[X86] Control-Flow Enforcement Technology - Shadow Stack support (LLVM side)

[oren_ben_simhon]

Summary

Control Flow Enforcement Technology (CET) provides HW capabilities to defend against Return Oriented Programming (ROP) attack and similarly Call/Jmp Oriented Programming (COP/JOP) attack.

Control flow subversion attacks are handled using two CET’s mechanisms:

1. Shadow Stack (SHSTK) – return address protection to defend against ROP.
2. Indirect Branch Tracking (IBT) – free branch protection to defend against JOP/COP.

Shadow stack solution introduces a new stack for return addresses only. The HW has a Shadow Stack Pointer (SSP) that points to the next return address. If we return to a different address, an exception is triggered.
The shadow stack is managed using a series of intrinsics that are introduced in this patch as well as the new register (SSP). The intrinsics are mapped to new instruction set that implements CET mechanism.
The patch also includes initial infrastructure support for IBT.

For more information, please see the following:
https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf

[craig.topper]

lib/Support/Host.cpp

1198

Please keep the leaf7 ECX bits with the others at line 1262.

lib/Target/X86/X86InstrSystem.td

498

Please limit lines to 80 columns.

[oren_ben_simhon]

Implemented comments posted until 11/19 (Thanks Craig)

[oren_ben_simhon]

Removed redundant FeatureCET flag.

[craig.topper]

lib/Support/Host.cpp

1208

This section needs to be rebased. Just make sure you put the bits in order when you do.

lib/Target/X86/X86InstrInfo.td

883

I don't think there's any need for the AssemblerPredicates. We don't do fine grained feature enabling for the assembler.

lib/Target/X86/X86InstrSystem.td

488

Don't put spaces around the color between GR32 and $src. Same with all the places below this.

[oren_ben_simhon]

Implemented comments posted until 11/21 (Thanks Craig)

[craig.topper]

LGTM

[gadi.haber]

Following comments by Craig and Simon, is it possible to add the MC tests for CET 32bit?
You can use the 32bit tests in https://reviews.llvm.org/D41329

include/llvm/IR/IntrinsicsX86.td

 
 // Read Performance-Monitoring Counter.
 let TargetPrefix = "x86" in {
   def int_x86_rdpmc : GCCBuiltin<"__builtin_ia32_rdpmc">,
               Intrinsic<[llvm_i64_ty], [llvm_i32_ty], []>;
 }
 
 //===----------------------------------------------------------------------===//
+// CET SS
+let TargetPrefix = "x86" in {
+  def int_x86_incsspd : GCCBuiltin<"__builtin_ia32_incsspd">,
+              Intrinsic<[], [llvm_i32_ty], []>;
+  def int_x86_incsspq : GCCBuiltin<"__builtin_ia32_incsspq">,
+              Intrinsic<[], [llvm_i64_ty], []>;
+  def int_x86_rdsspd : GCCBuiltin<"__builtin_ia32_rdsspd">,
+              Intrinsic<[llvm_i32_ty], [llvm_i32_ty], []>;
+  def int_x86_rdsspq : GCCBuiltin<"__builtin_ia32_rdsspq">,
+              Intrinsic<[llvm_i64_ty], [llvm_i64_ty], []>;
+  def int_x86_saveprevssp : GCCBuiltin<"__builtin_ia32_saveprevssp">,
+              Intrinsic<[], [], []>;
+  def int_x86_rstorssp : GCCBuiltin<"__builtin_ia32_rstorssp">,
+              Intrinsic<[], [llvm_ptr_ty], []>;
+  def int_x86_wrssd : GCCBuiltin<"__builtin_ia32_wrssd">,
+              Intrinsic<[], [llvm_i32_ty, llvm_ptr_ty], []>;
+  def int_x86_wrssq : GCCBuiltin<"__builtin_ia32_wrssq">,
+              Intrinsic<[], [llvm_i64_ty, llvm_ptr_ty], []>;
+  def int_x86_wrussd : GCCBuiltin<"__builtin_ia32_wrussd">,
+              Intrinsic<[], [llvm_i32_ty, llvm_ptr_ty], []>;
+  def int_x86_wrussq : GCCBuiltin<"__builtin_ia32_wrussq">,
+              Intrinsic<[], [llvm_i64_ty, llvm_ptr_ty], []>;
+  def int_x86_setssbsy : GCCBuiltin<"__builtin_ia32_setssbsy">,
+              Intrinsic<[], [], []>;
+  def int_x86_clrssbsy : GCCBuiltin<"__builtin_ia32_clrssbsy">,
+              Intrinsic<[], [llvm_ptr_ty], []>;
+}
+
+//===----------------------------------------------------------------------===//
 // 3DNow!
 
 let TargetPrefix = "x86" in {
   def int_x86_3dnow_pavgusb : GCCBuiltin<"__builtin_ia32_pavgusb">,
               Intrinsic<[llvm_x86mmx_ty], [llvm_x86mmx_ty, llvm_x86mmx_ty],
                         [IntrNoMem]>;
   def int_x86_3dnow_pf2id : GCCBuiltin<"__builtin_ia32_pf2id">,
               Intrinsic<[llvm_x86mmx_ty], [llvm_x86mmx_ty], [IntrNoMem]>;

lib/Support/Host.cpp

bool sys::getHostCPUFeatures(StringMap<bool> &Features) {
 
   bool HasLeaf7 =
       MaxLevel >= 7 && !getX86CpuIDAndInfoEx(0x7, 0x0, &EAX, &EBX, &ECX, &EDX);
 
   Features["fsgsbase"]   = HasLeaf7 && ((EBX >>  0) & 1);
   Features["sgx"]        = HasLeaf7 && ((EBX >>  2) & 1);
   Features["bmi"]        = HasLeaf7 && ((EBX >>  3) & 1);
   // AVX2 is only supported if we have the OS save support from AVX.
   Features["avx2"]       = HasLeaf7 && ((EBX >>  5) & 1) && HasAVXSave;
   Features["bmi2"]       = HasLeaf7 && ((EBX >>  8) & 1);
   Features["rtm"]        = HasLeaf7 && ((EBX >> 11) & 1);
   // AVX512 is only supported if the OS supports the context save for it.
   Features["avx512f"]    = HasLeaf7 && ((EBX >> 16) & 1) && HasAVX512Save;
   Features["avx512dq"]   = HasLeaf7 && ((EBX >> 17) & 1) && HasAVX512Save;
   Features["rdseed"]     = HasLeaf7 && ((EBX >> 18) & 1);
   Features["adx"]        = HasLeaf7 && ((EBX >> 19) & 1);
   Features["avx512ifma"] = HasLeaf7 && ((EBX >> 21) & 1) && HasAVX512Save;
   Features["clflushopt"] = HasLeaf7 && ((EBX >> 23) & 1);
   Features["clwb"]       = HasLeaf7 && ((EBX >> 24) & 1);
   Features["avx512pf"]   = HasLeaf7 && ((EBX >> 26) & 1) && HasAVX512Save;
   Features["avx512er"]   = HasLeaf7 && ((EBX >> 27) & 1) && HasAVX512Save;
   Features["avx512cd"]   = HasLeaf7 && ((EBX >> 28) & 1) && HasAVX512Save;
   Features["sha"]        = HasLeaf7 && ((EBX >> 29) & 1);
   Features["avx512bw"]   = HasLeaf7 && ((EBX >> 30) & 1) && HasAVX512Save;
   Features["avx512vl"]   = HasLeaf7 && ((EBX >> 31) & 1) && HasAVX512Save;
 
   Features["prefetchwt1"]     = HasLeaf7 && ((ECX >>  0) & 1);
   Features["avx512vbmi"]      = HasLeaf7 && ((ECX >>  1) & 1) && HasAVX512Save;
   Features["pku"]             = HasLeaf7 && ((ECX >>  4) & 1);
   Features["avx512vbmi2"]     = HasLeaf7 && ((ECX >>  6) & 1) && HasAVX512Save;
+  Features["shstk"]           = HasLeaf7 && ((ECX >>  7) & 1);
   Features["vaes"]            = HasLeaf7 && ((ECX >>  9) & 1) && HasAVXSave;
   Features["vpclmulqdq"]      = HasLeaf7 && ((ECX >> 10) & 1) && HasAVXSave;
   Features["avx512vnni"]      = HasLeaf7 && ((ECX >> 11) & 1) && HasAVX512Save;
   Features["avx512bitalg"]    = HasLeaf7 && ((ECX >> 12) & 1) && HasAVX512Save;
   Features["avx512vpopcntdq"] = HasLeaf7 && ((ECX >> 14) & 1) && HasAVX512Save;
 
+  Features["ibt"] = HasLeaf7 && ((EDX >> 20) & 1);
+
   bool HasLeafD = MaxLevel >= 0xd &&
                   !getX86CpuIDAndInfoEx(0xd, 0x1, &EAX, &EBX, &ECX, &EDX);
 
   // Only enable XSAVE if OS has enabled support for saving YMM state.
   Features["xsaveopt"] = HasLeafD && ((EAX >> 0) & 1) && HasAVXSave;
   Features["xsavec"]   = HasLeafD && ((EAX >> 1) & 1) && HasAVXSave;
   Features["xsaves"]   = HasLeafD && ((EAX >> 3) & 1) && HasAVXSave;
 

lib/Target/X86/X86.td

def FeatureBMI2    : SubtargetFeature<"bmi2", "HasBMI2", "true",
                                       "Support BMI2 instructions">;
 def FeatureRTM     : SubtargetFeature<"rtm", "HasRTM", "true",
                                       "Support RTM instructions">;
 def FeatureADX     : SubtargetFeature<"adx", "HasADX", "true",
                                       "Support ADX instructions">;
 def FeatureSHA     : SubtargetFeature<"sha", "HasSHA", "true",
                                       "Enable SHA instructions",
                                       [FeatureSSE2]>;
+def FeatureSHSTK   : SubtargetFeature<"shstk", "HasSHSTK", "true",
+                       "Support CET Shadow-Stack instructions">;
+def FeatureIBT     : SubtargetFeature<"ibt", "HasIBT", "true",
+                       "Support CET Indirect-Branch-Tracking instructions">;
 def FeaturePRFCHW  : SubtargetFeature<"prfchw", "HasPRFCHW", "true",
                                       "Support PRFCHW instructions">;
 def FeatureRDSEED  : SubtargetFeature<"rdseed", "HasRDSEED", "true",
                                       "Support RDSEED instruction">;
 def FeatureLAHFSAHF : SubtargetFeature<"sahf", "HasLAHFSAHF", "true",
                                        "Support LAHF and SAHF instructions">;
 def FeatureMWAITX  : SubtargetFeature<"mwaitx", "HasMWAITX", "true",
                                       "Enable MONITORX/MWAITX timer functionality">;

lib/Target/X86/X86InstrCompiler.td

 
 
 //===----------------------------------------------------------------------===//
 // Random Pseudo Instructions.
 
 // PIC base construction.  This expands to code that looks like this:
 //     call  $next_inst
 //     popl %destreg"
-let hasSideEffects = 0, isNotDuplicable = 1, Uses = [ESP] in
+let hasSideEffects = 0, isNotDuplicable = 1, Uses = [ESP, SSP] in
   def MOVPC32r : Ii32<0xE8, Pseudo, (outs GR32:$reg), (ins i32imm:$label),
                       "", []>;
 
 
 // ADJCALLSTACKDOWN/UP implicitly use/def ESP because they may be expanded into
 // a stack adjustment and the codegen must know that they may modify the stack
 // pointer before prolog-epilog rewriting occurs.
 // Pessimistically assume ADJCALLSTACKDOWN / ADJCALLSTACKUP will become
 // sub / add which can clobber EFLAGS.
-let Defs = [ESP, EFLAGS], Uses = [ESP] in {
+let Defs = [ESP, EFLAGS, SSP], Uses = [ESP, SSP] in {
 def ADJCALLSTACKDOWN32 : I<0, Pseudo, (outs),
                            (ins i32imm:$amt1, i32imm:$amt2, i32imm:$amt3),
                            "#ADJCALLSTACKDOWN",
                            []>,
                           Requires<[NotLP64]>;
 def ADJCALLSTACKUP32   : I<0, Pseudo, (outs), (ins i32imm:$amt1, i32imm:$amt2),
                            "#ADJCALLSTACKUP",
                            [(X86callseq_end timm:$amt1, timm:$amt2)]>,
                           Requires<[NotLP64]>;
 }
 def : Pat<(X86callseq_start timm:$amt1, timm:$amt2),
        (ADJCALLSTACKDOWN32 i32imm:$amt1, i32imm:$amt2, 0)>, Requires<[NotLP64]>;
 
 
 // ADJCALLSTACKDOWN/UP implicitly use/def RSP because they may be expanded into
 // a stack adjustment and the codegen must know that they may modify the stack
 // pointer before prolog-epilog rewriting occurs.
 // Pessimistically assume ADJCALLSTACKDOWN / ADJCALLSTACKUP will become
 // sub / add which can clobber EFLAGS.
-let Defs = [RSP, EFLAGS], Uses = [RSP] in {
+let Defs = [RSP, EFLAGS, SSP], Uses = [RSP, SSP] in {
 def ADJCALLSTACKDOWN64 : I<0, Pseudo, (outs),
                            (ins i32imm:$amt1, i32imm:$amt2, i32imm:$amt3),
                            "#ADJCALLSTACKDOWN",
                            []>,
                           Requires<[IsLP64]>;
 def ADJCALLSTACKUP64   : I<0, Pseudo, (outs), (ins i32imm:$amt1, i32imm:$amt2),
                            "#ADJCALLSTACKUP",
                            [(X86callseq_end timm:$amt1, timm:$amt2)]>,

 // All calls clobber the non-callee saved registers. ESP is marked as
 // a use to prevent stack-pointer assignments that appear immediately
 // before calls from potentially appearing dead.
 let Defs = [EAX, ECX, EDX, FP0, FP1, FP2, FP3, FP4, FP5, FP6, FP7,
             ST0, ST1, ST2, ST3, ST4, ST5, ST6, ST7,
             MM0, MM1, MM2, MM3, MM4, MM5, MM6, MM7,
             XMM0, XMM1, XMM2, XMM3, XMM4, XMM5, XMM6, XMM7,
             XMM8, XMM9, XMM10, XMM11, XMM12, XMM13, XMM14, XMM15, EFLAGS],
-    usesCustomInserter = 1, Uses = [ESP] in {
+    usesCustomInserter = 1, Uses = [ESP, SSP] in {
 def TLS_addr32 : I<0, Pseudo, (outs), (ins i32mem:$sym),
                   "# TLS_addr32",
                   [(X86tlsaddr tls32addr:$sym)]>,
                   Requires<[Not64BitMode]>;
 def TLS_base_addr32 : I<0, Pseudo, (outs), (ins i32mem:$sym),
                   "# TLS_base_addr32",
                   [(X86tlsbaseaddr tls32baseaddr:$sym)]>,
                   Requires<[Not64BitMode]>;
 }
 
 // All calls clobber the non-callee saved registers. RSP is marked as
 // a use to prevent stack-pointer assignments that appear immediately
 // before calls from potentially appearing dead.
 let Defs = [RAX, RCX, RDX, RSI, RDI, R8, R9, R10, R11,
             FP0, FP1, FP2, FP3, FP4, FP5, FP6, FP7,
             ST0, ST1, ST2, ST3, ST4, ST5, ST6, ST7,
             MM0, MM1, MM2, MM3, MM4, MM5, MM6, MM7,
             XMM0, XMM1, XMM2, XMM3, XMM4, XMM5, XMM6, XMM7,
             XMM8, XMM9, XMM10, XMM11, XMM12, XMM13, XMM14, XMM15, EFLAGS],
-    usesCustomInserter = 1, Uses = [RSP] in {
+    usesCustomInserter = 1, Uses = [RSP, SSP] in {
 def TLS_addr64 : I<0, Pseudo, (outs), (ins i64mem:$sym),
                    "# TLS_addr64",
                   [(X86tlsaddr tls64addr:$sym)]>,
                   Requires<[In64BitMode]>;
 def TLS_base_addr64 : I<0, Pseudo, (outs), (ins i64mem:$sym),
                    "# TLS_base_addr64",
                   [(X86tlsbaseaddr tls64baseaddr:$sym)]>,
                   Requires<[In64BitMode]>;
 }
 
 // Darwin TLS Support
 // For i386, the address of the thunk is passed on the stack, on return the
 // address of the variable is in %eax.  %ecx is trashed during the function
 // call.  All other registers are preserved.
 let Defs = [EAX, ECX, EFLAGS],
-    Uses = [ESP],
+    Uses = [ESP, SSP],
     usesCustomInserter = 1 in
 def TLSCall_32 : I<0, Pseudo, (outs), (ins i32mem:$sym),
                 "# TLSCall_32",
                 [(X86TLSCall addr:$sym)]>,
                 Requires<[Not64BitMode]>;
 
 // For x86_64, the address of the thunk is passed in %rdi, but the
 // pseudo directly use the symbol, so do not add an implicit use of
 // %rdi. The lowering will do the right thing with RDI.
 // On return the address of the variable is in %rax.  All other
 // registers are preserved.
 let Defs = [RAX, EFLAGS],
-    Uses = [RSP],
+    Uses = [RSP, SSP],
     usesCustomInserter = 1 in
 def TLSCall_64 : I<0, Pseudo, (outs), (ins i64mem:$sym),
                   "# TLSCall_64",
                   [(X86TLSCall addr:$sym)]>,
                   Requires<[In64BitMode]>;
 
 
 //===----------------------------------------------------------------------===//

lib/Target/X86/X86InstrControl.td

 //===----------------------------------------------------------------------===//
 //  Call Instructions...
 //
 let isCall = 1 in
   // All calls clobber the non-callee saved registers. ESP is marked as
   // a use to prevent stack-pointer assignments that appear immediately
   // before calls from potentially appearing dead. Uses for argument
   // registers are added manually.
-  let Uses = [ESP] in {
+  let Uses = [ESP, SSP] in {
     def CALLpcrel32 : Ii32PCRel<0xE8, RawFrm,
                            (outs), (ins i32imm_pcrel:$dst),
                            "call{l}\t$dst", [], IIC_CALL_RI>, OpSize32,
                       Requires<[Not64BitMode]>, Sched<[WriteJump]>;
     let hasSideEffects = 0 in
       def CALLpcrel16 : Ii16PCRel<0xE8, RawFrm,
                              (outs), (ins i16imm_pcrel:$dst),
                              "call{w}\t$dst", [], IIC_CALL_RI>, OpSize16,
def FARCALL32m  : I<0xFF, MRM3m, (outs), (ins opaque48mem:$dst),
                         "{l}call{l}\t{*}$dst", [], IIC_CALL_FAR_MEM>, OpSize32,
                       Sched<[WriteJumpLd]>;
   }
 
 
 // Tail call stuff.
 let isCall = 1, isTerminator = 1, isReturn = 1, isBarrier = 1,
     isCodeGenOnly = 1, SchedRW = [WriteJumpLd] in
-  let Uses = [ESP] in {
+  let Uses = [ESP, SSP] in {
   def TCRETURNdi : PseudoI<(outs),
                      (ins i32imm_pcrel:$dst, i32imm:$offset), []>, NotMemoryFoldable;
   def TCRETURNri : PseudoI<(outs),
                      (ins ptr_rc_tailcall:$dst, i32imm:$offset), []>, NotMemoryFoldable;
   let mayLoad = 1 in
   def TCRETURNmi : PseudoI<(outs),
                      (ins i32mem_TC:$dst, i32imm:$offset), []>;
 
let isCall = 1, isTerminator = 1, isReturn = 1, isBarrier = 1,
   def TAILJMPm : I<0xFF, MRM4m, (outs), (ins i32mem_TC:$dst),
                    "jmp{l}\t{*}$dst", [], IIC_JMP_MEM>;
 }
 
 // Conditional tail calls are similar to the above, but they are branches
 // rather than barriers, and they use EFLAGS.
 let isCall = 1, isTerminator = 1, isReturn = 1, isBranch = 1,
     isCodeGenOnly = 1, SchedRW = [WriteJumpLd] in
-  let Uses = [ESP, EFLAGS] in {
+  let Uses = [ESP, EFLAGS, SSP] in {
   def TCRETURNdicc : PseudoI<(outs),
                      (ins i32imm_pcrel:$dst, i32imm:$offset, i32imm:$cond), []>;
 
   // This gets substituted to a conditional jump instruction in MC lowering.
   def TAILJMPd_CC : Ii32PCRel<0x80, RawFrm, (outs),
                            (ins i32imm_pcrel:$dst, i32imm:$cond),
                            "",
                            [], IIC_JMP_REL>;
 }
 
 
 //===----------------------------------------------------------------------===//
 //  Call Instructions...
 //
 
 // RSP is marked as a use to prevent stack-pointer assignments that appear
 // immediately before calls from potentially appearing dead. Uses for argument
 // registers are added manually.
-let isCall = 1, Uses = [RSP], SchedRW = [WriteJump] in {
+let isCall = 1, Uses = [RSP, SSP], SchedRW = [WriteJump] in {
   // NOTE: this pattern doesn't match "X86call imm", because we do not know
   // that the offset between an arbitrary immediate and the call will fit in
   // the 32-bit pcrel field that we have.
   def CALL64pcrel32 : Ii32PCRel<0xE8, RawFrm,
                         (outs), (ins i64i32imm_pcrel:$dst),
                         "call{q}\t$dst", [], IIC_CALL_RI>, OpSize32,
                       Requires<[In64BitMode]>;
   def CALL64r       : I<0xFF, MRM2r, (outs), (ins GR64:$dst),
                         "call{q}\t{*}$dst", [(X86call GR64:$dst)],
                         IIC_CALL_RI>,
                       Requires<[In64BitMode]>;
   def CALL64m       : I<0xFF, MRM2m, (outs), (ins i64mem:$dst),
                         "call{q}\t{*}$dst", [(X86call (loadi64 addr:$dst))],
                         IIC_CALL_MEM>,
                       Requires<[In64BitMode,FavorMemIndirectCall]>;
 
   def FARCALL64   : RI<0xFF, MRM3m, (outs), (ins opaque80mem:$dst),
                        "lcall{q}\t{*}$dst", [], IIC_CALL_FAR_MEM>;
 }
 
 let isCall = 1, isTerminator = 1, isReturn = 1, isBarrier = 1,
-    isCodeGenOnly = 1, Uses = [RSP], usesCustomInserter = 1,
+    isCodeGenOnly = 1, Uses = [RSP, SSP], usesCustomInserter = 1,
     SchedRW = [WriteJump] in {
   def TCRETURNdi64   : PseudoI<(outs),
                         (ins i64i32imm_pcrel:$dst, i32imm:$offset),
                         []>;
   def TCRETURNri64   : PseudoI<(outs),
                         (ins ptr_rc_tailcall:$dst, i32imm:$offset), []>, NotMemoryFoldable;
   let mayLoad = 1 in
   def TCRETURNmi64   : PseudoI<(outs),
def TAILJMPm64_REX : I<0xFF, MRM4m, (outs), (ins i64mem_TC:$dst),
                            "rex64 jmp{q}\t{*}$dst", [], IIC_JMP_MEM>;
   }
 }
 
 // Conditional tail calls are similar to the above, but they are branches
 // rather than barriers, and they use EFLAGS.
 let isCall = 1, isTerminator = 1, isReturn = 1, isBranch = 1,
     isCodeGenOnly = 1, SchedRW = [WriteJumpLd] in
-  let Uses = [RSP, EFLAGS] in {
+  let Uses = [RSP, EFLAGS, SSP] in {
   def TCRETURNdi64cc : PseudoI<(outs),
                            (ins i64i32imm_pcrel:$dst, i32imm:$offset,
                             i32imm:$cond), []>;
 
   // This gets substituted to a conditional jump instruction in MC lowering.
   def TAILJMPd64_CC : Ii32PCRel<0x80, RawFrm, (outs),
                            (ins i64i32imm_pcrel:$dst, i32imm:$cond),
                            "",
                            [], IIC_JMP_REL>;
 }

lib/Target/X86/X86InstrInfo.td

 def HasRDSEED    : Predicate<"Subtarget->hasRDSEED()">;
 def HasPrefetchW : Predicate<"Subtarget->hasPRFCHW()">;
 def HasLAHFSAHF  : Predicate<"Subtarget->hasLAHFSAHF()">;
 def HasMWAITX    : Predicate<"Subtarget->hasMWAITX()">;
 def HasCLZERO    : Predicate<"Subtarget->hasCLZERO()">;
 def FPStackf32   : Predicate<"!Subtarget->hasSSE1()">;
 def FPStackf64   : Predicate<"!Subtarget->hasSSE2()">;
 def HasMPX       : Predicate<"Subtarget->hasMPX()">;
+def HasSHSTK     : Predicate<"Subtarget->hasSHSTK()">;
+def HasIBT       : Predicate<"Subtarget->hasIBT()">;
 def HasCLFLUSHOPT : Predicate<"Subtarget->hasCLFLUSHOPT()">;
 def HasCLWB      : Predicate<"Subtarget->hasCLWB()">;
 def HasCmpxchg16b: Predicate<"Subtarget->hasCmpxchg16b()">;
 def Not64BitMode : Predicate<"!Subtarget->is64Bit()">,
                              AssemblerPredicate<"!Mode64Bit", "Not 64-bit mode">;
 def In64BitMode  : Predicate<"Subtarget->is64Bit()">,
                              AssemblerPredicate<"Mode64Bit", "64-bit mode">;
 def IsLP64  : Predicate<"Subtarget->isTarget64BitLP64()">;

lib/Target/X86/X86InstrSystem.td

 //===----------------------------------------------------------------------===//
 // Cache instructions
 let SchedRW = [WriteSystem] in {
 def INVD : I<0x08, RawFrm, (outs), (ins), "invd", [], IIC_INVD>, TB;
 def WBINVD : I<0x09, RawFrm, (outs), (ins), "wbinvd", [], IIC_INVD>, TB;
 } // SchedRW
 
 //===----------------------------------------------------------------------===//
+// CET instructions
+let SchedRW = [WriteSystem], Predicates = [HasSHSTK]  in{
+  let Uses = [SSP] in {
+    let Defs = [SSP] in {
+      def INCSSPD : I<0xAE, MRM5r, (outs), (ins GR32:$src), "incsspd\t$src",
+                       [(int_x86_incsspd GR32:$src)]>, XS;
+      def INCSSPQ : RI<0xAE, MRM5r, (outs), (ins GR64:$src), "incsspq\t$src",
+                       [(int_x86_incsspq GR64:$src)]>, XS, 
+                       Requires<[In64BitMode]>;
+    } // Defs SSP
+
+    let Constraints = "$src = $dst" in {
+      def RDSSPD : I<0x1E, MRM1r, (outs GR32:$dst), (ins GR32:$src), 
+                     "rdsspd\t$dst",
+                     [(set GR32:$dst, (int_x86_rdsspd GR32:$src))]>, XS;
+      def RDSSPQ : RI<0x1E, MRM1r, (outs GR64:$dst), (ins GR64:$src), 
+                     "rdsspq\t$dst", 
+                     [(set GR64:$dst, (int_x86_rdsspq GR64:$src))]>, XS, 
+                     Requires<[In64BitMode]>;
+    }
+
+    let Defs = [SSP] in {
+      def SAVEPREVSSP : I<0x01, MRM_EA, (outs), (ins), "saveprevssp",
+                       [(int_x86_saveprevssp)]>, XS;
+      def RSTORSSP : I<0x01, MRM5m, (outs), (ins i32mem:$src), 
+                       "rstorssp\t$src",
+                       [(int_x86_rstorssp addr:$src)]>, XS;
+    } // Defs SSP
+  } // Uses SSP
+
+  def WRSSD : I<0xF6, MRMDestMem, (outs), (ins i32mem:$dst, GR32:$src),
+                "wrssd\t{$src, $dst|$dst, $src}",
+                [(int_x86_wrssd GR32:$src, addr:$dst)]>, T8;
+  def WRSSQ : RI<0xF6, MRMDestMem, (outs), (ins i64mem:$dst, GR64:$src),
+                 "wrssq\t{$src, $dst|$dst, $src}",
+                 [(int_x86_wrssq GR64:$src, addr:$dst)]>, T8, 
+                 Requires<[In64BitMode]>;
+  def WRUSSD : I<0xF5, MRMDestMem, (outs), (ins i32mem:$dst, GR32:$src),
+                 "wrussd\t{$src, $dst|$dst, $src}",
+                 [(int_x86_wrussd GR32:$src, addr:$dst)]>, T8PD;
+  def WRUSSQ : RI<0xF5, MRMDestMem, (outs), (ins i64mem:$dst, GR64:$src), 
+                  "wrussq\t{$src, $dst|$dst, $src}",
+                  [(int_x86_wrussq GR64:$src, addr:$dst)]>, T8PD, 
+                  Requires<[In64BitMode]>;
+
+  let Defs = [SSP] in {
+    let Uses = [SSP] in {
+        def SETSSBSY : I<0x01, MRM_E8, (outs), (ins), "setssbsy",
+                         [(int_x86_setssbsy)]>, XS;
+    } // Uses SSP
+
+    def CLRSSBSY : I<0xAE, MRM6m, (outs), (ins i32mem:$src), 
+                     "clrssbsy\t$src",
+                     [(int_x86_clrssbsy addr:$src)]>, XS;
+  } // Defs SSP
+} // SchedRW && HasSHSTK
+
+//===----------------------------------------------------------------------===//
 // XSAVE instructions
 let SchedRW = [WriteSystem] in {
 let Predicates = [HasXSAVE] in {
 let Defs = [EDX, EAX], Uses = [ECX] in
   def XGETBV : I<0x01, MRM_D0, (outs), (ins), "xgetbv", []>, TB;
 
 let Uses = [EDX, EAX, ECX] in
   def XSETBV : I<0x01, MRM_D1, (outs), (ins), 

lib/Target/X86/X86RegisterInfo.cpp

BitVector X86RegisterInfo::getReservedRegs(const MachineFunction &MF) const {
   BitVector Reserved(getNumRegs());
   const X86FrameLowering *TFI = getFrameLowering(MF);
 
   // Set the stack-pointer register and its aliases as reserved.
   for (MCSubRegIterator I(X86::RSP, this, /*IncludeSelf=*/true); I.isValid();
        ++I)
     Reserved.set(*I);
 
+  // Set the Shadow Stack Pointer as reserved.
+  Reserved.set(X86::SSP);
+
   // Set the instruction pointer register and its aliases as reserved.
   for (MCSubRegIterator I(X86::RIP, this, /*IncludeSelf=*/true); I.isValid();
        ++I)
     Reserved.set(*I);
 
   // Set the frame-pointer register and its aliases as reserved if needed.
   if (TFI->hasFP(MF)) {
     for (MCSubRegIterator I(X86::RBP, this, /*IncludeSelf=*/true); I.isValid();

lib/Target/X86/X86RegisterInfo.td

 def RIZ : X86Reg<"riz", 4>;
 
 // Bound registers, used in MPX instructions
 def BND0 : X86Reg<"bnd0",   0>;
 def BND1 : X86Reg<"bnd1",   1>;
 def BND2 : X86Reg<"bnd2",   2>;
 def BND3 : X86Reg<"bnd3",   3>;
 
+// CET registers - Shadow Stack Pointer
+def SSP : X86Reg<"ssp", 0>;
+
 //===----------------------------------------------------------------------===//
 // Register Class Definitions... now that we have all of the pieces, define the
 // top-level register classes.  The order specified in the register list is
 // implicitly defined to be the register allocation order.
 //
 
 // List call-clobbered registers before callee-save registers. RBX, RBP, (and
 // R12, R13, R14, and R15 for X86-64) are callee-save registers.

lib/Target/X86/X86Subtarget.h

protected:
   bool HasVNNI;
 
   /// Processor has AVX-512 Bit Algorithms instructions
   bool HasBITALG;
 
   /// Processor supports MPX - Memory Protection Extensions
   bool HasMPX;
 
+  /// Processor supports CET SHSTK - Control-Flow Enforcement Technology
+  /// using Shadow Stack
+  bool HasSHSTK;
+
+  /// Processor supports CET IBT - Control-Flow Enforcement Technology
+  /// using Indirect Branch Tracking
+  bool HasIBT;
+
   /// Processor has Software Guard Extensions
   bool HasSGX;
 
   /// Processor supports Flush Cache Line instruction
   bool HasCLFLUSHOPT;
 
   /// Processor supports Cache Line Write Back instruction
   bool HasCLWB;
public:
   bool hasERI() const { return HasERI; }
   bool hasDQI() const { return HasDQI; }
   bool hasBWI() const { return HasBWI; }
   bool hasVLX() const { return HasVLX; }
   bool hasPKU() const { return HasPKU; }
   bool hasVNNI() const { return HasVNNI; }
   bool hasBITALG() const { return HasBITALG; }
   bool hasMPX() const { return HasMPX; }
+  bool hasSHSTK() const { return HasSHSTK; }
+  bool hasIBT() const { return HasIBT; }
   bool hasCLFLUSHOPT() const { return HasCLFLUSHOPT; }
   bool hasCLWB() const { return HasCLWB; }
 
   bool isXRaySupported() const override { return is64Bit(); }
 
   X86ProcFamilyEnum getProcFamily() const { return X86ProcFamily; }
 
   /// TODO: to be removed later and replaced with suitable properties

lib/Target/X86/X86Subtarget.cpp

void X86Subtarget::initializeEnvironment() {
   HasBITALG = false;
   HasSHA = false;
   HasPRFCHW = false;
   HasRDSEED = false;
   HasLAHFSAHF = false;
   HasMWAITX = false;
   HasCLZERO = false;
   HasMPX = false;
+  HasSHSTK = false;
+  HasIBT = false;
   HasSGX = false;
   HasCLFLUSHOPT = false;
   HasCLWB = false;
   IsPMULLDSlow = false;
   IsSHLDSlow = false;
   IsUAMem16Slow = false;
   IsUAMem32Slow = false;
   HasSSEUnalignedMem = false;

test/CodeGen/MIR/X86/frame-info-save-restore-points.mir

bb.1:
 
     JMP_1 %bb.3.false
 
   bb.2.true:
     successors: %bb.3.false
     liveins: %eax
 
     MOV32mr %stack.0.tmp, 1, _, 0, _, killed %eax
-    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def dead %eflags, implicit %rsp
+    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %ssp, implicit-def dead %eflags, implicit %rsp, implicit %ssp
     %rsi = LEA64r %stack.0.tmp, 1, _, 0, _
     %edi = MOV32r0 implicit-def dead %eflags
-    CALL64pcrel32 @doSomething, csr_64, implicit %rsp, implicit %edi, implicit %rsi, implicit-def %rsp, implicit-def %eax
-    ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def dead %eflags, implicit %rsp
+    CALL64pcrel32 @doSomething, csr_64, implicit %rsp, implicit %ssp, implicit %edi, implicit %rsi, implicit-def %rsp, implicit-def %ssp, implicit-def %eax
+    ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %ssp, implicit-def dead %eflags, implicit %rsp, implicit %ssp
 
   bb.3.false:
     liveins: %eax
 
     RETQ %eax
 ...

test/CodeGen/X86/GlobalISel/irtranslator-callingconv.ll

;X32-NEXT:    RET 0, implicit %eax
 
   ret i32 * %p1;
 }
 
 declare void @trivial_callee()
 define void @test_trivial_call() {
 ; ALL-LABEL: name:            test_trivial_call
 
-; X32:      ADJCALLSTACKDOWN32 0, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32:      ADJCALLSTACKDOWN32 0, 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT: CALLpcrel32 @trivial_callee, csr_32, implicit %esp
-; X32-NEXT: ADJCALLSTACKUP32 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT: ADJCALLSTACKUP32 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT: RET 0
 
-; X64:      ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64:      ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT: CALL64pcrel32 @trivial_callee, csr_64, implicit %rsp
-; X64-NEXT: ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT: ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT: RET 0
 
   call void @trivial_callee()
   ret void
 }
 
 declare void @simple_arg_callee(i32 %in0, i32 %in1)
 define void @test_simple_arg(i32 %in0, i32 %in1) {
 ; ALL-LABEL: name:            test_simple_arg
 
 ; X32:      fixedStack:
 ; X32:   - { id: 0, type: default, offset: 4, size: 4, alignment: 4,
 ; X32-NEXT:  isImmutable: true,
 ; X32:   - { id: 1, type: default, offset: 0, size: 4, alignment: 16,
 ; X32-NEXT:  isImmutable: true,
 ; X32:      body:             |
 ; X32-NEXT:   bb.1 (%ir-block.0):
 ; X32-NEXT:     %2:_(p0) = G_FRAME_INDEX %fixed-stack.1
 ; X32-NEXT:     %0:_(s32) = G_LOAD %2(p0) :: (invariant load 4 from %fixed-stack.1, align 0)
 ; X32-NEXT:     %3:_(p0) = G_FRAME_INDEX %fixed-stack.0
 ; X32-NEXT:     %1:_(s32) = G_LOAD %3(p0) :: (invariant load 4 from %fixed-stack.0, align 0)
-; X32-NEXT:     ADJCALLSTACKDOWN32 8, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:     ADJCALLSTACKDOWN32 8, 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:     %4:_(p0) = COPY %esp
 ; X32-NEXT:     %5:_(s32) = G_CONSTANT i32 0
 ; X32-NEXT:     %6:_(p0) = G_GEP %4, %5(s32)
 ; X32-NEXT:     G_STORE %1(s32), %6(p0) :: (store 4 into stack, align 0)
 ; X32-NEXT:     %7:_(p0) = COPY %esp
 ; X32-NEXT:     %8:_(s32) = G_CONSTANT i32 4
 ; X32-NEXT:     %9:_(p0) = G_GEP %7, %8(s32)
 ; X32-NEXT:     G_STORE %0(s32), %9(p0) :: (store 4 into stack + 4, align 0)
 ; X32-NEXT:     CALLpcrel32 @simple_arg_callee, csr_32, implicit %esp
-; X32-NEXT:     ADJCALLSTACKUP32 8, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:     ADJCALLSTACKUP32 8, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:     RET 0
 
 ; X64:      %0:_(s32) = COPY %edi
 ; X64-NEXT: %1:_(s32) = COPY %esi
-; X64-NEXT: ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT: ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT: %edi = COPY %1(s32)
 ; X64-NEXT: %esi = COPY %0(s32)
-; X64-NEXT: CALL64pcrel32 @simple_arg_callee, csr_64, implicit %rsp, implicit %edi, implicit %esi
-; X64-NEXT: ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT: CALL64pcrel32 @simple_arg_callee, csr_64, implicit %rsp, implicit %ssp, implicit %edi, implicit %esi
+; X64-NEXT: ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT: RET 0
 
   call void @simple_arg_callee(i32 %in1, i32 %in0)
   ret void
 }
 
 declare void @simple_arg8_callee(i32 %arg1, i32 %arg2, i32 %arg3, i32 %arg4, i32 %arg5, i32 %arg6, i32 %arg7, i32 %arg8)
 define void @test_simple_arg8_call(i32 %in0) {
 ; ALL-LABEL: name:            test_simple_arg8_call
 
 ; X32:      fixedStack:
 ; X32:   - { id: 0, type: default, offset: 0, size: 4, alignment: 16,
 ; X32-NEXT:  isImmutable: true,
 ; X32:     body:             |
 ; X32-NEXT:   bb.1 (%ir-block.0):
 ; X32-NEXT:     %1:_(p0) = G_FRAME_INDEX %fixed-stack.0
 ; X32-NEXT:     %0:_(s32) = G_LOAD %1(p0) :: (invariant load 4 from %fixed-stack.0, align 0)
-; X32-NEXT:     ADJCALLSTACKDOWN32 32, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:     ADJCALLSTACKDOWN32 32, 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:     %2:_(p0) = COPY %esp
 ; X32-NEXT:     %3:_(s32) = G_CONSTANT i32 0
 ; X32-NEXT:     %4:_(p0) = G_GEP %2, %3(s32)
 ; X32-NEXT:     G_STORE %0(s32), %4(p0) :: (store 4 into stack, align 0)
 ; X32-NEXT:     %5:_(p0) = COPY %esp
 ; X32-NEXT:     %6:_(s32) = G_CONSTANT i32 4
 ; X32-NEXT:     %7:_(p0) = G_GEP %5, %6(s32)
 ; X32-NEXT:     G_STORE %0(s32), %7(p0) :: (store 4 into stack + 4, align 0)

 ; X32-NEXT:     %21:_(s32) = G_CONSTANT i32 24
 ; X32-NEXT:     %22:_(p0) = G_GEP %20, %21(s32)
 ; X32-NEXT:     G_STORE %0(s32), %22(p0) :: (store 4 into stack + 24, align 0)
 ; X32-NEXT:     %23:_(p0) = COPY %esp
 ; X32-NEXT:     %24:_(s32) = G_CONSTANT i32 28
 ; X32-NEXT:     %25:_(p0) = G_GEP %23, %24(s32)
 ; X32-NEXT:     G_STORE %0(s32), %25(p0) :: (store 4 into stack + 28, align 0)
 ; X32-NEXT:     CALLpcrel32 @simple_arg8_callee, csr_32, implicit %esp
-; X32-NEXT:     ADJCALLSTACKUP32 32, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:     ADJCALLSTACKUP32 32, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:     RET 0
 
 ; X64:          %0:_(s32) = COPY %edi
-; X64-NEXT:     ADJCALLSTACKDOWN64 16, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:     ADJCALLSTACKDOWN64 16, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:     %edi = COPY %0(s32)
 ; X64-NEXT:     %esi = COPY %0(s32)
 ; X64-NEXT:     %edx = COPY %0(s32)
 ; X64-NEXT:     %ecx = COPY %0(s32)
 ; X64-NEXT:     %r8d = COPY %0(s32)
 ; X64-NEXT:     %r9d = COPY %0(s32)
 ; X64-NEXT:     %1:_(p0) = COPY %rsp
 ; X64-NEXT:     %2:_(s64) = G_CONSTANT i64 0
 ; X64-NEXT:     %3:_(p0) = G_GEP %1, %2(s64)
 ; X64-NEXT:     G_STORE %0(s32), %3(p0) :: (store 4 into stack, align 0)
 ; X64-NEXT:     %4:_(p0) = COPY %rsp
 ; X64-NEXT:     %5:_(s64) = G_CONSTANT i64 8
 ; X64-NEXT:     %6:_(p0) = G_GEP %4, %5(s64)
 ; X64-NEXT:     G_STORE %0(s32), %6(p0) :: (store 4 into stack + 8, align 0)
-; X64-NEXT:     CALL64pcrel32 @simple_arg8_callee, csr_64, implicit %rsp, implicit %edi, implicit %esi, implicit %edx, implicit %ecx, implicit %r8d, implicit %r9d
-; X64-NEXT:     ADJCALLSTACKUP64 16, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:     CALL64pcrel32 @simple_arg8_callee, csr_64, implicit %rsp, implicit %ssp, implicit %edi, implicit %esi, implicit %edx, implicit %ecx, implicit %r8d, implicit %r9d
+; X64-NEXT:     ADJCALLSTACKUP64 16, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:     RET 0
 
   call void @simple_arg8_callee(i32 %in0, i32 %in0, i32 %in0, i32 %in0,i32 %in0, i32 %in0, i32 %in0, i32 %in0)
   ret void
 }
 
 declare i32 @simple_return_callee(i32 %in0)
 define i32 @test_simple_return_callee() {
 ; ALL-LABEL: name:            test_simple_return_callee
 
 ; X32:      %1:_(s32) = G_CONSTANT i32 5
-; X32-NEXT: ADJCALLSTACKDOWN32 4, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT: ADJCALLSTACKDOWN32 4, 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT: %2:_(p0) = COPY %esp
 ; X32-NEXT: %3:_(s32) = G_CONSTANT i32 0
 ; X32-NEXT: %4:_(p0) = G_GEP %2, %3(s32)
 ; X32-NEXT: G_STORE %1(s32), %4(p0) :: (store 4 into stack, align 0)
-; X32-NEXT: CALLpcrel32 @simple_return_callee, csr_32, implicit %esp, implicit-def %eax
+; X32-NEXT: CALLpcrel32 @simple_return_callee, csr_32, implicit %esp, implicit %ssp, implicit-def %eax
 ; X32-NEXT: %0:_(s32) = COPY %eax
-; X32-NEXT: ADJCALLSTACKUP32 4, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT: ADJCALLSTACKUP32 4, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT: %5:_(s32) = G_ADD %0, %0
 ; X32-NEXT: %eax = COPY %5(s32)
 ; X32-NEXT: RET 0, implicit %eax
 
 ; X64:      %1:_(s32) = G_CONSTANT i32 5
-; X64-NEXT: ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT: ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT: %edi = COPY %1(s32)
-; X64-NEXT: CALL64pcrel32 @simple_return_callee, csr_64, implicit %rsp, implicit %edi, implicit-def %eax
+; X64-NEXT: CALL64pcrel32 @simple_return_callee, csr_64, implicit %rsp, implicit %ssp, implicit %edi, implicit-def %eax
 ; X64-NEXT: %0:_(s32) = COPY %eax
-; X64-NEXT: ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT: ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT: %2:_(s32) = G_ADD %0, %0
 ; X64-NEXT: %eax = COPY %2(s32)
 ; X64-NEXT: RET 0, implicit %eax
 
   %call = call i32 @simple_return_callee(i32 5)
   %r = add i32 %call, %call
   ret i32 %r
 }
 
 declare <8 x i32> @split_return_callee(<8 x i32> %in0)
 define <8 x i32> @test_split_return_callee(<8 x i32> %arg1, <8 x i32> %arg2) {
 ; ALL-LABEL: name: test_split_return_callee
 
 ; X32:       fixedStack:
 ; X32-NEXT:   - { id: 0, type: default, offset: 0, size: 16, alignment: 16,
 ; X32-NEXT:       isImmutable: true,
 ; X32:       %2:_(<4 x s32>) = COPY %xmm0
 ; X32-NEXT:  %3:_(<4 x s32>) = COPY %xmm1
 ; X32-NEXT:  %4:_(<4 x s32>) = COPY %xmm2
 ; X32-NEXT:  %6:_(p0) = G_FRAME_INDEX %fixed-stack.0
 ; X32-NEXT:  %5:_(<4 x s32>) = G_LOAD %6(p0) :: (invariant load 16 from %fixed-stack.0, align 0)
 ; X32-NEXT:  %0:_(<8 x s32>) = G_MERGE_VALUES %2(<4 x s32>), %3(<4 x s32>)
 ; X32-NEXT:  %1:_(<8 x s32>) = G_MERGE_VALUES %4(<4 x s32>), %5(<4 x s32>)
-; X32-NEXT:  ADJCALLSTACKDOWN32 0, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:  ADJCALLSTACKDOWN32 0, 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:  %8:_(<4 x s32>), %9:_(<4 x s32>) = G_UNMERGE_VALUES %1(<8 x s32>)
 ; X32-NEXT:  %xmm0 = COPY %8(<4 x s32>)
 ; X32-NEXT:  %xmm1 = COPY %9(<4 x s32>)
-; X32-NEXT:  CALLpcrel32 @split_return_callee, csr_32, implicit %esp, implicit %xmm0, implicit %xmm1, implicit-def %xmm0, implicit-def %xmm1
+; X32-NEXT:  CALLpcrel32 @split_return_callee, csr_32, implicit %esp, implicit %ssp, implicit %xmm0, implicit %xmm1, implicit-def %xmm0, implicit-def %xmm1
 ; X32-NEXT:  %10:_(<4 x s32>) = COPY %xmm0
 ; X32-NEXT:  %11:_(<4 x s32>) = COPY %xmm1
 ; X32-NEXT:  %7:_(<8 x s32>) = G_MERGE_VALUES %10(<4 x s32>), %11(<4 x s32>)
-; X32-NEXT:  ADJCALLSTACKUP32 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:  ADJCALLSTACKUP32 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:  %12:_(<8 x s32>) = G_ADD %0, %7
 ; X32-NEXT:  %13:_(<4 x s32>), %14:_(<4 x s32>) = G_UNMERGE_VALUES %12(<8 x s32>)
 ; X32-NEXT:  %xmm0 = COPY %13(<4 x s32>)
 ; X32-NEXT:  %xmm1 = COPY %14(<4 x s32>)
 ; X32-NEXT:  RET 0, implicit %xmm0, implicit %xmm1
 
 ; X64:       %2:_(<4 x s32>) = COPY %xmm0
 ; X64-NEXT:  %3:_(<4 x s32>) = COPY %xmm1
 ; X64-NEXT:  %4:_(<4 x s32>) = COPY %xmm2
 ; X64-NEXT:  %5:_(<4 x s32>) = COPY %xmm3
 ; X64-NEXT:  %0:_(<8 x s32>) = G_MERGE_VALUES %2(<4 x s32>), %3(<4 x s32>)
 ; X64-NEXT:  %1:_(<8 x s32>) = G_MERGE_VALUES %4(<4 x s32>), %5(<4 x s32>)
-; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:  %7:_(<4 x s32>), %8:_(<4 x s32>) = G_UNMERGE_VALUES %1(<8 x s32>)
 ; X64-NEXT:  %xmm0 = COPY %7(<4 x s32>)
 ; X64-NEXT:  %xmm1 = COPY %8(<4 x s32>)
-; X64-NEXT:  CALL64pcrel32 @split_return_callee, csr_64, implicit %rsp, implicit %xmm0, implicit %xmm1, implicit-def %xmm0, implicit-def %xmm1
+; X64-NEXT:  CALL64pcrel32 @split_return_callee, csr_64, implicit %rsp, implicit %ssp, implicit %xmm0, implicit %xmm1, implicit-def %xmm0, implicit-def %xmm1
 ; X64-NEXT:  %9:_(<4 x s32>) = COPY %xmm0
 ; X64-NEXT:  %10:_(<4 x s32>) = COPY %xmm1
 ; X64-NEXT:  %6:_(<8 x s32>) = G_MERGE_VALUES %9(<4 x s32>), %10(<4 x s32>)
-; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:  %11:_(<8 x s32>) = G_ADD %0, %6
 ; X64-NEXT:  %12:_(<4 x s32>), %13:_(<4 x s32>) = G_UNMERGE_VALUES %11(<8 x s32>)
 ; X64-NEXT:  %xmm0 = COPY %12(<4 x s32>)
 ; X64-NEXT:  %xmm1 = COPY %13(<4 x s32>)
 ; X64-NEXT:  RET 0, implicit %xmm0, implicit %xmm1
 
   %call = call <8 x i32> @split_return_callee(<8 x i32> %arg2)
   %r = add <8 x i32> %arg1, %call
   ret  <8 x i32> %r
 }
 
 define void @test_indirect_call(void()* %func) {
 ; ALL-LABEL: name:            test_indirect_call
 
 ; X32:       registers:
 ; X32-NEXT:   - { id: 0, class: gr32, preferred-register: '' }
 ; X32-NEXT:   - { id: 1, class: _, preferred-register: '' }
 ; X32:       %1:_(p0) = G_FRAME_INDEX %fixed-stack.0
 ; X32-NEXT:  %0:gr32(p0) = G_LOAD %1(p0) :: (invariant load 4 from %fixed-stack.0, align 0)
-; X32-NEXT:  ADJCALLSTACKDOWN32 0, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:  ADJCALLSTACKDOWN32 0, 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:  CALL32r %0(p0), csr_32, implicit %esp
-; X32-NEXT:  ADJCALLSTACKUP32 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:  ADJCALLSTACKUP32 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:  RET 0
 
 ; X64:      registers:
 ; X64-NEXT:    - { id: 0, class: gr64, preferred-register: '' }
 ; X64:       %0:gr64(p0) = COPY %rdi
-; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:  CALL64r %0(p0), csr_64, implicit %rsp
-; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:  RET 0
 
   call void %func()
   ret void
 }
 
 
 declare void @take_char(i8)
 define void @test_abi_exts_call(i8* %addr) {
 ; ALL-LABEL: name:            test_abi_exts_call
 
 ; X32:       fixedStack:
 ; X32-NEXT:   - { id: 0, type: default, offset: 0, size: 4, alignment: 16,
 ; X32-NEXT:       isImmutable: true,
 ; X32:       %1:_(p0) = G_FRAME_INDEX %fixed-stack.0
 ; X32-NEXT:  %0:_(p0) = G_LOAD %1(p0) :: (invariant load 4 from %fixed-stack.0, align 0)
 ; X32-NEXT:  %2:_(s8) = G_LOAD %0(p0) :: (load 1 from %ir.addr)
-; X32-NEXT:  ADJCALLSTACKDOWN32 4, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:  ADJCALLSTACKDOWN32 4, 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:  %3:_(p0) = COPY %esp
 ; X32-NEXT:  %4:_(s32) = G_CONSTANT i32 0
 ; X32-NEXT:  %5:_(p0) = G_GEP %3, %4(s32)
 ; X32-NEXT:  %6:_(s32) = G_ANYEXT %2(s8)
 ; X32-NEXT:  G_STORE %6(s32), %5(p0) :: (store 4 into stack, align 0)
 ; X32-NEXT:  CALLpcrel32 @take_char, csr_32, implicit %esp
-; X32-NEXT:  ADJCALLSTACKUP32 4, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
-; X32-NEXT:  ADJCALLSTACKDOWN32 4, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:  ADJCALLSTACKUP32 4, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
+; X32-NEXT:  ADJCALLSTACKDOWN32 4, 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:  %7:_(p0) = COPY %esp
 ; X32-NEXT:  %8:_(s32) = G_CONSTANT i32 0
 ; X32-NEXT:  %9:_(p0) = G_GEP %7, %8(s32)
 ; X32-NEXT:  %10:_(s32) = G_SEXT %2(s8)
 ; X32-NEXT:  G_STORE %10(s32), %9(p0) :: (store 4 into stack, align 0)
 ; X32-NEXT:  CALLpcrel32 @take_char, csr_32, implicit %esp
-; X32-NEXT:  ADJCALLSTACKUP32 4, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
-; X32-NEXT:  ADJCALLSTACKDOWN32 4, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:  ADJCALLSTACKUP32 4, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
+; X32-NEXT:  ADJCALLSTACKDOWN32 4, 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:  %11:_(p0) = COPY %esp
 ; X32-NEXT:  %12:_(s32) = G_CONSTANT i32 0
 ; X32-NEXT:  %13:_(p0) = G_GEP %11, %12(s32)
 ; X32-NEXT:  %14:_(s32) = G_ZEXT %2(s8)
 ; X32-NEXT:  G_STORE %14(s32), %13(p0) :: (store 4 into stack, align 0)
 ; X32-NEXT:  CALLpcrel32 @take_char, csr_32, implicit %esp
-; X32-NEXT:  ADJCALLSTACKUP32 4, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:  ADJCALLSTACKUP32 4, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:  RET 0
 
 ; X64:       %0:_(p0) = COPY %rdi
 ; X64-NEXT:  %1:_(s8) = G_LOAD %0(p0) :: (load 1 from %ir.addr)
-; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:  %2:_(s32) = G_ANYEXT %1(s8)
 ; X64-NEXT:  %edi = COPY %2(s32)
-; X64-NEXT:  CALL64pcrel32 @take_char, csr_64, implicit %rsp, implicit %edi
-; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
-; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:  CALL64pcrel32 @take_char, csr_64, implicit %rsp, implicit %ssp, implicit %edi
+; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
+; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:  %3:_(s32) = G_SEXT %1(s8)
 ; X64-NEXT:  %edi = COPY %3(s32)
-; X64-NEXT:  CALL64pcrel32 @take_char, csr_64, implicit %rsp, implicit %edi
-; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
-; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:  CALL64pcrel32 @take_char, csr_64, implicit %rsp, implicit %ssp, implicit %edi
+; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
+; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:  %4:_(s32) = G_ZEXT %1(s8)
 ; X64-NEXT:  %edi = COPY %4(s32)
-; X64-NEXT:  CALL64pcrel32 @take_char, csr_64, implicit %rsp, implicit %edi
-; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:  CALL64pcrel32 @take_char, csr_64, implicit %rsp, implicit %ssp, implicit %edi
+; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:  RET 0
 
   %val = load i8, i8* %addr
   call void @take_char(i8 %val)
   call void @take_char(i8 signext %val)
   call void @take_char(i8 zeroext %val)
   ret void
 }
 
 declare void @variadic_callee(i8*, ...)
 define void @test_variadic_call_1(i8** %addr_ptr, i32* %val_ptr) {
 ; ALL-LABEL: name:            test_variadic_call_1
 
 ; X32:      fixedStack:
 ; X32-NEXT:  - { id: 0, type: default, offset: 4, size: 4, alignment: 4, stack-id: 0,
 ; X32-NEXT:      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true }
 ; X32-NEXT:  - { id: 1, type: default, offset: 0, size: 4, alignment: 16, stack-id: 0,
 ; X32-NEXT:      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true }
 ; X32:         %2:_(p0) = G_FRAME_INDEX %fixed-stack.1
 ; X32-NEXT:    %0:_(p0) = G_LOAD %2(p0) :: (invariant load 4 from %fixed-stack.1, align 0)
 ; X32-NEXT:    %3:_(p0) = G_FRAME_INDEX %fixed-stack.0
 ; X32-NEXT:    %1:_(p0) = G_LOAD %3(p0) :: (invariant load 4 from %fixed-stack.0, align 0)
 ; X32-NEXT:    %4:_(p0) = G_LOAD %0(p0) :: (load 4 from %ir.addr_ptr)
 ; X32-NEXT:    %5:_(s32) = G_LOAD %1(p0) :: (load 4 from %ir.val_ptr)
-; X32-NEXT:    ADJCALLSTACKDOWN32 8, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:    ADJCALLSTACKDOWN32 8, 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:    %6:_(p0) = COPY %esp
 ; X32-NEXT:    %7:_(s32) = G_CONSTANT i32 0
 ; X32-NEXT:    %8:_(p0) = G_GEP %6, %7(s32)
 ; X32-NEXT:    G_STORE %4(p0), %8(p0) :: (store 4 into stack, align 0)
 ; X32-NEXT:    %9:_(p0) = COPY %esp
 ; X32-NEXT:    %10:_(s32) = G_CONSTANT i32 4
 ; X32-NEXT:    %11:_(p0) = G_GEP %9, %10(s32)
 ; X32-NEXT:    G_STORE %5(s32), %11(p0) :: (store 4 into stack + 4, align 0)
 ; X32-NEXT:    CALLpcrel32 @variadic_callee, csr_32, implicit %esp
-; X32-NEXT:    ADJCALLSTACKUP32 8, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:    ADJCALLSTACKUP32 8, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:    RET 0
 
 ; X64:         %0:_(p0) = COPY %rdi
 ; X64-NEXT:    %1:_(p0) = COPY %rsi
 ; X64-NEXT:    %2:_(p0) = G_LOAD %0(p0) :: (load 8 from %ir.addr_ptr)
 ; X64-NEXT:    %3:_(s32) = G_LOAD %1(p0) :: (load 4 from %ir.val_ptr)
-; X64-NEXT:    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:    %rdi = COPY %2(p0)
 ; X64-NEXT:    %esi = COPY %3(s32)
 ; X64-NEXT:    %al = MOV8ri 0
-; X64-NEXT:    CALL64pcrel32 @variadic_callee, csr_64, implicit %rsp, implicit %rdi, implicit %esi, implicit %al
-; X64-NEXT:    ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:    CALL64pcrel32 @variadic_callee, csr_64, implicit %rsp, implicit %ssp, implicit %rdi, implicit %esi, implicit %al
+; X64-NEXT:    ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:    RET 0
 
   %addr = load i8*, i8** %addr_ptr
   %val = load i32, i32* %val_ptr
   call void (i8*, ...) @variadic_callee(i8* %addr, i32 %val)
   ret void
 }
 
 define void @test_variadic_call_2(i8** %addr_ptr, double* %val_ptr) {
 ; ALL-LABEL: name:            test_variadic_call_2
 
 ; X32:      fixedStack:
 ; X32-NEXT:  - { id: 0, type: default, offset: 4, size: 4, alignment: 4, stack-id: 0,
 ; X32-NEXT:      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true }
 ; X32-NEXT:  - { id: 1, type: default, offset: 0, size: 4, alignment: 16, stack-id: 0,
 ; X32-NEXT:      isImmutable: true, isAliased: false, callee-saved-register: '', callee-saved-restored: true }
 ; X32:         %2:_(p0) = G_FRAME_INDEX %fixed-stack.1
 ; X32-NEXT:    %0:_(p0) = G_LOAD %2(p0) :: (invariant load 4 from %fixed-stack.1, align 0)
 ; X32-NEXT:    %3:_(p0) = G_FRAME_INDEX %fixed-stack.0
 ; X32-NEXT:    %1:_(p0) = G_LOAD %3(p0) :: (invariant load 4 from %fixed-stack.0, align 0)
 ; X32-NEXT:    %4:_(p0) = G_LOAD %0(p0) :: (load 4 from %ir.addr_ptr)
 ; X32-NEXT:    %5:_(s64) = G_LOAD %1(p0) :: (load 8 from %ir.val_ptr, align 4)
-; X32-NEXT:    ADJCALLSTACKDOWN32 12, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:    ADJCALLSTACKDOWN32 12, 0, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:    %6:_(p0) = COPY %esp
 ; X32-NEXT:    %7:_(s32) = G_CONSTANT i32 0
 ; X32-NEXT:    %8:_(p0) = G_GEP %6, %7(s32)
 ; X32-NEXT:    G_STORE %4(p0), %8(p0) :: (store 4 into stack, align 0)
 ; X32-NEXT:    %9:_(p0) = COPY %esp
 ; X32-NEXT:    %10:_(s32) = G_CONSTANT i32 4
 ; X32-NEXT:    %11:_(p0) = G_GEP %9, %10(s32)
 ; X32-NEXT:    G_STORE %5(s64), %11(p0) :: (store 8 into stack + 4, align 0)
 ; X32-NEXT:    CALLpcrel32 @variadic_callee, csr_32, implicit %esp
-; X32-NEXT:    ADJCALLSTACKUP32 12, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
+; X32-NEXT:    ADJCALLSTACKUP32 12, 0, implicit-def %esp, implicit-def %eflags, implicit-def %ssp, implicit %esp, implicit %ssp
 ; X32-NEXT:    RET 0
 
 ; X64:         %1:_(p0) = COPY %rsi
 ; X64-NEXT:    %2:_(p0) = G_LOAD %0(p0) :: (load 8 from %ir.addr_ptr)
 ; X64-NEXT:    %3:_(s64) = G_LOAD %1(p0) :: (load 8 from %ir.val_ptr)
-; X64-NEXT:    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:    %rdi = COPY %2(p0)
 ; X64-NEXT:    %xmm0 = COPY %3(s64)
 ; X64-NEXT:    %al = MOV8ri 1
-; X64-NEXT:    CALL64pcrel32 @variadic_callee, csr_64, implicit %rsp, implicit %rdi, implicit %xmm0, implicit %al
-; X64-NEXT:    ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
+; X64-NEXT:    CALL64pcrel32 @variadic_callee, csr_64, implicit %rsp, implicit %ssp, implicit %rdi, implicit %xmm0, implicit %al
+; X64-NEXT:    ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit-def %ssp, implicit %rsp, implicit %ssp
 ; X64-NEXT:    RET 0
 
   %addr = load i8*, i8** %addr_ptr
   %val = load double, double* %val_ptr
   call void (i8*, ...) @variadic_callee(i8* %addr, double %val)
   ret void
 }

test/CodeGen/X86/ipra-reg-usage.ll

 ; RUN: llc -enable-ipra -print-regusage -o /dev/null 2>&1 < %s | FileCheck %s
 
 target triple = "x86_64-unknown-unknown"
 declare void @bar1()
 define preserve_allcc void @foo()#0 {
-; CHECK: foo Clobbered Registers: CS DS EFLAGS EIP EIZ ES FPSW FS GS IP RIP RIZ SS BND0 BND1 BND2 BND3 CR0 CR1 CR2 CR3 CR4 CR5 CR6 CR7 CR8 CR9 CR10 CR11 CR12 CR13 CR14 CR15 DR0 DR1 DR2 DR3 DR4 DR5 DR6 DR7 DR8 DR9 DR10 DR11 DR12 DR13 DR14 DR15 FP0 FP1 FP2 FP3 FP4 FP5 FP6 FP7 K0 K1 K2 K3 K4 K5 K6 K7 MM0 MM1 MM2 MM3 MM4 MM5 MM6 MM7 R11 ST0 ST1 ST2 ST3 ST4 ST5 ST6 ST7 XMM16 XMM17 XMM18 XMM19 XMM20 XMM21 XMM22 XMM23 XMM24 XMM25 XMM26 XMM27 XMM28 XMM29 XMM30 XMM31 YMM0 YMM1 YMM2 YMM3 YMM4 YMM5 YMM6 YMM7 YMM8 YMM9 YMM10 YMM11 YMM12 YMM13 YMM14 YMM15 YMM16 YMM17 YMM18 YMM19 YMM20 YMM21 YMM22 YMM23 YMM24 YMM25 YMM26 YMM27 YMM28 YMM29 YMM30 YMM31 ZMM0 ZMM1 ZMM2 ZMM3 ZMM4 ZMM5 ZMM6 ZMM7 ZMM8 ZMM9 ZMM10 ZMM11 ZMM12 ZMM13 ZMM14 ZMM15 ZMM16 ZMM17 ZMM18 ZMM19 ZMM20 ZMM21 ZMM22 ZMM23 ZMM24 ZMM25 ZMM26 ZMM27 ZMM28 ZMM29 ZMM30 ZMM31 R11B R11D R11W
+; CHECK: foo Clobbered Registers: CS DS EFLAGS EIP EIZ ES FPSW FS GS IP RIP RIZ SS SSP BND0 BND1 BND2 BND3 CR0 CR1 CR2 CR3 CR4 CR5 CR6 CR7 CR8 CR9 CR10 CR11 CR12 CR13 CR14 CR15 DR0 DR1 DR2 DR3 DR4 DR5 DR6 DR7 DR8 DR9 DR10 DR11 DR12 DR13 DR14 DR15 FP0 FP1 FP2 FP3 FP4 FP5 FP6 FP7 K0 K1 K2 K3 K4 K5 K6 K7 MM0 MM1 MM2 MM3 MM4 MM5 MM6 MM7 R11 ST0 ST1 ST2 ST3 ST4 ST5 ST6 ST7 XMM16 XMM17 XMM18 XMM19 XMM20 XMM21 XMM22 XMM23 XMM24 XMM25 XMM26 XMM27 XMM28 XMM29 XMM30 XMM31 YMM0 YMM1 YMM2 YMM3 YMM4 YMM5 YMM6 YMM7 YMM8 YMM9 YMM10 YMM11 YMM12 YMM13 YMM14 YMM15 YMM16 YMM17 YMM18 YMM19 YMM20 YMM21 YMM22 YMM23 YMM24 YMM25 YMM26 YMM27 YMM28 YMM29 YMM30 YMM31 ZMM0 ZMM1 ZMM2 ZMM3 ZMM4 ZMM5 ZMM6 ZMM7 ZMM8 ZMM9 ZMM10 ZMM11 ZMM12 ZMM13 ZMM14 ZMM15 ZMM16 ZMM17 ZMM18 ZMM19 ZMM20 ZMM21 ZMM22 ZMM23 ZMM24 ZMM25 ZMM26 ZMM27 ZMM28 ZMM29 ZMM30 ZMM31 R11B R11D R11W
   call void @bar1()
   call void @bar2()
   ret void
 }
 declare void @bar2()
 attributes #0 = {nounwind}

test/CodeGen/X86/movtopush.mir

--- |
   declare void @llvm.stackprotector(i8*, i8**) #1
 
   attributes #0 = { optsize }
   attributes #1 = { nounwind }
 
 ...
 ---
 # CHECK-LABEL: test9
-# CHECK:      ADJCALLSTACKDOWN32 16, 0, 16, implicit-def dead %esp, implicit-def dead %eflags, implicit %esp
+# CHECK:      ADJCALLSTACKDOWN32 16, 0, 16, implicit-def dead %esp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %esp, implicit %ssp
 # CHECK-NEXT: PUSH32i8 4, implicit-def %esp, implicit %esp
 # CHECK-NEXT: PUSH32i8 3, implicit-def %esp, implicit %esp
 # CHECK-NEXT: PUSH32i8 2, implicit-def %esp, implicit %esp
 # CHECK-NEXT: PUSH32i8 1, implicit-def %esp, implicit %esp
-# CHECK-NEXT: CALLpcrel32 @good, csr_32, implicit %esp, implicit-def %esp
-# CHECK-NEXT: ADJCALLSTACKUP32 16, 0, implicit-def dead %esp, implicit-def dead %eflags, implicit %esp
-# CHECK-NEXT: ADJCALLSTACKDOWN32 20, 0, 20, implicit-def dead %esp, implicit-def dead %eflags, implicit %esp
+# CHECK-NEXT: CALLpcrel32 @good, csr_32, implicit %esp, implicit %ssp, implicit-def %esp, implicit-def %ssp
+# CHECK-NEXT: ADJCALLSTACKUP32 16, 0, implicit-def dead %esp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %esp, implicit %ssp
+# CHECK-NEXT: ADJCALLSTACKDOWN32 20, 0, 20, implicit-def dead %esp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %esp, implicit %ssp
 # CHECK-NEXT: %1:gr32 = MOV32rm %stack.2.s, 1, _, 0, _ :: (load 4 from %stack.2.s, align 8)
 # CHECK-NEXT: %2:gr32 = MOV32rm %stack.2.s, 1, _, 4, _ :: (load 4 from %stack.2.s + 4)
 # CHECK-NEXT: %4:gr32 = LEA32r %stack.0.p, 1, _, 0, _
 # CHECK-NEXT: %5:gr32 = LEA32r %stack.1.q, 1, _, 0, _
 # CHECK-NEXT: PUSH32r %4, implicit-def %esp, implicit %esp
 # CHECK-NEXT: PUSH32r %5, implicit-def %esp, implicit %esp
 # CHECK-NEXT: PUSH32i8 6, implicit-def %esp, implicit %esp
 # CHECK-NEXT: PUSH32r %2, implicit-def %esp, implicit %esp
 # CHECK-NEXT: PUSH32r %1, implicit-def %esp, implicit %esp
-# CHECK-NEXT: CALLpcrel32 @struct, csr_32, implicit %esp, implicit-def %esp
-# CHECK-NEXT: ADJCALLSTACKUP32 20, 0, implicit-def dead %esp, implicit-def dead %eflags, implicit %esp
+# CHECK-NEXT: CALLpcrel32 @struct, csr_32, implicit %esp, implicit %ssp, implicit-def %esp, implicit-def %ssp
+# CHECK-NEXT: ADJCALLSTACKUP32 20, 0, implicit-def dead %esp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %esp, implicit %ssp
 # CHECK-NEXT: RET 0
 name:            test9
 alignment:       0
 exposesReturnsTwice: false
 legalized:       false
 regBankSelected: false
 selected:        false
 tracksRegLiveness: true
- { id: 1, name: q, type: default, offset: 0, size: 4, alignment: 4,
       stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
       di-variable: '', di-expression: '', di-location: '' }
   - { id: 2, name: s, type: default, offset: 0, size: 8, alignment: 8,
       stack-id: 0, callee-saved-register: '', callee-saved-restored: true,
       di-variable: '', di-expression: '', di-location: '' }
 constants:
 body:             |
   bb.0.entry:
-    ADJCALLSTACKDOWN32 16, 0, 0, implicit-def dead %esp, implicit-def dead %eflags, implicit %esp
+    ADJCALLSTACKDOWN32 16, 0, 0, implicit-def dead %esp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %esp, implicit %ssp
     %0 = COPY %esp
     MOV32mi %0, 1, _, 12, _, 4 :: (store 4 into stack + 12)
     MOV32mi %0, 1, _, 8, _, 3 :: (store 4 into stack + 8)
     MOV32mi %0, 1, _, 4, _, 2 :: (store 4 into stack + 4)
     MOV32mi %0, 1, _, 0, _, 1 :: (store 4 into stack)
-    CALLpcrel32 @good, csr_32, implicit %esp, implicit-def %esp
-    ADJCALLSTACKUP32 16, 0, implicit-def dead %esp, implicit-def dead %eflags, implicit %esp
-    ADJCALLSTACKDOWN32 20, 0, 0, implicit-def dead %esp, implicit-def dead %eflags, implicit %esp
+    CALLpcrel32 @good, csr_32, implicit %esp, implicit %ssp, implicit-def %esp, implicit-def %ssp
+    ADJCALLSTACKUP32 16, 0, implicit-def dead %esp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %esp, implicit %ssp
+    ADJCALLSTACKDOWN32 20, 0, 0, implicit-def dead %esp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %esp, implicit %ssp
     %1 = MOV32rm %stack.2.s, 1, _, 0, _ :: (load 4 from %stack.2.s, align 8)
     %2 = MOV32rm %stack.2.s, 1, _, 4, _ :: (load 4 from %stack.2.s + 4)
     %3 = COPY %esp
     MOV32mr %3, 1, _, 4, _, killed %2 :: (store 4)
     MOV32mr %3, 1, _, 0, _, killed %1 :: (store 4)
     %4 = LEA32r %stack.0.p, 1, _, 0, _
     MOV32mr %3, 1, _, 16, _, killed %4 :: (store 4 into stack + 16)
     %5 = LEA32r %stack.1.q, 1, _, 0, _
     MOV32mr %3, 1, _, 12, _, killed %5 :: (store 4 into stack + 12)
     MOV32mi %3, 1, _, 8, _, 6 :: (store 4 into stack + 8)
-    CALLpcrel32 @struct, csr_32, implicit %esp, implicit-def %esp
-    ADJCALLSTACKUP32 20, 0, implicit-def dead %esp, implicit-def dead %eflags, implicit %esp
+    CALLpcrel32 @struct, csr_32, implicit %esp, implicit %ssp, implicit-def %esp, implicit-def %ssp,
+    ADJCALLSTACKUP32 20, 0, implicit-def dead %esp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %esp, implicit %ssp
     RET 0
 
 ...

test/CodeGen/X86/tail-call-conditional.mir

body:             |
 
   ; CHECK: bb.1:
   ; CHECK-NEXT: successors: %bb.2({{[^)]+}}){{$}}
   ; CHECK-NEXT: liveins: %rax, %rsi
   ; CHECK-NEXT: {{^  $}}
   ; CHECK-NEXT: %rdi = COPY %rsi
   ; CHECK-NEXT: %rsi = COPY %rax
   ; CHECK-NEXT: CMP64ri8 %rax, 9, implicit-def %eflags
-  ; CHECK-NEXT: TCRETURNdi64cc @f1, 0, 3, csr_64, implicit %rsp, implicit %eflags, implicit %rsp, implicit %rdi, implicit %rsi, implicit %rax, implicit-def %rax, implicit %sil, implicit-def %sil, implicit %si, implicit-def %si, implicit %esi, implicit-def %esi, implicit %rsi, implicit-def %rsi, implicit %dil, implicit-def %dil, implicit %di, implicit-def %di, implicit %edi, implicit-def %edi, implicit %rdi, implicit-def %rdi, implicit %ah, implicit-def %ah, implicit %al, implicit-def %al, implicit %ax, implicit-def %ax, implicit %eax, implicit-def %eax
+  ; CHECK-NEXT: TCRETURNdi64cc @f1, 0, 3, csr_64, implicit %rsp, implicit %eflags, implicit %ssp, implicit %rsp, implicit %rdi, implicit %rsi, implicit %rax, implicit-def %rax, implicit %sil, implicit-def %sil, implicit %si, implicit-def %si, implicit %esi, implicit-def %esi, implicit %rsi, implicit-def %rsi, implicit %dil, implicit-def %dil, implicit %di, implicit-def %di, implicit %edi, implicit-def %edi, implicit %rdi, implicit-def %rdi, implicit %ah, implicit-def %ah, implicit %al, implicit-def %al, implicit %ax, implicit-def %ax, implicit %eax, implicit-def %eax
 
   bb.1:
     successors: %bb.2, %bb.3
     liveins: %rax, %rsi
 
     CMP64ri8 %rax, 9, implicit-def %eflags
     JA_1 %bb.3, implicit %eflags
     JMP_1 %bb.2

test/CodeGen/X86/x32-cet-intrinsics.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=i386-apple-darwin -mattr=+shstk -mattr=+ibt | FileCheck %s
+
+define void @test_incsspd(i32 %a) local_unnamed_addr {
+; CHECK-LABEL: test_incsspd:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    incsspd %eax
+; CHECK-NEXT:    retl
+entry:
+  tail call void @llvm.x86.incsspd(i32 %a)
+  ret void
+}
+
+declare void @llvm.x86.incsspd(i32)
+
+define i32 @test_rdsspd(i32 %a) {
+; CHECK-LABEL: test_rdsspd:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    rdsspd %eax
+; CHECK-NEXT:    retl
+entry:
+  %0 = call i32 @llvm.x86.rdsspd(i32 %a)
+  ret i32 %0
+}
+
+declare i32 @llvm.x86.rdsspd(i32)
+
+define void @test_saveprevssp() {
+; CHECK-LABEL: test_saveprevssp:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    saveprevssp
+; CHECK-NEXT:    retl
+entry:
+  tail call void @llvm.x86.saveprevssp()
+  ret void
+}
+
+declare void @llvm.x86.saveprevssp()
+
+define void @test_rstorssp(i8* %__p) {
+; CHECK-LABEL: test_rstorssp:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    rstorssp (%eax)
+; CHECK-NEXT:    retl
+entry:
+  tail call void @llvm.x86.rstorssp(i8* %__p)
+  ret void
+}
+
+declare void @llvm.x86.rstorssp(i8*)
+
+define void @test_wrssd(i32 %a, i8* %__p) {
+; CHECK-LABEL: test_wrssd:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    wrssd %eax, (%ecx)
+; CHECK-NEXT:    retl
+entry:
+  tail call void @llvm.x86.wrssd(i32 %a, i8* %__p)
+  ret void
+}
+
+declare void @llvm.x86.wrssd(i32, i8*)
+
+define void @test_wrussd(i32 %a, i8* %__p) {
+; CHECK-LABEL: test_wrussd:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    wrussd %eax, (%ecx)
+; CHECK-NEXT:    retl
+entry:
+  tail call void @llvm.x86.wrussd(i32 %a, i8* %__p)
+  ret void
+}
+
+declare void @llvm.x86.wrussd(i32, i8*)
+
+define void @test_setssbsy() {
+; CHECK-LABEL: test_setssbsy:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    setssbsy
+; CHECK-NEXT:    retl
+entry:
+  tail call void @llvm.x86.setssbsy()
+  ret void
+}
+
+declare void @llvm.x86.setssbsy()
+
+define void @test_clrssbsy(i8* %__p) {
+; CHECK-LABEL: test_clrssbsy:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
+; CHECK-NEXT:    clrssbsy (%eax)
+; CHECK-NEXT:    retl
+entry:
+  tail call void @llvm.x86.clrssbsy(i8* %__p)
+  ret void
+}
+
+declare void @llvm.x86.clrssbsy(i8* %__p)

test/CodeGen/X86/x64-cet-intrinsics.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=x86_64-apple-darwin -mattr=+shstk -mattr=+ibt | FileCheck %s
+
+define void @test_incsspd(i32 %a) local_unnamed_addr {
+; CHECK-LABEL: test_incsspd:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    incsspd %edi
+; CHECK-NEXT:    retq
+entry:
+  tail call void @llvm.x86.incsspd(i32 %a)
+  ret void
+}
+
+declare void @llvm.x86.incsspd(i32)
+
+define void @test_incsspq(i32 %a) local_unnamed_addr {
+; CHECK-LABEL: test_incsspq:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    movslq %edi, %rax
+; CHECK-NEXT:    incsspq %rax
+; CHECK-NEXT:    retq
+entry:
+  %conv.i = sext i32 %a to i64
+  tail call void @llvm.x86.incsspq(i64 %conv.i)
+  ret void
+}
+
+declare void @llvm.x86.incsspq(i64)
+
+define i32 @test_rdsspd(i32 %a) {
+; CHECK-LABEL: test_rdsspd:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    rdsspd %edi
+; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    retq
+entry:
+  %0 = call i32 @llvm.x86.rdsspd(i32 %a)
+  ret i32 %0
+}
+
+declare i32 @llvm.x86.rdsspd(i32)
+
+define i64 @test_rdsspq(i64 %a) {
+; CHECK-LABEL: test_rdsspq:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    rdsspq %rdi
+; CHECK-NEXT:    movq %rdi, %rax
+; CHECK-NEXT:    retq
+entry:
+  %0 = call i64 @llvm.x86.rdsspq(i64 %a)
+  ret i64 %0
+}
+
+declare i64 @llvm.x86.rdsspq(i64)
+
+define void @test_saveprevssp() {
+; CHECK-LABEL: test_saveprevssp:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    saveprevssp
+; CHECK-NEXT:    retq
+entry:
+  tail call void @llvm.x86.saveprevssp()
+  ret void
+}
+
+declare void @llvm.x86.saveprevssp()
+
+define void @test_rstorssp(i8* %__p) {
+; CHECK-LABEL: test_rstorssp:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    rstorssp (%rdi)
+; CHECK-NEXT:    retq
+entry:
+  tail call void @llvm.x86.rstorssp(i8* %__p)
+  ret void
+}
+
+declare void @llvm.x86.rstorssp(i8*)
+
+define void @test_wrssd(i32 %a, i8* %__p) {
+; CHECK-LABEL: test_wrssd:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    wrssd %edi, (%rsi)
+; CHECK-NEXT:    retq
+entry:
+  tail call void @llvm.x86.wrssd(i32 %a, i8* %__p)
+  ret void
+}
+
+declare void @llvm.x86.wrssd(i32, i8*)
+
+define void @test_wrssq(i64 %a, i8* %__p) {
+; CHECK-LABEL: test_wrssq:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    wrssq %rdi, (%rsi)
+; CHECK-NEXT:    retq
+entry:
+  tail call void @llvm.x86.wrssq(i64 %a, i8* %__p)
+  ret void
+}
+
+declare void @llvm.x86.wrssq(i64, i8*)
+
+define void @test_wrussd(i32 %a, i8* %__p) {
+; CHECK-LABEL: test_wrussd:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    wrussd %edi, (%rsi)
+; CHECK-NEXT:    retq
+entry:
+  tail call void @llvm.x86.wrussd(i32 %a, i8* %__p)
+  ret void
+}
+
+declare void @llvm.x86.wrussd(i32, i8*)
+
+define void @test_wrussq(i64 %a, i8* %__p) {
+; CHECK-LABEL: test_wrussq:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    wrussq %rdi, (%rsi)
+; CHECK-NEXT:    retq
+entry:
+  tail call void @llvm.x86.wrussq(i64 %a, i8* %__p)
+  ret void
+}
+
+declare void @llvm.x86.wrussq(i64, i8*)
+
+define void @test_setssbsy() {
+; CHECK-LABEL: test_setssbsy:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    setssbsy
+; CHECK-NEXT:    retq
+entry:
+  tail call void @llvm.x86.setssbsy()
+  ret void
+}
+
+declare void @llvm.x86.setssbsy()
+
+define void @test_clrssbsy(i8* %__p) {
+; CHECK-LABEL: test_clrssbsy:
+; CHECK:       ## BB#0: ## %entry
+; CHECK-NEXT:    clrssbsy (%rdi)
+; CHECK-NEXT:    retq
+entry:
+  tail call void @llvm.x86.clrssbsy(i8* %__p)
+  ret void
+}
+
+declare void @llvm.x86.clrssbsy(i8* %__p)

test/DebugInfo/MIR/X86/live-debug-vars-unused-arg-debugonly.mir

 body:             |
   bb.0.entry:
     DBG_VALUE debug-use %edi, debug-use _, !21, !DIExpression(), debug-location !25
     DBG_VALUE debug-use %rsi, debug-use _, !22, !DIExpression(), debug-location !26
     %2 = MOV32rm %rip, 1, _, @bar, _, debug-location !27 :: (dereferenceable load 4 from `i32* getelementptr inbounds ([2 x i32], [2 x i32]* @bar, i64 0, i64 0)`, !tbaa !28)
     DBG_VALUE debug-use %2, debug-use _, !23, !DIExpression(), debug-location !32
     %3 = MOV32rm %rip, 1, _, @bar + 4, _, debug-location !33 :: (dereferenceable load 4 from `i32* getelementptr inbounds ([2 x i32], [2 x i32]* @bar, i64 0, i64 1)`, !tbaa !28)
     DBG_VALUE debug-use %3, debug-use _, !24, !DIExpression(), debug-location !34
-    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit %rsp, debug-location !35
+    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %rsp, implicit %ssp, debug-location !35
     %edi = COPY %2, debug-location !35
     %esi = COPY %3, debug-location !35
-    CALL64pcrel32 @foo, csr_64, implicit %rsp, implicit killed %edi, implicit killed %esi, implicit-def %rsp, debug-location !35
-    ADJCALLSTACKUP64 0, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit %rsp, debug-location !35
+    CALL64pcrel32 @foo, csr_64, implicit %rsp, implicit %ssp, implicit killed %edi, implicit killed %esi, implicit-def %rsp, debug-location !35
+    ADJCALLSTACKUP64 0, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %rsp, implicit %ssp, debug-location !35
     %eax = MOV32r0 implicit-def dead %eflags, debug-location !36
     RET 0, killed %eax, debug-location !36
 
 ...
 
 # Let's verify that the slot index ranges for the unused variables argc/argv,
 # connected to physical regs %EDI and %RSI, does not overlap with the ranges
 # for %vreg2 and %vreg3. The register allocator is actually allocating the

test/DebugInfo/MIR/X86/live-debug-vars-unused-arg.mir

 body:             |
   bb.0.entry:
     DBG_VALUE debug-use %edi, debug-use _, !21, !DIExpression(), debug-location !25
     DBG_VALUE debug-use %rsi, debug-use _, !22, !DIExpression(), debug-location !26
     %2 = MOV32rm %rip, 1, _, @bar, _, debug-location !27 :: (dereferenceable load 4 from `i32* getelementptr inbounds ([2 x i32], [2 x i32]* @bar, i64 0, i64 0)`, !tbaa !28)
     DBG_VALUE debug-use %2, debug-use _, !23, !DIExpression(), debug-location !32
     %3 = MOV32rm %rip, 1, _, @bar + 4, _, debug-location !33 :: (dereferenceable load 4 from `i32* getelementptr inbounds ([2 x i32], [2 x i32]* @bar, i64 0, i64 1)`, !tbaa !28)
     DBG_VALUE debug-use %3, debug-use _, !24, !DIExpression(), debug-location !34
-    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit %rsp, debug-location !35
+    ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %rsp, implicit %ssp, debug-location !35
     %edi = COPY %2, debug-location !35
     %esi = COPY %3, debug-location !35
-    CALL64pcrel32 @foo, csr_64, implicit %rsp, implicit killed %edi, implicit killed %esi, implicit-def %rsp, debug-location !35
-    ADJCALLSTACKUP64 0, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit %rsp, debug-location !35
+    CALL64pcrel32 @foo, csr_64, implicit %rsp, implicit %ssp, implicit killed %edi, implicit killed %esi, implicit-def %rsp, debug-location !35
+    ADJCALLSTACKUP64 0, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %rsp, implicit %ssp, debug-location !35
     %eax = MOV32r0 implicit-def dead %eflags, debug-location !36
     RET 0, killed %eax, debug-location !36
 
 ...
 
 # Verify that we only get one DBG_VALUE for argc and one for argv.
 #
 # CHECKMIR: ![[ARGC:[0-9]+]] = !DILocalVariable(name: "argc", arg: 1

test/DebugInfo/X86/live-debug-vars-dse.mir

 constants:       
 body:             |
   bb.0.entry:
     liveins: %ecx
   
     %0 = COPY %ecx
     MOV32mr %stack.0.x.addr, 1, _, 0, _, %0 :: (store 4 into %ir.x.addr)
     DBG_VALUE %stack.0.x.addr, 0, !13, !DIExpression(), debug-location !14
-    ADJCALLSTACKDOWN64 32, 0, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit %rsp, debug-location !15
+    ADJCALLSTACKDOWN64 32, 0, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %rsp, implicit %ssp, debug-location !15
     %1 = LEA64r %stack.0.x.addr, 1, _, 0, _
     %rcx = COPY %1, debug-location !15
-    CALL64pcrel32 @escape, csr_win64, implicit %rsp, implicit %rcx, implicit-def %rsp, debug-location !15
-    ADJCALLSTACKUP64 32, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit %rsp, debug-location !15
+    CALL64pcrel32 @escape, csr_win64, implicit %rsp, implicit %ssp, implicit %rcx, implicit-def %rsp, implicit-def %ssp, debug-location !15
+    ADJCALLSTACKUP64 32, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %rsp, implicit %ssp, debug-location !15
     DBG_VALUE 1, debug-use _, !13, !DIExpression(), debug-location !16
     MOV32mi %rip, 1, _, @global, _, 1, debug-location !17 :: (store 4 into @global)
     DBG_VALUE %stack.0.x.addr, 0, !13, !DIExpression(), debug-location !18
     MOV32mi %stack.0.x.addr, 1, _, 0, _, 2, debug-location !18 :: (store 4 into %ir.x.addr)
-    ADJCALLSTACKDOWN64 32, 0, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit %rsp, debug-location !19
+    ADJCALLSTACKDOWN64 32, 0, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %rsp, implicit %ssp, debug-location !19
     %rcx = COPY %1, debug-location !19
-    CALL64pcrel32 @escape, csr_win64, implicit %rsp, implicit %rcx, implicit-def %rsp, debug-location !19
-    ADJCALLSTACKUP64 32, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit %rsp, debug-location !19
+    CALL64pcrel32 @escape, csr_win64, implicit %rsp, implicit %ssp, implicit %rcx, implicit-def %rsp, implicit-def %ssp, debug-location !19
+    ADJCALLSTACKUP64 32, 0, implicit-def dead %rsp, implicit-def dead %eflags, implicit-def dead %ssp, implicit %rsp, implicit %ssp, debug-location !19
     RET 0, debug-location !20
 
 ...

test/MC/X86/cet-encoding.s

+// RUN: llvm-mc -triple x86_64-unknown-unknown -mattr=+shstk --show-encoding %s | FileCheck %s
+
+// CHECK: incsspd %r13d
+// CHECK: # encoding: [0xf3,0x41,0x0f,0xae,0xed]
+          incsspd %r13d
+
+// CHECK: incsspq %r15
+// CHECK: # encoding: [0xf3,0x49,0x0f,0xae,0xef]
+          incsspq %r15
+
+// CHECK: rdsspq %r15
+// CHECK: # encoding: [0xf3,0x49,0x0f,0x1e,0xcf]
+          rdsspq %r15
+
+// CHECK: rdsspd %r13d
+// CHECK: # encoding: [0xf3,0x41,0x0f,0x1e,0xcd]
+          rdsspd %r13d
+
+// CHECK: saveprevssp
+// CHECK: # encoding: [0xf3,0x0f,0x01,0xea]
+          saveprevssp
+
+// CHECK: rstorssp 485498096
+// CHECK: # encoding: [0xf3,0x0f,0x01,0x2c,0x25,0xf0,0x1c,0xf0,0x1c]
+          rstorssp 485498096
+
+// CHECK: rstorssp (%rdx)
+// CHECK: # encoding: [0xf3,0x0f,0x01,0x2a]
+          rstorssp (%rdx)
+
+// CHECK: rstorssp 64(%rdx)
+// CHECK: # encoding: [0xf3,0x0f,0x01,0x6a,0x40]
+          rstorssp 64(%rdx)
+
+// CHECK: rstorssp 64(%rdx,%rax)
+// CHECK: # encoding: [0xf3,0x0f,0x01,0x6c,0x02,0x40]
+          rstorssp 64(%rdx,%rax)
+
+// CHECK: rstorssp 64(%rdx,%rax,4)
+// CHECK: # encoding: [0xf3,0x0f,0x01,0x6c,0x82,0x40]
+          rstorssp 64(%rdx,%rax,4)
+
+// CHECK: rstorssp -64(%rdx,%rax,4)
+// CHECK: # encoding: [0xf3,0x0f,0x01,0x6c,0x82,0xc0]
+          rstorssp -64(%rdx,%rax,4)
+
+// CHECK: wrssq %r15, 485498096
+// CHECK: # encoding: [0x4c,0x0f,0x38,0xf6,0x3c,0x25,0xf0,0x1c,0xf0,0x1c]
+          wrssq %r15, 485498096
+
+// CHECK: wrssq %r15, (%rdx)
+// CHECK: # encoding: [0x4c,0x0f,0x38,0xf6,0x3a]
+          wrssq %r15, (%rdx)
+
+// CHECK: wrssq %r15, 64(%rdx)
+// CHECK: # encoding: [0x4c,0x0f,0x38,0xf6,0x7a,0x40]
+          wrssq %r15, 64(%rdx)
+
+// CHECK: wrssq %r15, 64(%rdx,%rax)
+// CHECK: # encoding: [0x4c,0x0f,0x38,0xf6,0x7c,0x02,0x40]
+          wrssq %r15, 64(%rdx,%rax)
+
+// CHECK: wrssq %r15, 64(%rdx,%rax,4)
+// CHECK: # encoding: [0x4c,0x0f,0x38,0xf6,0x7c,0x82,0x40]
+          wrssq %r15, 64(%rdx,%rax,4)
+
+// CHECK: wrssq %r15, -64(%rdx,%rax,4)
+// CHECK: # encoding: [0x4c,0x0f,0x38,0xf6,0x7c,0x82,0xc0]
+          wrssq %r15, -64(%rdx,%rax,4)
+
+// CHECK: wrssd %r13d, 485498096
+// CHECK: # encoding: [0x44,0x0f,0x38,0xf6,0x2c,0x25,0xf0,0x1c,0xf0,0x1c]
+          wrssd %r13d, 485498096
+
+// CHECK: wrssd %r13d, (%rdx)
+// CHECK: # encoding: [0x44,0x0f,0x38,0xf6,0x2a]
+          wrssd %r13d, (%rdx)
+
+// CHECK: wrssd %r13d, 64(%rdx)
+// CHECK: # encoding: [0x44,0x0f,0x38,0xf6,0x6a,0x40]
+          wrssd %r13d, 64(%rdx)
+
+// CHECK: wrssd %r13d, 64(%rdx,%rax)
+// CHECK: # encoding: [0x44,0x0f,0x38,0xf6,0x6c,0x02,0x40]
+          wrssd %r13d, 64(%rdx,%rax)
+
+// CHECK: wrssd %r13d, 64(%rdx,%rax,4)
+// CHECK: # encoding: [0x44,0x0f,0x38,0xf6,0x6c,0x82,0x40]
+          wrssd %r13d, 64(%rdx,%rax,4)
+
+// CHECK: wrssd %r13d, -64(%rdx,%rax,4)
+// CHECK: # encoding: [0x44,0x0f,0x38,0xf6,0x6c,0x82,0xc0]
+          wrssd %r13d, -64(%rdx,%rax,4)
+
+// CHECK: wrussd %r13d, 485498096
+// CHECK: # encoding: [0x66,0x44,0x0f,0x38,0xf5,0x2c,0x25,0xf0,0x1c,0xf0,0x1c]
+          wrussd %r13d, 485498096
+
+// CHECK: wrussd %r13d, (%rdx)
+// CHECK: # encoding: [0x66,0x44,0x0f,0x38,0xf5,0x2a]
+          wrussd %r13d, (%rdx)
+
+// CHECK: wrussd %r13d, 64(%rdx)
+// CHECK: # encoding: [0x66,0x44,0x0f,0x38,0xf5,0x6a,0x40]
+          wrussd %r13d, 64(%rdx)
+
+// CHECK: wrussd %r13d, 64(%rdx,%rax)
+// CHECK: # encoding: [0x66,0x44,0x0f,0x38,0xf5,0x6c,0x02,0x40]
+          wrussd %r13d, 64(%rdx,%rax)
+
+// CHECK: wrussd %r13d, 64(%rdx,%rax,4)
+// CHECK: # encoding: [0x66,0x44,0x0f,0x38,0xf5,0x6c,0x82,0x40]
+          wrussd %r13d, 64(%rdx,%rax,4)
+
+// CHECK: wrussd %r13d, -64(%rdx,%rax,4)
+// CHECK: # encoding: [0x66,0x44,0x0f,0x38,0xf5,0x6c,0x82,0xc0]
+          wrussd %r13d, -64(%rdx,%rax,4)
+
+// CHECK: wrussq %r15, 485498096
+// CHECK: # encoding: [0x66,0x4c,0x0f,0x38,0xf5,0x3c,0x25,0xf0,0x1c,0xf0,0x1c]
+          wrussq %r15, 485498096
+
+// CHECK: wrussq %r15, (%rdx)
+// CHECK: # encoding: [0x66,0x4c,0x0f,0x38,0xf5,0x3a]
+          wrussq %r15, (%rdx)
+
+// CHECK: wrussq %r15, 64(%rdx)
+// CHECK: # encoding: [0x66,0x4c,0x0f,0x38,0xf5,0x7a,0x40]
+          wrussq %r15, 64(%rdx)
+
+// CHECK: wrussq %r15, 64(%rdx,%rax)
+// CHECK: # encoding: [0x66,0x4c,0x0f,0x38,0xf5,0x7c,0x02,0x40]
+          wrussq %r15, 64(%rdx,%rax)
+
+// CHECK: wrussq %r15, 64(%rdx,%rax,4)
+// CHECK: # encoding: [0x66,0x4c,0x0f,0x38,0xf5,0x7c,0x82,0x40]
+          wrussq %r15, 64(%rdx,%rax,4)
+
+// CHECK: wrussq %r15, -64(%rdx,%rax,4)
+// CHECK: # encoding: [0x66,0x4c,0x0f,0x38,0xf5,0x7c,0x82,0xc0]
+          wrussq %r15, -64(%rdx,%rax,4)
+
+// CHECK: clrssbsy 485498096
+// CHECK: # encoding: [0xf3,0x0f,0xae,0x34,0x25,0xf0,0x1c,0xf0,0x1c]
+          clrssbsy 485498096
+
+// CHECK: clrssbsy (%rdx)
+// CHECK: # encoding: [0xf3,0x0f,0xae,0x32]
+          clrssbsy (%rdx)
+
+// CHECK: clrssbsy 64(%rdx)
+// CHECK: # encoding: [0xf3,0x0f,0xae,0x72,0x40]
+          clrssbsy 64(%rdx)
+
+// CHECK: clrssbsy 64(%rdx,%rax)
+// CHECK: # encoding: [0xf3,0x0f,0xae,0x74,0x02,0x40]
+          clrssbsy 64(%rdx,%rax)
+
+// CHECK: clrssbsy 64(%rdx,%rax,4)
+// CHECK: # encoding: [0xf3,0x0f,0xae,0x74,0x82,0x40]
+          clrssbsy 64(%rdx,%rax,4)
+
+// CHECK: clrssbsy -64(%rdx,%rax,4)
+// CHECK: # encoding: [0xf3,0x0f,0xae,0x74,0x82,0xc0]
+          clrssbsy -64(%rdx,%rax,4)
+
+// CHECK: setssbsy
+// CHECK: # encoding: [0xf3,0x0f,0x01,0xe8]
+          setssbsy