[slh] x86 impl of ARM instrinsic + builtin for SLH

[zbrid]

Summary

This is similar to the work Kristof did for ARM here: https://reviews.llvm.org/D49072

For now, I have only implemented the version that lowers the intrinsic using an LFENCE. I'm workign on a version that can be lowered as an LFENCE or lowered using the control flow speculation available, so users have the option just as they do in the ARM patch.

This is intended to add to the discussion rather than be a definitive patch relating to the way we will handle spot mitigations as far as the final API/implementation in LLVM goes. Any comments about the API, the way implemented this, or anything else are welcome.

Note: This lowering to an lfence appears to be the same method that gcc currently uses in their implementation of the builtin.

• Below is copied from https://reviews.llvm.org/D49073 -----

This is part of implementing a technique to mitigate against Spectre v1,
similar in spirit to what has been proposed by Chandler for X86_64 at
http://lists.llvm.org/pipermail/llvm-dev/2018-March/122085.html.

This patch adds a new builtin function that provides a mechanism for
limiting the effects of miss-speculation by a CPU.
This patch provides the clang-side of the needed functionality; there is
also an llvm-side patch this patch is dependent on.

We've tried to design this in such a way that it can be used for any
target where this might be necessary. The patch provides a generic
implementation of the builtin, with most of the target-specific
support in the LLVM counter part to this clang patch.

The signature of the new, polymorphic, builtin is:

T __builtin_speculation_safe_value(T v)

T can be any integral type (signed or unsigned char, int, short, long,
etc) or any pointer type.

The builtin assures that value v will be made 0 on execution paths that
are being executed under control flow miss-speculation by the CPU, when
the miss-speculated path originated due to misprediction of a direct
conditional branch.

Whereas this still leaves open the possibility of execution on a
miss-speculated path starting at misprediction of other control flow
instructions, our believe is that the above guarantee is still useful in
mitigating vulnerability to Spectre v1-style attacks and implementable
for most, if not all, target instruction sets.

This also introduces the predefined pre-processor macro
__HAVE_SPECULATION_SAFE_VALUE, that allows users to check if their
version of the compiler supports this intrinsic.

[aheejin]

clang/test/Preprocessor/init.c

9678

Nit: Remove the whitespace to be consistent with adjacent lines? (I think having a whitespace is better in general though)

[zbrid]

update whitespace in wasm file to match surrounding

[kristof.beyls]

Thanks for picking this up, Zola!

I quickly looked through the patch - comparing it with what I had done under D49070 and D49073.
Apart from the point remarks inline, I had the following immediate thoughts:

1. Could you clang-format the patch?
2. Could you rebase the patch to top-of-trunk (it seems it is a bit behind ToT)?
3. For discussions, seeing the whole patch as it is might be helpful. OTOH, I think it also makes reviewing easier if the target-dependent and the target-independent parts would be split. I think that could also help others in implementing the intrinsics for their targets: they'd have guidance on what might be needed from that target-dependent implementation patches for X86 and AArch64.
4. Lowering to LFENCE seems a correct lowering to me, but someone more knowledgeable about x86 should confirm.
5. I think the LLVM-IR intrinsic should be target-independent, and not x86-specific. That would result in less duplication of code when implementing support for multiple architectures. I seem to remember that's how I implemented this in D49070. I didn't look so far at the SelectionDAG parts of this patch, as I think the differences between my implementation in D49070 and this patch may go away after making the intrinsic target-independent.

If we'd take the discussion about adding support for intrinsic T __builtin_speculation_safe_value(T v) further here, I'd be happy to abandon the patch series at D49073.
However, in that case, I think the explanation of the intrinsic there should be copied over here to provide a bit more context.

clang/lib/CodeGen/CGBuiltin.cpp

13

This line doesn't seem to be needed?

3987

line too long - run clang-format?

clang/lib/Sema/SemaChecking.cpp

1497

needs one more space of indentation?

5326

Should this be "TheCall->getNumArgs() > 1" (larger than rather then less than)?

clang/test/CodeGen/builtin-speculation-safe-value.c

2–3

When I wrote this test in D49073 this line read "REQUIRES: aarch64-registered-target". Looking at this now, I wonder why the requires might be needed, beyond the RUN line containing "-triple x86_64-linux-gnu". It'd be nice if this test didn't need a REQUIRES line.... But maybe there is a good reason it does need a requires line after all?

clang/test/Preprocessor/init.c

9678

It seems this is the only intended change in this file; all the other changes in this file were unintended for this patch?

llvm/include/llvm/IR/Intrinsics.td

1171

accidental new line diff?

llvm/include/llvm/IR/IntrinsicsX86.td

4819–4822 ↗

(On Diff #192297)

I think this needs to be a target independent LLVM IR intrinsic, not x86 specific. See D49070. This will also need documentation in LangRef.rst then, also see D49070 for a possible documentation I proposed for this intrinsic there.

llvm/lib/Target/X86/X86SpeculativeLoadHardening.cpp

615–629

The lowering of the intrinsic on a 32 bit and a 64 bit value looks identical to me, so the if statement isn't needed?

[zbrid]

update with clang-format

[zbrid]

fix test formatting; make target independent intrinsic; add doc

[zbrid]

remove unnecessary requirement from builtin test

[zbrid]

remove unnecessary if in x86 slh intrinsic lowering function

[zbrid]

llvm/lib/Target/X86/X86SpeculativeLoadHardening.cpp

615–629

Good catch.

[zbrid]

actually fix if statement

[kristof.beyls]

This intrinsic got added to gcc a while ago and should become available in the upcoming gcc 9 release.
In gcc however, the prototype of the intrinsic is slightly different (see https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html):
type __builtin_speculation_safe_value (type val, type failval)
It provides a second optional argument "failval". From the gcc documentation: "The function may use target-dependent speculation tracking state to cause failval to be returned when it is known that speculative execution has incorrectly predicted a conditional branch operation."
So, when implementing the intrinsic using a speculation barrier such as lfence, that failval argument doesn't have any effect. However, when lowering the intrinsic using speculation tracking similar to how that's used in SLH, this failval parameter is used to return a non-zero value on miss-speculation, in case the developer prefers that over the default zero value.

I think we should make the intrinsic compatible with the one introduced in gcc.

llvm/test/CodeGen/X86/speculative-load-hardening-intrinsic.ll

1

I guess the -mtriple command line option may not be needed since the IR file contain "target triple" and "target datalayout" information?

3–4

I guess this is not strictly necessary for this test, so should be removed?

8–62

Thanks for those updates, Zola. It makes it easier to compare this patch with the code I wrote earlier.
Doing that comparison, I see that I had a few changes too in target-independent SelectionDAG under lib/Codegen/SelectionDAG.
IIRC, you might find that you'll need that code if you also add tests here to test the correct thing happens when applying the intrinsic on other types than i32 or i64.
You probably also would want a test on a pointer data type here, I guess.

64–71

I guess this is not strictly necessary for this test, so should be removed?

clang/include/clang/Basic/Builtins.def

 BUILTIN(__xray_customevent, "vcC*z", "")
 BUILTIN(__xray_typedevent, "vzcC*z", "")
 
 // Win64-compatible va_list functions
 BUILTIN(__builtin_ms_va_start, "vc*&.", "nt")
 BUILTIN(__builtin_ms_va_end, "vc*&", "n")
 BUILTIN(__builtin_ms_va_copy, "vc*&c*&", "n")
 
+// T __builtin_speculation_safe_value (T val)
+BUILTIN(__builtin_speculation_safe_value, "vv", "t")
+
 #undef BUILTIN
 #undef LIBBUILTIN
 #undef LANGBUILTIN

clang/include/clang/Basic/DiagnosticSemaKinds.td

def err_spaceship_argument_narrowing : Error<
   "evaluates to %1, which cannot be narrowed to type %2}0">;
 def err_std_compare_type_not_supported : Error<
   "standard library implementation of %0 is not supported; "
    "%select{member '%2' does not have expected form|"
    "member '%2' is missing|"
    "the type is not trivially copyable|"
    "the type does not have the expected form}1">;
 
+def err_specsafevalue_builtin_must_be_pointer_or_integral : Error<
+  "argument to speculation_safe_value builtin must be a pointer or integer "
+  "(%0 invalid)">;
+
 def warn_dereference_of_noderef_type : Warning<
   "dereferencing %0; was declared with a 'noderef' type">, InGroup<NoDeref>;
 def warn_dereference_of_noderef_type_no_decl : Warning<
   "dereferencing expression marked as 'noderef'">, InGroup<NoDeref>;
 def warn_noderef_on_non_pointer_or_array : Warning<
   "'noderef' can only be used on an array or pointer type">, InGroup<IgnoredAttributes>;
 def warn_noderef_to_dereferenceable_pointer : Warning<
   "casting to dereferenceable pointer removes 'noderef' attribute">, InGroup<NoDeref>;
 
 def err_builtin_launder_invalid_arg : Error<
   "%select{non-pointer|function pointer|void pointer}0 argument to "
   "'__builtin_launder' is not allowed">;
 
 } // end of sema component.

clang/include/clang/Sema/Sema.h

private:
   bool SemaBuiltinLongjmp(CallExpr *TheCall);
   bool SemaBuiltinSetjmp(CallExpr *TheCall);
   ExprResult SemaBuiltinAtomicOverloaded(ExprResult TheCallResult);
   ExprResult SemaBuiltinNontemporalOverloaded(ExprResult TheCallResult);
   ExprResult SemaAtomicOpsOverloaded(ExprResult TheCallResult,
                                      AtomicExpr::AtomicOp Op);
   ExprResult SemaBuiltinOperatorNewDeleteOverloaded(ExprResult TheCallResult,
                                                     bool IsDelete);
+  ExprResult
+  SemaBuiltinSpeculationSafeValueOverloaded(ExprResult TheCallResult);
   bool SemaBuiltinConstantArg(CallExpr *TheCall, int ArgNum,
                               llvm::APSInt &Result);
   bool SemaBuiltinConstantArgRange(CallExpr *TheCall, int ArgNum, int Low,
                                    int High, bool RangeIsError = true);
   bool SemaBuiltinConstantArgMultiple(CallExpr *TheCall, int ArgNum,
                                       unsigned Multiple);
   bool SemaBuiltinARMSpecialReg(unsigned BuiltinID, CallExpr *TheCall,
                                 int ArgNum, unsigned ExpectedFieldNum,

clang/lib/CodeGen/CGBuiltin.cpp

 //===---- CGBuiltin.cpp - Emit LLVM Code for builtins ---------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This contains code to emit Builtin calls as LLVM code.
 //
 //===----------------------------------------------------------------------===//
 
 #include "CGCXXABI.h"
 #include "CGObjCRuntime.h"
 #include "CGOpenCLRuntime.h"
 #include "CGRecordLayout.h"
 #include "CodeGenFunction.h"
 #include "CodeGenModule.h"
 #include "ConstantEmitter.h"
 #include "TargetInfo.h"
 #include "clang/AST/ASTContext.h"
case Builtin::BI__builtin_ms_va_copy: {
     DestAddr = Address(Builder.CreateBitCast(DestAddr.getPointer(), BPP, "cp"),
                        DestAddr.getAlignment());
     SrcAddr = Address(Builder.CreateBitCast(SrcAddr.getPointer(), BPP, "ap"),
                       SrcAddr.getAlignment());
 
     Value *ArgPtr = Builder.CreateLoad(SrcAddr, "ap.val");
     return RValue::get(Builder.CreateStore(ArgPtr, DestAddr));
   }
+  case Builtin::BI__builtin_speculation_safe_value: {
+    Value *Val = EmitScalarExpr(E->getArg(0));
+
+    llvm::Type *T = ConvertType(E->getType());
+    assert((isa<llvm::IntegerType>(T) || isa<llvm::PointerType>(T)) &&
+           "unsupported type");
+
+    return RValue::get(Builder.CreateCall(
+        CGM.getIntrinsic(Intrinsic::speculationsafevalue, T), {Val}));
+  }
   }
 
   // If this is an alias for a lib function (e.g. __builtin_sin), emit
   // the call using the normal call path, but using the unmangled
   // version of the function name.
   if (getContext().BuiltinInfo.isLibFunction(BuiltinID))
     return emitLibraryCall(*this, FD, E,
                            CGM.getBuiltinLibFunction(FD, BuiltinID));

clang/lib/Frontend/InitPreprocessor.cpp

#include "clang/Basic/OpenCLExtensions.def"
   if (TI.hasInt128Type() && LangOpts.CPlusPlus && LangOpts.GNUMode) {
     // For each extended integer type, g++ defines a macro mapping the
     // index of the type (0 in this case) in some list of extended types
     // to the type.
     Builder.defineMacro("__GLIBCXX_TYPE_INT_N_0", "__int128");
     Builder.defineMacro("__GLIBCXX_BITSIZE_INT_N_0", "128");
   }
 
+  Builder.defineMacro("__HAVE_SPECULATION_SAFE_VALUE");
+
   // Get other target #defines.
   TI.getTargetDefines(LangOpts, Builder);
 }
 
 /// InitializePreprocessor - Initialize the preprocessor getting it and the
 /// environment ready to process a single file. This returns true on error.
 ///
 void clang::InitializePreprocessor(

clang/lib/Sema/SemaChecking.cpp

case Builtin::BIget_kernel_sub_group_count_for_ndrange:
     if (SemaOpenCLBuiltinNDRangeAndBlock(*this, TheCall))
       return ExprError();
     break;
   case Builtin::BI__builtin_os_log_format:
   case Builtin::BI__builtin_os_log_format_buffer_size:
     if (SemaBuiltinOSLogFormat(TheCall))
       return ExprError();
     break;
+  case Builtin::BI__builtin_speculation_safe_value:
+    return SemaBuiltinSpeculationSafeValueOverloaded(TheCallResult);
   }
 
   // Since the target specific builtins for each arch overlap, only check those
   // of the arch we are compiling for.
   if (Context.BuiltinInfo.isTSBuiltin(BuiltinID)) {
     switch (Context.getTargetInfo().getTriple().getArch()) {
       case llvm::Triple::arm:
       case llvm::Triple::armeb:
ExprResult Sema::SemaBuiltinNontemporalOverloaded(ExprResult TheCallResult) {
   if (ValArg.isInvalid())
     return ExprError();
 
   TheCall->setArg(0, ValArg.get());
   TheCall->setType(Context.VoidTy);
   return TheCallResult;
 }
 
+ExprResult
+Sema::SemaBuiltinSpeculationSafeValueOverloaded(ExprResult TheCallResult) {
+  CallExpr *TheCall = (CallExpr *)TheCallResult.get();
+  DeclRefExpr *DRE =
+      cast<DeclRefExpr>(TheCall->getCallee()->IgnoreParenCasts());
+  FunctionDecl *FDecl = cast<FunctionDecl>(DRE->getDecl());
+  unsigned BuiltinID = FDecl->getBuiltinID();
+  assert(BuiltinID == Builtin::BI__builtin_speculation_safe_value &&
+         "Unexpected speculation_Safe_value builtin!");
+
+  // Too few args
+  if (TheCall->getNumArgs() < 1)
+    return Diag(TheCall->getEndLoc(),
+                diag::err_typecheck_call_too_few_args_at_least)
+           << 0 /*function call*/ << 1 /* min args */ << TheCall->getNumArgs();
+
+  // Too many args
+  if (TheCall->getNumArgs() > 1)
+    return Diag(TheCall->getEndLoc(),
+                diag::err_typecheck_call_too_many_args_at_most)
+           << 0 /*function call*/ << 1 << TheCall->getNumArgs()
+           << SourceRange(TheCall->getArg(1)->getBeginLoc(),
+                          (*(TheCall->arg_end() - 1))->getEndLoc());
+
+  // Derive the return type from the pointer argument
+  ExprResult FirstArg =
+      DefaultFunctionArrayLvalueConversion(TheCall->getArg(0));
+  if (FirstArg.isInvalid())
+    return true;
+  TheCall->setArg(0, FirstArg.get());
+  QualType FirstArgType = FirstArg.get()->getType();
+
+  TheCall->setType(FirstArgType);
+
+  // The first argument must be a pointer or integer type.
+  if (!(FirstArgType->isIntegerType() || FirstArgType->isAnyPointerType()))
+    return Diag(TheCall->getArg(0)->getBeginLoc(),
+                diag::err_specsafevalue_builtin_must_be_pointer_or_integral)
+           << TheCall->getArg(0)->getType()
+           << TheCall->getArg(0)->getSourceRange();
+
+  return TheCallResult;
+}
+
 /// CheckObjCString - Checks that the argument to the builtin
 /// CFString constructor is correct
 /// Note: It might also make sense to do the UTF-16 conversion here (would
 /// simplify the backend).
 bool Sema::CheckObjCString(Expr *Arg) {
   Arg = Arg->IgnoreParenCasts();
   StringLiteral *Literal = dyn_cast<StringLiteral>(Arg);
 

clang/test/CodeGen/builtin-speculation-safe-value.c

+// RUN: %clang_cc1 -triple x86_64-linux-gnu -emit-llvm %s -o - | FileCheck -check-prefix=CHECK-SUPPORTED %s
+
+void test(char c, int i, void *p) {
+  // CHECK-LABEL-SUPPORTED: define void @test
+
+  char c_safe = __builtin_speculation_safe_value(c);
+  // CHECK-SUPPORTED: call i8 @llvm.speculationsafevalue.i8(i8 %{{[0-9a-z]+}})
+
+  int i_safe = __builtin_speculation_safe_value(i);
+  // CHECK-SUPPORTED: call i32 @llvm.speculationsafevalue.i32(i32 %{{[0-9a-z]+}})
+
+  void *p_safe = __builtin_speculation_safe_value(p);
+  // CHECK-SUPPORTED: call i8* @llvm.speculationsafevalue.p0i8(i8* %{{[0-9a-z]+}})
+
+  int arr[4];
+  int *arr_safe = __builtin_speculation_safe_value(arr);
+  // CHECK-SUPPORTED: call i32* @llvm.speculationsafevalue.p0i32(i32* %{{[0-9a-z]+}})
+}

clang/test/Preprocessor/init.c

 // WEBASSEMBLY-NEXT:#define __GCC_ATOMIC_SHORT_LOCK_FREE 2
 // WEBASSEMBLY-NEXT:#define __GCC_ATOMIC_TEST_AND_SET_TRUEVAL 1
 // WEBASSEMBLY-NEXT:#define __GCC_ATOMIC_WCHAR_T_LOCK_FREE 2
 // WEBASSEMBLY-NEXT:#define __GNUC_MINOR__ {{.*}}
 // WEBASSEMBLY-NEXT:#define __GNUC_PATCHLEVEL__ {{.*}}
 // WEBASSEMBLY-NEXT:#define __GNUC_STDC_INLINE__ 1
 // WEBASSEMBLY-NEXT:#define __GNUC__ {{.*}}
 // WEBASSEMBLY-NEXT:#define __GXX_ABI_VERSION 1002
+// WEBASSEMBLY-NEXT:#define __HAVE_SPECULATION_SAFE_VALUE 1
 // WEBASSEMBLY32-NEXT:#define __ILP32__ 1
 // WEBASSEMBLY64-NOT:#define __ILP32__
 // WEBASSEMBLY-NEXT:#define __INT16_C_SUFFIX__
 // WEBASSEMBLY-NEXT:#define __INT16_FMTd__ "hd"
 // WEBASSEMBLY-NEXT:#define __INT16_FMTi__ "hi"
 // WEBASSEMBLY-NEXT:#define __INT16_MAX__ 32767
 // WEBASSEMBLY-NEXT:#define __INT16_TYPE__ short
 // WEBASSEMBLY-NEXT:#define __INT32_C_SUFFIX__

llvm/docs/LangRef.rst

 obviously not constant. However, a call like
 ``llvm.is.constant.i32(i32 %param)`` *can* return true after the
 function is inlined, if the value passed to the function parameter was
 a constant.
 
 On the other hand, if constant folding is not run, it will never
 evaluate to true, even in simple cases.
 
+'``llvm.speculation_safe_value``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+This is an overloaded intrinsic. You can use llvm.speculation_safe_value on any
+integer type, and any pointer type.  However, not all targets support this
+intrinsic at the moment.
+
+::
+
+      declare T @llvm.speculation_safe_value.T(T %val)
+
+Overview:
+"""""""""
+
+The '``llvm.speculation_safe_value``' intrinsic.
+
+Arguments:
+""""""""""
+
+The first argument is a pointer or integer value.
+
+
+Semantics:
+""""""""""
+
+On a processor that predicts the direction and target of branches, code executes
+speculatively, i.e. before it is known if the code actually should be executed
+according to program logic.
+
+When the processor executes code speculatively that it should not execute
+according to program logic, the code is said to be executing on a
+miss-speculated path. Miss-speculated paths are caused by incorrect prediction
+of the direction or targets of branches.
+
+This intrinsic guarantees that for miss-speculated paths where at least the
+direction of one of the previously executed conditional branches was
+mispredicted, the intrinsic returns 0.
+
+On fully correctly predicted execution paths, it returns %val.
+
+For paths not covered by the above statements, it returns either 0 or %val.
+
 Stack Map Intrinsics
 --------------------
 
 LLVM provides experimental intrinsics to support runtime patching
 mechanisms commonly desired in dynamic language JITs. These intrinsics
 are described in :doc:`StackMaps`.
 
 Element Wise Atomic Memory Intrinsics

llvm/include/llvm/CodeGen/ISDOpcodes.h

enum NodeType {
     VECREDUCE_FMAX, VECREDUCE_FMIN,
     /// Integer reductions may have a result type larger than the vector element
     /// type. However, the reduction is performed using the vector element type
     /// and the value in the top bits is unspecified.
     VECREDUCE_ADD, VECREDUCE_MUL,
     VECREDUCE_AND, VECREDUCE_OR, VECREDUCE_XOR,
     VECREDUCE_SMAX, VECREDUCE_SMIN, VECREDUCE_UMAX, VECREDUCE_UMIN,
 
+    SPECULATION_SAFE_VALUE,
+
     /// BUILTIN_OP_END - This must be the last enum value in this list.
     /// The target-specific pre-isel opcode values start here.
     BUILTIN_OP_END
   };
 
   /// FIRST_TARGET_MEMORY_OPCODE - Target-specific pre-isel operations
   /// which do not reference a specific memory location should be less than
   /// this value. Those that do must not be less than this value, and can

llvm/include/llvm/IR/Intrinsics.td

 def int_experimental_vector_reduce_fmin : Intrinsic<[llvm_anyfloat_ty],
                                                     [llvm_anyvector_ty],
                                                     [IntrNoMem]>;
 
 //===----- Intrinsics that are used to provide predicate information -----===//
 
 def int_ssa_copy : Intrinsic<[llvm_any_ty], [LLVMMatchType<0>],
                              [IntrNoMem, Returned<0>]>;
+
+//===----- Intrinsics to mitigate against miss-speculation exploits -------===//
+
+def int_speculationsafevalue : Intrinsic<[llvm_any_ty], [LLVMMatchType<0>], []>;
 //===----------------------------------------------------------------------===//
 // Target-specific intrinsics
 //===----------------------------------------------------------------------===//
 
 include "llvm/IR/IntrinsicsPowerPC.td"
 include "llvm/IR/IntrinsicsX86.td"
 include "llvm/IR/IntrinsicsARM.td"
 include "llvm/IR/IntrinsicsAArch64.td"

llvm/include/llvm/Target/TargetSelectionDAG.td

 def SDTAtomicLoad : SDTypeProfile<1, 1, [
   SDTCisInt<0>, SDTCisPtrTy<1>
 ]>;
 
 def SDTConvertOp : SDTypeProfile<1, 5, [ //cvtss, su, us, uu, ff, fs, fu, sf, su
   SDTCisVT<2, OtherVT>, SDTCisVT<3, OtherVT>, SDTCisPtrTy<4>, SDTCisPtrTy<5>
 ]>;
 
+def SDTSpeculationSafe: SDTypeProfile<1, 1, [
+  SDTCisInt<1>, SDTCisSameAs<1, 0>
+]>;
+
 class SDCallSeqStart<list<SDTypeConstraint> constraints> :
         SDTypeProfile<0, 2, constraints>;
 class SDCallSeqEnd<list<SDTypeConstraint> constraints> :
         SDTypeProfile<0, 2, constraints>;
 
 //===----------------------------------------------------------------------===//
 // Selection DAG Node definitions.
 //

 def intrinsic_wo_chain : SDNode<"ISD::INTRINSIC_WO_CHAIN",
                                 SDTypeProfile<1, -1, [SDTCisPtrTy<1>]>, []>;
 
 def SDT_assertext : SDTypeProfile<1, 1,
   [SDTCisInt<0>, SDTCisInt<1>, SDTCisSameAs<1, 0>]>;
 def assertsext : SDNode<"ISD::AssertSext", SDT_assertext>;
 def assertzext : SDNode<"ISD::AssertZext", SDT_assertext>;
 
+def speculationsafevalue : SDNode<"ISD::SPECULATION_SAFE_VALUE",
+                                  SDTSpeculationSafe, []>;
 
 //===----------------------------------------------------------------------===//
 // Selection DAG Condition Codes
 
 class CondCode; // ISD::CondCode enums
 def SETOEQ : CondCode; def SETOGT : CondCode;
 def SETOGE : CondCode; def SETOLT : CondCode; def SETOLE : CondCode;
 def SETONE : CondCode; def SETO   : CondCode; def SETUO  : CondCode;

llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

#endif
   case ISD::VECREDUCE_OR:
   case ISD::VECREDUCE_XOR:
   case ISD::VECREDUCE_SMAX:
   case ISD::VECREDUCE_SMIN:
   case ISD::VECREDUCE_UMAX:
   case ISD::VECREDUCE_UMIN:
     Res = PromoteIntRes_VECREDUCE(N);
     break;
+  case ISD::SPECULATION_SAFE_VALUE:
+    Res = PromoteIntRes_SpeculationSafeValue(N);
+    break;
   }
 
   // If the result is null then the sub-method took care of registering it.
   if (Res.getNode())
     SetPromotedInteger(SDValue(N, ResNo), Res);
 }
 
 SDValue DAGTypeLegalizer::PromoteIntRes_MERGE_VALUES(SDNode *N,
SDValue DAGTypeLegalizer::PromoteIntRes_SimpleIntBinOp(SDNode *N) {
   // these operations don't care.  They may have weird bits going out, but
   // that too is okay if they are integer operations.
   SDValue LHS = GetPromotedInteger(N->getOperand(0));
   SDValue RHS = GetPromotedInteger(N->getOperand(1));
   return DAG.getNode(N->getOpcode(), SDLoc(N),
                      LHS.getValueType(), LHS, RHS);
 }
 
+SDValue DAGTypeLegalizer::PromoteIntRes_SpeculationSafeValue(SDNode *N) {
+  // Propagate size promotion through the intrinsic.
+  SDValue Op = GetPromotedInteger(N->getOperand(0));
+  return DAG.getNode(N->getOpcode(), SDLoc(N),
+                     Op.getValueType(), Op);
+}
+
 SDValue DAGTypeLegalizer::PromoteIntRes_SExtIntBinOp(SDNode *N) {
   // Sign extend the input.
   SDValue LHS = SExtPromotedInteger(N->getOperand(0));
   SDValue RHS = SExtPromotedInteger(N->getOperand(1));
   return DAG.getNode(N->getOpcode(), SDLoc(N),
                      LHS.getValueType(), LHS, RHS);
 }
 
#endif
 
   case ISD::SADDO:
   case ISD::SSUBO: ExpandIntRes_SADDSUBO(N, Lo, Hi); break;
   case ISD::UADDO:
   case ISD::USUBO: ExpandIntRes_UADDSUBO(N, Lo, Hi); break;
   case ISD::UMULO:
   case ISD::SMULO: ExpandIntRes_XMULO(N, Lo, Hi); break;
 
+  case ISD::SPECULATION_SAFE_VALUE:
+                   ExpandIntRes_SPECULATION_SAFE_VALUE(N, Lo, Hi); break;
+
   case ISD::SADDSAT:
   case ISD::UADDSAT:
   case ISD::SSUBSAT:
   case ISD::USUBSAT: ExpandIntRes_ADDSUBSAT(N, Lo, Hi); break;
   case ISD::SMULFIX:
   case ISD::UMULFIX: ExpandIntRes_MULFIX(N, Lo, Hi); break;
 
   case ISD::VECREDUCE_ADD:
void DAGTypeLegalizer::ExpandIntRes_CTPOP(SDNode *N,
   // ctpop(HiLo) -> ctpop(Hi)+ctpop(Lo)
   GetExpandedInteger(N->getOperand(0), Lo, Hi);
   EVT NVT = Lo.getValueType();
   Lo = DAG.getNode(ISD::ADD, dl, NVT, DAG.getNode(ISD::CTPOP, dl, NVT, Lo),
                    DAG.getNode(ISD::CTPOP, dl, NVT, Hi));
   Hi = DAG.getConstant(0, dl, NVT);
 }
 
+void DAGTypeLegalizer::ExpandIntRes_SPECULATION_SAFE_VALUE(SDNode *N,
+                                                           SDValue &Lo,
+                                                           SDValue &Hi) {
+  SDLoc dl(N);
+  GetExpandedInteger(N->getOperand(0), Lo, Hi);
+  Lo = DAG.getNode(N->getOpcode(), dl, Lo.getValueType(), Lo);
+  Hi = DAG.getNode(N->getOpcode(), dl, Hi.getValueType(), Hi);
+}
+
 void DAGTypeLegalizer::ExpandIntRes_CTTZ(SDNode *N,
                                          SDValue &Lo, SDValue &Hi) {
   SDLoc dl(N);
   // cttz (HiLo) -> Lo != 0 ? cttz(Lo) : (cttz(Hi)+32)
   GetExpandedInteger(N->getOperand(0), Lo, Hi);
   EVT NVT = Lo.getValueType();
 
   SDValue LoNotZero = DAG.getSetCC(dl, getSetCCResultType(NVT), Lo,

llvm/lib/CodeGen/SelectionDAG/LegalizeTypes.h

private:
   SDValue PromoteIntRes_SRA(SDNode *N);
   SDValue PromoteIntRes_SRL(SDNode *N);
   SDValue PromoteIntRes_TRUNCATE(SDNode *N);
   SDValue PromoteIntRes_UADDSUBO(SDNode *N, unsigned ResNo);
   SDValue PromoteIntRes_ADDSUBCARRY(SDNode *N, unsigned ResNo);
   SDValue PromoteIntRes_UNDEF(SDNode *N);
   SDValue PromoteIntRes_VAARG(SDNode *N);
   SDValue PromoteIntRes_XMULO(SDNode *N, unsigned ResNo);
+  SDValue PromoteIntRes_SpeculationSafeValue(SDNode *N);
   SDValue PromoteIntRes_ADDSUBSAT(SDNode *N);
   SDValue PromoteIntRes_MULFIX(SDNode *N);
   SDValue PromoteIntRes_FLT_ROUNDS(SDNode *N);
   SDValue PromoteIntRes_VECREDUCE(SDNode *N);
   SDValue PromoteIntRes_ABS(SDNode *N);
 
   // Integer Operand Promotion.
   bool PromoteIntegerOperand(SDNode *N, unsigned OpNo);
private:
   void ExpandIntRes_UREM              (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_Shift             (SDNode *N, SDValue &Lo, SDValue &Hi);
 
   void ExpandIntRes_MINMAX            (SDNode *N, SDValue &Lo, SDValue &Hi);
 
   void ExpandIntRes_SADDSUBO          (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_UADDSUBO          (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_XMULO             (SDNode *N, SDValue &Lo, SDValue &Hi);
+  void ExpandIntRes_SPECULATION_SAFE_VALUE(SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_ADDSUBSAT         (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_MULFIX            (SDNode *N, SDValue &Lo, SDValue &Hi);
 
   void ExpandIntRes_ATOMIC_LOAD       (SDNode *N, SDValue &Lo, SDValue &Hi);
   void ExpandIntRes_VECREDUCE         (SDNode *N, SDValue &Lo, SDValue &Hi);
 
   void ExpandShiftByConstant(SDNode *N, const APInt &Amt,
                              SDValue &Lo, SDValue &Hi);

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

case Intrinsic::ctlz: {
     return nullptr;
   }
   case Intrinsic::ctpop: {
     SDValue Arg = getValue(I.getArgOperand(0));
     EVT Ty = Arg.getValueType();
     setValue(&I, DAG.getNode(ISD::CTPOP, sdl, Ty, Arg));
     return nullptr;
   }
+  case Intrinsic::speculationsafevalue: {
+    SDValue Arg = getValue(I.getArgOperand(0));
+    EVT Ty = Arg.getValueType();
+    setValue(&I, DAG.getNode(ISD::SPECULATION_SAFE_VALUE, sdl, Ty, Arg));
+    return nullptr;
+  }
   case Intrinsic::fshl:
   case Intrinsic::fshr: {
     bool IsFSHL = Intrinsic == Intrinsic::fshl;
     SDValue X = getValue(I.getArgOperand(0));
     SDValue Y = getValue(I.getArgOperand(1));
     SDValue Z = getValue(I.getArgOperand(2));
     EVT VT = X.getValueType();
     SDValue BitWidthC = DAG.getConstant(VT.getScalarSizeInBits(), sdl, VT);

llvm/lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp

if (G) {
       if (Name) return Name;
       return "<<Unknown Target Node #" + utostr(getOpcode()) + ">>";
     }
     return "<<Unknown Node #" + utostr(getOpcode()) + ">>";
 
 #ifndef NDEBUG
   case ISD::DELETED_NODE:               return "<<Deleted Node!>>";
 #endif
+  case ISD::SPECULATION_SAFE_VALUE:     return "SpeculationSafeValue";
   case ISD::PREFETCH:                   return "Prefetch";
   case ISD::ATOMIC_FENCE:               return "AtomicFence";
   case ISD::ATOMIC_CMP_SWAP:            return "AtomicCmpSwap";
   case ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS: return "AtomicCmpSwapWithSuccess";
   case ISD::ATOMIC_SWAP:                return "AtomicSwap";
   case ISD::ATOMIC_LOAD_ADD:            return "AtomicLoadAdd";
   case ISD::ATOMIC_LOAD_SUB:            return "AtomicLoadSub";
   case ISD::ATOMIC_LOAD_AND:            return "AtomicLoadAnd";

llvm/lib/Target/X86/X86ISelLowering.h

namespace llvm {
   class X86TargetMachine;
 
   namespace X86ISD {
     // X86 Specific DAG Nodes
     enum NodeType : unsigned {
       // Start the numbering where the builtin ops leave off.
       FIRST_NUMBER = ISD::BUILTIN_OP_END,
 
+      SpeculationSafeValue,
+
       /// Bit scan forward.
       BSF,
       /// Bit scan reverse.
       BSR,
 
       /// Double shift instructions. These correspond to
       /// X86::SHLDxx and X86::SHRDxx instructions.
       SHLD,
private:
     std::vector<APFloat> LegalFPImmediates;
 
     /// Indicate that this x86 target can instruction
     /// select the specified FP immediate natively.
     void addLegalFPImmediate(const APFloat& Imm) {
       LegalFPImmediates.push_back(Imm);
     }
 
+    SDValue LowerSPECULATION_SAFE_VALUE(SDValue Op, SelectionDAG &DAG) const;
+
     SDValue LowerCallResult(SDValue Chain, SDValue InFlag,
                             CallingConv::ID CallConv, bool isVarArg,
                             const SmallVectorImpl<ISD::InputArg> &Ins,
                             const SDLoc &dl, SelectionDAG &DAG,
                             SmallVectorImpl<SDValue> &InVals,
                             uint32_t *RegMask) const;
     SDValue LowerMemArgument(SDValue Chain, CallingConv::ID CallConv,
                              const SmallVectorImpl<ISD::InputArg> &ArgInfo,

llvm/lib/Target/X86/X86ISelLowering.cpp

X86TargetLowering::X86TargetLowering(const X86TargetMachine &TM,
   // SETOEQ and SETUNE require checking two conditions.
   setCondCodeAction(ISD::SETOEQ, MVT::f32, Expand);
   setCondCodeAction(ISD::SETOEQ, MVT::f64, Expand);
   setCondCodeAction(ISD::SETOEQ, MVT::f80, Expand);
   setCondCodeAction(ISD::SETUNE, MVT::f32, Expand);
   setCondCodeAction(ISD::SETUNE, MVT::f64, Expand);
   setCondCodeAction(ISD::SETUNE, MVT::f80, Expand);
 
+  setOperationAction(ISD::SPECULATION_SAFE_VALUE, MVT::i32, Custom);
+  setOperationAction(ISD::SPECULATION_SAFE_VALUE, MVT::i64, Custom);
+
   // Integer absolute.
   if (Subtarget.hasCMov()) {
     setOperationAction(ISD::ABS            , MVT::i16  , Custom);
     setOperationAction(ISD::ABS            , MVT::i32  , Custom); 
   }
   setOperationAction(ISD::ABS              , MVT::i64  , Custom);
 
   // Funnel shifts.
static bool hasFPCMov(unsigned X86CC) {
   case X86::COND_A:
   case X86::COND_AE:
   case X86::COND_NE:
   case X86::COND_NP:
     return true;
   }
 }
 
+SDValue
+X86TargetLowering::LowerSPECULATION_SAFE_VALUE(SDValue Op,
+                                               SelectionDAG &DAG) const {
+
+  assert((Op.getValueType() == MVT::i64 || Op.getValueType() == MVT::i32) &&
+         "Unexpected lowering");
+
+  SDLoc DL(Op);
+  return DAG.getNode(X86ISD::SpeculationSafeValue, DL, Op.getValueType(),
+                     Op.getOperand(0));
+}
 
 bool X86TargetLowering::getTgtMemIntrinsic(IntrinsicInfo &Info,
                                            const CallInst &I,
                                            MachineFunction &MF,
                                            unsigned Intrinsic) const {
 
   const IntrinsicData* IntrData = getIntrinsicWithChain(Intrinsic);
   if (!IntrData)
SDValue X86TargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
   case ISD::FSINCOS:            return LowerFSINCOS(Op, Subtarget, DAG);
   case ISD::MLOAD:              return LowerMLOAD(Op, Subtarget, DAG);
   case ISD::MSTORE:             return LowerMSTORE(Op, Subtarget, DAG);
   case ISD::MGATHER:            return LowerMGATHER(Op, Subtarget, DAG);
   case ISD::MSCATTER:           return LowerMSCATTER(Op, Subtarget, DAG);
   case ISD::GC_TRANSITION_START:
                                 return LowerGC_TRANSITION_START(Op, DAG);
   case ISD::GC_TRANSITION_END:  return LowerGC_TRANSITION_END(Op, DAG);
+  case ISD::SPECULATION_SAFE_VALUE:
+                                return LowerSPECULATION_SAFE_VALUE(Op, DAG);
   }
 }
 
 /// Places new result values for the node in Results (their number
 /// and types must exactly match those of the original return values of
 /// the node), or leaves Results empty, which indicates that the node is not
 /// to be custom lowered after all.
 void X86TargetLowering::LowerOperationWrapper(SDNode *N,
const char *X86TargetLowering::getTargetNodeName(unsigned Opcode) const {
   case X86ISD::VPSHUFBITQMB:       return "X86ISD::VPSHUFBITQMB";
   case X86ISD::GF2P8MULB:          return "X86ISD::GF2P8MULB";
   case X86ISD::GF2P8AFFINEQB:      return "X86ISD::GF2P8AFFINEQB";
   case X86ISD::GF2P8AFFINEINVQB:   return "X86ISD::GF2P8AFFINEINVQB";
   case X86ISD::NT_CALL:            return "X86ISD::NT_CALL";
   case X86ISD::NT_BRIND:           return "X86ISD::NT_BRIND";
   case X86ISD::UMWAIT:             return "X86ISD::UMWAIT";
   case X86ISD::TPAUSE:             return "X86ISD::TPAUSE";
+  case X86ISD::SpeculationSafeValue:
+                                   return "X86ISD::SpeculationSafeValue";
   }
   return nullptr;
 }
 
 /// Return true if the addressing mode represented by AM is legal for this
 /// target, for a load/store of the specified type.
 bool X86TargetLowering::isLegalAddressingMode(const DataLayout &DL,
                                               const AddrMode &AM, Type *Ty,

llvm/lib/Target/X86/X86InstrInfo.td

def X86umwait : SDNode<"X86ISD::UMWAIT",
                                             SDTCisVT<2, i32>, SDTCisVT<3, i32>]>,
                        [SDNPHasChain, SDNPSideEffect]>;
 
 def X86tpause : SDNode<"X86ISD::TPAUSE",
                        SDTypeProfile<1, 3, [SDTCisVT<0, i32>, SDTCisInt<1>,
                                             SDTCisVT<2, i32>, SDTCisVT<3, i32>]>,
                        [SDNPHasChain, SDNPSideEffect]>;
 
+def X86SpeculationSafeValue : SDNode<"X86ISD::SpeculationSafeValue", SDTIntUnaryOp>;
+
+
 //===----------------------------------------------------------------------===//
 // X86 Operand Definitions.
 //
 
 // A version of ptr_rc which excludes SP, ESP, and RSP. This is used for
 // the index operand of an address, to conform to x86 encoding restrictions.
 def ptr_rc_nosp : PointerLikeRegClass<1>;
 

 def LEAVE64  : I<0xC9, RawFrm, (outs), (ins), "leave", []>,
                  Requires<[In64BitMode]>;
 } // SchedRW
 
 //===----------------------------------------------------------------------===//
 //  Miscellaneous Instructions.
 //
 
+let hasSideEffects = 1, isCodeGenOnly = 1 in {
+  def SpeculationSafeValue64
+    : PseudoI<(outs GR64:$dst), (ins GR64:$src),
+        [(set GR64:$dst, (X86SpeculationSafeValue GR64:$src))]>;
+  def SpeculationSafeValue32
+    : PseudoI<(outs GR32:$dst), (ins GR32:$src),
+        [(set GR32:$dst, (X86SpeculationSafeValue GR32:$src))]>;
+}
+
 let isBarrier = 1, hasSideEffects = 1, usesCustomInserter = 1,
     SchedRW = [WriteSystem] in
   def Int_eh_sjlj_setup_dispatch
     : PseudoI<(outs), (ins), [(X86eh_sjlj_setup_dispatch)]>;
 
 let Defs = [ESP], Uses = [ESP], hasSideEffects=0 in {
 let mayLoad = 1, SchedRW = [WriteLoad] in {
 def POP16r  : I<0x58, AddRegFrm, (outs GR16:$reg), (ins), "pop{w}\t$reg", []>,

llvm/lib/Target/X86/X86SpeculativeLoadHardening.cpp

unsigned hardenValueInRegister(unsigned Reg, MachineBasicBlock &MBB,
                                  MachineBasicBlock::iterator InsertPt,
                                  DebugLoc Loc);
   unsigned hardenPostLoad(MachineInstr &MI);
   void hardenReturnInstr(MachineInstr &MI);
   void tracePredStateThroughCall(MachineInstr &MI);
   void hardenIndirectCallOrJumpInstr(
       MachineInstr &MI,
       SmallDenseMap<unsigned, unsigned, 32> &AddrRegToHardenedReg);
+  bool lowerIntrinsic(MachineFunction &MF);
 };
 
 } // end anonymous namespace
 
 char X86SpeculativeLoadHardeningPass::ID = 0;
 
 void X86SpeculativeLoadHardeningPass::getAnalysisUsage(
     AnalysisUsage &AU) const {
static bool hasVulnerableLoad(MachineFunction &MF) {
   return false;
 }
 
 bool X86SpeculativeLoadHardeningPass::runOnMachineFunction(
     MachineFunction &MF) {
   LLVM_DEBUG(dbgs() << "********** " << getPassName() << " : " << MF.getName()
                     << " **********\n");
 
-  // Only run if this pass is forced enabled or we detect the relevant function
-  // attribute requesting SLH.
-  if (!EnableSpeculativeLoadHardening &&
-      !MF.getFunction().hasFnAttribute(Attribute::SpeculativeLoadHardening))
-    return false;
-
   Subtarget = &MF.getSubtarget<X86Subtarget>();
   MRI = &MF.getRegInfo();
   TII = Subtarget->getInstrInfo();
   TRI = Subtarget->getRegisterInfo();
+  bool Modified = lowerIntrinsic(MF);
+
+  // Only run this pass completely if it is forced enabled or if we detect the
+  // relevant function attribute requesting SLH. Otherwise we should only check
+  // for intrinsics that we must lower by adding an lfence.
+  if (!EnableSpeculativeLoadHardening &&
+      !MF.getFunction().hasFnAttribute(Attribute::SpeculativeLoadHardening)) {
+
+    return false || Modified;
+  }
 
   // FIXME: Support for 32-bit.
   PS.emplace(MF, &X86::GR64_NOSPRegClass);
 
   if (MF.begin() == MF.end())
     // Nothing to do for a degenerate empty function...
     return false;
 
void X86SpeculativeLoadHardeningPass::hardenEdgesWithLFENCE(
   for (MachineBasicBlock *MBB : Blocks) {
     auto InsertPt = MBB->SkipPHIsAndLabels(MBB->begin());
     BuildMI(*MBB, InsertPt, DebugLoc(), TII->get(X86::LFENCE));
     ++NumInstsInserted;
     ++NumLFENCEsInserted;
   }
 }
 
+bool X86SpeculativeLoadHardeningPass::lowerIntrinsic(MachineFunction &MF) {
+  bool Modified = false;
+  for (MachineBasicBlock &MBB : MF) {
+    MachineBasicBlock::iterator MBBI = MBB.begin();
+    MachineBasicBlock::iterator MBBE = MBB.end();
+    while (MBBI != MBBE) {
+      MachineBasicBlock::iterator NMBBI = std::next(MBBI);
+      MachineInstr &MI = *MBBI;
+      unsigned Opcode = MI.getOpcode();
+      if (Opcode == X86::SpeculationSafeValue64 ||
+          Opcode == X86::SpeculationSafeValue32) {
+        BuildMI(MBB, NMBBI, DebugLoc(), TII->get(X86::LFENCE));
+        ++NumInstsInserted;
+        ++NumLFENCEsInserted;
+        MRI->replaceRegWith(MI.getOperand(0).getReg(),
+                            MI.getOperand(1).getReg());
+        MI.eraseFromParent();
+        Modified = true;
+      }
+      MBBI = NMBBI;
+    }
+  }
+  return Modified;
+}
+
 SmallVector<X86SpeculativeLoadHardeningPass::BlockCondInfo, 16>
 X86SpeculativeLoadHardeningPass::collectBlockCondInfo(MachineFunction &MF) {
   SmallVector<BlockCondInfo, 16> Infos;
 
   // Walk the function and build up a summary for each block's conditions that
   // we need to trace through.
   for (MachineBasicBlock &MBB : MF) {
     // If there are no or only one successor, nothing to do here.

llvm/test/CodeGen/X86/speculative-load-hardening-intrinsic.ll

+; RUN: llc < %s -mtriple=x86_64-unknown-linux-gnu | FileCheck %s --check-prefix=X64
+
+; ModuleID = 'hello.cpp'
+source_filename = "hello.cpp"
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+; Function Attrs: noinline nounwind optnone uwtable
+define dso_local i32 @_Z5foo32i(i32 %a) #0 {
+entry:
+  %a.addr = alloca i32, align 4
+  %b = alloca i32, align 4
+  %b_safe = alloca i32, align 4
+  %c = alloca i32, align 4
+  store i32 %a, i32* %a.addr, align 4
+  %0 = load i32, i32* %a.addr, align 4
+  %mul = mul nsw i32 %0, 100
+  store i32 %mul, i32* %b, align 4
+  %1 = load i32, i32* %b, align 4
+  %2 = call i32 @llvm.speculationsafevalue.i32(i32 %1)
+; X64: movl -12(%rbp), %eax
+; X64: lfence
+; X64: movl %eax, -8(%rbp)
+  store i32 %2, i32* %b_safe, align 4
+  %3 = load i32, i32* %b_safe, align 4
+  %add = add nsw i32 %3, 100
+  store i32 %add, i32* %c, align 4
+  %4 = load i32, i32* %c, align 4
+  ret i32 %4
+}
+
+; Function Attrs: nounwind
+declare i32 @llvm.speculationsafevalue.i32(i32) #1
+
+; Function Attrs: noinline nounwind optnone uwtable
+define dso_local i32 @_Z5foo64i(i32 %a) #0 {
+entry:
+  %a.addr = alloca i32, align 4
+  %b = alloca i64, align 8
+  %b_safe = alloca i64, align 8
+  %c = alloca i64, align 8
+  store i32 %a, i32* %a.addr, align 4
+  %0 = load i32, i32* %a.addr, align 4
+  %mul = mul nsw i32 %0, 100
+  %conv = sext i32 %mul to i64
+  store i64 %conv, i64* %b, align 8
+  %1 = load i64, i64* %b, align 8
+  %2 = call i64 @llvm.speculationsafevalue.i64(i64 %1)
+; X64: movq -32(%rbp), %rax
+; X64: lfence
+; X64: movq %rax, -24(%rbp)
+  store i64 %2, i64* %b_safe, align 8
+  %3 = load i64, i64* %b_safe, align 8
+  %add = add nsw i64 %3, 100
+  store i64 %add, i64* %c, align 8
+  %4 = load i64, i64* %c, align 8
+  %conv1 = trunc i64 %4 to i32
+  ret i32 %conv1
+}
+
+; Function Attrs: nounwind
+declare i64 @llvm.speculationsafevalue.i64(i64) #1
+
+attributes #0 = { noinline nounwind optnone uwtable "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "less-precise-fpmad"="false" "min-legal-vector-width"="0" "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+fxsr,+mmx,+sse,+sse2,+x87" "unsafe-fp-math"="false" "use-soft-float"="false" }
+attributes #1 = { nounwind }
+
+!llvm.module.flags = !{!0}
+!llvm.ident = !{!1}
+
+!0 = !{i32 1, !"wchar_size", i32 4}
+!1 = !{!"clang version 9.0.0 (https://github.com/llvm/llvm-project.git 6fd90b5505fe7cddd0fd798fe9608ea0e0325302)"}