Add inline assembly load hardening mitigation for Load Value Injection (LVI) on X86 [6/6]

[sconstab]

Summary

Added code to X86AsmParser::emitInstruction() to add an LFENCE after each instruction that may load, and emit a warning if it encounters an instruction that may be vulnerable, but cannot be automatically mitigated.

[sconstab]

Added warnings for instructions that cannot be automatically mitigating by inserting an LFENCE.

[craig.topper]

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

3261

Can we move this code out to a separate method called from here? Something like performLVIMitigation?

[zbrid]

LGTM. This is awesome, thanks! Someone else should give another LGTM before submitting.

[sconstab]

Separated out the load/cfi hardening functionality into two separate member functions.

[sconstab]

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

3261

I created two new methods. Let me know what you think.

[mattdr]

This seems like a much more robust approach for mitigating inline asm. Thanks for that!

For now, my biggest question is around layering and separation of concerns. Do we really want to do transformations in the parser? It seems like that opens the door to making the parser API surprising and less helpful. For example, if the LVI mitigation is enabled the Parse and Emit operations of the parser no longer round-trip, which is very new and very nonobvious.

I definitely understand that you put the code here because it was the one path you could find that all attempts to emit ASM would go through. And I know it's frustrating I don't have a concrete suggestion for a different approach. Still working on that.

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

3165

Let's add a comment about how this list was created, or maybe a reference to public documentation if this list is pulled from there. Otherwise folks coming afterward have no way to tell if this list is correct.

3193

Once the details of LVI have faded, future maintainers will really appreciate a comment here about what this mitigation *is*, what it's doing, etc.

3199

Same comment as for the list above.

[sconstab]

Added comments with pointers to LVI public documentation.

[mattdr]

First, want to apologize for the high latency -- thanks to (gestures vaguely in the air) I've been mostly offline during the day and trying to catch up at night.

I think I've probably said my piece here. Summarizing themes from my comments:

1. the parser seems like a weird place to put this mitigation
2. the lists of instructions feel very magic because there's no explanation of how they were arrived at. They can also be incomplete or become out of date. Consider fixing both by matching instructions by interesting properties instead of opcodes. Include an explanation of why the properties make the instructions interesting in the context of LVI.
3. the warnings we fire aren't actionable

Ultimately, though, these are code-quality and maintainability issues and I'm not a maintainer. Especially since I'm only occasionally-online right now, please don't consider me a blocker for the review.

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

3174

Following up on my previous comment: it seems like this is supposed to be the full set of instructions that combine a load from memory with a control flow change dependent on that load.

How are we sure the list is complete? Just from a quick look at the return instructions LLVM has for X86, it looks like LRETL and LRETQ exist and would not be mitigated.

Can we just look at the MCInstDesc and warn for instructions that mayLoad and mayAffectControlFlow? Then this list will stay up to date even if new instructions are added.

3193

This warning should at least tell users *why* this is vulnerable, and ideally offer the next step for fixing it.

3213

I didn't find any mention of these instructions, why they would "require additional considerations", or what folks should actually *do* if they hit this warning by reading https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection.

Here's my best guess:
This is a list of [[ https://cpu.fyi/d/484#G7.340223 | string instructions that can be used with REP ]] and which update ZF in EFLAGS instead of relying on an a count in rcx to terminate". When used with REP or REPNE, these instructions become interesting because an attacker could use LVI to change the target's speculative control flow by injecting a value for a load. Since the instruction is indivisible, there's no place to put an LFENCE, so like RET these instructions need to be split into load and compare/branch components.

Is that accurate?

If so, we could explain it that way and replace the hardcoded list with hasImplicitDefOfPhysReg(X86::EFLAGS)

[zbrid]

LGTM, but I'm not sure whether or not this should be upstreamed due to Matt's concerns and due to my own lack of what's okay within LLVM for maintainers. Could @jyknight or @george.burgess.iv chime in? @craig.topper, is there an owner to this component who could chime in?

[sconstab]

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

3174

I looked into this and unfortunately the mayLoad attribute is not set for any of the RET family (see https://github.com/llvm/llvm-project/blob/b1d581019f5d4d176ad70ddffee13db247a13ef1/llvm/lib/Target/X86/X86InstrControl.td#L21). Either this is a bug or I don't actually understand what the mayLoad attribute actually means. It seems pretty strange since the POPr family has mayLoad. @craig.topper : opinion?

3193

The manual mitigation is a little bit more involved than what would typically be conveyed in a compiler warning. Would it be acceptable for the warning to point to the LVI deep dive document?

[craig.topper]

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

3174

I suspect it doesn't have the mayLoad flag because from the perspective of the codegen passes that normally use that flag, the load of the return address isn't really something we would care about. It could just as well be reading the return address from some stack dedicated return address stack or a stack implemented in hardware or something.

Not sure what the implications of just adding the flag would be to codegen.

3211

I don't think this works if the rep prefix is on the line before the cmps/scas instruction. That will assemble to the same thing but the parser sees it as two instructions.

[mattdr]

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

3174

Yikes, okay. It's incredibly surprising that mayLoad isn't set for the RET family, and if anything stops working when we set it that's almost certainly a bug we want to know about.

But I can accept that fixing that is out of scope for this change -- at least, for the initial version. If so, though, that makes it all the more important that we describe how we arrived at this particular list of instructions, in enough detail that someone else can retrace our steps.

3193

Sure, a link to an external site seems fine. Can we please link to the specific part of the page that explains the mitigations for this specific set of instructions?

3211

@craig.topper any notion what the best workaround would be? One option is keeping track of whether we've seen a REP or REPNE before this.

Although perhaps a simpler approach is just not worrying about whether we saw REP. We're here to try to warn users about instructions that we can't mitigate. What's the likelihood that we see a CMPSB without REP and it turns out it _doesn't_ need mitigation?

3213

And if we hit the same problem as above with RET, where somehow the instruction tables are inaccurate and we can't rely on them yet, then it's similarly okay with me if we leave the hardcoded list of instructions as long as we provide really specific comments describing why the list is all and only these instructions.

[sconstab]

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

3193

@mattdr Working on it!

3211

@mattdr The difference is that CMPSB without REP can be mitigated simply by following it with an LFENCE. REP CMPSB must be manually decomposed into a loop, into which an LFENCE can be inserted.

I wonder if it would suffice to simply emit a warning/error if a lone REP is encountered to indicate that the user may need to manually mitigate?

3213

Similar to providing a link to a public document in a warning message, I think it should suffice to have a link to said document (and specifically the relevant subsection of that document) in the comment.

[sconstab]

There is now a more complete set of documentation for instructions that must be manually mitigated:

https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection#specialinstructions

• The asm parser now emits a warning containing the above link whenever it encounters an instruction that cannot be automatically mitigated.
• Now emitting a warning if a REP/REPNE prefix is encountered without an accompanying instruction on the same line.

[sconstab]

@craig.topper @mattdr

[sconstab]

Added a CLI option to enable the inline asm hardening feature, which is now disabled by default. There is also a disclaimer in the CLI description that this feature is experimental.

[craig.topper]

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

3163

Move the command line option to the top of the file below the includes.

The two other examples I know of start with "-x86-experimental-" instead of having experimental at the end.

[sconstab]

Addressed comment by @craig.topper about the position of the CLI argument.

[sconstab]

Previous diff had an error.

[mattdr]

Whew. I still think putting this in the X86 assembly parser (so it doesn't round-trip) is crazypants, and I wish we were relying more on instruction metadata rather than hardcoded lists, but if I take those decisions as given then this implementation seems fine.

Thanks for all the new citations and your work to make the intent more obvious.

Take my "Accept" to mean "I can read this code and understand what is going on here". As for the rest, ¯\_(ツ)_/¯

[craig.topper]

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

3191

RET is a pseudo. It will never be parsed.
RETW is missing.
LRETL, LRETW, LRETQ, LRETIL, LRETIW, LRETIQ are missing based on what binutils does.

3200

JMP64m_REX is a special codegen pseudo instruction. It won't be parsed.

3201

binutils does not handle the FARJMP/FARCALL. Is that a miss or them or something else?

[sconstab]

Addressed comments from @craig.topper.

The coverage of the LVI mitigations for LLVM should match the coverage of the mitigations implemented for binutils. Both should adhere to the guidance outlined in this document: https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection.

Mitigations have only been recommended for SGX enclave code. Since SGX enclaves do not support far procedure calls and branches (and therefore far returns), these features do not need to be covered by either mitigation tool.

[sconstab]

Rebase onto master.

[craig.topper]

LGTM

[mattdr]

In D76158#2004182, @sconstab wrote:

The coverage of the LVI mitigations for LLVM should match the coverage of the mitigations implemented for binutils. Both should adhere to the guidance outlined in this document: https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection.

I mean, yes, sure, I want that too, but that doc just says the Intel patches to GNU binutils work by "by inserting an LFENCE instruction after each instruction that performs a load". And the binutils implementation does seem to be trying to make that true?

Mitigations have only been recommended for SGX enclave code. Since SGX enclaves do not support far procedure calls and branches (and therefore far returns), these features do not need to be covered by either mitigation tool.

That reasoning also tracks, but then here is the change where binutils explicitly adds support for LRET: https://github.com/bminor/binutils-gdb/commit/a09f656b267b9a684f038fba7cadfe98e2f18892#diff-c3c1bdcf15ebcd70b899275b3486272bR4594

Anyway, if the controlling factor is "we are only mitigating code that works in SGX and that excludes instructions like <blah>" that at least merits a comment for posterity. Otherwise -- because these are just magic lists of instructions without an explanation of how they were arrived at -- there is no way to tell the difference between an oversight and a purposeful omission, and others will probably independently come to the same conclusions @craig.topper did.

My "accept" remains with previously-described provisos.

[craig.topper]

In D76158#2026160, @mattdr wrote:

In D76158#2004182, @sconstab wrote:

The coverage of the LVI mitigations for LLVM should match the coverage of the mitigations implemented for binutils. Both should adhere to the guidance outlined in this document: https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection.

I mean, yes, sure, I want that too, but that doc just says the Intel patches to GNU binutils work by "by inserting an LFENCE instruction after each instruction that performs a load". And the binutils implementation does seem to be trying to make that true?

Mitigations have only been recommended for SGX enclave code. Since SGX enclaves do not support far procedure calls and branches (and therefore far returns), these features do not need to be covered by either mitigation tool.

That reasoning also tracks, but then here is the change where binutils explicitly adds support for LRET: https://github.com/bminor/binutils-gdb/commit/a09f656b267b9a684f038fba7cadfe98e2f18892#diff-c3c1bdcf15ebcd70b899275b3486272bR4594

Isn't that binutils patch adding IRET (opcode 0xcf)? LRET(opcodes 0xca and 0xcb) were already there.

Anyway, if the controlling factor is "we are only mitigating code that works in SGX and that excludes instructions like <blah>" that at least merits a comment for posterity. Otherwise -- because these are just magic lists of instructions without an explanation of how they were arrived at -- there is no way to tell the difference between an oversight and a purposeful omission, and others will probably independently come to the same conclusions @craig.topper did.

My "accept" remains with previously-described provisos.

[mattdr]

In D76158#2026209, @craig.topper wrote:

Isn't that binutils patch adding IRET (opcode 0xcf)? LRET(opcodes 0xca and 0xcb) were already there.

I admit I don't know the opcodes offhand, but I agree with your reading of the code. From what I can see that commit does seem to be the first time lret appears in comments, and it appears to add a number of lret test cases.

[sconstab]

In D76158#2026316, @mattdr wrote:

In D76158#2026209, @craig.topper wrote:

Isn't that binutils patch adding IRET (opcode 0xcf)? LRET(opcodes 0xca and 0xcb) were already there.

I admit I don't know the opcodes offhand, but I agree with your reading of the code. From what I can see that commit does seem to be the first time lret appears in comments, and it appears to add a number of lret test cases.

I just had a discussion with the author of that commit to binutils. He acknowledged that the inclusion of IRET and LRET was a mistake, and plans to fix it soon.

llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp

 #include "llvm/MC/MCParser/MCAsmParser.h"
 #include "llvm/MC/MCParser/MCParsedAsmOperand.h"
 #include "llvm/MC/MCParser/MCTargetAsmParser.h"
 #include "llvm/MC/MCRegisterInfo.h"
 #include "llvm/MC/MCSection.h"
 #include "llvm/MC/MCStreamer.h"
 #include "llvm/MC/MCSubtargetInfo.h"
 #include "llvm/MC/MCSymbol.h"
+#include "llvm/Support/CommandLine.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>
 #include <memory>
 
+static cl::opt<bool> LVIInlineAsmHardening(
+    "x86-experimental-lvi-inline-asm-hardening",
+    cl::desc("Harden inline assembly code that may be vulnerable to Load Value"
+             " Injection (LVI). This feature is experimental."), cl::Hidden);
+
 using namespace llvm;
 
 static bool checkScale(unsigned Scale, StringRef &ErrMsg) {
   if (Scale != 1 && Scale != 2 && Scale != 4 && Scale != 8) {
     ErrMsg = "scale factor in address must be 1, 2, 4 or 8";
     return true;
   }
   return false;
private:
   bool parseDirectiveSEHSaveXMM(SMLoc);
   bool parseDirectiveSEHPushFrame(SMLoc);
 
   unsigned checkTargetMatchPredicate(MCInst &Inst) override;
 
   bool validateInstruction(MCInst &Inst, const OperandVector &Ops);
   bool processInstruction(MCInst &Inst, const OperandVector &Ops);
 
+  // Load Value Injection (LVI) Mitigations for machine code
+  void emitWarningForSpecialLVIInstruction(SMLoc Loc);
+  bool applyLVICFIMitigation(MCInst &Inst);
+  bool applyLVILoadHardeningMitigation(MCInst &Inst, MCStreamer &Out);
+
   /// Wrapper around MCStreamer::emitInstruction(). Possibly adds
   /// instrumentation around Inst.
   void emitInstruction(MCInst &Inst, OperandVector &Operands, MCStreamer &Out);
 
   bool MatchAndEmitInstruction(SMLoc IDLoc, unsigned &Opcode,
                                OperandVector &Operands, MCStreamer &Out,
                                uint64_t &ErrorInfo,
                                bool MatchingInlineAsm) override;
bool X86AsmParser::validateInstruction(MCInst &Inst, const OperandVector &Ops) {
   }
   }
 
   return false;
 }
 
 static const char *getSubtargetFeatureName(uint64_t Val);
 
+void X86AsmParser::emitWarningForSpecialLVIInstruction(SMLoc Loc) {
+  Warning(Loc, "Instruction may be vulnerable to LVI and "
+               "requires manual mitigation");
+  Note(SMLoc(), "See https://software.intel.com/"
+                "security-software-guidance/insights/"
+                "deep-dive-load-value-injection#specialinstructions"
+                " for more information");
+}
+
+/// RET instructions and also instructions that indirect calls/jumps from memory
+/// combine a load and a branch within a single instruction. To mitigate these
+/// instructions against LVI, they must be decomposed into separate load and
+/// branch instructions, with an LFENCE in between. For more details, see:
+/// - X86LoadValueInjectionRetHardening.cpp
+/// - X86LoadValueInjectionIndirectThunks.cpp
+/// - https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection
+///
+/// Returns `true` if a mitigation was applied or warning was emitted.
+bool X86AsmParser::applyLVICFIMitigation(MCInst &Inst) {
+  // Information on control-flow instructions that require manual mitigation can
+  // be found here:
+  // https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection#specialinstructions
+  switch (Inst.getOpcode()) {
+  case X86::RET:
+  case X86::RETL:
+  case X86::RETQ:
+  case X86::RETIL:
+  case X86::RETIQ:
+  case X86::RETIW:
+  case X86::JMP16m:
+  case X86::JMP32m:
+  case X86::JMP64m:
+  case X86::JMP64m_REX:
+  case X86::FARJMP16m:
+  case X86::FARJMP32m:
+  case X86::FARJMP64:
+  case X86::CALL16m:
+  case X86::CALL32m:
+  case X86::CALL64m:
+  case X86::FARCALL16m:
+  case X86::FARCALL32m:
+  case X86::FARCALL64:
+    emitWarningForSpecialLVIInstruction(Inst.getLoc());
+    return true;
+  }
+  return false;
+}
+
+/// To mitigate LVI, every instruction that performs a load can be followed by
+/// an LFENCE instruction to squash any potential mis-speculation. There are
+/// some instructions that require additional considerations, and may requre
+/// manual mitigation. For more details, see:
+/// https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection
+///
+/// Returns `true` if a mitigation was applied or warning was emitted.
+bool X86AsmParser::applyLVILoadHardeningMitigation(MCInst &Inst,
+                                                   MCStreamer &Out) {
+  auto Opcode = Inst.getOpcode();
+  auto Flags = Inst.getFlags();
+  if ((Flags & X86::IP_HAS_REPEAT) || (Flags & X86::IP_HAS_REPEAT_NE)) {
+    // Information on REP string instructions that require manual mitigation can
+    // be found here:
+    // https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection#specialinstructions
+    switch (Opcode) {
+    case X86::CMPSB:
+    case X86::CMPSW:
+    case X86::CMPSL:
+    case X86::CMPSQ:
+    case X86::SCASB:
+    case X86::SCASW:
+    case X86::SCASL:
+    case X86::SCASQ:
+      emitWarningForSpecialLVIInstruction(Inst.getLoc());
+      return true;
+    }
+  } else if (Opcode == X86::REP_PREFIX || Opcode == X86::REPNE_PREFIX) {
+    // If a REP instruction is found on its own line, it may or may not be
+    // followed by a vulnerable instruction. Emit a warning just in case.
+    emitWarningForSpecialLVIInstruction(Inst.getLoc());
+    return true;
+  }
+
+  const MCInstrDesc &MCID = MII.get(Inst.getOpcode());
+  // LFENCE has the mayLoad property, don't double fence.
+  if (MCID.mayLoad() && Inst.getOpcode() != X86::LFENCE) {
+    MCInst FenceInst;
+    FenceInst.setOpcode(X86::LFENCE);
+    FenceInst.setLoc(Inst.getLoc());
+    Out.emitInstruction(FenceInst, getSTI());
+    return true;
+  }
+  return false;
+}
+
 void X86AsmParser::emitInstruction(MCInst &Inst, OperandVector &Operands,
                                    MCStreamer &Out) {
   Out.emitInstruction(Inst, getSTI());
+
+  if (LVIInlineAsmHardening) {
+    if (getSTI().getFeatureBits()[X86::FeatureLVIControlFlowIntegrity] &&
+        applyLVICFIMitigation(Inst))
+      return;
+    if (getSTI().getFeatureBits()[X86::FeatureLVILoadHardening])
+      applyLVILoadHardeningMitigation(Inst, Out);
+  }
 }
 
 bool X86AsmParser::MatchAndEmitInstruction(SMLoc IDLoc, unsigned &Opcode,
                                            OperandVector &Operands,
                                            MCStreamer &Out, uint64_t &ErrorInfo,
                                            bool MatchingInlineAsm) {
   if (isParsingIntelSyntax())
     return MatchAndEmitIntelInstruction(IDLoc, Opcode, Operands, Out, ErrorInfo,

llvm/test/CodeGen/X86/lvi-hardening-inline-asm.ll

+; RUN: llc -verify-machineinstrs -mtriple=x86_64-unknown -mattr=+lvi-load-hardening -mattr=+lvi-cfi -x86-lvi-inline-asm-hardening-experimental < %s -o %t.out 2> %t.err
+; RUN: FileCheck %s --check-prefix=X86 < %t.out
+; RUN: FileCheck %s --check-prefix=WARN < %t.err
+
+; Test module-level assembly
+module asm "pop %rbx"
+module asm "ret"
+; WARN:      warning: Instruction may be vulnerable to LVI
+; WARN-NEXT: ret
+; WARN-NEXT: ^
+; WARN-NEXT: note: See https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection#specialinstructions for more information
+
+; Function Attrs: noinline nounwind optnone uwtable
+define dso_local void @test_inline_asm() {
+entry:
+; X86-LABEL: test_inline_asm:
+  call void asm sideeffect "mov 0x3fed(%rip),%rax", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      movq  16365(%rip), %rax
+; X86-NEXT: lfence
+  call void asm sideeffect "movdqa 0x0(%rip),%xmm0", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      movdqa  (%rip), %xmm0
+; X86-NEXT: lfence
+  call void asm sideeffect "movslq 0x3e5d(%rip),%rbx", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      movslq  15965(%rip), %rbx
+; X86-NEXT: lfence
+  call void asm sideeffect "mov (%r12,%rax,8),%rax", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      movq  (%r12,%rax,8), %rax
+; X86-NEXT: lfence
+  call void asm sideeffect "movq (24)(%rsi), %r11", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      movq  24(%rsi), %r11
+; X86-NEXT: lfence
+  call void asm sideeffect "cmove %r12,%rax", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      cmoveq  %r12, %rax
+; X86-NOT:  lfence
+  call void asm sideeffect "cmove (%r12),%rax", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      cmoveq  (%r12), %rax
+; X86-NEXT: lfence
+  call void asm sideeffect "pop %rbx", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      popq  %rbx
+; X86-NEXT: lfence
+  call void asm sideeffect "popq %rbx", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      popq  %rbx
+; X86-NEXT: lfence
+  call void asm sideeffect "xchg (%r12),%rax", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      xchgq %rax, (%r12)
+; X86-NEXT: lfence
+  call void asm sideeffect "cmpxchg %r12,(%rax)", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      cmpxchgq  %r12, (%rax)
+; X86-NEXT: lfence
+  call void asm sideeffect "vpxor (%rcx,%rdx,1),%ymm1,%ymm0", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      vpxor (%rcx,%rdx), %ymm1, %ymm0
+; X86-NEXT: lfence
+  call void asm sideeffect "vpmuludq 0x20(%rsi),%ymm0,%ymm12", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      vpmuludq  32(%rsi), %ymm0, %ymm12
+; X86-NEXT: lfence
+  call void asm sideeffect "vpexpandq 0x40(%rdi),%zmm8{%k2}{z}", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      vpexpandq 64(%rdi), %zmm8 {%k2} {z}
+; X86-NEXT: lfence
+  call void asm sideeffect "addq (%r12),%rax", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      addq  (%r12), %rax
+; X86-NEXT: lfence
+  call void asm sideeffect "subq Lpoly+0(%rip), %rax", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      subq  Lpoly+0(%rip), %rax
+; X86-NEXT: lfence
+  call void asm sideeffect "adcq %r12,(%rax)", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      adcq  %r12, (%rax)
+; X86-NEXT: lfence
+  call void asm sideeffect "negq (%rax)", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      negq  (%rax)
+; X86-NEXT: lfence
+  call void asm sideeffect "incq %rax", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      incq  %rax
+; X86-NOT:  lfence
+  call void asm sideeffect "mulq (%rax)", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      mulq  (%rax)
+; X86-NEXT: lfence
+  call void asm sideeffect "imulq (%rax),%rdx", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      imulq (%rax), %rdx
+; X86-NEXT: lfence
+  call void asm sideeffect "shlq $$1,(%rax)", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      shlq  (%rax)
+; X86-NEXT: lfence
+  call void asm sideeffect "shrq $$1,(%rax)", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      shrq  (%rax)
+; X86-NEXT: lfence
+  call void asm sideeffect "repz cmpsb %es:(%rdi),%ds:(%rsi)", "~{dirflag},~{fpsr},~{flags}"() #1
+; WARN:      warning: Instruction may be vulnerable to LVI
+; WARN-NEXT: repz cmpsb %es:(%rdi),%ds:(%rsi)
+; WARN-NEXT: ^
+; WARN-NEXT: note: See https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection#specialinstructions for more information
+; X86:      rep cmpsb %es:(%rdi), %ds:(%rsi)
+; X86-NOT:  lfence
+  call void asm sideeffect "repnz scasb", "~{dirflag},~{fpsr},~{flags}"() #1
+; WARN:      warning: Instruction may be vulnerable to LVI
+; WARN-NEXT: repnz scasb
+; WARN-NEXT: ^
+; WARN-NEXT: note: See https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection#specialinstructions for more information
+; X86:      repne scasb %es:(%rdi), %al
+; X86-NOT:  lfence
+  call void asm sideeffect "repnz", ""() #1
+; WARN:      warning: Instruction may be vulnerable to LVI
+; WARN-NEXT: repnz
+; WARN-NEXT: ^
+; WARN-NEXT: note: See https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection#specialinstructions for more information
+  call void asm sideeffect "pinsrw $$0x6,(%eax),%xmm0", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      pinsrw  $6, (%eax), %xmm0
+; X86-NEXT: lfence
+  call void asm sideeffect "ret", "~{dirflag},~{fpsr},~{flags}"() #1
+; WARN:      warning: Instruction may be vulnerable to LVI
+; WARN-NEXT: ret
+; WARN-NEXT: ^
+; WARN-NEXT: note: See https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection#specialinstructions for more information
+; X86:      retq
+; X86-NOT:  lfence
+  call void asm sideeffect "ret $$8", "~{dirflag},~{fpsr},~{flags}"() #1
+; WARN:      warning: Instruction may be vulnerable to LVI
+; WARN-NEXT: ret $8
+; WARN-NEXT: ^
+; WARN-NEXT: note: See https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection#specialinstructions for more information
+; X86:      retq  $8
+; X86-NOT:  lfence
+  call void asm sideeffect "jmpq *(%rdx)", "~{dirflag},~{fpsr},~{flags}"() #1
+; WARN:      warning: Instruction may be vulnerable to LVI
+; WARN-NEXT: jmpq *(%rdx)
+; WARN-NEXT: ^
+; WARN-NEXT: note: See https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection#specialinstructions for more information
+; X86:      jmpq *(%rdx)
+; X86-NOT:  lfence
+  call void asm sideeffect "jmpq *0x100(%rdx)", "~{dirflag},~{fpsr},~{flags}"() #1
+; WARN:      warning: Instruction may be vulnerable to LVI
+; WARN-NEXT: jmpq *0x100(%rdx)
+; WARN-NEXT: ^
+; WARN-NEXT: note: See https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection#specialinstructions for more information
+; X86:      jmpq *256(%rdx)
+; X86-NOT:  lfence
+  call void asm sideeffect "callq *200(%rdx)", "~{dirflag},~{fpsr},~{flags}"() #1
+; WARN:      warning: Instruction may be vulnerable to LVI
+; WARN-NEXT: callq *200(%rdx)
+; WARN-NEXT: ^
+; WARN-NEXT: note: See https://software.intel.com/security-software-guidance/insights/deep-dive-load-value-injection#specialinstructions for more information
+; X86:      callq *200(%rdx)
+; X86-NOT:  lfence
+  call void asm sideeffect "fldt 0x8(%rbp)", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      fldt  8(%rbp)
+; X86-NEXT: lfence
+  call void asm sideeffect "fld %st(0)", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      fld %st(0)
+; X86-NOT:  lfence
+; Test assembler macros
+  call void asm sideeffect ".macro mplus1 x\0Aincq (\5Cx)\0A.endm\0Amplus1 %rcx", "~{dirflag},~{fpsr},~{flags}"() #1
+; X86:      incq (%rcx)
+; X86-NEXT: lfence
+  ret void
+}
+
+attributes #1 = { nounwind }