[asan-asm-instrumentation] Prologue and epilogue are moved out from InstrumentMemOperand().

[ygorshenin]

Summary

[asan-asm-instrumentation] Prologue and epilogue are moved out from InstrumentMemOperand().

• Prologue and epilogue are movedo ut from InstrumentMemOperand();
• Instrumentation for rep movs doesn't save/restore working register set between consecutive instrumentations for bounds of memory ranges;
• Fixed a bug that on x86_64 incorrect address was passed to __asan_report functions;

[ygorshenin]

Rebase.

[ygorshenin]

Rebase.

[ygorshenin]

PTAL

[eugenis]

lib/Target/X86/AsmParser/X86AsmInstrumentation.cpp

70

ExpectedShadowValueReg - if I understand this correctly, this is simply an available scratch register. Please find a better name.

73

AsanContext seems unnecessary and it mixes information about a memory access (which actually changes between prologue and epilogue in case of MOVS) and the instrumentation state (i.e. the spilled scratch register). Also, there can be no more than 1 outstanding AsanContext at any time, and it can be merged with X86AddressSanitizer class.

I'd rather

• pass accesssize and iswrite to InstrumentMemOperand as before.
• pass the scratch register to InstrumentMemOperandPrologue and save it right in X86AddressSanitizer (smth like a set of extra spilled registers that must be restored in the epilogue).

165

Too many InstrumentMemOperand* methods. Better repeat this prologue/epilogue calls in InstrumentMOV.

[ygorshenin]

Comments are addressed.

[ygorshenin]

PTAL

lib/Target/X86/AsmParser/X86AsmInstrumentation.cpp

70

Renamed to ScratchReg.

73

AccessSize and IsWrite are passed directly as arguments, but I'd prefer to keep scratch register in the register context.

165

Done.

[ygorshenin]

Removed target_link_libraries from CMakeLists.txt.

[eugenis]

LGTM w/ nit

lib/Target/X86/AsmParser/X86AsmInstrumentation.cpp

552

Is it possible to remove accesssize argument here and in prologue, and instead pass an invalid/zero value for the scratch register?
And check in instrumentmemoperandsmall that a scratch register is available, for sanity.

[ygorshenin]

Fixed RegisterContext ctor.

[ygorshenin]

Added checks in InstrumentMemOperandSmall().

[ygorshenin]

Many thanks!

lib/Target/X86/AsmParser/X86AsmInstrumentation.cpp

552

Done.

[ygorshenin]

r216869.

[chapuni]

Excuse me to comment the older commit.

This brought new dependency AsmParser on CodeGen. For example, llvm-mc requires LLVMX86CodeGen, despite other LLVM*CodGen(s) are not required.

Any better idea to dissolve such a dependency?

• Instantiate RegInfo also in X86AsmInstrumentation.cpp.
• Move X86RegisterInfo to another module
• Introduce a new module, X86RegisterInfo.

[eugenis]

Introduce a new module, X86RegisterInfo.

This sounds OK to me.

lib/Target/X86/AsmParser/CMakeLists.txt

   X86AsmInstrumentation.cpp
   X86AsmParser.cpp
   )
+
+target_link_libraries(LLVMX86AsmParser LLVMX86CodeGen)

lib/Target/X86/AsmParser/X86AsmInstrumentation.h

   // Tries to instrument and emit instruction.
   virtual void InstrumentAndEmitInstruction(
       const MCInst &Inst,
-      SmallVectorImpl<std::unique_ptr<MCParsedAsmOperand>> &Operands,
+      SmallVectorImpl<std::unique_ptr<MCParsedAsmOperand> > &Operands,
       MCContext &Ctx, const MCInstrInfo &MII, MCStreamer &Out);
 
 protected:

lib/Target/X86/AsmParser/X86AsmInstrumentation.cpp

 #include "MCTargetDesc/X86BaseInfo.h"
 #include "X86AsmInstrumentation.h"
 #include "X86Operand.h"
+#include "X86RegisterInfo.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/ADT/Triple.h"
+#include "llvm/CodeGen/MachineValueType.h"
 #include "llvm/IR/Function.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCInst.h"

   return Reg == X86::RSP || Reg == X86::ESP || Reg == X86::SP;
 }
 
+bool IsSmallMemAccess(unsigned AccessSize) { return AccessSize < 8; }
+
 std::string FuncName(unsigned AccessSize, bool IsWrite) {
   return std::string("__asan_report_") + (IsWrite ? "store" : "load") +
          utostr(AccessSize);

 
 class X86AddressSanitizer : public X86AsmInstrumentation {
 public:
+  struct RegisterContext {
+    RegisterContext()
+        : AddressReg(X86::RDI), ShadowReg(X86::RAX), ScratchReg(X86::RCX) {}
+
+    unsigned addressReg(MVT::SimpleValueType VT) const {
+      return getX86SubSuperRegister(AddressReg, VT);
+    }
+
+    unsigned shadowReg(MVT::SimpleValueType VT) const {
+      return getX86SubSuperRegister(ShadowReg, VT);
+    }
+
+    unsigned scratchReg(MVT::SimpleValueType VT) const {
+      return getX86SubSuperRegister(ScratchReg, VT);
+    }
+
+    unsigned AddressReg;
+    unsigned ShadowReg;
+    unsigned ScratchReg;
+  };
+
   X86AddressSanitizer(const MCSubtargetInfo &STI)
       : X86AsmInstrumentation(STI), RepPrefix(false) {}
   virtual ~X86AddressSanitizer() {}

   }
 
   // Should be implemented differently in x86_32 and x86_64 subclasses.
-  virtual void InstrumentMemOperandSmallImpl(X86Operand &Op,
-                                             unsigned AccessSize, bool IsWrite,
-                                             MCContext &Ctx,
-                                             MCStreamer &Out) = 0;
-  virtual void InstrumentMemOperandLargeImpl(X86Operand &Op,
-                                             unsigned AccessSize, bool IsWrite,
-                                             MCContext &Ctx,
-                                             MCStreamer &Out) = 0;
+  virtual void StoreFlags(MCStreamer &Out) = 0;
+
+  virtual void RestoreFlags(MCStreamer &Out) = 0;
+
+  // Adjusts up stack and saves all registers used in instrumentation.
+  virtual void InstrumentMemOperandPrologue(unsigned AccessSize,
+                                            const RegisterContext &RegCtx,
+                                            MCContext &Ctx,
+                                            MCStreamer &Out) = 0;
+
+  // Restores all registers used in instrumentation and adjusts stack.
+  virtual void InstrumentMemOperandEpilogue(unsigned AccessSize,
+                                            const RegisterContext &RegCtx,
+                                            MCContext &Ctx,
+                                            MCStreamer &Out) = 0;
+
+  virtual void InstrumentMemOperandSmall(X86Operand &Op, unsigned AccessSize,
+                                         bool IsWrite,
+                                         const RegisterContext &RegCtx,
+                                         MCContext &Ctx, MCStreamer &Out) = 0;
+  virtual void InstrumentMemOperandLarge(X86Operand &Op, unsigned AccessSize,
+                                         bool IsWrite,
+                                         const RegisterContext &RegCtx,
+                                         MCContext &Ctx, MCStreamer &Out) = 0;
+
   virtual void InstrumentMOVSImpl(unsigned AccessSize, MCContext &Ctx,
                                   MCStreamer &Out) = 0;
 
-  void InstrumentMemOperand(MCParsedAsmOperand &Op, unsigned AccessSize,
-                            bool IsWrite, MCContext &Ctx, MCStreamer &Out);
+  void InstrumentMemOperand(X86Operand &Op, unsigned AccessSize, bool IsWrite,
+                            const RegisterContext &RegCtx, MCContext &Ctx,
+                            MCStreamer &Out);
   void InstrumentMOVSBase(unsigned DstReg, unsigned SrcReg, unsigned CntReg,
                           unsigned AccessSize, MCContext &Ctx, MCStreamer &Out);
+
   void InstrumentMOVS(const MCInst &Inst, OperandVector &Operands,
                       MCContext &Ctx, const MCInstrInfo &MII, MCStreamer &Out);
   void InstrumentMOV(const MCInst &Inst, OperandVector &Operands,
                      MCContext &Ctx, const MCInstrInfo &MII, MCStreamer &Out);
 
+protected:
   void EmitLabel(MCStreamer &Out, MCSymbol *Label) { Out.EmitLabel(Label); }
 
-protected:
   // True when previous instruction was actually REP prefix.
   bool RepPrefix;
 };
 
-void X86AddressSanitizer::InstrumentMemOperand(MCParsedAsmOperand &Op,
-                                               unsigned AccessSize,
-                                               bool IsWrite, MCContext &Ctx,
-                                               MCStreamer &Out) {
+void X86AddressSanitizer::InstrumentMemOperand(
+    X86Operand &Op, unsigned AccessSize, bool IsWrite,
+    const RegisterContext &RegCtx, MCContext &Ctx, MCStreamer &Out) {
   assert(Op.isMem() && "Op should be a memory operand.");
   assert((AccessSize & (AccessSize - 1)) == 0 && AccessSize <= 16 &&
          "AccessSize should be a power of two, less or equal than 16.");
-
-  X86Operand &MemOp = static_cast<X86Operand &>(Op);
-  // FIXME: get rid of this limitation.
-  if (IsStackReg(MemOp.getMemBaseReg()) || IsStackReg(MemOp.getMemIndexReg()))
-    return;
-
   // FIXME: take into account load/store alignment.
-  if (AccessSize < 8)
-    InstrumentMemOperandSmallImpl(MemOp, AccessSize, IsWrite, Ctx, Out);
+  if (IsSmallMemAccess(AccessSize))
+    InstrumentMemOperandSmall(Op, AccessSize, IsWrite, RegCtx, Ctx, Out);
   else
-    InstrumentMemOperandLargeImpl(MemOp, AccessSize, IsWrite, Ctx, Out);
+    InstrumentMemOperandLarge(Op, AccessSize, IsWrite, RegCtx, Ctx, Out);
 }
 
 void X86AddressSanitizer::InstrumentMOVSBase(unsigned DstReg, unsigned SrcReg,

                                              MCContext &Ctx, MCStreamer &Out) {
   // FIXME: check whole ranges [DstReg .. DstReg + AccessSize * (CntReg - 1)]
   // and [SrcReg .. SrcReg + AccessSize * (CntReg - 1)].
+  RegisterContext RegCtx;
+  RegCtx.AddressReg = X86::RDX;
+  RegCtx.ShadowReg = X86::RAX;
+  RegCtx.ScratchReg = X86::RBX;
 
-  // FIXME: extract prolog and epilogue from InstrumentMemOperand()
-  // and optimize this sequence of InstrumentMemOperand() calls.
+  InstrumentMemOperandPrologue(AccessSize, RegCtx, Ctx, Out);
 
   // Test (%SrcReg)
   {
     const MCExpr *Disp = MCConstantExpr::Create(0, Ctx);
     std::unique_ptr<X86Operand> Op(X86Operand::CreateMem(
         0, Disp, SrcReg, 0, AccessSize, SMLoc(), SMLoc()));
-    InstrumentMemOperand(*Op, AccessSize, false /* IsWrite */, Ctx, Out);
+    InstrumentMemOperand(*Op, AccessSize, false /* IsWrite */, RegCtx, Ctx,
+                         Out);
   }
 
   // Test -1(%SrcReg, %CntReg, AccessSize)

     const MCExpr *Disp = MCConstantExpr::Create(-1, Ctx);
     std::unique_ptr<X86Operand> Op(X86Operand::CreateMem(
         0, Disp, SrcReg, CntReg, AccessSize, SMLoc(), SMLoc()));
-    InstrumentMemOperand(*Op, AccessSize, false /* IsWrite */, Ctx, Out);
+    InstrumentMemOperand(*Op, AccessSize, false /* IsWrite */, RegCtx, Ctx,
+                         Out);
   }
 
   // Test (%DstReg)

     const MCExpr *Disp = MCConstantExpr::Create(0, Ctx);
     std::unique_ptr<X86Operand> Op(X86Operand::CreateMem(
         0, Disp, DstReg, 0, AccessSize, SMLoc(), SMLoc()));
-    InstrumentMemOperand(*Op, AccessSize, true /* IsWrite */, Ctx, Out);
+    InstrumentMemOperand(*Op, AccessSize, true /* IsWrite */, RegCtx, Ctx, Out);
   }
 
   // Test -1(%DstReg, %CntReg, AccessSize)

     const MCExpr *Disp = MCConstantExpr::Create(-1, Ctx);
     std::unique_ptr<X86Operand> Op(X86Operand::CreateMem(
         0, Disp, DstReg, CntReg, AccessSize, SMLoc(), SMLoc()));
-    InstrumentMemOperand(*Op, AccessSize, true /* IsWrite */, Ctx, Out);
+    InstrumentMemOperand(*Op, AccessSize, true /* IsWrite */, RegCtx, Ctx, Out);
   }
+
+  InstrumentMemOperandEpilogue(AccessSize, RegCtx, Ctx, Out);
 }
 
 void X86AddressSanitizer::InstrumentMOVS(const MCInst &Inst,

   }
 
   const bool IsWrite = MII.get(Inst.getOpcode()).mayStore();
+  const RegisterContext RegCtx;
+
   for (unsigned Ix = 0; Ix < Operands.size(); ++Ix) {
     assert(Operands[Ix]);
     MCParsedAsmOperand &Op = *Operands[Ix];
-    if (Op.isMem())
-      InstrumentMemOperand(Op, AccessSize, IsWrite, Ctx, Out);
+    if (Op.isMem()) {
+      X86Operand &MemOp = static_cast<X86Operand &>(Op);
+      // FIXME: get rid of this limitation.
+      if (IsStackReg(MemOp.getMemBaseReg()) ||
+          IsStackReg(MemOp.getMemIndexReg())) {
+        continue;
+      }
+
+      InstrumentMemOperandPrologue(AccessSize, RegCtx, Ctx, Out);
+      InstrumentMemOperand(MemOp, AccessSize, IsWrite, RegCtx, Ctx, Out);
+      InstrumentMemOperandEpilogue(AccessSize, RegCtx, Ctx, Out);
+    }
   }
 }
 

       : X86AddressSanitizer(STI) {}
   virtual ~X86AddressSanitizer32() {}
 
-  virtual void InstrumentMemOperandSmallImpl(X86Operand &Op,
-                                             unsigned AccessSize, bool IsWrite,
-                                             MCContext &Ctx,
-                                             MCStreamer &Out) override;
-  virtual void InstrumentMemOperandLargeImpl(X86Operand &Op,
-                                             unsigned AccessSize, bool IsWrite,
-                                             MCContext &Ctx,
-                                             MCStreamer &Out) override;
+  virtual void StoreFlags(MCStreamer &Out) override {
+    EmitInstruction(Out, MCInstBuilder(X86::PUSHF32));
+  }
+
+  virtual void RestoreFlags(MCStreamer &Out) override {
+    EmitInstruction(Out, MCInstBuilder(X86::POPF32));
+  }
+
+  virtual void InstrumentMemOperandPrologue(unsigned AccessSize,
+                                            const RegisterContext &RegCtx,
+                                            MCContext &Ctx,
+                                            MCStreamer &Out) override {
+    EmitInstruction(
+        Out, MCInstBuilder(X86::PUSH32r).addReg(RegCtx.addressReg(MVT::i32)));
+    EmitInstruction(
+        Out, MCInstBuilder(X86::PUSH32r).addReg(RegCtx.shadowReg(MVT::i32)));
+    if (IsSmallMemAccess(AccessSize)) {
+      EmitInstruction(
+          Out, MCInstBuilder(X86::PUSH32r).addReg(RegCtx.scratchReg(MVT::i32)));
+    }
+    StoreFlags(Out);
+  }
+
+  virtual void InstrumentMemOperandEpilogue(unsigned AccessSize,
+                                            const RegisterContext &RegCtx,
+                                            MCContext &Ctx,
+                                            MCStreamer &Out) override {
+    RestoreFlags(Out);
+    if (IsSmallMemAccess(AccessSize)) {
+      EmitInstruction(
+          Out, MCInstBuilder(X86::POP32r).addReg(RegCtx.scratchReg(MVT::i32)));
+    }
+    EmitInstruction(
+        Out, MCInstBuilder(X86::POP32r).addReg(RegCtx.shadowReg(MVT::i32)));
+    EmitInstruction(
+        Out, MCInstBuilder(X86::POP32r).addReg(RegCtx.addressReg(MVT::i32)));
+  }
+
+  virtual void InstrumentMemOperandSmall(X86Operand &Op, unsigned AccessSize,
+                                         bool IsWrite,
+                                         const RegisterContext &RegCtx,
+                                         MCContext &Ctx,
+                                         MCStreamer &Out) override;
+  virtual void InstrumentMemOperandLarge(X86Operand &Op, unsigned AccessSize,
+                                         bool IsWrite,
+                                         const RegisterContext &RegCtx,
+                                         MCContext &Ctx,
+                                         MCStreamer &Out) override;
   virtual void InstrumentMOVSImpl(unsigned AccessSize, MCContext &Ctx,
                                   MCStreamer &Out) override;
 
 private:
-  void EmitCallAsanReport(MCContext &Ctx, MCStreamer &Out, unsigned AccessSize,
-                          bool IsWrite, unsigned AddressReg) {
+  void EmitCallAsanReport(unsigned AccessSize, bool IsWrite, MCContext &Ctx,
+                          MCStreamer &Out, const RegisterContext &RegCtx) {
     EmitInstruction(Out, MCInstBuilder(X86::CLD));
     EmitInstruction(Out, MCInstBuilder(X86::MMX_EMMS));
 

                              .addReg(X86::ESP)
                              .addReg(X86::ESP)
                              .addImm(-16));
-    EmitInstruction(Out, MCInstBuilder(X86::PUSH32r).addReg(AddressReg));
+    EmitInstruction(
+        Out, MCInstBuilder(X86::PUSH32r).addReg(RegCtx.addressReg(MVT::i32)));
 
     const std::string &Fn = FuncName(AccessSize, IsWrite);
     MCSymbol *FnSym = Ctx.GetOrCreateSymbol(StringRef(Fn));

   }
 };
 
-void X86AddressSanitizer32::InstrumentMemOperandSmallImpl(X86Operand &Op,
-                                                          unsigned AccessSize,
-                                                          bool IsWrite,
-                                                          MCContext &Ctx,
-                                                          MCStreamer &Out) {
-  EmitInstruction(Out, MCInstBuilder(X86::PUSH32r).addReg(X86::EAX));
-  EmitInstruction(Out, MCInstBuilder(X86::PUSH32r).addReg(X86::ECX));
-  EmitInstruction(Out, MCInstBuilder(X86::PUSH32r).addReg(X86::EDX));
-  EmitInstruction(Out, MCInstBuilder(X86::PUSHF32));
+void X86AddressSanitizer32::InstrumentMemOperandSmall(
+    X86Operand &Op, unsigned AccessSize, bool IsWrite,
+    const RegisterContext &RegCtx, MCContext &Ctx, MCStreamer &Out) {
+  unsigned AddressRegI32 = RegCtx.addressReg(MVT::i32);
+  unsigned ShadowRegI32 = RegCtx.shadowReg(MVT::i32);
+  unsigned ShadowRegI8 = RegCtx.shadowReg(MVT::i8);
+  unsigned ScratchRegI32 = RegCtx.scratchReg(MVT::i32);
 
   {
     MCInst Inst;
     Inst.setOpcode(X86::LEA32r);
-    Inst.addOperand(MCOperand::CreateReg(X86::EAX));
+    Inst.addOperand(MCOperand::CreateReg(AddressRegI32));
     Op.addMemOperands(Inst, 5);
     EmitInstruction(Out, Inst);
   }
 
-  EmitInstruction(
-      Out, MCInstBuilder(X86::MOV32rr).addReg(X86::ECX).addReg(X86::EAX));
-  EmitInstruction(
-      Out,
-      MCInstBuilder(X86::SHR32ri).addReg(X86::ECX).addReg(X86::ECX).addImm(3));
+  EmitInstruction(Out, MCInstBuilder(X86::MOV32rr).addReg(ShadowRegI32).addReg(
+                           AddressRegI32));
+  EmitInstruction(Out, MCInstBuilder(X86::SHR32ri)
+                           .addReg(ShadowRegI32)
+                           .addReg(ShadowRegI32)
+                           .addImm(3));
 
   {
     MCInst Inst;
     Inst.setOpcode(X86::MOV8rm);
-    Inst.addOperand(MCOperand::CreateReg(X86::CL));
+    Inst.addOperand(MCOperand::CreateReg(ShadowRegI8));
     const MCExpr *Disp = MCConstantExpr::Create(kShadowOffset, Ctx);
     std::unique_ptr<X86Operand> Op(
-        X86Operand::CreateMem(0, Disp, X86::ECX, 0, 1, SMLoc(), SMLoc()));
+        X86Operand::CreateMem(0, Disp, ShadowRegI32, 0, 1, SMLoc(), SMLoc()));
     Op->addMemOperands(Inst, 5);
     EmitInstruction(Out, Inst);
   }
 
-  EmitInstruction(Out,
-                  MCInstBuilder(X86::TEST8rr).addReg(X86::CL).addReg(X86::CL));
+  EmitInstruction(
+      Out, MCInstBuilder(X86::TEST8rr).addReg(ShadowRegI8).addReg(ShadowRegI8));
   MCSymbol *DoneSym = Ctx.CreateTempSymbol();
   const MCExpr *DoneExpr = MCSymbolRefExpr::Create(DoneSym, Ctx);
   EmitInstruction(Out, MCInstBuilder(X86::JE_4).addExpr(DoneExpr));
 
-  EmitInstruction(
-      Out, MCInstBuilder(X86::MOV32rr).addReg(X86::EDX).addReg(X86::EAX));
-  EmitInstruction(
-      Out,
-      MCInstBuilder(X86::AND32ri).addReg(X86::EDX).addReg(X86::EDX).addImm(7));
+  EmitInstruction(Out, MCInstBuilder(X86::MOV32rr).addReg(ScratchRegI32).addReg(
+                           AddressRegI32));
+  EmitInstruction(Out, MCInstBuilder(X86::AND32ri)
+                           .addReg(ScratchRegI32)
+                           .addReg(ScratchRegI32)
+                           .addImm(7));
 
   switch (AccessSize) {
   case 1:

   case 2: {
     MCInst Inst;
     Inst.setOpcode(X86::LEA32r);
-    Inst.addOperand(MCOperand::CreateReg(X86::EDX));
+    Inst.addOperand(MCOperand::CreateReg(ScratchRegI32));
 
     const MCExpr *Disp = MCConstantExpr::Create(1, Ctx);
     std::unique_ptr<X86Operand> Op(
-        X86Operand::CreateMem(0, Disp, X86::EDX, 0, 1, SMLoc(), SMLoc()));
+        X86Operand::CreateMem(0, Disp, ScratchRegI32, 0, 1, SMLoc(), SMLoc()));
     Op->addMemOperands(Inst, 5);
     EmitInstruction(Out, Inst);
     break;
   }
   case 4:
     EmitInstruction(Out, MCInstBuilder(X86::ADD32ri8)
-                             .addReg(X86::EDX)
-                             .addReg(X86::EDX)
+                             .addReg(ScratchRegI32)
+                             .addReg(ScratchRegI32)
                              .addImm(3));
     break;
   default:

   }
 
   EmitInstruction(
-      Out, MCInstBuilder(X86::MOVSX32rr8).addReg(X86::ECX).addReg(X86::CL));
-  EmitInstruction(
-      Out, MCInstBuilder(X86::CMP32rr).addReg(X86::EDX).addReg(X86::ECX));
+      Out,
+      MCInstBuilder(X86::MOVSX32rr8).addReg(ShadowRegI32).addReg(ShadowRegI8));
+  EmitInstruction(Out, MCInstBuilder(X86::CMP32rr).addReg(ScratchRegI32).addReg(
+                           ShadowRegI32));
   EmitInstruction(Out, MCInstBuilder(X86::JL_4).addExpr(DoneExpr));
 
-  EmitCallAsanReport(Ctx, Out, AccessSize, IsWrite, X86::EAX);
+  EmitCallAsanReport(AccessSize, IsWrite, Ctx, Out, RegCtx);
   EmitLabel(Out, DoneSym);
-
-  EmitInstruction(Out, MCInstBuilder(X86::POPF32));
-  EmitInstruction(Out, MCInstBuilder(X86::POP32r).addReg(X86::EDX));
-  EmitInstruction(Out, MCInstBuilder(X86::POP32r).addReg(X86::ECX));
-  EmitInstruction(Out, MCInstBuilder(X86::POP32r).addReg(X86::EAX));
 }
 
-void X86AddressSanitizer32::InstrumentMemOperandLargeImpl(X86Operand &Op,
-                                                          unsigned AccessSize,
-                                                          bool IsWrite,
-                                                          MCContext &Ctx,
-                                                          MCStreamer &Out) {
-  EmitInstruction(Out, MCInstBuilder(X86::PUSH32r).addReg(X86::EAX));
-  EmitInstruction(Out, MCInstBuilder(X86::PUSH32r).addReg(X86::ECX));
-  EmitInstruction(Out, MCInstBuilder(X86::PUSHF32));
+void X86AddressSanitizer32::InstrumentMemOperandLarge(
+    X86Operand &Op, unsigned AccessSize, bool IsWrite,
+    const RegisterContext &RegCtx, MCContext &Ctx, MCStreamer &Out) {
+  unsigned AddressRegI32 = RegCtx.addressReg(MVT::i32);
+  unsigned ShadowRegI32 = RegCtx.shadowReg(MVT::i32);
 
   {
     MCInst Inst;
     Inst.setOpcode(X86::LEA32r);
-    Inst.addOperand(MCOperand::CreateReg(X86::EAX));
+    Inst.addOperand(MCOperand::CreateReg(AddressRegI32));
     Op.addMemOperands(Inst, 5);
     EmitInstruction(Out, Inst);
   }
-  EmitInstruction(
-      Out, MCInstBuilder(X86::MOV32rr).addReg(X86::ECX).addReg(X86::EAX));
-  EmitInstruction(
-      Out,
-      MCInstBuilder(X86::SHR32ri).addReg(X86::ECX).addReg(X86::ECX).addImm(3));
+
+  EmitInstruction(Out, MCInstBuilder(X86::MOV32rr).addReg(ShadowRegI32).addReg(
+                           AddressRegI32));
+  EmitInstruction(Out, MCInstBuilder(X86::SHR32ri)
+                           .addReg(ShadowRegI32)
+                           .addReg(ShadowRegI32)
+                           .addImm(3));
   {
     MCInst Inst;
     switch (AccessSize) {

     }
     const MCExpr *Disp = MCConstantExpr::Create(kShadowOffset, Ctx);
     std::unique_ptr<X86Operand> Op(
-        X86Operand::CreateMem(0, Disp, X86::ECX, 0, 1, SMLoc(), SMLoc()));
+        X86Operand::CreateMem(0, Disp, ShadowRegI32, 0, 1, SMLoc(), SMLoc()));
     Op->addMemOperands(Inst, 5);
     Inst.addOperand(MCOperand::CreateImm(0));
     EmitInstruction(Out, Inst);

   const MCExpr *DoneExpr = MCSymbolRefExpr::Create(DoneSym, Ctx);
   EmitInstruction(Out, MCInstBuilder(X86::JE_4).addExpr(DoneExpr));
 
-  EmitCallAsanReport(Ctx, Out, AccessSize, IsWrite, X86::EAX);
+  EmitCallAsanReport(AccessSize, IsWrite, Ctx, Out, RegCtx);
   EmitLabel(Out, DoneSym);
-
-  EmitInstruction(Out, MCInstBuilder(X86::POPF32));
-  EmitInstruction(Out, MCInstBuilder(X86::POP32r).addReg(X86::ECX));
-  EmitInstruction(Out, MCInstBuilder(X86::POP32r).addReg(X86::EAX));
 }
 
 void X86AddressSanitizer32::InstrumentMOVSImpl(unsigned AccessSize,
                                                MCContext &Ctx,
                                                MCStreamer &Out) {
-  EmitInstruction(Out, MCInstBuilder(X86::PUSHF32));
+  StoreFlags(Out);
 
   // No need to test when ECX is equals to zero.
   MCSymbol *DoneSym = Ctx.CreateTempSymbol();

                      X86::ECX /* CntReg */, AccessSize, Ctx, Out);
 
   EmitLabel(Out, DoneSym);
-  EmitInstruction(Out, MCInstBuilder(X86::POPF32));
+  RestoreFlags(Out);
 }
 
 class X86AddressSanitizer64 : public X86AddressSanitizer {

       : X86AddressSanitizer(STI) {}
   virtual ~X86AddressSanitizer64() {}
 
-  virtual void InstrumentMemOperandSmallImpl(X86Operand &Op,
-                                             unsigned AccessSize, bool IsWrite,
-                                             MCContext &Ctx,
-                                             MCStreamer &Out) override;
-  virtual void InstrumentMemOperandLargeImpl(X86Operand &Op,
-                                             unsigned AccessSize, bool IsWrite,
-                                             MCContext &Ctx,
-                                             MCStreamer &Out) override;
+  virtual void StoreFlags(MCStreamer &Out) override {
+    EmitInstruction(Out, MCInstBuilder(X86::PUSHF64));
+  }
+
+  virtual void RestoreFlags(MCStreamer &Out) override {
+    EmitInstruction(Out, MCInstBuilder(X86::POPF64));
+  }
+
+  virtual void InstrumentMemOperandPrologue(unsigned AccessSize,
+                                            const RegisterContext &RegCtx,
+                                            MCContext &Ctx,
+                                            MCStreamer &Out) override {
+    EmitAdjustRSP(Ctx, Out, -128);
+    EmitInstruction(
+        Out, MCInstBuilder(X86::PUSH64r).addReg(RegCtx.shadowReg(MVT::i64)));
+    EmitInstruction(
+        Out, MCInstBuilder(X86::PUSH64r).addReg(RegCtx.addressReg(MVT::i64)));
+    if (IsSmallMemAccess(AccessSize)) {
+      EmitInstruction(
+          Out, MCInstBuilder(X86::PUSH64r).addReg(RegCtx.scratchReg(MVT::i64)));
+    }
+    StoreFlags(Out);
+  }
+
+  virtual void InstrumentMemOperandEpilogue(unsigned AccessSize,
+                                            const RegisterContext &RegCtx,
+                                            MCContext &Ctx,
+                                            MCStreamer &Out) override {
+    RestoreFlags(Out);
+    if (IsSmallMemAccess(AccessSize)) {
+      EmitInstruction(
+          Out, MCInstBuilder(X86::POP64r).addReg(RegCtx.scratchReg(MVT::i64)));
+    }
+    EmitInstruction(
+        Out, MCInstBuilder(X86::POP64r).addReg(RegCtx.addressReg(MVT::i64)));
+    EmitInstruction(
+        Out, MCInstBuilder(X86::POP64r).addReg(RegCtx.shadowReg(MVT::i64)));
+    EmitAdjustRSP(Ctx, Out, 128);
+  }
+
+  virtual void InstrumentMemOperandSmall(X86Operand &Op, unsigned AccessSize,
+                                         bool IsWrite,
+                                         const RegisterContext &RegCtx,
+                                         MCContext &Ctx,
+                                         MCStreamer &Out) override;
+  virtual void InstrumentMemOperandLarge(X86Operand &Op, unsigned AccessSize,
+                                         bool IsWrite,
+                                         const RegisterContext &RegCtx,
+                                         MCContext &Ctx,
+                                         MCStreamer &Out) override;
   virtual void InstrumentMOVSImpl(unsigned AccessSize, MCContext &Ctx,
                                   MCStreamer &Out) override;
 

     EmitInstruction(Out, Inst);
   }
 
-  void EmitCallAsanReport(MCContext &Ctx, MCStreamer &Out, unsigned AccessSize,
-                          bool IsWrite) {
+  void EmitCallAsanReport(unsigned AccessSize, bool IsWrite, MCContext &Ctx,
+                          MCStreamer &Out, const RegisterContext &RegCtx) {
     EmitInstruction(Out, MCInstBuilder(X86::CLD));
     EmitInstruction(Out, MCInstBuilder(X86::MMX_EMMS));
 

                              .addReg(X86::RSP)
                              .addImm(-16));
 
+    if (RegCtx.AddressReg != X86::RDI) {
+      EmitInstruction(Out, MCInstBuilder(X86::MOV64rr).addReg(X86::RDI).addReg(
+                               RegCtx.addressReg(MVT::i64)));
+    }
     const std::string &Fn = FuncName(AccessSize, IsWrite);
     MCSymbol *FnSym = Ctx.GetOrCreateSymbol(StringRef(Fn));
     const MCSymbolRefExpr *FnExpr =

   }
 };
 
-void X86AddressSanitizer64::InstrumentMemOperandSmallImpl(X86Operand &Op,
-                                                          unsigned AccessSize,
-                                                          bool IsWrite,
-                                                          MCContext &Ctx,
-                                                          MCStreamer &Out) {
-  EmitAdjustRSP(Ctx, Out, -128);
-  EmitInstruction(Out, MCInstBuilder(X86::PUSH64r).addReg(X86::RAX));
-  EmitInstruction(Out, MCInstBuilder(X86::PUSH64r).addReg(X86::RCX));
-  EmitInstruction(Out, MCInstBuilder(X86::PUSH64r).addReg(X86::RDI));
-  EmitInstruction(Out, MCInstBuilder(X86::PUSHF64));
+void X86AddressSanitizer64::InstrumentMemOperandSmall(
+    X86Operand &Op, unsigned AccessSize, bool IsWrite,
+    const RegisterContext &RegCtx, MCContext &Ctx, MCStreamer &Out) {
+  unsigned AddressRegI64 = RegCtx.addressReg(MVT::i64);
+  unsigned AddressRegI32 = RegCtx.addressReg(MVT::i32);
+  unsigned ShadowRegI64 = RegCtx.shadowReg(MVT::i64);
+  unsigned ShadowRegI32 = RegCtx.shadowReg(MVT::i32);
+  unsigned ShadowRegI8 = RegCtx.shadowReg(MVT::i8);
+  unsigned ScratchRegI32 = RegCtx.scratchReg(MVT::i32);
+
   {
     MCInst Inst;
     Inst.setOpcode(X86::LEA64r);
-    Inst.addOperand(MCOperand::CreateReg(X86::RDI));
+    Inst.addOperand(MCOperand::CreateReg(AddressRegI64));
     Op.addMemOperands(Inst, 5);
     EmitInstruction(Out, Inst);
   }
-  EmitInstruction(
-      Out, MCInstBuilder(X86::MOV64rr).addReg(X86::RAX).addReg(X86::RDI));
-  EmitInstruction(
-      Out,
-      MCInstBuilder(X86::SHR64ri).addReg(X86::RAX).addReg(X86::RAX).addImm(3));
+  EmitInstruction(Out, MCInstBuilder(X86::MOV64rr).addReg(ShadowRegI64).addReg(
+                           AddressRegI64));
+  EmitInstruction(Out, MCInstBuilder(X86::SHR64ri)
+                           .addReg(ShadowRegI64)
+                           .addReg(ShadowRegI64)
+                           .addImm(3));
   {
     MCInst Inst;
     Inst.setOpcode(X86::MOV8rm);
-    Inst.addOperand(MCOperand::CreateReg(X86::AL));
+    Inst.addOperand(MCOperand::CreateReg(ShadowRegI8));
     const MCExpr *Disp = MCConstantExpr::Create(kShadowOffset, Ctx);
     std::unique_ptr<X86Operand> Op(
-        X86Operand::CreateMem(0, Disp, X86::RAX, 0, 1, SMLoc(), SMLoc()));
+        X86Operand::CreateMem(0, Disp, ShadowRegI64, 0, 1, SMLoc(), SMLoc()));
     Op->addMemOperands(Inst, 5);
     EmitInstruction(Out, Inst);
   }
 
-  EmitInstruction(Out,
-                  MCInstBuilder(X86::TEST8rr).addReg(X86::AL).addReg(X86::AL));
+  EmitInstruction(
+      Out, MCInstBuilder(X86::TEST8rr).addReg(ShadowRegI8).addReg(ShadowRegI8));
   MCSymbol *DoneSym = Ctx.CreateTempSymbol();
   const MCExpr *DoneExpr = MCSymbolRefExpr::Create(DoneSym, Ctx);
   EmitInstruction(Out, MCInstBuilder(X86::JE_4).addExpr(DoneExpr));
 
-  EmitInstruction(
-      Out, MCInstBuilder(X86::MOV32rr).addReg(X86::ECX).addReg(X86::EDI));
-  EmitInstruction(
-      Out,
-      MCInstBuilder(X86::AND32ri).addReg(X86::ECX).addReg(X86::ECX).addImm(7));
+  EmitInstruction(Out, MCInstBuilder(X86::MOV32rr).addReg(ScratchRegI32).addReg(
+                           AddressRegI32));
+  EmitInstruction(Out, MCInstBuilder(X86::AND32ri)
+                           .addReg(ScratchRegI32)
+                           .addReg(ScratchRegI32)
+                           .addImm(7));
 
   switch (AccessSize) {
   case 1:

   case 2: {
     MCInst Inst;
     Inst.setOpcode(X86::LEA32r);
-    Inst.addOperand(MCOperand::CreateReg(X86::ECX));
+    Inst.addOperand(MCOperand::CreateReg(ScratchRegI32));
 
     const MCExpr *Disp = MCConstantExpr::Create(1, Ctx);
     std::unique_ptr<X86Operand> Op(
-        X86Operand::CreateMem(0, Disp, X86::ECX, 0, 1, SMLoc(), SMLoc()));
+        X86Operand::CreateMem(0, Disp, ScratchRegI32, 0, 1, SMLoc(), SMLoc()));
     Op->addMemOperands(Inst, 5);
     EmitInstruction(Out, Inst);
     break;
   }
   case 4:
     EmitInstruction(Out, MCInstBuilder(X86::ADD32ri8)
-                             .addReg(X86::ECX)
-                             .addReg(X86::ECX)
+                             .addReg(ScratchRegI32)
+                             .addReg(ScratchRegI32)
                              .addImm(3));
     break;
   default:

   }
 
   EmitInstruction(
-      Out, MCInstBuilder(X86::MOVSX32rr8).addReg(X86::EAX).addReg(X86::AL));
-  EmitInstruction(
-      Out, MCInstBuilder(X86::CMP32rr).addReg(X86::ECX).addReg(X86::EAX));
+      Out,
+      MCInstBuilder(X86::MOVSX32rr8).addReg(ShadowRegI32).addReg(ShadowRegI8));
+  EmitInstruction(Out, MCInstBuilder(X86::CMP32rr).addReg(ScratchRegI32).addReg(
+                           ShadowRegI32));
   EmitInstruction(Out, MCInstBuilder(X86::JL_4).addExpr(DoneExpr));
 
-  EmitCallAsanReport(Ctx, Out, AccessSize, IsWrite);
+  EmitCallAsanReport(AccessSize, IsWrite, Ctx, Out, RegCtx);
   EmitLabel(Out, DoneSym);
-
-  EmitInstruction(Out, MCInstBuilder(X86::POPF64));
-  EmitInstruction(Out, MCInstBuilder(X86::POP64r).addReg(X86::RDI));
-  EmitInstruction(Out, MCInstBuilder(X86::POP64r).addReg(X86::RCX));
-  EmitInstruction(Out, MCInstBuilder(X86::POP64r).addReg(X86::RAX));
-  EmitAdjustRSP(Ctx, Out, 128);
 }
 
-void X86AddressSanitizer64::InstrumentMemOperandLargeImpl(X86Operand &Op,
-                                                          unsigned AccessSize,
-                                                          bool IsWrite,
-                                                          MCContext &Ctx,
-                                                          MCStreamer &Out) {
-  EmitAdjustRSP(Ctx, Out, -128);
-  EmitInstruction(Out, MCInstBuilder(X86::PUSH64r).addReg(X86::RAX));
-  EmitInstruction(Out, MCInstBuilder(X86::PUSHF64));
+void X86AddressSanitizer64::InstrumentMemOperandLarge(
+    X86Operand &Op, unsigned AccessSize, bool IsWrite,
+    const RegisterContext &RegCtx, MCContext &Ctx, MCStreamer &Out) {
+  unsigned AddressRegI64 = RegCtx.addressReg(MVT::i64);
+  unsigned ShadowRegI64 = RegCtx.shadowReg(MVT::i64);
 
   {
     MCInst Inst;
     Inst.setOpcode(X86::LEA64r);
-    Inst.addOperand(MCOperand::CreateReg(X86::RAX));
+    Inst.addOperand(MCOperand::CreateReg(AddressRegI64));
     Op.addMemOperands(Inst, 5);
     EmitInstruction(Out, Inst);
   }
-  EmitInstruction(
-      Out,
-      MCInstBuilder(X86::SHR64ri).addReg(X86::RAX).addReg(X86::RAX).addImm(3));
+  EmitInstruction(Out, MCInstBuilder(X86::MOV64rr).addReg(ShadowRegI64).addReg(
+                           AddressRegI64));
+  EmitInstruction(Out, MCInstBuilder(X86::SHR64ri)
+                           .addReg(ShadowRegI64)
+                           .addReg(ShadowRegI64)
+                           .addImm(3));
   {
     MCInst Inst;
     switch (AccessSize) {

     }
     const MCExpr *Disp = MCConstantExpr::Create(kShadowOffset, Ctx);
     std::unique_ptr<X86Operand> Op(
-        X86Operand::CreateMem(0, Disp, X86::RAX, 0, 1, SMLoc(), SMLoc()));
+        X86Operand::CreateMem(0, Disp, ShadowRegI64, 0, 1, SMLoc(), SMLoc()));
     Op->addMemOperands(Inst, 5);
     Inst.addOperand(MCOperand::CreateImm(0));
     EmitInstruction(Out, Inst);

   const MCExpr *DoneExpr = MCSymbolRefExpr::Create(DoneSym, Ctx);
   EmitInstruction(Out, MCInstBuilder(X86::JE_4).addExpr(DoneExpr));
 
-  EmitCallAsanReport(Ctx, Out, AccessSize, IsWrite);
+  EmitCallAsanReport(AccessSize, IsWrite, Ctx, Out, RegCtx);
   EmitLabel(Out, DoneSym);
-
-  EmitInstruction(Out, MCInstBuilder(X86::POPF64));
-  EmitInstruction(Out, MCInstBuilder(X86::POP64r).addReg(X86::RAX));
-  EmitAdjustRSP(Ctx, Out, 128);
 }
 
 void X86AddressSanitizer64::InstrumentMOVSImpl(unsigned AccessSize,
                                                MCContext &Ctx,
                                                MCStreamer &Out) {
-  EmitInstruction(Out, MCInstBuilder(X86::PUSHF64));
+  StoreFlags(Out);
 
   // No need to test when RCX is equals to zero.
   MCSymbol *DoneSym = Ctx.CreateTempSymbol();

                      X86::RCX /* CntReg */, AccessSize, Ctx, Out);
 
   EmitLabel(Out, DoneSym);
-  EmitInstruction(Out, MCInstBuilder(X86::POPF64));
+  RestoreFlags(Out);
 }
 
 } // End anonymous namespace

test/Instrumentation/AddressSanitizer/X86/asm_mov.ll

 ; CHECK-LABEL: mov1b
 ; CHECK: leaq -128(%rsp), %rsp
 ; CHECK-NEXT: pushq %rax
-; CHECK-NEXT: pushq %rcx
 ; CHECK-NEXT: pushq %rdi
+; CHECK-NEXT: pushq %rcx
 ; CHECK-NEXT: pushfq
 ; CHECK-NEXT: leaq {{.*}}, %rdi
 ; CHECK-NEXT: movq %rdi, %rax

 ; CHECK-NEXT: callq __asan_report_load1@PLT
 ; CHECK-NEXT: [[A]]:
 ; CHECK-NEXT: popfq
-; CHECK-NEXT: popq %rdi
 ; CHECK-NEXT: popq %rcx
+; CHECK-NEXT: popq %rdi
 ; CHECK-NEXT: popq %rax
 ; CHECK-NEXT: leaq 128(%rsp), %rsp
 

 ; CHECK-LABEL: mov8b
 ; CHECK: leaq -128(%rsp), %rsp
 ; CHECK-NEXT: pushq %rax
+; CHECK-NEXT: pushq %rdi
 ; CHECK-NEXT: pushfq
-; CHECK-NEXT: leaq {{.*}}, %rax
+; CHECK-NEXT: leaq {{.*}}, %rdi
+; CHECK-NEXT: movq %rdi, %rax
 ; CHECK-NEXT: shrq $3, %rax
 ; CHECK-NEXT: cmpb $0, 2147450880(%rax)
 ; CHECK-NEXT: je [[A:.*]]

 ; CHECK-NEXT: callq __asan_report_load8@PLT
 ; CHECK-NEXT: [[A]]:
 ; CHECK-NEXT: popfq
+; CHECK-NEXT: popq %rdi
 ; CHECK-NEXT: popq %rax
 ; CHECK-NEXT: leaq 128(%rsp), %rsp
 
 ; CHECK: leaq -128(%rsp), %rsp
 ; CHECK-NEXT: pushq %rax
+; CHECK-NEXT: pushq %rdi
 ; CHECK-NEXT: pushfq
-; CHECK-NEXT: leaq {{.*}}, %rax
+; CHECK-NEXT: leaq {{.*}}, %rdi
+; CHECK-NEXT: movq %rdi, %rax
 ; CHECK-NEXT: shrq $3, %rax
 ; CHECK-NEXT: cmpb $0, 2147450880(%rax)
 ; CHECK-NEXT: je [[A:.*]]

 ; CHECK-NEXT: callq __asan_report_store8@PLT
 ; CHECK-NEXT: [[A]]:
 ; CHECK-NEXT: popfq
+; CHECK-NEXT: popq %rdi
 ; CHECK-NEXT: popq %rax
 ; CHECK-NEXT: leaq 128(%rsp), %rsp
 

test/Instrumentation/AddressSanitizer/X86/asm_rep_movs.ll

 ; CHECK-NEXT: testq %rcx, %rcx
 ; CHECK-NEXT: je [[B:.*]]
 
-; CHECK: leaq (%rsi), {{.*}}
-; CHECK: callq __asan_report_load1@PLT
-
-; CHECK: leaq -1(%rsi,%rcx), {{.*}}
-; CHECK: callq __asan_report_load1@PLT
-
-; CHECK: leaq (%rdi), {{.*}}
-; CHECK: callq __asan_report_store1@PLT
-
-; CHECK: leaq -1(%rdi,%rcx), {{.*}}
-; CHECK: callq __asan_report_store1@PLT
+; CHECK: leaq -128(%rsp), %rsp
+; CHECK-NEXT: pushq %rax
+; CHECK-NEXT: pushq %rdx
+; CHECK-NEXT: pushq %rbx
+; CHECK-NEXT: pushfq
+
+; CHECK: leaq (%rsi), %rdx
+; CHECK: movq %rdx, %rdi
+; CHECK-NEXT: callq __asan_report_load1@PLT
+
+; CHECK: leaq -1(%rsi,%rcx), %rdx
+; CHECK: movq %rdx, %rdi
+; CHECK-NEXT: callq __asan_report_load1@PLT
+
+; CHECK: leaq (%rdi), %rdx
+; CHECK: movq %rdx, %rdi
+; CHECK-NEXT: callq __asan_report_store1@PLT
+
+; CHECK: leaq -1(%rdi,%rcx), %rdx
+; CHECK: movq %rdx, %rdi
+; CHECK-NEXT: callq __asan_report_store1@PLT
+
+; CHECK: popfq
+; CHECK-NEXT: popq %rbx
+; CHECK-NEXT: popq %rdx
+; CHECK-NEXT: popq %rax
+; CHECK-NEXT: leaq 128(%rsp), %rsp
 
 ; CHECK: [[B]]:
 ; CHECK-NEXT: popfq

 ; CHECK-NEXT: testq %rcx, %rcx
 ; CHECK-NEXT: je [[Q:.*]]
 
-; CHECK: leaq (%rsi), {{.*}}
-; CHECK: callq __asan_report_load8@PLT
+; CHECK: leaq (%rsi), %rdx
+; CHECK: movq %rdx, %rdi
+; CHECK-NEXT: callq __asan_report_load8@PLT
 
-; CHECK: leaq -1(%rsi,%rcx,8), {{.*}}
-; CHECK: callq __asan_report_load8@PLT
+; CHECK: leaq -1(%rsi,%rcx,8), %rdx
+; CHECK: movq %rdx, %rdi
+; CHECK-NEXT: callq __asan_report_load8@PLT
 
-; CHECK: leaq (%rdi), {{.*}}
-; CHECK: callq __asan_report_store8@PLT
+; CHECK: leaq (%rdi), %rdx
+; CHECK: movq %rdx, %rdi
+; CHECK-NEXT: callq __asan_report_store8@PLT
 
-; CHECK: leaq -1(%rdi,%rcx,8), {{.*}}
-; CHECK: callq __asan_report_store8@PLT
+; CHECK: leaq -1(%rdi,%rcx,8), %rdx
+; CHECK: movq %rdx, %rdi
+; CHECK-NEXT: callq __asan_report_store8@PLT
 
 ; CHECK: [[Q]]:
 ; CHECK-NEXT: popfq