[X86] Create extra prolog/epilog for stack realignment

[LuoYuanke]

Summary

The base pointer register is reserved by compiler when there is
dynamic size alloca and stack realign in a function. However the
base pointer register is not defined in X86 ABI, so user can use
this register in inline assembly. The inline assembly would
clobber base pointer register without being awared by user. This
patch is to create extra prolog to save the stack pointer to a
scratch register and use this register to reference argument from
stack. For some calling convention (e.g. regcall), there may be
few scratch register.
Below is the example code for such case.

extern int bar(void *p);
long long foo(size_t size, char c, int id) {
  __attribute__((__aligned__(64))) int a;
  char *p = (char *)alloca(size);
  asm volatile ("nop"::"S"(405):);
  asm volatile ("movl %0, %1"::"r"(id), "m"(a):);
  p[2] = 8;
  memset(p, c, size);
  return bar(p);
}

And below prolog/epilog will be emit for this case.

leal    4(%esp), %ebx
.cfi_def_cfa %ebx, 0
andl    $-128, %esp
pushl   -4(%ebx)
...
leal    4(%ebx), %esp
.cfi_def_cfa %esp, 4

[LuoYuanke]

Add unwind attibute to test cfi instruction emit.

[LuoYuanke]

llvm/lib/Target/X86/X86ArgumentStackSlotRebase.cpp

80

About 60 lit test fails and need update if we create a new register class for it.

[pengfei]

llvm/lib/Target/X86/X86ArgumentStackSlotRebase.cpp

2

Update name

77

Are we replacing RBX with RBX on 64-bits?

80

Why R10? Do you want R11, according to ABI

%r10 temporary register, used for passing a function’s static chain pointer No
%r11 temporary register                                                     No

llvm/lib/Target/X86/X86FrameLowering.cpp

590–591

Remove

llvm/lib/Target/X86/X86RegisterInfo.td

436–439

Should remove it?

llvm/test/CodeGen/X86/swifttail-realign.ll

9 ↗

(On Diff #503625)

No sure if it's correct. I assume the old code used rsp to access %ptr but it is intentionally omitted.
It must be wrong if the new code still use rsp because it is not aligned to 32 now.

[LuoYuanke]

llvm/lib/Target/X86/X86ArgumentStackSlotRebase.cpp

77

Good catch, we may have to need virtual register since inline assembly may clobber any physical register.

80

Both R10 and R11 should be OK, but we should use virtual register.

llvm/test/CodeGen/X86/swifttail-realign.ll

9 ↗

(On Diff #503625)

There is alignment instruction andq $-32, %rax in both the old code and new code.

movq    %rsp, %rax
addq    $15, %rdi
andq    $-16, %rdi
subq    %rdi, %rax
andq    $-32, %rax
movq    %rax, %rsp
movq    var@GOTPCREL(%rip), %rcx
movq    %rax, (%rcx)

[LuoYuanke]

Address some of Phoebe's comments. Need another patch
to create new register class.

[LuoYuanke]

Address Phoebe's comments.
Use virtual register for argument stack slot rebase.

[LuoYuanke]

Update test cases.

[LuoYuanke]

Update test cases.

[LuoYuanke]

1. Use scratch register for argument base pointer.
2. Fix CFI instruction
3. Enhance replacement of frame pointer with argument base pointer.
4. c++ test case (https://reviews.llvm.org/D145926) can pass with -g.

[LuoYuanke]

llvm/test/CodeGen/MIR/X86/inline-asm-registers.mir

31–32

GR64 register class ID is changed from 67 to 68. Since register class is encoded in the flag, this test case need to be updated.

[LuoYuanke]

The patch is ready for review.

[LuoYuanke]

llvm/test/CodeGen/X86/i386-baseptr.ll

51

The argument base register is saved to a normal stack slot.

71

The argument base register is restored from a normal stack

[LuoYuanke]

llvm/test/CodeGen/X86/i386-baseptr.ll

50–53

Stack is aligned with 128 byte. (244 + 4 (ret addr) + 4 (ebp) +4 (esi) = 256)

[RKSimon]

llvm/lib/Target/X86/CMakeLists.txt

93

(style) keep this sorted if you can

[LuoYuanke]

Address Simon's comments.

[pengfei]

We had an offline discussion on this. I generally got the idea of the design. Let's land it first to see if there're unconsidered cases.

llvm/lib/Target/X86/X86ArgumentStackSlotRebase.cpp

97

Should assert rather than break?

146

Should assert rather than return?

llvm/lib/Target/X86/X86FrameLowering.cpp

4044

This can be auto.

4050

Use explicit type.

llvm/lib/Target/X86/X86RegisterInfo.cpp

819–820

The format looks odd, maybe clang-format?

llvm/lib/Target/X86/X86RegisterInfo.td

437

last .

443

ditto.

llvm/test/CodeGen/X86/i386-baseptr.ll

50

Nit, it seems a bit waste in space.

llvm/test/CodeGen/X86/swifttail-realign.ll

9 ↗

(On Diff #503625)

That looks good, thanks!

[LuoYuanke]

Address Phoebe's comments.

llvm/lib/Target/X86/X86ArgumentStackSlotRebase.cpp

97

If we use break it can return NoReg, so that nothing would be done in this pass and the extra prolog/epilog won't be inserted.

146

My original idea is to leave it as it is if we can't handle it.

llvm/test/CodeGen/X86/i386-baseptr.ll

50

Yes, but it is original code logic that is to align the stack address to 128.

[nikic]

It looks like this broke the expensive checks build (x86-64-baseptr.ll and i386-baseptr.ll fail).

[LuoYuanke]

In D145650#4209068, @nikic wrote:

It looks like this broke the expensive checks build (x86-64-baseptr.ll and i386-baseptr.ll fail).

I'll check it with "-DLLVM_ENABLE_EXPENSIVE_CHECKS=ON". Thanks for notifying.

[LuoYuanke]

There are live range issue for the physical register. Let me revert the patch first.

[rupprecht]

This relanded in e4ceb5a7bb9b8f6d730530345649286370dd3ff8, and we're seeing runtime errors in Halide when building with asan.

I don't have a standalone reproducer I can share yet, but the issue we see is when attempting to return from this method: https://github.com/halide/Halide/blob/main/src/runtime/x86_cpu_features.cpp. If I had to guess, it's a bad interaction with the cpuid implementation here: https://github.com/halide/Halide/blob/2a51f71a0add536ab986c675d1239db860dbc655/src/runtime/x86.ll#L145

The problem is $rsp is not pointing to the right place, and so ret jumps to a bogus location, so the next instruction faults:

... break on start address of x86_cpu_features ...
(lldb) x/2 $rsp
0x7fffffffb818: 0x5806dc35 0x00005555
(lldb) c
...
(lldb) di -l

repro_test`Halide::Runtime::Internal::halide_get_cpu_features:
    0x55555806e23e <+1246>: movq   -0x48(%rbp), %r10
    0x55555806e242 <+1250>: leaq   -0x28(%rbp), %rsp
    0x55555806e246 <+1254>: popq   %rbx
    0x55555806e247 <+1255>: popq   %r12
    0x55555806e249 <+1257>: popq   %r13
    0x55555806e24b <+1259>: popq   %r14
    0x55555806e24d <+1261>: popq   %r15
    0x55555806e24f <+1263>: popq   %rbp
    0x55555806e250 <+1264>: leaq   -0x10(%r10), %rsp
->  0x55555806e254 <+1268>: retq
(lldb) x/2 $rsp
0x7fffffffb810: 0xffffb890 0x00007fff
(lldb) x/2 0x7fffffffb818
0x7fffffffb818: 0x5806dc35 0x00005555

[LuoYuanke]

repro_test`Halide::Runtime::Internal::halide_get_cpu_features:

0x55555806e23e <+1246>: movq   -0x48(%rbp), %r10
0x55555806e242 <+1250>: leaq   -0x28(%rbp), %rsp
0x55555806e246 <+1254>: popq   %rbx
0x55555806e247 <+1255>: popq   %r12
0x55555806e249 <+1257>: popq   %r13
0x55555806e24b <+1259>: popq   %r14
0x55555806e24d <+1261>: popq   %r15
0x55555806e24f <+1263>: popq   %rbp
0x55555806e250 <+1264>: leaq   -0x10(%r10), %rsp

-> 0x55555806e254 <+1268>: retq
(lldb) x/2 $rsp
0x7fffffffb810: 0xffffb890 0x00007fff
(lldb) x/2 0x7fffffffb818
0x7fffffffb818: 0x5806dc35 0x00005555

Thank you very much for the indicator. It seems the offset is incorrect for leaq -0x10(%r10), %rsp. It should be leaq -0x8(%r10), %rsp. Sorry for inconvenience. I'll fix it as soon as possible and I'll develop more runtime test case for it.

[LuoYuanke]

@rupprecht, could you help to check if https://reviews.llvm.org/D146862 can fix your issue?

llvm/lib/Target/X86/CMakeLists.txt

 
 if (X86_GEN_FOLD_TABLES)
   tablegen(LLVM X86GenFoldTables.inc -gen-x86-fold-tables -asmwriternum=1)
 endif()
 
 add_public_tablegen_target(X86CommonTableGen)
 
 set(sources
+  X86ArgumentStackSlotRebase.cpp
   X86AsmPrinter.cpp
   X86AvoidTrailingCall.cpp
   X86CallFrameOptimization.cpp
   X86CallingConv.cpp
   X86CallLowering.cpp
   X86CmovConversion.cpp
   X86DomainReassignment.cpp
   X86DiscriminateMemOps.cpp
set(sources
   X86SpeculativeExecutionSideEffectSuppression.cpp
   X86Subtarget.cpp
   X86TargetMachine.cpp
   X86TargetObjectFile.cpp
   X86TargetTransformInfo.cpp
   X86VZeroUpper.cpp
   X86WinEHState.cpp
   X86InsertWait.cpp
   )
 
 add_llvm_target(X86CodeGen ${sources}
   LINK_COMPONENTS
   Analysis
   AsmPrinter
   CodeGen
   Core
   Instrumentation

llvm/lib/Target/X86/X86.h

 InstructionSelector *createX86InstructionSelector(const X86TargetMachine &TM,
                                                   X86Subtarget &,
                                                   X86RegisterBankInfo &);
 
 FunctionPass *createX86LoadValueInjectionLoadHardeningPass();
 FunctionPass *createX86LoadValueInjectionRetHardeningPass();
 FunctionPass *createX86SpeculativeLoadHardeningPass();
 FunctionPass *createX86SpeculativeExecutionSideEffectSuppression();
+FunctionPass *createX86ArgumentStackSlotPass();
 
 void initializeEvexToVexInstPassPass(PassRegistry &);
 void initializeFPSPass(PassRegistry &);
 void initializeFixupBWInstPassPass(PassRegistry &);
 void initializeFixupLEAPassPass(PassRegistry &);
+void initializeX86ArgumentStackSlotPassPass(PassRegistry &);
 void initializeX86FixupInstTuningPassPass(PassRegistry &);
 void initializeWinEHStatePassPass(PassRegistry &);
 void initializeX86AvoidSFBPassPass(PassRegistry &);
 void initializeX86AvoidTrailingCallPassPass(PassRegistry &);
 void initializeX86CallFrameOptimizationPass(PassRegistry &);
 void initializeX86CmovConverterPassPass(PassRegistry &);
 void initializeX86DAGToDAGISelPass(PassRegistry &);
 void initializeX86DomainReassignmentPass(PassRegistry &);

llvm/lib/Target/X86/X86ArgumentStackSlotRebase.cpp

+//===---- X86ArgumentStackSlotRebase.cpp - rebase argument stack slot -----===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This pass replace the frame register with a GPR virtual register and set
+// the stack offset for each instruction which reference argument from stack.
+//
+//===----------------------------------------------------------------------===//
+
+#include "X86.h"
+#include "X86InstrBuilder.h"
+#include "X86MachineFunctionInfo.h"
+#include "X86RegisterInfo.h"
+#include "X86Subtarget.h"
+#include "llvm/CodeGen/MachineBasicBlock.h"
+#include "llvm/CodeGen/MachineFrameInfo.h"
+#include "llvm/CodeGen/MachineFunction.h"
+#include "llvm/CodeGen/MachineFunctionPass.h"
+#include "llvm/CodeGen/MachineInstr.h"
+#include "llvm/CodeGen/MachineOperand.h"
+#include "llvm/CodeGen/MachineRegisterInfo.h"
+#include "llvm/CodeGen/TargetOpcodes.h"
+#include "llvm/CodeGen/TargetRegisterInfo.h"
+#include "llvm/CodeGen/TargetSubtargetInfo.h"
+#include "llvm/IR/Attributes.h"
+#include "llvm/IR/Function.h"
+#include "llvm/InitializePasses.h"
+#include "llvm/Pass.h"
+
+using namespace llvm;
+
+#define DEBUG_TYPE "x86argumentstackrebase"
+
+namespace {
+
+class X86ArgumentStackSlotPass : public MachineFunctionPass {
+
+public:
+  static char ID; // Pass identification, replacement for typeid
+
+  explicit X86ArgumentStackSlotPass() : MachineFunctionPass(ID) {
+    initializeX86ArgumentStackSlotPassPass(*PassRegistry::getPassRegistry());
+  }
+
+  bool runOnMachineFunction(MachineFunction &MF) override;
+
+  void getAnalysisUsage(AnalysisUsage &AU) const override {
+    AU.setPreservesCFG();
+    MachineFunctionPass::getAnalysisUsage(AU);
+  }
+};
+
+} // end anonymous namespace
+
+char X86ArgumentStackSlotPass::ID = 0;
+
+INITIALIZE_PASS(X86ArgumentStackSlotPass, DEBUG_TYPE, "Argument Stack Rebase",
+                false, false)
+
+FunctionPass *llvm::createX86ArgumentStackSlotPass() {
+  return new X86ArgumentStackSlotPass();
+}
+
+static Register getArgBaseReg(MachineFunction &MF) {
+  MachineRegisterInfo &MRI = MF.getRegInfo();
+  const X86Subtarget &STI = MF.getSubtarget<X86Subtarget>();
+  const Function &F = MF.getFunction();
+  CallingConv::ID CC = F.getCallingConv();
+  Register NoReg;
+  const TargetRegisterClass *RC = nullptr;
+  switch (CC) {
+  // We need a virtual register in case there is inline assembly
+  // clobber argument base register.
+  case CallingConv::C:
+    RC = STI.is64Bit() ? &X86::GR64_ArgRefRegClass : &X86::GR32_ArgRefRegClass;
+    break;
+  case CallingConv::X86_RegCall:
+    // FIXME: For regcall there is no scratch register on 32-bit target.
+    // We may use a callee saved register as argument base register and
+    // save it before being changed as base pointer. We need DW_CFA to
+    // indicate where the callee saved register is saved, so that it can
+    // be correctly unwind.
+    // push      ebx
+    // mov       ebx, esp
+    // and       esp, -128
+    // ...
+    // pop       ebx
+    // ret
+    RC = STI.is64Bit() ? &X86::GR64_ArgRefRegClass : nullptr;
+    break;
+  // TODO: Refine register class for each calling convention.
+  default:
+    break;
+  }
+  if (RC)
+    return MRI.createVirtualRegister(RC);
+  else
+    return NoReg;
+}
+
+bool X86ArgumentStackSlotPass::runOnMachineFunction(MachineFunction &MF) {
+  const Function &F = MF.getFunction();
+  MachineFrameInfo &MFI = MF.getFrameInfo();
+  const X86Subtarget &STI = MF.getSubtarget<X86Subtarget>();
+  const X86RegisterInfo *TRI = STI.getRegisterInfo();
+  const X86InstrInfo *TII = STI.getInstrInfo();
+  X86MachineFunctionInfo *X86FI = MF.getInfo<X86MachineFunctionInfo>();
+  bool Changed = false;
+
+  if (F.hasFnAttribute(Attribute::Naked))
+    return false;
+  // Only support Linux
+  if (!STI.isTargetLinux())
+    return false;
+  if (!TRI->hasBasePointer(MF))
+    return false;
+
+  Register BasePtr = TRI->getBaseRegister();
+  auto IsBaseRegisterClobbered = [&]() {
+    for (MachineBasicBlock &MBB : MF) {
+      for (MachineInstr &MI : MBB) {
+        if (!MI.isInlineAsm())
+          continue;
+        for (MachineOperand &MO : MI.operands()) {
+          if (!MO.isReg())
+            continue;
+          Register Reg = MO.getReg();
+          if (!Register::isPhysicalRegister(Reg))
+            continue;
+          if (TRI->isSuperOrSubRegisterEq(BasePtr, Reg))
+            return true;
+        }
+      }
+    }
+    return false;
+  };
+  if (!IsBaseRegisterClobbered())
+    return false;
+
+  Register ArgBaseReg = getArgBaseReg(MF);
+  if (!ArgBaseReg.isValid())
+    return false;
+  // leal    4(%esp), %reg
+  // FIXME: will the instruction be duplicated or eliminated? Should
+  // define a pseudo instruction for it?
+  MachineBasicBlock &MBB = MF.front();
+  MachineBasicBlock::iterator MBBI = MBB.begin();
+  DebugLoc DL;
+  // Emit instruction to copy get stack pointer to a virtual register
+  // and save the instruction to x86 machine functon info. We can get
+  // physical register of ArgBaseReg after register allocation. The
+  // stack slot is used to save/restore argument base pointer. We can
+  // get the index from the instruction.
+  unsigned SlotSize = TRI->getSlotSize();
+  int FI = MFI.CreateSpillStackObject(SlotSize, Align(SlotSize));
+  MachineInstr *LEA =
+      BuildMI(MBB, MBBI, DL,
+              TII->get(STI.is64Bit() ? X86::LEA64r : X86::LEA32r), ArgBaseReg)
+          .addFrameIndex(FI)
+          .addImm(1)
+          .addUse(X86::NoRegister)
+          .addImm(SlotSize)
+          .addUse(X86::NoRegister)
+          .setMIFlag(MachineInstr::FrameSetup);
+  X86FI->setStackPtrSaveMI(LEA);
+
+  for (MachineBasicBlock &MBB : MF) {
+    for (MachineInstr &MI : MBB) {
+      int I = 0;
+      for (MachineOperand &MO : MI.operands()) {
+        if (MO.isFI()) {
+          int Idx = MO.getIndex();
+          if (!MFI.isFixedObjectIndex(Idx))
+            continue;
+          int64_t Offset = MFI.getObjectOffset(Idx);
+          if (Offset < 0)
+            continue;
+          // TODO replace register for debug instruction
+          if (MI.isDebugInstr())
+            continue;
+          // Replace frame register with argument base pointer and its offset.
+          TRI->eliminateFrameIndex(MI.getIterator(), I, ArgBaseReg, Offset);
+          Changed = true;
+        }
+        ++I;
+      }
+    }
+  }
+
+  return Changed;
+}

llvm/lib/Target/X86/X86FrameLowering.cpp

 #include "llvm/CodeGen/WinEHFuncInfo.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/EHPersonalities.h"
 #include "llvm/IR/Function.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCObjectFileInfo.h"
 #include "llvm/MC/MCSymbol.h"
 #include "llvm/Support/Debug.h"
+#include "llvm/Support/LEB128.h"
 #include "llvm/Target/TargetOptions.h"
 #include <cstdlib>
 
 #define DEBUG_TYPE "x86-fl"
 
 STATISTIC(NumFrameLoopProbe, "Number of loop stack probes used in prologue");
 STATISTIC(NumFrameExtraProbe,
           "Number of extra stack probes generated in prologue");

 
 void X86FrameLowering::emitCalleeSavedFrameMoves(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI,
     const DebugLoc &DL, bool IsPrologue) const {
   MachineFunction &MF = *MBB.getParent();
   MachineFrameInfo &MFI = MF.getFrameInfo();
   MachineModuleInfo &MMI = MF.getMMI();
   const MCRegisterInfo *MRI = MMI.getContext().getRegisterInfo();
+  X86MachineFunctionInfo *X86FI = MF.getInfo<X86MachineFunctionInfo>();
 
   // Add callee saved registers to move list.
   const std::vector<CalleeSavedInfo> &CSI = MFI.getCalleeSavedInfo();
 
   // Calculate offsets.
   for (const CalleeSavedInfo &I : CSI) {
     int64_t Offset = MFI.getObjectOffset(I.getFrameIdx());
     Register Reg = I.getReg();
     unsigned DwarfReg = MRI->getDwarfRegNum(Reg, true);
 
     if (IsPrologue) {
+      if (X86FI->getStackPtrSaveMI()) {
+        // +2*SlotSize because there is return address and ebp at the bottom
+        // of the stack.
+        // | retaddr |
+        // | ebp     |
+        // |         |<--ebp
+        Offset += 2 * SlotSize;
+        SmallString<64> CfaExpr;
+        CfaExpr.push_back(dwarf::DW_CFA_expression);
+        uint8_t buffer[16];
+        CfaExpr.append(buffer, buffer + encodeULEB128(DwarfReg, buffer));
+        CfaExpr.push_back(2);
+        Register FramePtr = TRI->getFrameRegister(MF);
+        const Register MachineFramePtr =
+            STI.isTarget64BitILP32()
+                ? Register(getX86SubSuperRegister(FramePtr, 64))
+                : FramePtr;
+        unsigned DwarfFramePtr = MRI->getDwarfRegNum(MachineFramePtr, true);
+        CfaExpr.push_back((uint8_t)(dwarf::DW_OP_breg0 + DwarfFramePtr));
+        CfaExpr.append(buffer, buffer + encodeSLEB128(Offset, buffer));
+        BuildCFI(MBB, MBBI, DL,
+                 MCCFIInstruction::createEscape(nullptr, CfaExpr.str()),
+                 MachineInstr::FrameSetup);
+      } else {
         BuildCFI(MBB, MBBI, DL,
                  MCCFIInstruction::createOffset(nullptr, DwarfReg, Offset));
+      }
     } else {
       BuildCFI(MBB, MBBI, DL,
                MCCFIInstruction::createRestore(nullptr, DwarfReg));
     }
   }
+  if (auto *MI = X86FI->getStackPtrSaveMI()) {
+    int FI = MI->getOperand(1).getIndex();
+    int64_t Offset = MFI.getObjectOffset(FI) + 2 * SlotSize;
+    SmallString<64> CfaExpr;
+    Register FramePtr = TRI->getFrameRegister(MF);
+    const Register MachineFramePtr =
+        STI.isTarget64BitILP32()
+            ? Register(getX86SubSuperRegister(FramePtr, 64))
+            : FramePtr;
+    unsigned DwarfFramePtr = MRI->getDwarfRegNum(MachineFramePtr, true);
+    CfaExpr.push_back((uint8_t)(dwarf::DW_OP_breg0 + DwarfFramePtr));
+    uint8_t buffer[16];
+    CfaExpr.append(buffer, buffer + encodeSLEB128(Offset, buffer));
+    CfaExpr.push_back(dwarf::DW_OP_deref);
+
+    SmallString<64> DefCfaExpr;
+    DefCfaExpr.push_back(dwarf::DW_CFA_def_cfa_expression);
+    DefCfaExpr.append(buffer, buffer + encodeSLEB128(CfaExpr.size(), buffer));
+    DefCfaExpr.append(CfaExpr.str());
+    // DW_CFA_def_cfa_expression: DW_OP_breg5 offset, DW_OP_deref
+    BuildCFI(MBB, MBBI, DL,
+             MCCFIInstruction::createEscape(nullptr, DefCfaExpr.str()),
+             MachineInstr::FrameSetup);
+  }
 }
 
 void X86FrameLowering::emitZeroCallUsedRegs(BitVector RegsToZero,
                                             MachineBasicBlock &MBB) const {
   const MachineFunction &MF = *MBB.getParent();
 
   // Insertion point.
   MachineBasicBlock::iterator MBBI = MBB.getFirstTerminator();
if (TRI->isGeneralPurposeRegister(MF, Reg)) {
       GPRsToZero.set(getX86SubSuperRegister(Reg, 32));
       RegsToZero.reset(Reg);
     }
 
   for (MCRegister Reg : GPRsToZero.set_bits())
     BuildMI(MBB, MBBI, DL, TII.get(X86::XOR32rr), Reg)
         .addReg(Reg, RegState::Undef)
         .addReg(Reg, RegState::Undef);
 
   // Zero out registers.
   for (MCRegister Reg : RegsToZero.set_bits()) {
     if (ST.hasMMX() && X86::VR64RegClass.contains(Reg))
       // FIXME: Ignore MMX registers?
       continue;
 
     unsigned XorOp;
     if (X86::VR128RegClass.contains(Reg)) {
       // XMM#
const Register MachineFramePtr =
       STI.isTarget64BitILP32()
           ? Register(getX86SubSuperRegister(FramePtr, 64)) : FramePtr;
   Register BasePtr = TRI->getBaseRegister();
   bool HasWinCFI = false;
 
   // Debug location must be unknown since the first debug location is used
   // to determine the end of the prologue.
   DebugLoc DL;
+  Register ArgBaseReg;
+
+  // Emit extra prolog for argument stack slot reference.
+  if (auto *MI = X86FI->getStackPtrSaveMI()) {
+    // MI is lea instruction that created in X86ArgumentStackSlotPass.
+    // Creat extra prolog for stack realignment.
+    ArgBaseReg = MI->getOperand(0).getReg();
+    // leal    4(%esp), %basereg
+    // .cfi_def_cfa %basereg, 0
+    // andl    $-128, %esp
+    // pushl   -4(%basereg)
+    BuildMI(MBB, MBBI, DL, TII.get(Is64Bit ? X86::LEA64r : X86::LEA32r),
+            ArgBaseReg)
+        .addUse(StackPtr)
+        .addImm(1)
+        .addUse(X86::NoRegister)
+        .addImm(SlotSize)
+        .addUse(X86::NoRegister)
+        .setMIFlag(MachineInstr::FrameSetup);
+    if (NeedsDwarfCFI) {
+      // .cfi_def_cfa %basereg, 0
+      unsigned DwarfStackPtr = TRI->getDwarfRegNum(ArgBaseReg, true);
+      BuildCFI(MBB, MBBI, DL,
+               MCCFIInstruction::cfiDefCfa(nullptr, DwarfStackPtr, 0),
+               MachineInstr::FrameSetup);
+    }
+    BuildStackAlignAND(MBB, MBBI, DL, StackPtr, MaxAlign);
+    int64_t Offset = Is64Bit ? -2 * (int64_t)SlotSize : -1 * (int64_t)SlotSize;
+    BuildMI(MBB, MBBI, DL, TII.get(Is64Bit ? X86::PUSH64rmm: X86::PUSH32rmm))
+        .addReg(ArgBaseReg)
+        .addImm(1)
+        .addReg(X86::NoRegister)
+        .addImm(Offset)
+        .addReg(X86::NoRegister)
+        .setMIFlag(MachineInstr::FrameSetup);
+  }
 
   // Space reserved for stack-based arguments when making a (ABI-guaranteed)
   // tail call.
   unsigned TailCallArgReserveSize = -X86FI->getTCReturnAddrDelta();
   if (TailCallArgReserveSize  && IsWin64Prologue)
     report_fatal_error("Can't handle guaranteed tail call under win64 yet");
 
   const bool EmitStackProbeCall =
if (HasFP) {
     if (TRI->hasStackRealignment(MF) && !IsWin64Prologue)
       NumBytes = alignTo(NumBytes, MaxAlign);
 
     // Save EBP/RBP into the appropriate stack slot.
     BuildMI(MBB, MBBI, DL, TII.get(Is64Bit ? X86::PUSH64r : X86::PUSH32r))
       .addReg(MachineFramePtr, RegState::Kill)
       .setMIFlag(MachineInstr::FrameSetup);
 
-    if (NeedsDwarfCFI) {
+    if (NeedsDwarfCFI && !ArgBaseReg.isValid()) {
       // Mark the place where EBP/RBP was saved.
       // Define the current CFA rule to use the provided offset.
       assert(StackSize);
       BuildCFI(MBB, MBBI, DL,
                MCCFIInstruction::cfiDefCfaOffset(
                    nullptr, -2 * stackGrowth + (int)TailCallArgReserveSize),
                MachineInstr::FrameSetup);
 
if (!IsFunclet) {
         if (!X86FI->hasSwiftAsyncContext())
           BuildMI(MBB, MBBI, DL,
                   TII.get(Uses64BitFramePtr ? X86::MOV64rr : X86::MOV32rr),
                   FramePtr)
               .addReg(StackPtr)
               .setMIFlag(MachineInstr::FrameSetup);
 
         if (NeedsDwarfCFI) {
+          if (ArgBaseReg.isValid()) {
+            SmallString<64> CfaExpr;
+            CfaExpr.push_back(dwarf::DW_CFA_expression);
+            uint8_t buffer[16];
+            unsigned DwarfReg = TRI->getDwarfRegNum(MachineFramePtr, true);
+            CfaExpr.append(buffer, buffer + encodeULEB128(DwarfReg, buffer));
+            CfaExpr.push_back(2);
+            CfaExpr.push_back((uint8_t)(dwarf::DW_OP_breg0 + DwarfReg));
+            CfaExpr.push_back(0);
+            // DW_CFA_expression: reg5 DW_OP_breg5 +0
+            BuildCFI(MBB, MBBI, DL,
+                     MCCFIInstruction::createEscape(nullptr, CfaExpr.str()),
+                     MachineInstr::FrameSetup);
+          } else {
             // Mark effective beginning of when frame pointer becomes valid.
             // Define the current CFA to use the EBP/RBP register.
             unsigned DwarfFramePtr = TRI->getDwarfRegNum(MachineFramePtr, true);
             BuildCFI(
                 MBB, MBBI, DL,
                 MCCFIInstruction::createDefCfaRegister(nullptr, DwarfFramePtr),
                 MachineInstr::FrameSetup);
           }
+        }
 
         if (NeedsWinFPO) {
           // .cv_fpo_setframe $FramePtr
           HasWinCFI = true;
           BuildMI(MBB, MBBI, DL, TII.get(X86::SEH_SetFrame))
               .addImm(FramePtr)
               .addImm(0)
               .setMIFlag(MachineInstr::FrameSetup);
if (NeedsWinCFI) {
           .addImm(Reg)
           .setMIFlag(MachineInstr::FrameSetup);
     }
   }
 
   // Realign stack after we pushed callee-saved registers (so that we'll be
   // able to calculate their offsets from the frame pointer).
   // Don't do this for Win64, it needs to realign the stack after the prologue.
-  if (!IsWin64Prologue && !IsFunclet && TRI->hasStackRealignment(MF)) {
+  if (!IsWin64Prologue && !IsFunclet && TRI->hasStackRealignment(MF) &&
+      !ArgBaseReg.isValid()) {
     assert(HasFP && "There should be a frame pointer if stack is realigned.");
     BuildStackAlignAND(MBB, MBBI, DL, StackPtr, MaxAlign);
 
     if (NeedsWinCFI) {
       HasWinCFI = true;
       BuildMI(MBB, MBBI, DL, TII.get(X86::SEH_StackAlign))
           .addImm(MaxAlign)
           .setMIFlag(MachineInstr::FrameSetup);
if (X86FI->getHasSEHFramePtrSave() && !IsFunclet) {
           getFrameIndexReference(MF, X86FI->getSEHFramePtrSaveIndex(), UsedReg)
               .getFixed();
       assert(UsedReg == BasePtr);
       addRegOffset(BuildMI(MBB, MBBI, DL, TII.get(Opm)), UsedReg, true, Offset)
           .addReg(FramePtr)
           .setMIFlag(MachineInstr::FrameSetup);
     }
   }
+  if (ArgBaseReg.isValid()) {
+    // Save argument base pointer.
+    auto *MI = X86FI->getStackPtrSaveMI();
+    int FI = MI->getOperand(1).getIndex();
+    unsigned MOVmr = Is64Bit ? X86::MOV64mr : X86::MOV32mr;
+    // movl    %basereg, offset(%ebp)
+    addFrameReference(BuildMI(MBB, MBBI, DL, TII.get(MOVmr)), FI)
+        .addReg(ArgBaseReg)
+        .setMIFlag(MachineInstr::FrameSetup);
+  }
 
   if (((!HasFP && NumBytes) || PushedRegs) && NeedsDwarfCFI) {
     // Mark end of stack pointer adjustment.
     if (!HasFP && NumBytes) {
       // Define the current CFA rule to use the provided offset.
       assert(StackSize);
       BuildCFI(
           MBB, MBBI, DL,
void X86FrameLowering::emitEpilogue(MachineFunction &MF,
   unsigned TailCallArgReserveSize = -X86FI->getTCReturnAddrDelta();
   bool HasFP = hasFP(MF);
   uint64_t NumBytes = 0;
 
   bool NeedsDwarfCFI = (!MF.getTarget().getTargetTriple().isOSDarwin() &&
                         !MF.getTarget().getTargetTriple().isOSWindows()) &&
                        MF.needsFrameMoves();
 
+  Register ArgBaseReg;
+  if (auto *MI = X86FI->getStackPtrSaveMI()) {
+    unsigned Opc = X86::LEA32r;
+    Register StackReg = X86::ESP;
+    ArgBaseReg = MI->getOperand(0).getReg();
+    if (STI.is64Bit()) {
+      Opc = X86::LEA64r;
+      StackReg = X86::RSP;
+    }
+    // leal    -8(%basereg), %esp
+    // .cfi_def_cfa %esp, 4
+    BuildMI(MBB, MBBI, DL, TII.get(Opc), StackReg)
+        .addUse(ArgBaseReg)
+        .addImm(1)
+        .addUse(X86::NoRegister)
+        .addImm((int64_t)SlotSize * -2)
+        .addUse(X86::NoRegister)
+        .setMIFlag(MachineInstr::FrameDestroy);
+    if (NeedsDwarfCFI) {
+      unsigned DwarfStackPtr = TRI->getDwarfRegNum(StackReg, true);
+      BuildCFI(MBB, MBBI, DL,
+               MCCFIInstruction::cfiDefCfa(nullptr, DwarfStackPtr, SlotSize),
+               MachineInstr::FrameDestroy);
+      --MBBI;
+    }
+    --MBBI;
+  }
+
   if (IsFunclet) {
     assert(HasFP && "EH funclets without FP not yet implemented");
     NumBytes = getWinEHFuncletFrameSize(MF);
   } else if (HasFP) {
     // Calculate required stack adjustment.
     uint64_t FrameSize = StackSize - SlotSize;
     NumBytes = FrameSize - CSSize - TailCallArgReserveSize;
 
if (X86FI->hasSwiftAsyncContext()) {
       BuildMI(MBB, MBBI, DL, TII.get(X86::BTR64ri8),
               MachineFramePtr)
           .addUse(MachineFramePtr)
           .addImm(60)
           .setMIFlag(MachineInstr::FrameDestroy);
     }
 
     if (NeedsDwarfCFI) {
+      if (!ArgBaseReg.isValid()) {
         unsigned DwarfStackPtr =
             TRI->getDwarfRegNum(Is64Bit ? X86::RSP : X86::ESP, true);
         BuildCFI(MBB, MBBI, DL,
                  MCCFIInstruction::cfiDefCfa(nullptr, DwarfStackPtr, SlotSize),
                  MachineInstr::FrameDestroy);
+      }
       if (!MBB.succ_empty() && !MBB.isReturnBlock()) {
         unsigned DwarfFramePtr = TRI->getDwarfRegNum(MachineFramePtr, true);
         BuildCFI(MBB, AfterPop, DL,
                  MCCFIInstruction::createRestore(nullptr, DwarfFramePtr),
                  MachineInstr::FrameDestroy);
         --MBBI;
         --AfterPop;
       }
if (Opc != X86::DBG_VALUE && !PI->isTerminator()) {
           (Opc != X86::BTR64ri8 || !PI->getFlag(MachineInstr::FrameDestroy)) &&
           (Opc != X86::ADD64ri8 || !PI->getFlag(MachineInstr::FrameDestroy)))
         break;
       FirstCSPop = PI;
     }
 
     --MBBI;
   }
+  if (ArgBaseReg.isValid()) {
+    // Restore argument base pointer.
+    auto *MI = X86FI->getStackPtrSaveMI();
+    int FI = MI->getOperand(1).getIndex();
+    unsigned MOVrm = Is64Bit ? X86::MOV64rm : X86::MOV32rm;
+    // movl   offset(%ebp), %basereg
+    addFrameReference(BuildMI(MBB, MBBI, DL, TII.get(MOVrm), ArgBaseReg), FI)
+        .setMIFlag(MachineInstr::FrameDestroy);
+  }
   MBBI = FirstCSPop;
 
   if (IsFunclet && Terminator->getOpcode() == X86::CATCHRET)
     emitCatchRetReturnValue(MBB, FirstCSPop, &*Terminator);
 
   if (MBBI != MBB.end())
     DL = MBBI->getDebugLoc();
   // If there is an ADD32ri or SUB32ri of ESP immediately before this
void X86FrameLowering::adjustFrameForMsvcCxxEh(MachineFunction &MF) const {
   DebugLoc DL = MBB.findDebugLoc(MBBI);
   addFrameReference(BuildMI(MBB, MBBI, DL, TII.get(X86::MOV64mi32)),
                     UnwindHelpFI)
       .addImm(-2);
 }
 
 void X86FrameLowering::processFunctionBeforeFrameIndicesReplaced(
     MachineFunction &MF, RegScavenger *RS) const {
+  X86MachineFunctionInfo *X86FI = MF.getInfo<X86MachineFunctionInfo>();
+
   if (STI.is32Bit() && MF.hasEHFunclets())
     restoreWinEHStackPointersInParent(MF);
+  // We have emitted prolog and epilog. Don't need stack pointer saving
+  // instruction any more.
+  if (auto *MI = X86FI->getStackPtrSaveMI()) {
+    MI->eraseFromParent();
+    X86FI->setStackPtrSaveMI(nullptr);
+  }
 }
 
 void X86FrameLowering::restoreWinEHStackPointersInParent(
     MachineFunction &MF) const {
   // 32-bit functions have to restore stack pointers when control is transferred
   // back to the parent function. These blocks are identified as eh pads that
   // are not funclet entries.
   bool IsSEH = isAsynchronousEHPersonality(
       classifyEHPersonality(MF.getFunction().getPersonalityFn()));
   for (MachineBasicBlock &MBB : MF) {
     bool NeedsRestore = MBB.isEHPad() && !MBB.isEHFuncletEntry();
     if (NeedsRestore)
       restoreWin32EHStackPointers(MBB, MBB.begin(), DebugLoc(),
                                   /*RestoreSP=*/IsSEH);
   }
 }

llvm/lib/Target/X86/X86MachineFunctionInfo.h

class X86MachineFunctionInfo : public MachineFunctionInfo {
   /// addr]. If so, bit 60 of the in-memory frame pointer will be 1 to enable
   /// other tools to detect the extended record.
   bool HasSwiftAsyncContext = false;
 
   /// True if this function has tile virtual register. This is used to
   /// determine if we should insert tilerelease in frame lowering.
   bool HasVirtualTileReg = false;
 
+  MachineInstr *StackPtrSaveMI = nullptr;
+
   std::optional<int> SwiftAsyncContextFrameIdx;
 
   // Preallocated fields are only used during isel.
   // FIXME: Can we find somewhere else to store these?
   DenseMap<const Value *, size_t> PreallocatedIds;
   SmallVector<size_t, 0> PreallocatedStackSizes;
   SmallVector<SmallVector<size_t, 4>, 0> PreallocatedArgOffsets;
 
public:
   void setHasPreallocatedCall(bool v) { HasPreallocatedCall = v; }
 
   bool hasSwiftAsyncContext() const { return HasSwiftAsyncContext; }
   void setHasSwiftAsyncContext(bool v) { HasSwiftAsyncContext = v; }
 
   bool hasVirtualTileReg() const { return HasVirtualTileReg; }
   void setHasVirtualTileReg(bool v) { HasVirtualTileReg = v; }
 
+  void setStackPtrSaveMI(MachineInstr *MI) { StackPtrSaveMI = MI; }
+  MachineInstr *getStackPtrSaveMI() const { return StackPtrSaveMI; }
+
   std::optional<int> getSwiftAsyncContextFrameIdx() const {
     return SwiftAsyncContextFrameIdx;
   }
   void setSwiftAsyncContextFrameIdx(int v) { SwiftAsyncContextFrameIdx = v; }
 
   size_t getPreallocatedIdForCallSite(const Value *CS) {
     auto Insert = PreallocatedIds.insert({CS, PreallocatedIds.size()});
     if (Insert.second) {

llvm/lib/Target/X86/X86RegisterInfo.h

bool isFixedRegister(const MachineFunction &MF,
                        MCRegister PhysReg) const override;
 
   void adjustStackMapLiveOutMask(uint32_t *Mask) const override;
 
   bool hasBasePointer(const MachineFunction &MF) const;
 
   bool canRealignStack(const MachineFunction &MF) const override;
 
+  void eliminateFrameIndex(MachineBasicBlock::iterator II,
+                           unsigned FIOperandNum, Register BaseReg,
+                           int FIOffset) const;
+
   bool eliminateFrameIndex(MachineBasicBlock::iterator MI,
                            int SPAdj, unsigned FIOperandNum,
                            RegScavenger *RS = nullptr) const override;
 
   /// findDeadCallerSavedReg - Return a caller-saved register that isn't live
   /// when it reaches the "return" instruction. We can then pop a stack object
   /// to this register without worry about clobbering it.
   unsigned findDeadCallerSavedReg(MachineBasicBlock &MBB,

llvm/lib/Target/X86/X86RegisterInfo.cpp

 //===----------------------------------------------------------------------===//
 
 static bool CantUseSP(const MachineFrameInfo &MFI) {
   return MFI.hasVarSizedObjects() || MFI.hasOpaqueSPAdjustment();
 }
 
 bool X86RegisterInfo::hasBasePointer(const MachineFunction &MF) const {
   const X86MachineFunctionInfo *X86FI = MF.getInfo<X86MachineFunctionInfo>();
+  // We have a virtual register to reference argument, and don't need base
+  // pointer.
+  if (X86FI->getStackPtrSaveMI() != nullptr)
+    return false;
+
   if (X86FI->hasPreallocatedCall())
     return true;
 
   const MachineFrameInfo &MFI = MF.getFrameInfo();
 
   if (!EnableBasePointer)
     return false;
 
static bool isFuncletReturnInstr(MachineInstr &MI) {
   case X86::CLEANUPRET:
     return true;
   default:
     return false;
   }
   llvm_unreachable("impossible");
 }
 
+void X86RegisterInfo::eliminateFrameIndex(MachineBasicBlock::iterator II,
+                                          unsigned FIOperandNum,
+                                          Register BaseReg,
+                                          int FIOffset) const {
+  MachineInstr &MI = *II;
+  unsigned Opc = MI.getOpcode();
+  if (Opc == TargetOpcode::LOCAL_ESCAPE) {
+    MachineOperand &FI = MI.getOperand(FIOperandNum);
+    FI.ChangeToImmediate(FIOffset);
+    return;
+  }
+
+  MI.getOperand(FIOperandNum).ChangeToRegister(BaseReg, false);
+
+  // The frame index format for stackmaps and patchpoints is different from the
+  // X86 format. It only has a FI and an offset.
+  if (Opc == TargetOpcode::STACKMAP || Opc == TargetOpcode::PATCHPOINT) {
+    assert(BasePtr == FramePtr && "Expected the FP as base register");
+    int64_t Offset = MI.getOperand(FIOperandNum + 1).getImm() + FIOffset;
+    MI.getOperand(FIOperandNum + 1).ChangeToImmediate(Offset);
+    return;
+  }
+
+  if (MI.getOperand(FIOperandNum+3).isImm()) {
+    // Offset is a 32-bit integer.
+    int Imm = (int)(MI.getOperand(FIOperandNum + 3).getImm());
+    int Offset = FIOffset + Imm;
+    assert((!Is64Bit || isInt<32>((long long)FIOffset + Imm)) &&
+           "Requesting 64-bit offset in 32-bit immediate!");
+    if (Offset != 0 || !tryOptimizeLEAtoMOV(II))
+      MI.getOperand(FIOperandNum + 3).ChangeToImmediate(Offset);
+  } else {
+    // Offset is symbolic. This is extremely rare.
+    uint64_t Offset = FIOffset +
+      (uint64_t)MI.getOperand(FIOperandNum+3).getOffset();
+    MI.getOperand(FIOperandNum + 3).setOffset(Offset);
+  }
+}
+
 bool
 X86RegisterInfo::eliminateFrameIndex(MachineBasicBlock::iterator II,
                                      int SPAdj, unsigned FIOperandNum,
                                      RegScavenger *RS) const {
   MachineInstr &MI = *II;
   MachineBasicBlock &MBB = *MI.getParent();
   MachineFunction &MF = *MBB.getParent();
   MachineBasicBlock::iterator MBBI = MBB.getFirstTerminator();

llvm/lib/Target/X86/X86RegisterInfo.td

def GR64 : RegisterClass<"X86", [i64], 64,
                               RBX, R14, R15, R12, R13, RBP, RSP, RIP)>;
 
 // GR64PLTSafe - 64-bit GPRs without R10, R11, RSP and RIP. Could be used when
 // emitting code for intrinsics, which use implict input registers.
 def GR64PLTSafe : RegisterClass<"X86", [i64], 64,
                               (add RAX, RCX, RDX, RSI, RDI, R8, R9,
                                    RBX, R14, R15, R12, R13, RBP)>;
 
+// It includes the GPR that are used as scratch register for Linux64 calling
+// convention
+def GR64_ArgRef: RegisterClass<"X86", [i64], 64, (add R10, R11)> {
+  let GeneratePressureSet = 0;
+}
+
+// It includes the GPR that are used as scratch register for Linux32 calling
+// convention
+def GR32_ArgRef: RegisterClass<"X86", [i32], 32, (add ECX, EDX)> {
+  let GeneratePressureSet = 0;
+}
+
 // Segment registers for use by MOV instructions (and others) that have a
 //   segment register as one operand.  Always contain a 16-bit segment
 //   descriptor.
 def SEGMENT_REG : RegisterClass<"X86", [i16], 16, (add CS, DS, SS, ES, FS, GS)>;
 
 // Debug registers.
 def DEBUG_REG : RegisterClass<"X86", [i32], 32, (sequence "DR%u", 0, 15)>;
 

llvm/lib/Target/X86/X86TargetMachine.cpp

extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeX86Target() {
   initializeX86FlagsCopyLoweringPassPass(PR);
   initializeX86LoadValueInjectionLoadHardeningPassPass(PR);
   initializeX86LoadValueInjectionRetHardeningPassPass(PR);
   initializeX86OptimizeLEAPassPass(PR);
   initializeX86PartialReductionPass(PR);
   initializePseudoProbeInserterPass(PR);
   initializeX86ReturnThunksPass(PR);
   initializeX86DAGToDAGISelPass(PR);
+  initializeX86ArgumentStackSlotPassPass(PR);
 }
 
 static std::unique_ptr<TargetLoweringObjectFile> createTLOF(const Triple &TT) {
   if (TT.isOSBinFormatMachO()) {
     if (TT.getArch() == Triple::x86_64)
       return std::make_unique<X86_64MachoTargetObjectFile>();
     return std::make_unique<TargetLoweringObjectFileMachO>();
   }
bool X86PassConfig::addPreISel() {
   // Only add this pass for 32-bit x86 Windows.
   const Triple &TT = TM->getTargetTriple();
   if (TT.isOSWindows() && TT.getArch() == Triple::x86)
     addPass(createX86WinEHStatePass());
   return true;
 }
 
 void X86PassConfig::addPreRegAlloc() {
+  addPass(createX86ArgumentStackSlotPass());
   if (getOptLevel() != CodeGenOpt::None) {
     addPass(&LiveRangeShrinkID);
     addPass(createX86FixupSetCC());
     addPass(createX86OptimizeLEAs());
     addPass(createX86CallFrameOptimization());
     addPass(createX86AvoidStoreForwardingBlocks());
   }
 

llvm/test/CodeGen/MIR/X86/inline-asm-registers.mir

 liveins:
   - { reg: '$rdi' }
   - { reg: '$rsi' }
 body: |
   bb.0.entry:
     liveins: $rdi, $rsi
 
   ; CHECK-LABEL: name: test
-  ; CHECK: INLINEASM &foo, 0 /* attdialect */, 4456458 /* regdef:GR64 */, def $rsi, 4456458 /* regdef:GR64 */, def dead $rdi,
-    INLINEASM &foo, 0, 4456458, def $rsi, 4456458, def dead $rdi, 2147549193, killed $rdi, 2147483657, killed $rsi, 12, implicit-def dead early-clobber $eflags
+  ; CHECK: INLINEASM &foo, 0 /* attdialect */, 4521994 /* regdef:GR64 */, def $rsi, 4521994 /* regdef:GR64 */, def dead $rdi,
+    INLINEASM &foo, 0, 4521994, def $rsi, 4521994, def dead $rdi, 2147549193, killed $rdi, 2147483657, killed $rsi, 12, implicit-def dead early-clobber $eflags
     $rax = MOV64rr killed $rsi
     RET64 killed $rax
 ...
 ---
 name:            test2
 tracksRegLiveness: true
 liveins:
   - { reg: '$rdi' }
   - { reg: '$rsi' }
 body: |
   bb.0.entry:
     liveins: $rdi, $rsi
 
   ; Verify that the register ties are preserved.
   ; CHECK-LABEL: name: test2
-  ; CHECK: INLINEASM &foo, 0 /* attdialect */, 4456458 /* regdef:GR64 */, def $rsi, 4456458 /* regdef:GR64 */, def dead $rdi, 2147549193 /* reguse tiedto:$1 */, killed $rdi(tied-def 5), 2147483657 /* reguse tiedto:$0 */, killed $rsi(tied-def 3), 12 /* clobber */, implicit-def dead early-clobber $eflags
-    INLINEASM &foo, 0, 4456458, def $rsi, 4456458, def dead $rdi, 2147549193, killed $rdi(tied-def 5), 2147483657, killed $rsi(tied-def 3), 12, implicit-def dead early-clobber $eflags
+  ; CHECK: INLINEASM &foo, 0 /* attdialect */, 4521994 /* regdef:GR64 */, def $rsi, 4521994 /* regdef:GR64 */, def dead $rdi, 2147549193 /* reguse tiedto:$1 */, killed $rdi(tied-def 5), 2147483657 /* reguse tiedto:$0 */, killed $rsi(tied-def 3), 12 /* clobber */, implicit-def dead early-clobber $eflags
+    INLINEASM &foo, 0, 4521994, def $rsi, 4521994, def dead $rdi, 2147549193, killed $rdi(tied-def 5), 2147483657, killed $rsi(tied-def 3), 12, implicit-def dead early-clobber $eflags
     $rax = MOV64rr killed $rsi
     RET64 killed $rax
 ...

llvm/test/CodeGen/X86/O0-pipeline.ll

 ; CHECK-NEXT:       Safe Stack instrumentation pass
 ; CHECK-NEXT:       Insert stack protectors
 ; CHECK-NEXT:       Module Verifier
 ; CHECK-NEXT:       Assignment Tracking Analysis
 ; CHECK-NEXT:       X86 DAG->DAG Instruction Selection
 ; CHECK-NEXT:       X86 PIC Global Base Reg Initialization
 ; CHECK-NEXT:       Finalize ISel and expand pseudo-instructions
 ; CHECK-NEXT:       Local Stack Slot Allocation
+; CHECK-NEXT:       Argument Stack Rebase
 ; CHECK-NEXT:       X86 speculative load hardening
 ; CHECK-NEXT:       MachineDominator Tree Construction
 ; CHECK-NEXT:       X86 EFLAGS copy lowering
 ; CHECK-NEXT:       X86 DynAlloca Expander
 ; CHECK-NEXT:       Fast Tile Register Preconfigure
 ; CHECK-NEXT:       Eliminate PHI nodes for register allocation
 ; CHECK-NEXT:       Two-Address instruction pass
 ; CHECK-NEXT:       Fast Register Allocator

llvm/test/CodeGen/X86/i386-baseptr.ll

entry:
   %a = alloca i32, i32 %k, align 4
   store i32 0, ptr %a, align 4
   ret void
 }
 
 define void @clobber_base() #0 {
 ; CHECK-LABEL: clobber_base:
 ; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    leal {{[0-9]+}}(%esp), %ecx
+; CHECK-NEXT:    .cfi_def_cfa %ecx, 0
+; CHECK-NEXT:    andl $-128, %esp
+; CHECK-NEXT:    pushl -4(%ecx)
 ; CHECK-NEXT:    pushl %ebp
-; CHECK-NEXT:    .cfi_def_cfa_offset 8
-; CHECK-NEXT:    .cfi_offset %ebp, -8
 ; CHECK-NEXT:    movl %esp, %ebp
-; CHECK-NEXT:    .cfi_def_cfa_register %ebp
+; CHECK-NEXT:    .cfi_escape 0x10, 0x05, 0x02, 0x75, 0x00 #
 ; CHECK-NEXT:    pushl %esi
-; CHECK-NEXT:    andl $-128, %esp
-; CHECK-NEXT:    subl $128, %esp
-; CHECK-NEXT:    movl %esp, %esi
-; CHECK-NEXT:    .cfi_offset %esi, -12
+; CHECK-NEXT:    subl $244, %esp
+; CHECK-NEXT:    movl %ecx, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
+; CHECK-NEXT:    .cfi_escape 0x10, 0x06, 0x02, 0x75, 0x7c #
+; CHECK-NEXT:    .cfi_escape 0x0f, 0x04, 0x75, 0x84, 0x7f, 0x06 #
 ; CHECK-NEXT:    calll helper@PLT
 ; CHECK-NEXT:    movl %esp, %ecx
 ; CHECK-NEXT:    leal 31(,%eax,4), %eax
 ; CHECK-NEXT:    andl $-32, %eax
 ; CHECK-NEXT:    movl %ecx, %edx
 ; CHECK-NEXT:    subl %eax, %edx
 ; CHECK-NEXT:    movl %edx, %esp
 ; CHECK-NEXT:    negl %eax
 ; CHECK-NEXT:    movl $405, %esi # imm = 0x195
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:    movl $8, %edx
 ; CHECK-NEXT:    #APP
-; CHECK-NEXT:    movl %edx, (%esi)
+; CHECK-NEXT:    movl %edx, -120(%ebp)
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:    movl $0, (%ecx,%eax)
+; CHECK-NEXT:    movl {{[-0-9]+}}(%e{{[sb]}}p), %ecx # 4-byte Reload
 ; CHECK-NEXT:    leal -4(%ebp), %esp
 ; CHECK-NEXT:    popl %esi
 ; CHECK-NEXT:    popl %ebp
+; CHECK-NEXT:    leal -8(%ecx), %esp
 ; CHECK-NEXT:    .cfi_def_cfa %esp, 4
 ; CHECK-NEXT:    retl
 entry:
   %k = call i32 @helper()
   %a = alloca i32, align 128
   %b = alloca i32, i32 %k, align 4
   ; clobber base pointer register
   tail call void asm sideeffect "nop", "{si}"(i32 405)
entry:
   tail call void asm sideeffect "nop", "{si}"(i32 405)
   ; clobber argument pointer register
   tail call void asm sideeffect "nop", "{bx}"(i32 405)
   call void asm sideeffect "movl $0, $1", "r,*m"(i32 8, ptr elementtype(i32) %a)
   store i32 %param6, ptr %b, align 4
   ret void
 }
 
-attributes #0 = { "frame-pointer"="all"}
+attributes #0 = {"frame-pointer"="all"}
 !llvm.module.flags = !{!0}
 !0 = !{i32 2, !"override-stack-alignment", i32 32}

llvm/test/CodeGen/X86/opt-pipeline.ll

 ; CHECK-NEXT:       MachineDominator Tree Construction
 ; CHECK-NEXT:       Machine Block Frequency Analysis
 ; CHECK-NEXT:       Machine Common Subexpression Elimination
 ; CHECK-NEXT:       MachinePostDominator Tree Construction
 ; CHECK-NEXT:       Machine Cycle Info Analysis
 ; CHECK-NEXT:       Machine code sinking
 ; CHECK-NEXT:       Peephole Optimizations
 ; CHECK-NEXT:       Remove dead machine instructions
+; CHECK-NEXT:       Argument Stack Rebase
 ; CHECK-NEXT:       Live Range Shrink
 ; CHECK-NEXT:       X86 Fixup SetCC
 ; CHECK-NEXT:       Lazy Machine Block Frequency Analysis
 ; CHECK-NEXT:       X86 LEA Optimize
 ; CHECK-NEXT:       X86 Optimize Call Frame
 ; CHECK-NEXT:       X86 Avoid Store Forwarding Block
 ; CHECK-NEXT:       X86 speculative load hardening
 ; CHECK-NEXT:       MachineDominator Tree Construction

llvm/test/CodeGen/X86/statepoint-invoke-ra-enter-at-end.mir

body:             |
   ; CHECK-NEXT:   successors: %bb.7(0x80000000)
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[INC64r:%[0-9]+]]:gr64_with_sub_8bit = nuw nsw INC64r [[INC64r]], implicit-def dead $eflags
   ; CHECK-NEXT:   [[MOV64rm2:%[0-9]+]]:gr64 = MOV64rm undef %59:gr64, 1, $noreg, 0, $noreg :: (load unordered (s64) from `ptr addrspace(1) undef`, addrspace 1)
   ; CHECK-NEXT:   [[NOT64r2:%[0-9]+]]:gr64 = NOT64r [[NOT64r2]]
   ; CHECK-NEXT:   CMP64rr [[NOT64r2]], [[COPY6]], implicit-def $eflags
   ; CHECK-NEXT:   undef %100.sub_32bit:gr64_with_sub_8bit = MOV32ri 0
   ; CHECK-NEXT:   [[CMOV64rr:%[0-9]+]]:gr64 = CMOV64rr [[CMOV64rr]], %100, 4, implicit killed $eflags
-  ; CHECK-NEXT:   INLINEASM &"lock btsq $0,($1)", 1 /* sideeffect attdialect */, 4456457 /* reguse:GR64 */, %100, 4456457 /* reguse:GR64 */, undef %56:gr64, 12 /* clobber */, implicit-def dead early-clobber $df, 12 /* clobber */, implicit-def early-clobber $fpsw, 12 /* clobber */, implicit-def dead early-clobber $eflags
+  ; CHECK-NEXT:   INLINEASM &"lock btsq $0,($1)", 1 /* sideeffect attdialect */, 4521993 /* reguse:GR64 */, %100, 4521993 /* reguse:GR64 */, undef %56:gr64, 12 /* clobber */, implicit-def dead early-clobber $df, 12 /* clobber */, implicit-def early-clobber $fpsw, 12 /* clobber */, implicit-def dead early-clobber $eflags
   ; CHECK-NEXT:   LCMPXCHG32 undef %67:gr64, 1, $noreg, 0, $noreg, [[COPY5]], implicit-def dead $eax, implicit-def dead $eflags, implicit undef $eax :: (load store acquire monotonic (s32) on `ptr addrspace(1) undef`, addrspace 1)
   ; CHECK-NEXT:   ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp
   ; CHECK-NEXT:   $rdi = COPY [[COPY4]]
   ; CHECK-NEXT:   CALL64pcrel32 target-flags(x86-plt) @wobble, csr_64, implicit $rsp, implicit $ssp, implicit $rdi, implicit-def $rsp, implicit-def $ssp
   ; CHECK-NEXT:   ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp
   ; CHECK-NEXT:   [[MOV64rm2:%[0-9]+]]:gr64 = MOV64rm $rip, 1, $noreg, target-flags(x86-gotpcrel) @global, $noreg :: (load (s64) from got)
   ; CHECK-NEXT:   [[MOV64rm3:%[0-9]+]]:gr64 = MOV64rm [[MOV64rm2]], 1, $noreg, 0, $noreg :: (dereferenceable load unordered (s64) from @global)
   ; CHECK-NEXT:   [[NOT64r2:%[0-9]+]]:gr64 = NOT64r [[NOT64r2]]
body:             |
   bb.7.bb33:
     successors: %bb.7(0x80000000)
 
     %81:gr64_with_sub_8bit = nuw nsw INC64r %81, implicit-def dead $eflags
     %63:gr64 = MOV64rm undef %59:gr64, 1, $noreg, 0, $noreg :: (load unordered (s64) from `ptr addrspace(1) undef`, addrspace 1)
     %63:gr64 = NOT64r %63
     CMP64rr %63, %31, implicit-def $eflags
     %63:gr64 = CMOV64rr %63, %53, 4, implicit killed $eflags
-    INLINEASM &"lock btsq $0,($1)", 1 /* sideeffect attdialect */, 4456457 /* reguse:GR64 */, %53, 4456457 /* reguse:GR64 */, undef %56:gr64, 12 /* clobber */, implicit-def dead early-clobber $df, 12 /* clobber */, implicit-def early-clobber $fpsw, 12 /* clobber */, implicit-def dead early-clobber $eflags
+    INLINEASM &"lock btsq $0,($1)", 1 /* sideeffect attdialect */, 4521993 /* reguse:GR64 */, %53, 4521993 /* reguse:GR64 */, undef %56:gr64, 12 /* clobber */, implicit-def dead early-clobber $df, 12 /* clobber */, implicit-def early-clobber $fpsw, 12 /* clobber */, implicit-def dead early-clobber $eflags
     LCMPXCHG32 undef %67:gr64, 1, $noreg, 0, $noreg, %65, implicit-def dead $eax, implicit-def dead $eflags, implicit undef $eax :: (load store acquire monotonic (s32) on `ptr addrspace(1) undef`, addrspace 1)
     ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp
     $rdi = COPY %64
     CALL64pcrel32 target-flags(x86-plt) @wobble, csr_64, implicit $rsp, implicit $ssp, implicit killed $rdi, implicit-def $rsp, implicit-def $ssp
     ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp
     %71:gr64 = MOV64rm %46, 1, $noreg, 0, $noreg :: (dereferenceable load unordered (s64) from @global)
     %71:gr64 = NOT64r %71
     ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp

llvm/test/CodeGen/X86/x86-64-baseptr.ll

entry:
   %a = alloca i32, i32 %k, align 4
   store i32 0, ptr %a, align 4
   ret void
 }
 
 define void @clobber_base() #0 {
 ; CHECK-LABEL: clobber_base:
 ; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    leaq {{[0-9]+}}(%rsp), %r10
+; CHECK-NEXT:    .cfi_def_cfa %r10, 0
+; CHECK-NEXT:    andq $-128, %rsp
+; CHECK-NEXT:    pushq -16(%r10)
 ; CHECK-NEXT:    pushq %rbp
-; CHECK-NEXT:    .cfi_def_cfa_offset 16
-; CHECK-NEXT:    .cfi_offset %rbp, -16
 ; CHECK-NEXT:    movq %rsp, %rbp
-; CHECK-NEXT:    .cfi_def_cfa_register %rbp
+; CHECK-NEXT:    .cfi_escape 0x10, 0x06, 0x02, 0x76, 0x00 #
 ; CHECK-NEXT:    pushq %rbx
-; CHECK-NEXT:    andq $-128, %rsp
-; CHECK-NEXT:    subq $128, %rsp
-; CHECK-NEXT:    movq %rsp, %rbx
-; CHECK-NEXT:    .cfi_offset %rbx, -24
+; CHECK-NEXT:    subq $232, %rsp
+; CHECK-NEXT:    movq %r10, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
+; CHECK-NEXT:    .cfi_escape 0x10, 0x03, 0x02, 0x76, 0x78 #
+; CHECK-NEXT:    .cfi_escape 0x0f, 0x04, 0x76, 0x88, 0x7f, 0x06 #
 ; CHECK-NEXT:    callq helper@PLT
 ; CHECK-NEXT:    movq %rsp, %rcx
 ; CHECK-NEXT:    movl %eax, %eax
 ; CHECK-NEXT:    leaq 31(,%rax,4), %rax
 ; CHECK-NEXT:    andq $-32, %rax
 ; CHECK-NEXT:    movq %rcx, %rdx
 ; CHECK-NEXT:    subq %rax, %rdx
 ; CHECK-NEXT:    movq %rdx, %rsp
 ; CHECK-NEXT:    negq %rax
 ; CHECK-NEXT:    movl $405, %ebx # imm = 0x195
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:    movl $8, %edx
 ; CHECK-NEXT:    #APP
-; CHECK-NEXT:    movl %edx, (%rbx)
+; CHECK-NEXT:    movl %edx, -112(%rbp)
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:    movl $0, (%rcx,%rax)
+; CHECK-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %r10 # 8-byte Reload
 ; CHECK-NEXT:    leaq -8(%rbp), %rsp
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    popq %rbp
+; CHECK-NEXT:    leaq -16(%r10), %rsp
 ; CHECK-NEXT:    .cfi_def_cfa %rsp, 8
 ; CHECK-NEXT:    retq
 ;
 ; X32ABI-LABEL: clobber_base:
 ; X32ABI:       # %bb.0: # %entry
+; X32ABI-NEXT:    leaq {{[0-9]+}}(%esp), %r10
+; X32ABI-NEXT:    .cfi_def_cfa %r10, 0
+; X32ABI-NEXT:    andl $-128, %esp
+; X32ABI-NEXT:    pushq -16(%r10)
 ; X32ABI-NEXT:    pushq %rbp
-; X32ABI-NEXT:    .cfi_def_cfa_offset 16
-; X32ABI-NEXT:    .cfi_offset %rbp, -16
 ; X32ABI-NEXT:    movl %esp, %ebp
-; X32ABI-NEXT:    .cfi_def_cfa_register %rbp
+; X32ABI-NEXT:    .cfi_escape 0x10, 0x06, 0x02, 0x76, 0x00 #
 ; X32ABI-NEXT:    pushq %rbx
-; X32ABI-NEXT:    andl $-128, %esp
-; X32ABI-NEXT:    subl $128, %esp
-; X32ABI-NEXT:    movl %esp, %ebx
-; X32ABI-NEXT:    .cfi_offset %rbx, -24
+; X32ABI-NEXT:    subl $232, %esp
+; X32ABI-NEXT:    movq %r10, {{[-0-9]+}}(%e{{[sb]}}p) # 8-byte Spill
+; X32ABI-NEXT:    .cfi_escape 0x10, 0x03, 0x02, 0x76, 0x78 #
+; X32ABI-NEXT:    .cfi_escape 0x0f, 0x04, 0x76, 0x88, 0x7f, 0x06 #
 ; X32ABI-NEXT:    callq helper@PLT
 ; X32ABI-NEXT:    # kill: def $eax killed $eax def $rax
 ; X32ABI-NEXT:    leal 31(,%rax,4), %eax
 ; X32ABI-NEXT:    andl $-32, %eax
 ; X32ABI-NEXT:    movl %esp, %ecx
 ; X32ABI-NEXT:    movl %ecx, %edx
 ; X32ABI-NEXT:    subl %eax, %edx
 ; X32ABI-NEXT:    negl %eax
 ; X32ABI-NEXT:    movl %edx, %esp
 ; X32ABI-NEXT:    movl $405, %ebx # imm = 0x195
 ; X32ABI-NEXT:    #APP
 ; X32ABI-NEXT:    nop
 ; X32ABI-NEXT:    #NO_APP
 ; X32ABI-NEXT:    movl $8, %edx
 ; X32ABI-NEXT:    #APP
-; X32ABI-NEXT:    movl %edx, (%ebx)
+; X32ABI-NEXT:    movl %edx, -112(%ebp)
 ; X32ABI-NEXT:    #NO_APP
 ; X32ABI-NEXT:    movl $0, (%ecx,%eax)
+; X32ABI-NEXT:    movq {{[-0-9]+}}(%e{{[sb]}}p), %r10 # 8-byte Reload
 ; X32ABI-NEXT:    leal -8(%ebp), %esp
 ; X32ABI-NEXT:    popq %rbx
 ; X32ABI-NEXT:    popq %rbp
+; X32ABI-NEXT:    leaq -16(%r10), %rsp
 ; X32ABI-NEXT:    .cfi_def_cfa %rsp, 8
 ; X32ABI-NEXT:    retq
 entry:
   %k = call i32 @helper()
   %a = alloca i32, align 128
   %b = alloca i32, i32 %k, align 4
   ; clobber base pointer register
   tail call void asm sideeffect "nop", "{bx}"(i32 405)
   call void asm sideeffect "movl $0, $1", "r,*m"(i32 8, ptr elementtype(i32) %a)
   store i32 0, ptr %b, align 4
   ret void
 }
 
 define x86_regcallcc void @clobber_baseptr_argptr(i32 %param1, i32 %param2, i32 %param3, i32 %param4, i32 %param5, i32 %param6, i32 %param7, i32 %param8, i32 %param9, i32 %param10, i32 %param11, i32 %param12) #0 {
 ; CHECK-LABEL: clobber_baseptr_argptr:
 ; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    leaq {{[0-9]+}}(%rsp), %r10
+; CHECK-NEXT:    .cfi_def_cfa %r10, 0
+; CHECK-NEXT:    andq $-128, %rsp
+; CHECK-NEXT:    pushq -16(%r10)
 ; CHECK-NEXT:    pushq %rbp
-; CHECK-NEXT:    .cfi_def_cfa_offset 16
-; CHECK-NEXT:    .cfi_offset %rbp, -16
 ; CHECK-NEXT:    movq %rsp, %rbp
-; CHECK-NEXT:    .cfi_def_cfa_register %rbp
+; CHECK-NEXT:    .cfi_escape 0x10, 0x06, 0x02, 0x76, 0x00 #
 ; CHECK-NEXT:    pushq %rbx
-; CHECK-NEXT:    andq $-128, %rsp
-; CHECK-NEXT:    subq $256, %rsp # imm = 0x100
+; CHECK-NEXT:    subq $360, %rsp # imm = 0x168
 ; CHECK-NEXT:    movaps %xmm15, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
 ; CHECK-NEXT:    movaps %xmm14, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
 ; CHECK-NEXT:    movaps %xmm13, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
 ; CHECK-NEXT:    movaps %xmm12, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
 ; CHECK-NEXT:    movaps %xmm11, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
 ; CHECK-NEXT:    movaps %xmm10, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
 ; CHECK-NEXT:    movaps %xmm9, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
 ; CHECK-NEXT:    movaps %xmm8, {{[-0-9]+}}(%r{{[sb]}}p) # 16-byte Spill
-; CHECK-NEXT:    movq %rsp, %rbx
-; CHECK-NEXT:    .cfi_offset %rbx, -24
-; CHECK-NEXT:    .cfi_offset %xmm8, -160
-; CHECK-NEXT:    .cfi_offset %xmm9, -144
-; CHECK-NEXT:    .cfi_offset %xmm10, -128
-; CHECK-NEXT:    .cfi_offset %xmm11, -112
-; CHECK-NEXT:    .cfi_offset %xmm12, -96
-; CHECK-NEXT:    .cfi_offset %xmm13, -80
-; CHECK-NEXT:    .cfi_offset %xmm14, -64
-; CHECK-NEXT:    .cfi_offset %xmm15, -48
-; CHECK-NEXT:    movl 16(%rbp), %r14d
+; CHECK-NEXT:    movq %r10, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
+; CHECK-NEXT:    .cfi_escape 0x10, 0x03, 0x02, 0x76, 0x78 #
+; CHECK-NEXT:    .cfi_escape 0x10, 0x19, 0x02, 0x76, 0xf0, 0x7e #
+; CHECK-NEXT:    .cfi_escape 0x10, 0x1a, 0x02, 0x76, 0x80, 0x7f #
+; CHECK-NEXT:    .cfi_escape 0x10, 0x1b, 0x02, 0x76, 0x90, 0x7f #
+; CHECK-NEXT:    .cfi_escape 0x10, 0x1c, 0x02, 0x76, 0xa0, 0x7f #
+; CHECK-NEXT:    .cfi_escape 0x10, 0x1d, 0x02, 0x76, 0xb0, 0x7f #
+; CHECK-NEXT:    .cfi_escape 0x10, 0x1e, 0x02, 0x76, 0x40 #
+; CHECK-NEXT:    .cfi_escape 0x10, 0x1f, 0x02, 0x76, 0x50 #
+; CHECK-NEXT:    .cfi_escape 0x10, 0x20, 0x02, 0x76, 0x60 #
+; CHECK-NEXT:    .cfi_escape 0x0f, 0x04, 0x76, 0x88, 0x7e, 0x06 #
+; CHECK-NEXT:    movl (%r10), %r14d
 ; CHECK-NEXT:    callq helper@PLT
 ; CHECK-NEXT:    movq %rsp, %rcx
 ; CHECK-NEXT:    movl %eax, %eax
 ; CHECK-NEXT:    leaq 31(,%rax,4), %rax
 ; CHECK-NEXT:    andq $-32, %rax
 ; CHECK-NEXT:    movq %rcx, %rdx
 ; CHECK-NEXT:    subq %rax, %rdx
 ; CHECK-NEXT:    movq %rdx, %rsp
 ; CHECK-NEXT:    negq %rax
 ; CHECK-NEXT:    movl $405, %ebx # imm = 0x195
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:    #APP
 ; CHECK-NEXT:    nop
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:    movl $8, %edx
 ; CHECK-NEXT:    #APP
-; CHECK-NEXT:    movl %edx, (%rbx)
+; CHECK-NEXT:    movl %edx, -240(%rbp)
 ; CHECK-NEXT:    #NO_APP
 ; CHECK-NEXT:    movl %r14d, (%rcx,%rax)
 ; CHECK-NEXT:    movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm8 # 16-byte Reload
 ; CHECK-NEXT:    movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm9 # 16-byte Reload
 ; CHECK-NEXT:    movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm10 # 16-byte Reload
 ; CHECK-NEXT:    movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm11 # 16-byte Reload
 ; CHECK-NEXT:    movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm12 # 16-byte Reload
 ; CHECK-NEXT:    movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm13 # 16-byte Reload
 ; CHECK-NEXT:    movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm14 # 16-byte Reload
 ; CHECK-NEXT:    movaps {{[-0-9]+}}(%r{{[sb]}}p), %xmm15 # 16-byte Reload
+; CHECK-NEXT:    movq {{[-0-9]+}}(%r{{[sb]}}p), %r10 # 8-byte Reload
 ; CHECK-NEXT:    leaq -8(%rbp), %rsp
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    popq %rbp
+; CHECK-NEXT:    leaq -16(%r10), %rsp
 ; CHECK-NEXT:    .cfi_def_cfa %rsp, 8
 ; CHECK-NEXT:    retq
 ;
 ; X32ABI-LABEL: clobber_baseptr_argptr:
 ; X32ABI:       # %bb.0: # %entry
+; X32ABI-NEXT:    leaq {{[0-9]+}}(%esp), %r10
+; X32ABI-NEXT:    .cfi_def_cfa %r10, 0
+; X32ABI-NEXT:    andl $-128, %esp
+; X32ABI-NEXT:    pushq -16(%r10)
 ; X32ABI-NEXT:    pushq %rbp
-; X32ABI-NEXT:    .cfi_def_cfa_offset 16
-; X32ABI-NEXT:    .cfi_offset %rbp, -16
 ; X32ABI-NEXT:    movl %esp, %ebp
-; X32ABI-NEXT:    .cfi_def_cfa_register %rbp
+; X32ABI-NEXT:    .cfi_escape 0x10, 0x06, 0x02, 0x76, 0x00 #
 ; X32ABI-NEXT:    pushq %rbx
-; X32ABI-NEXT:    andl $-128, %esp
-; X32ABI-NEXT:    subl $256, %esp # imm = 0x100
+; X32ABI-NEXT:    subl $360, %esp # imm = 0x168
 ; X32ABI-NEXT:    movaps %xmm15, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
 ; X32ABI-NEXT:    movaps %xmm14, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
 ; X32ABI-NEXT:    movaps %xmm13, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
 ; X32ABI-NEXT:    movaps %xmm12, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
 ; X32ABI-NEXT:    movaps %xmm11, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
 ; X32ABI-NEXT:    movaps %xmm10, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
 ; X32ABI-NEXT:    movaps %xmm9, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
 ; X32ABI-NEXT:    movaps %xmm8, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
-; X32ABI-NEXT:    movl %esp, %ebx
-; X32ABI-NEXT:    .cfi_offset %rbx, -24
-; X32ABI-NEXT:    .cfi_offset %xmm8, -160
-; X32ABI-NEXT:    .cfi_offset %xmm9, -144
-; X32ABI-NEXT:    .cfi_offset %xmm10, -128
-; X32ABI-NEXT:    .cfi_offset %xmm11, -112
-; X32ABI-NEXT:    .cfi_offset %xmm12, -96
-; X32ABI-NEXT:    .cfi_offset %xmm13, -80
-; X32ABI-NEXT:    .cfi_offset %xmm14, -64
-; X32ABI-NEXT:    .cfi_offset %xmm15, -48
-; X32ABI-NEXT:    movl 16(%ebp), %r14d
+; X32ABI-NEXT:    movq %r10, {{[-0-9]+}}(%e{{[sb]}}p) # 8-byte Spill
+; X32ABI-NEXT:    .cfi_escape 0x10, 0x03, 0x02, 0x76, 0x78 #
+; X32ABI-NEXT:    .cfi_escape 0x10, 0x19, 0x02, 0x76, 0xf0, 0x7e #
+; X32ABI-NEXT:    .cfi_escape 0x10, 0x1a, 0x02, 0x76, 0x80, 0x7f #
+; X32ABI-NEXT:    .cfi_escape 0x10, 0x1b, 0x02, 0x76, 0x90, 0x7f #
+; X32ABI-NEXT:    .cfi_escape 0x10, 0x1c, 0x02, 0x76, 0xa0, 0x7f #
+; X32ABI-NEXT:    .cfi_escape 0x10, 0x1d, 0x02, 0x76, 0xb0, 0x7f #
+; X32ABI-NEXT:    .cfi_escape 0x10, 0x1e, 0x02, 0x76, 0x40 #
+; X32ABI-NEXT:    .cfi_escape 0x10, 0x1f, 0x02, 0x76, 0x50 #
+; X32ABI-NEXT:    .cfi_escape 0x10, 0x20, 0x02, 0x76, 0x60 #
+; X32ABI-NEXT:    .cfi_escape 0x0f, 0x04, 0x76, 0x88, 0x7e, 0x06 #
+; X32ABI-NEXT:    movl (%r10), %r14d
 ; X32ABI-NEXT:    callq helper@PLT
 ; X32ABI-NEXT:    # kill: def $eax killed $eax def $rax
 ; X32ABI-NEXT:    leal 31(,%rax,4), %eax
 ; X32ABI-NEXT:    andl $-32, %eax
 ; X32ABI-NEXT:    movl %esp, %ecx
 ; X32ABI-NEXT:    movl %ecx, %edx
 ; X32ABI-NEXT:    subl %eax, %edx
 ; X32ABI-NEXT:    negl %eax
 ; X32ABI-NEXT:    movl %edx, %esp
 ; X32ABI-NEXT:    movl $405, %ebx # imm = 0x195
 ; X32ABI-NEXT:    #APP
 ; X32ABI-NEXT:    nop
 ; X32ABI-NEXT:    #NO_APP
 ; X32ABI-NEXT:    #APP
 ; X32ABI-NEXT:    nop
 ; X32ABI-NEXT:    #NO_APP
 ; X32ABI-NEXT:    movl $8, %edx
 ; X32ABI-NEXT:    #APP
-; X32ABI-NEXT:    movl %edx, (%ebx)
+; X32ABI-NEXT:    movl %edx, -240(%ebp)
 ; X32ABI-NEXT:    #NO_APP
 ; X32ABI-NEXT:    movl %r14d, (%ecx,%eax)
 ; X32ABI-NEXT:    movaps {{[-0-9]+}}(%e{{[sb]}}p), %xmm8 # 16-byte Reload
 ; X32ABI-NEXT:    movaps {{[-0-9]+}}(%e{{[sb]}}p), %xmm9 # 16-byte Reload
 ; X32ABI-NEXT:    movaps {{[-0-9]+}}(%e{{[sb]}}p), %xmm10 # 16-byte Reload
 ; X32ABI-NEXT:    movaps {{[-0-9]+}}(%e{{[sb]}}p), %xmm11 # 16-byte Reload
 ; X32ABI-NEXT:    movaps {{[-0-9]+}}(%e{{[sb]}}p), %xmm12 # 16-byte Reload
 ; X32ABI-NEXT:    movaps {{[-0-9]+}}(%e{{[sb]}}p), %xmm13 # 16-byte Reload
 ; X32ABI-NEXT:    movaps {{[-0-9]+}}(%e{{[sb]}}p), %xmm14 # 16-byte Reload
 ; X32ABI-NEXT:    movaps {{[-0-9]+}}(%e{{[sb]}}p), %xmm15 # 16-byte Reload
+; X32ABI-NEXT:    movq {{[-0-9]+}}(%e{{[sb]}}p), %r10 # 8-byte Reload
 ; X32ABI-NEXT:    leal -8(%ebp), %esp
 ; X32ABI-NEXT:    popq %rbx
 ; X32ABI-NEXT:    popq %rbp
+; X32ABI-NEXT:    leaq -16(%r10), %rsp
 ; X32ABI-NEXT:    .cfi_def_cfa %rsp, 8
 ; X32ABI-NEXT:    retq
 entry:
   %k = call i32 @helper()
   %a = alloca i32, align 128
   %b = alloca i32, i32 %k, align 4
   ; clobber base pointer register
   tail call void asm sideeffect "nop", "{bx}"(i32 405)
   ; clobber argument pointer register
   tail call void asm sideeffect "nop", "~{bx},~{r10},~{r11}"()
   call void asm sideeffect "movl $0, $1", "r,*m"(i32 8, ptr elementtype(i32) %a)
   store i32 %param12, ptr %b, align 4
   ret void
 }
 
-attributes #0 = { "frame-pointer"="all"}
+attributes #0 = {"frame-pointer"="all"}
 !llvm.module.flags = !{!0}
 !0 = !{i32 2, !"override-stack-alignment", i32 32}