[MachineCopyPropagation] Extend pass to do COPY source forwarding

[gberry]

Summary

This change extends MachineCopyPropagation to do COPY source forwarding
and adds an additional run of the pass to the default pass pipeline just
after register allocation.

This version of this patch uses the newly added
MachineOperand::isRenamable bit to avoid forwarding registers is such a
way as to violate constraints that aren't captured in the
Machine IR (e.g. ABI or ISA constraints).

The AMDGPU target is modified by this change to mark more opcodes as
hasExtraSrcRegAllocReq so that their operands will be marked as not
renamable, to avoid copy forwarding violating the constraint that only
one operand may use the constant bus.

This change is a continuation of the work started in D30751.

[gberry]

@tstellar could you specifically take a look at the AMDGPU backend changes?

[mcrosier]

lib/CodeGen/MachineCopyPropagation.cpp

20 ↗

(On Diff #128974)

veg1 -> reg1

[gberry]

Ping?

[qcolombet]

Hi Geoff,

Looks mostly good for the generic parts.

Comments below.

Cheers,
-Quentin

lib/CodeGen/MachineCopyPropagation.cpp

249 ↗

(On Diff #128974)

You mean the source of Copy, right?

251 ↗

(On Diff #128974)

It looks like this function is meant to run only on allocated code.
If that's the case, say it in the comment.

260 ↗

(On Diff #128974)

Unless you expect CopySrcReg to always be a physical register, this check needs to be adapted.

262 ↗

(On Diff #128974)

Nit: to reduce indentation
if (!UseI.isCopy())

return false;

288 ↗

(On Diff #128974)

Just personal taste but I feel like a for loop would be more natural here.
for (const TargetRegisterClass *SuperRC = UseDstRC, TargetRegisterClass::sc_iterator SuperRCI = UseDstRC->getSuperClasses(), SuperRC; SuperRC = *SuperRCI++)

if (SuperRC->contains(CopySrcReg))
return true;

(Technically, I admit this has one more check, but I believe it doesn't matter for performance.)

346 ↗

(On Diff #128974)

I expect reserved registers shouldn't be renamable.
Am I missing something?

edit: Oh you may introduce this after forwarding a reserved register, if I am reading the code correctly. Then, unless I am missing something again, we should be fine rewriting this one as well.

360 ↗

(On Diff #128974)

Add a DEBUG message (and maybe an optimization remarks) for that case to see how often it happens and thus, if it is worth fixing.

Could be a follow-up patch.

373 ↗

(On Diff #128974)

Add a debug print saying we're skipping because of debug counter.

386 ↗

(On Diff #128974)

Hmm, I have mixed feeling about this.
Is renamable a property of the register or of the machine operand?

If this is the machine operand, then this particular use of the register will still be renamable.

What I am saying is depending on the semantic we give to renamable, this change makes sense or not.

[gberry]

lib/CodeGen/MachineCopyPropagation.cpp

251 ↗

(On Diff #128974)

This pass is meant to only run on allocated code. There is already a NoVRegs pass property check and an assert in CopyPropagateBlock that checks for virtual registers. I'll add 'physical' to the comment.

260 ↗

(On Diff #128974)

As noted above, I do expect it to always be a physical register. I can add asserts here, or perhaps rename the function to reduce confusion?

346 ↗

(On Diff #128974)

I believe this is a redundant check with the isRenamable that is there from the previous version. I'll confirm by doing some testing with this changed to an assert before removing it.

edit: After trying this out, it appears that there are a few cases of targets generating renamable reserved registers. One specific case is X86 and float-stack registers. I'll try adding this check to the verifier to see if we can flush out all these cases before committing this change.

386 ↗

(On Diff #128974)

If renamability is only a property of the opcode, then I don't think we need it, since we could always just check the opcode property.
My thinking is that it applies more to the individual physical register, and implies the value needs to be in that particular register, whether due to opcode constraints, ABI constraints, inline asm constrains, GC-interaction constraints, etc.

[qcolombet]

lib/CodeGen/MachineCopyPropagation.cpp

251 ↗

(On Diff #128974)

I thought we were running before the rewriter, but I guess we don't need that anymore since we have the renamable flag.
Thanks for pointing that out.

260 ↗

(On Diff #128974)

No given the pass itself doesn't expect vreg, we're fine.

346 ↗

(On Diff #128974)

Sounds good

386 ↗

(On Diff #128974)

In that case given we're really renaming the operand, I would think it should stay this way, but again, we could argue in both ways.

We should just agree what we're modeling with that flag and verify that in the machine verifier.

[gberry]

Updates based on Quentin's comments

[gberry]

I've uploaded a new version that addresses most of the issues brought up by @qcolombet.

@arsenm could you take a look at the AMDGPU part of this change?

lib/CodeGen/MachineCopyPropagation.cpp

386 ↗

(On Diff #128974)

I'm not sure exactly what you're suggesting we add to the machine verifier here.

[gberry]

D42449 adds the machine verifier check that no renamable operands are assigned to reserved registers

[nhaehnle]

The changes to AMDGPU tests and .td files look good to me.

It would be nice for the backend to have a hook that can be queried when renamable is false (but obviously this current patch doesn't have to introduce that).

[qcolombet]

LGTM

llvm/trunk/lib/CodeGen/MachineCopyPropagation.cpp

 //===- MachineCopyPropagation.cpp - Machine Copy Propagation Pass ---------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This is an extremely simple MachineInstr-level copy propagation pass.
 //
+// This pass forwards the source of COPYs to the users of their destinations
+// when doing so is legal.  For example:
+//
+//   %reg1 = COPY %reg0
+//   ...
+//   ... = OP %reg1
+//
+// If
+//   - %reg0 has not been clobbered by the time of the use of %reg1
+//   - the register class constraints are satisfied
+//   - the COPY def is the only value that reaches OP
+// then this pass replaces the above with:
+//
+//   %reg1 = COPY %reg0
+//   ...
+//   ... = OP %reg0
+//
+// This pass also removes some redundant COPYs.  For example:
+//
+//    %R1 = COPY %R0
+//    ... // No clobber of %R1
+//    %R0 = COPY %R1 <<< Removed
+//
+// or
+//
+//    %R1 = COPY %R0
+//    ... // No clobber of %R0
+//    %R1 = COPY %R0 <<< Removed
+//
 //===----------------------------------------------------------------------===//
 
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/ADT/SetVector.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/ADT/iterator_range.h"
 #include "llvm/CodeGen/MachineBasicBlock.h"
 #include "llvm/CodeGen/MachineFunction.h"
 #include "llvm/CodeGen/MachineFunctionPass.h"
 #include "llvm/CodeGen/MachineInstr.h"
 #include "llvm/CodeGen/MachineOperand.h"
 #include "llvm/CodeGen/MachineRegisterInfo.h"
+#include "llvm/CodeGen/TargetInstrInfo.h"
 #include "llvm/CodeGen/TargetRegisterInfo.h"
 #include "llvm/CodeGen/TargetSubtargetInfo.h"
 #include "llvm/MC/MCRegisterInfo.h"
 #include "llvm/Pass.h"
 #include "llvm/Support/Debug.h"
+#include "llvm/Support/DebugCounter.h"
 #include "llvm/Support/raw_ostream.h"
 #include <cassert>
 #include <iterator>
 
 using namespace llvm;
 
 #define DEBUG_TYPE "machine-cp"
 
 STATISTIC(NumDeletes, "Number of dead copies deleted");
+STATISTIC(NumCopyForwards, "Number of copy uses forwarded");
+DEBUG_COUNTER(FwdCounter, "machine-cp-fwd",
+              "Controls which register COPYs are forwarded");
 
 namespace {
 
 using RegList = SmallVector<unsigned, 4>;
 using SourceMap = DenseMap<unsigned, RegList>;
 using Reg2MIMap = DenseMap<unsigned, MachineInstr *>;
 
   class MachineCopyPropagation : public MachineFunctionPass {
MachineFunctionProperties getRequiredProperties() const override {
           MachineFunctionProperties::Property::NoVRegs);
     }
 
   private:
     void ClobberRegister(unsigned Reg);
     void ReadRegister(unsigned Reg);
     void CopyPropagateBlock(MachineBasicBlock &MBB);
     bool eraseIfRedundant(MachineInstr &Copy, unsigned Src, unsigned Def);
+    void forwardUses(MachineInstr &MI);
+    bool isForwardableRegClassCopy(const MachineInstr &Copy,
+                                   const MachineInstr &UseI, unsigned UseIdx);
+    bool hasImplicitOverlap(const MachineInstr &MI, const MachineOperand &Use);
 
     /// Candidates for deletion.
     SmallSetVector<MachineInstr*, 8> MaybeDeadCopies;
 
     /// Def -> available copies map.
     Reg2MIMap AvailCopyMap;
 
     /// Def -> copies map.
for (MachineInstr &MI :
     MI.clearRegisterKills(CopyDef, TRI);
 
   Copy.eraseFromParent();
   Changed = true;
   ++NumDeletes;
   return true;
 }
 
+/// Decide whether we should forward the source of \param Copy to its use in
+/// \param UseI based on the physical register class constraints of the opcode
+/// and avoiding introducing more cross-class COPYs.
+bool MachineCopyPropagation::isForwardableRegClassCopy(const MachineInstr &Copy,
+                                                       const MachineInstr &UseI,
+                                                       unsigned UseIdx) {
+
+  unsigned CopySrcReg = Copy.getOperand(1).getReg();
+
+  // If the new register meets the opcode register constraints, then allow
+  // forwarding.
+  if (const TargetRegisterClass *URC =
+          UseI.getRegClassConstraint(UseIdx, TII, TRI))
+    return URC->contains(CopySrcReg);
+
+  if (!UseI.isCopy())
+    return false;
+
+  /// COPYs don't have register class constraints, so if the user instruction
+  /// is a COPY, we just try to avoid introducing additional cross-class
+  /// COPYs.  For example:
+  ///
+  ///   RegClassA = COPY RegClassB  // Copy parameter
+  ///   ...
+  ///   RegClassB = COPY RegClassA  // UseI parameter
+  ///
+  /// which after forwarding becomes
+  ///
+  ///   RegClassA = COPY RegClassB
+  ///   ...
+  ///   RegClassB = COPY RegClassB
+  ///
+  /// so we have reduced the number of cross-class COPYs and potentially
+  /// introduced a nop COPY that can be removed.
+  const TargetRegisterClass *UseDstRC =
+      TRI->getMinimalPhysRegClass(UseI.getOperand(0).getReg());
+
+  const TargetRegisterClass *SuperRC = UseDstRC;
+  for (TargetRegisterClass::sc_iterator SuperRCI = UseDstRC->getSuperClasses();
+       SuperRC; SuperRC = *SuperRCI++)
+    if (SuperRC->contains(CopySrcReg))
+      return true;
+
+  return false;
+}
+
+/// Check that \p MI does not have implicit uses that overlap with it's \p Use
+/// operand (the register being replaced), since these can sometimes be
+/// implicitly tied to other operands.  For example, on AMDGPU:
+///
+/// V_MOVRELS_B32_e32 %VGPR2, %M0<imp-use>, %EXEC<imp-use>, %VGPR2_VGPR3_VGPR4_VGPR5<imp-use>
+///
+/// the %VGPR2 is implicitly tied to the larger reg operand, but we have no
+/// way of knowing we need to update the latter when updating the former.
+bool MachineCopyPropagation::hasImplicitOverlap(const MachineInstr &MI,
+                                                const MachineOperand &Use) {
+  for (const MachineOperand &MIUse : MI.uses())
+    if (&MIUse != &Use && MIUse.isReg() && MIUse.isImplicit() &&
+        MIUse.isUse() && TRI->regsOverlap(Use.getReg(), MIUse.getReg()))
+      return true;
+
+  return false;
+}
+
+/// Look for available copies whose destination register is used by \p MI and
+/// replace the use in \p MI with the copy's source register.
+void MachineCopyPropagation::forwardUses(MachineInstr &MI) {
+  if (AvailCopyMap.empty())
+    return;
+
+  // Look for non-tied explicit vreg uses that have an active COPY
+  // instruction that defines the physical register allocated to them.
+  // Replace the vreg with the source of the active COPY.
+  for (unsigned OpIdx = 0, OpEnd = MI.getNumOperands(); OpIdx < OpEnd;
+       ++OpIdx) {
+    MachineOperand &MOUse = MI.getOperand(OpIdx);
+    // Don't forward into undef use operands since doing so can cause problems
+    // with the machine verifier, since it doesn't treat undef reads as reads,
+    // so we can end up with a live range that ends on an undef read, leading to
+    // an error that the live range doesn't end on a read of the live range
+    // register.
+    if (!MOUse.isReg() || MOUse.isTied() || MOUse.isUndef() || MOUse.isDef() ||
+        MOUse.isImplicit())
+      continue;
+
+    if (!MOUse.getReg())
+      continue;
+
+    // Check that the register is marked 'renamable' so we know it is safe to
+    // rename it without violating any constraints that aren't expressed in the
+    // IR (e.g. ABI or opcode requirements).
+    if (!MOUse.isRenamable())
+      continue;
+
+    auto CI = AvailCopyMap.find(MOUse.getReg());
+    if (CI == AvailCopyMap.end())
+      continue;
+
+    MachineInstr &Copy = *CI->second;
+    unsigned CopyDstReg = Copy.getOperand(0).getReg();
+    const MachineOperand &CopySrc = Copy.getOperand(1);
+    unsigned CopySrcReg = CopySrc.getReg();
+
+    // FIXME: Don't handle partial uses of wider COPYs yet.
+    if (MOUse.getReg() != CopyDstReg) {
+      DEBUG(dbgs() << "MCP: FIXME! Not forwarding COPY to sub-register use:\n  "
+                   << MI);
+      continue;
+    }
+
+    // Don't forward COPYs of reserved regs unless they are constant.
+    if (MRI->isReserved(CopySrcReg) && !MRI->isConstantPhysReg(CopySrcReg))
+      continue;
+
+    if (!isForwardableRegClassCopy(Copy, MI, OpIdx))
+      continue;
+
+    if (hasImplicitOverlap(MI, MOUse))
+      continue;
+
+    if (!DebugCounter::shouldExecute(FwdCounter)) {
+      DEBUG(dbgs() << "MCP: Skipping forwarding due to debug counter:\n  "
+                   << MI);
+      continue;
+    }
+
+    DEBUG(dbgs() << "MCP: Replacing " << printReg(MOUse.getReg(), TRI)
+                 << "\n     with " << printReg(CopySrcReg, TRI) << "\n     in "
+                 << MI << "     from " << Copy);
+
+    MOUse.setReg(CopySrcReg);
+    if (!CopySrc.isRenamable())
+      MOUse.setIsRenamable(false);
+
+    DEBUG(dbgs() << "MCP: After replacement: " << MI << "\n");
+
+    // Clear kill markers that may have been invalidated.
+    for (MachineInstr &KMI :
+         make_range(Copy.getIterator(), std::next(MI.getIterator())))
+      KMI.clearRegisterKills(CopySrcReg, TRI);
+
+    ++NumCopyForwards;
+    Changed = true;
+  }
+}
+
 void MachineCopyPropagation::CopyPropagateBlock(MachineBasicBlock &MBB) {
   DEBUG(dbgs() << "MCP: CopyPropagateBlock " << MBB.getName() << "\n");
 
   for (MachineBasicBlock::iterator I = MBB.begin(), E = MBB.end(); I != E; ) {
     MachineInstr *MI = &*I;
     ++I;
 
     if (MI->isCopy()) {
if (MI->isCopy()) {
       //  %ecx = COPY %eax
       //  ... nothing clobbered eax.
       //  %ecx = COPY %eax
       // =>
       //  %ecx = COPY %eax
       if (eraseIfRedundant(*MI, Def, Src) || eraseIfRedundant(*MI, Src, Def))
         continue;
 
+      forwardUses(*MI);
+
+      // Src may have been changed by forwardUses()
+      Src = MI->getOperand(1).getReg();
+
       // If Src is defined by a previous copy, the previous copy cannot be
       // eliminated.
       ReadRegister(Src);
       for (const MachineOperand &MO : MI->implicit_operands()) {
         if (!MO.isReg() || !MO.readsReg())
           continue;
         unsigned Reg = MO.getReg();
         if (!Reg)
if (MI->isCopy()) {
       // it's no longer available for copy propagation.
       RegList &DestList = SrcMap[Src];
       if (!is_contained(DestList, Def))
           DestList.push_back(Def);
 
       continue;
     }
 
+    // Clobber any earlyclobber regs first.
+    for (const MachineOperand &MO : MI->operands())
+      if (MO.isReg() && MO.isEarlyClobber()) {
+        unsigned Reg = MO.getReg();
+        // If we have a tied earlyclobber, that means it is also read by this
+        // instruction, so we need to make sure we don't remove it as dead
+        // later.
+        if (MO.isTied())
+          ReadRegister(Reg);
+        ClobberRegister(Reg);
+      }
+
+    forwardUses(*MI);
+
     // Not a copy.
     SmallVector<unsigned, 2> Defs;
     const MachineOperand *RegMask = nullptr;
     for (const MachineOperand &MO : MI->operands()) {
       if (MO.isRegMask())
         RegMask = &MO;
       if (!MO.isReg())
         continue;
       unsigned Reg = MO.getReg();
       if (!Reg)
         continue;
 
       assert(!TargetRegisterInfo::isVirtualRegister(Reg) &&
              "MachineCopyPropagation should be run after register allocation!");
 
-      if (MO.isDef()) {
+      if (MO.isDef() && !MO.isEarlyClobber()) {
         Defs.push_back(Reg);
         continue;
       } else if (MO.readsReg())
         ReadRegister(Reg);
     }
 
     // The instruction has a register mask operand which means that it clobbers
     // a large set of registers.  Treat clobbered registers the same way as
for (unsigned Reg : Defs)
       ClobberRegister(Reg);
   }
 
   // If MBB doesn't have successors, delete the copies whose defs are not used.
   // If MBB does have successors, then conservative assume the defs are live-out
   // since we don't want to trust live-in lists.
   if (MBB.succ_empty()) {
     for (MachineInstr *MaybeDead : MaybeDeadCopies) {
+      DEBUG(dbgs() << "MCP: Removing copy due to no live-out succ: ";
+            MaybeDead->dump());
       assert(!MRI->isReserved(MaybeDead->getOperand(0).getReg()));
       MaybeDead->eraseFromParent();
       Changed = true;
       ++NumDeletes;
     }
   }
 
   MaybeDeadCopies.clear();

llvm/trunk/lib/CodeGen/TargetPassConfig.cpp

if (RegAllocPass) {
     addPass(&VirtRegRewriterID);
 
     // Perform stack slot coloring and post-ra machine LICM.
     //
     // FIXME: Re-enable coloring with register when it's capable of adding
     // kill markers.
     addPass(&StackSlotColoringID);
 
+    // Copy propagate to forward register uses and try to eliminate COPYs that
+    // were not coalesced.
+    addPass(&MachineCopyPropagationID);
+
     // Run post-ra machine LICM to hoist reloads / remats.
     //
     // FIXME: can this move into MachineLateOptimization?
     addPass(&MachineLICMID);
   }
 }
 
 //===---------------------------------------------------------------------===//

llvm/trunk/test/CodeGen/AArch64/aarch64-fold-lslfast.ll

 ; RUN: llc < %s -mtriple=aarch64-linux-gnu -mattr=+lsl-fast | FileCheck %s
 
 %struct.a = type [256 x i16]
 %struct.b = type [256 x i32]
 %struct.c = type [256 x i64]
 
 declare void @foo()
 define i16 @halfword(%struct.a* %ctx, i32 %xor72) nounwind {
 ; CHECK-LABEL: halfword:
 ; CHECK: ubfx [[REG:x[0-9]+]], x1, #9, #8
 ; CHECK: ldrh [[REG1:w[0-9]+]], [{{.*}}[[REG2:x[0-9]+]], [[REG]], lsl #1]
-; CHECK: strh [[REG1]], [{{.*}}[[REG2]], [[REG]], lsl #1]
+; CHECK: mov [[REG3:x[0-9]+]], [[REG2]]
+; CHECK: strh [[REG1]], [{{.*}}[[REG3]], [[REG]], lsl #1]
   %shr81 = lshr i32 %xor72, 9
   %conv82 = zext i32 %shr81 to i64
   %idxprom83 = and i64 %conv82, 255
   %arrayidx86 = getelementptr inbounds %struct.a, %struct.a* %ctx, i64 0, i64 %idxprom83
   %result = load i16, i16* %arrayidx86, align 2
   call void @foo()
   store i16 %result, i16* %arrayidx86, align 2
   ret i16 %result
 }
 
 define i32 @word(%struct.b* %ctx, i32 %xor72) nounwind {
 ; CHECK-LABEL: word:
 ; CHECK: ubfx [[REG:x[0-9]+]], x1, #9, #8
 ; CHECK: ldr [[REG1:w[0-9]+]], [{{.*}}[[REG2:x[0-9]+]], [[REG]], lsl #2]
-; CHECK: str [[REG1]], [{{.*}}[[REG2]], [[REG]], lsl #2]
+; CHECK: mov [[REG3:x[0-9]+]], [[REG2]]
+; CHECK: str [[REG1]], [{{.*}}[[REG3]], [[REG]], lsl #2]
   %shr81 = lshr i32 %xor72, 9
   %conv82 = zext i32 %shr81 to i64
   %idxprom83 = and i64 %conv82, 255
   %arrayidx86 = getelementptr inbounds %struct.b, %struct.b* %ctx, i64 0, i64 %idxprom83
   %result = load i32, i32* %arrayidx86, align 4
   call void @foo()
   store i32 %result, i32* %arrayidx86, align 4
   ret i32 %result
 }
 
 define i64 @doubleword(%struct.c* %ctx, i32 %xor72) nounwind {
 ; CHECK-LABEL: doubleword:
 ; CHECK: ubfx [[REG:x[0-9]+]], x1, #9, #8
 ; CHECK: ldr [[REG1:x[0-9]+]], [{{.*}}[[REG2:x[0-9]+]], [[REG]], lsl #3]
-; CHECK: str [[REG1]], [{{.*}}[[REG2]], [[REG]], lsl #3]
+; CHECK: mov [[REG3:x[0-9]+]], [[REG2]]
+; CHECK: str [[REG1]], [{{.*}}[[REG3]], [[REG]], lsl #3]
   %shr81 = lshr i32 %xor72, 9
   %conv82 = zext i32 %shr81 to i64
   %idxprom83 = and i64 %conv82, 255
   %arrayidx86 = getelementptr inbounds %struct.c, %struct.c* %ctx, i64 0, i64 %idxprom83
   %result = load i64, i64* %arrayidx86, align 8
   call void @foo()
   store i64 %result, i64* %arrayidx86, align 8
   ret i64 %result

llvm/trunk/test/CodeGen/AArch64/arm64-AdvSIMD-Scalar.ll

 ; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-eabi -aarch64-neon-syntax=apple -aarch64-enable-simd-scalar=true -asm-verbose=false -disable-adv-copy-opt=true | FileCheck %s -check-prefix=CHECK -check-prefix=CHECK-NOOPT
 ; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-eabi -aarch64-neon-syntax=apple -aarch64-enable-simd-scalar=true -asm-verbose=false -disable-adv-copy-opt=false | FileCheck %s -check-prefix=CHECK -check-prefix=CHECK-OPT
 ; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-eabi -aarch64-neon-syntax=generic -aarch64-enable-simd-scalar=true -asm-verbose=false -disable-adv-copy-opt=true | FileCheck %s -check-prefix=GENERIC -check-prefix=GENERIC-NOOPT
 ; RUN: llc < %s -verify-machineinstrs -mtriple=arm64-eabi -aarch64-neon-syntax=generic -aarch64-enable-simd-scalar=true -asm-verbose=false -disable-adv-copy-opt=false | FileCheck %s -check-prefix=GENERIC -check-prefix=GENERIC-OPT
 
 define <2 x i64> @bar(<2 x i64> %a, <2 x i64> %b) nounwind readnone {
 ; CHECK-LABEL: bar:
 ; CHECK: add.2d v[[REG:[0-9]+]], v0, v1
 ; CHECK: add  d[[REG3:[0-9]+]], d[[REG]], d1
 ; CHECK: sub  d[[REG2:[0-9]+]], d[[REG]], d1
-; Without advanced copy optimization, we end up with cross register
-; banks copies that cannot be coalesced.
-; CHECK-NOOPT: fmov [[COPY_REG3:x[0-9]+]], d[[REG3]]
-; With advanced copy optimization, we end up with just one copy
-; to insert the computed high part into the V register. 
-; CHECK-OPT-NOT: fmov
+; CHECK-NOT: fmov
 ; CHECK: fmov [[COPY_REG2:x[0-9]+]], d[[REG2]]
-; CHECK-NOOPT: fmov d0, [[COPY_REG3]]
-; CHECK-OPT-NOT: fmov
+; CHECK-NOT: fmov
 ; CHECK: mov.d v0[1], [[COPY_REG2]]
 ; CHECK-NEXT: ret
 ;
 ; GENERIC-LABEL: bar:
 ; GENERIC: add  v[[REG:[0-9]+]].2d, v0.2d, v1.2d
 ; GENERIC: add  d[[REG3:[0-9]+]], d[[REG]], d1
 ; GENERIC: sub  d[[REG2:[0-9]+]], d[[REG]], d1
-; GENERIC-NOOPT: fmov [[COPY_REG3:x[0-9]+]], d[[REG3]]
-; GENERIC-OPT-NOT: fmov
+; GENERIC-NOT: fmov
 ; GENERIC: fmov [[COPY_REG2:x[0-9]+]], d[[REG2]]
-; GENERIC-NOOPT: fmov d0, [[COPY_REG3]]
-; GENERIC-OPT-NOT: fmov
+; GENERIC-NOT: fmov
 ; GENERIC: mov v0.d[1], [[COPY_REG2]]
 ; GENERIC-NEXT: ret
   %add = add <2 x i64> %a, %b
   %vgetq_lane = extractelement <2 x i64> %add, i32 0
   %vgetq_lane2 = extractelement <2 x i64> %b, i32 0
   %add3 = add i64 %vgetq_lane, %vgetq_lane2
   %sub = sub i64 %vgetq_lane, %vgetq_lane2
   %vecinit = insertelement <2 x i64> undef, i64 %add3, i32 0

llvm/trunk/test/CodeGen/AArch64/arm64-zero-cycle-regmov.ll

 ; RUN: llc -mtriple=arm64-apple-ios -mcpu=cyclone < %s | FileCheck %s
 ; rdar://12254953
 
 define i32 @t(i32 %a, i32 %b, i32 %c, i32 %d) nounwind ssp {
 entry:
 ; CHECK-LABEL: t:
-; CHECK: mov x0, [[REG1:x[0-9]+]]
-; CHECK: mov x1, [[REG2:x[0-9]+]]
+; CHECK: mov [[REG2:x[0-9]+]], x3
+; CHECK: mov [[REG1:x[0-9]+]], x2
+; CHECK: mov x0, x2
+; CHECK: mov x1, x3
 ; CHECK: bl _foo
 ; CHECK: mov x0, [[REG1]]
 ; CHECK: mov x1, [[REG2]]
   %call = call i32 @foo(i32 %c, i32 %d) nounwind
   %call1 = call i32 @foo(i32 %c, i32 %d) nounwind
   unreachable
 }
 
 declare i32 @foo(i32, i32)

llvm/trunk/test/CodeGen/AArch64/cmpxchg-idioms.ll

 ; CHECK-NOT: cmp {{w[0-9]+}}, {{w[0-9]+}}
   ; FIXME: DAG combine should be able to deal with this.
 ; CHECK: orr [[TMP:w[0-9]+]], wzr, #0x1
 ; CHECK: eor w0, [[TMP]], #0x1
 ; CHECK: ret
 
 ; CHECK: [[FAILED]]:
 ; CHECK-NOT: cmp {{w[0-9]+}}, {{w[0-9]+}}
-; CHECK: mov [[TMP:w[0-9]+]], wzr
-; CHECK: eor w0, [[TMP]], #0x1
+; CHECK: eor w0, wzr, #0x1
 ; CHECK: ret
 
   %pair = cmpxchg i8* %value, i8 %oldValue, i8 %newValue acq_rel monotonic
   %success = extractvalue { i8, i1 } %pair, 1
   %failure = xor i1 %success, 1
   ret i1 %failure
 }
 

llvm/trunk/test/CodeGen/AArch64/copyprop.mir

+# RUN: llc -mtriple=aarch64-linux-gnu -run-pass machine-cp -o - %s | FileCheck %s
+# Tests for MachineCopyPropagation copy forwarding.
+---
+# Simple forwarding.
+# CHECK-LABEL: name: test1
+# CHECK: $x0 = SUBXri $x0, 1, 0
+name:            test1
+tracksRegLiveness: true
+body: |
+  bb.0:
+    liveins: $x0
+    renamable $x1 = COPY $x0
+    $x0 = SUBXri renamable $x1, 1, 0
+...
+---
+# Don't forward if not renamable.
+# CHECK-LABEL: name: test2
+# CHECK: $x0 = SUBXri $x1, 1, 0
+name:            test2
+tracksRegLiveness: true
+body: |
+  bb.0:
+    liveins: $x0
+    $x1 = COPY $x0
+    $x0 = SUBXri $x1, 1, 0
+...
+---
+# Don't forward reserved non-constant reg values.
+# CHECK-LABEL: name: test4
+# CHECK: $x0 = SUBXri renamable $x1, 1, 0
+name:            test4
+tracksRegLiveness: true
+body: |
+  bb.0:
+    liveins: $x0
+    $sp = SUBXri $sp, 16, 0
+    renamable $x1 = COPY $sp
+    $x0 = SUBXri renamable $x1, 1, 0
+    $sp = ADDXri $sp, 16, 0
+...
+---
+# Don't violate opcode constraints when forwarding.
+# CHECK-LABEL: name: test5
+# CHECK: $x0 = SUBXri renamable $x1, 1, 0
+name:            test5
+tracksRegLiveness: true
+body: |
+  bb.0:
+    liveins: $x0
+    renamable $x1 = COPY $xzr
+    $x0 = SUBXri renamable $x1, 1, 0
+...
+---
+# Test cross-class COPY forwarding.
+# CHECK-LABEL: name: test6
+# CHECK: $x2 = COPY $x0
+name:            test6
+tracksRegLiveness: true
+body: |
+  bb.0:
+    liveins: $x0
+    renamable $d1 = COPY $x0
+    $x2 = COPY renamable $d1
+    RET_ReallyLR implicit $x2
+...
+---
+# Don't forward if there are overlapping implicit operands.
+# CHECK-LABEL: name: test7
+# CHECK: $w0 = SUBWri killed renamable $w1, 1, 0
+name:            test7
+tracksRegLiveness: true
+body: |
+  bb.0:
+    liveins: $w0
+    renamable $w1 = COPY $w0
+    $w0 = SUBWri killed renamable $w1, 1, 0, implicit killed $x1
+...
+---
+# Check that kill flags are cleared.
+# CHECK-LABEL: name: test8
+# CHECK: $x2 = ADDXri $x0, 1, 0
+# CHECK: $x0 = SUBXri $x0, 1, 0
+name:            test8
+tracksRegLiveness: true
+body: |
+  bb.0:
+    liveins: $x0
+    renamable $x1 = COPY $x0
+    $x2 = ADDXri killed $x0, 1, 0
+    $x0 = SUBXri renamable $x1, 1, 0
+...
+---
+# Don't forward if value is clobbered.
+# CHECK-LABEL: name: test9
+# CHECK: $x2 = SUBXri renamable $x1, 1, 0
+name:            test9
+tracksRegLiveness: true
+body: |
+  bb.0:
+    liveins: $x0
+    renamable $x1 = COPY $x0
+    $x0 = ADDXri $x0, 1, 0
+    $x2 = SUBXri renamable $x1, 1, 0
+...

llvm/trunk/test/CodeGen/AArch64/f16-instructions.ll

then:
   ret void
 else:
   store i32 0, i32* %p2
   ret void
 }
 
 ; CHECK-COMMON-LABEL: test_phi:
 ; CHECK-COMMON: mov  x[[PTR:[0-9]+]], x0
-; CHECK-COMMON: ldr  h[[AB:[0-9]+]], [x[[PTR]]]
+; CHECK-COMMON: ldr  h[[AB:[0-9]+]], [x0]
 ; CHECK-COMMON: [[LOOP:LBB[0-9_]+]]:
 ; CHECK-COMMON: mov.16b  v[[R:[0-9]+]], v[[AB]]
 ; CHECK-COMMON: ldr  h[[AB]], [x[[PTR]]]
 ; CHECK-COMMON: mov  x0, x[[PTR]]
 ; CHECK-COMMON: bl {{_?}}test_dummy
 ; CHECK-COMMON: mov.16b  v0, v[[R]]
 ; CHECK-COMMON: ret
 define half @test_phi(half* %p1) #0 {

llvm/trunk/test/CodeGen/AArch64/flags-multiuse.ll

 ; CHECK-LABEL: test_multiflag:
 
   %test = icmp ne i32 %n, %m
 ; CHECK: cmp [[LHS:w[0-9]+]], [[RHS:w[0-9]+]]
 
   %val = zext i1 %test to i32
 ; CHECK: cset {{[xw][0-9]+}}, ne
 
+; CHECK: mov [[RHSCOPY:w[0-9]+]], [[RHS]]
+; CHECK: mov [[LHSCOPY:w[0-9]+]], [[LHS]]
+
   store i32 %val, i32* @var
 
   call void @bar()
 ; CHECK: bl bar
 
   ; Currently, the comparison is emitted again. An MSR/MRS pair would also be
   ; acceptable, but assuming the call preserves NZCV is not.
   br i1 %test, label %iftrue, label %iffalse
-; CHECK: cmp [[LHS]], [[RHS]]
+; CHECK: cmp [[LHSCOPY]], [[RHSCOPY]]
 ; CHECK: b.eq
 
 iftrue:
   ret i32 42
 iffalse:
   ret i32 0
 }

llvm/trunk/test/CodeGen/AArch64/ldst-opt.ll

entry:
   store i64 0, i64* %p7
   ret void
 }
 
 ; Check for bug 34674 where invalid add of xzr was being generated.
 ; CHECK-LABEL: bug34674:
 ; CHECK: // %entry
 ; CHECK-NEXT: mov [[ZREG:x[0-9]+]], xzr
-; CHECK-DAG: stp [[ZREG]], [[ZREG]], [x0]
+; CHECK-DAG: stp xzr, xzr, [x0]
 ; CHECK-DAG: add x{{[0-9]+}}, [[ZREG]], #1
 define i64 @bug34674(<2 x i64>* %p) {
 entry:
   store <2 x i64> zeroinitializer, <2 x i64>* %p
   %p2 = bitcast <2 x i64>* %p to i64*
   %ld = load i64, i64* %p2
   %add = add i64 %ld, 1
   ret i64 %add
 }

llvm/trunk/test/CodeGen/AArch64/merge-store-dependency.ll

 ; RUN: llc < %s -mcpu cortex-a53 -mtriple=aarch64-eabi | FileCheck %s --check-prefix=A53
 
 ; PR26827 - Merge stores causes wrong dependency.
 %struct1 = type { %struct1*, %struct1*, i32, i32, i16, i16, void (i32, i32, i8*)*, i8* }
 @gv0 = internal unnamed_addr global i32 0, align 4
 @gv1 = internal unnamed_addr global %struct1** null, align 8
 
 define void @test(%struct1* %fde, i32 %fd, void (i32, i32, i8*)* %func, i8* %arg)  {
 ;CHECK-LABEL: test
 entry:
 ; A53: mov [[DATA:w[0-9]+]], w1
 ; A53: str q{{[0-9]+}}, {{.*}}
 ; A53: str q{{[0-9]+}}, {{.*}}
-; A53: str [[DATA]], {{.*}}
+; A53: str w1, {{.*}}
 
   %0 = bitcast %struct1* %fde to i8*
   tail call void @llvm.memset.p0i8.i64(i8* align 8 %0, i8 0, i64 40, i1 false)
   %state = getelementptr inbounds %struct1, %struct1* %fde, i64 0, i32 4
   store i16 256, i16* %state, align 8
   %fd1 = getelementptr inbounds %struct1, %struct1* %fde, i64 0, i32 2
   store i32 %fd, i32* %fd1, align 8
   %force_eof = getelementptr inbounds %struct1, %struct1* %fde, i64 0, i32 3

llvm/trunk/test/CodeGen/AArch64/neg-imm.ll

 ; RUN: llc -mtriple=aarch64-linux-gnu -verify-machineinstrs -o - %s | FileCheck %s
 ; LSR used to pick a sub-optimal solution due to the target responding
 ; conservatively to isLegalAddImmediate for negative values.
 
 declare void @foo(i32)
 
 define void @test(i32 %px) {
 ; CHECK_LABEL: test:
 ; CHECK_LABEL: %entry
-; CHECK: subs
-; CHECK-NEXT: csel
+; CHECK: subs [[REG0:w[0-9]+]],
+; CHECK: csel {{w[0-9]+}}, wzr, [[REG0]]
 entry:
   %sub = add nsw i32 %px, -1
   %cmp = icmp slt i32 %px, 1
   %.sub = select i1 %cmp, i32 0, i32 %sub
   br label %for.body
 
 for.body:
 ; CHECK_LABEL: %for.body

llvm/trunk/test/CodeGen/AMDGPU/callee-special-input-sgprs.ll

define void @func_use_every_sgpr_input_call_use_workgroup_id_xyz() #1 {
   call void @use_workgroup_id_xyz()
   ret void
 }
 
 ; GCN-LABEL: {{^}}func_use_every_sgpr_input_call_use_workgroup_id_xyz_spill:
 ; GCN: s_mov_b32 s5, s32
 ; GCN: s_add_u32 s32, s32, 0x300
 
-; GCN-DAG: s_mov_b32 [[SAVE_X:s[0-9]+]], s14
-; GCN-DAG: s_mov_b32 [[SAVE_Y:s[0-9]+]], s15
-; GCN-DAG: s_mov_b32 [[SAVE_Z:s[0-9]+]], s16
+; GCN-DAG: s_mov_b32 [[SAVE_X:s[0-57-9][0-9]*]], s14
+; GCN-DAG: s_mov_b32 [[SAVE_Y:s[0-68-9][0-9]*]], s15
+; GCN-DAG: s_mov_b32 [[SAVE_Z:s[0-79][0-9]*]], s16
 ; GCN-DAG: s_mov_b64 {{s\[[0-9]+:[0-9]+\]}}, s[6:7]
 ; GCN-DAG: s_mov_b64 {{s\[[0-9]+:[0-9]+\]}}, s[8:9]
 ; GCN-DAG: s_mov_b64 {{s\[[0-9]+:[0-9]+\]}}, s[10:11]
 
-; GCN-DAG: s_mov_b32 s6, [[SAVE_X]]
-; GCN-DAG: s_mov_b32 s7, [[SAVE_Y]]
-; GCN-DAG: s_mov_b32 s8, [[SAVE_Z]]
+; GCN-DAG: s_mov_b32 s6, s14
+; GCN-DAG: s_mov_b32 s7, s15
+; GCN-DAG: s_mov_b32 s8, s16
 ; GCN: s_swappc_b64
 
 ; GCN: buffer_store_dword v{{[0-9]+}}, off, s[0:3], s5 offset:4
 ; GCN: s_load_dword s{{[0-9]+}},
 ; GCN: s_load_dword s{{[0-9]+}},
 ; GCN: s_load_dword s{{[0-9]+}},
 ; GCN: ; use
 ; GCN: ; use [[SAVE_X]]

llvm/trunk/test/CodeGen/AMDGPU/fix-vgpr-copies.mir

-# RUN: llc -march=amdgcn -start-after=greedy -stop-after=si-optimize-exec-masking -o - %s | FileCheck %s
+# RUN: llc -march=amdgcn -start-after=greedy -disable-copyprop -stop-after=si-optimize-exec-masking -o - %s | FileCheck %s
 # Check that we first do all vector instructions and only then change exec
 # CHECK-DAG:  COPY $vgpr10_vgpr11
 # CHECK-DAG:  COPY $vgpr12_vgpr13
 # CHECK:      $exec = COPY
 
 ---
 name:            main
 alignment:       0

llvm/trunk/test/CodeGen/AMDGPU/multilevel-break.ll

 
 ; GCN-LABEL: {{^}}multi_if_break_loop:
 ; GCN: s_mov_b64 [[BREAK_REG:s\[[0-9]+:[0-9]+\]]], 0{{$}}
 
 ; GCN: [[LOOP:BB[0-9]+_[0-9]+]]: ; %bb1{{$}}
 
 ; Uses a copy intsead of an or
 ; GCN: s_mov_b64 [[COPY:s\[[0-9]+:[0-9]+\]]], [[BREAK_REG]]
-; GCN: s_or_b64 [[BREAK_REG]], exec, [[COPY]]
+; GCN: s_or_b64 [[BREAK_REG]], exec, [[BREAK_REG]]
 define amdgpu_kernel void @multi_if_break_loop(i32 %arg) #0 {
 bb:
   %id = call i32 @llvm.amdgcn.workitem.id.x()
   %tmp = sub i32 %id, %arg
   br label %bb1
 
 bb1:
   %lsr.iv = phi i32 [ undef, %bb ], [ %lsr.iv.next, %case0 ], [ %lsr.iv.next, %case1 ]

llvm/trunk/test/CodeGen/AMDGPU/ret.ll

 ; RUN: llc -march=amdgcn -mcpu=tahiti -verify-machineinstrs < %s | FileCheck -check-prefix=GCN %s
 ; RUN: llc -march=amdgcn -mcpu=tonga -verify-machineinstrs < %s | FileCheck -check-prefix=GCN %s
 
 ; GCN-LABEL: {{^}}vgpr:
-; GCN: v_mov_b32_e32 v1, v0
-; GCN-DAG: v_add_f32_e32 v0, 1.0, v1
-; GCN-DAG: exp mrt0 v1, v1, v1, v1 done vm
+; GCN-DAG: v_mov_b32_e32 v1, v0
+; GCN-DAG: exp mrt0 v0, v0, v0, v0 done vm
 ; GCN: s_waitcnt expcnt(0)
+; GCN: v_add_f32_e32 v0, 1.0, v0
 ; GCN-NOT: s_endpgm
 define amdgpu_vs { float, float } @vgpr([9 x <16 x i8>] addrspace(2)* byval %arg, i32 inreg %arg1, i32 inreg %arg2, float %arg3) #0 {
 bb:
   call void @llvm.amdgcn.exp.f32(i32 0, i32 15, float %arg3, float %arg3, float %arg3, float %arg3, i1 true, i1 true) #0
   %x = fadd float %arg3, 1.000000e+00
   %a = insertvalue { float, float } undef, float %x, 0
   %b = insertvalue { float, float } %a, float %arg3, 1
   ret { float, float } %b

 ; GCN-NOT: s_endpgm
 define amdgpu_vs { i32, i32, i32, i32 } @sgpr_literal([9 x <16 x i8>] addrspace(2)* byval %arg, i32 inreg %arg1, i32 inreg %arg2, float %arg3) #0 {
 bb:
   %x = add i32 %arg2, 2
   ret { i32, i32, i32, i32 } { i32 5, i32 6, i32 7, i32 8 }
 }
 
 ; GCN-LABEL: {{^}}both:
-; GCN: v_mov_b32_e32 v1, v0
-; GCN-DAG: exp mrt0 v1, v1, v1, v1 done vm
-; GCN-DAG: v_add_f32_e32 v0, 1.0, v1
-; GCN-DAG: s_add_i32 s0, s3, 2
+; GCN-DAG: exp mrt0 v0, v0, v0, v0 done vm
+; GCN-DAG: v_mov_b32_e32 v1, v0
 ; GCN-DAG: s_mov_b32 s1, s2
-; GCN: s_mov_b32 s2, s3
 ; GCN: s_waitcnt expcnt(0)
+; GCN: v_add_f32_e32 v0, 1.0, v0
+; GCN-DAG: s_add_i32 s0, s3, 2
+; GCN-DAG: s_mov_b32 s2, s3
 ; GCN-NOT: s_endpgm
 define amdgpu_vs { float, i32, float, i32, i32 } @both([9 x <16 x i8>] addrspace(2)* byval %arg, i32 inreg %arg1, i32 inreg %arg2, float %arg3) #0 {
 bb:
   call void @llvm.amdgcn.exp.f32(i32 0, i32 15, float %arg3, float %arg3, float %arg3, float %arg3, i1 true, i1 true) #0
   %v = fadd float %arg3, 1.000000e+00
   %s = add i32 %arg2, 2
   %a0 = insertvalue { float, i32, float, i32, i32 } undef, float %v, 0
   %a1 = insertvalue { float, i32, float, i32, i32 } %a0, i32 %s, 1

llvm/trunk/test/CodeGen/ARM/atomic-op.ll

entry:
   ret i32 %0
 }
 
 define i32 @test_cmpxchg_fail_order(i32 *%addr, i32 %desired, i32 %new) {
 ; CHECK-LABEL: test_cmpxchg_fail_order:
 
   %pair = cmpxchg i32* %addr, i32 %desired, i32 %new seq_cst monotonic
   %oldval = extractvalue { i32, i1 } %pair, 0
-; CHECK-ARMV7:     ldrex   [[OLDVAL:r[0-9]+]], [r[[ADDR:[0-9]+]]]
+; CHECK-ARMV7:     mov     r[[ADDR:[0-9]+]], r0
+; CHECK-ARMV7:     ldrex   [[OLDVAL:r[0-9]+]], [r0]
 ; CHECK-ARMV7:     cmp     [[OLDVAL]], r1
 ; CHECK-ARMV7:     bne     [[FAIL_BB:\.?LBB[0-9]+_[0-9]+]]
 ; CHECK-ARMV7:     dmb ish
 ; CHECK-ARMV7: [[LOOP_BB:\.?LBB.*]]:
 ; CHECK-ARMV7:     strex   [[SUCCESS:r[0-9]+]], r2, [r[[ADDR]]]
 ; CHECK-ARMV7:     cmp     [[SUCCESS]], #0
 ; CHECK-ARMV7:     beq     [[SUCCESS_BB:\.?LBB.*]]
 ; CHECK-ARMV7:     ldrex   [[OLDVAL]], [r[[ADDR]]]
 ; CHECK-ARMV7:     cmp     [[OLDVAL]], r1
 ; CHECK-ARMV7:     beq     [[LOOP_BB]]
 ; CHECK-ARMV7: [[FAIL_BB]]:
 ; CHECK-ARMV7:     clrex
 ; CHECK-ARMV7:     bx      lr
 ; CHECK-ARMV7: [[SUCCESS_BB]]:
 ; CHECK-ARMV7:     dmb     ish
 ; CHECK-ARMV7:     bx      lr
 
-; CHECK-T2:     ldrex   [[OLDVAL:r[0-9]+]], [r[[ADDR:[0-9]+]]]
+; CHECK-T2:     mov     r[[ADDR:[0-9]+]], r0
+; CHECK-T2:     ldrex   [[OLDVAL:r[0-9]+]], [r0]
 ; CHECK-T2:     cmp     [[OLDVAL]], r1
 ; CHECK-T2:     bne     [[FAIL_BB:\.?LBB.*]]
 ; CHECK-T2:     dmb ish
 ; CHECK-T2: [[LOOP_BB:\.?LBB.*]]:
 ; CHECK-T2:     strex   [[SUCCESS:r[0-9]+]], r2, [r[[ADDR]]]
 ; CHECK-T2:     cmp     [[SUCCESS]], #0
 ; CHECK-T2:     dmbeq   ish
 ; CHECK-T2:     bxeq    lr

llvm/trunk/test/CodeGen/ARM/intrinsics-overflow.ll

define i32 @sadd_overflow(i32 %a, i32 %b) #0 {
   ; CHECK-LABEL: sadd_overflow:
 
   ; ARM: adds r[[R2:[0-9]+]], r[[R0:[0-9]+]], r[[R1:[0-9]+]]
   ; ARM: mov r[[R0]], #1
   ; ARM: movvc r[[R0]], #0
   ; ARM: mov pc, lr
 
   ; THUMBV6: mov  r[[R2:[0-9]+]], r[[R0:[0-9]+]]
-  ; THUMBV6: adds r[[R3:[0-9]+]], r[[R2]], r[[R1:[0-9]+]]
+  ; THUMBV6: adds r[[R3:[0-9]+]], r[[R0]], r[[R1:[0-9]+]]
   ; THUMBV6: movs r[[R0]], #0
   ; THUMBV6: movs r[[R1]], #1
   ; THUMBV6: cmp  r[[R3]], r[[R2]]
   ; THUMBV6: bvc  .L[[LABEL:.*]]
   ; THUMBV6: mov  r[[R0]], r[[R1]]
   ; THUMBV6: .L[[LABEL]]:
 
   ; THUMBV7: adds  r[[R2:[0-9]+]], r[[R0]], r[[R1:[0-9]+]]

llvm/trunk/test/CodeGen/ARM/select-imm.ll

 
 define void @t8(i32 %a) {
 entry:
 
 ; ARM scheduler emits icmp/zext before both calls, so isn't relevant
 
 ; ARMT2-LABEL: t8:
 ; ARMT2: mov     r1, r0
+; ARMT2: cmp     r0, #5
 ; ARMT2: mov     r0, #9
 ; ARMT2: mov     r4, #0
-; ARMT2: cmp     r1, #5
 ; ARMT2: movweq  r4, #1
 ; ARMT2: bl      t7
 
 ; THUMB1-LABEL: t8:
 ; THUMB1: mov     r1, r0
 ; THUMB1: movs    r4, #1
 ; THUMB1: movs    r0, #0
 ; THUMB1: cmp     r1, #5
 ; THUMB1: beq     .LBB7_2
 ; THUMB1: mov     r4, r0
 
 ; THUMB2-LABEL: t8:
 ; THUMB2: mov     r1, r0
-; THUMB2: movs    r4, #0
-; THUMB2: cmp     r1, #5
+; THUMB2: cmp     r0, #5
+; THUMB2: mov.w   r4, #0
 ; THUMB2: it      eq
 ; THUMB2: moveq   r4, #1
   %cmp = icmp eq i32 %a, 5
   %conv = zext i1 %cmp to i32
   %call = tail call i32 @t7(i32 9, i32 %a)
   tail call i32 @t7(i32 %conv, i32 %call)
   ret void
 }

llvm/trunk/test/CodeGen/ARM/swifterror.ll

 ; CHECK-APPLE-LABEL: foo_loop:
 ; CHECK-APPLE: mov [[CODE:r[0-9]+]], r0
 ; swifterror is kept in a register
 ; CHECK-APPLE: mov [[ID:r[0-9]+]], r8
 ; CHECK-APPLE: cmp [[CODE]], #0
 ; CHECK-APPLE: beq
 ; CHECK-APPLE: mov r0, #16
 ; CHECK-APPLE: malloc
-; CHECK-APPLE: strb r{{.*}}, [{{.*}}[[ID]], #8]
+; CHECK-APPLE: strb r{{.*}}, [r0, #8]
 ; CHECK-APPLE: ble
 ; CHECK-APPLE: mov r8, [[ID]]
 
 ; CHECK-O0-LABEL: foo_loop:
 ; CHECK-O0: mov r{{.*}}, r8
 ; CHECK-O0: cmp r{{.*}}, #0
 ; CHECK-O0: beq
 ; CHECK-O0-DAG: movw r{{.*}}, #1

llvm/trunk/test/CodeGen/Mips/llvm-ir/ashr.ll

 ; MMR3-NEXT:    sw $16, 40($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    .cfi_offset 17, -4
 ; MMR3-NEXT:    .cfi_offset 16, -8
 ; MMR3-NEXT:    move $8, $7
 ; MMR3-NEXT:    sw $6, 32($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    sw $5, 36($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    sw $4, 8($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    lw $16, 76($sp)
-; MMR3-NEXT:    srlv $4, $8, $16
+; MMR3-NEXT:    srlv $4, $7, $16
 ; MMR3-NEXT:    not16 $3, $16
 ; MMR3-NEXT:    sw $3, 24($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    sll16 $2, $6, 1
 ; MMR3-NEXT:    sllv $3, $2, $3
 ; MMR3-NEXT:    li16 $2, 64
 ; MMR3-NEXT:    or16 $3, $4
 ; MMR3-NEXT:    srlv $6, $6, $16
 ; MMR3-NEXT:    sw $6, 12($sp) # 4-byte Folded Spill

 ; MMR6-NEXT:    .cfi_offset 16, -8
 ; MMR6-NEXT:    move $1, $7
 ; MMR6-NEXT:    sw $6, 28($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    move $6, $5
 ; MMR6-NEXT:    sw $4, 12($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    lw $3, 68($sp)
 ; MMR6-NEXT:    li16 $2, 64
 ; MMR6-NEXT:    subu16 $7, $2, $3
-; MMR6-NEXT:    sllv $8, $6, $7
+; MMR6-NEXT:    sllv $8, $5, $7
 ; MMR6-NEXT:    andi16 $5, $7, 32
 ; MMR6-NEXT:    selnez $9, $8, $5
 ; MMR6-NEXT:    sllv $16, $4, $7
 ; MMR6-NEXT:    not16 $7, $7
 ; MMR6-NEXT:    srl16 $17, $6, 1
 ; MMR6-NEXT:    sw $6, 20($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    srlv $7, $17, $7
 ; MMR6-NEXT:    or16 $7, $16

llvm/trunk/test/CodeGen/Mips/llvm-ir/lshr.ll

 ; MMR3-NEXT:    .cfi_offset 17, -4
 ; MMR3-NEXT:    .cfi_offset 16, -8
 ; MMR3-NEXT:    move $8, $7
 ; MMR3-NEXT:    sw $6, 32($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    sw $5, 36($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    move $17, $5
 ; MMR3-NEXT:    sw $4, 8($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    lw $16, 76($sp)
-; MMR3-NEXT:    srlv $7, $8, $16
+; MMR3-NEXT:    srlv $7, $7, $16
 ; MMR3-NEXT:    not16 $3, $16
 ; MMR3-NEXT:    sw $3, 24($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    sll16 $2, $6, 1
 ; MMR3-NEXT:    sllv $3, $2, $3
 ; MMR3-NEXT:    li16 $4, 64
 ; MMR3-NEXT:    or16 $3, $7
 ; MMR3-NEXT:    srlv $5, $6, $16
 ; MMR3-NEXT:    sw $5, 12($sp) # 4-byte Folded Spill

 ; MMR6-NEXT:    .cfi_def_cfa_offset 48
 ; MMR6-NEXT:    sw $17, 44($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    sw $16, 40($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    .cfi_offset 17, -4
 ; MMR6-NEXT:    .cfi_offset 16, -8
 ; MMR6-NEXT:    move $1, $7
 ; MMR6-NEXT:    sw $5, 8($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    move $16, $4
-; MMR6-NEXT:    sw $16, 32($sp) # 4-byte Folded Spill
+; MMR6-NEXT:    sw $4, 32($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    lw $3, 76($sp)
-; MMR6-NEXT:    srlv $2, $1, $3
+; MMR6-NEXT:    srlv $2, $7, $3
 ; MMR6-NEXT:    not16 $5, $3
 ; MMR6-NEXT:    sw $5, 24($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    move $4, $6
-; MMR6-NEXT:    sw $4, 28($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    sll16 $6, $4, 1
+; MMR6-NEXT:    sw $6, 28($sp) # 4-byte Folded Spill
+; MMR6-NEXT:    sll16 $6, $6, 1
 ; MMR6-NEXT:    sllv $17, $6, $5
 ; MMR6-NEXT:    or16 $17, $2
 ; MMR6-NEXT:    addiu $7, $3, -64
 ; MMR6-NEXT:    sw $7, 36($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    lw $5, 8($sp) # 4-byte Folded Reload
 ; MMR6-NEXT:    srlv $6, $5, $7
 ; MMR6-NEXT:    sll16 $2, $16, 1
 ; MMR6-NEXT:    sw $2, 20($sp) # 4-byte Folded Spill

 ; MMR6-NEXT:    sw $2, 12($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    or $5, $9, $8
 ; MMR6-NEXT:    selnez $8, $11, $4
 ; MMR6-NEXT:    lw $2, 32($sp) # 4-byte Folded Reload
 ; MMR6-NEXT:    sllv $7, $2, $6
 ; MMR6-NEXT:    sw $7, 4($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    not16 $6, $6
 ; MMR6-NEXT:    move $7, $17
-; MMR6-NEXT:    srl16 $17, $7, 1
+; MMR6-NEXT:    srl16 $17, $17, 1
 ; MMR6-NEXT:    srlv $6, $17, $6
 ; MMR6-NEXT:    lw $17, 4($sp) # 4-byte Folded Reload
 ; MMR6-NEXT:    or16 $6, $17
 ; MMR6-NEXT:    seleqz $4, $6, $4
 ; MMR6-NEXT:    lw $6, 36($sp) # 4-byte Folded Reload
 ; MMR6-NEXT:    srlv $9, $2, $6
 ; MMR6-NEXT:    or $4, $8, $4
 ; MMR6-NEXT:    lw $2, 12($sp) # 4-byte Folded Reload

llvm/trunk/test/CodeGen/Mips/llvm-ir/shl.ll

 ; MMR3-NEXT:    .cfi_offset 17, -4
 ; MMR3-NEXT:    .cfi_offset 16, -8
 ; MMR3-NEXT:    sw $7, 8($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    sw $6, 36($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    move $17, $6
 ; MMR3-NEXT:    sw $5, 32($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    move $1, $4
 ; MMR3-NEXT:    lw $16, 76($sp)
-; MMR3-NEXT:    sllv $2, $1, $16
+; MMR3-NEXT:    sllv $2, $4, $16
 ; MMR3-NEXT:    not16 $4, $16
 ; MMR3-NEXT:    sw $4, 24($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    srl16 $3, $5, 1
 ; MMR3-NEXT:    srlv $4, $3, $4
 ; MMR3-NEXT:    li16 $3, 64
 ; MMR3-NEXT:    or16 $4, $2
 ; MMR3-NEXT:    sllv $6, $5, $16
 ; MMR3-NEXT:    sw $6, 20($sp) # 4-byte Folded Spill

 ; MMR6-NEXT:    .cfi_def_cfa_offset 32
 ; MMR6-NEXT:    sw $17, 28($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    sw $16, 24($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    .cfi_offset 17, -4
 ; MMR6-NEXT:    .cfi_offset 16, -8
 ; MMR6-NEXT:    sw $6, 4($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    move $1, $4
 ; MMR6-NEXT:    lw $3, 60($sp)
-; MMR6-NEXT:    sllv $2, $1, $3
+; MMR6-NEXT:    sllv $2, $4, $3
 ; MMR6-NEXT:    not16 $4, $3
 ; MMR6-NEXT:    sw $4, 16($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    sw $5, 20($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    srl16 $16, $5, 1
 ; MMR6-NEXT:    srlv $17, $16, $4
 ; MMR6-NEXT:    or16 $17, $2
 ; MMR6-NEXT:    sllv $8, $5, $3
 ; MMR6-NEXT:    andi16 $16, $3, 32

llvm/trunk/test/CodeGen/Mips/llvm-ir/sub.ll

 ; MMR3: subu16   $[[T16:[0-9]+]], $[[T15]], $[[T10]]
 ; MMR3: subu16   $[[T17:[0-9]+]], $6, $[[T1]]
 ; MMR3: subu16   $[[T18:[0-9]+]], $[[T17]], $7
 ; MMR3: lw       $[[T19:[0-9]+]], 8($sp)
 ; MMR3: lw       $[[T20:[0-9]+]], 0($sp)
 ; MMR3: subu16   $5, $[[T19]], $[[T20]]
 
 ; MMR6: move     $[[T0:[0-9]+]], $7
-; MMR6: sw       $[[T0]], 8($sp)
+; MMR6: sw       $7, 8($sp)
 ; MMR6: move     $[[T1:[0-9]+]], $5
 ; MMR6: sw       $4, 12($sp)
 ; MMR6: lw       $[[T2:[0-9]+]], 48($sp)
 ; MMR6: sltu     $[[T3:[0-9]+]], $6, $[[T2]]
 ; MMR6: xor      $[[T4:[0-9]+]], $6, $[[T2]]
 ; MMR6: sltiu    $[[T5:[0-9]+]], $[[T4]], 1
 ; MMR6: seleqz   $[[T6:[0-9]+]], $[[T3]], $[[T5]]
 ; MMR6: lw       $[[T7:[0-9]+]], 52($sp)

llvm/trunk/test/CodeGen/PowerPC/MCSE-caller-preserved-reg.ll

 define noalias i8* @_ZN2CC3funEv(%class.CC* %this) {
 ; CHECK-LABEL: _ZN2CC3funEv:
 ; CHECK:    mflr 0
 ; CHECK-NEXT:    std 0, 16(1)
 ; CHECK-NEXT:    stdu 1, -48(1)
 ; CHECK-NEXT:    .cfi_def_cfa_offset 48
 ; CHECK-NEXT:    .cfi_offset lr, 16
 ; CHECK-NEXT:    .cfi_offset r30, -16
+; CHECK-NEXT:    ld 12, 0(3)
 ; CHECK-NEXT:    std 30, 32(1)
 ; CHECK-NEXT:    mr 30, 3
-; CHECK-NEXT:    ld 12, 0(30)
 ; CHECK-NEXT:    std 2, 24(1)
 ; CHECK-NEXT:    mtctr 12
 ; CHECK-NEXT:    bctrl
 ; CHECK-NEXT:    ld 2, 24(1)
 ; CHECK-NEXT:    addis 3, 2, _ZN2CC2ccE@got@tlsgd@ha
 ; CHECK-NEXT:    addi 3, 3, _ZN2CC2ccE@got@tlsgd@l
 ; CHECK-NEXT:    bl __tls_get_addr(_ZN2CC2ccE@tlsgd)
 ; CHECK-NEXT:    nop

llvm/trunk/test/CodeGen/PowerPC/fma-mutate.ll

 ; Test several VSX FMA mutation opportunities.  The first one isn't a
 ; reasonable transformation because the killed product register is the
 ; same as the FMA target register.  The second one is legal.  The third
 ; one doesn't fit the feeding-copy pattern.
 
 ; RUN: llc -verify-machineinstrs < %s -mtriple=powerpc64-unknown-linux-gnu -mcpu=pwr7 -enable-unsafe-fp-math -mattr=+vsx -disable-ppc-vsx-fma-mutation=false | FileCheck %s
 target datalayout = "E-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-f128:128:128-v128:128:128-n32:64"
 target triple = "powerpc64-unknown-linux-gnu"
 
 declare double @llvm.sqrt.f64(double)
 
 define double @foo3(double %a) nounwind {
   %r = call double @llvm.sqrt.f64(double %a)
   ret double %r
 
 ; CHECK: @foo3
-; CHECK: xsnmsubadp [[REG:[0-9]+]], {{[0-9]+}}, [[REG]]
+; CHECK: fmr [[REG:[0-9]+]], [[REG2:[0-9]+]]
+; CHECK: xsnmsubadp [[REG]], {{[0-9]+}}, [[REG2]]
 ; CHECK: xsmaddmdp
 ; CHECK: xsmaddadp
 }
 

llvm/trunk/test/CodeGen/PowerPC/gpr-vsr-spill.ll

if.then:                                          ; preds = %entry
   %tmp = add i32 %add, %mul
   br label %if.end
 
 if.end:                                           ; preds = %if.then, %entry
   %e.0 = phi i32 [ %tmp, %if.then ], [ undef, %entry ]
   ret i32 %e.0
 ; CHECK: @foo
 ; CHECK: mr [[NEWREG:[0-9]+]], 3
+; CHECK: mr [[REG1:[0-9]+]], 4
 ; CHECK: mtvsrd [[NEWREG2:[0-9]+]], 4
-; CHECK: mffprd [[REG1:[0-9]+]], [[NEWREG2]]
 ; CHECK: add {{[0-9]+}}, [[NEWREG]], [[REG1]]
 ; CHECK: mffprd [[REG2:[0-9]+]], [[NEWREG2]]
 ; CHECK: add {{[0-9]+}}, [[REG2]], [[NEWREG]]
 }

llvm/trunk/test/CodeGen/PowerPC/licm-remat.ll

 ; Function Attrs: argmemonly nounwind
 declare void @llvm.memmove.p0i8.p0i8.i64(i8* nocapture, i8* nocapture readonly, i64, i1) #2
 ; Function Attrs: argmemonly nounwind
 declare void @llvm.memcpy.p0i8.p0i8.i64(i8* nocapture writeonly, i8* nocapture readonly, i64, i1) #2
 
 define linkonce_odr void @ZN6snappyDecompressor_(%"class.snappy::SnappyDecompressor"* %this, %"class.snappy::SnappyIOVecWriter"* %writer) {
 ; CHECK-LABEL: ZN6snappyDecompressor_:
 ; CHECK:       # %bb.0: # %entry
-; CHECK:       addis 3, 2, _ZN6snappy8internalL8wordmaskE@toc@ha
-; CHECK-DAG:   addi 25, 3, _ZN6snappy8internalL8wordmaskE@toc@l
+; CHECK:       addis 23, 2, _ZN6snappy8internalL8wordmaskE@toc@ha
+; CHECK-DAG:   addi 25, 23, _ZN6snappy8internalL8wordmaskE@toc@l
 ; CHECK-DAG:   addis 5, 2, _ZN6snappy8internalL10char_tableE@toc@ha
 ; CHECK-DAG:   addi 24, 5, _ZN6snappy8internalL10char_tableE@toc@l
 ; CHECK:       b .LBB0_2
 ; CHECK:       .LBB0_2: # %for.cond
 ; CHECK-NOT:   addis {{[0-9]+}}, 2, _ZN6snappy8internalL8wordmaskE@toc@ha
 ; CHECK-NOT:   addis {{[0-9]+}}, 2, _ZN6snappy8internalL10char_tableE@toc@ha
 ; CHECK:       bctrl
 entry:

llvm/trunk/test/CodeGen/PowerPC/opt-li-add-to-addi.ll

 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu -mcpu=pwr8 < %s | FileCheck %s
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu -mcpu=pwr8 < %s | FileCheck %s
 
 define i64 @testOptimizeLiAddToAddi(i64 %a) {
 ; CHECK-LABEL: testOptimizeLiAddToAddi:
-; CHECK:    addi 3, 30, 2444
+; CHECK:    addi 3, 3, 2444
 ; CHECK:    bl callv
 ; CHECK:    addi 3, 30, 234
 ; CHECK:    bl call
 ; CHECK:    blr
 entry:
   %cmp = icmp sgt i64 %a, 33
   br i1 %cmp, label %if.then, label %if.end
 

llvm/trunk/test/CodeGen/PowerPC/tail-dup-layout.ll

 ; so optional1 includes a copy of test2 at the end, and branches
 ; to test3 (at the top) or falls through to optional 2.
 ; The CHECK statements check for the whole string of tests
 ; and then check that the correct test has been duplicated into the end of
 ; the optional blocks and that the optional blocks are in the correct order.
 ;CHECK-LABEL: straight_test:
 ; test1 may have been merged with entry
 ;CHECK: mr [[TAGREG:[0-9]+]], 3
-;CHECK: andi. {{[0-9]+}}, [[TAGREG]], 1
+;CHECK: andi. {{[0-9]+}}, [[TAGREG:[0-9]+]], 1
 ;CHECK-NEXT: bc 12, 1, .[[OPT1LABEL:[_0-9A-Za-z]+]]
 ;CHECK-NEXT: # %test2
 ;CHECK-NEXT: rlwinm. {{[0-9]+}}, [[TAGREG]], 0, 30, 30
 ;CHECK-NEXT: bne 0, .[[OPT2LABEL:[_0-9A-Za-z]+]]
 ;CHECK-NEXT: .[[TEST3LABEL:[_0-9A-Za-z]+]]: # %test3
 ;CHECK-NEXT: rlwinm. {{[0-9]+}}, [[TAGREG]], 0, 29, 29
 ;CHECK-NEXT: bne 0, .[[OPT3LABEL:[_0-9A-Za-z]+]]
 ;CHECK-NEXT: .[[TEST4LABEL:[_0-9A-Za-z]+]]: # %test4

llvm/trunk/test/CodeGen/SPARC/32abi.ll

 ; CHECK-LABEL: call_floatarg:
 ; HARD: save %sp, -112, %sp
 ; HARD: mov %i2, %o1
 ; HARD-NEXT: mov %i1, %o0
 ; HARD-NEXT: st %i0, [%sp+104]
 ; HARD-NEXT: std %o0, [%sp+96]
 ; HARD-NEXT: st %o1, [%sp+92]
 ; HARD-NEXT: mov %i0, %o2
-; HARD-NEXT: mov %o0, %o3
+; HARD-NEXT: mov %i1, %o3
 ; HARD-NEXT: mov %o1, %o4
-; HARD-NEXT: mov %o0, %o5
+; HARD-NEXT: mov %i1, %o5
 ; HARD-NEXT: call floatarg
 ; HARD: std %f0, [%i4]
 ; SOFT: st %i0, [%sp+104]
 ; SOFT-NEXT:  st %i2, [%sp+100]
 ; SOFT-NEXT:  st %i1, [%sp+96]
 ; SOFT-NEXT:  st %i2, [%sp+92]
 ; SOFT-NEXT:  mov  %i1, %o0
 ; SOFT-NEXT:  mov  %i2, %o1

llvm/trunk/test/CodeGen/SPARC/atomics.ll

 define zeroext i16 @test_load_sub_i16(i16* %p, i16 zeroext %v) {
 entry:
   %0 = atomicrmw sub i16* %p, i16 %v seq_cst
   ret i16 %0
 }
 
 ; CHECK-LABEL: test_load_add_i32
 ; CHECK: membar
-; CHECK: add [[V:%[gilo][0-7]]], %o1, [[U:%[gilo][0-7]]]
-; CHECK: cas [%o0], [[V]], [[U]]
+; CHECK: mov [[U:%[gilo][0-7]]], [[V:%[gilo][0-7]]]
+; CHECK: add [[U:%[gilo][0-7]]], %o1, [[V2:%[gilo][0-7]]]
+; CHECK: cas [%o0], [[V]], [[V2]]
 ; CHECK: membar
 define zeroext i32 @test_load_add_i32(i32* %p, i32 zeroext %v) {
 entry:
   %0 = atomicrmw add i32* %p, i32 %v seq_cst
   ret i32 %0
 }
 
 ; CHECK-LABEL: test_load_sub_64

llvm/trunk/test/CodeGen/SystemZ/vec-sub-01.ll

 ; Test a v4f32 subtraction, as an example of an operation that needs to be
 ; scalarized and reassembled.  At present there's an unnecessary move that
 ; could be avoided with smarter ordering.  It also isn't important whether
 ; the VSLDBs use the result of the VLRs or use %v24 and %v26 directly.
 define <4 x float> @f5(<4 x float> %val1, <4 x float> %val2) {
 ; CHECK-LABEL: f5:
 ; CHECK-DAG: vlr %v[[A1:[0-5]]], %v24
 ; CHECK-DAG: vlr %v[[A2:[0-5]]], %v26
-; CHECK-DAG: vrepf %v[[B1:[0-5]]], %v[[A1]], 1
-; CHECK-DAG: vrepf %v[[B2:[0-5]]], %v[[A2]], 1
-; CHECK-DAG: vrepf %v[[C1:[0-5]]], %v[[A1]], 2
-; CHECK-DAG: vrepf %v[[C2:[0-5]]], %v[[A2]], 2
-; CHECK-DAG: vrepf %v[[D1:[0-5]]], %v[[A1]], 3
-; CHECK-DAG: vrepf %v[[D2:[0-5]]], %v[[A2]], 3
+; CHECK-DAG: vrepf %v[[B1:[0-5]]], %v24, 1
+; CHECK-DAG: vrepf %v[[B2:[0-5]]], %v26, 1
+; CHECK-DAG: vrepf %v[[C1:[0-5]]], %v24, 2
+; CHECK-DAG: vrepf %v[[C2:[0-5]]], %v26, 2
+; CHECK-DAG: vrepf %v[[D1:[0-5]]], %v24, 3
+; CHECK-DAG: vrepf %v[[D2:[0-5]]], %v26, 3
 ; CHECK-DAG: sebr %f[[A1]], %f[[A2]]
 ; CHECK-DAG: sebr %f[[B1]], %f[[B2]]
 ; CHECK-DAG: sebr %f[[C1]], %f[[C2]]
 ; CHECK-DAG: sebr %f[[D1]], %f[[D2]]
 ; CHECK-DAG: vmrhf [[HIGH:%v[0-9]+]], %v[[A1]], %v[[B1]]
 ; CHECK-DAG: vmrhf [[LOW:%v[0-9]+]], %v[[C1]], %v[[D1]]
 ; CHECK: vmrhg %v24, [[HIGH]], [[LOW]]
 ; CHECK: br %r14

llvm/trunk/test/CodeGen/Thumb/pr35836.ll

while.body:
   %add28 = add nuw nsw i64 %conv27, %conv25
   %add29 = add nuw nsw i64 %add28, %shr23
   %shr32 = lshr i64 %add29, 32
   br label %while.body
 }
 ; CHECK: adds r3, r0, r1
 ; CHECK: push {r5}
 ; CHECK: pop  {r1}
-; CHECK: adcs r1, r1
+; CHECK: adcs r1, r5
 ; CHECK: ldr  r0, [sp, #12]           @ 4-byte Reload
 ; CHECK: ldr  r2, [sp, #8]            @ 4-byte Reload
 ; CHECK: adds r2, r0, r2
 ; CHECK: push {r5}
 ; CHECK: pop  {r4}
-; CHECK: adcs r4, r4
+; CHECK: adcs r4, r5
 ; CHECK: adds r0, r2, r5
 ; CHECK: push {r3}
 ; CHECK: pop  {r0}
 ; CHECK: adcs r0, r4
 ; CHECK: ldr  r6, [sp, #4]            @ 4-byte Reload
 ; CHECK: str  r0, [r6]
 ; CHECK: ldr  r0, [r7]
 ; CHECK: ldr  r6, [sp]                @ 4-byte Reload
 ; CHECK: ldr  r6, [r6]
 ; CHECK: adds r0, r6, r0

llvm/trunk/test/CodeGen/Thumb/thumb-shrink-wrapping.ll

if.end:
   ret i32 42
 }
 
 declare void @abort() #0
 
 define i32 @b_to_bx(i32 %value) {
 ; CHECK-LABEL: b_to_bx:
 ; DISABLE: push {r7, lr}
-; CHECK: cmp r1, #49
+; CHECK: cmp r0, #49
 ; CHECK-NEXT: bgt [[ELSE_LABEL:LBB[0-9_]+]]
 ; ENABLE: push {r7, lr}
 
 ; CHECK: bl
 ; DISABLE-V5-NEXT: pop {r7, pc}
 ; DISABLE-V4T-NEXT: b [[END_LABEL:LBB[0-9_]+]]
 
 ; ENABLE-V5-NEXT: pop {r7, pc}

llvm/trunk/test/CodeGen/X86/2006-03-01-InstrSchedBug.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686-unknown-unknown | FileCheck %s
 
 define i32 @f(i32 %a, i32 %b) {
 ; CHECK-LABEL: f:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; CHECK-NEXT:    movl %ecx, %edx
-; CHECK-NEXT:    imull %edx, %edx
+; CHECK-NEXT:    imull %ecx, %edx
 ; CHECK-NEXT:    imull %eax, %ecx
 ; CHECK-NEXT:    imull %eax, %eax
 ; CHECK-NEXT:    addl %edx, %eax
 ; CHECK-NEXT:    leal (%eax,%ecx,2), %eax
 ; CHECK-NEXT:    retl
   %tmp.2 = mul i32 %a, %a
   %tmp.5 = shl i32 %a, 1
   %tmp.6 = mul i32 %tmp.5, %b
   %tmp.10 = mul i32 %b, %b
   %tmp.7 = add i32 %tmp.10, %tmp.2
   %tmp.11 = add i32 %tmp.7, %tmp.6
   ret i32 %tmp.11
 }
 

llvm/trunk/test/CodeGen/X86/arg-copy-elide.ll

 }
 
 ; CHECK-LABEL: _fastcc_split_i64:
 ; CHECK: pushl %ebp
 ; CHECK: movl %esp, %ebp
 ; CHECK-DAG: movl %edx, %[[r1:[^ ]*]]
 ; CHECK-DAG: movl 8(%ebp), %[[r2:[^ ]*]]
 ; CHECK-DAG: movl %[[r2]], 4(%esp)
-; CHECK-DAG: movl %[[r1]], (%esp)
+; CHECK-DAG: movl %edx, (%esp)
 ; CHECK: movl %esp, %[[reg:[^ ]*]]
 ; CHECK: pushl %[[reg]]
 ; CHECK: calll _addrof_i64
 ; CHECK: popl %ebp
 ; CHECK: retl
 
 
 ; We can't copy elide when it would reduce the user requested alignment.

llvm/trunk/test/CodeGen/X86/avx-load-store.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=avx,slow-unaligned-mem-32 | FileCheck %s
 ; RUN: llc -O0 < %s -mtriple=x86_64-unknown-unknown -mattr=avx,slow-unaligned-mem-32 | FileCheck %s -check-prefix=CHECK_O0
 
 define void @test_256_load(double* nocapture %d, float* nocapture %f, <4 x i64>* nocapture %i) nounwind {
 ; CHECK-LABEL: test_256_load:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    pushq %r15
 ; CHECK-NEXT:    pushq %r14
 ; CHECK-NEXT:    pushq %rbx
 ; CHECK-NEXT:    subq $96, %rsp
 ; CHECK-NEXT:    movq %rdx, %r14
 ; CHECK-NEXT:    movq %rsi, %r15
 ; CHECK-NEXT:    movq %rdi, %rbx
-; CHECK-NEXT:    vmovaps (%rbx), %ymm0
+; CHECK-NEXT:    vmovaps (%rdi), %ymm0
 ; CHECK-NEXT:    vmovups %ymm0, {{[0-9]+}}(%rsp) # 32-byte Spill
-; CHECK-NEXT:    vmovaps (%r15), %ymm1
+; CHECK-NEXT:    vmovaps (%rsi), %ymm1
 ; CHECK-NEXT:    vmovups %ymm1, {{[0-9]+}}(%rsp) # 32-byte Spill
-; CHECK-NEXT:    vmovaps (%r14), %ymm2
+; CHECK-NEXT:    vmovaps (%rdx), %ymm2
 ; CHECK-NEXT:    vmovups %ymm2, (%rsp) # 32-byte Spill
 ; CHECK-NEXT:    callq dummy
 ; CHECK-NEXT:    vmovups {{[0-9]+}}(%rsp), %ymm0 # 32-byte Reload
 ; CHECK-NEXT:    vmovaps %ymm0, (%rbx)
 ; CHECK-NEXT:    vmovups {{[0-9]+}}(%rsp), %ymm0 # 32-byte Reload
 ; CHECK-NEXT:    vmovaps %ymm0, (%r15)
 ; CHECK-NEXT:    vmovups (%rsp), %ymm0 # 32-byte Reload
 ; CHECK-NEXT:    vmovaps %ymm0, (%r14)

llvm/trunk/test/CodeGen/X86/avx512-bugfix-25270.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-apple-darwin -mcpu=knl | FileCheck %s
 
 declare void @Print__512(<16 x i32>) #0
 
 define void @bar__512(<16 x i32>* %var) #0 {
 ; CHECK-LABEL: bar__512:
 ; CHECK:       ## %bb.0: ## %allocas
 ; CHECK-NEXT:    pushq %rbx
 ; CHECK-NEXT:    subq $112, %rsp
 ; CHECK-NEXT:    movq %rdi, %rbx
-; CHECK-NEXT:    vmovups (%rbx), %zmm0
+; CHECK-NEXT:    vmovups (%rdi), %zmm0
 ; CHECK-NEXT:    vmovups %zmm0, (%rsp) ## 64-byte Spill
 ; CHECK-NEXT:    vbroadcastss {{.*}}(%rip), %zmm1
-; CHECK-NEXT:    vmovaps %zmm1, (%rbx)
+; CHECK-NEXT:    vmovaps %zmm1, (%rdi)
 ; CHECK-NEXT:    callq _Print__512
 ; CHECK-NEXT:    vmovups (%rsp), %zmm0 ## 64-byte Reload
 ; CHECK-NEXT:    callq _Print__512
 ; CHECK-NEXT:    vbroadcastss {{.*}}(%rip), %zmm0
 ; CHECK-NEXT:    vmovaps %zmm0, (%rbx)
 ; CHECK-NEXT:    addq $112, %rsp
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    retq

llvm/trunk/test/CodeGen/X86/avx512-calling-conv.ll

 ; KNL_X32-NEXT:    .cfi_offset %ebx, -8
 ; KNL_X32-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; KNL_X32-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; KNL_X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; KNL_X32-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; KNL_X32-NEXT:    movl %edi, (%esp)
 ; KNL_X32-NEXT:    calll _test11
 ; KNL_X32-NEXT:    movl %eax, %ebx
-; KNL_X32-NEXT:    movzbl %bl, %eax
+; KNL_X32-NEXT:    movzbl %al, %eax
 ; KNL_X32-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; KNL_X32-NEXT:    movl %esi, {{[0-9]+}}(%esp)
 ; KNL_X32-NEXT:    movl %edi, (%esp)
 ; KNL_X32-NEXT:    calll _test10
 ; KNL_X32-NEXT:    xorl %ecx, %ecx
 ; KNL_X32-NEXT:    testb $1, %bl
 ; KNL_X32-NEXT:    cmovel %ecx, %eax
 ; KNL_X32-NEXT:    addl $16, %esp

llvm/trunk/test/CodeGen/X86/avx512-regcall-NoMask.ll

 ; Test regcall when passing/retrieving 128 bit vector
 define x86_regcallcc <4 x i32> @test_CallargRet128Vector(<4 x i32> %a)  {
 ; X32-LABEL: test_CallargRet128Vector:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushl %esp
 ; X32-NEXT:    subl $24, %esp
 ; X32-NEXT:    vmovups %xmm4, (%esp) # 16-byte Spill
 ; X32-NEXT:    vmovdqa %xmm0, %xmm4
-; X32-NEXT:    vmovdqa %xmm4, %xmm1
+; X32-NEXT:    vmovdqa %xmm0, %xmm1
 ; X32-NEXT:    calll _test_argRet128Vector
 ; X32-NEXT:    vmovdqa32 %xmm4, %xmm0 {%k1}
 ; X32-NEXT:    vmovups (%esp), %xmm4 # 16-byte Reload
 ; X32-NEXT:    addl $24, %esp
 ; X32-NEXT:    popl %esp
 ; X32-NEXT:    retl
 ;
 ; WIN64-LABEL: test_CallargRet128Vector:
 ; WIN64:       # %bb.0:
 ; WIN64-NEXT:    pushq %rsp
 ; WIN64-NEXT:    .seh_pushreg 4
 ; WIN64-NEXT:    subq $16, %rsp
 ; WIN64-NEXT:    .seh_stackalloc 16
 ; WIN64-NEXT:    vmovaps %xmm8, (%rsp) # 16-byte Spill
 ; WIN64-NEXT:    .seh_savexmm 8, 0
 ; WIN64-NEXT:    .seh_endprologue
 ; WIN64-NEXT:    vmovdqa %xmm0, %xmm8
-; WIN64-NEXT:    vmovdqa %xmm8, %xmm1
+; WIN64-NEXT:    vmovdqa %xmm0, %xmm1
 ; WIN64-NEXT:    callq test_argRet128Vector
 ; WIN64-NEXT:    vmovdqa32 %xmm8, %xmm0 {%k1}
 ; WIN64-NEXT:    vmovaps (%rsp), %xmm8 # 16-byte Reload
 ; WIN64-NEXT:    addq $16, %rsp
 ; WIN64-NEXT:    popq %rsp
 ; WIN64-NEXT:    retq
 ; WIN64-NEXT:    .seh_handlerdata
 ; WIN64-NEXT:    .text
 ; WIN64-NEXT:    .seh_endproc
 ;
 ; LINUXOSX64-LABEL: test_CallargRet128Vector:
 ; LINUXOSX64:       # %bb.0:
 ; LINUXOSX64-NEXT:    pushq %rsp
 ; LINUXOSX64-NEXT:    .cfi_def_cfa_offset 16
 ; LINUXOSX64-NEXT:    subq $16, %rsp
 ; LINUXOSX64-NEXT:    vmovaps %xmm8, (%rsp) # 16-byte Spill
 ; LINUXOSX64-NEXT:    .cfi_def_cfa_offset 32
 ; LINUXOSX64-NEXT:    .cfi_offset %rsp, -16
 ; LINUXOSX64-NEXT:    .cfi_offset %xmm8, -32
 ; LINUXOSX64-NEXT:    vmovdqa %xmm0, %xmm8
-; LINUXOSX64-NEXT:    vmovdqa %xmm8, %xmm1
+; LINUXOSX64-NEXT:    vmovdqa %xmm0, %xmm1
 ; LINUXOSX64-NEXT:    callq test_argRet128Vector
 ; LINUXOSX64-NEXT:    vmovdqa32 %xmm8, %xmm0 {%k1}
 ; LINUXOSX64-NEXT:    vmovaps (%rsp), %xmm8 # 16-byte Reload
 ; LINUXOSX64-NEXT:    addq $16, %rsp
 ; LINUXOSX64-NEXT:    popq %rsp
 ; LINUXOSX64-NEXT:    retq
   %b = call x86_regcallcc <4 x i32> @test_argRet128Vector(<4 x i32> %a, <4 x i32> %a)
   %c = select <4 x i1> undef , <4 x i32> %a, <4 x i32> %b

 define x86_regcallcc i32 @testi32_inp(i32 %a1, i32 %a2, i32 %a3, i32 %a4, i32 %a5, i32 %a6, i32 %b1, i32 %b2, i32 %b3, i32 %b4, i32 %b5, i32 %b6) nounwind {
 ; X32-LABEL: testi32_inp:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushl %ebp
 ; X32-NEXT:    pushl %ebx
 ; X32-NEXT:    subl $20, %esp
 ; X32-NEXT:    movl %esi, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %edi, %esi
-; X32-NEXT:    movl %esi, {{[0-9]+}}(%esp) # 4-byte Spill
+; X32-NEXT:    movl %edi, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %ebx
-; X32-NEXT:    movl %ebx, (%esp) # 4-byte Spill
+; X32-NEXT:    movl %edx, (%esp) # 4-byte Spill
 ; X32-NEXT:    movl %ecx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, %edx
-; X32-NEXT:    movl %edx, {{[0-9]+}}(%esp) # 4-byte Spill
+; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    subl %ecx, %edx
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %edi
 ; X32-NEXT:    movl %edi, %ebp
 ; X32-NEXT:    subl {{[0-9]+}}(%esp), %ebp
 ; X32-NEXT:    imull %ebp, %edx
 ; X32-NEXT:    subl %esi, %ebx
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X32-NEXT:    movl %esi, %ecx

llvm/trunk/test/CodeGen/X86/avx512bw-intrinsics-fast-isel.ll

 ; X32-NEXT:    kmovd %edx, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    shrb $3, %cl
 ; X32-NEXT:    kmovd %ecx, %k2
 ; X32-NEXT:    movzwl %si, %edx
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    shrl $12, %ecx
 ; X32-NEXT:    kmovd %ecx, %k3
-; X32-NEXT:    movl %esi, %ecx
+; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    shrl $13, %ecx
 ; X32-NEXT:    andb $1, %cl
 ; X32-NEXT:    kmovd %ecx, %k4
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    shrl $14, %ecx
 ; X32-NEXT:    kmovd %ecx, %k5
 ; X32-NEXT:    shrl $16, %eax
 ; X32-NEXT:    movl %eax, %ebx

 ; X32-NEXT:    shrb $3, %dl
 ; X32-NEXT:    kmovd %edx, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $36, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $28, %k0, %k1
 ; X32-NEXT:    movl %esi, %ecx
-; X32-NEXT:    movl %ecx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    shrl $28, %eax
 ; X32-NEXT:    kmovd %eax, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $35, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $29, %k0, %k1
-; X32-NEXT:    movl %ecx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    shrl $29, %eax
 ; X32-NEXT:    andb $1, %al
 ; X32-NEXT:    kmovd %eax, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $34, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $30, %k0, %k1
-; X32-NEXT:    movl %ecx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    shrl $30, %eax
 ; X32-NEXT:    kmovd %eax, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $33, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $31, %k0, %k1
 ; X32-NEXT:    shrl $31, %ecx

 ; X32-NEXT:    kmovd %edx, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    shrb $3, %cl
 ; X32-NEXT:    kmovd %ecx, %k2
 ; X32-NEXT:    movzwl %si, %edx
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    shrl $12, %ecx
 ; X32-NEXT:    kmovd %ecx, %k3
-; X32-NEXT:    movl %esi, %ecx
+; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    shrl $13, %ecx
 ; X32-NEXT:    andb $1, %cl
 ; X32-NEXT:    kmovd %ecx, %k4
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    shrl $14, %ecx
 ; X32-NEXT:    kmovd %ecx, %k5
 ; X32-NEXT:    shrl $16, %eax
 ; X32-NEXT:    movl %eax, %ebx

 ; X32-NEXT:    shrb $3, %dl
 ; X32-NEXT:    kmovd %edx, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $36, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $28, %k0, %k1
 ; X32-NEXT:    movl %esi, %ecx
-; X32-NEXT:    movl %ecx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    shrl $28, %eax
 ; X32-NEXT:    kmovd %eax, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $35, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $29, %k0, %k1
-; X32-NEXT:    movl %ecx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    shrl $29, %eax
 ; X32-NEXT:    andb $1, %al
 ; X32-NEXT:    kmovd %eax, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $34, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $30, %k0, %k1
-; X32-NEXT:    movl %ecx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    shrl $30, %eax
 ; X32-NEXT:    kmovd %eax, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $33, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $31, %k0, %k1
 ; X32-NEXT:    shrl $31, %ecx

 ; X32-NEXT:    kmovd %edx, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    shrb $3, %cl
 ; X32-NEXT:    kmovd %ecx, %k2
 ; X32-NEXT:    movzwl %si, %edx
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    shrl $12, %ecx
 ; X32-NEXT:    kmovd %ecx, %k3
-; X32-NEXT:    movl %esi, %ecx
+; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    shrl $13, %ecx
 ; X32-NEXT:    andb $1, %cl
 ; X32-NEXT:    kmovd %ecx, %k4
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    shrl $14, %ecx
 ; X32-NEXT:    kmovd %ecx, %k5
 ; X32-NEXT:    shrl $16, %eax
 ; X32-NEXT:    movl %eax, %ebx

 ; X32-NEXT:    shrb $3, %dl
 ; X32-NEXT:    kmovd %edx, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $36, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $28, %k0, %k1
 ; X32-NEXT:    movl %esi, %ecx
-; X32-NEXT:    movl %ecx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    shrl $28, %eax
 ; X32-NEXT:    kmovd %eax, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $35, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $29, %k0, %k1
-; X32-NEXT:    movl %ecx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    shrl $29, %eax
 ; X32-NEXT:    andb $1, %al
 ; X32-NEXT:    kmovd %eax, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $34, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $30, %k0, %k1
-; X32-NEXT:    movl %ecx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    shrl $30, %eax
 ; X32-NEXT:    kmovd %eax, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $33, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $31, %k0, %k1
 ; X32-NEXT:    shrl $31, %ecx

 ; X32-NEXT:    kmovd %edx, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    shrb $3, %cl
 ; X32-NEXT:    kmovd %ecx, %k2
 ; X32-NEXT:    movzwl %si, %edx
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    shrl $12, %ecx
 ; X32-NEXT:    kmovd %ecx, %k3
-; X32-NEXT:    movl %esi, %ecx
+; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    shrl $13, %ecx
 ; X32-NEXT:    andb $1, %cl
 ; X32-NEXT:    kmovd %ecx, %k4
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    shrl $14, %ecx
 ; X32-NEXT:    kmovd %ecx, %k5
 ; X32-NEXT:    shrl $16, %eax
 ; X32-NEXT:    movl %eax, %ebx

 ; X32-NEXT:    shrb $3, %dl
 ; X32-NEXT:    kmovd %edx, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $36, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $28, %k0, %k1
 ; X32-NEXT:    movl %esi, %ecx
-; X32-NEXT:    movl %ecx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    shrl $28, %eax
 ; X32-NEXT:    kmovd %eax, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $35, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $29, %k0, %k1
-; X32-NEXT:    movl %ecx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    shrl $29, %eax
 ; X32-NEXT:    andb $1, %al
 ; X32-NEXT:    kmovd %eax, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $34, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $30, %k0, %k1
-; X32-NEXT:    movl %ecx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    shrl $30, %eax
 ; X32-NEXT:    kmovd %eax, %k2
 ; X32-NEXT:    kxorq %k2, %k1, %k1
 ; X32-NEXT:    kshiftlq $63, %k1, %k1
 ; X32-NEXT:    kshiftrq $33, %k1, %k1
 ; X32-NEXT:    kxorq %k1, %k0, %k0
 ; X32-NEXT:    kshiftrq $31, %k0, %k1
 ; X32-NEXT:    shrl $31, %ecx

llvm/trunk/test/CodeGen/X86/avx512bw-intrinsics-upgrade.ll

 ; AVX512F-32-NEXT:    andb $15, %dl
 ; AVX512F-32-NEXT:    shrb $3, %cl
 ; AVX512F-32-NEXT:    kshiftlq $63, %k7, %k7
 ; AVX512F-32-NEXT:    kshiftrq $59, %k7, %k7
 ; AVX512F-32-NEXT:    kxorq %k7, %k0, %k7
 ; AVX512F-32-NEXT:    kshiftrq $5, %k7, %k0
 ; AVX512F-32-NEXT:    kxorq %k4, %k0, %k4
 ; AVX512F-32-NEXT:    kmovd %ecx, %k0
-; AVX512F-32-NEXT:    movl %ebp, %ecx
+; AVX512F-32-NEXT:    movl %ebx, %ecx
 ; AVX512F-32-NEXT:    shrl $13, %ecx
 ; AVX512F-32-NEXT:    andb $1, %cl
 ; AVX512F-32-NEXT:    kshiftlq $63, %k4, %k4
 ; AVX512F-32-NEXT:    kshiftrq $58, %k4, %k4
 ; AVX512F-32-NEXT:    kxorq %k4, %k7, %k7
 ; AVX512F-32-NEXT:    kshiftrq $6, %k7, %k4
 ; AVX512F-32-NEXT:    kxorq %k6, %k4, %k6
 ; AVX512F-32-NEXT:    kmovd %ecx, %k4

 ; AVX512F-32-NEXT:    andb $15, %dl
 ; AVX512F-32-NEXT:    shrb $3, %cl
 ; AVX512F-32-NEXT:    kshiftlq $63, %k7, %k7
 ; AVX512F-32-NEXT:    kshiftrq $59, %k7, %k7
 ; AVX512F-32-NEXT:    kxorq %k7, %k0, %k7
 ; AVX512F-32-NEXT:    kshiftrq $5, %k7, %k0
 ; AVX512F-32-NEXT:    kxorq %k4, %k0, %k4
 ; AVX512F-32-NEXT:    kmovd %ecx, %k0
-; AVX512F-32-NEXT:    movl %ebp, %ecx
+; AVX512F-32-NEXT:    movl %ebx, %ecx
 ; AVX512F-32-NEXT:    shrl $13, %ecx
 ; AVX512F-32-NEXT:    andb $1, %cl
 ; AVX512F-32-NEXT:    kshiftlq $63, %k4, %k4
 ; AVX512F-32-NEXT:    kshiftrq $58, %k4, %k4
 ; AVX512F-32-NEXT:    kxorq %k4, %k7, %k7
 ; AVX512F-32-NEXT:    kshiftrq $6, %k7, %k4
 ; AVX512F-32-NEXT:    kxorq %k6, %k4, %k6
 ; AVX512F-32-NEXT:    kmovd %ecx, %k4

llvm/trunk/test/CodeGen/X86/buildvec-insertvec.ll

 
 ; Verify that the DAGCombiner doesn't wrongly fold a build_vector into a
 ; blend with a zero vector if the build_vector contains negative zero.
 
 define <4 x float> @test_negative_zero_1(<4 x float> %A) {
 ; SSE2-LABEL: test_negative_zero_1:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movaps %xmm0, %xmm1
-; SSE2-NEXT:    movhlps {{.*#+}} xmm1 = xmm1[1,1]
+; SSE2-NEXT:    movhlps {{.*#+}} xmm1 = xmm0[1],xmm1[1]
 ; SSE2-NEXT:    xorps %xmm2, %xmm2
 ; SSE2-NEXT:    movss {{.*#+}} xmm2 = xmm1[0],xmm2[1,2,3]
 ; SSE2-NEXT:    movss {{.*#+}} xmm1 = mem[0],zero,zero,zero
 ; SSE2-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
 ; SSE2-NEXT:    movlhps {{.*#+}} xmm0 = xmm0[0],xmm2[0]
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: test_negative_zero_1:

llvm/trunk/test/CodeGen/X86/combine-fcopysign.ll

 ; copysign(x, fp_extend(y)) -> copysign(x, y)
 define <4 x double> @combine_vec_fcopysign_fpext_sgn(<4 x double> %x, <4 x float> %y) {
 ; SSE-LABEL: combine_vec_fcopysign_fpext_sgn:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movaps %xmm2, %xmm3
 ; SSE-NEXT:    cvtss2sd %xmm2, %xmm4
 ; SSE-NEXT:    movshdup {{.*#+}} xmm5 = xmm2[1,1,3,3]
 ; SSE-NEXT:    movaps %xmm2, %xmm6
-; SSE-NEXT:    movhlps {{.*#+}} xmm6 = xmm6[1,1]
-; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[3,1,2,3]
+; SSE-NEXT:    movhlps {{.*#+}} xmm6 = xmm2[1],xmm6[1]
+; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[3,1],xmm2[2,3]
 ; SSE-NEXT:    movaps {{.*#+}} xmm7
 ; SSE-NEXT:    movaps %xmm0, %xmm2
 ; SSE-NEXT:    andps %xmm7, %xmm2
 ; SSE-NEXT:    movaps {{.*#+}} xmm8 = [-0.000000e+00,-0.000000e+00]
 ; SSE-NEXT:    andps %xmm8, %xmm4
 ; SSE-NEXT:    orps %xmm4, %xmm2
 ; SSE-NEXT:    movhlps {{.*#+}} xmm0 = xmm0[1,1]
 ; SSE-NEXT:    andps %xmm7, %xmm0
 ; SSE-NEXT:    xorps %xmm4, %xmm4
 ; SSE-NEXT:    cvtss2sd %xmm5, %xmm4
 ; SSE-NEXT:    andps %xmm8, %xmm4
 ; SSE-NEXT:    orps %xmm0, %xmm4
 ; SSE-NEXT:    movlhps {{.*#+}} xmm2 = xmm2[0],xmm4[0]
 ; SSE-NEXT:    movaps %xmm1, %xmm0
-; SSE-NEXT:    movhlps {{.*#+}} xmm0 = xmm0[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm0 = xmm1[1],xmm0[1]
 ; SSE-NEXT:    andps %xmm7, %xmm0
 ; SSE-NEXT:    cvtss2sd %xmm3, %xmm3
 ; SSE-NEXT:    andps %xmm8, %xmm3
 ; SSE-NEXT:    orps %xmm0, %xmm3
 ; SSE-NEXT:    andps %xmm7, %xmm1
 ; SSE-NEXT:    xorps %xmm0, %xmm0
 ; SSE-NEXT:    cvtss2sd %xmm6, %xmm0
 ; SSE-NEXT:    andps %xmm8, %xmm0

 ; SSE-NEXT:    movshdup {{.*#+}} xmm6 = xmm3[1,1,3,3]
 ; SSE-NEXT:    andps %xmm5, %xmm6
 ; SSE-NEXT:    movhlps {{.*#+}} xmm1 = xmm1[1,1]
 ; SSE-NEXT:    cvtsd2ss %xmm1, %xmm1
 ; SSE-NEXT:    andps %xmm4, %xmm1
 ; SSE-NEXT:    orps %xmm6, %xmm1
 ; SSE-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
 ; SSE-NEXT:    movaps %xmm3, %xmm1
-; SSE-NEXT:    movhlps {{.*#+}} xmm1 = xmm1[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm1 = xmm3[1],xmm1[1]
 ; SSE-NEXT:    andps %xmm5, %xmm1
 ; SSE-NEXT:    xorps %xmm6, %xmm6
 ; SSE-NEXT:    cvtsd2ss %xmm2, %xmm6
 ; SSE-NEXT:    andps %xmm4, %xmm6
 ; SSE-NEXT:    orps %xmm1, %xmm6
 ; SSE-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0,1],xmm6[0],xmm0[3]
 ; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[3,1,2,3]
 ; SSE-NEXT:    andps %xmm5, %xmm3

llvm/trunk/test/CodeGen/X86/combine-shl.ll

; AVX-NEXT:    retq
   ret <4 x i32> %2
 }
 
 ; fold (shl (ext (shl x, c1)), c2) -> (ext (shl x, (add c1, c2)))
 define <8 x i32> @combine_vec_shl_ext_shl0(<8 x i16> %x) {
 ; SSE-LABEL: combine_vec_shl_ext_shl0:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movdqa %xmm0, %xmm1
-; SSE-NEXT:    pmovzxwd {{.*#+}} xmm0 = xmm1[0],zero,xmm1[1],zero,xmm1[2],zero,xmm1[3],zero
+; SSE-NEXT:    pmovzxwd {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero
 ; SSE-NEXT:    punpckhwd {{.*#+}} xmm1 = xmm1[4],xmm0[4],xmm1[5],xmm0[5],xmm1[6],xmm0[6],xmm1[7],xmm0[7]
 ; SSE-NEXT:    pslld $20, %xmm1
 ; SSE-NEXT:    pslld $20, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: combine_vec_shl_ext_shl0:
 ; AVX:       # %bb.0:
 ; AVX-NEXT:    vpmovzxwd {{.*#+}} ymm0 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero,xmm0[4],zero,xmm0[5],zero,xmm0[6],zero,xmm0[7],zero

llvm/trunk/test/CodeGen/X86/complex-fastmath.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+sse4.2 | FileCheck %s --check-prefix=ALL --check-prefix=SSE
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx  | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=AVX1
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx2,+fma | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=FMA
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512f,+avx512vl | FileCheck %s --check-prefix=ALL --check-prefix=AVX --check-prefix=FMA
 
 ; PR31866
 ; complex float complex_square_f32(complex float x) {
 ;   return x*x;
 ; }
 
 define <2 x float> @complex_square_f32(<2 x float>) #0 {
 ; SSE-LABEL: complex_square_f32:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
 ; SSE-NEXT:    movaps %xmm0, %xmm2
-; SSE-NEXT:    addss %xmm2, %xmm2
+; SSE-NEXT:    addss %xmm0, %xmm2
 ; SSE-NEXT:    mulss %xmm1, %xmm2
 ; SSE-NEXT:    mulss %xmm0, %xmm0
 ; SSE-NEXT:    mulss %xmm1, %xmm1
 ; SSE-NEXT:    subss %xmm1, %xmm0
 ; SSE-NEXT:    insertps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[2,3]
 ; SSE-NEXT:    retq
 ;
 ; AVX1-LABEL: complex_square_f32:
; FMA-NEXT:    retq
   %10 = insertelement <2 x float> %9, float %5, i32 1
   ret <2 x float> %10
 }
 
 define <2 x double> @complex_square_f64(<2 x double>) #0 {
 ; SSE-LABEL: complex_square_f64:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movaps %xmm0, %xmm1
-; SSE-NEXT:    movhlps {{.*#+}} xmm1 = xmm1[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm1 = xmm0[1],xmm1[1]
 ; SSE-NEXT:    movaps %xmm0, %xmm2
-; SSE-NEXT:    addsd %xmm2, %xmm2
+; SSE-NEXT:    addsd %xmm0, %xmm2
 ; SSE-NEXT:    mulsd %xmm1, %xmm2
 ; SSE-NEXT:    mulsd %xmm0, %xmm0
 ; SSE-NEXT:    mulsd %xmm1, %xmm1
 ; SSE-NEXT:    subsd %xmm1, %xmm0
 ; SSE-NEXT:    unpcklpd {{.*#+}} xmm0 = xmm0[0],xmm2[0]
 ; SSE-NEXT:    retq
 ;
 ; AVX1-LABEL: complex_square_f64:
; FMA-NEXT:    retq
   %14 = insertelement <2 x float> %13, float %9, i32 1
   ret <2 x float> %14
 }
 
 define <2 x double> @complex_mul_f64(<2 x double>, <2 x double>) #0 {
 ; SSE-LABEL: complex_mul_f64:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movaps %xmm0, %xmm2
-; SSE-NEXT:    movhlps {{.*#+}} xmm2 = xmm2[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm2 = xmm0[1],xmm2[1]
 ; SSE-NEXT:    movaps %xmm1, %xmm3
-; SSE-NEXT:    movhlps {{.*#+}} xmm3 = xmm3[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm3 = xmm1[1],xmm3[1]
 ; SSE-NEXT:    movaps %xmm3, %xmm4
 ; SSE-NEXT:    mulsd %xmm0, %xmm4
 ; SSE-NEXT:    mulsd %xmm1, %xmm0
 ; SSE-NEXT:    mulsd %xmm2, %xmm1
 ; SSE-NEXT:    addsd %xmm4, %xmm1
 ; SSE-NEXT:    mulsd %xmm2, %xmm3
 ; SSE-NEXT:    subsd %xmm3, %xmm0
 ; SSE-NEXT:    unpcklpd {{.*#+}} xmm0 = xmm0[0],xmm1[0]

llvm/trunk/test/CodeGen/X86/divide-by-constant.ll

 ; X32-NEXT:    calll __udivdi3
 ; X32-NEXT:    addl $28, %esp
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: PR23590:
 ; X64:       # %bb.0: # %entry
 ; X64-NEXT:    movq %rdi, %rcx
 ; X64-NEXT:    movabsq $6120523590596543007, %rdx # imm = 0x54F077C718E7C21F
-; X64-NEXT:    movq %rcx, %rax
+; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    mulq %rdx
 ; X64-NEXT:    shrq $12, %rdx
 ; X64-NEXT:    imulq $12345, %rdx, %rax # imm = 0x3039
 ; X64-NEXT:    subq %rax, %rcx
 ; X64-NEXT:    movabsq $2635249153387078803, %rdx # imm = 0x2492492492492493
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    mulq %rdx
 ; X64-NEXT:    subq %rdx, %rcx

llvm/trunk/test/CodeGen/X86/fmaxnum.ll

 declare <2 x double> @llvm.maxnum.v2f64(<2 x double>, <2 x double>)
 declare <4 x double> @llvm.maxnum.v4f64(<4 x double>, <4 x double>)
 declare <8 x double> @llvm.maxnum.v8f64(<8 x double>, <8 x double>)
 
 ; FIXME: As the vector tests show, the SSE run shouldn't need this many moves.
 
 ; CHECK-LABEL: @test_fmaxf
 ; SSE:         movaps %xmm0, %xmm2
-; SSE-NEXT:    cmpunordss %xmm2, %xmm2
+; SSE-NEXT:    cmpunordss %xmm0, %xmm2
 ; SSE-NEXT:    movaps %xmm2, %xmm3
 ; SSE-NEXT:    andps %xmm1, %xmm3
 ; SSE-NEXT:    maxss %xmm0, %xmm1
 ; SSE-NEXT:    andnps %xmm1, %xmm2
 ; SSE-NEXT:    orps %xmm3, %xmm2
 ; SSE-NEXT:    movaps %xmm2, %xmm0
 ; SSE-NEXT:    retq
 ;
define float @test_fmaxf_minsize(float %x, float %y) minsize {
   %z = call float @fmaxf(float %x, float %y) readnone
   ret float %z
 }
 
 ; FIXME: As the vector tests show, the SSE run shouldn't need this many moves.
 
 ; CHECK-LABEL: @test_fmax
 ; SSE:         movapd %xmm0, %xmm2
-; SSE-NEXT:    cmpunordsd %xmm2, %xmm2
+; SSE-NEXT:    cmpunordsd %xmm0, %xmm2
 ; SSE-NEXT:    movapd %xmm2, %xmm3
 ; SSE-NEXT:    andpd %xmm1, %xmm3
 ; SSE-NEXT:    maxsd %xmm0, %xmm1
 ; SSE-NEXT:    andnpd %xmm1, %xmm2
 ; SSE-NEXT:    orpd %xmm3, %xmm2
 ; SSE-NEXT:    movapd %xmm2, %xmm0
 ; SSE-NEXT:    retq
 ;

 ; CHECK: callq fmaxl
 define x86_fp80 @test_fmaxl(x86_fp80 %x, x86_fp80 %y) {
   %z = call x86_fp80 @fmaxl(x86_fp80 %x, x86_fp80 %y) readnone
   ret x86_fp80 %z
 }
 
 ; CHECK-LABEL: @test_intrinsic_fmaxf
 ; SSE:         movaps %xmm0, %xmm2
-; SSE-NEXT:    cmpunordss %xmm2, %xmm2
+; SSE-NEXT:    cmpunordss %xmm0, %xmm2
 ; SSE-NEXT:    movaps %xmm2, %xmm3
 ; SSE-NEXT:    andps %xmm1, %xmm3
 ; SSE-NEXT:    maxss %xmm0, %xmm1
 ; SSE-NEXT:    andnps %xmm1, %xmm2
 ; SSE-NEXT:    orps %xmm3, %xmm2
 ; SSE-NEXT:    movaps %xmm2, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX:         vmaxss %xmm0, %xmm1, %xmm2
 ; AVX-NEXT:    vcmpunordss %xmm0, %xmm0, %xmm0
 ; AVX-NEXT:    vblendvps %xmm0, %xmm1, %xmm2, %xmm0
 ; AVX-NEXT:    retq
 define float @test_intrinsic_fmaxf(float %x, float %y) {
   %z = call float @llvm.maxnum.f32(float %x, float %y) readnone
   ret float %z
 }
 
 
 ; CHECK-LABEL: @test_intrinsic_fmax
 ; SSE:         movapd %xmm0, %xmm2
-; SSE-NEXT:    cmpunordsd %xmm2, %xmm2
+; SSE-NEXT:    cmpunordsd %xmm0, %xmm2
 ; SSE-NEXT:    movapd %xmm2, %xmm3
 ; SSE-NEXT:    andpd %xmm1, %xmm3
 ; SSE-NEXT:    maxsd %xmm0, %xmm1
 ; SSE-NEXT:    andnpd %xmm1, %xmm2
 ; SSE-NEXT:    orpd %xmm3, %xmm2
 ; SSE-NEXT:    movapd %xmm2, %xmm0
 ; SSE-NEXT:    retq
 ;

llvm/trunk/test/CodeGen/X86/fmf-flags.ll

 }
 
 declare float @llvm.fmuladd.f32(float %a, float %b, float %c);
 
 define float @fast_fmuladd_opts(float %a , float %b , float %c) {
 ; X64-LABEL: fast_fmuladd_opts:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movaps %xmm0, %xmm1
-; X64-NEXT:    addss %xmm1, %xmm1
+; X64-NEXT:    addss %xmm0, %xmm1
 ; X64-NEXT:    addss %xmm0, %xmm1
 ; X64-NEXT:    movaps %xmm1, %xmm0
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: fast_fmuladd_opts:
 ; X86:       # %bb.0:
 ; X86-NEXT:    flds {{[0-9]+}}(%esp)
 ; X86-NEXT:    fld %st(0)

llvm/trunk/test/CodeGen/X86/fminnum.ll

 declare <2 x double> @llvm.minnum.v2f64(<2 x double>, <2 x double>)
 declare <4 x double> @llvm.minnum.v4f64(<4 x double>, <4 x double>)
 declare <8 x double> @llvm.minnum.v8f64(<8 x double>, <8 x double>)
 
 ; FIXME: As the vector tests show, the SSE run shouldn't need this many moves.
 
 ; CHECK-LABEL: @test_fminf
 ; SSE:         movaps %xmm0, %xmm2
-; SSE-NEXT:    cmpunordss %xmm2, %xmm2
+; SSE-NEXT:    cmpunordss %xmm0, %xmm2
 ; SSE-NEXT:    movaps %xmm2, %xmm3
 ; SSE-NEXT:    andps %xmm1, %xmm3
 ; SSE-NEXT:    minss %xmm0, %xmm1
 ; SSE-NEXT:    andnps %xmm1, %xmm2
 ; SSE-NEXT:    orps %xmm3, %xmm2
 ; SSE-NEXT:    movaps %xmm2, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX:         vminss %xmm0, %xmm1, %xmm2
 ; AVX-NEXT:    vcmpunordss %xmm0, %xmm0, %xmm0
 ; AVX-NEXT:    vblendvps %xmm0, %xmm1, %xmm2, %xmm0
 ; AVX-NEXT:    retq
 define float @test_fminf(float %x, float %y) {
   %z = call float @fminf(float %x, float %y) readnone
   ret float %z
 }
 
 ; FIXME: As the vector tests show, the SSE run shouldn't need this many moves.
 
 ; CHECK-LABEL: @test_fmin
 ; SSE:         movapd %xmm0, %xmm2
-; SSE-NEXT:    cmpunordsd %xmm2, %xmm2
+; SSE-NEXT:    cmpunordsd %xmm0, %xmm2
 ; SSE-NEXT:    movapd %xmm2, %xmm3
 ; SSE-NEXT:    andpd %xmm1, %xmm3
 ; SSE-NEXT:    minsd %xmm0, %xmm1
 ; SSE-NEXT:    andnpd %xmm1, %xmm2
 ; SSE-NEXT:    orpd %xmm3, %xmm2
 ; SSE-NEXT:    movapd %xmm2, %xmm0
 ; SSE-NEXT:    retq
 ;

 ; CHECK: callq fminl
 define x86_fp80 @test_fminl(x86_fp80 %x, x86_fp80 %y) {
   %z = call x86_fp80 @fminl(x86_fp80 %x, x86_fp80 %y) readnone
   ret x86_fp80 %z
 }
 
 ; CHECK-LABEL: @test_intrinsic_fminf
 ; SSE:         movaps %xmm0, %xmm2
-; SSE-NEXT:    cmpunordss %xmm2, %xmm2
+; SSE-NEXT:    cmpunordss %xmm0, %xmm2
 ; SSE-NEXT:    movaps %xmm2, %xmm3
 ; SSE-NEXT:    andps %xmm1, %xmm3
 ; SSE-NEXT:    minss %xmm0, %xmm1
 ; SSE-NEXT:    andnps %xmm1, %xmm2
 ; SSE-NEXT:    orps %xmm3, %xmm2
 ; SSE-NEXT:    movaps %xmm2, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX:         vminss %xmm0, %xmm1, %xmm2
 ; AVX-NEXT:    vcmpunordss %xmm0, %xmm0, %xmm0
 ; AVX-NEXT:    vblendvps %xmm0, %xmm1, %xmm2, %xmm0
 ; AVX-NEXT:    retq
 define float @test_intrinsic_fminf(float %x, float %y) {
   %z = call float @llvm.minnum.f32(float %x, float %y) readnone
   ret float %z
 }
 
 ; CHECK-LABEL: @test_intrinsic_fmin
 ; SSE:         movapd %xmm0, %xmm2
-; SSE-NEXT:    cmpunordsd %xmm2, %xmm2
+; SSE-NEXT:    cmpunordsd %xmm0, %xmm2
 ; SSE-NEXT:    movapd %xmm2, %xmm3
 ; SSE-NEXT:    andpd %xmm1, %xmm3
 ; SSE-NEXT:    minsd %xmm0, %xmm1
 ; SSE-NEXT:    andnpd %xmm1, %xmm2
 ; SSE-NEXT:    orpd %xmm3, %xmm2
 ; SSE-NEXT:    movapd %xmm2, %xmm0
 ; SSE-NEXT:    retq
 ;

llvm/trunk/test/CodeGen/X86/fp128-i128.ll

 ;  df = u.e;
 ;  return x + df;
 ; }
 define fp128 @TestI128_4(fp128 %x) #0 {
 ; CHECK-LABEL: TestI128_4:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    subq $40, %rsp
 ; CHECK-NEXT:    movaps %xmm0, %xmm1
-; CHECK-NEXT:    movaps %xmm1, {{[0-9]+}}(%rsp)
+; CHECK-NEXT:    movaps %xmm0, {{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    movq {{[0-9]+}}(%rsp), %rax
 ; CHECK-NEXT:    movq %rax, {{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    movq $0, (%rsp)
 ; CHECK-NEXT:    movaps (%rsp), %xmm0
 ; CHECK-NEXT:    callq __addtf3
 ; CHECK-NEXT:    addq $40, %rsp
 ; CHECK-NEXT:    retq
 entry:
entry:
   ret void
 }
 
 define fp128 @acosl(fp128 %x) #0 {
 ; CHECK-LABEL: acosl:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    subq $40, %rsp
 ; CHECK-NEXT:    movaps %xmm0, %xmm1
-; CHECK-NEXT:    movaps %xmm1, {{[0-9]+}}(%rsp)
+; CHECK-NEXT:    movaps %xmm0, {{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    movq {{[0-9]+}}(%rsp), %rax
 ; CHECK-NEXT:    movq %rax, {{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    movq $0, (%rsp)
 ; CHECK-NEXT:    movaps (%rsp), %xmm0
 ; CHECK-NEXT:    callq __addtf3
 ; CHECK-NEXT:    addq $40, %rsp
 ; CHECK-NEXT:    retq
 entry:

llvm/trunk/test/CodeGen/X86/h-registers-1.ll

 ; CHECK-NEXT:    movq %r8, %rax
 ; CHECK-NEXT:    movzbl %ah, %ecx
 ; CHECK-NEXT:    movq %r9, %rax
 ; CHECK-NEXT:    movzbl %ah, %ebp
 ; CHECK-NEXT:    movl {{[0-9]+}}(%rsp), %eax
 ; CHECK-NEXT:    movzbl %ah, %eax
 ; CHECK-NEXT:    movl {{[0-9]+}}(%rsp), %ebx
 ; CHECK-NEXT:    movzbl %bh, %edi
-; CHECK-NEXT:    movq %r10, %r8
-; CHECK-NEXT:    addq %r8, %rsi
+; CHECK-NEXT:    addq %r10, %rsi
 ; CHECK-NEXT:    addq %r11, %rdx
 ; CHECK-NEXT:    addq %rsi, %rdx
 ; CHECK-NEXT:    addq %rbp, %rcx
 ; CHECK-NEXT:    addq %rdi, %rax
 ; CHECK-NEXT:    addq %rcx, %rax
 ; CHECK-NEXT:    addq %rdx, %rax
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    popq %rbp

 ; GNUX32-NEXT:    movq %r8, %rax
 ; GNUX32-NEXT:    movzbl %ah, %ecx
 ; GNUX32-NEXT:    movq %r9, %rax
 ; GNUX32-NEXT:    movzbl %ah, %ebp
 ; GNUX32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; GNUX32-NEXT:    movzbl %ah, %eax
 ; GNUX32-NEXT:    movl {{[0-9]+}}(%esp), %ebx
 ; GNUX32-NEXT:    movzbl %bh, %edi
-; GNUX32-NEXT:    movq %r10, %r8
-; GNUX32-NEXT:    addq %r8, %rsi
+; GNUX32-NEXT:    addq %r10, %rsi
 ; GNUX32-NEXT:    addq %r11, %rdx
 ; GNUX32-NEXT:    addq %rsi, %rdx
 ; GNUX32-NEXT:    addq %rbp, %rcx
 ; GNUX32-NEXT:    addq %rdi, %rax
 ; GNUX32-NEXT:    addq %rcx, %rax
 ; GNUX32-NEXT:    addq %rdx, %rax
 ; GNUX32-NEXT:    popq %rbx
 ; GNUX32-NEXT:    popq %rbp

llvm/trunk/test/CodeGen/X86/haddsub-2.ll

; AVX-NEXT:    retq
   %vecinit13 = insertelement <4 x i32> %vecinit9, i32 %sub12, i32 3
   ret <4 x i32> %vecinit13
 }
 
 define <4 x float> @not_a_hsub_2(<4 x float> %A, <4 x float> %B) {
 ; SSE-LABEL: not_a_hsub_2:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movaps %xmm0, %xmm2
-; SSE-NEXT:    movhlps {{.*#+}} xmm2 = xmm2[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm2 = xmm0[1],xmm2[1]
 ; SSE-NEXT:    movaps %xmm0, %xmm3
-; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[3,1,2,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[3,1],xmm0[2,3]
 ; SSE-NEXT:    subss %xmm3, %xmm2
 ; SSE-NEXT:    movshdup {{.*#+}} xmm3 = xmm0[1,1,3,3]
 ; SSE-NEXT:    subss %xmm3, %xmm0
 ; SSE-NEXT:    movaps %xmm1, %xmm3
-; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[3,1,2,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[3,1],xmm1[2,3]
 ; SSE-NEXT:    movaps %xmm1, %xmm4
-; SSE-NEXT:    movhlps {{.*#+}} xmm4 = xmm4[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm4 = xmm1[1],xmm4[1]
 ; SSE-NEXT:    subss %xmm4, %xmm3
 ; SSE-NEXT:    movshdup {{.*#+}} xmm4 = xmm1[1,1,3,3]
 ; SSE-NEXT:    subss %xmm4, %xmm1
 ; SSE-NEXT:    unpcklps {{.*#+}} xmm1 = xmm1[0],xmm3[0],xmm1[1],xmm3[1]
 ; SSE-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
 ; SSE-NEXT:    movlhps {{.*#+}} xmm0 = xmm0[0],xmm1[0]
 ; SSE-NEXT:    retq
 ;
; AVX-NEXT:    retq
   %vecinit13 = insertelement <4 x float> %vecinit9, float %sub12, i32 2
   ret <4 x float> %vecinit13
 }
 
 define <2 x double> @not_a_hsub_3(<2 x double> %A, <2 x double> %B) {
 ; SSE-LABEL: not_a_hsub_3:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movaps %xmm1, %xmm2
-; SSE-NEXT:    movhlps {{.*#+}} xmm2 = xmm2[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm2 = xmm1[1],xmm2[1]
 ; SSE-NEXT:    subsd %xmm2, %xmm1
 ; SSE-NEXT:    movaps %xmm0, %xmm2
-; SSE-NEXT:    movhlps {{.*#+}} xmm2 = xmm2[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm2 = xmm0[1],xmm2[1]
 ; SSE-NEXT:    subsd %xmm0, %xmm2
 ; SSE-NEXT:    unpcklpd {{.*#+}} xmm2 = xmm2[0],xmm1[0]
 ; SSE-NEXT:    movapd %xmm2, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: not_a_hsub_3:
 ; AVX:       # %bb.0:
 ; AVX-NEXT:    vpermilpd {{.*#+}} xmm2 = xmm1[1,0]

llvm/trunk/test/CodeGen/X86/haddsub-3.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown -mattr=+sse2 | FileCheck %s --check-prefixes=SSE,SSE2
 ; RUN: llc < %s -mtriple=x86_64-unknown -mattr=+ssse3 | FileCheck %s --check-prefixes=SSE,SSSE3
 ; RUN: llc < %s -mtriple=x86_64-unknown -mattr=+avx | FileCheck %s --check-prefixes=AVX
 
 define float @pr26491(<4 x float> %a0) {
 ; SSE2-LABEL: pr26491:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movaps %xmm0, %xmm1
-; SSE2-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,1,3,3]
+; SSE2-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,1],xmm0[3,3]
 ; SSE2-NEXT:    addps %xmm0, %xmm1
 ; SSE2-NEXT:    movaps %xmm1, %xmm0
-; SSE2-NEXT:    movhlps {{.*#+}} xmm0 = xmm0[1,1]
+; SSE2-NEXT:    movhlps {{.*#+}} xmm0 = xmm1[1],xmm0[1]
 ; SSE2-NEXT:    addss %xmm1, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: pr26491:
 ; SSSE3:       # %bb.0:
 ; SSSE3-NEXT:    movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
 ; SSSE3-NEXT:    addps %xmm0, %xmm1
 ; SSSE3-NEXT:    movaps %xmm1, %xmm0
-; SSSE3-NEXT:    movhlps {{.*#+}} xmm0 = xmm0[1,1]
+; SSSE3-NEXT:    movhlps {{.*#+}} xmm0 = xmm1[1],xmm0[1]
 ; SSSE3-NEXT:    addss %xmm1, %xmm0
 ; SSSE3-NEXT:    retq
 ;
 ; AVX-LABEL: pr26491:
 ; AVX:       # %bb.0:
 ; AVX-NEXT:    vmovshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
 ; AVX-NEXT:    vaddps %xmm0, %xmm1, %xmm0
 ; AVX-NEXT:    vpermilpd {{.*#+}} xmm1 = xmm0[1,0]

llvm/trunk/test/CodeGen/X86/haddsub-undef.ll

; AVX-NEXT:    retq
   %vecinit = insertelement <4 x float> undef, float %add, i32 0
   ret <4 x float> %vecinit
 }
 
 define <2 x double> @test5_undef(<2 x double> %a, <2 x double> %b) {
 ; SSE-LABEL: test5_undef:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movaps %xmm0, %xmm1
-; SSE-NEXT:    movhlps {{.*#+}} xmm1 = xmm1[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm1 = xmm0[1],xmm1[1]
 ; SSE-NEXT:    addsd %xmm0, %xmm1
 ; SSE-NEXT:    movapd %xmm1, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: test5_undef:
 ; AVX:       # %bb.0:
 ; AVX-NEXT:    vpermilpd {{.*#+}} xmm1 = xmm0[1,0]
 ; AVX-NEXT:    vaddsd %xmm1, %xmm0, %xmm0

 }
 
 define <4 x float> @test8_undef(<4 x float> %a, <4 x float> %b) {
 ; SSE-LABEL: test8_undef:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movshdup {{.*#+}} xmm1 = xmm0[1,1,3,3]
 ; SSE-NEXT:    addss %xmm0, %xmm1
 ; SSE-NEXT:    movaps %xmm0, %xmm2
-; SSE-NEXT:    movhlps {{.*#+}} xmm2 = xmm2[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm2 = xmm0[1],xmm2[1]
 ; SSE-NEXT:    shufps {{.*#+}} xmm0 = xmm0[3,1,2,3]
 ; SSE-NEXT:    addss %xmm2, %xmm0
 ; SSE-NEXT:    movlhps {{.*#+}} xmm1 = xmm1[0],xmm0[0]
 ; SSE-NEXT:    movaps %xmm1, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: test8_undef:
 ; AVX:       # %bb.0:

llvm/trunk/test/CodeGen/X86/half.ll

 }
 
 define <4 x float> @test_extend32_vec4(<4 x half>* %p) #0 {
 ; CHECK-LIBCALL-LABEL: test_extend32_vec4:
 ; CHECK-LIBCALL:       # %bb.0:
 ; CHECK-LIBCALL-NEXT:    pushq %rbx
 ; CHECK-LIBCALL-NEXT:    subq $48, %rsp
 ; CHECK-LIBCALL-NEXT:    movq %rdi, %rbx
-; CHECK-LIBCALL-NEXT:    movzwl (%rbx), %edi
+; CHECK-LIBCALL-NEXT:    movzwl (%rdi), %edi
 ; CHECK-LIBCALL-NEXT:    callq __gnu_h2f_ieee
 ; CHECK-LIBCALL-NEXT:    movaps %xmm0, {{[0-9]+}}(%rsp) # 16-byte Spill
 ; CHECK-LIBCALL-NEXT:    movzwl 2(%rbx), %edi
 ; CHECK-LIBCALL-NEXT:    callq __gnu_h2f_ieee
 ; CHECK-LIBCALL-NEXT:    movaps %xmm0, {{[0-9]+}}(%rsp) # 16-byte Spill
 ; CHECK-LIBCALL-NEXT:    movzwl 4(%rbx), %edi
 ; CHECK-LIBCALL-NEXT:    callq __gnu_h2f_ieee
 ; CHECK-LIBCALL-NEXT:    movaps %xmm0, (%rsp) # 16-byte Spill

 }
 
 define <4 x double> @test_extend64_vec4(<4 x half>* %p) #0 {
 ; CHECK-LIBCALL-LABEL: test_extend64_vec4:
 ; CHECK-LIBCALL:       # %bb.0:
 ; CHECK-LIBCALL-NEXT:    pushq %rbx
 ; CHECK-LIBCALL-NEXT:    subq $16, %rsp
 ; CHECK-LIBCALL-NEXT:    movq %rdi, %rbx
-; CHECK-LIBCALL-NEXT:    movzwl 4(%rbx), %edi
+; CHECK-LIBCALL-NEXT:    movzwl 4(%rdi), %edi
 ; CHECK-LIBCALL-NEXT:    callq __gnu_h2f_ieee
 ; CHECK-LIBCALL-NEXT:    movss %xmm0, {{[0-9]+}}(%rsp) # 4-byte Spill
 ; CHECK-LIBCALL-NEXT:    movzwl 6(%rbx), %edi
 ; CHECK-LIBCALL-NEXT:    callq __gnu_h2f_ieee
 ; CHECK-LIBCALL-NEXT:    movss %xmm0, {{[0-9]+}}(%rsp) # 4-byte Spill
 ; CHECK-LIBCALL-NEXT:    movzwl (%rbx), %edi
 ; CHECK-LIBCALL-NEXT:    callq __gnu_h2f_ieee
 ; CHECK-LIBCALL-NEXT:    movss %xmm0, {{[0-9]+}}(%rsp) # 4-byte Spill

 ; CHECK-I686-NEXT:    pushl %ebp
 ; CHECK-I686-NEXT:    pushl %ebx
 ; CHECK-I686-NEXT:    pushl %edi
 ; CHECK-I686-NEXT:    pushl %esi
 ; CHECK-I686-NEXT:    subl $44, %esp
 ; CHECK-I686-NEXT:    movaps %xmm0, {{[0-9]+}}(%esp) # 16-byte Spill
 ; CHECK-I686-NEXT:    movl {{[0-9]+}}(%esp), %ebp
 ; CHECK-I686-NEXT:    movaps %xmm0, %xmm1
-; CHECK-I686-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,1,2,3]
+; CHECK-I686-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,1],xmm0[2,3]
 ; CHECK-I686-NEXT:    movss %xmm1, (%esp)
 ; CHECK-I686-NEXT:    calll __gnu_f2h_ieee
 ; CHECK-I686-NEXT:    movw %ax, %si
 ; CHECK-I686-NEXT:    movaps {{[0-9]+}}(%esp), %xmm0 # 16-byte Reload
 ; CHECK-I686-NEXT:    movhlps {{.*#+}} xmm0 = xmm0[1,1]
 ; CHECK-I686-NEXT:    movss %xmm0, (%esp)
 ; CHECK-I686-NEXT:    calll __gnu_f2h_ieee
 ; CHECK-I686-NEXT:    movw %ax, %di

llvm/trunk/test/CodeGen/X86/horizontal-reduce-smax.ll

 ; X86-SSE2-NEXT:    movd %xmm3, %eax
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm3[1,1,2,3]
 ; X86-SSE2-NEXT:    movd %xmm0, %edx
 ; X86-SSE2-NEXT:    retl
 ;
 ; X86-SSE42-LABEL: test_reduce_v2i64:
 ; X86-SSE42:       ## %bb.0:
 ; X86-SSE42-NEXT:    movdqa %xmm0, %xmm1
-; X86-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
+; X86-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
 ; X86-SSE42-NEXT:    pcmpgtq %xmm2, %xmm0
 ; X86-SSE42-NEXT:    blendvpd %xmm0, %xmm1, %xmm2
 ; X86-SSE42-NEXT:    movd %xmm2, %eax
 ; X86-SSE42-NEXT:    pextrd $1, %xmm2, %edx
 ; X86-SSE42-NEXT:    retl
 ;
 ; X86-AVX-LABEL: test_reduce_v2i64:
 ; X86-AVX:       ## %bb.0:

 ; X64-SSE2-NEXT:    pandn %xmm1, %xmm3
 ; X64-SSE2-NEXT:    por %xmm0, %xmm3
 ; X64-SSE2-NEXT:    movq %xmm3, %rax
 ; X64-SSE2-NEXT:    retq
 ;
 ; X64-SSE42-LABEL: test_reduce_v2i64:
 ; X64-SSE42:       ## %bb.0:
 ; X64-SSE42-NEXT:    movdqa %xmm0, %xmm1
-; X64-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
+; X64-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
 ; X64-SSE42-NEXT:    pcmpgtq %xmm2, %xmm0
 ; X64-SSE42-NEXT:    blendvpd %xmm0, %xmm1, %xmm2
 ; X64-SSE42-NEXT:    movq %xmm2, %rax
 ; X64-SSE42-NEXT:    retq
 ;
 ; X64-AVX1-LABEL: test_reduce_v2i64:
 ; X64-AVX1:       ## %bb.0:
 ; X64-AVX1-NEXT:    vpshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]

 ;
 
 define i64 @test_reduce_v8i64(<8 x i64> %a0) {
 ; X86-SSE2-LABEL: test_reduce_v8i64:
 ; X86-SSE2:       ## %bb.0:
 ; X86-SSE2-NEXT:    subl $28, %esp
 ; X86-SSE2-NEXT:    .cfi_def_cfa_offset 32
 ; X86-SSE2-NEXT:    movdqa %xmm3, %xmm5
-; X86-SSE2-NEXT:    movdqa %xmm5, (%esp) ## 16-byte Spill
+; X86-SSE2-NEXT:    movdqa %xmm3, (%esp) ## 16-byte Spill
 ; X86-SSE2-NEXT:    movdqa %xmm2, %xmm3
 ; X86-SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; X86-SSE2-NEXT:    movdqa %xmm0, %xmm1
 ; X86-SSE2-NEXT:    movdqa {{.*#+}} xmm4 = [2147483648,0,2147483648,0]
 ; X86-SSE2-NEXT:    pxor %xmm4, %xmm5
 ; X86-SSE2-NEXT:    movdqa %xmm2, %xmm6
 ; X86-SSE2-NEXT:    pxor %xmm4, %xmm6
 ; X86-SSE2-NEXT:    movdqa %xmm6, %xmm7
 ; X86-SSE2-NEXT:    pcmpgtd %xmm5, %xmm7
 ; X86-SSE2-NEXT:    pcmpeqd %xmm5, %xmm6
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm7[0,0,2,2]
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm6[1,1,3,3]
 ; X86-SSE2-NEXT:    pand %xmm5, %xmm6
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm7[1,1,3,3]
 ; X86-SSE2-NEXT:    por %xmm6, %xmm5
 ; X86-SSE2-NEXT:    movdqa %xmm3, %xmm6
 ; X86-SSE2-NEXT:    pxor %xmm4, %xmm6
-; X86-SSE2-NEXT:    movdqa %xmm1, %xmm7
+; X86-SSE2-NEXT:    movdqa %xmm0, %xmm7
 ; X86-SSE2-NEXT:    pxor %xmm4, %xmm7
 ; X86-SSE2-NEXT:    movdqa %xmm7, %xmm0
 ; X86-SSE2-NEXT:    pcmpgtd %xmm6, %xmm0
 ; X86-SSE2-NEXT:    pcmpeqd %xmm6, %xmm7
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm7 = xmm7[1,1,3,3]
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm0[0,0,2,2]
 ; X86-SSE2-NEXT:    pand %xmm6, %xmm7
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm0[1,1,3,3]

 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm2[1,1,2,3]
 ; X86-SSE2-NEXT:    movd %xmm0, %edx
 ; X86-SSE2-NEXT:    addl $28, %esp
 ; X86-SSE2-NEXT:    retl
 ;
 ; X86-SSE42-LABEL: test_reduce_v8i64:
 ; X86-SSE42:       ## %bb.0:
 ; X86-SSE42-NEXT:    movdqa %xmm0, %xmm4
-; X86-SSE42-NEXT:    movdqa %xmm4, %xmm5
+; X86-SSE42-NEXT:    movdqa %xmm0, %xmm5
 ; X86-SSE42-NEXT:    pcmpgtq %xmm2, %xmm5
 ; X86-SSE42-NEXT:    movdqa %xmm1, %xmm0
 ; X86-SSE42-NEXT:    pcmpgtq %xmm3, %xmm0
 ; X86-SSE42-NEXT:    blendvpd %xmm0, %xmm1, %xmm3
 ; X86-SSE42-NEXT:    movdqa %xmm5, %xmm0
 ; X86-SSE42-NEXT:    blendvpd %xmm0, %xmm4, %xmm2
 ; X86-SSE42-NEXT:    movapd %xmm2, %xmm0
 ; X86-SSE42-NEXT:    pcmpgtq %xmm3, %xmm0

 ; X64-SSE2-NEXT:    pandn %xmm0, %xmm3
 ; X64-SSE2-NEXT:    por %xmm1, %xmm3
 ; X64-SSE2-NEXT:    movq %xmm3, %rax
 ; X64-SSE2-NEXT:    retq
 ;
 ; X64-SSE42-LABEL: test_reduce_v8i64:
 ; X64-SSE42:       ## %bb.0:
 ; X64-SSE42-NEXT:    movdqa %xmm0, %xmm4
-; X64-SSE42-NEXT:    movdqa %xmm4, %xmm5
+; X64-SSE42-NEXT:    movdqa %xmm0, %xmm5
 ; X64-SSE42-NEXT:    pcmpgtq %xmm2, %xmm5
 ; X64-SSE42-NEXT:    movdqa %xmm1, %xmm0
 ; X64-SSE42-NEXT:    pcmpgtq %xmm3, %xmm0
 ; X64-SSE42-NEXT:    blendvpd %xmm0, %xmm1, %xmm3
 ; X64-SSE42-NEXT:    movdqa %xmm5, %xmm0
 ; X64-SSE42-NEXT:    blendvpd %xmm0, %xmm4, %xmm2
 ; X64-SSE42-NEXT:    movapd %xmm2, %xmm0
 ; X64-SSE42-NEXT:    pcmpgtq %xmm3, %xmm0

llvm/trunk/test/CodeGen/X86/horizontal-reduce-smin.ll

 ; X86-SSE2-NEXT:    movd %xmm3, %eax
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm3[1,1,2,3]
 ; X86-SSE2-NEXT:    movd %xmm0, %edx
 ; X86-SSE2-NEXT:    retl
 ;
 ; X86-SSE42-LABEL: test_reduce_v2i64:
 ; X86-SSE42:       ## %bb.0:
 ; X86-SSE42-NEXT:    movdqa %xmm0, %xmm1
-; X86-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
+; X86-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
 ; X86-SSE42-NEXT:    movdqa %xmm2, %xmm0
 ; X86-SSE42-NEXT:    pcmpgtq %xmm1, %xmm0
 ; X86-SSE42-NEXT:    blendvpd %xmm0, %xmm1, %xmm2
 ; X86-SSE42-NEXT:    movd %xmm2, %eax
 ; X86-SSE42-NEXT:    pextrd $1, %xmm2, %edx
 ; X86-SSE42-NEXT:    retl
 ;
 ; X86-AVX-LABEL: test_reduce_v2i64:

 ; X64-SSE2-NEXT:    pandn %xmm1, %xmm3
 ; X64-SSE2-NEXT:    por %xmm0, %xmm3
 ; X64-SSE2-NEXT:    movq %xmm3, %rax
 ; X64-SSE2-NEXT:    retq
 ;
 ; X64-SSE42-LABEL: test_reduce_v2i64:
 ; X64-SSE42:       ## %bb.0:
 ; X64-SSE42-NEXT:    movdqa %xmm0, %xmm1
-; X64-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
+; X64-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
 ; X64-SSE42-NEXT:    movdqa %xmm2, %xmm0
 ; X64-SSE42-NEXT:    pcmpgtq %xmm1, %xmm0
 ; X64-SSE42-NEXT:    blendvpd %xmm0, %xmm1, %xmm2
 ; X64-SSE42-NEXT:    movq %xmm2, %rax
 ; X64-SSE42-NEXT:    retq
 ;
 ; X64-AVX1-LABEL: test_reduce_v2i64:
 ; X64-AVX1:       ## %bb.0:

 ;
 
 define i64 @test_reduce_v8i64(<8 x i64> %a0) {
 ; X86-SSE2-LABEL: test_reduce_v8i64:
 ; X86-SSE2:       ## %bb.0:
 ; X86-SSE2-NEXT:    subl $28, %esp
 ; X86-SSE2-NEXT:    .cfi_def_cfa_offset 32
 ; X86-SSE2-NEXT:    movdqa %xmm2, %xmm6
-; X86-SSE2-NEXT:    movdqa %xmm6, (%esp) ## 16-byte Spill
+; X86-SSE2-NEXT:    movdqa %xmm2, (%esp) ## 16-byte Spill
 ; X86-SSE2-NEXT:    movdqa %xmm0, %xmm2
 ; X86-SSE2-NEXT:    movdqa {{.*#+}} xmm4 = [2147483648,0,2147483648,0]
-; X86-SSE2-NEXT:    movdqa %xmm2, %xmm5
+; X86-SSE2-NEXT:    movdqa %xmm0, %xmm5
 ; X86-SSE2-NEXT:    pxor %xmm4, %xmm5
 ; X86-SSE2-NEXT:    pxor %xmm4, %xmm6
 ; X86-SSE2-NEXT:    movdqa %xmm6, %xmm7
 ; X86-SSE2-NEXT:    pcmpgtd %xmm5, %xmm7
 ; X86-SSE2-NEXT:    pcmpeqd %xmm5, %xmm6
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm7[0,0,2,2]
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm6[1,1,3,3]
 ; X86-SSE2-NEXT:    pand %xmm5, %xmm6

llvm/trunk/test/CodeGen/X86/horizontal-reduce-umax.ll

 ; X86-SSE2-NEXT:    movd %xmm3, %eax
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm3[1,1,2,3]
 ; X86-SSE2-NEXT:    movd %xmm0, %edx
 ; X86-SSE2-NEXT:    retl
 ;
 ; X86-SSE42-LABEL: test_reduce_v2i64:
 ; X86-SSE42:       ## %bb.0:
 ; X86-SSE42-NEXT:    movdqa %xmm0, %xmm1
-; X86-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
+; X86-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
 ; X86-SSE42-NEXT:    movdqa {{.*#+}} xmm3 = [0,2147483648,0,2147483648]
 ; X86-SSE42-NEXT:    pxor %xmm3, %xmm0
 ; X86-SSE42-NEXT:    pxor %xmm2, %xmm3
 ; X86-SSE42-NEXT:    pcmpgtq %xmm3, %xmm0
 ; X86-SSE42-NEXT:    blendvpd %xmm0, %xmm1, %xmm2
 ; X86-SSE42-NEXT:    movd %xmm2, %eax
 ; X86-SSE42-NEXT:    pextrd $1, %xmm2, %edx
 ; X86-SSE42-NEXT:    retl

 ; X64-SSE2-NEXT:    pandn %xmm1, %xmm3
 ; X64-SSE2-NEXT:    por %xmm0, %xmm3
 ; X64-SSE2-NEXT:    movq %xmm3, %rax
 ; X64-SSE2-NEXT:    retq
 ;
 ; X64-SSE42-LABEL: test_reduce_v2i64:
 ; X64-SSE42:       ## %bb.0:
 ; X64-SSE42-NEXT:    movdqa %xmm0, %xmm1
-; X64-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
+; X64-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
 ; X64-SSE42-NEXT:    movdqa {{.*#+}} xmm3 = [9223372036854775808,9223372036854775808]
 ; X64-SSE42-NEXT:    pxor %xmm3, %xmm0
 ; X64-SSE42-NEXT:    pxor %xmm2, %xmm3
 ; X64-SSE42-NEXT:    pcmpgtq %xmm3, %xmm0
 ; X64-SSE42-NEXT:    blendvpd %xmm0, %xmm1, %xmm2
 ; X64-SSE42-NEXT:    movq %xmm2, %rax
 ; X64-SSE42-NEXT:    retq
 ;

 ;
 
 define i64 @test_reduce_v8i64(<8 x i64> %a0) {
 ; X86-SSE2-LABEL: test_reduce_v8i64:
 ; X86-SSE2:       ## %bb.0:
 ; X86-SSE2-NEXT:    subl $28, %esp
 ; X86-SSE2-NEXT:    .cfi_def_cfa_offset 32
 ; X86-SSE2-NEXT:    movdqa %xmm3, %xmm5
-; X86-SSE2-NEXT:    movdqa %xmm5, (%esp) ## 16-byte Spill
+; X86-SSE2-NEXT:    movdqa %xmm3, (%esp) ## 16-byte Spill
 ; X86-SSE2-NEXT:    movdqa %xmm2, %xmm3
 ; X86-SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; X86-SSE2-NEXT:    movdqa %xmm0, %xmm1
 ; X86-SSE2-NEXT:    movdqa {{.*#+}} xmm4 = [2147483648,2147483648,2147483648,2147483648]
 ; X86-SSE2-NEXT:    pxor %xmm4, %xmm5
 ; X86-SSE2-NEXT:    movdqa %xmm2, %xmm6
 ; X86-SSE2-NEXT:    pxor %xmm4, %xmm6
 ; X86-SSE2-NEXT:    movdqa %xmm6, %xmm7
 ; X86-SSE2-NEXT:    pcmpgtd %xmm5, %xmm7
 ; X86-SSE2-NEXT:    pcmpeqd %xmm5, %xmm6
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm7[0,0,2,2]
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm6[1,1,3,3]
 ; X86-SSE2-NEXT:    pand %xmm5, %xmm6
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm7[1,1,3,3]
 ; X86-SSE2-NEXT:    por %xmm6, %xmm5
 ; X86-SSE2-NEXT:    movdqa %xmm3, %xmm6
 ; X86-SSE2-NEXT:    pxor %xmm4, %xmm6
-; X86-SSE2-NEXT:    movdqa %xmm1, %xmm7
+; X86-SSE2-NEXT:    movdqa %xmm0, %xmm7
 ; X86-SSE2-NEXT:    pxor %xmm4, %xmm7
 ; X86-SSE2-NEXT:    movdqa %xmm7, %xmm0
 ; X86-SSE2-NEXT:    pcmpgtd %xmm6, %xmm0
 ; X86-SSE2-NEXT:    pcmpeqd %xmm6, %xmm7
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm7 = xmm7[1,1,3,3]
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm0[0,0,2,2]
 ; X86-SSE2-NEXT:    pand %xmm6, %xmm7
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm0[1,1,3,3]

llvm/trunk/test/CodeGen/X86/horizontal-reduce-umin.ll

 ; X86-SSE2-NEXT:    movd %xmm3, %eax
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm3[1,1,2,3]
 ; X86-SSE2-NEXT:    movd %xmm0, %edx
 ; X86-SSE2-NEXT:    retl
 ;
 ; X86-SSE42-LABEL: test_reduce_v2i64:
 ; X86-SSE42:       ## %bb.0:
 ; X86-SSE42-NEXT:    movdqa %xmm0, %xmm1
-; X86-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
+; X86-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
 ; X86-SSE42-NEXT:    movdqa {{.*#+}} xmm0 = [0,2147483648,0,2147483648]
 ; X86-SSE42-NEXT:    movdqa %xmm1, %xmm3
 ; X86-SSE42-NEXT:    pxor %xmm0, %xmm3
 ; X86-SSE42-NEXT:    pxor %xmm2, %xmm0
 ; X86-SSE42-NEXT:    pcmpgtq %xmm3, %xmm0
 ; X86-SSE42-NEXT:    blendvpd %xmm0, %xmm1, %xmm2
 ; X86-SSE42-NEXT:    movd %xmm2, %eax
 ; X86-SSE42-NEXT:    pextrd $1, %xmm2, %edx

 ; X64-SSE2-NEXT:    pandn %xmm1, %xmm3
 ; X64-SSE2-NEXT:    por %xmm0, %xmm3
 ; X64-SSE2-NEXT:    movq %xmm3, %rax
 ; X64-SSE2-NEXT:    retq
 ;
 ; X64-SSE42-LABEL: test_reduce_v2i64:
 ; X64-SSE42:       ## %bb.0:
 ; X64-SSE42-NEXT:    movdqa %xmm0, %xmm1
-; X64-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
+; X64-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm0[2,3,0,1]
 ; X64-SSE42-NEXT:    movdqa {{.*#+}} xmm0 = [9223372036854775808,9223372036854775808]
 ; X64-SSE42-NEXT:    movdqa %xmm1, %xmm3
 ; X64-SSE42-NEXT:    pxor %xmm0, %xmm3
 ; X64-SSE42-NEXT:    pxor %xmm2, %xmm0
 ; X64-SSE42-NEXT:    pcmpgtq %xmm3, %xmm0
 ; X64-SSE42-NEXT:    blendvpd %xmm0, %xmm1, %xmm2
 ; X64-SSE42-NEXT:    movq %xmm2, %rax
 ; X64-SSE42-NEXT:    retq

 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm0 = xmm2[1,1,2,3]
 ; X86-SSE2-NEXT:    movd %xmm0, %edx
 ; X86-SSE2-NEXT:    retl
 ;
 ; X86-SSE42-LABEL: test_reduce_v4i64:
 ; X86-SSE42:       ## %bb.0:
 ; X86-SSE42-NEXT:    movdqa %xmm0, %xmm2
 ; X86-SSE42-NEXT:    movdqa {{.*#+}} xmm3 = [0,2147483648,0,2147483648]
-; X86-SSE42-NEXT:    movdqa %xmm2, %xmm4
+; X86-SSE42-NEXT:    movdqa %xmm0, %xmm4
 ; X86-SSE42-NEXT:    pxor %xmm3, %xmm4
 ; X86-SSE42-NEXT:    movdqa %xmm1, %xmm0
 ; X86-SSE42-NEXT:    pxor %xmm3, %xmm0
 ; X86-SSE42-NEXT:    pcmpgtq %xmm4, %xmm0
 ; X86-SSE42-NEXT:    blendvpd %xmm0, %xmm2, %xmm1
 ; X86-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
 ; X86-SSE42-NEXT:    movdqa %xmm1, %xmm0
 ; X86-SSE42-NEXT:    pxor %xmm3, %xmm0

 ; X64-SSE2-NEXT:    por %xmm4, %xmm2
 ; X64-SSE2-NEXT:    movq %xmm2, %rax
 ; X64-SSE2-NEXT:    retq
 ;
 ; X64-SSE42-LABEL: test_reduce_v4i64:
 ; X64-SSE42:       ## %bb.0:
 ; X64-SSE42-NEXT:    movdqa %xmm0, %xmm2
 ; X64-SSE42-NEXT:    movdqa {{.*#+}} xmm3 = [9223372036854775808,9223372036854775808]
-; X64-SSE42-NEXT:    movdqa %xmm2, %xmm4
+; X64-SSE42-NEXT:    movdqa %xmm0, %xmm4
 ; X64-SSE42-NEXT:    pxor %xmm3, %xmm4
 ; X64-SSE42-NEXT:    movdqa %xmm1, %xmm0
 ; X64-SSE42-NEXT:    pxor %xmm3, %xmm0
 ; X64-SSE42-NEXT:    pcmpgtq %xmm4, %xmm0
 ; X64-SSE42-NEXT:    blendvpd %xmm0, %xmm2, %xmm1
 ; X64-SSE42-NEXT:    pshufd {{.*#+}} xmm2 = xmm1[2,3,0,1]
 ; X64-SSE42-NEXT:    movdqa %xmm1, %xmm0
 ; X64-SSE42-NEXT:    pxor %xmm3, %xmm0

 ;
 
 define i64 @test_reduce_v8i64(<8 x i64> %a0) {
 ; X86-SSE2-LABEL: test_reduce_v8i64:
 ; X86-SSE2:       ## %bb.0:
 ; X86-SSE2-NEXT:    subl $28, %esp
 ; X86-SSE2-NEXT:    .cfi_def_cfa_offset 32
 ; X86-SSE2-NEXT:    movdqa %xmm2, %xmm6
-; X86-SSE2-NEXT:    movdqa %xmm6, (%esp) ## 16-byte Spill
+; X86-SSE2-NEXT:    movdqa %xmm2, (%esp) ## 16-byte Spill
 ; X86-SSE2-NEXT:    movdqa %xmm0, %xmm2
 ; X86-SSE2-NEXT:    movdqa {{.*#+}} xmm4 = [2147483648,2147483648,2147483648,2147483648]
-; X86-SSE2-NEXT:    movdqa %xmm2, %xmm5
+; X86-SSE2-NEXT:    movdqa %xmm0, %xmm5
 ; X86-SSE2-NEXT:    pxor %xmm4, %xmm5
 ; X86-SSE2-NEXT:    pxor %xmm4, %xmm6
 ; X86-SSE2-NEXT:    movdqa %xmm6, %xmm7
 ; X86-SSE2-NEXT:    pcmpgtd %xmm5, %xmm7
 ; X86-SSE2-NEXT:    pcmpeqd %xmm5, %xmm6
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm5 = xmm7[0,0,2,2]
 ; X86-SSE2-NEXT:    pshufd {{.*#+}} xmm6 = xmm6[1,1,3,3]
 ; X86-SSE2-NEXT:    pand %xmm5, %xmm6

llvm/trunk/test/CodeGen/X86/inline-asm-fpstack.ll

 ; CHECK-LABEL: testPR4459:
 ; CHECK:       ## %bb.0: ## %entry
 ; CHECK-NEXT:    subl $28, %esp
 ; CHECK-NEXT:    .cfi_def_cfa_offset 32
 ; CHECK-NEXT:    fldt {{[0-9]+}}(%esp)
 ; CHECK-NEXT:    fstpt (%esp)
 ; CHECK-NEXT:    calll _ceil
 ; CHECK-NEXT:    fld %st(0)
+; CHECK-NEXT:    fxch %st(1)
 ; CHECK-NEXT:    ## InlineAsm Start
 ; CHECK-NEXT:    fistpl %st(0)
 ; CHECK-NEXT:    ## InlineAsm End
 ; CHECK-NEXT:    fstpt (%esp)
 ; CHECK-NEXT:    calll _test3
 ; CHECK-NEXT:    addl $28, %esp
 ; CHECK-NEXT:    retl
 entry:

llvm/trunk/test/CodeGen/X86/ipra-local-linkage.ll

define void @bar(i32 %X) {
   call void asm sideeffect "movl  %r12d, $0", "{r15}~{r12}"(i32 %X)
   ; As R15 is clobbered by foo() when IPRA is enabled value of R15 should be
   ; saved if register containing orignal value is also getting clobbered
   ; and reloaded after foo(), here original value is loaded back into R15D after
   ; call to foo.
   call void @foo()
   ; CHECK-LABEL: bar:
   ; CHECK: callq foo
-  ; CHECK-NEXT: movl  %eax, %r15d
+  ; CHECK-NEXT: movl  %edi, %r15d
   call void asm sideeffect "movl  $0, %r12d", "{r15}~{r12}"(i32 %X)
   ret void
 }

llvm/trunk/test/CodeGen/X86/localescape.ll

define void @print_framealloc_from_fp(i8* %fp) {
   %b2 = getelementptr i32, i32* %b, i32 1
   %b2.val = load i32, i32* %b2
   call i32 (i8*, ...) @printf(i8* getelementptr ([10 x i8], [10 x i8]* @str, i32 0, i32 0), i32 %b2.val)
   ret void
 }
 
 ; X64-LABEL: print_framealloc_from_fp:
 ; X64: movq %rcx, %[[parent_fp:[a-z]+]]
-; X64: movl .Lalloc_func$frame_escape_0(%[[parent_fp]]), %edx
+; X64: movl .Lalloc_func$frame_escape_0(%rcx), %edx
 ; X64: leaq {{.*}}(%rip), %[[str:[a-z]+]]
 ; X64: movq %[[str]], %rcx
 ; X64: callq printf
 ; X64: movl .Lalloc_func$frame_escape_1(%[[parent_fp]]), %edx
 ; X64: movq %[[str]], %rcx
 ; X64: callq printf
 ; X64: movl    $42, .Lalloc_func$frame_escape_1(%[[parent_fp]])
 ; X64: retq

llvm/trunk/test/CodeGen/X86/machine-cp.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-apple-macosx -mattr=+sse2 -verify-machineinstrs | FileCheck %s
 
 ; After tail duplication, two copies in an early exit BB can be cancelled out.
 ; rdar://10640363
 define i32 @t1(i32 %a, i32 %b) nounwind  {
 ; CHECK-LABEL: t1:
 ; CHECK:       ## %bb.0: ## %entry
 ; CHECK-NEXT:    movl %esi, %edx
 ; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    testl %edx, %edx
+; CHECK-NEXT:    testl %esi, %esi
 ; CHECK-NEXT:    je LBB0_1
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  LBB0_2: ## %while.body
 ; CHECK-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    movl %edx, %ecx
 ; CHECK-NEXT:    cltd
 ; CHECK-NEXT:    idivl %ecx
 ; CHECK-NEXT:    testl %edx, %edx
entry:
   ret <8 x i16> %tmp8
 }
 
 define i32 @t3(i64 %a, i64 %b) nounwind  {
 ; CHECK-LABEL: t3:
 ; CHECK:       ## %bb.0: ## %entry
 ; CHECK-NEXT:    movq %rsi, %rdx
 ; CHECK-NEXT:    movq %rdi, %rax
-; CHECK-NEXT:    testq %rdx, %rdx
+; CHECK-NEXT:    testq %rsi, %rsi
 ; CHECK-NEXT:    je LBB2_1
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  LBB2_2: ## %while.body
 ; CHECK-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    movq %rdx, %rcx
 ; CHECK-NEXT:    cqto
 ; CHECK-NEXT:    idivq %rcx
 ; CHECK-NEXT:    testq %rdx, %rdx

llvm/trunk/test/CodeGen/X86/mul-i1024.ll

 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %edx, -400(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, -324(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    addl %ebx, %ecx
 ; X32-NEXT:    movl %edx, %eax
 ; X32-NEXT:    adcl %edi, %eax
 ; X32-NEXT:    movl %edi, %ecx
-; X32-NEXT:    movl %ecx, -204(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %edi, -204(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, -892(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl 12(%ebp), %eax
 ; X32-NEXT:    movl 36(%eax), %eax
 ; X32-NEXT:    movl %eax, -148(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %edx, %edx
 ; X32-NEXT:    mull %edx
 ; X32-NEXT:    movl %edx, -236(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, %edi
-; X32-NEXT:    movl %edi, -304(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %eax, -304(%ebp) # 4-byte Spill
 ; X32-NEXT:    addl %ecx, %edi
 ; X32-NEXT:    movl %edi, -80(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %eax
 ; X32-NEXT:    adcl $0, %eax
 ; X32-NEXT:    movl %eax, -220(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl 36(%esi), %eax
 ; X32-NEXT:    movl %eax, -316(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %ecx, %ecx
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %edx, %ecx
-; X32-NEXT:    movl %ecx, -124(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %edx, -124(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, -184(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, %edx
 ; X32-NEXT:    movl -400(%ebp), %esi # 4-byte Reload
 ; X32-NEXT:    addl %esi, %edx
 ; X32-NEXT:    adcl $0, %ecx
 ; X32-NEXT:    movl %ecx, -64(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -324(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    movl %ecx, %eax
 ; X32-NEXT:    movl %ebx, -100(%ebp) # 4-byte Spill
 ; X32-NEXT:    addl %ebx, %eax
 ; X32-NEXT:    movl %eax, -656(%ebp) # 4-byte Spill
 ; X32-NEXT:    leal (%ebx,%edi), %eax
 ; X32-NEXT:    movl %edx, %edi
-; X32-NEXT:    leal (%ecx,%edi), %edx
+; X32-NEXT:    leal (%ecx,%edx), %edx
 ; X32-NEXT:    adcl %eax, %edx
 ; X32-NEXT:    movl %edx, -700(%ebp) # 4-byte Spill
 ; X32-NEXT:    seto %al
 ; X32-NEXT:    lahf
 ; X32-NEXT:    movl %eax, %eax
 ; X32-NEXT:    movl %eax, -640(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, -96(%ebp) # 4-byte Spill
 ; X32-NEXT:    addl %ecx, %edi

 ; X32-NEXT:    adcl %edi, %eax
 ; X32-NEXT:    movl %eax, -764(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -324(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    addl %esi, %eax
 ; X32-NEXT:    movl %esi, %ecx
 ; X32-NEXT:    adcl %edi, %ebx
 ; X32-NEXT:    movl %ebx, -424(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edi, %ebx
-; X32-NEXT:    movl %ebx, -256(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %edi, -256(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -100(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    addl %eax, -80(%ebp) # 4-byte Folded Spill
 ; X32-NEXT:    movl -204(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    adcl %eax, -220(%ebp) # 4-byte Folded Spill
 ; X32-NEXT:    setb -388(%ebp) # 1-byte Folded Spill
 ; X32-NEXT:    movl 12(%ebp), %eax
 ; X32-NEXT:    movl 4(%eax), %eax
 ; X32-NEXT:    movl %eax, -92(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %edx, %edx
 ; X32-NEXT:    mull %edx
 ; X32-NEXT:    movl %eax, %edi
 ; X32-NEXT:    addl %ebx, %edi
 ; X32-NEXT:    movl %edx, %esi
 ; X32-NEXT:    adcl $0, %esi
 ; X32-NEXT:    movl %ecx, -28(%ebp) # 4-byte Spill
 ; X32-NEXT:    addl %ecx, %edi
 ; X32-NEXT:    movl %edi, -16(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl %ebx, %esi
 ; X32-NEXT:    setb %bh
 ; X32-NEXT:    addl %eax, %esi
 ; X32-NEXT:    movl %esi, -76(%ebp) # 4-byte Spill
 ; X32-NEXT:    movzbl %bh, %eax
 ; X32-NEXT:    adcl %edx, %eax
 ; X32-NEXT:    movl %eax, %edi
-; X32-NEXT:    movl %edi, -72(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %eax, -72(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl 12(%ebp), %eax
 ; X32-NEXT:    movl 8(%eax), %eax
 ; X32-NEXT:    movl %eax, -108(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %ebx, %ebx
 ; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %eax, -104(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, -156(%ebp) # 4-byte Spill
 ; X32-NEXT:    addl %eax, %ecx

 ; X32-NEXT:    movl %eax, -152(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl 8(%ebp), %eax
 ; X32-NEXT:    movl 40(%eax), %eax
 ; X32-NEXT:    movl %eax, -352(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %ecx, %ecx
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %eax, -364(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %ebx
-; X32-NEXT:    movl %ebx, -396(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %edx, -396(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -324(%ebp), %edx # 4-byte Reload
 ; X32-NEXT:    movl %edx, %edi
 ; X32-NEXT:    addl %eax, %edi
 ; X32-NEXT:    movl -400(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    adcl %ebx, %ecx
 ; X32-NEXT:    addl %esi, %edi
 ; X32-NEXT:    movl %edi, -44(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl -152(%ebp), %ecx # 4-byte Folded Reload

 ; X32-NEXT:    movl %eax, -508(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl 12(%ebp), %ecx
 ; X32-NEXT:    movl 16(%ecx), %eax
 ; X32-NEXT:    movl %eax, -212(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %ebx, %ebx
 ; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %eax, %edi
 ; X32-NEXT:    movl %edx, %esi
-; X32-NEXT:    movl %esi, -84(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %edx, -84(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl 20(%ecx), %eax
 ; X32-NEXT:    movl %eax, -252(%ebp) # 4-byte Spill
 ; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %eax, %ebx
 ; X32-NEXT:    addl %esi, %ebx
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    adcl $0, %ecx
 ; X32-NEXT:    addl %edi, %ebx

 ; X32-NEXT:    adcl %esi, %eax
 ; X32-NEXT:    movl %eax, -776(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -44(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    adcl %ebx, %eax
 ; X32-NEXT:    movl %eax, -772(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -52(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    adcl %edx, %eax
 ; X32-NEXT:    movl %edx, %ebx
-; X32-NEXT:    movl %ebx, -56(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %edx, -56(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, -780(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -132(%ebp), %edx # 4-byte Reload
 ; X32-NEXT:    movl %edx, %eax
 ; X32-NEXT:    addl %edi, %eax
 ; X32-NEXT:    movl -140(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    adcl %ecx, %eax
 ; X32-NEXT:    movl %eax, -448(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %eax

 ; X32-NEXT:    adcl %edx, %eax
 ; X32-NEXT:    movl %eax, -432(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl 8(%ebp), %eax
 ; X32-NEXT:    movl 8(%eax), %eax
 ; X32-NEXT:    movl %eax, -184(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %ecx, %ecx
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %eax, %ecx
-; X32-NEXT:    movl %ecx, -160(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %eax, -160(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, -268(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %ebx, %esi
-; X32-NEXT:    movl %esi, %eax
+; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    addl %ecx, %eax
 ; X32-NEXT:    movl -264(%ebp), %ebx # 4-byte Reload
 ; X32-NEXT:    movl %ebx, %ecx
 ; X32-NEXT:    adcl %edx, %ecx
 ; X32-NEXT:    addl %edi, %eax
 ; X32-NEXT:    movl %eax, -240(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl -432(%ebp), %ecx # 4-byte Folded Reload
 ; X32-NEXT:    movl %esi, %edx

 ; X32-NEXT:    movl %eax, %edx
 ; X32-NEXT:    adcl -120(%ebp), %edx # 4-byte Folded Reload
 ; X32-NEXT:    movl %edx, -532(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %ecx, %eax
 ; X32-NEXT:    movl %ecx, -172(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl -60(%ebp), %eax # 4-byte Folded Reload
 ; X32-NEXT:    movl %eax, -592(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %esi, %edx
-; X32-NEXT:    movl %edx, %eax
+; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    movl -116(%ebp), %esi # 4-byte Reload
 ; X32-NEXT:    addl %esi, %eax
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    movl -84(%ebp), %ebx # 4-byte Reload
 ; X32-NEXT:    adcl %ebx, %eax
 ; X32-NEXT:    movl %eax, -328(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %eax
 ; X32-NEXT:    addl %esi, %eax

 ; X32-NEXT:    adcl %edx, %eax
 ; X32-NEXT:    movl %eax, -628(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl 12(%ebp), %esi
 ; X32-NEXT:    movl 48(%esi), %eax
 ; X32-NEXT:    movl %eax, -300(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %ecx, %ecx
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %eax, %ebx
-; X32-NEXT:    movl %ebx, -336(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %eax, -336(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %edi
 ; X32-NEXT:    movl 52(%esi), %eax
 ; X32-NEXT:    movl %eax, -144(%ebp) # 4-byte Spill
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %eax, %esi
 ; X32-NEXT:    addl %edi, %esi
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    adcl $0, %ecx

 ; X32-NEXT:    movl %eax, -244(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %edx, %edx
 ; X32-NEXT:    mull %edx
 ; X32-NEXT:    movl %eax, -224(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, -360(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -336(%ebp), %ebx # 4-byte Reload
 ; X32-NEXT:    addl %eax, %ebx
 ; X32-NEXT:    movl %edi, %edx
-; X32-NEXT:    movl %edx, -176(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %edi, -176(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl -360(%ebp), %edi # 4-byte Folded Reload
 ; X32-NEXT:    addl %ecx, %ebx
 ; X32-NEXT:    movl %ebx, -472(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl %esi, %edi
 ; X32-NEXT:    movl %edi, -436(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -136(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    movl %ecx, %eax
 ; X32-NEXT:    movl -336(%ebp), %esi # 4-byte Reload

 ; X32-NEXT:    adcl %edi, %eax
 ; X32-NEXT:    movl %eax, -636(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl 8(%ebp), %eax
 ; X32-NEXT:    movl 64(%eax), %eax
 ; X32-NEXT:    movl %eax, -476(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %ecx, %ecx
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %edx, %esi
-; X32-NEXT:    movl %esi, -384(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %edx, -384(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -116(%ebp), %edi # 4-byte Reload
 ; X32-NEXT:    movl %edi, %ecx
 ; X32-NEXT:    movl %eax, %edx
-; X32-NEXT:    movl %edx, -480(%ebp) # 4-byte Spill
-; X32-NEXT:    addl %edx, %ecx
+; X32-NEXT:    movl %eax, -480(%ebp) # 4-byte Spill
+; X32-NEXT:    addl %eax, %ecx
 ; X32-NEXT:    movl -84(%ebp), %ebx # 4-byte Reload
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    adcl %esi, %eax
 ; X32-NEXT:    movl %eax, -920(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -28(%ebp), %esi # 4-byte Reload
 ; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    addl %edx, %eax
 ; X32-NEXT:    movl -256(%ebp), %ecx # 4-byte Reload

 ; X32-NEXT:    movl %eax, -976(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl 64(%ecx), %eax
 ; X32-NEXT:    movl %eax, -520(%ebp) # 4-byte Spill
 ; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %eax, -500(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    addl %esi, %ecx
 ; X32-NEXT:    movl %edx, %esi
-; X32-NEXT:    movl %esi, -496(%ebp) # 4-byte Spill
-; X32-NEXT:    movl %esi, %ecx
+; X32-NEXT:    movl %edx, -496(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    adcl %edi, %ecx
 ; X32-NEXT:    movl %ecx, -992(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    movl -180(%ebp), %edx # 4-byte Reload
 ; X32-NEXT:    addl %edx, %ecx
 ; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    movl -320(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    adcl %ecx, %eax

 ; X32-NEXT:    movzbl -96(%ebp), %edi # 1-byte Folded Reload
 ; X32-NEXT:    adcl %edx, %edi
 ; X32-NEXT:    movl 8(%ebp), %eax
 ; X32-NEXT:    movl 72(%eax), %eax
 ; X32-NEXT:    movl %eax, -516(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %edx, %edx
 ; X32-NEXT:    mull %edx
 ; X32-NEXT:    movl %eax, %esi
-; X32-NEXT:    movl %esi, -484(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %eax, -484(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, -488(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    addl %esi, %eax
 ; X32-NEXT:    movl -384(%ebp), %ebx # 4-byte Reload
 ; X32-NEXT:    adcl %edx, %ebx
 ; X32-NEXT:    addl %ecx, %eax
 ; X32-NEXT:    adcl %edi, %ebx
 ; X32-NEXT:    movl -116(%ebp), %ecx # 4-byte Reload

 ; X32-NEXT:    movl %esi, -696(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl -16(%ebp), %ecx # 4-byte Folded Reload
 ; X32-NEXT:    movl %ecx, -652(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl -120(%ebp), %eax # 4-byte Folded Reload
 ; X32-NEXT:    movl %eax, -924(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl -60(%ebp), %ebx # 4-byte Folded Reload
 ; X32-NEXT:    movl %ebx, -928(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl 8(%ebp), %ecx
-; X32-NEXT:    movl %ecx, %eax
-; X32-NEXT:    movl 84(%eax), %eax
+; X32-NEXT:    movl 84(%ecx), %eax
 ; X32-NEXT:    movl %eax, -544(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %ecx, %ecx
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %eax, %esi
 ; X32-NEXT:    movl -356(%ebp), %ebx # 4-byte Reload
 ; X32-NEXT:    addl %ebx, %esi
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    adcl $0, %ecx

 ; X32-NEXT:    movzbl %bl, %edi
 ; X32-NEXT:    adcl %edx, %edi
 ; X32-NEXT:    movl 12(%ebp), %eax
 ; X32-NEXT:    movl 88(%eax), %eax
 ; X32-NEXT:    movl %eax, -492(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %edx, %edx
 ; X32-NEXT:    mull %edx
 ; X32-NEXT:    movl %eax, %esi
-; X32-NEXT:    movl %esi, -556(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %eax, -556(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, -560(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -524(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    movl %eax, %ebx
 ; X32-NEXT:    addl %esi, %ebx
 ; X32-NEXT:    movl -528(%ebp), %esi # 4-byte Reload
 ; X32-NEXT:    adcl %edx, %esi
 ; X32-NEXT:    addl %ecx, %ebx
 ; X32-NEXT:    movl %ebx, -732(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl %edi, %esi
 ; X32-NEXT:    movl %esi, %edx
-; X32-NEXT:    movl %edx, -728(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %esi, -728(%ebp) # 4-byte Spill
 ; X32-NEXT:    addl -136(%ebp), %eax # 4-byte Folded Reload
 ; X32-NEXT:    movl %eax, -712(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -668(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    adcl -276(%ebp), %ecx # 4-byte Folded Reload
 ; X32-NEXT:    movl %ecx, -968(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    adcl -240(%ebp), %eax # 4-byte Folded Reload
 ; X32-NEXT:    movl %eax, -964(%ebp) # 4-byte Spill

 ; X32-NEXT:    movzbl -96(%ebp), %esi # 1-byte Folded Reload
 ; X32-NEXT:    adcl %edx, %esi
 ; X32-NEXT:    movl 12(%ebp), %eax
 ; X32-NEXT:    movl 72(%eax), %eax
 ; X32-NEXT:    movl %eax, -388(%ebp) # 4-byte Spill
 ; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %eax, -564(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %ebx
-; X32-NEXT:    movl %ebx, -568(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %edx, -568(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -500(%ebp), %edx # 4-byte Reload
 ; X32-NEXT:    movl %edx, %edi
 ; X32-NEXT:    addl %eax, %edi
 ; X32-NEXT:    movl -496(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    adcl %ebx, %eax
 ; X32-NEXT:    addl %ecx, %edi
 ; X32-NEXT:    adcl %esi, %eax
 ; X32-NEXT:    movl %eax, %ecx

 ; X32-NEXT:    addl %ecx, %edi
 ; X32-NEXT:    movl %edi, -232(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl %esi, %ebx
 ; X32-NEXT:    setb -88(%ebp) # 1-byte Folded Spill
 ; X32-NEXT:    addl %eax, %ebx
 ; X32-NEXT:    movzbl -88(%ebp), %eax # 1-byte Folded Reload
 ; X32-NEXT:    adcl %edx, %eax
 ; X32-NEXT:    movl %ecx, %edx
-; X32-NEXT:    addl %edx, %ebx
+; X32-NEXT:    addl %ecx, %ebx
 ; X32-NEXT:    adcl %esi, %eax
 ; X32-NEXT:    movl %eax, -88(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -28(%ebp), %edi # 4-byte Reload
 ; X32-NEXT:    movl -76(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    addl %edi, %ecx
 ; X32-NEXT:    movl -72(%ebp), %esi # 4-byte Reload
 ; X32-NEXT:    adcl -256(%ebp), %esi # 4-byte Folded Reload
 ; X32-NEXT:    addl %ecx, %edx

 ; X32-NEXT:    movl %edi, -56(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl 8(%ebp), %eax
 ; X32-NEXT:    movl 44(%eax), %eax
 ; X32-NEXT:    movl %eax, -120(%ebp) # 4-byte Spill
 ; X32-NEXT:    xorl %ecx, %ecx
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %edx, %edi
 ; X32-NEXT:    movl %eax, %ebx
-; X32-NEXT:    movl %ebx, %ecx
+; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    movl -396(%ebp), %esi # 4-byte Reload
 ; X32-NEXT:    addl %esi, %ecx
 ; X32-NEXT:    adcl $0, %edx
 ; X32-NEXT:    movl -364(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    addl %eax, %ecx
 ; X32-NEXT:    movl %ecx, -60(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl %esi, %edx
 ; X32-NEXT:    movl %esi, %ecx
 ; X32-NEXT:    setb -16(%ebp) # 1-byte Folded Spill
 ; X32-NEXT:    addl %ebx, %edx
 ; X32-NEXT:    movzbl -16(%ebp), %ebx # 1-byte Folded Reload
 ; X32-NEXT:    adcl %edi, %ebx
 ; X32-NEXT:    movl %eax, %esi
-; X32-NEXT:    addl %esi, %edx
+; X32-NEXT:    addl %eax, %edx
 ; X32-NEXT:    adcl %ecx, %ebx
 ; X32-NEXT:    movl -64(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    addl -324(%ebp), %eax # 4-byte Folded Reload
 ; X32-NEXT:    movl -152(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    adcl -400(%ebp), %ecx # 4-byte Folded Reload
 ; X32-NEXT:    addl %eax, %esi
 ; X32-NEXT:    movl %esi, -64(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -60(%ebp), %esi # 4-byte Reload

 ; X32-NEXT:    addl %edi, %ecx
 ; X32-NEXT:    movl %ecx, -24(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl %ebx, %edx
 ; X32-NEXT:    setb %cl
 ; X32-NEXT:    addl %eax, %edx
 ; X32-NEXT:    movzbl %cl, %eax
 ; X32-NEXT:    adcl %esi, %eax
 ; X32-NEXT:    movl %edi, %esi
-; X32-NEXT:    addl %esi, %edx
+; X32-NEXT:    addl %edi, %edx
 ; X32-NEXT:    adcl %ebx, %eax
 ; X32-NEXT:    movl %eax, -112(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -136(%ebp), %edi # 4-byte Reload
 ; X32-NEXT:    movl -584(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    addl %edi, %ecx
 ; X32-NEXT:    movl -432(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    adcl -264(%ebp), %eax # 4-byte Folded Reload
 ; X32-NEXT:    addl %ecx, %esi

 ; X32-NEXT:    addl %ecx, %ebx
 ; X32-NEXT:    movl %ebx, -36(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl %edi, %esi
 ; X32-NEXT:    setb %bl
 ; X32-NEXT:    addl %eax, %esi
 ; X32-NEXT:    movzbl %bl, %eax
 ; X32-NEXT:    adcl %edx, %eax
 ; X32-NEXT:    movl %ecx, %edx
-; X32-NEXT:    addl %edx, %esi
+; X32-NEXT:    addl %ecx, %esi
 ; X32-NEXT:    adcl %edi, %eax
 ; X32-NEXT:    movl %eax, -48(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -100(%ebp), %edi # 4-byte Reload
 ; X32-NEXT:    movl -220(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    addl %edi, %eax
 ; X32-NEXT:    movl -376(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    adcl -204(%ebp), %ecx # 4-byte Folded Reload
 ; X32-NEXT:    addl %eax, %edx

 ; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    addl %ebx, %ecx
 ; X32-NEXT:    adcl $0, %edi
 ; X32-NEXT:    movl 12(%ebp), %eax
 ; X32-NEXT:    movl 28(%eax), %ebx
 ; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %ebx, %esi
-; X32-NEXT:    movl %esi, -48(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %ebx, -48(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %ebx
 ; X32-NEXT:    addl %ecx, %eax
 ; X32-NEXT:    movl %eax, -64(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl %edi, %ebx
 ; X32-NEXT:    setb %cl
 ; X32-NEXT:    movl -316(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    mull %esi
 ; X32-NEXT:    addl %ebx, %eax

 ; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    addl -140(%ebp), %ecx # 4-byte Folded Reload
 ; X32-NEXT:    adcl $0, %edi
 ; X32-NEXT:    movl 8(%ebp), %eax
 ; X32-NEXT:    movl 92(%eax), %ebx
 ; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %ebx, %esi
-; X32-NEXT:    movl %esi, -140(%ebp) # 4-byte Spill
+; X32-NEXT:    movl %ebx, -140(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %ebx
 ; X32-NEXT:    addl %ecx, %eax
 ; X32-NEXT:    movl %eax, -56(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl %edi, %ebx
 ; X32-NEXT:    setb %cl
 ; X32-NEXT:    movl -92(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    mull %esi
 ; X32-NEXT:    addl %ebx, %eax

 ; X32-NEXT:    movl -92(%ebp), %esi # 4-byte Reload
 ; X32-NEXT:    imull %esi, %edi
 ; X32-NEXT:    movl -168(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %eax, -96(%ebp) # 4-byte Spill
 ; X32-NEXT:    addl %edi, %edx
 ; X32-NEXT:    movl 124(%ebx), %ebx
 ; X32-NEXT:    movl %ecx, %eax
-; X32-NEXT:    imull %eax, %ebx
+; X32-NEXT:    imull %ecx, %ebx
 ; X32-NEXT:    addl %edx, %ebx
 ; X32-NEXT:    movl -144(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    addl %ecx, -96(%ebp) # 4-byte Folded Spill
 ; X32-NEXT:    adcl -108(%ebp), %ebx # 4-byte Folded Reload
 ; X32-NEXT:    movl -156(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %edx, -144(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, -108(%ebp) # 4-byte Spill

 ; X32-NEXT:    movl %edx, %edi
 ; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    addl -128(%ebp), %ecx # 4-byte Folded Reload
 ; X32-NEXT:    adcl $0, %edi
 ; X32-NEXT:    movl 12(%ebp), %eax
 ; X32-NEXT:    movl 108(%eax), %edx
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    movl %edx, %ebx
-; X32-NEXT:    movl %ebx, -112(%ebp) # 4-byte Spill
-; X32-NEXT:    mull %ebx
+; X32-NEXT:    movl %edx, -112(%ebp) # 4-byte Spill
+; X32-NEXT:    mull %edx
 ; X32-NEXT:    movl %edx, %esi
 ; X32-NEXT:    addl %ecx, %eax
 ; X32-NEXT:    movl %eax, -128(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl %edi, %esi
 ; X32-NEXT:    setb -176(%ebp) # 1-byte Folded Spill
 ; X32-NEXT:    movl -124(%ebp), %eax # 4-byte Reload
 ; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %edx, %ecx

 ; X32-NEXT:    addl -188(%ebp), %esi # 4-byte Folded Reload
 ; X32-NEXT:    movl %esi, -136(%ebp) # 4-byte Spill
 ; X32-NEXT:    adcl -144(%ebp), %eax # 4-byte Folded Reload
 ; X32-NEXT:    movl %eax, -264(%ebp) # 4-byte Spill
 ; X32-NEXT:    setb -84(%ebp) # 1-byte Folded Spill
 ; X32-NEXT:    movl -184(%ebp), %ecx # 4-byte Reload
 ; X32-NEXT:    movl %ecx, %eax
 ; X32-NEXT:    movl %ebx, %esi
-; X32-NEXT:    mull %esi
+; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %edx, -144(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, -280(%ebp) # 4-byte Spill
 ; X32-NEXT:    movl -60(%ebp), %ebx # 4-byte Reload
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    mull %esi
 ; X32-NEXT:    movl %edx, %esi
 ; X32-NEXT:    movl %eax, %edi
 ; X32-NEXT:    addl -144(%ebp), %edi # 4-byte Folded Reload

 ; X64-NEXT:    movq %rbp, %rax
 ; X64-NEXT:    mulq %r8
 ; X64-NEXT:    movq %rax, %rbx
 ; X64-NEXT:    addq %rdi, %rbx
 ; X64-NEXT:    movq %rdx, %rbp
 ; X64-NEXT:    adcq $0, %rbp
 ; X64-NEXT:    addq %rcx, %rbx
 ; X64-NEXT:    movq %rbx, -{{[0-9]+}}(%rsp) # 8-byte Spill
-; X64-NEXT:    movq %rcx, %r11
 ; X64-NEXT:    adcq %rdi, %rbp
 ; X64-NEXT:    setb %bl
 ; X64-NEXT:    movzbl %bl, %ebx
 ; X64-NEXT:    addq %rax, %rbp
 ; X64-NEXT:    adcq %rdx, %rbx
 ; X64-NEXT:    movq %r9, %rax
 ; X64-NEXT:    mulq %r8
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rdx, {{[0-9]+}}(%rsp) # 8-byte Spill
-; X64-NEXT:    movq %r11, %r12
-; X64-NEXT:    movq %r11, %r8
+; X64-NEXT:    movq %rcx, %r12
+; X64-NEXT:    movq %rcx, %r8
 ; X64-NEXT:    addq %rax, %r12
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    movq %rdi, %r9
-; X64-NEXT:    movq %r9, (%rsp) # 8-byte Spill
+; X64-NEXT:    movq %rdi, (%rsp) # 8-byte Spill
 ; X64-NEXT:    adcq %rdx, %rax
 ; X64-NEXT:    addq %rbp, %r12
 ; X64-NEXT:    movq %r12, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    adcq %rbx, %rax
 ; X64-NEXT:    movq %rax, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq (%rsi), %rax
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    xorl %ebp, %ebp

 ; X64-NEXT:    adcq %rcx, %rbp
 ; X64-NEXT:    movq %rcx, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    setb %bl
 ; X64-NEXT:    addq %rax, %rbp
 ; X64-NEXT:    movzbl %bl, %ebx
 ; X64-NEXT:    adcq %rdx, %rbx
 ; X64-NEXT:    movq 16(%rsi), %rax
 ; X64-NEXT:    movq %rsi, %r13
-; X64-NEXT:    movq %r13, {{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    movq %rsi, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rax, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    mulq %r11
 ; X64-NEXT:    movq %rax, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rdx, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rdi, %r14
 ; X64-NEXT:    addq %rax, %r14
 ; X64-NEXT:    movq %rcx, %r11
 ; X64-NEXT:    adcq %rdx, %r11
 ; X64-NEXT:    addq %rbp, %r14
 ; X64-NEXT:    adcq %rbx, %r11
 ; X64-NEXT:    movq %r8, %rax
 ; X64-NEXT:    movq %r8, %rbp
-; X64-NEXT:    movq %rbp, -{{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    movq %r8, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    addq %rdi, %rax
 ; X64-NEXT:    movq %r9, %rax
 ; X64-NEXT:    adcq %rcx, %rax
 ; X64-NEXT:    movq %rax, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq (%r10), %rax
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    xorl %r8d, %r8d
 ; X64-NEXT:    mulq %r8
 ; X64-NEXT:    movq %rdx, %rsi
 ; X64-NEXT:    movq %rax, %rbx
 ; X64-NEXT:    addq %rdi, %rax
 ; X64-NEXT:    movq %rdi, %r9
-; X64-NEXT:    movq %rsi, %rax
+; X64-NEXT:    movq %rdx, %rax
 ; X64-NEXT:    adcq %rcx, %rax
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq 32(%r13), %rax
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    mulq %r8
 ; X64-NEXT:    xorl %r8d, %r8d
 ; X64-NEXT:    movq %rax, %r13
 ; X64-NEXT:    movq %rdx, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    movq %rbx, %rcx
 ; X64-NEXT:    addq %r13, %rax
 ; X64-NEXT:    movq %rsi, %rax
 ; X64-NEXT:    adcq %rdx, %rax
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rbp, %rax
-; X64-NEXT:    addq %r9, %rax
+; X64-NEXT:    addq %rdi, %rax
 ; X64-NEXT:    movq %rax, -{{[0-9]+}}(%rsp) # 8-byte Spill
-; X64-NEXT:    movq %r9, {{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    movq %rdi, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %rax # 8-byte Reload
 ; X64-NEXT:    adcq %r15, %rax
 ; X64-NEXT:    movq %rax, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    adcq %r14, %r12
 ; X64-NEXT:    movq %r12, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %rax # 8-byte Reload
 ; X64-NEXT:    adcq %r11, %rax
 ; X64-NEXT:    movq %rax, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %r11, %rdi
 ; X64-NEXT:    movq 8(%r10), %rax
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %r10, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    mulq %r8
 ; X64-NEXT:    movq %rax, %r11
 ; X64-NEXT:    addq %rsi, %r11
 ; X64-NEXT:    movq %rdx, %rbp
 ; X64-NEXT:    adcq $0, %rbp
-; X64-NEXT:    addq %rcx, %r11
+; X64-NEXT:    addq %rbx, %r11
 ; X64-NEXT:    adcq %rsi, %rbp
 ; X64-NEXT:    movq %rsi, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    setb %bl
 ; X64-NEXT:    addq %rax, %rbp
 ; X64-NEXT:    movzbl %bl, %ebx
 ; X64-NEXT:    adcq %rdx, %rbx
 ; X64-NEXT:    movq 16(%r10), %rax
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    mulq %r8
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rdx, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rcx, %r8
 ; X64-NEXT:    addq %rax, %r8
 ; X64-NEXT:    movq %rsi, %r10
 ; X64-NEXT:    adcq %rdx, %r10
 ; X64-NEXT:    addq %rbp, %r8
 ; X64-NEXT:    movq %r8, %rax
 ; X64-NEXT:    adcq %rbx, %r10
 ; X64-NEXT:    movq %rcx, %rdx
 ; X64-NEXT:    movq %rcx, %r12
-; X64-NEXT:    movq %r12, {{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    movq %rcx, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    addq %r9, %rdx
 ; X64-NEXT:    movq %rdx, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %r11, %r8
-; X64-NEXT:    adcq %r8, %r15
+; X64-NEXT:    adcq %r11, %r15
 ; X64-NEXT:    movq %r15, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    adcq %rax, %r14
 ; X64-NEXT:    movq %r14, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rax, %rcx
 ; X64-NEXT:    adcq %r10, %rdi
 ; X64-NEXT:    movq %rdi, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rsi # 8-byte Reload
 ; X64-NEXT:    movq 40(%rsi), %rax

 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %r15 # 8-byte Reload
 ; X64-NEXT:    addq {{[0-9]+}}(%rsp), %r15 # 8-byte Folded Reload
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %r12 # 8-byte Reload
 ; X64-NEXT:    adcq {{[0-9]+}}(%rsp), %r12 # 8-byte Folded Reload
 ; X64-NEXT:    addq %rax, %r15
 ; X64-NEXT:    adcq %rdx, %r12
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rcx # 8-byte Reload
 ; X64-NEXT:    movq %rcx, %rax
-; X64-NEXT:    movq %r10, %rbp
-; X64-NEXT:    mulq %rbp
+; X64-NEXT:    mulq %r10
 ; X64-NEXT:    movq %rdx, %rsi
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rdi # 8-byte Reload
 ; X64-NEXT:    movq %rdi, %rax
-; X64-NEXT:    mulq %rbp
+; X64-NEXT:    mulq %r10
 ; X64-NEXT:    movq %rdx, %rbp
 ; X64-NEXT:    movq %rax, %rbx
 ; X64-NEXT:    addq %rsi, %rbx
 ; X64-NEXT:    adcq $0, %rbp
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    movq %rcx, %r10
 ; X64-NEXT:    mulq %r11
 ; X64-NEXT:    movq %rdx, %rcx

 ; X64-NEXT:    adcq %rax, %r13
 ; X64-NEXT:    addq -{{[0-9]+}}(%rsp), %rsi # 8-byte Folded Reload
 ; X64-NEXT:    adcq -{{[0-9]+}}(%rsp), %r13 # 8-byte Folded Reload
 ; X64-NEXT:    addq %r9, %rsi
 ; X64-NEXT:    adcq %r8, %r13
 ; X64-NEXT:    adcq $0, %r15
 ; X64-NEXT:    adcq $0, %r12
 ; X64-NEXT:    movq %r10, %rbx
-; X64-NEXT:    movq %rbx, %rax
+; X64-NEXT:    movq %r10, %rax
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %r11 # 8-byte Reload
 ; X64-NEXT:    mulq %r11
 ; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    movq %rax, %r10
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    movq %rdi, %r9
 ; X64-NEXT:    mulq %r11
 ; X64-NEXT:    movq %rdx, %rdi
 ; X64-NEXT:    movq %rax, %rbp
 ; X64-NEXT:    addq %rcx, %rbp
 ; X64-NEXT:    adcq $0, %rdi
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rax # 8-byte Reload
 ; X64-NEXT:    movq 24(%rax), %rcx
 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rcx, %rbx
-; X64-NEXT:    movq %rbx, -{{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    movq %rcx, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    movq %rax, %r8
 ; X64-NEXT:    addq %rbp, %r8
 ; X64-NEXT:    adcq %rdi, %rcx
 ; X64-NEXT:    setb %dil
 ; X64-NEXT:    movq %r9, %rax
 ; X64-NEXT:    mulq %rbx
 ; X64-NEXT:    addq %rcx, %rax

 ; X64-NEXT:    adcq $0, %rbp
 ; X64-NEXT:    adcq $0, %rbx
 ; X64-NEXT:    addq %r15, %rbp
 ; X64-NEXT:    adcq %r12, %rbx
 ; X64-NEXT:    setb %r15b
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rcx # 8-byte Reload
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    movq %r11, %rsi
-; X64-NEXT:    mulq %rsi
+; X64-NEXT:    mulq %r11
 ; X64-NEXT:    movq %rdx, %r11
 ; X64-NEXT:    movq %rax, %r13
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %r12 # 8-byte Reload
 ; X64-NEXT:    movq %r12, %rax
 ; X64-NEXT:    mulq %rsi
 ; X64-NEXT:    movq %rdx, %rsi
 ; X64-NEXT:    movq %rax, %rdi
 ; X64-NEXT:    addq %r11, %rdi

 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %r8 # 8-byte Reload
 ; X64-NEXT:    addq {{[0-9]+}}(%rsp), %r8 # 8-byte Folded Reload
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %r10 # 8-byte Reload
 ; X64-NEXT:    adcq {{[0-9]+}}(%rsp), %r10 # 8-byte Folded Reload
 ; X64-NEXT:    addq %rax, %r8
 ; X64-NEXT:    adcq %rdx, %r10
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rcx # 8-byte Reload
 ; X64-NEXT:    movq %rcx, %rax
-; X64-NEXT:    movq %r11, %rbp
-; X64-NEXT:    mulq %rbp
+; X64-NEXT:    mulq %r11
 ; X64-NEXT:    movq %rdx, %rdi
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rsi # 8-byte Reload
 ; X64-NEXT:    movq %rsi, %rax
-; X64-NEXT:    mulq %rbp
+; X64-NEXT:    mulq %r11
 ; X64-NEXT:    movq %rdx, %rbp
 ; X64-NEXT:    movq %rax, %rbx
 ; X64-NEXT:    addq %rdi, %rbx
 ; X64-NEXT:    adcq $0, %rbp
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    movq %rcx, %r11
 ; X64-NEXT:    mulq %r9
 ; X64-NEXT:    movq %rdx, %rdi

 ; X64-NEXT:    adcq $0, -{{[0-9]+}}(%rsp) # 8-byte Folded Spill
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rdi # 8-byte Reload
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rcx # 8-byte Reload
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rdx, %rsi
 ; X64-NEXT:    movq %rax, %r14
 ; X64-NEXT:    movq %r8, %rbp
-; X64-NEXT:    movq %rbp, %rax
+; X64-NEXT:    movq %r8, %rax
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rcx, %r11
 ; X64-NEXT:    movq %rdx, %rbx
 ; X64-NEXT:    movq %rax, %rcx
 ; X64-NEXT:    addq %rsi, %rcx
 ; X64-NEXT:    adcq $0, %rbx
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rdi # 8-byte Reload

 ; X64-NEXT:    adcq %rax, %r15
 ; X64-NEXT:    addq {{[0-9]+}}(%rsp), %rbx # 8-byte Folded Reload
 ; X64-NEXT:    adcq {{[0-9]+}}(%rsp), %r15 # 8-byte Folded Reload
 ; X64-NEXT:    addq %r14, %rbx
 ; X64-NEXT:    adcq %r8, %r15
 ; X64-NEXT:    adcq $0, %r9
 ; X64-NEXT:    adcq $0, %r10
 ; X64-NEXT:    movq %rbp, %rsi
-; X64-NEXT:    movq %rsi, %rax
+; X64-NEXT:    movq %rbp, %rax
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %rcx # 8-byte Reload
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rdx, %r14
 ; X64-NEXT:    movq %rax, %r12
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    movq %rdi, %r8
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rdx, %rbp

 ; X64-NEXT:    movq %r10, %rax
 ; X64-NEXT:    mulq %rdi
 ; X64-NEXT:    movq %rdx, %r15
 ; X64-NEXT:    movq %rax, %rbx
 ; X64-NEXT:    addq %r9, %rbx
 ; X64-NEXT:    adcq $0, %r15
 ; X64-NEXT:    movq %rbp, %rax
 ; X64-NEXT:    movq %r8, %rdi
-; X64-NEXT:    movq %rdi, {{[0-9]+}}(%rsp) # 8-byte Spill
-; X64-NEXT:    mulq %rdi
+; X64-NEXT:    movq %r8, {{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    mulq %r8
 ; X64-NEXT:    movq %rdx, %r9
 ; X64-NEXT:    movq %rax, %r8
 ; X64-NEXT:    addq %rbx, %r8
 ; X64-NEXT:    adcq %r15, %r9
 ; X64-NEXT:    setb %bl
 ; X64-NEXT:    movq %r10, %rax
 ; X64-NEXT:    mulq %rdi
 ; X64-NEXT:    addq %r9, %rax

 ; X64-NEXT:    addq {{[0-9]+}}(%rsp), %r8 # 8-byte Folded Reload
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %rcx # 8-byte Reload
 ; X64-NEXT:    adcq {{[0-9]+}}(%rsp), %rcx # 8-byte Folded Reload
 ; X64-NEXT:    addq %rax, %r8
 ; X64-NEXT:    adcq %rdx, %rcx
 ; X64-NEXT:    movq %rcx, %r14
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rcx # 8-byte Reload
 ; X64-NEXT:    movq %rcx, %rax
-; X64-NEXT:    movq %r10, %rdi
-; X64-NEXT:    mulq %rdi
+; X64-NEXT:    mulq %r10
 ; X64-NEXT:    movq %rdx, %r11
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rsi # 8-byte Reload
 ; X64-NEXT:    movq %rsi, %rax
-; X64-NEXT:    mulq %rdi
+; X64-NEXT:    mulq %r10
 ; X64-NEXT:    movq %rdx, %rdi
 ; X64-NEXT:    movq %rax, %rbx
 ; X64-NEXT:    addq %r11, %rbx
 ; X64-NEXT:    adcq $0, %rdi
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    movq %rcx, %r13
 ; X64-NEXT:    mulq %r9
 ; X64-NEXT:    movq %rdx, %rcx

 ; X64-NEXT:    addq {{[0-9]+}}(%rsp), %rdi # 8-byte Folded Reload
 ; X64-NEXT:    adcq {{[0-9]+}}(%rsp), %r11 # 8-byte Folded Reload
 ; X64-NEXT:    addq {{[0-9]+}}(%rsp), %rdi # 8-byte Folded Reload
 ; X64-NEXT:    adcq %r12, %r11
 ; X64-NEXT:    adcq $0, %r8
 ; X64-NEXT:    movq %r8, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    adcq $0, %r14
 ; X64-NEXT:    movq %r14, -{{[0-9]+}}(%rsp) # 8-byte Spill
-; X64-NEXT:    movq %r13, %rbx
-; X64-NEXT:    movq %rbx, %rax
+; X64-NEXT:    movq %r13, %rax
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %rcx # 8-byte Reload
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rdx, %r8
 ; X64-NEXT:    movq %rax, %r12
 ; X64-NEXT:    movq %rsi, %rax
 ; X64-NEXT:    movq %rsi, %r9
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rcx, %r10
 ; X64-NEXT:    movq %rdx, %rsi
 ; X64-NEXT:    movq %rax, %rcx
 ; X64-NEXT:    addq %r8, %rcx
 ; X64-NEXT:    adcq $0, %rsi
-; X64-NEXT:    movq %rbx, %rax
+; X64-NEXT:    movq %r13, %rax
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %r13 # 8-byte Reload
 ; X64-NEXT:    mulq %r13
 ; X64-NEXT:    movq %rdx, %rbx
 ; X64-NEXT:    addq %rcx, %rax
 ; X64-NEXT:    movq %rax, %r8
 ; X64-NEXT:    adcq %rsi, %rbx
 ; X64-NEXT:    setb %cl
 ; X64-NEXT:    movq %r9, %rax

 ; X64-NEXT:    adcq $0, %rcx
 ; X64-NEXT:    addq -{{[0-9]+}}(%rsp), %rsi # 8-byte Folded Reload
 ; X64-NEXT:    movq %rsi, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    adcq -{{[0-9]+}}(%rsp), %rcx # 8-byte Folded Reload
 ; X64-NEXT:    movq %rcx, (%rsp) # 8-byte Spill
 ; X64-NEXT:    setb -{{[0-9]+}}(%rsp) # 1-byte Folded Spill
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rbx # 8-byte Reload
 ; X64-NEXT:    movq %rbx, %rax
-; X64-NEXT:    movq %r10, %rsi
-; X64-NEXT:    mulq %rsi
+; X64-NEXT:    mulq %r10
 ; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    movq %rax, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %r8 # 8-byte Reload
 ; X64-NEXT:    movq %r8, %rax
-; X64-NEXT:    mulq %rsi
+; X64-NEXT:    mulq %r10
 ; X64-NEXT:    movq %rdx, %rsi
 ; X64-NEXT:    movq %rax, %rdi
 ; X64-NEXT:    addq %rcx, %rdi
 ; X64-NEXT:    adcq $0, %rsi
 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    mulq %r9
 ; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    movq %rax, %r10

 ; X64-NEXT:    movq %rdx, %rsi
 ; X64-NEXT:    movq %rax, %r8
 ; X64-NEXT:    addq %rbx, %r8
 ; X64-NEXT:    adcq %rbp, %rsi
 ; X64-NEXT:    setb %bl
 ; X64-NEXT:    movq %r9, %rax
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rcx, %r10
-; X64-NEXT:    movq %r10, -{{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    movq %rcx, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    movq %rax, %rdi
 ; X64-NEXT:    addq %rsi, %rdi
 ; X64-NEXT:    movzbl %bl, %eax
 ; X64-NEXT:    adcq %rax, %rcx
 ; X64-NEXT:    movq %r11, %rax
 ; X64-NEXT:    xorl %edx, %edx
 ; X64-NEXT:    mulq %rdx
 ; X64-NEXT:    movq %rax, %rbx
 ; X64-NEXT:    movq %rdx, %r14
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %r12 # 8-byte Reload
-; X64-NEXT:    addq %rbx, %r12
+; X64-NEXT:    addq %rax, %r12
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %r15 # 8-byte Reload
-; X64-NEXT:    adcq %r14, %r15
+; X64-NEXT:    adcq %rdx, %r15
 ; X64-NEXT:    addq %rdi, %r12
 ; X64-NEXT:    adcq %rcx, %r15
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rcx # 8-byte Reload
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    movq %r11, %rsi
-; X64-NEXT:    movq %rsi, -{{[0-9]+}}(%rsp) # 8-byte Spill
-; X64-NEXT:    mulq %rsi
+; X64-NEXT:    movq %r11, -{{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    mulq %r11
 ; X64-NEXT:    movq %rdx, %r11
 ; X64-NEXT:    movq %rax, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %r9 # 8-byte Reload
 ; X64-NEXT:    movq %r9, %rax
 ; X64-NEXT:    mulq %rsi
 ; X64-NEXT:    movq %rdx, %rsi
 ; X64-NEXT:    movq %rax, %rdi
 ; X64-NEXT:    addq %r11, %rdi

 ; X64-NEXT:    movzbl %r11b, %eax
 ; X64-NEXT:    adcq %rax, %rcx
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    xorl %edx, %edx
 ; X64-NEXT:    mulq %rdx
 ; X64-NEXT:    movq %rdx, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rax, %r9
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rbp # 8-byte Reload
-; X64-NEXT:    addq %r9, %rbp
+; X64-NEXT:    addq %rax, %rbp
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rax # 8-byte Reload
 ; X64-NEXT:    adcq %rdx, %rax
 ; X64-NEXT:    addq %rsi, %rbp
 ; X64-NEXT:    adcq %rcx, %rax
 ; X64-NEXT:    addq %rbx, %r13
 ; X64-NEXT:    movq %r13, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    adcq %r14, %r8
 ; X64-NEXT:    movq %r8, {{[0-9]+}}(%rsp) # 8-byte Spill

 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rcx # 8-byte Reload
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rax, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rdx, %r8
 ; X64-NEXT:    movq 88(%rsi), %rax
 ; X64-NEXT:    movq %rsi, %r9
 ; X64-NEXT:    movq %rax, %rsi
-; X64-NEXT:    movq %rsi, -{{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    movq %rax, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rcx, %r11
 ; X64-NEXT:    movq %rdx, %rbp
 ; X64-NEXT:    movq %rax, %rbx
 ; X64-NEXT:    addq %r8, %rbx
 ; X64-NEXT:    adcq $0, %rbp
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    movq %rdi, -{{[0-9]+}}(%rsp) # 8-byte Spill

 ; X64-NEXT:    movq %rax, %rsi
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %r12 # 8-byte Reload
 ; X64-NEXT:    addq %r12, %rsi
 ; X64-NEXT:    movq %rdx, %r10
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %r8 # 8-byte Reload
 ; X64-NEXT:    adcq %r8, %r10
 ; X64-NEXT:    addq %rbx, %rsi
 ; X64-NEXT:    adcq %rbp, %r10
-; X64-NEXT:    movq %r9, %rdi
-; X64-NEXT:    movq 64(%rdi), %r13
+; X64-NEXT:    movq 64(%r9), %r13
 ; X64-NEXT:    movq %r13, %rax
 ; X64-NEXT:    mulq %r11
 ; X64-NEXT:    movq %rax, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rdx, %rcx
-; X64-NEXT:    movq 72(%rdi), %r9
+; X64-NEXT:    movq 72(%r9), %r9
 ; X64-NEXT:    movq %r9, %rax
 ; X64-NEXT:    mulq %r11
 ; X64-NEXT:    movq %rdx, %rbp
 ; X64-NEXT:    movq %rax, %rbx
 ; X64-NEXT:    addq %rcx, %rbx
 ; X64-NEXT:    adcq $0, %rbp
 ; X64-NEXT:    movq %r13, %rax
 ; X64-NEXT:    mulq %r15

 ; X64-NEXT:    movzbl %r11b, %eax
 ; X64-NEXT:    adcq %rax, %rbx
 ; X64-NEXT:    movq %r13, %rax
 ; X64-NEXT:    xorl %ecx, %ecx
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rdx, %r11
 ; X64-NEXT:    movq %rax, %r15
 ; X64-NEXT:    movq %r12, %rcx
-; X64-NEXT:    addq %r15, %rcx
-; X64-NEXT:    adcq %r11, %r8
+; X64-NEXT:    addq %rax, %rcx
+; X64-NEXT:    adcq %rdx, %r8
 ; X64-NEXT:    addq %rbp, %rcx
 ; X64-NEXT:    adcq %rbx, %r8
 ; X64-NEXT:    addq -{{[0-9]+}}(%rsp), %rcx # 8-byte Folded Reload
 ; X64-NEXT:    movq %rcx, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    adcq %r14, %r8
 ; X64-NEXT:    movq %r8, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    adcq $0, %rsi
 ; X64-NEXT:    adcq $0, %r10

 ; X64-NEXT:    movq %rbp, {{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    adcq $0, %r15
 ; X64-NEXT:    adcq $0, %r11
 ; X64-NEXT:    addq %rsi, %r15
 ; X64-NEXT:    adcq %r10, %r11
 ; X64-NEXT:    setb %r10b
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %rsi # 8-byte Reload
 ; X64-NEXT:    movq %rsi, %rax
-; X64-NEXT:    movq %r8, %rdi
-; X64-NEXT:    mulq %rdi
+; X64-NEXT:    mulq %r8
 ; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    movq %rax, %r9
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %rbp # 8-byte Reload
 ; X64-NEXT:    movq %rbp, %rax
-; X64-NEXT:    mulq %rdi
-; X64-NEXT:    movq %rdi, %r12
+; X64-NEXT:    mulq %r8
+; X64-NEXT:    movq %r8, %r12
 ; X64-NEXT:    movq %rdx, %rdi
 ; X64-NEXT:    movq %rax, %rbx
 ; X64-NEXT:    addq %rcx, %rbx
 ; X64-NEXT:    adcq $0, %rdi
 ; X64-NEXT:    movq %rsi, %rax
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %rsi # 8-byte Reload
 ; X64-NEXT:    mulq %rsi
 ; X64-NEXT:    movq %rdx, %rcx

 ; X64-NEXT:    movq %rsi, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    adcq $0, %rcx
 ; X64-NEXT:    movq %rcx, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq {{[0-9]+}}(%rsp), %rbp # 8-byte Reload
 ; X64-NEXT:    movq 96(%rbp), %rcx
 ; X64-NEXT:    imulq %rcx, %rdi
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    movq %r12, %rsi
-; X64-NEXT:    mulq %rsi
+; X64-NEXT:    mulq %r12
 ; X64-NEXT:    movq %rax, %r9
 ; X64-NEXT:    addq %rdi, %rdx
 ; X64-NEXT:    movq 104(%rbp), %r8
 ; X64-NEXT:    imulq %r8, %rsi
 ; X64-NEXT:    addq %rdx, %rsi
 ; X64-NEXT:    movq %rsi, %r11
 ; X64-NEXT:    movq 112(%rbp), %rax
 ; X64-NEXT:    movq %rbp, %rdi

llvm/trunk/test/CodeGen/X86/mul-i256.ll

 ; X32-NEXT:    movl %eax, %ebx
 ; X32-NEXT:    addl %ecx, %ebx
 ; X32-NEXT:    adcl $0, %esi
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    movl 4(%eax), %ecx
 ; X32-NEXT:    movl %edi, %eax
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %ecx, %edi
-; X32-NEXT:    movl %edi, {{[0-9]+}}(%esp) # 4-byte Spill
+; X32-NEXT:    movl %ecx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    addl %ebx, %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    adcl %esi, %ecx
 ; X32-NEXT:    setb {{[0-9]+}}(%esp) # 1-byte Folded Spill
 ; X32-NEXT:    movl %ebp, %eax
 ; X32-NEXT:    mull %edi
 ; X32-NEXT:    movl %edx, %ebx
 ; X32-NEXT:    movl %eax, %edi
 ; X32-NEXT:    addl %ecx, %edi
 ; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax # 1-byte Folded Reload
 ; X32-NEXT:    adcl %eax, %ebx
 ; X32-NEXT:    xorl %edx, %edx
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ecx # 4-byte Reload
 ; X32-NEXT:    movl %ecx, %eax
 ; X32-NEXT:    mull %edx
 ; X32-NEXT:    movl %edx, %ebp
-; X32-NEXT:    movl %ebp, {{[0-9]+}}(%esp) # 4-byte Spill
+; X32-NEXT:    movl %edx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, %esi
-; X32-NEXT:    movl %esi, (%esp) # 4-byte Spill
+; X32-NEXT:    movl %eax, (%esp) # 4-byte Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax # 4-byte Reload
 ; X32-NEXT:    xorl %edx, %edx
 ; X32-NEXT:    mull %edx
 ; X32-NEXT:    movl %edx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    addl %esi, %eax
 ; X32-NEXT:    adcl %ebp, %edx
 ; X32-NEXT:    addl %edi, %eax

 ; X32-NEXT:    addl {{[0-9]+}}(%esp), %ecx # 4-byte Folded Reload
 ; X32-NEXT:    movl %ecx, (%esp) # 4-byte Spill
 ; X32-NEXT:    adcl {{[0-9]+}}(%esp), %eax # 4-byte Folded Reload
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    adcl $0, {{[0-9]+}}(%esp) # 4-byte Folded Spill
 ; X32-NEXT:    adcl $0, {{[0-9]+}}(%esp) # 4-byte Folded Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    movl %eax, %ecx
-; X32-NEXT:    movl 8(%ecx), %ebx
+; X32-NEXT:    movl 8(%eax), %ebx
 ; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    movl %esi, %edi
 ; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %edx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax # 4-byte Reload
 ; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %edx, %ebp

 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax # 4-byte Reload
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %edx, %esi
 ; X32-NEXT:    movl %eax, %ebp
 ; X32-NEXT:    addl %edi, %ebp
 ; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax # 1-byte Folded Reload
 ; X32-NEXT:    adcl %eax, %esi
 ; X32-NEXT:    movl %ebx, %edi
-; X32-NEXT:    movl %edi, %eax
+; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    xorl %ecx, %ecx
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %edx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ebx # 4-byte Reload
 ; X32-NEXT:    addl %eax, %ebx
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax # 4-byte Reload
 ; X32-NEXT:    adcl %edx, %eax

llvm/trunk/test/CodeGen/X86/mul-i512.ll

 ; X32-NEXT:    addl %ecx, %edi
 ; X32-NEXT:    movl %edx, %ebx
 ; X32-NEXT:    adcl $0, %ebx
 ; X32-NEXT:    addl %esi, %edi
 ; X32-NEXT:    movl %esi, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %edi, (%esp) # 4-byte Spill
 ; X32-NEXT:    adcl %ecx, %ebx
 ; X32-NEXT:    movl %ecx, %edi
-; X32-NEXT:    movl %edi, {{[0-9]+}}(%esp) # 4-byte Spill
+; X32-NEXT:    movl %ecx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    setb %cl
 ; X32-NEXT:    addl %eax, %ebx
 ; X32-NEXT:    movzbl %cl, %ecx
 ; X32-NEXT:    adcl %edx, %ecx
 ; X32-NEXT:    movl 24(%ebp), %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    xorl %edx, %edx
 ; X32-NEXT:    mull %edx
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    addl %eax, %esi
 ; X32-NEXT:    adcl %edx, %edi
 ; X32-NEXT:    addl %ebx, %esi
 ; X32-NEXT:    movl %esi, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    adcl %ecx, %edi
 ; X32-NEXT:    movl %edi, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; X32-NEXT:    movl (%ecx), %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    xorl %ebx, %ebx
 ; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %eax, %ebp
 ; X32-NEXT:    movl %edx, %edi
-; X32-NEXT:    movl %edi, {{[0-9]+}}(%esp) # 4-byte Spill
+; X32-NEXT:    movl %edx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl 4(%ecx), %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %ecx, %esi
 ; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %eax, %ebx
 ; X32-NEXT:    addl %edi, %ebx
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    adcl $0, %ecx

 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax # 4-byte Reload
 ; X32-NEXT:    addl %ebp, %eax
 ; X32-NEXT:    movl %ebp, %esi
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax # 4-byte Reload
 ; X32-NEXT:    adcl %edi, %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ecx
-; X32-NEXT:    movl %ecx, %eax
-; X32-NEXT:    movl (%eax), %eax
+; X32-NEXT:    movl (%ecx), %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    xorl %ebp, %ebp
 ; X32-NEXT:    mull %ebp
 ; X32-NEXT:    movl %edx, %ebx
 ; X32-NEXT:    movl %eax, %ecx
-; X32-NEXT:    movl %ecx, %edx
+; X32-NEXT:    movl %eax, %edx
 ; X32-NEXT:    addl %esi, %edx
 ; X32-NEXT:    movl %esi, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    adcl %edi, %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    movl 16(%eax), %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    mull %ebp
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %ecx, %edi
 ; X32-NEXT:    movl %ecx, %ebp
-; X32-NEXT:    movl %ebp, {{[0-9]+}}(%esp) # 4-byte Spill
+; X32-NEXT:    movl %ecx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    addl %eax, %edi
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    adcl %edx, %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax # 4-byte Reload
 ; X32-NEXT:    addl %esi, %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl (%esp), %eax # 4-byte Reload

 ; X32-NEXT:    movl %eax, %esi
 ; X32-NEXT:    addl %ebx, %esi
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    adcl $0, %ecx
 ; X32-NEXT:    addl %ebp, %esi
 ; X32-NEXT:    movl %esi, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    adcl %ebx, %ecx
 ; X32-NEXT:    movl %ebx, %esi
-; X32-NEXT:    movl %esi, {{[0-9]+}}(%esp) # 4-byte Spill
+; X32-NEXT:    movl %ebx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    setb %bl
 ; X32-NEXT:    addl %eax, %ecx
 ; X32-NEXT:    movzbl %bl, %ebx
 ; X32-NEXT:    adcl %edx, %ebx
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    movl 8(%eax), %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    mull %edi

 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %esi # 4-byte Reload
 ; X32-NEXT:    mull %esi
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    addl %ebp, %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    adcl %ebx, %ecx
 ; X32-NEXT:    setb {{[0-9]+}}(%esp) # 1-byte Folded Spill
 ; X32-NEXT:    movl %edi, %ebp
-; X32-NEXT:    movl %ebp, %eax
+; X32-NEXT:    movl %edi, %eax
 ; X32-NEXT:    mull %esi
 ; X32-NEXT:    movl %edx, %edi
 ; X32-NEXT:    movl %eax, %ebx
 ; X32-NEXT:    addl %ecx, %ebx
 ; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax # 1-byte Folded Reload
 ; X32-NEXT:    adcl %eax, %edi
 ; X32-NEXT:    addl {{[0-9]+}}(%esp), %ebx # 4-byte Folded Reload
 ; X32-NEXT:    adcl {{[0-9]+}}(%esp), %edi # 4-byte Folded Reload

 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %esi # 4-byte Reload
 ; X32-NEXT:    mull %esi
 ; X32-NEXT:    movl %edx, %ecx
 ; X32-NEXT:    addl %ebp, %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    adcl %edi, %ecx
 ; X32-NEXT:    setb {{[0-9]+}}(%esp) # 1-byte Folded Spill
 ; X32-NEXT:    movl %ebx, %edi
-; X32-NEXT:    movl %edi, %eax
+; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    mull %esi
 ; X32-NEXT:    movl %eax, %ebp
 ; X32-NEXT:    addl %ecx, %ebp
 ; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax # 1-byte Folded Reload
 ; X32-NEXT:    adcl %eax, %edx
 ; X32-NEXT:    addl {{[0-9]+}}(%esp), %ebp # 4-byte Folded Reload
 ; X32-NEXT:    adcl {{[0-9]+}}(%esp), %edx # 4-byte Folded Reload
 ; X32-NEXT:    addl {{[0-9]+}}(%esp), %ebp # 4-byte Folded Reload

 ; X32-NEXT:    movl %eax, %ebx
 ; X32-NEXT:    addl %esi, %ebx
 ; X32-NEXT:    adcl $0, %edi
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    movl 36(%eax), %esi
 ; X32-NEXT:    movl %ecx, %eax
 ; X32-NEXT:    mull %esi
 ; X32-NEXT:    movl %esi, %ecx
-; X32-NEXT:    movl %ecx, {{[0-9]+}}(%esp) # 4-byte Spill
+; X32-NEXT:    movl %esi, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %esi
 ; X32-NEXT:    addl %ebx, %eax
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    adcl %edi, %esi
 ; X32-NEXT:    setb %bl
 ; X32-NEXT:    movl %ebp, %eax
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %edx, %ecx

 ; X32-NEXT:    adcl %edx, %esi
 ; X32-NEXT:    addl %ebp, %edi
 ; X32-NEXT:    movl %edi, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    adcl %ecx, %esi
 ; X32-NEXT:    movl %esi, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ecx # 4-byte Reload
 ; X32-NEXT:    movl %ecx, %eax
 ; X32-NEXT:    movl %ebx, %esi
-; X32-NEXT:    mull %esi
+; X32-NEXT:    mull %ebx
 ; X32-NEXT:    movl %edx, %edi
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ebx # 4-byte Reload
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    mull %esi
 ; X32-NEXT:    movl %edx, %esi
 ; X32-NEXT:    movl %eax, %ebp
 ; X32-NEXT:    addl %edi, %ebp

 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ebx # 4-byte Reload
 ; X32-NEXT:    imull %ebx, %esi
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %edi # 4-byte Reload
 ; X32-NEXT:    mull %edi
 ; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    addl %esi, %edx
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %esi # 4-byte Reload
 ; X32-NEXT:    movl %edi, %eax
-; X32-NEXT:    imull %eax, %esi
+; X32-NEXT:    imull %edi, %esi
 ; X32-NEXT:    addl %edx, %esi
 ; X32-NEXT:    addl {{[0-9]+}}(%esp), %ecx # 4-byte Folded Reload
 ; X32-NEXT:    movl %ecx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    adcl %ebp, %esi
 ; X32-NEXT:    movl %esi, %edi
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ecx # 4-byte Reload
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %edx, %ebp

 ; X32-NEXT:    adcl {{[0-9]+}}(%esp), %ebx # 4-byte Folded Reload
 ; X32-NEXT:    movl %ebx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    adcl {{[0-9]+}}(%esp), %eax # 4-byte Folded Reload
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    adcl {{[0-9]+}}(%esp), %edx # 4-byte Folded Reload
 ; X32-NEXT:    movl %edx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %esi
 ; X32-NEXT:    movl %esi, %ecx
-; X32-NEXT:    movl 40(%ecx), %ebx
+; X32-NEXT:    movl 40(%esi), %ebx
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    movl %ebx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %esi # 4-byte Reload
 ; X32-NEXT:    mull %esi
 ; X32-NEXT:    movl %eax, (%esp) # 4-byte Spill
 ; X32-NEXT:    movl %edx, %ebp
 ; X32-NEXT:    movl 44(%ecx), %ecx
 ; X32-NEXT:    movl %ecx, %eax

 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %esi # 4-byte Reload
 ; X32-NEXT:    imull %esi, %edi
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ecx # 4-byte Reload
 ; X32-NEXT:    mull %ecx
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    addl %edi, %edx
 ; X32-NEXT:    movl 60(%ebx), %ebx
 ; X32-NEXT:    movl %ecx, %eax
-; X32-NEXT:    imull %eax, %ebx
+; X32-NEXT:    imull %ecx, %ebx
 ; X32-NEXT:    addl %edx, %ebx
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ecx # 4-byte Reload
 ; X32-NEXT:    addl %ecx, {{[0-9]+}}(%esp) # 4-byte Folded Spill
 ; X32-NEXT:    adcl {{[0-9]+}}(%esp), %ebx # 4-byte Folded Reload
 ; X32-NEXT:    mull %ebp
 ; X32-NEXT:    movl %edx, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %eax, {{[0-9]+}}(%esp) # 4-byte Spill
 ; X32-NEXT:    movl %esi, %eax

 ; X64-NEXT:    movq %rdx, (%rsp) # 8-byte Spill
 ; X64-NEXT:    movq 24(%rdi), %r11
 ; X64-NEXT:    movq 16(%rdi), %r15
 ; X64-NEXT:    movq %rsi, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq (%rsi), %rdx
 ; X64-NEXT:    movq 8(%rsi), %rbp
 ; X64-NEXT:    movq %r15, %rax
 ; X64-NEXT:    movq %rdx, %rsi
-; X64-NEXT:    mulq %rsi
+; X64-NEXT:    mulq %rdx
 ; X64-NEXT:    movq %rdx, %r9
 ; X64-NEXT:    movq %rax, %r8
 ; X64-NEXT:    movq %r11, %rax
 ; X64-NEXT:    movq %r11, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    mulq %rsi
 ; X64-NEXT:    movq %rsi, %r10
 ; X64-NEXT:    movq %rdx, %rbx
 ; X64-NEXT:    movq %rax, %rsi
 ; X64-NEXT:    addq %r9, %rsi
 ; X64-NEXT:    adcq $0, %rbx
 ; X64-NEXT:    movq %r15, %rax
 ; X64-NEXT:    movq %r15, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    mulq %rbp
 ; X64-NEXT:    movq %rdx, %rcx
 ; X64-NEXT:    movq %rax, %r9
 ; X64-NEXT:    addq %rsi, %r9
 ; X64-NEXT:    adcq %rbx, %rcx
 ; X64-NEXT:    setb %al
 ; X64-NEXT:    movzbl %al, %ebx
 ; X64-NEXT:    movq %r11, %rax
 ; X64-NEXT:    mulq %rbp
 ; X64-NEXT:    movq %rbp, %r14
-; X64-NEXT:    movq %r14, -{{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    movq %rbp, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rdx, %rsi
 ; X64-NEXT:    movq %rax, %rbp
 ; X64-NEXT:    addq %rcx, %rbp
 ; X64-NEXT:    adcq %rbx, %rsi
 ; X64-NEXT:    xorl %ecx, %ecx
 ; X64-NEXT:    movq %r10, %rbx
-; X64-NEXT:    movq %rbx, -{{[0-9]+}}(%rsp) # 8-byte Spill
-; X64-NEXT:    movq %rbx, %rax
+; X64-NEXT:    movq %r10, -{{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    movq %r10, %rax
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rdx, %r13
 ; X64-NEXT:    movq %rax, %r10
 ; X64-NEXT:    movq %r15, %rax
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rdx, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rax, %r15
-; X64-NEXT:    movq %r15, -{{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    movq %rax, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    addq %r10, %r15
 ; X64-NEXT:    adcq %r13, %rdx
 ; X64-NEXT:    addq %rbp, %r15
 ; X64-NEXT:    adcq %rsi, %rdx
 ; X64-NEXT:    movq %rdx, %r12
 ; X64-NEXT:    movq %rdi, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq (%rdi), %rcx
 ; X64-NEXT:    movq %rcx, %rax

 ; X64-NEXT:    addq %rbx, %rbp
 ; X64-NEXT:    movzbl %r11b, %eax
 ; X64-NEXT:    adcq %rax, %rsi
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    xorl %edx, %edx
 ; X64-NEXT:    mulq %rdx
 ; X64-NEXT:    movq %rdx, %r14
 ; X64-NEXT:    movq %rax, %r11
-; X64-NEXT:    addq %r11, %r10
-; X64-NEXT:    adcq %r14, %r13
+; X64-NEXT:    addq %rax, %r10
+; X64-NEXT:    adcq %rdx, %r13
 ; X64-NEXT:    addq %rbp, %r10
 ; X64-NEXT:    adcq %rsi, %r13
 ; X64-NEXT:    addq %r8, %r10
 ; X64-NEXT:    adcq %r9, %r13
 ; X64-NEXT:    adcq $0, %r15
 ; X64-NEXT:    adcq $0, %r12
 ; X64-NEXT:    movq %r12, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %rsi # 8-byte Reload
 ; X64-NEXT:    movq 16(%rsi), %r8
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    movq %rcx, %r9
-; X64-NEXT:    movq %r9, -{{[0-9]+}}(%rsp) # 8-byte Spill
+; X64-NEXT:    movq %rcx, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    mulq %r8
 ; X64-NEXT:    movq %rdx, %rdi
 ; X64-NEXT:    movq %rax, %r12
 ; X64-NEXT:    movq -{{[0-9]+}}(%rsp), %rcx # 8-byte Reload
 ; X64-NEXT:    movq %rcx, %rax
 ; X64-NEXT:    mulq %r8
 ; X64-NEXT:    movq %rdx, %rbp
 ; X64-NEXT:    movq %rax, %rbx

 ; X64-NEXT:    addq %rsi, %r9
 ; X64-NEXT:    movzbl %bpl, %eax
 ; X64-NEXT:    adcq %rax, %rbx
 ; X64-NEXT:    movq %r8, %rax
 ; X64-NEXT:    xorl %ecx, %ecx
 ; X64-NEXT:    mulq %rcx
 ; X64-NEXT:    movq %rdx, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    movq %rax, %rbp
-; X64-NEXT:    addq %rbp, %r11
+; X64-NEXT:    addq %rax, %r11
 ; X64-NEXT:    adcq %rdx, %r14
 ; X64-NEXT:    addq %r9, %r11
 ; X64-NEXT:    adcq %rbx, %r14
 ; X64-NEXT:    addq %r10, %r12
 ; X64-NEXT:    movq %r12, -{{[0-9]+}}(%rsp) # 8-byte Spill
 ; X64-NEXT:    adcq %r13, -{{[0-9]+}}(%rsp) # 8-byte Folded Spill
 ; X64-NEXT:    adcq $0, %r11
 ; X64-NEXT:    adcq $0, %r14

llvm/trunk/test/CodeGen/X86/mul128.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown | FileCheck %s --check-prefix=X64
 ; RUN: llc < %s -mtriple=i386-unknown | FileCheck %s --check-prefix=X86
 
 define i128 @foo(i128 %t, i128 %u) {
 ; X64-LABEL: foo:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdx, %r8
 ; X64-NEXT:    imulq %rdi, %rcx
 ; X64-NEXT:    movq %rdi, %rax
-; X64-NEXT:    mulq %r8
+; X64-NEXT:    mulq %rdx
 ; X64-NEXT:    addq %rcx, %rdx
 ; X64-NEXT:    imulq %r8, %rsi
 ; X64-NEXT:    addq %rsi, %rdx
 ; X64-NEXT:    retq
 ;
 ; X86-LABEL: foo:
 ; X86:       # %bb.0:
 ; X86-NEXT:    pushl %ebp

llvm/trunk/test/CodeGen/X86/mulvi32.ll

 }
 
 ; very similar to mul4xi32 above
 ; there is no bitcast and the final shuffle is a little different
 define <4 x i64> @_mul4xi32toi64b(<4 x i32>, <4 x i32>) {
 ; SSE-LABEL: _mul4xi32toi64b:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movdqa %xmm0, %xmm2
-; SSE-NEXT:    pshufd {{.*#+}} xmm0 = xmm2[1,1,3,3]
+; SSE-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[1,1,3,3]
 ; SSE-NEXT:    pmuludq %xmm1, %xmm2
 ; SSE-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[1,1,3,3]
 ; SSE-NEXT:    pmuludq %xmm0, %xmm1
 ; SSE-NEXT:    movdqa %xmm2, %xmm0
 ; SSE-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm1[0]
 ; SSE-NEXT:    punpckhqdq {{.*#+}} xmm2 = xmm2[1],xmm1[1]
 ; SSE-NEXT:    movdqa %xmm2, %xmm1
 ; SSE-NEXT:    retq

llvm/trunk/test/CodeGen/X86/pmul.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown | FileCheck %s --check-prefix=SSE --check-prefix=SSE2
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+sse4.1 | FileCheck %s --check-prefix=SSE --check-prefix=SSE41
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx2 | FileCheck %s --check-prefix=AVX --check-prefix=AVX2
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512f | FileCheck %s --check-prefix=AVX --check-prefix=AVX512 --check-prefix=AVX512F
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512bw | FileCheck %s --check-prefix=AVX --check-prefix=AVX512 --check-prefix=AVX512BW
 
 define <16 x i8> @mul_v16i8c(<16 x i8> %i) nounwind  {
 ; SSE2-LABEL: mul_v16i8c:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm0, %xmm1
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm1 = xmm1[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm1 = xmm1[8],xmm0[8],xmm1[9],xmm0[9],xmm1[10],xmm0[10],xmm1[11],xmm0[11],xmm1[12],xmm0[12],xmm1[13],xmm0[13],xmm1[14],xmm0[14],xmm1[15],xmm0[15]
 ; SSE2-NEXT:    psraw $8, %xmm1
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm2 = [117,117,117,117,117,117,117,117]
 ; SSE2-NEXT:    pmullw %xmm2, %xmm1
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm3 = [255,255,255,255,255,255,255,255]
 ; SSE2-NEXT:    pand %xmm3, %xmm1
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm0
 ; SSE2-NEXT:    pmullw %xmm2, %xmm0
entry:
   %A = mul <2 x i64> %i, < i64 117, i64 117 >
   ret <2 x i64> %A
 }
 
 define <16 x i8> @mul_v16i8(<16 x i8> %i, <16 x i8> %j) nounwind  {
 ; SSE2-LABEL: mul_v16i8:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8],xmm1[8],xmm2[9],xmm1[9],xmm2[10],xmm1[10],xmm2[11],xmm1[11],xmm2[12],xmm1[12],xmm2[13],xmm1[13],xmm2[14],xmm1[14],xmm2[15],xmm1[15]
 ; SSE2-NEXT:    psraw $8, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm3
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm3 = xmm3[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm3 = xmm3[8],xmm0[8],xmm3[9],xmm0[9],xmm3[10],xmm0[10],xmm3[11],xmm0[11],xmm3[12],xmm0[12],xmm3[13],xmm0[13],xmm3[14],xmm0[14],xmm3[15],xmm0[15]
 ; SSE2-NEXT:    psraw $8, %xmm3
 ; SSE2-NEXT:    pmullw %xmm2, %xmm3
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm2 = [255,255,255,255,255,255,255,255]
 ; SSE2-NEXT:    pand %xmm2, %xmm3
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm1
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm0
entry:
   %A = mul <2 x i64> %i, %j
   ret <2 x i64> %A
 }
 
 define <32 x i8> @mul_v32i8c(<32 x i8> %i) nounwind  {
 ; SSE2-LABEL: mul_v32i8c:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm0, %xmm2
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8],xmm0[8],xmm2[9],xmm0[9],xmm2[10],xmm0[10],xmm2[11],xmm0[11],xmm2[12],xmm0[12],xmm2[13],xmm0[13],xmm2[14],xmm0[14],xmm2[15],xmm0[15]
 ; SSE2-NEXT:    psraw $8, %xmm2
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm3 = [117,117,117,117,117,117,117,117]
 ; SSE2-NEXT:    pmullw %xmm3, %xmm2
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm4 = [255,255,255,255,255,255,255,255]
 ; SSE2-NEXT:    pand %xmm4, %xmm2
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm0
 ; SSE2-NEXT:    pmullw %xmm3, %xmm0
 ; SSE2-NEXT:    pand %xmm4, %xmm0
 ; SSE2-NEXT:    packuswb %xmm2, %xmm0
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8],xmm1[8],xmm2[9],xmm1[9],xmm2[10],xmm1[10],xmm2[11],xmm1[11],xmm2[12],xmm1[12],xmm2[13],xmm1[13],xmm2[14],xmm1[14],xmm2[15],xmm1[15]
 ; SSE2-NEXT:    psraw $8, %xmm2
 ; SSE2-NEXT:    pmullw %xmm3, %xmm2
 ; SSE2-NEXT:    pand %xmm4, %xmm2
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm1
 ; SSE2-NEXT:    pmullw %xmm3, %xmm1
 ; SSE2-NEXT:    pand %xmm4, %xmm1
 ; SSE2-NEXT:    packuswb %xmm2, %xmm1
entry:
   %A = mul <4 x i64> %i, < i64 117, i64 117, i64 117, i64 117 >
   ret <4 x i64> %A
 }
 
 define <32 x i8> @mul_v32i8(<32 x i8> %i, <32 x i8> %j) nounwind  {
 ; SSE2-LABEL: mul_v32i8:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm2, %xmm4
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm4 = xmm4[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm4 = xmm4[8],xmm2[8],xmm4[9],xmm2[9],xmm4[10],xmm2[10],xmm4[11],xmm2[11],xmm4[12],xmm2[12],xmm4[13],xmm2[13],xmm4[14],xmm2[14],xmm4[15],xmm2[15]
 ; SSE2-NEXT:    psraw $8, %xmm4
 ; SSE2-NEXT:    movdqa %xmm0, %xmm5
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm5 = xmm5[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm5 = xmm5[8],xmm0[8],xmm5[9],xmm0[9],xmm5[10],xmm0[10],xmm5[11],xmm0[11],xmm5[12],xmm0[12],xmm5[13],xmm0[13],xmm5[14],xmm0[14],xmm5[15],xmm0[15]
 ; SSE2-NEXT:    psraw $8, %xmm5
 ; SSE2-NEXT:    pmullw %xmm4, %xmm5
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm4 = [255,255,255,255,255,255,255,255]
 ; SSE2-NEXT:    pand %xmm4, %xmm5
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm2 = xmm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm2
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm0
 ; SSE2-NEXT:    pmullw %xmm2, %xmm0
 ; SSE2-NEXT:    pand %xmm4, %xmm0
 ; SSE2-NEXT:    packuswb %xmm5, %xmm0
 ; SSE2-NEXT:    movdqa %xmm3, %xmm2
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8],xmm3[8],xmm2[9],xmm3[9],xmm2[10],xmm3[10],xmm2[11],xmm3[11],xmm2[12],xmm3[12],xmm2[13],xmm3[13],xmm2[14],xmm3[14],xmm2[15],xmm3[15]
 ; SSE2-NEXT:    psraw $8, %xmm2
 ; SSE2-NEXT:    movdqa %xmm1, %xmm5
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm5 = xmm5[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm5 = xmm5[8],xmm1[8],xmm5[9],xmm1[9],xmm5[10],xmm1[10],xmm5[11],xmm1[11],xmm5[12],xmm1[12],xmm5[13],xmm1[13],xmm5[14],xmm1[14],xmm5[15],xmm1[15]
 ; SSE2-NEXT:    psraw $8, %xmm5
 ; SSE2-NEXT:    pmullw %xmm2, %xmm5
 ; SSE2-NEXT:    pand %xmm4, %xmm5
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm3 = xmm3[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm3
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm1
 ; SSE2-NEXT:    pmullw %xmm3, %xmm1
entry:
   %A = mul <4 x i64> %i, %j
   ret <4 x i64> %A
 }
 
 define <64 x i8> @mul_v64i8c(<64 x i8> %i) nounwind  {
 ; SSE2-LABEL: mul_v64i8c:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm0, %xmm6
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm6 = xmm6[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm6 = xmm6[8],xmm0[8],xmm6[9],xmm0[9],xmm6[10],xmm0[10],xmm6[11],xmm0[11],xmm6[12],xmm0[12],xmm6[13],xmm0[13],xmm6[14],xmm0[14],xmm6[15],xmm0[15]
 ; SSE2-NEXT:    psraw $8, %xmm6
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm4 = [117,117,117,117,117,117,117,117]
 ; SSE2-NEXT:    pmullw %xmm4, %xmm6
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm5 = [255,255,255,255,255,255,255,255]
 ; SSE2-NEXT:    pand %xmm5, %xmm6
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm0
 ; SSE2-NEXT:    pmullw %xmm4, %xmm0
 ; SSE2-NEXT:    pand %xmm5, %xmm0
 ; SSE2-NEXT:    packuswb %xmm6, %xmm0
 ; SSE2-NEXT:    movdqa %xmm1, %xmm6
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm6 = xmm6[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm6 = xmm6[8],xmm1[8],xmm6[9],xmm1[9],xmm6[10],xmm1[10],xmm6[11],xmm1[11],xmm6[12],xmm1[12],xmm6[13],xmm1[13],xmm6[14],xmm1[14],xmm6[15],xmm1[15]
 ; SSE2-NEXT:    psraw $8, %xmm6
 ; SSE2-NEXT:    pmullw %xmm4, %xmm6
 ; SSE2-NEXT:    pand %xmm5, %xmm6
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm1
 ; SSE2-NEXT:    pmullw %xmm4, %xmm1
 ; SSE2-NEXT:    pand %xmm5, %xmm1
 ; SSE2-NEXT:    packuswb %xmm6, %xmm1
 ; SSE2-NEXT:    movdqa %xmm2, %xmm6
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm6 = xmm6[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm6 = xmm6[8],xmm2[8],xmm6[9],xmm2[9],xmm6[10],xmm2[10],xmm6[11],xmm2[11],xmm6[12],xmm2[12],xmm6[13],xmm2[13],xmm6[14],xmm2[14],xmm6[15],xmm2[15]
 ; SSE2-NEXT:    psraw $8, %xmm6
 ; SSE2-NEXT:    pmullw %xmm4, %xmm6
 ; SSE2-NEXT:    pand %xmm5, %xmm6
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm2 = xmm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm2
 ; SSE2-NEXT:    pmullw %xmm4, %xmm2
 ; SSE2-NEXT:    pand %xmm5, %xmm2
 ; SSE2-NEXT:    packuswb %xmm6, %xmm2
 ; SSE2-NEXT:    movdqa %xmm3, %xmm6
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm6 = xmm6[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm6 = xmm6[8],xmm3[8],xmm6[9],xmm3[9],xmm6[10],xmm3[10],xmm6[11],xmm3[11],xmm6[12],xmm3[12],xmm6[13],xmm3[13],xmm6[14],xmm3[14],xmm6[15],xmm3[15]
 ; SSE2-NEXT:    psraw $8, %xmm6
 ; SSE2-NEXT:    pmullw %xmm4, %xmm6
 ; SSE2-NEXT:    pand %xmm5, %xmm6
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm3 = xmm3[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm3
 ; SSE2-NEXT:    pmullw %xmm4, %xmm3
 ; SSE2-NEXT:    pand %xmm5, %xmm3
 ; SSE2-NEXT:    packuswb %xmm6, %xmm3
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: mul_v64i8c:
 ; SSE41:       # %bb.0: # %entry
 ; SSE41-NEXT:    movdqa %xmm1, %xmm4
 ; SSE41-NEXT:    movdqa %xmm0, %xmm1
-; SSE41-NEXT:    pmovsxbw %xmm1, %xmm0
+; SSE41-NEXT:    pmovsxbw %xmm0, %xmm0
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm6 = [117,117,117,117,117,117,117,117]
 ; SSE41-NEXT:    pmullw %xmm6, %xmm0
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm7 = [255,255,255,255,255,255,255,255]
 ; SSE41-NEXT:    pand %xmm7, %xmm0
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[2,3,0,1]
 ; SSE41-NEXT:    pmovsxbw %xmm1, %xmm1
 ; SSE41-NEXT:    pmullw %xmm6, %xmm1
 ; SSE41-NEXT:    pand %xmm7, %xmm1
entry:
   %A = mul <64 x i8> %i, < i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117, i8 117 >
   ret <64 x i8> %A
 }
 
 define <64 x i8> @mul_v64i8(<64 x i8> %i, <64 x i8> %j) nounwind  {
 ; SSE2-LABEL: mul_v64i8:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm4, %xmm8
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm8 = xmm8[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm8 = xmm8[8],xmm4[8],xmm8[9],xmm4[9],xmm8[10],xmm4[10],xmm8[11],xmm4[11],xmm8[12],xmm4[12],xmm8[13],xmm4[13],xmm8[14],xmm4[14],xmm8[15],xmm4[15]
 ; SSE2-NEXT:    psraw $8, %xmm8
 ; SSE2-NEXT:    movdqa %xmm0, %xmm9
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm9 = xmm9[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm9 = xmm9[8],xmm0[8],xmm9[9],xmm0[9],xmm9[10],xmm0[10],xmm9[11],xmm0[11],xmm9[12],xmm0[12],xmm9[13],xmm0[13],xmm9[14],xmm0[14],xmm9[15],xmm0[15]
 ; SSE2-NEXT:    psraw $8, %xmm9
 ; SSE2-NEXT:    pmullw %xmm8, %xmm9
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm8 = [255,255,255,255,255,255,255,255]
 ; SSE2-NEXT:    pand %xmm8, %xmm9
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm4 = xmm4[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm4
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm0
 ; SSE2-NEXT:    pmullw %xmm4, %xmm0
 ; SSE2-NEXT:    pand %xmm8, %xmm0
 ; SSE2-NEXT:    packuswb %xmm9, %xmm0
 ; SSE2-NEXT:    movdqa %xmm5, %xmm9
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm9 = xmm9[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm9 = xmm9[8],xmm5[8],xmm9[9],xmm5[9],xmm9[10],xmm5[10],xmm9[11],xmm5[11],xmm9[12],xmm5[12],xmm9[13],xmm5[13],xmm9[14],xmm5[14],xmm9[15],xmm5[15]
 ; SSE2-NEXT:    psraw $8, %xmm9
 ; SSE2-NEXT:    movdqa %xmm1, %xmm4
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm4 = xmm4[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm4 = xmm4[8],xmm1[8],xmm4[9],xmm1[9],xmm4[10],xmm1[10],xmm4[11],xmm1[11],xmm4[12],xmm1[12],xmm4[13],xmm1[13],xmm4[14],xmm1[14],xmm4[15],xmm1[15]
 ; SSE2-NEXT:    psraw $8, %xmm4
 ; SSE2-NEXT:    pmullw %xmm9, %xmm4
 ; SSE2-NEXT:    pand %xmm8, %xmm4
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm5 = xmm5[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm5
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm1
 ; SSE2-NEXT:    pmullw %xmm5, %xmm1
 ; SSE2-NEXT:    pand %xmm8, %xmm1
 ; SSE2-NEXT:    packuswb %xmm4, %xmm1
 ; SSE2-NEXT:    movdqa %xmm6, %xmm4
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm4 = xmm4[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm4 = xmm4[8],xmm6[8],xmm4[9],xmm6[9],xmm4[10],xmm6[10],xmm4[11],xmm6[11],xmm4[12],xmm6[12],xmm4[13],xmm6[13],xmm4[14],xmm6[14],xmm4[15],xmm6[15]
 ; SSE2-NEXT:    psraw $8, %xmm4
 ; SSE2-NEXT:    movdqa %xmm2, %xmm5
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm5 = xmm5[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm5 = xmm5[8],xmm2[8],xmm5[9],xmm2[9],xmm5[10],xmm2[10],xmm5[11],xmm2[11],xmm5[12],xmm2[12],xmm5[13],xmm2[13],xmm5[14],xmm2[14],xmm5[15],xmm2[15]
 ; SSE2-NEXT:    psraw $8, %xmm5
 ; SSE2-NEXT:    pmullw %xmm4, %xmm5
 ; SSE2-NEXT:    pand %xmm8, %xmm5
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm6 = xmm6[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm6
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm2 = xmm2[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm2
 ; SSE2-NEXT:    pmullw %xmm6, %xmm2
 ; SSE2-NEXT:    pand %xmm8, %xmm2
 ; SSE2-NEXT:    packuswb %xmm5, %xmm2
 ; SSE2-NEXT:    movdqa %xmm7, %xmm4
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm4 = xmm4[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm4 = xmm4[8],xmm7[8],xmm4[9],xmm7[9],xmm4[10],xmm7[10],xmm4[11],xmm7[11],xmm4[12],xmm7[12],xmm4[13],xmm7[13],xmm4[14],xmm7[14],xmm4[15],xmm7[15]
 ; SSE2-NEXT:    psraw $8, %xmm4
 ; SSE2-NEXT:    movdqa %xmm3, %xmm5
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm5 = xmm5[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm5 = xmm5[8],xmm3[8],xmm5[9],xmm3[9],xmm5[10],xmm3[10],xmm5[11],xmm3[11],xmm5[12],xmm3[12],xmm5[13],xmm3[13],xmm5[14],xmm3[14],xmm5[15],xmm3[15]
 ; SSE2-NEXT:    psraw $8, %xmm5
 ; SSE2-NEXT:    pmullw %xmm4, %xmm5
 ; SSE2-NEXT:    pand %xmm8, %xmm5
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm7 = xmm7[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm7
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm3 = xmm3[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm3
 ; SSE2-NEXT:    pmullw %xmm7, %xmm3
 ; SSE2-NEXT:    pand %xmm8, %xmm3
 ; SSE2-NEXT:    packuswb %xmm5, %xmm3
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: mul_v64i8:
 ; SSE41:       # %bb.0: # %entry
 ; SSE41-NEXT:    movdqa %xmm1, %xmm8
 ; SSE41-NEXT:    movdqa %xmm0, %xmm1
 ; SSE41-NEXT:    pmovsxbw %xmm4, %xmm9
-; SSE41-NEXT:    pmovsxbw %xmm1, %xmm0
+; SSE41-NEXT:    pmovsxbw %xmm0, %xmm0
 ; SSE41-NEXT:    pmullw %xmm9, %xmm0
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm9 = [255,255,255,255,255,255,255,255]
 ; SSE41-NEXT:    pand %xmm9, %xmm0
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm4 = xmm4[2,3,0,1]
 ; SSE41-NEXT:    pmovsxbw %xmm4, %xmm4
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[2,3,0,1]
 ; SSE41-NEXT:    pmovsxbw %xmm1, %xmm1
 ; SSE41-NEXT:    pmullw %xmm4, %xmm1
entry:
   ret <8 x i32> %res
 }
 
 define <8 x i64> @mul_v8i64_sext(<8 x i16> %val1, <8 x i32> %val2) {
 ; SSE2-LABEL: mul_v8i64_sext:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa %xmm1, %xmm4
 ; SSE2-NEXT:    movdqa %xmm0, %xmm1
-; SSE2-NEXT:    punpckhwd {{.*#+}} xmm9 = xmm9[4],xmm1[4],xmm9[5],xmm1[5],xmm9[6],xmm1[6],xmm9[7],xmm1[7]
+; SSE2-NEXT:    punpckhwd {{.*#+}} xmm9 = xmm9[4],xmm0[4],xmm9[5],xmm0[5],xmm9[6],xmm0[6],xmm9[7],xmm0[7]
 ; SSE2-NEXT:    movdqa %xmm9, %xmm0
 ; SSE2-NEXT:    psrad $31, %xmm0
 ; SSE2-NEXT:    psrad $16, %xmm9
 ; SSE2-NEXT:    punpckldq {{.*#+}} xmm9 = xmm9[0],xmm0[0],xmm9[1],xmm0[1]
 ; SSE2-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3]
 ; SSE2-NEXT:    movdqa %xmm0, %xmm3
 ; SSE2-NEXT:    psrad $31, %xmm3
 ; SSE2-NEXT:    psrad $16, %xmm0

llvm/trunk/test/CodeGen/X86/powi.ll

 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=sse2 | FileCheck %s
 ; Ideally this would compile to 5 multiplies.
 
 define double @pow_wrapper(double %a) nounwind readonly ssp noredzone {
 ; CHECK-LABEL: pow_wrapper:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movapd %xmm0, %xmm1
-; CHECK-NEXT:    mulsd %xmm1, %xmm1
+; CHECK-NEXT:    mulsd %xmm0, %xmm1
 ; CHECK-NEXT:    mulsd %xmm1, %xmm0
 ; CHECK-NEXT:    mulsd %xmm1, %xmm1
 ; CHECK-NEXT:    mulsd %xmm1, %xmm0
 ; CHECK-NEXT:    mulsd %xmm1, %xmm1
 ; CHECK-NEXT:    mulsd %xmm0, %xmm1
 ; CHECK-NEXT:    movapd %xmm1, %xmm0
 ; CHECK-NEXT:    retq
   %ret = tail call double @llvm.powi.f64(double %a, i32 15) nounwind ; <double> [#uses=1]

llvm/trunk/test/CodeGen/X86/pr11334.ll

 define <3 x double> @v3f2d_ext_vec(<3 x float> %v1) nounwind {
 ; SSE-LABEL: v3f2d_ext_vec:
 ; SSE:       # %bb.0: # %entry
 ; SSE-NEXT:    cvtps2pd %xmm0, %xmm2
 ; SSE-NEXT:    movhlps {{.*#+}} xmm0 = xmm0[1,1]
 ; SSE-NEXT:    cvtps2pd %xmm0, %xmm0
 ; SSE-NEXT:    movlps %xmm0, -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    movaps %xmm2, %xmm1
-; SSE-NEXT:    movhlps {{.*#+}} xmm1 = xmm1[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm1 = xmm2[1],xmm1[1]
 ; SSE-NEXT:    fldl -{{[0-9]+}}(%rsp)
 ; SSE-NEXT:    movaps %xmm2, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: v3f2d_ext_vec:
 ; AVX:       # %bb.0: # %entry
 ; AVX-NEXT:    vcvtps2pd %xmm0, %ymm0
 ; AVX-NEXT:    retq

llvm/trunk/test/CodeGen/X86/pr29112.ll

 ; CHECK-NEXT:    vinsertps {{.*#+}} xmm0 = xmm0[0,1,2],xmm13[0]
 ; CHECK-NEXT:    vinsertps {{.*#+}} xmm1 = xmm8[0],xmm13[0],xmm8[2,3]
 ; CHECK-NEXT:    vinsertps {{.*#+}} xmm1 = xmm1[0,1],xmm2[1],xmm1[3]
 ; CHECK-NEXT:    vinsertps {{.*#+}} xmm3 = xmm1[0,1,2],xmm3[1]
 ; CHECK-NEXT:    vinsertps {{.*#+}} xmm2 = xmm9[0,1],xmm2[3],xmm9[3]
 ; CHECK-NEXT:    vinsertps {{.*#+}} xmm2 = xmm2[0,1,2],xmm12[0]
 ; CHECK-NEXT:    vaddps %xmm3, %xmm2, %xmm2
 ; CHECK-NEXT:    vmovaps %xmm15, %xmm1
-; CHECK-NEXT:    vmovaps %xmm1, {{[0-9]+}}(%rsp) # 16-byte Spill
-; CHECK-NEXT:    vaddps %xmm0, %xmm1, %xmm9
+; CHECK-NEXT:    vmovaps %xmm15, {{[0-9]+}}(%rsp) # 16-byte Spill
+; CHECK-NEXT:    vaddps %xmm0, %xmm15, %xmm9
 ; CHECK-NEXT:    vaddps %xmm14, %xmm10, %xmm0
-; CHECK-NEXT:    vaddps %xmm1, %xmm1, %xmm8
+; CHECK-NEXT:    vaddps %xmm15, %xmm15, %xmm8
 ; CHECK-NEXT:    vaddps %xmm11, %xmm3, %xmm3
 ; CHECK-NEXT:    vaddps %xmm0, %xmm3, %xmm0
-; CHECK-NEXT:    vaddps %xmm0, %xmm1, %xmm0
+; CHECK-NEXT:    vaddps %xmm0, %xmm15, %xmm0
 ; CHECK-NEXT:    vmovaps %xmm8, {{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    vmovaps %xmm9, (%rsp)
 ; CHECK-NEXT:    vmovaps {{[0-9]+}}(%rsp), %xmm3 # 16-byte Reload
 ; CHECK-NEXT:    vzeroupper
 ; CHECK-NEXT:    callq foo
 ; CHECK-NEXT:    vmovaps {{[0-9]+}}(%rsp), %xmm1 # 16-byte Reload
 ; CHECK-NEXT:    vaddps {{[0-9]+}}(%rsp), %xmm1, %xmm1 # 16-byte Folded Reload
 ; CHECK-NEXT:    vaddps %xmm0, %xmm1, %xmm0

llvm/trunk/test/CodeGen/X86/pr34080-2.ll

 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    cmpl $3, 12(%ebx)
 ; CHECK-NEXT:    setl %al
 ; CHECK-NEXT:    subl %eax, %esi
 ; CHECK-NEXT:    movl $-1374389535, %ecx # imm = 0xAE147AE1
 ; CHECK-NEXT:    movl %esi, %eax
 ; CHECK-NEXT:    imull %ecx
 ; CHECK-NEXT:    movl %edx, %ecx
-; CHECK-NEXT:    movl %ecx, %eax
+; CHECK-NEXT:    movl %edx, %eax
 ; CHECK-NEXT:    shrl $31, %eax
 ; CHECK-NEXT:    sarl $5, %ecx
 ; CHECK-NEXT:    addl %eax, %ecx
 ; CHECK-NEXT:    movl $1374389535, %edx # imm = 0x51EB851F
 ; CHECK-NEXT:    movl %esi, %eax
 ; CHECK-NEXT:    imull %edx
 ; CHECK-NEXT:    movl %edx, %edi
-; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    movl %edx, %eax
 ; CHECK-NEXT:    shrl $31, %eax
 ; CHECK-NEXT:    sarl $7, %edi
 ; CHECK-NEXT:    addl %eax, %edi
 ; CHECK-NEXT:    imull $36525, %esi, %eax # imm = 0x8EAD
 ; CHECK-NEXT:    addl $172251900, %eax # imm = 0xA445AFC
 ; CHECK-NEXT:    movl $1374389535, %edx # imm = 0x51EB851F
 ; CHECK-NEXT:    imull %edx
 ; CHECK-NEXT:    movl %edx, %eax

llvm/trunk/test/CodeGen/X86/psubus.ll

 ; SSSE3-NEXT:    packuswb %xmm2, %xmm1
 ; SSSE3-NEXT:    packuswb %xmm3, %xmm1
 ; SSSE3-NEXT:    andnpd %xmm1, %xmm0
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: test14:
 ; SSE41:       # %bb.0: # %vector.ph
 ; SSE41-NEXT:    movdqa %xmm0, %xmm5
-; SSE41-NEXT:    pshufd {{.*#+}} xmm0 = xmm5[1,1,2,3]
+; SSE41-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[1,1,2,3]
 ; SSE41-NEXT:    pmovzxbd {{.*#+}} xmm8 = xmm0[0],zero,zero,zero,xmm0[1],zero,zero,zero,xmm0[2],zero,zero,zero,xmm0[3],zero,zero,zero
 ; SSE41-NEXT:    pmovzxbd {{.*#+}} xmm0 = xmm5[0],zero,zero,zero,xmm5[1],zero,zero,zero,xmm5[2],zero,zero,zero,xmm5[3],zero,zero,zero
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm6 = xmm5[2,3,0,1]
 ; SSE41-NEXT:    pmovzxbd {{.*#+}} xmm9 = xmm6[0],zero,zero,zero,xmm6[1],zero,zero,zero,xmm6[2],zero,zero,zero,xmm6[3],zero,zero,zero
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm5 = xmm5[3,1,2,3]
 ; SSE41-NEXT:    pmovzxbd {{.*#+}} xmm6 = xmm5[0],zero,zero,zero,xmm5[1],zero,zero,zero,xmm5[2],zero,zero,zero,xmm5[3],zero,zero,zero
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm5 = [2147483648,2147483648,2147483648,2147483648]
 ; SSE41-NEXT:    movdqa %xmm4, %xmm7

 define <32 x i16> @psubus_32i16_max(<32 x i16> %x, <32 x i16> %y) nounwind {
 ; SSE2-LABEL: psubus_32i16_max:
 ; SSE2:       # %bb.0: # %vector.ph
 ; SSE2-NEXT:    movdqa %xmm3, %xmm11
 ; SSE2-NEXT:    movdqa %xmm2, %xmm10
 ; SSE2-NEXT:    movdqa %xmm1, %xmm9
 ; SSE2-NEXT:    movdqa %xmm0, %xmm8
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm3 = [32768,32768,32768,32768,32768,32768,32768,32768]
-; SSE2-NEXT:    movdqa %xmm8, %xmm1
+; SSE2-NEXT:    movdqa %xmm0, %xmm1
 ; SSE2-NEXT:    pxor %xmm3, %xmm1
 ; SSE2-NEXT:    movdqa %xmm4, %xmm0
 ; SSE2-NEXT:    pxor %xmm3, %xmm0
 ; SSE2-NEXT:    pcmpgtw %xmm1, %xmm0
 ; SSE2-NEXT:    movdqa %xmm9, %xmm2
 ; SSE2-NEXT:    pxor %xmm3, %xmm2
 ; SSE2-NEXT:    movdqa %xmm5, %xmm1
 ; SSE2-NEXT:    pxor %xmm3, %xmm1

 ;
 ; SSSE3-LABEL: psubus_32i16_max:
 ; SSSE3:       # %bb.0: # %vector.ph
 ; SSSE3-NEXT:    movdqa %xmm3, %xmm11
 ; SSSE3-NEXT:    movdqa %xmm2, %xmm10
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm9
 ; SSSE3-NEXT:    movdqa %xmm0, %xmm8
 ; SSSE3-NEXT:    movdqa {{.*#+}} xmm3 = [32768,32768,32768,32768,32768,32768,32768,32768]
-; SSSE3-NEXT:    movdqa %xmm8, %xmm1
+; SSSE3-NEXT:    movdqa %xmm0, %xmm1
 ; SSSE3-NEXT:    pxor %xmm3, %xmm1
 ; SSSE3-NEXT:    movdqa %xmm4, %xmm0
 ; SSSE3-NEXT:    pxor %xmm3, %xmm0
 ; SSSE3-NEXT:    pcmpgtw %xmm1, %xmm0
 ; SSSE3-NEXT:    movdqa %xmm9, %xmm2
 ; SSSE3-NEXT:    pxor %xmm3, %xmm2
 ; SSSE3-NEXT:    movdqa %xmm5, %xmm1
 ; SSSE3-NEXT:    pxor %xmm3, %xmm1

 ; SSE2-LABEL: psubus_16i32_max:
 ; SSE2:       # %bb.0: # %vector.ph
 ; SSE2-NEXT:    movdqa %xmm1, %xmm8
 ; SSE2-NEXT:    movdqa %xmm0, %xmm9
 ; SSE2-NEXT:    pxor %xmm0, %xmm0
 ; SSE2-NEXT:    movdqa %xmm9, %xmm11
 ; SSE2-NEXT:    punpckhwd {{.*#+}} xmm11 = xmm11[4],xmm0[4],xmm11[5],xmm0[5],xmm11[6],xmm0[6],xmm11[7],xmm0[7]
 ; SSE2-NEXT:    punpcklwd {{.*#+}} xmm9 = xmm9[0],xmm0[0],xmm9[1],xmm0[1],xmm9[2],xmm0[2],xmm9[3],xmm0[3]
-; SSE2-NEXT:    movdqa %xmm8, %xmm10
+; SSE2-NEXT:    movdqa %xmm1, %xmm10
 ; SSE2-NEXT:    punpckhwd {{.*#+}} xmm10 = xmm10[4],xmm0[4],xmm10[5],xmm0[5],xmm10[6],xmm0[6],xmm10[7],xmm0[7]
 ; SSE2-NEXT:    punpcklwd {{.*#+}} xmm8 = xmm8[0],xmm0[0],xmm8[1],xmm0[1],xmm8[2],xmm0[2],xmm8[3],xmm0[3]
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm6 = [2147483648,2147483648,2147483648,2147483648]
 ; SSE2-NEXT:    movdqa %xmm4, %xmm1
 ; SSE2-NEXT:    pxor %xmm6, %xmm1
 ; SSE2-NEXT:    movdqa %xmm8, %xmm0
 ; SSE2-NEXT:    por %xmm6, %xmm0
 ; SSE2-NEXT:    pcmpgtd %xmm0, %xmm1

 ; SSSE3-LABEL: psubus_16i32_max:
 ; SSSE3:       # %bb.0: # %vector.ph
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm8
 ; SSSE3-NEXT:    movdqa %xmm0, %xmm9
 ; SSSE3-NEXT:    pxor %xmm0, %xmm0
 ; SSSE3-NEXT:    movdqa %xmm9, %xmm11
 ; SSSE3-NEXT:    punpckhwd {{.*#+}} xmm11 = xmm11[4],xmm0[4],xmm11[5],xmm0[5],xmm11[6],xmm0[6],xmm11[7],xmm0[7]
 ; SSSE3-NEXT:    punpcklwd {{.*#+}} xmm9 = xmm9[0],xmm0[0],xmm9[1],xmm0[1],xmm9[2],xmm0[2],xmm9[3],xmm0[3]
-; SSSE3-NEXT:    movdqa %xmm8, %xmm10
+; SSSE3-NEXT:    movdqa %xmm1, %xmm10
 ; SSSE3-NEXT:    punpckhwd {{.*#+}} xmm10 = xmm10[4],xmm0[4],xmm10[5],xmm0[5],xmm10[6],xmm0[6],xmm10[7],xmm0[7]
 ; SSSE3-NEXT:    punpcklwd {{.*#+}} xmm8 = xmm8[0],xmm0[0],xmm8[1],xmm0[1],xmm8[2],xmm0[2],xmm8[3],xmm0[3]
 ; SSSE3-NEXT:    movdqa {{.*#+}} xmm6 = [2147483648,2147483648,2147483648,2147483648]
 ; SSSE3-NEXT:    movdqa %xmm4, %xmm1
 ; SSSE3-NEXT:    pxor %xmm6, %xmm1
 ; SSSE3-NEXT:    movdqa %xmm8, %xmm0
 ; SSSE3-NEXT:    por %xmm6, %xmm0
 ; SSSE3-NEXT:    pcmpgtd %xmm0, %xmm1

llvm/trunk/test/CodeGen/X86/retpoline-external.ll

entry:
   tail call void @bar(i32 %x)
   tail call void %fp(i32 %x)
   ret void
 }
 
 ; X64-LABEL: icall_reg:
 ; X64-DAG:   movq %rdi, %[[fp:[^ ]*]]
 ; X64-DAG:   movl %esi, %[[x:[^ ]*]]
-; X64:       movl %[[x]], %edi
+; X64:       movl %esi, %edi
 ; X64:       callq bar
 ; X64-DAG:   movl %[[x]], %edi
 ; X64-DAG:   movq %[[fp]], %r11
 ; X64:       callq __llvm_external_retpoline_r11
 ; X64:       movl %[[x]], %edi
 ; X64:       callq bar
 ; X64-DAG:   movl %[[x]], %edi
 ; X64-DAG:   movq %[[fp]], %r11
define void @vcall(%struct.Foo* %obj) #0 {
   %fp = load void(%struct.Foo*)*, void(%struct.Foo*)** %vslot
   tail call void %fp(%struct.Foo* %obj)
   tail call void %fp(%struct.Foo* %obj)
   ret void
 }
 
 ; X64-LABEL: vcall:
 ; X64:       movq %rdi, %[[obj:[^ ]*]]
-; X64:       movq (%[[obj]]), %[[vptr:[^ ]*]]
+; X64:       movq (%rdi), %[[vptr:[^ ]*]]
 ; X64:       movq 8(%[[vptr]]), %[[fp:[^ ]*]]
 ; X64:       movq %[[fp]], %r11
 ; X64:       callq __llvm_external_retpoline_r11
 ; X64-DAG:   movq %[[obj]], %rdi
 ; X64-DAG:   movq %[[fp]], %r11
 ; X64:       jmp __llvm_external_retpoline_r11 # TAILCALL
 
 ; X64FAST-LABEL: vcall:

llvm/trunk/test/CodeGen/X86/retpoline.ll

entry:
   tail call void @bar(i32 %x)
   tail call void %fp(i32 %x)
   ret void
 }
 
 ; X64-LABEL: icall_reg:
 ; X64-DAG:   movq %rdi, %[[fp:[^ ]*]]
 ; X64-DAG:   movl %esi, %[[x:[^ ]*]]
-; X64:       movl %[[x]], %edi
+; X64:       movl %esi, %edi
 ; X64:       callq bar
 ; X64-DAG:   movl %[[x]], %edi
 ; X64-DAG:   movq %[[fp]], %r11
 ; X64:       callq __llvm_retpoline_r11
 ; X64:       movl %[[x]], %edi
 ; X64:       callq bar
 ; X64-DAG:   movl %[[x]], %edi
 ; X64-DAG:   movq %[[fp]], %r11
define void @vcall(%struct.Foo* %obj) #0 {
   %fp = load void(%struct.Foo*)*, void(%struct.Foo*)** %vslot
   tail call void %fp(%struct.Foo* %obj)
   tail call void %fp(%struct.Foo* %obj)
   ret void
 }
 
 ; X64-LABEL: vcall:
 ; X64:       movq %rdi, %[[obj:[^ ]*]]
-; X64:       movq (%[[obj]]), %[[vptr:[^ ]*]]
+; X64:       movq (%rdi), %[[vptr:[^ ]*]]
 ; X64:       movq 8(%[[vptr]]), %[[fp:[^ ]*]]
 ; X64:       movq %[[fp]], %r11
 ; X64:       callq __llvm_retpoline_r11
 ; X64-DAG:   movq %[[obj]], %rdi
 ; X64-DAG:   movq %[[fp]], %r11
 ; X64:       jmp __llvm_retpoline_r11 # TAILCALL
 
 ; X64FAST-LABEL: vcall:

llvm/trunk/test/CodeGen/X86/sad.ll

 ; SSE2-NEXT:    movdqa %xmm7, %xmm0
 ; SSE2-NEXT:    psrad $31, %xmm0
 ; SSE2-NEXT:    paddd %xmm0, %xmm7
 ; SSE2-NEXT:    pxor %xmm0, %xmm7
 ; SSE2-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm0 # 16-byte Reload
 ; SSE2-NEXT:    paddd %xmm7, %xmm0
 ; SSE2-NEXT:    movdqa %xmm0, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm13, %xmm1
-; SSE2-NEXT:    movdqa %xmm1, %xmm0
+; SSE2-NEXT:    movdqa %xmm13, %xmm0
 ; SSE2-NEXT:    psrad $31, %xmm0
 ; SSE2-NEXT:    paddd %xmm0, %xmm1
 ; SSE2-NEXT:    pxor %xmm0, %xmm1
 ; SSE2-NEXT:    movdqa {{[0-9]+}}(%rsp), %xmm0 # 16-byte Reload
 ; SSE2-NEXT:    paddd %xmm1, %xmm0
 ; SSE2-NEXT:    movdqa %xmm0, {{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    addq $4, %rax
 ; SSE2-NEXT:    jne .LBB2_1

llvm/trunk/test/CodeGen/X86/safestack.ll

 ; ANDROID-X64: movq %[[B]], %fs:72
 
 ; FUCHSIA-X64: movq %fs:24, %[[A:.*]]
 ; FUCHSIA-X64: leaq -16(%[[A]]), %[[B:.*]]
 ; FUCHSIA-X64: movq %[[B]], %fs:24
 
 ; LINUX-I386-PA: calll __safestack_pointer_address
 ; LINUX-I386-PA: movl %eax, %[[A:.*]]
-; LINUX-I386-PA: movl (%[[A]]), %[[B:.*]]
+; LINUX-I386-PA: movl (%eax), %[[B:.*]]
 ; LINUX-I386-PA: leal -16(%[[B]]), %[[C:.*]]
 ; LINUX-I386-PA: movl %[[C]], (%[[A]])

llvm/trunk/test/CodeGen/X86/safestack_inline.ll

 
 ; INLINE: movq p@GOTTPOFF(%rip), %[[A:.*]]
 ; INLINE: movq %fs:(%[[A]]), %[[B:.*]]
 ; INLINE: leaq -16(%[[B]]), %[[C:.*]]
 ; INLINE: movq %[[C]], %fs:(%[[A]])
 
 ; CALL: callq __safestack_pointer_address
 ; CALL: movq %rax, %[[A:.*]]
-; CALL: movq (%[[A]]), %[[B:.*]]
+; CALL: movq (%rax), %[[B:.*]]
 ; CALL: leaq -16(%[[B]]), %[[C:.*]]
 ; CALL: movq %[[C]], (%[[A]])

llvm/trunk/test/CodeGen/X86/scalar_widen_div.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-- -mattr=+sse4.2 |  FileCheck %s
 
 ; Verify when widening a divide/remainder operation, we only generate a
 ; divide/rem per element since divide/remainder can trap.
 
 ; CHECK: vectorDiv
 define void @vectorDiv (<2 x i32> addrspace(1)* %nsource, <2 x i32> addrspace(1)* %dsource, <2 x i32> addrspace(1)* %qdest) nounwind {
 ; CHECK-LABEL: vectorDiv:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movq %rdx, %r8
 ; CHECK-NEXT:    movq %rdi, -{{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    movq %rsi, -{{[0-9]+}}(%rsp)
-; CHECK-NEXT:    movq %r8, -{{[0-9]+}}(%rsp)
+; CHECK-NEXT:    movq %rdx, -{{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    movslq -{{[0-9]+}}(%rsp), %rcx
 ; CHECK-NEXT:    pmovsxdq (%rdi,%rcx,8), %xmm0
 ; CHECK-NEXT:    pmovsxdq (%rsi,%rcx,8), %xmm1
 ; CHECK-NEXT:    pextrq $1, %xmm0, %rax
 ; CHECK-NEXT:    pextrq $1, %xmm1, %rsi
 ; CHECK-NEXT:    cqto
 ; CHECK-NEXT:    idivq %rsi
 ; CHECK-NEXT:    movq %rax, %xmm2
; CHECK-NEXT:    retq
   ret <5 x i64>  %rem.r
 }
 
 ; CHECK: test_int_div
 define void @test_int_div(<3 x i32>* %dest, <3 x i32>* %old, i32 %n) {
 ; CHECK-LABEL: test_int_div:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl %edx, %r9d
-; CHECK-NEXT:    testl %r9d, %r9d
+; CHECK-NEXT:    testl %edx, %edx
 ; CHECK-NEXT:    jle .LBB12_3
 ; CHECK-NEXT:  # %bb.1: # %bb.nph
 ; CHECK-NEXT:    xorl %ecx, %ecx
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB12_2: # %for.body
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    movdqa (%rdi,%rcx), %xmm0
 ; CHECK-NEXT:    movdqa (%rsi,%rcx), %xmm1

llvm/trunk/test/CodeGen/X86/select.ll

 ; CHECK-NEXT:    retq
 ;
 ; MCU-LABEL: test1:
 ; MCU:       # %bb.0:
 ; MCU-NEXT:    testb $1, %cl
 ; MCU-NEXT:    jne .LBB0_1
 ; MCU-NEXT:  # %bb.2:
 ; MCU-NEXT:    addl $8, %edx
-; MCU-NEXT:    movl %edx, %eax
-; MCU-NEXT:    movl (%eax), %eax
+; MCU-NEXT:    movl (%edx), %eax
 ; MCU-NEXT:    retl
 ; MCU-NEXT:  .LBB0_1:
 ; MCU-NEXT:    addl $8, %eax
 ; MCU-NEXT:    movl (%eax), %eax
 ; MCU-NEXT:    retl
   %t0 = load %0, %0* %p
   %t1 = load %0, %0* %q
   %t4 = select i1 %r, %0 %t0, %0 %t1

llvm/trunk/test/CodeGen/X86/shrink-wrap-chkstk.ll

 
 false:
   %tmp.0 = phi i32 [ %tmp4, %true ], [ %a, %0 ]
   ret i32 %tmp.0
 }
 
 ; CHECK-LABEL: @use_eax_before_prologue@8: # @use_eax_before_prologue
 ; CHECK: movl %ecx, %eax
-; CHECK: cmpl %edx, %eax
+; CHECK: cmpl %edx, %ecx
 ; CHECK: jge LBB1_2
 ; CHECK: pushl %eax
 ; CHECK: movl $4092, %eax
 ; CHECK: calll __chkstk
 ; CHECK: movl 4092(%esp), %eax
 ; CHECK: calll _doSomething
 ; CHECK: LBB1_2:
 ; CHECK: retl

llvm/trunk/test/CodeGen/X86/slow-pmulld.ll

 ; SLM64-NEXT:    punpcklwd {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[1],xmm4[1],xmm2[2],xmm4[2],xmm2[3],xmm4[3]
 ; SLM64-NEXT:    retq
 ;
 ; SLOW32-LABEL: test_mul_v16i32_v16i16:
 ; SLOW32:       # %bb.0:
 ; SLOW32-NEXT:    movdqa %xmm1, %xmm3
 ; SLOW32-NEXT:    movdqa %xmm0, %xmm1
 ; SLOW32-NEXT:    movdqa {{.*#+}} xmm2 = [18778,18778,18778,18778,18778,18778,18778,18778]
-; SLOW32-NEXT:    movdqa %xmm1, %xmm4
+; SLOW32-NEXT:    movdqa %xmm0, %xmm4
 ; SLOW32-NEXT:    pmulhuw %xmm2, %xmm4
 ; SLOW32-NEXT:    pmullw %xmm2, %xmm1
 ; SLOW32-NEXT:    movdqa %xmm1, %xmm0
 ; SLOW32-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm4[0],xmm0[1],xmm4[1],xmm0[2],xmm4[2],xmm0[3],xmm4[3]
 ; SLOW32-NEXT:    punpckhwd {{.*#+}} xmm1 = xmm1[4],xmm4[4],xmm1[5],xmm4[5],xmm1[6],xmm4[6],xmm1[7],xmm4[7]
 ; SLOW32-NEXT:    movdqa %xmm3, %xmm4
 ; SLOW32-NEXT:    pmulhuw %xmm2, %xmm4
 ; SLOW32-NEXT:    pmullw %xmm2, %xmm3
 ; SLOW32-NEXT:    movdqa %xmm3, %xmm2
 ; SLOW32-NEXT:    punpcklwd {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[1],xmm4[1],xmm2[2],xmm4[2],xmm2[3],xmm4[3]
 ; SLOW32-NEXT:    punpckhwd {{.*#+}} xmm3 = xmm3[4],xmm4[4],xmm3[5],xmm4[5],xmm3[6],xmm4[6],xmm3[7],xmm4[7]
 ; SLOW32-NEXT:    retl
 ;
 ; SLOW64-LABEL: test_mul_v16i32_v16i16:
 ; SLOW64:       # %bb.0:
 ; SLOW64-NEXT:    movdqa %xmm1, %xmm3
 ; SLOW64-NEXT:    movdqa %xmm0, %xmm1
 ; SLOW64-NEXT:    movdqa {{.*#+}} xmm2 = [18778,18778,18778,18778,18778,18778,18778,18778]
-; SLOW64-NEXT:    movdqa %xmm1, %xmm4
+; SLOW64-NEXT:    movdqa %xmm0, %xmm4
 ; SLOW64-NEXT:    pmulhuw %xmm2, %xmm4
 ; SLOW64-NEXT:    pmullw %xmm2, %xmm1
 ; SLOW64-NEXT:    movdqa %xmm1, %xmm0
 ; SLOW64-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm4[0],xmm0[1],xmm4[1],xmm0[2],xmm4[2],xmm0[3],xmm4[3]
 ; SLOW64-NEXT:    punpckhwd {{.*#+}} xmm1 = xmm1[4],xmm4[4],xmm1[5],xmm4[5],xmm1[6],xmm4[6],xmm1[7],xmm4[7]
 ; SLOW64-NEXT:    movdqa %xmm3, %xmm4
 ; SLOW64-NEXT:    pmulhuw %xmm2, %xmm4
 ; SLOW64-NEXT:    pmullw %xmm2, %xmm3

llvm/trunk/test/CodeGen/X86/sqrt-fastmath.ll

; AVX-NEXT:    retq
   ret float %div
 }
 
 define float @f32_estimate(float %x) #1 {
 ; SSE-LABEL: f32_estimate:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    rsqrtss %xmm0, %xmm1
 ; SSE-NEXT:    movaps %xmm1, %xmm2
-; SSE-NEXT:    mulss %xmm2, %xmm2
+; SSE-NEXT:    mulss %xmm1, %xmm2
 ; SSE-NEXT:    mulss %xmm0, %xmm2
 ; SSE-NEXT:    addss {{.*}}(%rip), %xmm2
 ; SSE-NEXT:    mulss {{.*}}(%rip), %xmm1
 ; SSE-NEXT:    mulss %xmm2, %xmm1
 ; SSE-NEXT:    movaps %xmm1, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: f32_estimate:
; AVX-NEXT:    retq
   ret <4 x float> %div
 }
 
 define <4 x float> @v4f32_estimate(<4 x float> %x) #1 {
 ; SSE-LABEL: v4f32_estimate:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    rsqrtps %xmm0, %xmm1
 ; SSE-NEXT:    movaps %xmm1, %xmm2
-; SSE-NEXT:    mulps %xmm2, %xmm2
+; SSE-NEXT:    mulps %xmm1, %xmm2
 ; SSE-NEXT:    mulps %xmm0, %xmm2
 ; SSE-NEXT:    addps {{.*}}(%rip), %xmm2
 ; SSE-NEXT:    mulps {{.*}}(%rip), %xmm1
 ; SSE-NEXT:    mulps %xmm2, %xmm1
 ; SSE-NEXT:    movaps %xmm1, %xmm0
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: v4f32_estimate:

 }
 
 define <8 x float> @v8f32_estimate(<8 x float> %x) #1 {
 ; SSE-LABEL: v8f32_estimate:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    rsqrtps %xmm0, %xmm3
 ; SSE-NEXT:    movaps {{.*#+}} xmm4 = [-5.000000e-01,-5.000000e-01,-5.000000e-01,-5.000000e-01]
 ; SSE-NEXT:    movaps %xmm3, %xmm2
-; SSE-NEXT:    mulps %xmm2, %xmm2
+; SSE-NEXT:    mulps %xmm3, %xmm2
 ; SSE-NEXT:    mulps %xmm0, %xmm2
 ; SSE-NEXT:    movaps {{.*#+}} xmm0 = [-3.000000e+00,-3.000000e+00,-3.000000e+00,-3.000000e+00]
 ; SSE-NEXT:    addps %xmm0, %xmm2
 ; SSE-NEXT:    mulps %xmm4, %xmm2
 ; SSE-NEXT:    mulps %xmm3, %xmm2
 ; SSE-NEXT:    rsqrtps %xmm1, %xmm5
 ; SSE-NEXT:    movaps %xmm5, %xmm3
-; SSE-NEXT:    mulps %xmm3, %xmm3
+; SSE-NEXT:    mulps %xmm5, %xmm3
 ; SSE-NEXT:    mulps %xmm1, %xmm3
 ; SSE-NEXT:    addps %xmm0, %xmm3
 ; SSE-NEXT:    mulps %xmm4, %xmm3
 ; SSE-NEXT:    mulps %xmm5, %xmm3
 ; SSE-NEXT:    movaps %xmm2, %xmm0
 ; SSE-NEXT:    movaps %xmm3, %xmm1
 ; SSE-NEXT:    retq
 ;

llvm/trunk/test/CodeGen/X86/sse-scalar-fp-arith.ll

 }
 
 define <4 x float> @add_ss_mask(<4 x float> %a, <4 x float> %b, <4 x float> %c, i8 %mask) {
 ; SSE2-LABEL: add_ss_mask:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    testb $1, %dil
 ; SSE2-NEXT:    jne .LBB62_1
 ; SSE2-NEXT:  # %bb.2:
-; SSE2-NEXT:    movaps %xmm2, %xmm1
-; SSE2-NEXT:    movss {{.*#+}} xmm0 = xmm1[0],xmm0[1,2,3]
+; SSE2-NEXT:    movss {{.*#+}} xmm0 = xmm2[0],xmm0[1,2,3]
 ; SSE2-NEXT:    retq
 ; SSE2-NEXT:  .LBB62_1:
 ; SSE2-NEXT:    addss %xmm0, %xmm1
 ; SSE2-NEXT:    movss {{.*#+}} xmm0 = xmm1[0],xmm0[1,2,3]
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: add_ss_mask:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    testb $1, %dil
 ; SSE41-NEXT:    jne .LBB62_1
 ; SSE41-NEXT:  # %bb.2:
-; SSE41-NEXT:    movaps %xmm2, %xmm1
-; SSE41-NEXT:    blendps {{.*#+}} xmm0 = xmm1[0],xmm0[1,2,3]
+; SSE41-NEXT:    blendps {{.*#+}} xmm0 = xmm2[0],xmm0[1,2,3]
 ; SSE41-NEXT:    retq
 ; SSE41-NEXT:  .LBB62_1:
 ; SSE41-NEXT:    addss %xmm0, %xmm1
 ; SSE41-NEXT:    blendps {{.*#+}} xmm0 = xmm1[0],xmm0[1,2,3]
 ; SSE41-NEXT:    retq
 ;
 ; AVX1-LABEL: add_ss_mask:
 ; AVX1:       # %bb.0:

 }
 
 define <2 x double> @add_sd_mask(<2 x double> %a, <2 x double> %b, <2 x double> %c, i8 %mask) {
 ; SSE2-LABEL: add_sd_mask:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    testb $1, %dil
 ; SSE2-NEXT:    jne .LBB63_1
 ; SSE2-NEXT:  # %bb.2:
-; SSE2-NEXT:    movapd %xmm2, %xmm1
-; SSE2-NEXT:    movsd {{.*#+}} xmm0 = xmm1[0],xmm0[1]
+; SSE2-NEXT:    movsd {{.*#+}} xmm0 = xmm2[0],xmm0[1]
 ; SSE2-NEXT:    retq
 ; SSE2-NEXT:  .LBB63_1:
 ; SSE2-NEXT:    addsd %xmm0, %xmm1
 ; SSE2-NEXT:    movsd {{.*#+}} xmm0 = xmm1[0],xmm0[1]
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: add_sd_mask:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    testb $1, %dil
 ; SSE41-NEXT:    jne .LBB63_1
 ; SSE41-NEXT:  # %bb.2:
-; SSE41-NEXT:    movaps %xmm2, %xmm1
-; SSE41-NEXT:    blendps {{.*#+}} xmm0 = xmm1[0,1],xmm0[2,3]
+; SSE41-NEXT:    blendps {{.*#+}} xmm0 = xmm2[0,1],xmm0[2,3]
 ; SSE41-NEXT:    retq
 ; SSE41-NEXT:  .LBB63_1:
 ; SSE41-NEXT:    addsd %xmm0, %xmm1
 ; SSE41-NEXT:    blendpd {{.*#+}} xmm0 = xmm1[0],xmm0[1]
 ; SSE41-NEXT:    retq
 ;
 ; AVX1-LABEL: add_sd_mask:
 ; AVX1:       # %bb.0:

llvm/trunk/test/CodeGen/X86/sse1.ll

 
 ; This should not emit shuffles to populate the top 2 elements of the 4-element
 ; vector that this ends up returning.
 ; rdar://8368414
 define <2 x float> @test4(<2 x float> %A, <2 x float> %B) nounwind {
 ; X32-LABEL: test4:
 ; X32:       # %bb.0: # %entry
 ; X32-NEXT:    movaps %xmm0, %xmm2
-; X32-NEXT:    shufps {{.*#+}} xmm2 = xmm2[1,1,2,3]
+; X32-NEXT:    shufps {{.*#+}} xmm2 = xmm2[1,1],xmm0[2,3]
 ; X32-NEXT:    addss %xmm1, %xmm0
 ; X32-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,1,2,3]
 ; X32-NEXT:    subss %xmm1, %xmm2
 ; X32-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: test4:
 ; X64:       # %bb.0: # %entry
 ; X64-NEXT:    movaps %xmm0, %xmm2
-; X64-NEXT:    shufps {{.*#+}} xmm2 = xmm2[1,1,2,3]
+; X64-NEXT:    shufps {{.*#+}} xmm2 = xmm2[1,1],xmm0[2,3]
 ; X64-NEXT:    addss %xmm1, %xmm0
 ; X64-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,1,2,3]
 ; X64-NEXT:    subss %xmm1, %xmm2
 ; X64-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
 ; X64-NEXT:    retq
 entry:
   %tmp7 = extractelement <2 x float> %A, i32 0
   %tmp5 = extractelement <2 x float> %A, i32 1

llvm/trunk/test/CodeGen/X86/sse3-avx-addsub-2.ll

 }
 
 define <4 x float> @test16(<4 x float> %A, <4 x float> %B) {
 ; SSE-LABEL: test16:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movaps %xmm0, %xmm2
 ; SSE-NEXT:    subss %xmm0, %xmm2
 ; SSE-NEXT:    movaps %xmm0, %xmm3
-; SSE-NEXT:    movhlps {{.*#+}} xmm3 = xmm3[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm3 = xmm0[1],xmm3[1]
 ; SSE-NEXT:    movaps %xmm1, %xmm4
-; SSE-NEXT:    movhlps {{.*#+}} xmm4 = xmm4[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm4 = xmm1[1],xmm4[1]
 ; SSE-NEXT:    subss %xmm4, %xmm3
 ; SSE-NEXT:    movshdup {{.*#+}} xmm4 = xmm0[1,1,3,3]
 ; SSE-NEXT:    addss %xmm0, %xmm4
 ; SSE-NEXT:    shufps {{.*#+}} xmm0 = xmm0[3,1,2,3]
 ; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[3,1,2,3]
 ; SSE-NEXT:    addss %xmm0, %xmm1
 ; SSE-NEXT:    unpcklps {{.*#+}} xmm3 = xmm3[0],xmm1[0],xmm3[1],xmm1[1]
 ; SSE-NEXT:    unpcklps {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[1],xmm4[1]

llvm/trunk/test/CodeGen/X86/statepoint-live-in.ll

 ; CHECK-LABEL: test6:
 ; CHECK:       ## %bb.0: ## %entry
 ; CHECK-NEXT:    pushq %rbx
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    subq $16, %rsp
 ; CHECK-NEXT:    .cfi_def_cfa_offset 32
 ; CHECK-NEXT:    .cfi_offset %rbx, -16
 ; CHECK-NEXT:    movl %edi, %ebx
-; CHECK-NEXT:    movl %ebx, {{[0-9]+}}(%rsp)
+; CHECK-NEXT:    movl %edi, {{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    callq _baz
 ; CHECK-NEXT:  Ltmp6:
 ; CHECK-NEXT:    callq _bar
 ; CHECK-NEXT:  Ltmp7:
 ; CHECK-NEXT:    addq $16, %rsp
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    retq
 entry:

llvm/trunk/test/CodeGen/X86/statepoint-stack-usage.ll

 
 ; Check that we reuse the same stack slot across multiple calls.  The use of
 ; more than two calls here is critical.  We've had a bug which allowed reuse
 ; exactly once which went undetected for a long time.
 define i32 @back_to_back_deopt(i32 %a, i32 %b, i32 %c) #1 
   gc "statepoint-example" {
 ; CHECK-LABEL: back_to_back_deopt
 ; The exact stores don't matter, but there need to be three stack slots created
-; CHECK-DAG: movl %ebx, 12(%rsp)
-; CHECK-DAG: movl %ebp, 8(%rsp)
-; CHECK-DAG: movl %r14d, 4(%rsp)
+; CHECK-DAG: movl %edi, 12(%rsp)
+; CHECK-DAG: movl %esi, 8(%rsp)
+; CHECK-DAG: movl %edx, 4(%rsp)
 ; CHECK: callq
 ; CHECK-DAG: movl %ebx, 12(%rsp)
 ; CHECK-DAG: movl %ebp, 8(%rsp)
 ; CHECK-DAG: movl %r14d, 4(%rsp)
 ; CHECK: callq
 ; CHECK-DAG: movl %ebx, 12(%rsp)
 ; CHECK-DAG: movl %ebp, 8(%rsp)
 ; CHECK-DAG: movl %r14d, 4(%rsp)

llvm/trunk/test/CodeGen/X86/vec_fp_to_int.ll

 }
 
 define <4 x i64> @fptosi_4f32_to_4i64(<8 x float> %a) {
 ; SSE-LABEL: fptosi_4f32_to_4i64:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    cvttss2si %xmm0, %rax
 ; SSE-NEXT:    movq %rax, %xmm2
 ; SSE-NEXT:    movaps %xmm0, %xmm1
-; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,1,2,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,1],xmm0[2,3]
 ; SSE-NEXT:    cvttss2si %xmm1, %rax
 ; SSE-NEXT:    movq %rax, %xmm1
 ; SSE-NEXT:    punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm1[0]
 ; SSE-NEXT:    movaps %xmm0, %xmm1
-; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[3,1,2,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[3,1],xmm0[2,3]
 ; SSE-NEXT:    cvttss2si %xmm1, %rax
 ; SSE-NEXT:    movq %rax, %xmm3
 ; SSE-NEXT:    movhlps {{.*#+}} xmm0 = xmm0[1,1]
 ; SSE-NEXT:    cvttss2si %xmm0, %rax
 ; SSE-NEXT:    movq %rax, %xmm1
 ; SSE-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm3[0]
 ; SSE-NEXT:    movdqa %xmm2, %xmm0
 ; SSE-NEXT:    retq

 }
 
 define <4 x i64> @fptosi_8f32_to_4i64(<8 x float> %a) {
 ; SSE-LABEL: fptosi_8f32_to_4i64:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    cvttss2si %xmm0, %rax
 ; SSE-NEXT:    movq %rax, %xmm2
 ; SSE-NEXT:    movaps %xmm0, %xmm1
-; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,1,2,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,1],xmm0[2,3]
 ; SSE-NEXT:    cvttss2si %xmm1, %rax
 ; SSE-NEXT:    movq %rax, %xmm1
 ; SSE-NEXT:    punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm1[0]
 ; SSE-NEXT:    movaps %xmm0, %xmm1
-; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[3,1,2,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[3,1],xmm0[2,3]
 ; SSE-NEXT:    cvttss2si %xmm1, %rax
 ; SSE-NEXT:    movq %rax, %xmm3
 ; SSE-NEXT:    movhlps {{.*#+}} xmm0 = xmm0[1,1]
 ; SSE-NEXT:    cvttss2si %xmm0, %rax
 ; SSE-NEXT:    movq %rax, %xmm1
 ; SSE-NEXT:    punpcklqdq {{.*#+}} xmm1 = xmm1[0],xmm3[0]
 ; SSE-NEXT:    movdqa %xmm2, %xmm0
 ; SSE-NEXT:    retq
; AVX512VLDQ-NEXT:    retq
   %cvt = fptoui <2 x float> %a to <2 x i32>
   ret <2 x i32> %cvt
 }
 
 define <4 x i32> @fptoui_4f32_to_4i32(<4 x float> %a) {
 ; SSE-LABEL: fptoui_4f32_to_4i32:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movaps %xmm0, %xmm1
-; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[3,1,2,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[3,1],xmm0[2,3]
 ; SSE-NEXT:    cvttss2si %xmm1, %rax
 ; SSE-NEXT:    movd %eax, %xmm1
 ; SSE-NEXT:    movaps %xmm0, %xmm2
-; SSE-NEXT:    movhlps {{.*#+}} xmm2 = xmm2[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm2 = xmm0[1],xmm2[1]
 ; SSE-NEXT:    cvttss2si %xmm2, %rax
 ; SSE-NEXT:    movd %eax, %xmm2
 ; SSE-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1]
 ; SSE-NEXT:    cvttss2si %xmm0, %rax
 ; SSE-NEXT:    movd %eax, %xmm1
 ; SSE-NEXT:    shufps {{.*#+}} xmm0 = xmm0[1,1,2,3]
 ; SSE-NEXT:    cvttss2si %xmm0, %rax
 ; SSE-NEXT:    movd %eax, %xmm0

 define <8 x i32> @fptoui_8f32_to_8i32(<8 x float> %a) {
 ; SSE-LABEL: fptoui_8f32_to_8i32:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movaps %xmm0, %xmm2
 ; SSE-NEXT:    shufps {{.*#+}} xmm0 = xmm0[3,1,2,3]
 ; SSE-NEXT:    cvttss2si %xmm0, %rax
 ; SSE-NEXT:    movd %eax, %xmm0
 ; SSE-NEXT:    movaps %xmm2, %xmm3
-; SSE-NEXT:    movhlps {{.*#+}} xmm3 = xmm3[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm3 = xmm2[1],xmm3[1]
 ; SSE-NEXT:    cvttss2si %xmm3, %rax
 ; SSE-NEXT:    movd %eax, %xmm3
 ; SSE-NEXT:    punpckldq {{.*#+}} xmm3 = xmm3[0],xmm0[0],xmm3[1],xmm0[1]
 ; SSE-NEXT:    cvttss2si %xmm2, %rax
 ; SSE-NEXT:    movd %eax, %xmm0
 ; SSE-NEXT:    shufps {{.*#+}} xmm2 = xmm2[1,1,2,3]
 ; SSE-NEXT:    cvttss2si %xmm2, %rax
 ; SSE-NEXT:    movd %eax, %xmm2
 ; SSE-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
 ; SSE-NEXT:    punpcklqdq {{.*#+}} xmm0 = xmm0[0],xmm3[0]
 ; SSE-NEXT:    movaps %xmm1, %xmm2
-; SSE-NEXT:    shufps {{.*#+}} xmm2 = xmm2[3,1,2,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm2 = xmm2[3,1],xmm1[2,3]
 ; SSE-NEXT:    cvttss2si %xmm2, %rax
 ; SSE-NEXT:    movd %eax, %xmm2
 ; SSE-NEXT:    movaps %xmm1, %xmm3
-; SSE-NEXT:    movhlps {{.*#+}} xmm3 = xmm3[1,1]
+; SSE-NEXT:    movhlps {{.*#+}} xmm3 = xmm1[1],xmm3[1]
 ; SSE-NEXT:    cvttss2si %xmm3, %rax
 ; SSE-NEXT:    movd %eax, %xmm3
 ; SSE-NEXT:    punpckldq {{.*#+}} xmm3 = xmm3[0],xmm2[0],xmm3[1],xmm2[1]
 ; SSE-NEXT:    cvttss2si %xmm1, %rax
 ; SSE-NEXT:    movd %eax, %xmm2
 ; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,1,2,3]
 ; SSE-NEXT:    cvttss2si %xmm1, %rax
 ; SSE-NEXT:    movd %eax, %xmm1

 ; SSE-NEXT:    cvttss2si %xmm2, %rcx
 ; SSE-NEXT:    movabsq $-9223372036854775808, %rax # imm = 0x8000000000000000
 ; SSE-NEXT:    xorq %rax, %rcx
 ; SSE-NEXT:    cvttss2si %xmm0, %rdx
 ; SSE-NEXT:    ucomiss %xmm1, %xmm0
 ; SSE-NEXT:    cmovaeq %rcx, %rdx
 ; SSE-NEXT:    movq %rdx, %xmm2
 ; SSE-NEXT:    movaps %xmm0, %xmm3
-; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[1,1,2,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[1,1],xmm0[2,3]
 ; SSE-NEXT:    movaps %xmm3, %xmm4
 ; SSE-NEXT:    subss %xmm1, %xmm4
 ; SSE-NEXT:    cvttss2si %xmm4, %rcx
 ; SSE-NEXT:    xorq %rax, %rcx
 ; SSE-NEXT:    cvttss2si %xmm3, %rdx
 ; SSE-NEXT:    ucomiss %xmm1, %xmm3
 ; SSE-NEXT:    cmovaeq %rcx, %rdx
 ; SSE-NEXT:    movq %rdx, %xmm3
 ; SSE-NEXT:    punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm3[0]
 ; SSE-NEXT:    movaps %xmm0, %xmm3
-; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[3,1,2,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[3,1],xmm0[2,3]
 ; SSE-NEXT:    movaps %xmm3, %xmm4
 ; SSE-NEXT:    subss %xmm1, %xmm4
 ; SSE-NEXT:    cvttss2si %xmm4, %rcx
 ; SSE-NEXT:    xorq %rax, %rcx
 ; SSE-NEXT:    cvttss2si %xmm3, %rdx
 ; SSE-NEXT:    ucomiss %xmm1, %xmm3
 ; SSE-NEXT:    cmovaeq %rcx, %rdx
 ; SSE-NEXT:    movq %rdx, %xmm3

 ; SSE-NEXT:    cvttss2si %xmm2, %rcx
 ; SSE-NEXT:    movabsq $-9223372036854775808, %rax # imm = 0x8000000000000000
 ; SSE-NEXT:    xorq %rax, %rcx
 ; SSE-NEXT:    cvttss2si %xmm0, %rdx
 ; SSE-NEXT:    ucomiss %xmm1, %xmm0
 ; SSE-NEXT:    cmovaeq %rcx, %rdx
 ; SSE-NEXT:    movq %rdx, %xmm2
 ; SSE-NEXT:    movaps %xmm0, %xmm3
-; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[1,1,2,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[1,1],xmm0[2,3]
 ; SSE-NEXT:    movaps %xmm3, %xmm4
 ; SSE-NEXT:    subss %xmm1, %xmm4
 ; SSE-NEXT:    cvttss2si %xmm4, %rcx
 ; SSE-NEXT:    xorq %rax, %rcx
 ; SSE-NEXT:    cvttss2si %xmm3, %rdx
 ; SSE-NEXT:    ucomiss %xmm1, %xmm3
 ; SSE-NEXT:    cmovaeq %rcx, %rdx
 ; SSE-NEXT:    movq %rdx, %xmm3
 ; SSE-NEXT:    punpcklqdq {{.*#+}} xmm2 = xmm2[0],xmm3[0]
 ; SSE-NEXT:    movaps %xmm0, %xmm3
-; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[3,1,2,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm3 = xmm3[3,1],xmm0[2,3]
 ; SSE-NEXT:    movaps %xmm3, %xmm4
 ; SSE-NEXT:    subss %xmm1, %xmm4
 ; SSE-NEXT:    cvttss2si %xmm4, %rcx
 ; SSE-NEXT:    xorq %rax, %rcx
 ; SSE-NEXT:    cvttss2si %xmm3, %rdx
 ; SSE-NEXT:    ucomiss %xmm1, %xmm3
 ; SSE-NEXT:    cmovaeq %rcx, %rdx
 ; SSE-NEXT:    movq %rdx, %xmm3

llvm/trunk/test/CodeGen/X86/vec_int_to_fp.ll

 ;
 ; Unsigned Integer to Float
 ;
 
 define <4 x float> @uitofp_2i64_to_4f32(<2 x i64> %a) {
 ; SSE-LABEL: uitofp_2i64_to_4f32:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movdqa %xmm0, %xmm1
-; SSE-NEXT:    movq %xmm1, %rax
+; SSE-NEXT:    movq %xmm0, %rax
 ; SSE-NEXT:    testq %rax, %rax
 ; SSE-NEXT:    js .LBB39_1
 ; SSE-NEXT:  # %bb.2:
 ; SSE-NEXT:    xorps %xmm0, %xmm0
 ; SSE-NEXT:    cvtsi2ssq %rax, %xmm0
 ; SSE-NEXT:    jmp .LBB39_3
 ; SSE-NEXT:  .LBB39_1:
 ; SSE-NEXT:    movq %rax, %rcx
; AVX512VLDQ-NEXT:    retq
   %ext = shufflevector <2 x float> %cvt, <2 x float> zeroinitializer, <4 x i32> <i32 0, i32 1, i32 2, i32 3>
   ret <4 x float> %ext
 }
 
 define <4 x float> @uitofp_4i64_to_4f32_undef(<2 x i64> %a) {
 ; SSE-LABEL: uitofp_4i64_to_4f32_undef:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movdqa %xmm0, %xmm1
-; SSE-NEXT:    movq %xmm1, %rax
+; SSE-NEXT:    movq %xmm0, %rax
 ; SSE-NEXT:    testq %rax, %rax
 ; SSE-NEXT:    js .LBB41_1
 ; SSE-NEXT:  # %bb.2:
 ; SSE-NEXT:    xorps %xmm0, %xmm0
 ; SSE-NEXT:    cvtsi2ssq %rax, %xmm0
 ; SSE-NEXT:    jmp .LBB41_3
 ; SSE-NEXT:  .LBB41_1:
 ; SSE-NEXT:    movq %rax, %rcx

llvm/trunk/test/CodeGen/X86/vec_minmax_sint.ll

 ; SSE41-NEXT:    blendvpd %xmm0, %xmm2, %xmm1
 ; SSE41-NEXT:    movapd %xmm1, %xmm0
 ; SSE41-NEXT:    retq
 ;
 ; SSE42-LABEL: max_ge_v2i64:
 ; SSE42:       # %bb.0:
 ; SSE42-NEXT:    movdqa %xmm0, %xmm2
 ; SSE42-NEXT:    movdqa %xmm1, %xmm3
-; SSE42-NEXT:    pcmpgtq %xmm2, %xmm3
+; SSE42-NEXT:    pcmpgtq %xmm0, %xmm3
 ; SSE42-NEXT:    pcmpeqd %xmm0, %xmm0
 ; SSE42-NEXT:    pxor %xmm3, %xmm0
 ; SSE42-NEXT:    blendvpd %xmm0, %xmm2, %xmm1
 ; SSE42-NEXT:    movapd %xmm1, %xmm0
 ; SSE42-NEXT:    retq
 ;
 ; AVX1-LABEL: max_ge_v2i64:
 ; AVX1:       # %bb.0:

llvm/trunk/test/CodeGen/X86/vec_shift4.ll

; CHECK: pmulld
   ret <2 x i64> %tmp2
 }
 
 define <2 x i64> @shl2(<16 x i8> %r, <16 x i8> %a) nounwind readnone ssp {
 ; X32-LABEL: shl2:
 ; X32:       # %bb.0: # %entry
 ; X32-NEXT:    movdqa %xmm0, %xmm2
 ; X32-NEXT:    psllw $5, %xmm1
-; X32-NEXT:    movdqa %xmm2, %xmm3
+; X32-NEXT:    movdqa %xmm0, %xmm3
 ; X32-NEXT:    psllw $4, %xmm3
 ; X32-NEXT:    pand {{\.LCPI.*}}, %xmm3
 ; X32-NEXT:    movdqa %xmm1, %xmm0
 ; X32-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; X32-NEXT:    movdqa %xmm2, %xmm3
 ; X32-NEXT:    psllw $2, %xmm3
 ; X32-NEXT:    pand {{\.LCPI.*}}, %xmm3
 ; X32-NEXT:    paddb %xmm1, %xmm1
 ; X32-NEXT:    movdqa %xmm1, %xmm0
 ; X32-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; X32-NEXT:    movdqa %xmm2, %xmm3
-; X32-NEXT:    paddb %xmm3, %xmm3
+; X32-NEXT:    paddb %xmm2, %xmm3
 ; X32-NEXT:    paddb %xmm1, %xmm1
 ; X32-NEXT:    movdqa %xmm1, %xmm0
 ; X32-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; X32-NEXT:    movdqa %xmm2, %xmm0
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: shl2:
 ; X64:       # %bb.0: # %entry
 ; X64-NEXT:    movdqa %xmm0, %xmm2
 ; X64-NEXT:    psllw $5, %xmm1
-; X64-NEXT:    movdqa %xmm2, %xmm3
+; X64-NEXT:    movdqa %xmm0, %xmm3
 ; X64-NEXT:    psllw $4, %xmm3
 ; X64-NEXT:    pand {{.*}}(%rip), %xmm3
 ; X64-NEXT:    movdqa %xmm1, %xmm0
 ; X64-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; X64-NEXT:    movdqa %xmm2, %xmm3
 ; X64-NEXT:    psllw $2, %xmm3
 ; X64-NEXT:    pand {{.*}}(%rip), %xmm3
 ; X64-NEXT:    paddb %xmm1, %xmm1
 ; X64-NEXT:    movdqa %xmm1, %xmm0
 ; X64-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; X64-NEXT:    movdqa %xmm2, %xmm3
-; X64-NEXT:    paddb %xmm3, %xmm3
+; X64-NEXT:    paddb %xmm2, %xmm3
 ; X64-NEXT:    paddb %xmm1, %xmm1
 ; X64-NEXT:    movdqa %xmm1, %xmm0
 ; X64-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; X64-NEXT:    movdqa %xmm2, %xmm0
 ; X64-NEXT:    retq
 entry:
 ; CHECK-NOT: shlb
 ; CHECK: pblendvb
 ; CHECK: pblendvb
 ; CHECK: pblendvb
   %shl = shl <16 x i8> %r, %a                     ; <<16 x i8>> [#uses=1]
   %tmp2 = bitcast <16 x i8> %shl to <2 x i64>     ; <<2 x i64>> [#uses=1]
   ret <2 x i64> %tmp2
 }

llvm/trunk/test/CodeGen/X86/vector-blend.ll

 ; SSSE3-NEXT:    psubd %xmm0, %xmm1
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm0
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: blend_neg_logic_v4i32_2:
 ; SSE41:       # %bb.0: # %entry
 ; SSE41-NEXT:    movdqa %xmm0, %xmm2
 ; SSE41-NEXT:    pxor %xmm3, %xmm3
-; SSE41-NEXT:    psubd %xmm2, %xmm3
+; SSE41-NEXT:    psubd %xmm0, %xmm3
 ; SSE41-NEXT:    movaps %xmm1, %xmm0
 ; SSE41-NEXT:    blendvps %xmm0, %xmm2, %xmm3
 ; SSE41-NEXT:    movaps %xmm3, %xmm0
 ; SSE41-NEXT:    retq
 ;
 ; AVX-LABEL: blend_neg_logic_v4i32_2:
 ; AVX:       # %bb.0: # %entry
 ; AVX-NEXT:    vpxor %xmm2, %xmm2, %xmm2

llvm/trunk/test/CodeGen/X86/vector-idiv-sdiv-128.ll

; AVX-NEXT:    retq
   %res = sdiv <8 x i16> %a, <i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7>
   ret <8 x i16> %res
 }
 
 define <16 x i8> @test_div7_16i8(<16 x i8> %a) nounwind {
 ; SSE2-LABEL: test_div7_16i8:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa %xmm0, %xmm2
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8],xmm0[8],xmm2[9],xmm0[9],xmm2[10],xmm0[10],xmm2[11],xmm0[11],xmm2[12],xmm0[12],xmm2[13],xmm0[13],xmm2[14],xmm0[14],xmm2[15],xmm0[15]
 ; SSE2-NEXT:    psraw $8, %xmm2
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm3 = [65427,65427,65427,65427,65427,65427,65427,65427]
 ; SSE2-NEXT:    pmullw %xmm3, %xmm2
 ; SSE2-NEXT:    psrlw $8, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm1
-; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
+; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1],xmm1[2],xmm0[2],xmm1[3],xmm0[3],xmm1[4],xmm0[4],xmm1[5],xmm0[5],xmm1[6],xmm0[6],xmm1[7],xmm0[7]
 ; SSE2-NEXT:    psraw $8, %xmm1
 ; SSE2-NEXT:    pmullw %xmm3, %xmm1
 ; SSE2-NEXT:    psrlw $8, %xmm1
 ; SSE2-NEXT:    packuswb %xmm2, %xmm1
 ; SSE2-NEXT:    paddb %xmm0, %xmm1
 ; SSE2-NEXT:    movdqa %xmm1, %xmm0
 ; SSE2-NEXT:    psrlw $2, %xmm0
 ; SSE2-NEXT:    pand {{.*}}(%rip), %xmm0
; AVX-NEXT:    retq
   %res = srem <8 x i16> %a, <i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7, i16 7>
   ret <8 x i16> %res
 }
 
 define <16 x i8> @test_rem7_16i8(<16 x i8> %a) nounwind {
 ; SSE2-LABEL: test_rem7_16i8:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa %xmm0, %xmm2
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8],xmm0[8],xmm2[9],xmm0[9],xmm2[10],xmm0[10],xmm2[11],xmm0[11],xmm2[12],xmm0[12],xmm2[13],xmm0[13],xmm2[14],xmm0[14],xmm2[15],xmm0[15]
 ; SSE2-NEXT:    psraw $8, %xmm2
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm3 = [65427,65427,65427,65427,65427,65427,65427,65427]
 ; SSE2-NEXT:    pmullw %xmm3, %xmm2
 ; SSE2-NEXT:    psrlw $8, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm1
-; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
+; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1],xmm1[2],xmm0[2],xmm1[3],xmm0[3],xmm1[4],xmm0[4],xmm1[5],xmm0[5],xmm1[6],xmm0[6],xmm1[7],xmm0[7]
 ; SSE2-NEXT:    psraw $8, %xmm1
 ; SSE2-NEXT:    pmullw %xmm3, %xmm1
 ; SSE2-NEXT:    psrlw $8, %xmm1
 ; SSE2-NEXT:    packuswb %xmm2, %xmm1
 ; SSE2-NEXT:    paddb %xmm0, %xmm1
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; SSE2-NEXT:    psrlw $2, %xmm2
 ; SSE2-NEXT:    pand {{.*}}(%rip), %xmm2
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm3 = [32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32]
 ; SSE2-NEXT:    pxor %xmm3, %xmm2
 ; SSE2-NEXT:    psubb %xmm3, %xmm2
 ; SSE2-NEXT:    psrlw $7, %xmm1
 ; SSE2-NEXT:    pand {{.*}}(%rip), %xmm1
 ; SSE2-NEXT:    paddb %xmm2, %xmm1
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8],xmm1[8],xmm2[9],xmm1[9],xmm2[10],xmm1[10],xmm2[11],xmm1[11],xmm2[12],xmm1[12],xmm2[13],xmm1[13],xmm2[14],xmm1[14],xmm2[15],xmm1[15]
 ; SSE2-NEXT:    psraw $8, %xmm2
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm3 = [7,7,7,7,7,7,7,7]
 ; SSE2-NEXT:    pmullw %xmm3, %xmm2
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm4 = [255,255,255,255,255,255,255,255]
 ; SSE2-NEXT:    pand %xmm4, %xmm2
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm1
 ; SSE2-NEXT:    pmullw %xmm3, %xmm1

llvm/trunk/test/CodeGen/X86/vector-idiv-udiv-128.ll

 ; SSE2-NEXT:    movdqa %xmm0, %xmm1
 ; SSE2-NEXT:    psubb %xmm4, %xmm1
 ; SSE2-NEXT:    psrlw $1, %xmm1
 ; SSE2-NEXT:    pand {{.*}}(%rip), %xmm1
 ; SSE2-NEXT:    paddb %xmm4, %xmm1
 ; SSE2-NEXT:    psrlw $2, %xmm1
 ; SSE2-NEXT:    pand {{.*}}(%rip), %xmm1
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
-; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8,8,9,9,10,10,11,11,12,12,13,13,14,14,15,15]
+; SSE2-NEXT:    punpckhbw {{.*#+}} xmm2 = xmm2[8],xmm1[8],xmm2[9],xmm1[9],xmm2[10],xmm1[10],xmm2[11],xmm1[11],xmm2[12],xmm1[12],xmm2[13],xmm1[13],xmm2[14],xmm1[14],xmm2[15],xmm1[15]
 ; SSE2-NEXT:    psraw $8, %xmm2
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm3 = [7,7,7,7,7,7,7,7]
 ; SSE2-NEXT:    pmullw %xmm3, %xmm2
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm4 = [255,255,255,255,255,255,255,255]
 ; SSE2-NEXT:    pand %xmm4, %xmm2
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    psraw $8, %xmm1
 ; SSE2-NEXT:    pmullw %xmm3, %xmm1

llvm/trunk/test/CodeGen/X86/vector-mul.ll

; X64-AVX2-NEXT:    retq
   %1 = mul <8 x i16> %a0, <i16 1, i16 2, i16 4, i16 8, i16 16, i16 32, i16 64, i16 128>
   ret <8 x i16> %1
 }
 
 define <16 x i8> @mul_v16i8_1_2_4_8_1_2_4_8_1_2_4_8_1_2_4_8(<16 x i8> %a0) nounwind {
 ; X86-LABEL: mul_v16i8_1_2_4_8_1_2_4_8_1_2_4_8_1_2_4_8:
 ; X86:       # %bb.0:
 ; X86-NEXT:    movdqa %xmm0, %xmm1
-; X86-NEXT:    movdqa %xmm1, %xmm2
+; X86-NEXT:    movdqa %xmm0, %xmm2
 ; X86-NEXT:    psllw $4, %xmm2
 ; X86-NEXT:    pand {{\.LCPI.*}}, %xmm2
 ; X86-NEXT:    movdqa {{.*#+}} xmm0 = [8192,24640,8192,24640,8192,24640,8192,24640]
 ; X86-NEXT:    pblendvb %xmm0, %xmm2, %xmm1
 ; X86-NEXT:    movdqa %xmm1, %xmm2
 ; X86-NEXT:    psllw $2, %xmm2
 ; X86-NEXT:    pand {{\.LCPI.*}}, %xmm2
 ; X86-NEXT:    paddb %xmm0, %xmm0
 ; X86-NEXT:    pblendvb %xmm0, %xmm2, %xmm1
 ; X86-NEXT:    movdqa %xmm1, %xmm2
-; X86-NEXT:    paddb %xmm2, %xmm2
+; X86-NEXT:    paddb %xmm1, %xmm2
 ; X86-NEXT:    paddb %xmm0, %xmm0
 ; X86-NEXT:    pblendvb %xmm0, %xmm2, %xmm1
 ; X86-NEXT:    movdqa %xmm1, %xmm0
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: mul_v16i8_1_2_4_8_1_2_4_8_1_2_4_8_1_2_4_8:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movdqa %xmm0, %xmm1
-; X64-NEXT:    movdqa %xmm1, %xmm2
+; X64-NEXT:    movdqa %xmm0, %xmm2
 ; X64-NEXT:    psllw $4, %xmm2
 ; X64-NEXT:    pand {{.*}}(%rip), %xmm2
 ; X64-NEXT:    movdqa {{.*#+}} xmm0 = [8192,24640,8192,24640,8192,24640,8192,24640]
 ; X64-NEXT:    pblendvb %xmm0, %xmm2, %xmm1
 ; X64-NEXT:    movdqa %xmm1, %xmm2
 ; X64-NEXT:    psllw $2, %xmm2
 ; X64-NEXT:    pand {{.*}}(%rip), %xmm2
 ; X64-NEXT:    paddb %xmm0, %xmm0
 ; X64-NEXT:    pblendvb %xmm0, %xmm2, %xmm1
 ; X64-NEXT:    movdqa %xmm1, %xmm2
-; X64-NEXT:    paddb %xmm2, %xmm2
+; X64-NEXT:    paddb %xmm1, %xmm2
 ; X64-NEXT:    paddb %xmm0, %xmm0
 ; X64-NEXT:    pblendvb %xmm0, %xmm2, %xmm1
 ; X64-NEXT:    movdqa %xmm1, %xmm0
 ; X64-NEXT:    retq
 ;
 ; X64-XOP-LABEL: mul_v16i8_1_2_4_8_1_2_4_8_1_2_4_8_1_2_4_8:
 ; X64-XOP:       # %bb.0:
 ; X64-XOP-NEXT:    vpshlb {{.*}}(%rip), %xmm0, %xmm0

llvm/trunk/test/CodeGen/X86/vector-rotate-128.ll

 ; SSE41-NEXT:    movdqa %xmm0, %xmm3
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm2 = [16,16,16,16,16,16,16,16]
 ; SSE41-NEXT:    psubw %xmm1, %xmm2
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    psllw $12, %xmm0
 ; SSE41-NEXT:    psllw $4, %xmm1
 ; SSE41-NEXT:    por %xmm0, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm4
-; SSE41-NEXT:    paddw %xmm4, %xmm4
+; SSE41-NEXT:    paddw %xmm1, %xmm4
 ; SSE41-NEXT:    movdqa %xmm3, %xmm6
 ; SSE41-NEXT:    psllw $8, %xmm6
 ; SSE41-NEXT:    movdqa %xmm3, %xmm5
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm6, %xmm5
 ; SSE41-NEXT:    movdqa %xmm5, %xmm1
 ; SSE41-NEXT:    psllw $4, %xmm1
 ; SSE41-NEXT:    movdqa %xmm4, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm1, %xmm5
 ; SSE41-NEXT:    movdqa %xmm5, %xmm1
 ; SSE41-NEXT:    psllw $2, %xmm1
 ; SSE41-NEXT:    paddw %xmm4, %xmm4
 ; SSE41-NEXT:    movdqa %xmm4, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm1, %xmm5
 ; SSE41-NEXT:    movdqa %xmm5, %xmm1
 ; SSE41-NEXT:    psllw $1, %xmm1
 ; SSE41-NEXT:    paddw %xmm4, %xmm4
 ; SSE41-NEXT:    movdqa %xmm4, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm1, %xmm5
 ; SSE41-NEXT:    movdqa %xmm2, %xmm0
 ; SSE41-NEXT:    psllw $12, %xmm0
 ; SSE41-NEXT:    psllw $4, %xmm2
 ; SSE41-NEXT:    por %xmm0, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm1
-; SSE41-NEXT:    paddw %xmm1, %xmm1
+; SSE41-NEXT:    paddw %xmm2, %xmm1
 ; SSE41-NEXT:    movdqa %xmm3, %xmm4
 ; SSE41-NEXT:    psrlw $8, %xmm4
 ; SSE41-NEXT:    movdqa %xmm2, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm4, %xmm3
 ; SSE41-NEXT:    movdqa %xmm3, %xmm2
 ; SSE41-NEXT:    psrlw $4, %xmm2
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm2, %xmm3

 ;
 ; SSE41-LABEL: var_rotate_v16i8:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm1, %xmm3
 ; SSE41-NEXT:    movdqa %xmm0, %xmm1
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm2 = [8,8,8,8,8,8,8,8,8,8,8,8,8,8,8,8]
 ; SSE41-NEXT:    psubb %xmm3, %xmm2
 ; SSE41-NEXT:    psllw $5, %xmm3
-; SSE41-NEXT:    movdqa %xmm1, %xmm5
+; SSE41-NEXT:    movdqa %xmm0, %xmm5
 ; SSE41-NEXT:    psllw $4, %xmm5
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm5
-; SSE41-NEXT:    movdqa %xmm1, %xmm4
+; SSE41-NEXT:    movdqa %xmm0, %xmm4
 ; SSE41-NEXT:    movdqa %xmm3, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm5, %xmm4
 ; SSE41-NEXT:    movdqa %xmm4, %xmm5
 ; SSE41-NEXT:    psllw $2, %xmm5
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm5
 ; SSE41-NEXT:    paddb %xmm3, %xmm3
 ; SSE41-NEXT:    movdqa %xmm3, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm5, %xmm4
 ; SSE41-NEXT:    movdqa %xmm4, %xmm5
-; SSE41-NEXT:    paddb %xmm5, %xmm5
+; SSE41-NEXT:    paddb %xmm4, %xmm5
 ; SSE41-NEXT:    paddb %xmm3, %xmm3
 ; SSE41-NEXT:    movdqa %xmm3, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm5, %xmm4
 ; SSE41-NEXT:    psllw $5, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm3
-; SSE41-NEXT:    paddb %xmm3, %xmm3
+; SSE41-NEXT:    paddb %xmm2, %xmm3
 ; SSE41-NEXT:    movdqa %xmm1, %xmm5
 ; SSE41-NEXT:    psrlw $4, %xmm5
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm5
 ; SSE41-NEXT:    movdqa %xmm2, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm5, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm2
 ; SSE41-NEXT:    psrlw $2, %xmm2
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm2

 ; SSE2-NEXT:    por %xmm4, %xmm0
 ; SSE2-NEXT:    por %xmm3, %xmm0
 ; SSE2-NEXT:    por %xmm1, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: constant_rotate_v16i8:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm0, %xmm1
-; SSE41-NEXT:    movdqa %xmm1, %xmm3
+; SSE41-NEXT:    movdqa %xmm0, %xmm3
 ; SSE41-NEXT:    psllw $4, %xmm3
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm3
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm0 = [8192,24640,41088,57536,57600,41152,24704,8256]
 ; SSE41-NEXT:    movdqa %xmm1, %xmm2
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm3
 ; SSE41-NEXT:    psllw $2, %xmm3
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm3
 ; SSE41-NEXT:    paddb %xmm0, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm3
-; SSE41-NEXT:    paddb %xmm3, %xmm3
+; SSE41-NEXT:    paddb %xmm2, %xmm3
 ; SSE41-NEXT:    paddb %xmm0, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; SSE41-NEXT:    movdqa %xmm1, %xmm3
 ; SSE41-NEXT:    psrlw $4, %xmm3
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm3
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm0 = [57600,41152,24704,8256,8192,24640,41088,57536]
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm3, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm3

llvm/trunk/test/CodeGen/X86/vector-sext.ll

 ; SSE2-NEXT:    punpcklwd {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1],xmm1[2],xmm0[2],xmm1[3],xmm0[3]
 ; SSE2-NEXT:    psrad $24, %xmm1
 ; SSE2-NEXT:    movdqa %xmm2, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: sext_16i8_to_8i32:
 ; SSSE3:       # %bb.0: # %entry
 ; SSSE3-NEXT:    movdqa %xmm0, %xmm1
-; SSSE3-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3],xmm0[4],xmm1[4],xmm0[5],xmm1[5],xmm0[6],xmm1[6],xmm0[7],xmm1[7]
+; SSSE3-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSSE3-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3]
 ; SSSE3-NEXT:    psrad $24, %xmm0
 ; SSSE3-NEXT:    pshufb {{.*#+}} xmm1 = xmm1[u,u,u,4,u,u,u,5,u,u,u,6,u,u,u,7]
 ; SSSE3-NEXT:    psrad $24, %xmm1
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: sext_16i8_to_8i32:
 ; SSE41:       # %bb.0: # %entry

 ; SSE2-NEXT:    punpcklwd {{.*#+}} xmm3 = xmm3[0],xmm0[0],xmm3[1],xmm0[1],xmm3[2],xmm0[2],xmm3[3],xmm0[3]
 ; SSE2-NEXT:    psrad $24, %xmm3
 ; SSE2-NEXT:    movdqa %xmm4, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: sext_16i8_to_16i32:
 ; SSSE3:       # %bb.0: # %entry
 ; SSSE3-NEXT:    movdqa %xmm0, %xmm3
-; SSSE3-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[1],xmm3[1],xmm0[2],xmm3[2],xmm0[3],xmm3[3],xmm0[4],xmm3[4],xmm0[5],xmm3[5],xmm0[6],xmm3[6],xmm0[7],xmm3[7]
+; SSSE3-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSSE3-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3]
 ; SSSE3-NEXT:    psrad $24, %xmm0
 ; SSSE3-NEXT:    punpckhbw {{.*#+}} xmm1 = xmm1[8],xmm3[8],xmm1[9],xmm3[9],xmm1[10],xmm3[10],xmm1[11],xmm3[11],xmm1[12],xmm3[12],xmm1[13],xmm3[13],xmm1[14],xmm3[14],xmm1[15],xmm3[15]
 ; SSSE3-NEXT:    punpcklwd {{.*#+}} xmm2 = xmm2[0],xmm1[0],xmm2[1],xmm1[1],xmm2[2],xmm1[2],xmm2[3],xmm1[3]
 ; SSSE3-NEXT:    psrad $24, %xmm2
 ; SSSE3-NEXT:    movdqa %xmm3, %xmm1
 ; SSSE3-NEXT:    pshufb {{.*#+}} xmm1 = xmm1[u,u,u,4,u,u,u,5,u,u,u,6,u,u,u,7]
 ; SSSE3-NEXT:    psrad $24, %xmm1

 ; SSE2-NEXT:    psrad $24, %xmm1
 ; SSE2-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm0[0],xmm1[1],xmm0[1]
 ; SSE2-NEXT:    movdqa %xmm2, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: sext_16i8_to_4i64:
 ; SSSE3:       # %bb.0: # %entry
 ; SSSE3-NEXT:    movdqa %xmm0, %xmm1
-; SSSE3-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3],xmm0[4],xmm1[4],xmm0[5],xmm1[5],xmm0[6],xmm1[6],xmm0[7],xmm1[7]
+; SSSE3-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSSE3-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3]
 ; SSSE3-NEXT:    movdqa %xmm0, %xmm2
 ; SSSE3-NEXT:    psrad $31, %xmm2
 ; SSSE3-NEXT:    psrad $24, %xmm0
 ; SSSE3-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
 ; SSSE3-NEXT:    pshufb {{.*#+}} xmm1 = xmm1[u,u,u,2,u,u,u,3,u,u,u,u,u,u,u,u]
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm2
 ; SSSE3-NEXT:    psrad $31, %xmm2
entry:
   %C = sext <4 x i8> %B to <4 x i64>
   ret <4 x i64> %C
 }
 
 define <8 x i64> @sext_16i8_to_8i64(<16 x i8> %A) nounwind uwtable readnone ssp {
 ; SSE2-LABEL: sext_16i8_to_8i64:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm0, %xmm1
-; SSE2-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3],xmm0[4],xmm1[4],xmm0[5],xmm1[5],xmm0[6],xmm1[6],xmm0[7],xmm1[7]
+; SSE2-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; SSE2-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3]
 ; SSE2-NEXT:    movdqa %xmm0, %xmm2
 ; SSE2-NEXT:    psrad $31, %xmm2
 ; SSE2-NEXT:    psrad $24, %xmm0
 ; SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm1[1,1,2,3]
 ; SSE2-NEXT:    psrld $16, %xmm1
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]

 }
 
 define <8 x i64> @sext_8i32_to_8i64(<8 x i32> %A) nounwind uwtable readnone ssp {
 ; SSE2-LABEL: sext_8i32_to_8i64:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm3
 ; SSE2-NEXT:    psrad $31, %xmm3
-; SSE2-NEXT:    movdqa %xmm2, %xmm4
+; SSE2-NEXT:    movdqa %xmm1, %xmm4
 ; SSE2-NEXT:    psrad $31, %xmm4
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
 ; SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[1],xmm3[1]
 ; SSE2-NEXT:    movdqa %xmm1, %xmm3
 ; SSE2-NEXT:    psrad $31, %xmm3
 ; SSE2-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm3[0],xmm1[1],xmm3[1]
 ; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm2[2,3,0,1]
 ; SSE2-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[1],xmm4[1]
 ; SSE2-NEXT:    movdqa %xmm3, %xmm4
 ; SSE2-NEXT:    psrad $31, %xmm4
 ; SSE2-NEXT:    punpckldq {{.*#+}} xmm3 = xmm3[0],xmm4[0],xmm3[1],xmm4[1]
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: sext_8i32_to_8i64:
 ; SSSE3:       # %bb.0: # %entry
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm2
 ; SSSE3-NEXT:    movdqa %xmm0, %xmm3
 ; SSSE3-NEXT:    psrad $31, %xmm3
-; SSSE3-NEXT:    movdqa %xmm2, %xmm4
+; SSSE3-NEXT:    movdqa %xmm1, %xmm4
 ; SSSE3-NEXT:    psrad $31, %xmm4
 ; SSSE3-NEXT:    pshufd {{.*#+}} xmm1 = xmm0[2,3,0,1]
 ; SSSE3-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm3[0],xmm0[1],xmm3[1]
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm3
 ; SSSE3-NEXT:    psrad $31, %xmm3
 ; SSSE3-NEXT:    punpckldq {{.*#+}} xmm1 = xmm1[0],xmm3[0],xmm1[1],xmm3[1]
 ; SSSE3-NEXT:    pshufd {{.*#+}} xmm3 = xmm2[2,3,0,1]
 ; SSSE3-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[1],xmm4[1]

llvm/trunk/test/CodeGen/X86/vector-shift-ashr-128.ll

 ; SSE41-LABEL: var_shift_v8i16:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm0, %xmm2
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    psllw $12, %xmm0
 ; SSE41-NEXT:    psllw $4, %xmm1
 ; SSE41-NEXT:    por %xmm0, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm3
-; SSE41-NEXT:    paddw %xmm3, %xmm3
+; SSE41-NEXT:    paddw %xmm1, %xmm3
 ; SSE41-NEXT:    movdqa %xmm2, %xmm4
 ; SSE41-NEXT:    psraw $8, %xmm4
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm4, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm1
 ; SSE41-NEXT:    psraw $4, %xmm1
 ; SSE41-NEXT:    movdqa %xmm3, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm1, %xmm2

llvm/trunk/test/CodeGen/X86/vector-shift-lshr-128.ll

 ; SSE41-LABEL: var_shift_v8i16:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm0, %xmm2
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    psllw $12, %xmm0
 ; SSE41-NEXT:    psllw $4, %xmm1
 ; SSE41-NEXT:    por %xmm0, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm3
-; SSE41-NEXT:    paddw %xmm3, %xmm3
+; SSE41-NEXT:    paddw %xmm1, %xmm3
 ; SSE41-NEXT:    movdqa %xmm2, %xmm4
 ; SSE41-NEXT:    psrlw $8, %xmm4
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm4, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm1
 ; SSE41-NEXT:    psrlw $4, %xmm1
 ; SSE41-NEXT:    movdqa %xmm3, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm1, %xmm2

 ; SSE2-NEXT:    pand %xmm2, %xmm0
 ; SSE2-NEXT:    por %xmm1, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: var_shift_v16i8:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm0, %xmm2
 ; SSE41-NEXT:    psllw $5, %xmm1
-; SSE41-NEXT:    movdqa %xmm2, %xmm3
+; SSE41-NEXT:    movdqa %xmm0, %xmm3
 ; SSE41-NEXT:    psrlw $4, %xmm3
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm3
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm3
 ; SSE41-NEXT:    psrlw $2, %xmm3
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm3
 ; SSE41-NEXT:    paddb %xmm1, %xmm1

 ;
 ; SSE41-LABEL: splatvar_shift_v16i8:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm0, %xmm2
 ; SSE41-NEXT:    pxor %xmm0, %xmm0
 ; SSE41-NEXT:    pshufb %xmm0, %xmm1
 ; SSE41-NEXT:    psllw $5, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm3
-; SSE41-NEXT:    paddb %xmm3, %xmm3
+; SSE41-NEXT:    paddb %xmm1, %xmm3
 ; SSE41-NEXT:    movdqa %xmm2, %xmm4
 ; SSE41-NEXT:    psrlw $4, %xmm4
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm4
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm4, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm1
 ; SSE41-NEXT:    psrlw $2, %xmm1
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm1

 ; SSE2-NEXT:    pand {{.*}}(%rip), %xmm0
 ; SSE2-NEXT:    pand %xmm1, %xmm0
 ; SSE2-NEXT:    por %xmm2, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: constant_shift_v16i8:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm0, %xmm1
-; SSE41-NEXT:    movdqa %xmm1, %xmm2
+; SSE41-NEXT:    movdqa %xmm0, %xmm2
 ; SSE41-NEXT:    psrlw $4, %xmm2
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm2
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm0 = [8192,24640,41088,57536,49376,32928,16480,32]
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm2, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm2
 ; SSE41-NEXT:    psrlw $2, %xmm2
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm2
 ; SSE41-NEXT:    paddb %xmm0, %xmm0

llvm/trunk/test/CodeGen/X86/vector-shift-shl-128.ll

 ; SSE41-LABEL: var_shift_v8i16:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm0, %xmm2
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    psllw $12, %xmm0
 ; SSE41-NEXT:    psllw $4, %xmm1
 ; SSE41-NEXT:    por %xmm0, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm3
-; SSE41-NEXT:    paddw %xmm3, %xmm3
+; SSE41-NEXT:    paddw %xmm1, %xmm3
 ; SSE41-NEXT:    movdqa %xmm2, %xmm4
 ; SSE41-NEXT:    psllw $8, %xmm4
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm4, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm1
 ; SSE41-NEXT:    psllw $4, %xmm1
 ; SSE41-NEXT:    movdqa %xmm3, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm1, %xmm2

 ; SSE2-NEXT:    pand %xmm2, %xmm0
 ; SSE2-NEXT:    por %xmm1, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: var_shift_v16i8:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm0, %xmm2
 ; SSE41-NEXT:    psllw $5, %xmm1
-; SSE41-NEXT:    movdqa %xmm2, %xmm3
+; SSE41-NEXT:    movdqa %xmm0, %xmm3
 ; SSE41-NEXT:    psllw $4, %xmm3
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm3
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm3
 ; SSE41-NEXT:    psllw $2, %xmm3
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm3
 ; SSE41-NEXT:    paddb %xmm1, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm3
-; SSE41-NEXT:    paddb %xmm3, %xmm3
+; SSE41-NEXT:    paddb %xmm2, %xmm3
 ; SSE41-NEXT:    paddb %xmm1, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm3, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm0
 ; SSE41-NEXT:    retq
 ;
 ; AVX-LABEL: var_shift_v16i8:
 ; AVX:       # %bb.0:

 ;
 ; SSE41-LABEL: splatvar_shift_v16i8:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm0, %xmm2
 ; SSE41-NEXT:    pxor %xmm0, %xmm0
 ; SSE41-NEXT:    pshufb %xmm0, %xmm1
 ; SSE41-NEXT:    psllw $5, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm3
-; SSE41-NEXT:    paddb %xmm3, %xmm3
+; SSE41-NEXT:    paddb %xmm1, %xmm3
 ; SSE41-NEXT:    movdqa %xmm2, %xmm4
 ; SSE41-NEXT:    psllw $4, %xmm4
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm4
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm4, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm1
 ; SSE41-NEXT:    psllw $2, %xmm1
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm1
 ; SSE41-NEXT:    movdqa %xmm3, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm1, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm1
-; SSE41-NEXT:    paddb %xmm1, %xmm1
+; SSE41-NEXT:    paddb %xmm2, %xmm1
 ; SSE41-NEXT:    paddb %xmm3, %xmm3
 ; SSE41-NEXT:    movdqa %xmm3, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm1, %xmm2
 ; SSE41-NEXT:    movdqa %xmm2, %xmm0
 ; SSE41-NEXT:    retq
 ;
 ; AVX1-LABEL: splatvar_shift_v16i8:
 ; AVX1:       # %bb.0:

 ; SSE2-NEXT:    paddb %xmm0, %xmm0
 ; SSE2-NEXT:    pand %xmm1, %xmm0
 ; SSE2-NEXT:    por %xmm2, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSE41-LABEL: constant_shift_v16i8:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm0, %xmm1
-; SSE41-NEXT:    movdqa %xmm1, %xmm2
+; SSE41-NEXT:    movdqa %xmm0, %xmm2
 ; SSE41-NEXT:    psllw $4, %xmm2
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm2
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm0 = [8192,24640,41088,57536,49376,32928,16480,32]
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm2, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm2
 ; SSE41-NEXT:    psllw $2, %xmm2
 ; SSE41-NEXT:    pand {{.*}}(%rip), %xmm2
 ; SSE41-NEXT:    paddb %xmm0, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm2, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm2
-; SSE41-NEXT:    paddb %xmm2, %xmm2
+; SSE41-NEXT:    paddb %xmm1, %xmm2
 ; SSE41-NEXT:    paddb %xmm0, %xmm0
 ; SSE41-NEXT:    pblendvb %xmm0, %xmm2, %xmm1
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    retq
 ;
 ; AVX-LABEL: constant_shift_v16i8:
 ; AVX:       # %bb.0:
 ; AVX-NEXT:    vpsllw $4, %xmm0, %xmm1

llvm/trunk/test/CodeGen/X86/vector-shuffle-combining.ll

; AVX-NEXT:    retq
   %ret = shufflevector <4 x i32> %a0, <4 x i32> <i32 undef, i32 4, i32 5, i32 30>, <4 x i32> <i32 0, i32 5, i32 6, i32 7>
   ret <4 x i32> %ret
 }
 
 define <4 x float> @PR22377(<4 x float> %a, <4 x float> %b) {
 ; SSE-LABEL: PR22377:
 ; SSE:       # %bb.0: # %entry
 ; SSE-NEXT:    movaps %xmm0, %xmm1
-; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,3,1,3]
+; SSE-NEXT:    shufps {{.*#+}} xmm1 = xmm1[1,3],xmm0[1,3]
 ; SSE-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,2,0,2]
 ; SSE-NEXT:    addps %xmm0, %xmm1
 ; SSE-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1]
 ; SSE-NEXT:    retq
 ;
 ; AVX-LABEL: PR22377:
 ; AVX:       # %bb.0: # %entry
 ; AVX-NEXT:    vpermilps {{.*#+}} xmm1 = xmm0[1,3,1,3]

llvm/trunk/test/CodeGen/X86/vector-trunc-math.ll

 ;
 ; complex patterns - often created by vectorizer
 ;
 
 define <4 x i32> @mul_add_const_v4i64_v4i32(<4 x i32> %a0, <4 x i32> %a1) nounwind {
 ; SSE-LABEL: mul_add_const_v4i64_v4i32:
 ; SSE:       # %bb.0:
 ; SSE-NEXT:    movdqa %xmm0, %xmm2
-; SSE-NEXT:    pshufd {{.*#+}} xmm0 = xmm2[0,1,1,3]
+; SSE-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,1,1,3]
 ; SSE-NEXT:    pshufd {{.*#+}} xmm2 = xmm2[2,1,3,3]
 ; SSE-NEXT:    pshufd {{.*#+}} xmm3 = xmm1[0,1,1,3]
 ; SSE-NEXT:    pshufd {{.*#+}} xmm1 = xmm1[2,1,3,3]
 ; SSE-NEXT:    movdqa %xmm2, %xmm4
 ; SSE-NEXT:    psrlq $32, %xmm4
 ; SSE-NEXT:    pmuludq %xmm1, %xmm4
 ; SSE-NEXT:    movdqa %xmm1, %xmm5
 ; SSE-NEXT:    psrlq $32, %xmm5

llvm/trunk/test/CodeGen/X86/vector-trunc-packus.ll

 ; SSSE3-NEXT:    pand %xmm10, %xmm0
 ; SSSE3-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,2],xmm2[0,2]
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: trunc_packus_v8i64_v8i32:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm0, %xmm8
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm11 = [2147483648,0,2147483648,0]
-; SSE41-NEXT:    movdqa %xmm8, %xmm5
+; SSE41-NEXT:    movdqa %xmm0, %xmm5
 ; SSE41-NEXT:    pxor %xmm11, %xmm5
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm0 = [2147483647,2147483647]
 ; SSE41-NEXT:    movdqa %xmm0, %xmm6
 ; SSE41-NEXT:    pcmpgtd %xmm5, %xmm6
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm7 = xmm6[0,0,2,2]
 ; SSE41-NEXT:    pcmpeqd %xmm0, %xmm5
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm5 = xmm5[1,1,3,3]
 ; SSE41-NEXT:    pand %xmm7, %xmm5
; AVX512-NEXT:    retq
   ret <8 x i8> %5
 }
 
 define <16 x i8> @trunc_packus_v16i64_v16i8(<16 x i64> %a0) {
 ; SSE2-LABEL: trunc_packus_v16i64_v16i8:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm6, %xmm10
-; SSE2-NEXT:    movdqa %xmm10, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm5, %xmm11
-; SSE2-NEXT:    movdqa %xmm11, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm4, %xmm6
 ; SSE2-NEXT:    movdqa %xmm3, %xmm5
 ; SSE2-NEXT:    movdqa %xmm1, %xmm3
 ; SSE2-NEXT:    movdqa %xmm0, %xmm8
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm12 = [2147483648,0,2147483648,0]
 ; SSE2-NEXT:    movdqa %xmm7, %xmm1
 ; SSE2-NEXT:    pxor %xmm12, %xmm1
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm0 = [2147483903,2147483903]

 ; SSE2-NEXT:    packuswb %xmm15, %xmm0
 ; SSE2-NEXT:    packuswb %xmm11, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: trunc_packus_v16i64_v16i8:
 ; SSSE3:       # %bb.0:
 ; SSSE3-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa %xmm6, %xmm10
-; SSSE3-NEXT:    movdqa %xmm10, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSSE3-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa %xmm5, %xmm11
-; SSSE3-NEXT:    movdqa %xmm11, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSSE3-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa %xmm4, %xmm6
 ; SSSE3-NEXT:    movdqa %xmm3, %xmm5
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm3
 ; SSSE3-NEXT:    movdqa %xmm0, %xmm8
 ; SSSE3-NEXT:    movdqa {{.*#+}} xmm12 = [2147483648,0,2147483648,0]
 ; SSSE3-NEXT:    movdqa %xmm7, %xmm1
 ; SSSE3-NEXT:    pxor %xmm12, %xmm1
 ; SSSE3-NEXT:    movdqa {{.*#+}} xmm0 = [2147483903,2147483903]

 ; SSSE3-NEXT:    packuswb %xmm1, %xmm0
 ; SSSE3-NEXT:    packuswb %xmm15, %xmm0
 ; SSSE3-NEXT:    packuswb %xmm11, %xmm0
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: trunc_packus_v16i64_v16i8:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm7, %xmm8
-; SSE41-NEXT:    movdqa %xmm8, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE41-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movdqa %xmm6, %xmm7
 ; SSE41-NEXT:    movdqa %xmm5, %xmm6
 ; SSE41-NEXT:    movdqa %xmm4, %xmm5
 ; SSE41-NEXT:    movdqa %xmm3, %xmm4
 ; SSE41-NEXT:    movdqa %xmm0, %xmm11
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm13 = [2147483648,0,2147483648,0]
 ; SSE41-NEXT:    movdqa %xmm8, %xmm0
 ; SSE41-NEXT:    pxor %xmm13, %xmm0

llvm/trunk/test/CodeGen/X86/vector-trunc-ssat.ll

 ; SSSE3-NEXT:    por %xmm8, %xmm0
 ; SSSE3-NEXT:    shufps {{.*#+}} xmm0 = xmm0[0,2],xmm2[0,2]
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: trunc_ssat_v8i64_v8i32:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm0, %xmm8
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm4 = [2147483648,0,2147483648,0]
-; SSE41-NEXT:    movdqa %xmm8, %xmm5
+; SSE41-NEXT:    movdqa %xmm0, %xmm5
 ; SSE41-NEXT:    pxor %xmm4, %xmm5
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm10 = [4294967295,4294967295]
 ; SSE41-NEXT:    movdqa %xmm10, %xmm6
 ; SSE41-NEXT:    pcmpgtd %xmm5, %xmm6
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm7 = xmm6[0,0,2,2]
 ; SSE41-NEXT:    pcmpeqd %xmm10, %xmm5
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm5 = xmm5[1,1,3,3]
 ; SSE41-NEXT:    pand %xmm7, %xmm5

 ; SSE41-NEXT:    movdqa %xmm10, %xmm7
 ; SSE41-NEXT:    pcmpgtd %xmm5, %xmm7
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm6 = xmm7[0,0,2,2]
 ; SSE41-NEXT:    pcmpeqd %xmm10, %xmm5
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm5 = xmm5[1,1,3,3]
 ; SSE41-NEXT:    pand %xmm6, %xmm5
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm7 = xmm7[1,1,3,3]
 ; SSE41-NEXT:    por %xmm5, %xmm7
-; SSE41-NEXT:    movdqa %xmm8, %xmm5
+; SSE41-NEXT:    movdqa %xmm0, %xmm5
 ; SSE41-NEXT:    pxor %xmm4, %xmm5
 ; SSE41-NEXT:    movdqa %xmm10, %xmm6
 ; SSE41-NEXT:    pcmpgtd %xmm5, %xmm6
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm0 = xmm6[0,0,2,2]
 ; SSE41-NEXT:    pcmpeqd %xmm10, %xmm5
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm5 = xmm5[1,1,3,3]
 ; SSE41-NEXT:    pand %xmm0, %xmm5
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm0 = xmm6[1,1,3,3]

 ; SSE41-NEXT:    movdqa %xmm10, %xmm7
 ; SSE41-NEXT:    pcmpgtd %xmm5, %xmm7
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm6 = xmm7[0,0,2,2]
 ; SSE41-NEXT:    pcmpeqd %xmm10, %xmm5
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm5 = xmm5[1,1,3,3]
 ; SSE41-NEXT:    pand %xmm6, %xmm5
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm7 = xmm7[1,1,3,3]
 ; SSE41-NEXT:    por %xmm5, %xmm7
-; SSE41-NEXT:    movdqa %xmm8, %xmm5
+; SSE41-NEXT:    movdqa %xmm0, %xmm5
 ; SSE41-NEXT:    pxor %xmm4, %xmm5
 ; SSE41-NEXT:    movdqa %xmm10, %xmm6
 ; SSE41-NEXT:    pcmpgtd %xmm5, %xmm6
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm0 = xmm6[0,0,2,2]
 ; SSE41-NEXT:    pcmpeqd %xmm10, %xmm5
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm5 = xmm5[1,1,3,3]
 ; SSE41-NEXT:    pand %xmm0, %xmm5
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm0 = xmm6[1,1,3,3]

 }
 
 define <16 x i8> @trunc_ssat_v16i64_v16i8(<16 x i64> %a0) {
 ; SSE2-LABEL: trunc_ssat_v16i64_v16i8:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm5, %xmm8
-; SSE2-NEXT:    movdqa %xmm8, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm4, %xmm12
-; SSE2-NEXT:    movdqa %xmm12, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm4, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm3, %xmm5
 ; SSE2-NEXT:    movdqa %xmm2, %xmm4
 ; SSE2-NEXT:    movdqa %xmm1, %xmm3
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm11 = [2147483648,0,2147483648,0]
 ; SSE2-NEXT:    movdqa %xmm7, %xmm2
 ; SSE2-NEXT:    pxor %xmm11, %xmm2
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm7 = [2147483775,2147483775]
 ; SSE2-NEXT:    movdqa %xmm7, %xmm1

 ; SSE2-NEXT:    pand %xmm12, %xmm0
 ; SSE2-NEXT:    pandn %xmm1, %xmm12
 ; SSE2-NEXT:    por %xmm0, %xmm12
 ; SSE2-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm0 # 16-byte Reload
 ; SSE2-NEXT:    pand %xmm10, %xmm0
 ; SSE2-NEXT:    pandn %xmm1, %xmm10
 ; SSE2-NEXT:    por %xmm0, %xmm10
 ; SSE2-NEXT:    movdqa %xmm10, %xmm5
-; SSE2-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm10, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm0 # 16-byte Reload
 ; SSE2-NEXT:    pand %xmm9, %xmm0
 ; SSE2-NEXT:    pandn %xmm1, %xmm9
 ; SSE2-NEXT:    por %xmm0, %xmm9
 ; SSE2-NEXT:    movdqa %xmm9, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm0 # 16-byte Reload
 ; SSE2-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm2 # 16-byte Reload
 ; SSE2-NEXT:    pand %xmm0, %xmm2

 ; SSE2-NEXT:    packuswb %xmm4, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: trunc_ssat_v16i64_v16i8:
 ; SSSE3:       # %bb.0:
 ; SSSE3-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa %xmm5, %xmm8
-; SSSE3-NEXT:    movdqa %xmm8, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSSE3-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa %xmm4, %xmm12
-; SSSE3-NEXT:    movdqa %xmm12, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSSE3-NEXT:    movdqa %xmm4, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa %xmm3, %xmm5
 ; SSSE3-NEXT:    movdqa %xmm2, %xmm4
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm3
 ; SSSE3-NEXT:    movdqa {{.*#+}} xmm11 = [2147483648,0,2147483648,0]
 ; SSSE3-NEXT:    movdqa %xmm7, %xmm2
 ; SSSE3-NEXT:    pxor %xmm11, %xmm2
 ; SSSE3-NEXT:    movdqa {{.*#+}} xmm7 = [2147483775,2147483775]
 ; SSSE3-NEXT:    movdqa %xmm7, %xmm1

 ; SSSE3-NEXT:    pand %xmm12, %xmm0
 ; SSSE3-NEXT:    pandn %xmm1, %xmm12
 ; SSSE3-NEXT:    por %xmm0, %xmm12
 ; SSSE3-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm0 # 16-byte Reload
 ; SSSE3-NEXT:    pand %xmm10, %xmm0
 ; SSSE3-NEXT:    pandn %xmm1, %xmm10
 ; SSSE3-NEXT:    por %xmm0, %xmm10
 ; SSSE3-NEXT:    movdqa %xmm10, %xmm5
-; SSSE3-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSSE3-NEXT:    movdqa %xmm10, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm0 # 16-byte Reload
 ; SSSE3-NEXT:    pand %xmm9, %xmm0
 ; SSSE3-NEXT:    pandn %xmm1, %xmm9
 ; SSSE3-NEXT:    por %xmm0, %xmm9
 ; SSSE3-NEXT:    movdqa %xmm9, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm0 # 16-byte Reload
 ; SSSE3-NEXT:    movdqa -{{[0-9]+}}(%rsp), %xmm2 # 16-byte Reload
 ; SSSE3-NEXT:    pand %xmm0, %xmm2

 ; SSSE3-NEXT:    packuswb %xmm9, %xmm0
 ; SSSE3-NEXT:    packuswb %xmm4, %xmm0
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: trunc_ssat_v16i64_v16i8:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movdqa %xmm6, %xmm9
-; SSE41-NEXT:    movdqa %xmm9, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE41-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movaps %xmm4, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movdqa %xmm3, %xmm15
 ; SSE41-NEXT:    movdqa %xmm0, %xmm10
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm12 = [2147483648,0,2147483648,0]
 ; SSE41-NEXT:    movdqa %xmm7, %xmm0
 ; SSE41-NEXT:    pxor %xmm12, %xmm0
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm4 = [2147483775,2147483775]

 ; SSE41-NEXT:    por %xmm4, %xmm0
 ; SSE41-NEXT:    movapd %xmm13, %xmm5
 ; SSE41-NEXT:    blendvpd %xmm0, %xmm15, %xmm5
 ; SSE41-NEXT:    movapd %xmm5, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movapd %xmm13, %xmm2
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    blendvpd %xmm0, %xmm7, %xmm2
 ; SSE41-NEXT:    movapd %xmm2, %xmm4
-; SSE41-NEXT:    movapd %xmm4, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE41-NEXT:    movapd %xmm2, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movapd %xmm13, %xmm1
 ; SSE41-NEXT:    movdqa %xmm3, %xmm0
 ; SSE41-NEXT:    blendvpd %xmm0, -{{[0-9]+}}(%rsp), %xmm1 # 16-byte Folded Reload
 ; SSE41-NEXT:    movapd %xmm1, %xmm2
-; SSE41-NEXT:    movapd %xmm2, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE41-NEXT:    movapd %xmm1, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movapd %xmm13, %xmm1
 ; SSE41-NEXT:    movdqa %xmm9, %xmm0
 ; SSE41-NEXT:    blendvpd %xmm0, -{{[0-9]+}}(%rsp), %xmm1 # 16-byte Folded Reload
 ; SSE41-NEXT:    movapd %xmm1, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movdqa %xmm6, %xmm0
 ; SSE41-NEXT:    blendvpd %xmm0, -{{[0-9]+}}(%rsp), %xmm13 # 16-byte Folded Reload
 ; SSE41-NEXT:    movapd %xmm13, %xmm10
 ; SSE41-NEXT:    xorpd %xmm12, %xmm10

llvm/trunk/test/CodeGen/X86/vector-trunc-usat.ll

; AVX512BWVL-NEXT:    retq
   ret <8 x i16> %3
 }
 
 define <16 x i16> @trunc_usat_v16i32_v16i16(<16 x i32> %a0) {
 ; SSE2-LABEL: trunc_usat_v16i32_v16i16:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa %xmm1, %xmm8
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm7 = [2147483648,2147483648,2147483648,2147483648]
-; SSE2-NEXT:    movdqa %xmm8, %xmm5
+; SSE2-NEXT:    movdqa %xmm1, %xmm5
 ; SSE2-NEXT:    pxor %xmm7, %xmm5
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm1 = [2147549183,2147549183,2147549183,2147549183]
 ; SSE2-NEXT:    movdqa %xmm1, %xmm6
 ; SSE2-NEXT:    pcmpgtd %xmm5, %xmm6
 ; SSE2-NEXT:    movdqa %xmm0, %xmm5
 ; SSE2-NEXT:    pxor %xmm7, %xmm5
 ; SSE2-NEXT:    movdqa %xmm1, %xmm9
 ; SSE2-NEXT:    pcmpgtd %xmm5, %xmm9

 ; SSE2-NEXT:    psrad $16, %xmm1
 ; SSE2-NEXT:    packssdw %xmm5, %xmm1
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: trunc_usat_v16i32_v16i16:
 ; SSSE3:       # %bb.0:
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm8
 ; SSSE3-NEXT:    movdqa {{.*#+}} xmm7 = [2147483648,2147483648,2147483648,2147483648]
-; SSSE3-NEXT:    movdqa %xmm8, %xmm5
+; SSSE3-NEXT:    movdqa %xmm1, %xmm5
 ; SSSE3-NEXT:    pxor %xmm7, %xmm5
 ; SSSE3-NEXT:    movdqa {{.*#+}} xmm1 = [2147549183,2147549183,2147549183,2147549183]
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm6
 ; SSSE3-NEXT:    pcmpgtd %xmm5, %xmm6
 ; SSSE3-NEXT:    movdqa %xmm0, %xmm5
 ; SSSE3-NEXT:    pxor %xmm7, %xmm5
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm9
 ; SSSE3-NEXT:    pcmpgtd %xmm5, %xmm9

 
 define <16 x i8> @trunc_usat_v16i64_v16i8(<16 x i64> %a0) {
 ; SSE2-LABEL: trunc_usat_v16i64_v16i8:
 ; SSE2:       # %bb.0:
 ; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm4, %xmm12
-; SSE2-NEXT:    movdqa %xmm12, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm4, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm3, %xmm4
 ; SSE2-NEXT:    movdqa %xmm2, %xmm3
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm1
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm14 = [2147483648,2147483648,2147483648,2147483648]
 ; SSE2-NEXT:    movdqa %xmm7, %xmm0
 ; SSE2-NEXT:    pxor %xmm14, %xmm0
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm15 = [9223372039002259711,9223372039002259711]

 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: trunc_usat_v16i64_v16i8:
 ; SSSE3:       # %bb.0:
 ; SSSE3-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa %xmm4, %xmm12
-; SSSE3-NEXT:    movdqa %xmm12, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSSE3-NEXT:    movdqa %xmm4, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSSE3-NEXT:    movdqa %xmm3, %xmm4
 ; SSSE3-NEXT:    movdqa %xmm2, %xmm3
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm2
 ; SSSE3-NEXT:    movdqa %xmm0, %xmm1
 ; SSSE3-NEXT:    movdqa {{.*#+}} xmm14 = [2147483648,2147483648,2147483648,2147483648]
 ; SSSE3-NEXT:    movdqa %xmm7, %xmm0
 ; SSSE3-NEXT:    pxor %xmm14, %xmm0
 ; SSSE3-NEXT:    movdqa {{.*#+}} xmm15 = [9223372039002259711,9223372039002259711]

 ; SSSE3-NEXT:    packuswb %xmm10, %xmm1
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm0
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: trunc_usat_v16i64_v16i8:
 ; SSE41:       # %bb.0:
 ; SSE41-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movdqa %xmm6, %xmm11
-; SSE41-NEXT:    movdqa %xmm11, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE41-NEXT:    movdqa %xmm6, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movdqa %xmm5, %xmm12
-; SSE41-NEXT:    movdqa %xmm12, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE41-NEXT:    movdqa %xmm5, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movdqa %xmm4, %xmm13
-; SSE41-NEXT:    movdqa %xmm13, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE41-NEXT:    movdqa %xmm4, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE41-NEXT:    movdqa %xmm3, %xmm6
 ; SSE41-NEXT:    movdqa %xmm2, %xmm5
 ; SSE41-NEXT:    movdqa %xmm1, %xmm4
 ; SSE41-NEXT:    movdqa %xmm0, %xmm8
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm9 = [2147483648,2147483648,2147483648,2147483648]
 ; SSE41-NEXT:    movdqa %xmm7, %xmm10
 ; SSE41-NEXT:    pxor %xmm9, %xmm10
 ; SSE41-NEXT:    pxor %xmm9, %xmm11
 ; SSE41-NEXT:    pxor %xmm9, %xmm12
 ; SSE41-NEXT:    pxor %xmm9, %xmm13
-; SSE41-NEXT:    movdqa %xmm6, %xmm14
+; SSE41-NEXT:    movdqa %xmm3, %xmm14
 ; SSE41-NEXT:    pxor %xmm9, %xmm14
-; SSE41-NEXT:    movdqa %xmm5, %xmm3
+; SSE41-NEXT:    movdqa %xmm2, %xmm3
 ; SSE41-NEXT:    pxor %xmm9, %xmm3
-; SSE41-NEXT:    movdqa %xmm4, %xmm2
+; SSE41-NEXT:    movdqa %xmm1, %xmm2
 ; SSE41-NEXT:    pxor %xmm9, %xmm2
-; SSE41-NEXT:    pxor %xmm8, %xmm9
+; SSE41-NEXT:    pxor %xmm0, %xmm9
 ; SSE41-NEXT:    movdqa {{.*#+}} xmm1 = [9223372039002259711,9223372039002259711]
 ; SSE41-NEXT:    movdqa %xmm1, %xmm0
 ; SSE41-NEXT:    pcmpgtd %xmm9, %xmm0
 ; SSE41-NEXT:    pcmpeqd %xmm1, %xmm9
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm9 = xmm9[1,1,3,3]
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm7 = xmm0[0,0,2,2]
 ; SSE41-NEXT:    pand %xmm7, %xmm9
 ; SSE41-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[1,1,3,3]

llvm/trunk/test/CodeGen/X86/vector-zext.ll

entry:
   ret <8 x i32> %C
 }
 
 define <16 x i32> @zext_16i8_to_16i32(<16 x i8> %A) nounwind uwtable readnone ssp {
 ; SSE2-LABEL: zext_16i8_to_16i32:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm0, %xmm3
 ; SSE2-NEXT:    pxor %xmm4, %xmm4
-; SSE2-NEXT:    movdqa %xmm3, %xmm1
+; SSE2-NEXT:    movdqa %xmm0, %xmm1
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0],xmm4[0],xmm1[1],xmm4[1],xmm1[2],xmm4[2],xmm1[3],xmm4[3],xmm1[4],xmm4[4],xmm1[5],xmm4[5],xmm1[6],xmm4[6],xmm1[7],xmm4[7]
 ; SSE2-NEXT:    movdqa %xmm1, %xmm0
 ; SSE2-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm4[0],xmm0[1],xmm4[1],xmm0[2],xmm4[2],xmm0[3],xmm4[3]
 ; SSE2-NEXT:    punpckhwd {{.*#+}} xmm1 = xmm1[4],xmm4[4],xmm1[5],xmm4[5],xmm1[6],xmm4[6],xmm1[7],xmm4[7]
 ; SSE2-NEXT:    punpckhbw {{.*#+}} xmm3 = xmm3[8],xmm4[8],xmm3[9],xmm4[9],xmm3[10],xmm4[10],xmm3[11],xmm4[11],xmm3[12],xmm4[12],xmm3[13],xmm4[13],xmm3[14],xmm4[14],xmm3[15],xmm4[15]
 ; SSE2-NEXT:    movdqa %xmm3, %xmm2
 ; SSE2-NEXT:    punpcklwd {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[1],xmm4[1],xmm2[2],xmm4[2],xmm2[3],xmm4[3]
 ; SSE2-NEXT:    punpckhwd {{.*#+}} xmm3 = xmm3[4],xmm4[4],xmm3[5],xmm4[5],xmm3[6],xmm4[6],xmm3[7],xmm4[7]
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: zext_16i8_to_16i32:
 ; SSSE3:       # %bb.0: # %entry
 ; SSSE3-NEXT:    movdqa %xmm0, %xmm3
 ; SSSE3-NEXT:    pxor %xmm4, %xmm4
-; SSSE3-NEXT:    movdqa %xmm3, %xmm1
+; SSSE3-NEXT:    movdqa %xmm0, %xmm1
 ; SSSE3-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0],xmm4[0],xmm1[1],xmm4[1],xmm1[2],xmm4[2],xmm1[3],xmm4[3],xmm1[4],xmm4[4],xmm1[5],xmm4[5],xmm1[6],xmm4[6],xmm1[7],xmm4[7]
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm0
 ; SSSE3-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm4[0],xmm0[1],xmm4[1],xmm0[2],xmm4[2],xmm0[3],xmm4[3]
 ; SSSE3-NEXT:    punpckhwd {{.*#+}} xmm1 = xmm1[4],xmm4[4],xmm1[5],xmm4[5],xmm1[6],xmm4[6],xmm1[7],xmm4[7]
 ; SSSE3-NEXT:    punpckhbw {{.*#+}} xmm3 = xmm3[8],xmm4[8],xmm3[9],xmm4[9],xmm3[10],xmm4[10],xmm3[11],xmm4[11],xmm3[12],xmm4[12],xmm3[13],xmm4[13],xmm3[14],xmm4[14],xmm3[15],xmm4[15]
 ; SSSE3-NEXT:    movdqa %xmm3, %xmm2
 ; SSSE3-NEXT:    punpcklwd {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[1],xmm4[1],xmm2[2],xmm4[2],xmm2[3],xmm4[3]
 ; SSSE3-NEXT:    punpckhwd {{.*#+}} xmm3 = xmm3[4],xmm4[4],xmm3[5],xmm4[5],xmm3[6],xmm4[6],xmm3[7],xmm4[7]
entry:
   ret <4 x i64> %C
 }
 
 define <8 x i64> @zext_16i8_to_8i64(<16 x i8> %A) nounwind uwtable readnone ssp {
 ; SSE2-LABEL: zext_16i8_to_8i64:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm0, %xmm1
 ; SSE2-NEXT:    pxor %xmm4, %xmm4
-; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm1[1,1,2,3]
+; SSE2-NEXT:    pshufd {{.*#+}} xmm3 = xmm0[1,1,2,3]
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm1 = xmm1[0],xmm4[0],xmm1[1],xmm4[1],xmm1[2],xmm4[2],xmm1[3],xmm4[3],xmm1[4],xmm4[4],xmm1[5],xmm4[5],xmm1[6],xmm4[6],xmm1[7],xmm4[7]
 ; SSE2-NEXT:    punpcklwd {{.*#+}} xmm1 = xmm1[0],xmm4[0],xmm1[1],xmm4[1],xmm1[2],xmm4[2],xmm1[3],xmm4[3]
 ; SSE2-NEXT:    movdqa %xmm1, %xmm0
 ; SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm4[0],xmm0[1],xmm4[1]
 ; SSE2-NEXT:    punpckhdq {{.*#+}} xmm1 = xmm1[2],xmm4[2],xmm1[3],xmm4[3]
 ; SSE2-NEXT:    punpcklbw {{.*#+}} xmm3 = xmm3[0],xmm4[0],xmm3[1],xmm4[1],xmm3[2],xmm4[2],xmm3[3],xmm4[3],xmm3[4],xmm4[4],xmm3[5],xmm4[5],xmm3[6],xmm4[6],xmm3[7],xmm4[7]
 ; SSE2-NEXT:    punpcklwd {{.*#+}} xmm3 = xmm3[0],xmm4[0],xmm3[1],xmm4[1],xmm3[2],xmm4[2],xmm3[3],xmm4[3]
 ; SSE2-NEXT:    movdqa %xmm3, %xmm2
entry:
   ret <4 x i64> %C
 }
 
 define <8 x i64> @zext_8i16_to_8i64(<8 x i16> %A) nounwind uwtable readnone ssp {
 ; SSE2-LABEL: zext_8i16_to_8i64:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm0, %xmm3
 ; SSE2-NEXT:    pxor %xmm4, %xmm4
-; SSE2-NEXT:    movdqa %xmm3, %xmm1
+; SSE2-NEXT:    movdqa %xmm0, %xmm1
 ; SSE2-NEXT:    punpcklwd {{.*#+}} xmm1 = xmm1[0],xmm4[0],xmm1[1],xmm4[1],xmm1[2],xmm4[2],xmm1[3],xmm4[3]
 ; SSE2-NEXT:    movdqa %xmm1, %xmm0
 ; SSE2-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm4[0],xmm0[1],xmm4[1]
 ; SSE2-NEXT:    punpckhdq {{.*#+}} xmm1 = xmm1[2],xmm4[2],xmm1[3],xmm4[3]
 ; SSE2-NEXT:    punpckhwd {{.*#+}} xmm3 = xmm3[4],xmm4[4],xmm3[5],xmm4[5],xmm3[6],xmm4[6],xmm3[7],xmm4[7]
 ; SSE2-NEXT:    movdqa %xmm3, %xmm2
 ; SSE2-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[1],xmm4[1]
 ; SSE2-NEXT:    punpckhdq {{.*#+}} xmm3 = xmm3[2],xmm4[2],xmm3[3],xmm4[3]
 ; SSE2-NEXT:    retq
 ;
 ; SSSE3-LABEL: zext_8i16_to_8i64:
 ; SSSE3:       # %bb.0: # %entry
 ; SSSE3-NEXT:    movdqa %xmm0, %xmm3
 ; SSSE3-NEXT:    pxor %xmm4, %xmm4
-; SSSE3-NEXT:    movdqa %xmm3, %xmm1
+; SSSE3-NEXT:    movdqa %xmm0, %xmm1
 ; SSSE3-NEXT:    punpcklwd {{.*#+}} xmm1 = xmm1[0],xmm4[0],xmm1[1],xmm4[1],xmm1[2],xmm4[2],xmm1[3],xmm4[3]
 ; SSSE3-NEXT:    movdqa %xmm1, %xmm0
 ; SSSE3-NEXT:    punpckldq {{.*#+}} xmm0 = xmm0[0],xmm4[0],xmm0[1],xmm4[1]
 ; SSSE3-NEXT:    punpckhdq {{.*#+}} xmm1 = xmm1[2],xmm4[2],xmm1[3],xmm4[3]
 ; SSSE3-NEXT:    punpckhwd {{.*#+}} xmm3 = xmm3[4],xmm4[4],xmm3[5],xmm4[5],xmm3[6],xmm4[6],xmm3[7],xmm4[7]
 ; SSSE3-NEXT:    movdqa %xmm3, %xmm2
 ; SSSE3-NEXT:    punpckldq {{.*#+}} xmm2 = xmm2[0],xmm4[0],xmm2[1],xmm4[1]
 ; SSSE3-NEXT:    punpckhdq {{.*#+}} xmm3 = xmm3[2],xmm4[2],xmm3[3],xmm4[3]

 ; SSSE3-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1],xmm0[2],xmm2[2],xmm0[3],xmm2[3]
 ; SSSE3-NEXT:    punpckhwd {{.*#+}} xmm1 = xmm1[4],xmm2[4],xmm1[5],xmm2[5],xmm1[6],xmm2[6],xmm1[7],xmm2[7]
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: shuf_zext_8i16_to_8i32:
 ; SSE41:       # %bb.0: # %entry
 ; SSE41-NEXT:    movdqa %xmm0, %xmm1
 ; SSE41-NEXT:    pxor %xmm2, %xmm2
-; SSE41-NEXT:    pmovzxwd {{.*#+}} xmm0 = xmm1[0],zero,xmm1[1],zero,xmm1[2],zero,xmm1[3],zero
+; SSE41-NEXT:    pmovzxwd {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero
 ; SSE41-NEXT:    punpckhwd {{.*#+}} xmm1 = xmm1[4],xmm2[4],xmm1[5],xmm2[5],xmm1[6],xmm2[6],xmm1[7],xmm2[7]
 ; SSE41-NEXT:    retq
 ;
 ; AVX1-LABEL: shuf_zext_8i16_to_8i32:
 ; AVX1:       # %bb.0: # %entry
 ; AVX1-NEXT:    vpxor %xmm1, %xmm1, %xmm1
 ; AVX1-NEXT:    vpunpckhwd {{.*#+}} xmm1 = xmm0[4],xmm1[4],xmm0[5],xmm1[5],xmm0[6],xmm1[6],xmm0[7],xmm1[7]
 ; AVX1-NEXT:    vpmovzxwd {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero,xmm0[2],zero,xmm0[3],zero

 ; SSSE3-NEXT:    unpcklps {{.*#+}} xmm0 = xmm0[0],xmm2[0],xmm0[1],xmm2[1]
 ; SSSE3-NEXT:    unpckhps {{.*#+}} xmm1 = xmm1[2],xmm2[2],xmm1[3],xmm2[3]
 ; SSSE3-NEXT:    retq
 ;
 ; SSE41-LABEL: shuf_zext_4i32_to_4i64:
 ; SSE41:       # %bb.0: # %entry
 ; SSE41-NEXT:    movdqa %xmm0, %xmm1
 ; SSE41-NEXT:    pxor %xmm2, %xmm2
-; SSE41-NEXT:    pmovzxdq {{.*#+}} xmm0 = xmm1[0],zero,xmm1[1],zero
+; SSE41-NEXT:    pmovzxdq {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero
 ; SSE41-NEXT:    punpckhdq {{.*#+}} xmm1 = xmm1[2],xmm2[2],xmm1[3],xmm2[3]
 ; SSE41-NEXT:    retq
 ;
 ; AVX1-LABEL: shuf_zext_4i32_to_4i64:
 ; AVX1:       # %bb.0: # %entry
 ; AVX1-NEXT:    vpxor %xmm1, %xmm1, %xmm1
 ; AVX1-NEXT:    vpunpckhdq {{.*#+}} xmm1 = xmm0[2],xmm1[2],xmm0[3],xmm1[3]
 ; AVX1-NEXT:    vpmovzxdq {{.*#+}} xmm0 = xmm0[0],zero,xmm0[1],zero

llvm/trunk/test/CodeGen/X86/vselect-minmax.ll

entry:
   ret <64 x i8> %sel
 }
 
 define <64 x i8> @test98(<64 x i8> %a, <64 x i8> %b) {
 ; SSE2-LABEL: test98:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm3, %xmm8
 ; SSE2-NEXT:    movdqa %xmm2, %xmm9
-; SSE2-NEXT:    movdqa %xmm8, %xmm12
+; SSE2-NEXT:    movdqa %xmm3, %xmm12
 ; SSE2-NEXT:    pcmpgtb %xmm7, %xmm12
 ; SSE2-NEXT:    pcmpeqd %xmm13, %xmm13
 ; SSE2-NEXT:    movdqa %xmm12, %xmm3
 ; SSE2-NEXT:    pxor %xmm13, %xmm3
-; SSE2-NEXT:    movdqa %xmm9, %xmm14
+; SSE2-NEXT:    movdqa %xmm2, %xmm14
 ; SSE2-NEXT:    pcmpgtb %xmm6, %xmm14
 ; SSE2-NEXT:    movdqa %xmm14, %xmm2
 ; SSE2-NEXT:    pxor %xmm13, %xmm2
 ; SSE2-NEXT:    movdqa %xmm1, %xmm15
 ; SSE2-NEXT:    pcmpgtb %xmm5, %xmm15
 ; SSE2-NEXT:    movdqa %xmm15, %xmm10
 ; SSE2-NEXT:    pxor %xmm13, %xmm10
 ; SSE2-NEXT:    movdqa %xmm0, %xmm11

 
 define <64 x i8> @test100(<64 x i8> %a, <64 x i8> %b) {
 ; SSE2-LABEL: test100:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm3, %xmm8
 ; SSE2-NEXT:    movdqa %xmm2, %xmm9
 ; SSE2-NEXT:    movdqa %xmm0, %xmm10
 ; SSE2-NEXT:    movdqa %xmm7, %xmm12
-; SSE2-NEXT:    pcmpgtb %xmm8, %xmm12
+; SSE2-NEXT:    pcmpgtb %xmm3, %xmm12
 ; SSE2-NEXT:    pcmpeqd %xmm0, %xmm0
 ; SSE2-NEXT:    movdqa %xmm12, %xmm3
 ; SSE2-NEXT:    pxor %xmm0, %xmm3
 ; SSE2-NEXT:    movdqa %xmm6, %xmm13
-; SSE2-NEXT:    pcmpgtb %xmm9, %xmm13
+; SSE2-NEXT:    pcmpgtb %xmm2, %xmm13
 ; SSE2-NEXT:    movdqa %xmm13, %xmm2
 ; SSE2-NEXT:    pxor %xmm0, %xmm2
 ; SSE2-NEXT:    movdqa %xmm5, %xmm14
 ; SSE2-NEXT:    pcmpgtb %xmm1, %xmm14
 ; SSE2-NEXT:    movdqa %xmm14, %xmm11
 ; SSE2-NEXT:    pxor %xmm0, %xmm11
 ; SSE2-NEXT:    movdqa %xmm4, %xmm15
 ; SSE2-NEXT:    pcmpgtb %xmm10, %xmm15
entry:
   ret <16 x i32> %sel
 }
 
 define <16 x i32> @test114(<16 x i32> %a, <16 x i32> %b) {
 ; SSE2-LABEL: test114:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm3, %xmm8
 ; SSE2-NEXT:    movdqa %xmm2, %xmm9
-; SSE2-NEXT:    movdqa %xmm8, %xmm12
+; SSE2-NEXT:    movdqa %xmm3, %xmm12
 ; SSE2-NEXT:    pcmpgtd %xmm7, %xmm12
 ; SSE2-NEXT:    pcmpeqd %xmm13, %xmm13
 ; SSE2-NEXT:    movdqa %xmm12, %xmm3
 ; SSE2-NEXT:    pxor %xmm13, %xmm3
-; SSE2-NEXT:    movdqa %xmm9, %xmm14
+; SSE2-NEXT:    movdqa %xmm2, %xmm14
 ; SSE2-NEXT:    pcmpgtd %xmm6, %xmm14
 ; SSE2-NEXT:    movdqa %xmm14, %xmm2
 ; SSE2-NEXT:    pxor %xmm13, %xmm2
 ; SSE2-NEXT:    movdqa %xmm1, %xmm15
 ; SSE2-NEXT:    pcmpgtd %xmm5, %xmm15
 ; SSE2-NEXT:    movdqa %xmm15, %xmm10
 ; SSE2-NEXT:    pxor %xmm13, %xmm10
 ; SSE2-NEXT:    movdqa %xmm0, %xmm11

 
 define <16 x i32> @test116(<16 x i32> %a, <16 x i32> %b) {
 ; SSE2-LABEL: test116:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm3, %xmm8
 ; SSE2-NEXT:    movdqa %xmm2, %xmm9
 ; SSE2-NEXT:    movdqa %xmm0, %xmm10
 ; SSE2-NEXT:    movdqa %xmm7, %xmm12
-; SSE2-NEXT:    pcmpgtd %xmm8, %xmm12
+; SSE2-NEXT:    pcmpgtd %xmm3, %xmm12
 ; SSE2-NEXT:    pcmpeqd %xmm0, %xmm0
 ; SSE2-NEXT:    movdqa %xmm12, %xmm3
 ; SSE2-NEXT:    pxor %xmm0, %xmm3
 ; SSE2-NEXT:    movdqa %xmm6, %xmm13
-; SSE2-NEXT:    pcmpgtd %xmm9, %xmm13
+; SSE2-NEXT:    pcmpgtd %xmm2, %xmm13
 ; SSE2-NEXT:    movdqa %xmm13, %xmm2
 ; SSE2-NEXT:    pxor %xmm0, %xmm2
 ; SSE2-NEXT:    movdqa %xmm5, %xmm14
 ; SSE2-NEXT:    pcmpgtd %xmm1, %xmm14
 ; SSE2-NEXT:    movdqa %xmm14, %xmm11
 ; SSE2-NEXT:    pxor %xmm0, %xmm11
 ; SSE2-NEXT:    movdqa %xmm4, %xmm15
 ; SSE2-NEXT:    pcmpgtd %xmm10, %xmm15
entry:
   %sel = select <8 x i1> %cmp, <8 x i64> %a, <8 x i64> %b
   ret <8 x i64> %sel
 }
 
 define <8 x i64> @test122(<8 x i64> %a, <8 x i64> %b) {
 ; SSE2-LABEL: test122:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm7, %xmm8
-; SSE2-NEXT:    movdqa %xmm8, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm3, %xmm7
 ; SSE2-NEXT:    movdqa %xmm2, %xmm3
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm9
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm10 = [2147483648,0,2147483648,0]
 ; SSE2-NEXT:    pxor %xmm10, %xmm8
 ; SSE2-NEXT:    movdqa %xmm7, %xmm0
 ; SSE2-NEXT:    pxor %xmm10, %xmm0
entry:
   %sel = select <8 x i1> %cmp, <8 x i64> %a, <8 x i64> %b
   ret <8 x i64> %sel
 }
 
 define <8 x i64> @test124(<8 x i64> %a, <8 x i64> %b) {
 ; SSE2-LABEL: test124:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm7, %xmm11
-; SSE2-NEXT:    movdqa %xmm11, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm3, %xmm7
 ; SSE2-NEXT:    movdqa %xmm2, %xmm3
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm9
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm10 = [2147483648,0,2147483648,0]
 ; SSE2-NEXT:    movdqa %xmm7, %xmm8
 ; SSE2-NEXT:    pxor %xmm10, %xmm8
 ; SSE2-NEXT:    movdqa %xmm11, %xmm0
entry:
   %sel = select <8 x i1> %cmp, <8 x i64> %a, <8 x i64> %b
   ret <8 x i64> %sel
 }
 
 define <8 x i64> @test126(<8 x i64> %a, <8 x i64> %b) {
 ; SSE2-LABEL: test126:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm7, %xmm8
-; SSE2-NEXT:    movdqa %xmm8, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm3, %xmm7
 ; SSE2-NEXT:    movdqa %xmm2, %xmm3
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm9
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm10 = [2147483648,2147483648,2147483648,2147483648]
 ; SSE2-NEXT:    pxor %xmm10, %xmm8
 ; SSE2-NEXT:    movdqa %xmm7, %xmm0
 ; SSE2-NEXT:    pxor %xmm10, %xmm0
entry:
   %sel = select <8 x i1> %cmp, <8 x i64> %a, <8 x i64> %b
   ret <8 x i64> %sel
 }
 
 define <8 x i64> @test128(<8 x i64> %a, <8 x i64> %b) {
 ; SSE2-LABEL: test128:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm7, %xmm11
-; SSE2-NEXT:    movdqa %xmm11, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm3, %xmm7
 ; SSE2-NEXT:    movdqa %xmm2, %xmm3
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm9
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm10 = [2147483648,2147483648,2147483648,2147483648]
 ; SSE2-NEXT:    movdqa %xmm7, %xmm8
 ; SSE2-NEXT:    pxor %xmm10, %xmm8
 ; SSE2-NEXT:    movdqa %xmm11, %xmm0

 ; SSE2-LABEL: test130:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm2, %xmm8
 ; SSE2-NEXT:    movdqa %xmm3, %xmm12
 ; SSE2-NEXT:    pcmpgtb %xmm7, %xmm12
 ; SSE2-NEXT:    pcmpeqd %xmm13, %xmm13
 ; SSE2-NEXT:    movdqa %xmm12, %xmm9
 ; SSE2-NEXT:    pxor %xmm13, %xmm9
-; SSE2-NEXT:    movdqa %xmm8, %xmm14
+; SSE2-NEXT:    movdqa %xmm2, %xmm14
 ; SSE2-NEXT:    pcmpgtb %xmm6, %xmm14
 ; SSE2-NEXT:    movdqa %xmm14, %xmm2
 ; SSE2-NEXT:    pxor %xmm13, %xmm2
 ; SSE2-NEXT:    movdqa %xmm1, %xmm15
 ; SSE2-NEXT:    pcmpgtb %xmm5, %xmm15
 ; SSE2-NEXT:    movdqa %xmm15, %xmm10
 ; SSE2-NEXT:    pxor %xmm13, %xmm10
 ; SSE2-NEXT:    movdqa %xmm0, %xmm11

 ; SSE2-NEXT:    movdqa %xmm2, %xmm8
 ; SSE2-NEXT:    movdqa %xmm0, %xmm10
 ; SSE2-NEXT:    movdqa %xmm7, %xmm12
 ; SSE2-NEXT:    pcmpgtb %xmm3, %xmm12
 ; SSE2-NEXT:    pcmpeqd %xmm0, %xmm0
 ; SSE2-NEXT:    movdqa %xmm12, %xmm9
 ; SSE2-NEXT:    pxor %xmm0, %xmm9
 ; SSE2-NEXT:    movdqa %xmm6, %xmm13
-; SSE2-NEXT:    pcmpgtb %xmm8, %xmm13
+; SSE2-NEXT:    pcmpgtb %xmm2, %xmm13
 ; SSE2-NEXT:    movdqa %xmm13, %xmm2
 ; SSE2-NEXT:    pxor %xmm0, %xmm2
 ; SSE2-NEXT:    movdqa %xmm5, %xmm14
 ; SSE2-NEXT:    pcmpgtb %xmm1, %xmm14
 ; SSE2-NEXT:    movdqa %xmm14, %xmm11
 ; SSE2-NEXT:    pxor %xmm0, %xmm11
 ; SSE2-NEXT:    movdqa %xmm4, %xmm15
 ; SSE2-NEXT:    pcmpgtb %xmm10, %xmm15

 ; SSE2-LABEL: test146:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm2, %xmm8
 ; SSE2-NEXT:    movdqa %xmm3, %xmm12
 ; SSE2-NEXT:    pcmpgtd %xmm7, %xmm12
 ; SSE2-NEXT:    pcmpeqd %xmm13, %xmm13
 ; SSE2-NEXT:    movdqa %xmm12, %xmm9
 ; SSE2-NEXT:    pxor %xmm13, %xmm9
-; SSE2-NEXT:    movdqa %xmm8, %xmm14
+; SSE2-NEXT:    movdqa %xmm2, %xmm14
 ; SSE2-NEXT:    pcmpgtd %xmm6, %xmm14
 ; SSE2-NEXT:    movdqa %xmm14, %xmm2
 ; SSE2-NEXT:    pxor %xmm13, %xmm2
 ; SSE2-NEXT:    movdqa %xmm1, %xmm15
 ; SSE2-NEXT:    pcmpgtd %xmm5, %xmm15
 ; SSE2-NEXT:    movdqa %xmm15, %xmm10
 ; SSE2-NEXT:    pxor %xmm13, %xmm10
 ; SSE2-NEXT:    movdqa %xmm0, %xmm11

 ; SSE2-NEXT:    movdqa %xmm2, %xmm8
 ; SSE2-NEXT:    movdqa %xmm0, %xmm10
 ; SSE2-NEXT:    movdqa %xmm7, %xmm12
 ; SSE2-NEXT:    pcmpgtd %xmm3, %xmm12
 ; SSE2-NEXT:    pcmpeqd %xmm0, %xmm0
 ; SSE2-NEXT:    movdqa %xmm12, %xmm9
 ; SSE2-NEXT:    pxor %xmm0, %xmm9
 ; SSE2-NEXT:    movdqa %xmm6, %xmm13
-; SSE2-NEXT:    pcmpgtd %xmm8, %xmm13
+; SSE2-NEXT:    pcmpgtd %xmm2, %xmm13
 ; SSE2-NEXT:    movdqa %xmm13, %xmm2
 ; SSE2-NEXT:    pxor %xmm0, %xmm2
 ; SSE2-NEXT:    movdqa %xmm5, %xmm14
 ; SSE2-NEXT:    pcmpgtd %xmm1, %xmm14
 ; SSE2-NEXT:    movdqa %xmm14, %xmm11
 ; SSE2-NEXT:    pxor %xmm0, %xmm11
 ; SSE2-NEXT:    movdqa %xmm4, %xmm15
 ; SSE2-NEXT:    pcmpgtd %xmm10, %xmm15
entry:
   %sel = select <8 x i1> %cmp, <8 x i64> %b, <8 x i64> %a
   ret <8 x i64> %sel
 }
 
 define <8 x i64> @test154(<8 x i64> %a, <8 x i64> %b) {
 ; SSE2-LABEL: test154:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm7, %xmm8
-; SSE2-NEXT:    movdqa %xmm8, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm3, %xmm7
 ; SSE2-NEXT:    movdqa %xmm2, %xmm3
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm9
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm10 = [2147483648,0,2147483648,0]
 ; SSE2-NEXT:    pxor %xmm10, %xmm8
 ; SSE2-NEXT:    movdqa %xmm7, %xmm0
 ; SSE2-NEXT:    pxor %xmm10, %xmm0
entry:
   %sel = select <8 x i1> %cmp, <8 x i64> %b, <8 x i64> %a
   ret <8 x i64> %sel
 }
 
 define <8 x i64> @test156(<8 x i64> %a, <8 x i64> %b) {
 ; SSE2-LABEL: test156:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm7, %xmm11
-; SSE2-NEXT:    movdqa %xmm11, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm3, %xmm7
 ; SSE2-NEXT:    movdqa %xmm2, %xmm3
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm9
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm10 = [2147483648,0,2147483648,0]
 ; SSE2-NEXT:    movdqa %xmm7, %xmm8
 ; SSE2-NEXT:    pxor %xmm10, %xmm8
 ; SSE2-NEXT:    movdqa %xmm11, %xmm0
entry:
   %sel = select <8 x i1> %cmp, <8 x i64> %b, <8 x i64> %a
   ret <8 x i64> %sel
 }
 
 define <8 x i64> @test158(<8 x i64> %a, <8 x i64> %b) {
 ; SSE2-LABEL: test158:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm7, %xmm8
-; SSE2-NEXT:    movdqa %xmm8, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm3, %xmm7
 ; SSE2-NEXT:    movdqa %xmm2, %xmm3
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm9
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm10 = [2147483648,2147483648,2147483648,2147483648]
 ; SSE2-NEXT:    pxor %xmm10, %xmm8
 ; SSE2-NEXT:    movdqa %xmm7, %xmm0
 ; SSE2-NEXT:    pxor %xmm10, %xmm0
entry:
   %sel = select <8 x i1> %cmp, <8 x i64> %b, <8 x i64> %a
   ret <8 x i64> %sel
 }
 
 define <8 x i64> @test160(<8 x i64> %a, <8 x i64> %b) {
 ; SSE2-LABEL: test160:
 ; SSE2:       # %bb.0: # %entry
 ; SSE2-NEXT:    movdqa %xmm7, %xmm11
-; SSE2-NEXT:    movdqa %xmm11, -{{[0-9]+}}(%rsp) # 16-byte Spill
+; SSE2-NEXT:    movdqa %xmm7, -{{[0-9]+}}(%rsp) # 16-byte Spill
 ; SSE2-NEXT:    movdqa %xmm3, %xmm7
 ; SSE2-NEXT:    movdqa %xmm2, %xmm3
 ; SSE2-NEXT:    movdqa %xmm1, %xmm2
 ; SSE2-NEXT:    movdqa %xmm0, %xmm9
 ; SSE2-NEXT:    movdqa {{.*#+}} xmm10 = [2147483648,2147483648,2147483648,2147483648]
 ; SSE2-NEXT:    movdqa %xmm7, %xmm8
 ; SSE2-NEXT:    pxor %xmm10, %xmm8
 ; SSE2-NEXT:    movdqa %xmm11, %xmm0

 ; SSE2-NEXT:    por %xmm3, %xmm2
 ; SSE2-NEXT:    movdqa %xmm2, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSE4-LABEL: test180:
 ; SSE4:       # %bb.0: # %entry
 ; SSE4-NEXT:    movdqa %xmm0, %xmm2
 ; SSE4-NEXT:    movdqa %xmm1, %xmm3
-; SSE4-NEXT:    pcmpgtq %xmm2, %xmm3
+; SSE4-NEXT:    pcmpgtq %xmm0, %xmm3
 ; SSE4-NEXT:    pcmpeqd %xmm0, %xmm0
 ; SSE4-NEXT:    pxor %xmm3, %xmm0
 ; SSE4-NEXT:    blendvpd %xmm0, %xmm2, %xmm1
 ; SSE4-NEXT:    movapd %xmm1, %xmm0
 ; SSE4-NEXT:    retq
 ;
 ; AVX1-LABEL: test180:
 ; AVX1:       # %bb.0: # %entry

 ; SSE2-NEXT:    por %xmm3, %xmm2
 ; SSE2-NEXT:    movdqa %xmm2, %xmm0
 ; SSE2-NEXT:    retq
 ;
 ; SSE4-LABEL: test188:
 ; SSE4:       # %bb.0: # %entry
 ; SSE4-NEXT:    movdqa %xmm0, %xmm2
 ; SSE4-NEXT:    movdqa %xmm1, %xmm3
-; SSE4-NEXT:    pcmpgtq %xmm2, %xmm3
+; SSE4-NEXT:    pcmpgtq %xmm0, %xmm3
 ; SSE4-NEXT:    pcmpeqd %xmm0, %xmm0
 ; SSE4-NEXT:    pxor %xmm3, %xmm0
 ; SSE4-NEXT:    blendvpd %xmm0, %xmm1, %xmm2
 ; SSE4-NEXT:    movapd %xmm2, %xmm0
 ; SSE4-NEXT:    retq
 ;
 ; AVX1-LABEL: test188:
 ; AVX1:       # %bb.0: # %entry

llvm/trunk/test/CodeGen/X86/widen_conv-3.ll

 ; X86-SSE2-NEXT:    movzbl 2(%ecx), %ecx
 ; X86-SSE2-NEXT:    pinsrw $1, %ecx, %xmm0
 ; X86-SSE2-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3,4,4,5,5,6,6,7,7]
 ; X86-SSE2-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0,0,1,1,2,2,3,3]
 ; X86-SSE2-NEXT:    psrad $24, %xmm0
 ; X86-SSE2-NEXT:    cvtdq2ps %xmm0, %xmm0
 ; X86-SSE2-NEXT:    movss %xmm0, (%eax)
 ; X86-SSE2-NEXT:    movaps %xmm0, %xmm1
-; X86-SSE2-NEXT:    movhlps {{.*#+}} xmm1 = xmm1[1,1]
+; X86-SSE2-NEXT:    movhlps {{.*#+}} xmm1 = xmm0[1],xmm1[1]
 ; X86-SSE2-NEXT:    movss %xmm1, 8(%eax)
 ; X86-SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[1,1,2,3]
 ; X86-SSE2-NEXT:    movss %xmm0, 4(%eax)
 ; X86-SSE2-NEXT:    leal -4(%ebp), %esp
 ; X86-SSE2-NEXT:    popl %esi
 ; X86-SSE2-NEXT:    popl %ebp
 ; X86-SSE2-NEXT:    retl
 ;

llvm/trunk/test/CodeGen/X86/widen_conv-4.ll

 ; X86-SSE2-NEXT:    movdqa %xmm0, %xmm2
 ; X86-SSE2-NEXT:    punpckhwd {{.*#+}} xmm2 = xmm2[4],xmm1[4],xmm2[5],xmm1[5],xmm2[6],xmm1[6],xmm2[7],xmm1[7]
 ; X86-SSE2-NEXT:    cvtdq2ps %xmm2, %xmm2
 ; X86-SSE2-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3]
 ; X86-SSE2-NEXT:    cvtdq2ps %xmm0, %xmm0
 ; X86-SSE2-NEXT:    movups %xmm0, (%eax)
 ; X86-SSE2-NEXT:    movss %xmm2, 16(%eax)
 ; X86-SSE2-NEXT:    movaps %xmm2, %xmm0
-; X86-SSE2-NEXT:    movhlps {{.*#+}} xmm0 = xmm0[1,1]
+; X86-SSE2-NEXT:    movhlps {{.*#+}} xmm0 = xmm2[1],xmm0[1]
 ; X86-SSE2-NEXT:    movss %xmm0, 24(%eax)
 ; X86-SSE2-NEXT:    shufps {{.*#+}} xmm2 = xmm2[1,1,2,3]
 ; X86-SSE2-NEXT:    movss %xmm2, 20(%eax)
 ; X86-SSE2-NEXT:    retl
 ;
 ; X86-SSE42-LABEL: convert_v7i16_v7f32:
 ; X86-SSE42:       # %bb.0: # %entry
 ; X86-SSE42-NEXT:    movl {{[0-9]+}}(%esp), %eax

 ; X86-SSE2-NEXT:    movzbl 2(%ecx), %ecx
 ; X86-SSE2-NEXT:    pinsrw $1, %ecx, %xmm0
 ; X86-SSE2-NEXT:    pxor %xmm1, %xmm1
 ; X86-SSE2-NEXT:    punpcklbw {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3],xmm0[4],xmm1[4],xmm0[5],xmm1[5],xmm0[6],xmm1[6],xmm0[7],xmm1[7]
 ; X86-SSE2-NEXT:    punpcklwd {{.*#+}} xmm0 = xmm0[0],xmm1[0],xmm0[1],xmm1[1],xmm0[2],xmm1[2],xmm0[3],xmm1[3]
 ; X86-SSE2-NEXT:    cvtdq2ps %xmm0, %xmm0
 ; X86-SSE2-NEXT:    movss %xmm0, (%eax)
 ; X86-SSE2-NEXT:    movaps %xmm0, %xmm1
-; X86-SSE2-NEXT:    movhlps {{.*#+}} xmm1 = xmm1[1,1]
+; X86-SSE2-NEXT:    movhlps {{.*#+}} xmm1 = xmm0[1],xmm1[1]
 ; X86-SSE2-NEXT:    movss %xmm1, 8(%eax)
 ; X86-SSE2-NEXT:    shufps {{.*#+}} xmm0 = xmm0[1,1,2,3]
 ; X86-SSE2-NEXT:    movss %xmm0, 4(%eax)
 ; X86-SSE2-NEXT:    leal -4(%ebp), %esp
 ; X86-SSE2-NEXT:    popl %esi
 ; X86-SSE2-NEXT:    popl %ebp
 ; X86-SSE2-NEXT:    retl
 ;

llvm/trunk/test/CodeGen/X86/win64_frame.ll

 ; PUSHF-NEXT:    pushq %rdi
 ; PUSHF-NEXT:    .seh_pushreg 7
 ; PUSHF-NEXT:    subq $32, %rsp
 ; PUSHF-NEXT:    .seh_stackalloc 32
 ; PUSHF-NEXT:    leaq {{[0-9]+}}(%rsp), %rbp
 ; PUSHF-NEXT:    .seh_setframe 5, 32
 ; PUSHF-NEXT:    .seh_endprologue
 ; PUSHF-NEXT:    movq %rdx, %rsi
-; PUSHF-NEXT:    movq %rsi, %rax
+; PUSHF-NEXT:    movq %rdx, %rax
 ; PUSHF-NEXT:    lock cmpxchgq %r8, (%rcx)
 ; PUSHF-NEXT:    pushfq
 ; PUSHF-NEXT:    popq %rdi
 ; PUSHF-NEXT:    callq dummy
 ; PUSHF-NEXT:    pushq %rdi
 ; PUSHF-NEXT:    popfq
 ; PUSHF-NEXT:    cmovneq %rsi, %rax
 ; PUSHF-NEXT:    addq $32, %rsp

 ; SAHF-NEXT:    pushq %rdi
 ; SAHF-NEXT:    .seh_pushreg 7
 ; SAHF-NEXT:    subq $32, %rsp
 ; SAHF-NEXT:    .seh_stackalloc 32
 ; SAHF-NEXT:    leaq {{[0-9]+}}(%rsp), %rbp
 ; SAHF-NEXT:    .seh_setframe 5, 32
 ; SAHF-NEXT:    .seh_endprologue
 ; SAHF-NEXT:    movq %rdx, %rsi
-; SAHF-NEXT:    movq %rsi, %rax
+; SAHF-NEXT:    movq %rdx, %rax
 ; SAHF-NEXT:    lock cmpxchgq %r8, (%rcx)
 ; SAHF-NEXT:    seto %al
 ; SAHF-NEXT:    lahf
 ; SAHF-NEXT:    movq %rax, %rdi
 ; SAHF-NEXT:    callq dummy
 ; SAHF-NEXT:    pushq %rax
 ; SAHF-NEXT:    movq %rdi, %rax
 ; SAHF-NEXT:    addb $127, %al

llvm/trunk/test/CodeGen/X86/x86-interleaved-access.ll

 ; AVX1-NEXT:    vpunpckhbw {{.*#+}} xmm6 = xmm3[8],xmm1[8],xmm3[9],xmm1[9],xmm3[10],xmm1[10],xmm3[11],xmm1[11],xmm3[12],xmm1[12],xmm3[13],xmm1[13],xmm3[14],xmm1[14],xmm3[15],xmm1[15]
 ; AVX1-NEXT:    vextractf128 $1, %ymm7, %xmm3
 ; AVX1-NEXT:    vpunpckhbw {{.*#+}} xmm7 = xmm5[8],xmm7[8],xmm5[9],xmm7[9],xmm5[10],xmm7[10],xmm5[11],xmm7[11],xmm5[12],xmm7[12],xmm5[13],xmm7[13],xmm5[14],xmm7[14],xmm5[15],xmm7[15]
 ; AVX1-NEXT:    vextractf128 $1, %ymm5, %xmm4
 ; AVX1-NEXT:    vpunpcklbw {{.*#+}} xmm5 = xmm4[0],xmm3[0],xmm4[1],xmm3[1],xmm4[2],xmm3[2],xmm4[3],xmm3[3],xmm4[4],xmm3[4],xmm4[5],xmm3[5],xmm4[6],xmm3[6],xmm4[7],xmm3[7]
 ; AVX1-NEXT:    vpunpckhbw {{.*#+}} xmm3 = xmm4[8],xmm3[8],xmm4[9],xmm3[9],xmm4[10],xmm3[10],xmm4[11],xmm3[11],xmm4[12],xmm3[12],xmm4[13],xmm3[13],xmm4[14],xmm3[14],xmm4[15],xmm3[15]
 ; AVX1-NEXT:    vpunpcklwd {{.*#+}} xmm4 = xmm15[0],xmm5[0],xmm15[1],xmm5[1],xmm15[2],xmm5[2],xmm15[3],xmm5[3]
 ; AVX1-NEXT:    vmovdqa %xmm8, %xmm1
-; AVX1-NEXT:    vpunpcklwd {{.*#+}} xmm11 = xmm1[0],xmm2[0],xmm1[1],xmm2[1],xmm1[2],xmm2[2],xmm1[3],xmm2[3]
+; AVX1-NEXT:    vpunpcklwd {{.*#+}} xmm11 = xmm8[0],xmm2[0],xmm8[1],xmm2[1],xmm8[2],xmm2[2],xmm8[3],xmm2[3]
 ; AVX1-NEXT:    vinsertf128 $1, %xmm4, %ymm11, %ymm0
 ; AVX1-NEXT:    vmovups %ymm0, -{{[0-9]+}}(%rsp) # 32-byte Spill
 ; AVX1-NEXT:    vpunpcklwd {{.*#+}} xmm4 = xmm10[0],xmm9[0],xmm10[1],xmm9[1],xmm10[2],xmm9[2],xmm10[3],xmm9[3]
 ; AVX1-NEXT:    vmovdqa -{{[0-9]+}}(%rsp), %xmm0 # 16-byte Reload
 ; AVX1-NEXT:    vpunpcklwd {{.*#+}} xmm8 = xmm0[0],xmm14[0],xmm0[1],xmm14[1],xmm0[2],xmm14[2],xmm0[3],xmm14[3]
 ; AVX1-NEXT:    vinsertf128 $1, %xmm4, %ymm8, %ymm11
 ; AVX1-NEXT:    vpunpckhwd {{.*#+}} xmm4 = xmm15[4],xmm5[4],xmm15[5],xmm5[5],xmm15[6],xmm5[6],xmm15[7],xmm5[7]
 ; AVX1-NEXT:    vmovdqa %xmm4, -{{[0-9]+}}(%rsp) # 16-byte Spill

llvm/trunk/test/CodeGen/X86/x86-shrink-wrap-unwind.ll

 ;
 ; Prologue code.
 ; (What we push does not matter. It should be some random sratch register.)
 ; CHECK: pushq
 ;
 ; Compare the arguments and jump to exit.
 ; After the prologue is set.
 ; CHECK: movl %edi, [[ARG0CPY:%e[a-z]+]]
-; CHECK-NEXT: cmpl %esi, [[ARG0CPY]]
+; CHECK-NEXT: cmpl %esi, %edi
 ; CHECK-NEXT: jge [[EXIT_LABEL:LBB[0-9_]+]]
 ;
 ; Store %a in the alloca.
 ; CHECK: movl [[ARG0CPY]], 4(%rsp)
 ; Set the alloca address in the second argument.
 ; CHECK-NEXT: leaq 4(%rsp), %rsi
 ; Set the first argument to zero.
 ; CHECK-NEXT: xorl %edi, %edi

 ; CHECK-LABEL: frameUnwind:
 ;
 ; Compare the arguments and jump to exit.
 ; No prologue needed.
 ;
 ; Compare the arguments and jump to exit.
 ; After the prologue is set.
 ; CHECK: movl %edi, [[ARG0CPY:%e[a-z]+]]
-; CHECK-NEXT: cmpl %esi, [[ARG0CPY]]
+; CHECK-NEXT: cmpl %esi, %edi
 ; CHECK-NEXT: jge [[EXIT_LABEL:LBB[0-9_]+]]
 ;
 ; Prologue code.
 ; CHECK: pushq %rbp
 ; CHECK: movq %rsp, %rbp
 ;
 ; Store %a in the alloca.
 ; CHECK: movl [[ARG0CPY]], -4(%rbp)

 ; CHECK-LABEL: framelessnoUnwind:
 ;
 ; Compare the arguments and jump to exit.
 ; No prologue needed.
 ;
 ; Compare the arguments and jump to exit.
 ; After the prologue is set.
 ; CHECK: movl %edi, [[ARG0CPY:%e[a-z]+]]
-; CHECK-NEXT: cmpl %esi, [[ARG0CPY]]
+; CHECK-NEXT: cmpl %esi, %edi
 ; CHECK-NEXT: jge [[EXIT_LABEL:LBB[0-9_]+]]
 ;
 ; Prologue code.
 ; (What we push does not matter. It should be some random sratch register.)
 ; CHECK: pushq
 ;
 ; Store %a in the alloca.
 ; CHECK: movl [[ARG0CPY]], 4(%rsp)

llvm/trunk/test/CodeGen/X86/x86-shrink-wrapping.ll

 
 
 ; Initial motivating example: Simple diamond with a call just on one side.
 ; CHECK-LABEL: foo:
 ;
 ; Compare the arguments and jump to exit.
 ; No prologue needed.
 ; ENABLE: movl %edi, [[ARG0CPY:%e[a-z]+]]
-; ENABLE-NEXT: cmpl %esi, [[ARG0CPY]]
+; ENABLE-NEXT: cmpl %esi, %edi
 ; ENABLE-NEXT: jge [[EXIT_LABEL:LBB[0-9_]+]]
 ;
 ; Prologue code.
 ; (What we push does not matter. It should be some random sratch register.)
 ; CHECK: pushq
 ;
 ; Compare the arguments and jump to exit.
 ; After the prologue is set.
 ; DISABLE: movl %edi, [[ARG0CPY:%e[a-z]+]]
-; DISABLE-NEXT: cmpl %esi, [[ARG0CPY]]
+; DISABLE-NEXT: cmpl %esi, %edi
 ; DISABLE-NEXT: jge [[EXIT_LABEL:LBB[0-9_]+]]
 ;
 ; Store %a in the alloca.
 ; CHECK: movl [[ARG0CPY]], 4(%rsp)
 ; Set the alloca address in the second argument.
 ; CHECK-NEXT: leaq 4(%rsp), %rsi
 ; Set the first argument to zero.
 ; CHECK-NEXT: xorl %edi, %edi

llvm/trunk/test/DebugInfo/COFF/fpo-shrink-wrap.ll

 ;     doSomething(&c);
 ;   return doSomething(&c);
 ; }
 
 ; ASM: @shrink_wrap_basic@16:                  # @"\01@shrink_wrap_basic@16"
 ; ASM:         .cv_fpo_proc    @shrink_wrap_basic@16 8
 ; ASM:         .cv_loc 0 1 3 9                 # t.c:3:9
 ; ASM:         movl    %ecx, %eax
-; ASM:         cmpl    %edx, %eax
+; ASM:         cmpl    %edx, %ecx
 ; ASM:         jl      [[EPILOGUE:LBB0_[0-9]+]]
 
 ; ASM:         pushl   %ebx
 ; ASM:         .cv_fpo_pushreg %ebx
 ; ASM:         pushl   %edi
 ; ASM:         .cv_fpo_pushreg %edi
 ; ASM:         pushl   %esi
 ; ASM:         .cv_fpo_pushreg %esi

llvm/trunk/test/DebugInfo/X86/spill-nospill.ll

 ; CHECK: callq   g
 ; CHECK: movl    %eax, [[X_OFFS:[0-9]+]](%rsp)          # 4-byte Spill
 ; CHECK: #DEBUG_VALUE: f:x <- [DW_OP_plus_uconst [[X_OFFS]]] [$rsp+0]
 ; CHECK: #APP
 ; CHECK: #NO_APP
 ; CHECK: callq   g
 ; CHECK: movl    %eax, %[[CSR:[^ ]*]]
 ; CHECK: #DEBUG_VALUE: f:y <- $esi
-; CHECK: movl    %[[CSR]], %ecx
+; CHECK: movl    %eax, %ecx
 ; CHECK: callq   g
 ; CHECK: movl    %[[CSR]], %ecx
 ; CHECK: callq   g
 ; CHECK: movl    %[[CSR]], %ecx
 ; CHECK: callq   g
 ; CHECK: movl    [[X_OFFS]](%rsp), %eax          # 4-byte Reload
 ; CHECK: #DEBUG_VALUE: f:x <- $eax
 ; CHECK: addl    %[[CSR]], %eax