Teach LLVM about Power 9 D-Form VSX Instructions

[nemanjai]

Summary

This patch teaches LLVM how to generate the some of the Power 9 D-Form VSX instructions. Specifically, lxsd, lxssp, stxsd, and stxssp.

[nemanjai]

This is going to need some additional work to restrict the register sets for all the instructions. Of course, these are scalar loads/stores but they're restricted to the upper 32 VSX registers (the VMX registers) so we can't use the full vsfrc/vssrc register classes.

[cycheng]

+ Ehsan

[cycheng]

In D20310#432000, @nemanjai wrote:

This is going to need some additional work to restrict the register sets for all the instructions. Of course, these are scalar loads/stores but they're restricted to the upper 32 VSX registers (the VMX registers) so we can't use the full vsfrc/vssrc register classes.

I missed Nemanjai's comment, I did some test and looks like we can use vfrc register calss instead:

• assembler/dis-assembler:

static DecodeStatus DecodeVFRCRegisterClass(MCInst &Inst, uint64_t RegNo,
                                            uint64_t Address,
                                            const void *Decoder) {
  return decodeRegisterClass(Inst, RegNo, VSFRegs);
}

  void addRegVFRCOperands(MCInst &Inst, unsigned N) const {
    assert(N == 1 && "Invalid number of operands!");
    Inst.addOperand(MCOperand::createReg(VSFRegs[getVSReg()]));
  }

• PPCInstrVSX.td

def PPCRegVFRCAsmOperand : AsmOperandClass {
  let Name = "RegVFRC"; let PredicateMethod = "isVSRegNumber";
}
def vfrc : RegisterOperand<VFRC> {
  let ParserMatchClass = PPCRegVFRCAsmOperand;
}

  // Load DWord
  def LXSD  : DSForm_1<57, 2, (outs vfrc:$vD), (ins memrix:$src),
                       "lxsd $vD, $src", IIC_LdStLFD, []>;
  // Load SP from src, convert it to DP, and place in dword[0]
  def LXSSP : DSForm_1<57, 3, (outs vfrc:$vD), (ins memrix:$src),
                       "lxssp $vD, $src", IIC_LdStLFD, []>;

  // Store DWord
  def STXSD  : DSForm_1<61, 2, (outs), (ins vfrc:$vS, memrix:$dst),
                        "stxsd $vS, $dst", IIC_LdStSTFD, []>;
  // Convert DP of dword[0] to SP, and Store to dst
  def STXSSP : DSForm_1<61, 3, (outs), (ins vfrc:$vS, memrix:$dst),
                        "stxssp $vS, $dst", IIC_LdStSTFD, []>;

  let AddedComplexity = 500 in {
    def : Pat<(f64 (load iaddr:$src)), (LXSD  iaddr:$src)>;
    def : Pat<(f32 (load iaddr:$src)), (COPY_TO_REGCLASS (LXSSP iaddr:$src), VFRC)>;
    def : Pat<(f64 (extloadf32 iaddr:$src)),
            (COPY_TO_REGCLASS (LXSSP iaddr:$src), VFRC)>;
    def : Pat<(store f64:$vS, iaddr:$dst), (STXSD $vS, iaddr:$dst)>;
    def : Pat<(store f32:$vS, iaddr:$dst), (STXSSP (COPY_TO_REGCLASS $vS, VFRC), iaddr:$dst)>;
  }

• Test case result:

        lxsd 35, 8(3)
        lxsd 36, 16(3)
        lxsd 32, 24(3)
...

[nemanjai]

In D20310#435420, @cycheng wrote:

In D20310#432000, @nemanjai wrote:

This is going to need some additional work to restrict the register sets for all the instructions. Of course, these are scalar loads/stores but they're restricted to the upper 32 VSX registers (the VMX registers) so we can't use the full vsfrc/vssrc register classes.

I missed Nemanjai's comment, I did some test and looks like we can use vfrc register calss instead:

Unfortunately, this still isn't quite the correct semantics. Although this will target the right physical registers, the encoding is wrong. These really are VR registers and have 5-bit fields in the encoding. Things like:

lxsd 35, 8(3)

are not likely to produce the desired results. These instructions need the VR register to be specified in the 0-31 range which will actually mean VSR 32-63.
As far as I can tell, the idea with these instructions is that we get scalar floating point values using the nice D-Form loads into the remaining VSR's (the FP D-Form loads can be used for VSRs 0-31).

I think perhaps the best way to handle these would be to define a new register class which will alias the VRRC registers, but has 64-bit spill size and can hold f64/f32.

[cycheng]

Unfortunately, this still isn't quite the correct semantics. Although this will target the right physical registers, the encoding is wrong. These really are VR registers and have 5-bit fields in the encoding. Things like:

lxsd 35, 8(3)

are not likely to produce the desired results. These instructions need the VR register to be specified in the 0-31 range which will actually mean VSR 32-63.
As far as I can tell, the idea with these instructions is that we get scalar floating point values using the nice D-Form loads into the remaining VSR's (the FP D-Form loads can be used for VSRs 0-31).

I think perhaps the best way to handle these would be to define a new register class which will alias the VRRC registers, but has 64-bit spill size and can hold f64/f32.

Oh.. I see my mistake, thanks! I will fix this by defining a new register class.

[tjablin]

From CY:

This patch prevent from defining new register class for vsx that uses 64-bit altivec register, it reuses VFRC register class, and does required changes when printing assembly code.

VsxUseAltivecReg and VFRC are used in "PPCInstPrinter::printOperand", "PPCAsmPrinter::printOperand", in order to get correct register name for vsx that uses 64-bit altivec register.

[hfinkel]

In D20310#444384, @tjablin wrote:

From CY:

This patch prevent from defining new register class for vsx that uses 64-bit altivec register, it reuses VFRC register class, and does required changes when printing assembly code.

VsxUseAltivecReg and VFRC are used in "PPCInstPrinter::printOperand", "PPCAsmPrinter::printOperand", in order to get correct register name for vsx that uses 64-bit altivec register.

Thanks for working on this. I'd really like to see a unified solution here, both for this and for the high half of the VSX register file in general (i.e. using this same scheme to eliminate the VSRH registers).

Also, although I suggested using the instruction flag bits for this, I think we should be able to do this without them by looking at the operand descriptions's register class:

MI->getDesc().OpInfo[MO's index].RegClass == PPC::VFRCRegClassID

lib/Target/PowerPC/PPCInstrFormats.td

41

Please spell VSX here in all caps (and in the flag name).

[cycheng]

Thanks for working on this. I'd really like to see a unified solution here, both for this and for the high half of the VSX register file in general (i.e. using this same scheme to eliminate the VSRH registers).

Let me try again : )

So the unified solution, by my understanding, is: we should not define a new register class that is actually alias with existing register class, we can use customized instruction flags and custom c++ code to handle their difference part, e.g. name. Benefits I can image are: we don't have to teach backend that the two register class are the same, we can simplify register class hierarchy.

Our current register hierarchy:

• FPR (64-bit float register): F0 - F31 (f0 - f31)
• VF (64-bit VSX register): VF0 - VF31 (vs32 - vs63)
• VR (128-bit Altivec register): V0 - V31 (v0 - v31), overlap with VF (vs32 - vs63)
• VSRL (128-bit VSX register): VSL0 - VSL31 (vs0 - vs31), overlap with FPR (f0 - f31)
• VSRH (128-bit VSX register): VSH0 - VSH31 (vs32 - vs63), overlap with VR (v0 - v31)

Our current register class:

• F8RC: [f64] F0 - F31
• VFRC: [f64] VF0 - VF31
• VRRC: [v16i8,v8i16,v4i32,v2i64,v1i128,v4f32] V0 - V31
• VSLRC: [v4i32,v4f32,v2f64,v2i64] VSL0 - VSL31
• VSHRC:[v4i32,v4f32,v2f64,v2i64] VSH0 - VSH31
• VSRC: [v4i32,v4f32,v2f64,v2i64] VSLRC+VSHRC
• VSFRC: [f64] F8RC, VFRC
• VSSRC: [f32] F8RC, VFRC

I will remove VSRH, and below is my imaged modification:

In PPCRegisterInfo.td:

• Eliminate all VSRH and related register definitions

class VSRH<VR SubReg, string n> : PPCReg<n> { ... }

foreach Index = 0-31 in {
  def VSH#Index : VSRH<!cast<VR>("V"#Index), "vs" # !add(Index, 32)>,
                  DwarfRegAlias<!cast<VR>("V"#Index)>;
}

def VSHRC : RegisterClass<"PPC", [v4i32,v4f32,v2f64,v2i64], 128,
                          (add VSH2, VSH3, VSH4, VSH5, VSH0, VSH1, VSH6, VSH7, ... )>;

• For VSRC, change VSHRC to VFRC
  • I can image that we will also eliminate some code in PPCVSXCopy.cpp, because we won't have this copy: "VSRC.VSHRC <---> VSFRC.VFRC"

def VSRC  : RegisterClass<"PPC", [v4i32,v4f32,v2f64,v2i64], 128,
                          (add VSLRC, VFRC)>;

• Change VF's definition:

foreach Index = 0-31 in {
  def VF#Index : VF<Index, "v" #Index>,
                 DwarfRegNum<[!add(Index, 77), !add(Index, 77)];
}

• Add v2f64 to VRRC's supported types
• Because we eliminate VSHRC, so we should add its v2f64 to VRRC

def VRRC : RegisterClass<"PPC", [v16i8,v8i16,v4i32,v2i64,v2f64,v1i128,v4f32], 128,

In PPCISelLowering.cpp/PPCCallingConv.td

• revert changes of llvm c9de9e60b90 (r205041)

change1:
        case MVT::v2f64:
        case MVT::v2i64:
          RC = &PPC::VSHRCRegClass; => VRRCRegClass;

change2:
        MF.addLiveIn(VSRH[VR_idx], &PPC::VSHRCRegClass) : => remove (4 places)

change3:
  remove:
    CCIfType<[v2f64, v2i64], CCAssignToReg<[VSH2]>>
    CCIfType<[v2f64, v2i64], CCIfSubtarget<"hasVSX()",
             CCAssignToReg<[VSH2, VSH3, VSH4, VSH5, VSH6, VSH7, VSH8, VSH9]>>>

  add v2f64, v2i64 back to [v16i8, v8i16, v4i32, v4f32]

In PPCAsmPrinter.cpp/PPCInstPrinter.cpp

printOperand()
   if the register belongs to upper VSX registers (and now we use VFRC to represent it),
   we need translate it from v0-v31 to vs32-vs63

In PPCAsmParser.cpp/PPCDisassembler.cpp/PPCRegisterInfo.cpp/PPCVSXCopy.cpp

• Just remove VSH related code

[hfinkel]

• Original Message -----

From: "Chuang-Yu Cheng" <[e-mail]>
To: [e-mail], [e-mail], [e-mail], "nemanja i ibm" <[e-mail]>,
[e-mail]
Cc: [e-mail]
Sent: Wednesday, June 1, 2016 2:58:55 AM
Subject: Re: [PATCH] D20310: Teach LLVM about Power 9 D-Form VSX Instructions

cycheng added a comment.

Thanks for working on this. I'd really like to see a unified
solution here, both for this and for the high half of the VSX
register file in general (i.e. using this same scheme to eliminate
the VSRH registers).

Let me try again : )

So the unified solution, by my understanding, is: we should not
define a new register class that is actually alias with existing
register class, we can use customized instruction flags and custom
c++ code to handle their difference part, e.g. name. Benefits I can
image are: we don't have to teach backend that the two register
class are the same, we can simplify register class hierarchy.

Yes, except that I'm not sure that we want to remove the register classes, just the register definitions themselves. This makes the change smaller, and also does not force us to add VSX-only data types to the Altivec register classes. In short, add VR0-31 directly to VSHRC. Does that make sense? Then you'll need some, but not all, of the changes you outline below.

Thanks again,
Hal

Our current register hierarchy:

• FPR (64-bit float register): F0 - http://reviews.llvm.org/F31 (f0 -

f31)

• VF (64-bit VSX register): VF0 - VF31 (vs32 - vs63)
• VR (128-bit Altivec register): V0 - V31 (v0 - v31), overlap with VF

(vs32 - vs63)

• VSRL (128-bit VSX register): VSL0 - VSL31 (vs0 - vs31), overlap

with FPR (f0 - f31)

• VSRH (128-bit VSX register): VSH0 - VSH31 (vs32 - vs63), overlap

with VR (v0 - v31)

Our current register class:

• F8RC: [f64] F0 - http://reviews.llvm.org/F31
• VFRC: [f64] VF0 - VF31
• VRRC: [v16i8,v8i16,v4i32,v2i64,v1i128,v4f32] V0 - V31
• VSLRC: [v4i32,v4f32,v2f64,v2i64] VSL0 - VSL31
• VSHRC:[v4i32,v4f32,v2f64,v2i64] VSH0 - VSH31
• VSRC: [v4i32,v4f32,v2f64,v2i64] VSLRC+VSHRC
• VSFRC: [f64] F8RC, VFRC
• VSSRC: [f32] F8RC, VFRC

I will remove VSRH, and below is my imaged modification:

In PPCRegisterInfo.td:

• Eliminate all VSRH and related register definitions
  
  class VSRH<VR SubReg, string n> : PPCReg<n> { ... }
  
  foreach Index = 0-31 in { def VSH#Index : VSRH<!cast<VR>("V"#Index), "vs" # !add(Index, 32)>, DwarfRegAlias<!cast<VR>("V"#Index)>; }
  
  def VSHRC : RegisterClass<"PPC", [v4i32,v4f32,v2f64,v2i64], 128, (add VSH2, VSH3, VSH4, VSH5, VSH0, VSH1, VSH6, VSH7, ... )>;

• For VSRC, change VSHRC to VFRC
  • I can image that we will also eliminate some code in PPCVSXCopy.cpp, because we won't have this copy: "VSRC.VSHRC <---> VSFRC.VFRC"
    
    def VSRC : RegisterClass<"PPC", [v4i32,v4f32,v2f64,v2i64], 128, (add VSLRC, VFRC)>;

• Change VF's definition:
  
  foreach Index = 0-31 in { def VF#Index : VF<Index, "v" #Index>, DwarfRegNum<[!add(Index, 77), !add(Index, 77)]; }

• Add v2f64 to VRRC's supported types
• Because we eliminate VSHRC, so we should add its v2f64 to VRRC
  
  def VRRC : RegisterClass<"PPC", [v16i8,v8i16,v4i32,v2i64,v2f64,v1i128,v4f32], 128,

In PPCISelLowering.cpp/PPCCallingConv.td

• revert changes of llvm c9de9e60b90 (r205041)
  
  change1: case MVT::v2f64: case MVT::v2i64: RC = &PPC::VSHRCRegClass; => VRRCRegClass;
  
  change2: MF.addLiveIn(VSRH[VR_idx], &PPC::VSHRCRegClass) : => remove (4 places)
  
  change3: remove: CCIfType<[v2f64, v2i64], CCAssignToReg<[VSH2]>> CCIfType<[v2f64, v2i64], CCIfSubtarget<"hasVSX()", CCAssignToReg<[VSH2, VSH3, VSH4, VSH5, VSH6, VSH7, VSH8, VSH9]>>>
  
  add v2f64, v2i64 back to [v16i8, v8i16, v4i32, v4f32]

In PPCAsmPrinter.cpp/PPCInstPrinter.cpp

printOperand()
   if the register belongs to upper VSX registers (and now we use
   VFRC to represent it),
   we need translate it from v0-v31 to vs32-vs63

`In
PPCAsmParser.cpp/PPCDisassembler.cpp/PPCRegisterInfo.cpp/PPCVSXCopy.cpp`

• Just remove VSH related code

http://reviews.llvm.org/D20310

[cycheng]

Yes, except that I'm not sure that we want to remove the register classes, just the register definitions themselves. This makes the change smaller, and also does not force us to add VSX-only data types to the Altivec register classes. In short, add VR0-31 directly to VSHRC. Does that make sense? Then you'll need some, but not all, of the changes you outline below.

Thanks again,
Hal

Hi Hal,

I’ve seen some benefits because of this elimination, when I was fixing test case failures, I found we could generate shorter code than before. So I thought it is a right direction to simplify register hierarchy.

I had update 7 codegen test cases, but I still had one test case failure, that was encoding checking, I will fix it soon.

The new VSRC is composed of (VSLRC, VRRC), so I have removed all VSH related def and use.

CY

[hfinkel]

In D20310#452368, @cycheng wrote:

Yes, except that I'm not sure that we want to remove the register classes, just the register definitions themselves. This makes the change smaller, and also does not force us to add VSX-only data types to the Altivec register classes. In short, add VR0-31 directly to VSHRC. Does that make sense? Then you'll need some, but not all, of the changes you outline below.

Thanks again,
Hal

Hi Hal,

I’ve seen some benefits because of this elimination, when I was fixing test case failures, I found we could generate shorter code than before. So I thought it is a right direction to simplify register hierarchy.

I had update 7 codegen test cases, but I still had one test case failure, that was encoding checking, I will fix it soon.

The new VSRC is composed of (VSLRC, VRRC), so I have removed all VSH related def and use.

Okay, so you're saying that we generate even better code by eliminating VSHRC in addition to eliminating the register definitions themselves. In that case, I look forward to the updated patch :-)

Thanks again, Hal

CY

[cycheng]

Eliminating VSHRC brought up a new issue for me, but I have fixed it. Tom will upload the new patch later (the patch passed all of my testing on Pwr8).

The issue was:
Because for now, VSRC = (VSLRC, VRRC), so when spill vs0-63 or vr0-31 to stack, backend was possible to generate such code:

STXVD2X %VSL11<kill>, %X31, %X0<kill>, %RM<imp-use>; mem:ST16[FixedStack17]
...
%V3<def> = LVX %X31, %X0<kill>; mem:LD16[FixedStack17]

We spill vs11 by STXVD2X, but we reload it to vr3 by LVX.

Such test case is rare, I hit the issue when I was running 453.povray

// in texture.cpp, InitTextureTable()
int i;
for (i = 0; i < 4096; i++)
{
  hashTable[i] = i;
}

This simple for loop was translated into more than 1,000 lines assembly code (by the way, I thought the inefficient code gen here was related to non-legal vector type legalization that Nemanjai is fixing)

A lot of vmx and vsx registers were used, altivec and vsx instructions were interleaved, then we hit this issue.

[tjablin]

Another update from CY.

Changes:

Remove VSHRC, use existing VRRC instead, so we simplify register class hierarchy
Now, VSRC = (VSLRC, VRRC)
Because VSX can use vs0-vs63 or v0-v31, so we add a custom flag "UseVSXReg" to distinguish it, asm printer and PPCMCCodeEmitter will use that flag to print correct name and get correct encoding
Because VSRC = (VSLRC, VRRC), so backend is possible to spill/reload a vector register with incompatible vector store/load instruction, i.e. stxvd2x/lxv or stxv/lxvd2x, so we track stack slot to make sure we use correct store/load instruction.
Fix broken test cases because of this change.
select-i1-vs-i1.ll
Now we generate bettter code for all v4f32 test cases

vsx-p8.ll
vsx.ll
Now we generate better code when we use '-fast-isel -O0', because we don't need unnecessary register copy.

Testing:

Passed 3-stage bootstrap testing
Passed 3-stage llvm/clang check-all
Passed Spec2006 (binaries built by stage-3 clang/llvm) with:
(a) -m64 -mcpu=power8 -O3
(b) -m64 -mcpu=power8 -O1

[hfinkel]

lib/Target/PowerPC/InstPrinter/PPCInstPrinter.cpp

451

Can you please use something like this instead:

MI->getDesc().OpInfo[MO's index].RegClass == PPC::VFRCRegClassID

and then get rid of the UseVSXReg flag?

lib/Target/PowerPC/PPCMachineFunctionInfo.h

118 ↗

(On Diff #60689)

Use a DenseMap. You don't need to iterate over the ordering.

[amehsan]

As we discussed, before you commit the change, please add -verify-machineinstrs to your regression tests. No need to upload the patch again. Thanks.

[cycheng]

In D20310#457783, @amehsan wrote:

As we discussed, before you commit the change, please add -verify-machineinstrs to your regression tests. No need to upload the patch again. Thanks.

Done! Thanks!

Hi Hal,

Looks like we can eliminate my stack slot tracking by force using STXVD2X/LXVD2X when HasVSX is true. I have tested my full test-set, we only need to update 2 llc test cases.

By the way, Spec2006 config I used in testing:

1. -m64 -O3 -mcpu=power8
2. -m64 -O1 -mcpu=power8 -mno-vsx

CY

lib/Target/PowerPC/InstPrinter/PPCInstPrinter.cpp

451

1. I tried it, but I got failed, the reason was: RegClass return Super Register Class name, i.e. VSRC in our case, so when I get VSRC, I won't be able to know it is VRRC or VSLRC. And we only want to translate VRRC, not VSLRC.

const MCInstrDesc &MCID = MI->getDesc();

if (MCID.TSFlags & PPCII::UseVSXReg) {
  int RCID = MCID.OpInfo[OpNo].RegClass;

  if (RCID == PPC::VRRCRegClassID)
    Reg = PPC::VSX32 + (Reg - PPC::V0);
  else if (RCID == PPC::VFRCRegClassID)
    Reg = PPC::VSX32 + (Reg - PPC::VF0);
}

2. Probably not! Because the information is not enough. When the input register is VRRC, I won't know whether it is Altivec instruction, or VSX instruction, I need a flag for this. (so I need to tag the information in PPCInstrVSX.td)

So when UseVSXReg == true:

1. MI is a VSX
2. MI uses VSX registers

when UseVSXReg == false:

1. MI can be a VSX or Altivec, but they both use Altivec registers

[tjablin]

Posting for CY.

• Add -verify-machineinstrs for new testcases
• Eliminate stack slot tracking by force using STXVD2X/LXVD2X when HasVSX is true, also update two testcases, and add a new testcase to guard this new behavior
• Unfortunately, we might be able to remove "UseVSXReg" because when RC is VRRC, we need this flag to know we should treat it as vs32-vs63 or v0-v31.

[cycheng]

Does anybody have any comment on this patch?
Thanks!

[nemanjai]

This no longer applies an can be abandoned. We've implemented these instructions in two separate revisions and we've eliminated the VSHRC register class in yet another revision. There are no aspects of this that remain to be implemented.

[nemanjai]

Abandoning as per the previous comment.

lib/Target/PowerPC/AsmParser/PPCAsmParser.cpp

static const MCPhysReg FRegs[32] = {
   PPC::F4,  PPC::F5,  PPC::F6,  PPC::F7,
   PPC::F8,  PPC::F9,  PPC::F10, PPC::F11,
   PPC::F12, PPC::F13, PPC::F14, PPC::F15,
   PPC::F16, PPC::F17, PPC::F18, PPC::F19,
   PPC::F20, PPC::F21, PPC::F22, PPC::F23,
   PPC::F24, PPC::F25, PPC::F26, PPC::F27,
   PPC::F28, PPC::F29, PPC::F30, PPC::F31
 };
+static const MCPhysReg VFRegs[32] = {
+  PPC::VF0,  PPC::VF1,  PPC::VF2,  PPC::VF3,
+  PPC::VF4,  PPC::VF5,  PPC::VF6,  PPC::VF7,
+  PPC::VF8,  PPC::VF9,  PPC::VF10, PPC::VF11,
+  PPC::VF12, PPC::VF13, PPC::VF14, PPC::VF15,
+  PPC::VF16, PPC::VF17, PPC::VF18, PPC::VF19,
+  PPC::VF20, PPC::VF21, PPC::VF22, PPC::VF23,
+  PPC::VF24, PPC::VF25, PPC::VF26, PPC::VF27,
+  PPC::VF28, PPC::VF29, PPC::VF30, PPC::VF31
+};
 static const MCPhysReg VRegs[32] = {
   PPC::V0,  PPC::V1,  PPC::V2,  PPC::V3,
   PPC::V4,  PPC::V5,  PPC::V6,  PPC::V7,
   PPC::V8,  PPC::V9,  PPC::V10, PPC::V11,
   PPC::V12, PPC::V13, PPC::V14, PPC::V15,
   PPC::V16, PPC::V17, PPC::V18, PPC::V19,
   PPC::V20, PPC::V21, PPC::V22, PPC::V23,
   PPC::V24, PPC::V25, PPC::V26, PPC::V27,
   PPC::V28, PPC::V29, PPC::V30, PPC::V31
 };
 static const MCPhysReg VSRegs[64] = {
   PPC::VSL0,  PPC::VSL1,  PPC::VSL2,  PPC::VSL3,
   PPC::VSL4,  PPC::VSL5,  PPC::VSL6,  PPC::VSL7,
   PPC::VSL8,  PPC::VSL9,  PPC::VSL10, PPC::VSL11,
   PPC::VSL12, PPC::VSL13, PPC::VSL14, PPC::VSL15,
   PPC::VSL16, PPC::VSL17, PPC::VSL18, PPC::VSL19,
   PPC::VSL20, PPC::VSL21, PPC::VSL22, PPC::VSL23,
   PPC::VSL24, PPC::VSL25, PPC::VSL26, PPC::VSL27,
   PPC::VSL28, PPC::VSL29, PPC::VSL30, PPC::VSL31,
 
-  PPC::VSH0,  PPC::VSH1,  PPC::VSH2,  PPC::VSH3,
-  PPC::VSH4,  PPC::VSH5,  PPC::VSH6,  PPC::VSH7,
-  PPC::VSH8,  PPC::VSH9,  PPC::VSH10, PPC::VSH11,
-  PPC::VSH12, PPC::VSH13, PPC::VSH14, PPC::VSH15,
-  PPC::VSH16, PPC::VSH17, PPC::VSH18, PPC::VSH19,
-  PPC::VSH20, PPC::VSH21, PPC::VSH22, PPC::VSH23,
-  PPC::VSH24, PPC::VSH25, PPC::VSH26, PPC::VSH27,
-  PPC::VSH28, PPC::VSH29, PPC::VSH30, PPC::VSH31
+  PPC::V0,  PPC::V1,  PPC::V2,  PPC::V3,
+  PPC::V4,  PPC::V5,  PPC::V6,  PPC::V7,
+  PPC::V8,  PPC::V9,  PPC::V10, PPC::V11,
+  PPC::V12, PPC::V13, PPC::V14, PPC::V15,
+  PPC::V16, PPC::V17, PPC::V18, PPC::V19,
+  PPC::V20, PPC::V21, PPC::V22, PPC::V23,
+  PPC::V24, PPC::V25, PPC::V26, PPC::V27,
+  PPC::V28, PPC::V29, PPC::V30, PPC::V31
 };
 static const MCPhysReg VSFRegs[64] = {
   PPC::F0,  PPC::F1,  PPC::F2,  PPC::F3,
   PPC::F4,  PPC::F5,  PPC::F6,  PPC::F7,
   PPC::F8,  PPC::F9,  PPC::F10, PPC::F11,
   PPC::F12, PPC::F13, PPC::F14, PPC::F15,
   PPC::F16, PPC::F17, PPC::F18, PPC::F19,
   PPC::F20, PPC::F21, PPC::F22, PPC::F23,
void addRegF4RCOperands(MCInst &Inst, unsigned N) const {
     Inst.addOperand(MCOperand::createReg(FRegs[getReg()]));
   }
 
   void addRegF8RCOperands(MCInst &Inst, unsigned N) const {
     assert(N == 1 && "Invalid number of operands!");
     Inst.addOperand(MCOperand::createReg(FRegs[getReg()]));
   }
 
+  void addRegVFRCOperands(MCInst &Inst, unsigned N) const {
+    assert(N == 1 && "Invalid number of operands!");
+    Inst.addOperand(MCOperand::createReg(VFRegs[getReg()]));
+  }
+
   void addRegVRRCOperands(MCInst &Inst, unsigned N) const {
     assert(N == 1 && "Invalid number of operands!");
     Inst.addOperand(MCOperand::createReg(VRegs[getReg()]));
   }
 
   void addRegVSRCOperands(MCInst &Inst, unsigned N) const {
     assert(N == 1 && "Invalid number of operands!");
     Inst.addOperand(MCOperand::createReg(VSRegs[getVSReg()]));

lib/Target/PowerPC/Disassembler/PPCDisassembler.cpp

static const unsigned FRegs[] = {
   PPC::F8, PPC::F9, PPC::F10, PPC::F11,
   PPC::F12, PPC::F13, PPC::F14, PPC::F15,
   PPC::F16, PPC::F17, PPC::F18, PPC::F19,
   PPC::F20, PPC::F21, PPC::F22, PPC::F23,
   PPC::F24, PPC::F25, PPC::F26, PPC::F27,
   PPC::F28, PPC::F29, PPC::F30, PPC::F31
 };
 
+static const unsigned VFRegs[] = {
+  PPC::VF0, PPC::VF1, PPC::VF2, PPC::VF3,
+  PPC::VF4, PPC::VF5, PPC::VF6, PPC::VF7,
+  PPC::VF8, PPC::VF9, PPC::VF10, PPC::VF11,
+  PPC::VF12, PPC::VF13, PPC::VF14, PPC::VF15,
+  PPC::VF16, PPC::VF17, PPC::VF18, PPC::VF19,
+  PPC::VF20, PPC::VF21, PPC::VF22, PPC::VF23,
+  PPC::VF24, PPC::VF25, PPC::VF26, PPC::VF27,
+  PPC::VF28, PPC::VF29, PPC::VF30, PPC::VF31
+};
+
 static const unsigned VRegs[] = {
   PPC::V0, PPC::V1, PPC::V2, PPC::V3,
   PPC::V4, PPC::V5, PPC::V6, PPC::V7,
   PPC::V8, PPC::V9, PPC::V10, PPC::V11,
   PPC::V12, PPC::V13, PPC::V14, PPC::V15,
   PPC::V16, PPC::V17, PPC::V18, PPC::V19,
   PPC::V20, PPC::V21, PPC::V22, PPC::V23,
   PPC::V24, PPC::V25, PPC::V26, PPC::V27,
   PPC::V28, PPC::V29, PPC::V30, PPC::V31
 };
 
 static const unsigned VSRegs[] = {
   PPC::VSL0, PPC::VSL1, PPC::VSL2, PPC::VSL3,
   PPC::VSL4, PPC::VSL5, PPC::VSL6, PPC::VSL7,
   PPC::VSL8, PPC::VSL9, PPC::VSL10, PPC::VSL11,
   PPC::VSL12, PPC::VSL13, PPC::VSL14, PPC::VSL15,
   PPC::VSL16, PPC::VSL17, PPC::VSL18, PPC::VSL19,
   PPC::VSL20, PPC::VSL21, PPC::VSL22, PPC::VSL23,
   PPC::VSL24, PPC::VSL25, PPC::VSL26, PPC::VSL27,
   PPC::VSL28, PPC::VSL29, PPC::VSL30, PPC::VSL31,
 
-  PPC::VSH0, PPC::VSH1, PPC::VSH2, PPC::VSH3,
-  PPC::VSH4, PPC::VSH5, PPC::VSH6, PPC::VSH7,
-  PPC::VSH8, PPC::VSH9, PPC::VSH10, PPC::VSH11,
-  PPC::VSH12, PPC::VSH13, PPC::VSH14, PPC::VSH15,
-  PPC::VSH16, PPC::VSH17, PPC::VSH18, PPC::VSH19,
-  PPC::VSH20, PPC::VSH21, PPC::VSH22, PPC::VSH23,
-  PPC::VSH24, PPC::VSH25, PPC::VSH26, PPC::VSH27,
-  PPC::VSH28, PPC::VSH29, PPC::VSH30, PPC::VSH31
+  PPC::V0, PPC::V1, PPC::V2, PPC::V3,
+  PPC::V4, PPC::V5, PPC::V6, PPC::V7,
+  PPC::V8, PPC::V9, PPC::V10, PPC::V11,
+  PPC::V12, PPC::V13, PPC::V14, PPC::V15,
+  PPC::V16, PPC::V17, PPC::V18, PPC::V19,
+  PPC::V20, PPC::V21, PPC::V22, PPC::V23,
+  PPC::V24, PPC::V25, PPC::V26, PPC::V27,
+  PPC::V28, PPC::V29, PPC::V30, PPC::V31
 };
 
 static const unsigned VSFRegs[] = {
   PPC::F0, PPC::F1, PPC::F2, PPC::F3,
   PPC::F4, PPC::F5, PPC::F6, PPC::F7,
   PPC::F8, PPC::F9, PPC::F10, PPC::F11,
   PPC::F12, PPC::F13, PPC::F14, PPC::F15,
   PPC::F16, PPC::F17, PPC::F18, PPC::F19,

 }
 
 static DecodeStatus DecodeF8RCRegisterClass(MCInst &Inst, uint64_t RegNo,
                                             uint64_t Address,
                                             const void *Decoder) {
   return decodeRegisterClass(Inst, RegNo, FRegs);
 }
 
+static DecodeStatus DecodeVFRCRegisterClass(MCInst &Inst, uint64_t RegNo,
+                                            uint64_t Address,
+                                            const void *Decoder) {
+  return decodeRegisterClass(Inst, RegNo, VFRegs);
+}
+
 static DecodeStatus DecodeVRRCRegisterClass(MCInst &Inst, uint64_t RegNo,
                                             uint64_t Address,
                                             const void *Decoder) {
   return decodeRegisterClass(Inst, RegNo, VRegs);
 }
 
 static DecodeStatus DecodeVSRCRegisterClass(MCInst &Inst, uint64_t RegNo,
                                             uint64_t Address,

lib/Target/PowerPC/InstPrinter/PPCInstPrinter.cpp

 //===-- PPCInstPrinter.cpp - Convert PPC MCInst to assembly syntax --------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This class prints an PPC MCInst to a .s file.
 //
 //===----------------------------------------------------------------------===//
 
 #include "PPCInstPrinter.h"
+#include "PPCInstrInfo.h"
 #include "MCTargetDesc/PPCMCTargetDesc.h"
 #include "MCTargetDesc/PPCPredicates.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInst.h"
 #include "llvm/MC/MCInstrInfo.h"
 #include "llvm/MC/MCRegisterInfo.h"
 #include "llvm/MC/MCSubtargetInfo.h"
 #include "llvm/MC/MCSymbol.h"
if (RegName[1] == 's')
       return RegName + 2;
     return RegName + 1;
   case 'c': if (RegName[1] == 'r') return RegName + 2;
   }
   
   return RegName;
 }
 
+static bool isVFRegister(unsigned Reg) {
+  return Reg >= PPC::VF0 && Reg <= PPC::VF31;
+}
+
+static bool isVRRegister(unsigned Reg) {
+  return Reg >= PPC::V0 && Reg <= PPC::V31;
+}
+
 void PPCInstPrinter::printOperand(const MCInst *MI, unsigned OpNo,
                                   raw_ostream &O) {
   const MCOperand &Op = MI->getOperand(OpNo);
   if (Op.isReg()) {
-    const char *RegName = getRegisterName(Op.getReg());
+    unsigned Reg = Op.getReg();
+
+    // If this is a VSX instruction that uses vs32-vs63 register, the register
+    // will be V0-V31 (128-bit altivec) or VF0-VF31 (64-bit altivec), and their
+    // names are "v0-v31", so we need to map "v0-v31" to "vs32-vs63"
+    // (Please synchronize with PPCAsmPrinter::printOperand)
+    if ((MII.get(MI->getOpcode()).TSFlags & PPCII::UseVSXReg)) {
+      if (isVRRegister(Reg))
+        Reg = PPC::VSX32 + (Reg - PPC::V0);
+      else if (isVFRegister(Reg))
+        Reg = PPC::VSX32 + (Reg - PPC::VF0);
+    }
+
+    const char *RegName = getRegisterName(Reg);
     // The linux and AIX assembler does not take register prefixes.
     if (!isDarwinSyntax())
       RegName = stripRegisterPrefix(RegName);
     
     O << RegName;
     return;
   }
   

lib/Target/PowerPC/MCTargetDesc/PPCMCCodeEmitter.cpp

 //===-- PPCMCCodeEmitter.cpp - Convert PPC code to machine code -----------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the PPCMCCodeEmitter class.
 //
 //===----------------------------------------------------------------------===//
 
+#include "PPCInstrInfo.h"
 #include "MCTargetDesc/PPCMCTargetDesc.h"
 #include "MCTargetDesc/PPCFixupKinds.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCCodeEmitter.h"
 #include "llvm/MC/MCContext.h"
 #include "llvm/MC/MCExpr.h"
 #include "llvm/MC/MCInst.h"
get_crbitm_encoding(const MCInst &MI, unsigned OpNo,
                     const MCSubtargetInfo &STI) const {
   const MCOperand &MO = MI.getOperand(OpNo);
   assert((MI.getOpcode() == PPC::MTOCRF || MI.getOpcode() == PPC::MTOCRF8 ||
           MI.getOpcode() == PPC::MFOCRF || MI.getOpcode() == PPC::MFOCRF8) &&
          (MO.getReg() >= PPC::CR0 && MO.getReg() <= PPC::CR7));
   return 0x80 >> CTX.getRegisterInfo()->getEncodingValue(MO.getReg());
 }
 
+static bool isVRRegister(unsigned Reg) {
+  return Reg >= PPC::V0 && Reg <= PPC::V31;
+}
 
 unsigned PPCMCCodeEmitter::
 getMachineOpValue(const MCInst &MI, const MCOperand &MO,
                   SmallVectorImpl<MCFixup> &Fixups,
                   const MCSubtargetInfo &STI) const {
   if (MO.isReg()) {
     // MTOCRF/MFOCRF should go through get_crbitm_encoding for the CR operand.
     // The GPR operand should come through here though.
     assert((MI.getOpcode() != PPC::MTOCRF && MI.getOpcode() != PPC::MTOCRF8 &&
             MI.getOpcode() != PPC::MFOCRF && MI.getOpcode() != PPC::MFOCRF8) ||
            MO.getReg() < PPC::CR0 || MO.getReg() > PPC::CR7);
-    return CTX.getRegisterInfo()->getEncodingValue(MO.getReg());
+    unsigned Reg = MO.getReg();
+    unsigned Encode = CTX.getRegisterInfo()->getEncodingValue(Reg);
+
+    if ((MCII.get(MI.getOpcode()).TSFlags & PPCII::UseVSXReg))
+      if (isVRRegister(Reg))
+        Encode += 32;
+
+    return Encode;
   }
   
   assert(MO.isImm() &&
          "Relocation required in an instruction that we cannot encode!");
   return MO.getImm();
 }
 
 
 #include "PPCGenMCCodeEmitter.inc"

lib/Target/PowerPC/PPCAsmPrinter.cpp

case 'v':
         return RegName + 2;
       return RegName + 1;
     case 'c': if (RegName[1] == 'r') return RegName + 2;
   }
 
   return RegName;
 }
 
+static bool isVFRegister(unsigned Reg) {
+  return Reg >= PPC::VF0 && Reg <= PPC::VF31;
+}
+
+static bool isVRRegister(unsigned Reg) {
+  return Reg >= PPC::V0 && Reg <= PPC::V31;
+}
+
 void PPCAsmPrinter::printOperand(const MachineInstr *MI, unsigned OpNo,
                                  raw_ostream &O) {
   const DataLayout &DL = getDataLayout();
   const MachineOperand &MO = MI->getOperand(OpNo);
 
   switch (MO.getType()) {
   case MachineOperand::MO_Register: {
-    const char *RegName = PPCInstPrinter::getRegisterName(MO.getReg());
+    unsigned Reg = MO.getReg();
+
+    // If this is a VSX instruction that uses vs32-vs63 register, the register
+    // will be V0-V31 (128-bit altivec) or VF0-VF31 (64-bit altivec), and their
+    // names are "v0-v31", so we need to map "v0-v31" to "vs32-vs63"
+    // (Please synchronize with PPCInstPrinter::printOperand)
+    if (MI->getDesc().TSFlags & PPCII::UseVSXReg) {
+      if (isVRRegister(Reg))
+        Reg = PPC::VSX32 + (Reg - PPC::V0);
+      else if (isVFRegister(Reg))
+        Reg = PPC::VSX32 + (Reg - PPC::VF0);
+    }
+    const char *RegName = PPCInstPrinter::getRegisterName(Reg);
+
     // Linux assembler (Others?) does not take register mnemonics.
     // FIXME - What about special registers used in mfspr/mtspr?
     if (!Subtarget->isDarwin())
       RegName = stripRegisterPrefix(RegName);
     O << RegName;
     return;
   }
   case MachineOperand::MO_Immediate:

lib/Target/PowerPC/PPCCallingConv.td

def RetCC_PPC : CallingConv<[
   CCIfType<[f64], CCAssignToReg<[F1, F2, F3, F4, F5, F6, F7, F8]>>,
 
   // QPX vectors are returned in QF1 and QF2. 
   CCIfType<[v4f64, v4f32, v4i1],
            CCIfSubtarget<"hasQPX()", CCAssignToReg<[QF1, QF2]>>>,
  
   // Vector types returned as "direct" go into V2 .. V9; note that only the
   // ELFv2 ABI fully utilizes all these registers.
-  CCIfType<[v16i8, v8i16, v4i32, v2i64, v1i128, v4f32], 
+  CCIfType<[v16i8, v8i16, v4i32, v2f64, v2i64, v1i128, v4f32],
            CCIfSubtarget<"hasAltivec()",
-           CCAssignToReg<[V2, V3, V4, V5, V6, V7, V8, V9]>>>,
-  CCIfType<[v2f64, v2i64], CCIfSubtarget<"hasVSX()",
-           CCAssignToReg<[VSH2, VSH3, VSH4, VSH5, VSH6, VSH7, VSH8, VSH9]>>>
+           CCAssignToReg<[V2, V3, V4, V5, V6, V7, V8, V9]>>>
 ]>;
 
 // No explicit register is specified for the AnyReg calling convention. The
 // register allocator may assign the arguments to any free register.
 //
 // This calling convention is currently only supported by the stackmap and
 // patchpoint intrinsics. All other uses will result in an assert on Debug
 // builds. On Release builds we fallback to the PPC C calling convention.
def RetCC_PPC64_ELF_FIS : CallingConv<[
   CCIfType<[i16],  CCPromoteToType<i64>>,
   CCIfType<[i32],  CCPromoteToType<i64>>,
   CCIfType<[i64],  CCAssignToReg<[X3, X4]>>,
   CCIfType<[i128], CCAssignToReg<[X3, X4, X5, X6]>>,
   CCIfType<[f32],  CCAssignToReg<[F1, F2, F3, F4, F5, F6, F7, F8]>>,
   CCIfType<[f64],  CCAssignToReg<[F1, F2, F3, F4, F5, F6, F7, F8]>>,
   CCIfType<[v4f64, v4f32, v4i1],
            CCIfSubtarget<"hasQPX()", CCAssignToReg<[QF1, QF2]>>>,
-  CCIfType<[v16i8, v8i16, v4i32, v2i64, v1i128, v4f32], 
+  CCIfType<[v16i8, v8i16, v4i32, v2f64, v2i64, v1i128, v4f32],
            CCIfSubtarget<"hasAltivec()",
-           CCAssignToReg<[V2, V3, V4, V5, V6, V7, V8, V9]>>>,
-  CCIfType<[v2f64, v2i64], CCIfSubtarget<"hasVSX()",
-           CCAssignToReg<[VSH2, VSH3, VSH4, VSH5, VSH6, VSH7, VSH8, VSH9]>>>
+           CCAssignToReg<[V2, V3, V4, V5, V6, V7, V8, V9]>>>
 ]>;
 
 //===----------------------------------------------------------------------===//
 // PowerPC System V Release 4 32-bit ABI
 //===----------------------------------------------------------------------===//
 
 def CC_PPC32_SVR4_Common : CallingConv<[
   CCIfType<[i1], CCPromoteToType<i32>>,

 // In contrast to CC_PPC32_SVR4_VarArg, this calling convention first tries to
 // put vector arguments in vector registers before putting them on the stack.
 def CC_PPC32_SVR4 : CallingConv<[
   // QPX vectors mirror the scalar FP convention.
   CCIfType<[v4f64, v4f32, v4i1], CCIfSubtarget<"hasQPX()",
     CCAssignToReg<[QF1, QF2, QF3, QF4, QF5, QF6, QF7, QF8]>>>,
 
   // The first 12 Vector arguments are passed in AltiVec registers.
-  CCIfType<[v16i8, v8i16, v4i32, v2i64, v1i128, v4f32], 
+  CCIfType<[v16i8, v8i16, v4i32, v2f64, v2i64, v1i128, v4f32],
            CCIfSubtarget<"hasAltivec()", CCAssignToReg<[V2, V3, V4, V5, V6, V7,
                           V8, V9, V10, V11, V12, V13]>>>,
-  CCIfType<[v2f64, v2i64], CCIfSubtarget<"hasVSX()",
-           CCAssignToReg<[VSH2, VSH3, VSH4, VSH5, VSH6, VSH7, VSH8, VSH9,
-                          VSH10, VSH11, VSH12, VSH13]>>>,
            
   CCDelegateTo<CC_PPC32_SVR4_Common>
 ]>;  
 
 // Helper "calling convention" to handle aggregate by value arguments.
 // Aggregate by value arguments are always placed in the local variable space
 // of the caller. This calling convention is only used to assign those stack
 // offsets in the callers stack frame.
def CSR_64_AllRegs: CalleeSavedRegs<(add X0, (sequence "X%u", 3, 10),
                                              (sequence "X%u", 14, 31),
                                              (sequence "F%u", 0, 31),
                                              (sequence "CR%u", 0, 7))>;
 
 def CSR_64_AllRegs_Altivec : CalleeSavedRegs<(add CSR_64_AllRegs,
                                              (sequence "V%u", 0, 31))>;
 
 def CSR_64_AllRegs_VSX : CalleeSavedRegs<(add CSR_64_AllRegs_Altivec,
-                                         (sequence "VSL%u", 0, 31),
-                                         (sequence "VSH%u", 0, 31))>;
+                                         (sequence "VSL%u", 0, 31))>;
 

lib/Target/PowerPC/PPCISelLowering.cpp

if (VA.isRegLoc()) {
         case MVT::v4i32:
           RC = &PPC::VRRCRegClass;
           break;
         case MVT::v4f32:
           RC = Subtarget.hasQPX() ? &PPC::QSRCRegClass : &PPC::VRRCRegClass;
           break;
         case MVT::v2f64:
         case MVT::v2i64:
-          RC = &PPC::VSHRCRegClass;
+          RC = &PPC::VRRCRegClass;
           break;
         case MVT::v4f64:
           RC = &PPC::QFRCRegClass;
           break;
         case MVT::v4i1:
           RC = &PPC::QBRCRegClass;
           break;
       }
PPCTargetLowering::LowerFormalArguments_64SVR4(
   static const MCPhysReg GPR[] = {
     PPC::X3, PPC::X4, PPC::X5, PPC::X6,
     PPC::X7, PPC::X8, PPC::X9, PPC::X10,
   };
   static const MCPhysReg VR[] = {
     PPC::V2, PPC::V3, PPC::V4, PPC::V5, PPC::V6, PPC::V7, PPC::V8,
     PPC::V9, PPC::V10, PPC::V11, PPC::V12, PPC::V13
   };
-  static const MCPhysReg VSRH[] = {
-    PPC::VSH2, PPC::VSH3, PPC::VSH4, PPC::VSH5, PPC::VSH6, PPC::VSH7, PPC::VSH8,
-    PPC::VSH9, PPC::VSH10, PPC::VSH11, PPC::VSH12, PPC::VSH13
-  };
 
   const unsigned Num_GPR_Regs = array_lengthof(GPR);
   const unsigned Num_FPR_Regs = 13;
   const unsigned Num_VR_Regs  = array_lengthof(VR);
   const unsigned Num_QFPR_Regs = Num_FPR_Regs;
 
   // Do a first pass over the arguments to determine whether the ABI
   // guarantees that our caller has allocated the parameter save area
for (unsigned ArgNo = 0, e = Ins.size(); ArgNo != e; ++ArgNo) {
     case MVT::v2f64:
     case MVT::v2i64:
     case MVT::v1i128:
       if (!Subtarget.hasQPX()) {
       // These can be scalar arguments or elements of a vector array type
       // passed directly.  The latter are used to implement ELFv2 homogenous
       // vector aggregates.
       if (VR_idx != Num_VR_Regs) {
-        unsigned VReg = (ObjectVT == MVT::v2f64 || ObjectVT == MVT::v2i64) ?
-                        MF.addLiveIn(VSRH[VR_idx], &PPC::VSHRCRegClass) :
-                        MF.addLiveIn(VR[VR_idx], &PPC::VRRCRegClass);
+        unsigned VReg = MF.addLiveIn(VR[VR_idx], &PPC::VRRCRegClass);
         ArgVal = DAG.getCopyFromReg(Chain, dl, VReg, ObjectVT);
         ++VR_idx;
       } else {
         if (CallConv == CallingConv::Fast)
           ComputeArgOffset();
 
         needsLoad = true;
       }
PPCTargetLowering::LowerCall_64SVR4(SDValue Chain, SDValue Callee,
   static const MCPhysReg GPR[] = {
     PPC::X3, PPC::X4, PPC::X5, PPC::X6,
     PPC::X7, PPC::X8, PPC::X9, PPC::X10,
   };
   static const MCPhysReg VR[] = {
     PPC::V2, PPC::V3, PPC::V4, PPC::V5, PPC::V6, PPC::V7, PPC::V8,
     PPC::V9, PPC::V10, PPC::V11, PPC::V12, PPC::V13
   };
-  static const MCPhysReg VSRH[] = {
-    PPC::VSH2, PPC::VSH3, PPC::VSH4, PPC::VSH5, PPC::VSH6, PPC::VSH7, PPC::VSH8,
-    PPC::VSH9, PPC::VSH10, PPC::VSH11, PPC::VSH12, PPC::VSH13
-  };
 
   const unsigned NumGPRs = array_lengthof(GPR);
   const unsigned NumFPRs = 13;
   const unsigned NumVRs  = array_lengthof(VR);
   const unsigned NumQFPRs = NumFPRs;
 
   // When using the fast calling convention, we don't provide backing for
   // arguments that will be in registers.
case MVT::v1i128:
         SDValue Store = DAG.getStore(Chain, dl, Arg, PtrOff,
                                      MachinePointerInfo(), false, false, 0);
         MemOpChains.push_back(Store);
         if (VR_idx != NumVRs) {
           SDValue Load = DAG.getLoad(MVT::v4f32, dl, Store, PtrOff,
                                      MachinePointerInfo(),
                                      false, false, false, 0);
           MemOpChains.push_back(Load.getValue(1));
-
-          unsigned VReg = (Arg.getSimpleValueType() == MVT::v2f64 ||
-                           Arg.getSimpleValueType() == MVT::v2i64) ?
-                          VSRH[VR_idx] : VR[VR_idx];
-          ++VR_idx;
-
-          RegsToPass.push_back(std::make_pair(VReg, Load));
+          RegsToPass.push_back(std::make_pair(VR[VR_idx++], Load));
         }
         ArgOffset += 16;
         for (unsigned i=0; i<16; i+=PtrByteSize) {
           if (GPR_idx == NumGPRs)
             break;
           SDValue Ix = DAG.getNode(ISD::ADD, dl, PtrVT, PtrOff,
                                    DAG.getConstant(i, dl, PtrVT));
           SDValue Load = DAG.getLoad(PtrVT, dl, Store, Ix, MachinePointerInfo(),
                                      false, false, false, 0);
           MemOpChains.push_back(Load.getValue(1));
           RegsToPass.push_back(std::make_pair(GPR[GPR_idx++], Load));
         }
         break;
       }
 
       // Non-varargs Altivec params go into VRs or on the stack.
       if (VR_idx != NumVRs) {
-        unsigned VReg = (Arg.getSimpleValueType() == MVT::v2f64 ||
-                         Arg.getSimpleValueType() == MVT::v2i64) ?
-                        VSRH[VR_idx] : VR[VR_idx];
-        ++VR_idx;
-
-        RegsToPass.push_back(std::make_pair(VReg, Arg));
+        RegsToPass.push_back(std::make_pair(VR[VR_idx++], Arg));
       } else {
         if (CallConv == CallingConv::Fast)
           ComputePtrOff();
 
         LowerMemOpCallTo(DAG, MF, Chain, Arg, PtrOff, SPDiff, ArgOffset,
                          true, isTailCall, true, MemOpChains,
                          TailCallArguments, dl);
         if (CallConv == CallingConv::Fast)

lib/Target/PowerPC/PPCInstrFormats.td

class I<bits<6> opcode, dag OOL, dag IOL, string asmstr, InstrItinClass itin>
 
   /// These fields correspond to the fields in PPCInstrInfo.h.  Any changes to
   /// these must be reflected there!  See comments there for what these are.
   let TSFlags{0}   = PPC970_First;
   let TSFlags{1}   = PPC970_Single;
   let TSFlags{2}   = PPC970_Cracked;
   let TSFlags{5-3} = PPC970_Unit;
 
+  /// Indicate the (VSX) instruction that uses VSX register (vs0-vs63),
+  /// instead of VMX register (v0-v31).
+  /// Because VSX can use both vs32-vs63 (before ISA3.0) and v0-v31 (since
+  /// ISA3.0), and we don't define new register class for this new VMX usage,
+  /// so we use this flag to distinguish it.
+  bits<1> UseVSXReg = 0;
+  let TSFlags{6}   = UseVSXReg;
+
   // Fields used for relation models.
   string BaseName = "";
 
   // For cases where multiple instruction definitions really represent the
   // same underlying instruction but with one definition for 64-bit arguments
   // and one for 32-bit arguments, this bit breaks the degeneracy between
   // the two forms and allows TableGen to generate mapping tables.
   bit Interpretation64Bit = 0;
 }
 
 class PPC970_DGroup_First   { bits<1> PPC970_First = 1;  }
 class PPC970_DGroup_Single  { bits<1> PPC970_Single = 1; }
 class PPC970_DGroup_Cracked { bits<1> PPC970_Cracked = 1; }
 class PPC970_MicroCode;
 
 class PPC970_Unit_Pseudo   { bits<3> PPC970_Unit = 0;   }
 class PPC970_Unit_FXU      { bits<3> PPC970_Unit = 1;   }
 class PPC970_Unit_LSU      { bits<3> PPC970_Unit = 2;   }
 class PPC970_Unit_FPU      { bits<3> PPC970_Unit = 3;   }
 class PPC970_Unit_CRU      { bits<3> PPC970_Unit = 4;   }
 class PPC970_Unit_VALU     { bits<3> PPC970_Unit = 5;   }
 class PPC970_Unit_VPERM    { bits<3> PPC970_Unit = 6;   }
 class PPC970_Unit_BRU      { bits<3> PPC970_Unit = 7;   }
 
+class UseVSXReg { bits<1> UseVSXReg = 1; }
+
 // Two joined instructions; used to emit two adjacent instructions as one.
 // The itinerary from the first instruction is used for scheduling and
 // classification.
 class I2<bits<6> opcode1, bits<6> opcode2, dag OOL, dag IOL, string asmstr,
          InstrItinClass itin>
         : Instruction {
   field bits<64> Inst;
   field bits<64> SoftFail = 0;

lib/Target/PowerPC/PPCInstrInfo.h

enum PPC970_Unit {
   PPC970_FXU    = 1 << PPC970_Shift,   // Fixed Point (aka Integer/ALU) Unit
   PPC970_LSU    = 2 << PPC970_Shift,   // Load Store Unit
   PPC970_FPU    = 3 << PPC970_Shift,   // Floating Point Unit
   PPC970_CRU    = 4 << PPC970_Shift,   // Control Register Unit
   PPC970_VALU   = 5 << PPC970_Shift,   // Vector ALU
   PPC970_VPERM  = 6 << PPC970_Shift,   // Vector Permute Unit
   PPC970_BRU    = 7 << PPC970_Shift    // Branch Unit
 };
+
+enum {
+  /// Shift count to bypass PPC970 flags
+  NewDef_Shift = 6,
+
+  /// The VSX instruction that uses VSX register (vs0-vs63), instead of VMX
+  /// register (v0-v31).
+  UseVSXReg = 0x1 << NewDef_Shift
+};
 } // end namespace PPCII
 
 class PPCSubtarget;
 class PPCInstrInfo : public PPCGenInstrInfo {
   PPCSubtarget &Subtarget;
   const PPCRegisterInfo RI;
 
   bool StoreRegToStackSlot(MachineFunction &MF,

lib/Target/PowerPC/PPCInstrInfo.cpp

if (PPC::F8RCRegClass.contains(DestReg) &&
       PPC::VSRCRegClass.contains(SrcReg)) {
     unsigned SuperReg =
       TRI->getMatchingSuperReg(DestReg, PPC::sub_64, &PPC::VSRCRegClass);
 
     if (VSXSelfCopyCrash && SrcReg == SuperReg)
       llvm_unreachable("nop VSX copy");
 
     DestReg = SuperReg;
-  } else if (PPC::VRRCRegClass.contains(DestReg) &&
-             PPC::VSRCRegClass.contains(SrcReg)) {
-    unsigned SuperReg =
-      TRI->getMatchingSuperReg(DestReg, PPC::sub_128, &PPC::VSRCRegClass);
-
-    if (VSXSelfCopyCrash && SrcReg == SuperReg)
-      llvm_unreachable("nop VSX copy");
-
-    DestReg = SuperReg;
   } else if (PPC::F8RCRegClass.contains(SrcReg) &&
              PPC::VSRCRegClass.contains(DestReg)) {
     unsigned SuperReg =
       TRI->getMatchingSuperReg(SrcReg, PPC::sub_64, &PPC::VSRCRegClass);
 
     if (VSXSelfCopyCrash && DestReg == SuperReg)
       llvm_unreachable("nop VSX copy");
 
     SrcReg = SuperReg;
-  } else if (PPC::VRRCRegClass.contains(SrcReg) &&
-             PPC::VSRCRegClass.contains(DestReg)) {
-    unsigned SuperReg =
-      TRI->getMatchingSuperReg(SrcReg, PPC::sub_128, &PPC::VSRCRegClass);
-
-    if (VSXSelfCopyCrash && DestReg == SuperReg)
-      llvm_unreachable("nop VSX copy");
-
-    SrcReg = SuperReg;
   }
 
   // Different class register copy
   if (PPC::CRBITRCRegClass.contains(SrcReg) &&
       PPC::GPRCRegClass.contains(DestReg)) {
     unsigned CRReg = getCRFromCRBit(SrcReg);
     BuildMI(MBB, I, DL, get(PPC::MFOCRF), DestReg).addReg(CRReg);
     getKillRegState(KillSrc);
PPCInstrInfo::storeRegToStackSlot(MachineBasicBlock &MBB,
                                   const TargetRegisterClass *RC,
                                   const TargetRegisterInfo *TRI) const {
   MachineFunction &MF = *MBB.getParent();
   SmallVector<MachineInstr*, 4> NewMIs;
 
   PPCFunctionInfo *FuncInfo = MF.getInfo<PPCFunctionInfo>();
   FuncInfo->setHasSpills();
 
+  // Because VRRC and VSRC use incompatible store/load instruction, and backend
+  // can spill VRRC to a frame then reload it by using VSRC, we have to make
+  // sure this is not going to happen, so when target has VSX, we just
+  // spill/reload VRRC by VSRC.
+  if (Subtarget.hasVSX() && RC == &PPC::VRRCRegClass)
+    RC = &PPC::VSRCRegClass;
+
   bool NonRI = false, SpillsVRS = false;
   if (StoreRegToStackSlot(MF, SrcReg, isKill, FrameIdx, RC, NewMIs,
                           NonRI, SpillsVRS))
     FuncInfo->setSpillsCR();
 
   if (SpillsVRS)
     FuncInfo->setSpillsVRSAVE();
 
PPCInstrInfo::loadRegFromStackSlot(MachineBasicBlock &MBB,
   MachineFunction &MF = *MBB.getParent();
   SmallVector<MachineInstr*, 4> NewMIs;
   DebugLoc DL;
   if (MI != MBB.end()) DL = MI->getDebugLoc();
 
   PPCFunctionInfo *FuncInfo = MF.getInfo<PPCFunctionInfo>();
   FuncInfo->setHasSpills();
 
+  // Because VRRC and VSRC use incompatible store/load instruction, and backend
+  // can spill VRRC to a frame then reload it by using VSRC, we have to make
+  // sure this is not going to happen, so when target has VSX, we just
+  // spill/reload VRRC by VSRC.
+  if (Subtarget.hasVSX() && RC == &PPC::VRRCRegClass)
+    RC = &PPC::VSRCRegClass;
+
   bool NonRI = false, SpillsVRS = false;
   if (LoadRegFromStackSlot(MF, DL, DestReg, FrameIdx, RC, NewMIs,
                            NonRI, SpillsVRS))
     FuncInfo->setSpillsCR();
 
   if (SpillsVRS)
     FuncInfo->setSpillsVRSAVE();
 

lib/Target/PowerPC/PPCInstrVSX.td

 
 def PPCRegVSSRCAsmOperand : AsmOperandClass {
   let Name = "RegVSSRC"; let PredicateMethod = "isVSRegNumber";
 }
 def vssrc : RegisterOperand<VSSRC> {
   let ParserMatchClass = PPCRegVSSRCAsmOperand;
 }
 
+def PPCRegVFRCAsmOperand : AsmOperandClass {
+  let Name = "RegVFRC"; let PredicateMethod = "isRegNumber";
+}
+def vfrc : RegisterOperand<VFRC> {
+  let ParserMatchClass = PPCRegVFRCAsmOperand;
+}
+
 // Little-endian-specific nodes.
 def SDT_PPClxvd2x : SDTypeProfile<1, 1, [
   SDTCisVT<0, v2f64>, SDTCisPtrTy<1>
 ]>;
 def SDT_PPCstxvd2x : SDTypeProfile<0, 2, [
   SDTCisVT<0, v2f64>, SDTCisPtrTy<1>
 ]>;
 def SDT_PPCxxswapd : SDTypeProfile<1, 1, [

 }
 
 def HasVSX : Predicate<"PPCSubTarget->hasVSX()">;
 def IsLittleEndian : Predicate<"PPCSubTarget->isLittleEndian()">;
 def IsBigEndian : Predicate<"!PPCSubTarget->isLittleEndian()">;
 
 let Predicates = [HasVSX] in {
 let AddedComplexity = 400 in { // Prefer VSX patterns over non-VSX patterns.
+let UseVSXReg = 1 in {
 let hasSideEffects = 0 in { // VSX instructions don't have side effects.
 let Uses = [RM] in {
 
   // Load indexed instructions
   let mayLoad = 1 in {
     def LXSDX : XX1Form<31, 588,
                         (outs vsfrc:$XT), (ins memrr:$src),
                         "lxsdx $XT, $src", IIC_LdStLFD,
def XXSLDWI : XX3Form_2<60, 2,
                        (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB, u2imm:$SHW),
                        "xxsldwi $XT, $XA, $XB, $SHW", IIC_VecPerm, []>;
   def XXSPLTW : XX2Form_2<60, 164,
                        (outs vsrc:$XT), (ins vsrc:$XB, u2imm:$UIM),
                        "xxspltw $XT, $XB, $UIM", IIC_VecPerm,
                        [(set v4i32:$XT,
                              (PPCxxsplt v4i32:$XB, imm32SExt16:$UIM))]>;
 } // hasSideEffects
+} // UseVSXReg = 1
 
 // SELECT_CC_* - Used to implement the SELECT_CC DAG operation.  Expanded after
 // instruction selection into a branch sequence.
 let usesCustomInserter = 1,    // Expanded after instruction selection.
     PPC970_Single = 1 in {
 
   def SELECT_CC_VSRC: Pseudo<(outs vsrc:$dst),
                              (ins crrc:$cond, vsrc:$T, vsrc:$F, i32imm:$BROPC),

 /* FIXME: if the operands are v2i64, these patterns will not match.
    we should define new patterns or otherwise match the same patterns
    when the elements are larger than i32.
 */
 def HasP8Vector : Predicate<"PPCSubTarget->hasP8Vector()">;
 def HasDirectMove : Predicate<"PPCSubTarget->hasDirectMove()">;
 let Predicates = [HasP8Vector] in {
 let AddedComplexity = 400 in { // Prefer VSX patterns over non-VSX patterns.
-  let isCommutable = 1 in {
+  let isCommutable = 1, UseVSXReg = 1 in {
     def XXLEQV : XX3Form<60, 186,
                          (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                          "xxleqv $XT, $XA, $XB", IIC_VecGeneral,
                          [(set v4i32:$XT, (vnot_ppc (xor v4i32:$XA, v4i32:$XB)))]>;
     def XXLNAND : XX3Form<60, 178,
                           (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                           "xxlnand $XT, $XA, $XB", IIC_VecGeneral,
                           [(set v4i32:$XT, (vnot_ppc (and v4i32:$XA,
                                                     v4i32:$XB)))]>;
-  } // isCommutable
+  } // isCommutable, UseVSXReg
 
   def : Pat<(int_ppc_vsx_xxleqv v4i32:$A, v4i32:$B),
             (XXLEQV $A, $B)>;
 
+  let UseVSXReg = 1 in {
   def XXLORC : XX3Form<60, 170,
                        (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                        "xxlorc $XT, $XA, $XB", IIC_VecGeneral,
                        [(set v4i32:$XT, (or v4i32:$XA, (vnot_ppc v4i32:$XB)))]>;
 
   // VSX scalar loads introduced in ISA 2.07
   let mayLoad = 1 in {
     def LXSSPX : XX1Form<31, 524, (outs vssrc:$XT), (ins memrr:$src),
let AddedComplexity = 400 in { // Prefer VSX patterns over non-VSX patterns.
   let mayStore = 1 in {
     def STXSSPX : XX1Form<31, 652, (outs), (ins vssrc:$XT, memrr:$dst),
                           "stxsspx $XT, $dst", IIC_LdStSTFD,
                           [(store f32:$XT, xoaddr:$dst)]>;
     def STXSIWX : XX1Form<31, 140, (outs), (ins vsfrc:$XT, memrr:$dst),
                           "stxsiwx $XT, $dst", IIC_LdStSTFD,
                           [(PPCstfiwx f64:$XT, xoaddr:$dst)]>;
   } // mayStore
+  } // UseVSXReg = 1
 
   def : Pat<(f64 (extloadf32 xoaddr:$src)),
             (COPY_TO_REGCLASS (LXSSPX xoaddr:$src), VSFRC)>;
   def : Pat<(f64 (fextend f32:$src)),
             (COPY_TO_REGCLASS $src, VSFRC)>;
 
   def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETLT)),
             (SELECT_VSSRC (CRANDC $lhs, $rhs), $tval, $fval)>;
def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETUGE)),
             (SELECT_VSSRC (CRORC  $lhs, $rhs), $tval, $fval)>;
   def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETGT)),
             (SELECT_VSSRC (CRANDC $rhs, $lhs), $tval, $fval)>;
   def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETUGT)),
             (SELECT_VSSRC (CRANDC $lhs, $rhs), $tval, $fval)>;
   def : Pat<(f32 (selectcc i1:$lhs, i1:$rhs, f32:$tval, f32:$fval, SETNE)),
             (SELECT_VSSRC (CRXOR $lhs, $rhs), $tval, $fval)>;
 
+  let UseVSXReg = 1 in {
   // VSX Elementary Scalar FP arithmetic (SP)
   let isCommutable = 1 in {
     def XSADDSP : XX3Form<60, 0,
                           (outs vssrc:$XT), (ins vssrc:$XA, vssrc:$XB),
                           "xsaddsp $XT, $XA, $XB", IIC_VecFP,
                           [(set f32:$XT, (fadd f32:$XA, f32:$XB))]>;
     def XSMULSP : XX3Form<60, 16,
                           (outs vssrc:$XT), (ins vssrc:$XA, vssrc:$XB),
def XSCVUXDSP : XX2Form<60, 296,
                       [(set f32:$XT, (PPCfcfidus f64:$XB))]>;
 
   // Conversions between vector and scalar single precision
   def XSCVDPSPN : XX2Form<60, 267, (outs vsrc:$XT), (ins vssrc:$XB),
                           "xscvdpspn $XT, $XB", IIC_VecFP, []>;
   def XSCVSPDPN : XX2Form<60, 331, (outs vssrc:$XT), (ins vsrc:$XB),
                           "xscvspdpn $XT, $XB", IIC_VecFP, []>;
 
+  } // UseVSXReg = 1
 } // AddedComplexity = 400
 } // HasP8Vector
 
+let UseVSXReg = 1 in {
 let Predicates = [HasDirectMove] in {
   // VSX direct move instructions
   def MFVSRD : XX1_RS6_RD5_XO<31, 51, (outs g8rc:$rA), (ins vsfrc:$XT),
                               "mfvsrd $rA, $XT", IIC_VecGeneral,
                               [(set i64:$rA, (PPCmfvsr f64:$XT))]>,
       Requires<[In64BitMode]>;
   def MFVSRWZ : XX1_RS6_RD5_XO<31, 115, (outs gprc:$rA), (ins vsfrc:$XT),
                                "mfvsrwz $rA, $XT", IIC_VecGeneral,
def MTVSRDD: XX1Form<31, 435, (outs vsrc:$XT), (ins g8rc:$rA, g8rc:$rB),
                        "mtvsrdd $XT, $rA, $rB", IIC_VecGeneral,
                        []>, Requires<[In64BitMode]>;
 
   def MFVSRLD: XX1_RS6_RD5_XO<31, 307, (outs g8rc:$rA), (ins vsrc:$XT),
                               "mfvsrld $rA, $XT", IIC_VecGeneral,
                               []>, Requires<[In64BitMode]>;
 
 } // IsISA3_0, HasDirectMove
+} // UseVSXReg = 1
 
 /*  Direct moves of various widths from GPR's into VSR's. Each move lines
     the value up into element 0 (both BE and LE). Namely, entities smaller than
     a doubleword are shifted left and moved for BE. For LE, they're moved, then
     swapped to go into the least significant element of the VSR.
 */
 def MovesToVSR {
   dag BE_BYTE_0 =
let Predicates = [HasP9Vector] in {
 
   // [PO VRT XO VRB XO /], but the VRB is only used the left 64 bits (or less),
   // So we use different operand class for VRB
   class X_VT5_XO5_VB5_TyVB<bits<6> opcode, bits<5> xo2, bits<10> xo, string opc,
                            RegisterOperand vbtype, list<dag> pattern>
     : X_RD5_XO5_RS5<opcode, xo2, xo, (outs vrrc:$vT), (ins vbtype:$vB),
                     !strconcat(opc, " $vT, $vB"), IIC_VecFP, pattern>;
 
+  let UseVSXReg = 1 in {
   // [PO T XO B XO BX /]
   class XX2_RT5_XO5_XB6<bits<6> opcode, bits<5> xo2, bits<9> xo, string opc,
                         list<dag> pattern>
     : XX2_RD5_XO5_RS6<opcode, xo2, xo, (outs g8rc:$rT), (ins vsfrc:$XB),
                       !strconcat(opc, " $rT, $XB"), IIC_VecFP, pattern>;
 
   // [PO T XO B XO BX TX]
   class XX2_XT6_XO5_XB6<bits<6> opcode, bits<5> xo2, bits<9> xo, string opc,
                         RegisterOperand vtype, list<dag> pattern>
     : XX2_RD6_XO5_RS6<opcode, xo2, xo, (outs vtype:$XT), (ins vtype:$XB),
                       !strconcat(opc, " $XT, $XB"), IIC_VecFP, pattern>;
 
   // [PO T A B XO AX BX TX], src and dest register use different operand class
   class XX3_XT5_XA5_XB5<bits<6> opcode, bits<8> xo, string opc,
                   RegisterOperand xty, RegisterOperand aty, RegisterOperand bty,
                   InstrItinClass itin, list<dag> pattern>
     : XX3Form<opcode, xo, (outs xty:$XT), (ins aty:$XA, bty:$XB),
               !strconcat(opc, " $XT, $XA, $XB"), itin, pattern>;
+  } // end UseVSXReg = 1
 
   // [PO VRT VRA VRB XO /]
   class X_VT5_VA5_VB5<bits<6> opcode, bits<10> xo, string opc,
                       list<dag> pattern>
     : XForm_1<opcode, xo, (outs vrrc:$vT), (ins vrrc:$vA, vrrc:$vB),
               !strconcat(opc, " $vT, $vA, $vB"), IIC_VecFP, pattern>;
 
   // [PO VRT VRA VRB XO RO], Round to Odd version of [PO VRT VRA VRB XO /]
let Predicates = [HasP9Vector] in {
 
   // QP Compare Ordered/Unordered
   def XSCMPOQP : X_BF3_VA5_VB5<63, 132, "xscmpoqp", []>;
   def XSCMPUQP : X_BF3_VA5_VB5<63, 644, "xscmpuqp", []>;
 
   // DP/QP Compare Exponents
   def XSCMPEXPDP : XX3Form_1<60, 59,
                              (outs crrc:$crD), (ins vsfrc:$XA, vsfrc:$XB),
-                             "xscmpexpdp $crD, $XA, $XB", IIC_FPCompare, []>;
+                             "xscmpexpdp $crD, $XA, $XB", IIC_FPCompare, []>,
+                   UseVSXReg;
   def XSCMPEXPQP : X_BF3_VA5_VB5<63, 164, "xscmpexpqp", []>;
 
   // DP Compare ==, >=, >, !=
   // Use vsrc for XT, because the entire register of XT is set.
   // XT.dword[1] = 0x0000_0000_0000_0000
   def XSCMPEQDP : XX3_XT5_XA5_XB5<60,  3, "xscmpeqdp", vsrc, vsfrc, vsfrc,
                                   IIC_FPCompare, []>;
   def XSCMPGEDP : XX3_XT5_XA5_XB5<60, 19, "xscmpgedp", vsrc, vsfrc, vsfrc,
                                   IIC_FPCompare, []>;
   def XSCMPGTDP : XX3_XT5_XA5_XB5<60, 11, "xscmpgtdp", vsrc, vsfrc, vsfrc,
                                   IIC_FPCompare, []>;
   def XSCMPNEDP : XX3_XT5_XA5_XB5<60, 27, "xscmpnedp", vsrc, vsfrc, vsfrc,
                                   IIC_FPCompare, []>;
+  let UseVSXReg = 1 in {
   // Vector Compare Not Equal
   def XVCMPNEDP  : XX3Form_Rc<60, 123,
                               (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                               "xvcmpnedp  $XT, $XA, $XB", IIC_VecFPCompare, []>;
   let Defs = [CR6] in
   def XVCMPNEDPo : XX3Form_Rc<60, 123,
                               (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                               "xvcmpnedp. $XT, $XA, $XB", IIC_VecFPCompare, []>,
                               isDOT;
   def XVCMPNESP  : XX3Form_Rc<60,  91,
                               (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                               "xvcmpnesp  $XT, $XA, $XB", IIC_VecFPCompare, []>;
   let Defs = [CR6] in
   def XVCMPNESPo : XX3Form_Rc<60,  91,
                               (outs vsrc:$XT), (ins vsrc:$XA, vsrc:$XB),
                               "xvcmpnesp. $XT, $XA, $XB", IIC_VecFPCompare, []>,
                               isDOT;
+  } // end UseVSXReg = 1
 
   //===--------------------------------------------------------------------===//
   // Quad-Precision Floating-Point Conversion Instructions:
 
   // Convert DP -> QP
-  def XSCVDPQP  : X_VT5_XO5_VB5_TyVB<63, 22, 836, "xscvdpqp", vsfrc, []>;
+  def XSCVDPQP  : X_VT5_XO5_VB5_TyVB<63, 22, 836, "xscvdpqp", vfrc, []>;
 
   // Round & Convert QP -> DP (dword[1] is set to zero)
   def XSCVQPDP  : X_VT5_XO5_VB5   <63, 20, 836, "xscvqpdp" , []>;
   def XSCVQPDPO : X_VT5_XO5_VB5_Ro<63, 20, 836, "xscvqpdpo", []>;
 
   // Truncate & Convert QP -> (Un)Signed (D)Word (dword[1] is set to zero)
   def XSCVQPSDZ : X_VT5_XO5_VB5<63, 25, 836, "xscvqpsdz", []>;
   def XSCVQPSWZ : X_VT5_XO5_VB5<63,  9, 836, "xscvqpswz", []>;
   def XSCVQPUDZ : X_VT5_XO5_VB5<63, 17, 836, "xscvqpudz", []>;
   def XSCVQPUWZ : X_VT5_XO5_VB5<63,  1, 836, "xscvqpuwz", []>;
 
   // Convert (Un)Signed DWord -> QP
-  def XSCVSDQP  : X_VT5_XO5_VB5_TyVB<63, 10, 836, "xscvsdqp", vsfrc, []>;
-  def XSCVUDQP  : X_VT5_XO5_VB5_TyVB<63,  2, 836, "xscvudqp", vsfrc, []>;
+  def XSCVSDQP  : X_VT5_XO5_VB5_TyVB<63, 10, 836, "xscvsdqp", vfrc, []>;
+  def XSCVUDQP  : X_VT5_XO5_VB5_TyVB<63,  2, 836, "xscvudqp", vfrc, []>;
 
   //===--------------------------------------------------------------------===//
   // Round to Floating-Point Integer Instructions
 
   // (Round &) Convert DP <-> HP
   // Note! xscvdphp's src and dest register both use the left 64 bits, so we use
   // vsfrc for src and dest register. xscvhpdp's src only use the left 16 bits,
   // but we still use vsfrc for it.
let Predicates = [HasP9Vector] in {
   def XSRQPXP  : Z23_VT5_R1_VB5_RMC2_EX1<63, 37, 0, "xsrqpxp", []>;
 
   //===--------------------------------------------------------------------===//
   // Insert/Extract Instructions
 
   // Insert Exponent DP/QP
   // XT NOTE: XT.dword[1] = 0xUUUU_UUUU_UUUU_UUUU
   def XSIEXPDP : XX1Form <60, 918, (outs vsrc:$XT), (ins g8rc:$rA, g8rc:$rB),
-                          "xsiexpdp $XT, $rA, $rB", IIC_VecFP, []>;
+                          "xsiexpdp $XT, $rA, $rB", IIC_VecFP, []>, UseVSXReg;
   // vB NOTE: only vB.dword[0] is used, that's why we don't use
   //          X_VT5_VA5_VB5 form
   def XSIEXPQP : XForm_18<63, 868, (outs vrrc:$vT), (ins vrrc:$vA, vsfrc:$vB),
                           "xsiexpqp $vT, $vA, $vB", IIC_VecFP, []>;
 
   // Extract Exponent/Significand DP/QP
   def XSXEXPDP : XX2_RT5_XO5_XB6<60,  0, 347, "xsxexpdp", []>;
   def XSXSIGDP : XX2_RT5_XO5_XB6<60,  1, 347, "xsxsigdp", []>;
+
   def XSXEXPQP : X_VT5_XO5_VB5  <63,  2, 804, "xsxexpqp", []>;
   def XSXSIGQP : X_VT5_XO5_VB5  <63, 18, 804, "xsxsigqp", []>;
 
   // Vector Insert Word
+  let UseVSXReg = 1 in {
   // XB NOTE: Only XB.dword[1] is used, but we use vsrc on XB.
   def XXINSERTW   : XX2_RD6_UIM5_RS6<60, 181,
                                   (outs vsrc:$XT), (ins u4imm:$UIMM, vsrc:$XB),
                                   "xxinsertw $XT, $XB, $UIMM", IIC_VecFP, []>;
 
   // Vector Extract Unsigned Word
   def XXEXTRACTUW : XX2_RD6_UIM5_RS6<60, 165,
                                   (outs vsrc:$XT), (ins u4imm:$UIMM, vsrc:$XB),
                                   "xxextractuw $XT, $XB, $UIMM", IIC_VecFP, []>;
+  } // end UseVSXReg = 1
 
   // Vector Insert Exponent DP/SP
   def XVIEXPDP : XX3_XT5_XA5_XB5<60, 248, "xviexpdp", vsrc, vsrc, vsrc,
                                  IIC_VecFP, []>;
   def XVIEXPSP : XX3_XT5_XA5_XB5<60, 216, "xviexpsp", vsrc, vsrc, vsrc,
                                  IIC_VecFP, []>;
 
   // Vector Extract Exponent/Significand DP/SP
   def XVXEXPDP : XX2_XT6_XO5_XB6<60,  0, 475, "xvxexpdp", vsrc, []>;
   def XVXEXPSP : XX2_XT6_XO5_XB6<60,  8, 475, "xvxexpsp", vsrc, []>;
   def XVXSIGDP : XX2_XT6_XO5_XB6<60,  1, 475, "xvxsigdp", vsrc, []>;
   def XVXSIGSP : XX2_XT6_XO5_XB6<60,  9, 475, "xvxsigsp", vsrc, []>;
 
   //===--------------------------------------------------------------------===//
 
   // Test Data Class SP/DP/QP
+  let UseVSXReg = 1 in {
   def XSTSTDCSP : XX2_BF3_DCMX7_RS6<60, 298,
                               (outs crrc:$BF), (ins u7imm:$DCMX, vsfrc:$XB),
                               "xststdcsp $BF, $XB, $DCMX", IIC_VecFP, []>;
   def XSTSTDCDP : XX2_BF3_DCMX7_RS6<60, 362,
                               (outs crrc:$BF), (ins u7imm:$DCMX, vsfrc:$XB),
                               "xststdcdp $BF, $XB, $DCMX", IIC_VecFP, []>;
+  } // UseVSXReg = 1
   def XSTSTDCQP : X_BF3_DCMX7_RS5  <63, 708,
                               (outs crrc:$BF), (ins u7imm:$DCMX, vrrc:$vB),
                               "xststdcqp $BF, $vB, $DCMX", IIC_VecFP, []>;
 
   // Vector Test Data Class SP/DP
+  let UseVSXReg = 1 in {
   def XVTSTDCSP : XX2_RD6_DCMX7_RS6<60, 13, 5,
                               (outs vsrc:$XT), (ins u7imm:$DCMX, vsrc:$XB),
                               "xvtstdcsp $XT, $XB, $DCMX", IIC_VecFP, []>;
   def XVTSTDCDP : XX2_RD6_DCMX7_RS6<60, 15, 5,
                               (outs vsrc:$XT), (ins u7imm:$DCMX, vsrc:$XB),
                               "xvtstdcdp $XT, $XB, $DCMX", IIC_VecFP, []>;
+  } // UseVSXReg = 1
 
   //===--------------------------------------------------------------------===//
 
   // Maximum/Minimum Type-C/Type-J DP
   // XT.dword[1] = 0xUUUU_UUUU_UUUU_UUUU, so we use vsrc for XT
   def XSMAXCDP : XX3_XT5_XA5_XB5<60, 128, "xsmaxcdp", vsrc, vsfrc, vsfrc,
                                  IIC_VecFP, []>;
   def XSMAXJDP : XX3_XT5_XA5_XB5<60, 144, "xsmaxjdp", vsrc, vsfrc, vsfrc,
let Predicates = [HasP9Vector] in {
   // Vector Permute
   def XXPERM  : XX3_XT5_XA5_XB5<60, 26, "xxperm" , vsrc, vsrc, vsrc,
                                 IIC_VecPerm, []>;
   def XXPERMR : XX3_XT5_XA5_XB5<60, 58, "xxpermr", vsrc, vsrc, vsrc,
                                 IIC_VecPerm, []>;
 
   // Vector Splat Immediate Byte
   def XXSPLTIB : X_RD6_IMM8<60, 360, (outs vsrc:$XT), (ins u8imm:$IMM8),
-                            "xxspltib $XT, $IMM8", IIC_VecPerm, []>;
+                            "xxspltib $XT, $IMM8", IIC_VecPerm, []>, UseVSXReg;
 
   //===--------------------------------------------------------------------===//
   // Vector/Scalar Load/Store Instructions
 
   let mayLoad = 1 in {
   // Load Vector
   def LXV : DQ_RD6_RS5_DQ12<61, 1, (outs vsrc:$XT), (ins memrix16:$src),
-                            "lxv $XT, $src", IIC_LdStLFD, []>;
+                            "lxv $XT, $src", IIC_LdStLFD, []>, UseVSXReg;
   // Load DWord
-  def LXSD  : DSForm_1<57, 2, (outs vrrc:$vD), (ins memrix:$src),
+  def LXSD  : DSForm_1<57, 2, (outs vfrc:$vD), (ins memrix:$src),
                        "lxsd $vD, $src", IIC_LdStLFD, []>;
   // Load SP from src, convert it to DP, and place in dword[0]
-  def LXSSP : DSForm_1<57, 3, (outs vrrc:$vD), (ins memrix:$src),
+  def LXSSP : DSForm_1<57, 3, (outs vfrc:$vD), (ins memrix:$src),
                        "lxssp $vD, $src", IIC_LdStLFD, []>;
 
   // [PO T RA RB XO TX] almost equal to [PO S RA RB XO SX], but has different
   // "out" and "in" dag
   class X_XT6_RA5_RB5<bits<6> opcode, bits<10> xo, string opc,
                       RegisterOperand vtype, list<dag> pattern>
     : XX1Form<opcode, xo, (outs vtype:$XT), (ins memrr:$src),
-              !strconcat(opc, " $XT, $src"), IIC_LdStLFD, pattern>;
+              !strconcat(opc, " $XT, $src"), IIC_LdStLFD, pattern>, UseVSXReg;
 
   // Load as Integer Byte/Halfword & Zero Indexed
   def LXSIBZX : X_XT6_RA5_RB5<31, 781, "lxsibzx", vsfrc, []>;
   def LXSIHZX : X_XT6_RA5_RB5<31, 813, "lxsihzx", vsfrc, []>;
 
   // Load Vector Halfword*8/Byte*16 Indexed
   def LXVH8X  : X_XT6_RA5_RB5<31, 812, "lxvh8x" , vsrc, []>;
   def LXVB16X : X_XT6_RA5_RB5<31, 876, "lxvb16x", vsrc, []>;
 
   // Load Vector Indexed
   def LXVX    : X_XT6_RA5_RB5<31, 268, "lxvx"   , vsrc, []>;
 
   // Load Vector (Left-justified) with Length
   def LXVL    : X_XT6_RA5_RB5<31, 269, "lxvl"   , vsrc, []>;
   def LXVLL   : X_XT6_RA5_RB5<31, 301, "lxvll"  , vsrc, []>;
 
   // Load Vector Word & Splat Indexed
   def LXVWSX  : X_XT6_RA5_RB5<31, 364, "lxvwsx" , vsrc, []>;
   } // end mayLoad
 
   let mayStore = 1 in {
   // Store Vector
   def STXV : DQ_RD6_RS5_DQ12<61, 5, (outs), (ins vsrc:$XT, memrix16:$dst),
-                             "stxv $XT, $dst", IIC_LdStSTFD, []>;
+                             "stxv $XT, $dst", IIC_LdStSTFD, []>, UseVSXReg;
   // Store DWord
-  def STXSD  : DSForm_1<61, 2, (outs), (ins vrrc:$vS, memrix:$dst),
+  def STXSD  : DSForm_1<61, 2, (outs), (ins vfrc:$vS, memrix:$dst),
                         "stxsd $vS, $dst", IIC_LdStSTFD, []>;
   // Convert DP of dword[0] to SP, and Store to dst
-  def STXSSP : DSForm_1<61, 3, (outs), (ins vrrc:$vS, memrix:$dst),
+  def STXSSP : DSForm_1<61, 3, (outs), (ins vfrc:$vS, memrix:$dst),
                         "stxssp $vS, $dst", IIC_LdStSTFD, []>;
 
   // [PO S RA RB XO SX]
   class X_XS6_RA5_RB5<bits<6> opcode, bits<10> xo, string opc,
                       RegisterOperand vtype, list<dag> pattern>
     : XX1Form<opcode, xo, (outs), (ins vtype:$XT, memrr:$dst),
-              !strconcat(opc, " $XT, $dst"), IIC_LdStSTFD, pattern>;
+              !strconcat(opc, " $XT, $dst"), IIC_LdStSTFD, pattern>, UseVSXReg;
 
   // Store as Integer Byte/Halfword Indexed
   def STXSIBX  : X_XS6_RA5_RB5<31,  909, "stxsibx" , vsfrc, []>;
   def STXSIHX  : X_XS6_RA5_RB5<31,  941, "stxsihx" , vsfrc, []>;
 
   // Store Vector Halfword*8/Byte*16 Indexed
   def STXVH8X  : X_XS6_RA5_RB5<31,  940, "stxvh8x" , vsrc, []>;
   def STXVB16X : X_XS6_RA5_RB5<31, 1004, "stxvb16x", vsrc, []>;
 
   // Store Vector Indexed
   def STXVX    : X_XS6_RA5_RB5<31,  396, "stxvx"   , vsrc, []>;
 
   // Store Vector (Left-justified) with Length
   def STXVL    : X_XS6_RA5_RB5<31,  397, "stxvl"   , vsrc, []>;
   def STXVLL   : X_XS6_RA5_RB5<31,  429, "stxvll"  , vsrc, []>;
   } // end mayStore
+
+  // Prefer Power9 (aka Power v.3 instructions)
+  let AddedComplexity = 500 in {
+    def : Pat<(f64 (load iaddr:$src)), (LXSD  iaddr:$src)>;
+    def : Pat<(f32 (load iaddr:$src)),
+            (COPY_TO_REGCLASS (LXSSP iaddr:$src), VFRC)>;
+    def : Pat<(f64 (extloadf32 iaddr:$src)),
+            (COPY_TO_REGCLASS (LXSSP iaddr:$src), VFRC)>;
+    def : Pat<(store f64:$vS, iaddr:$dst), (STXSD $vS, iaddr:$dst)>;
+    def : Pat<(store f32:$vS, iaddr:$dst),
+            (STXSSP (COPY_TO_REGCLASS $vS, VFRC), iaddr:$dst)>;
+  }
 } // end HasP9Vector

lib/Target/PowerPC/PPCRegisterInfo.cpp

unsigned PPCRegisterInfo::getRegPressureLimit(const TargetRegisterClass *RC,
   case PPC::F8RCRegClassID:
   case PPC::F4RCRegClassID:
   case PPC::QFRCRegClassID:
   case PPC::QSRCRegClassID:
   case PPC::QBRCRegClassID:
   case PPC::VRRCRegClassID:
   case PPC::VFRCRegClassID:
   case PPC::VSLRCRegClassID:
-  case PPC::VSHRCRegClassID:
     return 32 - DefaultSafety;
   case PPC::VSRCRegClassID:
   case PPC::VSFRCRegClassID:
   case PPC::VSSRCRegClassID:
     return 64 - DefaultSafety;
   case PPC::CRRCRegClassID:
     return 8 - DefaultSafety;
   }

lib/Target/PowerPC/PPCRegisterInfo.td

 
 let Namespace = "PPC" in {
 def sub_lt : SubRegIndex<1>;
 def sub_gt : SubRegIndex<1, 1>;
 def sub_eq : SubRegIndex<1, 2>;
 def sub_un : SubRegIndex<1, 3>;
 def sub_32 : SubRegIndex<32>;
 def sub_64 : SubRegIndex<64>;
-def sub_128 : SubRegIndex<128>;
 }
 
 
 class PPCReg<string n> : Register<n> {
   let Namespace = "PPC";
 }
 
 // We identify all our registers with a 5-bit ID, for consistency's sake.

 // VSRL - One of the 32 128-bit VSX registers that overlap with the scalar
 // floating-point registers.
 class VSRL<FPR SubReg, string n> : PPCReg<n> {
   let HWEncoding = SubReg.HWEncoding;
   let SubRegs = [SubReg];
   let SubRegIndices = [sub_64];
 }
 
-// VSRH - One of the 32 128-bit VSX registers that overlap with the vector
-// registers.
-class VSRH<VR SubReg, string n> : PPCReg<n> {
-  let HWEncoding{4-0} = SubReg.HWEncoding{4-0};
-  let HWEncoding{5} = 1;
-  let SubRegs = [SubReg];
-  let SubRegIndices = [sub_128];
-}
-
 // CR - One of the 8 4-bit condition registers
 class CR<bits<3> num, string n, list<Register> subregs> : PPCReg<n> {
   let HWEncoding{2-0} = num;
   let SubRegs = subregs;
 }
 
 // CRBIT - One of the 32 1-bit condition register fields
 class CRBIT<bits<5> num, string n> : PPCReg<n> {

 }
 
 // Floating-point registers
 foreach Index = 0-31 in {
   def F#Index : FPR<Index, "f"#Index>,
                 DwarfRegNum<[!add(Index, 32), !add(Index, 32)]>;
 }
 
-// Floating-point vector subregisters (for VSX)
+// 64-bit Floating-point vector subregisters (for VSX)
+// Note: the register name is v0-v31, this is alias of vs32-vs63. Because VSX
+//   can use both v0-v31 (since ISA3.0) and vs32-vs63 (before ISA3.0). We only
+//   define one register class for both usage, and use custom c++ code to adjust
+//   asm printing and operand encoding.
 foreach Index = 0-31 in {
-  def VF#Index : VF<Index, "vs" # !add(Index, 32)>;
+  def VF#Index : VF<Index, "v" #Index>,
+                 DwarfRegNum<[!add(Index, 77), !add(Index, 77)]>;
 }
 
 // QPX Floating-point registers
 foreach Index = 0-31 in {
   def QF#Index : QFPR<!cast<FPR>("F"#Index), "q"#Index>,
                  DwarfRegNum<[!add(Index, 32), !add(Index, 32)]>;
 }
 
 // Vector registers
 foreach Index = 0-31 in {
   def V#Index : VR<!cast<VF>("VF"#Index), "v"#Index>,
                 DwarfRegNum<[!add(Index, 77), !add(Index, 77)]>;
 }
 
 // VSX registers
 foreach Index = 0-31 in {
   def VSL#Index : VSRL<!cast<FPR>("F"#Index), "vs"#Index>,
                   DwarfRegAlias<!cast<FPR>("F"#Index)>;
 }
-foreach Index = 0-31 in {
-  def VSH#Index : VSRH<!cast<VR>("V"#Index), "vs" # !add(Index, 32)>,
-                  DwarfRegAlias<!cast<VR>("V"#Index)>;
+
+// Dummy VSX registers, this defines string: "vs32"-"vs63", and is only used for
+// asm printing.
+foreach Index = 32-63 in {
+  def VSX#Index : PPCReg<"vs"#Index>;
 }
 
 // The reprsentation of r0 when treated as the constant 0.
 def ZERO  : GPR<0, "0">,    DwarfRegAlias<R0>;
 def ZERO8 : GP8<ZERO, "0">, DwarfRegAlias<X0>;
 
 // Representations of the frame pointer used by ISD::FRAMEADDR.
 def FP   : GPR<0 /* arbitrary */, "**FRAME POINTER**">;

 // register N is spilled to offset 8 * (32 - N) below the back chain word of the
 // previous stack frame. By allocating non-volatiles in reverse order we make
 // sure that the Floating-point register save area is always as small as
 // possible because there aren't any unused spill slots.
 def F8RC : RegisterClass<"PPC", [f64], 64, (add (sequence "F%u", 0, 13),
                                                 (sequence "F%u", 31, 14))>;
 def F4RC : RegisterClass<"PPC", [f32], 32, (add F8RC)>;
 
-def VRRC : RegisterClass<"PPC", [v16i8,v8i16,v4i32,v2i64,v1i128,v4f32], 128,
+def VRRC : RegisterClass<"PPC", [v16i8,v8i16,v4i32,v2i64,v2f64,v1i128,v4f32], 128,
                          (add V2, V3, V4, V5, V0, V1, V6, V7, V8, V9, V10, V11,
                              V12, V13, V14, V15, V16, V17, V18, V19, V31, V30,
                              V29, V28, V27, V26, V25, V24, V23, V22, V21, V20)>;
 
 // VSX register classes (the allocation order mirrors that of the corresponding
 // subregister classes).
 def VSLRC : RegisterClass<"PPC", [v4i32,v4f32,v2f64,v2i64], 128,
                           (add (sequence "VSL%u", 0, 13),
                                (sequence "VSL%u", 31, 14))>;
-def VSHRC : RegisterClass<"PPC", [v4i32,v4f32,v2f64,v2i64], 128,
-                          (add VSH2, VSH3, VSH4, VSH5, VSH0, VSH1, VSH6, VSH7,
-             VSH8, VSH9, VSH10, VSH11, VSH12, VSH13, VSH14,
-                               VSH15, VSH16, VSH17, VSH18, VSH19, VSH31, VSH30,
-                               VSH29, VSH28, VSH27, VSH26, VSH25, VSH24, VSH23,
-                               VSH22, VSH21, VSH20)>;
 def VSRC  : RegisterClass<"PPC", [v4i32,v4f32,v2f64,v2i64], 128,
-                          (add VSLRC, VSHRC)>;
+                          (add VSLRC, VRRC)>;
 
 // Register classes for the 64-bit "scalar" VSX subregisters.
 def VFRC :  RegisterClass<"PPC", [f64], 64,
                           (add VF2, VF3, VF4, VF5, VF0, VF1, VF6, VF7,
                                VF8, VF9, VF10, VF11, VF12, VF13, VF14,
                                VF15, VF16, VF17, VF18, VF19, VF31, VF30,
                                VF29, VF28, VF27, VF26, VF25, VF24, VF23,
                                VF22, VF21, VF20)>;
+
 def VSFRC : RegisterClass<"PPC", [f64], 64, (add F8RC, VFRC)>;
 
 // Register class for single precision scalars in VSX registers
 def VSSRC : RegisterClass<"PPC", [f32], 32, (add VSFRC)>;
 
 // For QPX
 def QFRC : RegisterClass<"PPC", [v4f64], 256, (add (sequence "QF%u", 0, 13),
                                                 (sequence "QF%u", 31, 14))>;

lib/Target/PowerPC/PPCVSXCopy.cpp

bool processBlock(MachineBasicBlock &MBB) {
         MachineOperand &DstMO = MI->getOperand(0);
         MachineOperand &SrcMO = MI->getOperand(1);
 
         if ( IsVSReg(DstMO.getReg(), MRI) &&
             !IsVSReg(SrcMO.getReg(), MRI)) {
           // This is a copy *to* a VSX register from a non-VSX register.
           Changed = true;
 
-          const TargetRegisterClass *SrcRC =
-            IsVRReg(SrcMO.getReg(), MRI) ? &PPC::VSHRCRegClass :
-                                           &PPC::VSLRCRegClass;
+          const TargetRegisterClass *SrcRC = &PPC::VSLRCRegClass;
           assert((IsF8Reg(SrcMO.getReg(), MRI) ||
-                  IsVRReg(SrcMO.getReg(), MRI) ||
                   IsVSSReg(SrcMO.getReg(), MRI) ||
                   IsVSFReg(SrcMO.getReg(), MRI)) &&
                  "Unknown source for a VSX copy");
 
           unsigned NewVReg = MRI.createVirtualRegister(SrcRC);
           BuildMI(MBB, MI, MI->getDebugLoc(),
                   TII->get(TargetOpcode::SUBREG_TO_REG), NewVReg)
             .addImm(1) // add 1, not 0, because there is no implicit clearing
                        // of the high bits.
             .addOperand(SrcMO)
-            .addImm(IsVRReg(SrcMO.getReg(), MRI) ? PPC::sub_128 :
-                                                   PPC::sub_64);
+            .addImm(PPC::sub_64);
 
           // The source of the original copy is now the new virtual register.
           SrcMO.setReg(NewVReg);
         } else if (!IsVSReg(DstMO.getReg(), MRI) &&
                     IsVSReg(SrcMO.getReg(), MRI)) {
           // This is a copy *from* a VSX register to a non-VSX register.
           Changed = true;
 
-          const TargetRegisterClass *DstRC =
-            IsVRReg(DstMO.getReg(), MRI) ? &PPC::VSHRCRegClass :
-                                           &PPC::VSLRCRegClass;
+          const TargetRegisterClass *DstRC = &PPC::VSLRCRegClass;
           assert((IsF8Reg(DstMO.getReg(), MRI) ||
                   IsVSFReg(DstMO.getReg(), MRI) ||
-                  IsVSSReg(DstMO.getReg(), MRI) ||
-                  IsVRReg(DstMO.getReg(), MRI)) &&
+                  IsVSSReg(DstMO.getReg(), MRI)) &&
                  "Unknown destination for a VSX copy");
 
           // Copy the VSX value into a new VSX register of the correct subclass.
           unsigned NewVReg = MRI.createVirtualRegister(DstRC);
           BuildMI(MBB, MI, MI->getDebugLoc(),
                   TII->get(TargetOpcode::COPY), NewVReg)
             .addOperand(SrcMO);
 
           // Transform the original copy into a subregister extraction copy.
           SrcMO.setReg(NewVReg);
-          SrcMO.setSubReg(IsVRReg(DstMO.getReg(), MRI) ? PPC::sub_128 :
-                                                         PPC::sub_64);
+          SrcMO.setSubReg(PPC::sub_64);
         }
       }
 
       return Changed;
     }
 
 public:
     bool runOnMachineFunction(MachineFunction &MF) override {

test/CodeGen/PowerPC/dform-test.ll

+; RUN: llc < %s -march=ppc64 -mcpu=pwr9 -verify-machineinstrs | \
+; RUN:   FileCheck %s -check-prefix=PWR9 -check-prefix=CHECK
+; RUN: llc < %s -march=ppc64 -mcpu=pwr8 -verify-machineinstrs | \
+; RUN:   FileCheck %s -check-prefix=PWR8 -check-prefix=CHECK
+
+; CHECK-LABEL: LXSD:
+define void @LXSD(i32 zeroext %N) {
+entry:
+  %cmp17 = icmp eq i32 %N, 0
+  br i1 %cmp17, label %for.cond.cleanup, label %for.body.preheader
+
+for.body.preheader:                               ; preds = %entry
+  br label %for.body
+
+for.cond.cleanup.loopexit:                        ; preds = %for.body
+  br label %for.cond.cleanup
+
+for.cond.cleanup:                                 ; preds = %for.cond.cleanup.loopexit, %entry
+  ret void
+
+for.body:                                         ; preds = %for.body.preheader, %for.body
+; v0 = vsx32
+; PWR9-DAG: lxsd 0
+; PWR9-DAG: lxsd 2
+; PWR9-DAG: lxsd 3
+; PWR9-DAG: lxsd 4
+; PWR9-DAG: lxssp 5
+; PWR9-DAG: xxmrghd 34, 37, 34
+; PWR9-DAG: xxmrghd 35, 37, 35
+; PWR9-DAG: xxmrghd 36, 37, 36
+; PWR9-DAG: xxmrghd 37, 37, 32
+; PWR8-NOT: lxsd{{[^x]}}
+; PWR8-NOT: lxssp{{[^x]}}
+  %i.018 = phi i32 [ %inc, %for.body ], [ 0, %for.body.preheader ]
+  %call = tail call double* @getDoublePtr()
+  %0 = load double, double* %call, align 8
+  %vecinit = insertelement <2 x double> <double 0.000000e+00, double undef>, double %0, i32 1
+  %arrayidx1 = getelementptr inbounds double, double* %call, i64 1
+  %1 = load double, double* %arrayidx1, align 8
+  %vecinit2 = insertelement <2 x double> <double 0.000000e+00, double undef>, double %1, i32 1
+  %arrayidx3 = getelementptr inbounds double, double* %call, i64 2
+  %2 = load double, double* %arrayidx3, align 8
+  %vecinit4 = insertelement <2 x double> <double 0.000000e+00, double undef>, double %2, i32 1
+  %arrayidx5 = getelementptr inbounds double, double* %call, i64 3
+  %3 = load double, double* %arrayidx5, align 8
+  %vecinit6 = insertelement <2 x double> <double 0.000000e+00, double undef>, double %3, i32 1
+  tail call void @passVSX(<2 x double> %vecinit, <2 x double> %vecinit2, <2 x double> %vecinit4, <2 x double> %vecinit6)
+  %inc = add nuw nsw i32 %i.018, 1
+  %exitcond = icmp eq i32 %inc, %N
+  br i1 %exitcond, label %for.cond.cleanup.loopexit, label %for.body
+}
+
+declare double* @getDoublePtr()
+
+; CHECK-LABEL: LXSSP:
+define void @LXSSP(i32 zeroext %N) #0 {
+entry:
+  %cmp20 = icmp eq i32 %N, 0
+  br i1 %cmp20, label %for.cond.cleanup, label %for.body.preheader
+
+for.body.preheader:                               ; preds = %entry
+  br label %for.body
+
+for.cond.cleanup.loopexit:                        ; preds = %for.body
+  br label %for.cond.cleanup
+
+for.cond.cleanup:                                 ; preds = %for.cond.cleanup.loopexit, %entry
+  ret void
+
+for.body:                                         ; preds = %for.body.preheader, %for.body
+
+; PWR9-DAG: lxssp 0
+; PWR9-DAG: lxssp 2
+; PWR9-DAG: lxssp 3
+; PWR9-DAG: lxssp 4
+; PWR9-DAG: lxssp 5
+; PWR9-DAG: xxmrghd 34, 37, 34
+; PWR9-DAG: xxmrghd 35, 37, 35
+; PWR9-DAG: xxmrghd 36, 37, 36
+; PWR9-DAG: xxmrghd 37, 37, 32
+; PWR8-NOT: lxssp{{[^x]}}
+  %i.021 = phi i32 [ %inc, %for.body ], [ 0, %for.body.preheader ]
+  %call = tail call float* @getFloatPtr()
+  %0 = load float, float* %call, align 4
+  %conv = fpext float %0 to double
+  %vecinit = insertelement <2 x double> <double 0.000000e+00, double undef>, double %conv, i32 1
+  %arrayidx1 = getelementptr inbounds float, float* %call, i64 1
+  %1 = load float, float* %arrayidx1, align 4
+  %conv2 = fpext float %1 to double
+  %vecinit3 = insertelement <2 x double> <double 0.000000e+00, double undef>, double %conv2, i32 1
+  %arrayidx4 = getelementptr inbounds float, float* %call, i64 2
+  %2 = load float, float* %arrayidx4, align 4
+  %conv5 = fpext float %2 to double
+  %vecinit6 = insertelement <2 x double> <double 0.000000e+00, double undef>, double %conv5, i32 1
+  %arrayidx7 = getelementptr inbounds float, float* %call, i64 3
+  %3 = load float, float* %arrayidx7, align 4
+  %conv8 = fpext float %3 to double
+  %vecinit9 = insertelement <2 x double> <double 0.000000e+00, double undef>, double %conv8, i32 1
+  tail call void @passVSX(<2 x double> %vecinit, <2 x double> %vecinit3, <2 x double> %vecinit6, <2 x double> %vecinit9)
+  %inc = add nuw nsw i32 %i.021, 1
+  %exitcond = icmp eq i32 %inc, %N
+  br i1 %exitcond, label %for.cond.cleanup.loopexit, label %for.body
+}
+
+declare float* @getFloatPtr()
+
+; CHECK-LABEL: STXSD
+define void @STXSD(i32 zeroext %N) {
+entry:
+  %cmp17 = icmp eq i32 %N, 0
+  br i1 %cmp17, label %for.cond.cleanup, label %for.body.preheader
+
+for.body.preheader:                               ; preds = %entry
+  br label %for.body
+
+for.cond.cleanup.loopexit:                        ; preds = %for.body
+  br label %for.cond.cleanup
+
+for.cond.cleanup:                                 ; preds = %for.cond.cleanup.loopexit, %entry
+  ret void
+
+for.body:                                         ; preds = %for.body.preheader, %for.body
+; PWR9: stxsd{{[^x]}}
+; PWR9: stxsd{{[^x]}}
+; PWR9: stxsd{{[^x]}}
+; PWR9: stxsd{{[^x]}}
+; PWR8-NOT: stxsd{{[^x]}}
+  %i.018 = phi i32 [ %inc, %for.body ], [ 0, %for.body.preheader ]
+  %call = tail call double* @getDoublePtr()
+  %call1 = tail call <2 x double> @getVSX()
+  %vecext = extractelement <2 x double> %call1, i32 0
+  store double %vecext, double* %call, align 8
+  %call2 = tail call <2 x double> @getVSX()
+  %vecext3 = extractelement <2 x double> %call2, i32 0
+  %arrayidx4 = getelementptr inbounds double, double* %call, i64 1
+  store double %vecext3, double* %arrayidx4, align 8
+  %call5 = tail call <2 x double> @getVSX()
+  %vecext6 = extractelement <2 x double> %call5, i32 0
+  %arrayidx7 = getelementptr inbounds double, double* %call, i64 2
+  store double %vecext6, double* %arrayidx7, align 8
+  %call8 = tail call <2 x double> @getVSX()
+  %vecext9 = extractelement <2 x double> %call8, i32 0
+  %arrayidx10 = getelementptr inbounds double, double* %call, i64 3
+  store double %vecext9, double* %arrayidx10, align 8
+  %inc = add nuw nsw i32 %i.018, 1
+  %exitcond = icmp eq i32 %inc, %N
+  br i1 %exitcond, label %for.cond.cleanup.loopexit, label %for.body
+}
+
+declare <2 x double> @getVSX()
+
+define void @STXSSP(i32 zeroext %N) {
+entry:
+  %cmp20 = icmp eq i32 %N, 0
+  br i1 %cmp20, label %for.cond.cleanup, label %for.body.preheader
+
+for.body.preheader:                               ; preds = %entry
+  br label %for.body
+
+for.cond.cleanup.loopexit:                        ; preds = %for.body
+  br label %for.cond.cleanup
+
+for.cond.cleanup:                                 ; preds = %for.cond.cleanup.loopexit, %entry
+  ret void
+
+for.body:                                         ; preds = %for.body.preheader, %for.body
+; v19 = vsx51
+; PWR9: xxlor 51
+; PWR9: stxssp 19
+; PWR9: xxlor 51
+; PWR9: stxssp 19
+; PWR9: xxlor 51
+; PWR9: stxssp 19
+; PWR9: xxlor 51
+; PWR9: stxssp 19
+; PWR8-NOT: stxssp{{[^x]}}
+  %i.021 = phi i32 [ %inc, %for.body ], [ 0, %for.body.preheader ]
+  %call = tail call float* @getFloatPtr()
+  %call1 = tail call <2 x double> @getVSX()
+  %vecext = extractelement <2 x double> %call1, i32 0
+  %conv = fptrunc double %vecext to float
+  store float %conv, float* %call, align 4
+  %call2 = tail call <2 x double> @getVSX()
+  %vecext3 = extractelement <2 x double> %call2, i32 0
+  %conv4 = fptrunc double %vecext3 to float
+  %arrayidx5 = getelementptr inbounds float, float* %call, i64 1
+  store float %conv4, float* %arrayidx5, align 4
+  %call6 = tail call <2 x double> @getVSX()
+  %vecext7 = extractelement <2 x double> %call6, i32 0
+  %conv8 = fptrunc double %vecext7 to float
+  %arrayidx9 = getelementptr inbounds float, float* %call, i64 2
+  store float %conv8, float* %arrayidx9, align 4
+  %call10 = tail call <2 x double> @getVSX()
+  %vecext11 = extractelement <2 x double> %call10, i32 0
+  %conv12 = fptrunc double %vecext11 to float
+  %arrayidx13 = getelementptr inbounds float, float* %call, i64 3
+  store float %conv12, float* %arrayidx13, align 4
+  %inc = add nuw nsw i32 %i.021, 1
+  %exitcond = icmp eq i32 %inc, %N
+  br i1 %exitcond, label %for.cond.cleanup.loopexit, label %for.body
+}
+
+declare void @passVSX(<2 x double>, <2 x double>, <2 x double>, <2 x double>)

test/CodeGen/PowerPC/p8-scalar_vector_conversions.ll

entry:
   %a.addr = alloca i64, align 8
   store i64 %a, i64* %a.addr, align 8
   %0 = load i64, i64* %a.addr, align 8
   %splat.splatinsert = insertelement <2 x i64> undef, i64 %0, i32 0
   %splat.splat = shufflevector <2 x i64> %splat.splatinsert, <2 x i64> undef, <2 x i32> zeroinitializer
   ret <2 x i64> %splat.splat
 ; CHECK: mtvsrd {{[0-9]+}}, 3
 ; CHECK-LE: mtvsrd [[REG1:[0-9]+]], 3
-; CHECK-LE: xxspltd [[REG1]], [[REG1]], 0
+; CHECK-LE: xxspltd 34, [[REG1]], 0
 }
 
 ; Function Attrs: nounwind
 define <4 x float> @buildf(float %a) {
 entry:
   %a.addr = alloca float, align 4
   store float %a, float* %a.addr, align 4
   %0 = load float, float* %a.addr, align 4

test/CodeGen/PowerPC/select-i1-vs-i1.ll

 define <4 x float> @testv4floatslt(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp slt i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
-; FIXME: This test (and the other v4f32 tests) should use the same bclr
-; technique as the v2f64 tests below.
-
 ; CHECK-LABEL: @testv4floatslt
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatult(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp ult i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatult
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatsle(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp sle i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatsle
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatule(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp ule i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatule
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floateq(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp eq i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floateq
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: creqv [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: creqv [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatsge(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp sge i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatsge
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatuge(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp uge i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatuge
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crorc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatsgt(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp sgt i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatsgt
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatugt(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp ugt i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatugt
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crandc [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define <4 x float> @testv4floatne(float %c1, float %c2, float %c3, float %c4, <4 x float> %a1, <4 x float> %a2) #0 {
 entry:
   %cmp1 = fcmp oeq float %c3, %c4
   %cmp3tmp = fcmp oeq float %c1, %c2
   %cmp3 = icmp ne i1 %cmp3tmp, %cmp1
   %cond = select i1 %cmp3, <4 x float> %a1, <4 x float> %a2
   ret <4 x float> %cond
 
 ; CHECK-LABEL: @testv4floatne
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 3, 4
 ; CHECK-DAG: fcmpu {{[0-9]+}}, 1, 2
-; CHECK-DAG: xxlor [[REG2:[0-9]+]], 34, 34
-; CHECK-DAG: crxor [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
-; CHECK: bc 12, [[REG1]], .LBB[[BB:[0-9_]+]]
-; CHECK: xxlor [[REG2]], 35, 35
-; CHECK: .LBB[[BB]]:
-; CHECK: xxlor 34, [[REG2]], [[REG2]]
+; CHECK: crxor [[REG1:[0-9]+]], {{[0-9]+}}, {{[0-9]+}}
+; CHECK: bclr 12, [[REG1]], 0
+; CHECK: vor 2, 3, 3
 ; CHECK: blr
 }
 
 define ppc_fp128 @testppc_fp128eq(ppc_fp128 %c1, ppc_fp128 %c2, ppc_fp128 %c3, ppc_fp128 %c4, ppc_fp128 %a1, ppc_fp128 %a2) #0 {
 entry:
   %cmp1 = fcmp oeq ppc_fp128 %c3, %c4
   %cmp3tmp = fcmp oeq ppc_fp128 %c1, %c2
   %cmp3 = icmp eq i1 %cmp3tmp, %cmp1

test/CodeGen/PowerPC/sjlj.ll

 ; FIXME: We should be saving VRSAVE on Darwin, but we're not!
 
 ; CHECK: @main
 ; CHECK: std
 ; Make sure that we're not saving VRSAVE on non-Darwin:
 ; CHECK-NOT: mfspr
 
 ; CHECK-DAG: stfd
-; CHECK-DAG: stvx
+; CHECK-DAG: stxvd2x
 
 ; CHECK-DAG: addis [[REG:[0-9]+]], 2, env_sigill@toc@ha
 ; CHECK-DAG: std 31, env_sigill@toc@l([[REG]])
 ; CHECK-DAG: addi [[REGA:[0-9]+]], [[REG]], env_sigill@toc@l
 ; CHECK-DAG: std [[REGA]], [[OFF:[0-9]+]](31)                  # 8-byte Folded Spill
 ; CHECK-DAG: std 1, 16([[REGA]])
 ; CHECK-DAG: std 2, 24([[REGA]])
 ; CHECK: bcl 20, 31, .LBB1_5
 ; CHECK: li 3, 1
 ; CHECK: #EH_SjLj_Setup .LBB1_5
 ; CHECK: b .LBB1_1
 
 ; CHECK: .LBB1_4:
 
 ; CHECK: lfd
-; CHECK: lvx
+; CHECK: lxvd2x
 ; CHECK: ld
 ; CHECK: blr
 
 ; CHECK: .LBB1_5:
 ; CHECK: mflr [[REGL:[0-9]+]]
 ; CHECK: ld [[REG2:[0-9]+]], [[OFF]](31)                   # 8-byte Folded Reload
 ; CHECK: std [[REGL]], 8([[REG2]])
 ; CHECK: li 3, 0
 
 ; CHECK-NOAV: @main
-; CHECK-NOAV-NOT: stvx
+; CHECK-NOAV-NOT: stxvd2x
 ; CHECK-NOAV: bcl
 ; CHECK-NOAV: mflr
 ; CHECK-NOAV: bl foo
-; CHECK-NOAV-NOT: lvx
+; CHECK-NOAV-NOT: lxvd2x
 ; CHECK-NOAV: blr
 }
 
 define signext i32 @main2() #0 {
 entry:
   %a = alloca i8, align 64
   call void @bar(i8* %a)
   %retval = alloca i32, align 4

test/CodeGen/PowerPC/vsx-args.ll

 ; RUN: llc < %s -mcpu=pwr7 -mattr=+vsx | FileCheck %s
-; RUN: llc < %s -mcpu=pwr7 -mattr=+vsx -fast-isel -O0 | FileCheck %s
+; RUN: llc < %s -mcpu=pwr7 -mattr=+vsx -fast-isel -O0 | FileCheck -check-prefix=CHECK-FISL %s
 target datalayout = "E-m:e-i64:64-n32:64"
 target triple = "powerpc64-unknown-linux-gnu"
 
 declare <2 x double> @sv(<2 x double>, <2 x i64>, <4 x float>) #0
 
 define <2 x double> @main(<4 x float> %a, <2 x double> %b, <2 x i64> %c) #1 {
 entry:
   %ca = tail call <2 x double> @sv(<2 x double> %b, <2 x i64> %c,  <4 x float> %a)
   %v = fadd <2 x double> %ca, <double 1.0, double 1.0>
   ret <2 x double> %v
 
 ; CHECK-LABEL: @main
 ; CHECK-DAG: vor [[V:[0-9]+]], 2, 2
-; CHECK-DAG: xxlor 34, 35, 35
-; CHECK-DAG: xxlor 35, 36, 36
+; CHECK-DAG: vor 2, 3, 3
+; CHECK-DAG: vor 3, 4, 4
 ; CHECK-DAG: vor 4, [[V]], [[V]]
-; CHECK-DAG: bl sv
-; CHECK-DAG: lxvd2x [[VC:[0-9]+]],
+; CHECK: bl sv
+; CHECK: lxvd2x [[VC:[0-9]+]],
 ; CHECK: xvadddp 34, 34, [[VC]]
 ; CHECK: blr
+
+; CHECK-FISL-LABEL: @main
+; CHECK-FISL: stxvd2x 34, 1, 3
+; CHECK-FISL: vor 2, 3, 3
+; CHECK-FISL: vor 3, 4, 4
+; CHECK-FISL: lxvd2x 36, 1, 3
+; CHECK-FISL: bl sv
+; CHECK-FISL: lxvd2x [[VC:[0-9]+]],
+; CHECK-FISL: xvadddp 34, 34, [[VC]]
+; CHECK-FISL: blr
 }
 
 attributes #0 = { noinline nounwind readnone }
 attributes #1 = { nounwind }
 

test/CodeGen/PowerPC/vsx-infl-copy1.ll

 ; RUN: llc -mcpu=pwr7 < %s | FileCheck %s
 target datalayout = "E-m:e-i64:64-n32:64"
 target triple = "powerpc64-unknown-linux-gnu"
 
 @ub = external global [1024 x i32], align 4
 @uc = external global [1024 x i32], align 4
 
 ; Function Attrs: noinline nounwind
 define <4 x i32> @_Z8example9Pj(<4 x i32>* %addr1, i64 %input1, i64 %input2) #0 {
 entry:
   br label %vector.body
 
 ; CHECK-LABEL: @_Z8example9Pj
-; CHECK: xxlor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
+; CHECK: vor
 
 vector.body:                                      ; preds = %vector.body, %entry
   %index = phi i64 [ 0, %entry ], [ %index.next, %vector.body ]
   %vec.phi = phi <4 x i32> [ zeroinitializer, %entry ], [ %43, %vector.body ]
   %vec.phi20 = phi <4 x i32> [ zeroinitializer, %entry ], [ %44, %vector.body ]
   %vec.phi21 = phi <4 x i32> [ zeroinitializer, %entry ], [ %45, %vector.body ]
   %vec.phi23 = phi <4 x i32> [ zeroinitializer, %entry ], [ %46, %vector.body ]
   %vec.phi24 = phi <4 x i32> [ zeroinitializer, %entry ], [ %47, %vector.body ]

test/CodeGen/PowerPC/vsx-p8.ll

define <4 x float> @test32u(<4 x float>* %a) {
   %v = load <4 x float>, <4 x float>* %a, align 8
   ret <4 x float> %v
 
 ; CHECK-REG-LABEL: @test32u
 ; CHECK-REG: lxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test32u
-; CHECK-FISL: lxvw4x 0, 0, 3
-; CHECK-FISL: xxlor 34, 0, 0
+; CHECK-FISL: lxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 }
 
 define void @test33u(<4 x float>* %a, <4 x float> %b) {
   store <4 x float> %b, <4 x float>* %a, align 8
   ret void
 
 ; CHECK-REG-LABEL: @test33u
 ; CHECK-REG: stxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test33u
-; CHECK-FISL: vor 3, 2, 2
-; CHECK-FISL: stxvw4x 35, 0, 3
+; CHECK-FISL: stxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 }
 

test/CodeGen/PowerPC/vsx-spill-norwstore.ll

 ; RUN: llc -mcpu=pwr7 -verify-machineinstrs < %s | FileCheck %s
 target datalayout = "E-m:e-i64:64-n32:64"
 target triple = "powerpc64-unknown-linux-gnu"
 
 @.str1 = external unnamed_addr constant [5 x i8], align 1
 @.str10 = external unnamed_addr constant [9 x i8], align 1
+@.v2f64 = external unnamed_addr constant <2 x double>, align 16
 
 ; Function Attrs: nounwind
 define void @main() #0 {
 ; CHECK-LABEL: @main
 ; Make sure that the stxvd2x passes -verify-machineinstrs
 ; CHECK: stxvd2x
 
 entry:
+  %val = load <2 x double>, <2 x double>* @.v2f64, align 16
   %0 = tail call <8 x i16> @llvm.ppc.altivec.vupkhsb(<16 x i8> <i8 0, i8 -1, i8 -1, i8 0, i8 0, i8 0, i8 -1, i8 0, i8 -1, i8 0, i8 0, i8 -1, i8 -1, i8 -1, i8 0, i8 -1>) #0
   %1 = tail call <8 x i16> @llvm.ppc.altivec.vupklsb(<16 x i8> <i8 0, i8 -1, i8 -1, i8 0, i8 0, i8 0, i8 -1, i8 0, i8 -1, i8 0, i8 0, i8 -1, i8 -1, i8 -1, i8 0, i8 -1>) #0
   br i1 false, label %if.then.i68.i, label %check.exit69.i
 
 if.then.i68.i:                                    ; preds = %entry
   unreachable
 
 check.exit69.i:                                   ; preds = %entry
   br i1 undef, label %if.then.i63.i, label %check.exit64.i
 
 if.then.i63.i:                                    ; preds = %check.exit69.i
-  tail call void (i8*, ...) @printf(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str10, i64 0, i64 0), i8* getelementptr inbounds ([5 x i8], [5 x i8]* @.str1, i64 0, i64 0)) #0
+  tail call void (i8*, ...) @printf(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str10, i64 0, i64 0), i8* getelementptr inbounds ([5 x i8], [5 x i8]* @.str1, i64 0, i64 0), <2 x double> %val) #0
   br label %check.exit64.i
 
 check.exit64.i:                                   ; preds = %if.then.i63.i, %check.exit69.i
   %2 = tail call i32 @llvm.ppc.altivec.vcmpequh.p(i32 2, <8 x i16> %0, <8 x i16> <i16 0, i16 -1, i16 -1, i16 0, i16 0, i16 0, i16 -1, i16 0>) #0
   %tobool.i55.i = icmp eq i32 %2, 0
   br i1 %tobool.i55.i, label %if.then.i58.i, label %check.exit59.i
 
 if.then.i58.i:                                    ; preds = %check.exit64.i

test/CodeGen/PowerPC/vsx-vec-spill.ll

+; RUN: llc < %s -march=ppc64 -mattr=+vsx -verify-machineinstrs | \
+; RUN:   FileCheck %s --check-prefix=VSX
+; RUN: llc < %s -march=ppc64 -mattr=-vsx -verify-machineinstrs | \
+; RUN:   FileCheck %s --check-prefix=NOVSX
+
+define <2 x double> @interleaving_VSX_VMX(
+  <2 x double> %a, <2 x double> %b, <2 x double> %c,
+  <2 x double> %d, <2 x double> %e, <2 x double> %f) {
+entry:
+  tail call void asm sideeffect "# clobbers",
+    "~{v14},~{v15},~{v16},~{v17},~{v18},~{v19},~{v20},~{v21},~{v22},~{v23},~{v24},~{v25},~{v26},~{v27},~{v28},~{v29},~{v30},~{v31}"() nounwind
+  tail call void @goo(<2 x double> %a) nounwind
+  %add = fadd <2 x double> %a, %b
+  %sub = fsub <2 x double> %a, %b
+  %mul = fmul <2 x double> %add, %sub
+  %add1 = fadd <2 x double> %c, %d
+  %sub2 = fsub <2 x double> %c, %d
+  %mul3 = fmul <2 x double> %add1, %sub2
+  %add4 = fadd <2 x double> %mul, %mul3
+  %add5 = fadd <2 x double> %e, %f
+  %sub6 = fsub <2 x double> %e, %f
+  %mul7 = fmul <2 x double> %add5, %sub6
+  %add8 = fadd <2 x double> %add4, %mul7
+  ret <2 x double> %add8
+; VSX-LABEL: interleaving_VSX_VMX
+; VSX-NOT: stvx
+; VSX-NOT: lvx
+
+; NOVSX-LABEL: interleaving_VSX_VMX
+; NOVSX-NOT: stxvd2x
+; NOVSX-NOT: lxvd2x
+}
+
+declare void @goo(<2 x double>)

test/CodeGen/PowerPC/vsx.ll

entry:
   %v = xor <4 x i32> %a, %b
   ret <4 x i32> %v
 
 ; CHECK-REG-LABEL: @test5
 ; CHECK-REG: xxlxor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test5
-; CHECK-FISL: vor
-; CHECK-FISL: vor
-; CHECK-FISL: xxlxor
-; CHECK-FISL: vor 2
+; CHECK-FISL: xxlxor 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test5
 ; CHECK-LE: xxlxor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <8 x i16> @test6(<8 x i16> %a, <8 x i16> %b) {
 entry:
   %v = xor <8 x i16> %a, %b
   ret <8 x i16> %v
 
 ; CHECK-REG-LABEL: @test6
 ; CHECK-REG: xxlxor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test6
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlxor 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlxor 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test6
 ; CHECK-LE: xxlxor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <16 x i8> @test7(<16 x i8> %a, <16 x i8> %b) {
 entry:
   %v = xor <16 x i8> %a, %b
   ret <16 x i8> %v
 
 ; CHECK-REG-LABEL: @test7
 ; CHECK-REG: xxlxor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test7
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlxor 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlxor 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test7
 ; CHECK-LE: xxlxor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <4 x i32> @test8(<4 x i32> %a, <4 x i32> %b) {
 entry:
   %v = or <4 x i32> %a, %b
   ret <4 x i32> %v
 
 ; CHECK-REG-LABEL: @test8
 ; CHECK-REG: xxlor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test8
-; CHECK-FISL: vor
-; CHECK-FISL: vor
-; CHECK-FISL: xxlor
-; CHECK-FISL: vor 2
+; CHECK-FISL: xxlor 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test8
 ; CHECK-LE: xxlor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <8 x i16> @test9(<8 x i16> %a, <8 x i16> %b) {
 entry:
   %v = or <8 x i16> %a, %b
   ret <8 x i16> %v
 
 ; CHECK-REG-LABEL: @test9
 ; CHECK-REG: xxlor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test9
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlor 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlor 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test9
 ; CHECK-LE: xxlor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <16 x i8> @test10(<16 x i8> %a, <16 x i8> %b) {
 entry:
   %v = or <16 x i8> %a, %b
   ret <16 x i8> %v
 
 ; CHECK-REG-LABEL: @test10
 ; CHECK-REG: xxlor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test10
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlor 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlor 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test10
 ; CHECK-LE: xxlor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <4 x i32> @test11(<4 x i32> %a, <4 x i32> %b) {
 entry:
   %v = and <4 x i32> %a, %b
   ret <4 x i32> %v
 
 ; CHECK-REG-LABEL: @test11
 ; CHECK-REG: xxland 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test11
-; CHECK-FISL: vor
-; CHECK-FISL: vor
-; CHECK-FISL: xxland
-; CHECK-FISL: vor 2
+; CHECK-FISL: xxland 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test11
 ; CHECK-LE: xxland 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <8 x i16> @test12(<8 x i16> %a, <8 x i16> %b) {
 entry:
   %v = and <8 x i16> %a, %b
   ret <8 x i16> %v
 
 ; CHECK-REG-LABEL: @test12
 ; CHECK-REG: xxland 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test12
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxland 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxland 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test12
 ; CHECK-LE: xxland 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <16 x i8> @test13(<16 x i8> %a, <16 x i8> %b) {
 entry:
   %v = and <16 x i8> %a, %b
   ret <16 x i8> %v
 
 ; CHECK-REG-LABEL: @test13
 ; CHECK-REG: xxland 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test13
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxland 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxland 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test13
 ; CHECK-LE: xxland 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <4 x i32> @test14(<4 x i32> %a, <4 x i32> %b) {
 entry:
   %v = or <4 x i32> %a, %b
   %w = xor <4 x i32> %v, <i32 -1, i32 -1, i32 -1, i32 -1>
   ret <4 x i32> %w
 
 ; CHECK-REG-LABEL: @test14
 ; CHECK-REG: xxlnor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test14
-; CHECK-FISL: vor 4, 3, 3
-; CHECK-FISL: vor 5, 2, 2
-; CHECK-FISL: xxlor 0, 37, 36
-; CHECK-FISL: xxlnor 36, 37, 36
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlor 0, 34, 35
+; CHECK-FISL: xxlnor 34, 34, 35
 ; CHECK-FISL: lis 0, -1
 ; CHECK-FISL: ori 0, 0, 65520
 ; CHECK-FISL: stxvd2x 0, 1, 0
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test14
 ; CHECK-LE: xxlnor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <8 x i16> @test15(<8 x i16> %a, <8 x i16> %b) {
 entry:
   %v = or <8 x i16> %a, %b
   %w = xor <8 x i16> %v, <i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1>
   ret <8 x i16> %w
 
 ; CHECK-REG-LABEL: @test15
 ; CHECK-REG: xxlnor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test15
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlor 36, 36, 37
-; CHECK-FISL: vor 0, 4, 4
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlnor 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlor 0, 34, 35
+; CHECK-FISL: xxlor 36, 0, 0
+; CHECK-FISL: xxlnor 0, 34, 35
+; CHECK-FISL: xxlor 34, 0, 0
 ; CHECK-FISL: lis 0, -1
 ; CHECK-FISL: ori 0, 0, 65520
-; CHECK-FISL: stvx 0, 1, 0
+; CHECK-FISL: stxvd2x 36, 1, 0
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test15
 ; CHECK-LE: xxlnor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <16 x i8> @test16(<16 x i8> %a, <16 x i8> %b) {
 entry:
   %v = or <16 x i8> %a, %b
   %w = xor <16 x i8> %v, <i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1>
   ret <16 x i8> %w
 
 ; CHECK-REG-LABEL: @test16
 ; CHECK-REG: xxlnor 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test16
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlor 36, 36, 37
-; CHECK-FISL: vor 0, 4, 4
-; CHECK-FISL: vor 4, 2, 2
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: xxlnor 36, 36, 37
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: xxlor 0, 34, 35
+; CHECK-FISL: xxlor 36, 0, 0
+; CHECK-FISL: xxlnor 0, 34, 35
+; CHECK-FISL: xxlor 34, 0, 0
 ; CHECK-FISL: lis 0, -1
 ; CHECK-FISL: ori 0, 0, 65520
-; CHECK-FISL: stvx 0, 1, 0
+; CHECK-FISL: stxvd2x 36, 1, 0
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test16
 ; CHECK-LE: xxlnor 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <4 x i32> @test17(<4 x i32> %a, <4 x i32> %b) {
 entry:
   %w = xor <4 x i32> %b, <i32 -1, i32 -1, i32 -1, i32 -1>
   %v = and <4 x i32> %a, %w
   ret <4 x i32> %v
 
 ; CHECK-REG-LABEL: @test17
 ; CHECK-REG: xxlandc 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test17
-; CHECK-FISL: vor 4, 3, 3
-; CHECK-FISL: vor 5, 2, 2
-; CHECK-FISL: vspltisb 2, -1
-; CHECK-FISL: vor 0, 2, 2
-; CHECK-FISL: xxlxor 36, 36, 32
-; CHECK-FISL: xxland 36, 37, 36
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: vspltisb 4, -1
+; CHECK-FISL: xxlxor 35, 35, 36
+; CHECK-FISL: xxland 34, 34, 35
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test17
 ; CHECK-LE: xxlandc 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <8 x i16> @test18(<8 x i16> %a, <8 x i16> %b) {
 entry:
   %w = xor <8 x i16> %b, <i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1, i16 -1>
   %v = and <8 x i16> %a, %w
   ret <8 x i16> %v
 
 ; CHECK-REG-LABEL: @test18
 ; CHECK-REG: xxlandc 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test18
 ; CHECK-FISL: vspltisb 4, -1
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: vor 0, 4, 4
-; CHECK-FISL: xxlxor 37, 37, 32
-; CHECK-FISL: vor 4, 5, 5
-; CHECK-FISL: vor 5, 2, 2
-; CHECK-FISL: vor 0, 3, 3
-; CHECK-FISL: xxlandc 37, 37, 32
-; CHECK-FISL: vor 2, 5, 5
+; CHECK-FISL: xxlxor 36, 35, 36
+; CHECK-FISL: xxlandc 34, 34, 35
 ; CHECK-FISL: lis 0, -1
 ; CHECK-FISL: ori 0, 0, 65520
-; CHECK-FISL: stvx 4, 1, 0
+; CHECK-FISL: stxvd2x 36, 1, 0
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test18
 ; CHECK-LE: xxlandc 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <16 x i8> @test19(<16 x i8> %a, <16 x i8> %b) {
 entry:
   %w = xor <16 x i8> %b, <i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1, i8 -1>
   %v = and <16 x i8> %a, %w
   ret <16 x i8> %v
 
 ; CHECK-REG-LABEL: @test19
 ; CHECK-REG: xxlandc 34, 34, 35
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test19
 ; CHECK-FISL: vspltisb 4, -1
-; CHECK-FISL: vor 5, 3, 3
-; CHECK-FISL: vor 0, 4, 4
-; CHECK-FISL: xxlxor 37, 37, 32
-; CHECK-FISL: vor 4, 5, 5
-; CHECK-FISL: vor 5, 2, 2
-; CHECK-FISL: vor 0, 3, 3
-; CHECK-FISL: xxlandc 37, 37, 32
-; CHECK-FISL: vor 2, 5, 5
+; CHECK-FISL: xxlxor 36, 35, 36
+; CHECK-FISL: xxlandc 34, 34, 35
 ; CHECK-FISL: lis 0, -1
 ; CHECK-FISL: ori 0, 0, 65520
-; CHECK-FISL: stvx 4, 1, 0
+; CHECK-FISL: stxvd2x 36, 1, 0
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test19
 ; CHECK-LE: xxlandc 34, 34, 35
 ; CHECK-LE: blr
 }
 
 define <4 x i32> @test20(<4 x i32> %a, <4 x i32> %b, <4 x i32> %c, <4 x i32> %d) {
 entry:
   %m = icmp eq <4 x i32> %c, %d
   %v = select <4 x i1> %m, <4 x i32> %a, <4 x i32> %b
   ret <4 x i32> %v
 
 ; CHECK-REG-LABEL: @test20
 ; CHECK-REG: vcmpequw {{[0-9]+}}, 4, 5
 ; CHECK-REG: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-REG: blr
 
-; FIXME: The fast-isel code is pretty miserable for this one.
-
 ; CHECK-FISL-LABEL: @test20
-; CHECK-FISL: vor 0, 5, 5
-; CHECK-FISL: vor 1, 4, 4
-; CHECK-FISL: vor 6, 3, 3
-; CHECK-FISL: vor 7, 2, 2
-; CHECK-FISL: vor 2, 1, 1
-; CHECK-FISL: vor 3, 0, 0
-; CHECK-FISL: vcmpequw 2, 2, 3
-; CHECK-FISL: vor 0, 2, 2
-; CHECK-FISL: xxsel 32, 38, 39, 32
-; CHECK-FISL: vor 2, 0, 0
+; CHECK-FISL: vcmpequw {{[0-9]+}}, 4, 5
+; CHECK-FISL: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test20
 ; CHECK-LE: vcmpequw {{[0-9]+}}, 4, 5
 ; CHECK-LE: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-LE: blr
 }
 
 define <4 x float> @test21(<4 x float> %a, <4 x float> %b, <4 x float> %c, <4 x float> %d) {
 entry:
   %m = fcmp oeq <4 x float> %c, %d
   %v = select <4 x i1> %m, <4 x float> %a, <4 x float> %b
   ret <4 x float> %v
 
 ; CHECK-REG-LABEL: @test21
 ; CHECK-REG: xvcmpeqsp [[V1:[0-9]+]], 36, 37
 ; CHECK-REG: xxsel 34, 35, 34, [[V1]]
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test21
-; CHECK-FISL: vor 0, 5, 5
-; CHECK-FISL: vor 1, 4, 4
-; CHECK-FISL: vor 6, 3, 3
-; CHECK-FISL: vor 7, 2, 2
-; CHECK-FISL: xvcmpeqsp 32, 33, 32
-; CHECK-FISL: xxsel 32, 38, 39, 32
-; CHECK-FISL: vor 2, 0, 0
+; CHECK-FISL: xvcmpeqsp [[V1:[0-9]+]], 36, 37
+; CHECK-FISL: xxsel 34, 35, 34, [[V1]]
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test21
 ; CHECK-LE: xvcmpeqsp [[V1:[0-9]+]], 36, 37
 ; CHECK-LE: xxsel 34, 35, 34, [[V1]]
 ; CHECK-LE: blr
 }
 

 ; CHECK-REG-DAG: xxlnor
 ; CHECK-REG-DAG: xxlnor
 ; CHECK-REG-DAG: xxlor
 ; CHECK-REG-DAG: xxlor
 ; CHECK-REG: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test22
-; CHECK-FISL-DAG: xvcmpeqsp {{[0-9]+}}, 33, 32
-; CHECK-FISL-DAG: xvcmpeqsp {{[0-9]+}}, 32, 32
-; CHECK-FISL-DAG: xvcmpeqsp {{[0-9]+}}, 33, 33
+; CHECK-FISL-DAG: xvcmpeqsp {{[0-9]+}}, 37, 37
+; CHECK-FISL-DAG: xvcmpeqsp {{[0-9]+}}, 36, 36
+; CHECK-FISL-DAG: xvcmpeqsp {{[0-9]+}}, 36, 37
 ; CHECK-FISL-DAG: xxlnor
 ; CHECK-FISL-DAG: xxlnor
 ; CHECK-FISL-DAG: xxlor
 ; CHECK-FISL-DAG: xxlor
-; CHECK-FISL: xxsel 0, 38, 39, {{[0-9]+}}
+; CHECK-FISL: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test22
 ; CHECK-LE-DAG: xvcmpeqsp {{[0-9]+}}, 37, 37
 ; CHECK-LE-DAG: xvcmpeqsp {{[0-9]+}}, 36, 36
 ; CHECK-LE-DAG: xvcmpeqsp {{[0-9]+}}, 36, 37
 ; CHECK-LE-DAG: xxlnor
 ; CHECK-LE-DAG: xxlnor

 
 ; CHECK-REG-LABEL: @test23
 ; CHECK-REG: vcmpequh {{[0-9]+}}, 4, 5
 ; CHECK-REG: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test23
 ; CHECK-FISL: vcmpequh 4, 4, 5
-; CHECK-FISL: vor 0, 3, 3
-; CHECK-FISL: vor 1, 2, 2
-; CHECK-FISL: vor 6, 4, 4
-; CHECK-FISL: xxsel 32, 32, 33, 38
-; CHECK-FISL: vor 2, 0, 
+; CHECK-FISL: xxsel 34, 35, 34, 36
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test23
 ; CHECK-LE: vcmpequh {{[0-9]+}}, 4, 5
 ; CHECK-LE: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-LE: blr
 }
 
 define <16 x i8> @test24(<16 x i8> %a, <16 x i8> %b, <16 x i8> %c, <16 x i8> %d) {
 entry:
   %m = icmp eq <16 x i8> %c, %d
   %v = select <16 x i1> %m, <16 x i8> %a, <16 x i8> %b
   ret <16 x i8> %v
 
 ; CHECK-REG-LABEL: @test24
 ; CHECK-REG: vcmpequb {{[0-9]+}}, 4, 5
 ; CHECK-REG: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test24
 ; CHECK-FISL: vcmpequb 4, 4, 5
-; CHECK-FISL: vor 0, 3, 3
-; CHECK-FISL: vor 1, 2, 2
-; CHECK-FISL: vor 6, 4, 4
-; CHECK-FISL: xxsel 32, 32, 33, 38
-; CHECK-FISL: vor 2, 0, 0
+; CHECK-FISL: xxsel 34, 35, 34, 36
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test24
 ; CHECK-LE: vcmpequb {{[0-9]+}}, 4, 5
 ; CHECK-LE: xxsel 34, 35, 34, {{[0-9]+}}
 ; CHECK-LE: blr
 }
 

 
 ; CHECK-REG-LABEL: @test30
 ; CHECK-REG: lxvd2x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test30
 ; CHECK-FISL: lxvd2x 0, 0, 3
 ; CHECK-FISL: xxlor 34, 0, 0
-; CHECK-FISL: vor 3, 2, 2
-; CHECK-FISL: vor 2, 3, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test30
 ; CHECK-LE: lxvd2x [[V1:[0-9]+]], 0, 3
 ; CHECK-LE: xxswapd 34, [[V1]]
 ; CHECK-LE: blr
 }
 
define <4 x float> @test32(<4 x float>* %a) {
   %v = load <4 x float>, <4 x float>* %a, align 16
   ret <4 x float> %v
 
 ; CHECK-REG-LABEL: @test32
 ; CHECK-REG: lxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test32
-; CHECK-FISL: lxvw4x 0, 0, 3
-; CHECK-FISL: xxlor 34, 0, 0
+; CHECK-FISL: lxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test32
 ; CHECK-LE: lxvd2x [[V1:[0-9]+]], 0, 3
 ; CHECK-LE: xxswapd 34, [[V1]]
 ; CHECK-LE: blr
 }
 
 define void @test33(<4 x float>* %a, <4 x float> %b) {
   store <4 x float> %b, <4 x float>* %a, align 16
   ret void
 
 ; CHECK-REG-LABEL: @test33
 ; CHECK-REG: stxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test33
-; CHECK-FISL: vor 3, 2, 2
-; CHECK-FISL: stxvw4x 35, 0, 3
+; CHECK-FISL: stxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test33
 ; CHECK-LE: xxswapd [[V1:[0-9]+]], 34
 ; CHECK-LE: stxvd2x [[V1]], 0, 3
 ; CHECK-LE: blr
 }
 
define void @test33u(<4 x float>* %a, <4 x float> %b) {
   store <4 x float> %b, <4 x float>* %a, align 8
   ret void
 
 ; CHECK-REG-LABEL: @test33u
 ; CHECK-REG: stxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test33u
-; CHECK-FISL: vor 3, 2, 2
-; CHECK-FISL: stxvw4x 35, 0, 3
+; CHECK-FISL: stxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test33u
 ; CHECK-LE: xxswapd [[V1:[0-9]+]], 34
 ; CHECK-LE: stxvd2x [[V1]], 0, 3
 ; CHECK-LE: blr
 }
 
 define <4 x i32> @test34(<4 x i32>* %a) {
   %v = load <4 x i32>, <4 x i32>* %a, align 16
   ret <4 x i32> %v
 
 ; CHECK-REG-LABEL: @test34
 ; CHECK-REG: lxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test34
-; CHECK-FISL: lxvw4x 0, 0, 3
-; CHECK-FISL: xxlor 34, 0, 0
+; CHECK-FISL: lxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test34
 ; CHECK-LE: lxvd2x [[V1:[0-9]+]], 0, 3
 ; CHECK-LE: xxswapd 34, [[V1]]
 ; CHECK-LE: blr
 }
 
 define void @test35(<4 x i32>* %a, <4 x i32> %b) {
   store <4 x i32> %b, <4 x i32>* %a, align 16
   ret void
 
 ; CHECK-REG-LABEL: @test35
 ; CHECK-REG: stxvw4x 34, 0, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test35
-; CHECK-FISL: vor 3, 2, 2
-; CHECK-FISL: stxvw4x 35, 0, 3
+; CHECK-FISL: stxvw4x 34, 0, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test35
 ; CHECK-LE: xxswapd [[V1:[0-9]+]], 34
 ; CHECK-LE: stxvd2x [[V1]], 0, 3
 ; CHECK-LE: blr
 }
 
define <2 x i1> @test65(<2 x i64> %a, <2 x i64> %b) {
   %w = icmp eq <2 x i64> %a, %b
   ret <2 x i1> %w
 
 ; CHECK-REG-LABEL: @test65
 ; CHECK-REG: vcmpequw 2, 2, 3
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test65
-; CHECK-FISL: vor 4, 3, 3
-; CHECK-FISL: vor 5, 2, 2
-; CHECK-FISL: vcmpequw 4, 5, 4
-; CHECK-FISL: vor 2, 4, 4
+; CHECK-FISL: vcmpequw 2, 2, 3
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test65
 ; CHECK-LE: vcmpequd 2, 2, 3
 ; CHECK-LE: blr
 }
 
 define <2 x i1> @test66(<2 x i64> %a, <2 x i64> %b) {
   %w = icmp ne <2 x i64> %a, %b
   ret <2 x i1> %w
 
 ; CHECK-REG-LABEL: @test66
 ; CHECK-REG: vcmpequw {{[0-9]+}}, 2, 3
 ; CHECK-REG: xxlnor 34, {{[0-9]+}}, {{[0-9]+}}
 ; CHECK-REG: blr
 
 ; CHECK-FISL-LABEL: @test66
-; CHECK-FISL: vcmpequw {{[0-9]+}}, 5, 4
-; CHECK-FISL: xxlnor 34, {{[0-9]+}}, {{[0-9]+}}
+; CHECK-FISL: vcmpequw 2, 2, 3
+; CHECK-FISL: xxlnor 34, 34, 34
 ; CHECK-FISL: blr
 
 ; CHECK-LE-LABEL: @test66
 ; CHECK-LE: vcmpequd {{[0-9]+}}, 2, 3
 ; CHECK-LE: xxlnor 34, {{[0-9]+}}, {{[0-9]+}}
 ; CHECK-LE: blr
 }