[NFC][PowerPC] Format and clean the logic for setOperationActions to make it easier to maintain

[steven.zhang]

Summary

We are adding more and more operation actions for different sub-features, and they are scatter over different places without clean logic.

if (hasVSX())
   ...
if (hasP9Vector())
  ...
...
if (hasVSX())
 ...

This NFC patch tried to set operation actions as following order:

By default -> hasAltivec -> hasVSX -> hasP8Altivec() -> hasP8Vector() -> hasP9Altivec() -> hasP9Vector()

So that, some operation actions could be override later and we will have clean picture on the place that setOperationAction should be added.

[qiucf]

Maybe better to add some comments before each section? Like D78132 or X86ISelLowering.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

162

These are not guarded by any predicate, why move them?

302

Better to move them down around init_trampoline since here others are related to float?

343

So, this patch does 'overrides' like:

// Before
if (A) { setLegal(X); } else { setCustom(X); }

// After
setCustom(X); if (A) { setLegal(X); }

Looks correct. But is this good practice?

[steven.zhang]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

162

We have other ATOMIC op later, I move them together. For BITREVERSE, I move them together with BITCAST.

330

Do we miss the case for setOperationAction(ISD::STRICT_FP_ROUND, MVT::f64, Legal); ?

343

For now, usually we have:

if (A) {
   if (B) {
     setLegal(X);
   } else {
      ...
   }
}

I am trying to flat them and move the same operation together to make it more clear. Regarding to your example, I have no good answer and that's something we can discuss. Personally, I think it makes sense as it means, by default, it is custom, if feature A enabled, it is legal. And we will have case like :

if (A) { setLegal(X); } else { setCustom(X); }
if (B) { setExpand(X);} else { setCustom(X); }

Now it is:

setCustom(X)
if (A)
   setLegal(X)
if (B)
   setExpand(X);

[qiucf]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

330

Looks yes. It's unexpectedly guarded by P9Vector predicate. I'll check in detail and fix it later.

[steven.zhang]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

330

Such kind of potential issue can be easily found with this re-work.

[qiucf]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

330

Oh, this is expected: setOperationAction(ISD::FP_ROUND, MVT::f64, Legal) sets fptrunc * to double as legal. But that's illegal on Power8 or earlier subtargets. Anyway, its operation settings are redundant.

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

 // FIXME: Remove this once the bug has been fixed!
 extern cl::opt<bool> ANDIGlueBug;
 
 PPCTargetLowering::PPCTargetLowering(const PPCTargetMachine &TM,
                                      const PPCSubtarget &STI)
     : TargetLowering(TM), Subtarget(STI) {
   // On PPC32/64, arguments smaller than 4/8 bytes are extended, so all
   // arguments are at least 4/8 bytes aligned.
-  bool isPPC64 = Subtarget.isPPC64();
-  setMinStackArgumentAlignment(isPPC64 ? Align(8) : Align(4));
+  setMinStackArgumentAlignment(Subtarget.isPPC64() ? Align(8) : Align(4));
 
   // Set up the register classes.
   addRegisterClass(MVT::i32, &PPC::GPRCRegClass);
-  if (!useSoftFloat()) {
-    if (hasSPE()) {
-      addRegisterClass(MVT::f32, &PPC::GPRCRegClass);
-      addRegisterClass(MVT::f64, &PPC::SPERCRegClass);
-    } else {
-      addRegisterClass(MVT::f32, &PPC::F4RCRegClass);
-      addRegisterClass(MVT::f64, &PPC::F8RCRegClass);
-    }
-  }
-
-  // Match BITREVERSE to customized fast code sequence in the td file.
-  setOperationAction(ISD::BITREVERSE, MVT::i32, Legal);
-  setOperationAction(ISD::BITREVERSE, MVT::i64, Legal);
-
-  // Sub-word ATOMIC_CMP_SWAP need to ensure that the input is zero-extended.
-  setOperationAction(ISD::ATOMIC_CMP_SWAP, MVT::i32, Custom);
 
   // PowerPC has an i16 but no i8 (or i1) SEXTLOAD.
   for (MVT VT : MVT::integer_valuetypes()) {
     setLoadExtAction(ISD::SEXTLOAD, VT, MVT::i1, Promote);
     setLoadExtAction(ISD::SEXTLOAD, VT, MVT::i8, Expand);
   }
 
-  if (Subtarget.isISA3_0()) {
-    setLoadExtAction(ISD::EXTLOAD, MVT::f64, MVT::f16, Legal);
-    setLoadExtAction(ISD::EXTLOAD, MVT::f32, MVT::f16, Legal);
-    setTruncStoreAction(MVT::f64, MVT::f16, Legal);
-    setTruncStoreAction(MVT::f32, MVT::f16, Legal);
-  } else {
-    // No extending loads from f16 or HW conversions back and forth.
-    setLoadExtAction(ISD::EXTLOAD, MVT::f64, MVT::f16, Expand);
-    setOperationAction(ISD::FP16_TO_FP, MVT::f64, Expand);
-    setOperationAction(ISD::FP_TO_FP16, MVT::f64, Expand);
-    setLoadExtAction(ISD::EXTLOAD, MVT::f32, MVT::f16, Expand);
-    setOperationAction(ISD::FP16_TO_FP, MVT::f32, Expand);
-    setOperationAction(ISD::FP_TO_FP16, MVT::f32, Expand);
-    setTruncStoreAction(MVT::f64, MVT::f16, Expand);
-    setTruncStoreAction(MVT::f32, MVT::f16, Expand);
-  }
-
-  setTruncStoreAction(MVT::f64, MVT::f32, Expand);
-
   // PowerPC has pre-inc load and store's.
-  setIndexedLoadAction(ISD::PRE_INC, MVT::i1, Legal);
-  setIndexedLoadAction(ISD::PRE_INC, MVT::i8, Legal);
-  setIndexedLoadAction(ISD::PRE_INC, MVT::i16, Legal);
-  setIndexedLoadAction(ISD::PRE_INC, MVT::i32, Legal);
-  setIndexedLoadAction(ISD::PRE_INC, MVT::i64, Legal);
-  setIndexedStoreAction(ISD::PRE_INC, MVT::i1, Legal);
-  setIndexedStoreAction(ISD::PRE_INC, MVT::i8, Legal);
-  setIndexedStoreAction(ISD::PRE_INC, MVT::i16, Legal);
-  setIndexedStoreAction(ISD::PRE_INC, MVT::i32, Legal);
-  setIndexedStoreAction(ISD::PRE_INC, MVT::i64, Legal);
-  if (!Subtarget.hasSPE()) {
-    setIndexedLoadAction(ISD::PRE_INC, MVT::f32, Legal);
-    setIndexedLoadAction(ISD::PRE_INC, MVT::f64, Legal);
-    setIndexedStoreAction(ISD::PRE_INC, MVT::f32, Legal);
-    setIndexedStoreAction(ISD::PRE_INC, MVT::f64, Legal);
+  for (MVT VT :
+       {MVT::i1, MVT::i8, MVT::i16, MVT::i32, MVT::i64, MVT::f32, MVT::f64}) {
+    setIndexedLoadAction(ISD::PRE_INC,  VT, Legal);
+    setIndexedStoreAction(ISD::PRE_INC, VT, Legal);
   }
 
+  for (MVT VT : {MVT::i32, MVT::i64}) {
     // PowerPC uses ADDC/ADDE/SUBC/SUBE to propagate carry.
-  const MVT ScalarIntVTs[] = { MVT::i32, MVT::i64 };
-  for (MVT VT : ScalarIntVTs) {
     setOperationAction(ISD::ADDC, VT, Legal);
     setOperationAction(ISD::ADDE, VT, Legal);
     setOperationAction(ISD::SUBC, VT, Legal);
     setOperationAction(ISD::SUBE, VT, Legal);
-  }
-
-  if (Subtarget.useCRBits()) {
-    setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::i1, Expand);
-
-    if (isPPC64 || Subtarget.hasFPCVT()) {
-      setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i1, Promote);
-      AddPromotedToType(ISD::STRICT_SINT_TO_FP, MVT::i1,
-                        isPPC64 ? MVT::i64 : MVT::i32);
-      setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i1, Promote);
-      AddPromotedToType(ISD::STRICT_UINT_TO_FP, MVT::i1,
-                        isPPC64 ? MVT::i64 : MVT::i32);
-
-      setOperationAction(ISD::SINT_TO_FP, MVT::i1, Promote);
-      AddPromotedToType (ISD::SINT_TO_FP, MVT::i1,
-                         isPPC64 ? MVT::i64 : MVT::i32);
-      setOperationAction(ISD::UINT_TO_FP, MVT::i1, Promote);
-      AddPromotedToType(ISD::UINT_TO_FP, MVT::i1,
-                        isPPC64 ? MVT::i64 : MVT::i32);
-
-      setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::i1, Promote);
-      AddPromotedToType(ISD::STRICT_FP_TO_SINT, MVT::i1,
-                        isPPC64 ? MVT::i64 : MVT::i32);
-      setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i1, Promote);
-      AddPromotedToType(ISD::STRICT_FP_TO_UINT, MVT::i1,
-                        isPPC64 ? MVT::i64 : MVT::i32);
-
-      setOperationAction(ISD::FP_TO_SINT, MVT::i1, Promote);
-      AddPromotedToType(ISD::FP_TO_SINT, MVT::i1,
-                        isPPC64 ? MVT::i64 : MVT::i32);
-      setOperationAction(ISD::FP_TO_UINT, MVT::i1, Promote);
-      AddPromotedToType(ISD::FP_TO_UINT, MVT::i1,
-                        isPPC64 ? MVT::i64 : MVT::i32);
-    } else {
-      setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i1, Custom);
-      setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i1, Custom);
-      setOperationAction(ISD::SINT_TO_FP, MVT::i1, Custom);
-      setOperationAction(ISD::UINT_TO_FP, MVT::i1, Custom);
-    }
-
-    // PowerPC does not support direct load/store of condition registers.
-    setOperationAction(ISD::LOAD, MVT::i1, Custom);
-    setOperationAction(ISD::STORE, MVT::i1, Custom);
 
-    // FIXME: Remove this once the ANDI glue bug is fixed:
-    if (ANDIGlueBug)
-      setOperationAction(ISD::TRUNCATE, MVT::i1, Custom);
-
-    for (MVT VT : MVT::integer_valuetypes()) {
-      setLoadExtAction(ISD::SEXTLOAD, VT, MVT::i1, Promote);
-      setLoadExtAction(ISD::ZEXTLOAD, VT, MVT::i1, Promote);
-      setTruncStoreAction(VT, MVT::i1, Expand);
-    }
+    // Match BITREVERSE to customized fast code sequence in the td file.
+    setOperationAction(ISD::BITREVERSE, VT, Legal);
 
-    addRegisterClass(MVT::i1, &PPC::CRBITRCRegClass);
-  }
+    setOperationAction(ISD::BITCAST, VT, Expand);
 
-  // Expand ppcf128 to i32 by hand for the benefit of llvm-gcc bootstrap on
-  // PPC (the libcall is not available).
-  setOperationAction(ISD::FP_TO_SINT, MVT::ppcf128, Custom);
-  setOperationAction(ISD::FP_TO_UINT, MVT::ppcf128, Custom);
-  setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::ppcf128, Custom);
-  setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::ppcf128, Custom);
+    setOperationAction(
+        ISD::CTPOP, VT,
+        Subtarget.hasPOPCNTD() == PPCSubtarget::POPCNTD_Fast ? Legal : Expand);
+    setOperationAction(ISD::CTTZ,  VT, Expand);
 
-  // We do not currently implement these libm ops for PowerPC.
-  setOperationAction(ISD::FFLOOR, MVT::ppcf128, Expand);
-  setOperationAction(ISD::FCEIL,  MVT::ppcf128, Expand);
-  setOperationAction(ISD::FTRUNC, MVT::ppcf128, Expand);
-  setOperationAction(ISD::FRINT,  MVT::ppcf128, Expand);
-  setOperationAction(ISD::FNEARBYINT, MVT::ppcf128, Expand);
-  setOperationAction(ISD::FREM, MVT::ppcf128, Expand);
+    // Don't use SMUL_LOHI/UMUL_LOHI or SDIVREM/UDIVREM to lower SREM/UREM.
+    setOperationAction(ISD::UMUL_LOHI, VT, Expand);
+    setOperationAction(ISD::SMUL_LOHI, VT, Expand);
+    setOperationAction(ISD::UDIVREM,   VT, Expand);
+    setOperationAction(ISD::SDIVREM,   VT, Expand);
 
     // PowerPC has no SREM/UREM instructions unless we are on P9
     // On P9 we may use a hardware instruction to compute the remainder.
     // When the result of both the remainder and the division is required it is
     // more efficient to compute the remainder from the result of the division
     // rather than use the remainder instruction. The instructions are legalized
-  // directly because the DivRemPairsPass performs the transformation at the IR
-  // level.
-  if (Subtarget.isISA3_0()) {
-    setOperationAction(ISD::SREM, MVT::i32, Legal);
-    setOperationAction(ISD::UREM, MVT::i32, Legal);
-    setOperationAction(ISD::SREM, MVT::i64, Legal);
-    setOperationAction(ISD::UREM, MVT::i64, Legal);
-  } else {
-    setOperationAction(ISD::SREM, MVT::i32, Expand);
-    setOperationAction(ISD::UREM, MVT::i32, Expand);
-    setOperationAction(ISD::SREM, MVT::i64, Expand);
-    setOperationAction(ISD::UREM, MVT::i64, Expand);
-  }
+    // directly because the DivRemPairsPass performs the transformation at the
+    // IR level.
+    setOperationAction(ISD::SREM, VT, Expand);
+    setOperationAction(ISD::UREM, VT, Expand);
 
-  // Don't use SMUL_LOHI/UMUL_LOHI or SDIVREM/UDIVREM to lower SREM/UREM.
-  setOperationAction(ISD::UMUL_LOHI, MVT::i32, Expand);
-  setOperationAction(ISD::SMUL_LOHI, MVT::i32, Expand);
-  setOperationAction(ISD::UMUL_LOHI, MVT::i64, Expand);
-  setOperationAction(ISD::SMUL_LOHI, MVT::i64, Expand);
-  setOperationAction(ISD::UDIVREM, MVT::i32, Expand);
-  setOperationAction(ISD::SDIVREM, MVT::i32, Expand);
-  setOperationAction(ISD::UDIVREM, MVT::i64, Expand);
-  setOperationAction(ISD::SDIVREM, MVT::i64, Expand);
+    // PowerPC does not have ROTR
+    setOperationAction(ISD::ROTR, VT, Expand);
 
-  // Handle constrained floating-point operations of scalar.
-  // TODO: Handle SPE specific operation.
-  setOperationAction(ISD::STRICT_FADD, MVT::f32, Legal);
-  setOperationAction(ISD::STRICT_FSUB, MVT::f32, Legal);
-  setOperationAction(ISD::STRICT_FMUL, MVT::f32, Legal);
-  setOperationAction(ISD::STRICT_FDIV, MVT::f32, Legal);
-  setOperationAction(ISD::STRICT_FMA, MVT::f32, Legal);
-  setOperationAction(ISD::STRICT_FP_ROUND, MVT::f32, Legal);
+    // PowerPC does not have BSWAP, but we can use vector BSWAP instruction
+    // xxbrd to speed up scalar BSWAP64. CTPOP or CTTZ were introduced in P8/P9
+    // respectively
+    setOperationAction(ISD::BSWAP, VT, Expand);
 
-  setOperationAction(ISD::STRICT_FADD, MVT::f64, Legal);
-  setOperationAction(ISD::STRICT_FSUB, MVT::f64, Legal);
-  setOperationAction(ISD::STRICT_FMUL, MVT::f64, Legal);
-  setOperationAction(ISD::STRICT_FDIV, MVT::f64, Legal);
-  setOperationAction(ISD::STRICT_FMA, MVT::f64, Legal);
-  if (Subtarget.hasVSX()) {
-    setOperationAction(ISD::STRICT_FRINT, MVT::f32, Legal);
-    setOperationAction(ISD::STRICT_FRINT, MVT::f64, Legal);
-  }
+    setOperationAction(ISD::SELECT, VT, Expand);
 
-  if (Subtarget.hasFSQRT()) {
-    setOperationAction(ISD::STRICT_FSQRT, MVT::f32, Legal);
-    setOperationAction(ISD::STRICT_FSQRT, MVT::f64, Legal);
-  }
+    // We want to legalize GlobalAddress and ConstantPool nodes into the
+    // appropriate instructions to materialize the address.
+    setOperationAction(ISD::GlobalAddress,    VT, Custom);
+    setOperationAction(ISD::GlobalTLSAddress, VT, Custom);
+    setOperationAction(ISD::BlockAddress,     VT, Custom);
+    setOperationAction(ISD::ConstantPool,     VT, Custom);
+    setOperationAction(ISD::JumpTable,        VT, Custom);
 
-  if (Subtarget.hasFPRND()) {
-    setOperationAction(ISD::STRICT_FFLOOR, MVT::f32, Legal);
-    setOperationAction(ISD::STRICT_FCEIL,  MVT::f32, Legal);
-    setOperationAction(ISD::STRICT_FTRUNC, MVT::f32, Legal);
-    setOperationAction(ISD::STRICT_FROUND, MVT::f32, Legal);
-
-    setOperationAction(ISD::STRICT_FFLOOR, MVT::f64, Legal);
-    setOperationAction(ISD::STRICT_FCEIL,  MVT::f64, Legal);
-    setOperationAction(ISD::STRICT_FTRUNC, MVT::f64, Legal);
-    setOperationAction(ISD::STRICT_FROUND, MVT::f64, Legal);
+    setOperationAction(ISD::DYNAMIC_STACKALLOC,      VT, Custom);
+    setOperationAction(ISD::GET_DYNAMIC_AREA_OFFSET, VT, Custom);
+
+    setOperationAction(ISD::EH_DWARF_CFA, VT, Custom);
   }
 
+  for (MVT VT :{MVT::f32, MVT::f64}) {
     // We don't support sin/cos/sqrt/fmod/pow
-  setOperationAction(ISD::FSIN , MVT::f64, Expand);
-  setOperationAction(ISD::FCOS , MVT::f64, Expand);
-  setOperationAction(ISD::FSINCOS, MVT::f64, Expand);
-  setOperationAction(ISD::FREM , MVT::f64, Expand);
-  setOperationAction(ISD::FPOW , MVT::f64, Expand);
-  setOperationAction(ISD::FSIN , MVT::f32, Expand);
-  setOperationAction(ISD::FCOS , MVT::f32, Expand);
-  setOperationAction(ISD::FSINCOS, MVT::f32, Expand);
-  setOperationAction(ISD::FREM , MVT::f32, Expand);
-  setOperationAction(ISD::FPOW , MVT::f32, Expand);
-  if (Subtarget.hasSPE()) {
-    setOperationAction(ISD::FMA  , MVT::f64, Expand);
-    setOperationAction(ISD::FMA  , MVT::f32, Expand);
-  } else {
-    setOperationAction(ISD::FMA  , MVT::f64, Legal);
-    setOperationAction(ISD::FMA  , MVT::f32, Legal);
-  }
+    setOperationAction(ISD::FSIN,      VT, Expand);
+    setOperationAction(ISD::FCOS,      VT, Expand);
+    setOperationAction(ISD::FREM,      VT, Expand);
+    setOperationAction(ISD::FPOW,      VT, Expand);
+    setOperationAction(ISD::FSQRT,     VT, Expand);
+    setOperationAction(ISD::FSINCOS,   VT, Expand);
+    setOperationAction(ISD::FCOPYSIGN, VT, Expand);
 
-  if (Subtarget.hasSPE())
-    setLoadExtAction(ISD::EXTLOAD, MVT::f64, MVT::f32, Expand);
+    setOperationAction(ISD::FMA, VT, Legal);
 
-  setOperationAction(ISD::FLT_ROUNDS_, MVT::i32, Custom);
+    setOperationAction(ISD::FP16_TO_FP, VT, Expand);
+    setOperationAction(ISD::FP_TO_FP16, VT, Expand);
 
-  // If we're enabling GP optimizations, use hardware square root
-  if (!Subtarget.hasFSQRT() &&
-      !(TM.Options.UnsafeFPMath && Subtarget.hasFRSQRTE() &&
-        Subtarget.hasFRE()))
-    setOperationAction(ISD::FSQRT, MVT::f64, Expand);
-
-  if (!Subtarget.hasFSQRT() &&
-      !(TM.Options.UnsafeFPMath && Subtarget.hasFRSQRTES() &&
-        Subtarget.hasFRES()))
-    setOperationAction(ISD::FSQRT, MVT::f32, Expand);
-
-  if (Subtarget.hasFCPSGN()) {
-    setOperationAction(ISD::FCOPYSIGN, MVT::f64, Legal);
-    setOperationAction(ISD::FCOPYSIGN, MVT::f32, Legal);
-  } else {
-    setOperationAction(ISD::FCOPYSIGN, MVT::f64, Expand);
-    setOperationAction(ISD::FCOPYSIGN, MVT::f32, Expand);
-  }
+    // Handle constrained floating-point operations of scalar.
+    // TODO: Handle SPE specific operation.
+    setOperationAction(ISD::STRICT_FADD,      VT, Legal);
+    setOperationAction(ISD::STRICT_FSUB,      VT, Legal);
+    setOperationAction(ISD::STRICT_FMUL,      VT, Legal);
+    setOperationAction(ISD::STRICT_FDIV,      VT, Legal);
+    setOperationAction(ISD::STRICT_FMA,       VT, Legal);
+    setOperationAction(ISD::STRICT_FP_EXTEND, VT, Legal);
+
+    setOperationAction(ISD::BITCAST, VT, Expand);
 
     if (Subtarget.hasFPRND()) {
-    setOperationAction(ISD::FFLOOR, MVT::f64, Legal);
-    setOperationAction(ISD::FCEIL,  MVT::f64, Legal);
-    setOperationAction(ISD::FTRUNC, MVT::f64, Legal);
-    setOperationAction(ISD::FROUND, MVT::f64, Legal);
-
-    setOperationAction(ISD::FFLOOR, MVT::f32, Legal);
-    setOperationAction(ISD::FCEIL,  MVT::f32, Legal);
-    setOperationAction(ISD::FTRUNC, MVT::f32, Legal);
-    setOperationAction(ISD::FROUND, MVT::f32, Legal);
-  }
-
-  // PowerPC does not have BSWAP, but we can use vector BSWAP instruction xxbrd
-  // to speed up scalar BSWAP64.
-  // CTPOP or CTTZ were introduced in P8/P9 respectively
-  setOperationAction(ISD::BSWAP, MVT::i32  , Expand);
-  if (Subtarget.hasP9Vector())
-    setOperationAction(ISD::BSWAP, MVT::i64  , Custom);
-  else
-    setOperationAction(ISD::BSWAP, MVT::i64  , Expand);
-  if (Subtarget.isISA3_0()) {
-    setOperationAction(ISD::CTTZ , MVT::i32  , Legal);
-    setOperationAction(ISD::CTTZ , MVT::i64  , Legal);
-  } else {
-    setOperationAction(ISD::CTTZ , MVT::i32  , Expand);
-    setOperationAction(ISD::CTTZ , MVT::i64  , Expand);
+      setOperationAction(ISD::STRICT_FFLOOR, VT, Legal);
+      setOperationAction(ISD::STRICT_FCEIL,  VT, Legal);
+      setOperationAction(ISD::STRICT_FTRUNC, VT, Legal);
+      setOperationAction(ISD::STRICT_FROUND, VT, Legal);
+
+      setOperationAction(ISD::FFLOOR, VT, Legal);
+      setOperationAction(ISD::FCEIL,  VT, Legal);
+      setOperationAction(ISD::FTRUNC, VT, Legal);
+      setOperationAction(ISD::FROUND, VT, Legal);
     }
 
-  if (Subtarget.hasPOPCNTD() == PPCSubtarget::POPCNTD_Fast) {
-    setOperationAction(ISD::CTPOP, MVT::i32  , Legal);
-    setOperationAction(ISD::CTPOP, MVT::i64  , Legal);
-  } else {
-    setOperationAction(ISD::CTPOP, MVT::i32  , Expand);
-    setOperationAction(ISD::CTPOP, MVT::i64  , Expand);
+    if (Subtarget.hasFSQRT()) {
+      setOperationAction(ISD::FSQRT,        VT, Legal);
+      setOperationAction(ISD::STRICT_FSQRT, VT, Legal);
     }
 
-  // PowerPC does not have ROTR
-  setOperationAction(ISD::ROTR, MVT::i32   , Expand);
-  setOperationAction(ISD::ROTR, MVT::i64   , Expand);
+    if (Subtarget.hasFCPSGN())
+      setOperationAction(ISD::FCOPYSIGN, VT, Legal);
 
-  if (!Subtarget.useCRBits()) {
-    // PowerPC does not have Select
-    setOperationAction(ISD::SELECT, MVT::i32, Expand);
-    setOperationAction(ISD::SELECT, MVT::i64, Expand);
-    setOperationAction(ISD::SELECT, MVT::f32, Expand);
-    setOperationAction(ISD::SELECT, MVT::f64, Expand);
+    if (Subtarget.hasFPU()) {
+      setOperationAction(ISD::STRICT_FSETCC,  VT, Legal);
+      setOperationAction(ISD::STRICT_FSETCCS, VT, Legal);
     }
 
     // PowerPC wants to turn select_cc of FP into fsel when possible.
-  setOperationAction(ISD::SELECT_CC, MVT::f32, Custom);
-  setOperationAction(ISD::SELECT_CC, MVT::f64, Custom);
-
-  // PowerPC wants to optimize integer setcc a bit
-  if (!Subtarget.useCRBits())
-    setOperationAction(ISD::SETCC, MVT::i32, Custom);
+    setOperationAction(ISD::SELECT_CC, VT, Custom);
+    // PowerPC does not have Select
+    setOperationAction(ISD::SELECT, VT, Expand);
 
-  if (Subtarget.hasFPU()) {
-    setOperationAction(ISD::STRICT_FSETCC, MVT::f32, Legal);
-    setOperationAction(ISD::STRICT_FSETCC, MVT::f64, Legal);
-    setOperationAction(ISD::STRICT_FSETCC, MVT::f128, Legal);
+    setCondCodeAction(ISD::SETULT, VT, Expand);
+    setCondCodeAction(ISD::SETUGT, VT, Expand);
+    setCondCodeAction(ISD::SETUEQ, VT, Expand);
+    setCondCodeAction(ISD::SETOGE, VT, Expand);
+    setCondCodeAction(ISD::SETOLE, VT, Expand);
+    setCondCodeAction(ISD::SETONE, VT, Expand);
 
-    setOperationAction(ISD::STRICT_FSETCCS, MVT::f32, Legal);
-    setOperationAction(ISD::STRICT_FSETCCS, MVT::f64, Legal);
-    setOperationAction(ISD::STRICT_FSETCCS, MVT::f128, Legal);
+    // No extending loads from f16 or HW conversions back and forth.
+    setLoadExtAction(ISD::EXTLOAD, VT, MVT::f16, Expand);
+    setTruncStoreAction(VT, MVT::f16, Expand);
   }
 
-  // PowerPC does not have BRCOND which requires SetCC
-  if (!Subtarget.useCRBits())
-    setOperationAction(ISD::BRCOND, MVT::Other, Expand);
+  // If we're enabling GP optimizations, use hardware square root
+  if (TM.Options.UnsafeFPMath && Subtarget.hasFRSQRTE() && Subtarget.hasFRE())
+    setOperationAction(ISD::FSQRT, MVT::f64, Legal);
 
-  setOperationAction(ISD::BR_JT,  MVT::Other, Expand);
+  if (TM.Options.UnsafeFPMath && Subtarget.hasFRSQRTES() && Subtarget.hasFRES())
+    setOperationAction(ISD::FSQRT, MVT::f32, Legal);
 
-  if (Subtarget.hasSPE()) {
-    // SPE has built-in conversions
-    setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::i32, Legal);
-    setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i32, Legal);
-    setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i32, Legal);
-    setOperationAction(ISD::FP_TO_SINT, MVT::i32, Legal);
-    setOperationAction(ISD::SINT_TO_FP, MVT::i32, Legal);
-    setOperationAction(ISD::UINT_TO_FP, MVT::i32, Legal);
-  } else {
-    // PowerPC turns FP_TO_SINT into FCTIWZ and some load/stores.
-    setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::i32, Custom);
-    setOperationAction(ISD::FP_TO_SINT, MVT::i32, Custom);
+  // We want to custom lower some of our intrinsics.
+  setOperationAction(ISD::INTRINSIC_WO_CHAIN, MVT::Other, Custom);
 
-    // PowerPC does not have [U|S]INT_TO_FP
-    setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i32, Expand);
-    setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i32, Expand);
-    setOperationAction(ISD::SINT_TO_FP, MVT::i32, Expand);
-    setOperationAction(ISD::UINT_TO_FP, MVT::i32, Expand);
-  }
+  // To handle counter-based loop conditions.
+  setOperationAction(ISD::INTRINSIC_W_CHAIN, MVT::i1, Custom);
+
+  setOperationAction(ISD::INTRINSIC_VOID, MVT::i8,    Custom);
+  setOperationAction(ISD::INTRINSIC_VOID, MVT::i16,   Custom);
+  setOperationAction(ISD::INTRINSIC_VOID, MVT::i32,   Custom);
+  setOperationAction(ISD::INTRINSIC_VOID, MVT::Other, Custom);
+
+  setOperationAction(ISD::FSHL, MVT::i32, Custom);
+  setOperationAction(ISD::FSHR, MVT::i32, Custom);
+
+  setTruncStoreAction(MVT::f64, MVT::f32, Expand);
+
+  setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::i32, Custom);
+  setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i32, Expand);
+  // PowerPC turns FP_TO_SINT into FCTIWZ and some load/stores.
+  setOperationAction(ISD::FP_TO_SINT, MVT::i32, Custom);
+  setOperationAction(ISD::FP_TO_UINT, MVT::i32, Expand);
+  // PowerPC does not have [U|S]INT_TO_FP
 
-  if (Subtarget.hasDirectMove() && isPPC64) {
-    setOperationAction(ISD::BITCAST, MVT::f32, Legal);
-    setOperationAction(ISD::BITCAST, MVT::i32, Legal);
-    setOperationAction(ISD::BITCAST, MVT::i64, Legal);
-    setOperationAction(ISD::BITCAST, MVT::f64, Legal);
-    if (TM.Options.UnsafeFPMath) {
-      setOperationAction(ISD::LRINT, MVT::f64, Legal);
-      setOperationAction(ISD::LRINT, MVT::f32, Legal);
-      setOperationAction(ISD::LLRINT, MVT::f64, Legal);
-      setOperationAction(ISD::LLRINT, MVT::f32, Legal);
-      setOperationAction(ISD::LROUND, MVT::f64, Legal);
-      setOperationAction(ISD::LROUND, MVT::f32, Legal);
-      setOperationAction(ISD::LLROUND, MVT::f64, Legal);
-      setOperationAction(ISD::LLROUND, MVT::f32, Legal);
-    }
-  } else {
-    setOperationAction(ISD::BITCAST, MVT::f32, Expand);
-    setOperationAction(ISD::BITCAST, MVT::i32, Expand);
-    setOperationAction(ISD::BITCAST, MVT::i64, Expand);
-    setOperationAction(ISD::BITCAST, MVT::f64, Expand);
+  // Expand ppcf128 to i32 by hand for the benefit of llvm-gcc bootstrap on
+  // PPC (the libcall is not available).
+  setOperationAction(ISD::FP_TO_SINT,        MVT::ppcf128, Custom);
+  setOperationAction(ISD::FP_TO_UINT,        MVT::ppcf128, Custom);
+  setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::ppcf128, Custom);
+  setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::ppcf128, Custom);
+
+  setOperationAction(ISD::STRICT_FP_ROUND,   MVT::f32, Legal);
+
+  if (Subtarget.hasFPU()) {
+    setOperationAction(ISD::STRICT_FSETCC,  MVT::f128, Legal);
+    setOperationAction(ISD::STRICT_FSETCCS, MVT::f128, Legal);
+
+  // With the instructions enabled under FPCVT, we can do everything.
+  if (Subtarget.hasFPCVT()) {
+    setOperationAction(ISD::FP_TO_SINT,        MVT::i32, Custom);
+    setOperationAction(ISD::FP_TO_UINT,        MVT::i32, Custom);
+    setOperationAction(ISD::SINT_TO_FP,        MVT::i32, Custom);
+    setOperationAction(ISD::UINT_TO_FP,        MVT::i32, Custom);
+    setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::i32, Custom);
+    setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i32, Custom);
+    setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i32, Custom);
+    setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i32, Custom);
   }
 
+  // We do not currently implement these libm ops for PowerPC.
+  setOperationAction(ISD::FFLOOR,     MVT::ppcf128, Expand);
+  setOperationAction(ISD::FCEIL,      MVT::ppcf128, Expand);
+  setOperationAction(ISD::FTRUNC,     MVT::ppcf128, Expand);
+  setOperationAction(ISD::FRINT,      MVT::ppcf128, Expand);
+  setOperationAction(ISD::FNEARBYINT, MVT::ppcf128, Expand);
+  setOperationAction(ISD::FREM,       MVT::ppcf128, Expand);
+
+  // 32-bit PowerPC wants to expand i64 shifts itself.
+  setOperationAction(ISD::SHL_PARTS, MVT::i32, Custom);
+  setOperationAction(ISD::SRA_PARTS, MVT::i32, Custom);
+  setOperationAction(ISD::SRL_PARTS, MVT::i32, Custom);
+
+  // PowerPC wants to optimize integer setcc a bit
+  setOperationAction(ISD::SETCC, MVT::i32, Custom);
+
+  // PowerPC does not have BRCOND which requires SetCC
+  setOperationAction(ISD::BRCOND, MVT::Other, Expand);
+  setOperationAction(ISD::BR_JT,  MVT::Other, Expand);
+
   // We cannot sextinreg(i1).  Expand to shifts.
   setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::i1, Expand);
 
+  setOperationAction(ISD::FLT_ROUNDS_,      MVT::i32, Custom);
+  setOperationAction(ISD::READCYCLECOUNTER, MVT::i64, Custom);
+
   // NOTE: EH_SJLJ_SETJMP/_LONGJMP supported here is NOT intended to support
   // SjLj exception handling but a light-weight setjmp/longjmp replacement to
   // support continuation, user-level threading, and etc.. As a result, no
   // other SjLj exception interfaces are implemented and please don't build
   // your own exception handling based on them.
   // LLVM/Clang supports zero-cost DWARF exception handling.
-  setOperationAction(ISD::EH_SJLJ_SETJMP, MVT::i32, Custom);
+  setOperationAction(ISD::EH_SJLJ_SETJMP,  MVT::i32, Custom);
   setOperationAction(ISD::EH_SJLJ_LONGJMP, MVT::Other, Custom);
 
-  // We want to legalize GlobalAddress and ConstantPool nodes into the
-  // appropriate instructions to materialize the address.
-  setOperationAction(ISD::GlobalAddress, MVT::i32, Custom);
-  setOperationAction(ISD::GlobalTLSAddress, MVT::i32, Custom);
-  setOperationAction(ISD::BlockAddress,  MVT::i32, Custom);
-  setOperationAction(ISD::ConstantPool,  MVT::i32, Custom);
-  setOperationAction(ISD::JumpTable,     MVT::i32, Custom);
-  setOperationAction(ISD::GlobalAddress, MVT::i64, Custom);
-  setOperationAction(ISD::GlobalTLSAddress, MVT::i64, Custom);
-  setOperationAction(ISD::BlockAddress,  MVT::i64, Custom);
-  setOperationAction(ISD::ConstantPool,  MVT::i64, Custom);
-  setOperationAction(ISD::JumpTable,     MVT::i64, Custom);
-
   // TRAP is legal.
   setOperationAction(ISD::TRAP, MVT::Other, Legal);
 
   // TRAMPOLINE is custom lowered.
-  setOperationAction(ISD::INIT_TRAMPOLINE, MVT::Other, Custom);
+  setOperationAction(ISD::INIT_TRAMPOLINE,   MVT::Other, Custom);
   setOperationAction(ISD::ADJUST_TRAMPOLINE, MVT::Other, Custom);
 
   // VASTART needs to be custom lowered to use the VarArgsFrameIndex
-  setOperationAction(ISD::VASTART           , MVT::Other, Custom);
-
-  if (Subtarget.is64BitELFABI()) {
-    // VAARG always uses double-word chunks, so promote anything smaller.
-    setOperationAction(ISD::VAARG, MVT::i1, Promote);
-    AddPromotedToType(ISD::VAARG, MVT::i1, MVT::i64);
-    setOperationAction(ISD::VAARG, MVT::i8, Promote);
-    AddPromotedToType(ISD::VAARG, MVT::i8, MVT::i64);
-    setOperationAction(ISD::VAARG, MVT::i16, Promote);
-    AddPromotedToType(ISD::VAARG, MVT::i16, MVT::i64);
-    setOperationAction(ISD::VAARG, MVT::i32, Promote);
-    AddPromotedToType(ISD::VAARG, MVT::i32, MVT::i64);
-    setOperationAction(ISD::VAARG, MVT::Other, Expand);
-  } else if (Subtarget.is32BitELFABI()) {
-    // VAARG is custom lowered with the 32-bit SVR4 ABI.
-    setOperationAction(ISD::VAARG, MVT::Other, Custom);
-    setOperationAction(ISD::VAARG, MVT::i64, Custom);
-  } else
-    setOperationAction(ISD::VAARG, MVT::Other, Expand);
-
-  // VACOPY is custom lowered with the 32-bit SVR4 ABI.
-  if (Subtarget.is32BitELFABI())
-    setOperationAction(ISD::VACOPY            , MVT::Other, Custom);
-  else
-    setOperationAction(ISD::VACOPY            , MVT::Other, Expand);
-
-  // Use the default implementation.
-  setOperationAction(ISD::VAEND             , MVT::Other, Expand);
-  setOperationAction(ISD::STACKSAVE         , MVT::Other, Expand);
-  setOperationAction(ISD::STACKRESTORE      , MVT::Other, Custom);
-  setOperationAction(ISD::DYNAMIC_STACKALLOC, MVT::i32  , Custom);
-  setOperationAction(ISD::DYNAMIC_STACKALLOC, MVT::i64  , Custom);
-  setOperationAction(ISD::GET_DYNAMIC_AREA_OFFSET, MVT::i32, Custom);
-  setOperationAction(ISD::GET_DYNAMIC_AREA_OFFSET, MVT::i64, Custom);
-  setOperationAction(ISD::EH_DWARF_CFA, MVT::i32, Custom);
-  setOperationAction(ISD::EH_DWARF_CFA, MVT::i64, Custom);
+  setOperationAction(ISD::VASTART, MVT::Other, Custom);
+  setOperationAction(ISD::VAARG,   MVT::Other, Expand);
+  setOperationAction(ISD::VACOPY,  MVT::Other, Expand);
+  setOperationAction(ISD::VAEND,   MVT::Other, Expand);
+
+  setOperationAction(ISD::STACKSAVE,    MVT::Other, Expand);
+  setOperationAction(ISD::STACKRESTORE, MVT::Other, Custom);
 
-  // We want to custom lower some of our intrinsics.
-  setOperationAction(ISD::INTRINSIC_WO_CHAIN, MVT::Other, Custom);
+  // Sub-word ATOMIC_CMP_SWAP need to ensure that the input is zero-extended.
+  setOperationAction(ISD::ATOMIC_CMP_SWAP, MVT::i32, Custom);
+  setOperationAction(ISD::ATOMIC_LOAD,     MVT::i64, Expand);
+  setOperationAction(ISD::ATOMIC_STORE,    MVT::i64, Expand);
 
-  // To handle counter-based loop conditions.
-  setOperationAction(ISD::INTRINSIC_W_CHAIN, MVT::i1, Custom);
+  if (!useSoftFloat()) {
+    if (hasSPE()) {
+      addRegisterClass(MVT::f32, &PPC::GPRCRegClass);
+      addRegisterClass(MVT::f64, &PPC::SPERCRegClass);
+    } else {
+      addRegisterClass(MVT::f32, &PPC::F4RCRegClass);
+      addRegisterClass(MVT::f64, &PPC::F8RCRegClass);
+    }
+  }
 
-  setOperationAction(ISD::INTRINSIC_VOID, MVT::i8, Custom);
-  setOperationAction(ISD::INTRINSIC_VOID, MVT::i16, Custom);
-  setOperationAction(ISD::INTRINSIC_VOID, MVT::i32, Custom);
-  setOperationAction(ISD::INTRINSIC_VOID, MVT::Other, Custom);
+  // Specify operation actions for SPE.
+  if (Subtarget.hasSPE()) {
+    // SPE has built-in conversions
+    setOperationAction(ISD::FP_TO_SINT,        MVT::i32, Legal);
+    setOperationAction(ISD::FP_TO_UINT,        MVT::i32, Legal);
+    setOperationAction(ISD::SINT_TO_FP,        MVT::i32, Legal);
+    setOperationAction(ISD::UINT_TO_FP,        MVT::i32, Legal);
+    setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::i32, Legal);
+    setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i32, Legal);
+    setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i32, Legal);
+    setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i32, Legal);
+
+    for (MVT VT : {MVT::f32, MVT::f64}) {
+      setOperationAction(ISD::FMA, VT, Expand);
 
       // Comparisons that require checking two conditions.
-  if (Subtarget.hasSPE()) {
-    setCondCodeAction(ISD::SETO, MVT::f32, Expand);
-    setCondCodeAction(ISD::SETO, MVT::f64, Expand);
-    setCondCodeAction(ISD::SETUO, MVT::f32, Expand);
-    setCondCodeAction(ISD::SETUO, MVT::f64, Expand);
-  }
-  setCondCodeAction(ISD::SETULT, MVT::f32, Expand);
-  setCondCodeAction(ISD::SETULT, MVT::f64, Expand);
-  setCondCodeAction(ISD::SETUGT, MVT::f32, Expand);
-  setCondCodeAction(ISD::SETUGT, MVT::f64, Expand);
-  setCondCodeAction(ISD::SETUEQ, MVT::f32, Expand);
-  setCondCodeAction(ISD::SETUEQ, MVT::f64, Expand);
-  setCondCodeAction(ISD::SETOGE, MVT::f32, Expand);
-  setCondCodeAction(ISD::SETOGE, MVT::f64, Expand);
-  setCondCodeAction(ISD::SETOLE, MVT::f32, Expand);
-  setCondCodeAction(ISD::SETOLE, MVT::f64, Expand);
-  setCondCodeAction(ISD::SETONE, MVT::f32, Expand);
-  setCondCodeAction(ISD::SETONE, MVT::f64, Expand);
+      setCondCodeAction(ISD::SETO,  VT, Expand);
+      setCondCodeAction(ISD::SETUO, VT, Expand);
 
-  setOperationAction(ISD::STRICT_FP_EXTEND, MVT::f32, Legal);
-  setOperationAction(ISD::STRICT_FP_EXTEND, MVT::f64, Legal);
+      setIndexedLoadAction(ISD::PRE_INC,  VT, Expand);
+      setIndexedStoreAction(ISD::PRE_INC, VT, Expand);
+    }
+
+    setLoadExtAction(ISD::EXTLOAD, MVT::f64, MVT::f32, Expand);
+  }
+
+  if (Subtarget.is64BitELFABI()) {
+    // VAARG always uses double-word chunks, so promote anything smaller.
+    for (MVT VT : {MVT::i1, MVT::i8, MVT::i16, MVT::i32}) {
+      setOperationAction(ISD::VAARG, VT, Promote);
+      AddPromotedToType(ISD::VAARG,  VT, MVT::i64);
+    }
+  }
+
+  if (Subtarget.is32BitELFABI()) {
+    // VAARG is custom lowered with the 32-bit SVR4 ABI.
+    setOperationAction(ISD::VAARG, MVT::Other, Custom);
+    setOperationAction(ISD::VAARG, MVT::i64, Custom);
+
+    // VACOPY is custom lowered with the 32-bit SVR4 ABI.
+    setOperationAction(ISD::VACOPY, MVT::Other, Custom);
+  }
 
   if (Subtarget.has64BitSupport()) {
     // They also have instructions for converting between i64 and fp.
-    setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::i64, Custom);
-    setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i64, Expand);
-    setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i64, Custom);
-    setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i64, Expand);
-    setOperationAction(ISD::FP_TO_SINT, MVT::i64, Custom);
-    setOperationAction(ISD::FP_TO_UINT, MVT::i64, Expand);
-    setOperationAction(ISD::SINT_TO_FP, MVT::i64, Custom);
-    setOperationAction(ISD::UINT_TO_FP, MVT::i64, Expand);
+    setOperationAction(ISD::FP_TO_SINT,        MVT::i64, Custom);
+    setOperationAction(ISD::FP_TO_UINT,        MVT::i64, Expand);
+    setOperationAction(ISD::SINT_TO_FP,        MVT::i64, Custom);
+    setOperationAction(ISD::UINT_TO_FP,        MVT::i64, Expand);
+    setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::i64, Custom);
+    setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i64, Expand);
+    setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i64, Custom);
+    setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i64, Expand);
     // This is just the low 32 bits of a (signed) fp->i64 conversion.
     // We cannot do this with Promote because i64 is not a legal type.
     setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i32, Custom);
-    setOperationAction(ISD::FP_TO_UINT, MVT::i32, Custom);
+    setOperationAction(ISD::FP_TO_UINT,        MVT::i32, Custom);
 
     if (Subtarget.hasLFIWAX() || Subtarget.isPPC64()) {
-      setOperationAction(ISD::SINT_TO_FP, MVT::i32, Custom);
+      setOperationAction(ISD::SINT_TO_FP,        MVT::i32, Custom);
       setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i32, Custom);
     }
-  } else {
-    // PowerPC does not have FP_TO_UINT on 32-bit implementations.
-    if (Subtarget.hasSPE()) {
-      setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i32, Legal);
-      setOperationAction(ISD::FP_TO_UINT, MVT::i32, Legal);
-    } else {
-      setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i32, Expand);
-      setOperationAction(ISD::FP_TO_UINT, MVT::i32, Expand);
-    }
-  }
+    setOperationAction(ISD::PREFETCH, MVT::Other, Legal);
+
+    // PowerPC has better expansions for funnel shifts than the generic
+    // TargetLowering::expandFunnelShift.
+    setOperationAction(ISD::FSHL, MVT::i64, Custom);
+    setOperationAction(ISD::FSHR, MVT::i64, Custom);
 
-  // With the instructions enabled under FPCVT, we can do everything.
     if (Subtarget.hasFPCVT()) {
-    if (Subtarget.has64BitSupport()) {
-      setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::i64, Custom);
-      setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i64, Custom);
-      setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i64, Custom);
-      setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i64, Custom);
-      setOperationAction(ISD::FP_TO_SINT, MVT::i64, Custom);
-      setOperationAction(ISD::FP_TO_UINT, MVT::i64, Custom);
-      setOperationAction(ISD::SINT_TO_FP, MVT::i64, Custom);
-      setOperationAction(ISD::UINT_TO_FP, MVT::i64, Custom);
+      setOperationAction(ISD::FP_TO_UINT,        MVT::i64, Custom);
+      setOperationAction(ISD::UINT_TO_FP,        MVT::i64, Custom);
+      setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i64, Custom);
+      setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i64, Custom);
     }
-
-    setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::i32, Custom);
-    setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i32, Custom);
-    setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i32, Custom);
-    setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i32, Custom);
-    setOperationAction(ISD::FP_TO_SINT, MVT::i32, Custom);
-    setOperationAction(ISD::FP_TO_UINT, MVT::i32, Custom);
-    setOperationAction(ISD::SINT_TO_FP, MVT::i32, Custom);
-    setOperationAction(ISD::UINT_TO_FP, MVT::i32, Custom);
   }
 
   if (Subtarget.use64BitRegs()) {
     // 64-bit PowerPC implementations can support i64 types directly
     addRegisterClass(MVT::i64, &PPC::G8RCRegClass);
     // BUILD_PAIR can't be handled natively, and should be expanded to shl/or
     setOperationAction(ISD::BUILD_PAIR, MVT::i64, Expand);
     // 64-bit PowerPC wants to expand i128 shifts itself.
     setOperationAction(ISD::SHL_PARTS, MVT::i64, Custom);
     setOperationAction(ISD::SRA_PARTS, MVT::i64, Custom);
     setOperationAction(ISD::SRL_PARTS, MVT::i64, Custom);
-  } else {
-    // 32-bit PowerPC wants to expand i64 shifts itself.
-    setOperationAction(ISD::SHL_PARTS, MVT::i32, Custom);
-    setOperationAction(ISD::SRA_PARTS, MVT::i32, Custom);
-    setOperationAction(ISD::SRL_PARTS, MVT::i32, Custom);
+
+    setOperationAction(ISD::SHL_PARTS, MVT::i32, Legal);
+    setOperationAction(ISD::SRA_PARTS, MVT::i32, Legal);
+    setOperationAction(ISD::SRL_PARTS, MVT::i32, Legal);
   }
 
-  // PowerPC has better expansions for funnel shifts than the generic
-  // TargetLowering::expandFunnelShift.
-  if (Subtarget.has64BitSupport()) {
-    setOperationAction(ISD::FSHL, MVT::i64, Custom);
-    setOperationAction(ISD::FSHR, MVT::i64, Custom);
+  if (Subtarget.isPPC64()) {
+    if (Subtarget.hasDirectMove()) {
+      setOperationAction(ISD::BITCAST, MVT::i32, Legal);
+      setOperationAction(ISD::BITCAST, MVT::i64, Legal);
+
+      for (MVT VT : {MVT::f32, MVT::f64}) {
+        setOperationAction(ISD::BITCAST, VT, Legal);
+        if (TM.Options.UnsafeFPMath) {
+          setOperationAction(ISD::LRINT,   VT, Legal);
+          setOperationAction(ISD::LLRINT,  VT, Legal);
+          setOperationAction(ISD::LROUND,  VT, Legal);
+          setOperationAction(ISD::LLROUND, VT, Legal);
+        }
+      }
     }
-  setOperationAction(ISD::FSHL, MVT::i32, Custom);
-  setOperationAction(ISD::FSHR, MVT::i32, Custom);
 
-  if (Subtarget.hasVSX()) {
-    setOperationAction(ISD::FMAXNUM_IEEE, MVT::f64, Legal);
-    setOperationAction(ISD::FMAXNUM_IEEE, MVT::f32, Legal);
-    setOperationAction(ISD::FMINNUM_IEEE, MVT::f64, Legal);
-    setOperationAction(ISD::FMINNUM_IEEE, MVT::f32, Legal);
+    setOperationAction(ISD::READCYCLECOUNTER, MVT::i64, Legal);
+    setOperationAction(ISD::ATOMIC_LOAD,      MVT::i64, Legal);
+    setOperationAction(ISD::ATOMIC_STORE,     MVT::i64, Legal);
   }
 
+  // Specify operation actions if altivec enabled.
   if (Subtarget.hasAltivec()) {
-    for (MVT VT : { MVT::v16i8, MVT::v8i16, MVT::v4i32 }) {
+    addRegisterClass(MVT::v4f32, &PPC::VRRCRegClass);
+    addRegisterClass(MVT::v4i32, &PPC::VRRCRegClass);
+    addRegisterClass(MVT::v8i16, &PPC::VRRCRegClass);
+    addRegisterClass(MVT::v16i8, &PPC::VRRCRegClass);
+
+    for (MVT VT : {MVT::v16i8, MVT::v8i16, MVT::v4i32}) {
       setOperationAction(ISD::SADDSAT, VT, Legal);
       setOperationAction(ISD::SSUBSAT, VT, Legal);
       setOperationAction(ISD::UADDSAT, VT, Legal);
       setOperationAction(ISD::USUBSAT, VT, Legal);
     }
     // First set operation action for all vector types to expand. Then we
     // will selectively turn on ones that can be effectively codegen'd.
     for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
       // add/sub are legal for all supported vector VT's.
       setOperationAction(ISD::ADD, VT, Legal);
       setOperationAction(ISD::SUB, VT, Legal);
 
-      // For v2i64, these are only valid with P8Vector. This is corrected after
-      // the loop.
-      if (VT.getSizeInBits() <= 128 && VT.getScalarSizeInBits() <= 64) {
+      setOperationAction(ISD::VSELECT, VT, Legal);
+
+      // For v2i64, these are only valid with P8Vector.
+      if (VT.getSizeInBits() <= 128 && VT.getScalarSizeInBits() <= 64 &&
+          VT != MVT::v2i64) {
         setOperationAction(ISD::SMAX, VT, Legal);
         setOperationAction(ISD::SMIN, VT, Legal);
         setOperationAction(ISD::UMAX, VT, Legal);
         setOperationAction(ISD::UMIN, VT, Legal);
       }
-      else {
-        setOperationAction(ISD::SMAX, VT, Expand);
-        setOperationAction(ISD::SMIN, VT, Expand);
-        setOperationAction(ISD::UMAX, VT, Expand);
-        setOperationAction(ISD::UMIN, VT, Expand);
-      }
-
-      if (Subtarget.hasVSX()) {
-        setOperationAction(ISD::FMAXNUM, VT, Legal);
-        setOperationAction(ISD::FMINNUM, VT, Legal);
-      }
-
-      // Vector instructions introduced in P8
-      if (Subtarget.hasP8Altivec() && (VT.SimpleTy != MVT::v1i128)) {
-        setOperationAction(ISD::CTPOP, VT, Legal);
-        setOperationAction(ISD::CTLZ, VT, Legal);
-      }
-      else {
-        setOperationAction(ISD::CTPOP, VT, Expand);
-        setOperationAction(ISD::CTLZ, VT, Expand);
-      }
-
-      // Vector instructions introduced in P9
-      if (Subtarget.hasP9Altivec() && (VT.SimpleTy != MVT::v1i128))
-        setOperationAction(ISD::CTTZ, VT, Legal);
-      else
-        setOperationAction(ISD::CTTZ, VT, Expand);
 
       // We promote all shuffles to v16i8.
       setOperationAction(ISD::VECTOR_SHUFFLE, VT, Promote);
-      AddPromotedToType (ISD::VECTOR_SHUFFLE, VT, MVT::v16i8);
+      AddPromotedToType(ISD::VECTOR_SHUFFLE,  VT, MVT::v16i8);
 
       // We promote all non-typed operations to v4i32.
-      setOperationAction(ISD::AND   , VT, Promote);
-      AddPromotedToType (ISD::AND   , VT, MVT::v4i32);
-      setOperationAction(ISD::OR    , VT, Promote);
-      AddPromotedToType (ISD::OR    , VT, MVT::v4i32);
-      setOperationAction(ISD::XOR   , VT, Promote);
-      AddPromotedToType (ISD::XOR   , VT, MVT::v4i32);
-      setOperationAction(ISD::LOAD  , VT, Promote);
-      AddPromotedToType (ISD::LOAD  , VT, MVT::v4i32);
-      setOperationAction(ISD::SELECT, VT, Promote);
-      AddPromotedToType (ISD::SELECT, VT, MVT::v4i32);
-      setOperationAction(ISD::VSELECT, VT, Legal);
+      setOperationAction(ISD::AND,       VT, Promote);
+      setOperationAction(ISD::OR,        VT, Promote);
+      setOperationAction(ISD::XOR,       VT, Promote);
+      setOperationAction(ISD::LOAD,      VT, Promote);
+      setOperationAction(ISD::STORE,     VT, Promote);
+      setOperationAction(ISD::SELECT,    VT, Promote);
       setOperationAction(ISD::SELECT_CC, VT, Promote);
-      AddPromotedToType (ISD::SELECT_CC, VT, MVT::v4i32);
-      setOperationAction(ISD::STORE, VT, Promote);
-      AddPromotedToType (ISD::STORE, VT, MVT::v4i32);
+      AddPromotedToType(ISD::AND,       VT, MVT::v4i32);
+      AddPromotedToType(ISD::OR,        VT, MVT::v4i32);
+      AddPromotedToType(ISD::XOR,       VT, MVT::v4i32);
+      AddPromotedToType(ISD::LOAD,      VT, MVT::v4i32);
+      AddPromotedToType(ISD::STORE,     VT, MVT::v4i32);
+      AddPromotedToType(ISD::SELECT,    VT, MVT::v4i32);
+      AddPromotedToType(ISD::SELECT_CC, VT, MVT::v4i32);
 
       // No other operations are legal.
-      setOperationAction(ISD::MUL , VT, Expand);
-      setOperationAction(ISD::SDIV, VT, Expand);
-      setOperationAction(ISD::SREM, VT, Expand);
-      setOperationAction(ISD::UDIV, VT, Expand);
-      setOperationAction(ISD::UREM, VT, Expand);
-      setOperationAction(ISD::FDIV, VT, Expand);
-      setOperationAction(ISD::FREM, VT, Expand);
-      setOperationAction(ISD::FNEG, VT, Expand);
-      setOperationAction(ISD::FSQRT, VT, Expand);
-      setOperationAction(ISD::FLOG, VT, Expand);
-      setOperationAction(ISD::FLOG10, VT, Expand);
-      setOperationAction(ISD::FLOG2, VT, Expand);
-      setOperationAction(ISD::FEXP, VT, Expand);
-      setOperationAction(ISD::FEXP2, VT, Expand);
-      setOperationAction(ISD::FSIN, VT, Expand);
-      setOperationAction(ISD::FCOS, VT, Expand);
-      setOperationAction(ISD::FABS, VT, Expand);
-      setOperationAction(ISD::FFLOOR, VT, Expand);
-      setOperationAction(ISD::FCEIL,  VT, Expand);
-      setOperationAction(ISD::FTRUNC, VT, Expand);
-      setOperationAction(ISD::FRINT,  VT, Expand);
-      setOperationAction(ISD::FNEARBYINT, VT, Expand);
-      setOperationAction(ISD::EXTRACT_VECTOR_ELT, VT, Expand);
-      setOperationAction(ISD::INSERT_VECTOR_ELT, VT, Expand);
-      setOperationAction(ISD::BUILD_VECTOR, VT, Expand);
-      setOperationAction(ISD::MULHU, VT, Expand);
-      setOperationAction(ISD::MULHS, VT, Expand);
+      setOperationAction(ISD::MUL,       VT, Expand);
+      setOperationAction(ISD::SDIV,      VT, Expand);
+      setOperationAction(ISD::SREM,      VT, Expand);
+      setOperationAction(ISD::UDIV,      VT, Expand);
+      setOperationAction(ISD::UREM,      VT, Expand);
+      setOperationAction(ISD::FDIV,      VT, Expand);
+      setOperationAction(ISD::FREM,      VT, Expand);
+      setOperationAction(ISD::FNEG,      VT, Expand);
+      setOperationAction(ISD::FSQRT,     VT, Expand);
+      setOperationAction(ISD::FLOG,      VT, Expand);
+      setOperationAction(ISD::FLOG10,    VT, Expand);
+      setOperationAction(ISD::FLOG2,     VT, Expand);
+      setOperationAction(ISD::FEXP,      VT, Expand);
+      setOperationAction(ISD::FEXP2,     VT, Expand);
+      setOperationAction(ISD::FSIN,      VT, Expand);
+      setOperationAction(ISD::FCOS,      VT, Expand);
+      setOperationAction(ISD::FABS,      VT, Expand);
+      setOperationAction(ISD::FFLOOR,    VT, Expand);
+      setOperationAction(ISD::FCEIL,     VT, Expand);
+      setOperationAction(ISD::FTRUNC,    VT, Expand);
+      setOperationAction(ISD::FRINT,     VT, Expand);
+      setOperationAction(ISD::MULHU,     VT, Expand);
+      setOperationAction(ISD::MULHS,     VT, Expand);
       setOperationAction(ISD::UMUL_LOHI, VT, Expand);
       setOperationAction(ISD::SMUL_LOHI, VT, Expand);
-      setOperationAction(ISD::UDIVREM, VT, Expand);
-      setOperationAction(ISD::SDIVREM, VT, Expand);
-      setOperationAction(ISD::SCALAR_TO_VECTOR, VT, Expand);
-      setOperationAction(ISD::FPOW, VT, Expand);
-      setOperationAction(ISD::BSWAP, VT, Expand);
-      setOperationAction(ISD::SIGN_EXTEND_INREG, VT, Expand);
-      setOperationAction(ISD::ROTL, VT, Expand);
-      setOperationAction(ISD::ROTR, VT, Expand);
+      setOperationAction(ISD::UDIVREM,   VT, Expand);
+      setOperationAction(ISD::SDIVREM,   VT, Expand);
+      setOperationAction(ISD::FPOW,      VT, Expand);
+      setOperationAction(ISD::BSWAP,     VT, Expand);
+      setOperationAction(ISD::ROTL,      VT, Expand);
+      setOperationAction(ISD::ROTR,      VT, Expand);
+      setOperationAction(ISD::CTPOP,     VT, Expand);
+      setOperationAction(ISD::CTLZ,      VT, Expand);
+      setOperationAction(ISD::CTTZ,      VT, Expand);
+
+      setOperationAction(ISD::SCALAR_TO_VECTOR,   VT, Expand);
+      setOperationAction(ISD::SIGN_EXTEND_INREG,  VT, Expand);
+      setOperationAction(ISD::EXTRACT_VECTOR_ELT, VT, Expand);
+      setOperationAction(ISD::INSERT_VECTOR_ELT,  VT, Expand);
+      setOperationAction(ISD::FNEARBYINT,         VT, Expand);
+      setOperationAction(ISD::BUILD_VECTOR,       VT, Expand);
 
       for (MVT InnerVT : MVT::fixedlen_vector_valuetypes()) {
         setTruncStoreAction(VT, InnerVT, Expand);
         setLoadExtAction(ISD::SEXTLOAD, VT, InnerVT, Expand);
         setLoadExtAction(ISD::ZEXTLOAD, VT, InnerVT, Expand);
-        setLoadExtAction(ISD::EXTLOAD, VT, InnerVT, Expand);
+        setLoadExtAction(ISD::EXTLOAD,  VT, InnerVT, Expand);
       }
     }
+
+    setOperationAction(ISD::SELECT,    MVT::v4i32, Expand);
     setOperationAction(ISD::SELECT_CC, MVT::v4i32, Expand);
-    if (!Subtarget.hasP8Vector()) {
-      setOperationAction(ISD::SMAX, MVT::v2i64, Expand);
-      setOperationAction(ISD::SMIN, MVT::v2i64, Expand);
-      setOperationAction(ISD::UMAX, MVT::v2i64, Expand);
-      setOperationAction(ISD::UMIN, MVT::v2i64, Expand);
-    }
 
     // We can custom expand all VECTOR_SHUFFLEs to VPERM, others we can handle
     // with merges, splats, etc.
     setOperationAction(ISD::VECTOR_SHUFFLE, MVT::v16i8, Custom);
 
     // Vector truncates to sub-word integer that fit in an Altivec/VSX register
     // are cheap, so handle them before they get expanded to scalar.
-    setOperationAction(ISD::TRUNCATE, MVT::v8i8, Custom);
-    setOperationAction(ISD::TRUNCATE, MVT::v4i8, Custom);
-    setOperationAction(ISD::TRUNCATE, MVT::v2i8, Custom);
-    setOperationAction(ISD::TRUNCATE, MVT::v4i16, Custom);
-    setOperationAction(ISD::TRUNCATE, MVT::v2i16, Custom);
+    for (MVT VT : {MVT::v8i8, MVT::v4i8, MVT::v2i8, MVT::v4i16, MVT::v2i16})
+      setOperationAction(ISD::TRUNCATE, VT, Custom);
 
-    setOperationAction(ISD::AND   , MVT::v4i32, Legal);
-    setOperationAction(ISD::OR    , MVT::v4i32, Legal);
-    setOperationAction(ISD::XOR   , MVT::v4i32, Legal);
-    setOperationAction(ISD::LOAD  , MVT::v4i32, Legal);
-    setOperationAction(ISD::SELECT, MVT::v4i32,
-                       Subtarget.useCRBits() ? Legal : Expand);
-    setOperationAction(ISD::STORE , MVT::v4i32, Legal);
-    setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::v4i32, Legal);
-    setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::v4i32, Legal);
-    setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::v4i32, Legal);
-    setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::v4i32, Legal);
-    setOperationAction(ISD::FP_TO_SINT, MVT::v4i32, Legal);
-    setOperationAction(ISD::FP_TO_UINT, MVT::v4i32, Legal);
-    setOperationAction(ISD::SINT_TO_FP, MVT::v4i32, Legal);
-    setOperationAction(ISD::UINT_TO_FP, MVT::v4i32, Legal);
-    setOperationAction(ISD::FFLOOR, MVT::v4f32, Legal);
-    setOperationAction(ISD::FCEIL, MVT::v4f32, Legal);
-    setOperationAction(ISD::FTRUNC, MVT::v4f32, Legal);
-    setOperationAction(ISD::FNEARBYINT, MVT::v4f32, Legal);
+    setOperationAction(ISD::AND,               MVT::v4i32, Legal);
+    setOperationAction(ISD::OR,                MVT::v4i32, Legal);
+    setOperationAction(ISD::XOR,               MVT::v4i32, Legal);
+    setOperationAction(ISD::MUL,               MVT::v4f32, Legal);
+    setOperationAction(ISD::MUL,               MVT::v4i32, Custom);
+    setOperationAction(ISD::FMA,               MVT::v4f32, Legal);
+    setOperationAction(ISD::LOAD,              MVT::v4i32, Legal);
+    setOperationAction(ISD::STORE,             MVT::v4i32, Legal);
+    setOperationAction(ISD::FFLOOR,            MVT::v4f32, Legal);
+    setOperationAction(ISD::FCEIL,             MVT::v4f32, Legal);
+    setOperationAction(ISD::FTRUNC,            MVT::v4f32, Legal);
+    setOperationAction(ISD::FNEARBYINT,        MVT::v4f32, Legal);
+    setOperationAction(ISD::FP_TO_SINT,        MVT::v4i32, Legal);
+    setOperationAction(ISD::FP_TO_UINT,        MVT::v4i32, Legal);
+    setOperationAction(ISD::SINT_TO_FP,        MVT::v4i32, Legal);
+    setOperationAction(ISD::UINT_TO_FP,        MVT::v4i32, Legal);
+    setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::v4i32, Legal);
+    setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::v4i32, Legal);
+    setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::v4i32, Legal);
+    setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::v4i32, Legal);
 
     // Custom lowering ROTL v1i128 to VECTOR_SHUFFLE v16i8.
     setOperationAction(ISD::ROTL, MVT::v1i128, Custom);
     // With hasAltivec set, we can lower ISD::ROTL to vrl(b|h|w).
-    if (Subtarget.hasAltivec())
-      for (auto VT : {MVT::v4i32, MVT::v8i16, MVT::v16i8})
-        setOperationAction(ISD::ROTL, VT, Legal);
-    // With hasP8Altivec set, we can lower ISD::ROTL to vrld.
-    if (Subtarget.hasP8Altivec())
-      setOperationAction(ISD::ROTL, MVT::v2i64, Legal);
-
-    addRegisterClass(MVT::v4f32, &PPC::VRRCRegClass);
-    addRegisterClass(MVT::v4i32, &PPC::VRRCRegClass);
-    addRegisterClass(MVT::v8i16, &PPC::VRRCRegClass);
-    addRegisterClass(MVT::v16i8, &PPC::VRRCRegClass);
-
-    setOperationAction(ISD::MUL, MVT::v4f32, Legal);
-    setOperationAction(ISD::FMA, MVT::v4f32, Legal);
-
-    if (Subtarget.hasVSX()) {
-      setOperationAction(ISD::FDIV, MVT::v4f32, Legal);
-      setOperationAction(ISD::FSQRT, MVT::v4f32, Legal);
-    }
-
-    if (Subtarget.hasP8Altivec())
-      setOperationAction(ISD::MUL, MVT::v4i32, Legal);
-    else
-      setOperationAction(ISD::MUL, MVT::v4i32, Custom);
 
     if (Subtarget.isISA3_1()) {
-      setOperationAction(ISD::MUL, MVT::v2i64, Legal);
-      setOperationAction(ISD::MULHS, MVT::v2i64, Legal);
-      setOperationAction(ISD::MULHU, MVT::v2i64, Legal);
-      setOperationAction(ISD::MULHS, MVT::v4i32, Legal);
-      setOperationAction(ISD::MULHU, MVT::v4i32, Legal);
-      setOperationAction(ISD::UDIV, MVT::v2i64, Legal);
-      setOperationAction(ISD::SDIV, MVT::v2i64, Legal);
-      setOperationAction(ISD::UDIV, MVT::v4i32, Legal);
-      setOperationAction(ISD::SDIV, MVT::v4i32, Legal);
-      setOperationAction(ISD::UREM, MVT::v2i64, Legal);
-      setOperationAction(ISD::SREM, MVT::v2i64, Legal);
-      setOperationAction(ISD::UREM, MVT::v4i32, Legal);
-      setOperationAction(ISD::SREM, MVT::v4i32, Legal);
+      for (MVT VT : {MVT::v4i32, MVT::v2i64}) {
+        setOperationAction(ISD::MULHS, VT, Legal);
+        setOperationAction(ISD::MULHU, VT, Legal);
+        setOperationAction(ISD::UDIV,  VT, Legal);
+        setOperationAction(ISD::SDIV,  VT, Legal);
+        setOperationAction(ISD::UREM,  VT, Legal);
+        setOperationAction(ISD::SREM,  VT, Legal);
+      }
+      setOperationAction(ISD::MUL,  MVT::v2i64,  Legal);
       setOperationAction(ISD::UREM, MVT::v1i128, Legal);
       setOperationAction(ISD::SREM, MVT::v1i128, Legal);
       setOperationAction(ISD::UDIV, MVT::v1i128, Legal);
       setOperationAction(ISD::SDIV, MVT::v1i128, Legal);
       setOperationAction(ISD::ROTL, MVT::v1i128, Legal);
     }
 
     setOperationAction(ISD::MUL, MVT::v8i16, Legal);
     setOperationAction(ISD::MUL, MVT::v16i8, Custom);
 
     setOperationAction(ISD::SCALAR_TO_VECTOR, MVT::v4f32, Custom);
     setOperationAction(ISD::SCALAR_TO_VECTOR, MVT::v4i32, Custom);
 
     setOperationAction(ISD::BUILD_VECTOR, MVT::v16i8, Custom);
     setOperationAction(ISD::BUILD_VECTOR, MVT::v8i16, Custom);
     setOperationAction(ISD::BUILD_VECTOR, MVT::v4i32, Custom);
     setOperationAction(ISD::BUILD_VECTOR, MVT::v4f32, Custom);
 
     // Altivec does not contain unordered floating-point compare instructions
-    setCondCodeAction(ISD::SETUO, MVT::v4f32, Expand);
+    setCondCodeAction(ISD::SETUO,  MVT::v4f32, Expand);
     setCondCodeAction(ISD::SETUEQ, MVT::v4f32, Expand);
     setCondCodeAction(ISD::SETO,   MVT::v4f32, Expand);
     setCondCodeAction(ISD::SETONE, MVT::v4f32, Expand);
 
-    if (Subtarget.hasVSX()) {
-      setOperationAction(ISD::SCALAR_TO_VECTOR, MVT::v2f64, Legal);
-      setOperationAction(ISD::EXTRACT_VECTOR_ELT, MVT::v2f64, Legal);
-      if (Subtarget.hasP8Vector()) {
-        setOperationAction(ISD::SCALAR_TO_VECTOR, MVT::v4f32, Legal);
-        setOperationAction(ISD::EXTRACT_VECTOR_ELT, MVT::v4f32, Legal);
-      }
-      if (Subtarget.hasDirectMove() && isPPC64) {
-        setOperationAction(ISD::SCALAR_TO_VECTOR, MVT::v16i8, Legal);
-        setOperationAction(ISD::SCALAR_TO_VECTOR, MVT::v8i16, Legal);
-        setOperationAction(ISD::SCALAR_TO_VECTOR, MVT::v4i32, Legal);
-        setOperationAction(ISD::SCALAR_TO_VECTOR, MVT::v2i64, Legal);
-        setOperationAction(ISD::EXTRACT_VECTOR_ELT, MVT::v16i8, Legal);
-        setOperationAction(ISD::EXTRACT_VECTOR_ELT, MVT::v8i16, Legal);
-        setOperationAction(ISD::EXTRACT_VECTOR_ELT, MVT::v4i32, Legal);
-        setOperationAction(ISD::EXTRACT_VECTOR_ELT, MVT::v2i64, Legal);
-      }
-      setOperationAction(ISD::EXTRACT_VECTOR_ELT, MVT::v2f64, Legal);
+    if (EnableSoftFP128) {
+      addRegisterClass(MVT::f128, &PPC::VRRCRegClass);
+
+      for (MVT FPT : MVT::fp_valuetypes())
+        setLoadExtAction(ISD::EXTLOAD, MVT::f128, FPT, Expand);
+
+      setOperationAction(ISD::LOAD,  MVT::f128, Promote);
+      setOperationAction(ISD::STORE, MVT::f128, Promote);
+
+      AddPromotedToType(ISD::LOAD,  MVT::f128, MVT::v4i32);
+      AddPromotedToType(ISD::STORE, MVT::f128, MVT::v4i32);
+
+      // Set FADD/FSUB as libcall to avoid the legalizer to expand the
+      // fp_to_uint and int_to_fp.
+      setOperationAction(ISD::FADD,      MVT::f128, LibCall);
+      setOperationAction(ISD::FSUB,      MVT::f128, LibCall);
+      setOperationAction(ISD::FMUL,      MVT::f128, Expand);
+      setOperationAction(ISD::FDIV,      MVT::f128, Expand);
+      setOperationAction(ISD::FNEG,      MVT::f128, Expand);
+      setOperationAction(ISD::FABS,      MVT::f128, Expand);
+      setOperationAction(ISD::FSIN,      MVT::f128, Expand);
+      setOperationAction(ISD::FCOS,      MVT::f128, Expand);
+      setOperationAction(ISD::FPOW,      MVT::f128, Expand);
+      setOperationAction(ISD::FPOWI,     MVT::f128, Expand);
+      setOperationAction(ISD::FREM,      MVT::f128, Expand);
+      setOperationAction(ISD::FSQRT,     MVT::f128, Expand);
+      setOperationAction(ISD::FMA,       MVT::f128, Expand);
+      setOperationAction(ISD::FCOPYSIGN, MVT::f128, Expand);
+
+      // Expand the fp_extend if the target type is fp128.
+      setOperationAction(ISD::FP_EXTEND,        MVT::f128, Expand);
+      setOperationAction(ISD::STRICT_FP_EXTEND, MVT::f128, Expand);
+
+      // Expand the fp_round if the source type is fp128.
+      for (MVT VT : {MVT::f32, MVT::f64}) {
+        setOperationAction(ISD::FP_ROUND,        VT, Custom);
+        setOperationAction(ISD::STRICT_FP_ROUND, VT, Custom);
+        setTruncStoreAction(MVT::f128,           VT, Expand);
+      }
+
+  // Specify operation actions if VSX enabled.
+  if (Subtarget.hasVSX()) {
+    addRegisterClass(MVT::f64,   &PPC::VSFRCRegClass);
+    addRegisterClass(MVT::v4i32, &PPC::VSRCRegClass);
+    addRegisterClass(MVT::v2i64, &PPC::VSRCRegClass);
+    addRegisterClass(MVT::v4f32, &PPC::VSRCRegClass);
+    addRegisterClass(MVT::v2f64, &PPC::VSRCRegClass);
+
+    for (MVT VT : {MVT::f32, MVT::f64}) {
+      setOperationAction(ISD::STRICT_FRINT, VT, Legal);
+      setOperationAction(ISD::FMAXNUM_IEEE, VT, Legal);
+      setOperationAction(ISD::FMINNUM_IEEE, VT, Legal);
+      setOperationAction(ISD::FROUND,       VT, Legal);
+      setOperationAction(ISD::FRINT,        VT, Legal);
 
       // The nearbyint variants are not allowed to raise the inexact exception
       // so we can only code-gen them with unsafe math.
-      if (TM.Options.UnsafeFPMath) {
-        setOperationAction(ISD::FNEARBYINT, MVT::f64, Legal);
-        setOperationAction(ISD::FNEARBYINT, MVT::f32, Legal);
-      }
+      if (TM.Options.UnsafeFPMath)
+        setOperationAction(ISD::FNEARBYINT, VT, Legal);
+    }
+
+    for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
+      setOperationAction(ISD::FMAXNUM, VT, Legal);
+      setOperationAction(ISD::FMINNUM, VT, Legal);
+    }
+
+    if (Subtarget.hasDirectMove() && Subtarget.isPPC64()) {
+      for (MVT VT : {MVT::v16i8, MVT::v8i16, MVT::v4i32, MVT::v2i64}) {
+        setOperationAction(ISD::SCALAR_TO_VECTOR,   VT, Legal);
+        setOperationAction(ISD::EXTRACT_VECTOR_ELT, VT, Legal);
+      }
+    }
+
+    for (MVT VT : {MVT::v2f64, MVT::v4f32}) {
+      setOperationAction(ISD::FRINT,      VT, Legal);
+      setOperationAction(ISD::FROUND,     VT, Legal);
+      setOperationAction(ISD::FNEARBYINT, VT, Legal);
+      setOperationAction(ISD::FDIV,       VT, Legal);
+      setOperationAction(ISD::FSQRT,      VT, Legal);
+      setOperationAction(ISD::FNEG,       VT, Legal);
+      setOperationAction(ISD::FABS,       VT, Legal);
+      setOperationAction(ISD::FCOPYSIGN,  VT, Legal);
+
+      // Handle constrained floating-point operations of vector.
+      // The predictor is `hasVSX` because altivec instruction has
+      // no exception but VSX vector instruction has.
+      setOperationAction(ISD::STRICT_FADD,    VT, Legal);
+      setOperationAction(ISD::STRICT_FSUB,    VT, Legal);
+      setOperationAction(ISD::STRICT_FMUL,    VT, Legal);
+      setOperationAction(ISD::STRICT_FDIV,    VT, Legal);
+      setOperationAction(ISD::STRICT_FMA,     VT, Legal);
+      setOperationAction(ISD::STRICT_FSQRT,   VT, Legal);
+      setOperationAction(ISD::STRICT_FMAXNUM, VT, Legal);
+      setOperationAction(ISD::STRICT_FMINNUM, VT, Legal);
+      setOperationAction(ISD::STRICT_FRINT,   VT, Legal);
+      setOperationAction(ISD::STRICT_FFLOOR,  VT, Legal);
+      setOperationAction(ISD::STRICT_FCEIL,   VT, Legal);
+      setOperationAction(ISD::STRICT_FTRUNC,  VT, Legal);
+      setOperationAction(ISD::STRICT_FROUND,  VT, Legal);
 
-      setOperationAction(ISD::FFLOOR, MVT::v2f64, Legal);
-      setOperationAction(ISD::FCEIL, MVT::v2f64, Legal);
-      setOperationAction(ISD::FTRUNC, MVT::v2f64, Legal);
-      setOperationAction(ISD::FNEARBYINT, MVT::v2f64, Legal);
-      setOperationAction(ISD::FRINT, MVT::v2f64, Legal);
-      setOperationAction(ISD::FROUND, MVT::v2f64, Legal);
-      setOperationAction(ISD::FROUND, MVT::f64, Legal);
-      setOperationAction(ISD::FRINT, MVT::f64, Legal);
-
-      setOperationAction(ISD::FNEARBYINT, MVT::v4f32, Legal);
-      setOperationAction(ISD::FRINT, MVT::v4f32, Legal);
-      setOperationAction(ISD::FROUND, MVT::v4f32, Legal);
-      setOperationAction(ISD::FROUND, MVT::f32, Legal);
-      setOperationAction(ISD::FRINT, MVT::f32, Legal);
-
-      setOperationAction(ISD::MUL, MVT::v2f64, Legal);
-      setOperationAction(ISD::FMA, MVT::v2f64, Legal);
-
-      setOperationAction(ISD::FDIV, MVT::v2f64, Legal);
-      setOperationAction(ISD::FSQRT, MVT::v2f64, Legal);
+    setOperationAction(ISD::FFLOOR,             MVT::v2f64, Legal);
+    setOperationAction(ISD::FCEIL,              MVT::v2f64, Legal);
+    setOperationAction(ISD::FTRUNC,             MVT::v2f64, Legal);
+    setOperationAction(ISD::MUL,                MVT::v2f64, Legal);
+    setOperationAction(ISD::FMA,                MVT::v2f64, Legal);
+    setOperationAction(ISD::LOAD,               MVT::v2f64, Legal);
+    setOperationAction(ISD::STORE,              MVT::v2f64, Legal);
+    setOperationAction(ISD::VECTOR_SHUFFLE,     MVT::v2f64, Legal);
+    setOperationAction(ISD::SCALAR_TO_VECTOR,   MVT::v2f64, Legal);
+    setOperationAction(ISD::EXTRACT_VECTOR_ELT, MVT::v2f64, Legal);
 
-      // Share the Altivec comparison restrictions.
-      setCondCodeAction(ISD::SETUO, MVT::v2f64, Expand);
-      setCondCodeAction(ISD::SETUEQ, MVT::v2f64, Expand);
-      setCondCodeAction(ISD::SETO,   MVT::v2f64, Expand);
-      setCondCodeAction(ISD::SETONE, MVT::v2f64, Expand);
+    setCondCodeAction(ISD::SETUO,  MVT::v2f64, Expand);
 
-      setOperationAction(ISD::LOAD, MVT::v2f64, Legal);
-      setOperationAction(ISD::STORE, MVT::v2f64, Legal);
-
-      setOperationAction(ISD::VECTOR_SHUFFLE, MVT::v2f64, Legal);
-
-      if (Subtarget.hasP8Vector())
-        addRegisterClass(MVT::f32, &PPC::VSSRCRegClass);
-
-      addRegisterClass(MVT::f64, &PPC::VSFRCRegClass);
-
-      addRegisterClass(MVT::v4i32, &PPC::VSRCRegClass);
-      addRegisterClass(MVT::v4f32, &PPC::VSRCRegClass);
-      addRegisterClass(MVT::v2f64, &PPC::VSRCRegClass);
-
-      if (Subtarget.hasP8Altivec()) {
-        setOperationAction(ISD::SHL, MVT::v2i64, Legal);
-        setOperationAction(ISD::SRA, MVT::v2i64, Legal);
-        setOperationAction(ISD::SRL, MVT::v2i64, Legal);
-
-        // 128 bit shifts can be accomplished via 3 instructions for SHL and
-        // SRL, but not for SRA because of the instructions available:
-        // VS{RL} and VS{RL}O. However due to direct move costs, it's not worth
-        // doing
-        setOperationAction(ISD::SHL, MVT::v1i128, Expand);
-        setOperationAction(ISD::SRL, MVT::v1i128, Expand);
-        setOperationAction(ISD::SRA, MVT::v1i128, Expand);
+    // VSX v2i64 only supports non-arithmetic operations.
+    setOperationAction(ISD::ADD, MVT::v2i64, Expand);
+    setOperationAction(ISD::SUB, MVT::v2i64, Expand);
 
-        setOperationAction(ISD::SETCC, MVT::v2i64, Legal);
-      }
-      else {
-        setOperationAction(ISD::SHL, MVT::v2i64, Expand);
-        setOperationAction(ISD::SRA, MVT::v2i64, Expand);
-        setOperationAction(ISD::SRL, MVT::v2i64, Expand);
 
-        setOperationAction(ISD::SETCC, MVT::v2i64, Custom);
 
-        // VSX v2i64 only supports non-arithmetic operations.
-        setOperationAction(ISD::ADD, MVT::v2i64, Expand);
-        setOperationAction(ISD::SUB, MVT::v2i64, Expand);
-      }
-
-      if (Subtarget.isISA3_1())
-        setOperationAction(ISD::SETCC, MVT::v1i128, Legal);
-      else
-        setOperationAction(ISD::SETCC, MVT::v1i128, Expand);
-
-      setOperationAction(ISD::LOAD, MVT::v2i64, Promote);
-      AddPromotedToType (ISD::LOAD, MVT::v2i64, MVT::v2f64);
-      setOperationAction(ISD::STORE, MVT::v2i64, Promote);
-      AddPromotedToType (ISD::STORE, MVT::v2i64, MVT::v2f64);
+    setOperationAction(ISD::LOAD,  MVT::v2i64, Promote);
+    AddPromotedToType(ISD::LOAD,   MVT::v2i64, MVT::v2f64);
+    AddPromotedToType(ISD::STORE,  MVT::v2i64, MVT::v2f64);
 
-      setOperationAction(ISD::VECTOR_SHUFFLE, MVT::v2i64, Legal);
+    setOperationAction(ISD::VECTOR_SHUFFLE,    MVT::v2i64, Legal);
 
-      setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::v2i64, Legal);
-      setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::v2i64, Legal);
-      setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::v2i64, Legal);
-      setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::v2i64, Legal);
-      setOperationAction(ISD::SINT_TO_FP, MVT::v2i64, Legal);
-      setOperationAction(ISD::UINT_TO_FP, MVT::v2i64, Legal);
-      setOperationAction(ISD::FP_TO_SINT, MVT::v2i64, Legal);
-      setOperationAction(ISD::FP_TO_UINT, MVT::v2i64, Legal);
+    setOperationAction(ISD::SINT_TO_FP,        MVT::v2i64, Legal);
+    setOperationAction(ISD::UINT_TO_FP,        MVT::v2i64, Legal);
+    setOperationAction(ISD::FP_TO_SINT,        MVT::v2i64, Legal);
+    setOperationAction(ISD::FP_TO_UINT,        MVT::v2i64, Legal);
 
-      // Custom handling for partial vectors of integers converted to
-      // floating point. We already have optimal handling for v2i32 through
-      // the DAG combine, so those aren't necessary.
-      setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::v2i8, Custom);
-      setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::v4i8, Custom);
-      setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::v2i16, Custom);
-      setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::v4i16, Custom);
-      setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::v2i8, Custom);
-      setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::v4i8, Custom);
-      setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::v2i16, Custom);
-      setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::v4i16, Custom);
-      setOperationAction(ISD::UINT_TO_FP, MVT::v2i8, Custom);
-      setOperationAction(ISD::UINT_TO_FP, MVT::v4i8, Custom);
-      setOperationAction(ISD::UINT_TO_FP, MVT::v2i16, Custom);
-      setOperationAction(ISD::UINT_TO_FP, MVT::v4i16, Custom);
-      setOperationAction(ISD::SINT_TO_FP, MVT::v2i8, Custom);
-      setOperationAction(ISD::SINT_TO_FP, MVT::v4i8, Custom);
-      setOperationAction(ISD::SINT_TO_FP, MVT::v2i16, Custom);
-      setOperationAction(ISD::SINT_TO_FP, MVT::v4i16, Custom);
-
-      setOperationAction(ISD::FNEG, MVT::v4f32, Legal);
-      setOperationAction(ISD::FNEG, MVT::v2f64, Legal);
-      setOperationAction(ISD::FABS, MVT::v4f32, Legal);
-      setOperationAction(ISD::FABS, MVT::v2f64, Legal);
-      setOperationAction(ISD::FCOPYSIGN, MVT::v4f32, Legal);
-      setOperationAction(ISD::FCOPYSIGN, MVT::v2f64, Legal);
+    for (MVT VT : {MVT::v2i8, MVT::v4i8, MVT::v2i16, MVT::v4i16}) {
+      setOperationAction(ISD::STRICT_UINT_TO_FP, VT, Custom);
+      setOperationAction(ISD::STRICT_SINT_TO_FP, VT, Custom);
+      setOperationAction(ISD::UINT_TO_FP,        VT, Custom);
+      setOperationAction(ISD::SINT_TO_FP,        VT, Custom);
+    }
 
-      if (Subtarget.hasDirectMove())
-        setOperationAction(ISD::BUILD_VECTOR, MVT::v2i64, Custom);
-      setOperationAction(ISD::BUILD_VECTOR, MVT::v2f64, Custom);
 
-      // Handle constrained floating-point operations of vector.
-      // The predictor is `hasVSX` because altivec instruction has
-      // no exception but VSX vector instruction has.
-      setOperationAction(ISD::STRICT_FADD, MVT::v4f32, Legal);
-      setOperationAction(ISD::STRICT_FSUB, MVT::v4f32, Legal);
-      setOperationAction(ISD::STRICT_FMUL, MVT::v4f32, Legal);
-      setOperationAction(ISD::STRICT_FDIV, MVT::v4f32, Legal);
-      setOperationAction(ISD::STRICT_FMA, MVT::v4f32, Legal);
-      setOperationAction(ISD::STRICT_FSQRT, MVT::v4f32, Legal);
-      setOperationAction(ISD::STRICT_FMAXNUM, MVT::v4f32, Legal);
-      setOperationAction(ISD::STRICT_FMINNUM, MVT::v4f32, Legal);
-      setOperationAction(ISD::STRICT_FRINT, MVT::v4f32, Legal);
-      setOperationAction(ISD::STRICT_FFLOOR, MVT::v4f32, Legal);
-      setOperationAction(ISD::STRICT_FCEIL,  MVT::v4f32, Legal);
-      setOperationAction(ISD::STRICT_FTRUNC, MVT::v4f32, Legal);
-      setOperationAction(ISD::STRICT_FROUND, MVT::v4f32, Legal);
-
-      setOperationAction(ISD::STRICT_FADD, MVT::v2f64, Legal);
-      setOperationAction(ISD::STRICT_FSUB, MVT::v2f64, Legal);
-      setOperationAction(ISD::STRICT_FMUL, MVT::v2f64, Legal);
-      setOperationAction(ISD::STRICT_FDIV, MVT::v2f64, Legal);
-      setOperationAction(ISD::STRICT_FMA, MVT::v2f64, Legal);
-      setOperationAction(ISD::STRICT_FSQRT, MVT::v2f64, Legal);
-      setOperationAction(ISD::STRICT_FMAXNUM, MVT::v2f64, Legal);
-      setOperationAction(ISD::STRICT_FMINNUM, MVT::v2f64, Legal);
-      setOperationAction(ISD::STRICT_FRINT, MVT::v2f64, Legal);
-      setOperationAction(ISD::STRICT_FFLOOR, MVT::v2f64, Legal);
-      setOperationAction(ISD::STRICT_FCEIL,  MVT::v2f64, Legal);
-      setOperationAction(ISD::STRICT_FTRUNC, MVT::v2f64, Legal);
-      setOperationAction(ISD::STRICT_FROUND, MVT::v2f64, Legal);
-
-      addRegisterClass(MVT::v2i64, &PPC::VSRCRegClass);
-    }
+    if (Subtarget.isISA3_1())
+      setOperationAction(ISD::SETCC, MVT::v1i128, Legal);
+    else
+      setOperationAction(ISD::SETCC, MVT::v1i128, Expand);
 
-    if (Subtarget.hasP8Altivec()) {
-      addRegisterClass(MVT::v2i64, &PPC::VRRCRegClass);
-      addRegisterClass(MVT::v1i128, &PPC::VRRCRegClass);
-    }
+  // Specify operation actions if Power8 altivec enabled.
 
-    if (Subtarget.hasP9Vector()) {
-      setOperationAction(ISD::INSERT_VECTOR_ELT, MVT::v4i32, Custom);
-      setOperationAction(ISD::INSERT_VECTOR_ELT, MVT::v4f32, Custom);
+    // With hasP8Altivec set, we can lower ISD::ROTL to vrld.
+    setOperationAction(ISD::ROTL,  MVT::v2i64, Legal);
+    setOperationAction(ISD::SHL,   MVT::v2i64, Legal);
+    setOperationAction(ISD::SRA,   MVT::v2i64, Legal);
+    setOperationAction(ISD::SRL,   MVT::v2i64, Legal);
+    setOperationAction(ISD::ADD,   MVT::v2i64, Legal);
+    setOperationAction(ISD::SUB,   MVT::v2i64, Legal);
+    setOperationAction(ISD::SETCC, MVT::v2i64, Legal);
+    setOperationAction(ISD::MUL,   MVT::v4i32, Legal);
 
+    for (MVT VT : MVT::fixedlen_vector_valuetypes()) {
+      // Vector instructions introduced in P8
+      if (VT.SimpleTy != MVT::v1i128) {
+        setOperationAction(ISD::CTPOP, VT, Legal);
+        setOperationAction(ISD::CTLZ, VT, Legal);
+      }
+    }
+
+    if (Subtarget.hasVSX()) {
       // 128 bit shifts can be accomplished via 3 instructions for SHL and
       // SRL, but not for SRA because of the instructions available:
-      // VS{RL} and VS{RL}O.
-      setOperationAction(ISD::SHL, MVT::v1i128, Legal);
-      setOperationAction(ISD::SRL, MVT::v1i128, Legal);
+      // VS{RL} and VS{RL}O. However due to direct move costs, it's not worth
+      // doing
+      setOperationAction(ISD::SHL, MVT::v1i128, Expand);
+      setOperationAction(ISD::SRL, MVT::v1i128, Expand);
       setOperationAction(ISD::SRA, MVT::v1i128, Expand);
+    }
+  }
+
+  // Specify operation actions if Power8 vector enabled.
+  if (Subtarget.hasP8Vector()) {
+    addRegisterClass(MVT::f32, &PPC::VSSRCRegClass);
+
+    setOperationAction(ISD::SMAX, MVT::v2i64, Legal);
+    setOperationAction(ISD::SMIN, MVT::v2i64, Legal);
+    setOperationAction(ISD::UMAX, MVT::v2i64, Legal);
+    setOperationAction(ISD::UMIN, MVT::v2i64, Legal);
+
+    setOperationAction(ISD::SCALAR_TO_VECTOR,   MVT::v4f32, Legal);
+    setOperationAction(ISD::EXTRACT_VECTOR_ELT, MVT::v4f32, Legal);
+  }
 
-      addRegisterClass(MVT::f128, &PPC::VRRCRegClass);
-      setOperationAction(ISD::FADD, MVT::f128, Legal);
-      setOperationAction(ISD::FSUB, MVT::f128, Legal);
-      setOperationAction(ISD::FDIV, MVT::f128, Legal);
-      setOperationAction(ISD::FMUL, MVT::f128, Legal);
-      setOperationAction(ISD::FP_EXTEND, MVT::f128, Legal);
-      // No extending loads to f128 on PPC.
-      for (MVT FPT : MVT::fp_valuetypes())
-        setLoadExtAction(ISD::EXTLOAD, MVT::f128, FPT, Expand);
-      setOperationAction(ISD::FMA, MVT::f128, Legal);
-      setCondCodeAction(ISD::SETULT, MVT::f128, Expand);
-      setCondCodeAction(ISD::SETUGT, MVT::f128, Expand);
-      setCondCodeAction(ISD::SETUEQ, MVT::f128, Expand);
-      setCondCodeAction(ISD::SETOGE, MVT::f128, Expand);
-      setCondCodeAction(ISD::SETOLE, MVT::f128, Expand);
-      setCondCodeAction(ISD::SETONE, MVT::f128, Expand);
-
-      setOperationAction(ISD::FTRUNC, MVT::f128, Legal);
-      setOperationAction(ISD::FRINT, MVT::f128, Legal);
-      setOperationAction(ISD::FFLOOR, MVT::f128, Legal);
-      setOperationAction(ISD::FCEIL, MVT::f128, Legal);
-      setOperationAction(ISD::FNEARBYINT, MVT::f128, Legal);
-      setOperationAction(ISD::FROUND, MVT::f128, Legal);
+  if (Subtarget.isISA3_0()) {
+    for (MVT VT : {MVT::i32, MVT::i64}) {
+      setOperationAction(ISD::SREM, VT, Legal);
+      setOperationAction(ISD::UREM, VT, Legal);
+      setOperationAction(ISD::CTTZ, VT, Legal);
+    }
+
+    for (MVT VT : {MVT::f32, MVT::f64}) {
+      setOperationAction(ISD::FP16_TO_FP, VT, Legal);
+      setOperationAction(ISD::FP_TO_FP16, VT, Legal);
+
+      setLoadExtAction(ISD::EXTLOAD, VT, MVT::f16, Legal);
+      setTruncStoreAction(VT,            MVT::f16, Legal);
+    }
+  }
+
+  // Specify operation actions if Power9 altivec enabled.
+  if (Subtarget.hasP9Altivec()) {
+    setOperationAction(ISD::INSERT_VECTOR_ELT, MVT::v8i16, Custom);
+    setOperationAction(ISD::INSERT_VECTOR_ELT, MVT::v16i8, Custom);
+
+    for (MVT VT : {MVT::v4i8, MVT::v4i16, MVT::v4i32, MVT::v2i8, MVT::v2i16,
+                   MVT::v2i32, MVT::v2i64})
+      setOperationAction(ISD::SIGN_EXTEND_INREG, VT, Legal);
+
+    for (MVT VT : MVT::fixedlen_vector_valuetypes())
+      // Vector instructions introduced in P9
+      if (VT.SimpleTy != MVT::v1i128)
+        setOperationAction(ISD::CTTZ, VT, Legal);
+  }
+
+  // Specify operation actions if Power9 vector enabled.
+  if (Subtarget.hasP9Vector()) {
+
+    setOperationAction(ISD::FADD,       MVT::f128, Legal);
+    setOperationAction(ISD::FSUB,       MVT::f128, Legal);
+    setOperationAction(ISD::FDIV,       MVT::f128, Legal);
+    setOperationAction(ISD::FMUL,       MVT::f128, Legal);
+    setOperationAction(ISD::FP_EXTEND,  MVT::f128, Legal);
+    setOperationAction(ISD::FMA,        MVT::f128, Legal);
+    setOperationAction(ISD::FTRUNC,     MVT::f128, Legal);
+    setOperationAction(ISD::FRINT,      MVT::f128, Legal);
+    setOperationAction(ISD::FFLOOR,     MVT::f128, Legal);
+    setOperationAction(ISD::FCEIL,      MVT::f128, Legal);
+    setOperationAction(ISD::FROUND,     MVT::f128, Legal);
 
-      setOperationAction(ISD::SELECT, MVT::f128, Expand);
-      setOperationAction(ISD::FP_ROUND, MVT::f64, Legal);
-      setOperationAction(ISD::FP_ROUND, MVT::f32, Legal);
-      setTruncStoreAction(MVT::f128, MVT::f64, Expand);
-      setTruncStoreAction(MVT::f128, MVT::f32, Expand);
-      setOperationAction(ISD::BITCAST, MVT::i128, Custom);
-      // No implementation for these ops for PowerPC.
-      setOperationAction(ISD::FSIN, MVT::f128, Expand);
-      setOperationAction(ISD::FCOS, MVT::f128, Expand);
-      setOperationAction(ISD::FPOW, MVT::f128, Expand);
-      setOperationAction(ISD::FPOWI, MVT::f128, Expand);
-      setOperationAction(ISD::FREM, MVT::f128, Expand);
+    setOperationAction(ISD::FSIN,    MVT::f128, Expand);
+    setOperationAction(ISD::FCOS,    MVT::f128, Expand);
+    setOperationAction(ISD::FPOW,    MVT::f128, Expand);
+    setOperationAction(ISD::FPOWI,   MVT::f128, Expand);
+    setOperationAction(ISD::FREM,    MVT::f128, Expand);
+    setOperationAction(ISD::SELECT,  MVT::f128, Expand);
+    setOperationAction(ISD::BITCAST, MVT::i128, Custom);
 
-      // Handle constrained floating-point operations of fp128
-      setOperationAction(ISD::STRICT_FADD, MVT::f128, Legal);
-      setOperationAction(ISD::STRICT_FSUB, MVT::f128, Legal);
-      setOperationAction(ISD::STRICT_FMUL, MVT::f128, Legal);
-      setOperationAction(ISD::STRICT_FDIV, MVT::f128, Legal);
-      setOperationAction(ISD::STRICT_FMA, MVT::f128, Legal);
-      setOperationAction(ISD::STRICT_FSQRT, MVT::f128, Legal);
-      setOperationAction(ISD::STRICT_FP_EXTEND, MVT::f128, Legal);
-      setOperationAction(ISD::STRICT_FP_ROUND, MVT::f64, Legal);
-      setOperationAction(ISD::STRICT_FP_ROUND, MVT::f32, Legal);
-      setOperationAction(ISD::STRICT_FRINT, MVT::f128, Legal);
-      setOperationAction(ISD::STRICT_FNEARBYINT, MVT::f128, Legal);
-      setOperationAction(ISD::STRICT_FFLOOR, MVT::f128, Legal);
-      setOperationAction(ISD::STRICT_FCEIL, MVT::f128, Legal);
-      setOperationAction(ISD::STRICT_FTRUNC, MVT::f128, Legal);
-      setOperationAction(ISD::STRICT_FROUND, MVT::f128, Legal);
-      setOperationAction(ISD::FP_EXTEND, MVT::v2f32, Custom);
-      setOperationAction(ISD::BSWAP, MVT::v8i16, Legal);
-      setOperationAction(ISD::BSWAP, MVT::v4i32, Legal);
-      setOperationAction(ISD::BSWAP, MVT::v2i64, Legal);
-      setOperationAction(ISD::BSWAP, MVT::v1i128, Legal);
-    } else if (Subtarget.hasAltivec() && EnableSoftFP128) {
-      addRegisterClass(MVT::f128, &PPC::VRRCRegClass);
-
-      for (MVT FPT : MVT::fp_valuetypes())
-        setLoadExtAction(ISD::EXTLOAD, MVT::f128, FPT, Expand);
+    setOperationAction(ISD::STRICT_FADD,       MVT::f128, Legal);
+    setOperationAction(ISD::STRICT_FSUB,       MVT::f128, Legal);
+    setOperationAction(ISD::STRICT_FMUL,       MVT::f128, Legal);
+    setOperationAction(ISD::STRICT_FDIV,       MVT::f128, Legal);
+    setOperationAction(ISD::STRICT_FMA,        MVT::f128, Legal);
+    setOperationAction(ISD::STRICT_FSQRT,      MVT::f128, Legal);
+    setOperationAction(ISD::STRICT_FP_EXTEND,  MVT::f128, Legal);
+    setOperationAction(ISD::STRICT_FRINT,      MVT::f128, Legal);
+    setOperationAction(ISD::STRICT_FFLOOR,     MVT::f128, Legal);
+    setOperationAction(ISD::STRICT_FCEIL,      MVT::f128, Legal);
+    setOperationAction(ISD::STRICT_FTRUNC,     MVT::f128, Legal);
+    setOperationAction(ISD::STRICT_FROUND,     MVT::f128, Legal);
+
+    setOperationAction(ISD::FP_ROUND,        MVT::f64, Legal);
+    setOperationAction(ISD::FP_ROUND,        MVT::f32, Legal);
+    setOperationAction(ISD::STRICT_FP_ROUND, MVT::f64, Legal);
+    setOperationAction(ISD::STRICT_FP_ROUND, MVT::f32, Legal);
+
+    setOperationAction(ISD::BSWAP, MVT::i64,    Custom);
+    setOperationAction(ISD::BSWAP, MVT::v8i16,  Legal);
+    setOperationAction(ISD::BSWAP, MVT::v4i32,  Legal);
+    setOperationAction(ISD::BSWAP, MVT::v2i64,  Legal);
 
-      setOperationAction(ISD::LOAD, MVT::f128, Promote);
-      setOperationAction(ISD::STORE, MVT::f128, Promote);
+    setOperationAction(ISD::FP_EXTEND,         MVT::v2f32, Custom);
+    setOperationAction(ISD::INSERT_VECTOR_ELT, MVT::v4i32, Custom);
+    setOperationAction(ISD::INSERT_VECTOR_ELT, MVT::v4f32, Custom);
 
-      AddPromotedToType(ISD::LOAD, MVT::f128, MVT::v4i32);
-      AddPromotedToType(ISD::STORE, MVT::f128, MVT::v4i32);
+    // 128 bit shifts can be accomplished via 3 instructions for SHL and
+    // SRL, but not for SRA because of the instructions available:
+    // VS{RL} and VS{RL}O.
+    setOperationAction(ISD::SHL, MVT::v1i128, Legal);
+    setOperationAction(ISD::SRL, MVT::v1i128, Legal);
+    setOperationAction(ISD::SRA, MVT::v1i128, Expand);
 
-      // Set FADD/FSUB as libcall to avoid the legalizer to expand the
-      // fp_to_uint and int_to_fp.
-      setOperationAction(ISD::FADD, MVT::f128, LibCall);
-      setOperationAction(ISD::FSUB, MVT::f128, LibCall);
+    setCondCodeAction(ISD::SETULT, MVT::f128, Expand);
+    setCondCodeAction(ISD::SETUGT, MVT::f128, Expand);
+    setCondCodeAction(ISD::SETUEQ, MVT::f128, Expand);
+    setCondCodeAction(ISD::SETOGE, MVT::f128, Expand);
+    setCondCodeAction(ISD::SETOLE, MVT::f128, Expand);
+    setCondCodeAction(ISD::SETONE, MVT::f128, Expand);
 
-      setOperationAction(ISD::FMUL, MVT::f128, Expand);
-      setOperationAction(ISD::FDIV, MVT::f128, Expand);
-      setOperationAction(ISD::FNEG, MVT::f128, Expand);
-      setOperationAction(ISD::FABS, MVT::f128, Expand);
-      setOperationAction(ISD::FSIN, MVT::f128, Expand);
-      setOperationAction(ISD::FCOS, MVT::f128, Expand);
-      setOperationAction(ISD::FPOW, MVT::f128, Expand);
-      setOperationAction(ISD::FPOWI, MVT::f128, Expand);
-      setOperationAction(ISD::FREM, MVT::f128, Expand);
-      setOperationAction(ISD::FSQRT, MVT::f128, Expand);
-      setOperationAction(ISD::FMA, MVT::f128, Expand);
-      setOperationAction(ISD::FCOPYSIGN, MVT::f128, Expand);
+    // No extending loads to f128 on PPC.
+    for (MVT FPT : MVT::fp_valuetypes())
+      setLoadExtAction(ISD::EXTLOAD, MVT::f128, FPT, Expand);
 
-      setTruncStoreAction(MVT::f128, MVT::f64, Expand);
-      setTruncStoreAction(MVT::f128, MVT::f32, Expand);
+  }
 
-      // Expand the fp_extend if the target type is fp128.
-      setOperationAction(ISD::FP_EXTEND, MVT::f128, Expand);
-      setOperationAction(ISD::STRICT_FP_EXTEND, MVT::f128, Expand);
+  // Specify the operation actions with crbits enabled.
+  if (Subtarget.useCRBits()) {
+    addRegisterClass(MVT::i1, &PPC::CRBITRCRegClass);
 
-      // Expand the fp_round if the source type is fp128.
-      for (MVT VT : {MVT::f32, MVT::f64}) {
-        setOperationAction(ISD::FP_ROUND, VT, Custom);
-        setOperationAction(ISD::STRICT_FP_ROUND, VT, Custom);
-      }
+    if (Subtarget.isPPC64() || Subtarget.hasFPCVT()) {
+      MVT VT = Subtarget.isPPC64() ? MVT::i64 : MVT::i32;
+      setOperationAction(ISD::SINT_TO_FP,        MVT::i1, Promote);
+      setOperationAction(ISD::UINT_TO_FP,        MVT::i1, Promote);
+      setOperationAction(ISD::FP_TO_SINT,        MVT::i1, Promote);
+      setOperationAction(ISD::FP_TO_UINT,        MVT::i1, Promote);
+      setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i1, Promote);
+      setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i1, Promote);
+      setOperationAction(ISD::STRICT_FP_TO_SINT, MVT::i1, Promote);
+      setOperationAction(ISD::STRICT_FP_TO_UINT, MVT::i1, Promote);
+
+      AddPromotedToType(ISD::SINT_TO_FP,         MVT::i1, VT);
+      AddPromotedToType(ISD::UINT_TO_FP,         MVT::i1, VT);
+      AddPromotedToType(ISD::FP_TO_SINT,         MVT::i1, VT);
+      AddPromotedToType(ISD::FP_TO_UINT,         MVT::i1, VT);
+      AddPromotedToType(ISD::STRICT_UINT_TO_FP,  MVT::i1, VT);
+      AddPromotedToType(ISD::STRICT_SINT_TO_FP,  MVT::i1, VT);
+      AddPromotedToType(ISD::STRICT_FP_TO_SINT,  MVT::i1, VT);
+      AddPromotedToType(ISD::STRICT_FP_TO_UINT,  MVT::i1, VT);
+
+    } else {
+      setOperationAction(ISD::STRICT_SINT_TO_FP, MVT::i1, Custom);
+      setOperationAction(ISD::STRICT_UINT_TO_FP, MVT::i1, Custom);
+      setOperationAction(ISD::SINT_TO_FP,        MVT::i1, Custom);
+      setOperationAction(ISD::UINT_TO_FP,        MVT::i1, Custom);
     }
 
-    if (Subtarget.hasP9Altivec()) {
-      setOperationAction(ISD::INSERT_VECTOR_ELT, MVT::v8i16, Custom);
-      setOperationAction(ISD::INSERT_VECTOR_ELT, MVT::v16i8, Custom);
+    // PowerPC does not support direct load/store of condition registers.
+    setOperationAction(ISD::LOAD,  MVT::i1, Custom);
+    setOperationAction(ISD::STORE, MVT::i1, Custom);
+
+    setOperationAction(ISD::SELECT, MVT::i32, Legal);
+    setOperationAction(ISD::SELECT, MVT::i64, Legal);
+    setOperationAction(ISD::SELECT, MVT::f32, Legal);
+    setOperationAction(ISD::SELECT, MVT::f64, Legal);
+    if (Subtarget.hasAltivec())
+      setOperationAction(ISD::SELECT, MVT::v4i32, Legal);
 
-      setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::v4i8,  Legal);
-      setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::v4i16, Legal);
-      setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::v4i32, Legal);
-      setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::v2i8,  Legal);
-      setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::v2i16, Legal);
-      setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::v2i32, Legal);
-      setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::v2i64, Legal);
+    setOperationAction(ISD::SETCC,  MVT::i32,   Legal);
+    setOperationAction(ISD::BRCOND, MVT::Other, Legal);
+
+    setOperationAction(ISD::SIGN_EXTEND_INREG, MVT::i1, Expand);
+
+    // FIXME: Remove this once the ANDI glue bug is fixed:
+    if (ANDIGlueBug)
+      setOperationAction(ISD::TRUNCATE, MVT::i1, Custom);
+
+    for (MVT VT : MVT::integer_valuetypes()) {
+      setLoadExtAction(ISD::SEXTLOAD, VT, MVT::i1, Promote);
+      setLoadExtAction(ISD::ZEXTLOAD, VT, MVT::i1, Promote);
+      setTruncStoreAction(VT, MVT::i1, Expand);
     }
   }
 
   if (Subtarget.pairedVectorMemops()) {
     addRegisterClass(MVT::v256i1, &PPC::VSRpRCRegClass);
-    setOperationAction(ISD::LOAD, MVT::v256i1, Custom);
+
+    setOperationAction(ISD::LOAD,  MVT::v256i1, Custom);
     setOperationAction(ISD::STORE, MVT::v256i1, Custom);
   }
+
   if (Subtarget.hasMMA()) {
     addRegisterClass(MVT::v512i1, &PPC::UACCRCRegClass);
-    setOperationAction(ISD::LOAD, MVT::v512i1, Custom);
-    setOperationAction(ISD::STORE, MVT::v512i1, Custom);
+
+    setOperationAction(ISD::LOAD,         MVT::v512i1, Custom);
+    setOperationAction(ISD::STORE,        MVT::v512i1, Custom);
     setOperationAction(ISD::BUILD_VECTOR, MVT::v512i1, Custom);
   }
 
-  if (Subtarget.has64BitSupport())
-    setOperationAction(ISD::PREFETCH, MVT::Other, Legal);
-
   if (Subtarget.isISA3_1())
     setOperationAction(ISD::SRA, MVT::v1i128, Legal);
 
-  setOperationAction(ISD::READCYCLECOUNTER, MVT::i64, isPPC64 ? Legal : Custom);
-
-  if (!isPPC64) {
-    setOperationAction(ISD::ATOMIC_LOAD,  MVT::i64, Expand);
-    setOperationAction(ISD::ATOMIC_STORE, MVT::i64, Expand);
-  }
-
   setBooleanContents(ZeroOrOneBooleanContent);
 
   if (Subtarget.hasAltivec()) {
     // Altivec instructions set fields to all zeros or all ones.
     setBooleanVectorContents(ZeroOrNegativeOneBooleanContent);
   }
 
-  if (!isPPC64) {
+  if (!Subtarget.isPPC64()) {
     // These libcalls are not available in 32-bit.
     setLibcallName(RTLIB::SHL_I128, nullptr);
     setLibcallName(RTLIB::SRL_I128, nullptr);
     setLibcallName(RTLIB::SRA_I128, nullptr);
   }
 
-  if (!isPPC64)
+  if (!Subtarget.isPPC64())
     setMaxAtomicSizeInBitsSupported(32);
 
-  setStackPointerRegisterToSaveRestore(isPPC64 ? PPC::X1 : PPC::R1);
+  setStackPointerRegisterToSaveRestore(Subtarget.isPPC64() ? PPC::X1 : PPC::R1);
 
   // We have target-specific dag combine patterns for the following nodes:
   setTargetDAGCombine(ISD::ADD);
   setTargetDAGCombine(ISD::SHL);
   setTargetDAGCombine(ISD::SRA);
   setTargetDAGCombine(ISD::SRL);
   setTargetDAGCombine(ISD::MUL);
   setTargetDAGCombine(ISD::FMA);