[PowerPC] Enable CR bits support for Power8 and above.

[amyk]

Summary

This patch turns on support for CR bit accesses for Power8 and above. The reason why CR bits are turned
on as the default for Power8 and above is that because later architectures make use of builtins and
instructions that require CR bit accesses (such as the use of setbc in the vector string isolate predicate
and bcd builtins on Power10).

This patch also adds the clang portion to allow for turning on CR bits in the front end if the user so desires to.

[shchenz]

This patch turns on support for CR bit accesses for Power8 and above. The reason why CR bits are turned on as the default for Power8 and above is that because later architectures make use of builtins and instructions that require CR bit accesses (such as the use of setbc in the vector string isolate predicate and bcd builtins on Power10).

Maybe we also can add some comments in docs/ClangCommandLineReference.rst to explicitly say that -mcrbits will be default to on when PowerPC arch is no smaller than 8. I believe some cr-bit operations also exist on Power7 or even Power6?

clang/lib/Basic/Targets/PPC.cpp

520

If we set the +crbits by the arch name, do we still need the customization (Turn on crbits for O2 and above) in computeFSAdditions()?

clang/test/Driver/ppc-crbits.cpp

51

Do we need some cases for AIX?

[amyk]

Address review comments of adding documentation to clang/docs/ClangCommandLineReference.rst and adding AIX checks to clang/test/Driver/ppc-crbits.cpp.

[amyk]

clang/lib/Basic/Targets/PPC.cpp

520

Yeah, that's a good point. I looked into this previously, and it appears that addition of -mcrbits inside computeFSAdditions() may still be necessary.

In particular, we have test cases that test pre-POWER8 with optimizations on (or, if no optimization level is provided, then -O2 is assumed the default). In these cases, much of the code changes if the customization inside computeFSAdditions() is removed because we would no longer be using crbits pre-P8.

clang/test/Driver/ppc-crbits.cpp

51

Thanks, I've added some.

[shchenz]

LGTM. Thanks for fixing.

clang/lib/Basic/Targets/PPC.cpp

520

hmm, OK, then we still have the same issue that this patch fixes for pre-POWER8. I'm ok with leaving it for now as IMO Power7/Power6 should not be major versions for PowerPC.

[nemanjai]

clang/lib/Basic/Targets/PPC.cpp

520

Up until Power8, the only instructions we had that require crbits were CR-logicals (which existed since the POWER architecture - i.e. before PowerPC). In Power8, we added BCD instructions that also require crbits. So that's why we turn it on for Power8 and up.

[nemanjai]

clang/docs/ClangCommandLineReference.rst

3578

... is the default for POWER8 and above as well as for all other CPUs when optimization is applied (-O2 and above).

But please check if it is also applied at -O1 and correct that statement accordingly.

llvm/test/CodeGen/PowerPC/fast-isel-fcmp-nan.ll

5

Why do we not emit the VSX instructions here any longer? How are crbits related?

[shchenz]

clang/lib/Basic/Targets/PPC.cpp

520

Fail enough.

[shchenz]

clang/lib/Basic/Targets/PPC.cpp

520

haha, I mean fair....

[amyk]

llvm/test/CodeGen/PowerPC/fast-isel-fcmp-nan.ll

5

Nemanja and I discussed this outside of the revision. This test case is ran with FastISel by default.

It looks like for fcmp ULT/UEQ/UGT/OLE/ONE/OGE, we're not able to find a compare predicate, so we don't use FastIISel for these changed test cases.
With my patch (turning on CR bits), we end up matching to fcmpu in the td patterns, but when CR bits are not present, we match to the VSX instructions in PPCISelDAGToDAG.
This should be something we investigate at a later date.

clang/docs/ClangCommandLineReference.rst

 PowerPC
 -------
 .. option:: -maltivec, -mno-altivec
 
 .. option:: -mcmpb, -mno-cmpb
 
 .. option:: -mcrbits, -mno-crbits
 
+Control the CR-bit tracking feature on PowerPC. ``-mcrbits`` (the enablement of CR-bit tracking support) is the default for POWER8 and above, as well as for all other CPUs when optimization is applied (-O2 and above).
+
 .. option:: -mcrypto, -mno-crypto
 
 .. option:: -mdirect-move, -mno-direct-move
 
 .. option:: -mefpu2
 
 .. option:: -mfloat128, -mno-float128
 

clang/lib/Basic/Targets/PPC.h

class LLVM_LIBRARY_VISIBILITY PPCTargetInfo : public TargetInfo {
   enum PPCFloatABI { HardFloat, SoftFloat } FloatABI;
 
   // Target cpu features.
   bool HasAltivec = false;
   bool HasMMA = false;
   bool HasROPProtect = false;
   bool HasPrivileged = false;
   bool HasVSX = false;
+  bool UseCRBits = false;
   bool HasP8Vector = false;
   bool HasP8Crypto = false;
   bool HasDirectMove = false;
   bool HasHTM = false;
   bool HasBPERMD = false;
   bool HasExtDiv = false;
   bool HasP9Vector = false;
   bool HasSPE = false;

clang/lib/Basic/Targets/PPC.cpp

 bool PPCTargetInfo::handleTargetFeatures(std::vector<std::string> &Features,
                                          DiagnosticsEngine &Diags) {
   FloatABI = HardFloat;
   for (const auto &Feature : Features) {
     if (Feature == "+altivec") {
       HasAltivec = true;
     } else if (Feature == "+vsx") {
       HasVSX = true;
+    } else if (Feature == "+crbits") {
+      UseCRBits = true;
     } else if (Feature == "+bpermd") {
       HasBPERMD = true;
     } else if (Feature == "+extdiv") {
       HasExtDiv = true;
     } else if (Feature == "+power8-vector") {
       HasP8Vector = true;
     } else if (Feature == "+crypto") {
       HasP8Crypto = true;
Features["extdiv"] = llvm::StringSwitch<bool>(CPU)
                            .Case("pwr8", true)
                            .Case("pwr7", true)
                            .Default(false);
   Features["direct-move"] = llvm::StringSwitch<bool>(CPU)
                                 .Case("ppc64le", true)
                                 .Case("pwr9", true)
                                 .Case("pwr8", true)
                                 .Default(false);
+  Features["crbits"] = llvm::StringSwitch<bool>(CPU)
+                                .Case("ppc64le", true)
+                                .Case("pwr9", true)
+                                .Case("pwr8", true)
+                                .Default(false);
   Features["vsx"] = llvm::StringSwitch<bool>(CPU)
                         .Case("ppc64le", true)
                         .Case("pwr9", true)
                         .Case("pwr8", true)
                         .Case("pwr7", true)
                         .Default(false);
   Features["htm"] = llvm::StringSwitch<bool>(CPU)
                         .Case("ppc64le", true)

 void PPCTargetInfo::addFutureSpecificFeatures(
     llvm::StringMap<bool> &Features) const {}
 
 bool PPCTargetInfo::hasFeature(StringRef Feature) const {
   return llvm::StringSwitch<bool>(Feature)
       .Case("powerpc", true)
       .Case("altivec", HasAltivec)
       .Case("vsx", HasVSX)
+      .Case("crbits", UseCRBits)
       .Case("power8-vector", HasP8Vector)
       .Case("crypto", HasP8Crypto)
       .Case("direct-move", HasDirectMove)
       .Case("htm", HasHTM)
       .Case("bpermd", HasBPERMD)
       .Case("extdiv", HasExtDiv)
       .Case("float128", HasFloat128)
       .Case("power9-vector", HasP9Vector)

clang/test/Driver/ppc-crbits.cpp

+// RUN: %clang -target powerpc64le-unknown-linux-gnu %s -### -mcpu=pwr10 \
+// RUN:   -mcrbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-CRBITS %s
+// RUN: %clang -target powerpc64le-unknown-linux-gnu %s -### -mcpu=pwr10 \
+// RUN:   -mno-crbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-NOCRBITS %s
+
+// RUN: %clang -target powerpc64le-unknown-linux-gnu %s -### -mcpu=pwr9 \
+// RUN:   -mcrbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-CRBITS %s
+// RUN: %clang -target powerpc64le-unknown-linux-gnu %s -### -mcpu=pwr9 \
+// RUN:   -mno-crbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-NOCRBITS %s
+
+// RUN: %clang -target powerpc64le-unknown-linux-gnu %s -### -mcpu=pwr8 \
+// RUN:   -mcrbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-CRBITS %s
+// RUN: %clang -target powerpc64le-unknown-linux-gnu %s -### -mcpu=pwr8 \
+// RUN:   -mno-crbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-NOCRBITS %s
+
+// RUN: %clang -target powerpc64le-unknown-linux-gnu %s -### -mcpu=pwr7 \
+// RUN:   -mcrbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-CRBITS %s
+// RUN: %clang -target powerpc64le-unknown-linux-gnu %s -### -mcpu=pwr7 \
+// RUN:   -mno-crbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-NOCRBITS %s
+
+// RUN: %clang -target powerpc-ibm-aix %s -### -mcpu=pwr10 \
+// RUN:   -mcrbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-CRBITS %s
+// RUN: %clang -target powerpc-ibm-aix %s -### -mcpu=pwr10 \
+// RUN:   -mno-crbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-NOCRBITS %s
+
+// RUN: %clang -target powerpc-ibm-aix %s -### -mcpu=pwr9 \
+// RUN:   -mcrbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-CRBITS %s
+// RUN: %clang -target powerpc-ibm-aix %s -### -mcpu=pwr9 \
+// RUN:   -mno-crbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-NOCRBITS %s
+
+// RUN: %clang -target powerpc-ibm-aix %s -### -mcpu=pwr8 \
+// RUN:   -mcrbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-CRBITS %s
+// RUN: %clang -target powerpc-ibm-aix %s -### -mcpu=pwr8 \
+// RUN:   -mno-crbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-NOCRBITS %s
+
+// RUN: %clang -target powerpc-ibm-aix %s -### -mcpu=pwr7 \
+// RUN:   -mcrbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-CRBITS %s
+// RUN: %clang -target powerpc-ibm-aix %s -### -mcpu=pwr7 \
+// RUN:   -mno-crbits -o %t.o 2>&1 | FileCheck -check-prefix=CHECK-NOCRBITS %s
+
+
+// CHECK-NOCRBITS: "-target-feature" "-crbits"
+// CHECK-CRBITS: "-target-feature" "+crbits"
+
+
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -mcpu=pwr10 -emit-llvm \
+// RUN:   -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -mcpu=pwr10 -mcrbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -mcpu=pwr10 -mno-crbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-NOCRBITS
+
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -mcpu=pwr9 -emit-llvm \
+// RUN:   -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -mcpu=pwr9 -mcrbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -mcpu=pwr9 -mno-crbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-NOCRBITS
+
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -mcpu=pwr8 -emit-llvm \
+// RUN:   -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -mcpu=pwr8 -mcrbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -mcpu=pwr8 -mno-crbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-NOCRBITS
+
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -mcpu=pwr7 -emit-llvm \
+// RUN:   -S %s -o - | FileCheck %s --check-prefix=HAS-NOCRBITS
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -mcpu=pwr7 -mcrbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc64le-unknown-linux-gnu -mcpu=pwr7 -mno-crbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-NOCRBITS
+
+// RUN: %clang -target powerpc-ibm-aix -mcpu=pwr10 -emit-llvm \
+// RUN:   -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc-ibm-aix -mcpu=pwr10 -mcrbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc-ibm-aix -mcpu=pwr10 -mno-crbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-NOCRBITS
+
+// RUN: %clang -target powerpc-ibm-aix -mcpu=pwr9 -emit-llvm \
+// RUN:   -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc-ibm-aix -mcpu=pwr9 -mcrbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc-ibm-aix -mcpu=pwr9 -mno-crbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-NOCRBITS
+
+// RUN: %clang -target powerpc-ibm-aix -mcpu=pwr8 -emit-llvm \
+// RUN:   -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc-ibm-aix -mcpu=pwr8 -mcrbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc-ibm-aix -mcpu=pwr8 -mno-crbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-NOCRBITS
+
+// RUN: %clang -target powerpc-ibm-aix -mcpu=pwr7 -emit-llvm \
+// RUN:   -S %s -o - | FileCheck %s --check-prefix=HAS-NOCRBITS
+// RUN: %clang -target powerpc-ibm-aix -mcpu=pwr7 -mcrbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-CRBITS
+// RUN: %clang -target powerpc-ibm-aix -mcpu=pwr7 -mno-crbits \
+// RUN:   -emit-llvm -S %s -o - | FileCheck %s --check-prefix=HAS-NOCRBITS
+
+
+// HAS-CRBITS: main(
+// HAS-CRBITS: attributes #0 = {
+// HAS-CRBITS-SAME: +crbits
+// HAS-NOCRBITS: main(
+// HAS-NOCRBITS: attributes #0 = {
+// HAS-NOCRBITS-SAME: -crbits
+
+int main(int argc, char *argv[]) {
+  return 0;
+}

llvm/lib/Target/PowerPC/PPC.td

list<SubtargetFeature> P8AdditionalFeatures =
      FeatureP8Vector,
      FeatureP8Crypto,
      FeatureHTM,
      FeatureDirectMove,
      FeatureICBT,
      FeaturePartwordAtomic,
      FeatureQuadwordAtomic,
      FeaturePredictableSelectIsExpensive,
-     FeatureISA2_07
+     FeatureISA2_07,
+     FeatureCRBits
     ];
 
   list<SubtargetFeature> P8SpecificFeatures = [FeatureAddiLoadFusion,
                                                FeatureAddisLoadFusion];
   list<SubtargetFeature> P8InheritableFeatures =
     !listconcat(P7InheritableFeatures, P8AdditionalFeatures);
   list<SubtargetFeature> P8Features =
     !listconcat(P8InheritableFeatures, P8SpecificFeatures);

llvm/test/CodeGen/PowerPC/addegluecrash.ll

 ; CHECK-NEXT:    addc 6, 6, 7
 ; CHECK-NEXT:    addze 3, 3
 ; CHECK-NEXT:    ld 5, 8(5)
 ; CHECK-NEXT:    mulld 7, 5, 8
 ; CHECK-NEXT:    mulhdu 5, 5, 8
 ; CHECK-NEXT:    addc 6, 6, 7
 ; CHECK-NEXT:    addze 5, 5
 ; CHECK-NEXT:    add 3, 5, 3
-; CHECK-NEXT:    cmpld 7, 3, 5
-; CHECK-NEXT:    mfocrf 3, 1
-; CHECK-NEXT:    rlwinm 5, 3, 29, 31, 31
-; CHECK-NEXT:    # implicit-def: $x3
-; CHECK-NEXT:    mr 3, 5
-; CHECK-NEXT:    clrldi 3, 3, 32
+; CHECK-NEXT:    cmpld 3, 5
+; CHECK-NEXT:    crmove 20, 0
+; CHECK-NEXT:    li 5, 0
+; CHECK-NEXT:    li 3, 1
+; CHECK-NEXT:    isel 3, 3, 5, 20
 ; CHECK-NEXT:    std 3, 0(4)
 ; CHECK-NEXT:    blr
   %1 = load i64, i64* %a, align 8
   %conv = zext i64 %1 to i128
   %2 = load i64, i64* %b, align 8
   %conv2 = zext i64 %2 to i128
   %mul = mul nuw i128 %conv2, %conv
   %shr = lshr i128 %mul, 64

llvm/test/CodeGen/PowerPC/f128-branch-cond.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=powerpc64le-unknown-linux -mcpu=pwr8 -O0 < %s | \
 ; RUN:   FileCheck %s -check-prefix=P8
 ; RUN: llc -mtriple=powerpc64le-unknown-linux -mcpu=pwr9 -O0 < %s | \
 ; RUN:   FileCheck %s -check-prefix=P9
 
 define i32 @test_choice1(fp128 %a, fp128 %b) #0 {
 ; P8-LABEL: test_choice1:
 ; P8:       # %bb.0: # %entry
 ; P8-NEXT:    mflr 0
 ; P8-NEXT:    std 0, 16(1)
 ; P8-NEXT:    stdu 1, -112(1)
 ; P8-NEXT:    bl __nekf2
 ; P8-NEXT:    nop
 ; P8-NEXT:    # kill: def $r3 killed $r3 killed $x3
 ; P8-NEXT:    cmplwi 3, 0
-; P8-NEXT:    li 3, 0
-; P8-NEXT:    li 4, 1
-; P8-NEXT:    iseleq 3, 3, 4
-; P8-NEXT:    cmplwi 3, 0
 ; P8-NEXT:    bne 0, .LBB0_2
 ; P8-NEXT:    b .LBB0_1
 ; P8-NEXT:  .LBB0_1: # %if.true
 ; P8-NEXT:    bl foo
 ; P8-NEXT:    nop
 ; P8-NEXT:    stw 3, 108(1) # 4-byte Folded Spill
 ; P8-NEXT:    b .LBB0_3
 ; P8-NEXT:  .LBB0_2: # %if.false

 ; P8:       # %bb.0: # %entry
 ; P8-NEXT:    mflr 0
 ; P8-NEXT:    std 0, 16(1)
 ; P8-NEXT:    stdu 1, -112(1)
 ; P8-NEXT:    bl __lekf2
 ; P8-NEXT:    nop
 ; P8-NEXT:    # kill: def $r3 killed $r3 killed $x3
 ; P8-NEXT:    cmpwi 3, 1
-; P8-NEXT:    li 4, 0
-; P8-NEXT:    li 3, 1
-; P8-NEXT:    isellt 3, 3, 4
-; P8-NEXT:    cmplwi 3, 0
-; P8-NEXT:    bne 0, .LBB1_2
+; P8-NEXT:    blt 0, .LBB1_2
 ; P8-NEXT:    b .LBB1_1
 ; P8-NEXT:  .LBB1_1: # %if.true
 ; P8-NEXT:    bl foo
 ; P8-NEXT:    nop
 ; P8-NEXT:    stw 3, 108(1) # 4-byte Folded Spill
 ; P8-NEXT:    b .LBB1_3
 ; P8-NEXT:  .LBB1_2: # %if.false
 ; P8-NEXT:    bl bar
 ; P8-NEXT:    nop
 ; P8-NEXT:    stw 3, 108(1) # 4-byte Folded Spill
 ; P8-NEXT:  .LBB1_3: # %final
 ; P8-NEXT:    lwz 3, 108(1) # 4-byte Folded Reload
 ; P8-NEXT:    clrldi 3, 3, 32
 ; P8-NEXT:    addi 1, 1, 112
 ; P8-NEXT:    ld 0, 16(1)
 ; P8-NEXT:    mtlr 0
 ; P8-NEXT:    blr
 ;
 ; P9-LABEL: test_choice2:
 ; P9:       # %bb.0: # %entry
 ; P9-NEXT:    mflr 0
 ; P9-NEXT:    std 0, 16(1)
-; P9-NEXT:    stdu 1, -128(1)
-; P9-NEXT:    xscmpuqp 7, 2, 3
-; P9-NEXT:    mfocrf 3, 1
-; P9-NEXT:    rotlwi 3, 3, 28
-; P9-NEXT:    stw 3, 124(1)
-; P9-NEXT:    mfocrf 3, 1
-; P9-NEXT:    lwz 4, 124(1)
-; P9-NEXT:    rotlwi 4, 4, 4
-; P9-NEXT:    mtocrf 1, 4
-; P9-NEXT:    clrlwi 3, 3, 31
-; P9-NEXT:    xori 4, 3, 1
-; P9-NEXT:    mfocrf 3, 1
-; P9-NEXT:    rlwinm 3, 3, 30, 31, 31
-; P9-NEXT:    xori 3, 3, 1
-; P9-NEXT:    and 3, 3, 4
-; P9-NEXT:    cmplwi 3, 0
-; P9-NEXT:    bne 0, .LBB1_2
+; P9-NEXT:    stdu 1, -112(1)
+; P9-NEXT:    xscmpuqp 0, 2, 3
+; P9-NEXT:    crmove 20, 3
+; P9-NEXT:    crnot 21, 20
+; P9-NEXT:    crmove 20, 1
+; P9-NEXT:    crnot 20, 20
+; P9-NEXT:    crand 20, 20, 21
+; P9-NEXT:    crxor 21, 21, 21
+; P9-NEXT:    crxor 20, 20, 21
+; P9-NEXT:    bc 12, 20, .LBB1_2
 ; P9-NEXT:    b .LBB1_1
 ; P9-NEXT:  .LBB1_1: # %if.true
 ; P9-NEXT:    bl foo
 ; P9-NEXT:    nop
-; P9-NEXT:    stw 3, 120(1) # 4-byte Folded Spill
+; P9-NEXT:    stw 3, 108(1) # 4-byte Folded Spill
 ; P9-NEXT:    b .LBB1_3
 ; P9-NEXT:  .LBB1_2: # %if.false
 ; P9-NEXT:    bl bar
 ; P9-NEXT:    nop
-; P9-NEXT:    stw 3, 120(1) # 4-byte Folded Spill
+; P9-NEXT:    stw 3, 108(1) # 4-byte Folded Spill
 ; P9-NEXT:  .LBB1_3: # %final
-; P9-NEXT:    lwz 3, 120(1) # 4-byte Folded Reload
+; P9-NEXT:    lwz 3, 108(1) # 4-byte Folded Reload
 ; P9-NEXT:    clrldi 3, 3, 32
-; P9-NEXT:    addi 1, 1, 128
+; P9-NEXT:    addi 1, 1, 112
 ; P9-NEXT:    ld 0, 16(1)
 ; P9-NEXT:    mtlr 0
 ; P9-NEXT:    blr
 entry:
   %cmp = fcmp ugt fp128 %a, %b
   br i1 %cmp, label %if.true, label %if.false
 
 if.true:

 ; P8-LABEL: test_choice3:
 ; P8:       # %bb.0: # %entry
 ; P8-NEXT:    mflr 0
 ; P8-NEXT:    std 0, 16(1)
 ; P8-NEXT:    stdu 1, -112(1)
 ; P8-NEXT:    bl __ltkf2
 ; P8-NEXT:    nop
 ; P8-NEXT:    # kill: def $r3 killed $r3 killed $x3
-; P8-NEXT:    not 3, 3
-; P8-NEXT:    srwi 3, 3, 31
-; P8-NEXT:    cmplwi 3, 0
-; P8-NEXT:    bne 0, .LBB2_2
+; P8-NEXT:    cmpwi 3, -1
+; P8-NEXT:    bgt 0, .LBB2_2
 ; P8-NEXT:    b .LBB2_1
 ; P8-NEXT:  .LBB2_1: # %if.true
 ; P8-NEXT:    bl foo
 ; P8-NEXT:    nop
 ; P8-NEXT:    stw 3, 108(1) # 4-byte Folded Spill
 ; P8-NEXT:    b .LBB2_3
 ; P8-NEXT:  .LBB2_2: # %if.false
 ; P8-NEXT:    bl bar

llvm/test/CodeGen/PowerPC/fast-isel-fcmp-nan.ll

 ; RUN: llc -verify-machineinstrs -mtriple powerpc64le-unknown-linux-gnu -fast-isel -O0 < %s | FileCheck %s
 
 define i1 @TestULT(double %t0) {
 ; CHECK-LABEL: TestULT:
-; CHECK: xscmpudp
+; CHECK: fcmpu
 ; CHECK: blr
 entry:
   %t1 = fcmp ult double %t0, 0.000000e+00
   br i1 %t1, label %good, label %bad
 
 bad:
   ret i1 false
 
bad:
   ret i1 false
 
 good:
   ret i1 true
 }
 
 define i1 @TestUEQ(double %t0) {
 ; CHECK-LABEL: TestUEQ:
-; CHECK: xscmpudp
+; CHECK: fcmpu
 ; CHECK: blr
 entry:
   %t1 = fcmp ueq double %t0, 0.000000e+00
   br i1 %t1, label %good, label %bad
 
 bad:
   ret i1 false
 
 good:
   ret i1 true
 }
 
 define i1 @TestUGT(double %t0) {
 ; CHECK-LABEL: TestUGT:
-; CHECK: xscmpudp
+; CHECK: fcmpu
 ; CHECK: blr
 entry:
   %t1 = fcmp ugt double %t0, 0.000000e+00
   br i1 %t1, label %good, label %bad
 
 bad:
   ret i1 false
 
bad:
   ret i1 false
 
 good:
   ret i1 true
 }
 
 define i1 @TestOLE(double %t0) {
 ; CHECK-LABEL: TestOLE:
-; CHECK: xscmpudp
+; CHECK: fcmpu
 ; CHECK: blr
 entry:
   %t1 = fcmp ole double %t0, 0.000000e+00
   br i1 %t1, label %good, label %bad
 
 bad:
   ret i1 false
 
 good:
   ret i1 true
 }
 
 define i1 @TestONE(double %t0) {
 ; CHECK-LABEL: TestONE:
-; CHECK: xscmpudp
+; CHECK: fcmpu
 ; CHECK: blr
 entry:
   %t1 = fcmp one double %t0, 0.000000e+00
   br i1 %t1, label %good, label %bad
 
 bad:
   ret i1 false
 
bad:
   ret i1 false
 
 good:
   ret i1 true
 }
 
 define i1 @TestOGE(double %t0) {
 ; CHECK-LABEL: TestOGE:
-; CHECK: xscmpudp
+; CHECK: fcmpu
 ; CHECK: blr
 entry:
   %t1 = fcmp oge double %t0, 0.000000e+00
   br i1 %t1, label %good, label %bad
 
 bad:
   ret i1 false
 
 good:
   ret i1 true
 }

llvm/test/CodeGen/PowerPC/fp-strict-fcmp-noopt.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs -ppc-asm-full-reg-names -ppc-vsr-nums-as-vr \
 ; RUN:   < %s -mtriple=powerpc64-unknown-linux -mcpu=pwr9 -O0 | FileCheck %s
 
 define i32 @une_ppcf128(ppc_fp128 %a, ppc_fp128 %b) #0 {
 ; CHECK-LABEL: une_ppcf128:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    xscmpudp cr7, f1, f3
-; CHECK-NEXT:    mfocrf r3, 1
-; CHECK-NEXT:    rlwinm r3, r3, 31, 31, 31
-; CHECK-NEXT:    xscmpudp cr7, f2, f4
-; CHECK-NEXT:    mfocrf r4, 1
-; CHECK-NEXT:    rlwinm r4, r4, 31, 31, 31
-; CHECK-NEXT:    xori r4, r4, 1
-; CHECK-NEXT:    and r4, r3, r4
-; CHECK-NEXT:    xscmpudp cr7, f1, f3
-; CHECK-NEXT:    mfocrf r3, 1
-; CHECK-NEXT:    rlwinm r3, r3, 31, 31, 31
-; CHECK-NEXT:    xori r3, r3, 1
-; CHECK-NEXT:    xscmpudp cr7, f1, f3
-; CHECK-NEXT:    mfocrf r5, 1
-; CHECK-NEXT:    rlwinm r5, r5, 31, 31, 31
-; CHECK-NEXT:    xori r5, r5, 1
-; CHECK-NEXT:    and r3, r3, r5
-; CHECK-NEXT:    or r3, r3, r4
-; CHECK-NEXT:    # kill: def $r4 killed $r3
+; CHECK-NEXT:    fcmpu cr0, f1, f3
+; CHECK-NEXT:    crmove 4*cr5+lt, eq
+; CHECK-NEXT:    fcmpu cr1, f2, f4
+; CHECK-NEXT:    crmove 4*cr5+gt, 4*cr1+eq
+; CHECK-NEXT:    crnot 4*cr5+gt, 4*cr5+gt
+; CHECK-NEXT:    crand 4*cr5+gt, 4*cr5+lt, 4*cr5+gt
+; CHECK-NEXT:    crmove 4*cr5+lt, eq
+; CHECK-NEXT:    crnot 4*cr5+lt, 4*cr5+lt
+; CHECK-NEXT:    crand 4*cr5+lt, 4*cr5+lt, 4*cr5+lt
+; CHECK-NEXT:    cror 4*cr5+lt, 4*cr5+lt, 4*cr5+gt
+; CHECK-NEXT:    li r4, 0
+; CHECK-NEXT:    li r3, 1
+; CHECK-NEXT:    isel r3, r3, r4, 4*cr5+lt
 ; CHECK-NEXT:    clrldi r3, r3, 32
 ; CHECK-NEXT:    blr
 entry:
   %0 = call i1 @llvm.experimental.constrained.fcmp.ppcf128(ppc_fp128 %a, ppc_fp128 %b, metadata !"une", metadata !"fpexcept.strict") #0
   %1 = zext i1 %0 to i32
   ret i32 %1
 }
 
 ; This is a different branch from une
 define i32 @ogt_ppcf128(ppc_fp128 %a, ppc_fp128 %b) #0 {
 ; CHECK-LABEL: ogt_ppcf128:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    xscmpudp cr7, f1, f3
-; CHECK-NEXT:    mfocrf r3, 1
-; CHECK-NEXT:    rlwinm r3, r3, 31, 31, 31
-; CHECK-NEXT:    xscmpudp cr7, f2, f4
-; CHECK-NEXT:    mfocrf r4, 1
-; CHECK-NEXT:    rlwinm r4, r4, 30, 31, 31
-; CHECK-NEXT:    and r4, r3, r4
-; CHECK-NEXT:    xscmpudp cr0, f1, f3
-; CHECK-NEXT:    mfocrf r3, 128
-; CHECK-NEXT:    stw r3, -4(r1)
-; CHECK-NEXT:    xscmpudp cr7, f1, f3
-; CHECK-NEXT:    mfocrf r3, 1
-; CHECK-NEXT:    lwz r5, -4(r1)
-; CHECK-NEXT:    rotlwi r5, r5, 4
-; CHECK-NEXT:    mtocrf 1, r5
-; CHECK-NEXT:    rlwinm r5, r3, 30, 31, 31
-; CHECK-NEXT:    mfocrf r3, 1
-; CHECK-NEXT:    rlwinm r3, r3, 31, 31, 31
-; CHECK-NEXT:    xori r3, r3, 1
-; CHECK-NEXT:    and r3, r3, r5
-; CHECK-NEXT:    or r3, r3, r4
-; CHECK-NEXT:    # kill: def $r4 killed $r3
+; CHECK-NEXT:    fcmpu cr0, f1, f3
+; CHECK-NEXT:    crmove 4*cr5+lt, eq
+; CHECK-NEXT:    fcmpu cr1, f2, f4
+; CHECK-NEXT:    crmove 4*cr5+gt, 4*cr1+gt
+; CHECK-NEXT:    crand 4*cr5+gt, 4*cr5+lt, 4*cr5+gt
+; CHECK-NEXT:    crmove 4*cr5+lt, eq
+; CHECK-NEXT:    crnot 4*cr5+lt, 4*cr5+lt
+; CHECK-NEXT:    crmove 4*cr5+eq, gt
+; CHECK-NEXT:    crand 4*cr5+lt, 4*cr5+lt, 4*cr5+eq
+; CHECK-NEXT:    cror 4*cr5+lt, 4*cr5+lt, 4*cr5+gt
+; CHECK-NEXT:    li r4, 0
+; CHECK-NEXT:    li r3, 1
+; CHECK-NEXT:    isel r3, r3, r4, 4*cr5+lt
 ; CHECK-NEXT:    clrldi r3, r3, 32
 ; CHECK-NEXT:    blr
 entry:
   %0 = call i1 @llvm.experimental.constrained.fcmp.ppcf128(ppc_fp128 %a, ppc_fp128 %b, metadata !"ogt", metadata !"fpexcept.strict") #0
   %1 = zext i1 %0 to i32
   ret i32 %1
 }
 
 define i1 @test_f128(fp128 %a, fp128 %b) #0 {
 ; CHECK-LABEL: test_f128:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    xscmpuqp cr7, v2, v3
-; CHECK-NEXT:    mfocrf r3, 1
-; CHECK-NEXT:    rlwinm r3, r3, 31, 31, 31
-; CHECK-NEXT:    xori r4, r3, 1
-; CHECK-NEXT:    # implicit-def: $x3
-; CHECK-NEXT:    mr r3, r4
+; CHECK-NEXT:    xscmpuqp cr0, v2, v3
+; CHECK-NEXT:    crmove 4*cr5+lt, eq
+; CHECK-NEXT:    xscmpuqp cr0, v2, v3
+; CHECK-NEXT:    crmove 4*cr5+gt, eq
+; CHECK-NEXT:    crnor 4*cr5+lt, 4*cr5+lt, 4*cr5+gt
+; CHECK-NEXT:    li r4, 0
+; CHECK-NEXT:    li r3, 1
+; CHECK-NEXT:    isel r3, r3, r4, 4*cr5+lt
 ; CHECK-NEXT:    blr
 entry:
   %0 = call i1 @llvm.experimental.constrained.fcmp.f128(fp128 %a, fp128 %b, metadata !"une", metadata !"fpexcept.strict") #0
   ret i1 %0
 }
 
 define i1 @testbr_f64(double %a, double %b) #0 {
 ; CHECK-LABEL: testbr_f64:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    xscmpudp cr7, f1, f2
-; CHECK-NEXT:    mfocrf r3, 1
-; CHECK-NEXT:    rlwinm r3, r3, 31, 31, 31
-; CHECK-NEXT:    cmplwi r3, 0
-; CHECK-NEXT:    bne cr0, .LBB3_2
+; CHECK-NEXT:    fcmpu cr0, f1, f2
+; CHECK-NEXT:    crmove 4*cr5+lt, eq
+; CHECK-NEXT:    bc 12, 4*cr5+lt, .LBB3_2
 ; CHECK-NEXT:    b .LBB3_1
 ; CHECK-NEXT:  .LBB3_1: # %tr
 ; CHECK-NEXT:    li r3, -1
 ; CHECK-NEXT:    blr
 ; CHECK-NEXT:  .LBB3_2: # %fl
 ; CHECK-NEXT:    li r3, 0
 ; CHECK-NEXT:    blr
 entry:
   %0 = call i1 @llvm.experimental.constrained.fcmp.f64(double %a, double %b, metadata !"une", metadata !"fpexcept.strict") #0
   br i1 %0, label %tr, label %fl
 tr:
   ret i1 true
 fl:
   ret i1 false
 }
 
 define i1 @testbr_f32(float %a, float %b) #0 {
 ; CHECK-LABEL: testbr_f32:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    fcmpu cr7, f1, f2
-; CHECK-NEXT:    mfocrf r3, 1
-; CHECK-NEXT:    rlwinm r3, r3, 31, 31, 31
-; CHECK-NEXT:    cmplwi r3, 0
-; CHECK-NEXT:    bne cr0, .LBB4_2
+; CHECK-NEXT:    fcmpu cr0, f1, f2
+; CHECK-NEXT:    crmove 4*cr5+lt, eq
+; CHECK-NEXT:    bc 12, 4*cr5+lt, .LBB4_2
 ; CHECK-NEXT:    b .LBB4_1
 ; CHECK-NEXT:  .LBB4_1: # %tr
 ; CHECK-NEXT:    li r3, -1
 ; CHECK-NEXT:    blr
 ; CHECK-NEXT:  .LBB4_2: # %fl
 ; CHECK-NEXT:    li r3, 0
 ; CHECK-NEXT:    blr
 entry:

llvm/test/CodeGen/PowerPC/fp64-to-int16.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -O0 < %s | FileCheck %s
 target triple = "powerpc64le--linux-gnu"
 
 define i1 @Test(double %a) {
 ; CHECK-LABEL: Test:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    xscvdpsxws 0, 1
 ; CHECK-NEXT:    mffprwz 3, 0
-; CHECK-NEXT:    xori 3, 3, 65534
-; CHECK-NEXT:    cntlzw 3, 3
-; CHECK-NEXT:    srwi 4, 3, 5
-; CHECK-NEXT:    # implicit-def: $x3
-; CHECK-NEXT:    mr 3, 4
+; CHECK-NEXT:    cmplwi 3, 65534
+; CHECK-NEXT:    crmove 20, 2
+; CHECK-NEXT:    li 4, 0
+; CHECK-NEXT:    li 3, 1
+; CHECK-NEXT:    isel 3, 3, 4, 20
 ; CHECK-NEXT:    blr
 entry:
   %conv = fptoui double %a to i16
   %cmp = icmp eq i16 %conv, -2
   ret i1 %cmp
 }

llvm/test/CodeGen/PowerPC/pcrel-byte-loads.ll

entry:
   %0 = load i1, i1* getelementptr inbounds ([20 x i1], [20 x i1]* @GlobLd1, i64 0, i64 0), align 1
   ret i1 %0
 }
 
 define void @i32_ZextLoad_i1() {
 ; CHECK-LE-LABEL: i32_ZextLoad_i1:
 ; CHECK-LE:       # %bb.0: # %entry
 ; CHECK-LE-NEXT:    plbz r3, GlobLd1@PCREL(0), 1
+; CHECK-LE-NEXT:    clrldi r3, r3, 63
 ; CHECK-LE-NEXT:    pstb r3, GlobSt1@PCREL(0), 1
 ; CHECK-LE-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: i32_ZextLoad_i1:
 ; CHECK-BE:       # %bb.0: # %entry
 ; CHECK-BE-NEXT:    addis r3, r2, GlobLd1@toc@ha
 ; CHECK-BE-NEXT:    addis r4, r2, GlobSt1@toc@ha
 ; CHECK-BE-NEXT:    lbz r3, GlobLd1@toc@l(r3)
+; CHECK-BE-NEXT:    clrldi r3, r3, 63
 ; CHECK-BE-NEXT:    stb r3, GlobSt1@toc@l(r4)
 ; CHECK-BE-NEXT:    blr
 entry:
   %0 = load i1, i1* getelementptr inbounds ([20 x i1], [20 x i1]* @GlobLd1, i64 0, i64 0), align 1
   store i1 %0, i1* getelementptr inbounds ([20 x i1], [20 x i1]* @GlobSt1, i64 0, i64 0), align 1
   ret void
 }
 
 %1 = type { i64 }
 @Glob1 = external dso_local global %1, align 8
 @Glob2 = external dso_local unnamed_addr constant [11 x i8], align 1
 declare i32 @Decl(%1*, i8*) local_unnamed_addr #0
 
 define dso_local i1 @i32_ExtLoad_i1() local_unnamed_addr #0 {
 ; CHECK-LE-LABEL: i32_ExtLoad_i1:
 ; CHECK-LE:       # %bb.0: # %bb
 ; CHECK-LE-NEXT:    mflr r0
 ; CHECK-LE-NEXT:    std r0, 16(r1)
 ; CHECK-LE-NEXT:    stdu r1, -32(r1)
 ; CHECK-LE-NEXT:    .cfi_def_cfa_offset 32
 ; CHECK-LE-NEXT:    .cfi_offset lr, 16
 ; CHECK-LE-NEXT:    paddi r3, 0, Glob1@PCREL, 1
 ; CHECK-LE-NEXT:    paddi r4, 0, Glob2@PCREL, 1
 ; CHECK-LE-NEXT:    bl Decl@notoc
-; CHECK-LE-NEXT:    plbz r4, GlobLd1@PCREL(0), 1
-; CHECK-LE-NEXT:    cmplwi r3, 0
-; CHECK-LE-NEXT:    li r3, 1
-; CHECK-LE-NEXT:    iseleq r3, 0, r3
-; CHECK-LE-NEXT:    and r3, r3, r4
+; CHECK-LE-NEXT:    cmpwi cr1, r3, 0
+; CHECK-LE-NEXT:    plbz r3, GlobLd1@PCREL(0), 1
+; CHECK-LE-NEXT:    andi. r3, r3, 1
+; CHECK-LE-NEXT:    crandc 4*cr5+lt, gt, 4*cr1+eq
+; CHECK-LE-NEXT:    setbc r3, 4*cr5+lt
 ; CHECK-LE-NEXT:    addi r1, r1, 32
 ; CHECK-LE-NEXT:    ld r0, 16(r1)
 ; CHECK-LE-NEXT:    mtlr r0
 ; CHECK-LE-NEXT:    blr
 ;
 ; CHECK-BE-LABEL: i32_ExtLoad_i1:
 ; CHECK-BE:       # %bb.0: # %bb
 ; CHECK-BE-NEXT:    mflr r0
 ; CHECK-BE-NEXT:    std r0, 16(r1)
 ; CHECK-BE-NEXT:    stdu r1, -112(r1)
 ; CHECK-BE-NEXT:    .cfi_def_cfa_offset 112
 ; CHECK-BE-NEXT:    .cfi_offset lr, 16
 ; CHECK-BE-NEXT:    addis r3, r2, Glob1@toc@ha
 ; CHECK-BE-NEXT:    addis r4, r2, Glob2@toc@ha
 ; CHECK-BE-NEXT:    addi r3, r3, Glob1@toc@l
 ; CHECK-BE-NEXT:    addi r4, r4, Glob2@toc@l
 ; CHECK-BE-NEXT:    bl Decl
 ; CHECK-BE-NEXT:    nop
-; CHECK-BE-NEXT:    addis r4, r2, GlobLd1@toc@ha
-; CHECK-BE-NEXT:    cmplwi r3, 0
-; CHECK-BE-NEXT:    li r3, 1
-; CHECK-BE-NEXT:    lbz r4, GlobLd1@toc@l(r4)
-; CHECK-BE-NEXT:    iseleq r3, 0, r3
-; CHECK-BE-NEXT:    and r3, r3, r4
+; CHECK-BE-NEXT:    cmpwi cr1, r3, 0
+; CHECK-BE-NEXT:    addis r3, r2, GlobLd1@toc@ha
+; CHECK-BE-NEXT:    lbz r3, GlobLd1@toc@l(r3)
+; CHECK-BE-NEXT:    andi. r3, r3, 1
+; CHECK-BE-NEXT:    crandc 4*cr5+lt, gt, 4*cr1+eq
+; CHECK-BE-NEXT:    setbc r3, 4*cr5+lt
 ; CHECK-BE-NEXT:    addi r1, r1, 112
 ; CHECK-BE-NEXT:    ld r0, 16(r1)
 ; CHECK-BE-NEXT:    mtlr r0
 ; CHECK-BE-NEXT:    blr
 bb:
   %i = call signext i32 @Decl(%1* nonnull dereferenceable(32) @Glob1, i8* getelementptr inbounds ([11 x i8], [11 x i8]* @Glob2, i64 0, i64 0)) #1
   %i1 = icmp eq i32 %i, 0
   %i2 = load i1, i1* getelementptr inbounds ([20 x i1], [20 x i1]* @GlobLd1, i64 0, i64 0), align 1
   %i3 = select i1 %i1, i1 false, i1 %i2
   ret i1 %i3
 }