[PowerPC] Store, load, move from and to registers related builtins

[Conanap]

Summary

This patch implements store, load, move from and to registers related
builtins. The patch aims to provide feature parady with xlC on AIX.

[lei]

please add sema checking for pwr8 builtins.

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-LoadReserve-StoreCond.ll

80

remove all reference to attributes, local_unnamed_addr #[0..9] since it's not in the IR.

[Conanap]

Added more sema checking, test case update

[lei]

clang/lib/Sema/SemaChecking.cpp

3407

need tests for these.

llvm/include/llvm/IR/IntrinsicsPowerPC.td

1568

do these loads not need IntrReadMem?

[nemanjai]

We have encountered an issue with lwarx/ldarx that required that they emit inline asm rather than an intrinsic. What makes lbarx/lharx different?

clang/lib/Sema/SemaChecking.cpp

3406

Why do these need the extdiv feature?

[Conanap]

Added more tests, corrected sema checking and intrinsic flag

[lei]

llvm/include/llvm/IR/IntrinsicsPowerPC.td

1686–1706

nit: un-related line deletion

llvm/lib/Target/PowerPC/PPCInstrInfo.td

5493

EXTSH should not be needed and we should not be using xoaddr

llvm/test/CodeGen/builtins-ppc-xlcompat-move-tofrom-regs.ll

8

this is confusing... maybe this shouldjust be CHECK-32BIT

[Conanap]

Changed xoaddr, removed extws, changed check prefix

[Conanap]

Changed more xoaddr to ForceXForm

[nemanjai]

Taking this off the review queue until lharx/lbarx are changed to emit inline asm in line with lwarx/ldarx.

[Conanap]

Updated lharx and lbarx to inline asm implementation, implemented stfiw.

[nemanjai]

This is getting close to approval. The newly added __stfiw needs to be fixed and some nits need to be addressed.

clang/lib/Sema/SemaChecking.cpp

3405

This is not correct. The instruction (non-VSX version) has existed since Power3. The VSX version was added in Power8. No changes to the instruction came in Power9 so I have no idea where the decision to add this check came from.

In fact, this would also blow up in the back end if you compiled with something like -mcpu=pwr9 -mno-altivec or -mcpu=pwr9 -mno-vsx.

llvm/include/llvm/IR/IntrinsicsPowerPC.td

1567

Nit: line too long.

1576

Nit: indentation is inconsistent here.

llvm/lib/Target/PowerPC/PPCInstrVSX.td

4085

This needs the non-VSX pattern as well.

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-stfiw.ll

2

One of the run lines should be with -mattr=-vsx.

[Conanap]

Added non-vsx pattern for stfiw, extra testline for that pattern,
some nits

[Conanap]

Updated a test case

[nemanjai]

llvm/lib/Target/PowerPC/PPCInstrVSX.td

4085

I just realized this is in the wrong place. STXSIWX was added in Power8 not in Power9.

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-stfiw.ll

11

Don't make all of these pwr9. Make some of them pwr7 and some pwr8.

[lei]

clang/test/CodeGen/builtins-ppc-xlcompat-LoadReseve-StoreCond.c

16

where is the check for CHECK-NON-PWR8-ERR:?

37

CHECK-NON-PWR8-ERR: check?

clang/test/CodeGen/builtins-ppc-xlcompat-stfiw.c

3

why are we only testing this for pwr9 vs pwr7/8 similar to other tests added?

[Conanap]

Updated test cases

[Conanap]

Addressed comments

clang/test/CodeGen/builtins-ppc-xlcompat-LoadReseve-StoreCond.c

16

lwarx and stwcx are both available before power8, so the check is not needed.

[Conanap]

Moved pattern to a more appropriate place, updated test cases

[nemanjai]

LGTM.

[Conanap]

Changed flags for intrinsic of dcbtt and dcbtstt

[Conanap]

Fixed a typo

[Conanap]

I'm aware of the getting target issue; the fix will be up soon.

[Conanap]

issue should be fixed now; pushed with this: https://reviews.llvm.org/D106130#change-PZi4uueeCg9i
(I just had to move the test files into the PowerPC folder).

Will continue to monitor

[NeHuang]

@jroelofs committed f6769b663a0d4432b5e00e0c03904a5dfba7b077 to move the backend test cases from CodeGen -> CodeGen/PowerPC so they don't fail when the PPC backend isn't built.

clang/include/clang/Basic/BuiltinsPPC.def

 BUILTIN(__builtin_ppc_fetch_and_and, "UiUiD*Ui", "")
 BUILTIN(__builtin_ppc_fetch_and_andlp, "ULiULiD*ULi", "")
 BUILTIN(__builtin_ppc_fetch_and_or, "UiUiD*Ui", "")
 BUILTIN(__builtin_ppc_fetch_and_orlp, "ULiULiD*ULi", "")
 BUILTIN(__builtin_ppc_fetch_and_swap, "UiUiD*Ui", "")
 BUILTIN(__builtin_ppc_fetch_and_swaplp, "ULiULiD*ULi", "")
 BUILTIN(__builtin_ppc_ldarx, "LiLiD*", "")
 BUILTIN(__builtin_ppc_lwarx, "iiD*", "")
+BUILTIN(__builtin_ppc_lharx, "isD*", "")
+BUILTIN(__builtin_ppc_lbarx, "UiUcD*", "")
 BUILTIN(__builtin_ppc_stdcx, "iLiD*Li", "")
 BUILTIN(__builtin_ppc_stwcx, "iiD*i", "")
+BUILTIN(__builtin_ppc_sthcx, "isD*s", "")
 BUILTIN(__builtin_ppc_tdw, "vLLiLLiIUi", "")
 BUILTIN(__builtin_ppc_tw, "viiIUi", "")
 BUILTIN(__builtin_ppc_trap, "vi", "")
 BUILTIN(__builtin_ppc_trapd, "vLi", "")
 BUILTIN(__builtin_ppc_fcfid, "dd", "")
 BUILTIN(__builtin_ppc_fcfud, "dd", "")
 BUILTIN(__builtin_ppc_fctid, "dd", "")
 BUILTIN(__builtin_ppc_fctidz, "dd", "")

 BUILTIN(__builtin_ppc_fmsub, "dddd", "")
 BUILTIN(__builtin_ppc_fmsubs, "ffff", "")
 BUILTIN(__builtin_ppc_fnmadd, "dddd", "")
 BUILTIN(__builtin_ppc_fnmadds, "ffff", "")
 BUILTIN(__builtin_ppc_fnmsub, "dddd", "")
 BUILTIN(__builtin_ppc_fnmsubs, "ffff", "")
 BUILTIN(__builtin_ppc_fre, "dd", "")
 BUILTIN(__builtin_ppc_fres, "ff", "")
+BUILTIN(__builtin_ppc_dcbtstt, "vv*", "")
+BUILTIN(__builtin_ppc_dcbtt, "vv*", "")
+BUILTIN(__builtin_ppc_mftbu, "Ui","")
+BUILTIN(__builtin_ppc_mfmsr, "Ui", "")
+BUILTIN(__builtin_ppc_stfiw, "viC*d", "")
 
 BUILTIN(__builtin_ppc_get_timebase, "ULLi", "n")
 
 // This is just a placeholder, the types and attributes are wrong.
 BUILTIN(__builtin_altivec_vaddcuw, "V4UiV4UiV4Ui", "")
 
 BUILTIN(__builtin_altivec_vaddsbs, "V16ScV16ScV16Sc", "")
 BUILTIN(__builtin_altivec_vaddubs, "V16UcV16UcV16Uc", "")

clang/lib/Basic/Targets/PPC.cpp

static void defineXLCompatMacros(MacroBuilder &Builder) {
   Builder.defineMacro("__fetch_and_and", "__builtin_ppc_fetch_and_and");
   Builder.defineMacro("__fetch_and_andlp", "__builtin_ppc_fetch_and_andlp");
   Builder.defineMacro("__fetch_and_or", "__builtin_ppc_fetch_and_or");
   Builder.defineMacro("__fetch_and_orlp", "__builtin_ppc_fetch_and_orlp");
   Builder.defineMacro("__fetch_and_swap", "__builtin_ppc_fetch_and_swap");
   Builder.defineMacro("__fetch_and_swaplp", "__builtin_ppc_fetch_and_swaplp");
   Builder.defineMacro("__ldarx", "__builtin_ppc_ldarx");
   Builder.defineMacro("__lwarx", "__builtin_ppc_lwarx");
+  Builder.defineMacro("__lharx", "__builtin_ppc_lharx");
+  Builder.defineMacro("__lbarx", "__builtin_ppc_lbarx");
+  Builder.defineMacro("__stfiw", "__builtin_ppc_stfiw");
   Builder.defineMacro("__stdcx", "__builtin_ppc_stdcx");
   Builder.defineMacro("__stwcx", "__builtin_ppc_stwcx");
+  Builder.defineMacro("__sthcx", "__builtin_ppc_sthcx");
   Builder.defineMacro("__tdw", "__builtin_ppc_tdw");
   Builder.defineMacro("__tw", "__builtin_ppc_tw");
   Builder.defineMacro("__trap", "__builtin_ppc_trap");
   Builder.defineMacro("__trapd", "__builtin_ppc_trapd");
   Builder.defineMacro("__fcfid", "__builtin_ppc_fcfid");
   Builder.defineMacro("__fcfud", "__builtin_ppc_fcfud");
   Builder.defineMacro("__fctid", "__builtin_ppc_fctid");
   Builder.defineMacro("__fctidz", "__builtin_ppc_fctidz");
static void defineXLCompatMacros(MacroBuilder &Builder) {
   Builder.defineMacro("__popcnt4", "__builtin_popcount");
   Builder.defineMacro("__popcnt8", "__builtin_popcountll");
   Builder.defineMacro("__readflm", "__builtin_readflm");
   Builder.defineMacro("__rotatel4", "__builtin_rotateleft32");
   Builder.defineMacro("__rotatel8", "__builtin_rotateleft64");
   Builder.defineMacro("__rdlam", "__builtin_ppc_rdlam");
   Builder.defineMacro("__setflm", "__builtin_setflm");
   Builder.defineMacro("__setrnd", "__builtin_setrnd");
+  Builder.defineMacro("__dcbtstt", "__builtin_ppc_dcbtstt");
+  Builder.defineMacro("__dcbtt", "__builtin_ppc_dcbtt");
+  Builder.defineMacro("__mftbu", "__builtin_ppc_mftbu");
+  Builder.defineMacro("__mfmsr", "__builtin_ppc_mfmsr");
 }
 
 /// PPCTargetInfo::getTargetDefines - Return a set of the PowerPC-specific
 /// #defines that are not tied to a specific subtarget.
 void PPCTargetInfo::getTargetDefines(const LangOptions &Opts,
                                      MacroBuilder &Builder) const {
 
   defineXLCompatMacros(Builder);

clang/lib/CodeGen/CGBuiltin.cpp

static llvm::Value *emitPPCLoadReserveIntrinsic(CodeGenFunction &CGF,
   case clang::PPC::BI__builtin_ppc_ldarx:
     AsmOS << "ldarx ";
     RetType = CGF.Int64Ty;
     break;
   case clang::PPC::BI__builtin_ppc_lwarx:
     AsmOS << "lwarx ";
     RetType = CGF.Int32Ty;
     break;
+  case clang::PPC::BI__builtin_ppc_lharx:
+    AsmOS << "lharx ";
+    RetType = CGF.Int16Ty;
+    break;
+  case clang::PPC::BI__builtin_ppc_lbarx:
+    AsmOS << "lbarx ";
+    RetType = CGF.Int8Ty;
+    break;
   default:
     llvm_unreachable("Expected only PowerPC load reserve intrinsics");
   }
 
   AsmOS << "$0, ${1:y}";
 
   std::string Constraints = "=r,*Z,~{memory}";
   std::string MachineClobbers = CGF.getTarget().getClobbers();
Value *Unpacked = Builder.CreateBitCast(
         Ops[0], llvm::FixedVectorType::get(ConvertType(E->getType()), 2));
 
     if (getTarget().isLittleEndian())
       Index = ConstantInt::get(Index->getType(), 1 - Index->getZExtValue());
 
     return Builder.CreateExtractElement(Unpacked, Index);
   }
 
+  case PPC::BI__builtin_ppc_sthcx: {
+    llvm::Function *F = CGM.getIntrinsic(Intrinsic::ppc_sthcx);
+    Ops[0] = Builder.CreateBitCast(Ops[0], Int8PtrTy);
+    Ops[1] = Builder.CreateSExt(Ops[1], Int32Ty);
+    return Builder.CreateCall(F, Ops);
+  }
+
   // The PPC MMA builtins take a pointer to a __vector_quad as an argument.
   // Some of the MMA instructions accumulate their result into an existing
   // accumulator whereas the others generate a new accumulator. So we need to
   // use custom code generation to expand a builtin call with a pointer to a
   // load (if the corresponding instruction accumulates its result) followed by
   // the call to the intrinsic and a store of the result.
 #define CUSTOM_BUILTIN(Name, Intr, Types, Accumulate) \
   case PPC::BI__builtin_##Name:
#include "clang/Basic/BuiltinsPPC.def"
   }
   case PPC::BI__builtin_ppc_fetch_and_swap:
   case PPC::BI__builtin_ppc_fetch_and_swaplp: {
     return MakeBinaryAtomicValue(*this, AtomicRMWInst::Xchg, E,
                                  llvm::AtomicOrdering::Monotonic);
   }
   case PPC::BI__builtin_ppc_ldarx:
   case PPC::BI__builtin_ppc_lwarx:
+  case PPC::BI__builtin_ppc_lharx:
+  case PPC::BI__builtin_ppc_lbarx:
     return emitPPCLoadReserveIntrinsic(*this, BuiltinID, E);
   case PPC::BI__builtin_ppc_popcntb: {
     Value *ArgValue = EmitScalarExpr(E->getArg(0));
     llvm::Type *ArgType = ArgValue->getType();
     Function *F = CGM.getIntrinsic(Intrinsic::ppc_popcntb, {ArgType, ArgType});
     return Builder.CreateCall(F, Ops, "popcntb");
   }
   case PPC::BI__builtin_ppc_mtfsf: {

clang/lib/Sema/SemaChecking.cpp

bool Sema::CheckPPCBuiltinFunctionCall(const TargetInfo &TI, unsigned BuiltinID,
   case PPC::BI__builtin_ppc_tdw:
     return SemaBuiltinConstantArgRange(TheCall, 2, 0, 31);
   case PPC::BI__builtin_ppc_cmpeqb:
   case PPC::BI__builtin_ppc_setb:
   case PPC::BI__builtin_ppc_maddhd:
   case PPC::BI__builtin_ppc_maddhdu:
   case PPC::BI__builtin_ppc_maddld:
     return SemaFeatureCheck(*this, TheCall, "isa-v30-instructions",
                             diag::err_ppc_builtin_only_on_arch, "9");
   case PPC::BI__builtin_ppc_cmprb:
     return SemaFeatureCheck(*this, TheCall, "isa-v30-instructions",
                             diag::err_ppc_builtin_only_on_arch, "9") ||
            SemaBuiltinConstantArgRange(TheCall, 0, 0, 1);
   // For __rlwnm, __rlwimi and __rldimi, the last parameter mask must
   // be a constant that represents a contiguous bit field.
   case PPC::BI__builtin_ppc_rlwnm:
     return SemaBuiltinConstantArg(TheCall, 1, Result) ||
            SemaValueIsRunOfOnes(TheCall, 2);
   case PPC::BI__builtin_ppc_rlwimi:
return SemaBuiltinConstantArgRange(TheCall, 0, 0, 7) ||
            SemaBuiltinConstantArgRange(TheCall, 1, 0, 15);
   case PPC::BI__builtin_ppc_alignx:
     return SemaBuiltinConstantArgPower2(TheCall, 0);
   case PPC::BI__builtin_ppc_rdlam:
     return SemaValueIsRunOfOnes(TheCall, 2);
   case PPC::BI__builtin_ppc_icbt:
     return SemaFeatureCheck(*this, TheCall, "isa-v207-instructions",
                             diag::err_ppc_builtin_only_on_arch, "8");
+  case PPC::BI__builtin_ppc_sthcx:
+  case PPC::BI__builtin_ppc_lharx:
+  case PPC::BI__builtin_ppc_lbarx:
+    return SemaFeatureCheck(*this, TheCall, "isa-v207-instructions",
+                            diag::err_ppc_builtin_only_on_arch, "8");
 #define CUSTOM_BUILTIN(Name, Intr, Types, Acc) \
   case PPC::BI__builtin_##Name: \
     return SemaBuiltinPPCMMACall(TheCall, Types);
 #include "clang/Basic/BuiltinsPPC.def"
   }
   return SemaBuiltinConstantArgRange(TheCall, i, l, u);
 }
 

clang/test/CodeGen/builtins-ppc-xlcompat-LoadReseve-StoreCond.c

 // REQUIRES: powerpc-registered-target
-// RUN: %clang_cc1 -O2 -triple=powerpc-unknown-aix -emit-llvm %s -o - | \
-// RUN: FileCheck %s
-// RUN: %clang_cc1 -O2 -triple=powerpc64-unknown-aix -emit-llvm %s -o - | \
-// RUN: FileCheck %s
-// RUN: %clang_cc1 -O2 -triple=powerpc64le-unknown-unknown -emit-llvm %s \
-// RUN:  -o - | FileCheck %s
-// RUN: %clang_cc1 -O2 -triple=powerpc64-unknown-unknown -emit-llvm %s \
-// RUN:  -o - | FileCheck %s
+// RUN: %clang_cc1 -O2 -target-cpu pwr8 -triple=powerpc-unknown-aix \
+// RUN:  -emit-llvm %s -o - | FileCheck %s
+// RUN: %clang_cc1 -O2 -target-cpu pwr8 -triple=powerpc64-unknown-aix \
+// RUN:  -emit-llvm %s -o - | FileCheck %s
+// RUN: %clang_cc1 -O2 -target-cpu pwr8 -triple=powerpc64le-unknown-unknown \
+// RUN:  -emit-llvm %s -o - | FileCheck %s
+// RUN: %clang_cc1 -O2 -target-cpu pwr8 -triple=powerpc64-unknown-unknown \
+// RUN:  -emit-llvm %s -o - | FileCheck %s
+// RAUN: not %clang_cc1 -O2 -target-cpu pwr7 -triple=powerpc-unknown-aix \
+// RAUN:  -emit-llvm %s -o - 2>&1 | FileCheck %s \
+// RAUN:  --check-prefix=CHECK-NON-PWR8-ERR
 
 int test_lwarx(volatile int* a) {
-  // CHECK: @test_lwarx
+  // CHECK-LABEL: @test_lwarx
   // CHECK: %0 = tail call i32 asm sideeffect "lwarx $0, ${1:y}", "=r,*Z,~{memory}"(i32* %a)
   return __lwarx(a);
 }
+
+short test_lharx(volatile short *a) {
+  // CHECK-LABEL: @test_lharx
+  // CHECK: %0 = tail call i16 asm sideeffect "lharx $0, ${1:y}", "=r,*Z,~{memory}"(i16* %a)
+  // CHECK-NON-PWR8-ERR:  error: this builtin is only valid on POWER8 or later CPUs
+  return __lharx(a);
+}
+
+char test_lbarx(volatile unsigned char *a) {
+  // CHECK-LABEL: @test_lbarx
+  // CHECK: %0 = tail call i8 asm sideeffect "lbarx $0, ${1:y}", "=r,*Z,~{memory}"(i8* %a)
+  // CHECK-NON-PWR8-ERR:  error: this builtin is only valid on POWER8 or later CPUs
+  return __lbarx(a);
+}
+
 int test_stwcx(volatile int* a, int val) {
-  // CHECK: @test_stwcx
+  // CHECK-LABEL: @test_stwcx
   // CHECK: %0 = bitcast i32* %a to i8*
   // CHECK: %1 = tail call i32 @llvm.ppc.stwcx(i8* %0, i32 %val)
   return __stwcx(a, val);
 }
+
+int test_sthcx(volatile short *a, short val) {
+  // CHECK-LABEL: @test_sthcx
+  // CHECK: %0 = bitcast i16* %a to i8*
+  // CHECK: %1 = sext i16 %val to i32
+  // CHECK: %2 = tail call i32 @llvm.ppc.sthcx(i8* %0, i32 %1)
+  // CHECK-NON-PWR8-ERR:  error: this builtin is only valid on POWER8 or later CPUs
+  return __sthcx(a, val);
+}

clang/test/CodeGen/builtins-ppc-xlcompat-move-tofrom-regs.c

+// RUN: %clang_cc1 -O2 -triple powerpc64-unknown-unknown \
+// RUN:   -emit-llvm %s -o - -target-cpu pwr7 | FileCheck %s
+// RUN: %clang_cc1 -O2 -triple powerpc64le-unknown-unknown \
+// RUN:   -emit-llvm %s -o - -target-cpu pwr8 | FileCheck %s
+// RUN: %clang_cc1 -O2 -triple powerpc-unknown-aix \
+// RUN:   -emit-llvm %s -o - -target-cpu pwr7 | FileCheck %s
+// RUN: %clang_cc1 -O2 -triple powerpc64-unknown-aix \
+// RUN:   -emit-llvm %s -o - -target-cpu pwr7 | FileCheck %s
+
+unsigned int test_mftbu(void) {
+  // CHECK-LABEL: @test_mftbu
+  // CHECK: %0 = tail call i32 @llvm.ppc.mftbu()
+  return __mftbu();
+}
+
+unsigned long test_mfmsr(void) {
+  // CHECK-LABEL: @test_mfmsr
+  // CHECK: %0 = tail call i32 @llvm.ppc.mfmsr()
+  return __mfmsr();
+}

clang/test/CodeGen/builtins-ppc-xlcompat-prefetch.c

+// RUN: %clang_cc1 -O2 -triple powerpc64-unknown-unknown \
+// RUN:   -emit-llvm %s -o - -target-cpu pwr7 | FileCheck %s
+// RUN: %clang_cc1 -O2 -triple powerpc64le-unknown-unknown \
+// RUN:   -emit-llvm %s -o - -target-cpu pwr8 | FileCheck %s
+// RUN: %clang_cc1 -O2 -triple powerpc-unknown-aix \
+// RUN:   -emit-llvm %s -o - -target-cpu pwr7 | FileCheck %s
+// RUN: %clang_cc1 -O2 -triple powerpc64-unknown-aix \
+// RUN:   -emit-llvm %s -o - -target-cpu pwr7 | FileCheck %s
+
+extern void *vpa;
+
+void test_dcbtstt(void) {
+  // CHECK-LABEL: @test_dcbtstt
+  // CHECK: %0 = load i8*, i8** @vpa
+  // CHECK: tail call void @llvm.ppc.dcbtstt(i8* %0)
+  // CHECK: ret void
+  __dcbtstt(vpa);
+}
+
+void test_dcbtt(void) {
+  // CHECK-LABEL: @test_dcbt
+  // CHECK: %0 = load i8*, i8** @vpa
+  // CHECK: tail call void @llvm.ppc.dcbtt(i8* %0)
+  // CHECK: ret void
+  __dcbtt(vpa);
+}

clang/test/CodeGen/builtins-ppc-xlcompat-stfiw.c

+// RUN: %clang_cc1 -O2 -triple powerpc64-unknown-unknown \
+// RUN:   -emit-llvm %s -o - -target-cpu pwr7 | FileCheck %s
+// RUN: %clang_cc1 -O2 -triple powerpc64le-unknown-unknown \
+// RUN:   -emit-llvm %s -o - -target-cpu pwr8 | FileCheck %s
+// RUN: %clang_cc1 -O2 -triple powerpc-unknown-aix \
+// RUN:   -emit-llvm %s -o - -target-cpu pwr8 | FileCheck %s
+// RUN: %clang_cc1 -O2 -triple powerpc64-unknown-aix \
+// RUN:   -emit-llvm %s -o - -target-cpu pwr8 | FileCheck %s
+
+extern const int *cia;
+extern double da;
+
+void test_stfiw() {
+  // CHECK-LABEL: test_stfiw
+  // CHECK: void @llvm.ppc.stfiw(i8* %0, double %1)
+  __builtin_ppc_stfiw(cia, da);
+}
+
+void test_xl_stfiw() {
+  // CHECK-LABEL: test_xl_stfiw
+  // CHECK: void @llvm.ppc.stfiw(i8* %0, double %1)
+  __stfiw(cia, da);
+}

llvm/include/llvm/IR/IntrinsicsPowerPC.td

let TargetPrefix = "ppc" in {
   def int_ppc_iospace_eieio : GCCBuiltin<"__builtin_ppc_iospace_eieio">,
                               Intrinsic<[],[],[]>;
   def int_ppc_stdcx : GCCBuiltin<"__builtin_ppc_stdcx">,
                       Intrinsic<[llvm_i32_ty], [llvm_ptr_ty, llvm_i64_ty],
                                 [IntrWriteMem]>;
   def int_ppc_stwcx : GCCBuiltin<"__builtin_ppc_stwcx">,
                       Intrinsic<[llvm_i32_ty], [llvm_ptr_ty, llvm_i32_ty],
                                 [IntrWriteMem]>;
+  def int_ppc_sthcx
+      : Intrinsic<[llvm_i32_ty], [ llvm_ptr_ty, llvm_i32_ty ], [IntrWriteMem]>;
+  def int_ppc_dcbtstt : GCCBuiltin<"__builtin_ppc_dcbtstt">,
+                        Intrinsic<[], [llvm_ptr_ty],
+                                  [IntrArgMemOnly, NoCapture<ArgIndex<0>>]>;
+  def int_ppc_dcbtt : GCCBuiltin<"__builtin_ppc_dcbtt">,
+                      Intrinsic<[], [llvm_ptr_ty],
+                                [IntrArgMemOnly, NoCapture<ArgIndex<0>>]>;
+  def int_ppc_mftbu : GCCBuiltin<"__builtin_ppc_mftbu">,
+                      Intrinsic<[llvm_i32_ty], [], [IntrNoMem]>;
+  def int_ppc_mfmsr : GCCBuiltin<"__builtin_ppc_mfmsr">,
+                      Intrinsic<[llvm_i32_ty], [], [IntrNoMem]>;
+  def int_ppc_stfiw : GCCBuiltin<"__builtin_ppc_stfiw">,
+                      Intrinsic<[], [llvm_ptr_ty, llvm_double_ty],
+                                [IntrWriteMem]>;
   // compare
   def int_ppc_cmpeqb
       : GCCBuiltin<"__builtin_ppc_cmpeqb">,
         Intrinsic<[llvm_i64_ty], [llvm_i64_ty, llvm_i64_ty], [IntrNoMem]>;
   def int_ppc_cmprb
       : GCCBuiltin<"__builtin_ppc_cmprb">,
         Intrinsic<[llvm_i32_ty], [llvm_i32_ty, llvm_i32_ty, llvm_i32_ty],
                   [IntrNoMem, ImmArg<ArgIndex<0>>]>;
def int_ppc_fnmsubs
                    [IntrNoMem]>;
   def int_ppc_fre
       : GCCBuiltin<"__builtin_ppc_fre">,
         Intrinsic <[llvm_double_ty], [llvm_double_ty], [IntrNoMem]>;
   def int_ppc_fres
       : GCCBuiltin<"__builtin_ppc_fres">,
         Intrinsic <[llvm_float_ty], [llvm_float_ty], [IntrNoMem]>;
 }
 
 //===----------------------------------------------------------------------===//
 // PowerPC Atomic Intrinsic Definitions.
 let TargetPrefix = "ppc" in {
   class AtomicRMW128Intrinsic
     : Intrinsic<[llvm_i64_ty, llvm_i64_ty],
                 [llvm_ptr_ty, llvm_i64_ty, llvm_i64_ty],
                 [IntrArgMemOnly, NoCapture<ArgIndex<0>>]>;
   def int_ppc_atomicrmw_xchg_i128 : AtomicRMW128Intrinsic;
   def int_ppc_atomicrmw_add_i128  : AtomicRMW128Intrinsic;
   def int_ppc_atomicrmw_sub_i128  : AtomicRMW128Intrinsic;
   def int_ppc_atomicrmw_and_i128  : AtomicRMW128Intrinsic;
   def int_ppc_atomicrmw_or_i128   : AtomicRMW128Intrinsic;
   def int_ppc_atomicrmw_xor_i128  : AtomicRMW128Intrinsic;
   def int_ppc_atomicrmw_nand_i128 : AtomicRMW128Intrinsic;
   def int_ppc_cmpxchg_i128 : Intrinsic<[llvm_i64_ty, llvm_i64_ty],
                                        [llvm_ptr_ty,
                                         llvm_i64_ty, llvm_i64_ty,
                                         llvm_i64_ty, llvm_i64_ty],
                                        [IntrArgMemOnly, NoCapture<ArgIndex<0>>]>;
 }

llvm/lib/Target/PowerPC/PPC.td

list<SubtargetFeature> P8AdditionalFeatures =
      FeatureP8Altivec,
      FeatureP8Vector,
      FeatureP8Crypto,
      FeatureHTM,
      FeatureDirectMove,
      FeatureICBT,
      FeaturePartwordAtomic,
      FeatureQuadwordAtomic,
-     FeaturePredictableSelectIsExpensive
+     FeaturePredictableSelectIsExpensive,
+     FeatureISA2_07
     ];
 
   list<SubtargetFeature> P8SpecificFeatures = [FeatureAddiLoadFusion,
                                                FeatureAddisLoadFusion];
   list<SubtargetFeature> P8InheritableFeatures =
     !listconcat(P7InheritableFeatures, P8AdditionalFeatures);
   list<SubtargetFeature> P8Features =
     !listconcat(P8InheritableFeatures, P8SpecificFeatures);

llvm/lib/Target/PowerPC/PPCInstrInfo.td

 def : Pat<(int_ppc_fctiw f64:$A),
         (FCTIW $A)>;
 def : Pat<(int_ppc_fctiwz f64:$A),
         (XSCVDPSXWS $A)>;
 def : Pat<(int_ppc_fctudz f64:$A),
         (XSCVDPUXDS $A)>;
 def : Pat<(int_ppc_fctuwz f64:$A),
         (XSCVDPUXWS $A)>;
+
+def : Pat<(int_ppc_mfmsr), (MFMSR)>;
+def : Pat<(int_ppc_mftbu), (MFTB 269)>;
+
+let Predicates = [IsISA2_07] in {
+  def : Pat<(int_ppc_sthcx ForceXForm:$dst, gprc:$A),
+          (STHCX gprc:$A, ForceXForm:$dst)>;
+}
+def : Pat<(int_ppc_dcbtstt ForceXForm:$dst),
+          (DCBTST 16, ForceXForm:$dst)>;
+def : Pat<(int_ppc_dcbtt ForceXForm:$dst),
+          (DCBT 16, ForceXForm:$dst)>;
+
+def : Pat<(int_ppc_stfiw ForceXForm:$dst, f64:$XT),
+          (STFIWX f64:$XT, ForceXForm:$dst)>;

llvm/lib/Target/PowerPC/PPCInstrVSX.td

 def : Pat<(int_ppc_fnmadds f32:$A, f32:$B, f32:$C), (XSNMADDMSP $A, $B, $C)>;
 def : Pat<(int_ppc_fres f32:$A), (XSRESP $A)>;
 def : Pat<(i32 (int_ppc_extract_exp f64:$A)),
           (EXTRACT_SUBREG (XSXEXPDP (COPY_TO_REGCLASS $A, VSFRC)), sub_32)>;
 def : Pat<(int_ppc_extract_sig f64:$A),
           (XSXSIGDP (COPY_TO_REGCLASS $A, VSFRC))>;
 def : Pat<(f64 (int_ppc_insert_exp f64:$A, i64:$B)),
           (COPY_TO_REGCLASS (XSIEXPDP (COPY_TO_REGCLASS $A, G8RC), $B), F8RC)>;
+
+def : Pat<(int_ppc_stfiw ForceXForm:$dst, f64:$XT),
+          (STXSIWX f64:$XT, ForceXForm:$dst)>;
 } // HasVSX, HasP8Vector
 
 // Any big endian Power8 VSX subtarget.
 let Predicates = [HasVSX, HasP8Vector, IsBigEndian] in {
 def : Pat<DWToSPExtractConv.El0SS1,
           (f32 (XSCVSXDSP (COPY_TO_REGCLASS $S1, VSFRC)))>;
 def : Pat<DWToSPExtractConv.El1SS1,
           (f32 (XSCVSXDSP (COPY_TO_REGCLASS (XXPERMDI $S1, $S1, 2), VSFRC)))>;
defm : ScalToVecWPermute<
   (SUBREG_TO_REG
      (i64 1),
      (XSCVDPUXDS (COPY_TO_REGCLASS (DFLOADf32 DSForm:$A), VSFRC)), sub_64)>;
 def : Pat<(v4f32 (PPCldsplat ForceXForm:$A)),
           (v4f32 (LXVWSX ForceXForm:$A))>;
 def : Pat<(v4i32 (PPCldsplat ForceXForm:$A)),
           (v4i32 (LXVWSX ForceXForm:$A))>;
 } // HasVSX, HasP9Vector
 
 // Any Power9 VSX subtarget with equivalent length but better Power10 VSX
 // patterns.
 // Two identical blocks are required due to the slightly different predicates:
 // One without P10 instructions, the other is BigEndian only with P10 instructions.
 let Predicates = [HasVSX, HasP9Vector, NoP10Vector] in {
 // Little endian Power10 subtargets produce a shorter pattern but require a
 // COPY_TO_REGCLASS. The COPY_TO_REGCLASS makes it appear to need two instructions 
 // to perform the operation, when only one instruction is produced in practice.

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-LoadReserve-StoreCond.ll

 ; CHECK-32-NEXT:    mfocrf 3, 128
 ; CHECK-32-NEXT:    srwi 3, 3, 28
 ; CHECK-32-NEXT:    blr
 entry:
   %0 = bitcast i32* %a to i8*
   %1 = tail call i32 @llvm.ppc.stwcx(i8* %0, i32 %b)
   ret i32 %1
 }
+
+declare i32 @llvm.ppc.sthcx(i8*, i32)
+define dso_local signext i32 @test_sthcx(i16* %a, i16 signext %val) {
+; CHECK-64-LABEL: test_sthcx:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    sthcx. 4, 0, 3
+; CHECK-64-NEXT:    mfocrf 3, 128
+; CHECK-64-NEXT:    srwi 3, 3, 28
+; CHECK-64-NEXT:    extsw 3, 3
+; CHECK-64-NEXT:    blr
+;
+; CHECK-32-LABEL: test_sthcx:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    sthcx. 4, 0, 3
+; CHECK-32-NEXT:    mfocrf 3, 128
+; CHECK-32-NEXT:    srwi 3, 3, 28
+; CHECK-32-NEXT:    blr
+entry:
+  %0 = bitcast i16* %a to i8*
+  %1 = sext i16 %val to i32
+  %2 = tail call i32 @llvm.ppc.sthcx(i8* %0, i32 %1)
+  ret i32 %2
+}
+
+define dso_local signext i16 @test_lharx(i16* %a) {
+; CHECK-64-LABEL: test_lharx:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    #APP
+; CHECK-64-NEXT:    lharx 3, 0, 3
+; CHECK-64-NEXT:    #NO_APP
+; CHECK-64-NEXT:    extsh 3, 3
+; CHECK-64-NEXT:    blr
+;
+; CHECK-32-LABEL: test_lharx:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    #APP
+; CHECK-32-NEXT:    lharx 3, 0, 3
+; CHECK-32-NEXT:    #NO_APP
+; CHECK-32-NEXT:    extsh 3, 3
+; CHECK-32-NEXT:    blr
+entry:
+  %0 = tail call i16 asm sideeffect "lharx $0, ${1:y}", "=r,*Z,~{memory}"(i16* %a)
+  ret i16 %0
+}
+
+; Function Attrs: nounwind uwtable
+define dso_local zeroext i8 @test_lbarx(i8* %a) {
+; CHECK-64-LABEL: test_lbarx:
+; CHECK-64:       # %bb.0: # %entry
+; CHECK-64-NEXT:    #APP
+; CHECK-64-NEXT:    lbarx 3, 0, 3
+; CHECK-64-NEXT:    #NO_APP
+; CHECK-64-NEXT:    clrldi 3, 3, 56
+; CHECK-64-NEXT:    blr
+;
+; CHECK-32-LABEL: test_lbarx:
+; CHECK-32:       # %bb.0: # %entry
+; CHECK-32-NEXT:    #APP
+; CHECK-32-NEXT:    lbarx 3, 0, 3
+; CHECK-32-NEXT:    #NO_APP
+; CHECK-32-NEXT:    clrlwi 3, 3, 24
+; CHECK-32-NEXT:    blr
+entry:
+  %0 = tail call i8 asm sideeffect "lbarx $0, ${1:y}", "=r,*Z,~{memory}"(i8* %a)
+  ret i8 %0
+}

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-stfiw.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
+; RUN:   -mcpu=pwr8 < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
+; RUN:   -mcpu=pwr9 -mattr -vsx < %s | FileCheck %s --check-prefix=CHECK-NO-VSX
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
+; RUN:   -mcpu=pwr9 < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu \
+; RUN:   -mcpu=pwr7 < %s | FileCheck %s --check-prefix=CHECK-NO-VSX
+; RUN: llc -verify-machineinstrs -mtriple=powerpc-unknown-aix \
+; RUN:   -mcpu=pwr7 < %s | FileCheck %s --check-prefix=CHECK-32BIT
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-aix \
+; RUN:   -mcpu=pwr8 < %s | FileCheck %s
+
+declare void @llvm.ppc.stfiw(i8*, double)
+define dso_local void @test_stfiw(i32* %cia, double %da) {
+; CHECK-LABEL: test_stfiw:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    stxsiwx 1, 0, 3
+; CHECK-NEXT:    blr
+;
+; CHECK-NO-VSX-LABEL: test_stfiw:
+; CHECK-NO-VSX:       # %bb.0: # %entry
+; CHECK-NO-VSX-NEXT:    stfiwx 1, 0, 3
+; CHECK-NO-VSX-NEXT:    blr
+;
+; CHECK-32BIT-LABEL: test_stfiw:
+; CHECK-32BIT:       # %bb.0: # %entry
+; CHECK-32BIT-NEXT:    stfiwx 1, 0, 3
+; CHECK-32BIT-NEXT:    blr
+; CHECK-PWR9-LABEL: test_stfiw:
+; CHECK-PWR9:       # %bb.0: # %entry
+; CHECK-PWR9-NEXT:    stxsiwx 1, 0, 3
+; CHECK-PWR9-NEXT:    blr
+entry:
+  %0 = bitcast i32* %cia to i8*
+  tail call void @llvm.ppc.stfiw(i8* %0, double %da)
+  ret void
+}
+
+define dso_local void @test_xl_stfiw(i32* %cia, double %da) {
+; CHECK-LABEL: test_xl_stfiw:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    stxsiwx 1, 0, 3
+; CHECK-NEXT:    blr
+;
+; CHECK-NO-VSX-LABEL: test_xl_stfiw:
+; CHECK-NO-VSX:       # %bb.0: # %entry
+; CHECK-NO-VSX-NEXT:    stfiwx 1, 0, 3
+; CHECK-NO-VSX-NEXT:    blr
+;
+; CHECK-32BIT-LABEL: test_xl_stfiw:
+; CHECK-32BIT:       # %bb.0: # %entry
+; CHECK-32BIT-NEXT:    stfiwx 1, 0, 3
+; CHECK-32BIT-NEXT:    blr
+; CHECK-PWR9-LABEL: test_xl_stfiw:
+; CHECK-PWR9:       # %bb.0: # %entry
+; CHECK-PWR9-NEXT:    stxsiwx 1, 0, 3
+; CHECK-PWR9-NEXT:    blr
+entry:
+  %0 = bitcast i32* %cia to i8*
+  tail call void @llvm.ppc.stfiw(i8* %0, double %da)
+  ret void
+}

llvm/test/CodeGen/builtins-ppc-xlcompat-move-tofrom-regs.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
+; RUN:   -mcpu=pwr8 < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu \
+; RUN:   -mcpu=pwr7 < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc-unknown-aix \
+; RUN:   -mcpu=pwr7 < %s | FileCheck %s --check-prefix=CHECK-32BIT
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-aix \
+; RUN:   -mcpu=pwr7 < %s | FileCheck %s
+
+declare i32 @llvm.ppc.mftbu()
+declare i32 @llvm.ppc.mfmsr()
+
+define dso_local zeroext i32 @test_mftbu() {
+; CHECK-LABEL: test_mftbu:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    mftbu 3
+; CHECK-NEXT:    clrldi 3, 3, 32
+; CHECK-NEXT:    blr
+;
+; CHECK-32BIT-LABEL: test_mftbu:
+; CHECK-32BIT:       # %bb.0: # %entry
+; CHECK-32BIT-NEXT:    mftbu 3
+; CHECK-32BIT-NEXT:    blr
+entry:
+  %0 = tail call i32 @llvm.ppc.mftbu()
+  ret i32 %0
+}
+
+define dso_local i64 @test_mfmsr() {
+; CHECK-LABEL: test_mfmsr:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    mfmsr 3
+; CHECK-NEXT:    clrldi 3, 3, 32
+; CHECK-NEXT:    blr
+;
+; CHECK-32BIT-LABEL: test_mfmsr:
+; CHECK-32BIT:       # %bb.0: # %entry
+; CHECK-32BIT-NEXT:    mfmsr 4
+; CHECK-32BIT-NEXT:    li 3, 0
+; CHECK-32BIT-NEXT:    blr
+entry:
+  %0 = tail call i32 @llvm.ppc.mfmsr()
+  %conv = zext i32 %0 to i64
+  ret i64 %conv
+}

llvm/test/CodeGen/builtins-ppc-xlcompat-prefetch.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
+; RUN:   -mcpu=pwr8 < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu \
+; RUN:   -mcpu=pwr7 < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc-unknown-aix \
+; RUN:   -mcpu=pwr7 < %s | FileCheck %s --check-prefix=CHECK-AIX
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-aix \
+; RUN:   -mcpu=pwr7 < %s | FileCheck %s --check-prefix=CHECK-AIX64
+
+declare void @llvm.ppc.dcbtstt(i8*)
+declare void @llvm.ppc.dcbtt(i8*)
+
+@vpa = external local_unnamed_addr global i8*, align 8
+
+define dso_local void @test_dcbtstt() {
+; CHECK-LABEL: test_dcbtstt:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    addis 3, 2, .LC0@toc@ha
+; CHECK-NEXT:    ld 3, .LC0@toc@l(3)
+; CHECK-NEXT:    ld 3, 0(3)
+; CHECK-NEXT:    dcbtstt 0, 3
+; CHECK-NEXT:    blr
+;
+; CHECK-AIX-LABEL: test_dcbtstt:
+; CHECK-AIX:       # %bb.0: # %entry
+; CHECK-AIX-NEXT:    lwz 3, L..C0(2) # @vpa
+; CHECK-AIX-NEXT:    lwz 3, 0(3)
+; CHECK-AIX-NEXT:    dcbtstt 0, 3
+; CHECK-AIX-NEXT:    blr
+;
+; CHECK-AIX64-LABEL: test_dcbtstt:
+; CHECK-AIX64:       # %bb.0: # %entry
+; CHECK-AIX64-NEXT:    ld 3, L..C0(2) # @vpa
+; CHECK-AIX64-NEXT:    ld 3, 0(3)
+; CHECK-AIX64-NEXT:    dcbtstt 0, 3
+; CHECK-AIX64-NEXT:    blr
+entry:
+  %0 = load i8*, i8** @vpa, align 8
+  tail call void @llvm.ppc.dcbtstt(i8* %0)
+  ret void
+}
+
+
+define dso_local void @test_dcbtt() {
+; CHECK-LABEL: test_dcbtt:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    addis 3, 2, .LC0@toc@ha
+; CHECK-NEXT:    ld 3, .LC0@toc@l(3)
+; CHECK-NEXT:    ld 3, 0(3)
+; CHECK-NEXT:    dcbtt 0, 3
+; CHECK-NEXT:    blr
+;
+; CHECK-AIX-LABEL: test_dcbtt:
+; CHECK-AIX:       # %bb.0: # %entry
+; CHECK-AIX-NEXT:    lwz 3, L..C0(2) # @vpa
+; CHECK-AIX-NEXT:    lwz 3, 0(3)
+; CHECK-AIX-NEXT:    dcbtt 0, 3
+; CHECK-AIX-NEXT:    blr
+;
+; CHECK-AIX64-LABEL: test_dcbtt:
+; CHECK-AIX64:       # %bb.0: # %entry
+; CHECK-AIX64-NEXT:    ld 3, L..C0(2) # @vpa
+; CHECK-AIX64-NEXT:    ld 3, 0(3)
+; CHECK-AIX64-NEXT:    dcbtt 0, 3
+; CHECK-AIX64-NEXT:    blr
+entry:
+  %0 = load i8*, i8** @vpa, align 8
+  tail call void @llvm.ppc.dcbtt(i8* %0)
+  ret void
+}