[PowerPC] PPC backend optimization to lower int_ppc_tdw/int_ppc_tw intrinsics to TDI/TWI machine instructions

[NeHuang]

Summary

This patch adds the backend optimization to match XL behavior for the two builtins __tdw and __tw that when the second input argument is an immediate, emitting tdi/twi instructions instead of td/tw.

[nemanjai]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

5003

I think a couple of improvements can be made here:

1. Populate the Opcode and Ops array based on various conditions and add a single call to SelectNodeTo()
2. Handle the first input being constant (and then flip the condition)

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-64bit-only.ll

131

Can we add -ppc-asm-full-reg-names to the RUN lines so it is more clear which operand is a register and which is an immediate. This works on AIX now since https://reviews.llvm.org/D94282 landed.

[amyk]

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-64bit-only.ll

131

Maybe it would be good to pre-commit the change with -ppc-asm-full-reg-names added to the run lines so then this patch can only contain the pertinent td/tdi/tw/twi changes.

[NeHuang]

Addressed review comments from @nemanjai and @amyk

[amyk]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

5001

Might be a good idea to save N->getConstantOperandVal(1) since it is being accessed quite a few times here.

5003

I see we emit TDI/TWI in 2/3 cases, so I was wondering if it make sense pull out setting the opcode in the second and third case to have the default opcode be:

Opcode = N->getConstantOperandVal(1) == Intrinsic::ppc_tdw ? PPC::TDI
                                                           : PPC::TWI;

And then we just set the opcode to TD/TW in the first case?

5038

nit: Curly braces can be removed.

5042

nit: Curly braces can be removed.

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-64bit-only.ll

131

I meant, maybe it is a better idea to commit the test cases with -ppc-asm-full-reg-names first, so then this revision does not contain the additional updates of adding the registers in places that is not affected by your patch. However, perhaps if Nemanja thinks adding the option to this patch is OK, then that's fine with me, too.

[NeHuang]

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-64bit-only.ll

131

Good catch. Let me rebase this patch with ToT. The NFC patch was committed at 40cad47fd82ecaf253ba9b11fcd34f67dd557e9d.

[nemanjai]

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

5001

+1

5003

+1
Same thing with the Ops vector. Pre-populate it and then only change the operand that needs to be changed.

5012

Nit: complete sentences please. Here and in other comments.

Also, please add a comment stating that the imm + imm form will be optimized to either an unconditional trap or a nop in a later pass.

5036

This will be an uninitialized variable when compiled without asserts.

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-64bit-only.ll

3–7

This change should just be pre-committed

[NeHuang]

Addressed review comments from @amy

[NeHuang]

Address review comments from @nemanjai

[nemanjai]

LGTM other than a number of stylistic changes. Feel free to address those on the commit. You also might want to give @amyk a bit of time to ensure her comments were adequately addressed.

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

5017

Nit: no braces for a single statement.

5020

Nit: keep the else on the line immediately following the end of the if block so it is visually easy to match them up (the comments for else/else if go into the block). Also, rather than the structure:

if (something) {
} else {
  if (something else)
    ...
  else
    ...
}

Opt for a more flat structure of

if (something)
else if (something else)
else

5021

If you initialize Opcode this way at the declaration, you won't need this here and can flatten this as per my above comment.

5025

Nit: no braces with a single statement.

5037

s/fifthy/fifth

[amyk]

Aside from Nemanja's comments, this patch LGTM. Thanks for addressing the comments!

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

void PPCDAGToDAGISel::Select(SDNode *N) {
 
   case ISD::Constant:
     if (N->getValueType(0) == MVT::i64) {
       ReplaceNode(N, selectI64Imm(CurDAG, N));
       return;
     }
     break;
 
+  case ISD::INTRINSIC_VOID: {
+    auto IntrinsicID = N->getConstantOperandVal(1);
+    if (IntrinsicID == Intrinsic::ppc_tdw || IntrinsicID == Intrinsic::ppc_tw) {
+      unsigned Opcode = IntrinsicID == Intrinsic::ppc_tdw ? PPC::TDI : PPC::TWI;
+      SDValue Ops[] = {N->getOperand(4), N->getOperand(2), N->getOperand(3)};
+      int16_t SImmOperand2;
+      int16_t SImmOperand3;
+      int16_t SImmOperand4;
+      bool isOperand2IntS16Immediate =
+          isIntS16Immediate(N->getOperand(2), SImmOperand2);
+      bool isOperand3IntS16Immediate =
+          isIntS16Immediate(N->getOperand(3), SImmOperand3);
+      // We will emit PPC::TD or PPC::TW if the 2nd and 3rd operands are reg +
+      // reg or imm + imm. The imm + imm form will be optimized to either an
+      // unconditional trap or a nop in a later pass.
+      if (isOperand2IntS16Immediate == isOperand3IntS16Immediate)
+        Opcode = IntrinsicID == Intrinsic::ppc_tdw ? PPC::TD : PPC::TW;
+      else if (isOperand3IntS16Immediate)
+        // The 2nd and 3rd operands are reg + imm.
+        Ops[2] = getI32Imm(int(SImmOperand3) & 0xFFFF, dl);
+      else {
+        // The 2nd and 3rd operands are imm + reg.
+        bool isOperand4IntS16Immediate =
+            isIntS16Immediate(N->getOperand(4), SImmOperand4);
+        assert(isOperand4IntS16Immediate &&
+               "The 4th operand is not an Immediate");
+        // We need to flip the condition immediate TO.
+        int16_t TO = int(SImmOperand4) & 0x1F;
+        // We swap the first and second bit of TO if they are not same.
+        if ((TO & 0x1) != ((TO & 0x2) >> 1))
+          TO = (TO & 0x1) ? TO + 1 : TO - 1;
+        // We swap the fourth and fifth bit of TO if they are not same.
+        if ((TO & 0x8) != ((TO & 0x10) >> 1))
+          TO = (TO & 0x8) ? TO + 8 : TO - 8;
+        Ops[0] = getI32Imm(TO, dl);
+        Ops[1] = N->getOperand(3);
+        Ops[2] = getI32Imm(int(SImmOperand2) & 0xFFFF, dl);
+      }
+      CurDAG->SelectNodeTo(N, Opcode, MVT::Other, Ops);
+      return;
+    }
+    break;
+  }
+
   case ISD::INTRINSIC_WO_CHAIN: {
     // We emit the PPC::FSELS instruction here because of type conflicts with
     // the comparison operand. The FSELS instruction is defined to use an 8-byte
     // comparison like the FSELD version. The fsels intrinsic takes a 4-byte
     // value for the comparison. When selecting through a .td file, a type
     // error is raised. Must check this first so we never break on the
     // !Subtarget->isISA3_1() check.
     if (N->getConstantOperandVal(0) == Intrinsic::ppc_fsels) {

llvm/lib/Target/PowerPC/PPCInstr64Bit.td

def SLBIEG : XForm_26<31, 466, (outs), (ins gprc:$RS, gprc:$RB),
                       "slbieg $RS, $RB", IIC_SprSLBIEG, []>;
 // SLB Synchronize
 def SLBSYNC : XForm_0<31, 338, (outs), (ins), "slbsync", IIC_SprSLBSYNC, []>;
 
 } // IsISA3_0
 
 def : Pat<(int_ppc_stdcx ForceXForm:$dst, g8rc:$A),
           (STDCX g8rc:$A, ForceXForm:$dst)>;
-def : Pat<(int_ppc_tdw g8rc:$A, g8rc:$B, i32:$IMM),
-          (TD $IMM, $A, $B)>;
 
 // trapd
 def : Pat<(int_ppc_trapd g8rc:$A),
           (TDI 24, $A, 0)>;
 def : Pat<(i64 (int_ppc_mfspr timm:$SPR)),
           (MFSPR8 $SPR)>;
 def : Pat<(int_ppc_mtspr timm:$SPR, g8rc:$RT),
           (MTSPR8 $SPR, $RT)>;

llvm/lib/Target/PowerPC/PPCInstrInfo.td

 // swap the high word and low word.
 def : Pat<(i64 (bitreverse i64:$A)),
   (OR8 (RLDICR DWBytes7654.DWord, 32, 31), DWBytes3210.DWord)>;
 
 def : Pat<(int_ppc_stwcx ForceXForm:$dst, gprc:$A),
           (STWCX gprc:$A, ForceXForm:$dst)>;
 def : Pat<(int_ppc_stbcx ForceXForm:$dst, gprc:$A),
           (STBCX gprc:$A, ForceXForm:$dst)>;
-def : Pat<(int_ppc_tw gprc:$A, gprc:$B, i32:$IMM),
-          (TW $IMM, $A, $B)>;
 def : Pat<(int_ppc_trap gprc:$A),
           (TWI 24, $A, 0)>;
 
 def : Pat<(int_ppc_fcfid f64:$A),
         (XSCVSXDDP $A)>;
 def : Pat<(int_ppc_fcfud f64:$A),
         (XSCVUXDDP $A)>;
 def : Pat<(int_ppc_fctid f64:$A),

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-64bit-only.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
 ; RUN:   --ppc-asm-full-reg-names -mcpu=pwr8 < %s | FileCheck %s
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu \
 ; RUN:   --ppc-asm-full-reg-names -mcpu=pwr7 < %s | FileCheck %s
 ; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-aix \
 ; RUN:   --ppc-asm-full-reg-names -mcpu=pwr8 < %s | FileCheck %s
 
 ; tdw
 declare void @llvm.ppc.tdw(i64 %a, i64 %b, i32 immarg)
 define dso_local void @test__tdwlgt(i64 %a, i64 %b) {
 ; CHECK-LABEL: test__tdwlgt:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    tdlgt r3, r4
 ; CHECK-NEXT:    blr

 ; CHECK-LABEL: test__tdw_no_match:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    td 13, r3, r4
 ; CHECK-NEXT:    blr
   call void @llvm.ppc.tdw(i64 %a, i64 %b, i32 13)
   ret void
 }
 
+; tdw -> tdi
+define dso_local void @test__tdi_reg_imm_boundary(i64 %a) {
+; CHECK-LABEL: test__tdi_reg_imm_boundary:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdi 3, r3, 32767
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 %a, i64 32767, i32 3)
+  ret void
+}
+
+define dso_local void @test__tdi_imm_reg_boundary(i64 %a) {
+; CHECK-LABEL: test__tdi_imm_reg_boundary:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdi 3, r3, 32767
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 32767, i64 %a, i32 3)
+  ret void
+}
+
+define dso_local void @test__tdi_reg_imm_boundary1(i64 %a) {
+; CHECK-LABEL: test__tdi_reg_imm_boundary1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdi 3, r3, -32768
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 %a, i64 -32768, i32 3)
+  ret void
+}
+
+define dso_local void @test__tdi_imm_reg_boundary1(i64 %a) {
+; CHECK-LABEL: test__tdi_imm_reg_boundary1:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdi 3, r3, -32768
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 -32768, i64 %a, i32 3)
+  ret void
+}
+
+define dso_local void @test__td_reg_imm_boundary2(i64 %a) {
+; CHECK-LABEL: test__td_reg_imm_boundary2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    li r4, 0
+; CHECK-NEXT:    ori r4, r4, 32768
+; CHECK-NEXT:    td 3, r4, r3
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 32768, i64 %a, i32 3)
+  ret void
+}
+
+define dso_local void @test__td_imm_reg_boundary2(i64 %a) {
+; CHECK-LABEL: test__td_imm_reg_boundary2:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    li r4, 0
+; CHECK-NEXT:    ori r4, r4, 32768
+; CHECK-NEXT:    td 3, r3, r4
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 %a, i64 32768, i32 3)
+  ret void
+}
+
+define dso_local void @test__td_reg_imm_boundary3(i64 %a) {
+; CHECK-LABEL: test__td_reg_imm_boundary3:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    lis r4, -1
+; CHECK-NEXT:    ori r4, r4, 32767
+; CHECK-NEXT:    td 3, r3, r4
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 %a, i64 -32769, i32 3)
+  ret void
+}
+
+define dso_local void @test__td_imm_reg_boundary3(i64 %a) {
+; CHECK-LABEL: test__td_imm_reg_boundary3:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    lis r4, -1
+; CHECK-NEXT:    ori r4, r4, 32767
+; CHECK-NEXT:    td 3, r3, r4
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 %a, i64 -32769, i32 3)
+  ret void
+}
+
+define dso_local void @test__tdlgti_reg_imm(i64 %a) {
+; CHECK-LABEL: test__tdlgti_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdlgti r3, 0
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 %a, i64 0, i32 1)
+  ret void
+}
+
+define dso_local void @test__tdllti_imm_reg(i64 %a) {
+; CHECK-LABEL: test__tdllti_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdllti r3, 0
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 0, i64 %a, i32 1)
+  ret void
+}
+
+define dso_local void @test__tdllti_reg_imm(i64 %a) {
+; CHECK-LABEL: test__tdllti_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdllti r3, 1
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 %a, i64 1, i32 2)
+  ret void
+}
+
+define dso_local void @test__tdlgti_imm_reg(i64 %a) {
+; CHECK-LABEL: test__tdlgti_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdlgti r3, 1
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 1, i64 %a, i32 2)
+  ret void
+}
+
+define dso_local void @test__tdeqi_reg_imm(i64 %a) {
+; CHECK-LABEL: test__tdeqi_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdeqi r3, 2
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 %a, i64 2, i32 4)
+  ret void
+}
+
+define dso_local void @test__tdeqi_imm_reg(i64 %a) {
+; CHECK-LABEL: test__tdeqi_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdeqi r3, 2
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 2, i64 %a, i32 4)
+  ret void
+}
+
+define dso_local void @test__tdgti_reg_imm(i64 %a) {
+; CHECK-LABEL: test__tdgti_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdgti r3, 16
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 %a, i64 16, i32 8)
+  ret void
+}
+
+define dso_local void @test__tdlti_imm_reg(i64 %a) {
+; CHECK-LABEL: test__tdlti_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdlti r3, 16
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 16, i64 %a, i32 8)
+  ret void
+}
+
+define dso_local void @test__tdlti_reg_imm(i64 %a) {
+; CHECK-LABEL: test__tdlti_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdlti r3, 64
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 %a, i64 64, i32 16)
+  ret void
+}
+
+define dso_local void @test__tdgti_imm_reg(i64 %a) {
+; CHECK-LABEL: test__tdgti_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdgti r3, 64
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 64, i64 %a, i32 16)
+  ret void
+}
+
+define dso_local void @test__tdnei_reg_imm(i64 %a) {
+; CHECK-LABEL: test__tdnei_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdnei r3, 256
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 %a, i64 256, i32 24)
+  ret void
+}
+
+define dso_local void @test__tdnei_imm_reg(i64 %a) {
+; CHECK-LABEL: test__tdnei_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdnei r3, 256
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 256, i64 %a, i32 24)
+  ret void
+}
+
+define dso_local void @test__tdui_reg_imm(i64 %a) {
+; CHECK-LABEL: test__tdui_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdui r3, 512
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 %a, i64 512, i32 31)
+  ret void
+}
+
+define dso_local void @test__tdui_imm_reg(i64 %a) {
+; CHECK-LABEL: test__tdui_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdui r3, 512
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 512, i64 %a, i32 31)
+  ret void
+}
+
 ; trapd
 declare void @llvm.ppc.trapd(i64 %a)
 define dso_local void @test__trapd(i64 %a) {
 ; CHECK-LABEL: test__trapd:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    tdnei r3, 0
 ; CHECK-NEXT:    blr
   call void @llvm.ppc.trapd(i64 %a)
   ret void
 }

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap.ll

 ; CHECK-LABEL: test__tw_no_match:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    tw 13, r3, r4
 ; CHECK-NEXT:    blr
   call void @llvm.ppc.tw(i32 %a, i32 %b, i32 13)
   ret void
 }
 
+; tw -> twi
+define dso_local void @test__twi_boundary_reg_imm(i32 %a) {
+; CHECK-LABEL: test__twi_boundary_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twi 3, r3, 32767
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 %a, i32 32767, i32 3)
+  ret void
+}
+
+define dso_local void @test__twi_boundary_imm_reg(i32 %a) {
+; CHECK-LABEL: test__twi_boundary_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twi 3, r3, 32767
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 32767, i32 %a, i32 3)
+  ret void
+}
+
+define dso_local void @test__twi_boundary1_reg_imm(i32 %a) {
+; CHECK-LABEL: test__twi_boundary1_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twi 3, r3, -32768
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 %a, i32 -32768, i32 3)
+  ret void
+}
+
+define dso_local void @test__twi_boundary1_imm_reg(i32 %a) {
+; CHECK-LABEL: test__twi_boundary1_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twi 3, r3, -32768
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 -32768, i32 %a, i32 3)
+  ret void
+}
+
+define dso_local void @test__tw_boundary2_reg_imm(i32 %a) {
+; CHECK-LABEL: test__tw_boundary2_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    lis r4, 0
+; CHECK-NEXT:    ori r4, r4, 32768
+; CHECK-NEXT:    tw 3, r3, r4
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 %a, i32 32768, i32 3)
+  ret void
+}
+
+define dso_local void @test__tw_boundary2_imm_reg(i32 %a) {
+; CHECK-LABEL: test__tw_boundary2_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    lis r4, 0
+; CHECK-NEXT:    ori r4, r4, 32768
+; CHECK-NEXT:    tw 3, r4, r3
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 32768, i32 %a, i32 3)
+  ret void
+}
+
+define dso_local void @test__tw_boundary3_reg_imm(i32 %a) {
+; CHECK-LABEL: test__tw_boundary3_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    lis r4, -1
+; CHECK-NEXT:    ori r4, r4, 32767
+; CHECK-NEXT:    tw 3, r3, r4
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 %a, i32 -32769, i32 3)
+  ret void
+}
+
+define dso_local void @test__tw_boundary3_imm_reg(i32 %a) {
+; CHECK-LABEL: test__tw_boundary3_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    lis r4, -1
+; CHECK-NEXT:    ori r4, r4, 32767
+; CHECK-NEXT:    tw 3, r4, r3
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 -32769, i32 %a, i32 3)
+  ret void
+}
+
+define dso_local void @test__twlgti_reg_imm(i32 %a) {
+; CHECK-LABEL: test__twlgti_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twlgti r3, 0
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 %a, i32 0, i32 1)
+  ret void
+}
+
+define dso_local void @test__twllti_imm_reg(i32 %a) {
+; CHECK-LABEL: test__twllti_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twllti r3, 0
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 0, i32 %a, i32 1)
+  ret void
+}
+
+define dso_local void @test__twllti_reg_imm(i32 %a) {
+; CHECK-LABEL: test__twllti_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twllti r3, 1
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 %a, i32 1, i32 2)
+  ret void
+}
+
+define dso_local void @test__twlgti_imm_reg(i32 %a) {
+; CHECK-LABEL: test__twlgti_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twlgti r3, 1
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 1, i32 %a, i32 2)
+  ret void
+}
+
+define dso_local void @test__tweqi_reg_imm(i32 %a) {
+; CHECK-LABEL: test__tweqi_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tweqi r3, 2
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 %a, i32 2, i32 4)
+  ret void
+}
+
+define dso_local void @test__tweqi_imm_reg(i32 %a) {
+; CHECK-LABEL: test__tweqi_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tweqi r3, 2
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 2, i32 %a, i32 4)
+  ret void
+}
+
+define dso_local void @test__twgti_reg_imm(i32 %a) {
+; CHECK-LABEL: test__twgti_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twgti r3, 16
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 %a, i32 16, i32 8)
+  ret void
+}
+
+define dso_local void @test__twlti_imm_reg(i32 %a) {
+; CHECK-LABEL: test__twlti_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twlti r3, 16
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 16, i32 %a, i32 8)
+  ret void
+}
+
+define dso_local void @test__twlti_reg_imm(i32 %a) {
+; CHECK-LABEL: test__twlti_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twlti r3, 64
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 %a, i32 64, i32 16)
+  ret void
+}
+
+define dso_local void @test__twgti_imm_reg(i32 %a) {
+; CHECK-LABEL: test__twgti_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twgti r3, 64
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 64, i32 %a, i32 16)
+  ret void
+}
+
+define dso_local void @test__twnei_reg_imm(i32 %a) {
+; CHECK-LABEL: test__twnei_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twnei r3, 256
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 %a, i32 256, i32 24)
+  ret void
+}
+
+define dso_local void @test__twnei_imm_reg(i32 %a) {
+; CHECK-LABEL: test__twnei_imm_reg:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twnei r3, 256
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 256, i32 %a, i32 24)
+  ret void
+}
+
+define dso_local void @test__twui_reg_imm(i32 %a) {
+; CHECK-LABEL: test__twui_reg_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twui r3, 512
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 %a, i32 512, i32 31)
+  ret void
+}
+
+define dso_local void @test__twui_imm_imm(i32 %a) {
+; CHECK-LABEL: test__twui_imm_imm:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twui r3, 512
+; CHECK-NEXT:    blr
+  call void @llvm.ppc.tw(i32 512, i32 %a, i32 31)
+  ret void
+}
+
 ; trap
 declare void @llvm.ppc.trap(i32 %a)
 define dso_local void @test__trap(i32 %a) {
 ; CHECK-LABEL: test__trap:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    twnei r3, 0
 ; CHECK-NEXT:    blr
   call void @llvm.ppc.trap(i32 %a)
   ret void
 }