[PowerPC] XCOFF exception section support on the direct assembler path and exception language and reason code lowering from trap intrinsics

[pscoro]

Summary

This feature implements support for making entries in the exception section on XCOFF on the direct assembly path using the ".except" pseudo-op. It also provides functionality to lower entries (comprised of language and reason codes) into the exception section through the use of annotation metadata attached to llvm.ppc.trap/trapd/tw/tdw intrinsics. Integrated assembler support will be provided in another review. https://reviews.llvm.org/D133030 needs to merge first for LIT tests

[shchenz]

Add more reviewers as we add a target hook TargetLowering::LowerTargetIntrinsic in the target independent part while building target intrinsic.

[RKSimon]

llvm/include/llvm/CodeGen/TargetLowering.h

4628

Please document the instruction and its arguments. Is Op necessary - should it be returned instead? Use SmallVectorImpl<SDValue>?

llvm/include/llvm/MC/MCStreamer.h

639

is it really necessary to add const to the integer args?

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

4868

Do you need Result? Should this be returning anything?

[pscoro]

Added integrated assembly support

[pscoro]

formatting

[pscoro]

small fix to lit test

[pscoro]

whitespace fix

[shchenz]

You should not add the integrated-as mode change to the existing non integrated-as change.
Better to post two patches for these two functionalities.

[RKSimon]

In D132146#3785383, @shchenz wrote:

You should not add the integrated-as mode change to the existing non integrated-as change.
Better to post two patches for these two functionalities.

+1 Please split these

[pscoro]

In D132146#3785383, @shchenz wrote:

You should not add the integrated-as mode change to the existing non integrated-as change.
Better to post two patches for these two functionalities.

In D132146#3785915, @RKSimon wrote:

In D132146#3785383, @shchenz wrote:

You should not add the integrated-as mode change to the existing non integrated-as change.
Better to post two patches for these two functionalities.

+1 Please split these

How would you recommend splitting these up? Beside the lit tests there is only one file (MCAsmStreamer) that has code specific to the direct assembly path, and only MCXCOFFStreamer and XCOFFObjectWriter are specific to the integrated-as path. The rest is shared code. My assumption was that if there is this much shared code it would be safer to make one review because the second review can't merge until this first one has and for a given feature, the full body of code that is needed for it is present in its review. That being said, do you prefer if I just move MCXCOFFStreamer, XCOFFObjectWriter and the object lit tests to a separate review?

[pscoro]

Split review

[pscoro]

Added temporary llvm_unreachable for integrated assembler path

[shchenz]

Thanks, looks almost good to me. Some minor comments.

llvm/include/llvm/MC/MCObjectWriter.h

108 ↗

(On Diff #461234)

This seems not belonging to this patch?

llvm/lib/MC/MCAsmStreamer.cpp

949

format

llvm/lib/MC/MCStreamer.cpp

1193

Please use clang-format to reformat all the changes. The format here seems not right.

1197

report_fatal_error seems like a better choice. llvm_unreachable may not stop the program in some configurations.

llvm/lib/MC/MCXCOFFStreamer.cpp

89

report_fatal_error

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-annotations-64bit.ll

11 ↗

(On Diff #461234)

This seems not right.

You already added RUN line for obj file type. You can just check: (can not redirect stderr to /dev/null)

: OBJ: LLVM ERROR: emitXCOFFExceptDirective not yet supported for integrated assembler path.

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-annotations.ll

11 ↗

(On Diff #461234)

Ditto. add a new runline for obj file type and check the error messages.

[pscoro]

fixes

[pscoro]

changes to LowerTargetIntrinsic

[shchenz]

llvm/lib/MC/MCStreamer.cpp

1199

nit: format

llvm/lib/MC/MCXCOFFStreamer.cpp

89

nit: a space after "integrated" and reformat please

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-annotations-64bit.ll

1 ↗

(On Diff #461978)

is the utils/update_llc_test_checks.py able to handle not --crash llc RUN line? I tested this, seems it is recognized as WARNING: Skipping non-llc RUN line: not --crash llc. If so, maybe we should delete this line as we are manually changing this file?

11 ↗

(On Diff #461978)

nit: space between integrated and assembler

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-annotations.ll

1 ↗

(On Diff #461978)

ditto.

2 ↗

(On Diff #461978)

Seems like we are testing 32 bit targets in this file? If so, we should use 32 bit triple, like powerpc-unknown-linux-gnu powerpc-unknown-aix

13 ↗

(On Diff #461978)

nit: space between integrated and assembler

[pscoro]

addressing nits

[pscoro]

llvm/lib/MC/MCXCOFFStreamer.cpp

89

ran clang-format after adding the space, no formatting was performed

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-annotations.ll

2 ↗

(On Diff #461978)

powerpc-unknown-aix already existed, I added powerpc-unknown-linux-gnu, no changes to assembly generated.

[shchenz]

llvm/lib/MC/MCXCOFFStreamer.cpp

89

Nit: Seems for a long string, the usual way is to put the space at the end of first line, instead of at the start of the second line. Need format here if the space is put at the end of integrated

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-annotations.ll

2 ↗

(On Diff #461978)

hmm, there are still some lines testing 64 bit targets, like the first/fourth/fifth RUN lines.

[pscoro]

changed lit test file names

[shchenz]

LGTM. Thanks for your patience.

[RKSimon]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

16247

This is always SDvalue()? If you're never going to create a node and just collect operands, I'd suggest you don't call this 'LowerTargetIntrinsic' and replace it with something more suitable (collectTargetIntrinsicOperands?)

[pscoro]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

16247

The idea was to keep the hook somewhat open ended. This particular intrinsic does not require anything besides operand collection but in general another target intrinsic may require additional custom lowering steps. Since its unavoidable that this hook is called for every target intrinsic, shouldn't the hook be general enough that additional intrinsic cases can be added to it later? Let me know what you think.

[RKSimon]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

16247

I agree, but its caller doesn't use the return value at all.

[pscoro]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

16247

Oh I see, the result gets overwritten anyway. In this case I suppose collectTargetIntrinsicOperands is the simplest solution. Will make the change shortly.

[pscoro]

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

16247

Also since the result is never used I'm assuming I should revert this function back to returning void?

[pscoro]

changed LowerTargetIntrinsic to CollectTargetIntrinsicOperands

[RKSimon]

Thanks, LGTM

[pscoro]

@shchenz I do not have commit privileges yet. If there are no further revision requests, could you commit this when you get the chance? Thanks

[shchenz]

@shchenz I do not have commit privileges yet. If there are no further revision requests, could you commit this when you get the chance? Thanks

Sure. Will commit this soon.

llvm/include/llvm/CodeGen/TargetLowering.h

virtual void LowerAsmOperandForConstraint(SDValue Op, std::string &Constraint,
                                             SelectionDAG &DAG) const;
 
   // Lower custom output constraints. If invalid, return SDValue().
   virtual SDValue LowerAsmOutputForConstraint(SDValue &Chain, SDValue &Flag,
                                               const SDLoc &DL,
                                               const AsmOperandInfo &OpInfo,
                                               SelectionDAG &DAG) const;
 
+  // Targets may override this function to collect operands from the CallInst
+  // and for example, lower them into the SelectionDAG operands. 
+  virtual void CollectTargetIntrinsicOperands(const CallInst &I,
+                                              SmallVectorImpl<SDValue> &Ops,
+                                              SelectionDAG &DAG) const;
+
   //===--------------------------------------------------------------------===//
   // Div utility functions
   //
 
   SDValue BuildSDIV(SDNode *N, SelectionDAG &DAG, bool IsAfterLegalization,
                     SmallVectorImpl<SDNode *> &Created) const;
   SDValue BuildUDIV(SDNode *N, SelectionDAG &DAG, bool IsAfterLegalization,
                     SmallVectorImpl<SDNode *> &Created) const;

llvm/include/llvm/MC/MCStreamer.h

public:
   /// undesirable name.
   ///
   /// \param Name - The name used internally in the assembly for references to
   /// the symbol.
   /// \param Rename - The value to which the Name parameter is
   /// changed at the end of assembly.
   virtual void emitXCOFFRenameDirective(const MCSymbol *Name, StringRef Rename);
 
+  /// Emit an XCOFF .except directive which adds information about
+  /// a trap instruction to the object file exception section
+  ///
+  /// \param Symbol - The function containing the trap.
+  /// \param Lang - The language code for the exception entry.
+  /// \param Reason - The reason code for the exception entry.
+  virtual void emitXCOFFExceptDirective(const MCSymbol *Symbol, MCSymbol *Trap,
+                                        unsigned Lang, unsigned Reason,
+                                        unsigned FunctionSize, bool hasDebug);
+
   /// Emit a XCOFF .ref directive which creates R_REF type entry in the
   /// relocation table for one or more symbols.
   ///
   /// \param Sym - The symbol on the .ref directive.
   virtual void emitXCOFFRefDirective(StringRef Sym);
 
   /// Emit an ELF .size directive.
   ///

llvm/include/llvm/MC/MCXCOFFStreamer.h

void emitXCOFFRefDirective(StringRef Name) override {
     report_fatal_error("emitXCOFFRefDirective is not implemented yet on object"
                        "generation path");
   }
   void emitXCOFFRenameDirective(const MCSymbol *Name,
                                 StringRef Rename) override {
     report_fatal_error("emitXCOFFRenameDirective is not implemented yet on "
                        "object generation path");
   }
+  void emitXCOFFExceptDirective(const MCSymbol *Symbol, MCSymbol *Trap,
+                                unsigned Lang, unsigned Reason,
+                                unsigned FunctionSize, bool hasDebug) override;
 };
 
 } // end namespace llvm
 
 #endif // LLVM_MC_MCXCOFFSTREAMER_H

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

void SelectionDAGBuilder::visitTargetIntrinsic(const CallInst &I,
   // Propagate fast-math-flags from IR to node(s).
   SDNodeFlags Flags;
   if (auto *FPMO = dyn_cast<FPMathOperator>(&I))
     Flags.copyFMF(*FPMO);
   SelectionDAG::FlagInserter FlagsInserter(DAG, Flags);
 
   // Create the node.
   SDValue Result;
+  // In some cases, custom collection of operands from CallInst I may be needed.
+  TLI.CollectTargetIntrinsicOperands(I, Ops, DAG);
   if (IsTgtIntrinsic) {
     // This is target intrinsic that touches memory
     Result =
         DAG.getMemIntrinsicNode(Info.opc, getCurSDLoc(), VTs, Ops, Info.memVT,
                                 MachinePointerInfo(Info.ptrVal, Info.offset),
                                 Info.align, Info.flags, Info.size,
                                 I.getAAMetadata());
   } else if (!HasChain) {

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

while (true) {
       }
       return;
     }
     break;
   }
   }
 }
 
+void TargetLowering::CollectTargetIntrinsicOperands(const CallInst &I,
+                                           SmallVectorImpl<SDValue> &Ops,
+                                           SelectionDAG &DAG) const {
+  return;
+}
+
 std::pair<unsigned, const TargetRegisterClass *>
 TargetLowering::getRegForInlineAsmConstraint(const TargetRegisterInfo *RI,
                                              StringRef Constraint,
                                              MVT VT) const {
   if (Constraint.empty() || Constraint[0] != '{')
     return std::make_pair(0u, static_cast<TargetRegisterClass *>(nullptr));
   assert(*(Constraint.end() - 1) == '}' && "Not a brace enclosed constraint?");
 

llvm/lib/MC/MCAsmStreamer.cpp

public:
   void emitXCOFFSymbolLinkageWithVisibility(MCSymbol *Symbol,
                                             MCSymbolAttr Linakge,
                                             MCSymbolAttr Visibility) override;
   void emitXCOFFRenameDirective(const MCSymbol *Name,
                                 StringRef Rename) override;
 
   void emitXCOFFRefDirective(StringRef Name) override;
 
+  void emitXCOFFExceptDirective(const MCSymbol *Symbol, MCSymbol *Trap,
+                                unsigned Lang, unsigned Reason,
+                                unsigned FunctionSize, bool hasDebug) override;
+
   void emitELFSize(MCSymbol *Symbol, const MCExpr *Value) override;
   void emitCommonSymbol(MCSymbol *Symbol, uint64_t Size,
                         unsigned ByteAlignment) override;
 
   /// Emit a local common (.lcomm) symbol.
   ///
   /// @param Symbol - The common symbol to emit.
   /// @param Size - The size of the common symbol.
void MCAsmStreamer::emitXCOFFRenameDirective(const MCSymbol *Name,
   EmitEOL();
 }
 
 void MCAsmStreamer::emitXCOFFRefDirective(StringRef Name) {
   OS << "\t.ref " << Name;
   EmitEOL();
 }
 
+void MCAsmStreamer::emitXCOFFExceptDirective(const MCSymbol *Symbol,
+                                             MCSymbol *Trap, unsigned Lang,
+                                             unsigned Reason,
+                                             unsigned FunctionSize,
+                                             bool hasDebug) {
+  OS << "\t.except\t";
+  Symbol->print(OS, MAI);
+  OS << ", " << Lang << ", " << Reason;
+  EmitEOL();
+}
+
 void MCAsmStreamer::emitELFSize(MCSymbol *Symbol, const MCExpr *Value) {
   assert(MAI->hasDotTypeDotSizeDirective());
   OS << "\t.size\t";
   Symbol->print(OS, MAI);
   OS << ", ";
   Value->print(OS, MAI);
   EmitEOL();
 }

llvm/lib/MC/MCStreamer.cpp

void MCStreamer::emitXCOFFRenameDirective(const MCSymbol *Name,
   llvm_unreachable("emitXCOFFRenameDirective is only supported on "
                    "XCOFF targets");
 }
 
 void MCStreamer::emitXCOFFRefDirective(StringRef Name) {
   llvm_unreachable("emitXCOFFRefDirective is only supported on XCOFF targets");
 }
 
+void MCStreamer::emitXCOFFExceptDirective(const MCSymbol *Symbol,
+                                          MCSymbol *Trap, unsigned Lang,
+                                          unsigned Reason,
+                                          unsigned FunctionSize,
+                                          bool hasDebug) {
+  report_fatal_error("emitXCOFFExceptDirective is only supported on "
+                     "XCOFF targets");
+}
+
 void MCStreamer::emitELFSize(MCSymbol *Symbol, const MCExpr *Value) {}
 void MCStreamer::emitELFSymverDirective(const MCSymbol *OriginalSym,
                                         StringRef Name, bool KeepOriginalSym) {}
 void MCStreamer::emitLocalCommonSymbol(MCSymbol *Symbol, uint64_t Size,
                                        unsigned ByteAlignment) {}
 void MCStreamer::emitTBSSSymbol(MCSection *Section, MCSymbol *Symbol,
                                 uint64_t Size, unsigned ByteAlignment) {}
 void MCStreamer::changeSection(MCSection *, const MCExpr *) {}

llvm/lib/MC/MCXCOFFStreamer.cpp

void MCXCOFFStreamer::emitXCOFFSymbolLinkageWithVisibility(
 
   // When the caller passes `MCSA_Invalid` for the visibility, do not emit one.
   if (Visibility == MCSA_Invalid)
     return;
 
   emitSymbolAttribute(Symbol, Visibility);
 }
 
+void MCXCOFFStreamer::emitXCOFFExceptDirective(const MCSymbol *Symbol,
+                                               MCSymbol *Trap, unsigned Lang,
+                                               unsigned Reason,
+                                               unsigned FunctionSize,
+                                               bool hasDebug) {
+  report_fatal_error(
+      "emitXCOFFExceptDirective not yet supported for integrated "
+      "assembler path.");
+}
+
 void MCXCOFFStreamer::emitCommonSymbol(MCSymbol *Symbol, uint64_t Size,
                                        unsigned ByteAlignment) {
   getAssembler().registerSymbol(*Symbol);
   Symbol->setExternal(cast<MCSymbolXCOFF>(Symbol)->getStorageClass() !=
                       XCOFF::C_HIDEXT);
   Symbol->setCommon(Size, ByteAlignment);
 
   // Default csect align is 4, but common symbols have explicit alignment values

llvm/lib/Target/PowerPC/PPCAsmPrinter.cpp

bool PPCAIXAsmPrinter::doInitialization(Module &M) {
 
   return Result;
 }
 
 void PPCAIXAsmPrinter::emitInstruction(const MachineInstr *MI) {
   switch (MI->getOpcode()) {
   default:
     break;
+  case PPC::TW:
+  case PPC::TWI:
+  case PPC::TD:
+  case PPC::TDI: {
+    if (MI->getNumOperands() < 5)
+      break; 
+    const MachineOperand &LangMO = MI->getOperand(3);
+    const MachineOperand &ReasonMO = MI->getOperand(4);
+    if (!LangMO.isImm() || !ReasonMO.isImm())
+      break;
+    MCSymbol *TempSym = OutContext.createNamedTempSymbol();
+    OutStreamer->emitLabel(TempSym);
+    OutStreamer->emitXCOFFExceptDirective(CurrentFnSym, TempSym,
+                 LangMO.getImm(), ReasonMO.getImm(),
+                 Subtarget->isPPC64() ? MI->getMF()->getInstructionCount() * 8 :
+                 MI->getMF()->getInstructionCount() * 4,
+     MMI->hasDebugInfo());
+    break;
+  }
   case PPC::GETtlsADDR64AIX:
   case PPC::GETtlsADDR32AIX: {
     // The reference to .__tls_get_addr is unknown to the assembler
     // so we need to emit an external symbol reference.
     MCSymbol *TlsGetAddr = createMCSymbolForTlsGetAddr(OutContext);
     ExtSymSDNodeSymbols.insert(TlsGetAddr);
     break;
   }

llvm/lib/Target/PowerPC/PPCISelDAGToDAG.cpp

case ISD::Constant:
     if (N->getValueType(0) == MVT::i64) {
       ReplaceNode(N, selectI64Imm(CurDAG, N));
       return;
     }
     break;
 
   case ISD::INTRINSIC_VOID: {
     auto IntrinsicID = N->getConstantOperandVal(1);
-    if (IntrinsicID == Intrinsic::ppc_tdw || IntrinsicID == Intrinsic::ppc_tw) {
-      unsigned Opcode = IntrinsicID == Intrinsic::ppc_tdw ? PPC::TDI : PPC::TWI;
+    if (IntrinsicID != Intrinsic::ppc_tdw && IntrinsicID != Intrinsic::ppc_tw &&
+        IntrinsicID != Intrinsic::ppc_trapd &&
+        IntrinsicID != Intrinsic::ppc_trap)
+        break;
+    unsigned Opcode = (IntrinsicID == Intrinsic::ppc_tdw ||
+                       IntrinsicID == Intrinsic::ppc_trapd)
+                          ? PPC::TDI
+                          : PPC::TWI;
+    SmallVector<SDValue, 4> OpsWithMD;
+    unsigned MDIndex;
+    if (IntrinsicID == Intrinsic::ppc_tdw ||
+        IntrinsicID == Intrinsic::ppc_tw) {
       SDValue Ops[] = {N->getOperand(4), N->getOperand(2), N->getOperand(3)};
       int16_t SImmOperand2;
       int16_t SImmOperand3;
       int16_t SImmOperand4;
       bool isOperand2IntS16Immediate =
           isIntS16Immediate(N->getOperand(2), SImmOperand2);
       bool isOperand3IntS16Immediate =
           isIntS16Immediate(N->getOperand(3), SImmOperand3);
if (IntrinsicID == Intrinsic::ppc_tdw ||
           TO = (TO & 0x1) ? TO + 1 : TO - 1;
         // We swap the fourth and fifth bit of TO if they are not same.
         if ((TO & 0x8) != ((TO & 0x10) >> 1))
           TO = (TO & 0x8) ? TO + 8 : TO - 8;
         Ops[0] = getI32Imm(TO, dl);
         Ops[1] = N->getOperand(3);
         Ops[2] = getI32Imm(int(SImmOperand2) & 0xFFFF, dl);
       }
-      CurDAG->SelectNodeTo(N, Opcode, MVT::Other, Ops);
-      return;
+      OpsWithMD = {Ops[0], Ops[1], Ops[2]};
+      MDIndex = 5;
+    } else {
+      OpsWithMD = {getI32Imm(24, dl), N->getOperand(2), getI32Imm(0, dl)};
+      MDIndex = 3;
     }
-    break;
+
+    if (N->getNumOperands() > MDIndex) {
+      SDValue MDV = N->getOperand(MDIndex);
+      const MDNode *MD = cast<MDNodeSDNode>(MDV)->getMD();
+      assert(MD->getNumOperands() != 0 && "Empty MDNode in operands!");
+      assert((isa<MDString>(MD->getOperand(0)) && cast<MDString>(
+           MD->getOperand(0))->getString().equals("ppc-trap-reason")) 
+           && "Unsupported annotation data type!");
+      for (unsigned i = 1; i < MD->getNumOperands(); i++) {
+        assert(isa<MDString>(MD->getOperand(i)) && 
+               "Invalid data type for annotation ppc-trap-reason!");
+        OpsWithMD.push_back(
+            getI32Imm(std::stoi(cast<MDString>(
+                      MD->getOperand(i))->getString().str()), dl));
+      }
+    }
+    OpsWithMD.push_back(N->getOperand(0)); // chain
+    CurDAG->SelectNodeTo(N, Opcode, MVT::Other, OpsWithMD);
+    return;
   }
 
   case ISD::INTRINSIC_WO_CHAIN: {
     // We emit the PPC::FSELS instruction here because of type conflicts with
     // the comparison operand. The FSELS instruction is defined to use an 8-byte
     // comparison like the FSELD version. The fsels intrinsic takes a 4-byte
     // value for the comparison. When selecting through a .td file, a type
     // error is raised. Must check this first so we never break on the

llvm/lib/Target/PowerPC/PPCISelLowering.h

getInlineAsmMemConstraint(StringRef ConstraintCode) const override {
         return InlineAsm::Constraint_Q;
       else if (ConstraintCode == "Z")
         return InlineAsm::Constraint_Z;
       else if (ConstraintCode == "Zy")
         return InlineAsm::Constraint_Zy;
       return TargetLowering::getInlineAsmMemConstraint(ConstraintCode);
     }
 
+    void CollectTargetIntrinsicOperands(const CallInst &I,
+                                 SmallVectorImpl<SDValue> &Ops,
+                                 SelectionDAG &DAG) const override;
+
     /// isLegalAddressingMode - Return true if the addressing mode represented
     /// by AM is legal for this target, for a load/store of the specified type.
     bool isLegalAddressingMode(const DataLayout &DL, const AddrMode &AM,
                                Type *Ty, unsigned AS,
                                Instruction *I = nullptr) const override;
 
     /// isLegalICmpImmediate - Return true if the specified immediate is legal
     /// icmp immediate, that is the target has icmp instructions which can

llvm/lib/Target/PowerPC/PPCISelLowering.cpp

if (Result.getNode()) {
     Ops.push_back(Result);
     return;
   }
 
   // Handle standard constraint letters.
   TargetLowering::LowerAsmOperandForConstraint(Op, Constraint, Ops, DAG);
 }
 
+void PPCTargetLowering::CollectTargetIntrinsicOperands(const CallInst &I,
+                                              SmallVectorImpl<SDValue> &Ops,
+                                              SelectionDAG &DAG) const {
+  if (I.getNumOperands() <= 1)
+    return;
+  if (!isa<ConstantSDNode>(Ops[1].getNode()))
+    return;
+  auto IntrinsicID = cast<ConstantSDNode>(Ops[1].getNode())->getZExtValue();
+  if (IntrinsicID != Intrinsic::ppc_tdw && IntrinsicID != Intrinsic::ppc_tw &&
+      IntrinsicID != Intrinsic::ppc_trapd && IntrinsicID != Intrinsic::ppc_trap)
+    return;
+
+  if (I.hasMetadata("annotation")) {
+    MDNode *MDN = I.getMetadata("annotation");
+    Ops.push_back(DAG.getMDNode(MDN));
+  }
+}
+
 // isLegalAddressingMode - Return true if the addressing mode represented
 // by AM is legal for this target, for a load/store of the specified type.
 bool PPCTargetLowering::isLegalAddressingMode(const DataLayout &DL,
                                               const AddrMode &AM, Type *Ty,
                                               unsigned AS,
                                               Instruction *I) const {
   // Vector type r+i form is supported since power9 as DQ form. We don't check
   // the offset matching DQ form requirement(off % 16 == 0), because on PowerPC,

llvm/lib/Target/PowerPC/PPCInstr64Bit.td

 // SLB Synchronize
 def SLBSYNC : XForm_0<31, 338, (outs), (ins), "slbsync", IIC_SprSLBSYNC, []>;
 
 } // IsISA3_0
 
 def : Pat<(int_ppc_stdcx ForceXForm:$dst, g8rc:$A),
           (STDCX g8rc:$A, ForceXForm:$dst)>;
 
-// trapd
-def : Pat<(int_ppc_trapd g8rc:$A),
-          (TDI 24, $A, 0)>;
 def : Pat<(i64 (int_ppc_mfspr timm:$SPR)),
           (MFSPR8 $SPR)>;
 def : Pat<(int_ppc_mtspr timm:$SPR, g8rc:$RT),
           (MTSPR8 $SPR, $RT)>;

llvm/lib/Target/PowerPC/PPCInstrInfo.td

 let mayStore = 1, mayLoad = 0, hasSideEffects = 0 in
 def STWAT : X_RD5_RS5_IM5<31, 710, (outs), (ins gprc:$rS, gprc:$rA, u5imm:$FC),
                           "stwat $rS, $rA, $FC", IIC_LdStStore>,
             Requires<[IsISA3_0]>;
 
 let isTerminator = 1, isBarrier = 1, hasCtrlDep = 1 in
 def TRAP  : XForm_24<31, 4, (outs), (ins), "trap", IIC_LdStLoad, [(trap)]>;
 
-def TWI : DForm_base<3, (outs), (ins u5imm:$to, gprc:$rA, s16imm:$imm),
+def TWI : DForm_base<3, (outs), (ins u5imm:$to, gprc:$rA, s16imm:$imm, variable_ops),
                      "twi $to, $rA, $imm", IIC_IntTrapW, []>;
-def TW : XForm_1<31, 4, (outs), (ins u5imm:$to, gprc:$rA, gprc:$rB),
+def TW : XForm_1<31, 4, (outs), (ins u5imm:$to, gprc:$rA, gprc:$rB, variable_ops),
                  "tw $to, $rA, $rB", IIC_IntTrapW, []>;
-def TDI : DForm_base<2, (outs), (ins u5imm:$to, g8rc:$rA, s16imm:$imm),
+def TDI : DForm_base<2, (outs), (ins u5imm:$to, g8rc:$rA, s16imm:$imm, variable_ops),
                      "tdi $to, $rA, $imm", IIC_IntTrapD, []>;
-def TD : XForm_1<31, 68, (outs), (ins u5imm:$to, g8rc:$rA, g8rc:$rB),
+def TD : XForm_1<31, 68, (outs), (ins u5imm:$to, g8rc:$rA, g8rc:$rB, variable_ops),
                  "td $to, $rA, $rB", IIC_IntTrapD, []>;
 
 def POPCNTB : XForm_11<31, 122, (outs gprc:$rA), (ins gprc:$rS),
                        "popcntb $rA, $rS", IIC_IntGeneral,
                        [(set i32:$rA, (int_ppc_popcntb i32:$rS))]>;
 
 //===----------------------------------------------------------------------===//
 // PPC32 Load Instructions.

 // swap the high word and low word.
 def : Pat<(i64 (bitreverse i64:$A)),
   (OR8 (RLDICR DWBytes7654.DWord, 32, 31), DWBytes3210.DWord)>;
 
 def : Pat<(int_ppc_stwcx ForceXForm:$dst, gprc:$A),
           (STWCX gprc:$A, ForceXForm:$dst)>;
 def : Pat<(int_ppc_stbcx ForceXForm:$dst, gprc:$A),
           (STBCX gprc:$A, ForceXForm:$dst)>;
-def : Pat<(int_ppc_trap gprc:$A),
-          (TWI 24, $A, 0)>;
 
 def : Pat<(int_ppc_fcfid f64:$A),
         (XSCVSXDDP $A)>;
 def : Pat<(int_ppc_fcfud f64:$A),
         (XSCVUXDDP $A)>;
 def : Pat<(int_ppc_fctid f64:$A),
         (FCTID $A)>;
 def : Pat<(int_ppc_fctidz f64:$A),

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-annotations-td.ll

+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
+; RUN:   --ppc-asm-full-reg-names -mcpu=pwr8 < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu \
+; RUN:   --ppc-asm-full-reg-names -mcpu=pwr7 < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-aix \
+; RUN:   --ppc-asm-full-reg-names -mcpu=pwr8 < %s | FileCheck %s -check-prefix=AIX
+; RUN: not --crash llc -verify-machineinstrs -mtriple=powerpc64-unknown-aix \
+; RUN:   --ppc-asm-full-reg-names -mcpu=pwr8 --filetype=obj -o /dev/null %s 2>&1 | FileCheck %s -check-prefix=OBJ
+
+; OBJ: LLVM ERROR: emitXCOFFExceptDirective not yet supported for integrated assembler path.
+
+!1 = !{!"ppc-trap-reason", !"1", !"2"}
+declare void @llvm.ppc.trapd(i64 %a)
+declare void @llvm.ppc.tdw(i64 %a, i64 %b, i32 immarg)
+
+define dso_local void @test__trapd_annotation(i64 %a) {
+; CHECK-LABEL: test__trapd_annotation:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdi 24, r3, 0
+; CHECK-NEXT:    blr
+;
+; AIX-LABEL: test__trapd_annotation:
+; AIX:       # %bb.0:
+; AIX-NEXT:  L..tmp0:
+; AIX-NEXT:    .except .test__trapd_annotation, 1, 2
+; AIX-NEXT:    tdi 24, r3, 0
+; AIX-NEXT:    blr
+  call void @llvm.ppc.trapd(i64 %a), !annotation !1
+  ret void
+}
+
+define dso_local void @test__tdw_annotation(i64 %a) {
+; CHECK-LABEL: test__tdw_annotation:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    tdi 0, r3, 4
+; CHECK-NEXT:    blr
+;
+; AIX-LABEL: test__tdw_annotation:
+; AIX:       # %bb.0:
+; AIX-NEXT:  L..tmp1:
+; AIX-NEXT:    .except .test__tdw_annotation, 1, 2
+; AIX-NEXT:    tdi 0, r3, 4
+; AIX-NEXT:    blr
+  call void @llvm.ppc.tdw(i64 4, i64 %a, i32 0), !annotation !1
+  ret void
+}

llvm/test/CodeGen/PowerPC/builtins-ppc-xlcompat-trap-annotations-tw.ll

+; RUN: llc -verify-machineinstrs -mtriple=powerpc64le-unknown-linux-gnu \
+; RUN:   --ppc-asm-full-reg-names -mcpu=pwr8 < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-linux-gnu \
+; RUN:   --ppc-asm-full-reg-names -mcpu=pwr7 < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc-unknown-linux-gnu \
+; RUN:   --ppc-asm-full-reg-names -mcpu=pwr7 < %s | FileCheck %s
+; RUN: llc -verify-machineinstrs -mtriple=powerpc-unknown-aix \
+; RUN:   --ppc-asm-full-reg-names -mcpu=pwr8 < %s | FileCheck %s -check-prefix=AIX
+; RUN: llc -verify-machineinstrs -mtriple=powerpc64-unknown-aix \
+; RUN:   --ppc-asm-full-reg-names -mcpu=pwr8 < %s | FileCheck %s -check-prefix=AIX
+; RUN: not --crash llc -verify-machineinstrs -mtriple=powerpc64-unknown-aix \
+; RUN:   --ppc-asm-full-reg-names -mcpu=pwr8 --filetype=obj -o /dev/null %s 2>&1 | FileCheck %s -check-prefix=OBJ
+
+; OBJ: LLVM ERROR: emitXCOFFExceptDirective not yet supported for integrated assembler path.
+
+!1 = !{!"ppc-trap-reason", !"1", !"2"}
+declare void @llvm.ppc.trap(i32 %a)
+declare void @llvm.ppc.tw(i32 %a, i32 %b, i32 immarg)
+define dso_local void @test__trap_annotation(i32 %a) {
+; CHECK-LABEL: test__trap_annotation:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twi 24, r3, 0
+; CHECK-NEXT:    blr
+;
+; AIX-LABEL: test__trap_annotation:
+; AIX:       # %bb.0:
+; AIX-NEXT:  L..tmp0:
+; AIX-NEXT:    .except .test__trap_annotation, 1, 2
+; AIX-NEXT:    twi 24, r3, 0
+; AIX-NEXT:    blr
+  call void @llvm.ppc.trap(i32 %a), !annotation !1
+  ret void
+}
+
+define dso_local void @test__tw_annotation(i32 %a) {
+; CHECK-LABEL: test__tw_annotation:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    twi 0, r3, 4
+; CHECK-NEXT:    blr
+;
+; AIX-LABEL: test__tw_annotation:
+; AIX:       # %bb.0:
+; AIX-NEXT:  L..tmp1:
+; AIX-NEXT:    .except .test__tw_annotation, 1, 2
+; AIX-NEXT:    twi 0, r3, 4
+; AIX-NEXT:    blr
+  call void @llvm.ppc.tw(i32 4, i32 %a, i32 0), !annotation !1
+  ret void
+}