Function-relative jump-table encoding

[joerg]

Summary

This patch is a work-in-progress base for solving D23250 and the jump-table handling on PPC64. It contains two parts:

1. A FUNCBASE ISD node that the target should turn into an appropriate address expression. The x86 implementation is provided.
2. A set of three different jump-table encodings, using different sizes for the delta. Since the value range is limited by the function size, the 32bit encoding works well for large code models as long as functions are still smaller than 4GB. A new peep hole pass could also choose the correct encoding on a per-function base.

I'd like some comments on the general direction of the patch. If someone with a bit familiarity of PPC wants to provide the ISD hook, it would be appreciated. I have no idea how to implement the peep hole pass right now. For 32bit x86, the PIC version results in a slightly more complex instruction, i.e. before it was:

movl    .LJTI0_0@GOTOFF(%eax,%ecx,4), %edx
addl    %eax, %edx

Afterward it is:

movl    .LJTI0_0@GOTOFF(%eax,%ecx,4), %edx
leal    .L0$fb(%edx,%eax), %edx

On 64bit x86, it adds a full instruction because the rip-relative addressing can't fold the offset addition into a single lea. The difference between the different encoding versions would be movl/movslq vs movsbl/movsbq vs movswl/movswq.

[joerg]

I think I can implement the selection of the jump table encoding as fixup pass on x86. The biggest concern here is that at least for x86_64, the funcbase-difference jump table encodings require one more register. For 32bit mode, that's not a concern as the address computation can be folded with lea into a single operation. So being able to use 16bit encodings for the jump table would imply an additional scratch register to be used, even if the function turns out to be too large. I'm not sure how much that really matters though.

include/llvm/CodeGen/ISDOpcodes.h

enum NodeType {
     BasicBlock, VALUETYPE, CONDCODE, Register, RegisterMask,
     Constant, ConstantFP,
     GlobalAddress, GlobalTLSAddress, FrameIndex,
     JumpTable, ConstantPool, ExternalSymbol, BlockAddress,
 
     /// The address of the GOT
     GLOBAL_OFFSET_TABLE,
 
+    /// The address of the entry point of the function.
+    FUNCBASE,
+
     /// FRAMEADDR, RETURNADDR - These nodes represent llvm.frameaddress and
     /// llvm.returnaddress on the DAG.  These nodes take one operand, the index
     /// of the frame or return address to return.  An index of zero corresponds
     /// to the current function's frame or return address, an index of one to
     /// the parent's frame or return address, and so on.
     FRAMEADDR, RETURNADDR, ADDROFRETURNADDR,
 
     /// LOCAL_RECOVER - Represents the llvm.localrecover intrinsic.

include/llvm/CodeGen/MIRYamlMapping.h

static void enumeration(yaml::IO &IO,
     IO.enumCase(EntryKind, "block-address",
                 MachineJumpTableInfo::EK_BlockAddress);
     IO.enumCase(EntryKind, "gp-rel64-block-address",
                 MachineJumpTableInfo::EK_GPRel64BlockAddress);
     IO.enumCase(EntryKind, "gp-rel32-block-address",
                 MachineJumpTableInfo::EK_GPRel32BlockAddress);
     IO.enumCase(EntryKind, "label-difference32",
                 MachineJumpTableInfo::EK_LabelDifference32);
+    IO.enumCase(EntryKind, "funcbase-difference32",
+                MachineJumpTableInfo::EK_FuncBaseDifference32);
     IO.enumCase(EntryKind, "inline", MachineJumpTableInfo::EK_Inline);
     IO.enumCase(EntryKind, "custom32", MachineJumpTableInfo::EK_Custom32);
   }
 };
 
 } // end namespace yaml
 } // end namespace llvm
 

include/llvm/CodeGen/MachineFunction.h

public:
   /// If isLinkerPrivate is specified, an 'l' label is returned, otherwise a
   /// normal 'L' label is returned.
   MCSymbol *getJTISymbol(unsigned JTI, MCContext &Ctx,
                          bool isLinkerPrivate = false) const;
 
   /// getPICBaseSymbol - Return a function-local symbol to represent the PIC
   /// base.
   MCSymbol *getPICBaseSymbol() const;
+
+  /// getFuncBaseSymbol - Return a function-local symbol to represent the entry
+  /// point.
+  MCSymbol *getFuncBaseSymbol() const;
 };
 
 //===--------------------------------------------------------------------===//
 // GraphTraits specializations for function basic block graphs (CFGs)
 //===--------------------------------------------------------------------===//
 
 // Provide specializations of GraphTraits to be able to treat a
 // machine function as a graph of machine basic blocks... these are

include/llvm/CodeGen/MachineJumpTableInfo.h

enum JTEntryKind {
     /// the address of the jump table.  This is used for PIC jump tables where
     /// gprel32 is not supported.  e.g.:
     ///      .word LBB123 - LJTI1_2
     /// If the .set directive is supported, this is emitted as:
     ///      .set L4_5_set_123, LBB123 - LJTI1_2
     ///      .word L4_5_set_123
     EK_LabelDifference32,
 
+    /// EK_FuncBaseDifference32 - Each entry is the address of the block minus
+    /// the start of the function.  This is used for position-independent code
+    /// and in medium and large code models.
+    EK_FuncBaseDifference32,
+    EK_FuncBaseDifference16,
+    EK_FuncBaseDifference8,
+
     /// EK_Inline - Jump table entries are emitted inline at their point of
     /// use. It is the responsibility of the target to emit the entries.
     EK_Inline,
 
     /// EK_Custom32 - Each entry is a 32-bit value that is custom lowered by the
     /// TargetLowering::LowerCustomJumpTableEntry hook.
     EK_Custom32
   };

include/llvm/CodeGen/SelectionDAG.h

SDValue getUNDEF(EVT VT) {
     return getNode(ISD::UNDEF, SDLoc(), VT);
   }
 
   /// Return a GLOBAL_OFFSET_TABLE node. This does not have a useful SDLoc.
   SDValue getGLOBAL_OFFSET_TABLE(EVT VT) {
     return getNode(ISD::GLOBAL_OFFSET_TABLE, SDLoc(), VT);
   }
 
+  /// Return a FUNCBASE node. This does not have a useful SDLoc.
+  SDValue getFUNCBASE(EVT VT) {
+    return getNode(ISD::FUNCBASE, SDLoc(), VT);
+  }
+
   /// Gets or creates the specified node.
   ///
   SDValue getNode(unsigned Opcode, const SDLoc &DL, EVT VT,
                   ArrayRef<SDUse> Ops);
   SDValue getNode(unsigned Opcode, const SDLoc &DL, EVT VT,
                   ArrayRef<SDValue> Ops, const SDNodeFlags *Flags = nullptr);
   SDValue getNode(unsigned Opcode, const SDLoc &DL, ArrayRef<EVT> ResultTys,
                   ArrayRef<SDValue> Ops);

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

void AsmPrinter::EmitFunctionHeader() {
   // Emit the prefix data.
   if (F->hasPrefixData())
     EmitGlobalConstant(F->getParent()->getDataLayout(), F->getPrefixData());
 
   // Emit the CurrentFnSym.  This is a virtual function to allow targets to
   // do their wild and crazy things as required.
   EmitFunctionEntryLabel();
 
+  if (const MachineJumpTableInfo *MJTI = MF->getJumpTableInfo()) {
+    switch (MJTI->getEntryKind()) {
+    default:
+      break;
+    case MachineJumpTableInfo::EK_FuncBaseDifference32:
+    case MachineJumpTableInfo::EK_FuncBaseDifference16:
+    case MachineJumpTableInfo::EK_FuncBaseDifference8:
+      OutStreamer->EmitLabel(MF->getFuncBaseSymbol());
+    }
+  }
+
   // If the function had address-taken blocks that got deleted, then we have
   // references to the dangling symbols.  Emit them at the start of the function
   // so that we don't get references to undefined symbols.
   std::vector<MCSymbol*> DeadBlockSyms;
   MMI->takeDeletedSymbolsForFunction(F, DeadBlockSyms);
   for (unsigned i = 0, e = DeadBlockSyms.size(); i != e; ++i) {
     OutStreamer->AddComment("Address taken block that was later removed");
     OutStreamer->EmitLabel(DeadBlockSyms[i]);
void AsmPrinter::EmitJumpTableInfo() {
   const std::vector<MachineJumpTableEntry> &JT = MJTI->getJumpTables();
   if (JT.empty()) return;
 
   // Pick the directive to use to print the jump table entries, and switch to
   // the appropriate section.
   const Function *F = MF->getFunction();
   const TargetLoweringObjectFile &TLOF = getObjFileLowering();
   bool JTInDiffSection = !TLOF.shouldPutJumpTableInFunctionSection(
-      MJTI->getEntryKind() == MachineJumpTableInfo::EK_LabelDifference32,
+      MJTI->getEntryKind() == MachineJumpTableInfo::EK_LabelDifference32 ||
+      MJTI->getEntryKind() == MachineJumpTableInfo::EK_FuncBaseDifference32 ||
+      MJTI->getEntryKind() == MachineJumpTableInfo::EK_FuncBaseDifference16 ||
+      MJTI->getEntryKind() == MachineJumpTableInfo::EK_FuncBaseDifference8,
       *F);
   if (JTInDiffSection) {
     // Drop it in the readonly section.
     MCSection *ReadOnlySection = TLOF.getSectionForJumpTable(*F, TM);
     OutStreamer->SwitchSection(ReadOnlySection);
   }
 
   EmitAlignment(Log2_32(MJTI->getEntryAlignment(DL)));
if (MAI->doesSetDirectiveSuppressReloc()) {
       break;
     }
     Value = MCSymbolRefExpr::create(MBB->getSymbol(), OutContext);
     const TargetLowering *TLI = MF->getSubtarget().getTargetLowering();
     const MCExpr *Base = TLI->getPICJumpTableRelocBaseExpr(MF, UID, OutContext);
     Value = MCBinaryExpr::createSub(Value, Base, OutContext);
     break;
   }
+
+  case MachineJumpTableInfo::EK_FuncBaseDifference32:
+  case MachineJumpTableInfo::EK_FuncBaseDifference16:
+  case MachineJumpTableInfo::EK_FuncBaseDifference8: {
+    // Each entry is the address of the block minus the address of
+    // the entry point of the function.
+    Value = MCSymbolRefExpr::create(MBB->getSymbol(), OutContext);
+    MCSymbol *FB = MF->getFuncBaseSymbol();
+
+    const MCExpr *Base = MCSymbolRefExpr::create(FB, OutContext);
+    Value = MCBinaryExpr::createSub(Value, Base, OutContext);
+    break;
+  }
   }
 
   assert(Value && "Unknown entry kind!");
 
   unsigned EntrySize = MJTI->getEntrySize(getDataLayout());
   OutStreamer->EmitValue(Value, EntrySize);
 }
 

lib/CodeGen/MachineFunction.cpp

 
 /// Return a function-local symbol to represent the PIC base.
 MCSymbol *MachineFunction::getPICBaseSymbol() const {
   const DataLayout &DL = getDataLayout();
   return Ctx.getOrCreateSymbol(Twine(DL.getPrivateGlobalPrefix()) +
                                Twine(getFunctionNumber()) + "$pb");
 }
 
+/// Return a function-local symbol to represent the entry point.
+MCSymbol *MachineFunction::getFuncBaseSymbol() const {
+  const DataLayout &DL = getDataLayout();
+  return Ctx.getOrCreateSymbol(Twine(DL.getPrivateGlobalPrefix()) +
+                               Twine(getFunctionNumber()) + "$fb");
+}
+
 //===----------------------------------------------------------------------===//
 //  MachineFrameInfo implementation
 //===----------------------------------------------------------------------===//
 
 /// Make sure the function is at least Align bytes aligned.
 void MachineFrameInfo::ensureMaxAlignment(unsigned Align) {
   if (!StackRealignable)
     assert(Align <= StackAlignment &&

 #endif
 
 //===----------------------------------------------------------------------===//
 //  MachineJumpTableInfo implementation
 //===----------------------------------------------------------------------===//
 
 /// Return the size of each entry in the jump table.
 unsigned MachineJumpTableInfo::getEntrySize(const DataLayout &TD) const {
-  // The size of a jump table entry is 4 bytes unless the entry is just the
-  // address of a block, in which case it is the pointer size.
   switch (getEntryKind()) {
   case MachineJumpTableInfo::EK_BlockAddress:
     return TD.getPointerSize();
   case MachineJumpTableInfo::EK_GPRel64BlockAddress:
     return 8;
   case MachineJumpTableInfo::EK_GPRel32BlockAddress:
   case MachineJumpTableInfo::EK_LabelDifference32:
+  case MachineJumpTableInfo::EK_FuncBaseDifference32:
   case MachineJumpTableInfo::EK_Custom32:
     return 4;
+  case MachineJumpTableInfo::EK_FuncBaseDifference16:
+    return 2;
+  case MachineJumpTableInfo::EK_FuncBaseDifference8:
+    return 1;
   case MachineJumpTableInfo::EK_Inline:
     return 0;
   }
   llvm_unreachable("Unknown jump table encoding!");
 }
 
 /// Return the alignment of each entry in the jump table.
 unsigned MachineJumpTableInfo::getEntryAlignment(const DataLayout &TD) const {
   // The alignment of a jump table entry is the alignment of int32 unless the
   // entry is just the address of a block, in which case it is the pointer
   // alignment.
   switch (getEntryKind()) {
   case MachineJumpTableInfo::EK_BlockAddress:
     return TD.getPointerABIAlignment();
   case MachineJumpTableInfo::EK_GPRel64BlockAddress:
     return TD.getABIIntegerTypeAlignment(64);
   case MachineJumpTableInfo::EK_GPRel32BlockAddress:
   case MachineJumpTableInfo::EK_LabelDifference32:
+  case MachineJumpTableInfo::EK_FuncBaseDifference32:
   case MachineJumpTableInfo::EK_Custom32:
     return TD.getABIIntegerTypeAlignment(32);
+  case MachineJumpTableInfo::EK_FuncBaseDifference16:
+    return TD.getABIIntegerTypeAlignment(16);
+  case MachineJumpTableInfo::EK_FuncBaseDifference8:
+    return TD.getABIIntegerTypeAlignment(8);
   case MachineJumpTableInfo::EK_Inline:
     return 1;
   }
   llvm_unreachable("Unknown jump table encoding!");
 }
 
 /// Create a new jump table entry in the jump table info.
 unsigned MachineJumpTableInfo::createJumpTableIndex(

lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

     if (Action == TargetLowering::Legal)
       Action = TargetLowering::Expand;
     break;
   case ISD::INIT_TRAMPOLINE:
   case ISD::ADJUST_TRAMPOLINE:
   case ISD::FRAMEADDR:
   case ISD::RETURNADDR:
   case ISD::ADDROFRETURNADDR:
+  case ISD::FUNCBASE:
     // These operations lie about being legal: when they claim to be legal,
     // they should actually be custom-lowered.
     Action = TLI.getOperationAction(Node->getOpcode(), Node->getValueType(0));
     if (Action == TargetLowering::Legal)
       Action = TargetLowering::Custom;
     break;
   case ISD::READCYCLECOUNTER:
     // READCYCLECOUNTER returns an i64, even if type legalization might have

     SDValue Addr = DAG.getNode(ISD::ADD, dl, Index.getValueType(),
                                Index, Table);
 
     EVT MemVT = EVT::getIntegerVT(*DAG.getContext(), EntrySize * 8);
     SDValue LD = DAG.getExtLoad(
         ISD::SEXTLOAD, dl, PTy, Chain, Addr,
         MachinePointerInfo::getJumpTable(DAG.getMachineFunction()), MemVT);
     Addr = LD;
-    if (TM.isPositionIndependent()) {
+    if (TM.isPositionIndependent() ||
+        TLI.getJumpTableEncoding() == MachineJumpTableInfo::EK_FuncBaseDifference32 ||
+        TLI.getJumpTableEncoding() == MachineJumpTableInfo::EK_FuncBaseDifference16 ||
+        TLI.getJumpTableEncoding() == MachineJumpTableInfo::EK_FuncBaseDifference8) {
       // For PIC, the sequence is:
       // BRIND(load(Jumptable + index) + RelocBase)
       // RelocBase can be JumpTable, GOT or some sort of global base.
       Addr = DAG.getNode(ISD::ADD, dl, PTy, Addr,
                           TLI.getPICJumpTableRelocBase(Table, DAG));
     }
     Tmp1 = DAG.getNode(ISD::BRIND, dl, MVT::Other, LD.getValue(1), Addr);
     Results.push_back(Tmp1);

lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp

if (cast<ConstantSDNode>(this)->isOpaque())
       return "OpaqueConstant";
     return "Constant";
   case ISD::ConstantFP:                 return "ConstantFP";
   case ISD::GlobalAddress:              return "GlobalAddress";
   case ISD::GlobalTLSAddress:           return "GlobalTLSAddress";
   case ISD::FrameIndex:                 return "FrameIndex";
   case ISD::JumpTable:                  return "JumpTable";
   case ISD::GLOBAL_OFFSET_TABLE:        return "GLOBAL_OFFSET_TABLE";
+  case ISD::FUNCBASE:                   return "FUNCBASE";
   case ISD::RETURNADDR:                 return "RETURNADDR";
   case ISD::ADDROFRETURNADDR:           return "ADDROFRETURNADDR";
   case ISD::FRAMEADDR:                  return "FRAMEADDR";
   case ISD::LOCAL_RECOVER:              return "LOCAL_RECOVER";
   case ISD::READ_REGISTER:              return "READ_REGISTER";
   case ISD::WRITE_REGISTER:             return "WRITE_REGISTER";
   case ISD::FRAME_TO_ARGS_OFFSET:       return "FRAME_TO_ARGS_OFFSET";
   case ISD::EH_DWARF_CFA:               return "EH_DWARF_CFA";

lib/CodeGen/SelectionDAG/TargetLowering.cpp

if (getTargetMachine().getMCAsmInfo()->getGPRel32Directive() != nullptr)
     return MachineJumpTableInfo::EK_GPRel32BlockAddress;
 
   // Otherwise, use a label difference.
   return MachineJumpTableInfo::EK_LabelDifference32;
 }
 
 SDValue TargetLowering::getPICJumpTableRelocBase(SDValue Table,
                                                  SelectionDAG &DAG) const {
-  // If our PIC model is GP relative, use the global offset table as the base.
-  unsigned JTEncoding = getJumpTableEncoding();
-
-  if ((JTEncoding == MachineJumpTableInfo::EK_GPRel64BlockAddress) ||
-      (JTEncoding == MachineJumpTableInfo::EK_GPRel32BlockAddress))
+  switch (getJumpTableEncoding()) {
+  case MachineJumpTableInfo::EK_GPRel32BlockAddress:
+  case MachineJumpTableInfo::EK_GPRel64BlockAddress:
     return DAG.getGLOBAL_OFFSET_TABLE(getPointerTy(DAG.getDataLayout()));
 
+  case MachineJumpTableInfo::EK_FuncBaseDifference32:
+  case MachineJumpTableInfo::EK_FuncBaseDifference16:
+  case MachineJumpTableInfo::EK_FuncBaseDifference8:
+    return DAG.getFUNCBASE(getPointerTy(DAG.getDataLayout()));
+
+  default:
     return Table;
   }
+}
 
 /// This returns the relocation base for the given PIC jumptable, the same as
 /// getPICJumpTableRelocBase, but as an MCExpr.
 const MCExpr *
 TargetLowering::getPICJumpTableRelocBaseExpr(const MachineFunction *MF,
                                              unsigned JTI,MCContext &Ctx) const{
+  switch (getJumpTableEncoding()) {
+  case MachineJumpTableInfo::EK_FuncBaseDifference32:
+  case MachineJumpTableInfo::EK_FuncBaseDifference16:
+  case MachineJumpTableInfo::EK_FuncBaseDifference8:
+    return MCSymbolRefExpr::create(MF->getPICBaseSymbol(), Ctx);
+  default:
     // The normal PIC reloc base is the label at the start of the jump table.
     return MCSymbolRefExpr::create(MF->getJTISymbol(JTI, Ctx), Ctx);
   }
+}
 
 bool
 TargetLowering::isOffsetFoldingLegal(const GlobalAddressSDNode *GA) const {
   const TargetMachine &TM = getTargetMachine();
   const GlobalValue *GV = GA->getGlobal();
 
   // If the address is not even local to this DSO we will have to load it from
   // a got and then add the offset.

lib/Target/X86/X86ISelLowering.h

     SDValue LowerSETCC(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerSETCCE(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerSELECT(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerBRCOND(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerJumpTable(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerDYNAMIC_STACKALLOC(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerVASTART(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerVAARG(SDValue Op, SelectionDAG &DAG) const;
+    SDValue LowerFUNCBASE(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerRETURNADDR(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerADDROFRETURNADDR(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFRAME_TO_ARGS_OFFSET(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerEH_RETURN(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerEH_SJLJ_SETJMP(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerEH_SJLJ_LONGJMP(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerEH_SJLJ_SETUP_DISPATCH(SDValue Op, SelectionDAG &DAG) const;

lib/Target/X86/X86ISelLowering.cpp

     SDValue VMask = getMaskNode(Mask, MaskVT, Subtarget, DAG, dl);
     return DAG.getMaskedLoad(VT, dl, Chain, Addr, VMask, PassThru, VT,
                              MemIntr->getMemOperand(), ISD::NON_EXTLOAD,
                              true /* expanding */);
   }
   }
 }
 
+SDValue X86TargetLowering::LowerFUNCBASE(SDValue Op, SelectionDAG &DAG) const {
+  unsigned char OpFlag = Subtarget.classifyLocalReference(nullptr);
+  MachineFunction &MF = DAG.getMachineFunction();
+  EVT PtrVT = getPointerTy(DAG.getDataLayout());
+  SDLoc DL;
+
+  SDValue Result = DAG.getMCSymbol(MF.getFuncBaseSymbol(), PtrVT);
+  Result = DAG.getNode(getGlobalWrapperKind(), DL, PtrVT, Result);
+
+  // With PIC, the address is actually $g + Offset.
+  if (OpFlag)
+    Result =
+        DAG.getNode(ISD::ADD, DL, PtrVT,
+                    DAG.getNode(X86ISD::GlobalBaseReg, SDLoc(), PtrVT), Result);
+
+  return Result;
+}
+
 SDValue X86TargetLowering::LowerRETURNADDR(SDValue Op,
                                            SelectionDAG &DAG) const {
   MachineFrameInfo &MFI = DAG.getMachineFunction().getFrameInfo();
   MFI.setReturnAddressIsTaken(true);
 
   if (verifyReturnAddressArgumentIsConstant(Op, DAG))
     return SDValue();
 

   case ISD::BRCOND:             return LowerBRCOND(Op, DAG);
   case ISD::JumpTable:          return LowerJumpTable(Op, DAG);
   case ISD::VASTART:            return LowerVASTART(Op, DAG);
   case ISD::VAARG:              return LowerVAARG(Op, DAG);
   case ISD::VACOPY:             return LowerVACOPY(Op, Subtarget, DAG);
   case ISD::INTRINSIC_WO_CHAIN: return LowerINTRINSIC_WO_CHAIN(Op, Subtarget, DAG);
   case ISD::INTRINSIC_VOID:
   case ISD::INTRINSIC_W_CHAIN:  return LowerINTRINSIC_W_CHAIN(Op, Subtarget, DAG);
+  case ISD::FUNCBASE:            return LowerFUNCBASE(Op, DAG);
   case ISD::RETURNADDR:         return LowerRETURNADDR(Op, DAG);
   case ISD::ADDROFRETURNADDR:   return LowerADDROFRETURNADDR(Op, DAG);
   case ISD::FRAMEADDR:          return LowerFRAMEADDR(Op, DAG);
   case ISD::FRAME_TO_ARGS_OFFSET:
                                 return LowerFRAME_TO_ARGS_OFFSET(Op, DAG);
   case ISD::DYNAMIC_STACKALLOC: return LowerDYNAMIC_STACKALLOC(Op, DAG);
   case ISD::EH_RETURN:          return LowerEH_RETURN(Op, DAG);
   case ISD::EH_SJLJ_SETJMP:     return lowerEH_SJLJ_SETJMP(Op, DAG);