CodeGen: Implement IR generation for the relative vtable ABI (PR26723).

[pcc]

Summary

Also add documentation for the new feature.

The new IR scheme for virtual calls
(http://lists.llvm.org/pipermail/llvm-dev/2016-February/096146.html) will
be implemented in a follow-up patch. This patch does some refactoring that
puts us in a good position to do that.

Depends on http://reviews.llvm.org/D17893

Depends on http://reviews.llvm.org/D17938

[pcc]

There's a correctness issue with this change that I'll need to resolve before this goes in; see http://lists.llvm.org/pipermail/cfe-dev/2016-March/048025.html

[pcc]

• Do not set virtual index in debug info for relative-ABI virtual functions

[pcc]

• Update test to match committed debug info change

[rsmith]

lib/CodeGen/CGDebugInfo.cpp

1153

Is this what debug info consumers expect for the non-virtual case?

[pcc]

• Use a vindex of 0 for non-virtuals

lib/CodeGen/CGDebugInfo.cpp

1153

The debug info emitter doesn't use the virtual index of non-virtuals [1], but it's probably better to use zero for non-virtuals, as that cuts down on IR writer output. Done.

[1] http://llvm-cs.pcc.me.uk/lib/CodeGen/AsmPrinter/DwarfUnit.cpp#1223

[rnk]

lib/CodeGen/ItaniumCXXABI.cpp

1625

Maybe this should live in CGVTables.cpp or CGClass.cpp instead of ItaniumCXXABI.cpp?

[pcc]

lib/CodeGen/ItaniumCXXABI.cpp

1625

Yes, sorry, I forgot to move this somewhere more sensible before sending this out for review.

[pcc]

• Use llvm.load.relative (D18367)

[pcc]

• Compute a correct byte offset for struct vtable bitsets

docs/UsersManual.rst

.. code-block:: console
       // Make an aligned typedef of the AVX-512 16-int vector type.
       typedef __v16si __aligned_v16si __attribute__((aligned(64)));
 
       void initialize_vector(__aligned_v16si *v) {
         // The compiler may assume that ‘v’ is 64-byte aligned, regardless of the
         // value of -fmax-type-align.
       }
 
+**-funstable-c++-abi-classes**
+
+**-funstable-c++-abi-classes=PATH**
+
+   These flags control whether the compiler uses a more space-efficient
+   virtual table ABI, known as the relative ABI. The first flag enables the
+   ABI for all classes in the program, while the second flag enables the
+   ABI for classes in namespaces named in the file at ``PATH``.
+
+   This ABI is not guaranteed to be stable between versions of Clang. This
+   means that it is an ODR violation to define a class with the relative
+   ABI in two translation units compiled with different versions of
+   Clang. Specifically, mixing different head revisions or major releases
+   is not allowed, but mixing different point releases is fine.
+
+   The format of the namespace list file is a series of lines ending in
+   either ``*`` or ``**``. Preceding that is a namespace specifier delimited
+   by double-colons followed by ``::``, or the empty string to denote the
+   global namespace. ``#`` serves as a comment character. Each entry in the
+   list indicates that classes in that namespace, including nested classes,
+   classes defined in enclosed anonymous namespaces, and classes defined
+   within member functions of those classes, may use the relative ABI.
+
+   If the line ends in ``*`` this applies to the given namespace only,
+   while if the line ends in ``**`` it applies to the given namespace and
+   any enclosed namespaces.
+
+   It is not guaranteed that all such classes will use the relative ABI. For
+   example, if a base class uses the platform ABI and declares virtual functions,
+   that will force any derived classes to use the platform ABI. The compiler
+   will diagnose any cases where a class derives from multiple incompatible
+   base classes.
+
+   Example:
+
+   .. code-block:: console
+
+      # Enables the unstable ABI for classes in the global namespace.
+      *
+
+      # Enables the unstable ABI for classes in the base namespace, and
+      # any enclosed namespaces.
+      base::**
+
+   Note that it is recommended to use the ``[[clang::unstable_abi]]``
+   attribute instead of these flags wherever possible.
 
 Profile Guided Optimization
 ---------------------------
 
 Profile information enables better optimization. For example, knowing that a
 branch is taken very frequently helps the compiler make better decisions when
 ordering basic blocks. Knowing that a function ``foo`` is called more
 frequently than another function ``bar`` helps the inliner.

lib/CodeGen/CGClass.cpp

llvm::Value *CodeGenFunction::GetVTablePtr(Address This,
 
   if (CGM.getCodeGenOpts().OptimizationLevel > 0 &&
       CGM.getCodeGenOpts().StrictVTablePointers)
     CGM.DecorateInstructionWithInvariantGroup(VTable, RD);
 
   return VTable;
 }
 
+llvm::Value *CodeGenFunction::GetVirtualFunctionFromVTable(const CXXRecordDecl *RD,
+                                                           llvm::Value *VTable,
+                                                           uint64_t VTableIndex,
+                                                           llvm::Type *Ty) {
+  if (!CGM.getVTables().shouldUseRelativeABI(RD)) {
+    VTable = Builder.CreateBitCast(VTable, Ty->getPointerTo()->getPointerTo());
+    llvm::Value *VTableSlotPtr =
+        Builder.CreateConstInBoundsGEP1_64(VTable, VTableIndex, "vfn");
+    return Builder.CreateAlignedLoad(VTableSlotPtr, getPointerAlign());
+  }
+
+  VTable = Builder.CreateBitCast(VTable, Int8PtrTy);
+  llvm::Value *Load = Builder.CreateCall(
+      CGM.getIntrinsic(llvm::Intrinsic::load_relative, {Int32Ty}),
+      {VTable, llvm::ConstantInt::get(Int32Ty, 4 * VTableIndex)});
+  return Builder.CreateBitCast(Load, Ty->getPointerTo());
+}
+
 // If a class has a single non-virtual base and does not introduce or override
 // virtual member functions or fields, it will have the same layout as its base.
 // This function returns the least derived such class.
 //
 // Casting an instance of a base class to such a derived class is technically
 // undefined behavior, but it is a relatively common hack for introducing member
 // functions on class instances with specific properties (e.g. llvm::Operator)
 // that works under most compilers and should not have security implications, so

lib/CodeGen/CGDebugInfo.cpp

llvm::DISubprogram *CGDebugInfo::CreateCXXMemberFunction(
   if (!Method->isImplicit()) {
     MethodDefUnit = getOrCreateFile(Method->getLocation());
     MethodLine = getLineNumber(Method->getLocation());
   }
 
   // Collect virtual method info.
   llvm::DIType *ContainingType = nullptr;
   unsigned Virtuality = 0;
   unsigned VIndex = 0;
 
   if (Method->isVirtual()) {
     if (Method->isPure())
       Virtuality = llvm::dwarf::DW_VIRTUALITY_pure_virtual;
     else
       Virtuality = llvm::dwarf::DW_VIRTUALITY_virtual;
 
     // It doesn't make sense to give a virtual destructor a vtable index,
     // since a single destructor has two entries in the vtable.
     // FIXME: Add proper support for debug info for virtual calls in
     // the Microsoft ABI, where we may use multiple vptrs to make a vftable
     // lookup if we have multiple or virtual inheritance.
+    // FIXME: Add support for debug info for virtual calls in the relative
+    // ABI, where the virtual function address needs to be calculated from
+    // the virtual table address.
     if (!isa<CXXDestructorDecl>(Method) &&
-        !CGM.getTarget().getCXXABI().isMicrosoft())
+        !CGM.getTarget().getCXXABI().isMicrosoft() &&
+        !CGM.getVTables().shouldUseRelativeABI(Method->getParent()))
       VIndex = CGM.getItaniumVTableContext().getMethodVTableIndex(Method);
+    else
+      // -1u indicates that the vtable index is non-representable.
+      VIndex = -1u;
     ContainingType = RecordTy;
   }
 
   unsigned Flags = 0;
   if (Method->isImplicit())
     Flags |= llvm::DINode::FlagArtificial;
   Flags |= getAccessFlag(Method->getAccess(), Method->getParent());
   if (const CXXConstructorDecl *CXXC = dyn_cast<CXXConstructorDecl>(Method)) {

lib/CodeGen/CGVTables.h

class CodeGenVTables {
   /// emitThunk - Emit a single thunk.
   void emitThunk(GlobalDecl GD, const ThunkInfo &Thunk, bool ForVTable);
 
   /// maybeEmitThunkForVTable - Emit the given thunk for the vtable if needed by
   /// the ABI.
   void maybeEmitThunkForVTable(GlobalDecl GD, const ThunkInfo &Thunk);
 
 public:
-  /// CreateVTableInitializer - Create a vtable initializer for the given record
-  /// decl.
+  /// SetVTableInitializer - Set VTable's initializer to the vtable initializer
+  /// for the given record decl.
   /// \param Components - The vtable components; this is really an array of
   /// VTableComponents.
-  llvm::Constant *CreateVTableInitializer(
-      const CXXRecordDecl *RD, const VTableComponent *Components,
-      unsigned NumComponents, const VTableLayout::VTableThunkTy *VTableThunks,
+  void SetVTableInitializer(llvm::GlobalVariable *VTable,
+                            const CXXRecordDecl *RD,
+                            const VTableComponent *Components,
+                            unsigned NumComponents,
+                            const VTableLayout::VTableThunkTy *VTableThunks,
                             unsigned NumVTableThunks, llvm::Constant *RTTI);
 
   CodeGenVTables(CodeGenModule &CGM);
 
   ItaniumVTableContext &getItaniumVTableContext() {
     return *cast<ItaniumVTableContext>(VTContext);
   }
 
   MicrosoftVTableContext &getMicrosoftVTableContext() {
public:
     
   /// GenerateClassData - Generate all the class data required to be
   /// generated upon definition of a KeyFunction.  This includes the
   /// vtable, the RTTI data structure (if RTTI is enabled) and the VTT
   /// (if the class has virtual bases).
   void GenerateClassData(const CXXRecordDecl *RD);
 
   bool isVTableExternal(const CXXRecordDecl *RD);
+
+  /// Whether to use the relative vtable ABI for this record.
+  bool shouldUseRelativeABI(const CXXRecordDecl *RD);
+
+  /// Returns the type of a vtable with the given layout. Normally an array of
+  /// pointers, but may be a struct under the relative vtable ABI.
+  llvm::Type *GetVTableType(const CXXRecordDecl *RD,
+                            const VTableLayout &VTLayout);
 };
 
 } // end namespace CodeGen
 } // end namespace clang
 #endif

lib/CodeGen/CGVTables.cpp

void CodeGenVTables::EmitThunks(GlobalDecl GD)
 
   if (!ThunkInfoVector)
     return;
 
   for (const ThunkInfo& Thunk : *ThunkInfoVector)
     emitThunk(GD, Thunk, /*ForVTable=*/false);
 }
 
-llvm::Constant *CodeGenVTables::CreateVTableInitializer(
-    const CXXRecordDecl *RD, const VTableComponent *Components,
-    unsigned NumComponents, const VTableLayout::VTableThunkTy *VTableThunks,
-    unsigned NumVTableThunks, llvm::Constant *RTTI) {
+bool CodeGenVTables::shouldUseRelativeABI(const CXXRecordDecl *RD) {
+  return RD->isUnstableCXXABI();
+}
+
+llvm::Type *CodeGenVTables::GetVTableType(const CXXRecordDecl *RD,
+                                          const VTableLayout &VTLayout) {
+  if (!shouldUseRelativeABI(RD))
+    return llvm::ArrayType::get(CGM.Int8PtrTy,
+                                VTLayout.getNumVTableComponents());
+
+  std::vector<llvm::Type *> Types;
+  for (auto &Comp : VTLayout.vtable_components()) {
+    if (Comp.isFunctionPointerKind())
+      Types.push_back(CGM.Int32Ty);
+    else
+      Types.push_back(CGM.Int8PtrTy);
+  }
+  return llvm::StructType::get(CGM.getLLVMContext(), Types);
+}
+
+void CodeGenVTables::SetVTableInitializer(
+    llvm::GlobalVariable *VTable, const CXXRecordDecl *RD,
+    const VTableComponent *Components, unsigned NumComponents,
+    const VTableLayout::VTableThunkTy *VTableThunks, unsigned NumVTableThunks,
+    llvm::Constant *RTTI) {
   SmallVector<llvm::Constant *, 64> Inits;
 
   llvm::Type *Int8PtrTy = CGM.Int8PtrTy;
+  llvm::Type *Int32Ty = CGM.Int32Ty;
   
   llvm::Type *PtrDiffTy = 
     CGM.getTypes().ConvertType(CGM.getContext().getPointerDiffType());
 
   unsigned NextVTableThunkIndex = 0;
 
   llvm::Constant *PureVirtualFn = nullptr, *DeletedVirtualFn = nullptr;
 
+  bool RelativeABI = shouldUseRelativeABI(RD);
+  llvm::Type *VTableTy = VTable->getValueType();
+  llvm::Type *FunctionPtrTy = RelativeABI ? Int32Ty : Int8PtrTy;
+  llvm::Constant *AddrPointInt;
+
+  auto maybeMakeRelative = [&](llvm::Constant *C) {
+    if (!RelativeABI)
+      return C;
+    return llvm::ConstantExpr::getIntegerCast(
+        llvm::ConstantExpr::getSub(
+            llvm::ConstantExpr::getPtrToInt(C, PtrDiffTy), AddrPointInt),
+        Int32Ty, /*isSigned=*/true);
+  };
+
   for (unsigned I = 0; I != NumComponents; ++I) {
     VTableComponent Component = Components[I];
 
+    if (RelativeABI && Component.isFunctionPointerKind() &&
+        (I == 0 || !Components[I - 1].isFunctionPointerKind())) {
+      // FIXME: Need a better way of identifying address points that works with
+      // the Itanium and MS ABIs.
+      AddrPointInt = llvm::ConstantExpr::getPtrToInt(
+          llvm::ConstantExpr::getGetElementPtr(
+              VTableTy, VTable,
+              llvm::ArrayRef<llvm::Constant *>{
+                  llvm::ConstantInt::get(Int32Ty, 0),
+                  llvm::ConstantInt::get(Int32Ty, I)}),
+          PtrDiffTy);
+    }
+
     llvm::Constant *Init = nullptr;
 
     switch (Component.getKind()) {
     case VTableComponent::CK_VCallOffset:
       Init = llvm::ConstantInt::get(PtrDiffTy, 
                                     Component.getVCallOffset().getQuantity());
       Init = llvm::ConstantExpr::getIntToPtr(Init, Int8PtrTy);
       break;
case VTableComponent::CK_DeletingDtorPointer: {
         const CXXMethodDecl *MD = cast<CXXMethodDecl>(GD.getDecl());
         // OK on device side: functions w/ __device__ attribute
         // OK on host side: anything except __device__-only functions.
         bool CanEmitMethod = CGM.getLangOpts().CUDAIsDevice
                                  ? MD->hasAttr<CUDADeviceAttr>()
                                  : (MD->hasAttr<CUDAHostAttr>() ||
                                     !MD->hasAttr<CUDADeviceAttr>());
         if (!CanEmitMethod) {
-          Init = llvm::ConstantExpr::getNullValue(Int8PtrTy);
+          Init = llvm::ConstantExpr::getNullValue(FunctionPtrTy);
           break;
         }
         // Method is acceptable, continue processing as usual.
       }
 
       if (cast<CXXMethodDecl>(GD.getDecl())->isPure()) {
         // We have a pure virtual member function.
         if (!PureVirtualFn) {
           llvm::FunctionType *Ty = 
             llvm::FunctionType::get(CGM.VoidTy, /*isVarArg=*/false);
           StringRef PureCallName = CGM.getCXXABI().GetPureVirtualCallName();
           PureVirtualFn = CGM.CreateRuntimeFunction(Ty, PureCallName);
           if (auto *F = dyn_cast<llvm::Function>(PureVirtualFn))
             F->setUnnamedAddr(true);
           PureVirtualFn = llvm::ConstantExpr::getBitCast(PureVirtualFn,
                                                          CGM.Int8PtrTy);
         }
-        Init = PureVirtualFn;
+        Init = maybeMakeRelative(PureVirtualFn);
       } else if (cast<CXXMethodDecl>(GD.getDecl())->isDeleted()) {
         if (!DeletedVirtualFn) {
           llvm::FunctionType *Ty =
             llvm::FunctionType::get(CGM.VoidTy, /*isVarArg=*/false);
           StringRef DeletedCallName =
             CGM.getCXXABI().GetDeletedVirtualCallName();
           DeletedVirtualFn = CGM.CreateRuntimeFunction(Ty, DeletedCallName);
           if (auto *F = dyn_cast<llvm::Function>(DeletedVirtualFn))
             F->setUnnamedAddr(true);
           DeletedVirtualFn = llvm::ConstantExpr::getBitCast(DeletedVirtualFn,
                                                          CGM.Int8PtrTy);
         }
-        Init = DeletedVirtualFn;
+        Init = maybeMakeRelative(DeletedVirtualFn);
       } else {
         // Check if we should use a thunk.
         if (NextVTableThunkIndex < NumVTableThunks &&
             VTableThunks[NextVTableThunkIndex].first == I) {
           const ThunkInfo &Thunk = VTableThunks[NextVTableThunkIndex].second;
         
           maybeEmitThunkForVTable(GD, Thunk);
           Init = CGM.GetAddrOfThunk(GD, Thunk);
 
           NextVTableThunkIndex++;
         } else {
           llvm::Type *Ty = CGM.getTypes().GetFunctionTypeForVTable(GD);
         
           Init = CGM.GetAddrOfFunction(GD, Ty, /*ForVTable=*/true);
         }
 
         Init = llvm::ConstantExpr::getBitCast(Init, Int8PtrTy);
+        Init = maybeMakeRelative(Init);
       }
       break;
     }
 
     case VTableComponent::CK_UnusedFunctionPointer:
-      Init = llvm::ConstantExpr::getNullValue(Int8PtrTy);
+      Init = llvm::ConstantExpr::getNullValue(FunctionPtrTy);
       break;
     };
     
     Inits.push_back(Init);
   }
-  
-  llvm::ArrayType *ArrayType = llvm::ArrayType::get(Int8PtrTy, NumComponents);
-  return llvm::ConstantArray::get(ArrayType, Inits);
+  if (RelativeABI) {
+    VTable->setInitializer(
+        llvm::ConstantStruct::get(cast<llvm::StructType>(VTableTy), Inits));
+  } else {
+    VTable->setInitializer(
+        llvm::ConstantArray::get(cast<llvm::ArrayType>(VTableTy), Inits));
+  }
 }
 
 llvm::GlobalVariable *
 CodeGenVTables::GenerateConstructionVTable(const CXXRecordDecl *RD, 
                                       const BaseSubobject &Base, 
                                       bool BaseIsVirtual, 
                                    llvm::GlobalVariable::LinkageTypes Linkage,
                                       VTableAddressPointsMapTy& AddressPoints) {
CodeGenVTables::GenerateConstructionVTable(const CXXRecordDecl *RD,
   // Get the mangled construction vtable name.
   SmallString<256> OutName;
   llvm::raw_svector_ostream Out(OutName);
   cast<ItaniumMangleContext>(CGM.getCXXABI().getMangleContext())
       .mangleCXXCtorVTable(RD, Base.getBaseOffset().getQuantity(),
                            Base.getBase(), Out);
   StringRef Name = OutName.str();
 
-  llvm::ArrayType *ArrayType = 
-    llvm::ArrayType::get(CGM.Int8PtrTy, VTLayout->getNumVTableComponents());
+  llvm::Type *VTType = GetVTableType(RD, *VTLayout);
 
   // Construction vtable symbols are not part of the Itanium ABI, so we cannot
   // guarantee that they actually will be available externally. Instead, when
   // emitting an available_externally VTT, we provide references to an internal
   // linkage construction vtable. The ABI only requires complete-object vtables
   // to be the same for all instances of a type, not construction vtables.
   if (Linkage == llvm::GlobalVariable::AvailableExternallyLinkage)
     Linkage = llvm::GlobalVariable::InternalLinkage;
 
   // Create the variable that will hold the construction vtable.
   llvm::GlobalVariable *VTable = 
-    CGM.CreateOrReplaceCXXRuntimeVariable(Name, ArrayType, Linkage);
+    CGM.CreateOrReplaceCXXRuntimeVariable(Name, VTType, Linkage);
   CGM.setGlobalVisibility(VTable, RD);
 
   // V-tables are always unnamed_addr.
   VTable->setUnnamedAddr(true);
 
   llvm::Constant *RTTI = CGM.GetAddrOfRTTIDescriptor(
       CGM.getContext().getTagDeclType(Base.getBase()));
 
   // Create and set the initializer.
-  llvm::Constant *Init = CreateVTableInitializer(
-      Base.getBase(), VTLayout->vtable_component_begin(),
+  SetVTableInitializer(
+      VTable, Base.getBase(), VTLayout->vtable_component_begin(),
       VTLayout->getNumVTableComponents(), VTLayout->vtable_thunk_begin(),
       VTLayout->getNumVTableThunks(), RTTI);
-  VTable->setInitializer(Init);
-  
   CGM.EmitVTableBitSetEntries(VTable, *VTLayout.get());
 
   return VTable;
 }
 
 static bool shouldEmitAvailableExternallyVTable(const CodeGenModule &CGM,
                                                 const CXXRecordDecl *RD) {
   return CGM.getCodeGenOpts().OptimizationLevel > 0 &&
std::sort(BitsetEntries.begin(), BitsetEntries.end(),
     if (S1 != S2)
       return false;
 
     return E1.second < E2.second;
   });
 
   llvm::NamedMDNode *BitsetsMD =
       getModule().getOrInsertNamedMetadata("llvm.bitsets");
-  for (auto BitsetEntry : BitsetEntries)
-    CreateVTableBitSetEntry(BitsetsMD, VTable,
-                            PointerWidth * BitsetEntry.second,
-                            BitsetEntry.first);
+
+  const llvm::StructLayout *SL = nullptr;
+  if (auto *ST = dyn_cast<llvm::StructType>(VTable->getValueType()))
+    SL = getDataLayout().getStructLayout(ST);
+
+  for (auto BitsetEntry : BitsetEntries) {
+    CharUnits ByteOffset;
+    if (SL)
+      ByteOffset =
+          CharUnits::fromQuantity(SL->getElementOffset(BitsetEntry.second));
+    else
+      ByteOffset = PointerWidth * BitsetEntry.second;
+    CreateVTableBitSetEntry(BitsetsMD, VTable, ByteOffset, BitsetEntry.first);
+  }
 }

lib/CodeGen/CodeGenFunction.h

public:
 
   void InitializeVTablePointers(const CXXRecordDecl *ClassDecl);
 
   /// GetVTablePtr - Return the Value of the vtable pointer member pointed
   /// to by This.
   llvm::Value *GetVTablePtr(Address This, llvm::Type *VTableTy,
                             const CXXRecordDecl *VTableClass);
 
+  llvm::Value *GetVirtualFunctionFromVTable(const CXXRecordDecl *RD,
+                                            llvm::Value *VTable,
+                                            uint64_t VTableIndex,
+                                            llvm::Type *Ty);
+
   enum CFITypeCheckKind {
     CFITCK_VCall,
     CFITCK_NVCall,
     CFITCK_DerivedCast,
     CFITCK_UnrelatedCast,
     CFITCK_ICall,
   };
 

lib/CodeGen/ItaniumCXXABI.cpp

llvm::Value *ItaniumCXXABI::EmitLoadOfMemberFunctionPointer(
   // Cast the adjusted this to a pointer to vtable pointer and load.
   llvm::Type *VTableTy = Builder.getInt8PtrTy();
   CharUnits VTablePtrAlign =
     CGF.CGM.getDynamicOffsetAlignment(ThisAddr.getAlignment(), RD,
                                       CGF.getPointerAlign());
   llvm::Value *VTable =
     CGF.GetVTablePtr(Address(This, VTablePtrAlign), VTableTy, RD);
 
-  // Apply the offset.
   llvm::Value *VTableOffset = FnAsInt;
   if (!UseARMMethodPtrABI)
     VTableOffset = Builder.CreateSub(VTableOffset, ptrdiff_1);
-  VTable = Builder.CreateGEP(VTable, VTableOffset);
 
   // Load the virtual function to call.
-  VTable = Builder.CreateBitCast(VTable, FTy->getPointerTo()->getPointerTo());
-  llvm::Value *VirtualFn =
-    Builder.CreateAlignedLoad(VTable, CGF.getPointerAlign(),
+  llvm::Value *VirtualFn;
+  if (CGM.getVTables().shouldUseRelativeABI(RD)) {
+    VirtualFn =
+        Builder.CreateCall(CGM.getIntrinsic(llvm::Intrinsic::load_relative,
+                                            {VTableOffset->getType()}),
+                           {VTable, VTableOffset});
+    VirtualFn = Builder.CreateBitCast(VirtualFn, FTy->getPointerTo());
+  } else {
+    // Apply the offset.
+    llvm::Value *VTableSlotPtr = Builder.CreateGEP(VTable, VTableOffset);
+    VTableSlotPtr = Builder.CreateBitCast(VTableSlotPtr,
+                                          FTy->getPointerTo()->getPointerTo());
+    VirtualFn = Builder.CreateAlignedLoad(VTableSlotPtr, CGF.getPointerAlign(),
                                           "memptr.virtualfn");
+  }
   CGF.EmitBranch(FnEnd);
 
   // In the non-virtual path, the function pointer is actually a
   // function pointer.
   CGF.EmitBlock(FnNonVirtual);
   llvm::Value *NonVirtualFn =
     Builder.CreateIntToPtr(FnAsInt, FTy->getPointerTo(), "memptr.nonvirtualfn");
   
llvm::Constant *ItaniumCXXABI::BuildMemberPointer(const CXXMethodDecl *MD,
   // Get the function pointer (or index if this is a virtual function).
   llvm::Constant *MemPtr[2];
   if (MD->isVirtual()) {
     uint64_t Index = CGM.getItaniumVTableContext().getMethodVTableIndex(MD);
 
     const ASTContext &Context = getContext();
     CharUnits PointerWidth =
       Context.toCharUnitsFromBits(Context.getTargetInfo().getPointerWidth(0));
-    uint64_t VTableOffset = (Index * PointerWidth.getQuantity());
+    uint64_t VTableOffset = Index;
+    if (CGM.getVTables().shouldUseRelativeABI(MD->getParent()))
+      VTableOffset *= 4;
+    else
+      VTableOffset *= PointerWidth.getQuantity();
 
     if (UseARMMethodPtrABI) {
       // ARM C++ ABI 3.2.1:
       //   This ABI specifies that adj contains twice the this
       //   adjustment, plus 1 if the member function is virtual. The
       //   least significant bit of adj then makes exactly the same
       //   discrimination as the least significant bit of ptr does for
       //   Itanium.
void ItaniumCXXABI::emitVTableDefinitions(CodeGenVTables &CGVT,
 
   ItaniumVTableContext &VTContext = CGM.getItaniumVTableContext();
   const VTableLayout &VTLayout = VTContext.getVTableLayout(RD);
   llvm::GlobalVariable::LinkageTypes Linkage = CGM.getVTableLinkage(RD);
   llvm::Constant *RTTI =
       CGM.GetAddrOfRTTIDescriptor(CGM.getContext().getTagDeclType(RD));
 
   // Create and set the initializer.
-  llvm::Constant *Init = CGVT.CreateVTableInitializer(
-      RD, VTLayout.vtable_component_begin(), VTLayout.getNumVTableComponents(),
-      VTLayout.vtable_thunk_begin(), VTLayout.getNumVTableThunks(), RTTI);
-  VTable->setInitializer(Init);
+  CGVT.SetVTableInitializer(VTable, RD, VTLayout.vtable_component_begin(),
+                            VTLayout.getNumVTableComponents(),
+                            VTLayout.vtable_thunk_begin(),
+                            VTLayout.getNumVTableThunks(), RTTI);
 
   // Set the correct linkage.
   VTable->setLinkage(Linkage);
 
   if (CGM.supportsCOMDAT() && VTable->isWeakForLinker())
     VTable->setComdat(CGM.getModule().getOrInsertComdat(VTable->getName()));
 
   // Set the right visibility.
llvm::GlobalVariable *ItaniumCXXABI::getAddrOfVTable(const CXXRecordDecl *RD,
 
   // Queue up this vtable for possible deferred emission.
   CGM.addDeferredVTable(RD);
 
   SmallString<256> Name;
   llvm::raw_svector_ostream Out(Name);
   getMangleContext().mangleCXXVTable(RD, Out);
 
-  ItaniumVTableContext &VTContext = CGM.getItaniumVTableContext();
-  llvm::ArrayType *ArrayType = llvm::ArrayType::get(
-      CGM.Int8PtrTy, VTContext.getVTableLayout(RD).getNumVTableComponents());
+  const VTableLayout &VTLayout =
+      CGM.getItaniumVTableContext().getVTableLayout(RD);
+  llvm::Type *VTableType = CGM.getVTables().GetVTableType(RD, VTLayout);
 
   VTable = CGM.CreateOrReplaceCXXRuntimeVariable(
-      Name, ArrayType, llvm::GlobalValue::ExternalLinkage);
+      Name, VTableType, llvm::GlobalValue::ExternalLinkage);
   VTable->setUnnamedAddr(true);
 
   if (RD->hasAttr<DLLImportAttr>())
     VTable->setDLLStorageClass(llvm::GlobalValue::DLLImportStorageClass);
   else if (RD->hasAttr<DLLExportAttr>())
     VTable->setDLLStorageClass(llvm::GlobalValue::DLLExportStorageClass);
 
   return VTable;
 }
 
 llvm::Value *ItaniumCXXABI::getVirtualFunctionPointer(CodeGenFunction &CGF,
                                                       GlobalDecl GD,
                                                       Address This,
                                                       llvm::Type *Ty,
                                                       SourceLocation Loc) {
   GD = GD.getCanonicalDecl();
-  Ty = Ty->getPointerTo()->getPointerTo();
   auto *MethodDecl = cast<CXXMethodDecl>(GD.getDecl());
-  llvm::Value *VTable = CGF.GetVTablePtr(This, Ty, MethodDecl->getParent());
+  llvm::Value *VTable = CGF.GetVTablePtr(
+      This, Ty->getPointerTo()->getPointerTo(), MethodDecl->getParent());
 
   CGF.EmitBitSetCodeForVCall(MethodDecl->getParent(), VTable, Loc);
 
   uint64_t VTableIndex = CGM.getItaniumVTableContext().getMethodVTableIndex(GD);
-  llvm::Value *VFuncPtr =
-      CGF.Builder.CreateConstInBoundsGEP1_64(VTable, VTableIndex, "vfn");
-  return CGF.Builder.CreateAlignedLoad(VFuncPtr, CGF.getPointerAlign());
+  return CGF.GetVirtualFunctionFromVTable(MethodDecl->getParent(), VTable,
+                                          VTableIndex, Ty);
 }
 
 llvm::Value *ItaniumCXXABI::EmitVirtualDestructorCall(
     CodeGenFunction &CGF, const CXXDestructorDecl *Dtor, CXXDtorType DtorType,
     Address This, const CXXMemberCallExpr *CE) {
   assert(CE == nullptr || CE->arg_begin() == CE->arg_end());
   assert(DtorType == Dtor_Deleting || DtorType == Dtor_Complete);
 
   const CGFunctionInfo *FInfo = &CGM.getTypes().arrangeCXXStructorDeclaration(
       Dtor, getFromDtorType(DtorType));
   llvm::Type *Ty = CGF.CGM.getTypes().GetFunctionType(*FInfo);

lib/CodeGen/MicrosoftCXXABI.cpp

for (VPtrInfo *Info : VFPtrs) {
     const VTableLayout &VTLayout =
       VFTContext.getVFTableLayout(RD, Info->FullOffsetInMDC);
 
     llvm::Constant *RTTI = nullptr;
     if (any_of(VTLayout.vtable_components(),
                [](const VTableComponent &VTC) { return VTC.isRTTIKind(); }))
       RTTI = getMSCompleteObjectLocator(RD, Info);
 
-    llvm::Constant *Init = CGVT.CreateVTableInitializer(
-        RD, VTLayout.vtable_component_begin(),
-        VTLayout.getNumVTableComponents(), VTLayout.vtable_thunk_begin(),
+    CGVT.SetVTableInitializer(VTable, RD, VTLayout.vtable_component_begin(),
+                              VTLayout.getNumVTableComponents(),
+                              VTLayout.vtable_thunk_begin(),
                               VTLayout.getNumVTableThunks(), RTTI);
 
-    VTable->setInitializer(Init);
-
     emitVTableBitSetEntries(Info, RD, VTable);
   }
 }
 
 bool MicrosoftCXXABI::isVirtualOffsetNeededForVTableField(
     CodeGenFunction &CGF, CodeGenFunction::VPtr Vptr) {
   return Vptr.NearestVBase != nullptr;
 }
if (llvm::GlobalValue *VFTable =
     VFTablesMap[ID] = VFTable;
     VTable = VTableAliasIsRequred
                  ? cast<llvm::GlobalVariable>(
                        cast<llvm::GlobalAlias>(VFTable)->getBaseObject())
                  : cast<llvm::GlobalVariable>(VFTable);
     return VTable;
   }
 
-  uint64_t NumVTableSlots =
-      VTContext.getVFTableLayout(RD, VFPtr->FullOffsetInMDC)
-          .getNumVTableComponents();
+  const VTableLayout &VTLayout =
+      VTContext.getVFTableLayout(RD, VFPtr->FullOffsetInMDC);
   llvm::GlobalValue::LinkageTypes VTableLinkage =
       VTableAliasIsRequred ? llvm::GlobalValue::PrivateLinkage : VFTableLinkage;
 
   StringRef VTableName = VTableAliasIsRequred ? StringRef() : VFTableName.str();
 
-  llvm::ArrayType *VTableType =
-      llvm::ArrayType::get(CGM.Int8PtrTy, NumVTableSlots);
+  llvm::Type *VTableType = CGM.getVTables().GetVTableType(RD, VTLayout);
 
   // Create a backing variable for the contents of VTable.  The VTable may
   // or may not include space for a pointer to RTTI data.
   llvm::GlobalValue *VFTable;
   VTable = new llvm::GlobalVariable(CGM.getModule(), VTableType,
                                     /*isConstant=*/true, VTableLinkage,
                                     /*Initializer=*/nullptr, VTableName);
   VTable->setUnnamedAddr(true);
 
   llvm::Comdat *C = nullptr;
   if (!VFTableComesFromAnotherTU &&
       (llvm::GlobalValue::isWeakForLinker(VFTableLinkage) ||
        (llvm::GlobalValue::isLocalLinkage(VFTableLinkage) &&
         VTableAliasIsRequred)))
     C = CGM.getModule().getOrInsertComdat(VFTableName.str());
 
   // Only insert a pointer into the VFTable for RTTI data if we are not
   // importing it.  We never reference the RTTI data directly so there is no
   // need to make room for it.
   if (VTableAliasIsRequred) {
-    llvm::Value *GEPIndices[] = {llvm::ConstantInt::get(CGM.IntTy, 0),
-                                 llvm::ConstantInt::get(CGM.IntTy, 1)};
+    llvm::Value *GEPIndices[] = {llvm::ConstantInt::get(CGM.Int32Ty, 0),
+                                 llvm::ConstantInt::get(CGM.Int32Ty, 1)};
     // Create a GEP which points just after the first entry in the VFTable,
     // this should be the location of the first virtual method.
     llvm::Constant *VTableGEP = llvm::ConstantExpr::getInBoundsGetElementPtr(
         VTable->getValueType(), VTable, GEPIndices);
     if (llvm::GlobalValue::isWeakForLinker(VFTableLinkage)) {
       VFTableLinkage = llvm::GlobalValue::ExternalLinkage;
       if (C)
         C->setSelectionKind(llvm::Comdat::Largest);
     }
+    VTableGEP = llvm::ConstantExpr::getBitCast(VTableGEP, CGM.Int8PtrPtrTy);
     VFTable = llvm::GlobalAlias::create(CGM.Int8PtrTy,
                                         /*AddressSpace=*/0, VFTableLinkage,
                                         VFTableName.str(), VTableGEP,
                                         &CGM.getModule());
     VFTable->setUnnamedAddr(true);
   } else {
     // We don't need a GlobalAlias to be a symbol for the VTable if we won't
     // be referencing any RTTI data.

 }
 
 llvm::Value *MicrosoftCXXABI::getVirtualFunctionPointer(CodeGenFunction &CGF,
                                                         GlobalDecl GD,
                                                         Address This,
                                                         llvm::Type *Ty,
                                                         SourceLocation Loc) {
   GD = GD.getCanonicalDecl();
-  CGBuilderTy &Builder = CGF.Builder;
 
-  Ty = Ty->getPointerTo()->getPointerTo();
   Address VPtr =
       adjustThisArgumentForVirtualFunctionCall(CGF, GD, This, true);
 
   auto *MethodDecl = cast<CXXMethodDecl>(GD.getDecl());
-  llvm::Value *VTable = CGF.GetVTablePtr(VPtr, Ty, MethodDecl->getParent());
+  llvm::Value *VTable = CGF.GetVTablePtr(
+      VPtr, Ty->getPointerTo()->getPointerTo(), MethodDecl->getParent());
 
   MicrosoftVTableContext::MethodVFTableLocation ML =
       CGM.getMicrosoftVTableContext().getMethodVFTableLocation(GD);
   if (CGM.NeedVTableBitSets())
     CGF.EmitBitSetCodeForVCall(getClassAtVTableLocation(getContext(), GD, ML),
                                VTable, Loc);
 
-  llvm::Value *VFuncPtr =
-      Builder.CreateConstInBoundsGEP1_64(VTable, ML.Index, "vfn");
-  return Builder.CreateAlignedLoad(VFuncPtr, CGF.getPointerAlign());
+  return CGF.GetVirtualFunctionFromVTable(MethodDecl->getParent(), VTable,
+                                          ML.Index, Ty);
 }
 
 llvm::Value *MicrosoftCXXABI::EmitVirtualDestructorCall(
     CodeGenFunction &CGF, const CXXDestructorDecl *Dtor, CXXDtorType DtorType,
     Address This, const CXXMemberCallExpr *CE) {
   assert(CE == nullptr || CE->arg_begin() == CE->arg_end());
   assert(DtorType == Dtor_Deleting || DtorType == Dtor_Complete);
 
llvm::Function *MicrosoftCXXABI::EmitVirtualMemPtrThunk(
   // Start defining the function.
   CGF.StartFunction(GlobalDecl(), FnInfo.getReturnType(), ThunkFn, FnInfo,
                     FunctionArgs, MD->getLocation(), SourceLocation());
   EmitThisParam(CGF);
 
   // Load the vfptr and then callee from the vftable.  The callee should have
   // adjusted 'this' so that the vfptr is at offset zero.
   llvm::Value *VTable = CGF.GetVTablePtr(
-      getThisAddress(CGF), ThunkTy->getPointerTo()->getPointerTo(), MD->getParent());
+      getThisAddress(CGF), ThunkTy->getPointerTo()->getPointerTo(),
+      MD->getParent());
 
-  llvm::Value *VFuncPtr =
-      CGF.Builder.CreateConstInBoundsGEP1_64(VTable, ML.Index, "vfn");
-  llvm::Value *Callee =
-    CGF.Builder.CreateAlignedLoad(VFuncPtr, CGF.getPointerAlign());
+  llvm::Value *Callee = CGF.GetVirtualFunctionFromVTable(
+      MD->getParent(), VTable, ML.Index, ThunkTy);
 
   CGF.EmitMustTailThunk(MD, getThisValue(CGF), Callee);
 
   return ThunkFn;
 }
 
 void MicrosoftCXXABI::emitVirtualInheritanceTables(const CXXRecordDecl *RD) {
   const VBTableGlobals &VBGlobals = enumerateVBTables(RD);

test/CodeGenCXX/bitsets.cpp

 // Tests for the cfi-vcall feature:
 // RUN: %clang_cc1 -triple x86_64-unknown-linux -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=ITANIUM --check-prefix=ITANIUM-NDIAG --check-prefix=NDIAG %s
 // RUN: %clang_cc1 -triple x86_64-unknown-linux -fsanitize=cfi-vcall -emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=ITANIUM --check-prefix=ITANIUM-DIAG --check-prefix=DIAG --check-prefix=DIAG-ABORT %s
 // RUN: %clang_cc1 -triple x86_64-unknown-linux -fsanitize=cfi-vcall -fsanitize-recover=cfi-vcall -emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=ITANIUM --check-prefix=DIAG --check-prefix=DIAG-RECOVER %s
 // RUN: %clang_cc1 -triple x86_64-pc-windows-msvc -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=MS --check-prefix=NDIAG %s
 
 // Tests for the whole-program-vtables feature:
-// RUN: %clang_cc1 -triple x86_64-unknown-linux -fwhole-program-vtables -emit-llvm -o - %s | FileCheck --check-prefix=VTABLE-OPT --check-prefix=ITANIUM %s
-// RUN: %clang_cc1 -triple x86_64-pc-windows-msvc -fwhole-program-vtables -emit-llvm -o - %s | FileCheck --check-prefix=VTABLE-OPT --check-prefix=MS %s
+// RUN: %clang_cc1 -triple x86_64-unknown-linux -fwhole-program-vtables -emit-llvm -o - %s | FileCheck --check-prefix=VTABLE-OPT --check-prefix=ITANIUM --check-prefix=ITANIUM-STABLE %s
+// RUN: %clang_cc1 -triple x86_64-pc-windows-msvc -fwhole-program-vtables -emit-llvm -o - %s | FileCheck --check-prefix=VTABLE-OPT --check-prefix=MS --check-prefix=MS-STABLE %s
+// RUN: %clang_cc1 -triple x86_64-unknown-linux -fwhole-program-vtables -funstable-c++-abi-classes -emit-llvm -o - %s | FileCheck --check-prefix=VTABLE-OPT --check-prefix=ITANIUM --check-prefix=ITANIUM-UNSTABLE %s
+// RUN: %clang_cc1 -triple x86_64-pc-windows-msvc -fwhole-program-vtables -funstable-c++-abi-classes -emit-llvm -o - %s | FileCheck --check-prefix=VTABLE-OPT --check-prefix=MS --check-prefix=MS-UNSTABLE %s
 
 // MS: @[[VTA:[0-9]*]] {{.*}} comdat($"\01??_7A@@6B@")
 // MS: @[[VTB:[0-9]*]] {{.*}} comdat($"\01??_7B@@6B0@@")
 // MS: @[[VTAinB:[0-9]*]] {{.*}} comdat($"\01??_7B@@6BA@@@")
 // MS: @[[VTAinC:[0-9]*]] {{.*}} comdat($"\01??_7C@@6B@")
 // MS: @[[VTBinD:[0-9]*]] {{.*}} comdat($"\01??_7D@?A@@6BB@@@")
 // MS: @[[VTAinBinD:[0-9]*]] {{.*}} comdat($"\01??_7D@?A@@6BA@@@")
 // MS: @[[VTFA:[0-9]*]] {{.*}} comdat($"\01??_7FA@?1??foo@@YAXXZ@6B@")

 }
 
 // Check for the expected number of elements (15 or 23 respectively).
 // MS-NDIAG: !llvm.bitsets = !{[[X:[^,]*(,[^,]*){9}]]}
 // MS-DIAG: !llvm.bitsets = !{[[X:[^,]*(,[^,]*){15}]]}
 // ITANIUM-NDIAG: !llvm.bitsets = !{[[X:[^,]*(,[^,]*){14}]]}
 // ITANIUM-DIAG: !llvm.bitsets = !{[[X:[^,]*(,[^,]*){23}]]}
 
-// ITANIUM-DAG: !{!"_ZTS1A", [3 x i8*]* @_ZTV1A, i64 16}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1A", [3 x i8*]* @_ZTV1A, i64 16}
 // ITANIUM-DIAG-DAG: !{!"all-vtables", [3 x i8*]* @_ZTV1A, i64 16}
-// ITANIUM-DAG: !{!"_ZTS1A", [7 x i8*]* @_ZTCN12_GLOBAL__N_11DE0_1B, i64 32}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1A", [7 x i8*]* @_ZTCN12_GLOBAL__N_11DE0_1B, i64 32}
 // ITANIUM-DIAG-DAG: !{!"all-vtables", [7 x i8*]* @_ZTCN12_GLOBAL__N_11DE0_1B, i64 32}
-// ITANIUM-DAG: !{!"_ZTS1B", [7 x i8*]* @_ZTCN12_GLOBAL__N_11DE0_1B, i64 32}
-// ITANIUM-DAG: !{!"_ZTS1A", [9 x i8*]* @_ZTCN12_GLOBAL__N_11DE8_1C, i64 64}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1B", [7 x i8*]* @_ZTCN12_GLOBAL__N_11DE0_1B, i64 32}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1A", [9 x i8*]* @_ZTCN12_GLOBAL__N_11DE8_1C, i64 64}
 // ITANIUM-DIAG-DAG: !{!"all-vtables", [9 x i8*]* @_ZTCN12_GLOBAL__N_11DE8_1C, i64 64}
-// ITANIUM-DAG: !{!"_ZTS1C", [9 x i8*]* @_ZTCN12_GLOBAL__N_11DE8_1C, i64 32}
-// ITANIUM-DAG: !{!"_ZTS1A", [12 x i8*]* @_ZTVN12_GLOBAL__N_11DE, i64 32}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1C", [9 x i8*]* @_ZTCN12_GLOBAL__N_11DE8_1C, i64 32}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1A", [12 x i8*]* @_ZTVN12_GLOBAL__N_11DE, i64 32}
 // ITANIUM-DIAG-DAG: !{!"all-vtables", [12 x i8*]* @_ZTVN12_GLOBAL__N_11DE, i64 32}
-// ITANIUM-DAG: !{!"_ZTS1B", [12 x i8*]* @_ZTVN12_GLOBAL__N_11DE, i64 32}
-// ITANIUM-DAG: !{!"_ZTS1C", [12 x i8*]* @_ZTVN12_GLOBAL__N_11DE, i64 88}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1B", [12 x i8*]* @_ZTVN12_GLOBAL__N_11DE, i64 32}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1C", [12 x i8*]* @_ZTVN12_GLOBAL__N_11DE, i64 88}
 // ITANIUM-DIAG-DAG: !{!"all-vtables", [12 x i8*]* @_ZTVN12_GLOBAL__N_11DE, i64 88}
-// ITANIUM-DAG: !{![[DTYPE]], [12 x i8*]* @_ZTVN12_GLOBAL__N_11DE, i64 32}
-// ITANIUM-DAG: !{!"_ZTS1A", [7 x i8*]* @_ZTV1B, i64 32}
+// ITANIUM-STABLE-DAG: !{![[DTYPE]], [12 x i8*]* @_ZTVN12_GLOBAL__N_11DE, i64 32}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1A", [7 x i8*]* @_ZTV1B, i64 32}
 // ITANIUM-DIAG-DAG: !{!"all-vtables", [7 x i8*]* @_ZTV1B, i64 32}
-// ITANIUM-DAG: !{!"_ZTS1B", [7 x i8*]* @_ZTV1B, i64 32}
-// ITANIUM-DAG: !{!"_ZTS1A", [5 x i8*]* @_ZTV1C, i64 32}
-// ITANIUM-DAG: !{!"_ZTS1C", [5 x i8*]* @_ZTV1C, i64 32}
-// ITANIUM-DAG: !{!"_ZTS1A", [3 x i8*]* @_ZTVZ3foovE2FA, i64 16}
-// ITANIUM-DAG: !{!{{[0-9]+}}, [3 x i8*]* @_ZTVZ3foovE2FA, i64 16}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1B", [7 x i8*]* @_ZTV1B, i64 32}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1A", [5 x i8*]* @_ZTV1C, i64 32}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1C", [5 x i8*]* @_ZTV1C, i64 32}
+// ITANIUM-STABLE-DAG: !{!"_ZTS1A", [3 x i8*]* @_ZTVZ3foovE2FA, i64 16}
+// ITANIUM-STABLE-DAG: !{!{{[0-9]+}}, [3 x i8*]* @_ZTVZ3foovE2FA, i64 16}
+
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1A", { i8*, i8*, i32 }* @_ZTV1A, i64 16}
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1A", { i8*, i8*, i8*, i8*, i32, i32, i32 }* @_ZTV1B, i64 32}
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1B", { i8*, i8*, i8*, i8*, i32, i32, i32 }* @_ZTV1B, i64 32}
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1A", { i8*, i8*, i8*, i8*, i32 }* @_ZTV1C, i64 32}
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1C", { i8*, i8*, i8*, i8*, i32 }* @_ZTV1C, i64 32}
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1A", { i8*, i8*, i8*, i8*, i32, i32, i32 }* @_ZTCN12_GLOBAL__N_11DE0_1B, i64 32}
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1B", { i8*, i8*, i8*, i8*, i32, i32, i32 }* @_ZTCN12_GLOBAL__N_11DE0_1B, i64 32}
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1A", { i8*, i8*, i8*, i8*, i32, i8*, i8*, i8*, i32 }* @_ZTCN12_GLOBAL__N_11DE8_1C, i64 64}
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1C", { i8*, i8*, i8*, i8*, i32, i8*, i8*, i8*, i32 }* @_ZTCN12_GLOBAL__N_11DE8_1C, i64 32}
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1A", { i8*, i8*, i8*, i8*, i32, i32, i32, i8*, i8*, i8*, i8*, i32 }* @_ZTVN12_GLOBAL__N_11DE, i64 32}
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1B", { i8*, i8*, i8*, i8*, i32, i32, i32, i8*, i8*, i8*, i8*, i32 }* @_ZTVN12_GLOBAL__N_11DE, i64 32}
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1C", { i8*, i8*, i8*, i8*, i32, i32, i32, i8*, i8*, i8*, i8*, i32 }* @_ZTVN12_GLOBAL__N_11DE, i64 80}
+// ITANIUM-UNSTABLE-DAG: !{![[DTYPE]], { i8*, i8*, i8*, i8*, i32, i32, i32, i8*, i8*, i8*, i8*, i32 }* @_ZTVN12_GLOBAL__N_11DE, i64 32}
+// ITANIUM-UNSTABLE-DAG: !{!"_ZTS1A", { i8*, i8*, i32 }* @_ZTVZ3foovE2FA, i64 16}
+// ITANIUM-UNSTABLE-DAG: !{!{{[0-9]+}}, { i8*, i8*, i32 }* @_ZTVZ3foovE2FA, i64 16}
 
-// MS-DAG: !{!"?AUA@@", [2 x i8*]* @[[VTA]], i64 8}
+// MS-STABLE-DAG: !{!"?AUA@@", [2 x i8*]* @[[VTA]], i64 8}
 // MS-DIAG-DAG: !{!"all-vtables", [2 x i8*]* @[[VTA]], i64 8}
-// MS-DAG: !{!"?AUB@@", [3 x i8*]* @[[VTB]], i64 8}
+// MS-STABLE-DAG: !{!"?AUB@@", [3 x i8*]* @[[VTB]], i64 8}
 // MS-DIAG-DAG: !{!"all-vtables", [3 x i8*]* @[[VTB]], i64 8}
-// MS-DAG: !{!"?AUA@@", [2 x i8*]* @[[VTAinB]], i64 8}
+// MS-STABLE-DAG: !{!"?AUA@@", [2 x i8*]* @[[VTAinB]], i64 8}
 // MS-DIAG-DAG: !{!"all-vtables", [2 x i8*]* @[[VTAinB]], i64 8}
-// MS-DAG: !{!"?AUA@@", [2 x i8*]* @[[VTAinC]], i64 8}
+// MS-STABLE-DAG: !{!"?AUA@@", [2 x i8*]* @[[VTAinC]], i64 8}
 // MS-DIAG-DAG: !{!"all-vtables", [2 x i8*]* @[[VTAinC]], i64 8}
-// MS-DAG: !{!"?AUB@@", [3 x i8*]* @[[VTBinD]], i64 8}
+// MS-STABLE-DAG: !{!"?AUB@@", [3 x i8*]* @[[VTBinD]], i64 8}
 // MS-DIAG-DAG: !{!"all-vtables", [3 x i8*]* @[[VTBinD]], i64 8}
-// MS-DAG: !{![[DTYPE]], [3 x i8*]* @[[VTBinD]], i64 8}
-// MS-DAG: !{!"?AUA@@", [2 x i8*]* @[[VTAinBinD]], i64 8}
+// MS-STABLE-DAG: !{![[DTYPE]], [3 x i8*]* @[[VTBinD]], i64 8}
+// MS-STABLE-DAG: !{!"?AUA@@", [2 x i8*]* @[[VTAinBinD]], i64 8}
 // MS-DIAG-DAG: !{!"all-vtables", [2 x i8*]* @[[VTAinBinD]], i64 8}
-// MS-DAG: !{!"?AUA@@", [2 x i8*]* @[[VTFA]], i64 8}
+// MS-STABLE-DAG: !{!"?AUA@@", [2 x i8*]* @[[VTFA]], i64 8}
 // MS-DIAG-DAG: !{!"all-vtables", [2 x i8*]* @[[VTFA]], i64 8}
-// MS-DAG: !{!{{[0-9]+}}, [2 x i8*]* @[[VTFA]], i64 8}
+// MS-STABLE-DAG: !{!{{[0-9]+}}, [2 x i8*]* @[[VTFA]], i64 8}
+
+// MS-UNSTABLE-DAG: !{!"?AUA@@", { i8*, i32 }* @[[VTA]], i64 8}
+// MS-UNSTABLE-DAG: !{!"?AUB@@", { i8*, i32, i32 }* @[[VTB]], i64 8}
+// MS-UNSTABLE-DAG: !{!"?AUA@@", { i8*, i32 }* @[[VTAinB]], i64 8}
+// MS-UNSTABLE-DAG: !{!"?AUA@@", { i8*, i32 }* @[[VTAinC]], i64 8}
+// MS-UNSTABLE-DAG: !{!"?AUB@@", { i8*, i32, i32 }* @[[VTBinD]], i64 8}
+// MS-UNSTABLE-DAG: !{![[DTYPE]], { i8*, i32, i32 }* @[[VTBinD]], i64 8}
+// MS-UNSTABLE-DAG: !{!"?AUA@@", { i8*, i32 }* @[[VTAinBinD]], i64 8}
+// MS-UNSTABLE-DAG: !{!"?AUA@@", { i8*, i32 }* @[[VTFA]], i64 8}
+// MS-UNSTABLE-DAG: !{!{{[0-9]+}}, { i8*, i32 }* @[[VTFA]], i64 8}

test/CodeGenCXX/debug-info-virtual-fn-relative.cpp

+// RUN: %clang_cc1 -emit-llvm -debug-info-kind=limited %s -o - | FileCheck --check-prefix=CHECK-STABLE %s
+// RUN: %clang_cc1 -funstable-c++-abi-classes -emit-llvm -debug-info-kind=limited %s -o - | FileCheck --check-prefix=CHECK-UNSTABLE %s
+
+struct S {
+  S();
+  virtual void f();
+};
+
+// CHECK-STABLE: virtualIndex: 0,
+// CHECK-UNSTABLE: virtualIndex: 4294967295,
+S::S() {}
+void S::f() {}

test/CodeGenCXX/vtable-relative-abi.cpp

+// RUN: %clang_cc1 %s -triple x86_64-unknown-linux-gnu -funstable-c++-abi-classes -emit-llvm -o - | FileCheck --check-prefix=CHECK --check-prefix=CHECK-ITANIUM %s
+// RUN: %clang_cc1 %s -triple x86_64-unknown-windows-msvc -funstable-c++-abi-classes -emit-llvm -o - | FileCheck --check-prefix=CHECK --check-prefix=CHECK-MS %s
+
+// CHECK-ITANIUM: @_ZTV1S = unnamed_addr constant { i8*, i8*, i32, i32 } { i8* null, i8* bitcast ({ i8*, i8* }* @_ZTI1S to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.S*)* @_ZN1S2f1Ev to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i8*, i32, i32 }, { i8*, i8*, i32, i32 }* @_ZTV1S, i32 0, i32 2) to i64)) to i32), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.S*)* @_ZN1S2f2Ev to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i8*, i32, i32 }, { i8*, i8*, i32, i32 }* @_ZTV1S, i32 0, i32 2) to i64)) to i32) }, align 8
+// CHECK-MS: @0 = private unnamed_addr constant { i8*, i32, i32 } { i8* bitcast (%rtti.CompleteObjectLocator* @"\01??_R4S@@6B@" to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.S*)* @"\01?f1@S@@UEAAXXZ" to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i32, i32 }, { i8*, i32, i32 }* @0, i32 0, i32 1) to i64)) to i32), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.S*)* @"\01?f2@S@@UEAAXXZ" to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i32, i32 }, { i8*, i32, i32 }* @0, i32 0, i32 1) to i64)) to i32) }, comdat($"\01??_7S@@6B@")
+struct S {
+  S();
+  virtual void f1();
+  virtual void f2();
+};
+
+// CHECK-ITANIUM: @_ZTV1T = unnamed_addr constant { i8*, i8*, i32 } { i8* null, i8* bitcast ({ i8*, i8* }* @_ZTI1T to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.T*)* @_ZN1T1gEv to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i8*, i32 }, { i8*, i8*, i32 }* @_ZTV1T, i32 0, i32 2) to i64)) to i32) }
+// CHECK-MS: @1 = private unnamed_addr constant { i8*, i32 } { i8* bitcast (%rtti.CompleteObjectLocator* @"\01??_R4T@@6B@" to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.T*)* @"\01?g@T@@UEAAXXZ" to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i32 }, { i8*, i32 }* @1, i32 0, i32 1) to i64)) to i32) }, comdat($"\01??_7T@@6B@")
+struct T {
+  T();
+  virtual void g();
+};
+
+// CHECK-ITANIUM: @_ZTV1U = unnamed_addr constant { i8*, i8*, i32, i32, i8*, i8*, i32 } { i8* null, i8* bitcast ({ i8*, i8*, i32, i32, i8*, i64, i8*, i64 }* @_ZTI1U to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.U*)* @_ZN1U2f1Ev to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i8*, i32, i32, i8*, i8*, i32 }, { i8*, i8*, i32, i32, i8*, i8*, i32 }* @_ZTV1U, i32 0, i32 2) to i64)) to i32), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.S*)* @_ZN1S2f2Ev to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i8*, i32, i32, i8*, i8*, i32 }, { i8*, i8*, i32, i32, i8*, i8*, i32 }* @_ZTV1U, i32 0, i32 2) to i64)) to i32), i8* inttoptr (i64 -8 to i8*), i8* bitcast ({ i8*, i8*, i32, i32, i8*, i64, i8*, i64 }* @_ZTI1U to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.T*)* @_ZN1T1gEv to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i8*, i32, i32, i8*, i8*, i32 }, { i8*, i8*, i32, i32, i8*, i8*, i32 }* @_ZTV1U, i32 0, i32 6) to i64)) to i32) }, align 8
+// CHECK-MS: @2 = private unnamed_addr constant { i8*, i32, i32 } { i8* bitcast (%rtti.CompleteObjectLocator* @"\01??_R4U@@6BS@@@" to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.U*)* @"\01?f1@U@@UEAAXXZ" to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i32, i32 }, { i8*, i32, i32 }* @2, i32 0, i32 1) to i64)) to i32), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.S*)* @"\01?f2@S@@UEAAXXZ" to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i32, i32 }, { i8*, i32, i32 }* @2, i32 0, i32 1) to i64)) to i32) }, comdat($"\01??_7U@@6BS@@@")
+// CHECK-MS: @3 = private unnamed_addr constant { i8*, i32 } { i8* bitcast (%rtti.CompleteObjectLocator* @"\01??_R4U@@6BT@@@" to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.T*)* @"\01?g@T@@UEAAXXZ" to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i32 }, { i8*, i32 }* @3, i32 0, i32 1) to i64)) to i32) }, comdat($"\01??_7U@@6BT@@@")
+struct U : S, T {
+  U();
+  virtual void f1();
+};
+
+S::S() {}
+void S::f1() {}
+
+T::T() {}
+void T::g() {}
+
+U::U() {}
+void U::f1() {}
+
+struct V {
+  virtual void f();
+};
+
+struct V1 : virtual V {
+};
+
+struct V2 : virtual V {
+};
+
+// CHECK-ITANIUM: @_ZTC2V30_2V1 = linkonce_odr unnamed_addr constant { i8*, i8*, i8*, i8*, i32 } { i8* null, i8* null, i8* null, i8* bitcast ({ i8*, i8*, i32, i32, i8*, i64 }* @_ZTI2V1 to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.V*)* @_ZN1V1fEv to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i8*, i8*, i8*, i32 }, { i8*, i8*, i8*, i8*, i32 }* @_ZTC2V30_2V1, i32 0, i32 4) to i64)) to i32) }, comdat
+// CHECK-ITANIUM: @_ZTC2V38_2V2 = linkonce_odr unnamed_addr constant { i8*, i8*, i8*, i8*, i32, i8*, i8*, i8*, i32 } { i8* inttoptr (i64 -8 to i8*), i8* inttoptr (i64 -8 to i8*), i8* null, i8* bitcast ({ i8*, i8*, i32, i32, i8*, i64 }* @_ZTI2V2 to i8*), i32 0, i8* null, i8* inttoptr (i64 8 to i8*), i8* bitcast ({ i8*, i8*, i32, i32, i8*, i64 }* @_ZTI2V2 to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.V*)* @_ZN1V1fEv to i64), i64 ptrtoint (i32* getelementptr inbounds ({ i8*, i8*, i8*, i8*, i32, i8*, i8*, i8*, i32 }, { i8*, i8*, i8*, i8*, i32, i8*, i8*, i8*, i32 }* @_ZTC2V38_2V2, i32 0, i32 8) to i64)) to i32) }, comdat
+struct V3 : V1, V2 {
+  V3();
+};
+
+V3::V3() {}
+
+// CHECK-ITANIUM: define void @_Z5call1P1S
+// CHECK-MS: define void @"\01?call1@@YAXPEAUS@@@Z"
+void call1(S *s) {
+  // CHECK: [[VTABLE:%.*]] = load void
+  // CHECK: [[VTABLEI8:%.*]] = bitcast {{.*}} [[VTABLE]] to i8*
+  // CHECK: [[VFN:%.*]] = call i8* @llvm.load.relative.i32(i8* [[VTABLEI8]], i32 0)
+  // CHECK: [[VFNCASTED:%.*]] = bitcast i8* [[VFN]] to void (
+  // CHECK: call void [[VFNCASTED]](
+  s->f1();
+}
+
+// CHECK-ITANIUM: define void @_Z5call2P1S
+// CHECK-MS: define void @"\01?call2@@YAXPEAUS@@@Z"
+void call2(S *s) {
+  // CHECK: [[VTABLE:%.*]] = load void
+  // CHECK: [[VTABLEI8:%.*]] = bitcast {{.*}} [[VTABLE]] to i8*
+  // CHECK: [[VFN:%.*]] = call i8* @llvm.load.relative.i32(i8* [[VTABLEI8]], i32 4)
+  // CHECK: [[VFNCASTED:%.*]] = bitcast i8* [[VFN]] to void (
+  // CHECK: call void [[VFNCASTED]](
+  s->f2();
+}
+
+typedef void (S::*mfp)();
+
+// CHECK-ITANIUM: define { i64, i64 } @_Z7getmfp1v()
+// CHECK-ITANIUM: ret { i64, i64 } { i64 1, i64 0 }
+// CHECK-MS: define i8* @"\01?getmfp1@@YAP8S@@EAAXXZXZ"()
+// CHECK-MS: ret i8* bitcast {{.*}} @"\01??_9S@@$BA@AA"
+// CHECK-MS: define linkonce_odr void @"\01??_9S@@$BA@AA"
+// CHECK-MS: [[VTABLE:%.*]] = load void
+// CHECK-MS: [[VTABLEI8:%.*]] = bitcast {{.*}} [[VTABLE]] to i8*
+// CHECK-MS: [[VFN:%.*]] = call i8* @llvm.load.relative.i32(i8* [[VTABLEI8]], i32 0)
+// CHECK-MS: [[VFNCASTED:%.*]] = bitcast i8* [[VFN]] to void (
+// CHECK-MS: musttail call {{.*}} [[VFNCASTED]](
+mfp getmfp1() {
+  return &S::f1;
+}
+
+// CHECK-ITANIUM: define { i64, i64 } @_Z7getmfp2v()
+// CHECK-ITANIUM: ret { i64, i64 } { i64 5, i64 0 }
+// CHECK-MS: define i8* @"\01?getmfp2@@YAP8S@@EAAXXZXZ"()
+// CHECK-MS: ret i8* bitcast {{.*}} @"\01??_9S@@$B7AA"
+// CHECK-MS: define linkonce_odr void @"\01??_9S@@$B7AA"
+// CHECK-MS: [[VTABLE:%.*]] = load void
+// CHECK-MS: [[VTABLEI8:%.*]] = bitcast {{.*}} [[VTABLE]] to i8*
+// CHECK-MS: [[VFN:%.*]] = call i8* @llvm.load.relative.i32(i8* [[VTABLEI8]], i32 4)
+// CHECK-MS: [[VFNCASTED:%.*]] = bitcast i8* [[VFN]] to void (
+// CHECK-MS: musttail call {{.*}} [[VFNCASTED]](
+mfp getmfp2() {
+  return &S::f2;
+}
+
+// In the MS ABI virtual member function calls use thunks (which we already
+// tested above), so there's nothing to test that's specific to the relative
+// ABI.
+
+// CHECK-ITANIUM: define void @_Z7callmfpP1SMS_FvvE
+void callmfp(S *s, mfp p) {
+  // CHECK-ITANIUM: [[VTOFFSET:%.*]] = sub i64 {{.*}}, 1
+  // CHECK-ITANIUM: [[VFN:%.*]] = call i8* @llvm.load.relative.i64(i8* {{.*}}, i64 [[VTOFFSET]])
+  // CHECK-ITANIUM: bitcast i8* [[VFN]] to void (
+  (s->*p)();
+}