Support for relative vtables

[leonardchan]

Summary

This is a revisit to D20749 which introduces a flag that enables the relative ABI described in https://bugs.llvm.org/show_bug.cgi?id=26723.

This is still a work in progress and am still in the process of debugging some issues I ran into when building Fuchsia with this, but wanted to raise some awareness and see if anyone has any thoughts or comments.

[pcc]

Can we start with a patch that just exposes a flag that enables the relative ABI unconditionally, and remove all the platform ABI compatibility stuff?

[leonardchan]

In D58321#1407362, @pcc wrote:

Can we start with a patch that just exposes a flag that enables the relative ABI unconditionally, and remove all the platform ABI compatibility stuff?

Done. Removed the checks requiring LTO, checks in checkClassABI, and relevant tests for them.

[rjmccall]

clang/include/clang/Driver/Options.td

1265

Please make this a -cc1 option or otherwise mark it clearly experimental.

[leonardchan]

Update to current working version without needing LTO

[leonardchan]

• Remove hasHiddenLTOVisibility from CXXRecordDecl and rename the flag to -frelative-c++-abi-vtables

[riccibruno]

I think that you need to update the serialization code in Serialization/ASTReaderDecl.cpp and Serialization/ASTWriterDecl.cpp to account for the new flag.

[leonardchan]

Uploading my latest version of this patch and reporting some results. Will proceed with formatting, cleanup, and addressing the existing comments and ask for official reviews later.

---

After toying around with this for a bit, just enabling this flag on all c++ code in zircon shows about a 1% size increase in the final image (x64.zbi), but does its job of moving many vtables from .data.rel.ro to rodata. For each shared library that's included in the image, some of them decrease while most increase in size. These changes though are pretty small and rounded up align to page sizes (4kB), resulting in the overall 1% increase. This is mostly from extra instructions and extra PLT entries, through we have an idea for how we could reduce the PLT size.

We still get the benefits of moving vtables into rodata and having them shared between processes. Below is the section dump for one of our shared libraries (omitting the debug sections):

Without relative ABI

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .note.gnu.build-id NOTE            0000000000000270 000270 000018 00   A  0   0  4
  [ 2] .dynsym           DYNSYM          0000000000000288 000288 000540 18   A  5   1  8
  [ 3] .gnu.hash         GNU_HASH        00000000000007c8 0007c8 000050 00   A  2   0  8
  [ 4] .dynamic          DYNAMIC         0000000000000818 000818 000170 10   A  5   0  8
  [ 5] .dynstr           STRTAB          0000000000000988 000988 000424 00   A  0   0  1
  [ 6] .rela.dyn         RELA            0000000000000db0 000db0 0000a8 18   A  2   0  8  # missing
  [ 7] .relr.dyn         00000013: <unknown> 0000000000000e58 000e58 000080 08   A  0   0  8
  [ 8] .rela.plt         RELA            0000000000000ed8 000ed8 000468 18   A  2  15  8
  [ 9] .rodata           PROGBITS        0000000000001340 001340 0025c8 00 AMS  0   0 16
  [10] .eh_frame_hdr     PROGBITS        0000000000003908 003908 000a1c 00   A  0   0  4
  [11] .eh_frame         PROGBITS        0000000000004328 004328 0032cc 00   A  0   0  8
  [12] .text             PROGBITS        0000000000008000 008000 014c1b 00  AX  0   0 16
  [13] .plt              PROGBITS        000000000001cc20 01cc20 000300 00  AX  0   0 16
  [14] .data.rel.ro      PROGBITS        000000000001d000 01d000 001948 00  WA  0   0 16
  [15] .got.plt          PROGBITS        000000000001e948 01e948 000190 00  WA  0   0  8

With relative ABI

Section Headers:
  [Nr] Name              Type            Address          Off    Size   ES Flg Lk Inf Al
  [ 0]                   NULL            0000000000000000 000000 000000 00      0   0  0
  [ 1] .note.gnu.build-id NOTE            0000000000000270 000270 000018 00   A  0   0  4
  [ 2] .dynsym           DYNSYM          0000000000000288 000288 000540 18   A  5   1  8
  [ 3] .gnu.hash         GNU_HASH        00000000000007c8 0007c8 000050 00   A  2   0  8
  [ 4] .dynamic          DYNAMIC         0000000000000818 000818 000140 10   A  5   0  8
  [ 5] .dynstr           STRTAB          0000000000000958 000958 000424 00   A  0   0  1
  [ 6] .relr.dyn         00000013: <unknown> 0000000000000d80 000d80 000068 08   A  0   0  8
  [ 7] .rela.plt         RELA            0000000000000de8 000de8 000480 18   A  2  14  8
  [ 8] .rodata           PROGBITS        0000000000001270 001270 002990 00 AMS  0   0 16
  [ 9] .eh_frame_hdr     PROGBITS        0000000000003c00 003c00 000a1c 00   A  0   0  4
  [10] .eh_frame         PROGBITS        0000000000004620 004620 0032cc 00   A  0   0  8
  [11] .text             PROGBITS        0000000000008000 008000 014ffb 00  AX  0   0 16
  [12] .plt              PROGBITS        000000000001d000 01d000 000310 00  AX  0   0 16
  [13] .data.rel.ro      PROGBITS        000000000001e000 01e000 0012d0 00  WA  0   0 16
  [14] .got.plt          PROGBITS        000000000001f2d0 01f2d0 000198 00  WA  0   0  8
  [15] .data             PROGBITS        0000000000020000 020000 000038 00  WA  0   0 16
  [16] .bss              NOBITS          0000000000020038 020038 000100 00  WA  0   0  8

Sections that grow are .rela.plt, .rodata, .text, .plt, and .got.plt whereas sections that shrink are .rela.dyn and .data.rel.ro.

We were also able to build libc++ with relative ABI and got ~2% decrease in size for the final shared library.

Other stuff in this patch:

• Remove any LTO related code
• Ensure needsRelocation() catches the relative offset function pointers

[leonardchan]

Ok. Formally requesting for code reviews now.

• Made the flag an experimental flag.
• Added changes to relevant AST Serialization code.

[leonardchan]

*ping*

[rjmccall]

clang/include/clang/AST/DeclCXX.h

531 ↗

(On Diff #197196)

Should we proactively generalize this as a "CXXABIVariant" enum, which for now can just be "Standard" and "RelativeVTables"?

Also, I don't want to pre-empt your secret plans, but if Fuchsia is just going to use this as its system C++ ABI, maybe we should plan for that, too.

clang/include/clang/AST/VTableBuilder.h

267

const VTableComponent &, I think.

clang/include/clang/Basic/LangOptions.def

330

Yeah, see, this plays into the question above. I would not want to provide this as a language option for general use. The attribute seems good enough for testing, and if you want a -cc1 option to apply the attribute by default for experimentation during Fuchsia bring-up that's fair, but I don't want something that suggests to users that it's okay to pass this attribute and change the system default.

clang/include/clang/Sema/Sema.h

10976 ↗

(On Diff #197196)

Comment / method-name mismatch?

[pcc]

clang/include/clang/AST/DeclCXX.h

531 ↗

(On Diff #197196)

At this point I probably would remove this bitfield entirely. This implementation does not support enabling the ABI on a per-class basis, so everywhere that we are currently checking this field we should just be able to check RelativeCXXABIVTables in LangOptions.

[leonardchan]

clang/include/clang/AST/DeclCXX.h

531 ↗

(On Diff #197196)

The goal for us is to just enable this for all vtables so we don't need this on a per-class basis. Removed the bitfield.

clang/include/clang/Basic/LangOptions.def

330

Ok. So is this the reason for the white list approach mentioned in D17893? As an alternative, would you also be ok with creating a FuchsiaCXXABI that subclasses ItaniumCXXABI, similar to the ARM and WebAssembly ones? This way it doesn't change the system default.

clang/include/clang/Sema/Sema.h

10976 ↗

(On Diff #197196)

Removed this method since the bitfield we set here is removed.

[rjmccall]

clang/include/clang/Basic/LangOptions.def

330

That design for relative v-tables was intended (at least as it was described to me) as an optimization for certain classes in a project that still needed to interoperate with system and other non-project C++ code. It was initially attempting to apply the optimization eagerly to all classes that didn't match a blacklist of classes (and namespaces of classes) that needed to follow the system ABI, which I objected to, which is why I asked for it to be redesigned around applying the optimization only to whitelisted classes/namespaces. None of that applies if you're changing the system ABI rule unless you decide you want to allow classes to opt into standard Itanium v-tables.

If introducing a FuchsiaCXXABI class makes the code easier, go for it.

clang/include/clang/AST/VTableBuilder.h

VTableLayout(ArrayRef<size_t> VTableIndices,
                ArrayRef<VTableThunkTy> VTableThunks,
                const AddressPointsMapTy &AddressPoints);
   ~VTableLayout();
 
   ArrayRef<VTableComponent> vtable_components() const {
     return VTableComponents;
   }
 
+  const VTableComponent &getVTableComponent(size_t i) const {
+    return VTableComponents[i];
+  }
+
   ArrayRef<VTableThunkTy> vtable_thunks() const {
     return VTableThunks;
   }
 
   AddressPointLocation getAddressPoint(BaseSubobject Base) const {
     assert(AddressPoints.count(Base) && "Did not find address point!");
     return AddressPoints.find(Base)->second;
   }

clang/include/clang/Basic/LangOptions.def

 COMPATIBLE_VALUE_LANGOPT(FunctionAlignment, 5, 0, "Default alignment for functions")
 
 LANGOPT(FixedPoint, 1, 0, "fixed point types")
 LANGOPT(PaddingOnUnsignedFixedPoint, 1, 0,
         "unsigned fixed point types having one extra padding bit")
 
 LANGOPT(RegisterStaticDestructors, 1, 1, "Register C++ static destructors")
 
+LANGOPT(RelativeCXXABIVTables, 1, 0,
+        "Whether to use clang's relative C++ ABI "
+        "for classes with vtables")
+
 #undef LANGOPT
 #undef COMPATIBLE_LANGOPT
 #undef BENIGN_LANGOPT
 #undef ENUM_LANGOPT
 #undef COMPATIBLE_ENUM_LANGOPT
 #undef BENIGN_ENUM_LANGOPT
 #undef VALUE_LANGOPT
 #undef COMPATIBLE_VALUE_LANGOPT
 #undef BENIGN_VALUE_LANGOPT

clang/include/clang/Driver/Options.td

 
 def ffine_grained_bitfield_accesses : Flag<["-"],
   "ffine-grained-bitfield-accesses">, Group<f_clang_Group>, Flags<[CC1Option]>,
   HelpText<"Use separate accesses for consecutive bitfield runs with legal widths and alignments.">;
 def fno_fine_grained_bitfield_accesses : Flag<["-"],
   "fno-fine-grained-bitfield-accesses">, Group<f_clang_Group>, Flags<[CC1Option]>,
   HelpText<"Use large-integer access for consecutive bitfield runs.">;
 
+def fexperimental_relative_cxx_abi_vtables : Flag<["-"], "fexperimental-relative-c++-abi-vtables">,
+  Group<f_Group>, Flags<[CC1Option]>,
+  HelpText<"Use the experimental C++ class ABI for classes with virtual tables">;
+
 def flat__namespace : Flag<["-"], "flat_namespace">;
 def flax_vector_conversions : Flag<["-"], "flax-vector-conversions">, Group<f_Group>;
 def flimited_precision_EQ : Joined<["-"], "flimited-precision=">, Group<f_Group>;
 def fapple_link_rtlib : Flag<["-"], "fapple-link-rtlib">, Group<f_Group>,
   HelpText<"Force linking the clang builtins runtime library">;
 def flto_EQ : Joined<["-"], "flto=">, Flags<[CoreOption, CC1Option]>, Group<f_Group>,
   HelpText<"Set LTO mode to either 'full' or 'thin'">, Values<"thin,full">;
 def flto : Flag<["-"], "flto">, Flags<[CoreOption, CC1Option]>, Group<f_Group>,
   HelpText<"Enable LTO in 'full' mode">;
 def fno_lto : Flag<["-"], "fno-lto">, Group<f_Group>,
   HelpText<"Disable LTO mode (default)">;
 def flto_jobs_EQ : Joined<["-"], "flto-jobs=">,
   Flags<[CC1Option]>, Group<f_Group>,
   HelpText<"Controls the backend parallelism of -flto=thin (default "
            "of 0 means the number of threads will be derived from "
            "the number of CPUs detected)">;
 def fthinlto_index_EQ : Joined<["-"], "fthinlto-index=">,
   Flags<[CC1Option]>, Group<f_Group>,
   HelpText<"Perform ThinLTO importing using provided function summary index">;
 def fmacro_backtrace_limit_EQ : Joined<["-"], "fmacro-backtrace-limit=">,
                                 Group<f_Group>, Flags<[DriverOption, CoreOption]>;
 def fmerge_all_constants : Flag<["-"], "fmerge-all-constants">, Group<f_Group>,

clang/lib/AST/DeclCXX.cpp

 #include "clang/Basic/Diagnostic.h"
 #include "clang/Basic/IdentifierTable.h"
 #include "clang/Basic/LLVM.h"
 #include "clang/Basic/LangOptions.h"
 #include "clang/Basic/OperatorKinds.h"
 #include "clang/Basic/PartialDiagnostic.h"
 #include "clang/Basic/SourceLocation.h"
 #include "clang/Basic/Specifiers.h"
+#include "clang/Basic/TargetInfo.h"
 #include "llvm/ADT/None.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/SmallVector.h"
 #include "llvm/ADT/iterator_range.h"
 #include "llvm/Support/Casting.h"
 #include "llvm/Support/ErrorHandling.h"
 #include "llvm/Support/raw_ostream.h"
 #include <algorithm>

clang/lib/CodeGen/CGClass.cpp

llvm::Value *CodeGenFunction::GetVTablePtr(Address This,
 
   if (CGM.getCodeGenOpts().OptimizationLevel > 0 &&
       CGM.getCodeGenOpts().StrictVTablePointers)
     CGM.DecorateInstructionWithInvariantGroup(VTable, RD);
 
   return VTable;
 }
 
+llvm::Value *CodeGenFunction::GetVirtualFunctionFromVTable(
+    const CXXRecordDecl *RD, llvm::Value *VTable, uint64_t VTableIndex,
+    llvm::Type *Ty) {
+  if (!getLangOpts().RelativeCXXABIVTables) {
+    VTable = Builder.CreateBitCast(VTable, Ty->getPointerTo()->getPointerTo());
+    llvm::Value *VTableSlotPtr =
+        Builder.CreateConstInBoundsGEP1_64(VTable, VTableIndex, "vfn");
+    return Builder.CreateAlignedLoad(VTableSlotPtr, getPointerAlign());
+  }
+
+  VTable = Builder.CreateBitCast(VTable, Int8PtrTy);
+  llvm::Value *Load = Builder.CreateCall(
+      CGM.getIntrinsic(llvm::Intrinsic::load_relative, {Int32Ty}),
+      {VTable, llvm::ConstantInt::get(Int32Ty, 4 * VTableIndex)});
+  return Builder.CreateBitCast(Load, Ty->getPointerTo());
+}
+
 // If a class has a single non-virtual base and does not introduce or override
 // virtual member functions or fields, it will have the same layout as its base.
 // This function returns the least derived such class.
 //
 // Casting an instance of a base class to such a derived class is technically
 // undefined behavior, but it is a relatively common hack for introducing member
 // functions on class instances with specific properties (e.g. llvm::Operator)
 // that works under most compilers and should not have security implications, so

clang/lib/CodeGen/CGDebugInfo.cpp

if (Method->isVirtual()) {
     if (Method->isPure())
       SPFlags |= llvm::DISubprogram::SPFlagPureVirtual;
     else
       SPFlags |= llvm::DISubprogram::SPFlagVirtual;
 
     if (CGM.getTarget().getCXXABI().isItaniumFamily()) {
       // It doesn't make sense to give a virtual destructor a vtable index,
       // since a single destructor has two entries in the vtable.
-      if (!isa<CXXDestructorDecl>(Method))
+      if (!isa<CXXDestructorDecl>(Method) &&
+          !CGM.getLangOpts().RelativeCXXABIVTables)
         VIndex = CGM.getItaniumVTableContext().getMethodVTableIndex(Method);
+      else
+        VIndex = -1u;
     } else {
       // Emit MS ABI vftable information.  There is only one entry for the
       // deleting dtor.
       const auto *DD = dyn_cast<CXXDestructorDecl>(Method);
       GlobalDecl GD = DD ? GlobalDecl(DD, Dtor_Deleting) : GlobalDecl(Method);
       MethodVFTableLocation ML =
           CGM.getMicrosoftVTableContext().getMethodVFTableLocation(GD);
       VIndex = ML.Index;

clang/lib/CodeGen/CGVTables.h

class CodeGenVTables {
   /// Cache for the deleted virtual member call function.
   llvm::Constant *DeletedVirtualFn = nullptr;
 
   /// Get the address of a thunk and emit it if necessary.
   llvm::Constant *maybeEmitThunk(GlobalDecl GD,
                                  const ThunkInfo &ThunkAdjustments,
                                  bool ForVTable);
 
-  void addVTableComponent(ConstantArrayBuilder &builder,
-                          const VTableLayout &layout, unsigned idx,
+  /// Get the constant to be added to a VTable.
+  llvm::Constant *getVTableComponent(const VTableLayout &layout,
+                                     unsigned componentIdx, unsigned vtableIdx,
                                      llvm::Constant *rtti,
                                      unsigned &nextVTableThunkIndex);
 
 public:
   /// Add vtable components for the given vtable layout to the given
   /// global initializer.
-  void createVTableInitializer(ConstantStructBuilder &builder,
-                               const VTableLayout &layout,
-                               llvm::Constant *rtti);
+  void SetVTableInitializer(llvm::GlobalVariable *VTable,
+                            const VTableLayout &VTLayout, llvm::Constant *RTTI);
+
+  /// Make a function pointer into a relative integer when using the relative
+  /// vtables ABI.
+  llvm::Constant *makeRelative(llvm::Constant *C, llvm::GlobalVariable *VTable,
+                               unsigned vtableIdx,
+                               unsigned lastAddrPoint) const;
+
+  /// Populate the fields of a single vtable element. This accepts either a
+  /// StructBuilder or ArrayBuilder.
+  template <class BuilderTy>
+  void buildVTableElement(BuilderTy &builder, llvm::GlobalVariable *VTable,
+                          const VTableLayout &VTLayout, llvm::Constant *RTTI,
+                          unsigned &nextVTableThunkIndex, unsigned vtableIdx);
 
   CodeGenVTables(CodeGenModule &CGM);
 
   ItaniumVTableContext &getItaniumVTableContext() {
     return *cast<ItaniumVTableContext>(VTContext);
   }
 
   MicrosoftVTableContext &getMicrosoftVTableContext() {
public:
   /// (if the class has virtual bases).
   void GenerateClassData(const CXXRecordDecl *RD);
 
   bool isVTableExternal(const CXXRecordDecl *RD);
 
   /// Returns the type of a vtable with the given layout. Normally a struct of
   /// arrays of pointers, with one struct element for each vtable in the vtable
   /// group.
-  llvm::Type *getVTableType(const VTableLayout &layout);
+  llvm::Type *getVTableType(const CXXRecordDecl *RD,
+                            const VTableLayout &layout);
 };
 
 } // end namespace CodeGen
 } // end namespace clang
 #endif

clang/lib/CodeGen/CGVTables.cpp

void CodeGenVTables::EmitThunks(GlobalDecl GD) {
 
   if (!ThunkInfoVector)
     return;
 
   for (const ThunkInfo& Thunk : *ThunkInfoVector)
     maybeEmitThunk(GD, Thunk, /*ForVTable=*/false);
 }
 
-void CodeGenVTables::addVTableComponent(
-    ConstantArrayBuilder &builder, const VTableLayout &layout,
-    unsigned idx, llvm::Constant *rtti, unsigned &nextVTableThunkIndex) {
-  auto &component = layout.vtable_components()[idx];
+llvm::Constant *CodeGenVTables::makeRelative(llvm::Constant *C,
+                                             llvm::GlobalVariable *VTable,
+                                             unsigned vtableIdx,
+                                             unsigned lastAddrPoint) const {
+  llvm::Type *Int32Ty = CGM.Int32Ty;
+  llvm::Type *PtrDiffTy =
+      CGM.getTypes().ConvertType(CGM.getContext().getPointerDiffType());
+  llvm::Type *VTableTy = VTable->getValueType();
+  llvm::Constant *gep = llvm::ConstantExpr::getGetElementPtr(
+      VTableTy, VTable,
+      llvm::ArrayRef<llvm::Constant *>{
+          llvm::ConstantInt::get(Int32Ty, 0),
+          llvm::ConstantInt::get(Int32Ty, vtableIdx),
+          llvm::ConstantInt::get(Int32Ty, lastAddrPoint)});
+
+  llvm::Constant *AddrPointInt =
+      llvm::ConstantExpr::getPtrToInt(gep, PtrDiffTy);
+  llvm::Constant *ptrToInt = llvm::ConstantExpr::getPtrToInt(C, PtrDiffTy);
+
+  return llvm::ConstantExpr::getIntegerCast(
+      llvm::ConstantExpr::getSub(ptrToInt, AddrPointInt), Int32Ty,
+      /*isSigned=*/true);
+}
+
+llvm::Constant *CodeGenVTables::getVTableComponent(
+    const VTableLayout &layout, unsigned componentIdx, unsigned vtableIdx,
+    llvm::Constant *rtti, unsigned &nextVTableThunkIndex) {
+  auto &component = layout.getVTableComponent(componentIdx);
+
+  llvm::Type *Int8PtrTy = CGM.Int8PtrTy;
+  llvm::Type *Int32Ty = CGM.Int32Ty;
+  llvm::Type *FunctionPtrTy =
+      CGM.getLangOpts().RelativeCXXABIVTables ? Int32Ty : Int8PtrTy;
 
   auto addOffsetConstant = [&](CharUnits offset) {
-    builder.add(llvm::ConstantExpr::getIntToPtr(
+    return llvm::ConstantExpr::getIntToPtr(
         llvm::ConstantInt::get(CGM.PtrDiffTy, offset.getQuantity()),
-        CGM.Int8PtrTy));
+        CGM.Int8PtrTy);
   };
 
   switch (component.getKind()) {
   case VTableComponent::CK_VCallOffset:
     return addOffsetConstant(component.getVCallOffset());
 
   case VTableComponent::CK_VBaseOffset:
     return addOffsetConstant(component.getVBaseOffset());
 
   case VTableComponent::CK_OffsetToTop:
     return addOffsetConstant(component.getOffsetToTop());
 
   case VTableComponent::CK_RTTI:
-    return builder.add(llvm::ConstantExpr::getBitCast(rtti, CGM.Int8PtrTy));
+    return llvm::ConstantExpr::getBitCast(rtti, CGM.Int8PtrTy);
 
   case VTableComponent::CK_FunctionPointer:
   case VTableComponent::CK_CompleteDtorPointer:
   case VTableComponent::CK_DeletingDtorPointer: {
     GlobalDecl GD;
 
     // Get the right global decl.
     switch (component.getKind()) {
if (CGM.getLangOpts().CUDA) {
       const CXXMethodDecl *MD = cast<CXXMethodDecl>(GD.getDecl());
       // OK on device side: functions w/ __device__ attribute
       // OK on host side: anything except __device__-only functions.
       bool CanEmitMethod =
           CGM.getLangOpts().CUDAIsDevice
               ? MD->hasAttr<CUDADeviceAttr>()
               : (MD->hasAttr<CUDAHostAttr>() || !MD->hasAttr<CUDADeviceAttr>());
       if (!CanEmitMethod)
-        return builder.addNullPointer(CGM.Int8PtrTy);
+        return llvm::ConstantExpr::getNullValue(FunctionPtrTy);
       // Method is acceptable, continue processing as usual.
     }
 
     auto getSpecialVirtualFn = [&](StringRef name) {
       llvm::FunctionType *fnTy =
           llvm::FunctionType::get(CGM.VoidTy, /*isVarArg=*/false);
       llvm::Constant *fn = cast<llvm::Constant>(
           CGM.CreateRuntimeFunction(fnTy, name).getCallee());
case VTableComponent::CK_DeletingDtorPointer: {
     } else if (cast<CXXMethodDecl>(GD.getDecl())->isDeleted()) {
       if (!DeletedVirtualFn)
         DeletedVirtualFn =
           getSpecialVirtualFn(CGM.getCXXABI().GetDeletedVirtualCallName());
       fnPtr = DeletedVirtualFn;
 
     // Thunks.
     } else if (nextVTableThunkIndex < layout.vtable_thunks().size() &&
-               layout.vtable_thunks()[nextVTableThunkIndex].first == idx) {
+               layout.vtable_thunks()[nextVTableThunkIndex].first ==
+                   componentIdx) {
       auto &thunkInfo = layout.vtable_thunks()[nextVTableThunkIndex].second;
 
       nextVTableThunkIndex++;
       fnPtr = maybeEmitThunk(GD, thunkInfo, /*ForVTable=*/true);
 
     // Otherwise we can use the method definition directly.
     } else {
       llvm::Type *fnTy = CGM.getTypes().GetFunctionTypeForVTable(GD);
       fnPtr = CGM.GetAddrOfFunction(GD, fnTy, /*ForVTable=*/true);
     }
 
-    fnPtr = llvm::ConstantExpr::getBitCast(fnPtr, CGM.Int8PtrTy);
-    builder.add(fnPtr);
-    return;
+    return llvm::ConstantExpr::getBitCast(fnPtr, CGM.Int8PtrTy);
   }
 
   case VTableComponent::CK_UnusedFunctionPointer:
-    return builder.addNullPointer(CGM.Int8PtrTy);
+    return llvm::ConstantExpr::getNullValue(FunctionPtrTy);
   }
 
   llvm_unreachable("Unexpected vtable component kind");
 }
 
-llvm::Type *CodeGenVTables::getVTableType(const VTableLayout &layout) {
-  SmallVector<llvm::Type *, 4> tys;
+llvm::Type *CodeGenVTables::getVTableType(const CXXRecordDecl *RD,
+                                          const VTableLayout &layout) {
+  SmallVector<llvm::Type *, 4> types;
   for (unsigned i = 0, e = layout.getNumVTables(); i != e; ++i) {
-    tys.push_back(llvm::ArrayType::get(CGM.Int8PtrTy, layout.getVTableSize(i)));
-  }
+    if (!CGM.getLangOpts().RelativeCXXABIVTables) {
+      types.push_back(
+          llvm::ArrayType::get(CGM.Int8PtrTy, layout.getVTableSize(i)));
+    } else {
+      SmallVector<llvm::Type *, 4> innerTypes;
+      size_t thisIndex = layout.getVTableOffset(i);
+      size_t nextIndex = thisIndex + layout.getVTableSize(i);
+      for (unsigned componentIdx = thisIndex; componentIdx != nextIndex;
+           ++componentIdx) {
+        auto &component = layout.getVTableComponent(componentIdx);
+        if (component.isFunctionPointerKind())
+          innerTypes.push_back(CGM.Int32Ty);
+        else
+          innerTypes.push_back(CGM.Int8PtrTy);
+      }
+      types.push_back(llvm::StructType::get(CGM.getLLVMContext(), innerTypes));
+    }
+  }
+
+  return llvm::StructType::get(CGM.getLLVMContext(), types);
 
-  return llvm::StructType::get(CGM.getLLVMContext(), tys);
+template <class BuilderTy>
+void CodeGenVTables::buildVTableElement(BuilderTy &builder,
+                                        llvm::GlobalVariable *VTable,
+                                        const VTableLayout &VTLayout,
+                                        llvm::Constant *RTTI,
+                                        unsigned &nextVTableThunkIndex,
+                                        unsigned vtableIdx) {
+  // We want to point to the first function in the vtable. This would
+  // normally be the 3rd component under Itanium ABI (2nd for Microsoft),
+  // but not in the case of virtual inheritance. For both of these ABIs, the
+  // first virtual function comes after the RTTI component, so we can just
+  // check if the previous component was RTTI.
+  //
+  // FIXME: Need a better way of identifying address points that works with
+  // the Itanium and MS ABIs.
+  unsigned lastAddrPoint = 0;
+
+  size_t thisIndex = VTLayout.getVTableOffset(vtableIdx);
+  size_t nextIndex = thisIndex + VTLayout.getVTableSize(vtableIdx);
+  for (unsigned componentIdx = thisIndex; componentIdx != nextIndex;
+       ++componentIdx) {
+    if (componentIdx > 0 &&
+        VTLayout.getVTableComponent(componentIdx - 1).isRTTIKind())
+      lastAddrPoint = componentIdx;
+
+    llvm::Constant *component = getVTableComponent(
+        VTLayout, componentIdx, vtableIdx, RTTI, nextVTableThunkIndex);
+
+    auto &layoutComponent = VTLayout.getVTableComponent(componentIdx);
+    if (CGM.getLangOpts().RelativeCXXABIVTables &&
+        layoutComponent.isUsedFunctionPointerKind())
+      component =
+          makeRelative(component, VTable, vtableIdx, lastAddrPoint - thisIndex);
+
+    builder.add(component);
+  }
 }
 
-void CodeGenVTables::createVTableInitializer(ConstantStructBuilder &builder,
-                                             const VTableLayout &layout,
-                                             llvm::Constant *rtti) {
+void CodeGenVTables::SetVTableInitializer(llvm::GlobalVariable *VTable,
+                                          const VTableLayout &VTLayout,
+                                          llvm::Constant *RTTI) {
+  ConstantInitBuilder builder(CGM);
+  auto components = builder.beginStruct();
+
   unsigned nextVTableThunkIndex = 0;
-  for (unsigned i = 0, e = layout.getNumVTables(); i != e; ++i) {
-    auto vtableElem = builder.beginArray(CGM.Int8PtrTy);
-    size_t thisIndex = layout.getVTableOffset(i);
-    size_t nextIndex = thisIndex + layout.getVTableSize(i);
-    for (unsigned i = thisIndex; i != nextIndex; ++i) {
-      addVTableComponent(vtableElem, layout, i, rtti, nextVTableThunkIndex);
+  for (unsigned vtableIdx = 0; vtableIdx != VTLayout.getNumVTables();
+       ++vtableIdx) {
+    if (!CGM.getLangOpts().RelativeCXXABIVTables) {
+      auto vtableElem = components.beginArray(CGM.Int8PtrTy);
+      buildVTableElement(vtableElem, VTable, VTLayout, RTTI,
+                         nextVTableThunkIndex, vtableIdx);
+      vtableElem.finishAndAddTo(components);
+    } else {
+      auto vtableElem = components.beginStruct();
+      buildVTableElement(vtableElem, VTable, VTLayout, RTTI,
+                         nextVTableThunkIndex, vtableIdx);
+      vtableElem.finishAndAddTo(components);
     }
-    vtableElem.finishAndAddTo(builder);
   }
+
+  components.finishAndSetAsInitializer(VTable);
 }
 
 llvm::GlobalVariable *
 CodeGenVTables::GenerateConstructionVTable(const CXXRecordDecl *RD,
                                       const BaseSubobject &Base,
                                       bool BaseIsVirtual,
                                    llvm::GlobalVariable::LinkageTypes Linkage,
                                       VTableAddressPointsMapTy& AddressPoints) {
CodeGenVTables::GenerateConstructionVTable(const CXXRecordDecl *RD,
   // Get the mangled construction vtable name.
   SmallString<256> OutName;
   llvm::raw_svector_ostream Out(OutName);
   cast<ItaniumMangleContext>(CGM.getCXXABI().getMangleContext())
       .mangleCXXCtorVTable(RD, Base.getBaseOffset().getQuantity(),
                            Base.getBase(), Out);
   StringRef Name = OutName.str();
 
-  llvm::Type *VTType = getVTableType(*VTLayout);
+  llvm::Type *VTType = getVTableType(RD, *VTLayout);
 
   // Construction vtable symbols are not part of the Itanium ABI, so we cannot
   // guarantee that they actually will be available externally. Instead, when
   // emitting an available_externally VTT, we provide references to an internal
   // linkage construction vtable. The ABI only requires complete-object vtables
   // to be the same for all instances of a type, not construction vtables.
   if (Linkage == llvm::GlobalVariable::AvailableExternallyLinkage)
     Linkage = llvm::GlobalVariable::InternalLinkage;
 
   unsigned Align = CGM.getDataLayout().getABITypeAlignment(VTType);
 
   // Create the variable that will hold the construction vtable.
   llvm::GlobalVariable *VTable =
       CGM.CreateOrReplaceCXXRuntimeVariable(Name, VTType, Linkage, Align);
 
   // V-tables are always unnamed_addr.
   VTable->setUnnamedAddr(llvm::GlobalValue::UnnamedAddr::Global);
 
   llvm::Constant *RTTI = CGM.GetAddrOfRTTIDescriptor(
       CGM.getContext().getTagDeclType(Base.getBase()));
 
   // Create and set the initializer.
-  ConstantInitBuilder builder(CGM);
-  auto components = builder.beginStruct();
-  createVTableInitializer(components, *VTLayout, RTTI);
-  components.finishAndSetAsInitializer(VTable);
+  SetVTableInitializer(VTable, *VTLayout, RTTI);
 
   // Set properties only after the initializer has been set to ensure that the
   // GV is treated as definition and not declaration.
   assert(!VTable->isDeclaration() && "Shouldn't set properties on declaration");
   CGM.setGVProperties(VTable, RD);
 
   CGM.EmitVTableTypeMetadata(VTable, *VTLayout.get());
 
llvm::sort(AddressPoints, [this](const AddressPoint &AP1,
     if (S1 != S2)
       return false;
 
     return AP1.second < AP2.second;
   });
 
   ArrayRef<VTableComponent> Comps = VTLayout.vtable_components();
   for (auto AP : AddressPoints) {
+    CharUnits ByteOffset = PointerWidth * AP.second;
+
     // Create type metadata for the address point.
-    AddVTableTypeMetadata(VTable, PointerWidth * AP.second, AP.first);
+    AddVTableTypeMetadata(VTable, ByteOffset, AP.first);
 
     // The class associated with each address point could also potentially be
     // used for indirect calls via a member function pointer, so we need to
     // annotate the address of each function pointer with the appropriate member
     // function pointer type.
     for (unsigned I = 0; I != Comps.size(); ++I) {
       if (Comps[I].getKind() != VTableComponent::CK_FunctionPointer)
         continue;
       llvm::Metadata *MD = CreateMetadataIdentifierForVirtualMemPtrType(
           Context.getMemberPointerType(
               Comps[I].getFunctionDecl()->getType(),
               Context.getRecordType(AP.first).getTypePtr()));
       VTable->addTypeMetadata((PointerWidth * I).getQuantity(), MD);
     }
   }
 }

clang/lib/CodeGen/CodeGenFunction.h

public:
 
   void InitializeVTablePointers(const CXXRecordDecl *ClassDecl);
 
   /// GetVTablePtr - Return the Value of the vtable pointer member pointed
   /// to by This.
   llvm::Value *GetVTablePtr(Address This, llvm::Type *VTableTy,
                             const CXXRecordDecl *VTableClass);
 
+  llvm::Value *GetVirtualFunctionFromVTable(const CXXRecordDecl *RD,
+                                            llvm::Value *VTable,
+                                            uint64_t VTableIndex,
+                                            llvm::Type *Ty);
+
   enum CFITypeCheckKind {
     CFITCK_VCall,
     CFITCK_NVCall,
     CFITCK_DerivedCast,
     CFITCK_UnrelatedCast,
     CFITCK_ICall,
     CFITCK_NVMFCall,
     CFITCK_VMFCall,

clang/lib/CodeGen/ItaniumCXXABI.cpp

if (CGM.getCodeGenOpts().SanitizeTrap.has(SanitizerKind::CFIMFCall)) {
                     SanitizerHandler::CFICheckFail, StaticData,
                     {VTable, ValidVtable});
     }
 
     FnVirtual = Builder.GetInsertBlock();
   }
 
   // Load the virtual function to call.
-  VFPAddr = Builder.CreateBitCast(VFPAddr, FTy->getPointerTo()->getPointerTo());
-  llvm::Value *VirtualFn = Builder.CreateAlignedLoad(
-      VFPAddr, CGF.getPointerAlign(), "memptr.virtualfn");
+  llvm::Value *VirtualFn;
+  if (CGM.getLangOpts().RelativeCXXABIVTables) {
+    VirtualFn =
+        Builder.CreateCall(CGM.getIntrinsic(llvm::Intrinsic::load_relative,
+                                            {VTableOffset->getType()}),
+                           {VTable, VTableOffset});
+    VirtualFn = Builder.CreateBitCast(VirtualFn, FTy->getPointerTo());
+  } else {
+    // Apply the offset.
+    llvm::Value *VTableSlotPtr =
+        Builder.CreateBitCast(VFPAddr, FTy->getPointerTo()->getPointerTo());
+    VirtualFn = Builder.CreateAlignedLoad(VTableSlotPtr, CGF.getPointerAlign(),
+                                          "memptr.virtualfn");
+  }
   CGF.EmitBranch(FnEnd);
 
   // In the non-virtual path, the function pointer is actually a
   // function pointer.
   CGF.EmitBlock(FnNonVirtual);
   llvm::Value *NonVirtualFn =
     Builder.CreateIntToPtr(FnAsInt, FTy->getPointerTo(), "memptr.nonvirtualfn");
 
llvm::Constant *ItaniumCXXABI::BuildMemberPointer(const CXXMethodDecl *MD,
   // Get the function pointer (or index if this is a virtual function).
   llvm::Constant *MemPtr[2];
   if (MD->isVirtual()) {
     uint64_t Index = CGM.getItaniumVTableContext().getMethodVTableIndex(MD);
 
     const ASTContext &Context = getContext();
     CharUnits PointerWidth =
       Context.toCharUnitsFromBits(Context.getTargetInfo().getPointerWidth(0));
-    uint64_t VTableOffset = (Index * PointerWidth.getQuantity());
+    uint64_t VTableOffset = Index;
+    if (CGM.getLangOpts().RelativeCXXABIVTables)
+      VTableOffset *= 4;
+    else
+      VTableOffset *= PointerWidth.getQuantity();
 
     if (UseARMMethodPtrABI) {
       // ARM C++ ABI 3.2.1:
       //   This ABI specifies that adj contains twice the this
       //   adjustment, plus 1 if the member function is virtual. The
       //   least significant bit of adj then makes exactly the same
       //   discrimination as the least significant bit of ptr does for
       //   Itanium.
void ItaniumCXXABI::emitVTableDefinitions(CodeGenVTables &CGVT,
 
   ItaniumVTableContext &VTContext = CGM.getItaniumVTableContext();
   const VTableLayout &VTLayout = VTContext.getVTableLayout(RD);
   llvm::GlobalVariable::LinkageTypes Linkage = CGM.getVTableLinkage(RD);
   llvm::Constant *RTTI =
       CGM.GetAddrOfRTTIDescriptor(CGM.getContext().getTagDeclType(RD));
 
   // Create and set the initializer.
-  ConstantInitBuilder Builder(CGM);
-  auto Components = Builder.beginStruct();
-  CGVT.createVTableInitializer(Components, VTLayout, RTTI);
-  Components.finishAndSetAsInitializer(VTable);
+  CGVT.SetVTableInitializer(VTable, VTLayout, RTTI);
 
   // Set the correct linkage.
   VTable->setLinkage(Linkage);
 
   if (CGM.supportsCOMDAT() && VTable->isWeakForLinker())
     VTable->setComdat(CGM.getModule().getOrInsertComdat(VTable->getName()));
 
   // Set the right visibility.
llvm::GlobalVariable *ItaniumCXXABI::getAddrOfVTable(const CXXRecordDecl *RD,
   CGM.addDeferredVTable(RD);
 
   SmallString<256> Name;
   llvm::raw_svector_ostream Out(Name);
   getMangleContext().mangleCXXVTable(RD, Out);
 
   const VTableLayout &VTLayout =
       CGM.getItaniumVTableContext().getVTableLayout(RD);
-  llvm::Type *VTableType = CGM.getVTables().getVTableType(VTLayout);
+  llvm::Type *VTableType = CGM.getVTables().getVTableType(RD, VTLayout);
 
   // Use pointer alignment for the vtable. Otherwise we would align them based
   // on the size of the initializer which doesn't make sense as only single
   // values are read.
   unsigned PAlign = CGM.getTarget().getPointerAlign(0);
 
   VTable = CGM.CreateOrReplaceCXXRuntimeVariable(
       Name, VTableType, llvm::GlobalValue::ExternalLinkage,
       getContext().toCharUnitsFromBits(PAlign).getQuantity());
   VTable->setUnnamedAddr(llvm::GlobalValue::UnnamedAddr::Global);
 
   CGM.setGVProperties(VTable, RD);
 
   return VTable;
 }
 
 CGCallee ItaniumCXXABI::getVirtualFunctionPointer(CodeGenFunction &CGF,
                                                   GlobalDecl GD,
                                                   Address This,
                                                   llvm::Type *Ty,
                                                   SourceLocation Loc) {
-  Ty = Ty->getPointerTo()->getPointerTo();
   auto *MethodDecl = cast<CXXMethodDecl>(GD.getDecl());
-  llvm::Value *VTable = CGF.GetVTablePtr(This, Ty, MethodDecl->getParent());
+  llvm::Value *VTable = CGF.GetVTablePtr(
+      This, Ty->getPointerTo()->getPointerTo(), MethodDecl->getParent());
 
   uint64_t VTableIndex = CGM.getItaniumVTableContext().getMethodVTableIndex(GD);
   llvm::Value *VFunc;
   if (CGF.ShouldEmitVTableTypeCheckedLoad(MethodDecl->getParent())) {
     VFunc = CGF.EmitVTableTypeCheckedLoad(
         MethodDecl->getParent(), VTable,
         VTableIndex * CGM.getContext().getTargetInfo().getPointerWidth(0) / 8);
   } else {
     CGF.EmitTypeMetadataCodeForVCall(MethodDecl->getParent(), VTable, Loc);
 
-    llvm::Value *VFuncPtr =
-        CGF.Builder.CreateConstInBoundsGEP1_64(VTable, VTableIndex, "vfn");
-    auto *VFuncLoad =
-        CGF.Builder.CreateAlignedLoad(VFuncPtr, CGF.getPointerAlign());
+    auto *VFuncLoad = CGF.GetVirtualFunctionFromVTable(MethodDecl->getParent(),
+                                                       VTable, VTableIndex, Ty);
 
     // Add !invariant.load md to virtual function load to indicate that
     // function didn't change inside vtable.
     // It's safe to add it without -fstrict-vtable-pointers, but it would not
     // help in devirtualization because it will only matter if we will have 2
     // the same virtual function loads from the same vtable load, which won't
     // happen without enabled devirtualization with -fstrict-vtable-pointers.
     if (CGM.getCodeGenOpts().OptimizationLevel > 0 &&
-        CGM.getCodeGenOpts().StrictVTablePointers)
-      VFuncLoad->setMetadata(
+        CGM.getCodeGenOpts().StrictVTablePointers) {
+      if (auto *VFuncLoadInstr = dyn_cast<llvm::Instruction>(VFuncLoad)) {
+        VFuncLoadInstr->setMetadata(
             llvm::LLVMContext::MD_invariant_load,
             llvm::MDNode::get(CGM.getLLVMContext(),
                               llvm::ArrayRef<llvm::Metadata *>()));
+      }
+    }
     VFunc = VFuncLoad;
   }
 
   CGCallee Callee(GD, VFunc);
   return Callee;
 }
 
 llvm::Value *ItaniumCXXABI::EmitVirtualDestructorCall(

clang/lib/CodeGen/MicrosoftCXXABI.cpp

for (const std::unique_ptr<VPtrInfo>& Info : VFPtrs) {
     const VTableLayout &VTLayout =
       VFTContext.getVFTableLayout(RD, Info->FullOffsetInMDC);
 
     llvm::Constant *RTTI = nullptr;
     if (any_of(VTLayout.vtable_components(),
                [](const VTableComponent &VTC) { return VTC.isRTTIKind(); }))
       RTTI = getMSCompleteObjectLocator(RD, *Info);
 
-    ConstantInitBuilder Builder(CGM);
-    auto Components = Builder.beginStruct();
-    CGVT.createVTableInitializer(Components, VTLayout, RTTI);
-    Components.finishAndSetAsInitializer(VTable);
+    CGVT.SetVTableInitializer(VTable, VTLayout, RTTI);
 
     emitVTableTypeMetadata(*Info, RD, VTable);
   }
 }
 
 bool MicrosoftCXXABI::isVirtualOffsetNeededForVTableField(
     CodeGenFunction &CGF, CodeGenFunction::VPtr Vptr) {
   return Vptr.NearestVBase != nullptr;
#endif
 
   const VTableLayout &VTLayout =
       VTContext.getVFTableLayout(RD, VFPtr->FullOffsetInMDC);
   llvm::GlobalValue::LinkageTypes VTableLinkage =
       VTableAliasIsRequred ? llvm::GlobalValue::PrivateLinkage : VFTableLinkage;
 
   StringRef VTableName = VTableAliasIsRequred ? StringRef() : VFTableName.str();
 
-  llvm::Type *VTableType = CGM.getVTables().getVTableType(VTLayout);
+  llvm::Type *VTableType = CGM.getVTables().getVTableType(RD, VTLayout);
 
   // Create a backing variable for the contents of VTable.  The VTable may
   // or may not include space for a pointer to RTTI data.
   llvm::GlobalValue *VFTable;
   VTable = new llvm::GlobalVariable(CGM.getModule(), VTableType,
                                     /*isConstant=*/true, VTableLinkage,
                                     /*Initializer=*/nullptr, VTableName);
   VTable->setUnnamedAddr(llvm::GlobalValue::UnnamedAddr::Global);
if (VTableAliasIsRequred) {
     // this should be the location of the first virtual method.
     llvm::Constant *VTableGEP = llvm::ConstantExpr::getInBoundsGetElementPtr(
         VTable->getValueType(), VTable, GEPIndices);
     if (llvm::GlobalValue::isWeakForLinker(VFTableLinkage)) {
       VFTableLinkage = llvm::GlobalValue::ExternalLinkage;
       if (C)
         C->setSelectionKind(llvm::Comdat::Largest);
     }
+    VTableGEP = llvm::ConstantExpr::getBitCast(VTableGEP, CGM.Int8PtrPtrTy);
     VFTable = llvm::GlobalAlias::create(CGM.Int8PtrTy,
                                         /*AddressSpace=*/0, VFTableLinkage,
                                         VFTableName.str(), VTableGEP,
                                         &CGM.getModule());
     VFTable->setUnnamedAddr(llvm::GlobalValue::UnnamedAddr::Global);
   } else {
     // We don't need a GlobalAlias to be a symbol for the VTable if we won't
     // be referencing any RTTI data.
#endif
   return VTable;
 }
 
 CGCallee MicrosoftCXXABI::getVirtualFunctionPointer(CodeGenFunction &CGF,
                                                     GlobalDecl GD,
                                                     Address This,
                                                     llvm::Type *Ty,
                                                     SourceLocation Loc) {
-  CGBuilderTy &Builder = CGF.Builder;
-
-  Ty = Ty->getPointerTo()->getPointerTo();
   Address VPtr =
       adjustThisArgumentForVirtualFunctionCall(CGF, GD, This, true);
 
   auto *MethodDecl = cast<CXXMethodDecl>(GD.getDecl());
-  llvm::Value *VTable = CGF.GetVTablePtr(VPtr, Ty, MethodDecl->getParent());
+  llvm::Value *VTable = CGF.GetVTablePtr(
+      VPtr, Ty->getPointerTo()->getPointerTo(), MethodDecl->getParent());
 
   MicrosoftVTableContext &VFTContext = CGM.getMicrosoftVTableContext();
   MethodVFTableLocation ML = VFTContext.getMethodVFTableLocation(GD);
 
   // Compute the identity of the most derived class whose virtual table is
   // located at the MethodVFTableLocation ML.
   auto getObjectWithVPtr = [&] {
     return llvm::find_if(VFTContext.getVFPtrOffsets(
CGCallee MicrosoftCXXABI::getVirtualFunctionPointer(CodeGenFunction &CGF,
   if (CGF.ShouldEmitVTableTypeCheckedLoad(MethodDecl->getParent())) {
     VFunc = CGF.EmitVTableTypeCheckedLoad(
         getObjectWithVPtr(), VTable,
         ML.Index * CGM.getContext().getTargetInfo().getPointerWidth(0) / 8);
   } else {
     if (CGM.getCodeGenOpts().PrepareForLTO)
       CGF.EmitTypeMetadataCodeForVCall(getObjectWithVPtr(), VTable, Loc);
 
-    llvm::Value *VFuncPtr =
-        Builder.CreateConstInBoundsGEP1_64(VTable, ML.Index, "vfn");
-    VFunc = Builder.CreateAlignedLoad(VFuncPtr, CGF.getPointerAlign());
+    VFunc = CGF.GetVirtualFunctionFromVTable(MethodDecl->getParent(), VTable,
+                                             ML.Index, Ty);
   }
 
   CGCallee Callee(GD, VFunc);
   return Callee;
 }
 
 llvm::Value *MicrosoftCXXABI::EmitVirtualDestructorCall(
     CodeGenFunction &CGF, const CXXDestructorDecl *Dtor, CXXDtorType DtorType,
CGF.StartFunction(GlobalDecl(), FnInfo.getReturnType(), ThunkFn, FnInfo,
                     FunctionArgs, MD->getLocation(), SourceLocation());
   setCXXABIThisValue(CGF, loadIncomingCXXThis(CGF));
 
   // Load the vfptr and then callee from the vftable.  The callee should have
   // adjusted 'this' so that the vfptr is at offset zero.
   llvm::Value *VTable = CGF.GetVTablePtr(
       getThisAddress(CGF), ThunkTy->getPointerTo()->getPointerTo(), MD->getParent());
 
-  llvm::Value *VFuncPtr =
-      CGF.Builder.CreateConstInBoundsGEP1_64(VTable, ML.Index, "vfn");
-  llvm::Value *Callee =
-    CGF.Builder.CreateAlignedLoad(VFuncPtr, CGF.getPointerAlign());
+  llvm::Value *Callee = CGF.GetVirtualFunctionFromVTable(
+      MD->getParent(), VTable, ML.Index, ThunkTy);
 
   CGF.EmitMustTailThunk(MD, getThisValue(CGF), {ThunkTy, Callee});
 
   return ThunkFn;
 }
 
 void MicrosoftCXXABI::emitVirtualInheritanceTables(const CXXRecordDecl *RD) {
   const VBTableGlobals &VBGlobals = enumerateVBTables(RD);

clang/lib/Driver/ToolChains/Clang.cpp

   if (WholeProgramVTables) {
     if (!D.isUsingLTO())
       D.Diag(diag::err_drv_argument_only_allowed_with)
           << "-fwhole-program-vtables"
           << "-flto";
     CmdArgs.push_back("-fwhole-program-vtables");
   }
 
+  // Add unstable C++ ABI flags.
+  if (Args.hasArg(options::OPT_fexperimental_relative_cxx_abi_vtables))
+    CmdArgs.push_back("-fexperimental-relative-c++-abi-vtables");
+
   bool RequiresSplitLTOUnit = WholeProgramVTables || Sanitize.needsLTO();
   bool SplitLTOUnit =
       Args.hasFlag(options::OPT_fsplit_lto_unit,
                    options::OPT_fno_split_lto_unit, RequiresSplitLTOUnit);
   if (RequiresSplitLTOUnit && !SplitLTOUnit)
     D.Diag(diag::err_drv_argument_not_allowed_with)
         << "-fno-split-lto-unit"
         << (WholeProgramVTables ? "-fwhole-program-vtables" : "-fsanitize=cfi");

clang/lib/Frontend/CompilerInvocation.cpp

if (Arg *A = Args.getLastArg(OPT_fclang_abi_compat_EQ)) {
     } else if (Ver != "latest") {
       Diags.Report(diag::err_drv_invalid_value)
           << A->getAsString(Args) << A->getValue();
     }
   }
 
   Opts.CompleteMemberPointers = Args.hasArg(OPT_fcomplete_member_pointers);
   Opts.BuildingPCHWithObjectFile = Args.hasArg(OPT_building_pch_with_obj);
+
+  Opts.RelativeCXXABIVTables =
+      Args.hasArg(OPT_fexperimental_relative_cxx_abi_vtables);
 }
 
 static bool isStrictlyPreprocessorAction(frontend::ActionKind Action) {
   switch (Action) {
   case frontend::ASTDeclList:
   case frontend::ASTDump:
   case frontend::ASTPrint:
   case frontend::ASTView:

clang/test/CodeGenCXX/debug-info-virtual-fn-relative.cpp

+// RUN: %clang_cc1 -emit-llvm -debug-info-kind=limited %s -o - | FileCheck --check-prefix=CHECK-STABLE %s
+// RUN: %clang_cc1 -fexperimental-relative-c++-abi-vtables -emit-llvm -debug-info-kind=limited %s -o - | FileCheck --check-prefix=CHECK-UNSTABLE %s
+
+struct S {
+  S();
+  virtual void f();
+};
+
+// CHECK-STABLE: virtualIndex: 0,
+// CHECK-UNSTABLE: virtualIndex: 4294967295,
+S::S() {}
+void S::f() {}

clang/test/CodeGenCXX/vtable-relative-abi.cpp

+// RUN: %clang_cc1 %s -triple x86_64-unknown-linux-gnu -fexperimental-relative-c++-abi-vtables -emit-llvm -o - | FileCheck --check-prefix=CHECK --check-prefix=CHECK-ITANIUM %s
+// RUN: %clang_cc1 %s -triple x86_64-unknown-windows-msvc -fexperimental-relative-c++-abi-vtables -emit-llvm -o - | FileCheck --check-prefix=CHECK --check-prefix=CHECK-MS %s
+
+// CHECK-ITANIUM: @_ZTV1S = unnamed_addr constant { { i8*, i8*, i32, i32 } } { { i8*, i8*, i32, i32 } { i8* null, i8* bitcast ({ i8*, i8* }* @_ZTI1S to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.S*)* @_ZN1S2f1Ev to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i8*, i32, i32 } }, { { i8*, i8*, i32, i32 } }* @_ZTV1S, i32 0, i32 0, i32 2) to i64)) to i32), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.S*)* @_ZN1S2f2Ev to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i8*, i32, i32 } }, { { i8*, i8*, i32, i32 } }* @_ZTV1S, i32 0, i32 0, i32 2) to i64)) to i32) } }, align 8
+// CHECK-MS: @0 = private unnamed_addr constant { { i8*, i32, i32 } } { { i8*, i32, i32 } { i8* bitcast (%rtti.CompleteObjectLocator* @"??_R4S@@6B@" to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.S*)* @"?f1@S@@UEAAXXZ" to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i32, i32 } }, { { i8*, i32, i32 } }* @0, i32 0, i32 0, i32 1) to i64)) to i32), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.S*)* @"?f2@S@@UEAAXXZ" to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i32, i32 } }, { { i8*, i32, i32 } }* @0, i32 0, i32 0, i32 1) to i64)) to i32) } }, comdat($"??_7S@@6B@")
+
+struct S {
+  S();
+  virtual void f1();
+  virtual void f2();
+};
+
+// CHECK-ITANIUM: @_ZTV1T = unnamed_addr constant { { i8*, i8*, i32 } } { { i8*, i8*, i32 } { i8* null, i8* bitcast ({ i8*, i8* }* @_ZTI1T to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.T*)* @_ZN1T1gEv to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i8*, i32 } }, { { i8*, i8*, i32 } }* @_ZTV1T, i32 0, i32 0, i32 2) to i64)) to i32) } }, align 8
+// CHECK-MS: @1 = private unnamed_addr constant { { i8*, i32 } } { { i8*, i32 } { i8* bitcast (%rtti.CompleteObjectLocator* @"??_R4T@@6B@" to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.T*)* @"?g@T@@UEAAXXZ" to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i32 } }, { { i8*, i32 } }* @1, i32 0, i32 0, i32 1) to i64)) to i32) } }, comdat($"??_7T@@6B@")
+struct T {
+  T();
+  virtual void g();
+};
+
+// CHECK-ITANIUM: @_ZTV1U = unnamed_addr constant { { i8*, i8*, i32, i32 }, { i8*, i8*, i32 } } { { i8*, i8*, i32, i32 } { i8* null, i8* bitcast ({ i8*, i8*, i32, i32, i8*, i64, i8*, i64 }* @_ZTI1U to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.U*)* @_ZN1U2f1Ev to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i8*, i32, i32 }, { i8*, i8*, i32 } }, { { i8*, i8*, i32, i32 }, { i8*, i8*, i32 } }* @_ZTV1U, i32 0, i32 0, i32 2) to i64)) to i32), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.S*)* @_ZN1S2f2Ev to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i8*, i32, i32 }, { i8*, i8*, i32 } }, { { i8*, i8*, i32, i32 }, { i8*, i8*, i32 } }* @_ZTV1U, i32 0, i32 0, i32 2) to i64)) to i32) }, { i8*, i8*, i32 } { i8* inttoptr (i64 -8 to i8*), i8* bitcast ({ i8*, i8*, i32, i32, i8*, i64, i8*, i64 }* @_ZTI1U to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.T*)* @_ZN1T1gEv to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i8*, i32, i32 }, { i8*, i8*, i32 } }, { { i8*, i8*, i32, i32 }, { i8*, i8*, i32 } }* @_ZTV1U, i32 0, i32 1, i32 2) to i64)) to i32) } }, align 8
+// CHECK-MS: @2 = private unnamed_addr constant { { i8*, i32, i32 } } { { i8*, i32, i32 } { i8* bitcast (%rtti.CompleteObjectLocator* @"??_R4U@@6BS@@@" to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.U*)* @"?f1@U@@UEAAXXZ" to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i32, i32 } }, { { i8*, i32, i32 } }* @2, i32 0, i32 0, i32 1) to i64)) to i32), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.S*)* @"?f2@S@@UEAAXXZ" to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i32, i32 } }, { { i8*, i32, i32 } }* @2, i32 0, i32 0, i32 1) to i64)) to i32) } }, comdat($"??_7U@@6BS@@@")
+// CHECK-MS: @3 = private unnamed_addr constant { { i8*, i32 } } { { i8*, i32 } { i8* bitcast (%rtti.CompleteObjectLocator* @"??_R4U@@6BT@@@" to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.T*)* @"?g@T@@UEAAXXZ" to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i32 } }, { { i8*, i32 } }* @3, i32 0, i32 0, i32 1) to i64)) to i32) } }, comdat($"??_7U@@6BT@@@")
+struct U : S, T {
+  U();
+  virtual void f1();
+};
+
+S::S() {}
+void S::f1() {}
+
+T::T() {}
+void T::g() {}
+
+U::U() {}
+void U::f1() {}
+
+struct V {
+  virtual void f();
+};
+
+struct V1 : virtual V {
+};
+
+struct V2 : virtual V {
+};
+
+// CHECK-ITANIUM: @_ZTC2V30_2V1 = linkonce_odr unnamed_addr constant { { i8*, i8*, i8*, i8*, i32 } } { { i8*, i8*, i8*, i8*, i32 } { i8* null, i8* null, i8* null, i8* bitcast ({ i8*, i8*, i32, i32, i8*, i64 }* @_ZTI2V1 to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.V*)* @_ZN1V1fEv to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i8*, i8*, i8*, i32 } }, { { i8*, i8*, i8*, i8*, i32 } }* @_ZTC2V30_2V1, i32 0, i32 0, i32 4) to i64)) to i32) } }, comdat, align 8
+// CHECK-ITANIUM: @_ZTC2V38_2V2 = linkonce_odr unnamed_addr constant { { i8*, i8*, i8*, i8*, i32 }, { i8*, i8*, i8*, i32 } } { { i8*, i8*, i8*, i8*, i32 } { i8* inttoptr (i64 -8 to i8*), i8* inttoptr (i64 -8 to i8*), i8* null, i8* bitcast ({ i8*, i8*, i32, i32, i8*, i64 }* @_ZTI2V2 to i8*), i32 0 }, { i8*, i8*, i8*, i32 } { i8* null, i8* inttoptr (i64 8 to i8*), i8* bitcast ({ i8*, i8*, i32, i32, i8*, i64 }* @_ZTI2V2 to i8*), i32 trunc (i64 sub (i64 ptrtoint (void (%struct.V*)* @_ZN1V1fEv to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i8*, i8*, i8*, i8*, i32 }, { i8*, i8*, i8*, i32 } }, { { i8*, i8*, i8*, i8*, i32 }, { i8*, i8*, i8*, i32 } }* @_ZTC2V38_2V2, i32 0, i32 1, i32 3) to i64)) to i32) } }, comdat, align 8
+struct V3 : V1, V2 {
+  V3();
+};
+
+V3::V3() {}
+
+// CHECK-ITANIUM: define void @_Z5call1P1S
+// CHECK-MS: define {{.*}} void @"?call1@@YAXPEAUS@@@Z"
+void call1(S *s) {
+  // CHECK: [[VTABLE:%.*]] = load void
+  // CHECK: [[VTABLEI8:%.*]] = bitcast {{.*}} [[VTABLE]] to i8*
+  // CHECK: [[VFN:%.*]] = call i8* @llvm.load.relative.i32(i8* [[VTABLEI8]], i32 0)
+  // CHECK: [[VFNCASTED:%.*]] = bitcast i8* [[VFN]] to void (
+  // CHECK: call void [[VFNCASTED]](
+  s->f1();
+}
+
+// CHECK-ITANIUM: define void @_Z5call2P1S
+// CHECK-MS: define {{.*}} void @"?call2@@YAXPEAUS@@@Z"
+void call2(S *s) {
+  // CHECK: [[VTABLE:%.*]] = load void
+  // CHECK: [[VTABLEI8:%.*]] = bitcast {{.*}} [[VTABLE]] to i8*
+  // CHECK: [[VFN:%.*]] = call i8* @llvm.load.relative.i32(i8* [[VTABLEI8]], i32 4)
+  // CHECK: [[VFNCASTED:%.*]] = bitcast i8* [[VFN]] to void (
+  // CHECK: call void [[VFNCASTED]](
+  s->f2();
+}
+
+typedef void (S::*mfp)();
+
+// CHECK-ITANIUM: define { i64, i64 } @_Z7getmfp1v()
+// CHECK-ITANIUM: ret { i64, i64 } { i64 1, i64 0 }
+// CHECK-MS: define {{.*}} i8* @"?getmfp1@@YAP8S@@EAAXXZXZ"()
+// CHECK-MS: ret i8* bitcast {{.*}} @"??_9S@@$BA@AA"
+// CHECK-MS: define linkonce_odr void @"??_9S@@$BA@AA"
+// CHECK-MS: [[VTABLE:%.*]] = load void
+// CHECK-MS: [[VTABLEI8:%.*]] = bitcast {{.*}} [[VTABLE]] to i8*
+// CHECK-MS: [[VFN:%.*]] = call i8* @llvm.load.relative.i32(i8* [[VTABLEI8]], i32 0)
+// CHECK-MS: [[VFNCASTED:%.*]] = bitcast i8* [[VFN]] to void (
+// CHECK-MS: musttail call {{.*}} [[VFNCASTED]](
+mfp getmfp1() {
+  return &S::f1;
+}
+
+// CHECK-ITANIUM: define { i64, i64 } @_Z7getmfp2v()
+// CHECK-ITANIUM: ret { i64, i64 } { i64 5, i64 0 }
+// CHECK-MS: define {{.*}} i8* @"?getmfp2@@YAP8S@@EAAXXZXZ"()
+// CHECK-MS: ret i8* bitcast {{.*}} @"??_9S@@$B7AA"
+// CHECK-MS: define linkonce_odr void @"??_9S@@$B7AA"
+// CHECK-MS: [[VTABLE:%.*]] = load void
+// CHECK-MS: [[VTABLEI8:%.*]] = bitcast {{.*}} [[VTABLE]] to i8*
+// CHECK-MS: [[VFN:%.*]] = call i8* @llvm.load.relative.i32(i8* [[VTABLEI8]], i32 4)
+// CHECK-MS: [[VFNCASTED:%.*]] = bitcast i8* [[VFN]] to void (
+// CHECK-MS: musttail call {{.*}} [[VFNCASTED]](
+mfp getmfp2() {
+  return &S::f2;
+}
+
+// In the MS ABI virtual member function calls use thunks (which we already
+// tested above), so there's nothing to test that's specific to the relative
+// ABI.
+
+// CHECK-ITANIUM: define void @_Z7callmfpP1SMS_FvvE
+void callmfp(S *s, mfp p) {
+  // CHECK-ITANIUM: [[VTOFFSET:%.*]] = sub i64 {{.*}}, 1
+  // CHECK-ITANIUM: [[VFN:%.*]] = call i8* @llvm.load.relative.i64(i8* {{.*}}, i64 [[VTOFFSET]])
+  // CHECK-ITANIUM: bitcast i8* [[VFN]] to void (
+  (s->*p)();
+}

llvm/lib/IR/Constants.cpp

if (!UC || isa<GlobalValue>(UC))
       return true;
 
     if (UC->isConstantUsed())
       return true;
   }
   return false;
 }
 
+/// Returns true if this constant mathes the relative referencing for virtual
+/// functions introduced in the relative ABI. This is the difference between a
+/// function and its corresponding vtable entry.
+///
+/// i64 sub (
+///   i64 ptrtoint (i8* obj to i64),
+//    i64 ptrtoint (
+//      i32* getelementptr inbounds (%object_vtable, %object_vtable*
+//        @_ZTV12derived_virt, i32 0, i32 0, i32 2) to i64)
+static bool MatchesRelativeOffset(const Constant *C) {
+  const auto *CE = dyn_cast<ConstantExpr>(C);
+  if (!CE || CE->getOpcode() != Instruction::Sub)
+    return false;
+
+  const auto *LHS = dyn_cast<ConstantExpr>(CE->getOperand(0));
+  const auto *RHS = dyn_cast<ConstantExpr>(CE->getOperand(1));
+  if (!LHS || !RHS || LHS->getOpcode() != Instruction::PtrToInt ||
+      RHS->getOpcode() != Instruction::PtrToInt)
+    return false;
+
+  RHS = dyn_cast<ConstantExpr>(RHS->getOperand(0));
+  if (!RHS)
+    return false;
+
+  return (isa<GlobalValue>(LHS->getOperand(0)) &&
+          RHS->getOpcode() == Instruction::GetElementPtr &&
+          isa<GlobalValue>(RHS->getOperand(0)));
+}
+
 bool Constant::needsRelocation() const {
   if (isa<GlobalValue>(this))
     return true; // Global reference.
 
   if (const BlockAddress *BA = dyn_cast<BlockAddress>(this))
     return BA->getFunction()->needsRelocation();
 
   // While raw uses of blockaddress need to be relocated, differences between
   // two of them don't when they are for labels in the same function.  This is a
   // common idiom when creating a table for the indirect goto extension, so we
   // handle it efficiently here.
   if (const ConstantExpr *CE = dyn_cast<ConstantExpr>(this))
     if (CE->getOpcode() == Instruction::Sub) {
       ConstantExpr *LHS = dyn_cast<ConstantExpr>(CE->getOperand(0));
       ConstantExpr *RHS = dyn_cast<ConstantExpr>(CE->getOperand(1));
       if (LHS && RHS && LHS->getOpcode() == Instruction::PtrToInt &&
           RHS->getOpcode() == Instruction::PtrToInt &&
           isa<BlockAddress>(LHS->getOperand(0)) &&
           isa<BlockAddress>(RHS->getOperand(0)) &&
           cast<BlockAddress>(LHS->getOperand(0))->getFunction() ==
               cast<BlockAddress>(RHS->getOperand(0))->getFunction())
         return false;
     }
 
+  // This results in an R_PLT_PC reloc which can be computed at link time.
+  if (MatchesRelativeOffset(this))
+    return false;
+
   bool Result = false;
   for (unsigned i = 0, e = getNumOperands(); i != e; ++i)
     Result |= cast<Constant>(getOperand(i))->needsRelocation();
 
   return Result;
 }
 
 /// If the specified constantexpr is dead, remove it. This involves recursively