Front-end of the implementation of the interleaving algorithm

[zhaomo]

Summary

This patch includes the front-end of the interleaving algorithm implemented in https://reviews.llvm.org/D50754.

This patch introduces a new front-end option named -enable-vtable-interleaving. When this option is set, the LTO visibility of the module is hidden, and the cfi-vcall is enabled for the module, this front-end creates a placeholder (marked with a new type of metadata called "offset_type" metadata) for each used offset in a vtable. These placeholders will be replaced with the actually offsets in the interleaved layout when the interleaved layout is decided, which is implemented in https://reviews.llvm.org/D50754.

[pcc]

clang/lib/CodeGen/CGCXXABI.h

52

Why does this need to be stored separately on the CGCXXABI? Can't we always access it via the CodeGenOptions on CGM?

clang/lib/CodeGen/CGClass.cpp

2798–2799

Do you need this function as well as the one you're adding? Can't you change the code in MicrosoftCXXABI.cpp to form a constant?

clang/lib/CodeGen/CodeGenModule.cpp

5779

Comment should say why.

clang/lib/CodeGen/ItaniumCXXABI.cpp

993

I wouldn't add a function to GlobalObject for this, you can add the metadata here directly since this is the only place in the code where you need to add it.

1787

Remind me why this needs to be here? (And the explanation needs to be in a comment.)

clang/lib/Frontend/CompilerInvocation.cpp

1400

Use hasArg here instead, there's no need to support a flag for disabling interleaving in the frontend.

[vlad.tsyrklevich]

This change causes all compiler-rt cfi tests to be UNSUPPORTED for me locally, do you have any idea why that might be? The lit changes don't make it immediately clear.

clang/lib/CodeGen/CGVTables.cpp

1146

member function pointers*

clang/lib/CodeGen/ItaniumCXXABI.cpp

949

doxygen comments normally go in the corresponding header file.

compiler-rt/test/cfi/CMakeLists.txt

62

nit: spacing

compiler-rt/test/cfi/cross-dso-diagnostic.cpp

7 ↗

(On Diff #164783)

Why? This test has to do with error printing to do with vcalls across DSOs, not cross-DSO CFI, so I wouldn't expect that this should be affected by this change.

compiler-rt/test/cfi/lit.cfg

33 ↗

(On Diff #164783)

Can get rid of at least save-temps here, do we need to pass -enable-vtable-interleaving to the linker?

[zhaomo]

clang/lib/CodeGen/ItaniumCXXABI.cpp

1787

The calculation of address point is essentially base_vtable_addr + offset, where offset is from the indices of gep.
In the interleaving pass, we replace base_vtable_addr with (addr_point_in_interleaved_layout - offset).

The LLVM language reference says that the base address of a inbounds gep must be an in bound address of the object. The new base address addr_point_in_interleaved_layout - offset, however, may not be an in bound address.

[zhaomo]

clang/lib/CodeGen/ItaniumCXXABI.cpp

949

I just followed the style of the rest of the file. All the comments above functions use ///. Do I need to change them?

[zhaomo]

compiler-rt/test/cfi/lit.cfg

33 ↗

(On Diff #164783)

I was debugging with save-temps and I forgot to remove it.

Currently we need to pass -enable-vtable-interleaving to the linker because the front-end and back-end are controlled by two separate flags.

[rsmith]

clang/include/clang/Driver/CC1Options.td

409–410

We usually only expose the non-default flag in -cc1, so that there are no ordering concerns and we can determine whether a feature is enabled with just hasArg. Also, -fvtable-interleaving would seem like a more natural flag name to me.

clang/include/clang/Frontend/CodeGenOptions.h

274 ↗

(On Diff #164783)

Enable here seems redundant. Is thee a reason to declare this explicitly rather than adding it to CodeGenOptions.def?

clang/lib/CodeGen/ItaniumCXXABI.cpp

983–986

The __s here are atypical. If you want a name that can't collide with a name generated by the frontend, putting a period in it is the usual strategy. Also, a name prefix that indicates what this global is for would make the IR more readable and make this less likely to collide with other things. Maybe clang.vtable.index.<CLASS>.<OFFSET> or similar?

Also, there's no guarantee that RD->getNameAsString() produces something unique within the translation unit, so you'll need to make sure that name collisions are handled somehow (I *think* GlobalVariable already deals with this for you, but I'm not sure... however, I think its strategy is to add a number to the end of the mangling, which will lead to some quite peculiar results given that you already have a number there). It might be easier to base the name of this symbol on the mangled name of the class; then you could give the global variable the same linkage as the class, which might save your interleaving pass a little bit of work later on as the duplicates will already have been combined -- but that's up to you, and I don't have enough of the big picture here to give you advice.

997

Representing this offset as a ptrtoint on a global variable seems really strange. Is there really no better way to model this in IR? (I mean, if not, then carry on as you are, but this seems like a hack.)

1787

I still find the need for this confusing. Suppose we have:

struct A { virtual void f(); };
struct B { virtual void g(); virtual void h(); };
struct C : A, B {};

such that C has a naive vtable containing 7 elements {{offset, typeid, &A::f}, {offset, typeid, &B::f, &B::h}}, with address point indexes (0, 2) and (1, 2). Here we emit a gep inbounds @vtable, 0, 1, 2 to get the B-in-C vtable. Now, if this were instead emitted as gep inbounds (cast @vtable to i8*), 24, and the interleaving process were replacing @vtable with a non-inbounds GEP on its interleaved ubervtable, I could understand the need to drop inbounds here -- because the replacement for @vtable might be before the start of the ubervtable. But based on my reading of the algorithm description, that's not what happens: first we split @vtable into multiple individual vtables, and that splitting process presumably must identify these GEPs to figure out which slice of the vtable they need. That presumably either removes this GEP entirely, or leaves this being a correct and trivial inbounds GEP on a split-out piece of the vtable. (I'm imagining we effectively rewrite gep inbounds @vtable, 0, 1, 2 as @vtable-slice-2 as part of the splitting.) And then after interleaving, I'm imagining we replace all uses of @vtable-slice-2 with the appropriate address point in the ubervtable, which is again an inbounds GEP (but that would be a new GEP, unrelated to this one).

That is, if you need this change, it seems like that indicates a bug in the vtable splitter. But maybe I'm misunderstanding how this all fits together.

1862

Do you really need to change this to load the vptr at the "wrong" type and then cast it? (Could the cast, if necessary at all, live in EmitVTableTypeCheckedLoad instead?)

3776–3780

Shouldn't you also do this if the base is non-virtual? In that case,Offset is an offset within the class layout, not an index into the vtable.

[zhaomo]

In D51905#1231383, @vlad.tsyrklevich wrote:

This change causes all compiler-rt cfi tests to be UNSUPPORTED for me locally, do you have any idea why that might be? The lit changes don't make it immediately clear.

Not sure why that happened. How did you run the compiler-rt tests? Did you use ninja check-cfi?

[zhaomo]

Fixed issues pointed out by Peter and Vlad.

Richard:
I just saw comments but I will travel tomorrow and this weekend so I will reply to your comments when I get a chance.

[vlad.tsyrklevich]

In D51905#1234308, @zhaomo wrote:

In D51905#1231383, @vlad.tsyrklevich wrote:

This change causes all compiler-rt cfi tests to be UNSUPPORTED for me locally, do you have any idea why that might be? The lit changes don't make it immediately clear.

Not sure why that happened. How did you run the compiler-rt tests? Did you use ninja check-cfi?

I ran them with llvm-lit against the tests under test/cfi. It looks like something caused the default test suite to change where all of the tests were marked unsupported (the test suite ran without lld and my default linker otherwise is bfd so LTO was unsupported.) Not a problem with the change.

clang/lib/CodeGen/ItaniumCXXABI.cpp

949

Disregard that, I hadn't realized that was the style elsewhere.

[zhaomo]

Fixed bugs in the previous patch.

[zhaomo]

Addressed @rsmith's comments.

clang/include/clang/Driver/CC1Options.td

409–410

Changed it in the latest patch.

clang/include/clang/Frontend/CodeGenOptions.h

274 ↗

(On Diff #164783)

Changed it in the latest patch.

clang/lib/CodeGen/ItaniumCXXABI.cpp

983–986

The encoding of the name of such a global variable is just for debugging purposes. The back-end pass does not care about the name. Instead, it relies on the metadata associated with the global variable, and the metadata does not rely on RD->getNameAsString() . Also, the back-end pass does not care how many global variables associated with the same metadata.

I did not know that it is using periods is a convention in LLVM. I changed the name to clang.vtable.offset.<CLASS>.<OFFSET>.

997

The global variable we create for each referenced vtable offset is actually a zero-length array, and we use the address of the array as the placeholder for a referenced vtable offset. I was told that this is a common trick used in LLVM.

@pcc: Peter, would you provide more information here?

1787

GlobalSplit, the pass that splits vtables, relies on inrange instead of inbounds.

From the language reference:

If the inrange keyword is present before any index, loading from or storing to any pointer derived from the getelementptr has undefined behavior if the load or store would access memory outside of the bounds of the element selected by the index marked as inrange.

inbounds means something different. Again from the language reference:

If the inbounds keyword is present, the result value of the getelementptr is a poison value if the base pointer is not an in bounds address of an allocated object, or if any of the addresses that would be formed by successive addition of the offsets implied by the indices to the base address with infinitely precise signed arithmetic are not an in bounds address of that allocated object.

We need to drop inbounds because that the base pointer may not be in bounds address of an interleaved vtable.

Here is how an address point is calculated in an original vtable. original_address_pt = start_original_vtable + original_offset, where start_original_vtable is the address of the original vtable and original_offset is the offset of the address point in the original vtable.

Our interleaving pass calculates the address of this address point in the interleaved object. Let's call it new_address_pt and new_address_pt = start_interleaved_vtable + new_offset.
Then we replace start_original_vtable with (new_address_pt - original_offset). Because start_original_vtable is the base pointer of the gep, and new_offset may be smaller than original_offset, we have to drop inbounds.

I'm imagining we effectively rewrite gep inbounds @vtable, 0, 1, 2 as @vtable-slice-2

gep inbounds @vtable, 0, 1, 2 is rewritten to gep inbounds @vtable-slice-2, 0, 2 in your example.

1862

I have to because both branches of the if statement rely on OffsetConstant, which is a byte offset.

3776–3780

You are right. This bug is fixed in the latest patch.

[zhaomo]

In this new patch, I added a "level" field to type metadata when vtable interleaving is enabled.
Currently type metadata is a (byte offset, type) pair. However, suppose T1 and T2 are compatible with the same offset of a vtable, the interleaving pass cannot always determine which type is the less derived one because T1 and T2 may have the same number of compatible vtables. An observation is that types that are compatible with the same address point like T1 and T2 form a linear inheritance hierarchy. Based on this, I added a third field to the type metadata representing the level of the type in the linear type hierarchy. all-vtables' level value is 1 and any other type's level is greater than 1. The more derived a type is, the higher the level is.

clang/include/clang/AST/VTableBuilder.h

private:
   int64_t Value;
 };
 
 class VTableLayout {
 public:
   typedef std::pair<uint64_t, ThunkInfo> VTableThunkTy;
   struct AddressPointLocation {
     unsigned VTableIndex, AddressPointIndex;
+    // This flag indicates whether the vtable containing this address point
+    // has vcall offsets for the type associated with this address point.
+    bool HasVCallOffsets;
   };
   typedef llvm::DenseMap<BaseSubobject, AddressPointLocation>
       AddressPointsMapTy;
 
 private:
   // Stores the component indices of the first component of each virtual table in
   // the virtual table group. To save a little memory in the common case where
   // the vtable group contains a single vtable, an empty vector here represents

clang/include/clang/Basic/ABI.h

 /// C++ destructor types.
 enum CXXDtorType {
     Dtor_Deleting, ///< Deleting dtor
     Dtor_Complete, ///< Complete object dtor
     Dtor_Base,     ///< Base object dtor
     Dtor_Comdat    ///< The COMDAT used for dtors
 };
 
+class CXXRecordDecl;
+
 /// A return adjustment.
 struct ReturnAdjustment {
   /// The non-virtual adjustment from the derived object to its
   /// nearest virtual base.
   int64_t NonVirtual;
 
   /// Holds the ABI-specific information about the virtual return
   /// adjustment, if needed.
bool isEmpty() const {
       return Equals(Zero);
     }
 
     bool Less(const VirtualAdjustment &RHS) const {
       return memcmp(this, &RHS, sizeof(RHS)) < 0;
     }
   } Virtual;
 
-  ReturnAdjustment() : NonVirtual(0) {}
+  /// The type for which VirtualAdjustment is applied.
+  /// This field is only valid when the virtual return adjust is non-zero.
+  const CXXRecordDecl *VirtuallyAdjustedType;
+
+  ReturnAdjustment() : NonVirtual(0), VirtuallyAdjustedType(nullptr) {}
+  ReturnAdjustment(const CXXRecordDecl *Type)
+      : NonVirtual(0), VirtuallyAdjustedType(Type) {}
 
   bool isEmpty() const { return !NonVirtual && Virtual.isEmpty(); }
 
   friend bool operator==(const ReturnAdjustment &LHS,
                          const ReturnAdjustment &RHS) {
     return LHS.NonVirtual == RHS.NonVirtual && LHS.Virtual.Equals(RHS.Virtual);
   }
 
bool isEmpty() const {
       return Equals(Zero);
     }
 
     bool Less(const VirtualAdjustment &RHS) const {
       return memcmp(this, &RHS, sizeof(RHS)) < 0;
     }
   } Virtual;
 
-  ThisAdjustment() : NonVirtual(0) { }
+  /// The virtual base type for which VirtualAdjustment is applied.
+  /// This field is only valid when the virtual this adjustment is non-zero.
+  const CXXRecordDecl *VirtuallyAdjustedType;
+
+  ThisAdjustment() : NonVirtual(0), VirtuallyAdjustedType(nullptr) {}
+  ThisAdjustment(const CXXRecordDecl *Type)
+      : NonVirtual(0), VirtuallyAdjustedType(Type) {}
 
   bool isEmpty() const { return !NonVirtual && Virtual.isEmpty(); }
 
   friend bool operator==(const ThisAdjustment &LHS,
                          const ThisAdjustment &RHS) {
     return LHS.NonVirtual == RHS.NonVirtual && LHS.Virtual.Equals(RHS.Virtual);
   }
 

clang/include/clang/Basic/CodeGenOptions.def

 
 ENUM_CODEGENOPT(SignReturnAddress, SignReturnAddressScope, 2, None)
 ENUM_CODEGENOPT(SignReturnAddressKey, SignReturnAddressKeyValue, 1, AKey)
 CODEGENOPT(BranchTargetEnforcement, 1, 0)
 
 /// Whether to emit unused static constants.
 CODEGENOPT(KeepStaticConsts, 1, 0)
 
+/// Whether to interleave VTables.
+CODEGENOPT(VTableInterleaving, 1, 0)
+
 #undef CODEGENOPT
 #undef ENUM_CODEGENOPT
 #undef VALUE_CODEGENOPT
 

clang/include/clang/Driver/CC1Options.td

def fno_debug_pass_manager : Flag<["-"], "fno-debug-pass-manager">,
     HelpText<"Disables debug printing for the new pass manager">;
 // The driver option takes the key as a parameter to the -msign-return-address=
 // and -mbranch-protection= options, but CC1 has a separate option so we
 // don't have to parse the parameter twice.
 def msign_return_address_key_EQ : Joined<["-"], "msign-return-address-key=">,
     Values<"a_key,b_key">;
 def mbranch_target_enforce : Flag<["-"], "mbranch-target-enforce">;
 def fno_dllexport_inlines : Flag<["-"], "fno-dllexport-inlines">;
+def vtable_interleaving : Flag<["-"], "vtable-interleaving">,
+    HelpText<"Enable VTable interleaving">;
 
 //===----------------------------------------------------------------------===//
 // Dependency Output Options
 //===----------------------------------------------------------------------===//
 
 def sys_header_deps : Flag<["-"], "sys-header-deps">,
   HelpText<"Include system headers in dependency output">;
 def module_file_deps : Flag<["-"], "module-file-deps">,
   HelpText<"Include module files in dependency output">;
 def header_include_file : Separate<["-"], "header-include-file">,
   HelpText<"Filename (or -) to write header include output to">;
 def show_includes : Flag<["--"], "show-includes">,

clang/lib/AST/VTableBuilder.cpp

private:
   /// VBaseOffsetOffsets - Contains the offsets of the virtual base offsets,
   /// relative to the address point.
   VBaseOffsetOffsetsMapTy VBaseOffsetOffsets;
 
   /// FinalOverriders - The final overriders of the most derived class.
   /// (Can be null when we're not building a vtable of the most derived class).
   const FinalOverriders *Overriders;
 
+  /// VCallOffsetBases - The set of BaseSubobjects whose vtables contain vcall
+  /// offsets.
+  llvm::DenseSet<BaseSubobject> VCallOffsetBases;
+
   /// AddVCallAndVBaseOffsets - Add vcall offsets and vbase offsets for the
   /// given base subobject.
   void AddVCallAndVBaseOffsets(BaseSubobject Base, bool BaseIsVirtual,
                                CharUnits RealBaseOffset);
 
   /// AddVCallOffsets - Add vcall offsets for the given base subobject.
-  void AddVCallOffsets(BaseSubobject Base, CharUnits VBaseOffset);
+  /// VBase is the virtual base whose vtable contains the vcall offsets that are
+  /// collected during this function.
+  void AddVCallOffsets(BaseSubobject Base, CharUnits VBaseOffset,
+                       BaseSubobject VBase);
 
   /// AddVBaseOffsets - Add vbase offsets for the given class.
   void AddVBaseOffsets(const CXXRecordDecl *Base,
                        CharUnits OffsetInLayoutClass);
 
   /// getCurrentOffsetOffset - Get the current vcall or vbase offset offset in
   /// chars, relative to the vtable address point.
   CharUnits getCurrentOffsetOffset() const;
public:
   typedef VTableComponentVectorTy::const_reverse_iterator const_iterator;
   const_iterator components_begin() const { return Components.rbegin(); }
   const_iterator components_end() const { return Components.rend(); }
 
   const VCallOffsetMap &getVCallOffsets() const { return VCallOffsets; }
   const VBaseOffsetOffsetsMapTy &getVBaseOffsetOffsets() const {
     return VBaseOffsetOffsets;
   }
+
+  /// isVCallOffsetBase - Check whether Base's vtable has vcall offsets.
+  bool isVCallOffsetBase(BaseSubobject Base) {
+    return VCallOffsetBases.find(Base) != VCallOffsetBases.end();
+  }
 };
 
 void
 VCallAndVBaseOffsetBuilder::AddVCallAndVBaseOffsets(BaseSubobject Base,
                                                     bool BaseIsVirtual,
                                                     CharUnits RealBaseOffset) {
   const ASTRecordLayout &Layout = Context.getASTRecordLayout(Base.getBase());
 
AddVCallAndVBaseOffsets(
       BaseSubobject(PrimaryBase,PrimaryBaseOffset),
       PrimaryBaseIsVirtual, RealBaseOffset);
   }
 
   AddVBaseOffsets(Base.getBase(), RealBaseOffset);
 
   // We only want to add vcall offsets for virtual bases.
   if (BaseIsVirtual)
-    AddVCallOffsets(Base, RealBaseOffset);
+    AddVCallOffsets(Base, RealBaseOffset, Base);
 }
 
 CharUnits VCallAndVBaseOffsetBuilder::getCurrentOffsetOffset() const {
   // OffsetIndex is the index of this vcall or vbase offset, relative to the
   // vtable address point. (We subtract 3 to account for the information just
   // above the address point, the RTTI info, the offset to top, and the
   // vcall offset itself).
   int64_t OffsetIndex = -(int64_t)(3 + Components.size());
 
   CharUnits PointerWidth =
     Context.toCharUnitsFromBits(Context.getTargetInfo().getPointerWidth(0));
   CharUnits OffsetOffset = PointerWidth * OffsetIndex;
   return OffsetOffset;
 }
 
 void VCallAndVBaseOffsetBuilder::AddVCallOffsets(BaseSubobject Base,
-                                                 CharUnits VBaseOffset) {
+                                                 CharUnits VBaseOffset,
+                                                 BaseSubobject VBase) {
   const CXXRecordDecl *RD = Base.getBase();
   const ASTRecordLayout &Layout = Context.getASTRecordLayout(RD);
 
   const CXXRecordDecl *PrimaryBase = Layout.getPrimaryBase();
 
   // Handle the primary base first.
   // We only want to add vcall offsets if the base is non-virtual; a virtual
   // primary base will have its vcall and vbase offsets emitted already.
   if (PrimaryBase && !Layout.isPrimaryBaseVirtual()) {
     // Get the base offset of the primary base.
     assert(Layout.getBaseClassOffset(PrimaryBase).isZero() &&
            "Primary base should have a zero offset!");
 
     AddVCallOffsets(BaseSubobject(PrimaryBase, Base.getBaseOffset()),
-                    VBaseOffset);
+                    VBaseOffset, VBase);
   }
 
   // Add the vcall offsets.
   for (const auto *MD : RD->methods()) {
     if (!MD->isVirtual())
       continue;
     MD = MD->getCanonicalDecl();
 
     CharUnits OffsetOffset = getCurrentOffsetOffset();
 
+    // Now we know that VBase at least has one vcall offset in its vtable.
+    VCallOffsetBases.insert(VBase);
+
     // Don't add a vcall offset if we already have one for this member function
     // signature.
     if (!VCallOffsets.AddVCallOffset(MD, OffsetOffset))
       continue;
 
     CharUnits Offset = CharUnits::Zero();
 
     if (Overriders) {
for (const auto &B : RD->bases()) {
     const CXXRecordDecl *BaseDecl = B.getType()->getAsCXXRecordDecl();
     if (BaseDecl == PrimaryBase)
       continue;
 
     // Get the base offset of this base.
     CharUnits BaseOffset = Base.getBaseOffset() +
       Layout.getBaseClassOffset(BaseDecl);
 
-    AddVCallOffsets(BaseSubobject(BaseDecl, BaseOffset),
-                    VBaseOffset);
+    AddVCallOffsets(BaseSubobject(BaseDecl, BaseOffset), VBaseOffset, VBase);
   }
 }
 
 void
 VCallAndVBaseOffsetBuilder::AddVBaseOffsets(const CXXRecordDecl *RD,
                                             CharUnits OffsetInLayoutClass) {
   const ASTRecordLayout &LayoutClassLayout =
     Context.getASTRecordLayout(LayoutClass);
for (const auto &TI : VTableThunks) {
 
     if (MD->getParent() == MostDerivedClass)
       AddThunk(MD, Thunk);
   }
 }
 
 ReturnAdjustment
 ItaniumVTableBuilder::ComputeReturnAdjustment(BaseOffset Offset) {
-  ReturnAdjustment Adjustment;
+  ReturnAdjustment Adjustment(Offset.DerivedClass);
 
   if (!Offset.isEmpty()) {
     if (Offset.VirtualBase) {
       // Get the virtual base offset offset.
       if (Offset.DerivedClass == MostDerivedClass) {
         // We can get the offset offset directly from our map.
         Adjustment.Virtual.Itanium.VBaseOffsetOffset =
           VBaseOffsetOffsets.lookup(Offset.VirtualBase).getQuantity();
BaseSubobject OverriderBaseSubobject(Overrider.Method->getParent(),
                                        Overrider.Offset);
 
   // Compute the adjustment offset.
   BaseOffset Offset = ComputeThisAdjustmentBaseOffset(OverriddenBaseSubobject,
                                                       OverriderBaseSubobject);
   if (Offset.isEmpty())
     return ThisAdjustment();
 
-  ThisAdjustment Adjustment;
+  ThisAdjustment Adjustment(Offset.VirtualBase);
 
   if (Offset.VirtualBase) {
     // Get the vcall offset map for this virtual base.
     VCallOffsetMap &VCallOffsets = VCallOffsetsForVBases[Offset.VirtualBase];
 
     if (VCallOffsets.empty()) {
       // We don't have vcall offsets for this virtual base, go ahead and
       // build them.
if (RD == MostDerivedClass) {
     }
   }
 
   // Compute 'this' pointer adjustments.
   ComputeThisAdjustments();
 
   // Add all address points.
   while (true) {
-    AddressPoints.insert(
-        std::make_pair(BaseSubobject(RD, OffsetInLayoutClass),
-                       VTableLayout::AddressPointLocation{
-                           unsigned(VTableIndices.size() - 1),
-                           unsigned(AddressPoint - VTableIndex)}));
+    BaseSubobject CurBaseSubObj(RD, OffsetInLayoutClass);
+    bool HasVCallOffset = Builder.isVCallOffsetBase(CurBaseSubObj);
+    AddressPoints.insert(std::make_pair(
+        CurBaseSubObj,
+        VTableLayout::AddressPointLocation{unsigned(VTableIndices.size() - 1),
+                                           unsigned(AddressPoint - VTableIndex),
+                                           HasVCallOffset}));
 
     const ASTRecordLayout &Layout = Context.getASTRecordLayout(RD);
     const CXXRecordDecl *PrimaryBase = Layout.getPrimaryBase();
 
     if (!PrimaryBase)
       break;
 
     if (Layout.isPrimaryBaseVirtual()) {

clang/lib/CodeGen/CGCXXABI.h

 protected:
   CodeGenModule &CGM;
   std::unique_ptr<MangleContext> MangleCtx;
 
   CGCXXABI(CodeGenModule &CGM)
     : CGM(CGM), MangleCtx(CGM.getContext().createMangleContext()) {}
 
 protected:
   ImplicitParamDecl *getThisDecl(CodeGenFunction &CGF) {
     return CGF.CXXABIThisDecl;
   }
   llvm::Value *getThisValue(CodeGenFunction &CGF) {
     return CGF.CXXABIThisValue;
   }
   Address getThisAddress(CodeGenFunction &CGF) {
     return Address(CGF.CXXABIThisValue, CGF.CXXABIThisAlignment);
   }
public:
   /// this emits virtual table tables.  For the MSVC++ ABI, this emits virtual
   /// base tables.
   virtual void emitVirtualInheritanceTables(const CXXRecordDecl *RD) = 0;
 
   virtual bool exportThunk() = 0;
   virtual void setThunkLinkage(llvm::Function *Thunk, bool ForVTable,
                                GlobalDecl GD, bool ReturnAdjustment) = 0;
 
-  virtual llvm::Value *performThisAdjustment(CodeGenFunction &CGF,
-                                             Address This,
+  virtual llvm::Value *performThisAdjustment(CodeGenFunction &CGF, Address This,
                                              const ThisAdjustment &TA) = 0;
 
   virtual llvm::Value *performReturnAdjustment(CodeGenFunction &CGF,
                                                Address Ret,
                                                const ReturnAdjustment &RA) = 0;
 
   virtual void EmitReturnFromThunk(CodeGenFunction &CGF,
                                    RValue RV, QualType ResultType);

clang/lib/CodeGen/CGClass.cpp

if (!CGM.getCodeGenOpts().WholeProgramVTables ||
     return false;
 
   std::string TypeName = RD->getQualifiedNameAsString();
   return !getContext().getSanitizerBlacklist().isBlacklistedType(
       SanitizerKind::CFIVCall, TypeName);
 }
 
 llvm::Value *CodeGenFunction::EmitVTableTypeCheckedLoad(
-    const CXXRecordDecl *RD, llvm::Value *VTable, uint64_t VTableByteOffset) {
+    const CXXRecordDecl *RD, llvm::Value *CastedVTable, llvm::Type *ReturnType,
+    llvm::Value *VTableByteOffset) {
   SanitizerScope SanScope(this);
 
   EmitSanitizerStatReport(llvm::SanStat_CFI_VCall);
 
   llvm::Metadata *MD =
       CGM.CreateMetadataIdentifierForType(QualType(RD->getTypeForDecl(), 0));
   llvm::Value *TypeId = llvm::MetadataAsValue::get(CGM.getLLVMContext(), MD);
 
-  llvm::Value *CastedVTable = Builder.CreateBitCast(VTable, Int8PtrTy);
-  llvm::Value *CheckedLoad = Builder.CreateCall(
-      CGM.getIntrinsic(llvm::Intrinsic::type_checked_load),
-      {CastedVTable, llvm::ConstantInt::get(Int32Ty, VTableByteOffset),
-       TypeId});
+  llvm::Value *CheckedLoad =
+      Builder.CreateCall(CGM.getIntrinsic(llvm::Intrinsic::type_checked_load),
+                         {CastedVTable, VTableByteOffset, TypeId});
   llvm::Value *CheckResult = Builder.CreateExtractValue(CheckedLoad, 1);
 
   EmitCheck(std::make_pair(CheckResult, SanitizerKind::CFIVCall),
             SanitizerHandler::CFICheckFail, nullptr, nullptr);
 
-  return Builder.CreateBitCast(
-      Builder.CreateExtractValue(CheckedLoad, 0),
-      cast<llvm::PointerType>(VTable->getType())->getElementType());
+  return Builder.CreateBitCast(Builder.CreateExtractValue(CheckedLoad, 0),
+                               ReturnType);
 }
 
 void CodeGenFunction::EmitForwardingCallToLambda(
                                       const CXXMethodDecl *callOperator,
                                       CallArgList &callArgs) {
   // Get the address of the call operator.
   const CGFunctionInfo &calleeFnInfo =
     CGM.getTypes().arrangeCXXMethodDeclaration(callOperator);

clang/lib/CodeGen/CGVTables.cpp

 #include "clang/Basic/CodeGenOptions.h"
 #include "clang/CodeGen/CGFunctionInfo.h"
 #include "clang/CodeGen/ConstantInitBuilder.h"
 #include "llvm/IR/IntrinsicInst.h"
 #include "llvm/Support/Format.h"
 #include "llvm/Transforms/Utils/Cloning.h"
 #include <algorithm>
 #include <cstdio>
+#include <map>
+#include <set>
 
 using namespace clang;
 using namespace CodeGen;
 
 CodeGenVTables::CodeGenVTables(CodeGenModule &CGM)
     : CGM(CGM), VTContext(CGM.getContext().getVTableContext()) {}
 
 llvm::Constant *CodeGenModule::GetAddrOfThunk(StringRef Name, llvm::Type *FnTy,
if (NullCheckValue) {
 
     llvm::Value *IsNull = CGF.Builder.CreateIsNull(ReturnValue);
     CGF.Builder.CreateCondBr(IsNull, AdjustNull, AdjustNotNull);
     CGF.EmitBlock(AdjustNotNull);
   }
 
   auto ClassDecl = ResultType->getPointeeType()->getAsCXXRecordDecl();
   auto ClassAlign = CGF.CGM.getClassPointerAlignment(ClassDecl);
-  ReturnValue = CGF.CGM.getCXXABI().performReturnAdjustment(CGF,
-                                            Address(ReturnValue, ClassAlign),
-                                            Thunk.Return);
+  ReturnValue = CGF.CGM.getCXXABI().performReturnAdjustment(
+      CGF, Address(ReturnValue, ClassAlign), Thunk.Return);
 
   if (NullCheckValue) {
     CGF.Builder.CreateBr(AdjustEnd);
     CGF.EmitBlock(AdjustNull);
     CGF.Builder.CreateBr(AdjustEnd);
     CGF.EmitBlock(AdjustEnd);
 
     llvm::PHINode *PHI = CGF.Builder.CreatePHI(ReturnValue->getType(), 2);
void CodeGenFunction::EmitCallAndReturnForThunk(llvm::FunctionCallee Callee,
                                                 const ThunkInfo *Thunk,
                                                 bool IsUnprototyped) {
   assert(isa<CXXMethodDecl>(CurGD.getDecl()) &&
          "Please use a new CGF for this thunk");
   const CXXMethodDecl *MD = cast<CXXMethodDecl>(CurGD.getDecl());
 
   // Adjust the 'this' pointer if necessary
   llvm::Value *AdjustedThisPtr =
-    Thunk ? CGM.getCXXABI().performThisAdjustment(
-                          *this, LoadCXXThisAddress(), Thunk->This)
+      Thunk ? CGM.getCXXABI().performThisAdjustment(*this, LoadCXXThisAddress(),
+                                                    Thunk->This)
             : LoadCXXThis();
 
   // If perfect forwarding is required a variadic method, a method using
   // inalloca, or an unprototyped thunk, use musttail. Emit an error if this
   // thunk requires a return adjustment, since that is impossible with musttail.
   if (CurFnInfo->usesInAlloca() || CurFnInfo->isVariadic() || IsUnprototyped) {
     if (Thunk && !Thunk->Return.isEmpty()) {
       if (IsUnprototyped)
         CGM.ErrorUnsupported(

 void CodeGenModule::EmitVTableTypeMetadata(llvm::GlobalVariable *VTable,
                                            const VTableLayout &VTLayout) {
   if (!getCodeGenOpts().LTOUnit)
     return;
 
   CharUnits PointerWidth =
       Context.toCharUnitsFromBits(Context.getTargetInfo().getPointerWidth(0));
 
-  typedef std::pair<const CXXRecordDecl *, unsigned> AddressPoint;
-  std::vector<AddressPoint> AddressPoints;
+  typedef std::tuple<const CXXRecordDecl *, unsigned, bool> AddressPoint;
+
+  // We use a set first to filter out duplicates.
+  std::set<AddressPoint> AddressPointSet;
   for (auto &&AP : VTLayout.getAddressPoints())
-    AddressPoints.push_back(std::make_pair(
-        AP.first.getBase(), VTLayout.getVTableOffset(AP.second.VTableIndex) +
-                                AP.second.AddressPointIndex));
+    AddressPointSet.insert(
+        std::make_tuple(AP.first.getBase(),
+                        VTLayout.getVTableOffset(AP.second.VTableIndex) +
+                            AP.second.AddressPointIndex,
+                        AP.second.HasVCallOffsets));
+
+  std::vector<AddressPoint> AddressPoints(AddressPointSet.begin(),
+                                          AddressPointSet.end());
 
   // Sort the address points for determinism.
   llvm::sort(AddressPoints, [this](const AddressPoint &AP1,
                                    const AddressPoint &AP2) {
     if (&AP1 == &AP2)
       return false;
 
+    if (getCodeGenOpts().VTableInterleaving) {
+      // When vtable interleaving is enabled, sort the address points by:
+      // 1) the increasing order of the byte offset;
+      // 2) (for the address points at the same offset) the order from less
+      // derived to more derived.
+      if (std::get<1>(AP1) != std::get<1>(AP2))
+        return std::get<1>(AP1) < std::get<1>(AP2);
+      return std::get<0>(AP2)->isDerivedFrom(std::get<0>(AP1));
+    } else {
       std::string S1;
       llvm::raw_string_ostream O1(S1);
       getCXXABI().getMangleContext().mangleTypeName(
-        QualType(AP1.first->getTypeForDecl(), 0), O1);
+          QualType(std::get<0>(AP1)->getTypeForDecl(), 0), O1);
       O1.flush();
 
       std::string S2;
       llvm::raw_string_ostream O2(S2);
       getCXXABI().getMangleContext().mangleTypeName(
-        QualType(AP2.first->getTypeForDecl(), 0), O2);
+          QualType(std::get<0>(AP2)->getTypeForDecl(), 0), O2);
       O2.flush();
 
       if (S1 < S2)
         return true;
       if (S1 != S2)
         return false;
 
-    return AP1.second < AP2.second;
+      return std::get<1>(AP1) < std::get<1>(AP2);
+    }
   });
 
+  // When vtable interleaving is enabled, we need to keep inheritance
+  // relationships among types that have address points at the same address.
+  // Note that the types that have address points at the same offset in the same
+  // vtable are guaranteed to form a linear inheritance hierarchy. To retain
+  // inheritance hierarchy among types in IR, we append a "level" field to each
+  // piece of type metadata. For the type metadata associated with the same
+  // offset in the same vtable, the higher the level value is, the more derived
+  // the associated type is. The minimum value for level is 1.
+
+  // Map from each address point offset to the current level of that address.
+  std::map<unsigned, unsigned> CurLevelMap;
+
   ArrayRef<VTableComponent> Comps = VTLayout.vtable_components();
   for (auto AP : AddressPoints) {
+    const CXXRecordDecl *Type = std::get<0>(AP);
+    unsigned Offset = std::get<1>(AP);
+    bool HasVCallOffsets = std::get<2>(AP);
+
+    if (getCodeGenOpts().VTableInterleaving) {
+      if (CurLevelMap.find(Offset) == CurLevelMap.end())
+        CurLevelMap[Offset] = 2;
+
+      // Create type metadata for the address point.
+      AddVTableTypeMetadata(VTable, PointerWidth * Offset, Type, false,
+                            CurLevelMap[Offset]++);
+
+      // If the address point of some type T is in a vtable that contains vcall
+      // offsets for T, this address in the vtable is also an address point for
+      // the new type T.vcall_offset, which is intended to group all the vtables
+      // of T that contain vcall offsets together for interleaving.
+      if (HasVCallOffsets)
+        AddVTableTypeMetadata(VTable, PointerWidth * Offset, Type, true,
+                              CurLevelMap[Offset]++);
+
+      // When vtable interleaving is enabled, we don't emit type metadata for
+      // virtual member functions because the interleaving pass uses type
+      // metadata to determine address points of vtables.
+      continue;
+    }
+
     // Create type metadata for the address point.
-    AddVTableTypeMetadata(VTable, PointerWidth * AP.second, AP.first);
+    AddVTableTypeMetadata(VTable, PointerWidth * Offset, Type);
 
     // The class associated with each address point could also potentially be
     // used for indirect calls via a member function pointer, so we need to
     // annotate the address of each function pointer with the appropriate member
     // function pointer type.
     for (unsigned I = 0; I != Comps.size(); ++I) {
       if (Comps[I].getKind() != VTableComponent::CK_FunctionPointer)
         continue;
       llvm::Metadata *MD = CreateMetadataIdentifierForVirtualMemPtrType(
           Context.getMemberPointerType(
               Comps[I].getFunctionDecl()->getType(),
-              Context.getRecordType(AP.first).getTypePtr()));
-      VTable->addTypeMetadata((PointerWidth * I).getQuantity(), MD);
+              Context.getRecordType(std::get<0>(AP)).getTypePtr()));
+      VTable->addTypeMetadata((PointerWidth * I).getQuantity(), MD, 0);
     }
   }
 }

clang/lib/CodeGen/CodeGenFunction.h

public:
   void EmitTypeMetadataCodeForVCall(const CXXRecordDecl *RD,
                                     llvm::Value *VTable, SourceLocation Loc);
 
   /// Returns whether we should perform a type checked load when loading a
   /// virtual function for virtual calls to members of RD. This is generally
   /// true when both vcall CFI and whole-program-vtables are enabled.
   bool ShouldEmitVTableTypeCheckedLoad(const CXXRecordDecl *RD);
 
-  /// Emit a type checked load from the given vtable.
-  llvm::Value *EmitVTableTypeCheckedLoad(const CXXRecordDecl *RD, llvm::Value *VTable,
-                                         uint64_t VTableByteOffset);
+  /// Emit a type checked load from the given vtable. CastedVTable
+  /// is of int8ptr type and VTableByteOffset is a byte offset.
+  llvm::Value *EmitVTableTypeCheckedLoad(const CXXRecordDecl *RD,
+                                         llvm::Value *CastedVTable,
+                                         llvm::Type *EntryType,
+                                         llvm::Value *VTableByteOffset);
 
   /// EnterDtorCleanups - Enter the cleanups necessary to complete the
   /// given phase of destruction for a destructor.  The end result
   /// should call destructors on members and base classes in reverse
   /// order of their construction.
   void EnterDtorCleanups(const CXXDestructorDecl *Dtor, CXXDtorType Type);
 
   /// ShouldInstrumentFunction - Return true if the current function should be

clang/lib/CodeGen/CodeGenModule.h

private:
 
   /// Mapping from canonical types to their metadata identifiers. We need to
   /// maintain this mapping because identifiers may be formed from distinct
   /// MDNodes.
   typedef llvm::DenseMap<QualType, llvm::Metadata *> MetadataTypeMap;
   MetadataTypeMap MetadataIdMap;
   MetadataTypeMap VirtualMetadataIdMap;
   MetadataTypeMap GeneralizedMetadataIdMap;
+  MetadataTypeMap VCallOffsetMetadataIdMap;
 
 public:
   CodeGenModule(ASTContext &C, const HeaderSearchOptions &headersearchopts,
                 const PreprocessorOptions &ppopts,
                 const CodeGenOptions &CodeGenOpts, llvm::Module &M,
                 DiagnosticsEngine &Diags,
                 CoverageSourceInfo *CoverageInfo = nullptr);
 
public:
   /// calls via a member function pointer.
   llvm::Metadata *CreateMetadataIdentifierForVirtualMemPtrType(QualType T);
 
   /// Create a metadata identifier for the generalization of the given type.
   /// This may either be an MDString (for external identifiers) or a distinct
   /// unnamed MDNode (for internal identifiers).
   llvm::Metadata *CreateMetadataIdentifierGeneralized(QualType T);
 
+  /// Create a metadata identifier that is intended to be used to group
+  /// T's vtables containing vcall offsets together for vtable interleaving.
+  llvm::Metadata *CreateMetadataIdentifierForVCallOffsetType(QualType T);
+
   /// Create and attach type metadata to the given function.
   void CreateFunctionTypeMetadataForIcall(const FunctionDecl *FD,
                                           llvm::Function *F);
 
   /// Returns whether this module needs the "all-vtables" type identifier.
   bool NeedAllVtablesTypeId() const;
 
   /// Create and attach type metadata for the given vtable.
   void AddVTableTypeMetadata(llvm::GlobalVariable *VTable, CharUnits Offset,
-                             const CXXRecordDecl *RD);
+                             const CXXRecordDecl *RD,
+                             bool IsVCallOffsetType = false,
+                             unsigned Level = 0);
 
   /// Return a vector of most-base classes for RD. This is used to implement
   /// control flow integrity checks for member function pointers.
   ///
   /// A most-base class of a class C is defined as a recursive base class of C,
   /// including C itself, that does not have any bases.
   std::vector<const CXXRecordDecl *>
   getMostBaseClasses(const CXXRecordDecl *RD);

clang/lib/CodeGen/CodeGenModule.cpp

void CodeGenModule::SetLLVMFunctionAttributesForDefinition(const Decl *D,
   // These are only ever necessary on definitions; we're guaranteed that the
   // definition will be present in the LTO unit as a result of LTO visibility.
   auto *MD = dyn_cast<CXXMethodDecl>(D);
   if (MD && requiresMemberFunctionPointerTypeMetadata(*this, MD)) {
     for (const CXXRecordDecl *Base : getMostBaseClasses(MD->getParent())) {
       llvm::Metadata *Id =
           CreateMetadataIdentifierForType(Context.getMemberPointerType(
               MD->getType(), Context.getRecordType(Base).getTypePtr()));
-      F->addTypeMetadata(0, Id);
+      F->addTypeMetadata(0, Id, 0);
     }
   }
 }
 
 void CodeGenModule::SetCommonAttributes(GlobalDecl GD, llvm::GlobalValue *GV) {
   const Decl *D = GD.getDecl();
   if (dyn_cast_or_null<NamedDecl>(D))
     setGVProperties(GV, GD);
if (!LangOpts.Sanitize.has(SanitizerKind::CFIICall))
     return;
 
   // Non-static class methods are handled via vtable or member function pointer
   // checks elsewhere.
   if (isa<CXXMethodDecl>(FD) && !cast<CXXMethodDecl>(FD)->isStatic())
     return;
 
   llvm::Metadata *MD = CreateMetadataIdentifierForType(FD->getType());
-  F->addTypeMetadata(0, MD);
-  F->addTypeMetadata(0, CreateMetadataIdentifierGeneralized(FD->getType()));
+  F->addTypeMetadata(0, MD, 0);
+  F->addTypeMetadata(0, CreateMetadataIdentifierGeneralized(FD->getType()), 0);
 
   // Emit a hash-based bit set entry for cross-DSO calls.
   if (CodeGenOpts.SanitizeCfiCrossDso)
     if (auto CrossDsoTypeId = CreateCrossDsoCfiTypeId(MD))
-      F->addTypeMetadata(0, llvm::ConstantAsMetadata::get(CrossDsoTypeId));
+      F->addTypeMetadata(0, llvm::ConstantAsMetadata::get(CrossDsoTypeId), 0);
 }
 
 void CodeGenModule::SetFunctionAttributes(GlobalDecl GD, llvm::Function *F,
                                           bool IsIncompleteFunction,
                                           bool IsThunk) {
 
   if (llvm::Intrinsic::ID IID = F->getIntrinsicID()) {
     // If this is an intrinsic function, set the function's attributes
static QualType GeneralizeFunctionType(ASTContext &Ctx, QualType Ty) {
   llvm_unreachable("Encountered unknown FunctionType");
 }
 
 llvm::Metadata *CodeGenModule::CreateMetadataIdentifierGeneralized(QualType T) {
   return CreateMetadataIdentifierImpl(GeneralizeFunctionType(getContext(), T),
                                       GeneralizedMetadataIdMap, ".generalized");
 }
 
+llvm::Metadata *
+CodeGenModule::CreateMetadataIdentifierForVCallOffsetType(QualType T) {
+  return CreateMetadataIdentifierImpl(T, VCallOffsetMetadataIdMap,
+                                      ".vcall_offset");
+}
+
 /// Returns whether this module needs the "all-vtables" type identifier.
 bool CodeGenModule::NeedAllVtablesTypeId() const {
   // Returns true if at least one of vtable-based CFI checkers is enabled and
   // is not in the trapping mode.
   return ((LangOpts.Sanitize.has(SanitizerKind::CFIVCall) &&
            !CodeGenOpts.SanitizeTrap.has(SanitizerKind::CFIVCall)) ||
           (LangOpts.Sanitize.has(SanitizerKind::CFINVCall) &&
            !CodeGenOpts.SanitizeTrap.has(SanitizerKind::CFINVCall)) ||
           (LangOpts.Sanitize.has(SanitizerKind::CFIDerivedCast) &&
            !CodeGenOpts.SanitizeTrap.has(SanitizerKind::CFIDerivedCast)) ||
           (LangOpts.Sanitize.has(SanitizerKind::CFIUnrelatedCast) &&
            !CodeGenOpts.SanitizeTrap.has(SanitizerKind::CFIUnrelatedCast)));
 }
 
 void CodeGenModule::AddVTableTypeMetadata(llvm::GlobalVariable *VTable,
                                           CharUnits Offset,
-                                          const CXXRecordDecl *RD) {
+                                          const CXXRecordDecl *RD,
+                                          bool IsVCallOffsetType,
+                                          unsigned Level) {
+
+  // We only create type metadata with the level field if Level > 0,
+  // which indicates that vtable interleaving is enabled.
   llvm::Metadata *MD =
-      CreateMetadataIdentifierForType(QualType(RD->getTypeForDecl(), 0));
-  VTable->addTypeMetadata(Offset.getQuantity(), MD);
+      IsVCallOffsetType
+          ? CreateMetadataIdentifierForVCallOffsetType(
+                QualType(RD->getTypeForDecl(), 0))
+          : CreateMetadataIdentifierForType(QualType(RD->getTypeForDecl(), 0));
+  VTable->addTypeMetadata(Offset.getQuantity(), MD, Level);
 
   if (CodeGenOpts.SanitizeCfiCrossDso)
     if (auto CrossDsoTypeId = CreateCrossDsoCfiTypeId(MD))
       VTable->addTypeMetadata(Offset.getQuantity(),
-                              llvm::ConstantAsMetadata::get(CrossDsoTypeId));
+                              llvm::ConstantAsMetadata::get(CrossDsoTypeId),
+                              Level);
 
+  // all-vtables's level is always 1, which is the minimum value for level.
   if (NeedAllVtablesTypeId()) {
     llvm::Metadata *MD = llvm::MDString::get(getLLVMContext(), "all-vtables");
-    VTable->addTypeMetadata(Offset.getQuantity(), MD);
+    VTable->addTypeMetadata(Offset.getQuantity(), MD, Level > 0 ? 1 : 0);
   }
 }
 
 TargetAttr::ParsedTargetAttr CodeGenModule::filterFunctionTargetAttrs(const TargetAttr *TD) {
   assert(TD != nullptr);
   TargetAttr::ParsedTargetAttr ParsedAttr = TD->parse();
 
   ParsedAttr.Features.erase(

clang/lib/CodeGen/ItaniumCXXABI.cpp

 
 #include "CGCXXABI.h"
 #include "CGCleanup.h"
 #include "CGRecordLayout.h"
 #include "CGVTables.h"
 #include "CodeGenFunction.h"
 #include "CodeGenModule.h"
 #include "TargetInfo.h"
-#include "clang/CodeGen/ConstantInitBuilder.h"
 #include "clang/AST/Mangle.h"
-#include "clang/AST/Type.h"
 #include "clang/AST/StmtCXX.h"
+#include "clang/AST/Type.h"
+#include "clang/CodeGen/ConstantInitBuilder.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/GlobalValue.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/Intrinsics.h"
+#include "llvm/IR/Metadata.h"
 #include "llvm/IR/Value.h"
 #include "llvm/Support/ScopedPrinter.h"
 
 using namespace clang;
 using namespace CodeGen;
 
 namespace {
 class ItaniumCXXABI : public CodeGen::CGCXXABI {
   /// VTables - All the vtables which have been defined.
   llvm::DenseMap<const CXXRecordDecl *, llvm::GlobalVariable *> VTables;
+  /// Map each entry group to the corresponding offset global variable.
+  /// This map is needed for vtable interleaving.
+  llvm::DenseMap<std::pair<llvm::Metadata *, int64_t>, llvm::GlobalVariable *>
+      OffsetMap;
 
   /// All the thread wrapper functions that have been used.
   llvm::SmallVector<std::pair<const VarDecl *, llvm::Function *>, 8>
       ThreadWrappers;
 
 protected:
   bool UseARMMethodPtrABI;
   bool UseARMGuardVarABI;
   bool Use32BitVTableOffsetABI;
 
   ItaniumMangleContext &getMangleContext() {
     return cast<ItaniumMangleContext>(CodeGen::CGCXXABI::getMangleContext());
   }
 
 public:
-  ItaniumCXXABI(CodeGen::CodeGenModule &CGM,
-                bool UseARMMethodPtrABI = false,
-                bool UseARMGuardVarABI = false) :
-    CGCXXABI(CGM), UseARMMethodPtrABI(UseARMMethodPtrABI),
-    UseARMGuardVarABI(UseARMGuardVarABI),
-    Use32BitVTableOffsetABI(false) { }
+  ItaniumCXXABI(CodeGen::CodeGenModule &CGM, bool UseARMMethodPtrABI = false,
+                bool UseARMGuardVarABI = false)
+      : CGCXXABI(CGM), UseARMMethodPtrABI(UseARMMethodPtrABI),
+        UseARMGuardVarABI(UseARMGuardVarABI), Use32BitVTableOffsetABI(false) {}
 
   bool classifyReturnType(CGFunctionInfo &FI) const override;
 
   RecordArgABI getRecordArgABI(const CXXRecordDecl *RD) const override {
     // If C++ prohibits us from making a copy, pass by address.
     if (!RD->canPassInRegisters())
       return RAA_Indirect;
     return RAA_Default;
void setThunkLinkage(llvm::Function *Thunk, bool ForVTable, GlobalDecl GD,
     // linkage together with vtables when needed.
     if (ForVTable && !Thunk->hasLocalLinkage())
       Thunk->setLinkage(llvm::GlobalValue::AvailableExternallyLinkage);
     CGM.setGVProperties(Thunk, GD);
   }
 
   bool exportThunk() override { return true; }
 
+  // VirtuallyAdjustedType is used for vtable interleaving. It is the type
+  // of the vtable in which the vcall offset entry (if it is a this adjustment)
+  // or the virtual base entry (if it is a return adjustment) is defined. For a
+  // return adjustment, VirtuallyAdjustedType is the derived type; For a this
+  // adjustment, VirtuallyAdjustedType is the virtual base type.
+  llvm::Value *performTypeAdjustment(CodeGenFunction &CGF, Address InitialPtr,
+                                     int64_t NonVirtualAdjustment,
+                                     const CXXRecordDecl *VirtuallyAdjustedType,
+                                     int64_t VirtualAdjustment,
+                                     bool IsReturnAdjustment);
   llvm::Value *performThisAdjustment(CodeGenFunction &CGF, Address This,
                                      const ThisAdjustment &TA) override;
 
   llvm::Value *performReturnAdjustment(CodeGenFunction &CGF, Address Ret,
                                        const ReturnAdjustment &RA) override;
 
   size_t getSrcArgforCopyCtor(const CXXConstructorDecl *,
                               FunctionArgList &Args) const override {
public:
   bool usesThreadWrapperFunction(const VarDecl *VD) const override {
     return !isEmittedWithConstantInitializer(VD);
   }
   LValue EmitThreadLocalVarDeclLValue(CodeGenFunction &CGF, const VarDecl *VD,
                                       QualType LValType) override;
 
   bool NeedsVTTParameter(GlobalDecl GD) override;
 
+  bool shouldInterleaveVTables(const CXXRecordDecl *RD);
+  llvm::Constant *getOffsetConstant(const CXXRecordDecl *RD, int64_t Offset,
+                                    llvm::Type *OffsetType,
+                                    bool AccessVCallOffset = false);
+  llvm::Value *getVTableEntryPointer(CodeGenFunction &CGF,
+                                     const CXXRecordDecl *RD,
+                                     bool AccessVCallOffset,
+                                     llvm::Value *VTable, int64_t Offset,
+                                     const llvm::Twine &Name = "");
+
   /**************************** RTTI Uniqueness ******************************/
 
 protected:
   /// Returns true if the ABI requires RTTI type_info objects to be unique
   /// across a program.
   virtual bool shouldRTTIBeUnique() const { return true; }
 
 public:
CodeGen::CGCXXABI *CodeGen::CreateItaniumCXXABI(CodeGenModule &CGM) {
   case TargetCXXABI::GenericItanium:
     if (CGM.getContext().getTargetInfo().getTriple().getArch()
         == llvm::Triple::le32) {
       // For PNaCl, use ARM-style method pointers so that PNaCl code
       // does not assume anything about the alignment of function
       // pointers.
       return new ItaniumCXXABI(CGM, /*UseARMMethodPtrABI=*/true);
     }
-    return new ItaniumCXXABI(CGM);
+    return new ItaniumCXXABI(CGM, /* UseARMMethodPtrABI = */ false,
+                             /* UseARMGuardVarABI = */ false);
 
   case TargetCXXABI::Microsoft:
     llvm_unreachable("Microsoft ABI is not Itanium-based");
   }
   llvm_unreachable("bad ABI kind");
 }
 
 llvm::Type *
CGCallee ItaniumCXXABI::EmitLoadOfMemberFunctionPointer(
   // Compute the address of the virtual function pointer.
   llvm::Value *VFPAddr = Builder.CreateGEP(VTable, VTableOffset);
 
   // Check the address of the function pointer if CFI on member function
   // pointers is enabled.
   llvm::Constant *CheckSourceLocation;
   llvm::Constant *CheckTypeDesc;
   bool ShouldEmitCFICheck = CGF.SanOpts.has(SanitizerKind::CFIMFCall) &&
-                            CGM.HasHiddenLTOVisibility(RD);
+                            CGM.HasHiddenLTOVisibility(RD) &&
+                            !CGM.getCodeGenOpts().VTableInterleaving;
   if (ShouldEmitCFICheck) {
     CodeGenFunction::SanitizerScope SanScope(&CGF);
 
     CheckSourceLocation = CGF.EmitCheckSourceLocation(E->getBeginLoc());
     CheckTypeDesc = CGF.EmitCheckTypeDescriptor(QualType(MPT, 0));
     llvm::Constant *StaticData[] = {
         llvm::ConstantInt::get(CGF.Int8Ty, CodeGenFunction::CFITCK_VMFCall),
         CheckSourceLocation,
ItaniumCXXABI::EmitMemberDataPointer(const MemberPointerType *MPT,
   return llvm::ConstantInt::get(CGM.PtrDiffTy, offset.getQuantity());
 }
 
 llvm::Constant *
 ItaniumCXXABI::EmitMemberFunctionPointer(const CXXMethodDecl *MD) {
   return BuildMemberPointer(MD, CharUnits::Zero());
 }
 
+/// We should only interleave vtables when the type has cfi-vcall enabled, has
+/// the hidden LTO visibility, and VTableInterleaving is set.
+bool ItaniumCXXABI::shouldInterleaveVTables(const CXXRecordDecl *RD) {
+  // For now vtable interleaving relies on cfi-vcall.
+  SanitizerMask M = SanitizerKind::CFIVCall;
+  std::string TypeName = RD->getQualifiedNameAsString();
+  if (CGM.getContext().getSanitizerBlacklist().isBlacklistedType(M, TypeName))
+    return false;
+
+  return CGM.HasHiddenLTOVisibility(RD) &&
+         CGM.getCodeGenOpts().VTableInterleaving;
+}
+
+/// When interleaving is disabled, this function simply returns
+/// a constant of the offset. Otherwise, it create or returns the
+/// offset placeholder of the specified type for the entry group identified
+/// by the pair (TypeId, Offset). An entry group is a set of vtable entries
+/// that must have the same offset from their corresponding address points.
+/// To interleave vtables, we replace offsets into vtables with offset
+/// placeholders, which will be replaced with real offsets once the interleaved
+/// layout is decided.
+llvm::Constant *ItaniumCXXABI::getOffsetConstant(const CXXRecordDecl *RD,
+                                                 int64_t Offset,
+                                                 llvm::Type *OffsetType,
+                                                 bool IsVCallOffset) {
+  if (!shouldInterleaveVTables(RD))
+    return llvm::ConstantInt::get(OffsetType, Offset, true);
+
+  // AccessVCallOffset indicates whether we are getting the offset to a vcall
+  // offset or not. If it is true, we need to use the type id of the added vcall
+  // offset subtype of RD whose vtables are ones containing vcall offsets of RD.
+  llvm::Metadata *TypeId = IsVCallOffset
+                               ? CGM.CreateMetadataIdentifierForVCallOffsetType(
+                                     QualType(RD->getTypeForDecl(), 0))
+                               : CGM.CreateMetadataIdentifierForType(
+                                     QualType(RD->getTypeForDecl(), 0));
+  std::pair<llvm::Metadata *, int64_t> P = std::make_pair(TypeId, Offset);
+
+  // The offset global variable for the entry group.
+  llvm::GlobalVariable *OffsetGV;
+  if (OffsetMap.find(P) != OffsetMap.end())
+    OffsetGV = OffsetMap[P];
+  else {
+    // If we haven't seen this offset, create an offset global variable for it.
+    llvm::Type *GVType = llvm::ArrayType::get(CGM.Int32Ty, 0);
+    // The name of a offset global variable has the format "__[type
+    // id]$[offset]".
+    std::string Name =
+        "__" + RD->getNameAsString() + "$" + std::to_string(Offset);
+    OffsetGV = new llvm::GlobalVariable(
+        CGM.getModule(), GVType,
+        /*Constant=*/true, llvm::GlobalValue::InternalLinkage,
+        llvm::Constant::getNullValue(GVType), Name);
+
+    // Add offset.type metadata to the newly created offset global variable.
+    OffsetGV->addMetadata(
+        llvm::LLVMContext::MD_offset_type,
+        *llvm::MDTuple::get(
+            CGM.getLLVMContext(),
+            {TypeId, llvm::ConstantAsMetadata::get(
+                         llvm::ConstantInt::get(CGM.Int64Ty, Offset))}));
+
+    OffsetMap[P] = OffsetGV;
+  }
+  return llvm::ConstantExpr::getPtrToInt(OffsetGV, OffsetType);
+}
+
+/// Return a pointer to the specified vtable entry. When vtable interleaving
+/// is enabled, the calculation of this pointer is based on a vtable offset
+/// placeholder, which will be replaced when the interleaved layout is decided.
+llvm::Value *ItaniumCXXABI::getVTableEntryPointer(
+    CodeGenFunction &CGF, const CXXRecordDecl *RD, bool AccessVCallOffset,
+    llvm::Value *VTablePtr, int64_t Offset, const llvm::Twine &Name) {
+  llvm::Constant *OffsetConstant =
+      getOffsetConstant(RD, Offset, CGM.Int64Ty, AccessVCallOffset);
+  // Use the placeholder to calculate the pointer to the entry.
+  return CGF.Builder.CreateInBoundsGEP(nullptr, VTablePtr, OffsetConstant,
+                                       Name);
+}
+
 llvm::Constant *ItaniumCXXABI::BuildMemberPointer(const CXXMethodDecl *MD,
                                                   CharUnits ThisAdjustment) {
   assert(MD->isInstance() && "Member function must not be static!");
 
   CodeGenTypes &Types = CGM.getTypes();
 
   // Get the function pointer (or index if this is a virtual function).
   llvm::Constant *MemPtr[2];
   if (MD->isVirtual()) {
     uint64_t Index = CGM.getItaniumVTableContext().getMethodVTableIndex(MD);
 
     const ASTContext &Context = getContext();
     CharUnits PointerWidth =
       Context.toCharUnitsFromBits(Context.getTargetInfo().getPointerWidth(0));
     uint64_t VTableOffset = (Index * PointerWidth.getQuantity());
+    const CXXRecordDecl *RD = MD->getParent();
 
     if (UseARMMethodPtrABI) {
       // ARM C++ ABI 3.2.1:
       //   This ABI specifies that adj contains twice the this
       //   adjustment, plus 1 if the member function is virtual. The
       //   least significant bit of adj then makes exactly the same
       //   discrimination as the least significant bit of ptr does for
       //   Itanium.
-      MemPtr[0] = llvm::ConstantInt::get(CGM.PtrDiffTy, VTableOffset);
+      MemPtr[0] = getOffsetConstant(RD, (int64_t)VTableOffset, CGM.PtrDiffTy);
       MemPtr[1] = llvm::ConstantInt::get(CGM.PtrDiffTy,
                                          2 * ThisAdjustment.getQuantity() + 1);
     } else {
       // Itanium C++ ABI 2.3:
       //   For a virtual function, [the pointer field] is 1 plus the
       //   virtual table offset (in bytes) of the function,
       //   represented as a ptrdiff_t.
+      if (shouldInterleaveVTables(RD)) {
+        llvm::Constant *Placeholder =
+            getOffsetConstant(RD, (int64_t)VTableOffset, CGM.PtrDiffTy);
+        llvm::Constant *ConstantOne = llvm::ConstantInt::get(CGM.PtrDiffTy, 1);
+        MemPtr[0] = llvm::ConstantExpr::getAdd(Placeholder, ConstantOne);
+      } else
         MemPtr[0] = llvm::ConstantInt::get(CGM.PtrDiffTy, VTableOffset + 1);
       MemPtr[1] = llvm::ConstantInt::get(CGM.PtrDiffTy,
                                          ThisAdjustment.getQuantity());
     }
   } else {
     const FunctionProtoType *FPT = MD->getType()->castAs<FunctionProtoType>();
     llvm::Type *Ty;
     // Check whether the function has a computable LLVM signature.
     if (Types.isFuncTypeConvertible(FPT)) {
ItaniumCXXABI::GetVirtualBaseClassOffset(CodeGenFunction &CGF,
                                          Address This,
                                          const CXXRecordDecl *ClassDecl,
                                          const CXXRecordDecl *BaseClassDecl) {
   llvm::Value *VTablePtr = CGF.GetVTablePtr(This, CGM.Int8PtrTy, ClassDecl);
   CharUnits VBaseOffsetOffset =
       CGM.getItaniumVTableContext().getVirtualBaseOffsetOffset(ClassDecl,
                                                                BaseClassDecl);
 
-  llvm::Value *VBaseOffsetPtr =
-    CGF.Builder.CreateConstGEP1_64(VTablePtr, VBaseOffsetOffset.getQuantity(),
-                                   "vbase.offset.ptr");
+  llvm::Value *VBaseOffsetPtr = getVTableEntryPointer(
+      CGF, ClassDecl, false, VTablePtr, VBaseOffsetOffset.getQuantity(),
   VBaseOffsetPtr = CGF.Builder.CreateBitCast(VBaseOffsetPtr,
                                              CGM.PtrDiffTy->getPointerTo());
 
   llvm::Value *VBaseOffset =
     CGF.Builder.CreateAlignedLoad(VBaseOffsetPtr, CGF.getPointerAlign(),
                                   "vbase.offset");
 
   return VBaseOffset;
VTableLayout::AddressPointLocation AddressPoint =
           .getVTableLayout(VTableClass)
           .getAddressPoint(Base);
   llvm::Value *Indices[] = {
     llvm::ConstantInt::get(CGM.Int32Ty, 0),
     llvm::ConstantInt::get(CGM.Int32Ty, AddressPoint.VTableIndex),
     llvm::ConstantInt::get(CGM.Int32Ty, AddressPoint.AddressPointIndex),
   };
 
+  // The address point calculated by vtable_start + offset where vtable_start is
+  // the base address of the gep. When interleaving is enabled, we replace
+  // vtable_start with the new address point in the interleaved layout minus
+  // offset, which may not be an in bound address of the object.
+  bool InBounds = shouldInterleaveVTables(VTableClass) ? false : true;
   return llvm::ConstantExpr::getGetElementPtr(VTable->getValueType(), VTable,
-                                              Indices, /*InBounds=*/true,
+                                              Indices, /*InBounds=*/InBounds,
                                               /*InRangeIndex=*/1);
 }
 
 llvm::Value *ItaniumCXXABI::getVTableAddressPointInStructorWithVTT(
     CodeGenFunction &CGF, const CXXRecordDecl *VTableClass, BaseSubobject Base,
     const CXXRecordDecl *NearestVBase) {
   assert((Base.getBase()->getNumVBases() || NearestVBase != nullptr) &&
          NeedsVTTParameter(CGF.CurGD) && "This class doesn't have VTT");
llvm::GlobalVariable *ItaniumCXXABI::getAddrOfVTable(const CXXRecordDecl *RD,
   return VTable;
 }
 
 CGCallee ItaniumCXXABI::getVirtualFunctionPointer(CodeGenFunction &CGF,
                                                   GlobalDecl GD,
                                                   Address This,
                                                   llvm::Type *Ty,
                                                   SourceLocation Loc) {
-  Ty = Ty->getPointerTo()->getPointerTo();
   auto *MethodDecl = cast<CXXMethodDecl>(GD.getDecl());
-  llvm::Value *VTable = CGF.GetVTablePtr(This, Ty, MethodDecl->getParent());
+  auto *RecordDecl = MethodDecl->getParent();
+  llvm::Value *VTable = CGF.GetVTablePtr(This, CGM.Int8PtrTy, RecordDecl);
 
   uint64_t VTableIndex = CGM.getItaniumVTableContext().getMethodVTableIndex(GD);
+  int64_t ByteOffset =
+      VTableIndex * CGM.getContext().getTargetInfo().getPointerWidth(0) / 8;
+  llvm::Value *OffsetConstant =
+      getOffsetConstant(RecordDecl, ByteOffset, CGM.Int32Ty);
   llvm::Value *VFunc;
-  if (CGF.ShouldEmitVTableTypeCheckedLoad(MethodDecl->getParent())) {
-    VFunc = CGF.EmitVTableTypeCheckedLoad(
-        MethodDecl->getParent(), VTable,
-        VTableIndex * CGM.getContext().getTargetInfo().getPointerWidth(0) / 8);
-  } else {
-    CGF.EmitTypeMetadataCodeForVCall(MethodDecl->getParent(), VTable, Loc);
-
+  if (CGF.ShouldEmitVTableTypeCheckedLoad(RecordDecl))
+    VFunc = CGF.EmitVTableTypeCheckedLoad(RecordDecl, VTable,
+                                          Ty->getPointerTo(), OffsetConstant);
+  else {
+    CGF.EmitTypeMetadataCodeForVCall(RecordDecl, VTable, Loc);
     llvm::Value *VFuncPtr =
-        CGF.Builder.CreateConstInBoundsGEP1_64(VTable, VTableIndex, "vfn");
+        CGF.Builder.CreateInBoundsGEP(nullptr, VTable, OffsetConstant, "vfn");
+    VFuncPtr =
+        CGF.Builder.CreateBitCast(VFuncPtr, Ty->getPointerTo()->getPointerTo());
     auto *VFuncLoad =
         CGF.Builder.CreateAlignedLoad(VFuncPtr, CGF.getPointerAlign());
 
     // Add !invariant.load md to virtual function load to indicate that
     // function didn't change inside vtable.
     // It's safe to add it without -fstrict-vtable-pointers, but it would not
     // help in devirtualization because it will only matter if we will have 2
     // the same virtual function loads from the same vtable load, which won't
for (const auto &B : RD->vbases()) {
     if (!BRD->isDynamicClass())
       continue;
     if (!canSpeculativelyEmitVTableAsBaseClass(BRD))
       return false;
   }
 
   return true;
 }
-static llvm::Value *performTypeAdjustment(CodeGenFunction &CGF,
-                                          Address InitialPtr,
-                                          int64_t NonVirtualAdjustment,
-                                          int64_t VirtualAdjustment,
-                                          bool IsReturnAdjustment) {
+
+llvm::Value *ItaniumCXXABI::performTypeAdjustment(
+    CodeGenFunction &CGF, Address InitialPtr, int64_t NonVirtualAdjustment,
+    const CXXRecordDecl *VirtuallyAdjustedType, int64_t VirtualAdjustment,
   if (!NonVirtualAdjustment && !VirtualAdjustment)
     return InitialPtr.getPointer();
 
   Address V = CGF.Builder.CreateElementBitCast(InitialPtr, CGF.Int8Ty);
 
   // In a base-to-derived cast, the non-virtual adjustment is applied first.
   if (NonVirtualAdjustment && !IsReturnAdjustment) {
     V = CGF.Builder.CreateConstInBoundsByteGEP(V,
                               CharUnits::fromQuantity(NonVirtualAdjustment));
   }
 
   // Perform the virtual adjustment if we have one.
   llvm::Value *ResultPtr;
   if (VirtualAdjustment) {
+    assert(VirtuallyAdjustedType != nullptr &&
+           "The type being virtually adjusted is not available");
+
     llvm::Type *PtrDiffTy =
         CGF.ConvertType(CGF.getContext().getPointerDiffType());
 
     Address VTablePtrPtr = CGF.Builder.CreateElementBitCast(V, CGF.Int8PtrTy);
     llvm::Value *VTablePtr = CGF.Builder.CreateLoad(VTablePtrPtr);
-
+    // If there is virtual adjustment and this is a this adjustment, we know
+    // that we read the vcall offset from the vtable.
     llvm::Value *OffsetPtr =
-        CGF.Builder.CreateConstInBoundsGEP1_64(VTablePtr, VirtualAdjustment);
-
+        getVTableEntryPointer(CGF, VirtuallyAdjustedType, !IsReturnAdjustment,
+                              VTablePtr, VirtualAdjustment);
     OffsetPtr = CGF.Builder.CreateBitCast(OffsetPtr, PtrDiffTy->getPointerTo());
 
     // Load the adjustment offset from the vtable.
     llvm::Value *Offset =
       CGF.Builder.CreateAlignedLoad(OffsetPtr, CGF.getPointerAlign());
 
     // Adjust our pointer.
     ResultPtr = CGF.Builder.CreateInBoundsGEP(V.getPointer(), Offset);
llvm::Value *ItaniumCXXABI::performTypeAdjustment(
   // Cast back to the original type.
   return CGF.Builder.CreateBitCast(ResultPtr, InitialPtr.getType());
 }
 
 llvm::Value *ItaniumCXXABI::performThisAdjustment(CodeGenFunction &CGF,
                                                   Address This,
                                                   const ThisAdjustment &TA) {
   return performTypeAdjustment(CGF, This, TA.NonVirtual,
+                               TA.VirtuallyAdjustedType,
                                TA.Virtual.Itanium.VCallOffsetOffset,
                                /*IsReturnAdjustment=*/false);
 }
 
 llvm::Value *
 ItaniumCXXABI::performReturnAdjustment(CodeGenFunction &CGF, Address Ret,
                                        const ReturnAdjustment &RA) {
   return performTypeAdjustment(CGF, Ret, RA.NonVirtual,
+                               RA.VirtuallyAdjustedType,
                                RA.Virtual.Itanium.VBaseOffsetOffset,
                                /*IsReturnAdjustment=*/true);
 }
 
 void ARMCXXABI::EmitReturnFromThunk(CodeGenFunction &CGF,
                                     RValue RV, QualType ResultType) {
   if (!isa<CXXDestructorDecl>(CGF.CurGD.getDecl()))
     return ItaniumCXXABI::EmitReturnFromThunk(CGF, RV, ResultType);
bool ItaniumCXXABI::NeedsVTTParameter(GlobalDecl GD) {
 
   return false;
 }
 
 namespace {
 class ItaniumRTTIBuilder {
   CodeGenModule &CGM;  // Per-module state.
   llvm::LLVMContext &VMContext;
-  const ItaniumCXXABI &CXXABI;  // Per-module state.
+  ItaniumCXXABI &CXXABI; // Per-module state.
 
   /// Fields - The fields of the RTTI descriptor currently being built.
   SmallVector<llvm::Constant *, 16> Fields;
 
   /// GetAddrOfTypeName - Returns the mangled type name of the given type.
   llvm::GlobalVariable *
   GetAddrOfTypeName(QualType Ty, llvm::GlobalVariable::LinkageTypes Linkage);
 
   /// GetAddrOfExternalRTTIDescriptor - Returns the constant for the RTTI
   /// descriptor of the given type.
   llvm::Constant *GetAddrOfExternalRTTIDescriptor(QualType Ty);
 
   /// BuildVTablePointer - Build the vtable pointer for the given type.
   void BuildVTablePointer(const Type *Ty);
 
   /// BuildSIClassTypeInfo - Build an abi::__si_class_type_info, used for single
   /// inheritance, according to the Itanium C++ ABI, 2.9.5p6b.
   void BuildSIClassTypeInfo(const CXXRecordDecl *RD);
 
+  /// GetOffsetFlag - Build a constant representing __offset_flags.
+  /// When vtable interleaving is enabled and Offset is an virtual base offset,
+  /// this constant will be constructed from the corresponding offset
+  /// placeholder.
+  llvm::Constant *GetOffsetFlag(const CXXRecordDecl *RD, int64_t Offset,
+                                uint64_t LowerBits, llvm::Type *OffsetFlagsTy);
+
   /// BuildVMIClassTypeInfo - Build an abi::__vmi_class_type_info, used for
   /// classes with bases that do not satisfy the abi::__si_class_type_info
   /// constraints, according ti the Itanium C++ ABI, 2.9.5p5c.
   void BuildVMIClassTypeInfo(const CXXRecordDecl *RD);
 
   /// BuildPointerTypeInfo - Build an abi::__pointer_type_info struct, used
   /// for pointer types.
   void BuildPointerTypeInfo(QualType PointeeTy);
 
   /// BuildObjCObjectTypeInfo - Build the appropriate kind of
   /// type_info for an object type.
   void BuildObjCObjectTypeInfo(const ObjCObjectType *Ty);
 
   /// BuildPointerToMemberTypeInfo - Build an abi::__pointer_to_member_type_info
   /// struct, used for member pointer types.
   void BuildPointerToMemberTypeInfo(const MemberPointerType *Ty);
 
 public:
-  ItaniumRTTIBuilder(const ItaniumCXXABI &ABI)
+  ItaniumRTTIBuilder(ItaniumCXXABI &ABI)
       : CGM(ABI.CGM), VMContext(CGM.getModule().getContext()), CXXABI(ABI) {}
 
   // Pointer type info flags.
   enum {
     /// PTI_Const - Type has const qualifier.
     PTI_Const = 0x1,
 
     /// PTI_Volatile - Type has volatile qualifier.
static const char * const ClassTypeInfo =
     "_ZTVN10__cxxabiv117__class_type_infoE";
   // abi::__si_class_type_info.
   static const char * const SIClassTypeInfo =
     "_ZTVN10__cxxabiv120__si_class_type_infoE";
   // abi::__vmi_class_type_info.
   static const char * const VMIClassTypeInfo =
     "_ZTVN10__cxxabiv121__vmi_class_type_infoE";
 
+  llvm::Type *VTableTy1 =
+      llvm::StructType::get(llvm::ArrayType::get(CGM.Int8PtrTy, 7));
+  llvm::Type *VTableTy2 =
+      llvm::StructType::get(llvm::ArrayType::get(CGM.Int8PtrTy, 10));
+  llvm::Type *CurVTableTy = nullptr;
   const char *VTableName = nullptr;
 
   switch (Ty->getTypeClass()) {
 #define TYPE(Class, Base)
 #define ABSTRACT_TYPE(Class, Base)
 #define NON_CANONICAL_UNLESS_DEPENDENT_TYPE(Class, Base) case Type::Class:
 #define NON_CANONICAL_TYPE(Class, Base) case Type::Class:
 #define DEPENDENT_TYPE(Class, Base) case Type::Class:
#include "clang/AST/TypeNodes.def"
   case Type::Vector:
   case Type::ExtVector:
   case Type::Complex:
   case Type::Atomic:
   // FIXME: GCC treats block pointers as fundamental types?!
   case Type::BlockPointer:
     // abi::__fundamental_type_info.
     VTableName = "_ZTVN10__cxxabiv123__fundamental_type_infoE";
+    CurVTableTy = VTableTy1;
     break;
 
   case Type::ConstantArray:
   case Type::IncompleteArray:
   case Type::VariableArray:
     // abi::__array_type_info.
     VTableName = "_ZTVN10__cxxabiv117__array_type_infoE";
+    CurVTableTy = VTableTy1;
     break;
 
   case Type::FunctionNoProto:
   case Type::FunctionProto:
     // abi::__function_type_info.
     VTableName = "_ZTVN10__cxxabiv120__function_type_infoE";
+    CurVTableTy = VTableTy1;
     break;
 
   case Type::Enum:
     // abi::__enum_type_info.
     VTableName = "_ZTVN10__cxxabiv116__enum_type_infoE";
+    CurVTableTy = VTableTy1;
     break;
 
   case Type::Record: {
     const CXXRecordDecl *RD =
       cast<CXXRecordDecl>(cast<RecordType>(Ty)->getDecl());
 
     if (!RD->hasDefinition() || !RD->getNumBases()) {
       VTableName = ClassTypeInfo;
     } else if (CanUseSingleInheritance(RD)) {
       VTableName = SIClassTypeInfo;
     } else {
       VTableName = VMIClassTypeInfo;
     }
-
+    CurVTableTy = VTableTy2;
     break;
   }
 
   case Type::ObjCObject:
     // Ignore protocol qualifiers.
     Ty = cast<ObjCObjectType>(Ty)->getBaseType().getTypePtr();
 
     // Handle id and Class.
     if (isa<BuiltinType>(Ty)) {
       VTableName = ClassTypeInfo;
+      CurVTableTy = VTableTy2;
       break;
     }
 
     assert(isa<ObjCInterfaceType>(Ty));
     LLVM_FALLTHROUGH;
 
   case Type::ObjCInterface:
     if (cast<ObjCInterfaceType>(Ty)->getDecl()->getSuperClass()) {
       VTableName = SIClassTypeInfo;
     } else {
       VTableName = ClassTypeInfo;
     }
+    CurVTableTy = VTableTy2;
     break;
 
   case Type::ObjCObjectPointer:
   case Type::Pointer:
     // abi::__pointer_type_info.
     VTableName = "_ZTVN10__cxxabiv119__pointer_type_infoE";
+    CurVTableTy = VTableTy1;
     break;
 
   case Type::MemberPointer:
     // abi::__pointer_to_member_type_info.
     VTableName = "_ZTVN10__cxxabiv129__pointer_to_member_type_infoE";
+    CurVTableTy = VTableTy1;
     break;
   }
 
-  llvm::Constant *VTable =
-    CGM.getModule().getOrInsertGlobal(VTableName, CGM.Int8PtrTy);
+  llvm::Constant *VTable;
+  // FIXME When interleaving is enabled, we assume that
+  // libc++abi is used and we use vtable types for typeinfo classes
+  // defined in libc++abi.
+  if (CGM.getCodeGenOpts().VTableInterleaving) {
+    VTable = CGM.getModule().getOrInsertGlobal(VTableName, CurVTableTy);
+    CGM.setDSOLocal(cast<llvm::GlobalValue>(VTable->stripPointerCasts()));
+    llvm::Value *Indices[] = {
+        llvm::ConstantInt::get(CGM.Int32Ty, 0),
+        llvm::ConstantInt::get(CGM.Int32Ty, 0),
+        llvm::ConstantInt::get(CGM.Int32Ty, 2),
+    };
+    VTable = llvm::ConstantExpr::getGetElementPtr(CurVTableTy, VTable, Indices,
+                                                  /*InBounds=*/true,
+                                                  /*InRangeIndex=*/1);
+    VTable = llvm::ConstantExpr::getBitCast(VTable, CGM.Int8PtrTy);
+  } else {
+    VTable = CGM.getModule().getOrInsertGlobal(VTableName, CGM.Int8PtrTy);
     CGM.setDSOLocal(cast<llvm::GlobalValue>(VTable->stripPointerCasts()));
 
     llvm::Type *PtrDiffTy =
         CGM.getTypes().ConvertType(CGM.getContext().getPointerDiffType());
 
     // The vtable address point is 2.
     llvm::Constant *Two = llvm::ConstantInt::get(PtrDiffTy, 2);
-  VTable =
-      llvm::ConstantExpr::getInBoundsGetElementPtr(CGM.Int8PtrTy, VTable, Two);
+    VTable = llvm::ConstantExpr::getInBoundsGetElementPtr(CGM.Int8PtrTy, VTable,
+                                                          Two);
     VTable = llvm::ConstantExpr::getBitCast(VTable, CGM.Int8PtrTy);
-
+  }
   Fields.push_back(VTable);
 }
 
 /// Return the linkage that the type info and type info name constants
 /// should have for the given type.
 static llvm::GlobalVariable::LinkageTypes getTypeInfoLinkage(CodeGenModule &CGM,
                                                              QualType Ty) {
   // Itanium C++ ABI 2.9.5p7:
static unsigned ComputeVMIClassTypeInfoFlags(const CXXRecordDecl *RD) {
 
   // Walk all bases.
   for (const auto &I : RD->bases())
     Flags |= ComputeVMIClassTypeInfoFlags(&I, Bases);
 
   return Flags;
 }
 
+llvm::Constant *ItaniumRTTIBuilder::GetOffsetFlag(const CXXRecordDecl *RD,
+                                                  int64_t Offset,
+                                                  uint64_t LowerBits,
+                                                  llvm::Type *OffsetFlagsTy) {
+  // If interleaving is not enabled or Offset is not a virtual base offset,
+  // we don't need to create a offset placeholder for vtable interleaving.
+  if (!CXXABI.shouldInterleaveVTables(RD) || !(LowerBits & BCTI_Virtual)) {
+    uint64_t UpperBits = (uint64_t)Offset << 8;
+    uint64_t OffsetFlags = UpperBits | LowerBits;
+    return llvm::ConstantInt::get(OffsetFlagsTy, OffsetFlags);
+  }
+
+  llvm::Constant *OffsetConstant =
+      CXXABI.getOffsetConstant(RD, Offset, OffsetFlagsTy);
+  llvm::Constant *ShiftAmount = llvm::ConstantInt::get(OffsetFlagsTy, 8);
+  llvm::Constant *UpperBitsConstant =
+      llvm::ConstantExpr::getNSWShl(OffsetConstant, ShiftAmount);
+  llvm::Constant *LowerBitsConstant =
+      llvm::ConstantInt::get(OffsetFlagsTy, LowerBits);
+  return llvm::ConstantExpr::getOr(UpperBitsConstant, LowerBitsConstant);
+}
+
 /// BuildVMIClassTypeInfo - Build an abi::__vmi_class_type_info, used for
 /// classes with bases that do not satisfy the abi::__si_class_type_info
 /// constraints, according ti the Itanium C++ ABI, 2.9.5p5c.
 void ItaniumRTTIBuilder::BuildVMIClassTypeInfo(const CXXRecordDecl *RD) {
   llvm::Type *UnsignedIntLTy =
     CGM.getTypes().ConvertType(CGM.getContext().UnsignedIntTy);
 
   // Itanium C++ ABI 2.9.5p6c:
void ItaniumRTTIBuilder::BuildVMIClassTypeInfo(const CXXRecordDecl *RD) {
 
   for (const auto &Base : RD->bases()) {
     // The __base_type member points to the RTTI for the base type.
     Fields.push_back(ItaniumRTTIBuilder(CXXABI).BuildTypeInfo(Base.getType()));
 
     const CXXRecordDecl *BaseDecl =
       cast<CXXRecordDecl>(Base.getType()->getAs<RecordType>()->getDecl());
 
-    int64_t OffsetFlags = 0;
-
     // All but the lower 8 bits of __offset_flags are a signed offset.
     // For a non-virtual base, this is the offset in the object of the base
     // subobject. For a virtual base, this is the offset in the virtual table of
     // the virtual base offset for the virtual base referenced (negative).
     CharUnits Offset;
     if (Base.isVirtual())
       Offset =
         CGM.getItaniumVTableContext().getVirtualBaseOffsetOffset(RD, BaseDecl);
     else {
       const ASTRecordLayout &Layout = CGM.getContext().getASTRecordLayout(RD);
       Offset = Layout.getBaseClassOffset(BaseDecl);
     };
 
-    OffsetFlags = uint64_t(Offset.getQuantity()) << 8;
+    int64_t OffsetInt = Offset.getQuantity();
+    uint64_t Flags = 0;
 
     // The low-order byte of __offset_flags contains flags, as given by the
     // masks from the enumeration __offset_flags_masks.
     if (Base.isVirtual())
-      OffsetFlags |= BCTI_Virtual;
+      Flags |= BCTI_Virtual;
     if (Base.getAccessSpecifier() == AS_public)
-      OffsetFlags |= BCTI_Public;
+      Flags |= BCTI_Public;
 
-    Fields.push_back(llvm::ConstantInt::get(OffsetFlagsLTy, OffsetFlags));
+    Fields.push_back(GetOffsetFlag(RD, OffsetInt, Flags, OffsetFlagsLTy));
   }
 }
 
 /// Compute the flags for a __pbase_type_info, and remove the corresponding
 /// pieces from \p Type.
 static unsigned extractPBaseFlags(ASTContext &Ctx, QualType &Type) {
   unsigned Flags = 0;
 

clang/lib/CodeGen/MicrosoftCXXABI.cpp

 }
 
 CGCallee MicrosoftCXXABI::getVirtualFunctionPointer(CodeGenFunction &CGF,
                                                     GlobalDecl GD,
                                                     Address This,
                                                     llvm::Type *Ty,
                                                     SourceLocation Loc) {
   CGBuilderTy &Builder = CGF.Builder;
-
-  Ty = Ty->getPointerTo()->getPointerTo();
+  llvm::Type *VTablePtrTy = Ty->getPointerTo()->getPointerTo();
+  llvm::Type *FuncPtrTy = Ty->getPointerTo();
   Address VPtr =
       adjustThisArgumentForVirtualFunctionCall(CGF, GD, This, true);
 
   auto *MethodDecl = cast<CXXMethodDecl>(GD.getDecl());
-  llvm::Value *VTable = CGF.GetVTablePtr(VPtr, Ty, MethodDecl->getParent());
+  llvm::Value *VTable =
+      CGF.GetVTablePtr(VPtr, VTablePtrTy, MethodDecl->getParent());
 
   MicrosoftVTableContext &VFTContext = CGM.getMicrosoftVTableContext();
   MethodVFTableLocation ML = VFTContext.getMethodVFTableLocation(GD);
 
   // Compute the identity of the most derived class whose virtual table is
   // located at the MethodVFTableLocation ML.
   auto getObjectWithVPtr = [&] {
     return llvm::find_if(VFTContext.getVFPtrOffsets(
                              ML.VBase ? ML.VBase : MethodDecl->getParent()),
                          [&](const std::unique_ptr<VPtrInfo> &Info) {
                            return Info->FullOffsetInMDC == ML.VFPtrOffset;
                          })
         ->get()
         ->ObjectWithVPtr;
   };
 
   llvm::Value *VFunc;
   if (CGF.ShouldEmitVTableTypeCheckedLoad(MethodDecl->getParent())) {
-    VFunc = CGF.EmitVTableTypeCheckedLoad(
-        getObjectWithVPtr(), VTable,
-        ML.Index * CGM.getContext().getTargetInfo().getPointerWidth(0) / 8);
+    int64_t ByteOffset =
+        ML.Index * CGM.getContext().getTargetInfo().getPointerWidth(0) / 8;
+    llvm::Value *OffsetConstant =
+        llvm::ConstantInt::get(CGM.Int32Ty, ByteOffset);
+    llvm::Value *CastedVTable = Builder.CreateBitCast(VTable, CGM.Int8PtrTy);
+    VFunc = CGF.EmitVTableTypeCheckedLoad(getObjectWithVPtr(), CastedVTable,
+                                          FuncPtrTy, OffsetConstant);
   } else {
     if (CGM.getCodeGenOpts().PrepareForLTO)
       CGF.EmitTypeMetadataCodeForVCall(getObjectWithVPtr(), VTable, Loc);
 
     llvm::Value *VFuncPtr =
         Builder.CreateConstInBoundsGEP1_64(VTable, ML.Index, "vfn");
     VFunc = Builder.CreateAlignedLoad(VFuncPtr, CGF.getPointerAlign());
   }

clang/lib/Frontend/CompilerInvocation.cpp

static bool ParseCodeGenArgs(CodeGenOptions &Opts, ArgList &Args, InputKind IK,
   }
 
   Opts.BranchTargetEnforcement = Args.hasArg(OPT_mbranch_target_enforce);
 
   Opts.KeepStaticConsts = Args.hasArg(OPT_fkeep_static_consts);
 
   Opts.SpeculativeLoadHardening = Args.hasArg(OPT_mspeculative_load_hardening);
 
   Opts.DefaultFunctionAttrs = Args.getAllArgValues(OPT_default_function_attr);
 
   Opts.PassPlugins = Args.getAllArgValues(OPT_fpass_plugin_EQ);
 
   Opts.SymbolPartition = Args.getLastArgValue(OPT_fsymbol_partition_EQ);
 
+  Opts.VTableInterleaving = Args.hasArg(OPT_vtable_interleaving);
+
   return Success;
 }
 
 static void ParseDependencyOutputArgs(DependencyOutputOptions &Opts,
                                       ArgList &Args) {
   Opts.OutputFile = Args.getLastArgValue(OPT_dependency_file);
   Opts.Targets = Args.getAllArgValues(OPT_MT);
   Opts.IncludeSystemHeaders = Args.hasArg(OPT_sys_header_deps);

clang/test/CodeGen/tbaa-for-vptr.cpp

-// RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -o - -fsanitize=thread %s | FileCheck %s --check-prefixes=CHECK,OLD-PATH
-// RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -o - -O1 -fno-experimental-new-pass-manager %s | FileCheck %s --check-prefixes=CHECK,OLD-PATH
-// RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -o - -O1 -fno-experimental-new-pass-manager -relaxed-aliasing -fsanitize=thread %s | FileCheck %s --check-prefixes=CHECK,OLD-PATH
+// RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -o - -fsanitize=thread %s | FileCheck %s --check-prefixes=CHECK,OLD-PATH,UNOPTIMIZED
+// RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -o - -O1 -fno-experimental-new-pass-manager %s | FileCheck %s --check-prefixes=CHECK,OLD-PATH,OPTIMIZED
+// RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -o - -O1 -fno-experimental-new-pass-manager -relaxed-aliasing -fsanitize=thread %s | FileCheck %s --check-prefixes=CHECK,OLD-PATH,OPTIMIZED
 //
-// RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -new-struct-path-tbaa -o - -fsanitize=thread %s | FileCheck %s --check-prefixes=CHECK,NEW-PATH
-// RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -new-struct-path-tbaa -o - -O1 -fno-experimental-new-pass-manager %s | FileCheck %s --check-prefixes=CHECK,NEW-PATH
-// RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -new-struct-path-tbaa -o - -O1 -fno-experimental-new-pass-manager -relaxed-aliasing -fsanitize=thread %s | FileCheck %s --check-prefixes=CHECK,NEW-PATH
+// RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -new-struct-path-tbaa -o - -fsanitize=thread %s | FileCheck %s --check-prefixes=CHECK,NEW-PATH,UNOPTIMIZED
+// RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -new-struct-path-tbaa -o - -O1 -fno-experimental-new-pass-manager %s | FileCheck %s --check-prefixes=CHECK,NEW-PATH,OPTIMIZED
+// RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -new-struct-path-tbaa -o - -O1 -fno-experimental-new-pass-manager -relaxed-aliasing -fsanitize=thread %s | FileCheck %s --check-prefixes=CHECK,NEW-PATH,OPTIMIZED
 //
 // RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -o - %s | FileCheck %s --check-prefix=NOTBAA
 // RUN: %clang_cc1 -triple %itanium_abi_triple -emit-llvm -o - -O2 -fno-experimental-new-pass-manager -relaxed-aliasing %s | FileCheck %s --check-prefix=NOTBAA
 //
 // Check that we generate TBAA for vtable pointer loads and stores.
 // When -fsanitize=thread is used TBAA should be generated at all opt levels
 // even if -relaxed-aliasing is present.
 struct A {
   virtual int foo() const ;
   virtual ~A();
 };
 
 void CreateA() {
   new A;
 }
 
 void CallFoo(A *a, int (A::*fp)() const) {
   a->foo();
   (a->*fp)();
 }
 
 // CHECK-LABEL: @_Z7CallFoo
-// CHECK: %{{.*}} = load i32 (%struct.A*)**, {{.*}} !tbaa ![[NUM:[0-9]+]]
+// UNOPTIMIZED: [[R0:%[^ ]*]] = bitcast %struct.A* {{.*}} to i8**
+// UNOPTIMIZED: %{{.*}} = load i8*, i8** [[R0]], {{.*}} !tbaa ![[NUM:[0-9]+]]
+// OPTIMIZED: %{{.*}} = load i32 (%struct.A*)**, {{.*}} !tbaa ![[NUM:[0-9]+]]
 // CHECK: br i1
 // CHECK: load i8*, {{.*}}, !tbaa ![[NUM]]
 //
 // CHECK-LABEL: @_ZN1AC2Ev
 // CHECK: store i32 (...)** {{.*}}, !tbaa ![[NUM]]
 //
 // OLD-PATH: [[NUM]] = !{[[TYPE:!.*]], [[TYPE]], i64 0}
 // OLD-PATH: [[TYPE]] = !{!"vtable pointer", !{{.*}}
 // NEW-PATH: [[NUM]] = !{[[TYPE:!.*]], [[TYPE]], i64 0, i64 [[POINTER_SIZE:.*]]}
 // NEW-PATH: [[TYPE]] = !{!{{.*}}, i64 [[POINTER_SIZE]], !"vtable pointer"}
 // NOTBAA-NOT: = !{!"Simple C++ TBAA"}

clang/test/CodeGenCXX/alignment.cpp

namespace test1 {
   }
 
   // CHECK-LABEL: @_ZN5test11bERNS_1BE
   void b(B &b) {
     // CHECK: [[RESULT:%.*]] = alloca [[ARRAY]], align 64
     // CHECK: [[B_P:%.*]] = load [[B:%.*]]*, [[B]]**
     // CHECK: [[VPTR_P:%.*]] = bitcast [[B]]* [[B_P]] to i8**
     // CHECK: [[VPTR:%.*]] = load i8*, i8** [[VPTR_P]], align 8
-    // CHECK: [[T0:%.*]] = getelementptr i8, i8* [[VPTR]], i64 -24
+    // CHECK: [[T0:%.*]] = getelementptr inbounds i8, i8* [[VPTR]], i64 -24
     // CHECK: [[OFFSET_P:%.*]] = bitcast i8* [[T0]] to i64*
     // CHECK: [[OFFSET:%.*]] = load i64, i64* [[OFFSET_P]], align 8
     // CHECK: [[T0:%.*]] = bitcast [[B]]* [[B_P]] to i8*
     // CHECK: [[T1:%.*]] = getelementptr inbounds i8, i8* [[T0]], i64 [[OFFSET]]
     // CHECK: [[A_P:%.*]] = bitcast i8* [[T1]] to [[A]]*
     // CHECK: [[ARRAY_P:%.*]] = getelementptr inbounds [[A]], [[A]]* [[A_P]], i32 0, i32 0
     // CHECK: [[T0:%.*]] = bitcast [[ARRAY]]* [[RESULT]] to i8*
     // CHECK: [[T1:%.*]] = bitcast [[ARRAY]]* [[ARRAY_P]] to i8*
namespace test1 {
 
   // CHECK-LABEL: @_ZN5test11fEv
   void f() {
     // TODO: we should devirtualize this derived-to-base conversion.
     // CHECK: [[D_P:%.*]] = alloca [[D:%.*]], align 16
     // CHECK: [[RESULT:%.*]] = alloca [[ARRAY]], align 64
     // CHECK: [[VPTR_P:%.*]] = bitcast [[D]]* [[D_P]] to i8**
     // CHECK: [[VPTR:%.*]] = load i8*, i8** [[VPTR_P]], align 16
-    // CHECK: [[T0:%.*]] = getelementptr i8, i8* [[VPTR]], i64 -24
+    // CHECK: [[T0:%.*]] = getelementptr inbounds i8, i8* [[VPTR]], i64 -24
     // CHECK: [[OFFSET_P:%.*]] = bitcast i8* [[T0]] to i64*
     // CHECK: [[OFFSET:%.*]] = load i64, i64* [[OFFSET_P]], align 8
     // CHECK: [[T0:%.*]] = bitcast [[D]]* [[D_P]] to i8*
     // CHECK: [[T1:%.*]] = getelementptr inbounds i8, i8* [[T0]], i64 [[OFFSET]]
     // CHECK: [[A_P:%.*]] = bitcast i8* [[T1]] to [[A]]*
     // CHECK: [[ARRAY_P:%.*]] = getelementptr inbounds [[A]], [[A]]* [[A_P]], i32 0, i32 0
     // CHECK: [[T0:%.*]] = bitcast [[ARRAY]]* [[RESULT]] to i8*
     // CHECK: [[T1:%.*]] = bitcast [[ARRAY]]* [[ARRAY_P]] to i8*

clang/test/CodeGenCXX/arm.cpp

namespace test6 {
 
   // CHECK-LABEL: define void @_ZN5test64testEPNS_1AE
   void test(A *a) {
     // CHECK:      [[AVAR:%.*]] = alloca [[A:%.*]]*, align 4
     // CHECK-NEXT: store [[A]]* {{.*}}, [[A]]** [[AVAR]], align 4
     // CHECK-NEXT: [[V:%.*]] = load [[A]]*, [[A]]** [[AVAR]], align 4
     // CHECK-NEXT: [[ISNULL:%.*]] = icmp eq [[A]]* [[V]], null
     // CHECK-NEXT: br i1 [[ISNULL]]
-    // CHECK:      [[T0:%.*]] = bitcast [[A]]* [[V]] to void ([[A]]*)***
-    // CHECK-NEXT: [[T1:%.*]] = load void ([[A]]*)**, void ([[A]]*)*** [[T0]]
-    // CHECK-NEXT: [[T2:%.*]] = getelementptr inbounds void ([[A]]*)*, void ([[A]]*)** [[T1]], i64 1
-    // CHECK-NEXT: [[T3:%.*]] = load void ([[A]]*)*, void ([[A]]*)** [[T2]]
-    // CHECK-NEXT: call void [[T3]]([[A]]* [[V]])
+    // CHECK:      [[T0:%.*]] = bitcast [[A]]* [[V]] to i8**
+    // CHECK-NEXT: [[T1:%.*]] = load i8*, i8** [[T0]]
+    // CHECK-NEXT: [[T2:%.*]] = getelementptr inbounds i8, i8* [[T1]], i32 4
+    // CHECK:      [[T3:%.*]] = bitcast i8* [[T2]] to void ([[A]]*)**
+    // CHECK-NEXT: [[T4:%.*]] = load void ([[A]]*)*, void ([[A]]*)** [[T3]]
+    // CHECK-NEXT: call void [[T4]]([[A]]* [[V]])
     // CHECK-NEXT: br label
     // CHECK:      ret void
     delete a;
   }
 }
 
 namespace test7 {
   int foo();

clang/test/CodeGenCXX/cfi-cross-dso.cpp

 
 void g() {
   B b;
   b.f();
 }
 
 // MS: @[[B_VTABLE:.*]] = private unnamed_addr constant { [2 x i8*] } {{.*}}@"??_R4B@?A0x{{[^@]*}}@@6B@"{{.*}}@"?f@B@?A0x{{[^@]*}}@@UEAAXXZ"
 
-// CHECK:   %[[VT:.*]] = load void (%struct.A*)**, void (%struct.A*)***
-// CHECK:   %[[VT2:.*]] = bitcast {{.*}}%[[VT]] to i8*, !nosanitize
+// CHECK:   %[[R0:.*]] = load %struct.A*, %struct.A** %a.addr
+// ITANIUM:   %[[R1:.*]] = bitcast %struct.A* %[[R0]] to i8**
+// ITANIUM:   %[[VT2:.*]] = load i8*, i8** %[[R1]]
+// MS:   %[[R1:.*]] = bitcast %struct.A* %[[R0]] to void (%struct.A*)***
+// MS:   %[[VT:.*]] = load void (%struct.A*)**, void (%struct.A*)*** %[[R1]]
+// MS:   %[[VT2:.*]] = bitcast {{.*}}%[[VT]] to i8*
 // ITANIUM:   %[[TEST:.*]] = call i1 @llvm.type.test(i8* %[[VT2]], metadata !"_ZTS1A"), !nosanitize
 // MS:   %[[TEST:.*]] = call i1 @llvm.type.test(i8* %[[VT2]], metadata !"?AUA@@"), !nosanitize
 // CHECK:   br i1 %[[TEST]], label %[[CONT:.*]], label %[[SLOW:.*]], {{.*}} !nosanitize
 // CHECK: [[SLOW]]
 // ITANIUM:   call void @__cfi_slowpath_diag(i64 7004155349499253778, i8* %[[VT2]], {{.*}}) {{.*}} !nosanitize
 // MS:   call void @__cfi_slowpath_diag(i64 -8005289897957287421, i8* %[[VT2]], {{.*}}) {{.*}} !nosanitize
 // CHECK:   br label %[[CONT]], !nosanitize
 // CHECK: [[CONT]]
 // CHECK:   call void %{{.*}}(%struct.A* %{{.*}})
 
 // No hash-based bit set entry for (anonymous namespace)::B
 // ITANIUM-NOT: !{i64 {{.*}}, [3 x i8*]* @_ZTVN12_GLOBAL__N_11BE,
 // MS-NOT: !{i64 {{.*}}, [2 x i8*]* @[[B_VTABLE]],

clang/test/CodeGenCXX/constructor-destructor-return-this.cpp

void test_destructor() {
   E* e2 = gete();
   e2->~E();
 }
 
 // CHECKARM-LABEL: define void @_Z15test_destructorv()
 
 // Verify that virtual calls to destructors are not marked with a 'returned'
 // this parameter at the call site...
-// CHECKARM: [[VFN:%.*]] = getelementptr inbounds %class.E* (%class.E*)*, %class.E* (%class.E*)**
+// CHECKARM: [[VFPTR:%.*]] = getelementptr inbounds i8, i8*
+// CHECKARM: [[VFN:%.*]] = bitcast i8* [[VFPTR]] to %class.E* (%class.E*)**
 // CHECKARM: [[THUNK:%.*]] = load %class.E* (%class.E*)*, %class.E* (%class.E*)** [[VFN]]
 // CHECKARM: call %class.E* [[THUNK]](%class.E* %
 
 // ...but static calls create declarations with 'returned' this
 // CHECKARM: {{%.*}} = call %class.E* @_ZN1ED1Ev(%class.E* %
 
 // CHECKARM: declare %class.E* @_ZN1ED1Ev(%class.E* returned)

clang/test/CodeGenCXX/constructor-init.cpp

struct B : A {
     virtual int foo();
     B();
     B(int);
   };
 
   // CHECK-LABEL: define void @_ZN10InitVTable1BC2Ev(%"struct.InitVTable::B"* %this) unnamed_addr
   // CHECK:      [[T0:%.*]] = bitcast [[B:%.*]]* [[THIS:%.*]] to i32 (...)***
   // CHECK-NEXT: store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTVN10InitVTable1BE, i32 0, inrange i32 0, i32 2) to i32 (...)**), i32 (...)*** [[T0]]
-  // CHECK:      [[VTBL:%.*]] = load i32 ([[B]]*)**, i32 ([[B]]*)*** {{%.*}}
-  // CHECK-NEXT: [[FNP:%.*]] = getelementptr inbounds i32 ([[B]]*)*, i32 ([[B]]*)** [[VTBL]], i64 0
+  // CHECK:      [[VTBL:%.*]] = load i8*, i8** {{%.*}}
+  // CHECK-NEXT: [[FNBP:%.*]] = getelementptr inbounds i8, i8* [[VTBL]], i32 0
+  // CHECK-NEXT: [[FNP:%.*]] = bitcast i8* [[FNBP]] to i32 ([[B]]*)**
   // CHECK-NEXT: [[FN:%.*]] = load i32 ([[B]]*)*, i32 ([[B]]*)** [[FNP]]
   // CHECK-NEXT: [[ARG:%.*]] = call i32 [[FN]]([[B]]* [[THIS]])
   // CHECK-NEXT: call void @_ZN10InitVTable1AC2Ei({{.*}}* {{%.*}}, i32 [[ARG]])
   // CHECK-NEXT: [[T0:%.*]] = bitcast [[B]]* [[THIS]] to i32 (...)***
   // CHECK-NEXT: store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTVN10InitVTable1BE, i32 0, inrange i32 0, i32 2) to i32 (...)**), i32 (...)*** [[T0]]
   // CHECK-NEXT: ret void
   B::B() : A(foo()) {}
 

clang/test/CodeGenCXX/delete.cpp

void global_delete_virtual(X *xp) {
     // CHECK:      [[T0:%.*]] = bitcast [[X:%.*]]* [[XP:%.*]] to i64**
     // CHECK-NEXT: [[VTABLE:%.*]] = load i64*, i64** [[T0]]
     // CHECK-NEXT: [[T0:%.*]] = getelementptr inbounds i64, i64* [[VTABLE]], i64 -2
     // CHECK-NEXT: [[OFFSET:%.*]] = load i64, i64* [[T0]], align 8
     // CHECK-NEXT: [[T0:%.*]] = bitcast [[X]]* [[XP]] to i8*
     // CHECK-NEXT: [[ALLOCATED:%.*]] = getelementptr inbounds i8, i8* [[T0]], i64 [[OFFSET]]
     //   Load the complete-object destructor (not the deleting destructor)
     //   and call it.
-    // CHECK-NEXT: [[T0:%.*]] = bitcast [[X:%.*]]* [[XP:%.*]] to void ([[X]]*)***
-    // CHECK-NEXT: [[VTABLE:%.*]] = load void ([[X]]*)**, void ([[X]]*)*** [[T0]]
-    // CHECK-NEXT: [[T0:%.*]] = getelementptr inbounds void ([[X]]*)*, void ([[X]]*)** [[VTABLE]], i64 0
-    // CHECK-NEXT: [[DTOR:%.*]] = load void ([[X]]*)*, void ([[X]]*)** [[T0]]
+    // CHECK-NEXT: [[T0:%.*]] = bitcast [[X:%.*]]* [[XP:%.*]] to i8**
+    // CHECK-NEXT: [[VTABLE:%.*]] = load i8*, i8** [[T0]]
+    // CHECK-NEXT: [[T0:%.*]] = getelementptr inbounds i8, i8* [[VTABLE]], i32 0
+    // CHECK-NEXT: [[T1:%.*]] = bitcast i8* [[T0]] to void ([[X]]*)**
+    // CHECK-NEXT: [[DTOR:%.*]] = load void ([[X]]*)*, void ([[X]]*)** [[T1]]
     // CHECK-NEXT: call void [[DTOR]]([[X]]* [[OBJ:%.*]])
     //   Call the global operator delete.
     // CHECK-NEXT: call void @_ZdlPv(i8* [[ALLOCATED]]) [[NUW:#[0-9]+]]
     ::delete xp;
   }
 }
 
 namespace test5 {

clang/test/CodeGenCXX/devirtualize-virtual-function-calls-final.cpp

namespace Test5 {
   struct C final : B {
   };
 
   // CHECK-LABEL: define void @_ZN5Test51fEPNS_1CE
   void f(C* d) {
     // FIXME: It should be possible to devirtualize this case, but that is
     // not implemented yet.
     // CHECK: getelementptr
+    // CHECK-NEXT: bitcast
     // CHECK-NEXT: %[[FUNC:.*]] = load
     // CHECK-NEXT: call void %[[FUNC]]
     static_cast<A*>(d)->f();
   }
   // CHECK-LABEL: define void @_ZN5Test53fopEPNS_1CE
   void fop(C* d) {
     // FIXME: It should be possible to devirtualize this case, but that is
     // not implemented yet.
     // CHECK: getelementptr
+    // CHECK-NEXT: bitcast
     // CHECK-NEXT: %[[FUNC:.*]] = load
     // CHECK-NEXT: call i32 %[[FUNC]]
     -static_cast<A&>(*d);
   }
 }
 
 namespace Test6 {
   struct A {
namespace Test9 {
   A *f(RC *x) {
     // FIXME: It should be possible to devirtualize this case, but that is
     // not implemented yet.
     // CHECK: load
     // CHECK: bitcast
     // CHECK: [[F_PTR_RA:%.+]] = bitcast
     // CHECK: [[VTABLE:%.+]] = load {{.+}} [[F_PTR_RA]]
     // CHECK: [[VFN:%.+]] = getelementptr inbounds {{.+}} [[VTABLE]], i{{[0-9]+}} 0
-    // CHECK-NEXT: %[[FUNC:.*]] = load {{.+}} [[VFN]]
+    // CHECK-NEXT: [[VFP:%.*]] = bitcast {{.+}} [[VFN]]
+    // CHECK-NEXT: %[[FUNC:.*]] = load {{.+}} [[VFP]]
     // CHECK-NEXT: = call {{.*}} %[[FUNC]]
     return static_cast<RA*>(x)->f();
   }
   // CHECK: define {{.*}} @_ZN5Test93fopEPNS_2RCE
   A *fop(RC *x) {
     // FIXME: It should be possible to devirtualize this case, but that is
     // not implemented yet.
     // CHECK: load
     // CHECK: bitcast
     // CHECK: [[F_PTR_RA:%.+]] = bitcast
     // CHECK: [[VTABLE:%.+]] = load {{.+}} [[F_PTR_RA]]
-    // CHECK: [[VFN:%.+]] = getelementptr inbounds {{.+}} [[VTABLE]], i{{[0-9]+}} 1
-    // CHECK-NEXT: %[[FUNC:.*]] = load {{.+}} [[VFN]]
+    // CHECK: [[VFN:%.+]] = getelementptr inbounds {{.+}} [[VTABLE]], i{{[0-9]+}} 4
+    // CHECK-NEXT: [[VFP:%.*]] = bitcast {{.+}} [[VFN]]
+    // CHECK-NEXT: %[[FUNC:.*]] = load {{.+}} [[VFP]]
     // CHECK-NEXT: = call {{.*}} %[[FUNC]]
     return -static_cast<RA&>(*x);
   }
 }
 
 namespace Test10 {
   struct A {
     virtual int f();

clang/test/CodeGenCXX/interleaving-member-function-pointers.cpp

+// This file tests if accesses to vbase entries are correctly generated when vtable interleaving is enabled.
+// This test is based on member-function-pointers.cpp.
+
+// RUN: %clang_cc1 %s -flto -fvisibility hidden -fsanitize=cfi-vcall -vtable-interleaving -emit-llvm -o - -triple=x86_64-unknown-unknown | FileCheck -check-prefix CODE-LP64 %s
+// RUN: %clang_cc1 %s -flto -fvisibility hidden -fsanitize=cfi-vcall -vtable-interleaving -emit-llvm -o - -triple=i386-unknown-unknown | FileCheck -check-prefix CODE-LP32 %s
+// RUN: %clang_cc1 %s -flto -fvisibility hidden -fsanitize=cfi-vcall -vtable-interleaving -emit-llvm -o - -triple=x86_64-unknown-unknown | FileCheck -check-prefix GLOBAL-LP64 %s
+// RUN: %clang_cc1 %s -flto -fvisibility hidden -fsanitize=cfi-vcall -vtable-interleaving -emit-llvm -o - -triple=i386-unknown-unknown | FileCheck -check-prefix GLOBAL-LP32 %s
+// RUN: %clang_cc1 %s -flto -fvisibility hidden -fsanitize=cfi-vcall -vtable-interleaving -emit-llvm -o - -triple=armv7-unknown-unknown | FileCheck -check-prefix GLOBAL-ARM %s
+
+// PNaCl uses the same representation of method pointers as ARM.
+// RUN: %clang_cc1 %s -flto -fvisibility hidden -fsanitize=cfi-vcall -vtable-interleaving -emit-llvm -o - -triple=le32-unknown-nacl | FileCheck -check-prefix GLOBAL-ARM %s
+// MIPS uses the same representation of method pointers as ARM.
+// RUN: %clang_cc1 %s -flto -fvisibility hidden -fsanitize=cfi-vcall -vtable-interleaving -emit-llvm -o - -triple=mips-unknown-linux-gnu | FileCheck -check-prefix GLOBAL-ARM %s
+// WebAssembly uses the same representation of method pointers as ARM.
+// RUN: %clang_cc1 %s -flto -fvisibility hidden -fsanitize=cfi-vcall -vtable-interleaving -emit-llvm -o - -triple=wasm32-unknown-unknown | FileCheck -check-prefix GLOBAL-ARM %s
+
+namespace test1 {
+struct A {
+  int a;
+  void f();
+  virtual void vf1();
+  virtual void vf2();
+};
+struct B {
+  int b;
+  virtual void g();
+};
+struct C : B, A {};
+
+void (A::*pa)();
+void (A::*volatile vpa)();
+void (B::*pb)();
+void (C::*pc)();
+
+// GLOBAL-LP64: @_ZN5test13pa2E = hidden global { i64, i64 } { i64 ptrtoint (void (%"struct.test1::A"*)* @_ZN5test11A1fEv to i64), i64 0 }, align 8
+void (A::*pa2)() = &A::f;
+
+// GLOBAL-LP64: @_ZN5test13pa3E = hidden global { i64, i64 } { i64 add (i64 ptrtoint ([0 x i32]* @"__A$0" to i64), i64 1), i64 0 }, align 8
+// GLOBAL-LP32: @_ZN5test13pa3E = hidden global { i32, i32 } { i32 add (i32 ptrtoint ([0 x i32]* @"__A$0" to i32), i32 1), i32 0 }, align 4
+void (A::*pa3)() = &A::vf1;
+
+// GLOBAL-LP64: @_ZN5test13pa4E = hidden global { i64, i64 } { i64 add (i64 ptrtoint ([0 x i32]* @"__A$8" to i64), i64 1), i64 0 }, align 8
+// GLOBAL-LP32: @_ZN5test13pa4E = hidden global { i32, i32 } { i32 add (i32 ptrtoint ([0 x i32]* @"__A$4" to i32), i32 1), i32 0 }, align 4
+void (A::*pa4)() = &A::vf2;
+
+// GLOBAL-LP64: @_ZN5test13pc2E = hidden global { i64, i64 } { i64 ptrtoint (void (%"struct.test1::A"*)* @_ZN5test11A1fEv to i64), i64 16 }, align 8
+void (C::*pc2)() = &C::f;
+
+// GLOBAL-LP64: @_ZN5test13pc3E = hidden global { i64, i64 } {  i64 add (i64 ptrtoint ([0 x i32]* @"__A$0" to i64), i64 1), i64 0 }, align 8
+void (A::*pc3)() = &A::vf1;
+
+void f() {
+  // CODE-LP64: store { i64, i64 } zeroinitializer, { i64, i64 }* @_ZN5test12paE
+  pa = 0;
+
+  // Is this okay?  What are LLVM's volatile semantics for structs?
+  // CODE-LP64: store volatile { i64, i64 } zeroinitializer, { i64, i64 }* @_ZN5test13vpaE
+  vpa = 0;
+
+  // CODE-LP64: [[TMP:%.*]] = load { i64, i64 }, { i64, i64 }* @_ZN5test12paE, align 8
+  // CODE-LP64: [[TMPADJ:%.*]] = extractvalue { i64, i64 } [[TMP]], 1
+  // CODE-LP64: [[ADJ:%.*]] = add nsw i64 [[TMPADJ]], 16
+  // CODE-LP64: [[RES:%.*]] = insertvalue { i64, i64 } [[TMP]], i64 [[ADJ]], 1
+  // CODE-LP64: store { i64, i64 } [[RES]], { i64, i64 }* @_ZN5test12pcE, align 8
+  pc = pa;
+
+  // CODE-LP64: [[TMP:%.*]] = load { i64, i64 }, { i64, i64 }* @_ZN5test12pcE, align 8
+  // CODE-LP64: [[TMPADJ:%.*]] = extractvalue { i64, i64 } [[TMP]], 1
+  // CODE-LP64: [[ADJ:%.*]] = sub nsw i64 [[TMPADJ]], 16
+  // CODE-LP64: [[RES:%.*]] = insertvalue { i64, i64 } [[TMP]], i64 [[ADJ]], 1
+  // CODE-LP64: store { i64, i64 } [[RES]], { i64, i64 }* @_ZN5test12paE, align 8
+  pa = static_cast<void (A::*)()>(pc);
+}
+
+void f2() {
+  // CODE-LP64: store { i64, i64 } { i64 ptrtoint (void (%"struct.test1::A"*)* @_ZN5test11A1fEv to i64), i64 0 }
+  void (A::*pa2)() = &A::f;
+
+  // CODE-LP64: store { i64, i64 } { i64 add (i64 ptrtoint ([0 x i32]* @"__A$0" to i64), i64 1), i64 0 }
+  // CODE-LP32: store { i32, i32 } { i32 add (i32 ptrtoint ([0 x i32]* @"__A$0" to i32), i32 1), i32 0 }
+  void (A::*pa3)() = &A::vf1;
+
+  // CODE-LP64: store { i64, i64 } { i64 add (i64 ptrtoint ([0 x i32]* @"__A$8" to i64), i64 1), i64 0 }
+  // CODE-LP32: store { i32, i32 } { i32 add (i32 ptrtoint ([0 x i32]* @"__A$4" to i32), i32 1), i32 0 }
+  void (A::*pa4)() = &A::vf2;
+}
+
+void f3(A *a, A &ar) {
+  (a->*pa)();
+  (ar.*pa)();
+}
+
+bool f4() {
+  return pa;
+}
+} // namespace test1
+
+namespace test2 {
+struct A {
+  void foo();
+  virtual void vfoo();
+};
+struct B {
+  void foo();
+  virtual void vfoo();
+};
+struct C : A, B {
+  void foo();
+  virtual void vfoo();
+};
+
+// GLOBAL-ARM: @_ZN5test24ptr0E = hidden global {{.*}} { i32 ptrtoint ({{.*}}* @_ZN5test21A3fooEv to i32), i32 0 }
+// GLOBAL-ARM: @_ZN5test24ptr1E = hidden global {{.*}} { i32 ptrtoint ({{.*}}* @_ZN5test21B3fooEv to i32), i32 8 }
+// GLOBAL-ARM: @_ZN5test24ptr2E = hidden global {{.*}} { i32 ptrtoint ({{.*}}* @_ZN5test21C3fooEv to i32), i32 0 }
+// GLOBAL-ARM: @_ZN5test24ptr3E = hidden global {{.*}} { i32 ptrtoint ([0 x i32]* @"__A$0.1" to i32), i32 1 }
+// GLOBAL-ARM: @_ZN5test24ptr4E = hidden global {{.*}} { i32 ptrtoint ([0 x i32]* @"__B$0" to i32), i32 9 }
+// GLOBAL-ARM: @_ZN5test24ptr5E = hidden global {{.*}} { i32 ptrtoint ([0 x i32]* @"__C$0" to i32), i32 1 }
+void (C::*ptr0)() = &A::foo;
+void (C::*ptr1)() = &B::foo;
+void (C::*ptr2)() = &C::foo;
+void (C::*ptr3)() = &A::vfoo;
+void (C::*ptr4)() = &B::vfoo;
+void (C::*ptr5)() = &C::vfoo;
+} // namespace test2
+
+// rdar://problem/10815683 - Verify that we can emit reinterprets of
+// member pointers as constant initializers.  For added trickiness,
+// we also add some non-trivial adjustments.
+namespace test3 {
+struct A {
+  int nonEmpty;
+  void foo();
+};
+struct B : public A {
+  virtual void requireNonZeroAdjustment();
+};
+struct C {
+  int nonEmpty;
+};
+struct D : public C {
+  virtual void requireNonZeroAdjustment();
+};
+
+// It's not that the offsets are doubled on ARM, it's that they're left-shifted by 1.
+
+// GLOBAL-LP64: @_ZN5test31aE = hidden global { i64, i64 } { i64 ptrtoint (void (%"struct.test3::A"*)* @_ZN5test31A3fooEv to i64), i64 0 }, align 8
+// GLOBAL-LP32: @_ZN5test31aE = hidden global { i32, i32 } { i32 ptrtoint (void (%"struct.test3::A"*)* @_ZN5test31A3fooEv to i32), i32 0 }, align 4
+// GLOBAL-ARM:  @_ZN5test31aE = hidden global { i32, i32 } { i32 ptrtoint (void (%"struct.test3::A"*)* @_ZN5test31A3fooEv to i32), i32 0 }, align 4
+void (A::*a)() = &A::foo;
+
+// GLOBAL-LP64: @_ZN5test31bE = hidden global { i64, i64 } { i64 ptrtoint (void (%"struct.test3::A"*)* @_ZN5test31A3fooEv to i64), i64 8 }, align 8
+// GLOBAL-LP32: @_ZN5test31bE = hidden global { i32, i32 } { i32 ptrtoint (void (%"struct.test3::A"*)* @_ZN5test31A3fooEv to i32), i32 4 }, align 4
+// GLOBAL-ARM:  @_ZN5test31bE = hidden global { i32, i32 } { i32 ptrtoint (void (%"struct.test3::A"*)* @_ZN5test31A3fooEv to i32), i32 8 }, align 4
+void (B::*b)() = (void (B::*)()) & A::foo;
+
+// GLOBAL-LP64: @_ZN5test31cE = hidden global { i64, i64 } { i64 ptrtoint (void (%"struct.test3::A"*)* @_ZN5test31A3fooEv to i64), i64 8 }, align 8
+// GLOBAL-LP32: @_ZN5test31cE = hidden global { i32, i32 } { i32 ptrtoint (void (%"struct.test3::A"*)* @_ZN5test31A3fooEv to i32), i32 4 }, align 4
+// GLOBAL-ARM:  @_ZN5test31cE = hidden global { i32, i32 } { i32 ptrtoint (void (%"struct.test3::A"*)* @_ZN5test31A3fooEv to i32), i32 8 }, align 4
+void (C::*c)() = (void (C::*)())(void (B::*)()) & A::foo;
+
+// GLOBAL-LP64: @_ZN5test31dE = hidden global { i64, i64 } { i64 ptrtoint (void (%"struct.test3::A"*)* @_ZN5test31A3fooEv to i64), i64 16 }, align 8
+// GLOBAL-LP32: @_ZN5test31dE = hidden global { i32, i32 } { i32 ptrtoint (void (%"struct.test3::A"*)* @_ZN5test31A3fooEv to i32), i32 8 }, align 4
+// GLOBAL-ARM:  @_ZN5test31dE = hidden global { i32, i32 } { i32 ptrtoint (void (%"struct.test3::A"*)* @_ZN5test31A3fooEv to i32), i32 16 }, align 4
+void (D::*d)() = (void (C::*)())(void (B::*)()) & A::foo;
+} // namespace test3
+
+namespace test4 {
+struct A {
+  virtual void a();
+};
+struct B : A {};
+struct C : B {
+  virtual void a();
+};
+void (C::*x)() = &C::a;
+
+// GLOBAL-LP64: @_ZN5test41xE = hidden global { i64, i64 } { i64 add (i64 ptrtoint ([0 x i32]* @"__C$0.2" to i64), i64 1), i64 0 }
+// GLOBAL-LP32: @_ZN5test41xE = hidden global { i32, i32 } { i32 add (i32 ptrtoint ([0 x i32]* @"__C$0.2" to i32), i32 1), i32 0 }
+// GLOBAL-ARM:  @_ZN5test41xE = hidden global { i32, i32 } { i32 ptrtoint ([0 x i32]* @"__C$0.2" to i32), i32 1 }
+} // namespace test4

clang/test/CodeGenCXX/interleaving-virtual-base.cpp

+// This file tests if accesses to vbase entries are correctly generated when vtable interleaving is enabled.
+// This test is based on virtual-base-cast.cpp.
+
+// RUN: %clang_cc1 -flto -fvisibility hidden  -fsanitize=cfi-vcall -vtable-interleaving -emit-llvm %s -o - -triple i686-pc-linux-gnu | FileCheck %s
+
+struct A {
+  int a;
+  virtual int aa();
+};
+struct B {
+  int b;
+  virtual int bb();
+};
+struct C : virtual A, virtual B {
+  int c;
+  virtual int aa();
+  virtual int bb();
+};
+struct AA {
+  int a;
+  virtual int aa();
+};
+struct BB {
+  int b;
+  virtual int bb();
+};
+struct CC : AA, BB {
+  virtual int aa();
+  virtual int bb();
+  virtual int cc();
+};
+struct D : virtual C, virtual CC {
+  int e;
+};
+
+D *x;
+
+A *a() { return x; }
+// CHECK: @_Z1av() [[NUW:#[0-9]+]]
+// CHECK: [[VBASEOFFSETPTRA:%[a-zA-Z0-9\.]+]] = getelementptr inbounds i8, i8* {{.*}}, i64 ptrtoint ([0 x i32]* @"__D$-16" to i64)
+// CHECK: [[CASTVBASEOFFSETPTRA:%[a-zA-Z0-9\.]+]] = bitcast i8* [[VBASEOFFSETPTRA]] to i32*
+// CHECK: load i32, i32* [[CASTVBASEOFFSETPTRA]]
+// CHECK: }
+
+B *b() { return x; }
+// CHECK: @_Z1bv() [[NUW]]
+// CHECK: [[VBASEOFFSETPTRA:%[a-zA-Z0-9\.]+]] = getelementptr inbounds i8, i8* {{.*}}, i64 ptrtoint ([0 x i32]* @"__D$-20" to i64)
+// CHECK: [[CASTVBASEOFFSETPTRA:%[a-zA-Z0-9\.]+]] = bitcast i8* [[VBASEOFFSETPTRA]] to i32*
+// CHECK: load i32, i32* [[CASTVBASEOFFSETPTRA]]
+// CHECK: }
+
+BB *c() { return x; }
+// CHECK: @_Z1cv() [[NUW]]
+// CHECK: [[VBASEOFFSETPTRC:%[a-zA-Z0-9\.]+]] = getelementptr inbounds i8, i8* {{.*}}, i64 ptrtoint ([0 x i32]* @"__D$-24" to i64)
+// CHECK: [[CASTVBASEOFFSETPTRC:%[a-zA-Z0-9\.]+]] = bitcast i8* [[VBASEOFFSETPTRC]] to i32*
+// CHECK: [[VBASEOFFSETC:%[a-zA-Z0-9\.]+]] = load i32, i32* [[CASTVBASEOFFSETPTRC]]
+// CHECK: add i32 [[VBASEOFFSETC]], 8
+// CHECK: }
+
+// Put the vbptr at a non-zero offset inside a non-virtual base.
+struct E {
+  int e;
+};
+struct F : E, D {
+  int f;
+};
+
+F *y;
+
+BB *d() { return y; }
+
+// CHECK: attributes [[NUW]] = { noinline nounwind{{.*}} }

clang/test/CodeGenCXX/interleaving-virtual-calls.cpp

+// This file tests if accesses to virtual function pointer entries are correctly generated when vtable interleaving is enabled.
+
+// RUN: %clang_cc1 -vtable-interleaving -flto -fvisibility hidden -fsanitize=cfi-vcall -emit-llvm %s -o - -triple x86_64-unknown-linux | FileCheck %s
+
+struct A {
+  int a;
+  virtual int foo1();
+  virtual int foo2();
+};
+struct B : A {
+  int b;
+  virtual int foo1();
+};
+struct C {
+  int c;
+  virtual int foo3();
+  virtual int foo4();
+};
+struct D : B, C {
+  virtual int foo2();
+  virtual int foo3();
+};
+
+// CHECK: @_Z5test1P1A
+int test1(A *a) {
+  // CHECK: [[R0:%.*]] = bitcast %struct.A* {{.*}} to i8**
+  // CHECK: [[VTABLE:%.*]] = load i8*, i8** [[R0]]
+  // CHECK: getelementptr inbounds i8, i8* [[VTABLE]], i32 ptrtoint ([0 x i32]* @"__A$0" to i32)
+  return a->foo1();
+}
+
+// CHECK: @_Z5test2P1C
+int test2(C *c) {
+  // CHECK: [[R0:%.*]] = bitcast %struct.C* {{.*}} to i8**
+  // CHECK: [[VTABLE:%.*]] = load i8*, i8** [[R0]]
+  // CHECK: getelementptr inbounds i8, i8* [[VTABLE]], i32 ptrtoint ([0 x i32]* @"__C$8" to i32)
+  return c->foo4();
+}

clang/test/CodeGenCXX/type-metadata.cpp

 // Tests for the cfi-vcall feature:
 // RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux -fvisibility hidden -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT --check-prefix=ITANIUM --check-prefix=TT-ITANIUM --check-prefix=NDIAG %s
 // RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux -fvisibility hidden -fsanitize=cfi-vcall -emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT --check-prefix=ITANIUM --check-prefix=TT-ITANIUM --check-prefix=ITANIUM-DIAG --check-prefix=DIAG --check-prefix=DIAG-ABORT %s
 // RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux -fvisibility hidden -fsanitize=cfi-vcall -fsanitize-recover=cfi-vcall -emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT --check-prefix=ITANIUM --check-prefix=TT-ITANIUM --check-prefix=ITANIUM-DIAG --check-prefix=DIAG --check-prefix=DIAG-RECOVER %s
 // RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT --check-prefix=MS --check-prefix=TT-MS --check-prefix=NDIAG %s
 
 // Tests for the whole-program-vtables feature:
 // RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux -fvisibility hidden -fwhole-program-vtables -emit-llvm -o - %s | FileCheck --check-prefix=VTABLE-OPT --check-prefix=ITANIUM --check-prefix=TT-ITANIUM %s
 // RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc -fwhole-program-vtables -emit-llvm -o - %s | FileCheck --check-prefix=VTABLE-OPT --check-prefix=MS --check-prefix=TT-MS %s
 
 // Tests for cfi + whole-program-vtables:
 // RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux -fvisibility hidden -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -fwhole-program-vtables -emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-VT --check-prefix=ITANIUM --check-prefix=TC-ITANIUM %s
 // RUN: %clang_cc1 -flto -flto-unit -triple x86_64-pc-windows-msvc -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -fwhole-program-vtables -emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-VT --check-prefix=MS --check-prefix=TC-MS %s
 
+// Tests for the cfi-vcall with vtable interleaving:
+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux -fvisibility hidden -fsanitize=cfi-vcall -fsanitize-trap=cfi-vcall -vtable-interleaving -emit-llvm -o - %s | FileCheck --check-prefix=CFI --check-prefix=CFI-NVT --check-prefix=INTER --check-prefix=TT-ITANIUM --check-prefix=NDIAG %s
+
 // ITANIUM: @_ZTV1A = {{[^!]*}}, !type [[A16:![0-9]+]]
 // ITANIUM-DIAG-SAME: !type [[ALL16:![0-9]+]]
 // ITANIUM-SAME: !type [[AF16:![0-9]+]]
 
+// INTER: @_ZTV1A = {{[^!]*}}, !type [[A16:![0-9]+]]
+
 // ITANIUM: @_ZTV1B = {{[^!]*}}, !type [[A32:![0-9]+]]
 // ITANIUM-DIAG-SAME: !type [[ALL32:![0-9]+]]
 // ITANIUM-SAME: !type [[AF32:![0-9]+]]
 // ITANIUM-SAME: !type [[AF40:![0-9]+]]
 // ITANIUM-SAME: !type [[AF48:![0-9]+]]
 // ITANIUM-SAME: !type [[B32:![0-9]+]]
 // ITANIUM-DIAG-SAME: !type [[ALL32]]
 // ITANIUM-SAME: !type [[BF32:![0-9]+]]
 // ITANIUM-SAME: !type [[BF40:![0-9]+]]
 // ITANIUM-SAME: !type [[BF48:![0-9]+]]
 
+// INTER: @_ZTV1B = {{[^!]*}}, !type [[A32:![0-9]+]]
+// INTER-SAME: !type [[A_VCALL_OFFSET32:![0-9]+]]
+// INTER-SAME: !type [[B32:![0-9]+]]
+
 // ITANIUM: @_ZTV1C = {{[^!]*}}, !type [[A32]]
 // ITANIUM-DIAG-SAME: !type [[ALL32]]
 // ITANIUM-SAME: !type [[AF32]]
 // ITANIUM-SAME: !type [[C32:![0-9]+]]
 // ITANIUM-DIAG-SAME: !type [[ALL32]]
 // ITANIUM-SAME: !type [[CF32:![0-9]+]]
 
+// INTER: @_ZTV1C = {{[^!]*}}, !type [[A32]]
+// INTER-SAME: !type [[A_VCALL_OFFSET32]]
+// INTER-SAME: !type [[C32_4:![0-9]+]]
+
 // DIAG: @[[SRC:.*]] = private unnamed_addr constant [{{.*}} x i8] c"{{.*}}type-metadata.cpp\00", align 1
 // DIAG: @[[TYPE:.*]] = private unnamed_addr constant { i16, i16, [4 x i8] } { i16 -1, i16 0, [4 x i8] c"'A'\00" }
 // DIAG: @[[BADTYPESTATIC:.*]] = private unnamed_addr global { i8, { [{{.*}} x i8]*, i32, i32 }, { i16, i16, [4 x i8] }* } { i8 0, { [{{.*}} x i8]*, i32, i32 } { [{{.*}} x i8]* @[[SRC]], i32 123, i32 3 }, { i16, i16, [4 x i8] }* @[[TYPE]] }
 
 // ITANIUM: @_ZTVN12_GLOBAL__N_11DE = {{[^!]*}}, !type [[A32]]
 // ITANIUM-DIAG-SAME: !type [[ALL32]]
 // ITANIUM-SAME: !type [[AF32]]
 // ITANIUM-SAME: !type [[AF40]]

 // ITANIUM-SAME: !type [[CF40:![0-9]+]]
 // ITANIUM-SAME: !type [[CF48:![0-9]+]]
 // ITANIUM-SAME: !type [[D32:![0-9]+]]
 // ITANIUM-DIAG-SAME: !type [[ALL32]]
 // ITANIUM-SAME: !type [[DF32:![0-9]+]]
 // ITANIUM-SAME: !type [[DF40:![0-9]+]]
 // ITANIUM-SAME: !type [[DF48:![0-9]+]]
 
+// INTER: @_ZTVN12_GLOBAL__N_11DE = {{[^!]*}}, !type [[A32]]
+// INTER-SAME: !type [[A_VCALL_OFFSET32]]
+// INTER-SAME: !type [[B32]]
+// INTER-SAME: !type [[D32:![0-9]+]]
+// INTER-SAME: !type [[C88:![0-9]+]]
+
 // ITANIUM: @_ZTCN12_GLOBAL__N_11DE0_1B = {{[^!]*}}, !type [[A32]]
 // ITANIUM-DIAG-SAME: !type [[ALL32]]
 // ITANIUM-SAME: !type [[B32]]
 // ITANIUM-DIAG-SAME: !type [[ALL32]]
 
+// INTER: @_ZTCN12_GLOBAL__N_11DE0_1B = {{[^!]*}}, !type [[A32]]
+// INTER-SAME: !type [[A_VCALL_OFFSET32]]
+// INTER-SAME: !type [[B32]]
+
 // ITANIUM: @_ZTCN12_GLOBAL__N_11DE8_1C = {{[^!]*}}, !type [[A64:![0-9]+]]
 // ITANIUM-DIAG-SAME: !type [[ALL64:![0-9]+]]
 // ITANIUM-SAME: !type [[AF64:![0-9]+]]
 // ITANIUM-SAME: !type [[C32]]
 // ITANIUM-DIAG-SAME: !type [[ALL32]]
 // ITANIUM-SAME: !type [[CF64:![0-9]+]]
 
+// INTER: @_ZTCN12_GLOBAL__N_11DE8_1C = {{[^!]*}}, !type [[C32_2:![0-9]+]]
+// INTER-SAME: !type [[A64:![0-9]+]]
+// INTER-SAME: !type [[A_VCALL_OFFSET64:![0-9]+]]
+
 // ITANIUM: @_ZTVZ3foovE2FA = {{[^!]*}}, !type [[A16]]
 // ITANIUM-DIAG-SAME: !type [[ALL16]]
 // ITANIUM-SAME: !type [[AF16]]
 // ITANIUM-SAME: !type [[FA16:![0-9]+]]
 // ITANIUM-DIAG-SAME: !type [[ALL16]]
 // ITANIUM-SAME: !type [[FAF16:![0-9]+]]
 
+// INTER: @_ZTVZ3foovE2FA = {{[^!]*}}, !type [[A16]]
+// INTER-SAME: !type [[FA16:![0-9]+]]
+
 // MS: comdat($"??_7A@@6B@"), !type [[A8:![0-9]+]]
 // MS: comdat($"??_7B@@6B0@@"), !type [[B8:![0-9]+]]
 // MS: comdat($"??_7B@@6BA@@@"), !type [[A8]]
 // MS: comdat($"??_7C@@6B@"), !type [[A8]]
 // MS: comdat($"??_7D@?A0x{{[^@]*}}@@6BB@@@"), !type [[B8]], !type [[D8:![0-9]+]]
 // MS: comdat($"??_7D@?A0x{{[^@]*}}@@6BA@@@"), !type [[A8]]
 // MS: comdat($"??_7FA@?1??foo@@YAXXZ@6B@"), !type [[A8]], !type [[FA8:![0-9]+]]
 

 
 void D::f() {
 }
 
 void D::h() {
 }
 
 // ITANIUM: define hidden void @_Z2afP1A
+// INTER: define hidden void @_Z2afP1A
 // MS: define dso_local void @"?af@@YAXPEAUA@@@Z"
 void af(A *a) {
   // TT-ITANIUM: [[P:%[^ ]*]] = call i1 @llvm.type.test(i8* [[VT:%[^ ]*]], metadata !"_ZTS1A")
   // TT-MS: [[P:%[^ ]*]] = call i1 @llvm.type.test(i8* [[VT:%[^ ]*]], metadata !"?AUA@@")
   // TC-ITANIUM: [[PAIR:%[^ ]*]] = call { i8*, i1 } @llvm.type.checked.load(i8* {{%[^ ]*}}, i32 0, metadata !"_ZTS1A")
   // TC-MS: [[PAIR:%[^ ]*]] = call { i8*, i1 } @llvm.type.checked.load(i8* {{%[^ ]*}}, i32 0, metadata !"?AUA@@")
   // CFI-VT: [[P:%[^ ]*]] = extractvalue { i8*, i1 } [[PAIR]], 1
   // DIAG-NEXT: [[VTVALID0:%[^ ]*]] = call i1 @llvm.type.test(i8* [[VT]], metadata !"all-vtables")
void af(A *a) {
   // CFI-VT: [[PTRI8:%[^ ]*]] = extractvalue { i8*, i1 } [[PAIR]], 0
   // CFI-VT: [[PTR:%[^ ]*]] = bitcast i8* [[PTRI8]] to
   // CFI: call void [[PTR]]
 #line 123
   a->f();
 }
 
 // ITANIUM: define internal void @_Z3df1PN12_GLOBAL__N_11DE
+// INTER: define internal void @_Z3df1PN12_GLOBAL__N_11DE
 // MS: define internal void @"?df1@@YAXPEAUD@?A0x{{[^@]*}}@@@Z"
 void df1(D *d) {
   // TT-ITANIUM: {{%[^ ]*}} = call i1 @llvm.type.test(i8* {{%[^ ]*}}, metadata ![[DTYPE:[0-9]+]])
   // TT-MS: {{%[^ ]*}} = call i1 @llvm.type.test(i8* {{%[^ ]*}}, metadata !"?AUA@@")
   // TC-ITANIUM: {{%[^ ]*}} = call { i8*, i1 } @llvm.type.checked.load(i8* {{%[^ ]*}}, i32 0, metadata ![[DTYPE:[0-9]+]])
   // TC-MS: {{%[^ ]*}} = call { i8*, i1 } @llvm.type.checked.load(i8* {{%[^ ]*}}, i32 0, metadata !"?AUA@@")
   d->f();
 }
 
 // ITANIUM: define internal void @_Z3dg1PN12_GLOBAL__N_11DE
+// INTER: define internal void @_Z3dg1PN12_GLOBAL__N_11DE
 // MS: define internal void @"?dg1@@YAXPEAUD@?A0x{{[^@]*}}@@@Z"
 void dg1(D *d) {
   // TT-ITANIUM: {{%[^ ]*}} = call i1 @llvm.type.test(i8* {{%[^ ]*}}, metadata !"_ZTS1B")
   // TT-MS: {{%[^ ]*}} = call i1 @llvm.type.test(i8* {{%[^ ]*}}, metadata !"?AUB@@")
   // TC-ITANIUM: {{%[^ ]*}} = call { i8*, i1 } @llvm.type.checked.load(i8* {{%[^ ]*}}, i32 8, metadata !"_ZTS1B")
   // TC-MS: {{%[^ ]*}} = call { i8*, i1 } @llvm.type.checked.load(i8* {{%[^ ]*}}, i32 0, metadata !"?AUB@@")
   d->g();
 }
 
 // ITANIUM: define internal void @_Z3dh1PN12_GLOBAL__N_11DE
+// INTER: define internal void @_Z3dh1PN12_GLOBAL__N_11DE
 // MS: define internal void @"?dh1@@YAXPEAUD@?A0x{{[^@]*}}@@@Z"
 void dh1(D *d) {
   // TT-ITANIUM: {{%[^ ]*}} = call i1 @llvm.type.test(i8* {{%[^ ]*}}, metadata ![[DTYPE]])
   // TT-MS: {{%[^ ]*}} = call i1 @llvm.type.test(i8* {{%[^ ]*}}, metadata ![[DTYPE:[0-9]+]])
   // TC-ITANIUM: {{%[^ ]*}} = call { i8*, i1 } @llvm.type.checked.load(i8* {{%[^ ]*}}, i32 16, metadata ![[DTYPE]])
   // TC-MS: {{%[^ ]*}} = call { i8*, i1 } @llvm.type.checked.load(i8* {{%[^ ]*}}, i32 8, metadata ![[DTYPE:[0-9]+]])
   d->h();
 }
 
 // ITANIUM: define internal void @_Z3df2PN12_GLOBAL__N_11DE
+// INTER: define internal void @_Z3df2PN12_GLOBAL__N_11DE
 // MS: define internal void @"?df2@@YAXPEAUD@?A0x{{[^@]*}}@@@Z"
 __attribute__((no_sanitize("cfi")))
 void df2(D *d) {
   // CFI-NVT-NOT: call i1 @llvm.type.test
   // CFI-VT: [[P:%[^ ]*]] = call i1 @llvm.type.test
   // CFI-VT: call void @llvm.assume(i1 [[P]])
   d->f();
 }
 
 // ITANIUM: define internal void @_Z3df3PN12_GLOBAL__N_11DE
+// INTER: define internal void @_Z3df3PN12_GLOBAL__N_11DE
 // MS: define internal void @"?df3@@YAXPEAUD@?A0x{{[^@]*}}@@@Z"
 __attribute__((no_sanitize("address"))) __attribute__((no_sanitize("cfi-vcall")))
 void df3(D *d) {
   // CFI-NVT-NOT: call i1 @llvm.type.test
   // CFI-VT: [[P:%[^ ]*]] = call i1 @llvm.type.test
   // CFI-VT: call void @llvm.assume(i1 [[P]])
   d->f();
 }
struct B {
   virtual void m_fn2();
 };
 struct C : B, A {};
 struct D : C {
   void m_fn1();
 };
 
 // ITANIUM: define hidden void @_ZN5test21fEPNS_1DE
+// INTER: define hidden void @_ZN5test21fEPNS_1DE
 // MS: define dso_local void @"?f@test2@@YAXPEAUD@1@@Z"
 void f(D *d) {
   // TT-ITANIUM: {{%[^ ]*}} = call i1 @llvm.type.test(i8* {{%[^ ]*}}, metadata !"_ZTSN5test21DE")
   // TT-MS: {{%[^ ]*}} = call i1 @llvm.type.test(i8* {{%[^ ]*}}, metadata !"?AUA@test2@@")
   // TC-ITANIUM: {{%[^ ]*}} = call { i8*, i1 } @llvm.type.checked.load(i8* {{%[^ ]*}}, i32 8, metadata !"_ZTSN5test21DE")
   // TC-MS: {{%[^ ]*}} = call { i8*, i1 } @llvm.type.checked.load(i8* {{%[^ ]*}}, i32 0, metadata !"?AUA@test2@@")
   d->m_fn1();
 }

 // ITANIUM-DIAG: [[ALL64]] = !{i64 64, !"all-vtables"}
 // ITANIUM: [[AF64]] = !{i64 64, !"_ZTSM1AFvvE.virtual"}
 // ITANIUM: [[CF64]] = !{i64 64, !"_ZTSM1CFvvE.virtual"}
 // ITANIUM: [[FA16]] = !{i64 16, [[FA_ID:![0-9]+]]}
 // ITANIUM: [[FA_ID]] = distinct !{}
 // ITANIUM: [[FAF16]] = !{i64 16, [[FAF_ID:![0-9]+]]}
 // ITANIUM: [[FAF_ID]] = distinct !{}
 
+// INTER: [[A16]] = !{i64 16, !"_ZTS1A", i64 2}
+// INTER: [[A32]] = !{i64 32, !"_ZTS1A", i64 2}
+// INTER: [[A_VCALL_OFFSET32]] = !{i64 32, !"_ZTS1A.vcall_offset", i64 3}
+// INTER: [[B32]] = !{i64 32, !"_ZTS1B", i64 4}
+// INTER: [[C32_4]] = !{i64 32, !"_ZTS1C", i64 4}
+// INTER: [[D32]] = !{i64 32, [[D_ID:![0-9]+]], i64 5}
+// INTER: [[D_ID]] = distinct !{}
+// INTER: [[C88]] = !{i64 88, !"_ZTS1C", i64 2}
+// INTER: [[C32_2]] = !{i64 32, !"_ZTS1C", i64 2}
+// INTER: [[A64]] = !{i64 64, !"_ZTS1A", i64 2}
+// INTER: [[A_VCALL_OFFSET64]] = !{i64 64, !"_ZTS1A.vcall_offset", i64 3}
+// INTER: [[FA16]] = !{i64 16, [[FA_ID:![0-9]+]], i64 3}
+// INTER: [[FA_ID]] = distinct !{}
+
 // MS: [[A8]] = !{i64 8, !"?AUA@@"}
 // MS: [[B8]] = !{i64 8, !"?AUB@@"}
 // MS: [[D8]] = !{i64 8, [[D_ID:![0-9]+]]}
 // MS: [[D_ID]] = distinct !{}
 // MS: [[FA8]] = !{i64 8, [[FA_ID:![0-9]+]]}
 // MS: [[FA_ID]] = distinct !{}

clang/test/CodeGenCXX/ubsan-vtable-checks.cpp

 // MSABI: define dso_local i32 @"?get_v
 int get_v(T* t) {
   // First, we check that vtable is not loaded before a type check.
   // CHECK-NULL-NOT: load {{.*}} (%struct.T*{{.*}})**, {{.*}} (%struct.T*{{.*}})***
   // CHECK-NULL: [[UBSAN_CMP_RES:%[0-9]+]] = icmp ne %struct.T* %{{[_a-z0-9]+}}, null
   // CHECK-NULL-NEXT: br i1 [[UBSAN_CMP_RES]], label %{{.*}}, label %{{.*}}
   // CHECK-NULL: call void @__ubsan_handle_type_mismatch_v1_abort
   // Second, we check that vtable is actually loaded once the type check is done.
-  // CHECK-NULL: load {{.*}} (%struct.T*{{.*}})**, {{.*}} (%struct.T*{{.*}})***
+  // ITANIUM: [[R0:%.*]] = bitcast %struct.T* {{.*}} to i8**
+  // ITANIUM-NEXT: load i8*, i8** [[R0]]
+  // MSABI: [[R0:%.*]] = bitcast %struct.T* {{.*}} to i32 (%struct.T*)***
+  // MSABI-NEXT: load i32 (%struct.T*)**, i32 (%struct.T*)*** [[R0]]
   return t->v();
 }
 
 // ITANIUM: define void @_Z9delete_itP1T
 // MSABI: define dso_local void @"?delete_it
 void delete_it(T *t) {
   // First, we check that vtable is not loaded before a type check.
   // CHECK-VPTR-NOT: load {{.*}} (%struct.T*{{.*}})**, {{.*}} (%struct.T*{{.*}})***
   // CHECK-VPTR: br i1 {{.*}} label %{{.*}}
   // CHECK-VPTR: call void @__ubsan_handle_dynamic_type_cache_miss_abort
   // Second, we check that vtable is actually loaded once the type check is done.
-  // CHECK-VPTR: load {{.*}} (%struct.T*{{.*}})**, {{.*}} (%struct.T*{{.*}})***
+  // ITANIUM: [[vtable:%.*]] = bitcast %struct.T* {{.*}} to i8**
+  // ITANIUM-NEXT: load i8*, i8** [[vtable]]
+  // MSABI: [[R0:%.*]] = bitcast %struct.T* {{.*}} to i8* (%struct.T*, i32)***
+  // MSABI-NEXT: load i8* (%struct.T*, i32)**, i8* (%struct.T*, i32)*** [[R0]]
   delete t;
 }
 
 // ITANIUM: define %struct.U* @_Z7dyncastP1T
 // MSABI: define dso_local %struct.U* @"?dyncast
 U* dyncast(T *t) {
   // First, we check that dynamic_cast is not called before a type check.
   // CHECK-VPTR-NOT: call i8* @__{{dynamic_cast|RTDynamicCast}}
   // CHECK-VPTR: br i1 {{.*}} label %{{.*}}
   // CHECK-VPTR: call void @__ubsan_handle_dynamic_type_cache_miss_abort
   // Second, we check that dynamic_cast is actually called once the type check is done.
   // CHECK-VPTR: call i8* @__{{dynamic_cast|RTDynamicCast}}
   return dynamic_cast<U*>(t);
 }

clang/test/CodeGenCXX/virtual-base-cast.cpp

 // RUN: %clang_cc1 -emit-llvm %s -o - -triple i686-pc-linux-gnu | FileCheck %s
 // RUN: %clang_cc1 -emit-llvm %s -o - -triple i686-pc-win32 | FileCheck -check-prefix MSVC %s
 
 struct A { int a; virtual int aa(); };
 struct B { int b; virtual int bb(); };
 struct C : virtual A, virtual B { int c; virtual int aa(); virtual int bb(); };
 struct AA { int a; virtual int aa(); };
 struct BB { int b; virtual int bb(); };
 struct CC : AA, BB { virtual int aa(); virtual int bb(); virtual int cc(); };
 struct D : virtual C, virtual CC { int e; };
 
 D* x;
 
 A* a() { return x; }
 // CHECK: @_Z1av() [[NUW:#[0-9]+]]
-// CHECK: [[VBASEOFFSETPTRA:%[a-zA-Z0-9\.]+]] = getelementptr i8, i8* {{.*}}, i64 -16
+// CHECK: [[VBASEOFFSETPTRA:%[a-zA-Z0-9\.]+]] = getelementptr inbounds i8, i8* {{.*}}, i64 -16
 // CHECK: [[CASTVBASEOFFSETPTRA:%[a-zA-Z0-9\.]+]] = bitcast i8* [[VBASEOFFSETPTRA]] to i32*
 // CHECK: load i32, i32* [[CASTVBASEOFFSETPTRA]]
 // CHECK: }
 
 // MSVC: @"?a@@YAPAUA@@XZ"() [[NUW:#[0-9]+]] {
 // MSVC:   %[[vbptr_off:.*]] = getelementptr inbounds i8, i8* {{.*}}, i32 0
 // MSVC:   %[[vbptr:.*]] = bitcast i8* %[[vbptr_off]] to i32**
 // MSVC:   %[[vbtable:.*]] = load i32*, i32** %[[vbptr]]
 // MSVC:   %[[entry:.*]] = getelementptr inbounds i32, i32* {{.*}}, i32 1
 // MSVC:   %[[offset:.*]] = load i32, i32* %[[entry]]
 // MSVC:   add nsw i32 0, %[[offset]]
 // MSVC: }
 
 B* b() { return x; }
 // CHECK: @_Z1bv() [[NUW]]
-// CHECK: [[VBASEOFFSETPTRA:%[a-zA-Z0-9\.]+]] = getelementptr i8, i8* {{.*}}, i64 -20
+// CHECK: [[VBASEOFFSETPTRA:%[a-zA-Z0-9\.]+]] = getelementptr inbounds i8, i8* {{.*}}, i64 -20
 // CHECK: [[CASTVBASEOFFSETPTRA:%[a-zA-Z0-9\.]+]] = bitcast i8* [[VBASEOFFSETPTRA]] to i32*
 // CHECK: load i32, i32* [[CASTVBASEOFFSETPTRA]]
 // CHECK: }
 
 // Same as 'a' except we use a different vbtable offset.
 // MSVC: @"?b@@YAPAUB@@XZ"() [[NUW:#[0-9]+]] {
 // MSVC:   %[[vbptr_off:.*]] = getelementptr inbounds i8, i8* {{.*}}, i32 0
 // MSVC:   %[[vbptr:.*]] = bitcast i8* %[[vbptr_off]] to i32**
 // MSVC:   %[[vbtable:.*]] = load i32*, i32** %[[vbptr]]
 // MSVC:   %[[entry:.*]] = getelementptr inbounds i32, i32* {{.*}}, i32 2
 // MSVC:   %[[offset:.*]] = load i32, i32* %[[entry]]
 // MSVC:   add nsw i32 0, %[[offset]]
 // MSVC: }
 
 
 BB* c() { return x; }
 // CHECK: @_Z1cv() [[NUW]]
-// CHECK: [[VBASEOFFSETPTRC:%[a-zA-Z0-9\.]+]] = getelementptr i8, i8* {{.*}}, i64 -24
+// CHECK: [[VBASEOFFSETPTRC:%[a-zA-Z0-9\.]+]] = getelementptr inbounds i8, i8* {{.*}}, i64 -24
 // CHECK: [[CASTVBASEOFFSETPTRC:%[a-zA-Z0-9\.]+]] = bitcast i8* [[VBASEOFFSETPTRC]] to i32*
 // CHECK: [[VBASEOFFSETC:%[a-zA-Z0-9\.]+]] = load i32, i32* [[CASTVBASEOFFSETPTRC]]
 // CHECK: add i32 [[VBASEOFFSETC]], 8
 // CHECK: }
 
 // Same as 'a' except we use a different vbtable offset.
 // MSVC: @"?c@@YAPAUBB@@XZ"() [[NUW:#[0-9]+]] {
 // MSVC:   %[[vbptr_off:.*]] = getelementptr inbounds i8, i8* {{.*}}, i32 0

compiler-rt/test/cfi/CMakeLists.txt

 set(CFI_TESTSUITES)
 
-macro (add_cfi_test_suites lld thinlto newpm)
+macro (add_cfi_test_suites lld thinlto newpm interleaving)
   set(suffix)
   if (${lld})
     set(suffix ${suffix}-lld)
   endif()
   if (${thinlto})
     set(suffix ${suffix}-thinlto)
   endif()
   if (${newpm})
     set(suffix ${suffix}-newpm)
   endif()
+  if (${interleaving})
+    set(suffix ${suffix}-interleaving)
+  endif()
   set(suffix ${suffix}-${CFI_TEST_TARGET_ARCH})
 
   set(CFI_TEST_USE_LLD ${lld})
   set(CFI_TEST_USE_THINLTO ${thinlto})
   set(CFI_TEST_USE_NEWPM ${newpm})
+  set(CFI_TEST_USE_INTERLEAVING ${interleaving})
 
   set(CFI_LIT_TEST_MODE Standalone)
   set(CFI_TEST_CONFIG_SUFFIX -standalone${suffix})
   configure_lit_site_cfg(
     ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in
     ${CMAKE_CURRENT_BINARY_DIR}/Standalone${suffix}/lit.site.cfg.py
     )
   list(APPEND CFI_TESTSUITES ${CMAKE_CURRENT_BINARY_DIR}/Standalone${suffix})
if(APPLE)
   darwin_filter_host_archs(CFI_SUPPORTED_ARCH CFI_TEST_ARCH)
 endif()
 
 foreach(arch ${CFI_TEST_ARCH})
   set(CFI_TEST_TARGET_ARCH ${arch})
   get_test_cc_for_arch(${arch} CFI_TEST_TARGET_CC CFI_TEST_TARGET_CFLAGS)
   if (APPLE)
     # FIXME: enable ThinLTO tests after fixing http://llvm.org/pr32741
-    add_cfi_test_suites(False False False)
+    add_cfi_test_suites(False False False False)
   elseif(WIN32)
-    add_cfi_test_suites(True False False)
-    add_cfi_test_suites(True True False)
+    add_cfi_test_suites(True False False False)
+    add_cfi_test_suites(True True False False)
   else()
-    add_cfi_test_suites(False False False)
-    add_cfi_test_suites(False True False)
-    add_cfi_test_suites(False False True)
-    add_cfi_test_suites(False True True)
-    if (COMPILER_RT_HAS_LLD AND NOT arch STREQUAL "i386")
-      add_cfi_test_suites(True False False)
-      add_cfi_test_suites(True True False)
-    endif()
+    add_cfi_test_suites(False False False False)
+    add_cfi_test_suites(False True False False)
+    add_cfi_test_suites(False False True False)
+    add_cfi_test_suites(False True True False)   
+    add_cfi_test_suites(True False False False)
+  add_cfi_test_suites(True True False False)
+  add_cfi_test_suites(True False False True)
+  add_cfi_test_suites(False False False True)
   endif()
 endforeach()
 
 set(CFI_TEST_DEPS ${SANITIZER_COMMON_LIT_TEST_DEPS})
 list(APPEND CFI_TEST_DEPS
   ubsan
   stats
 )

compiler-rt/test/cfi/anon-namespace.cpp

 // RUN: %clangxx -c -DTU1 -o %t1.o %s
 // RUN: %clangxx -c -DTU2 -o %t2.o %S/../cfi/anon-namespace.cpp
 // RUN: %clangxx -o %t5 %t1.o %t2.o
 // RUN: %run %t5 2>&1 | FileCheck --check-prefix=NCFI %s
 
 // RUN: %clangxx_cfi_diag -c -DTU1 -o %t1.o %s
 // RUN: %clangxx_cfi_diag -c -DTU2 -o %t2.o %S/../cfi/anon-namespace.cpp
 // RUN: %clangxx_cfi_diag -o %t6 %t1.o %t2.o
-// RUN: %run %t6 2>&1 | FileCheck --check-prefix=CFI-DIAG %s
+// RUN: %interleave_diag %run %t6 2>&1 | FileCheck --check-prefix=CFI-DIAG %s
 
 // Tests that the CFI mechanism treats classes in the anonymous namespace in
 // different translation units as having distinct identities. This is done by
 // compiling two translation units TU1 and TU2 containing a class named B in an
 // anonymous namespace, and testing that the program crashes if TU2 attempts to
 // use a TU1 B as a TU2 B.
 
 // FIXME: This test should not require that the paths supplied to the compiler

compiler-rt/test/cfi/create-derivers.test

 REQUIRES: asserts
+UNSUPPORTED: interleaving
 
 %% Explicit -flto to override possible -flto=thin in %clangxx_cfi
 RUN: %clangxx_cfi -flto -c -o %t1.o %S/simple-fail.cpp
-RUN: opt -lowertypetests -debug-only=lowertypetests -o /dev/null %t1.o 2>&1 | FileCheck --check-prefix=B0 %s
+RUN: opt -%opt_flag -debug-only=%opt_flag -o /dev/null %t1.o 2>&1 | FileCheck --check-prefix=B0 %s
 B0: {{1B|B@@}}: {{.*}} size 1
 
 RUN: %clangxx_cfi -DB32 -flto -c -o %t2.o %S/simple-fail.cpp
-RUN: opt -lowertypetests -debug-only=lowertypetests -o /dev/null %t2.o 2>&1 | FileCheck --check-prefix=B32 %s
+RUN: opt -%opt_flag -debug-only=%opt_flag -o /dev/null %t2.o 2>&1 | FileCheck --check-prefix=B32 %s
 B32: {{1B|B@@}}: {{.*}} size 2{{3|4}}
 B32-NOT: all-ones
 
 RUN: %clangxx_cfi -DB64 -flto -c -o %t3.o %S/simple-fail.cpp
-RUN: opt -lowertypetests -debug-only=lowertypetests -o /dev/null %t3.o 2>&1 | FileCheck --check-prefix=B64 %s
+RUN: opt -%opt_flag -debug-only=%opt_flag -o /dev/null %t3.o 2>&1 | FileCheck --check-prefix=B64 %s
 B64: {{1B|B@@}}: {{.*}} size 5{{3|4}}
 B64-NOT: all-ones
 
 RUN: %clangxx_cfi -DBM -flto -c -o %t4.o %S/simple-fail.cpp
-RUN: opt -lowertypetests -debug-only=lowertypetests -o /dev/null %t4.o 2>&1 | FileCheck --check-prefix=BM %s
+RUN: opt -%opt_flag -debug-only=%opt_flag -o /dev/null %t4.o 2>&1 | FileCheck --check-prefix=BM %s
 BM: {{1B|B@@}}: {{.*}} size 8{{3|4}}
 BM-NOT: all-ones

compiler-rt/test/cfi/cross-dso/icall/lit.local.cfg.py

 # The cfi-icall checker is only supported on x86 and x86_64 for now.
 if config.root.host_arch not in ['x86', 'x86_64']:
   config.unsupported = True
+
+if 'interleaving' in config.available_features:
+  config.unsupported = True  
 No newline at end of file

compiler-rt/test/cfi/cross-dso/lit.local.cfg.py

 def getRoot(config):
   if not config.parent:
     return config
   return getRoot(config.parent)
 
 root = getRoot(config)
 
 if root.host_os not in ['Linux', 'FreeBSD', 'NetBSD']:
   config.unsupported = True
 
 # Android O (API level 26) has support for cross-dso cfi in libdl.so.
 if config.android and 'android-26' not in config.available_features:
   config.unsupported = True
+
+if 'interleaving' in config.available_features:
+  config.unsupported = True  
 No newline at end of file

compiler-rt/test/cfi/lit.cfg.py

 config.substitutions.append((r"%clang ", clang + ' '))
 config.substitutions.append((r"%clangxx ", clangxx + ' '))
 if config.lto_supported:
   clang_cfi = clang + '-fsanitize=cfi '
 
   if config.cfi_lit_test_mode == "Devirt":
     config.available_features.add('devirt')
     clang_cfi += '-fwhole-program-vtables '
+    if config.use_interleaving:
+      config.substitutions.append((r"%expect_crash_unless_devirt ", config.expect_crash))
+    else:
       config.substitutions.append((r"%expect_crash_unless_devirt ", ""))
   else:
     config.substitutions.append((r"%expect_crash_unless_devirt ", config.expect_crash))
 
   cxx = ' '.join(config.cxx_mode_flags) + ' '
   diag = '-fno-sanitize-trap=cfi -fsanitize-recover=cfi '
   non_dso = '-fvisibility=hidden '
   dso = '-fsanitize-cfi-cross-dso -fvisibility=default '
+  interleave = '-Xclang -vtable-interleaving -Wl,-plugin-opt,-vtable-interleaving -fno-sanitize=cfi-mfcall '
   if config.android:
     dso += '-include ' + config.test_source_root + '/cross-dso/util/cfi_stubs.h '
+
   config.substitutions.append((r"%clang_cfi ", clang_cfi + non_dso))
-  config.substitutions.append((r"%clangxx_cfi ", clang_cfi + cxx + non_dso))
   config.substitutions.append((r"%clang_cfi_diag ", clang_cfi + non_dso + diag))
-  config.substitutions.append((r"%clangxx_cfi_diag ", clang_cfi + cxx + non_dso + diag))
+  
+  if config.use_interleaving:   
+    config.available_features.add('interleaving')
+    config.substitutions.append((r"%clangxx_cfi ", clang_cfi + cxx + non_dso + interleave))
+    config.substitutions.append((r"%clangxx_cfi_diag ", clang_cfi + cxx + non_dso + diag + interleave))
+    config.substitutions.append((r"%interleave_diag ", "not "))
+  else:
+    config.substitutions.append((r"%clangxx_cfi ", clang_cfi + cxx + non_dso))
     config.substitutions.append((r"%clangxx_cfi_dso ", clang_cfi + cxx + dso))
     config.substitutions.append((r"%clangxx_cfi_dso_diag ", clang_cfi + cxx + dso + diag))
+    config.substitutions.append((r"%clangxx_cfi_diag ", clang_cfi + cxx + non_dso + diag))
+    config.substitutions.append((r"%interleave_diag ", ""))
+    config.substitutions.append((r"%opt_flag ", "lowertypetests "))
+  
+  config.substitutions.append((r"%clangxx_cfi_no_interleaving ", clang_cfi + cxx + non_dso))
+  
   config.substitutions.append((r"%debug_info_flags", ' '.join(config.debug_info_flags)))
 else:
   config.unsupported = True
 
 if config.default_sanitizer_opts:
   config.environment['UBSAN_OPTIONS'] = ':'.join(config.default_sanitizer_opts)
 
 if lit_config.params.get('check_supported', None) and config.unsupported:
   raise BaseException("Tests unsupported")
+  

compiler-rt/test/cfi/lit.site.cfg.py.in

 @LIT_SITE_CFG_IN_HEADER@
 
 config.name_suffix = "@CFI_TEST_CONFIG_SUFFIX@"
 config.cfi_lit_test_mode = "@CFI_LIT_TEST_MODE@"
 config.target_arch = "@CFI_TEST_TARGET_ARCH@"
 config.target_cflags = "@CFI_TEST_TARGET_CFLAGS@"
 config.use_lld = @CFI_TEST_USE_LLD@
 config.use_lto = True # CFI *requires* LTO.
 config.use_thinlto = @CFI_TEST_USE_THINLTO@
 config.use_newpm = @CFI_TEST_USE_NEWPM@
+config.use_interleaving = @CFI_TEST_USE_INTERLEAVING@
 
 lit_config.load_config(config, "@COMPILER_RT_BINARY_DIR@/test/lit.common.configured")
 lit_config.load_config(config, "@CMAKE_CURRENT_SOURCE_DIR@/lit.cfg.py")

compiler-rt/test/cfi/mfcall.cpp

-// UNSUPPORTED: windows-msvc
+// UNSUPPORTED: windows-msvc, interleaving
 
 // RUN: %clangxx_cfi -o %t %s
 // RUN: %expect_crash %run %t a
 // RUN: %expect_crash %run %t b
 // RUN: %expect_crash %run %t c
 // RUN: %expect_crash %run %t d
 // RUN: %expect_crash %run %t e
 // RUN: %run %t f

compiler-rt/test/cfi/multiple-inheritance.cpp

 // RUN: %expect_crash %run %t4 2>&1 | FileCheck --check-prefix=CFI %s
 // RUN: %expect_crash %run %t4 x 2>&1 | FileCheck --check-prefix=CFI %s
 
 // RUN: %clangxx -o %t5 %s
 // RUN: %run %t5 2>&1 | FileCheck --check-prefix=NCFI %s
 // RUN: %run %t5 x 2>&1 | FileCheck --check-prefix=NCFI %s
 
 // RUN: %clangxx_cfi_diag -o %t6 %s
-// RUN: %run %t6 2>&1 | FileCheck --check-prefix=CFI-DIAG2 %s
-// RUN: %run %t6 x 2>&1 | FileCheck --check-prefix=CFI-DIAG1 %s
+// RUN: %interleave_diag %run %t6 2>&1 | FileCheck --check-prefix=CFI-DIAG2 %s
+// RUN: %interleave_diag %run %t6 x 2>&1 | FileCheck --check-prefix=CFI-DIAG1 %s
 
 // Tests that the CFI mechanism is sensitive to multiple inheritance and only
 // permits calls via virtual tables for the correct base class.
 
 // REQUIRES: cxxabi
 
 #include <stdio.h>
 #include "utils.h"

compiler-rt/test/cfi/simple-fail.cpp

 
 // RUN: %clangxx_cfi -O3 -DB64 -o %t15 %s
 // RUN: %expect_crash %run %t15 2>&1 | FileCheck --check-prefix=CFI %s
 
 // RUN: %clangxx_cfi -O3 -DBM -o %t16 %s
 // RUN: %expect_crash %run %t16 2>&1 | FileCheck --check-prefix=CFI %s
 
 // RUN: %clangxx_cfi_diag -o %t17 %s
-// RUN: %run %t17 2>&1 | FileCheck --check-prefix=CFI-DIAG %s
+// RUN: %interleave_diag %run %t17 2>&1 | FileCheck --check-prefix=CFI-DIAG %s
 
 // RUN: %clangxx -o %t18 %s
 // RUN: %run %t18 2>&1 | FileCheck --check-prefix=NCFI %s
 
-// RUN: %clangxx_cfi -DCHECK_NO_SANITIZE_CFI -o %t19 %s
+// RUN: %clangxx_cfi_no_interleaving -DCHECK_NO_SANITIZE_CFI -o %t19 %s
 // RUN: %run %t19 2>&1 | FileCheck --check-prefix=NCFI %s
 
 // Tests that the CFI mechanism crashes the program when making a virtual call
 // to an object of the wrong class but with a compatible vtable, by casting a
 // pointer to such an object and attempting to make a call through it.
 
 // REQUIRES: cxxabi
 
int main() {
   fprintf(stderr, "1\n");
 
   // CFI-DIAG: runtime error: control flow integrity check for type 'B' failed during cast to unrelated type
   // CFI-DIAG-NEXT: note: vtable is of type '{{(struct )?}}A'
   // CFI-DIAG: runtime error: control flow integrity check for type 'B' failed during virtual call
   // CFI-DIAG-NEXT: note: vtable is of type '{{(struct )?}}A'
   ((B *)a)->f(); // UB here
 
+  // INTERLEAVING-NCFI-NOT: {{^2$}}
   // CFI-NOT: {{^2$}}
   // NCFI: {{^2$}}
   fprintf(stderr, "2\n");
 }

compiler-rt/test/cfi/vdtor.cpp

 // RUN: %clangxx_cfi -o %t1 %s
 // RUN: %expect_crash %run %t1 2>&1 | FileCheck --check-prefix=CFI %s
 
 // RUN: %clangxx_cfi -DB32 -o %t2 %s
 // RUN: %expect_crash %run %t2 2>&1 | FileCheck --check-prefix=CFI %s
 
 // RUN: %clangxx_cfi -DB64 -o %t3 %s
 // RUN: %expect_crash %run %t3 2>&1 | FileCheck --check-prefix=CFI %s
 
 // RUN: %clangxx_cfi -DBM -o %t4 %s
 // RUN: %expect_crash %run %t4 2>&1 | FileCheck --check-prefix=CFI %s
 
 // RUN: %clangxx -o %t5 %s
 // RUN: %run %t5 2>&1 | FileCheck --check-prefix=NCFI %s
 
 // RUN: %clangxx_cfi_diag -o %t6 %s
-// RUN: %run %t6 2>&1 | FileCheck --check-prefix=CFI-DIAG %s
+// RUN: %interleave_diag %run %t6 2>&1 | FileCheck --check-prefix=CFI-DIAG %s
 
 // Tests that the CFI enforcement also applies to virtual destructor calls made
 // via 'delete'.
 
 // REQUIRES: cxxabi
 
 #include <stdio.h>
 #include "utils.h"

compiler-rt/test/lit.common.configured.in

 set_default("apple_platform", "osx")
 set_default("sanitizer_can_use_cxxabi", @SANITIZER_CAN_USE_CXXABI_PYBOOL@)
 set_default("has_lld", @COMPILER_RT_HAS_LLD_PYBOOL@)
 set_default("can_symbolize", @CAN_SYMBOLIZE@)
 set_default("use_lld", False)
 set_default("use_thinlto", False)
 set_default("use_lto", config.use_thinlto)
 set_default("use_newpm", False)
+set_default("use_interleaving", False)
 set_default("android", @ANDROID_PYBOOL@)
 set_default("android_ndk_version", @ANDROID_NDK_VERSION@)
 set_default("android_serial", "@ANDROID_SERIAL_FOR_TESTING@")
 set_default("android_files_to_push", [])
 set_default("have_rpc_xdr_h", @HAVE_RPC_XDR_H@)
 set_default("gwp_asan", @COMPILER_RT_HAS_GWP_ASAN_PYBOOL@)
 config.available_features.add('target-is-%s' % config.target_arch)