[GlobalDCE] Dead Virtual Function Elimination

[ostannard]

Summary

Currently, it is hard for the compiler to remove unused C++ virtual
functions, because they are all referenced from vtables, which are referenced
by constructors. This means that if the constructor is called from any live
code, then we keep every virtual function in the final link, even if there
are no call sites which can use it.

This patch allows unused virtual functions to be removed during LTO (and
regular compilation in limited circumstances) by using type metadata to match
virtual function call sites to the vtable slots they might load from. This
information can then be used in the global dead code elimination pass instead
of the references from vtables to virtual functions, to more accurately
determine which functions are reachable.

To make this transformation safe, I have changed clang's code-generation to
always load virtual function pointers using the llvm.type.checked.load
intrinsic, instead of regular load instructions. I originally tried writing
this using clang's existing code-generation, which uses the llvm.type.test
and llvm.assume intrinsics after doing a normal load. However, it is possible
for optimisations to obscure the relationship between the GEP, load and
llvm.type.test, causing GlobalDCE to fail to find virtual function call
sites.

The existing linkage and visibility types don't accurately describe the scope
in which a virtual call could be made which uses a given vtable. This is
wider than the visibility of the type itself, because a virtual function call
could be made using a more-visible base class. I've added a new
!vcall_visibility metadata type to represent this, described in
TypeMetadata.rst. The internalization pass and libLTO have been updated to
change this metadata when linking is performed.

This doesn't currently work with ThinLTO, because it needs to see every call
to llvm.type.checked.load in the linkage unit. It might be possible to
extend this optimisation to be able to use the ThinLTO summary, as was done
for devirtualization, but until then that combination is rejected in the
clang driver.

To test this, I've written a fuzzer which generates random C++ programs with
complex class inheritance graphs, and virtual functions called through object
and function pointers of different types. The programs are spread across
multiple translation units and DSOs to test the different visibility
restrictions.

I've also tried doing bootstrap builds of LLVM to test this. This isn't
ideal, because only classes in anonymous namespaces can be optimised with
-fvisibility=default, and some parts of LLVM (plugins and bugpoint) do not
work correctly with -fvisibility=hidden. However, there are only 12 test
failures when building with -fvisibility=hidden (and an unmodified compiler),
and this change does not cause any new failures for either value of
-fvisibility.

On the 7 C++ sub-benchmarks of SPEC2006, this gives a geomean code-size
reduction of ~6%, over a baseline compiled with "-O2 -flto
-fvisibility=hidden -fwhole-program-vtables". The best cases are reductions
of ~14% in 450.soplex and 483.xalancbmk, and there are no code size
increases.

I've also run this on a set of 8 mbed-os examples compiled for Armv7M, which
show a geomean size reduction of ~3%, again with no size increases.

I had hoped that this would have no effect on performance, which would allow
it to awlays be enabled (when using -fwhole-program-vtables). However, the
changes in clang to use the llvm.type.checked.load intrinsic are causing ~1%
performance regression in the C++ parts of SPEC2006. It should be possible to
recover some of this perf loss by teaching optimisations about the
llvm.type.checked.load intrinsic, which would make it worth turning this on
by default (though it's still dependent on -fwhole-program-vtables).

[lebedev.ri]

To make this transformation safe, I have changed clang's code-generation to
always load virtual function pointers using the llvm.type.checked.load
intrinsic, instead of regular load instructions. I originally tried writing
this using clang's existing code-generation, which uses the llvm.type.test
and llvm.assume intrinsics after doing a normal load. However, it is possible
for optimisations to obscure the relationship between the GEP, load and
llvm.type.test, causing GlobalDCE to fail to find virtual function call
sites.

Can you please clarify, that is saying that without switching to
llvm.type.checked.load the optimization would be less efficient
since it would trigger less, not that the optimization would trigger
in some cases where it shouldn't trigger, thus miscompiling the code,
correct ?

[ostannard]

Without switching to llvm.type.checked.load, the optimisation would trigger in some cases where it shouldn't trigger, thus miscompiling the code. This is because it needs to know about _every_ virtual call site, if we lose the information about some of them then it will incorrectly think a virtual function is unused.

[ostannard]

Ping.

[pcc]

Hi Oliver, thanks for working on this. This is something that I've always wanted to do with the type metadata but never had time to work on. I'll try to take a more in depth look later this week.

[ostannard]

Ping.

[pcc]

It doesn't seem sound to create the distinction between translation unit and linkage unit visibility and then allow passes to relax visibility. For example, imagine that you have an executable with (compiled with -fvisibility=default):

struct PluginInterface {
  virtual void foo() = 0;
}

struct Plugin1 : PluginInterface {
  virtual void foo();
};

void Plugin1::foo() {
 // ...
}

And a DSO loaded with dlopen:

void call_foo(PluginInterface *plugin) {
  plugin->foo();
}

Note that the lack of attribute means that the vtables have public LTO visibility. Depending on the contents of the DSO, the PluginInterface and Plugin1 vtable symbols may not be referenced at all by the DSO. If the DSO is loaded via dlopen it will definitely not be referenced at link time. If we allow LTO to relax PluginInterface and Plugin1 vtables to what you're calling VCallVisibilityTranslationUnit in the main executable and there are no calls to foo() in the main executable, we will incorrectly drop the definition of Plugin1::foo().

The approach that I can see working involves basically the same structure as CFI and WPD: the LTO visibility is decided by the frontend and not relaxed by passes or LTO. There is only one "flavour" of !vcall_visibility metadata: the one that means the entire hierarchy below the vtable has hidden LTO visibility.

However, the
changes in clang to use the llvm.type.checked.load intrinsic are causing ~1%
performance regression in the C++ parts of SPEC2006.

Have you checked the assembly to see whether an unused CFI check is being emitted?

clang/lib/CodeGen/CGClass.cpp

2826

There should be no need to emit this llvm.assume intrinsic. We can rely on the unspecified behaviour in the case where the second element returned by llvm.type.checked.load is false.

llvm/docs/TypeMetadata.rst

240

s/"public"/"default"/

llvm/lib/Transforms/IPO/GlobalDCE.cpp

256

Not necessarily, at least in this implementation, because "vtable symbol can be internalized" doesn't imply "all vcalls are visible". See main response.

[ostannard]

In that example, with everything having default ELF visibility, all of the vtables will get vcall_visibility public, which can't be optimised by VFE, and won't ever be relaxed to one of the tighter visibilities.

The cases which can be optimised are (assuming "visibility" means visibility of the most-visible dynamic base class):

• Classes in an anonymous namespace, which aren't visible outside their translation unit. Probably doesn't happen very often, but can be optimised without LTO.
• Classes visible outside the translation unit, but not outside the LTO unit. This generally means hidden ELF visibility, unless the lto_visibility_public attribute is used. These can't be optimised without LTO, but can be with it.

I implemented the second case by adding the LinkageUnit visibility, which can't be optimised by VFE, but can be reduced to TranslationUnit when LTO internalisation happens. This could also have been done by not changing the visibility at LTO time, and instead leting GlobalDCE know if it is running post-LTO. Both of these should give the same results, but the way I used represents the visibility fully in the IR, without having the extra state of "are we doing LTO?".

I also don't think it matters here whether the DSO is loaded by dlopen or by being linked against it. Is there something I've missed here, or was dlopen not relevant to your example.

Have you checked the assembly to see whether an unused CFI check is being emitted?

Not yet, I wanted to make sure that this optimisation was valid and correctly implemented before tracking down the performance regressions.

llvm/lib/Transforms/IPO/GlobalDCE.cpp

256

This is checking the vcall_visibility, which will only be VCallVisibilityTranslationUnit if all base classes which contain virtual functions are private to this translation unit, which does imply "all vcalls are visible".

[ostannard]

• Rebase
• Don't emit llvm.assume when not necessary (we already weren't checking for it's presence in GlobalDCE)
• s/"public"/"default"/ in IR docs

[pcc]

In D63932#1606248, @ostannard wrote:

In that example, with everything having default ELF visibility, all of the vtables will get vcall_visibility public, which can't be optimised by VFE, and won't ever be relaxed to one of the tighter visibilities.

The cases which can be optimised are (assuming "visibility" means visibility of the most-visible dynamic base class):

• Classes in an anonymous namespace, which aren't visible outside their translation unit. Probably doesn't happen very often, but can be optimised without LTO.
• Classes visible outside the translation unit, but not outside the LTO unit. This generally means hidden ELF visibility, unless the lto_visibility_public attribute is used. These can't be optimised without LTO, but can be with it.

I implemented the second case by adding the LinkageUnit visibility, which can't be optimised by VFE, but can be reduced to TranslationUnit when LTO internalisation happens. This could also have been done by not changing the visibility at LTO time, and instead leting GlobalDCE know if it is running post-LTO. Both of these should give the same results, but the way I used represents the visibility fully in the IR, without having the extra state of "are we doing LTO?".

Okay, I somehow missed that the relaxation to TranslationUnit was guarded on it being LinkageUnit to start with. So at least from that perspective I don't see any soundness issues. It still doesn't seem like a good idea to do the relaxation in LTO though, since the visibility-outside-of-LTO of the vtable symbols is not a reliable indicator whether the type is used externally (I think that's what confused me to begin with). Here are a couple of cases where things might go wrong:

• Take the example from my earlier message, give the "main executable" and "DSO" hidden visibility, build the "main executable" with LTO and the "DSO" without LTO, and statically link them both into the same executable. We run into the same problem where the Plugin1 vtable is potentially not referenced and thus misoptimised. Yes, this is a violation of the LTO visibility rules, but the example shows that we only detect it sometimes. I think that if we did want to detect cases where the LTO visibility rules are clearly being violated, the outcome should be to issue a diagnostic and not to silently proceed with optimizations disabled, since the violation might be masking other undetected issues. That really seems orthogonal to this patch, though.
• Imagine linking part of a program with ld -r with LTO -- all symbols including vtable symbols will appear to be externally visible, which is not necessarily a violation of the LTO visibility rules because they may not be used externally in the final link. So we end up pessimising unnecessarily.

I gave this some more thought and it seems that the frontend has all of the information required to make a determination about whether to allow VFE, without needing to teach LTO to relax visibility. We can make the frontend do this:

• If -flto was passed (see CodeGenOptions::PrepareForLTO), attach "allow VFE" metadata if class has hidden LTO visibility.
• Otherwise, attach "allow VFE" metadata if class is not isExternallyVisible.

Now the behaviour of GlobalDCE can be made conditional on the "allow VFE" metadata alone. This also has the advantage of simplifying the IR by making "allow VFE" a boolean rather than a tri-state as it is now.

llvm/lib/Transforms/IPO/GlobalDCE.cpp

256

Ack.

[ostannard]

• Take the example from my earlier message, give the "main executable" and "DSO" hidden visibility, build the "main executable" with LTO and the "DSO" without LTO, and statically link them both into the same executable. We run into the same problem where the Plugin1 vtable is potentially not referenced and thus misoptimised. Yes, this is a violation of the LTO visibility rules, but the example shows that we only detect it sometimes. I think that if we did want to detect cases where the LTO visibility rules are clearly being violated, the outcome should be to issue a diagnostic and not to silently proceed with optimizations disabled, since the violation might be masking other undetected issues. That really seems orthogonal to this patch, though.

As you say, this is a violation of the LTO visibility rules, so [[clang::lto_visibility_public]] must be used. I've gated this optimisation behind -fwhole-program-vtables, which if I've understood https://clang.llvm.org/docs/LTOVisibility.html correctly, is effectively the user asserting to the compiler that they've followed the LTO visility rules correctly.

• Imagine linking part of a program with ld -r with LTO -- all symbols including vtable symbols will appear to be externally visible, which is not necessarily a violation of the LTO visibility rules because they may not be used externally in the final link. So we end up pessimising unnecessarily.

I'm not really familiar with partial linking, is there any good documentation on how it is supposed to interact with LTO?

I've tried to make the changed to vcall_visibility as similar to the normal LTO internalisation process as possible, it happens at the same time and under the same conditions. If partial linking prevents normal internalisation, which will in turn prevent most of the existing optimisations done by GlobalDCE, then surely it should also prevent internalisation of vcall_visibility?

I gave this some more thought and it seems that the frontend has all of the information required to make a determination about whether to allow VFE, without needing to teach LTO to relax visibility. We can make the frontend do this:

• If -flto was passed (see CodeGenOptions::PrepareForLTO), attach "allow VFE" metadata if class has hidden LTO visibility.
• Otherwise, attach "allow VFE" metadata if class is not isExternallyVisible.

Now the behaviour of GlobalDCE can be made conditional on the "allow VFE" metadata alone. This also has the advantage of simplifying the IR by making "allow VFE" a boolean rather than a tri-state as it is now.

The problem with that is that GlobalDCE is still run in the pre-linking pipeline when emiting an LTO object, because it can remove code/data which already has internal linkage. To make this work, we'd have to do one of:

• Tell GlobalDCE whether it is running in LTO post-linking or not, so it can avoid doing VFE before internalisation. This breaks the idea that the IR contains all the information needed to determine if an optimisation is legal or not.
• Remove the pre-linking runs of GlobalDCE when doing LTO. This will result in more known-dead code reaching the LTO link stage, and results in a pass which is only valid at certain points in the pipeline.
• Split VFE out of GlobalDCE. This would result in worse optimisation, because chains of dependencies involving both virtual calls and regular references won't be followed fully.

[pcc]

In D63932#1608235, @ostannard wrote:

• Take the example from my earlier message, give the "main executable" and "DSO" hidden visibility, build the "main executable" with LTO and the "DSO" without LTO, and statically link them both into the same executable. We run into the same problem where the Plugin1 vtable is potentially not referenced and thus misoptimised. Yes, this is a violation of the LTO visibility rules, but the example shows that we only detect it sometimes. I think that if we did want to detect cases where the LTO visibility rules are clearly being violated, the outcome should be to issue a diagnostic and not to silently proceed with optimizations disabled, since the violation might be masking other undetected issues. That really seems orthogonal to this patch, though.

As you say, this is a violation of the LTO visibility rules, so [[clang::lto_visibility_public]] must be used. I've gated this optimisation behind -fwhole-program-vtables, which if I've understood https://clang.llvm.org/docs/LTOVisibility.html correctly, is effectively the user asserting to the compiler that they've followed the LTO visility rules correctly.

Yes, that's correct.

• Imagine linking part of a program with ld -r with LTO -- all symbols including vtable symbols will appear to be externally visible, which is not necessarily a violation of the LTO visibility rules because they may not be used externally in the final link. So we end up pessimising unnecessarily.

I'm not really familiar with partial linking, is there any good documentation on how it is supposed to interact with LTO?

I'm not aware of any. The way that I think about it is that a partial link operation with LTO inputs can still produce an LTO unit, but that LTO unit may not be combined with other LTO units, either by passing bitcode files directly to the final link or by adding another ld -r output with an LTO unit to the final link.

I've tried to make the changed to vcall_visibility as similar to the normal LTO internalisation process as possible, it happens at the same time and under the same conditions. If partial linking prevents normal internalisation, which will in turn prevent most of the existing optimisations done by GlobalDCE, then surely it should also prevent internalisation of vcall_visibility?

Partial linking will indeed prevent dropping the virtual functions, but it should not prevent clearing the pointer to the virtual function in the vtable. The linker should then be able to drop the virtual function body as part of --gc-sections during the final link.

I gave this some more thought and it seems that the frontend has all of the information required to make a determination about whether to allow VFE, without needing to teach LTO to relax visibility. We can make the frontend do this:

• If -flto was passed (see CodeGenOptions::PrepareForLTO), attach "allow VFE" metadata if class has hidden LTO visibility.
• Otherwise, attach "allow VFE" metadata if class is not isExternallyVisible.

Now the behaviour of GlobalDCE can be made conditional on the "allow VFE" metadata alone. This also has the advantage of simplifying the IR by making "allow VFE" a boolean rather than a tri-state as it is now.

The problem with that is that GlobalDCE is still run in the pre-linking pipeline when emiting an LTO object, because it can remove code/data which already has internal linkage.

Right, of course. Sorry, I forgot about that. It seems that if we want to allow VFE on internal linkage types without LTO, we would still need the tri-state.

To make this work, we'd have to do one of:

• Tell GlobalDCE whether it is running in LTO post-linking or not, so it can avoid doing VFE before internalisation. This breaks the idea that the IR contains all the information needed to determine if an optimisation is legal or not.
• Remove the pre-linking runs of GlobalDCE when doing LTO. This will result in more known-dead code reaching the LTO link stage, and results in a pass which is only valid at certain points in the pipeline.
• Split VFE out of GlobalDCE. This would result in worse optimisation, because chains of dependencies involving both virtual calls and regular references won't be followed fully.

I think I would favour something closer to your first suggestion, but instead of telling GlobalDCE specifically about this, we represent the "post-link" flag in the IR (e.g. as a module flag) in order to keep the IR self-contained. LTO would then be taught to add this flag at the right time, and the logic inside GlobalDCE would be:

• If post-link flag not set, allow VFE if linkage <= TranslationUnit.
• If post-link flag set, allow VFE if linkage <= LinkageUnit.

This would also help address a correctness issue with the CFI and WPD passes, which is that it is currently unsound to run them at compile time. If we let them run at compile time, we would in principle be able to do CFI and WPD on internal linkage types without LTO.

[ostannard]

Partial linking will indeed prevent dropping the virtual functions, but it should not prevent clearing the pointer to the virtual function in the vtable. The linker should then be able to drop the virtual function body as part of --gc-sections during the final link.

If partial linking isn't doing internalisation, I'd expect that to prevent a lot of other LTO optimisations, not just VFE. Is this a common use-case which we need to care about?

I think I would favour something closer to your first suggestion, but instead of telling GlobalDCE specifically about this, we represent the "post-link" flag in the IR (e.g. as a module flag) in order to keep the IR self-contained. LTO would then be taught to add this flag at the right time, and the logic inside GlobalDCE would be:

• If post-link flag not set, allow VFE if linkage <= TranslationUnit.
• If post-link flag set, allow VFE if linkage <= LinkageUnit.

This would also help address a correctness issue with the CFI and WPD passes, which is that it is currently unsound to run them at compile time. If we let them run at compile time, we would in principle be able to do CFI and WPD on internal linkage types without LTO.

Ok, sounds reasonable, though I suspect WPD and CFI will need a slightly different definition of type visibility - they care about the possibility of unseen code adding new derived classes, so the visibility of base classes isn't important, and they might be able to make use of the final specifier.

[ostannard]

• Add LTOPostLink metadata, instead of internalising vcall visibility at LTO time

[pcc]

In D63932#1610182, @ostannard wrote:

Partial linking will indeed prevent dropping the virtual functions, but it should not prevent clearing the pointer to the virtual function in the vtable. The linker should then be able to drop the virtual function body as part of --gc-sections during the final link.

If partial linking isn't doing internalisation, I'd expect that to prevent a lot of other LTO optimisations, not just VFE. Is this a common use-case which we need to care about?

It isn't that common, but it seems worth doing if it can be done easily.

That said, I note that it does appear that your implementation will end up preserving the pointer in the vtable in this case because you're relying on the use list to make decisions about what to GC. So it doesn't seem easy to do at this point, but if for example we made this compatible with ThinLTO at some point we would probably not be able to rely on the use list, and the resulting changes to this feature might make this easier to do.

I think I would favour something closer to your first suggestion, but instead of telling GlobalDCE specifically about this, we represent the "post-link" flag in the IR (e.g. as a module flag) in order to keep the IR self-contained. LTO would then be taught to add this flag at the right time, and the logic inside GlobalDCE would be:

• If post-link flag not set, allow VFE if linkage <= TranslationUnit.
• If post-link flag set, allow VFE if linkage <= LinkageUnit.

This would also help address a correctness issue with the CFI and WPD passes, which is that it is currently unsound to run them at compile time. If we let them run at compile time, we would in principle be able to do CFI and WPD on internal linkage types without LTO.

Ok, sounds reasonable, though I suspect WPD and CFI will need a slightly different definition of type visibility - they care about the possibility of unseen code adding new derived classes, so the visibility of base classes isn't important, and they might be able to make use of the final specifier.

Internal linkage types use distinct metadata (i.e. metadata that can't be merged with other TUs), so there's no possibility of a new derived class being added. I suspect that we could make these passes check the distinctness of metadata if the post-link flag is not set, but that probably deserves more thought.

llvm/lib/Transforms/IPO/GlobalDCE.cpp

409

Could be just ConstantPointerNull::get(F->getType()).

llvm/test/LTO/ARM/lto-linking-metadata.ll

3

Could you add a test for the new LTO API as well as the legacy one?

llvm/test/Transforms/GlobalDCE/virtual-functions-derived-pointer-call.ll

31

Remove word "or".

llvm/test/Transforms/GlobalDCE/virtual-functions-visibility-post-lto.ll

14

Could you simplify this test (and others) by removing the RTTI, please? We should also have a test showing that RTTI is preserved.

[ostannard]

It isn't that common, but it seems worth doing if it can be done easily.

That said, I note that it does appear that your implementation will end up preserving the pointer in the vtable in this case because you're relying on the use list to make decisions about what to GC. So it doesn't seem easy to do at this point, but if for example we made this compatible with ThinLTO at some point we would probably not be able to rely on the use list, and the resulting changes to this feature might make this easier to do.

Ok, I think that it makes sense to leave this for a separate patch, as long as we currently generate correct code. I've added partial linking of the LTO unit to my fuzzer, and haven't found any problems.

[ostannard]

Ping

[ostannard]

Ping

[mehdi_amini]

we represent the "post-link" flag in the IR (e.g. as a module flag) in order to keep the IR self-contained.

This makes the IR self-contained, which is good, but it also make the interpretation of the IR modal, which isn't great. It means that suddenly the rules of interpretation of what is valid to do or not changes according to this module flag.

[ostannard]

This makes the IR self-contained, which is good, but it also make the interpretation of the IR modal, which isn't great. It means that suddenly the rules of interpretation of what is valid to do or not changes according to this module flag.

I think the original version was better from that perspective - most compiler passes only need to check for one value of the attribute, and only the linker needed to care about the difference between link-unit and public visibility. Do you think we should go back to that design, or do you have a different idea?

[mehdi_amini]

In D63932#1679910, @ostannard wrote:

This makes the IR self-contained, which is good, but it also make the interpretation of the IR modal, which isn't great. It means that suddenly the rules of interpretation of what is valid to do or not changes according to this module flag.

I think the original version was better from that perspective - most compiler passes only need to check for one value of the attribute, and only the linker needed to care about the difference between link-unit and public visibility.

The *linker* needed to care about the difference is fine, but that's different from a *compiler pass* during LTO. In general it is an important design point in LLVM so far to encode these information in the IR (for instance the linkage type of globals would be changed and not the interpretation of these with a flag).

Do you think we should go back to that design, or do you have a different idea?

I didn't review the original version, but pcc@ did apparently, so I'm willing to trust them on what's the best way forward here.

[ostannard]

Ping

[ostannard]

Ping

[pcc]

LGTM

[sylvestre.ledru]

@ostannard Maybe you could add that to the release notes? Thanks

clang/include/clang/Basic/CodeGenOptions.def

CODEGENOPT(DebugFwdTemplateParams, 1, 0) ///< Whether to emit complete
                                          ///< forward declarations (versus just
                                          ///< including them in the name).
 
 CODEGENOPT(EmitLLVMUseLists, 1, 0) ///< Control whether to serialize use-lists.
 
 CODEGENOPT(WholeProgramVTables, 1, 0) ///< Whether to apply whole-program
                                       ///  vtable optimization.
 
+CODEGENOPT(VirtualFunctionElimination, 1, 0) ///< Whether to apply the dead
+                                             /// virtual function elimination
+                                             /// optimization.
+
 /// Whether to use public LTO visibility for entities in std and stdext
 /// namespaces. This is enabled by clang-cl's /MT and /MTd flags.
 CODEGENOPT(LTOVisibilityPublicStd, 1, 0)
 
 /// The user specified number of registers to be used for integral arguments,
 /// or 0 if unspecified.
 VALUE_CODEGENOPT(NumRegisterParameters, 32, 0)
 

clang/include/clang/Driver/Options.td

def fsplit_lto_unit : Flag<["-"], "fsplit-lto-unit">, Group<f_Group>,
   HelpText<"Enables splitting of the LTO unit.">;
 def fno_split_lto_unit : Flag<["-"], "fno-split-lto-unit">, Group<f_Group>,
   Flags<[CoreOption]>;
 def fforce_emit_vtables : Flag<["-"], "fforce-emit-vtables">, Group<f_Group>,
     Flags<[CC1Option]>,
     HelpText<"Emits more virtual tables to improve devirtualization">;
 def fno_force_emit_vtables : Flag<["-"], "fno-force-emit-vtables">, Group<f_Group>,
   Flags<[CoreOption]>;
+
+def fvirtual_function_elimination : Flag<["-"], "fvirtual-function-elimination">, Group<f_Group>,
+  Flags<[CoreOption, CC1Option]>,
+  HelpText<"Enables dead virtual function elimination optimization. Requires -flto=full">;
+def fno_virtual_function_elimination : Flag<["-"], "fno-virtual-function_elimination">, Group<f_Group>,
+  Flags<[CoreOption]>;
+
 def fwrapv : Flag<["-"], "fwrapv">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Treat signed integer overflow as two's complement">;
 def fwritable_strings : Flag<["-"], "fwritable-strings">, Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Store string literals as writable data">;
 def fzero_initialized_in_bss : Flag<["-"], "fzero-initialized-in-bss">, Group<f_Group>;
 def ffunction_sections : Flag<["-"], "ffunction-sections">, Group<f_Group>,
   Flags<[CC1Option]>,
   HelpText<"Place each function in its own section (ELF Only)">;

clang/lib/CodeGen/CGClass.cpp

void CodeGenFunction::EmitVTablePtrCheck(const CXXRecordDecl *RD,
   llvm::Value *ValidVtable = Builder.CreateCall(
       CGM.getIntrinsic(llvm::Intrinsic::type_test), {CastedVTable, AllVtables});
   EmitCheck(std::make_pair(TypeTest, M), SanitizerHandler::CFICheckFail,
             StaticData, {CastedVTable, ValidVtable});
 }
 
 bool CodeGenFunction::ShouldEmitVTableTypeCheckedLoad(const CXXRecordDecl *RD) {
   if (!CGM.getCodeGenOpts().WholeProgramVTables ||
-      !SanOpts.has(SanitizerKind::CFIVCall) ||
-      !CGM.getCodeGenOpts().SanitizeTrap.has(SanitizerKind::CFIVCall) ||
       !CGM.HasHiddenLTOVisibility(RD))
     return false;
 
+  if (CGM.getCodeGenOpts().VirtualFunctionElimination)
+    return true;
+
+  if (!SanOpts.has(SanitizerKind::CFIVCall) ||
+      !CGM.getCodeGenOpts().SanitizeTrap.has(SanitizerKind::CFIVCall))
+    return false;
+
   std::string TypeName = RD->getQualifiedNameAsString();
   return !getContext().getSanitizerBlacklist().isBlacklistedType(
       SanitizerKind::CFIVCall, TypeName);
 }
 
 llvm::Value *CodeGenFunction::EmitVTableTypeCheckedLoad(
     const CXXRecordDecl *RD, llvm::Value *VTable, uint64_t VTableByteOffset) {
   SanitizerScope SanScope(this);
 
   EmitSanitizerStatReport(llvm::SanStat_CFI_VCall);
 
   llvm::Metadata *MD =
       CGM.CreateMetadataIdentifierForType(QualType(RD->getTypeForDecl(), 0));
   llvm::Value *TypeId = llvm::MetadataAsValue::get(CGM.getLLVMContext(), MD);
 
   llvm::Value *CastedVTable = Builder.CreateBitCast(VTable, Int8PtrTy);
   llvm::Value *CheckedLoad = Builder.CreateCall(
       CGM.getIntrinsic(llvm::Intrinsic::type_checked_load),
       {CastedVTable, llvm::ConstantInt::get(Int32Ty, VTableByteOffset),
        TypeId});
   llvm::Value *CheckResult = Builder.CreateExtractValue(CheckedLoad, 1);
 
+  std::string TypeName = RD->getQualifiedNameAsString();
+  if (SanOpts.has(SanitizerKind::CFIVCall) &&
+      !getContext().getSanitizerBlacklist().isBlacklistedType(
+          SanitizerKind::CFIVCall, TypeName)) {
     EmitCheck(std::make_pair(CheckResult, SanitizerKind::CFIVCall),
-            SanitizerHandler::CFICheckFail, nullptr, nullptr);
+              SanitizerHandler::CFICheckFail, {}, {});
+  }
 
   return Builder.CreateBitCast(
       Builder.CreateExtractValue(CheckedLoad, 0),
       cast<llvm::PointerType>(VTable->getType())->getElementType());
 }
 
 void CodeGenFunction::EmitForwardingCallToLambda(
                                       const CXXMethodDecl *callOperator,
                                       CallArgList &callArgs) {

clang/lib/CodeGen/CGVTables.cpp

CodeGenVTables::GenerateConstructionVTable(const CXXRecordDecl *RD,
   createVTableInitializer(components, *VTLayout, RTTI);
   components.finishAndSetAsInitializer(VTable);
 
   // Set properties only after the initializer has been set to ensure that the
   // GV is treated as definition and not declaration.
   assert(!VTable->isDeclaration() && "Shouldn't set properties on declaration");
   CGM.setGVProperties(VTable, RD);
 
-  CGM.EmitVTableTypeMetadata(VTable, *VTLayout.get());
+  CGM.EmitVTableTypeMetadata(RD, VTable, *VTLayout.get());
 
   return VTable;
 }
 
 static bool shouldEmitAvailableExternallyVTable(const CodeGenModule &CGM,
                                                 const CXXRecordDecl *RD) {
   return CGM.getCodeGenOpts().OptimizationLevel > 0 &&
          CGM.getCXXABI().canSpeculativelyEmitVTable(RD);
while (1) {
         break;
       }
     }
   }
 
   return true;
 }
 
-void CodeGenModule::EmitVTableTypeMetadata(llvm::GlobalVariable *VTable,
+llvm::GlobalObject::VCallVisibility
+CodeGenModule::GetVCallVisibilityLevel(const CXXRecordDecl *RD) {
+  LinkageInfo LV = RD->getLinkageAndVisibility();
+  llvm::GlobalObject::VCallVisibility TypeVis;
+  if (!isExternallyVisible(LV.getLinkage()))
+    TypeVis = llvm::GlobalObject::VCallVisibilityTranslationUnit;
+  else if (HasHiddenLTOVisibility(RD))
+    TypeVis = llvm::GlobalObject::VCallVisibilityLinkageUnit;
+  else
+    TypeVis = llvm::GlobalObject::VCallVisibilityPublic;
+
+  for (auto B : RD->bases())
+    if (B.getType()->getAsCXXRecordDecl()->isDynamicClass())
+      TypeVis = std::min(TypeVis,
+                    GetVCallVisibilityLevel(B.getType()->getAsCXXRecordDecl()));
+
+  for (auto B : RD->vbases())
+    if (B.getType()->getAsCXXRecordDecl()->isDynamicClass())
+      TypeVis = std::min(TypeVis,
+                    GetVCallVisibilityLevel(B.getType()->getAsCXXRecordDecl()));
+
+  return TypeVis;
+}
+
+void CodeGenModule::EmitVTableTypeMetadata(const CXXRecordDecl *RD,
+                                           llvm::GlobalVariable *VTable,
                                            const VTableLayout &VTLayout) {
   if (!getCodeGenOpts().LTOUnit)
     return;
 
   CharUnits PointerWidth =
       Context.toCharUnitsFromBits(Context.getTargetInfo().getPointerWidth(0));
 
   typedef std::pair<const CXXRecordDecl *, unsigned> AddressPoint;
for (unsigned I = 0; I != Comps.size(); ++I) {
         continue;
       llvm::Metadata *MD = CreateMetadataIdentifierForVirtualMemPtrType(
           Context.getMemberPointerType(
               Comps[I].getFunctionDecl()->getType(),
               Context.getRecordType(AP.first).getTypePtr()));
       VTable->addTypeMetadata((PointerWidth * I).getQuantity(), MD);
     }
   }
+
+  if (getCodeGenOpts().VirtualFunctionElimination) {
+    llvm::GlobalObject::VCallVisibility TypeVis = GetVCallVisibilityLevel(RD);
+    if (TypeVis != llvm::GlobalObject::VCallVisibilityPublic)
+      VTable->addVCallVisibilityMetadata(TypeVis);
+  }
 }

clang/lib/CodeGen/CodeGenModule.h

public:
   void emitOpenMPDeviceFunctionRedefinition(GlobalDecl OldGD, GlobalDecl NewGD,
                                             llvm::GlobalValue *GV);
 
   /// Returns whether the given record has hidden LTO visibility and therefore
   /// may participate in (single-module) CFI and whole-program vtable
   /// optimization.
   bool HasHiddenLTOVisibility(const CXXRecordDecl *RD);
 
+  /// Returns the vcall visibility of the given type. This is the scope in which
+  /// a virtual function call could be made which ends up being dispatched to a
+  /// member function of this class. This scope can be wider than the visibility
+  /// of the class itself when the class has a more-visible dynamic base class.
+  llvm::GlobalObject::VCallVisibility
+  GetVCallVisibilityLevel(const CXXRecordDecl *RD);
+
   /// Emit type metadata for the given vtable using the given layout.
-  void EmitVTableTypeMetadata(llvm::GlobalVariable *VTable,
+  void EmitVTableTypeMetadata(const CXXRecordDecl *RD,
+                              llvm::GlobalVariable *VTable,
                               const VTableLayout &VTLayout);
 
   /// Generate a cross-DSO type identifier for MD.
   llvm::ConstantInt *CreateCrossDsoCfiTypeId(llvm::Metadata *MD);
 
   /// Create a metadata identifier for the given type. This may either be an
   /// MDString (for external identifiers) or a distinct unnamed MDNode (for
   /// internal identifiers).

clang/lib/CodeGen/ItaniumCXXABI.cpp

CGCallee ItaniumCXXABI::EmitLoadOfMemberFunctionPointer(
   // we only pay attention to the low 32 bits of the offset.
   llvm::Value *VTableOffset = FnAsInt;
   if (!UseARMMethodPtrABI)
     VTableOffset = Builder.CreateSub(VTableOffset, ptrdiff_1);
   if (Use32BitVTableOffsetABI) {
     VTableOffset = Builder.CreateTrunc(VTableOffset, CGF.Int32Ty);
     VTableOffset = Builder.CreateZExt(VTableOffset, CGM.PtrDiffTy);
   }
-  // Compute the address of the virtual function pointer.
-  llvm::Value *VFPAddr = Builder.CreateGEP(VTable, VTableOffset);
 
   // Check the address of the function pointer if CFI on member function
   // pointers is enabled.
   llvm::Constant *CheckSourceLocation;
   llvm::Constant *CheckTypeDesc;
   bool ShouldEmitCFICheck = CGF.SanOpts.has(SanitizerKind::CFIMFCall) &&
                             CGM.HasHiddenLTOVisibility(RD);
-  if (ShouldEmitCFICheck) {
+  bool ShouldEmitVFEInfo = CGM.getCodeGenOpts().VirtualFunctionElimination &&
+                           CGM.HasHiddenLTOVisibility(RD);
+  llvm::Value *VirtualFn = nullptr;
+
+  {
     CodeGenFunction::SanitizerScope SanScope(&CGF);
+    llvm::Value *TypeId = nullptr;
+    llvm::Value *CheckResult = nullptr;
 
+    if (ShouldEmitCFICheck || ShouldEmitVFEInfo) {
+      // If doing CFI or VFE, we will need the metadata node to check against.
+      llvm::Metadata *MD =
+          CGM.CreateMetadataIdentifierForVirtualMemPtrType(QualType(MPT, 0));
+      TypeId = llvm::MetadataAsValue::get(CGF.getLLVMContext(), MD);
+    }
+
+    llvm::Value *VFPAddr = Builder.CreateGEP(VTable, VTableOffset);
+
+    if (ShouldEmitVFEInfo) {
+      // If doing VFE, load from the vtable with a type.checked.load intrinsic
+      // call. Note that we use the GEP to calculate the address to load from
+      // and pass 0 as the offset to the intrinsic. This is because every
+      // vtable slot of the correct type is marked with matching metadata, and
+      // we know that the load must be from one of these slots.
+      llvm::Value *CheckedLoad = Builder.CreateCall(
+          CGM.getIntrinsic(llvm::Intrinsic::type_checked_load),
+          {VFPAddr, llvm::ConstantInt::get(CGM.Int32Ty, 0), TypeId});
+      CheckResult = Builder.CreateExtractValue(CheckedLoad, 1);
+      VirtualFn = Builder.CreateExtractValue(CheckedLoad, 0);
+      VirtualFn = Builder.CreateBitCast(VirtualFn, FTy->getPointerTo(),
+                                        "memptr.virtualfn");
+    } else {
+      // When not doing VFE, emit a normal load, as it allows more
+      // optimisations than type.checked.load.
+      if (ShouldEmitCFICheck) {
+        CheckResult = Builder.CreateCall(
+            CGM.getIntrinsic(llvm::Intrinsic::type_test),
+            {Builder.CreateBitCast(VFPAddr, CGF.Int8PtrTy), TypeId});
+      }
+      VFPAddr =
+          Builder.CreateBitCast(VFPAddr, FTy->getPointerTo()->getPointerTo());
+      VirtualFn = Builder.CreateAlignedLoad(VFPAddr, CGF.getPointerAlign(),
+                                            "memptr.virtualfn");
+    }
+    assert(VirtualFn && "Virtual fuction pointer not created!");
+    assert((!ShouldEmitCFICheck || !ShouldEmitVFEInfo || CheckResult) &&
+           "Check result required but not created!");
+
+    if (ShouldEmitCFICheck) {
+      // If doing CFI, emit the check.
       CheckSourceLocation = CGF.EmitCheckSourceLocation(E->getBeginLoc());
       CheckTypeDesc = CGF.EmitCheckTypeDescriptor(QualType(MPT, 0));
       llvm::Constant *StaticData[] = {
           llvm::ConstantInt::get(CGF.Int8Ty, CodeGenFunction::CFITCK_VMFCall),
           CheckSourceLocation,
           CheckTypeDesc,
       };
 
-    llvm::Metadata *MD =
-        CGM.CreateMetadataIdentifierForVirtualMemPtrType(QualType(MPT, 0));
-    llvm::Value *TypeId = llvm::MetadataAsValue::get(CGF.getLLVMContext(), MD);
-
-    llvm::Value *TypeTest = Builder.CreateCall(
-        CGM.getIntrinsic(llvm::Intrinsic::type_test), {VFPAddr, TypeId});
-
       if (CGM.getCodeGenOpts().SanitizeTrap.has(SanitizerKind::CFIMFCall)) {
-      CGF.EmitTrapCheck(TypeTest);
+        CGF.EmitTrapCheck(CheckResult);
       } else {
         llvm::Value *AllVtables = llvm::MetadataAsValue::get(
             CGM.getLLVMContext(),
             llvm::MDString::get(CGM.getLLVMContext(), "all-vtables"));
         llvm::Value *ValidVtable = Builder.CreateCall(
             CGM.getIntrinsic(llvm::Intrinsic::type_test), {VTable, AllVtables});
-      CGF.EmitCheck(std::make_pair(TypeTest, SanitizerKind::CFIMFCall),
+        CGF.EmitCheck(std::make_pair(CheckResult, SanitizerKind::CFIMFCall),
                       SanitizerHandler::CFICheckFail, StaticData,
                       {VTable, ValidVtable});
       }
 
       FnVirtual = Builder.GetInsertBlock();
     }
+  } // End of sanitizer scope
 
-  // Load the virtual function to call.
-  VFPAddr = Builder.CreateBitCast(VFPAddr, FTy->getPointerTo()->getPointerTo());
-  llvm::Value *VirtualFn = Builder.CreateAlignedLoad(
-      VFPAddr, CGF.getPointerAlign(), "memptr.virtualfn");
   CGF.EmitBranch(FnEnd);
 
   // In the non-virtual path, the function pointer is actually a
   // function pointer.
   CGF.EmitBlock(FnNonVirtual);
   llvm::Value *NonVirtualFn =
     Builder.CreateIntToPtr(FnAsInt, FTy->getPointerTo(), "memptr.nonvirtualfn");
 
void ItaniumCXXABI::emitVTableDefinitions(CodeGenVTables &CGVT,
   if (RD->getIdentifier() &&
       RD->getIdentifier()->isStr("__fundamental_type_info") &&
       isa<NamespaceDecl>(DC) && cast<NamespaceDecl>(DC)->getIdentifier() &&
       cast<NamespaceDecl>(DC)->getIdentifier()->isStr("__cxxabiv1") &&
       DC->getParent()->isTranslationUnit())
     EmitFundamentalRTTIDescriptors(RD);
 
   if (!VTable->isDeclarationForLinker())
-    CGM.EmitVTableTypeMetadata(VTable, VTLayout);
+    CGM.EmitVTableTypeMetadata(RD, VTable, VTLayout);
 }
 
 bool ItaniumCXXABI::isVirtualOffsetNeededForVTableField(
     CodeGenFunction &CGF, CodeGenFunction::VPtr Vptr) {
   if (Vptr.NearestVBase == nullptr)
     return false;
   return NeedsVTTParameter(CGF.CurGD);
 }

clang/lib/Driver/ToolChains/Clang.cpp

for (unsigned i = 0; i < Tgts->getNumValues(); ++i) {
       // We need to get the string from the triple because it may be not exactly
       // the same as the one we get directly from the arguments.
       llvm::Triple T(Tgts->getValue(i));
       TargetInfo += T.getTriple();
     }
     CmdArgs.push_back(Args.MakeArgString(TargetInfo.str()));
   }
 
-  bool WholeProgramVTables =
-      Args.hasFlag(options::OPT_fwhole_program_vtables,
-                   options::OPT_fno_whole_program_vtables, false);
+  bool VirtualFunctionElimination =
+      Args.hasFlag(options::OPT_fvirtual_function_elimination,
+                   options::OPT_fno_virtual_function_elimination, false);
+  if (VirtualFunctionElimination) {
+    // VFE requires full LTO (currently, this might be relaxed to allow ThinLTO
+    // in the future).
+    if (D.getLTOMode() != LTOK_Full)
+      D.Diag(diag::err_drv_argument_only_allowed_with)
+          << "-fvirtual-function-elimination"
+          << "-flto=full";
+
+    CmdArgs.push_back("-fvirtual-function-elimination");
+  }
+
+  // VFE requires whole-program-vtables, and enables it by default.
+  bool WholeProgramVTables = Args.hasFlag(
+      options::OPT_fwhole_program_vtables,
+      options::OPT_fno_whole_program_vtables, VirtualFunctionElimination);
+  if (VirtualFunctionElimination && !WholeProgramVTables) {
+    D.Diag(diag::err_drv_argument_not_allowed_with)
+        << "-fno-whole-program-vtables"
+        << "-fvirtual-function-elimination";
+  }
+
   if (WholeProgramVTables) {
     if (!D.isUsingLTO())
       D.Diag(diag::err_drv_argument_only_allowed_with)
           << "-fwhole-program-vtables"
           << "-flto";
     CmdArgs.push_back("-fwhole-program-vtables");
   }
 

clang/lib/Frontend/CompilerInvocation.cpp

else
       Opts.setDebuggerTuning(static_cast<llvm::DebuggerKind>(Val));
   }
   Opts.DwarfVersion = getLastArgIntValue(Args, OPT_dwarf_version_EQ, 0, Diags);
   Opts.DebugColumnInfo = Args.hasArg(OPT_dwarf_column_info);
   Opts.EmitCodeView = Args.hasArg(OPT_gcodeview);
   Opts.CodeViewGHash = Args.hasArg(OPT_gcodeview_ghash);
   Opts.MacroDebugInfo = Args.hasArg(OPT_debug_info_macro);
   Opts.WholeProgramVTables = Args.hasArg(OPT_fwhole_program_vtables);
+  Opts.VirtualFunctionElimination =
+      Args.hasArg(OPT_fvirtual_function_elimination);
   Opts.LTOVisibilityPublicStd = Args.hasArg(OPT_flto_visibility_public_std);
   Opts.SplitDwarfFile = Args.getLastArgValue(OPT_split_dwarf_file);
   Opts.SplitDwarfOutput = Args.getLastArgValue(OPT_split_dwarf_output);
   Opts.SplitDwarfInlining = !Args.hasArg(OPT_fno_split_dwarf_inlining);
   Opts.DebugTypeExtRefs = Args.hasArg(OPT_dwarf_ext_refs);
   Opts.DebugExplicitImport = Args.hasArg(OPT_dwarf_explicit_import);
   Opts.DebugFwdTemplateParams = Args.hasArg(OPT_debug_forward_template_params);
   Opts.EmbedSource = Args.hasArg(OPT_gembed_source);

clang/test/CodeGenCXX/vcall-visibility-metadata.cpp

+// RUN: %clang_cc1 -flto -flto-unit -triple x86_64-unknown-linux -emit-llvm -fvirtual-function-elimination -fwhole-program-vtables -o - %s | FileCheck %s
+
+
+// Anonymous namespace.
+namespace {
+// CHECK: @_ZTVN12_GLOBAL__N_11AE = {{.*}} !vcall_visibility [[VIS_TU:![0-9]+]]
+struct A {
+  A() {}
+  virtual int f() { return 1; }
+};
+}
+void *construct_A() {
+  return new A();
+}
+
+
+// Hidden visibility.
+// CHECK: @_ZTV1B = {{.*}} !vcall_visibility [[VIS_DSO:![0-9]+]]
+struct __attribute__((visibility("hidden"))) B {
+  B() {}
+  virtual int f() { return 1; }
+};
+B *construct_B() {
+  return new B();
+}
+
+
+// Default visibility.
+// CHECK-NOT: @_ZTV1C = {{.*}} !vcall_visibility
+struct __attribute__((visibility("default"))) C {
+  C() {}
+  virtual int f() { return 1; }
+};
+C *construct_C() {
+  return new C();
+}
+
+
+// Hidden visibility, public LTO visibility.
+// CHECK-NOT: @_ZTV1D = {{.*}} !vcall_visibility
+struct __attribute__((visibility("hidden"))) [[clang::lto_visibility_public]] D {
+  D() {}
+  virtual int f() { return 1; }
+};
+D *construct_D() {
+  return new D();
+}
+
+
+// Hidden visibility, but inherits from class with default visibility.
+// CHECK-NOT: @_ZTV1E = {{.*}} !vcall_visibility
+struct __attribute__((visibility("hidden"))) E : C {
+  E() {}
+  virtual int f() { return 1; }
+};
+E *construct_E() {
+  return new E();
+}
+
+
+// Anonymous namespace, but inherits from class with default visibility.
+// CHECK-NOT: @_ZTVN12_GLOBAL__N_11FE = {{.*}} !vcall_visibility
+namespace {
+struct __attribute__((visibility("hidden"))) F : C {
+  F() {}
+  virtual int f() { return 1; }
+};
+}
+void *construct_F() {
+  return new F();
+}
+
+
+// Anonymous namespace, but inherits from class with hidden visibility.
+// CHECK: @_ZTVN12_GLOBAL__N_11GE = {{.*}} !vcall_visibility [[VIS_DSO:![0-9]+]]
+namespace {
+struct __attribute__((visibility("hidden"))) G : B {
+  G() {}
+  virtual int f() { return 1; }
+};
+}
+void *construct_G() {
+  return new G();
+}
+
+
+// CHECK-DAG: [[VIS_DSO]] = !{i64 1}
+// CHECK-DAG: [[VIS_TU]] = !{i64 2}

clang/test/CodeGenCXX/virtual-function-elimination.cpp

+// RUN: %clang_cc1 -triple x86_64-unknown-linux -flto -flto-unit -fvirtual-function-elimination -fwhole-program-vtables -emit-llvm -o - %s | FileCheck %s
+
+
+struct __attribute__((visibility("default"))) A {
+  virtual void foo();
+};
+
+void test_1(A *p) {
+  // A has default visibility, so no need for type.checked.load.
+// CHECK-LABEL: define void @_Z6test_1P1A
+// CHECK: [[FN_PTR_ADDR:%.+]] = getelementptr inbounds void (%struct.A*)*, void (%struct.A*)** {{%.+}}, i64 0
+// CHECK: [[FN_PTR:%.+]] = load void (%struct.A*)*, void (%struct.A*)** [[FN_PTR_ADDR]]
+// CHECK: call void [[FN_PTR]](
+  p->foo();
+}
+
+
+struct __attribute__((visibility("hidden"))) [[clang::lto_visibility_public]] B {
+  virtual void foo();
+};
+
+void test_2(B *p) {
+  // B has public LTO visibility, so no need for type.checked.load.
+// CHECK-LABEL: define void @_Z6test_2P1B
+// CHECK: [[FN_PTR_ADDR:%.+]] = getelementptr inbounds void (%struct.B*)*, void (%struct.B*)** {{%.+}}, i64 0
+// CHECK: [[FN_PTR:%.+]] = load void (%struct.B*)*, void (%struct.B*)** [[FN_PTR_ADDR]]
+// CHECK: call void [[FN_PTR]](
+  p->foo();
+}
+
+
+struct __attribute__((visibility("hidden"))) C {
+  virtual void foo();
+  virtual void bar();
+};
+
+void test_3(C *p) {
+  // C has hidden visibility, so we generate type.checked.load to allow VFE.
+// CHECK-LABEL: define void @_Z6test_3P1C
+// CHECK: [[LOAD:%.+]] = call { i8*, i1 } @llvm.type.checked.load(i8* {{%.+}}, i32 0, metadata !"_ZTS1C")
+// CHECK: [[FN_PTR_I8:%.+]] = extractvalue { i8*, i1 } [[LOAD]], 0
+// CHECK: [[FN_PTR:%.+]] = bitcast i8* [[FN_PTR_I8]] to void (%struct.C*)*
+// CHECK: call void [[FN_PTR]](
+  p->foo();
+}
+
+void test_4(C *p) {
+  // When using type.checked.load, we pass the vtable offset to the intrinsic,
+  // rather than adding it to the pointer with a GEP.
+// CHECK-LABEL: define void @_Z6test_4P1C
+// CHECK: [[LOAD:%.+]] = call { i8*, i1 } @llvm.type.checked.load(i8* {{%.+}}, i32 8, metadata !"_ZTS1C")
+// CHECK: [[FN_PTR_I8:%.+]] = extractvalue { i8*, i1 } [[LOAD]], 0
+// CHECK: [[FN_PTR:%.+]] = bitcast i8* [[FN_PTR_I8]] to void (%struct.C*)*
+// CHECK: call void [[FN_PTR]](
+  p->bar();
+}
+
+void test_5(C *p, void (C::*q)(void)) {
+  // We also use type.checked.load for the virtual side of member function
+  // pointer calls. We use a GEP to calculate the address to load from and pass
+  // 0 as the offset to the intrinsic, because we know that the load must be
+  // from exactly the point marked by one of the function-type metadatas (in
+  // this case "_ZTSM1CFvvE.virtual"). If we passed the offset from the member
+  // function pointer to the intrinsic, this information would be lost. No
+  // codegen changes on the non-virtual side.
+// CHECK-LABEL: define void @_Z6test_5P1CMS_FvvE(
+// CHECK: [[FN_PTR_ADDR:%.+]] = getelementptr i8, i8* %vtable, i64 {{%.+}}
+// CHECK: [[LOAD:%.+]] = call { i8*, i1 } @llvm.type.checked.load(i8* [[FN_PTR_ADDR]], i32 0, metadata !"_ZTSM1CFvvE.virtual")
+// CHECK: [[FN_PTR_I8:%.+]] = extractvalue { i8*, i1 } [[LOAD]], 0
+// CHECK: [[FN_PTR:%.+]] = bitcast i8* [[FN_PTR_I8]] to void (%struct.C*)*
+
+// CHECK: [[PHI:%.+]] = phi void (%struct.C*)* {{.*}}[ [[FN_PTR]], {{.*}} ]
+// CHECK: call void [[PHI]](
+  (p->*q)();
+}

clang/test/Driver/virtual-function-elimination.cpp

+// RUN: %clang -target x86_64-unknown-linux -fvirtual-function-elimination -### %s 2>&1 | FileCheck --check-prefix=BAD-LTO %s
+// RUN: %clang -target x86_64-unknown-linux -fvirtual-function-elimination -flto=thin -### %s 2>&1 | FileCheck --check-prefix=BAD-LTO %s
+// BAD-LTO: invalid argument '-fvirtual-function-elimination' only allowed with '-flto=full'
+
+// RUN: %clang -target x86_64-unknown-linux -fvirtual-function-elimination -flto -### %s 2>&1 | FileCheck --check-prefix=GOOD %s
+// RUN: %clang -target x86_64-unknown-linux -fvirtual-function-elimination -flto=full -### %s 2>&1 | FileCheck --check-prefix=GOOD %s
+// RUN: %clang -target x86_64-unknown-linux -fvirtual-function-elimination -flto -fwhole-program-vtables -### %s 2>&1 | FileCheck --check-prefix=GOOD %s
+// GOOD: "-fvirtual-function-elimination" "-fwhole-program-vtables"
+
+// RUN: %clang -target x86_64-unknown-linux -fvirtual-function-elimination -fno-whole-program-vtables -flto -### %s 2>&1 | FileCheck --check-prefix=NO-WHOLE-PROGRAM-VTABLES %s
+// NO-WHOLE-PROGRAM-VTABLES: invalid argument '-fno-whole-program-vtables' not allowed with '-fvirtual-function-elimination'

llvm/docs/LangRef.rst

 For example, the following metadata section specifies that the module was
 compiled with a ``wchar_t`` width of 4 bytes, and the underlying type of an
 enum is the smallest type which can represent all of its values::
 
     !llvm.module.flags = !{!0, !1}
     !0 = !{i32 1, !"short_wchar", i32 1}
     !1 = !{i32 1, !"short_enum", i32 0}
 
+LTO Post-Link Module Flags Metadata
+-----------------------------------
+
+Some optimisations are only when the entire LTO unit is present in the current
+module. This is represented by the ``LTOPostLink`` module flags metadata, which
+will be created with a value of ``1`` when LTO linking occurs.
+
 Automatic Linker Flags Named Metadata
 =====================================
 
 Some targets support embedding of flags to the linker inside individual object
 files. Typically this is used in conjunction with language extensions which
 allow source files to contain linker command line options, and have these
 automatically be transmitted to the linker via object files.
 

 metadata object representing a :doc:`type identifier <TypeMetadata>`.
 
 Overview:
 """""""""
 
 The ``llvm.type.test`` intrinsic tests whether the given pointer is associated
 with the given type identifier.
 
+.. _type.checked.load:
+
 '``llvm.type.checked.load``' Intrinsic
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Syntax:
 """""""
 
 ::
 

llvm/docs/TypeMetadata.rst

define void @main() {
       %d12 = call i1 @bar(i32* getelementptr ([2 x i32]* @d, i32 0, i32 1)) ; returns 1
       %e = call i1 @baz(void ()* @e) ; returns 1
       %f = call i1 @baz(void ()* @f) ; returns 0
       %g = call i1 @baz(void ()* @g) ; returns 1
       ret void
     }
 
 .. _GlobalLayoutBuilder: https://github.com/llvm/llvm-project/blob/master/llvm/include/llvm/Transforms/IPO/LowerTypeTests.h
+
+``!vcall_visibility`` Metadata
+==============================
+
+In order to allow removing unused function pointers from vtables, we need to
+know whether every virtual call which could use it is known to the compiler, or
+whether another translation unit could introduce more calls through the vtable.
+This is not the same as the linkage of the vtable, because call sites could be
+using a pointer of a more widely-visible base class. For example, consider this
+code:
+
+.. code-block:: c++
+
+  __attribute__((visibility("default")))
+  struct A {
+    virtual void f();
+  };
+
+  __attribute__((visibility("hidden")))
+  struct B : A {
+    virtual void f();
+  };
+
+With LTO, we know that all code which can see the declaration of ``B`` is
+visible to us. However, a pointer to a ``B`` could be cast to ``A*`` and passed
+to another linkage unit, which could then call ``f`` on it. This call would
+load from the vtable for ``B`` (using the object pointer), and then call
+``B::f``. This means we can't remove the function pointer from ``B``'s vtable,
+or the implementation of ``B::f``. However, if we can see all code which knows
+about any dynamic base class (which would be the case if ``B`` only inherited
+from classes with hidden visibility), then this optimisation would be valid.
+
+This concept is represented in IR by the ``!vcall_visibility`` metadata
+attached to vtable objects, with the following values:
+
+.. list-table::
+   :header-rows: 1
+   :widths: 10 90
+
+   * - Value
+     - Behavior
+
+   * - 0 (or omitted)
+     - **Public**
+           Virtual function calls using this vtable could be made from external
+           code.
+
+   * - 1
+     - **Linkage Unit**
+           All virtual function calls which might use this vtable are in the
+           current LTO unit, meaning they will be in the current module once
+           LTO linking has been performed.
+
+   * - 2
+     - **Translation Unit**
+           All virtual function calls which might use this vtable are in the
+           current module.
+
+In addition, all function pointer loads from a vtable marked with the
+``!vcall_visibility`` metadata (with a non-zero value) must be done using the
+:ref:`llvm.type.checked.load <type.checked.load>` intrinsic, so that virtual
+calls sites can be correlated with the vtables which they might load from.
+Other parts of the vtable (RTTI, offset-to-top, ...) can still be accessed with
+normal loads.

llvm/include/llvm/Analysis/TypeMetadataUtils.h

 
 /// Given a call to the intrinsic \@llvm.type.checked.load, find all
 /// devirtualizable call sites based on the call and return them in DevirtCalls.
 void findDevirtualizableCallsForTypeCheckedLoad(
     SmallVectorImpl<DevirtCallSite> &DevirtCalls,
     SmallVectorImpl<Instruction *> &LoadedPtrs,
     SmallVectorImpl<Instruction *> &Preds, bool &HasNonCallUses,
     const CallInst *CI, DominatorTree &DT);
+
+Constant *getPointerAtOffset(Constant *I, uint64_t Offset, Module &M);
 }
 
 #endif

llvm/include/llvm/IR/FixedMetadataKinds.def

 LLVM_FIXED_MD_KIND(MD_absolute_symbol, "absolute_symbol", 21)
 LLVM_FIXED_MD_KIND(MD_associated, "associated", 22)
 LLVM_FIXED_MD_KIND(MD_callees, "callees", 23)
 LLVM_FIXED_MD_KIND(MD_irr_loop, "irr_loop", 24)
 LLVM_FIXED_MD_KIND(MD_access_group, "llvm.access.group", 25)
 LLVM_FIXED_MD_KIND(MD_callback, "callback", 26)
 LLVM_FIXED_MD_KIND(MD_preserve_access_index, "llvm.preserve.access.index", 27)
 LLVM_FIXED_MD_KIND(MD_misexpect, "misexpect", 28)
+LLVM_FIXED_MD_KIND(MD_vcall_visibility, "vcall_visibility", 29)

llvm/include/llvm/IR/GlobalObject.h

 
 namespace llvm {
 
 class Comdat;
 class MDNode;
 class Metadata;
 
 class GlobalObject : public GlobalValue {
+public:
+  // VCallVisibility - values for visibility metadata attached to vtables. This
+  // describes the scope in which a virtual call could end up being dispatched
+  // through this vtable.
+  enum VCallVisibility {
+    // Type is potentially visible to external code.
+    VCallVisibilityPublic = 0,
+    // Type is only visible to code which will be in the current Module after
+    // LTO internalization.
+    VCallVisibilityLinkageUnit = 1,
+    // Type is only visible to code in the current Module.
+    VCallVisibilityTranslationUnit = 2,
+  };
+
 protected:
   GlobalObject(Type *Ty, ValueTy VTy, Use *Ops, unsigned NumOps,
                LinkageTypes Linkage, const Twine &Name,
                unsigned AddressSpace = 0)
       : GlobalValue(Ty, VTy, Ops, NumOps, Linkage, Name, AddressSpace),
         ObjComdat(nullptr) {
     setGlobalValueSubClassData(0);
   }
public:
   ///
   /// \returns true if any metadata was removed.
   bool eraseMetadata(unsigned KindID);
 
   /// Copy metadata from Src, adjusting offsets by Offset.
   void copyMetadata(const GlobalObject *Src, unsigned Offset);
 
   void addTypeMetadata(unsigned Offset, Metadata *TypeID);
+  void addVCallVisibilityMetadata(VCallVisibility Visibility);
+  VCallVisibility getVCallVisibility() const;
 
 protected:
   void copyAttributesFrom(const GlobalObject *Src);
 
 public:
   // Methods for support type inquiry through isa, cast, and dyn_cast:
   static bool classof(const Value *V) {
     return V->getValueID() == Value::FunctionVal ||

llvm/include/llvm/Transforms/IPO/GlobalDCE.h

private:
 
   /// Constant -> Globals that use this global cache.
   std::unordered_map<Constant *, SmallPtrSet<GlobalValue *, 8>>
       ConstantDependenciesCache;
 
   /// Comdat -> Globals in that Comdat section.
   std::unordered_multimap<Comdat *, GlobalValue *> ComdatMembers;
 
+  /// !type metadata -> set of (vtable, offset) pairs
+  DenseMap<Metadata *, SmallSet<std::pair<GlobalVariable *, uint64_t>, 4>>
+      TypeIdMap;
+
+  // Global variables which are vtables, and which we have enough information
+  // about to safely do dead virtual function elimination.
+  SmallPtrSet<GlobalValue *, 32> VFESafeVTables;
+
   void UpdateGVDependencies(GlobalValue &GV);
   void MarkLive(GlobalValue &GV,
                 SmallVectorImpl<GlobalValue *> *Updates = nullptr);
   bool RemoveUnusedGlobalValue(GlobalValue &GV);
 
+  // Dead virtual function elimination.
+  void AddVirtualFunctionDependencies(Module &M);
+  void ScanVTables(Module &M);
+  void ScanTypeCheckedLoadIntrinsics(Module &M);
+  void ScanVTableLoad(Function *Caller, Metadata *TypeId, uint64_t CallOffset);
+
   void ComputeDependencies(Value *V, SmallPtrSetImpl<GlobalValue *> &U);
 };
 
 }
 
 #endif // LLVM_TRANSFORMS_IPO_GLOBALDCE_H

llvm/lib/Analysis/TypeMetadataUtils.cpp

for (const Use &U : CI->uses()) {
     }
     HasNonCallUses = true;
   }
 
   for (Value *LoadedPtr : LoadedPtrs)
     findCallsAtConstantOffset(DevirtCalls, &HasNonCallUses, LoadedPtr,
                               Offset->getZExtValue(), CI, DT);
 }
+
+Constant *llvm::getPointerAtOffset(Constant *I, uint64_t Offset, Module &M) {
+  if (I->getType()->isPointerTy()) {
+    if (Offset == 0)
+      return I;
+    return nullptr;
+  }
+
+  const DataLayout &DL = M.getDataLayout();
+
+  if (auto *C = dyn_cast<ConstantStruct>(I)) {
+    const StructLayout *SL = DL.getStructLayout(C->getType());
+    if (Offset >= SL->getSizeInBytes())
+      return nullptr;
+
+    unsigned Op = SL->getElementContainingOffset(Offset);
+    return getPointerAtOffset(cast<Constant>(I->getOperand(Op)),
+                              Offset - SL->getElementOffset(Op), M);
+  }
+  if (auto *C = dyn_cast<ConstantArray>(I)) {
+    ArrayType *VTableTy = C->getType();
+    uint64_t ElemSize = DL.getTypeAllocSize(VTableTy->getElementType());
+
+    unsigned Op = Offset / ElemSize;
+    if (Op >= C->getNumOperands())
+      return nullptr;
+
+    return getPointerAtOffset(cast<Constant>(I->getOperand(Op)),
+                              Offset % ElemSize, M);
+  }
+  return nullptr;
+}

llvm/lib/IR/Metadata.cpp

void GlobalObject::addTypeMetadata(unsigned Offset, Metadata *TypeID) {
   addMetadata(
       LLVMContext::MD_type,
       *MDTuple::get(getContext(),
                     {ConstantAsMetadata::get(ConstantInt::get(
                          Type::getInt64Ty(getContext()), Offset)),
                      TypeID}));
 }
 
+void GlobalObject::addVCallVisibilityMetadata(VCallVisibility Visibility) {
+  addMetadata(LLVMContext::MD_vcall_visibility,
+              *MDNode::get(getContext(),
+                           {ConstantAsMetadata::get(ConstantInt::get(
+                               Type::getInt64Ty(getContext()), Visibility))}));
+}
+
+GlobalObject::VCallVisibility GlobalObject::getVCallVisibility() const {
+  if (MDNode *MD = getMetadata(LLVMContext::MD_vcall_visibility)) {
+    uint64_t Val = cast<ConstantInt>(
+                       cast<ConstantAsMetadata>(MD->getOperand(0))->getValue())
+                       ->getZExtValue();
+    assert((Val >= 0 && Val <= 2) && "unknown vcall visibility!");
+    return (VCallVisibility)Val;
+  }
+  return VCallVisibility::VCallVisibilityPublic;
+}
+
 void Function::setSubprogram(DISubprogram *SP) {
   setMetadata(LLVMContext::MD_dbg, SP);
 }
 
 DISubprogram *Function::getSubprogram() const {
   return cast_or_null<DISubprogram>(getMetadata(LLVMContext::MD_dbg));
 }
 

llvm/lib/LTO/LTO.cpp

for (const auto &R : GlobalResolutions) {
       if (!GV || GV->hasLocalLinkage() || GV->isDeclaration())
         continue;
       GV->setUnnamedAddr(R.second.UnnamedAddr ? GlobalValue::UnnamedAddr::Global
                                               : GlobalValue::UnnamedAddr::None);
       if (EnableLTOInternalization && R.second.Partition == 0)
         GV->setLinkage(GlobalValue::InternalLinkage);
     }
 
+    RegularLTO.CombinedModule->addModuleFlag(Module::Error, "LTOPostLink", 1);
+
     if (Conf.PostInternalizeModuleHook &&
         !Conf.PostInternalizeModuleHook(0, *RegularLTO.CombinedModule))
       return Error::success();
   }
   return backend(Conf, AddStream, RegularLTO.ParallelCodeGenParallelismLevel,
                  std::move(RegularLTO.CombinedModule), ThinLTO.CombinedIndex);
 }
 

llvm/lib/LTO/LTOCodeGenerator.cpp

void LTOCodeGenerator::applyScopeRestrictions() {
   }
 
   // Update the llvm.compiler_used globals to force preserving libcalls and
   // symbols referenced from asm
   updateCompilerUsed(*MergedModule, *TargetMach, AsmUndefinedRefs);
 
   internalizeModule(*MergedModule, mustPreserveGV);
 
+  MergedModule->addModuleFlag(Module::Error, "LTOPostLink", 1);
+
   ScopeRestrictionsDone = true;
 }
 
 /// Restore original linkage for symbols that may have been internalized
 void LTOCodeGenerator::restoreLinkageForExternals() {
   if (!ShouldInternalize || !ShouldRestoreGlobalsLinkage)
     return;
 

llvm/lib/Transforms/IPO/GlobalDCE.cpp

 // deletes whatever is left over.  This allows it to delete recursive chunks of
 // the program which are unreachable.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Transforms/IPO/GlobalDCE.h"
 #include "llvm/ADT/SmallPtrSet.h"
 #include "llvm/ADT/Statistic.h"
+#include "llvm/Analysis/TypeMetadataUtils.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/Module.h"
+#include "llvm/IR/Operator.h"
 #include "llvm/Pass.h"
 #include "llvm/Transforms/IPO.h"
 #include "llvm/Transforms/Utils/CtorUtils.h"
 #include "llvm/Transforms/Utils/GlobalStatus.h"
 
 using namespace llvm;
 
 #define DEBUG_TYPE "globaldce"
 
+static cl::opt<bool>
+    ClEnableVFE("enable-vfe", cl::Hidden, cl::init(true), cl::ZeroOrMore,
+                cl::desc("Enable virtual function elimination"));
+
 STATISTIC(NumAliases  , "Number of global aliases removed");
 STATISTIC(NumFunctions, "Number of functions removed");
 STATISTIC(NumIFuncs,    "Number of indirect functions removed");
 STATISTIC(NumVariables, "Number of global variables removed");
+STATISTIC(NumVFuncs,    "Number of virtual functions removed");
 
 namespace {
   class GlobalDCELegacyPass : public ModulePass {
   public:
     static char ID; // Pass identification, replacement for typeid
     GlobalDCELegacyPass() : ModulePass(ID) {
       initializeGlobalDCELegacyPassPass(*PassRegistry::getPassRegistry());
     }

 }
 
 void GlobalDCEPass::UpdateGVDependencies(GlobalValue &GV) {
   SmallPtrSet<GlobalValue *, 8> Deps;
   for (User *User : GV.users())
     ComputeDependencies(User, Deps);
   Deps.erase(&GV); // Remove self-reference.
   for (GlobalValue *GVU : Deps) {
+    // If this is a dep from a vtable to a virtual function, and we have
+    // complete information about all virtual call sites which could call
+    // though this vtable, then skip it, because the call site information will
+    // be more precise.
+    if (VFESafeVTables.count(GVU) && isa<Function>(&GV)) {
+      LLVM_DEBUG(dbgs() << "Ignoring dep " << GVU->getName() << " -> "
+                        << GV.getName() << "\n");
+      continue;
+    }
     GVDependencies[GVU].insert(&GV);
   }
 }
 
 /// Mark Global value as Live
 void GlobalDCEPass::MarkLive(GlobalValue &GV,
                              SmallVectorImpl<GlobalValue *> *Updates) {
   auto const Ret = AliveGlobals.insert(&GV);
   if (!Ret.second)
     return;
 
   if (Updates)
     Updates->push_back(&GV);
   if (Comdat *C = GV.getComdat()) {
-    for (auto &&CM : make_range(ComdatMembers.equal_range(C)))
+    for (auto &&CM : make_range(ComdatMembers.equal_range(C))) {
       MarkLive(*CM.second, Updates); // Recursion depth is only two because only
                                      // globals in the same comdat are visited.
     }
   }
+}
+
+void GlobalDCEPass::ScanVTables(Module &M) {
+  SmallVector<MDNode *, 2> Types;
+  LLVM_DEBUG(dbgs() << "Building type info -> vtable map\n");
+
+  auto *LTOPostLinkMD =
+      cast_or_null<ConstantAsMetadata>(M.getModuleFlag("LTOPostLink"));
+  bool LTOPostLink =
+      LTOPostLinkMD &&
+      (cast<ConstantInt>(LTOPostLinkMD->getValue())->getZExtValue() != 0);
+
+  for (GlobalVariable &GV : M.globals()) {
+    Types.clear();
+    GV.getMetadata(LLVMContext::MD_type, Types);
+    if (GV.isDeclaration() || Types.empty())
+      continue;
+
+    // Use the typeid metadata on the vtable to build a mapping from typeids to
+    // the list of (GV, offset) pairs which are the possible vtables for that
+    // typeid.
+    for (MDNode *Type : Types) {
+      Metadata *TypeID = Type->getOperand(1).get();
+
+      uint64_t Offset =
+          cast<ConstantInt>(
+              cast<ConstantAsMetadata>(Type->getOperand(0))->getValue())
+              ->getZExtValue();
+
+      TypeIdMap[TypeID].insert(std::make_pair(&GV, Offset));
+    }
+
+    // If the type corresponding to the vtable is private to this translation
+    // unit, we know that we can see all virtual functions which might use it,
+    // so VFE is safe.
+    if (auto GO = dyn_cast<GlobalObject>(&GV)) {
+      GlobalObject::VCallVisibility TypeVis = GV.getVCallVisibility();
+      if (TypeVis == GlobalObject::VCallVisibilityTranslationUnit ||
+          (LTOPostLink &&
+           TypeVis == GlobalObject::VCallVisibilityLinkageUnit)) {
+        LLVM_DEBUG(dbgs() << GV.getName() << " is safe for VFE\n");
+        VFESafeVTables.insert(&GV);
+      }
+    }
+  }
+}
+
+void GlobalDCEPass::ScanVTableLoad(Function *Caller, Metadata *TypeId,
+                                   uint64_t CallOffset) {
+  for (auto &VTableInfo : TypeIdMap[TypeId]) {
+    GlobalVariable *VTable = VTableInfo.first;
+    uint64_t VTableOffset = VTableInfo.second;
+
+    Constant *Ptr =
+        getPointerAtOffset(VTable->getInitializer(), VTableOffset + CallOffset,
+                           *Caller->getParent());
+    if (!Ptr) {
+      LLVM_DEBUG(dbgs() << "can't find pointer in vtable!\n");
+      VFESafeVTables.erase(VTable);
+      return;
+    }
+
+    auto Callee = dyn_cast<Function>(Ptr->stripPointerCasts());
+    if (!Callee) {
+      LLVM_DEBUG(dbgs() << "vtable entry is not function pointer!\n");
+      VFESafeVTables.erase(VTable);
+      return;
+    }
+
+    LLVM_DEBUG(dbgs() << "vfunc dep " << Caller->getName() << " -> "
+                      << Callee->getName() << "\n");
+    GVDependencies[Caller].insert(Callee);
+  }
+}
+
+void GlobalDCEPass::ScanTypeCheckedLoadIntrinsics(Module &M) {
+  LLVM_DEBUG(dbgs() << "Scanning type.checked.load intrinsics\n");
+  Function *TypeCheckedLoadFunc =
+      M.getFunction(Intrinsic::getName(Intrinsic::type_checked_load));
+
+  if (!TypeCheckedLoadFunc)
+    return;
+
+  for (auto U : TypeCheckedLoadFunc->users()) {
+    auto CI = dyn_cast<CallInst>(U);
+    if (!CI)
+      continue;
+
+    auto *Offset = dyn_cast<ConstantInt>(CI->getArgOperand(1));
+    Value *TypeIdValue = CI->getArgOperand(2);
+    auto *TypeId = cast<MetadataAsValue>(TypeIdValue)->getMetadata();
+
+    if (Offset) {
+      ScanVTableLoad(CI->getFunction(), TypeId, Offset->getZExtValue());
+    } else {
+      // type.checked.load with a non-constant offset, so assume every entry in
+      // every matching vtable is used.
+      for (auto &VTableInfo : TypeIdMap[TypeId]) {
+        VFESafeVTables.erase(VTableInfo.first);
+      }
+    }
+  }
+}
+
+void GlobalDCEPass::AddVirtualFunctionDependencies(Module &M) {
+  if (!ClEnableVFE)
+    return;
+
+  ScanVTables(M);
+
+  if (VFESafeVTables.empty())
+    return;
+
+  ScanTypeCheckedLoadIntrinsics(M);
+
+  LLVM_DEBUG(
+    dbgs() << "VFE safe vtables:\n";
+    for (auto *VTable : VFESafeVTables)
+      dbgs() << "  " << VTable->getName() << "\n";
+  );
+}
 
 PreservedAnalyses GlobalDCEPass::run(Module &M, ModuleAnalysisManager &MAM) {
   bool Changed = false;
 
   // The algorithm first computes the set L of global variables that are
   // trivially live.  Then it walks the initialization of these variables to
   // compute the globals used to initialize them, which effectively builds a
   // directed graph where nodes are global variables, and an edge from A to B
if (Comdat *C = F.getComdat())
       ComdatMembers.insert(std::make_pair(C, &F));
   for (GlobalVariable &GV : M.globals())
     if (Comdat *C = GV.getComdat())
       ComdatMembers.insert(std::make_pair(C, &GV));
   for (GlobalAlias &GA : M.aliases())
     if (Comdat *C = GA.getComdat())
       ComdatMembers.insert(std::make_pair(C, &GA));
 
+  // Add dependencies between virtual call sites and the virtual functions they
+  // might call, if we have that information.
+  AddVirtualFunctionDependencies(M);
+
   // Loop over the module, adding globals which are obviously necessary.
   for (GlobalObject &GO : M.global_objects()) {
     Changed |= RemoveUnusedGlobalValue(GO);
     // Functions with external linkage are needed if they have a body.
     // Externally visible & appending globals are needed, if they have an
     // initializer.
     if (!GO.isDeclaration())
       if (!GO.isDiscardableIfUnused())
PreservedAnalyses GlobalDCEPass::run(Module &M, ModuleAnalysisManager &MAM) {
   // themselves.
   auto EraseUnusedGlobalValue = [&](GlobalValue *GV) {
     RemoveUnusedGlobalValue(*GV);
     GV->eraseFromParent();
     Changed = true;
   };
 
   NumFunctions += DeadFunctions.size();
-  for (Function *F : DeadFunctions)
+  for (Function *F : DeadFunctions) {
+    if (!F->use_empty()) {
+      // Virtual functions might still be referenced by one or more vtables,
+      // but if we've proven them to be unused then it's safe to replace the
+      // virtual function pointers with null, allowing us to remove the
+      // function itself.
+      ++NumVFuncs;
+      F->replaceAllUsesWith(ConstantPointerNull::get(F->getType()));
+    }
     EraseUnusedGlobalValue(F);
+  }
 
   NumVariables += DeadGlobalVars.size();
   for (GlobalVariable *GV : DeadGlobalVars)
     EraseUnusedGlobalValue(GV);
 
   NumAliases += DeadAliases.size();
   for (GlobalAlias *GA : DeadAliases)
     EraseUnusedGlobalValue(GA);
 
   NumIFuncs += DeadIFuncs.size();
   for (GlobalIFunc *GIF : DeadIFuncs)
     EraseUnusedGlobalValue(GIF);
 
   // Make sure that all memory is released
   AliveGlobals.clear();
   ConstantDependenciesCache.clear();
   GVDependencies.clear();
   ComdatMembers.clear();
+  TypeIdMap.clear();
+  VFESafeVTables.clear();
 
   if (Changed)
     return PreservedAnalyses::none();
   return PreservedAnalyses::all();
 }
 
 // RemoveUnusedGlobalValue - Loop over all of the uses of the specified
 // GlobalValue, looking for the constant pointer ref that may be pointing to it.

llvm/lib/Transforms/IPO/WholeProgramDevirt.cpp

struct DevirtModule {
   bool areRemarksEnabled();
 
   void scanTypeTestUsers(Function *TypeTestFunc, Function *AssumeFunc);
   void scanTypeCheckedLoadUsers(Function *TypeCheckedLoadFunc);
 
   void buildTypeIdentifierMap(
       std::vector<VTableBits> &Bits,
       DenseMap<Metadata *, std::set<TypeMemberInfo>> &TypeIdMap);
-  Constant *getPointerAtOffset(Constant *I, uint64_t Offset);
   bool
   tryFindVirtualCallTargets(std::vector<VirtualCallTarget> &TargetsForSlot,
                             const std::set<TypeMemberInfo> &TypeMemberInfos,
                             uint64_t ByteOffset);
 
   void applySingleImplDevirt(VTableSlotInfo &SlotInfo, Constant *TheFn,
                              bool &IsExported);
   bool trySingleImplDevirt(MutableArrayRef<VirtualCallTarget> TargetsForSlot,
for (MDNode *Type : Types) {
               cast<ConstantAsMetadata>(Type->getOperand(0))->getValue())
               ->getZExtValue();
 
       TypeIdMap[TypeID].insert({BitsPtr, Offset});
     }
   }
 }
 
-Constant *DevirtModule::getPointerAtOffset(Constant *I, uint64_t Offset) {
-  if (I->getType()->isPointerTy()) {
-    if (Offset == 0)
-      return I;
-    return nullptr;
-  }
-
-  const DataLayout &DL = M.getDataLayout();
-
-  if (auto *C = dyn_cast<ConstantStruct>(I)) {
-    const StructLayout *SL = DL.getStructLayout(C->getType());
-    if (Offset >= SL->getSizeInBytes())
-      return nullptr;
-
-    unsigned Op = SL->getElementContainingOffset(Offset);
-    return getPointerAtOffset(cast<Constant>(I->getOperand(Op)),
-                              Offset - SL->getElementOffset(Op));
-  }
-  if (auto *C = dyn_cast<ConstantArray>(I)) {
-    ArrayType *VTableTy = C->getType();
-    uint64_t ElemSize = DL.getTypeAllocSize(VTableTy->getElementType());
-
-    unsigned Op = Offset / ElemSize;
-    if (Op >= C->getNumOperands())
-      return nullptr;
-
-    return getPointerAtOffset(cast<Constant>(I->getOperand(Op)),
-                              Offset % ElemSize);
-  }
-  return nullptr;
-}
-
 bool DevirtModule::tryFindVirtualCallTargets(
     std::vector<VirtualCallTarget> &TargetsForSlot,
     const std::set<TypeMemberInfo> &TypeMemberInfos, uint64_t ByteOffset) {
   for (const TypeMemberInfo &TM : TypeMemberInfos) {
     if (!TM.Bits->GV->isConstant())
       return false;
 
     Constant *Ptr = getPointerAtOffset(TM.Bits->GV->getInitializer(),
-                                       TM.Offset + ByteOffset);
+                                       TM.Offset + ByteOffset, M);
     if (!Ptr)
       return false;
 
     auto Fn = dyn_cast<Function>(Ptr->stripPointerCasts());
     if (!Fn)
       return false;
 
     // We can disregard __cxa_pure_virtual as a possible call target, as
bool DevirtModule::run() {
   removeRedundantTypeTests();
 
   // Rebuild each global we touched as part of virtual constant propagation to
   // include the before and after bytes.
   if (DidVirtualConstProp)
     for (VTableBits &B : Bits)
       rebuildGlobal(B);
 
+  // We have lowered or deleted the type checked load intrinsics, so we no
+  // longer have enough information to reason about the liveness of virtual
+  // function pointers in GlobalDCE.
+  for (GlobalVariable &GV : M.globals())
+    GV.eraseMetadata(LLVMContext::MD_vcall_visibility);
+
   return true;
 }
 
 void DevirtIndex::run() {
   if (ExportSummary.typeIdCompatibleVtableMap().empty())
     return;
 
   DenseMap<GlobalValue::GUID, std::vector<StringRef>> NameByGUID;

llvm/test/LTO/ARM/lto-linking-metadata.ll

+; RUN: opt %s -o %t1.bc
+
+; RUN: llvm-lto %t1.bc -o %t1.save.opt -save-merged-module -O1 --exported-symbol=foo
+; RUN: llvm-dis < %t1.save.opt.merged.bc | FileCheck %s
+
+; RUN: llvm-lto2 run %t1.bc -o %t.out.o -save-temps \
+; RUN:     -r=%t1.bc,foo,pxl
+; RUN: llvm-dis < %t.out.o.0.2.internalize.bc | FileCheck  %s
+
+target datalayout = "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64"
+target triple = "armv7a-unknown-linux"
+
+define void @foo() {
+entry:
+  ret void
+}
+
+; CHECK: !llvm.module.flags = !{[[MD_NUM:![0-9]+]]}
+; CHECK: [[MD_NUM]] = !{i32 1, !"LTOPostLink", i32 1}

llvm/test/ThinLTO/X86/lazyload_metadata.ll

 ; Do setup work for all below tests: generate bitcode and combined index
 ; RUN: opt -module-summary %s -o %t.bc -bitcode-mdindex-threshold=0
 ; RUN: opt -module-summary %p/Inputs/lazyload_metadata.ll -o %t2.bc -bitcode-mdindex-threshold=0
 ; RUN: llvm-lto -thinlto-action=thinlink -o %t3.bc %t.bc %t2.bc
 ; REQUIRES: asserts
 
 ; Check that importing @globalfunc1 does not trigger loading all the global
 ; metadata for @globalfunc2 and @globalfunc3
 
 ; RUN: llvm-lto -thinlto-action=import %t2.bc -thinlto-index=%t3.bc \
 ; RUN:          -o /dev/null -stats \
 ; RUN:  2>&1 | FileCheck %s -check-prefix=LAZY
-; LAZY: 63 bitcode-reader  - Number of Metadata records loaded
+; LAZY: 65 bitcode-reader  - Number of Metadata records loaded
 ; LAZY: 2 bitcode-reader  - Number of MDStrings loaded
 
 ; RUN: llvm-lto -thinlto-action=import %t2.bc -thinlto-index=%t3.bc \
 ; RUN:          -o /dev/null -disable-ondemand-mds-loading -stats \
 ; RUN:  2>&1 | FileCheck %s -check-prefix=NOTLAZY
-; NOTLAZY: 72 bitcode-reader  - Number of Metadata records loaded
+; NOTLAZY: 74 bitcode-reader  - Number of Metadata records loaded
 ; NOTLAZY: 7 bitcode-reader  - Number of MDStrings loaded
 
 
 target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-apple-macosx10.11.0"
 
 define void @globalfunc1(i32 %arg) {
   %x = call i1 @llvm.type.test(i8* undef, metadata !"typeid1")

llvm/test/Transforms/GlobalDCE/virtual-functions-base-call.ll

+; RUN: opt < %s -globaldce -S | FileCheck %s
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+
+; struct A {
+;   A();
+;   virtual int foo();
+; };
+; 
+; struct B : A {
+;   B();
+;   virtual int foo();
+; };
+; 
+; A::A() {}
+; B::B() {}
+; int A::foo() { return 42; }
+; int B::foo() { return 1337; }
+; 
+; extern "C" int test(A *p) { return p->foo(); }
+
+; The virtual call in test could be dispatched to either A::foo or B::foo, so
+; both must be retained.
+
+%struct.A = type { i32 (...)** }
+%struct.B = type { %struct.A }
+
+; CHECK: @_ZTV1A = internal unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.A*)* @_ZN1A3fooEv to i8*)] }
+@_ZTV1A = internal unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.A*)* @_ZN1A3fooEv to i8*)] }, align 8, !type !0, !type !1, !vcall_visibility !2
+
+; CHECK: @_ZTV1B = internal unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.B*)* @_ZN1B3fooEv to i8*)] }
+@_ZTV1B = internal unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.B*)* @_ZN1B3fooEv to i8*)] }, align 8, !type !0, !type !1, !type !3, !type !4, !vcall_visibility !2
+
+; CHECK: define internal i32 @_ZN1A3fooEv(
+define internal i32 @_ZN1A3fooEv(%struct.A* nocapture readnone %this) {
+entry:
+  ret i32 42
+}
+
+; CHECK: define internal i32 @_ZN1B3fooEv(
+define internal i32 @_ZN1B3fooEv(%struct.B* nocapture readnone %this) {
+entry:
+  ret i32 1337
+}
+
+define hidden void @_ZN1AC2Ev(%struct.A* nocapture %this) {
+entry:
+  %0 = getelementptr inbounds %struct.A, %struct.A* %this, i64 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTV1A, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+define hidden void @_ZN1BC2Ev(%struct.B* nocapture %this) {
+entry:
+  %0 = getelementptr inbounds %struct.B, %struct.B* %this, i64 0, i32 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTV1B, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+define hidden i32 @test(%struct.A* %p) {
+entry:
+  %0 = bitcast %struct.A* %p to i8**
+  %vtable1 = load i8*, i8** %0, align 8
+  %1 = tail call { i8*, i1 } @llvm.type.checked.load(i8* %vtable1, i32 0, metadata !"_ZTS1A"), !nosanitize !10
+  %2 = extractvalue { i8*, i1 } %1, 0, !nosanitize !10
+  %3 = bitcast i8* %2 to i32 (%struct.A*)*, !nosanitize !10
+  %call = tail call i32 %3(%struct.A* %p)
+  ret i32 %call
+}
+
+declare { i8*, i1 } @llvm.type.checked.load(i8*, i32, metadata) #2
+
+!0 = !{i64 16, !"_ZTS1A"}
+!1 = !{i64 16, !"_ZTSM1AFivE.virtual"}
+!2 = !{i64 2}
+!3 = !{i64 16, !"_ZTS1B"}
+!4 = !{i64 16, !"_ZTSM1BFivE.virtual"}
+!10 = !{}

llvm/test/Transforms/GlobalDCE/virtual-functions-base-pointer-call.ll

+; RUN: opt < %s -globaldce -S | FileCheck %s
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+
+; struct A {
+;   A();
+;   virtual int foo(int);
+;   virtual int bar(float);
+; };
+; 
+; struct B : A {
+;   B();
+;   virtual int foo(int);
+;   virtual int bar(float);
+; };
+; 
+; A::A() {}
+; B::B() {}
+; int A::foo(int)   { return 1; }
+; int A::bar(float) { return 2; }
+; int B::foo(int)   { return 3; }
+; int B::bar(float) { return 4; }
+; 
+; extern "C" int test(A *p, int (A::*q)(int)) { return (p->*q)(42); }
+
+; Member function pointers are tracked by the combination of their object type
+; and function type, which must both be compatible. Here, the call is through a
+; pointer of type "int (A::*q)(int)", so the call could be dispatched to A::foo
+; or B::foo. It can't be dispatched to A::bar or B::bar as the function pointer
+; does not match, so those can be removed.
+
+%struct.A = type { i32 (...)** }
+%struct.B = type { %struct.A }
+
+; CHECK: @_ZTV1A = internal unnamed_addr constant { [4 x i8*] } { [4 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.A*, i32)* @_ZN1A3fooEi to i8*), i8* null] }
+@_ZTV1A = internal unnamed_addr constant { [4 x i8*] } { [4 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.A*, i32)* @_ZN1A3fooEi to i8*), i8* bitcast (i32 (%struct.A*, float)* @_ZN1A3barEf to i8*)] }, align 8, !type !0, !type !1, !type !2, !vcall_visibility !3
+; CHECK: @_ZTV1B = internal unnamed_addr constant { [4 x i8*] } { [4 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.B*, i32)* @_ZN1B3fooEi to i8*), i8* null] }
+@_ZTV1B = internal unnamed_addr constant { [4 x i8*] } { [4 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.B*, i32)* @_ZN1B3fooEi to i8*), i8* bitcast (i32 (%struct.B*, float)* @_ZN1B3barEf to i8*)] }, align 8, !type !0, !type !1, !type !2, !type !4, !type !5, !type !6, !vcall_visibility !3
+
+
+; CHECK: define internal i32 @_ZN1A3fooEi(
+define internal i32 @_ZN1A3fooEi(%struct.A* nocapture readnone %this, i32) unnamed_addr #1 align 2 {
+entry:
+  ret i32 1
+}
+
+; CHECK-NOT: define internal i32 @_ZN1A3barEf(
+define internal i32 @_ZN1A3barEf(%struct.A* nocapture readnone %this, float) unnamed_addr #1 align 2 {
+entry:
+  ret i32 2
+}
+
+; CHECK: define internal i32 @_ZN1B3fooEi(
+define internal i32 @_ZN1B3fooEi(%struct.B* nocapture readnone %this, i32) unnamed_addr #1 align 2 {
+entry:
+  ret i32 3
+}
+
+; CHECK-NOT: define internal i32 @_ZN1B3barEf(
+define internal i32 @_ZN1B3barEf(%struct.B* nocapture readnone %this, float) unnamed_addr #1 align 2 {
+entry:
+  ret i32 4
+}
+
+
+define hidden void @_ZN1AC2Ev(%struct.A* nocapture %this) {
+entry:
+  %0 = getelementptr inbounds %struct.A, %struct.A* %this, i64 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [4 x i8*] }, { [4 x i8*] }* @_ZTV1A, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+define hidden void @_ZN1BC2Ev(%struct.B* nocapture %this) {
+entry:
+  %0 = getelementptr inbounds %struct.B, %struct.B* %this, i64 0, i32 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [4 x i8*] }, { [4 x i8*] }* @_ZTV1B, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+define hidden i32 @test(%struct.A* %p, i64 %q.coerce0, i64 %q.coerce1) {
+entry:
+  %0 = bitcast %struct.A* %p to i8*
+  %1 = getelementptr inbounds i8, i8* %0, i64 %q.coerce1
+  %this.adjusted = bitcast i8* %1 to %struct.A*
+  %2 = and i64 %q.coerce0, 1
+  %memptr.isvirtual = icmp eq i64 %2, 0
+  br i1 %memptr.isvirtual, label %memptr.nonvirtual, label %memptr.virtual
+
+memptr.virtual:                                   ; preds = %entry
+  %3 = bitcast i8* %1 to i8**
+  %vtable = load i8*, i8** %3, align 8
+  %4 = add i64 %q.coerce0, -1
+  %5 = getelementptr i8, i8* %vtable, i64 %4, !nosanitize !12
+  %6 = tail call { i8*, i1 } @llvm.type.checked.load(i8* %5, i32 0, metadata !"_ZTSM1AFiiE.virtual"), !nosanitize !12
+  %7 = extractvalue { i8*, i1 } %6, 0, !nosanitize !12
+  %memptr.virtualfn = bitcast i8* %7 to i32 (%struct.A*, i32)*, !nosanitize !12
+  br label %memptr.end
+
+memptr.nonvirtual:                                ; preds = %entry
+  %memptr.nonvirtualfn = inttoptr i64 %q.coerce0 to i32 (%struct.A*, i32)*
+  br label %memptr.end
+
+memptr.end:                                       ; preds = %memptr.nonvirtual, %memptr.virtual
+  %8 = phi i32 (%struct.A*, i32)* [ %memptr.virtualfn, %memptr.virtual ], [ %memptr.nonvirtualfn, %memptr.nonvirtual ]
+  %call = tail call i32 %8(%struct.A* %this.adjusted, i32 42)
+  ret i32 %call
+}
+
+declare { i8*, i1 } @llvm.type.checked.load(i8*, i32, metadata)
+
+!0 = !{i64 16, !"_ZTS1A"}
+!1 = !{i64 16, !"_ZTSM1AFiiE.virtual"}
+!2 = !{i64 24, !"_ZTSM1AFifE.virtual"}
+!3 = !{i64 2}
+!4 = !{i64 16, !"_ZTS1B"}
+!5 = !{i64 16, !"_ZTSM1BFiiE.virtual"}
+!6 = !{i64 24, !"_ZTSM1BFifE.virtual"}
+!12 = !{}

llvm/test/Transforms/GlobalDCE/virtual-functions-derived-call.ll

+; RUN: opt < %s -globaldce -S | FileCheck %s
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+
+; struct A {
+;   A();
+;   virtual int foo();
+; };
+; 
+; struct B : A {
+;   B();
+;   virtual int foo();
+; };
+; 
+; A::A() {}
+; B::B() {}
+; int A::foo() { return 42; }
+; int B::foo() { return 1337; }
+; 
+; extern "C" int test(B *p) { return p->foo(); }
+
+; The virtual call in test can only be dispatched to B::foo (or a more-derived
+; class, if there was one), so A::foo can be removed.
+
+%struct.A = type { i32 (...)** }
+%struct.B = type { %struct.A }
+
+; CHECK: @_ZTV1A = internal unnamed_addr constant { [3 x i8*] } zeroinitializer
+@_ZTV1A = internal unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.A*)* @_ZN1A3fooEv to i8*)] }, align 8, !type !0, !type !1, !vcall_visibility !2
+
+; CHECK: @_ZTV1B = internal unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.B*)* @_ZN1B3fooEv to i8*)] }
+@_ZTV1B = internal unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.B*)* @_ZN1B3fooEv to i8*)] }, align 8, !type !0, !type !1, !type !3, !type !4, !vcall_visibility !2
+
+; CHECK-NOT: define internal i32 @_ZN1A3fooEv(
+define internal i32 @_ZN1A3fooEv(%struct.A* nocapture readnone %this) {
+entry:
+  ret i32 42
+}
+
+; CHECK: define internal i32 @_ZN1B3fooEv(
+define internal i32 @_ZN1B3fooEv(%struct.B* nocapture readnone %this) {
+entry:
+  ret i32 1337
+}
+
+define hidden void @_ZN1AC2Ev(%struct.A* nocapture %this) {
+entry:
+  %0 = getelementptr inbounds %struct.A, %struct.A* %this, i64 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTV1A, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+define hidden void @_ZN1BC2Ev(%struct.B* nocapture %this) {
+entry:
+  %0 = getelementptr inbounds %struct.B, %struct.B* %this, i64 0, i32 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTV1B, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+define hidden i32 @test(%struct.B* %p) {
+entry:
+  %0 = bitcast %struct.B* %p to i8**
+  %vtable1 = load i8*, i8** %0, align 8
+  %1 = tail call { i8*, i1 } @llvm.type.checked.load(i8* %vtable1, i32 0, metadata !"_ZTS1B"), !nosanitize !10
+  %2 = extractvalue { i8*, i1 } %1, 0, !nosanitize !10
+  %3 = bitcast i8* %2 to i32 (%struct.B*)*, !nosanitize !10
+  %call = tail call i32 %3(%struct.B* %p)
+  ret i32 %call
+}
+
+declare { i8*, i1 } @llvm.type.checked.load(i8*, i32, metadata) #2
+
+!0 = !{i64 16, !"_ZTS1A"}
+!1 = !{i64 16, !"_ZTSM1AFivE.virtual"}
+!2 = !{i64 2}
+!3 = !{i64 16, !"_ZTS1B"}
+!4 = !{i64 16, !"_ZTSM1BFivE.virtual"}
+!10 = !{}

llvm/test/Transforms/GlobalDCE/virtual-functions-derived-pointer-call.ll

+
+; RUN: opt < %s -globaldce -S | FileCheck %s
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+
+; struct A {
+;   A();
+;   virtual int foo(int);
+;   virtual int bar(float);
+; };
+; 
+; struct B : A {
+;   B();
+;   virtual int foo(int);
+;   virtual int bar(float);
+; };
+; 
+; A::A() {}
+; B::B() {}
+; int A::foo(int)   { return 1; }
+; int A::bar(float) { return 2; }
+; int B::foo(int)   { return 3; }
+; int B::bar(float) { return 4; }
+; 
+; extern "C" int test(B *p, int (B::*q)(int)) { return (p->*q)(42); }
+
+; Member function pointers are tracked by the combination of their object type
+; and function type, which must both be compatible. Here, the call is through a
+; pointer of type "int (B::*q)(int)", so the call could only be dispatched to
+; B::foo. It can't be dispatched to A::bar or B::bar as the function pointer
+; does not match, and it can't be dispatched to A::foo as the object type
+; doesn't match, so those can be removed.
+
+%struct.A = type { i32 (...)** }
+%struct.B = type { %struct.A }
+
+; CHECK: @_ZTV1A = internal unnamed_addr constant { [4 x i8*] } zeroinitializer
+@_ZTV1A = internal unnamed_addr constant { [4 x i8*] } { [4 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.A*, i32)* @_ZN1A3fooEi to i8*), i8* bitcast (i32 (%struct.A*, float)* @_ZN1A3barEf to i8*)] }, align 8, !type !0, !type !1, !type !2, !vcall_visibility !3
+; CHECK: @_ZTV1B = internal unnamed_addr constant { [4 x i8*] } { [4 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.B*, i32)* @_ZN1B3fooEi to i8*), i8* null] }
+@_ZTV1B = internal unnamed_addr constant { [4 x i8*] } { [4 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.B*, i32)* @_ZN1B3fooEi to i8*), i8* bitcast (i32 (%struct.B*, float)* @_ZN1B3barEf to i8*)] }, align 8, !type !0, !type !1, !type !2, !type !4, !type !5, !type !6, !vcall_visibility !3
+
+
+; CHECK-NOT: define internal i32 @_ZN1A3fooEi(
+define internal i32 @_ZN1A3fooEi(%struct.A* nocapture readnone %this, i32) unnamed_addr #1 align 2 {
+entry:
+  ret i32 1
+}
+
+; CHECK-NOT: define internal i32 @_ZN1A3barEf(
+define internal i32 @_ZN1A3barEf(%struct.A* nocapture readnone %this, float) unnamed_addr #1 align 2 {
+entry:
+  ret i32 2
+}
+
+; CHECK: define internal i32 @_ZN1B3fooEi(
+define internal i32 @_ZN1B3fooEi(%struct.B* nocapture readnone %this, i32) unnamed_addr #1 align 2 {
+entry:
+  ret i32 3
+}
+
+; CHECK-NOT: define internal i32 @_ZN1B3barEf(
+define internal i32 @_ZN1B3barEf(%struct.B* nocapture readnone %this, float) unnamed_addr #1 align 2 {
+entry:
+  ret i32 4
+}
+
+
+define hidden void @_ZN1AC2Ev(%struct.A* nocapture %this) {
+entry:
+  %0 = getelementptr inbounds %struct.A, %struct.A* %this, i64 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [4 x i8*] }, { [4 x i8*] }* @_ZTV1A, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+define hidden void @_ZN1BC2Ev(%struct.B* nocapture %this) {
+entry:
+  %0 = getelementptr inbounds %struct.B, %struct.B* %this, i64 0, i32 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [4 x i8*] }, { [4 x i8*] }* @_ZTV1B, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+define hidden i32 @test(%struct.B* %p, i64 %q.coerce0, i64 %q.coerce1) {
+entry:
+  %0 = bitcast %struct.B* %p to i8*
+  %1 = getelementptr inbounds i8, i8* %0, i64 %q.coerce1
+  %this.adjusted = bitcast i8* %1 to %struct.B*
+  %2 = and i64 %q.coerce0, 1
+  %memptr.isvirtual = icmp eq i64 %2, 0
+  br i1 %memptr.isvirtual, label %memptr.nonvirtual, label %memptr.virtual
+
+memptr.virtual:                                   ; preds = %entry
+  %3 = bitcast i8* %1 to i8**
+  %vtable = load i8*, i8** %3, align 8
+  %4 = add i64 %q.coerce0, -1
+  %5 = getelementptr i8, i8* %vtable, i64 %4, !nosanitize !12
+  %6 = tail call { i8*, i1 } @llvm.type.checked.load(i8* %5, i32 0, metadata !"_ZTSM1BFiiE.virtual"), !nosanitize !12
+  %7 = extractvalue { i8*, i1 } %6, 0, !nosanitize !12
+  %memptr.virtualfn = bitcast i8* %7 to i32 (%struct.B*, i32)*, !nosanitize !12
+  br label %memptr.end
+
+memptr.nonvirtual:                                ; preds = %entry
+  %memptr.nonvirtualfn = inttoptr i64 %q.coerce0 to i32 (%struct.B*, i32)*
+  br label %memptr.end
+
+memptr.end:                                       ; preds = %memptr.nonvirtual, %memptr.virtual
+  %8 = phi i32 (%struct.B*, i32)* [ %memptr.virtualfn, %memptr.virtual ], [ %memptr.nonvirtualfn, %memptr.nonvirtual ]
+  %call = tail call i32 %8(%struct.B* %this.adjusted, i32 42)
+  ret i32 %call
+}
+
+declare { i8*, i1 } @llvm.type.checked.load(i8*, i32, metadata)
+
+!0 = !{i64 16, !"_ZTS1A"}
+!1 = !{i64 16, !"_ZTSM1AFiiE.virtual"}
+!2 = !{i64 24, !"_ZTSM1AFifE.virtual"}
+!3 = !{i64 2}
+!4 = !{i64 16, !"_ZTS1B"}
+!5 = !{i64 16, !"_ZTSM1BFiiE.virtual"}
+!6 = !{i64 24, !"_ZTSM1BFifE.virtual"}
+!12 = !{}

llvm/test/Transforms/GlobalDCE/virtual-functions-visibility-post-lto.ll

+; RUN: opt < %s -globaldce -S | FileCheck %s
+
+; structs A, B and C have vcall_visibility of public, linkage-unit and
+; translation-unit respectively. This test is run after LTO linking (the
+; LTOPostLink metadata is present), so B and C can be VFE'd.
+
+target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
+
+%struct.A = type { i32 (...)** }
+
+@_ZTV1A = hidden unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (void (%struct.A*)* @_ZN1A3fooEv to i8*)] }, align 8, !type !0, !type !1, !vcall_visibility !2
+
+define internal void @_ZN1AC2Ev(%struct.A* %this) {
+entry:
+  %0 = getelementptr inbounds %struct.A, %struct.A* %this, i64 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTV1A, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+; CHECK: define {{.*}} @_ZN1A3fooEv(
+define internal void @_ZN1A3fooEv(%struct.A* nocapture %this) {
+entry:
+  ret void
+}
+
+define dso_local i8* @_Z6make_Av() {
+entry:
+  %call = tail call i8* @_Znwm(i64 8)
+  %0 = bitcast i8* %call to %struct.A*
+  tail call void @_ZN1AC2Ev(%struct.A* %0)
+  ret i8* %call
+}
+
+
+%struct.B = type { i32 (...)** }
+
+@_ZTV1B = hidden unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (void (%struct.B*)* @_ZN1B3fooEv to i8*)] }, align 8, !type !0, !type !1, !vcall_visibility !3
+
+define internal void @_ZN1BC2Ev(%struct.B* %this) {
+entry:
+  %0 = getelementptr inbounds %struct.B, %struct.B* %this, i64 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTV1B, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+; CHECK-NOT: define {{.*}} @_ZN1B3fooEv(
+define internal void @_ZN1B3fooEv(%struct.B* nocapture %this) {
+entry:
+  ret void
+}
+
+define dso_local i8* @_Z6make_Bv() {
+entry:
+  %call = tail call i8* @_Znwm(i64 8)
+  %0 = bitcast i8* %call to %struct.B*
+  tail call void @_ZN1BC2Ev(%struct.B* %0)
+  ret i8* %call
+}
+
+
+%struct.C = type { i32 (...)** }
+
+@_ZTV1C = hidden unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (void (%struct.C*)* @_ZN1C3fooEv to i8*)] }, align 8, !type !0, !type !1, !vcall_visibility !4
+
+define internal void @_ZN1CC2Ev(%struct.C* %this) {
+entry:
+  %0 = getelementptr inbounds %struct.C, %struct.C* %this, i64 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTV1C, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+; CHECK-NOT: define {{.*}} @_ZN1C3fooEv(
+define internal void @_ZN1C3fooEv(%struct.C* nocapture %this) {
+entry:
+  ret void
+}
+
+define dso_local i8* @_Z6make_Cv() {
+entry:
+  %call = tail call i8* @_Znwm(i64 8)
+  %0 = bitcast i8* %call to %struct.C*
+  tail call void @_ZN1CC2Ev(%struct.C* %0)
+  ret i8* %call
+}
+
+declare dso_local noalias nonnull i8* @_Znwm(i64)
+
+!llvm.module.flags = !{!5}
+
+!0 = !{i64 16, !"_ZTS1A"}
+!1 = !{i64 16, !"_ZTSM1AFvvE.virtual"}
+!2 = !{i64 0} ; public vcall visibility
+!3 = !{i64 1} ; linkage-unit vcall visibility
+!4 = !{i64 2} ; translation-unit vcall visibility
+!5 = !{i32 1, !"LTOPostLink", i32 1}

llvm/test/Transforms/GlobalDCE/virtual-functions-visibility-pre-lto.ll

+; RUN: opt < %s -globaldce -S | FileCheck %s
+
+; structs A, B and C have vcall_visibility of public, linkage-unit and
+; translation-unit respectively. This test is run before LTO linking occurs
+; (the LTOPostLink metadata is not present), so only C can be VFE'd.
+
+target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
+
+%struct.A = type { i32 (...)** }
+
+@_ZTV1A = hidden unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (void (%struct.A*)* @_ZN1A3fooEv to i8*)] }, align 8, !type !0, !type !1, !vcall_visibility !2
+
+define internal void @_ZN1AC2Ev(%struct.A* %this) {
+entry:
+  %0 = getelementptr inbounds %struct.A, %struct.A* %this, i64 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTV1A, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+; CHECK: define {{.*}} @_ZN1A3fooEv(
+define internal void @_ZN1A3fooEv(%struct.A* nocapture %this) {
+entry:
+  ret void
+}
+
+define dso_local i8* @_Z6make_Av() {
+entry:
+  %call = tail call i8* @_Znwm(i64 8)
+  %0 = bitcast i8* %call to %struct.A*
+  tail call void @_ZN1AC2Ev(%struct.A* %0)
+  ret i8* %call
+}
+
+
+%struct.B = type { i32 (...)** }
+
+@_ZTV1B = hidden unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (void (%struct.B*)* @_ZN1B3fooEv to i8*)] }, align 8, !type !0, !type !1, !vcall_visibility !3
+
+define internal void @_ZN1BC2Ev(%struct.B* %this) {
+entry:
+  %0 = getelementptr inbounds %struct.B, %struct.B* %this, i64 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTV1B, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+; CHECK: define {{.*}} @_ZN1B3fooEv(
+define internal void @_ZN1B3fooEv(%struct.B* nocapture %this) {
+entry:
+  ret void
+}
+
+define dso_local i8* @_Z6make_Bv() {
+entry:
+  %call = tail call i8* @_Znwm(i64 8)
+  %0 = bitcast i8* %call to %struct.B*
+  tail call void @_ZN1BC2Ev(%struct.B* %0)
+  ret i8* %call
+}
+
+
+%struct.C = type { i32 (...)** }
+
+@_ZTV1C = hidden unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (void (%struct.C*)* @_ZN1C3fooEv to i8*)] }, align 8, !type !0, !type !1, !vcall_visibility !4
+
+define internal void @_ZN1CC2Ev(%struct.C* %this) {
+entry:
+  %0 = getelementptr inbounds %struct.C, %struct.C* %this, i64 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTV1C, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+; CHECK-NOT: define {{.*}} @_ZN1C3fooEv(
+define internal void @_ZN1C3fooEv(%struct.C* nocapture %this) {
+entry:
+  ret void
+}
+
+define dso_local i8* @_Z6make_Cv() {
+entry:
+  %call = tail call i8* @_Znwm(i64 8)
+  %0 = bitcast i8* %call to %struct.C*
+  tail call void @_ZN1CC2Ev(%struct.C* %0)
+  ret i8* %call
+}
+
+declare dso_local noalias nonnull i8* @_Znwm(i64)
+
+!llvm.module.flags = !{}
+
+!0 = !{i64 16, !"_ZTS1A"}
+!1 = !{i64 16, !"_ZTSM1AFvvE.virtual"}
+!2 = !{i64 0} ; public vcall visibility
+!3 = !{i64 1} ; linkage-unit vcall visibility
+!4 = !{i64 2} ; translation-unit vcall visibility

llvm/test/Transforms/GlobalDCE/virtual-functions.ll

+; RUN: opt < %s -globaldce -S | FileCheck %s
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+
+declare dso_local noalias nonnull i8* @_Znwm(i64)
+declare { i8*, i1 } @llvm.type.checked.load(i8*, i32, metadata)
+
+; %struct.A is a C++ struct with two virtual functions, A::foo and A::bar. The
+; !vcall_visibility metadata is set on the vtable, so we know that all virtual
+; calls through this vtable are visible and use the @llvm.type.checked.load
+; intrinsic. Function test_A makes a call to A::foo, but there is no call to
+; A::bar anywhere, so A::bar can be deleted, and its vtable slot replaced with
+; null.
+
+%struct.A = type { i32 (...)** }
+
+; The pointer to A::bar in the vtable can be removed, because it will never be
+; loaded. We replace it with null to keep the layout the same. Because it is at
+; the end of the vtable we could potentially shrink the vtable, but don't
+; currently do that.
+; CHECK: @_ZTV1A = internal unnamed_addr constant { [4 x i8*] } { [4 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.A*)* @_ZN1A3fooEv to i8*), i8* null] }
+@_ZTV1A = internal unnamed_addr constant { [4 x i8*] } { [4 x i8*] [i8* null, i8* null, i8* bitcast (i32 (%struct.A*)* @_ZN1A3fooEv to i8*), i8* bitcast (i32 (%struct.A*)* @_ZN1A3barEv to i8*)] }, align 8, !type !0, !type !1, !type !2, !vcall_visibility !3
+
+; A::foo is called, so must be retained.
+; CHECK: define internal i32 @_ZN1A3fooEv(
+define internal i32 @_ZN1A3fooEv(%struct.A* nocapture readnone %this) {
+entry:
+  ret i32 42
+}
+
+; A::bar is not used, so can be deleted.
+; CHECK-NOT: define internal i32 @_ZN1A3barEv(
+define internal i32 @_ZN1A3barEv(%struct.A* nocapture readnone %this) {
+entry:
+  ret i32 1337
+}
+
+define dso_local i32 @test_A() {
+entry:
+  %call = tail call i8* @_Znwm(i64 8)
+  %0 = bitcast i8* %call to %struct.A*
+  %1 = bitcast i8* %call to i32 (...)***
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [4 x i8*] }, { [4 x i8*] }* @_ZTV1A, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %1, align 8
+  %2 = tail call { i8*, i1 } @llvm.type.checked.load(i8* bitcast (i8** getelementptr inbounds ({ [4 x i8*] }, { [4 x i8*] }* @_ZTV1A, i64 0, inrange i32 0, i64 2) to i8*), i32 0, metadata !"_ZTS1A"), !nosanitize !9
+  %3 = extractvalue { i8*, i1 } %2, 0, !nosanitize !9
+  %4 = bitcast i8* %3 to i32 (%struct.A*)*, !nosanitize !9
+  %call1 = tail call i32 %4(%struct.A* nonnull %0)
+  ret i32 %call1
+}
+
+!0 = !{i64 16, !"_ZTS1A"}
+!1 = !{i64 16, !"_ZTSM1AFivE.virtual"}
+!2 = !{i64 24, !"_ZTSM1AFivE.virtual"}
+!3 = !{i64 2}
+!9 = !{}

llvm/test/Transforms/GlobalDCE/vtable-rtti.ll

+; RUN: opt < %s -globaldce -S | FileCheck %s
+
+; We currently only use llvm.type.checked.load for virtual function pointers,
+; not any other part of the vtable, so we can't remove the RTTI pointer even if
+; it's never going to be loaded from.
+
+target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
+
+%struct.A = type { i32 (...)** }
+
+; CHECK: @_ZTV1A = hidden unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* bitcast ({ i8*, i8* }* @_ZTI1A to i8*), i8* null] }, align 8, !type !0, !type !1, !vcall_visibility !2
+
+@_ZTV1A = hidden unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* bitcast ({ i8*, i8* }* @_ZTI1A to i8*), i8* bitcast (void (%struct.A*)* @_ZN1A3fooEv to i8*)] }, align 8, !type !0, !type !1, !vcall_visibility !2
+@_ZTS1A = hidden constant [3 x i8] c"1A\00", align 1
+@_ZTI1A = hidden constant { i8*, i8* } { i8* bitcast (i8** getelementptr inbounds (i8*, i8** @_ZTVN10__cxxabiv117__class_type_infoE, i64 2) to i8*), i8* getelementptr inbounds ([3 x i8], [3 x i8]* @_ZTS1A, i32 0, i32 0) }, align 8
+
+define internal void @_ZN1AC2Ev(%struct.A* %this) {
+entry:
+  %0 = getelementptr inbounds %struct.A, %struct.A* %this, i64 0, i32 0
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTV1A, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret void
+}
+
+; CHECK-NOT: define {{.*}} @_ZN1A3fooEv(
+define internal void @_ZN1A3fooEv(%struct.A* nocapture %this) {
+entry:
+  ret void
+}
+
+define dso_local i8* @_Z6make_Av() {
+entry:
+  %call = tail call i8* @_Znwm(i64 8)
+  %0 = bitcast i8* %call to %struct.A*
+  tail call void @_ZN1AC2Ev(%struct.A* %0)
+  ret i8* %call
+}
+
+
+declare dso_local noalias nonnull i8* @_Znwm(i64)
+@_ZTVN10__cxxabiv117__class_type_infoE = external dso_local global i8*
+
+!llvm.module.flags = !{!3}
+
+!0 = !{i64 16, !"_ZTS1A"}
+!1 = !{i64 16, !"_ZTSM1AFvvE.virtual"}
+!2 = !{i64 2} ; translation-unit vcall visibility
+!3 = !{i32 1, !"LTOPostLink", i32 1}

llvm/test/Transforms/Internalize/vcall-visibility.ll

+; RUN: opt < %s -internalize -S | FileCheck %s
+
+%struct.A = type { i32 (...)** }
+%struct.B = type { i32 (...)** }
+%struct.C = type { i32 (...)** }
+
+; Class A has default visibility, so has no !vcall_visibility metadata before
+; or after LTO.
+; CHECK-NOT: @_ZTV1A = {{.*}}!vcall_visibility
+@_ZTV1A = dso_local unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (void (%struct.A*)* @_ZN1A3fooEv to i8*)] }, align 8, !type !0, !type !1
+
+; Class B has hidden visibility but public LTO visibility, so has no
+; !vcall_visibility metadata before or after LTO.
+; CHECK-NOT: @_ZTV1B = {{.*}}!vcall_visibility
+@_ZTV1B = hidden unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (void (%struct.B*)* @_ZN1B3fooEv to i8*)] }, align 8, !type !2, !type !3
+
+; Class C has hidden visibility, so the !vcall_visibility metadata is set to 1
+; (linkage unit) before LTO, and 2 (translation unit) after LTO.
+; CHECK: @_ZTV1C ={{.*}}!vcall_visibility [[MD_TU_VIS:![0-9]+]]
+@_ZTV1C = hidden unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* null, i8* bitcast (void (%struct.C*)* @_ZN1C3fooEv to i8*)] }, align 8, !type !4, !type !5, !vcall_visibility !6
+
+; Class D has translation unit visibility before LTO, and this is not changed
+; by LTO.
+; CHECK: @_ZTVN12_GLOBAL__N_11DE = {{.*}}!vcall_visibility [[MD_TU_VIS:![0-9]+]]
+@_ZTVN12_GLOBAL__N_11DE = internal unnamed_addr constant { [3 x i8*] } zeroinitializer, align 8, !type !7, !type !9, !vcall_visibility !11
+
+define dso_local void @_ZN1A3fooEv(%struct.A* nocapture %this) {
+entry:
+  ret void
+}
+
+define hidden void @_ZN1B3fooEv(%struct.B* nocapture %this) {
+entry:
+  ret void
+}
+
+define hidden void @_ZN1C3fooEv(%struct.C* nocapture %this) {
+entry:
+  ret void
+}
+
+define hidden noalias nonnull i8* @_Z6make_dv() {
+entry:
+  %call = tail call i8* @_Znwm(i64 8) #3
+  %0 = bitcast i8* %call to i32 (...)***
+  store i32 (...)** bitcast (i8** getelementptr inbounds ({ [3 x i8*] }, { [3 x i8*] }* @_ZTVN12_GLOBAL__N_11DE, i64 0, inrange i32 0, i64 2) to i32 (...)**), i32 (...)*** %0, align 8
+  ret i8* %call
+}
+
+declare dso_local noalias nonnull i8* @_Znwm(i64)
+
+; CHECK: [[MD_TU_VIS]] = !{i64 2}
+!0 = !{i64 16, !"_ZTS1A"}
+!1 = !{i64 16, !"_ZTSM1AFvvE.virtual"}
+!2 = !{i64 16, !"_ZTS1B"}
+!3 = !{i64 16, !"_ZTSM1BFvvE.virtual"}
+!4 = !{i64 16, !"_ZTS1C"}
+!5 = !{i64 16, !"_ZTSM1CFvvE.virtual"}
+!6 = !{i64 1}
+!7 = !{i64 16, !8}
+!8 = distinct !{}
+!9 = !{i64 16, !10}
+!10 = distinct !{}
+!11 = !{i64 2}