[Docs] Document -lto-whole-program-visibility

[tejohnson]

Summary

Documents interaction of linker option added in D71913 with LTO
visibility.

[pcc]

clang/docs/LTOVisibility.rst

40

Isn't it spelled whole-program-visibility?

45

I thought that the motivation was avoiding duplicate work in the case where you need varying LTO visibility? Otherwise you could just build with and without -fvisibility=hidden and it would be the same from an LTO visibility perspective.

[tejohnson]

clang/docs/LTOVisibility.rst

40

good catch, yes

45

Yeah I started to write that out but it seemed too verbose (essentially we can't share a single bitcode object anymore without this, because it isn't safe for all dependent links, which to me is included in "situations where it is not safe to specify -fvisibility=hidden"). Also, on D71913, @evgeny777 mentioned he had a case were the symbol visibility constraints didn't allow for -fvisibility=hidden, but where LTO visibility for vtables can be hidden. I could add something more verbose about my case if you prefer.

[tejohnson]

Fix gold plugin option

[tejohnson]

ping

[pcc]

clang/docs/LTOVisibility.rst

45

@evgeny777 mentioned he had a case were the symbol visibility constraints didn't allow for -fvisibility=hidden, but where LTO visibility for vtables can be hidden.

That case seems somewhat questionable to me. If the symbols are being exported, it is presumably for the purpose of allowing the symbols to be used outside of the defining DSO. But LTO visibility based optimizations could make any such use of the symbols unsafe. For example with WPD it's unsafe to derive outside of the defining DSO and with dead virtual function elimination it's unsafe to call virtual functions outside of the defining DSO.

That makes me think that -lto-whole-program-visibility should imply that associated symbols are forced to hidden to prevent such use, unless we can clearly define what it means to export such a symbol.

[evgeny777]

@pcc

That case seems somewhat questionable to me. If the symbols are being exported, it is presumably for the purpose of allowing the symbols to be used outside of the defining DSO. But LTO visibility based optimizations could make any such use of the symbols unsafe. For example with WPD it's unsafe to derive outside of the defining DSO and with dead virtual function elimination it's unsafe to call virtual functions outside of the defining DSO.

True, but still direct (cross DSO) calls and globals accesses are possible and do not require explicitly setting visibility everywhere. That was my point.

[tejohnson]

In D75655#1920036, @evgeny777 wrote:

@pcc

That case seems somewhat questionable to me. If the symbols are being exported, it is presumably for the purpose of allowing the symbols to be used outside of the defining DSO. But LTO visibility based optimizations could make any such use of the symbols unsafe. For example with WPD it's unsafe to derive outside of the defining DSO and with dead virtual function elimination it's unsafe to call virtual functions outside of the defining DSO.

True, but still direct (cross DSO) calls and globals accesses are possible and do not require explicitly setting visibility everywhere. That was my point.

Not sure if we have come to a resolution here. @evgeny777 it sounds like you want symbols that can be direct called not to be marked hidden while still allowing for hidden LTO visibility for virtual functions. Which my change will support, but it sounds like it could be dangerous if not used carefully (i.e. if you don't know for sure that virtual symbols are only called within their defining DSO). Is this a correct summary?

[tejohnson]

@pcc ping. I'd like to get something in for LLVM 11 if at all possible. How about for now I make the last sentence something like "This is useful in situations where it is not safe to specify`-fvisibility=hidden` at compile time, for example when bitcode objects will be consumed by different LTO links that don't all have whole program visibility." ?

[MaskRay]

clang/docs/LTOVisibility.rst

39

This is now

``--lto-whole-program-visibility``

(I am currently preparing some lld/ELF release notes for 11.0.0...)

[pcc]

I think that the part of the description beginning "This can be used when..." is somewhat misleading since you could pretty much say the same thing about specifying -fvisibility=hidden -fwhole-program-vtables at compile time.

I think it would be better to focus on the specific capability that --lto-visibility-hidden gives you over just compiling with -fvisibility=hidden. Let's keep @evgeny777 's use case out of this because I still have concerns about it being dangerous. Can you simply say:

"This flag can be used to defer specifying whether classes have hidden LTO visibility until link time."

Then, in order to try to forbid inappropriate use of the symbols, you could add:

"Due to an implementation limitation, symbols associated with classes with hidden LTO visibility may still be exported from the binary when using this flag. It is unsafe to refer to these symbols, and their visibility may be relaxed to hidden in a future compiler release."

[tejohnson]

In D75655#2237474, @pcc wrote:

I think that the part of the description beginning "This can be used when..." is somewhat misleading since you could pretty much say the same thing about specifying -fvisibility=hidden -fwhole-program-vtables at compile time.

I think it would be better to focus on the specific capability that --lto-visibility-hidden gives you over just compiling with -fvisibility=hidden. Let's keep @evgeny777 's use case out of this because I still have concerns about it being dangerous. Can you simply say:

"This flag can be used to defer specifying whether classes have hidden LTO visibility until link time."

Then, in order to try to forbid inappropriate use of the symbols, you could add:

"Due to an implementation limitation, symbols associated with classes with hidden LTO visibility may still be exported from the binary when using this flag. It is unsafe to refer to these symbols, and their visibility may be relaxed to hidden in a future compiler release."

Added these, but added in my proposed wording about why you would want to defer (to allow bitcode sharing).

clang/docs/LTOVisibility.rst

39

Fixed here and in summary

[tejohnson]

Address comments

[pcc]

LGTM

Thanks and sorry for the delay.

clang/docs/LTOVisibility.rst

 flag) or, on the Windows platform, the dllimport and dllexport attributes. When
 targeting non-Windows platforms, classes with a visibility other than hidden
 visibility receive public LTO visibility. When targeting Windows, classes
 with dllimport or dllexport attributes receive public LTO visibility. All
 other classes receive hidden LTO visibility. Classes with internal linkage
 (e.g. classes declared in unnamed namespaces) also receive hidden LTO
 visibility.
 
+During the LTO link, all classes with public LTO visibility will be refined
+to hidden LTO visibility when the ``--lto-whole-program-visibility`` lld linker
+option is applied (``-plugin-opt=whole-program-visibility`` for gold). This flag
+can be used to defer specifying whether classes have hidden LTO visibility until
+link time, to allow bitcode objects to be shared by different LTO links.
+Due to an implementation limitation, symbols associated with classes with hidden
+LTO visibility may still be exported from the binary when using this flag. It is
+unsafe to refer to these symbols, and their visibility may be relaxed to hidden
+in a future compiler release.
+
 A class defined in a translation unit built without LTO receives public
 LTO visibility regardless of its object file visibility, linkage or other
 attributes.
 
 This mechanism will produce the correct result in most cases, but there are
 two cases where it may wrongly infer hidden LTO visibility.
 
 1. As a corollary of the above rules, if a linkage unit is produced from a