This is an archive of the discontinued LLVM Phabricator instance.

[compiler-rt/profile] Reland mark __llvm_profile_raw_version as hidden
ClosedPublic

Authored by pirama on Oct 13 2021, 3:13 PM.

Details

Summary

Since libclang_rt.profile is added later in the command line, a
definition of __llvm_profile_raw_version is not included if it is
provided from an earlier object, e.g. from a shared dependency.

This causes an extra dependence edge where if libA.so depends on libB.so
and both are coverage-instrumented, libA.so uses libB.so's definition of
__llvm_profile_raw_version. This leads to a runtime link failure if the
libB.so available at runtime does not provide this symbol (but provides
the other dependent symbols). Such a scenario can occur in Android's
mainline modules.
E.g.:

ld -o libB.so libclang_rt.profile-x86_64.a
ld -o libA.so -l B libclang_rt.profile-x86_64.a

libB.so has a global definition of llvm_profile_raw_version. libA.so
uses libB.so's definition of
llvm_profile_raw_version. At runtime,
libB.so may not be coverage-instrumented (i.e. not export
__llvm_profile_raw_version) so runtime linking of libA.so will fail.

Marking this symbol as hidden forces each binary to use the definition
of __llvm_profile_raw_version from libclang_rt.profile. The visiblity
is unchanged for Apple platforms where its presence is checked by the
TAPI tool.

Diff Detail

Event Timeline

pirama created this revision.Oct 13 2021, 3:13 PM
pirama requested review of this revision.Oct 13 2021, 3:13 PM
Herald added a project: Restricted Project. · View Herald TranscriptOct 13 2021, 3:13 PM
Herald added a subscriber: Restricted Project. · View Herald Transcript
phosek accepted this revision.Oct 13 2021, 3:18 PM

LGTM

This revision is now accepted and ready to land.Oct 13 2021, 3:18 PM

This leads to a runtime link failure if the libB.so available at runtime does not provide this symbol (but provides the other dependent symbols). Such a scenario can occur in Android's mainline modules.

The symbol is defined at link time but not available at run time? That's a pilot error.

compiler-rt/lib/profile/InstrProfiling.h
312

References don't need a visibility attribute.

pirama updated this revision to Diff 379536.Oct 13 2021, 3:35 PM
pirama edited the summary of this revision. (Show Details)

Fix typos

In the summary, do you mean 'not excluded' instead of 'not included' ?

pirama updated this revision to Diff 379539.Oct 13 2021, 3:53 PM

Remove attribute from reference.

In the summary, do you mean 'not excluded' instead of 'not included' ?

I did mean "not included".

`ld -o libA.so -lB ...`

If libB.so has an externally visible __llvm_profile_raw_version, then a definition of that symbol is not included in libA.so.

pirama updated this revision to Diff 379799.Oct 14 2021, 12:12 PM

Apply clang-format linter suggestion.

This leads to a runtime link failure if the libB.so available at runtime does not provide this symbol (but provides the other dependent symbols). Such a scenario can occur in Android's mainline modules.

The symbol is defined at link time but not available at run time? That's a pilot error.

Normally I'd agree. However in this instance, this symbol is added by the coverage instrumentation. I think we should keep instrumentation as low-touch and avoid breaking workflows that otherwise work without coverage instrumentation.

MaskRay added a comment.EditedOct 15 2021, 10:42 AM

This leads to a runtime link failure if the libB.so available at runtime does not provide this symbol (but provides the other dependent symbols). Such a scenario can occur in Android's mainline modules.

I guess my confusion is from the first paragraph:

Since it is marked as weak, a definition of __llvm_profile_raw_version is not included if it is available at runtime from another object, e.g. from a shared dependency.

I fail to connect the cause and the effect. How is weak related in this situation?

The Android use case needs a clear description. If someone needs to untangle the history, this can make contributors confused.

pirama updated this revision to Diff 380066.Oct 15 2021, 11:08 AM
pirama edited the summary of this revision. (Show Details)

This is not related to weak attribute to __llvm_profile_raw_version

pirama updated this revision to Diff 380068.Oct 15 2021, 11:14 AM
pirama edited the summary of this revision. (Show Details)

Describe Android use case.

Thanks for the description update. It is clear now :)

Maybe something like Hide __llvm_profile_raw_version so as not to resolve reference from a dependent shared object can clarify the intention.

MaskRay accepted this revision.Oct 15 2021, 11:32 AM

Looks great!

This revision was automatically updated to reflect the committed changes.

This causes instrprof-darwin-exports.c test failure on MacOS:
ld: warning: cannot export hidden symbol ___llvm_profile_raw_version from /Users/zequanwu/llvm-project/build/release/lib/clang/14.0.0/lib/darwin/libclang_rt.profile_osx.a(InstrProfilingVersionVar.c.o)

This causes instrprof-darwin-exports.c test failure on MacOS:
ld: warning: cannot export hidden symbol ___llvm_profile_raw_version from /Users/zequanwu/llvm-project/build/release/lib/clang/14.0.0/lib/darwin/libclang_rt.profile_osx.a(InstrProfilingVersionVar.c.o)

I think we should stop exporting ___llvm_profile_raw_version from Darwin::addProfileRTLibs() in clang/lib/Driver/ToolChains/Darwin.cpp. @vsk Can you confirm?

This causes instrprof-darwin-exports.c test failure on MacOS:
ld: warning: cannot export hidden symbol ___llvm_profile_raw_version from /Users/zequanwu/llvm-project/build/release/lib/clang/14.0.0/lib/darwin/libclang_rt.profile_osx.a(InstrProfilingVersionVar.c.o)

I think we should stop exporting ___llvm_profile_raw_version from Darwin::addProfileRTLibs() in clang/lib/Driver/ToolChains/Darwin.cpp. @vsk Can you confirm?

Can you revert it for now to unblock the test failure on MacOS bots?

pirama reopened this revision.Oct 21 2021, 11:00 AM

This causes instrprof-darwin-exports.c test failure on MacOS:
ld: warning: cannot export hidden symbol ___llvm_profile_raw_version from /Users/zequanwu/llvm-project/build/release/lib/clang/14.0.0/lib/darwin/libclang_rt.profile_osx.a(InstrProfilingVersionVar.c.o)

I think we should stop exporting ___llvm_profile_raw_version from Darwin::addProfileRTLibs() in clang/lib/Driver/ToolChains/Darwin.cpp. @vsk Can you confirm?

Can you revert it for now to unblock the test failure on MacOS bots?

Done. Zequan, can you help test if removing https://github.com/llvm/llvm-project/blob/ab3d5d0533678ee93c93bd62db79c091741096f0/clang/lib/Driver/ToolChains/Darwin.cpp#L1292 fixes all failures? I'll submit this again once @vsk confirms if that is acceptable.

This revision is now accepted and ready to land.Oct 21 2021, 11:00 AM

This causes instrprof-darwin-exports.c test failure on MacOS:
ld: warning: cannot export hidden symbol ___llvm_profile_raw_version from /Users/zequanwu/llvm-project/build/release/lib/clang/14.0.0/lib/darwin/libclang_rt.profile_osx.a(InstrProfilingVersionVar.c.o)

I think we should stop exporting ___llvm_profile_raw_version from Darwin::addProfileRTLibs() in clang/lib/Driver/ToolChains/Darwin.cpp. @vsk Can you confirm?

Can you revert it for now to unblock the test failure on MacOS bots?

Done. Zequan, can you help test if removing https://github.com/llvm/llvm-project/blob/ab3d5d0533678ee93c93bd62db79c091741096f0/clang/lib/Driver/ToolChains/Darwin.cpp#L1292 fixes all failures? I'll submit this again once @vsk confirms if that is acceptable.

That doesn't help. I got different error:

Command Output (stdout):
--
2d1
< ___llvm_profile_raw_version

--

This causes instrprof-darwin-exports.c test failure on MacOS:
ld: warning: cannot export hidden symbol ___llvm_profile_raw_version from /Users/zequanwu/llvm-project/build/release/lib/clang/14.0.0/lib/darwin/libclang_rt.profile_osx.a(InstrProfilingVersionVar.c.o)

I think we should stop exporting ___llvm_profile_raw_version from Darwin::addProfileRTLibs() in clang/lib/Driver/ToolChains/Darwin.cpp. @vsk Can you confirm?

Can you revert it for now to unblock the test failure on MacOS bots?

Done. Zequan, can you help test if removing https://github.com/llvm/llvm-project/blob/ab3d5d0533678ee93c93bd62db79c091741096f0/clang/lib/Driver/ToolChains/Darwin.cpp#L1292 fixes all failures? I'll submit this again once @vsk confirms if that is acceptable.

That doesn't help. I got different error:

Command Output (stdout):
--
2d1
< ___llvm_profile_raw_version

--

Is this the same test? What is the 'RUN: ' line that's failing?

This causes instrprof-darwin-exports.c test failure on MacOS:
ld: warning: cannot export hidden symbol ___llvm_profile_raw_version from /Users/zequanwu/llvm-project/build/release/lib/clang/14.0.0/lib/darwin/libclang_rt.profile_osx.a(InstrProfilingVersionVar.c.o)

I think we should stop exporting ___llvm_profile_raw_version from Darwin::addProfileRTLibs() in clang/lib/Driver/ToolChains/Darwin.cpp. @vsk Can you confirm?

Can you revert it for now to unblock the test failure on MacOS bots?

Done. Zequan, can you help test if removing https://github.com/llvm/llvm-project/blob/ab3d5d0533678ee93c93bd62db79c091741096f0/clang/lib/Driver/ToolChains/Darwin.cpp#L1292 fixes all failures? I'll submit this again once @vsk confirms if that is acceptable.

That doesn't help. I got different error:

Command Output (stdout):
--
2d1
< ___llvm_profile_raw_version

--

Is this the same test? What is the 'RUN: ' line that's failing?

It's the same test failing at line 26 after removing.

MaskRay added a comment.EditedOct 21 2021, 12:52 PM

According to https://reviews.llvm.org/rGf5380185623be243ba0f1b18d4bd594ac5cc7163 Apple's API verification tool (tapi) checks the exported __llvm_profile_raw_version.
However, ld64 cannot export a private-extern (hidden) symbol.

I wonder whether Apple's tool should lift the restriction. If not, we may need to special case __APPLE__.

According to https://reviews.llvm.org/rGf5380185623be243ba0f1b18d4bd594ac5cc7163 Apple's API verification tool (tapi) checks the exported __llvm_profile_raw_version.
However, ld64 cannot export a private-extern (hidden) symbol.

I wonder whether Apple's tool should lift the restriction. If not, we may need to special case __APPLE__.

Special-casing __APPLE__ SGTM but I'll wait a few days for @vsk's response. Though disabling the export and fixing the test seems the right thing to do. Removing the addExportedSymbol(CmdArgs, "___llvm_profile_raw_version"); ought to have fixed it but there seems to be some nuance to this test that I'm missing.

gulfem added a subscriber: gulfem.Nov 9 2021, 8:45 AM
pirama updated this revision to Diff 387453.Nov 15 2021, 5:23 PM

Do not hide __llvm_profile_raw_version on Apple platforms.

pirama updated this revision to Diff 387725.Nov 16 2021, 11:35 AM
pirama retitled this revision from [compiler-rt/profile] Mark __llvm_profile_raw_version as hidden to [compiler-rt/profile] Reland mark __llvm_profile_raw_version as hidden.
pirama edited the summary of this revision. (Show Details)

arc diff --verbatim to update phabricator message

This revision was landed with ongoing or failed builds.Nov 16 2021, 11:46 AM
This revision was automatically updated to reflect the committed changes.