[InstrProfiling] Use ELF section groups for counters, data and values

[phosek]

Summary

start_/stop_ references retain C identifier name sections such as
__llvm_prf_*. Putting these into a section group disables this logic.

The ELF section group semantics ensures that group members are retained
or discarded as a unit. When a function symbol is discarded, this allows
allows linker to discard counters, data and values associated with that
function symbol as well.

Note that noduplicates COMDAT is lowered to zero-flag section group in
ELF. We only set this for functions that aren't already in a COMDAT and
for those that don't have available_externally linkage since we already
use regular COMDAT groups for those.

[phosek]

This is an alternative to D76802 which uses zero-flag ELF section groups instead of SHF_LINK_ORDER. While testing D76802, we keep running into issues in real-world code, but this variant seems to be working.

[davidxl]

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

832–833

maybe this:

bool UseSectGroup = (NeedComdat || TT.isOSBinFormatELF());

if (UseSectGroup) {

auto GroupName = ..
Comdat *C = M->getOrInsertComdat(GroupName);
 if (!NeedComdat)
    C->setSelectionKind(Comdat::NoDuplicates);
 GV->setComdat(C);

}

[davidxl]

Can this be handled by other linkers (gold, bfd)?

[phosek]

In D96757#2566973, @davidxl wrote:

Can this be handled by other linkers (gold, bfd)?

They handle this without problems (I tested it locally) which is another advantage over D76802. However, they aren't yet able to GC those sections so this change is currently a no-op for those linkers. They would need to make a change similar to D96753. There's currently an ongoing discussion about that.

[davidxl]

Looks good to me. Wait for maskray's comments.

[MaskRay]

Does this mean that for non-COMDAT __llvm_prf_* sections, we now need one section header (.group) for -ffunction-sections use cases? sizeof(Elf64_Shdr)=64 the size overhead can be quite significant.

This is an alternative to D76802 which uses zero-flag ELF section groups instead of SHF_LINK_ORDER. While testing D76802, we keep running into issues in real-world code, but this variant seems to be working.

Would be good to know the root cause. SHF_LINK_ORDER is the perfect alternative to COFF comdat nodeduplicates here as both solutions don't have space overhead.

[davidxl]

Good point about the section header. The objective of the patch is to reduce binary image size (loadable sections) though the object size may increase. It would be good to know the object size overhead because some of our large binaries may get hit during link time (of instr binary).

[phosek]

In D96757#2567122, @MaskRay wrote:

Would be good to know the root cause. SHF_LINK_ORDER is the perfect alternative to COFF comdat nodeduplicates here as both solutions don't have space overhead.

The current issue we're stuck on is in the build of zstd (as discussed privately):

ld.lld: error: host_arm64-profile/obj/third_party/zstd/libzstd.a(libzstd.zstd_decompress.c.o):(__llvm_prf_cnts): sh_link points to discarded section host_arm64-profile/obj/third_party/zstd/libzstd.a(libzstd.zstd_decompress.c.o):(.text.ZSTD_getDDict)

In D96757#2567128, @davidxl wrote:

Good point about the section header. The objective of the patch is to reduce binary image size (loadable sections) though the object size may increase. It would be good to know the object size overhead because some of our large binaries may get hit during link time (of instr binary).

I'll do a full build of Fuchsia with this change to measure the impact.

[phosek]

I tested this change in the Fuchsia build. The total size of all .o files without this change is 31665652 (30.2GB) and with this change it increases to 32287840 (30.8GB) so the increase is about 1.9%. In terms of impact on the final system image (by enabling --gc-sections on instrumented binaries), the size of boot image decreases from 116M to 63M so roughly 46% reduction, and the size of system image decreases from 5.8GB to 4.2GB which is roughly 28% reduction.

[davidxl]

Nice reduction. Is the total .o size quoted from instrumentation build (with --gc-sections)?

[phosek]

In D96757#2568675, @davidxl wrote:

Nice reduction. Is the total .o size quoted from instrumentation build (with --gc-sections)?

In both cases the .o files are built with -fprofile-instr-generate -ffunction-sections -fdata-sections and --gc-sections for linking.

[MaskRay]

I have checked that the .o size increase should not be a concern for my internal users. So, LG, but the description needs some clarification.

C identifier name input sections such as __llvm_prf_* are GC roots so they cannot be discarded.

Sorry, my previous wording is inaccurate. It should be: __start_/__stop_ references retain C identifier name sections.
This is different from being GC roots.

The ELF section group semantics ensures that group members are retained or discarded as a unit. When a function symbol is discarded, this allows allows linker to discard counters, data and values associated with that function symbol as well.

I don't find incoming or outgoing relocations with __llvm_prf_names, but it appears to be a parallel table to __llvm_prf_cnts/__llvm_prf_data.
The idea is that __llvm_prf_cnts can be properly handled with the traditional GC semantics (text sections keep it live).
We use a section group to let __llvm_prf_cnts retain the associated __llvm_prf_names (and probably also __llvm_prf_vals).
Hope @davidxl can elaborate on the usage of these sections to make this paragraph accurate.

[davidxl]

llvm_prf_names are cross function for the purpose of compression. This section is not loadable, so it is does not really matter that much.

[phosek]

@rnk based on your comment in https://groups.google.com/g/llvm-dev/c/4Bxvjt_rcCg/m/9TsRuI4FAgAJ, I think this should work for COFF as well, is that correct?

[phosek]

I plan on improving how __llvm_prf_names is constructed using the approach suggested in PR49155 in a follow up change.

[yroux]

Hi Petr,

This commit broke AArch64 LLD bot, logs are available here:

http://lab.llvm.org:8011/#/builders/53/builds/1366

Cheers,
Yvan

[llvm-commits]

Thanks for the heads up. I've landed
https://github.com/llvm/llvm-project/commit/97184ab99c46e35ae94f828ee90f5d6af2c47e11
which should have addressed that issue, but now the bot is failing with a
different issue that looks like a stale output. I'm still looking into it,
but if I cannot find a culprit quickly, I'll revert the change.

• smime.p7s3 KBDownload
• msg-20355-53.txt163 BDownload

[phosek]

Check the size of __llvm_prf_cnts section rather than its content.

[aeubanks]

instrprof-gc-sections.c is failing for me:

FAIL: Profile-x86_64 :: instrprof-gc-sections.c (1 of 1)
******************** TEST 'Profile-x86_64 :: instrprof-gc-sections.c' FAILED ********************
Script:
--
: 'RUN: at line 3';      /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/./bin/clang   -m64  -ldl  -fprofile-instr-generate=/usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.profraw -fuse-ld=lld -fcoverage-mapping -mllvm -enable-name-compression=false -DCODE=1 -ffunction-sections -fdata-sections -Wl,--gc-sections -o /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c
: 'RUN: at line 4';    /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp
: 'RUN: at line 5';   llvm-profdata merge -o /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.profdata /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.profraw
: 'RUN: at line 6';   llvm-profdata show --all-functions /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.profdata | FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=PROF
: 'RUN: at line 7';   llvm-cov show /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp -instr-profile /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.profdata | FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=COV
: 'RUN: at line 8';   llvm-nm /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp | FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=NM
: 'RUN: at line 9';   llvm-readelf -x __llvm_prf_names /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp | FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=PRF_NAMES
: 'RUN: at line 10';   llvm-size -A /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp | FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=PRF_CNTS
: 'RUN: at line 12';      /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/./bin/clang   -m64  -ldl  -fprofile-instr-generate=/usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.lto.profraw -fuse-ld=lld -fcoverage-mapping -mllvm -enable-name-compression=false -DCODE=1 -ffunction-sections -fdata-sections -Wl,--gc-sections -flto -o /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.lto /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c
: 'RUN: at line 13';    /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.lto
: 'RUN: at line 14';   llvm-profdata merge -o /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.lto.profdata /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.lto.profraw
: 'RUN: at line 15';   llvm-profdata show --all-functions /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.lto.profdata | FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=PROF
: 'RUN: at line 16';   llvm-cov show /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.lto -instr-profile /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.lto.profdata | FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=COV
: 'RUN: at line 17';   llvm-nm /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.lto | FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=NM
: 'RUN: at line 18';   llvm-readelf -x __llvm_prf_names /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.lto | FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=PRF_NAMES
: 'RUN: at line 19';   llvm-size -A /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.lto | FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=PRF_CNTS
: 'RUN: at line 30';      /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/./bin/clang   -m64  -ldl  -fprofile-generate=/usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.pgo.profraw -fuse-ld=lld -DCODE=1 -ffunction-sections -fdata-sections -Wl,--gc-sections -o /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.pgo /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c
: 'RUN: at line 31';    /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.pgo
: 'RUN: at line 32';   llvm-profdata merge -o /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.pgo.profdata /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.pgo.profraw
: 'RUN: at line 33';   llvm-profdata show --all-functions /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.pgo.profdata | FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=PGO
: 'RUN: at line 34';   llvm-nm /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.pgo | FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=NM
--
Exit Code: 1

Command Output (stderr):
--
+ : 'RUN: at line 3'
+ /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/./bin/clang -m64 -ldl -fprofile-instr-generate=/usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.profraw -fuse-ld=lld -fcoverage-mapping -mllvm -enable-name-compression=false -DCODE=1 -ffunction-sections -fdata-sections -Wl,--gc-sections -o /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c
+ : 'RUN: at line 4'
+ /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp
+ : 'RUN: at line 5'
+ llvm-profdata merge -o /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.profdata /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.profraw
+ : 'RUN: at line 6'
+ llvm-profdata show --all-functions /usr/local/google/home/aeubanks/repos/llvm-project2/build/cmake/projects/compiler-rt/test/profile/Profile-x86_64/Output/instrprof-gc-sections.c.tmp.profdata
+ FileCheck /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c -check-prefix=PROF
/usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c:53:15: error: PROF-NEXT: is not on the line after the previous match
// PROF-NEXT: Instrumentation level: Front-end
              ^
<stdin>:10:1: note: 'next' match was here
Instrumentation level: Front-end
^
<stdin>:5:19: note: previous match ended here
 Function count: 1
                  ^
<stdin>:6:1: note: non-matching line after previous match is here
 foo:
^

Input file: <stdin>
Check file: /usr/local/google/home/aeubanks/repos/llvm-project2/compiler-rt/test/profile/instrprof-gc-sections.c

-dump-input=help explains the following input dump.

Input was:
<<<<<<
         1: Counters:
         2:  main:
         3:  Hash: 0x0000000000000018
         4:  Counters: 1
         5:  Function count: 1
         6:  foo:
         7:  Hash: 0x0000000000000000
         8:  Counters: 1
         9:  Function count: 0
        10: Instrumentation level: Front-end
next:53     !~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: match on wrong line
        11: Functions shown: 2
        12: Total functions: 2
        13: Maximum function count: 1
        14: Maximum internal block count: 0
>>>>>>

--

********************
********************
Failed Tests (1):
  Profile-x86_64 :: instrprof-gc-sections.c


Testing Time: 0.12s
  Failed: 1

cmake invocation:

cmake -G Ninja -DLLVM_LOCAL_RPATH=/usr/local/google/home/aeubanks/repos/chromium/src/third_party/llvm-build-tools/gcc-10.2.0-trusty/lib64 '-DCOMPILER_RT_TEST_COMPILER_CFLAGS=--gcc-toolchain=/usr/local/google/home/aeubanks/repos/chromium/src/third_party/llvm-build-tools/gcc-10.2.0-trusty -Wl,-rpath,/usr/local/google/home/aeubanks/repos/chromium/src/third_party/llvm-build-tools/gcc-10.2.0-trusty/lib64 -Wl,-rpath,/usr/local/google/home/aeubanks/repos/chromium/src/third_party/llvm-build-tools/gcc-10.2.0-trusty/lib32' -DLLVM_ENABLE_LIBXML2=FORCE_ON -DLLVM_TARGETS_TO_BUILD=X86 '-DLLVM_ENABLE_PROJECTS=clang;lld;compiler-rt' -DCMAKE_C_FLAGS= -DCMAKE_CXX_FLAGS= -DCMAKE_EXE_LINKER_FLAGS= -DCMAKE_SHARED_LINKER_FLAGS= -DCMAKE_MODULE_LINKER_FLAGS= -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_C_COMPILER=/usr/local/google/home/aeubanks/repos/chromium/src/third_party/llvm-build-tools/gcc-10.2.0-trusty/bin/gcc -DCMAKE_CXX_COMPILER=/usr/local/google/home/aeubanks/repos/chromium/src/third_party/llvm-build-tools/gcc-10.2.0-trusty/bin/g++  ../../llvm

[aeubanks]

Looks like lld isn't getting built when I run ninja check-profile. Manually building it then running check-profile works.

[phosek]

In D96757#2580469, @aeubanks wrote:

instrprof-gc-sections.c is failing for me:

Do you have lld installed on your host? This looks like as if the test was using an older lld and not the one from your build.

[MaskRay]

In D96757#2580519, @phosek wrote:

In D96757#2580469, @aeubanks wrote:

instrprof-gc-sections.c is failing for me:

Do you have lld installed on your host? This looks like as if the test was using an older lld and not the one from your build.

lld-available may be a bit tricky. I can never tell when lld-available or lld should be used:( Perhaps disable the test temporarily (would be better for @aeubanks to do that as he can test the configuration)

[phosek]

In D96757#2580518, @aeubanks wrote:

Looks like lld isn't getting built when I run ninja check-profile. Manually building it then running check-profile works.

I think the problem is that depending on the order in which projects are processed, TARGET lld on https://github.com/llvm/llvm-project/blob/06e25d56451977ef5b7052282faacfe3d42acb65/compiler-rt/test/profile/CMakeLists.txt#L8 might return FALSE in case compiler-rt is processed before lld so this is not sufficient. What's not clear to me is how does this work for other runtimes like ASan which use the same pattern, see https://github.com/llvm/llvm-project/blob/main/compiler-rt/test/asan/CMakeLists.txt#L29 (or maybe it doesn't and nobody noticed).

I'll try to come up with a fix if you're not in a hurry, otherwise I can revert the change and reland it later.

[aeubanks]

I'm ok for now, I was trying to investigate a bunch of other failing check-profile tests on our bots when this separate one came up locally :)

[phosek]

In D96757#2580522, @MaskRay wrote:

In D96757#2580519, @phosek wrote:

In D96757#2580469, @aeubanks wrote:

instrprof-gc-sections.c is failing for me:

Do you have lld installed on your host? This looks like as if the test was using an older lld and not the one from your build.

lld-available may be a bit tricky. I can never tell when lld-available or lld should be used:( Perhaps disable the test temporarily (would be better for @aeubanks to do that as he can test the configuration)

lld-available is set based on COMPILER_RT_HAS_LLD (https://github.com/llvm/llvm-project/blob/06e25d56451977ef5b7052282faacfe3d42acb65/compiler-rt/CMakeLists.txt#L595) which may be TRUE but TARGET lld may still return FALSE depending on the order in which projects are processed.

compiler-rt/test/CMakeLists.txt

 
 # When ANDROID, we build tests with the host compiler (i.e. CMAKE_C_COMPILER),
 # and run tests with tools from the host toolchain.
 if(NOT ANDROID)
   if(NOT COMPILER_RT_STANDALONE_BUILD AND NOT RUNTIMES_BUILD)
     # Use LLVM utils and Clang from the same build tree.
     list(APPEND SANITIZER_COMMON_LIT_TEST_DEPS
       clang clang-resource-headers FileCheck count not llvm-config llvm-nm llvm-objdump
-      llvm-readobj llvm-symbolizer compiler-rt-headers sancov)
+      llvm-readelf llvm-readobj llvm-size llvm-symbolizer compiler-rt-headers sancov)
     if (WIN32)
       list(APPEND SANITIZER_COMMON_LIT_TEST_DEPS KillTheDoctor)
     endif()
   endif()
 endif()
 
 function(compiler_rt_test_runtime runtime)
   string(TOUPPER ${runtime} runtime_uppercase)

compiler-rt/test/profile/CMakeLists.txt

 set(PROFILE_LIT_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
 set(PROFILE_LIT_BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR})
 
 set(PROFILE_TESTSUITES)
 set(PROFILE_TEST_DEPS ${SANITIZER_COMMON_LIT_TEST_DEPS})
 if(NOT COMPILER_RT_STANDALONE_BUILD)
   list(APPEND PROFILE_TEST_DEPS profile llvm-profdata llvm-cov)
+  if(NOT APPLE AND COMPILER_RT_HAS_LLD AND TARGET lld)
+    list(APPEND PROFILE_TEST_DEPS lld)
+  endif()
 endif()
 
 set(PROFILE_TEST_ARCH ${PROFILE_SUPPORTED_ARCH})
 if(APPLE)
   darwin_filter_host_archs(PROFILE_SUPPORTED_ARCH PROFILE_TEST_ARCH)
 endif()
 
 foreach(arch ${PROFILE_TEST_ARCH})

compiler-rt/test/profile/instrprof-gc-sections.c

+// REQUIRES: linux, lld-available
+
+// RUN: %clang_profgen=%t.profraw -fuse-ld=lld -fcoverage-mapping -mllvm -enable-name-compression=false -DCODE=1 -ffunction-sections -fdata-sections -Wl,--gc-sections -o %t %s
+// RUN: %run %t
+// RUN: llvm-profdata merge -o %t.profdata %t.profraw
+// RUN: llvm-profdata show --all-functions %t.profdata | FileCheck %s -check-prefix=PROF
+// RUN: llvm-cov show %t -instr-profile %t.profdata | FileCheck %s -check-prefix=COV
+// RUN: llvm-nm %t | FileCheck %s -check-prefix=NM
+// RUN: llvm-readelf -x __llvm_prf_names %t | FileCheck %s -check-prefix=PRF_NAMES
+// RUN: llvm-size -A %t | FileCheck %s -check-prefix=PRF_CNTS
+
+// RUN: %clang_lto_profgen=%t.lto.profraw -fuse-ld=lld -fcoverage-mapping -mllvm -enable-name-compression=false -DCODE=1 -ffunction-sections -fdata-sections -Wl,--gc-sections -flto -o %t.lto %s
+// RUN: %run %t.lto
+// RUN: llvm-profdata merge -o %t.lto.profdata %t.lto.profraw
+// RUN: llvm-profdata show --all-functions %t.lto.profdata | FileCheck %s -check-prefix=PROF
+// RUN: llvm-cov show %t.lto -instr-profile %t.lto.profdata | FileCheck %s -check-prefix=COV
+// RUN: llvm-nm %t.lto | FileCheck %s -check-prefix=NM
+// RUN: llvm-readelf -x __llvm_prf_names %t.lto | FileCheck %s -check-prefix=PRF_NAMES
+// RUN: llvm-size -A %t.lto | FileCheck %s -check-prefix=PRF_CNTS
+
+// Note: We expect foo() and some of the profiling data associated with it to
+// be garbage collected.
+
+// Note: When there is no code in a program, we expect to see the exact same
+// set of external functions provided by the profile runtime.
+
+// Note: We also check the IR instrumentation and expect foo() to be garbage
+// collected as well.
+
+// RUN: %clang_pgogen=%t.pgo.profraw -fuse-ld=lld -DCODE=1 -ffunction-sections -fdata-sections -Wl,--gc-sections -o %t.pgo %s
+// RUN: %run %t.pgo
+// RUN: llvm-profdata merge -o %t.pgo.profdata %t.pgo.profraw
+// RUN: llvm-profdata show --all-functions %t.pgo.profdata | FileCheck %s -check-prefix=PGO
+// RUN: llvm-nm %t.pgo | FileCheck %s -check-prefix=NM
+
+#ifdef CODE
+
+// COV: [[@LINE+1]]{{ *}}|{{ *}}0|void foo()
+void foo() {}
+
+// COV: [[@LINE+1]]{{ *}}|{{ *}}1|int main
+int main() { return 0; }
+
+#endif // CODE
+
+// NM-NOT: foo
+
+// PROF: Counters:
+// PROF-NEXT:   main:
+// PROF-NEXT:     Hash:
+// PROF-NEXT:     Counters: 1
+// PROF-NEXT:     Function count: 1
+// PROF-NEXT: Instrumentation level: Front-end
+// PROF-NEXT: Functions shown: 1
+// PROF-NEXT: Total functions: 1
+// PROF-NEXT: Maximum function count:
+// PROF-NEXT: Maximum internal block count:
+
+// Note: We don't expect the names of garbage collected functions to disappear
+// from __llvm_prf_names, because collectPGOFuncNameStrings() glues the names
+// together.
+
+// PRF_NAMES: Hex dump of section '__llvm_prf_names':
+// PRF_NAMES-NEXT: {{.*}} 0800666f 6f016d61 696e{{.*$}}
+//                        | | f o  o # m a  i n
+//                        | |___________|
+//                        |             |
+//               UncompressedLen = 8    |
+//                                      |
+//                               CompressedLen = 0
+
+// Note: We expect the profile counters for garbage collected functions to also
+// be garbage collected.
+
+// PRF_CNTS: __llvm_prf_cnts 8
+
+// PGO: Counters:
+// PGO-NEXT:   main:
+// PGO-NEXT:     Hash:
+// PGO-NEXT:     Counters: 1
+// PGO-NEXT: Instrumentation level: IR
+// PGO-NEXT: Functions shown: 1
+// PGO-NEXT: Total functions: 1
+// PGO-NEXT: Maximum function count:
+// PGO-NEXT: Maximum internal block count:

llvm/lib/Transforms/Instrumentation/InstrProfiling.cpp

if (TT.isOSBinFormatCOFF()) {
       // comdats. We can't use a group because the Visual C++ linker will
       // report duplicate symbol errors if there are multiple external symbols
       // with the same name marked IMAGE_COMDAT_SELECT_ASSOCIATIVE.
       Linkage = GlobalValue::LinkOnceODRLinkage;
       Visibility = GlobalValue::HiddenVisibility;
     }
   }
   std::string DataVarName = getVarName(Inc, getInstrProfDataVarPrefix());
   auto MaybeSetComdat = [=](GlobalVariable *GV) {
-    if (NeedComdat)
-      GV->setComdat(M->getOrInsertComdat(TT.isOSBinFormatCOFF() ? GV->getName()
-                                                                : DataVarName));
+    // For ELF, when not using COMDAT, put counters, data and values into
+    // a noduplicates COMDAT which is lowered to a zero-flag section group.
+    // This allows linker GC to discard the entire group when the function
+    // is discarded.
+    bool UseComdat = (NeedComdat || TT.isOSBinFormatELF());
+    if (UseComdat) {
+      auto GroupName = TT.isOSBinFormatCOFF() ? GV->getName() : DataVarName;
+      Comdat *C = M->getOrInsertComdat(GroupName);
+      if (!NeedComdat)
+        C->setSelectionKind(Comdat::NoDuplicates);
+      GV->setComdat(C);
+    }
   };
 
   uint64_t NumCounters = Inc->getNumCounters()->getZExtValue();
   LLVMContext &Ctx = M->getContext();
   ArrayType *CounterTy = ArrayType::get(Type::getInt64Ty(Ctx), NumCounters);
 
   // Create the counters variable.
   auto *CounterPtr =

llvm/test/Instrumentation/InstrProfiling/icall.ll

 ; Function Attrs: nounwind
 declare void @llvm.instrprof.increment(i8*, i64, i32, i32) #0
 
 ; Function Attrs: nounwind
 declare void @llvm.instrprof.value.profile(i8*, i64, i64, i32, i32) #0
 
 attributes #0 = { nounwind }
 
-; STATIC: @__profvp_foo = private global [1 x i64] zeroinitializer, section "{{[^"]+}}", align 8
-; STATIC: @__profvp_bar = private global [1 x i64] zeroinitializer, section "{{[^"]+}}", comdat($__profd_bar), align 8
+; STATIC: @__profvp_foo = private global [1 x i64] zeroinitializer, section "{{[^"]+}}"
+; STATIC: @__profvp_bar = private global [1 x i64] zeroinitializer, section "{{[^"]+}}", comdat($__profd_bar)
 ; STATIC: @__llvm_prf_vnodes
 
 ; DYN-NOT: @__profvp_foo
 ; DYN-NOT: @__llvm_prf_vnodes
 
 ; STATIC: call void @__llvm_profile_instrument_target(i64 %3, i8* bitcast ({ i64, i64, i64*, i8*, i8*, i32, [2 x i16] }* @__profd_foo to i8*), i32 0)
 ; STATIC-EXT: call void @__llvm_profile_instrument_target(i64 %3, i8* bitcast ({ i64, i64, i64*, i8*, i8*, i32, [2 x i16] }* @__profd_foo to i8*), i32 zeroext 0)
 ; STATIC-SEXT: call void @__llvm_profile_instrument_target(i64 %3, i8* bitcast ({ i64, i64, i64*, i8*, i8*, i32, [2 x i16] }* @__profd_foo to i8*), i32 signext 0)
 
 ; STATIC: declare void @__llvm_profile_instrument_target(i64, i8*, i32)
 ; STATIC-EXT: declare void @__llvm_profile_instrument_target(i64, i8*, i32 zeroext)
 ; STATIC-SEXT: declare void @__llvm_profile_instrument_target(i64, i8*, i32 signext)

llvm/test/Instrumentation/InstrProfiling/linkage.ll

 ;; Check that runtime symbols get appropriate linkage.
 
-; RUN: opt < %s -mtriple=x86_64-apple-macosx10.10.0 -instrprof -S | FileCheck %s --check-prefixes=POSIX,MACHO
-; RUN: opt < %s -mtriple=x86_64-unknown-linux -instrprof -S | FileCheck %s --check-prefixes=POSIX,LINUX
-; RUN: opt < %s -mtriple=x86_64-unknown-fuchsia -instrprof -S | FileCheck %s --check-prefixes=POSIX,LINUX
+; RUN: opt < %s -mtriple=x86_64-apple-macosx10.10.0 -instrprof -S | FileCheck %s --check-prefixes=MACHO
+; RUN: opt < %s -mtriple=x86_64-unknown-linux -instrprof -S | FileCheck %s --check-prefixes=ELF
+; RUN: opt < %s -mtriple=x86_64-unknown-fuchsia -instrprof -S | FileCheck %s --check-prefixes=ELF
 ; RUN: opt < %s  -mtriple=x86_64-pc-win32-coff -instrprof -S | FileCheck %s --check-prefixes=COFF
-; RUN: opt < %s -mtriple=x86_64-apple-macosx10.10.0 -passes=instrprof -S | FileCheck %s --check-prefixes=POSIX,MACHO
-; RUN: opt < %s -mtriple=x86_64-unknown-linux -passes=instrprof -S | FileCheck %s --check-prefixes=POSIX,LINUX
-; RUN: opt < %s -mtriple=x86_64-unknown-fuchsia -passes=instrprof -S | FileCheck %s --check-prefixes=POSIX,LINUX
+; RUN: opt < %s -mtriple=x86_64-apple-macosx10.10.0 -passes=instrprof -S | FileCheck %s --check-prefixes=MACHO
+; RUN: opt < %s -mtriple=x86_64-unknown-linux -passes=instrprof -S | FileCheck %s --check-prefixes=ELF
+; RUN: opt < %s -mtriple=x86_64-unknown-fuchsia -passes=instrprof -S | FileCheck %s --check-prefixes=ELF
 ; RUN: opt < %s  -mtriple=x86_64-pc-win32-coff -passes=instrprof -S | FileCheck %s --check-prefixes=COFF
 
 ; MACHO: @__llvm_profile_runtime = external global i32
-; LINUX-NOT: @__llvm_profile_runtime = external global i32
+; ELF-NOT: @__llvm_profile_runtime = external global i32
+
+; ELF: $__profd_foo = comdat noduplicates
+; ELF: $__profd_foo_weak = comdat noduplicates
+; ELF: $"__profd_linkage.ll:foo_internal" = comdat noduplicates
+; ELF: $__profd_foo_inline = comdat noduplicates
+; ELF: $__profd_foo_extern = comdat any
 
 @__profn_foo = hidden constant [3 x i8] c"foo"
 @__profn_foo_weak = weak hidden constant [8 x i8] c"foo_weak"
 @"__profn_linkage.ll:foo_internal" = internal constant [23 x i8] c"linkage.ll:foo_internal"
 @__profn_foo_inline = linkonce_odr hidden constant [10 x i8] c"foo_inline"
 @__profn_foo_extern = linkonce_odr hidden constant [10 x i8] c"foo_extern"
 
-; POSIX: @__profc_foo = hidden global
-; POSIX: @__profd_foo = hidden global
+; ELF: @__profc_foo = hidden global{{.*}}section "__llvm_prf_cnts", comdat($__profd_foo)
+; ELF: @__profd_foo = hidden global{{.*}}section "__llvm_prf_data", comdat
+; MACHO: @__profc_foo = hidden global
+; MACHO: @__profd_foo = hidden global
 ; COFF: @__profc_foo = internal global
 ; COFF-NOT: comdat
 ; COFF: @__profd_foo = internal global
 define void @foo() {
   call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__profn_foo, i32 0, i32 0), i64 0, i32 1, i32 0)
   ret void
 }
 
-; POSIX: @__profc_foo_weak = weak hidden global
-; POSIX: @__profd_foo_weak = weak hidden global
+; ELF: @__profc_foo_weak = weak hidden global{{.*}}section "__llvm_prf_cnts", comdat($__profd_foo_weak)
+; ELF: @__profd_foo_weak = weak hidden global{{.*}}section "__llvm_prf_data", comdat
+; MACHO: @__profc_foo_weak = weak hidden global
+; MACHO: @__profd_foo_weak = weak hidden global
 ; COFF: @__profc_foo_weak = internal global
 ; COFF: @__profd_foo_weak = internal global
 define weak void @foo_weak() {
   call void @llvm.instrprof.increment(i8* getelementptr inbounds ([8 x i8], [8 x i8]* @__profn_foo_weak, i32 0, i32 0), i64 0, i32 1, i32 0)
   ret void
 }
 
-; POSIX: @"__profc_linkage.ll:foo_internal" = internal global
-; POSIX: @"__profd_linkage.ll:foo_internal" = internal global
+; ELF: @"__profc_linkage.ll:foo_internal" = internal global{{.*}}section "__llvm_prf_cnts", comdat($"__profd_linkage.ll:foo_internal")
+; ELF: @"__profd_linkage.ll:foo_internal" = internal global{{.*}}section "__llvm_prf_data", comdat
+; MACHO: @"__profc_linkage.ll:foo_internal" = internal global
+; MACHO: @"__profd_linkage.ll:foo_internal" = internal global
 ; COFF: @"__profc_linkage.ll:foo_internal" = internal global
 ; COFF: @"__profd_linkage.ll:foo_internal" = internal global
 define internal void @foo_internal() {
   call void @llvm.instrprof.increment(i8* getelementptr inbounds ([23 x i8], [23 x i8]* @"__profn_linkage.ll:foo_internal", i32 0, i32 0), i64 0, i32 1, i32 0)
   ret void
 }
 
-; POSIX: @__profc_foo_inline = linkonce_odr hidden global
-; POSIX: @__profd_foo_inline = linkonce_odr hidden global
+; ELF: @__profc_foo_inline = linkonce_odr hidden global{{.*}}section "__llvm_prf_cnts", comdat($__profd_foo_inline)
+; ELF: @__profd_foo_inline = linkonce_odr hidden global{{.*}}section "__llvm_prf_data", comdat
+; MACHO: @__profc_foo_inline = linkonce_odr hidden global
+; MACHO: @__profd_foo_inline = linkonce_odr hidden global
 ; COFF: @__profc_foo_inline = internal global{{.*}} section ".lprfc$M", align 8
 ; COFF: @__profd_foo_inline = internal global{{.*}} section ".lprfd$M", align 8
 define linkonce_odr void @foo_inline() {
   call void @llvm.instrprof.increment(i8* getelementptr inbounds ([10 x i8], [10 x i8]* @__profn_foo_inline, i32 0, i32 0), i64 0, i32 1, i32 0)
   ret void
 }
 
-; LINUX: @__profc_foo_extern = linkonce_odr hidden global {{.*}}section "__llvm_prf_cnts", comdat($__profd_foo_extern), align 8
-; LINUX: @__profd_foo_extern = linkonce_odr hidden global {{.*}}section "__llvm_prf_data", comdat, align 8
+; ELF: @__profc_foo_extern = linkonce_odr hidden global {{.*}}section "__llvm_prf_cnts", comdat($__profd_foo_extern), align 8
+; ELF: @__profd_foo_extern = linkonce_odr hidden global {{.*}}section "__llvm_prf_data", comdat, align 8
 ; MACHO: @__profc_foo_extern = linkonce_odr hidden global
 ; MACHO: @__profd_foo_extern = linkonce_odr hidden global
 ; COFF: @__profc_foo_extern = linkonce_odr hidden global {{.*}}section ".lprfc$M", comdat, align 8
 ; COFF: @__profd_foo_extern = linkonce_odr hidden global {{.*}}section ".lprfd$M", comdat, align 8
 define available_externally void @foo_extern() {
   call void @llvm.instrprof.increment(i8* getelementptr inbounds ([10 x i8], [10 x i8]* @__profn_foo_extern, i32 0, i32 0), i64 0, i32 1, i32 0)
   ret void
 }
 
 declare void @llvm.instrprof.increment(i8*, i64, i32, i32)
 
 ; MACHO: define linkonce_odr hidden i32 @__llvm_profile_runtime_user() {{.*}} {
 ; MACHO:   %[[REG:.*]] = load i32, i32* @__llvm_profile_runtime
 ; MACHO:   ret i32 %[[REG]]
 ; MACHO: }
 ; COFF: define linkonce_odr hidden i32 @__llvm_profile_runtime_user() {{.*}} comdat {
-; LINUX-NOT: define linkonce_odr hidden i32 @__llvm_profile_runtime_user() {{.*}} {
-; LINUX-NOT:   %[[REG:.*]] = load i32, i32* @__llvm_profile_runtime
+; ELF-NOT: define linkonce_odr hidden i32 @__llvm_profile_runtime_user() {{.*}} {
+; ELF-NOT:   %[[REG:.*]] = load i32, i32* @__llvm_profile_runtime

llvm/test/Instrumentation/InstrProfiling/platform.ll

 ; RUN: opt < %s -mtriple=x86_64-pc-windows -passes=instrprof -S | FileCheck %s -check-prefix=WINDOWS
 
 @__profn_foo = hidden constant [3 x i8] c"foo"
 ; MACHO-NOT: __profn_foo
 ; ELF-NOT: __profn_foo
 ; WINDOWS-NOT: __profn_foo
 
 ; MACHO: @__profc_foo = hidden global [1 x i64] zeroinitializer, section "__DATA,__llvm_prf_cnts", align 8
-; ELF: @__profc_foo = hidden global [1 x i64] zeroinitializer, section "__llvm_prf_cnts", align 8
+; ELF: @__profc_foo = hidden global [1 x i64] zeroinitializer, section "__llvm_prf_cnts", comdat($__profd_foo), align 8
 ; WINDOWS: @__profc_foo = internal global [1 x i64] zeroinitializer, section ".lprfc$M", align 8
 
 ; MACHO: @__profd_foo = hidden {{.*}}, section "__DATA,__llvm_prf_data,regular,live_support", align 8
-; ELF: @__profd_foo = hidden {{.*}}, section "__llvm_prf_data", align 8
+; ELF: @__profd_foo = hidden {{.*}}, section "__llvm_prf_data", comdat, align 8
 ; WINDOWS: @__profd_foo = internal global {{.*}}, section ".lprfd$M", align 8
 
 ; ELF: @__llvm_prf_nm = private constant [{{.*}} x i8] c"{{.*}}", section "{{.*}}__llvm_prf_names", align 1
 ; WINDOWS: @__llvm_prf_nm = private constant [{{.*}} x i8] c"{{.*}}", section "{{.*}}lprfn$M", align 1
 
 define void @foo() {
   call void @llvm.instrprof.increment(i8* getelementptr inbounds ([3 x i8], [3 x i8]* @__profn_foo, i32 0, i32 0), i64 0, i32 1, i32 0)
   ret void

llvm/test/Transforms/PGOProfile/counter_promo.ll

 ; NONATOMIC_PROMO-NEXT: add {{.*}} %[[PROMO2]], %[[LIVEOUT2]]
 ; NONATOMIC_PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}1)
 ; NONATOMIC_PROMO-NEXT: %[[PROMO3:[a-z0-9.]+]] = load {{.*}} @__profc_foo{{.*}} 2)
 ; NONATOMIC_PROMO-NEXT: add {{.*}} %[[PROMO3]], %[[LIVEOUT3]]
 ; NONATOMIC_PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}2)
 ; ATOMIC_PROMO: atomicrmw add {{.*}} @__profc_foo{{.*}}0), i64 %[[LIVEOUT1]] seq_cst
 ; ATOMIC_PROMO-NEXT: atomicrmw add {{.*}} @__profc_foo{{.*}}1), i64 %[[LIVEOUT2]] seq_cst
 ; ATOMIC_PROMO-NEXT: atomicrmw add {{.*}} @__profc_foo{{.*}}2), i64 %[[LIVEOUT3]] seq_cst
-; PROMO-NOT: @__profc_foo
+; PROMO-NOT: @__profc_foo{{.*}})
 
 
 }
 
 declare void @bar(i32)

llvm/test/Transforms/PGOProfile/counter_promo_mexits.ll

 ; PROMO-NEXT: add 
 ; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}1)
 ; PROMO-NEXT:  %pgocount.promoted{{.*}} = load {{.*}} @__profc_foo{{.*}} 2)
 ; PROMO-NEXT: add 
 ; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}2)
 ; PROMO-NEXT:  %pgocount{{.*}} = load {{.*}} @__profc_foo{{.*}} 4)
 ; PROMO-NEXT: add 
 ; PROMO-NEXT: store {{.*}}@__profc_foo{{.*}}4)
-; PROMO-NOT: @__profc_foo
+; PROMO-NOT: @__profc_foo{{.*}})
 
 
 bb15:                                             ; preds = %bb14, %bb4
   tail call void @bar(i32 1)
   ret void
 }
 
 declare void @bar(i32) local_unnamed_addr