[CSPGO] Set module flag for CSPGO to avoid IRPGO value loss in LTO

[YolandaCY]

Summary

The IRPGOFlag symbol (__llvm_profile_raw_version) will be dropped when identified
as non-prevailing for either regular or thin LTO during the mixed-LTO mode compilation.
This caused an unresolved symbol linker error when enable CSPGO over ThinLTO
after https://reviews.llvm.org/D103717.

Try set a CSIRPGO flag for CSPGO to avoid this symbol resolution caused value loss.

[YolandaCY]

Use a new flag to avoid test failures

[YolandaCY]

Fix for CSPGO only

[YolandaCY]

cleanup

[YolandaCY]

lint fix

[YolandaCY]

rebase

[YolandaCY]

Could you help take a look? This is trying to address a linker error for CSPGO with ThinLTO on Windows after a recent CL to keep __profd_ variable private.

[rnk]

Should we set the "EnableValueProfiling" module flag as an alternative way to express that value profiling is enabled? That flag wouldn't be subject to this kind of symbol resolution.

[YolandaCY]

Thank you Reid for the comment. (Sorry I missed your comment last week.)

Yes, setting "EnableValueProfiling" flag can be an option. Just I encountered a test failure at "llvm/test/Instrumentation/InstrProfiling/comdat.ll" that will change the generated "__prof*" variable name with a trailing 0. The test may need be revised. I'm not quite sure if "EnableValueProfiling" flag can be equal to the IRFlag value, if there's no objection, I can try this method.

[YolandaCY]

Change to set EnableValueProfiling module flag

[YolandaCY]

cleanup unrelated changes

[rnk]

llvm/lib/LTO/LTO.cpp

1236 ↗

(On Diff #365957)

My only concern is that we have to do this in two places. Is there a better place we can do this? Maybe the frontend, or wherever we enable IR PGO? These feel like the wrong places.

[YolandaCY]

Set the flag earlier when generate the IRPGO var

[YolandaCY]

llvm/lib/LTO/LTO.cpp

1236 ↗

(On Diff #365957)

Thank you Reid for the suggestion. Agree that setting this ealier when we enable the IR PGO is better. Revised the patch accordingly.

[rnk]

lgtm

@MaskRay, please take a look.

[MaskRay]

The IRPGOFlag symbol (__llvm_profile_raw_version) will be dropped when identified as non-prevailing for either regular or thin LTO during the mixed-LTO mode compilation. This caused an unresolved symbol linker error when enable CSPGO over ThinLTO after https://reviews.llvm.org/D103717.

I am concerned that EnableValueProfiling might not be added to the correct place. (Why specifically to CSPGO?) Can you give a concrete example for the mixed LTO scenario?
What's the "duplicate symbol" error?

llvm/test/Transforms/PGOProfile/lto_cspgo_gen.ll

12 ↗

(On Diff #366226)

This test is unclear why EnableValueProfiling is set for CSPGO.

[MaskRay]

llvm/lib/ProfileData/InstrProf.cpp

1148

The way comdat any is used, it is unsupported if you mix non-CS and CS PGO object files. The first copy is picked by the linker.

This does not seem a proper fix to me. If you can give a concrete example, if may help me to understand.

[YolandaCY]

In D107034#2944459, @MaskRay wrote:

The IRPGOFlag symbol (__llvm_profile_raw_version) will be dropped when identified as non-prevailing for either regular or thin LTO during the mixed-LTO mode compilation. This caused an unresolved symbol linker error when enable CSPGO over ThinLTO after https://reviews.llvm.org/D103717.

I am concerned that EnableValueProfiling might not be added to the correct place. (Why specifically to CSPGO?) Can you give a concrete example for the mixed LTO scenario?
What's the "duplicate symbol" error?

The background is when enable CSPGO over ThinLTO, it requires a seperate pass to create IRPGO flag in prelink, hence the var will later be subject to symbol resolution during LTO. This is not the case for PGO though. See references at:
https://github.com/llvm/llvm-project/blob/main/llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp#L1614
https://github.com/llvm/llvm-project/blob/main/llvm/lib/Passes/PassBuilder.cpp#L1183

For mixed LTO, I think it should relate to flag "-fsplit-lto-unit" (also subject to the whole program devirtualization optimization pass). I included a test case for hybrid lto in a previous revision, see: https://reviews.llvm.org/D107034?vs=on&id=363666#toc
You may find a "...!"ThinLTO", i32 0}" mudule flag which indicate this file will be added to regular LTO, while the other without it will be added for ThinLTO.

Here are some more references on why mixed LTO is needed and how:
https://github.com/llvm/llvm-project/blob/main/llvm/lib/Transforms/IPO/WholeProgramDevirt.cpp#L34
https://github.com/llvm/llvm-project/blob/main/llvm/lib/Transforms/IPO/ThinLTOBitcodeWriter.cpp#L269

The duplicate symbol error if when I simply try to reserve the IRPGO flag in regular LTO when it is identified as non-prevailing, I will get such error for ELF, but not COFF.

Hope this helps.

llvm/lib/ProfileData/InstrProf.cpp

1148

Yes, I think mixing non-CS and CS PGO seems not a normal practice. As CSPGO is trying collect profile after inlining by applying profile created by PGO.
The user guide for CSPGO is run after PGO as described here:
https://clang.llvm.org/docs/UsersManual.html#cmdoption-fcs-profile-generate

Hope this makes sense now.

[MaskRay]

Can you give concrete instructions to reproduce the linker error?

[YolandaCY]

In D107034#2945640, @MaskRay wrote:

Can you give concrete instructions to reproduce the linker error?

I tried to create a small reproducer but still encountered several issues. The linker error may be explained with below example:

class Str {
  int size;

public:
  char *p;

  Str(int size) {
    p = new char[size];
  }

  ~Str() {
    delete[] p;
  }

  void assign_mem(char *a, char *b, int size) {
    memcpy(a, b, size);
  }

  int assign(Str &b) {
    if (b.size < size) return -1;
    assign_mem(p, b.p, size);
    return 1;
  }
};

If function assign inlines assign_mem, it does contain mem op and should require a value profile var, but not vice versa.
The case encounterd the linker error is that a function similar to assign (e.g. basic_string::operator=) does not inline the callee assign_mem in the splitted regular LTO link, thus it will generate a __profd_assign* with private linkage, while in a later ThinLTO module, the assign inlined the callee containing mem op, thus it may generate a __profd_assign with non-private linkage (due to NS!=0). During the final link together with both regular and thin LTO modules for COFF, symbols from regular LTO are inserted earlier as prevailing, and the private __profd_assign* is not inserted. Whereas the symbol from thin module has external attribute, and is associated to __profc_assign* who is non-prevailing and should not be inserted. While there is a prevailing __profc_assign* existing in symbol table, there is no __profd_assign*, thus reported as unresolved symbol.

I think by correcting the profile var name with a hash suffix when IRPGO enabled should avoid such inconsistency in different translation units. And the linker error helped exposing the issue.

I'm implementing a CL to build Chromium that depends on this. The steps described in the CL should help reproduce the error if you are interested @MaskRay. The original error looks like following:

lld-link: error: undefined symbol: __profd_??4?$basic_string@DU?$char_traits@D@__1@std@@V?$allocator@D@23@@__1@std@@QEAAAEAV012@AEBV012@@Z
>>> referenced by .\..\..\buildtools\third_party\libc++\trunk\include\string:2401
>>>               ./chrome.dll.lto.147.obj:(public: class std::__1::basic_string<char, struct std::__1::char_traits<char>, class std::__1::allocator<char>> & __cdecl std::td::__1::allocator<char>>::operator=(class std::__1::basic_string<char, struct std::__1::char_traits<char>, class std::__1::allocator<char>> const &))
>>> referenced by .\..\..\buildtools\third_party\libc++\trunk\include\string:2401
>>>               ./chrome.dll.lto.147.obj:(public: class std::__1::basic_string<char, struct std::__1::char_traits<char>, class std::__1::allocator<char>> & __cdecl std::td::__1::allocator<char>>::operator=(class std::__1::basic_string<char, struct std::__1::char_traits<char>, class std::__1::allocator<char>> const &))

Please let me know if more information is needed.

[MaskRay]

So, this appears to be a PE/COFF specific problem, due to our wrestling with link.exe's comdat limitation....

I am still unclear yet whether this is only specific to post-inlining instrumentation. If yes, this is a CSPGO specific problem. Perhaps we can think about dropping link.exe compatibility if this turns out to be too much of a hassle...

---

I do not understand this part:

Whereas the symbol from thin module has external attribute, and is associated to profc_assign* who is non-prevailing and should not be inserted. While there is a prevailing profc_assign* existing in symbol table, there is no __profd_assign*, thus reported as unresolved symbol.

Summary of the issue:

• The final (after implicit or distributed ThinLTO generated object files) linker command line looks like lld-link ... regular.o thin.o.
• regular.o (regular LTO) emits __prof[cd]_assign. __profc_assign as the comdat leader is prevailing. profc is linkonce_odr while profd is private.
• thin.o (Thin LTO) emits __prof[cd]_assign. __profc_assign as the comdat leader is non-prevailing. Both profc and profd are linkonce_odr.

Does your change make regular.o __profc_assign linkonce_odr?
Which variable in a non-prevailing comdat is referenced, causing an undefined symbol error?

I think by correcting the profile var name with a hash suffix when IRPGO enabled should avoid such inconsistency in different translation units. And the linker error helped exposing the issue.

Any idea why a hash suffix isn't used?
See canRenameComdatFunc, is the address of assign taken?

[MaskRay]

llvm/lib/ProfileData/InstrProf.cpp

1149

According to your description, various profc could cause linker errors. So the comment "The IRPGO flag var (__llvm_profile_raw_version) has external linkage," is incomplete/incorrect.

[MaskRay]

I edited my comment a bit so your email notification may not catch it.
This looks to me a PE/COFF specific issue. I wonder whether we can drop link.exe support for PGO / CSPGO.
link.exe has the limitation "non-leader members of a comdat must be private".
I recall that PE/COFF can entirely use the ELF scheme if we can lift the limitation.
(https://lists.llvm.org/pipermail/llvm-dev/2021-May/150758.html)

[YolandaCY]

In D107034#2953641, @MaskRay wrote:

So, this appears to be a PE/COFF specific problem, due to our wrestling with link.exe's comdat limitation....

I am still unclear yet whether this is only specific to post-inlining instrumentation. If yes, this is a CSPGO specific problem. Perhaps we can think about dropping link.exe compatibility if this turns out to be too much of a hassle...

---

I do not understand this part:

Whereas the symbol from thin module has external attribute, and is associated to profc_assign* who is non-prevailing and should not be inserted. While there is a prevailing profc_assign* existing in symbol table, there is no __profd_assign*, thus reported as unresolved symbol.

Summary of the issue:

• The final (after implicit or distributed ThinLTO generated object files) linker command line looks like lld-link ... regular.o thin.o.
• regular.o (regular LTO) emits __prof[cd]_assign. __profc_assign as the comdat leader is prevailing. profc is linkonce_odr while profd is private.
• thin.o (Thin LTO) emits __prof[cd]_assign. __profc_assign as the comdat leader is non-prevailing. Both profc and profd are linkonce_odr.

In thin.o, both profc and profd are weak_odr linkage.

Does your change make regular.o __profc_assign linkonce_odr?

Yes. Just make sure the ValueProfiling flag available and will keep it as it is for COFF.

Which variable in a non-prevailing comdat is referenced, causing an undefined symbol error?

It's profd_assign. As its associate parent (profc_assign) is non-prevailing, it will get discarded as the parent. See https://github.com/llvm/llvm-project/blob/main/lld/COFF/InputFiles.cpp#L349

I think by correcting the profile var name with a hash suffix when IRPGO enabled should avoid such inconsistency in different translation units. And the linker error helped exposing the issue.

Any idea why a hash suffix isn't used?
See canRenameComdatFunc, is the address of assign taken?

The canRenameComdatFunc is turned off by default by flag DoComdatRenaming. I can have a try later.
The getVarName for profile variables will add FunHash as suffix in this change.

[YolandaCY]

In D107034#2953729, @MaskRay wrote:

I edited my comment a bit so your email notification may not catch it.
This looks to me a PE/COFF specific issue. I wonder whether we can drop link.exe support for PGO / CSPGO.
link.exe has the limitation "non-leader members of a comdat must be private".
I recall that PE/COFF can entirely use the ELF scheme if we can lift the limitation.
(https://lists.llvm.org/pipermail/llvm-dev/2021-May/150758.html)

I think I used lld-link.exe, not MSVC's link.exe. As mentioned in the link,

lld-link doesn't have this limitation.

I hope my last reply will help resolve some confusion.

[MaskRay]

In D107034#2954190, @YolandaCY wrote:

In D107034#2953641, @MaskRay wrote:

So, this appears to be a PE/COFF specific problem, due to our wrestling with link.exe's comdat limitation....

I am still unclear yet whether this is only specific to post-inlining instrumentation. If yes, this is a CSPGO specific problem. Perhaps we can think about dropping link.exe compatibility if this turns out to be too much of a hassle...

---

I do not understand this part:

Whereas the symbol from thin module has external attribute, and is associated to profc_assign* who is non-prevailing and should not be inserted. While there is a prevailing profc_assign* existing in symbol table, there is no __profd_assign*, thus reported as unresolved symbol.

Summary of the issue:

• The final (after implicit or distributed ThinLTO generated object files) linker command line looks like lld-link ... regular.o thin.o.
• regular.o (regular LTO) emits __prof[cd]_assign. __profc_assign as the comdat leader is prevailing. profc is linkonce_odr while profd is private.
• thin.o (Thin LTO) emits __prof[cd]_assign. __profc_assign as the comdat leader is non-prevailing. Both profc and profd are linkonce_odr.

In thin.o, both profc and profd are weak_odr linkage.

OK, clang emitted weak_odr directly, perhaps due to explicit instantiation of a template (GVA_StrongODR).
Since comdat renaming is disabled in this case, profd should not be private.

Does your change make regular.o __profc_assign linkonce_odr?

Yes. Just make sure the ValueProfiling flag available and will keep it as it is for COFF.

Which variable in a non-prevailing comdat is referenced, causing an undefined symbol error?

It's profd_assign. As its associate parent (profc_assign) is non-prevailing, it will get discarded as the parent. See https://github.com/llvm/llvm-project/blob/main/lld/COFF/InputFiles.cpp#L349

Is __profd_assign referenced by a non-assign function? The __profd_assign reference is a result of inlining.

I think by correcting the profile var name with a hash suffix when IRPGO enabled should avoid such inconsistency in different translation units. And the linker error helped exposing the issue.

Any idea why a hash suffix isn't used?
See canRenameComdatFunc, is the address of assign taken?

The canRenameComdatFunc is turned off by default by flag DoComdatRenaming. I can have a try later.
The getVarName for profile variables will add FunHash as suffix in this change.

DoComdatRenaming is PGOInstrumentation.cpp's comdat renaming.
canRenameComdatFunc is InstrProfiling.cpp's comdat renaming. It is not disabled.

linkonce_odr functions are generally renamed.
The linkage in question is weak_odr, so the renaming is disabled for canRenameComdatFunc.

So now this change looks strange to me:

if (!DoHashBasedCounterSplit || !enablesValueProfiling(*M) ||
    !canRenameComdatFunc(*F))
  return (Prefix + Name).str();

canRenameComdatFunc(*F) is false.

[MaskRay]

Perhaps the right fix is D108432. I need to update its description.

[YolandaCY]

In D107034#2956440, @MaskRay wrote:

Perhaps the right fix is D108432. I need to update its description.

Thank you for the patch. I will have a try with this fix and get back to you.

There might be multiple solutions to resolve the issues. Regardingless the linker error, the issue with IRPGO flag missing should be addressed I think. Does that sound good to you from CSPGO point of view?

[MaskRay]

About this patch M.addModuleFlag(llvm::Module::Warning, "EnableValueProfiling", 1);: I am not so sure we should do it.

Note that we only set "EnableValueProfiling" to 0 for the clang -fprofile-instr-generate case. The value is not set to 1 (without poking into the internal -enable-value-profiling option).

clang -fprofile-generate with value profiling doesn't set this module flag, so -fcs-profile-generate doesn't need to set it, either.

lld/COFF/LTO.cpp

225

drop unrelated change

[YolandaCY]

In D107034#2956730, @MaskRay wrote:

About this patch M.addModuleFlag(llvm::Module::Warning, "EnableValueProfiling", 1);: I am not so sure we should do it.

From enablesValueProfiling both IRPGOFlag and EnableValueProfiling mudule flag are examined as indicators. My understanding IRPGOFlag should imply "EnableValueProfiling" to 1.

This alternative solution was recommended by Reid, and no objection from Xu. Could you help clarify with Fangrui? @rnk @xur

Note that we only set "EnableValueProfiling" to 0 for the clang -fprofile-instr-generate case. The value is not set to 1 (without poking into the internal -enable-value-profiling option).

clang -fprofile-generate with value profiling doesn't set this module flag, so -fcs-profile-generate doesn't need to set it, either.

I think I have clarified why this change is unnecessary to PGO in previous reply.

I can drop the unrelated change regarding /lldsavetemps for sure. That should be an easy fix.

[MaskRay]

In D107034#2956812, @YolandaCY wrote:

In D107034#2956730, @MaskRay wrote:

About this patch M.addModuleFlag(llvm::Module::Warning, "EnableValueProfiling", 1);: I am not so sure we should do it.

From enablesValueProfiling both IRPGOFlag and EnableValueProfiling mudule flag are examined as indicators. My understanding IRPGOFlag should imply "EnableValueProfiling" to 1.

This alternative solution was recommended by Reid, and no objection from Xu. Could you help clarify with Fangrui? @rnk @xur

"EnableValueProfiling" is set to 0 by -fprofile-instr-generate to enable a size saving optimization (D102818).

IR profiling doesn't set 1. It is inconsistent for CSPGO to set "EnableValueProfiling" to 1, while -fprofile-generate keeps omitting the module flag.
-disable-vp can disable value profiling. This patch doesn't respect that option.

In addition, I think changing the condition in getVarName is incorrect. Coverage and PGO have different use cases while share some infrastructure (InstrProfiling.cpp).
Coverage suppresses renaming to have better coverage results. The decision is independent of whether value profiling is enabled or not.
On the other hand, PGO's preferring renaming behavior is also independent of value profiling.

Some confusion here is that -fprofile-instr-generate was used to do PGO (perhaps all groups except Apple have migrated away).
When "EnableValueProfiling" was introduced, I think the intention was to prevent breakage to them.
Otherwise, PGO use cases could just ignore "EnableValueProfiling" everywhere.

Actually, I am thinking of dropping "EnableValueProfiling". It needs some COFF side work (D44543) (or we just unsupport that use case).

Note that we only set "EnableValueProfiling" to 0 for the clang -fprofile-instr-generate case. The value is not set to 1 (without poking into the internal -enable-value-profiling option).

clang -fprofile-generate with value profiling doesn't set this module flag, so -fcs-profile-generate doesn't need to set it, either.

I think I have clarified why this change is unnecessary to PGO in previous reply.

I can drop the unrelated change regarding /lldsavetemps for sure. That should be an easy fix.

As previously explained, I think the patch is not correct. So keeping the "Request Changes" state.

[xur]

Sorry that I did not follow this patch closely. But after reading this I don't think this is a proper fix.

Are you sure IRPGOFlag is dropped? CSPGO won't work properly if this happens. You might work-around the linker issue (unresolved symbols), but the resulted profile will not be marked properly.
But I really doubt IRPGOFlag is indeed dropped. From the explanation of the example test case, it seems you just see the unsats from value profiling variables. Do you have other evidence that IRPGOFlag variable is dropped?

A test case is really helping here (or the build log).

Did you try MasRay's patch
https://reviews.llvm.org/D108432
to see if that fixes the problem?

[YolandaCY]

In D107034#2958541, @xur wrote:

Sorry that I did not follow this patch closely. But after reading this I don't think this is a proper fix.

Are you sure IRPGOFlag is dropped? CSPGO won't work properly if this happens. You might work-around the linker issue (unresolved symbols), but the resulted profile will not be marked properly.
But I really doubt IRPGOFlag is indeed dropped. From the explanation of the example test case, it seems you just see the unsats from value profiling variables. Do you have other evidence that IRPGOFlag variable is dropped?

This variable is not dropped for the final binary, it’s just dropped for each module (if non-prevailing) during LTO/ThinLTO as there’s no uses or references. In the final instrumented binary, the variable is still kept. The example in discussion was trying to explain why it will introduce a unresolved symbol error, can be a seperate issue.

A test case is really helping here (or the build log).

For the loss of IRPGOFlag in LTO should be easy to reproduce. (Just hard for the linker issue)
You may have a try to the test I added lto_cspgo_gen.ll, if exaimine output of the last command "llvm-dis %t.0.0.preopt.bc -o - ", you will find the variable "__llvm_profile_raw_version" is not emitted.

Did you try MasRay's patch
https://reviews.llvm.org/D108432
to see if that fixes the problem?

MaskRay's patch may help avoid the linker error, but it wont fix the IRPGOFlag missing issue.

I actually have an alternative patch in a previous diff, if we have concern on using EnableValueProfiling flag.
Not sure if this looks better?

[MaskRay]

In D107034#2958599, @YolandaCY wrote:

In D107034#2958541, @xur wrote:

Sorry that I did not follow this patch closely. But after reading this I don't think this is a proper fix.

Are you sure IRPGOFlag is dropped? CSPGO won't work properly if this happens. You might work-around the linker issue (unresolved symbols), but the resulted profile will not be marked properly.
But I really doubt IRPGOFlag is indeed dropped. From the explanation of the example test case, it seems you just see the unsats from value profiling variables. Do you have other evidence that IRPGOFlag variable is dropped?

This variable is not dropped for the final binary, it’s just dropped for each module (if non-prevailing) during LTO/ThinLTO as there’s no uses or references. In the final instrumented binary, the variable is still kept. The example in discussion was trying to explain why it will introduce a unresolved symbol error, can be a seperate issue.

A test case is really helping here (or the build log).

For the loss of IRPGOFlag in LTO should be easy to reproduce. (Just hard for the linker issue)
You may have a try to the test I added lto_cspgo_gen.ll, if exaimine output of the last command "llvm-dis %t.0.0.preopt.bc -o - ", you will find the variable "__llvm_profile_raw_version" is not emitted.

Did you try MasRay's patch
https://reviews.llvm.org/D108432
to see if that fixes the problem?

MaskRay's patch may help avoid the linker error, but it wont fix the IRPGOFlag missing issue.

I actually have an alternative patch in a previous diff, if we have concern on using EnableValueProfiling flag.
Not sure if this looks better?

I am afraid https://reviews.llvm.org/D107034?vs=on&id=363777#toc is incorrect, either.

My understanding below. @xur is the expert and may correct me:)

ThinLTO+CSPGO instrumentation is:

# After regular PGO instrumentation, profile collection, llvm-profdata merge
...
clang -flto=thin -fprofile-use=... -fcs-profile-generate=... a.c b.c # ThinLTO prelink phase
clang -flto=thin -fcs-profile-generate=... a.o b.o # ThinLTO postlink (backend compile) phase

The ThinLTO bitcode files need the -fcs-profile-generate= option to set __llvm_profile_raw_version in the PGOInstrumentationGenCreateVar pass.

ld.lld does comdat selection but the prevailing/non-prevailing decisions do not drop non-prevailing __llvm_profile_raw_version variables from bitcode files.

During ThinLTO postlink (backend compile), InstrProfiling.cpp doesn't need to add __llvm_profile_raw_version, because the variable is already there.

---

I think we need better documentation for CSPGO.

[YolandaCY]

@xur This is also reproducible using the existing test case PGOProfile/thinlto_cspgo_gen.ll. If evaluating the generated "%t.2.4.opt.bc" containing non-prevailing IRPGO flag, you will find the value is missed. (The value is still reserved in "%t.2.3.import.bc".)

In the LTO case, the symbol is dropped at *.preopt phase.

In summary after tracing:

1. In LTO, the value is dropped when addRegularLTO as it has external linkage and non-prevailing.
2. In ThinLTO, the value is first changed to available_external linkage by thinLTOResolvePrevailingInModule and thinLTOResolvePrevailingGUID. It is dropped later in the backend GlobleOpt pass by deleteIfDead.

Hence to keep the value we may need to identify the IRPGO variable in all above places, make sure it is reserved, and then drop it after instrumentation to avoid duplicate symbols in later linkage. My feeling is that using a module flag may avoid too much hassle across multiple places and more complexity to the symbol resolution stuff. (as suggested by Reid)
Agree that mixing EnableValueProfiling with IRPGO flag is improper as it seems Clang also may support value profiling (from comment of shouldRecordFunctionAddr ).
Maybe we can rename the module flag to "CSIRPGO"? We can also remove the flag after instrumentation as no need for later passes.

[YolandaCY]

change flag name and remove unrelated fix

[xur]

In D107034#2959896, @YolandaCY wrote:

@xur This is also reproducible using the existing test case PGOProfile/thinlto_cspgo_gen.ll. If evaluating the generated "%t.2.4.opt.bc" containing non-prevailing IRPGO flag, you will find the value is missed. (The value is still reserved in "%t.2.3.import.bc".)

In the LTO case, the symbol is dropped at *.preopt phase.

In summary after tracing:

1. In LTO, the value is dropped when addRegularLTO as it has external linkage and non-prevailing.
2. In ThinLTO, the value is first changed to available_external linkage by thinLTOResolvePrevailingInModule and thinLTOResolvePrevailingGUID. It is dropped later in the backend GlobleOpt pass by deleteIfDead.

Hence to keep the value we may need to identify the IRPGO variable in all above places, make sure it is reserved, and then drop it after instrumentation to avoid duplicate symbols in later linkage. My feeling is that using a module flag may avoid too much hassle across multiple places and more complexity to the symbol resolution stuff. (as suggested by Reid)
Agree that mixing EnableValueProfiling with IRPGO flag is improper as it seems Clang also may support value profiling (from comment of shouldRecordFunctionAddr ).
Maybe we can rename the module flag to "CSIRPGO"? We can also remove the flag after instrumentation as no need for later passes.

OK. I now see the problem.

I added IRPGOFlag was to only used to mark the profile as IR instrumentation (rather clang instrumentation).
But it was later used to coordinate instrumentation pass and lowering pass. This breaks in thin-lto and lto.

IRPGOFlag now not only is used in enabling value profiling, but also in comdat renaming.

I still don't want to use module flag -- using IRPGOFlag still preferred.

I will fix this.

[xur]

I sent my version of patch as https://reviews.llvm.org/D108581

[YolandaCY]

Issue is fixed at D108581.

lld/COFF/LTO.cpp

for (unsigned i = 0; i != maxTasks; ++i) {
     if (files[i])
       objBuf = files[i]->getBuffer();
     else
       objBuf = buf[i];
     if (objBuf.empty())
       continue;
 
     if (config->saveTemps)
-      saveBuffer(buf[i], ltoObjName);
+      saveBuffer(objBuf, ltoObjName);
     ret.push_back(make<ObjFile>(MemoryBufferRef(objBuf, ltoObjName)));
   }
 
   return ret;
 }

llvm/include/llvm/ProfileData/InstrProf.h

 } // end namespace RawInstrProf
 
 // Parse MemOP Size range option.
 void getMemOPSizeRangeFromOption(StringRef Str, int64_t &RangeStart,
                                  int64_t &RangeLast);
 
 // Create a COMDAT variable INSTR_PROF_RAW_VERSION_VAR to make the runtime
 // aware this is an ir_level profile so it can set the version flag.
 void createIRLevelProfileFlagVar(Module &M, bool IsCS,
-                                 bool InstrEntryBBEnabled);
+                                 bool InstrEntryBBEnabled,
+                                 bool IsPrivate = false);
 
 // Create the variable for the profile file name.
 void createProfileFileNameVar(Module &M, StringRef InstrProfileOutput);
 
 // Whether to compress function names in profile records, and filenames in
 // code coverage mappings. Used by the Instrumentation library and unit tests.
 extern cl::opt<bool> DoInstrProfNameCompression;
 
 } // end namespace llvm
 #endif // LLVM_PROFILEDATA_INSTRPROF_H

llvm/lib/ProfileData/InstrProf.cpp

bool needsComdatForCounter(const Function &F, const Module &M) {
   return true;
 }
 
 // Check if INSTR_PROF_RAW_VERSION_VAR is defined.
 bool isIRPGOFlagSet(const Module *M) {
   auto IRInstrVar =
       M->getNamedGlobal(INSTR_PROF_QUOTE(INSTR_PROF_RAW_VERSION_VAR));
   if (!IRInstrVar || IRInstrVar->isDeclaration() ||
-      IRInstrVar->hasLocalLinkage())
+      (IRInstrVar->hasLocalLinkage() && !IRInstrVar->hasPrivateLinkage()))
     return false;
 
   // Check if the flag is set.
   if (!IRInstrVar->hasInitializer())
     return false;
 
   auto *InitVal = dyn_cast_or_null<ConstantInt>(IRInstrVar->getInitializer());
   if (!InitVal)
if (!F.hasComdat()) {
     assert(F.getLinkage() == GlobalValue::AvailableExternallyLinkage);
     return true;
   }
   return true;
 }
 
 // Create a COMDAT variable INSTR_PROF_RAW_VERSION_VAR to make the runtime
 // aware this is an ir_level profile so it can set the version flag.
 void createIRLevelProfileFlagVar(Module &M, bool IsCS,
-                                 bool InstrEntryBBEnabled) {
+                                 bool InstrEntryBBEnabled, bool IsPrivate) {
   const StringRef VarName(INSTR_PROF_QUOTE(INSTR_PROF_RAW_VERSION_VAR));
   Type *IntTy64 = Type::getInt64Ty(M.getContext());
   uint64_t ProfileVersion = (INSTR_PROF_RAW_VERSION | VARIANT_MASK_IR_PROF);
   if (IsCS)
     ProfileVersion |= VARIANT_MASK_CSIR_PROF;
   if (InstrEntryBBEnabled)
     ProfileVersion |= VARIANT_MASK_INSTR_ENTRY;
   auto IRLevelVersionVariable = new GlobalVariable(
       M, IntTy64, true, GlobalValue::WeakAnyLinkage,
       Constant::getIntegerValue(IntTy64, APInt(64, ProfileVersion)), VarName);
   IRLevelVersionVariable->setVisibility(GlobalValue::DefaultVisibility);
   Triple TT(M.getTargetTriple());
-  if (TT.supportsCOMDAT()) {
+  if (IsPrivate) {
+    IRLevelVersionVariable->setLinkage(GlobalValue::PrivateLinkage);
+  } else if (TT.supportsCOMDAT()) {
     IRLevelVersionVariable->setLinkage(GlobalValue::ExternalLinkage);
     IRLevelVersionVariable->setComdat(M.getOrInsertComdat(VarName));
   }
 }
 
 // Create the variable for the profile file name.
 void createProfileFileNameVar(Module &M, StringRef InstrProfileOutput) {
   if (InstrProfileOutput.empty())

llvm/lib/Transforms/Instrumentation/PGOInstrumentation.cpp

 }
 
 static bool InstrumentAllFunctions(
     Module &M, function_ref<TargetLibraryInfo &(Function &)> LookupTLI,
     function_ref<BranchProbabilityInfo *(Function &)> LookupBPI,
     function_ref<BlockFrequencyInfo *(Function &)> LookupBFI, bool IsCS) {
   // For the context-sensitve instrumentation, we should have a separated pass
   // (before LTO/ThinLTO linking) to create these variables.
-  if (!IsCS)
-    createIRLevelProfileFlagVar(M, /* IsCS */ false, PGOInstrumentEntry);
+  // During LTO/ThinLTO linking, if the IRPGOFlag var is non-prevailing, it will
+  // be dropped for opt. In such case, create the symbol with private linkage
+  // for opt that should be eliminated later to avoid duplicate symbols.
+  if (!IsCS || !isIRPGOFlagSet(&M))
+    createIRLevelProfileFlagVar(M, /* IsCS */ false, PGOInstrumentEntry,
+                    /* IsPrivate */ IsCS);
+
   std::unordered_multimap<Comdat *, GlobalValue *> ComdatMembers;
   collectComdatMembers(M, ComdatMembers);
 
   for (auto &F : M) {
     if (F.isDeclaration())
       continue;
     if (F.hasFnAttribute(llvm::Attribute::NoProfile))
       continue;

llvm/test/Transforms/PGOProfile/Inputs/hybrid_thinlto_cspgo_gen.ll

+; ModuleID = 'test-v1.bc'
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+%class.Child = type { %class.Base.base, [4 x i8] }
+%class.Base.base = type <{ i32 (...)**, i32 }>
+
+$__llvm_profile_filename = comdat any
+
+$__llvm_profile_raw_version = comdat any
+
+@_ZTV5Child = external unnamed_addr constant { [3 x i8*] }, align 8
+@_ZTVN10__cxxabiv121__vmi_class_type_infoE = external dso_local global i8*
+@_ZTS5Child = dso_local constant [7 x i8] c"5Child\00", align 1
+@_ZTI4Base = external dso_local constant i8*
+@_ZTI5Child = dso_local constant { i8*, i8*, i32, i32, i8*, i64 } { i8* bitcast (i8** getelementptr inbounds (i8*, i8** @_ZTVN10__cxxabiv121__vmi_class_type_infoE, i64 2) to i8*), i8* getelementptr inbounds ([7 x i8], [7 x i8]* @_ZTS5Child, i32 0, i32 0), i32 0, i32 1, i8* bitcast (i8** @_ZTI4Base to i8*), i64 0 }, align 8
+@__llvm_profile_filename = local_unnamed_addr constant [19 x i8] c"default_%m.profraw\00", comdat
+@__llvm_profile_raw_version = local_unnamed_addr constant i64 216172782113783814, comdat
+
+; Function Attrs: mustprogress nofree norecurse nosync nounwind readnone uwtable willreturn
+define dso_local i32 @_ZN5Child3getEv(%class.Child* nocapture readnone align 8 %this) unnamed_addr #0 align 2 {
+entry:
+  ret i32 1
+}
+
+attributes #0 = { "target-cpu"="x86-64" }
+
+!llvm.module.flags = !{!0, !1, !2}
+
+!0 = !{i32 1, !"wchar_size", i32 4}
+!1 = !{i32 7, !"uwtable", i32 1}
+!2 = !{i32 1, !"EnableSplitLTOUnit", i32 1}
+
+^0 = module: (path: "test-v1.bc", hash: (1837240108, 3253769035, 1388040350, 3007262886, 1978890690))
+^1 = gv: (name: "__llvm_profile_raw_version", summaries: (variable: (module: ^0, flags: (linkage: external, visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 0, canAutoHide: 0), varFlags: (readonly: 0, writeonly: 0, constant: 1)))) ; guid = 3271499267116101737
+^2 = gv: (name: "_ZTV5Child") ; guid = 5226103449856009057
+^3 = gv: (name: "_ZN5Child3getEv", summaries: (function: (module: ^0, flags: (linkage: external, visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 1, canAutoHide: 0), insts: 1, funcFlags: (readNone: 1, readOnly: 0, noRecurse: 1, returnDoesNotAlias: 0, noInline: 0, alwaysInline: 0)))) ; guid = 7125659195073751167
+^4 = gv: (name: "_ZTS5Child", summaries: (variable: (module: ^0, flags: (linkage: external, visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 1, canAutoHide: 0), varFlags: (readonly: 1, writeonly: 0, constant: 1)))) ; guid = 7547014829888929871
+^5 = gv: (name: "_ZTI5Child", summaries: (variable: (module: ^0, flags: (linkage: external, visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 1, canAutoHide: 0), varFlags: (readonly: 1, writeonly: 0, constant: 1), refs: (^7, ^4, ^6)))) ; guid = 12661005278488252785
+^6 = gv: (name: "_ZTI4Base") ; guid = 13357340906568957678
+^7 = gv: (name: "_ZTVN10__cxxabiv121__vmi_class_type_infoE") ; guid = 15154540976975978366
+^8 = gv: (name: "__llvm_profile_filename", summaries: (variable: (module: ^0, flags: (linkage: external, visibility: default, notEligibleToImport: 0, live: 0, dsoLocal: 0, canAutoHide: 0), varFlags: (readonly: 0, writeonly: 0, constant: 1)))) ; guid = 17292341786426384195
+^9 = flags: 8
+^10 = blockcount: 1

llvm/test/Transforms/PGOProfile/hybrid_lto_cspgo_gen.ll

+; REQUIRES: x86-registered-target
+
+; RUN: opt -module-summary %S/Inputs/hybrid_thinlto_cspgo_gen.ll -o %t1.bc
+; RUN: opt -module-summary %s -o %t2.bc
+
+;; Test IRPGO is set and profiling variables will contain a FuncHash suffix.
+; RUN: ld.lld --lto-cs-profile-file=alloc --lto-cs-profile-generate --lto-O2 --save-temps --shared -o %t %t1.bc %t2.bc
+; RUN: llvm-dis %t.0.4.opt.bc -o - | FileCheck %s --check-prefix=ENABLE-IRPGO
+
+; ENABLE-IRPGO: $__profc__ZN5Child3getEv.[[HASH:[0-9]+]]
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+%class.Child.1 = type { %class.Base.base.0, [4 x i8] }
+%class.Base.base.0 = type <{ i32 (...)**, i32 }>
+
+@_ZTV5Child = dso_local unnamed_addr constant { [3 x i8*] } { [3 x i8*] [i8* null, i8* bitcast ({ i8*, i8*, i32, i32, i8*, i64 }* @_ZTI5Child to i8*), i8* bitcast (i32 (%class.Child.1*)* @_ZN5Child3getEv to i8*)] }, align 8, !type !0, !type !1, !type !2, !type !3
+@_ZTI5Child = external dso_local constant { i8*, i8*, i32, i32, i8*, i64 }, align 8
+
+; Function Attrs: mustprogress nofree norecurse nosync nounwind readnone uwtable willreturn
+define available_externally dso_local i32 @_ZN5Child3getEv(%class.Child.1* nocapture readnone align 8 %this) unnamed_addr #0 align 2 {
+entry:
+  ret i32 1
+}
+
+attributes #0 = { "target-cpu"="x86-64" }
+
+!llvm.module.flags = !{!4, !5, !6, !7}
+
+!0 = !{i64 16, !"_ZTS4Base"}
+!1 = !{i64 16, !"_ZTSM4BaseFivE.virtual"}
+!2 = !{i64 16, !"_ZTS5Child"}
+!3 = !{i64 16, !"_ZTSM5ChildFivE.virtual"}
+!4 = !{i32 1, !"wchar_size", i32 4}
+!5 = !{i32 7, !"uwtable", i32 1}
+!6 = !{i32 1, !"EnableSplitLTOUnit", i32 1}
+!7 = !{i32 1, !"ThinLTO", i32 0}
+
+^0 = module: (path: "test-v1.bc", hash: (0, 0, 0, 0, 0))
+^1 = gv: (name: "_ZTV5Child", summaries: (variable: (module: ^0, flags: (linkage: external, visibility: default, notEligibleToImport: 1, live: 0, dsoLocal: 1, canAutoHide: 0), varFlags: (readonly: 1, writeonly: 0, constant: 1), refs: (^3, ^2)))) ; guid = 5226103449856009057
+^2 = gv: (name: "_ZN5Child3getEv", summaries: (function: (module: ^0, flags: (linkage: available_externally, visibility: default, notEligibleToImport: 1, live: 0, dsoLocal: 1, canAutoHide: 0), insts: 1, funcFlags: (readNone: 1, readOnly: 0, noRecurse: 1, returnDoesNotAlias: 0, noInline: 0, alwaysInline: 0)))) ; guid = 7125659195073751167
+^3 = gv: (name: "_ZTI5Child") ; guid = 12661005278488252785
+^4 = flags: 8
+^5 = blockcount: 1