[Inliner] Preserve !prof metadata when converting call to invoke.

[hoy]

Summary

When a callee function is inlined via an invoke instruction, every function call inside the callee, if not an invoke, will be converted to an invoke after cloned to the caller body. I found that during the conversion the !prof metadata was dropped. This in turned caused a cloned indirect call not properly promoted in subsequent passes.

The particular scenario I was investigating was with AutoFDO and thinLTO. In prelink, no ICP was triggered (neither by the sample loader nor PGO ICP), no indirect call was promoted. This is because 1) the particular indirect call did not have inlined samples; and 2) PGO ICP was intentionally disabled. After inlining, the prof metadata was dropped. Then in postlink, PGO ICP jumped in but didn't do anything. Thus the opportunity was missed.

I'm making a simple fix to preserve !prof metadata when converting call to invoke.

[davidxl]

Does the data need to be scaled first?

[wenlei]

In D125249#3501442, @davidxl wrote:

Does the data need to be scaled first?

Scaling is done right after inlining, separately. Here when just converting call to invoke, we can preserved the counts faithfully.

[davidxl]

Should we have a test case to cover the post-inlining scaling. Otherwise looks good.

[hoy]

In D125249#3501442, @davidxl wrote:

Does the data need to be scaled first?

Good point. The scaling should be done earlier. Similarly for !dbg metadata.

[hoy]

In D125249#3501454, @davidxl wrote:

Should we have a test case to cover the post-inlining scaling. Otherwise looks good.

Will do.

[hoy]

Extending test to check scaling.

[davidxl]

lgtm

llvm/lib/Transforms/Utils/Local.cpp

BasicBlock *llvm::changeToInvokeAndSplitBasicBlock(CallInst *CI,
   // as of this time.
 
   InvokeInst *II =
       InvokeInst::Create(CI->getFunctionType(), CI->getCalledOperand(), Split,
                          UnwindEdge, InvokeArgs, OpBundles, CI->getName(), BB);
   II->setDebugLoc(CI->getDebugLoc());
   II->setCallingConv(CI->getCallingConv());
   II->setAttributes(CI->getAttributes());
+  II->setMetadata(LLVMContext::MD_prof, CI->getMetadata(LLVMContext::MD_prof));
 
   if (DTU)
     DTU->applyUpdates({{DominatorTree::Insert, BB, UnwindEdge}});
 
   // Make sure that anything using the call now uses the invoke!  This also
   // updates the CallGraph if present, because it uses a WeakTrackingVH.
   CI->replaceAllUsesWith(II);
 

llvm/test/Transforms/Inline/profile_meta_invoke.ll

+; Make sure that profile metadata is preserved when cloning a call.
+; RUN: opt < %s -passes='require<profile-summary>,cgscc(inline)' -S | FileCheck %s
+
+declare i32 @__gxx_personality_v0(...)
+
+define void @callee(void ()* %func) !prof !15 {
+  call void %func(), !prof !16
+  ret void
+}
+
+define void @caller(void ()* %func) personality i32 (...)* @__gxx_personality_v0 {
+  invoke void @callee(void ()* %func)
+          to label %ret unwind label %lpad, !prof !17
+
+ret:
+  ret void
+
+lpad:
+  %exn = landingpad {i8*, i32}
+          cleanup
+  unreachable
+}
+
+!llvm.module.flags = !{!1}
+!1 = !{i32 1, !"ProfileSummary", !2}
+!2 = !{!3, !4, !5, !6, !7, !8, !9, !10}
+!3 = !{!"ProfileFormat", !"SampleProfile"}
+!4 = !{!"TotalCount", i64 10000}
+!5 = !{!"MaxCount", i64 10}
+!6 = !{!"MaxInternalCount", i64 1}
+!7 = !{!"MaxFunctionCount", i64 2000}
+!8 = !{!"NumCounts", i64 2}
+!9 = !{!"NumFunctions", i64 2}
+!10 = !{!"DetailedSummary", !11}
+!11 = !{!12, !13, !14}
+!12 = !{i32 10000, i64 100, i32 1}
+!13 = !{i32 999000, i64 100, i32 1}
+!14 = !{i32 999999, i64 1, i32 2}
+!15 = !{!"function_entry_count", i64 1000}
+!16 = !{!"VP", i32 0, i64 1000, i64 9191153033785521275, i64 400, i64 -1069303473483922844, i64 600}
+!17 = !{!"branch_weights", i32 500}
+
+; CHECK-LABEL: @caller(
+; CHECK:  invoke void %func()
+; CHECK-NEXT: {{.*}} !prof ![[PROF:[0-9]+]]
+; CHECK: ![[PROF]] = !{!"VP", i32 0, i64 500, i64 9191153033785521275, i64 200, i64 -1069303473483922844, i64 300}