[AMDGPU] Don't constrain callees with inlinehint from inlining on MaxBB check to prevent link failure

[vpykhtin]

Summary

Function bodies marked inline in an opencl source are eliminated but MaxBB check may prevent inlining them leaving undefined references.

[arsenm]

llvm/lib/Target/AMDGPU/AMDGPUInline.cpp

221–222

inlinehint shouldn't be impacting correctness

[scott.linder]

The undefined references in the linker are OK. The user has to provide an external definition, or link against one. I don't think there is anything to do about this in LLVM.

[vpykhtin]

updated description

[rampitec]

LGTM. Not for a correctness, but for the sake of performance. We have been here before, if we restrict inlinehint functions for the compilation speed we are losing too much.
That said, it is also a reasonable w/a for specific issue, although we still need to contact ISV to fix their code.

llvm/lib/Target/AMDGPU/AMDGPUInline.cpp

std::function<AssumptionCache &(Function &)> GetAssumptionCache =
       [this](Function &F) -> AssumptionCache & {
     return ACT->getAssumptionCache(F);
   };
 
   auto IC = llvm::getInlineCost(cast<CallBase>(*CS.getInstruction()), Callee,
                              LocalParams, TTI, GetAssumptionCache, None, PSI,
                              RemarksEnabled ? &ORE : nullptr);
 
-  if (IC && !IC.isAlways()) {
+  // Don't constrain callees with inlinehint as it may result in link failure
+  if (IC && !IC.isAlways() && !Callee->hasFnAttribute(Attribute::InlineHint)) {
     // Single BB does not increase total BB amount, thus subtract 1
     size_t Size = Caller->size() + Callee->size() - 1;
     if (MaxBB && Size > MaxBB)
       return llvm::InlineCost::getNever("max number of bb exceeded");
   }
   return IC;
 }

llvm/test/CodeGen/AMDGPU/inline-maxbb.ll

 ; NOINL-NOT: mul i32
 ; NOINL: call i32
 
 define amdgpu_kernel void @caller(i32 %x) {
   %res = call i32 @callee(i32 %x)
   store volatile i32 %res, i32 addrspace(1)* undef
   ret void
 }
+
+
+; inlinehint
+define i32 @callee_hint(i32 %x) #0 {
+entry:
+  %cc = icmp eq i32 %x, 1
+  br i1 %cc, label %ret_res, label %mulx
+
+mulx:
+  %mul1 = mul i32 %x, %x
+  %mul2 = mul i32 %mul1, %x
+  %mul3 = mul i32 %mul1, %mul2
+  %mul4 = mul i32 %mul3, %mul2
+  %mul5 = mul i32 %mul4, %mul3
+  br label %ret_res
+
+ret_res:
+  %r = phi i32 [ %mul5, %mulx ], [ %x, %entry ]
+  ret i32 %r
+}
+
+; INL-LABEL: @caller_hint
+; NOINL-LABEL: @caller_hint
+; INL: mul i32
+; INL-NOT: call i32
+; NOINL: mul i32
+; NOINL-NOT: call i32
+
+define amdgpu_kernel void @caller_hint(i32 %x) {
+  %res = call i32 @callee_hint(i32 %x)
+  store volatile i32 %res, i32 addrspace(1)* undef
+  ret void
+}
+
+attributes #0 = { inlinehint }