[AMDGPU] Add an option to disable manual ctor / dtor lowering

[jhuber6]

Summary

Currently AMDGPU offers extra ctor / dtor lowering by emitting a kernel
that can be called. It's possible to handle ctors and dtors using the
standard method as shown in D149340's commit message. In which case we
on't need these extra kernels as they won't be called. This patch simply
adds a way to conditionally turn off this handling if we do not want to
get extra kernels in the output.

Unrelated, but we could convert this handling to an ODR function that simply
calls the code in D149340 constructed via LLVM-IR. That would handle priority
correctly and would then be correct if not run in LTO mode.

[arsenm]

Typo "on't" in commit message.

What's the long term plan here? I'd prefer to move towards something that acts like a consistent platform rather than sometimes lowering these based on a debug flag. What would be setting this?

[jhuber6]

In D150565#4346193, @arsenm wrote:

Typo "on't" in commit message.

What's the long term plan here? I'd prefer to move towards something that acts like a consistent platform rather than sometimes lowering these based on a debug flag. What would be setting this?

I'm currently working on a patch to make this more common at least so we simply traverse the init / fini sections inside this kernel call. The reason for this patch is simply that this method of construction requires a kernel call while the libc project I'm working on chooses to perform this step manually. So I would simply like the idea to optionally disable the extra handling if it's not needed.

[jhuber6]

ping

[yaxunl]

LGTM

llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp

EnableEarlyIfConversion("amdgpu-early-ifcvt", cl::Hidden,
                         cl::desc("Run early if-conversion"),
                         cl::init(false));
 
 static cl::opt<bool>
 OptExecMaskPreRA("amdgpu-opt-exec-mask-pre-ra", cl::Hidden,
             cl::desc("Run pre-RA exec mask optimizations"),
             cl::init(true));
 
+static cl::opt<bool>
+    LowerCtorDtor("amdgpu-lower-global-ctor-dtor",
+                  cl::desc("Lower GPU ctor / dtors to globals on the device."),
+                  cl::init(true), cl::Hidden);
+
 // Option to disable vectorizer for tests.
 static cl::opt<bool> EnableLoadStoreVectorizer(
   "amdgpu-load-store-vectorizer",
   cl::desc("Enable load store vectorizer"),
   cl::init(true),
   cl::Hidden);
 
 // Option to control global loads scalarization
void AMDGPUPassConfig::addIRPasses() {
   const AMDGPUTargetMachine &TM = getAMDGPUTargetMachine();
 
   // There is no reason to run these.
   disablePass(&StackMapLivenessID);
   disablePass(&FuncletLayoutID);
   disablePass(&PatchableFunctionID);
 
   addPass(createAMDGPUPrintfRuntimeBinding());
+  if (LowerCtorDtor)
     addPass(createAMDGPUCtorDtorLoweringLegacyPass());
 
   // A call to propagate attributes pass in the backend in case opt was not run.
   addPass(createAMDGPUPropagateAttributesEarlyPass(&TM));
 
   addPass(createAMDGPULowerIntrinsicsPass());
 
   // Function calls are not supported, so make sure we inline everything.
   addPass(createAMDGPUAlwaysInlinePass());

llvm/test/CodeGen/AMDGPU/lower-ctor-dtor.ll

 ; RUN: opt -S -mtriple=amdgcn-- -amdgpu-lower-ctor-dtor < %s | FileCheck %s
 ; RUN: opt -S -mtriple=amdgcn-- -passes=amdgpu-lower-ctor-dtor < %s | FileCheck %s
 
 ; Make sure we get the same result if we run multiple times
 ; RUN: opt -S -mtriple=amdgcn-- -passes=amdgpu-lower-ctor-dtor,amdgpu-lower-ctor-dtor < %s | FileCheck %s
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx700 -filetype=obj -o - < %s | llvm-readelf -s - 2>&1 | FileCheck %s -check-prefix=VISIBILITY
 ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx700 -filetype=obj -o - < %s | llvm-readelf -S - 2>&1 | FileCheck %s -check-prefix=SECTION
+; RUN: llc -mtriple=amdgcn-amd-amdhsa -amdgpu-lower-global-ctor-dtor=0 -mcpu=gfx700 -filetype=obj -o - < %s | llvm-readelf -s - 2>&1 | FileCheck %s -check-prefix=DISABLED
 
 @llvm.global_ctors = appending addrspace(1) global [1 x { i32, ptr, ptr }] [{ i32, ptr, ptr } { i32 1, ptr @foo, ptr null }]
 @llvm.global_dtors = appending addrspace(1) global [1 x { i32, ptr, ptr }] [{ i32, ptr, ptr } { i32 1, ptr @bar, ptr null }]
 
 ; CHECK: @llvm.used = appending global [2 x ptr] [ptr @amdgcn.device.init, ptr @amdgcn.device.fini]
 
 ; CHECK-LABEL: amdgpu_kernel void @amdgcn.device.init() #0
 ; CHECK-NEXT: call void @foo
 ; CHECK-NEXT: ret void
 
 ; CHECK-LABEL: amdgpu_kernel void @amdgcn.device.fini() #1
 ; CHECK-NEXT: call void @bar
 ; CHECK-NEXT: ret void
 
 ; CHECK-NOT: amdgcn.device.
 
 ; VISIBILITY: FUNC   GLOBAL PROTECTED {{.*}} amdgcn.device.init
 ; VISIBILITY: OBJECT GLOBAL DEFAULT {{.*}} amdgcn.device.init.kd
 ; VISIBILITY: FUNC   GLOBAL PROTECTED {{.*}} amdgcn.device.fini
 ; VISIBILITY: OBJECT   GLOBAL DEFAULT {{.*}} amdgcn.device.fini.kd
 ; SECTION: .init_array.1     INIT_ARRAY      {{.*}} {{.*}} 000008 00  WA  0   0  8
 ; SECTION: .fini_array.1     FINI_ARRAY      {{.*}} {{.*}} 000008 00  WA  0   0  8
+; DISABLED-NOT: FUNC   GLOBAL PROTECTED {{.*}} amdgcn.device.init
+; DISABLED-NOT: OBJECT GLOBAL DEFAULT {{.*}} amdgcn.device.init.kd
+; DISABLED-NOT: FUNC   GLOBAL PROTECTED {{.*}} amdgcn.device.fini
+; DISABLED-NOT: OBJECT   GLOBAL DEFAULT {{.*}} amdgcn.device.fini.kd
 
 define internal void @foo() {
   ret void
 }
 
 define internal void @bar() {
   ret void
 }
 
 ; CHECK: attributes #0 = { "device-init" }
 ; CHECK: attributes #1 = { "device-fini" }