[CUDA][HIP] Check calling convention based on function target

[yaxunl]

Summary

MSVC header files using vectorcall to differentiate overloaded functions, which
causes failure for AMDGPU target. This is because clang does not check function
calling convention based on function target.

This patch checks calling convention using the proper target info.

[yaxunl]

My last fix is not right. This patch fixes the issue by checking calling convention based on whether it is host or device function.

[tra]

lib/Sema/SemaDeclAttr.cpp

4622–4639

This is rather convoluted.
Perhaps it would be easier to understand if the code is restructured. along these lines:

// Returns a tuple indicating whether we need to check TargetInfo on host/device side of the compilation
bool CheckHost = false, CheckDevice=false;

switch (CudaTarget) { 
    case CFT_HostDevice: 
      CheckHost = true; CheckDevice=true; break;
    case CFT_Host: 
      CheckHost = true; break;
    case CFG_Device: 
    case CFG_Global: 
      CheckDevice=true; break;
  }
} ();
TargetInfo *HostTI = CudaIsDevice ? Aux : &TI;
TargetInfo * DeviceTI = CudaIsDevice ? &TI : Aux;
if (CheckHost && HostTI)
  HostTI->checkCallingConvention(CC);
if (CheckDevice && DeviceTI)
    DeviceTI->checkCallingConvention(CC);

test/SemaCUDA/amdgpu-windows-vectorcall.cu

4

It may be good to add a check where we *would* expect to see the diagnostics.

[yaxunl]

Revised by Artem's comments.

[tra]

test/SemaCUDA/amdgpu-windows-vectorcall.cu

4

It may be good to add a check where we *would* expect to see the diagnostics.

Scratch that. Wrong calling convention attribute will be ignored.

10–11

I don't think we need templates here. We're only making sure that the function attributes are handled correctly.
Can we get by with a regular function declaration or definition?

[yaxunl]

test/SemaCUDA/amdgpu-windows-vectorcall.cu

10–11

Here we want to make sure _Ret (__cdecl _Arg0::*)(_Types) and _Ret (__vectorcall _Arg0::*)(_Types) are different types because calling convention is part of type, otherwise it results in compilation error due to duplicate template instantation. I can come up with a test without template.

[yaxunl]

modify test to use non-template functions.

[yaxunl]

ping

[tra]

LGTM.

[rjmccall]

lib/Sema/SemaDeclAttr.cpp

4620–4647

Please sink these declarations into the CUDA-specific block. Also, please add some comments to explain why different logic is needed for CUDA.

[yaxunl]

lib/Sema/SemaDeclAttr.cpp

4620–4647

will do. thanks.

[yaxunl]

lib/Sema/SemaDeclAttr.cpp

4620–4647

variables A and TI are also used by else part of the if statement, so only variable Aux can be moved in the if part.

[rjmccall]

lib/Sema/SemaDeclAttr.cpp

4620–4647

Yes, sorry, only your new declaration needs to be sunk.

lib/Sema/SemaDeclAttr.cpp

case ParsedAttr::AT_PreserveMost:
     CC = CC_PreserveMost;
     break;
   case ParsedAttr::AT_PreserveAll:
     CC = CC_PreserveAll;
     break;
   default: llvm_unreachable("unexpected attribute kind");
   }
 
+  TargetInfo::CallingConvCheckResult A = TargetInfo::CCCR_OK;
   const TargetInfo &TI = Context.getTargetInfo();
-  TargetInfo::CallingConvCheckResult A = TI.checkCallingConvention(CC);
+  auto *Aux = Context.getAuxTargetInfo();
+  if (LangOpts.CUDA) {
+    auto CudaTarget = IdentifyCUDATarget(FD);
+    bool CheckHost = false, CheckDevice = false;
+    switch (CudaTarget) {
+    case CFT_HostDevice:
+      CheckHost = true;
+      CheckDevice = true;
+      break;
+    case CFT_Host:
+      CheckHost = true;
+      break;
+    case CFT_Device:
+    case CFT_Global:
+      CheckDevice = true;
+      break;
+    case CFT_InvalidTarget:
+      llvm_unreachable("unexpected cuda target");
+    }
+    auto *HostTI = LangOpts.CUDAIsDevice ? Aux : &TI;
+    auto *DeviceTI = LangOpts.CUDAIsDevice ? &TI : Aux;
+    if (CheckHost && HostTI)
+      A = HostTI->checkCallingConvention(CC);
+    if (A == TargetInfo::CCCR_OK && CheckDevice && DeviceTI)
+      A = DeviceTI->checkCallingConvention(CC);
+  } else {
+    A = TI.checkCallingConvention(CC);
+  }
   if (A != TargetInfo::CCCR_OK) {
     if (A == TargetInfo::CCCR_Warning)
       Diag(Attrs.getLoc(), diag::warn_cconv_ignored) << Attrs;
 
     // This convention is not valid for the target. Use the default function or
     // method calling convention.
     bool IsCXXMethod = false, IsVariadic = false;
     if (FD) {

test/SemaCUDA/amdgpu-windows-vectorcall.cu

+// RUN: %clang_cc1 -triple amdgcn-amd-amdhsa -aux-triple x86_64-pc-windows-msvc -fms-compatibility -fcuda-is-device -fsyntax-only -verify %s
+
+__cdecl void hostf1();
+__vectorcall void (*hostf2)() = hostf1; // expected-error {{cannot initialize a variable of type 'void ((*))() __attribute__((vectorcall))' with an lvalue of type 'void () __attribute__((cdecl))'}}