[Clang][OpenMP] Fix the issue that temp cubin files are not removed after compilation when using new OpenMP driver

[tianshilei1992]

Summary

The root cause of this is, in NVPTX::Assembler::ConstructJob, the output file name might not match the Output's file name passed into the function because CudaToolChain::getInputFilename is a specialized version. That means the real output file is not added to the temp files list, which will be all removed in the d'tor of Compilation. In order to "fix" it, in the function NVPTX::OpenMPLinker::ConstructJob, before calling clang-nvlink-wrapper, the function calls getToolChain().getInputFilename(II) to get the right output file name for each input, and add it to temp file, and then they can be removed w/o any issue. However, this whole logic doesn't work when using the new OpenMP driver because NVPTX::OpenMPLinker::ConstructJob is not called at all, which causing the issue that the cubin file generated in each single unit compilation is out of track.

In this patch, we add the real output file into temp files if its name doesn't match Output. We add it when the file is an output instead of doing it when it is an input, like what we did in NVPTX::OpenMPLinker::ConstructJob, which makes more sense.

[jhuber6]

Is this really the cause? nvptx::assemble should call createOutputFile which makes a temp file to output to that is added to TempFiles.

[tianshilei1992]

In D124253#3467375, @jhuber6 wrote:

Is this really the cause? nvptx::assemble should call createOutputFile which makes a temp file to output to that is added to TempFiles.

Hmm, that's interesting. After adding that line, the temp file gets removed. Let me check that function.

[tianshilei1992]

The issue is in clang/lib/Driver/ToolChains/Cuda.cpp.

[tianshilei1992]

update

[jhuber6]

LG

clang/lib/Driver/ToolChains/Cuda.cpp

void NVPTX::Assembler::ConstructJob(Compilation &C, const JobAction &JA,
 
   // Pass -v to ptxas if it was passed to the driver.
   if (Args.hasArg(options::OPT_v))
     CmdArgs.push_back("-v");
 
   CmdArgs.push_back("--gpu-name");
   CmdArgs.push_back(Args.MakeArgString(CudaArchToString(gpu_arch)));
   CmdArgs.push_back("--output-file");
-  CmdArgs.push_back(Args.MakeArgString(TC.getInputFilename(Output)));
+  const char *OutputFileName = Args.MakeArgString(TC.getInputFilename(Output));
+  if (std::string(OutputFileName) != std::string(Output.getFilename()))
+    C.addTempFile(OutputFileName);
+  CmdArgs.push_back(OutputFileName);
   for (const auto& II : Inputs)
     CmdArgs.push_back(Args.MakeArgString(II.getFilename()));
 
   for (const auto& A : Args.getAllArgValues(options::OPT_Xcuda_ptxas))
     CmdArgs.push_back(Args.MakeArgString(A));
 
   bool Relocatable = false;
   if (JA.isOffloading(Action::OFK_OpenMP))
if (II.getType() == types::TY_LLVM_IR ||
       continue;
     }
 
     // Currently, we only pass the input files to the linker, we do not pass
     // any libraries that may be valid only for the host.
     if (!II.isFilename())
       continue;
 
-    const char *CubinF = C.addTempFile(
-        C.getArgs().MakeArgString(getToolChain().getInputFilename(II)));
+    const char *CubinF =
+        C.getArgs().MakeArgString(getToolChain().getInputFilename(II));
 
     CmdArgs.push_back(CubinF);
   }
 
   AddStaticDeviceLibsLinking(C, *this, JA, Inputs, Args, CmdArgs, "nvptx",
                              GPUArch, /*isBitCodeSDL=*/false,
                              /*postClangLink=*/false);