revised by Artem's comments

refactor CUDACheckLambdaCapture and add comments

LGTM. Thanks.

revised by John's comments.

revised by Artem's and Paul's comments

Added diagnostics for capturing host variables on device lambda and made lambda host device by default.

I think the point of -ffp-contract=fast is that it allows emitting fma across statements if there are such opportunities whereas -ffp-contract=on only allows fma in one statement

Would you please add the following lit test

return the unified features.

Fix issue about -mattr. Also use generic LTO option translation and use -plugin-opt.

Only make non-capturing lambda host and device by default.

LGTM. Thanks.

In D78655#2107016, @tra wrote:

In D78655#2105058, @yaxunl wrote:

• lambdas with any lambda-capture (which must therefore have an enclosing function) inherit the enclosing function's HDness.

Nit: *any* capture does not necessarily imply existence of the enclosing function. One can explicitly capture variables in the global scope. E.g.:

int x;
auto l = [x=x](){ return x;};

In D81176#2105944, @zhuhan0 wrote:

This broke a test clang/test/Tooling/clang-check-offload.cpp for a critical Linux distro at Facebook. With this change, the test adds a -include __clang_hip_runtime_wrapper argument. The wrapper includes some standard c++ headers, but our distro don't have those headers in the default include paths, thus causing a break.

I notice this behavior doesn't happen for CUDA tests, which also rely on a similar __clang_cuda_runtime_wrapper. I think what's causing the difference is the different handling of nogpuinc/nogpulib option. My knowledge on this area is limited, so correct me if I'm wrong. CUDA seems to respect nogpuinc and doesn't include its wrapper if the flag is provided: https://github.com/llvm/llvm-project/blob/master/clang/lib/Driver/ToolChains/Cuda.cpp#L255. But based on this change, HIP does things differently: https://github.com/llvm/llvm-project/blob/master/clang/lib/Driver/ToolChains/AMDGPU.cpp#L226.

If I modify RocmInstallationDetector::AddHIPIncludeArgs to also respect nogpuinc/nogpulib, the test will pass for us. Is it a mistake for HIP to always include the wrapper file? Could you provide a fix for this issue? Thanks!

In D78655#2019290, @rsmith wrote:

There are two behaviors that seem to make sense:

• Treat lambdas as implicitly HD (like constexpr functions) in all CUDA / HIP language modes. I don't think it makes sense for lambdas to become implicitly HD in C++17 simply because they become implicitly constexpr, nor for their HDness to depend on whether their parameter types happen to be literal types, etc. So in C++17, where lambdas are constexpr whenever they can be, the logical behavior would seem to be that lambdas are implicitly HD. And then for consistency with that, I'd expect them to be implicitly HD across all language modes.
• Implicitly give lambdas the same HD-ness as the enclosing function (if there is one).
  
  I would think the best choice may be to do both of these things: if there is an enclosing function, inherit its host or device attributes. And if not, then treat the lambda as implicitly HD. A slight variation on that, that might be better: lambdas with no lambda-capture are implicitly HD; lambdas with any lambda-capture (which must therefore have an enclosing function) inherit the enclosing function's HDness.
  
  (Note that if we go this way, it makes no difference if there are reference captures, because they're always references on the same "side".)

improve diagnostic message

Revised by Richard's comments.

I will commit this patch since it allows hipcc to use old rocm device library. Will fix other issues later.

The only thing that concerns me is that you called clang-offload-unbundler twice to unbundle a fat binary, one for host, one for device. One call of clang-offload-unbundler is supposed to unbundle everything.

In D81959#2096616, @arsenm wrote:

Isn't the internalization implied by LTO? I thought part of the appeal of LTO is killing this off

fix tests

use -mcpu instead of march in HIP toolchain.

fix typo in tests

In D81627#2090499, @tra wrote:

LGTM. Good to go if @arsenm is OK with fixing -fgpu-rdc in a separate patch.

In D60620#2067134, @tra wrote:

Do you expect users to specify these IDs? How do you see it being used in practice? I think you do need to implement a user-friendly shortcut and expand it to the detailed offload-id internally. I'm fine with allowing explicit offload id as a hidden argument, but I don't think it's suitable for something that will be used by everyone who can't be expected to be aware of all the gory details of particular GPU features.

The good thing about this target id is that it is backward compatible with GPU arch. For common users who are not concerned with specific GPU configurations, they can just use the old GPU arch and nothing changes. This is because GPU arch without features implies default value for these features, which work on all configurations. For advanced users who do need to build for specific GPU configurations, they should already have the knowledge about the name and meaning of these configurations by reading the AMDGPU user guide (http://llvm.org/docs/AMDGPUUsage.html). Therefore a target id in the form of gfx908:xnack+ is not something cryptic to them. On the other hand, an encoded GPU arch like gfx908a is cryptic since it has no meaning at all.

In D81713#2089672, @arsenm wrote:

Can you add tests for this? Is this also sufficient with the directory layout change?

revised by Artem's comments.

In D81627#2088502, @arsenm wrote:

In D81627#2088500, @arsenm wrote:

It doesn't matter if we don't support isa linking. We should just use clang and default to -flto. LTO "just works" as is

This is a step forward, but the lack of ISA linking shouldn't block eliminating the use of llc/opt

LGTM. thanks

In D81176#2077940, @thakis wrote:

One more example failure: http://lab.llvm.org:8011/builders/clang-x64-windows-msvc/builds/16333

(Note I had re-reverted your reland, see the 2 comments above your comment.)

LGTM

In D81176#2077333, @yaxunl wrote:

In D81176#2077319, @thakis wrote:

Still broken: http://45.33.8.238/win/16976/summary.html

reverted.

The difficulty for fixing these failures is that they only show up on some systems I do not have access and cannot see the complete output.

In D81176#2077319, @thakis wrote:

Still broken: http://45.33.8.238/win/16976/summary.html

In D81176#2077254, @thakis wrote:

Also still failing on mac: http://45.33.8.238/mac/14992/step_7.txt

@echristo Aaron has addressed most of the comments. The major remaining issue is that the general linker
calls AddGenerateObjFileFromHIPFatBinary, which is supposed to be part of HIP
toolchain. This issue is not new, since the existing gnutools::Linker::ConstructJob
already did the same thing. Is it OK to land this patch and leave the refactoring of
AddGenerateObjFileFromHIPFatBinary to a separate patch? Thanks.

In D80450#2071696, @tra wrote:

In D80450#2055463, @tra wrote:

Is this patch supposed to be used with D79526 or instead of it?

^^^ I don't think this has been answered. I would like to test this change before it lands.

ping