This is an archive of the discontinued LLVM Phabricator instance.

[Clang] Add captures to the instantiation scope of lambda call operators
ClosedPublic

Authored by cor3ntin on Aug 29 2023, 11:10 AM.

Details

Summary

Like concepts checking, a trailing return type of a lambda
in a dependent context may refer to captures in which case
they may need to be rebuilt, so the map of local decl
should include captures.

This patch reveal a pre-existing issue.
this is always recomputed by TreeTransform.

*this (like all captures) only become const
after the parameter list.

However, if try to recompute the value of this (in a parameter)
during template instantiation while determining the type of the call operator,
we will determine it to be const (unless the lambda is mutable).

There is no good way to know at that point that we are in a parameter
or not, the easiest/best solution is to transform the type of this.

Note that doing so break a handful of HLSL tests.
So this is a prototype at this point.

Fixes #65067
Fixes #63675

Diff Detail

Event Timeline

cor3ntin created this revision.Aug 29 2023, 11:10 AM
Herald added a project: Restricted Project. · View Herald TranscriptAug 29 2023, 11:10 AM
cor3ntin requested review of this revision.Aug 29 2023, 11:10 AM
Herald added a project: Restricted Project. · View Herald TranscriptAug 29 2023, 11:10 AM
Herald added a subscriber: cfe-commits. · View Herald Transcript

@beanz The changes to TreeTransform.h cause a lot of HLSL test failures similar to this one:

clang: /home/cor3ntin/dev/compilers/LLVM/llvm-project/clang/lib/AST/ExprClassification.cpp:57: Cl clang::Expr::ClassifyImpl(clang::ASTContext &, clang::SourceLocation *) const: Assertion `isLValue()' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: /home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang -cc1 -internal-isystem /home/cor3ntin/dev/compilers/LLVM/build-release/lib/clang/18/include -nostdsysteminc -triple dxil-pc-shadermodel6.0-compute -emit-llvm -o - -O0 /home/cor3ntin/dev/compilers/LLVM/llvm-project/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl
1.      <eof> parser at end of file
2.      instantiating function definition 'hlsl::RWBuffer<int>::RWBuffer'
 #0 0x00005562645f33c7 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x3d663c7)
 #1 0x00005562645f0f9e llvm::sys::RunSignalHandlers() (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x3d63f9e)
 #2 0x00005562645f3bba SignalHandler(int) Signals.cpp:0:0
 #3 0x00007fc625c3c4b0 (/lib/x86_64-linux-gnu/libc.so.6+0x3c4b0)
 #4 0x00007fc625c90ffb __pthread_kill_implementation ./nptl/pthread_kill.c:44:76
 #5 0x00007fc625c90ffb __pthread_kill_internal ./nptl/pthread_kill.c:78:10
 #6 0x00007fc625c90ffb pthread_kill ./nptl/pthread_kill.c:89:10
 #7 0x00007fc625c3c406 raise ./signal/../sysdeps/posix/raise.c:27:6
 #8 0x00007fc625c2287c abort ./stdlib/abort.c:81:7
 #9 0x00007fc625c2279b _nl_load_domain ./intl/loadmsgcat.c:1177:9
#10 0x00007fc625c33b86 (/lib/x86_64-linux-gnu/libc.so.6+0x33b86)
#11 0x0000556267a3f1a9 clang::Expr::ClassifyImpl(clang::ASTContext&, clang::SourceLocation*) const (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x71b21a9)
#12 0x0000556267a3fa37 clang::Expr::isModifiableLvalue(clang::ASTContext&, clang::SourceLocation*) const (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x71b2a37)
#13 0x0000556266ffbd1a CheckForModifiableLvalue(clang::Expr*, clang::SourceLocation, clang::Sema&) SemaExpr.cpp:0:0
#14 0x0000556266ffb4b4 clang::Sema::CheckAssignmentOperands(clang::Expr*, clang::ActionResult<clang::Expr*, true>&, clang::SourceLocation, clang::QualType, clang::BinaryOperatorKind) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x676e4b4)
#15 0x0000556266fddd2e clang::Sema::CreateBuiltinBinOp(clang::SourceLocation, clang::BinaryOperatorKind, clang::Expr*, clang::Expr*) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x6750d2e)
#16 0x0000556267572779 clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformBinaryOperator(clang::BinaryOperator*) SemaTemplateInstantiate.cpp:0:0
#17 0x000055626756ceb5 clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformStmt(clang::Stmt*, clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::StmtDiscardKind) SemaTemplateInstantiate.cpp:0:0
#18 0x000055626758c4ba clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformCompoundStmt(clang::CompoundStmt*, bool) SemaTemplateInstantiate.cpp:0:0
#19 0x000055626756ce43 clang::Sema::SubstStmt(clang::Stmt*, clang::MultiLevelTemplateArgumentList const&) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x6cdfe43)
#20 0x00005562675c6bab clang::Sema::InstantiateFunctionDefinition(clang::SourceLocation, clang::FunctionDecl*, bool, bool, bool) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x6d39bab)
#21 0x00005562675c9cbe clang::Sema::PerformPendingInstantiations(bool) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x6d3ccbe)
#22 0x0000556266bf08a5 clang::Sema::ActOnEndOfTranslationUnitFragment(clang::Sema::TUFragmentKind) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x63638a5)
#23 0x0000556266bf1028 clang::Sema::ActOnEndOfTranslationUnit() (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x6364028)
#24 0x0000556266ab548c clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, clang::Sema::ModuleImportState&) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x622848c)
#25 0x0000556266ab007e clang::ParseAST(clang::Sema&, bool, bool) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x622307e)
#26 0x000055626519ac60 clang::FrontendAction::Execute() (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x490dc60)
#27 0x000055626510c96f clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x487f96f)
#28 0x000055626527cc5b clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x49efc5b)
#29 0x000055626341cf64 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x2b8ff64)
#30 0x0000556263419e3d ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&, llvm::ToolContext const&) driver.cpp:0:0
#31 0x0000556263418b9e clang_main(int, char**, llvm::ToolContext const&) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x2b8bb9e)
#32 0x00005562634298d1 main (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x2b9c8d1)
#33 0x00007fc625c23a90 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#34 0x00007fc625c23b49 call_init ./csu/../csu/libc-start.c:128:20
#35 0x00007fc625c23b49 __libc_start_main ./csu/../csu/libc-start.c:347:5
#36 0x0000556263416325 _start (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x2b89325)
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /home/cor3ntin/dev/compilers/LLVM/build-release/bin/FileCheck /home/cor3ntin/dev/compilers/LLVM/llvm-project/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl

any idea would be appreciated! Thanks

@beanz The changes to TreeTransform.h cause a lot of HLSL test failures similar to this one:

clang: /home/cor3ntin/dev/compilers/LLVM/llvm-project/clang/lib/AST/ExprClassification.cpp:57: Cl clang::Expr::ClassifyImpl(clang::ASTContext &, clang::SourceLocation *) const: Assertion `isLValue()' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.
Stack dump:
0.      Program arguments: /home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang -cc1 -internal-isystem /home/cor3ntin/dev/compilers/LLVM/build-release/lib/clang/18/include -nostdsysteminc -triple dxil-pc-shadermodel6.0-compute -emit-llvm -o - -O0 /home/cor3ntin/dev/compilers/LLVM/llvm-project/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl
1.      <eof> parser at end of file
2.      instantiating function definition 'hlsl::RWBuffer<int>::RWBuffer'
 #0 0x00005562645f33c7 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x3d663c7)
 #1 0x00005562645f0f9e llvm::sys::RunSignalHandlers() (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x3d63f9e)
 #2 0x00005562645f3bba SignalHandler(int) Signals.cpp:0:0
 #3 0x00007fc625c3c4b0 (/lib/x86_64-linux-gnu/libc.so.6+0x3c4b0)
 #4 0x00007fc625c90ffb __pthread_kill_implementation ./nptl/pthread_kill.c:44:76
 #5 0x00007fc625c90ffb __pthread_kill_internal ./nptl/pthread_kill.c:78:10
 #6 0x00007fc625c90ffb pthread_kill ./nptl/pthread_kill.c:89:10
 #7 0x00007fc625c3c406 raise ./signal/../sysdeps/posix/raise.c:27:6
 #8 0x00007fc625c2287c abort ./stdlib/abort.c:81:7
 #9 0x00007fc625c2279b _nl_load_domain ./intl/loadmsgcat.c:1177:9
#10 0x00007fc625c33b86 (/lib/x86_64-linux-gnu/libc.so.6+0x33b86)
#11 0x0000556267a3f1a9 clang::Expr::ClassifyImpl(clang::ASTContext&, clang::SourceLocation*) const (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x71b21a9)
#12 0x0000556267a3fa37 clang::Expr::isModifiableLvalue(clang::ASTContext&, clang::SourceLocation*) const (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x71b2a37)
#13 0x0000556266ffbd1a CheckForModifiableLvalue(clang::Expr*, clang::SourceLocation, clang::Sema&) SemaExpr.cpp:0:0
#14 0x0000556266ffb4b4 clang::Sema::CheckAssignmentOperands(clang::Expr*, clang::ActionResult<clang::Expr*, true>&, clang::SourceLocation, clang::QualType, clang::BinaryOperatorKind) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x676e4b4)
#15 0x0000556266fddd2e clang::Sema::CreateBuiltinBinOp(clang::SourceLocation, clang::BinaryOperatorKind, clang::Expr*, clang::Expr*) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x6750d2e)
#16 0x0000556267572779 clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformBinaryOperator(clang::BinaryOperator*) SemaTemplateInstantiate.cpp:0:0
#17 0x000055626756ceb5 clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformStmt(clang::Stmt*, clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::StmtDiscardKind) SemaTemplateInstantiate.cpp:0:0
#18 0x000055626758c4ba clang::TreeTransform<(anonymous namespace)::TemplateInstantiator>::TransformCompoundStmt(clang::CompoundStmt*, bool) SemaTemplateInstantiate.cpp:0:0
#19 0x000055626756ce43 clang::Sema::SubstStmt(clang::Stmt*, clang::MultiLevelTemplateArgumentList const&) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x6cdfe43)
#20 0x00005562675c6bab clang::Sema::InstantiateFunctionDefinition(clang::SourceLocation, clang::FunctionDecl*, bool, bool, bool) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x6d39bab)
#21 0x00005562675c9cbe clang::Sema::PerformPendingInstantiations(bool) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x6d3ccbe)
#22 0x0000556266bf08a5 clang::Sema::ActOnEndOfTranslationUnitFragment(clang::Sema::TUFragmentKind) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x63638a5)
#23 0x0000556266bf1028 clang::Sema::ActOnEndOfTranslationUnit() (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x6364028)
#24 0x0000556266ab548c clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, clang::Sema::ModuleImportState&) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x622848c)
#25 0x0000556266ab007e clang::ParseAST(clang::Sema&, bool, bool) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x622307e)
#26 0x000055626519ac60 clang::FrontendAction::Execute() (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x490dc60)
#27 0x000055626510c96f clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x487f96f)
#28 0x000055626527cc5b clang::ExecuteCompilerInvocation(clang::CompilerInstance*) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x49efc5b)
#29 0x000055626341cf64 cc1_main(llvm::ArrayRef<char const*>, char const*, void*) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x2b8ff64)
#30 0x0000556263419e3d ExecuteCC1Tool(llvm::SmallVectorImpl<char const*>&, llvm::ToolContext const&) driver.cpp:0:0
#31 0x0000556263418b9e clang_main(int, char**, llvm::ToolContext const&) (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x2b8bb9e)
#32 0x00005562634298d1 main (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x2b9c8d1)
#33 0x00007fc625c23a90 __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#34 0x00007fc625c23b49 call_init ./csu/../csu/libc-start.c:128:20
#35 0x00007fc625c23b49 __libc_start_main ./csu/../csu/libc-start.c:347:5
#36 0x0000556263416325 _start (/home/cor3ntin/dev/compilers/LLVM/build-release/bin/clang+0x2b89325)
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /home/cor3ntin/dev/compilers/LLVM/build-release/bin/FileCheck /home/cor3ntin/dev/compilers/LLVM/llvm-project/clang/test/CodeGenHLSL/builtins/RWBuffer-subscript.hlsl

any idea would be appreciated! Thanks

CC @bogner as well

I'll apply this patch and debug the issue today.

beanz added a comment.Aug 29 2023, 2:14 PM

@cor3ntin, I know what the problem is and I think I can put up a review for a fix tonight or (more likely) tomorrow. Is that okay?

@beanz Oh wow, excellent, i did not expect a solution so fast! I'm glad to know there may indeed be an issue with HLSL rather than this patch because I'm not sure I could have come up with a different fix!

beanz added a comment.Aug 29 2023, 2:21 PM

Yea, the gist of it is that in HLSL this is a reference not a pointer, which means the CXXThisExpr is always an LValue.

I think the right fix for this is to cleanup the CXXThisExpr creation code and create a CXXThisExpr::Create method like the other AST nodes. Then I can have a simpler way to force the CXXThisExpr to always be an LValue for HLSL.

@aaron.ballman, does that sound reasonable?

cor3ntin updated this revision to Diff 555974.Sep 6 2023, 12:22 AM

Rebase and cleanups

@erichkeane @aaron.ballman This now pass the tests and is ready for review :)
Thanks to @beanz for the quick turnaround on the HLSL issue

1 suggestion, else LGTM.

clang/lib/Sema/SemaLambda.cpp
2263

Wonder if LambdaScopeForCallOperatorInstantiationRAII should just inherit from `FunctionScopeRAII? Am I missing why not?

cor3ntin added inline comments.Sep 6 2023, 8:15 AM
clang/lib/Sema/SemaLambda.cpp
2263

I suppose it could yes. I think it would have to be protected so that disable is not misused

erichkeane accepted this revision.Sep 6 2023, 8:36 AM
erichkeane added inline comments.
clang/lib/Sema/SemaLambda.cpp
2263

Ah, this isn't the object I thought it was (the actual scope), just the RAII, so I think this is all right/not much benefit to it.

I was afraid it was the actual 'scope' object, so introspection into the list of active scopes would pull us into some private variable/init somewhere rather than the ctor of the containing object. No reason to change this now.

This revision is now accepted and ready to land.Sep 6 2023, 8:36 AM
cor3ntin updated this revision to Diff 556068.Sep 6 2023, 12:40 PM

Inherit from FunctionScopeRAII as per @erichkeane

cor3ntin added inline comments.Sep 6 2023, 12:46 PM
clang/lib/Sema/SemaLambda.cpp
2263

Oups, I missed your reply and changed it. Hopefully you are still happy with it!

erichkeane added inline comments.Sep 6 2023, 12:47 PM
clang/lib/Sema/SemaLambda.cpp
2263

Yep, no problem with it now, a bit of a brainfart when I suggested it in the first place. Still LGTM.

This revision was landed with ongoing or failed builds.Sep 6 2023, 1:00 PM
This revision was automatically updated to reflect the committed changes.

I'm seeing libcxx failures after this patch. See https://ci.chromium.org/ui/p/fuchsia/builders/toolchain.ci/clang-linux-x64/b8770673168839783889/overview

# .---command stderr------------
# | /b/s/w/ir/x/w/llvm-llvm-project/libcxx/test/std/ranges/range.adaptors/range.all/range.owning.view/empty.pass.cpp:38:19: error: static assertion failed due to requirement '!HasEmpty<const std::ranges::owning_view<ComparableIters> &>'
# |    38 |     static_assert(!HasEmpty<const OwningView&>);
# |       |                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | /b/s/w/ir/x/w/llvm-llvm-project/libcxx/test/std/ranges/range.adaptors/range.all/range.owning.view/empty.pass.cpp:39:19: error: static assertion failed due to requirement '!HasEmpty<const std::ranges::owning_view<ComparableIters> &&>'
# |    39 |     static_assert(!HasEmpty<const OwningView&&>);
# |       |                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# | 2 errors generated.
# `-----------------------------

I've verified that reverting this patch fixes those errors. Would you mind taking a look?

@abrachet Thanks, i've reverted the patch while i investigate a fix

I managed to reduced it to

template <class b>
concept cs = requires(b bj) {
  bj.begin();
};
struct {
  template <class b>
  requires cs<b>
  auto operator()(b &&) {}
} begin;
template <class>
concept cu = requires {
  begin;
};
template <class b>
concept cy = requires(b bj) {
  begin(bj);
};
struct {
  template <cy b> void operator()(b &&);
} cz;
template <cu d> class dc {
  d dd;

public:
  void cz() const requires requires { cz(dd); };
};

template <class de>
concept e = requires(de f) {
  f.cz();
};
void g() {
  struct dg {
    void begin();
  };
  using dh = dc<dg>;
  static_assert(!e<dh>);
}

The changes to clang/lib/Sema/TreeTransform.h are what cause the issue

cor3ntin reopened this revision.Sep 7 2023, 12:54 AM
This revision is now accepted and ready to land.Sep 7 2023, 12:54 AM
cor3ntin updated this revision to Diff 556115.Sep 7 2023, 1:37 AM

Fix libc++ compile

@erichkeane I struggle to reduce further, do you think it's worth adding this inscrutable
test case somewhere?

Fix libc++ compile

@erichkeane I struggle to reduce further, do you think it's worth adding this inscrutable
test case somewhere?

There's definitely value to putting it in somewhere, I'd suggest giving meaningful names to the identifiers however.

cor3ntin updated this revision to Diff 556209.Sep 7 2023, 2:42 PM

Add the test case that caused a build failure

@erichkeane Ok, I added a test!

erichkeane accepted this revision.Sep 8 2023, 6:34 AM
This revision was landed with ongoing or failed builds.Sep 8 2023, 8:50 AM
This revision was automatically updated to reflect the committed changes.
thakis added a subscriber: thakis.Sep 8 2023, 9:00 AM

Looks like the test is missing a RUN line and hence makes lit fail: http://45.33.8.238/linux/117631/step_7.txt

Please take a look and revert for now if it takes a while to fix.

Looks like the test is missing a RUN line and hence makes lit fail: http://45.33.8.238/linux/117631/step_7.txt

Please take a look and revert for now if it takes a while to fix.

Won't take a while, it is a pretty simple change, I'll work on it now.

Looks like the test is missing a RUN line and hence makes lit fail: http://45.33.8.238/linux/117631/step_7.txt

Please take a look and revert for now if it takes a while to fix.

Won't take a while, it is a pretty simple change, I'll work on it now.

Should be fixed in 390b48675be80420f471bd3be74577495b1b1897. We were ALSO missing a 'expected-no-diagnostics' line