[clang-repl] Support statements on global scope in incremental mode.

[v.g.vassilev]

Summary

[clang-repl] Support statements on global scope in incremental mode.

This patch teaches clang to parse statements on the global scope to allow:

 ./bin/clang-repl
clang-repl> int i = 12;
clang-repl> ++i;
clang-repl> extern "C" int printf(const char*,...);
clang-repl> printf("%d\n", i);
13
clang-repl> %quit

Generally, disambiguating between statements and declarations is a non-trivial task for a C++ parser. The challenge is to allow both standard C++ to be translated as if this patch does not exist and in the cases where the user typed a statement to be executed as if it were in a function body.

Clang's Parser does pretty well in disambiguating between declarations and expressions. We have added DisambiguatingWithExpression flag which allows us to preserve the existing and optimized behavior where needed and implement the extra rules for disambiguating. Only few cases require additional attention:

• Constructors/destructors -- Parser::isConstructorDeclarator was used in to disambiguate between ctor-looking declarations and statements on the global scope(eg. Ns::f()).
• The template keyword -- the template keyword can appear in both declarations and statements. This patch considers the template keyword to be a declaration starter which breaks a few cases in incremental mode which will be tackled later.
• The inline (and similar) keyword -- looking at the first token in many cases allows us to classify what is a declaration.
• Other language keywords and specifiers -- ObjC/ObjC++/OpenCL/OpenMP rely on pragmas or special tokens which will be handled in subsequent patches.

The patch conceptually models a "top-level" statement into a TopLevelStmtDecl. The TopLevelStmtDecl is lowered into a void function with no arguments. We attach this function to the global initializer list to execute the statement blocks in the correct order.

[v.g.vassilev]

Rely on the CXXScopeSpec to detect more accurately if we are parsing a constructor declarator.

[v.g.vassilev]

clang/lib/Parse/ParseDecl.cpp

3434–3436

The only remaining failure is in test/Parser/cxx-class.cpp where we have something similar to:

namespace ctor_error {
  class Ctor { // expected-note{{not complete until the closing '}'}}
    Ctor(f)(int); // ok
    Ctor(g(int)); // ok
  };

  Ctor::Ctor (x) = { 0 }; // \
    // expected-error{{qualified reference to 'Ctor' is a constructor name}}
}

Now, Ctor::Ctor is recognized as a constructor and the diagnostic message regresses to:

error: 'error' diagnostics expected but not seen: 
  File /home/vvassilev/workspace/sources/llvm-project/clang/test/Parser/cxx-class.cpp Line 116: qualified reference to 'Ctor' is a constructor name
error: 'error' diagnostics seen but not expected: 
  File /home/vvassilev/workspace/sources/llvm-project/clang/test/Parser/cxx-class.cpp Line 116: unknown type name 'x'
2 errors generated.

It seems that the more accurate diagnostic used to come from Actions.getTypeName... Any ideas how to fix this?

[v.g.vassilev]

ping...

[v.g.vassilev]

Adding Aaron as a reviewer who hopefully could speed up the review.

[aaron.ballman]

The patch conceptually models the possible "top-level" statements between two top-level declarations into a block which is emitted as part of the global init function.
Currently we model this "statement block" as a function body to a void function taking no arguments. We attach this function to the global initializer list to execute the statement blocks in the correct order.

I think some of the tests have confusing behavior based on this conceptual model because the tests do things like declare namespaces and call functions; you can declare a namespace at file scope, not at function scope; you can't call a function at file scope (unless it's part of an initializer). So it the semantics wind up feeling confused because you can't tell whether you're at file scope or at function scope because you seem to be in both at the same time.

clang/include/clang/AST/ASTConsumer.h

54–59 ↗

(On Diff #449066)

In C and C++, the only thing that's at file scope (that the parser sees) are declarations, not statements, so the phrasing here seems off. I suspect this is because you don't *know* you're only going to get declarations (because it could be that we're trying to parse in the context of an imaginary function body)? If so, perhaps the comments could be updated to clarify that.

clang/include/clang/Parse/Parser.h

468

Drive by punctuation fix. I'm not certain I understand "as that is the only one used", but that's from the original comment. It's possible the comment holds no real weight any longer?

clang/lib/CodeGen/CodeGenModule.cpp

5290–5294

You should fix the other identifiers in this function to match the usual coding style.

clang/lib/Parse/ParseDecl.cpp

5660–5667

A constructor declaration must be a definition, so if we don't see a {, :, or try after the declarator then it's an expression, not a declaration.

Consider:

struct S {
  S();
};

S::S() = default;

The out-of-line constructor definition is still a kind of declaration despite not ending in {, :, or try.

clang/lib/Parse/ParseTentative.cpp

54

http://eel.is/c++draft/dcl.dcl#nt:empty-declaration

60

You should assert this at the top of the file; we shouldn't call into here for C code.

61

Do you need special handling for:

struct S {
  operator int();
};

S::operator int() { return 0; }

clang/lib/Parse/Parser.cpp

737–738

How close are we to having this fixme fixed?

743

clang/test/Interpreter/disambiguate-decl-stmt.cpp

3

Why is this not covered by the REQUIRES above?

29–34

This behavior is very confusing to me -- you cannot declare a namespace at function scope and you cannot call a function at file scope, so one of these two seems like it should not work.

(Same kinds of confusion happen elsewhere in the tests.)

clang/test/Interpreter/execute-stmts.cpp

10–12

Should this code be removed or get a FIXME comment?

[v.g.vassilev]

Address review comments

[v.g.vassilev]

clang/include/clang/AST/ASTConsumer.h

54–59 ↗

(On Diff #449066)

I am a bit confused. Could you clarify what wording changes you had in mind?

clang/include/clang/Parse/Parser.h

468

It does not indeed carry weight. Let me try to improve it.

clang/lib/CodeGen/CodeGenModule.cpp

5290–5294

I have not paid too much attention in the coding style here as it was mostly for initiating a design discussion.

The body of this function breaks the design of clang which assumes AST to be created by the frontend and be essentially immutable in CodeGen. This FunctionDecl needs to be treated as a global initializer. That is, it needs to be run in a particular order to the other initializer following the lexical occurrence of the statement block. That leaves us with two options, either to expose the registration of global initializers outside of CodeGen, or we need to create that here.

Alternatively, we could make HandleTopLevelStmts take a FunctionDecl and we can create it in the IncrementalParser. That blurs the responsibility and coherence of the interfaces. We will have public interface HandleTopLevelStmts (or similar) which handles top-level statements but take a FunctionDecl as a parameter which is created by the user...

clang/lib/Parse/ParseDecl.cpp

5660–5667

That came up a lot in the testsuite but I've added a test for it.

clang/lib/Parse/ParseTentative.cpp

54

IIUC, clang models the ; as a NullStmt.

60

That's a good point.

61

We seem to need that. Thanks. I've added a FIXME.

clang/lib/Parse/Parser.cpp

737–738

2000 tests away. The tests are mostly in OpenCL, OpenMP, etc, which just need adding more entries in places such as isCXXDeclarationSpecifier. I've decided to keep the patch small and then add that once we merge this one.

clang/test/Interpreter/disambiguate-decl-stmt.cpp

3

Good question, @sunho, @lhames do you remember if we still need this exclusion here?

IIRC, there are platforms which support the llvm-jit but with a limited set of features, eg. some platforms do not support reliably exceptions, others relocations (eg some ppc little endian machines).

29–34

The mental model has been that we have a mix of statements and declarations on the global scope. The sequence of statements are only considered to be as if they have been wrapped in a function body. For example:

namespace Ns {namespace Ns { void Ns(); void Fs();}}
void Ns::Ns::Ns() { printf("void Ns::Ns::Ns()\n"); }
void Ns::Ns::Fs() {}
Ns::Ns::Fs();
Ns::Ns::Ns();

should be semantically equivalent to:

namespace Ns {namespace Ns { void Ns(); void Fs();}}
void Ns::Ns::Ns() { printf("void Ns::Ns::Ns()\n"); }
void Ns::Ns::Fs() {}

void stmt_block1() {
  Ns::Ns::Fs();
  Ns::Ns::Ns();
}
auto _ = (stmt_block1(), 1);

Does that help in making it less confusing when thinking about it?

[aaron.ballman]

clang/include/clang/AST/ASTConsumer.h

54–59 ↗

(On Diff #449066)

It wasn't clear to me whether this was misnamed (HandleTopLevelDecls()) or whether the comments didn't make it clear that this is actually "statements and/or declarations and we don't know whether they're at file scope or not."

clang/lib/CodeGen/CodeGenModule.cpp

5290–5294

I have not paid too much attention in the coding style here as it was mostly for initiating a design discussion.

No worries, so long as you run clang-format over the patch before landing, it'll be fine.

either to expose the registration of global initializers outside of CodeGen, or we need to create that here.

Ideally, this should be generated outside of CodeGen. It seems to me that this could perhaps be done once the TU has finished but before codegen has started (in ActOnEndOfTranslationUnit() or ActOnEndOfTranslationUnitFragment())?

clang/lib/Parse/ParseTentative.cpp

54

That depends on where the ; is: https://godbolt.org/z/Kz9zjjKoe

clang/lib/Parse/Parser.cpp

737–738

LOL, okay, so a ways away still. :-D

clang/test/Interpreter/disambiguate-decl-stmt.cpp

29–34

The mental model has been that we have a mix of statements and declarations on the global scope.

I don't understand that mental model because neither C nor C++ allow you to have statements at the global scope. So I'm not certain how to think about statements showing up there (and more importantly, the languages and the compiler wouldn't expect that anyway). For example, consider this:

int n = 12;
int array[n];

This is valid at block scope and invalid at global scope: https://godbolt.org/z/8zxn3h79E

Similarly, consider this code:

template <typename Ty>
void func();

This is valid at file scope but invalid at block scope.

My concern here is that it will be a game of whack-a-mole to identify all of these circumstances and try to get the semantics somewhat close to correct. What is the harm with the REPL always being a full TU, so if the user wants a block scope, they're responsible for writing a function and calling it from main()?

[rsmith]

In terms of the high-level direction here, I think it would make sense to approach this by adding a full-fledged language extension to Clang to allow statements at the top level (with a -f flag to enable it), and then enable that extension in the interpreter. The major change that shift in viewpoint provides is that we should have explicit modeling of these top-level statements in the AST, rather than having them only transiently exist until they get passed off to the AST consumer. I think I'd want to see something like TopLevelStmtDecl added to explicitly model the case where we parse a statement at the top level. You can then extend the various parts of Clang that deal with global variable initializers to also handle TopLevelStmtDecls.

[aaron.ballman]

In D127284#3776036, @rsmith wrote:

In terms of the high-level direction here, I think it would make sense to approach this by adding a full-fledged language extension to Clang to allow statements at the top level (with a -f flag to enable it), and then enable that extension in the interpreter. The major change that shift in viewpoint provides is that we should have explicit modeling of these top-level statements in the AST, rather than having them only transiently exist until they get passed off to the AST consumer. I think I'd want to see something like TopLevelStmtDecl added to explicitly model the case where we parse a statement at the top level. You can then extend the various parts of Clang that deal with global variable initializers to also handle TopLevelStmtDecls.

Thank you for the suggestion! I actually have a different view on this that I was thinking about last night, but it's somewhat similar to yours in that it involves a flag to opt into behavior.

I don't think we should add a feature flag for this to Clang or support this functionality in Clang's AST -- it does not meet our language extension requirements (this won't be proposed to any standards body, etc). Further, I don't think Clang maintainers should have to play cognitive whack-a-mole as new statements and features are added to C and C++, wondering how they should behave if at the top level. Instead, I think it would make sense to add a flag to clang-repl so the user can decide whether they want their REPL experience to be "whole TU" or "pretend we're wrapped in a function". For users who want their experience to be both at the same time: that's an interesting idea, but it is not a C or C++ REPL experience; I think those users can go with the "whole TU" approach and write an extra line of code + some braces, as needed.

The reason I'm so uncomfortable with putting this into Clang is because neither language is designed to expect this sort of behavior and the resulting code will misbehave in mysterious ways when we guess wrong and it will be very difficult to address them all. I expect some things to be easy to recognize (see the template keyword and you know you can't be within a function scope), other things will require a bit of token lookahead (unsigned long long _Thread_local i = 12; cannot appear at local scope), and still others will require looking ahead through a potentially arbitrary number of statements (like with a VLA). I think the user should know explicitly which context they're in given that it matters to the source languages (and this is before we start to think about REPL over something like ObjC/OpenCL/HLSL/etc which may have even more interesting situations to worry about).

[aaron.ballman]

In D127284#3776805, @aaron.ballman wrote:

In D127284#3776036, @rsmith wrote:

In terms of the high-level direction here, I think it would make sense to approach this by adding a full-fledged language extension to Clang to allow statements at the top level (with a -f flag to enable it), and then enable that extension in the interpreter. The major change that shift in viewpoint provides is that we should have explicit modeling of these top-level statements in the AST, rather than having them only transiently exist until they get passed off to the AST consumer. I think I'd want to see something like TopLevelStmtDecl added to explicitly model the case where we parse a statement at the top level. You can then extend the various parts of Clang that deal with global variable initializers to also handle TopLevelStmtDecls.

Thank you for the suggestion! I actually have a different view on this that I was thinking about last night, but it's somewhat similar to yours in that it involves a flag to opt into behavior.

I don't think we should add a feature flag for this to Clang or support this functionality in Clang's AST -- it does not meet our language extension requirements (this won't be proposed to any standards body, etc). Further, I don't think Clang maintainers should have to play cognitive whack-a-mole as new statements and features are added to C and C++, wondering how they should behave if at the top level. Instead, I think it would make sense to add a flag to clang-repl so the user can decide whether they want their REPL experience to be "whole TU" or "pretend we're wrapped in a function". For users who want their experience to be both at the same time: that's an interesting idea, but it is not a C or C++ REPL experience; I think those users can go with the "whole TU" approach and write an extra line of code + some braces, as needed.

The reason I'm so uncomfortable with putting this into Clang is because neither language is designed to expect this sort of behavior and the resulting code will misbehave in mysterious ways when we guess wrong and it will be very difficult to address them all. I expect some things to be easy to recognize (see the template keyword and you know you can't be within a function scope), other things will require a bit of token lookahead (unsigned long long _Thread_local i = 12; cannot appear at local scope), and still others will require looking ahead through a potentially arbitrary number of statements (like with a VLA). I think the user should know explicitly which context they're in given that it matters to the source languages (and this is before we start to think about REPL over something like ObjC/OpenCL/HLSL/etc which may have even more interesting situations to worry about).

I had a really great conversation with @v.g.vassilev off-list about my concerns and the clang-repl needs/desires (thank you for taking the time to have that chat with me!), and this is a summary of what we think makes sense as a way forward:

Technical:

• clang gets a -cc1 (only) flag (or some other kind of not-user-facing option) to enable a special mode where you can mix statements and declarations at TU scope.
• clang-repl uses that new flag.
• clang gets a new AST node for TopLevelStmtDecl that's a Decl AST node wrapping a Stmt node; it's documented as only existing to support REPL. Add an assertion somewhere sensible that we never see one of these AST nodes outside of a clang-repl context.
• clang-repl uses that new AST node as-needed.

Administrative:

• clang-repl continues to push on standardization efforts in WG21 (and potentially WG14).
  • If those efforts lead to an explicit rejection of the idea, we should discuss how to proceed at that time, but I would envision that we'd remove the new AST node and the flag to enable this functionality, and introduce interfaces allowing us to synthesize code during codegen to try to break clang-repl users as little as possible. This way we aren't eating into standards body design space that will potentially give us problems in the future if the committees elect to do something *incompatible* in this space. (Basically: in a fight between clang-repl users and a new standard feature, the new standard feature "wins".) This will break clang-repl users, so we should consider documenting the potential for this up front (basically, call it an experimental feature of clang-repl rather than promising backwards compatibility for it).
• Any failures related to TopLevelStmtDecl will require collaboration between clang-repl and clang maintainers, but should mostly be driven by clang-repl maintainers.

[v.g.vassilev]

Address comments.

[v.g.vassilev]

Rebase.

clang/test/Interpreter/disambiguate-decl-stmt.cpp

29–34

That was clarified in the off-list discussion. I will try to summarize it for having proper history:

Our new facility teaches the parser to recognize statements when they were not declarations. If they were declarations we prefer the language standard. If they were valid statements which are spelled out on the global scope we pretend they were written in block scope, form a function dedicated definition and order it for global initialization.

int n = 12;
int array[n];

is still invalid on global scope.

template <typename Ty>
void func();

is untouched.

[aaron.ballman]

It looks like precommit CI caught a relevant issue that should be addressed.

Also, there's an in-progress patch that might be worth keeping an eye on in case it changes the behavior for clang-repl in bad ways: https://reviews.llvm.org/D137020 -- the patch causes unknown declaration specifiers to be parsed as if they were a type specifier rather than an expression, which could potentially change repl behavior.

clang/include/clang/AST/Decl.h

4239

4242

4255–4260

I think we probably want this to assert(Statements.empty()) rather than silently fail to set the statements.

4258

It's a noop change, but perhaps std::unitialized_copy instead?

4271

Should we add an overload pair here for better const correctness? e.g.,

ArrayRef<const Stmt *> statements() const { return {Stmts, NumStmts}; }
ArrayRef<Stmt *> statements() { return {Stmts, NumStmts}; }

clang/include/clang/Basic/DeclNodes.td

97

Oh boy, that node name is a hoot. It ends in Stmt but it's a Decl. :-D I think it's fine because it matches the convention used for all the other identifiers here, but it may be worth thinking about whether we want a different name for the AST node like TopLevelPseudoStmtDecl or FileScopePseudoStatementDecl. I don't insist on a change here though, just something to consider.

clang/include/clang/Lex/Preprocessor.h

1789–1792

We should be able to drop this as part of this patch, right? (I think you can modify the IncrementalAction object so that it calls CI.getLangOpts().IncrementalExtensions = true; if needed, but you're passing the cc1 flag to the invocation and so I think you can maybe remove this call entirely.)

clang/lib/AST/Decl.cpp

5209

5216

5235–5238

Some renaming for style, but also, this gets a name that's actually a reserved identifier instead of one that the user could potentially be using.

5241–5242

5247

clang/lib/CodeGen/CodeGenModule.cpp

6358

clang/lib/Parse/ParseDecl.cpp

5363–5364

Are you planning on doing this as part of this patch?

clang/lib/Sema/SemaDecl.cpp

19540

Can use auto here since the type is spelled out in the initialization (for some reason Phab won't let me suggest an edit here).

clang/test/Interpreter/disambiguate-decl-stmt.cpp

3

Still wondering on this bit.

[v.g.vassilev]

Address more comments.

[v.g.vassilev]

clang/include/clang/Basic/DeclNodes.td

97

Indeed, I'd put the pseudo to qualify the decl, like TopLevelStmtPseudoDecl but I still don't think either is an improvement to what we have currently.

clang/include/clang/Lex/Preprocessor.h

1789–1792

I wanted to do this is a separate commit. I am worried of breaking downstream users. I remember long time ago @akyrtzi was using this logic.

There are also a bunch of tests in clang and lldb.

clang/lib/CodeGen/CodeGenModule.cpp

6358

I'd like to keep this as the FunctionDecl type makes it move obvious what getOrConvertToFunction returns.

clang/lib/Parse/ParseDecl.cpp

5363–5364

I'd like to do it in a following patch as the patch will mostly touch other languages and would spill this one.

clang/test/Interpreter/disambiguate-decl-stmt.cpp

3

// REQUIRES: host-supports-jit ensures that the system supports spawning a JIT.
// UNSUPPORTED: system-aix means that this test is not supported on AIX.

These options were discovered through a lot of pain and suffering by a lot of reverts. I would prefer to keep UNSUPPORTED line in this patch, and try to undo it in a small and atomic commit later to see if that test can be executed on AIX.

[v.g.vassilev]

In D127284#3902487, @aaron.ballman wrote:

It looks like precommit CI caught a relevant issue that should be addressed.

Yes, I have a comment (starting with "The only remaining failure is in test/Parser/cxx-class.cpp where we have something similar to") asking for help. I have accidentally marked it as resolved but it is not. I will spend some time tomorrow trying to figure out what's the best way forward.

Also, there's an in-progress patch that might be worth keeping an eye on in case it changes the behavior for clang-repl in bad ways: https://reviews.llvm.org/D137020 -- the patch causes unknown declaration specifiers to be parsed as if they were a type specifier rather than an expression, which could potentially change repl behavior.

Thanks for the heads up!

[rsmith]

clang/lib/Parse/ParseDecl.cpp

5660–5685

I don't think any of this should be necessary. Per the function comment, it's a precondition of this function that we're looking at either ClassName or ClassNameScope::ClassName and we shouldn't be re-checking that.

I also think we should not be returning true here if we see ClassName::ClassName without looking at the rest of the tokens. That'll cause diagnostic quality regressions like the one you're seeing. We should continue to the rest of this function to see if we have (Type or () next.

clang/lib/Parse/ParseTentative.cpp

61

I think you should check isCurrentClassName before calling this, like the other callers of this function do. My understanding is that the intent is for isConstructorDeclarator to only check the (potential) function declarator that appears after the name, not to check the name itself.

[v.g.vassilev]

Address comments. Call isCurrentClassName before calling isConstructorDeclarator.

clang/lib/Parse/ParseDecl.cpp

5660–5685

Indeed. I've reverted that and I did not need to have changes in this routine anymore. Thanks!!

clang/lib/Parse/ParseTentative.cpp

61

Ah, okay, that was the missing bit.

[v.g.vassilev]

Remove several fixmes. Now we can deal with templates, deduction guides and operators.

[v.g.vassilev]

clang/lib/Parse/Parser.cpp

1033

There is a remaining challenge which probably could be addressed outside of this patch.

Consider this statement block:

int i =  12;
++i; 
i--;

template<typename T> struct A { };

Ideally we should model ++i; i--; as a single TopLevelStmtDecl as the statement block is contiguous. That would require the creation of 2 AST nodes per block (one for the TopLevelStmtDecl and one for its conversion to FunctionDecl). This will give us also a nice property on the REPL side where the user could decide to squash multiple statements into a statement block to save on memory.

To do so, we will need to use isDeclarationStatement as a stop rule in ParseTopLevelDecl. In turn, this would mean that we should duplicate all of the switch cases described in the ParseExternalDeclaration function here. [We need teach isDeclarationStatement everything we know about declarations, eg. it must tell us to stop when we see definition struct A].

The last version of this patch goes in the opposite direction, trying to minimize the code duplication (bloat?) by wrapping each global statement into a TopLevelStmtDecl, reusing the logic in ParseExternalDeclaration. However, we pay the price for 2 AST node allocations per global statement. That is a serious hit for people that want to control the parsing granularity of an interpreter.

I wonder if we can do something better hitting both requirements in some smart way I cannot see...

[v.g.vassilev]

Rebase + fix the failure on windows

[rsmith]

clang/lib/AST/Decl.cpp

5230

I would hope that we can remove this. Instead, I think we can teach CodeGen to emit a sequence of TopLevelStmtDecls directly as an LLVM IR function -- if it's not emitted anything else nor flushed its IR output since it last emitted a TopLevelStmtDecl, then reuse and extend the previous Function, otherwise create a new one. That would also allow us to make TopLevelStmtDecl model exactly one Stmt, which seems cleaner.

clang/lib/Parse/ParseTentative.cpp

52–53

Can we sink this into the switch on the token kind below?

clang/lib/Parse/Parser.cpp

1033

It seems to me that the big cost here is creating a FunctionDecl and all of its ancillary components; a TopLevelStmtDecl is pretty cheap. I don't think it should be necessary to create that FunctionDecl at all -- we should be able to go straight from TopLevelStmtDecl to an IR function like we go straight from a VarDecl for a global function to its initializer IR function without creating a FunctionDecl.

[v.g.vassilev]

Address comments, support better C in clang-repl.

[v.g.vassilev]

Teach CodeGen to squash contiguous top-level stmt decl blocks into a single llvm::Function. Rebase.

[aaron.ballman]

In general, this looks like an improvement over the last iteration (thank you for the suggestions, @rsmith!). I think this LGTM but there's enough changes in CodeGen that I'd appreciate a second set of eyes on those changes before giving the go-ahead. I've added a few more reviewers to hopefully help with that.

clang/include/clang/AST/Decl.h

4248–4251

Both are now unused.

clang/include/clang/Basic/DeclNodes.td

97

Yeah, I'm "okay" with the name as-is.

clang/include/clang/Lex/Preprocessor.h

1789–1792

I wanted to do this is a separate commit. I am worried of breaking downstream users.

Downstream users have no expectation of this interface remaining stable to begin with, so I'd rather we remove the code unless someone speaks up with a concrete technical problem. That said, I'm fine doing it in a separate commit so that it's easier to raise awareness for downstreams if you think this will be disruptive to them.

clang/lib/AST/Decl.cpp

5215–5216

Might as well make this form of the constructor take a Stmt * instead of doing a two-step initialization.

clang/lib/AST/DeclPrinter.cpp

936–938

getStmt() can return null; should this assert we're not calling it in such a case?

clang/test/Interpreter/disambiguate-decl-stmt.cpp

3

SGTM, thank you for investigating!

[rsmith]

This is looking good to me. No further non-trivial concerns on my side.

clang/lib/CodeGen/CodeGenModule.cpp

6155

Instead of a name comparison, can you check whether CXXGlobalInits.back() == CurCGF->CurFn?

6162

Maybe add a TODO to ask the ABI name mangler to pick a name?

clang/lib/Parse/ParseDecl.cpp

5370

I guess this happens in some pragma situations? Can you put a real diagnostic in here rather than an assert?

clang/lib/Parse/Parser.cpp

1028–1030

Have you tried this with the latest version of the patch? Can the FIXME be removed?

[v.g.vassilev]

I think I have addressed all comments from @aaron.ballman and @rsmith.

clang/include/clang/Lex/Preprocessor.h

1789–1792

I'd prefer doing it in a separate commit. This patch is bulky and we may need to revert it making all bots happy. That'd be probably make downstream consumers green/red for a while and generate a some email traffic ;)

clang/lib/CodeGen/CodeGenModule.cpp

6155

Awesome! Thanks!

clang/lib/Parse/ParseDecl.cpp

5370

That's what I understood from the code. Thanks for clarifying. I added a diagnostic.

clang/lib/Parse/Parser.cpp

1028–1030

I have tried. The failures are a couple of hundred now. However, the failing tests are due to the fact that the tests check for expected diagnostics from ill-formed code. There the decisions if something that's a statement or a declaration are harder and sometimes we produce unexpected diagnostics. I am not sure if we should be fixing the test files.

I will investigate more thoroughly but I have removed the FIXME as the majority of the failures are resolved by the current version of the patch.

[v.g.vassilev]

rebase.

[rsmith]

clang/lib/Parse/Parser.cpp

1028–1030

OK, I think that's reasonable. I think it would be interesting to try building some non-trivial C++ code, perhaps bootstrapping Clang, with -Xclang -fincremental-extensions, as a check for whether there are any cases where we disambiguate a valid top-level declaration as a statement. But I don't think that should be a blocker for landing this patch.

[v.g.vassilev]

clang/lib/Parse/Parser.cpp

1028–1030

I don't want to hold off that patch any longer as well.

I started looking into that. Currently I have decided to track it here https://reviews.llvm.org/D139016 as some of the changes might require further review/discussion.

Parsing things seems fine, I get up to libSupport but then there are some linker failures I need to investigate.

[aaron.ballman]

clang/include/clang/Lex/Preprocessor.h

1789–1792

Doing it as a separate commit is fine by me.

clang/lib/Parse/ParseDecl.cpp

5370–5375

Why is this using a custom diagnostic instead of adding a typical diagnostic to DiagnosticParseKinds.td?

[v.g.vassilev]

clang/lib/Parse/ParseDecl.cpp

5370–5375

I don't have a problem converting this to a DiagnosticParseKind. However, this is a temporary diagnostic and we risk once the FIXME is resolved to leave a stray diagnostic id.

[aaron.ballman]

clang/lib/Parse/ParseDecl.cpp

5370–5375

It's been my experience that there's nothing more permanent than a temporary solution, so I'd say we should go with the regular diagnostic. Also, it should be worded unsupported statement at global scope to fit the usual diagnostic wording style.

[v.g.vassilev]

Address comments, fix a fixme.

clang/lib/Parse/ParseDecl.cpp

5370–5375

I took the liberty to address the FIXME.

[v.g.vassilev]

I screwed up with the last diff. Let me fix it in a bit...

[v.g.vassilev]

Fixed the diff. I accidentally have added the commits from https://reviews.llvm.org/D139016

[aaron.ballman]

LGTM!

[Michael137]

FYI, looks like this broke the LLDB incremental buildbot. Following tests started failing:

lldb-api.commands/expression/argument_passing_restrictions.TestArgumentPassingRestrictions.py
lldb-api.commands/expression/deleting-implicit-copy-constructor.TestDeletingImplicitCopyConstructor.py
lldb-api.commands/expression/diagnostics.TestExprDiagnostics.py
lldb-api.commands/expression/import-std-module/array.TestArrayFromStdModule.py
lldb-api.commands/expression/import-std-module/module-build-errors.TestStdModuleBuildErrors.py
lldb-api.commands/expression/rdar44436068.Test128BitsInteger.py
lldb-api.commands/expression/weak_symbols.TestWeakSymbols.py
lldb-api.commands/target/dump-pcm-info.TestDumpPCMInfo.py
lldb-api.functionalities/data-formatter/compactvectors.TestCompactVectors.py
lldb-api.functionalities/tail_call_frames/cross_object.TestCrossObjectTailCalls.py
lldb-api.functionalities/target_var/no_vars.TestTargetVarNoVars.py
lldb-api.lang/c/modules.TestCModules.py
lldb-api.lang/cpp/accelerator-table.TestCPPAccelerator.py
lldb-api.lang/cpp/modules-import.TestCXXModulesImport.py
lldb-api.lang/objc/modules.TestObjCModules.py
lldb-api.lang/objc/modules-auto-import.TestModulesAutoImport.py
lldb-api.lang/objc/modules-cache.TestClangModulesCache.py
lldb-api.lang/objc/modules-compile-error.TestModulesCompileError.py
lldb-api.lang/objc/modules-incomplete.TestIncompleteModules.py
lldb-api.lang/objc/modules-inline-functions.TestModulesInlineFunctions.py
lldb-api.lang/objc/modules-non-objc-target.TestObjCModulesNonObjCTarget.py
lldb-api.lang/objc/modules-objc-property.TestModulesObjCProperty.py
lldb-api.tools/lldb-server.TestLldbGdbServer.py

https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48872/
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48873/

Last successful build: https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48871/

[v.g.vassilev]

In D127284#3969225, @Michael137 wrote:

FYI, looks like this broke the LLDB incremental buildbot. Following tests started failing:

lldb-api.commands/expression/argument_passing_restrictions.TestArgumentPassingRestrictions.py
lldb-api.commands/expression/deleting-implicit-copy-constructor.TestDeletingImplicitCopyConstructor.py
lldb-api.commands/expression/diagnostics.TestExprDiagnostics.py
lldb-api.commands/expression/import-std-module/array.TestArrayFromStdModule.py
lldb-api.commands/expression/import-std-module/module-build-errors.TestStdModuleBuildErrors.py
lldb-api.commands/expression/rdar44436068.Test128BitsInteger.py
lldb-api.commands/expression/weak_symbols.TestWeakSymbols.py
lldb-api.commands/target/dump-pcm-info.TestDumpPCMInfo.py
lldb-api.functionalities/data-formatter/compactvectors.TestCompactVectors.py
lldb-api.functionalities/tail_call_frames/cross_object.TestCrossObjectTailCalls.py
lldb-api.functionalities/target_var/no_vars.TestTargetVarNoVars.py
lldb-api.lang/c/modules.TestCModules.py
lldb-api.lang/cpp/accelerator-table.TestCPPAccelerator.py
lldb-api.lang/cpp/modules-import.TestCXXModulesImport.py
lldb-api.lang/objc/modules.TestObjCModules.py
lldb-api.lang/objc/modules-auto-import.TestModulesAutoImport.py
lldb-api.lang/objc/modules-cache.TestClangModulesCache.py
lldb-api.lang/objc/modules-compile-error.TestModulesCompileError.py
lldb-api.lang/objc/modules-incomplete.TestIncompleteModules.py
lldb-api.lang/objc/modules-inline-functions.TestModulesInlineFunctions.py
lldb-api.lang/objc/modules-non-objc-target.TestObjCModulesNonObjCTarget.py
lldb-api.lang/objc/modules-objc-property.TestModulesObjCProperty.py
lldb-api.tools/lldb-server.TestLldbGdbServer.py

https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48872/
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48873/

Last successful build: https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48871/

Hi @Michael137, I cannot reproduce these failures on unix. Is there some special way I need to compile. I will try reproducing them on OSX. Did you have time to debug where the tests fail?

[v.g.vassilev]

In D127284#3969249, @v.g.vassilev wrote:

In D127284#3969225, @Michael137 wrote:

FYI, looks like this broke the LLDB incremental buildbot. Following tests started failing:

lldb-api.commands/expression/argument_passing_restrictions.TestArgumentPassingRestrictions.py
lldb-api.commands/expression/deleting-implicit-copy-constructor.TestDeletingImplicitCopyConstructor.py
lldb-api.commands/expression/diagnostics.TestExprDiagnostics.py
lldb-api.commands/expression/import-std-module/array.TestArrayFromStdModule.py
lldb-api.commands/expression/import-std-module/module-build-errors.TestStdModuleBuildErrors.py
lldb-api.commands/expression/rdar44436068.Test128BitsInteger.py
lldb-api.commands/expression/weak_symbols.TestWeakSymbols.py
lldb-api.commands/target/dump-pcm-info.TestDumpPCMInfo.py
lldb-api.functionalities/data-formatter/compactvectors.TestCompactVectors.py
lldb-api.functionalities/tail_call_frames/cross_object.TestCrossObjectTailCalls.py
lldb-api.functionalities/target_var/no_vars.TestTargetVarNoVars.py
lldb-api.lang/c/modules.TestCModules.py
lldb-api.lang/cpp/accelerator-table.TestCPPAccelerator.py
lldb-api.lang/cpp/modules-import.TestCXXModulesImport.py
lldb-api.lang/objc/modules.TestObjCModules.py
lldb-api.lang/objc/modules-auto-import.TestModulesAutoImport.py
lldb-api.lang/objc/modules-cache.TestClangModulesCache.py
lldb-api.lang/objc/modules-compile-error.TestModulesCompileError.py
lldb-api.lang/objc/modules-incomplete.TestIncompleteModules.py
lldb-api.lang/objc/modules-inline-functions.TestModulesInlineFunctions.py
lldb-api.lang/objc/modules-non-objc-target.TestObjCModulesNonObjCTarget.py
lldb-api.lang/objc/modules-objc-property.TestModulesObjCProperty.py
lldb-api.tools/lldb-server.TestLldbGdbServer.py

https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48872/
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48873/

Last successful build: https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48871/

Hi @Michael137, I cannot reproduce these failures on unix. Is there some special way I need to compile. I will try reproducing them on OSX. Did you have time to debug where the tests fail?

I cannot reproduce this on osx either.

[Michael137]

In D127284#3969308, @v.g.vassilev wrote:

In D127284#3969249, @v.g.vassilev wrote:

In D127284#3969225, @Michael137 wrote:

FYI, looks like this broke the LLDB incremental buildbot. Following tests started failing:

lldb-api.commands/expression/argument_passing_restrictions.TestArgumentPassingRestrictions.py
lldb-api.commands/expression/deleting-implicit-copy-constructor.TestDeletingImplicitCopyConstructor.py
lldb-api.commands/expression/diagnostics.TestExprDiagnostics.py
lldb-api.commands/expression/import-std-module/array.TestArrayFromStdModule.py
lldb-api.commands/expression/import-std-module/module-build-errors.TestStdModuleBuildErrors.py
lldb-api.commands/expression/rdar44436068.Test128BitsInteger.py
lldb-api.commands/expression/weak_symbols.TestWeakSymbols.py
lldb-api.commands/target/dump-pcm-info.TestDumpPCMInfo.py
lldb-api.functionalities/data-formatter/compactvectors.TestCompactVectors.py
lldb-api.functionalities/tail_call_frames/cross_object.TestCrossObjectTailCalls.py
lldb-api.functionalities/target_var/no_vars.TestTargetVarNoVars.py
lldb-api.lang/c/modules.TestCModules.py
lldb-api.lang/cpp/accelerator-table.TestCPPAccelerator.py
lldb-api.lang/cpp/modules-import.TestCXXModulesImport.py
lldb-api.lang/objc/modules.TestObjCModules.py
lldb-api.lang/objc/modules-auto-import.TestModulesAutoImport.py
lldb-api.lang/objc/modules-cache.TestClangModulesCache.py
lldb-api.lang/objc/modules-compile-error.TestModulesCompileError.py
lldb-api.lang/objc/modules-incomplete.TestIncompleteModules.py
lldb-api.lang/objc/modules-inline-functions.TestModulesInlineFunctions.py
lldb-api.lang/objc/modules-non-objc-target.TestObjCModulesNonObjCTarget.py
lldb-api.lang/objc/modules-objc-property.TestModulesObjCProperty.py
lldb-api.tools/lldb-server.TestLldbGdbServer.py

https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48872/
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48873/

Last successful build: https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48871/

Hi @Michael137, I cannot reproduce these failures on unix. Is there some special way I need to compile. I will try reproducing them on OSX. Did you have time to debug where the tests fail?

I cannot reproduce this on osx either.

Hmmm how are you running the tests? It reproduces pretty consistently with the patch and doesn't without it. I just run ninja check-lldb-api (assuming you built with LLDB_INCLUDE_TESTS=ON)

Let me try with a clean build

UPDATE: fails with clean build too

Might be best to revert it for now while we figure out what's wrong

[v.g.vassilev]

In D127284#3969398, @Michael137 wrote:

In D127284#3969308, @v.g.vassilev wrote:

In D127284#3969249, @v.g.vassilev wrote:

In D127284#3969225, @Michael137 wrote:

FYI, looks like this broke the LLDB incremental buildbot. Following tests started failing:

lldb-api.commands/expression/argument_passing_restrictions.TestArgumentPassingRestrictions.py
lldb-api.commands/expression/deleting-implicit-copy-constructor.TestDeletingImplicitCopyConstructor.py
lldb-api.commands/expression/diagnostics.TestExprDiagnostics.py
lldb-api.commands/expression/import-std-module/array.TestArrayFromStdModule.py
lldb-api.commands/expression/import-std-module/module-build-errors.TestStdModuleBuildErrors.py
lldb-api.commands/expression/rdar44436068.Test128BitsInteger.py
lldb-api.commands/expression/weak_symbols.TestWeakSymbols.py
lldb-api.commands/target/dump-pcm-info.TestDumpPCMInfo.py
lldb-api.functionalities/data-formatter/compactvectors.TestCompactVectors.py
lldb-api.functionalities/tail_call_frames/cross_object.TestCrossObjectTailCalls.py
lldb-api.functionalities/target_var/no_vars.TestTargetVarNoVars.py
lldb-api.lang/c/modules.TestCModules.py
lldb-api.lang/cpp/accelerator-table.TestCPPAccelerator.py
lldb-api.lang/cpp/modules-import.TestCXXModulesImport.py
lldb-api.lang/objc/modules.TestObjCModules.py
lldb-api.lang/objc/modules-auto-import.TestModulesAutoImport.py
lldb-api.lang/objc/modules-cache.TestClangModulesCache.py
lldb-api.lang/objc/modules-compile-error.TestModulesCompileError.py
lldb-api.lang/objc/modules-incomplete.TestIncompleteModules.py
lldb-api.lang/objc/modules-inline-functions.TestModulesInlineFunctions.py
lldb-api.lang/objc/modules-non-objc-target.TestObjCModulesNonObjCTarget.py
lldb-api.lang/objc/modules-objc-property.TestModulesObjCProperty.py
lldb-api.tools/lldb-server.TestLldbGdbServer.py

https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48872/
https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48873/

Last successful build: https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48871/

Hi @Michael137, I cannot reproduce these failures on unix. Is there some special way I need to compile. I will try reproducing them on OSX. Did you have time to debug where the tests fail?

I cannot reproduce this on osx either.

Hmmm how are you running the tests? It reproduces pretty consistently with the patch and doesn't without it. I just run ninja check-lldb-api (assuming you built with LLDB_INCLUDE_TESTS=ON)

Let me try with a clean build

I used cmake -C ../../sources/llvm-project/lldb/cmake/caches/Apple-lldb-macOS.cmake -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi" -DLLDB_USE_SYSTEM_DEBUGSERVER=ON -DLLVM_ENABLE_PROJECTS="clang;lldb" -DLLVM_TARGETS_TO_BUILD=host -DCMAKE_BUILD_TYPE=Debug -DLLDB_INCLUDE_TESTS=ON -G Ninja ../../sources/llvm-project/llvm

I run:

ninja -j20 check-lldb-api -v
[0/3] cd /Users/vvassilev/workspace/builds/llvm-project/runtimes/runtimes-bins && /usr/local/Cellar/cmake/3.20.2/bin/cmake --build /Users/vvassilev/workspace/builds/llvm-project/runtimes/runtimes-bins/ --target cxx --config Debug
ninja: no work to do.
[1/3] cd /Users/vvassilev/workspace/builds/llvm-project/tools/lldb/test/API && /usr/local/Cellar/cmake/3.20.2/bin/cmake -E copy_if_different /Applications/Xcode.app/Contents/Developer/../SharedFrameworks/LLDB.framework/Resources/debugserver /Users/vvassilev/workspace/builds/llvm-project/./bin
[2/3] cd /Users/vvassilev/workspace/builds/llvm-project/tools/lldb/test/API && /Users/vvassilev/miniconda/bin/python3.9 /Users/vvassilev/workspace/builds/llvm-project/./bin/llvm-lit -sv /Users/vvassilev/workspace/builds/llvm-project/tools/lldb/test/API

Testing Time: 0.38s
  Unsupported: 1070

For some reasons the failing tests are marked as unresolved for me. I have osx 12.2.1

[v.g.vassilev]

UPDATE: fails with clean build too

Might be best to revert it for now while we figure out what's wrong

I fail to reproduce it, can you give me access to some node where I can debug? I suspect that should be something easy to fix.

UPDATE: Reproduced it. Debugging.

[v.g.vassilev]

In D127284#3969446, @v.g.vassilev wrote:

UPDATE: fails with clean build too

Might be best to revert it for now while we figure out what's wrong

I fail to reproduce it, can you give me access to some node where I can debug? I suspect that should be something easy to fix.

UPDATE: Reproduced it. Debugging.

@Michael137, I think I understand what happens. This patch introduces a new language option which is not benign from modules perspective. However, lldb sets up modules the old way and then switches to the incremental processing mode. We have two ways to fix this:

• Pass -Xclang -fincremental-extensions in ClangModulesDeclVendor.cpp and delete instance->getPreprocessor().enableIncrementalProcessing();; or
• Apply https://reviews.llvm.org/D139258 which already does that.

Can you check if https://reviews.llvm.org/D139258 passes the lldb testsuite. It does pass my reproduction by hand (I still cannot run full of the lldb-api tests)?

[Michael137]

In D127284#3969458, @v.g.vassilev wrote:

In D127284#3969446, @v.g.vassilev wrote:

UPDATE: fails with clean build too

Might be best to revert it for now while we figure out what's wrong

I fail to reproduce it, can you give me access to some node where I can debug? I suspect that should be something easy to fix.

UPDATE: Reproduced it. Debugging.

@Michael137, I think I understand what happens. This patch introduces a new language option which is not benign from modules perspective. However, lldb sets up modules the old way and then switches to the incremental processing mode. We have two ways to fix this:

• Pass -Xclang -fincremental-extensions in ClangModulesDeclVendor.cpp and delete instance->getPreprocessor().enableIncrementalProcessing();; or
• Apply https://reviews.llvm.org/D139258 which already does that.

Can you check if https://reviews.llvm.org/D139258 passes the lldb testsuite. It does pass my reproduction by hand (I still cannot run full of the lldb-api tests)?

Thanks for taking a look. I tried with the suggested patch on the Objective-C API tests and there's only 1 test failure remaining there:

lang/objc/modules-compile-error/TestModulesCompileError.py

Expecting sub string: "module.h:4:1: error: unknown type name 'syntax_error_for_lldb_to_find'" (was not found)

The test expects an error string that looks like unknown type name 'syntax_error_for_lldb_to_find' but gets use of 'undeclared identifier 'syntax_error_for_lldb_to_find'

Haven't looked much further than that. Does that sound familiar to you? I see a similar test fix as part of this patch. Maybe it's just a matter of fixing up the expected string.

I think we should revert for now until https://reviews.llvm.org/D139258 is ready to go. Just to unblock the buildbot

[v.g.vassilev]

In D127284#3969593, @Michael137 wrote:

In D127284#3969458, @v.g.vassilev wrote:

In D127284#3969446, @v.g.vassilev wrote:

UPDATE: fails with clean build too

Might be best to revert it for now while we figure out what's wrong

I fail to reproduce it, can you give me access to some node where I can debug? I suspect that should be something easy to fix.

UPDATE: Reproduced it. Debugging.

@Michael137, I think I understand what happens. This patch introduces a new language option which is not benign from modules perspective. However, lldb sets up modules the old way and then switches to the incremental processing mode. We have two ways to fix this:

• Pass -Xclang -fincremental-extensions in ClangModulesDeclVendor.cpp and delete instance->getPreprocessor().enableIncrementalProcessing();; or
• Apply https://reviews.llvm.org/D139258 which already does that.

Can you check if https://reviews.llvm.org/D139258 passes the lldb testsuite. It does pass my reproduction by hand (I still cannot run full of the lldb-api tests)?

Thanks for taking a look. I tried with the suggested patch on the Objective-C API tests and there's only 1 test failure remaining there:

lang/objc/modules-compile-error/TestModulesCompileError.py

Expecting sub string: "module.h:4:1: error: unknown type name 'syntax_error_for_lldb_to_find'" (was not found)

The test expects an error string that looks like unknown type name 'syntax_error_for_lldb_to_find' but gets use of 'undeclared identifier 'syntax_error_for_lldb_to_find'

Haven't looked much further than that. Does that sound familiar to you? I see a similar test fix as part of this patch. Maybe it's just a matter of fixing up the expected string.

I think we should revert for now until https://reviews.llvm.org/D139258 is ready to go. Just to unblock the buildbot

I have pushed a fix in https://github.com/llvm/llvm-project/commit/c95a0c91c0de66eb1066f23c69332522656f188e That should unblock the bot. If that does not work, I will revert.

[Michael137]

In D127284#3969975, @v.g.vassilev wrote:

In D127284#3969593, @Michael137 wrote:

In D127284#3969458, @v.g.vassilev wrote:

In D127284#3969446, @v.g.vassilev wrote:

UPDATE: fails with clean build too

Might be best to revert it for now while we figure out what's wrong

I fail to reproduce it, can you give me access to some node where I can debug? I suspect that should be something easy to fix.

UPDATE: Reproduced it. Debugging.

@Michael137, I think I understand what happens. This patch introduces a new language option which is not benign from modules perspective. However, lldb sets up modules the old way and then switches to the incremental processing mode. We have two ways to fix this:

• Pass -Xclang -fincremental-extensions in ClangModulesDeclVendor.cpp and delete instance->getPreprocessor().enableIncrementalProcessing();; or
• Apply https://reviews.llvm.org/D139258 which already does that.

Can you check if https://reviews.llvm.org/D139258 passes the lldb testsuite. It does pass my reproduction by hand (I still cannot run full of the lldb-api tests)?

Thanks for taking a look. I tried with the suggested patch on the Objective-C API tests and there's only 1 test failure remaining there:

lang/objc/modules-compile-error/TestModulesCompileError.py

Expecting sub string: "module.h:4:1: error: unknown type name 'syntax_error_for_lldb_to_find'" (was not found)

The test expects an error string that looks like unknown type name 'syntax_error_for_lldb_to_find' but gets use of 'undeclared identifier 'syntax_error_for_lldb_to_find'

Haven't looked much further than that. Does that sound familiar to you? I see a similar test fix as part of this patch. Maybe it's just a matter of fixing up the expected string.

I think we should revert for now until https://reviews.llvm.org/D139258 is ready to go. Just to unblock the buildbot

I have pushed a fix in https://github.com/llvm/llvm-project/commit/c95a0c91c0de66eb1066f23c69332522656f188e That should unblock the bot. If that does not work, I will revert.

Latest test run: https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48911/execution/node/74/log/
I modified the expected string slightly to make it work: https://reviews.llvm.org/rG811ad246ac7b
Should be all good now. Thanks for looking into this!

[v.g.vassilev]

Latest test run: https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/48911/execution/node/74/log/
I modified the expected string slightly to make it work: https://reviews.llvm.org/rG811ad246ac7b
Should be all good now. Thanks for looking into this!

Thanks a lot for pinging! Glad that it worked with a relatively little number of iterations considering the nature of this patch ;)

[UnionType]

clang/test/Interpreter/execute-stmts.cpp

35

It seems that top-level statements wrapped in braces cannot be handled, maybe I made a mistake.

for (; i > 4; --i) { printf("i = %d\n", i); }

{ i++; }

[v.g.vassilev]

clang/test/Interpreter/execute-stmts.cpp

35

Ah, you are right. I have a fix for this case here: https://reviews.llvm.org/D139798

[UnionType]

@v.g.vassilev Thanks for your reply, the patch is clean and clear.

[tahonermann]

This change appears to have negatively impacted some users that were dependent on the previous Preprocessing::enableIncrementalProcessing(true) behavior. See https://github.com/llvm/llvm-project/issues/62413.

[v.g.vassilev]

In D127284#4303390, @tahonermann wrote:

This change appears to have negatively impacted some users that were dependent on the previous Preprocessing::enableIncrementalProcessing(true) behavior. See https://github.com/llvm/llvm-project/issues/62413.

Thank you for the ping. I’ve replied there.

clang/include/clang/AST/ASTNodeTraverser.h

void VisitBindingDecl(const BindingDecl *D) {
     if (const auto *E = D->getBinding())
       Visit(E);
   }
 
   void VisitFileScopeAsmDecl(const FileScopeAsmDecl *D) {
     Visit(D->getAsmString());
   }
 
+  void VisitTopLevelStmtDecl(const TopLevelStmtDecl *D) {
+    for (const Stmt *S : D->statements())
+      Visit(S);
+  }
+
   void VisitCapturedDecl(const CapturedDecl *D) { Visit(D->getBody()); }
 
   void VisitOMPThreadPrivateDecl(const OMPThreadPrivateDecl *D) {
     for (const auto *E : D->varlists())
       Visit(E);
   }
 
   void VisitOMPDeclareReductionDecl(const OMPDeclareReductionDecl *D) {

clang/include/clang/AST/Decl.h

public:
   const StringLiteral *getAsmString() const { return AsmString; }
   StringLiteral *getAsmString() { return AsmString; }
   void setAsmString(StringLiteral *Asm) { AsmString = Asm; }
 
   static bool classof(const Decl *D) { return classofKind(D->getKind()); }
   static bool classofKind(Kind K) { return K == FileScopeAsm; }
 };
 
+/// A declaration that models statements on the global scope. This declaration
+/// supports incremental and interactive C/C++.
+///
+///\note This is used in libInterpreter, clang -cc1 -fincremental-extensions and
+/// in tools such as clang-repl.
+class TopLevelStmtDecl : public Decl {
+  friend class ASTDeclReader;
+  friend class ASTDeclWriter;
+
+  Stmt **Stmts = nullptr;
+  unsigned NumStmts = 0;
+  FunctionDecl *FD = nullptr;
+
+  TopLevelStmtDecl(DeclContext *DC, SourceLocation L)
+      : Decl(TopLevelStmt, DC, L) {}
+
+  void setStmts(ASTContext &C, ArrayRef<Stmt *> Statements) {
+    if (!Statements.empty()) {
+      Stmts = new (C) Stmt *[Statements.size()];
+      std::copy(Statements.begin(), Statements.end(), Stmts);
+      NumStmts = Statements.size();
+    }
+  }
+
+  virtual void anchor();
+
+public:
+  static TopLevelStmtDecl *Create(ASTContext &C, ArrayRef<Stmt *> Stmts);
+  static TopLevelStmtDecl *CreateDeserialized(ASTContext &C, unsigned ID);
+
+  SourceRange getSourceRange() const override LLVM_READONLY;
+  FunctionDecl *getOrConvertToFunction();
+  ArrayRef<Stmt *> statements() const { return {Stmts, NumStmts}; }
+
+  static bool classof(const Decl *D) { return classofKind(D->getKind()); }
+  static bool classofKind(Kind K) { return K == TopLevelStmt; }
+};
+
 /// Represents a block literal declaration, which is like an
 /// unnamed FunctionDecl.  For example:
 /// ^{ statement-body }   or   ^(int arg1, float arg2){ statement-body }
 class BlockDecl : public Decl, public DeclContext {
   // This class stores some data in DeclContext::BlockDeclBits
   // to save some space. Use the provided accessors to access it.
 public:
   /// A class which contains all the information about a particular

clang/include/clang/AST/RecursiveASTVisitor.h

 
 DEF_TRAVERSE_DECL(LifetimeExtendedTemporaryDecl, {
   TRY_TO(TraverseStmt(D->getTemporaryExpr()));
 })
 
 DEF_TRAVERSE_DECL(FileScopeAsmDecl,
                   { TRY_TO(TraverseStmt(D->getAsmString())); })
 
+DEF_TRAVERSE_DECL(TopLevelStmtDecl, {
+  for (auto *I : D->statements())
+    TRY_TO(TraverseStmt(I));
+})
+
 DEF_TRAVERSE_DECL(ImportDecl, {})
 
 DEF_TRAVERSE_DECL(FriendDecl, {
   // Friend is either decl or a type.
   if (D->getFriendType()) {
     TRY_TO(TraverseTypeLoc(D->getFriendType()->getTypeLoc()));
     // Traverse any CXXRecordDecl owned by this type, since
     // it will not be in the parent context:

clang/include/clang/Basic/DeclNodes.td

def ObjCImpl
       def ObjCCategoryImpl : DeclNode<ObjCImpl>;
       def ObjCImplementation : DeclNode<ObjCImpl>;
   def ObjCProperty : DeclNode<Named, "Objective-C properties">;
   def ObjCCompatibleAlias : DeclNode<Named>;
 def LinkageSpec : DeclNode<Decl>, DeclContext;
 def Export : DeclNode<Decl>, DeclContext;
 def ObjCPropertyImpl : DeclNode<Decl>;
 def FileScopeAsm : DeclNode<Decl>;
+def TopLevelStmt : DeclNode<Decl>;
 def AccessSpec : DeclNode<Decl>;
 def Friend : DeclNode<Decl>;
 def FriendTemplate : DeclNode<Decl>;
 def StaticAssert : DeclNode<Decl>;
 def Block : DeclNode<Decl, "blocks">, DeclContext;
 def Captured : DeclNode<Decl>, DeclContext;
 def ClassScopeFunctionSpecialization : DeclNode<Decl>;
 def Import : DeclNode<Decl>;
 def OMPThreadPrivate : DeclNode<Decl>;
 def OMPAllocate : DeclNode<Decl>;
 def OMPRequires : DeclNode<Decl>;
 def Empty : DeclNode<Decl>;
 def RequiresExprBody : DeclNode<Decl>, DeclContext;
 def LifetimeExtendedTemporary : DeclNode<Decl>;
 def HLSLBuffer : DeclNode<Named, "HLSLBuffer">, DeclContext;

clang/include/clang/Basic/LangOptions.def

 
 VALUE_LANGOPT(FuchsiaAPILevel, 32, 0, "Fuchsia API level")
 
 // This option will be removed in the future once the backend
 // supports all operations (like division or float-to-integer conversion)
 // on large _BitInts.
 BENIGN_VALUE_LANGOPT(MaxBitIntWidth, 32, 128, "Maximum width of a _BitInt")
 
+LANGOPT(IncrementalExtensions, 1, 0, " True if we want to process statements"
+        "on the global scope, ignore EOF token and continue later on (thus "
+        "avoid tearing the Lexer and etc. down). Controlled by "
+        "-fincremental-extensions.")
+
 #undef LANGOPT
 #undef COMPATIBLE_LANGOPT
 #undef BENIGN_LANGOPT
 #undef ENUM_LANGOPT
 #undef COMPATIBLE_ENUM_LANGOPT
 #undef BENIGN_ENUM_LANGOPT
 #undef VALUE_LANGOPT
 #undef COMPATIBLE_VALUE_LANGOPT
 #undef BENIGN_VALUE_LANGOPT

clang/include/clang/Driver/Options.td

defm modules_validate_system_headers : BoolOption<"f", "modules-validate-system-headers",
   NegFlag<SetFalse, [NoXarchOption]>>, Group<i_Group>;
 def fno_modules_validate_textual_header_includes :
   Flag<["-"], "fno-modules-validate-textual-header-includes">,
   Group<f_Group>, Flags<[CC1Option, NoXarchOption]>,
   MarshallingInfoNegativeFlag<LangOpts<"ModulesValidateTextualHeaderIncludes">>,
   HelpText<"Do not enforce -fmodules-decluse and private header restrictions for textual headers. "
            "This flag will be removed in a future Clang release.">;
 
+def fincremental_extensions :
+  Flag<["-"], "fincremental-extensions">,
+  Group<f_Group>, Flags<[CC1Option]>,
+  HelpText<"Enable incremental processing extensions such as processing"
+           "statements on the global scope.">,
+  MarshallingInfoFlag<LangOpts<"IncrementalExtensions">>;
+
+
 def fvalidate_ast_input_files_content:
   Flag <["-"], "fvalidate-ast-input-files-content">,
   Group<f_Group>, Flags<[CC1Option]>,
   HelpText<"Compute and store the hash of input files used to build an AST."
            " Files with mismatching mtime's are considered valid"
            " if both contents is identical">,
   MarshallingInfoFlag<HeaderSearchOpts<"ValidateASTInputFilesContent">>;
 def fmodules_validate_input_files_content:

clang/include/clang/Lex/Preprocessor.h

class Preprocessor {
 
   /// Tracks all of the comment handlers that the client registered
   /// with this preprocessor.
   std::vector<CommentHandler *> CommentHandlers;
 
   /// Empty line handler.
   EmptylineHandler *Emptyline = nullptr;
 
-  /// True if we want to ignore EOF token and continue later on (thus
-  /// avoid tearing the Lexer and etc. down).
-  bool IncrementalProcessing = false;
-
 public:
   /// The kind of translation unit we are processing.
   const TranslationUnitKind TUKind;
 
 private:
   /// The code-completion handler.
   CodeCompletionHandler *CodeComplete = nullptr;
 
if (CachedLexPos != 0 && isBacktrackEnabled())
       CachedTokens[CachedLexPos-1] = Tok;
   }
 
   /// Recompute the current lexer kind based on the CurLexer/
   /// CurTokenLexer pointers.
   void recomputeCurLexerKind();
 
   /// Returns true if incremental processing is enabled
-  bool isIncrementalProcessingEnabled() const { return IncrementalProcessing; }
+  bool isIncrementalProcessingEnabled() const {
+    return getLangOpts().IncrementalExtensions;
+  }
 
   /// Enables the incremental processing
   void enableIncrementalProcessing(bool value = true) {
-    IncrementalProcessing = value;
+    // FIXME: Drop this interface.
+    const_cast<LangOptions &>(getLangOpts()).IncrementalExtensions = value;
   }
 
   /// Specify the point at which code-completion will be performed.
   ///
   /// \param File the file in which code completion should occur. If
   /// this file is included multiple times, code-completion will
   /// perform completion the first time it is included. If NULL, this
   /// function clears out the code-completion point.
   ///

clang/include/clang/Parse/Parser.h

public:
   // different actual classes based on the actions in place.
   typedef OpaquePtr<DeclGroupRef> DeclGroupPtrTy;
   typedef OpaquePtr<TemplateName> TemplateTy;
 
   typedef SmallVector<TemplateParameterList *, 4> TemplateParameterLists;
 
   typedef Sema::FullExprArg FullExprArg;
 
+  /// A SmallVector of statements.
+  typedef SmallVector<Stmt *, 32> StmtVector;
+
   // Parsing methods.
 
   /// Initialize - Warm up the parser.
   ///
   void Initialize();
 
   /// Parse the first top-level declaration in a translation unit.
   bool ParseFirstTopLevelDecl(DeclGroupPtrTy &Result,
private:
   ExprResult ParseAssignmentExprWithObjCMessageExprStart(
       SourceLocation LBracloc, SourceLocation SuperLoc,
       ParsedType ReceiverType, Expr *ReceiverExpr);
   bool ParseObjCXXMessageReceiver(bool &IsExpr, void *&TypeOrExpr);
 
   //===--------------------------------------------------------------------===//
   // C99 6.8: Statements and Blocks.
 
-  /// A SmallVector of statements, with stack size 32 (as that is the only one
-  /// used.)
-  typedef SmallVector<Stmt*, 32> StmtVector;
-  /// A SmallVector of expressions, with stack size 12 (the maximum used.)
+  /// A SmallVector of expressions.
   typedef SmallVector<Expr*, 12> ExprVector;
   /// A SmallVector of types.
   typedef SmallVector<ParsedType, 12> TypeVector;
 
   StmtResult
   ParseStatement(SourceLocation *TrailingElseLoc = nullptr,
                  ParsedStmtContext StmtCtx = ParsedStmtContext::SubStmt);
   StmtResult ParseStatementOrDeclaration(
private:
   void ParseEnumBody(SourceLocation StartLoc, Decl *TagDecl);
   void ParseStructUnionBody(SourceLocation StartLoc, DeclSpec::TST TagType,
                             RecordDecl *TagDecl);
 
   void ParseStructDeclaration(
       ParsingDeclSpec &DS,
       llvm::function_ref<void(ParsingFieldDeclarator &)> FieldsCallback);
 
+  Decl *ParseTopLevelStmtDecl();
+
   bool isDeclarationSpecifier(ImplicitTypenameContext AllowImplicitTypename,
                               bool DisambiguatingWithExpression = false);
   bool isTypeSpecifierQualifier();
 
   /// isKnownToBeTypeSpecifier - Return true if we know that the specified token
   /// is definitely a type-specifier.  Return false if it isn't part of a type
   /// specifier or if we're not sure.
   bool isKnownToBeTypeSpecifier(const Token &Tok) const;
 
   /// Return true if we know that we are definitely looking at a
   /// decl-specifier, and isn't part of an expression such as a function-style
   /// cast. Return false if it's no a decl-specifier, or we're not sure.
   bool isKnownToBeDeclarationSpecifier() {
     if (getLangOpts().CPlusPlus)
       return isCXXDeclarationSpecifier(ImplicitTypenameContext::No) ==
              TPResult::True;
     return isDeclarationSpecifier(ImplicitTypenameContext::No, true);
   }
 
   /// isDeclarationStatement - Disambiguates between a declaration or an
   /// expression statement, when parsing function bodies.
+  ///
+  /// \param DisambiguatingWithExpression - True to indicate that the purpose of
+  /// this check is to disambiguate between an expression and a declaration.
   /// Returns true for declaration, false for expression.
-  bool isDeclarationStatement() {
+  bool isDeclarationStatement(bool DisambiguatingWithExpression = false) {
     if (getLangOpts().CPlusPlus)
-      return isCXXDeclarationStatement();
+      return isCXXDeclarationStatement(DisambiguatingWithExpression);
     return isDeclarationSpecifier(ImplicitTypenameContext::No, true);
   }
 
   /// isForInitDeclaration - Disambiguates between a declaration or an
   /// expression in the context of the C 'clause-1' or the C++
   // 'for-init-statement' part of a 'for' statement.
   /// Returns true for declaration, false for expression.
   bool isForInitDeclaration() {
private:
   /// class message that appears to be missing the open bracket '['.
   bool isStartOfObjCClassMessageMissingOpenBracket();
 
   /// Starting with a scope specifier, identifier, or
   /// template-id that refers to the current class, determine whether
   /// this is a constructor declarator.
   bool isConstructorDeclarator(
       bool Unqualified, bool DeductionGuide = false,
-      DeclSpec::FriendSpecified IsFriend = DeclSpec::FriendSpecified::No);
+      DeclSpec::FriendSpecified IsFriend = DeclSpec::FriendSpecified::No,
+      bool DisambiguatingWithExpression = false);
 
   /// Specifies the context in which type-id/expression
   /// disambiguation will occur.
   enum TentativeCXXTypeIdContext {
     TypeIdInParens,
     TypeIdUnambiguous,
     TypeIdAsTemplateArgument
   };
bool isTypeIdUnambiguously() {
     if (getLangOpts().CPlusPlus)
       return isCXXTypeId(TypeIdUnambiguous, IsAmbiguous);
     return isTypeSpecifierQualifier();
   }
 
   /// isCXXDeclarationStatement - C++-specialized function that disambiguates
   /// between a declaration or an expression statement, when parsing function
   /// bodies. Returns true for declaration, false for expression.
-  bool isCXXDeclarationStatement();
+  bool isCXXDeclarationStatement(bool DisambiguatingWithExpression = false);
 
   /// isCXXSimpleDeclaration - C++-specialized function that disambiguates
   /// between a simple-declaration or an expression-statement.
   /// If during the disambiguation process a parsing error is encountered,
   /// the function returns true to let the declaration parsing code handle it.
   /// Returns false if the statement is disambiguated as expression.
   bool isCXXSimpleDeclaration(bool AllowForRangeDecl);
 

clang/include/clang/Sema/Sema.h

public:
   DiagnoseSizeOfParametersAndReturnValue(ArrayRef<ParmVarDecl *> Parameters,
                                          QualType ReturnTy, NamedDecl *D);
 
   void DiagnoseInvalidJumps(Stmt *Body);
   Decl *ActOnFileScopeAsmDecl(Expr *expr,
                               SourceLocation AsmLoc,
                               SourceLocation RParenLoc);
 
+  Decl *ActOnTopLevelStmtDecl(const SmallVectorImpl<Stmt *> &Stmts);
+
   /// Handle a C++11 empty-declaration and attribute-declaration.
   Decl *ActOnEmptyDeclaration(Scope *S, const ParsedAttributesView &AttrList,
                               SourceLocation SemiLoc);
 
   enum class ModuleDeclKind {
     Interface,               ///< 'export module X;'
     Implementation,          ///< 'module X;'
     PartitionInterface,      ///< 'export module X:Y;'

clang/include/clang/Sema/Template.h

 // Define all the decl visitors using DeclNodes.inc
 #define DECL(DERIVED, BASE) \
     Decl *Visit ## DERIVED ## Decl(DERIVED ## Decl *D);
 #define ABSTRACT_DECL(DECL)
 
 // Decls which never appear inside a class or function.
 #define OBJCCONTAINER(DERIVED, BASE)
 #define FILESCOPEASM(DERIVED, BASE)
+#define TOPLEVELSTMT(DERIVED, BASE)
 #define IMPORT(DERIVED, BASE)
 #define EXPORT(DERIVED, BASE)
 #define LINKAGESPEC(DERIVED, BASE)
 #define OBJCCOMPATIBLEALIAS(DERIVED, BASE)
 #define OBJCMETHOD(DERIVED, BASE)
 #define OBJCTYPEPARAM(DERIVED, BASE)
 #define OBJCIVAR(DERIVED, BASE)
 #define OBJCPROPERTY(DERIVED, BASE)

clang/include/clang/Serialization/ASTBitCodes.h

enum DeclCode {
   DECL_DECOMPOSITION,
 
   /// A BindingDecl record.
   DECL_BINDING,
 
   /// A FileScopeAsmDecl record.
   DECL_FILE_SCOPE_ASM,
 
+  /// A TopLevelStmtDecl record.
+  DECL_TOP_LEVEL_STMT_DECL,
+
   /// A BlockDecl record.
   DECL_BLOCK,
 
   /// A CapturedDecl record.
   DECL_CAPTURED,
 
   /// A record that stores the set of declarations that are
   /// lexically stored within a given DeclContext.

clang/lib/AST/Decl.cpp

 }
 
 FileScopeAsmDecl *FileScopeAsmDecl::CreateDeserialized(ASTContext &C,
                                                        unsigned ID) {
   return new (C, ID) FileScopeAsmDecl(nullptr, nullptr, SourceLocation(),
                                       SourceLocation());
 }
 
+void TopLevelStmtDecl::anchor() {}
+
+TopLevelStmtDecl *TopLevelStmtDecl::Create(ASTContext &C,
+                                           llvm::ArrayRef<Stmt *> Stmts) {
+  assert(Stmts.size());
+  assert(C.getLangOpts().IncrementalExtensions &&
+         "Must be used only in incremental mode");
+
+  SourceLocation BeginLoc = Stmts[0]->getBeginLoc();
+  DeclContext *DC = C.getTranslationUnitDecl();
+
+  TopLevelStmtDecl *TLSD = new (C, DC) TopLevelStmtDecl(DC, BeginLoc);
+  TLSD->setStmts(C, Stmts);
+  return TLSD;
+}
+
+TopLevelStmtDecl *TopLevelStmtDecl::CreateDeserialized(ASTContext &C,
+                                                       unsigned ID) {
+  return new (C, ID) TopLevelStmtDecl(/*DC=*/nullptr, SourceLocation());
+}
+
+SourceRange TopLevelStmtDecl::getSourceRange() const {
+  return SourceRange(getLocation(), Stmts[NumStmts - 1]->getEndLoc());
+}
+
+FunctionDecl *TopLevelStmtDecl::getOrConvertToFunction() {
+  if (FD)
+    return FD;
+
+  ASTContext &C = getASTContext();
+  IdentifierInfo *name =
+      &C.Idents.get("_stmts_" + llvm::utostr((uintptr_t)this));
+  SourceLocation noLoc;
+  SourceLocation beginLoc = Stmts[0]->getBeginLoc();
+  FunctionProtoType::ExtProtoInfo EPI;
+  QualType FunctionTy = C.getFunctionType(C.VoidTy, llvm::None, EPI);
+  auto *TSI = C.getTrivialTypeSourceInfo(FunctionTy);
+  TranslationUnitDecl *TUDecl = cast<TranslationUnitDecl>(getDeclContext());
+  FD = FunctionDecl::Create(C, TUDecl, getBeginLoc(), noLoc, name, FunctionTy,
+                            TSI, SC_None);
+
+  auto StmtArrayRef = llvm::makeArrayRef(Stmts, NumStmts);
+  CompoundStmt *Body = CompoundStmt::Create(
+      C, StmtArrayRef, FPOptionsOverride(), beginLoc, getEndLoc());
+  FD->setBody(Body);
+  return FD;
+}
+
 void EmptyDecl::anchor() {}
 
 EmptyDecl *EmptyDecl::Create(ASTContext &C, DeclContext *DC, SourceLocation L) {
   return new (C, DC) EmptyDecl(DC, L);
 }
 
 EmptyDecl *EmptyDecl::CreateDeserialized(ASTContext &C, unsigned ID) {
   return new (C, ID) EmptyDecl(nullptr, SourceLocation());

clang/lib/AST/DeclBase.cpp

switch (DeclKind) {
 
     // Never have names.
     case Friend:
     case FriendTemplate:
     case AccessSpec:
     case LinkageSpec:
     case Export:
     case FileScopeAsm:
+    case TopLevelStmt:
     case StaticAssert:
     case ObjCPropertyImpl:
     case PragmaComment:
     case PragmaDetectMismatch:
     case Block:
     case Captured:
     case TranslationUnit:
     case ExternCContext:

clang/lib/AST/DeclPrinter.cpp

public:
     void VisitEmptyDecl(EmptyDecl *D);
     void VisitFunctionDecl(FunctionDecl *D);
     void VisitFriendDecl(FriendDecl *D);
     void VisitFieldDecl(FieldDecl *D);
     void VisitVarDecl(VarDecl *D);
     void VisitLabelDecl(LabelDecl *D);
     void VisitParmVarDecl(ParmVarDecl *D);
     void VisitFileScopeAsmDecl(FileScopeAsmDecl *D);
+    void VisitTopLevelStmtDecl(TopLevelStmtDecl *D);
     void VisitImportDecl(ImportDecl *D);
     void VisitStaticAssertDecl(StaticAssertDecl *D);
     void VisitNamespaceDecl(NamespaceDecl *D);
     void VisitUsingDirectiveDecl(UsingDirectiveDecl *D);
     void VisitNamespaceAliasDecl(NamespaceAliasDecl *D);
     void VisitCXXRecordDecl(CXXRecordDecl *D);
     void VisitLinkageSpecDecl(LinkageSpecDecl *D);
     void VisitTemplateDecl(const TemplateDecl *D);

 
 void DeclPrinter::VisitFileScopeAsmDecl(FileScopeAsmDecl *D) {
   Out << "__asm (";
   D->getAsmString()->printPretty(Out, nullptr, Policy, Indentation, "\n",
                                  &Context);
   Out << ")";
 }
 
+void DeclPrinter::VisitTopLevelStmtDecl(TopLevelStmtDecl *D) {
+  for (const Stmt *S : D->statements())
+    S->printPretty(Out, nullptr, Policy, Indentation, "\n", &Context);
+}
+
 void DeclPrinter::VisitImportDecl(ImportDecl *D) {
   Out << "@import " << D->getImportedModule()->getFullModuleName()
       << ";\n";
 }
 
 void DeclPrinter::VisitStaticAssertDecl(StaticAssertDecl *D) {
   Out << "static_assert(";
   D->getAssertExpr()->printPretty(Out, nullptr, Policy, Indentation, "\n",

clang/lib/CodeGen/CGDecl.cpp

void CodeGenFunction::EmitDecl(const Decl &D) {
   case Decl::ObjCCompatibleAlias:
   case Decl::PragmaComment:
   case Decl::PragmaDetectMismatch:
   case Decl::AccessSpec:
   case Decl::LinkageSpec:
   case Decl::Export:
   case Decl::ObjCPropertyImpl:
   case Decl::FileScopeAsm:
+  case Decl::TopLevelStmt:
   case Decl::Friend:
   case Decl::FriendTemplate:
   case Decl::Block:
   case Decl::Captured:
   case Decl::ClassScopeFunctionSpecialization:
   case Decl::UsingShadow:
   case Decl::ConstructorUsingShadow:
   case Decl::ObjCTypeParam:

clang/lib/CodeGen/CodeGenModule.cpp

void CodeGenModule::HandleCXXStaticMemberVarInstantiation(VarDecl *VD) {
 
   EmitTopLevelDecl(VD);
 }
 
 void CodeGenModule::EmitGlobalFunctionDefinition(GlobalDecl GD,
                                                  llvm::GlobalValue *GV) {
   const auto *D = cast<FunctionDecl>(GD.getDecl());
 
   // Compute the function info and LLVM type.
   const CGFunctionInfo &FI = getTypes().arrangeGlobalDeclaration(GD);
   llvm::FunctionType *Ty = getTypes().GetFunctionType(FI);
 
   // Get or create the prototype for the function.
   if (!GV || (GV->getValueType() != Ty))
     GV = cast<llvm::GlobalValue>(GetAddrOfFunction(GD, Ty, /*ForVTable=*/false,
                                                    /*DontDefer=*/true,
                                                    ForDefinition));
 
   // Already emitted.
   if (!GV->isDeclaration())
     return;

 }
 
 void CodeGenModule::EmitDeclContext(const DeclContext *DC) {
   for (auto *I : DC->decls()) {
     // Unlike other DeclContexts, the contents of an ObjCImplDecl at TU scope
     // are themselves considered "top-level", so EmitTopLevelDecl on an
     // ObjCImplDecl does not recursively visit them. We need to do that in
     // case they're nested inside another construct (LinkageSpecDecl /
     // ExportDecl) that does stop them from being considered "top-level".
     if (auto *OID = dyn_cast<ObjCImplDecl>(I)) {
       for (auto *M : OID->methods())
         EmitTopLevelDecl(M);
     }
 
     EmitTopLevelDecl(I);
   }
 }
 
 /// EmitTopLevelDecl - Emit code for a single top level declaration.
 void CodeGenModule::EmitTopLevelDecl(Decl *D) {
   // Ignore dependent declarations.
   if (D->isTemplated())
     return;
 
if (LangOpts.OpenMPIsDevice)
       break;
     // File-scope asm is ignored during device-side SYCL compilation.
     if (LangOpts.SYCLIsDevice)
       break;
     auto *AD = cast<FileScopeAsmDecl>(D);
     getModule().appendModuleInlineAsm(AD->getAsmString()->getString());
     break;
   }
+  case Decl::TopLevelStmt: {
+    FunctionDecl *FD = cast<TopLevelStmtDecl>(D)->getOrConvertToFunction();
+    EmitTopLevelDecl(FD);
+    GlobalDecl GD(FD);
+    auto *Fn = cast<llvm::Function>(GetGlobalValue(getMangledName(GD)));
+    CXXGlobalInits.push_back(Fn);
+    break;
+  }
 
   case Decl::Import: {
     auto *Import = cast<ImportDecl>(D);
 
     // If we've already imported this module, we're done.
     if (!ImportedModules.insert(Import->getImportedModule()))
       break;
 

clang/lib/CodeGen/ModuleBuilder.cpp

public:
     void HandleCXXStaticMemberVarInstantiation(VarDecl *VD) override {
       if (Diags.hasErrorOccurred())
         return;
 
       Builder->HandleCXXStaticMemberVarInstantiation(VD);
     }
 
     bool HandleTopLevelDecl(DeclGroupRef DG) override {
+      // FIXME: Why not return false and abort parsing?
       if (Diags.hasErrorOccurred())
         return true;
 
       HandlingTopLevelDeclRAII HandlingDecl(*this);
 
       // Make sure to emit all elements of a Decl.
       for (DeclGroupRef::iterator I = DG.begin(), E = DG.end(); I != E; ++I)
         Builder->EmitTopLevelDecl(*I);

clang/lib/Interpreter/IncrementalParser.cpp

if (hasCodeCompletionSupport() &&
       CI.createCodeCompletionConsumer();
 
     // Use a code completion consumer?
     CodeCompleteConsumer *CompletionConsumer = nullptr;
     if (CI.hasCodeCompletionConsumer())
       CompletionConsumer = &CI.getCodeCompletionConsumer();
 
     Preprocessor &PP = CI.getPreprocessor();
-    PP.enableIncrementalProcessing();
     PP.EnterMainSourceFile();
 
     if (!CI.hasSema())
       CI.createSema(getTranslationUnitKind(), CompletionConsumer);
   }
 
   // Do not terminate after processing the input. This allows us to keep various
   // clang objects alive and to incrementally grow the current TU.
if (P->getCurToken().is(tok::eof)) {
     P->EnterScope(Scope::DeclScope);
     S.ActOnTranslationUnitScope(P->getCurScope());
   }
 
   Parser::DeclGroupPtrTy ADecl;
   Sema::ModuleImportState ImportState;
   for (bool AtEOF = P->ParseFirstTopLevelDecl(ADecl, ImportState); !AtEOF;
        AtEOF = P->ParseTopLevelDecl(ADecl, ImportState)) {
-    // If we got a null return and something *was* parsed, ignore it.  This
-    // is due to a top-level semicolon, an action override, or a parse error
-    // skipping something.
     if (ADecl && !Consumer->HandleTopLevelDecl(ADecl.get()))
       return llvm::make_error<llvm::StringError>("Parsing failed. "
                                                  "The consumer rejected a decl",
                                                  std::error_code());
   }
 
   DiagnosticsEngine &Diags = getCI()->getDiagnostics();
   if (Diags.hasErrorOccurred()) {

clang/lib/Interpreter/Interpreter.cpp

IncrementalCompilerBuilder::create(std::vector<const char *> &ClangArgv) {
 
   ClangArgv.insert(ClangArgv.begin(), MainExecutableName.c_str());
 
   // Prepending -c to force the driver to do something if no action was
   // specified. By prepending we allow users to override the default
   // action and use other actions in incremental mode.
   // FIXME: Print proper driver diagnostics if the driver flags are wrong.
   ClangArgv.insert(ClangArgv.begin() + 1, "-c");
+  ClangArgv.insert(ClangArgv.begin() + 2, "-Xclang");
+  ClangArgv.insert(ClangArgv.begin() + 3, "-fincremental-extensions");
 
   if (!llvm::is_contained(ClangArgv, " -x")) {
     // We do C++ by default; append right after argv[0] if no "-x" given
     ClangArgv.push_back("-x");
     ClangArgv.push_back("c++");
   }
 
   // Put a dummy C++ file on to ensure there's at least one compile job for the

clang/lib/Parse/ParseDecl.cpp

case tok::annot_cxxscope: {
 
       // Check whether this is a constructor declaration. If we're in a
       // context where the identifier could be a class name, and it has the
       // shape of a constructor declaration, process it as one.
       if ((DSContext == DeclSpecContext::DSC_top_level ||
            DSContext == DeclSpecContext::DSC_class) &&
           Actions.isCurrentClassName(*Next.getIdentifierInfo(), getCurScope(),
                                      &SS) &&
           isConstructorDeclarator(/*Unqualified=*/false,
                                   /*DeductionGuide=*/false,
                                   DS.isFriendSpecified()))
         goto DoneWithDeclSpec;
 
       // C++20 [temp.spec] 13.9/6.
       // This disables the access checking rules for function template explicit
       // instantiation and explicit specialization:
       // - `return type`.
       SuppressAccessChecks SAC(*this, IsTemplateSpecOrInst);
 
case tok::kw_private:
     return getLangOpts().OpenCL;
 
   // C11 _Atomic
   case tok::kw__Atomic:
     return true;
   }
 }
 
+Decl *Parser::ParseTopLevelStmtDecl() {
+  if (!PP.isIncrementalProcessingEnabled())
+    return nullptr;
+
+  // FIXME: Remove the incremental processing pre-condition and verify clang
+  // still can pass its test suite, which will harden `isDeclarationStatement`.
+  // Parse a block of top-level-stmts.
+  Parser::StmtVector Stmts;
+  while (!isDeclarationStatement(/*DisambiguatingWithExpression=*/true)) {
+    ParsedStmtContext SubStmtCtx = ParsedStmtContext();
+    StmtResult R = ParseStatementOrDeclaration(Stmts, SubStmtCtx);
+    if (!R.isUsable())
+      return nullptr;
+
+    Stmts.push_back(R.get());
+
+    if (Tok.isOneOf(tok::eof, tok::annot_module_end))
+      break;
+  }
+
+  if (!Stmts.empty())
+    return Actions.ActOnTopLevelStmtDecl(Stmts);
+
+  return nullptr;
+}
+
 /// isDeclarationSpecifier() - Return true if the current token is part of a
 /// declaration specifier.
 ///
 /// \param AllowImplicitTypename whether this is a context where T::type [T
 /// dependent] can appear.
 /// \param DisambiguatingWithExpression True to indicate that the purpose of
 /// this check is to disambiguate between an expression and a declaration.
 bool Parser::isDeclarationSpecifier(
     ImplicitTypenameContext AllowImplicitTypename,
     bool DisambiguatingWithExpression) {
   switch (Tok.getKind()) {
   default: return false;
 
   // OpenCL 2.0 and later define this keyword.
   case tok::kw_pipe:
     return getLangOpts().OpenCL &&
            getLangOpts().getOpenCLCompatibleVersion() >= 200;
 
   case tok::identifier:   // foo::bar
     // Unfortunate hack to support "Class.factoryMethod" notation.
     if (getLangOpts().ObjC && NextToken().is(tok::period))
       return false;
     if (TryAltiVecVectorToken())
       return true;
     [[fallthrough]];
   case tok::kw_decltype: // decltype(T())::type
   case tok::kw_typename: // typename T::type
     // Annotate typenames and C++ scope specifiers.  If we get one, just
#include "clang/Basic/OpenCLImageTypes.def"
 
     return true;
 
   case tok::kw_private:
     return getLangOpts().OpenCL;
   }
 }
 
-bool Parser::isConstructorDeclarator(bool IsUnqualified, bool DeductionGuide,
-                                     DeclSpec::FriendSpecified IsFriend) {
+bool Parser::isConstructorDeclarator(
+    bool IsUnqualified, bool DeductionGuide, DeclSpec::FriendSpecified IsFriend,
+    bool DisambiguatingWithExpression /*=false*/) {
   TentativeParsingAction TPA(*this);
 
   // Parse the C++ scope specifier.
   CXXScopeSpec SS;
   if (ParseOptionalCXXScopeSpecifier(SS, /*ObjectType=*/nullptr,
                                      /*ObjectHasErrors=*/false,
                                      /*EnteringContext=*/true)) {
     TPA.Revert();
     return false;
   }
 
+  IdentifierInfo *CtorII = nullptr;
   // Parse the constructor name.
   if (Tok.is(tok::identifier)) {
     // We already know that we have a constructor name; just consume
     // the token.
+    CtorII = Tok.getIdentifierInfo();
     ConsumeToken();
   } else if (Tok.is(tok::annot_template_id)) {
     ConsumeAnnotationToken();
   } else {
     TPA.Revert();
     return false;
   }
 
   // There may be attributes here, appertaining to the constructor name or type
   // we just stepped past.
   SkipCXX11Attributes();
 
   // Current class name must be followed by a left parenthesis.
   if (Tok.isNot(tok::l_paren)) {
     TPA.Revert();
     return false;
   }
   ConsumeParen();
 
+  if (SS.isNotEmpty() && SS.getScopeRep()) {
+    NestedNameSpecifier *NNS = SS.getScopeRep();
+    if (NNS->getAsNamespace() || NNS->getAsNamespaceAlias()) {
+      TPA.Revert();
+      return false;
+    }
+    if (CtorII) {
+      if (NamedDecl *CXXRD = NNS->getAsRecordDecl()) {
+        bool IsCtor = CXXRD->getIdentifier() == CtorII;
+        TPA.Revert();
+        return IsCtor;
+      }
+    }
+  }
+
+  // FIXME: This case is only valid in a member declarator contexts where we
+  // declare a constructor in the class body.
+  if (!DisambiguatingWithExpression) {
     // A right parenthesis, or ellipsis followed by a right parenthesis signals
     // that we have a constructor.
     if (Tok.is(tok::r_paren) ||
         (Tok.is(tok::ellipsis) && NextToken().is(tok::r_paren))) {
       TPA.Revert();
       return true;
     }
+  }
 
   // A C++11 attribute here signals that we have a constructor, and is an
   // attribute on the first constructor parameter.
   if (getLangOpts().CPlusPlus11 &&
       isCXX11AttributeSpecifier(/*Disambiguate*/ false,
                                 /*OuterMightBeMessageSend*/ true)) {
     TPA.Revert();
     return true;

clang/lib/Parse/ParseTentative.cpp

 ///           'using' typename[opt] '::'[opt] nested-name-specifier
 ///                 unqualified-id ';'
 ///           'using' '::' unqualified-id ;
 ///
 ///         using-directive:
 ///           'using' 'namespace' '::'[opt] nested-name-specifier[opt]
 ///                 namespace-name ';'
 ///
-bool Parser::isCXXDeclarationStatement() {
+bool Parser::isCXXDeclarationStatement(
+    bool DisambiguatingWithExpression /*=false*/) {
+  assert(getLangOpts().CPlusPlus && "Must be called for C++ only.");
+  if (DisambiguatingWithExpression) {
+    switch (Tok.getKind()) {
+    case tok::semi: // Not a stmt but can be extra terminator in `namespace{};`
+    case tok::kw_template: // FIXME: Stmts such as `t::template f<int>();`?
+    case tok::kw_inline:
+    case tok::annot_pragma_vis: // FIXME: How to add a generic pragma
+      return true;
+    case tok::identifier:
+      if (isConstructorDeclarator(/*Unqualified=*/false,
+                                  /*DeductionGuide=*/false,
+                                  /*IsFriend=*/DeclSpec::FriendSpecified::No,
+                                  /*DisambiguatingWithExpression=*/true))
+        return true;
+      // Check if this is a dtor.
+      if (!TryAnnotateTypeOrScopeToken() && Tok.is(tok::annot_cxxscope) &&
+          NextToken().is(tok::tilde))
+        return true;
+      break;
+    default:
+      break;
+    }
+  }
+
   switch (Tok.getKind()) {
     // asm-definition
   case tok::kw_asm:
     // namespace-alias-definition
   case tok::kw_namespace:
     // using-declaration
     // using-directive
   case tok::kw_using:
     // static_assert-declaration
   case tok::kw_static_assert:
   case tok::kw__Static_assert:
     return true;
     // simple-declaration
   default:
     return isCXXSimpleDeclaration(/*AllowForRangeDecl=*/false);
   }
 }
 
 /// isCXXSimpleDeclaration - C++-specialized function that disambiguates
 /// between a simple-declaration or an expression-statement.
 /// If during the disambiguation process a parsing error is encountered,
 /// the function returns true to let the declaration parsing code handle it.
 /// Returns false if the statement is disambiguated as expression.
Parser::isCXXDeclarationSpecifier(ImplicitTypenameContext AllowImplicitTypename,
   case tok::kw_enum:
     // cv-qualifier
   case tok::kw_const:
   case tok::kw_volatile:
     return TPResult::True;
 
     // OpenCL address space qualifiers
   case tok::kw_private:
     if (!getLangOpts().OpenCL)
       return TPResult::False;
     [[fallthrough]];
   case tok::kw___private:
   case tok::kw___local:
   case tok::kw___global:
   case tok::kw___constant:
   case tok::kw___generic:
     // OpenCL access qualifiers
   case tok::kw___read_only:

clang/lib/Parse/Parser.cpp

bool Parser::ParseTopLevelDecl(DeclGroupPtrTy &Result,
 
   default:
     break;
   }
 
   ParsedAttributes attrs(AttrFactory);
   MaybeParseCXX11Attributes(attrs);
 
   Result = ParseExternalDeclaration(attrs);
   // An empty Result might mean a line with ';' or some parsing error, ignore
   // it.
   if (Result) {
     if (ImportState == Sema::ModuleImportState::FirstDecl)
       // First decl was not modular.
       ImportState = Sema::ModuleImportState::NotACXX20Module;
     else if (ImportState == Sema::ModuleImportState::ImportAllowed)
       // Non-imports disallow further imports.
       ImportState = Sema::ModuleImportState::ImportFinished;
   }
   return false;
 }
 
 /// ParseExternalDeclaration:
case tok::kw_module:
     return nullptr;
 
   default:
   dont_know:
     if (Tok.isEditorPlaceholder()) {
       ConsumeToken();
       return nullptr;
     }
+
+    SingleDecl = ParseTopLevelStmtDecl();
+
     // We can't tell whether this is a function-definition or declaration yet.
+    if (!SingleDecl)
       return ParseDeclarationOrFunctionDefinition(Attrs, DS);
   }
 
   // This routine returns a DeclGroup, if the thing we parsed only contains a
   // single decl, convert it now.
   return Actions.ConvertDeclToDeclGroup(SingleDecl);
 }
 
 /// Determine whether the current token, if it occurs after a

clang/lib/Sema/SemaDecl.cpp

 
   FileScopeAsmDecl *New = FileScopeAsmDecl::Create(Context, CurContext,
                                                    AsmString, StartLoc,
                                                    EndLoc);
   CurContext->addDecl(New);
   return New;
 }
 
+Decl *Sema::ActOnTopLevelStmtDecl(const SmallVectorImpl<Stmt *> &Stmts) {
+  TopLevelStmtDecl *New = TopLevelStmtDecl::Create(Context, Stmts);
+  Context.getTranslationUnitDecl()->addDecl(New);
+  return New;
+}
+
 void Sema::ActOnPragmaRedefineExtname(IdentifierInfo* Name,
                                       IdentifierInfo* AliasName,
                                       SourceLocation PragmaLoc,
                                       SourceLocation NameLoc,
                                       SourceLocation AliasNameLoc) {
   NamedDecl *PrevDecl = LookupSingleName(TUScope, Name, NameLoc,
                                          LookupOrdinaryName);
   AttributeCommonInfo Info(AliasName, SourceRange(AliasNameLoc),

                       getASTContext().GetGVALinkageForFunction(Def));
   };
 
   if (LangOpts.OpenMPIsDevice) {
     // In OpenMP device mode we will not emit host only functions, or functions
     // we don't need due to their linkage.
     Optional<OMPDeclareTargetDeclAttr::DevTypeTy> DevTy =
         OMPDeclareTargetDeclAttr::getDeviceType(FD->getCanonicalDecl());
     // DevTy may be changed later by
     //  #pragma omp declare target to(*) device_type(*).
     // Therefore DevTy having no value does not imply host. The emission status
     // will be checked again at the end of compilation unit with Final = true.
     if (DevTy)
       if (*DevTy == OMPDeclareTargetDeclAttr::DT_Host)
         return FunctionEmissionStatus::OMPDiscarded;
     // If we have an explicit value for the device type, or we are in a target
     // declare context, we need to emit all extern and used symbols.

clang/lib/Serialization/ASTCommon.cpp

bool serialization::isRedeclarableDeclKind(unsigned Kind) {
   case Decl::ObjCProperty:
   case Decl::ObjCCompatibleAlias:
   case Decl::LinkageSpec:
   case Decl::Export:
   case Decl::ObjCPropertyImpl:
   case Decl::PragmaComment:
   case Decl::PragmaDetectMismatch:
   case Decl::FileScopeAsm:
+  case Decl::TopLevelStmt:
   case Decl::AccessSpec:
   case Decl::Friend:
   case Decl::FriendTemplate:
   case Decl::StaticAssert:
   case Decl::Block:
   case Decl::Captured:
   case Decl::ClassScopeFunctionSpecialization:
   case Decl::Import:

clang/lib/Serialization/ASTReaderDecl.cpp

public:
     void VisitUsingDecl(UsingDecl *D);
     void VisitUsingEnumDecl(UsingEnumDecl *D);
     void VisitUsingPackDecl(UsingPackDecl *D);
     void VisitUsingShadowDecl(UsingShadowDecl *D);
     void VisitConstructorUsingShadowDecl(ConstructorUsingShadowDecl *D);
     void VisitLinkageSpecDecl(LinkageSpecDecl *D);
     void VisitExportDecl(ExportDecl *D);
     void VisitFileScopeAsmDecl(FileScopeAsmDecl *AD);
+    void VisitTopLevelStmtDecl(TopLevelStmtDecl *D);
     void VisitImportDecl(ImportDecl *D);
     void VisitAccessSpecDecl(AccessSpecDecl *D);
     void VisitFriendDecl(FriendDecl *D);
     void VisitFriendTemplateDecl(FriendTemplateDecl *D);
     void VisitStaticAssertDecl(StaticAssertDecl *D);
     void VisitBlockDecl(BlockDecl *BD);
     void VisitCapturedDecl(CapturedDecl *CD);
     void VisitEmptyDecl(EmptyDecl *D);

 }
 
 void ASTDeclReader::VisitFileScopeAsmDecl(FileScopeAsmDecl *AD) {
   VisitDecl(AD);
   AD->setAsmString(cast<StringLiteral>(Record.readExpr()));
   AD->setRParenLoc(readSourceLocation());
 }
 
+void ASTDeclReader::VisitTopLevelStmtDecl(TopLevelStmtDecl *D) {
+  VisitDecl(D);
+  unsigned NumStmts = Record.readInt();
+  llvm::SmallVector<Stmt *, 32> Statements;
+  Statements.reserve(NumStmts);
+  for (unsigned I = 0; I != NumStmts; ++I)
+    Statements.push_back(Record.readStmt());
+  D->setStmts(Reader.getContext(), llvm::makeArrayRef(Statements));
+}
+
 void ASTDeclReader::VisitBlockDecl(BlockDecl *BD) {
   VisitDecl(BD);
   BD->setBody(cast_or_null<CompoundStmt>(Record.readStmt()));
   BD->setSignatureAsWritten(readTypeSourceInfo());
   unsigned NumParams = Record.readInt();
   SmallVector<ParmVarDecl *, 16> Params;
   Params.reserve(NumParams);
   for (unsigned I = 0; I != NumParams; ++I)
static bool isConsumerInterestedIn(ASTContext &Ctx, Decl *D, bool HasBody) {
   // emitted when we import the relevant module.
   if (isPartOfPerModuleInitializer(D)) {
     auto *M = D->getImportedOwningModule();
     if (M && M->Kind == Module::ModuleMapModule &&
         Ctx.DeclMustBeEmitted(D))
       return false;
   }
 
-  if (isa<FileScopeAsmDecl>(D) ||
-      isa<ObjCProtocolDecl>(D) ||
-      isa<ObjCImplDecl>(D) ||
-      isa<ImportDecl>(D) ||
-      isa<PragmaCommentDecl>(D) ||
-      isa<PragmaDetectMismatchDecl>(D))
+  if (isa<FileScopeAsmDecl>(D) || isa<TopLevelStmtDecl>(D) ||
+      isa<ObjCProtocolDecl>(D) || isa<ObjCImplDecl>(D) || isa<ImportDecl>(D) ||
+      isa<PragmaCommentDecl>(D) || isa<PragmaDetectMismatchDecl>(D))
     return true;
   if (isa<OMPThreadPrivateDecl>(D) || isa<OMPDeclareReductionDecl>(D) ||
       isa<OMPDeclareMapperDecl>(D) || isa<OMPAllocateDecl>(D) ||
       isa<OMPRequiresDecl>(D))
     return !D->getDeclContext()->isFunctionOrMethod();
   if (const auto *Var = dyn_cast<VarDecl>(D))
     return Var->isFileVarDecl() &&
            (Var->isThisDeclarationADefinition() == VarDecl::Definition ||
case DECL_DECOMPOSITION:
     D = DecompositionDecl::CreateDeserialized(Context, ID, Record.readInt());
     break;
   case DECL_BINDING:
     D = BindingDecl::CreateDeserialized(Context, ID);
     break;
   case DECL_FILE_SCOPE_ASM:
     D = FileScopeAsmDecl::CreateDeserialized(Context, ID);
     break;
+  case DECL_TOP_LEVEL_STMT_DECL:
+    D = TopLevelStmtDecl::CreateDeserialized(Context, ID);
+    break;
   case DECL_BLOCK:
     D = BlockDecl::CreateDeserialized(Context, ID);
     break;
   case DECL_MS_PROPERTY:
     D = MSPropertyDecl::CreateDeserialized(Context, ID);
     break;
   case DECL_MS_GUID:
     D = MSGuidDecl::CreateDeserialized(Context, ID);

clang/lib/Serialization/ASTWriterDecl.cpp

public:
     void VisitUsingDecl(UsingDecl *D);
     void VisitUsingEnumDecl(UsingEnumDecl *D);
     void VisitUsingPackDecl(UsingPackDecl *D);
     void VisitUsingShadowDecl(UsingShadowDecl *D);
     void VisitConstructorUsingShadowDecl(ConstructorUsingShadowDecl *D);
     void VisitLinkageSpecDecl(LinkageSpecDecl *D);
     void VisitExportDecl(ExportDecl *D);
     void VisitFileScopeAsmDecl(FileScopeAsmDecl *D);
+    void VisitTopLevelStmtDecl(TopLevelStmtDecl *D);
     void VisitImportDecl(ImportDecl *D);
     void VisitAccessSpecDecl(AccessSpecDecl *D);
     void VisitFriendDecl(FriendDecl *D);
     void VisitFriendTemplateDecl(FriendTemplateDecl *D);
     void VisitStaticAssertDecl(StaticAssertDecl *D);
     void VisitBlockDecl(BlockDecl *D);
     void VisitCapturedDecl(CapturedDecl *D);
     void VisitEmptyDecl(EmptyDecl *D);

 
 void ASTDeclWriter::VisitFileScopeAsmDecl(FileScopeAsmDecl *D) {
   VisitDecl(D);
   Record.AddStmt(D->getAsmString());
   Record.AddSourceLocation(D->getRParenLoc());
   Code = serialization::DECL_FILE_SCOPE_ASM;
 }
 
+void ASTDeclWriter::VisitTopLevelStmtDecl(TopLevelStmtDecl *D) {
+  VisitDecl(D);
+  Record.push_back(D->statements().size());
+  for (Stmt *S : D->statements())
+    Record.AddStmt(S);
+  Code = serialization::DECL_TOP_LEVEL_STMT_DECL;
+}
+
 void ASTDeclWriter::VisitEmptyDecl(EmptyDecl *D) {
   VisitDecl(D);
   Code = serialization::DECL_EMPTY;
 }
 
 void ASTDeclWriter::VisitLifetimeExtendedTemporaryDecl(
     LifetimeExtendedTemporaryDecl *D) {
   VisitDecl(D);

 /// decls.
 static bool isRequiredDecl(const Decl *D, ASTContext &Context,
                            bool WritingModule) {
   // An ObjCMethodDecl is never considered as "required" because its
   // implementation container always is.
 
   // File scoped assembly or obj-c or OMP declare target implementation must be
   // seen.
-  if (isa<FileScopeAsmDecl>(D) || isa<ObjCImplDecl>(D))
+  if (isa<FileScopeAsmDecl>(D) || isa<TopLevelStmtDecl>(D) ||
+      isa<ObjCImplDecl>(D))
     return true;
 
   if (WritingModule && isPartOfPerModuleInitializer(D)) {
     // These declarations are part of the module initializer, and are emitted
     // if and when the module is imported, rather than being emitted eagerly.
     return false;
   }
 

clang/test/Interpreter/disambiguate-decl-stmt.cpp

+// REQUIRES: host-supports-jit
+// UNSUPPORTED: system-aix
+// RUN: cat %s | clang-repl -Xcc -Xclang -Xcc -verify | FileCheck %s
+// RUN: %clang_cc1 -fsyntax-only -verify -fincremental-extensions %s
+
+// expected-no-diagnostics
+
+extern "C" int printf(const char*,...);
+
+// Decls which are hard to disambiguate
+
+// FIXME: Support operators.
+// struct S1 { operator int(); };
+// S1::operator int() { return 0; }
+
+
+// Dtors
+using I = int;
+I x = 10;
+x.I::~I();
+x = 20;
+
+// Ctors
+
+// FIXME: Support deduction guide
+// template<typename T> struct A { A(); A(T); };
+// A() -> A<int>;
+
+struct S2 { S2(); };
+S2::S2() = default;
+
+namespace N { struct S { S(); }; }
+N::S::S() { printf("N::S::S()\n"); }
+N::S s;
+// CHECK: N::S::S()
+
+namespace Ns {namespace Ns { void Ns(); void Fs();}}
+void Ns::Ns::Ns() { printf("void Ns::Ns::Ns()\n"); }
+void Ns::Ns::Fs() {}
+
+Ns::Ns::Fs();
+Ns::Ns::Ns();
+// CHECK-NEXT: void Ns::Ns::Ns()
+
+struct Attrs1 { Attrs1(); };
+Attrs1::Attrs1() __attribute((pure)) = default;
+
+struct Attrs2 { Attrs2(); };
+__attribute((pure)) Attrs2::Attrs2() = default;
+
+// Extra semicolon
+namespace N {};

clang/test/Interpreter/execute-stmts.cpp

+// REQUIRES: host-supports-jit
+// UNSUPPORTED: system-aix
+// RUN: cat %s | clang-repl -Xcc -Xclang -Xcc  -verify | FileCheck %s
+// RUN: %clang_cc1 -fsyntax-only -verify -fincremental-extensions %s
+
+// expected-no-diagnostics
+
+extern "C" int printf(const char*,...);
+
+// FIXME: Support template instantiation calls.
+//template <typename T> T call() { printf("call\n"); return T(); }
+//call<int>();
+//// C: call
+
+int i = 1;
+++i;
+printf("i = %d\n", i);
+// CHECK: i = 2
+
+namespace Ns { void f(){ i++; } }
+Ns::f();
+
+void g() { ++i; }
+g();
+::g();
+
+printf("i = %d\n", i);
+// CHECK-NEXT: i = 5
+
+for (; i > 4; --i) printf("i = %d\n", i);
+// CHECK-NEXT: i = 5
+
+int j = i; printf("j = %d\n", j);
+// CHECK-NEXT: j = 4

clang/test/Interpreter/stmt-serialization.cpp

+// RUN: rm -rf %t
+// RUN: %clang_cc1 -std=c++20 -fincremental-extensions -fmodules-cache-path=%t \
+// RUN:            -x c++ %s -verify
+// expected-no-diagnostics
+
+#pragma clang module build TopLevelStmt
+module TopLevelStmt { module Statements {} }
+#pragma clang module contents
+
+#pragma clang module begin TopLevelStmt.Statements
+extern "C" int printf(const char*,...);
+int i = 0;
+i++;
+#pragma clang module end /*TopLevelStmt.Statements*/
+#pragma clang module endbuild /*TopLevelStmt*/
+
+#pragma clang module import TopLevelStmt.Statements
+
+printf("Value of i is '%d'", i);

clang/unittests/Interpreter/InterpreterTest.cpp

TEST(InterpreterTest, DeclsAndStatements) {
   // gtest doesn't expand into explicit bool conversions.
   EXPECT_TRUE(!!R1);
 
   auto *PTU1 = R1->TUPart;
   EXPECT_EQ(2U, DeclsSize(PTU1));
 
   // FIXME: Add support for wrapping and running statements.
   auto R2 = Interp->Parse("var1++; printf(\"var1 value %d\\n\", var1);");
-  EXPECT_FALSE(!!R2);
-  using ::testing::HasSubstr;
-  EXPECT_THAT(DiagnosticsOS.str(),
-              HasSubstr("error: unknown type name 'var1'"));
-  auto Err = R2.takeError();
-  EXPECT_EQ("Parsing failed.", llvm::toString(std::move(Err)));
+  EXPECT_TRUE(!!R2);
 }
 
 TEST(InterpreterTest, UndoCommand) {
   Args ExtraArgs = {"-Xclang", "-diagnostic-log-file", "-Xclang", "-"};
 
   // Create the diagnostic engine with unowned consumer.
   std::string DiagnosticOutput;
   llvm::raw_string_ostream DiagnosticsOS(DiagnosticOutput);