[lldb] Allow evaluating expressions in C++20 mode

[Michael137]

Summary

This patch allows users to evaluate expressions using
expr -l c++20. Currently DWARF keeps the CU's at
DW_AT_language at DW_LANG_C_plus_plus_14 even
when compiling with -std=c++20. So even in "C++20
programs" expression evaluation will by default be
performed in C++11 mode for now.

Enabling C++14 has been previously attempted at
https://reviews.llvm.org/D80308

There are some remaining issues around evaluating C++20
expressions. Mainly, lack of support for C++20 AST nodes in
clang::ASTImporter. But these can be addressed in follow-up
patches.

[Michael137]

• Remove redundant change

[aprantl]

lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp

522

Why no case C++17?

525

Why does this not set C++14? Because it's effectively equivalent?

[aprantl]

lldb/source/Target/Language.cpp

271

Can you check if we already have a similar function in Dwarf.h in LLVM?

[Michael137]

lldb/source/Target/Language.cpp

271

we do in fact :)

dwarf::isCplusPlus

The only remaining quirk is that we one of the language constants (DW_LANG_Mips_Assembler) doesn't match the DWARF constants. Which isn't really a problem for this particular function

[Michael137]

lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp

525

I think it's because we haven't done the work to make c++14 in the expression evaluator work, but the default clang language is c++14. So we downgrade it silently to c++11 here.

We should really fix that, but I think this broke some tests in the past when we tried enabling it (see 8cb75745412e4bc9592d2409cc6cfa4a2940d9e7, https://reviews.llvm.org/D80308)

[Michael137]

lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp

525

But looking at the specific test failures from back then there doesn't seem like a good reason for why we can't re-enable that patch now. And also add a C++17 version.

Let me try that

[Michael137]

lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp

522

Fair point, can add one

[Michael137]

• Add FIXME

[JDevlieghere]

lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp

518

s/evaluatino/evaluation/

lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionParser.cpp

case lldb::eLanguageTypeObjC:
     // Clang now sets as default C++14 as the default standard (with
     // GNU extensions), so we do the same here to avoid mismatches that
     // cause compiler error when evaluating expressions (e.g. nullptr not found
     // as it's a C++11 feature). Currently lldb evaluates C++14 as C++11 (see
     // two lines below) so we decide to be consistent with that, but this could
     // be re-evaluated in the future.
     lang_opts.CPlusPlus11 = true;
     break;
+  case lldb::eLanguageTypeC_plus_plus_20:
+    lang_opts.CPlusPlus20 = true;
+    LLVM_FALLTHROUGH;
+  case lldb::eLanguageTypeC_plus_plus_17:
+    // FIXME: add a separate case for CPlusPlus14. Currently folded into C++17
+    // because C++14 is the default standard for Clang but enabling CPlusPlus14
+    // expression evaluatino doesn't pass the test-suite cleanly.
+    lang_opts.CPlusPlus14 = true;
+    lang_opts.CPlusPlus17 = true;
+    LLVM_FALLTHROUGH;
   case lldb::eLanguageTypeC_plus_plus:
   case lldb::eLanguageTypeC_plus_plus_11:
   case lldb::eLanguageTypeC_plus_plus_14:
     lang_opts.CPlusPlus11 = true;
     m_compiler->getHeaderSearchOpts().UseLibcxx = true;
     [[fallthrough]];
   case lldb::eLanguageTypeC_plus_plus_03:
     lang_opts.CPlusPlus = true;
     if (process_sp)
       lang_opts.ObjC =
           process_sp->GetLanguageRuntime(lldb::eLanguageTypeObjC) != nullptr;
     break;

lldb/source/Plugins/ExpressionParser/Clang/ClangExpressionSourceCode.cpp

TokenVerifier::TokenVerifier(std::string body) {
 
   FileID FID = SM.createFileID(buf->getMemBufferRef());
 
   // Let's just enable the latest ObjC and C++ which should get most tokens
   // right.
   LangOptions Opts;
   Opts.ObjC = true;
   Opts.DollarIdents = true;
-  Opts.CPlusPlus17 = true;
+  Opts.CPlusPlus20 = true;
   Opts.LineComment = true;
 
   Lexer lex(FID, buf->getMemBufferRef(), SM, Opts);
 
   Token token;
   bool exit = false;
   while (!exit) {
     // Returns true if this is the last token we get from the lexer.

lldb/source/Plugins/Language/CPlusPlus/CPlusPlusNameParser.cpp

static const clang::LangOptions &GetLangOptions() {
   llvm::call_once(g_once_flag, []() {
     g_options.LineComment = true;
     g_options.C99 = true;
     g_options.C11 = true;
     g_options.CPlusPlus = true;
     g_options.CPlusPlus11 = true;
     g_options.CPlusPlus14 = true;
     g_options.CPlusPlus17 = true;
+    g_options.CPlusPlus20 = true;
   });
   return g_options;
 }
 
 static const llvm::StringMap<tok::TokenKind> &GetKeywordsMap() {
   static llvm::StringMap<tok::TokenKind> g_map{
 #define KEYWORD(Name, Flags) {llvm::StringRef(#Name), tok::kw_##Name},
 #include "clang/Basic/TokenKinds.def"

lldb/source/Plugins/TypeSystem/Clang/TypeSystemClang.cpp

op_kind = StringSwitch<clang::OverloadedOperatorKind>(name)
                 .Case("|=", clang::OO_PipeEqual)
                 .Case("||", clang::OO_PipePipe)
                 .Case("~", clang::OO_Tilde)
                 .Case("!", clang::OO_Exclaim)
                 .Case("!=", clang::OO_ExclaimEqual)
                 .Case("=", clang::OO_Equal)
                 .Case("==", clang::OO_EqualEqual)
                 .Case("<", clang::OO_Less)
+                .Case("<=>", clang::OO_Spaceship)
                 .Case("<<", clang::OO_LessLess)
                 .Case("<<=", clang::OO_LessLessEqual)
                 .Case("<=", clang::OO_LessEqual)
                 .Case(">", clang::OO_Greater)
                 .Case(">>", clang::OO_GreaterGreater)
                 .Case(">>=", clang::OO_GreaterGreaterEqual)
                 .Case(">=", clang::OO_GreaterEqual)
                 .Case("()", clang::OO_Call)
if (LangStd == LangStandard::lang_unspecified) {
     }
   }
 
   const LangStandard &Std = LangStandard::getLangStandardForKind(LangStd);
   Opts.LineComment = Std.hasLineComments();
   Opts.C99 = Std.isC99();
   Opts.CPlusPlus = Std.isCPlusPlus();
   Opts.CPlusPlus11 = Std.isCPlusPlus11();
+  Opts.CPlusPlus14 = Std.isCPlusPlus14();
+  Opts.CPlusPlus17 = Std.isCPlusPlus17();
+  Opts.CPlusPlus20 = Std.isCPlusPlus20();
   Opts.Digraphs = Std.hasDigraphs();
   Opts.GNUMode = Std.isGNUMode();
   Opts.GNUInline = !Std.isC99();
   Opts.HexFloats = Std.hasHexFloats();
 
   Opts.WChar = true;
 
   // OpenCL has some additional defaults.
LanguageSet TypeSystemClang::GetSupportedLanguagesForTypes() {
   languages.Insert(lldb::eLanguageTypeC_plus_plus);
   languages.Insert(lldb::eLanguageTypeC99);
   languages.Insert(lldb::eLanguageTypeObjC);
   languages.Insert(lldb::eLanguageTypeObjC_plus_plus);
   languages.Insert(lldb::eLanguageTypeC_plus_plus_03);
   languages.Insert(lldb::eLanguageTypeC_plus_plus_11);
   languages.Insert(lldb::eLanguageTypeC11);
   languages.Insert(lldb::eLanguageTypeC_plus_plus_14);
+  languages.Insert(lldb::eLanguageTypeC_plus_plus_17);
+  languages.Insert(lldb::eLanguageTypeC_plus_plus_20);
   return languages;
 }
 
 LanguageSet TypeSystemClang::GetSupportedLanguagesForExpressions() {
   LanguageSet languages;
   languages.Insert(lldb::eLanguageTypeC_plus_plus);
   languages.Insert(lldb::eLanguageTypeObjC_plus_plus);
   languages.Insert(lldb::eLanguageTypeC_plus_plus_03);
   languages.Insert(lldb::eLanguageTypeC_plus_plus_11);
   languages.Insert(lldb::eLanguageTypeC_plus_plus_14);
+  languages.Insert(lldb::eLanguageTypeC_plus_plus_17);
+  languages.Insert(lldb::eLanguageTypeC_plus_plus_20);
   return languages;
 }
 
 void TypeSystemClang::Initialize() {
   PluginManager::RegisterPlugin(
       GetPluginNameStatic(), "clang base AST context plug-in", CreateInstance,
       GetSupportedLanguagesForTypes(), GetSupportedLanguagesForExpressions());
 }

lldb/source/Target/Language.cpp

 }
 
 bool Language::LanguageIsCPlusPlus(LanguageType language) {
   switch (language) {
   case eLanguageTypeC_plus_plus:
   case eLanguageTypeC_plus_plus_03:
   case eLanguageTypeC_plus_plus_11:
   case eLanguageTypeC_plus_plus_14:
+  case eLanguageTypeC_plus_plus_17:
+  case eLanguageTypeC_plus_plus_20:
   case eLanguageTypeObjC_plus_plus:
     return true;
   default:
     return false;
   }
 }
 
 bool Language::LanguageIsObjC(LanguageType language) {
bool Language::LanguageIsCFamily(LanguageType language) {
   case eLanguageTypeC:
   case eLanguageTypeC89:
   case eLanguageTypeC99:
   case eLanguageTypeC11:
   case eLanguageTypeC_plus_plus:
   case eLanguageTypeC_plus_plus_03:
   case eLanguageTypeC_plus_plus_11:
   case eLanguageTypeC_plus_plus_14:
+  case eLanguageTypeC_plus_plus_17:
+  case eLanguageTypeC_plus_plus_20:
   case eLanguageTypeObjC_plus_plus:
   case eLanguageTypeObjC:
     return true;
   default:
     return false;
   }
 }
 
 bool Language::LanguageIsPascal(LanguageType language) {
   switch (language) {
   case eLanguageTypePascal83:
     return true;
   default:
     return false;
   }
 }
 
 LanguageType Language::GetPrimaryLanguage(LanguageType language) {
   switch (language) {
   case eLanguageTypeC_plus_plus:
   case eLanguageTypeC_plus_plus_03:
   case eLanguageTypeC_plus_plus_11:
   case eLanguageTypeC_plus_plus_14:
+  case eLanguageTypeC_plus_plus_17:
+  case eLanguageTypeC_plus_plus_20:
     return eLanguageTypeC_plus_plus;
   case eLanguageTypeC:
   case eLanguageTypeC89:
   case eLanguageTypeC99:
   case eLanguageTypeC11:
     return eLanguageTypeC;
   case eLanguageTypeObjC:
   case eLanguageTypeObjC_plus_plus:

lldb/test/API/lang/cpp/standards/cpp20/Makefile

+CXX_SOURCES := main.cpp
+CXXFLAGS_EXTRAS := -std=c++20
+
+include Makefile.rules

lldb/test/API/lang/cpp/standards/cpp20/TestCPP20Standard.py

+import lldb
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+
+class TestCPP20Standard(TestBase):
+    def test_cpp20(self):
+        """
+        Tests that we can evaluate an expression in C++20 mode
+        """
+        self.build()
+        lldbutil.run_to_source_breakpoint(self, "Foo{}", lldb.SBFileSpec("main.cpp"))
+
+        self.expect("expr -l c++11 -- Foo{} <=> Foo{}", error=True, substrs=["'<=>' is a single token in C++20; add a space to avoid a change in behavior"])
+
+        self.expect("expr -l c++20 -- Foo{} <=> Foo{}", substrs=["(bool) $0 = true"])

lldb/test/API/lang/cpp/standards/cpp20/main.cpp

+#include <compare>
+
+struct Foo {
+  friend auto operator<=>(Foo const &, Foo const &) { return true; }
+};
+
+int main() { return Foo{} <=> Foo{}; }

lldb/unittests/Language/CPlusPlus/CPlusPlusLanguageTest.cpp

TestCase test_cases[] = {
        "std::__1::error_code::operator bool[abi:v160000]"},
 
       {"auto ns::foo::operator[][abi:v160000](size_t) const", "auto", "ns::foo",
        "operator[][abi:v160000]", "(size_t)", "const",
        "ns::foo::operator[][abi:v160000]"},
 
       {"auto Foo[abi:abc]<int>::operator<<<Foo[abi:abc]<int>>(int) &", "auto",
        "Foo[abi:abc]<int>", "operator<<<Foo[abi:abc]<int>>", "(int)", "&",
-       "Foo[abi:abc]<int>::operator<<<Foo[abi:abc]<int>>"}};
+       "Foo[abi:abc]<int>::operator<<<Foo[abi:abc]<int>>"},
+
+      {"auto A::operator<=>[abi:tag]<A::B>()", "auto", "A",
+       "operator<=>[abi:tag]<A::B>", "()", "",
+       "A::operator<=>[abi:tag]<A::B>"}};
 
   for (const auto &test : test_cases) {
     CPlusPlusLanguage::MethodName method(ConstString(test.input));
     EXPECT_TRUE(method.IsValid()) << test.input;
     if (method.IsValid()) {
       EXPECT_EQ(test.return_type, method.GetReturnType().str());
       EXPECT_EQ(test.context, method.GetContext().str());
       EXPECT_EQ(test.basename, method.GetBasename().str());
       EXPECT_EQ(test.arguments, method.GetArguments().str());
       EXPECT_EQ(test.qualifiers, method.GetQualifiers().str());
       EXPECT_EQ(test.scope_qualified_name, method.GetScopeQualifiedName());
     }
   }
 }
 
 TEST(CPlusPlusLanguage, InvalidMethodNameParsing) {
   // Tests that we correctly reject malformed function names
 
   std::string test_cases[] = {
       "int Foo::operator[]<[10>()",
       "Foo::operator bool[10]()",
-      "auto A::operator<=>[abi:tag]<A::B>()",
       "auto A::operator<<<(int)",
       "auto A::operator>>>(int)",
       "auto A::operator<<<Type[abi:tag]<>(int)",
       "auto A::operator<<<Type[abi:tag]<Type<int>>(int)",
       "auto A::foo[(int)",
       "auto A::foo[](int)",
       "auto A::foo[bar](int)",
       "auto A::foo[abi](int)",
EXPECT_FALSE(CPlusPlusLanguage::ExtractContextAndIdentifier(
       "selector:", context, basename));
   EXPECT_FALSE(CPlusPlusLanguage::ExtractContextAndIdentifier(
       "selector:otherField:", context, basename));
   EXPECT_FALSE(CPlusPlusLanguage::ExtractContextAndIdentifier(
       "abc::", context, basename));
   EXPECT_FALSE(CPlusPlusLanguage::ExtractContextAndIdentifier(
       "f<A<B><C>>", context, basename));
 
-  // We expect these cases to fail until we turn on C++2a
-  EXPECT_FALSE(CPlusPlusLanguage::ExtractContextAndIdentifier(
+  EXPECT_TRUE(CPlusPlusLanguage::ExtractContextAndIdentifier(
       "A::operator<=><A::B>", context, basename));
-  EXPECT_FALSE(CPlusPlusLanguage::ExtractContextAndIdentifier(
+  EXPECT_TRUE(CPlusPlusLanguage::ExtractContextAndIdentifier(
       "operator<=><A::B>", context, basename));
 }
 
 static std::vector<std::string> GenerateAlternate(llvm::StringRef Name) {
   std::vector<std::string> Strings;
   if (Language *CPlusPlusLang =
           Language::FindPlugin(lldb::eLanguageTypeC_plus_plus)) {
     std::vector<ConstString> Results =