NFC: Reorganize the demangler a bit

[erik.pilkington]

Summary

With this patch, the copies of the files ItaniumDemangle.h, StringView.h, and Utility.h are kept byte-for-byte in sync between libcxxabi and llvm. All differences (namespaces, fallthrough, and unreachable macros) are defined in each copies' DemanglerConfig.h. This patch also adds a script to copy changes from libcxxabi (cp-to-llvm.sh), and a README.txt explaining the situation.

Thanks for taking a look!

[dexonsmith]

This is awesome.

1. Should we document in the README.txt that we want to split "demangle" out to a separate top-level directory some time after the monorepo move?
2. I suggest adding a bot that runs the script and confirms that git status is clean.

[erik.pilkington]

1. Should we document in the README.txt that we want to split "demangle" out to a separate top-level directory some time after the monorepo move?

Sure, new patch adds this.

2. I suggest adding a bot that runs the script and confirms that git status is clean.

That's a good idea. I don't know how to go about doing that, but it seems like it should be pretty straightforward.

[andrew.w.kaylor]

llvm/include/llvm/Demangle/DemangleConfig.h

75 ↗

(On Diff #170738)

Is DEMANGLE_ENABLE_DUMP ever going to be defined? LLVM_ENABLE_DUMP gets defined in llvm-config.h in response to a CMake variable. I don't see anything analogous here.

[labath]

I like this change a lot. I was hoping something like this could be done, when I saw that we were copying patches around.

(Not clicking accept as I don't feel like an owner here.)

libcxxabi/src/demangle/README.txt

51 ↗

(On Diff #170738)

Use an impersonal form here? "It is also highly recommended...", or at least change "I" to "we".

[erik.pilkington]

Use an impersonal tone in the README, remove DEMANGLE_ENABLE_DUMP.

libcxxabi/src/demangle/README.txt

51 ↗

(On Diff #170738)

Sure, fixed in the update.

llvm/include/llvm/Demangle/DemangleConfig.h

75 ↗

(On Diff #170738)

Ah, no it's not, sorry! Just went through this file too quickly I guess.

[erik.pilkington]

I think I'll just land this, I have some dependent work that I want to do.

libcxxabi/trunk/src/cxa_demangle.cpp

 //===-------------------------- cxa_demangle.cpp --------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 // FIXME: (possibly) incomplete list of features that clang mangles that this
 // file does not yet support:
 //   - C++ modules TS
 
-#define _LIBCPP_NO_EXCEPTIONS
-
-#include "__cxxabi_config.h"
-
 #include "demangle/ItaniumDemangle.h"
-
+#include "__cxxabi_config.h"
 #include <cassert>
 #include <cctype>
 #include <cstdio>
 #include <cstdlib>
 #include <cstring>
 #include <functional>
 #include <numeric>
 #include <utility>
-#include <vector>
 
 using namespace itanium_demangle;
 
 constexpr const char *itanium_demangle::FloatData<float>::spec;
 constexpr const char *itanium_demangle::FloatData<double>::spec;
 constexpr const char *itanium_demangle::FloatData<long double>::spec;
 
 // <discriminator> := _ <non-negative number>      # when number < 10

libcxxabi/trunk/src/demangle/Compiler.h

-//===--- Compiler.h ---------------------------------------------*- C++ -*-===//
-//
-//                     The LLVM Compiler Infrastructure
-//
-// This file is distributed under the University of Illinois Open Source
-// License. See LICENSE.TXT for details.
-//
-// This file is contains a subset of macros copied from
-// llvm/lib/Demangle/Compiler.h.
-//===----------------------------------------------------------------------===//
-
-#ifndef LIBCXX_DEMANGLE_COMPILER_H
-#define LIBCXX_DEMANGLE_COMPILER_H
-
-#ifdef _MSC_VER
-// snprintf is implemented in VS 2015
-#if _MSC_VER < 1900
-#define snprintf _snprintf_s
-#endif
-#endif
-
-#ifndef __has_attribute
-#define __has_attribute(x) 0
-#endif
-
-#ifndef NDEBUG
-#if __has_attribute(noinline) && __has_attribute(used)
-#define DUMP_METHOD __attribute__((noinline, used))
-#else
-#define DUMP_METHOD
-#endif
-#endif
-
-#endif

libcxxabi/trunk/src/demangle/DemangleConfig.h

+//===--- Compiler.h ---------------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+// This file is contains a subset of macros copied from
+// llvm/lib/Demangle/Compiler.h.
+//===----------------------------------------------------------------------===//
+
+#ifndef LIBCXX_DEMANGLE_COMPILER_H
+#define LIBCXX_DEMANGLE_COMPILER_H
+
+#include "__config"
+
+#ifdef _MSC_VER
+// snprintf is implemented in VS 2015
+#if _MSC_VER < 1900
+#define snprintf _snprintf_s
+#endif
+#endif
+
+#ifndef __has_attribute
+#define __has_attribute(x) 0
+#endif
+
+#ifndef NDEBUG
+#if __has_attribute(noinline) && __has_attribute(used)
+#define DEMANGLE_DUMP_METHOD __attribute__((noinline, used))
+#else
+#define DEMANGLE_DUMP_METHOD
+#endif
+#endif
+
+#define DEMANGLE_FALLTHROUGH _LIBCPP_FALLTHROUGH()
+#define DEMANGLE_UNREACHABLE _LIBCPP_UNREACHABLE()
+
+#define DEMANGLE_NAMESPACE_BEGIN namespace { namespace itanium_demangle {
+#define DEMANGLE_NAMESPACE_END } }
+
+#endif

libcxxabi/trunk/src/demangle/ItaniumDemangle.h

 //===------------------------- ItaniumDemangle.h ----------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
-// WARNING: This file defines its contents within an anonymous namespace. It
-// should not be included anywhere other than cxa_demangle.h.
+// Generic itanium demangler library. This file has two byte-per-byte identical
+// copies in the source tree, one in libcxxabi, and the other in llvm.
 //
 //===----------------------------------------------------------------------===//
 
-#ifndef LIBCXX_DEMANGLE_ITANIUMDEMANGLE_H
-#define LIBCXX_DEMANGLE_ITANIUMDEMANGLE_H
+#ifndef DEMANGLE_ITANIUMDEMANGLE_H
+#define DEMANGLE_ITANIUMDEMANGLE_H
 
 // FIXME: (possibly) incomplete list of features that clang mangles that this
 // file does not yet support:
 //   - C++ modules TS
 
-#include "Compiler.h"
+#include "DemangleConfig.h"
 #include "StringView.h"
 #include "Utility.h"
-
 #include <cassert>
 #include <cctype>
 #include <cstdio>
 #include <cstdlib>
 #include <cstring>
 #include <numeric>
 #include <utility>
 
#define FOR_EACH_NODE_KIND(X) \
     X(IntegerCastExpr) \
     X(IntegerLiteral) \
     X(FloatLiteral) \
     X(DoubleLiteral) \
     X(LongDoubleLiteral) \
     X(BracedExpr) \
     X(BracedRangeExpr)
 
-namespace {
-namespace itanium_demangle {
+DEMANGLE_NAMESPACE_BEGIN
+
 // Base class of all AST nodes. The AST is built by the parser, then is
 // traversed by the printLeft/Right functions to produce a demangled string.
 class Node {
 public:
   enum Kind : unsigned char {
 #define ENUMERATOR(NodeKind) K ## NodeKind,
     FOR_EACH_NODE_KIND(ENUMERATOR)
 #undef ENUMERATOR
public:
   virtual void printRight(OutputStream &) const {}
 
   virtual StringView getBaseName() const { return StringView(); }
 
   // Silence compiler warnings, this dtor will never be called.
   virtual ~Node() = default;
 
 #ifndef NDEBUG
-  DUMP_METHOD void dump() const;
+  DEMANGLE_DUMP_METHOD void dump() const;
 #endif
 };
 
 class NodeArray {
   Node **Elements;
   size_t NumElements;
 
 public:
case SpecialSubKind::string:
       return StringView("basic_string");
     case SpecialSubKind::istream:
       return StringView("basic_istream");
     case SpecialSubKind::ostream:
       return StringView("basic_ostream");
     case SpecialSubKind::iostream:
       return StringView("basic_iostream");
     }
-    _LIBCPP_UNREACHABLE();
+    DEMANGLE_UNREACHABLE;
   }
 
   void printLeft(OutputStream &S) const override {
     switch (SSK) {
     case SpecialSubKind::allocator:
       S += "std::allocator";
       break;
     case SpecialSubKind::basic_string:
case SpecialSubKind::string:
       return StringView("string");
     case SpecialSubKind::istream:
       return StringView("istream");
     case SpecialSubKind::ostream:
       return StringView("ostream");
     case SpecialSubKind::iostream:
       return StringView("iostream");
     }
-    _LIBCPP_UNREACHABLE();
+    DEMANGLE_UNREACHABLE;
   }
 
   void printLeft(OutputStream &S) const override {
     switch (SSK) {
     case SpecialSubKind::allocator:
       S += "std::allocator";
       break;
     case SpecialSubKind::basic_string:
case 'K': {
 
     if (look(AfterQuals) == 'F' ||
         (look(AfterQuals) == 'D' &&
          (look(AfterQuals + 1) == 'o' || look(AfterQuals + 1) == 'O' ||
           look(AfterQuals + 1) == 'w' || look(AfterQuals + 1) == 'x'))) {
       Result = getDerived().parseFunctionType();
       break;
     }
-    _LIBCPP_FALLTHROUGH();
+    DEMANGLE_FALLTHROUGH;
   }
   case 'U': {
     Result = getDerived().parseQualifiedType();
     break;
   }
   // <builtin-type> ::= v    # void
   case 'v':
     ++First;
if (look(1) && look(1) != 't') {
         Result = make<NameWithTemplateArgs>(Sub, TA);
         break;
       }
 
       // If all we parsed was a substitution, don't re-insert into the
       // substitution table.
       return Sub;
     }
-    _LIBCPP_FALLTHROUGH();
+    DEMANGLE_FALLTHROUGH;
   }
   //        ::= <class-enum-type>
   default: {
     Result = getDerived().parseClassEnumType();
     break;
   }
   }
 

 }
 
 template <typename Alloc>
 struct ManglingParser : AbstractManglingParser<ManglingParser<Alloc>, Alloc> {
   using AbstractManglingParser<ManglingParser<Alloc>,
                                Alloc>::AbstractManglingParser;
 };
 
-}  // namespace itanium_demangle
-}  // namespace
+DEMANGLE_NAMESPACE_END
 
-#endif // LIBCXX_DEMANGLE_ITANIUMDEMANGLE_H
+#endif // DEMANGLE_ITANIUMDEMANGLE_H

libcxxabi/trunk/src/demangle/README.txt

+Itanium Name Demangler Library
+==============================
+
+Introduction
+------------
+
+This directory contains the generic itanium name demangler library. The main
+purpose of the library is to demangle C++ symbols, i.e. convert the string
+"_Z1fv" into "f()". You can also use the CRTP base ManglingParser to perform
+some simple analysis on the mangled name, or (in LLVM) use the opaque
+ItaniumPartialDemangler to query the demangled AST.
+
+Why are there multiple copies of the this library in the source tree?
+---------------------------------------------------------------------
+
+This directory is mirrored between libcxxabi/demangle and
+llvm/include/llvm/Demangle. The simple reason for this is that both projects
+need to demangle symbols, but neither can depend on each other. libcxxabi needs
+the demangler to implement __cxa_demangle, which is part of the itanium ABI
+spec. LLVM needs a copy for a bunch of places, but doesn't want to use the
+system's __cxa_demangle because it a) might not be available (i.e., on Windows),
+and b) probably isn't that up-to-date on the latest language features.
+
+The copy of the demangler in LLVM has some extra stuff that aren't needed in
+libcxxabi (ie, the MSVC demangler, ItaniumPartialDemangler), which depend on the
+shared generic components. Despite these differences, we want to keep the "core"
+generic demangling library identical between both copies to simplify development
+and testing.
+
+If you're working on the generic library, then do the work first in libcxxabi,
+then run the cp-to-llvm.sh script in src/demangle. This script takes as an
+argument the path to llvm, and re-copies the changes you made to libcxxabi over.
+Note that this script just blindly overwrites all changes to the generic library
+in llvm, so be careful.
+
+Because the core demangler needs to work in libcxxabi, everything needs to be
+declared in an anonymous namespace (see DEMANGLE_NAMESPACE_BEGIN), and you can't
+introduce any code that depends on the libcxx dylib.
+
+Hopefully, when LLVM becomes a monorepo, we can de-duplicate this code, and have
+both LLVM and libcxxabi depend on a shared demangler library.
+
+Testing
+-------
+
+The tests are split up between libcxxabi/test/{unit,}test_demangle.cpp, and
+llvm/unittest/Demangle. The llvm directory should only get tests for stuff not
+included in the core library. In the future though, we should probably move all
+the tests to LLVM.
+
+It is also a really good idea to run libFuzzer after non-trivial changes, see
+libcxxabi/fuzz/cxa_demangle_fuzzer.cpp and https://llvm.org/docs/LibFuzzer.html.

libcxxabi/trunk/src/demangle/StringView.h

 //===--- StringView.h -------------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
+//===----------------------------------------------------------------------===//
+//
+// FIXME: Use std::string_view instead when we support C++17.
 //
-// This file is copied from llvm/lib/Demangle/StringView.h.
 //===----------------------------------------------------------------------===//
 
-#ifndef LIBCXX_DEMANGLE_STRINGVIEW_H
-#define LIBCXX_DEMANGLE_STRINGVIEW_H
+#ifndef DEMANGLE_STRINGVIEW_H
+#define DEMANGLE_STRINGVIEW_H
 
+#include "DemangleConfig.h"
 #include <algorithm>
 #include <cassert>
 #include <cstring>
 
-namespace {
+DEMANGLE_NAMESPACE_BEGIN
+
 class StringView {
   const char *First;
   const char *Last;
 
 public:
+  static const size_t npos = ~size_t(0);
+
   template <size_t N>
   StringView(const char (&Str)[N]) : First(Str), Last(Str + N - 1) {}
   StringView(const char *First_, const char *Last_)
       : First(First_), Last(Last_) {}
   StringView(const char *First_, size_t Len)
       : First(First_), Last(First_ + Len) {}
   StringView(const char *Str) : First(Str), Last(Str + std::strlen(Str)) {}
   StringView() : First(nullptr), Last(nullptr) {}
 
   StringView substr(size_t From) const {
     return StringView(begin() + From, size() - From);
   }
 
+  size_t find(char C, size_t From = 0) const {
+    size_t FindBegin = std::min(From, size());
+    // Avoid calling memchr with nullptr.
+    if (FindBegin < size()) {
+      // Just forward to memchr, which is faster than a hand-rolled loop.
+      if (const void *P = ::memchr(First + FindBegin, C, size() - FindBegin))
+        return size_t(static_cast<const char *>(P) - First);
+    }
+    return npos;
+  }
+
   StringView substr(size_t From, size_t To) const {
     if (To >= size())
       To = size() - 1;
     if (From >= size())
       From = size() - 1;
     return StringView(First + From, First + To);
   }
 
   StringView dropFront(size_t N = 1) const {
     if (N >= size())
-      N = size() - 1;
+      N = size();
     return StringView(First + N, Last);
   }
 
+  StringView dropBack(size_t N = 1) const {
+    if (N >= size())
+      N = size();
+    return StringView(First, Last - N);
+  }
+
   char front() const {
     assert(!empty());
     return *begin();
   }
 
+  char back() const {
+    assert(!empty());
+    return *(end() - 1);
+  }
+
   char popFront() {
     assert(!empty());
     return *First++;
   }
 
   bool consumeFront(char C) {
     if (!startsWith(C))
       return false;
public:
   size_t size() const { return static_cast<size_t>(Last - First); }
   bool empty() const { return First == Last; }
 };
 
 inline bool operator==(const StringView &LHS, const StringView &RHS) {
   return LHS.size() == RHS.size() &&
          std::equal(LHS.begin(), LHS.end(), RHS.begin());
 }
-} // namespace
+
+DEMANGLE_NAMESPACE_END
 
 #endif

libcxxabi/trunk/src/demangle/Utility.h

 //===--- Utility.h ----------------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
+//===----------------------------------------------------------------------===//
+//
+// Provide some utility classes for use in the demangler(s).
 //
-// This file is copied from llvm/lib/Demangle/Utility.h.
 //===----------------------------------------------------------------------===//
 
-#ifndef LIBCXX_DEMANGLE_UTILITY_H
-#define LIBCXX_DEMANGLE_UTILITY_H
+#ifndef DEMANGLE_UTILITY_H
+#define DEMANGLE_UTILITY_H
 
 #include "StringView.h"
-
 #include <cstdint>
 #include <cstdlib>
 #include <cstring>
 #include <iterator>
 #include <limits>
 
-namespace {
+DEMANGLE_NAMESPACE_BEGIN
+
 // Stream that AST nodes write their string representation into after the AST
 // has been parsed.
 class OutputStream {
   char *Buffer;
   size_t CurrentPosition;
   size_t BufferCapacity;
 
   // Ensure there is at least n more positions in buffer.
if (Buf == nullptr) {
     BufferSize = InitSize;
   } else
     BufferSize = *N;
 
   S.reset(Buf, BufferSize);
   return true;
 }
 
-} // namespace
+DEMANGLE_NAMESPACE_END
 
 #endif

libcxxabi/trunk/src/demangle/cp-to-llvm.sh

+#!/bin/bash
+
+# Copies the 'demangle' library, excluding 'DemangleConfig.h', to llvm. If no
+# llvm directory is specified, then assume a monorepo layout.
+
+set -e
+
+FILES="ItaniumDemangle.h StringView.h Utility.h README.txt"
+LLVM_DEMANGLE_DIR=$1
+
+if [[ -z "$LLVM_DEMANGLE_DIR" ]]; then
+    LLVM_DEMANGLE_DIR="../../../llvm/include/llvm/Demangle"
+fi
+
+if [[ ! -d "$LLVM_DEMANGLE_DIR" ]]; then
+    echo "No such directory: $LLVM_DEMANGLE_DIR" >&2
+    exit 1
+fi
+
+read -p "This will overwrite the copies of $FILES in $LLVM_DEMANGLE_DIR; are you sure? [y/N]" -n 1 -r ANSWER
+echo
+
+if [[ $ANSWER =~ ^[Yy]$ ]]; then
+    for I in $FILES ; do
+        cp $I $LLVM_DEMANGLE_DIR/$I
+    done
+fi

llvm/trunk/include/llvm/Demangle/Compiler.h

-//===--- Compiler.h ---------------------------------------------*- C++ -*-===//
-//
-//                     The LLVM Compiler Infrastructure
-//
-// This file is distributed under the University of Illinois Open Source
-// License. See LICENSE.TXT for details.
-//
-//
-// This file contains a variety of feature test macros copied from
-// include/llvm/Support/Compiler.h so that LLVMDemangle does not need to take
-// a dependency on LLVMSupport.
-//===----------------------------------------------------------------------===//
-
-#ifndef LLVM_DEMANGLE_COMPILER_H
-#define LLVM_DEMANGLE_COMPILER_H
-
-#ifdef _MSC_VER
-// snprintf is implemented in VS 2015
-#if _MSC_VER < 1900
-#define snprintf _snprintf_s
-#endif
-#endif
-
-#ifndef __has_feature
-#define __has_feature(x) 0
-#endif
-
-#ifndef __has_cpp_attribute
-#define __has_cpp_attribute(x) 0
-#endif
-
-#ifndef __has_attribute
-#define __has_attribute(x) 0
-#endif
-
-#ifndef __has_builtin
-#define __has_builtin(x) 0
-#endif
-
-#ifndef LLVM_GNUC_PREREQ
-#if defined(__GNUC__) && defined(__GNUC_MINOR__) && defined(__GNUC_PATCHLEVEL__)
-#define LLVM_GNUC_PREREQ(maj, min, patch)                                      \
-  ((__GNUC__ << 20) + (__GNUC_MINOR__ << 10) + __GNUC_PATCHLEVEL__ >=          \
-   ((maj) << 20) + ((min) << 10) + (patch))
-#elif defined(__GNUC__) && defined(__GNUC_MINOR__)
-#define LLVM_GNUC_PREREQ(maj, min, patch)                                      \
-  ((__GNUC__ << 20) + (__GNUC_MINOR__ << 10) >= ((maj) << 20) + ((min) << 10))
-#else
-#define LLVM_GNUC_PREREQ(maj, min, patch) 0
-#endif
-#endif
-
-#if __has_attribute(used) || LLVM_GNUC_PREREQ(3, 1, 0)
-#define LLVM_ATTRIBUTE_USED __attribute__((__used__))
-#else
-#define LLVM_ATTRIBUTE_USED
-#endif
-
-#if __has_builtin(__builtin_unreachable) || LLVM_GNUC_PREREQ(4, 5, 0)
-#define LLVM_BUILTIN_UNREACHABLE __builtin_unreachable()
-#elif defined(_MSC_VER)
-#define LLVM_BUILTIN_UNREACHABLE __assume(false)
-#endif
-
-#if __has_attribute(noinline) || LLVM_GNUC_PREREQ(3, 4, 0)
-#define LLVM_ATTRIBUTE_NOINLINE __attribute__((noinline))
-#elif defined(_MSC_VER)
-#define LLVM_ATTRIBUTE_NOINLINE __declspec(noinline)
-#else
-#define LLVM_ATTRIBUTE_NOINLINE
-#endif
-
-#if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
-#define LLVM_DUMP_METHOD LLVM_ATTRIBUTE_NOINLINE LLVM_ATTRIBUTE_USED
-#else
-#define LLVM_DUMP_METHOD LLVM_ATTRIBUTE_NOINLINE
-#endif
-
-#if __cplusplus > 201402L && __has_cpp_attribute(fallthrough)
-#define LLVM_FALLTHROUGH [[fallthrough]]
-#elif __has_cpp_attribute(gnu::fallthrough)
-#define LLVM_FALLTHROUGH [[gnu::fallthrough]]
-#elif !__cplusplus
-// Workaround for llvm.org/PR23435, since clang 3.6 and below emit a spurious
-// error when __has_cpp_attribute is given a scoped attribute in C mode.
-#define LLVM_FALLTHROUGH
-#elif __has_cpp_attribute(clang::fallthrough)
-#define LLVM_FALLTHROUGH [[clang::fallthrough]]
-#else
-#define LLVM_FALLTHROUGH
-#endif
-
-#endif

llvm/trunk/include/llvm/Demangle/DemangleConfig.h

+//===--- DemangleConfig.h ---------------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains a variety of feature test macros copied from
+// include/llvm/Support/Compiler.h so that LLVMDemangle does not need to take
+// a dependency on LLVMSupport.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_DEMANGLE_COMPILER_H
+#define LLVM_DEMANGLE_COMPILER_H
+
+#ifdef _MSC_VER
+// snprintf is implemented in VS 2015
+#if _MSC_VER < 1900
+#define snprintf _snprintf_s
+#endif
+#endif
+
+#ifndef __has_feature
+#define __has_feature(x) 0
+#endif
+
+#ifndef __has_cpp_attribute
+#define __has_cpp_attribute(x) 0
+#endif
+
+#ifndef __has_attribute
+#define __has_attribute(x) 0
+#endif
+
+#ifndef __has_builtin
+#define __has_builtin(x) 0
+#endif
+
+#ifndef DEMANGLE_GNUC_PREREQ
+#if defined(__GNUC__) && defined(__GNUC_MINOR__) && defined(__GNUC_PATCHLEVEL__)
+#define DEMANGLE_GNUC_PREREQ(maj, min, patch)                           \
+  ((__GNUC__ << 20) + (__GNUC_MINOR__ << 10) + __GNUC_PATCHLEVEL__ >=          \
+   ((maj) << 20) + ((min) << 10) + (patch))
+#elif defined(__GNUC__) && defined(__GNUC_MINOR__)
+#define DEMANGLE_GNUC_PREREQ(maj, min, patch)                           \
+  ((__GNUC__ << 20) + (__GNUC_MINOR__ << 10) >= ((maj) << 20) + ((min) << 10))
+#else
+#define DEMANGLE_GNUC_PREREQ(maj, min, patch) 0
+#endif
+#endif
+
+#if __has_attribute(used) || DEMANGLE_GNUC_PREREQ(3, 1, 0)
+#define DEMANGLE_ATTRIBUTE_USED __attribute__((__used__))
+#else
+#define DEMANGLE_ATTRIBUTE_USED
+#endif
+
+#if __has_builtin(__builtin_unreachable) || DEMANGLE_GNUC_PREREQ(4, 5, 0)
+#define DEMANGLE_UNREACHABLE __builtin_unreachable()
+#elif defined(_MSC_VER)
+#define DEMANGLE_BUILTIN_UNREACHABLE __assume(false)
+#endif
+
+#if __has_attribute(noinline) || DEMANGLE_GNUC_PREREQ(3, 4, 0)
+#define DEMANGLE_ATTRIBUTE_NOINLINE __attribute__((noinline))
+#elif defined(_MSC_VER)
+#define DEMANGLE_ATTRIBUTE_NOINLINE __declspec(noinline)
+#else
+#define DEMANGLE_ATTRIBUTE_NOINLINE
+#endif
+
+#if !defined(NDEBUG)
+#define DEMANGLE_DUMP_METHOD DEMANGLE_ATTRIBUTE_NOINLINE DEMANGLE_ATTRIBUTE_USED
+#else
+#define DEMANGLE_DUMP_METHOD DEMANGLE_ATTRIBUTE_NOINLINE
+#endif
+
+#if __cplusplus > 201402L && __has_cpp_attribute(fallthrough)
+#define DEMANGLE_FALLTHROUGH [[fallthrough]]
+#elif __has_cpp_attribute(gnu::fallthrough)
+#define DEMANGLE_FALLTHROUGH [[gnu::fallthrough]]
+#elif !__cplusplus
+// Workaround for llvm.org/PR23435, since clang 3.6 and below emit a spurious
+// error when __has_cpp_attribute is given a scoped attribute in C mode.
+#define DEMANGLE_FALLTHROUGH
+#elif __has_cpp_attribute(clang::fallthrough)
+#define DEMANGLE_FALLTHROUGH [[clang::fallthrough]]
+#else
+#define DEMANGLE_FALLTHROUGH
+#endif
+
+#define DEMANGLE_NAMESPACE_BEGIN namespace llvm { namespace itanium_demangle {
+#define DEMANGLE_NAMESPACE_END } }
+
+#endif

llvm/trunk/include/llvm/Demangle/ItaniumDemangle.h

 //===------------------------- ItaniumDemangle.h ----------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
+//
+// Generic itanium demangler library. This file has two byte-per-byte identical
+// copies in the source tree, one in libcxxabi, and the other in llvm.
+//
+//===----------------------------------------------------------------------===//
 
-#ifndef LLVM_DEMANGLE_ITANIUMDEMANGLE_H
-#define LLVM_DEMANGLE_ITANIUMDEMANGLE_H
+#ifndef DEMANGLE_ITANIUMDEMANGLE_H
+#define DEMANGLE_ITANIUMDEMANGLE_H
 
 // FIXME: (possibly) incomplete list of features that clang mangles that this
 // file does not yet support:
 //   - C++ modules TS
 
-#include "llvm/Demangle/Compiler.h"
-#include "llvm/Demangle/StringView.h"
-#include "llvm/Demangle/Utility.h"
-
+#include "DemangleConfig.h"
+#include "StringView.h"
+#include "Utility.h"
 #include <cassert>
 #include <cctype>
 #include <cstdio>
 #include <cstdlib>
 #include <cstring>
 #include <numeric>
 #include <utility>
 
#define FOR_EACH_NODE_KIND(X) \
     X(IntegerCastExpr) \
     X(IntegerLiteral) \
     X(FloatLiteral) \
     X(DoubleLiteral) \
     X(LongDoubleLiteral) \
     X(BracedExpr) \
     X(BracedRangeExpr)
 
-namespace llvm {
-namespace itanium_demangle {
+DEMANGLE_NAMESPACE_BEGIN
+
 // Base class of all AST nodes. The AST is built by the parser, then is
 // traversed by the printLeft/Right functions to produce a demangled string.
 class Node {
 public:
   enum Kind : unsigned char {
 #define ENUMERATOR(NodeKind) K ## NodeKind,
     FOR_EACH_NODE_KIND(ENUMERATOR)
 #undef ENUMERATOR
public:
   virtual void printRight(OutputStream &) const {}
 
   virtual StringView getBaseName() const { return StringView(); }
 
   // Silence compiler warnings, this dtor will never be called.
   virtual ~Node() = default;
 
 #ifndef NDEBUG
-  LLVM_DUMP_METHOD void dump() const;
+  DEMANGLE_DUMP_METHOD void dump() const;
 #endif
 };
 
 class NodeArray {
   Node **Elements;
   size_t NumElements;
 
 public:
case SpecialSubKind::string:
       return StringView("basic_string");
     case SpecialSubKind::istream:
       return StringView("basic_istream");
     case SpecialSubKind::ostream:
       return StringView("basic_ostream");
     case SpecialSubKind::iostream:
       return StringView("basic_iostream");
     }
-    LLVM_BUILTIN_UNREACHABLE;
+    DEMANGLE_UNREACHABLE;
   }
 
   void printLeft(OutputStream &S) const override {
     switch (SSK) {
     case SpecialSubKind::allocator:
       S += "std::allocator";
       break;
     case SpecialSubKind::basic_string:
case SpecialSubKind::string:
       return StringView("string");
     case SpecialSubKind::istream:
       return StringView("istream");
     case SpecialSubKind::ostream:
       return StringView("ostream");
     case SpecialSubKind::iostream:
       return StringView("iostream");
     }
-    LLVM_BUILTIN_UNREACHABLE;
+    DEMANGLE_UNREACHABLE;
   }
 
   void printLeft(OutputStream &S) const override {
     switch (SSK) {
     case SpecialSubKind::allocator:
       S += "std::allocator";
       break;
     case SpecialSubKind::basic_string:
case 'K': {
 
     if (look(AfterQuals) == 'F' ||
         (look(AfterQuals) == 'D' &&
          (look(AfterQuals + 1) == 'o' || look(AfterQuals + 1) == 'O' ||
           look(AfterQuals + 1) == 'w' || look(AfterQuals + 1) == 'x'))) {
       Result = getDerived().parseFunctionType();
       break;
     }
-    LLVM_FALLTHROUGH;
+    DEMANGLE_FALLTHROUGH;
   }
   case 'U': {
     Result = getDerived().parseQualifiedType();
     break;
   }
   // <builtin-type> ::= v    # void
   case 'v':
     ++First;
if (look(1) && look(1) != 't') {
         Result = make<NameWithTemplateArgs>(Sub, TA);
         break;
       }
 
       // If all we parsed was a substitution, don't re-insert into the
       // substitution table.
       return Sub;
     }
-    LLVM_FALLTHROUGH;
+    DEMANGLE_FALLTHROUGH;
   }
   //        ::= <class-enum-type>
   default: {
     Result = getDerived().parseClassEnumType();
     break;
   }
   }
 

 }
 
 template <typename Alloc>
 struct ManglingParser : AbstractManglingParser<ManglingParser<Alloc>, Alloc> {
   using AbstractManglingParser<ManglingParser<Alloc>,
                                Alloc>::AbstractManglingParser;
 };
 
-}  // namespace itanium_demangle
-}  // namespace llvm
+DEMANGLE_NAMESPACE_END
 
-#endif // LLVM_DEMANGLE_ITANIUMDEMANGLE_H
+#endif // DEMANGLE_ITANIUMDEMANGLE_H

llvm/trunk/include/llvm/Demangle/MicrosoftDemangle.h

 //===------------------------- MicrosoftDemangle.h --------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_DEMANGLE_MICROSOFT_DEMANGLE_H
 #define LLVM_DEMANGLE_MICROSOFT_DEMANGLE_H
 
-#include "llvm/Demangle/Compiler.h"
+#include "llvm/Demangle/DemangleConfig.h"
 #include "llvm/Demangle/MicrosoftDemangleNodes.h"
 #include "llvm/Demangle/StringView.h"
 #include "llvm/Demangle/Utility.h"
 
 #include <utility>
 
 namespace llvm {
 namespace ms_demangle {

llvm/trunk/include/llvm/Demangle/MicrosoftDemangleNodes.h

+//===- MicrosoftDemangleNodes.h ---------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is dual licensed under the MIT and the University of Illinois Open
+// Source Licenses. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines the AST nodes used in the MSVC demangler.
+//
+//===----------------------------------------------------------------------===//
+
 #ifndef LLVM_SUPPORT_MICROSOFTDEMANGLENODES_H
 #define LLVM_SUPPORT_MICROSOFTDEMANGLENODES_H
 
-#include "llvm/Demangle/Compiler.h"
+#include "llvm/Demangle/DemangleConfig.h"
 #include "llvm/Demangle/StringView.h"
 #include <array>
 
+namespace llvm {
+namespace itanium_demangle {
 class OutputStream;
+}
+}
+
+using llvm::itanium_demangle::OutputStream;
+using llvm::itanium_demangle::StringView;
 
 namespace llvm {
 namespace ms_demangle {
 
 // Storage classes
 enum Qualifiers : uint8_t {
   Q_None = 0,
   Q_Const = 1 << 0,
struct FunctionSymbolNode : public SymbolNode {
   void output(OutputStream &OS, OutputFlags Flags) const override;
 
   FunctionSignatureNode *Signature = nullptr;
 };
 
 } // namespace ms_demangle
 } // namespace llvm
 
-#endif
+#endif
 

llvm/trunk/include/llvm/Demangle/README.txt

+Itanium Name Demangler Library
+==============================
+
+Introduction
+------------
+
+This directory contains the generic itanium name demangler library. The main
+purpose of the library is to demangle C++ symbols, i.e. convert the string
+"_Z1fv" into "f()". You can also use the CRTP base ManglingParser to perform
+some simple analysis on the mangled name, or (in LLVM) use the opaque
+ItaniumPartialDemangler to query the demangled AST.
+
+Why are there multiple copies of the this library in the source tree?
+---------------------------------------------------------------------
+
+This directory is mirrored between libcxxabi/demangle and
+llvm/include/llvm/Demangle. The simple reason for this is that both projects
+need to demangle symbols, but neither can depend on each other. libcxxabi needs
+the demangler to implement __cxa_demangle, which is part of the itanium ABI
+spec. LLVM needs a copy for a bunch of places, but doesn't want to use the
+system's __cxa_demangle because it a) might not be available (i.e., on Windows),
+and b) probably isn't that up-to-date on the latest language features.
+
+The copy of the demangler in LLVM has some extra stuff that aren't needed in
+libcxxabi (ie, the MSVC demangler, ItaniumPartialDemangler), which depend on the
+shared generic components. Despite these differences, we want to keep the "core"
+generic demangling library identical between both copies to simplify development
+and testing.
+
+If you're working on the generic library, then do the work first in libcxxabi,
+then run the cp-to-llvm.sh script in src/demangle. This script takes as an
+argument the path to llvm, and re-copies the changes you made to libcxxabi over.
+Note that this script just blindly overwrites all changes to the generic library
+in llvm, so be careful.
+
+Because the core demangler needs to work in libcxxabi, everything needs to be
+declared in an anonymous namespace (see DEMANGLE_NAMESPACE_BEGIN), and you can't
+introduce any code that depends on the libcxx dylib.
+
+Hopefully, when LLVM becomes a monorepo, we can de-duplicate this code, and have
+both LLVM and libcxxabi depend on a shared demangler library.
+
+Testing
+-------
+
+The tests are split up between libcxxabi/test/{unit,}test_demangle.cpp, and
+llvm/unittest/Demangle. The llvm directory should only get tests for stuff not
+included in the core library. In the future though, we should probably move all
+the tests to LLVM.
+
+It is also a really good idea to run libFuzzer after non-trivial changes, see
+libcxxabi/fuzz/cxa_demangle_fuzzer.cpp and https://llvm.org/docs/LibFuzzer.html.

llvm/trunk/include/llvm/Demangle/StringView.h

 //===--- StringView.h -------------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
+//===----------------------------------------------------------------------===//
+//
+// FIXME: Use std::string_view instead when we support C++17.
 //
-// This file contains a limited version of LLVM's StringView class.  It is
-// copied here so that LLVMDemangle need not take a dependency on LLVMSupport.
 //===----------------------------------------------------------------------===//
 
-#ifndef LLVM_DEMANGLE_STRINGVIEW_H
-#define LLVM_DEMANGLE_STRINGVIEW_H
+#ifndef DEMANGLE_STRINGVIEW_H
+#define DEMANGLE_STRINGVIEW_H
 
+#include "DemangleConfig.h"
 #include <algorithm>
 #include <cassert>
 #include <cstring>
 
+DEMANGLE_NAMESPACE_BEGIN
+
 class StringView {
   const char *First;
   const char *Last;
 
 public:
   static const size_t npos = ~size_t(0);
 
   template <size_t N>
public:
   }
 
   size_t find(char C, size_t From = 0) const {
     size_t FindBegin = std::min(From, size());
     // Avoid calling memchr with nullptr.
     if (FindBegin < size()) {
       // Just forward to memchr, which is faster than a hand-rolled loop.
       if (const void *P = ::memchr(First + FindBegin, C, size() - FindBegin))
-        return static_cast<const char *>(P) - First;
+        return size_t(static_cast<const char *>(P) - First);
     }
     return npos;
   }
 
   StringView substr(size_t From, size_t To) const {
     if (To >= size())
       To = size() - 1;
     if (From >= size())
public:
   bool empty() const { return First == Last; }
 };
 
 inline bool operator==(const StringView &LHS, const StringView &RHS) {
   return LHS.size() == RHS.size() &&
          std::equal(LHS.begin(), LHS.end(), RHS.begin());
 }
 
+DEMANGLE_NAMESPACE_END
+
 #endif

llvm/trunk/include/llvm/Demangle/Utility.h

 //===--- Utility.h ----------------------------------------------*- C++ -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
+//===----------------------------------------------------------------------===//
+//
+// Provide some utility classes for use in the demangler(s).
 //
-// This file contains several utility classes used by the demangle library.
 //===----------------------------------------------------------------------===//
 
-#ifndef LLVM_DEMANGLE_UTILITY_H
-#define LLVM_DEMANGLE_UTILITY_H
+#ifndef DEMANGLE_UTILITY_H
+#define DEMANGLE_UTILITY_H
 
 #include "StringView.h"
-
 #include <cstdint>
 #include <cstdlib>
 #include <cstring>
 #include <iterator>
 #include <limits>
 
+DEMANGLE_NAMESPACE_BEGIN
+
 // Stream that AST nodes write their string representation into after the AST
 // has been parsed.
 class OutputStream {
   char *Buffer;
   size_t CurrentPosition;
   size_t BufferCapacity;
 
   // Ensure there is at least n more positions in buffer.
if (Buf == nullptr) {
     BufferSize = InitSize;
   } else
     BufferSize = *N;
 
   S.reset(Buf, BufferSize);
   return true;
 }
 
+DEMANGLE_NAMESPACE_END
+
 #endif

llvm/trunk/lib/Demangle/MicrosoftDemangle.cpp

 // easily reused in other programs such as libcxxabi.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Demangle/MicrosoftDemangle.h"
 #include "llvm/Demangle/Demangle.h"
 #include "llvm/Demangle/MicrosoftDemangleNodes.h"
 
-#include "llvm/Demangle/Compiler.h"
+#include "llvm/Demangle/DemangleConfig.h"
 #include "llvm/Demangle/StringView.h"
 #include "llvm/Demangle/Utility.h"
 
 #include <array>
 #include <cctype>
 #include <cstdio>
 #include <tuple>
 
case SpecialIntrinsicKind::Vbtable:
     break;
   case SpecialIntrinsicKind::LocalVftable:
     NI->Name = "`local vftable'";
     break;
   case SpecialIntrinsicKind::RttiCompleteObjLocator:
     NI->Name = "`RTTI Complete Object Locator'";
     break;
   default:
-    LLVM_BUILTIN_UNREACHABLE;
+    DEMANGLE_UNREACHABLE;
   }
   QualifiedNameNode *QN = demangleNameScopeChain(MangledName, NI);
   SpecialTableSymbolNode *STSN = Arena.alloc<SpecialTableSymbolNode>();
   STSN->Name = QN;
   bool IsMember = false;
   char Front = MangledName.popFront();
   if (Front != '6' && Front != '7') {
     Error = true;
translateIntrinsicFunctionCode(char CH, FunctionIdentifierCodeGroup Group) {
   switch (Group) {
   case FunctionIdentifierCodeGroup::Basic:
     return Basic[Index];
   case FunctionIdentifierCodeGroup::Under:
     return Under[Index];
   case FunctionIdentifierCodeGroup::DoubleUnder:
     return DoubleUnder[Index];
   }
-  LLVM_BUILTIN_UNREACHABLE;
+  DEMANGLE_UNREACHABLE;
 }
 
 IdentifierNode *
 Demangler::demangleFunctionIdentifierCode(StringView &MangledName,
                                           FunctionIdentifierCodeGroup Group) {
   switch (Group) {
   case FunctionIdentifierCodeGroup::Basic:
     switch (char CH = MangledName.popFront()) {
if (!MangledName.consumeFront("@_"))
     goto StringLiteralError;
   if (MangledName.empty())
     goto StringLiteralError;
 
   // Char Type (regular or wchar_t)
   switch (MangledName.popFront()) {
   case '1':
     IsWcharT = true;
-    LLVM_FALLTHROUGH;
+    DEMANGLE_FALLTHROUGH;
   case '0':
     break;
   default:
     goto StringLiteralError;
   }
 
   // Encoded Length
   std::tie(StringByteSize, IsNegative) = demangleNumber(MangledName);
case 1:
       break;
     case 2:
       Result->Char = CharKind::Char16;
       break;
     case 4:
       Result->Char = CharKind::Char32;
       break;
     default:
-      LLVM_BUILTIN_UNREACHABLE;
+      DEMANGLE_UNREACHABLE;
     }
     const unsigned NumChars = BytesDecoded / CharBytes;
     for (unsigned CharIndex = 0; CharIndex < NumChars; ++CharIndex) {
       unsigned NextChar =
           decodeMultiByteChar(StringBytes, CharIndex, CharBytes);
       if (CharIndex + 1 < NumChars || Result->IsTruncated)
         outputEscapedChar(OS, NextChar);
     }
if (MangledName.consumeFront("$$Y")) {
         S = parse(MangledName);
         memorizeIdentifier(S->Name->getUnqualifiedIdentifier());
       }
 
       switch (InheritanceSpecifier) {
       case 'J':
         TPRN->ThunkOffsets[TPRN->ThunkOffsetCount++] =
             demangleSigned(MangledName);
-        LLVM_FALLTHROUGH;
+        DEMANGLE_FALLTHROUGH;
       case 'I':
         TPRN->ThunkOffsets[TPRN->ThunkOffsetCount++] =
             demangleSigned(MangledName);
-        LLVM_FALLTHROUGH;
+        DEMANGLE_FALLTHROUGH;
       case 'H':
         TPRN->ThunkOffsets[TPRN->ThunkOffsetCount++] =
             demangleSigned(MangledName);
-        LLVM_FALLTHROUGH;
+        DEMANGLE_FALLTHROUGH;
       case '1':
         break;
       default:
         Error = true;
         break;
       }
       TPRN->Affinity = PointerAffinity::Pointer;
       TPRN->Symbol = S;
if (MangledName.consumeFront("$$Y")) {
       // Data member pointer.
       MangledName = MangledName.dropFront();
       char InheritanceSpecifier = MangledName.popFront();
 
       switch (InheritanceSpecifier) {
       case 'G':
         TPRN->ThunkOffsets[TPRN->ThunkOffsetCount++] =
             demangleSigned(MangledName);
-        LLVM_FALLTHROUGH;
+        DEMANGLE_FALLTHROUGH;
       case 'F':
         TPRN->ThunkOffsets[TPRN->ThunkOffsetCount++] =
             demangleSigned(MangledName);
         TPRN->ThunkOffsets[TPRN->ThunkOffsetCount++] =
             demangleSigned(MangledName);
-        LLVM_FALLTHROUGH;
+        DEMANGLE_FALLTHROUGH;
       case '0':
         break;
       default:
         Error = true;
         break;
       }
       TPRN->IsMemberPointer = true;
 

llvm/trunk/lib/Demangle/MicrosoftDemangleNodes.cpp

 //===- MicrosoftDemangle.cpp ----------------------------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is dual licensed under the MIT and the University of Illinois Open
 // Source Licenses. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file defines a demangler for MSVC-style mangled symbols.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/Demangle/MicrosoftDemangleNodes.h"
-#include "llvm/Demangle/Compiler.h"
+#include "llvm/Demangle/DemangleConfig.h"
 #include "llvm/Demangle/Utility.h"
 #include <cctype>
 #include <string>
 
 using namespace llvm;
 using namespace ms_demangle;
 
 #define OUTPUT_ENUM_CLASS_VALUE(Enum, Value, Desc)                             \

llvm/trunk/lib/Support/ItaniumManglingCanonicalizer.cpp

 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/FoldingSet.h"
 #include "llvm/ADT/StringRef.h"
 
 using namespace llvm;
 using llvm::itanium_demangle::ForwardTemplateReference;
 using llvm::itanium_demangle::Node;
 using llvm::itanium_demangle::NodeKind;
+using llvm::itanium_demangle::StringView;
 
 namespace {
 struct FoldingSetNodeIDBuilder {
   llvm::FoldingSetNodeID &ID;
   void operator()(const Node *P) { ID.AddPointer(P); }
   void operator()(StringView Str) {
     ID.AddString(llvm::StringRef(Str.begin(), Str.size()));
   }