[LibTooling] Add RangeSelector library for defining source ranges based on bound AST nodes.

[ymandel]

Summary

The RangeSelector library defines a combinator language for specifying source
ranges based on bound id for AST nodes. The combinator approach follows the
design of the AST matchers. The RangeSelectors defined here will be used in
both RewriteRule, for specifying source affected by edit, and in Stencil for
specifying source to use constructively in a replacement.

[ymandel]

Ilya, this revision follows up from our discussion on the doc. It adds some more selectors. If this change is to big, I'm happy to split it up (e.g. moving the changes to SourceCode to a separate revision and/or splitting up the RangeSelector changes). Also, I have the diff updating Transformer ready if you want to see that. I can post a preview or created a stacked revision. Let me know...

[ilya-biryukov]

Thanks! Mostly NITs from me, the design looks nice! Would you mind adding some tests before we land this?

The only major thing that I'd argue against is adding helper functions like findPreviousToken to SourceCode.h.
It's fine to have them in the .cpp files if they make the implementation simpler, but we want to design them more carefully if we put them in the public API.

clang/include/clang/Tooling/Refactoring/RangeSelector.h

30

Maybe try putting it into the tooling namespace directly?
Or are there immediate and unfortunate conflicts with existing names?

36

NIT: maybe explain what Id is?

37

NIT: I believe LLVM naming rules would mandate to name it ID

38

Maybe a shorter description could suffice?

Selects a node and a trailing semicolon. Useful for selecting expression statements.

44

NIT: maybe name it statement?

47

Could this be an overload with the same name?

54

NIT: this is super-specialized, but has a very generic name, therefore might cause confusion. Maybe call it ctorInitializerName?

58

Same thing, maybe rename to callExprArgs?
And other nodes too

75

NIT: maybe call it expansion?

contraction is a new term, might be confusing to use.

clang/include/clang/Tooling/Refactoring/SourceCode.h

76 ↗

(On Diff #198969)

Not sure this should be a public API. Maybe keep it private to the use-site?

[ymandel]

Add tests and back out changes to SourceCode.

[ymandel]

comment tweak

[ymandel]

Thanks for the review. Added the tests and undid changes to SourceCode.

clang/include/clang/Tooling/Refactoring/RangeSelector.h

30

No conflicts. Was just being cautious.

37

here and throughout.

54

It works with others as well, particularly NamedDecl. This is one place where typed node ids would be nice, b/c that would allow us to make this interface typed, like the matchers.

I kept the name but tried to clarify the comments. WDYT?

58

i'd like to keep these as lightweight as possible. Compromised on callArgs?

75

Sigh. You're right. Its just that the existing terminology is confusing. that said, anyone using this combinator is already doing advanced work, so consistency is probably more important than clarity.

clang/include/clang/Tooling/Refactoring/SourceCode.h

76 ↗

(On Diff #198969)

Good point. Originally, we needed these in two different files -- but the range selectors actually obviate that.

[ilya-biryukov]

clang/include/clang/Tooling/Refactoring/RangeSelector.h

30

I can see why a separate namespace would make sense, but since the tooling namespace is not densely populated I hope we can get away with putting things here directly and saving some typing on the use-sites.

Hope that does not backfire in the future, though.

54

Ah, sorry, misread the original comment. The name actually makes sense in that case.
Am I correct to assume it will only select the final identifier of a qualified name, but not the qualifier?
E.g. for ::foo::bar::baz, it would only select baz, right?
What happens when we also have template args? E.g. for ::foo::bar::baz<int>, it would only select baz, right?

Maybe add those examples to the documentation? It's something that's very easy to get wrong.

58

callArgs LG, thanks!

clang/unittests/Tooling/RangeSelectorTest.cpp

40

NIT: use ASTUnit to match LLVM naming rules

144

Consider restructuring the code to place assertions into the test function itself.
This wildly improves locations reported in case of test failures and makes tests easier to read.

I.e. having something like

auto AST = buildASTAndMatch(Code, Matcher);
EXPECT_EQ(applySelector(range(Arg0, Arg1), AST),  "3, 7");

is arguably both easier to read and produces better location information on failures than a function that runs the test.
Even though it's a bit more code.

Note that it's a bit more complicated if you need to deal with Expected<> return types, but still possible:

EXPECT_THAT_EXPECTED(applySelector(range(Arg0, Arg1), AST),  HasValue("3, 7"));
EXPECT_THAT_EXPECTED(applySelector(range(Arg1, Arg0), AST),  Failed());

[ymandel]

Restructured tests to simplify.

[ymandel]

clang/include/clang/Tooling/Refactoring/RangeSelector.h

30

SGTM.

clang/unittests/Tooling/RangeSelectorTest.cpp

144

Thanks for the suggestion. Thoroughly reworked the tests along these lines. I think the result is much clearer.

[ymandel]

fixed formatting.

[ilya-biryukov]

Thanks! LGTM!

[ymandel]

updated some comments.

[ymandel]

Test file broke gcc builds. Fix in progress...

[ymandel]

Fixed with r361160.

[penzn]

cfe/trunk/lib/Tooling/Refactoring/RangeSelector.cpp

229 ↗

(On Diff #200265)

Sorry for posting here, haven't gotten my bugzilla access yet (requested though).

This breaks with Visual Studio 2017 (15.7.6):

RangeSelector.cpp(229): error C2971: '`anonymous-namespace'::RelativeSelector': template parameter 'Func': 'getStatementsRange': a variable with non-static storage duration cannot be used as a non-type argument

[penzn]

cfe/trunk/lib/Tooling/Refactoring/RangeSelector.cpp

229 ↗

(On Diff #200265)

Fixed in https://reviews.llvm.org/D62202

clang/include/clang/Tooling/Refactoring/RangeSelector.h

+//===--- RangeSelector.h - Source-selection library ---------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+///  \file
+///  Defines a combinator library supporting the definition of _selectors_,
+///  which select source ranges based on (bound) AST nodes.
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLING_REFACTOR_RANGE_SELECTOR_H_
+#define LLVM_CLANG_TOOLING_REFACTOR_RANGE_SELECTOR_H_
+
+#include "clang/ASTMatchers/ASTMatchFinder.h"
+#include "clang/Basic/SourceLocation.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Error.h"
+#include <functional>
+
+namespace clang {
+namespace tooling {
+using RangeSelector = std::function<Expected<CharSourceRange>(
+    const ast_matchers::MatchFinder::MatchResult &)>;
+
+inline RangeSelector charRange(CharSourceRange R) {
+  return [R](const ast_matchers::MatchFinder::MatchResult &)
+             -> Expected<CharSourceRange> { return R; };
+}
+
+/// Selects from the start of \p Begin and to the end of \p End.
+RangeSelector range(RangeSelector Begin, RangeSelector End);
+
+/// Convenience version of \c range where end-points are bound nodes.
+RangeSelector range(StringRef BeginID, StringRef EndID);
+
+/// Selects a node, including trailing semicolon (for non-expression
+/// statements). \p ID is the node's binding in the match result.
+RangeSelector node(StringRef ID);
+
+/// Selects a node, including trailing semicolon (always). Useful for selecting
+/// expression statements. \p ID is the node's binding in the match result.
+RangeSelector statement(StringRef ID);
+
+/// Given a \c MemberExpr, selects the member token. \p ID is the node's
+/// binding in the match result.
+RangeSelector member(StringRef ID);
+
+/// Given a node with a "name", (like \c NamedDecl, \c DeclRefExpr or \c
+/// CxxCtorInitializer) selects the name's token.  Only selects the final
+/// identifier of a qualified name, but not any qualifiers or template
+/// arguments.  For example, for `::foo::bar::baz` and `::foo::bar::baz<int>`,
+/// it selects only `baz`.
+///
+/// \param ID is the node's binding in the match result.
+RangeSelector name(StringRef ID);
+
+// Given a \c CallExpr (bound to \p ID), selects the arguments' source text (all
+// source between the call's parentheses).
+RangeSelector callArgs(StringRef ID);
+
+// Given a \c CompoundStmt (bound to \p ID), selects the source of the
+// statements (all source between the braces).
+RangeSelector statements(StringRef ID);
+
+// Given a \c InitListExpr (bound to \p ID), selects the range of the elements
+// (all source between the braces).
+RangeSelector initListElements(StringRef ID);
+
+/// Selects the range from which `S` was expanded (possibly along with other
+/// source), if `S` is an expansion, and `S` itself, otherwise.  Corresponds to
+/// `SourceManager::getExpansionRange`.
+RangeSelector expansion(RangeSelector S);
+} // namespace tooling
+} // namespace clang
+
+#endif // LLVM_CLANG_TOOLING_REFACTOR_RANGE_SELECTOR_H_

clang/lib/Tooling/Refactoring/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS Support)
 
 add_clang_library(clangToolingRefactor
   ASTSelection.cpp
   ASTSelectionRequirements.cpp
   AtomicChange.cpp
   Extract/Extract.cpp
   Extract/SourceExtraction.cpp
+  RangeSelector.cpp
   RefactoringActions.cpp
   Rename/RenamingAction.cpp
   Rename/SymbolOccurrences.cpp
   Rename/USRFinder.cpp
   Rename/USRFindingAction.cpp
   Rename/USRLocFinder.cpp
   SourceCode.cpp
   Stencil.cpp

clang/lib/Tooling/Refactoring/RangeSelector.cpp

+//===--- Transformer.cpp - Transformer library implementation ---*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Tooling/Refactoring/RangeSelector.h"
+#include "clang/AST/Expr.h"
+#include "clang/ASTMatchers/ASTMatchFinder.h"
+#include "clang/Basic/SourceLocation.h"
+#include "clang/Lex/Lexer.h"
+#include "clang/Tooling/Refactoring/SourceCode.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Errc.h"
+#include "llvm/Support/Error.h"
+#include <string>
+#include <utility>
+#include <vector>
+
+using namespace clang;
+using namespace tooling;
+
+using ast_matchers::MatchFinder;
+using ast_type_traits::ASTNodeKind;
+using ast_type_traits::DynTypedNode;
+using llvm::Error;
+using llvm::StringError;
+
+using MatchResult = MatchFinder::MatchResult;
+
+static Error invalidArgumentError(Twine Message) {
+  return llvm::make_error<StringError>(llvm::errc::invalid_argument, Message);
+}
+
+static Error typeError(StringRef ID, const ASTNodeKind &Kind) {
+  return invalidArgumentError("mismatched type (node id=" + ID +
+                              " kind=" + Kind.asStringRef() + ")");
+}
+
+static Error typeError(StringRef ID, const ASTNodeKind &Kind,
+                       Twine ExpectedType) {
+  return invalidArgumentError("mismatched type: expected one of " +
+                              ExpectedType + " (node id=" + ID +
+                              " kind=" + Kind.asStringRef() + ")");
+}
+
+static Error missingPropertyError(StringRef ID, Twine Description,
+                                  StringRef Property) {
+  return invalidArgumentError(Description + " requires property '" + Property +
+                              "' (node id=" + ID + ")");
+}
+
+static Expected<DynTypedNode> getNode(const ast_matchers::BoundNodes &Nodes,
+                                      StringRef ID) {
+  auto &NodesMap = Nodes.getMap();
+  auto It = NodesMap.find(ID);
+  if (It == NodesMap.end())
+    return invalidArgumentError("ID not bound: " + ID);
+  return It->second;
+}
+
+static SourceLocation findPreviousTokenStart(SourceLocation Start,
+                                             const SourceManager &SM,
+                                             const LangOptions &LangOpts) {
+  if (Start.isInvalid() || Start.isMacroID())
+    return SourceLocation();
+
+  SourceLocation BeforeStart = Start.getLocWithOffset(-1);
+  if (BeforeStart.isInvalid() || BeforeStart.isMacroID())
+    return SourceLocation();
+
+  return Lexer::GetBeginningOfToken(BeforeStart, SM, LangOpts);
+}
+
+static SourceLocation findPreviousTokenKind(SourceLocation Start,
+                                            const SourceManager &SM,
+                                            const LangOptions &LangOpts,
+                                            tok::TokenKind TK) {
+  while (true) {
+    SourceLocation L = findPreviousTokenStart(Start, SM, LangOpts);
+    if (L.isInvalid() || L.isMacroID())
+      return SourceLocation();
+
+    Token T;
+    if (Lexer::getRawToken(L, T, SM, LangOpts, /*IgnoreWhiteSpace=*/true))
+      return SourceLocation();
+
+    if (T.is(TK))
+      return T.getLocation();
+
+    Start = L;
+  }
+}
+
+static SourceLocation findOpenParen(const CallExpr &E, const SourceManager &SM,
+                                    const LangOptions &LangOpts) {
+  SourceLocation EndLoc =
+      E.getNumArgs() == 0 ? E.getRParenLoc() : E.getArg(0)->getBeginLoc();
+  return findPreviousTokenKind(EndLoc, SM, LangOpts, tok::TokenKind::l_paren);
+}
+
+RangeSelector tooling::node(StringRef ID) {
+  return [ID](const MatchResult &Result) -> Expected<CharSourceRange> {
+    Expected<DynTypedNode> Node = getNode(Result.Nodes, ID);
+    if (!Node)
+      return Node.takeError();
+    return Node->get<Stmt>() != nullptr && Node->get<Expr>() == nullptr
+               ? getExtendedRange(*Node, tok::TokenKind::semi, *Result.Context)
+               : CharSourceRange::getTokenRange(Node->getSourceRange());
+  };
+}
+
+RangeSelector tooling::statement(StringRef ID) {
+  return [ID](const MatchResult &Result) -> Expected<CharSourceRange> {
+    Expected<DynTypedNode> Node = getNode(Result.Nodes, ID);
+    if (!Node)
+      return Node.takeError();
+    return getExtendedRange(*Node, tok::TokenKind::semi, *Result.Context);
+  };
+}
+
+RangeSelector tooling::range(RangeSelector Begin, RangeSelector End) {
+  return [Begin, End](const MatchResult &Result) -> Expected<CharSourceRange> {
+    Expected<CharSourceRange> BeginRange = Begin(Result);
+    if (!BeginRange)
+      return BeginRange.takeError();
+    Expected<CharSourceRange> EndRange = End(Result);
+    if (!EndRange)
+      return EndRange.takeError();
+    SourceLocation B = BeginRange->getBegin();
+    SourceLocation E = EndRange->getEnd();
+    // Note: we are precluding the possibility of sub-token ranges in the case
+    // that EndRange is a token range.
+    if (Result.SourceManager->isBeforeInTranslationUnit(E, B)) {
+      return invalidArgumentError("Bad range: out of order");
+    }
+    return CharSourceRange(SourceRange(B, E), EndRange->isTokenRange());
+  };
+}
+
+RangeSelector tooling::range(StringRef BeginID, StringRef EndID) {
+  return tooling::range(node(BeginID), node(EndID));
+}
+
+RangeSelector tooling::member(StringRef ID) {
+  return [ID](const MatchResult &Result) -> Expected<CharSourceRange> {
+    Expected<DynTypedNode> Node = getNode(Result.Nodes, ID);
+    if (!Node)
+      return Node.takeError();
+    if (auto *M = Node->get<clang::MemberExpr>())
+      return CharSourceRange::getTokenRange(
+          M->getMemberNameInfo().getSourceRange());
+    return typeError(ID, Node->getNodeKind(), "MemberExpr");
+  };
+}
+
+RangeSelector tooling::name(StringRef ID) {
+  return [ID](const MatchResult &Result) -> Expected<CharSourceRange> {
+    Expected<DynTypedNode> N = getNode(Result.Nodes, ID);
+    if (!N)
+      return N.takeError();
+    auto &Node = *N;
+    if (const auto *D = Node.get<NamedDecl>()) {
+      if (!D->getDeclName().isIdentifier())
+        return missingPropertyError(ID, "name", "identifier");
+      SourceLocation L = D->getLocation();
+      auto R = CharSourceRange::getTokenRange(L, L);
+      // Verify that the range covers exactly the name.
+      // FIXME: extend this code to support cases like `operator +` or
+      // `foo<int>` for which this range will be too short.  Doing so will
+      // require subcasing `NamedDecl`, because it doesn't provide virtual
+      // access to the \c DeclarationNameInfo.
+      if (getText(R, *Result.Context) != D->getName())
+        return CharSourceRange();
+      return R;
+    }
+    if (const auto *E = Node.get<DeclRefExpr>()) {
+      if (!E->getNameInfo().getName().isIdentifier())
+        return missingPropertyError(ID, "name", "identifier");
+      SourceLocation L = E->getLocation();
+      return CharSourceRange::getTokenRange(L, L);
+    }
+    if (const auto *I = Node.get<CXXCtorInitializer>()) {
+      if (!I->isMemberInitializer() && I->isWritten())
+        return missingPropertyError(ID, "name", "explicit member initializer");
+      SourceLocation L = I->getMemberLocation();
+      return CharSourceRange::getTokenRange(L, L);
+    }
+    return typeError(ID, Node.getNodeKind(),
+                     "DeclRefExpr, NamedDecl, CXXCtorInitializer");
+  };
+}
+
+namespace {
+// Creates a selector from a range-selection function `Func`, which selects a
+// range that is relative to a bound node id.  `ArgT` is the node type expected
+// by `Func`.
+template <typename T, CharSourceRange (*Func)(const MatchResult &, const T &)>
+class RelativeSelector {
+  std::string ID;
+
+public:
+  RelativeSelector(StringRef ID) : ID(ID) {}
+
+  Expected<CharSourceRange> operator()(const MatchResult &Result) {
+    Expected<DynTypedNode> N = getNode(Result.Nodes, ID);
+    if (!N)
+      return N.takeError();
+    if (const auto *Arg = N->get<T>())
+      return Func(Result, *Arg);
+    return typeError(ID, N->getNodeKind());
+  }
+};
+} // namespace
+
+// Returns the range of the statements (all source between the braces).
+static CharSourceRange getStatementsRange(const MatchResult &,
+                                          const CompoundStmt &CS) {
+  return CharSourceRange::getCharRange(CS.getLBracLoc().getLocWithOffset(1),
+                                       CS.getRBracLoc());
+}
+
+RangeSelector tooling::statements(StringRef ID) {
+  return RelativeSelector<CompoundStmt, getStatementsRange>(ID);
+}
+
+// Returns the range of the source between the call's parentheses.
+static CharSourceRange getCallArgumentsRange(const MatchResult &Result,
+                                             const CallExpr &CE) {
+  return CharSourceRange::getCharRange(
+      findOpenParen(CE, *Result.SourceManager, Result.Context->getLangOpts())
+          .getLocWithOffset(1),
+      CE.getRParenLoc());
+}
+
+RangeSelector tooling::callArgs(StringRef ID) {
+  return RelativeSelector<CallExpr, getCallArgumentsRange>(ID);
+}
+
+// Returns the range of the elements of the initializer list. Includes all
+// source between the braces.
+static CharSourceRange getElementsRange(const MatchResult &,
+                                        const InitListExpr &E) {
+  return CharSourceRange::getCharRange(E.getLBraceLoc().getLocWithOffset(1),
+                                       E.getRBraceLoc());
+}
+
+RangeSelector tooling::initListElements(StringRef ID) {
+  return RelativeSelector<InitListExpr, getElementsRange>(ID);
+}
+
+RangeSelector tooling::expansion(RangeSelector S) {
+  return [S](const MatchResult &Result) -> Expected<CharSourceRange> {
+    Expected<CharSourceRange> SRange = S(Result);
+    if (!SRange)
+      return SRange.takeError();
+    return Result.SourceManager->getExpansionRange(*SRange);
+  };
+}

clang/unittests/Tooling/CMakeLists.txt

add_clang_unittest(ToolingTests
   CompilationDatabaseTest.cpp
   DiagnosticsYamlTest.cpp
   ExecutionTest.cpp
   FixItTest.cpp
   HeaderIncludesTest.cpp
   LexicallyOrderedRecursiveASTVisitorTest.cpp
   LookupTest.cpp
   QualTypeNamesTest.cpp
+  RangeSelectorTest.cpp
   RecursiveASTVisitorTests/Attr.cpp
   RecursiveASTVisitorTests/Class.cpp
   RecursiveASTVisitorTests/ConstructExpr.cpp
   RecursiveASTVisitorTests/CXXBoolLiteralExpr.cpp
   RecursiveASTVisitorTests/CXXMemberCall.cpp
   RecursiveASTVisitorTests/CXXOperatorCallExprTraverser.cpp
   RecursiveASTVisitorTests/DeclRefExpr.cpp
   RecursiveASTVisitorTests/ImplicitCtor.cpp
target_link_libraries(ToolingTests
   clangFrontend
   clangLex
   clangRewrite
   clangSerialization
   clangTooling
   clangToolingCore
   clangToolingInclusions
   clangToolingRefactor
+  LLVMSupport
+  LLVMTestingSupport
   )

clang/unittests/Tooling/RangeSelectorTest.cpp

+//===- unittest/Tooling/RangeSelectorTest.cpp
+//-----------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Tooling/Refactoring/RangeSelector.h"
+#include "clang/ASTMatchers/ASTMatchers.h"
+#include "clang/Tooling/FixIt.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Testing/Support/Error.h"
+#include "gmock/gmock.h"
+#include "gtest/gtest.h"
+
+using namespace clang;
+using namespace tooling;
+using namespace ast_matchers;
+
+namespace {
+using ::testing::AllOf;
+using ::testing::HasSubstr;
+using MatchResult = MatchFinder::MatchResult;
+using ::llvm::Expected;
+using ::llvm::Failed;
+using ::llvm::HasValue;
+using ::llvm::StringError;
+
+struct TestMatch {
+  // The AST unit from which `result` is built. We bundle it because it backs
+  // the result. Users are not expected to access it.
+  std::unique_ptr<ASTUnit> ASTUnit;
+  // The result to use in the test. References `ast_unit`.
+  MatchResult Result;
+};
+
+template <typename M> TestMatch matchCode(StringRef Code, M Matcher) {
+  auto ASTUnit = buildASTFromCode(Code);
+  assert(ASTUnit != nullptr && "AST construction failed");
+
+  ASTContext &Context = ASTUnit->getASTContext();
+  assert(!Context.getDiagnostics().hasErrorOccurred() && "Compilation error");
+
+  auto Matches = ast_matchers::match(Matcher, Context);
+  // We expect a single, exact match.
+  assert(Matches.size() != 0 && "no matches found");
+  assert(Matches.size() == 1 && "too many matches");
+
+  return TestMatch{std::move(ASTUnit), MatchResult(Matches[0], &Context)};
+}
+
+// Applies \p Selector to \p Match and, on success, returns the selected source.
+Expected<StringRef> apply(RangeSelector Selector, const TestMatch &Match) {
+  Expected<CharSourceRange> Range = Selector(Match.Result);
+  if (!Range)
+    return Range.takeError();
+  return fixit::internal::getText(*Range, *Match.Result.Context);
+}
+
+// Applies \p Selector to a trivial match with only a single bound node with id
+// "bound_node_id".  For use in testing unbound-node errors.
+Expected<CharSourceRange> applyToTrivial(const RangeSelector &Selector) {
+  // We need to bind the result to something, or the match will fail. Use a
+  // binding that is not used in the unbound node tests.
+  TestMatch Match =
+      matchCode("static int x = 0;", varDecl().bind("bound_node_id"));
+  return Selector(Match.Result);
+}
+
+// Matches the message expected for unbound-node failures.
+testing::Matcher<StringError> withUnboundNodeMessage() {
+  return testing::Property(
+      &StringError::getMessage,
+      AllOf(HasSubstr("unbound_id"), HasSubstr("not bound")));
+}
+
+// Applies \p Selector to code containing assorted node types, where the match
+// binds each one: a statement ("stmt"), a (non-member) ctor-initializer
+// ("init"), an expression ("expr") and a (nameless) declaration ("decl").  Used
+// to test failures caused by applying selectors to nodes of the wrong type.
+Expected<CharSourceRange> applyToAssorted(RangeSelector Selector) {
+  StringRef Code = R"cc(
+      struct A {};
+      class F : public A {
+       public:
+        F(int) {}
+      };
+      void g() { F f(1); }
+    )cc";
+
+  auto Matcher =
+      compoundStmt(
+          hasDescendant(
+              cxxConstructExpr(
+                  hasDeclaration(
+                      decl(hasDescendant(cxxCtorInitializer(isBaseInitializer())
+                                             .bind("init")))
+                          .bind("decl")))
+                  .bind("expr")))
+          .bind("stmt");
+
+  return Selector(matchCode(Code, Matcher).Result);
+}
+
+// Matches the message expected for type-error failures.
+testing::Matcher<StringError> withTypeErrorMessage(StringRef NodeID) {
+  return testing::Property(
+      &StringError::getMessage,
+      AllOf(HasSubstr(NodeID), HasSubstr("mismatched type")));
+}
+
+TEST(RangeSelectorTest, UnboundNode) {
+  EXPECT_THAT_EXPECTED(applyToTrivial(node("unbound_id")),
+                       Failed<StringError>(withUnboundNodeMessage()));
+}
+
+TEST(RangeSelectorTest, RangeOp) {
+  StringRef Code = R"cc(
+    int f(int x, int y, int z) { return 3; }
+    int g() { return f(/* comment */ 3, 7 /* comment */, 9); }
+  )cc";
+  StringRef Arg0 = "a0";
+  StringRef Arg1 = "a1";
+  StringRef Call = "call";
+  auto Matcher = callExpr(hasArgument(0, expr().bind(Arg0)),
+                          hasArgument(1, expr().bind(Arg1)))
+                     .bind(Call);
+  TestMatch Match = matchCode(Code, Matcher);
+
+  // Node-id specific version:
+  EXPECT_THAT_EXPECTED(apply(range(Arg0, Arg1), Match), HasValue("3, 7"));
+  // General version:
+  EXPECT_THAT_EXPECTED(apply(range(node(Arg0), node(Arg1)), Match),
+                       HasValue("3, 7"));
+}
+
+TEST(RangeSelectorTest, NodeOpStatement) {
+  StringRef Code = "int f() { return 3; }";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, returnStmt().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(node(ID), Match), HasValue("return 3;"));
+}
+
+TEST(RangeSelectorTest, NodeOpExpression) {
+  StringRef Code = "int f() { return 3; }";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, expr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(node(ID), Match), HasValue("3"));
+}
+
+TEST(RangeSelectorTest, StatementOp) {
+  StringRef Code = "int f() { return 3; }";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, expr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(statement(ID), Match), HasValue("3;"));
+}
+
+TEST(RangeSelectorTest, MemberOp) {
+  StringRef Code = R"cc(
+    struct S {
+      int member;
+    };
+    int g() {
+      S s;
+      return s.member;
+    }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, memberExpr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(member(ID), Match), HasValue("member"));
+}
+
+// Tests that member does not select any qualifiers on the member name.
+TEST(RangeSelectorTest, MemberOpQualified) {
+  StringRef Code = R"cc(
+    struct S {
+      int member;
+    };
+    struct T : public S {
+      int field;
+    };
+    int g() {
+      T t;
+      return t.S::member;
+    }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, memberExpr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(member(ID), Match), HasValue("member"));
+}
+
+TEST(RangeSelectorTest, MemberOpTemplate) {
+  StringRef Code = R"cc(
+    struct S {
+      template <typename T> T foo(T t);
+    };
+    int f(int x) {
+      S s;
+      return s.template foo<int>(3);
+    }
+  )cc";
+
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, memberExpr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(member(ID), Match), HasValue("foo"));
+}
+
+TEST(RangeSelectorTest, MemberOpOperator) {
+  StringRef Code = R"cc(
+    struct S {
+      int operator*();
+    };
+    int f(int x) {
+      S s;
+      return s.operator *();
+    }
+  )cc";
+
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, memberExpr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(member(ID), Match), HasValue("operator *"));
+}
+
+TEST(RangeSelectorTest, NameOpNamedDecl) {
+  StringRef Code = R"cc(
+    int myfun() {
+      return 3;
+    }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, functionDecl().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(name(ID), Match), HasValue("myfun"));
+}
+
+TEST(RangeSelectorTest, NameOpDeclRef) {
+  StringRef Code = R"cc(
+    int foo(int x) {
+      return x;
+    }
+    int g(int x) { return foo(x) * x; }
+  )cc";
+  StringRef Ref = "ref";
+  TestMatch Match = matchCode(Code, declRefExpr(to(functionDecl())).bind(Ref));
+  EXPECT_THAT_EXPECTED(apply(name(Ref), Match), HasValue("foo"));
+}
+
+TEST(RangeSelectorTest, NameOpCtorInitializer) {
+  StringRef Code = R"cc(
+    class C {
+     public:
+      C() : field(3) {}
+      int field;
+    };
+  )cc";
+  StringRef Init = "init";
+  TestMatch Match = matchCode(Code, cxxCtorInitializer().bind(Init));
+  EXPECT_THAT_EXPECTED(apply(name(Init), Match), HasValue("field"));
+}
+
+TEST(RangeSelectorTest, NameOpErrors) {
+  EXPECT_THAT_EXPECTED(applyToTrivial(name("unbound_id")),
+                       Failed<StringError>(withUnboundNodeMessage()));
+  EXPECT_THAT_EXPECTED(applyToAssorted(name("stmt")),
+                       Failed<StringError>(withTypeErrorMessage("stmt")));
+}
+
+TEST(RangeSelectorTest, NameOpDeclRefError) {
+  StringRef Code = R"cc(
+    struct S {
+      int operator*();
+    };
+    int f(int x) {
+      S s;
+      return *s + x;
+    }
+  )cc";
+  StringRef Ref = "ref";
+  TestMatch Match = matchCode(Code, declRefExpr(to(functionDecl())).bind(Ref));
+  EXPECT_THAT_EXPECTED(
+      name(Ref)(Match.Result),
+      Failed<StringError>(testing::Property(
+          &StringError::getMessage,
+          AllOf(HasSubstr(Ref), HasSubstr("requires property 'identifier'")))));
+}
+
+TEST(RangeSelectorTest, CallArgsOp) {
+  const StringRef Code = R"cc(
+    struct C {
+      int bar(int, int);
+    };
+    int f() {
+      C x;
+      return x.bar(3, 4);
+    }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, callExpr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(callArgs(ID), Match), HasValue("3, 4"));
+}
+
+TEST(RangeSelectorTest, CallArgsOpNoArgs) {
+  const StringRef Code = R"cc(
+    struct C {
+      int bar();
+    };
+    int f() {
+      C x;
+      return x.bar();
+    }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, callExpr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(callArgs(ID), Match), HasValue(""));
+}
+
+TEST(RangeSelectorTest, CallArgsOpNoArgsWithComments) {
+  const StringRef Code = R"cc(
+    struct C {
+      int bar();
+    };
+    int f() {
+      C x;
+      return x.bar(/*empty*/);
+    }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, callExpr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(callArgs(ID), Match), HasValue("/*empty*/"));
+}
+
+// Tests that arguments are extracted correctly when a temporary (with parens)
+// is used.
+TEST(RangeSelectorTest, CallArgsOpWithParens) {
+  const StringRef Code = R"cc(
+    struct C {
+      int bar(int, int) { return 3; }
+    };
+    int f() {
+      C x;
+      return C().bar(3, 4);
+    }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match =
+      matchCode(Code, callExpr(callee(functionDecl(hasName("bar")))).bind(ID));
+  EXPECT_THAT_EXPECTED(apply(callArgs(ID), Match), HasValue("3, 4"));
+}
+
+TEST(RangeSelectorTest, CallArgsOpLeadingComments) {
+  const StringRef Code = R"cc(
+    struct C {
+      int bar(int, int) { return 3; }
+    };
+    int f() {
+      C x;
+      return x.bar(/*leading*/ 3, 4);
+    }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, callExpr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(callArgs(ID), Match),
+                       HasValue("/*leading*/ 3, 4"));
+}
+
+TEST(RangeSelectorTest, CallArgsOpTrailingComments) {
+  const StringRef Code = R"cc(
+    struct C {
+      int bar(int, int) { return 3; }
+    };
+    int f() {
+      C x;
+      return x.bar(3 /*trailing*/, 4);
+    }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, callExpr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(callArgs(ID), Match),
+                       HasValue("3 /*trailing*/, 4"));
+}
+
+TEST(RangeSelectorTest, CallArgsOpEolComments) {
+  const StringRef Code = R"cc(
+    struct C {
+      int bar(int, int) { return 3; }
+    };
+    int f() {
+      C x;
+      return x.bar(  // Header
+          1,           // foo
+          2            // bar
+      );
+    }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, callExpr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(callArgs(ID), Match), HasValue(R"(  // Header
+          1,           // foo
+          2            // bar
+      )"));
+}
+
+TEST(RangeSelectorTest, CallArgsErrors) {
+  EXPECT_THAT_EXPECTED(applyToTrivial(callArgs("unbound_id")),
+                       Failed<StringError>(withUnboundNodeMessage()));
+  EXPECT_THAT_EXPECTED(applyToAssorted(callArgs("stmt")),
+                       Failed<StringError>(withTypeErrorMessage("stmt")));
+}
+
+TEST(RangeSelectorTest, StatementsOp) {
+  StringRef Code = R"cc(
+    void g();
+    void f() { /* comment */ g(); /* comment */ g(); /* comment */ }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, compoundStmt().bind(ID));
+  EXPECT_THAT_EXPECTED(
+      apply(statements(ID), Match),
+      HasValue(" /* comment */ g(); /* comment */ g(); /* comment */ "));
+}
+
+TEST(RangeSelectorTest, StatementsOpEmptyList) {
+  StringRef Code = "void f() {}";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, compoundStmt().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(statements(ID), Match), HasValue(""));
+}
+
+TEST(RangeSelectorTest, StatementsOpErrors) {
+  EXPECT_THAT_EXPECTED(applyToTrivial(statements("unbound_id")),
+                       Failed<StringError>(withUnboundNodeMessage()));
+  EXPECT_THAT_EXPECTED(applyToAssorted(statements("decl")),
+                       Failed<StringError>(withTypeErrorMessage("decl")));
+}
+
+TEST(RangeSelectorTest, ElementsOp) {
+  StringRef Code = R"cc(
+    void f() {
+      int v[] = {/* comment */ 3, /* comment*/ 4 /* comment */};
+      (void)v;
+    }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, initListExpr().bind(ID));
+  EXPECT_THAT_EXPECTED(
+      apply(initListElements(ID), Match),
+      HasValue("/* comment */ 3, /* comment*/ 4 /* comment */"));
+}
+
+TEST(RangeSelectorTest, ElementsOpEmptyList) {
+  StringRef Code = R"cc(
+    void f() {
+      int v[] = {};
+      (void)v;
+    }
+  )cc";
+  StringRef ID = "id";
+  TestMatch Match = matchCode(Code, initListExpr().bind(ID));
+  EXPECT_THAT_EXPECTED(apply(initListElements(ID), Match), HasValue(""));
+}
+
+TEST(RangeSelectorTest, ElementsOpErrors) {
+  EXPECT_THAT_EXPECTED(applyToTrivial(initListElements("unbound_id")),
+                       Failed<StringError>(withUnboundNodeMessage()));
+  EXPECT_THAT_EXPECTED(applyToAssorted(initListElements("stmt")),
+                       Failed<StringError>(withTypeErrorMessage("stmt")));
+}
+
+// Tests case where the matched node is the complete expanded text.
+TEST(RangeSelectorTest, ExpansionOp) {
+  StringRef Code = R"cc(
+#define BADDECL(E) int bad(int x) { return E; }
+    BADDECL(x * x)
+  )cc";
+
+  StringRef Fun = "Fun";
+  TestMatch Match = matchCode(Code, functionDecl(hasName("bad")).bind(Fun));
+  EXPECT_THAT_EXPECTED(apply(expansion(node(Fun)), Match),
+                       HasValue("BADDECL(x * x)"));
+}
+
+// Tests case where the matched node is (only) part of the expanded text.
+TEST(RangeSelectorTest, ExpansionOpPartial) {
+  StringRef Code = R"cc(
+#define BADDECL(E) int bad(int x) { return E; }
+    BADDECL(x * x)
+  )cc";
+
+  StringRef Ret = "Ret";
+  TestMatch Match = matchCode(Code, returnStmt().bind(Ret));
+  EXPECT_THAT_EXPECTED(apply(expansion(node(Ret)), Match),
+                       HasValue("BADDECL(x * x)"));
+}
+
+} // namespace