Add clang-llvm-rename tool.

[ruiu]

Summary

Currently, this tool can rename variables in lld's source tree
without breaking it, but it is very unlikely that it will work
on any other LLVM subdirectories. I hand-tuned some renaming rules
to adopt it to lld's codebase.

This patch is not intended to be submitted, since this is a special-
purpose tool.

[Eugene.Zelenko]

There is clang-rename already. May be new functionality should be added there?

[ruiu]

• removed a special rule for E
• do not lowercase global variables whose name is all uppercase
• OSec -> osec

[ruiu]

In D64123#1568096, @Eugene.Zelenko wrote:

There is clang-rename already. May be new functionality should be added there?

clang-rename seems to focus on renaming a variable in a selected region with the interactive use in mind. I.e. a user selects a variable on an IDE and rename its definition and all references.

I actually tried to create this tool based on clang-rename but failed because clang-rename was too slow for my purpose. It takes a non-negligible amount of time even for renaming a single variable, and I wanted to rename thousands or tens of thousands variables.

There are a few other significant differences between this tool and clang-rename as shown below:

• this tool renames a variable only when it can see the definition of the variable, while clang-rename renames a specified variable unconditionally
• clang-rename is an interactive tool, and you need to specify a new name by hand. On the other hand, this tool automatically generates a new name for each variable
• the core part of my tool is to find variables to rename, while clang-rename doesn't do anything for that

Overall, this tool is I think significantly different from clang-rename.

[ruiu]

• updated a few special mappings

[michaelplatings]

Hi @ruiu,
Can you comment on how this compares to clang-tidy? I had assumed that the readability-identifier-naming clang-tidy rule would largely do the trick.

[ruiu]

I wasn't aware that clang-tidy had a such feature. readability-identifier-naming rule doesn't seem to work for this purpose out of the box. That being said, in hindsight, maybe I should have written this as a patch to clang-tidy instead of a new tool (which should have saved my time!)

[ruiu]

• Add a comment as to how to build and run clang-llvm-rename tool

clang-tools-extra/CMakeLists.txt

 include(CMakeDependentOption)
 
 add_subdirectory(clang-apply-replacements)
 add_subdirectory(clang-reorder-fields)
 add_subdirectory(modularize)
 add_subdirectory(clang-tidy)
 add_subdirectory(clang-tidy-vs)
+add_subdirectory(clang-llvm-rename)
 
 add_subdirectory(clang-change-namespace)
 add_subdirectory(clang-doc)
 add_subdirectory(clang-include-fixer)
 add_subdirectory(clang-move)
 add_subdirectory(clang-query)
 add_subdirectory(pp-trace)
 add_subdirectory(tool-template)

clang-tools-extra/clang-llvm-rename/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS
+  Support
+  )
+
+add_clang_executable(clang-llvm-rename
+  ClangLLVMRename.cpp
+  )
+
+target_link_libraries(clang-llvm-rename
+  PRIVATE
+  clangAST
+  clangASTMatchers
+  clangBasic
+  clangFrontend
+  clangTooling
+  clangToolingRefactoring
+  )
+set(LLVM_LINK_COMPONENTS
+  support
+  )

clang-tools-extra/clang-llvm-rename/ClangLLVMRename.cpp

+//===---- ClangLLVMRename.cpp ---------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This is a refactoring tool to rename variables so that they start with a
+// lowercase letter. This tool is intended to be used to rename variables in
+// LLVM codebase in which variable names start with an uppercase letter at
+// the moment.
+//
+//   Usage:
+//   clang-llmv-rename <file1> <file2> ...
+//
+// <file1> ... specify the paths of files in the CMake source tree. This
+// path is looked up in the compile command database.
+//
+//
+// For each variable in given files, the tool first check whether the
+// variable's definition is in one of given files or not, and rename it if
+// and only if it can find a definition of the variable. If the tool cannot
+// modify a definition of a variable, it doesn't rename it, in order to keep
+// a program compiles.
+//
+// Note that this tool is not perfect; it doesn't resolve or even detect
+// name conflicts caused by renaming. You may need to rename variables
+// before using this tool so that your program is free from name conflicts
+// due to lowercase/uppercase renaming.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/ASTMatchers/ASTMatchFinder.h"
+#include "clang/ASTMatchers/ASTMatchers.h"
+#include "clang/Basic/SourceManager.h"
+#include "clang/Frontend/FrontendActions.h"
+#include "clang/Frontend/TextDiagnosticPrinter.h"
+#include "clang/Lex/Lexer.h"
+#include "clang/Rewrite/Core/Rewriter.h"
+#include "clang/Tooling/CommonOptionsParser.h"
+#include "clang/Tooling/Execution.h"
+#include "clang/Tooling/Refactoring.h"
+#include "clang/Tooling/Refactoring/AtomicChange.h"
+#include "clang/Tooling/Tooling.h"
+#include "llvm/ADT/StringSet.h"
+#include "llvm/ADT/StringSwitch.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/Signals.h"
+
+using namespace clang;
+using namespace clang::ast_matchers;
+using namespace clang::tooling;
+using namespace llvm;
+
+namespace {
+class RenameCallback : public MatchFinder::MatchCallback {
+public:
+  RenameCallback(
+      std::map<std::string, tooling::Replacements> &FileToReplacements,
+      ArrayRef<std::string> Paths)
+      : FileToReplacements(FileToReplacements) {
+    for (StringRef S : Paths)
+      InputFiles.insert(canonicalizePath(S));
+  }
+
+  // This function is called for each AST pattern matche.
+  void run(const MatchFinder::MatchResult &Result) override {
+    SourceManager &SM = *Result.SourceManager;
+
+    if (auto *D = Result.Nodes.getNodeAs<NamedDecl>("VarDecl")) {
+      if (isa<ParmVarDecl>(D))
+        return;
+      if (isConstant(D->getName()))
+        return;
+      convert(SM, D->getLocation(), D->getName());
+      return;
+    }
+
+    if (auto *D = Result.Nodes.getNodeAs<ParmVarDecl>("ParmVarDecl")) {
+      if (auto *Fn =
+              dyn_cast_or_null<FunctionDecl>(D->getParentFunctionOrMethod()))
+        if (Fn->isImplicit())
+          return;
+      convert(SM, D->getLocation(), "");
+      return;
+    }
+
+    if (auto *D = Result.Nodes.getNodeAs<FieldDecl>("FieldDecl")) {
+      convert(SM, D->getLocation(), "");
+      return;
+    }
+
+    if (auto *D = Result.Nodes.getNodeAs<DeclRefExpr>("DeclRefExpr")) {
+      if (!isInGivenFiles(SM, D->getFoundDecl()->getLocation()))
+        return;
+      if (isa<EnumConstantDecl>(D->getDecl()) ||
+          isa<NonTypeTemplateParmDecl>(D->getDecl()))
+        return;
+      if (isConstant(D->getDecl()->getName()))
+        return;
+      if (D->getDecl()->getName().empty())
+        return;
+      convert(SM, D->getLocation(), "");
+      return;
+    }
+
+    if (auto *D = Result.Nodes.getNodeAs<MemberExpr>("MemberExpr")) {
+      if (!isInGivenFiles(SM, D->getFoundDecl()->getLocation()))
+        return;
+      if (D->getMemberDecl()->getName().empty())
+        return;
+      convert(SM, D->getMemberLoc(), D->getMemberDecl()->getName());
+      return;
+    }
+
+    if (auto *D =
+            Result.Nodes.getNodeAs<CXXCtorInitializer>("CXXCtorInitializer")) {
+      if (!isInGivenFiles(SM, D->getMemberLocation()))
+        return;
+      convert(SM, D->getSourceLocation(), D->getMember()->getName());
+    }
+
+    if (auto *D = Result.Nodes.getNodeAs<CXXDependentScopeMemberExpr>(
+            "CXXDependentScopeMemberExpr")) {
+      if (!isInGivenFiles(SM, D->getMemberLoc()))
+        return;
+
+      SourceLocation Loc = D->getBeginLoc();
+      int Len = Lexer::MeasureTokenLength(Loc, SM, LangOptions());
+      std::string Name = std::string(SM.getCharacterData(Loc), Len);
+      if (Name == "ELFT" || Name == "RelTy")
+        return;
+
+      convert(SM, D->getMemberLoc(),
+              D->getMemberNameInfo().getName().getAsString());
+    }
+  }
+
+  // Setup an AST matcher.
+  void registerMatchers(MatchFinder &M) {
+    M.addMatcher(varDecl(isExpansionInMainFile()).bind("VarDecl"), this);
+    M.addMatcher(parmVarDecl(isExpansionInMainFile()).bind("ParmVarDecl"),
+                 this);
+    M.addMatcher(fieldDecl(isExpansionInMainFile()).bind("FieldDecl"), this);
+    M.addMatcher(declRefExpr(isExpansionInMainFile()).bind("DeclRefExpr"),
+                 this);
+    M.addMatcher(memberExpr(isExpansionInMainFile()).bind("MemberExpr"), this);
+    M.addMatcher(cxxCtorInitializer().bind("CXXCtorInitializer"), this);
+    M.addMatcher(cxxDependentScopeMemberExpr(isExpansionInMainFile())
+                     .bind("CXXDependentScopeMemberExpr"),
+                 this);
+  }
+
+private:
+  // Returns true if a given source location is in an input file.
+  bool isInGivenFiles(SourceManager &SM, SourceLocation Loc) {
+    std::string Path = canonicalizePath(SM.getFilename(SM.getSpellingLoc(Loc)));
+    return InputFiles.count(Path);
+  }
+
+  // Returns true if S looks something like ALL_UPPERCASE_NAME.
+  bool isConstant(StringRef S) {
+    return S.size() > 3 &&
+           S.find_first_not_of("0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ_") ==
+               StringRef::npos;
+  }
+
+  // Renames an identifier so that it starts with a lowercase letter.
+  void convert(SourceManager &SM, SourceLocation Loc, StringRef MustMatch) {
+    if (!Loc.isValid())
+      return;
+
+    SourceLocation SpellingLoc = SM.getSpellingLoc(Loc);
+    if (!SM.isInMainFile(SpellingLoc))
+      return;
+
+    int Len = Lexer::MeasureTokenLength(SpellingLoc, SM, LangOptions());
+    std::string OldName = std::string(SM.getCharacterData(SpellingLoc), Len);
+
+    if (!MustMatch.empty() && OldName != MustMatch)
+      return;
+
+    std::string NewName = tolower(OldName);
+    tooling::Replacement Repl(SM, SpellingLoc, OldName.size(), NewName);
+    if (!Repl.getFilePath().empty())
+      consumeError(FileToReplacements[Repl.getFilePath()].add(Repl));
+  }
+
+  // Returns a new identifier name for a given identifier. Basically, a new
+  // name is differnet only at the first character, but for some identifiers
+  // such as all capital ones, there are special renaming rules.
+  std::string tolower(StringRef S) {
+    // Names that we do not want to convert.
+    if (S == "E")
+      return S;
+
+    // This is a list of special renaming rule.
+    std::string Ret = StringSwitch<std::string>(S)
+                          .Case("MBOrErr", "mbOrErr")
+                          .Case("CGProfile", "cgProfile")
+                          .Case("ISAddr", "isecAddr")
+                          .Case("ISLoc", "isecLoc")
+                          .Case("ISLimit", "isecLimit")
+                          .Case("TSBase", "tsBase")
+                          .Case("TSLimit", "tsLimit")
+                          .Case("Static", "isStatic")
+                          .Case("Pic", "isPic")
+                          .Case("Main", "mainPart")
+                          .Case("Class", "klass")
+                          .Default("");
+    if (!Ret.empty())
+      return Ret;
+
+    // If a new name would be a reserved word, don't rename.
+    for (StringRef S2 : {"new", "case", "default"})
+      if (S.lower() == S2)
+        return S;
+
+    // Convert all-uppercase names to all-smallcase.
+    if (S.find_first_not_of("0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ") ==
+        StringRef::npos)
+      return S.lower();
+
+    // They are converted to lowercase names.
+    for (StringRef S2 : {"MBRef", "EFlags", "EKind", "EMachine"})
+      if (S == S2)
+        return S2.lower();
+
+    // If an identifier starts with one of the folloiwng prefixes, the
+    // prefix will be converted to lowercase.
+    for (StringRef Prefix : {"LTO", "ARM", "PPC", "RISCV", "LMA", "VMA", "ELF",
+                             "ISD", "ABI", "LHS", "RHS", "VFP", "DTP", "OPT_"})
+      if (S.startswith(Prefix))
+        return (Prefix.lower() + S.substr(Prefix.size())).str();
+
+    // Otherwise, lowercase the first character.
+    Ret = S.str();
+    Ret[0] = std::tolower(Ret[0]);
+    return Ret;
+  }
+
+  // Removes "../" and make a resulting path absolute.
+  std::string canonicalizePath(StringRef Path) {
+    SmallString<128> S = Path;
+    sys::fs::make_absolute(S);
+    sys::path::remove_dots(S, /*remove_dot_dot=*/true);
+    return S.str();
+  }
+
+  std::map<std::string, tooling::Replacements> &FileToReplacements;
+  StringSet<> InputFiles;
+};
+} // end anonymous namespace
+
+// Set up the command line options
+static cl::extrahelp CommonHelp(CommonOptionsParser::HelpMessage);
+static cl::OptionCategory ToolTemplateCategory("tool-template options");
+
+int main(int argc, const char **argv) {
+  sys::PrintStackTraceOnErrorSignal(argv[0]);
+
+  tooling::CommonOptionsParser OptParser(argc, argv, ToolTemplateCategory);
+  const std::vector<std::string> &Files = OptParser.getSourcePathList();
+  tooling::RefactoringTool Tool(OptParser.getCompilations(), Files);
+
+  RenameCallback Callback(Tool.getReplacements(), Files);
+  MatchFinder Finder;
+  Callback.registerMatchers(Finder);
+
+  std::unique_ptr<tooling::FrontendActionFactory> Factory =
+      tooling::newFrontendActionFactory(&Finder);
+  return Tool.runAndSave(Factory.get());
+}