[clang-tidy] Fix a crash issue when clang-tidy runs with compilation database.

[hokein]

Summary

The clang-tidy will trigger an assertion if it's not in the building directory.

TEST:
cd <llvm-repo>/
./build/bin/clang-tidy --checks=-*,modernize-use-nullptr -p build tools/clang/tools/extra/clang-tidy/ClangTidy.cpp

The crash issue is gone after applying this patch.

Fixes PR24834, PR26241

[alexfh]

Thank you for working on this!

This change needs a lit test (ensure it fails/crashes without the patch and works fine after the patch).

clang-tidy/ClangTidyDiagnosticConsumer.cpp

346

Should this code handle possible errors from getCurrentWorkingDirectory()?

clang-tidy/ClangTidyDiagnosticConsumer.h

62

Please add a comment and explain also how this relates to ClangTidyMessage::FilePath and clang::tooling::Replacement::FilePath.

Additionally, describe possible values. For example, can this be an absolute path, a relative path or both? Can this be empty?

[hokein]

• Add lit test.
• Address review comments.

[hokein]

clang-tidy/ClangTidyDiagnosticConsumer.cpp

346

Yeah, you are right. We need to handle the error here since in some cases like the source file is not exit, the SourceManager is unavailable.

[alexfh]

clang-tidy/ClangTidy.cpp

121

Maybe Change the directory to the one used during the analysis.?

clang-tidy/ClangTidyDiagnosticConsumer.cpp

122

This should be a StringRef for consistency.

clang-tidy/ClangTidyDiagnosticConsumer.h

67

compile_commands.json is one format of a compilation database, there are others. So it might be not very useful to name it here.

71

does not exist

test/clang-tidy/clang-tidy-run-with-database.cpp

2

I think, this test now needs // REQUIRES: shell

3

The test shouldn't change anything in the source directory. It can create any setup it needs in the temporary directory though (the %t placeholder would be useful for this).

[hokein]

• Address review comments.

[hokein]

clang-tidy/ClangTidyDiagnosticConsumer.cpp

122

We might not use StringRef here since
SourceManager.getFileManager().getVirtualFileSystem().getCurrentWorkingDirectory() returns std::string.

clang-tidy/ClangTidyDiagnosticConsumer.h

67

However, I see the compile_commands.json is hardcoded in source. http://clang.llvm.org/doxygen/JSONCompilationDatabase_8cpp_source.html#l00111

[alexfh]

clang-tidy/ClangTidyDiagnosticConsumer.cpp

122

As discussed offline, it makes sense to avoid calling SourceManager.getFileManager().getVirtualFileSystem().getCurrentWorkingDirectory() for each error, since it already has a cost of at least a string allocation/copy, but might also be an actual I/O operation. We can assume the working directory doesn't change while parsing a file, so we can cache it in the ClangTidyContext together with the current file name. Once we do this, we can get a StringRef to the working directory stored in ClangTidyContext and pass it here (but still store a std::string to avoid dependencies on a lifetime of an external string).

clang-tidy/ClangTidyDiagnosticConsumer.h

67

Yes, but this is just one implementation of a compilation database. There are other implementations as well.

67

nit: "the compilation database"

[hokein]

• Don't save BuildDirectory for every error in each check.

[hokein]

Some cleanup.

[hokein]

Remove an extra blank line.

[hokein]

Address one comment.

[hokein]

clang-tidy/ClangTidyDiagnosticConsumer.cpp

122

Done. Now there is no need to add BuildDirectory in each ClangTidyError.

[alexfh]

In D17335#358613, @hokein wrote:

• Don't save BuildDirectory for every error in each check.

I think, it's not going to work with multiple files. There's just one Stats instance.

test/clang-tidy/clang-tidy-run-with-database.cpp

3

We need to test more stuff here, at least these come to mind:

1. -fix
2. multiple files in one clang-tidy invocation
3. files with the same name in different directories

[hokein]

Update:

• Update test to apply multiple files and fixes.
• Make it work with multiple files.

[hokein]

Make apply-fix work on relative path in command field of compilation database.

[alexfh]

clang-tidy/ClangTidyDiagnosticConsumer.h

75

As discussed offline, I'd like ClangTidyError to be independent of ClangTidyContext, unless this particular optimization saves significant resources.

[hokein]

Save the build directory in each error, making ClangTidyError not rely on ClangTidyContext.
The benchmark result shows that using StringRef doesn't improve performance greatly.

[hokein]

Don't modify unrelevant code.

[hokein]

Simplify test code.

[alexfh]

test/clang-tidy/clang-tidy-run-with-database.cpp

3

You don't need to create the root directory, since the mkdir -p calls below will take care of this.

16

I'm not sure how the rest of the test works. This grep -v call leaves everything except for the CHECK-lines in the %t.cpp, doesn't it?

17

Why don't you use %s here?

[hokein]

Address review comments.

[hokein]

test/clang-tidy/clang-tidy-run-with-database.cpp

17

Yeah, exactly.

[alexfh]

Looks good. Thank you for fixing this!

[hokein]

Nit: correct a comment.

clang-tidy/ClangTidy.h

runClangTidy(std::unique_ptr<ClangTidyOptionsProvider> OptionsProvider,
              ProfileData *Profile = nullptr);
 
 // FIXME: This interface will need to be significantly extended to be useful.
 // FIXME: Implement confidence levels for displaying/fixing errors.
 //
 /// \brief Displays the found \p Errors to the users. If \p Fix is true, \p
 /// Errors containing fixes are automatically applied.
 void handleErrors(const std::vector<ClangTidyError> &Errors, bool Fix,
-                  unsigned &WarningsAsErrorsCount);
+                  unsigned &WarningsAsErrorsCount,
+                  StringRef BuildDirectory);
 
 /// \brief Serializes replacements into YAML and writes them to the specified
 /// output stream.
 void exportReplacements(const std::vector<ClangTidyError> &Errors,
                         raw_ostream &OS);
 
 } // end namespace tidy
 } // end namespace clang
 
 #endif // LLVM_CLANG_TOOLS_EXTRA_CLANG_TIDY_CLANGTIDY_H

clang-tidy/ClangTidy.cpp

public:
   bool supportsCrossFileDiagnostics() const override { return true; }
 
 private:
   ClangTidyContext &Context;
 };
 
 class ErrorReporter {
 public:
-  ErrorReporter(bool ApplyFixes)
+  ErrorReporter(bool ApplyFixes, StringRef BuildDirectory)
       : Files(FileSystemOptions()), DiagOpts(new DiagnosticOptions()),
         DiagPrinter(new TextDiagnosticPrinter(llvm::outs(), &*DiagOpts)),
         Diags(IntrusiveRefCntPtr<DiagnosticIDs>(new DiagnosticIDs), &*DiagOpts,
               DiagPrinter),
         SourceMgr(Diags, Files), Rewrite(SourceMgr, LangOpts),
         ApplyFixes(ApplyFixes), TotalFixes(0), AppliedFixes(0),
         WarningsAsErrors(0) {
+    // By default, the working directory of file system is the current
+    // clang-tidy running directory.
+    //
+    // Change the directory to the one used during the analysis.
+    if (!BuildDirectory.empty())
+      Files.getVirtualFileSystem()->setCurrentWorkingDirectory(BuildDirectory);
     DiagOpts->ShowColors = llvm::sys::Process::StandardOutHasColors();
     DiagPrinter->BeginSourceFile(LangOpts);
   }
 
   void reportDiagnostic(const ClangTidyError &Error) {
     const ClangTidyMessage &Message = Error.Message;
     SourceLocation Loc = getLocation(Message.FilePath, Message.FileOffset);
     // Contains a pair for each attempted fix: location and whether the fix was
     // applied successfully.
     SmallVector<std::pair<SourceLocation, bool>, 4> FixLocations;
     {
       auto Level = static_cast<DiagnosticsEngine::Level>(Error.DiagLevel);
       std::string Name = Error.CheckName;
       if (Error.IsWarningAsError) {
         Name += ",-warnings-as-errors";
         Level = DiagnosticsEngine::Error;
         WarningsAsErrors++;
       }

 std::unique_ptr<clang::ASTConsumer>
 ClangTidyASTConsumerFactory::CreateASTConsumer(
     clang::CompilerInstance &Compiler, StringRef File) {
   // FIXME: Move this to a separate method, so that CreateASTConsumer doesn't
   // modify Compiler.
   Context.setSourceManager(&Compiler.getSourceManager());
   Context.setCurrentFile(File);
   Context.setASTContext(&Compiler.getASTContext());
+  auto WorkingDir = Compiler.getSourceManager()
+                        .getFileManager()
+                        .getVirtualFileSystem()
+                        ->getCurrentWorkingDirectory();
+  if (WorkingDir)
+    Context.setBuildDirectoryForStats(WorkingDir.get());
 
   std::vector<std::unique_ptr<ClangTidyCheck>> Checks;
   CheckFactories->createChecks(&Context, Checks);
 
   ast_matchers::MatchFinder::MatchFinderOptions FinderOptions;
   if (auto *P = Context.getCheckProfileData())
     FinderOptions.CheckProfiling.emplace(P->Records);
 
runClangTidy(std::unique_ptr<ClangTidyOptionsProvider> OptionsProvider,
 
   ActionFactory Factory(Context);
   Tool.run(&Factory);
   *Errors = Context.getErrors();
   return Context.getStats();
 }
 
 void handleErrors(const std::vector<ClangTidyError> &Errors, bool Fix,
-                  unsigned &WarningsAsErrorsCount) {
-  ErrorReporter Reporter(Fix);
+                  unsigned &WarningsAsErrorsCount,
+                  StringRef BuildDirectory) {
+  ErrorReporter Reporter(Fix, BuildDirectory);
   for (const ClangTidyError &Error : Errors)
     Reporter.reportDiagnostic(Error);
   Reporter.Finish();
   WarningsAsErrorsCount += Reporter.getWarningsAsErrorsCount();
 }
 
 void exportReplacements(const std::vector<ClangTidyError> &Errors,
                         raw_ostream &OS) {

clang-tidy/ClangTidyDiagnosticConsumer.h

 struct ClangTidyError {
   enum Level {
     Warning = DiagnosticsEngine::Warning,
     Error = DiagnosticsEngine::Error
   };
 
   ClangTidyError(StringRef CheckName, Level DiagLevel, bool IsWarningAsError);
 
   std::string CheckName;
   ClangTidyMessage Message;
   tooling::Replacements Fix;
   SmallVector<ClangTidyMessage, 1> Notes;
 
   Level DiagLevel;
   bool IsWarningAsError;
 };
 
 /// \brief Read-only set of strings represented as a list of positive and
 /// negative globs. Positive globs add all matched strings to the set, negative
 /// globs remove them in the order of appearance in the list.
 class GlobList {
 public:
   /// \brief \p GlobList is a comma-separated list of globs (only '*'
   /// metacharacter is supported) with optional '-' prefix to denote exclusion.
   GlobList(StringRef Globs);
 
   /// \brief Returns \c true if the pattern matches \p S. The result is the last
   /// matching glob's Positive flag.
   bool contains(StringRef S) { return contains(S, false); }
 
ClangTidyStats()
       : ErrorsDisplayed(0), ErrorsIgnoredCheckFilter(0), ErrorsIgnoredNOLINT(0),
         ErrorsIgnoredNonUserCode(0), ErrorsIgnoredLineFilter(0) {}
 
   unsigned ErrorsDisplayed;
   unsigned ErrorsIgnoredCheckFilter;
   unsigned ErrorsIgnoredNOLINT;
   unsigned ErrorsIgnoredNonUserCode;
   unsigned ErrorsIgnoredLineFilter;
+  // A build directory of the source file containing the error.
+  //
+  // It's an absolute path which is `directory` field of the source file in
+  // compilation database. If users don't specify compilation database
+  // directory, it is the current directory where clang-tidy runs.
+  //
+  // It can be empty in some cases, e.g. the source file does not exist.
+  std::string BuildDirectory;
 
   unsigned errorsIgnored() const {
     return ErrorsIgnoredNOLINT + ErrorsIgnoredCheckFilter +
            ErrorsIgnoredNonUserCode + ErrorsIgnoredLineFilter;
   }
 };
 
 /// \brief Container for clang-tidy profiling data.
public:
   /// \brief Returns options for \c File. Does not change or depend on
   /// \c CurrentFile.
   ClangTidyOptions getOptionsForFile(StringRef File) const;
 
   /// \brief Returns \c ClangTidyStats containing issued and ignored diagnostic
   /// counters.
   const ClangTidyStats &getStats() const { return Stats; }
 
+  void setBuildDirectoryForStats(StringRef BuildDirectory);
+
   /// \brief Returns all collected errors.
   const std::vector<ClangTidyError> &getErrors() const { return Errors; }
 
   /// \brief Clears collected errors.
   void clearErrors() { Errors.clear(); }
 
   /// \brief Set the output struct for profile data.
   ///

clang-tidy/ClangTidyDiagnosticConsumer.cpp

ClangTidyMessage::ClangTidyMessage(StringRef Message,
   FileOffset = Sources.getFileOffset(Loc);
 }
 
 ClangTidyError::ClangTidyError(StringRef CheckName,
                                ClangTidyError::Level DiagLevel,
                                bool IsWarningAsError)
     : CheckName(CheckName), DiagLevel(DiagLevel),
       IsWarningAsError(IsWarningAsError) {}
 
 // Returns true if GlobList starts with the negative indicator ('-'), removes it
 // from the GlobList.
 static bool ConsumeNegativeIndicator(StringRef &GlobList) {
   if (GlobList.startswith("-")) {
     GlobList = GlobList.substr(1);
     return true;
   }
   return false;

 
 ClangTidyOptions ClangTidyContext::getOptionsForFile(StringRef File) const {
   // Merge options on top of getDefaults() as a safeguard against options with
   // unset values.
   return ClangTidyOptions::getDefaults().mergeWith(
       OptionsProvider->getOptions(File));
 }
 
+void ClangTidyContext::setBuildDirectoryForStats(StringRef BuildDirectory) {
+  Stats.BuildDirectory = BuildDirectory;
+}
+
 void ClangTidyContext::setCheckProfileData(ProfileData *P) { Profile = P; }
 
 GlobList &ClangTidyContext::getChecksFilter() {
   assert(CheckFilter != nullptr);
   return *CheckFilter;
 }
 
 bool ClangTidyContext::isCheckEnabled(StringRef CheckName) const {
if (DiagLevel == DiagnosticsEngine::Note) {
     }
     bool IsWarningAsError =
         DiagLevel == DiagnosticsEngine::Warning &&
         Context.getWarningAsErrorFilter().contains(CheckName);
     Errors.push_back(ClangTidyError(CheckName, Level, IsWarningAsError));
   }
 
   // FIXME: Provide correct LangOptions for each file.
   LangOptions LangOpts;
   ClangTidyDiagnosticRenderer Converter(
       LangOpts, &Context.DiagEngine->getDiagnosticOptions(), Errors.back());
   SmallString<100> Message;
   Info.FormatDiagnostic(Message);
   SourceManager *Sources = nullptr;
   if (Info.hasSourceManager())
     Sources = &Info.getSourceManager();
   Converter.emitDiagnostic(Info.getLocation(), DiagLevel, Message,

clang-tidy/tool/ClangTidyMain.cpp

bool FoundErrors =
         return E.DiagLevel == ClangTidyError::Error;
       }) != Errors.end();
 
   const bool DisableFixes = Fix && FoundErrors && !FixErrors;
 
   unsigned WErrorCount = 0;
 
   // -fix-errors implies -fix.
-  handleErrors(Errors, (FixErrors || Fix) && !DisableFixes, WErrorCount);
+  handleErrors(Errors, (FixErrors || Fix) && !DisableFixes, WErrorCount,
+               Stats.BuildDirectory);
 
   if (!ExportFixes.empty() && !Errors.empty()) {
     std::error_code EC;
     llvm::raw_fd_ostream OS(ExportFixes, EC, llvm::sys::fs::F_None);
     if (EC) {
       llvm::errs() << "Error opening output file: " << EC.message() << '\n';
       return 1;
     }

test/clang-tidy/Inputs/compilation-database/header.h

+#define NULL 0
+
+int *a = NULL;

test/clang-tidy/Inputs/compilation-database/template.json

+[
+{
+  "directory": "test_dir",
+  "command": "clang++ -o test.o clang-tidy-run-with-database.cpp",
+  "file": "test_dir/clang-tidy-run-with-database.cpp"
+}
+]

test/clang-tidy/clang-tidy-run-with-database.cpp

+// REQUIRES: shell
+// RUN: sed 's|test_dir|%S|g' %S/Inputs/compilation-database/template.json > %T/compile_commands.json
+// RUN: clang-tidy --checks=-*,modernize-use-nullptr -p %T %s -header-filter=.* | FileCheck %s -implicit-check-not="{{warning:}}"
+
+#include "./Inputs/compilation-database/header.h"
+// CHECK: warning: use nullptr [modernize-use-nullptr]