[Tooling] add a Heuristic field indicating that a CompileCommand was guessed.

[sammccall]

Summary

Use cases:

• a tool that dumps the heuristic used for each header in a project can be used to evaluate changes to the heuristic
• we want to expose this information to users in clangd as it affects accuracy/reliability of editor features
• express interpolation tests more directly

[ilya-biryukov]

I'll leave the LGTM to Manuel. I don't have a strong opinion here, but here are some thoughts.

The approach taken seems to be the least-leaky that I can think of unless we want to change the return type of CompilationDatabase::getCompileCommand or add a new method that would explicitly be used to infer commands. Both options are rather tricky to implement.
Yet, having the new field in the CompileCommand feels wrong, especially in combination with a fact that CompilationDatabase returns a vector of compile commands rather just a single command, i.e. after this patch implementations can technically return a combination of inferred and non-inferred commands for the same file, which might force clients to think about proper prioritization and filtering commands, something I'd rather keep hidden.

[ilya-biryukov]

LGTM, Manuel does not mind having this and the string heuristic does look like the easiest approach to convey this information in the API.

[sammccall]

In D60194#1454633, @ilya-biryukov wrote:

Yet, having the new field in the CompileCommand feels wrong, especially in combination with a fact that CompilationDatabase returns a vector of compile commands rather just a single command, i.e. after this patch implementations can technically return a combination of inferred and non-inferred commands for the same file, which might force clients to think about proper prioritization and filtering commands, something I'd rather keep hidden.

I do think the vector<CompileCommand> return isn't ideal, though I'm not sure it's really related to this change - the CDB can return inferred and non-inferred commands today, and there's no guidance on which to pick.
(I was thinking of adding some, but different tools do different things e.g. pick first, or iterate over all, and I'm not sure which is best :-/)

cfe/trunk/include/clang/Tooling/CompilationDatabase.h

struct CompileCommand {
   std::string Filename;
 
   /// The command line that was executed.
   std::vector<std::string> CommandLine;
 
   /// The output file associated with the command.
   std::string Output;
 
+  /// If this compile command was guessed rather than read from an authoritative
+  /// source, a short human-readable explanation.
+  /// e.g. "inferred from foo/bar.h".
+  std::string Heuristic;
+
   friend bool operator==(const CompileCommand &LHS, const CompileCommand &RHS) {
     return LHS.Directory == RHS.Directory && LHS.Filename == RHS.Filename &&
-           LHS.CommandLine == RHS.CommandLine && LHS.Output == RHS.Output;
+           LHS.CommandLine == RHS.CommandLine && LHS.Output == RHS.Output &&
+           LHS.Heuristic == RHS.Heuristic;
   }
 
   friend bool operator!=(const CompileCommand &LHS, const CompileCommand &RHS) {
     return !(LHS == RHS);
   }
 };
 
 /// Interface for compilation databases.

cfe/trunk/lib/Tooling/InterpolatingCompilationDatabase.cpp

CompileCommand transferTo(StringRef Filename) const {
     // --std flag may only be transferred if the language is the same.
     // We may consider "translating" these, e.g. c++11 -> c11.
     if (Std != LangStandard::lang_unspecified && foldType(TargetType) == Type) {
       Result.CommandLine.emplace_back((
           llvm::Twine(ClangCLMode ? "/std:" : "-std=") +
           LangStandard::getLangStandardForKind(Std).getName()).str());
     }
     Result.CommandLine.push_back(Filename);
+    Result.Heuristic = "inferred from " + Cmd.Filename;
     return Result;
   }
 
 private:
   // Determine whether the given command line is intended for the CL driver.
   static bool checkIsCLMode(ArrayRef<std::string> CmdLine) {
     // First look for --driver-mode.
     for (StringRef S : llvm::reverse(CmdLine)) {

cfe/trunk/unittests/Tooling/CompilationDatabaseTest.cpp

if (Results.empty())
       return "none";
     // drop the input file argument, so tests don't have to deal with path().
     EXPECT_EQ(Results[0].CommandLine.back(), path(F))
         << "Last arg should be the file";
     Results[0].CommandLine.pop_back();
     return llvm::join(Results[0].CommandLine, " ");
   }
 
+  // Parse the file whose command was used out of the Heuristic string.
+  std::string getProxy(llvm::StringRef F) {
+    auto Results =
+        inferMissingCompileCommands(llvm::make_unique<MemCDB>(Entries))
+            ->getCompileCommands(path(F));
+    if (Results.empty())
+      return "none";
+    StringRef Proxy = Results.front().Heuristic;
+    if (!Proxy.consume_front("inferred from "))
+      return "";
+    // We have a proxy file, convert back to a unix relative path.
+    // This is a bit messy, but we do need to test these strings somehow...
+    llvm::SmallString<32> TempDir;
+    llvm::sys::path::system_temp_directory(false, TempDir);
+    Proxy.consume_front(TempDir);
+    Proxy.consume_front(llvm::sys::path::get_separator());
+    llvm::SmallString<32> Result = Proxy;
+    llvm::sys::path::native(Result, llvm::sys::path::Style::posix);
+    return Result.str();
+  }
+
   MemCDB::EntryMap Entries;
 };
 
 TEST_F(InterpolateTest, Nearby) {
   add("dir/foo.cpp");
   add("dir/bar.cpp");
   add("an/other/foo.cpp");
 
   // great: dir and name both match (prefix or full, case insensitive)
-  EXPECT_EQ(getCommand("dir/f.cpp"), "clang -D dir/foo.cpp");
-  EXPECT_EQ(getCommand("dir/FOO.cpp"), "clang -D dir/foo.cpp");
+  EXPECT_EQ(getProxy("dir/f.cpp"), "dir/foo.cpp");
+  EXPECT_EQ(getProxy("dir/FOO.cpp"), "dir/foo.cpp");
   // no name match. prefer matching dir, break ties by alpha
-  EXPECT_EQ(getCommand("dir/a.cpp"), "clang -D dir/bar.cpp");
+  EXPECT_EQ(getProxy("dir/a.cpp"), "dir/bar.cpp");
   // an exact name match beats one segment of directory match
-  EXPECT_EQ(getCommand("some/other/bar.h"),
-            "clang -D dir/bar.cpp -x c++-header");
+  EXPECT_EQ(getProxy("some/other/bar.h"), "dir/bar.cpp");
   // two segments of directory match beat a prefix name match
-  EXPECT_EQ(getCommand("an/other/b.cpp"), "clang -D an/other/foo.cpp");
+  EXPECT_EQ(getProxy("an/other/b.cpp"), "an/other/foo.cpp");
   // if nothing matches at all, we still get the closest alpha match
-  EXPECT_EQ(getCommand("below/some/obscure/path.cpp"),
-            "clang -D an/other/foo.cpp");
+  EXPECT_EQ(getProxy("below/some/obscure/path.cpp"), "an/other/foo.cpp");
 }
 
 TEST_F(InterpolateTest, Language) {
   add("dir/foo.cpp", "-std=c++17");
   add("dir/bar.c", "");
   add("dir/baz.cee", "-x c");
 
   // .h is ambiguous, so we add explicit language flags
TEST_F(InterpolateTest, Strip) {
   // the -o option and the input file are removed, but -Wall is preserved.
   EXPECT_EQ(getCommand("dir/bar.cpp"), "clang -D dir/foo.cpp -Wall");
 }
 
 TEST_F(InterpolateTest, Case) {
   add("FOO/BAR/BAZ/SHOUT.cc");
   add("foo/bar/baz/quiet.cc");
   // Case mismatches are completely ignored, so we choose the name match.
-  EXPECT_EQ(getCommand("foo/bar/baz/shout.C"), "clang -D FOO/BAR/BAZ/SHOUT.cc");
+  EXPECT_EQ(getProxy("foo/bar/baz/shout.C"), "FOO/BAR/BAZ/SHOUT.cc");
 }
 
 TEST_F(InterpolateTest, Aliasing) {
   add("foo.cpp", "-faligned-new");
 
   // The interpolated command should keep the given flag as written, even though
   // the flag is internally represented as an alias.
   EXPECT_EQ(getCommand("foo.hpp"), "clang -D foo.cpp -faligned-new");