[clangd] Adjust compile flags so they work when applied to other file(type)s.

[sammccall]

Summary

It's reasonable to want to use the command from one file to compile another.
In particular, the command from a translation unit to parse a related header:

{"file": "foo.h", "command": "clang foo.cpp"}

This is largely what InterpolatingCompilationDatabase tries to do.
To do this correctly can require nontrivial changes to the argv, because the
file extension affects semantics. e.g. here we must add "-x c++header".

When external tools compile commands for different files, we should apply the
same adjustments. This is better than telling people to "fix their tools":

• simple e.g. python scripts shouldn't have to interpret clang argv
• this is a good way to represent the intent "parse header X in the context of file Y", which can work even if X is not self-contained. clangd does not support this today, but some other tools do, and we may one day.

This issue is discussed in https://github.com/clangd/clangd/issues/519

[kadircet]

thanks, lgtm!

clang-tools-extra/clangd/CompileCommands.cpp

287

maybe perform this before inserting -- Filename and do it once afterwards?

295

agreed it already inserts -- in some cases today, i think i was trying to be conservative at the time. i don't see any reason for not doing that all the time (possibly apart from test failures and tons of new places that don't expect -- in the flags).

clang-tools-extra/clangd/CompileCommands.cpp

for (auto *Input : ArgList.filtered(driver::options::OPT_arch)) {
     ++ArchOptCount;
     for (auto I = 0U; I <= Input->getNumValues(); ++I)
       IndicesToDrop.push_back(Input->getIndex() + I);
   }
   // If there is a single `-arch` option, keep it.
   if (ArchOptCount < 2)
     IndicesToDrop.clear();
 
+  // In some cases people may try to reuse the command from another file, e.g.
+  //   { File: "foo.h", CommandLine: "clang foo.cpp" }.
+  // We assume the intent is to parse foo.h the same way as foo.cpp, or as if
+  // it were being included from foo.cpp.
+  //
+  // We're going to rewrite the command to refer to foo.h, and this may change
+  // its semantics (e.g. by parsing the file as C). If we do this, we should
+  // use transferCompileCommand to adjust the argv.
+  // In practice only the extension of the file matters, so do this only when
+  // it differs.
+  llvm::StringRef FileExtension = llvm::sys::path::extension(File);
+  llvm::Optional<std::string> TransferFrom;
+  auto SawInput = [&](llvm::StringRef Input) {
+    if (llvm::sys::path::extension(Input) != FileExtension)
+      TransferFrom.emplace(Input);
+  };
+
   // Strip all the inputs and `--`. We'll put the input for the requested file
   // explicitly at the end of the flags. This ensures modifications done in the
   // following steps apply in more cases (like setting -x, which only affects
   // inputs that come after it).
-  for (auto *Input : ArgList.filtered(driver::options::OPT_INPUT))
+  for (auto *Input : ArgList.filtered(driver::options::OPT_INPUT)) {
+    SawInput(Input->getValue(0));
     IndicesToDrop.push_back(Input->getIndex());
+  }
   // Anything after `--` is also treated as input, drop them as well.
   if (auto *DashDash =
           ArgList.getLastArgNoClaim(driver::options::OPT__DASH_DASH)) {
-    Cmd.resize(DashDash->getIndex() + 1); // +1 to account for Cmd[0].
+    auto DashDashIndex = DashDash->getIndex() + 1; // +1 accounts for Cmd[0]
+    for (unsigned I = DashDashIndex; I < Cmd.size(); ++I)
+      SawInput(Cmd[I]);
+    Cmd.resize(DashDashIndex);
   }
   llvm::sort(IndicesToDrop);
   llvm::for_each(llvm::reverse(IndicesToDrop),
                  // +1 to account for the executable name in Cmd[0] that
                  // doesn't exist in ArgList.
                  [&Cmd](unsigned Idx) { Cmd.erase(Cmd.begin() + Idx + 1); });
   // All the inputs are stripped, append the name for the requested file. Rest
   // of the modifications should respect `--`.
   Cmd.push_back("--");
   Cmd.push_back(File.str());
 
+  if (TransferFrom) {
+    tooling::CompileCommand TransferCmd;
+    TransferCmd.Filename = std::move(*TransferFrom);
+    TransferCmd.CommandLine = std::move(Cmd);
+    TransferCmd = transferCompileCommand(std::move(TransferCmd), File);
+    Cmd = std::move(TransferCmd.CommandLine);
+
+    // Restore the canonical "driver --opts -- filename" form we expect.
+    // FIXME: This is ugly and coupled. Make transferCompileCommand ensure it?
+    assert(!Cmd.empty() && Cmd.back() == File);
+    Cmd.pop_back();
+    if (!Cmd.empty() && Cmd.back() == "--")
+      Cmd.pop_back();
+    assert(!llvm::is_contained(Cmd, "--"));
+    Cmd.push_back("--");
+    Cmd.push_back(File.str());
+  }
+
   for (auto &Edit : Config::current().CompileFlags.Edits)
     Edit(Cmd);
 
   // Check whether the flag exists, either as -flag or -flag=*
   auto Has = [&](llvm::StringRef Flag) {
     for (llvm::StringRef Arg : Cmd) {
       if (Arg.consume_front(Flag) && (Arg.empty() || Arg[0] == '='))
         return true;

clang-tools-extra/clangd/unittests/CompileCommandsTests.cpp

TEST(CommandMangler, Everything) {
   std::vector<std::string> Cmd = {"clang++", "--", "foo.cc", "bar.cc"};
   Mangler.adjust(Cmd, "foo.cc");
   EXPECT_THAT(Cmd, ElementsAre(testPath("fake/clang++"),
                                "-resource-dir=" + testPath("fake/resources"),
                                "-isysroot", testPath("fake/sysroot"), "--",
                                "foo.cc"));
 }
 
+TEST(CommandMangler, FilenameMismatch) {
+  auto Mangler = CommandMangler::forTests();
+  Mangler.ClangPath = testPath("clang");
+  // Our compile flags refer to foo.cc...
+  std::vector<std::string> Cmd = {"clang", "foo.cc"};
+  // but we're applying it to foo.h...
+  Mangler.adjust(Cmd, "foo.h");
+  // so transferCompileCommand should add -x c++-header to preserve semantics.
+  EXPECT_THAT(
+      Cmd, ElementsAre(testPath("clang"), "-x", "c++-header", "--", "foo.h"));
+}
+
 TEST(CommandMangler, ResourceDir) {
   auto Mangler = CommandMangler::forTests();
   Mangler.ResourceDir = testPath("fake/resources");
   std::vector<std::string> Cmd = {"clang++", "foo.cc"};
   Mangler.adjust(Cmd, "foo.cc");
   EXPECT_THAT(Cmd, Contains("-resource-dir=" + testPath("fake/resources")));
 }