[LibTooling] Allow compilation database to explicitly specify compilation database file and source root

[zturner]

Summary

Allow explicit specification of a compilation database file and source root.

While trying to create a compilation database test for D23455, I ran into the problem that compilation database tests require a command shell, evidenced by the fact that at the top of clang-tidy-run-with-database-cpp contains the line REQUIRES: SHELL. This makes it impossible to write the test that was requested in D23455 (not to mention it means the existing test is not running on Windows either).

I solve this by introducing a new command line parameter and slightly enhancing the functionality of an existing one, detailed below:

1. There was no way to specify an arbitrary compilation database. At best you could specify the directory of a compilation database, but it would still be forced to look for compile_commands.json. This is an inflexible design when you want to have multiple tests which each use their own compilation database. You could put them in different folders and call each one compile_commands.json, but that leads to many different directories with only one file, which is kind of silly.

The -p option let you specify a build path, which would be the folder where a compilation database existed, but it did not let you specify a *specific file* to use as a compilation database. I improved expanded this behavior of this option to check if the specified path is a regular file or a directory. If it is a regular file it doesn't search for a compilation database, and instead just uses the file you specify. The nice thing about this is that now we can put many compilation databases for various tests in the same directory, and have the run line of the test reference the exact compilation database needed for the test.

2. Instead of a "real" compilation database, the test consisted of a template. And the test used shell in order to invoke sed to modify the template so that the files and directories would be correct. So the test would output the modified template into the CMake output directory, and then it had to use more shell commands to copy the source files over to the CMake output directory so that they would be in the right hierarchy in order to be found by the relative paths written to the compilation database.

In order to remove the dependency on shell, we need a way to reference the source files in their original location in the source tree, and not require them to be copied to the output tree. To do this I introduced a new command line option -r that lets you specify the root at which relative paths are resolved against.

These two things combined eliminate the need to modify the compilation database with sed and write it to the output folder because we can write *actual* source code on disk and check it in for the tests, and the compilation database can refer to these files using actual relative paths. Then because of #1, the run line can point directly to the compilation database in question.

I re-wrote the compilation database test to use this new method, and it now no longer depends on shell. I will upload the re-written test in a separate patch since I can't do cross-repo diffs.

include/clang/Tooling/CompilationDatabase.h

public:
   ///
   /// Returns NULL and sets ErrorMessage if we were not able to build up a
   /// compilation database for the build directory.
   ///
   /// FIXME: Currently only supports JSON compilation databases, which
   /// are named 'compile_commands.json' in the given directory. Extend this
   /// for other build types (like ninja build files).
   static std::unique_ptr<CompilationDatabase>
-  loadFromDirectory(StringRef BuildDirectory, std::string &ErrorMessage);
+  loadFromDirectory(StringRef BuildDirectory, StringRef SourceRoot,
+                    std::string &ErrorMessage);
+
+  /// \brief Attempts to load a compilation database from the specified file.
+  ///
+  /// Returns NULL and sets ErrorMessage if the specified compilation database
+  /// does not exist or is in an unrecognized format.  Relative paths will
+  /// be resolved starting from 'SourceRoot'.  If 'SourceRoot' is empty
+  /// they will be resolved from the current working directory.
+  static std::unique_ptr<CompilationDatabase>
+  loadFromFile(StringRef File, StringRef SourceRoot, std::string &ErrorMessage);
 
   /// \brief Tries to detect a compilation database location and load it.
   ///
   /// Looks for a compilation database in all parent paths of file 'SourceFile'
   /// by calling loadFromDirectory.
   static std::unique_ptr<CompilationDatabase>
   autoDetectFromSource(StringRef SourceFile, std::string &ErrorMessage);
 
   /// \brief Tries to detect a compilation database location and load it.
   ///
-  /// Looks for a compilation database in directory 'SourceDir' and all
-  /// its parent paths by calling loadFromDirectory.
+  /// Looks for a compilation database in directory 'DatabaseDir' and all
+  /// its parent paths by calling loadFromDirectory.  Relative paths will
+  /// be resolved starting from 'SourceRoot'.  If 'SourceRoot' is empty
+  /// they will be resolved from the current working directory.
   static std::unique_ptr<CompilationDatabase>
-  autoDetectFromDirectory(StringRef SourceDir, std::string &ErrorMessage);
+  autoDetectFromDirectory(StringRef DatabaseDir, StringRef SourceRoot,
+                          std::string &ErrorMessage);
 
   /// \brief Returns all compile commands in which the specified file was
   /// compiled.
   ///
   /// This includes compile comamnds that span multiple source files.
   /// For example, consider a project with the following compilations:
   /// $ clang++ -o test a.cc b.cc t.cc
   /// $ clang++ -o production a.cc b.cc -DPRODUCTION

 class CompilationDatabasePlugin {
 public:
   virtual ~CompilationDatabasePlugin();
 
   /// \brief Loads a compilation database from a build directory.
   ///
   /// \see CompilationDatabase::loadFromDirectory().
   virtual std::unique_ptr<CompilationDatabase>
-  loadFromDirectory(StringRef Directory, std::string &ErrorMessage) = 0;
+  loadFromDirectory(StringRef Directory, StringRef SourceRoot,
+                    std::string &ErrorMessage) = 0;
+
+  /// \brief Loads a compilation database from the specified file.
+  ///
+  /// \see CompilationDatabase::loadFromFile().
+  virtual std::unique_ptr<CompilationDatabase>
+  loadFromFile(StringRef File, StringRef SourceRoot,
+               std::string &ErrorMessage) = 0;
 };
 
 /// \brief A compilation database that returns a single compile command line.
 ///
 /// Useful when we want a tool to behave more like a compiler invocation.
 class FixedCompilationDatabase : public CompilationDatabase {
 public:
   /// \brief Creates a FixedCompilationDatabase from the arguments after "--".

include/clang/Tooling/JSONCompilationDatabase.h

 /// by setting the flag -DCMAKE_EXPORT_COMPILE_COMMANDS.
 class JSONCompilationDatabase : public CompilationDatabase {
 public:
   /// \brief Loads a JSON compilation database from the specified file.
   ///
   /// Returns NULL and sets ErrorMessage if the database could not be
   /// loaded from the given file.
   static std::unique_ptr<JSONCompilationDatabase>
-  loadFromFile(StringRef FilePath, std::string &ErrorMessage);
+  loadFromFile(StringRef FilePath, StringRef SourceRoot,
+               std::string &ErrorMessage);
 
   /// \brief Loads a JSON compilation database from a data buffer.
   ///
   /// Returns NULL and sets ErrorMessage if the database could not be loaded.
   static std::unique_ptr<JSONCompilationDatabase>
   loadFromBuffer(StringRef DatabaseString, std::string &ErrorMessage);
 
   /// \brief Returns all compile comamnds in which the specified file was
public:
   std::vector<std::string> getAllFiles() const override;
 
   /// \brief Returns all compile commands for all the files in the compilation
   /// database.
   std::vector<CompileCommand> getAllCompileCommands() const override;
 
 private:
   /// \brief Constructs a JSON compilation database on a memory buffer.
-  JSONCompilationDatabase(std::unique_ptr<llvm::MemoryBuffer> Database)
-      : Database(std::move(Database)),
+  /// If a source root is given, it is used to resolve relative paths.
+  JSONCompilationDatabase(StringRef SourceRoot,
+                          std::unique_ptr<llvm::MemoryBuffer> Database)
+      : Database(std::move(Database)), SourceRoot(SourceRoot),
         YAMLStream(this->Database->getBuffer(), SM) {}
 
   /// \brief Parses the database file and creates the index.
   ///
   /// Returns whether parsing succeeded. Sets ErrorMessage if parsing
   /// failed.
   bool parse(std::string &ErrorMessage);
 
   // Tuple (directory, filename, commandline) where 'commandline' points to the
   // corresponding scalar nodes in the YAML stream.
   // If the command line contains a single argument, it is a shell-escaped
   // command line.
   // Otherwise, each entry in the command line vector is a literal
   // argument to the compiler.
   typedef std::tuple<llvm::yaml::ScalarNode *,
                      llvm::yaml::ScalarNode *,
                     std::vector<llvm::yaml::ScalarNode *>> CompileCommandRef;
 
   /// \brief Converts the given array of CompileCommandRefs to CompileCommands.
   void getCommands(ArrayRef<CompileCommandRef> CommandsRef,
                    std::vector<CompileCommand> &Commands) const;
 
+  void getDirectoryForCommand(llvm::yaml::ScalarNode &JsonDir,
+                              llvm::SmallVectorImpl<char> &Dir) const;
+
   // Maps file paths to the compile command lines for that file.
   llvm::StringMap<std::vector<CompileCommandRef>> IndexByFile;
 
   /// All the compile commands in the order that they were provided in the
   /// JSON stream.
   std::vector<CompileCommandRef> AllCommands;
 
   FileMatchTrie MatchTrie;
 
   std::unique_ptr<llvm::MemoryBuffer> Database;
+  StringRef SourceRoot;
   llvm::SourceMgr SM;
   llvm::yaml::Stream YAMLStream;
 };
 
 } // end namespace tooling
 } // end namespace clang
 
 #endif

lib/Tooling/CommonOptionsParser.cpp

 
 using namespace clang::tooling;
 using namespace llvm;
 
 const char *const CommonOptionsParser::HelpMessage =
     "\n"
     "-p <build-path> is used to read a compile command database.\n"
     "\n"
-    "\tFor example, it can be a CMake build directory in which a file named\n"
-    "\tcompile_commands.json exists (use -DCMAKE_EXPORT_COMPILE_COMMANDS=ON\n"
-    "\tCMake option to get this output). When no build path is specified,\n"
+    "\t<build-path> can be either a file or a directory.  If it is a file,\n"
+    "\tit will be treated as a JSON compile command database.  If it is a\n"
+    "\tdirectory, the directory will be searched for a file named\n"
+    "\tcompile_commands.json (CMake, for example, generates such a file in\n"
+    "\tthe build output directory when used with the special option\n"
+    "\t-DCMAKE_EXPORT_COMPILE_COMMANDS=ON.  When this option is not present,\n"
     "\ta search for compile_commands.json will be attempted through all\n"
     "\tparent paths of the first input file . See:\n"
     "\thttp://clang.llvm.org/docs/HowToSetupToolingForLLVM.html for an\n"
     "\texample of setting up Clang Tooling on a source tree.\n"
     "\n"
+    "-r <source-root> is used to set the base folder from which to resolve\n"
+    "\trelative paths in the compile command database.  By default relative\n"
+    "\tpaths are expected to be found relative to the current working\n"
+    "\tdirectory, as described below."
     "<source0> ... specify the paths of source files. These paths are\n"
     "\tlooked up in the compile command database. If the path of a file is\n"
     "\tabsolute, it needs to point into CMake's source tree. If the path is\n"
     "\trelative, the current working directory needs to be in the CMake\n"
     "\tsource tree and the file must be in a subdirectory of the current\n"
     "\tworking directory. \"./\" prefixes in the relative files will be\n"
     "\tautomatically removed, but the rest of a relative path must be a\n"
     "\tsuffix of a path in the compile command database.\n"

 
 CommonOptionsParser::CommonOptionsParser(
     int &argc, const char **argv, cl::OptionCategory &Category,
     llvm::cl::NumOccurrencesFlag OccurrencesFlag, const char *Overview) {
   static cl::opt<bool> Help("h", cl::desc("Alias for -help"), cl::Hidden);
 
   static cl::opt<std::string> BuildPath("p", cl::desc("Build path"),
                                         cl::Optional, cl::cat(Category));
-
+  static cl::opt<std::string> SourceRoot("r", cl::desc("Source Root"),
+                                         cl::Optional, cl::cat(Category));
   static cl::list<std::string> SourcePaths(
       cl::Positional, cl::desc("<source0> [... <sourceN>]"), OccurrencesFlag,
       cl::cat(Category));
 
   static cl::list<std::string> ArgsAfter(
       "extra-arg",
       cl::desc("Additional argument to append to the compiler command line"),
       cl::cat(Category));
CommonOptionsParser::CommonOptionsParser(
 
   SourcePathList = SourcePaths;
   if ((OccurrencesFlag == cl::ZeroOrMore || OccurrencesFlag == cl::Optional) &&
       SourcePathList.empty())
     return;
   if (!Compilations) {
     std::string ErrorMessage;
     if (!BuildPath.empty()) {
-      Compilations =
-          CompilationDatabase::autoDetectFromDirectory(BuildPath, ErrorMessage);
+      if (llvm::sys::fs::is_regular_file(BuildPath)) {
+        Compilations = CompilationDatabase::loadFromFile(BuildPath, SourceRoot,
+                                                         ErrorMessage);
+        if (!Compilations)
+          ErrorMessage = "Could not load compilation database \"" + BuildPath +
+                         "\"\n" + ErrorMessage;
+      } else {
+        Compilations = CompilationDatabase::autoDetectFromDirectory(
+            BuildPath, SourceRoot, ErrorMessage);
+      }
     } else {
       Compilations = CompilationDatabase::autoDetectFromSource(SourcePaths[0],
                                                                ErrorMessage);
     }
     if (!Compilations) {
       llvm::errs() << "Error while trying to load a compilation database:\n"
                    << ErrorMessage << "Running without flags.\n";
       Compilations.reset(

lib/Tooling/CompilationDatabase.cpp

 #include <system_error>
 using namespace clang;
 using namespace tooling;
 
 LLVM_INSTANTIATE_REGISTRY(CompilationDatabasePluginRegistry)
 
 CompilationDatabase::~CompilationDatabase() {}
 
+std::unique_ptr<CompilationDatabase> CompilationDatabase::loadFromDirectory(
+    StringRef BuildDirectory, StringRef SourceRoot, std::string &ErrorMessage) {
+  std::stringstream ErrorStream;
+  for (CompilationDatabasePluginRegistry::iterator
+       It = CompilationDatabasePluginRegistry::begin(),
+       Ie = CompilationDatabasePluginRegistry::end();
+       It != Ie; ++It) {
+    std::string DatabaseErrorMessage;
+    std::unique_ptr<CompilationDatabasePlugin> Plugin(It->instantiate());
+    if (std::unique_ptr<CompilationDatabase> DB = Plugin->loadFromDirectory(
+            BuildDirectory, SourceRoot, DatabaseErrorMessage))
+      return DB;
+    ErrorStream << It->getName() << ": " << DatabaseErrorMessage << "\n";
+  }
+  ErrorMessage = ErrorStream.str();
+  return nullptr;
+}
+
 std::unique_ptr<CompilationDatabase>
-CompilationDatabase::loadFromDirectory(StringRef BuildDirectory,
-                                       std::string &ErrorMessage) {
+CompilationDatabase::loadFromFile(StringRef File, StringRef SourceRoot,
   std::stringstream ErrorStream;
   for (CompilationDatabasePluginRegistry::iterator
            It = CompilationDatabasePluginRegistry::begin(),
            Ie = CompilationDatabasePluginRegistry::end();
        It != Ie; ++It) {
     std::string DatabaseErrorMessage;
     std::unique_ptr<CompilationDatabasePlugin> Plugin(It->instantiate());
     if (std::unique_ptr<CompilationDatabase> DB =
-            Plugin->loadFromDirectory(BuildDirectory, DatabaseErrorMessage))
+            Plugin->loadFromFile(File, SourceRoot, DatabaseErrorMessage))
       return DB;
     ErrorStream << It->getName() << ": " << DatabaseErrorMessage << "\n";
   }
   ErrorMessage = ErrorStream.str();
   return nullptr;
 }
 
 static std::unique_ptr<CompilationDatabase>
-findCompilationDatabaseFromDirectory(StringRef Directory,
+findCompilationDatabaseFromDirectory(StringRef Directory, StringRef SourceRoot,
                                      std::string &ErrorMessage) {
   std::stringstream ErrorStream;
   bool HasErrorMessage = false;
   while (!Directory.empty()) {
     std::string LoadErrorMessage;
 
     if (std::unique_ptr<CompilationDatabase> DB =
-            CompilationDatabase::loadFromDirectory(Directory, LoadErrorMessage))
+            CompilationDatabase::loadFromDirectory(Directory, SourceRoot,
+                                                   LoadErrorMessage))
       return DB;
 
     if (!HasErrorMessage) {
       ErrorStream << "No compilation database found in " << Directory.str()
                   << " or any parent directory\n" << LoadErrorMessage;
       HasErrorMessage = true;
     }
 
     Directory = llvm::sys::path::parent_path(Directory);
   }
   ErrorMessage = ErrorStream.str();
   return nullptr;
 }
 
 std::unique_ptr<CompilationDatabase>
 CompilationDatabase::autoDetectFromSource(StringRef SourceFile,
                                           std::string &ErrorMessage) {
   SmallString<1024> AbsolutePath(getAbsolutePath(SourceFile));
   StringRef Directory = llvm::sys::path::parent_path(AbsolutePath);
 
   std::unique_ptr<CompilationDatabase> DB =
-      findCompilationDatabaseFromDirectory(Directory, ErrorMessage);
+      findCompilationDatabaseFromDirectory(Directory, "", ErrorMessage);
 
   if (!DB)
     ErrorMessage = ("Could not auto-detect compilation database for file \"" +
-                   SourceFile + "\"\n" + ErrorMessage).str();
+                    SourceFile + "\"\n" + ErrorMessage)
+                       .str();
   return DB;
 }
 
 std::unique_ptr<CompilationDatabase>
 CompilationDatabase::autoDetectFromDirectory(StringRef SourceDir,
+                                             StringRef SourceRoot,
                                              std::string &ErrorMessage) {
   SmallString<1024> AbsolutePath(getAbsolutePath(SourceDir));
 
   std::unique_ptr<CompilationDatabase> DB =
-      findCompilationDatabaseFromDirectory(AbsolutePath, ErrorMessage);
+      findCompilationDatabaseFromDirectory(AbsolutePath, SourceRoot,
+                                           ErrorMessage);
 
   if (!DB)
     ErrorMessage = ("Could not auto-detect compilation database from directory \"" +
                    SourceDir + "\"\n" + ErrorMessage).str();
   return DB;
 }
 
 CompilationDatabasePlugin::~CompilationDatabasePlugin() {}

lib/Tooling/JSONCompilationDatabase.cpp

 //
 //===----------------------------------------------------------------------===//
 
 #include "clang/Tooling/JSONCompilationDatabase.h"
 #include "clang/Tooling/CompilationDatabase.h"
 #include "clang/Tooling/CompilationDatabasePluginRegistry.h"
 #include "clang/Tooling/Tooling.h"
 #include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/Triple.h"
 #include "llvm/Support/Allocator.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/StringSaver.h"
+#include <direct.h>
 #include <system_error>
 
 namespace clang {
 namespace tooling {
 
 namespace {
 
 /// \brief A parser for escaped strings of command line arguments.
private:
 
   const StringRef Input;
   StringRef::iterator Position;
   std::vector<std::string> CommandLine;
 };
 
 std::vector<std::string> unescapeCommandLine(
     StringRef EscapedCommandLine) {
-#if defined(LLVM_ON_WIN32)
+  llvm::Triple Triple(llvm::sys::getDefaultTargetTriple());
+  if (Triple.getOS() == llvm::Triple::OSType::Win32) {
     llvm::BumpPtrAllocator Alloc;
     llvm::StringSaver Saver(Alloc);
     llvm::SmallVector<const char *, 64> T;
     llvm::cl::TokenizeWindowsCommandLine(EscapedCommandLine, Saver, T);
     std::vector<std::string> Result(T.begin(), T.end());
     return Result;
-#else
+  } else {
     CommandLineArgumentParser parser(EscapedCommandLine);
     return parser.parse();
-#endif
+  }
 }
 
 class JSONCompilationDatabasePlugin : public CompilationDatabasePlugin {
   std::unique_ptr<CompilationDatabase>
-  loadFromDirectory(StringRef Directory, std::string &ErrorMessage) override {
+  loadFromDirectory(StringRef Directory, StringRef SourceRoot,
+                    std::string &ErrorMessage) override {
     SmallString<1024> JSONDatabasePath(Directory);
     llvm::sys::path::append(JSONDatabasePath, "compile_commands.json");
+    return loadFromFile(JSONDatabasePath, SourceRoot, ErrorMessage);
+  }
+
+  std::unique_ptr<CompilationDatabase>
+  loadFromFile(StringRef File, StringRef SourceRoot,
+               std::string &ErrorMessage) override {
     std::unique_ptr<CompilationDatabase> Database(
-        JSONCompilationDatabase::loadFromFile(JSONDatabasePath, ErrorMessage));
+        JSONCompilationDatabase::loadFromFile(File, SourceRoot, ErrorMessage));
     if (!Database)
       return nullptr;
     return Database;
   }
 };
 
 } // end namespace
 
 // Register the JSONCompilationDatabasePlugin with the
 // CompilationDatabasePluginRegistry using this statically initialized variable.
 static CompilationDatabasePluginRegistry::Add<JSONCompilationDatabasePlugin>
 X("json-compilation-database", "Reads JSON formatted compilation databases");
 
 // This anchor is used to force the linker to link in the generated object file
 // and thus register the JSONCompilationDatabasePlugin.
 volatile int JSONAnchorSource = 0;
 
 std::unique_ptr<JSONCompilationDatabase>
-JSONCompilationDatabase::loadFromFile(StringRef FilePath,
+JSONCompilationDatabase::loadFromFile(StringRef FilePath, StringRef SourceRoot,
                                       std::string &ErrorMessage) {
   llvm::ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> DatabaseBuffer =
       llvm::MemoryBuffer::getFile(FilePath);
   if (std::error_code Result = DatabaseBuffer.getError()) {
     ErrorMessage = "Error while opening JSON database: " + Result.message();
     return nullptr;
   }
   std::unique_ptr<JSONCompilationDatabase> Database(
-      new JSONCompilationDatabase(std::move(*DatabaseBuffer)));
+      new JSONCompilationDatabase(SourceRoot, std::move(*DatabaseBuffer)));
   if (!Database->parse(ErrorMessage))
     return nullptr;
   return Database;
 }
 
 std::unique_ptr<JSONCompilationDatabase>
 JSONCompilationDatabase::loadFromBuffer(StringRef DatabaseString,
                                         std::string &ErrorMessage) {
   std::unique_ptr<llvm::MemoryBuffer> DatabaseBuffer(
       llvm::MemoryBuffer::getMemBuffer(DatabaseString));
   std::unique_ptr<JSONCompilationDatabase> Database(
-      new JSONCompilationDatabase(std::move(DatabaseBuffer)));
+      new JSONCompilationDatabase("", std::move(DatabaseBuffer)));
   if (!Database->parse(ErrorMessage))
     return nullptr;
   return Database;
 }
 
 std::vector<CompileCommand>
 JSONCompilationDatabase::getCompileCommands(StringRef FilePath) const {
   SmallString<128> NativeFilePath;
nodeToCommandLine(const std::vector<llvm::yaml::ScalarNode *> &Nodes) {
   }
   return Arguments;
 }
 
 void JSONCompilationDatabase::getCommands(
     ArrayRef<CompileCommandRef> CommandsRef,
     std::vector<CompileCommand> &Commands) const {
   for (int I = 0, E = CommandsRef.size(); I != E; ++I) {
-    SmallString<8> DirectoryStorage;
+    SmallString<128> DirectoryStorage;
     SmallString<32> FilenameStorage;
+    getDirectoryForCommand(*std::get<0>(CommandsRef[I]), DirectoryStorage);
     Commands.emplace_back(
-      std::get<0>(CommandsRef[I])->getValue(DirectoryStorage),
+        DirectoryStorage,
         std::get<1>(CommandsRef[I])->getValue(FilenameStorage),
         nodeToCommandLine(std::get<2>(CommandsRef[I])));
   }
 }
 
 bool JSONCompilationDatabase::parse(std::string &ErrorMessage) {
   llvm::yaml::document_iterator I = YAMLStream.begin();
   if (I == YAMLStream.end()) {
     ErrorMessage = "Error while parsing YAML.";
     return false;
for (auto& NextObject : *Array) {
     if (!Directory) {
       ErrorMessage = "Missing key: \"directory\".";
       return false;
     }
     SmallString<8> FileStorage;
     StringRef FileName = File->getValue(FileStorage);
     SmallString<128> NativeFilePath;
     if (llvm::sys::path::is_relative(FileName)) {
-      SmallString<8> DirectoryStorage;
-      SmallString<128> AbsolutePath(
-          Directory->getValue(DirectoryStorage));
-      llvm::sys::path::append(AbsolutePath, FileName);
-      llvm::sys::path::native(AbsolutePath, NativeFilePath);
-    } else {
-      llvm::sys::path::native(FileName, NativeFilePath);
+      SmallString<128> TempStorage(Directory->getValue(NativeFilePath));
+      getDirectoryForCommand(*Directory, TempStorage);
+      llvm::sys::path::append(TempStorage, FileName);
+      FileStorage.swap(TempStorage);
     }
+
+    llvm::sys::path::native(FileStorage, NativeFilePath);
     auto Cmd = CompileCommandRef(Directory, File, *Command);
     IndexByFile[NativeFilePath].push_back(Cmd);
     AllCommands.push_back(Cmd);
     MatchTrie.insert(NativeFilePath);
   }
   return true;
 }
 
+void JSONCompilationDatabase::getDirectoryForCommand(
+    llvm::yaml::ScalarNode &JsonDir, llvm::SmallVectorImpl<char> &Dir) const {
+  SmallString<128> DirectoryStorage;
+  Dir.clear();
+  StringRef JsonVal = JsonDir.getValue(DirectoryStorage);
+  Dir.append(JsonVal.begin(), JsonVal.end());
+  if (llvm::sys::path::is_relative(Dir)) {
+    SmallString<128> TempStorage(SourceRoot);
+    llvm::sys::path::append(TempStorage, Dir);
+    Dir.swap(TempStorage);
+  }
+  llvm::sys::fs::make_absolute(Dir);
+  llvm::sys::path::native(Dir);
+}
+
 } // end namespace tooling
 } // end namespace clang

lib/Tooling/Tooling.cpp

for (const auto &SourcePath : SourcePaths) {
     for (CompileCommand &CompileCommand : CompileCommandsForFile) {
       // FIXME: chdir is thread hostile; on the other hand, creating the same
       // behavior as chdir is complex: chdir resolves the path once, thus
       // guaranteeing that all subsequent relative path operations work
       // on the same path the original chdir resulted in. This makes a
       // difference for example on network filesystems, where symlinks might be
       // switched during runtime of the tool. Fixing this depends on having a
       // file system abstraction that allows openat() style interactions.
+      if (!CompileCommand.Directory.empty()) {
         if (OverlayFileSystem->setCurrentWorkingDirectory(
                 CompileCommand.Directory))
           llvm::report_fatal_error("Cannot chdir into \"" +
                                    Twine(CompileCommand.Directory) + "\n!");
+      }
 
       // Now fill the in-memory VFS with the relative file mappings so it will
       // have the correct relative paths. We never remove mappings but that
       // should be fine.
       if (SeenWorkingDirectories.insert(CompileCommand.Directory).second)
         for (const auto &MappedFile : MappedFileContents)
           if (!llvm::sys::path::is_absolute(MappedFile.first))
             InMemoryFileSystem->addFile(