[Tooling] Acknowledge that many CompilationDatabases don't support enumeration.

[sammccall]

Summary

Provide default implementations so that only getCompileCommands() is mandatory.

[grandinj]

include/clang/Tooling/CompilationDatabase.h

122

I know very little about LLVM's standards, so ignore me if I'm wrong, but shouldn't this be returning a pair of (begin,end) iterators rather than potentially a copy of a very large array of strings?

And shouldn't it be returning an iteration over StringRef rather then std::string, which will require copying the actual data?

133

similarly here

[hokein]

LGTM.

[sammccall]

include/clang/Tooling/CompilationDatabase.h

122

You might well be right. But this is an existing interface designed as an extension point for out-of-tree build systems. I'd rather not break them unless we have evidence of an actual performance problem.
(Note my change here is backwards compatible and doesn't change the signature)

include/clang/Tooling/CompilationDatabase.h

struct CompileCommand {
   std::vector<std::string> CommandLine;
 
   /// The output file associated with the command.
   std::string Output;
 };
 
 /// \brief Interface for compilation databases.
 ///
-/// A compilation database allows the user to retrieve all compile command lines
-/// that a specified file is compiled with in a project.
-/// The retrieved compile command lines can be used to run clang tools over
-/// a subset of the files in a project.
+/// A compilation database allows the user to retrieve compile command lines
+/// for the files in a project.
+///
+/// Many implementations are enumerable, allowing all command lines to be
+/// retrieved. These can be used to run clang tools over a subset of the files
+/// in a project.
 class CompilationDatabase {
 public:
   virtual ~CompilationDatabase();
 
   /// \brief Loads a compilation database from a build directory.
   ///
   /// Looks at the specified 'BuildDirectory' and creates a compilation database
   /// that allows to query compile commands for source files in the
public:
   /// $ clang++ -o test a.cc b.cc t.cc
   /// $ clang++ -o production a.cc b.cc -DPRODUCTION
   /// A compilation database representing the project would return both command
   /// lines for a.cc and b.cc and only the first command line for t.cc.
   virtual std::vector<CompileCommand> getCompileCommands(
     StringRef FilePath) const = 0;
 
   /// \brief Returns the list of all files available in the compilation database.
-  virtual std::vector<std::string> getAllFiles() const = 0;
+  ///
+  /// By default, returns nothing. Implementations should override this if they
+  /// can enumerate their source files.
+  virtual std::vector<std::string> getAllFiles() const { return {}; }
 
   /// \brief Returns all compile commands for all the files in the compilation
   /// database.
   ///
   /// FIXME: Add a layer in Tooling that provides an interface to run a tool
   /// over all files in a compilation database. Not all build systems have the
   /// ability to provide a feasible implementation for \c getAllCompileCommands.
-  virtual std::vector<CompileCommand> getAllCompileCommands() const = 0;
+  ///
+  /// By default, this is implemented in terms of getAllFiles() and
+  /// getCompileCommands(). Subclasses may override this for efficiency.
+  virtual std::vector<CompileCommand> getAllCompileCommands() const;
 };
 
 /// \brief Interface for compilation database plugins.
 ///
 /// A compilation database plugin allows the user to register custom compilation
 /// databases that are picked up as compilation database if the corresponding
 /// library is linked in. To register a plugin, declare a static variable like:
 ///
public:
   /// \see CompilationDatabase::loadFromDirectory().
   virtual std::unique_ptr<CompilationDatabase>
   loadFromDirectory(StringRef Directory, std::string &ErrorMessage) = 0;
 };
 
 /// \brief A compilation database that returns a single compile command line.
 ///
 /// Useful when we want a tool to behave more like a compiler invocation.
+/// This compilation database is not enumerable: getAllFiles() returns {}.
 class FixedCompilationDatabase : public CompilationDatabase {
 public:
   /// \brief Creates a FixedCompilationDatabase from the arguments after "--".
   ///
   /// Parses the given command line for "--". If "--" is found, the rest of
   /// the arguments will make up the command line in the returned
   /// FixedCompilationDatabase.
   /// The arguments after "--" must not include positional parameters or the
public:
   /// \brief Returns the given compile command.
   ///
   /// Will always return a vector with one entry that contains the directory
   /// and command line specified at construction with "clang-tool" as argv[0]
   /// and 'FilePath' as positional argument.
   std::vector<CompileCommand>
   getCompileCommands(StringRef FilePath) const override;
 
-  /// \brief Returns the list of all files available in the compilation database.
-  ///
-  /// Note: This is always an empty list for the fixed compilation database.
-  std::vector<std::string> getAllFiles() const override;
-
-  /// \brief Returns all compile commands for all the files in the compilation
-  /// database.
-  ///
-  /// Note: This is always an empty list for the fixed compilation database.
-  std::vector<CompileCommand> getAllCompileCommands() const override;
-
 private:
   /// This is built up to contain a single entry vector to be returned from
   /// getCompileCommands after adding the positional argument.
   std::vector<CompileCommand> CompileCommands;
 };
 
 } // end namespace tooling
 } // end namespace clang
 
 #endif

lib/Tooling/CompilationDatabase.cpp

std::unique_ptr<CompilationDatabase> DB =
       findCompilationDatabaseFromDirectory(AbsolutePath, ErrorMessage);
 
   if (!DB)
     ErrorMessage = ("Could not auto-detect compilation database from directory \"" +
                    SourceDir + "\"\n" + ErrorMessage).str();
   return DB;
 }
 
+std::vector<CompileCommand> CompilationDatabase::getAllCompileCommands() const {
+  std::vector<CompileCommand> Result;
+  for (const auto &File : getAllFiles()) {
+    auto C = getCompileCommands(File);
+    std::move(C.begin(), C.end(), std::back_inserter(Result));
+  }
+  return Result;
+}
+
 CompilationDatabasePlugin::~CompilationDatabasePlugin() {}
 
 namespace {
 // Helper for recursively searching through a chain of actions and collecting
 // all inputs, direct and indirect, of compile jobs.
 struct CompileJobAnalyzer {
   void run(const driver::Action *A) {
     runImpl(A, false);

 std::vector<CompileCommand>
 FixedCompilationDatabase::getCompileCommands(StringRef FilePath) const {
   std::vector<CompileCommand> Result(CompileCommands);
   Result[0].CommandLine.push_back(FilePath);
   Result[0].Filename = FilePath;
   return Result;
 }
 
-std::vector<std::string>
-FixedCompilationDatabase::getAllFiles() const {
-  return std::vector<std::string>();
-}
-
-std::vector<CompileCommand>
-FixedCompilationDatabase::getAllCompileCommands() const {
-  return std::vector<CompileCommand>();
-}
-
 namespace {
 
 class FixedCompilationDatabasePlugin : public CompilationDatabasePlugin {
   std::unique_ptr<CompilationDatabase>
   loadFromDirectory(StringRef Directory, std::string &ErrorMessage) override {
     SmallString<1024> DatabasePath(Directory);
     llvm::sys::path::append(DatabasePath, "compile_flags.txt");
     return FixedCompilationDatabase::loadFromFile(DatabasePath, ErrorMessage);