Add index-while-building support to Clang - Part 3

[jkorous]

Summary

See Part 1 (https://reviews.llvm.org/D39050) for an overview of the new functionality.

Part 1 added a new -index-store-path option that added a new FrontendAction – WrappingIndexRecordAction – to collect indexing information during compilation. Part 2 contained the changes to write the collected information out to the directory supplied by the -index-store-path option in the LLVM Bitstream format, as a collection of unit and record files.
This part adds a new IndexStore library to help clients read and manage the indexing data in index store directory. It abstracts away the directory layout and file format of the store and, combined with the new DirectoryWatcher, also notifies clients whenever a unit file is updated.

[nathawes]

This needs to be reworked to remove its dependence on blocks.

[nathawes]

Updated to apply on the updated first and second patches + formatting fixes.

[yvvan]

I can't really test this one because INDEXSTORE_HAS_BLOCKS is apple-specific and it's required to rewrite all block-related stuff to use that code

[yvvan]

Will it work if I replace every block-related code here with llvm::function_ref<>?

[nathawes]

@yvvan I'm not sure, sorry. I plan on removing/adding alternatives for the Apple specific bits and opening these up for review over the next two weeks if you're ok to wait a little longer.

[yvvan]

@nathawes I've already started changing so I can hopefully share my code with you when I fix all undefined symbols :)

[nathawes]

That would be great @yvvan – thanks!

[yvvan]

So I have that working on windows with clang-cl (probably broke something else during the porting, need to recheck.
One more commit I have in my working tree is YAML export to be able to use it for clangd. The quality of code is currently not so good so I probably need some extra time to make it ok-ish. But i hope to have it posted to reviews this week (unless I have something urgent)

[jkorous]

It's time to officially abandon these patches in favor of new push for upstreaming index-while-building.

Current reviews in progress
https://reviews.llvm.org/D58749
https://reviews.llvm.org/D58418

RFC
http://lists.llvm.org/pipermail/cfe-dev/2019-February/061432.html

[jkorous]

I'll address comments for this patch in the new set of patches and I'll add original reviewers.

include/clang/DirectoryWatcher/DirectoryWatcher.h

+//===- DirectoryWatcher.h - Listens for directory file changes --*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+/// \file
+/// \brief Utility class for listening for file system changes in a directory.
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_DIRECTORYWATCHER_DIRECTORYWATCHER_H
+#define LLVM_CLANG_DIRECTORYWATCHER_DIRECTORYWATCHER_H
+
+#include "clang/Basic/LLVM.h"
+#include "clang/Index/IndexDataStore.h"
+#include <functional>
+#include <memory>
+#include <string>
+
+namespace clang {
+
+/// Provides notifications for file system changes in a directory.
+///
+/// Guarantees that the first time the directory is processed, the receiver will
+/// be invoked even if the directory is empty.
+class DirectoryWatcher : public index::AbstractDirectoryWatcher {
+  struct Implementation;
+  Implementation &Impl;
+
+  DirectoryWatcher();
+
+  DirectoryWatcher(const DirectoryWatcher &) = delete;
+  DirectoryWatcher &operator=(const DirectoryWatcher &) = delete;
+
+public:
+  ~DirectoryWatcher();
+
+  static std::unique_ptr<DirectoryWatcher> create(StringRef Path,
+                                                  EventReceiver Receiver,
+                                                  bool waitInitialSync,
+                                                  std::string &Error);
+};
+
+} // namespace clang
+
+#endif

include/clang/Index/IndexDataStore.h

+//===--- IndexDataStore.h - Index data store info -------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_INDEX_INDEXDATASTORE_H
+#define LLVM_CLANG_INDEX_INDEXDATASTORE_H
+
+#include "clang/Basic/LLVM.h"
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/StringRef.h"
+#include <functional>
+#include <memory>
+#include <string>
+#include <vector>
+
+namespace clang {
+namespace index {
+
+class AbstractDirectoryWatcher {
+public:
+  enum class EventKind {
+    /// A file was added.
+    Added,
+    /// A file was removed.
+    Removed,
+    /// A file was modified.
+    Modified,
+    /// The watched directory got deleted. No more events will follow.
+    DirectoryDeleted,
+  };
+
+  struct Event {
+    EventKind Kind;
+    std::string Filename;
+    timespec ModTime;
+  };
+
+  typedef std::function<void(ArrayRef<Event> Events, bool isInitial)>
+      EventReceiver;
+  typedef std::unique_ptr<AbstractDirectoryWatcher>(CreateFnTy)(
+      StringRef Path, EventReceiver Receiver, bool waitInitialSync,
+      std::string &Error);
+
+  virtual ~AbstractDirectoryWatcher() {}
+};
+
+class IndexDataStore {
+public:
+  ~IndexDataStore();
+
+  static std::unique_ptr<IndexDataStore> create(StringRef IndexStorePath,
+                                                std::string &Error);
+
+  StringRef getFilePath() const;
+  bool foreachUnitName(bool sorted,
+                       llvm::function_ref<bool(StringRef unitName)> receiver);
+
+  static unsigned getFormatVersion();
+
+  enum class UnitEventKind {
+    Added,
+    Removed,
+    Modified,
+    /// The directory got deleted. No more events will follow.
+    DirectoryDeleted,
+  };
+  struct UnitEvent {
+    UnitEventKind Kind;
+    StringRef UnitName;
+    timespec ModTime;
+  };
+  struct UnitEventNotification {
+    bool IsInitial;
+    ArrayRef<UnitEvent> Events;
+  };
+  typedef std::function<void(UnitEventNotification)> UnitEventHandler;
+
+  void setUnitEventHandler(UnitEventHandler Handler);
+  /// \returns true if an error occurred.
+  bool startEventListening(
+      llvm::function_ref<AbstractDirectoryWatcher::CreateFnTy> createFn,
+      bool waitInitialSync, std::string &Error);
+  void stopEventListening();
+
+  void discardUnit(StringRef UnitName);
+  void discardRecord(StringRef RecordName);
+
+  void purgeStaleData();
+
+private:
+  IndexDataStore(void *Impl) : Impl(Impl) {}
+
+  void *Impl; // An IndexDataStoreImpl.
+};
+
+} // namespace index
+} // namespace clang
+
+#endif

include/clang/Index/IndexDataStoreSymbolUtils.h

 #define LLVM_CLANG_INDEX_INDEXDATASTORESYMBOLUTILS_H
 
 #include "indexstore/indexstore.h"
 #include "clang/Index/IndexSymbol.h"
 
 namespace clang {
 namespace index {
 
+/// Map an indexstore_symbol_kind_t to a SymbolKind, handling unknown values.
+SymbolKind getSymbolKind(indexstore_symbol_kind_t K);
+
+/// Map an indexstore_symbol_subkind_t to a SymbolSubKind, handling unknown
+/// values.
+SymbolSubKind getSymbolSubKind(indexstore_symbol_subkind_t K);
+
+/// Map an indexstore_symbol_language_t to a SymbolLanguage, handling unknown
+/// values.
+SymbolLanguage getSymbolLanguage(indexstore_symbol_language_t L);
+
+/// Map an indexstore representation to a SymbolPropertySet, handling
+/// unknown values.
+SymbolPropertySet getSymbolProperties(uint64_t Props);
+
+/// Map an indexstore representation to a SymbolRoleSet, handling unknown
+/// values.
+SymbolRoleSet getSymbolRoles(uint64_t Roles);
+
 /// Map a SymbolKind to an indexstore_symbol_kind_t.
 indexstore_symbol_kind_t getIndexStoreKind(SymbolKind K);
 
 /// Map a SymbolSubKind to an indexstore_symbol_subkind_t.
 indexstore_symbol_subkind_t getIndexStoreSubKind(SymbolSubKind K);
 
 /// Map a SymbolLanguage to an indexstore_symbol_language_t.
 indexstore_symbol_language_t getIndexStoreLang(SymbolLanguage L);

include/clang/Index/IndexRecordReader.h

+//===--- IndexRecordReader.h - Index record deserialization ---------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_INDEX_INDEXRECORDREADER_H
+#define LLVM_CLANG_INDEX_INDEXRECORDREADER_H
+
+#include "clang/Index/IndexSymbol.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+#include <memory>
+
+namespace llvm {
+class MemoryBuffer;
+}
+
+namespace clang {
+namespace index {
+
+struct IndexRecordDecl {
+  unsigned DeclID;
+  SymbolInfo SymInfo;
+  SymbolRoleSet Roles;
+  SymbolRoleSet RelatedRoles;
+  StringRef Name;
+  StringRef USR;
+  StringRef CodeGenName;
+};
+
+struct IndexRecordRelation {
+  SymbolRoleSet Roles;
+  const IndexRecordDecl *Dcl = nullptr;
+
+  IndexRecordRelation() = default;
+  IndexRecordRelation(SymbolRoleSet Roles, const IndexRecordDecl *Dcl)
+      : Roles(Roles), Dcl(Dcl) {}
+};
+
+struct IndexRecordOccurrence {
+  const IndexRecordDecl *Dcl;
+  SmallVector<IndexRecordRelation, 4> Relations;
+  SymbolRoleSet Roles;
+  unsigned Line;
+  unsigned Column;
+};
+
+class IndexRecordReader {
+  IndexRecordReader();
+
+public:
+  static std::unique_ptr<IndexRecordReader>
+  createWithRecordFilename(StringRef RecordFilename, StringRef StorePath,
+                           std::string &Error);
+
+  static std::unique_ptr<IndexRecordReader>
+  createWithFilePath(StringRef FilePath, std::string &Error);
+
+  static std::unique_ptr<IndexRecordReader>
+  createWithBuffer(std::unique_ptr<llvm::MemoryBuffer> Buffer,
+                   std::string &Error);
+
+  ~IndexRecordReader();
+
+  struct DeclSearchReturn {
+    bool AcceptDecl;
+    bool ContinueSearch;
+  };
+  typedef DeclSearchReturn(DeclSearchCheck)(const IndexRecordDecl &);
+
+  /// Goes through and passes record decls, after filtering using a \c Checker
+  /// function.
+  ///
+  /// Resulting decls can be used as filter for \c foreachOccurrence. This
+  /// allows allocating memory only for the record decls that the caller is
+  /// interested in.
+  bool searchDecls(llvm::function_ref<DeclSearchCheck> Checker,
+                   llvm::function_ref<void(const IndexRecordDecl *)> Receiver);
+
+  /// \param NoCache if true, avoids allocating memory for the decls.
+  /// Useful when the caller does not intend to keep \c IndexRecordReader
+  /// for more queries.
+  bool foreachDecl(bool NoCache,
+                   llvm::function_ref<bool(const IndexRecordDecl *)> Receiver);
+
+  /// \param DeclsFilter if non-empty indicates the list of decls that we want
+  /// to get occurrences for. An empty array indicates that we want occurrences
+  /// for all decls.
+  /// \param RelatedDeclsFilter Same as \c DeclsFilter but for related decls.
+  bool foreachOccurrence(
+      ArrayRef<const IndexRecordDecl *> DeclsFilter,
+      ArrayRef<const IndexRecordDecl *> RelatedDeclsFilter,
+      llvm::function_ref<bool(const IndexRecordOccurrence &)> Receiver);
+
+  bool foreachOccurrence(
+      llvm::function_ref<bool(const IndexRecordOccurrence &)> Receiver);
+
+  bool foreachOccurrenceInLineRange(
+      unsigned lineStart, unsigned lineCount,
+      llvm::function_ref<bool(const IndexRecordOccurrence &)> Receiver);
+
+  struct Implementation;
+
+private:
+  Implementation &Impl;
+};
+
+} // namespace index
+} // namespace clang
+
+#endif

include/clang/Index/IndexUnitReader.h

+//===--- IndexUnitReader.h - Index unit deserialization -------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_INDEX_INDEXUNITREADER_H
+#define LLVM_CLANG_INDEX_INDEXUNITREADER_H
+
+#include "clang/Basic/LLVM.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Chrono.h"
+
+namespace clang {
+namespace index {
+
+class IndexUnitReader {
+public:
+  enum class DependencyKind {
+    Unit,
+    Record,
+    File,
+  };
+
+  ~IndexUnitReader();
+
+  static std::unique_ptr<IndexUnitReader>
+  createWithUnitFilename(StringRef UnitFilename, StringRef StorePath,
+                         std::string &Error);
+
+  static std::unique_ptr<IndexUnitReader>
+  createWithFilePath(StringRef FilePath, std::string &Error);
+
+  static Optional<llvm::sys::TimePoint<>>
+  getModificationTimeForUnit(StringRef UnitFilename, StringRef StorePath,
+                             std::string &Error);
+
+  StringRef getProviderIdentifier() const;
+  StringRef getProviderVersion() const;
+
+  llvm::sys::TimePoint<> getModificationTime() const;
+  StringRef getWorkingDirectory() const;
+  StringRef getOutputFile() const;
+  StringRef getSysrootPath() const;
+  StringRef getMainFilePath() const;
+  StringRef getModuleName() const;
+  StringRef getTarget() const;
+  bool hasMainFile() const;
+  bool isSystemUnit() const;
+  bool isModuleUnit() const;
+  bool isDebugCompilation() const;
+
+  struct DependencyInfo {
+    DependencyKind Kind;
+    bool IsSystem;
+    StringRef UnitOrRecordName;
+    StringRef FilePath;
+    StringRef ModuleName;
+    size_t FileSize;
+    time_t ModTime;
+  };
+
+  struct IncludeInfo {
+    StringRef SourcePath;
+    unsigned SourceLine;
+    StringRef TargetPath;
+  };
+
+  /// Unit dependencies are provided ahead of record ones, record ones
+  /// ahead of the file ones.
+  bool foreachDependency(
+      llvm::function_ref<bool(const DependencyInfo &Info)> Receiver);
+
+  bool
+  foreachInclude(llvm::function_ref<bool(const IncludeInfo &Info)> Receiver);
+
+private:
+  IndexUnitReader(void *Impl) : Impl(Impl) {}
+
+  void *Impl; // An IndexUnitReaderImpl.
+};
+
+} // namespace index
+} // namespace clang
+
+#endif

include/indexstore/IndexStoreCXX.h

+//===--- IndexStoreCXX.h - C++ wrapper for the Index Store C API. ---------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Header-only C++ wrapper for the Index Store C API.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_INDEXSTORE_INDEXSTORECXX_H
+#define LLVM_CLANG_INDEXSTORE_INDEXSTORECXX_H
+
+#include "indexstore/indexstore.h"
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/Optional.h"
+#include "llvm/ADT/STLExtras.h"
+#include "llvm/ADT/SmallString.h"
+#include <ctime>
+
+namespace indexstore {
+using llvm::ArrayRef;
+using llvm::Optional;
+using llvm::StringRef;
+
+static inline StringRef
+stringFromIndexStoreStringRef(indexstore_string_ref_t str) {
+  return StringRef(str.data, str.length);
+}
+
+class IndexRecordSymbol {
+  indexstore_symbol_t obj;
+  friend class IndexRecordReader;
+
+public:
+  IndexRecordSymbol(indexstore_symbol_t obj) : obj(obj) {}
+
+  indexstore_symbol_language_t getLanguage() {
+    return indexstore_symbol_get_language(obj);
+  }
+  indexstore_symbol_kind_t getKind() { return indexstore_symbol_get_kind(obj); }
+  indexstore_symbol_subkind_t getSubKind() {
+    return indexstore_symbol_get_subkind(obj);
+  }
+  uint64_t getProperties() { return indexstore_symbol_get_properties(obj); }
+  uint64_t getRoles() { return indexstore_symbol_get_roles(obj); }
+  uint64_t getRelatedRoles() {
+    return indexstore_symbol_get_related_roles(obj);
+  }
+  StringRef getName() {
+    return stringFromIndexStoreStringRef(indexstore_symbol_get_name(obj));
+  }
+  StringRef getUSR() {
+    return stringFromIndexStoreStringRef(indexstore_symbol_get_usr(obj));
+  }
+  StringRef getCodegenName() {
+    return stringFromIndexStoreStringRef(
+        indexstore_symbol_get_codegen_name(obj));
+  }
+};
+
+class IndexSymbolRelation {
+  indexstore_symbol_relation_t obj;
+
+public:
+  IndexSymbolRelation(indexstore_symbol_relation_t obj) : obj(obj) {}
+
+  uint64_t getRoles() { return indexstore_symbol_relation_get_roles(obj); }
+  IndexRecordSymbol getSymbol() {
+    return indexstore_symbol_relation_get_symbol(obj);
+  }
+};
+
+class IndexRecordOccurrence {
+  indexstore_occurrence_t obj;
+
+public:
+  IndexRecordOccurrence(indexstore_occurrence_t obj) : obj(obj) {}
+
+  IndexRecordSymbol getSymbol() {
+    return indexstore_occurrence_get_symbol(obj);
+  }
+  uint64_t getRoles() { return indexstore_occurrence_get_roles(obj); }
+
+  bool foreachRelation(llvm::function_ref<bool(IndexSymbolRelation)> receiver) {
+#if INDEXSTORE_HAS_BLOCKS
+    return indexstore_occurrence_relations_apply(
+        obj, ^bool(indexstore_symbol_relation_t sym_rel) {
+          return receiver(sym_rel);
+        });
+#else
+    return false;
+#endif
+  }
+
+  std::pair<unsigned, unsigned> getLineCol() {
+    unsigned line, col;
+    indexstore_occurrence_get_line_col(obj, &line, &col);
+    return std::make_pair(line, col);
+  }
+};
+
+class IndexStore;
+typedef std::shared_ptr<IndexStore> IndexStoreRef;
+
+class IndexStore {
+  indexstore_t obj;
+  friend class IndexRecordReader;
+  friend class IndexUnitReader;
+
+public:
+  IndexStore(StringRef path, std::string &error) {
+    llvm::SmallString<64> buf = path;
+    indexstore_error_t c_err = nullptr;
+    obj = indexstore_store_create(buf.c_str(), &c_err);
+    if (c_err) {
+      error = indexstore_error_get_description(c_err);
+      indexstore_error_dispose(c_err);
+    }
+  }
+
+  IndexStore(IndexStore &&other) : obj(other.obj) { other.obj = nullptr; }
+
+  ~IndexStore() { indexstore_store_dispose(obj); }
+
+  static IndexStoreRef create(StringRef path, std::string &error) {
+    auto storeRef = std::make_shared<IndexStore>(path, error);
+    if (storeRef->isInvalid())
+      return nullptr;
+    return storeRef;
+  }
+
+  static unsigned formatVersion() { return indexstore_format_version(); }
+
+  bool isValid() const { return obj; }
+  bool isInvalid() const { return !isValid(); }
+  explicit operator bool() const { return isValid(); }
+
+  bool foreachUnit(bool sorted,
+                   llvm::function_ref<bool(StringRef unitName)> receiver) {
+#if INDEXSTORE_HAS_BLOCKS
+    return indexstore_store_units_apply(
+        obj, sorted, ^bool(indexstore_string_ref_t unit_name) {
+          return receiver(stringFromIndexStoreStringRef(unit_name));
+        });
+#else
+    return false;
+#endif
+  }
+
+  class UnitEvent {
+    indexstore_unit_event_t obj;
+
+  public:
+    UnitEvent(indexstore_unit_event_t obj) : obj(obj) {}
+
+    enum class Kind {
+      Added,
+      Removed,
+      Modified,
+      DirectoryDeleted,
+    };
+
+    Kind getKind() const {
+      indexstore_unit_event_kind_t c_k = indexstore_unit_event_get_kind(obj);
+      Kind K;
+      switch (c_k) {
+      case INDEXSTORE_UNIT_EVENT_ADDED:
+        K = Kind::Added;
+        break;
+      case INDEXSTORE_UNIT_EVENT_REMOVED:
+        K = Kind::Removed;
+        break;
+      case INDEXSTORE_UNIT_EVENT_MODIFIED:
+        K = Kind::Modified;
+        break;
+      case INDEXSTORE_UNIT_EVENT_DIRECTORY_DELETED:
+        K = Kind::DirectoryDeleted;
+        break;
+      }
+      return K;
+    }
+
+    StringRef getUnitName() const {
+      return stringFromIndexStoreStringRef(
+          indexstore_unit_event_get_unit_name(obj));
+    }
+
+    timespec getModificationTime() const {
+      return indexstore_unit_event_get_modification_time(obj);
+    }
+  };
+
+  class UnitEventNotification {
+    indexstore_unit_event_notification_t obj;
+
+  public:
+    UnitEventNotification(indexstore_unit_event_notification_t obj)
+        : obj(obj) {}
+
+    bool isInitial() const {
+      return indexstore_unit_event_notification_is_initial(obj);
+    }
+    size_t getEventsCount() const {
+      return indexstore_unit_event_notification_get_events_count(obj);
+    }
+    UnitEvent getEvent(size_t index) const {
+      return indexstore_unit_event_notification_get_event(obj, index);
+    }
+  };
+
+  typedef std::function<void(UnitEventNotification)> UnitEventHandler;
+
+  void setUnitEventHandler(UnitEventHandler handler) {
+#if INDEXSTORE_HAS_BLOCKS
+    if (!handler) {
+      indexstore_store_set_unit_event_handler(obj, nullptr);
+      return;
+    }
+
+    indexstore_store_set_unit_event_handler(
+        obj, ^(indexstore_unit_event_notification_t evt_note) {
+          handler(UnitEventNotification(evt_note));
+        });
+#endif
+  }
+
+  bool startEventListening(bool waitInitialSync, std::string &error) {
+    indexstore_unit_event_listen_options_t opts;
+    opts.wait_initial_sync = waitInitialSync;
+    indexstore_error_t c_err = nullptr;
+    bool ret = indexstore_store_start_unit_event_listening(
+        obj, &opts, sizeof(opts), &c_err);
+    if (c_err) {
+      error = indexstore_error_get_description(c_err);
+      indexstore_error_dispose(c_err);
+    }
+    return ret;
+  }
+
+  void stopEventListening() {
+    return indexstore_store_stop_unit_event_listening(obj);
+  }
+
+  void discardUnit(StringRef UnitName) {
+    llvm::SmallString<64> buf = UnitName;
+    indexstore_store_discard_unit(obj, buf.c_str());
+  }
+
+  void discardRecord(StringRef RecordName) {
+    llvm::SmallString<64> buf = RecordName;
+    indexstore_store_discard_record(obj, buf.c_str());
+  }
+
+  void getUnitNameFromOutputPath(StringRef outputPath,
+                                 llvm::SmallVectorImpl<char> &nameBuf) {
+    llvm::SmallString<256> buf = outputPath;
+    size_t nameLen = indexstore_store_get_unit_name_from_output_path(
+        obj, buf.c_str(), nameBuf.data(), nameBuf.size());
+    if (nameLen + 1 > nameBuf.size()) {
+      nameBuf.resize(nameLen + 1);
+      indexstore_store_get_unit_name_from_output_path(
+          obj, buf.c_str(), nameBuf.data(), nameBuf.size());
+    }
+  }
+
+  llvm::Optional<timespec> getUnitModificationTime(StringRef unitName,
+                                                   std::string &error) {
+    llvm::SmallString<64> buf = unitName;
+    int64_t seconds, nanoseconds;
+    indexstore_error_t c_err = nullptr;
+    bool err = indexstore_store_get_unit_modification_time(
+        obj, buf.c_str(), &seconds, &nanoseconds, &c_err);
+    if (err && c_err) {
+      error = indexstore_error_get_description(c_err);
+      indexstore_error_dispose(c_err);
+      return llvm::None;
+    }
+    timespec ts;
+    ts.tv_sec = seconds;
+    ts.tv_nsec = nanoseconds;
+    return ts;
+  }
+
+  void purgeStaleData() { indexstore_store_purge_stale_data(obj); }
+};
+
+class IndexRecordReader {
+  indexstore_record_reader_t obj;
+
+public:
+  IndexRecordReader(IndexStore &store, StringRef recordName,
+                    std::string &error) {
+    llvm::SmallString<64> buf = recordName;
+    indexstore_error_t c_err = nullptr;
+    obj = indexstore_record_reader_create(store.obj, buf.c_str(), &c_err);
+    if (c_err) {
+      error = indexstore_error_get_description(c_err);
+      indexstore_error_dispose(c_err);
+    }
+  }
+
+  IndexRecordReader(IndexRecordReader &&other) : obj(other.obj) {
+    other.obj = nullptr;
+  }
+
+  ~IndexRecordReader() { indexstore_record_reader_dispose(obj); }
+
+  bool isValid() const { return obj; }
+  bool isInvalid() const { return !isValid(); }
+  explicit operator bool() const { return isValid(); }
+
+  /// Goes through and passes record decls, after filtering using a \c Checker
+  /// function.
+  ///
+  /// Resulting decls can be used as filter for \c foreachOccurrence. This
+  /// allows allocating memory only for the record decls that the caller is
+  /// interested in.
+  bool
+  searchSymbols(llvm::function_ref<bool(IndexRecordSymbol, bool &stop)> filter,
+                llvm::function_ref<void(IndexRecordSymbol)> receiver) {
+#if INDEXSTORE_HAS_BLOCKS
+    return indexstore_record_reader_search_symbols(
+        obj,
+        ^bool(indexstore_symbol_t symbol, bool *stop) {
+          return filter(symbol, *stop);
+        },
+        ^(indexstore_symbol_t symbol) {
+          receiver(symbol);
+        });
+#else
+    return false;
+#endif
+  }
+
+  bool foreachSymbol(bool noCache,
+                     llvm::function_ref<bool(IndexRecordSymbol)> receiver) {
+#if INDEXSTORE_HAS_BLOCKS
+    return indexstore_record_reader_symbols_apply(
+        obj, noCache, ^bool(indexstore_symbol_t sym) {
+          return receiver(sym);
+        });
+#else
+    return false;
+#endif
+  }
+
+  /// \param DeclsFilter if non-empty indicates the list of decls that we want
+  /// to get occurrences for. An empty array indicates that we want occurrences
+  /// for all decls.
+  /// \param RelatedDeclsFilter Same as \c DeclsFilter but for related decls.
+  bool
+  foreachOccurrence(ArrayRef<IndexRecordSymbol> symbolsFilter,
+                    ArrayRef<IndexRecordSymbol> relatedSymbolsFilter,
+                    llvm::function_ref<bool(IndexRecordOccurrence)> receiver) {
+#if INDEXSTORE_HAS_BLOCKS
+    llvm::SmallVector<indexstore_symbol_t, 16> c_symbolsFilter;
+    c_symbolsFilter.reserve(symbolsFilter.size());
+    for (IndexRecordSymbol sym : symbolsFilter) {
+      c_symbolsFilter.push_back(sym.obj);
+    }
+    llvm::SmallVector<indexstore_symbol_t, 16> c_relatedSymbolsFilter;
+    c_relatedSymbolsFilter.reserve(relatedSymbolsFilter.size());
+    for (IndexRecordSymbol sym : relatedSymbolsFilter) {
+      c_relatedSymbolsFilter.push_back(sym.obj);
+    }
+    return indexstore_record_reader_occurrences_of_symbols_apply(
+        obj, c_symbolsFilter.data(), c_symbolsFilter.size(),
+        c_relatedSymbolsFilter.data(), c_relatedSymbolsFilter.size(),
+        ^bool(indexstore_occurrence_t occur) {
+          return receiver(occur);
+        });
+#else
+    return false;
+#endif
+  }
+
+  bool
+  foreachOccurrence(llvm::function_ref<bool(IndexRecordOccurrence)> receiver) {
+#if INDEXSTORE_HAS_BLOCKS
+    return indexstore_record_reader_occurrences_apply(
+        obj, ^bool(indexstore_occurrence_t occur) {
+          return receiver(occur);
+        });
+#else
+    return false;
+#endif
+  }
+
+  bool foreachOccurrenceInLineRange(
+      unsigned lineStart, unsigned lineEnd,
+      llvm::function_ref<bool(IndexRecordOccurrence)> receiver) {
+#if INDEXSTORE_HAS_BLOCKS
+    return indexstore_record_reader_occurrences_in_line_range_apply(
+        obj, lineStart, lineEnd, ^bool(indexstore_occurrence_t occur) {
+          return receiver(occur);
+        });
+#else
+    return false;
+#endif
+  }
+};
+
+class IndexUnitDependency {
+  indexstore_unit_dependency_t obj;
+  friend class IndexUnitReader;
+
+public:
+  IndexUnitDependency(indexstore_unit_dependency_t obj) : obj(obj) {}
+
+  enum class DependencyKind {
+    Unit,
+    Record,
+    File,
+  };
+  DependencyKind getKind() {
+    switch (indexstore_unit_dependency_get_kind(obj)) {
+    case INDEXSTORE_UNIT_DEPENDENCY_UNIT:
+      return DependencyKind::Unit;
+    case INDEXSTORE_UNIT_DEPENDENCY_RECORD:
+      return DependencyKind::Record;
+    case INDEXSTORE_UNIT_DEPENDENCY_FILE:
+      return DependencyKind::File;
+    }
+  }
+  bool isSystem() { return indexstore_unit_dependency_is_system(obj); }
+  StringRef getName() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_dependency_get_name(obj));
+  }
+  StringRef getFilePath() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_dependency_get_filepath(obj));
+  }
+  StringRef getModuleName() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_dependency_get_modulename(obj));
+  }
+  time_t getModificationTime() {
+    return indexstore_unit_dependency_get_modification_time(obj);
+  }
+  size_t getFileSize() { return indexstore_unit_dependency_get_file_size(obj); }
+};
+
+class IndexUnitInclude {
+  indexstore_unit_include_t obj;
+  friend class IndexUnitReader;
+
+public:
+  IndexUnitInclude(indexstore_unit_include_t obj) : obj(obj) {}
+
+  StringRef getSourcePath() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_include_get_source_path(obj));
+  }
+  StringRef getTargetPath() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_include_get_target_path(obj));
+  }
+  unsigned getSourceLine() {
+    return indexstore_unit_include_get_source_line(obj);
+  }
+};
+
+class IndexUnitReader {
+  indexstore_unit_reader_t obj;
+
+public:
+  IndexUnitReader(IndexStore &store, StringRef unitName, std::string &error) {
+    llvm::SmallString<64> buf = unitName;
+    indexstore_error_t c_err = nullptr;
+    obj = indexstore_unit_reader_create(store.obj, buf.c_str(), &c_err);
+    if (c_err) {
+      error = indexstore_error_get_description(c_err);
+      indexstore_error_dispose(c_err);
+    }
+  }
+
+  IndexUnitReader(IndexUnitReader &&other) : obj(other.obj) {
+    other.obj = nullptr;
+  }
+
+  ~IndexUnitReader() { indexstore_unit_reader_dispose(obj); }
+
+  bool isValid() const { return obj; }
+  bool isInvalid() const { return !isValid(); }
+  explicit operator bool() const { return isValid(); }
+
+  StringRef getProviderIdentifier() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_reader_get_provider_identifier(obj));
+  }
+  StringRef getProviderVersion() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_reader_get_provider_version(obj));
+  }
+
+  timespec getModificationTime() {
+    int64_t seconds, nanoseconds;
+    indexstore_unit_reader_get_modification_time(obj, &seconds, &nanoseconds);
+    timespec ts;
+    ts.tv_sec = seconds;
+    ts.tv_nsec = nanoseconds;
+    return ts;
+  }
+
+  bool isSystemUnit() { return indexstore_unit_reader_is_system_unit(obj); }
+  bool isModuleUnit() { return indexstore_unit_reader_is_module_unit(obj); }
+  bool isDebugCompilation() {
+    return indexstore_unit_reader_is_debug_compilation(obj);
+  }
+  bool hasMainFile() { return indexstore_unit_reader_has_main_file(obj); }
+
+  StringRef getMainFilePath() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_reader_get_main_file(obj));
+  }
+  StringRef getModuleName() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_reader_get_module_name(obj));
+  }
+  StringRef getWorkingDirectory() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_reader_get_working_dir(obj));
+  }
+  StringRef getOutputFile() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_reader_get_output_file(obj));
+  }
+  StringRef getSysrootPath() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_reader_get_sysroot_path(obj));
+  }
+  StringRef getTarget() {
+    return stringFromIndexStoreStringRef(
+        indexstore_unit_reader_get_target(obj));
+  }
+
+  bool
+  foreachDependency(llvm::function_ref<bool(IndexUnitDependency)> receiver) {
+#if INDEXSTORE_HAS_BLOCKS
+    return indexstore_unit_reader_dependencies_apply(
+        obj, ^bool(indexstore_unit_dependency_t dep) {
+          return receiver(dep);
+        });
+#else
+    return false;
+#endif
+  }
+
+  bool foreachInclude(llvm::function_ref<bool(IndexUnitInclude)> receiver) {
+#if INDEXSTORE_HAS_BLOCKS
+    return indexstore_unit_reader_includes_apply(
+        obj, ^bool(indexstore_unit_include_t inc) {
+          return receiver(inc);
+        });
+#else
+    return false;
+#endif
+  }
+};
+
+} // namespace indexstore
+
+#endif

include/indexstore/indexstore.h

 #ifdef __cplusplus
 #define INDEXSTORE_BEGIN_DECLS extern "C" {
 #define INDEXSTORE_END_DECLS }
 #else
 #define INDEXSTORE_BEGIN_DECLS
 #define INDEXSTORE_END_DECLS
 #endif
 
+#ifndef INDEXSTORE_PUBLIC
+#if defined(_MSC_VER)
+#define INDEXSTORE_PUBLIC __declspec(dllimport)
+#else
+#define INDEXSTORE_PUBLIC
+#endif
+#endif
+
+#ifndef __has_feature
+#define __has_feature(x) 0
+#endif
+
+#if __has_feature(blocks)
+#define INDEXSTORE_HAS_BLOCKS 1
+#else
+#define INDEXSTORE_HAS_BLOCKS 0
+#endif
+
 INDEXSTORE_BEGIN_DECLS
 
+typedef void *indexstore_error_t;
+
+INDEXSTORE_PUBLIC const char *
+indexstore_error_get_description(indexstore_error_t error);
+
+INDEXSTORE_PUBLIC void indexstore_error_dispose(indexstore_error_t error);
+
+typedef struct {
+  const char *data;
+  size_t length;
+} indexstore_string_ref_t;
+
+INDEXSTORE_PUBLIC unsigned indexstore_format_version(void);
+
+typedef void *indexstore_t;
+
+INDEXSTORE_PUBLIC indexstore_t
+indexstore_store_create(const char *store_path, indexstore_error_t *error);
+
+INDEXSTORE_PUBLIC void indexstore_store_dispose(indexstore_t store);
+
+#if INDEXSTORE_HAS_BLOCKS
+INDEXSTORE_PUBLIC bool indexstore_store_units_apply(
+    indexstore_t, unsigned sorted,
+    bool (^applier)(indexstore_string_ref_t unit_name));
+#endif
+
+typedef void *indexstore_unit_event_notification_t;
+typedef void *indexstore_unit_event_t;
+
+INDEXSTORE_PUBLIC size_t indexstore_unit_event_notification_get_events_count(
+    indexstore_unit_event_notification_t notification);
+
+INDEXSTORE_PUBLIC indexstore_unit_event_t
+indexstore_unit_event_notification_get_event(
+    indexstore_unit_event_notification_t notification, size_t index);
+
+INDEXSTORE_PUBLIC bool indexstore_unit_event_notification_is_initial(
+    indexstore_unit_event_notification_t notification);
+
+typedef enum {
+  INDEXSTORE_UNIT_EVENT_ADDED = 1,
+  INDEXSTORE_UNIT_EVENT_REMOVED = 2,
+  INDEXSTORE_UNIT_EVENT_MODIFIED = 3,
+  INDEXSTORE_UNIT_EVENT_DIRECTORY_DELETED = 4,
+} indexstore_unit_event_kind_t;
+
+INDEXSTORE_PUBLIC indexstore_unit_event_kind_t
+indexstore_unit_event_get_kind(indexstore_unit_event_t event);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_event_get_unit_name(indexstore_unit_event_t event);
+
+INDEXSTORE_PUBLIC timespec
+indexstore_unit_event_get_modification_time(indexstore_unit_event_t event);
+
+#if INDEXSTORE_HAS_BLOCKS
+typedef void (^indexstore_unit_event_handler_t)(
+    indexstore_unit_event_notification_t);
+
+INDEXSTORE_PUBLIC void indexstore_store_set_unit_event_handler(
+    indexstore_t store, indexstore_unit_event_handler_t handler);
+#endif
+
+typedef struct {
+  /// If true, \c indexstore_store_start_unit_event_listening will block until
+  /// the initial set of units is passed to the unit event handler, otherwise
+  /// the function will return and the initial set will be passed
+  /// asynchronously.
+  bool wait_initial_sync;
+} indexstore_unit_event_listen_options_t;
+
+INDEXSTORE_PUBLIC bool indexstore_store_start_unit_event_listening(
+    indexstore_t store, indexstore_unit_event_listen_options_t *options,
+    size_t listen_options_struct_size, indexstore_error_t *error);
+
+INDEXSTORE_PUBLIC void
+indexstore_store_stop_unit_event_listening(indexstore_t store);
+
+INDEXSTORE_PUBLIC void indexstore_store_discard_unit(indexstore_t store,
+                                                     const char *unit_name);
+
+INDEXSTORE_PUBLIC void indexstore_store_discard_record(indexstore_t store,
+                                                       const char *record_name);
+
+INDEXSTORE_PUBLIC void indexstore_store_purge_stale_data(indexstore_t store);
+
+/// Determines the unit name from the \c output_path and writes it out in the
+/// \c name_buf buffer. It doesn't write more than \c buf_size.
+/// \returns the length of the name. If this is larger than \c buf_size, the
+/// caller should call the function again with a buffer of the appropriate size.
+INDEXSTORE_PUBLIC size_t indexstore_store_get_unit_name_from_output_path(
+    indexstore_t store, const char *output_path, char *name_buf,
+    size_t buf_size);
+
+/// \returns true if an error occurred, false otherwise.
+INDEXSTORE_PUBLIC bool indexstore_store_get_unit_modification_time(
+    indexstore_t store, const char *unit_name, int64_t *seconds,
+    int64_t *nanoseconds, indexstore_error_t *error);
+
+typedef void *indexstore_symbol_t;
+
 typedef enum {
   INDEXSTORE_SYMBOL_KIND_UNKNOWN = 0,
   INDEXSTORE_SYMBOL_KIND_MODULE = 1,
   INDEXSTORE_SYMBOL_KIND_NAMESPACE = 2,
   INDEXSTORE_SYMBOL_KIND_NAMESPACEALIAS = 3,
   INDEXSTORE_SYMBOL_KIND_MACRO = 4,
   INDEXSTORE_SYMBOL_KIND_ENUM = 5,
   INDEXSTORE_SYMBOL_KIND_STRUCT = 6,
typedef enum {
   INDEXSTORE_SYMBOL_ROLE_REL_CALLEDBY = 1 << 13,
   INDEXSTORE_SYMBOL_ROLE_REL_EXTENDEDBY = 1 << 14,
   INDEXSTORE_SYMBOL_ROLE_REL_ACCESSOROF = 1 << 15,
   INDEXSTORE_SYMBOL_ROLE_REL_CONTAINEDBY = 1 << 16,
   INDEXSTORE_SYMBOL_ROLE_REL_IBTYPEOF = 1 << 17,
   INDEXSTORE_SYMBOL_ROLE_REL_SPECIALIZATIONOF = 1 << 18,
 } indexstore_symbol_role_t;
 
+INDEXSTORE_PUBLIC indexstore_symbol_language_t
+indexstore_symbol_get_language(indexstore_symbol_t symbol);
+
+INDEXSTORE_PUBLIC indexstore_symbol_kind_t
+indexstore_symbol_get_kind(indexstore_symbol_t symbol);
+
+INDEXSTORE_PUBLIC indexstore_symbol_subkind_t
+indexstore_symbol_get_subkind(indexstore_symbol_t symbol);
+
+INDEXSTORE_PUBLIC uint64_t
+indexstore_symbol_get_properties(indexstore_symbol_t symbol);
+
+INDEXSTORE_PUBLIC uint64_t
+indexstore_symbol_get_roles(indexstore_symbol_t symbol);
+
+INDEXSTORE_PUBLIC uint64_t
+indexstore_symbol_get_related_roles(indexstore_symbol_t symbol);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_symbol_get_name(indexstore_symbol_t symbol);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_symbol_get_usr(indexstore_symbol_t symbol);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_symbol_get_codegen_name(indexstore_symbol_t symbol);
+
+typedef void *indexstore_symbol_relation_t;
+
+INDEXSTORE_PUBLIC uint64_t
+indexstore_symbol_relation_get_roles(indexstore_symbol_relation_t relation);
+
+INDEXSTORE_PUBLIC indexstore_symbol_t
+indexstore_symbol_relation_get_symbol(indexstore_symbol_relation_t relation);
+
+typedef void *indexstore_occurrence_t;
+
+INDEXSTORE_PUBLIC indexstore_symbol_t
+indexstore_occurrence_get_symbol(indexstore_occurrence_t occur);
+
+#if INDEXSTORE_HAS_BLOCKS
+INDEXSTORE_PUBLIC bool indexstore_occurrence_relations_apply(
+    indexstore_occurrence_t occur,
+    bool (^applier)(indexstore_symbol_relation_t relation));
+#endif
+
+INDEXSTORE_PUBLIC uint64_t
+indexstore_occurrence_get_roles(indexstore_occurrence_t occur);
+
+INDEXSTORE_PUBLIC void
+indexstore_occurrence_get_line_col(indexstore_occurrence_t occur,
+                                   unsigned *line, unsigned *column);
+
+typedef void *indexstore_record_reader_t;
+
+INDEXSTORE_PUBLIC indexstore_record_reader_t indexstore_record_reader_create(
+    indexstore_t store, const char *record_name, indexstore_error_t *error);
+
+INDEXSTORE_PUBLIC void
+indexstore_record_reader_dispose(indexstore_record_reader_t reader);
+
+#if INDEXSTORE_HAS_BLOCKS
+/// Goes through the symbol data and passes symbols to \c receiver, for the
+/// symbol data that \c filter returns true on.
+///
+/// This allows allocating memory only for the record symbols that the caller is
+/// interested in.
+INDEXSTORE_PUBLIC bool indexstore_record_reader_search_symbols(
+    indexstore_record_reader_t reader,
+    bool (^filter)(indexstore_symbol_t symbol, bool *stop),
+    void (^receiver)(indexstore_symbol_t symbol));
+
+/// \param nocache if true, avoids allocating memory for the symbols.
+/// Useful when the caller does not intend to keep \c indexstore_record_reader_t
+/// for more queries.
+INDEXSTORE_PUBLIC bool indexstore_record_reader_symbols_apply(
+    indexstore_record_reader_t reader, bool nocache,
+    bool (^applier)(indexstore_symbol_t symbol));
+
+INDEXSTORE_PUBLIC bool indexstore_record_reader_occurrences_apply(
+    indexstore_record_reader_t reader,
+    bool (^applier)(indexstore_occurrence_t occur));
+
+INDEXSTORE_PUBLIC bool indexstore_record_reader_occurrences_in_line_range_apply(
+    indexstore_record_reader_t reader, unsigned line_start, unsigned line_count,
+    bool (^applier)(indexstore_occurrence_t occur));
+
+/// \param symbols if non-zero \c symbols_count, indicates the list of symbols
+/// that we want to get occurrences for. An empty array indicates that we want
+/// occurrences for all symbols.
+/// \param related_symbols Same as \c symbols but for related symbols.
+INDEXSTORE_PUBLIC bool indexstore_record_reader_occurrences_of_symbols_apply(
+    indexstore_record_reader_t reader, indexstore_symbol_t *symbols,
+    size_t symbols_count, indexstore_symbol_t *related_symbols,
+    size_t related_symbols_count,
+    bool (^applier)(indexstore_occurrence_t occur));
+#endif
+
+typedef void *indexstore_unit_reader_t;
+
+INDEXSTORE_PUBLIC indexstore_unit_reader_t indexstore_unit_reader_create(
+    indexstore_t store, const char *unit_name, indexstore_error_t *error);
+
+INDEXSTORE_PUBLIC void
+indexstore_unit_reader_dispose(indexstore_unit_reader_t reader);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_reader_get_provider_identifier(indexstore_unit_reader_t reader);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_reader_get_provider_version(indexstore_unit_reader_t reader);
+
+INDEXSTORE_PUBLIC void indexstore_unit_reader_get_modification_time(
+    indexstore_unit_reader_t reader, int64_t *seconds, int64_t *nanoseconds);
+
+INDEXSTORE_PUBLIC bool
+indexstore_unit_reader_is_system_unit(indexstore_unit_reader_t reader);
+
+INDEXSTORE_PUBLIC bool
+indexstore_unit_reader_is_module_unit(indexstore_unit_reader_t reader);
+
+INDEXSTORE_PUBLIC bool
+indexstore_unit_reader_is_debug_compilation(indexstore_unit_reader_t reader);
+
+INDEXSTORE_PUBLIC bool
+indexstore_unit_reader_has_main_file(indexstore_unit_reader_t reader);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_reader_get_main_file(indexstore_unit_reader_t reader);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_reader_get_module_name(indexstore_unit_reader_t reader);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_reader_get_working_dir(indexstore_unit_reader_t reader);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_reader_get_output_file(indexstore_unit_reader_t reader);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_reader_get_sysroot_path(indexstore_unit_reader_t reader);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_reader_get_target(indexstore_unit_reader_t reader);
+
+typedef void *indexstore_unit_dependency_t;
+typedef void *indexstore_unit_include_t;
+
+typedef enum {
+  INDEXSTORE_UNIT_DEPENDENCY_UNIT = 1,
+  INDEXSTORE_UNIT_DEPENDENCY_RECORD = 2,
+  INDEXSTORE_UNIT_DEPENDENCY_FILE = 3,
+} indexstore_unit_dependency_kind_t;
+
+INDEXSTORE_PUBLIC indexstore_unit_dependency_kind_t
+indexstore_unit_dependency_get_kind(indexstore_unit_dependency_t dep);
+
+INDEXSTORE_PUBLIC bool
+indexstore_unit_dependency_is_system(indexstore_unit_dependency_t dep);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_dependency_get_filepath(indexstore_unit_dependency_t dep);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_dependency_get_modulename(indexstore_unit_dependency_t dep);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_dependency_get_name(indexstore_unit_dependency_t dep);
+
+INDEXSTORE_PUBLIC time_t indexstore_unit_dependency_get_modification_time(
+    indexstore_unit_dependency_t dep);
+
+INDEXSTORE_PUBLIC size_t
+indexstore_unit_dependency_get_file_size(indexstore_unit_dependency_t dep);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_include_get_source_path(indexstore_unit_include_t dep);
+
+INDEXSTORE_PUBLIC indexstore_string_ref_t
+indexstore_unit_include_get_target_path(indexstore_unit_include_t dep);
+
+INDEXSTORE_PUBLIC unsigned
+indexstore_unit_include_get_source_line(indexstore_unit_include_t dep);
+
+#if INDEXSTORE_HAS_BLOCKS
+INDEXSTORE_PUBLIC bool indexstore_unit_reader_dependencies_apply(
+    indexstore_unit_reader_t reader,
+    bool (^applier)(indexstore_unit_dependency_t dep));
+
+INDEXSTORE_PUBLIC bool indexstore_unit_reader_includes_apply(
+    indexstore_unit_reader_t reader,
+    bool (^applier)(indexstore_unit_include_t include));
+
+#endif
+
 INDEXSTORE_END_DECLS
 
 #endif

lib/CMakeLists.txt

if(CLANG_ENABLE_ARCMT)
   add_subdirectory(ARCMigrate)
 endif()
 add_subdirectory(Driver)
 add_subdirectory(Serialization)
 add_subdirectory(Frontend)
 add_subdirectory(FrontendTool)
 add_subdirectory(Tooling)
 add_subdirectory(Index)
+add_subdirectory(DirectoryWatcher)
 if(CLANG_ENABLE_STATIC_ANALYZER)
   add_subdirectory(StaticAnalyzer)
 endif()
 add_subdirectory(Format)

lib/DirectoryWatcher/CMakeLists.txt

+set(LLVM_LINK_COMPONENTS support)
+
+add_clang_library(clangDirectoryWatcher
+  DirectoryWatcher.cpp
+
+  LINK_LIBS
+  clangBasic
+  )

lib/DirectoryWatcher/DirectoryWatcher.cpp

+//===- DirectoryWatcher.cpp - Listens for directory file changes ----------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+/// \file
+/// \brief Utility class for listening for file system changes in a directory.
+//===----------------------------------------------------------------------===//
+
+#include "clang/DirectoryWatcher/DirectoryWatcher.h"
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/raw_ostream.h"
+
+#define HAVE_CORESERVICES 0
+
+#if defined(__has_include)
+#if __has_include(<CoreServices/CoreServices.h>)
+
+#include <CoreServices/CoreServices.h>
+#undef HAVE_CORESERVICES
+#define HAVE_CORESERVICES 1
+
+#endif
+#endif
+
+using namespace clang;
+using namespace llvm;
+
+static timespec toTimeSpec(sys::TimePoint<> tp) {
+  std::chrono::seconds sec =
+      std::chrono::time_point_cast<std::chrono::seconds>(tp).time_since_epoch();
+  std::chrono::nanoseconds nsec =
+      std::chrono::time_point_cast<std::chrono::nanoseconds>(tp - sec)
+          .time_since_epoch();
+  timespec ts;
+  ts.tv_sec = sec.count();
+  ts.tv_nsec = nsec.count();
+  return ts;
+}
+
+static Optional<timespec> getModTime(StringRef path) {
+  sys::fs::file_status Status;
+  std::error_code EC = status(path, Status);
+  if (EC)
+    return None;
+  return toTimeSpec(Status.getLastModificationTime());
+}
+
+struct DirectoryWatcher::Implementation {
+#if HAVE_CORESERVICES
+  FSEventStreamRef EventStream = nullptr;
+
+  bool setupFSEventStream(StringRef path, EventReceiver receiver,
+                          dispatch_queue_t queue);
+  void stopFSEventStream();
+
+  ~Implementation() { stopFSEventStream(); };
+#endif
+};
+
+#if HAVE_CORESERVICES
+namespace {
+struct EventStreamContextData {
+  std::string WatchedPath;
+  DirectoryWatcher::EventReceiver Receiver;
+
+  EventStreamContextData(std::string watchedPath,
+                         DirectoryWatcher::EventReceiver receiver)
+      : WatchedPath(std::move(watchedPath)), Receiver(std::move(receiver)) {}
+
+  static void dispose(const void *ctx) {
+    delete static_cast<const EventStreamContextData *>(ctx);
+  }
+};
+} // namespace
+
+static void eventStreamCallback(ConstFSEventStreamRef stream,
+                                void *clientCallBackInfo, size_t numEvents,
+                                void *eventPaths,
+                                const FSEventStreamEventFlags eventFlags[],
+                                const FSEventStreamEventId eventIds[]) {
+  auto *ctx = static_cast<EventStreamContextData *>(clientCallBackInfo);
+
+  std::vector<DirectoryWatcher::Event> Events;
+  for (size_t i = 0; i < numEvents; ++i) {
+    StringRef path = ((const char **)eventPaths)[i];
+    const FSEventStreamEventFlags flags = eventFlags[i];
+    if (!(flags & kFSEventStreamEventFlagItemIsFile)) {
+      if ((flags & kFSEventStreamEventFlagItemRemoved) &&
+          path == ctx->WatchedPath) {
+        DirectoryWatcher::Event Evt{
+            DirectoryWatcher::EventKind::DirectoryDeleted, path, timespec{}};
+        Events.push_back(Evt);
+        break;
+      }
+      continue;
+    }
+    DirectoryWatcher::EventKind K = DirectoryWatcher::EventKind::Modified;
+    if ((flags & kFSEventStreamEventFlagItemCreated) ||
+        (flags & kFSEventStreamEventFlagItemRenamed))
+      K = DirectoryWatcher::EventKind::Added;
+    if (flags & kFSEventStreamEventFlagItemRemoved)
+      K = DirectoryWatcher::EventKind::Removed;
+    timespec modTime{};
+    if (K != DirectoryWatcher::EventKind::Removed) {
+      auto modTimeOpt = getModTime(path);
+      if (!modTimeOpt.hasValue())
+        continue;
+      modTime = modTimeOpt.getValue();
+    }
+    DirectoryWatcher::Event Evt{K, path, modTime};
+    Events.push_back(Evt);
+  }
+
+  ctx->Receiver(Events, /*isInitial=*/false);
+}
+
+bool DirectoryWatcher::Implementation::setupFSEventStream(
+    StringRef path, EventReceiver receiver, dispatch_queue_t queue) {
+  if (path.empty())
+    return true;
+
+  CFMutableArrayRef pathsToWatch =
+      CFArrayCreateMutable(nullptr, 0, &kCFTypeArrayCallBacks);
+  CFStringRef cfPathStr =
+      CFStringCreateWithBytes(nullptr, (const UInt8 *)path.data(), path.size(),
+                              kCFStringEncodingUTF8, false);
+  CFArrayAppendValue(pathsToWatch, cfPathStr);
+  CFRelease(cfPathStr);
+  CFAbsoluteTime latency = 0.2; // Latency in seconds.
+
+  std::string realPath;
+  {
+    SmallString<128> Storage;
+    StringRef P = llvm::Twine(path).toNullTerminatedStringRef(Storage);
+    char Buffer[PATH_MAX];
+    // Use ::realpath to get the real path name
+    if (::realpath(P.begin(), Buffer) != nullptr)
+      realPath = Buffer;
+    else
+      realPath = path;
+  }
+
+  EventStreamContextData *ctxData =
+      new EventStreamContextData(std::move(realPath), std::move(receiver));
+  FSEventStreamContext context;
+  context.version = 0;
+  context.info = ctxData;
+  context.retain = nullptr;
+  context.release = EventStreamContextData::dispose;
+  context.copyDescription = nullptr;
+
+  EventStream = FSEventStreamCreate(
+      nullptr, eventStreamCallback, &context, pathsToWatch,
+      kFSEventStreamEventIdSinceNow, latency,
+      kFSEventStreamCreateFlagFileEvents | kFSEventStreamCreateFlagNoDefer);
+  CFRelease(pathsToWatch);
+  if (!EventStream) {
+    return true;
+  }
+  FSEventStreamSetDispatchQueue(EventStream, queue);
+  FSEventStreamStart(EventStream);
+  return false;
+}
+
+void DirectoryWatcher::Implementation::stopFSEventStream() {
+  if (!EventStream)
+    return;
+  FSEventStreamStop(EventStream);
+  FSEventStreamInvalidate(EventStream);
+  FSEventStreamRelease(EventStream);
+  EventStream = nullptr;
+}
+#endif
+
+DirectoryWatcher::DirectoryWatcher() : Impl(*new Implementation()) {}
+
+DirectoryWatcher::~DirectoryWatcher() { delete &Impl; }
+
+#if HAVE_CORESERVICES
+static std::vector<DirectoryWatcher::Event> scanDirectory(StringRef Path) {
+  using namespace llvm::sys;
+
+  std::vector<DirectoryWatcher::Event> Events;
+  std::error_code EC;
+  for (auto It = fs::directory_iterator(Path, EC),
+            End = fs::directory_iterator();
+       !EC && It != End; It.increment(EC)) {
+    auto modTime = getModTime(It->path());
+    if (!modTime.hasValue())
+      continue;
+    DirectoryWatcher::Event Event{DirectoryWatcher::EventKind::Added,
+                                  It->path(), modTime.getValue()};
+    Events.push_back(std::move(Event));
+  }
+  return Events;
+}
+#endif
+
+std::unique_ptr<DirectoryWatcher>
+DirectoryWatcher::create(StringRef Path, EventReceiver Receiver,
+                         bool waitInitialSync, std::string &Error) {
+#if HAVE_CORESERVICES
+
+  using namespace llvm::sys;
+
+  if (!fs::exists(Path)) {
+    std::error_code EC = fs::create_directories(Path);
+    if (EC) {
+      Error = EC.message();
+      return nullptr;
+    }
+  }
+
+  bool IsDir;
+  std::error_code EC = fs::is_directory(Path, IsDir);
+  if (EC) {
+    Error = EC.message();
+    return nullptr;
+  }
+  if (!IsDir) {
+    Error = "path is not a directory: ";
+    Error += Path;
+    return nullptr;
+  }
+
+  std::unique_ptr<DirectoryWatcher> DirWatch;
+  DirWatch.reset(new DirectoryWatcher());
+  auto &Impl = DirWatch->Impl;
+
+  dispatch_queue_t queue =
+      dispatch_queue_create("DirectoryWatcher", DISPATCH_QUEUE_SERIAL);
+  dispatch_semaphore_t initScanSema = dispatch_semaphore_create(0);
+  dispatch_semaphore_t setupFSEventsSema = dispatch_semaphore_create(0);
+
+  std::string copiedPath = Path;
+  dispatch_retain(initScanSema);
+  dispatch_retain(setupFSEventsSema);
+  dispatch_async(queue, ^{
+    // Wait for the event stream to be setup before doing the initial scan,
+    // to make sure we won't miss any events.
+    dispatch_semaphore_wait(setupFSEventsSema, DISPATCH_TIME_FOREVER);
+    auto events = scanDirectory(copiedPath);
+    Receiver(events, /*isInitial=*/true);
+    dispatch_semaphore_signal(initScanSema);
+    dispatch_release(setupFSEventsSema);
+    dispatch_release(initScanSema);
+  });
+  bool fsErr = Impl.setupFSEventStream(Path, Receiver, queue);
+  dispatch_semaphore_signal(setupFSEventsSema);
+
+  if (waitInitialSync) {
+    dispatch_semaphore_wait(initScanSema, DISPATCH_TIME_FOREVER);
+  }
+  dispatch_release(setupFSEventsSema);
+  dispatch_release(initScanSema);
+  dispatch_release(queue);
+
+  if (fsErr) {
+    raw_string_ostream(Error)
+        << "failed to setup FSEvents stream for path: " << Path;
+    return nullptr;
+  }
+
+  return DirWatch;
+#else
+  return nullptr;
+#endif
+}

lib/Index/CMakeLists.txt

add_clang_library(clangIndex
   FileIndexData.cpp
   IndexBody.cpp
   IndexDataStore.cpp
   IndexDataStoreUtils.cpp
   IndexDecl.cpp
   IndexingAction.cpp
   IndexingContext.cpp
   IndexRecordHasher.cpp
+  IndexRecordReader.cpp
   IndexRecordWriter.cpp
   IndexSymbol.cpp
   IndexTypeSourceInfo.cpp
   UnitIndexDataRecorder.cpp
+  IndexUnitReader.cpp
   IndexUnitWriter.cpp
   USRGeneration.cpp
 
   ADDITIONAL_HEADERS
   IndexingContext.h
   SimpleFormatContext.h
 
   LINK_LIBS

lib/Index/IndexDataStore.cpp

 //===--- IndexDataStore.cpp - Index data store info -----------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
+#include "clang/Index/IndexDataStore.h"
 #include "IndexDataStoreUtils.h"
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Mutex.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/raw_ostream.h"
 

 void store::appendInteriorRecordPath(StringRef RecordName,
                                      SmallVectorImpl<char> &PathBuf) {
   // To avoid putting a huge number of files into the records directory, create
   // subdirectories based on the last 2 characters from the hash.
   StringRef hash2chars = RecordName.substr(RecordName.size() - 2);
   sys::path::append(PathBuf, hash2chars);
   sys::path::append(PathBuf, RecordName);
 }
+
+//===----------------------------------------------------------------------===//
+// IndexDataStore
+//===----------------------------------------------------------------------===//
+
+namespace {
+
+class UnitEventHandlerData {
+  mutable sys::Mutex Mtx;
+  IndexDataStore::UnitEventHandler Handler;
+
+public:
+  void setHandler(IndexDataStore::UnitEventHandler handler) {
+    sys::ScopedLock L(Mtx);
+    Handler = std::move(handler);
+  }
+  IndexDataStore::UnitEventHandler getHandler() const {
+    sys::ScopedLock L(Mtx);
+    return Handler;
+  }
+};
+
+class IndexDataStoreImpl {
+  std::string FilePath;
+  std::shared_ptr<UnitEventHandlerData> TheUnitEventHandlerData;
+  std::unique_ptr<AbstractDirectoryWatcher> DirWatcher;
+
+public:
+  explicit IndexDataStoreImpl(StringRef indexStorePath)
+      : FilePath(indexStorePath) {
+    TheUnitEventHandlerData = std::make_shared<UnitEventHandlerData>();
+  }
+
+  StringRef getFilePath() const { return FilePath; }
+  bool foreachUnitName(bool sorted,
+                       llvm::function_ref<bool(StringRef unitName)> receiver);
+  void setUnitEventHandler(IndexDataStore::UnitEventHandler Handler);
+  bool startEventListening(
+      llvm::function_ref<AbstractDirectoryWatcher::CreateFnTy> createFn,
+      bool waitInitialSync, std::string &Error);
+  void stopEventListening();
+  void discardUnit(StringRef UnitName);
+  void discardRecord(StringRef RecordName);
+  void purgeStaleData();
+};
+
+} // anonymous namespace
+
+bool IndexDataStoreImpl::foreachUnitName(
+    bool sorted, llvm::function_ref<bool(StringRef unitName)> receiver) {
+  SmallString<128> UnitPath;
+  UnitPath = FilePath;
+  appendUnitSubDir(UnitPath);
+
+  std::vector<std::string> filenames;
+
+  std::error_code EC;
+  for (auto It = sys::fs::directory_iterator(UnitPath, EC),
+            End = sys::fs::directory_iterator();
+       !EC && It != End; It.increment(EC)) {
+    StringRef unitName = sys::path::filename(It->path());
+    if (!sorted) {
+      if (!receiver(unitName))
+        return false;
+    } else {
+      filenames.push_back(unitName);
+    }
+  }
+
+  if (sorted) {
+    llvm::array_pod_sort(filenames.begin(), filenames.end());
+    for (auto &fname : filenames)
+      if (!receiver(fname))
+        return false;
+  }
+  return true;
+}
+
+void IndexDataStoreImpl::setUnitEventHandler(
+    IndexDataStore::UnitEventHandler handler) {
+  TheUnitEventHandlerData->setHandler(std::move(handler));
+}
+
+bool IndexDataStoreImpl::startEventListening(
+    llvm::function_ref<AbstractDirectoryWatcher::CreateFnTy> createFn,
+    bool waitInitialSync, std::string &Error) {
+  if (DirWatcher) {
+    Error = "event listener already active";
+    return true;
+  }
+
+  SmallString<128> UnitPath;
+  UnitPath = FilePath;
+  appendUnitSubDir(UnitPath);
+
+  auto localUnitEventHandlerData = TheUnitEventHandlerData;
+  auto OnUnitsChange = [localUnitEventHandlerData](
+                           ArrayRef<AbstractDirectoryWatcher::Event> Events,
+                           bool isInitial) {
+    SmallVector<IndexDataStore::UnitEvent, 16> UnitEvents;
+    UnitEvents.reserve(Events.size());
+    for (const AbstractDirectoryWatcher::Event &evt : Events) {
+      IndexDataStore::UnitEventKind K;
+      StringRef UnitName = sys::path::filename(evt.Filename);
+      switch (evt.Kind) {
+      case AbstractDirectoryWatcher::EventKind::Added:
+        K = IndexDataStore::UnitEventKind::Added;
+        break;
+      case AbstractDirectoryWatcher::EventKind::Removed:
+        K = IndexDataStore::UnitEventKind::Removed;
+        break;
+      case AbstractDirectoryWatcher::EventKind::Modified:
+        K = IndexDataStore::UnitEventKind::Modified;
+        break;
+      case AbstractDirectoryWatcher::EventKind::DirectoryDeleted:
+        K = IndexDataStore::UnitEventKind::DirectoryDeleted;
+        UnitName = StringRef();
+        break;
+      }
+      UnitEvents.push_back(IndexDataStore::UnitEvent{K, UnitName, evt.ModTime});
+    }
+
+    if (auto handler = localUnitEventHandlerData->getHandler()) {
+      IndexDataStore::UnitEventNotification EventNote{isInitial, UnitEvents};
+      handler(EventNote);
+    }
+  };
+
+  DirWatcher = createFn(UnitPath.str(), OnUnitsChange, waitInitialSync, Error);
+  if (!DirWatcher)
+    return true;
+
+  return false;
+}
+
+void IndexDataStoreImpl::stopEventListening() { DirWatcher.reset(); }
+
+void IndexDataStoreImpl::discardUnit(StringRef UnitName) {
+  SmallString<128> UnitPath;
+  UnitPath = FilePath;
+  appendUnitSubDir(UnitPath);
+  appendInteriorUnitPath(UnitName, UnitPath);
+  sys::fs::remove(UnitPath);
+}
+
+void IndexDataStoreImpl::discardRecord(StringRef RecordName) {
+  SmallString<128> RecordPath;
+  RecordPath = FilePath;
+  appendRecordSubDir(RecordPath);
+  appendInteriorRecordPath(RecordName, RecordPath);
+  sys::fs::remove(RecordPath);
+}
+
+void IndexDataStoreImpl::purgeStaleData() {
+  // FIXME: Implement.
+}
+
+std::unique_ptr<IndexDataStore> IndexDataStore::create(StringRef IndexStorePath,
+                                                       std::string &Error) {
+  if (!sys::fs::exists(IndexStorePath)) {
+    raw_string_ostream OS(Error);
+    OS << "index store path does not exist: " << IndexStorePath;
+    return nullptr;
+  }
+
+  return std::unique_ptr<IndexDataStore>(
+      new IndexDataStore(new IndexDataStoreImpl(IndexStorePath)));
+}
+
+#define IMPL static_cast<IndexDataStoreImpl *>(Impl)
+
+IndexDataStore::~IndexDataStore() { delete IMPL; }
+
+StringRef IndexDataStore::getFilePath() const { return IMPL->getFilePath(); }
+
+bool IndexDataStore::foreachUnitName(
+    bool sorted, llvm::function_ref<bool(StringRef unitName)> receiver) {
+  return IMPL->foreachUnitName(sorted, std::move(receiver));
+}
+
+unsigned IndexDataStore::getFormatVersion() { return STORE_FORMAT_VERSION; }
+
+void IndexDataStore::setUnitEventHandler(UnitEventHandler Handler) {
+  return IMPL->setUnitEventHandler(std::move(Handler));
+}
+
+bool IndexDataStore::startEventListening(
+    llvm::function_ref<AbstractDirectoryWatcher::CreateFnTy> createFn,
+    bool waitInitialSync, std::string &Error) {
+  return IMPL->startEventListening(std::move(createFn), waitInitialSync, Error);
+}
+
+void IndexDataStore::stopEventListening() { return IMPL->stopEventListening(); }
+
+void IndexDataStore::discardUnit(StringRef UnitName) {
+  IMPL->discardUnit(UnitName);
+}
+
+void IndexDataStore::discardRecord(StringRef RecordName) {
+  IMPL->discardRecord(RecordName);
+}
+
+void IndexDataStore::purgeStaleData() { IMPL->purgeStaleData(); }

lib/Index/IndexDataStoreUtils.cpp

void store::emitRecordID(unsigned ID, const char *Name, BitstreamWriter &Stream,
                          RecordDataImpl &Record) {
   Record.clear();
   Record.push_back(ID);
   while (*Name)
     Record.push_back(*Name++);
   Stream.EmitRecord(bitc::BLOCKINFO_CODE_SETRECORDNAME, Record);
 }
 
+SymbolKind index::getSymbolKind(indexstore_symbol_kind_t K) {
+  switch ((uint64_t)K) {
+  default:
+  case INDEXSTORE_SYMBOL_KIND_UNKNOWN:
+    return SymbolKind::Unknown;
+  case INDEXSTORE_SYMBOL_KIND_MODULE:
+    return SymbolKind::Module;
+  case INDEXSTORE_SYMBOL_KIND_NAMESPACE:
+    return SymbolKind::Namespace;
+  case INDEXSTORE_SYMBOL_KIND_NAMESPACEALIAS:
+    return SymbolKind::NamespaceAlias;
+  case INDEXSTORE_SYMBOL_KIND_MACRO:
+    return SymbolKind::Macro;
+  case INDEXSTORE_SYMBOL_KIND_ENUM:
+    return SymbolKind::Enum;
+  case INDEXSTORE_SYMBOL_KIND_STRUCT:
+    return SymbolKind::Struct;
+  case INDEXSTORE_SYMBOL_KIND_CLASS:
+    return SymbolKind::Class;
+  case INDEXSTORE_SYMBOL_KIND_PROTOCOL:
+    return SymbolKind::Protocol;
+  case INDEXSTORE_SYMBOL_KIND_EXTENSION:
+    return SymbolKind::Extension;
+  case INDEXSTORE_SYMBOL_KIND_UNION:
+    return SymbolKind::Union;
+  case INDEXSTORE_SYMBOL_KIND_TYPEALIAS:
+    return SymbolKind::TypeAlias;
+  case INDEXSTORE_SYMBOL_KIND_FUNCTION:
+    return SymbolKind::Function;
+  case INDEXSTORE_SYMBOL_KIND_VARIABLE:
+    return SymbolKind::Variable;
+  case INDEXSTORE_SYMBOL_KIND_FIELD:
+    return SymbolKind::Field;
+  case INDEXSTORE_SYMBOL_KIND_ENUMCONSTANT:
+    return SymbolKind::EnumConstant;
+  case INDEXSTORE_SYMBOL_KIND_INSTANCEMETHOD:
+    return SymbolKind::InstanceMethod;
+  case INDEXSTORE_SYMBOL_KIND_CLASSMETHOD:
+    return SymbolKind::ClassMethod;
+  case INDEXSTORE_SYMBOL_KIND_STATICMETHOD:
+    return SymbolKind::StaticMethod;
+  case INDEXSTORE_SYMBOL_KIND_INSTANCEPROPERTY:
+    return SymbolKind::InstanceProperty;
+  case INDEXSTORE_SYMBOL_KIND_CLASSPROPERTY:
+    return SymbolKind::ClassProperty;
+  case INDEXSTORE_SYMBOL_KIND_STATICPROPERTY:
+    return SymbolKind::StaticProperty;
+  case INDEXSTORE_SYMBOL_KIND_CONSTRUCTOR:
+    return SymbolKind::Constructor;
+  case INDEXSTORE_SYMBOL_KIND_DESTRUCTOR:
+    return SymbolKind::Destructor;
+  case INDEXSTORE_SYMBOL_KIND_CONVERSIONFUNCTION:
+    return SymbolKind::ConversionFunction;
+  case INDEXSTORE_SYMBOL_KIND_PARAMETER:
+    return SymbolKind::Parameter;
+  case INDEXSTORE_SYMBOL_KIND_USING:
+    return SymbolKind::Using;
+  }
+}
+
+SymbolSubKind index::getSymbolSubKind(indexstore_symbol_subkind_t K) {
+  switch ((uint64_t)K) {
+  default:
+  case INDEXSTORE_SYMBOL_SUBKIND_NONE:
+    return SymbolSubKind::None;
+  case INDEXSTORE_SYMBOL_SUBKIND_CXXCOPYCONSTRUCTOR:
+    return SymbolSubKind::CXXCopyConstructor;
+  case INDEXSTORE_SYMBOL_SUBKIND_CXXMOVECONSTRUCTOR:
+    return SymbolSubKind::CXXMoveConstructor;
+  case INDEXSTORE_SYMBOL_SUBKIND_ACCESSORGETTER:
+    return SymbolSubKind::AccessorGetter;
+  case INDEXSTORE_SYMBOL_SUBKIND_ACCESSORSETTER:
+    return SymbolSubKind::AccessorSetter;
+  case INDEXSTORE_SYMBOL_SUBKIND_USINGTYPENAME:
+    return SymbolSubKind::UsingTypename;
+  case INDEXSTORE_SYMBOL_SUBKIND_USINGVALUE:
+    return SymbolSubKind::UsingValue;
+  }
+}
+
+SymbolLanguage index::getSymbolLanguage(indexstore_symbol_language_t L) {
+  switch ((uint64_t)L) {
+  default: // FIXME: add an unknown language?
+  case INDEXSTORE_SYMBOL_LANG_C:
+    return SymbolLanguage::C;
+  case INDEXSTORE_SYMBOL_LANG_OBJC:
+    return SymbolLanguage::ObjC;
+  case INDEXSTORE_SYMBOL_LANG_CXX:
+    return SymbolLanguage::CXX;
+  case INDEXSTORE_SYMBOL_LANG_SWIFT:
+    return SymbolLanguage::Swift;
+  }
+}
+
+SymbolPropertySet index::getSymbolProperties(uint64_t Props) {
+  // FIXME: currently these enums must be kept in sync.
+  return (uint64_t)Props;
+}
+
+SymbolRoleSet index::getSymbolRoles(uint64_t Roles) {
+  // FIXME: currently these enums must be kept in sync.
+  return (uint64_t)Roles;
+}
+
 indexstore_symbol_kind_t index::getIndexStoreKind(SymbolKind K) {
   switch (K) {
   case SymbolKind::Unknown:
     return INDEXSTORE_SYMBOL_KIND_UNKNOWN;
   case SymbolKind::Module:
     return INDEXSTORE_SYMBOL_KIND_MODULE;
   case SymbolKind::Namespace:
     return INDEXSTORE_SYMBOL_KIND_NAMESPACE;

lib/Index/IndexRecordHasher.cpp

for (auto &Info : Occurs) {
     }
   }
   return Hash;
 }
 
 hash_code IndexRecordHasher::hash(const Decl *D) {
   assert(D->isCanonicalDecl());
 
-  if (isa<TagDecl>(D) || isa<ObjCContainerDecl>(D)) {
+  if (isa<TagDecl>(D) || isa<ObjCContainerDecl>(D))
     return tryCache(D, D);
-  } else if (auto *NS = dyn_cast<NamespaceDecl>(D)) {
+  if (auto *NS = dyn_cast<NamespaceDecl>(D)) {
     if (NS->isAnonymousNamespace())
       return hash_value(StringRef("@aN"));
     return tryCache(D, D);
-  } else {
-    // There's a balance between caching results and not growing the cache too
-    // much. Measurements showed that avoiding caching all decls is beneficial
-    // particularly when including all of Cocoa.
-    return hashImpl(D);
-  }
-}
+  }
 
 hash_code IndexRecordHasher::hash(QualType NonCanTy) {
   CanQualType CanTy = Ctx.getCanonicalType(NonCanTy);
   return hash(CanTy);
 }
 
 hash_code IndexRecordHasher::hash(CanQualType CT) {
   // Do some hashing without going to the cache, for example we can avoid
if (const PackExpansionType *Expansion = dyn_cast<PackExpansionType>(T)) {
       return COMBINE_HASH('P', hash(asCanon(Expansion->getPattern())));
     }
     if (const RValueReferenceType *RT = dyn_cast<RValueReferenceType>(T)) {
       return COMBINE_HASH('%', hash(asCanon(RT->getPointeeType())));
     }
     if (const ComplexType *CT = dyn_cast<ComplexType>(T)) {
       return COMBINE_HASH('<', hash(asCanon(CT->getElementType())));
     }
-
     break;
   }
 
   return COMBINE_HASH(tryCache(CT.getAsOpaquePtr(), CT));
 }
 
 hash_code IndexRecordHasher::hash(DeclarationName Name) {
   assert(!Name.isEmpty());

lib/Index/IndexRecordReader.cpp

+//===--- IndexRecordReader.cpp - Index record deserialization -------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Index/IndexRecordReader.h"
+#include "BitstreamVisitor.h"
+#include "IndexDataStoreUtils.h"
+#include "clang/Index/IndexDataStoreSymbolUtils.h"
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/Bitcode/BitstreamReader.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/raw_ostream.h"
+
+using namespace clang;
+using namespace clang::index;
+using namespace clang::index::store;
+using namespace llvm;
+
+struct IndexRecordReader::Implementation {
+  BumpPtrAllocator Allocator;
+  std::unique_ptr<MemoryBuffer> Buffer;
+  llvm::BitstreamCursor DeclCursor;
+  llvm::BitstreamCursor OccurCursor;
+  ArrayRef<uint32_t> DeclOffsets;
+  const IndexRecordDecl **Decls;
+
+  void setDeclOffsets(ArrayRef<uint32_t> Offs) {
+    DeclOffsets = Offs;
+    Decls = Allocator.Allocate<const IndexRecordDecl *>(Offs.size());
+    memset(Decls, 0, sizeof(IndexRecordDecl *) * Offs.size());
+  }
+
+  unsigned getNumDecls() const { return DeclOffsets.size(); }
+
+  const IndexRecordDecl *getDeclByID(unsigned DeclID) {
+    if (DeclID == 0)
+      return nullptr;
+    return getDecl(DeclID - 1);
+  }
+
+  const IndexRecordDecl *getDecl(unsigned Index) {
+    assert(Index < getNumDecls());
+    if (const IndexRecordDecl *D = Decls[Index])
+      return D;
+
+    IndexRecordDecl *D = Allocator.Allocate<IndexRecordDecl>();
+    readDecl(Index, *D);
+    Decls[Index] = D;
+    return D;
+  }
+
+  /// Goes through the decls and populates a vector of record decls, based on
+  /// what the given function returns.
+  ///
+  /// The advantage of this function is to allocate memory only for the record
+  /// decls that the caller is interested in.
+  bool searchDecls(llvm::function_ref<DeclSearchCheck> Checker,
+                   llvm::function_ref<void(const IndexRecordDecl *)> Receiver) {
+    for (unsigned I = 0, E = getNumDecls(); I != E; ++I) {
+      if (const IndexRecordDecl *D = Decls[I]) {
+        DeclSearchReturn Ret = Checker(*D);
+        if (Ret.AcceptDecl)
+          Receiver(D);
+        if (!Ret.ContinueSearch)
+          return false;
+        continue;
+      }
+
+      IndexRecordDecl LocalD;
+      readDecl(I, LocalD);
+      DeclSearchReturn Ret = Checker(LocalD);
+      if (Ret.AcceptDecl) {
+        IndexRecordDecl *D = Allocator.Allocate<IndexRecordDecl>();
+        *D = LocalD;
+        Decls[I] = D;
+        Receiver(D);
+      }
+      if (!Ret.ContinueSearch)
+        return false;
+    }
+    return true;
+  }
+
+  void readDecl(unsigned Index, IndexRecordDecl &RecD) {
+    RecordData Record;
+    StringRef Blob;
+    DeclCursor.JumpToBit(DeclOffsets[Index]);
+    unsigned Code = DeclCursor.ReadCode();
+    unsigned RecID = DeclCursor.readRecord(Code, Record, &Blob);
+    assert(RecID == REC_DECLINFO);
+    (void)RecID;
+
+    unsigned I = 0;
+    RecD.DeclID = Index + 1;
+    RecD.SymInfo.Kind =
+        getSymbolKind((indexstore_symbol_kind_t)read(Record, I));
+    RecD.SymInfo.SubKind =
+        getSymbolSubKind((indexstore_symbol_subkind_t)read(Record, I));
+    RecD.SymInfo.Lang =
+        getSymbolLanguage((indexstore_symbol_language_t)read(Record, I));
+    RecD.SymInfo.Properties = getSymbolProperties(read(Record, I));
+    RecD.Roles = getSymbolRoles(read(Record, I));
+    RecD.RelatedRoles = getSymbolRoles(read(Record, I));
+    size_t NameLen = read(Record, I);
+    size_t USRLen = read(Record, I);
+    RecD.Name = Blob.substr(0, NameLen);
+    RecD.USR = Blob.substr(NameLen, USRLen);
+    RecD.CodeGenName = Blob.substr(NameLen + USRLen);
+  }
+
+  /// Reads occurrence data.
+  /// \param DeclsFilter if non-empty indicates the list of decls that we want
+  /// to get occurrences for. If empty then indicates that we want occurrences
+  /// for all decls.
+  /// \param RelatedDeclsFilter Same as \c DeclsFilter but for related decls.
+  /// \returns true if the occurrence info was filled out, false if occurrence
+  /// was ignored.
+  bool readOccurrence(RecordDataImpl &Record, StringRef Blob,
+                      ArrayRef<const IndexRecordDecl *> DeclsFilter,
+                      ArrayRef<const IndexRecordDecl *> RelatedDeclsFilter,
+                      IndexRecordOccurrence &RecOccur) {
+
+    auto isDeclIDContained = [](unsigned DeclID,
+                                ArrayRef<const IndexRecordDecl *> Ds) -> bool {
+      if (Ds.empty())
+        return true; // empty means accept all.
+      auto pred = [DeclID](const IndexRecordDecl *D) {
+        return D->DeclID == DeclID;
+      };
+      return std::find_if(Ds.begin(), Ds.end(), pred) != Ds.end();
+    };
+
+    unsigned I = 0;
+    unsigned DeclID = read(Record, I);
+    if (!isDeclIDContained(DeclID, DeclsFilter))
+      return false;
+
+    if (!RelatedDeclsFilter.empty()) {
+      unsigned RelI = I + 3;
+      unsigned NumRelated = read(Record, RelI);
+      bool FoundRelated = false;
+      while (NumRelated--) {
+        ++RelI; // roles;
+        unsigned RelDID = read(Record, RelI);
+        if (isDeclIDContained(RelDID, RelatedDeclsFilter)) {
+          FoundRelated = true;
+          break;
+        }
+      }
+      if (!FoundRelated)
+        return false;
+    }
+
+    RecOccur.Dcl = getDeclByID(DeclID);
+    RecOccur.Roles = getSymbolRoles(read(Record, I));
+    RecOccur.Line = read(Record, I);
+    RecOccur.Column = read(Record, I);
+
+    unsigned NumRelated = read(Record, I);
+    while (NumRelated--) {
+      SymbolRoleSet RelRoles = getSymbolRoles(read(Record, I));
+      const IndexRecordDecl *RelD = getDeclByID(read(Record, I));
+      RecOccur.Relations.emplace_back(RelRoles, RelD);
+    }
+
+    return true;
+  }
+
+  bool foreachDecl(bool NoCache,
+                   function_ref<bool(const IndexRecordDecl *)> Receiver) {
+    for (unsigned I = 0, E = getNumDecls(); I != E; ++I) {
+      if (const IndexRecordDecl *D = Decls[I]) {
+        if (!Receiver(D))
+          return false;
+        continue;
+      }
+
+      if (NoCache) {
+        IndexRecordDecl LocalD;
+        readDecl(I, LocalD);
+        if (!Receiver(&LocalD))
+          return false;
+      } else {
+        if (!Receiver(getDecl(I)))
+          return false;
+      }
+    }
+    return true;
+  }
+
+  bool foreachOccurrence(
+      ArrayRef<const IndexRecordDecl *> DeclsFilter,
+      ArrayRef<const IndexRecordDecl *> RelatedDeclsFilter,
+      function_ref<bool(const IndexRecordOccurrence &)> Receiver) {
+
+    class OccurBitVisitor : public BitstreamVisitor<OccurBitVisitor> {
+      IndexRecordReader::Implementation &Reader;
+      ArrayRef<const IndexRecordDecl *> DeclsFilter;
+      ArrayRef<const IndexRecordDecl *> RelatedDeclsFilter;
+      function_ref<bool(const IndexRecordOccurrence &)> Receiver;
+
+    public:
+      OccurBitVisitor(
+          llvm::BitstreamCursor &Stream,
+          IndexRecordReader::Implementation &Reader,
+          ArrayRef<const IndexRecordDecl *> DeclsFilter,
+          ArrayRef<const IndexRecordDecl *> RelatedDeclsFilter,
+          function_ref<bool(const IndexRecordOccurrence &)> Receiver)
+          : BitstreamVisitor(Stream), Reader(Reader), DeclsFilter(DeclsFilter),
+            RelatedDeclsFilter(RelatedDeclsFilter),
+            Receiver(std::move(Receiver)) {}
+
+      StreamVisit visitRecord(unsigned BlockID, unsigned RecID,
+                              RecordDataImpl &Record, StringRef Blob) {
+        assert(RecID == REC_DECLOCCURRENCE);
+        IndexRecordOccurrence RecOccur;
+        if (Reader.readOccurrence(Record, Blob, DeclsFilter, RelatedDeclsFilter,
+                                  RecOccur))
+          if (!Receiver(RecOccur))
+            return StreamVisit::Abort;
+        return StreamVisit::Continue;
+      }
+    };
+
+    SavedStreamPosition SavedPosition(OccurCursor);
+    OccurBitVisitor Visitor(OccurCursor, *this, DeclsFilter, RelatedDeclsFilter,
+                            Receiver);
+    std::string Error;
+    return Visitor.visit(Error);
+  }
+
+  bool foreachOccurrenceInLineRange(
+      unsigned lineStart, unsigned lineCount,
+      llvm::function_ref<bool(const IndexRecordOccurrence &)> receiver) {
+    // FIXME: Use binary search and make this more efficient.
+    unsigned lineEnd = lineStart + lineCount;
+    return foreachOccurrence(None, None,
+                             [&](const IndexRecordOccurrence &occur) -> bool {
+                               if (occur.Line > lineEnd)
+                                 return false; // we're done.
+                               if (occur.Line >= lineStart) {
+                                 if (!receiver(occur))
+                                   return false;
+                               }
+                               return true;
+                             });
+  }
+
+  static uint64_t read(RecordDataImpl &Record, unsigned &I) {
+    return Record[I++];
+  }
+};
+
+namespace {
+
+class IndexBitstreamVisitor : public BitstreamVisitor<IndexBitstreamVisitor> {
+  IndexRecordReader::Implementation &Reader;
+
+public:
+  IndexBitstreamVisitor(llvm::BitstreamCursor &Stream,
+                        IndexRecordReader::Implementation &Reader)
+      : BitstreamVisitor(Stream), Reader(Reader) {}
+
+  StreamVisit visitBlock(unsigned ID) {
+    switch ((RecordBitBlock)ID) {
+    case REC_VERSION_BLOCK_ID:
+    case REC_DECLOFFSETS_BLOCK_ID:
+      return StreamVisit::Continue;
+
+    case REC_DECLS_BLOCK_ID:
+      Reader.DeclCursor = Stream;
+      if (Reader.DeclCursor.EnterSubBlock(ID)) {
+        *Error = "malformed block record";
+        return StreamVisit::Abort;
+      }
+      readBlockAbbrevs(Reader.DeclCursor);
+      return StreamVisit::Skip;
+
+    case REC_DECLOCCURRENCES_BLOCK_ID:
+      Reader.OccurCursor = Stream;
+      if (Reader.OccurCursor.EnterSubBlock(ID)) {
+        *Error = "malformed block record";
+        return StreamVisit::Abort;
+      }
+      readBlockAbbrevs(Reader.OccurCursor);
+      return StreamVisit::Skip;
+    }
+
+    // Some newly introduced block in a minor version update that we cannot
+    // handle.
+    return StreamVisit::Skip;
+  }
+
+  StreamVisit visitRecord(unsigned BlockID, unsigned RecID,
+                          RecordDataImpl &Record, StringRef Blob) {
+    switch (BlockID) {
+    case REC_VERSION_BLOCK_ID: {
+      unsigned StoreFormatVersion = Record[0];
+      if (StoreFormatVersion != STORE_FORMAT_VERSION) {
+        llvm::raw_string_ostream OS(*Error);
+        OS << "Store format version mismatch: " << StoreFormatVersion;
+        OS << " , expected: " << STORE_FORMAT_VERSION;
+        return StreamVisit::Abort;
+      }
+      break;
+    }
+    case REC_DECLOFFSETS_BLOCK_ID:
+      assert(RecID == REC_DECLOFFSETS);
+      Reader.setDeclOffsets(
+          makeArrayRef((const uint32_t *)Blob.data(), Record[0]));
+      break;
+
+    case REC_DECLS_BLOCK_ID:
+    case REC_DECLOCCURRENCES_BLOCK_ID:
+      llvm_unreachable("shouldn't visit this block'");
+    }
+    return StreamVisit::Continue;
+  }
+};
+
+} // anonymous namespace
+
+std::unique_ptr<IndexRecordReader> IndexRecordReader::createWithRecordFilename(
+    StringRef RecordFilename, StringRef StorePath, std::string &Error) {
+  SmallString<128> PathBuf = StorePath;
+  appendRecordSubDir(PathBuf);
+  appendInteriorRecordPath(RecordFilename, PathBuf);
+  return createWithFilePath(PathBuf.str(), Error);
+}
+
+std::unique_ptr<IndexRecordReader>
+IndexRecordReader::createWithFilePath(StringRef FilePath, std::string &Error) {
+  auto ErrOrBuf = MemoryBuffer::getFile(FilePath, /*FileSize=*/-1,
+                                        /*RequiresNullTerminator=*/false);
+  if (!ErrOrBuf) {
+    raw_string_ostream(Error) << "failed opening index record '" << FilePath
+                              << "': " << ErrOrBuf.getError().message();
+    return nullptr;
+  }
+  return createWithBuffer(std::move(*ErrOrBuf), Error);
+}
+
+std::unique_ptr<IndexRecordReader>
+IndexRecordReader::createWithBuffer(std::unique_ptr<llvm::MemoryBuffer> Buffer,
+                                    std::string &Error) {
+
+  std::unique_ptr<IndexRecordReader> Reader;
+  Reader.reset(new IndexRecordReader());
+  auto &Impl = Reader->Impl;
+  Impl.Buffer = std::move(Buffer);
+  llvm::BitstreamCursor Stream(*Impl.Buffer);
+
+  // Sniff for the signature.
+  if (Stream.Read(8) != 'I' || Stream.Read(8) != 'D' || Stream.Read(8) != 'X' ||
+      Stream.Read(8) != 'R') {
+    Error = "not a serialized index record file";
+    return nullptr;
+  }
+
+  IndexBitstreamVisitor BitVisitor(Stream, Impl);
+  if (!BitVisitor.visit(Error))
+    return nullptr;
+
+  return Reader;
+}
+
+IndexRecordReader::IndexRecordReader() : Impl(*new Implementation()) {}
+
+IndexRecordReader::~IndexRecordReader() { delete &Impl; }
+
+bool IndexRecordReader::searchDecls(
+    llvm::function_ref<DeclSearchCheck> Checker,
+    llvm::function_ref<void(const IndexRecordDecl *)> Receiver) {
+  return Impl.searchDecls(std::move(Checker), std::move(Receiver));
+}
+
+bool IndexRecordReader::foreachDecl(
+    bool NoCache, function_ref<bool(const IndexRecordDecl *)> Receiver) {
+  return Impl.foreachDecl(NoCache, std::move(Receiver));
+}
+
+bool IndexRecordReader::foreachOccurrence(
+    ArrayRef<const IndexRecordDecl *> DeclsFilter,
+    ArrayRef<const IndexRecordDecl *> RelatedDeclsFilter,
+    function_ref<bool(const IndexRecordOccurrence &)> Receiver) {
+  return Impl.foreachOccurrence(DeclsFilter, RelatedDeclsFilter,
+                                std::move(Receiver));
+}
+
+bool IndexRecordReader::foreachOccurrence(
+    llvm::function_ref<bool(const IndexRecordOccurrence &)> Receiver) {
+  return foreachOccurrence(None, None, std::move(Receiver));
+}
+
+bool IndexRecordReader::foreachOccurrenceInLineRange(
+    unsigned lineStart, unsigned lineCount,
+    llvm::function_ref<bool(const IndexRecordOccurrence &)> Receiver) {
+  return Impl.foreachOccurrenceInLineRange(lineStart, lineCount, Receiver);
+}

lib/Index/IndexUnitReader.cpp

+//===--- IndexUnitReader.cpp - Index unit deserialization -----------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Index/IndexUnitReader.h"
+#include "BitstreamVisitor.h"
+#include "IndexDataStoreUtils.h"
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/Optional.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Bitcode/BitstreamReader.h"
+#include "llvm/Support/Chrono.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include <unistd.h>
+
+using namespace clang;
+using namespace clang::index;
+using namespace clang::index::store;
+using namespace llvm;
+
+namespace {
+
+typedef function_ref<bool(const IndexUnitReader::DependencyInfo &)>
+    DependencyReceiver;
+typedef function_ref<bool(const IndexUnitReader::IncludeInfo &)>
+    IncludeReceiver;
+
+class IndexUnitReaderImpl {
+  sys::TimePoint<> ModTime;
+  std::unique_ptr<MemoryBuffer> MemBuf;
+
+public:
+  StringRef ProviderIdentifier;
+  StringRef ProviderVersion;
+  llvm::BitstreamCursor DependCursor;
+  llvm::BitstreamCursor IncludeCursor;
+  bool IsSystemUnit;
+  bool IsModuleUnit;
+  bool IsDebugCompilation;
+  StringRef WorkingDir;
+  StringRef OutputFile;
+  StringRef SysrootPath;
+  StringRef ModuleName;
+  SmallString<128> MainFilePath;
+  StringRef Target;
+  std::vector<FileBitPath> Paths;
+  StringRef PathsBuffer;
+
+  struct ModuleInfo {
+    unsigned NameOffset;
+    unsigned NameSize;
+  };
+  std::vector<ModuleInfo> Modules;
+  StringRef ModuleNamesBuffer;
+
+  bool init(std::unique_ptr<MemoryBuffer> Buf, sys::TimePoint<> ModTime,
+            std::string &Error);
+
+  StringRef getProviderIdentifier() const { return ProviderIdentifier; }
+  StringRef getProviderVersion() const { return ProviderVersion; }
+
+  sys::TimePoint<> getModificationTime() const { return ModTime; }
+  StringRef getWorkingDirectory() const { return WorkingDir; }
+  StringRef getOutputFile() const { return OutputFile; }
+  StringRef getSysrootPath() const { return SysrootPath; }
+  StringRef getTarget() const { return Target; }
+
+  StringRef getModuleName() const { return ModuleName; }
+  StringRef getMainFilePath() const { return MainFilePath.str(); }
+  bool hasMainFile() const { return !MainFilePath.empty(); }
+  bool isSystemUnit() const { return IsSystemUnit; }
+  bool isModuleUnit() const { return IsModuleUnit; }
+  bool isDebugCompilation() const { return IsDebugCompilation; }
+
+  /// Unit dependencies are provided ahead of record ones, record ones
+  /// ahead of the file ones.
+  bool foreachDependency(DependencyReceiver Receiver);
+
+  bool foreachInclude(IncludeReceiver Receiver);
+
+  StringRef getPathFromBuffer(size_t Offset, size_t Size) {
+    return PathsBuffer.substr(Offset, Size);
+  }
+
+  void constructFilePath(SmallVectorImpl<char> &Path, int PathIndex);
+
+  StringRef getModuleName(int ModuleIndex);
+};
+
+class IndexUnitBitstreamVisitor
+    : public BitstreamVisitor<IndexUnitBitstreamVisitor> {
+  IndexUnitReaderImpl &Reader;
+  size_t WorkDirOffset;
+  size_t WorkDirSize;
+  size_t OutputFileOffset;
+  size_t OutputFileSize;
+  size_t SysrootOffset;
+  size_t SysrootSize;
+  int MainPathIndex;
+
+public:
+  IndexUnitBitstreamVisitor(llvm::BitstreamCursor &Stream,
+                            IndexUnitReaderImpl &Reader)
+      : BitstreamVisitor(Stream), Reader(Reader) {}
+
+  StreamVisit visitBlock(unsigned ID) {
+    switch ((UnitBitBlock)ID) {
+    case UNIT_VERSION_BLOCK_ID:
+    case UNIT_INFO_BLOCK_ID:
+    case UNIT_PATHS_BLOCK_ID:
+    case UNIT_MODULES_BLOCK_ID:
+      return StreamVisit::Continue;
+
+    case UNIT_DEPENDENCIES_BLOCK_ID:
+      Reader.DependCursor = Stream;
+      if (Reader.DependCursor.EnterSubBlock(ID)) {
+        *Error = "malformed unit dependencies block record";
+        return StreamVisit::Abort;
+      }
+      readBlockAbbrevs(Reader.DependCursor);
+      return StreamVisit::Skip;
+    case UNIT_INCLUDES_BLOCK_ID:
+      Reader.IncludeCursor = Stream;
+      if (Reader.IncludeCursor.EnterSubBlock(ID)) {
+        *Error = "malformed unit includes block record";
+        return StreamVisit::Abort;
+      }
+      readBlockAbbrevs(Reader.IncludeCursor);
+      return StreamVisit::Skip;
+    }
+
+    // Some newly introduced block in a minor version update that we cannot
+    // handle.
+    return StreamVisit::Skip;
+  }
+
+  StreamVisit visitRecord(unsigned BlockID, unsigned RecID,
+                          RecordDataImpl &Record, StringRef Blob) {
+    switch (BlockID) {
+    case UNIT_VERSION_BLOCK_ID: {
+      unsigned StoreFormatVersion = Record[0];
+      if (StoreFormatVersion != STORE_FORMAT_VERSION) {
+        llvm::raw_string_ostream OS(*Error);
+        OS << "Store format version mismatch: " << StoreFormatVersion;
+        OS << " , expected: " << STORE_FORMAT_VERSION;
+        return StreamVisit::Abort;
+      }
+      break;
+    }
+
+    case UNIT_INFO_BLOCK_ID: {
+      assert(RecID == UNIT_INFO);
+      unsigned I = 0;
+      Reader.IsSystemUnit = Record[I++];
+
+      // Save these to lookup them up after we get the paths buffer.
+      WorkDirOffset = Record[I++];
+      WorkDirSize = Record[I++];
+      OutputFileOffset = Record[I++];
+      OutputFileSize = Record[I++];
+      SysrootOffset = Record[I++];
+      SysrootSize = Record[I++];
+      MainPathIndex = (int)Record[I++] - 1;
+      Reader.IsDebugCompilation = Record[I++];
+      Reader.IsModuleUnit = Record[I++];
+
+      size_t moduleNameSize = Record[I++];
+      size_t providerIdentifierSize = Record[I++];
+      size_t providerVersionSize = Record[I++];
+      I++; // Reserved for ProviderDataVersion.
+      Reader.ModuleName = Blob.substr(0, moduleNameSize);
+      Blob = Blob.drop_front(moduleNameSize);
+      Reader.ProviderIdentifier = Blob.substr(0, providerIdentifierSize);
+      Blob = Blob.drop_front(providerIdentifierSize);
+      Reader.ProviderVersion = Blob.substr(0, providerVersionSize);
+      Reader.Target = Blob.drop_front(providerVersionSize);
+      break;
+    }
+
+    case UNIT_PATHS_BLOCK_ID:
+      switch (RecID) {
+      case UNIT_PATH: {
+        unsigned I = 0;
+        UnitFilePathPrefixKind Kind = (UnitFilePathPrefixKind)Record[I++];
+        size_t DirOffset = Record[I++];
+        size_t DirSize = Record[I++];
+        size_t FilenameOffset = Record[I++];
+        size_t FilenameSize = Record[I++];
+
+        Reader.Paths.emplace_back(
+            Kind, BitPathComponent(DirOffset, DirSize),
+            BitPathComponent(FilenameOffset, FilenameSize));
+      } break;
+      case UNIT_PATH_BUFFER:
+        Reader.PathsBuffer = Blob;
+        Reader.WorkingDir =
+            Reader.getPathFromBuffer(WorkDirOffset, WorkDirSize);
+        Reader.OutputFile =
+            Reader.getPathFromBuffer(OutputFileOffset, OutputFileSize);
+        Reader.SysrootPath =
+            Reader.getPathFromBuffer(SysrootOffset, SysrootSize);
+
+        // now we can populate the main file's path
+        Reader.constructFilePath(Reader.MainFilePath, MainPathIndex);
+        break;
+      default:
+        llvm_unreachable("shouldn't visit this record");
+      }
+      break;
+
+    case UNIT_MODULES_BLOCK_ID:
+      switch (RecID) {
+      case UNIT_MODULE: {
+        unsigned I = 0;
+        unsigned NameOffset = Record[I++];
+        unsigned NameSize = Record[I++];
+        Reader.Modules.push_back({NameOffset, NameSize});
+      } break;
+      case UNIT_MODULE_BUFFER:
+        Reader.ModuleNamesBuffer = Blob;
+        break;
+      default:
+        llvm_unreachable("shouldn't visit this record");
+      }
+      break;
+
+    case UNIT_DEPENDENCIES_BLOCK_ID:
+    case UNIT_INCLUDES_BLOCK_ID:
+      llvm_unreachable("shouldn't visit this block'");
+    }
+    return StreamVisit::Continue;
+  }
+};
+
+typedef std::function<bool(RecordDataImpl &Record, StringRef Blob)>
+    BlockVisitorCallback;
+
+class IndexUnitBlockBitstreamVisitor
+    : public BitstreamVisitor<IndexUnitBlockBitstreamVisitor> {
+  unsigned RecID;
+  BlockVisitorCallback Visit;
+
+public:
+  IndexUnitBlockBitstreamVisitor(unsigned RecID,
+                                 llvm::BitstreamCursor &BlockStream,
+                                 BlockVisitorCallback Visit)
+      : BitstreamVisitor(BlockStream), RecID(RecID), Visit(std::move(Visit)) {}
+
+  StreamVisit visitRecord(unsigned BlockID, unsigned RecID,
+                          RecordDataImpl &Record, StringRef Blob) {
+    if (RecID != this->RecID)
+      llvm_unreachable("shouldn't be called with this RecID");
+
+    if (Visit(Record, Blob))
+      return StreamVisit::Continue;
+    return StreamVisit::Abort;
+  }
+};
+
+} // anonymous namespace
+
+bool IndexUnitReaderImpl::init(std::unique_ptr<MemoryBuffer> Buf,
+                               sys::TimePoint<> ModTime, std::string &Error) {
+  this->ModTime = ModTime;
+  this->MemBuf = std::move(Buf);
+  llvm::BitstreamCursor Stream(*MemBuf);
+
+  // Sniff for the signature.
+  if (Stream.Read(8) != 'I' || Stream.Read(8) != 'D' || Stream.Read(8) != 'X' ||
+      Stream.Read(8) != 'U') {
+    Error = "not a serialized index unit file";
+    return true;
+  }
+
+  IndexUnitBitstreamVisitor BitVisitor(Stream, *this);
+  return !BitVisitor.visit(Error);
+}
+
+/// Unit dependencies are provided ahead of record ones, record ones
+/// ahead of the file ones.
+bool IndexUnitReaderImpl::foreachDependency(DependencyReceiver Receiver) {
+  store::SavedStreamPosition SavedDepPosition(DependCursor);
+  IndexUnitBlockBitstreamVisitor Visitor(
+      UNIT_DEPENDENCY, DependCursor,
+      [&](RecordDataImpl &Record, StringRef Blob) {
+        unsigned I = 0;
+        UnitDependencyKind UnitDepKind = (UnitDependencyKind)Record[I++];
+        bool IsSystem = Record[I++];
+        int PathIndex = (int)Record[I++] - 1;
+        int ModuleIndex = (int)Record[I++] - 1;
+        time_t ModTime = (time_t)Record[I++];
+        size_t FileSize = Record[I++];
+        StringRef Name = Blob;
+
+        IndexUnitReader::DependencyKind DepKind;
+        switch (UnitDepKind) {
+        case UNIT_DEPEND_KIND_UNIT:
+          DepKind = IndexUnitReader::DependencyKind::Unit;
+          break;
+        case UNIT_DEPEND_KIND_RECORD:
+          DepKind = IndexUnitReader::DependencyKind::Record;
+          break;
+        case UNIT_DEPEND_KIND_FILE:
+          DepKind = IndexUnitReader::DependencyKind::File;
+          break;
+        }
+
+        SmallString<512> PathBuf;
+        this->constructFilePath(PathBuf, PathIndex);
+        StringRef ModuleName = this->getModuleName(ModuleIndex);
+
+        return Receiver(IndexUnitReader::DependencyInfo{
+            DepKind, IsSystem, Name, PathBuf.str(), ModuleName, FileSize,
+            ModTime});
+      });
+
+  std::string Error;
+  return Visitor.visit(Error);
+}
+
+bool IndexUnitReaderImpl::foreachInclude(IncludeReceiver Receiver) {
+  store::SavedStreamPosition SavedIncPosition(IncludeCursor);
+  IndexUnitBlockBitstreamVisitor Visitor(
+      UNIT_INCLUDE, IncludeCursor, [&](RecordDataImpl &Record, StringRef Blob) {
+        unsigned I = 0;
+        int SourcePathIndex = (int)Record[I++] - 1;
+        unsigned Line = Record[I++];
+        int TargetPathIndex = (int)Record[I++] - 1;
+
+        SmallString<512> SourceBuf, TargetBuf;
+        this->constructFilePath(SourceBuf, SourcePathIndex);
+        this->constructFilePath(TargetBuf, TargetPathIndex);
+        return Receiver(IndexUnitReader::IncludeInfo{SourceBuf.str(), Line,
+                                                     TargetBuf.str()});
+      });
+
+  std::string Error;
+  return Visitor.visit(Error);
+}
+
+void IndexUnitReaderImpl::constructFilePath(SmallVectorImpl<char> &PathBuf,
+                                            int PathIndex) {
+  if (PathIndex < 0)
+    return;
+  FileBitPath &Path = Paths[PathIndex];
+  StringRef Prefix;
+  switch (Path.PrefixKind) {
+  case UNIT_PATH_PREFIX_NONE:
+    break;
+  case UNIT_PATH_PREFIX_WORKDIR:
+    Prefix = getWorkingDirectory();
+    break;
+  case UNIT_PATH_PREFIX_SYSROOT:
+    Prefix = getSysrootPath();
+    break;
+  }
+  PathBuf.append(Prefix.begin(), Prefix.end());
+  sys::path::append(
+      PathBuf, getPathFromBuffer(Path.Dir.Offset, Path.Dir.Size),
+      getPathFromBuffer(Path.Filename.Offset, Path.Filename.Size));
+}
+
+StringRef IndexUnitReaderImpl::getModuleName(int ModuleIndex) {
+  if (ModuleIndex < 0)
+    return StringRef();
+  auto &ModInfo = Modules[ModuleIndex];
+  return StringRef(ModuleNamesBuffer.data() + ModInfo.NameOffset,
+                   ModInfo.NameSize);
+}
+
+//===----------------------------------------------------------------------===//
+// IndexUnitReader
+//===----------------------------------------------------------------------===//
+
+std::unique_ptr<IndexUnitReader> IndexUnitReader::createWithUnitFilename(
+    StringRef UnitFilename, StringRef StorePath, std::string &Error) {
+  SmallString<128> PathBuf = StorePath;
+  appendUnitSubDir(PathBuf);
+  sys::path::append(PathBuf, UnitFilename);
+  return createWithFilePath(PathBuf.str(), Error);
+}
+
+std::unique_ptr<IndexUnitReader>
+IndexUnitReader::createWithFilePath(StringRef FilePath, std::string &Error) {
+  int FD;
+  std::error_code EC = sys::fs::openFileForRead(FilePath, FD);
+  if (EC) {
+    raw_string_ostream(Error)
+        << "Failed opening '" << FilePath << "': " << EC.message();
+    return nullptr;
+  }
+
+  assert(FD != -1);
+  struct AutoFDClose {
+    int FD;
+    AutoFDClose(int FD) : FD(FD) {}
+    ~AutoFDClose() { ::close(FD); }
+  } AutoFDClose(FD);
+
+  sys::fs::file_status FileStat;
+  EC = sys::fs::status(FD, FileStat);
+  if (EC) {
+    Error = EC.message();
+    return nullptr;
+  }
+
+  auto ErrOrBuf = MemoryBuffer::getOpenFile(FD, FilePath, /*FileSize=*/-1,
+                                            /*RequiresNullTerminator=*/false);
+  if (!ErrOrBuf) {
+    raw_string_ostream(Error) << "Failed opening '" << FilePath
+                              << "': " << ErrOrBuf.getError().message();
+    return nullptr;
+  }
+
+  std::unique_ptr<IndexUnitReaderImpl> Impl(new IndexUnitReaderImpl());
+  bool Err = Impl->init(std::move(*ErrOrBuf),
+                        FileStat.getLastModificationTime(), Error);
+  if (Err)
+    return nullptr;
+
+  std::unique_ptr<IndexUnitReader> Reader;
+  Reader.reset(new IndexUnitReader(Impl.release()));
+  return Reader;
+}
+
+Optional<sys::TimePoint<>> IndexUnitReader::getModificationTimeForUnit(
+    StringRef UnitFilename, StringRef StorePath, std::string &Error) {
+  SmallString<128> PathBuf = StorePath;
+  appendUnitSubDir(PathBuf);
+  sys::path::append(PathBuf, UnitFilename);
+
+  sys::fs::file_status FileStat;
+  std::error_code EC = sys::fs::status(PathBuf.str(), FileStat);
+  if (EC) {
+    Error = EC.message();
+    return None;
+  }
+  return FileStat.getLastModificationTime();
+}
+
+#define IMPL static_cast<IndexUnitReaderImpl *>(Impl)
+
+IndexUnitReader::~IndexUnitReader() { delete IMPL; }
+
+StringRef IndexUnitReader::getProviderIdentifier() const {
+  return IMPL->getProviderIdentifier();
+}
+
+StringRef IndexUnitReader::getProviderVersion() const {
+  return IMPL->getProviderVersion();
+}
+
+llvm::sys::TimePoint<> IndexUnitReader::getModificationTime() const {
+  return IMPL->getModificationTime();
+}
+
+StringRef IndexUnitReader::getWorkingDirectory() const {
+  return IMPL->getWorkingDirectory();
+}
+
+StringRef IndexUnitReader::getOutputFile() const {
+  return IMPL->getOutputFile();
+}
+
+StringRef IndexUnitReader::getSysrootPath() const {
+  return IMPL->getSysrootPath();
+}
+
+StringRef IndexUnitReader::getMainFilePath() const {
+  return IMPL->getMainFilePath();
+}
+
+StringRef IndexUnitReader::getModuleName() const {
+  return IMPL->getModuleName();
+}
+
+StringRef IndexUnitReader::getTarget() const { return IMPL->getTarget(); }
+
+bool IndexUnitReader::hasMainFile() const { return IMPL->hasMainFile(); }
+
+bool IndexUnitReader::isSystemUnit() const { return IMPL->isSystemUnit(); }
+
+bool IndexUnitReader::isModuleUnit() const { return IMPL->isModuleUnit(); }
+
+bool IndexUnitReader::isDebugCompilation() const {
+  return IMPL->isDebugCompilation();
+}
+
+/// \c Index is the index in the \c getDependencies array.
+/// Unit dependencies are provided ahead of record ones.
+bool IndexUnitReader::foreachDependency(DependencyReceiver Receiver) {
+  return IMPL->foreachDependency(std::move(Receiver));
+}
+
+bool IndexUnitReader::foreachInclude(IncludeReceiver Receiver) {
+  return IMPL->foreachInclude(std::move(Receiver));
+}

test/Index/Store/Inputs/head.h

+
+extern void test1_func(void);
+extern void test2_func(void);

test/Index/Store/Inputs/json.c.json

+{
+  "files": [
+    "/test1.o",
+    "/Inputs/test1.c",
+    "/Inputs/head.h",
+    "/test2.o",
+    "/Inputs/test2.c",
+    "/test3.o",
+    "/Inputs/test3.cpp"
+  ],
+  "symbols": [
+    {
+      "kind": "function",
+      "lang": "C",
+      "usr": "c:@F@test1_func",
+      "name": "test1_func",
+      "codegen": "_test1_func",
+      "roles": "Decl,Def"
+    },
+    {
+      "kind": "function",
+      "lang": "C",
+      "usr": "c:@F@test2_func",
+      "name": "test2_func",
+      "codegen": "_test2_func",
+      "roles": "Decl,Def"
+    },
+    {
+      "kind": "class",
+      "lang": "C++",
+      "usr": "c:@S@Base",
+      "name": "Base",
+      "roles": "Def,Ref,RelBase,RelCont"
+    },
+    {
+      "kind": "class",
+      "lang": "C++",
+      "usr": "c:@S@Sub",
+      "name": "Sub",
+      "roles": "Def",
+      "rel-roles": "RelBase,RelCont"
+    }
+  ],
+  "records": [
+    {
+      "occurrences": [
+        {
+          "symbol": 0,
+          "line": 3,
+          "col": 6,
+          "roles": "Def"
+        }
+      ]
+    },
+    {
+      "occurrences": [
+        {
+          "symbol": 0,
+          "line": 2,
+          "col": 13,
+          "roles": "Decl"
+        },
+        {
+          "symbol": 1,
+          "line": 3,
+          "col": 13,
+          "roles": "Decl"
+        }
+      ]
+    },
+    {
+      "occurrences": [
+        {
+          "symbol": 1,
+          "line": 3,
+          "col": 6,
+          "roles": "Def"
+        }
+      ]
+    },
+    {
+      "occurrences": [
+        {
+          "symbol": 2,
+          "line": 1,
+          "col": 7,
+          "roles": "Def"
+        },
+        {
+          "symbol": 3,
+          "line": 2,
+          "col": 7,
+          "roles": "Def"
+        },
+        {
+          "symbol": 2,
+          "line": 2,
+          "col": 20,
+          "roles": "Ref,RelBase,RelCont",
+          "relations": [
+            {
+              "symbol": 3,
+              "rel-roles": "RelBase,RelCont"
+            }
+          ]
+
+        }
+      ]
+    }
+  ],
+  "units": [
+    {
+      "triple": "x86_64-apple-macosx10.7.0",
+      "out-file": 0,
+      "sources": [
+        {
+          "file": 1,
+          "records": [0]
+        },
+        {
+          "file": 2,
+          "records": [1]
+        }
+      ]
+    },
+    {
+      "triple": "x86_64-apple-macosx10.7.0",
+      "out-file": 3,
+      "sources": [
+        {
+          "file": 4,
+          "records": [2]
+        },
+        {
+          "file": 2,
+          "records": [1]
+        }
+      ]
+    },
+    {
+      "triple": "x86_64-apple-macosx10.7.0",
+      "out-file": 5,
+      "sources": [
+        {
+          "file": 6,
+          "records": [3]
+        }
+      ]
+    }
+  ]
+}

test/Index/Store/Inputs/module/ModDep.h

+#include "ModTop.h"
+
+void ModDep_func(ModTopStruct s);

test/Index/Store/Inputs/module/ModSystem.h

+
+typedef struct {
+} ModSystemStruct;
+
+void ModSystem_func(void);

test/Index/Store/Inputs/module/ModTop.h

+
+typedef struct {
+} ModTopStruct;
+
+void ModTop_func(void);

test/Index/Store/Inputs/module/ModTopSub1.h

+void ModTopSub1_func(void);

test/Index/Store/Inputs/module/ModTopSub2.h

+// This header has no symbols, intended to show up as file dependency.

test/Index/Store/Inputs/module/module.modulemap

+module ModTop {
+  header "ModTop.h"
+  export *
+  module Sub1 {
+    header "ModTopSub1.h"
+  }
+  module Sub2 {
+    header "ModTopSub2.h"
+  }
+}
+module ModDep { header "ModDep.h" export * }
+module ModSystem [system] { header "ModSystem.h" export * }

test/Index/Store/Inputs/overlay.yaml

+{
+  'version': 0,
+  'roots': [{ 'type': 'file', 'name': 'OUT_DIR/using-overlay.h',
+             'external-contents': 'INPUT_DIR/using-overlay.h'
+           }]
+}

test/Index/Store/Inputs/print-unit.h

+#include "head.h"
+#include "using-overlay.h"

test/Index/Store/Inputs/sys/another.h

+
+extern void sys_another_func(void);

test/Index/Store/Inputs/sys/syshead.h

+
+#include "another.h"
+
+extern void sys_test1_func(void);

test/Index/Store/Inputs/test1.c

+#include "head.h"
+
+void test1_func(void) {}

test/Index/Store/Inputs/test2.c

+#include "head.h"
+
+void test2_func(void) {}

test/Index/Store/Inputs/test3.cpp

+class Base {};
+class Sub : public Base {};

test/Index/Store/Inputs/using-overlay.h

+void using_overlay(void);

test/Index/Store/external-source-symbol-hash.m

+// RUN: rm -rf %t.idx
+// RUN: %clang_cc1 %s -index-store-path %t.idx -D USE_EXTERNAL
+// RUN: c-index-test core -print-record %t.idx | FileCheck %s
+// RUN: %clang_cc1 %s -index-store-path %t.idx
+// RUN: find %t.idx/*/records -name "external-source-symbol-hash*" | count 2
+
+#ifdef USE_EXTERNAL
+#define EXT_DECL(mod_name) __attribute__((external_source_symbol(language = "Swift", defined_in = mod_name)))
+#else
+#define EXT_DECL(mod_name)
+#endif
+
+#define NS_ENUM(_name, _type) \
+  enum _name : _type _name;   \
+  enum _name : _type
+
+// Forward declarations should pick up the attribute from later decls
+@protocol P1;
+// CHECK: [[@LINE-1]]:11 | protocol/Swift | c:@M@some_module@objc(pl)P1 | Ref | rel: 0
+@class I2;
+// CHECK: [[@LINE-1]]:8 | class/Swift | c:@M@other_module@objc(cs)I2 | Ref | rel: 0
+enum E3 : int;
+// CHECK: [[@LINE-1]]:6 | enum/Swift | c:@M@third_module@E@E3 | Decl | rel: 0
+
+void test(id<P1> first, I2 *second, enum E3 third) {}
+// CHECK: [[@LINE-1]]:14 | protocol/Swift | c:@M@some_module@objc(pl)P1 | Ref,RelCont | rel: 1
+// CHECK: [[@LINE-2]]:25 | class/Swift | c:@M@other_module@objc(cs)I2 | Ref,RelCont | rel: 1
+// CHECK: [[@LINE-3]]:42 | enum/Swift | c:@M@third_module@E@E3 | Ref,RelCont | rel: 1
+
+EXT_DECL("some_module")
+@protocol P1
+// CHECK: [[@LINE-1]]:11 | protocol/Swift | c:@M@some_module@objc(pl)P1 | Decl | rel: 0
+- (void)method;
+// CHECK: [[@LINE-1]]:9 | instance-method/Swift | c:@M@some_module@objc(pl)P1(im)method | Decl,Dyn,RelChild | rel: 1
+@end
+
+EXT_DECL("other_module")
+@interface I2
+// CHECK: [[@LINE-1]]:12 | class/Swift | c:@M@other_module@objc(cs)I2 | Decl | rel: 0
+- (void)method;
+// CHECK: [[@LINE-1]]:9 | instance-method/Swift | c:@M@other_module@objc(cs)I2(im)method | Decl,Dyn,RelChild | rel: 1
+@end
+
+typedef NS_ENUM(E3, int) {
+  // CHECK: [[@LINE-1]]:17 | enum/Swift | c:@M@third_module@E@E3 | Def | rel: 0
+  firstCase = 1,
+  // CHECK: [[@LINE-1]]:3 | enumerator/Swift | c:@M@third_module@E@E3@firstCase | Def,RelChild | rel: 1
+} EXT_DECL("third_module");

test/Index/Store/handle-prebuilt-module.m

+// RUN: rm -rf %t
+// RUN: mkdir %t
+// RUN: %clang -arch x86_64 -mmacosx-version-min=10.7 -fsyntax-only %s -o %t.o -index-store-path %t/idx1 -fmodules -fmodules-cache-path=%t/mcp -I %S/Inputs/module -Rindex-store 2> %t.err1
+// RUN: %clang -arch x86_64 -mmacosx-version-min=10.7 -fsyntax-only %s -o %t.o -index-store-path %t/idx2 -fmodules -fmodules-cache-path=%t/mcp -I %S/Inputs/module -Rindex-store 2> %t.err2
+// RUN: %clang -arch x86_64 -mmacosx-version-min=10.7 -fsyntax-only %s -o %t.o -index-store-path %t/idx2 -fmodules -fmodules-cache-path=%t/mcp -I %S/Inputs/module -Rindex-store 2> %t.err3
+// RUN: FileCheck -input-file=%t.err1 -check-prefix=CREATING_MODULES %s -allow-empty
+// RUN: FileCheck -input-file=%t.err2 -check-prefix=CREATING_INDEX_DATA_FROM_MODULE_FILES %s
+// RUN: FileCheck -input-file=%t.err3 -check-prefix=EXISTING_INDEX_DATA_FROM_MODULE_FILES %s -allow-empty
+// RUN: c-index-test core -print-unit %t/idx1 > %t/all-units1.txt
+// RUN: c-index-test core -print-unit %t/idx2 > %t/all-units2.txt
+// RUN: c-index-test core -print-record %t/idx1 > %t/all-records1.txt
+// RUN: c-index-test core -print-record %t/idx2 > %t/all-records2.txt
+// RUN: diff -u %t/all-units1.txt %t/all-units2.txt
+// RUN: diff -u %t/all-records1.txt %t/all-records2.txt
+
+@import ModDep;
+
+// CREATING_MODULES-NOT: remark:
+
+// CREATING_INDEX_DATA_FROM_MODULE_FILES: remark: producing index data for module file {{.*}}ModDep{{.*}}.pcm
+// CREATING_INDEX_DATA_FROM_MODULE_FILES: remark: producing index data for module file {{.*}}ModTop{{.*}}.pcm
+
+// EXISTING_INDEX_DATA_FROM_MODULE_FILES-NOT: remark:

test/Index/Store/json-with-module.m

+// RUN: rm -rf %t.idx %t.mcp
+// RUN: %clang -arch x86_64 -mmacosx-version-min=10.7 -c %s -o %t.o -index-store-path %t.idx -fmodules -fmodules-cache-path=%t.mcp -Xclang -fdisable-module-hash -I %S/Inputs/module
+// RUN: c-index-test core -aggregate-json %t.idx -o %t.json
+// RUN: sed -e "s:%S::g" -e "s:%T::g" %t.json > %t.final.json
+// RUN: diff -u %s.json %t.final.json
+
+@import ModDep;

test/Index/Store/json-with-module.m.json

+{
+  "files": [
+    "/json-with-module.m.tmp.mcp/ModDep.pcm",
+    "/json-with-module.m.tmp.mcp/ModTop.pcm",
+    "/Inputs/module/ModDep.h",
+    "/Inputs/module/ModTop.h",
+    "/Inputs/module/ModTopSub1.h",
+    "/Inputs/module/ModTopSub2.h",
+    "/json-with-module.m.tmp.o",
+    "/json-with-module.m",
+    "/Inputs/module/module.modulemap"
+  ],
+  "symbols": [
+    {
+      "kind": "function",
+      "lang": "C",
+      "usr": "c:@F@ModDep_func",
+      "name": "ModDep_func",
+      "roles": "Decl",
+      "rel-roles": "RelCont"
+    },
+    {
+      "kind": "type-alias",
+      "lang": "C",
+      "usr": "c:@T@ModTopStruct",
+      "name": "ModTopStruct",
+      "roles": "Def,Ref,RelCont"
+    },
+    {
+      "kind": "struct",
+      "lang": "C",
+      "usr": "c:@SA@ModTopStruct",
+      "name": "",
+      "roles": "Def"
+    },
+    {
+      "kind": "function",
+      "lang": "C",
+      "usr": "c:@F@ModTop_func",
+      "name": "ModTop_func",
+      "roles": "Decl"
+    },
+    {
+      "kind": "function",
+      "lang": "C",
+      "usr": "c:@F@ModTopSub1_func",
+      "name": "ModTopSub1_func",
+      "roles": "Decl"
+    }
+  ],
+  "records": [
+    {
+      "occurrences": [
+        {
+          "symbol": 0,
+          "line": 3,
+          "col": 6,
+          "roles": "Decl"
+        },
+        {
+          "symbol": 1,
+          "line": 3,
+          "col": 18,
+          "roles": "Ref,RelCont",
+          "relations": [
+            {
+              "symbol": 0,
+              "rel-roles": "RelCont"
+            }
+          ]
+
+        }
+      ]
+    },
+    {
+      "occurrences": [
+        {
+          "symbol": 2,
+          "line": 2,
+          "col": 9,
+          "roles": "Def"
+        },
+        {
+          "symbol": 1,
+          "line": 3,
+          "col": 3,
+          "roles": "Def"
+        },
+        {
+          "symbol": 3,
+          "line": 5,
+          "col": 6,
+          "roles": "Decl"
+        }
+      ]
+    },
+    {
+      "occurrences": [
+        {
+          "symbol": 4,
+          "line": 1,
+          "col": 6,
+          "roles": "Decl"
+        }
+      ]
+    }
+  ],
+  "units": [
+    {
+      "triple": "x86_64-apple-macosx10.7.0",
+      "out-file": 0,
+      "unit-dependencies": [1],
+      "sources": [
+        {
+          "file": 2,
+          "records": [0]
+        }
+      ]
+    },
+    {
+      "triple": "x86_64-apple-macosx10.7.0",
+      "out-file": 1,
+      "sources": [
+        {
+          "file": 3,
+          "records": [1]
+        },
+        {
+          "file": 4,
+          "records": [2]
+        },
+        {
+          "file": 5
+        }
+      ]
+    },
+    {
+      "triple": "x86_64-apple-macosx10.7.0",
+      "out-file": 6,
+      "unit-dependencies": [0],
+      "sources": [
+        {
+          "file": 7
+        },
+        {
+          "file": 8
+        }
+      ]
+    }
+  ]
+}

test/Index/Store/json-with-pch.c

+// RUN: rm -rf %t.idx
+// RUN: %clang -arch x86_64 -mmacosx-version-min=10.7 -x c-header %S/Inputs/head.h -o %t.h.pch -index-store-path %t.idx
+// RUN: %clang -arch x86_64 -mmacosx-version-min=10.7 -c %s -o %t.o -index-store-path %t.idx -include %t.h -Werror
+// RUN: c-index-test core -aggregate-json %t.idx -o %t.json
+// RUN: sed -e "s:%S::g" -e "s:%T::g" %t.json > %t.final.json
+// RUN: diff -u %s.json %t.final.json
+
+int main() {
+  test1_func();
+}

test/Index/Store/json-with-pch.c.json

+{
+  "files": [
+    "/json-with-pch.c.tmp.h.pch",
+    "/Inputs/head.h",
+    "/json-with-pch.c.tmp.o",
+    "/json-with-pch.c"
+  ],
+  "symbols": [
+    {
+      "kind": "function",
+      "lang": "C",
+      "usr": "c:@F@test1_func",
+      "name": "test1_func",
+      "roles": "Decl,Ref,Call,RelCall,RelCont"
+    },
+    {
+      "kind": "function",
+      "lang": "C",
+      "usr": "c:@F@test2_func",
+      "name": "test2_func",
+      "roles": "Decl"
+    },
+    {
+      "kind": "function",
+      "lang": "C",
+      "usr": "c:@F@main",
+      "name": "main",
+      "roles": "Def",
+      "rel-roles": "RelCall,RelCont"
+    }
+  ],
+  "records": [
+    {
+      "occurrences": [
+        {
+          "symbol": 0,
+          "line": 2,
+          "col": 13,
+          "roles": "Decl"
+        },
+        {
+          "symbol": 1,
+          "line": 3,
+          "col": 13,
+          "roles": "Decl"
+        }
+      ]
+    },
+    {
+      "occurrences": [
+        {
+          "symbol": 2,
+          "line": 8,
+          "col": 5,
+          "roles": "Def"
+        },
+        {
+          "symbol": 0,
+          "line": 9,
+          "col": 3,
+          "roles": "Ref,Call,RelCall,RelCont",
+          "relations": [
+            {
+              "symbol": 2,
+              "rel-roles": "RelCall,RelCont"
+            }
+          ]
+
+        }
+      ]
+    }
+  ],
+  "units": [
+    {
+      "triple": "x86_64-apple-macosx10.7.0",
+      "out-file": 0,
+      "sources": [
+        {
+          "file": 1,
+          "records": [0]
+        }
+      ]
+    },
+    {
+      "triple": "x86_64-apple-macosx10.7.0",
+      "out-file": 2,
+      "unit-dependencies": [0],
+      "sources": [
+        {
+          "file": 3,
+          "records": [1]
+        }
+      ]
+    }
+  ]
+}

test/Index/Store/json.c

+// RUN: rm -rf %t.idx
+// RUN: mkdir -p %t.o
+// RUN: %clang -target x86_64-apple-macosx10.7 -arch x86_64 -mmacosx-version-min=10.7 -c %S/Inputs/test1.c -o %t.o/test1.o -index-store-path %t.idx -index-ignore-system-symbols -index-record-codegen-name
+// RUN: %clang -target x86_64-apple-macosx10.7 -arch x86_64 -mmacosx-version-min=10.7 -c %S/Inputs/test2.c -o %t.o/test2.o -index-store-path %t.idx -index-ignore-system-symbols -index-record-codegen-name
+// RUN: %clang -target x86_64-apple-macosx10.7 -arch x86_64 -mmacosx-version-min=10.7 -c %S/Inputs/test3.cpp -o %t.o/test3.o -index-store-path %t.idx -index-ignore-system-symbols -index-record-codegen-name
+// RUN: c-index-test core -aggregate-json %t.idx -o %t.json
+// RUN: sed -e "s:%S::g" -e "s:%t.o::g" %t.json > %t.final.json
+// RUN: diff -u %S/Inputs/json.c.json %t.final.json

test/Index/Store/print-record.mm

+// RUN: rm -rf %t.idx
+// RUN: %clang_cc1 %s -index-store-path %t.idx
+// RUN: c-index-test core -print-record %t.idx | FileCheck %s
+
+@class MyCls;
+
+@interface MyCls
+@end
+
+// CHECK: [[@LINE+2]]:6 | function/C | c:@F@foo#*$objc(cs)MyCls# | Decl | rel: 0
+// CHECK: [[@LINE+1]]:10 | class/ObjC | c:objc(cs)MyCls | Ref,RelCont | rel: 1
+void foo(MyCls *p);
+
+// RANGE-NOT: before_range
+void before_range();
+
+// RANGE: [[@LINE+1]]:6 | function/C | c:@F@in_range1# | Decl
+void in_range1();
+// RANGE: [[@LINE+1]]:6 | function/C | c:@F@in_range2# | Decl
+void in_range2();
+
+// RANGE-NOT: after_range
+void after_range();
+
+// RUN: c-index-test core -print-record %t.idx -filepath %s:18:20 | FileCheck -check-prefix=RANGE %s

test/Index/Store/print-unit.c

+#include "print-unit.h"
+#include "syshead.h"
+
+void foo(int i);
+
+// RUN: rm -rf %t
+// RUN: %clang_cc1 -I %S/Inputs -isystem %S/Inputs/sys -index-store-path %t/idx %s -triple x86_64-apple-macosx10.8
+// RUN: c-index-test core -print-unit %t/idx | FileCheck %s
+// RUN: %clang_cc1 -I %S/Inputs -isystem %S/Inputs/sys -index-store-path %t/idx_opt1 %s -triple x86_64-apple-macosx10.8 -O2
+// RUN: c-index-test core -print-unit %t/idx_opt1 | FileCheck %s -check-prefix=OPT
+// RUN: %clang_cc1 -I %S/Inputs -isystem %S/Inputs/sys -index-store-path %t/idx_opt2 %s -triple x86_64-apple-macosx10.8 -Os
+// RUN: c-index-test core -print-unit %t/idx_opt2 | FileCheck %s -check-prefix=OPT
+
+// CHECK: print-unit.c.o
+// CHECK: provider: clang-
+// CHECK: is-system: 0
+// CHECK: has-main: 1
+// CHECK: main-path: {{.*}}/print-unit.c
+// CHECK: out-file: {{.*}}/print-unit.c.o
+// CHECK: target: x86_64-apple-macosx10.8
+// CHECK: is-debug: 1
+// CHECK: DEPEND START
+// CHECK: Record | user | {{.*}}/print-unit.c | print-unit.c-
+// CHECK: Record | user | {{.*}}/Inputs/head.h | head.h-
+// CHECK: Record | user | {{.*}}/Inputs/using-overlay.h | using-overlay.h-
+// CHECK: Record | system | {{.*}}/Inputs/sys/syshead.h | syshead.h-
+// CHECK: Record | system | {{.*}}/Inputs/sys/another.h | another.h-
+// CHECK: File | user | {{.*}}/Inputs/print-unit.h | | {{[0-9]*$}}
+// CHECK: DEPEND END (6)
+// CHECK: INCLUDE START
+// CHECK: {{.*}}/print-unit.c:1 | {{.*}}/Inputs/print-unit.h
+// CHECK: {{.*}}/print-unit.c:2 | {{.*}}/Inputs/sys/syshead.h
+// CHECK: {{.*}}/Inputs/print-unit.h:1 | {{.*}}/Inputs/head.h
+// CHECK: {{.*}}/Inputs/print-unit.h:2 | {{.*}}/Inputs/using-overlay.h
+// CHECK: INCLUDE END (4)
+
+// OPT: is-debug: 0

test/Index/Store/print-units-with-modules.m

+// RUN: rm -rf %t.idx %t.mcp
+// RUN: %clang -arch x86_64 -mmacosx-version-min=10.7 -c %s -o %t.o -index-store-path %t.idx -fmodules -fmodules-cache-path=%t.mcp -Xclang -fdisable-module-hash -I %S/Inputs/module
+// RUN: c-index-test core -print-unit %t.idx | FileCheck %s
+
+@import ModDep;
+@import ModSystem;
+
+// CHECK: ModDep.pcm
+// CHECK: provider: clang-
+// CHECK: is-system: 0
+// CHECK: is-module: 1
+// CHECK: module-name: ModDep
+// CHECK: has-main: 0
+// CHECK: main-path: {{$}}
+// CHECK: out-file: {{.*}}/ModDep.pcm
+// CHECK: DEPEND START
+// CHECK: Unit | user | ModTop | {{.*}}/ModTop.pcm | ModTop.pcm
+// CHECK: Record | user | ModDep | {{.*}}/Inputs/module/ModDep.h | ModDep.h
+// CHECK: DEPEND END (2)
+
+// CHECK: ModSystem.pcm
+// CHECK: is-system: 1
+// CHECK: is-module: 1
+// CHECK: module-name: ModSystem
+// CHECK: has-main: 0
+// CHECK: main-path: {{$}}
+// CHECK: out-file: {{.*}}/ModSystem.pcm
+// CHECK: DEPEND START
+// CHECK: Record | system | ModSystem | {{.*}}/Inputs/module/ModSystem.h | ModSystem.h
+// CHECK: DEPEND END (1)
+
+// CHECK: ModTop.pcm
+// CHECK: is-system: 0
+// CHECK: is-module: 1
+// CHECK: module-name: ModTop
+// CHECK: has-main: 0
+// CHECK: main-path: {{$}}
+// CHECK: out-file: {{.*}}/ModTop.pcm
+// CHECK: DEPEND START
+// CHECK: Record | user | ModTop | {{.*}}/Inputs/module/ModTop.h | ModTop.h
+// CHECK: Record | user | ModTop.Sub1 | {{.*}}/Inputs/module/ModTopSub1.h | ModTopSub1.h
+// CHECK: File | user | ModTop.Sub2 | {{.*}}/Inputs/module/ModTopSub2.h | | {{[0-9]*$}}
+// CHECK: DEPEND END (3)
+
+// CHECK: print-units-with-modules.m.tmp.o
+// CHECK: is-system: 0
+// CHECK: is-module: 0
+// CHECK: module-name: <none>
+// CHECK: has-main: 1
+// CHECK: main-path: {{.*}}/print-units-with-modules.m
+// CHECK: out-file: {{.*}}/print-units-with-modules.m.tmp.o
+// CHECK: DEPEND START
+// CHECK: Unit | user | ModDep | {{.*}}/ModDep.pcm | ModDep.pcm
+// CHECK: Unit | system | ModSystem | {{.*}}/ModSystem.pcm | ModSystem.pcm
+// CHECK: File | user | {{.*}}/print-units-with-modules.m | | {{[0-9]*$}}
+// CHECK: File | user | {{.*}}/Inputs/module/module.modulemap | | {{[0-9]*$}}
+// CHECK: DEPEND END (4)

test/Index/Store/print-units-with-pch.c

+// RUN: rm -rf %t.idx
+// RUN: %clang -arch x86_64 -mmacosx-version-min=10.7 -x c-header %S/Inputs/head.h -o %t.h.pch -index-store-path %t.idx
+// RUN: %clang -arch x86_64 -mmacosx-version-min=10.7 -c %s -o %t.o -index-store-path %t.idx -include %t.h -Werror
+// RUN: c-index-test core -print-unit %t.idx | FileCheck %s
+
+int main() {
+  test1_func();
+}
+
+// CHECK: print-units-with-pch.c.tmp.h.pch
+// CHECK: is-system: 0
+// CHECK: has-main: 0
+// CHECK: main-path: {{$}}
+// CHECK: out-file: {{.*}}/print-units-with-pch.c.tmp.h.pch
+// CHECK: DEPEND START
+// CHECK: Record | user | {{.*}}/Inputs/head.h | head.h
+// CHECK: DEPEND END (1)
+
+// CHECK: print-units-with-pch.c.tmp.o
+// CHECK: is-system: 0
+// CHECK: has-main: 1
+// CHECK: main-path: {{.*}}/print-units-with-pch.c
+// CHECK: out-file: {{.*}}/print-units-with-pch.c.tmp.o
+// CHECK: DEPEND START
+// CHECK: Unit | user | {{.*}}/print-units-with-pch.c.tmp.h.pch | print-units-with-pch.c.tmp.h.pch
+// CHECK: Record | user | {{.*}}/print-units-with-pch.c | print-units-with-pch.c
+// CHECK: DEPEND END (2)

test/Index/Store/record-hash-crash-invalid-name.cpp

+// Makes sure it doesn't crash.
+
+// RUN: rm -rf %t
+// RUN: not %clang_cc1 %s -index-store-path %t/idx -std=c++14
+// RUN: c-index-test core -print-record %t/idx | FileCheck %s
+
+namespace rdar32474406 {
+void foo();
+typedef void (*Func_t)();
+// CHECK: [[@LINE+4]]:1 | type-alias/C | c:record-hash-crash-invalid-name.cpp@N@rdar32474406@T@Func_t | Ref,RelCont | rel: 1
+// CHECK-NEXT: RelCont | c:@N@rdar32474406
+// CHECK: [[@LINE+2]]:13 | function/C | c:@N@rdar32474406@F@foo# | Ref,RelCont | rel: 1
+// CHECK-NEXT: RelCont | c:@N@rdar32474406
+Func_t[] = {foo}; // invalid decomposition
+} // namespace rdar32474406

test/Index/Store/record-hash-crash.cpp

+// Makes sure it doesn't crash.
+
+// RUN: rm -rf %t
+// RUN: %clang_cc1 %s -index-store-path %t/idx -std=c++14
+// RUN: c-index-test core -print-record %t/idx | FileCheck %s
+
+namespace crash1 {
+// CHECK: [[@LINE+1]]:6 | function/C
+auto getit() {
+  return []() {};
+}
+} // namespace crash1
+
+namespace crash2 {
+// CHECK: [[@LINE+2]]:7 | class(Gen)/C++ | c:@N@crash2@ST>1#T@Foo | Decl,RelChild | rel: 1
+template <typename T>
+class Foo; // canonical decl
+
+// CHECK: [[@LINE+2]]:7 | class(Gen)/C++ | c:@N@crash2@ST>1#T@Foo | Def,RelChild | rel: 1
+template <typename T>
+class Foo {};
+
+// CHECK: [[@LINE+2]]:8 | struct(Gen)/C++ | c:@N@crash2@ST>1#t>1#pT@Wrapper | Def,RelChild | rel: 1
+template <template <typename... ARGS> class TYPE>
+struct Wrapper {};
+
+// CHECK: [[@LINE+3]]:8 | struct(Gen,TS)/C++ | c:@N@crash2@S@Wrapper>#@N@crash2@ST>1#T@Foo | Def,RelChild,RelSpecialization | rel: 2
+// CHECK: RelSpecialization | c:@N@crash2@ST>1#t>1#pT@Wrapper
+template <>
+struct Wrapper<Foo> {};
+} // namespace crash2

test/Index/Store/syntax-only.c

+// RUN: rm -rf %t.idx
+// RUN: %clang -fsyntax-only %s -index-store-path %t.idx -o %T/syntax-only.c.myoutfile
+// RUN: c-index-test core -print-unit %t.idx | FileCheck %s -check-prefix=CHECK-UNIT
+// RUN: c-index-test core -print-record %t.idx | FileCheck %s -check-prefix=CHECK-RECORD
+
+// CHECK-UNIT: out-file: {{.*}}/syntax-only.c.myoutfile
+// CHECK-RECORD: function/C | foo | c:@F@foo
+
+void foo();

test/Index/Store/unit-with-vfs.c

+// RUN: sed -e "s:INPUT_DIR:%S/Inputs:g" -e "s:OUT_DIR:%t:g" %S/Inputs/overlay.yaml > %t.yaml
+// REQUIRES: shell
+
+#include "using-overlay.h"
+
+// RUN: rm -rf %t.idx
+// RUN: %clang_cc1 %s -index-store-path %t.idx -I %t -ivfsoverlay %t.yaml
+// RUN: c-index-test core -print-unit %t.idx | FileCheck %s
+
+// CHECK: Record | user | {{.*}}test/Index/Store/Inputs/using-overlay.h

test/Index/Store/unit-workdir-prefix.c

+#include "header.h"
+
+void foo(void) {
+  bar();
+}
+
+// RUN: rm -rf %t
+// RUN: mkdir -p %t/Directory
+// RUN: mkdir -p %t/Directory.surprise
+// RUN: mkdir -p %t/sdk
+// RUN: mkdir -p %t/sdk_other
+// RUN: echo "void bar(void);" > %t/sdk_other/header.h
+// RUN: cp %s %t/Directory.surprise/main.c
+//
+// RUN: %clang_cc1 -isystem %t/sdk_other -isysroot %t/sdk -index-store-path %t/idx %t/Directory.surprise/main.c -triple x86_64-apple-macosx10.8 -working-directory %t/Directory
+// RUN: c-index-test core -print-unit %t/idx | FileCheck %s
+
+// CHECK: main.c.o
+// CHECK: provider: clang-
+// CHECK: is-system: 0
+// CHECK: has-main: 1
+// CHECK: main-path: {{.*}}Directory.surprise{{/|\\}}main.c
+// CHECK: out-file: {{.*}}Directory.surprise{{/|\\}}main.c.o
+// CHECK: target: x86_64-apple-macosx10.8
+// CHECK: is-debug: 1
+// CHECK: DEPEND START
+// CHECK: Record | user | {{.*}}Directory.surprise{{/|\\}}main.c | main.c-
+// CHECK: Record | system | {{.*}}sdk_other{{/|\\}}header.h | header.h-

test/Index/Store/using-libstdcpp-arc.mm

+// Test to make sure we don't crash, rdar://30816887.
+
+// RUN: rm -rf %t.idx
+// RUN: %clang_cc1 %s -index-store-path %t.idx -fobjc-arc -fobjc-arc-cxxlib=libstdc++
+// RUN: c-index-test core -print-record %t.idx | FileCheck %s
+
+// CHECK: [[@LINE+1]]:6 | function/C
+void test1(void);

tools/CMakeLists.txt

 create_subdirectory_options(CLANG TOOL)
 
 add_clang_subdirectory(diagtool)
 add_clang_subdirectory(driver)
 add_clang_subdirectory(clang-diff)
 add_clang_subdirectory(clang-format)
 add_clang_subdirectory(clang-format-vs)
 add_clang_subdirectory(clang-fuzzer)
 add_clang_subdirectory(clang-import-test)
 add_clang_subdirectory(clang-offload-bundler)
 
 add_clang_subdirectory(c-index-test)
+add_clang_subdirectory(IndexStore)
 
 add_clang_subdirectory(clang-rename)
 add_clang_subdirectory(clang-refactor)
 
 if(CLANG_ENABLE_ARCMT)
   add_clang_subdirectory(arcmt-test)
   add_clang_subdirectory(c-arcmt-test)
 endif()

tools/IndexStore/CMakeLists.txt

+include(CheckIncludeFiles)
+
+set(SOURCES
+  IndexStore.cpp
+
+  ADDITIONAL_HEADERS
+  ../../include/indexstore/indexstore.h
+  ../../include/indexstore/IndexStoreCXX.h
+  )
+
+set(LIBS
+  clangDirectoryWatcher
+  clangIndex
+)
+
+set(LLVM_EXPORTED_SYMBOL_FILE ${CMAKE_CURRENT_SOURCE_DIR}/IndexStore.exports)
+
+set(ENABLE_SHARED SHARED)
+
+if(WIN32)
+  set(output_name "libIndexStore")
+else()
+  set(output_name "IndexStore")
+endif()
+
+# FIXME: needs to be ported to non-Apple platforms.
+if(APPLE)
+
+add_clang_library(IndexStore ${ENABLE_SHARED} ${ENABLE_STATIC}
+  OUTPUT_NAME ${output_name}
+  ${SOURCES}
+
+  LINK_LIBS
+  ${LIBS}
+
+  LINK_COMPONENTS
+  ${LLVM_TARGETS_TO_BUILD}
+  Core
+  Support
+  )
+
+set(INDEXSTORE_LIBRARY_VERSION "${LLVM_VERSION_MAJOR}.${LLVM_VERSION_MINOR}.${LLVM_VERSION_PATCH}")
+
+if(ENABLE_SHARED)
+  if(WIN32)
+    set_target_properties(IndexStore
+      PROPERTIES
+      VERSION ${INDEXSTORE_LIBRARY_VERSION}
+      DEFINE_SYMBOL _CINDEX_LIB_)
+  elseif(APPLE)
+    set(INDEXSTORE_LINK_FLAGS " -Wl,-compatibility_version -Wl,1")
+    set(INDEXSTORE_LINK_FLAGS "${INDEXSTORE_LINK_FLAGS} -Wl,-current_version -Wl,${INDEXSTORE_LIBRARY_VERSION}")
+
+    check_include_files("CoreServices/CoreServices.h" HAVE_CORESERVICES_H)
+    if(HAVE_CORESERVICES_H)
+      set(INDEXSTORE_LINK_FLAGS "${INDEXSTORE_LINK_FLAGS} -framework CoreServices")
+    endif()
+
+    set_property(TARGET IndexStore APPEND_STRING PROPERTY
+                 LINK_FLAGS ${INDEXSTORE_LINK_FLAGS})
+  else()
+    set_target_properties(IndexStore
+      PROPERTIES
+      VERSION ${INDEXSTORE_LIBRARY_VERSION}
+      DEFINE_SYMBOL _CINDEX_LIB_)
+  endif()
+endif()
+
+if (LLVM_INSTALL_TOOLCHAIN_ONLY)
+  install(TARGETS IndexStore
+    COMPONENT IndexStore
+    LIBRARY DESTINATION lib${LLVM_LIBDIR_SUFFIX}
+    ARCHIVE DESTINATION lib${LLVM_LIBDIR_SUFFIX}
+    RUNTIME DESTINATION bin)
+
+  if (NOT CMAKE_CONFIGURATION_TYPES)
+    add_custom_target(install-IndexStore
+                      DEPENDS IndexStore
+                      COMMAND "${CMAKE_COMMAND}"
+                              -DCMAKE_INSTALL_COMPONENT=IndexStore
+                              -P "${CMAKE_BINARY_DIR}/cmake_install.cmake")
+  endif()
+endif()
+
+set(INDEXSTORE_HEADERS_INSTALL_DESTINATION "local/include")
+
+install(DIRECTORY ../../include/indexstore
+  COMPONENT IndexStore
+  DESTINATION "${INDEXSTORE_HEADERS_INSTALL_DESTINATION}"
+  FILES_MATCHING
+  PATTERN "*.h"
+  PATTERN ".svn" EXCLUDE
+  )
+endif()

tools/IndexStore/IndexStore.cpp

+//===- IndexStore.cpp - Index store API -----------------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the API for the index store.
+//
+//===----------------------------------------------------------------------===//
+
+#include "indexstore/indexstore.h"
+#include "clang/DirectoryWatcher/DirectoryWatcher.h"
+#include "clang/Index/IndexDataStore.h"
+#include "clang/Index/IndexDataStoreSymbolUtils.h"
+#include "clang/Index/IndexRecordReader.h"
+#include "clang/Index/IndexUnitReader.h"
+#include "clang/Index/IndexUnitWriter.h"
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/Optional.h"
+#include "llvm/ADT/SmallString.h"
+#include "llvm/Support/Chrono.h"
+#include <Block.h>
+
+using namespace clang;
+using namespace clang::index;
+using namespace llvm;
+
+static indexstore_string_ref_t toIndexStoreString(StringRef str) {
+  return indexstore_string_ref_t{str.data(), str.size()};
+}
+
+static timespec toTimeSpec(sys::TimePoint<> tp) {
+  std::chrono::seconds sec =
+      std::chrono::time_point_cast<std::chrono::seconds>(tp).time_since_epoch();
+  std::chrono::nanoseconds nsec =
+      std::chrono::time_point_cast<std::chrono::nanoseconds>(tp - sec)
+          .time_since_epoch();
+  timespec ts;
+  ts.tv_sec = sec.count();
+  ts.tv_nsec = nsec.count();
+  return ts;
+}
+
+namespace {
+
+struct IndexStoreError {
+  std::string Error;
+};
+
+} // anonymous namespace
+
+const char *indexstore_error_get_description(indexstore_error_t err) {
+  return static_cast<IndexStoreError *>(err)->Error.c_str();
+}
+
+void indexstore_error_dispose(indexstore_error_t err) {
+  delete static_cast<IndexStoreError *>(err);
+}
+
+unsigned indexstore_format_version(void) {
+  return IndexDataStore::getFormatVersion();
+}
+
+indexstore_t indexstore_store_create(const char *store_path,
+                                     indexstore_error_t *c_error) {
+  std::unique_ptr<IndexDataStore> store;
+  std::string error;
+  store = IndexDataStore::create(store_path, error);
+  if (!store) {
+    if (c_error)
+      *c_error = new IndexStoreError{error};
+    return nullptr;
+  }
+  return store.release();
+}
+
+void indexstore_store_dispose(indexstore_t store) {
+  delete static_cast<IndexDataStore *>(store);
+}
+
+#if INDEXSTORE_HAS_BLOCKS
+bool indexstore_store_units_apply(
+    indexstore_t c_store, unsigned sorted,
+    bool (^applier)(indexstore_string_ref_t unit_name)) {
+  IndexDataStore *store = static_cast<IndexDataStore *>(c_store);
+  return store->foreachUnitName(sorted, [&](StringRef unitName) -> bool {
+    return applier(toIndexStoreString(unitName));
+  });
+}
+
+size_t indexstore_unit_event_notification_get_events_count(
+    indexstore_unit_event_notification_t c_evtnote) {
+  auto *evtnote =
+      static_cast<IndexDataStore::UnitEventNotification *>(c_evtnote);
+  return evtnote->Events.size();
+}
+
+indexstore_unit_event_t indexstore_unit_event_notification_get_event(
+    indexstore_unit_event_notification_t c_evtnote, size_t index) {
+  auto *evtnote =
+      static_cast<IndexDataStore::UnitEventNotification *>(c_evtnote);
+  return (indexstore_unit_event_t)&evtnote->Events[index];
+}
+
+bool indexstore_unit_event_notification_is_initial(
+    indexstore_unit_event_notification_t c_evtnote) {
+  auto *evtnote =
+      static_cast<IndexDataStore::UnitEventNotification *>(c_evtnote);
+  return evtnote->IsInitial;
+}
+
+indexstore_unit_event_kind_t
+indexstore_unit_event_get_kind(indexstore_unit_event_t c_evt) {
+  auto *evt = static_cast<IndexDataStore::UnitEvent *>(c_evt);
+  indexstore_unit_event_kind_t k;
+  switch (evt->Kind) {
+  case IndexDataStore::UnitEventKind::Added:
+    k = INDEXSTORE_UNIT_EVENT_ADDED;
+    break;
+  case IndexDataStore::UnitEventKind::Removed:
+    k = INDEXSTORE_UNIT_EVENT_REMOVED;
+    break;
+  case IndexDataStore::UnitEventKind::Modified:
+    k = INDEXSTORE_UNIT_EVENT_MODIFIED;
+    break;
+  case IndexDataStore::UnitEventKind::DirectoryDeleted:
+    k = INDEXSTORE_UNIT_EVENT_DIRECTORY_DELETED;
+    break;
+  }
+  return k;
+}
+
+indexstore_string_ref_t
+indexstore_unit_event_get_unit_name(indexstore_unit_event_t c_evt) {
+  auto *evt = static_cast<IndexDataStore::UnitEvent *>(c_evt);
+  return toIndexStoreString(evt->UnitName);
+}
+
+timespec
+indexstore_unit_event_get_modification_time(indexstore_unit_event_t c_evt) {
+  auto *evt = static_cast<IndexDataStore::UnitEvent *>(c_evt);
+  return evt->ModTime;
+}
+
+void indexstore_store_set_unit_event_handler(
+    indexstore_t c_store, indexstore_unit_event_handler_t blk_handler) {
+  IndexDataStore *store = static_cast<IndexDataStore *>(c_store);
+  if (!blk_handler) {
+    store->setUnitEventHandler(nullptr);
+    return;
+  }
+
+  class BlockWrapper {
+    indexstore_unit_event_handler_t blk_handler;
+
+  public:
+    BlockWrapper(indexstore_unit_event_handler_t handler) {
+      blk_handler = Block_copy(handler);
+    }
+    BlockWrapper(const BlockWrapper &other) {
+      blk_handler = Block_copy(other.blk_handler);
+    }
+    ~BlockWrapper() { Block_release(blk_handler); }
+
+    void operator()(indexstore_unit_event_notification_t evt_note) const {
+      blk_handler(evt_note);
+    }
+  };
+
+  BlockWrapper handler(blk_handler);
+
+  store->setUnitEventHandler(
+      [handler](IndexDataStore::UnitEventNotification evtNote) {
+        handler(&evtNote);
+      });
+}
+#endif
+
+bool indexstore_store_start_unit_event_listening(
+    indexstore_t c_store, indexstore_unit_event_listen_options_t *client_opts,
+    size_t listen_options_struct_size, indexstore_error_t *c_error) {
+  IndexDataStore *store = static_cast<IndexDataStore *>(c_store);
+  indexstore_unit_event_listen_options_t listen_opts;
+  memset(&listen_opts, 0, sizeof(listen_opts));
+  unsigned clientOptSize = listen_options_struct_size < sizeof(listen_opts)
+                               ? listen_options_struct_size
+                               : sizeof(listen_opts);
+  memcpy(&listen_opts, client_opts, clientOptSize);
+
+  std::string error;
+  auto createFn =
+      [](StringRef Path, AbstractDirectoryWatcher::EventReceiver Receiver,
+         bool waitInitialSync,
+         std::string &Error) -> std::unique_ptr<AbstractDirectoryWatcher> {
+    return DirectoryWatcher::create(Path, std::move(Receiver), waitInitialSync,
+                                    Error);
+  };
+  bool err = store->startEventListening(createFn, listen_opts.wait_initial_sync,
+                                        error);
+  if (err && c_error)
+    *c_error = new IndexStoreError{error};
+  return err;
+}
+
+void indexstore_store_stop_unit_event_listening(indexstore_t c_store) {
+  IndexDataStore *store = static_cast<IndexDataStore *>(c_store);
+  store->stopEventListening();
+}
+
+void indexstore_store_discard_unit(indexstore_t c_store,
+                                   const char *unit_name) {
+  IndexDataStore *store = static_cast<IndexDataStore *>(c_store);
+  store->discardUnit(unit_name);
+}
+
+void indexstore_store_discard_record(indexstore_t c_store,
+                                     const char *record_name) {
+  IndexDataStore *store = static_cast<IndexDataStore *>(c_store);
+  store->discardRecord(record_name);
+}
+
+void indexstore_store_purge_stale_data(indexstore_t c_store) {
+  IndexDataStore *store = static_cast<IndexDataStore *>(c_store);
+  store->purgeStaleData();
+}
+
+indexstore_symbol_kind_t indexstore_symbol_get_kind(indexstore_symbol_t sym) {
+  return getIndexStoreKind(static_cast<IndexRecordDecl *>(sym)->SymInfo.Kind);
+}
+
+indexstore_symbol_subkind_t
+indexstore_symbol_get_subkind(indexstore_symbol_t sym) {
+  return getIndexStoreSubKind(
+      static_cast<IndexRecordDecl *>(sym)->SymInfo.SubKind);
+}
+
+indexstore_symbol_language_t
+indexstore_symbol_get_language(indexstore_symbol_t sym) {
+  return getIndexStoreLang(static_cast<IndexRecordDecl *>(sym)->SymInfo.Lang);
+}
+
+uint64_t indexstore_symbol_get_properties(indexstore_symbol_t sym) {
+  return getIndexStoreProperties(
+      static_cast<IndexRecordDecl *>(sym)->SymInfo.Properties);
+}
+
+uint64_t indexstore_symbol_get_roles(indexstore_symbol_t sym) {
+  return getIndexStoreRoles(static_cast<IndexRecordDecl *>(sym)->Roles);
+}
+
+uint64_t indexstore_symbol_get_related_roles(indexstore_symbol_t sym) {
+  return getIndexStoreRoles(static_cast<IndexRecordDecl *>(sym)->RelatedRoles);
+}
+
+indexstore_string_ref_t indexstore_symbol_get_name(indexstore_symbol_t sym) {
+  auto *D = static_cast<IndexRecordDecl *>(sym);
+  return toIndexStoreString(D->Name);
+}
+
+indexstore_string_ref_t indexstore_symbol_get_usr(indexstore_symbol_t sym) {
+  auto *D = static_cast<IndexRecordDecl *>(sym);
+  return toIndexStoreString(D->USR);
+}
+
+indexstore_string_ref_t
+indexstore_symbol_get_codegen_name(indexstore_symbol_t sym) {
+  auto *D = static_cast<IndexRecordDecl *>(sym);
+  return toIndexStoreString(D->CodeGenName);
+}
+
+uint64_t
+indexstore_symbol_relation_get_roles(indexstore_symbol_relation_t sym_rel) {
+  return getIndexStoreRoles(static_cast<IndexRecordRelation *>(sym_rel)->Roles);
+}
+
+indexstore_symbol_t
+indexstore_symbol_relation_get_symbol(indexstore_symbol_relation_t sym_rel) {
+  return (indexstore_symbol_t) static_cast<IndexRecordRelation *>(sym_rel)->Dcl;
+}
+
+indexstore_symbol_t
+indexstore_occurrence_get_symbol(indexstore_occurrence_t occur) {
+  return (indexstore_symbol_t) static_cast<IndexRecordOccurrence *>(occur)->Dcl;
+}
+
+#if INDEXSTORE_HAS_BLOCKS
+bool indexstore_occurrence_relations_apply(
+    indexstore_occurrence_t occur,
+    bool (^applier)(indexstore_symbol_relation_t symbol_rel)) {
+  auto *recOccur = static_cast<IndexRecordOccurrence *>(occur);
+  for (auto &rel : recOccur->Relations) {
+    if (!applier(&rel))
+      return false;
+  }
+  return true;
+}
+#endif
+
+uint64_t indexstore_occurrence_get_roles(indexstore_occurrence_t occur) {
+  return static_cast<IndexRecordOccurrence *>(occur)->Roles;
+}
+
+void indexstore_occurrence_get_line_col(indexstore_occurrence_t occur,
+                                        unsigned *line, unsigned *column) {
+  auto *recOccur = static_cast<IndexRecordOccurrence *>(occur);
+  if (line)
+    *line = recOccur->Line;
+  if (column)
+    *column = recOccur->Column;
+}
+
+typedef void *indexstore_record_reader_t;
+
+indexstore_record_reader_t
+indexstore_record_reader_create(indexstore_t c_store, const char *record_name,
+                                indexstore_error_t *c_error) {
+  IndexDataStore *store = static_cast<IndexDataStore *>(c_store);
+  std::unique_ptr<IndexRecordReader> reader;
+  std::string error;
+  reader = IndexRecordReader::createWithRecordFilename(
+      record_name, store->getFilePath(), error);
+  if (!reader) {
+    if (c_error)
+      *c_error = new IndexStoreError{error};
+    return nullptr;
+  }
+  return reader.release();
+}
+
+void indexstore_record_reader_dispose(indexstore_record_reader_t rdr) {
+  auto *reader = static_cast<IndexRecordReader *>(rdr);
+  delete reader;
+}
+
+#if INDEXSTORE_HAS_BLOCKS
+/// Goes through the symbol data and passes symbols to \c receiver, for the
+/// symbol data that \c filter returns true on.
+///
+/// This allows allocating memory only for the record symbols that the caller is
+/// interested in.
+bool indexstore_record_reader_search_symbols(
+    indexstore_record_reader_t rdr,
+    bool (^filter)(indexstore_symbol_t symbol, bool *stop),
+    void (^receiver)(indexstore_symbol_t symbol)) {
+  auto *reader = static_cast<IndexRecordReader *>(rdr);
+
+  auto filterFn =
+      [&](const IndexRecordDecl &D) -> IndexRecordReader::DeclSearchReturn {
+    bool stop = false;
+    bool accept = filter((indexstore_symbol_t)&D, &stop);
+    return {accept, !stop};
+  };
+  auto receiverFn = [&](const IndexRecordDecl *D) {
+    receiver((indexstore_symbol_t)D);
+  };
+
+  return reader->searchDecls(filterFn, receiverFn);
+}
+
+bool indexstore_record_reader_symbols_apply(
+    indexstore_record_reader_t rdr, bool nocache,
+    bool (^applier)(indexstore_symbol_t symbol)) {
+  auto *reader = static_cast<IndexRecordReader *>(rdr);
+  auto receiverFn = [&](const IndexRecordDecl *D) -> bool {
+    return applier((indexstore_symbol_t)D);
+  };
+  return reader->foreachDecl(nocache, receiverFn);
+}
+
+bool indexstore_record_reader_occurrences_apply(
+    indexstore_record_reader_t rdr,
+    bool (^applier)(indexstore_occurrence_t occur)) {
+  auto *reader = static_cast<IndexRecordReader *>(rdr);
+  auto receiverFn = [&](const IndexRecordOccurrence &RO) -> bool {
+    return applier((indexstore_occurrence_t)&RO);
+  };
+  return reader->foreachOccurrence(receiverFn);
+}
+
+bool indexstore_record_reader_occurrences_in_line_range_apply(
+    indexstore_record_reader_t rdr, unsigned line_start, unsigned line_count,
+    bool (^applier)(indexstore_occurrence_t occur)) {
+  auto *reader = static_cast<IndexRecordReader *>(rdr);
+  auto receiverFn = [&](const IndexRecordOccurrence &RO) -> bool {
+    return applier((indexstore_occurrence_t)&RO);
+  };
+  return reader->foreachOccurrenceInLineRange(line_start, line_count,
+                                              receiverFn);
+}
+
+/// \param symbols if non-zero \c symbols_count, indicates the list of symbols
+/// that we want to get occurrences for. An empty array indicates that we want
+/// occurrences for all symbols.
+/// \param related_symbols Same as \c symbols but for related symbols.
+bool indexstore_record_reader_occurrences_of_symbols_apply(
+    indexstore_record_reader_t rdr, indexstore_symbol_t *symbols,
+    size_t symbols_count, indexstore_symbol_t *related_symbols,
+    size_t related_symbols_count,
+    bool (^applier)(indexstore_occurrence_t occur)) {
+  auto *reader = static_cast<IndexRecordReader *>(rdr);
+  auto receiverFn = [&](const IndexRecordOccurrence &RO) -> bool {
+    return applier((indexstore_occurrence_t)&RO);
+  };
+  return reader->foreachOccurrence(
+      {(IndexRecordDecl **)symbols, symbols_count},
+      {(IndexRecordDecl **)related_symbols, related_symbols_count}, receiverFn);
+}
+#endif
+
+size_t indexstore_store_get_unit_name_from_output_path(indexstore_t store,
+                                                       const char *output_path,
+                                                       char *name_buf,
+                                                       size_t buf_size) {
+  SmallString<256> unitName;
+  IndexUnitWriter::getUnitNameForAbsoluteOutputFile(output_path, unitName);
+  size_t nameLen = unitName.size();
+  strlcpy(name_buf, unitName.c_str(), buf_size);
+  return nameLen;
+}
+
+bool indexstore_store_get_unit_modification_time(indexstore_t c_store,
+                                                 const char *unit_name,
+                                                 int64_t *seconds,
+                                                 int64_t *nanoseconds,
+                                                 indexstore_error_t *c_error) {
+  IndexDataStore *store = static_cast<IndexDataStore *>(c_store);
+  std::string error;
+  // FIXME: This provides mod time with second-only accuracy.
+  auto optModTime = IndexUnitReader::getModificationTimeForUnit(
+      unit_name, store->getFilePath(), error);
+  if (!optModTime) {
+    if (c_error)
+      *c_error = new IndexStoreError{error};
+    return true;
+  }
+
+  timespec ts = toTimeSpec(*optModTime);
+  if (seconds)
+    *seconds = ts.tv_sec;
+  if (nanoseconds)
+    *nanoseconds = ts.tv_nsec;
+
+  return false;
+}
+
+indexstore_unit_reader_t
+indexstore_unit_reader_create(indexstore_t c_store, const char *unit_name,
+                              indexstore_error_t *c_error) {
+  IndexDataStore *store = static_cast<IndexDataStore *>(c_store);
+  std::unique_ptr<IndexUnitReader> reader;
+  std::string error;
+  reader = IndexUnitReader::createWithUnitFilename(unit_name,
+                                                   store->getFilePath(), error);
+  if (!reader) {
+    if (c_error)
+      *c_error = new IndexStoreError{error};
+    return nullptr;
+  }
+  return reader.release();
+}
+
+void indexstore_unit_reader_dispose(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  delete reader;
+}
+
+indexstore_string_ref_t
+indexstore_unit_reader_get_provider_identifier(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return toIndexStoreString(reader->getProviderIdentifier());
+}
+
+indexstore_string_ref_t
+indexstore_unit_reader_get_provider_version(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return toIndexStoreString(reader->getProviderVersion());
+}
+
+void indexstore_unit_reader_get_modification_time(indexstore_unit_reader_t rdr,
+                                                  int64_t *seconds,
+                                                  int64_t *nanoseconds) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  // FIXME: This provides mod time with second-only accuracy.
+  sys::TimePoint<> timeVal = reader->getModificationTime();
+  timespec ts = toTimeSpec(timeVal);
+  if (seconds)
+    *seconds = ts.tv_sec;
+  if (nanoseconds)
+    *nanoseconds = ts.tv_nsec;
+}
+
+bool indexstore_unit_reader_is_system_unit(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return reader->isSystemUnit();
+}
+
+bool indexstore_unit_reader_is_module_unit(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return reader->isModuleUnit();
+}
+
+bool indexstore_unit_reader_is_debug_compilation(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return reader->isDebugCompilation();
+}
+
+bool indexstore_unit_reader_has_main_file(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return reader->hasMainFile();
+}
+
+indexstore_string_ref_t
+indexstore_unit_reader_get_main_file(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return toIndexStoreString(reader->getMainFilePath());
+}
+
+indexstore_string_ref_t
+indexstore_unit_reader_get_module_name(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return toIndexStoreString(reader->getModuleName());
+}
+
+indexstore_string_ref_t
+indexstore_unit_reader_get_working_dir(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return toIndexStoreString(reader->getWorkingDirectory());
+}
+
+indexstore_string_ref_t
+indexstore_unit_reader_get_output_file(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return toIndexStoreString(reader->getOutputFile());
+}
+
+indexstore_string_ref_t
+indexstore_unit_reader_get_sysroot_path(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return toIndexStoreString(reader->getSysrootPath());
+}
+
+indexstore_string_ref_t
+indexstore_unit_reader_get_target(indexstore_unit_reader_t rdr) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return toIndexStoreString(reader->getTarget());
+}
+
+indexstore_unit_dependency_kind_t
+indexstore_unit_dependency_get_kind(indexstore_unit_dependency_t c_dep) {
+  auto dep = static_cast<const IndexUnitReader::DependencyInfo *>(c_dep);
+  switch (dep->Kind) {
+  case IndexUnitReader::DependencyKind::Unit:
+    return INDEXSTORE_UNIT_DEPENDENCY_UNIT;
+  case IndexUnitReader::DependencyKind::Record:
+    return INDEXSTORE_UNIT_DEPENDENCY_RECORD;
+  case IndexUnitReader::DependencyKind::File:
+    return INDEXSTORE_UNIT_DEPENDENCY_FILE;
+  }
+}
+
+bool indexstore_unit_dependency_is_system(indexstore_unit_dependency_t c_dep) {
+  auto dep = static_cast<const IndexUnitReader::DependencyInfo *>(c_dep);
+  return dep->IsSystem;
+}
+
+indexstore_string_ref_t
+indexstore_unit_dependency_get_filepath(indexstore_unit_dependency_t c_dep) {
+  auto dep = static_cast<const IndexUnitReader::DependencyInfo *>(c_dep);
+  return toIndexStoreString(dep->FilePath);
+}
+
+indexstore_string_ref_t
+indexstore_unit_dependency_get_modulename(indexstore_unit_dependency_t c_dep) {
+  auto dep = static_cast<const IndexUnitReader::DependencyInfo *>(c_dep);
+  return toIndexStoreString(dep->ModuleName);
+}
+
+indexstore_string_ref_t
+indexstore_unit_dependency_get_name(indexstore_unit_dependency_t c_dep) {
+  auto dep = static_cast<const IndexUnitReader::DependencyInfo *>(c_dep);
+  return toIndexStoreString(dep->UnitOrRecordName);
+}
+
+time_t indexstore_unit_dependency_get_modification_time(
+    indexstore_unit_dependency_t c_dep) {
+  auto dep = static_cast<const IndexUnitReader::DependencyInfo *>(c_dep);
+  return dep->ModTime;
+}
+
+size_t
+indexstore_unit_dependency_get_file_size(indexstore_unit_dependency_t c_dep) {
+  auto dep = static_cast<const IndexUnitReader::DependencyInfo *>(c_dep);
+  return dep->FileSize;
+}
+
+indexstore_string_ref_t
+indexstore_unit_include_get_source_path(indexstore_unit_include_t c_inc) {
+  auto inc = static_cast<const IndexUnitReader::IncludeInfo *>(c_inc);
+  return toIndexStoreString(inc->SourcePath);
+}
+
+indexstore_string_ref_t
+indexstore_unit_include_get_target_path(indexstore_unit_include_t c_inc) {
+  auto inc = static_cast<const IndexUnitReader::IncludeInfo *>(c_inc);
+  return toIndexStoreString(inc->TargetPath);
+}
+
+unsigned
+indexstore_unit_include_get_source_line(indexstore_unit_include_t c_inc) {
+  auto inc = static_cast<const IndexUnitReader::IncludeInfo *>(c_inc);
+  return inc->SourceLine;
+}
+
+#if INDEXSTORE_HAS_BLOCKS
+bool indexstore_unit_reader_dependencies_apply(
+    indexstore_unit_reader_t rdr,
+    bool (^applier)(indexstore_unit_dependency_t)) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return reader->foreachDependency(
+      [&](const IndexUnitReader::DependencyInfo &depInfo) -> bool {
+        return applier((void *)&depInfo);
+      });
+}
+
+bool indexstore_unit_reader_includes_apply(
+    indexstore_unit_reader_t rdr, bool (^applier)(indexstore_unit_include_t)) {
+  auto reader = static_cast<IndexUnitReader *>(rdr);
+  return reader->foreachInclude(
+      [&](const IndexUnitReader::IncludeInfo &incInfo) -> bool {
+        return applier((void *)&incInfo);
+      });
+}
+#endif

tools/IndexStore/IndexStore.exports

+indexstore_error_get_description
+indexstore_error_dispose
+indexstore_format_version
+indexstore_store_create
+indexstore_store_dispose
+indexstore_store_get_unit_modification_time
+indexstore_store_get_unit_name_from_output_path
+indexstore_store_units_apply
+indexstore_store_set_unit_event_handler
+indexstore_store_start_unit_event_listening
+indexstore_store_stop_unit_event_listening
+indexstore_store_discard_unit
+indexstore_store_discard_record
+indexstore_store_purge_stale_data
+indexstore_symbol_get_kind
+indexstore_symbol_get_language
+indexstore_symbol_get_properties
+indexstore_symbol_get_roles
+indexstore_symbol_get_related_roles
+indexstore_symbol_get_subkind
+indexstore_symbol_get_name
+indexstore_symbol_get_usr
+indexstore_symbol_get_codegen_name
+indexstore_symbol_relation_get_roles
+indexstore_symbol_relation_get_symbol
+indexstore_occurrence_get_symbol
+indexstore_occurrence_get_roles
+indexstore_occurrence_get_line_col
+indexstore_occurrence_relations_apply
+indexstore_record_reader_create
+indexstore_record_reader_dispose
+indexstore_record_reader_search_symbols
+indexstore_record_reader_symbols_apply
+indexstore_record_reader_occurrences_apply
+indexstore_record_reader_occurrences_in_line_range_apply
+indexstore_record_reader_occurrences_of_symbols_apply
+indexstore_unit_dependency_get_kind
+indexstore_unit_dependency_get_filepath
+indexstore_unit_dependency_get_file_size
+indexstore_unit_dependency_get_modification_time
+indexstore_unit_dependency_get_modulename
+indexstore_unit_dependency_get_name
+indexstore_unit_dependency_is_system
+indexstore_unit_event_get_kind
+indexstore_unit_event_get_modification_time
+indexstore_unit_event_get_unit_name
+indexstore_unit_event_notification_get_event
+indexstore_unit_event_notification_get_events_count
+indexstore_unit_event_notification_is_initial
+indexstore_unit_reader_create
+indexstore_unit_reader_dispose
+indexstore_unit_reader_get_main_file
+indexstore_unit_reader_get_modification_time
+indexstore_unit_reader_get_module_name
+indexstore_unit_reader_get_provider_identifier
+indexstore_unit_reader_get_provider_version
+indexstore_unit_reader_get_working_dir
+indexstore_unit_reader_get_output_file
+indexstore_unit_reader_get_sysroot_path
+indexstore_unit_reader_get_target
+indexstore_unit_reader_dependencies_apply
+indexstore_unit_reader_includes_apply
+indexstore_unit_reader_has_main_file
+indexstore_unit_reader_is_debug_compilation
+indexstore_unit_reader_is_module_unit
+indexstore_unit_reader_is_system_unit
+indexstore_unit_include_get_source_path
+indexstore_unit_include_get_target_path
+indexstore_unit_include_get_source_line

tools/c-index-test/CMakeLists.txt

+include(CheckIncludeFiles)
+
 set(LLVM_LINK_COMPONENTS
   support
 )
 
 add_clang_executable(c-index-test
   c-index-test.c
   core_main.cpp
+  JSONAggregation.cpp
   )
 
+set(INDEXSTORE_LIB)
+set(CINDEXTEST_LIBS)
+if(APPLE)
+  set(INDEXSTORE_LIB IndexStore)
+endif()
+
 if(NOT MSVC)
   set_property(
     SOURCE c-index-test.c
     PROPERTY COMPILE_FLAGS "-std=gnu89"
     )
 endif()
 
 if (LLVM_BUILD_STATIC)
   target_link_libraries(c-index-test
     PRIVATE
     libclang_static
     clangCodeGen
     clangIndex
+    ${CINDEXTEST_LIBS}
   )
 else()
   target_link_libraries(c-index-test
     PRIVATE
     libclang
+    ${INDEXSTORE_LIB}
     clangAST
     clangBasic
     clangCodeGen
+    clangDirectoryWatcher
     clangFrontend
     clangIndex
     clangSerialization
+    ${CINDEXTEST_LIBS}
   )
 endif()
 
 set_target_properties(c-index-test
   PROPERTIES
   LINKER_LANGUAGE CXX)
 
 # If libxml2 is available, make it available for c-index-test.
 if (CLANG_HAVE_LIBXML)
   if ((CMAKE_OSX_SYSROOT) AND (EXISTS ${CMAKE_OSX_SYSROOT}/${LIBXML2_INCLUDE_DIR}))
     include_directories(SYSTEM ${CMAKE_OSX_SYSROOT}/${LIBXML2_INCLUDE_DIR})
   else()
     include_directories(SYSTEM ${LIBXML2_INCLUDE_DIR})
   endif()
   target_link_libraries(c-index-test PRIVATE ${LIBXML2_LIBRARIES})
 endif()
 
+if(APPLE)
+  check_include_files("CoreServices/CoreServices.h" HAVE_CORESERVICES_H)
+  if(HAVE_CORESERVICES_H)
+    target_link_libraries(c-index-test PRIVATE "-framework CoreServices")
+  endif()
+endif()
+
 if (NOT LLVM_INSTALL_TOOLCHAIN_ONLY)
   if(INTERNAL_INSTALL_PREFIX)
     set(INSTALL_DESTINATION "${INTERNAL_INSTALL_PREFIX}/bin")
     set_property(TARGET c-index-test APPEND PROPERTY INSTALL_RPATH
        "@executable_path/../../lib")
   else()
     set(INSTALL_DESTINATION bin)
   endif()

tools/c-index-test/JSONAggregation.h

+//===--- JSONAggregation.h - Index data aggregation in JSON format --------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLS_CINDEXTEST_JSONAGGREGATION_H
+#define LLVM_CLANG_TOOLS_CINDEXTEST_JSONAGGREGATION_H
+
+#include "clang/Basic/LLVM.h"
+
+namespace clang {
+namespace index {
+
+/// Returns true if an error occurred, false otherwise.
+bool aggregateDataAsJSON(StringRef StorePath, raw_ostream &OS);
+
+} // end namespace index
+} // end namespace clang
+
+#endif

tools/c-index-test/JSONAggregation.cpp

+//===--- JSONAggregation.cpp - Index data aggregation in JSON format ------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "JSONAggregation.h"
+#include "indexstore/IndexStoreCXX.h"
+#include "clang/Index/IndexDataStoreSymbolUtils.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/Support/Allocator.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/raw_ostream.h"
+
+using namespace clang;
+using namespace clang::index;
+using namespace indexstore;
+using namespace llvm;
+
+#if INDEXSTORE_HAS_BLOCKS
+
+namespace {
+
+typedef size_t FilePathIndex;
+typedef size_t RecordIndex;
+typedef size_t SymbolIndex;
+
+struct UnitSourceInfo {
+  FilePathIndex FilePath;
+  SmallVector<RecordIndex, 2> AssociatedRecords;
+};
+
+struct UnitInfo {
+  std::string Name;
+  SmallVector<UnitSourceInfo, 8> Sources;
+  SmallVector<std::string, 3> UnitDepends;
+  FilePathIndex OutFile;
+  StringRef Triple;
+};
+
+struct SymbolInfo {
+  SymbolKind Kind;
+  SymbolLanguage Lang;
+  StringRef USR;
+  StringRef Name;
+  StringRef CodegenName;
+  SymbolRoleSet Roles = 0;
+  SymbolRoleSet RelatedRoles = 0;
+};
+
+struct SymbolRelationInfo {
+  SymbolIndex RelatedSymbol;
+  SymbolRoleSet Roles;
+  SymbolRelationInfo(SymbolIndex relSymbol, SymbolRoleSet roles)
+      : RelatedSymbol(relSymbol), Roles(roles) {}
+};
+
+struct SymbolOccurrenceInfo {
+  SymbolIndex Symbol;
+  SymbolRoleSet Roles = 0;
+  std::vector<SymbolRelationInfo> Relations;
+  unsigned Line;
+  unsigned Column;
+};
+
+struct RecordInfo {
+  SmallVector<SymbolOccurrenceInfo, 8> Occurrences;
+};
+
+class Aggregator {
+  IndexStore Store;
+
+  BumpPtrAllocator Allocator;
+
+  StringMap<FilePathIndex, BumpPtrAllocator &> FilePathIndices;
+  std::vector<StringRef> FilePaths;
+  StringMap<char, BumpPtrAllocator &> Triples;
+
+  std::vector<std::unique_ptr<UnitInfo>> Units;
+
+  StringMap<RecordIndex, BumpPtrAllocator &> RecordIndices;
+  std::vector<std::unique_ptr<RecordInfo>> Records;
+
+  StringMap<SymbolIndex, BumpPtrAllocator &> SymbolIndices;
+  std::vector<SymbolInfo> Symbols;
+
+public:
+  explicit Aggregator(IndexStore store)
+      : Store(std::move(store)), FilePathIndices(Allocator), Triples(Allocator),
+        RecordIndices(Allocator), SymbolIndices(Allocator) {}
+
+  bool process();
+  void processUnit(StringRef name, IndexUnitReader &UnitReader);
+  void dumpJSON(raw_ostream &OS);
+
+private:
+  StringRef copyStr(StringRef str) {
+    if (str.empty())
+      return StringRef();
+    char *buf = Allocator.Allocate<char>(str.size());
+    std::copy(str.begin(), str.end(), buf);
+    return StringRef(buf, str.size());
+  }
+
+  StringRef getTripleString(StringRef inputTriple) {
+    return Triples.insert(std::make_pair(inputTriple, 0)).first->first();
+  }
+
+  FilePathIndex getFilePathIndex(StringRef path, StringRef workingDir);
+  RecordIndex getRecordIndex(StringRef recordFile);
+  SymbolIndex getSymbolIndex(IndexRecordSymbol sym);
+  std::unique_ptr<RecordInfo> processRecord(StringRef recordFile);
+};
+
+} // anonymous namespace
+
+bool Aggregator::process() {
+  bool succ =
+      Store.foreachUnit(/*sorted=*/true, [&](StringRef unitName) -> bool {
+        std::string error;
+        auto unitReader = IndexUnitReader(Store, unitName, error);
+        if (!unitReader) {
+          errs() << "error opening unit file '" << unitName << "': " << error
+                 << '\n';
+          return false;
+        }
+
+        processUnit(unitName, unitReader);
+        return true;
+      });
+
+  return !succ;
+}
+
+void Aggregator::processUnit(StringRef name, IndexUnitReader &UnitReader) {
+  auto workDir = UnitReader.getWorkingDirectory();
+  auto unit = llvm::make_unique<UnitInfo>();
+  unit->Name = name;
+  unit->Triple = getTripleString(UnitReader.getTarget());
+  unit->OutFile = getFilePathIndex(UnitReader.getOutputFile(), workDir);
+
+  struct DepInfo {
+    UnitSourceInfo source;
+    std::string unitName;
+  };
+  SmallVector<DepInfo, 32> Deps;
+  UnitReader.foreachDependency([&](IndexUnitDependency dep) -> bool {
+    Deps.resize(Deps.size() + 1);
+    auto &depInfo = Deps.back();
+    switch (dep.getKind()) {
+    case IndexUnitDependency::DependencyKind::Unit: {
+      depInfo.unitName = dep.getName();
+      StringRef filePath = dep.getFilePath();
+      if (!filePath.empty())
+        depInfo.source.FilePath = getFilePathIndex(filePath, workDir);
+      break;
+    }
+    case IndexUnitDependency::DependencyKind::Record: {
+      depInfo.source.FilePath = getFilePathIndex(dep.getFilePath(), workDir);
+      RecordIndex recIndex = getRecordIndex(dep.getName());
+      depInfo.source.AssociatedRecords.push_back(recIndex);
+      break;
+    }
+    case IndexUnitDependency::DependencyKind::File:
+      depInfo.source.FilePath = getFilePathIndex(dep.getFilePath(), workDir);
+    }
+    return true;
+  });
+
+  unit->Sources.reserve(Deps.size());
+  for (auto &dep : Deps) {
+    if (!dep.unitName.empty()) {
+      unit->UnitDepends.emplace_back(std::move(dep.unitName));
+    } else {
+      unit->Sources.push_back(std::move(dep.source));
+    }
+  }
+
+  Units.push_back(std::move(unit));
+}
+
+FilePathIndex Aggregator::getFilePathIndex(StringRef path,
+                                           StringRef workingDir) {
+  StringRef absPath;
+  SmallString<128> absPathBuf;
+  if (sys::path::is_absolute(path) || workingDir.empty()) {
+    absPath = path;
+  } else {
+    absPathBuf = workingDir;
+    sys::path::append(absPathBuf, path);
+    absPath = absPathBuf.str();
+  }
+
+  auto pair = FilePathIndices.insert(std::make_pair(absPath, FilePaths.size()));
+  bool wasInserted = pair.second;
+  if (wasInserted) {
+    FilePaths.push_back(pair.first->first());
+  }
+  return pair.first->second;
+}
+
+RecordIndex Aggregator::getRecordIndex(StringRef recordFile) {
+  auto pair = RecordIndices.insert(std::make_pair(recordFile, Records.size()));
+  bool wasInserted = pair.second;
+  if (wasInserted) {
+    Records.push_back(processRecord(recordFile));
+  }
+  return pair.first->second;
+}
+
+std::unique_ptr<RecordInfo> Aggregator::processRecord(StringRef recordFile) {
+  std::string error;
+  auto recordReader = IndexRecordReader(Store, recordFile, error);
+  if (!recordReader) {
+    errs() << "failed reading record file: " << recordFile << '\n';
+    ::exit(1);
+  }
+  auto record = llvm::make_unique<RecordInfo>();
+  recordReader.foreachOccurrence([&](IndexRecordOccurrence idxOccur) -> bool {
+    SymbolIndex symIdx = getSymbolIndex(idxOccur.getSymbol());
+    SymbolInfo &symInfo = Symbols[symIdx];
+    symInfo.Roles |= idxOccur.getRoles();
+    SymbolOccurrenceInfo occurInfo;
+    occurInfo.Symbol = symIdx;
+    idxOccur.foreachRelation([&](IndexSymbolRelation rel) -> bool {
+      SymbolIndex relsymIdx = getSymbolIndex(rel.getSymbol());
+      SymbolInfo &relsymInfo = Symbols[relsymIdx];
+      relsymInfo.RelatedRoles |= rel.getRoles();
+      occurInfo.Relations.emplace_back(relsymIdx, rel.getRoles());
+      return true;
+    });
+    occurInfo.Roles = idxOccur.getRoles();
+    std::tie(occurInfo.Line, occurInfo.Column) = idxOccur.getLineCol();
+    record->Occurrences.push_back(std::move(occurInfo));
+    return true;
+  });
+  return record;
+}
+
+SymbolIndex Aggregator::getSymbolIndex(IndexRecordSymbol sym) {
+  auto pair =
+      SymbolIndices.insert(std::make_pair(sym.getUSR(), Symbols.size()));
+  bool wasInserted = pair.second;
+  if (wasInserted) {
+    SymbolInfo symInfo;
+    symInfo.Kind = getSymbolKind(sym.getKind());
+    symInfo.Lang = getSymbolLanguage(sym.getLanguage());
+    symInfo.USR = pair.first->first();
+    symInfo.Name = copyStr(sym.getName());
+    symInfo.CodegenName = copyStr(sym.getCodegenName());
+    Symbols.push_back(std::move(symInfo));
+  }
+  return pair.first->second;
+}
+
+void Aggregator::dumpJSON(raw_ostream &OS) {
+  OS << "{\n";
+  OS.indent(2) << "\"files\": [\n";
+  for (unsigned i = 0, e = FilePaths.size(); i != e; ++i) {
+    OS.indent(4) << '\"' << FilePaths[i] << '\"';
+    if (i < e - 1)
+      OS << ',';
+    OS << '\n';
+  }
+  OS.indent(2) << "],\n";
+
+  OS.indent(2) << "\"symbols\": [\n";
+  for (unsigned i = 0, e = Symbols.size(); i != e; ++i) {
+    OS.indent(4) << "{\n";
+    SymbolInfo &symInfo = Symbols[i];
+    OS.indent(6) << "\"kind\": \"" << getSymbolKindString(symInfo.Kind)
+                 << "\",\n";
+    OS.indent(6) << "\"lang\": \"" << getSymbolLanguageString(symInfo.Lang)
+                 << "\",\n";
+    OS.indent(6) << "\"usr\": \"" << symInfo.USR << "\",\n";
+    OS.indent(6) << "\"name\": \"" << symInfo.Name << "\",\n";
+    if (!symInfo.CodegenName.empty())
+      OS.indent(6) << "\"codegen\": \"" << symInfo.CodegenName << "\",\n";
+    OS.indent(6) << "\"roles\": \"";
+    printSymbolRoles(symInfo.Roles, OS);
+    OS << '\"';
+    if (symInfo.RelatedRoles != 0) {
+      OS << ",\n";
+      OS.indent(6) << "\"rel-roles\": \"";
+      printSymbolRoles(symInfo.RelatedRoles, OS);
+      OS << '\"';
+    }
+    OS << '\n';
+    OS.indent(4) << "}";
+    if (i < e - 1)
+      OS << ',';
+    OS << '\n';
+  }
+  OS.indent(2) << "],\n";
+
+  OS.indent(2) << "\"records\": [\n";
+  for (unsigned i = 0, e = Records.size(); i != e; ++i) {
+    OS.indent(4) << "{\n";
+    RecordInfo &recInfo = *Records[i];
+    OS.indent(6) << "\"occurrences\": [\n";
+    for (unsigned oi = 0, oe = recInfo.Occurrences.size(); oi != oe; ++oi) {
+      OS.indent(8) << "{\n";
+      SymbolOccurrenceInfo &occurInfo = recInfo.Occurrences[oi];
+      OS.indent(10) << "\"symbol\": " << occurInfo.Symbol << ",\n";
+      OS.indent(10) << "\"line\": " << occurInfo.Line << ",\n";
+      OS.indent(10) << "\"col\": " << occurInfo.Column << ",\n";
+      OS.indent(10) << "\"roles\": \"";
+      printSymbolRoles(occurInfo.Roles, OS);
+      OS << '\"';
+      if (!occurInfo.Relations.empty()) {
+        OS << ",\n";
+        OS.indent(10) << "\"relations\": [\n";
+        for (unsigned ri = 0, re = occurInfo.Relations.size(); ri != re; ++ri) {
+          OS.indent(12) << "{\n";
+          SymbolRelationInfo &relInfo = occurInfo.Relations[ri];
+          OS.indent(14) << "\"symbol\": " << relInfo.RelatedSymbol << ",\n";
+          OS.indent(14) << "\"rel-roles\": \"";
+          printSymbolRoles(relInfo.Roles, OS);
+          OS << "\"\n";
+          OS.indent(12) << "}";
+          if (ri < re - 1)
+            OS << ',';
+          OS << '\n';
+        }
+        OS.indent(10) << "]\n";
+      }
+      OS << '\n';
+      OS.indent(8) << "}";
+      if (oi < oe - 1)
+        OS << ',';
+      OS << '\n';
+    }
+    OS.indent(6) << "]\n";
+    OS.indent(4) << "}";
+    if (i < e - 1)
+      OS << ',';
+    OS << '\n';
+  }
+  OS.indent(2) << "],\n";
+
+  StringMap<size_t> UnitIndicesByName;
+  for (unsigned i = 0, e = Units.size(); i != e; ++i) {
+    UnitInfo &unit = *Units[i];
+    UnitIndicesByName[unit.Name] = i;
+  }
+
+  OS.indent(2) << "\"units\": [\n";
+  for (unsigned i = 0, e = Units.size(); i != e; ++i) {
+    OS.indent(4) << "{\n";
+    UnitInfo &unit = *Units[i];
+    OS.indent(6) << "\"triple\": \"" << unit.Triple << "\",\n";
+    OS.indent(6) << "\"out-file\": " << unit.OutFile << ",\n";
+    if (!unit.UnitDepends.empty()) {
+      OS.indent(6) << "\"unit-dependencies\": [";
+      for (unsigned ui = 0, ue = unit.UnitDepends.size(); ui != ue; ++ui) {
+        OS << UnitIndicesByName[unit.UnitDepends[ui]];
+        if (ui < ue - 1)
+          OS << ", ";
+      }
+      OS << "],\n";
+    }
+    OS.indent(6) << "\"sources\": [\n";
+    for (unsigned si = 0, se = unit.Sources.size(); si != se; ++si) {
+      OS.indent(8) << "{\n";
+      UnitSourceInfo &source = unit.Sources[si];
+      OS.indent(10) << "\"file\": " << source.FilePath;
+      if (!source.AssociatedRecords.empty()) {
+        OS << ",\n";
+        OS.indent(10) << "\"records\": [";
+        for (unsigned ri = 0, re = source.AssociatedRecords.size(); ri != re;
+             ++ri) {
+          OS << source.AssociatedRecords[ri];
+          if (ri < re - 1)
+            OS << ", ";
+        }
+        OS << ']';
+      }
+      OS << '\n';
+      OS.indent(8) << "}";
+      if (si < se - 1)
+        OS << ',';
+      OS << '\n';
+    }
+    OS.indent(6) << "]\n";
+    OS.indent(4) << "}";
+    if (i < e - 1)
+      OS << ',';
+    OS << '\n';
+  }
+  OS.indent(2) << "]\n";
+  OS << "}\n";
+}
+
+bool index::aggregateDataAsJSON(StringRef StorePath, raw_ostream &OS) {
+  std::string error;
+  auto dataStore = IndexStore(StorePath, error);
+  if (!dataStore) {
+    errs() << "error opening store path '" << StorePath << "': " << error
+           << '\n';
+    return true;
+  }
+
+  // Explicitely avoid doing any memory cleanup for aggregator since the process
+  // is going to exit when we are done.
+  Aggregator *aggregator = new Aggregator(std::move(dataStore));
+  bool err = aggregator->process();
+  if (err)
+    return true;
+  aggregator->dumpJSON(OS);
+  return false;
+}
+
+#else
+
+bool index::aggregateDataAsJSON(StringRef StorePath, raw_ostream &OS) {
+  return true;
+}
+#endif

tools/c-index-test/core_main.cpp

 //===-- core_main.cpp - Core Index Tool testbed ---------------------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
+#include "JSONAggregation.h"
+#include "indexstore/IndexStoreCXX.h"
 #include "clang/CodeGen/ObjectFilePCHContainerOperations.h"
+#include "clang/DirectoryWatcher/DirectoryWatcher.h"
 #include "clang/Frontend/ASTUnit.h"
 #include "clang/Frontend/CompilerInstance.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/FrontendAction.h"
 #include "clang/Index/CodegenNameGenerator.h"
 #include "clang/Index/IndexDataConsumer.h"
+#include "clang/Index/IndexDataStoreSymbolUtils.h"
+#include "clang/Index/IndexRecordReader.h"
+#include "clang/Index/IndexUnitReader.h"
 #include "clang/Index/USRGeneration.h"
 #include "clang/Index/UnitIndexDataConsumer.h"
 #include "clang/Index/UnitIndexingAction.h"
 #include "clang/Serialization/ASTReader.h"
 #include "llvm/Support/CommandLine.h"
+#include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/PrettyStackTrace.h"
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/raw_ostream.h"
 
+#define HAVE_CORESERVICES 0
+
+#if defined(__has_include)
+#if __has_include(<CoreServices/CoreServices.h>)
+
+#include <CoreServices/CoreServices.h>
+#undef HAVE_CORESERVICES
+#define HAVE_CORESERVICES 1
+
+#endif
+#endif
+
 using namespace clang;
 using namespace clang::index;
 using namespace llvm;
 
 extern "C" int indextest_core_main(int argc, const char **argv);
 
 namespace {
 
 enum class ActionType {
   None,
   PrintSourceSymbols,
   PrintSourceUnit,
+  PrintRecord,
+  PrintUnit,
+  PrintStoreFormatVersion,
+  AggregateAsJSON,
+  WatchDir,
 };
 
 namespace options {
 
 static cl::OptionCategory IndexTestCoreCategory("index-test-core options");
 
 static cl::opt<ActionType> Action(
     cl::desc("Action:"), cl::init(ActionType::None),
-    cl::values(clEnumValN(ActionType::PrintSourceSymbols,
-                          "print-source-symbols", "Print symbols from source"),
-               clEnumValN(ActionType::PrintSourceUnit, "print-source-unit",
-                          "Print unit info from source")),
+    cl::values(
+        clEnumValN(ActionType::PrintSourceSymbols, "print-source-symbols",
+                   "Print symbols from source"),
+                   "Print unit info from source"),
+        clEnumValN(ActionType::PrintRecord, "print-record",
+                   "Print record file info"),
+        clEnumValN(ActionType::PrintUnit, "print-unit", "Print unit file info"),
+        clEnumValN(ActionType::PrintStoreFormatVersion,
+                   "print-store-format-version", "Print store format version"),
+        clEnumValN(ActionType::AggregateAsJSON, "aggregate-json",
+                   "Aggregate index data in JSON format"),
+        clEnumValN(ActionType::WatchDir, "watch-dir",
+                   "Watch directory for file events")),
     cl::cat(IndexTestCoreCategory));
 
+static cl::opt<std::string> OutputFile("o", cl::desc("output file"),
+                                       cl::cat(IndexTestCoreCategory));
+
+static cl::list<std::string> InputFiles(cl::Positional,
+                                        cl::desc("<filename>..."));
+
 static cl::extrahelp MoreHelp(
   "\nAdd \"-- <compiler arguments>\" at the end to setup the compiler "
   "invocation\n"
 );
 
 static cl::opt<bool>
 DumpModuleImports("dump-imported-module-files",
                cl::desc("Print symbols and input files from imported modules"));
 
 static cl::opt<bool>
 IncludeLocals("include-locals", cl::desc("Print local symbols"));
 
 static cl::opt<std::string>
 ModuleFilePath("module-file",
                cl::desc("Path to module file to print symbols from"));
 static cl::opt<std::string>
   ModuleFormat("fmodule-format", cl::init("raw"),
         cl::desc("Container format for clang modules and PCH, 'raw' or 'obj'"));
 
+static cl::opt<std::string> FilePathAndRange(
+    "filepath", cl::desc("File path that can optionally include a line range"));
 }
 } // anonymous namespace
 
 static void printSymbolInfo(SymbolInfo SymInfo, raw_ostream &OS);
 static void printSymbolNameAndUSR(const Decl *D, ASTContext &Ctx,
                                   raw_ostream &OS);
 
 static void printDeclOccurrence(const Decl *D, SymbolRoleSet Roles,
public:
   }
 
   bool handleDeclOccurence(const Decl *D, SymbolRoleSet Roles,
                            ArrayRef<SymbolRelation> Relations,
                            SourceLocation Loc, bool IsInSystemFile,
                            ASTNodeInfo ASTNode) override {
     ASTContext &Ctx = D->getASTContext();
     SourceManager &SM = Ctx.getSourceManager();
-
     Loc = SM.getFileLoc(Loc);
     FileID FID = SM.getFileID(Loc);
     unsigned Offset = SM.getFileOffset(Loc);
     printDeclOccurrence(D, Roles, Relations, FID, Offset, IsInSystemFile,
                         *CGNameGen, OS);
     return true;
   }
 
static bool printSourceUnit(ArrayRef<const char *> Args, bool IndexLocals,
 
   auto PCHContainerOps = std::make_shared<PCHContainerOperations>();
   std::unique_ptr<ASTUnit> Unit(ASTUnit::LoadFromCompilerInvocationAction(
       std::move(CInvok), PCHContainerOps, Diags, IndexAction.get()));
 
   return !Unit;
 }
 
+#if INDEXSTORE_HAS_BLOCKS
+
+//===----------------------------------------------------------------------===//
+// Print Record
+//===----------------------------------------------------------------------===//
+
+static void printSymbol(const IndexRecordDecl &Rec, raw_ostream &OS);
+static void printSymbol(const IndexRecordOccurrence &Rec, raw_ostream &OS);
+
+static int printRecord(StringRef Filename, raw_ostream &OS) {
+  std::string Error;
+  auto Reader = IndexRecordReader::createWithFilePath(Filename, Error);
+  if (!Reader) {
+    errs() << Error << '\n';
+    return true;
+  }
+
+  Reader->foreachDecl(/*noCache=*/true,
+                      [&](const IndexRecordDecl *Rec) -> bool {
+                        printSymbol(*Rec, OS);
+                        return true;
+                      });
+  OS << "------------\n";
+  Reader->foreachOccurrence([&](const IndexRecordOccurrence &Rec) -> bool {
+    printSymbol(Rec, OS);
+    return true;
+  });
+
+  return false;
+};
+
+//===----------------------------------------------------------------------===//
+// Print Store Records
+//===----------------------------------------------------------------------===//
+
+static void printSymbol(indexstore::IndexRecordSymbol Sym, raw_ostream &OS);
+static void printSymbol(indexstore::IndexRecordOccurrence Occur,
+                        raw_ostream &OS);
+
+static bool printStoreRecord(indexstore::IndexStore &Store, StringRef RecName,
+                             StringRef FilePath, raw_ostream &OS) {
+  std::string Error;
+  indexstore::IndexRecordReader Reader(Store, RecName, Error);
+  if (!Reader) {
+    errs() << "error loading record: " << Error << "\n";
+    return true;
+  }
+
+  StringRef Filename = sys::path::filename(FilePath);
+  OS << Filename << '\n';
+  OS << "------------\n";
+  Reader.foreachSymbol(/*noCache=*/true,
+                       [&](indexstore::IndexRecordSymbol Sym) -> bool {
+                         printSymbol(Sym, OS);
+                         return true;
+                       });
+  OS << "------------\n";
+  Reader.foreachOccurrence(
+      [&](indexstore::IndexRecordOccurrence Occur) -> bool {
+        printSymbol(Occur, OS);
+        return true;
+      });
+
+  return false;
+}
+
+static int printStoreRecords(StringRef StorePath, raw_ostream &OS) {
+  std::string Error;
+  indexstore::IndexStore Store(StorePath, Error);
+  if (!Store) {
+    errs() << "error loading store: " << Error << "\n";
+    return 1;
+  }
+
+  bool Success =
+      Store.foreachUnit(/*sorted=*/true, [&](StringRef UnitName) -> bool {
+        indexstore::IndexUnitReader Reader(Store, UnitName, Error);
+        if (!Reader) {
+          errs() << "error loading unit: " << Error << "\n";
+          return false;
+        }
+        return Reader.foreachDependency(
+            [&](indexstore::IndexUnitDependency Dep) -> bool {
+              if (Dep.getKind() ==
+                  indexstore::IndexUnitDependency::DependencyKind::Record) {
+                bool Err = printStoreRecord(Store, Dep.getName(),
+                                            Dep.getFilePath(), OS);
+                OS << '\n';
+                return !Err;
+              }
+              return true;
+            });
+      });
+
+  return !Success;
+}
+
+static std::string findRecordNameForFile(indexstore::IndexStore &store,
+                                         StringRef filePath) {
+  std::string recName;
+  store.foreachUnit(/*sorted=*/false, [&](StringRef unitName) -> bool {
+    std::string error;
+    indexstore::IndexUnitReader Reader(store, unitName, error);
+    if (!Reader) {
+      errs() << "error loading unit: " << error << "\n";
+      return false;
+    }
+    Reader.foreachDependency([&](indexstore::IndexUnitDependency Dep) -> bool {
+      if (Dep.getKind() ==
+          indexstore::IndexUnitDependency::DependencyKind::Record) {
+        if (Dep.getFilePath() == filePath) {
+          recName = Dep.getName();
+          return false;
+        }
+        return true;
+      }
+      return true;
+    });
+    return true;
+  });
+  return recName;
+}
+
+static int printStoreFileRecord(StringRef storePath, StringRef filePath,
+                                Optional<unsigned> lineStart,
+                                unsigned lineCount, raw_ostream &OS) {
+  std::string error;
+  indexstore::IndexStore store(storePath, error);
+  if (!store) {
+    errs() << "error loading store: " << error << "\n";
+    return 1;
+  }
+
+  std::string recName = findRecordNameForFile(store, filePath);
+  if (recName.empty()) {
+    errs() << "could not find record for '" << filePath << "'\n";
+    return 1;
+  }
+
+  if (!lineStart.hasValue())
+    return printStoreRecord(store, recName, filePath, OS);
+
+  indexstore::IndexRecordReader Reader(store, recName, error);
+  if (!Reader) {
+    errs() << "error loading record: " << error << "\n";
+    return 1;
+  }
+
+  Reader.foreachOccurrenceInLineRange(
+      *lineStart, lineCount,
+      [&](indexstore::IndexRecordOccurrence Occur) -> bool {
+        printSymbol(Occur, OS);
+        return true;
+      });
+
+  return 0;
+}
+
+//===----------------------------------------------------------------------===//
+// Print Unit
+//===----------------------------------------------------------------------===//
+
+static int printUnit(StringRef Filename, raw_ostream &OS) {
+  std::string Error;
+  auto Reader = IndexUnitReader::createWithFilePath(Filename, Error);
+  if (!Reader) {
+    errs() << Error << '\n';
+    return true;
+  }
+
+  OS << "provider: " << Reader->getProviderIdentifier() << '-'
+     << Reader->getProviderVersion() << '\n';
+  OS << "is-system: " << Reader->isSystemUnit() << '\n';
+  OS << "is-module: " << Reader->isModuleUnit() << '\n';
+  OS << "module-name: "
+     << (Reader->getModuleName().empty() ? "<none>" : Reader->getModuleName())
+     << '\n';
+  OS << "has-main: " << Reader->hasMainFile() << '\n';
+  OS << "main-path: " << Reader->getMainFilePath() << '\n';
+  OS << "work-dir: " << Reader->getWorkingDirectory() << '\n';
+  OS << "out-file: " << Reader->getOutputFile() << '\n';
+  OS << "target: " << Reader->getTarget() << '\n';
+  OS << "is-debug: " << Reader->isDebugCompilation() << '\n';
+  OS << "DEPEND START\n";
+  unsigned NumDepends = 0;
+  Reader->foreachDependency(
+      [&](const IndexUnitReader::DependencyInfo &Dep) -> bool {
+        switch (Dep.Kind) {
+        case IndexUnitReader::DependencyKind::Unit:
+          OS << "Unit | ";
+          break;
+        case IndexUnitReader::DependencyKind::Record:
+          OS << "Record | ";
+          break;
+        case IndexUnitReader::DependencyKind::File:
+          OS << "File | ";
+          break;
+        }
+        OS << (Dep.IsSystem ? "system" : "user");
+        OS << " | ";
+        if (!Dep.ModuleName.empty())
+          OS << Dep.ModuleName << " | ";
+        OS << Dep.FilePath << " | " << Dep.UnitOrRecordName << " | ";
+        OS << Dep.ModTime << " | " << Dep.FileSize << '\n';
+        ++NumDepends;
+        return true;
+      });
+  OS << "DEPEND END (" << NumDepends << ")\n";
+  OS << "INCLUDE START\n";
+  unsigned NumIncludes = 0;
+  Reader->foreachInclude([&](const IndexUnitReader::IncludeInfo &Inc) -> bool {
+    OS << Inc.SourcePath << ":" << Inc.SourceLine << " | ";
+    OS << Inc.TargetPath << '\n';
+    ++NumIncludes;
+    return true;
+  });
+  OS << "INCLUDE END (" << NumIncludes << ")\n";
+
+  return false;
+};
+
+//===----------------------------------------------------------------------===//
+// Print Store Units
+//===----------------------------------------------------------------------===//
+
+static bool printStoreUnit(indexstore::IndexStore &Store, StringRef UnitName,
+                           raw_ostream &OS) {
+  std::string Error;
+  indexstore::IndexUnitReader Reader(Store, UnitName, Error);
+  if (!Reader) {
+    errs() << "error loading unit: " << Error << "\n";
+    return true;
+  }
+
+  OS << "provider: " << Reader.getProviderIdentifier() << '-'
+     << Reader.getProviderVersion() << '\n';
+  OS << "is-system: " << Reader.isSystemUnit() << '\n';
+  OS << "is-module: " << Reader.isModuleUnit() << '\n';
+  OS << "module-name: "
+     << (Reader.getModuleName().empty() ? "<none>" : Reader.getModuleName())
+     << '\n';
+  OS << "has-main: " << Reader.hasMainFile() << '\n';
+  OS << "main-path: " << Reader.getMainFilePath() << '\n';
+  OS << "work-dir: " << Reader.getWorkingDirectory() << '\n';
+  OS << "out-file: " << Reader.getOutputFile() << '\n';
+  OS << "target: " << Reader.getTarget() << '\n';
+  OS << "is-debug: " << Reader.isDebugCompilation() << '\n';
+  OS << "DEPEND START\n";
+  unsigned NumDepends = 0;
+  Reader.foreachDependency([&](indexstore::IndexUnitDependency Dep) -> bool {
+    switch (Dep.getKind()) {
+    case indexstore::IndexUnitDependency::DependencyKind::Unit:
+      OS << "Unit | ";
+      break;
+    case indexstore::IndexUnitDependency::DependencyKind::Record:
+      OS << "Record | ";
+      break;
+    case indexstore::IndexUnitDependency::DependencyKind::File:
+      OS << "File | ";
+      break;
+    }
+    OS << (Dep.isSystem() ? "system" : "user");
+    OS << " | ";
+    if (!Dep.getModuleName().empty())
+      OS << Dep.getModuleName() << " | ";
+    OS << Dep.getFilePath() << " | " << Dep.getName() << " | ";
+    OS << Dep.getModificationTime() << '\n';
+    ++NumDepends;
+    return true;
+  });
+  OS << "DEPEND END (" << NumDepends << ")\n";
+  OS << "INCLUDE START\n";
+  unsigned NumIncludes = 0;
+  Reader.foreachInclude([&](indexstore::IndexUnitInclude Inc) -> bool {
+    OS << Inc.getSourcePath() << ":" << Inc.getSourceLine() << " | ";
+    OS << Inc.getTargetPath() << '\n';
+    ++NumIncludes;
+    return true;
+  });
+  OS << "INCLUDE END (" << NumIncludes << ")\n";
+
+  return false;
+}
+
+static int printStoreUnits(StringRef StorePath, raw_ostream &OS) {
+  std::string Error;
+  indexstore::IndexStore Store(StorePath, Error);
+  if (!Store) {
+    errs() << "error loading store: " << Error << "\n";
+    return 1;
+  }
+
+  bool Success =
+      Store.foreachUnit(/*sorted=*/true, [&](StringRef UnitName) -> bool {
+        OS << UnitName << '\n';
+        OS << "--------\n";
+        bool err = printStoreUnit(Store, UnitName, OS);
+        OS << '\n';
+        return !err;
+      });
+
+  return !Success;
+}
+
+#else
+
+static int printUnit(StringRef Filename, raw_ostream &OS) { return 1; }
+
+static int printStoreUnits(StringRef StorePath, raw_ostream &OS) { return 1; }
+
+static int printStoreFileRecord(StringRef storePath, StringRef filePath,
+                                Optional<unsigned> lineStart,
+                                unsigned lineCount, raw_ostream &OS) {
+  return 1;
+}
+
+#endif
+
 //===----------------------------------------------------------------------===//
 // Helper Utils
 //===----------------------------------------------------------------------===//
 
 static void printSymbolInfo(SymbolInfo SymInfo, raw_ostream &OS) {
   OS << getSymbolKindString(SymInfo.Kind);
   if (SymInfo.SubKind != SymbolSubKind::None)
     OS << '/' << getSymbolSubKindString(SymInfo.SubKind);
static void printSymbolNameAndUSR(const Decl *D, ASTContext &Ctx,
   SmallString<256> USRBuf;
   if (generateUSRForDecl(D, USRBuf)) {
     OS << "<no-usr>";
   } else {
     OS << USRBuf;
   }
 }
 
+#if INDEXSTORE_HAS_BLOCKS
+
+static void printSymbol(const IndexRecordDecl &Rec, raw_ostream &OS) {
+  printSymbolInfo(Rec.SymInfo, OS);
+  OS << " | ";
+
+  if (Rec.Name.empty())
+    OS << "<no-name>";
+  else
+    OS << Rec.Name;
+  OS << " | ";
+
+  if (Rec.USR.empty())
+    OS << "<no-usr>";
+  else
+    OS << Rec.USR;
+  OS << " | ";
+
+  if (Rec.CodeGenName.empty())
+    OS << "<no-cgname>";
+  else
+    OS << Rec.CodeGenName;
+  OS << " | ";
+
+  printSymbolRoles(Rec.Roles, OS);
+  OS << " - ";
+  printSymbolRoles(Rec.RelatedRoles, OS);
+  OS << '\n';
+}
+
+static void printSymbol(const IndexRecordOccurrence &Rec, raw_ostream &OS) {
+  OS << Rec.Line << ':' << Rec.Column << " | ";
+  printSymbolInfo(Rec.Dcl->SymInfo, OS);
+  OS << " | ";
+
+  if (Rec.Dcl->USR.empty())
+    OS << "<no-usr>";
+  else
+    OS << Rec.Dcl->USR;
+  OS << " | ";
+
+  printSymbolRoles(Rec.Roles, OS);
+  OS << " | ";
+  OS << "rel: " << Rec.Relations.size() << '\n';
+  for (auto &Rel : Rec.Relations) {
+    OS << '\t';
+    printSymbolRoles(Rel.Roles, OS);
+    OS << " | ";
+    if (Rel.Dcl->USR.empty())
+      OS << "<no-usr>";
+    else
+      OS << Rel.Dcl->USR;
+    OS << '\n';
+  }
+}
+
+static void printSymbol(indexstore::IndexRecordSymbol Sym, raw_ostream &OS) {
+  SymbolInfo SymInfo{getSymbolKind(Sym.getKind()),
+                     getSymbolSubKind(Sym.getSubKind()),
+                     getSymbolLanguage(Sym.getLanguage()),
+                     SymbolPropertySet(Sym.getProperties())};
+
+  printSymbolInfo(SymInfo, OS);
+  OS << " | ";
+
+  if (Sym.getName().empty())
+    OS << "<no-name>";
+  else
+    OS << Sym.getName();
+  OS << " | ";
+
+  if (Sym.getUSR().empty())
+    OS << "<no-usr>";
+  else
+    OS << Sym.getUSR();
+  OS << " | ";
+
+  if (Sym.getCodegenName().empty())
+    OS << "<no-cgname>";
+  else
+    OS << Sym.getCodegenName();
+  OS << " | ";
+
+  printSymbolRoles(Sym.getRoles(), OS);
+  OS << " - ";
+  printSymbolRoles(Sym.getRelatedRoles(), OS);
+  OS << '\n';
+}
+
+static void printSymbol(indexstore::IndexRecordOccurrence Occur,
+                        raw_ostream &OS) {
+  OS << Occur.getLineCol().first << ':' << Occur.getLineCol().second << " | ";
+  auto Sym = Occur.getSymbol();
+  SymbolInfo SymInfo{getSymbolKind(Sym.getKind()),
+                     getSymbolSubKind(Sym.getSubKind()),
+                     getSymbolLanguage(Sym.getLanguage()),
+                     SymbolPropertySet(Sym.getProperties())};
+
+  printSymbolInfo(SymInfo, OS);
+  OS << " | ";
+
+  if (Sym.getUSR().empty())
+    OS << "<no-usr>";
+  else
+    OS << Sym.getUSR();
+  OS << " | ";
+
+  unsigned NumRelations = 0;
+  Occur.foreachRelation([&](indexstore::IndexSymbolRelation) {
+    ++NumRelations;
+    return true;
+  });
+
+  printSymbolRoles(Occur.getRoles(), OS);
+  OS << " | ";
+  OS << "rel: " << NumRelations << '\n';
+  Occur.foreachRelation([&](indexstore::IndexSymbolRelation Rel) {
+    OS << '\t';
+    printSymbolRoles(Rel.getRoles(), OS);
+    OS << " | ";
+    auto Sym = Rel.getSymbol();
+    if (Sym.getUSR().empty())
+      OS << "<no-usr>";
+    else
+      OS << Sym.getUSR();
+    OS << '\n';
+    return true;
+  });
+}
+
+#else
+
+static int printRecord(StringRef Filename, raw_ostream &OS) { return 1; }
+static int printStoreRecords(StringRef StorePath, raw_ostream &OS) { return 1; }
+
+#endif
+
+static int watchDirectory(StringRef dirPath) {
+  raw_ostream &OS = outs();
+  auto receiver = [&](ArrayRef<DirectoryWatcher::Event> Events,
+                      bool isInitial) {
+    for (auto evt : Events) {
+      switch (evt.Kind) {
+      case DirectoryWatcher::EventKind::Added:
+        OS << "added: ";
+        break;
+      case DirectoryWatcher::EventKind::Modified:
+        OS << "modified: ";
+        break;
+      case DirectoryWatcher::EventKind::Removed:
+        OS << "removed: ";
+        break;
+      case DirectoryWatcher::EventKind::DirectoryDeleted:
+        OS << "dir deleted: ";
+        break;
+      }
+      OS << evt.Filename << '\n';
+    }
+  };
+  std::string Error;
+  auto watcher = DirectoryWatcher::create(dirPath, receiver,
+                                          /*waitInitialSync=*/true, Error);
+  if (!watcher) {
+    errs() << "failed creating directory watcher: " << Error << '\n';
+    return 1;
+  }
+#if HAVE_CORESERVICES
+  dispatch_main();
+#else
+  return 1;
+#endif
+}
+
 //===----------------------------------------------------------------------===//
 // Command line processing.
 //===----------------------------------------------------------------------===//
 
+bool deconstructPathAndRange(StringRef input, std::string &filepath,
+                             Optional<unsigned> &lineStart,
+                             unsigned &lineCount) {
+  StringRef path, range;
+  std::tie(path, range) = input.split(':');
+  StringRef start, end;
+  std::tie(start, end) = range.split(':');
+  filepath = path;
+  lineCount = 0;
+  if (start.empty())
+    return false;
+  unsigned num;
+  if (start.getAsInteger(10, num)) {
+    errs() << "couldn't convert to integer: " << start << '\n';
+    return true;
+  }
+  lineStart = num;
+  if (end.empty())
+    return false;
+  if (end.getAsInteger(10, num)) {
+    errs() << "couldn't convert to integer: " << end << '\n';
+    return true;
+  }
+  lineCount = num - lineStart.getValue();
+  return false;
+}
+
 int indextest_core_main(int argc, const char **argv) {
   sys::PrintStackTraceOnErrorSignal(argv[0]);
   PrettyStackTraceProgram X(argc, argv);
 
   assert(argv[1] == StringRef("core"));
   ++argv;
   --argc;
 
if (CompArgs.empty()) {
       errs()
           << "error: missing compiler args; pass '-- <compiler arguments>'\n";
       return 1;
     }
     return printSourceUnit(CompArgs, options::IncludeLocals,
                            /*IndexModDepedencies=*/true);
   }
 
+  if (options::Action == ActionType::PrintRecord) {
+    if (!options::FilePathAndRange.empty()) {
+      std::string filepath;
+      Optional<unsigned> lineStart;
+      unsigned lineCount;
+      if (deconstructPathAndRange(options::FilePathAndRange, filepath,
+                                  lineStart, lineCount))
+        return 1;
+
+      if (options::InputFiles.empty()) {
+        errs() << "error: missing index store path\n";
+        return 1;
+      }
+      return printStoreFileRecord(options::InputFiles[0], filepath, lineStart,
+                                  lineCount, outs());
+    }
+
+    if (options::InputFiles.empty()) {
+      errs() << "error: missing input file or directory\n";
+      return 1;
+    }
+
+    if (sys::fs::is_directory(options::InputFiles[0]))
+      return printStoreRecords(options::InputFiles[0], outs());
+    else
+      return printRecord(options::InputFiles[0], outs());
+  }
+
+  if (options::Action == ActionType::PrintUnit) {
+    if (options::InputFiles.empty()) {
+      errs() << "error: missing input file or directory\n";
+      return 1;
+    }
+
+    if (sys::fs::is_directory(options::InputFiles[0]))
+      return printStoreUnits(options::InputFiles[0], outs());
+    else
+      return printUnit(options::InputFiles[0], outs());
+  }
+
+#if INDEXSTORE_HAS_BLOCKS
+  if (options::Action == ActionType::PrintStoreFormatVersion) {
+    outs() << indexstore::IndexStore::formatVersion() << '\n';
+  }
+#endif
+
+  if (options::Action == ActionType::AggregateAsJSON) {
+    if (options::InputFiles.empty()) {
+      errs() << "error: missing input data store directory\n";
+      return 1;
+    }
+    StringRef storePath = options::InputFiles[0];
+    if (options::OutputFile.empty())
+      return aggregateDataAsJSON(storePath, outs());
+    std::error_code EC;
+    raw_fd_ostream OS(options::OutputFile, EC, llvm::sys::fs::F_None);
+    if (EC) {
+      errs() << "failed to open output file: " << EC.message() << '\n';
+      return 1;
+    }
+    return aggregateDataAsJSON(storePath, OS);
+  }
+
+  if (options::Action == ActionType::WatchDir) {
+    if (options::InputFiles.empty()) {
+      errs() << "error: missing directory path\n";
+      return 1;
+    }
+    return watchDirectory(options::InputFiles[0]);
+  }
+
   return 0;
 }