Added the ability to cache the finalized symbol tables subsequent debug sessions to start faster.

[clayborg]

Summary

This is an updated version of the https://reviews.llvm.org/D113789 patch with the following changes:

• We no longer modify modification times of the cache files
• Use LLVM caching and cache pruning instead of making a new cache mechanism (See DataFileCache.h/.cpp)
• Add signature to start of each file since we are not using modification times so we can tell when caches are stale and remove and re-create the cache file as files are changed
• Add settings to control the cache size, disk percentage and expiration in days to keep cache size under control

This patch enables symbol tables to be cached in the LLDB index cache directory. All cache files are in a single directory and the files use unique names to ensure that files from the same path will re-use the same file as files get modified. This means as files change, their cache files will be deleted and updated. The modification time of each of the cache files is not modified so that access based pruning of the cache can be implemented.

The symbol table cache files start with a signature that uniquely identifies a file on disk and contains one or more of the following items:

• object file UUID if available
• object file mod time if available
• object name for BSD archive .o files that are in .a files if available

If none of these signature items are available, then the file will not be cached. This keeps temporary object files from expressions from being cached.

When the cache files are loaded on subsequent debug sessions, the signature is compare and if the file has been modified (uuid changes, mod time changes, or object file mod time changes) then the cache file is deleted and re-created.

Module caching must be enabled by the user before this can be used:

symbols.enable-lldb-index-cache (boolean) = false

(lldb) settings set symbols.enable-lldb-index-cache true

There is also a setting that allows the user to specify a module cache directory that defaults to a directory that defaults to being next to the symbols.clang-modules-cache-path directory in a temp directory:

(lldb) settings show symbols.lldb-index-cache-path
/var/folders/9p/472sr0c55l9b20x2zg36b91h0000gn/C/lldb/IndexCache

If this setting is enabled, the finalized symbol tables will be serialized and saved to disc so they can be quickly loaded next time you debug.

Each module can cache one or more files in the index cache directory. The cache file names must be unique to a file on disk and its architecture and object name for .o files in BSD archives. This allows universal mach-o files to support caching multuple architectures in the same module cache directory. Making the file based on the this info allows this cache file to be deleted and replaced when the file gets updated on disk. This keeps the cache from growing over time during the compile/edit/debug cycle and prevents out of space issues.

If the cache is enabled, the symbol table will be loaded from the cache the next time you debug if the module has not changed.

The cache also has settings to control the size of the cache on disk. Each time LLDB starts up with the index cache enable, the cache will be pruned to ensure it stays within the user defined settings:

(lldb) settings set symbols.lldb-index-cache-expiration-days <days>

A value of zero will disable cache files from expiring when the cache is pruned. The default value is 7 currently.

(lldb) settings set symbols.lldb-index-cache-max-byte-size <size>

A value of zero will disable pruning based on a total byte size. The default value is zero currently.
(lldb) settings set symbols.lldb-index-cache-max-percent <percentage-of-disk-space>

A value of 100 will allow the disc to be filled to the max, a value of zero will disable percentage pruning. The default value is zero.

[clayborg]

Added string tables to the cache files.

This allowed me to add the symbol table name indexes to the same symbol table cache file and share any strings from the normal symbol table.

[wallace]

much nicer than the first version. I'm just asking a few minor things.

lldb/include/lldb/Host/FileSystem.h

147

lldb/include/lldb/Symbol/Symbol.h

256

better mention that the return value means

lldb/include/lldb/Symbol/Symtab.h

227

ditto

lldb/include/lldb/Utility/DataFileCache.h

59 ↗

(On Diff #394041)

175 ↗

(On Diff #394041)

I'd just call this ConstStringTable for callers to know that this is using ConstString as its storage pool and that the data doesn't just go away when the object is disposed.

lldb/source/Utility/DataFileCache.cpp

58 ↗

(On Diff #394041)

could you create a new lldb log channel where this information is logged? it'll help investigating issues with the cache (if they ever happen). Same for everywhere llvm::Error's or lldb::Status messages are lost.

[clayborg]

Address all review comments.

[clayborg]

lldb/source/Utility/DataFileCache.cpp

58 ↗

(On Diff #394041)

I didn't create a new channel because we are out of log bits. I can expand the log bits to 64 in a separate patch.

[wallace]

lgtm

[labath]

The patch is slightly larger than I would prefer for a through review, but here's my first pass at it. I appreciate the difficulties in bootstrapping something like this incrementally, but I do see at least a couple of opportunities to make things smaller. I think the DataEncoder, FileSystem, Symbol, and Mangled changes could go into separate patch(es). I don't know if it's feasible to separate the construction of the symtab cache content, from the actual act of storing it in the cache, but if those two processes could be separated, it would help things a lot.

lldb/source/Host/common/FileSystem.cpp

523

What's the purpose of this check ?

lldb/source/Symbol/Symbol.cpp

645

This is quite dodgy. It would be better to change the storage so that we don't have to do this. Perhaps something like

union {
  uint16_t flag_storage;
  struct {
    uint16_t m_type_data_resolved : 1;
    ...
  };
};

lldb/source/Symbol/Symtab.cpp

1187

constexpr llvm::StringLiteral

1223

constexpr uint32_t

1245–1249

I am troubled by code like this. Have you tried creating a mock object file in the test?

1331

please clang-format the patch

lldb/source/Utility/DataFileCache.cpp

10–13 ↗

(On Diff #394141)

These are not appropriate dependencies for code in the utility library. Either reimplement it without them or move the class somewhere else.

51 ↗

(On Diff #394141)

[&] on a lambda that outlives the enclosing function is dangerous.

61–64 ↗

(On Diff #394141)

there's an LLDB_LOG_ERROR macro for this

[clayborg]

Changes:

• Updated Symtab class to not have to work around not having an object file and created a test with an object file
• Don't use hack to encode Symbol bitfield values.
• Remove file system IsDirectory check that wasn't needed
• Switch to use constexpr llvm::StringLiteral where requested
• Switch to use constexpr uint32_t for CURRENT_CACHE_VERSION
• Moved DataFileCache to Core since it relies on ModuleList's properties
• Fixed unsafe capture in lambda
• Switch log calls in DataFileCache.cpp to use LLDB_LOG_ERROR
• Updated test to create a module/object file from yaml to avoid having to hack Symtab to deal with NULL m_objfile

[clayborg]

Pavel: I fixed all issues you identified. Let me know if there is anything else you would like to see changed.

lldb/source/Host/common/FileSystem.cpp

523

I checked the llvm code, and it already correctly handles not deleting a file if it is a directory, so this isn't needed.

lldb/source/Symbol/Symbol.cpp

645

I encoded them manually now. It was too disruptive to the code to try and add an anonymous union with an anonymous struct as there were many compiler warnings about it being a GNU extension and made the code way too messy, especially in the constructor as you can't init each value easily.

[labath]

Thanks for the quick response.

I did another pass over the patch, and I think I have a fairly good understanding of how it works. I have another round of comments, but nothing major.

lldb/include/lldb/Core/DataFileCache.h

10

130–133

the Optional class already has comparison operators that do the right thing.

lldb/include/lldb/Core/Mangled.h

72–73

lldb/source/Core/Module.cpp

1678

The cache key and cache signature algorithms are fairly similar but subtly different (IIUC, one is used to locate the cache entry on the disk and the second one to verify that it refers to the correct file). I think it'd be better to have the two functions next to each other, to highlight the differences. If you moved the signature construction function here (by changing it from a Module constructor to a CacheSignature getter function), then that would also the dependency problem I mentioned last time as I believe these constructors were the only non-utility dependency of that code (I don't think that code has to be moved back to Utility, but it *could* be moved, it need arises).

lldb/source/Symbol/Symbol.cpp

645

That works too.

649

I guess this isn't a "trick" anymore, just manual encoding of boolean bitfields into an int.

lldb/source/Symbol/Symtab.cpp

118–147

If I managed to understand what this is doing correctly, it could be more simply implemented as:

if (pair.second.IsEmpty())
  continue;
s->Printf("\n0x%2.2x name to index table:\n", pair.first);
std::map<uint32_t, std::vector<ConstString>> index_to_names;
for (const auto &entry: pair.second)
  index_to_names[entry.value].push_back(entry.cstring); // automatically creates a new map entry for the first string
for (const auto &pair: index_to_names) {
  s->Printf("0x%8.8x: ", pair.first);
  for (const auto &name: pair.second) // vector always non-empty
    s->Printf("\"%s\" ", name.AsCString());
  s->EOL();
}

272

I guess this is no longer relevant.

1182

this is not a good use of auto per the llvm coding standards

1189

add static (for the next function as well)

lldb/test/API/functionalities/module_cache/universal/TestModuleCacheUniversal.py

22

leftover from debugging?

58–59

IIUC, the python tests check the caching behavior, while the (c++) unit tests check that the actual contents of the cache entries can be read/written correctly. I like that.

lldb/unittests/Symbol/SymbolTest.cpp

186

Our c++ test layout mirrors the layout of the code it is testing, so this test should go to MangledTest.cpp and the next one to SymtabTest.cpp. (Although it might be nicer if all the (de)serialization code lived in a single central place, and then the tests for it would live in a single place as well.)

[clayborg]

• Fixed all review issues
• Moved tests to separate test files in SymbolTests

[clayborg]

lldb/source/Core/Module.cpp

1678

So the signature only contains an optional UUID, optional mod time, and optional object mod time, so they really are not related. The cache key must be the same for the same file on disk even if the file gets updated. So the cache key is designed to stay the same for the same file on disk, but also allow for the same file on disk to have more than one architecture (universal files) or for a file to contain multiple files like .a files that contain .o files. Since we only include the basename of the path for the module file, we also include the hash from Module::Hash() which _does_ include the full file path. That way we can have many "a.out" cache files from different directories and the main thing that will differ is the hash that is appended at the end.

The DataCacheFile.cpp was relying on the ModuleList.cpp in order to read the "symbols.* preferences via "ModuleList::GetGlobalModuleListProperties()", so the ModuleList is actually what was the part that violated the dependency issue.

lldb/source/Symbol/Symbol.cpp

649

It is. I tried doing your approach of making an anonymous union and an anonymous struct but I got a bunch of warnings that these were GNU extensions and I feared that some buildbots would fail with warnings as errors. I also didn't know if this would mess up the windows buildbots in case they didn't support these. So to be safe I just manually encode stuff to be safe.

lldb/source/Symbol/Symtab.cpp

118–147

Whoops this was just dumping code I added to verify that the name tables were being reproduced correctly. No one will want to see these. I will remove the code.

lldb/test/API/functionalities/module_cache/universal/TestModuleCacheUniversal.py

22

yes! good catch

58–59

Exactly. I wanted to make sure that encoding could be done with unit testing as that is easier. I could also verify that every bit in the Symbol class was encoded and decoded properly since I was playing tricks before with the bitfields, but it did help issues when I did things manually as well.

[clayborg]

Pavel: I fixed all issues you identified. Let me know if there is anything else you would like to see changed.

[labath]

I think this is in a pretty good shape now. Thanks for your patience.

lldb/source/Symbol/Symbol.cpp

649

Yes, I saw what you're doing, and that's fine. My point was that I would use the word "trick" to describe something clever/subtle/dangerous (which the original implementation was). This isn't tricky, its just tedious. :)

But that's not a big deal.

[stella.stamenova]

The new test (TestModuleCacheSimple) is failing on the windows lldb bot. It looks like the deletion of the executable is failing due to permissions and other similar tests are skipped on windows for that reason. I haven't looked at the test in more detail to see if it would be appropriate to modify somehow so it would work:

https://lab.llvm.org/buildbot/#/builders/83/builds/12914

(The bot is rather angry at me for the VS2019 update, but everything should be working OK again)

[clayborg]

In D115324#3198913, @stella.stamenova wrote:

The new test (TestModuleCacheSimple) is failing on the windows lldb bot. It looks like the deletion of the executable is failing due to permissions and other similar tests are skipped on windows for that reason. I haven't looked at the test in more detail to see if it would be appropriate to modify somehow so it would work:

https://lab.llvm.org/buildbot/#/builders/83/builds/12914

(The bot is rather angry at me for the VS2019 update, but everything should be working OK again)

Fixed with:

commit 59f1d0eed58ce0f23c3a5a324d122a99f9757228 (HEAD -> main, origin/main, origin/HEAD)
Author: Greg Clayton <[e-mail]>
Date: Thu Dec 16 16:13:58 2021 -0800

Fix windows buildbots after https://reviews.llvm.org/D115324

Windows has trouble deleting the executable due to permissions.

In D115324#3198913, @stella.stamenova wrote:

The new test (TestModuleCacheSimple) is failing on the windows lldb bot. It looks like the deletion of the executable is failing due to permissions and other similar tests are skipped on windows for that reason. I haven't looked at the test in more detail to see if it would be appropriate to modify somehow so it would work:

https://lab.llvm.org/buildbot/#/builders/83/builds/12914

(The bot is rather angry at me for the VS2019 update, but everything should be working OK again)

[clayborg]

Some mac buildbots were failing due to "lldb/test/API/functionalities/module_cache/universal/TestModuleCacheUniversal.py" failing to build the executable with the makefile due to lack of arm64 support in the SDK that was being used.

commit 2a844c8869909960fb006a668d30c7349f55ee7e (HEAD -> main, origin/main, origin/HEAD)
Author: Greg Clayton <[e-mail]>
Date: Fri Dec 17 12:14:44 2021 -0800

Fix macOS buildbots after https://reviews.llvm.org/D115324.

The test was attempting to make a universal x86_64/arm64 binary, but some older bots don't have a macOS SDK that can handle this. Switching over to using a yaml file instead should solve the problem.

[clayborg]

The unit testing I added to this patch was unable to catch a serious issue that I have fixed and is needed if anyone wants to enable this feature. So please make sure if you use this patch to also make sure that the following patch is also applied to your repository if you are not using top of tree:

https://reviews.llvm.org/D124572

[dblaikie]

Been experimenting with this recently and I noticed that loading in the cached indexes seems to do a lot of loading - specifically interning a lot of strings from the index and the symtab. Does this happen when reading a built-in index (apple_names/debug_names) (I don't have an immediately easy way to test this, or I Would've before asking)? I'd be surprised if that was the case, which is also confusing me as to why it's the case for these cached indexes? I'd have expected the cached index to look basically the same as the apple_names/debug_names builtin index and have similar performance properties, but maybe that's not the case?

[clayborg]

In D115324#3857081, @dblaikie wrote:

Been experimenting with this recently and I noticed that loading in the cached indexes seems to do a lot of loading - specifically interning a lot of strings from the index and the symtab. Does this happen when reading a built-in index (apple_names/debug_names) (I don't have an immediately easy way to test this, or I Would've before asking)? I'd be surprised if that was the case, which is also confusing me as to why it's the case for these cached indexes? I'd have expected the cached index to look basically the same as the apple_names/debug_names builtin index and have similar performance properties, but maybe that's not the case?

Yeah, the caches are currently designed to just serialize the cache to disc and allow it to be loaded into the same data structure as is used when the cache isn't used.

The DWARF indexes are more efficient where there is a base class that represents the cache, and the manual index will create its own data structures and then do lookups using the base class' API. It allows the API calls to do the lookups as efficiently as possible without interning any strings. An improvement to this caching could do the same kind of thing, but that isn't what we have right now. We would need to continue to serialize/deserialize the symbol table, but the symbol table index and debug info index could be modified to indirect the cache lookups through a base class so the serialized indexes could be made more efficient.

The index cache is purely a serialize/deserialize of the same cache structure that is used when manually parsing and indexing.

lldb/include/lldb/Core/DataFileCache.h

+//===-- DataFileCache.h -----------------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLDB_CORE_DATAFILECACHE_H
+#define LLDB_CORE_DATAFILECACHE_H
+
+#include "lldb/Utility/FileSpec.h"
+#include "lldb/Utility/Status.h"
+#include "lldb/Utility/UUID.h"
+#include "lldb/lldb-forward.h"
+#include "llvm/Support/Caching.h"
+#include <mutex>
+
+namespace lldb_private {
+
+/// This class enables data to be cached into a directory using the llvm
+/// caching code. Data can be stored and accessed using a unique string key.
+/// The data will be stored in the directory that is specified in the
+/// DataFileCache constructor. The data will be stored in files that start with
+/// "llvmcache-<key>" where <key> is the key name specified when getting to
+/// setting cached data.
+///
+/// Sample code for how to use the cache:
+///
+///   DataFileCache cache("/tmp/lldb-test-cache");
+///   StringRef key("Key1");
+///   auto mem_buffer_up = cache.GetCachedData(key);
+///   if (mem_buffer_up) {
+///     printf("cached data:\n%s", mem_buffer_up->getBufferStart());
+///   } else {
+///     std::vector<uint8_t> data = { 'h', 'e', 'l', 'l', 'o', '\n' };
+///     cache.SetCachedData(key, data);
+///   }
+
+class DataFileCache {
+public:
+  /// Create a data file cache in the directory path that is specified.
+  ///
+  /// Data will be cached in files created in this directory when clients call
+  /// DataFileCache::SetCacheData.
+  DataFileCache(llvm::StringRef path);
+
+  /// Get cached data from the cache directory for the specified key.
+  ///
+  /// Keys must be unique for any given data. This function attempts to see if
+  /// the data is available for the specified key and will return a valid memory
+  /// buffer is data is available.
+  ///
+  /// \param key
+  ///   The unique string key that identifies data being cached.
+  ///
+  /// \return
+  ///   A valid unique pointer to a memory buffer if the data is available, or
+  ///   a unique pointer that contains NULL if the data is not available.
+  std::unique_ptr<llvm::MemoryBuffer> GetCachedData(llvm::StringRef key);
+
+  /// Set cached data for the specified key.
+  ///
+  /// Setting the cached data will save a file in the cache directory to contain
+  /// the specified data.
+  ///
+  /// \param key
+  ///   The unique string key that identifies data being cached.
+  ///
+  /// \return
+  ///   True if the data was successfully cached, false otherwise.
+  bool SetCachedData(llvm::StringRef key, llvm::ArrayRef<uint8_t> data);
+
+  /// Remove the cache file associated with the key.
+  Status RemoveCacheFile(llvm::StringRef key);
+
+private:
+  /// Return the cache file that is associated with the key.
+  FileSpec GetCacheFilePath(llvm::StringRef key);
+
+  llvm::FileCache m_cache_callback;
+  FileSpec m_cache_dir;
+  std::mutex m_mutex;
+  std::unique_ptr<llvm::MemoryBuffer> m_mem_buff_up;
+  bool m_take_ownership = false;
+};
+
+/// A signature for a given file on disk.
+///
+/// Any files that are cached in the LLDB index cached need some data that
+/// uniquely identifies a file on disk and this information should be written
+/// into each cache file so we can validate if the cache file still matches
+/// the file we are trying to load cached data for. Objects can fill out this
+/// signature and then encode and decode them to validate the signatures
+/// match. If they do not match, the cache file on disk should be removed as
+/// it is out of date.
+struct CacheSignature {
+  /// UUID of object file or module.
+  llvm::Optional<UUID> m_uuid = llvm::None;
+  /// Modification time of file on disk.
+  llvm::Optional<std::time_t> m_mod_time = llvm::None;
+  /// If this describes a .o file with a BSD archive, the BSD archive's
+  /// modification time will be in m_mod_time, and the .o file's modification
+  /// time will be in this m_obj_mod_time.
+  llvm::Optional<std::time_t> m_obj_mod_time = llvm::None;
+
+  CacheSignature() = default;
+
+  /// Create a signature from a module.
+  CacheSignature(lldb_private::Module *module);
+
+  /// Create a signature from an object file.
+  CacheSignature(lldb_private::ObjectFile *objfile);
+
+  void Clear() {
+    m_uuid = llvm::None;
+    m_mod_time = llvm::None;
+    m_obj_mod_time = llvm::None;
+  }
+
+  /// Return true if any of the signature member variables have valid values.
+  bool IsValid() const {
+    return m_uuid.hasValue() || m_mod_time.hasValue() ||
+           m_obj_mod_time.hasValue();
+  }
+
+  /// Check if two signatures are the same.
+  bool operator!=(const CacheSignature &rhs) {
+    if (m_uuid != rhs.m_uuid)
+      return true;
+    if (m_mod_time != rhs.m_mod_time)
+      return true;
+    if (m_obj_mod_time != rhs.m_obj_mod_time)
+      return true;
+    return false;
+  }
+  /// Encode this object into a data encoder object.
+  ///
+  /// This allows this object to be serialized to disk. The CacheSignature
+  /// object must have at least one member variable that has a value in order to
+  /// be serialized so that we can match this data to when the cached file is
+  /// loaded at a later time.
+  ///
+  /// \param encoder
+  ///   A data encoder object that serialized bytes will be encoded into.
+  ///
+  /// \return
+  ///   True if a signature was encoded, and false if there were no member
+  ///   variables that had value. False indicates this data should not be
+  ///   cached to disk because we were unable to encode a valid signature.
+  bool Encode(DataEncoder &encoder);
+
+  /// Decode a serialized version of this object from data.
+  ///
+  /// \param data
+  ///   The decoder object that references the serialized data.
+  ///
+  /// \param offset_ptr
+  ///   A pointer that contains the offset from which the data will be decoded
+  ///   from that gets updated as data gets decoded.
+  ///
+  /// \return
+  ///   True if the signature was successfully decoded, false otherwise.
+  bool Decode(const DataExtractor &data, lldb::offset_t *offset_ptr);
+};
+
+/// Many cache files require string tables to store data efficiently. This
+/// class helps create string tables.
+class ConstStringTable {
+public:
+  ConstStringTable() = default;
+  /// Add a string into the string table.
+  ///
+  /// Add a string to the string table will only add the same string one time
+  /// and will return the offset in the string table buffer to that string.
+  /// String tables are easy to build with ConstString objects since most LLDB
+  /// classes for symbol or debug info use them already and they provide
+  /// permanent storage for the string.
+  ///
+  /// \param s
+  ///   The string to insert into the string table.
+  ///
+  /// \return
+  ///   The byte offset from the start of the string table for the inserted
+  ///   string. Duplicate strings that get inserted will return the same
+  ///   byte offset.
+  uint32_t Add(ConstString s);
+
+  bool Encode(DataEncoder &encoder);
+
+private:
+  std::vector<ConstString> m_strings;
+  std::map<ConstString, uint32_t> m_string_to_offset;
+  /// Skip one byte to start the string table off with an empty string.
+  uint32_t m_next_offset = 1;
+};
+
+/// Many cache files require string tables to store data efficiently. This
+/// class helps give out strings from a string table that was read from a
+/// cache file.
+class StringTableReader {
+public:
+  StringTableReader() = default;
+
+  llvm::StringRef Get(uint32_t offset) const;
+
+  bool Decode(const DataExtractor &data, lldb::offset_t *offset_ptr);
+
+protected:
+  /// All of the strings in the string table are contained in m_data.
+  llvm::StringRef m_data;
+};
+
+} // namespace lldb_private
+
+#endif // LLDB_CORE_DATAFILECACHE_H

lldb/include/lldb/Core/Mangled.h

 //===-- Mangled.h -----------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLDB_CORE_MANGLED_H
 #define LLDB_CORE_MANGLED_H
 #if defined(__cplusplus)
 
 #include "lldb/lldb-enumerations.h"
 #include "lldb/lldb-forward.h"
-
+#include "lldb/lldb-types.h"
 #include "lldb/Utility/ConstString.h"
-
 #include "llvm/ADT/StringRef.h"
 
 #include <cstddef>
 #include <memory>
 
 namespace lldb_private {
 
 /// \class Mangled Mangled.h "lldb/Core/Mangled.h"
public:
   /// mangled or not.
   ///
   /// \param[in] name
   ///     The already const name to copy into this object.
   explicit Mangled(ConstString name);
 
   explicit Mangled(llvm::StringRef name);
 
+  bool operator==(const Mangled &rhs) const {
+    return m_mangled == rhs.m_mangled &&
+           GetDemangledName() == rhs.GetDemangledName();
+  }
+
+  bool operator!=(const Mangled &rhs) const {
+    return !(*this == rhs);
+  }
+
   /// Convert to pointer operator.
   ///
   /// This allows code to check a Mangled object to see if it contains a valid
   /// mangled name using code such as:
   ///
   /// \code
   /// Mangled mangled(...);
   /// if (mangled)
public:
   /// \param[in] name
   ///     The name we are attempting to identify the mangling scheme for.
   ///
   /// \return
   ///     eManglingSchemeNone if no known mangling scheme could be identified
   ///     for s, otherwise the enumerator for the mangling scheme detected.
   static Mangled::ManglingScheme GetManglingScheme(llvm::StringRef const name);
 
+  /// Decode a serialized version of this object from data.
+  ///
+  /// \param data
+  ///   The decoder object that references the serialized data.
+  ///
+  /// \param offset_ptr
+  ///   A pointer that contains the offset from which the data will be decoded
+  ///   from that gets updated as data gets decoded.
+  ///
+  /// \param strtab
+  ///   All strings in cache files are put into string tables for efficiency
+  ///   and cache file size reduction. Strings are stored as uint32_t string
+  ///   table offsets in the cache data.
+  bool Decode(const DataExtractor &data, lldb::offset_t *offset_ptr,
+              const StringTableReader &strtab);
+
+  /// Encode this object into a data encoder object.
+  ///
+  /// This allows this object to be serialized to disk.
+  ///
+  /// \param encoder
+  ///   A data encoder object that serialized bytes will be encoded into.
+  ///
+  /// \param strtab
+  ///   All strings in cache files are put into string tables for efficiency
+  ///   and cache file size reduction. Strings are stored as uint32_t string
+  ///   table offsets in the cache data.
+  void Encode(DataEncoder &encoder, ConstStringTable &strtab) const;
+
 private:
   /// Mangled member variables.
   ConstString m_mangled;           ///< The mangled version of the name
   mutable ConstString m_demangled; ///< Mutable so we can get it on demand with
                                    ///a const version of this object
 };
 
 Stream &operator<<(Stream &s, const Mangled &obj);
 
 } // namespace lldb_private
 
 #endif // #if defined(__cplusplus)
 #endif // LLDB_CORE_MANGLED_H

lldb/include/lldb/Core/Module.h

protected:
     /// names we are looking for
     lldb::FunctionNameType m_name_type_mask = lldb::eFunctionNameTypeNone;
 
     ///< If \b true, then demangled names that match will need to contain
     ///< "m_name" in order to be considered a match
     bool m_match_name_after_lookup = false;
   };
 
+  /// Get a unique hash for this module.
+  ///
+  /// The hash should be enough to identify the file on disk and the
+  /// architecture of the file. If the module represents an object inside of a
+  /// file, then the hash should include the object name and object offset to
+  /// ensure a unique hash. Some examples:
+  /// - just a regular object file (mach-o, elf, coff, etc) should create a hash
+  /// - a universal mach-o file that contains to multiple architectures,
+  ///   each architecture slice should have a unique hash even though they come
+  ///   from the same file
+  /// - a .o file inside of a BSD archive. Each .o file will have an object name
+  ///   and object offset that should produce a unique hash. The object offset
+  ///   is needed as BSD archive files can contain multiple .o files that have
+  ///   the same name.
+  uint32_t Hash();
+
+  /// Get a unique cache key for the current module.
+  ///
+  /// The cache key must be unique for a file on disk and not change if the file
+  /// is updated. This allows cache data to use this key as a prefix and as
+  /// files are modified in disk, we will overwrite the cache files. If one file
+  /// can contain multiple files, like a universal mach-o file or like a BSD
+  /// archive, the cache key must contain enough information to differentiate
+  /// these different files.
+  std::string GetCacheKey();
+
+  /// Get the global index file cache.
+  ///
+  /// LLDB can cache data for a module between runs. This cache directory can be
+  /// used to stored data that previously was manually created each time you debug.
+  /// Examples include debug information indexes, symbol tables, symbol table
+  /// indexes, and more.
+  ///
+  /// \returns
+  ///   If caching is enabled in the lldb settings, return a pointer to the data
+  ///   file cache. If caching is not enabled, return NULL.
+  static DataFileCache *GetIndexCache();
 protected:
   // Member Variables
   mutable std::recursive_mutex m_mutex; ///< A mutex to keep this object happy
                                         /// in multi-threaded environments.
 
   /// The modification time for this module when it was created.
   llvm::sys::TimePoint<> m_mod_time;
 

lldb/include/lldb/Core/ModuleList.h

 
 public:
   ModuleListProperties();
 
   FileSpec GetClangModulesCachePath() const;
   bool SetClangModulesCachePath(const FileSpec &path);
   bool GetEnableExternalLookup() const;
   bool SetEnableExternalLookup(bool new_value);
+  bool GetEnableLLDBIndexCache() const;
+  bool SetEnableLLDBIndexCache(bool new_value);
+  uint64_t GetLLDBIndexCacheMaxByteSize();
+  uint64_t GetLLDBIndexCacheMaxPercent();
+  uint64_t GetLLDBIndexCacheExpirationDays();
+  FileSpec GetLLDBIndexCachePath() const;
+  bool SetLLDBIndexCachePath(const FileSpec &path);
 
   PathMappingList GetSymlinkMappings() const;
 };
 
 /// \class ModuleList ModuleList.h "lldb/Core/ModuleList.h"
 /// A collection class for Module objects.
 ///
 /// Modules in the module collection class are stored as reference counted

lldb/include/lldb/Host/FileSystem.h

public:
   /// \}
 
   /// Resolve path to make it canonical.
   /// \{
   void Resolve(llvm::SmallVectorImpl<char> &path);
   void Resolve(FileSpec &file_spec);
   /// \}
 
+  /// Remove a single file.
+  ///
+  /// The path must specify a file and not a directory.
+  /// \{
+  Status RemoveFile(const FileSpec &file_spec);
+  Status RemoveFile(const llvm::Twine &path);
+  /// \}
+
   //// Create memory buffer from path.
   /// \{
   std::shared_ptr<DataBufferLLVM> CreateDataBuffer(const llvm::Twine &path,
                                                    uint64_t size = 0,
                                                    uint64_t offset = 0);
   std::shared_ptr<DataBufferLLVM> CreateDataBuffer(const FileSpec &file_spec,
                                                    uint64_t size = 0,
                                                    uint64_t offset = 0);

lldb/include/lldb/Symbol/ObjectFile.h

 #include "lldb/Core/PluginInterface.h"
 #include "lldb/Symbol/Symtab.h"
 #include "lldb/Symbol/UnwindTable.h"
 #include "lldb/Utility/DataExtractor.h"
 #include "lldb/Utility/Endian.h"
 #include "lldb/Utility/FileSpec.h"
 #include "lldb/Utility/UUID.h"
 #include "lldb/lldb-private.h"
+#include "llvm/ADT/Optional.h"
 #include "llvm/Support/Threading.h"
 #include "llvm/Support/VersionTuple.h"
 
 namespace lldb_private {
 
 class ObjectFileJITDelegate {
 public:
   ObjectFileJITDelegate() = default;
public:
   ///
   /// \return
   ///     Returns true if any binaries were loaded.
 
   virtual bool LoadCoreFileImages(lldb_private::Process &process) {
     return false;
   }
 
+  /// Get a hash that can be used for caching object file releated information.
+  ///
+  /// Data for object files can be cached between runs of debug sessions and
+  /// a module can end up using a main file and a symbol file, both of which
+  /// can be object files. So we need a unique hash that identifies an object
+  /// file when storing cached data.
+  uint32_t GetCacheHash();
+
+
 protected:
   // Member variables.
   FileSpec m_file;
   Type m_type;
   Strata m_strata;
   lldb::addr_t m_file_offset; ///< The offset in bytes into the file, or the
                               ///address in memory
   lldb::addr_t m_length; ///< The length of this object file if it is known (can
                          ///be zero if length is unknown or can't be
                          ///determined).
   DataExtractor
       m_data; ///< The data for this object file so things can be parsed lazily.
   lldb::ProcessWP m_process_wp;
   const lldb::addr_t m_memory_addr;
   std::unique_ptr<lldb_private::SectionList> m_sections_up;
   std::unique_ptr<lldb_private::Symtab> m_symtab_up;
   /// We need a llvm::once_flag that we can use to avoid locking the module
   /// lock and deadlocking LLDB. See comments in ObjectFile::GetSymtab() for
   /// the full details. We also need to be able to clear the symbol table, so we
   /// need to use a std::unique_ptr to a llvm::once_flag so if we clear the
   /// symbol table, we can have a new once flag to use when it is created again.
   std::unique_ptr<llvm::once_flag> m_symtab_once_up;
+  llvm::Optional<uint32_t> m_cache_hash;
 
   /// Sets the architecture for a module.  At present the architecture can
   /// only be set if it is invalid.  It is not allowed to switch from one
   /// concrete architecture to another.
   ///
   /// \param[in] new_arch
   ///     The architecture this module will be set to.
   ///

lldb/include/lldb/Symbol/Symbol.h

bool GetDisassembly(const ExecutionContext &exe_ctx, const char *flavor,
                       bool prefer_file_cache, Stream &strm);
 
   bool ContainsFileAddress(lldb::addr_t file_addr) const;
 
   static llvm::StringRef GetSyntheticSymbolPrefix() {
     return "___lldb_unnamed_symbol";
   }
 
+  /// Decode a serialized version of this object from data.
+  ///
+  /// \param data
+  ///   The decoder object that references the serialized data.
+  ///
+  /// \param offset_ptr
+  ///   A pointer that contains the offset from which the data will be decoded
+  ///   from that gets updated as data gets decoded.
+  ///
+  /// \param section_list
+  ///   A section list that allows lldb_private::Address objects to be filled
+  ///   in. The address information for symbols are serilized as file addresses
+  ///   and must be converted into Address objects with the right section and
+  ///   offset.
+  ///
+  /// \param strtab
+  ///   All strings in cache files are put into string tables for efficiency
+  ///   and cache file size reduction. Strings are stored as uint32_t string
+  ///   table offsets in the cache data.
+  ///
+  /// \return
+  ///   True if the symbol is successfully decoded, false otherwise.
+  bool Decode(const DataExtractor &data, lldb::offset_t *offset_ptr,
+              const SectionList *section_list, const StringTableReader &strtab);
+
+  /// Encode this object into a data encoder object.
+  ///
+  /// This allows this object to be serialized to disk.
+  ///
+  /// \param encoder
+  ///   A data encoder object that serialized bytes will be encoded into.
+  ///
+  /// \param strtab
+  ///   All strings in cache files are put into string tables for efficiency
+  ///   and cache file size reduction. Strings are stored as uint32_t string
+  ///   table offsets in the cache data.
+  void Encode(DataEncoder &encoder, ConstStringTable &strtab) const;
+
+  bool operator==(const Symbol &rhs) const;
+
 protected:
   // This is the internal guts of ResolveReExportedSymbol, it assumes
   // reexport_name is not null, and that module_spec is valid.  We track the
   // modules we've already seen to make sure we don't get caught in a cycle.
 
   Symbol *ResolveReExportedSymbolInModuleSpec(
       Target &target, ConstString &reexport_name,
       lldb_private::ModuleSpec &module_spec,

lldb/include/lldb/Symbol/Symtab.h

public:
   static void DumpSymbolHeader(Stream *s);
 
   void Finalize();
 
   void AppendSymbolNamesToMap(const IndexCollection &indexes,
                               bool add_demangled, bool add_mangled,
                               NameToIndexMap &name_to_index_map) const;
 
-  ObjectFile *GetObjectFile() { return m_objfile; }
+  ObjectFile *GetObjectFile() const { return m_objfile; }
+
+  /// Decode a serialized version of this object from data.
+  ///
+  /// \param data
+  ///   The decoder object that references the serialized data.
+  ///
+  /// \param offset_ptr
+  ///   A pointer that contains the offset from which the data will be decoded
+  ///   from that gets updated as data gets decoded.
+  ///
+  /// \param[out] uuid_mismatch
+  ///   Set to true if a cache file exists but the UUID didn't match, false
+  ///   otherwise.
+  ///
+  /// \return
+  ///   True if the symbol table is successfully decoded and can be used,
+  ///   false otherwise.
+  bool Decode(const DataExtractor &data, lldb::offset_t *offset_ptr,
+              bool &uuid_mismatch);
+
+  /// Encode this object into a data encoder object.
+  ///
+  /// This allows this object to be serialized to disk. The object file must
+  /// have a valid Signature in order to be serialized as it is used to make
+  /// sure the cached information matches when cached data is loaded at a later
+  /// time. If the object file doesn't have a valid signature false will be
+  /// returned and it will indicate we should not cache this data.
+  ///
+  /// \param encoder
+  ///   A data encoder object that serialized bytes will be encoded into.
+  ///
+  /// \return
+  ///   True if the symbol table's object file can generate a valid signature
+  ///   and all data for the symbol table was encoded, false otherwise.
+  bool Encode(DataEncoder &encoder) const;
+
+  /// Get the cache key string for this symbol table.
+  ///
+  /// The cache key must start with the module's cache key and is followed
+  /// by information that indicates this key is for caching the symbol table
+  /// contents and should also include the has of the object file. A module can
+  /// be represented by an ObjectFile object for the main executable, but can
+  /// also have a symbol file that is from the same or a different object file.
+  /// This means we might have two symbol tables cached in the index cache, one
+  /// for the main executable and one for the symbol file.
+  ///
+  /// \return
+  ///   The unique cache key used to save and retrieve data from the index cache.
+  std::string GetCacheKey();
+
+  /// Save the symbol table data out into a cache.
+  ///
+  /// The symbol table will only be saved to a cache file if caching is enabled.
+  ///
+  /// We cache the contents of the symbol table since symbol tables in LLDB take
+  /// some time to initialize. This is due to the many sources for data that are
+  /// used to create a symbol table:
+  /// - standard symbol table
+  /// - dynamic symbol table (ELF)
+  /// - compressed debug info sections
+  /// - unwind information
+  /// - function pointers found in runtimes for global constructor/destructors
+  /// - other sources.
+  /// All of the above sources are combined and one symbol table results after
+  /// all sources have been considered.
+  void SaveToCache();
+
+  /// Load the symbol table from the index cache.
+  ///
+  /// Quickly load the finalized symbol table from the index cache. This saves
+  /// time when the debugger starts up. The index cache file for the symbol
+  /// table has the modification time set to the same time as the main module.
+  /// If the cache file exists and the modification times match, we will load
+  /// the symbol table from the serlized cache file.
+  ///
+  /// \return
+  ///   True if the symbol table was successfully loaded from the index cache,
+  ///   false if the symbol table wasn't cached or was out of date.
+  bool LoadFromCache();
 
 protected:
   typedef std::vector<Symbol> collection;
   typedef collection::iterator iterator;
   typedef collection::const_iterator const_iterator;
   class FileRangeToIndexMapCompare {
   public:
     FileRangeToIndexMapCompare(const Symtab &symtab) : m_symtab(symtab) {}
     bool operator()(const uint32_t a_data, const uint32_t b_data) const {
       return rank(a_data) > rank(b_data);
     }
 
   private:
     // How much preferred is this symbol?
     int rank(const uint32_t data) const {
       const Symbol &symbol = *m_symtab.SymbolAtIndex(data);
       if (symbol.IsExternal())
         return 3;
       if (symbol.IsWeak())
         return 2;
       if (symbol.IsDebug())
         return 0;

lldb/include/lldb/Utility/DataEncoder.h

public:
   /// Append the bytes contained in the string reference. This function will
   /// not append a NULL termination character for a C string. Use the
   /// AppendCString function for this purpose.
   ///
   /// \param data
   ///     A string reference that contains bytes to append.
   void AppendData(llvm::StringRef data);
 
+  /// Append a bytes to the end of the owned data.
+  ///
+  /// Append the bytes contained in the array reference.
+  ///
+  /// \param data
+  ///     A array reference that contains bytes to append.
+  void AppendData(llvm::ArrayRef<uint8_t> data);
+
   /// Append a C string to the end of the owned data.
   ///
   /// Append the bytes contained in the string reference along with an extra
   /// NULL termination character if the StringRef bytes doesn't include one as
   /// the last byte.
   ///
   /// \param data
   ///     A string reference that contains bytes to append.
public:
   ///
   /// This value will always return the data that this object references even if
   /// the object was constructed with caller owned data.
   ///
   /// \return
   ///     A array reference to the data that this object references.
   llvm::ArrayRef<uint8_t> GetData() const;
 
+  /// Get the number of bytes contained in this object.
+  ///
+  /// \return
+  ///     The total number of bytes of data this object refers to.
+  size_t GetByteSize() const;
+
+  lldb::ByteOrder GetByteOrder() const { return m_byte_order; }
+
+  /// The address size to use when encoding pointers or addresses.
+  uint8_t GetAddressByteSize() const { return m_addr_size; }
+
 private:
   uint32_t BytesLeft(uint32_t offset) const {
     const uint32_t size = GetByteSize();
     if (size > offset)
       return size - offset;
     return 0;
   }
 
   /// Test the availability of \a length bytes of data from \a offset.
   ///
   /// \return
   ///     \b true if \a offset is a valid offset and there are \a
   ///     length bytes available at that offset, \b false otherwise.
   bool ValidOffsetForDataOfSize(uint32_t offset, uint32_t length) const {
     return length <= BytesLeft(offset);
   }
 
   /// Test the validity of \a offset.
   ///
   /// \return
   ///     \b true if \a offset is a valid offset into the data in this
   ///     object, \b false otherwise.
   bool ValidOffset(uint32_t offset) const { return offset < GetByteSize(); }
 
-  /// Get the number of bytes contained in this object.
-  ///
-  /// \return
-  ///     The total number of bytes of data this object refers to.
-  size_t GetByteSize() const;
-
   /// The shared pointer to data that can grow as data is added
   std::shared_ptr<lldb_private::DataBufferHeap> m_data_sp;
 
   /// The byte order of the data we are encoding to.
-  lldb::ByteOrder m_byte_order;
+  const lldb::ByteOrder m_byte_order;
 
   /// The address size to use when encoding pointers or addresses.
-  uint8_t m_addr_size;
+  const uint8_t m_addr_size;
 
   DataEncoder(const DataEncoder &) = delete;
   const DataEncoder &operator=(const DataEncoder &) = delete;
 };
 
 } // namespace lldb_private
 
 #endif // #if defined (__cplusplus)
 #endif // LLDB_UTILITY_DATAENCODER_H

lldb/include/lldb/lldb-forward.h

 class BreakpointPrecondition;
 class BreakpointResolver;
 class BreakpointSite;
 class BreakpointSiteList;
 class BroadcastEventSpec;
 class Broadcaster;
 class BroadcasterManager;
 class CXXSyntheticChildren;
+struct CacheSignature;
 class CallFrameInfo;
 class CommandInterpreter;
 class CommandInterpreterRunOptions;
 class CommandObject;
 class CommandObjectMultiword;
 class CommandReturnObject;
 class Communication;
 class CompactUnwindInfo;
 class CompileUnit;
 class CompilerDecl;
 class CompilerDeclContext;
 class CompilerType;
 class Connection;
 class ConnectionFileDescriptor;
 class ConstString;
+class ConstStringTable;
 class DWARFCallFrameInfo;
 class DWARFDataExtractor;
 class DWARFExpression;
 class DataBuffer;
 class DataBufferHeap;
 class DataEncoder;
 class DataExtractor;
+class DataFileCache;
 class Debugger;
 class Declaration;
 class DiagnosticManager;
 class Disassembler;
 class DumpValueObjectOptions;
 class DynamicCheckerFunctions;
 class DynamicLoader;
 class Editline;

 class Status;
 class StopInfo;
 class Stoppoint;
 class StoppointCallbackContext;
 class Stream;
 class StreamFile;
 class StreamString;
 class StringList;
+class StringTableReader;
 class StructuredDataImpl;
 class StructuredDataPlugin;
 class Symbol;
 class SymbolContext;
 class SymbolContextList;
 class SymbolContextScope;
 class SymbolContextSpecifier;
 class SymbolFile;

lldb/packages/Python/lldbsuite/test/lldbtest.py

""".format(cmd_status, filecheck_cmd, output, cmd_stdout, cmd_stderr)
 
     """Assert that an lldb.SBError is in the "success" state."""
     def assertSuccess(self, obj, msg=None):
         if not obj.Success():
             error = obj.GetCString()
             self.fail(self._formatMessage(msg,
                 "'{}' is not success".format(error)))
 
-    def createTestTarget(self, file_path=None, msg=None):
+    def createTestTarget(self, file_path=None, msg=None,
+                         load_dependent_modules=True):
         """
         Creates a target from the file found at the given file path.
         Asserts that the resulting target is valid.
         :param file_path: The file path that should be used to create the target.
                           The default argument opens the current default test
                           executable in the current test directory.
         :param msg: A custom error message.
         """
         if file_path is None:
             file_path = self.getBuildArtifact("a.out")
         error = lldb.SBError()
         triple = ""
         platform = ""
-        load_dependent_modules = True
         target = self.dbg.CreateTarget(file_path, triple, platform,
                                        load_dependent_modules, error)
         if error.Fail():
             err = "Couldn't create target for path '{}': {}".format(file_path,
                                                                     str(error))
             self.fail(self._formatMessage(msg, err))
 
         self.assertTrue(target.IsValid(), "Got invalid target without error")

lldb/source/Core/CMakeLists.txt

 endif()
 
 add_lldb_library(lldbCore
   Address.cpp
   AddressRange.cpp
   AddressResolver.cpp
   AddressResolverFileLine.cpp
   Communication.cpp
+  DataFileCache.cpp
   Debugger.cpp
   Declaration.cpp
   Disassembler.cpp
   DumpDataExtractor.cpp
   DumpRegisterValue.cpp
   DynamicLoader.cpp
   EmulateInstruction.cpp
   FileLineResolver.cpp

lldb/source/Core/CoreProperties.td

 include "../../include/lldb/Core/PropertiesBase.td"
 
 let Definition = "modulelist" in {
   def EnableExternalLookup: Property<"enable-external-lookup", "Boolean">,
     Global,
     DefaultTrue,
     Desc<"Control the use of external tools and repositories to locate symbol files. Directories listed in target.debug-file-search-paths and directory of the executable are always checked first for separate debug info files. Then depending on this setting: On macOS, Spotlight would be also used to locate a matching .dSYM bundle based on the UUID of the executable. On NetBSD, directory /usr/libdata/debug would be also searched. On platforms other than NetBSD directory /usr/lib/debug would be also searched.">;
   def ClangModulesCachePath: Property<"clang-modules-cache-path", "FileSpec">,
     Global,
     DefaultStringValue<"">,
     Desc<"The path to the clang modules cache directory (-fmodules-cache-path).">;
   def SymLinkPaths: Property<"debug-info-symlink-paths", "FileSpecList">,
     Global,
     DefaultStringValue<"">,
     Desc<"Debug info path which should be resolved while parsing, relative to the host filesystem.">;
+  def EnableLLDBIndexCache: Property<"enable-lldb-index-cache", "Boolean">,
+    Global,
+    DefaultFalse,
+    Desc<"Enable caching for debug sessions in LLDB. LLDB can cache data for each module for improved performance in subsequent debug sessions.">;
+  def LLDBIndexCachePath: Property<"lldb-index-cache-path", "FileSpec">,
+    Global,
+    DefaultStringValue<"">,
+    Desc<"The path to the LLDB index cache directory.">;
+  def LLDBIndexCacheMaxByteSize: Property<"lldb-index-cache-max-byte-size", "UInt64">,
+    Global,
+    DefaultUnsignedValue<0>,
+    Desc<"The maximum size for the LLDB index cache directory in bytes. A value over the amount of available space on the disk will be reduced to the amount of available space. A value of 0 disables the absolute size-based pruning.">;
+  def LLDBIndexCacheMaxPercent: Property<"lldb-index-cache-max-percent", "UInt64">,
+    Global,
+    DefaultUnsignedValue<0>,
+    Desc<"The maximum size for the cache directory in terms of percentage of the available space on the disk. Set to 100 to indicate no limit, 50 to indicate that the cache size will not be left over half the available disk space. A value over 100 will be reduced to 100. A value of 0 disables the percentage size-based pruning.">;
+  def LLDBIndexCacheExpirationDays: Property<"lldb-index-cache-expiration-days", "UInt64">,
+    Global,
+    DefaultUnsignedValue<7>,
+    Desc<"The expiration time in days for a file. When a file hasn't been accessed for the specified amount of days, it is removed from the cache. A value of 0 disables the expiration-based pruning.">;
 }
 
 let Definition = "debugger" in {
   def AutoConfirm: Property<"auto-confirm", "Boolean">,
     Global,
     DefaultFalse,
     Desc<"If true all confirmation prompts will receive their default reply.">;
   def DisassemblyFormat: Property<"disassembly-format", "FormatEntity">,

lldb/source/Core/DataFileCache.cpp

+//===-- DataFileCache.cpp -------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "lldb/Core/DataFileCache.h"
+#include "lldb/Core/Module.h"
+#include "lldb/Core/ModuleList.h"
+#include "lldb/Host/FileSystem.h"
+#include "lldb/Symbol/ObjectFile.h"
+#include "lldb/Utility/DataEncoder.h"
+#include "lldb/Utility/Log.h"
+#include "lldb/Utility/Logging.h"
+#include "llvm/Support/CachePruning.h"
+#include "llvm/Support/MemoryBuffer.h"
+
+using namespace llvm;
+using namespace lldb_private;
+
+DataFileCache::DataFileCache(StringRef path) {
+  m_cache_dir.SetPath(path);
+
+  // Prune the cache based off of the LLDB settings each time we create a cache
+  // object.
+  ModuleListProperties &properties =
+      ModuleList::GetGlobalModuleListProperties();
+  CachePruningPolicy policy;
+  // Only scan once an hour. If we have lots of debug sessions we don't want
+  // to scan this directory too often. A timestamp file is written to the
+  // directory to ensure different processes don't scan the directory too often.
+  // This setting doesn't mean that a thread will continually scan the cache
+  // directory within this process.
+  policy.Interval = std::chrono::hours(1);
+  // Get the user settings for pruning.
+  policy.MaxSizeBytes = properties.GetLLDBIndexCacheMaxByteSize();
+  policy.MaxSizePercentageOfAvailableSpace =
+      properties.GetLLDBIndexCacheMaxPercent();
+  policy.Expiration =
+      std::chrono::hours(properties.GetLLDBIndexCacheExpirationDays() * 24);
+  pruneCache(path, policy);
+
+  // This lambda will get called when the data is gotten from the cache and
+  // also after the data was set for a given key. We only need to take
+  // ownership of the data if we are geting the data, so we use the
+  // m_take_ownership member variable to indicate if we need to take
+  // ownership.
+
+  auto add_buffer = [this](unsigned task, std::unique_ptr<llvm::MemoryBuffer> m) {
+    if (m_take_ownership)
+      m_mem_buff_up = std::move(m);
+  };
+  Expected<FileCache> cache_or_err =
+      llvm::localCache("LLDBModuleCache", "lldb-module", path, add_buffer);
+  if (cache_or_err)
+    m_cache_callback = std::move(*cache_or_err);
+  else {
+    Log *log(lldb_private::GetLogIfAnyCategoriesSet(LIBLLDB_LOG_MODULES));
+    LLDB_LOG_ERROR(log, cache_or_err.takeError(),
+                   "failed to create lldb index cache directory: {0}");
+  }
+}
+
+std::unique_ptr<llvm::MemoryBuffer>
+DataFileCache::GetCachedData(StringRef key) {
+  std::lock_guard<std::mutex> guard(m_mutex);
+
+  const unsigned task = 1;
+  m_take_ownership = true;
+  // If we call the "m_cache_callback" function and the data is cached, it will
+  // call the "add_buffer" lambda function from the constructor which will in
+  // turn take ownership of the member buffer that is passed to the callback and
+  // put it into a member variable.
+  Expected<AddStreamFn> add_stream_or_err = m_cache_callback(task, key);
+  m_take_ownership = false;
+  // At this point we either already called the "add_buffer" lambda with
+  // the data or we haven't. We can tell if we got the cached data by checking
+  // the add_stream function pointer value below.
+  if (add_stream_or_err) {
+    AddStreamFn &add_stream = *add_stream_or_err;
+    // If the "add_stream" is nullptr, then the data was cached and we already
+    // called the "add_buffer" lambda. If it is valid, then if we were to call
+    // the add_stream function it would cause a cache file to get generated
+    // and we would be expected to fill in the data. In this function we only
+    // want to check if the data was cached, so we don't want to call
+    // "add_stream" in this function.
+    if (!add_stream)
+      return std::move(m_mem_buff_up);
+  } else {
+    Log *log(lldb_private::GetLogIfAnyCategoriesSet(LIBLLDB_LOG_MODULES));
+    LLDB_LOG_ERROR(log, add_stream_or_err.takeError(),
+                   "failed to get the cache add stream callback for key: {0}");
+  }
+  // Data was not cached.
+  return std::unique_ptr<llvm::MemoryBuffer>();
+}
+
+bool DataFileCache::SetCachedData(StringRef key, llvm::ArrayRef<uint8_t> data) {
+  std::lock_guard<std::mutex> guard(m_mutex);
+  const unsigned task = 2;
+  // If we call this function and the data is cached, it will call the
+  // add_buffer lambda function from the constructor which will ignore the
+  // data.
+  Expected<AddStreamFn> add_stream_or_err = m_cache_callback(task, key);
+  // If we reach this code then we either already called the callback with
+  // the data or we haven't. We can tell if we had the cached data by checking
+  // the CacheAddStream function pointer value below.
+  if (add_stream_or_err) {
+    AddStreamFn &add_stream = *add_stream_or_err;
+    // If the "add_stream" is nullptr, then the data was cached. If it is
+    // valid, then if we call the add_stream function with a task it will
+    // cause the file to get generated, but we only want to check if the data
+    // is cached here, so we don't want to call it here. Note that the
+    // add_buffer will also get called in this case after the data has been
+    // provided, but we won't take ownership of the memory buffer as we just
+    // want to write the data.
+    if (add_stream) {
+      Expected<std::unique_ptr<CachedFileStream>> file_or_err =
+          add_stream(task);
+      if (file_or_err) {
+        CachedFileStream *cfs = file_or_err->get();
+        cfs->OS->write((const char *)data.data(), data.size());
+        return true;
+      } else {
+        Log *log(lldb_private::GetLogIfAnyCategoriesSet(LIBLLDB_LOG_MODULES));
+        LLDB_LOG_ERROR(log, file_or_err.takeError(),
+                       "failed to get the cache file stream for key: {0}");
+      }
+    }
+  } else {
+    Log *log(lldb_private::GetLogIfAnyCategoriesSet(LIBLLDB_LOG_MODULES));
+    LLDB_LOG_ERROR(log, add_stream_or_err.takeError(),
+                   "failed to get the cache add stream callback for key: {0}");
+  }
+  return false;
+}
+
+FileSpec DataFileCache::GetCacheFilePath(llvm::StringRef key) {
+  FileSpec cache_file(m_cache_dir);
+  std::string filename("llvmcache-");
+  filename += key.str();
+  cache_file.AppendPathComponent(filename);
+  return cache_file;
+}
+
+Status DataFileCache::RemoveCacheFile(llvm::StringRef key) {
+  FileSpec cache_file = GetCacheFilePath(key);
+  FileSystem &fs = FileSystem::Instance();
+  if (!fs.Exists(cache_file))
+    return Status();
+  return fs.RemoveFile(cache_file);
+}
+
+CacheSignature::CacheSignature(lldb_private::Module *module) {
+  Clear();
+  UUID uuid = module->GetUUID();
+  if (uuid.IsValid())
+    m_uuid = uuid;
+
+  std::time_t mod_time = 0;
+  mod_time = llvm::sys::toTimeT(module->GetModificationTime());
+  if (mod_time != 0)
+    m_mod_time = mod_time;
+
+  mod_time = llvm::sys::toTimeT(module->GetObjectModificationTime());
+  if (mod_time != 0)
+    m_obj_mod_time = mod_time;
+}
+
+CacheSignature::CacheSignature(lldb_private::ObjectFile *objfile) {
+  Clear();
+  UUID uuid = objfile->GetUUID();
+  if (uuid.IsValid())
+    m_uuid = uuid;
+
+  std::time_t mod_time = 0;
+  // Grab the modification time of the object file's file. It isn't always the
+  // same as the module's file when you have a executable file as the main
+  // executable, and you have a object file for a symbol file.
+  FileSystem &fs = FileSystem::Instance();
+  mod_time = llvm::sys::toTimeT(fs.GetModificationTime(objfile->GetFileSpec()));
+  if (mod_time != 0)
+    m_mod_time = mod_time;
+
+  mod_time =
+      llvm::sys::toTimeT(objfile->GetModule()->GetObjectModificationTime());
+  if (mod_time != 0)
+    m_obj_mod_time = mod_time;
+}
+
+enum SignatureEncoding {
+  eSignatureUUID = 1u,
+  eSignatureModTime = 2u,
+  eSignatureObjectModTime = 3u,
+  eSignatureEnd = 255u,
+};
+
+bool CacheSignature::Encode(DataEncoder &encoder) {
+  if (!IsValid())
+    return false; // Invalid signature, return false!
+
+  if (m_uuid.hasValue()) {
+    llvm::ArrayRef<uint8_t> uuid_bytes = m_uuid->GetBytes();
+    encoder.AppendU8(eSignatureUUID);
+    encoder.AppendU8(uuid_bytes.size());
+    encoder.AppendData(uuid_bytes);
+  }
+  if (m_mod_time.hasValue()) {
+    encoder.AppendU8(eSignatureModTime);
+    encoder.AppendU32(*m_mod_time);
+  }
+  if (m_obj_mod_time.hasValue()) {
+    encoder.AppendU8(eSignatureObjectModTime);
+    encoder.AppendU32(*m_obj_mod_time);
+  }
+  encoder.AppendU8(eSignatureEnd);
+  return true;
+}
+
+bool CacheSignature::Decode(const DataExtractor &data,
+                            lldb::offset_t *offset_ptr) {
+  Clear();
+  while (uint8_t sig_encoding = data.GetU8(offset_ptr)) {
+    switch (sig_encoding) {
+    case eSignatureUUID: {
+      const uint8_t length = data.GetU8(offset_ptr);
+      const uint8_t *bytes = (const uint8_t *)data.GetData(offset_ptr, length);
+      if (bytes != nullptr && length > 0)
+        m_uuid = UUID::fromData(llvm::ArrayRef<uint8_t>(bytes, length));
+    } break;
+    case eSignatureModTime: {
+      uint32_t mod_time = data.GetU32(offset_ptr);
+      if (mod_time > 0)
+        m_mod_time = mod_time;
+    } break;
+    case eSignatureObjectModTime: {
+      uint32_t mod_time = data.GetU32(offset_ptr);
+      if (mod_time > 0)
+        m_mod_time = mod_time;
+    } break;
+    case eSignatureEnd:
+      return true;
+    default:
+      break;
+    }
+  }
+  return false;
+}
+
+uint32_t ConstStringTable::Add(ConstString s) {
+  auto pos = m_string_to_offset.find(s);
+  if (pos != m_string_to_offset.end())
+    return pos->second;
+  const uint32_t offset = m_next_offset;
+  m_strings.push_back(s);
+  m_string_to_offset[s] = offset;
+  m_next_offset += s.GetLength() + 1;
+  return offset;
+}
+
+static const llvm::StringRef kStringTableIdentifier("STAB");
+
+bool ConstStringTable::Encode(DataEncoder &encoder) {
+  // Write an 4 character code into the stream. This will help us when decoding
+  // to make sure we find this identifier when decoding the string table to make
+  // sure we have the rigth data. It also helps to identify the string table
+  // when dumping the hex bytes in a cache file.
+  encoder.AppendData(kStringTableIdentifier);
+  size_t length_offset = encoder.GetByteSize();
+  encoder.AppendU32(0); // Total length of all strings which will be fixed up.
+  size_t strtab_offset = encoder.GetByteSize();
+  encoder.AppendU8(0); // Start the string table with with an empty string.
+  for (auto s: m_strings) {
+    // Make sure all of the offsets match up with what we handed out!
+    const size_t stroff = encoder.GetByteSize() - strtab_offset;
+    auto pos = m_string_to_offset.find(s);
+    assert(pos->second == stroff);
+    // Append the C string into the encoder
+    encoder.AppendCString(s.GetStringRef());
+  }
+  // Fixup the string table length.
+  encoder.PutU32(length_offset, encoder.GetByteSize() - strtab_offset);
+  return true;
+}
+
+bool StringTableReader::Decode(const DataExtractor &data,
+                               lldb::offset_t *offset_ptr) {
+  llvm::StringRef identifier((const char *)data.GetData(offset_ptr, 4), 4);
+  if (identifier != kStringTableIdentifier)
+    return false;
+  const uint32_t length = data.GetU32(offset_ptr);
+  // We always have at least one byte for the empty string at offset zero.
+  if (length == 0)
+    return false;
+  const char *bytes = (const char *)data.GetData(offset_ptr, length);
+  if (bytes == nullptr)
+    return false;
+  m_data = StringRef(bytes, length);
+  return true;
+}
+
+StringRef StringTableReader::Get(uint32_t offset) const {
+  if (offset >= m_data.size())
+    return StringRef();
+  return StringRef(m_data.data() + offset);
+}

lldb/source/Core/Mangled.cpp

 //===-- Mangled.cpp -------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lldb/Core/Mangled.h"
 
+#include "lldb/Core/DataFileCache.h"
 #include "lldb/Core/RichManglingContext.h"
 #include "lldb/Target/Language.h"
 #include "lldb/Utility/ConstString.h"
+#include "lldb/Utility/DataEncoder.h"
 #include "lldb/Utility/Log.h"
 #include "lldb/Utility/Logging.h"
 #include "lldb/Utility/RegularExpression.h"
 #include "lldb/Utility/Stream.h"
 #include "lldb/lldb-enumerations.h"
 
 #include "llvm/ADT/StringRef.h"
 #include "llvm/Demangle/Demangle.h"
Stream &operator<<(Stream &s, const Mangled &obj) {
 
   ConstString demangled = obj.GetDemangledName();
   if (demangled)
     s << ", demangled = '" << demangled << '\'';
   else
     s << ", demangled = <error>";
   return s;
 }
+
+// When encoding Mangled objects we can get away with encoding as little
+// information as is required. The enumeration below helps us to efficiently
+// encode Mangled objects.
+enum MangledEncoding {
+  /// If the Mangled object has neither a mangled name or demangled name we can
+  /// encode the object with one zero byte using the Empty enumeration.
+  Empty = 0u,
+  /// If the Mangled object has only a demangled name and no mangled named, we
+  /// can encode only the demangled name.
+  DemangledOnly = 1u,
+  /// If the mangle name can calculate the demangled name (it is the
+  /// mangled/demangled counterpart), then we only need to encode the mangled
+  /// name as the demangled name can be recomputed.
+  MangledOnly = 2u,
+  /// If we have a Mangled object with two different names that are not related
+  /// then we need to save both strings. This can happen if we have a name that
+  /// isn't a true mangled name, but we want to be able to lookup a symbol by
+  /// name and type in the symbol table. We do this for Objective C symbols like
+  /// "OBJC_CLASS_$_NSValue" where the mangled named will be set to
+  /// "OBJC_CLASS_$_NSValue" and the demangled name will be manually set to
+  /// "NSValue". If we tried to demangled the name "OBJC_CLASS_$_NSValue" it
+  /// would fail, but in these cases we want these unrelated names to be
+  /// preserved.
+  MangledAndDemangled = 3u
+};
+
+bool Mangled::Decode(const DataExtractor &data, lldb::offset_t *offset_ptr,
+                     const StringTableReader &strtab) {
+  m_mangled.Clear();
+  m_demangled.Clear();
+  MangledEncoding encoding = (MangledEncoding)data.GetU8(offset_ptr);
+  switch (encoding) {
+    case Empty:
+      return true;
+
+    case DemangledOnly:
+      m_demangled.SetString(strtab.Get(data.GetU32(offset_ptr)));
+      return true;
+
+    case MangledOnly:
+      m_mangled.SetString(strtab.Get(data.GetU32(offset_ptr)));
+      return true;
+
+    case MangledAndDemangled:
+      m_mangled.SetString(strtab.Get(data.GetU32(offset_ptr)));
+      m_demangled.SetString(strtab.Get(data.GetU32(offset_ptr)));
+      return true;
+  }
+  return false;
+}
+/// The encoding format for the Mangled object is as follows:
+///
+/// uint8_t encoding;
+/// char str1[]; (only if DemangledOnly, MangledOnly)
+/// char str2[]; (only if MangledAndDemangled)
+///
+/// The strings are stored as NULL terminated UTF8 strings and str1 and str2
+/// are only saved if we need them based on the encoding.
+///
+/// Some mangled names have a mangled name that can be demangled by the built
+/// in demanglers. These kinds of mangled objects know when the mangled and
+/// demangled names are the counterparts for each other. This is done because
+/// demangling is very expensive and avoiding demangling the same name twice
+/// saves us a lot of compute time. For these kinds of names we only need to
+/// save the mangled name and have the encoding set to "MangledOnly".
+///
+/// If a mangled obejct has only a demangled name, then we save only that string
+/// and have the encoding set to "DemangledOnly".
+///
+/// Some mangled objects have both mangled and demangled names, but the
+/// demangled name can not be computed from the mangled name. This is often used
+/// for runtime named, like Objective C runtime V2 and V3 names. Both these
+/// names must be saved and the encoding is set to "MangledAndDemangled".
+///
+/// For a Mangled object with no names, we only need to set the encoding to
+/// "Empty" and not store any string values.
+void Mangled::Encode(DataEncoder &file, ConstStringTable &strtab) const {
+  MangledEncoding encoding = Empty;
+  if (m_mangled) {
+    encoding = MangledOnly;
+    if (m_demangled) {
+      // We have both mangled and demangled names. If the demangled name is the
+      // counterpart of the mangled name, then we only need to save the mangled
+      // named. If they are different, we need to save both.
+      ConstString s;
+      if (!(m_mangled.GetMangledCounterpart(s) && s == m_demangled))
+        encoding = MangledAndDemangled;
+    }
+  } else if (m_demangled) {
+    encoding = DemangledOnly;
+  }
+  file.AppendU8(encoding);
+  switch (encoding) {
+    case Empty:
+      break;
+    case DemangledOnly:
+      file.AppendU32(strtab.Add(m_demangled));
+      break;
+    case MangledOnly:
+      file.AppendU32(strtab.Add(m_mangled));
+      break;
+    case MangledAndDemangled:
+      file.AppendU32(strtab.Add(m_mangled));
+      file.AppendU32(strtab.Add(m_demangled));
+      break;
+  }
+}

lldb/source/Core/Module.cpp

 //===-- Module.cpp --------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include "lldb/Core/Module.h"
 
 #include "lldb/Core/AddressRange.h"
 #include "lldb/Core/AddressResolverFileLine.h"
+#include "lldb/Core/DataFileCache.h"
 #include "lldb/Core/Debugger.h"
 #include "lldb/Core/FileSpecList.h"
 #include "lldb/Core/Mangled.h"
 #include "lldb/Core/ModuleSpec.h"
 #include "lldb/Core/SearchFilter.h"
 #include "lldb/Core/Section.h"
 #include "lldb/Host/FileSystem.h"
 #include "lldb/Host/Host.h"

 #include "lldb/Host/windows/PosixApi.h"
 #endif
 
 #include "Plugins/Language/CPlusPlus/CPlusPlusLanguage.h"
 #include "Plugins/Language/ObjC/ObjCLanguage.h"
 
 #include "llvm/ADT/STLExtras.h"
 #include "llvm/Support/Compiler.h"
+#include "llvm/Support/DJB.h"
 #include "llvm/Support/FileSystem.h"
+#include "llvm/Support/FormatVariadic.h"
+#include "llvm/Support/JSON.h"
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/raw_ostream.h"
 
 #include <cassert>
 #include <cinttypes>
 #include <cstdarg>
 #include <cstdint>
 #include <cstring>

 bool Module::GetIsDynamicLinkEditor() {
   ObjectFile *obj_file = GetObjectFile();
 
   if (obj_file)
     return obj_file->GetIsDynamicLinkEditor();
 
   return false;
 }
+
+uint32_t Module::Hash() {
+  std::string identifier;
+  llvm::raw_string_ostream id_strm(identifier);
+  id_strm << m_arch.GetTriple().str() << '-' << m_file.GetPath();
+  if (m_object_name)
+    id_strm << '(' << m_object_name.GetStringRef() << ')';
+  if (m_object_offset > 0)
+    id_strm << m_object_offset;
+  const auto mtime = llvm::sys::toTimeT(m_object_mod_time);
+  if (mtime > 0)
+    id_strm << mtime;
+  return llvm::djbHash(id_strm.str());
+}
+
+std::string Module::GetCacheKey() {
+  std::string key;
+  llvm::raw_string_ostream strm(key);
+  strm << m_arch.GetTriple().str() << '-' << m_file.GetFilename();
+  if (m_object_name)
+    strm << '(' << m_object_name.GetStringRef() << ')';
+  strm << '-' << llvm::format_hex(Hash(), 10);
+  return strm.str();
+}
+
+DataFileCache *Module::GetIndexCache() {
+  if (!ModuleList::GetGlobalModuleListProperties().GetEnableLLDBIndexCache())
+    return nullptr;
+  // NOTE: intentional leak so we don't crash if global destructor chain gets
+  // called as other threads still use the result of this function
+  static DataFileCache *g_data_file_cache = new DataFileCache(ModuleList::GetGlobalModuleListProperties().GetLLDBIndexCachePath().GetPath());
+  return g_data_file_cache;
+}

lldb/source/Core/ModuleList.cpp

ModuleListProperties::ModuleListProperties() {
   m_collection_sp->Initialize(g_modulelist_properties);
   m_collection_sp->SetValueChangedCallback(ePropertySymLinkPaths,
                                            [this] { UpdateSymlinkMappings(); });
 
   llvm::SmallString<128> path;
   if (clang::driver::Driver::getDefaultModuleCachePath(path)) {
     lldbassert(SetClangModulesCachePath(FileSpec(path)));
   }
+
+  path.clear();
+  if (llvm::sys::path::cache_directory(path)) {
+    llvm::sys::path::append(path, "lldb");
+    llvm::sys::path::append(path, "IndexCache");
+    lldbassert(SetLLDBIndexCachePath(FileSpec(path)));
+  }
+
 }
 
 bool ModuleListProperties::GetEnableExternalLookup() const {
   const uint32_t idx = ePropertyEnableExternalLookup;
   return m_collection_sp->GetPropertyAtIndexAsBoolean(
       nullptr, idx, g_modulelist_properties[idx].default_uint_value != 0);
 }
 
return m_collection_sp
       ->GetCurrentValue();
 }
 
 bool ModuleListProperties::SetClangModulesCachePath(const FileSpec &path) {
   return m_collection_sp->SetPropertyAtIndexAsFileSpec(
       nullptr, ePropertyClangModulesCachePath, path);
 }
 
+FileSpec ModuleListProperties::GetLLDBIndexCachePath() const {
+  return m_collection_sp
+      ->GetPropertyAtIndexAsOptionValueFileSpec(nullptr, false,
+                                                ePropertyLLDBIndexCachePath)
+      ->GetCurrentValue();
+}
+
+bool ModuleListProperties::SetLLDBIndexCachePath(const FileSpec &path) {
+  return m_collection_sp->SetPropertyAtIndexAsFileSpec(
+      nullptr, ePropertyLLDBIndexCachePath, path);
+}
+
+bool ModuleListProperties::GetEnableLLDBIndexCache() const {
+  const uint32_t idx = ePropertyEnableLLDBIndexCache;
+  return m_collection_sp->GetPropertyAtIndexAsBoolean(
+      nullptr, idx, g_modulelist_properties[idx].default_uint_value != 0);
+}
+
+bool ModuleListProperties::SetEnableLLDBIndexCache(bool new_value) {
+  return m_collection_sp->SetPropertyAtIndexAsBoolean(
+      nullptr, ePropertyEnableLLDBIndexCache, new_value);
+}
+
+uint64_t ModuleListProperties::GetLLDBIndexCacheMaxByteSize() {
+  const uint32_t idx = ePropertyLLDBIndexCacheMaxByteSize;
+  return m_collection_sp->GetPropertyAtIndexAsUInt64(
+      nullptr, idx, g_modulelist_properties[idx].default_uint_value);
+}
+
+uint64_t ModuleListProperties::GetLLDBIndexCacheMaxPercent() {
+  const uint32_t idx = ePropertyLLDBIndexCacheMaxPercent;
+  return m_collection_sp->GetPropertyAtIndexAsUInt64(
+      nullptr, idx, g_modulelist_properties[idx].default_uint_value);
+}
+
+uint64_t ModuleListProperties::GetLLDBIndexCacheExpirationDays() {
+  const uint32_t idx = ePropertyLLDBIndexCacheExpirationDays;
+  return m_collection_sp->GetPropertyAtIndexAsUInt64(
+      nullptr, idx, g_modulelist_properties[idx].default_uint_value);
+}
+
 void ModuleListProperties::UpdateSymlinkMappings() {
   FileSpecList list = m_collection_sp
                           ->GetPropertyAtIndexAsOptionValueFileSpecList(
                               nullptr, false, ePropertySymLinkPaths)
                           ->GetCurrentValue();
   llvm::sys::ScopedWriter lock(m_symlink_paths_mutex);
   const bool notify = false;
   m_symlink_paths.Clear(notify);

lldb/source/Host/common/FileSystem.cpp

if (llvm::sys::fs::is_directory(file))
     m_collector->addDirectory(file);
   else
     m_collector->addFile(file);
 }
 
 void FileSystem::SetHomeDirectory(std::string home_directory) {
   m_home_directory = std::move(home_directory);
 }
+
+Status FileSystem::RemoveFile(const FileSpec &file_spec) {
+  return RemoveFile(file_spec.GetPath());
+}
+
+Status FileSystem::RemoveFile(const llvm::Twine &path) {
+  return Status(llvm::sys::fs::remove(path));
+}

lldb/source/Symbol/ObjectFile.cpp

 #include "lldb/Target/SectionLoadList.h"
 #include "lldb/Target/Target.h"
 #include "lldb/Utility/DataBuffer.h"
 #include "lldb/Utility/DataBufferHeap.h"
 #include "lldb/Utility/Log.h"
 #include "lldb/Utility/Timer.h"
 #include "lldb/lldb-private.h"
 
+#include "llvm/Support/DJB.h"
+
 using namespace lldb;
 using namespace lldb_private;
 
 char ObjectFile::ID;
 
 static ObjectFileSP
 CreateObjectFromContainer(const lldb::ModuleSP &module_sp, const FileSpec *file,
                           lldb::offset_t file_offset, lldb::offset_t file_size,
if (module_sp) {
     // data which requires the symbol table.
     //
     // So to work around this, we create the symbol table one time using
     // llvm::once_flag, lock it, and then set the unique pointer. Any other
     // thread that gets ahold of the symbol table before parsing is done, will
     // not be able to access the symbol table contents since all APIs in Symtab
     // are protected by a mutex in the Symtab object itself.
     llvm::call_once(*m_symtab_once_up, [&]() {
-       ElapsedTime elapsed(module_sp->GetSymtabParseTime());
-       Symtab *symtab = new Symtab(this);
-       std::lock_guard<std::recursive_mutex> symtab_guard(symtab->GetMutex());
-       m_symtab_up.reset(symtab);
+      if (!m_symtab_up->LoadFromCache()) {
+        ElapsedTime elapsed(module_sp->GetSymtabParseTime());
         ParseSymtab(*m_symtab_up);
         m_symtab_up->Finalize();
+      }
     });
   }
   return m_symtab_up.get();
 }
+
+uint32_t ObjectFile::GetCacheHash() {
+  if (m_cache_hash)
+    return *m_cache_hash;
+  StreamString strm;
+  strm.Format("{0}-{1}-{2}", m_file, GetType(), GetStrata());
+  m_cache_hash = llvm::djbHash(strm.GetString());
+  return *m_cache_hash;
+}

lldb/source/Symbol/Symbol.cpp

 #include "lldb/Core/ModuleSpec.h"
 #include "lldb/Core/Section.h"
 #include "lldb/Symbol/Function.h"
 #include "lldb/Symbol/ObjectFile.h"
 #include "lldb/Symbol/SymbolVendor.h"
 #include "lldb/Symbol/Symtab.h"
 #include "lldb/Target/Process.h"
 #include "lldb/Target/Target.h"
+#include "lldb/Utility/DataEncoder.h"
 #include "lldb/Utility/Stream.h"
 
 using namespace lldb;
 using namespace lldb_private;
 
 Symbol::Symbol()
     : SymbolContextScope(), m_type_data_resolved(false), m_is_synthetic(false),
       m_is_debug(false), m_is_external(false), m_size_is_sibling(false),
if (m_is_synthetic && !m_mangled) {
     // sessions, to understand crashes in those symbols and to reliably set
     // breakpoints on them.
     llvm::SmallString<256> name;
     llvm::raw_svector_ostream os(name);
     os << GetSyntheticSymbolPrefix() << GetID();
     m_mangled.SetDemangledName(ConstString(os.str()));
   }
 }
+
+bool Symbol::Decode(const DataExtractor &data, lldb::offset_t *offset_ptr,
+                    const SectionList *section_list,
+                    const StringTableReader &strtab) {
+  if (!data.ValidOffsetForDataOfSize(*offset_ptr, 8))
+    return false;
+  m_uid = data.GetU32(offset_ptr);
+  m_type_data = data.GetU16(offset_ptr);
+  const uint16_t bitfields = data.GetU16(offset_ptr);
+  m_type_data_resolved = (1u << 15 & bitfields) != 0;
+  m_is_synthetic = (1u << 14 & bitfields) != 0;
+  m_is_debug = (1u << 13 & bitfields) != 0;
+  m_is_external = (1u << 12 & bitfields) != 0;
+  m_size_is_sibling = (1u << 11 & bitfields) != 0;
+  m_size_is_synthesized = (1u << 10 & bitfields) != 0;
+  m_size_is_valid = (1u << 9 & bitfields) != 0;
+  m_demangled_is_synthesized = (1u << 8 & bitfields) != 0;
+  m_contains_linker_annotations = (1u << 7 & bitfields) != 0;
+  m_is_weak = (1u << 6 & bitfields) != 0;
+  m_type = bitfields & 0x003f;
+  if (!m_mangled.Decode(data, offset_ptr, strtab))
+    return false;
+  if (!data.ValidOffsetForDataOfSize(*offset_ptr, 20))
+    return false;
+  const bool is_addr = data.GetU8(offset_ptr) != 0;
+  const uint64_t value = data.GetU64(offset_ptr);
+  if (is_addr) {
+    m_addr_range.GetBaseAddress().ResolveAddressUsingFileSections(
+        value, section_list);
+  } else {
+    m_addr_range.GetBaseAddress().Clear();
+    m_addr_range.GetBaseAddress().SetOffset(value);
+  }
+  m_addr_range.SetByteSize(data.GetU64(offset_ptr));
+  m_flags =  data.GetU32(offset_ptr);
+  return true;
+}
+
+/// The encoding format for the symbol is as follows:
+///
+/// uint32_t m_uid;
+/// uint16_t m_type_data;
+/// uint16_t bitfield_data;
+/// Mangled mangled;
+/// uint8_t is_addr;
+/// uint64_t file_addr_or_value;
+/// uint64_t size;
+/// uint32_t flags;
+///
+/// The only tricky thing in this encoding is encoding all of the bits in the
+/// bitfields. We use a trick to store all bitfields as a 16 bit value and we
+/// do the same thing when decoding the symbol. There are test that ensure this
+/// encoding works for each individual bit. Everything else is very easy to
+/// store.
+void Symbol::Encode(DataEncoder &file, ConstStringTable &strtab) const {
+  file.AppendU32(m_uid);
+  file.AppendU16(m_type_data);
+  uint16_t bitfields = m_type;
+  if (m_type_data_resolved)
+    bitfields |= 1u << 15;
+  if (m_is_synthetic)
+    bitfields |= 1u << 14;
+  if (m_is_debug)
+    bitfields |= 1u << 13;
+  if (m_is_external)
+    bitfields |= 1u << 12;
+  if (m_size_is_sibling)
+    bitfields |= 1u << 11;
+  if (m_size_is_synthesized)
+    bitfields |= 1u << 10;
+  if (m_size_is_valid)
+    bitfields |= 1u << 9;
+  if (m_demangled_is_synthesized)
+    bitfields |= 1u << 8;
+  if (m_contains_linker_annotations)
+    bitfields |= 1u << 7;
+  if (m_is_weak)
+    bitfields |= 1u << 6;
+  file.AppendU16(bitfields);
+  m_mangled.Encode(file, strtab);
+  // A symbol's value might be an address, or it might be a constant. If the
+  // symbol's base address doesn't have a section, then it is a constant value.
+  // If it does have a section, we will encode the file address and re-resolve
+  // the address when we decode it.
+  bool is_addr = m_addr_range.GetBaseAddress().GetSection().get() != NULL;
+  file.AppendU8(is_addr);
+  file.AppendU64(m_addr_range.GetBaseAddress().GetFileAddress());
+  file.AppendU64(m_addr_range.GetByteSize());
+  file.AppendU32(m_flags);
+}
+
+bool Symbol::operator==(const Symbol &rhs) const {
+  if (m_uid != rhs.m_uid)
+    return false;
+  if (m_type_data != rhs.m_type_data)
+    return false;
+  if (m_type_data_resolved != rhs.m_type_data_resolved)
+    return false;
+  if (m_is_synthetic != rhs.m_is_synthetic)
+    return false;
+  if (m_is_debug != rhs.m_is_debug)
+    return false;
+  if (m_is_external != rhs.m_is_external)
+    return false;
+  if (m_size_is_sibling != rhs.m_size_is_sibling)
+    return false;
+  if (m_size_is_synthesized != rhs.m_size_is_synthesized)
+    return false;
+  if (m_size_is_valid != rhs.m_size_is_valid)
+    return false;
+  if (m_demangled_is_synthesized != rhs.m_demangled_is_synthesized)
+    return false;
+  if (m_contains_linker_annotations != rhs.m_contains_linker_annotations)
+    return false;
+  if (m_is_weak != rhs.m_is_weak)
+    return false;
+  if (m_type != rhs.m_type)
+    return false;
+  if (m_mangled != rhs.m_mangled)
+    return false;
+  if (m_addr_range.GetBaseAddress() != rhs.m_addr_range.GetBaseAddress())
+    return false;
+  if (m_addr_range.GetByteSize() != rhs.m_addr_range.GetByteSize())
+    return false;
+  if (m_flags != rhs.m_flags)
+    return false;
+  return true;
+}

lldb/source/Symbol/Symtab.cpp

 //===-- Symtab.cpp --------------------------------------------------------===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #include <map>
 #include <set>
 
+#include "lldb/Core/DataFileCache.h"
 #include "lldb/Core/Module.h"
 #include "lldb/Core/RichManglingContext.h"
 #include "lldb/Core/Section.h"
 #include "lldb/Symbol/ObjectFile.h"
 #include "lldb/Symbol/Symbol.h"
 #include "lldb/Symbol/SymbolContext.h"
 #include "lldb/Symbol/Symtab.h"
 #include "lldb/Target/Language.h"
+#include "lldb/Utility/DataEncoder.h"
+#include "lldb/Utility/Endian.h"
 #include "lldb/Utility/RegularExpression.h"
 #include "lldb/Utility/Stream.h"
 #include "lldb/Utility/Timer.h"
 
+#include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/StringRef.h"
+#include "llvm/Support/DJB.h"
 
 using namespace lldb;
 using namespace lldb_private;
 
 Symtab::Symtab(ObjectFile *objfile)
     : m_objfile(objfile), m_symbols(), m_file_addr_to_index(*this),
       m_name_to_symbol_indices(), m_mutex(),
       m_file_addr_to_index_computed(false), m_name_indexes_computed(false) {
case eSortOrderNone: {
       s->PutCString(":\n");
       DumpSymbolHeader(s);
       const_iterator begin = m_symbols.begin();
       const_iterator end = m_symbols.end();
       for (const_iterator pos = m_symbols.begin(); pos != end; ++pos) {
         s->Indent();
         pos->Dump(s, target, std::distance(begin, pos), name_preference);
       }
-    } break;
+    }
+    break;
 
     case eSortOrderByName: {
       // Although we maintain a lookup by exact name map, the table isn't
       // sorted by name. So we must make the ordered symbol list up ourselves.
       s->PutCString(" (sorted by name):\n");
       DumpSymbolHeader(s);
 
       std::multimap<llvm::StringRef, const Symbol *> name_map;
       for (const_iterator pos = m_symbols.begin(), end = m_symbols.end();
            pos != end; ++pos) {
         const char *name = pos->GetName().AsCString();
         if (name && name[0])
           name_map.insert(std::make_pair(name, &(*pos)));
       }
 
       for (const auto &name_to_symbol : name_map) {
         const Symbol *symbol = name_to_symbol.second;
         s->Indent();
         symbol->Dump(s, target, symbol - &m_symbols[0], name_preference);
       }
     } break;
 
     case eSortOrderByAddress:
       s->PutCString(" (sorted by address):\n");
       DumpSymbolHeader(s);
       if (!m_file_addr_to_index_computed)
         InitAddressIndexes();
       const size_t num_entries = m_file_addr_to_index.GetSize();
       for (size_t i = 0; i < num_entries; ++i) {
         s->Indent();
         const uint32_t symbol_idx = m_file_addr_to_index.GetEntryRef(i).data;
         m_symbols[symbol_idx].Dump(s, target, symbol_idx, name_preference);
       }
       break;
     }
   } else {
     s->PutCString("\n");
static bool lldb_skip_name(llvm::StringRef mangled,
   }
   llvm_unreachable("unknown scheme!");
 }
 
 void Symtab::InitNameIndexes() {
   // Protected function, no need to lock mutex...
   if (!m_name_indexes_computed) {
     m_name_indexes_computed = true;
     ElapsedTime elapsed(m_objfile->GetModule()->GetSymtabIndexTime());
     LLDB_SCOPED_TIMER();
 
     // Collect all loaded language plugins.
     std::vector<Language *> languages;
     Language::ForEach([&languages](Language *l) {
       languages.push_back(l);
       return true;
     });
void Symtab::Finalize() {
   std::lock_guard<std::recursive_mutex> guard(m_mutex);
   // Calculate the size of symbols inside InitAddressIndexes.
   InitAddressIndexes();
   // Shrink to fit the symbols so we don't waste memory
   if (m_symbols.capacity() > m_symbols.size()) {
     collection new_symbols(m_symbols.begin(), m_symbols.end());
     m_symbols.swap(new_symbols);
   }
+  SaveToCache();
 }
 
 Symbol *Symtab::FindSymbolAtFileAddress(addr_t file_addr) {
   std::lock_guard<std::recursive_mutex> guard(m_mutex);
   if (!m_file_addr_to_index_computed)
     InitAddressIndexes();
 
   const FileRangeToIndexMap::Entry *entry =
for (uint32_t idx = child_idx - 1; idx != UINT32_MAX; --idx) {
       const Symbol *symbol = SymbolAtIndex(idx);
       const uint32_t sibling_idx = symbol->GetSiblingIndex();
       if (sibling_idx != UINT32_MAX && sibling_idx > child_idx)
         return symbol;
     }
   }
   return nullptr;
 }
+
+std::string Symtab::GetCacheKey() {
+  std::string key;
+  llvm::raw_string_ostream strm(key);
+  // Symbol table can come from different object files for the same module. A
+  // module can have one object file as the main executable and might have
+  // another object file in a separate symbol file.
+  strm << m_objfile->GetModule()->GetCacheKey() << "-symtab-"
+      << llvm::format_hex(m_objfile->GetCacheHash(), 10);
+  return strm.str();
+}
+
+void Symtab::SaveToCache() {
+  DataFileCache *cache = Module::GetIndexCache();
+  if (!cache)
+    return; // Caching is not enabled.
+  InitNameIndexes(); // Init the name indexes so we can cache them as well.
+  const auto byte_order = endian::InlHostByteOrder();
+  DataEncoder file(byte_order, /*addr_size=*/8);
+  // Encode will return false if the symbol table's object file doesn't have
+  // anything to make a signature from.
+  if (Encode(file))
+    cache->SetCachedData(GetCacheKey(), file.GetData());
+}
+
+constexpr llvm::StringLiteral kIdentifierCStrMap("CMAP");
+
+static void EncodeCStrMap(DataEncoder &encoder, ConstStringTable &strtab,
+                          const UniqueCStringMap<uint32_t> &cstr_map) {
+  encoder.AppendData(kIdentifierCStrMap);
+  encoder.AppendU32(cstr_map.GetSize());
+  for (const auto &entry: cstr_map) {
+    // Make sure there are no empty strings.
+    assert((bool)entry.cstring);
+    encoder.AppendU32(strtab.Add(entry.cstring));
+    encoder.AppendU32(entry.value);
+  }
+}
+
+bool DecodeCStrMap(const DataExtractor &data, lldb::offset_t *offset_ptr,
+                   const StringTableReader &strtab,
+                   UniqueCStringMap<uint32_t> &cstr_map) {
+  llvm::StringRef identifier((const char *)data.GetData(offset_ptr, 4), 4);
+  if (identifier != kIdentifierCStrMap)
+    return false;
+  const uint32_t count = data.GetU32(offset_ptr);
+  for (uint32_t i=0; i<count; ++i)
+  {
+    llvm::StringRef str(strtab.Get(data.GetU32(offset_ptr)));
+    uint32_t value = data.GetU32(offset_ptr);
+    // No empty strings in the name indexes in Symtab
+    if (str.empty())
+      return false;
+    cstr_map.Append(ConstString(str), value);
+  }
+  return true;
+}
+
+constexpr llvm::StringLiteral kIdentifierSymbolTable("SYMB");
+constexpr uint32_t CURRENT_CACHE_VERSION = 1;
+
+/// The encoding format for the symbol table is as follows:
+///
+/// Signature signature;
+/// ConstStringTable strtab;
+/// Identifier four character code: 'SYMB'
+/// uint32_t version;
+/// uint32_t num_symbols;
+/// Symbol symbols[num_symbols];
+/// uint8_t num_cstr_maps;
+/// UniqueCStringMap<uint32_t> cstr_maps[num_cstr_maps]
+bool Symtab::Encode(DataEncoder &encoder) const {
+  // Name indexes must be computed before calling this function.
+  assert(m_name_indexes_computed);
+
+  // Encode the object file's signature
+  CacheSignature signature(m_objfile);
+  if (!signature.Encode(encoder))
+    return false;
+  ConstStringTable strtab;
+
+  // Encoder the symbol table into a separate encoder first. This allows us
+  // gather all of the strings we willl need in "strtab" as we will need to
+  // write the string table out before the symbol table.
+  DataEncoder symtab_encoder(encoder.GetByteOrder(),
+                              encoder.GetAddressByteSize());
+  symtab_encoder.AppendData(kIdentifierSymbolTable);
+  // Encode the symtab data version.
+  symtab_encoder.AppendU32(CURRENT_CACHE_VERSION);
+  // Encode the number of symbols.
+  symtab_encoder.AppendU32(m_symbols.size());
+  // Encode the symbol data for all symbols.
+  for (const auto &symbol: m_symbols)
+    symbol.Encode(symtab_encoder, strtab);
+
+  // Emit a byte for how many C string maps we emit. We will fix this up after
+  // we emit the C string maps since we skip emitting C string maps if they are
+  // empty.
+  size_t num_cmaps_offset = symtab_encoder.GetByteSize();
+  uint8_t num_cmaps = 0;
+  symtab_encoder.AppendU8(0);
+  for (const auto &pair: m_name_to_symbol_indices) {
+    if (pair.second.IsEmpty())
+      continue;
+    ++num_cmaps;
+    symtab_encoder.AppendU8(pair.first);
+    EncodeCStrMap(symtab_encoder, strtab, pair.second);
+  }
+  if (num_cmaps > 0)
+    symtab_encoder.PutU8(num_cmaps_offset, num_cmaps);
+
+  // Now that all strings have been gathered, we will emit the string table.
+  strtab.Encode(encoder);
+  // Followed the the symbol table data.
+  encoder.AppendData(symtab_encoder.GetData());
+  return true;
+}
+
+bool Symtab::Decode(const DataExtractor &data, lldb::offset_t *offset_ptr,
+                    bool &signature_mismatch) {
+  signature_mismatch = false;
+  CacheSignature signature;
+  StringTableReader strtab;
+  { // Scope for "elapsed" object below so it can measure the time parse.
+    ElapsedTime elapsed(m_objfile->GetModule()->GetSymtabParseTime());
+    if (!signature.Decode(data, offset_ptr))
+      return false;
+    if (CacheSignature(m_objfile) != signature) {
+      signature_mismatch = true;
+      return false;
+    }
+    // We now decode the string table for all strings in the data cache file.
+    if (!strtab.Decode(data, offset_ptr))
+      return false;
+
+    // And now we can decode the symbol table with string table we just decoded.
+    llvm::StringRef identifier((const char *)data.GetData(offset_ptr, 4), 4);
+    if (identifier != kIdentifierSymbolTable)
+      return false;
+    const uint32_t version = data.GetU32(offset_ptr);
+    if (version != CURRENT_CACHE_VERSION)
+      return false;
+    const uint32_t num_symbols = data.GetU32(offset_ptr);
+    if (num_symbols == 0)
+      return true;
+    m_symbols.resize(num_symbols);
+    SectionList *sections = m_objfile->GetModule()->GetSectionList();
+    for (uint32_t i=0; i<num_symbols; ++i) {
+      if (!m_symbols[i].Decode(data, offset_ptr, sections, strtab))
+        return false;
+    }
+  }
+
+  { // Scope for "elapsed" object below so it can measure the time to index.
+    ElapsedTime elapsed(m_objfile->GetModule()->GetSymtabIndexTime());
+    const uint8_t num_cstr_maps = data.GetU8(offset_ptr);
+    for (uint8_t i=0; i<num_cstr_maps; ++i) {
+      uint8_t type = data.GetU8(offset_ptr);
+      UniqueCStringMap<uint32_t> &cstr_map =
+          GetNameToSymbolIndexMap((lldb::FunctionNameType)type);
+      if (!DecodeCStrMap(data, offset_ptr, strtab, cstr_map))
+        return false;
+    }
+    m_name_indexes_computed = true;
+  }
+  return true;
+}
+
+bool Symtab::LoadFromCache() {
+  DataFileCache *cache = Module::GetIndexCache();
+  if (!cache)
+    return false;
+
+  std::unique_ptr<llvm::MemoryBuffer> mem_buffer_up =
+      cache->GetCachedData(GetCacheKey());
+  if (!mem_buffer_up)
+    return false;
+  DataExtractor data(mem_buffer_up->getBufferStart(),
+                     mem_buffer_up->getBufferSize(),
+                     m_objfile->GetByteOrder(),
+                     m_objfile->GetAddressByteSize());
+  bool signature_mismatch = false;
+  lldb::offset_t offset = 0;
+  const bool result = Decode(data, &offset, signature_mismatch);
+  if (signature_mismatch)
+    cache->RemoveCacheFile(GetCacheKey());
+  return result;
+}

lldb/source/Utility/DataEncoder.cpp

 
 void DataEncoder::AppendData(llvm::StringRef data) {
   const char *bytes = data.data();
   const size_t length = data.size();
   if (bytes && length > 0)
     m_data_sp->AppendData(bytes, length);
 }
 
+void DataEncoder::AppendData(llvm::ArrayRef<uint8_t> data) {
+  const uint8_t *bytes = data.data();
+  const size_t length = data.size();
+  if (bytes && length > 0)
+    m_data_sp->AppendData(bytes, length);
+}
+
 void DataEncoder::AppendCString(llvm::StringRef data) {
   const char *bytes = data.data();
   const size_t length = data.size();
   if (bytes) {
     if (length > 0)
       m_data_sp->AppendData(bytes, length);
     if (length == 0 || bytes[length - 1] != '\0')
       AppendU8(0);
   }
 }

lldb/test/API/functionalities/module_cache/bsd/Makefile

+C_SOURCES := main.c a.c b.c c.c
+EXE :=  # Define a.out explicitly
+MAKE_DSYM := NO
+
+all: a.out
+
+a.out: main.o libfoo.a
+  $(LD) $(LDFLAGS) $^ -o $@
+
+lib_ab.a: a.o b.o
+  $(AR) $(ARFLAGS) $@ $^
+  $(RM) $^
+
+# Here we make a .a file that has two a.o files with different modification
+# times and different content by first creating libfoo.a with only a.o and b.o,
+# then we sleep for 2 seconds, touch c.o to ensure it has a different
+# modification time, and then rename c.o to a.o and then add it to the .a file
+# again. This is to help test that the module cache will create different
+# directories for the two different a.o files.
+libfoo.a: lib_ab.a c.o
+  sleep 2
+  touch c.o
+  mv c.o a.o
+  $(AR) $(ARFLAGS) $@ lib_ab.a a.o
+  $(RM) a.o
+
+include Makefile.rules

lldb/test/API/functionalities/module_cache/bsd/TestModuleCacheBSD.py

+"""Test the LLDB module cache funcionality."""
+
+import glob
+import lldb
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+import os
+import time
+
+
+class ModuleCacheTestcaseBSD(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    def setUp(self):
+        # Call super's setUp().
+        TestBase.setUp(self)
+        # Find the line number in a(int) to break at.
+        self.line_a = line_number(
+            'a.c', '// Set file and line breakpoint inside a().')
+        self.line_b = line_number(
+            'b.c', '// Set file and line breakpoint inside b().')
+        self.line_c = line_number(
+            'c.c', '// Set file and line breakpoint inside c().')
+        self.cache_dir = os.path.join(self.getBuildDir(), 'lldb-module-cache')
+        # Set the lldb module cache directory to a directory inside the build
+        # artifacts directory so no other tests are interfered with.
+        self.runCmd('settings set symbols.lldb-index-cache-path "%s"' % (self.cache_dir))
+        self.runCmd('settings set symbols.enable-lldb-index-cache true')
+        self.build()
+
+
+    def get_module_cache_files(self, basename):
+        module_cache_glob = os.path.join(self.cache_dir, "llvmcache-*%s*symtab*" % (basename))
+        return glob.glob(module_cache_glob)
+
+
+    # Requires no dSYM, so we let the Makefile make the right stuff for us
+    @no_debug_info_test
+    @skipUnlessDarwin
+    def test(self):
+        """
+            Test module cache functionality for bsd archive object files.
+
+            This will test that if we enable the module cache, we have a
+            corresponding cache entry for the .o files in libfoo.a.
+
+            The static library has two entries for "a.o":
+            - one from a.c
+            - one from c.c which had c.o renamed to a.o and then put into the
+              libfoo.a as an extra .o file with different contents from the
+              original a.o
+
+            We do this to test that we can correctly cache duplicate .o files
+            that appear in .a files.
+
+            This test only works on darwin because of the way DWARF is stored
+            where the debug map will refer to .o files inside of .a files.
+        """
+        exe = self.getBuildArtifact("a.out")
+
+        # Create a module with no depedencies.
+        target = self.createTestTarget(load_dependent_modules=False)
+
+        self.runCmd('breakpoint set -f a.c -l %d' % (self.line_a))
+        self.runCmd('breakpoint set -f b.c -l %d' % (self.line_b))
+        self.runCmd('breakpoint set -f c.c -l %d' % (self.line_c))
+
+        # Get the executable module and get the number of symbols to make
+        # sure the symbol table gets parsed and cached. The module cache is
+        # enabled in the setUp() function.
+        main_module = target.GetModuleAtIndex(0)
+        self.assertTrue(main_module.IsValid())
+        # Make sure the symbol table gets loaded and cached
+        main_module.GetNumSymbols()
+        a_o_cache_files = self.get_module_cache_files("libfoo.a(a.o)")
+        b_o_cache_files = self.get_module_cache_files("libfoo.a(b.o)")
+        # We expect the directory for a.o to have two cache directories:
+        # - 1 for the a.o with a earlier mod time
+        # - 1 for the a.o that was renamed from c.o that should be 2 seconds older
+        self.assertEqual(len(a_o_cache_files), 2,
+                "make sure there are two files in the module cache directory (%s) for libfoo.a(a.o)" % (self.cache_dir))
+        self.assertEqual(len(b_o_cache_files), 1,
+                "make sure there are two files in the module cache directory (%s) for libfoo.a(b.o)" % (self.cache_dir))

lldb/test/API/functionalities/module_cache/bsd/a.c

+int __a_global = 1;
+
+int a(int arg) {
+    int result = arg + __a_global;
+    return result; // Set file and line breakpoint inside a().
+}

lldb/test/API/functionalities/module_cache/bsd/b.c

+static int __b_global = 2;
+
+int b(int arg) {
+    int result = arg + __b_global;
+    return result; // Set file and line breakpoint inside b().
+}

lldb/test/API/functionalities/module_cache/bsd/c.c

+static int __c_global = 3;
+
+int c(int arg) {
+    int result = arg + __c_global;
+    return result; // Set file and line breakpoint inside c().
+}

lldb/test/API/functionalities/module_cache/bsd/main.c

+#include <stdio.h>
+
+extern int a(int);
+extern int b(int);
+extern int c(int);
+int main (int argc, char const *argv[])
+{
+    printf ("a(1) returns %d\n", a(1));
+    printf ("b(2) returns %d\n", b(2));
+    printf ("c(2) returns %d\n", c(2));
+}

lldb/test/API/functionalities/module_cache/simple_exe/Makefile

+C_SOURCES := main.c
+
+include Makefile.rules

lldb/test/API/functionalities/module_cache/simple_exe/TestModuleCacheSimple.py

+"""Test the LLDB module cache funcionality."""
+
+import glob
+import lldb
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+import os
+import time
+
+
+class ModuleCacheTestcaseSimple(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    def setUp(self):
+        # Call super's setUp().
+        TestBase.setUp(self)
+        # Find the line number in a(int) to break at.
+        self.cache_dir = os.path.join(self.getBuildDir(), 'lldb-module-cache')
+        # Set the lldb module cache directory to a directory inside the build
+        # artifacts directory so no other tests are interfered with.
+        self.runCmd('settings set symbols.lldb-index-cache-path "%s"' % (self.cache_dir))
+        self.runCmd('settings set symbols.enable-lldb-index-cache true')
+        self.build()
+
+
+    def get_module_cache_files(self, basename):
+        module_file_glob = os.path.join(self.cache_dir, "llvmcache-*%s*" % (basename))
+        return glob.glob(module_file_glob)
+
+    # Doesn't depend on any specific debug information.
+    @no_debug_info_test
+    def test(self):
+        """
+            Test module cache functionality for a simple object file.
+
+            This will test that if we enable the module cache, we have a
+            corresponding index cache entry for the symbol table for the
+            executable. It also removes the executable, rebuilds so that the
+            modification time of the binary gets updated, and then creates a new
+            target and should cause the cache to get updated so the cache file
+            should get an updated modification time.
+        """
+        exe = self.getBuildArtifact("a.out")
+
+        # Create a module with no depedencies.
+        target = self.createTestTarget(load_dependent_modules=False)
+
+        # Get the executable module and get the number of symbols to make
+        # sure the symbol table gets parsed and cached. The module cache is
+        # enabled in the setUp() function.
+        main_module = target.GetModuleAtIndex(0)
+        self.assertTrue(main_module.IsValid())
+        # Make sure the symbol table gets loaded and cached
+        main_module.GetNumSymbols()
+        cache_files = self.get_module_cache_files("a.out")
+        self.assertEqual(len(cache_files), 1,
+                         "make sure there is only one cache file for 'a.out'")
+        symtab_cache_path = cache_files[0]
+        exe_mtime_1 = os.path.getmtime(exe)
+        symtab_mtime_1 = os.path.getmtime(symtab_cache_path)
+        # Now remove the executable and sleep for a few seconds to make sure we
+        # get a different creation and modification time for the file since some
+        # OSs store the modification time in seconds since Jan 1, 1970.
+        os.remove(exe)
+        self.assertEqual(os.path.exists(exe), False,
+                         'make sure we were able to remove the executable')
+        time.sleep(2)
+        # Now rebuild the binary so it has a different content which should
+        # update the UUID to make the cache miss when it tries to load the
+        # symbol table from the binary at the same path.
+        self.build(dictionary={'CFLAGS_EXTRAS': '-DEXTRA_FUNCTION'})
+        self.assertEqual(os.path.exists(exe), True,
+                         'make sure executable exists after rebuild')
+        # Make sure the modification time has changed or this test will fail.
+        exe_mtime_2 = os.path.getmtime(exe)
+        self.assertNotEqual(
+                exe_mtime_1,
+                exe_mtime_2,
+                "make sure the modification time of the executable has changed")
+        # Make sure the module cache still has an out of date cache with the
+        # same old modification time.
+        self.assertEqual(symtab_mtime_1,
+                         os.path.getmtime(symtab_cache_path),
+                         "check that the 'symtab' cache file modification time doesn't match the executable modification time after rebuild")
+        # Create a new target and get the symbols again, and make sure the cache
+        # gets updated for the symbol table cache
+        target = self.createTestTarget(load_dependent_modules=False)
+        main_module = target.GetModuleAtIndex(0)
+        self.assertTrue(main_module.IsValid())
+        main_module.GetNumSymbols()
+        self.assertEqual(os.path.exists(symtab_cache_path), True,
+                         'make sure "symtab" cache files exists after cache is updated')
+        symtab_mtime_2 = os.path.getmtime(symtab_cache_path)
+        self.assertNotEqual(
+                symtab_mtime_1,
+                symtab_mtime_2,
+                'make sure modification time of "symtab-..." changed')

lldb/test/API/functionalities/module_cache/simple_exe/main.c

+#ifdef EXTRA_FUNCTION
+int foo(int i) {
+  return i*3;
+}
+#endif
+
+int main (int argc, char const *argv[]) {
+#ifdef EXTRA_FUNCTION
+  return foo(argc);
+#else
+  return 0;
+#endif
+}

lldb/test/API/functionalities/module_cache/universal/Makefile

+EXE := testit
+
+include Makefile.rules
+
+all: testit
+
+testit: testit.x86_64 testit.arm64
+  lipo -create -o testit $^
+
+testit.arm64: testit.arm64.o
+  $(CC) -isysroot $(SDKROOT) -target arm64-apple-macosx10.9 -o testit.arm64 $<
+
+testit.x86_64: testit.x86_64.o
+  $(CC) -isysroot $(SDKROOT) -target x86_64-apple-macosx10.9 -o testit.x86_64 $<
+
+testit.arm64.o: main.c
+  $(CC) -isysroot $(SDKROOT) -g -O0 -target arm64-apple-macosx10.9 -c -o testit.arm64.o $<
+
+testit.x86_64.o: main.c
+  $(CC) -isysroot $(SDKROOT) -g -O0 -target x86_64-apple-macosx10.9 -c -o testit.x86_64.o $<

lldb/test/API/functionalities/module_cache/universal/TestModuleCacheUniversal.py

+"""Test the LLDB module cache funcionality for universal mach-o files."""
+
+import glob
+import lldb
+from lldbsuite.test.decorators import *
+from lldbsuite.test.lldbtest import *
+from lldbsuite.test import lldbutil
+import os
+import time
+
+
+class ModuleCacheTestcaseUniversal(TestBase):
+
+    mydir = TestBase.compute_mydir(__file__)
+
+    def setUp(self):
+        # Call super's setUp().
+        TestBase.setUp(self)
+        # Find the line number in a(int) to break at.
+        self.cache_dir = os.path.join(self.getBuildDir(), 'lldb-module-cache')
+        # Set the lldb module cache directory to a directory inside the build
+        # artifacts directory so no other tests are interfered with.
+        self.runCmd('settings set symbols.lldb-index-cache-path "%s"' % (self.cache_dir))
+        self.runCmd('settings set symbols.enable-lldb-index-cache true')
+        self.build()
+
+
+    def get_module_cache_files(self, basename):
+        module_file_glob = os.path.join(self.cache_dir, "llvmcache-*%s*" % (basename))
+        return glob.glob(module_file_glob)
+
+
+    # Doesn't depend on any specific debug information.
+    @no_debug_info_test
+    @skipUnlessDarwin
+    @skipIfDarwinEmbedded # this test file assumes we're targetting an x86 system
+    def test(self):
+        """
+            Test module cache functionality for a universal mach-o files.
+
+            This will test that if we enable the module cache, we can create
+            lldb module caches for each slice of a universal mach-o file and
+            they will each have a unique directory.
+        """
+        exe_basename = "testit"
+        exe = self.getBuildArtifact(exe_basename)
+
+        # Create a module with no depedencies.
+        self.runCmd('target create -d --arch x86_64 %s' % (exe))
+        self.runCmd('image dump symtab %s' % (exe_basename))
+        self.runCmd('target create -d --arch arm64 %s' % (exe))
+        self.runCmd('image dump symtab %s' % (exe_basename))
+
+        cache_files = self.get_module_cache_files(exe_basename)
+
+        self.assertEqual(len(cache_files), 2,
+                "make sure there are two files in the module cache directory (%s) for %s" % (self.cache_dir, exe_basename))

lldb/test/API/functionalities/module_cache/universal/main.c

+int main(int argc, const char **argv) {
+  return 0;
+}

lldb/unittests/Symbol/CMakeLists.txt

 add_lldb_unittest(SymbolTests
   LocateSymbolFileTest.cpp
+  MangledTest.cpp
   PostfixExpressionTest.cpp
+  SymbolTest.cpp
+  SymtabTest.cpp
   TestTypeSystem.cpp
   TestTypeSystemClang.cpp
   TestClangASTImporter.cpp
   TestDWARFCallFrameInfo.cpp
   TestType.cpp
   TestLineEntry.cpp
 
   LINK_LIBS
+    lldbCore
     lldbHost
     lldbSymbol
     lldbUtilityHelpers
     lldbPluginObjectFileELF
     lldbPluginObjectFileMachO
     lldbPluginSymbolFileDWARF
     lldbPluginSymbolFileSymtab
     lldbPluginTypeSystemClang
     LLVMTestingSupport
   )
 
 set(test_inputs
   inlined-functions.yaml
   )
 add_unittest_inputs(SymbolTests "${test_inputs}")

lldb/unittests/Symbol/MangledTest.cpp

+//===-- SymbolTest.cpp ----------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "lldb/Core/Mangled.h"
+#include "lldb/Core/DataFileCache.h"
+#include "lldb/Utility/DataEncoder.h"
+#include "lldb/Utility/DataExtractor.h"
+
+#include "gtest/gtest.h"
+
+using namespace lldb;
+using namespace lldb_private;
+
+static void EncodeDecode(const Mangled &object, ByteOrder byte_order) {
+  const uint8_t addr_size = 8;
+  DataEncoder file(byte_order, addr_size);
+  DataEncoder strtab_encoder(byte_order, addr_size);
+  ConstStringTable const_strtab;
+
+  object.Encode(file, const_strtab);
+
+  llvm::ArrayRef<uint8_t> bytes = file.GetData();
+  DataExtractor data(bytes.data(), bytes.size(), byte_order, addr_size);
+
+  const_strtab.Encode(strtab_encoder);
+  llvm::ArrayRef<uint8_t> strtab_bytes = strtab_encoder.GetData();
+  DataExtractor strtab_data(strtab_bytes.data(), strtab_bytes.size(),
+                            byte_order, addr_size);
+  StringTableReader strtab_reader;
+  offset_t strtab_data_offset = 0;
+  ASSERT_EQ(strtab_reader.Decode(strtab_data, &strtab_data_offset), true);
+
+  Mangled decoded_object;
+  offset_t data_offset = 0;
+  decoded_object.Decode(data, &data_offset, strtab_reader);
+  EXPECT_EQ(object, decoded_object);
+}
+
+static void EncodeDecode(const Mangled &object) {
+  EncodeDecode(object, eByteOrderLittle);
+  EncodeDecode(object, eByteOrderBig);
+}
+
+TEST(MangledTest, EncodeDecodeMangled) {
+  Mangled mangled;
+  // Test encoding and decoding an empty mangled object.
+  EncodeDecode(mangled);
+
+  // Test encoding a mangled object that hasn't demangled its name yet.
+  mangled.SetMangledName(ConstString("_Z3fooi"));
+  EncodeDecode(mangled);
+
+  // Test encoding a mangled object that has demangled its name by computing it.
+  mangled.GetDemangledName();
+  // EncodeDecode(mangled);
+
+  // Test encoding a mangled object that has just a demangled name
+  mangled.SetMangledName(ConstString());
+  mangled.SetDemangledName(ConstString("hello"));
+  EncodeDecode(mangled);
+
+  // Test encoding a mangled name that has both a mangled and demangled name
+  // that are not mangled/demangled counterparts of each other.
+  mangled.SetMangledName(ConstString("world"));
+  EncodeDecode(mangled);
+}

lldb/unittests/Symbol/SymbolTest.cpp

+//===-- SymbolTest.cpp ----------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "lldb/Symbol/Symbol.h"
+#include "lldb/Core/DataFileCache.h"
+#include "lldb/Core/Section.h"
+#include "lldb/Utility/DataEncoder.h"
+#include "lldb/Utility/DataExtractor.h"
+
+#include "gtest/gtest.h"
+
+using namespace lldb;
+using namespace lldb_private;
+
+static void EncodeDecode(const Symbol &object, const SectionList *sect_list,
+                         ByteOrder byte_order) {
+  const uint8_t addr_size = 8;
+  DataEncoder file(byte_order, addr_size);
+  DataEncoder strtab_encoder(byte_order, addr_size);
+  ConstStringTable const_strtab;
+  object.Encode(file, const_strtab);
+  llvm::ArrayRef<uint8_t> bytes = file.GetData();
+  DataExtractor data(bytes.data(), bytes.size(), byte_order, addr_size);
+
+  const_strtab.Encode(strtab_encoder);
+  llvm::ArrayRef<uint8_t> strtab_bytes = strtab_encoder.GetData();
+  DataExtractor strtab_data(strtab_bytes.data(), strtab_bytes.size(),
+                            byte_order, addr_size);
+  StringTableReader strtab_reader;
+  offset_t strtab_data_offset = 0;
+  ASSERT_EQ(strtab_reader.Decode(strtab_data, &strtab_data_offset), true);
+
+  Symbol decoded_object;
+  offset_t data_offset = 0;
+  decoded_object.Decode(data, &data_offset, sect_list, strtab_reader);
+  EXPECT_EQ(object, decoded_object);
+}
+
+static void EncodeDecode(const Symbol &object, const SectionList *sect_list) {
+  EncodeDecode(object, sect_list, eByteOrderLittle);
+  EncodeDecode(object, sect_list, eByteOrderBig);
+}
+
+TEST(SymbolTest, EncodeDecodeSymbol) {
+
+  SectionSP sect_sp(new Section(
+      /*module_sp=*/ModuleSP(),
+      /*obj_file=*/nullptr,
+      /*sect_id=*/1,
+      /*name=*/ConstString(".text"),
+      /*sect_type=*/eSectionTypeCode,
+      /*file_vm_addr=*/0x1000,
+      /*vm_size=*/0x1000,
+      /*file_offset=*/0,
+      /*file_size=*/0,
+      /*log2align=*/5,
+      /*flags=*/0x10203040));
+
+  SectionList sect_list;
+  sect_list.AddSection(sect_sp);
+
+  Symbol symbol(
+      /*symID=*/0x10203040,
+      /*name=*/"main",
+      /*type=*/eSymbolTypeCode,
+      /*bool external=*/false,
+      /*bool is_debug=*/false,
+      /*bool is_trampoline=*/false,
+      /*bool is_artificial=*/false,
+      /*section_sp=*/sect_sp,
+      /*offset=*/0x0,
+      /*size=*/0x100,
+      /*size_is_valid=*/true,
+      /*contains_linker_annotations=*/false,
+      /*flags=*/0x11223344);
+
+  // Test encoding a symbol with an address.
+  EncodeDecode(symbol, &sect_list);
+
+  // Test that encoding the bits in the bitfield works for all endianness
+  // combos.
+
+  // Test Symbol.m_is_synthetic
+  symbol.SetIsSynthetic(true);
+  EncodeDecode(symbol, &sect_list);
+  symbol.SetIsSynthetic(false);
+
+  // Test Symbol.m_is_debug
+  symbol.SetDebug(true);
+  EncodeDecode(symbol, &sect_list);
+  symbol.SetDebug(false);
+
+  // Test Symbol.m_is_external
+  symbol.SetExternal(true);
+  EncodeDecode(symbol, &sect_list);
+  symbol.SetExternal(false);
+
+  // Test Symbol.m_size_is_sibling
+  symbol.SetSizeIsSibling(true);
+  EncodeDecode(symbol, &sect_list);
+  symbol.SetSizeIsSibling(false);
+
+  // Test Symbol.m_size_is_synthesized
+  symbol.SetSizeIsSynthesized(true);
+  EncodeDecode(symbol, &sect_list);
+  symbol.SetSizeIsSynthesized(false);
+
+  // Test Symbol.m_size_is_synthesized
+  symbol.SetByteSize(0);
+  EncodeDecode(symbol, &sect_list);
+  symbol.SetByteSize(0x100);
+
+  // Test Symbol.m_demangled_is_synthesized
+  symbol.SetDemangledNameIsSynthesized(true);
+  EncodeDecode(symbol, &sect_list);
+  symbol.SetDemangledNameIsSynthesized(false);
+
+  // Test Symbol.m_contains_linker_annotations
+  symbol.SetContainsLinkerAnnotations(true);
+  EncodeDecode(symbol, &sect_list);
+  symbol.SetContainsLinkerAnnotations(false);
+
+  // Test Symbol.m_is_weak
+  symbol.SetIsWeak(true);
+  EncodeDecode(symbol, &sect_list);
+  symbol.SetIsWeak(false);
+
+  // Test encoding a symbol with no address.
+  symbol.GetAddressRef().SetSection(SectionSP());
+  EncodeDecode(symbol, &sect_list);
+}

lldb/unittests/Symbol/SymtabTest.cpp

+//===-- SymbolTest.cpp ----------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "Plugins/ObjectFile/Mach-O/ObjectFileMachO.h"
+#include "Plugins/SymbolFile/DWARF/SymbolFileDWARF.h"
+#include "Plugins/TypeSystem/Clang/TypeSystemClang.h"
+#include "TestingSupport/SubsystemRAII.h"
+#include "TestingSupport/TestUtilities.h"
+
+#include "lldb/Core/DataFileCache.h"
+#include "lldb/Core/Module.h"
+#include "lldb/Host/FileSystem.h"
+#include "lldb/Host/HostInfo.h"
+#include "lldb/Symbol/Symbol.h"
+#include "lldb/Symbol/Symtab.h"
+#include "lldb/Utility/DataEncoder.h"
+#include "lldb/Utility/DataExtractor.h"
+
+#include <memory>
+
+#include "gtest/gtest.h"
+
+using namespace lldb;
+using namespace lldb_private;
+
+class SymtabTest : public testing::Test {
+  SubsystemRAII<FileSystem, HostInfo, ObjectFileMachO, SymbolFileDWARF,
+                TypeSystemClang>
+      subsystem;
+};
+
+static void EncodeDecode(const Symtab &object, ByteOrder byte_order) {
+  const uint8_t addr_size = 8;
+  DataEncoder file(byte_order, addr_size);
+
+  object.Encode(file);
+  llvm::ArrayRef<uint8_t> bytes = file.GetData();
+  DataExtractor data(bytes.data(), bytes.size(), byte_order, addr_size);
+  Symtab decoded_object(object.GetObjectFile());
+  offset_t data_offset = 0;
+  bool uuid_mismatch = false;
+  decoded_object.Decode(data, &data_offset, uuid_mismatch);
+  ASSERT_EQ(object.GetNumSymbols(), decoded_object.GetNumSymbols());
+  for (size_t i = 0; i < object.GetNumSymbols(); ++i)
+    EXPECT_EQ(*object.SymbolAtIndex(i), *decoded_object.SymbolAtIndex(i));
+}
+
+static void EncodeDecode(const Symtab &object) {
+  EncodeDecode(object, eByteOrderLittle);
+  EncodeDecode(object, eByteOrderBig);
+}
+
+TEST_F(SymtabTest, EncodeDecodeSymtab) {
+
+  auto ExpectedFile = TestFile::fromYaml(R"(
+--- !mach-o
+FileHeader:
+  magic:           0xFEEDFACF
+  cputype:         0x100000C
+  cpusubtype:      0x0
+  filetype:        0x2
+  ncmds:           17
+  sizeofcmds:      792
+  flags:           0x200085
+  reserved:        0x0
+LoadCommands:
+  - cmd:             LC_SEGMENT_64
+    cmdsize:         72
+    segname:         __PAGEZERO
+    vmaddr:          0
+    vmsize:          4294967296
+    fileoff:         0
+    filesize:        0
+    maxprot:         0
+    initprot:        0
+    nsects:          0
+    flags:           0
+  - cmd:             LC_SEGMENT_64
+    cmdsize:         232
+    segname:         __TEXT
+    vmaddr:          4294967296
+    vmsize:          16384
+    fileoff:         0
+    filesize:        16384
+    maxprot:         5
+    initprot:        5
+    nsects:          2
+    flags:           0
+    Sections:
+      - sectname:        __text
+        segname:         __TEXT
+        addr:            0x100003F94
+        size:            36
+        offset:          0x3F94
+        align:           2
+        reloff:          0x0
+        nreloc:          0
+        flags:           0x80000400
+        reserved1:       0x0
+        reserved2:       0x0
+        reserved3:       0x0
+        content:         FF8300D1E80300AA00008052FF1F00B9E81B00B9E10B00F9E20700F9FF830091C0035FD6
+      - sectname:        __unwind_info
+        segname:         __TEXT
+        addr:            0x100003FB8
+        size:            72
+        offset:          0x3FB8
+        align:           2
+        reloff:          0x0
+        nreloc:          0
+        flags:           0x0
+        reserved1:       0x0
+        reserved2:       0x0
+        reserved3:       0x0
+        content:         010000001C000000000000001C000000000000001C00000002000000943F00003400000034000000B93F00000000000034000000030000000C000100100001000000000000200002
+  - cmd:             LC_SEGMENT_64
+    cmdsize:         72
+    segname:         __LINKEDIT
+    vmaddr:          4294983680
+    vmsize:          16384
+    fileoff:         16384
+    filesize:        674
+    maxprot:         1
+    initprot:        1
+    nsects:          0
+    flags:           0
+  - cmd:             LC_DYLD_CHAINED_FIXUPS
+    cmdsize:         16
+    dataoff:         16384
+    datasize:        56
+  - cmd:             LC_DYLD_EXPORTS_TRIE
+    cmdsize:         16
+    dataoff:         16440
+    datasize:        48
+  - cmd:             LC_SYMTAB
+    cmdsize:         24
+    symoff:          16496
+    nsyms:           10
+    stroff:          16656
+    strsize:         128
+  - cmd:             LC_DYSYMTAB
+    cmdsize:         80
+    ilocalsym:       0
+    nlocalsym:       8
+    iextdefsym:      8
+    nextdefsym:      2
+    iundefsym:       10
+    nundefsym:       0
+    tocoff:          0
+    ntoc:            0
+    modtaboff:       0
+    nmodtab:         0
+    extrefsymoff:    0
+    nextrefsyms:     0
+    indirectsymoff:  0
+    nindirectsyms:   0
+    extreloff:       0
+    nextrel:         0
+    locreloff:       0
+    nlocrel:         0
+  - cmd:             LC_LOAD_DYLINKER
+    cmdsize:         32
+    name:            12
+    Content:         '/usr/lib/dyld'
+    ZeroPadBytes:    7
+  - cmd:             LC_UUID
+    cmdsize:         24
+    uuid:            1EECD2B8-16EA-3FEC-AB3C-F46139DBD0E2
+  - cmd:             LC_BUILD_VERSION
+    cmdsize:         32
+    platform:        1
+    minos:           786432
+    sdk:             786432
+    ntools:          1
+    Tools:
+      - tool:            3
+        version:         46596096
+  - cmd:             LC_SOURCE_VERSION
+    cmdsize:         16
+    version:         0
+  - cmd:             LC_MAIN
+    cmdsize:         24
+    entryoff:        16276
+    stacksize:       0
+  - cmd:             LC_LOAD_DYLIB
+    cmdsize:         48
+    dylib:
+      name:            24
+      timestamp:       2
+      current_version: 78643968
+      compatibility_version: 65536
+    Content:         '/usr/lib/libc++.1.dylib'
+    ZeroPadBytes:    1
+  - cmd:             LC_LOAD_DYLIB
+    cmdsize:         56
+    dylib:
+      name:            24
+      timestamp:       2
+      current_version: 85917696
+      compatibility_version: 65536
+    Content:         '/usr/lib/libSystem.B.dylib'
+    ZeroPadBytes:    6
+  - cmd:             LC_FUNCTION_STARTS
+    cmdsize:         16
+    dataoff:         16488
+    datasize:        8
+  - cmd:             LC_DATA_IN_CODE
+    cmdsize:         16
+    dataoff:         16496
+    datasize:        0
+  - cmd:             LC_CODE_SIGNATURE
+    cmdsize:         16
+    dataoff:         16784
+    datasize:        274
+LinkEditData:
+  NameList:
+    - n_strx:          28
+      n_type:          0x64
+      n_sect:          0
+      n_desc:          0
+      n_value:         0
+    - n_strx:          64
+      n_type:          0x64
+      n_sect:          0
+      n_desc:          0
+      n_value:         0
+    - n_strx:          73
+      n_type:          0x66
+      n_sect:          0
+      n_desc:          1
+      n_value:         1639532873
+    - n_strx:          1
+      n_type:          0x2E
+      n_sect:          1
+      n_desc:          0
+      n_value:         4294983572
+    - n_strx:          115
+      n_type:          0x24
+      n_sect:          1
+      n_desc:          0
+      n_value:         4294983572
+    - n_strx:          1
+      n_type:          0x24
+      n_sect:          0
+      n_desc:          0
+      n_value:         36
+    - n_strx:          1
+      n_type:          0x4E
+      n_sect:          1
+      n_desc:          0
+      n_value:         36
+    - n_strx:          1
+      n_type:          0x64
+      n_sect:          1
+      n_desc:          0
+      n_value:         0
+    - n_strx:          2
+      n_type:          0xF
+      n_sect:          1
+      n_desc:          16
+      n_value:         4294967296
+    - n_strx:          22
+      n_type:          0xF
+      n_sect:          1
+      n_desc:          0
+      n_value:         4294983572
+  StringTable:
+    - ' '
+    - __mh_execute_header
+    - _main
+    - '/Users/gclayton/Documents/src/args/'
+    - main.cpp
+    - '/Users/gclayton/Documents/src/args/main.o'
+    - _main
+    - ''
+    - ''
+    - ''
+    - ''
+    - ''
+    - ''
+    - ''
+...
+)");
+
+  ASSERT_THAT_EXPECTED(ExpectedFile, llvm::Succeeded());
+  auto module_sp = std::make_shared<Module>(ExpectedFile->moduleSpec());
+  ObjectFile *objfile = module_sp->GetObjectFile();
+  ASSERT_NE(objfile, nullptr);
+
+  // Test encoding and decoding an empty symbol table.
+  Symtab symtab(objfile);
+  symtab.PreloadSymbols();
+  EncodeDecode(symtab);
+
+  // Now encode and decode an actual symbol table from our yaml.
+  Symtab *module_symtab = module_sp->GetSymtab();
+  ASSERT_NE(module_symtab, nullptr);
+  module_symtab->PreloadSymbols();
+  EncodeDecode(*module_symtab);
+}