[lld/mac] Add support for distributed ThinLTO

[thakis]

Summary

Adds support for the following flags:

• --thinlto-index-only, --thinlto-index-only=
• --thinlto-emit-imports-files
• --thinlto-emit-index-files
• --thinlto-object-suffix-replace=
• --thinlto-prefix-replace=

See https://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html
for some words on --thinlto-index-only.

I don't really need the other flags, but they were in the vicinity
and _someone_ might need them, so I figured I'd add them too.

-object_path_lto now sets c.AlwaysEmitRegularLTOObj as in the other ports,
which means it can now only point to a filename for non-thin LTO.
I think that was the intent of D129705 anyways, so update
test/MachO/lto-object-path.ll to use a non-thin bitcode file for that test.

[tschuett]

Distraction: I always thought a local distributed link process is less scary to ninja, because *most* linker processes are single-threaded.

[thakis]

fix test

[thakis]

@MaskRay, would you be comfortable reviewing this? Afaict you did quite a bit of work on the elf version. This should look fairly similar to that.

[thakis]

lld/MachO/LTO.cpp

193

This here got extracted to the outputFilePath lambda, since I need this path in the index-only codepath as well.

(If desired, I could pull the minor refactoring and reordering in this function into its own patch. I figured it's small enough that that's not needed.)

247

This is moving up above the pruneCache call because I need objPathIsDir in the index only path, and the index-only handling happens before cache pruning. Moving this up should be safe.

[thakis]

add thinlto-obj-path.ll; forgot to git add lld/test/MachO/thinlto-obj-path.ll previously

[MaskRay]

I'd like to. I am going to be out of town for one week starting from tomorrow (then another week starting from 12-06) so I am likely slow to getting to this...

[thakis]

No rush, this isn't urgent. Thanks!

[thakis]

Friendly ping, the week of 12/6 is over :)

(But there's still no rush, as long as it isn't forgotten.)

[MaskRay]

(Mostly out of town 2022-12-01 ~ 2022-12-05, and have a long pile of backlog now...)

[MaskRay]

Generally looks good. lld/MachO/CMakeLists.txt should add BitWriter to fix -DBUILD_SHARED_LIBS=on builds.

lld/MachO/LTO.cpp

250

Why does this have /tmp/?

lld/test/MachO/thinlto-index-only.ll

3

if you append cd %t, the %t/ uses below can be removed.

11

not test -e %t/%t/3 this test is unneeded?

26

lld/test/MachO/thinlto-no-index.ll

2

This test can be merged into thinlto-index-only.ll.

I'll remove test/ELF/lto/thinlto-no-index.ll

10

lld/test/MachO/thinlto-prefix-replace.ll

14

[MaskRay]

lld/test/MachO/thinlto-obj-path.ll

6 ↗

(On Diff #477489)

FYI I just improved ELF/lto/thinlto-obj-path.ll and ELF/lto/obj-path.ll a bit.

[thakis]

address comments

[thakis]

Thanks!

lld/MachO/LTO.cpp

250

This just moved up from below. But it's for when you don't pass -object_path_lto, then the .o files go into /tmp/lto.tmp. (See https://reviews.llvm.org/D138451#inline-1337643)

lld/test/MachO/thinlto-index-only.ll

3

I'm not a fan of that style: I often copy failing RUN lines from lit output into my terminal, and I don't want to have to cd there.

11

Sorry, this was meant to say not test -e %t/3. It tests that the lld invocation two lines above doesn't write its output in --thinlto-index-only mode.

lld/test/MachO/thinlto-no-index.ll

2

(That happened in b5ab42af84de – merged into this one too)

[MaskRay]

LGTM. Perhaps another #lld-macho folk wants to take a look as well.

lld/MachO/Driver.cpp

1838

lld/MachO/InputFiles.cpp

2218

You can use structured binding declaration now that C++17 can be used.

lld/test/MachO/thinlto-index-only.ll

3

OK, it's fine. I use a custom script to parse RUN lines and track CWD, so cd %t isn't really a problem for me...
I find %t especially convenient when multiple output files are created. In rare occasions whether an output file exists prior to the test execution matters and ensuring everything is removed makes the test more robust.

[MaskRay]

thinlto (subject)

ThinLTO

Adds support for the following flags:

Better to add double dashes to the following bullet points.

for some words on -thinlto-index-only.

--thinlto-index-only (the patch only allows double-dash forms which match lld/ELF).

[thakis]

tweaks

[thakis]

Thanks! I'll wait another day, but this has been sitting here for a while, so I think folks had plenty time to comment :)

[JOE1994]

FYI:

This introduced build failure (due to link time error : undefined reference to "writeIndexToFile"),
and was later fixed by https://github.com/llvm/llvm-project/commit/87bea593626a103ea4d63ab4a027ec956ccc2b56

[haowei]

We are seeing a linker error on Mac builder today when we try to roll the clang toolchain for Flutter, error message (build task: https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20Unopt/25679/overview):

[3356/6995] LINK ./geometry_benchmarks
[3357/6995] SOLINK libtessellator.dylib libtessellator.dylib.TOC
FAILED: libtessellator.dylib libtessellator.dylib.TOC 
if [ ! -e ./libtessellator.dylib -o ! -e ./libtessellator.dylib.TOC ] || otool -l ./libtessellator.dylib | grep -q LC_REEXPORT_DYLIB ; then ../../buildtools/mac-x64/clang/bin/clang++ -shared -isysroot /Volumes/Work/s/w/ir/cache/osx_sdk/XCode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.0.sdk -mmacosx-version-min=10.14.0  -Wl,-object_path_lto,./lto_libtessellator.o -arch x86_64 -nostdlib++ -Wl,-search_paths_first -L. -Wl,-rpath,@loader_path/. -Wl,-rpath,@loader_path/../../.. -o ./libtessellator.dylib -Wl,-filelist,./libtessellator.dylib.rsp   -framework Foundation && { otool -l ./libtessellator.dylib | grep LC_ID_DYLIB -A 5; nm -gP ./libtessellator.dylib | cut -f1-2 -d' ' | grep -v U$$; true; } > ./libtessellator.dylib.TOC; else ../../buildtools/mac-x64/clang/bin/clang++ -shared -isysroot /Volumes/Work/s/w/ir/cache/osx_sdk/XCode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.0.sdk -mmacosx-version-min=10.14.0  -Wl,-object_path_lto,./lto_libtessellator.o -arch x86_64 -nostdlib++ -Wl,-search_paths_first -L. -Wl,-rpath,@loader_path/. -Wl,-rpath,@loader_path/../../.. -o ./libtessellator.dylib -Wl,-filelist,./libtessellator.dylib.rsp   -framework Foundation && { otool -l ./libtessellator.dylib | grep LC_ID_DYLIB -A 5; nm -gP ./libtessellator.dylib | cut -f1-2 -d' ' | grep -v U$$; true; } > ./libtessellator.dylib.tmp && if ! cmp -s ./libtessellator.dylib.tmp ./libtessellator.dylib.TOC; then mv ./libtessellator.dylib.tmp ./libtessellator.dylib.TOC ; fi; fi
ld64.lld: error: No available targets are compatible with triple ""
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

We did a manual bisecting clang prebuilts and we think this patch is causing the issue.

The bug can be reproduced on a Mac x64 with following steps:

• Create a source file foo.c containing int foo() { return 123; }
• Run clang++ -c -o foo.o foo.c
• Run clang++ -shared -nostdlib -Wl,-object_path_lto,foo.o -o foo.dylib foo.o

With this patch, it will output: ld64.lld: error: No available targets are compatible with triple "". Without this patch, the clang++ will return 0.
Would you mind take a look please?

We are still trying to understand why this patch is causing this issue, maybe it uncovered a bug somewhere else.

[thakis]

In D138451#4151685, @haowei wrote:

We are seeing a linker error on Mac builder today when we try to roll the clang toolchain for Flutter, error message (build task: https://ci.chromium.org/ui/p/flutter/builders/try/Mac%20Unopt/25679/overview):

I was away on leave for a bit. Are you still seeing this? It feels like it should be unrelated (?)

[glandium]

In D138451#4151685, @haowei wrote:

With this patch, it will output: ld64.lld: error: No available targets are compatible with triple "". Without this patch, the clang++ will return 0.
Would you mind take a look please?

We are still trying to understand why this patch is causing this issue, maybe it uncovered a bug somewhere else.

We're seeing the same with Firefox, and this comes from the AlwaysEmitRegularLTOObj change. The problem we have is that we use -object_path_lto to have a deterministic path that keeps the objects around for dsymutil (otherwise, being temporary files, they are removed, although I guess we could use -save-temps instead, but that would remove the determinism), and /some/ things we build end up not using LTO, so the LTO dir ends up empty for them, but with AlwaysEmitRegularLTOObj being true, the error is emitted. I guess there's a case to be made that maybe that error should not be emitted when the directory is empty because there was nothing to LTO.

lld/MachO/Config.h

struct Configuration {
   bool saveTemps = false;
   bool adhocCodesign = false;
   bool emitFunctionStarts = false;
   bool emitBitcodeBundle = false;
   bool emitDataInCodeInfo = false;
   bool emitEncryptionInfo = false;
   bool emitInitOffsets = false;
   bool emitChainedFixups = false;
+  bool thinLTOEmitImportsFiles;
+  bool thinLTOEmitIndexFiles;
+  bool thinLTOIndexOnly;
   bool timeTraceEnabled = false;
   bool dataConst = false;
   bool dedupLiterals = true;
   bool deadStripDuplicates = false;
   bool omitDebugInfo = false;
   bool warnDylibInstallName = false;
   bool ignoreOptimizationHints = false;
   bool forceExactCpuSubtypeMatch = false;
struct Configuration {
   llvm::StringRef installName;
   llvm::StringRef mapFile;
   llvm::StringRef ltoObjPath;
   llvm::StringRef thinLTOJobs;
   llvm::StringRef umbrella;
   uint32_t ltoo = 2;
   llvm::CachePruningPolicy thinLTOCachePolicy;
   llvm::StringRef thinLTOCacheDir;
+  llvm::StringRef thinLTOIndexOnlyArg;
+  std::pair<llvm::StringRef, llvm::StringRef> thinLTOObjectSuffixReplace;
+  std::pair<llvm::StringRef, llvm::StringRef> thinLTOPrefixReplace;
   bool deadStripDylibs = false;
   bool demangle = false;
   bool deadStrip = false;
   bool errorForArchMismatch = false;
   bool ignoreAutoLink = false;
   // ld64 allows invalid auto link options as long as the link succeeds. LLD
   // does not, but there are cases in the wild where the invalid linker options
   // exist. This allows users to ignore the specific invalid options in the case

lld/MachO/Driver.cpp

 }
 
 static const char *getReproduceOption(InputArgList &args) {
   if (const Arg *arg = args.getLastArg(OPT_reproduce))
     return arg->getValue();
   return getenv("LLD_REPRODUCE");
 }
 
+// Parse options of the form "old;new".
+static std::pair<StringRef, StringRef> getOldNewOptions(opt::InputArgList &args,
+                                                        unsigned id) {
+  auto *arg = args.getLastArg(id);
+  if (!arg)
+    return {"", ""};
+
+  StringRef s = arg->getValue();
+  std::pair<StringRef, StringRef> ret = s.split(';');
+  if (ret.second.empty())
+    error(arg->getSpelling() + " expects 'old;new' format, but got " + s);
+  return ret;
+}
+
 static void parseClangOption(StringRef opt, const Twine &msg) {
   std::string err;
   raw_string_ostream os(err);
 
   const char *argv[] = {"lld", opt.data()};
   if (cl::ParseCommandLineOptions(2, argv, "", &os))
     return;
   os.flush();
if (const Arg *arg = args.getLastArg(OPT_umbrella)) {
     config->umbrella = arg->getValue();
   }
   config->ltoObjPath = args.getLastArgValue(OPT_object_path_lto);
   config->ltoo = args::getInteger(args, OPT_lto_O, 2);
   if (config->ltoo > 3)
     error("--lto-O: invalid optimization level: " + Twine(config->ltoo));
   config->thinLTOCacheDir = args.getLastArgValue(OPT_cache_path_lto);
   config->thinLTOCachePolicy = getLTOCachePolicy(args);
+  config->thinLTOEmitImportsFiles = args.hasArg(OPT_thinlto_emit_imports_files);
+  config->thinLTOEmitIndexFiles = args.hasArg(OPT_thinlto_emit_index_files) ||
+                                  args.hasArg(OPT_thinlto_index_only) ||
+                                  args.hasArg(OPT_thinlto_index_only_eq);
+  config->thinLTOIndexOnly = args.hasArg(OPT_thinlto_index_only) ||
+                             args.hasArg(OPT_thinlto_index_only_eq);
+  config->thinLTOIndexOnlyArg = args.getLastArgValue(OPT_thinlto_index_only_eq);
+  config->thinLTOObjectSuffixReplace =
+      getOldNewOptions(args, OPT_thinlto_object_suffix_replace_eq);
+  config->thinLTOPrefixReplace =
+      getOldNewOptions(args, OPT_thinlto_prefix_replace_eq);
+  if (config->thinLTOEmitIndexFiles && !config->thinLTOIndexOnly) {
+    if (args.hasArg(OPT_thinlto_object_suffix_replace_eq))
+      error("--thinlto-object-suffix-replace is not supported with "
+            "--thinlto-emit-index-files");
+    else if (args.hasArg(OPT_thinlto_prefix_replace_eq))
+      error("--thinlto-prefix-replace is not supported with "
+            "--thinlto-emit-index-files");
+  }
   config->runtimePaths = args::getStrings(args, OPT_rpath);
   config->allLoad = args.hasFlag(OPT_all_load, OPT_noall_load, false);
   config->archMultiple = args.hasArg(OPT_arch_multiple);
   config->applicationExtension = args.hasFlag(
       OPT_application_extension, OPT_no_application_extension, false);
   config->exportDynamic = args.hasArg(OPT_export_dynamic);
   config->forceLoadObjC = args.hasArg(OPT_ObjC);
   config->forceLoadSwift = args.hasArg(OPT_force_load_swift_libs);
if (config->timeTraceEnabled)
     createSyntheticSymbols();
     addSynthenticMethnames();
 
     createAliases();
     // If we are in "explicit exports" mode, hide everything that isn't
     // explicitly exported. Do this before running LTO so that LTO can better
     // optimize.
     handleExplicitExports();
+
+    bool didCompileBitcodeFiles = compileBitcodeFiles();
+
+    // If -thinlto-index-only is given, we should create only "index
+    // files" and not object files. Index file creation is already done
+    // in compileBitcodeFiles, so we are done if that's the case.
+    if (config->thinLTOIndexOnly)
+      return errorCount() == 0;
+
     // LTO may emit a non-hidden (extern) object file symbol even if the
     // corresponding bitcode symbol is hidden. In particular, this happens for
     // cross-module references to hidden symbols under ThinLTO. Thus, if we
     // compiled any bitcode files, we must redo the symbol hiding.
-    if (compileBitcodeFiles())
+    if (didCompileBitcodeFiles)
       handleExplicitExports();
     replaceCommonSymbols();
 
     StringRef orderFile = args.getLastArgValue(OPT_order_file);
     if (!orderFile.empty())
       priorityBuilder.parseOrderFile(orderFile);
 
     referenceStubBinder();

lld/MachO/InputFiles.h

 }
 
 template <class CommandType = llvm::MachO::load_command, class... Types>
 std::vector<const CommandType *> findCommands(const void *anyHdr,
                                               Types... types) {
   return detail::findCommands<CommandType>(anyHdr, 0, types...);
 }
 
+std::string replaceThinLTOSuffix(StringRef path);
 } // namespace macho
 
 std::string toString(const macho::InputFile *file);
 std::string toString(const macho::Section &);
 } // namespace lld
 
 #endif

lld/MachO/InputFiles.cpp

return symtab->addDefined(name, &file, /*isec=*/nullptr, /*value=*/0,
                             /*isWeakDefCanBeHidden=*/false);
 }
 
 BitcodeFile::BitcodeFile(MemoryBufferRef mb, StringRef archiveName,
                          uint64_t offsetInArchive, bool lazy, bool forceHidden)
     : InputFile(BitcodeKind, mb, lazy), forceHidden(forceHidden) {
   this->archiveName = std::string(archiveName);
   std::string path = mb.getBufferIdentifier().str();
+  if (config->thinLTOIndexOnly)
+    path = replaceThinLTOSuffix(mb.getBufferIdentifier());
+
   // ThinLTO assumes that all MemoryBufferRefs given to it have a unique
   // name. If two members with the same name are provided, this causes a
   // collision and ThinLTO can't proceed.
   // So, we append the archive name to disambiguate two members with the same
   // name from multiple different archives, and offset within the archive to
   // disambiguate two members of the same name from a single archive.
   MemoryBufferRef mbref(mb.getBuffer(),
                         saver().save(archiveName.empty()
for (const auto &[i, objSym] : llvm::enumerate(obj->symbols())) {
     if (!objSym.isUndefined()) {
       symbols[i] = symtab->addLazyObject(saver().save(objSym.getName()), *this);
       if (!lazy)
         break;
     }
   }
 }
 
+std::string macho::replaceThinLTOSuffix(StringRef path) {
+  StringRef suffix = config->thinLTOObjectSuffixReplace.first;
+  StringRef repl = config->thinLTOObjectSuffixReplace.second;
+
+  if (path.consume_back(suffix))
+    return (path + repl).str();
+  return std::string(path);
+}
+
 void macho::extract(InputFile &file, StringRef reason) {
   assert(file.lazy);
   file.lazy = false;
   printArchiveMemberLoad(reason, &file);
   if (auto *bitcode = dyn_cast<BitcodeFile>(&file)) {
     bitcode->parse();
   } else {
     auto &f = cast<ObjFile>(file);
     if (target->wordSize == 8)
       f.parse<LP64>();
     else
       f.parse<ILP32>();
   }
 }
 
 template void ObjFile::parse<LP64>();

lld/MachO/LTO.h

 //===- LTO.h ----------------------------------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLD_MACHO_LTO_H
 #define LLD_MACHO_LTO_H
 
+#include "lld/Common/LLVM.h"
+#include "llvm/ADT/DenseSet.h"
 #include "llvm/ADT/SmallString.h"
 #include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/raw_ostream.h"
 #include <memory>
 #include <vector>
 
 namespace llvm::lto {
 class LTO;
 } // namespace llvm::lto
 
 namespace lld::macho {
 
 class BitcodeFile;
 class ObjFile;
 
 class BitcodeCompiler {
 public:
   BitcodeCompiler();
 
   void add(BitcodeFile &f);
   std::vector<ObjFile *> compile();
 
 private:
   std::unique_ptr<llvm::lto::LTO> ltoObj;
   std::vector<llvm::SmallString<0>> buf;
   std::vector<std::unique_ptr<llvm::MemoryBuffer>> files;
+  std::unique_ptr<llvm::raw_fd_ostream> indexFile;
+  llvm::DenseSet<StringRef> thinIndices;
 };
 
 } // namespace lld::macho
 
 #endif

lld/MachO/LTO.cpp

 #include "InputFiles.h"
 #include "Symbols.h"
 #include "Target.h"
 
 #include "lld/Common/Args.h"
 #include "lld/Common/CommonLinkerContext.h"
 #include "lld/Common/Strings.h"
 #include "lld/Common/TargetOptionsCommandFlags.h"
+#include "llvm/Bitcode/BitcodeWriter.h"
 #include "llvm/LTO/Config.h"
 #include "llvm/LTO/LTO.h"
 #include "llvm/Support/Caching.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Transforms/ObjCARC.h"
 
 using namespace lld;
 using namespace lld::macho;
 using namespace llvm;
 using namespace llvm::MachO;
 using namespace llvm::sys;
 
+// Creates an empty file to store a list of object files for final
+// linking of distributed ThinLTO.
+static std::unique_ptr<raw_fd_ostream> openFile(StringRef file) {
+  std::error_code ec;
+  auto ret =
+      std::make_unique<raw_fd_ostream>(file, ec, sys::fs::OpenFlags::OF_None);
+  if (ec) {
+    error("cannot open " + file + ": " + ec.message());
+    return nullptr;
+  }
+  return ret;
+}
+
+static std::string getThinLTOOutputFile(StringRef modulePath) {
+  return lto::getThinLTOOutputFile(
+      std::string(modulePath), std::string(config->thinLTOPrefixReplace.first),
+      std::string(config->thinLTOPrefixReplace.second));
+}
+
 static lto::Config createConfig() {
   lto::Config c;
   c.Options = initTargetOptionsFromCodeGenFlags();
   c.Options.EmitAddrsig = config->icfLevel == ICFLevel::safe;
   for (StringRef C : config->mllvmOpts)
     c.MllvmArgs.emplace_back(C.str());
   c.CodeModel = getCodeModelFromCMModel();
   c.CPU = getCPUStr();
   c.MAttrs = getMAttrs();
   c.DiagHandler = diagnosticHandler;
   c.PreCodeGenPassesHook = [](legacy::PassManager &pm) {
     pm.add(createObjCARCContractPass());
   };
+
+  c.AlwaysEmitRegularLTOObj = !config->ltoObjPath.empty();
+
   c.TimeTraceEnabled = config->timeTraceEnabled;
   c.TimeTraceGranularity = config->timeTraceGranularity;
   c.OptLevel = config->ltoo;
   c.CGOptLevel = args::getCGOptLevel(config->ltoo);
   if (config->saveTemps)
     checkError(c.addSaveTemps(config->outputFile.str() + ".",
                               /*UseInputModulePath=*/true));
   return c;
 }
 
 // If `originalPath` exists, hardlinks `path` to `originalPath`. If that fails,
 // or `originalPath` is not set, saves `buffer` to `path`.
 static void saveOrHardlinkBuffer(StringRef buffer, const Twine &path,
                                  Optional<StringRef> originalPath) {
   if (originalPath) {
     auto err = fs::create_hard_link(*originalPath, path);
     if (!err)
       return;
   }
   saveBuffer(buffer, path);
 }
 
 BitcodeCompiler::BitcodeCompiler() {
-  lto::ThinBackend backend = lto::createInProcessThinBackend(
-      heavyweight_hardware_concurrency(config->thinLTOJobs));
+  // Initialize indexFile.
+  if (!config->thinLTOIndexOnlyArg.empty())
+    indexFile = openFile(config->thinLTOIndexOnlyArg);
+
+  // Initialize ltoObj.
+  lto::ThinBackend backend;
+  auto onIndexWrite = [&](StringRef S) { thinIndices.erase(S); };
+  if (config->thinLTOIndexOnly) {
+    backend = lto::createWriteIndexesThinBackend(
+        std::string(config->thinLTOPrefixReplace.first),
+        std::string(config->thinLTOPrefixReplace.second),
+        config->thinLTOEmitImportsFiles, indexFile.get(), onIndexWrite);
+  } else {
+    backend = lto::createInProcessThinBackend(
+        llvm::heavyweight_hardware_concurrency(config->thinLTOJobs),
+        onIndexWrite, config->thinLTOEmitIndexFiles,
+        config->thinLTOEmitImportsFiles);
+  }
+
   ltoObj = std::make_unique<lto::LTO>(createConfig(), backend);
 }
 
 void BitcodeCompiler::add(BitcodeFile &f) {
-  ArrayRef<lto::InputFile::Symbol> objSyms = f.obj->symbols();
+  lto::InputFile &obj = *f.obj;
+
+  if (config->thinLTOEmitIndexFiles)
+    thinIndices.insert(obj.getName());
+
+  ArrayRef<lto::InputFile::Symbol> objSyms = obj.symbols();
   std::vector<lto::SymbolResolution> resols;
   resols.reserve(objSyms.size());
 
   // Provide a resolution to the LTO API for each symbol.
   bool exportDynamic =
       config->outputType != MH_EXECUTE || config->exportDynamic;
   auto symIt = f.symbols.begin();
   for (const lto::InputFile::Symbol &objSym : objSyms) {
if (r.Prevailing)
       replaceSymbol<Undefined>(sym, sym->getName(), sym->getFile(),
                                RefState::Strong, /*wasBitcodeSymbol=*/true);
 
     // TODO: set the other resolution configs properly
   }
   checkError(ltoObj->add(std::move(f.obj), resols));
 }
 
+// If LazyObjFile has not been added to link, emit empty index files.
+// This is needed because this is what GNU gold plugin does and we have a
+// distributed build system that depends on that behavior.
+static void thinLTOCreateEmptyIndexFiles() {
+  DenseSet<StringRef> linkedBitCodeFiles;
+  for (InputFile *file : inputFiles)
+    if (auto *f = dyn_cast<BitcodeFile>(file))
+      if (!f->lazy)
+        linkedBitCodeFiles.insert(f->getName());
+
+  for (InputFile *file : inputFiles) {
+    if (auto *f = dyn_cast<BitcodeFile>(file)) {
+      if (!f->lazy)
+        continue;
+      if (linkedBitCodeFiles.contains(f->getName()))
+        continue;
+      std::string path =
+          replaceThinLTOSuffix(getThinLTOOutputFile(f->obj->getName()));
+      std::unique_ptr<raw_fd_ostream> os = openFile(path + ".thinlto.bc");
+      if (!os)
+        continue;
+
+      ModuleSummaryIndex m(/*HaveGVs=*/false);
+      m.setSkipModuleByDistributedBackend();
+      writeIndexToFile(m, *os);
+      if (config->thinLTOEmitImportsFiles)
+        openFile(path + ".imports");
+    }
+  }
+}
+
 // Merge all the bitcode files we have seen, codegen the result
 // and return the resulting ObjectFile(s).
 std::vector<ObjFile *> BitcodeCompiler::compile() {
   unsigned maxTasks = ltoObj->getMaxTasks();
   buf.resize(maxTasks);
   files.resize(maxTasks);
 
   // The -cache_path_lto option specifies the path to a directory in which
std::vector<ObjFile *> BitcodeCompiler::compile() {
 
   checkError(ltoObj->run(
       [&](size_t task) {
         return std::make_unique<CachedFileStream>(
             std::make_unique<raw_svector_ostream>(buf[task]));
       },
       cache));
 
-  if (!config->thinLTOCacheDir.empty())
-    pruneCache(config->thinLTOCacheDir, config->thinLTOCachePolicy, files);
+  // Emit empty index files for non-indexed files
+  for (StringRef s : thinIndices) {
+    std::string path = getThinLTOOutputFile(s);
+    openFile(path + ".thinlto.bc");
+    if (config->thinLTOEmitImportsFiles)
+      openFile(path + ".imports");
+  }
+
+  if (config->thinLTOEmitIndexFiles)
+    thinLTOCreateEmptyIndexFiles();
 
   // In ThinLTO mode, Clang passes a temporary directory in -object_path_lto,
   // while the argument is a single file in FullLTO mode.
   bool objPathIsDir = true;
   if (!config->ltoObjPath.empty()) {
     if (std::error_code ec = fs::create_directories(config->ltoObjPath))
       fatal("cannot create LTO object path " + config->ltoObjPath + ": " +
             ec.message());
 
     if (!fs::is_directory(config->ltoObjPath)) {
       objPathIsDir = false;
       unsigned objCount =
           count_if(buf, [](const SmallString<0> &b) { return !b.empty(); });
       if (objCount > 1)
         fatal("-object_path_lto must specify a directory when using ThinLTO");
     }
   }
 
+  auto outputFilePath = [objPathIsDir](int i) {
+    SmallString<261> filePath("/tmp/lto.tmp");
+    if (!config->ltoObjPath.empty()) {
+      filePath = config->ltoObjPath;
+      if (objPathIsDir)
+        path::append(filePath, Twine(i) + "." +
+                                   getArchitectureName(config->arch()) +
+                                   ".lto.o");
+    }
+    return filePath;
+  };
+
+  // ThinLTO with index only option is required to generate only the index
+  // files. After that, we exit from linker and ThinLTO backend runs in a
+  // distributed environment.
+  if (config->thinLTOIndexOnly) {
+    if (!config->ltoObjPath.empty())
+      saveBuffer(buf[0], outputFilePath(0));
+    if (indexFile)
+      indexFile->close();
+    return {};
+  }
+
+  if (!config->thinLTOCacheDir.empty())
+    pruneCache(config->thinLTOCacheDir, config->thinLTOCachePolicy, files);
+
   std::vector<ObjFile *> ret;
   for (unsigned i = 0; i < maxTasks; ++i) {
     // Get the native object contents either from the cache or from memory.  Do
     // not use the cached MemoryBuffer directly to ensure dsymutil does not
     // race with the cache pruner.
     StringRef objBuf;
     Optional<StringRef> cachePath = llvm::None;
     if (files[i]) {
       objBuf = files[i]->getBuffer();
       cachePath = files[i]->getBufferIdentifier();
     } else {
       objBuf = buf[i];
     }
     if (objBuf.empty())
       continue;
 
     // FIXME: should `saveTemps` and `ltoObjPath` use the same file name?
     if (config->saveTemps)
       saveBuffer(objBuf,
                  config->outputFile + ((i == 0) ? "" : Twine(i)) + ".lto.o");
 
-    SmallString<261> filePath("/tmp/lto.tmp");
+    auto filePath = outputFilePath(i);
     uint32_t modTime = 0;
     if (!config->ltoObjPath.empty()) {
-      filePath = config->ltoObjPath;
-      if (objPathIsDir)
-        path::append(filePath, Twine(i) + "." +
-                                   getArchitectureName(config->arch()) +
-                                   ".lto.o");
       saveOrHardlinkBuffer(objBuf, filePath, cachePath);
       modTime = getModTime(filePath);
     }
     ret.push_back(make<ObjFile>(
         MemoryBufferRef(objBuf, saver().save(filePath.str())), modTime, ""));
   }
 
   return ret;
 }

lld/MachO/Options.td

def no_color_diagnostics: Flag<["--"], "no-color-diagnostics">,
     Group<grp_lld>;
 def color_diagnostics_eq: Joined<["--"], "color-diagnostics=">,
     HelpText<"Use colors in diagnostics (default: auto)">,
     MetaVarName<"[auto,always,never]">,
     Group<grp_lld>;
 def threads_eq : Joined<["--"], "threads=">,
     HelpText<"Number of threads. '1' disables multi-threading. By default all available hardware threads are used">,
     Group<grp_lld>;
+def thinlto_emit_imports_files: Flag<["--"], "thinlto-emit-imports-files">,
+    Group<grp_lld>;
+def thinlto_emit_index_files: Flag<["--"], "thinlto-emit-index-files">,
+    Group<grp_lld>;
+def thinlto_index_only: Flag<["--"], "thinlto-index-only">,
+    Group<grp_lld>;
+def thinlto_index_only_eq: Joined<["--"], "thinlto-index-only=">,
+    Group<grp_lld>;
 def thinlto_jobs_eq : Joined<["--"], "thinlto-jobs=">,
     HelpText<"Number of ThinLTO jobs. Default to --threads=">,
     Group<grp_lld>;
+def thinlto_object_suffix_replace_eq:
+    Joined<["--"], "thinlto-object-suffix-replace=">,
+    Group<grp_lld>;
+def thinlto_prefix_replace_eq: Joined<["--"], "thinlto-prefix-replace=">,
+    Group<grp_lld>;
 def reproduce: Separate<["--"], "reproduce">,
     Group<grp_lld>;
 def reproduce_eq: Joined<["--"], "reproduce=">,
     Alias<!cast<Separate>(reproduce)>,
     HelpText<"Write tar file containing inputs and command to reproduce link">,
     Group<grp_lld>;
 def version: Flag<["--"], "version">,
     HelpText<"Display the version number and exit">,

lld/test/ELF/lto/thinlto-obj-path.ll

 
 ;; Ensure lld emits empty combined module if specific obj-path.
 ; RUN: rm -fr %t.dir/objpath && mkdir -p %t.dir/objpath
 ; RUN: ld.lld --plugin-opt=obj-path=%t4.o -shared %t1.o %t2.o -o %t.dir/objpath/a.out --save-temps
 ; RUN: ls %t.dir/objpath/a.out*.lto.* | count 3
 
 ;; Ensure lld does not emit empty combined module in default.
 ; RUN: rm -fr %t.dir/objpath && mkdir -p %t.dir/objpath
-; RUN: ld.lld %t1.o %t2.o -o %t.dir/objpath/a.out --save-temps
+; RUN: ld.lld -shared %t1.o %t2.o -o %t.dir/objpath/a.out --save-temps
 ; RUN: ls %t.dir/objpath/a.out*.lto.* | count 2
 
 ; CHECK: Format: elf64-x86-64
 
 target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 declare void @g(...)
 
 define void @f() {
 entry:
   call void (...) @g()
   ret void
 }

lld/test/MachO/lto-object-path.ll

 ; REQUIRES: x86
 ; UNSUPPORTED: system-windows
 
 ; RUN: rm -rf %t; mkdir %t
 ; RUN: opt -thinlto-bc %s -o %t/test.o
+; RUN: opt %s -o %t/test-nonthin.o
 
 ; RUN: %lld %t/test.o -o %t/test
 ; RUN: llvm-nm -pa %t/test | FileCheck %s --check-prefixes CHECK,NOOBJPATH
 
 ; RUN: ZERO_AR_DATE=0 %lld %t/test.o -o %t/test -object_path_lto %t/lto-temps
 ; RUN: llvm-nm -pa %t/test | FileCheck %s --check-prefixes CHECK,OBJPATH-DIR -DDIR=%t/lto-temps
 
 ;; Ensure object path is still used if the cache is used

 ;;    to ensure that the cache is pruned immediately; we want the opposite here).
 ; RUN: rm -rf %t/lto-temps
 ; RUN: %lld %t/test.o -o %t/test -object_path_lto %t/lto-temps -prune_interval_lto 20 -cache_path_lto %t/cache --thinlto-cache-policy=cache_size_files=10:cache_size_bytes=10000
 ;; Get the link count.
 ; RUN: ls -l %t/lto-temps/1.x86_64.lto.o | tr -s ' ' | cut -f 2 -d ' ' | FileCheck %s --check-prefix=HARDLINK
 
 ;; check that the object path can be an existing file
 ; RUN: touch %t/lto-tmp.o
-; RUN: ZERO_AR_DATE=0 %lld %t/test.o -o %t/test -object_path_lto %t/lto-tmp.o
+; RUN: ZERO_AR_DATE=0 %lld %t/test-nonthin.o -o %t/test -object_path_lto %t/lto-tmp.o
 ; RUN: llvm-nm -pa %t/test | FileCheck %s --check-prefixes CHECK,OBJPATH-FILE -DFILE=%t/lto-tmp.o
 
 
 ; CHECK:             0000000000000000                - 00 0000    SO /tmp/test.cpp
 ; NOOBJPATH-NEXT:    0000000000000000                - 03 0001   OSO /tmp/lto.tmp
 ;; check that modTime is nonzero when `-object_path_lto` is provided
 ; OBJPATH-DIR-NEXT:  {{[0-9a-f]*[1-9a-f]+[0-9a-f]*}} - 03 0001   OSO [[DIR]]/1.x86_64.lto.o
 ; OBJPATH-FILE-NEXT: {{[0-9a-f]*[1-9a-f]+[0-9a-f]*}} - 03 0001   OSO [[FILE]]

lld/test/MachO/thinlto-emit-imports.ll

+; REQUIRES: x86
+; RUN: rm -rf %t; split-file %s %t
+
+; Generate summary sections and test lld handling.
+; RUN: opt -module-summary %t/f.ll -o %t1.o
+; RUN: opt -module-summary %t/g.ll -o %t2.o
+
+; Include a file with an empty module summary index, to ensure that the expected
+; output files are created regardless, for a distributed build system.
+; RUN: opt -module-summary %t/empty.ll -o %t3.o
+
+; Ensure lld generates imports files if requested for distributed backends.
+; RUN: rm -f %t3.o.imports %t3.o.thinlto.bc
+; RUN: %lld --thinlto-index-only --thinlto-emit-imports-files -dylib %t1.o %t2.o %t3.o -o %t4
+
+; The imports file for this module contains the bitcode file for %t/g.ll
+; RUN: count 1 < %t1.o.imports
+; RUN: FileCheck %s --check-prefix=IMPORTS1 < %t1.o.imports
+; IMPORTS1: thinlto-emit-imports.ll.tmp2.o
+
+; The imports file for g.ll is empty as it does not import anything.
+; RUN: count 0 < %t2.o.imports
+
+; The imports file for empty.ll is empty but should exist.
+; RUN: count 0 < %t3.o.imports
+
+; The index file should be created even for the input with an empty summary.
+; RUN: ls %t3.o.thinlto.bc
+
+; Ensure lld generates error if unable to write to imports file.
+; RUN: rm -f %t3.o.imports
+; RUN: touch %t3.o.imports
+; RUN: chmod 400 %t3.o.imports
+; RUN: not %lld --thinlto-index-only --thinlto-emit-imports-files -dylib %t1.o %t2.o %t3.o -o /dev/null 2>&1 \
+; RUN:     | FileCheck -DMSG=%errc_EACCES %s --check-prefix=ERR
+; ERR: cannot open {{.*}}3.o.imports: [[MSG]]
+
+; Ensure lld doesn't generate import files when thinlto-index-only is not enabled
+; RUN: rm -f %t1.o.imports
+; RUN: rm -f %t2.o.imports
+; RUN: rm -f %t3.o.imports
+; RUN: %lld --thinlto-emit-imports-files -dylib %t1.o %t2.o %t3.o -o %t4
+; RUN: not ls %t1.o.imports
+; RUN: not ls %t2.o.imports
+; RUN: not ls %t3.o.imports
+
+;--- f.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+declare void @g(...)
+
+define void @f() {
+entry:
+  call void (...) @g()
+  ret void
+}
+
+;--- g.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @g() {
+entry:
+  ret void
+}
+
+;--- empty.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"

lld/test/MachO/thinlto-emit-index.ll

+; REQUIRES: x86
+; RUN: rm -rf %t; split-file %s %t
+
+;; Mostly copied/updated from thinlto-index-only.ll
+;; First ensure that the ThinLTO handling in lld handles
+;; bitcode without summary sections gracefully and generates index file.
+; RUN: llvm-as %t/f.ll -o %t/1.o
+; RUN: llvm-as %t/g.ll -o %t/2.o
+; RUN: %lld --thinlto-emit-index-files -dylib %t/1.o %t/2.o -o %t/3
+; RUN: ls %t/2.o.thinlto.bc
+; RUN: ls %t/3
+; RUN: %lld -dylib %t/1.o %t/2.o -o %t/3
+; RUN: llvm-nm %t/3 | FileCheck %s --check-prefix=NM
+
+;; Basic ThinLTO tests.
+; RUN: opt -module-summary %t/f.ll -o %t/1.o
+; RUN: opt -module-summary %t/g.ll -o %t/2.o
+; RUN: opt -module-summary %t/empty.ll -o %t/3.o
+
+;; Ensure lld generates an index and also a binary if requested.
+; RUN: %lld --thinlto-emit-index-files -dylib %t/1.o %t/2.o -o %t/4
+; RUN: llvm-bcanalyzer -dump %t/1.o.thinlto.bc | FileCheck %s -DP1=%t/1.o -DP2=%t/2.o --check-prefix=BACKEND1
+; RUN: llvm-bcanalyzer -dump %t/2.o.thinlto.bc | FileCheck %s -DP2=%t/2.o --check-prefix=BACKEND2
+; RUN: ls %t/4
+
+;; Ensure lld generates an index and not a binary if both emit-index and index-only are present.
+; RUN: %lld --thinlto-emit-index-files --thinlto-index-only -dylib %t/1.o %t/2.o -o %t/5
+; RUN: not ls %t/5
+
+;; Ensure lld generates an index even if the file is wrapped in --start-lib/--end-lib
+; RUN: rm -f %t/2.o.thinlto.bc
+; RUN: %lld --thinlto-emit-index-files -dylib %t/1.o %t/3.o --start-lib %t/2.o --end-lib -o %t/6
+; RUN: llvm-dis < %t/2.o.thinlto.bc | grep -q '\^0 = module:'
+; RUN: ls %t/6
+
+;; Test that LLD generates an empty index even for lazy object file that is not added to link.
+;; Test that LLD also generates empty imports file with the --thinlto-emit-imports-files option.
+; RUN: rm -f %t/1.o.thinlto.bc %t/1.o.imports
+; RUN: %lld --thinlto-emit-index-files -dylib %t/2.o --start-lib %t/1.o --end-lib \
+; RUN: --thinlto-emit-imports-files -o %t/7
+; RUN: ls %t/7
+; RUN: ls %t/1.o.thinlto.bc
+; RUN: ls %t/1.o.imports
+
+;; Ensure LLD generates an empty index for each bitcode file even if all bitcode files are lazy.
+; RUN: rm -f %t/1.o.thinlto.bc
+; RUN: llvm-mc -filetype=obj -triple=x86_64-apple-darwin /dev/null -o dummy.o
+; RUN: %lld --thinlto-emit-index-files -dylib dummy.o --start-lib %t/1.o --end-lib -o %t/8
+; RUN: ls %t/8
+; RUN: ls %t/1.o.thinlto.bc
+
+;; Test that LLD errors out when run with suffix replacement, or prefix replacement
+; RUN: not %lld --thinlto-emit-index-files -dylib %t/2.o --start-lib %t/1.o --end-lib \
+; RUN: --thinlto-prefix-replace="abc;xyz" 2>&1 | FileCheck %s --check-prefix=ERR1
+; ERR1: --thinlto-prefix-replace is not supported with --thinlto-emit-index-files
+
+; RUN: not %lld --thinlto-emit-index-files -dylib %t/2.o --start-lib %t/1.o --end-lib \
+; RUN: --thinlto-object-suffix-replace="abc;xyz" 2>&1 | FileCheck %s --check-prefix=ERR2
+; ERR2: --thinlto-object-suffix-replace is not supported with --thinlto-emit-index-files
+
+;; But not when passed with index only as well
+; RUN: %lld --thinlto-emit-index-files -dylib %t/2.o --start-lib %t/1.o --end-lib \
+; RUN: --thinlto-prefix-replace="abc;xyz" --thinlto-index-only
+
+; RUN: %lld --thinlto-emit-index-files -dylib %t/2.o --start-lib %t/1.o --end-lib \
+; RUN: --thinlto-object-suffix-replace="abc;xyz" --thinlto-index-only
+
+; NM: T _f
+
+;; The backend index for this module contains summaries from itself and
+;; Inputs/thinlto.ll, as it imports from the latter.
+; BACKEND1: <MODULE_STRTAB_BLOCK
+; BACKEND1-NEXT: <ENTRY {{.*}} record string = '[[P1]]'
+; BACKEND1-NEXT: <ENTRY {{.*}} record string = '[[P2]]'
+; BACKEND1-NEXT: </MODULE_STRTAB_BLOCK
+; BACKEND1: <GLOBALVAL_SUMMARY_BLOCK
+; BACKEND1: <VERSION
+; BACKEND1: <FLAGS
+; BACKEND1: <VALUE_GUID op0={{1|2}} op1={{-3706093650706652785|-5300342847281564238}}
+; BACKEND1: <VALUE_GUID op0={{1|2}} op1={{-3706093650706652785|-5300342847281564238}}
+; BACKEND1: <COMBINED
+; BACKEND1: <COMBINED
+; BACKEND1: </GLOBALVAL_SUMMARY_BLOCK
+
+;; The backend index for Input/thinlto.ll contains summaries from itself only,
+;; as it does not import anything.
+; BACKEND2: <MODULE_STRTAB_BLOCK
+; BACKEND2-NEXT: <ENTRY {{.*}} record string = '[[P2]]'
+; BACKEND2-NEXT: </MODULE_STRTAB_BLOCK
+; BACKEND2-NEXT: <GLOBALVAL_SUMMARY_BLOCK
+; BACKEND2-NEXT: <VERSION
+; BACKEND2-NEXT: <FLAGS
+; BACKEND2-NEXT: <VALUE_GUID op0=1 op1=-5300342847281564238
+; BACKEND2-NEXT: <COMBINED
+; BACKEND2-NEXT: <BLOCK_COUNT op0=2/>
+; BACKEND2-NEXT: </GLOBALVAL_SUMMARY_BLOCK
+
+
+;--- f.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+declare void @g(...)
+
+define void @f() {
+entry:
+  call void (...) @g()
+  ret void
+}
+
+;--- g.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @g() {
+entry:
+  ret void
+}
+
+;--- empty.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"

lld/test/MachO/thinlto-index-file.ll

+; REQUIRES: x86
+; RUN: rm -rf %t; split-file %s %t
+
+; RUN: opt -module-summary %t/f.ll -o %t/1.o
+; RUN: opt -module-summary %t/g.ll -o %t/2.o
+; RUN: opt -module-summary %t/empty.ll -o %t/3.o
+
+;; Ensure lld writes linked files to linked objects file
+; RUN: %lld --thinlto-index-only=%t/1.txt -dylib %t/1.o %t/2.o %t/3.o -o /dev/null
+; RUN: FileCheck %s < %t/1.txt
+; CHECK: 1.o
+; CHECK: 2.o
+; CHECK: 3.o
+
+;--- f.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+declare void @g(...)
+
+define void @f() {
+entry:
+  call void (...) @g()
+  ret void
+}
+
+;--- g.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @g() {
+entry:
+  ret void
+}
+
+;--- empty.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"

lld/test/MachO/thinlto-index-only.ll

+; REQUIRES: x86
+; RUN: rm -rf %t; split-file %s %t
+
+;; First ensure that the ThinLTO handling in lld handles
+;; bitcode without summary sections gracefully and generates index file.
+; RUN: llvm-as %t/f.ll -o %t/1.o
+; RUN: llvm-as %t/g.ll -o %t/2.o
+; RUN: %lld --thinlto-index-only -dylib %t/1.o %t/2.o -o %t/3
+; RUN: ls %t/2.o.thinlto.bc
+; RUN: not test -e %t/%t/3
+; RUN: %lld -dylib %t/1.o %t/2.o -o %t/3
+; RUN: llvm-nm %t/3 | FileCheck %s --check-prefix=NM
+
+;; Basic ThinLTO tests.
+; RUN: opt -module-summary %t/f.ll -o %t/1.o
+; RUN: opt -module-summary %t/g.ll -o %t/2.o
+; RUN: opt -module-summary %t/empty.ll -o %t/3.o
+
+;; Ensure lld generates an index and not a binary if requested.
+; RUN: %lld --thinlto-index-only -dylib %t/1.o %t/2.o -o %t/4
+; RUN: llvm-bcanalyzer -dump %t/1.o.thinlto.bc | FileCheck %s -DP1=%t/1.o -DP2=%t/2.o --check-prefix=BACKEND1
+; RUN: llvm-bcanalyzer -dump %t/2.o.thinlto.bc | FileCheck %s -DP2=%t/2.o --check-prefix=BACKEND2
+; RUN: not test -e %t/4
+
+;; Ensure lld generates an index even if the file is wrapped in --start-lib/--end-lib
+; RUN: rm -f %t/2.o.thinlto.bc %t/4
+; RUN: %lld --thinlto-index-only -dylib %t/1.o %t/3.o --start-lib %t/2.o --end-lib -o %t/4
+; RUN: llvm-dis < %t/2.o.thinlto.bc | grep -q '\^0 = module:'
+; RUN: not test -e %t/4
+
+;; Test that LLD generates an empty index even for lazy object file that is not added to link.
+;; Test LLD generates empty imports file either because of thinlto-emit-imports-files option.
+; RUN: rm -f %t/1.o.thinlto.bc %t/1.o.imports
+; RUN: %lld --thinlto-index-only -dylib %t/2.o --start-lib %t/1.o --end-lib \
+; RUN:      --thinlto-emit-imports-files -o %t/3
+; RUN: ls %t/1.o.thinlto.bc
+; RUN: ls %t/1.o.imports
+
+;; Ensure LLD generates an empty index for each bitcode file even if all bitcode files are lazy.
+; RUN: rm -f %t/1.o.thinlto.bc
+; RUN: llvm-mc -filetype=obj -triple=x86_64-apple-darwin /dev/null -o dummy.o
+; RUN: %lld --thinlto-index-only -dylib dummy.o --start-lib %t/1.o --end-lib -o /dev/null
+; RUN: ls %t/1.o.thinlto.bc
+
+;; Ensure when the same bitcode object is given as both lazy and non-lazy,
+;; LLD does not generate an empty index for the lazy object.
+; RUN: rm -f %t/2.o.thinlto.bc
+; RUN: %lld --thinlto-index-only -dylib %t/1.o %t/2.o --start-lib %t/2.o --end-lib -o /dev/null
+; RUN: llvm-dis < %t/2.o.thinlto.bc | grep -q '\^0 = module:'
+; RUN: rm -f %t/2.o.thinlto.bc
+; RUN: %lld --thinlto-index-only -dylib --start-lib %t/2.o --end-lib %t/2.o %t/1.o -o /dev/null
+; RUN: llvm-dis < %t/2.o.thinlto.bc | grep -q '\^0 = module:'
+
+;; Ensure when the same lazy bitcode object is given multiple times,
+;; no empty index file is generated if one of the copies is linked.
+; RUN: rm -f %t/2.o.thinlto.bc
+; RUN: %lld --thinlto-index-only -dylib %t/1.o --start-lib %t/2.o --end-lib --start-lib %t/2.o --end-lib -o /dev/null
+; RUN: llvm-dis < %t/2.o.thinlto.bc | grep -q '\^0 = module:'
+
+; NM: T _f
+
+;; The backend index for this module contains summaries from itself and
+;; g.ll, as it imports from the latter.
+; BACKEND1: <MODULE_STRTAB_BLOCK
+; BACKEND1-NEXT: <ENTRY {{.*}} record string = '[[P1]]'
+; BACKEND1-NEXT: <ENTRY {{.*}} record string = '[[P2]]'
+; BACKEND1-NEXT: </MODULE_STRTAB_BLOCK
+; BACKEND1: <GLOBALVAL_SUMMARY_BLOCK
+; BACKEND1: <VERSION
+; BACKEND1: <FLAGS
+; BACKEND1: <VALUE_GUID op0={{1|2}} op1={{-3706093650706652785|-5300342847281564238}}
+; BACKEND1: <VALUE_GUID op0={{1|2}} op1={{-3706093650706652785|-5300342847281564238}}
+; BACKEND1: <COMBINED
+; BACKEND1: <COMBINED
+; BACKEND1: </GLOBALVAL_SUMMARY_BLOCK
+
+;; The backend index for g.ll contains summaries from itself only,
+;; as it does not import anything.
+; BACKEND2: <MODULE_STRTAB_BLOCK
+; BACKEND2-NEXT: <ENTRY {{.*}} record string = '[[P2]]'
+; BACKEND2-NEXT: </MODULE_STRTAB_BLOCK
+; BACKEND2-NEXT: <GLOBALVAL_SUMMARY_BLOCK
+; BACKEND2-NEXT: <VERSION
+; BACKEND2-NEXT: <FLAGS
+; BACKEND2-NEXT: <VALUE_GUID op0=1 op1=-5300342847281564238
+; BACKEND2-NEXT: <COMBINED
+; BACKEND2-NEXT: <BLOCK_COUNT op0=2/>
+; BACKEND2-NEXT: </GLOBALVAL_SUMMARY_BLOCK
+
+;--- f.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+declare void @g(...)
+
+define void @f() {
+entry:
+  call void (...) @g()
+  ret void
+}
+
+;--- g.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @g() {
+entry:
+  ret void
+}
+
+;--- empty.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"

lld/test/MachO/thinlto-no-index.ll

+; REQUIRES: x86
+; RUN: rm -rf %t; split-file %s %t
+
+; Basic ThinLTO tests.
+; RUN: opt -module-summary %t/f.ll -o %t1.o
+; RUN: opt -module-summary %t/g.ll -o %t2.o
+; RUN: opt -module-summary %t/empty.ll -o %t3.o
+
+; Ensure lld doesn't generates index files when thinlto-index-only is not enabled
+; RUN: rm -f %t1.o.thinlto.bc %t2.o.thinlto.bc %t3.o.thinlto.bc
+; RUN: %lld -dylib %t1.o %t2.o %t3.o -o %t4
+; RUN: not ls %t1.o.thinlto.bc
+; RUN: not ls %t2.o.thinlto.bc
+; RUN: not ls %t3.o.thinlto.bc
+
+;--- f.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+declare void @g(...)
+
+define void @f() {
+entry:
+  call void (...) @g()
+  ret void
+}
+
+;--- g.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @g() {
+entry:
+  ret void
+}
+
+;--- empty.ll
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"

lld/test/MachO/thinlto-object-suffix-replace.ll

+; REQUIRES: x86
+;; Test to make sure the thinlto-object-suffix-replace option is handled
+;; correctly.
+; RUN: rm -rf %t && mkdir %t && cd %t
+
+;; Generate bitcode file with summary, as well as a minimized bitcode without
+; the debug metadata for the thin link.
+; RUN: opt --thinlto-bc %s -thin-link-bitcode-file=1.thinlink.bc -o 1.o
+
+;; First perform the thin link on the normal bitcode file, and save the
+;; resulting index.
+; RUN: %lld --thinlto-index-only -dylib 1.o -o 3
+; RUN: cp 1.o.thinlto.bc 1.o.thinlto.bc.orig
+
+;; Next perform the thin link on the minimized bitcode file, and compare dump
+;; of the resulting index to the above dump to ensure they are identical.
+; RUN: rm -f 1.o.thinlto.bc
+;; Make sure it isn't inadvertently using the regular bitcode file.
+; RUN: rm -f 1.o
+; RUN: %lld --thinlto-index-only --thinlto-object-suffix-replace=".thinlink.bc;.o" \
+; RUN:   -dylib 1.thinlink.bc -o 3
+; RUN: cmp 1.o.thinlto.bc.orig 1.o.thinlto.bc
+
+;; Ensure lld generates error if object suffix replace option does not have 'old;new' format
+; RUN: rm -f 1.o.thinlto.bc
+; RUN: not %lld --thinlto-index-only --thinlto-object-suffix-replace="abc:def" -dylib 1.thinlink.bc \
+; RUN:   -o 3 2>&1 | FileCheck %s --check-prefix=ERR1
+; ERR1: --thinlto-object-suffix-replace= expects 'old;new' format, but got abc:def
+
+;; If filename does not end with old suffix, no suffix change should occur,
+;; so ".thinlto.bc" will simply be appended to the input file name.
+; RUN: rm -f 1.thinlink.bc.thinlto.bc
+; RUN: %lld --thinlto-index-only --thinlto-object-suffix-replace=".abc;.o" -dylib 1.thinlink.bc -o /dev/null
+; RUN: ls 1.thinlink.bc.thinlto.bc
+
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @f() {
+entry:
+  ret void
+}
+
+!llvm.dbg.cu = !{}
+
+!1 = !{i32 2, !"Debug Info Version", i32 3}
+!llvm.module.flags = !{!1}

lld/test/MachO/thinlto-prefix-replace.ll

+; REQUIRES: x86
+
+;; Check that changing the output path via thinlto-prefix-replace works
+; RUN: mkdir -p %t/oldpath
+; RUN: opt -module-summary %s -o %t/oldpath/thinlto_prefix_replace.o
+
+;; Ensure that there is no existing file at the new path, so we properly
+;; test the creation of the new file there.
+; RUN: rm -f %t/newpath/thinlto_prefix_replace.o.thinlto.bc
+; RUN: %lld --thinlto-index-only --thinlto-prefix-replace="%t/oldpath/;%t/newpath/" -dylib %t/oldpath/thinlto_prefix_replace.o -o %t/thinlto_prefix_replace
+; RUN: ls %t/newpath/thinlto_prefix_replace.o.thinlto.bc
+
+;; Ensure that lld generates error if prefix replace option does not have 'old;new' format
+; RUN: rm -f %t/newpath/thinlto_prefix_replace.o.thinlto.bc
+; RUN: not %lld --thinlto-index-only --thinlto-prefix-replace=abc:def -dylib %t/oldpath/thinlto_prefix_replace.o -o /dev/null 2>&1 | FileCheck %s --check-prefix=ERR
+; ERR: --thinlto-prefix-replace= expects 'old;new' format, but got abc:def
+
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @f() {
+entry:
+  ret void
+}