[lld] Support separate minimized bitcode file path in --thinlto-prefix-replace

[itf]

Summary

Currently, the --thinlto-suffix-replace="oldsuffix;newsuffix" option
is used during distributed ThinLTO thin links to specify the mapping
of the input bitcode object file to the minimized bitcode file.
--thinlto-suffix-replace is the only option specifying the mapping
between those files.

This forces the full bitcode file and the minimized bitcode file to be in
the same directory.

This patch expands the --thinlto-prefix-replace option to allow a
separate directory tree mapping to be specified for the minimized
bitcode file paths.

This is important to support builds and build systems where a directory
that is the output of a single action must either be fully included or
not included as the input of another action (e.g. the thin link, it must take as its input
the minimized bitcode files but not take as input the full bitcode files).

The new format is: --thinlto-prefix-replace="origpath;outpath[;objpath][;minimizedbitcodepath]"

If minimizedbitcodepath is specified it replaces minimizedbitcodepath
of the minimized bitcode files with origpath when mapping the minimized
bitcode to the full bitcode, otherwise it falls back to the old behavior of
assuming both files are in the same directory. The rewritten path is written
into the index and imports files for the subsequent distributed ThinLTO backends.

With this change instead of mapping the minimized bitcode to the full
bitcode using a suffix replace, we can instead use a prefix replace.

[itf]

This is a continuation of D144596 , based on D144596#4182628.

[tejohnson]

Thanks!

If minimizedbitcodepath is specified it replaces minimizedbitcodepath of the minimized bitcode files with origpath when mapping the minimized bitcode to the full bitcode, otherwise it falls back to the old behavior of assuming both files are in the same directory.

It would be clearer to specify what this rewritten path is used for (i.e. written into the index and imports files for the subsequent distributed ThinLTO backends).

Also, similar to what I suggest below for the variables, origpath is somewhat of a misnomer if we have a minimizedbitcodepath (since the latter is actually the original input path to the thin link). So maybe origpath should be fullbitcodepath?

With this change, --thinlto-suffix-replace can now be deprecated. Instead of mapping the minimized bitcode to the full bitcode using a suffix replace, we can instead use a prefix replace.

I don't think it should be deprecated until and unless all build systems using it are moved to the new handling. Which might just be bazel but I'm not totally sure? If there are other users they may prefer the existing mechanism and directory structure.

lld/COFF/Config.h

227

This should be thinLTOPrefixReplaceMinimizedBitcode not thinLTOPrefixReplaceMinimizedIndex to match the code (and MinimizedBitcode is a better name IMO).

I also think with this change that we should rename thinLTOPrefixReplaceOld to thinLTOPrefixReplaceFullBitcode, and thinLTOPrefixReplaceNew with thinLTOPrefixReplaceOutputFile or something similar, to clarify what each of these paths are.

Then I suggest making the comment something like "replace the prefix of the minimized thin link input bitcode files, thinLTOPrefixReplaceMinimizedBitcode, with thinLTOPrefixReplaceFullBitcode, when the bitcode path is written to output index and imports files, allowing..."

lld/ELF/Config.h

179–180

Similar name changes here and in other Config.h to what I suggested above.

@MaskRay, would it be good to add comments about what these variables are, as is done in the COFF config file? I notice that there are no comments on any of the variables, but I think it might be helpful for these in particular.

lld/ELF/InputFiles.cpp

1586

The way the code is written, both suffix and prefix replacing can be done. If we don't want people to rely on that, and want to eventually move off of suffix replacement, we probably should give an error somewhere, i.e. in the option parsing, if both are specified. Then maybe add a comment here that we have already checked that only one is in effect.

1587

The name getThinLTOOutputFile is no longer very accurate. Perhaps make this getThinLTOReplacePathPrefix or something?

llvm/tools/gold/gold-plugin.cpp

897

Remove the extra space added here.

[itf]

lld/COFF/Config.h

227

Changing the name of the variables makes it more clear. I replaced it here and on the other files

lld/ELF/InputFiles.cpp

1586

I think it is fine to have both suffix and prefix replace, as they don't interfere with one another.

I can add a check so that this is not possible any longer and check the error in the tests, but if we intend to deprecate suffix replace this would just add more code to be removed in the future.

[MaskRay]

Apologies, I'm on an intense trip and will have little time studying patches before April 12...

lld/ELF/Config.h

179–180

Yes, having a comment will be useful.

lld/test/COFF/thinlto-minimized-bitcode-prefix-replace.ll

4

... && mkdir -p %t/minimized %t/full %t/indexing

[MaskRay]

Sorry for the delay. I finished a long trip and was spending some time to clean up my email queue:)

Currently, the --thinlto-suffix-replace="oldsuffix;newsuffix" option is used during distributed ThinLTO thin links to specify the mapping...

I feel that this paragraph can be made clearer. How about this:

For distributed ThinLTO, --thinlto-index-only= emits index files (*.thinlink.bc) and import files (*.imports) that reference IR module names.
When minimized bitcode files like a.indexing.o are used as input, we use --thinlto-suffix-replace='.indexing.o;.o' so that the IR module names record a.o instead of a.indexing.o.
Backend compiles need the full bitcode file paths, not the minimized bitcode file paths.

However, this forces the full bitcode file and its corresponding minimized bitcode file to be in the same directory.

This patch adds the fourth component to the value of --thinlto-prefix-replace=
to allow a separate directory for minimized bitcode files, whose hierarchy map the directory for full bitcode files.

This is important to support builds and build systems where a directory that is the output of a single action must either be fully included or not included as the input of another action (e.g. the thin link, it must take as its input the minimized bitcode files but not take as input the full bitcode files).

This paragraph can change the subject to be clearer.

In Bazel, when we have an action that may take an tree artifact as an input, the action takes as input all the files inside the directory of the tree artifact or none of them.
If a full bitcode file and its corresponding minimized bitcode file within the same directory, Bazel will have to ship both files to a backend compile action, causing waste.

---

The error message is

error: --thinlto-prefix-replace=old_dir;new_dir;obj_dir must be used with --thinlto-index-only=

It can be changed to:

error: --thinlto-prefix-replace=full;index;obj;minimized when obj is non-empty must be used with --thinlto-index-only=

lld/test/COFF/thinlto-minimized-bitcode-prefix-replace.ll

13

Bazel currently uses a.indexing.o for minimized bitcode files. Using %t/indexing will cause confusion for the emitted index files. Suggest: %t/index

Also fix tests for other ports.

24

Using %t/unused as both the output response file and the directory in -thinlto-prefix-replace: leads to confusion.

Suggest -thinlto-index-only:%t/a.rsp -thinlto-prefix-replace:'%t/full;%t/index;%t/obj;%t/minimized'

Prefer single quotes to double quotes.

Add a RUN line to inspect FileCheck --check-prefix=xxx < %t/a.rsp. The output is useful.

39

Fix: No newline at end of file

lld/test/ELF/lto/thinlto-minimized-bitcode-prefix-replace.ll

2

This file just duplicates thinlto-object-suffix-replace.ll ?

[MaskRay]

lld/COFF/Config.h

234

Perhaps it's clearer to name all variables thinLTOPrefixReplace*Dir, e.g. thinLTOPrefixReplace{Full,Index,Obj,Minimized}Dir?

lld/COFF/Driver.cpp

94–95

Perhaps full;index[;obj][;minimized]

lld/COFF/Config.h

struct Configuration {
   // Used for /mapinfo.
   bool mapInfo = false;
 
   // Used for /thinlto-index-only:
   llvm::StringRef thinLTOIndexOnlyArg;
 
   // Used for /thinlto-prefix-replace:
   // Replace the prefix in paths generated for ThinLTO, replacing
-  // thinLTOPrefixReplaceOld with thinLTOPrefixReplaceNew. If
+  // thinLTOPrefixReplaceFullBitcode with thinLTOPrefixReplaceOutput. If
   // thinLTOPrefixReplaceNativeObject is defined, replace the prefix of object
   // file paths written to the response file given in the
   // --thinlto-index-only=${response} option with
-  // thinLTOPrefixReplaceNativeObject, instead of thinLTOPrefixReplaceNew.
-  llvm::StringRef thinLTOPrefixReplaceOld;
-  llvm::StringRef thinLTOPrefixReplaceNew;
+  // thinLTOPrefixReplaceNativeObject, instead of thinLTOPrefixReplaceOutput.
+  // If thinLTOPrefixReplaceMinimizedBitcode, replace the prefix of the
+  // minimized thin link input bitcode files,
+  // thinLTOPrefixReplaceMinimizedBitcode, with thinLTOPrefixReplaceFullBitcode,
+  // when the bitcode path is written to output index and imports files,
+  // allowing minimized thin link input bitcode files to be in a different
+  // directory as the full bitcode files
+  llvm::StringRef thinLTOPrefixReplaceFullBitcode;
+  llvm::StringRef thinLTOPrefixReplaceOutput;
   llvm::StringRef thinLTOPrefixReplaceNativeObject;
+  llvm::StringRef thinLTOPrefixReplaceMinimizedBitcode;
 
   // Used for /thinlto-object-suffix-replace:
   std::pair<llvm::StringRef, llvm::StringRef> thinLTOObjectSuffixReplace;
 
   // Used for /lto-obj-path:
   llvm::StringRef ltoObjPath;
 
   // Used for /lto-cs-profile-generate:

lld/COFF/Driver.cpp

static std::pair<StringRef, StringRef> getOldNewOptions(opt::InputArgList &args,
 
   StringRef s = arg->getValue();
   std::pair<StringRef, StringRef> ret = s.split(';');
   if (ret.second.empty())
     error(arg->getSpelling() + " expects 'old;new' format, but got " + s);
   return ret;
 }
 
-// Parse options of the form "old;new[;extra]".
-static std::tuple<StringRef, StringRef, StringRef>
+// Parse options of the form "old;new[;extra][;extra2]".
+static std::tuple<StringRef, StringRef, StringRef, StringRef>
 getOldNewOptionsExtra(opt::InputArgList &args, unsigned id) {
   auto [oldDir, second] = getOldNewOptions(args, id);
-  auto [newDir, extraDir] = second.split(';');
-  return {oldDir, newDir, extraDir};
+  auto [newDir, third] = second.split(';');
+  auto [extraDir, extraDir2] = third.split(';');
+  return {oldDir, newDir, extraDir, extraDir2};
 }
 
 // Drop directory components and replace extension with
 // ".exe", ".dll" or ".sys".
 static std::string getOutputPath(StringRef path, bool isDll, bool isDriver) {
   StringRef ext = ".exe";
   if (isDll)
     ext = ".dll";
if (!config->manifestInput.empty() &&
     fatal("/manifestinput: requires /manifest:embed");
   }
 
   config->thinLTOEmitImportsFiles = args.hasArg(OPT_thinlto_emit_imports_files);
   config->thinLTOIndexOnly = args.hasArg(OPT_thinlto_index_only) ||
                              args.hasArg(OPT_thinlto_index_only_arg);
   config->thinLTOIndexOnlyArg =
       args.getLastArgValue(OPT_thinlto_index_only_arg);
-  std::tie(config->thinLTOPrefixReplaceOld, config->thinLTOPrefixReplaceNew,
-           config->thinLTOPrefixReplaceNativeObject) =
+  std::tie(config->thinLTOPrefixReplaceFullBitcode,
+           config->thinLTOPrefixReplaceOutput,
+           config->thinLTOPrefixReplaceNativeObject,
+           config->thinLTOPrefixReplaceMinimizedBitcode) =
       getOldNewOptionsExtra(args, OPT_thinlto_prefix_replace);
   config->thinLTOObjectSuffixReplace =
       getOldNewOptions(args, OPT_thinlto_object_suffix_replace);
   config->ltoObjPath = args.getLastArgValue(OPT_lto_obj_path);
   config->ltoCSProfileGenerate = args.hasArg(OPT_lto_cs_profile_generate);
   config->ltoCSProfileFile = args.getLastArgValue(OPT_lto_cs_profile_file);
   // Handle miscellaneous boolean flags.
   config->ltoPGOWarnMismatch = args.hasFlag(OPT_lto_pgo_warn_mismatch,

lld/COFF/InputFiles.cpp

thunkSym = ctx.symtab.addImportThunk(
         name, cast_or_null<DefinedImportData>(impSym), hdr->Machine);
 }
 
 BitcodeFile::BitcodeFile(COFFLinkerContext &ctx, MemoryBufferRef mb,
                          StringRef archiveName, uint64_t offsetInArchive,
                          bool lazy)
     : InputFile(ctx, BitcodeKind, mb, lazy) {
   std::string path = mb.getBufferIdentifier().str();
-  if (ctx.config.thinLTOIndexOnly)
+  if (ctx.config.thinLTOIndexOnly) {
     path = replaceThinLTOSuffix(mb.getBufferIdentifier(),
                                 ctx.config.thinLTOObjectSuffixReplace.first,
                                 ctx.config.thinLTOObjectSuffixReplace.second);
+    if (!ctx.config.thinLTOPrefixReplaceMinimizedBitcode.empty()) {
+      path = lto::getThinLTOReplacePathPrefix(
+          path, ctx.config.thinLTOPrefixReplaceMinimizedBitcode.str(),
+          ctx.config.thinLTOPrefixReplaceFullBitcode.str());
+    }
+  }
 
   // ThinLTO assumes that all MemoryBufferRefs given to it have a unique
   // name. If two archives define two members with the same name, this
   // causes a collision which result in only one of the objects being taken
   // into consideration at LTO time (which very likely causes undefined
   // symbols later in the link stage). So we append file offset to make
   // filename unique.
   MemoryBufferRef mbref(mb.getBuffer(),

lld/COFF/LTO.h

 private:
   std::unique_ptr<llvm::lto::LTO> ltoObj;
   std::vector<std::pair<std::string, SmallString<0>>> buf;
   std::vector<std::unique_ptr<MemoryBuffer>> files;
   std::vector<std::string> file_names;
   std::unique_ptr<llvm::raw_fd_ostream> indexFile;
   llvm::DenseSet<StringRef> thinIndices;
 
-  std::string getThinLTOOutputFile(StringRef path);
+  std::string getThinLTOReplacePathPrefix(StringRef path);
   llvm::lto::Config createConfig();
 
   COFFLinkerContext &ctx;
 };
 }
 
 #endif

lld/COFF/LTO.cpp

auto ret =
       std::make_unique<raw_fd_ostream>(file, ec, sys::fs::OpenFlags::OF_None);
   if (ec) {
     error("cannot open " + file + ": " + ec.message());
     return nullptr;
   }
   return ret;
 }
 
-std::string BitcodeCompiler::getThinLTOOutputFile(StringRef path) {
-  return lto::getThinLTOOutputFile(
-      std::string(path), std::string(ctx.config.thinLTOPrefixReplaceOld),
-      std::string(ctx.config.thinLTOPrefixReplaceNew));
+std::string BitcodeCompiler::getThinLTOReplacePathPrefix(StringRef path) {
+  return lto::getThinLTOReplacePathPrefix(
+      std::string(path),
+      std::string(ctx.config.thinLTOPrefixReplaceFullBitcode),
+      std::string(ctx.config.thinLTOPrefixReplaceOutput));
 }
 
 lto::Config BitcodeCompiler::createConfig() {
   lto::Config c;
   c.Options = initTargetOptionsFromCodeGenFlags();
   c.Options.EmitAddrsig = true;
   for (StringRef C : ctx.config.mllvmOpts)
     c.MllvmArgs.emplace_back(C.str());
BitcodeCompiler::BitcodeCompiler(COFFLinkerContext &c) : ctx(c) {
   if (!ctx.config.thinLTOIndexOnlyArg.empty())
     indexFile = openFile(ctx.config.thinLTOIndexOnlyArg);
 
   // Initialize ltoObj.
   lto::ThinBackend backend;
   if (ctx.config.thinLTOIndexOnly) {
     auto OnIndexWrite = [&](StringRef S) { thinIndices.erase(S); };
     backend = lto::createWriteIndexesThinBackend(
-        std::string(ctx.config.thinLTOPrefixReplaceOld),
-        std::string(ctx.config.thinLTOPrefixReplaceNew),
+        std::string(ctx.config.thinLTOPrefixReplaceFullBitcode),
+        std::string(ctx.config.thinLTOPrefixReplaceOutput),
         std::string(ctx.config.thinLTOPrefixReplaceNativeObject),
         ctx.config.thinLTOEmitImportsFiles, indexFile.get(), OnIndexWrite);
   } else {
     backend = lto::createInProcessThinBackend(
         llvm::heavyweight_hardware_concurrency(ctx.config.thinLTOJobs));
   }
 
   ltoObj = std::make_unique<lto::LTO>(createConfig(), backend,
checkError(ltoObj->run(
         buf[task].first = moduleName.str();
         return std::make_unique<CachedFileStream>(
             std::make_unique<raw_svector_ostream>(buf[task].second));
       },
       cache));
 
   // Emit empty index files for non-indexed files
   for (StringRef s : thinIndices) {
-    std::string path = getThinLTOOutputFile(s);
+    std::string path = getThinLTOReplacePathPrefix(s);
     openFile(path + ".thinlto.bc");
     if (ctx.config.thinLTOEmitImportsFiles)
       openFile(path + ".imports");
   }
 
   // ThinLTO with index only option is required to generate only the index
   // files. After that, we exit from linker and ThinLTO backend runs in a
   // distributed environment.

lld/ELF/Config.h

struct Config {
   llvm::StringRef soName;
   llvm::StringRef sysroot;
   llvm::StringRef thinLTOCacheDir;
   llvm::StringRef thinLTOIndexOnlyArg;
   llvm::StringRef whyExtract;
   StringRef zBtiReport = "none";
   StringRef zCetReport = "none";
   llvm::StringRef ltoBasicBlockSections;
   std::pair<llvm::StringRef, llvm::StringRef> thinLTOObjectSuffixReplace;
-  llvm::StringRef thinLTOPrefixReplaceOld;
-  llvm::StringRef thinLTOPrefixReplaceNew;
+  llvm::StringRef thinLTOPrefixReplaceFullBitcode;
+  llvm::StringRef thinLTOPrefixReplaceOutput;
   llvm::StringRef thinLTOPrefixReplaceNativeObject;
+  llvm::StringRef thinLTOPrefixReplaceMinimizedBitcode;
   std::string rpath;
   llvm::SmallVector<VersionDefinition, 0> versionDefinitions;
   llvm::SmallVector<llvm::StringRef, 0> auxiliaryList;
   llvm::SmallVector<llvm::StringRef, 0> filterList;
   llvm::SmallVector<llvm::StringRef, 0> passPlugins;
   llvm::SmallVector<llvm::StringRef, 0> searchPaths;
   llvm::SmallVector<llvm::StringRef, 0> symbolOrderingFile;
   llvm::SmallVector<llvm::StringRef, 0> thinLTOModulesToCompile;

lld/ELF/Driver.cpp

static std::pair<StringRef, StringRef> getOldNewOptions(opt::InputArgList &args,
 
   StringRef s = arg->getValue();
   std::pair<StringRef, StringRef> ret = s.split(';');
   if (ret.second.empty())
     error(getAliasSpelling(arg) + " expects 'old;new' format, but got " + s);
   return ret;
 }
 
-// Parse options of the form "old;new[;extra]".
-static std::tuple<StringRef, StringRef, StringRef>
+// Parse options of the form "old;new[;extra][;extra2]".
+static std::tuple<StringRef, StringRef, StringRef, StringRef>
 getOldNewOptionsExtra(opt::InputArgList &args, unsigned id) {
   auto [oldDir, second] = getOldNewOptions(args, id);
-  auto [newDir, extraDir] = second.split(';');
-  return {oldDir, newDir, extraDir};
+  auto [newDir, third] = second.split(';');
+  auto [extraDir, extraDir2] = third.split(';');
+  return {oldDir, newDir, extraDir, extraDir2};
 }
 
 // Parse the symbol ordering file and warn for any duplicate entries.
 static SmallVector<StringRef, 0> getSymbolOrderingFile(MemoryBufferRef mb) {
   SetVector<StringRef, SmallVector<StringRef, 0>> names;
   for (StringRef s : args::getLines(mb))
     if (!names.insert(s) && config->warnSymbolOrdering)
       warn(mb.getBufferIdentifier() + ": duplicate ordered symbol: " + s);
static void readConfigs(opt::InputArgList &args) {
   config->thinLTOEmitIndexFiles = args.hasArg(OPT_thinlto_emit_index_files) ||
                                   args.hasArg(OPT_thinlto_index_only) ||
                                   args.hasArg(OPT_thinlto_index_only_eq);
   config->thinLTOIndexOnly = args.hasArg(OPT_thinlto_index_only) ||
                              args.hasArg(OPT_thinlto_index_only_eq);
   config->thinLTOIndexOnlyArg = args.getLastArgValue(OPT_thinlto_index_only_eq);
   config->thinLTOObjectSuffixReplace =
       getOldNewOptions(args, OPT_thinlto_object_suffix_replace_eq);
-  std::tie(config->thinLTOPrefixReplaceOld, config->thinLTOPrefixReplaceNew,
-           config->thinLTOPrefixReplaceNativeObject) =
+  std::tie(config->thinLTOPrefixReplaceFullBitcode,
+           config->thinLTOPrefixReplaceOutput,
+           config->thinLTOPrefixReplaceNativeObject,
+           config->thinLTOPrefixReplaceMinimizedBitcode) =
       getOldNewOptionsExtra(args, OPT_thinlto_prefix_replace_eq);
   if (config->thinLTOEmitIndexFiles && !config->thinLTOIndexOnly) {
     if (args.hasArg(OPT_thinlto_object_suffix_replace_eq))
       error("--thinlto-object-suffix-replace is not supported with "
             "--thinlto-emit-index-files");
     else if (args.hasArg(OPT_thinlto_prefix_replace_eq))
       error("--thinlto-prefix-replace is not supported with "
             "--thinlto-emit-index-files");

lld/ELF/InputFiles.cpp

 
 BitcodeFile::BitcodeFile(MemoryBufferRef mb, StringRef archiveName,
                          uint64_t offsetInArchive, bool lazy)
     : InputFile(BitcodeKind, mb) {
   this->archiveName = archiveName;
   this->lazy = lazy;
 
   std::string path = mb.getBufferIdentifier().str();
-  if (config->thinLTOIndexOnly)
+  if (config->thinLTOIndexOnly) {
     path = replaceThinLTOSuffix(mb.getBufferIdentifier());
+    if (!config->thinLTOPrefixReplaceMinimizedBitcode.empty()) {
+      path = lto::getThinLTOReplacePathPrefix(
+          path, config->thinLTOPrefixReplaceMinimizedBitcode.str(),
+          config->thinLTOPrefixReplaceFullBitcode.str());
+    }
+  }
 
   // ThinLTO assumes that all MemoryBufferRefs given to it have a unique
   // name. If two archives define two members with the same name, this
   // causes a collision which result in only one of the objects being taken
   // into consideration at LTO time (which very likely causes undefined
   // symbols later in the link stage). So we append file offset to make
   // filename unique.
   StringRef name = archiveName.empty()

lld/ELF/LTO.cpp

static std::unique_ptr<raw_fd_ostream> openLTOOutputFile(StringRef file) {
   std::error_code ec;
   std::unique_ptr<raw_fd_ostream> fs =
       std::make_unique<raw_fd_stream>(file, ec);
   if (!ec)
     return fs;
   return openFile(file);
 }
 
-static std::string getThinLTOOutputFile(StringRef modulePath) {
-  return lto::getThinLTOOutputFile(
-      std::string(modulePath), std::string(config->thinLTOPrefixReplaceOld),
-      std::string(config->thinLTOPrefixReplaceNew));
+static std::string getThinLTOReplacePathPrefix(StringRef modulePath) {
+  return lto::getThinLTOReplacePathPrefix(
+      std::string(modulePath),
+      std::string(config->thinLTOPrefixReplaceFullBitcode),
+      std::string(config->thinLTOPrefixReplaceOutput));
 }
 
 static lto::Config createConfig() {
   lto::Config c;
 
   // LLD supports the new relocations and address-significance tables.
   c.Options = initTargetOptionsFromCodeGenFlags();
   c.Options.EmitAddrsig = true;
BitcodeCompiler::BitcodeCompiler() {
   if (!config->thinLTOIndexOnlyArg.empty())
     indexFile = openFile(config->thinLTOIndexOnlyArg);
 
   // Initialize ltoObj.
   lto::ThinBackend backend;
   auto onIndexWrite = [&](StringRef s) { thinIndices.erase(s); };
   if (config->thinLTOIndexOnly) {
     backend = lto::createWriteIndexesThinBackend(
-        std::string(config->thinLTOPrefixReplaceOld),
-        std::string(config->thinLTOPrefixReplaceNew),
+        std::string(config->thinLTOPrefixReplaceFullBitcode),
+        std::string(config->thinLTOPrefixReplaceOutput),
         std::string(config->thinLTOPrefixReplaceNativeObject),
         config->thinLTOEmitImportsFiles, indexFile.get(), onIndexWrite);
   } else {
     backend = lto::createInProcessThinBackend(
         llvm::heavyweight_hardware_concurrency(config->thinLTOJobs),
         onIndexWrite, config->thinLTOEmitIndexFiles,
         config->thinLTOEmitImportsFiles);
   }
for (BitcodeFile *f : ctx.bitcodeFiles)
     linkedBitCodeFiles.insert(f->getName());
 
   for (BitcodeFile *f : ctx.lazyBitcodeFiles) {
     if (!f->lazy)
       continue;
     if (linkedBitCodeFiles.contains(f->getName()))
       continue;
     std::string path =
-        replaceThinLTOSuffix(getThinLTOOutputFile(f->obj->getName()));
+        replaceThinLTOSuffix(getThinLTOReplacePathPrefix(f->obj->getName()));
     std::unique_ptr<raw_fd_ostream> os = openFile(path + ".thinlto.bc");
     if (!os)
       continue;
 
     ModuleSummaryIndex m(/*HaveGVs*/ false);
     m.setSkipModuleByDistributedBackend();
     writeIndexToFile(m, *os);
     if (config->thinLTOEmitImportsFiles)
checkError(ltoObj->run(
           return std::make_unique<CachedFileStream>(
               std::make_unique<raw_svector_ostream>(buf[task]));
         },
         cache));
 
   // Emit empty index files for non-indexed files but not in single-module mode.
   if (config->thinLTOModulesToCompile.empty()) {
     for (StringRef s : thinIndices) {
-      std::string path = getThinLTOOutputFile(s);
+      std::string path = getThinLTOReplacePathPrefix(s);
       openFile(path + ".thinlto.bc");
       if (config->thinLTOEmitImportsFiles)
         openFile(path + ".imports");
     }
   }
 
   if (config->thinLTOEmitIndexFiles)
     thinLTOCreateEmptyIndexFiles();

lld/MachO/Config.h

struct Configuration {
   llvm::StringRef thinLTOJobs;
   llvm::StringRef umbrella;
   uint32_t ltoo = 2;
   llvm::CodeGenOpt::Level ltoCgo;
   llvm::CachePruningPolicy thinLTOCachePolicy;
   llvm::StringRef thinLTOCacheDir;
   llvm::StringRef thinLTOIndexOnlyArg;
   std::pair<llvm::StringRef, llvm::StringRef> thinLTOObjectSuffixReplace;
-  llvm::StringRef thinLTOPrefixReplaceOld;
-  llvm::StringRef thinLTOPrefixReplaceNew;
+  llvm::StringRef thinLTOPrefixReplaceFullBitcode;
+  llvm::StringRef thinLTOPrefixReplaceOutput;
   llvm::StringRef thinLTOPrefixReplaceNativeObject;
+  llvm::StringRef thinLTOPrefixReplaceMinimizedBitcode;
   bool deadStripDylibs = false;
   bool demangle = false;
   bool deadStrip = false;
   bool errorForArchMismatch = false;
   bool ignoreAutoLink = false;
   // ld64 allows invalid auto link options as long as the link succeeds. LLD
   // does not, but there are cases in the wild where the invalid linker options
   // exist. This allows users to ignore the specific invalid options in the case

lld/MachO/Driver.cpp

static std::pair<StringRef, StringRef> getOldNewOptions(opt::InputArgList &args,
 
   StringRef s = arg->getValue();
   std::pair<StringRef, StringRef> ret = s.split(';');
   if (ret.second.empty())
     error(arg->getSpelling() + " expects 'old;new' format, but got " + s);
   return ret;
 }
 
-// Parse options of the form "old;new[;extra]".
-static std::tuple<StringRef, StringRef, StringRef>
+// Parse options of the form "old;new[;extra][;extra2]".
+static std::tuple<StringRef, StringRef, StringRef, StringRef>
 getOldNewOptionsExtra(opt::InputArgList &args, unsigned id) {
   auto [oldDir, second] = getOldNewOptions(args, id);
-  auto [newDir, extraDir] = second.split(';');
-  return {oldDir, newDir, extraDir};
+  auto [newDir, third] = second.split(';');
+  auto [extraDir, extraDir2] = third.split(';');
+  return {oldDir, newDir, extraDir, extraDir2};
 }
 
 static void parseClangOption(StringRef opt, const Twine &msg) {
   std::string err;
   raw_string_ostream os(err);
 
   const char *argv[] = {"lld", opt.data()};
   if (cl::ParseCommandLineOptions(2, argv, "", &os))
bool macho::link(ArrayRef<const char *> argsArr, llvm::raw_ostream &stdoutOS,
   config->thinLTOEmitIndexFiles = args.hasArg(OPT_thinlto_emit_index_files) ||
                                   args.hasArg(OPT_thinlto_index_only) ||
                                   args.hasArg(OPT_thinlto_index_only_eq);
   config->thinLTOIndexOnly = args.hasArg(OPT_thinlto_index_only) ||
                              args.hasArg(OPT_thinlto_index_only_eq);
   config->thinLTOIndexOnlyArg = args.getLastArgValue(OPT_thinlto_index_only_eq);
   config->thinLTOObjectSuffixReplace =
       getOldNewOptions(args, OPT_thinlto_object_suffix_replace_eq);
-  std::tie(config->thinLTOPrefixReplaceOld, config->thinLTOPrefixReplaceNew,
-           config->thinLTOPrefixReplaceNativeObject) =
+  std::tie(config->thinLTOPrefixReplaceFullBitcode,
+           config->thinLTOPrefixReplaceOutput,
+           config->thinLTOPrefixReplaceNativeObject,
+           config->thinLTOPrefixReplaceMinimizedBitcode) =
       getOldNewOptionsExtra(args, OPT_thinlto_prefix_replace_eq);
   if (config->thinLTOEmitIndexFiles && !config->thinLTOIndexOnly) {
     if (args.hasArg(OPT_thinlto_object_suffix_replace_eq))
       error("--thinlto-object-suffix-replace is not supported with "
             "--thinlto-emit-index-files");
     else if (args.hasArg(OPT_thinlto_prefix_replace_eq))
       error("--thinlto-prefix-replace is not supported with "
             "--thinlto-emit-index-files");

lld/MachO/InputFiles.cpp

return symtab->addDefined(name, &file, /*isec=*/nullptr, /*value=*/0,
                             /*isWeakDefCanBeHidden=*/false);
 }
 
 BitcodeFile::BitcodeFile(MemoryBufferRef mb, StringRef archiveName,
                          uint64_t offsetInArchive, bool lazy, bool forceHidden)
     : InputFile(BitcodeKind, mb, lazy), forceHidden(forceHidden) {
   this->archiveName = std::string(archiveName);
   std::string path = mb.getBufferIdentifier().str();
-  if (config->thinLTOIndexOnly)
+  if (config->thinLTOIndexOnly) {
     path = replaceThinLTOSuffix(mb.getBufferIdentifier());
+    if (!config->thinLTOPrefixReplaceMinimizedBitcode.empty()) {
+      path = lto::getThinLTOReplacePathPrefix(
+          path, config->thinLTOPrefixReplaceMinimizedBitcode.str(),
+          config->thinLTOPrefixReplaceFullBitcode.str());
+    }
+  }
 
   // ThinLTO assumes that all MemoryBufferRefs given to it have a unique
   // name. If two members with the same name are provided, this causes a
   // collision and ThinLTO can't proceed.
   // So, we append the archive name to disambiguate two members with the same
   // name from multiple different archives, and offset within the archive to
   // disambiguate two members of the same name from a single archive.
   MemoryBufferRef mbref(mb.getBuffer(),

lld/MachO/LTO.cpp

auto ret =
       std::make_unique<raw_fd_ostream>(file, ec, sys::fs::OpenFlags::OF_None);
   if (ec) {
     error("cannot open " + file + ": " + ec.message());
     return nullptr;
   }
   return ret;
 }
 
-static std::string getThinLTOOutputFile(StringRef modulePath) {
-  return lto::getThinLTOOutputFile(
-      std::string(modulePath), std::string(config->thinLTOPrefixReplaceOld),
-      std::string(config->thinLTOPrefixReplaceNew));
+static std::string getThinLTOReplacePathPrefix(StringRef modulePath) {
+  return lto::getThinLTOReplacePathPrefix(
+      std::string(modulePath),
+      std::string(config->thinLTOPrefixReplaceFullBitcode),
+      std::string(config->thinLTOPrefixReplaceOutput));
 }
 
 static lto::Config createConfig() {
   lto::Config c;
   c.Options = initTargetOptionsFromCodeGenFlags();
   c.Options.EmitAddrsig = config->icfLevel == ICFLevel::safe;
   for (StringRef C : config->mllvmOpts)
     c.MllvmArgs.emplace_back(C.str());
BitcodeCompiler::BitcodeCompiler() {
   if (!config->thinLTOIndexOnlyArg.empty())
     indexFile = openFile(config->thinLTOIndexOnlyArg);
 
   // Initialize ltoObj.
   lto::ThinBackend backend;
   auto onIndexWrite = [&](StringRef S) { thinIndices.erase(S); };
   if (config->thinLTOIndexOnly) {
     backend = lto::createWriteIndexesThinBackend(
-        std::string(config->thinLTOPrefixReplaceOld),
-        std::string(config->thinLTOPrefixReplaceNew),
+        std::string(config->thinLTOPrefixReplaceFullBitcode),
+        std::string(config->thinLTOPrefixReplaceOutput),
         std::string(config->thinLTOPrefixReplaceNativeObject),
         config->thinLTOEmitImportsFiles, indexFile.get(), onIndexWrite);
   } else {
     backend = lto::createInProcessThinBackend(
         llvm::heavyweight_hardware_concurrency(config->thinLTOJobs),
         onIndexWrite, config->thinLTOEmitIndexFiles,
         config->thinLTOEmitImportsFiles);
   }
static void thinLTOCreateEmptyIndexFiles() {
 
   for (InputFile *file : inputFiles) {
     if (auto *f = dyn_cast<BitcodeFile>(file)) {
       if (!f->lazy)
         continue;
       if (linkedBitCodeFiles.contains(f->getName()))
         continue;
       std::string path =
-          replaceThinLTOSuffix(getThinLTOOutputFile(f->obj->getName()));
+          replaceThinLTOSuffix(getThinLTOReplacePathPrefix(f->obj->getName()));
       std::unique_ptr<raw_fd_ostream> os = openFile(path + ".thinlto.bc");
       if (!os)
         continue;
 
       ModuleSummaryIndex m(/*HaveGVs=*/false);
       m.setSkipModuleByDistributedBackend();
       writeIndexToFile(m, *os);
       if (config->thinLTOEmitImportsFiles)
checkError(ltoObj->run(
         [&](size_t task, const Twine &moduleName) {
           return std::make_unique<CachedFileStream>(
               std::make_unique<raw_svector_ostream>(buf[task]));
         },
         cache));
 
   // Emit empty index files for non-indexed files
   for (StringRef s : thinIndices) {
-    std::string path = getThinLTOOutputFile(s);
+    std::string path = getThinLTOReplacePathPrefix(s);
     openFile(path + ".thinlto.bc");
     if (config->thinLTOEmitImportsFiles)
       openFile(path + ".imports");
   }
 
   if (config->thinLTOEmitIndexFiles)
     thinLTOCreateEmptyIndexFiles();
 

lld/test/COFF/thinlto-minimized-bitcode-prefix-replace.ll

+; REQUIRES: x86
+;; Test to make sure the thinlto-prefix-replace option can replace the
+;; prefix of minimized bitcode files
+; RUN: rm -rf %t && mkdir %t &&  mkdir %t/minimized && mkdir %t/full && mkdir %t/indexing
+
+;; Generate bitcode file with summary, as well as a minimized bitcode without
+; the debug metadata for the thin link.
+; RUN: opt -thinlto-bc %s -thin-link-bitcode-file=%t/minimized/1.obj -o %t/full/1.obj
+
+;; First perform the thin link on the normal bitcode file, and save the
+;; resulting index.
+; RUN: lld-link -entry:main -thinlto-index-only -thinlto-prefix-replace:"%t/full;%t/indexing" %t/full/1.obj -out:%t/t.exe
+; RUN: cp %t/indexing/1.obj.thinlto.bc %t/indexing/1.obj.thinlto.bc.orig
+
+;; Next perform the thin link on the minimized bitcode file, and compare dump
+;; of the resulting index to the above dump to ensure they are identical.
+;; Make sure it isn't inadvertently using the regular bitcode file.
+; RUN: rm -f %t/full/1.obj
+; RUN: lld-link -entry:main -thinlto-index-only -thinlto-prefix-replace:"%t/full;%t/indexing;;%t/minimized" \
+; RUN:     %t/minimized/1.obj -out:%t/t.exe
+; RUN: cmp %t/indexing/1.obj.thinlto.bc.orig %t/indexing/1.obj.thinlto.bc
+;; Make sure the third string to thinlto-prefix-replace does not affect the test
+; RUN: lld-link -entry:main -thinlto-index-only:"%t/unused" -thinlto-prefix-replace:"%t/full;%t/indexing;%t/unused;%t/minimized" \
+; RUN:     %t/minimized/1.obj -out:%t/t.exe
+; RUN: cmp %t/indexing/1.obj.thinlto.bc.orig %t/indexing/1.obj.thinlto.bc
+
+target datalayout = "e-m:w-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-pc-windows-msvc19.0.24215"
+
+define void @main() {
+entry:
+  ret void
+}
+
+!llvm.dbg.cu = !{}
+
+!1 = !{i32 2, !"Debug Info Version", i32 3}
+!llvm.module.flags = !{!1}
 No newline at end of file

lld/test/ELF/lto/thinlto-minimized-bitcode-prefix-replace.ll

+; REQUIRES: x86
+;; Test to make sure the thinlto-object-suffix-replace option is handled
+;; correctly.
+; RUN: rm -rf %t && mkdir %t && cd %t
+
+;; Generate bitcode file with summary, as well as a minimized bitcode without
+; the debug metadata for the thin link.
+; RUN: opt --thinlto-bc %s -thin-link-bitcode-file=1.thinlink.bc -o 1.o
+
+;; First perform the thin link on the normal bitcode file, and save the
+;; resulting index.
+; RUN: ld.lld --plugin-opt=thinlto-index-only -shared 1.o -o 3
+; RUN: cp 1.o.thinlto.bc 1.o.thinlto.bc.orig
+
+;; Next perform the thin link on the minimized bitcode file, and compare dump
+;; of the resulting index to the above dump to ensure they are identical.
+; RUN: rm -f 1.o.thinlto.bc
+;; Make sure it isn't inadvertently using the regular bitcode file.
+; RUN: rm -f 1.o
+; RUN: ld.lld --plugin-opt=thinlto-index-only --plugin-opt=thinlto-object-suffix-replace=".thinlink.bc;.o" \
+; RUN:   -shared 1.thinlink.bc -o 3
+; RUN: cmp 1.o.thinlto.bc.orig 1.o.thinlto.bc
+;; Also check that this works without the --plugin-opt= prefix.
+; RUN: ld.lld --thinlto-index-only --thinlto-object-suffix-replace=".thinlink.bc;.o" \
+; RUN:   -shared 1.thinlink.bc -o 3
+; RUN: cmp 1.o.thinlto.bc.orig 1.o.thinlto.bc
+
+;; Ensure lld generates error if object suffix replace option does not have 'old;new' format
+; RUN: rm -f 1.o.thinlto.bc
+; RUN: not ld.lld --plugin-opt=thinlto-index-only --plugin-opt=thinlto-object-suffix-replace="abc:def" -shared 1.thinlink.bc \
+; RUN:   -o 3 2>&1 | FileCheck %s --check-prefix=ERR1
+; ERR1: --plugin-opt=thinlto-object-suffix-replace= expects 'old;new' format, but got abc:def
+
+;; If filename does not end with old suffix, no suffix change should occur,
+;; so ".thinlto.bc" will simply be appended to the input file name.
+; RUN: rm -f 1.thinlink.bc.thinlto.bc
+; RUN: ld.lld --plugin-opt=thinlto-index-only --plugin-opt=thinlto-object-suffix-replace=".abc;.o" -shared 1.thinlink.bc -o /dev/null
+; RUN: ls 1.thinlink.bc.thinlto.bc
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define void @f() {
+entry:
+  ret void
+}
+
+!llvm.dbg.cu = !{}
+
+!1 = !{i32 2, !"Debug Info Version", i32 3}
+!llvm.module.flags = !{!1}

lld/test/MachO/thinlto-minimized-bitcode-prefix-replace.ll

+; REQUIRES: x86
+;; Test to make sure the thinlto-prefix-replace option can replace the
+;; prefix of minimized bitcode files
+; RUN: rm -rf %t && mkdir %t &&  mkdir %t/minimized && mkdir %t/full && mkdir %t/indexing
+
+;; Generate bitcode file with summary, as well as a minimized bitcode without
+; the debug metadata for the thin link.
+; RUN: opt --thinlto-bc %s -thin-link-bitcode-file=%t/minimized/1.o -o %t/full/1.o
+
+;; First perform the thin link on the normal bitcode file, and save the
+;; resulting index.
+; RUN: %lld --thinlto-index-only --thinlto-prefix-replace="%t/full;%t/indexing" -dylib %t/full/1.o -o 3
+; RUN: cp %t/indexing/1.o.thinlto.bc %t/indexing/1.o.thinlto.bc.orig
+
+;; Next perform the thin link on the minimized bitcode file, and compare dump
+;; of the resulting index to the above dump to ensure they are identical.
+;; Make sure it isn't inadvertently using the regular bitcode file.
+; RUN: rm -f %t/full/1.o
+; RUN: %lld --thinlto-index-only --thinlto-prefix-replace="%t/full;%t/indexing;;%t/minimized" \
+; RUN:   -dylib %t/minimized/1.o -o 3
+; RUN: cmp %t/indexing/1.o.thinlto.bc.orig %t/indexing/1.o.thinlto.bc
+;; Make sure the third string to thinlto-prefix-replace does not affect the test
+; RUN: %lld --thinlto-index-only="%t/unused" --thinlto-prefix-replace="%t/full;%t/indexing;%t/unused;%t/minimized" \
+; RUN:   -dylib %t/minimized/1.o -o 3
+; RUN: cmp %t/indexing/1.o.thinlto.bc.orig %t/indexing/1.o.thinlto.bc
+
+target datalayout = "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-apple-darwin"
+
+define void @f() {
+entry:
+  ret void
+}
+
+!llvm.dbg.cu = !{}
+
+!1 = !{i32 2, !"Debug Info Version", i32 3}
+!llvm.module.flags = !{!1}
 No newline at end of file

llvm/include/llvm/LTO/LTO.h

void computeLTOCacheKey(
     const std::set<GlobalValue::GUID> &CfiFunctionDefs = {},
     const std::set<GlobalValue::GUID> &CfiFunctionDecls = {});
 
 namespace lto {
 
 /// Given the original \p Path to an output file, replace any path
 /// prefix matching \p OldPrefix with \p NewPrefix. Also, create the
 /// resulting directory if it does not yet exist.
-std::string getThinLTOOutputFile(const std::string &Path,
+std::string getThinLTOReplacePathPrefix(const std::string &Path,
                                         const std::string &OldPrefix,
                                         const std::string &NewPrefix);
 
 /// Setup optimization remarks.
 Expected<std::unique_ptr<ToolOutputFile>> setupLLVMOptimizationRemarks(
     LLVMContext &Context, StringRef RemarksFilename, StringRef RemarksPasses,
     StringRef RemarksFormat, bool RemarksWithHotness,
     std::optional<uint64_t> RemarksHotnessThreshold = 0, int Count = -1);
 
 /// Setups the output file for saving statistics.

llvm/lib/LTO/LTO.cpp

return std::make_unique<InProcessThinBackend>(
         Conf, CombinedIndex, Parallelism, ModuleToDefinedGVSummaries, AddStream,
         Cache, OnWrite, ShouldEmitIndexFiles, ShouldEmitImportsFiles);
   };
 }
 
 // Given the original \p Path to an output file, replace any path
 // prefix matching \p OldPrefix with \p NewPrefix. Also, create the
 // resulting directory if it does not yet exist.
-std::string lto::getThinLTOOutputFile(const std::string &Path,
+std::string lto::getThinLTOReplacePathPrefix(const std::string &Path,
                                              const std::string &OldPrefix,
                                              const std::string &NewPrefix) {
   if (OldPrefix.empty() && NewPrefix.empty())
     return Path;
   SmallString<128> NewPath(Path);
   llvm::sys::path::replace_path_prefix(NewPath, OldPrefix, NewPrefix);
   StringRef ParentPath = llvm::sys::path::parent_path(NewPath.str());
   if (!ParentPath.empty()) {
     // Make sure the new directory exists, creating it if necessary.
     if (std::error_code EC = llvm::sys::fs::create_directories(ParentPath))
public:
 
   Error start(
       unsigned Task, BitcodeModule BM,
       const FunctionImporter::ImportMapTy &ImportList,
       const FunctionImporter::ExportSetTy &ExportList,
       const std::map<GlobalValue::GUID, GlobalValue::LinkageTypes> &ResolvedODR,
       MapVector<StringRef, BitcodeModule> &ModuleMap) override {
     StringRef ModulePath = BM.getModuleIdentifier();
-    std::string NewModulePath =
-        getThinLTOOutputFile(std::string(ModulePath), OldPrefix, NewPrefix);
+    std::string NewModulePath = getThinLTOReplacePathPrefix(
+        std::string(ModulePath), OldPrefix, NewPrefix);
 
     if (LinkedObjectsFile) {
       std::string ObjectPrefix =
           NativeObjectPrefix.empty() ? NewPrefix : NativeObjectPrefix;
-      std::string LinkedObjectsFilePath = getThinLTOOutputFile(
+      std::string LinkedObjectsFilePath = getThinLTOReplacePathPrefix(
           std::string(ModulePath), OldPrefix, ObjectPrefix);
       *LinkedObjectsFile << LinkedObjectsFilePath << '\n';
     }
 
     if (auto E = emitFiles(ImportList, ModulePath, NewModulePath))
       return E;
 
     if (OnWrite)

llvm/tools/gold/gold-plugin.cpp

static std::unique_ptr<LTO> createLTO(IndexWriteCallback OnIndexWrite,
   Conf.AlwaysEmitRegularLTOObj = !options::obj_path.empty();
 
   if (options::thinlto_index_only) {
     std::string OldPrefix, NewPrefix;
     getThinLTOOldAndNewPrefix(OldPrefix, NewPrefix);
     Backend = createWriteIndexesThinBackend(
         OldPrefix, NewPrefix,
         // TODO: Add support for optional native object path in
         // thinlto_prefix_replace option to match lld.
         /*NativeObjectPrefix=*/"", options::thinlto_emit_imports_files,
         LinkedObjectsFile, OnIndexWrite);
   } else {
     Backend = createInProcessThinBackend(
         llvm::heavyweight_hardware_concurrency(options::Parallelism));
   }
 
   Conf.OverrideTriple = options::triple;
for (claimed_file &F : Modules) {
     if (options::thinlto && !HandleToInputFile.count(F.leader_handle))
       HandleToInputFile.insert(std::make_pair(
           F.leader_handle, std::make_unique<PluginInputFile>(F.handle)));
     // In case we are thin linking with a minimized bitcode file, ensure
     // the module paths encoded in the index reflect where the backends
     // will locate the full bitcode files for compiling/importing.
     std::string Identifier =
         getThinLTOObjectFileName(F.name, OldSuffix, NewSuffix);
+    // TODO: Add support for optional minimized bitcode file path in
+    // thinlto_prefix_replace option to match lld.
     auto ObjFilename = ObjectToIndexFileState.insert({Identifier, false});
     assert(ObjFilename.second);
     if (const void *View = getSymbolsAndView(F))
       addModule(*Lto, F, View, ObjFilename.first->first());
     else if (options::thinlto_index_only) {
       ObjFilename.first->second = true;
       writeEmptyDistributedBuildOutputs(Identifier, OldPrefix, NewPrefix,
                                         /* SkipModule */ true);