[ThinLTO] Use clang's existing code gen handling for ThinLTO backends

[tejohnson]

Summary

We noticed that when invoking the thinBackend via clang (for the
distributed build case) that flags like -ffunction-sections and
-emit-llvm were not having the intended effect. This could have been
fixed by setting up the TargetOptions and the CodeGenFileType in the LTO
Config, but since clang already has handling for all of this, it is
straightforward to just let it do the handling (just below the
early return after runThinLTOBackend being removed here).

Depends on D31100.

[tejohnson]

As discussed in the review threads for D31114 and D31100, only fall
back to clang's output file emission for -emit-llvm and -emit-llvm-bc.

[tejohnson]

lib/CodeGen/BackendUtil.cpp

1007

I just noticed that EmitAssembly does a lot more than just emission - it is also setting up an optimization pipeline in CreatePasses, which we don't want to do in the ThinLTO backend case as we already do the opt in LTO. Which makes me think that the simplest and most consistent solution, especially if as in discussed in D31114 we'll be using the LTO API for emitting object and assembly code, is to use LTO for all file emission in the ThinLTO backend case. It would mean extending lto::Config to be able to flag when we want to emit-llvm or emit-llvm-bc, and invoke the appropriate module writer/printer instead of normal codegen. That way -emit-llvm* will always get output corresponding to the native object/assembly in the ThinLTO backend case. WDYT?

[tejohnson]

lib/CodeGen/BackendUtil.cpp

1007

I have a new patch set that will do just this that I am uploading momentarily

[tejohnson]

Use LTO to emit LLVM IR

[tejohnson]

Subsumed by D31534

lib/CodeGen/BackendUtil.cpp

Expected<BitcodeModule> clang::FindThinLTOModule(MemoryBufferRef MBRef) {
   }
 
   return make_error<StringError>("Could not find module summary",
                                  inconvertibleErrorCode());
 }
 
 static void runThinLTOBackend(ModuleSummaryIndex *CombinedIndex, Module *M,
                               std::unique_ptr<raw_pwrite_stream> OS,
-                              std::string SampleProfile) {
+                              std::string SampleProfile,
+                              bool SkipCodeGen) {
   StringMap<std::map<GlobalValue::GUID, GlobalValueSummary *>>
       ModuleToDefinedGVSummaries;
   CombinedIndex->collectDefinedGVSummariesPerModule(ModuleToDefinedGVSummaries);
 
   // We can simply import the values mentioned in the combined index, since
   // we should only invoke this using the individual indexes written out
   // via a WriteIndexesThinBackend.
   FunctionImporter::ImportMapTy ImportList;
for (auto &I : ImportList) {
 
     OwnedImports.push_back(std::move(*MBOrErr));
   }
   auto AddStream = [&](size_t Task) {
     return llvm::make_unique<lto::NativeObjectStream>(std::move(OS));
   };
   lto::Config Conf;
   Conf.SampleProfile = SampleProfile;
+  Conf.SkipCodeGen = SkipCodeGen;
   if (Error E = thinBackend(
           Conf, 0, AddStream, *M, *CombinedIndex, ImportList,
           ModuleToDefinedGVSummaries[M->getModuleIdentifier()], ModuleMap)) {
     handleAllErrors(std::move(E), [&](ErrorInfoBase &EIB) {
       errs() << "Error running ThinLTO backend: " << EIB.message() << '\n';
     });
   }
 }
if (!IndexOrErr) {
       return;
     }
     std::unique_ptr<ModuleSummaryIndex> CombinedIndex = std::move(*IndexOrErr);
     // A null CombinedIndex means we should skip ThinLTO compilation
     // (LLVM will optionally ignore empty index files, returning null instead
     // of an error).
     bool DoThinLTOBackend = CombinedIndex != nullptr;
     if (DoThinLTOBackend) {
+      bool UsesCodeGen = (Action != Backend_EmitNothing &&
+                          Action != Backend_EmitBC &&
+                          Action != Backend_EmitLL);
+      if (UsesCodeGen) {
         runThinLTOBackend(CombinedIndex.get(), M, std::move(OS),
-                        CGOpts.SampleProfileFile);
+                          CGOpts.SampleProfileFile, /*SkipCodeGen*/false);
         return;
+      } else
+        runThinLTOBackend(CombinedIndex.get(), M, nullptr,
+                          CGOpts.SampleProfileFile, /*SkipCodeGen*/true);
     }
   }
 
   EmitAssemblyHelper AsmHelper(Diags, HeaderOpts, CGOpts, TOpts, LOpts, M);
 
   if (CGOpts.ExperimentalNewPassManager)
     AsmHelper.EmitAssemblyWithNewPassManager(Action, std::move(OS));
   else
     AsmHelper.EmitAssembly(Action, std::move(OS));
 
   // Verify clang's TargetInfo DataLayout against the LLVM TargetMachine's
   // DataLayout.
   if (AsmHelper.TM) {
     std::string DLDesc = M->getDataLayout().getStringRepresentation();
     if (DLDesc != TDesc.getStringRepresentation()) {
       unsigned DiagID = Diags.getCustomDiagID(
           DiagnosticsEngine::Error, "backend data layout '%0' does not match "

test/CodeGen/thinlto-emit-llvm.c

+// Test to ensure -emit-llvm and -emit-llvm-bc work when invoking the
+// ThinLTO backend path.
+// RUN: %clang -O2 %s -flto=thin -c -o %t.o
+// RUN: llvm-lto -thinlto -o %t %t.o
+// RUN: %clang_cc1 -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -emit-llvm -o - | FileCheck %s
+// RUN: %clang_cc1 -O2 -x ir %t.o -fthinlto-index=%t.thinlto.bc -emit-llvm-bc -o - | llvm-dis -o - | FileCheck %s
+
+// CHECK: define void @foo()
+void foo() {
+}