[LTO] set data layout for module before optimize&codegen

[ychen]

Summary

This is currently done for Clang & legacy regular LTO API. For
consistency, add the same logic to new LTO API, legacy ThinLTO API.

check-all shows no regression.

[tejohnson]

Sorry, completely missed this earlier. Is this fixing a bug? What is the effect on behavior?

[ychen]

Not for a concrete bug, but for consistency only. Basically, it makes the module data layout match target data layout before opt&codegen starts under all circumstances (LTO/clang).

[pcc]

I don't think we should do this unless it fixes a bug. What happens if you remove it from the other two places?

[ychen]

In D90782#2459370, @pcc wrote:

I don't think we should do this unless it fixes a bug. What happens if you remove it from the other two places?

A bunch of tests fail after removing the datalayout set from the other two places. One failed test is llvm/test/LTO/X86/attrs.ll

Exit Code: 134

Command Output (stderr):
--
+ : 'RUN: at line 1'
+ /home/ychen2/Src/u/build-clang-native-checking/bin/llvm-as
+ : 'RUN: at line 2'
+ /home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto -exported-symbol=test_x86_aesni_aeskeygenassist -mattr=+aes -o /home/ychen2/Src/u/build-clang-native-checking/test/LTO/X86/Output/attrs.ll.tmp2 /home/ychen2/Src/u/build-clang-native-checking/test/LTO/X86/Output/attrs.ll.tmp1
llvm-lto: /home/ychen2/Src/u/llvm/lib/CodeGen/MachineFunction.cpp:203: void llvm::MachineFunction::init(): Assertion `Target.isCompatibleDataLayout(getDataLayout()) && "Can't create a MachineFunction using a Module with a " "Target-incompatible DataLayout attached\n"' failed.
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0.      Program arguments: /home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto -exported-symbol=test_x86_aesni_aeskeygenassist -mattr=+aes -o /home/ychen2/Src/u/build-clang-native-checking/test/LTO/X86/Output/attrs.ll.tmp2 /home/ychen2/Src/u/build-clang-native-checking/test/LTO/X86/Output/attrs.ll.tmp1
1.      Running pass 'Function Pass Manager' on module 'ld-temp.o'.
2.      Running pass 'X86 DAG->DAG Instruction Selection' on function '@test_x86_aesni_aeskeygenassist'
 #0 0x0000000003a54873 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x3a54873)
 #1 0x0000000003a5270e llvm::sys::RunSignalHandlers() (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x3a5270e)
 #2 0x0000000003a54d25 SignalHandler(int) (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x3a54d25)
 #3 0x00007fe2e8563980 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12980)
 #4 0x00007fe2e700cfb7 raise /build/glibc-S7xCS9/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:51:0
 #5 0x00007fe2e700e921 abort /build/glibc-S7xCS9/glibc-2.27/stdlib/abort.c:81:0
 #6 0x00007fe2e6ffe48a __assert_fail_base /build/glibc-S7xCS9/glibc-2.27/assert/assert.c:89:0
 #7 0x00007fe2e6ffe502 (/lib/x86_64-linux-gnu/libc.so.6+0x30502)
 #8 0x000000000342558d (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x342558d)
 #9 0x000000000344e8e2 llvm::MachineModuleInfo::getOrCreateMachineFunction(llvm::Function&) (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x344e8e2)
#10 0x000000000342f219 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x342f219)
#11 0x000000000384cf28 llvm::FPPassManager::runOnFunction(llvm::Function&) (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x384cf28)
#12 0x0000000003852f38 llvm::FPPassManager::runOnModule(llvm::Module&) (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x3852f38)
#13 0x000000000384d547 llvm::legacy::PassManagerImpl::run(llvm::Module&) (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x384d547)
#14 0x00000000038d4be7 llvm::splitCodeGen(std::unique_ptr<llvm::Module, std::default_delete<llvm::Module> >, llvm::ArrayRef<llvm::raw_pwrite_stream*>, llvm::ArrayRef<llvm::raw_pwrite_stream*>, std::function<std::unique_ptr<llvm::TargetMachine, std::default_delete<llvm::TargetMachine> > ()> const&, llvm::CodeGenFileType, bool) (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x38d4be7)
#15 0x00000000038d2c1a llvm::LTOCodeGenerator::compileOptimized(llvm::ArrayRef<llvm::raw_pwrite_stream*>) (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x38d2c1a)
#16 0x00000000022c86a0 main (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x22c86a0)
#17 0x00007fe2e6fefbf7 __libc_start_main /build/glibc-S7xCS9/glibc-2.27/csu/../csu/libc-start.c:344:0
#18 0x00000000022c45da _start (/home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto+0x22c45da)
/home/ychen2/Src/u/build-clang-native-checking/test/LTO/X86/Output/attrs.ll.script: line 4: 16574 Aborted                 (core dumped) /home/ychen2/Src/u/build-clang-native-checking/bin/llvm-lto -exported-symbol=test_x86_aesni_aeskeygenassist -mattr=+aes -o /home/ychen2/Src/u/build-clang-native-checking/test/LTO/X86/Output/attrs.ll.tmp2 /home/ychen2/Src/u/build-clang-native-checking/test/LTO/X86/Output/attrs.ll.tmp1

--

********************
********************
Failed Tests (1):
  LLVM :: LTO/X86/attrs.ll

So it looks like the existing datalayout set is helpful at least in test regard. https://llvm.org/docs/LangRef.html#data-layout says "A module may specify a target specific data layout string ...". So the test input IR is not broken in that sense.

llvm/lib/LTO/LTOBackend.cpp

Error lto::backend(const Config &C, AddStreamFn AddStream,
                    unsigned ParallelCodeGenParallelismLevel,
                    std::unique_ptr<Module> Mod,
                    ModuleSummaryIndex &CombinedIndex) {
   Expected<const Target *> TOrErr = initAndLookupTarget(C, *Mod);
   if (!TOrErr)
     return TOrErr.takeError();
 
   std::unique_ptr<TargetMachine> TM = createTargetMachine(C, *TOrErr, *Mod);
+  if (TM)
+    Mod->setDataLayout(TM->createDataLayout());
 
   if (!C.CodeGenOnly) {
     if (!opt(C, TM.get(), 0, *Mod, /*IsThinLTO=*/false,
              /*ExportSummary=*/&CombinedIndex, /*ImportSummary=*/nullptr))
       return Error::success();
   }
 
   if (ParallelCodeGenParallelismLevel == 1) {
Error lto::thinBackend(const Config &Conf, unsigned Task, AddStreamFn AddStream,
                        const GVSummaryMapTy &DefinedGlobals,
                        MapVector<StringRef, BitcodeModule> &ModuleMap,
                        const std::vector<uint8_t> *CmdArgs) {
   Expected<const Target *> TOrErr = initAndLookupTarget(Conf, Mod);
   if (!TOrErr)
     return TOrErr.takeError();
 
   std::unique_ptr<TargetMachine> TM = createTargetMachine(Conf, *TOrErr, Mod);
+  if (TM)
+    Mod.setDataLayout(TM->createDataLayout());
 
   // Setup optimization remarks.
   auto DiagFileOrErr = lto::setupLLVMOptimizationRemarks(
       Mod.getContext(), Conf.RemarksFilename, Conf.RemarksPasses,
       Conf.RemarksFormat, Conf.RemarksWithHotness, Task);
   if (!DiagFileOrErr)
     return DiagFileOrErr.takeError();
   auto DiagnosticOutputFile = std::move(*DiagFileOrErr);
if (!BMOrErr) {
       });
       return false;
     }
     ModuleMap.insert({I.first(), *BMOrErr});
     OwnedImportsLifetimeManager.push_back(std::move(*MBOrErr));
   }
   return true;
 }
 No newline at end of file

llvm/lib/LTO/ThinLTOCodeGenerator.cpp

crossImportIntoModule(Module &TheModule, const ModuleSummaryIndex &Index,
   }
   // Verify again after cross-importing.
   verifyLoadedModule(TheModule);
 }
 
 static void optimizeModule(Module &TheModule, TargetMachine &TM,
                            unsigned OptLevel, bool Freestanding,
                            ModuleSummaryIndex *Index) {
+  TheModule.setDataLayout(TM.createDataLayout());
+
   // Populate the PassManager
   PassManagerBuilder PMB;
   PMB.LibraryInfo = new TargetLibraryInfoImpl(TM.getTargetTriple());
   if (Freestanding)
     PMB.LibraryInfo->disableAllFunctions();
   PMB.Inliner = createFunctionInliningPass();
   // FIXME: should get it from the bitcode?
   PMB.OptLevel = OptLevel;