Infer relocation model from module flags in relocatable LTO link

[eugenis]

Summary

Fix for PR33096.

[eugenis]

LLD fix coming in a separate patch.
I've opted not to change current behavior for shared and executable linking, but now I'm having second thoughts. Non-LTO pipeline allows the user to put PIC code in a non-PIE executable, or non-PIC code in a shared library if they wish so. Maybe we should completely ignore the output type here and strictly go by module flags, to be consistent.

[pcc]

Non-LTO pipeline allows the user to put PIC code in a non-PIE executable, or non-PIC code in a shared library if they wish so. Maybe we should completely ignore the output type here and strictly go by module flags, to be consistent.

Disabling PIC can sometimes act as an optimization. We may want to replace the reloc model field in lto::Config with a boolean that gives the backend permission to generate position-dependent code. Before we do that though, I think we will need to decide what to do about the other reloc models (dynamic-no-pic, rwpi, ropi, ropi-rwpi).

lib/LTO/LTO.cpp

117–120

I'd rewrite this with if/else and drop the .hasValue() here.

lib/LTO/LTOBackend.cpp

139

Can you pass the module into createTargetMachine and inline this function into it?

140

Drop .hasValue().

[joerg]

In D33372#760060, @pcc wrote:

Non-LTO pipeline allows the user to put PIC code in a non-PIE executable, or non-PIC code in a shared library if they wish so. Maybe we should completely ignore the output type here and strictly go by module flags, to be consistent.

Disabling PIC can sometimes act as an optimization.

Disabling PIC was a performance hack on 32bit x86 due to the extreme register starvation of the architecture.
It is also generally agreed that it is fundamentally broken from a design perspective. I don't see good reasons for supporting that, frankly.

[pcc]

In D33372#760139, @joerg wrote:

In D33372#760060, @pcc wrote:

Non-LTO pipeline allows the user to put PIC code in a non-PIE executable, or non-PIC code in a shared library if they wish so. Maybe we should completely ignore the output type here and strictly go by module flags, to be consistent.

Disabling PIC can sometimes act as an optimization.

Disabling PIC was a performance hack on 32bit x86 due to the extreme register starvation of the architecture.
It is also generally agreed that it is fundamentally broken from a design perspective. I don't see good reasons for supporting that, frankly.

Disabling PIC on its own doesn't seem that problematic to me. The problematic part is the assumption in the backend that symbols are DSO-local, which leads to hacks such as copy relocations. But with LTO the linker knows whether each symbol is DSO-local, and it could in principle pass that information to the backend, which could then generate appropriate code for accessing each symbol. This is blocked on something like D20217 though.

[pcc]

LGTM with one nit

lib/LTO/LTOBackend.cpp

119–120

Remove TheTriple argument, it can always be computed from the module.

[eugenis]

r303578

include/llvm/LTO/Config.h

 /// LTO configuration. A linker can configure LTO by setting fields in this data
 /// structure and passing it to the lto::LTO constructor.
 struct Config {
   // Note: when adding fields here, consider whether they need to be added to
   // computeCacheKey in LTO.cpp.
   std::string CPU;
   TargetOptions Options;
   std::vector<std::string> MAttrs;
-  Reloc::Model RelocModel = Reloc::PIC_;
+  Optional<Reloc::Model> RelocModel = Reloc::PIC_;
   CodeModel::Model CodeModel = CodeModel::Default;
   CodeGenOpt::Level CGOptLevel = CodeGenOpt::Default;
   TargetMachine::CodeGenFileType CGFileType = TargetMachine::CGFT_ObjectFile;
   unsigned OptLevel = 2;
   bool DisableVerify = false;
 
   /// Disable entirely the optimizer, including importing for ThinLTO
   bool CodeGenOnly = false;

lib/LTO/LTO.cpp

#endif
   // RelaxELFRelocations. The clang driver can also pass FunctionSections,
   // DataSections and DebuggerTuning via command line flags.
   AddUnsigned(Conf.Options.RelaxELFRelocations);
   AddUnsigned(Conf.Options.FunctionSections);
   AddUnsigned(Conf.Options.DataSections);
   AddUnsigned((unsigned)Conf.Options.DebuggerTuning);
   for (auto &A : Conf.MAttrs)
     AddString(A);
-  AddUnsigned(Conf.RelocModel);
+  if (Conf.RelocModel)
+    AddUnsigned(*Conf.RelocModel);
+  else
+    AddUnsigned(-1);
   AddUnsigned(Conf.CodeModel);
   AddUnsigned(Conf.CGOptLevel);
   AddUnsigned(Conf.CGFileType);
   AddUnsigned(Conf.OptLevel);
   AddString(Conf.OptPipeline);
   AddString(Conf.AAPipeline);
   AddString(Conf.OverrideTriple);
   AddString(Conf.DefaultTriple);

lib/LTO/LTOBackend.cpp

CombinedIndexHook = [=](const ModuleSummaryIndex &Index) {
     return true;
   };
 
   return Error::success();
 }
 
 namespace {
 
 std::unique_ptr<TargetMachine>
-createTargetMachine(Config &Conf, StringRef TheTriple,
-                    const Target *TheTarget) {
+createTargetMachine(Config &Conf, const Target *TheTarget, Module &M) {
+  StringRef TheTriple = M.getTargetTriple();
   SubtargetFeatures Features;
   Features.getDefaultSubtargetFeatures(Triple(TheTriple));
   for (const std::string &A : Conf.MAttrs)
     Features.AddFeature(A);
 
+  Reloc::Model RelocModel;
+  if (Conf.RelocModel)
+    RelocModel = *Conf.RelocModel;
+  else
+    RelocModel =
+        M.getPICLevel() == PICLevel::NotPIC ? Reloc::Static : Reloc::PIC_;
+
   return std::unique_ptr<TargetMachine>(TheTarget->createTargetMachine(
-      TheTriple, Conf.CPU, Features.getString(), Conf.Options, Conf.RelocModel,
+      TheTriple, Conf.CPU, Features.getString(), Conf.Options, RelocModel,
       Conf.CodeModel, Conf.CGOptLevel));
 }
 
 static void runNewPMPasses(Module &Mod, TargetMachine *TM, unsigned OptLevel) {
   PassBuilder PB(TM);
   AAManager AA;
 
   // Parse a custom AA pipeline if asked to.
   assert(PB.parseAAPipeline(AA, "default"));
 
   LoopAnalysisManager LAM;
   FunctionAnalysisManager FAM;
   CGSCCAnalysisManager CGAM;
SplitModule(
               Expected<std::unique_ptr<Module>> MOrErr = parseBitcodeFile(
                   MemoryBufferRef(StringRef(BC.data(), BC.size()), "ld-temp.o"),
                   Ctx);
               if (!MOrErr)
                 report_fatal_error("Failed to read bitcode");
               std::unique_ptr<Module> MPartInCtx = std::move(MOrErr.get());
 
               std::unique_ptr<TargetMachine> TM =
-                  createTargetMachine(C, MPartInCtx->getTargetTriple(), T);
+                  createTargetMachine(C, T, *MPartInCtx);
 
               codegen(C, TM.get(), AddStream, ThreadId, *MPartInCtx);
             },
             // Pass BC using std::move to ensure that it get moved rather than
             // copied into the thread's context.
             std::move(BC), ThreadCount++);
       },
       false);

 Error lto::backend(Config &C, AddStreamFn AddStream,
                    unsigned ParallelCodeGenParallelismLevel,
                    std::unique_ptr<Module> Mod,
                    ModuleSummaryIndex &CombinedIndex) {
   Expected<const Target *> TOrErr = initAndLookupTarget(C, *Mod);
   if (!TOrErr)
     return TOrErr.takeError();
 
-  std::unique_ptr<TargetMachine> TM =
-      createTargetMachine(C, Mod->getTargetTriple(), *TOrErr);
+  std::unique_ptr<TargetMachine> TM = createTargetMachine(C, *TOrErr, *Mod);
 
   // Setup optimization remarks.
   auto DiagFileOrErr = lto::setupOptimizationRemarks(
       Mod->getContext(), C.RemarksFilename, C.RemarksWithHotness);
   if (!DiagFileOrErr)
     return DiagFileOrErr.takeError();
   auto DiagnosticOutputFile = std::move(*DiagFileOrErr);
 
Error lto::thinBackend(Config &Conf, unsigned Task, AddStreamFn AddStream,
                        Module &Mod, const ModuleSummaryIndex &CombinedIndex,
                        const FunctionImporter::ImportMapTy &ImportList,
                        const GVSummaryMapTy &DefinedGlobals,
                        MapVector<StringRef, BitcodeModule> &ModuleMap) {
   Expected<const Target *> TOrErr = initAndLookupTarget(Conf, Mod);
   if (!TOrErr)
     return TOrErr.takeError();
 
-  std::unique_ptr<TargetMachine> TM =
-      createTargetMachine(Conf, Mod.getTargetTriple(), *TOrErr);
+  std::unique_ptr<TargetMachine> TM = createTargetMachine(Conf, *TOrErr, Mod);
 
   if (Conf.CodeGenOnly) {
     codegen(Conf, TM.get(), AddStream, Task, Mod);
     return Error::success();
   }
 
   if (Conf.PreOptModuleHook && !Conf.PreOptModuleHook(Task, Mod))
     return Error::success();

test/tools/gold/X86/relocation-model-pic.ll

+; RUN: cat %s >%t.pic.ll
+; RUN: echo '!llvm.module.flags = !{!0}' >>%t.pic.ll
+; RUN: echo '!0 = !{i32 1, !"PIC Level", i32 2}' >>%t.pic.ll
+
+; RUN: llvm-as %s -o %t.o
+; RUN: llvm-as %t.pic.ll -o %t.pic.o
+
+;; Non-PIC source.
+
+; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
+; RUN:    --shared \
+; RUN:    --plugin-opt=save-temps %t.o -o %t-out
+; RUN: llvm-readobj -r %t-out.o | FileCheck %s --check-prefix=PIC
+
+; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
+; RUN:    --export-dynamic --noinhibit-exec -pie \
+; RUN:    --plugin-opt=save-temps %t.o -o %t-out
+; RUN: llvm-readobj -r %t-out.o | FileCheck %s --check-prefix=PIC
+
+; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
+; RUN:    --export-dynamic --noinhibit-exec \
+; RUN:    --plugin-opt=save-temps %t.o -o %t-out
+; RUN: llvm-readobj -r %t-out.o | FileCheck %s --check-prefix=STATIC
+
+; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
+; RUN:    -r \
+; RUN:    --plugin-opt=save-temps %t.o -o %t-out
+; RUN: llvm-readobj -r %t-out.o | FileCheck %s --check-prefix=STATIC
+
+;; PIC source.
+
+; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
+; RUN:    --shared \
+; RUN:    --plugin-opt=save-temps %t.pic.o -o %t-out
+; RUN: llvm-readobj -r %t-out.o | FileCheck %s --check-prefix=PIC
+
+; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
+; RUN:    --export-dynamic --noinhibit-exec -pie \
+; RUN:    --plugin-opt=save-temps %t.pic.o -o %t-out
+; RUN: llvm-readobj -r %t-out.o | FileCheck %s --check-prefix=PIC
+
+; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
+; RUN:    --export-dynamic --noinhibit-exec \
+; RUN:    --plugin-opt=save-temps %t.pic.o -o %t-out
+; RUN: llvm-readobj -r %t-out.o | FileCheck %s --check-prefix=STATIC
+
+; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
+; RUN:    -r \
+; RUN:    --plugin-opt=save-temps %t.pic.o -o %t-out
+; RUN: llvm-readobj -r %t-out.o | FileCheck %s --check-prefix=PIC
+
+
+; PIC: R_X86_64_GOTPCREL foo
+; STATIC: R_X86_64_PC32 foo
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+@foo = external global i32
+define i32 @main() {
+  %t = load i32, i32* @foo
+  ret i32 %t
+}

tools/gold/gold-plugin.cpp

 }
 
 static ld_plugin_add_symbols add_symbols = nullptr;
 static ld_plugin_get_symbols get_symbols = nullptr;
 static ld_plugin_add_input_file add_input_file = nullptr;
 static ld_plugin_set_extra_library_path set_extra_library_path = nullptr;
 static ld_plugin_get_view get_view = nullptr;
 static bool IsExecutable = false;
-static Optional<Reloc::Model> RelocationModel;
+static Optional<Reloc::Model> RelocationModel = None;
 static std::string output_name = "";
 static std::list<claimed_file> Modules;
 static DenseMap<int, void *> FDToLeaderHandle;
 static StringMap<ResolutionInfo> ResInfo;
 static std::vector<std::string> Cleanup;
 
 namespace options {
   enum OutputType {
for (; tv->tv_tag != LDPT_NULL; ++tv) {
     // header.
     switch (static_cast<int>(tv->tv_tag)) {
     case LDPT_OUTPUT_NAME:
       output_name = tv->tv_u.tv_string;
       break;
     case LDPT_LINKER_OUTPUT:
       switch (tv->tv_u.tv_val) {
       case LDPO_REL: // .o
+        IsExecutable = false;
+        break;
       case LDPO_DYN: // .so
         IsExecutable = false;
         RelocationModel = Reloc::PIC_;
         break;
       case LDPO_PIE: // position independent executable
         IsExecutable = true;
         RelocationModel = Reloc::PIC_;
         break;
static std::unique_ptr<LTO> createLTO() {
   Conf.CPU = options::mcpu;
   Conf.Options = InitTargetOptionsFromCodeGenFlags();
 
   // Disable the new X86 relax relocations since gold might not support them.
   // FIXME: Check the gold version or add a new option to enable them.
   Conf.Options.RelaxELFRelocations = false;
 
   Conf.MAttrs = MAttrs;
-  Conf.RelocModel = *RelocationModel;
+  Conf.RelocModel = RelocationModel;
   Conf.CGOptLevel = getCGOptLevel();
   Conf.DisableVerify = options::DisableVerify;
   Conf.OptLevel = options::OptLevel;
   if (options::Parallelism)
     Backend = createInProcessThinBackend(options::Parallelism);
   if (options::thinlto_index_only) {
     std::string OldPrefix, NewPrefix;
     getThinLTOOldAndNewPrefix(OldPrefix, NewPrefix);