[LLD] [COFF] Add options for disabling auto import and runtime pseudo relocs

[mstorsjo]

Summary

Allow disabling either the full auto import feature, or just forbidding the cases that require runtime fixups.

As long as all auto imported variables are referenced from separate .refptr$<name> sections, we can alias them on top of the IAT entries and don't actually need any runtime fixups via pseudo relocations. LLVM generates references to variables in .refptr stubs, if it isn't known that the variable for sure is defined in the same object module. Runtime pseudo relocs are needed if the addresses of auto imported variables are used in constant initializers though.

Fixing up runtime pseudo relocations requires the use of VirtualProtect (which is disallowed in WinStore/UWP apps) or VirtualProtectFromApp. To allow any risk of ambiguity, allow rejecting cases that would require this at the linker stage.

This adds support for the --disable-runtime-pseudo-reloc and --disable-auto-import options in the MinGW driver (matching GNU ld.bfd) with corresponding lld private options in the COFF driver.

[mstorsjo]

lld/COFF/Options.td

194

It would be nice to be able to omit the HelpText for the two options here (to hide the option from the help listing), when using B<> style options for private options, but the current form of multiclass B<... doesn't allow omitting those strings and unconditionally adds HelpText, which includes it in the listing. I'm not nearly familiar enough with tablegen to know what's possible to do, though.

[rnk]

lgtm

lld/COFF/Options.td

194

I suppose the best option I can think of is two def Flag variants.

[mstorsjo]

In D78923#2021685, @rnk wrote:

lgtm

Thanks for the review - but I realized I'll want to revise this a bit before landing - ld.bfd actually has two different sets of options, --disable-auto-import and --disable-runtime-pseudo-relocs - and it might be good to keep the distinction. I.e. --disable-auto-import opts out of the whole process, requiring strict use of dllimports, while --disable-runtime-pseudo-relocs just makes it error out if pseudo relocs really are needed. Because thanks to .refptr$<sym> stubs, we actually don't need to create any pseudo relocs at all in most cases, as we alias the refptr stub on the IAT entry in those cases.

Also, did you specifically look at the part of the patch about allowing this lld private option in .drectve sections? I'm not entirely convinced about whether that's a good thing to do and would appreciate your opinion.

lld/COFF/Options.td

194

I realized the best might be to just make another multiclass, B_priv or something like that, without the help texts.

[mstorsjo]

Split out the support for these options in .drectve to a later patch, to get that discussed separately/explicitly.

Added two separate options, one for auto import in general, and one for pseudo relocations (which auto imports may or may not require).

[mstorsjo]

@rnk - can you have another look? I changed it a bit more than just touch-ups since the first review.

[mstorsjo]

Ping @rnk, can you have another look?

[rnk]

Looks good.

In D78923#2024678, @mstorsjo wrote:

Split out the support for these options in .drectve to a later patch, to get that discussed separately/explicitly.

Got it, I'll go look at that there. That could be interesting.

lld/COFF/Options.td

21

👍

[ruiu]

LGTM

[thakis]

Looks like you need to allow a backslash too: http://45.33.8.238/win/15234/step_10.txt

lld/COFF/Config.h

struct Configuration {
   bool incremental = true;
   bool integrityCheck = false;
   bool killAt = false;
   bool repro = false;
   bool swaprunCD = false;
   bool swaprunNet = false;
   bool thinLTOEmitImportsFiles;
   bool thinLTOIndexOnly;
+  bool autoImport = false;
+  bool pseudoRelocs = false;
 };
 
 extern Configuration *config;
 
 } // namespace coff
 } // namespace lld
 
 #endif

lld/COFF/Driver.cpp

void LinkerDriver::link(ArrayRef<const char *> argsArr) {
   config->nxCompat = args.hasFlag(OPT_nxcompat, OPT_nxcompat_no, true);
   for (auto *arg : args.filtered(OPT_swaprun))
     parseSwaprun(arg->getValue());
   config->terminalServerAware =
       !config->dll && args.hasFlag(OPT_tsaware, OPT_tsaware_no, true);
   config->debugDwarf = debug == DebugKind::Dwarf;
   config->debugGHashes = debug == DebugKind::GHash;
   config->debugSymtab = debug == DebugKind::Symtab;
+  config->autoImport =
+      args.hasFlag(OPT_auto_import, OPT_auto_import_no, config->mingw);
+  config->pseudoRelocs = args.hasFlag(
+      OPT_runtime_pseudo_reloc, OPT_runtime_pseudo_reloc_no, config->mingw);
 
   // Don't warn about long section names, such as .debug_info, for mingw or when
   // -debug:dwarf is requested.
   if (config->mingw || config->debugDwarf)
     config->warnLongSectionNames = false;
 
   config->lldmapFile = getMapFile(args, OPT_lldmap, OPT_lldmap_file);
   config->mapFile = getMapFile(args, OPT_map, OPT_map_file);
void LinkerDriver::link(ArrayRef<const char *> argsArr) {
   symtab->addAbsolute(mangle("__guard_flags"), 0);
   symtab->addAbsolute(mangle("__guard_iat_count"), 0);
   symtab->addAbsolute(mangle("__guard_iat_table"), 0);
   symtab->addAbsolute(mangle("__guard_longjmp_count"), 0);
   symtab->addAbsolute(mangle("__guard_longjmp_table"), 0);
   // Needed for MSVC 2017 15.5 CRT.
   symtab->addAbsolute(mangle("__enclave_config"), 0);
 
-  if (config->mingw) {
+  if (config->pseudoRelocs) {
     symtab->addAbsolute(mangle("__RUNTIME_PSEUDO_RELOC_LIST__"), 0);
     symtab->addAbsolute(mangle("__RUNTIME_PSEUDO_RELOC_LIST_END__"), 0);
+  }
+  if (config->mingw) {
     symtab->addAbsolute(mangle("__CTOR_LIST__"), 0);
     symtab->addAbsolute(mangle("__DTOR_LIST__"), 0);
   }
 
   // This code may add new undefined symbols to the link, which may enqueue more
   // symbol resolution tasks, so we need to continue executing tasks until we
   // converge.
   do {
void LinkerDriver::link(ArrayRef<const char *> argsArr) {
   if (args.hasArg(OPT_include_optional)) {
     // Handle /includeoptional
     for (auto *arg : args.filtered(OPT_include_optional))
       if (dyn_cast_or_null<LazyArchive>(symtab->find(arg->getValue())))
         addUndefined(arg->getValue());
     while (run());
   }
 
-  if (config->mingw) {
+  if (config->autoImport) {
+    // MinGW specific.
     // Load any further object files that might be needed for doing automatic
     // imports.
     //
     // For cases with no automatically imported symbols, this iterates once
     // over the symbol table and doesn't do anything.
     //
     // For the normal case with a few automatically imported symbols, this
     // should only need to be run once, since each new object file imported

lld/COFF/Options.td

 
 // Boolean flag which can be suffixed by ":no". Using it unsuffixed turns the
 // flag on and using it suffixed by ":no" turns it off.
 multiclass B<string name, string help_on, string help_off> {
   def "" : F<name>, HelpText<help_on>;
   def _no : F<name#":no">, HelpText<help_off>;
 }
 
+// Same as B<> above, but without help texts, for private undocumented
+// options.
+multiclass B_priv<string name> {
+  def "" : F<name>;
+  def _no : F<name#":no">;
+}
+
 def align   : P<"align", "Section alignment">;
 def aligncomm : P<"aligncomm", "Set common symbol alignment">;
 def alternatename : P<"alternatename", "Define weak alias">;
 def base    : P<"base", "Base address of the program">;
 def color_diagnostics: Flag<["--"], "color-diagnostics">,
     HelpText<"Use colors in diagnostics">;
 def color_diagnostics_eq: Joined<["--"], "color-diagnostics=">,
     HelpText<"Use colors in diagnostics; one of 'always', 'never', 'auto'">;
defm tsaware  : B<"tsaware",
                   "Create non-Terminal Server aware executable">;
 
 def help : F<"help">;
 
 // /?? and -?? must be before /? and -? to not confuse lib/Options.
 def help_q : Flag<["/??", "-??", "/?", "-?"], "">, Alias<help>;
 
 // LLD extensions
+defm auto_import : B_priv<"auto-import">;
+defm runtime_pseudo_reloc : B_priv<"runtime-pseudo-reloc">;
 def end_lib : F<"end-lib">,
   HelpText<"Ends group of objects treated as if they were in a library">;
 def exclude_all_symbols : F<"exclude-all-symbols">;
 def export_all_symbols : F<"export-all-symbols">;
 defm demangle : B<"demangle",
     "Demangle symbols in output (default)",
     "Do not demangle symbols in output">;
 def include_optional : Joined<["/", "-", "/?", "-?"], "includeoptional:">,

lld/COFF/SymbolTable.cpp

if (name.startswith("__imp_")) {
       }
     }
 
     // We don't want to report missing Microsoft precompiled headers symbols.
     // A proper message will be emitted instead in PDBLinker::aquirePrecompObj
     if (name.contains("_PchSym_"))
       continue;
 
-    if (config->mingw && handleMinGWAutomaticImport(sym, name))
+    if (config->autoImport && handleMinGWAutomaticImport(sym, name))
       continue;
 
     // Remaining undefined symbols are not fatal if /force is specified.
     // They are replaced with dummy defined symbols.
     if (config->forceUnresolved)
       replaceSymbol<DefinedAbsolute>(sym, name, 0);
     undefs.insert(sym);
   }

lld/COFF/Writer.cpp

void Writer::createMiscChunks() {
   // Create SEH table. x86-only.
   if (config->safeSEH)
     createSEHTable();
 
   // Create /guard:cf tables if requested.
   if (config->guardCF != GuardCFLevel::Off)
     createGuardCFTables();
 
-  if (config->mingw) {
+  if (config->autoImport)
     createRuntimePseudoRelocs();
 
+  if (config->mingw)
     insertCtorDtorSymbols();
-  }
-}
 
 // Create .idata section for the DLL-imported symbol table.
 // The format of this section is inherently Windows-specific.
 // IdataContents class abstracted away the details for us,
 // so we just let it create chunks and add them to the section.
 void Writer::createImportTables() {
   // Initialize DLLOrder so that import entries are ordered in
   // the same order as in the command line. (That affects DLL
void Writer::createRuntimePseudoRelocs() {
 
   for (Chunk *c : symtab->getChunks()) {
     auto *sc = dyn_cast<SectionChunk>(c);
     if (!sc || !sc->live)
       continue;
     sc->getRuntimePseudoRelocs(rels);
   }
 
+  if (!config->pseudoRelocs) {
+    // Not writing any pseudo relocs; if some were needed, error out and
+    // indicate what required them.
+    for (const RuntimePseudoReloc &rpr : rels)
+      error("automatic dllimport of " + rpr.sym->getName() + " in " +
+            toString(rpr.target->file) + " requires pseudo relocations");
+    return;
+  }
+
   if (!rels.empty())
     log("Writing " + Twine(rels.size()) + " runtime pseudo relocations");
   PseudoRelocTableChunk *table = make<PseudoRelocTableChunk>(rels);
   rdataSec->addChunk(table);
   EmptyChunk *endOfList = make<EmptyChunk>();
   rdataSec->addChunk(endOfList);
 
   Symbol *headSym = symtab->findUnderscore("__RUNTIME_PSEUDO_RELOC_LIST__");

lld/MinGW/Driver.cpp

bool mingw::link(ArrayRef<const char *> argsArr, bool canExitEarly,
   if (args.hasFlag(OPT_no_insert_timestamp, OPT_insert_timestamp, false))
     add("-timestamp:0");
 
   if (args.hasFlag(OPT_gc_sections, OPT_no_gc_sections, false))
     add("-opt:ref");
   else
     add("-opt:noref");
 
+  if (args.hasFlag(OPT_enable_auto_import, OPT_disable_auto_import, true))
+    add("-auto-import");
+  else
+    add("-auto-import:no");
+  if (args.hasFlag(OPT_enable_runtime_pseudo_reloc,
+                   OPT_disable_runtime_pseudo_reloc, true))
+    add("-runtime-pseudo-reloc");
+  else
+    add("-runtime-pseudo-reloc:no");
+
   if (auto *a = args.getLastArg(OPT_icf)) {
     StringRef s = a->getValue();
     if (s == "all")
       add("-opt:icf");
     else if (s == "safe" || s == "none")
       add("-opt:noicf");
     else
       error("unknown parameter: --icf=" + s);

lld/MinGW/Options.td

multiclass EqLong<string name, string help> {
   def NAME # _eq: Joined<["--"], name # "=">, Alias<!cast<Separate>(NAME)>,
     HelpText<help>;
 }
 
 def L: JoinedOrSeparate<["-"], "L">, MetaVarName<"<dir>">,
   HelpText<"Add a directory to the library search path">;
 def Bdynamic: F<"Bdynamic">, HelpText<"Link against shared libraries">;
 def Bstatic: F<"Bstatic">, HelpText<"Do not link against shared libraries">;
+def disable_auto_import: F<"disable-auto-import">,
+    HelpText<"Don't automatically import data symbols from other DLLs without dllimport">;
+def disable_runtime_pseudo_reloc: F<"disable-runtime-pseudo-reloc">,
+    HelpText<"Don't do automatic imports that require runtime fixups">;
 def dynamicbase: F<"dynamicbase">, HelpText<"Enable ASLR">;
+def enable_auto_import: F<"enable-auto-import">,
+    HelpText<"Automatically import data symbols from other DLLs where needed">;
+def enable_runtime_pseudo_reloc: F<"enable-runtime-pseudo-reloc">,
+    HelpText<"Allow automatic imports that require runtime fixups">;
 defm entry: Eq<"entry", "Name of entry point symbol">, MetaVarName<"<entry>">;
 def exclude_all_symbols: F<"exclude-all-symbols">,
     HelpText<"Don't automatically export any symbols">;
 def export_all_symbols: F<"export-all-symbols">,
     HelpText<"Export all symbols even if a def file or dllexport attributes are used">;
 def gc_sections: F<"gc-sections">, HelpText<"Remove unused sections">;
 def help: F<"help">, HelpText<"Print option help">;
 def icf: J<"icf=">, HelpText<"Identical code folding">;

 def alias_strip_S: Flag<["-"], "S">, Alias<strip_debug>;
 def alias_undefined_u: JoinedOrSeparate<["-"], "u">, Alias<undefined>;
 
 // Ignored options
 def: Joined<["-"], "O">;
 def: F<"build-id">;
 def: F<"disable-auto-image-base">;
 def: F<"enable-auto-image-base">;
-def: F<"enable-auto-import">, HelpText<"Ignored; listed for libtool compatibility">;
 def: F<"end-group">;
 def: Flag<["--"], "full-shutdown">;
 def: F<"high-entropy-va">;
 def: S<"major-image-version">;
 def: S<"minor-image-version">;
 def: F<"no-seh">;
 def: F<"nxcompat">;
 def: F<"pic-executable">;
 def: S<"plugin">;
 def: J<"plugin=">;
 def: S<"plugin-opt">;
 def: J<"plugin-opt=">;
 def: J<"sysroot">;
 def: F<"start-group">;
 def: F<"tsaware">;

lld/test/COFF/autoimport-refptr.s

 # REQUIRES: x86
 
 # RUN: echo -e ".global variable\n.global DllMainCRTStartup\n.text\nDllMainCRTStartup:\nret\n.data\nvariable:\n.long 42" > %t-lib.s
 # RUN: llvm-mc -triple=x86_64-windows-gnu %t-lib.s -filetype=obj -o %t-lib.obj
 # RUN: lld-link -out:%t-lib.dll -dll -entry:DllMainCRTStartup %t-lib.obj -lldmingw -implib:%t-lib.lib
 
-# RUN: llvm-mc -triple=x86_64-windows-gnu %s -filetype=obj -o %t.obj
+# RUN: llvm-mc -triple=x86_64-windows-gnu %s -defsym listptrs=1 -filetype=obj -o %t.obj
 # RUN: lld-link -lldmingw -out:%t.exe -entry:main %t.obj %t-lib.lib -verbose
 
 # RUN: llvm-readobj --coff-imports %t.exe | FileCheck -check-prefix=IMPORTS %s
 # RUN: llvm-objdump -d %t.exe | FileCheck --check-prefix=DISASM %s
 # RUN: llvm-objdump -s %t.exe | FileCheck --check-prefix=CONTENTS %s
 
+## Check that we can autoimport these variables with pseudo relocs disabled.
+# RUN: llvm-mc -triple=x86_64-windows-gnu %s -defsym listptrs=0 -filetype=obj -o %t.noptrs.obj
+# RUN: lld-link -lldmingw -runtime-pseudo-reloc:no -out:%t.exe -entry:main %t.noptrs.obj %t-lib.lib
+
+## Check that we can't autoimport them with autoimport disabled.
+# RUN: not lld-link -lldmingw -auto-import:no -out:%t.exe -entry:main %t.noptrs.obj %t-lib.lib 2>&1 | FileCheck --check-prefix=NO-AUTOIMPORT %s
+
 # IMPORTS: Import {
 # IMPORTS-NEXT: Name: autoimport-refptr.s.tmp-lib.dll
 # IMPORTS-NEXT: ImportLookupTableRVA: 0x2050
 # IMPORTS-NEXT: ImportAddressTableRVA: 0x2060
 # IMPORTS-NEXT: Symbol: variable (0)
 # IMPORTS-NEXT: }
 
 # DISASM: Disassembly of section .text:

 # DISASM: 140001012:      c3      retq
 
 # relocs: pointing at an empty list of runtime pseudo relocs.
 # localvar: 42
 # CONTENTS: Contents of section .data:
 # CONTENTS:  140003000 08200040 01000000 08200040 01000000
 # CONTENTS:  140003010 2a000000
 
+# NO-AUTOIMPORT: error: undefined symbol: variable
+
     .global main
     .global localvar
     .text
 main:
     movq .refptr.variable(%rip), %rax
     movl (%rax), %eax
     movq .refptr.localvar(%rip), %rcx
     addl (%rcx), %eax
     ret
 
     .data
+.if listptrs==1
 relocs:
     .quad __RUNTIME_PSEUDO_RELOC_LIST__
     .quad __RUNTIME_PSEUDO_RELOC_LIST_END__
+.endif
 localvar:
     .int 42
 
 # Normally the compiler wouldn't emit a stub for a variable that is
 # emitted in the same translation unit.
     .section .rdata$.refptr.localvar,"dr",discard,.refptr.localvar
     .global .refptr.localvar
 .refptr.localvar:
     .quad localvar
 
     .section .rdata$.refptr.variable,"dr",discard,.refptr.variable
     .global .refptr.variable
 .refptr.variable:
     .quad variable

lld/test/COFF/autoimport-x86.s

 # REQUIRES: x86
 
 # RUN: echo -e ".global variable\n.global DllMainCRTStartup\n.text\nDllMainCRTStartup:\nret\n.data\nvariable:\n.long 42" > %t-lib.s
 # RUN: llvm-mc -triple=x86_64-windows-gnu %t-lib.s -filetype=obj -o %t-lib.obj
 # RUN: lld-link -out:%t-lib.dll -dll -entry:DllMainCRTStartup %t-lib.obj -lldmingw -implib:%t-lib.lib
 
-# RUN: llvm-mc -triple=x86_64-windows-gnu %s -filetype=obj -o %t.obj
+# RUN: llvm-mc -triple=x86_64-windows-gnu -defsym listptrs=1 %s -filetype=obj -o %t.obj
 # RUN: lld-link -lldmingw -debug:symtab -out:%t.exe -entry:main %t.obj %t-lib.lib -verbose
 
+# RUN: llvm-mc -triple=x86_64-windows-gnu -defsym listptrs=0 %s -filetype=obj -o %t.noptrs.obj
+# RUN: not lld-link -lldmingw -runtime-pseudo-reloc:no -debug:symtab -out:%t.exe -entry:main %t.noptrs.obj %t-lib.lib 2>&1 | FileCheck --check-prefix=DISABLED %s
+
 # RUN: llvm-readobj --coff-imports %t.exe | FileCheck -check-prefix=IMPORTS %s
 # RUN: llvm-objdump -d %t.exe | FileCheck --check-prefix=DISASM %s
 # RUN: llvm-objdump -s %t.exe | FileCheck --check-prefix=CONTENTS %s
 # RUN: llvm-nm %t.exe | FileCheck -check-prefix=SYMBOLS %s
 
 # IMPORTS: Import {
 # IMPORTS-NEXT: Name: autoimport-x86.s.tmp-lib.dll
 # IMPORTS-NEXT: ImportLookupTableRVA: 0x2070

 # CONTENTS: Contents of section .data:
 # CONTENTS:  140003000 80200040 01000000 00200040 01000000
 # CONTENTS:  140003010 24200040 01000000
 
 # Check that the automatically imported symbol "variable" is not listed in
 # the symbol table.
 # SYMBOLS-NOT: variable
 
+# DISABLED: error: automatic dllimport of variable in {{.*}}/autoimport-x86.s.tmp.noptrs.obj requires pseudo relocations
+
     .global main
     .text
 main:
     movl variable(%rip), %eax
     ret
     .data
 ptr:
     .quad variable
+.if listptrs==1
 relocs:
     .quad __RUNTIME_PSEUDO_RELOC_LIST__
     .quad __RUNTIME_PSEUDO_RELOC_LIST_END__
+.endif

lld/test/MinGW/driver.test

 RUN: not ld.lld -m i386pep -entry 2>&1 | FileCheck -check-prefix MISSING_ARG %s
 MISSING_ARG: error: -entry: missing argument
 
 RUN: not ld.lld -m i386pep --foo 2>&1 | FileCheck -check-prefix UNKNOWN_ARG %s
 UNKNOWN_ARG: error: unknown argument: --foo
 
 RUN: not ld.lld -m i386pep 2>&1 | FileCheck -check-prefix NO_INPUT_FILES %s
 NO_INPUT_FILES: error: no input files
+
+RUN: ld.lld -### -m i386pep foo.o | FileCheck -check-prefix ENABLE_AUTO_IMPORT %s
+RUN: ld.lld -### -m i386pep foo.o --disable-auto-import --enable-auto-import | FileCheck -check-prefix ENABLE_AUTO_IMPORT %s
+RUN: ld.lld -### -m i386pep foo.o -enable-auto-import | FileCheck -check-prefix ENABLE_AUTO_IMPORT %s
+ENABLE_AUTO_IMPORT: -auto-import{{ }}
+
+RUN: ld.lld -### -m i386pep foo.o --disable-auto-import | FileCheck -check-prefix DISABLE_AUTO_IMPORT %s
+RUN: ld.lld -### -m i386pep foo.o -disable-auto-import | FileCheck -check-prefix DISABLE_AUTO_IMPORT %s
+DISABLE_AUTO_IMPORT: -auto-import:no
+
+RUN: ld.lld -### -m i386pep foo.o | FileCheck -check-prefix ENABLE_RUNTIME_PSEUDO_RELOC %s
+RUN: ld.lld -### -m i386pep foo.o --disable-runtime-pseudo-reloc --enable-runtime-pseudo-reloc | FileCheck -check-prefix ENABLE_RUNTIME_PSEUDO_RELOC %s
+RUN: ld.lld -### -m i386pep foo.o -enable-runtime-pseudo-reloc | FileCheck -check-prefix ENABLE_RUNTIME_PSEUDO_RELOC %s
+ENABLE_RUNTIME_PSEUDO_RELOC: -runtime-pseudo-reloc{{ }}
+
+RUN: ld.lld -### -m i386pep foo.o --disable-runtime-pseudo-reloc | FileCheck -check-prefix DISABLE_RUNTIME_PSEUDO_RELOC %s
+RUN: ld.lld -### -m i386pep foo.o -disable-runtime-pseudo-reloc | FileCheck -check-prefix DISABLE_RUNTIME_PSEUDO_RELOC %s
+DISABLE_RUNTIME_PSEUDO_RELOC: -runtime-pseudo-reloc:no