[LLD] [COFF] Add an lld specific option /includeoptional

[mstorsjo]

Summary

This works like /include, but is not fatal if the requested symbol wasn't found. This allows implementing the GNU ld option -u.

[ruiu]

Our algorithm for the ELF -u option is as follows:

1. resolve all symbols normally
2. for each symbol given by -u, look up the symbol table, and if a returned symbol is a lazy symbol, call fetch

This way seems better than the way how it is implemented in this patch for the following two reasons:

1. we don't need any Config member as -u option values are consumed in-place
2. The cost of -u is proportional to the number of -u options instead of the number of (undefined) symbols

[mstorsjo]

COFF/SymbolTable.cpp

247 ↗

(On Diff #203426)

This won't do the right thing, if there actually are proper undefined to the same symbol as well. Any ideas/suggestions, @rnk @ruiu?

[ruiu]

I guess that my previous comment addresses the issue?

[mstorsjo]

In D62976#1533841, @ruiu wrote:

I guess that my previous comment addresses the issue?

Yes, probably - thanks!

Since we fetch objects iteratively in the COFF linker, we probably need to wait until the normal link has converged, then fetch lazy objs for -u, and then keep iterating until it converges again?

[ruiu]

In D62976#1533960, @mstorsjo wrote:

In D62976#1533841, @ruiu wrote:

I guess that my previous comment addresses the issue?

Yes, probably - thanks!

Since we fetch objects iteratively in the COFF linker, we probably need to wait until the normal link has converged, then fetch lazy objs for -u, and then keep iterating until it converges again?

Yeah, I guess so. Technically, fetching an object file can add a new archive to the input file set via a .drctive section which could in turn make other -u symbols can be resolved, so we need to iterate the whole process until it converges. That said, I don't think we have to cover that case though.

[mstorsjo]

Adjusted to make it work more like how this is implemented in the ELF linker; still failing (and properly so) if there's real undefined references to a symbol that was requested with the new option (-u in GNU ld).

[ruiu]

LGTM

COFF/Driver.cpp

1687–1690 ↗

(On Diff #203596)

I think you can write them in one line

if (auto *Sym = dyn_cast_or_null<Lazy>(Symtab->find(Arg->getValue()))

COFF/Options.td

170 ↗

(On Diff #203596)

We use the same spellings for identifiers representing a symbol name, so I'd name this include_optional.

lld/trunk/COFF/Driver.cpp

void LinkerDriver::link(ArrayRef<const char *> ArgsArr) {
   if (errorCount())
     return;
 
   // Do LTO by compiling bitcode input files to a set of native COFF files then
   // link those files.
   Symtab->addCombinedLTOObjects();
   run();
 
+  if (Args.hasArg(OPT_include_optional)) {
+    // Handle /includeoptional
+    for (auto *Arg : Args.filtered(OPT_include_optional))
+      if (dyn_cast_or_null<Lazy>(Symtab->find(Arg->getValue())))
+        addUndefined(Arg->getValue());
+    while (run());
+  }
+
   if (Config->MinGW) {
     // Load any further object files that might be needed for doing automatic
     // imports.
     //
     // For cases with no automatically imported symbols, this iterates once
     // over the symbol table and doesn't do anything.
     //
     // For the normal case with a few automatically imported symbols, this

lld/trunk/COFF/Options.td

 def help_q : Flag<["/??", "-??", "/?", "-?"], "">, Alias<help>;
 
 // LLD extensions
 def exclude_all_symbols : F<"exclude-all-symbols">;
 def export_all_symbols : F<"export-all-symbols">;
 defm demangle : B<"demangle",
     "Demangle symbols in output (default)",
     "Do not demangle symbols in output">;
+def include_optional : Joined<["/", "-", "/?", "-?"], "includeoptional:">,
+    HelpText<"Add symbol as undefined, but allow it to remain undefined">;
 def kill_at : F<"kill-at">;
 def lldmingw : F<"lldmingw">;
 def output_def : Joined<["/", "-", "/?", "-?"], "output-def:">;
 def pdb_source_path : P<"pdbsourcepath",
                         "Base path used to make relative source file path absolute in PDB">;
 def rsp_quoting : Joined<["--"], "rsp-quoting=">,
   HelpText<"Quoting style for response files, 'windows' (default) or 'posix'">;
 def dash_dash_version : Flag<["--"], "version">,

lld/trunk/test/COFF/includeoptional.yaml

+# RUN: yaml2obj < %s > %t.main.obj
+# RUN: yaml2obj < %p/Inputs/include1c.yaml > %t.lib.obj
+# RUN: rm -f %t.lib.lib
+# RUN: llvm-ar cru %t.lib.lib %t.lib.obj
+# RUN: lld-link /out:%t.exe /entry:main /includeoptional:bar /includeoptional:actuallymissing %t.main.obj %t.lib.lib /verbose >& %t.log
+# RUN: FileCheck %s < %t.log
+
+# CHECK: includeoptional.yaml.tmp.main.obj
+# CHECK: includeoptional.yaml.tmp.lib.lib
+# CHECK: includeoptional.yaml.tmp.lib.lib(includeoptional.yaml.tmp.lib.obj) for bar
+--- !COFF
+header:
+  Machine:         IMAGE_FILE_MACHINE_AMD64
+  Characteristics: []
+sections:
+  - Name:            .text
+    Characteristics: [ IMAGE_SCN_CNT_CODE, IMAGE_SCN_MEM_EXECUTE, IMAGE_SCN_MEM_READ ]
+    Alignment:       4
+    SectionData:     B800000000506800000000680000000050E80000000050E800000000
+symbols:
+  - Name:            .text
+    Value:           0
+    SectionNumber:   1
+    SimpleType:      IMAGE_SYM_TYPE_NULL
+    ComplexType:     IMAGE_SYM_DTYPE_NULL
+    StorageClass:    IMAGE_SYM_CLASS_STATIC
+    SectionDefinition:
+      Length:          28
+      NumberOfRelocations: 4
+      NumberOfLinenumbers: 0
+      CheckSum:        0
+      Number:          0
+  - Name:            main
+    Value:           0
+    SectionNumber:   1
+    SimpleType:      IMAGE_SYM_TYPE_NULL
+    ComplexType:     IMAGE_SYM_DTYPE_NULL
+    StorageClass:    IMAGE_SYM_CLASS_EXTERNAL
+...

lld/trunk/test/COFF/includeoptional2.yaml

+# RUN: yaml2obj < %s > %t.obj
+# RUN: not lld-link /out:%t.exe /entry:main /includeoptional:undeffunc %t.obj /verbose 2>&1 | FileCheck %s
+
+# CHECK: error: undefined symbol: undeffunc
+
+--- !COFF
+header:
+  Machine:         IMAGE_FILE_MACHINE_AMD64
+  Characteristics: []
+sections:
+  - Name:            .text
+    Characteristics: [ IMAGE_SCN_CNT_CODE, IMAGE_SCN_MEM_EXECUTE, IMAGE_SCN_MEM_READ ]
+    Alignment:       4
+    SectionData:     B800000000506800000000680000000050E80000000050E800000000
+    Relocations:
+      - VirtualAddress:  17
+        SymbolName:      undeffunc
+        Type:            IMAGE_REL_AMD64_REL32
+symbols:
+  - Name:            .text
+    Value:           0
+    SectionNumber:   1
+    SimpleType:      IMAGE_SYM_TYPE_NULL
+    ComplexType:     IMAGE_SYM_DTYPE_NULL
+    StorageClass:    IMAGE_SYM_CLASS_STATIC
+    SectionDefinition:
+      Length:          28
+      NumberOfRelocations: 1
+      NumberOfLinenumbers: 0
+      CheckSum:        0
+      Number:          0
+  - Name:            main
+    Value:           0
+    SectionNumber:   1
+    SimpleType:      IMAGE_SYM_TYPE_NULL
+    ComplexType:     IMAGE_SYM_DTYPE_NULL
+    StorageClass:    IMAGE_SYM_CLASS_EXTERNAL
+  - Name:            undeffunc
+    Value:           0
+    SectionNumber:   0
+    SimpleType:      IMAGE_SYM_TYPE_NULL
+    ComplexType:     IMAGE_SYM_DTYPE_NULL
+    StorageClass:    IMAGE_SYM_CLASS_EXTERNAL
+...