[LTO][Doc] Document gotchas with LTO and libcalls.

[mysterymath]

Summary

The interaction of LTO and libcalls is subtle, and footguns abound.
Libcalls can cause lazy object files to be added to the link after LTO
code generation completes, which can cause issues if COFF/ELF dependent
libraries are used, or if the libcalls are implemented in bitcode.

This change documents the current state of affairs, along with
prescriptions for rules to follow to avoid undefined behavior.

[mysterymath]

Just wanted to ping this; if we can't reasonably improve the semantic collision between LTO, deplibs, and libcalls, it seems like it should at least be written down somewhere.

llvm/docs/LinkTimeOptimization.rst

 
 .. code-block:: c
 
   lto_codegen_compile(lto_code_gen_t, size*)
 
 which returns a pointer to a buffer containing the generated native object file.
 The linker then parses that and links it with the rest of the native object
 files.
+
+Libcalls
+========
+
+Code generation can insert new calls to library functions ("libcalls") to
+implement various IR operations. This can happen during LTO code generation as
+well, which can create a variety of subtle problems.
+
+Bitcode
+-------
+
+Libcall implementations may be pulled into the link after LTO code generation
+completes. If they are implemented in bitcode, LLVM's LTO implementation will
+not attempt to compile them, which can cause the link to fail. Accordingly,
+bitcode libcalls must be pulled into the link before code generation occurs. The
+symbols must also be marked as "used", since they may be unreferenced until code
+generation completes, but they must not be stripped out.
+
+``libLTO`` maintains a list of possible libcalls for this purpose:
+
+.. code-block:: c
+
+  extern const char *const *lto_runtime_lib_symbols_list(size_t *size)
+
+Members of this list will be automatically annotated to prevent premature
+internalization or removal. The various LLD implementations also use this list
+to summarily add any bitcall implementation to the link. The ``gold`` plugin
+doesn't yet do this, so compiler runtime libraries that wish to work with this
+plugin must not implement any libcalls in bitcode.
+
+Note that the list of libcalls is not exhaustive. Libcalls not on the list must
+not be implemented in bitcode.
+
+Dependent libraries
+-------------------
+
+COFF and ELF LLD allow an object file to declare dependencies on libraries. (For
+example, ``#pragma comment(lib, "name")``.) When such an object file is added to
+the link, the dependencies are also added. This also causes problems with LTO,
+since library dependencies for libcalls may enter the link after LTO code
+generation has completed.
+
+LTO code generation takes as input the linker's resolution table for each symbol
+present in the LTO module. For each such symbol, the linker records:
+
+1. Whether the symbol is referenced from outside of the LTO module.
+2. Whether the address of the symbol is significant.
+3. Whether the LTO module contains the prevailing definition of the symbol.
+
+Generally, the code generated by LTO will only be correct if these properties
+are not changed by libraries added afterwards.
+
+For ELF LLD, this isn't an issue, since the linker simply refuses to add any
+libraries after LTO code generation. This can cause link failures, but not
+miscompilation. On the other hand, COFF LLD does add libraries afterwards.
+
+For COFF, if the library had already been added to the link before LTO code
+generation, there's no additional risk of miscompile or link failure.
+Otherwise, certain rules must be followed to avoid undefined behavior.
+
+The first rule falls on the library dependency itself. It must not contain any
+undefined or weak references that might resolve to symbols in the LTO compile.
+
+The remaining rules fall on the LTO modules. These must not include any weak
+symbols that could resolve to symbols in the library. Neither may they include
+any strong symbols that could interpose on symbols in the library.