lld-link: Several tweaks to default entry point selection.

[thakis]

Summary

Three related changes:

First:. link.exe uses the presence of main and wmain to decide if it should call mainCRTStartup or wmainCRTStartup, even if /nodefaultlib is passed. For compatibility, remove FindMain logic.

Second: Default to the non-wide entrypoint if main is not found. This has two effects:

A: In normal links, lld-link now prints

lld-link: error: undefined symbol: _main
>>> referenced by f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:78
>>>               libcmt.lib(exe_main.obj):("int __cdecl invoke_main(void)" (?invoke_main@@YAHXZ))
>>> referenced by f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:283
>>>               libcmt.lib(exe_main.obj):("int __cdecl __scrt_common_main_seh(void)" (?__scrt_common_main_seh@@YAHXZ))

instead of

lld-link: error: entry point must be defined

This is arguably a better error message, since it now mentions that _main is missing. (This matches link.exe's diagnostic in this case.)

B: With /nodefautlib, we now default to mainCRTStartup if no main() is present, again matching link.exe. This makes r337407 obsolete.

This means if you have a cc file containing both mainCRTStartup and wmainCRTStartup and you pass /nodefaultlib /subsystem:console, lld-link will now call mainCRTStartup, matching link.exe

Third: Print a warning if both main and wmain are present, similar to link.exe's LNK4067.

[ruiu]

Can you add a test for the new warning message?

[thakis]

add warning test

[ruiu]

LGTM

COFF/Driver.cpp

 //
 // There are four different entry point functions for Windows executables,
 // each of which corresponds to a user-defined "main" function. This function
 // infers an entry point from a user-defined "main" function.
 StringRef LinkerDriver::findDefaultEntry() {
   assert(Config->Subsystem != IMAGE_SUBSYSTEM_UNKNOWN &&
          "must handle /subsystem before calling this");
 
-  // As a special case, if /nodefaultlib is given, we directly look for an
-  // entry point. This is because, if no default library is linked, users
-  // need to define an entry point instead of a "main".
-  bool FindMain = !Config->NoDefaultLibAll;
   if (Config->Subsystem == IMAGE_SUBSYSTEM_WINDOWS_GUI) {
-    if (findUnderscoreMangle(FindMain ? "WinMain" : "WinMainCRTStartup"))
-      return mangle("WinMainCRTStartup");
-    if (findUnderscoreMangle(FindMain ? "wWinMain" : "wWinMainCRTStartup"))
+    if (findUnderscoreMangle("wWinMain")) {
+      if (!findUnderscoreMangle("WinMain"))
         return mangle("wWinMainCRTStartup");
+      warn("found both wWinMain and WinMain; using latter");
     }
-  if (findUnderscoreMangle(FindMain ? "main" : "mainCRTStartup"))
-    return mangle("mainCRTStartup");
-  if (findUnderscoreMangle(FindMain ? "wmain" : "wmainCRTStartup"))
+    return mangle("WinMainCRTStartup");
+  }
+  if (findUnderscoreMangle("wmain")) {
+    if (!findUnderscoreMangle("main"))
       return mangle("wmainCRTStartup");
-  return "";
+    warn("found both wmain and main; using latter");
+  }
+  return mangle("mainCRTStartup");
 }
 
 WindowsSubsystem LinkerDriver::inferSubsystem() {
   if (Config->DLL)
     return IMAGE_SUBSYSTEM_WINDOWS_GUI;
+  // Note that link.exe infers the subsystem from the presence of these
+  // functions even if /entry: or /nodefaultlib are passed which causes them
+  // to not be called.
   bool HaveMain = findUnderscoreMangle("main");
   bool HaveWMain = findUnderscoreMangle("wmain");
   bool HaveWinMain = findUnderscoreMangle("WinMain");
   bool HaveWWinMain = findUnderscoreMangle("wWinMain");
   if (HaveMain || HaveWMain) {
     if (HaveWinMain || HaveWWinMain) {
       warn(std::string("found ") + (HaveMain ? "main" : "wmain") + " and " +
            (HaveWinMain ? "WinMain" : "wWinMain") +

test/COFF/entry-inference.test

 # RUN: sed -e s/ENTRYNAME/main/ %s | yaml2obj > %t.obj
 # RUN: not lld-link /out:%t.exe %t.obj > %t.log 2>&1
 # RUN: FileCheck -check-prefix=MAIN %s < %t.log
+# RUN: not lld-link /nodefaultlib /out:%t.exe %t.obj > %t.log 2>&1
+# RUN: FileCheck -check-prefix=MAIN %s < %t.log
 
 # RUN: sed s/ENTRYNAME/wmain/ %s | yaml2obj > %t.obj
 # RUN: not lld-link /out:%t.exe %t.obj > %t.log 2>&1
 # RUN: FileCheck -check-prefix=WMAIN %s < %t.log
+# RUN: not lld-link /nodefaultlib /out:%t.exe %t.obj > %t.log 2>&1
+# RUN: FileCheck -check-prefix=WMAIN %s < %t.log
 
 # RUN: sed s/ENTRYNAME/WinMain/ %s | yaml2obj > %t.obj
 # RUN: not lld-link /out:%t.exe %t.obj > %t.log 2>&1
 # RUN: FileCheck -check-prefix=WINMAIN %s < %t.log
+# RUN: not lld-link /nodefaultlib /out:%t.exe %t.obj > %t.log 2>&1
+# RUN: FileCheck -check-prefix=WINMAIN %s < %t.log
 
 # RUN: sed s/ENTRYNAME/wWinMain/ %s | yaml2obj > %t.obj
 # RUN: not lld-link /out:%t.exe %t.obj > %t.log 2>&1
 # RUN: FileCheck -check-prefix=WWINMAIN %s < %t.log
+# RUN: not lld-link /nodefaultlib /out:%t.exe %t.obj > %t.log 2>&1
+# RUN: FileCheck -check-prefix=WWINMAIN %s < %t.log
 
 # MAIN:     error: <root>: undefined symbol: mainCRTStartup
 # WMAIN:    error: <root>: undefined symbol: wmainCRTStartup
 # WINMAIN:  error: <root>: undefined symbol: WinMainCRTStartup
 # WWINMAIN: error: <root>: undefined symbol: wWinMainCRTStartup
 
 --- !COFF
 header:

test/COFF/entry-inference4.test

 # RUN: sed 's/ENTRY1/WinMain/;s/ENTRY2/main/' %s | yaml2obj > %t.obj
 # RUN: not lld-link /subsystem:windows /out:%t.exe %t.obj > %t.log 2>&1
 # RUN: FileCheck -check-prefix=WINMAIN %s < %t.log
 
 # RUN: sed 's/ENTRY1/wWinMain/;s/ENTRY2/main/' %s | yaml2obj > %t.obj
 # RUN: not lld-link /subsystem:windows /out:%t.exe %t.obj > %t.log 2>&1
 # RUN: FileCheck -check-prefix=WWINMAIN %s < %t.log
 
 # RUN: sed 's/ENTRY1/WinMain/;s/ENTRY2/main/' %s | yaml2obj > %t.obj
 # RUN: not lld-link /subsystem:console /out:%t.exe %t.obj > %t.log 2>&1
 # RUN: FileCheck -check-prefix=MAIN %s < %t.log
 
 # RUN: sed 's/ENTRY1/WinMain/;s/ENTRY2/wmain/' %s | yaml2obj > %t.obj
 # RUN: not lld-link /subsystem:console /out:%t.exe %t.obj > %t.log 2>&1
 # RUN: FileCheck -check-prefix=WMAIN %s < %t.log
 
-# MAIN:     error: <root>: undefined symbol: mainCRTStartup
-# WMAIN:    error: <root>: undefined symbol: wmainCRTStartup
-# WINMAIN:  error: <root>: undefined symbol: WinMainCRTStartup
-# WWINMAIN: error: <root>: undefined symbol: wWinMainCRTStartup
+# RUN: sed 's/ENTRY1/wmain/;s/ENTRY2/main/' %s | yaml2obj > %t.obj
+# RUN: not lld-link /out:%t.exe %t.obj > %t.log 2>&1
+# RUN: FileCheck -check-prefix=MAINWMAIN %s < %t.log
+
+# RUN: sed 's/ENTRY1/wWinMain/;s/ENTRY2/WinMain/' %s | yaml2obj > %t.obj
+# RUN: not lld-link /out:%t.exe %t.obj > %t.log 2>&1
+# RUN: FileCheck -check-prefix=WINMAINWWINMAIN %s < %t.log
+
+# MAIN:            error: <root>: undefined symbol: mainCRTStartup
+# WMAIN:           error: <root>: undefined symbol: wmainCRTStartup
+# MAINWMAIN:       warning: found both wmain and main; using latter
+# MAINWMAIN:       error: <root>: undefined symbol: mainCRTStartup
+# WINMAIN:         error: <root>: undefined symbol: WinMainCRTStartup
+# WWINMAIN:        error: <root>: undefined symbol: wWinMainCRTStartup
+# WINMAINWWINMAIN: warning: found both wWinMain and WinMain; using latter
+# WINMAINWWINMAIN: error: <root>: undefined symbol: WinMainCRTStartup
 
 --- !COFF
 header:
   Machine:         IMAGE_FILE_MACHINE_AMD64
   Characteristics: []
 sections:
   - Name:            .text
     Characteristics: [ IMAGE_SCN_CNT_CODE, IMAGE_SCN_MEM_EXECUTE, IMAGE_SCN_MEM_READ ]