[COFF] Improve the check for functions that should get an extra underscore

[mstorsjo]

Summary

This fixes exporting functions starting with an underscore, and fully decorated fastcall/vectorcall functions.

Tests will be added in the lld repo.

Also update a related comment to match the current behaviour, although it doesn't need any functional changes there.

[ruiu]

LGTM. Thank you for doing this!

lib/Object/COFFModuleDefinition.cpp

 
 struct Token {
   explicit Token(Kind T = Unknown, StringRef S = "") : K(T), Value(S) {}
   Kind K;
   StringRef Value;
 };
 
 static bool isDecorated(StringRef Sym, bool MingwDef) {
-  // mingw does not prepend "_".
-  return (!MingwDef && Sym.startswith("_")) || Sym.startswith("@") ||
-         Sym.startswith("?");
+  // In def files, the symbols can either be listed decorated or undecorated.
+  //
+  // - For cdecl symbols, only the undecorated form is allowed.
+  // - For fastcall and vectorcall symbols, both fully decorated or
+  //   undecorated forms can be present.
+  // - For stdcall symbols in non-MinGW environments, the decorated form is
+  //   fully decorated with leading underscore and trailing stack argument
+  //   size - like "_Func@0".
+  // - In MinGW def files, a decorated stdcall symbol does not include the
+  //   leading underscore though, like "Func@0".
+
+  // This function controls whether a leading underscore should be added to
+  // the given symbol name or not. For MinGW, treat a stdcall symbol name such
+  // as "Func@0" as undecorated, i.e. a leading underscore must be added.
+  // For non-MinGW, look for '@' in the whole string and consider "_Func@0"
+  // as decorated, i.e. don't add any more leading underscores.
+  // We can't check for a leading underscore here, since function names
+  // themselves can start with an underscore, while a second one still needs
+  // to be added.
+  return Sym.startswith("@") || Sym.contains("@@") || Sym.startswith("?") ||
+         (!MingwDef && Sym.contains('@'));
 }
 
 static Error createError(const Twine &Err) {
   return make_error<StringError>(StringRef(Err.str()),
                                  object_error::parse_failed);
 }
 
 class Lexer {

lib/ToolDrivers/llvm-dlltool/DlltoolDriver.cpp

int llvm::dlltoolDriverMain(llvm::ArrayRef<const char *> ArgsArr) {
 
   if (Machine == IMAGE_FILE_MACHINE_I386 && Args.getLastArg(OPT_k)) {
     for (COFFShortExport& E : Def->Exports) {
       if (E.isWeak() || (!E.Name.empty() && E.Name[0] == '?'))
         continue;
       E.SymbolName = E.Name;
       // Trim off the trailing decoration. Symbols will always have a
       // starting prefix here (either _ for cdecl/stdcall, @ for fastcall
-      // or ? for C++ functions). (Vectorcall functions also will end up having
-      // a prefix here, even if they shouldn't.)
+      // or ? for C++ functions). Vectorcall functions won't have any
+      // fixed prefix, but the function base name will still be at least
+      // one char.
       E.Name = E.Name.substr(0, E.Name.find('@', 1));
       // By making sure E.SymbolName != E.Name for decorated symbols,
       // writeImportLibrary writes these symbols with the type
       // IMPORT_NAME_UNDECORATE.
     }
   }
 
   if (writeImportLibrary(Def->OutputFile, Path, Def->Exports, Machine, true))
     return 1;
   return 0;
 }