[lld-macho]Limit "cannot-export-hidden-symbol" warnings to only 3 to avoid crowding logs

[oontvoo]

Summary

Details:
We often use wildcard symbols in the exported_symbols list, and sometimes they match autohide symbols, which triggers these "cannot export hidden symbols" warnings that can be a bit noisy.

It'd be more user-friendly if LLD could truncate these.

[oontvoo]

refactor

[thakis]

How does ld64 handle this situation?

[oontvoo]

In D159095#4626864, @thakis wrote:

How does ld64 handle this situation?

LD64 would issue similar warnings. It's not that LLD is doing anything wrong here.
It's just that in our internal use case, we have a huge list of exported-symbols (with wild-cards) so these warnings don't really offer anything useful to us and often crowd the log.
Having a way to silence them would drastically improve the readability of logs (and potentially speed up the linkage too, due to less I/O), especially when debugging other linker errors.

[oontvoo]

Per off-line discussion, will rework this so that LLD only emit the warning 3 times (or some smaller number, can decide/debate later)

[oontvoo]

updated patch to truncate the warnings

[thakis]

Thanks!

lld/MachO/Driver.cpp

1388

why an explicit call to .load() here but not in the line above? be self-consistent.

[oontvoo]

removed .load() call

[oontvoo]

Thanks!

lld/MachO/Driver.cpp

if (auto *objFile = dyn_cast<ObjFile>(file)) {
                 " for " + alias->getName());
         }
       }
     }
   }
 }
 
 static void handleExplicitExports() {
+  static constexpr int kMaxWarnings = 3;
   if (config->hasExplicitExports) {
-    parallelForEach(symtab->getSymbols(), [](Symbol *sym) {
+    std::atomic<uint64_t> warningsCount{0};
+    parallelForEach(symtab->getSymbols(), [&warningsCount](Symbol *sym) {
       if (auto *defined = dyn_cast<Defined>(sym)) {
         if (config->exportedSymbols.match(sym->getName())) {
           if (defined->privateExtern) {
             if (defined->weakDefCanBeHidden) {
               // weak_def_can_be_hidden symbols behave similarly to
               // private_extern symbols in most cases, except for when
               // it is explicitly exported.
               // The former can be exported but the latter cannot.
               defined->privateExtern = false;
             } else {
+              // Only print the first 3 warnings verbosely, and
+              // shorten the rest to avoid crowding logs.
+              if (warningsCount.fetch_add(1, std::memory_order_relaxed) <
+                  kMaxWarnings)
                 warn("cannot export hidden symbol " + toString(*defined) +
                      "\n>>> defined in " + toString(defined->getFile()));
             }
           }
         } else {
           defined->privateExtern = true;
         }
       } else if (auto *dysym = dyn_cast<DylibSymbol>(sym)) {
         dysym->shouldReexport = config->exportedSymbols.match(sym->getName());
       }
     });
+    if (warningsCount > kMaxWarnings)
+      warn("<... " + Twine(warningsCount - kMaxWarnings) +
+           " more similar warnings...>");
   } else if (!config->unexportedSymbols.empty()) {
     parallelForEach(symtab->getSymbols(), [](Symbol *sym) {
       if (auto *defined = dyn_cast<Defined>(sym))
         if (config->unexportedSymbols.match(defined->getName()))
           defined->privateExtern = true;
     });
   }
 }

lld/test/MachO/export-options.s

 # REQUIRES: x86
 
 # RUN: rm -rf %t; split-file %s %t
 # RUN: echo "" | llvm-mc -filetype=obj -triple=x86_64-apple-macos -o %t/empty.o
 # RUN: llvm-mc -filetype=obj -triple=x86_64-apple-macos %t/default.s -o %t/default.o
 # RUN: llvm-mc -filetype=obj -triple=x86_64-apple-macos %t/lazydef.s -o %t/lazydef.o
+# RUN: llvm-mc -filetype=obj -triple=x86_64-apple-macos %t/too-many-warnings.s -o %t/too-many-warnings.o
 # RUN: llvm-ar --format=darwin rcs %t/lazydef.a %t/lazydef.o
 
 ## Check that mixing exported and unexported symbol options yields an error
 # RUN: not %lld -dylib %t/default.o -o /dev/null \
 # RUN:         -exported_symbol a -unexported_symbol b 2>&1 | \
 # RUN:     FileCheck --check-prefix=CONFLICT %s
 #
 # RUN: not %lld -dylib %t/default.o -o /dev/null \

 
 ## -unexported_symbol will not make us re-export symbols in dylibs.
 # RUN: %lld -dylib %t/default.o -o %t/libdefault.dylib
 # RUN: %lld -dylib %t/empty.o %t/libdefault.dylib -unexported_symbol _tlv \
 # RUN:   -o %t/unexport-dylib
 # RUN: llvm-objdump --macho --exports-trie %t/unexport-dylib | FileCheck %s \
 # RUN:   --check-prefix=EMPTY-TRIE
 
+## Check that warnings are truncated to the first 3 only.
+# RUN: %no-fatal-warnings-lld -dylib %t/too-many-warnings.o -o %t/too-many.out \
+# RUN:         -exported_symbol "_private_extern*" 2>&1 | \
+# RUN:     FileCheck --check-prefix=TRUNCATE %s
+
+# TRUNCATE: warning: cannot export hidden symbol _private_extern{{.+}}
+# TRUNCATE: warning: cannot export hidden symbol _private_extern{{.+}}
+# TRUNCATE: warning: cannot export hidden symbol _private_extern{{.+}}
+# TRUNCATE: warning: <... 7 more similar warnings...>
+# TRUNCATE-EMPTY:
+
 #--- default.s
 
 .globl _keep_globl, _hide_globl, _tlv
 _keep_globl:
   retq
 _hide_globl:
   retq
 .private_extern _private_extern
_foo:
   retq
 
 #--- weak-private-extern.s
 .global _foo
 .weak_definition _foo
 .private_extern _foo
 _foo:
   retq
+
+#--- too-many-warnings.s
+.private_extern _private_extern1
+.private_extern _private_extern2
+.private_extern _private_extern3
+.private_extern _private_extern4
+.private_extern _private_extern5
+.private_extern _private_extern6
+.private_extern _private_extern7
+.private_extern _private_extern8
+.private_extern _private_extern9
+.private_extern _private_extern10
+
+_private_extern1:
+  retq
+
+_private_extern2:
+  retq
+
+_private_extern3:
+  retq
+
+_private_extern4:
+  retq
+
+_private_extern5:
+  retq
+
+_private_extern6:
+  retq
+
+_private_extern7:
+  retq
+
+_private_extern8:
+  retq
+
+_private_extern9:
+  retq
+
+_private_extern10:
+  retq