recommend using llvm-ar when finding undefined references and empty archives

[inglorion]

Summary

When we perform LTO builds with a version of ar that does not
understand LLVM bitcode objects, we end up with undefined references,
because our archive files do not list the bitcode symbols in their
indices. The error messages do not make it clear what the real problem
is. This change adds a note that points out the likely problem and
solution. It is similar in spirit to r282633, but aims to avoid false
positives by only triggering when we see both undefined references and
archives without symbols in their indices.

Fixes PR32281.

[inglorion]

ELF/Error.cpp

87 ↗

(On Diff #91954)

Since we can't lock a mutex we already hold, exitLld must be called without holding the lock on Mu. This makes the structure of the code here quite unfortunate. Is there a more elegant way to get this to work?

[ruiu]

I think this patch changes too many files. You could simplify this by constructing an error message string in ArchiveFile::parse (to briefly say something like that "foo.a: no symbol table found. If you are attempting LTO, you need to use llvm-ar instead of ar to create static archives.") and store that to Config. So the code to add parse would be something like

Config->EmptyArchiveErrors.push_back(toString(*this) + ":  symbol table missing ...");

where Config->EmptyArchiveErrors is of std::vector<std::string>.

And then print out these messages in reportUndefined.

[inglorion]

@ruiu: Yes, that would be a lot simpler. Let me explain why I did it the way I did it, and then I'll be happy to simplify it if you still feel that's better.

I thought about emitting the message from Relocations.cpp, but then I thought about ErrorLimit. If I emit the note about empty archives after the undefined symbols, it is quite likely that it would not be emitted because the error limit is reached before getting to that point. If I emit the note first, it's easily missed because there may be a whole bunch of undefined reference errors after it. Thinking about that some, I thought it would be most useful if the note is emitted just before lld exits - and that this is actually something that may be more generically useful. That's why I added support for this to Errors. Using this new mechanism, the note is not affected by ErrorLimit and is emitted last, where it should be most effective.

I also thought about reporting the names of the archives in which we found no or empty indices, but my guess is that there isn't a lot of value to that; the most important thing is that this situation may be affecting your build, and switching ar implementations will fix the problem if that's the case. Having the list of file names is more text on the screen, and more memory usage to carry them around, and I didn't feel the benefits outweigh the cost, so I went with a simple boolean.

[ruiu]

ErrorLimit is 20. Isn't it enough to print out both unresolved symbols and suspicious archive files, is it? If it is not enough, you can print out an error only for the first archive file that doesn't contain a symbol table.

[inglorion]

@ruiu, how is this?

[ruiu]

ELF/InputFiles.h

253

In this way, you need to reset the value in main because LLD should be able to be invoked more than once in the same process if it's used as a library. You want to store this boolean variable to Config.

ELF/Relocations.cpp

619

You only want to print out this message only once, no?

(nit: remove trailing "\n". I believe message() automatically appends that.)

[inglorion]

@ruiu's comments

[davide]

Reid (@rnk) pointed out this comes up during Chromium build so please make sure to verify the output is still reasonable with this patch (if I recall correctly libtool emits something similar (but it's more verbose) and the Chromium folks have to filter it out).

[inglorion]

Yes, I'll check Chromium builds once we're happy with the implementation.

[ruiu]

Let's try this. LGTM.

ELF/InputFiles.cpp

567

Format.

Let's eliminate the local varaible. You could replace NoSymbols with File->symbols().empty(). File->symbols() is cheap so you don't need to save time here.

ELF/Relocations.cpp

623

Remove the space before ".

[davide]

ELF/InputFiles.cpp

561–568

Why do you need two different flags?

567

should live on two different lines, no? (in general, clang-format).

ELF/Relocations.cpp

621

I'd say "the bitcode files".

[davide]

Other than the comments, this looks good

[inglorion]

ELF/InputFiles.cpp

561–568

By "two different flags" do you mean NoSymbols and ArchiveWithoutSymbolsSeen? I'll get rid of that by following ruiu's suggestion.

567

W.r.t. "two different lines": that's what I thought, too, but clang-format actually changed it to be one line. Shall I just go ahead and change it back to two lines?

ELF/Relocations.cpp

621

Hmm, yeah. I added the message because I ran into trouble with bitcode files, but it really is a more general situation, which is why I went with "object files" in the message. Is it ok to leave that as-is or shall I reword the message to something like "This can happen when creating archives with bitcode files (such as created by -flto) using a version of ar that does not know how to index those"?

[ruiu]

ELF/InputFiles.cpp

567

You want to make sure that you are using the LLVM style when formatting it with clang-format. Clang-format reflowed this line to use two lines for me.

[inglorion]

clang-format done right and removed NoSymbols local

[ruiu]

Still LGTM

[davide]

In D31011#704298, @ruiu wrote:

Still LGTM

Same.

ELF/InputFiles.h

private:
   bool Seen = false;
 };
 
 // An ArchiveFile object represents a .a file.
 class ArchiveFile : public InputFile {
 public:
   explicit ArchiveFile(MemoryBufferRef M) : InputFile(ArchiveKind, M) {}
   static bool classof(const InputFile *F) { return F->kind() == ArchiveKind; }
+  static bool archiveWithoutSymbolsSeen() { return ArchiveWithoutSymbolsSeen; }
   template <class ELFT> void parse();
 
   // Returns a memory buffer for a given symbol and the offset in the archive
   // for the member. An empty memory buffer and an offset of zero
   // is returned if we have already returned the same memory buffer.
   // (So that we don't instantiate same members more than once.)
   std::pair<MemoryBufferRef, uint64_t> getMember(const Archive::Symbol *Sym);
 
 private:
+  static bool ArchiveWithoutSymbolsSeen;
   std::unique_ptr<Archive> File;
   llvm::DenseSet<uint64_t> Seen;
 };
 
 class BitcodeFile : public InputFile {
 public:
   explicit BitcodeFile(MemoryBufferRef M);
   static bool classof(const InputFile *F) { return F->kind() == BitcodeKind; }

ELF/InputFiles.cpp

uint64_t getSectionLoadAddress(const object::SectionRef &Sec) const override {
     return static_cast<const ELFSectionRef &>(Sec).getOffset();
   }
   std::unique_ptr<LoadedObjectInfo> clone() const override {
     return std::unique_ptr<LoadedObjectInfo>();
   }
 };
 }
 
+bool ArchiveFile::ArchiveWithoutSymbolsSeen = false;
+
 Optional<MemoryBufferRef> elf::readFile(StringRef Path) {
   log(Path);
   auto MBOrErr = MemoryBuffer::getFile(Path);
   if (auto EC = MBOrErr.getError()) {
     error("cannot open " + Path + ": " + EC.message());
     return None;
   }
 
SymbolBody *elf::ObjectFile<ELFT>::createSymbolBody(const Elf_Sym *Sym) {
   }
 }
 
 template <class ELFT> void ArchiveFile::parse() {
   File = check(Archive::create(MB),
                MB.getBufferIdentifier() + ": failed to parse archive");
 
   // Read the symbol table to construct Lazy objects.
-  for (const Archive::Symbol &Sym : File->symbols())
+  bool NoSymbols = true;
+  for (const Archive::Symbol &Sym : File->symbols()) {
     Symtab<ELFT>::X->addLazyArchive(this, Sym);
+    NoSymbols = false;
+  }
+
+  if (NoSymbols) ArchiveWithoutSymbolsSeen = true;
 }
 
 // Returns a buffer pointing to a member file containing a given symbol.
 std::pair<MemoryBufferRef, uint64_t>
 ArchiveFile::getMember(const Archive::Symbol *Sym) {
   Archive::Child C =
       check(Sym->getMember(),
             "could not get the member for symbol " + Sym->getName());
 

ELF/Relocations.cpp

static void reportUndefined(SymbolBody &Sym, InputSectionBase &S,
   if (Config->UnresolvedSymbols == UnresolvedPolicy::IgnoreAll ||
       (Config->UnresolvedSymbols == UnresolvedPolicy::Ignore && CanBeExternal))
     return;
 
   std::string Msg = S.getLocation<ELFT>(Offset) + ": undefined symbol '" +
                     toString(Sym) + "'";
 
   if (Config->UnresolvedSymbols == UnresolvedPolicy::WarnAll ||
-      (Config->UnresolvedSymbols == UnresolvedPolicy::Warn && CanBeExternal))
+      (Config->UnresolvedSymbols == UnresolvedPolicy::Warn && CanBeExternal)) {
     warn(Msg);
-  else
+  } else {
     error(Msg);
+    if (ArchiveFile::archiveWithoutSymbolsSeen())
+      message("At least one archive listed no symbols in its index."
+              " This can happen when creating archives with a version"
+              " of ar that does not understand the object files in"
+              " the archive. For example, if you are using LLVM"
+              " bitcode objects (such as created by -flto), you may "
+              " need to use llvm-ar or GNU ar with a plugin.\n");
+  }
 }
 
 template <class RelTy>
 static std::pair<uint32_t, uint32_t>
 mergeMipsN32RelTypes(uint32_t Type, uint32_t Offset, RelTy *I, RelTy *E) {
   // MIPS N32 ABI treats series of successive relocations with the same offset
   // as a single relocation. The similar approach used by N64 ABI, but this ABI
   // packs all relocations into the single relocation record. Here we emulate

test/ELF/lto/archive-no-index.ll

+; REQUIRES: x86
+; Tests that we suggest that LTO symbols missing from an archive index
+; may be the cause of undefined references, but only if we both
+; encountered an empty archive index and undefined references (to prevent
+; noisy false alarms).
+
+; RUN: rm -fr %T/archive-no-index
+; RUN: mkdir %T/archive-no-index
+; RUN: llvm-as %S/Inputs/archive.ll -o %T/archive-no-index/f.o
+; RUN: llvm-ar cr %T/archive-no-index/libf.a
+; RUN: llvm-ar qS %T/archive-no-index/libf.a %T/archive-no-index/f.o
+; RUN: llvm-as %s -o %t.o
+; RUN: not ld.lld -emain -m elf_x86_64 %t.o -o %t %T/archive-no-index/libf.a \
+; RUN:     2>&1 | FileCheck --check-prefix=NOTE %s
+
+; RUN: llvm-ar crs %T/archive-no-index/libfs.a %T/archive-no-index/f.o
+; RUN: ld.lld -emain -m elf_x86_64 %t.o -o %t %T/archive-no-index/libf.a \
+; RUN:     %T/archive-no-index/libfs.a
+
+; RUN: llvm-as %S/Inputs/archive-3.ll -o %T/archive-no-index/foo.o
+; RUN: llvm-ar crs %T/archive-no-index/libfoo.a %T/archive-no-index/foo.o
+; RUN: not ld.lld -emain -m elf_x86_64 %t.o -o %t %T/archive-no-index/libfoo.a \
+; RUN:     2>&1 | FileCheck --check-prefix=NO-NOTE %s
+
+; NOTE: undefined symbol 'f'
+; NOTE: archive listed no symbols
+
+; NO-NOTE: undefined symbol 'f'
+; NO-NOTE-NOT: archive listed no symbols
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @f()
+
+define i32 @main() {
+  call void @f()
+  ret i32 0
+}