[ELF] Create Undefined for local STT_SECTION symbol if the section is discarded

[MaskRay]

Summary

This allows us to catch some errors when there are relocations to such
discarded sections (e.g. PR41693). Updated comdat-discarded-error.s
checks we emit friendly error message.

For relocatable-eh-frame.s, ld.lld -r a.o a.o will now error
"STT_SECTION symbol should be defined" because the section .eh_frame
refers to is now an Undefined instead of a Defined.
So I have to change error() to warn() to retain the output.

[MaskRay]

Update

[MaskRay]

Rebase

[MaskRay]

🤪

[ruiu]

LGTM

[phosek]

We're now getting a lot of STT_SECTION symbol should be defined warnings when building libFuzzer: https://logs.chromium.org/logs/fuchsia/buildbucket/cr-buildbucket.appspot.com/8911634509805270864/+/steps/clang/0/steps/build/0/stdout. The libFuzzer uses lld to produce a single relocatable file with libc++ linked in: https://github.com/llvm/llvm-project/blob/master/compiler-rt/lib/fuzzer/CMakeLists.txt#L132. I haven't yet figured out what's going on (the warning could be also a little more specific, e.g. which symbol and which section). Do you have any idea what's going on?

[phosek]

https://storage.googleapis.com/fuchsia-build/reproducer/fuzzer.tar.bz2 is the reproducer.

[MaskRay]

In D61583#1528517, @phosek wrote:

https://storage.googleapis.com/fuchsia-build/reproducer/fuzzer.tar.bz2 is the reproducer.

Thanks for the reproduce. I think this error (added in rLLD319404) should probably be suppressed for non-alloc sections. I'll post a patch.

[MaskRay]

In D61583#1528522, @MaskRay wrote:

In D61583#1528517, @phosek wrote:

https://storage.googleapis.com/fuchsia-build/reproducer/fuzzer.tar.bz2 is the reproducer.

Thanks for the reproduce. I think this error (added in rLLD319404) should probably be suppressed for non-alloc sections. I'll post a patch.

D62840

[MaskRay]

@sfertile I'm confused with the recent revert and revert of revert.. The problem this change caused should have been fixed by D62840. Did you find other problems? (The warn() can be deleted if you suspect there is something wrong.)

[sfertile]

In D61583#1533390, @MaskRay wrote:

@sfertile I'm confused with the recent revert and revert of revert.. The problem this change caused should have been fixed by D62840. Did you find other problems? (The warn() can be deleted if you suspect there is somethin

The multistage power-pc build bot is broken

69.906 [3746/40/441] Linking CXX executable bin/llvm-tblgen
FAILED: bin/llvm-tblgen 
: && /home/buildbots/ppc64le-lld-multistage-test/ppc64le-lld-multistage-test/install/stage1/bin/clang++  -fPIC -fvisibility-inlines-hidden -Werror -Werror=date-time -Werror=unguarded-availability-new -std=c++11 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -O3  -fuse-ld=lld -Wl,--color-diagnostics -Wl,-allow-shlib-undefined    -Wl,-O3 -Wl,--gc-sections utils/TableGen/CMakeFiles/llvm-tblgen.dir/AsmMatcherEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/AsmWriterEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/AsmWriterInst.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/Attributes.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CallingConvEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeEmitterGen.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenDAGPatterns.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenHwModes.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenInstruction.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenMapTable.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenRegisters.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenSchedule.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenTarget.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DAGISelEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DAGISelMatcherEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DAGISelMatcherGen.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DAGISelMatcherOpt.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DAGISelMatcher.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DFAPacketizerEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/DisassemblerEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/ExegesisEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/FastISelEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/FixedLenDecoderEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/GlobalISelEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/InfoByHwMode.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/InstrInfoEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/InstrDocsEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/IntrinsicEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/OptParserEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/PredicateExpander.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/PseudoLoweringEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/RISCVCompressInstEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/RegisterBankEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/RegisterInfoEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/SDNodeProperties.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/SearchableTableEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/SubtargetEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/SubtargetFeatureInfo.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/TableGen.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/Types.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/X86DisassemblerTables.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/X86EVEX2VEXTablesEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/X86FoldTablesEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/X86ModRMFilters.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/X86RecognizableInstr.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/WebAssemblyDisassemblerEmitter.cpp.o utils/TableGen/CMakeFiles/llvm-tblgen.dir/CTagsEmitter.cpp.o  -o bin/llvm-tblgen  -Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMSupport.a lib/libLLVMTableGen.a -lpthread lib/libLLVMSupport.a -lz -lrt -ldl -ltinfo -lpthread -lm lib/libLLVMDemangle.a && :
ld.lld: error: relocation refers to a discarded section: .rodata._ZNK4llvm3MVT13getSizeInBitsEv
>>> defined in utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenRegisters.cpp.o
>>> referenced by CodeGenRegisters.cpp
>>>               utils/TableGen/CMakeFiles/llvm-tblgen.dir/CodeGenRegisters.cpp.o:(.toc+0x0)

ld.lld: error: relocation refers to a discarded section: .rodata._ZNK4llvm3MVT13getSizeInBitsEv
>>> defined in utils/TableGen/CMakeFiles/llvm-tblgen.dir/GlobalISelEmitter.cpp.o
>>> referenced by GlobalISelEmitter.cpp
>>>               utils/TableGen/CMakeFiles/llvm-tblgen.dir/GlobalISelEmitter.cpp.o:(.toc+0x20)

ld.lld: error: relocation refers to a discarded section: .rodata._ZNK4llvm3MVT20getVectorElementTypeEv
>>> defined in utils/TableGen/CMakeFiles/llvm-tblgen.dir/GlobalISelEmitter.cpp.o
>>> referenced by GlobalISelEmitter.cpp
>>>               utils/TableGen/CMakeFiles/llvm-tblgen.dir/GlobalISelEmitter.cpp.o:(.toc+0x28)

ld.lld: error: relocation refers to a discarded section: .rodata._ZNK4llvm3MVT20getVectorElementTypeEv
>>> defined in utils/TableGen/CMakeFiles/llvm-tblgen.dir/IntrinsicEmitter.cpp.o
>>> referenced by IntrinsicEmitter.cpp
>>>               utils/TableGen/CMakeFiles/llvm-tblgen.dir/IntrinsicEmitter.cpp.o:(.toc+0x38)

ld.lld: error: relocation refers to a discarded section: .rodata._ZNK4llvm3MVT13getSizeInBitsEv
>>> defined in utils/TableGen/CMakeFiles/llvm-tblgen.dir/IntrinsicEmitter.cpp.o
>>> referenced by IntrinsicEmitter.cpp
>>>               utils/TableGen/CMakeFiles/llvm-tblgen.dir/IntrinsicEmitter.cpp.o:(.toc+0x40)

ld.lld: error: relocation refers to a discarded section: .rodata._ZN4llvm4yaml11needsQuotesENS_9StringRefE
>>> defined in lib/libLLVMSupport.a(Timer.cpp.o)
>>> referenced by Timer.cpp
>>>               Timer.cpp.o:(.toc+0x30) in archive lib/libLLVMSupport.a
clang-9: error: linker command failed with exit code 1 (use -v to see invocation)

The build monitor narrowed it down to this patch so I was pulling it to turn the bot back to green until we can figure out why this patch only causes a problem on PowerPC. I don't think there is anything necessarily wrong with the warning. but I do think we need to see if we are discarding the same sections before/after this patch. I'm not familiar with when/why we discard sections but it seems quite odd to me we are discarding a section with relocations referencing it.

[sfertile]

@MaskRay Are you planning on pulling this? The buildbot is still red and there have been numerous Power related changes going in with no buildbot coverage.

ELF/InputFiles.cpp

template <class ELFT> Symbol *ObjFile<ELFT>::createSymbol(const Elf_Sym *Sym) {
   if (Binding == STB_LOCAL) {
     if (Sym->getType() == STT_FILE)
       SourceFile = CHECK(Sym->getName(this->StringTable), this);
 
     if (this->StringTable.size() <= Sym->st_name)
       fatal(toString(this) + ": invalid symbol name offset");
 
     StringRefZ Name = this->StringTable.data() + Sym->st_name;
-    if (Sym->st_shndx == SHN_UNDEF)
+    if (Sym->st_shndx == SHN_UNDEF || Sec == &InputSection::Discarded)
       return make<Undefined>(this, Name, Binding, StOther, Type);
 
     return make<Defined>(this, Name, Binding, StOther, Type, Value, Size, Sec);
   }
 
   StringRef Name = CHECK(Sym->getName(this->StringTable), this);
 
   switch (Sym->st_shndx) {