ar_to_bc.sh: Ignore non-bitcode files in archives

[MaskRay]

Summary

The script uses llvm-link to link LLVM bitcode files.
5426da8ffa4a6d55adab21026ce6ebe8f1cc6ef2 used -DLLVM_DISABLE_ASSEMBLY_FILES=ON
to ignore object files compiled from lib/Support/BLAKE3/*.S.

A better approach (which fits Bazel better) is to ignore non-bitcode files.

[akyrtzi]

Should this patch also remove -DLLVM_DISABLE_ASSEMBLY_FILES=ON from symbolizer/scripts/build_symbolizer.sh?

[MaskRay]

remove -DLLVM_DISABLE_ASSEMBLY_FILES=ON

[MaskRay]

In D126728#3548617, @akyrtzi wrote:

Should this patch also remove -DLLVM_DISABLE_ASSEMBLY_FILES=ON from symbolizer/scripts/build_symbolizer.sh?

Thanks for the quick review:) I think it's a good idea. I suspect LLVM_DISABLE_ASSEMBLY_FILES can be removed, but that can be a separate change.

[akyrtzi]

It's not clear to me what symbolizer/scripts/ar_to_bc.sh is doing exactly, will there not be an issue (e.g. linker error?) when it's trying to link the object files while the .s derived ones were excluded?

[MaskRay]

In D126728#3548686, @akyrtzi wrote:

It's not clear to me what symbolizer/scripts/ar_to_bc.sh is doing exactly, will there not be an issue (e.g. linker error?) when it's trying to link the object files while the .s derived ones were excluded?

Seems that it is used by compiler-rt's "internal symbolizer". But I am unclear how it is plugged into the CMake build system.

Using clang -flto plus llvm-link is a way to use LLVMSymbolize without exposing llvm:: symbols in sanitizer runtime files.

[MatzeB]

compiler-rt/lib/sanitizer_common/symbolizer/scripts/ar_to_bc.sh

33–38

I tracked down a build failure here, to the file utility reporting LLVM bitcode instead of the expected LLVM IR bitcode :-(

bash-5.0$ file --version
file-5.11
magic file from /etc/magic:/usr/share/misc/magic

[MatzeB]

compiler-rt/lib/sanitizer_common/symbolizer/scripts/ar_to_bc.sh

33–38

I put up a way to avoid this whole script in D128729

compiler-rt/lib/sanitizer_common/symbolizer/scripts/ar_to_bc.sh

 SCRATCH_DIR=$(mktemp -d)
 ln -s $INPUTS $SCRATCH_DIR/
 
 pushd $SCRATCH_DIR
 
 for INPUT in *; do
   for OBJ in $($AR t $INPUT); do
     $AR x $INPUT $OBJ
+    if [[ $(file $OBJ) =~ 'LLVM IR bitcode' ]]; then
       mv -f $OBJ $(basename $INPUT).$OBJ
+    else
+      # Skip $OBJ which may come from an assembly file (e.g. Support/BLAKE3/*.S).
+      rm -f $OBJ
+    fi
   done
 done
 
 $LINK *.o -o $OUTPUT
 
 rm -rf $SCRATCH_DIR

compiler-rt/lib/sanitizer_common/symbolizer/scripts/build_symbolizer.sh

cmake -GNinja \
     -DCMAKE_C_COMPILER=$CC \
     -DCMAKE_CXX_COMPILER=$CXX \
     -DCMAKE_C_FLAGS_RELEASE="${LLVM_FLAGS}" \
     -DCMAKE_CXX_FLAGS_RELEASE="${LLVM_FLAGS}" \
     -DLLVM_TABLEGEN=$TBLGEN \
     -DLLVM_ENABLE_ZLIB=ON \
     -DLLVM_ENABLE_TERMINFO=OFF \
     -DLLVM_ENABLE_THREADS=OFF \
-    -DLLVM_DISABLE_ASSEMBLY_FILES=ON \
   $LLVM_SRC
 fi
 cd ${LLVM_BUILD}
 ninja LLVMSymbolize LLVMObject LLVMBinaryFormat LLVMDebugInfoDWARF LLVMSupport LLVMDebugInfoPDB LLVMDebuginfod LLVMMC LLVMDemangle LLVMTextAPI
 
 cd ${BUILD_DIR}
 rm -rf ${SYMBOLIZER_BUILD}
 mkdir ${SYMBOLIZER_BUILD}