This is an archive of the discontinued LLVM Phabricator instance.

[Clang][AArch64][Darwin] Enable GlobalISel by default for Darwin ARM64 platforms.
ClosedPublic

Authored by aemerson on Nov 2 2022, 10:07 AM.

Diff Detail

Event Timeline

aemerson created this revision.Nov 2 2022, 10:07 AM
Herald added a project: Restricted Project. · View Herald TranscriptNov 2 2022, 10:07 AM
aemerson requested review of this revision.Nov 2 2022, 10:07 AM
Herald added a project: Restricted Project. · View Herald TranscriptNov 2 2022, 10:07 AM
Herald added subscribers: MaskRay, wdng. · View Herald Transcript

maybe link back to the discourse thread in the commit message?

arsenm added inline comments.Nov 2 2022, 10:14 AM
clang/lib/Driver/ToolChains/Clang.cpp
7224–7225

Why abort=0? I can understand abort=1 or 2

arsenm added inline comments.Nov 2 2022, 10:16 AM
clang/lib/Driver/ToolChains/Darwin.cpp
388

Actually I'm confused why we're duplicating this logic in both these places

aemerson added inline comments.Nov 2 2022, 10:48 AM
clang/lib/Driver/ToolChains/Clang.cpp
7224–7225

Abort=1 would mean the compiler crashes on fallback, and 2 would mean we emit diagnostics, neither of which we want for an on-by-default configuration.

clang/lib/Driver/ToolChains/Darwin.cpp
388

This one is for adding linker args which we need for LTO builds.

aemerson edited the summary of this revision. (Show Details)Nov 2 2022, 10:48 AM

A release note?

aemerson updated this revision to Diff 472753.Nov 2 2022, 2:03 PM

Add a clang release note.

paquette accepted this revision.Nov 2 2022, 2:09 PM

If there's nothing else to add wrt telling people what's changed, I think this looks good?

This revision is now accepted and ready to land.Nov 2 2022, 2:09 PM
fhahn added a subscriber: fhahn.Nov 9 2022, 2:19 AM

It looks like GISel crashes when building llvm-test-suite with -O3 on ARM64. If it isn't trivial to fix we should probably revert the patch to bring things back to green.

Reproducer:

; llc  -global-isel -O3
target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
target triple = "arm64-apple-macosx"

define void @widget(ptr %arg, i1 %arg1) {
bb:
  br i1 %arg1, label %bb2, label %bb3

bb2:                                              ; preds = %bb
  %tmp = icmp slt i32 0, 0
  br i1 %tmp, label %bb3, label %bb9

bb3:                                              ; preds = %bb2, %bb
  %tmp4 = phi i32 [ 1, %bb ], [ 0, %bb2 ]
  %tmp5 = zext i32 %tmp4 to i64
  br label %bb6

bb6:                                              ; preds = %bb6, %bb3
  %tmp7 = getelementptr i32, ptr %arg, i64 %tmp5
  %tmp8 = getelementptr i32, ptr %tmp7, i64 4
  store <4 x i32> zeroinitializer, ptr %tmp8, align 4
  br label %bb6

bb9:                                              ; preds = %bb2
  ret void
}

https://llvm.godbolt.org/z/d5oTWxvaG

nikic added a subscriber: nikic.Nov 9 2022, 2:22 AM

I'd like to object to enabling this via the frontend. This means that non-clang frontends will now use a non-default configuration that is not extensively tested by upstream anymore.

If you don't want to change tests, you can adjust the RUN lines to explicitly disable globalisel.

I believe this patch might be the cause of some assertion failures we are seeing in a two stage lto build, as it's only happening for arm64-darwin. See https://ci.chromium.org/ui/p/fuchsia/builders/prod/clang-mac-xarm64/b8798097341668696785/overview and it's corresponding build logs https://logs.chromium.org/logs/fuchsia/buildbucket/cr-buildbucket/8798097341668696785/+/u/clang/build/stdout

[4060/4274](20) Linking CXX executable bin/clang-refactor
FAILED: bin/clang-refactor 
: && /opt/s/w/ir/x/w/staging/llvm_build/./bin/clang++ --target=arm64-apple-darwin --sysroot=/opt/s/w/ir/cache/macos_sdk/XCode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk -stdlib=libc++ -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wno-comment -Wstring-conversion -Wmisleading-indentation -Wctad-maybe-unsupported -fdiagnostics-color -flto -ffile-prefix-map=/opt/s/w/ir/x/w/staging/llvm_build/tools/clang/stage2-bins=../staging/llvm_build/tools/clang/stage2-bins -ffile-prefix-map=/opt/s/w/ir/x/w/llvm-llvm-project/= -no-canonical-prefixes -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG -isysroot /opt/s/w/ir/cache/macos_sdk/XCode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk -mmacosx-version-min=10.13 -Wl,-headerpad_max_install_names -nostdlib++ /opt/s/w/ir/x/w/staging/llvm_build/lib/libc++.a -stdlib=libc++ -static-libstdc++ -fno-pie -fuse-ld=lld -Wl,--color-diagnostics -flto -Wl,-lto_library -Wl,/opt/s/w/ir/x/w/staging/llvm_build/./lib/libLTO.dylib    -Wl,-dead_strip tools/clang/tools/clang-refactor/CMakeFiles/clang-refactor.dir/ClangRefactor.cpp.o tools/clang/tools/clang-refactor/CMakeFiles/clang-refactor.dir/TestSupport.cpp.o -o bin/clang-refactor  -Wl,-rpath,@loader_path/../lib  lib/libLLVMOption.a  lib/libLLVMSupport.a  lib/libclangAST.a  lib/libclangBasic.a  lib/libclangFormat.a  lib/libclangFrontend.a  lib/libclangLex.a  lib/libclangRewrite.a  lib/libclangSerialization.a  lib/libclangTooling.a  lib/libclangToolingCore.a  lib/libclangToolingRefactoring.a  lib/libclangIndex.a  lib/libclangFormat.a  lib/libclangToolingInclusions.a  lib/libclangFrontend.a  lib/libclangDriver.a  lib/libLLVMWindowsDriver.a  lib/libLLVMOption.a  lib/libclangParse.a  lib/libclangSerialization.a  lib/libclangSema.a  lib/libclangEdit.a  lib/libclangAnalysis.a  lib/libclangASTMatchers.a  lib/libclangAST.a  lib/libLLVMFrontendOpenMP.a  lib/libLLVMScalarOpts.a  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a  lib/libLLVMProfileData.a  lib/libLLVMSymbolize.a  lib/libLLVMDebugInfoPDB.a  lib/libLLVMDebugInfoMSF.a  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMObject.a  lib/libLLVMMCParser.a  lib/libLLVMMC.a  lib/libLLVMDebugInfoCodeView.a  lib/libLLVMIRReader.a  lib/libLLVMAsmParser.a  lib/libLLVMTextAPI.a  lib/libclangSupport.a  lib/libLLVMFrontendHLSL.a  lib/libLLVMBitReader.a  lib/libclangToolingCore.a  lib/libclangRewrite.a  lib/libclangLex.a  lib/libclangBasic.a  lib/libLLVMCore.a  lib/libLLVMBinaryFormat.a  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMSupport.a  -lm  /opt/s/w/ir/x/w/staging/zlib_install/lib/libz.a  /opt/s/w/ir/x/w/staging/zstd_install_prefix/lib/libzstd.a  lib/libLLVMDemangle.a && :
clang++: warning: argument unused during compilation: '-static-libstdc++' [-Wunused-command-line-argument]
Assertion failed: (BitWidth == RHS.BitWidth && "Bit widths must be the same"), function operator+=, file /opt/s/w/ir/x/w/llvm-llvm-project/llvm/lib/Support/APInt.cpp, line 189.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.	Program arguments: /opt/s/w/ir/x/w/staging/llvm_build/./bin/ld64.lld -demangle -dynamic -arch arm64 -platform_version macos 11.0.0 11.0 -no_pie -mllvm -global-isel -mllvm -global-isel-abort=0 -syslibroot /opt/s/w/ir/cache/macos_sdk/XCode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk -o bin/clang-refactor -headerpad_max_install_names /opt/s/w/ir/x/w/staging/llvm_build/lib/libc++.a --color-diagnostics -lto_library /opt/s/w/ir/x/w/staging/llvm_build/./lib/libLTO.dylib -dead_strip tools/clang/tools/clang-refactor/CMakeFiles/clang-refactor.dir/ClangRefactor.cpp.o tools/clang/tools/clang-refactor/CMakeFiles/clang-refactor.dir/TestSupport.cpp.o -rpath @loader_path/../lib lib/libLLVMOption.a lib/libLLVMSupport.a lib/libclangAST.a lib/libclangBasic.a lib/libclangFormat.a lib/libclangFrontend.a lib/libclangLex.a lib/libclangRewrite.a lib/libclangSerialization.a lib/libclangTooling.a lib/libclangToolingCore.a lib/libclangToolingRefactoring.a lib/libclangIndex.a lib/libclangFormat.a lib/libclangToolingInclusions.a lib/libclangFrontend.a lib/libclangDriver.a lib/libLLVMWindowsDriver.a lib/libLLVMOption.a lib/libclangParse.a lib/libclangSerialization.a lib/libclangSema.a lib/libclangEdit.a lib/libclangAnalysis.a lib/libclangASTMatchers.a lib/libclangAST.a lib/libLLVMFrontendOpenMP.a lib/libLLVMScalarOpts.a lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMProfileData.a lib/libLLVMSymbolize.a lib/libLLVMDebugInfoPDB.a lib/libLLVMDebugInfoMSF.a lib/libLLVMDebugInfoDWARF.a lib/libLLVMObject.a lib/libLLVMMCParser.a lib/libLLVMMC.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMTextAPI.a lib/libclangSupport.a lib/libLLVMFrontendHLSL.a lib/libLLVMBitReader.a lib/libclangToolingCore.a lib/libclangRewrite.a lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCore.a lib/libLLVMBinaryFormat.a lib/libLLVMRemarks.a lib/libLLVMBitstreamReader.a lib/libLLVMSupport.a -lm /opt/s/w/ir/x/w/staging/zlib_install/lib/libz.a /opt/s/w/ir/x/w/staging/zstd_install_prefix/lib/libzstd.a lib/libLLVMDemangle.a -lSystem /opt/s/w/ir/x/w/staging/llvm_build/./lib/clang/16.0.0/lib/darwin/libclang_rt.osx.a
1.	Running pass 'Function Pass Manager' on module 'ld-temp.o'.
2.	Running pass 'AArch64PreLegalizerCombiner' on function '@_ZN5clang4Sema21CheckTemplateArgumentEPNS_9NamedDeclERNS_19TemplateArgumentLocES2_NS_14SourceLocationES5_jRN4llvm15SmallVectorImplINS_16TemplateArgumentEEESA_NS0_25CheckTemplateArgumentKindE'
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  ld64.lld                 0x000000010a3c929e llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 46
1  ld64.lld                 0x000000010a3c80e8 llvm::sys::RunSignalHandlers() + 248
2  ld64.lld                 0x000000010a3c98ee SignalHandler(int) + 270
3  libsystem_platform.dylib 0x00007fff208d2d7d _sigtramp + 29
4  libsystem_platform.dylib 000000000000000000 _sigtramp + 18446603339970040480
5  libsystem_c.dylib        0x00007fff207e2406 abort + 125
6  libsystem_c.dylib        0x00007fff207e17d8 err + 0
7  ld64.lld                 0x000000010a33d9ff llvm::APInt::operator+=(llvm::APInt const&) + 223
8  ld64.lld                 0x000000010b23cc5f void std::__2::__function::__policy_invoker<void (llvm::MachineIRBuilder&)>::__call_impl<std::__2::__function::__default_alloc_func<llvm::CombinerHelper::matchReassocFoldConstantsInSubTree(llvm::GPtrAdd&, llvm::MachineInstr*, llvm::MachineInstr*, std::__2::function<void (llvm::MachineIRBuilder&)>&)::$_0, void (llvm::MachineIRBuilder&)>>(std::__2::__function::__policy_storage const*, llvm::MachineIRBuilder&) + 143
9  ld64.lld                 0x000000010af99e50 (anonymous namespace)::AArch64GenPreLegalizerCombinerHelper::tryCombineAll(llvm::GISelChangeObserver&, llvm::MachineInstr&, llvm::MachineIRBuilder&) const + 9952
10 ld64.lld                 0x000000010af95030 (anonymous namespace)::AArch64PreLegalizerCombinerInfo::combine(llvm::GISelChangeObserver&, llvm::MachineInstr&, llvm::MachineIRBuilder&) const + 144
11 ld64.lld                 0x000000010b214107 llvm::Combiner::combineMachineInstrs(llvm::MachineFunction&, llvm::GISelCSEInfo*) + 3015
12 ld64.lld                 0x000000010af94a7d (anonymous namespace)::AArch64PreLegalizerCombiner::runOnMachineFunction(llvm::MachineFunction&) + 781
13 ld64.lld                 0x000000010b6ef714 llvm::MachineFunctionPass::runOnFunction(llvm::Function&) + 660
14 ld64.lld                 0x000000010d18ffd2 llvm::FPPassManager::runOnFunction(llvm::Function&) + 1618
15 ld64.lld                 0x000000010d198624 llvm::FPPassManager::runOnModule(llvm::Module&) + 68
16 ld64.lld                 0x000000010d190a7c llvm::legacy::PassManagerImpl::run(llvm::Module&) + 1964
17 ld64.lld                 0x000000010b5cc32f codegen(llvm::lto::Config const&, llvm::TargetMachine*, std::__2::function<llvm::Expected<std::__2::unique_ptr<llvm::CachedFileStream, std::__2::default_delete<llvm::CachedFileStream>>> (unsigned int)>, unsigned int, llvm::Module&, llvm::ModuleSummaryIndex const&) + 1727
18 ld64.lld                 0x000000010b5cb43c llvm::lto::backend(llvm::lto::Config const&, std::__2::function<llvm::Expected<std::__2::unique_ptr<llvm::CachedFileStream, std::__2::default_delete<llvm::CachedFileStream>>> (unsigned int)>, unsigned int, llvm::Module&, llvm::ModuleSummaryIndex&) + 364
19 ld64.lld                 0x000000010b5bd8b4 llvm::lto::LTO::runRegularLTO(std::__2::function<llvm::Expected<std::__2::unique_ptr<llvm::CachedFileStream, std::__2::default_delete<llvm::CachedFileStream>>> (unsigned int)>) + 2068
20 ld64.lld                 0x000000010b5bccf2 llvm::lto::LTO::run(std::__2::function<llvm::Expected<std::__2::unique_ptr<llvm::CachedFileStream, std::__2::default_delete<llvm::CachedFileStream>>> (unsigned int)>, std::__2::function<llvm::Expected<std::__2::function<llvm::Expected<std::__2::unique_ptr<llvm::CachedFileStream, std::__2::default_delete<llvm::CachedFileStream>>> (unsigned int)>> (unsigned int, llvm::StringRef)>) + 1122
21 ld64.lld                 0x000000010a6fcade lld::macho::BitcodeCompiler::compile() + 830
22 ld64.lld                 0x000000010a6cc840 lld::macho::link(llvm::ArrayRef<char const*>, llvm::raw_ostream&, llvm::raw_ostream&, bool, bool) + 35856
23 ld64.lld                 0x000000010a30d6b0 lldMain(int, char const**, llvm::raw_ostream&, llvm::raw_ostream&, bool) + 1872
24 ld64.lld                 0x000000010a30cdad lld_main(int, char**) + 285
25 libdyld.dylib            0x00007fff208a8f3d start + 1
clang++: error: unable to execute command: Abort trap: 6
clang++: error: linker command failed due to signal (use -v to see invocation)
[4061/4274](19) Linking CXX executable bin/llvm-lipo

It looks like GISel crashes when building llvm-test-suite with -O3 on ARM64. If it isn't trivial to fix we should probably revert the patch to bring things back to green.

Reproducer:

; llc  -global-isel -O3
target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
target triple = "arm64-apple-macosx"

define void @widget(ptr %arg, i1 %arg1) {
bb:
  br i1 %arg1, label %bb2, label %bb3

bb2:                                              ; preds = %bb
  %tmp = icmp slt i32 0, 0
  br i1 %tmp, label %bb3, label %bb9

bb3:                                              ; preds = %bb2, %bb
  %tmp4 = phi i32 [ 1, %bb ], [ 0, %bb2 ]
  %tmp5 = zext i32 %tmp4 to i64
  br label %bb6

bb6:                                              ; preds = %bb6, %bb3
  %tmp7 = getelementptr i32, ptr %arg, i64 %tmp5
  %tmp8 = getelementptr i32, ptr %tmp7, i64 4
  store <4 x i32> zeroinitializer, ptr %tmp8, align 4
  br label %bb6

bb9:                                              ; preds = %bb2
  ret void
}

https://llvm.godbolt.org/z/d5oTWxvaG

There are a few minor issues here. There are a handle of g_ptr_add combines which are creating offsets with the non-canonical size, and then others that don't consider that the size could be different

hans added a subscriber: hans.Nov 10 2022, 10:52 AM

Looks like there are a few issues flagged already. Is someone reverting, or what's the plan?

I'd like to object to enabling this via the frontend. This means that non-clang frontends will now use a non-default configuration that is not extensively tested by upstream anymore.

If you don't want to change tests, you can adjust the RUN lines to explicitly disable globalisel.

+1, even in-tree if you use lld/LTO you won't be using globalisel

Sorry, for some reason I hadn’t received any emails about this so I only noticed now. I’ll look into the issues and come back once the issues are resolved.

thakis added a subscriber: thakis.Nov 14 2022, 9:02 AM

Heads-up: This very visibly broke web page rendering in Chromium (but only when using LTO) (https://crbug.com/1383873) . Not clear yet if this exposed UB on our end or not.

One thing that we did notice is that adding -fno-global-isel to cflags and ldflags only had an effect when manually blowing away the thinlto cache. Maybe whatever decides when to invalidate the thinlto cache needs to learn to invalidate it when the -global-isel flag changes?

clang/lib/Driver/ToolChains/Clang.cpp
7224–7225

Any reason why we pass different flags to ld's LTO than to cc1?

One thing that we did notice is that adding -fno-global-isel to cflags and ldflags only had an effect when manually blowing away the thinlto cache. Maybe whatever decides when to invalidate the thinlto cache needs to learn to invalidate it when the -global-isel flag changes?

Looks like we'll have to do D134013 for the Mach-O and COFF lld ports as well. I'll make a CL for that. Thanks to @hans for pointing that out.

thakis added a comment.Dec 7 2022, 9:23 AM

Heads-up: This very visibly broke web page rendering in Chromium (but only when using LTO) (https://crbug.com/1383873) . Not clear yet if this exposed UB on our end or not.

After looking at this for a while, this does look like a miscompile. I filed https://github.com/llvm/llvm-project/issues/59376 for this, with a somewhat reduced repro (down to a single TU, but the TU is still fairly large.)