This is an archive of the discontinued LLVM Phabricator instance.

Lower `@llvm.global_dtors` using `__cxa_atexit` on MachO
ClosedPublic

Authored by yln on Mar 9 2022, 1:07 PM.

Details

Summary

For MachO, lower .llvm.global_dtors into .llvm_global_ctors with
__cxa_atexit calls to avoid emitting the deprecated __mod_term_func.

Reuse the existing WebAssemblyLowerGlobalDtors.cpp to accomplish this.

Diff Detail

Event Timeline

yln created this revision.Mar 9 2022, 1:07 PM
yln requested review of this revision.Mar 9 2022, 1:07 PM
Herald added a project: Restricted Project. · View Herald TranscriptMar 9 2022, 1:07 PM
yln added a comment.Mar 9 2022, 1:19 PM

I wanted to check in to get your feedback on the overall approach while I am working on adding tests for the behavior change.

On Darwin (MachO really) we want to transition away from the deprecated __mod_term_funcs section (relevant: D45578). However, currently @llvm.global_dtors, i.e., llvm::appendToGlobalDtors callers, are lowered to it.
Here I am trying to reuse an existing lowering pass for WebAssembly (which doesn't have a destructor section, D41211) to transform @llvm.global_dtors into functions registered as constructors (@llvm.global_ctors) that then call __cxa_atexit to register the destructor.

The overall approach here makes sense to me. The tests test/CodeGen/WebAssembly/global_dtors.ll and test/CodeGen/WebAssembly/lower-global-dtors.ll cover the main interesting corner cases that we've hit with this pass; they currently have wasm targets and wasm-specific CHECK lines, but could likely be ported to other targets.

@yln I like the approach here as it solves problem for all of LLVM, not just ASan specifically :). This is better than what I was going to do originally. I think the only missing there here are some tests.

yln updated this revision to Diff 414540.EditedMar 10 2022, 6:09 PM

Make pass compatible with new pass manager to ease testing with opt. Add and adapt tests. Add documentation. Make clang-format happy.

yln added inline comments.Mar 10 2022, 6:30 PM
llvm/lib/CodeGen/TargetPassConfig.cpp
898–902

Is TargetPassConfig::addIRPasses() the right place to hook this pass?

yln added inline comments.Mar 10 2022, 6:32 PM
llvm/include/llvm/Transforms/Utils/LowerGlobalDtors.h
2

Copypasta Eliminate Invoke instructions is fixed.

yln updated this revision to Diff 414556.Mar 10 2022, 6:55 PM

Declare preserved analyses in new pass format.

aralisza accepted this revision.Mar 11 2022, 11:04 AM
This revision is now accepted and ready to land.Mar 11 2022, 11:04 AM
delcypher accepted this revision.Mar 11 2022, 1:43 PM
delcypher added inline comments.
llvm/lib/CodeGen/TargetPassConfig.cpp
900

Random thought. Do we want to support the legacy way of calling destructors, rather than removing it entirely? If we were to do such a thing I'd suspect we'd guard using the legacy way on the OS deployment target.

Just to be clear. I'm happy with the patch the way it is. I'm just wondering if we should consider allowing the legacy way as well. I can't see an obvious use case for it because the new way should work on older OSs too but maybe there's a use case I haven't thought about?

yln added inline comments.Mar 11 2022, 2:45 PM
llvm/lib/CodeGen/TargetPassConfig.cpp
900

Having a way to explicitly request the old behavior sounds like a good idea. I will look into it.

arphaman accepted this revision.Mar 14 2022, 3:02 PM
yln updated this revision to Diff 415273.Mar 14 2022, 4:48 PM

Add support for an escape hatch to fallback to the old behavior: via Clang driver flag
(-fregister-global-dtors-with-atexit) or llc / code generation flag
(-lower-global-dtors-via-cxa-atexit). This escape hatch will be
removed in the future.

Herald added a project: Restricted Project. · View Herald TranscriptMar 14 2022, 4:48 PM
yln marked an inline comment as done.Mar 14 2022, 5:02 PM
yln added inline comments.
llvm/lib/CodeGen/TargetPassConfig.cpp
900

I added an escape hatch to fallback to the old behavior:

  • via Clang driver flag -fregister-global-dtors-with-atexit
  • llc / code generation flag -lower-global-dtors-via-cxa-atexit.

This escape hatch will be removed in the future.

yln added a comment.Mar 14 2022, 5:16 PM

@sunfish
Hi Dan, I hope you are still happy with this change. I didn't change any WebAssembly tests, but rather added a new IR-level test, so all existing WebAssembly behavior should stay the same. Let me know if you have any concerns.

delcypher added inline comments.Mar 14 2022, 5:20 PM
llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp
1181

If you have access to Options.LowerGlobalDtorsViaCxaAtExit then you could do something like

if (Options.LowerGlobalDtorsViaCxaAtExit)
  report_fatal_error("@llvm.global_dtors should have been lowered already");

so that we error when appropriate.

I'm not sure if you have easy access to the option from this class, so if it's difficult then we can ignore this problem.

llvm/lib/CodeGen/TargetPassConfig.cpp
900

@yln I don't see any modifications to the driver. How is -fregister-global-dtors-with-atexit added as a driver flag?

yln updated this revision to Diff 415282.Mar 14 2022, 5:29 PM

Report fatal error when we can, instead of never.

@sunfish
Hi Dan, I hope you are still happy with this change. I didn't change any WebAssembly tests, but rather added a new IR-level test, so all existing WebAssembly behavior should stay the same. Let me know if you have any concerns.

Yes, the change looks good to me!

yln marked 2 inline comments as done.Mar 14 2022, 5:48 PM
yln added inline comments.
llvm/lib/CodeGen/TargetPassConfig.cpp
900

This is an existing flag in the frontend added as part of this change: D45578

Add a command line option `fregister_global_dtors_with_atexit` to register destructor functions annotated with __attribute__((destructor)) using __cxa_atexit or atexit.

It's on by default on Darwin:

if (Args.hasFlag(options::OPT_fregister_global_dtors_with_atexit,
                 options::OPT_fno_register_global_dtors_with_atexit,
                 RawTriple.isOSDarwin() && !KernelOrKext))
  CmdArgs.push_back("-fregister-global-dtors-with-atexit");

Essentially we switched over the frontend to avoid emitting the deprecated __mod_term_func section a long time ago and now I am using the same technique to lower .llvm.global_dtors in the backend. So we could say the flag is exactly what we wanted to express (it makes sense to key off it), but now it applies more fully.

Before D45578, destructor functions annotated with __attribute__((destructor)) were added to .llvm.global_dtor. I do not know why we decided to lower them in the frontend (where it didn't apply to .llvm.global_dtors) instead of the backend (this change).

There is certainly an opportunity here to refactor this and make the frontend use llvm.global_dtor again to reduce code duplication (after making sure we cover all the necessary cases). That's a big and wide-reaching change and I am not intending to do this as part of this change.

This revision was landed with ongoing or failed builds.Mar 14 2022, 5:51 PM
This revision was automatically updated to reflect the committed changes.
yln marked an inline comment as done.

Hi @yln, the test you added llvm/test/Transforms/LowerGlobalDestructors/lower-global-dtors.ll is failing on some bots, e.g. https://lab.llvm.org/buildbot/#/builders/139/builds/18527 (builder llvm-clang-x86_64-sie-ubuntu-fast). Please can you take a look?

@yln I've reverted your commit at rG7262eacd41997d7ca262d83367e28998662c1b21 to try and get the buildbots green again

yln added a comment.Mar 15 2022, 1:28 PM

Hi @Orlando and @RKSimon!

Thanks for pointing out the test failure and reverting the change. I see this failure:

: 'RUN: at line 1';   /home/buildbot/buildbot-root/llvm-clang-x86_64-sie-ubuntu-fast/build/bin/opt        -lower-global-dtors -S < /home/buildbot/buildbot-root/llvm-project/llvm/test/Transforms/LowerGlobalDestructors/lower-global-dtors.ll | /home/buildbot/buildbot-root/llvm-clang-x86_64-sie-ubuntu-fast/build/bin/FileCheck /home/buildbot/buildbot-root/llvm-project/llvm/test/Transforms/LowerGlobalDestructors/lower-global-dtors.ll --implicit-check-not=llvm.global_dtors
: 'RUN: at line 2';   /home/buildbot/buildbot-root/llvm-clang-x86_64-sie-ubuntu-fast/build/bin/opt -passes=lower-global-dtors -S < /home/buildbot/buildbot-root/llvm-project/llvm/test/Transforms/LowerGlobalDestructors/lower-global-dtors.ll | /home/buildbot/buildbot-root/llvm-clang-x86_64-sie-ubuntu-fast/build/bin/FileCheck /home/buildbot/buildbot-root/llvm-project/llvm/test/Transforms/LowerGlobalDestructors/lower-global-dtors.ll --implicit-check-not=llvm.global_dtors
--
Exit Code: 2
Command Output (stderr):
--
opt: Unknown command line argument '-lower-global-dtors'.  Try: '/home/buildbot/buildbot-root/llvm-clang-x86_64-sie-ubuntu-fast/build/bin/opt --help'
opt: Did you mean '--join-globalcopies'?

This looks like my changes are not sufficiently registering the pass for opt on every platform, but so far I haven't found anything that is obviously wrong/missing. Do you know who I can reach out to help me with "pass registration"?

yln added a comment.Mar 15 2022, 1:42 PM

New revision here: D121736

New revision here: D121736

FYI - you could have continued to use this phab for review, it makes it easier to compare the changes you've made