This is an archive of the discontinued LLVM Phabricator instance.

Disable llvm-symbolizer on some of the driver tests that are timing out
Needs ReviewPublic

Authored by ahatanak on Apr 20 2023, 3:38 PM.

Details

Summary

These tests don't require invoking llvm-symbolizer.

Diff Detail

Event Timeline

ahatanak created this revision.Apr 20 2023, 3:38 PM
Herald added a project: Restricted Project. · View Herald TranscriptApr 20 2023, 3:38 PM
ahatanak requested review of this revision.Apr 20 2023, 3:38 PM
shafik added a subscriber: shafik.Apr 20 2023, 5:34 PM

Do you know why these started timing out? I saw this locally the other day but could not figure out the root cause.

MaskRay added a comment.EditedApr 20 2023, 5:37 PM

Do you know why these started timing out? I saw this locally the other day but could not figure out the root cause.

D86170 provides some information about llvm-symbolizer performance with an unoptimized debug build (slow llvm-symbolizer + huge debug info => slowness in crashes).
I think we probably should add LLVM_DISABLE_SYMBOLIZATION=1 to the lit level, not in individual tests.

I'm OK with this, though I wouldn't mind a more robust/general solution to this - especially all gunit death tests have this problem too - they crash and spend significant time symbolizing, I think? So it'd be great if we could find some way to run those with symbolization disabled too, not sure if there would be a more general place that'd cover both of these cases (but it might be interesting to you & perhaps you can find a fix for gunit too, even if it's a distinct solution, if the problem in general is of interest)

I think we probably should add LLVM_DISABLE_SYMBOLIZATION=1 to the lit level, not in individual tests.

Though I'm not sure how to do that in a way that it doesn't apply to test that are genuinely failing, where a symbolized backtrace is helpful/important/exactly what want?

ahatanak updated this revision to Diff 518610.May 1 2023, 5:31 PM

Disable llvm-symbolizer when running lit tests.

Herald added a project: Restricted Project. · View Herald TranscriptMay 1 2023, 5:31 PM
ahatanak updated this revision to Diff 518630.May 1 2023, 9:32 PM

Enable llvm-symbolizer when running disable-crash-reports.test.

Disable llvm-symbolizer when running lit tests.

This seems problematic though - when lit tests fail it's quite helpful to get a symbolized stack trace.

shafik added a comment.May 8 2023, 5:53 PM

Disable llvm-symbolizer when running lit tests.

This seems problematic though - when lit tests fail it's quite helpful to get a symbolized stack trace.

While I have seen this again just today with the driver tests and it is annoying, I would also prefer a more targeted solution if possible.

Disable llvm-symbolizer when running lit tests.

This seems problematic though - when lit tests fail it's quite helpful to get a symbolized stack trace.

While I have seen this again just today with the driver tests and it is annoying, I would also prefer a more targeted solution if possible.

OK, I can go back to the original solution if it's important to keep the symbolizer enabled.

ahatanak updated this revision to Diff 520861.May 9 2023, 4:59 PM

Disable llvm-symbolizer individually.

not will set LLVM_DISABLE_SYMBOLIZATION if we use not --crash

are these tests supposed to crash or gracefully fail? if they're supposed to crash then we should use not --crash

arsenm added a subscriber: arsenm.May 19 2023, 12:35 PM

I also think we need to revert or disable https://github.com/llvm/llvm-project/commit/cead4eceb01b935fae07bf4a7e91911b344d2fec

The symbolizer is unusably slow with it

I think we probably should add LLVM_DISABLE_SYMBOLIZATION=1 to the lit level, not in individual tests.

Though I'm not sure how to do that in a way that it doesn't apply to test that are genuinely failing, where a symbolized backtrace is helpful/important/exactly what want?

Ping @MaskRay thoughts on the tension between disabling it entirely, and it being useful for developers investigating real failures?

I also think we need to revert or disable https://github.com/llvm/llvm-project/commit/cead4eceb01b935fae07bf4a7e91911b344d2fec

The symbolizer is unusably slow with it

I believe that issue's been addressed by 02e8eb1a438bdb1dc9a97aea75a8c9c748048039 / D145009 ?

@ahatanak Thanks for working on this. I think that annotating these particular tests so that's clear that they are supposed to crash (and therefore symbolication is of no use) is a good change. If we want to make a more global change for tests I think that should be handled separately.

However, if not --crash does the job of disabling symbolication I think we should use that because it also documents that we expect the binary being executed to crash. If you make that change then I'm happy to approve.

clang/test/Driver/crash-diagnostics-dir-3.c
4 ↗(On Diff #520861)

Should we use not --crash instead? It's more explicit that we are expecting a crash and IIRC should disable symbolization as well.