Disable llvm-symbolizer on some of the driver tests that are timing out

[ahatanak]

Summary

These tests don't require invoking llvm-symbolizer.

[ahatanak]

These tests are all timing out after 600 seconds. https://green.lab.llvm.org/green/job/clang-stage1-RA/34005/#showFailuresLink

[shafik]

Do you know why these started timing out? I saw this locally the other day but could not figure out the root cause.

[MaskRay]

In D148851#4285583, @shafik wrote:

Do you know why these started timing out? I saw this locally the other day but could not figure out the root cause.

D86170 provides some information about llvm-symbolizer performance with an unoptimized debug build (slow llvm-symbolizer + huge debug info => slowness in crashes).
I think we probably should add LLVM_DISABLE_SYMBOLIZATION=1 to the lit level, not in individual tests.

[dblaikie]

I'm OK with this, though I wouldn't mind a more robust/general solution to this - especially all gunit death tests have this problem too - they crash and spend significant time symbolizing, I think? So it'd be great if we could find some way to run those with symbolization disabled too, not sure if there would be a more general place that'd cover both of these cases (but it might be interesting to you & perhaps you can find a fix for gunit too, even if it's a distinct solution, if the problem in general is of interest)

[dblaikie]

I think we probably should add LLVM_DISABLE_SYMBOLIZATION=1 to the lit level, not in individual tests.

Though I'm not sure how to do that in a way that it doesn't apply to test that are genuinely failing, where a symbolized backtrace is helpful/important/exactly what want?

[ahatanak]

Disable llvm-symbolizer when running lit tests.

[ahatanak]

Enable llvm-symbolizer when running disable-crash-reports.test.

[dblaikie]

In D148851#4311266, @ahatanak wrote:

Disable llvm-symbolizer when running lit tests.

This seems problematic though - when lit tests fail it's quite helpful to get a symbolized stack trace.

[shafik]

In D148851#4328084, @dblaikie wrote:

In D148851#4311266, @ahatanak wrote:

Disable llvm-symbolizer when running lit tests.

This seems problematic though - when lit tests fail it's quite helpful to get a symbolized stack trace.

While I have seen this again just today with the driver tests and it is annoying, I would also prefer a more targeted solution if possible.

[ahatanak]

In D148851#4328345, @shafik wrote:

In D148851#4328084, @dblaikie wrote:

In D148851#4311266, @ahatanak wrote:

Disable llvm-symbolizer when running lit tests.

This seems problematic though - when lit tests fail it's quite helpful to get a symbolized stack trace.

While I have seen this again just today with the driver tests and it is annoying, I would also prefer a more targeted solution if possible.

OK, I can go back to the original solution if it's important to keep the symbolizer enabled.

[ahatanak]

Disable llvm-symbolizer individually.

[ahatanak]

ping

[aeubanks]

not will set LLVM_DISABLE_SYMBOLIZATION if we use not --crash

are these tests supposed to crash or gracefully fail? if they're supposed to crash then we should use not --crash

[arsenm]

I also think we need to revert or disable https://github.com/llvm/llvm-project/commit/cead4eceb01b935fae07bf4a7e91911b344d2fec

The symbolizer is unusably slow with it

[dblaikie]

In D148851#4285705, @dblaikie wrote:

I think we probably should add LLVM_DISABLE_SYMBOLIZATION=1 to the lit level, not in individual tests.

Though I'm not sure how to do that in a way that it doesn't apply to test that are genuinely failing, where a symbolized backtrace is helpful/important/exactly what want?

Ping @MaskRay thoughts on the tension between disabling it entirely, and it being useful for developers investigating real failures?

[dblaikie]

In D148851#4357360, @arsenm wrote:

I also think we need to revert or disable https://github.com/llvm/llvm-project/commit/cead4eceb01b935fae07bf4a7e91911b344d2fec

The symbolizer is unusably slow with it

I believe that issue's been addressed by 02e8eb1a438bdb1dc9a97aea75a8c9c748048039 / D145009 ?

[delcypher]

@ahatanak Thanks for working on this. I think that annotating these particular tests so that's clear that they are supposed to crash (and therefore symbolication is of no use) is a good change. If we want to make a more global change for tests I think that should be handled separately.

However, if not --crash does the job of disabling symbolication I think we should use that because it also documents that we expect the binary being executed to crash. If you make that change then I'm happy to approve.

clang/test/Driver/crash-diagnostics-dir-3.c

4 ↗

(On Diff #520861)

Should we use not --crash instead? It's more explicit that we are expecting a crash and IIRC should disable symbolization as well.

llvm/utils/lit/lit/TestingConfig.py

def fromdefaults(litConfig):
 
         Create a TestingConfig object with default values.
         """
         # Set the environment based on the command line arguments.
         environment = {
             'PATH' : os.pathsep.join(litConfig.path +
                                      [os.environ.get('PATH','')]),
             'LLVM_DISABLE_CRASH_REPORT' : '1',
+            'LLVM_DISABLE_SYMBOLIZATION' : '1',
             }
 
         pass_vars = [
             'LIBRARY_PATH',
             'LD_LIBRARY_PATH',
             'SYSTEMROOT',
             'TERM',
             'CLANG',