[clang-tidy] Suppress one unittest on macOS.

[NoQ]

Summary

It's failing on macOS under AddressSanitizer for no obvious reason - i.e., no report is produced, just the code isn't working correctly. Might possibly be a bug in ASan but I want to silence an internal buildbot before a deeper investigation. I didn't try platforms other than macOS.

The failure can be reproduced somewhere around rG0b2a92224630f6e177d091b8391cfa943764aba5 by self-hosting. The unittest has to be compiled as -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_SANITIZER="Address" in order to reproduce the issue (RelWithDebInfo works too, but not Debug).

[vsavchenko]

LGTM, thanks for taking care of this!

[njames93]

I was having issues with this test case under macos in D82188.
It would fail for seemingly no apparent reason until I disable a test in a different translation unit.
This made me think there is a subtle bug in the linker used on macos. That could also explain why asan is having a hard time with this as well.
I got as far as seeing that runCheckOnCode was called, the check was instantiated but its matchers never got registered effectively meaning it didn't run.

[NoQ]

In D84453#2171548, @njames93 wrote:

I was having issues with this test case under macos in D82188.
It would fail for seemingly no apparent reason until I disable a test in a different translation unit.
This made me think there is a subtle bug in the linker used on macos. That could also explain why asan is having a hard time with this as well.
I got as far as seeing that runCheckOnCode was called, the check was instantiated but its matchers never got registered effectively meaning it didn't run.

Damn, i'm very glad you showed up. I can reproduce your problem as well, so ASan is probably not at fault. I'll try to talk to linker folks to see if they can fix this.

I guess i'll still commit this patch so that to unbreak the buildbots but i'll keep an eye on this issue.

[NoQ]

Update comments with the updated investigation details.

[njames93]

In D84453#2177665, @NoQ wrote:

In D84453#2171548, @njames93 wrote:

I was having issues with this test case under macos in D82188.
It would fail for seemingly no apparent reason until I disable a test in a different translation unit.
This made me think there is a subtle bug in the linker used on macos. That could also explain why asan is having a hard time with this as well.
I got as far as seeing that runCheckOnCode was called, the check was instantiated but its matchers never got registered effectively meaning it didn't run.

Damn, i'm very glad you showed up. I can reproduce your problem as well, so ASan is probably not at fault. I'll try to talk to linker folks to see if they can fix this.

I guess i'll still commit this patch so that to unbreak the buildbots but i'll keep an eye on this issue.

If it is an issue with the linking and the issue lies within the system ld binary on mac os then we really can't do much here apart from raise an issue with apple folks.
Be interesting to see if this bug is present when using lld on mac os, however I don't have a mac so its kind of hard for me to test.

clang-tools-extra/unittests/clang-tidy/ClangTidyDiagnosticConsumerTest.cpp

public:
   void check(const ast_matchers::MatchFinder::MatchResult &Result) override {
     const auto *Var = Result.Nodes.getNodeAs<VarDecl>("var");
     // Add diagnostics in the wrong order.
     diag(Var->getLocation(), "variable");
     diag(Var->getTypeSpecStartLoc(), "type specifier");
   }
 };
 
+// FIXME: This test seems to cause a strange linking interference
+// with the ValidConfiguration.ValidEnumOptions test on macOS.
+// If both tests are enabled, this test will fail as if
+// runCheckOnCode() is not invoked at all. Looks like a linker bug.
+// For now both tests are disabled on macOS. It is not sufficient
+// to only disable the other test because this test keeps failing
+// under Address Sanitizer, which may be an indication of more
+// such linking interference with other tests and this test
+// seems to be in the center of it.
+#ifndef __APPLE__
 TEST(ClangTidyDiagnosticConsumer, SortsErrors) {
   std::vector<ClangTidyError> Errors;
   runCheckOnCode<TestCheck>("int a;", &Errors);
   EXPECT_EQ(2ul, Errors.size());
   EXPECT_EQ("type specifier", Errors[0].Message.Message);
   EXPECT_EQ("variable", Errors[1].Message.Message);
 }
+#endif
 
 } // namespace test
 } // namespace tidy
 } // namespace clang

clang-tools-extra/unittests/clang-tidy/ClangTidyOptionsTest.cpp

#define CHECK_ERROR_INT(Name, Expected)                                        \
   CHECK_ERROR_INT(TestCheck.getIntLocal<bool>("BoolCaseMismatch"),
                   "invalid configuration value 'True' for option "
                   "'test.BoolCaseMismatch'; expected a bool");
 
 #undef CHECK_ERROR_INT
 }
 
 // FIXME: Figure out why this test causes crashes on mac os.
+// See also comments around the ClangTidyDiagnosticConsumer.SortsErrors test.
 #ifndef __APPLE__
 TEST(ValidConfiguration, ValidEnumOptions) {
 
   ClangTidyOptions Options;
   auto &CheckOptions = Options.CheckOptions;
 
   CheckOptions["test.Valid"] = "Red";
   CheckOptions["test.Invalid"] = "Scarlet";