[test] Fix plugin tests

[hintonda]

Summary

The following changes were required to fix these tests:

1. Change LLVM_ENABLE_PLUGINS to an option and move it to llvm/CMakeLists.txt with an appropriate default -- which matches the original default behavior.

2. Move the plugins directory from clang/test/Analysis clang/lib/Analysis. It's not enough to add an exclude to the lit.local.cfg file because add_lit_testsuites recurses the tree and automatically adds the appropriate check- targets, which don't make sense for the plugins because they aren't tests and don't have RUN statements.
   
   Here's a list of the clang-check-anlysis* targets with this change:

$ ninja -t targets all| sed -n "s/.*\/\(check[^:]*\):.*/\1/p" | sort -u | grep clang-analysis
check-clang-analysis
check-clang-analysis-checkers
check-clang-analysis-copypaste
check-clang-analysis-diagnostics
check-clang-analysis-engine
check-clang-analysis-exploration_order
check-clang-analysis-html_diagnostics
check-clang-analysis-html_diagnostics-relevant_lines
check-clang-analysis-inlining
check-clang-analysis-objc
check-clang-analysis-unified-sources
check-clang-analysis-z3

3. Simplify the logic and only include the subdirectories under clang/lib/Analysis/plugins if LLVM_ENABLE_PLUGINS is set.

[NoQ]

Thanks!!

I think we're losing the Z3 constraint manager test suite this way after all (details inline), but i feel we should downprioritize this when it comes to fixing everybody's buildbots.

clang/test/CMakeLists.txt

141–147

As far as i remember, this chunk of code is responsible for running the whole analyzer test suite with different parameters (with Z3 as a constraint manager instead of the ad-hoc range constraint manager), rather than running a smaller chunk of the suite.

@mikhail.ramalho, @ddcc - i think i should leave it up to you to decide if you want to keep this working. Right now these extra tests aren't run under any buildbot, and the facility that they're testing is probably never going to be used by actual users (as opposed to the z3 refutation). But i wouldn't love losing this facility because it's a great way of evaluating the static analyzer, i.e. figuring out how good *could* it have been with a good constraint solver, so that we had something to look forward to.

[hintonda]

clang/test/CMakeLists.txt

141–147

Ah, that was unintentional. I didn't intend to touch Z3.

Let me see if I can put that part back.

[hintonda]

clang/test/CMakeLists.txt

141–147

Actually, on second thought, these get picked up automatically, so they shouldn't be added like this.

[hintonda]

This patch broke some of the bots, so reopening to track the fix.

[hintonda]

• Fix dependencies so required plugins get built before tests.

[davezarzycki]

This broke non-PIC builds. Fixed: r362399

[hintonda]

In D62445#1527368, @davezarzycki wrote:

This broke non-PIC builds. Fixed: r362399

Sorry for the breakage, and thanks for the fix!

[nathanchance]

cfe/trunk/lib/Analysis/plugins/CMakeLists.txt

1 ↗

(On Diff #202605)

I think this should have a dependency on CLANG_ENABLE_STATIC_ANALYZER like in clang/test/CMakeLists.txt, otherwise my build (which disables CLANG_ENABLE_STATIC_ANALYZER) fails because these plugins are being added to the build.

[2058/2605] Linking CXX shared module lib/CheckerOptionHandlingAnalyzerPlugin.so
FAILED: lib/CheckerOptionHandlingAnalyzerPlugin.so 
: && /usr/bin/g++ -fPIC -O2 -march=native -mtune=native -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -w -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -fno-strict-aliasing -O3 -DNDEBUG  -Wl,-z,nodelete -fuse-ld=gold   -Wl,-O3 -Wl,--gc-sections  -Wl,--version-script,"/tc-build2/build/llvm/stage1/tools/clang/lib/Analysis/plugins/CheckerOptionHandling/CheckerOptionHandlingAnalyzerPlugin.exports" -shared  -o lib/CheckerOptionHandlingAnalyzerPlugin.so tools/clang/lib/Analysis/plugins/CheckerOptionHandling/CMakeFiles/CheckerOptionHandlingAnalyzerPlugin.dir/CheckerOptionHandling.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMSupport.a lib/libclangAnalysis.a lib/libclangAST.a -lclangStaticAnalyzerCore -lclangStaticAnalyzerFrontend lib/libLLVMSupport.a lib/libclangASTMatchers.a lib/libclangAST.a lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCore.a lib/libLLVMRemarks.a lib/libLLVMMC.a lib/libLLVMBinaryFormat.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMDebugInfoMSF.a lib/libLLVMSupport.a -lz -lrt -ldl -lpthread -lm lib/libLLVMDemangle.a && :
/usr/bin/ld.gold: error: cannot find -lclangStaticAnalyzerCore
/usr/bin/ld.gold: error: cannot find -lclangStaticAnalyzerFrontend
collect2: error: ld returned 1 exit status
[2059/2605] Building CXX object tools/clang/tools/clang-refactor/CMakeFiles/clang-refactor.dir/TestSupport.cpp.o
[2060/2605] Linking CXX shared module lib/CheckerDependencyHandlingAnalyzerPlugin.so
FAILED: lib/CheckerDependencyHandlingAnalyzerPlugin.so 
: && /usr/bin/g++ -fPIC -O2 -march=native -mtune=native -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -w -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -fno-strict-aliasing -O3 -DNDEBUG  -Wl,-z,nodelete -fuse-ld=gold   -Wl,-O3 -Wl,--gc-sections  -Wl,--version-script,"/tc-build2/build/llvm/stage1/tools/clang/lib/Analysis/plugins/CheckerDependencyHandling/CheckerDependencyHandlingAnalyzerPlugin.exports" -shared  -o lib/CheckerDependencyHandlingAnalyzerPlugin.so tools/clang/lib/Analysis/plugins/CheckerDependencyHandling/CMakeFiles/CheckerDependencyHandlingAnalyzerPlugin.dir/CheckerDependencyHandling.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMSupport.a lib/libclangAnalysis.a lib/libclangAST.a -lclangStaticAnalyzerCore -lclangStaticAnalyzerFrontend lib/libLLVMSupport.a lib/libclangASTMatchers.a lib/libclangAST.a lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCore.a lib/libLLVMRemarks.a lib/libLLVMMC.a lib/libLLVMBinaryFormat.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMDebugInfoMSF.a lib/libLLVMSupport.a -lz -lrt -ldl -lpthread -lm lib/libLLVMDemangle.a && :
/usr/bin/ld.gold: error: cannot find -lclangStaticAnalyzerCore
/usr/bin/ld.gold: error: cannot find -lclangStaticAnalyzerFrontend
collect2: error: ld returned 1 exit status
[2061/2605] Building CXX object tools/clang/lib/Tooling/CMakeFiles/obj.clangTooling.dir/CompilationDatabase.cpp.o
[2062/2605] Linking CXX shared module lib/SampleAnalyzerPlugin.so
FAILED: lib/SampleAnalyzerPlugin.so 
: && /usr/bin/g++ -fPIC -O2 -march=native -mtune=native -fPIC -fvisibility-inlines-hidden -Werror=date-time -std=c++11 -w -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -fno-strict-aliasing -O3 -DNDEBUG  -Wl,-z,nodelete -fuse-ld=gold   -Wl,-O3 -Wl,--gc-sections  -Wl,--version-script,"/tc-build2/build/llvm/stage1/tools/clang/lib/Analysis/plugins/SampleAnalyzer/SampleAnalyzerPlugin.exports" -shared  -o lib/SampleAnalyzerPlugin.so tools/clang/lib/Analysis/plugins/SampleAnalyzer/CMakeFiles/SampleAnalyzerPlugin.dir/MainCallChecker.cpp.o  -Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMSupport.a lib/libclangAnalysis.a lib/libclangAST.a -lclangStaticAnalyzerCore -lclangStaticAnalyzerFrontend lib/libLLVMSupport.a lib/libclangASTMatchers.a lib/libclangAST.a lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCore.a lib/libLLVMRemarks.a lib/libLLVMMC.a lib/libLLVMBinaryFormat.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMDebugInfoMSF.a lib/libLLVMSupport.a -lz -lrt -ldl -lpthread -lm lib/libLLVMDemangle.a && :
/usr/bin/ld.gold: error: cannot find -lclangStaticAnalyzerCore
/usr/bin/ld.gold: error: cannot find -lclangStaticAnalyzerFrontend
collect2: error: ld returned 1 exit status

[hintonda]

cfe/trunk/lib/Analysis/plugins/CMakeLists.txt

1 ↗

(On Diff #202605)

Sounds reasonable, and sorry for the breakage -- I'll post a fix shortly.

[hintonda]

cfe/trunk/lib/Analysis/plugins/CMakeLists.txt

1 ↗

(On Diff #202605)

Fixed in r362555.

Thanks again for reporting this...

[NoQ]

Ugh, there seems to be one more forgotten buildbot with plugins problems: http://green.lab.llvm.org/green/job/clang-stage2-cmake-RgSan/6406/consoleText

FAIL: Clang :: Analysis/checker-plugins.c (467 of 14955)
******************** TEST 'Clang :: Analysis/checker-plugins.c' FAILED ********************
Script:
--
: 'RUN: at line 3';   /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range -verify /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/SampleAnalyzerPlugin.dylib    -analyzer-checker='example.MainCallChecker'
: 'RUN: at line 15';   /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/CheckerDependencyHandlingAnalyzerPlugin.dylib   -analyzer-checker=example.DependendentChecker    -analyzer-list-enabled-checkers    2>&1 | /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/FileCheck /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c -check-prefix=CHECK-IMPLICITLY-ENABLED
: 'RUN: at line 24';   /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/CheckerDependencyHandlingAnalyzerPlugin.dylib   -analyzer-checker=example.DependendentChecker    -analyzer-disable-checker=example.Dependency    -analyzer-list-enabled-checkers    2>&1 | /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/FileCheck /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c -check-prefix=CHECK-IMPLICITLY-DISABLED
: 'RUN: at line 34';   /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/CheckerOptionHandlingAnalyzerPlugin.dylib   -analyzer-checker=example.MyChecker    2>&1 | /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/FileCheck /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c -check-prefix=CHECK-CHECKER-OPTION-OUTPUT
: 'RUN: at line 41';   /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/CheckerOptionHandlingAnalyzerPlugin.dylib   -analyzer-checker=example.MyChecker    -analyzer-config example.MyChecker:ExampleOption=true    2>&1 | /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/FileCheck /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c -check-prefix=CHECK-CHECKER-OPTION-OUTPUT-TRUE
: 'RUN: at line 49';   /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/CheckerOptionHandlingAnalyzerPlugin.dylib   -analyzer-checker=example.MyChecker    -analyzer-checker=debug.ConfigDumper    2>&1 | /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/FileCheck /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c -check-prefix=CHECK-CHECKER-OPTION
: 'RUN: at line 57';   /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/CheckerOptionHandlingAnalyzerPlugin.dylib   -analyzer-checker=example.MyChecker    -analyzer-checker=debug.ConfigDumper    -analyzer-config example.MyChecker:ExampleOption=true    2>&1 | /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/FileCheck /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c -check-prefix=CHECK-CHECKER-OPTION-TRUE
: 'RUN: at line 66';   not /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range -verify /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/CheckerOptionHandlingAnalyzerPlugin.dylib   -analyzer-checker=example.MyChecker    -analyzer-config example.MyChecker:Example=true    2>&1 | /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/FileCheck /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c -check-prefix=CHECK-NON-EXISTENT-CHECKER-OPTION
: 'RUN: at line 75';   not /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range -verify /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/CheckerOptionHandlingAnalyzerPlugin.dylib   -analyzer-checker=example.MyChecker    -analyzer-config-compatibility-mode=true    -analyzer-config example.MyChecker:Example=true
: 'RUN: at line 82';   not /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range -verify /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/CheckerOptionHandlingAnalyzerPlugin.dylib   -analyzer-checker=example.MyChecker    -analyzer-config example.MyChecker:ExampleOption=example    2>&1 | /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/FileCheck /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c -check-prefix=CHECK-INVALID-BOOL-VALUE
: 'RUN: at line 92';   not /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range -verify /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/CheckerOptionHandlingAnalyzerPlugin.dylib   -analyzer-checker=example.MyChecker    -analyzer-config-compatibility-mode=true    -analyzer-config example.MyChecker:ExampleOption=example
: 'RUN: at line 98';   /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/CheckerOptionHandlingAnalyzerPlugin.dylib   -analyzer-checker=example.MyChecker    -analyzer-checker=debug.ConfigDumper    -analyzer-config-compatibility-mode=true    -analyzer-config example.MyChecker:ExampleOption=example    2>&1 | /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/FileCheck /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c -check-prefix=CHECK-CORRECTED-BOOL-VALUE
: 'RUN: at line 108';   /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/CheckerOptionHandlingAnalyzerPlugin.dylib   -analyzer-checker=example.MyChecker    -analyzer-checker-option-help    2>&1 | /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/FileCheck /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c -check-prefix=CHECK-CHECKER-OPTION-HELP
--
Exit Code: 1

The error message doesn't seem to be very expressive. I also can't reproduce it locally.

It got suddenly fixed in http://green.lab.llvm.org/green/job/clang-stage2-cmake-RgSan/6409/ but then immediately failed again later and it's still failing in a similar manner, and nobody noticed for two months =/

I'll keep looking into this but i'd love to hear if you have any immediate thoughts on that :)

[hintonda]

In D62445#1613268, @NoQ wrote:

Ugh, there seems to be one more forgotten buildbot with plugins problems: http://green.lab.llvm.org/green/job/clang-stage2-cmake-RgSan/6406/consoleText

FAIL: Clang :: Analysis/checker-plugins.c (467 of 14955)
******************** TEST 'Clang :: Analysis/checker-plugins.c' FAILED ********************
Script:
--
: 'RUN: at line 3';   /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/bin/clang -cc1 -internal-isystem /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/lib/clang/9.0.0/include -nostdsysteminc -analyze -analyzer-constraints=range -verify /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/llvm/tools/clang/test/Analysis/checker-plugins.c    -load /Users/buildslave/jenkins/workspace/clang-stage2-cmake-RgSan/clang-build/./lib/SampleAnalyzerPlugin.dylib    -analyzer-checker='example.MainCallChecker'

<snip>

Exit Code: 1

The error message doesn't seem to be very expressive. I also can't reproduce it locally.

It got suddenly fixed in http://green.lab.llvm.org/green/job/clang-stage2-cmake-RgSan/6409/ but then immediately failed again later and it's still failing in a similar manner, and nobody noticed for two months =/

I'll keep looking into this but i'd love to hear if you have any immediate thoughts on that :)

I don't see anything obvious, but could change the bot to pass "-vv" to lit instead of just "-v"? That way the actual error will make to the log.

[hintonda]

In D62445#1613268, @NoQ wrote:

Ugh, there seems to be one more forgotten buildbot with plugins problems: http://green.lab.llvm.org/green/job/clang-stage2-cmake-RgSan/6406/consoleText

<snip>

It got suddenly fixed in http://green.lab.llvm.org/green/job/clang-stage2-cmake-RgSan/6409/ but then immediately failed again later and it's still failing in a similar manner, and nobody noticed for two months =/

The fix was accidental and fortuitous.

r362399 tried to set LLVM_ENABLE_PLUGINS_default to the value of LLVM_ENABLE_PIC, but had a typo that actually turn off LLVM_ENABLE_PLUGINS, so the test was marked unsupported in build #6409. That typo was fixed in r362492 which caused the next build, #6411 and all subsequent builds, to fail.

Will now look into cause/fix.

[davezarzycki]

Just FYI – Having LLVM_ENABLE_PLUGINS_default depend on LLVM_ENABLE_PIC is a hack that should go away someday. Doing so requires that plugin dependencies either always build PIC (and therefore ignore LLVM_ENABLE_PIC), or are linked into the executables that load them and the plugin knows how to find them in the executable. In the case of the latter and on Mach-O platforms, this requires passing -bundle_loader $PATH_TO_EXECUTABLE to the linker when creating the plugin (a.k.a. "bundle"), but on ELF platforms, I'm not sure what needs to be done.

[davezarzycki]

In D62445#1616218, @davezarzycki wrote:

Just FYI – Having LLVM_ENABLE_PLUGINS_default depend on LLVM_ENABLE_PIC is a hack that should go away someday. Doing so requires that plugin dependencies either always build PIC (and therefore ignore LLVM_ENABLE_PIC), or are linked into the executables that load them and the plugin knows how to find them in the executable. In the case of the latter and on Mach-O platforms, this requires passing -bundle_loader $PATH_TO_EXECUTABLE to the linker when creating the plugin (a.k.a. "bundle"), but on ELF platforms, I'm not sure what needs to be done.

It just occurred to me: option 3 is to statically link the plugins if PIC is disabled.

These are all just tradeoffs of course, and the worst among them is what we have today: conflating "enable plugins" with the default PIC behavior for the project.

[NoQ]

I didn't make much progress so far, but i marked the test as // UNSUPPORTED: darwin in rC368765, so the buildbot is now green starting with http://green.lab.llvm.org/green/job/clang-stage2-cmake-RgSan/6519/.

[hintonda]

In D62445#1630518, @NoQ wrote:

I didn't make much progress so far, but i marked the test as // UNSUPPORTED: darwin in rC368765, so the buildbot is now green starting with http://green.lab.llvm.org/green/job/clang-stage2-cmake-RgSan/6519/.

I haven't had a chance to check this, but I suspect these plugin tests need LLVM_NO_DEAD_STRIP, since the failing build is RelWithDebInfo.

[mikhail.ramalho]

Hi, I just found this revision.

Could you point out how can I run the CSA tests with Z3 as backend? We used to do $ ninja check-clang-analyzer-z3, however, it now fails and suggests:

 $ ninja check-clang-analyzer-z3
ninja: error: unknown target 'check-clang-analyzer-z3', did you mean 'check-clang-analysis-z3'?

check-clang-analysis-z3 only runs one test though.

[thakis]

cfe/trunk/test/CMakeLists.txt

140 ↗

(On Diff #202605)

Did you intentionally delete the 2 targets here?

[hintonda]

cfe/trunk/test/CMakeLists.txt

140 ↗

(On Diff #202605)

Not intentionally, although I don't remember the details right now. Probably just a mistake on my part.

Do you want me to re-add them?

clang/lib/Analysis/CMakeLists.txt

add_clang_library(clangAnalysis
   UninitializedValues.cpp
 
   LINK_LIBS
   clangAST
   clangASTMatchers
   clangBasic
   clangLex
   )
+
+add_subdirectory(plugins)

clang/lib/Analysis/plugins/CMakeLists.txt

+if(LLVM_ENABLE_PLUGINS)
+  add_subdirectory(SampleAnalyzer)
+  add_subdirectory(CheckerDependencyHandling)
+  add_subdirectory(CheckerOptionHandling)
+endif()

clang/lib/Analysis/plugins/CheckerDependencyHandling/CMakeLists.txt

 set(LLVM_EXPORTED_SYMBOL_FILE ${CMAKE_CURRENT_SOURCE_DIR}/CheckerDependencyHandlingAnalyzerPlugin.exports)
 add_llvm_library(CheckerDependencyHandlingAnalyzerPlugin MODULE CheckerDependencyHandling.cpp PLUGIN_TOOL clang)
 
-if(LLVM_ENABLE_PLUGINS AND (WIN32 OR CYGWIN))
-  target_link_libraries(CheckerDependencyHandlingAnalyzerPlugin PRIVATE
-    clangAnalysis
-    clangAST
-    clangStaticAnalyzerCore
-    LLVMSupport
-    )
-endif()
+  clangStaticAnalyzerFrontend

clang/lib/Analysis/plugins/CheckerDependencyHandling/CheckerDependencyHandling.cpp

 #include "clang/StaticAnalyzer/Core/BugReporter/BugType.h"
 #include "clang/StaticAnalyzer/Core/Checker.h"
 #include "clang/StaticAnalyzer/Core/PathSensitive/CheckerContext.h"
 #include "clang/StaticAnalyzer/Frontend/CheckerRegistry.h"
 
 using namespace clang;
 using namespace ento;
 
 namespace {
 struct Dependency : public Checker<check::BeginFunction> {
   void checkBeginFunction(CheckerContext &Ctx) const {}
 };
 struct DependendentChecker : public Checker<check::BeginFunction> {
   void checkBeginFunction(CheckerContext &Ctx) const {}
 };
 } // end anonymous namespace
 
 // Register plugin!
 extern "C" void clang_registerCheckers(CheckerRegistry &registry) {
   registry.addChecker<Dependency>("example.Dependency", "", "");
   registry.addChecker<DependendentChecker>("example.DependendentChecker", "",
                                            "");
 
   registry.addDependency("example.DependendentChecker", "example.Dependency");
 }
 
 extern "C" const char clang_analyzerAPIVersionString[] =
     CLANG_ANALYZER_API_VERSION_STRING;

clang/lib/Analysis/plugins/CheckerDependencyHandling/CheckerDependencyHandlingAnalyzerPlugin.exports

 clang_registerCheckers
 clang_analyzerAPIVersionString

clang/lib/Analysis/plugins/CheckerOptionHandling/CMakeLists.txt

 set(LLVM_EXPORTED_SYMBOL_FILE ${CMAKE_CURRENT_SOURCE_DIR}/CheckerOptionHandlingAnalyzerPlugin.exports)
 add_llvm_library(CheckerOptionHandlingAnalyzerPlugin MODULE CheckerOptionHandling.cpp PLUGIN_TOOL clang)
 
-if(LLVM_ENABLE_PLUGINS AND (WIN32 OR CYGWIN))
-  target_link_libraries(CheckerOptionHandlingAnalyzerPlugin PRIVATE
-    clangAnalysis
-    clangAST
-    clangStaticAnalyzerCore
-    LLVMSupport
-    )
-endif()
+  clangStaticAnalyzerFrontend

clang/lib/Analysis/plugins/CheckerOptionHandling/CheckerOptionHandling.cpp

 #include "clang/StaticAnalyzer/Core/BugReporter/BugType.h"
 #include "clang/StaticAnalyzer/Core/Checker.h"
 #include "clang/StaticAnalyzer/Core/PathSensitive/CheckerContext.h"
 #include "clang/StaticAnalyzer/Frontend/CheckerRegistry.h"
 
 using namespace clang;
 using namespace ento;
 
 namespace {
 struct MyChecker : public Checker<check::BeginFunction> {
   void checkBeginFunction(CheckerContext &Ctx) const {}
 };
 
 void registerMyChecker(CheckerManager &Mgr) {
   MyChecker *Checker = Mgr.registerChecker<MyChecker>();
   llvm::outs() << "Example option is set to "
                << (Mgr.getAnalyzerOptions().getCheckerBooleanOption(
                        Checker, "ExampleOption")
                        ? "true"
                        : "false")
                << '\n';
 }
 
 bool shouldRegisterMyChecker(const LangOptions &LO) { return true; }
 
 } // end anonymous namespace
 
 // Register plugin!
 extern "C" void clang_registerCheckers(CheckerRegistry &registry) {
   registry.addChecker(registerMyChecker, shouldRegisterMyChecker,
                       "example.MyChecker", "Example Description",
                       "example.mychecker.documentation.nonexistent.html",
                       /*isHidden*/false);
 
   registry.addCheckerOption(/*OptionType*/ "bool",
                             /*CheckerFullName*/ "example.MyChecker",
                             /*OptionName*/ "ExampleOption",
                             /*DefaultValStr*/ "false",
                             /*Description*/ "This is an example checker opt.",
                             /*DevelopmentStage*/ "released");
 }
 
 extern "C" const char clang_analyzerAPIVersionString[] =
     CLANG_ANALYZER_API_VERSION_STRING;

clang/lib/Analysis/plugins/CheckerOptionHandling/CheckerOptionHandlingAnalyzerPlugin.exports

 clang_registerCheckers
 clang_analyzerAPIVersionString

clang/lib/Analysis/plugins/SampleAnalyzer/CMakeLists.txt

+set(LLVM_EXPORTED_SYMBOL_FILE ${CMAKE_CURRENT_SOURCE_DIR}/SampleAnalyzerPlugin.exports)
+add_llvm_library(SampleAnalyzerPlugin MODULE MainCallChecker.cpp PLUGIN_TOOL clang)
+
+target_link_libraries(SampleAnalyzerPlugin PRIVATE
+  clangAnalysis
+  clangAST
+  clangStaticAnalyzerCore
+  clangStaticAnalyzerFrontend
+  LLVMSupport
+  )

clang/lib/Analysis/plugins/SampleAnalyzer/MainCallChecker.cpp

 #include "clang/StaticAnalyzer/Core/BugReporter/BugType.h"
 #include "clang/StaticAnalyzer/Core/Checker.h"
 #include "clang/StaticAnalyzer/Core/PathSensitive/CheckerContext.h"
 #include "clang/StaticAnalyzer/Frontend/CheckerRegistry.h"
 
 using namespace clang;
 using namespace ento;
 
 namespace {
 class MainCallChecker : public Checker<check::PreStmt<CallExpr>> {
   mutable std::unique_ptr<BugType> BT;
 
 public:
   void checkPreStmt(const CallExpr *CE, CheckerContext &C) const;
 };
 } // end anonymous namespace
 
 void MainCallChecker::checkPreStmt(const CallExpr *CE,
                                    CheckerContext &C) const {
   const Expr *Callee = CE->getCallee();
   const FunctionDecl *FD = C.getSVal(Callee).getAsFunctionDecl();
 
   if (!FD)
     return;
 
   // Get the name of the callee.
   IdentifierInfo *II = FD->getIdentifier();
   if (!II) // if no identifier, not a simple C function
     return;
 
   if (II->isStr("main")) {
     ExplodedNode *N = C.generateErrorNode();
     if (!N)
       return;
 
     if (!BT)
       BT.reset(new BugType(this, "call to main", "example analyzer plugin"));
 
     std::unique_ptr<BugReport> report =
         llvm::make_unique<BugReport>(*BT, BT->getName(), N);
     report->addRange(Callee->getSourceRange());
     C.emitReport(std::move(report));
   }
 }
 
 // Register plugin!
 extern "C" void clang_registerCheckers(CheckerRegistry &registry) {
   registry.addChecker<MainCallChecker>(
       "example.MainCallChecker", "Disallows calls to functions called main",
       "");
 }
 
 extern "C" const char clang_analyzerAPIVersionString[] =
     CLANG_ANALYZER_API_VERSION_STRING;

clang/lib/Analysis/plugins/SampleAnalyzer/SampleAnalyzerPlugin.exports

 clang_registerCheckers
 clang_analyzerAPIVersionString

clang/test/Analysis/lit.local.cfg

 # with reference output)
 config.substitutions.append(('%diff_plist',
     'diff -u -w -I "<string>/" -I "<string>.:" -I "version"'))
 
 # Diff command for testing SARIF output to reference output.
 config.substitutions.append(('%diff_sarif',
     '''diff -U1 -w -I ".*file:.*%basename_t" -I '"version":' -I "2\.0\.0\-csd\.[0-9]*\.beta\."'''))
 
-config.excludes.add('plugins')
-
 if not config.root.clang_staticanalyzer:
     config.unsupported = True

clang/test/Analysis/plugins/CMakeLists.txt

-add_subdirectory(SampleAnalyzer)
-add_subdirectory(CheckerDependencyHandling)
-add_subdirectory(CheckerOptionHandling)
-
-set(CLANG_ANALYZER_PLUGIN_DEPS
-  SampleAnalyzerPlugin
-  CheckerDependencyHandlingAnalyzerPlugin
-  CheckerOptionHandlingAnalyzerPlugin
-  )
-
-add_custom_target(clang-analyzer-plugin
-  DEPENDS ${CLANG_ANALYZER_PLUGIN_DEPS})

clang/test/Analysis/plugins/SampleAnalyzer/CMakeLists.txt

-set(LLVM_EXPORTED_SYMBOL_FILE ${CMAKE_CURRENT_SOURCE_DIR}/SampleAnalyzerPlugin.exports)
-add_llvm_library(SampleAnalyzerPlugin MODULE MainCallChecker.cpp PLUGIN_TOOL clang)
-
-if(LLVM_ENABLE_PLUGINS AND (WIN32 OR CYGWIN))
-  target_link_libraries(SampleAnalyzerPlugin PRIVATE
-    clangAnalysis
-    clangAST
-    clangStaticAnalyzerCore
-    LLVMSupport
-    )
-endif()

clang/test/CMakeLists.txt

 if (CLANG_BUILD_EXAMPLES)
   list(APPEND CLANG_TEST_DEPS
     AnnotateFunctions
     clang-interpreter
     PrintFunctionNames
     )
 endif ()
 
-if (CLANG_ENABLE_STATIC_ANALYZER AND CLANG_BUILD_EXAMPLES)
-  list(APPEND CLANG_TEST_DEPS
-    SampleAnalyzerPlugin
-    )
-endif ()
-
 set(CLANG_TEST_PARAMS
   clang_site_config=${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg
   USE_Z3_SOLVER=0
   )
 
 set(ANALYZER_TEST_PARAMS
   USE_Z3_SOLVER=0)
 
list(APPEND CLANG_TEST_DEPS
     )
 
   if(TARGET llvm-lto)
     list(APPEND CLANG_TEST_DEPS llvm-lto)
   endif()
 endif()
 
 if (CLANG_ENABLE_STATIC_ANALYZER)
-  add_subdirectory(Analysis/plugins)
-  list(APPEND CLANG_TEST_DEPS clang-analyzer-plugin)
-
-  # check-all would launch those tests via check-clang.
-  set(EXCLUDE_FROM_ALL ON)
-
-  add_lit_testsuite(check-clang-analyzer "Running the Clang analyzer tests"
-    ${CMAKE_CURRENT_BINARY_DIR}/Analysis
-    PARAMS ${ANALYZER_TEST_PARAMS}
-    DEPENDS ${CLANG_TEST_DEPS})
-  set_target_properties(check-clang-analyzer PROPERTIES FOLDER "Clang tests")
-
-  if (LLVM_WITH_Z3)
-    add_lit_testsuite(check-clang-analyzer-z3 "Running the Clang analyzer tests, using Z3 as a solver"
-      ${CMAKE_CURRENT_BINARY_DIR}/Analysis
-      PARAMS ${ANALYZER_TEST_PARAMS_Z3}
-      DEPENDS ${CLANG_TEST_DEPS})
-    set_target_properties(check-clang-analyzer-z3 PROPERTIES FOLDER "Clang tests")
+  if (LLVM_ENABLE_PLUGINS)
+    set(CLANG_TEST_DEPS
+      SampleAnalyzerPlugin
+      CheckerDependencyHandlingAnalyzerPlugin
+      CheckerOptionHandlingAnalyzerPlugin
+      )
   endif()
-
-  set(EXCLUDE_FROM_ALL OFF)
 endif()
 
 add_custom_target(clang-test-depends DEPENDS ${CLANG_TEST_DEPS})
 set_target_properties(clang-test-depends PROPERTIES FOLDER "Clang tests")
 
 add_lit_testsuite(check-clang "Running the Clang regression tests"
   ${CMAKE_CURRENT_BINARY_DIR}
   #LIT ${LLVM_LIT}

llvm/CMakeLists.txt

 # By default, we target the host, but this can be overridden at CMake
 # invocation time.
 set(LLVM_DEFAULT_TARGET_TRIPLE "${LLVM_HOST_TRIPLE}" CACHE STRING
   "Default target for which LLVM will generate code." )
 set(TARGET_TRIPLE "${LLVM_DEFAULT_TARGET_TRIPLE}")
 message(STATUS "LLVM host triple: ${LLVM_HOST_TRIPLE}")
 message(STATUS "LLVM default target triple: ${LLVM_DEFAULT_TARGET_TRIPLE}")
 
+if(WIN32 OR CYGWIN)
+  if(BUILD_SHARED_LIBS OR LLVM_BUILD_LLVM_DYLIB)
+    set(LLVM_ENABLE_PLUGINS_default ON)
+  else()
+    set(LLVM_ENABLE_PLUGINS_default OFF)
+  endif()
+else()
+  set(LLVM_ENABLE_PLUGINS_default ON)
+endif()
+option(LLVM_ENABLE_PLUGINS "Enable plugin support" ${LLVM_ENABLE_PLUGINS_default})
+
 include(HandleLLVMOptions)
 
 # Verify that we can find a Python 2 interpreter.  Python 3 is unsupported.
 # FIXME: We should support systems with only Python 3, but that requires work
 # on LLDB.
 set(Python_ADDITIONAL_VERSIONS 2.7)
 include(FindPythonInterp)
 if( NOT PYTHONINTERP_FOUND )

llvm/cmake/modules/HandleLLVMOptions.cmake

 option(LLVM_EXPORT_SYMBOLS_FOR_PLUGINS "Export symbols from LLVM tools so that plugins can import them" OFF)
 if(BUILD_SHARED_LIBS AND LLVM_EXPORT_SYMBOLS_FOR_PLUGINS)
   message(FATAL_ERROR "BUILD_SHARED_LIBS not compatible with LLVM_EXPORT_SYMBOLS_FOR_PLUGINS")
 endif()
 if(LLVM_LINK_LLVM_DYLIB AND LLVM_EXPORT_SYMBOLS_FOR_PLUGINS)
   message(FATAL_ERROR "LLVM_LINK_LLVM_DYLIB not compatible with LLVM_EXPORT_SYMBOLS_FOR_PLUGINS")
 endif()
 
-# Plugin support
-# FIXME: Make this configurable.
-if(BUILD_SHARED_LIBS OR LLVM_BUILD_LLVM_DYLIB)
-  set(LLVM_ENABLE_PLUGINS ON)
-else()
-  set(LLVM_ENABLE_PLUGINS OFF)
-endif()
-
 # By default we should enable LLVM_ENABLE_IDE only for multi-configuration
 # generators. This option disables optional build system features that make IDEs
 # less usable.
 set(LLVM_ENABLE_IDE_default OFF)
 if (CMAKE_CONFIGURATION_TYPES)
   set(LLVM_ENABLE_IDE_default ON)
 endif()
 option(LLVM_ENABLE_IDE