This is an archive of the discontinued LLVM Phabricator instance.

Differential D24040

codechecker tool core
Needs ReviewPublic

Authored by o.gyorgy on Aug 30 2016, 10:01 AM.

Details

Reviewers
dcoughlin
zaks.anna
alexfh
rizsotto.mailinglist
xazax.hun
Summary

This is the first patch for the codechecker tool. (https://github.com/Ericsson/codechecker/issues/359)

This patch contains only the core files, documentation and unit tests.

  • libcodechecker
  • report_server (handle report storage)
  • report_viewer (handle viewer client requests (web, command line))
  • thrift api (for storing and getting results by the analyzers)
  • unit tests
  • config files (runtime and packaging)
  • command line client to view results

The next patch with functional tests have additional infrastructure requirements (PostgreSQL) which should be further discussed.
The third patch will contain the web GUI part of codechecker.

Diff Detail

Event Timeline

o.gyorgy updated this revision to Diff 69714.Aug 30 2016, 10:01 AM
o.gyorgy retitled this revision from to codechecker tool core.
o.gyorgy updated this object.
o.gyorgy added reviewers: zaks.anna, dcoughlin, artem.tamazov, alexfh, rizsotto.mailinglist, xazax.hun.
o.gyorgy added subscribers: dkrupp, baloghadamsoftware, tmsblgh.
NoQ added a subscriber: NoQ.Aug 30 2016, 10:47 AM
o.gyorgy added a subscriber: cfe-commits.Aug 31 2016, 1:49 AM
r0mai added a subscriber: r0mai.Aug 31 2016, 2:34 AM
rizsotto.mailinglist edited edge metadata.Aug 31 2016, 7:00 AM

Gyorgy and the ericsson team, thanks for doing this. very good job! good targeted functionality. i don't want to underestimate the complexity it requires, but to me this is a giant code. i do miss the explanation of the overall functional description what a module does and how it relates to other modules. i found a gap between the high level overview and the code comments.
generally speaking i found this code under documented and a bit verbose. the comments in the code is not really paint the picture that i need in order to fix a bug, or implement a feature. and it might only be my personal taste, but found that creating types (or classes this case) in a dynamically typed script language, it makes the code very verbose while it does not implement much.
was commented only on two random modules, but found the same patterns in many other ones.
found that this code is not pep8 conform. pylint found a lot of errors and warnings, and a couple of duplicate codes.
i hope we can fix these findings and that would make this code more solid and understood by others.

tools/codechecker/bin/CodeChecker
35

wow, that looks woodoo. can you explain (in code comment) why is that necessary?

67

can you explain (in code comment) why are you playing with the environment? what problem does it solve and why do you need a backup?

70

can you backport tempfile.TemporaryDirectory (from python 3.2) in one of your modules and use it inside with for proper resource guarding?

85

why not store the subprocess.Popen object and send signal via send_signal? (that would make it more portable too.)

tools/codechecker/libcodechecker/build_manager.py
30

is there any reason not to use subprocess.call instead? if you want that to be in silent, you shall pass {'stdout': subprocess.PIPE, 'stderr': subprocess.STDOUT} to it, otherwise it prints the output without you doing the copy. also noticed the shell=True which makes it very non-portable. and the command parameter is a string instead of a list. is there any reason for doing like this?

55

save-to-file and restore-from-file the environment would deserve a separate module i would say. is there any reason why it's not having those utility methods?

131

why not have a command line parameter accessor method, that would either return the value if that was given or None? i found this pattern multiple files. that would get rid of the try block completely.

150

if intercept_build_executable is None: would be more python.

169

the comment is not really helpful!
why does it need a touch? or you just create an empty one? what for?

177

logger.exception is logging the exception value implicitly at ERROR level too. but what i'm trying to say is, there is no error message in this line. you just throw the exception into the user face. can you add some explanation or instruction for the user?

artem.tamazov resigned from this revision.Aug 31 2016, 7:38 AM
artem.tamazov removed a reviewer: artem.tamazov.

It seems that my name got into reviewers list by mistake.

mehdi_amini added a subscriber: mehdi_amini.Aug 31 2016, 10:29 AM

This looks like a fairly large tool. Should it get its own "subproject level" directory in the SVN instead of being nested within clang?

tools/codechecker/README.md
50

Is it outdated?

mehdi_amini added a comment.Aug 31 2016, 10:31 AM

This looks like a fairly large tool. Should it get its own "subproject level" directory in the SVN instead of being nested within clang?

I'd add that the clang-tools-extra are more closely tied to clang than what this seems to be. Is there a strong rev-lock with clang that I missed?

szepet added a subscriber: szepet.Aug 31 2016, 11:22 PM
o.gyorgy added a comment.Sep 2 2016, 3:23 AM
In D24040#530293, @rizsotto.mailinglist wrote:

Gyorgy and the ericsson team, thanks for doing this. very good job! good targeted functionality. i don't want to underestimate the complexity it requires, but to me this is a giant code. i do miss the explanation of the overall functional description what a module does and how it relates to other modules. i found a gap between the high level overview and the code comments.

Would that help If update architecture.md documentation (which gives a high level overview) to be more in sync with the source code, with additional comments and descriptions in the modules?

generally speaking i found this code under documented and a bit verbose. the comments in the code is not really paint the picture that i need in order to fix a bug, or implement a feature. and it might only be my personal taste, but found that creating types (or classes this case) in a dynamically typed script language, it makes the code very verbose while it does not implement much.

We use classes to handle and setup environment, build, analyzer and other configurations. In the infrastructure this makes available to easily introduce new analyzers and handle the output of them. Multiple output handling is supported for each analyzer (print to stdout, store to database).

was commented only on two random modules, but found the same patterns in many other ones.
found that this code is not pep8 conform. pylint found a lot of errors and warnings, and a couple of duplicate codes.
i hope we can fix these findings and that would make this code more solid and understood by others.

I will address these issues and update our code.

Thanks for the feedback.

o.gyorgy added a comment.Sep 2 2016, 3:31 AM
In D24040#530546, @mehdi_amini wrote:

This looks like a fairly large tool. Should it get its own "subproject level" directory in the SVN instead of being nested within clang?

I'd add that the clang-tools-extra are more closely tied to clang than what this seems to be. Is there a strong rev-lock with clang that I missed?

I've put it here because it is similar to scan-build/scan-build-py and on OSX it uses the intercept-build from scan-build-py. Right now there is an alternative logger on linux (not in this patch) but we plan to move to use only the intercept-build.
We process the output generated by clang and clang-tidy and we use the command line options for these tools which could change with newer revisions, in that case this tool needs to be updated too.

I will update the readme too.

rizsotto.mailinglist added a comment.Sep 2 2016, 6:11 AM
In D24040#532718, @o.gyorgy wrote:
In D24040#530293, @rizsotto.mailinglist wrote:

Gyorgy and the ericsson team, thanks for doing this. very good job! good targeted functionality. i don't want to underestimate the complexity it requires, but to me this is a giant code. i do miss the explanation of the overall functional description what a module does and how it relates to other modules. i found a gap between the high level overview and the code comments.

Would that help If update architecture.md documentation (which gives a high level overview) to be more in sync with the source code, with additional comments and descriptions in the modules?

i prefer python docstrings instead of the markdown files. found that the __init__.py files are empty. those are good candidates for a higher overview of the package architecture. now module headers are one liners most of the places, which is a missed opportunity to document the connections between the modules. method docstrings are non descriptive enough. (echoing back the function names with spaces.) would read about the usage of the method, contract between the caller and the callee, or the implementation difficulties.

generally speaking i found this code under documented and a bit verbose. the comments in the code is not really paint the picture that i need in order to fix a bug, or implement a feature. and it might only be my personal taste, but found that creating types (or classes this case) in a dynamically typed script language, it makes the code very verbose while it does not implement much.

We use classes to handle and setup environment, build, analyzer and other configurations. In the infrastructure this makes available to easily introduce new analyzers and handle the output of them. Multiple output handling is supported for each analyzer (print to stdout, store to database).

yes, i got that you were using OO. and try to re-use code this way... my point is, this abstraction, in this language, ends up to have an abstract class with empty methods, a couple of implementations and the user of the abstract class, which enjoys the polymorphism. to understand a single implementation i have to read (at least) three different modules/files. it's very fragmented to me... my proposal would be, to write util methods. implement the different analyzer commands as a combination of the util methods. this abstraction fits better to a script language. it also make your code more dense, less verbose, easier to maintain, test or reason about... those was my two cents.

mehdi_amini added a comment.Sep 7 2016, 3:57 PM
In D24040#532719, @o.gyorgy wrote:
In D24040#530546, @mehdi_amini wrote:

This looks like a fairly large tool. Should it get its own "subproject level" directory in the SVN instead of being nested within clang?

I'd add that the clang-tools-extra are more closely tied to clang than what this seems to be. Is there a strong rev-lock with clang that I missed?

I've put it here because it is similar to scan-build/scan-build-py and on OSX it uses the intercept-build from scan-build-py. Right now there is an alternative logger on linux (not in this patch) but we plan to move to use only the intercept-build.
We process the output generated by clang and clang-tidy and we use the command line options for these tools which could change with newer revisions, in that case this tool needs to be updated too.

By strong rev-lock, I meant "Using the internal C++ API" of clang itself.
The scan-build.py thing seems to me like a tiny script compared to CodeChecker.

whisperity added a subscriber: whisperity.Jan 1 2017, 8:35 AM

Revision Contents

PathSize
tools/
codechecker/
131 lines
67 lines
bin/
106 lines
401 lines
config/
123 lines
46 lines
22 lines
13 lines
41 lines
10 lines
docs/
79 lines
6 lines
checker_docs/
3 lines
38 lines
73 lines
33 lines
images/
15 lines
71 lines
88 lines
542 lines
libcodechecker/
5 lines
197 lines
156 lines
77 lines
68 lines
analyzers/
5 lines
132 lines
131 lines
168 lines
376 lines
188 lines
114 lines
58 lines
232 lines
59 lines
36 lines
218 lines
128 lines
465 lines
147 lines
180 lines
38 lines
243 lines
cmdline_client/
5 lines
378 lines
108 lines
195 lines
485 lines
db_migrate/
80 lines
24 lines
versions/
76 lines
48 lines
24 lines
32 lines
37 lines
154 lines
27 lines
db_model/
5 lines
284 lines
21 lines
93 lines
46 lines
247 lines
39 lines
136 lines
87 lines
90 lines
423 lines
284 lines
90 lines
113 lines
171 lines
65 lines
storage_server/
5 lines
640 lines
150 lines
179 lines
391 lines
198 lines
viewer_server/
5 lines
1295 lines
223 lines
5 lines
tests/
unit/
5 lines
plist_test_files/
633 lines
633 lines
633 lines
14 lines
636 lines
642 lines
gen_plist/
9 lines
12 lines
12 lines
10 lines
17 lines
31 lines
source_suppress_test_files/
73 lines
5 lines
146 lines
209 lines
133 lines
253 lines
tidy_output_test_files/
9 lines
12 lines
files/
9 lines
14 lines
7 lines
10 lines
9 lines
122 lines
18 lines
282 lines
25 lines
425 lines
thrift_api/
104 lines
262 lines
59 lines
9 lines

Diff 69714

tools/codechecker/README.md

  • This file was added.
-----
# Introduction
CodeChecker is a static analysis infrastructure built on [Clang Static Analyzer](http://clang-analyzer.llvm.org/).
CodeChecker replaces [scan-build](http://clang-analyzer.llvm.org/scan-build.html) in Clang Static Analyzer in Linux systems.
Main features:
* store the result of multiple large analysis run results efficiently
* run multiple analyzers, currently Clang Static Analyzer and Clang-Tidy is supported
* dynamic web based defect viewer
* a PostgreSQL/SQLite based defect storage & management (both are optional, results can be shown on standard output in quickcheck mode)
* update analyzer results only for modified files (depends on the build system)
* compare analysis results (new/resolved/unresolved bugs compared to a baseline)
* filter analysis results (checker name, severity, source file name ...)
* skip analysis in specific source directories if required
* suppression of false positives (in config file or in the source)
* Thrift API based server-client model for storing bugs and viewing results.
* It is possible to connect multiple bug viewers. Currently a web-based viewer and a command line viewer are provided.
You can find a high level overview about the infrastructure in the presentation
at the [2015 Euro LLVM](http://llvm.org/devmtg/2015-04/) Conference:
__Industrial Experiences with the Clang Static Analysis Toolset
_Daniel Krupp, Gyorgy Orban, Gabor Horvath and Bence Babati___ ([ Slides](http://llvm.org/devmtg/2015-04/slides/Clang_static_analysis_toolset_final.pdf))
## Important Limitations
CodeChecker requires some new features from clang to work properly.
If your clang version does not have these features you will see in debug log the following messages:
* `Check name wasn't found in the plist file.` --> use clang = 3.7 or trunk@r228624; otherwise CodeChecker makes a guess based on the report message
* `Hash value wasn't found in the plist file.` --> update for a newer clang version; otherwise CodeChecker generates a simple hash based on the filename and the line content, this method is applied for Clang Tidy results too, because Clang Tidy does not support bug identifier hash generation currently
## Linux
For a more detailed dependency list see [Requirements](docs/deps.md)
### Basic dependecy install & setup
Tested on Ubuntu LTS 14.04.2
~~~~~~{.sh}
# get ubuntu packages
# clang-3.6 can be replaced by any later versions of clang
sudo apt-get install clang-3.6 doxygen build-essential thrift-compiler python-virtualenv gcc-multilib git wget
# create new python virtualenv
virtualenv -p /usr/bin/python2.7 ~/checker_env
# activate virtualenv
source ~/checker_env/bin/activate
# get source code
git clone https://github.com/Ericsson/codechecker.git
mehdi_aminiUnsubmitted
Not Done
ReplyInline Actions

Is it outdated?

mehdi_amini: Is it outdated?
cd codechecker
# install required basic python modules
pip install -r .ci/basic_python_requirements
# create codechecker package
./build_package.py -o ~/codechecker_package
cd ..
~~~~~~
### Check a test project
#### Check if clang or clang tidy is available
~~~~~~{.sh}
which clang
which clang-tidy
~~~~~~
If 'clang' or 'clang-tidy' commands are not available the package can be configured to use another/newer clang binary for the analisys.
Edit the 'CodeChecker/config/package_layout.json' config files "runtime/analyzers"
section in the generated package and modify the analyzers section to the analyzers
available in the PATH
```
"analyzers" : {
"clangsa" : "clang-3.6",
"clang-tidy" : "clang-tidy-3.6"
},
```
#### Activate virtualenv
~~~~~~{.sh}
source ~/checker_env/bin/activate
~~~~~~
#### Add package bin directory to PATH.
This step can be skipped if you always give the path of CodeChecker command.
~~~~~~{.sh}
export PATH=~/codechecker_package/CodeChecker/bin:$PATH
~~~~~~
#### Check the project
Check the project using SQLite. The database is placed in the working
directory which can be provided by -w flag (~/.codechecker by default).
~~~~~~{.sh}
CodeChecker check -n test_project_check -b "cd my_test_project && make clean && make"
~~~~~~
#### Start web server to view the results
~~~~~~{.sh}
CodeChecker server
~~~~~~
#### View the results with firefox
~~~~~~{.sh}
firefox http://localhost:8001
~~~~~~
See user guide for further configuration and check options.
## Additional documentations
[User guide](docs/user_guide.md)
[Use with PostgreSQL database](docs/postgresql_setup.md)
[Command line usage_examples](docs/usage.md)
[Checker documentation](docs/checker_docs.md)
[Architecture overview](docs/architecture.md)
[Requirements](docs/deps.md)
[Package layout](docs/package_layout.md)
[Thrift api](thrift_api/thrift_api.md)
[External source dependencies](docs/deps.md)
[Test documentation](tests/functional/package_test.md)
[Database schema migration](docs/db_schema_guide.md)

tools/codechecker/alembic.ini

  • This file was added.
# A generic, single database configuration.
[alembic]
# path to migration scripts
script_location = libcodechecker/db_migrate
# template used to generate migration files
# file_template = %%(rev)s_%%(slug)s
# max length of characters to apply to the
# "slug" field
#truncate_slug_length = 40
# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false
# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false
# version location specification; this defaults
# to database/versions. When using multiple version
# directories, initial revisions must be specified with --version-path
# version_locations = %(here)s/bar %(here)s/bat database/versions
# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8
sqlalchemy.url = postgres://codechecker@localhost:5432/codechecker
# Logging configuration
[loggers]
keys = root,sqlalchemy,alembic
[handlers]
keys = console
[formatters]
keys = generic
[logger_root]
level = WARN
handlers = console
qualname =
[logger_sqlalchemy]
level = WARN
handlers =
qualname = sqlalchemy.engine
[logger_alembic]
level = INFO
handlers =
qualname = alembic
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic
[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

tools/codechecker/bin/CodeChecker

  • This file was added.
PropertyOld ValueNew Value
File Modenull100755
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
used to kickstart codechecker
save original environment without modifications
used to run the logging in the same env
'''
import os
import sys
import signal
import subprocess
import pickle
import ntpath
import tempfile
import shutil
procPool = []
def run_codechecker(original_env, checker_env):
'''
run the codechecker
original_env - will be saved for later usage by the build action logger
checker_env - codechecker will be run in the checker env
'''
package_bin = os.path.dirname(os.path.realpath(__file__))
package_root, bin_dir = ntpath.split(package_bin)
python = os.path.join('python')
rizsotto.mailinglistUnsubmitted
Not Done
ReplyInline Actions

wow, that looks woodoo. can you explain (in code comment) why is that necessary?

rizsotto.mailinglist: wow, that looks woodoo. can you explain (in code comment) why is that necessary?
common_lib = os.path.join(package_root,
'cc_lib',
'python2.7')
gen_lib = os.path.join(package_root,
'cc_lib',
'python2.7',
'codechecker_gen')
checker_env['PYTHONPATH'] = common_lib + ':' + gen_lib
checker_env['CC_PACKAGE_ROOT'] = \
os.path.realpath(os.path.join(package_bin, os.pardir))
codechecker_main = os.path.join(package_root,
'cc_bin',
'CodeChecker.py')
checker_cmd = []
checker_cmd.append(python)
checker_cmd.append(codechecker_main)
checker_cmd.extend(sys.argv[1:])
proc = subprocess.Popen(checker_cmd, env=checker_env)
procPool.append(proc.pid)
proc.wait()
sys.exit(proc.returncode)
def main():
original_env = os.environ.copy()
rizsotto.mailinglistUnsubmitted
Not Done
ReplyInline Actions

can you explain (in code comment) why are you playing with the environment? what problem does it solve and why do you need a backup?

rizsotto.mailinglist: can you explain (in code comment) why are you playing with the environment? what problem does…
checker_env = original_env
tmp_dir = tempfile.mkdtemp()
rizsotto.mailinglistUnsubmitted
Not Done
ReplyInline Actions

can you backport tempfile.TemporaryDirectory (from python 3.2) in one of your modules and use it inside with for proper resource guarding?

rizsotto.mailinglist: can you backport `tempfile.TemporaryDirectory` (from python 3.2) in one of your modules and use…
original_env_file = os.path.join(tmp_dir, 'original_env.pickle')
try:
with open(original_env_file, 'wb') as env_save:
pickle.dump(original_env, env_save)
checker_env['CODECHECKER_ORIGINAL_BUILD_ENV'] = original_env_file
except Exception as ex:
print('Saving original build environment failed')
print(ex)
def signal_term_handler(sig, frame):
pid = os.getpid()
for p in procPool:
rizsotto.mailinglistUnsubmitted
Not Done
ReplyInline Actions

why not store the subprocess.Popen object and send signal via send_signal? (that would make it more portable too.)

rizsotto.mailinglist: why not store the `subprocess.Popen` object and send signal via `send_signal`? (that would make…
os.kill(p, signal.SIGINT)
# remove temporary directory
try:
shutil.rmtree(tmp_dir)
except Exception as ex:
print('Failed to remove temporary directory: ' + tmp_dir)
print('Manual cleanup is required.')
print(ex)
sys.exit(1)
signal.signal(signal.SIGTERM, signal_term_handler)
signal.signal(signal.SIGINT, signal_term_handler)
run_codechecker(original_env, checker_env)
shutil.rmtree(tmp_dir)
if __name__ == "__main__":
main()

tools/codechecker/bin/CodeChecker.py

  • This file was added.
PropertyOld ValueNew Value
File Modenull100755
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
main codechecker script
'''
import os
import sys
import signal
import argparse
import shared
from libcodechecker import logger
from libcodechecker import arg_handler
from libcodechecker import util
from libcodechecker.analyzers import analyzer_types
from libcodechecker.cmdline_client import cmd_line_client
LOG = logger.get_new_logger('MAIN')
analyzers = ' '.join(list(analyzer_types.supported_analyzers))
class OrderedCheckersAction(argparse.Action):
'''
Action to store enabled and disabled checkers
and keep ordering from command line
Create separate lists based on the checker names for
each analyzer
'''
def __init__(self, option_strings, dest, nargs=None, **kwargs):
if nargs is not None:
raise ValueError("nargs not allowed")
super(OrderedCheckersAction, self).__init__(option_strings, dest, **kwargs)
def __call__(self, parser, namespace, value, option_string=None):
if 'ordered_checkers' not in namespace:
setattr(namespace, 'ordered_checkers', [])
ordered_checkers = namespace.ordered_checkers
if self.dest == 'enable':
ordered_checkers.append((value, True))
else:
ordered_checkers.append((value, False))
setattr(namespace, 'ordered_checkers', ordered_checkers)
# ------------------------------------------------------------------------------
class DeprecatedOptionAction(argparse.Action):
'''
Deprecated argument action
'''
def __init__(self,
option_strings,
dest,
nargs=None,
const=None,
default=None,
type=None,
choices=None,
required=False,
help=None,
metavar=None):
super(DeprecatedOptionAction, self).__init__(option_strings,
dest,
nargs='?',
const='deprecated_option',
default=None,
type=None,
choices=None,
required=False,
help='DEPRECATED argument!',
metavar='DEPRECATED')
def __call__(self, parser, namespace, value=None, option_string=None):
LOG.warning("Deprecated command line option in use: '" + option_string + "'")
def add_database_arguments(parser):
'''Helper method for adding database arguments to an argument parser.'''
parser.add_argument('--sqlite', action=DeprecatedOptionAction)
parser.add_argument('--postgresql', dest="postgresql",
action='store_true', required=False,
help='Use PostgreSQL database.')
parser.add_argument('--dbport', type=int, dest="dbport",
default=5432, required=False,
help='Postgres server port.')
# WARNING dbaddress default value influences workspace creation (SQLite)
parser.add_argument('--dbaddress', type=str, dest="dbaddress",
default="localhost", required=False,
help='Postgres database server address')
parser.add_argument('--dbname', type=str, dest="dbname",
default="codechecker", required=False,
help='Name of the database.')
parser.add_argument('--dbusername', type=str, dest="dbusername",
default='codechecker', required=False,
help='Database user name.')
def add_analyzer_arguments(parser):
"""
analyzer related arguments
"""
parser.add_argument('-e', '--enable',
default=argparse.SUPPRESS,
action=OrderedCheckersAction,
help='Enable checker.')
parser.add_argument('-d', '--disable',
default=argparse.SUPPRESS,
action=OrderedCheckersAction,
help='Disable checker.')
parser.add_argument('--keep-tmp', action="store_true",
dest="keep_tmp", required=False,
help='''\
Keep temporary report files generated during the analysis.''')
parser.add_argument('--analyzers', nargs='+',
dest="analyzers", required=False,
default=[analyzer_types.CLANG_SA, analyzer_types.CLANG_TIDY],
help="""Select which analyzer should be enabled.\nCurrently supported analyzers are: """ + analyzers + """\ne.g. '--analyzers """ + analyzers +"'")
parser.add_argument('--saargs', dest="clangsa_args_cfg_file",
required=False, default=argparse.SUPPRESS,
help='''\
File with arguments which will be forwarded directly to the Clang static analyzer without modifiaction''')
parser.add_argument('--tidyargs', dest="tidy_args_cfg_file",
required=False, default=argparse.SUPPRESS,
help='''\
File with arguments which will be forwarded directly to the Clang tidy analyzer without modifiaction''')
# ------------------------------------------------------------------------------
def main():
'''
codechecker main command line
'''
def signal_handler(sig, frame):
'''
Without this handler the postgreSql
server does not terminated at signal
'''
sys.exit(1)
signal.signal(signal.SIGINT, signal_handler)
signal.signal(signal.SIGTERM, signal_handler)
try:
parser = argparse.ArgumentParser(
prog='CodeChecker',
formatter_class=argparse.RawDescriptionHelpFormatter,
description='''
Run the CodeChecker source analyzer framework.
See the subcommands for specific features.''',
epilog='''
Example usage:
--------------
Analyzing a project with default settings:
CodeChecker check -w ~/workspace -b "cd ~/myproject && make" -n myproject
Start the viewer to see the results:
CodeChecker server -w ~/workspace
See the results in a web browser: localhost:8001
See results in the command line: CodeChecker cmd results -p 8001 -n myproject
To analyze a small project quickcheck feature can be used.
The results will be printed only to the standard output. (No database will be used)
CodeChecker quickcheck -w ~/workspace -b "cd ~/myproject && make"
''')
subparsers = parser.add_subparsers(help='commands')
workspace_help_msg = """Directory where the codechecker can store analysis related data."""
name_help_msg = """Name of the analysis."""
jobs_help_msg = """Number of jobs. Start multiple processes for faster analisys""";
log_argument_help_msg="""Path to the log file which is created during the build. \nIf there is an already generated log file with the compilation commands\ngenerated by 'CodeChecker log' or 'cmake -DCMAKE_EXPORT_COMPILE_COMMANDS' \nCodechecker check can use it for the analisys in that case running the original build will \nbe left out from the analysis process (no log is needed)."""
# --------------------------------------
# check commands
check_parser = subparsers.add_parser('check',
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
help='''\
Run the supported source code analyzers on a project''')
check_parser.add_argument('-w', '--workspace', type=str,
default=util.get_default_workspace(),
dest="workspace",
help=workspace_help_msg)
check_parser.add_argument('-n', '--name', type=str,
dest="name", required=True,
default=argparse.SUPPRESS,
help=name_help_msg)
checkgroup = check_parser.add_mutually_exclusive_group(required=True)
checkgroup.add_argument('-b', '--build', type=str, dest="command",
default=argparse.SUPPRESS,
required=False, help='''\
Build command which is used to build the project''')
checkgroup.add_argument('-l', '--log', type=str, dest="logfile",
default=argparse.SUPPRESS,
required=False,
help=log_argument_help_msg)
check_parser.add_argument('-j', '--jobs', type=int, dest="jobs",
default=1, required=False,
help=jobs_help_msg)
check_parser.add_argument('-u', '--suppress', type=str, dest="suppress",
default=argparse.SUPPRESS,
required=False, help="""Path to suppress file. \nSuppress file can be used to suppress analysis results during the analisys.\nIt is based on the bug identifier generated by the compiler which is experimental.\nDo not depend too much on this file because identifier or file format can be changed.\nFor other in source suppress features see the user guide.""")
check_parser.add_argument('-c', '--clean',
default=argparse.SUPPRESS,
action=DeprecatedOptionAction)
check_parser.add_argument('--update', action=DeprecatedOptionAction,
dest="update", default=False, required=False,
help='''\
Incremental parsing, update the results of a previous run.
Only the files changed since the last build will be reanalyzed. Depends on the build system.''')
check_parser.add_argument('--force', action="store_true",
dest="force", default=False, required=False,
help='''\
Delete analisys results form the database if a run with the given name already exists''')
check_parser.add_argument('-s', '--skip', type=str, dest="skipfile",
default=argparse.SUPPRESS,
required=False, help='Path to skip file.')
add_analyzer_arguments(check_parser)
add_database_arguments(check_parser)
check_parser.set_defaults(func=arg_handler.handle_check)
# --------------------------------------
# quickcheck commands
qcheck_parser = subparsers.add_parser('quickcheck',
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
help='''\
Run CodeChecker for a project without database.''')
qcheckgroup = qcheck_parser.add_mutually_exclusive_group(required=True)
qcheckgroup.add_argument('-b', '--build', type=str, dest="command",
default=argparse.SUPPRESS,
required=False, help='Build command.')
qcheckgroup.add_argument('-l', '--log', type=str, dest="logfile",
required=False,
default=argparse.SUPPRESS,
help=log_argument_help_msg)
qcheck_parser.add_argument('-s', '--steps', action="store_true",
dest="print_steps", help='Print steps.')
add_analyzer_arguments(qcheck_parser)
qcheck_parser.set_defaults(func=arg_handler.handle_quickcheck)
# --------------------------------------
# log commands
logging_parser = subparsers.add_parser('log',
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
help='''\
Runs the given build command. During the build the compilation commands are collected and
stored into a compilation command json file (no analisys is done during the build).''')
logging_parser.add_argument('-o', '--output', type=str, dest="logfile",
default=argparse.SUPPRESS,
required=True, help='Path to the log file.')
logging_parser.add_argument('-b', '--build', type=str, dest="command",
default=argparse.SUPPRESS,
required=True, help='Build command.')
logging_parser.set_defaults(func=arg_handler.handle_log)
# --------------------------------------
# checkers parser
checkers_parser = subparsers.add_parser('checkers',
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
help="""List the available checkers for the supported analyzers and show their default status (+ for being enabled, - for being disabled by default).""")
checkers_parser.add_argument('--analyzers', nargs='+',
dest="analyzers", required=False,
help="""Select which analyzer checkers should be listed.\nCurrently supported analyzers:\n""" + analyzers)
checkers_parser.set_defaults(func=arg_handler.handle_list_checkers)
# --------------------------------------
# server
server_parser = subparsers.add_parser('server',
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
help='Start the codechecker web server.')
server_parser.add_argument('-w', '--workspace', type=str,
dest="workspace",
default=util.get_default_workspace(),
help=workspace_help_msg)
server_parser.add_argument('-v', '--view-port', type=int, dest="view_port",
default=8001, required=False,
help='Port used for viewing.')
server_parser.add_argument('-u', '--suppress', type=str, dest="suppress",
required=False, help='Path to suppress file.')
server_parser.add_argument('--not-host-only', action="store_true",
dest="not_host_only",
help='Viewing the results is possible not \
only by browsers or clients started locally.')
server_parser.add_argument('--check-port', type=int, dest="check_port",
default=None, required=False,
help='Port used for checking.')
server_parser.add_argument('--check-address', type=str,
dest="check_address", default="localhost",
required=False, help='Server address.')
add_database_arguments(server_parser)
server_parser.set_defaults(func=arg_handler.handle_server)
# --------------------------------------
# cmd_line
cmd_line_parser = subparsers.add_parser('cmd',
help='Command line client')
cmd_line_client.register_client_command_line(cmd_line_parser)
# --------------------------------------
# debug parser
debug_parser = subparsers.add_parser('debug',
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
help="""Generate gdb debug dump files for all the failed compilation commands in the last analyzer run.\nRequires a database with the failed compilation commands""")
debug_parser.add_argument('-w', '--workspace', type=str,
dest="workspace",
default=util.get_default_workspace(),
help=workspace_help_msg)
debug_parser.add_argument('-f', '--force', action="store_true",
dest="force", required=False, default=False,
help='Overwrite already generated files.')
add_database_arguments(debug_parser)
debug_parser.set_defaults(func=arg_handler.handle_debug)
# --------------------------------------
# plist parser
plist_parser = subparsers.add_parser('plist',
formatter_class=argparse.ArgumentDefaultsHelpFormatter,
help='Parse plist files in the given directory.')
plist_parser.add_argument('-w', '--workspace', type=str,
dest="workspace",
default=util.get_default_workspace(),
help=workspace_help_msg)
plist_parser.add_argument('-n', '--name', type=str,
dest="name", required=True,
default=argparse.SUPPRESS,
help=name_help_msg)
plist_parser.add_argument('-d', '--directory', type=str,
dest="directory", required=True,
help='Path of a directory containing plist files to parse.')
plist_parser.add_argument('-j', '--jobs', type=int, dest="jobs",
default=1, required=False,
help=jobs_help_msg)
plist_parser.add_argument('-s', '--steps', action="store_true",
dest="print_steps", help='Print steps.')
plist_parser.add_argument('--stdout', action="store_true", dest="stdout",
required=False, default=False,
help="Print results to stdout instead of database.")
plist_parser.add_argument('--force', action="store_true",
dest="force", default=False, required=False,
help='''\
Delete analisys results form the database if a run with the given name already exists''')
add_database_arguments(plist_parser)
plist_parser.set_defaults(func=arg_handler.handle_plist)
# --------------------------------------
# package version info
version_parser = subparsers.add_parser('version',
help='Print package version information.')
version_parser.set_defaults(func=arg_handler.handle_version_info)
args = parser.parse_args()
args.func(args)
except KeyboardInterrupt as kb_err:
LOG.info(str(kb_err))
LOG.info("Interupted by user...")
sys.exit(1)
except shared.ttypes.RequestFailed as thrift_ex:
LOG.info("Server error.")
LOG.info("Error code: " + str(thrift_ex.error_code))
LOG.info("Error message: " + str(thrift_ex.message))
sys.exit(1)
# Handle all exception, but print stacktrace. It is needed for atexit.
# atexit does not work correctly when an unhandled exception occured.
# So in this case, the servers left running when the script exited.
except Exception:
import traceback
traceback.print_exc(file=sys.stdout)
sys.exit(2)
# ------------------------------------------------------------------------------
if __name__ == "__main__":
LOG.debug(sys.path)
LOG.debug(sys.version)
LOG.debug(sys.executable)
LOG.debug(os.environ.get('LD_LIBRARY_PATH'))
main()

tools/codechecker/config/checker_severity_map.json

  • This file was added.
{
"security.FloatLoopCounter": "MEDIUM",
"security.insecureAPI.UncheckedReturn": "MEDIUM",
"security.insecureAPI.getpw": "MEDIUM",
"security.insecureAPI.gets": "MEDIUM",
"security.insecureAPI.mkstemp": "MEDIUM",
"security.insecureAPI.mktemp": "MEDIUM",
"security.insecureAPI.rand": "MEDIUM",
"security.insecureAPI.strcpy": "MEDIUM",
"security.insecureAPI.vfork": "MEDIUM",
"unix.API": "MEDIUM",
"unix.Malloc": "MEDIUM",
"unix.MallocSizeof": "MEDIUM",
"unix.MismatchedDeallocator": "MEDIUM",
"unix.Vfork": "MEDIUM",
"unix.cstring.BadSizeArg": "MEDIUM",
"unix.cstring.NullArg": "MEDIUM",
"nullability.NullPassedToNonnull": "HIGH",
"nullability.NullReturnedFromNonnull": "HIGH",
"nullability.NullableDereferenced": "MEDIUM",
"nullability.NullablePassedToNonnull": "MEDIUM",
"nullability.NullablePassedToNonnull": "MEDIUM",
"core.CallAndMessage": "HIGH",
"core.DivideZero": "HIGH",
"core.DynamicTypePropagation": "MEDIUM",
"core.NonNullParamChecker": "HIGH",
"core.NullDereference": "HIGH",
"core.StackAddressEscape": "HIGH",
"core.UndefinedBinaryOperatorResult": "MEDIUM",
"core.VLASize": "MEDIUM",
"core.builtin.BuiltinFunctions": "MEDIUM",
"core.builtin.NoReturnFunctions": "MEDIUM",
"core.uninitialized.ArraySubscript": "MEDIUM",
"core.uninitialized.Assign": "MEDIUM",
"core.uninitialized.Branch": "MEDIUM",
"core.uninitialized.CapturedBlockVariable": "MEDIUM",
"core.uninitialized.UndefReturn": "HIGH",
"cplusplus.NewDelete": "HIGH",
"cplusplus.NewDeleteLeaks": "HIGH",
"deadcode.DeadStores": "LOW",
"llvm.Conventions": "LOW",
"cert-dcl03-c": "MEDIUM",
"cert-dcl50-cpp": "LOW",
"cert-dcl54-cpp": "MEDIUM",
"cert-dcl59-cpp": "MEDIUM",
"cert-err52-cpp": "LOW",
"cert-err60-cpp": "MEDIUM",
"cert-err61-cpp": "MEDIUM",
"cert-fio38-c": "HIGH",
"cert-oop11-cpp": "MEDIUM",
"cppcoreguidelines-c-copy-assignment-signature": "MEDIUM",
"cppcoreguidelines-pro-bounds-array-to-pointer-decay": "LOW",
"cppcoreguidelines-pro-bounds-pointer-arithmetic": "LOW",
"cppcoreguidelines-pro-type-const-cast": "LOW",
"cppcoreguidelines-pro-type-cstyle-cast": "LOW",
"cppcoreguidelines-pro-type-reinterpret-cast": "LOW",
"cppcoreguidelines-pro-type-static-cast-downcast": "LOW",
"cppcoreguidelines-pro-type-union-access": "LOW",
"cppcoreguidelines-pro-type-vararg": "LOW",
"google-build-explicit-make-pair": "MEDIUM",
"google-build-namespaces": "MEDIUM",
"google-build-using-namespace": "LOW",
"google-explicit-constructor": "MEDIUM",
"google-global-names-in-headers": "HIGH",
"google-readability-braces-around-statements": "LOW",
"google-readability-casting": "LOW",
"google-readability-function-size": "LOW",
"google-readability-namespace-comments": "LOW",
"google-readability-redundant-smartptr-get": "MEDIUM",
"google-readability-todo": "LOW",
"google-runtime-int": "LOW",
"google-runtime-member-string-references": "LOW",
"google-runtime-memset": "HIGH",
"google-runtime-operator": "MEDIUM",
"llvm-header-guard": "LOW",
"llvm-include-order": "LOW",
"llvm-namespace-comment": "LOW",
"llvm-twine-local": "LOW",
"misc-argument-comment": "LOW",
"misc-assert-side-effect": "MEDIUM",
"misc-assign-operator-signature": "MEDIUM",
"misc-bool-pointer-implicit-conversion": "LOW",
"misc-inaccurate-erase": "HIGH",
"misc-inefficient-algorithm": "MEDIUM",
"misc-macro-parentheses": "MEDIUM",
"misc-macro-repeated-side-effects": "MEDIUM",
"misc-move-const-arg": "MEDIUM",
"misc-move-constructor-init": "MEDIUM",
"misc-new-delete-overloads": "MEDIUM",
"misc-noexcept-move-constructor": "MEDIUM",
"misc-non-copyable-objects": "HIGH",
"misc-sizeof-container": "HIGH",
"misc-static-assert": "MEDIUM",
"misc-swapped-arguments": "HIGH",
"misc-throw-by-value-catch-by-reference": "MEDIUM",
"misc-undelegated-constructor": "MEDIUM",
"misc-uniqueptr-reset-release": "MEDIUM",
"misc-unused-alias-decls": "LOW",
"misc-unused-parameters": "LOW",
"misc-unused-raii": "HIGH",
"modernize-loop-convert": "LOW",
"modernize-make-unique": "LOW",
"modernize-pass-by-value": "LOW",
"modernize-redundant-void-arg": "LOW",
"modernize-replace-auto-ptr": "LOW",
"modernize-shrink-to-fit": "LOW",
"modernize-use-auto": "LOW",
"modernize-use-default": "LOW",
"modernize-use-nullptr": "LOW",
"modernize-use-override": "LOW",
"readability-braces-around-statements": "LOW",
"readability-container-size-empty": "LOW",
"readability-else-after-return": "LOW",
"readability-function-size": "LOW",
"readability-identifier-naming": "LOW",
"readability-implicit-bool-cast": "LOW",
"readability-inconsistent-declaration-parameter-name": "LOW",
"readability-named-parameter": "LOW",
"readability-redundant-smartptr-get": "LOW",
"readability-redundant-string-cstr": "LOW",
"readability-simplify-boolean-expr": "MEDIUM",
"readability-uniqueptr-delete-release": "LOW"
}

tools/codechecker/config/config.json

  • This file was added.
{
"environment_variables": {
"env_package_root" : "CC_PACKAGE_ROOT",
"env_verbose_name" : "CC_VERBOSE_LEVEL",
"env_alchemy_verbose_name" : "CC_ALCHEMY_LOG_LEVEL",
"env_path" : "PATH",
"env_ld_lib_path" : "LD_LIBRARY_PATH",
"cc_logger_bin" : "CC_LOGGER_BIN",
"cc_logger_file" : "CC_LOGGER_FILE",
"cc_logger_compiles" : "CC_LOGGER_GCC_LIKE",
"ld_preload" : "LD_PRELOAD",
"codechecker_enable_check" : "CODECHECKER_ENABLE_CHECK",
"codechecker_disable_check" : "CODECHECKER_DISABLE_CHECK",
"codechecker_workspace" : "CODECHECKER_WORKSPACE"
},
"package_variables": {
"default_db_username" : "codechecker",
"pgsql_data_dir_name" : "pgsql_data",
"path_dumps_name" : "dumps"
},
"checker_config": {
"clangsa_checkers" : [
{"core" : true },
{"unix" : true },
{"deadcode" : true },
{"cplusplus" : true },
{"security.insecureAPI.UncheckedReturn" : true },
{"security.insecureAPI.getpw" : true },
{"security.insecureAPI.gets" : true },
{"security.insecureAPI.mktemp" : true },
{"security.insecureAPI.mkstemp" : true },
{"security.insecureAPI.vfork" : true }
],
"clang-tidy_checkers" : [
{"clang-diagnostic-" : true },
{"cert-fio38-c" : true },
{"google-global-names-in-headers" : true },
{"google-runtime-memset" : true },
{"misc-inaccurate-erase" : true },
{"misc-non-copyable-objects" : true },
{"misc-sizeof-container" : true },
{"misc-swapped-arguments" : true },
{"misc-unused-raii" : true }
]
}
}

tools/codechecker/config/config.md

  • This file was added.
# Package configuration
### Checker severity map
checker_severity_map.json file contains a mapping between a
checker name and a severity level. Severity levels can be found in the shared.thrift file.
### Package configuration
* environment variables section
Contains enviroment variable names set and used during the static analysis
* package variables section
Default database username which will be used to initialize postgres database.
* checker config section
+ checkers
This section contains the default checkers set used for analysis.
The order of the checkers will be kept. (To enable set to true, to disable set to false)
### gdb script
Contains an automated gdb script which can be used for debug. In debug mode the failed build commands will be rerun with gdb.
### version
Version file contains version information for the package

tools/codechecker/config/gdbScript.gdb

  • This file was added.
set follow-fork-mode child
set print pretty on
set print array on
set print demangle on
run
where
backtrace full
frame
info args
info locals
show print demangle
list
quit

tools/codechecker/config/package_layout.json

  • This file was added.
{
"static" : {
"bin": "bin",
"cc_bin": "cc_bin",
"lib": "lib",
"plugin": "plugin",
"www": "www",
"docs": "www/docs",
"checker_md_docs": "www/docs/checker_md_docs",
"web_client": "www/scripts/codechecker-api",
"web_client_plugins": "www/scripts/plugins",
"web_client_codemirror": "www/scripts/plugins/codemirror",
"web_client_dojo": "www/scripts/plugins/dojo",
"web_client_highlightjs": "www/scripts/plugins/highlightjs",
"web_client_jsplumb": "www/scripts/plugins/jsplumb",
"web_client_marked": "www/scripts/plugins/marked",
"js_thrift": "www/scripts/plugins/thrift",
"config": "config",
"ld_logger": "ld_logger",
"codechecker_modules": "cc_lib/python2.7",
"libcodechecker": "cc_lib/python2.7/libcodechecker",
"codechecker_gen": "cc_lib/python2.7/codechecker_gen",
"codechecker_db_model": "cc_lib/python2.7/db_model",
"codechecker_db_migrate": "cc_lib/python2.7/db_migrate",
"storage_server_modules": "cc_lib/python2.7/storage_server",
"viewer_server_modules": "cc_lib/python2.7/viewer_server",
"cmdline_client": "cc_lib/python2.7/cmdline_client"
},
"runtime" : {
"analyzers" : {
"clangsa" : "clang",
"clang-tidy" : "clang-tidy"
},
"ld_logger_bin" : "bin/ldlogger",
"ld_logger_lib_name" : "ldlogger.so",
"ld_logger_lib_path" : "ld_logger/lib",
"version_file" : "config/version.json",
"checkers_severity_map_file" : "config/checker_severity_map.json",
"gdb_config_file" : "config/gdbScript.gdb"
}
}

tools/codechecker/config/version.json

  • This file was added.
{
"version":{
"major" : "5",
"minor" : "4"
},
"db_version":{
"major" : "5",
"minor" : "0"
}
}

tools/codechecker/docs/architecture.md

  • This file was added.
# Architecture overview
```
Architecture
.---------.
| Web |
.-------------. .----------. | browser |
| Buildaction | ---> .----------. | client |
| job list | ---> .----------.------. '---------'.---------.
| | ---> | Analyzer |------| ^ | Command |
'-------------' | client |------| |---->| line |
^ '----------' | | | client |
| | | '---------'
| Binary protocol Json protocol
.-----------------. | |
| Compilation | v v
| database (json) | .------------.------------.
'-----------------' | Thrift API | Thrift API |
^ '------------'------------'
| | Report | Report |
| | storage | viewer |
.---------------. | server | server |
| Buildlogger / | '------------'------------'
| CMake | ^ ^
'---------------' | |
SQLAlchemy ORM
| |
v v
SQLite/PostgreSQL
_.-----._
.- -.
|-_ _-|
| ~-----~ |
| |
`._ _.'
"-----"
```
## Buildlogger / CMake
## Buildaction job list
Based on the Compilation database a Buildaction is created for each analyzer.
## Analyzer client
Multiple analyzer clients can run parallel.
Each Analyzer client:
- processes one Buildaction
- constructs the analysis command
- runs an analyzer
- postprocesses analysis results if required
- sends the analisys results trough Thift binary protocol to the Report storage server
In quickcheck mode the results are only printed to the standard output. Nothing is stored into a database.
## Report Storage server
- Provides a Thrift API for result storage.
- Stores the analysis results to the database.
- Detects duplicate results (result in header file detected by multiple analyzer runs).
- Handles that source file contents are stored only once.
- Uses SQLAlchemy to connect to a database backend.
## Database
- Store multiple analyzer run results.
- Data can be used to generate analysis statistics.
## Report viewer server
- Multiple clients can connect simultaneously to the report viewer server.
- Uses SQLAlchemy to connect to a database backend.
- Provides a simple https webserver to view documentation.
## Command line client
- Simple client to view/compare results.
- Can be used for automated scripts for result processing.
- Provides plaintext and json output.
## Web browser client
- Client to view/filter/compare analysis results.
- Results are dynamically rendered based on the database content.

tools/codechecker/docs/checker_docs.md

  • This file was added.
Checker documentation
----------------------
Checker doc map is required by the viewer server to show documentation for each checker.
This map contains a checker name and the corresponding md documentation file name.
\include checker_docs/checker_doc_map.json

tools/codechecker/docs/checker_docs/checker_doc_map.json

  • This file was added.
{
"core.DivideZero": "core_division_by_zero.md"
}

tools/codechecker/docs/checker_docs/core_division_by_zero.md

  • This file was added.
DivisionByZero
================
Description
-------------
This checker finds bugs when division by zero happens.
Example
-------------
~~~~~~{.cpp}
int doubleOrNull(int a)
{
return a > 0 ? 2*a : 0;
}
void f()
{
int a = 0.0;
int b = doubleOrNull(a);
a = a / b; // Division by zero
}
~~~~~~
Configuration
-------------
None
Limitations
-------------
None
Solution
-------------
Check the divisor for zero.
For example, here is the fix for the previous code.
if (b != 0) a = a / b;

tools/codechecker/docs/db_schema_guide.md

  • This file was added.
# How to modify the database schema
CodeChecker is developed in rolling release model so it is important to update
the database schema in a backward compatible way. This is achieved using the
[Alembic](http://alembic.readthedocs.org/en/latest/index.html) database
migration tool.
This is a step-by-step guide, how to modify the schema in a backward compatible
way using migration scripts.
## Step 1: Update the database model
CodeChecker uses [SQLAlchemy](http://www.sqlalchemy.org/) for database
operations. You can find the database model in libcodechecker/db_model/orm_model.py file. When
you want to modify the database schema you should update this file first.
Please read [SQLAlchemy declarative syntax documentation](http://docs.sqlalchemy.org/en/latest/orm/extensions/declarative/)
for syntax and semantics.
## Step 2: Write a migration script
When you change the database schema, it's essential to write a migration script.
When you start CodeChecker with an existing database, it will automatically
migrate the schema to the latest version using Alembic migration scripts.
You can write the migration script manually or you can use Alembic's
'autogenerate' feature.
### Step 2.A: Generating migration scripts using autogenerate
Alembic can compare the table metadata against the status of a database and
based on this comparison it generates the migration script. Even though this
method has it's [limitations](
http://alembic.readthedocs.org/en/latest/autogenerate.html#what-does-autogenerate-detect-and-what-does-it-not-detect)
, in most cases it works well.
To generate a migration script, do the following steps:
1. Start a database with the original, unmodified schema.
2. Go to CodeChecker's source root
3. Edit the sqlalchemy.url option in [alembic.ini](
http://alembic.readthedocs.org/en/latest/tutorial.html#editing-the-ini-file)
according to your database configuration.
4. Generate the migration script using alembic:
```
alembic revision --autogenerate -m "Change description"
```
5. The new migration script db_migrate/versions/{hash}_change_description.py is
generated. **You must always check the generated script because sometimes it
isn't correct.**
6. Run all test cases. **All unit tests must pass**.
7. Don't forget to commit the migration script with your other changes.
Further reading:
- [Auto Generating Migrations](http://alembic.readthedocs.org/en/latest/autogenerate.html)
- [Alembic tutorial](http://alembic.readthedocs.org/en/latest/tutorial.html)
- [Alembic Operation Reference](http://alembic.readthedocs.org/en/latest/ops.html)
### Step 2.B: Writing migration scripts by hand
Navigate to the root directory of CodeChecker source code and create an empty
migration script using `alembic revision`:
```
alembic revision -m "Change description"
```
The new file db_migrate/versions/{hash}_change_description.py is generated. This
file contains an empty `upgrade` and a `downgrade` function. You should always
implement the `upgrade` function. Downgrading is not supported right now.
You should also read the [Alembic tutorial](http://alembic.readthedocs.org/en/latest/tutorial.html#create-a-migration-script)
and the [Operation Reference](http://alembic.readthedocs.org/en/latest/ops.html)
for details.

tools/codechecker/docs/deps.md

  • This file was added.
### Packaging requirements
* [Git](https://git-scm.com/) (> 1.9.1)
* [Thrift compiler](https://thrift.apache.org/) (> 0.9.2) required to generate python and javascript files
* [Doxygen](http://www.stack.nl/~dimitri/doxygen/) (> 1.8) to generate documentation
* Build logging
- It is possible to build package without the ld-logger. In that case no automatic compilation database generation is done. To build ld-logger 32 and 64 bit versions automatically, `gcc multilib` and `make` is required.
- Compilation command database can be generated with CMake during the build (run `cmake` with the 'CMAKE_EXPORT_COMPILE_COMMANDS' option). CodeChecker can process the generated compilation database at runtime.
* Other external dependencies are automatically downloaded and
copied to the necessary directories in the package.
### Runtime requirements
#### Basic
Javascript dependencies are automatically downloaded based on the ext_source_deps_config.json file during package build.
* [Clang Static analyzer](http://clang-analyzer.llvm.org/) (latest stable or [trunk](http://clang.llvm.org/get_started.html))
* [Clang Tidy](http://clang.llvm.org/extra/clang-tidy/) (available in the clang tools extra repository [trunk](http://clang.llvm.org/get_started.html))
* [Python2](https://www.python.org/) (> 2.7)
* [Alembic](https://pypi.python.org/pypi/alembic) (>= 0.8.2) database migration support is available only for PostgreSQL database
* [SQLAlchemy](http://www.sqlalchemy.org/) (>= 1.0.9) Python SQL toolkit and Object Relational Mapper, for supporting multiple database backends
- [PyPi SQLAlchemy](https://pypi.python.org/pypi/SQLAlchemy) (> 1.0.2)
* Thrift python modules. Cross-language service building framework to handle data transfer for report storage and result viewer clients
- [PyPi thrift](https://pypi.python.org/pypi/thrift/0.9.2)(> 0.9.2 )
* [Codemirror](https://codemirror.net/) (MIT) - view source code in the browser
* [Jsplumb](https://jsplumbtoolkit.com/) (community edition, MIT) - draw bug paths
* [Marked](https://github.com/chjj/marked) (BSD) - view documentation for checkers written in markdown (generated dynamically)
* [Dojotoolkit](https://dojotoolkit.org/) (BSD) - main framework for the web UI
* [Highlightjs](https://highlightjs.org/) (BSD) - required for highlighting the source code
#### PostgreSQL
For the additional PostgreSQL dependecies see the [PostgreSQL setup](postgresql_setup.md) documentation.
\include ext_source_deps_config.json

tools/codechecker/docs/images/viewer.png

  • This binary file was added.

tools/codechecker/docs/package_layout.md

  • This file was added.
# Codechecker package builder
Short description for the package layout of the generic codechecker package.
Package creation is based on the package layout config file.
It has two main parts a static and a runtime part.
## Static section
Static part is used to create the main package skeleton (directory structure) where the CodeChecker finds the required files.
### External checker libraries
External checker libraries can be used in the package. The shared object files should be in the plugin directory and will be automatically loaded at runtime.
## Runtime section
The runtime part contains informations which will be used only at runtime
to find files during the checking process.

tools/codechecker/docs/postgresql_setup.md

  • This file was added.
## PostgreSQL
## Extra runtime requirements for PostgreSQL database support
* [PostgreSQL](http://www.postgresql.org/ "PostgreSQL") (> 9.3.5) (optional)
* [psycopg2](http://initd.org/psycopg/ "psycopg2") (> 2.5.4) or [pg8000](https://github.com/mfenniak/pg8000 "pg8000") (>= 1.10.0) at least one database connector is required for postgreSQL database support (both are supported)
- [PyPi psycopg2](https://pypi.python.org/pypi/psycopg2/2.6.1) __requires lbpq!__
- [PyPi pg8000](https://pypi.python.org/pypi/pg8000)
## Install & setup additional dependencies
Tested on Ubuntu LTS 14.04.2
~~~~~~{.sh}
# get the extra postgresql packages
sudo apt-get install libpq-dev python-dev postgresql postgresql-client-common postgresql-common
# Note: The following PostgreSQL specific steps are only needed when PostgreSQL
# is used for checking. By default CodeChecker uses SQLite.
# setup database for a test_user
sudo -i -u postgres
# add a test user with "test_pwd" password
createuser --createdb --login --pwprompt test_user
exit
# PostgreSQL authentication
# PGPASSFILE environment variable should be set to a pgpass file
# For format and further information see PostgreSQL documentation:
# http://www.postgresql.org/docs/current/static/libpq-pgpass.html
echo "*:5432:*:test_user:test_pwd" >> ~/.pgpass
chmod 0600 ~/.pgpass
# activate the already created virtualenv in the basic setup
source ~/checker_env/bin/activate
# install required python modules
pip install -r .ci/python_requirements
# create codechecker package
./build_package.py -o ~/codechecker_package
cd ..
~~~~~~
## Run CodeChecker
Activate virtualenv.
~~~~~~{.sh}
source ~/checker_env/bin/activate
~~~~~~
Add package bin directory to PATH.
This step can be skipped if you always give the path of CodeChecker command.
~~~~~~{.sh}
export PATH=~/codechecker_package/CodeChecker/bin:$PATH
~~~~~~
Check a test project.
~~~~~~{.sh}
CodeChecker check --dbusername test_user --postgresql -n test_project_check -b "cd my_test_project && make clean && make"
~~~~~~
Start web server to view the results.
~~~~~~{.sh}
CodeChecker server --dbusername test_user --postgresql
~~~~~~
View the results with firefox.
~~~~~~{.sh}
firefox http://localhost:8001
~~~~~~

tools/codechecker/docs/usage.md

  • This file was added.
#CodeChecker command line examples
###1. Quickly check some files and print results in command line
```
CodeChecker quickcheck -b make
```
###2. Check a project, update earlier results and view results from a browser
Runs make, logs build, run analyzers and store the results in sqlite db.
```
CodeChecker check -b make
```
Start webserver, which listens on default port localhost:8001
results can be viewed in a browser
```
CodeChecker server
```
The developer may also suppress false positives
At the end, the project can be rechecked.
```
CodeChecker check make
```
###3. Same as use case 2., but the developer would like to enable alpha checkers and llvm checkers
```
CodeChecker check -e alpha -e llvm -b make
```
###4. Same as use case 2., but the developer stores suppressed false positives in a text file that is checked in into version control
```
CodeChecker check –u /home/myuser/myproject/suppress.list -b make
CodeChecker server –u /home/myuser/myproject/suppress.list
#suppress list will be stored in suppress.list with the bug hashes in the file
#the user checks in the suppress.list file with the source code
```
###5. Same as use case 2., but the developer wants to give extra Config to clang-tidy or clang-sa
```
CodeChecker check --saargs clangsa.config --tidyargs clang-tidy.config -b make
#clang-sa and clang-tidy parameters are stored in simple text files,
#is expected to be in the same format as clang-sa/tidy
#command line parameters and will be passed to every clang-sa/tidy calls
```
###6. Asking for command line help for the check subcommand (all other subcommands would be the same: server, checkers,cmd…)
```
CodeChecker check -h
```
###7. Run analysis on 2 versions of the project
Analyze a large project from a script/Jenkins job periodically. Developers view the results on a central web-server.
If a hit is false positive, developers can mark it and comment it on the web interface and the suppress hashes are stored in a text file that can be version controlled.
Different versions of the project can be compared for new/resolved/unresolved bugs. Differences between runs can be viewed in the web browser or from command line (and can be sent in email if needed).
Large projects should use postgresql for performance reasons.
```
CodeChecker check -n myproject_v1 –postgresql -b make
CodeChecker check -n myproject_v2 –postgresql -b make
#runs analysis, assumes that postgres is available on the default 5432 TCP port,
#codechecker db user exists and codechecker db can be created
#please note that this use case is also supported with SQLITE db
```
Start the webserver and view the diff between the two results in the web browser
```
CodeChecker server –postgresql
firefox localhost:8001
```
Start the webserver and view the diff between the two results in command line
```
CodeChecker cmd diff –b myproject_v1 –n myproject_v2 –p 8001 -new
#assumes that the server is started on port 8001
#then shows the new bugs in myproject_v2 compared to baseline myproject_v1
CodeChecker cmd diff –b myproject_v1 –n myproject_v2 –p 8001 -unresolved
#assumes that the server is started on port 8001
#then shows the unresolved bugs in myproject_v2 compared to baseline myproject_v1
CodeChecker cmd diff –b myproject_v1 –n myproject_v2 –p 8001 -resolved
#assumes that the server is started on port 8001
#then shows the resolved bugs in myproject_v2 compared to baseline myproject_v1
```

tools/codechecker/docs/user_guide.md

  • This file was added.
# CodeChecker Userguide
## CodeChecker usage
First of all, you have to setup the environment for CodeChecker.
Codechecker server uses SQLite database (by default) to store the results which is also packed into the package.
The next step is to start the CodeChecker main script.
The main script can be started with different options.
~~~~~~~~~~~~~~~~~~~~~
usage: CodeChecker.py [-h] {check,log,checkers,server,cmd,debug} ...
Run the codechecker script.
positional arguments:
{check,log,checkers,server,cmd,debug}
commands
check Run codechecker for a project.
log Build the project and only create a log file (no
checking).
checkers List available checkers.
server Start the codechecker database server.
cmd Command line client
debug Create debug logs for failed actions
optional arguments:
-h, --help show this help message and exit
~~~~~~~~~~~~~~~~~~~~~
## Default configuration:
Used ports:
* 5432 - PostgreSQL
* 8001 - CodeChecker result viewer
## 1. log mode:
Just build your project and create a log file but do not invoke the source code analysis.
~~~~~~~~~~~~~~~~~~~~~
$CodeChecker log --help
usage: CodeChecker.py log [-h] -o LOGFILE -b COMMAND
optional arguments:
-h, --help show this help message and exit
-o LOGFILE, --output LOGFILE
Path to the log file.
-b COMMAND, --build COMMAND
Build command.
~~~~~~~~~~~~~~~~~~~~~
You can change the compilers that should be logged.
Set CC_LOGGER_GCC_LIKE environment variable to a colon separated list.
For example (default):
~~~~~~~~~~~~~~~~~~~~~
export CC_LOGGER_GCC_LIKE="gcc:g++:clang"
~~~~~~~~~~~~~~~~~~~~~
Example:
~~~~~~~~~~~~~~~~~~~~~
CodeChecker log -o ../codechecker_myProject_build.log -b "make -j2"
~~~~~~~~~~~~~~~~~~~~~
Note:
In case you want to analyze your whole project, do not forget to clean your build tree before logging.
## 2. check mode:
### Basic Usage
Database and connections will be automatically configured.
The main script starts and setups everything what is required for analyzing a project (database server, tables ...).
~~~~~~~~~~~~~~~~~~~~~
CodeChecker check -w codechecker_workspace -n myTestProject -b "make"
~~~~~~~~~~~~~~~~~~~~~
Static analysis can be started also by using an already generated buildlog (see log mode).
If log is not available the analyzer will automatically create it.
An already created CMake json compilation database can be used as well.
~~~~~~~~~~~~~~~~~~~~~
CodeChecker check -w ~/codechecker_wp -n myProject -l ~/codechecker_wp/build_log.json
~~~~~~~~~~~~~~~~~~~~~
### Advanced Usage
~~~~~~~~~~~~~~~~~~~~~
usage: CodeChecker check [-h] [-w WORKSPACE] -n NAME (-b COMMAND | -l LOGFILE)
[-j JOBS] [-u SUPPRESS] [-c [DEPRECATED]]
[--update [DEPRECATED]] [--force] [-s SKIPFILE]
[-e ENABLE] [-d DISABLE] [--keep-tmp]
[--analyzers ANALYZERS [ANALYZERS ...]]
[--saargs CLANGSA_ARGS_CFG_FILE]
[--tidyargs TIDY_ARGS_CFG_FILE] [--sqlite]
[--dbport DBPORT] [--dbaddress DBADDRESS]
[--dbname DBNAME] [--dbusername DBUSERNAME]
optional arguments:
-h, --help show this help message and exit
-w WORKSPACE, --workspace WORKSPACE
Directory where the codechecker can store analysis
related data. (default: /home/<user_home>/.codechecker)
-n NAME, --name NAME Name of the analysis.
-b COMMAND, --build COMMAND
Build command which is used to build the project
-l LOGFILE, --log LOGFILE
Path to the log file which is created during the
build. If there is an already generated log file with
the compilation commands generated by 'CodeChecker
log' or 'cmake -DCMAKE_EXPORT_COMPILE_COMMANDS'
CodeChecker check can use it for the analisys in that
case running the original build will be left out from
the analysis process (no log is needed).
-j JOBS, --jobs JOBS Number of jobs. Start multiple processes for faster
analisys (default: 1)
-u SUPPRESS, --suppress SUPPRESS
Path to suppress file. Suppress file can be used to
suppress analysis results during the analisys. It is
based on the bug identifier generated by the compiler
which is experimental. Do not depend too much on this
file because identifier or file format can be changed.
For other in source suppress features see the user
guide.
-c [DEPRECATED], --clean [DEPRECATED]
DEPRECATED argument! (default: None)
--update [DEPRECATED]
DEPRECATED argument! (default: None)
--force Delete analisys results form the database if a run
with the given name already exists (default: False)
-s SKIPFILE, --skip SKIPFILE
Path to skip file.
-e ENABLE, --enable ENABLE
Enable checker.
-d DISABLE, --disable DISABLE
Disable checker.
--keep-tmp Keep temporary report files generated during the
analysis. (default: False)
--analyzers ANALYZERS [ANALYZERS ...]
Select which analyzer should be enabled. Currently
supported analyzers are: clangsa clang-tidy e.g. '--
analyzers clangsa clang-tidy' (default: ['clangsa',
'clang-tidy'])
--saargs CLANGSA_ARGS_CFG_FILE
File with arguments which will be forwarded directly
to the Clang static analyzer without modifiaction
--tidyargs TIDY_ARGS_CFG_FILE
File with arguments which will be forwarded directly
to the Clang tidy analyzer without modifiaction
--sqlite [DEPRECATED]
DEPRECATED argument! (default: None)
--postgresql Use PostgreSQL database. (default: False)
--dbport DBPORT Postgres server port. (default: 5432)
--dbaddress DBADDRESS
Postgres database server address (default: localhost)
--dbname DBNAME Name of the database. (default: codechecker)
--dbusername DBUSERNAME
Database user name. (default: codechecker)
~~~~~~~~~~~~~~~~~~~~~
CodeChecker is able to handle several analyzer tools. Currently CodeChecker
supports Clang Static Analyzer and Clang Tidy. ```CodeChecker checkers```
command lists all checkers from each analyzers. These can be switched on and off
by ```-e``` and ```-d``` flags. Furthermore ```--analyzers``` specifies which
analyzer tool should be used (both by default). The tools are completely
independent, so either can be omitted if not present as these are provided by
different binaries.
#### Forward compiler options
These options can modify the compilation actions logged by the build logger or
created by cmake (exporting compile commands). The extra compiler options can be
given in config files which are provided by the flags described below.
The config files can contain placeholders in `$(ENV_VAR)` format. If the
`ENV_VAR` environment variable is set then the placeholder is replaced to its
value. Otherwise an error message is logged saying that the variable is not set,
and in this case an empty string is inserted in the place of the placeholder.
##### Clang Static Analyzer
Use the ```--saargs``` argument to a file which contains compilation options.
````
CodeChecker check --saargs extra_compile_flags -n myProject -b "make -j4"
````
Where the extra_compile_flags file contains additional compilation options.
Config file example:
```
-I~/include/for/analysis -I$(MY_LIB)/include -DDEBUG
```
where `MY_LIB` is the path of a library code.
##### Clang-tidy
Use the ```--tidyargs``` argument to a file which contains compilation options.
````
CodeChecker check --tidyargs extra_tidy_compile_flags -n myProject -b "make -j4"
````
Where the extra_compile_flags file contains additional compilation flags.
Clang tidy requires a different format to add compilation options.
Compilation options can be added before ( ```-extra-arg-before=<string>``` ) and
after (```-extra-arg=<string>```) the original compilation options.
Config file example:
```
-extra-arg-before='-I~/include/for/analysis' -extra-arg-before='-I~/other/include/for/analysis/' -extra-arg-before='-I$(MY_LIB)/include' -extra-arg='-DDEBUG'
```
where `MY_LIB` is the path of a library code.
### Using SQLite for database:
CodeChecker can also use SQLite for storing the results. In this case the
SQLite database will be created in the workspace directory.
In order to use PostgreSQL instead of SQLite, use the ```--postgresql``` command
line argument for ```CodeChecker server``` and ```CodeChecker check```
commands. If ```--postgresql``` is not given then SQLite is used by default in
which case ```--dbport```, ```--dbaddress```, ```--dbname```, and
```--dbusername``` command line arguments are ignored.
#### Note:
Schema migration is not supported with SQLite. This means if you upgrade your
CodeChecker to a newer version, you might need to re-check your project.
### Suppression in the source:
Suppress comments can be used in the source to suppress specific or all checker results found in a source line.
Suppress comment should be above the line where the bug was found no empty lines are allowed between the line with the bug and the suppress comment.
Only comment lines staring with "//" are supported
Supported comment formats:
~~~~~~~~~~~~~~~~~~~~~
void test() {
int x;
// codechecker_suppress [deadcode.DeadStores] suppress deadcode
x = 1; // warn
}
~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~
void test() {
int x;
// codechecker_suppress [all] suppress all checker results
x = 1; // warn
}
~~~~~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~
void test() {
int x;
// codechecker_suppress [all] suppress all
// checker resuls
// with a long
// comment
x = 1; // warn
}
~~~~~~~~~~~~~~~~~~~~~
### Suppress file:
~~~~~~~~~~~~~~~~~~~~~
-u SUPPRESS
~~~~~~~~~~~~~~~~~~~~~
Suppress file can contain bug hashes and comments.
Suppressed bugs will not be showed in the viewer by default.
Usually a reason to suppress a bug is a false positive result (reporting a non-existent bug). Such false positives should be reported, so we can fix the checkers.
A comment can be added to suppressed reports that describes why that report is false positive. You should not edit suppress file by hand. The server should handle it.
The suppress file can be checked into the source code repository.
Bugs can be suppressed on the viewer even when suppress file was not set by command line arguments. This case the suppress will not be permanent. For this reason it is
advised to always provide (the same) suppress file for the checks.
### Skip file:
~~~~~~~~~~~~~~~~~~~~~
-s SKIPFILE, --skip SKIPFILE
~~~~~~~~~~~~~~~~~~~~~
With a skip file you can filter which files should or shouldn't be checked.
Each line in a skip file should start with a '-' or '+' character followed by a path glob pattern. A minus character means that if a checked file path - including the headers - matches with the pattern, the file will not be checked. The plus character means the opposite: if a file path matches with the pattern, it will be checked.
CodeChecker reads the file from top to bottom and stops at the first matching pattern.
For example:
~~~~~~~~~~~~~~~~~~~~~
-/skip/all/source/in/directory*
-/do/not/check/this.file
+/dir/check.this.file
-/dir/*
~~~~~~~~~~~~~~~~~~~~~
### Enable/Disable checkers
~~~~~~~~~~~~~~~~~~~~~
-e ENABLE, --enable ENABLE
-d DISABLE, --disable DISABLE
~~~~~~~~~~~~~~~~~~~~~
You can enable or disable checkers or checker groups. If you want to enable more checker groups use -e multiple times. To get the actual list of checkers run ```CodeChecer checkers``` command.
For example if you want to enable core and security checkers, but want to disable alpha checkers use
~~~~~~~~~~~~~~~~~~~~~
CodeChecker check -e core -e security -d alpha ...
~~~~~~~~~~~~~~~~~~~~~
### Multithreaded Checking
~~~~~~~~~~~~~~~~~~~~~
-j JOBS, --jobs JOBS Number of jobs.
~~~~~~~~~~~~~~~~~~~~~
CodeChecker will execute analysis on as many threads as specified after -j argument.
### Various deployment possibilities
The codechecker server can be started separately when desired.
In that case multiple clients can use the same database to store new results or view old ones.
#### Codechecker server and database on the same machine
Codechecker server and the database are running on the same machine but the database server is started manually.
In this case the database handler and the database can be started manually by running the server command.
The workspace needs to be provided for both the server and the check commands.
~~~~~~~~~~~~~~~~~~~~~
CodeChecker server -w ~/codechecker_wp --dbname myProjectdb --dbport 5432 --dbaddress localhost --view-port 8001
~~~~~~~~~~~~~~~~~~~~~
The checking process can be started separately on the same machine
~~~~~~~~~~~~~~~~~~~~~
CodeChecker check -w ~/codechecker_wp -n myProject -b "make -j 4" --dbname myProjectdb --dbaddress localhost --dbport 5432
~~~~~~~~~~~~~~~~~~~~~
or on a different machine
~~~~~~~~~~~~~~~~~~~~~
CodeChecker check -w ~/codechecker_wp -n myProject -b "make -j 4" --dbname myProjectdb --dbaddress 192.168.1.1 --dbport 5432
~~~~~~~~~~~~~~~~~~~~~
#### Codechecker server and database are on different machines
It is possible that the codechecker server and the PostgreSQL database that contains the analysis results are on different machines. To setup PostgreSQL see later section.
In this case the codechecker server can be started using the following command:
~~~~~~~~~~~~~~~~~~~~~
CodeChecker server --dbname myProjectdb --dbport 5432 --dbaddress 192.168.1.2 --view-port 8001
~~~~~~~~~~~~~~~~~~~~~
Start codechecker server locally which connects to a remote database (which is started separately). Workspace is not required in this case.
Start the checking as explained previously.
~~~~~~~~~~~~~~~~~~~~~
CodeChecker check -w ~/codechecker_wp -n myProject -b "make -j 4" --dbname myProjectdb --dbaddress 192.168.1.2 --dbport 5432
~~~~~~~~~~~~~~~~~~~~~
## 3. Quick check mode:
It's possible to quickly check a small project (set of files) for bugs without
storing the results into a database. In this case only the build command is
required and the defect list appears on the console. The defect list doesn't
shows the bug paths by default but you can turn it on using the --steps command
line parameter.
Basic usage:
~~~~~~~~~~~~~~~~~~~~~
CodeChecker quickcheck -b 'make'
~~~~~~~~~~~~~~~~~~~~~
Enabling bug path:
~~~~~~~~~~~~~~~~~~~~~
CodeChecker quickcheck -b 'make' --steps
~~~~~~~~~~~~~~~~~~~~~
Usage:
~~~~~~~~~~~~~~~~~~~~~
usage: CodeChecker.py quickcheck [-h] (-b COMMAND | -l LOGFILE) [-e ENABLE]
[-d DISABLE] [-s]
optional arguments:
-h, --help show this help message and exit
-b COMMAND, --build COMMAND
Build command.
-l LOGFILE, --log LOGFILE
Path to the log file which is created during the
build.
-e ENABLE, --enable ENABLE
Enable checker.
-d DISABLE, --disable DISABLE
Disable checker.
-s, --steps Print steps.
~~~~~~~~~~~~~~~~~~~~~
## 4. checkers mode:
List all available checkers.
The ```+``` (or ```-```) sign before a name of a checker shows whether the checker is enabled (or disabled) by default.
~~~~~~~~~~~~~~~~~~~~~
CodeChecker checkers
~~~~~~~~~~~~~~~~~~~~~
## 5. cmd mode:
A lightweigh command line interface to query the results of an analysis.
It is a suitable client to integrate with continous integration, schedule maintenance tasks and verifying correct analysis process.
The commands always need a viewer port of an already running CodeChecker server instance (which can be started using CodeChecker server command).
~~~~~~~~~~~~~~~~~~~~~
usage: CodeChecker.py cmd [-h] {runs,results,sum,del} ...
positional arguments:
{runs,results,diff,sum,del}
runs Get the run data.
results List results.
diff Diff two run.
sum Sum results.
del Remove run results.
optional arguments:
-h, --help show this help message and exit
~~~~~~~~~~~~~~~~~~~~~
## 6. debug mode:
In debug mode CodeChecker can generate logs for failed build actions. The logs can be helpful debugging the checkers.
## Example Usage
### Checking files
Checking with some extra checkers disabled and enabled
~~~~~~~~~~~~~~~~~~~~~
CodeChecker check -w ~/Develop/workspace -j 4 -b "cd ~/Libraries/myproject && make clean && make -j4" -s ~/Develop/skip.list -u ~/Develop/suppress.txt -e unix.Malloc -d core.uninitialized.Branch -n MyLittleProject -c --dbport 5432 --dbname cctestdb
~~~~~~~~~~~~~~~~~~~~~
### View results
To view the results CodeChecker sever needs to be started.
~~~~~~~~~~~~~~~~~~~~~
CodeChecker server -w ~/codes/package/checker_ws/ --dbport 5432 --dbaddress localhost
~~~~~~~~~~~~~~~~~~~~~
After the server has started open the outputed link to the browser (localhost:8001 in this example).
~~~~~~~~~~~~~~~~~~~~~
[11318] - WARNING! No suppress file was given, suppressed results will be only stored in the database.
[11318] - Checking for database
[11318] - Database is not running yet
[11318] - Starting database
[11318] - Waiting for client requests on [localhost:8001]
~~~~~~~~~~~~~~~~~~~~~
### Run CodeChecker distributed in a cluster
You may want to configure codechecker to do the analysis on separate machines in a distributed way.
Start the postgres database on a central machine (in this example it is called codechecker.central) on a remotely accessible address and port and then run
```CodeChecker check``` on multiple machines (called host1 and host2), specify the remote dbaddress and dbport and use the same run name.
Create and start an empty database to which the codechecker server can connect.
#### Setup PostgreSQL (one time only)
Before the first use, you have to setup PostgreSQL.
PostgreSQL stores its data files in a data directory, so before you start the PostgreSQL server you have to create and init this data directory.
I will call the data directory to pgsql_data.
Do the following steps:
~~~~~~~~~~~~~~~~~~~~~
# on machine codechecker.central
mkdir -p /path/to/pgsql_data
initdb -U codechecker -D /path/to/pgsql_data -E "SQL_ASCII"
# Start PostgreSQL server on port 5432
postgres -U codechecker -D /path/to/pgsql_data -p 5432 &>pgsql_log &
~~~~~~~~~~~~~~~~~~~~~
#### Run CodeChecker on multiple hosts
Then you can run codechecker on multiple hosts but using the same run name (in this example this is called "distributed_run".
postgres is listening on codechecker.central port 9999.
~~~~~~~~~~~~~~~~~~~~~
# On host1 we check module1
CodeChecker check -w /tmp/codechecker_ws -b "cd module_1;make" --dbport 5432 --dbaddress codechecker.central -n distributed_run
# On host2 we check module2
CodeChecker check -w /tmp/codechecker_ws -b "cd module_2;make" --dbport 5432 --dbaddress codechecker.central -n disributed_run
~~~~~~~~~~~~~~~~~~~~~
#### PostgreSQL authentication (optional)
If a CodeChecker is run with a user that needs database authentication, the
PGPASSFILE environment variable should be set to a pgpass file
For format and further information see PostgreSQL documentation:
http://www.postgresql.org/docs/current/static/libpq-pgpass.html
## Debugging CodeChecker
Environment variables can be used to turn on CodeChecker debug mode.
Turn on analysis related logging
~~~~~~~~~~~~~~~~~~~~~
export CODECHECKER_VERBOSE=debug_analyzer
~~~~~~~~~~~~~~~~~~~~~
Turn on CodeChecker debug level logging
~~~~~~~~~~~~~~~~~~~~~
export CODECHECKER_VERBOSE=debug
~~~~~~~~~~~~~~~~~~~~~
If debug logging is enabled and PostgreSQL database is used, PostgreSQL logs are written to postgresql.log in the workspace directory.
Turn on SQL_ALCHEMY debug level logging
~~~~~~~~~~~~~~~~~~~~~
export CODECHECKER_ALCHEMY_LOG=True
~~~~~~~~~~~~~~~~~~~~~

tools/codechecker/libcodechecker/__init__.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.

tools/codechecker/libcodechecker/analysis_manager.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
'''
import os
import sys
import signal
import multiprocessing
import ntpath
import traceback
import shutil
from collections import defaultdict
from libcodechecker import logger
from libcodechecker import analyzer_env
from libcodechecker.analyzers import analyzer_types
LOG = logger.get_new_logger('ANALISYS MANAGER')
def worker_result_handler(results):
"""
print the analisys summary
"""
successful_analysis = defaultdict(int)
failed_analisys = defaultdict(int)
skipped_num = 0
for res, skipped, analyzer_type in results:
if skipped:
skipped_num += 1
else:
if res == 0:
successful_analysis[analyzer_type] += 1
else:
failed_analisys[analyzer_type] += 1
LOG.info("----==== Summary ====----")
LOG.info('Total compilation commands: ' + str(len(results)))
if successful_analysis:
LOG.info('Successfully analyzed')
for analyzer_type, res in successful_analysis.iteritems():
LOG.info(' ' + analyzer_type + ': ' + str(res))
if failed_analisys:
LOG.info("Failed to analyze")
for analyzer_type, res in failed_analisys.iteritems():
LOG.info(' ' + analyzer_type + ': ' + str(res))
if skipped_num:
LOG.info('Skipped compilation commands: ' + str(skipped_num))
LOG.info("----=================----")
def check(check_data):
"""
Invoke clang with an action which called by processes.
Different analyzer object belongs to for each build action
skiplist handler is None if no skip file was configured
"""
args, action, context, analyzer_config_map, skp_handler, \
report_output_dir, use_db = check_data
skipped = False
try:
# if one analysis fails the check fails
return_codes = 0
skipped = False
for source in action.sources:
# if there is no skiplist handler there was no skip list file
# in the command line
# cpp file skipping is handled here
_, source_file_name = ntpath.split(source)
if skp_handler and skp_handler.should_skip(source):
LOG.debug_analyzer(source_file_name + ' is skipped')
skipped = True
continue
# construct analyzer env
analyzer_environment = analyzer_env.get_check_env(context.path_env_extra,
context.ld_lib_path_extra)
run_id = context.run_id
rh = analyzer_types.construct_result_handler(args,
action,
run_id,
report_output_dir,
context.severity_map,
skp_handler,
use_db)
#LOG.info('Analysing ' + source_file_name)
# create a source analyzer
source_analyzer = analyzer_types.construct_analyzer(action,
analyzer_config_map)
# source is the currently analyzed source file
# there can be more in one buildaction
source_analyzer.source_file = source
# fills up the result handler with the analyzer information
source_analyzer.analyze(rh, analyzer_environment)
if rh.analyzer_returncode == 0:
# analysis was successful
# processing results
if rh.analyzer_stdout != '':
LOG.debug_analyzer('\n' + rh.analyzer_stdout)
if rh.analyzer_stderr != '':
LOG.debug_analyzer('\n' + rh.analyzer_stderr)
rh.postprocess_result()
rh.handle_results()
else:
# analisys failed
LOG.error('Analyzing ' + source_file_name + ' failed.')
if rh.analyzer_stdout != '':
LOG.error(rh.analyzer_stdout)
if rh.analyzer_stderr != '':
LOG.error(rh.analyzer_stderr)
return_codes = rh.analyzer_returncode
if not args.keep_tmp:
rh.clean_results()
return (return_codes, skipped, action.analyzer_type)
except Exception as e:
LOG.debug_analyzer(str(e))
traceback.print_exc(file=sys.stdout)
return (1, skipped, action.analyzer_type)
def start_workers(args, actions, context, analyzer_config_map, skp_handler):
"""
start the workers in the process pool
for every buildaction there is worker which makes the analysis
"""
# Handle SIGINT to stop this script running
def signal_handler(*arg, **kwarg):
try:
pool.terminate()
finally:
sys.exit(1)
signal.signal(signal.SIGINT, signal_handler)
# create report output dir this will be used by the result handlers for each
# analyzer to store analyzer results or temporary files
# each analyzer instance does its own cleanup
report_output = os.path.join(context.codechecker_workspace,
args.name + '_reports')
if not os.path.exists(report_output):
os.mkdir(report_output)
# Start checking parallel
pool = multiprocessing.Pool(args.jobs)
# pool.map(check, actions, 1)
try:
# Workaround, equialent of map
# The main script does not get signal
# while map or map_async function is running
# It is a python bug, this does not happen if a timeout is specified;
# then receive the interrupt immediately
analyzed_actions = [(args,
build_action,
context,
analyzer_config_map,
skp_handler,
report_output,
True ) for build_action in actions]
pool.map_async(check,
analyzed_actions,
1,
callback=worker_result_handler).get(float('inf'))
pool.close()
except Exception:
pool.terminate()
raise
finally:
pool.join()
if not args.keep_tmp:
LOG.debug('Removing temporary directory: ' + report_output)
shutil.rmtree(report_output)

tools/codechecker/libcodechecker/analyzer.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
"""
Prepare and start different analisys types
"""
import os
import sys
import json
import time
import copy
from libcodechecker import client
from libcodechecker import logger
from libcodechecker import analysis_manager
from libcodechecker import skiplist_handler
from libcodechecker.analyzers import analyzer_types
LOG = logger.get_new_logger('ANALYZER')
def prepare_actions(actions, enabled_analyzers):
"""
set the analyzer type for each buildaction
muliply actions if multiple source analyzers are set
"""
res = []
for ea in enabled_analyzers:
for action in actions:
new_action = copy.deepcopy(action)
new_action.analyzer_type = ea
res.append(new_action)
return res
def run_check(args, actions, context):
"""
prepare:
- analyzer config handlers
- skiplist handling
- analyzer severity levels
stores analysis related data to the database and starts the analysis
"""
if args.jobs <= 0:
args.jobs = 1
LOG.debug_analyzer("Checking supported analyzers.")
enabled_analyzers = analyzer_types.check_supported_analyzers(
args.analyzers,
context)
# load severity map from config file
LOG.debug_analyzer("Loading checker severity map.")
if os.path.exists(context.checkers_severity_map_file):
with open(context.checkers_severity_map_file, 'r') as sev_conf_file:
severity_config = sev_conf_file.read()
context.severity_map = json.loads(severity_config)
actions = prepare_actions(actions, enabled_analyzers)
analyzer_config_map = {}
package_version = context.version['major'] + '.' + context.version['minor']
suppress_file = ''
try:
suppress_file = os.path.realpath(args.suppress)
except AttributeError:
LOG.debug_analyzer('Suppress file was not set in the command line')
# Create one skip list handler shared between the analysis manager workers
skip_handler = None
try:
if args.skipfile:
LOG.debug_analyzer("Creating skiplist handler.")
skip_handler = skiplist_handler.SkipListHandler(args.skipfile)
except AttributeError:
LOG.debug_analyzer('Skip file was not set in the command line')
with client.get_connection() as connection:
context.run_id = connection.add_checker_run(' '.join(sys.argv),
args.name,
package_version,
args.force)
# clean previous suppress information
client.clean_suppress(connection, context.run_id)
if os.path.exists(suppress_file):
client.send_suppress(context.run_id, connection, suppress_file)
analyzer_config_map = analyzer_types.build_config_handlers(args,
context,
enabled_analyzers,
connection)
if skip_handler:
connection.add_skip_paths(context.run_id,
skip_handler.get_skiplist())
LOG.info("Static analysis is starting ...")
start_time = time.time()
analysis_manager.start_workers(args,
actions,
context,
analyzer_config_map,
skip_handler)
end_time = time.time()
with client.get_connection() as connection:
connection.finish_checker_run(context.run_id)
LOG.info("Analysis length: " + str(end_time - start_time) + " sec.")
def run_quick_check(args,
context,
actions):
'''
This function implements the "quickcheck" feature.
No result is stored to a database
'''
enabled_analyzers = set()
enabled_analyzers = analyzer_types.check_supported_analyzers(args.analyzers,
context)
actions = prepare_actions(actions, enabled_analyzers)
analyzer_config_map = {}
analyzer_config_map = analyzer_types.build_config_handlers(args,
context,
enabled_analyzers)
for action in actions:
check_data = (args,
action,
context,
analyzer_config_map,
None,
args.workspace,
False)
analysis_manager.check(check_data)

tools/codechecker/libcodechecker/analyzer_crash_handler.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''Module to handle analyzer crash.'''
import subprocess
import shlex
import tempfile
import signal
from libcodechecker import logger
class AnalyzerCrashHandler(object):
def __init__(self, context, analyzer_env):
self._context = context
self._analyzer_env = analyzer_env
self._logger = logger.get_new_logger('ANALYZER_CRASH_HANDLER')
# --------------------------------------------------------------------------
def get_crash_info(self, build_cmd):
''' Get the crash info by running the build command
with gdb again if there was some crash.
'''
def signal_handler(*args, **kwargs):
try:
result.terminate()
finally:
raise KeyboardInterrupt('CTRL+C')
signal.signal(signal.SIGINT, signal_handler)
gdb_cmd = []
gdb_cmd.append('gdb')
gdb_cmd.append('--batch')
# add gdb script path is in the DT unter the config folder
gdb_cmd.append('--command=' + self._context.gdb_config_file)
gdb_cmd.append('--args')
gdb_cmd.extend(build_cmd)
tmp_stdout = tempfile.TemporaryFile()
tmp_stderr = tempfile.TemporaryFile()
result = ""
# gdb uses python3, so it is crashed when any python2 module in PYTHONPATH
# bug: https://bugs.launchpad.net/ubuntu/+source/apport/+bug/1398033
self._analyzer_env['PYTHONPATH'] = ''
try:
result = subprocess.Popen(shlex.split(' '.join(gdb_cmd)),
env=self._analyzer_env,
stdout=tmp_stdout, stderr=tmp_stderr)
result.wait()
tmp_stdout.seek(0)
tmp_stderr.seek(0)
output_stdout = tmp_stdout.read()
output_stderr = tmp_stderr.read()
result = output_stdout + '\n' + output_stderr
except KeyboardInterrupt:
raise
except:
result = 'Failed to get extra debug information using gdb:\n' + \
' '.join(gdb_cmd)
finally:
tmp_stdout.close()
tmp_stderr.close()
return result
# --------------------------------------------------------------------------

tools/codechecker/libcodechecker/analyzer_env.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
''''''
import os
from libcodechecker import logger
LOG = logger.get_new_logger('ENV')
# ------------------------------------------------------------------------------
def get_log_env(logfile, context, original_env):
'''
Environment for logging. With the ld logger.
Keep the original environment unmodified as possible
Only environment variables required for logging are changed
'''
new_env = original_env
new_env[context.env_var_cc_logger_bin] = context.path_logger_bin
new_env['LD_PRELOAD'] = context.logger_lib_name
try:
original_ld_library_path = new_env['LD_LIBRARY_PATH']
new_env['LD_LIBRARY_PATH'] = context.path_logger_lib + \
':' + original_ld_library_path
except:
new_env['LD_LIBRARY_PATH'] = context.path_logger_lib
# set ld logger logfile
new_env[context.env_var_cc_logger_file] = logfile
return new_env
# -----------------------------------------------------------------------------
def get_check_env(path_env_extra, ld_lib_path_extra):
'''
Extending the checker environment.
Check environment is extended to find tools if they ar not on
the default places
'''
new_env = os.environ.copy()
if len(path_env_extra) > 0:
extra_path = ':'.join(path_env_extra)
LOG.debug_analyzer('Extending PATH environment variable with: ' + extra_path)
try:
new_env['PATH'] = extra_path + ':' + new_env['PATH']
except:
new_env['PATH'] = extra_path
if len(ld_lib_path_extra) > 0:
extra_lib = ':'.join(ld_lib_path_extra)
LOG.debug_analyzer('Extending LD_LIBRARY_PATH environment variable with: ' + extra_lib)
try:
original_ld_library_path = new_env['LD_LIBRARY_PATH']
new_env['LD_LIBRARY_PATH'] = extra_lib + ':' + original_ld_library_path
except:
new_env['LD_LIBRARY_PATH'] = extra_lib
return new_env

tools/codechecker/libcodechecker/analyzers/__init__.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.

tools/codechecker/libcodechecker/analyzers/analyzer_base.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import os
import sys
import shlex
import signal
import subprocess
from abc import ABCMeta, abstractmethod
from libcodechecker import logger
LOG = logger.get_new_logger('ANALYZER_BASE')
class SourceAnalyzer(object):
"""
base class for different source analyzers
"""
__metaclass__ = ABCMeta
def __init__(self, config_handler, buildaction):
self.__config_handler = config_handler
self.__buildaction = buildaction
self.__source_file = ''
self.__checkers = []
@property
def checkers(self):
return self.__checkers
@property
def buildaction(self):
return self.__buildaction
@property
def config_handler(self):
return self.__config_handler
@property
def source_file(self):
"""
the currently analyzed source file
"""
return self.__source_file
@source_file.setter
def source_file(self, file_path):
"""
the currently analyzed source file
"""
self.__source_file = file_path
@abstractmethod
def construct_analyzer_cmd(self, result_handler):
"""
construct the analyzer command
"""
pass
def analyze(self, res_handler, env=None):
"""
run the analyzer
"""
LOG.debug('Running analyzer ...')
def signal_handler(*args, **kwargs):
# Clang does not kill its child processes, so I have to
try:
g_pid = proc.pid
os.killpg(g_pid, signal.SIGTERM)
finally:
sys.exit(os.EX_OK)
signal.signal(signal.SIGINT, signal_handler)
# NOTICE!
# the currently analyzed source file needs to be set beforer the
# analyzer command is constructed
# the analyzer output file is based on the currently analyzed source
res_handler.analyzed_source_file = self.source_file
# construct the analyzer cmd
analyzer_cmd = self.construct_analyzer_cmd(res_handler)
LOG.debug('\n' + ' '.join(analyzer_cmd))
res_handler.analyzer_cmd = analyzer_cmd
analyzer_cmd = ' '.join(analyzer_cmd)
try:
ret_code, stdout, stderr = SourceAnalyzer.run_proc(analyzer_cmd,
env)
res_handler.analyzer_returncode = ret_code
res_handler.analyzer_stdout = stdout
res_handler.analyzer_stderr = stderr
return res_handler
except Exception as ex:
LOG.error(ex)
res_handler.analyzer_returncode = 1
return res_handler
@abstractmethod
def get_analyzer_checkers(self, config_handler, env):
"""
return the checkers available in the analyzer
"""
pass
@staticmethod
def run_proc(command, env=None, cwd=None):
"""
Just run the given command and return the returncode
and the stdout and stderr outputs of the process.
"""
cmd = shlex.split(command, posix=False)
proc = subprocess.Popen(cmd,
bufsize=-1,
env=env,
preexec_fn=os.setsid,
cwd=cwd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
(stdout, stderr) = proc.communicate()
return proc.returncode, stdout, stderr

tools/codechecker/libcodechecker/analyzers/analyzer_clang_tidy.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
'''
import re
import subprocess
import shlex
from libcodechecker import logger
from libcodechecker.analyzers import analyzer_base
LOG = logger.get_new_logger('CLANG TIDY')
class ClangTidy(analyzer_base.SourceAnalyzer):
"""
constructs the clang tidy analyzer commands
"""
def __parse_checkers(self, tidy_output):
"""
parse clang tidy checkers list
skip clang static analyzer checkers
store them to checkers
"""
for line in tidy_output.splitlines():
line = line.strip()
if re.match(r'^Enabled checks:', line) or line == '':
continue
elif line.startswith('clang-analyzer-'):
continue
else:
match = re.match(r'^\S+$', line)
if match:
self.checkers.append((match.group(0), ''))
def get_analyzer_checkers(self, config_handler, env):
"""
return the list of the supported checkers
"""
if not self.checkers:
analyzer_binary = config_handler.analyzer_binary
command = [analyzer_binary]
command.append("-list-checks")
command.append("-checks='*'")
command.append("-")
command.append("--")
try:
command = shlex.split(' '.join(command))
result = subprocess.check_output(command,
env=env)
except subprocess.CalledProcessError as cperr:
LOG.error(cperr)
return {}
self.__parse_checkers(result)
return self.checkers
def construct_analyzer_cmd(self, res_handler):
"""
"""
try:
config = self.config_handler
analyzer_bin = config.analyzer_binary
analyzer_cmd = []
analyzer_cmd.append(analyzer_bin)
# disable all checkers by default
checkers_cmdline = '-*'
# config handler stores which checkers are enabled or disabled
for checker_name, value in config.checks().iteritems():
enabled, _ = value
if enabled:
checkers_cmdline += ',' + checker_name
else:
checkers_cmdline += ',-' + checker_name
analyzer_cmd.append("-checks='" + checkers_cmdline.lstrip(',') + "'")
LOG.debug(config.analyzer_extra_arguments)
analyzer_cmd.append(config.analyzer_extra_arguments)
analyzer_cmd.append(self.source_file)
analyzer_cmd.append("--")
extra_arguments_before = []
if len(config.compiler_resource_dirs) > 0:
for inc_dir in config.compiler_resource_dirs:
extra_arguments_before.append('-resource-dir')
extra_arguments_before.append(inc_dir)
extra_arguments_before.append('-isystem')
extra_arguments_before.append(inc_dir)
if config.compiler_sysroot:
extra_arguments_before.append('--sysroot')
extra_arguments_before.append(config.compiler_sysroot)
for path in config.system_includes:
extra_arguments_before.append('-isystem')
extra_arguments_before.append(path)
for path in config.includes:
extra_arguments_before.append('-I')
extra_arguments_before.append(path)
# Set lang
extra_arguments_before.append('-x')
extra_arguments_before.append(self.buildaction.lang)
analyzer_cmd.append(' '.join(extra_arguments_before))
analyzer_cmd.extend(self.buildaction.analyzer_options)
return analyzer_cmd
except Exception as ex:
LOG.error(ex)
return []

tools/codechecker/libcodechecker/analyzers/analyzer_clangsa.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import re
import shlex
import subprocess
from libcodechecker import logger
from libcodechecker.analyzers import analyzer_base
LOG = logger.get_new_logger('CLANGSA')
class ClangSA(analyzer_base.SourceAnalyzer):
"""
constructs clang static analyzer commmands
"""
def __parse_checkers(self, clangsa_output):
"""
parse clang static analyzer checkers
store them to checkers
"""
# checker name and description in one line
pattern = re.compile(r'^\s\s(?P<checker_name>\S*)\s*(?P<description>.*)')
checker_name = None
for line in clangsa_output.splitlines():
if re.match(r'^CHECKERS:', line) or line == '':
continue
elif checker_name and not re.match(r'^\s\s\S', line):
# collect description for the checker name
self.checkers.append((checker_name, line.strip()))
checker_name = None
elif re.match(r'^\s\s\S+$', line.rstrip()):
# only checker name is in the line
checker_name = line.strip()
else:
# checker name and description is in one line
match = pattern.match(line.rstrip())
if match:
current = match.groupdict()
self.checkers.append((current['checker_name'],
current['description']))
def get_analyzer_checkers(self, config_handler, env):
"""
return the list of the supported checkers
"""
if not self.checkers:
analyzer_binary = config_handler.analyzer_binary
command = [analyzer_binary, "-cc1"]
for plugin in config_handler.analyzer_plugins:
command.append("-load")
command.append(plugin)
command.append("-analyzer-checker-help")
try:
command = shlex.split(' '.join(command))
result = subprocess.check_output(command,
env=env)
except subprocess.CalledProcessError as cperr:
LOG.error(cperr)
return {}
self.__parse_checkers(result)
return self.checkers
def construct_analyzer_cmd(self, res_handler):
"""
called by the analyzer method
construct the analyzer command
"""
try:
# get an putput file from the result handler
analyzer_output_file = res_handler.get_analyzer_result_file()
analyzer_mode = 'plist-multi-file'
# get the checkers list from the config_handler
# checker order matters
config = self.config_handler
analyzer_bin = config.analyzer_binary
analyzer_cmd = []
analyzer_cmd.append(analyzer_bin)
if len(config.compiler_resource_dirs) > 0:
for inc_dir in config.compiler_resource_dirs:
analyzer_cmd.append('-resource-dir')
analyzer_cmd.append(inc_dir)
analyzer_cmd.append('-isystem')
analyzer_cmd.append(inc_dir)
# compiling is enough
analyzer_cmd.append('-c')
analyzer_cmd.append('--analyze')
# turn off clang hardcoded checkers list
analyzer_cmd.append('--analyzer-no-default-checks')
for plugin in config.analyzer_plugins:
analyzer_cmd.append("-Xclang")
analyzer_cmd.append("-plugin")
analyzer_cmd.append("-Xclang")
analyzer_cmd.append("checkercfg")
analyzer_cmd.append("-Xclang")
analyzer_cmd.append("-load")
analyzer_cmd.append("-Xclang")
analyzer_cmd.append(plugin)
analyzer_cmd.append('-Xclang')
analyzer_cmd.append('-analyzer-opt-analyze-headers')
analyzer_cmd.append('-Xclang')
analyzer_cmd.append('-analyzer-output=' + analyzer_mode)
if config.compiler_sysroot:
analyzer_cmd.append('--sysroot')
analyzer_cmd.append(config.compiler_sysroot)
for path in config.system_includes:
analyzer_cmd.append('-isystem')
analyzer_cmd.append(path)
for path in config.includes:
analyzer_cmd.append('-I')
analyzer_cmd.append(path)
analyzer_cmd.append('-o')
analyzer_cmd.append(analyzer_output_file)
# config handler stores which checkers are enabled or disabled
for checker_name, value in config.checks().iteritems():
enabled, description = value
if enabled:
analyzer_cmd.append('-Xclang')
analyzer_cmd.append('-analyzer-checker=' + checker_name)
else:
analyzer_cmd.append('-Xclang')
analyzer_cmd.append('-analyzer-disable-checker')
analyzer_cmd.append('-Xclang')
analyzer_cmd.append(checker_name)
# Set lang
analyzer_cmd.append('-x')
analyzer_cmd.append(self.buildaction.lang)
analyzer_cmd.append(config.analyzer_extra_arguments)
analyzer_cmd.extend(self.buildaction.analyzer_options)
analyzer_cmd.append(self.source_file)
return analyzer_cmd
except Exception as ex:
LOG.error(ex)
return []

tools/codechecker/libcodechecker/analyzers/analyzer_types.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
"""
supported analyzer types
"""
import os
import re
import sys
from libcodechecker import logger
from libcodechecker import analyzer_env
from libcodechecker import host_check
from libcodechecker import client
from libcodechecker.analyzers import analyzer_clangsa
from libcodechecker.analyzers import config_handler_clangsa
from libcodechecker.analyzers import result_handler_clangsa
from libcodechecker.analyzers import analyzer_clang_tidy
from libcodechecker.analyzers import result_handler_clang_tidy
from libcodechecker.analyzers import config_handler_clang_tidy
LOG = logger.get_new_logger('ANALYZER TYPES')
CLANG_SA = 'clangsa'
CLANG_TIDY = 'clang-tidy'
supported_analyzers = {CLANG_SA, CLANG_TIDY}
def is_sa_checker_name(checker_name):
"""
match for Clang Static analyzer names like
unix
unix.Malloc
security.insecureAPI
security.insecureAPI.gets
"""
# no '-' is alowed in the checker name
sa_checker_name = r'^[^-]+$'
ptn = re.compile(sa_checker_name)
if ptn.match(checker_name):
return True
return False
def is_tidy_checker_name(checker_name):
"""
match for Clang Tidy analyzer names like
-*
modernize-*
clang-diagnostic-*
cert-fio38-c
google-global-names-in-headers
"""
# must contain at least one '-'
tidy_checker_name = r'^(?=.*[\-]).+$'
ptn = re.compile(tidy_checker_name)
if ptn.match(checker_name):
return True
return False
def check_supported_analyzers(analyzers, context):
"""
check if the selected analyzers are supported
"""
check_env = analyzer_env.get_check_env(context.path_env_extra,
context.ld_lib_path_extra)
analyzer_binaries = context.analyzer_binaries
enabled_analyzers = set()
for analyzer_name in analyzers:
if analyzer_name not in supported_analyzers:
LOG.error('Unsupported analyzer ' + analyzer_name + ' !')
sys.exit(1)
else:
# get the compiler binary to check if it can run
available_analyzer = True
analyzer_bin = analyzer_binaries.get(analyzer_name)
if not analyzer_bin:
LOG.debug_analyzer('Failed to detect analyzer binary ' + analyzer_name)
available_analyzer = False
if not host_check.check_clang(analyzer_bin, check_env):
LOG.warning('Failed to run analyzer '
+ analyzer_name + ' !')
available_analyzer = False
if available_analyzer:
enabled_analyzers.add(analyzer_name)
return enabled_analyzers
def construct_analyzer_type(analyzer_type, config_handler, buildaction):
"""
construct a specific analyzer based on the type
"""
if analyzer_type == CLANG_SA:
LOG.debug_analyzer('Constructing clangSA analyzer')
analyzer = analyzer_clangsa.ClangSA(config_handler,
buildaction)
return analyzer
elif analyzer_type == CLANG_TIDY:
LOG.debug_analyzer("Constructing clang-tidy analyzer")
analyzer = analyzer_clang_tidy.ClangTidy(config_handler,
buildaction)
return analyzer
else:
LOG.error('Not supported analyzer type')
return None
def construct_analyzer(buildaction,
analyzer_config_map):
"""
construct an analyzer
"""
try:
LOG.debug_analyzer('Constructing analyzer')
analyzer_type = buildaction.analyzer_type
# get the proper config handler for this analyzer type
config_handler = analyzer_config_map.get(analyzer_type)
analyzer = construct_analyzer_type(analyzer_type,
config_handler,
buildaction)
return analyzer
except Exception as ex:
LOG.debug_analyzer(ex)
return None
def initialize_checkers(config_handler,
checkers,
default_checkers=None,
cmdline_checkers=None):
# by default disable all checkers
for checker_name, description in checkers:
config_handler.add_checker(checker_name, False, description)
# set default enabled or disabled checkers
if default_checkers:
for checker in default_checkers:
for checker_name, enabled in checker.items():
if enabled:
config_handler.enable_checker(checker_name)
else:
config_handler.disable_checker(checker_name)
# set user defined enabled or disabled checkers from the
# command line
if cmdline_checkers:
for checker_name, enabled in cmdline_checkers:
if enabled:
config_handler.enable_checker(checker_name)
else:
config_handler.disable_checker(checker_name)
def __replace_env_var(cfg_file):
def replacer(matchobj):
env_var = matchobj.group(1)
if matchobj.group(1) not in os.environ:
LOG.error(env_var + ' environment variable not set in ' + cfg_file)
return ''
return os.environ[env_var]
return replacer
def __build_clangsa_config_handler(args, context):
"""
Build the config handler for clang static analyzer
Handle config options from the command line and config files
"""
config_handler = config_handler_clangsa.ClangSAConfigHandler()
config_handler.analyzer_plugins_dir = context.checker_plugin
config_handler.analyzer_binary = context.analyzer_binaries.get(CLANG_SA)
config_handler.compiler_resource_dirs = context.compiler_resource_dirs
config_handler.compiler_sysroot = context.compiler_sysroot
config_handler.system_includes = context.extra_system_includes
config_handler.includes = context.extra_includes
try:
with open(args.clangsa_args_cfg_file, 'rb') as sa_cfg:
config_handler.analyzer_extra_arguments = \
re.sub('\$\((.*?)\)',
__replace_env_var(args.clangsa_args_cfg_file),
sa_cfg.read().strip())
except IOError as ioerr:
LOG.debug_analyzer(ioerr)
except AttributeError as aerr:
# no clangsa arguments file was given in the command line
LOG.debug_analyzer(aerr)
analyzer = construct_analyzer_type(CLANG_SA, config_handler, None)
check_env = analyzer_env.get_check_env(context.path_env_extra,
context.ld_lib_path_extra)
checkers = analyzer.get_analyzer_checkers(config_handler, check_env)
# read clang-tidy checkers from the config file
clang_sa_checkers = context.default_checkers_config.get(CLANG_SA +
'_checkers')
try:
cmdline_checkers = args.ordered_checkers
except AttributeError:
LOG.debug_analyzer('No checkers were defined in the command line for' +
CLANG_SA)
cmdline_checkers = None
initialize_checkers(config_handler,
checkers,
clang_sa_checkers,
cmdline_checkers)
return config_handler
def __build_clang_tidy_config_handler(args, context):
"""
Build the config handler for clang tidy analyzer
Handle config options from the command line and config files
"""
config_handler = config_handler_clang_tidy.ClangTidyConfigHandler()
config_handler.analyzer_binary = context.analyzer_binaries.get(CLANG_TIDY)
config_handler.compiler_resource_dirs = context.compiler_resource_dirs
config_handler.compiler_sysroot = context.compiler_sysroot
config_handler.system_includes = context.extra_system_includes
config_handler.includes = context.extra_includes
try:
with open(args.tidy_args_cfg_file, 'rb') as tidy_cfg:
config_handler.analyzer_extra_arguments = \
re.sub('\$\((.*?)\)', __replace_env_var, tidy_cfg.read().strip())
except IOError as ioerr:
LOG.debug_analyzer(ioerr)
except AttributeError as aerr:
# no clang tidy arguments file was given in the command line
LOG.debug_analyzer(aerr)
analyzer = construct_analyzer_type(CLANG_TIDY, config_handler, None)
check_env = analyzer_env.get_check_env(context.path_env_extra,
context.ld_lib_path_extra)
checkers = analyzer.get_analyzer_checkers(config_handler, check_env)
# read clang-tidy checkers from the config file
clang_tidy_checkers = context.default_checkers_config.get(CLANG_TIDY +
'_checkers')
try:
cmdline_checkers = args.ordered_checkers
except AttributeError:
LOG.debug_analyzer('No checkers were defined in the command line for ' +
CLANG_TIDY)
cmdline_checkers = None
initialize_checkers(config_handler,
checkers,
clang_tidy_checkers,
cmdline_checkers)
return config_handler
def build_config_handlers(args, context, enabled_analyzers, connection=None):
"""
construct multiple config handlers and if there is a connection
store configs into the database
handle config from command line or from config file if no command line
config is given
supported command line config format is in JSON tidy supports YAML also but
no standard lib for yaml parsing is available in python
"""
run_id = context.run_id
analyzer_config_map = {}
for ea in enabled_analyzers:
if ea == CLANG_SA:
config_handler = __build_clangsa_config_handler(args, context)
analyzer_config_map[ea] = config_handler
elif ea == CLANG_TIDY:
config_handler = __build_clang_tidy_config_handler(args, context)
analyzer_config_map[ea] = config_handler
else:
LOG.debug_analyzer('Not supported analyzer type. No configuration handler will be created')
if connection:
# collect all configuration options and store them together
configs = []
for _, config_handler in analyzer_config_map.iteritems():
configs.extend(config_handler.get_checker_configs())
client.replace_config_in_db(run_id, connection, configs)
return analyzer_config_map
def construct_result_handler(args,
buildaction,
run_id,
report_output,
severity_map,
skiplist_handler,
store_to_db=False):
"""
construct a result handler
"""
if store_to_db:
# create a result handler which stores the results into a database
if buildaction.analyzer_type == CLANG_SA:
csa_res_handler = result_handler_clangsa.ClangSAPlistToDB(
buildaction,
report_output,
run_id)
csa_res_handler.severity_map = severity_map
csa_res_handler.skiplist_handler = skiplist_handler
return csa_res_handler
elif buildaction.analyzer_type == CLANG_TIDY:
ct_res_handler = result_handler_clang_tidy.ClangTidyPlistToDB(
buildaction,
report_output,
run_id)
ct_res_handler.severity_map = severity_map
ct_res_handler.skiplist_handler = skiplist_handler
return ct_res_handler
else:
LOG.error('Not supported analyzer type')
return None
else:
if buildaction.analyzer_type == CLANG_SA:
csa_res_handler = result_handler_clangsa.ClangSAPlistToStdout(
buildaction,
report_output)
csa_res_handler.print_steps = args.print_steps
return csa_res_handler
elif buildaction.analyzer_type == CLANG_TIDY:
ct_res_handler = result_handler_clang_tidy.ClangTidyPlistToStdout(
buildaction,
report_output)
ct_res_handler.severity_map = severity_map
ct_res_handler.skiplist_handler = skiplist_handler
return ct_res_handler
else:
LOG.error('Not supported analyzer type')
return None

tools/codechecker/libcodechecker/analyzers/config_handler.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import os
import collections
from abc import ABCMeta, abstractmethod
from libcodechecker import logger
LOG = logger.get_new_logger('CONFIG HANDLER')
class AnalyzerConfigHandler(object):
"""
handle the checker configurations
and enabled disabled checkers lists
"""
__metaclass__ = ABCMeta
def __init__(self):
self.__analyzer_binary = None
self.__analyzer_plugins_dir = None
self.__compiler_sysroot = None
self.__compiler_resource_dirs = []
self.__sys_inc = []
self.__includes = []
self.__analyzer_extra_arguments = ''
# the key is the checker name, the value is a tuple
# False if disabled (should be by default)
# True if checker is enabled
# (False/True, 'checker_description')
self.__available_checkers = collections.OrderedDict()
@property
def analyzer_plugins_dir(self):
"""
get directory from where shared objects with checkers should be loaded
"""
return self.__analyzer_plugins_dir
@analyzer_plugins_dir.setter
def analyzer_plugins_dir(self, value):
"""
set the directory where shared objects with checkers can be found
"""
self.__analyzer_plugins_dir = value
@property
def analyzer_plugins(self):
"""
full path of the analyzer plugins
"""
plugin_dir = self.__analyzer_plugins_dir
analyzer_plugins = [os.path.join(plugin_dir, f)
for f in os.listdir(plugin_dir)
if os.path.isfile(os.path.join(plugin_dir, f))]
return analyzer_plugins
@property
def analyzer_binary(self):
return self.__analyzer_binary
@analyzer_binary.setter
def analyzer_binary(self, value):
self.__analyzer_binary = value
@abstractmethod
def get_checker_configs(self):
"""
return a lis of (checker_name, key, key_valye) tuples
"""
pass
def add_checker(self, checker_name, enabled, description):
"""
add additional checker
tuple of (checker_name, True\False)
"""
self.__available_checkers[checker_name] = (enabled, description)
def enable_checker(self, checker_name, description=None):
"""
enable checker, keep description if already set
"""
for ch_name, values in self.__available_checkers.iteritems():
if ch_name.startswith(checker_name):
_, description = values
self.__available_checkers[ch_name] = (True, description)
def disable_checker(self, checker_name, description=None):
"""
disable checker, keep description if already set
"""
for ch_name, values in self.__available_checkers.iteritems():
if ch_name.startswith(checker_name):
_, description = values
self.__available_checkers[ch_name] = (False, description)
def checks(self):
"""
return the checkers
"""
return self.__available_checkers
@property
def compiler_sysroot(self):
"""
get compiler sysroot
"""
return self.__compiler_sysroot
@compiler_sysroot.setter
def compiler_sysroot(self, compiler_sysroot):
"""
set compiler sysroot
"""
self.__compiler_sysroot = compiler_sysroot
@property
def compiler_resource_dirs(self):
"""
set compiler resource directories
"""
return self.__compiler_resource_dirs
@compiler_resource_dirs.setter
def compiler_resource_dirs(self, resource_dirs):
"""
set compiler resource directories
"""
self.__compiler_resource_dirs = resource_dirs
@property
def system_includes(self):
"""
"""
return self.__sys_inc
@system_includes.setter
def system_includes(self, includes):
"""
"""
self.__sys_inc = includes
def add_system_includes(self, sys_inc):
"""
add additional system includes if needed
"""
self.__sys_inc.append(sys_inc)
@property
def includes(self):
"""
add additional includes if needed
"""
return self.__includes
@includes.setter
def includes(self, includes):
"""
add additional includes if needed
"""
self.__includes = includes
def add_includes(self, inc):
"""
add additional include paths
"""
self.__includes.append(inc)
@property
def analyzer_extra_arguments(self):
"""
extra arguments fowarded to the analyzer without modification
"""
return self.__analyzer_extra_arguments
@analyzer_extra_arguments.setter
def analyzer_extra_arguments(self, value):
"""
extra arguments fowarded to the analyzer without modification
"""
self.__analyzer_extra_arguments = value

tools/codechecker/libcodechecker/analyzers/config_handler_clang_tidy.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import re
import json
import shlex
import argparse
from libcodechecker.analyzers import config_handler
from libcodechecker import logger
LOG = logger.get_new_logger('CLANG TIDY CONFIG')
class ClangTidyConfigHandler(config_handler.AnalyzerConfigHandler):
'''
Configuration handler for Clang-tidy analyzer
'''
def __init__(self):
super(ClangTidyConfigHandler, self).__init__()
def get_checker_configs(self):
"""
process the raw extra analyzer arguments and get the configuration
data ('-config=' argument for Clang tidy) for the checkers
Clang tidy accepts YAML or JSON formatted config, right now
parsing only the JSON format is supported
return a lis of tuples
(checker_name, key, key_value) list
{
"CheckOptions": [
{
"key": "readability-simplify-boolean-expr.ChainedConditionalReturn",
"value": 1
},
{
"key": "google-readability-namespace-comments.SpacesBeforeComments",
"value": 2
},
{
"key": "modernize-loop-convert.NamingStyle",
"value": "UPPER_CASE"
},
{
"key": "clang-analyzer-unix.Malloc:Optimistic",
"value": true
},
{
"key": "clang-analyzer-unix.test.Checker:Optimistic",
"value": true
}
]
}
"""
LOG.debug(self.analyzer_extra_arguments)
res = []
# match for clang static analyzer names and attributes
clang_sa_checker_rex = r'^clang-analyzer-(?P<checker_name>([^:]+))\:(?P<checker_attribute>([^:]+))$'
# match for clang tidy analyzer names and attributes
clang_tidy_checker_rex = r'^(?P<checker_name>([^.]+))\.(?P<checker_attribute>([^.]+))$'
clangsa_pattern = re.compile(clang_sa_checker_rex)
tidy_pattern = re.compile(clang_tidy_checker_rex)
# get config from the extra arguments if there is any
try:
tidy_config_parser = argparse.ArgumentParser()
tidy_config_parser.add_argument('-config',
dest='tidy_config',
default='',
type=str)
args, _ = tidy_config_parser.parse_known_args(
shlex.split(self.analyzer_extra_arguments))
except Exception as ex:
LOG.debug('No config found in the tidy extra args.')
LOG.debug(ex)
return res
try:
tidy_config = json.loads(args.tidy_config)
for checker_option in tidy_config.get('CheckOptions', []):
value = checker_option['value']
key_values = re.match(clangsa_pattern, checker_option['key'])
key_values_tidy = re.match(tidy_pattern, checker_option['key'])
if key_values:
checker_name = key_values.group('checker_name')
checker_attr = key_values.group('checker_attribute')
res.append((checker_name, checker_attr, value))
elif key_values_tidy:
checker_name = key_values_tidy.group('checker_name')
checker_attr = key_values_tidy.group('checker_attribute')
res.append((checker_name, checker_attr, value))
else:
LOG.debug('no match')
except ValueError as verr:
LOG.debug('Failed to parse config')
LOG.debug(verr)
except Exception as ex:
LOG.debug('Failed to process config')
LOG.debug(ex)
return res

tools/codechecker/libcodechecker/analyzers/config_handler_clangsa.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import re
from libcodechecker import logger
from libcodechecker.analyzers import config_handler
LOG = logger.get_new_logger('CLANGSA CONFIG HANDLER')
class ClangSAConfigHandler(config_handler.AnalyzerConfigHandler):
"""
Configuration handler for the clang static analyzer
"""
def __init__(self):
super(ClangSAConfigHandler, self).__init__()
self.__checker_configs = []
def add_checker_config(self, config):
"""
add a (checker_name, key, value) tuple to the list
"""
self.__checker_configs.append(config)
def get_checker_configs(self):
"""
Process raw config data (from the command line)
Filter out checker related configuration values
like unix:Optimistic=true in the command line:
'-Xanalyzer -analyzer-config -Xanalyzer unix:Optimistic=true'
return a list of (checker_name, key, value) tuples
"""
checker_config_pattern = r'(?P<checker_name>([^: ]+))\:(?P<checker_attr>([^:=]+))\=(?P<attr_value>([^:\. ]+))'
raw_config = self.analyzer_extra_arguments
LOG.debug_analyzer(raw_config)
checker_configs = re.finditer(checker_config_pattern, raw_config)
if checker_configs:
for cfg in checker_configs:
checker_name = cfg.group('checker_name')
checker_attr = cfg.group('checker_attr')
attr_value = cfg.group('attr_value')
self.__checker_configs.append((checker_name,
checker_attr,
attr_value))
LOG.debug_analyzer(self.__checker_configs)
return self.__checker_configs

tools/codechecker/libcodechecker/analyzers/result_handler_base.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import os
import uuid
import ntpath
from abc import ABCMeta, abstractmethod
from libcodechecker import logger
LOG = logger.get_new_logger('RESULT_HANDLER_BASE')
class ResultHandler(object):
"""
handle and store the results at runtime for the analyzer
stdout, stderr, temporarly generated files
do result postprocessing if required
For each buildaction there is one result handler
the result handler can handle multiple results if there
are more than one analyzed source file in one buildaction
handle_results() handles the results of a static analysis tool processed on
a build action. Alternatively one can also handle results given by plist
files for which handle_plist() can be used.
For each build action
- postprocess_result and handle_results can be called multiple times
for the source files in one buildaction they will be analyzed separately
with the same build options
Method call order should be this
postprocess_result()
handle_results()
"""
__metaclass__ = ABCMeta
# handle the output stdout, or plist or both for an analyzer
def __init__(self, action, workspace):
"""
put the temporary files for the workspace
"""
self.__workspace = workspace
self.__analyzer_cmd = []
self.__analyzer_stdout = ''
self.__analyzer_stderr = ''
self.__severity_map = {}
self.__skiplist_handler = None
self.__analyzed_source_file = None
self.__analyzer_returncode = 1
self.__buildaction = action
self.__res_file = None
@property
def buildaction(self):
"""
"""
return self.__buildaction
@property
def analyzer_cmd(self):
"""
set the analyzer cmd
"""
return self.__analyzer_cmd
@analyzer_cmd.setter
def analyzer_cmd(self, cmd):
"""
set the analyzer cmd
"""
self.__analyzer_cmd = cmd
@property
def result_files(self):
"""
return the analyzer result files
"""
return self.__result_files
@property
def skiplist_handler(self):
"""
"""
return self.__skiplist_handler
@skiplist_handler.setter
def skiplist_handler(self, handler):
"""
used to check if analyzer result should be
handled or just skipped
"""
self.__skiplist_handler = handler
@property
def severity_map(self):
"""
severity map for each checker
"""
return self.__severity_map
@severity_map.setter
def severity_map(self, value):
"""
severity map for each checker
"""
self.__severity_map = value
@property
def workspace(self):
"""
workspace where the analysis results should go
temporarly generated files
"""
return self.__workspace
@property
def analyzer_returncode(self):
"""
"""
return self.__analyzer_returncode
@analyzer_returncode.setter
def analyzer_returncode(self, return_code):
"""
"""
self.__analyzer_returncode = return_code
@property
def analyzer_stdout(self):
"""
get the stdout from the analyzer
"""
return self.__analyzer_stdout
@analyzer_stdout.setter
def analyzer_stdout(self, stdout):
"""
set the stdout of the analyzer
"""
self.__analyzer_stdout = stdout
@property
def analyzer_stderr(self):
"""
get stderr of the analyzer
"""
return self.__analyzer_stderr
@analyzer_stderr.setter
def analyzer_stderr(self, stderr):
"""
set the stderr of the analyzer
"""
self.__analyzer_stderr = stderr
@property
def analyzed_source_file(self):
"""
the source file which is analyzed
"""
return self.__analyzed_source_file
@analyzed_source_file.setter
def analyzed_source_file(self, file_path):
"""
the source file which is analyzed
"""
self.__analyzed_source_file = file_path
def get_analyzer_result_file(self):
"""
generate a result filename where the analyzer should put the analyzer result
result file should be removed by the result handler if not required anymore
"""
if not self.__res_file:
analyzed_file = self.analyzed_source_file
_, analyzed_file_name = ntpath.split(analyzed_file)
uid = str(uuid.uuid1()).split('-')[0]
out_file_name = str(self.buildaction.analyzer_type) + \
'_' + analyzed_file_name + '_' + uid + '.plist'
out_file = os.path.join(self.__workspace, out_file_name)
self.__res_file = out_file
return self.__res_file
def clean_results(self):
"""
should be called after the postprocessing and result handling is done
"""
if self.__res_file:
try:
os.remove(self.__res_file)
except OSError as oserr:
# there might be no result file if analysis failed
LOG.debug(oserr)
pass
@abstractmethod
def postprocess_result(self):
"""
postprocess result if needed
should be called after the analisys finished
"""
pass
@abstractmethod
def handle_plist(self, plist):
"""
handle the results directly from the given plist file
"""
pass
@abstractmethod
def handle_results(self):
"""
handle the results
"""
pass

tools/codechecker/libcodechecker/analyzers/result_handler_clang_tidy.py

  • This file was added.
# -------------------------------------------------------------------------
# The CodeChecker Infrastructure
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
# -------------------------------------------------------------------------
from libcodechecker import logger
from libcodechecker import tidy_output_converter
from libcodechecker.analyzers.result_handler_plist_to_db import PlistToDB
from libcodechecker.analyzers.result_handler_plist_to_stdout import PlistToStdout
LOG = logger.get_new_logger('CLANG_TIDY_RESULT_HANDLER')
def generate_plist_from_tidy_result(output_file, tidy_stdout):
"""
Generate a plist file from the clang tidy analyzer results
"""
parser = tidy_output_converter.OutputParser()
messages = parser.parse_messages(tidy_stdout)
plist_converter = tidy_output_converter.PListConverter()
plist_converter.add_messages(messages)
plist_converter.write_to_file(output_file)
class ClangTidyPlistToDB(PlistToDB):
"""
Store clang tidy plist results to a database
"""
def postprocess_result(self):
"""
Generate plist file which can be parsed and processed for
results which can be stored into the database
"""
output_file = self.get_analyzer_result_file()
LOG.debug_analyzer(self.analyzer_stdout)
tidy_stdout = self.analyzer_stdout.splitlines()
generate_plist_from_tidy_result(output_file, tidy_stdout)
class ClangTidyPlistToStdout(PlistToStdout):
"""
Print the clang tidy results to the standatd output
"""
def postprocess_result(self):
"""
For the same output format with the clang static analyzer the
Clang tidy results are postprocessed into a plist file
"""
output_file = self.get_analyzer_result_file()
tidy_stdout = self.analyzer_stdout.splitlines()
generate_plist_from_tidy_result(output_file, tidy_stdout)

tools/codechecker/libcodechecker/analyzers/result_handler_clangsa.py

  • This file was added.
# -------------------------------------------------------------------------
# The CodeChecker Infrastructure
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
# -------------------------------------------------------------------------
"""
result handlers for the Clang Static Analyzer
"""
from libcodechecker.analyzers.result_handler_plist_to_db import PlistToDB
from libcodechecker.analyzers.result_handler_plist_to_stdout import PlistToStdout
class ClangSAPlistToDB(PlistToDB):
"""
Store the result plist generated by the Clang Static Analyzer
to the database
"""
def postprocess_result(self):
"""
No postprocessing required for clang static analyzer
"""
pass
class ClangSAPlistToStdout(PlistToStdout):
"""
Print the results from the generated plist file to the standard output
"""
def postprocess_result(self):
"""
No postprocessing required for clang static analyzer
"""
pass

tools/codechecker/libcodechecker/analyzers/result_handler_plist_to_db.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import os
import zlib
import ntpath
from abc import ABCMeta
from libcodechecker import client
from libcodechecker import logger
from libcodechecker import plist_parser
from libcodechecker import suppress_handler
from libcodechecker.analyzers.result_handler_base import ResultHandler
import shared
LOG = logger.get_new_logger('PLIST TO DB')
class PlistToDB(ResultHandler):
"""
Result handler for processing a plist file with the
analysis results and stores them to the database.
"""
__metaclass__ = ABCMeta
def __init__(self, buildaction, workspace, run_id):
super(PlistToDB, self).__init__(buildaction, workspace)
self.__run_id = run_id
def __store_bugs(self, files, bugs, connection, analisys_id):
file_ids = {}
# Send content of file to the server if needed
for file_name in files:
file_descriptor = connection.need_file_content(self.__run_id,
file_name)
file_ids[file_name] = file_descriptor.fileId
# sometimes the file doesn't exist, e.g. when the input of the
# analysis is pure plist files
if not os.path.isfile(file_name):
LOG.debug(file_name + ' not found, and will not be stored')
continue
if file_descriptor.needed:
with open(file_name, 'r') as source_file:
file_content = source_file.read()
compressed_file = zlib.compress(file_content,
zlib.Z_BEST_COMPRESSION)
# TODO: we may not use the file content in the end
# depending on skippaths
LOG.debug('storing file content to the database')
connection.add_file_content(file_descriptor.fileId,
compressed_file)
# skipping bugs in header files handled here
report_ids = []
for bug in bugs:
events = bug.events()
# skip list handler can be None if no config file is set
if self.skiplist_handler:
if events and self.skiplist_handler.should_skip(
events[-1].start_pos.file_path):
# Issue #20: this bug is in a file which should be skipped
LOG.debug(bug.hash_value + ' is skipped (in ' +
events[-1].start_pos.file_path + ")")
continue
# create remaining data for bugs and send them to the server
bug_paths = []
for path in bug.paths():
bug_paths.append(
shared.ttypes.BugPathPos(path.start_pos.line,
path.start_pos.col,
path.end_pos.line,
path.end_pos.col,
file_ids[path.start_pos.file_path]))
bug_events = []
for event in bug.events():
bug_events.append(shared.ttypes.BugPathEvent(
event.start_pos.line,
event.start_pos.col,
event.end_pos.line,
event.end_pos.col,
event.msg,
file_ids[event.start_pos.file_path]))
bug_hash = bug.hash_value
severity_name = self.severity_map.get(bug.checker_name,
'UNSPECIFIED')
severity = shared.ttypes.Severity._NAMES_TO_VALUES[severity_name]
suppress = False
source_file = bug.file_path
last_bug_event = bug.events()[-1]
bug_line = last_bug_event.start_pos.line
sp_handler = suppress_handler.SourceSuppressHandler(source_file,
bug_line)
# check for suppress comment
supp = sp_handler.check_source_suppress()
if supp:
# something shoud be suppressed
suppress_checkers = sp_handler.suppressed_checkers()
if bug.checker_name in suppress_checkers or \
suppress_checkers == ['all']:
suppress = True
file_path, file_name = ntpath.split(source_file)
# checker_hash, file_name, comment
to_suppress = (bug_hash,
file_name,
sp_handler.suppress_comment())
LOG.debug(to_suppress)
connection.add_suppress_bug(self.__run_id, [to_suppress])
LOG.debug('Storing check results to the database')
report_id = connection.add_report(analisys_id,
file_ids[bug.file_path],
bug_hash,
bug.msg,
bug_paths,
bug_events,
bug.checker_name,
bug.category,
bug.type,
severity,
suppress)
report_ids.append(report_id)
def handle_plist(self, plist):
with client.get_connection() as connection:
# TODO: When the analyzer name can be read from PList, then it
# should be passed too.
# TODO: File name should be read from the PList and passed.
analisys_id = connection.add_build_action(self.__run_id,
plist,
'Build action from plist',
'',
'')
try:
files, bugs = plist_parser.parse_plist(plist)
except Exception as ex:
msg = 'Parsing the generated result file failed'
LOG.error(msg + ' ' + plist)
connection.finish_build_action(analysis_id, msg)
return 1
self.__store_bugs(files, bugs, connection, analisys_id)
connection.finish_build_action(analisys_id, self.analyzer_stderr)
def handle_results(self):
"""
send the plist content to the database
server API calls should be used in one connection
- addBuildAction
- addReport
- needFileContent
- addFileContent
- finishBuildAction
"""
with client.get_connection() as connection:
LOG.debug('Storing original build and analyzer command to the database')
_, source_file_name = ntpath.split(self.analyzed_source_file)
analisys_id = connection.add_build_action(self.__run_id,
self.buildaction.original_command,
' '.join(self.analyzer_cmd),
self.buildaction.analyzer_type,
source_file_name)
# store buildaction and analyzer command to the database
if self.analyzer_returncode == 0:
LOG.info(self.buildaction.analyzer_type + ' analyzed ' +
source_file_name + ' successfully.')
plist_file = self.get_analyzer_result_file()
try:
files, bugs = plist_parser.parse_plist(plist_file)
except Exception as ex:
LOG.debug(str(ex))
msg = 'Parsing the generated result file failed'
LOG.error(msg + ' ' + plist_file)
connection.finish_build_action(analisys_id, msg)
return 1
self.__store_bugs(files, bugs, connection, analisys_id)
else:
LOG.info('Analysing ' + source_file_name +
' with ' + self.buildaction.analyzer_type +
' failed.')
connection.finish_build_action(analisys_id, self.analyzer_stderr)

tools/codechecker/libcodechecker/analyzers/result_handler_plist_to_stdout.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import os
import sys
import math
import ntpath
import linecache
from abc import ABCMeta
from libcodechecker import logger
from libcodechecker import plist_parser
from libcodechecker.analyzers.result_handler_base import ResultHandler
LOG = logger.get_new_logger('PLIST TO STDOUT')
class PlistToStdout(ResultHandler):
"""
Result handler for processing a plist file with the
analysis results and print them to the standard output.
"""
__metaclass__ = ABCMeta
def __init__(self, buildaction, workspace):
super(PlistToStdout, self).__init__(buildaction, workspace)
self.__print_steps = False
self.__output = sys.stdout
@property
def print_steps(self):
"""
Print the multiple steps for a bug if there is any
"""
return self.__print_steps
@print_steps.setter
def print_steps(self, value):
"""
Print the multiple steps for a bug if there is any
"""
self.__print_steps = value
def __format_location(self, event):
pos = event.start_pos
line = linecache.getline(pos.file_path, pos.line)
if line == '':
return line
marker_line = line[0:(pos.col-1)]
marker_line = ' ' * (len(marker_line) + marker_line.count('\t'))
return '%s%s^' % (line.replace('\t', ' '), marker_line)
def __format_bug_event(self, event):
pos = event.start_pos
fname = os.path.basename(pos.file_path)
return '%s:%d:%d: %s' % (fname, pos.line, pos.col, event.msg)
def __print_bugs(self, bugs):
index_format = ' %%%dd, ' % int(math.floor(math.log10(len(bugs)))+1)
for bug in bugs:
last_event = bug.get_last_event()
self.__output.write(self.__format_bug_event(last_event))
self.__output.write('\n')
self.__output.write(self.__format_location(last_event))
self.__output.write('\n')
if self.__print_steps:
self.__output.write(' Steps:\n')
for index, event in enumerate(bug.events()):
self.__output.write(index_format % (index + 1))
self.__output.write(self.__format_bug_event(event))
self.__output.write('\n')
self.__output.write('\n')
def handle_plist(self, plist):
try:
_, bugs = plist_parser.parse_plist(plist)
except Exception as ex:
LOG.error('The generated plist is not valid!')
LOG.error(ex)
return 1
if len(bugs) > 0:
self.__output.write("%s contains %d defect(s)\n\n" % (plist, len(bugs)))
self.__print_bugs(bugs)
else:
self.__output.write("%s doesn't contain any defects\n" % plist)
def handle_results(self):
source = self.analyzed_source_file
_, source_file_name = ntpath.split(source)
plist = self.get_analyzer_result_file()
try:
_, bugs = plist_parser.parse_plist(plist)
except Exception as ex:
LOG.error('The generated plist is not valid!')
LOG.error(ex)
return 1
err_code = self.analyzer_returncode
if err_code == 0:
if len(bugs) > 0:
self.__output.write('%s found %d defect(s) while analyzing %s\n\n' %
(self.buildaction.analyzer_type, len(bugs),
source_file_name))
self.__print_bugs(bugs)
else:
self.__output.write('%s found no defects while analyzing %s\n' %
(self.buildaction.analyzer_type,
source_file_name))
return err_code
else:
self.__output.write('Analyzing %s with %s failed.\n' %
(source_file_name,
self.buildaction.analyzer_type))
return err_code

tools/codechecker/libcodechecker/arg_handler.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
handle command line arguments
'''
import json
import multiprocessing
import os
import shutil
import sys
import tempfile
from libcodechecker import analyzer
from libcodechecker import analyzer_env
from libcodechecker import build_action
from libcodechecker import build_manager
from libcodechecker import client
from libcodechecker import debug_reporter
from libcodechecker import generic_package_context
from libcodechecker import generic_package_suppress_handler
from libcodechecker import host_check
from libcodechecker import log_parser
from libcodechecker import logger
from libcodechecker import util
from libcodechecker.analyzers import analyzer_types
from libcodechecker.database_handler import SQLServer
from libcodechecker.viewer_server import client_db_access_server
LOG = logger.get_new_logger('ARG_HANDLER')
def handle_list_checkers(args):
"""
list the supported checkers by the analyzers
list the default enabled and disabled checkers
in the config
"""
context = generic_package_context.get_context()
enabled_analyzers = args.analyzers
analyzer_environment = analyzer_env.get_check_env(context.path_env_extra,
context.ld_lib_path_extra)
if not enabled_analyzers:
# noting set list checkers for all supported analyzers
enabled_analyzers = list(analyzer_types.supported_analyzers)
enabled_analyzer_types = set()
for ea in enabled_analyzers:
if ea not in analyzer_types.supported_analyzers:
LOG.info('Not supported analyzer ' + str(ea))
sys.exit(1)
else:
enabled_analyzer_types.add(ea)
analyzer_config_map = analyzer_types.build_config_handlers(args,
context,
enabled_analyzer_types)
for ea in enabled_analyzers:
# get the config
config_handler = analyzer_config_map.get(ea)
source_analyzer = analyzer_types.construct_analyzer_type(ea,
config_handler,
None)
checkers = source_analyzer.get_analyzer_checkers(config_handler,
analyzer_environment)
default_checker_cfg = context.default_checkers_config.get(ea+'_checkers')
analyzer_types.initialize_checkers(config_handler,
checkers,
default_checker_cfg)
for checker_name, value in config_handler.checks().iteritems():
enabled, description = value
if enabled:
print(' + {0:50} {1}'.format(checker_name, description))
else:
print(' - {0:50} {1}'.format(checker_name, description))
def handle_server(args):
"""
starts the report viewer server
"""
if not host_check.check_zlib():
LOG.error("zlib error")
sys.exit(1)
try:
workspace = args.workspace
except AttributeError:
# if no workspace value was set for some reason
# in args set the default value
workspace = util.get_default_workspace()
# WARNING
# in case of SQLite args.dbaddress default value is used
# for which the is_localhost should return true
local_db = util.is_localhost(args.dbaddress)
if local_db and not os.path.exists(workspace):
os.makedirs(workspace)
if args.suppress is None:
LOG.warning('WARNING! No suppress file was given, suppressed results will be only stored in the database.')
else:
if not os.path.exists(args.suppress):
LOG.error('Suppress file '+args.suppress+' not found!')
sys.exit(1)
context = generic_package_context.get_context()
context.codechecker_workspace = workspace
context.db_username = args.dbusername
check_env = analyzer_env.get_check_env(context.path_env_extra,
context.ld_lib_path_extra)
sql_server = SQLServer.from_cmdline_args(args,
context.codechecker_workspace,
context.migration_root,
check_env)
conn_mgr = client.ConnectionManager(sql_server, args.check_address, args.check_port)
if args.check_port:
LOG.debug('Starting codechecker server and database server.')
sql_server.start(context.db_version_info, wait_for_start=True, init=True)
conn_mgr.start_report_server(context.db_version_info)
else:
LOG.debug('Starting database.')
sql_server.start(context.db_version_info, wait_for_start=True, init=True)
# start database viewer
db_connection_string = sql_server.get_connection_string()
suppress_handler = generic_package_suppress_handler.GenericSuppressHandler()
try:
suppress_handler.suppress_file = args.suppress
LOG.debug('Using suppress file: ' + str(suppress_handler.suppress_file))
except AttributeError as aerr:
# suppress file was not set
LOG.debug(aerr)
package_data = {}
package_data['www_root'] = context.www_root
package_data['doc_root'] = context.doc_root
checker_md_docs = os.path.join(context.doc_root, 'checker_md_docs')
checker_md_docs_map = os.path.join(checker_md_docs,
'checker_doc_map.json')
package_data['checker_md_docs'] = checker_md_docs
with open(checker_md_docs_map, 'r') as dFile:
checker_md_docs_map = json.load(dFile)
package_data['checker_md_docs_map'] = checker_md_docs_map
client_db_access_server.start_server(package_data,
args.view_port,
db_connection_string,
suppress_handler,
args.not_host_only,
context.db_version_info)
def handle_log(args):
"""
Generates a build log by running the original build command
No analysis is done
"""
args.logfile = os.path.realpath(args.logfile)
if os.path.exists(args.logfile):
os.remove(args.logfile)
context = generic_package_context.get_context()
open(args.logfile, 'a').close() # same as linux's touch
build_manager.perform_build_command(args.logfile,
args.command,
context)
def handle_debug(args):
"""
Runs a debug command on the buildactions where the analysis
failed for some reason
"""
context = generic_package_context.get_context()
try:
workspace = args.workspace
except AttributeError:
# if no workspace value was set for some reason
# in args set the default value
workspace = util.get_default_workspace()
context.codechecker_workspace = workspace
context.db_username = args.dbusername
check_env = analyzer_env.get_check_env(context.path_env_extra,
context.ld_lib_path_extra)
sql_server = SQLServer.from_cmdline_args(args,
context.codechecker_workspace,
context.migration_root,
check_env)
sql_server.start(context.db_version_info, wait_for_start=True, init=False)
debug_reporter.debug(context, sql_server.get_connection_string(), args.force)
def handle_check(args):
"""
Runs the original build and logs the buildactions
Based on the log runs the analysis
"""
try:
if not host_check.check_zlib():
LOG.error("zlib error")
sys.exit(1)
try:
workspace = args.workspace
except AttributeError:
# if no workspace value was set for some reason
# in args set the default value
workspace = util.get_default_workspace()
workspace = os.path.realpath(workspace)
if not os.path.isdir(workspace):
os.mkdir(workspace)
context = generic_package_context.get_context()
context.codechecker_workspace = workspace
context.db_username = args.dbusername
log_file = build_manager.check_log_file(args)
if not log_file:
log_file = build_manager.generate_log_file(args,
context)
if not log_file:
LOG.error("Failed to generate compilation command file: " +
log_file)
sys.exit(1)
try:
actions = log_parser.parse_log(log_file)
except Exception as ex:
LOG.error(ex)
sys.exit(1)
if not actions:
LOG.warning('There are no build actions in the log file.')
sys.exit(1)
check_env = analyzer_env.get_check_env(context.path_env_extra,
context.ld_lib_path_extra)
sql_server = SQLServer.from_cmdline_args(args,
context.codechecker_workspace,
context.migration_root,
check_env)
conn_mgr = client.ConnectionManager(sql_server, 'localhost', util.get_free_port())
sql_server.start(context.db_version_info, wait_for_start=True, init=True)
conn_mgr.start_report_server(context.db_version_info)
LOG.debug("Checker server started.")
analyzer.run_check(args,
actions,
context)
LOG.info("Analysis has finished.")
db_data = ""
if args.postgresql:
db_data += " --postgresql" \
+ " --dbname " + args.dbname \
+ " --dbport " + str(args.dbport) \
+ " --dbusername " + args.dbusername
LOG.info("To view results run:\nCodeChecker server -w " +
workspace + db_data)
except Exception as ex:
LOG.error(ex)
import traceback
print(traceback.format_exc())
def _do_quickcheck(args):
'''
Handles the "quickcheck" command.
For arguments see main function in CodeChecker.py. It also requires an extra
property in args object, namely workspace which is a directory path as a
string. This function is called from handle_quickcheck.
'''
context = generic_package_context.get_context()
try:
workspace = args.workspace
except AttributeError:
# if no workspace value was set for some reason
# in args set the default value
workspace = util.get_default_workspace()
context.codechecker_workspace = workspace
# load severity map from config file
if os.path.exists(context.checkers_severity_map_file):
with open(context.checkers_severity_map_file, 'r') as sev_conf_file:
severity_config = sev_conf_file.read()
context.severity_map = json.loads(severity_config)
log_file = build_manager.check_log_file(args)
if not log_file:
log_file = build_manager.generate_log_file(args,
context)
if not log_file:
LOG.error("Failed to generate compilation command file: " + log_file)
sys.exit(1)
try:
actions = log_parser.parse_log(log_file)
except Exception as ex:
LOG.error(ex)
sys.exit(1)
if not actions:
LOG.warning('There are no build actions in the log file.')
sys.exit(1)
analyzer.run_quick_check(args, context, actions)
def handle_quickcheck(args):
'''
Handles the "quickcheck" command using _do_quickcheck function.
It creates a new temporary directory and sets it as workspace directory.
After _do_quickcheck call it deletes the temporary directory and its
content.
'''
args.workspace = tempfile.mkdtemp(prefix='codechecker-qc')
try:
_do_quickcheck(args)
finally:
shutil.rmtree(args.workspace)
def consume_plist(item):
plist, args, context = item
LOG.info('Consuming ' + plist)
action = build_action.BuildAction()
action.analyzer_type = analyzer_types.CLANG_SA
action.original_command = 'Imported from PList directly'
rh = analyzer_types.construct_result_handler(args,
action,
context.run_id,
args.directory,
{},
None,
not args.stdout)
rh.handle_plist(os.path.join(args.directory, plist))
def handle_plist(args):
context = generic_package_context.get_context()
context.codechecker_workspace = args.workspace
context.db_username = args.dbusername
if not args.stdout:
args.workspace = os.path.realpath(args.workspace)
if not os.path.isdir(args.workspace):
os.mkdir(args.workspace)
check_env = analyzer_env.get_check_env(context.path_env_extra,
context.ld_lib_path_extra)
sql_server = SQLServer.from_cmdline_args(args,
context.codechecker_workspace,
context.migration_root,
check_env)
conn_mgr = client.ConnectionManager(sql_server,
'localhost',
util.get_free_port())
sql_server.start(context.db_version_info, wait_for_start=True, init=True)
conn_mgr.start_report_server(context.db_version_info)
with client.get_connection() as connection:
package_version = context.version['major'] + '.' + context.version['minor']
context.run_id = connection.add_checker_run(' '.join(sys.argv),
args.name,
package_version,
args.force)
pool = multiprocessing.Pool(args.jobs)
try:
items = [(plist, args, context) for plist in os.listdir(args.directory)]
pool.map_async(consume_plist, items, 1).get(float('inf'))
pool.close()
except Exception:
pool.terminate()
raise
finally:
pool.join()
def handle_version_info(args):
''' Get and print the version informations from the
version config file and thrift API versions'''
context = generic_package_context.get_context()
version_file = context.version_file
v_data = ''
try:
with open(version_file) as v_file:
v_data = v_file.read()
version_data = json.loads(v_data)
base_version = version_data['version']['major'] + \
'.' + version_data['version']['minor']
db_schema_version = version_data['db_version']['major'] + \
'.'+version_data['db_version']['minor']
print('Base package version: \t' + base_version).expandtabs(30)
print('Package build date: \t' +
version_data['package_build_date']).expandtabs(30)
print('Git hash: \t' + version_data['git_hash']).expandtabs(30)
print('DB schema version: \t' + db_schema_version).expandtabs(30)
except ValueError as verr:
LOG.error('Failed to decode version information from the config file.')
LOG.error(verr)
sys.exit(1)
except IOError as ioerr:
LOG.error('Failed to read version config file: ' + version_file)
LOG.error(ioerr)
sys.exit(1)
# thift api version for the clients
from codeCheckerDBAccess import constants
print('Thrift client api version: \t' +
constants.API_VERSION).expandtabs(30)

tools/codechecker/libcodechecker/build_action.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
''''''
import hashlib
class BuildAction(object):
def __init__(self, build_action_id=0):
self._id = build_action_id
self._analyzer_options = []
self.analyzer_type = -1
self._original_command = ''
self._directory = ''
self._output = ''
self._lang = None
self._target = ''
self._source_count = 0
self._sources = []
self._skip = True
def __str__(self):
# for debugging
return ('Id: {0} ,\nOriginal command: {1},\n'
'Analyzer type: {2},\n Analyzer options: {3},\n'
'Directory: {4},\nOutput: {5},\nLang: {6},\nTarget: {7},\n'
'Source count {8},\nSources: {9}').\
format(self._id, self._original_command,
self._analyzer_type, self._analyzer_options,
self._directory, self._output, self._lang, self._target,
self._source_count, self._sources)
@property
def analyzer_type(self):
"""
stores which type of analyzer should be run in this
buildaction
"""
return self._analyzer_type
@analyzer_type.setter
def analyzer_type(self, value):
"""
stores which type of analyzer should be run in this
buildaction
"""
self._analyzer_type = value
@property
def id(self):
return self._id
@property
def analyzer_options(self):
return self._analyzer_options
@analyzer_options.setter
def analyzer_options(self, value):
'''A filtered compile arguments which will be forwarded to the analyzer.'''
self._analyzer_options = value
@property
def original_command(self):
return self._original_command
@original_command.setter
def original_command(self, value):
self._original_command = value
@property
def directory(self):
return self._directory
@directory.setter
def directory(self, value):
self._directory = value
@property
def output(self):
return self._output
@output.setter
def output(self, value):
self._output = value
@property
def source_count(self):
return self._source_count
@source_count.setter
def source_count(self, count):
self._source_count = count
@property
def sources(self):
for source in self._sources:
yield source
@sources.setter
def sources(self, value):
self._sources.append(value)
self._source_count += 1
@property
def lang(self):
return self._lang
@lang.setter
def lang(self, value):
self._lang = value
@property
def target(self):
return self._target
@target.setter
def target(self, value):
self._target = value
@property
def skip(self):
return self._skip
@skip.setter
def skip(self, value):
self._skip = value
def __eq__(self, other):
return other._original_command == self._original_command
@property
def cmp_key(self):
'''
If the compilation database contains the same compilation action
multiple times it should be checked only once.
Use this key to compare compilation commands for the analysis.
'''
hash_content = []
hash_content.extend(self.analyzer_options)
hash_content.append(str(self.analyzer_type))
hash_content.append(self.output)
hash_content.append(self.target)
hash_content.extend(self.sources)
return hashlib.sha1(''.join(hash_content)).hexdigest()

tools/codechecker/libcodechecker/build_manager.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
"""
build and log related stuff
"""
import os
import pickle
import subprocess
import sys
import shutil
import platform
from libcodechecker import logger
from libcodechecker import analyzer_env
from libcodechecker import host_check
from distutils.spawn import find_executable
LOG = logger.get_new_logger('BUILD MANAGER')
def execute_buildcmd(command, silent=False, env=None, cwd=None):
"""
Execute the the build command and continously write
the output from the process to the standard output.
"""
proc = subprocess.Popen(command,
rizsotto.mailinglistUnsubmitted
Not Done
ReplyInline Actions

is there any reason not to use subprocess.call instead? if you want that to be in silent, you shall pass {'stdout': subprocess.PIPE, 'stderr': subprocess.STDOUT} to it, otherwise it prints the output without you doing the copy. also noticed the shell=True which makes it very non-portable. and the command parameter is a string instead of a list. is there any reason for doing like this?

rizsotto.mailinglist: is there any reason not to use `subprocess.call` instead? if you want that to be in silent, you…
bufsize=-1,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
cwd=cwd,
shell=True)
while True:
line = proc.stdout.readline()
if not silent:
sys.stdout.write(line)
if line == '' and proc.poll() is not None:
break
return proc.returncode
def perform_build_command(logfile, command, context, silent=False):
"""
Build the project and create a log file.
"""
if not silent:
LOG.info("Starting build ...")
try:
rizsotto.mailinglistUnsubmitted
Not Done
ReplyInline Actions

save-to-file and restore-from-file the environment would deserve a separate module i would say. is there any reason why it's not having those utility methods?

rizsotto.mailinglist: save-to-file and restore-from-file the environment would deserve a separate module i would say.
original_env_file = os.environ['CODECHECKER_ORIGINAL_BUILD_ENV']
LOG.debug_analyzer('Loading original build env from: ' + original_env_file)
with open(original_env_file, 'rb') as env_file:
original_env = pickle.load(env_file)
except Exception as ex:
LOG.warning(str(ex))
LOG.warning('Failed to get saved original_env using a current copy for logging')
original_env = os.environ.copy()
return_code = 0
# Run user's commands with intercept
if host_check.check_intercept(original_env):
LOG.debug_analyzer("with intercept ...")
final_command = command
command = "intercept-build " + "--cdb "+ logfile + " " + final_command
log_env = original_env
LOG.debug_analyzer(command)
# Run user's commands in shell
else:
# TODO better platform detection
if platform.system() == 'Linux':
LOG.debug_analyzer("with ld logger ...")
log_env = analyzer_env.get_log_env(logfile, context, original_env)
if 'CC_LOGGER_GCC_LIKE' not in log_env:
log_env['CC_LOGGER_GCC_LIKE'] = 'gcc:g++:clang:clang++:cc:c++'
else:
LOG.error("Intercept-build is required to run CodeChecker in OS X.")
sys.exit(1)
LOG.debug_analyzer(log_env)
try:
ret_code = execute_buildcmd(command, silent, log_env)
if not silent:
if ret_code == 0:
LOG.info("Build finished successfully.")
LOG.debug_analyzer("The logfile is: " + logfile)
else:
LOG.info("Build failed.")
sys.exit(ret_code)
except Exception as ex:
LOG.error("Calling original build command failed")
LOG.error(str(ex))
sys.exit(1)
def default_compilation_db(workspace_path):
"""
default compilation commands database file in the workspace
"""
compilation_commands = os.path.join(workspace_path,
'compilation_commands.json')
return compilation_commands
def check_log_file(args):
"""
check if the compilation command file was set in the command line
if not check if it is in the workspace directory
"""
log_file = None
try:
if args.logfile:
log_file = os.path.realpath(args.logfile)
else:
# log file could be in the workspace directory
log_file = default_compilation_db(args.workspace)
if not os.path.exists(log_file):
LOG.debug_analyzer("Compilation database file does not exists.")
return None
except AttributeError as ex:
rizsotto.mailinglistUnsubmitted
Not Done
ReplyInline Actions

why not have a command line parameter accessor method, that would either return the value if that was given or None? i found this pattern multiple files. that would get rid of the try block completely.

rizsotto.mailinglist: why not have a command line parameter accessor method, that would either return the value if…
# args.log_file was not set
LOG.debug_analyzer(ex)
LOG.debug_analyzer("Compilation database file was not set in the command line.")
finally:
return log_file
def generate_log_file(args, context, silent=False):
"""
Returns a build command log file for check/quickcheck command.
"""
log_file = None
try:
if args.command:
intercept_build_executable = find_executable('intercept-build')
if intercept_build_executable == None:
rizsotto.mailinglistUnsubmitted
Not Done
ReplyInline Actions

if intercept_build_executable is None: would be more python.

rizsotto.mailinglist: `if intercept_build_executable is None:` would be more python.
if platform.system() == 'Linux':
# check if logger bin exists
if not os.path.isfile(context.path_logger_bin):
LOG.error('Logger binary not found! Required for logging.')
sys.exit(1)
# check if logger lib exists
if not os.path.exists(context.path_logger_lib):
LOG.error('Logger library directory not found! Libs are requires' \
'for logging.')
sys.exit(1)
log_file = default_compilation_db(args.workspace)
if os.path.exists(log_file):
LOG.debug_analyzer("Removing previous compilation command file: " +
log_file)
os.remove(log_file)
open(log_file, 'a').close() # same as linux's touch
rizsotto.mailinglistUnsubmitted
Not Done
ReplyInline Actions

the comment is not really helpful!
why does it need a touch? or you just create an empty one? what for?

rizsotto.mailinglist: the comment is not really helpful! why does it need a touch? or you just create an empty one?
perform_build_command(log_file,
args.command,
context,
silent=silent)
except AttributeError as aerr:
LOG.error(aerr)
rizsotto.mailinglistUnsubmitted
Not Done
ReplyInline Actions

logger.exception is logging the exception value implicitly at ERROR level too. but what i'm trying to say is, there is no error message in this line. you just throw the exception into the user face. can you add some explanation or instruction for the user?

rizsotto.mailinglist: `logger.exception` is logging the exception value implicitly at `ERROR` level too. but what i'm…
sys.exit(1)
return log_file

tools/codechecker/libcodechecker/checkers.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
from codechecker import util
from codechecker import globalConfig
from codechecker import logger
def get_env_checkers_list(env_name):
env_set_checkers = util.get_env_var(env_name)
LOG = logger.get_new_logger("CHECKERS")
LOG.debug_analyzer('Getting checkers list from environment variable %s' % (env_name))
if env_set_checkers:
checkers_list = env_set_checkers.split(':')
LOG.debug_analyzer('Checkers list is -> ' + env_set_checkers)
return sorted(checkers_list)
else:
LOG.debug_analyzer('No checkers list was defined.')
return None
def get_enabled_checkers():
config = globalConfig.GlobalConfig()
env_checkers = get_env_checkers_list(config.envEnableCheckersName)
return env_checkers
def get_disabled_checkers():
config = globalConfig.GlobalConfig()
env_checkers = get_env_checkers_list(config.envDisableCheckersName)
return env_checkers

tools/codechecker/libcodechecker/client.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import sys
import time
import atexit
import contextlib
import multiprocessing
import os
import codecs
import shared
from libcodechecker.storage_server import report_server
from DBThriftAPI import CheckerReport
from DBThriftAPI.ttypes import SuppressBugData
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from libcodechecker import logger
from libcodechecker import suppress_file_handler
LOG = logger.get_new_logger('CLIENT')
# -----------------------------------------------------------------------------
def clean_suppress(connection, run_id):
"""
clean all the suppress information from the database
"""
connection.clean_suppress_data(run_id)
# -----------------------------------------------------------------------------
def send_suppress(run_id, connection, file_name):
"""
collect suppress information from the suppress file to be stored
in the database
"""
suppress_data = []
if os.path.exists(file_name):
with codecs.open(file_name, 'r', 'UTF-8') as s_file:
suppress_data = suppress_file_handler.get_suppress_data(s_file)
if len(suppress_data) > 0:
connection.add_suppress_bug(run_id, suppress_data)
# -----------------------------------------------------------------------------
def replace_config_in_db(run_id, connection, configs):
configuration_list = []
for checker_name, key, key_value in configs:
configuration_list.append(shared.ttypes.ConfigValue(checker_name,
key,
str(key_value)))
# store checker configs to the database
connection.replace_config_info(run_id, configuration_list)
# -----------------------------------------------------------------------------
@contextlib.contextmanager
def get_connection():
''' Automatic Connection handler via ContextManager idiom.
You can use this in with statement.'''
connection = ConnectionManager.instance.create_connection()
try:
yield connection
finally:
connection.close_connection()
# -----------------------------------------------------------------------------
class Connection(object):
''' Represent a connection to the server.
In contstructor establish the connection and
you have to call close_connection function to close it.
So, you should set it up before create a connection.'''
def __init__(self, host, port):
''' Establish the connection beetwen client and server. '''
tries_count = 0
while True:
try:
self._transport = TSocket.TSocket(host, port)
self._transport = TTransport.TBufferedTransport(self._transport)
self._protocol = TBinaryProtocol.TBinaryProtocol(self._transport)
self._client = CheckerReport.Client(self._protocol)
self._transport.open()
break
except Thrift.TException as thrift_exc:
if tries_count > 3:
LOG.error('The client cannot establish the connection '
'with the server!')
LOG.error('%s' % (thrift_exc.message))
sys.exit(2)
else:
tries_count += 1
time.sleep(1)
def close_connection(self):
''' Close connection. '''
self._transport.close()
def add_checker_run(self, command, name, version, force):
run_id = self._client.addCheckerRun(command, name, version, force)
return run_id
def finish_checker_run(self, run_id):
''' bool finishCheckerRun(1: i64 run_id) '''
self._client.finishCheckerRun(run_id)
def clean_suppress_data(self, run_id):
"""
clean suppress data
"""
self._client.cleanSuppressData(run_id)
def add_suppress_bug(self, run_id, suppress_data):
"""
process and send suppress data
which should be sent to the report server
"""
bugs_to_suppress = []
for checker_hash, file_name, comment in suppress_data:
comment = comment.encode('UTF-8')
suppress_bug = SuppressBugData(checker_hash,
file_name,
comment)
bugs_to_suppress.append(suppress_bug)
return self._client.addSuppressBug(run_id,
bugs_to_suppress)
def add_skip_paths(self, run_id, paths):
''' bool addSkipPath(1: i64 run_id, 2: map<string, string> paths) '''
# convert before sending through thrift
converted = {}
for path, comment in paths.items():
converted[path] = comment.encode('UTF-8')
return self._client.addSkipPath(run_id, converted)
def replace_config_info(self, run_id, config_list):
''' bool replaceConfigInfo(1: i64 run_id, 2: list<ConfigValue> values) '''
return self._client.replaceConfigInfo(run_id, config_list)
def add_build_action(self, run_id, build_cmd, check_cmd, analyzer_type,
analyzed_source_file):
''' i64 addBuildAction(1: i64 run_id, 2: string build_cmd,
3: string check_cmd, 4: string analyzer_type, 5: string analyzed_source_file)
'''
return self._client.addBuildAction(run_id,
build_cmd,
check_cmd,
analyzer_type,
analyzed_source_file)
def finish_build_action(self, action_id, failure):
''' bool finishBuildAction(1: i64 action_id, 2: string failure) '''
return self._client.finishBuildAction(action_id, failure)
def add_report(self, build_action_id, file_id, bug_hash,
checker_message, bugpath, events, checker_id, checker_cat,
bug_type, severity, suppress):
''' '''
return self._client.addReport(build_action_id, file_id, bug_hash,
checker_message, bugpath,
events, checker_id, checker_cat,
bug_type, severity, suppress)
def need_file_content(self, run_id, filepath):
''' NeedFileResult needFileContent(1: i64 run_id, 2: string filepath)
'''
return self._client.needFileContent(run_id, filepath)
def add_file_content(self, file_id, file_content):
''' bool addFileContent(1: i64 file_id, 2: binary file_content) '''
return self._client.addFileContent(file_id, file_content)
# -----------------------------------------------------------------------------
class ConnectionManager(object):
'''
ContextManager class for handling connections.
Store common information for about connection.
Start and stop the server.
'''
run_id = None
def __init__(self, database_server, host, port):
self.database_server = database_server
self.host = host
self.port = port
ConnectionManager.instance = self
def create_connection(self):
return Connection(self.host, self.port)
def start_report_server(self, db_version_info):
is_server_started = multiprocessing.Event()
server = multiprocessing.Process(target=report_server.run_server,
args=(
self.port,
self.database_server.get_connection_string(),
db_version_info,
is_server_started))
server.daemon = True
server.start()
# Wait a bit
counter = 0
while not is_server_started.is_set() and counter < 4:
LOG.debug('Waiting for checker server to start.')
time.sleep(3)
counter += 1
if counter >= 4 or not server.is_alive():
# last chance to start
if server.exitcode is None:
# it is possible that the database starts really slow
time.sleep(5)
if not is_server_started.is_set():
LOG.error('Failed to start checker server.')
sys.exit(1)
else:
LOG.error('Failed to start checker server.')
LOG.error('Checker server exit code: ' +
str(server.exitcode))
sys.exit(1)
atexit.register(server.terminate)
self.server = server
LOG.debug('Checker server start sequence done.')

tools/codechecker/libcodechecker/cmdline_client/__init__.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.

tools/codechecker/libcodechecker/cmdline_client/cmd_line_client.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import sys
import argparse
import json
import thrift_helper
import shared
import codeCheckerDBAccess
SUPPORTED_VERSION = '4.0'
def check_API_version(client):
''' check if server API is supported by the client'''
version = client.getAPIVersion()
supp_major_version = SUPPORTED_VERSION.split('.')[0]
api_major_version = version.split('.')[0]
if supp_major_version != api_major_version:
return False
return True
class CmdLineOutputEncoder(json.JSONEncoder):
def default(self, obj):
d = {}
d.update(obj.__dict__)
return d
def setupClient(host, port, uri):
''' setup the thrift client and check API version'''
client = thrift_helper.ThriftClientHelper(host, port, uri)
# test if client can work with thrift API getVersion
if not check_API_version(client):
print('Backward incompatible change was in the API.')
print('Please update client. Server version is not supported')
sys.exit(1)
return client
def print_table(lines, separate_head=True):
"""Prints a formatted table given a 2 dimensional array"""
# Count the column width
widths = []
for line in lines:
for i, size in enumerate([len(x) for x in line]):
while i >= len(widths):
widths.append(0)
if size > widths[i]:
widths[i] = size
# Generate the format string to pad the columns
print_string = ""
for i, width in enumerate(widths):
print_string += "{" + str(i) + ":" + str(width) + "} | "
if (len(print_string) == 0):
return
print_string = print_string[:-3]
# Print the actual data
print("-"*(sum(widths)+3*(len(widths)-1)))
for i, line in enumerate(lines):
print(print_string.format(*line))
if (i == 0 and separate_head):
print("-"*(sum(widths)+3*(len(widths)-1)))
print("-"*(sum(widths)+3*(len(widths)-1)))
print ''
def get_run_ids(client):
''' returns a map for run names and run_ids '''
runs = client.getRunData()
run_data = {}
for run in runs:
run_data[run.name] = (run.runId, run.runDate)
return run_data
def check_run_names(client, check_names):
run_info = get_run_ids(client)
if not check_names:
return run_info
missing_name = False
for name in check_names:
if not run_info.get(name):
print('No check name found: ' + name)
missing_name = True
if missing_name:
print('Possible check names are:')
for name, info in run_info.items():
print(name)
sys.exit(1)
return run_info
def handle_list_runs(args):
client = setupClient(args.host, args.port, '/')
runs = client.getRunData()
if args.output_format == 'json':
results = []
for run in runs:
results.append({run.name: run})
print CmdLineOutputEncoder().encode(results)
else:
rows = []
rows.append(('Name', 'ResultCount', 'RunDate'))
for run in runs:
rows.append((run.name, str(run.resultCount), run.runDate))
print_table(rows)
def handle_list_results(args):
client = setupClient(args.host, args.port, '/')
run_info = check_run_names(client, [args.name])
# for name, info in run_info.items()
run_id, run_date = run_info.get(args.name)
limit = 500
offset = 0
filters = []
if args.suppressed:
report_filter = codeCheckerDBAccess.ttypes.ReportFilter(suppressed=True)
else:
report_filter = codeCheckerDBAccess.ttypes.ReportFilter(suppressed=False)
filters.append(report_filter)
results = client.getRunResults(run_id, limit, offset, None, filters)
if args.output_format == 'json':
print CmdLineOutputEncoder().encode(results)
else:
rows = []
if args.suppressed:
rows.append(('File', 'Checker', 'Severity', 'Msg', 'Suppress comment'))
while results:
for res in results:
bug_line = res.lastBugPosition.startLine
checked_file = res.checkedFile+' @ '+str(bug_line)
sev = shared.ttypes.Severity._VALUES_TO_NAMES[res.severity]
rows.append((checked_file, res.checkerId, sev, res.checkerMsg, res.suppressComment))
offset += limit
results = client.getRunResults(run_id, limit, offset, None, filters)
else:
rows.append(('File', 'Checker', 'Severity', 'Msg'))
while results:
for res in results:
bug_line = res.lastBugPosition.startLine
sev = shared.ttypes.Severity._VALUES_TO_NAMES[res.severity]
checked_file = res.checkedFile+' @ '+str(bug_line)
rows.append((checked_file, res.checkerId, sev, res.checkerMsg))
offset += limit
results = client.getRunResults(run_id, limit, offset, None, filters)
print_table(rows)
def handle_list_result_types(args):
client = setupClient(args.host, args.port, '/')
filters = []
if args.suppressed:
report_filter = codeCheckerDBAccess.ttypes.ReportFilter(suppressed=True)
else:
report_filter = codeCheckerDBAccess.ttypes.ReportFilter(suppressed=False)
filters.append(report_filter)
if args.all_results:
run_info = check_run_names(client, None)
results_collector = []
for name, run_info in run_info.items():
run_id, run_date = run_info
results = client.getRunResultTypes(run_id, filters)
if args.output_format == 'json':
results_collector.append({name: results})
else:
print('Check date: '+run_date)
print('Check name: '+name)
rows = []
rows.append(('Checker', 'Severity', 'Count'))
for res in results:
sev = shared.ttypes.Severity._VALUES_TO_NAMES[res.severity]
rows.append((res.checkerId, sev, str(res.count)))
print_table(rows)
if args.output_format == 'json':
print CmdLineOutputEncoder().encode(results_collector)
else:
run_info = check_run_names(client, args.names)
for name in args.names:
run_id, run_date = run_info.get(name)
results = client.getRunResultTypes(run_id, filters)
if args.output_format == 'json':
print CmdLineOutputEncoder().encode(results)
else:
print('Check date: '+run_date)
print('Check name: '+name)
rows = []
rows.append(('Checker', 'Severity', 'Count'))
for res in results:
sev = shared.ttypes.Severity._VALUES_TO_NAMES[res.severity]
rows.append((res.checkerId, sev, str(res.count)))
print_table(rows)
# ------------------------------------------------------------
def handle_remove_run_results(args):
client = setupClient(args.host, args.port, '/')
run_info = check_run_names(client, args.name)
# FIXME LIST comprehension
run_ids_to_delete = []
for name, info in run_info.items():
run_id, run_date = info
if name in args.name:
run_ids_to_delete.append(run_id)
client.removeRunResults(run_ids_to_delete)
print('Done.')
def handle_diff_results(args):
def printResult(getterFn, baseid, newid, suppr, output_format):
report_filter = [codeCheckerDBAccess.ttypes.ReportFilter(suppressed=suppr)]
rows, sort_type, limit, offset = [], None, 500, 0
rows.append(('File', 'Checker', 'Severity', 'Msg'))
results = getterFn(baseid, newid, limit, offset, sort_type, report_filter)
if output_format == 'json':
print CmdLineOutputEncoder().encode(results)
else:
while results:
for res in results:
bug_line = res.lastBugPosition.startLine
sev = shared.ttypes.Severity._VALUES_TO_NAMES[res.severity]
checked_file = res.checkedFile+' @ '+str(bug_line)
rows.append((checked_file, res.checkerId, sev, res.checkerMsg))
offset += limit
results = getterFn(baseid, newid, limit, offset, sort_type, report_filter)
print_table(rows)
client = setupClient(args.host, args.port, '/')
run_info = check_run_names(client, [args.basename, args.newname])
baseid = run_info[args.basename][0]
newid = run_info[args.newname][0]
if args.new:
printResult(client.getNewResults, baseid, newid, args.suppressed, args.output_format)
elif args.unresolved:
printResult(client.getUnresolvedResults, baseid, newid, args.suppressed, args.output_format)
elif args.resolved:
printResult(client.getResolvedResults, baseid, newid, args.suppressed, args.output_format)
def register_client_command_line(argument_parser):
''' should be used to extend the already existing arguments
extend the argument parser with extra commands'''
subparsers = argument_parser.add_subparsers()
# list runs
listruns_parser = subparsers.add_parser('runs', help='Get the run data.')
listruns_parser.add_argument('--host', type=str, dest="host", default='localhost',
help='Server host.')
listruns_parser.add_argument('-p', '--port', type=str, dest="port", default=11444,
required=True, help='Server port.')
listruns_parser.add_argument('-o', choices=['plaintext', 'json'], default='plaintext', type=str, dest="output_format", help='Output format.')
listruns_parser.set_defaults(func=handle_list_runs)
# list results
listresults_parser = subparsers.add_parser('results', help='List results.')
listresults_parser.add_argument('--host', type=str, dest="host", default='localhost',
help='Server host.')
listresults_parser.add_argument('-p', '--port', type=str, dest="port", default=11444,
required=True, help='Server port.')
listresults_parser.add_argument('-n', '--name', type=str, dest="name", required=True,
help='Check name.')
listresults_parser.add_argument('-s', '--suppressed', action="store_true", dest="suppressed", help='Suppressed results.')
listresults_parser.add_argument('-o', choices=['plaintext', 'json'], default='plaintext', type=str, dest="output_format", help='Output format.')
listresults_parser.set_defaults(func=handle_list_results)
# list diffs
diff_parser = subparsers.add_parser('diff', help='Diff two run.')
diff_parser.add_argument('--host', type=str, dest="host", default='localhost',
help='Server host.')
diff_parser.add_argument('-p', '--port', type=str, dest="port", default=11444,
required=True, help='Server port.')
diff_parser.add_argument('-b', '--basename', type=str, dest="basename", required=True,
help='Base name.')
diff_parser.add_argument('-n', '--newname', type=str, dest="newname", required=True,
help='New name.')
diff_parser.add_argument('-s', '--suppressed', action="store_true", dest="suppressed", default=False,
required=False, help='Show suppressed bugs.')
diff_parser.add_argument('-o', choices=['plaintext', 'json'], default='plaintext', type=str, dest="output_format", help='Output format.')
group = diff_parser.add_mutually_exclusive_group(required=True)
group.add_argument('--new', action="store_true", dest="new", help="Show new results.")
group.add_argument('--unresolved', action="store_true", dest="unresolved", help="Show unresolved results.")
group.add_argument('--resolved', action="store_true", dest="resolved", help="Show resolved results.")
diff_parser.set_defaults(func=handle_diff_results)
# list resulttypes
sum_parser = subparsers.add_parser('sum', help='Sum results.')
sum_parser.add_argument('--host', type=str, dest="host", default='localhost',
help='Server host.')
sum_parser.add_argument('-p', '--port', type=str, dest="port", default=11444,
required=True, help='Server port.')
name_group = sum_parser.add_mutually_exclusive_group(required=True)
name_group.add_argument('-n', '--name', nargs='+', type=str, dest="names", help='Check name.')
name_group.add_argument('-a', '--all', action='store_true', dest="all_results", help='All results.')
sum_parser.add_argument('-s', '--suppressed', action="store_true", dest="suppressed", help='Suppressed results.')
sum_parser.add_argument('-o', choices=['plaintext', 'json'], default='plaintext', type=str, dest="output_format", help='Output format.')
sum_parser.set_defaults(func=handle_list_result_types)
# list resulttypes
sum_parser = subparsers.add_parser('del', help='Remove run results.')
sum_parser.add_argument('--host', type=str, dest="host", default='localhost',
help='Server host.')
sum_parser.add_argument('-p', '--port', type=str, dest="port", default=11444,
required=True, help='Server port.')
sum_parser.add_argument('-n', '--name', nargs='+', type=str, dest="name", required=True, help='Server port.')
sum_parser.set_defaults(func=handle_remove_run_results)
def main():
parser = argparse.ArgumentParser(description='Simple command line client for codechecker.')
register_client_command_line(parser)
args = parser.parse_args()
args.func(args)
if __name__ == "__main__":
main()

tools/codechecker/libcodechecker/cmdline_client/thrift_helper.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import os
import sys
#import datetime
import socket
from thrift.transport import THttpClient
from thrift.protocol import TJSONProtocol
from codeCheckerDBAccess import codeCheckerDBAccess
import shared
class ThriftClientHelper():
def __init__(self, host, port, uri):
self.transport = THttpClient.THttpClient(host, port, uri)
self.protocol = TJSONProtocol.TJSONProtocol(self.transport)
self.client = codeCheckerDBAccess.Client(self.protocol)
# ------------------------------------------------------------
def ThriftClientCall(function):
#print type(function)
funcName = function.__name__
def wrapper(self, *args, **kwargs):
#print('['+host+':'+str(port)+'] >>>>> ['+funcName+']')
#before = datetime.datetime.now()
self.transport.open()
func = getattr(self.client,funcName)
try:
res = func(*args, **kwargs)
except shared.ttypes.RequestFailed as reqfailure:
if reqfailure.error_code == shared.ttypes.ErrorCode.DATABASE:
print(str(reqfailure.message))
else:
print('Other error')
print(str(reqfailure))
sys.exit(1)
except socket.error as serr:
errCause = os.strerror(serr.errno)
print(errCause)
print(str(serr))
sys.exit(1)
#after = datetime.datetime.now()
#timediff = after - before
#diff = timediff.microseconds/1000
#print('['+str(diff)+'ms] <<<<< ['+host+':'+str(port)+']')
#print res
self.transport.close()
return res
return wrapper
# ------------------------------------------------------------
@ThriftClientCall
def getRunData():
pass
# ------------------------------------------------------------
@ThriftClientCall
def getRunResults(self, runId, resultBegin, resultEnd, sortType, reportFilters):
pass
# ------------------------------------------------------------
@ThriftClientCall
def getRunResultCount(self, runId, reportFilters):
pass
#-----------------------------------------------------------------------
@ThriftClientCall
def getRunResultTypes(self, runId, reportFilters):
pass
#-----------------------------------------------------------------------
@ThriftClientCall
def getAPIVersion(self):
pass
#-----------------------------------------------------------------------
@ThriftClientCall
def removeRunResults(self, run_ids):
pass
#-----------------------------------------------------------------------
@ThriftClientCall
def getNewResults(self, base_run_id, new_run_id, limit, offset, sortType, reportFilters):
pass
#-----------------------------------------------------------------------
@ThriftClientCall
def getUnresolvedResults(self, base_run_id, new_run_id, limit, offset, sortType, reportFilters):
pass
#-----------------------------------------------------------------------
@ThriftClientCall
def getResolvedResults(self, base_run_id, new_run_id, limit, offset, sortType, reportFilters):
pass

tools/codechecker/libcodechecker/context_base.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import os
import abc
# -----------------------------------------------------------------------------
class ContextBase(object):
__metaclass__ = abc.ABCMeta
_package_root = None
_verbose_level = None
_alchemy_log_level = None
_env_path = None
_env_vars = None
_extra_include_paths = []
_compiler_sysroot = None
_extra_system_include_paths = []
_codechecker_enable_checkers = set()
_codechecker_disable_checkers = set()
_codechecker_workspace = None
_db_username = None
_module_id = ''
_run_id = None
_severity_map = dict()
def load_data(self, env_vars, pckg_layout, variables):
self.pckg_layout = pckg_layout
self.variables = variables
self.env_vars = env_vars
self.set_env(env_vars)
self._db_username = self.variables['default_db_username']
@abc.abstractmethod
def set_env(self, env_vars):
self._package_root = os.environ.get(env_vars['env_package_root'])
self._verbose_level = os.environ.get(env_vars['env_verbose_name'])
self._alchemy_log_level = \
os.environ.get(env_vars['env_alchemy_verbose_name'])
self._env_path = os.environ.get(env_vars['env_path'])
env_enabled_checkers = os.environ.get(
env_vars['codechecker_enable_check'])
if env_enabled_checkers:
self._codechecker_enable_checkers = \
set(env_enabled_checkers.split(':'))
env_disabled_checkers = os.environ.get(
env_vars['codechecker_disable_check'])
if env_disabled_checkers:
self._codechecker_disable_checkers = \
set(env_disabled_checkers.split(':'))
codechecker_workspace = os.environ.get('codechecker_workspace')
if codechecker_workspace:
self._codechecker_workspace = codechecker_workspace
@property
def package_root(self):
return self._package_root
@property
def verbose_level(self):
return self._verbose_level
@property
def checker_plugin(self):
return os.path.join(self._package_root,
self.pckg_layout['plugin'])
@property
def clang_include(self):
return os.path.join(self._package_root,
self.pckg_layout['compiler_include'])
@property
def extra_includes(self):
return self._extra_include_paths
@extra_includes.setter
def add_extra_includes(self, path):
self._extra_include_paths.append(path)
@property
def extra_system_includes(self):
return self._extra_system_include_paths
@extra_includes.setter
def add_extra_system_includes(self, path):
self._extra_system_include_paths.append(path)
@property
def gdb_config_file(self):
return os.path.join(self._package_root,
self.pckg_layout['gdb_config_file'])
@property
def checkers_severity_map_file(self):
return os.path.join(self._package_root,
self.pckg_layout['checkers_severity_map_file'])
@property
def doc_root(self):
return os.path.join(self._package_root,
self.pckg_layout['docs'])
@property
def www_root(self):
return os.path.join(self._package_root,
self.pckg_layout['www'])
@property
def migration_root(self):
return os.path.join(self._package_root,
self.pckg_layout['codechecker_db_migrate'])
@property
def db_username(self):
return self._db_username
@db_username.setter
def db_username(self, value):
self._db_username = value
@property
def pgsql_data_dir_name(self):
return self.variables['pgsql_data_dir_name']
@property
def env_enabled_checkers(self):
return self._codechecker_enable_checkers
@env_enabled_checkers.setter
def env_enabled_checkers(self, value):
self._codechecker_enable_checkers = \
self._codechecker_enable_checkers.union(value)
@property
def env_disabled_checkers(self):
return self._codechecker_disable_checkers
@env_disabled_checkers.setter
def env_disabled_checkers(self, value):
self._codechecker_disable_checkers = \
self._codechecker_disable_checkers.union(value)
@property
def codechecker_workspace(self):
return self._codechecker_workspace
@codechecker_workspace.setter
def codechecker_workspace(self, value):
self._codechecker_workspace = value
@property
def database_path(self):
return os.path.join(self.codechecker_workspace,
self.pgsql_data_dir_name)
@property
def compiler_sysroot(self):
return self._compiler_sysroot
@compiler_sysroot.setter
def compiler_sysroot(self, value):
self._compiler_sysroot = value
@property
def module_id(self):
return self._module_id
@module_id.setter
def module_id(self, value):
self._module_id = value
@property
def run_id(self):
return self._run_id
@run_id.setter
def run_id(self, value):
self._run_id = value
@property
def severity_map(self):
return self._severity_map
@severity_map.setter
def severity_map(self, value):
self._severity_map = value

tools/codechecker/libcodechecker/database_handler.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
Database server handling.
'''
import os
import subprocess
import atexit
import time
import sys
from abc import ABCMeta, abstractmethod
from libcodechecker import util
from libcodechecker import logger
from libcodechecker import pgpass
from libcodechecker import host_check
from libcodechecker.db_model.orm_model import CC_META
from libcodechecker.db_model.orm_model import DBVersion
from libcodechecker.db_model.orm_model import CreateSession
import sqlalchemy
from sqlalchemy.engine.url import URL, make_url
from sqlalchemy.sql.elements import quoted_name
from sqlalchemy.engine import Engine
from sqlalchemy import event
from alembic import command, config
LOG = logger.get_new_logger('DB_HANDLER')
class SQLServer(object):
'''
Abstract base class for database server handling. An SQLServer instance is
responsible for the initialization, starting, and stopping the database
server, and also for connection string management.
SQLServer implementations are created via SQLServer.from_cmdline_args().
How to add a new database server implementation:
1, Derive from SQLServer and implement the abstract methods
2, Add/modify some command line options in CodeChecker.py
3, Modify SQLServer.from_cmdline_args() in order to create an
instance of the new server type if needed
'''
__metaclass__ = ABCMeta
def __init__(self, migration_root):
'''
Sets self.migration_root. migration_root should be the path to the
alembic migration scripts.
'''
self.migration_root = migration_root
def _create_or_update_schema(self, use_migration=True):
'''
Creates or updates the database schema. The database server should be
started before this method is called.
If use_migration is True, this method runs an alembic upgrade to HEAD.
In the False case, there is no migration support and only SQLAlchemy
meta data is used for schema creation.
On error sys.exit(1) is called.
'''
try:
db_uri = self.get_connection_string()
engine = SQLServer.create_engine(db_uri)
LOG.debug('Update/create database schema')
if use_migration:
LOG.debug('Creating new database session')
session = CreateSession(engine)
connection = session.connection()
cfg = config.Config()
cfg.set_main_option("script_location", self.migration_root)
cfg.attributes["connection"] = connection
command.upgrade(cfg, "head")
session.commit()
else:
CC_META.create_all(engine)
engine.dispose()
LOG.debug('Update/create database schema done')
return True
except sqlalchemy.exc.SQLAlchemyError as alch_err:
LOG.error(str(alch_err))
sys.exit(1)
@abstractmethod
def start(self, db_version_info, wait_for_start=True, init=False):
'''
Starts the database server and initializes the database server.
On wait_for_start == True, this method returns when the server is up
and ready for connections. Otherwise it only starts the server and
returns immediately.
On init == True, this it also initializes the database data and schema
if needed.
On error sys.exit(1) should be called.
'''
pass
@abstractmethod
def stop(self):
'''
Terminates the database server.
On error sys.exit(1) should be called.
'''
pass
@abstractmethod
def get_connection_string(self):
'''
Returns the connection string for SQLAlchemy.
DO NOT LOG THE CONNECTION STRING BECAUSE IT MAY CONTAIN THE PASSWORD
FOR THE DATABASE!
'''
pass
@staticmethod
def create_engine(connection_string):
'''
Creates a new SQLAlchemy engine.
'''
if make_url(connection_string).drivername == 'sqlite+pysqlite':
# FIXME: workaround for locking errors
return sqlalchemy.create_engine(connection_string,
encoding='utf8',
connect_args={'timeout': 600})
else:
return sqlalchemy.create_engine(connection_string,
encoding='utf8')
@staticmethod
def from_cmdline_args(args, workspace, migration_root, env=None):
'''
Normally only this method is called form outside of this module in
order to instance the proper server implementation.
Parameters:
args: the command line arguments from CodeChecker.py
workspace: path to the CodeChecker workspace directory
migration_root: path to the database migration scripts
env: a run environment dictionary.
'''
if not host_check.check_sql_driver(args.postgresql):
LOG.error("The selected SQL driver is not available.")
sys.exit(1)
if args.postgresql:
LOG.debug("Using PostgreSQLServer")
return PostgreSQLServer(workspace,
migration_root,
args.dbaddress,
args.dbport,
args.dbusername,
args.dbname,
run_env=env)
else:
LOG.debug("Using SQLiteDatabase")
return SQLiteDatabase(workspace, migration_root, run_env=env)
def check_db_version(self, db_version_info, session=None):
'''
Checks the database version and prints an error message on database
version mismatch.
- On mismatching or on missing version a sys.exit(1) is called.
- On missing DBVersion table, it returns False
- On compatible DB version, it returns True
Parameters:
db_version_info (db_version.DBVersionInfo): required database
version.
session: an open database session or None. If session is None, a
new session is created.
'''
try:
dispose_engine = False
if session is None:
engine = SQLServer.create_engine(self.get_connection_string())
dispose_engine = True
session = CreateSession(engine)
else:
engine = session.get_bind()
if not engine.has_table(quoted_name(DBVersion.__tablename__, True)):
LOG.debug("Missing DBVersion table!")
return False
version = session.query(DBVersion).first()
if version is None:
# Version is not populated yet
LOG.error('No version information found in the database.')
sys.exit(1)
elif not db_version_info.is_compatible(version.major, version.minor):
LOG.error('Version mismatch. Expected database version: ' +
str(db_version_info))
version_from_db = 'v'+str(version.major)+'.'+str(version.minor)
LOG.error('Version from the database is: ' + version_from_db)
LOG.error('Please update your database.')
sys.exit(1)
LOG.debug("Database version is compatible.")
return True
finally:
session.commit()
if dispose_engine:
engine.dispose()
def _add_version(self, db_version_info, session=None):
'''
Fills the DBVersion table.
'''
engine = None
if session is None:
engine = SQLServer.create_engine(self.get_connection_string())
session = CreateSession(engine)
expected = db_version_info.get_expected_version()
LOG.debug('Adding DB version: ' + str(expected))
session.add(DBVersion(expected[0], expected[1]))
session.commit()
if engine:
engine.dispose()
LOG.debug('Adding DB version done!')
class PostgreSQLServer(SQLServer):
'''
Handler for PostgreSQL.
'''
def __init__(self, workspace, migration_root, host, port, user, database, password = None, run_env=None):
super(PostgreSQLServer, self).__init__(migration_root)
self.path = os.path.join(workspace, 'pgsql_data')
self.host = host
self.port = port
self.user = user
self.database = database
self.password = password
self.run_env = run_env
self.workspace = workspace
self.proc = None
def _is_database_data_exist(self):
'''Check the PostgreSQL instance existence in a given path.'''
LOG.debug('Checking for database at ' + self.path)
return os.path.exists(self.path) and \
os.path.exists(os.path.join(self.path, 'PG_VERSION')) and \
os.path.exists(os.path.join(self.path, 'base'))
def _initialize_database_data(self):
'''Initialize a PostgreSQL instance with initdb. '''
LOG.debug('Initializing database at ' + self.path)
init_db = ['initdb', '-U', self.user, '-D', self.path, '-E SQL_ASCII']
err, code = util.call_command(init_db, self.run_env)
# logger -> print error
return code == 0
def _get_connection_string(self, database):
'''
Helper method for getting the connection string for the given database.
'''
port = str(self.port)
driver = host_check.get_postgresql_driver_name()
password = self.password
if driver == 'pg8000' and not password:
pfilepath = os.environ.get('PGPASSFILE')
if pfilepath:
password = pgpass.get_password_from_file(pfilepath,
self.host,
port,
database,
self.user)
extra_args = {'client_encoding': 'utf8'}
return str(URL('postgresql+' + driver,
username=self.user,
password=password,
host=self.host,
port=port,
database=database,
query=extra_args))
def _wait_or_die(self):
'''
Wait for database if the database process was stared
with a different client. No polling is possible.
'''
LOG.debug('Waiting for PostgreSQL')
tries_count = 0
max_try = 20
timeout = 5
while not self._is_running() and tries_count < max_try:
tries_count += 1
time.sleep(timeout)
if tries_count >= max_try:
LOG.error('Failed to start database.')
sys.exit(1)
def _create_database(self):
try:
LOG.debug('Creating new database if not exists')
db_uri = self._get_connection_string('postgres')
engine = SQLServer.create_engine(db_uri)
text = "SELECT 1 FROM pg_database WHERE datname='%s'" % self.database
if not bool(engine.execute(text).scalar()):
conn = engine.connect()
# From sqlalchemy documentation:
# The psycopg2 and pg8000 dialects also offer the special level AUTOCOMMIT.
conn = conn.execution_options(isolation_level="AUTOCOMMIT")
conn.execute('CREATE DATABASE "%s"' % self.database)
conn.close()
LOG.debug('Database created: ' + self.database)
LOG.debug('Database already exists: ' + self.database)
except sqlalchemy.exc.SQLAlchemyError as alch_err:
LOG.error('Failed to create database!')
LOG.error(str(alch_err))
sys.exit(1)
def _is_running(self):
'''Is there PostgreSQL instance running on a given host and port.'''
LOG.debug('Checking if database is running at ' + self.host + ':' + str(self.port))
check_db = ['psql', '-U', self.user, '-l', '-p', str(self.port), '-h', self.host]
err, code = util.call_command(check_db, self.run_env)
return code == 0
def start(self, db_version_info, wait_for_start=True, init=False):
'''
Start a PostgreSQL instance with given path, host and port.
Return with process instance
'''
LOG.debug('Starting/connecting to database')
if not self._is_running():
if not util.is_localhost(self.host):
LOG.info('Database is not running yet')
sys.exit(1)
if not self._is_database_data_exist():
if not init:
# The database does not exists
LOG.error('Database data is missing!')
LOG.error('Please check your configuration!')
sys.exit(1)
elif not self._initialize_database_data():
# The database does not exist and cannot create
LOG.error('Database data is missing and the initialization '
'of a new failed!')
LOG.error('Please check your configuration!')
sys.exit(1)
LOG.info('Starting database')
LOG.debug('Starting database at ' + self.host + ':' + str(self.port) + ' ' + self.path)
db_logfile = os.path.join(self.workspace, 'postgresql.log') \
if logger.get_log_level() == logger.DEBUG else os.devnull
self._db_log = open(db_logfile, 'wb')
start_db = ['postgres', '-i', '-D', self.path, '-p', str(self.port), '-h', self.host]
self.proc = subprocess.Popen(start_db,
bufsize=-1,
env=self.run_env,
stdout=self._db_log,
stderr=subprocess.STDOUT)
add_version = False
if init:
self._wait_or_die()
self._create_database()
add_version = not self.check_db_version(db_version_info)
self._create_or_update_schema()
elif wait_for_start:
self._wait_or_die()
add_version = not self.check_db_version(db_version_info)
if add_version:
self._add_version(db_version_info)
atexit.register(self.stop)
LOG.debug('Done')
def stop(self):
if self.proc:
LOG.debug('Terminating database')
self.proc.terminate()
self._db_log.close()
def get_connection_string(self):
return self._get_connection_string(self.database)
class SQLiteDatabase(SQLServer):
'''
Handler for SQLite.
'''
def __init__(self, workspace, migration_root, run_env=None):
super(SQLiteDatabase, self).__init__(migration_root)
self.dbpath = os.path.join(workspace, 'codechecker.sqlite')
self.run_env = run_env
def _set_sqlite_pragma(dbapi_connection, connection_record):
cursor = dbapi_connection.cursor()
cursor.execute("PRAGMA foreign_keys=ON")
cursor.close()
event.listen(Engine, 'connect', _set_sqlite_pragma)
def start(self, db_version_info, wait_for_start=True, init=False):
if init:
add_version = not self.check_db_version(db_version_info)
self._create_or_update_schema(use_migration=False)
if add_version:
self._add_version(db_version_info)
if not os.path.exists(self.dbpath):
# The database does not exists
LOG.error('Database (%s) is missing!' % self.dbpath)
sys.exit(1)
def stop(self):
pass
def get_connection_string(self):
return str(URL('sqlite+pysqlite', None, None, None, None, self.dbpath))

tools/codechecker/libcodechecker/db_migrate/env.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
from __future__ import with_statement
from alembic import context
from sqlalchemy import engine_from_config, pool
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
# add your model's MetaData object here
# for 'autogenerate' support
try:
from libcodechecker.db_model.orm_model import Base
except ImportError:
# Assume we are in the source directory
import sys
import os
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__),
"..")))
from db_model.orm_model import Base
target_metadata = Base.metadata
# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.
def run_migrations_offline():
"""Run migrations in 'offline' mode.
This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.
Calls to context.execute() here emit the given string to the
script output.
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url, target_metadata=target_metadata, literal_binds=True)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online():
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
connectable = config.attributes.get('connection', None)
if connectable is None:
connectable = engine_from_config(
config.get_section(config.config_ini_section),
prefix='sqlalchemy.',
poolclass=pool.NullPool)
with connectable.connect() as connection:
context.configure(
connection=connection,
target_metadata=target_metadata
)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()

tools/codechecker/libcodechecker/db_migrate/script.py.mako

  • This file was added.
"""${message}
Revision ID: ${up_revision}
Revises: ${down_revision | comma,n}
Create Date: ${create_date}
"""
# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}
branch_labels = ${repr(branch_labels)}
depends_on = ${repr(depends_on)}
from alembic import op
import sqlalchemy as sa
${imports if imports else ""}
def upgrade():
${upgrades if upgrades else "pass"}
def downgrade():
${downgrades if downgrades else "pass"}

tools/codechecker/libcodechecker/db_migrate/versions/2b23d1a4fb96_udpate_suppress_handling.py

  • This file was added.
"""udpate suppress handling
Revision ID: 2b23d1a4fb96
Revises: 63efc03c2a5
Create Date: 2015-10-06 12:40:06.669407
"""
# revision identifiers, used by Alembic.
revision = '2b23d1a4fb96'
down_revision = '63efc03c2a5'
branch_labels = None
depends_on = None
import sys
from alembic import op
import sqlalchemy as sa
from sqlalchemy.schema import Sequence, CreateSequence
# NOTICE!
# kept here only for later reference if required for other
# db upgrades if full table copy or data modification is required
#
# helper table
#suppress_bug_helper = sa.Table(
# 'suppress_bug',
# sa.MetaData(),
# sa.Column('file_name', sa.String()),
# sa.Column('hash', sa.String()),
#)
# NOTICE END
def upgrade():
op.drop_constraint('pk_suppress_bug', 'suppress_bug', type_='primary')
op.add_column('suppress_bug',
sa.Column('file_name', sa.String(), nullable=True))
# add new id primary key
op.add_column('suppress_bug',
sa.Column('id', sa.Integer(), autoincrement=True, primary_key=True))
# create constraint
op.create_primary_key(
"pk_suppress_bug", "suppress_bug",
["id"]
)
# required to fill up autoincrement values for the id
op.execute(CreateSequence(Sequence('suppress_bug_id')))
op.alter_column("suppress_bug",
"id",
nullable=False,
server_default=sa.text("nextval('suppress_bug_id'::regclass)"))
# NOTICE!
# kept here only for later reference if required for other
# db upgrades if full table copy or data modification is required
# copy the full table and during the copy modify values if required
# use the current connection
#connection = op.get_bind()
#for sp_bug in connection.execute(suppress_bug_helper.select()):
# connection.execute(
# suppress_bug_helper.update().where(
# suppress_bug_helper.c.hash == sp_bug.hash
# ).values(
# file_name='',
# )
# )
# NOTICE END
def downgrade():
# downgrade is not supported
sys.exit(1)

tools/codechecker/libcodechecker/db_migrate/versions/30e41fdf2e85_store_analyzer_type_and_analyzed_source_.py

  • This file was added.
"""Store analyzer type and analyzed source file to the database for each buildaction
Revision ID: 30e41fdf2e85
Revises: 4e97419519b3
Create Date: 2016-07-04 15:36:26.208047
"""
# revision identifiers, used by Alembic.
revision = '30e41fdf2e85'
down_revision = '4e97419519b3'
branch_labels = None
depends_on = None
import sys
from alembic import op
import sqlalchemy as sa
def upgrade():
'''
extend build_actions table with columns to identify if
the results for a build_action should be deleted in update mode
analyzer_type: is required to identify the analyzer which analyzer analyzed the build action
analyzed_source_file: is required to identify which source file was analyzed in the build action (it is possible to contain multiple source files)
'''
op.add_column('build_actions',
sa.Column('analyzed_source_file',
sa.String(),
nullable=False,
server_default='')
)
op.add_column('build_actions',
sa.Column('analyzer_type',
sa.String(),
nullable=False,
server_default='')
)
def downgrade():
# downgrade is not supported
sys.exit(1)

tools/codechecker/libcodechecker/db_migrate/versions/453083da7cce_quick_delete.py

  • This file was added.
"""quick delete
Revision ID: 453083da7cce
Revises: f6791c2b40d
Create Date: 2015-11-26 15:11:36.790627
"""
# revision identifiers, used by Alembic.
revision = '453083da7cce'
down_revision = 'f6791c2b40d'
branch_labels = None
depends_on = None
from alembic import op
import sys
def upgrade():
op.create_index(op.f('ix_reports_end_bugevent'), 'reports', ['end_bugevent'], unique=False)
op.create_index(op.f('ix_reports_start_bugevent'), 'reports', ['start_bugevent'], unique=False)
def downgrade():
sys.exit(-1)

tools/codechecker/libcodechecker/db_migrate/versions/464168cc48ad_remove_bug_hash_type.py

  • This file was added.
"""remove_bug_hash_type
Revision ID: 464168cc48ad
Revises: 2b23d1a4fb96
Create Date: 2015-11-04 17:45:32.136366
"""
# revision identifiers, used by Alembic.
revision = '464168cc48ad'
down_revision = '2b23d1a4fb96'
branch_labels = None
depends_on = None
import sys
from alembic import op
def upgrade():
"""
remove bug hash type
"""
op.drop_column('reports', 'bug_id_type')
op.drop_column('suppress_bug', 'type')
def downgrade():
"""
downgrade is not supported
"""
sys.exit(1)

tools/codechecker/libcodechecker/db_migrate/versions/4e97419519b3_remove_run_id_primary_key_from_skippath_.py

  • This file was added.
"""Remove run_id primary key from SkipPath table
Revision ID: 4e97419519b3
Revises: 453083da7cce
Create Date: 2016-02-12 13:34:23.302931
"""
# revision identifiers, used by Alembic.
revision = '4e97419519b3'
down_revision = '453083da7cce'
branch_labels = None
depends_on = None
import sys
from alembic import op
import sqlalchemy as sa
def upgrade():
"""
remove primary key constraint for run_id in the skip path table
"""
op.drop_constraint('pk_skip_path', 'skip_path', type_='primary')
# create new primary key constraint for the id only
op.create_primary_key(
"pk_skip_path", "skip_path",
["id"]
)
op.alter_column('skip_path',
sa.Column('run_id', sa.Integer(), nullable=False))
def downgrade():
# downgrade is not supported
sys.exit(1)

tools/codechecker/libcodechecker/db_migrate/versions/63efc03c2a5_initial_schema.py

  • This file was added.
"""Initial schema
Revision ID: 63efc03c2a5
Revises:
Create Date: 2015-09-25 14:31:57.381866
"""
# revision identifiers, used by Alembic.
revision = '63efc03c2a5'
down_revision = None
branch_labels = None
depends_on = None
from alembic import op
import sqlalchemy as sa
def upgrade():
op.create_table('db_version',
sa.Column('major', sa.Integer(), nullable=False),
sa.Column('minor', sa.Integer(), nullable=False),
sa.PrimaryKeyConstraint('major', 'minor', name=op.f('pk_db_version'))
)
op.create_table('runs',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('date', sa.DateTime(), nullable=True),
sa.Column('duration', sa.Integer(), nullable=True),
sa.Column('name', sa.String(), nullable=True),
sa.Column('version', sa.String(), nullable=True),
sa.Column('command', sa.String(), nullable=True),
sa.Column('inc_count', sa.Integer(), nullable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_runs')),
sa.UniqueConstraint('name', name=op.f('uq_runs_name'))
)
op.create_table('build_actions',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('run_id', sa.Integer(), nullable=True),
sa.Column('build_cmd', sa.String(), nullable=True),
sa.Column('check_cmd', sa.String(), nullable=True),
sa.Column('failure_txt', sa.String(), nullable=True),
sa.Column('date', sa.DateTime(), nullable=True),
sa.Column('duration', sa.Integer(), nullable=True),
sa.ForeignKeyConstraint(['run_id'], [u'runs.id'], name=op.f('fk_build_actions_run_id_runs'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_build_actions'))
)
op.create_table('configs',
sa.Column('run_id', sa.Integer(), nullable=False),
sa.Column('checker_name', sa.String(), nullable=False),
sa.Column('attribute', sa.String(), nullable=False),
sa.Column('value', sa.String(), nullable=False),
sa.ForeignKeyConstraint(['run_id'], [u'runs.id'], name=op.f('fk_configs_run_id_runs'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.PrimaryKeyConstraint('run_id', 'checker_name', 'attribute', 'value', name=op.f('pk_configs'))
)
op.create_table('files',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('run_id', sa.Integer(), nullable=True),
sa.Column('filepath', sa.String(), nullable=True),
sa.Column('content', sa.Binary(), nullable=True),
sa.Column('inc_count', sa.Integer(), nullable=True),
sa.ForeignKeyConstraint(['run_id'], [u'runs.id'], name=op.f('fk_files_run_id_runs'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_files'))
)
op.create_table('skip_path',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('path', sa.String(), nullable=True),
sa.Column('run_id', sa.Integer(), nullable=False),
sa.Column('comment', sa.Binary(), nullable=True),
sa.ForeignKeyConstraint(['run_id'], [u'runs.id'], name=op.f('fk_skip_path_run_id_runs'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.PrimaryKeyConstraint('id', 'run_id', name=op.f('pk_skip_path'))
)
op.create_table('suppress_bug',
sa.Column('hash', sa.String(), nullable=False),
sa.Column('run_id', sa.Integer(), nullable=False),
sa.Column('type', sa.Integer(), nullable=True),
sa.Column('comment', sa.Binary(), nullable=True),
sa.ForeignKeyConstraint(['run_id'], [u'runs.id'], name=op.f('fk_suppress_bug_run_id_runs'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.PrimaryKeyConstraint('hash', 'run_id', name=op.f('pk_suppress_bug'))
)
op.create_table('bug_path_events',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('line_begin', sa.Integer(), nullable=True),
sa.Column('col_begin', sa.Integer(), nullable=True),
sa.Column('line_end', sa.Integer(), nullable=True),
sa.Column('col_end', sa.Integer(), nullable=True),
sa.Column('msg', sa.String(), nullable=True),
sa.Column('file_id', sa.Integer(), nullable=True),
sa.Column('next', sa.Integer(), nullable=True),
sa.Column('prev', sa.Integer(), nullable=True),
sa.ForeignKeyConstraint(['file_id'], [u'files.id'], name=op.f('fk_bug_path_events_file_id_files'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_bug_path_events'))
)
op.create_index(op.f('ix_bug_path_events_file_id'), 'bug_path_events', ['file_id'], unique=False)
op.create_table('bug_report_points',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('line_begin', sa.Integer(), nullable=True),
sa.Column('col_begin', sa.Integer(), nullable=True),
sa.Column('line_end', sa.Integer(), nullable=True),
sa.Column('col_end', sa.Integer(), nullable=True),
sa.Column('file_id', sa.Integer(), nullable=True),
sa.Column('next', sa.Integer(), nullable=True),
sa.ForeignKeyConstraint(['file_id'], [u'files.id'], name=op.f('fk_bug_report_points_file_id_files'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_bug_report_points'))
)
op.create_index(op.f('ix_bug_report_points_file_id'), 'bug_report_points', ['file_id'], unique=False)
op.create_table('reports',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('file_id', sa.Integer(), nullable=True),
sa.Column('run_id', sa.Integer(), nullable=True),
sa.Column('bug_id', sa.String(), nullable=True),
sa.Column('bug_id_type', sa.Integer(), nullable=True),
sa.Column('checker_id', sa.String(), nullable=True),
sa.Column('checker_cat', sa.String(), nullable=True),
sa.Column('bug_type', sa.String(), nullable=True),
sa.Column('severity', sa.Integer(), nullable=True),
sa.Column('checker_message', sa.String(), nullable=True),
sa.Column('start_bugpoint', sa.Integer(), nullable=True),
sa.Column('start_bugevent', sa.Integer(), nullable=True),
sa.Column('end_bugevent', sa.Integer(), nullable=True),
sa.Column('suppressed', sa.Boolean(), nullable=True),
sa.ForeignKeyConstraint(['end_bugevent'], [u'bug_path_events.id'], name=op.f('fk_reports_end_bugevent_bug_path_events'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.ForeignKeyConstraint(['file_id'], [u'files.id'], name=op.f('fk_reports_file_id_files'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.ForeignKeyConstraint(['run_id'], [u'runs.id'], name=op.f('fk_reports_run_id_runs'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.ForeignKeyConstraint(['start_bugevent'], [u'bug_path_events.id'], name=op.f('fk_reports_start_bugevent_bug_path_events'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.ForeignKeyConstraint(['start_bugpoint'], [u'bug_report_points.id'], name=op.f('fk_reports_start_bugpoint_bug_report_points'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.PrimaryKeyConstraint('id', name=op.f('pk_reports'))
)
op.create_index(op.f('ix_reports_bug_id'), 'reports', ['bug_id'], unique=False)
op.create_index(op.f('ix_reports_run_id'), 'reports', ['run_id'], unique=False)
op.create_table('reports_to_build_actions',
sa.Column('report_id', sa.Integer(), nullable=False),
sa.Column('build_action_id', sa.Integer(), nullable=False),
sa.ForeignKeyConstraint(['build_action_id'], [u'build_actions.id'], name=op.f('fk_reports_to_build_actions_build_action_id_build_actions'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.ForeignKeyConstraint(['report_id'], [u'reports.id'], name=op.f('fk_reports_to_build_actions_report_id_reports'), ondelete=u'CASCADE', initially=u'DEFERRED', deferrable=True),
sa.PrimaryKeyConstraint('report_id', 'build_action_id', name=op.f('pk_reports_to_build_actions'))
)
def downgrade():
op.drop_table('reports_to_build_actions')
op.drop_index(op.f('ix_reports_run_id'), table_name='reports')
op.drop_index(op.f('ix_reports_bug_id'), table_name='reports')
op.drop_table('reports')
op.drop_index(op.f('ix_bug_report_points_file_id'), table_name='bug_report_points')
op.drop_table('bug_report_points')
op.drop_index(op.f('ix_bug_path_events_file_id'), table_name='bug_path_events')
op.drop_table('bug_path_events')
op.drop_table('suppress_bug')
op.drop_table('skip_path')
op.drop_table('files')
op.drop_table('configs')
op.drop_table('build_actions')
op.drop_table('runs')
op.drop_table('db_version')

tools/codechecker/libcodechecker/db_migrate/versions/f6791c2b40d_add_can_delete_to_run_table.py

  • This file was added.
"""Add can_delete to Run table
Revision ID: f6791c2b40d
Revises: 464168cc48ad
Create Date: 2015-11-17 16:22:02.793689
"""
# revision identifiers, used by Alembic.
revision = 'f6791c2b40d'
down_revision = '464168cc48ad'
branch_labels = None
depends_on = None
import sys
from alembic import op
import sqlalchemy as sa
def upgrade():
op.add_column('runs',
sa.Column('can_delete', sa.Boolean(), nullable=False, server_default=sa.sql.expression.true(), default=True))
def downgrade():
# downgrade is not supported
sys.exit(1)

tools/codechecker/libcodechecker/db_model/__init__.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.

tools/codechecker/libcodechecker/db_model/orm_model.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
ORM model.
'''
from __future__ import print_function
from __future__ import unicode_literals
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.sql.expression import true
from sqlalchemy.ext.declarative import declarative_base
from datetime import datetime
from math import ceil
CC_META = MetaData(naming_convention={
"ix": 'ix_%(column_0_label)s',
"uq": "uq_%(table_name)s_%(column_0_name)s",
"ck": "ck_%(table_name)s_%(column_0_name)s",
"fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
"pk": "pk_%(table_name)s"
})
# Create base class for ORM classes
Base = declarative_base(metadata=CC_META)
# Start of ORM classes
class DBVersion(Base):
__tablename__ = 'db_version'
# TODO: constraint, only one line in this table
major = Column(Integer, primary_key=True)
minor = Column(Integer, primary_key=True)
def __init__(self, major, minor):
self.major = major
self.minor = minor
class Run(Base):
__tablename__ = 'runs'
__table_args__ = (
UniqueConstraint('name'),
)
id = Column(Integer, autoincrement=True, primary_key=True)
date = Column(DateTime)
duration = Column(Integer) # Seconds, -1 if unfinished.
name = Column(String)
version = Column(String)
command = Column(String)
inc_count = Column(Integer)
can_delete = Column(Boolean, nullable=False, server_default=true(), default=True)
# Relationships (One to Many).
configlist = relationship('Config', cascade="all, delete-orphan", passive_deletes=True)
buildactionlist = relationship('BuildAction', cascade="all, delete-orphan", passive_deletes=True)
def __init__(self, name, version, command):
self.date, self.name, self.version, self.command = \
datetime.now(), name, version, command
self.duration = -1
self.inc_count = 0
def mark_finished(self):
self.duration = ceil((datetime.now() - self.date).total_seconds())
class Config(Base):
__tablename__ = 'configs'
run_id = Column(Integer, ForeignKey('runs.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), primary_key=True)
checker_name = Column(String, primary_key=True)
attribute = Column(String, primary_key=True)
value = Column(String, primary_key=True)
def __init__(self, run_id, checker_name, attribute, value):
self.attribute, self.value = attribute, value
self.checker_name, self.run_id = checker_name, run_id
class File(Base):
__tablename__ = 'files'
id = Column(Integer, autoincrement=True, primary_key=True)
run_id = Column(Integer, ForeignKey('runs.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'))
filepath = Column(String)
content = Column(Binary)
inc_count = Column(Integer)
def __init__(self, run_id, filepath):
self.run_id, self.filepath = run_id, filepath
self.inc_count = 0
def addContent(self, content):
self.content = content
class BuildAction(Base):
__tablename__ = 'build_actions'
id = Column(Integer, autoincrement=True, primary_key=True)
run_id = Column(Integer, ForeignKey('runs.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'))
build_cmd = Column(String)
analyzer_type = Column(String, nullable=False)
analyzed_source_file = Column(String, nullable=False)
check_cmd = Column(String)
# No failure if the text is empty.
failure_txt = Column(String)
date = Column(DateTime)
# Seconds, -1 if unfinished.
duration = Column(Integer)
def __init__(self, run_id, build_cmd, check_cmd, analyzer_type, analyzed_source_file):
self.run_id, self.build_cmd, self.check_cmd, self.failure_txt = \
run_id, build_cmd, check_cmd, ''
self.date = datetime.now()
self.analyzer_type = analyzer_type
self.analyzed_source_file = analyzed_source_file
self.duration = -1
def mark_finished(self, failure_txt):
self.failure_txt = failure_txt
self.duration = (datetime.now() - self.date).total_seconds()
class BugPathEvent(Base):
__tablename__ = 'bug_path_events'
id = Column(Integer, autoincrement=True, primary_key=True)
line_begin = Column(Integer)
col_begin = Column(Integer)
line_end = Column(Integer)
col_end = Column(Integer)
msg = Column(String)
file_id = Column(Integer, ForeignKey('files.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), index = True)
next = Column(Integer)
prev = Column(Integer)
def __init__(self, line_begin, col_begin, line_end, col_end, msg, file_id):
self.line_begin, self.col_begin, self.line_end, self.col_end, self.msg = \
line_begin, col_begin, line_end, col_end, msg
self.file_id = file_id
def addPrev(self, prev):
self.prev = prev
def addNext(self, next):
self.next = next
def isLast(self):
return self.next is None
def isFirst(self):
return self.prev is None
class BugReportPoint(Base):
__tablename__ = 'bug_report_points'
id = Column(Integer, autoincrement=True, primary_key=True)
line_begin = Column(Integer)
col_begin = Column(Integer)
line_end = Column(Integer)
col_end = Column(Integer)
file_id = Column(Integer, ForeignKey('files.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), index = True)
# TODO: Add check, the value must be an existing id or null.
# Be careful when inserting.
next = Column(Integer)
def __init__(self, line_begin, col_begin, line_end, col_end, file_id):
self.line_begin, self.col_begin, self.line_end, self.col_end = \
line_begin, col_begin, line_end, col_end
self.file_id = file_id
def addNext(self, next):
self.next = next
def isLast(self):
return self.next is None
class Report(Base):
__tablename__ = 'reports'
id = Column(Integer, autoincrement=True, primary_key=True)
file_id = Column(Integer, ForeignKey('files.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'))
run_id = Column(Integer, ForeignKey('runs.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), index = True)
bug_id = Column(String, index = True)
checker_id = Column(String)
checker_cat = Column(String)
bug_type = Column(String)
severity = Column(Integer)
# TODO: multiple messages to multiple source locations?
checker_message = Column(String)
start_bugpoint = Column(Integer, ForeignKey('bug_report_points.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'))
start_bugevent = Column(Integer, ForeignKey('bug_path_events.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), index = True)
end_bugevent = Column(Integer, ForeignKey('bug_path_events.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), index = True)
suppressed = Column(Boolean)
# Cascade delete might remove rows SQLAlchemy warns about this
# to remove warnings about already deleted items set this to False.
__mapper_args__ = {
'confirm_deleted_rows' : False
}
# Priority/severity etc...
def __init__(self, run_id, bug_id, file_id, checker_message, start_bugpoint, start_bugevent, end_bugevent, checker_id, checker_cat, bug_type, severity, suppressed):
self.run_id = run_id
self.file_id = file_id
self.bug_id, self.checker_message = bug_id, checker_message
self.start_bugpoint = start_bugpoint
self.start_bugevent = start_bugevent
self.end_bugevent = end_bugevent
self.severity = severity
self.checker_id, self.checker_cat, self.bug_type = checker_id, checker_cat, bug_type
self.suppressed = suppressed
class ReportsToBuildActions(Base):
__tablename__ = 'reports_to_build_actions'
report_id = Column(Integer, ForeignKey('reports.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), primary_key=True)
build_action_id = Column(
Integer, ForeignKey('build_actions.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), primary_key=True)
def __init__(self, report_id, build_action_id):
self.report_id = report_id
self.build_action_id = build_action_id
class SuppressBug(Base):
__tablename__ = 'suppress_bug'
id = Column(Integer, autoincrement=True, primary_key=True)
hash = Column(String, nullable=False)
file_name = Column(String)
run_id = Column(Integer, ForeignKey('runs.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), nullable=False)
comment = Column(Binary)
def __init__(self, run_id, hash, file_name, comment):
self.hash, self.run_id = hash, run_id
self.comment = comment
self.file_name = file_name
class SkipPath(Base):
__tablename__ = 'skip_path'
id = Column(Integer, autoincrement=True, primary_key=True)
path = Column(String)
run_id = Column(Integer, ForeignKey('runs.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), nullable=False)
comment = Column(Binary)
def __init__(self, run_id, path, comment):
self.path = path
self.run_id = run_id
self.comment = comment
# End of ORM classes.
def CreateSchema(engine):
""" Creates the schema if it does not exists.
Do not check version or do migration yet. """
Base.metadata.create_all(engine)
def CreateSession(engine):
""" Creates a scoped session factory that can act like a session.
The factory uses a thread_local registry, so every thread have
its own session. """
SessionFactory = scoped_session(sessionmaker(bind=engine))
return SessionFactory

tools/codechecker/libcodechecker/db_version.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
class DBVersionInfo(object):
def __init__(self, major, minor):
self.major = int(major)
self.minor = int(minor)
def is_compatible(self, major, minor):
return major == self.major
def get_expected_version(self):
return (self.major, self.minor)
def __str__(self):
return 'v'+str(self.major)+'.'+str(self.minor)

tools/codechecker/libcodechecker/debug_reporter.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import os
import sys
import sqlalchemy
from sqlalchemy.sql import and_
from db_model.orm_model import BuildAction
from db_model.orm_model import Run
from libcodechecker import analyzer_env
from libcodechecker import logger
from libcodechecker import analyzer_crash_handler
from libcodechecker import database_handler
LOG = logger.get_new_logger('DEBUG_REPORTER')
# -----------------------------------------------------------------------------
def get_dump_file_name(run_id, action_id):
return 'action_' + str(run_id) + '_' + str(action_id) + '_dump.log'
# -----------------------------------------------------------------------------
def debug(context, connection_string, force):
try:
engine = database_handler.SQLServer.create_engine(connection_string)
session = sqlalchemy.orm.scoped_session(
sqlalchemy.orm.sessionmaker(bind=engine))
# Get latest run id
last_run = session.query(Run).order_by(Run.id.desc()).first()
# Get all failed actions
actions = session.query(BuildAction).filter(and_(
BuildAction.run_id == last_run.id,
sqlalchemy.sql.func.length(BuildAction.failure_txt) != 0))
debug_env = analyzer_env.get_check_env(context.path_env_extra,
context.ld_lib_path_extra)
crash_handler = analyzer_crash_handler.AnalyzerCrashHandler(context,
debug_env)
dumps_dir = context.dump_output_dir
if not os.path.exists(dumps_dir):
os.mkdir(dumps_dir)
LOG.info('Generating gdb dump files to : ' + dumps_dir)
for action in actions:
LOG.info('Processing action ' + str(action.id) + '.')
debug_log_file = \
os.path.join(dumps_dir, get_dump_file_name(last_run.id, action.id))
if not force and os.path.exists(debug_log_file):
LOG.info('This file already exists.')
continue
LOG.info('Generating stacktrace with gdb.')
gdb_result = \
crash_handler.get_crash_info(str(action.check_cmd).split())
LOG.info('Writing debug info to file.')
with open(debug_log_file, 'w') as log_file:
log_file.write('========================\n')
log_file.write('Original build command: \n')
log_file.write('========================\n')
log_file.write(action.build_cmd + '\n')
log_file.write('===============\n')
log_file.write('Check command: \n')
log_file.write('===============\n')
log_file.write(action.check_cmd + '\n')
log_file.write('==============\n')
log_file.write('Failure text: \n')
log_file.write('==============\n')
log_file.write(action.failure_txt + '\n')
log_file.write('==========\n')
log_file.write('GDB info: \n')
log_file.write('==========\n')
log_file.write(gdb_result)
LOG.info('All new debug files are placed in ' + dumps_dir)
except KeyboardInterrupt as kb_exc:
LOG.error(str(kb_exc))
sys.exit(1)

tools/codechecker/libcodechecker/decorators.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import time
import shared
import sqlalchemy
def timeit(method):
def timed(*args, **kw):
timer_begin = time.time()
result = method(*args, **kw)
timer_end = time.time()
print '%r (%r, %r) %2.2f sec' % (method.__name__, args, kw,
timer_end - timer_begin)
return result
return timed
def trace(method):
def wrapped(*args, **kw):
print('Stepped into ' + method.__name__)
result = method(*args, **kw)
print('Stepped out ' + method.__name__)
return result
return wrapped
def catch_sqlalchemy(method):
def wrapped(*args, **kw):
try:
return method(*args, **kw)
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE,
msg)
return wrapped

tools/codechecker/libcodechecker/generic_package_context.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import os
import sys
import json
from libcodechecker import context_base
from libcodechecker import logger
from libcodechecker import db_version
from libcodechecker.analyzers import analyzer_types
LOG = logger.get_new_logger('CONTEXT')
# -----------------------------------------------------------------------------
class Context(context_base.ContextBase):
''' generic package specific context'''
__instance = None
logger_bin = None
logger_file = None
logger_compilers = None
ld_preload = None
__package_version = None
__package_root = None
def __init__(self, package_root, pckg_layout, cfg_dict):
env_vars = cfg_dict['environment_variables']
variables = cfg_dict['package_variables']
self.__checker_config = cfg_dict['checker_config']
# Base class constructor gets the common environment variables
super(Context, self).__init__()
super(Context, self).load_data(env_vars, pckg_layout, variables)
# get package specific environment variables
self.set_env(env_vars)
self.__package_root = package_root
version, database_version = self.__get_version(self.version_file)
self.__package_version = version
self.__db_version_info = db_version.DBVersionInfo(database_version['major'],
database_version['minor'])
Context.__instance = self
def set_env(self, env_vars):
'''
get the environment variables
'''
super(Context, self).set_env(env_vars)
# get generic package specific environment variables
self.logger_bin = os.environ.get(env_vars['cc_logger_bin'])
self.logger_file = os.environ.get(env_vars['cc_logger_file'])
self.logger_compilers = os.environ.get(env_vars['cc_logger_compiles'])
self.ld_preload = os.environ.get(env_vars['ld_preload'])
self.ld_lib_path = env_vars['env_ld_lib_path']
def __get_version(self, version_file):
'''
get the package version fron the verison config file
'''
try:
with open(version_file, 'r') as vfile:
vfile_data = json.loads(vfile.read())
package_version = vfile_data['version']
db_version = vfile_data['db_version']
except ValueError as verr:
# db_version is required to know if the db schema is compatible
LOG.error('Failed to get version info from the version file')
LOG.error(verr)
sys.exit(1)
return package_version, db_version
@property
def default_checkers_config(self):
return self.__checker_config
@property
def version(self):
return self.__package_version
@property
def db_version_info(self):
return self.__db_version_info
@property
def version_file(self):
return os.path.join(self.__package_root, self.pckg_layout['version_file'])
@property
def env_var_cc_logger_bin(self):
return self.env_vars['cc_logger_bin']
@property
def env_var_ld_preload(self):
return self.env_vars['ld_preload']
@property
def env_var_cc_logger_compiles(self):
return self.env_vars['cc_logger_compiles']
@property
def env_var_cc_logger_file(self):
return self.env_vars['cc_logger_file']
@property
def path_logger_bin(self):
return os.path.join(self.package_root,
self.pckg_layout['ld_logger_bin'])
@property
def path_logger_lib(self):
return os.path.join(self.package_root,
self.pckg_layout['ld_logger_lib_path'])
@property
def logger_lib_name(self):
return self.pckg_layout['ld_logger_lib_name']
@property
def dumps_dir_name(self):
return self.variables['path_dumps_name']
@property
def pg_data_dir(self):
return os.path.join(self.codechecker_workspace,
self.pgsql_data_dir_name)
@property
def dump_output_dir(self):
return os.path.join(self.codechecker_workspace,
self.variables['path_dumps_name'])
@property
def compiler_resource_dirs(self):
compiler_resource_dirs = self.pckg_layout.get('compiler_resource_dirs')
if not compiler_resource_dirs:
return []
else:
inc_dirs = []
for path in compiler_resource_dirs:
if os.path.isabs(path):
inc_dirs.append(path)
else:
inc_dirs.append(os.path.join(self.__package_root, path))
return inc_dirs
@property
def path_env_extra(self):
extra_paths = self.pckg_layout.get('path_env_extra')
if not extra_paths:
return []
else:
paths = []
for path in extra_paths:
if os.path.isabs(path):
paths.append(path)
else:
paths.append(os.path.join(self.__package_root, path))
return paths
@property
def ld_lib_path_extra(self):
extra_lib = self.pckg_layout.get('ld_lib_path_extra')
if not extra_lib:
return []
else:
ld_paths = []
for path in extra_lib:
if os.path.isabs(path):
ld_paths.append(path)
else:
ld_paths.append(os.path.join(self.__package_root, path))
return ld_paths
@property
def analyzer_binaries(self):
analyzers = {}
compiler_binaries = self.pckg_layout.get('analyzers')
if not compiler_binaries:
# set default analyzers assume they are in the PATH
# will be checked later
# key naming in the dict should be the same as in
# the supported analyzers list
analyzers[analyzer_types.CLANG_SA] = 'clang'
analyzers[analyzer_types.CLANG_TIDY] = 'clang-tidy'
else:
for name, value in compiler_binaries.iteritems():
if os.path.isabs(value):
# check if it is an absolute path
analyzers[name] = value
elif os.path.dirname(value):
# check if it is a package relative path
analyzers[name] = os.path.join(self.__package_root, value)
else:
analyzers[name] = value
return analyzers
def get_context():
LOG.debug('Loading package config')
package_root = os.environ['CC_PACKAGE_ROOT']
pckg_config_file = os.path.join(package_root, "config", "config.json")
LOG.debug('Reading config: ' + pckg_config_file)
with open(pckg_config_file, 'r') as cfg:
cfg_dict = json.loads(cfg.read())
LOG.debug(cfg_dict)
LOG.debug('Loading layout config')
layout_cfg_file = os.path.join(package_root, "config", "package_layout.json")
LOG.debug(layout_cfg_file)
with open(layout_cfg_file, 'r') as lcfg:
lcfg_dict = json.loads(lcfg.read())
# merge static and runtime layout
layout_config = lcfg_dict['static'].copy()
layout_config.update(lcfg_dict['runtime'])
LOG.debug(layout_config)
try:
return Context(package_root,
layout_config,
cfg_dict)
except KeyError as kerr:
LOG.error(kerr)
sys.exit(1)

tools/codechecker/libcodechecker/generic_package_suppress_handler.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
handler for suppressing a bug
'''
from libcodechecker import logger
from libcodechecker import suppress_handler
from libcodechecker import suppress_file_handler
# Warning! this logger should only be used in this module
LOG = logger.get_new_logger('SUPPRESS')
class GenericSuppressHandler(suppress_handler.SuppressHandler):
def store_suppress_bug_id(self, bug_id, file_name, comment):
if self.suppress_file is None:
return False
ret = suppress_file_handler.write_to_suppress_file(self.suppress_file,
bug_id,
file_name,
comment)
return ret
def remove_suppress_bug_id(self, bug_id, file_name):
if self.suppress_file is None:
return False
ret = suppress_file_handler.remove_from_suppress_file(self.suppress_file,
bug_id,
file_name)
return ret

tools/codechecker/libcodechecker/host_check.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import errno
import subprocess
import os
from libcodechecker import logger
LOG = logger.get_new_logger('HOST CHECK')
# -----------------------------------------------------------------------------
def check_zlib():
''' Check if zlib compression is available
if wrong libraries are installed on the host machine it is
possible the the compression failes which is required to
store data into the database.
'''
try:
import zlib
return True
except Exception as ex:
LOG.error(str(ex))
LOG.error('Failed to import zlib module')
return False
try:
zlib.compress('Compress this')
return True
except Exception as ex:
LOG.error(str(ex))
LOG.error('Zlib copression error', zlib.Z_BEST_COMPRESSION)
return False
# -----------------------------------------------------------------------------
def get_postgresql_driver_name():
try:
driver = os.getenv('CODECHECKER_DB_DRIVER')
if driver:
return driver
try:
import psycopg2 # NOQA
return "psycopg2"
except Exception:
import pg8000 # NOQA
return "pg8000"
except Exception as ex:
LOG.error(str(ex))
LOG.error('Failed to import psycopg2 or pg8000 module.')
raise
# -----------------------------------------------------------------------------
def check_postgresql_driver():
try:
get_postgresql_driver_name()
return True
except Exception as ex:
LOG.debug(ex)
return False
# -----------------------------------------------------------------------------
def check_sql_driver(check_postgresql):
if check_postgresql:
try:
get_postgresql_driver_name()
return True
except Exception as ex:
return False
else:
try:
try:
import pysqlite2
except Exception as ex:
import sqlite3
except Exception as ex:
LOG.debug(ex)
return False
return True
# -----------------------------------------------------------------------------
def check_clang(compiler_bin, env):
'''
simple check if clang is available
'''
clang_version_cmd = [compiler_bin, '--version']
LOG.debug_analyzer(' '.join(clang_version_cmd))
try:
res = subprocess.call(clang_version_cmd,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
if not res:
return True
else:
LOG.debug_analyzer('Failed to run: "' + ' '.join(clang_version_cmd) + '"')
return False
except OSError as oerr:
if oerr[0] == errno.ENOENT:
LOG.error(oerr)
LOG.error('Failed to run: ' + ' '.join(clang_version_cmd) + '"')
return False
# -----------------------------------------------------------------------------
def check_intercept(env):
'''
simple check if intercept (scan-build-py) is available
'''
intercept_cmd = ['intercept-build']
try:
res = subprocess.call(intercept_cmd,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
if not res:
return True
else:
LOG.debug('Failed to run: "' + ' '.join(intercept_cmd) + '"')
return False
except OSError as oerr:
if oerr[0] == errno.ENOENT:
# not just intercept-build can be used for logging
# it is possible that another build logger is available
LOG.debug(oerr)
LOG.debug('Failed to run: ' + ' '.join(intercept_cmd) + '"')
return False

tools/codechecker/libcodechecker/log_parser.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import os
import traceback
from libcodechecker import build_action
from libcodechecker import logger
from libcodechecker import option_parser
LOG = logger.get_new_logger('LOG PARSER')
# -----------------------------------------------------------------------------
def parse_compile_commands_json(logfile):
import json
actions = []
filtered_build_actions = {}
logfile.seek(0)
data = json.load(logfile)
counter = 0
for entry in data:
sourcefile = entry['file']
lang = option_parser.get_language(sourcefile[sourcefile.rfind('.'):])
if not lang:
continue
action = build_action.BuildAction(counter)
command = entry['command']
results = option_parser.parse_options(command)
action.original_command = command
action.analyzer_options = results.compile_opts
action.lang = results.lang
action.target = results.arch
if results.action == option_parser.ActionType.COMPILE or \
results.action == option_parser.ActionType.LINK:
action.skip = False
# TODO: check arch
action.directory = entry['directory']
action.sources = sourcefile
action.lang = lang
# filter out duplicate compilation commands
unique_key = action.cmp_key
if filtered_build_actions.get(unique_key) is None:
filtered_build_actions[unique_key] = action
del action
counter += 1
for ba_hash, ba in filtered_build_actions.iteritems():
actions.append(ba)
return actions
# -----------------------------------------------------------------------------
def parse_log(logfilepath):
LOG.debug_analyzer('Parsing log file: ' + logfilepath)
actions = []
with open(logfilepath) as logfile:
try:
actions = parse_compile_commands_json(logfile)
except (ValueError, KeyError, TypeError) as ex:
if os.stat(logfilepath).st_size == 0:
LOG.error('The compile database is empty.')
else:
LOG.error('The compile database is not valid.')
LOG.debug(traceback.format_exc())
LOG.debug(ex)
LOG.debug_analyzer('Parsing log file done.')
return actions

tools/codechecker/libcodechecker/logger.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
'''
import os
import sys
import logging
# the logging leves can be accesses without
# importing the logging module in other modules
DEBUG = logging.DEBUG
INFO = logging.INFO
WARNING = logging.WARNING
ERROR = logging.ERROR
CRITICAL = logging.CRITICAL
NOTSET = logging.NOTSET
class BColors(object):
HEADER = '\033[95m'
OKBLUE = '\033[94m'
OKGREEN = '\033[92m'
WARNING = '\033[93m'
FAIL = '\033[91m'
ENDC = '\033[0m'
# ------------------------------------------------------------------------------
logging.DEBUG_ANALYZER = 15
logging.addLevelName(logging.DEBUG_ANALYZER, 'DEBUG_ANALYZER')
class CCLogger(logging.Logger):
def __init__(self, name, level=NOTSET):
return super(CCLogger, self).__init__(name, level)
def debug_analyzer(self, msg, *args, **kwargs):
if self.isEnabledFor(logging.DEBUG_ANALYZER):
self._log(logging.DEBUG_ANALYZER, msg, args, **kwargs)
logging.setLoggerClass(CCLogger)
# ------------------------------------------------------------------------------
def get_log_level():
'''
'''
level = os.getenv('CODECHECKER_VERBOSE')
if level:
if level == 'debug':
return logging.DEBUG
elif level == 'debug_analyzer':
return logging.DEBUG_ANALYZER
return logging.INFO
# ------------------------------------------------------------------------------
def get_new_logger(logger_name, out_stream=sys.stdout):
'''
'''
loglevel = get_log_level()
logger = logging.getLogger('[' + logger_name + ']')
stdout_handler = logging.StreamHandler(stream=out_stream)
if not getattr(logger, 'handlers'):
if loglevel == logging.DEBUG:
# FIXME create a new handler to write all log messages into a file
# FIXME filter stdout messages only from analyzer, parser,
# report, sourceDep (any other?) are printed out to the stdout
logger.setLevel(logging.DEBUG)
stdout_handler.setLevel(logging.DEBUG)
format_str = '[%(process)d] <%(thread)d> - %(filename)s:%(lineno)d %(funcName)s() - %(message)s'
msg_formatter = logging.Formatter(format_str)
stdout_handler.setFormatter(msg_formatter)
logger.addHandler(stdout_handler)
elif loglevel == logging.DEBUG_ANALYZER:
logger.setLevel(logging.DEBUG_ANALYZER)
stdout_handler.setLevel(logging.DEBUG_ANALYZER)
msg_formatter = logging.Formatter('[%(levelname)s] - %(message)s')
stdout_handler.setFormatter(msg_formatter)
logger.addHandler(stdout_handler)
else:
logger.setLevel(logging.INFO)
stdout_handler.setLevel(logging.INFO)
msg_formatter = logging.Formatter('[%(levelname)s] - %(message)s')
stdout_handler.setFormatter(msg_formatter)
logger.addHandler(stdout_handler)
return logger

tools/codechecker/libcodechecker/option_parser.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
import re
import shlex
from libcodechecker import logger
LOG = logger.get_new_logger('OPTION PARSER')
# TODO: make modular option handling system,
# configuring possibility from config file
# class OpenFilterPluginBase(object):
# __metaclass__ = abc.ABCMeta
#
# @abc.abstractproperty
# def compiler_name(self):
# ''' Compiler name (regex expression)'''
# return None
#
# @abc.abstactmethod
# def get_exclude_rules(self):
# ''' Remove options, x -> nothing'''
# return {}
#
# @abc.abstactmethod
# def get_modify_rules(self):
# ''' Modify options, x -> y'''
# return {}
#
# @abc.abstactmethod
# def get_add_rules(self):
# ''' Add rules,::x '''
# return []
#
# class DefaultFilterPlugin(OpenFilterPluginBase):
# __compiler_name = '.*'
#
# @property
# def compiler_name(self):
# return __compiler_name
#
# def get_exclude_rules(self):
# return {}
#
# def get_modify_rules(self):
# return {}
#
# def get_add_rules(self):
# return []
# ---------------------------------------------------------------------------- #
# Lookup tables. From ScanBuild: clang-analyzer.llvm.org/scan-build.html
# ---------------------------------------------------------------------------- #
COMPILE_OPTION_MAP = {
'-nostdinc': 0,
'-include': 1,
'-idirafter': 1,
'-imacros': 1,
'-iprefix': 1,
'-isystem': 1,
'-iwithprefix': 1,
'-iwithprefixbefore': 1
}
COMPILE_OPTION_MAP_REGEX = {
'-O([1-3]|s)?$': 0,
'-std=.*': 0,
'^-f.*': 0,
'-m.*': 0,
'^-Wno-.*': 0,
'^-m(32|64)$': 0
}
COMPILE_OPTION_MAP_MERGED = [
'^-iquote(.*)$',
'^-[DIU](.*)$',
'^-F(.+)$'
]
COMPILER_LINKER_OPTION_MAP = {
'-write-strings': 0,
'-ftrapv-handler': 1,
'-mios-simulator-version-min': 0,
'-sysroot': 1,
'-stdlib': 0,
'-target': 1,
'-v': 0,
'-mmacosx-version-min': 0,
'-miphoneos-version-min': 0
}
LINKER_OPTION_MAP = {
'-framework': 1,
'-fobjc-link-runtime': 0
}
IGNORED_OPTION_MAP = {
'-MT': 1,
'-MF': 1,
'-fsyntax-only': 0,
'-save-temps': 0,
'-install_name': 1,
'-exported_symbols_list': 1,
'-current_version': 1,
'-compatibility_version': 1,
'-init': 1,
'-e': 1,
'-seg1addr': 1,
'-bundle_loader': 1,
'-multiply_defined': 1,
'-sectorder': 3,
'--param': 1,
'-u': 1,
'--serialize-diagnostics': 1
}
UNIQUE_OPTIONS = {
'-isysroot': 1
}
REPLACE_OPTIONS_MAP = {
'-mips32' : [ '-target', 'mips', '-mips32' ],
'-mips64' : [ '-target', 'mips64', '-mips64' ],
'-mpowerpc' : [ '-target', 'powerpc' ],
'-mpowerpc64' : [ '-target', 'powerpc64' ]
}
UNKNOWN_OPTIONS_MAP_REGEX = {
'^-fcall-saved-.*': 0,
'^-fcond-mismatch': 0,
'^-fconserve-stack': 0,
'^-fcrossjumping': 0,
'^-fcse-follow-jumps': 0,
'^-fcse-skip-blocks': 0,
'^-ffixed-r2': 0,
'^-fgcse-lm': 0,
'^-fhoist-adjacent-loads': 0,
'^-findirect-inlining': 0,
'^-finline-limit.*': 0,
'^-fipa-sra': 0,
'^-fno-delete-null-pointer-checks': 0,
'^-fno-strength-reduce': 0,
'^-fno-toplevel-reorder': 0,
'^-fno-unit-at-a-time': 0,
'^-fno-var-tracking-assignments': 0,
'^-fpartial-inlining': 0,
'^-fpeephole2': 0,
'^-fregmove': 0,
'^-frename-registers': 0,
'^-freorder-functions': 0,
'^-frerun-cse-after-loop': 0,
'^-fsched-spec': 0,
'^-fthread-jumps': 0,
'^-ftree-pre': 0,
'^-ftree-switch-conversion': 0,
'^-ftree-tail-merge': 0,
'^-m(no-)?sdata.*$': 0,
'^-m(no-)?spe.*': 0,
'^-m(no-)?string$': 0,
'^-maccumulate-outgoing-args': 0,
'^-mfix-cortex-m3-ldrd$': 0,
'^-mmultiple$': 0,
'^-mthumb-interwork$': 0,
'^-mupdate$': 0
}
# -----------------------------------------------------------------------------
class ActionType(object):
LINK, COMPILE, PREPROCESS, INFO = range(4)
# -----------------------------------------------------------------------------
class OptionParserResult(object):
def __init__(self):
self._action = ActionType.LINK
self._compile_opts = []
self._link_opts = []
self._files = []
self._arch = ''
self._lang = None
self._output = ''
@property
def action(self):
return self._action
@action.setter
def action(self, value):
self._action = value
@property
def compile_opts(self):
return self._compile_opts
@compile_opts.setter
def compile_opts(self, value):
self._compile_opts = value
@property
def link_opts(self):
return self._link_opts
@link_opts.setter
def link_opts(self, value):
self._link_opts = value
@property
def files(self):
return self._files
@files.setter
def files(self, value):
self._files = value
@property
def arch(self):
return self._arch
@arch.setter
def arch(self, value):
self._arch = value
@property
def lang(self):
return self._lang
@lang.setter
def lang(self, value):
self._lang = value
@property
def output(self):
return self._output
@output.setter
def output(self, value):
self._output = value
# -----------------------------------------------------------------------------
class OptionIterator(object):
def __init__(self, args):
self._item = None
self._it = iter(args)
def next(self):
self._item = next(self._it)
return self # ._item
def __iter__(self):
return self
@property
def item(self):
return self._item
# -----------------------------------------------------------------------------
def arg_check(it, result):
def regex_match(string, pattern):
regexp = re.compile(pattern)
match = regexp.match(string)
return match is not None
# Handler functions for options
def append_to_list(table, target_list, regex=False):
'''Append n item from iterator to to result[att_name] list.'''
def wrapped(value):
def append_n(size):
target_list.append(it.item)
for x in xrange(0, size):
it.next()
target_list.append(it.item)
if regex:
for pattern in table:
if regex_match(value, pattern):
append_n(table[pattern])
return True
elif value in table:
append_n(table[value])
return True
return False
return wrapped
def append_merged_to_list(table, target_list):
''' Append one or two item to the list.
1: if there is no space between two option.
2: otherwise.
'''
def wrapped(value):
for pattern in table:
match = re.match(pattern, value)
if match is not None:
tmp = it.item
if match.group(1) == '':
it.next()
tmp = tmp + it.item
target_list.append(tmp)
return True
return False
return wrapped
def append_to_list_from_file(arg, target_list):
'''Append items from file to to result[att_name] list.'''
def wrapped(value):
if value == arg:
it.next()
with open(it.item) as file:
for line in file:
target_list.append(line.strip())
return True
return False
return wrapped
def append_replacement_to_list(table, target_list, regex=False):
'''Append replacement items from table to to result[att_name] list.'''
def wrapped(value):
def append_replacement(items):
for item in items:
target_list.append(item)
it.next()
if regex:
for pattern in table:
if regex_match(value, pattern):
append_replacement(table[pattern])
return True
elif value in table:
append_replacement(table[value])
return True
return False
return wrapped
def set_attr(arg, attr_name, attr_value=None, regex=None):
'''Set an attr value. If no value given then read next from iterator.'''
def wrapped(value):
if (regex and regex_match(value, arg)) or value == arg:
tmp = attr_value
if attr_value is None:
it.next()
tmp = it.item
attr_name = tmp
return True
return False
return wrapped
def skip(table, regex=False):
'''Skip n item in iterator.'''
def wrapped(value):
def skip_n(size):
for x in xrange(0, size):
it.next()
if regex:
for pattern in table:
if regex_match(value, pattern):
table[pattern]
return True
elif value in table:
skip_n(table[value])
return True
return False
return wrapped
# Defines handler functions for tables and single options
arg_collection = [
append_replacement_to_list(REPLACE_OPTIONS_MAP, result.compile_opts),
skip(UNKNOWN_OPTIONS_MAP_REGEX, True),
append_to_list(COMPILE_OPTION_MAP, result.compile_opts),
append_to_list(COMPILER_LINKER_OPTION_MAP, result.compile_opts),
append_to_list(COMPILE_OPTION_MAP_REGEX, result.compile_opts, True),
append_merged_to_list(COMPILE_OPTION_MAP_MERGED, result.compile_opts),
set_attr('-x', result.lang),
set_attr('-o', result.output),
set_attr('-arch', result.arch),
set_attr('-c', result.action, ActionType.COMPILE),
set_attr('^-(E|MM?)$', result.action, ActionType.PREPROCESS, True),
set_attr('-print-prog-name', result.action, ActionType.INFO),
skip(LINKER_OPTION_MAP),
skip(IGNORED_OPTION_MAP),
skip(UNIQUE_OPTIONS),
append_to_list_from_file('-filelist', result.files),
append_to_list({'^[^-].+': 0}, result.files, True)]
return any((collection(it.item) for collection in arg_collection))
# -----------------------------------------------------------------------------
def parse_options(args):
'''Requires a full compile command with the compiler, not only arguments.'''
# keep " characters
args = args.replace('"', '\\"')
result_map = OptionParserResult()
for it in OptionIterator(shlex.split(args)[1:]):
arg_check(it, result_map) # TODO: do sth at False result, actually skip
return result_map
# -----------------------------------------------------------------------------
def get_language(extension):
mapping = {'.c': 'c',
'.cp': 'c++',
'.cpp': 'c++',
'.cxx': 'c++',
'.txx': 'c++',
'.cc': 'c++',
'.C': 'c++',
'.ii': 'c++',
'.m': 'objective-c',
'.mm': 'objective-c++'}
lang = mapping.get(extension) if extension in mapping else None
return lang

tools/codechecker/libcodechecker/orm_model.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
ORM model.
'''
from __future__ import print_function
from __future__ import unicode_literals
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.sql.expression import true
from sqlalchemy.ext.declarative import declarative_base
from datetime import datetime
from math import ceil
CC_META = MetaData(naming_convention={
"ix": 'ix_%(column_0_label)s',
"uq": "uq_%(table_name)s_%(column_0_name)s",
"ck": "ck_%(table_name)s_%(column_0_name)s",
"fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
"pk": "pk_%(table_name)s"
})
# Create base class for ORM classes
Base = declarative_base(metadata=CC_META)
# Start of ORM classes
class DBVersion(Base):
__tablename__ = 'db_version'
# TODO: constraint, only one line in this table
major = Column(Integer, primary_key=True)
minor = Column(Integer, primary_key=True)
def __init__(self, major, minor):
self.major = major
self.minor = minor
class Run(Base):
__tablename__ = 'runs'
__table_args__ = (
UniqueConstraint('name'),
)
id = Column(Integer, autoincrement=True, primary_key=True)
date = Column(DateTime)
duration = Column(Integer) # Seconds, -1 if unfinished.
name = Column(String)
version = Column(String)
command = Column(String)
inc_count = Column(Integer)
can_delete = Column(Boolean, nullable=False, server_default=true(), default=True)
# Relationships (One to Many).
configlist = relationship('Config', cascade="all, delete-orphan", passive_deletes=True)
buildactionlist = relationship('BuildAction', cascade="all, delete-orphan", passive_deletes=True)
def __init__(self, name, version, command):
self.date, self.name, self.version, self.command = \
datetime.now(), name, version, command
self.duration = -1
self.inc_count = 0
def mark_finished(self):
self.duration = ceil((datetime.now() - self.date).total_seconds())
class Config(Base):
__tablename__ = 'configs'
run_id = Column(Integer, ForeignKey('runs.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), primary_key=True)
checker_name = Column(String, primary_key=True)
attribute = Column(String, primary_key=True)
value = Column(String, primary_key=True)
def __init__(self, run_id, checker_name, attribute, value):
self.attribute, self.value = attribute, value
self.checker_name, self.run_id = checker_name, run_id
class File(Base):
__tablename__ = 'files'
id = Column(Integer, autoincrement=True, primary_key=True)
run_id = Column(Integer, ForeignKey('runs.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'))
filepath = Column(String)
content = Column(Binary)
inc_count = Column(Integer)
def __init__(self, run_id, filepath):
self.run_id, self.filepath = run_id, filepath
self.inc_count = 0
def addContent(self, content):
self.content = content
class BuildAction(Base):
__tablename__ = 'build_actions'
id = Column(Integer, autoincrement=True, primary_key=True)
run_id = Column(Integer, ForeignKey('runs.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'))
build_cmd = Column(String)
analyzer_type = Column(String, nullable=False)
analyzed_source_file = Column(String, nullable=False)
check_cmd = Column(String)
# No failure if the text is empty.
failure_txt = Column(String)
date = Column(DateTime)
# Seconds, -1 if unfinished.
duration = Column(Integer)
def __init__(self, run_id, build_cmd, check_cmd, analyzer_type, analyzed_source_file):
self.run_id, self.build_cmd, self.check_cmd, self.failure_txt = \
run_id, build_cmd, check_cmd, ''
self.date = datetime.now()
self.analyzer_type = analyzer_type
self.analyzed_source_file = analyzed_source_file
self.duration = -1
def mark_finished(self, failure_txt):
self.failure_txt = failure_txt
self.duration = (datetime.now() - self.date).total_seconds()
class BugPathEvent(Base):
__tablename__ = 'bug_path_events'
id = Column(Integer, autoincrement=True, primary_key=True)
line_begin = Column(Integer)
col_begin = Column(Integer)
line_end = Column(Integer)
col_end = Column(Integer)
msg = Column(String)
file_id = Column(Integer, ForeignKey('files.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), index = True)
next = Column(Integer)
prev = Column(Integer)
def __init__(self, line_begin, col_begin, line_end, col_end, msg, file_id):
self.line_begin, self.col_begin, self.line_end, self.col_end, self.msg = \
line_begin, col_begin, line_end, col_end, msg
self.file_id = file_id
def addPrev(self, prev):
self.prev = prev
def addNext(self, next):
self.next = next
def isLast(self):
return self.next is None
def isFirst(self):
return self.prev is None
class BugReportPoint(Base):
__tablename__ = 'bug_report_points'
id = Column(Integer, autoincrement=True, primary_key=True)
line_begin = Column(Integer)
col_begin = Column(Integer)
line_end = Column(Integer)
col_end = Column(Integer)
file_id = Column(Integer, ForeignKey('files.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), index = True)
# TODO: Add check, the value must be an existing id or null.
# Be careful when inserting.
next = Column(Integer)
def __init__(self, line_begin, col_begin, line_end, col_end, file_id):
self.line_begin, self.col_begin, self.line_end, self.col_end = \
line_begin, col_begin, line_end, col_end
self.file_id = file_id
def addNext(self, next):
self.next = next
def isLast(self):
return self.next is None
class Report(Base):
__tablename__ = 'reports'
id = Column(Integer, autoincrement=True, primary_key=True)
file_id = Column(Integer, ForeignKey('files.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'))
run_id = Column(Integer, ForeignKey('runs.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), index = True)
bug_id = Column(String, index = True)
checker_id = Column(String)
checker_cat = Column(String)
bug_type = Column(String)
severity = Column(Integer)
# TODO: multiple messages to multiple source locations?
checker_message = Column(String)
start_bugpoint = Column(Integer, ForeignKey('bug_report_points.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'))
start_bugevent = Column(Integer, ForeignKey('bug_path_events.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), index = True)
end_bugevent = Column(Integer, ForeignKey('bug_path_events.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), index = True)
suppressed = Column(Boolean)
# Cascade delete might remove rows SQLAlchemy warns about this
# to remove warnings about already deleted items set this to False.
__mapper_args__ = {
'confirm_deleted_rows' : False
}
# Priority/severity etc...
def __init__(self, run_id, bug_id, file_id, checker_message, start_bugpoint, start_bugevent, end_bugevent, checker_id, checker_cat, bug_type, severity, suppressed):
self.run_id = run_id
self.file_id = file_id
self.bug_id, self.checker_message = bug_id, checker_message
self.start_bugpoint = start_bugpoint
self.start_bugevent = start_bugevent
self.end_bugevent = end_bugevent
self.severity = severity
self.checker_id, self.checker_cat, self.bug_type = checker_id, checker_cat, bug_type
self.suppressed = suppressed
class ReportsToBuildActions(Base):
__tablename__ = 'reports_to_build_actions'
report_id = Column(Integer, ForeignKey('reports.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), primary_key=True)
build_action_id = Column(
Integer, ForeignKey('build_actions.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), primary_key=True)
def __init__(self, report_id, build_action_id):
self.report_id = report_id
self.build_action_id = build_action_id
class SuppressBug(Base):
__tablename__ = 'suppress_bug'
id = Column(Integer, autoincrement=True, primary_key=True)
hash = Column(String, nullable=False)
file_name = Column(String)
run_id = Column(Integer, ForeignKey('runs.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), nullable=False)
comment = Column(Binary)
def __init__(self, run_id, hash, file_name, comment):
self.hash, self.run_id = hash, run_id
self.comment = comment
self.file_name = file_name
class SkipPath(Base):
__tablename__ = 'skip_path'
id = Column(Integer, autoincrement=True, primary_key=True)
path = Column(String)
run_id = Column(Integer, ForeignKey('runs.id', deferrable = True, initially = "DEFERRED", ondelete='CASCADE'), nullable=False)
comment = Column(Binary)
def __init__(self, run_id, path, comment):
self.path = path
self.run_id = run_id
self.comment = comment
# End of ORM classes.
def CreateSchema(engine):
""" Creates the schema if it does not exists.
Do not check version or do migration yet. """
Base.metadata.create_all(engine)
def CreateSession(engine):
""" Creates a scoped session factory that can act like a session.
The factory uses a thread_local registry, so every thread have
its own session. """
SessionFactory = scoped_session(sessionmaker(bind=engine))
return SessionFactory

tools/codechecker/libcodechecker/pgpass.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
Parser for PostgreSQL libpq password file.
'''
def _match_field(line, field):
'''
Return the remaining part of the line in case of matching field,
otherwise it returns None
'''
if line is None or len(line) < 2:
return None
if line[0] == '*' and line[1] == ':':
# match (star)
return line[2:]
escaped = False
while len(line) > 0:
if not escaped and line[0] == '\\':
line = line[1:]
escaped = True
if len(line) == 0:
return None
if not escaped and line[0] == ':' and len(field) == 0:
# match
return line[1:]
escaped = False
if len(field) == 0:
return None
elif field[0] == line[0]:
line = line[1:]
field = field[1:]
else:
return None
return None
def _match_line(line, hostname, port, database, username):
'''
Tries to match the given line to the given hostname, port, database, and username.
Returns non None on match.
'''
pw = _match_field(line, hostname)
pw = _match_field(pw, port)
pw = _match_field(pw, database)
pw = _match_field(pw, username)
if pw is None:
return pw
# the password is still escaped
escaped = False
password = ''
for c in pw:
if not escaped and c == '\\':
escaped = True
continue
escaped = False
password += c
return password
def get_password_from_file(passfile_path, hostname, port, database, username):
'''
Parser for PostgreSQL libpq password file.
Returns None on no matching entry, otherwise it returns the password.
For file format see http://www.postgresql.org/docs/current/static/libpq-pgpass.html
'''
if len(hostname) == 0 or len(port) == 0 or len(database) == 0 or len(username) == 0:
return None
with open(passfile_path, 'r') as passfile:
for line in passfile:
pw = _match_line(line.strip(), hostname, port, database, username)
if pw:
return pw
return None

tools/codechecker/libcodechecker/plist_helper.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
''' This file contains some workaround for codechecker
to work with older clang versions. It is for demonstration purposes only.
The names, hashes will change after switching to a newer clang version.
'''
import hashlib
import linecache
import os
import re
from libcodechecker import logger
LOG = logger.get_new_logger('PLIST_HELPER')
def gen_bug_hash(bug):
line_content = linecache.getline(bug.file_path, bug.from_line)
if line_content == '' and not os.path.isfile(bug.file_path):
LOG.debug('%s does not exists!' % bug.file_path)
file_name = os.path.basename(bug.file_path)
l = [file_name, bug.checker_name, bug.msg, line_content,
str(bug.from_col), str(bug.until_col)]
for p in bug.paths():
l.append(str(p.start_pos.col))
l.append(str(p.end_pos.col))
string_to_hash = '|||'.join(l)
return hashlib.md5(string_to_hash.encode()).hexdigest()
def levenshtein(a, b): # http://hetland.org/coding/python/levenshtein.py
"Calculates the Levenshtein distance between a and b."
n, m = len(a), len(b)
if n > m:
# Make sure n <= m, to use O(min(n,m)) space
a, b = b, a
n, m = m, n
current = range(n+1)
for i in range(1, m+1):
previous, current = current, [i]+[0]*n
for j in range(1, n+1):
add, delete = previous[j]+1, current[j-1]+1
change = previous[j-1]
if a[j-1] != b[i-1]:
change = change + 1
current[j] = min(add, delete, change)
return current[n]
def get_check_name(current_msg):
# clean message from variable and class name
clean_msg = re.sub(r"'.*?'", '', current_msg)
closest_msg = ''
min_dist = len(clean_msg) // 4
for msg in checker_message_map.keys():
tmp_dist = levenshtein(clean_msg, msg)
if tmp_dist < min_dist:
closest_msg = msg
min_dist = tmp_dist
return checker_message_map[closest_msg]
# this map needs extending
checker_message_map = \
{
"": "NOT FOUND",
"Access out-of-bound array element (buffer overflow)": "alpha.security.ArrayBound",
"Address of stack memory associated with local variable returned to caller": "core.StackAddressEscape",
"Argument to is uninitialized": "core.CallAndMessage",
"Argument to free() is the address of the global variable , which is not memory allocated by malloc()": "alpha.unix.MallocWithAnnotations",
"Argument to free() is the address of the local variable , which is not memory allocated by malloc()": "alpha.unix.MallocWithAnnotations",
"Assigned value is garbage or undefined": "core.uninitialized.Assign",
"Attempt to free released memory": "alpha.unix.MallocWithAnnotations",
"Branch condition evaluates to a garbage value": "core.uninitialized.Branch",
"Call to function is extremely insecure as it can always result in a buffer overflow": "security.insecureAPI.gets",
"Call to function is insecure as it always creates or uses insecure temporary file. Use instead": "security.insecureAPI.mktemp",
"Call to function is insecure as it does not provide bounding of the memory buffer. Replace unbounded copy functions with analogous functions that support length arguments such as . CWE-119": "security.insecureAPI.strcpy",
"Cast a region whose size is not a multiple of the destination type size": "alpha.core.CastSize",
"Dereference of null pointer (loaded from variable )": "core.NullDereference",
"Dereference of undefined pointer value": "core.NullDereference",
"Division by zero": "core.DivideZero",
"Function call argument is a pointer to uninitialized value": "core.CallAndMessage",
"Function call argument is an uninitialized value": "core.CallAndMessage",
"identical expressions on both sides of logical operator": "alpha.core.IdenticalExpr",
"Memory allocated by should be deallocated by , not ": "unix.MismatchedDeallocator",
"Memory allocated by should be deallocated by , not free()": "unix.MismatchedDeallocator",
"No call of chdir(\"/\") immediately after chroot": "alpha.unix.Chroot",
"Opened File never closed. Potential Resource leak": "alpha.unix.Stream",
"Potential leak of memory pointed to by ": "alpha.unix.MallocWithAnnotations",
"Potential leak of memory pointed to by ": "cplusplus.NewDelete",
"Potential memory leak": "cplusplus.NewDelete",
"Result of is converted to a pointer of type , which is incompatible with sizeof operand type ": "unix.MallocSizeof",
"Size argument is greater than the length of the destination buffer": "alpha.unix.cstring.OutOfBounds",
"The code calls sizeof() on a pointer type. This can produce an unexpected result": "alpha.core.SizeofPtr",
"the computation of the size of the memory allocation may overflow": "alpha.security.MallocOverflow",
"The left operand of is a garbage value": "core.UndefinedBinaryOperatorResult",
"The right operand of is a garbage value": "core.UndefinedBinaryOperatorResult",
"This statement is never executed": "alpha.deadcode.UnreachableCode",
"Undefined or garbage value returned to caller": "core.uninitialized.UndefReturn",
"Use of memory after it is freed": "alpha.unix.MallocWithAnnotations",
"Value stored to during its initialization is never read": "deadcode.DeadStores",
"Value stored to is never read": "deadcode.DeadStores"
}

tools/codechecker/libcodechecker/plist_parser.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
''''''
# FIXME: This file contains some workarounds.
# Remove them as soon as a proper clang version comes out.
import plistlib
from xml.parsers.expat import ExpatError
import plist_helper
from libcodechecker import logger
LOG = logger.get_new_logger('PLIST_PARSER')
class GenericEquality(object):
def __eq__(self, other):
return (isinstance(other, self.__class__) and
self.__dict__ == other.__dict__)
def __ne__(self, other):
return not self.__eq__(other)
# -----------------------------------------------------------------------------
class Position(GenericEquality):
'''Represent a postion.'''
def __init__(self, x, y, f):
self.line = x
self.col = y
self.file_path = f
# -----------------------------------------------------------------------------
class Range(GenericEquality):
'''Represent a location in the bug path.'''
def __init__(self, start_pos, end_pos, msg=''):
self.start_pos = start_pos
self.end_pos = end_pos
self.msg = msg
# -----------------------------------------------------------------------------
def make_position(pos_map, files):
return Position(pos_map.line, pos_map.col, files[pos_map.file])
# -----------------------------------------------------------------------------
def make_range(array, files):
if len(array) == 2:
start = make_position(array[0], files)
end = make_position(array[1], files)
return Range(start, end)
# -----------------------------------------------------------------------------
class Bug(object):
'''The bug with all information, it include bugpath too.'''
def __init__(self, file, from_pos, until_pos=None, msg=None, category=None,
type=None, hash_value=''):
self.file_path = file
self.msg = msg
self._paths = []
self._events = []
self.hash_value = hash_value
if not until_pos:
until_pos = from_pos
(self.from_line, self.from_col) = from_pos
(self.until_line, self.until_col) = until_pos
def paths(self):
return self._paths
def events(self):
return self._events
def add_to_path(self, new_range):
self._paths.append(new_range)
def add_to_events(self, new_range):
self._events.append(new_range)
def get_last_path(self):
return self._paths[-1] if len(self._paths) > 0 else None
def get_last_event(self):
return self._events[-1] if len(self._events) > 0 else None
# -----------------------------------------------------------------------------
def parse_plist(path):
"""
parse the plist file
"""
bugs = []
files = []
try:
plist = plistlib.readPlist(path)
files = plist['files']
for diag in plist['diagnostics']:
current = Bug(files[diag['location']['file']],
(diag['location']['line'], diag['location']['col']))
for item in diag['path']:
if item['kind'] == 'event':
message = item['message']
if 'ranges' in item:
for arr in item['ranges']:
source_range = make_range(arr, files)
source_range.msg = message
current.add_to_events(source_range)
else:
location = make_position(item['location'], files)
source_range = Range(location, location, message)
source_range.msg = message
current.add_to_events(source_range)
elif item['kind'] == 'control':
for edge in item['edges']:
start = make_range(edge.start, files)
end = make_range(edge.end, files)
if start != current.get_last_path():
current.add_to_path(start)
current.add_to_path(end)
current.msg = diag['description']
current.category = diag['category']
current.type = diag['type']
try:
current.checker_name = diag['check_name']
except KeyError as kerr:
LOG.debug("Check name wasn't found in the plist file. "
'Read the user guide!')
current.checker_name = plist_helper.get_check_name(current.msg)
LOG.debug('Guessed check name: ' + current.checker_name)
try:
current.hash_value = diag['issue_hash_content_of_line_in_context']
except KeyError as kerr:
# hash was not found
# generate some hash for older clang versions
LOG.debug(kerr)
LOG.debug("Hash value wasn't found in the plist file. "
'Read the user guide!')
current.hash_value = plist_helper.gen_bug_hash(current)
bugs.append(current)
except ExpatError as err:
LOG.debug('Failed to process plist file: ' + path)
LOG.debug(err)
finally:
return files, bugs

tools/codechecker/libcodechecker/skiplist_handler.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
"""
"""
import re
import fnmatch
from libcodechecker import logger
LOG = logger.get_new_logger('SKIPLIST_HANDLER')
class SkipListHandler(object):
"""
Skiplist file format:
-/skip/all/source/in/directory*
-/do/not/check/this.file
+/dir/check.this.file
-/dir/*
"""
def __init__(self, skip_file):
"""
read up the skip file
"""
self.__skip = []
skip_file_content = []
with open(skip_file, 'r') as skip_file:
skip_file_content = [line.strip() for line in skip_file if line.strip() != '']
for line in skip_file_content:
if len(line) < 2 or line[0] not in ['-', '+']:
LOG.warning("Skipping malformed skipfile pattern: " + line)
continue
rexpr = re.compile(fnmatch.translate(line[1:].strip() + '*'))
self.__skip.append((line, rexpr))
def should_skip(self, source):
"""
check if the given source sourld be skipped
Should the analyzer skip the given source file?
"""
for line, rexpr in self.__skip:
if rexpr.match(source):
sign = line[0]
return sign == '-'
return False
def get_skiplist(self):
"""
Read skip file and return with its content in a list.
"""
skiplist_with_comment = {}
for line, rexpr in self.__skip:
skiplist_with_comment[line] = ''
return skiplist_with_comment

tools/codechecker/libcodechecker/storage_server/__init__.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.

tools/codechecker/libcodechecker/storage_server/report_server.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
from __future__ import print_function
from __future__ import unicode_literals
import sys
import os
import datetime
import socket
import errno
import ntpath
import sqlalchemy
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol
from thrift.server import TServer
from codechecker_gen.DBThriftAPI import CheckerReport
from codechecker_gen.DBThriftAPI.ttypes import *
import shared
from libcodechecker.db_model.orm_model import *
from libcodechecker import logger
from libcodechecker import decorators
from libcodechecker import database_handler
LOG = logger.get_new_logger('CC SERVER')
if os.environ.get('CODECHECKER_ALCHEMY_LOG') is not None:
import logging
logging.basicConfig()
logging.getLogger('sqlalchemy.engine').setLevel(logging.DEBUG)
logging.getLogger('sqlalchemy.orm').setLevel(logging.DEBUG)
class CheckerReportHandler(object):
'''
Class to handle requests from the codechecker script to store run
information to the database.
'''
def __sequence_deleter(self, table, first_id):
'''Delete points of sequnce in a general way.'''
next_id = first_id
while next_id:
item = self.session.query(table).get(next_id)
if item:
next_id = item.next
self.session.delete(item)
else:
break
def __del_source_file_for_report(self, run_id, report_id, report_file_id):
'''
Delete the stored file if there are no report references to it
in the database.
'''
report_reference_to_file = self.session.query(Report) \
.filter(
and_(Report.run_id == run_id,
Report.file_id == report_file_id,
Report.id != report_id))
rep_ref_count = report_reference_to_file.count()
if rep_ref_count == 0:
LOG.debug("No other references to the source file \n id: " +
str(report_file_id) + " can be deleted.")
# There are no other references to the file, it can be deleted.
self.session.query(File).filter(File.id == report_file_id)\
.delete()
return rep_ref_count
def __del_buildaction_results(self, build_action_id, run_id):
"""
Delete the build action and related analysis results from the database.
Report entry will be deleted by ReportsToBuildActions cascade delete.
"""
LOG.debug("Cleaning old buildactions")
try:
rep_to_ba = self.session.query(ReportsToBuildActions) \
.filter(ReportsToBuildActions.build_action_id ==
build_action_id)
reports_to_delete = [r.report_id for r in rep_to_ba]
LOG.debug("Trying to delete reports belonging to the buildaction:")
LOG.debug(reports_to_delete)
for report_id in reports_to_delete:
# Check if there is another reference to this report from
# other buildactions.
other_reference = self.session.query(ReportsToBuildActions) \
.filter(
and_(ReportsToBuildActions.report_id == report_id,
ReportsToBuildActions.build_action_id != build_action_id))
LOG.debug("Checking report id:" + str(report_id))
LOG.debug("Report id " + str(report_id) +
" reference count: " +
str(other_reference.count()))
if other_reference.count() == 0:
# There is no other reference, data related to the report
# can be deleted.
report = self.session.query(Report).get(report_id)
LOG.debug("Removing bug path events")
self.__sequence_deleter(BugPathEvent, report.start_bugevent)
LOG.debug("Removing bug report points")
self.__sequence_deleter(BugReportPoint, report.start_bugpoint)
if self.__del_source_file_for_report(run_id, report.id, report.file_id):
LOG.debug("Stored source file needs to be kept, there is reference to it from another report.")
# report needs to be deleted if there is no reference the
# file cascade delete will remove it
# else manual cleanup is needed
self.session.delete(report)
self.session.query(BuildAction).filter(BuildAction.id == build_action_id)\
.delete()
self.session.query(ReportsToBuildActions).filter(
ReportsToBuildActions.build_action_id == build_action_id).delete()
except Exception as ex:
raise shared.ttypes.RequestFailed(
shared.ttypes.ErrorCode.GENERAL,
str(ex))
@decorators.catch_sqlalchemy
def addCheckerRun(self, command, name, version, force):
'''
Store checker run related data to the database.
By default updates the results if name already exists.
Using the force flag removes existing analysis results for a run.
'''
run = self.session.query(Run).filter(Run.name == name).first()
if run and force:
# Clean already collected results.
if not run.can_delete:
# Deletion is already in progress.
msg = "Can't delete " + str(run.id)
LOG.debug(msg)
raise shared.ttypes.RequestFailed(
shared.ttypes.ErrorCode.DATABASE,
msg)
LOG.info('Removing previous analisys results ...')
self.session.delete(run)
self.session.commit()
checker_run = Run(name, version, command)
self.session.add(checker_run)
self.session.commit()
return checker_run.id
elif run:
# There is already a run, update the results.
run.date = datetime.now()
# Increment update counter.
run.inc_count += 1
self.session.commit()
return run.id
else:
# There is no run create new.
checker_run = Run(name, version, command)
self.session.add(checker_run)
self.session.commit()
return checker_run.id
@decorators.catch_sqlalchemy
def finishCheckerRun(self, run_id):
'''
'''
run = self.session.query(Run).get(run_id)
if not run:
return False
run.mark_finished()
self.session.commit()
return True
@decorators.catch_sqlalchemy
def replaceConfigInfo(self, run_id, config_values):
'''
Removes all the previously stored config informations
and stores the new values.
'''
count = self.session.query(Config) \
.filter(Config.run_id == run_id) \
.delete()
LOG.debug('Config: ' + str(count) + ' removed item.')
configs = [Config(
run_id, info.checker_name, info.attribute, info.value) for
info in config_values]
self.session.bulk_save_objects(configs)
self.session.commit()
return True
@decorators.catch_sqlalchemy
def addBuildAction(self,
run_id,
build_cmd,
check_cmd,
analyzer_type,
analyzed_source_file):
'''
'''
try:
build_actions = \
self.session.query(BuildAction) \
.filter(and_(BuildAction.run_id == run_id,
BuildAction.build_cmd == build_cmd,
or_(
and_(BuildAction.analyzer_type == analyzer_type,
BuildAction.analyzed_source_file == analyzed_source_file),
and_(BuildAction.analyzer_type == "",
BuildAction.analyzed_source_file == "")
)))\
.all()
if build_actions:
# Delete the already stored buildaction and analysis results.
for build_action in build_actions:
self.__del_buildaction_results(build_action.id, run_id)
self.session.commit()
action = BuildAction(run_id,
build_cmd,
check_cmd,
analyzer_type,
analyzed_source_file)
self.session.add(action)
self.session.commit()
except Exception as ex:
LOG.error(ex)
raise
return action.id
@decorators.catch_sqlalchemy
def finishBuildAction(self, action_id, failure):
'''
'''
action = self.session.query(BuildAction).get(action_id)
if action is None:
# TODO: if file is not needed update reportstobuildactions.
return False
action.mark_finished(failure)
self.session.commit()
return True
@decorators.catch_sqlalchemy
def needFileContent(self, run_id, filepath):
'''
'''
try:
f = self.session.query(File) \
.filter(and_(File.run_id == run_id,
File.filepath == filepath)) \
.one_or_none()
except Exception as ex:
raise shared.ttypes.RequestFailed(
shared.ttypes.ErrorCode.GENERAL,
str(ex))
run_inc_count = self.session.query(Run).get(run_id).inc_count
needed = False
if not f:
needed = True
f = File(run_id, filepath)
self.session.add(f)
self.session.commit()
elif f.inc_count < run_inc_count:
needed = True
f.inc_count = run_inc_count
return NeedFileResult(needed, f.id)
@decorators.catch_sqlalchemy
def addFileContent(self, id, content):
'''
'''
f = self.session.query(File).get(id)
assert f is not None
f.addContent(content)
return True
def __is_same_event_path(self, start_bugevent_id, events):
'''
Checks if the given event path is the same as the one in the
events argument.
'''
try:
# There should be at least one bug event.
point2 = self.session.query(BugPathEvent).get(start_bugevent_id)
for point1 in events:
if point1.startLine != point2.line_begin or \
point1.startCol != point2.col_begin or \
point1.endLine != point2.line_end or \
point1.endCol != point2.col_end or \
point1.msg != point2.msg or \
point1.fileId != point2.file_id:
return False
if point2.next is None:
return point1 == events[-1]
point2 = self.session.query(BugPathEvent).get(point2.next)
except Exception as ex:
raise shared.ttypes.RequestFailed(
shared.ttypes.ErrorCode.GENERAL,
str(ex))
def storeReportInfo(self,
action,
file_id,
bug_hash,
msg,
bugpath,
events,
checker_id,
checker_cat,
bug_type,
severity,
suppressed = False):
'''
'''
try:
path_ids = self.storeBugPath(bugpath)
event_ids = self.storeBugEvents(events)
path_start = path_ids[0].id if len(path_ids) > 0 else None
source_file = self.session.query(File).get(file_id)
source_file_path, source_file_name = ntpath.split(source_file.filepath)
# Old suppress format did not contain file name.
suppressed = self.session.query(SuppressBug) \
.filter(
and_(SuppressBug.run_id == action.run_id,
SuppressBug.hash == bug_hash,
or_(SuppressBug.file_name == source_file_name,
SuppressBug.file_name == u''))) \
.count() > 0
report = Report(action.run_id,
bug_hash,
file_id,
msg,
path_start,
event_ids[0].id,
event_ids[-1].id,
checker_id,
checker_cat,
bug_type,
severity,
suppressed)
self.session.add(report)
self.session.commit()
# Commit required to get the ID of the newly added report.
reportToActions = ReportsToBuildActions(report.id, action.id)
self.session.add(reportToActions)
# Avoid data loss for duplicate keys.
self.session.commit()
return report.id
except Exception as ex:
raise shared.ttypes.RequestFailed(
shared.ttypes.ErrorCode.GENERAL,
str(ex))
@decorators.catch_sqlalchemy
def addReport(self,
build_action_id,
file_id,
bug_hash,
msg,
bugpath,
events,
checker_id,
checker_cat,
bug_type,
severity,
suppress):
'''
'''
try:
action = self.session.query(BuildAction).get(build_action_id)
assert action is not None
# TODO: perfomance issues when executing the following query on large
# databaseses?
reports = self.session.query(self.report_ident) \
.filter(and_(self.report_ident.c.bug_id == bug_hash,
self.report_ident.c.run_id == action.run_id))
try:
# Check for duplicates by bug hash.
if reports.count() != 0:
for possib_dup in reports:
# It's a duplicate or a hash clash. Check checker name,
# file id, and position.
dup_report_obj = self.session.query(Report).get(
possib_dup.report_ident.id)
if dup_report_obj and dup_report_obj.checker_id == checker_id and \
dup_report_obj.file_id == file_id and \
self.__is_same_event_path(dup_report_obj.start_bugevent, events):
# It's a duplicate.
rtp = self.session.query(ReportsToBuildActions) \
.get((dup_report_obj.id,
action.id))
if not rtp:
reportToActions = ReportsToBuildActions(
dup_report_obj.id, action.id)
self.session.add(reportToActions)
self.session.commit()
return dup_report_obj.id
return self.storeReportInfo(action,
file_id,
bug_hash,
msg,
bugpath,
events,
checker_id,
checker_cat,
bug_type,
severity,
suppress)
except sqlalchemy.exc.IntegrityError as ex:
self.session.rollback()
reports = self.session.query(self.report_ident) \
.filter(and_(self.report_ident.c.bug_id == bug_hash,
self.report_ident.c.run_id == action.run_id))
if reports.count() != 0:
return reports.first().report_ident.id
else:
raise
except Exception as ex:
raise shared.ttypes.RequestFailed(
shared.ttypes.ErrorCode.GENERAL,
str(ex))
def storeBugEvents(self, bugevents):
'''
'''
events = []
for event in bugevents:
bpe = BugPathEvent(event.startLine,
event.startCol,
event.endLine,
event.endCol,
event.msg,
event.fileId)
self.session.add(bpe)
events.append(bpe)
self.session.flush()
if len(events) > 1:
for i in xrange(len(events)-1):
events[i].addNext(events[i+1].id)
events[i+1].addPrev(events[i].id)
events[-1].addPrev(events[-2].id)
return events
def storeBugPath(self, bugpath):
paths = []
for i in xrange(len(bugpath)):
brp = BugReportPoint(bugpath[i].startLine,
bugpath[i].startCol,
bugpath[i].endLine,
bugpath[i].endCol,
bugpath[i].fileId)
self.session.add(brp)
paths.append(brp)
self.session.flush()
for i in xrange(len(paths)-1):
paths[i].addNext(paths[i+1].id)
return paths
@decorators.catch_sqlalchemy
def addSuppressBug(self, run_id, bugs_to_suppress):
'''
Supppress multiple bugs for a run. This can be used before storing
the suppress file content.
'''
try:
suppressList = []
for bug_to_suppress in bugs_to_suppress:
suppress_bug = SuppressBug(run_id,
bug_to_suppress.bug_hash,
bug_to_suppress.file_name,
bug_to_suppress.comment)
suppressList.append(suppress_bug)
self.session.bulk_save_objects(suppressList)
self.session.commit()
except Exception as ex:
LOG.error(str(ex))
return False
return True
@decorators.catch_sqlalchemy
def cleanSuppressData(self, run_id):
'''
Clean the suppress bug entries for a run
and remove suppressed flags for the suppressed reports.
Only the database is modified.
'''
try:
count = self.session.query(SuppressBug) \
.filter(SuppressBug.run_id == run_id) \
.delete()
LOG.debug('Cleaning previous suppress entries from the database. '
+ str(count) + ' removed items.')
reports = self.session.query(Report) \
.filter(and_(Report.run_id == run_id,
Report.suppressed == True)) \
.all()
for report in reports:
report.suppressed = False
self.session.commit()
except Exception as ex:
LOG.error(str(ex))
return False
return True
@decorators.catch_sqlalchemy
def addSkipPath(self, run_id, paths):
'''
'''
count = self.session.query(SkipPath) \
.filter(SkipPath.run_id == run_id) \
.delete()
LOG.debug('SkipPath: ' + str(count) + ' removed item.')
skipPathList = []
for path, comment in paths.items():
skipPath = SkipPath(run_id, path, comment)
skipPathList.append(skipPath)
self.session.bulk_save_objects(skipPathList)
self.session.commit()
return True
@decorators.catch_sqlalchemy
def stopServer(self):
'''
'''
self.session.commit()
sys.exit(0)
def __init__(self, session, lockDB):
self.session = session
self.report_ident = sqlalchemy.orm.query.Bundle('report_ident',
Report.id,
Report.bug_id,
Report.run_id,
Report.start_bugevent,
Report.start_bugpoint)
def run_server(port, db_uri, db_version_info, callback_event=None):
LOG.debug('Starting codechecker server ...')
try:
engine = database_handler.SQLServer.create_engine(db_uri)
LOG.debug('Creating new database session')
session = CreateSession(engine)
except sqlalchemy.exc.SQLAlchemyError as alch_err:
LOG.error(str(alch_err))
sys.exit(1)
session.autoflush = False # Autoflush is enabled by default.
LOG.debug('Starting thrift server')
try:
# Start thrift server.
handler = CheckerReportHandler(session, True)
processor = CheckerReport.Processor(handler)
transport = TSocket.TServerSocket(port=port)
tfactory = TTransport.TBufferedTransportFactory()
pfactory = TBinaryProtocol.TBinaryProtocolFactory()
server = TServer.TThreadPoolServer(processor,
transport,
tfactory,
pfactory,
daemon=True)
LOG.info('Waiting for check results on ['+str(port)+']')
if callback_event:
callback_event.set()
LOG.debug('Starting to serve')
server.serve()
session.commit()
except socket.error as sockerr:
LOG.error(str(sockerr))
if sockerr.errno == errno.EADDRINUSE:
LOG.error('Checker port '+str(port)+' is already used')
sys.exit(1)
except Exception as err:
LOG.error(str(err))
session.commit()
sys.exit(1)

tools/codechecker/libcodechecker/suppress_file_handler.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
''' suppress file format
# this is the old format
123324353456463442341242342343#1 || bug hash comment
# this is the new format
123324353456463442341242342343#1 || filename || bug hash comment
after removing the hash_value_type the generated format is:
123324353456463442341242342343 || filename || bug hash comment
for backward compatibility the hash_value_type is an optional filed
'''
import re
import os
import codecs
from libcodechecker import logger
LOG = logger.get_new_logger('SUPPRESS_FILE_HANDLER')
COMMENT_SEPARATOR = '||'
HASH_TYPE_SEPARATOR = '#'
def get_suppress_data(suppress_file):
"""
processs a file object for suppress information
"""
old_format_pattern = r"^(?P<bug_hash>[\d\w]{32})(\#(?P<bug_hash_type>\d))?\s*\|\|\s*(?P<comment>[^\|]*)$"
old_format = re.compile(old_format_pattern, re.UNICODE)
new_format_pattern = r"^(?P<bug_hash>[\d\w]{32})(\#(?P<bug_hash_type>\d))?\s*\|\|\s*(?P<file_name>[^\\\|]+)\s*\|\|\s*(?P<comment>[^\|]*)$"
new_format = re.compile(new_format_pattern, re.UNICODE)
suppress_data = []
for line in suppress_file:
new_format_match = re.match(new_format, line.strip())
if new_format_match:
LOG.debug('Match for new suppress entry format:')
new_format_match = new_format_match.groupdict()
LOG.debug(new_format_match)
suppress_data.append((new_format_match['bug_hash'],
new_format_match['file_name'],
new_format_match['comment']))
continue
old_format_match = re.match(old_format, line.strip())
if old_format_match:
LOG.debug('Match for old suppress entry format:')
old_format_match = old_format_match.groupdict()
LOG.debug(old_format_match)
suppress_data.append((old_format_match['bug_hash'],
u'', # empty file name
old_format_match['comment']))
continue
if line.strip() != '':
LOG.warning('Malformed suppress line: ' + line)
return suppress_data
# ---------------------------------------------------------------------------
def write_to_suppress_file(suppress_file, value, file_name, comment=''):
comment = comment.decode('UTF-8')
LOG.debug('Processing suppress file: '+suppress_file)
try:
with codecs.open(suppress_file, 'r', 'UTF-8') as s_file:
suppress_data = get_suppress_data(s_file)
if not os.stat(suppress_file)[6] == 0:
# file is not empty
res = filter(lambda x: (x[0] == value and x[1] == file_name) or (x[0] == value and x[1] == ''), suppress_data)
if res:
LOG.debug("Already found in\n %s" % (suppress_file))
return True
s_file = codecs.open(suppress_file, 'a', 'UTF-8')
s_file.write(value+COMMENT_SEPARATOR+file_name+COMMENT_SEPARATOR+comment+'\n')
s_file.close()
return True
except Exception as ex:
LOG.error(str(ex))
LOG.error("Failed to write: %s" % (suppress_file))
return False
def remove_from_suppress_file(suppress_file, value, file_name):
"""
remove suppress information from the suppress file
old and new format is supported
"""
LOG.debug('Removing ' + value + ' from \n' + suppress_file)
try:
s_file = codecs.open(suppress_file, 'r+', 'UTF-8')
lines = s_file.readlines()
# filter out new format first because it is more specific
old_format_pattern = r"^" + value + r"(\#\d)?\s*\|\|\s*(?P<comment>[^\|]*)$"
old_format = re.compile(old_format_pattern, re.UNICODE)
new_format_pattern = r"^" + value + r"(\#d)?\s*\|\|\s*" + file_name + r"\s*\|\|\s*(?P<comment>[^\|]*)$"
new_format = re.compile(new_format_pattern, re.UNICODE)
def check_for_match(line):
"""
check if the line matches the new or old format
"""
line = line.strip()
if re.match(new_format, line.strip()):
return False
if re.match(old_format, line.strip()):
return False
else:
return True
# filter out lines which should be removed
lines = filter(lambda line: check_for_match(line), lines)
s_file.seek(0)
s_file.truncate()
s_file.writelines(lines)
s_file.close()
return True
except Exception as ex:
LOG.error(str(ex))
LOG.error("Failed to write: %s" % (suppress_file))
return False

tools/codechecker/libcodechecker/suppress_handler.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
suppress handling
'''
import re
import abc
import linecache
from libcodechecker import logger
LOG = logger.get_new_logger('SUPPRESS HANDLER')
class SuppressHandler(object):
""" suppress handler base class """
__metaclass__ = abc.ABCMeta
__suppressfile = None
@abc.abstractmethod
def store_suppress_bug_id(self,
source_file_path,
bug_id,
file_name,
comment):
""" store the suppress bug_id """
pass
@abc.abstractmethod
def remove_suppress_bug_id(self,
bug_id,
file_name):
""" remove the suppress bug_id """
pass
@property
def suppress_file(self):
"""" file on the filesystem where the suppress
data will be written """
return self.__suppressfile
@suppress_file.setter
def suppress_file(self, value):
""" set the suppress file"""
self.__suppressfile = value
class SourceSuppressHandler(object):
"""
handle bug suppression in the source
"""
suppress_marker = 'codechecker_suppress'
def __init__(self, source_file, bug_line):
"""
source line number indexing starts at 1
"""
self.__source_file = source_file
self.__bug_line = bug_line
self.__suppressed_checkers = []
self.__suppress_comment = None
def __check_if_comment(self, line):
"""
check if the line is a comment
accepted comment format is only if line starts with '//'
"""
return line.strip().startswith('//')
def __process_suppress_info(self, source_section):
"""
return true if suppress comment found and matches the
required format
Accepted source suppress format only above the bug line no
empty lines are accepted between the comment and the bug line
For suppressing all checker results:
// codechecker_suppress [all] some multi line
// comment
For suppressing some specific checker results:
// codechecker_suppress [checker.name1, checker.name2] some
// multi line comment
"""
nocomment = source_section.replace('//', '')
# remove extra spaces if any
formatted = ' '.join(nocomment.split())
# check for codechecker suppress comment
pattern = r'^\s*codechecker_suppress\s*\[\s*(?P<checkers>(.*))\s*\]\s*(?P<comment>.*)$'
ptn = re.compile(pattern)
res = re.match(ptn, formatted)
if res:
checkers = res.group('checkers')
if checkers == "all":
self.__suppressed_checkers.append('all')
else:
suppress_checker_list = re.findall(r"[^,\s]+", checkers.strip())
self.__suppressed_checkers.extend(suppress_checker_list)
comment = res.group('comment')
if comment == '':
self.__suppress_comment = "WARNING! suppress comment is missing"
else:
self.__suppress_comment = res.group('comment')
return True
else:
return False
def check_source_suppress(self):
"""
return true if there is a suppress comment or false if not
"""
source_file = self.__source_file
LOG.debug('Checking for suppress comment in the source file: '
+ self.__source_file)
previous_line_num = self.__bug_line - 1
suppression_result = False
if previous_line_num > 0:
marker_found = False
comment_line = True
collected_lines = []
while (not marker_found and comment_line):
source_line = linecache.getline(source_file, previous_line_num)
if(self.__check_if_comment(source_line)):
# it is a comment
if self.suppress_marker in source_line:
# found the marker
collected_lines.append(source_line.strip())
marker_found = True
break
else:
collected_lines.append(source_line.strip())
comment_line = True
else:
# this is not a comment
comment_line = False
break
if(previous_line_num > 0):
previous_line_num -= 1
else:
break
# collected comment lines upward from bug line
rev = list(reversed(collected_lines))
if marker_found:
suppression_result = self.__process_suppress_info(''.join(rev))
LOG.debug('Suppress comment found: ' + str(suppression_result))
return suppression_result
def suppressed_checkers(self):
"""
get the suppressed checkers list
"""
return self.__suppressed_checkers
def suppress_comment(self):
"""
get the suppress comment
"""
return self.__suppress_comment

tools/codechecker/libcodechecker/tidy_output_converter.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
This module is responsible for parsing clang-tidy output and generating plist
for the plist_parser module.
'''
import re
import os
import copy
import plistlib
from libcodechecker import logger
LOG = logger.get_new_logger('TIDY_OUTPUT_HANDLER')
class Note(object):
'''
Represents a note and also this is the base class of Message.
'''
def __init__(self, path, line, column, message):
self.path = path
self.line = line
self.column = column
self.message = message
def __eq__(self, other):
return self.path == other.path and \
self.line == other.line and \
self.column == other.column and \
self.message == other.message
def __str__(self):
return ('path=%s, line=%d, column=%s, message=%s') % \
(self.path, self.line, self.column, self.message)
class Message(Note):
'''
Represents a clang-tidy message with an optional fixit message.
'''
def __init__(self, path, line, column, message, checker, fixits=None, notes=None):
super(Message, self).__init__(path, line, column, message)
self.checker = checker
self.fixits = fixits if fixits else []
self.notes = notes if notes else []
def __eq__(self, other):
return super(Message, self).__eq__(other) and \
self.checker == other.checker and \
self.fixits == other.fixits and \
self.notes == other.notes
def __str__(self):
return ('%s, checker=%s, fixits=%s, notes=%s') % \
(super(Message, self).__str__(), self.checker,
[str(fixit) for fixit in self.fixits],
[str(note) for note in self.notes])
class OutputParser(object):
'''
Parser for clang-tidy console output.
'''
# Regex for parsing a clang-tidy message
message_line_re = re.compile(
# File path followed by a ':'
'^(?P<path>[\S]+):'
# Line number followed by a ':'
'(?P<line>\d+):'
# Column number followed by a ':' and a space
'(?P<column>\d+):\ '
# Severity followed by a ':'
'(?P<severity>\w+):'
# Checker message
'(?P<message>[\S \t]+)\s*'
# Checker name
'\[(?P<checker>.*)\]')
# Matches a note
note_line_re = re.compile(
# File path followed by a ':'
'^(?P<path>[\S]+):'
# Line number followed by a ':'
'(?P<line>\d+):'
# Column number followed by a ':' and a space
'(?P<column>\d+):\ '
# Severity == note
'note:'
# Checker message
'(?P<message>.*)')
def __init__(self):
self.messages = []
def parse_messages_from_file(self, path):
'''
Parse clang-tidy output dump (redirected output).
'''
with open(path, 'r') as file:
return self.parse_messages(file)
def parse_messages(self, tidy_out):
'''
Parse the given clang-tidy output. This method calls iter(tidy_out).
The iterator should return lines.
Parameters:
tidy_out: something iterable (e.g.: a file object)
'''
titer = iter(tidy_out)
try:
next_line = titer.next()
while True:
message, next_line = self._parse_message(titer, next_line)
if message != None:
self.messages.append(message)
except StopIteration:
pass
return self.messages
def _parse_message(self, titer, line):
'''
Parse the given line. Returns a (message, next_line) pair or throws a
StopIteration. The message could be None.
Parameters:
titer: clang-tidy output iterator
line: the current line
'''
match = OutputParser.message_line_re.match(line)
if match is None:
return None, titer.next()
message = Message(
os.path.abspath(match.group('path')),
int(match.group('line')),
int(match.group('column')),
match.group('message').strip(),
match.group('checker').strip())
try:
line = titer.next()
line = self._parse_code(message, titer, line)
line = self._parse_fixits(message, titer, line)
line = self._parse_notes(message, titer, line)
return message, line
except StopIteration:
return message, line
def _parse_code(self, message, titer, line):
# eat code line
if OutputParser.note_line_re.match(line) or \
OutputParser.message_line_re.match(line):
LOG.debug("Unexpected line: %s. Expected a code line!" % line)
return line
# eat arrow line
# FIXME: range support?
line = titer.next()
if '^' not in line:
LOG.debug("Unexpected line: %s. Expected an arrow line!" % line)
return line
return titer.next()
def _parse_fixits(self, message, titer, line):
'''Parses fixit messages'''
while OutputParser.message_line_re.match(line) is None and \
OutputParser.note_line_re.match(line) is None:
message_text = line.strip()
if message_text == '':
continue
message.fixits.append(Note(message.path, message.line,
line.find(message_text) + 1, message_text))
line = titer.next()
return line
def _parse_notes(self, message, titer, line):
'''Parses note messages'''
while OutputParser.message_line_re.match(line) is None:
match = OutputParser.note_line_re.match(line)
if match is None:
LOG.debug("Unexpected line: %s" % line)
return titer.next()
message.notes.append(Note(os.path.abspath(match.group('path')),
int(match.group('line')),
int(match.group('column')),
match.group('message').strip()))
line = titer.next()
line = self._parse_code(message, titer, line)
return line
class PListConverter(object):
'''
clang-tidy messages to plist converter.
'''
def __init__(self):
self.plist = {
'files' : [],
'diagnostics' : []
}
def _add_files_from_messages(self, messages):
'''
Adds the new files from the given message array to the plist's "files"
key, and returns a path to file index dictionary.
'''
fmap = {}
for message in messages:
try:
# This file is already in the plist
idx = self.plist['files'].index(message.path)
fmap[message.path] = idx
except ValueError:
# New file
fmap[message.path] = len(self.plist['files'])
self.plist['files'].append(message.path)
# collect file paths from the message notes
for nt in message.notes:
try:
# This file is already in the plist
idx = self.plist['files'].index(nt.path)
fmap[nt.path] = idx
except ValueError:
# New file
fmap[nt.path] = len(self.plist['files'])
self.plist['files'].append(nt.path)
return fmap
def _add_diagnostics(self, messages, fmap):
'''
Adds the messages to the plist as diagnostics.
'''
for message in messages:
diag = PListConverter._create_diag(message, fmap)
self.plist['diagnostics'].append(diag)
@staticmethod
def _get_checker_category(checker):
'''
Returns the check's category.
'''
parts = checker.split('-')
if len(parts) == 0:
# I don't know if it's possible
return 'unknown'
else:
return parts[0]
@staticmethod
def _create_diag(message, fmap):
'''
Creates a new plist diagnostic from a single clang-tidy message.
'''
diag = {}
diag['location'] = PListConverter._create_location(message, fmap)
diag['check_name'] = message.checker
diag['description'] = message.message
diag['category'] = PListConverter._get_checker_category(message.checker)
diag['type'] = 'clang-tidy'
diag['path'] = [PListConverter._create_event_from_note(message, fmap)]
PListConverter._add_fixits(diag, message, fmap)
PListConverter._add_notes(diag, message, fmap)
return diag
@staticmethod
def _create_location(note, fmap):
return {
'line' : note.line,
'col' : note.column,
'file' : fmap[note.path]
}
@staticmethod
def _create_event_from_note(note, fmap):
return {
'kind' : 'event',
'location' : PListConverter._create_location(note, fmap),
'depth' : 0, # I don't know WTF is this
'message' : note.message
}
@staticmethod
def _create_edge(start_note, end_note, fmap):
start_loc = PListConverter._create_location(start_note, fmap)
end_loc = PListConverter._create_location(end_note, fmap)
return {
'start' : [start_loc, start_loc],
'end' : [end_loc, end_loc]
}
@staticmethod
def _add_fixits(diag, message, fmap):
'''
Adds fixits as events to the diagnostics.
'''
for fixit in message.fixits:
mf = copy.deepcopy(fixit)
mf.message = '%s (fixit)' % fixit.message
diag['path'].append(PListConverter._create_event_from_note(
mf, fmap))
@staticmethod
def _add_notes(diag, message, fmap):
'''
Adds notes as events to the diagnostics. It also creates edges between
the notes.
'''
edges = []
last = None
for note in message.notes:
if last is not None:
edges.append(PListConverter._create_edge(last, note, fmap))
diag['path'].append(PListConverter._create_event_from_note(
note, fmap))
last = note
diag['path'].append({
'kind' : 'control',
'edges' : edges
})
def add_messages(self, messages):
'''
Adds the given clang-tidy messages to the plist.
'''
fmap = self._add_files_from_messages(messages)
self._add_diagnostics(messages, fmap)
def write_to_file(self, path):
'''
Writes out the plist XML to the given path.
'''
with open(path, 'wb') as file:
self.write(file)
def write(self, file):
'''
Writes out the plist XML using the given file object.
'''
plistlib.writePlist(self.plist, file)

tools/codechecker/libcodechecker/util.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
util module
'''
import datetime
import os
import hashlib
import ntpath
import sys
import glob
import socket
import shutil
import subprocess
from libcodechecker import logger
# WARNING! LOG should be only used in this module
LOG = logger.get_new_logger('UTIL')
# ---------------------------------------------------------------------
def get_free_port():
''' get a free port from the os'''
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('', 0))
free_port = s.getsockname()[1]
s.close()
return free_port
# ---------------------------------------------------------------------
def is_localhost(address):
'''
Check if address is one of the valid values and try to get the
IP-addresses from the system.
'''
valid_values = ['localhost', '0.0.0.0', '*']
try:
valid_values.append(socket.gethostbyname('localhost'))
except Exception:
# failed to get ip address for localhost
pass
try:
valid_values.append(socket.gethostbyname(socket.gethostname()))
except Exception:
# failed to get ip address for host_name
pass
return address in valid_values
# ---------------------------------------------------------------------
def match_file_name(file_name, pattern):
file_name_parts = file_name.split('--')
if file_name_parts[0] == pattern:
# print('%s == %s : TRUE'% (file_name_parts[0],pattern))
return True
else:
# print('%s == %s : TRUE'% (file_name_parts[0],pattern))
return False
# ---------------------------------------------------------------------
def get_file_last_modification_time(file):
'''
Returns the last modification time of a file
'''
return datetime.datetime.fromtimestamp(os.path.getmtime(file))
# ---------------------------------------------------------------------
def get_env_var(env_var, needed=False):
'''
Read the environment variables and handle the exception if a necessary
environment variable is missing
'''
value = os.getenv(env_var)
if needed and not value:
LOG.critical('Failed to read necessary environment variable %s.'
' (Maybe codechecker was not configured properly)'
% (env_var))
sys.exit(1)
return value
# -------------------------------------------------------------------------
def get_tmp_dir_hash():
'''Generate a hash based on the current time and process id.'''
pid = os.getpid()
time = datetime.datetime.now()
data = str(pid) + str(time)
dir_hash = hashlib.md5()
dir_hash.update(data)
LOG.debug('The generated temporary directory hash is %s'
% (dir_hash.hexdigest()))
return dir_hash.hexdigest()
# -------------------------------------------------------------------------
def get_file_name_from_path(path):
'''Get the filename from a path.'''
head, tail = ntpath.split(path)
return head, tail
# -------------------------------------------------------------------------
def get_obj_target(object_file_path):
return os.path.split(os.path.abspath(dir))[-2]
# -------------------------------------------------------------------------
def create_dir(path):
'''Create a directory safely if it does not exist yet.
This may be called from several processes or threads, creating the same
directory, and it fails only if the directory is not created.
'''
if not os.path.isdir(path):
try:
LOG.debug('Creating directory %s' % (path))
os.makedirs(path)
except Exception as e:
if not os.path.isdir(path):
LOG.error('Failed to create directory %s' % (path))
raise e
return
# -------------------------------------------------------------------------
def get_file_list(path, pattern):
glob_pattern = os.path.join(path, pattern)
return glob.glob(glob_pattern)
# -------------------------------------------------------------------------
def remove_file_list(file_list):
for rfile in file_list:
LOG.debug(rfile)
try:
os.remove(rfile)
except OSError:
# maybe another thread has already deleted it
LOG.debug('Failed to remove file %s' % (rfile))
return
# -------------------------------------------------------------------------
def remove_dir(path):
def error_handler(*args):
LOG.warning('Failed to remove directory %s' % (path))
shutil.rmtree(path, onerror=error_handler)
def call_command(command, env=None):
''' Call an external command and return with (output, return_code).'''
try:
LOG.debug('Run ' + ' '.join(command))
out = subprocess.check_output(command,
bufsize=-1,
env=env,
stderr=subprocess.STDOUT)
LOG.debug(out)
return out, 0
except subprocess.CalledProcessError as ex:
LOG.debug('Running command "' + ' '.join(command) + '" Failed')
LOG.debug(str(ex.returncode))
LOG.debug(ex.output)
return ex.output, ex.returncode
def get_default_workspace():
"""
default workspace in the users home directory
"""
workspace = os.path.join(os.path.expanduser("~"), '.codechecker')
return workspace

tools/codechecker/libcodechecker/viewer_server/__init__.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.

tools/codechecker/libcodechecker/viewer_server/client_db_access_handler.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
Handle thrift requests
'''
import zlib
import os
import datetime
from collections import defaultdict
import ntpath
import codecs
import sqlalchemy
from sqlalchemy import asc, desc
from sqlalchemy.sql import or_, and_, func
from sqlalchemy.sql.expression import literal_column
from libcodechecker.db_model.orm_model import *
import shared
from codeCheckerDBAccess import constants
from codeCheckerDBAccess.ttypes import *
from libcodechecker import logger
LOG = logger.get_new_logger('ACCESS HANDLER')
# -----------------------------------------------------------------------
def timefunc(function):
'''
timer function
'''
func_name = function.__name__
def debug_wrapper(*args, **kwargs):
'''
wrapper for debug log
'''
before = datetime.now()
res = function(*args, **kwargs)
after = datetime.now()
timediff = after - before
diff = timediff.microseconds/1000
LOG.debug('['+str(diff)+'ms] ' + func_name)
return res
def release_wrapper(*args, **kwargs):
'''
no logging
'''
res = function(*args, **kwargs)
return res
if logger.get_log_level() == logger.DEBUG:
return debug_wrapper
else:
return release_wrapper
def conv(text):
'''
Convert * to % got from clients for the database queries
'''
if text is None:
return '%'
return text.replace('*', '%')
def construct_report_filter(report_filters):
'''
construct the report filter for reports and suppressed reports
'''
OR = []
if report_filters is None:
AND = []
AND.append(Report.checker_message.like('%'))
AND.append(Report.checker_id.like('%'))
AND.append(File.filepath.like('%'))
OR.append(and_(*AND))
filter_expression = or_(*OR)
return filter_expression
for report_filter in report_filters:
AND = []
if report_filter.checkerMsg:
AND.append(Report.checker_message.ilike(
conv(report_filter.checkerMsg)))
if report_filter.checkerId:
AND.append(Report.checker_id.ilike(
conv(report_filter.checkerId)))
if report_filter.filepath:
AND.append(File.filepath.ilike(
conv(report_filter.filepath)))
if report_filter.severity is not None:
# severity value can be 0
AND.append(Report.severity == report_filter.severity)
if report_filter.suppressed:
AND.append(Report.suppressed == True)
else:
AND.append(Report.suppressed == False)
OR.append(and_(*AND))
filter_expression = or_(*OR)
return filter_expression
class ThriftRequestHandler():
'''
Connect to database and handle thrift client requests
'''
def __init__(self,
session,
checker_md_docs,
checker_md_docs_map,
suppress_handler,
db_version_info):
self.__checker_md_docs = checker_md_docs
self.__checker_doc_map = checker_md_docs_map
self.__suppress_handler = suppress_handler
self.__session = session
def __queryReport(self, reportId):
session = self.__session
try:
q = session.query(Report,
File,
BugPathEvent,
SuppressBug) \
.filter(Report.id == reportId) \
.outerjoin(File,
Report.file_id == File.id) \
.outerjoin(BugPathEvent,
Report.end_bugevent == BugPathEvent.id) \
.outerjoin(SuppressBug,
SuppressBug.hash == Report.bug_id)
results = q.limit(1).all()
if len(results) < 1:
raise shared.ttypes.RequestFailed(
shared.ttypes.ErrorCode.DATABASE,
"Report " + reportId + " not found!")
report, source_file, lbpe, suppress_bug = results[0]
last_event_pos = \
shared.ttypes.BugPathEvent(
startLine=lbpe.line_begin,
startCol=lbpe.col_begin,
endLine=lbpe.line_end,
endCol=lbpe.col_end,
msg=lbpe.msg,
fileId=lbpe.file_id,
filePath=source_file.filepath)
if suppress_bug:
suppress_comment = suppress_bug.comment
else:
suppress_comment = None
return ReportData(
bugHash=report.bug_id,
checkedFile=source_file.filepath,
checkerMsg=report.checker_message,
suppressed=report.suppressed,
reportId=report.id,
fileId=source_file.id,
lastBugPosition=last_event_pos,
checkerId=report.checker_id,
severity=report.severity,
suppressComment=suppress_comment)
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(
shared.ttypes.ErrorCode.DATABASE,
msg)
def __sortResultsQuery(self, query, sort_types=None):
'''
Helper method for __queryDiffResults and __queryResults to apply sorting.
'''
# get a list of sort_types which will be a nested ORDER BY
sort_type_map = {}
sort_type_map[SortType.FILENAME] = [File.filepath, BugPathEvent.line_begin]
sort_type_map[SortType.CHECKER_NAME] = [Report.checker_id]
sort_type_map[SortType.SEVERITY] = [Report.severity]
# mapping the SQLAlchemy functions
order_type_map = {}
order_type_map[Order.ASC] = asc
order_type_map[Order.DESC] = desc
if sort_types is None:
sort_types = [SortMode(SortType.FILENAME, Order.ASC)]
for sort in sort_types:
sorttypes = sort_type_map.get(sort.type)
for sorttype in sorttypes:
order_type = order_type_map.get(sort.ord)
query = query.order_by(order_type(sorttype))
return query
def __queryResults(self, run_id, limit, offset, sort_types, report_filters):
max_query_limit = constants.MAX_QUERY_SIZE
if limit > max_query_limit:
LOG.debug('Query limit ' + str(limit) +
' was larger than max query limit ' +
str(max_query_limit) + ', setting limit to ' +
str(max_query_limit))
limit = max_query_limit
session = self.__session
filter_expression = construct_report_filter(report_filters)
try:
q = session.query(Report,
File,
BugPathEvent,
SuppressBug) \
.filter(Report.run_id == run_id) \
.outerjoin(File,
and_(Report.file_id == File.id,
File.run_id == run_id)) \
.outerjoin(BugPathEvent,
Report.end_bugevent == BugPathEvent.id) \
.outerjoin(SuppressBug,
and_(SuppressBug.hash == Report.bug_id,
SuppressBug.run_id == run_id)) \
.filter(filter_expression)
q = self.__sortResultsQuery(q, sort_types)
results = []
for report, source_file, lbpe, suppress_bug in \
q.limit(limit).offset(offset):
last_event_pos = \
shared.ttypes.BugPathEvent(startLine=lbpe.line_begin,
startCol=lbpe.col_begin,
endLine=lbpe.line_end,
endCol=lbpe.col_end,
msg=lbpe.msg,
fileId=lbpe.file_id,
filePath=source_file.filepath)
if suppress_bug:
suppress_comment = suppress_bug.comment
else:
suppress_comment = None
results.append(
ReportData(bugHash=report.bug_id,
checkedFile=source_file.filepath,
checkerMsg=report.checker_message,
suppressed=report.suppressed,
reportId=report.id,
fileId=source_file.id,
lastBugPosition=last_event_pos,
checkerId=report.checker_id,
severity=report.severity,
suppressComment=suppress_comment)
)
return results
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE, msg)
@timefunc
def getRunData(self):
session = self.__session
results = []
try:
# count the reports subquery
stmt = session.query(Report.run_id,
func.count(literal_column('*')).label('report_count')) \
.filter(Report.suppressed == False) \
.group_by(Report.run_id) \
.subquery()
q = session.query(Run, stmt.c.report_count) \
.outerjoin(stmt, Run.id == stmt.c.run_id) \
.order_by(Run.date)
for instance, reportCount in q:
if reportCount is None:
reportCount = 0
results.append(RunData(instance.id,
str(instance.date),
instance.name,
instance.duration,
reportCount,
instance.command
))
return results
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE, msg)
@timefunc
def getReport(self, reportId):
return self.__queryReport(reportId)
@timefunc
def getRunResults(self, run_id, limit, offset, sort_types, report_filters):
return self.__queryResults(run_id,
limit,
offset,
sort_types,
report_filters)
@timefunc
def getRunResultCount(self, run_id, report_filters):
filter_expression = construct_report_filter(report_filters)
session = self.__session
try:
reportCount = session.query(Report) \
.filter(Report.run_id == run_id) \
.outerjoin(File,
and_(Report.file_id == File.id,
File.run_id == run_id)) \
.outerjoin(BugPathEvent,
Report.end_bugevent == BugPathEvent.id) \
.outerjoin(SuppressBug,
and_(SuppressBug.hash == Report.bug_id,
SuppressBug.run_id == run_id)) \
.filter(filter_expression)\
.count()
if reportCount is None:
reportCount = 0
return reportCount
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE, msg)
@timefunc
def __construct_bug_event_list(self, session, start_bug_event):
file_path_cache = {}
bug_events = []
event = session.query(BugPathEvent).get(start_bug_event)
file_path = file_path_cache.get(event.file_id)
if not file_path:
f = session.query(File).get(event.file_id)
file_path = f.filepath
file_path_cache[event.file_id] = file_path
bug_events.append((event, file_path))
while event.next is not None:
event = session.query(BugPathEvent).get(event.next)
file_path = file_path_cache.get(event.file_id)
if not file_path:
f = session.query(File).get(event.file_id)
file_path = f.filepath
file_path_cache[event.file_id] = file_path
bug_events.append((event, file_path))
return bug_events
@timefunc
def __construct_bug_point_list(self, session, start_bug_point):
# start_bug_point can be None
file_path_cache = {}
bug_points = []
if start_bug_point:
bug_point = session.query(BugReportPoint).get(start_bug_point)
file_path = file_path_cache.get(bug_point.file_id)
if not file_path:
f = session.query(File).get(bug_point.file_id)
file_path = f.filepath
file_path_cache[bug_point.file_id] = file_path
bug_points.append((bug_point, file_path))
while bug_point.next is not None:
bug_point = session.query(BugReportPoint).get(bug_point.next)
file_path = file_path_cache.get(bug_point.file_id)
if not file_path:
f = session.query(File).get(bug_point.file_id)
file_path = f.filepath
file_path_cache[bug_point.file_id] = file_path
bug_points.append((bug_point, file_path))
return bug_points
@timefunc
def getReportDetails(self, reportId):
"""
Parameters:
- reportId
"""
session = self.__session
try:
report = session.query(Report).get(reportId)
events = self.__construct_bug_event_list(session,
report.start_bugevent)
bug_events_list = []
for (event, file_path) in events:
bug_events_list.append(
shared.ttypes.BugPathEvent(
startLine=event.line_begin,
startCol=event.col_begin,
endLine=event.line_end,
endCol=event.col_end,
msg=event.msg,
fileId=event.file_id,
filePath=file_path))
points = self.__construct_bug_point_list(session,
report.start_bugpoint)
bug_point_list = []
for (bug_point, file_path) in points:
bug_point_list.append(
shared.ttypes.BugPathPos(
startLine=bug_point.line_begin,
startCol=bug_point.col_begin,
endLine=bug_point.line_end,
endCol=bug_point.col_end,
fileId=bug_point.file_id,
filePath=file_path))
return ReportDetails(bug_events_list, bug_point_list)
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE, msg)
def __set_report_suppress_flag(self,
session,
run_ids,
bug_id_hash,
source_file_name,
suppress_flag):
"""
update the suppress flag for multiple report entries based on the
filter
"""
if not run_ids:
# there are no run ids where the report should be suppressed
return
def check_filename(data):
report, file_obj = data
source_file_path, f_name = ntpath.split(file_obj.filepath)
if f_name == source_file_name:
return True
else:
return False
reports = session.query(Report, File) \
.filter(and_(Report.bug_id == bug_id_hash,
Report.run_id.in_(run_ids))) \
.outerjoin(File, File.id == Report.file_id) \
.all()
reports = filter(check_filename, reports)
for report, file_obj in reports:
report.suppressed = suppress_flag
def __update_suppress_storage_data(self,
run_ids,
report,
suppress,
comment=u''):
"""
update suppress information in the database and in the suppress file
can be used to suppress or unsuppress a report for multiple runs
"""
session = self.__session
report_id = report.id
bug_id_hash = report.bug_id
source_file = session.query(File).get(report.file_id)
source_file_path, source_file_name = ntpath.split(source_file.filepath)
LOG.debug('Updating suppress data for: ' + str(report_id) + ' bug id ' +
bug_id_hash + ' file name ' + source_file_name +
' suppressing ' + str(suppress))
# check if it is already suppressed for any run ids
suppressed = session.query(SuppressBug) \
.filter(or_( \
and_(SuppressBug.hash == bug_id_hash,
SuppressBug.file_name == source_file_name,
SuppressBug.run_id.in_(run_ids)),
and_(SuppressBug.hash == bug_id_hash,
SuppressBug.file_name == '',
SuppressBug.run_id.in_(run_ids))
)) \
.all()
if not suppressed and suppress:
# the bug is not suppressed for any run_id, suppressing it
LOG.debug('Bug is not suppressed in any runs')
for rId in run_ids:
suppress_bug = SuppressBug(rId,
bug_id_hash,
source_file_name,
comment)
session.add(suppress_bug)
# update report entries
self.__set_report_suppress_flag(session,
run_ids,
bug_id_hash,
source_file_name,
suppress_flag=suppress)
elif suppressed and suppress:
# already suppressed for some run ids check if other suppression
# is needed for other run id
suppressed_runids = set([r.run_id for r in suppressed])
LOG.debug('Bug is suppressed in these runs:' +
' '.join([str(r) for r in suppressed_runids]))
suppress_in_these_runs = set(run_ids).difference(suppressed_runids)
for run_id in suppress_in_these_runs:
suppress_bug = SuppressBug(run_id,
bug_id_hash,
source_file_name,
comment)
session.add(suppress_bug)
self.__set_report_suppress_flag(session,
suppress_in_these_runs,
bug_id_hash,
source_file_name,
suppress_flag=suppress)
elif suppressed and not suppress:
# already suppressed for some run ids
# remove those entries
already_suppressed_runids = \
filter(lambda bug: bug.run_id in run_ids, set(suppressed))
unsuppress_in_these_runs = \
{bug.run_id for bug in already_suppressed_runids}
LOG.debug('Already suppressed, unsuppressing now')
suppressed = session.query(SuppressBug) \
.filter(and_(SuppressBug.hash == bug_id_hash,
SuppressBug.file_name == source_file_name,
SuppressBug.run_id.in_(unsuppress_in_these_runs)))
# delete supppress bug entries
for sp in suppressed:
session.delete(sp)
# update report entries
self.__set_report_suppress_flag(session,
unsuppress_in_these_runs,
bug_id_hash,
source_file_name,
suppress_flag=suppress)
#elif suppressed is None and not suppress:
# # check only in the file if there is anything that should be
# # removed the database has no entries in the suppressBug table
ret = True
if suppress:
# store to suppress file
ret = self.__suppress_handler \
.store_suppress_bug_id(bug_id_hash,
source_file_name,
comment)
else:
# remove from suppress file
ret = self.__suppress_handler \
.remove_suppress_bug_id(bug_id_hash,
source_file_name)
if not ret:
session.rollback()
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.IOERROR,
'Failed to store suppress bug id')
else:
session.commit()
return True
@timefunc
def suppressBug(self, run_ids, report_id, comment):
"""
Add suppress bug entry to the SuppressBug table.
Set the suppressed flag for the selected report.
"""
session = self.__session
try:
report = session.query(Report).get(report_id)
if report:
return self.__update_suppress_storage_data(run_ids,
report,
True,
comment)
else:
msg = 'Report id ' + str(report_id) + \
' was not found in the database.'
LOG.error(msg)
raise shared.ttypes.RequestFailed(
shared.ttypes.ErrorCode.DATABASE, msg)
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(
shared.ttypes.ErrorCode.DATABASE, msg)
except Exception as ex:
msg = str(ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(
shared.ttypes.ErrorCode.IOERROR, msg)
@timefunc
def unSuppressBug(self, run_ids, report_id):
"""
Remove the suppress flag from the reports in multiple runs if given.
Cleanup the SuppressBug table to remove suppress entries.
"""
session = self.__session
try:
report = session.query(Report).get(report_id)
if report:
return self.__update_suppress_storage_data(run_ids,
report,
False)
else:
msg = 'Report id ' + str(report_id) + \
' was not found in the database.'
LOG.error(msg)
raise shared.ttypes.RequestFailed(
shared.ttypes.ErrorCode.DATABASE, msg)
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE,
msg)
except Exception as ex:
msg = str(ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.IOERROR,
msg)
def getCheckerDoc(self, checkerId):
"""
Parameters:
- checkerId
"""
text = "No documentation found for checker: " + checkerId + \
"\n\nPlease refer to the documentation at the "
if "." in checkerId:
text += "[ClangSA](http://clang-analyzer.llvm.org/available_checks.html)"
elif "-" in checkerId:
text += "[ClangTidy](http://clang.llvm.org/extra/clang-tidy/checks/list.html)"
text += " homepage."
try:
md_file = self.__checker_doc_map.get(checkerId)
if md_file:
md_file = os.path.join(self.__checker_md_docs, md_file)
with open(md_file, 'r') as md_content:
text = md_content.read()
return text
except Exception as ex:
msg = str(ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.IOERROR, msg)
def getCheckerConfigs(self, run_id):
"""
Parameters:
- run_id
"""
session = self.__session
try:
configs = session.query(Config) \
.filter(Config.run_id == run_id) \
.all()
configs = [(c.checker_name, c.attribute, c.value)
for c in configs]
res = []
for cName, attribute, value in configs:
res.append(shared.ttypes.ConfigValue(cName, attribute, value))
return res
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE, msg)
@timefunc
def getSkipPaths(self, run_id):
session = self.__session
try:
suppressed_paths = session.query(SkipPath) \
.filter(SkipPath.run_id == run_id) \
.all()
results = []
for sp in suppressed_paths:
encoded_path = sp.path
encoded_comment = sp.comment
results.append(SkipPathData(encoded_path, encoded_comment))
return results
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE, msg)
@timefunc
def getBuildActions(self, reportId):
session = self.__session
try:
build_actions = session.query(BuildAction) \
.outerjoin(ReportsToBuildActions) \
.filter(ReportsToBuildActions.report_id == reportId) \
.all()
return [ba.build_cmd for ba in build_actions]
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE, msg)
@timefunc
def getFileId(self, run_id, path):
session = self.__session
try:
sourcefile = session.query(File) \
.filter(File.run_id == run_id,
File.filepath == path) \
.first()
if sourcefile is None:
return -1
return sourcefile.id
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE, msg)
@timefunc
def getSourceFileData(self, fileId, fileContent):
"""
Parameters:
- fileId
- fileContent
"""
session = self.__session
try:
sourcefile = session.query(File) \
.filter(File.id == fileId).first()
if sourcefile is None:
return SourceFileData()
if fileContent and sourcefile.content:
source = zlib.decompress(sourcefile.content)
source = codecs.decode(source, 'utf-8', 'replace')
return SourceFileData(fileId=sourcefile.id,
filePath=sourcefile.filepath,
fileContent=source)
else:
return SourceFileData(fileId=sourcefile.id,
filePath=sourcefile.filepath)
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE, msg)
@timefunc
def getRunResultTypes(self, run_id, report_filters):
session = self.__session
try:
filter_expression = construct_report_filter(report_filters)
q = session.query(Report) \
.filter(Report.run_id == run_id) \
.outerjoin(File,
Report.file_id == File.id) \
.outerjoin(BugPathEvent,
Report.end_bugevent == BugPathEvent.id) \
.outerjoin(SuppressBug,
and_(SuppressBug.hash == Report.bug_id,
SuppressBug.run_id == run_id)) \
.order_by(Report.checker_id) \
.filter(filter_expression) \
.all()
count_results = defaultdict(int)
result_reports = defaultdict()
# count and filter out the results for the same checker_id
for r in q:
count_results[r.checker_id] += 1
result_reports[r.checker_id] = r
results = []
for checker_id, res in result_reports.items():
results.append(ReportDataTypeCount(res.checker_id, res.severity, count_results[res.checker_id]))
# result count ascending
results = sorted(results, key=lambda rep: rep.count, reverse=True)
return results
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE, msg)
# -----------------------------------------------------------------------
@timefunc
def __get_hashes_for_diff(self, session, base_run_id, new_run_id):
LOG.debug('query all baseline hashes')
# keyed tuple list is returned
base_line_hashes = session.query(Report.bug_id) \
.filter(Report.run_id == base_run_id) \
.all()
# LOG.debug(len(base_line_hashes))
LOG.debug('query all new check hashes')
# keyed tuple list is returned
new_check_hashes = session.query(Report.bug_id) \
.filter(Report.run_id == new_run_id) \
.all()
# LOG.debug(len(new_check_hashes))
base_line_hashes = set([t[0] for t in base_line_hashes])
new_check_hashes = set([t[0] for t in new_check_hashes])
return base_line_hashes, new_check_hashes
# -----------------------------------------------------------------------
@timefunc
def __queryDiffResults(self,
session,
diff_hash_list,
run_id,
limit,
offset,
sort_types=None,
report_filters=None):
max_query_limit = constants.MAX_QUERY_SIZE
if limit > max_query_limit:
LOG.debug('Query limit ' + str(limit) +
' was larger than max query limit ' +
str(max_query_limit) + ', setting limit to ' +
str(max_query_limit))
limit = max_query_limit
filter_expression = construct_report_filter(report_filters)
try:
q = session.query(Report,
File,
BugPathEvent,
SuppressBug) \
.filter(Report.run_id == run_id) \
.outerjoin(File,
and_(Report.file_id == File.id,
File.run_id == run_id)) \
.outerjoin(BugPathEvent,
Report.end_bugevent == BugPathEvent.id) \
.outerjoin(SuppressBug,
and_(SuppressBug.hash == Report.bug_id,
SuppressBug.run_id == run_id)) \
.filter(Report.bug_id.in_(diff_hash_list)) \
.filter(filter_expression)
q = self.__sortResultsQuery(q, sort_types)
results = []
for report, source_file, lbpe, suppress_bug \
in q.limit(limit).offset(offset):
lastEventPos = \
shared.ttypes.BugPathEvent(startLine=lbpe.line_begin,
startCol=lbpe.col_begin,
endLine=lbpe.line_end,
endCol=lbpe.col_end,
msg=lbpe.msg,
fileId=lbpe.file_id)
if suppress_bug:
suppress_comment = suppress_bug.comment
else:
suppress_comment = None
results.append(ReportData(bugHash=report.bug_id,
checkedFile=source_file.filepath,
checkerMsg=report.checker_message,
suppressed=report.suppressed,
reportId=report.id,
fileId=source_file.id,
lastBugPosition=lastEventPos,
checkerId=report.checker_id,
severity=report.severity,
suppressComment=suppress_comment))
return results
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE, msg)
# -----------------------------------------------------------------------
@timefunc
def getNewResults(self,
base_run_id,
new_run_id,
limit,
offset,
sort_types=None,
report_filters=None):
session = self.__session
base_line_hashes, new_check_hashes = \
self.__get_hashes_for_diff(session,
base_run_id,
new_run_id)
diff_hashes = list(new_check_hashes.difference(base_line_hashes))
LOG.debug(len(diff_hashes))
LOG.debug(diff_hashes)
if len(diff_hashes) == 0:
return []
return self.__queryDiffResults(session,
diff_hashes,
new_run_id,
limit,
offset,
sort_types,
report_filters)
# -----------------------------------------------------------------------
@timefunc
def getResolvedResults(self,
base_run_id,
new_run_id,
limit,
offset,
sort_types=None,
report_filters=None):
session = self.__session
base_line_hashes, new_check_hashes = \
self.__get_hashes_for_diff(session,
base_run_id,
new_run_id)
diff_hashes = list(base_line_hashes.difference(new_check_hashes))
LOG.debug(len(diff_hashes))
LOG.debug(diff_hashes)
if len(diff_hashes) == 0:
return []
return self.__queryDiffResults(session,
diff_hashes,
base_run_id,
limit,
offset,
sort_types,
report_filters)
# -----------------------------------------------------------------------
@timefunc
def getUnresolvedResults(self,
base_run_id,
new_run_id,
limit,
offset,
sort_types=None,
report_filters=None):
session = self.__session
base_line_hashes, new_check_hashes = \
self.__get_hashes_for_diff(session,
base_run_id,
new_run_id)
diff_hashes = list(base_line_hashes.intersection(new_check_hashes))
LOG.debug('diff hashes' + str(len(diff_hashes)))
LOG.debug(diff_hashes)
if len(diff_hashes) == 0:
return []
return self.__queryDiffResults(session,
diff_hashes,
new_run_id,
limit,
offset,
sort_types,
report_filters)
# -----------------------------------------------------------------------
@timefunc
def getAPIVersion(self):
# returns the thrift api version
return constants.API_VERSION
# -----------------------------------------------------------------------
@timefunc
def removeRunResults(self, run_ids):
session = self.__session
runs_to_delete = []
for run_id in run_ids:
LOG.debug('run ids to delete')
LOG.debug(run_id)
run_to_delete = session.query(Run).get(run_id)
if not run_to_delete.can_delete:
LOG.debug("Can't delete " + str(run_id))
continue
run_to_delete.can_delete = False
session.commit()
runs_to_delete.append(run_to_delete)
for run_to_delete in runs_to_delete:
session.delete(run_to_delete)
session.commit()
return True
# -----------------------------------------------------------------------
def getSuppressFile(self):
"""
return the suppress file path or empty string if not set
"""
suppress_file = self.__suppress_handler.suppress_file
if suppress_file:
return suppress_file
return ''
# -----------------------------------------------------------------------
def __queryDiffResultsCount(self,
session,
diff_hash_list,
run_id,
report_filters=None):
"""
count results for a hash list with filters
"""
filter_expression = construct_report_filter(report_filters)
try:
report_count = session.query(Report) \
.filter(Report.run_id == run_id) \
.outerjoin(File,
and_(Report.file_id == File.id,
File.run_id == run_id)) \
.outerjoin(BugPathEvent,
Report.end_bugevent == BugPathEvent.id) \
.outerjoin(SuppressBug,
and_(SuppressBug.hash == Report.bug_id,
SuppressBug.run_id == run_id)) \
.filter(Report.bug_id.in_(diff_hash_list)) \
.filter(filter_expression) \
.count()
if report_count is None:
report_count = 0
return report_count
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE,
msg)
# -----------------------------------------------------------------------
@timefunc
def getDiffResultCount(self,
base_run_id,
new_run_id,
diff_type,
report_filters):
"""
count the diff results
"""
session = self.__session
base_line_hashes, new_check_hashes = \
self.__get_hashes_for_diff(session,
base_run_id,
new_run_id)
run_id = None
diff_hashes = []
if diff_type == DiffType.NEW:
diff_hashes = list(new_check_hashes.difference(base_line_hashes))
if not diff_hashes:
return 0
run_id = new_run_id
elif diff_type == DiffType.RESOLVED:
diff_hashes = list(base_line_hashes.difference(new_check_hashes))
if not diff_hashes:
return 0
run_id = base_run_id
elif diff_type == DiffType.UNRESOLVED:
diff_hashes = list(base_line_hashes.intersection(new_check_hashes))
if not diff_hashes:
return 0
run_id = new_run_id
else:
msg = 'Unsupported diff type: ' + str(diff_type)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE,
msg)
return self.__queryDiffResultsCount(session,
diff_hashes,
run_id,
report_filters)
# -----------------------------------------------------------------------
def __queryDiffResultTypes(self,
session,
diff_hash_list,
run_id,
report_filters):
"""
query and count results for a hash list with filters
"""
try:
filter_expression = construct_report_filter(report_filters)
q = session.query(Report) \
.filter(Report.run_id == run_id) \
.outerjoin(File,
Report.file_id == File.id) \
.outerjoin(BugPathEvent,
Report.end_bugevent == BugPathEvent.id) \
.outerjoin(SuppressBug,
and_(SuppressBug.hash == Report.bug_id,
SuppressBug.run_id == run_id)) \
.order_by(Report.checker_id) \
.filter(Report.bug_id.in_(diff_hash_list)) \
.filter(filter_expression) \
.all()
count_results = defaultdict(int)
result_reports = defaultdict()
# count and filter out the results for the same checker_id
for r in q:
count_results[r.checker_id] += 1
result_reports[r.checker_id] = r
results = []
for checker_id, res in result_reports.items():
results.append(ReportDataTypeCount(res.checker_id,
res.severity,
count_results[res.checker_id]))
# result count ascending
results = sorted(results, key=lambda rep: rep.count, reverse=True)
return results
except sqlalchemy.exc.SQLAlchemyError as alchemy_ex:
msg = str(alchemy_ex)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE,
msg)
# -----------------------------------------------------------------------
@timefunc
def getDiffResultTypes(self,
base_run_id,
new_run_id,
diff_type,
report_filters):
session = self.__session
base_line_hashes, new_check_hashes = \
self.__get_hashes_for_diff(session,
base_run_id,
new_run_id)
run_id = None
diff_hashes = []
if diff_type == DiffType.NEW:
diff_hashes = list(new_check_hashes.difference(base_line_hashes))
if not diff_hashes:
return diff_hashes
run_id = new_run_id
elif diff_type == DiffType.RESOLVED:
diff_hashes = list(base_line_hashes.difference(new_check_hashes))
if not diff_hashes:
return diff_hashes
run_id = base_run_id
elif diff_type == DiffType.UNRESOLVED:
diff_hashes = list(base_line_hashes.intersection(new_check_hashes))
if not diff_hashes:
return diff_hashes
run_id = new_run_id
else:
msg = 'Unsupported diff type: ' + str(diff_type)
LOG.error(msg)
raise shared.ttypes.RequestFailed(shared.ttypes.ErrorCode.DATABASE,
msg)
return self.__queryDiffResultTypes(session,
diff_hashes,
run_id,
report_filters)

tools/codechecker/libcodechecker/viewer_server/client_db_access_server.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
'''
Main viewer server starts a http server which handles thrift clienta
and browser requests
'''
import os
import posixpath
import urllib
import errno
import socket
from multiprocessing.pool import ThreadPool
from sqlalchemy.orm import sessionmaker
from sqlalchemy.orm import scoped_session
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
from SimpleHTTPServer import SimpleHTTPRequestHandler
from thrift.transport import TTransport
from thrift.protocol import TJSONProtocol
from codeCheckerDBAccess import codeCheckerDBAccess
from client_db_access_handler import ThriftRequestHandler
from libcodechecker import logger
from libcodechecker import database_handler
LOG = logger.get_new_logger('DB ACCESS')
class RequestHander(SimpleHTTPRequestHandler):
'''
Handle thrift and browser requests
Simply modified and extended version of SimpleHTTPRequestHandler
'''
def __init__(self, request, client_address, server):
self.sc_session = server.sc_session
self.db_version_info = server.db_version_info
BaseHTTPRequestHandler.__init__(self,
request,
client_address,
server)
def log_message(self, msg_format, *args):
''' silenting http server '''
return
def do_POST(self):
''' handling thrift messages '''
client_host, client_port = self.client_address
LOG.debug('Processing request from: ' +
client_host + ':' +
str(client_port))
# create new thrift handler
checker_md_docs = self.server.checker_md_docs
checker_md_docs_map = self.server.checker_md_docs_map
suppress_handler = self.server.suppress_handler
protocol_factory = TJSONProtocol.TJSONProtocolFactory()
input_protocol_factory = protocol_factory
output_protocol_factory = protocol_factory
itrans = TTransport.TFileObjectTransport(self.rfile)
otrans = TTransport.TFileObjectTransport(self.wfile)
itrans = TTransport.TBufferedTransport(itrans,
int(self.headers['Content-Length']))
otrans = TTransport.TMemoryBuffer()
iprot = input_protocol_factory.getProtocol(itrans)
oprot = output_protocol_factory.getProtocol(otrans)
try:
session = self.sc_session()
acc_handler = ThriftRequestHandler(session,
checker_md_docs,
checker_md_docs_map,
suppress_handler,
self.db_version_info)
processor = codeCheckerDBAccess.Processor(acc_handler)
processor.process(iprot, oprot)
result = otrans.getvalue()
self.sc_session.remove()
self.send_response(200)
self.send_header("content-type", "application/x-thrift")
self.send_header("Content-Length", len(result))
self.end_headers()
self.wfile.write(result)
return
except Exception as exn:
LOG.error(str(exn))
self.send_error(404, "Request failed.")
# self.send_header("content-type", "application/x-thrift")
# self.end_headers()
# self.wfile.write('')
return
def list_directory(self, path):
''' disable directory listing '''
self.send_error(404, "No permission to list directory")
return None
def translate_path(self, path):
'''
modified version from SimpleHTTPRequestHandler
path is set to www_root
'''
# abandon query parameters
path = path.split('?', 1)[0]
path = path.split('#', 1)[0]
path = posixpath.normpath(urllib.unquote(path))
words = path.split('/')
words = filter(None, words)
path = self.server.www_root
for word in words:
drive, word = os.path.splitdrive(word)
head, word = os.path.split(word)
if word in (os.curdir, os.pardir):
continue
path = os.path.join(path, word)
return path
class CCSimpleHttpServer(HTTPServer):
'''
Simple http server to handle requests from the clients
'''
daemon_threads = False
def __init__(self,
server_address,
RequestHandlerClass,
db_conn_string,
pckg_data,
suppress_handler,
db_version_info):
LOG.debug('Initializing HTTP server')
self.www_root = pckg_data['www_root']
self.doc_root = pckg_data['doc_root']
self.checker_md_docs = pckg_data['checker_md_docs']
self.checker_md_docs_map = pckg_data['checker_md_docs_map']
self.suppress_handler = suppress_handler
self.db_version_info = db_version_info
self.__engine =database_handler.SQLServer.create_engine(db_conn_string)
Session = scoped_session(sessionmaker())
Session.configure(bind=self.__engine)
self.sc_session = Session
self.__request_handlers = ThreadPool(processes=10)
HTTPServer.__init__(self, server_address,
RequestHandlerClass,
bind_and_activate=True)
def process_request_thread(self, request, client_address):
try:
# finish_request instatiates request handler class
self.finish_request(request, client_address)
self.shutdown_request(request)
except socket.error as serr:
if serr[0] == errno.EPIPE:
LOG.debug('Broken pipe')
LOG.debug(serr)
self.shutdown_request(request)
except Exception as ex:
LOG.debug(ex)
self.handle_error(request, client_address)
self.shutdown_request(request)
def process_request(self, request, client_address):
# sock_name = request.getsockname()
# LOG.debug('PROCESSING request: '+str(sock_name)+' from: '
# +str(client_address))
self.__request_handlers.apply_async(self.process_request_thread,
(request, client_address))
def start_server(package_data, port, db_conn_string, suppress_handler,
not_host_only, db_version_info):
'''
start http server to handle web client and thrift requests
'''
LOG.debug('Starting the codechecker DB access server')
if not_host_only:
access_server_host = ''
else:
access_server_host = 'localhost'
LOG.debug('Suppressing to ' + str(suppress_handler.suppress_file))
server_addr = (access_server_host, port)
http_server = CCSimpleHttpServer(server_addr,
RequestHander,
db_conn_string,
package_data,
suppress_handler,
db_version_info)
LOG.info('Waiting for client requests on [' +
access_server_host + ':' + str(port) + ']')
http_server.serve_forever()
LOG.info('done.')

tools/codechecker/requirements.txt

  • This file was added.
sqlalchemy>=1.0.9
alembic>=0.8.2
psycopg2>=2.5.4
pg8000>=1.10.2
thrift>=0.9.1

tools/codechecker/tests/unit/__init__.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.

tools/codechecker/tests/unit/plist_test_files/clang-3.4.plist

  • This file was added.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>clang_version</key>
<string>Ubuntu clang version 3.4.2-13ubuntu2 (tags/RELEASE_34/dot2-final) (based on LLVM 3.4.2)</string>
<key>files</key>
<array>
<string>test.cpp</string>
<string>./test.h</string>
</array>
<key>diagnostics</key>
<array>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>19</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>19</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>12</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>&apos;base&apos; initialized to 0</string>
<key>message</key>
<string>&apos;base&apos; initialized to 0</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>13</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>15</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>15</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>18</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Passing the value 0 via 1st parameter &apos;base&apos;</string>
<key>message</key>
<string>Passing the value 0 via 1st parameter &apos;base&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>19</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Calling &apos;test_func&apos;</string>
<key>message</key>
<string>Calling &apos;test_func&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Entered call from &apos;main&apos;</string>
<key>message</key>
<string>Entered call from &apos;main&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>4</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Passing the value 0 via 1st parameter &apos;num&apos;</string>
<key>message</key>
<string>Passing the value 0 via 1st parameter &apos;num&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>20</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>26</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Calling &apos;generate_id&apos;</string>
<key>message</key>
<string>Calling &apos;generate_id&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>1</integer>
</dict>
<key>depth</key><integer>2</integer>
<key>extended_message</key>
<string>Entered call from &apos;test_func&apos;</string>
<key>message</key>
<string>Entered call from &apos;test_func&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>12</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>17</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
</array>
<key>depth</key><integer>2</integer>
<key>extended_message</key>
<string>Division by zero</string>
<key>message</key>
<string>Division by zero</string>
</dict>
</array>
<key>description</key><string>Division by zero</string>
<key>category</key><string>Logic error</string>
<key>type</key><string>Division by zero</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>generate_id</string>
<key>issue_hash</key><string>1</string>
<key>location</key>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
</dict>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>26</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Value stored to &apos;id&apos; is never read</string>
<key>message</key>
<string>Value stored to &apos;id&apos; is never read</string>
</dict>
</array>
<key>description</key><string>Value stored to &apos;id&apos; is never read</string>
<key>category</key><string>Dead store</string>
<key>type</key><string>Dead assignment</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>test_func</string>
<key>issue_hash</key><string>2</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
</dict>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>29</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
<key>message</key>
<string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
</dict>
</array>
<key>description</key><string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
<key>category</key><string>Logic error</string>
<key>type</key><string>Stack address stored into global variable</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>test</string>
<key>issue_hash</key><string>3</string>
<key>location</key>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
</dict>
</array>
</dict>
</plist>
No newline at end of file

tools/codechecker/tests/unit/plist_test_files/clang-3.5.plist

  • This file was added.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>clang_version</key>
<string>Ubuntu clang version 3.5.2-2 (tags/RELEASE_352/final) (based on LLVM 3.5.2)</string>
<key>files</key>
<array>
<string>test.cpp</string>
<string>./test.h</string>
</array>
<key>diagnostics</key>
<array>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>19</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>19</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>12</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>&apos;base&apos; initialized to 0</string>
<key>message</key>
<string>&apos;base&apos; initialized to 0</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>13</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>15</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>15</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>18</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Passing the value 0 via 1st parameter &apos;base&apos;</string>
<key>message</key>
<string>Passing the value 0 via 1st parameter &apos;base&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>19</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Calling &apos;test_func&apos;</string>
<key>message</key>
<string>Calling &apos;test_func&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Entered call from &apos;main&apos;</string>
<key>message</key>
<string>Entered call from &apos;main&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>4</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Passing the value 0 via 1st parameter &apos;num&apos;</string>
<key>message</key>
<string>Passing the value 0 via 1st parameter &apos;num&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>20</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>26</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Calling &apos;generate_id&apos;</string>
<key>message</key>
<string>Calling &apos;generate_id&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>1</integer>
</dict>
<key>depth</key><integer>2</integer>
<key>extended_message</key>
<string>Entered call from &apos;test_func&apos;</string>
<key>message</key>
<string>Entered call from &apos;test_func&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>12</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>17</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
</array>
<key>depth</key><integer>2</integer>
<key>extended_message</key>
<string>Division by zero</string>
<key>message</key>
<string>Division by zero</string>
</dict>
</array>
<key>description</key><string>Division by zero</string>
<key>category</key><string>Logic error</string>
<key>type</key><string>Division by zero</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>generate_id</string>
<key>issue_hash</key><string>1</string>
<key>location</key>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
</dict>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>26</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Value stored to &apos;id&apos; is never read</string>
<key>message</key>
<string>Value stored to &apos;id&apos; is never read</string>
</dict>
</array>
<key>description</key><string>Value stored to &apos;id&apos; is never read</string>
<key>category</key><string>Dead store</string>
<key>type</key><string>Dead assignment</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>test_func</string>
<key>issue_hash</key><string>2</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
</dict>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>29</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
<key>message</key>
<string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
</dict>
</array>
<key>description</key><string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
<key>category</key><string>Logic error</string>
<key>type</key><string>Stack address stored into global variable</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>test</string>
<key>issue_hash</key><string>3</string>
<key>location</key>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
</dict>
</array>
</dict>
</plist>
No newline at end of file

tools/codechecker/tests/unit/plist_test_files/clang-3.6.plist

  • This file was added.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>clang_version</key>
<string>Ubuntu clang version 3.6.2-1 (tags/RELEASE_362/final) (based on LLVM 3.6.2)</string>
<key>files</key>
<array>
<string>test.cpp</string>
<string>./test.h</string>
</array>
<key>diagnostics</key>
<array>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>19</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>19</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>12</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>&apos;base&apos; initialized to 0</string>
<key>message</key>
<string>&apos;base&apos; initialized to 0</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>13</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>15</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>15</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>18</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Passing the value 0 via 1st parameter &apos;base&apos;</string>
<key>message</key>
<string>Passing the value 0 via 1st parameter &apos;base&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>19</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Calling &apos;test_func&apos;</string>
<key>message</key>
<string>Calling &apos;test_func&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Entered call from &apos;main&apos;</string>
<key>message</key>
<string>Entered call from &apos;main&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>4</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Passing the value 0 via 1st parameter &apos;num&apos;</string>
<key>message</key>
<string>Passing the value 0 via 1st parameter &apos;num&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>20</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>26</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Calling &apos;generate_id&apos;</string>
<key>message</key>
<string>Calling &apos;generate_id&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>1</integer>
</dict>
<key>depth</key><integer>2</integer>
<key>extended_message</key>
<string>Entered call from &apos;test_func&apos;</string>
<key>message</key>
<string>Entered call from &apos;test_func&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>12</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>17</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
</array>
<key>depth</key><integer>2</integer>
<key>extended_message</key>
<string>Division by zero</string>
<key>message</key>
<string>Division by zero</string>
</dict>
</array>
<key>description</key><string>Division by zero</string>
<key>category</key><string>Logic error</string>
<key>type</key><string>Division by zero</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>generate_id</string>
<key>issue_hash</key><string>1</string>
<key>location</key>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
</dict>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>26</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Value stored to &apos;id&apos; is never read</string>
<key>message</key>
<string>Value stored to &apos;id&apos; is never read</string>
</dict>
</array>
<key>description</key><string>Value stored to &apos;id&apos; is never read</string>
<key>category</key><string>Dead store</string>
<key>type</key><string>Dead assignment</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>test_func</string>
<key>issue_hash</key><string>2</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
</dict>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>29</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
<key>message</key>
<string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
</dict>
</array>
<key>description</key><string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
<key>category</key><string>Logic error</string>
<key>type</key><string>Stack address stored into global variable</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>test</string>
<key>issue_hash</key><string>3</string>
<key>location</key>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
</dict>
</array>
</dict>
</plist>
No newline at end of file

tools/codechecker/tests/unit/plist_test_files/clang-3.7-noerror.plist

  • This file was added.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>clang_version</key>
<string>Ubuntu clang version 3.7.0-2ubuntu1 (tags/RELEASE_370/final) (based on LLVM 3.7.0)</string>
<key>files</key>
<array>
</array>
<key>diagnostics</key>
<array>
</array>
</dict>
</plist>
No newline at end of file

tools/codechecker/tests/unit/plist_test_files/clang-3.7.plist

  • This file was added.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>clang_version</key>
<string>Ubuntu clang version 3.7.0-2ubuntu1 (tags/RELEASE_370/final) (based on LLVM 3.7.0)</string>
<key>files</key>
<array>
<string>test.cpp</string>
<string>./test.h</string>
</array>
<key>diagnostics</key>
<array>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>19</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>19</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>12</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>&apos;base&apos; initialized to 0</string>
<key>message</key>
<string>&apos;base&apos; initialized to 0</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>13</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>15</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>15</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>18</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Passing the value 0 via 1st parameter &apos;base&apos;</string>
<key>message</key>
<string>Passing the value 0 via 1st parameter &apos;base&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>19</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Calling &apos;test_func&apos;</string>
<key>message</key>
<string>Calling &apos;test_func&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Entered call from &apos;main&apos;</string>
<key>message</key>
<string>Entered call from &apos;main&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>4</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Passing the value 0 via 1st parameter &apos;num&apos;</string>
<key>message</key>
<string>Passing the value 0 via 1st parameter &apos;num&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>20</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>26</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Calling &apos;generate_id&apos;</string>
<key>message</key>
<string>Calling &apos;generate_id&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>1</integer>
</dict>
<key>depth</key><integer>2</integer>
<key>extended_message</key>
<string>Entered call from &apos;test_func&apos;</string>
<key>message</key>
<string>Entered call from &apos;test_func&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>12</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>17</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
</array>
<key>depth</key><integer>2</integer>
<key>extended_message</key>
<string>Division by zero</string>
<key>message</key>
<string>Division by zero</string>
</dict>
</array>
<key>description</key><string>Division by zero</string>
<key>category</key><string>Logic error</string>
<key>type</key><string>Division by zero</string>
<key>check_name</key><string>core.DivideZero</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>generate_id</string>
<key>issue_hash</key><string>1</string>
<key>location</key>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
</dict>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>26</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Value stored to &apos;id&apos; is never read</string>
<key>message</key>
<string>Value stored to &apos;id&apos; is never read</string>
</dict>
</array>
<key>description</key><string>Value stored to &apos;id&apos; is never read</string>
<key>category</key><string>Dead store</string>
<key>type</key><string>Dead assignment</string>
<key>check_name</key><string>deadcode.DeadStores</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>test_func</string>
<key>issue_hash</key><string>2</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
</dict>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>29</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
<key>message</key>
<string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
</dict>
</array>
<key>description</key><string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
<key>category</key><string>Logic error</string>
<key>type</key><string>Stack address stored into global variable</string>
<key>check_name</key><string>core.StackAddressEscape</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>test</string>
<key>issue_hash</key><string>3</string>
<key>location</key>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
</dict>
</array>
</dict>
</plist>
No newline at end of file

tools/codechecker/tests/unit/plist_test_files/clang-3.8-trunk.plist

  • This file was added.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>clang_version</key>
<string>clang version 3.8.0 (trunk 251510)</string>
<key>files</key>
<array>
<string>test.cpp</string>
<string>./test.h</string>
</array>
<key>diagnostics</key>
<array>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>19</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>19</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>12</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>&apos;base&apos; initialized to 0</string>
<key>message</key>
<string>&apos;base&apos; initialized to 0</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>20</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>13</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>15</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>15</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>18</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Passing the value 0 via 1st parameter &apos;base&apos;</string>
<key>message</key>
<string>Passing the value 0 via 1st parameter &apos;base&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>21</integer>
<key>col</key><integer>19</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Calling &apos;test_func&apos;</string>
<key>message</key>
<string>Calling &apos;test_func&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Entered call from &apos;main&apos;</string>
<key>message</key>
<string>Entered call from &apos;main&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>4</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>7</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Passing the value 0 via 1st parameter &apos;num&apos;</string>
<key>message</key>
<string>Passing the value 0 via 1st parameter &apos;num&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>22</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>25</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>20</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>26</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>1</integer>
<key>extended_message</key>
<string>Calling &apos;generate_id&apos;</string>
<key>message</key>
<string>Calling &apos;generate_id&apos;</string>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>1</integer>
</dict>
<key>depth</key><integer>2</integer>
<key>extended_message</key>
<string>Entered call from &apos;test_func&apos;</string>
<key>message</key>
<string>Entered call from &apos;test_func&apos;</string>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>6</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>12</integer>
<key>file</key><integer>1</integer>
</dict>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>17</integer>
<key>file</key><integer>1</integer>
</dict>
</array>
</array>
<key>depth</key><integer>2</integer>
<key>extended_message</key>
<string>Division by zero</string>
<key>message</key>
<string>Division by zero</string>
</dict>
</array>
<key>description</key><string>Division by zero</string>
<key>category</key><string>Logic error</string>
<key>type</key><string>Division by zero</string>
<key>check_name</key><string>core.DivideZero</string>
<!-- This hash is experimental and going to change! -->
<key>issue_hash_content_of_line_in_context</key><string>79e31a6ba028f0b7d9779faf4a6cb9cf</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>generate_id</string>
<key>issue_hash_function_offset</key><string>1</string>
<key>location</key>
<dict>
<key>line</key><integer>7</integer>
<key>col</key><integer>14</integer>
<key>file</key><integer>1</integer>
</dict>
</dict>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>10</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>26</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Value stored to &apos;id&apos; is never read</string>
<key>message</key>
<string>Value stored to &apos;id&apos; is never read</string>
</dict>
</array>
<key>description</key><string>Value stored to &apos;id&apos; is never read</string>
<key>category</key><string>Dead store</string>
<key>type</key><string>Dead assignment</string>
<key>check_name</key><string>deadcode.DeadStores</string>
<!-- This hash is experimental and going to change! -->
<key>issue_hash_content_of_line_in_context</key><string>8714f42d8328bc78d5d7bff6ced918cc</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>test_func</string>
<key>issue_hash_function_offset</key><string>2</string>
<key>location</key>
<dict>
<key>line</key><integer>8</integer>
<key>col</key><integer>5</integer>
<key>file</key><integer>0</integer>
</dict>
</dict>
<dict>
<key>path</key>
<array>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>6</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>control</string>
<key>edges</key>
<array>
<dict>
<key>start</key>
<array>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>15</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
<key>end</key>
<array>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</dict>
</array>
</dict>
<dict>
<key>kind</key><string>event</string>
<key>location</key>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
<key>ranges</key>
<array>
<array>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>3</integer>
<key>file</key><integer>0</integer>
</dict>
<dict>
<key>line</key><integer>14</integer>
<key>col</key><integer>29</integer>
<key>file</key><integer>0</integer>
</dict>
</array>
</array>
<key>depth</key><integer>0</integer>
<key>extended_message</key>
<string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
<key>message</key>
<string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
</dict>
</array>
<key>description</key><string>Address of stack memory associated with local variable &apos;str&apos; is still referred to by the global variable &apos;p&apos; upon returning to the caller. This will be a dangling reference</string>
<key>category</key><string>Logic error</string>
<key>type</key><string>Stack address stored into global variable</string>
<key>check_name</key><string>core.StackAddressEscape</string>
<!-- This hash is experimental and going to change! -->
<key>issue_hash_content_of_line_in_context</key><string>f7b5072d428e890f2d309217f3ead16f</string>
<key>issue_context_kind</key><string>function</string>
<key>issue_context</key><string>test</string>
<key>issue_hash_function_offset</key><string>3</string>
<key>location</key>
<dict>
<key>line</key><integer>16</integer>
<key>col</key><integer>1</integer>
<key>file</key><integer>0</integer>
</dict>
</dict>
</array>
</dict>
</plist>
No newline at end of file

tools/codechecker/tests/unit/plist_test_files/empty_file

  • This file was added.
This is an empty file.

tools/codechecker/tests/unit/plist_test_files/gen_plist/README.md

  • This file was added.
## Generate plist files
Use these scripts and source files to generate plist report files.
call gen_plist script with the clang binary and the plist output filename
Eg.:
~~~~~~~~~~~
./gen_plist clang-3.7 clang-3.7.plist
~~~~~~~~~~~

tools/codechecker/tests/unit/plist_test_files/gen_plist/gen_noerror_plist

  • This file was added.
PropertyOld ValueNew Value
File Modenull100755
#!/bin/bash
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
set -x # echo on
CLANG=$1
OUT=$2
$CLANG --analyze -Xclang -analyzer-opt-analyze-headers -Xclang -analyzer-output=plist-multi-file -o $OUT noerror.cpp

tools/codechecker/tests/unit/plist_test_files/gen_plist/gen_plist

  • This file was added.
PropertyOld ValueNew Value
File Modenull100755
#!/bin/bash
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
set -x # echo on
CLANG=$1
OUT=$2
$CLANG --analyze -Xclang -analyzer-opt-analyze-headers -Xclang -analyzer-output=plist-multi-file -o $OUT test.cpp -I.

tools/codechecker/tests/unit/plist_test_files/gen_plist/noerror.cpp

  • This file was added.
/* -*- coding: utf-8 -*-
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
*/
int main(){
return 0;
}

tools/codechecker/tests/unit/plist_test_files/gen_plist/test.h

  • This file was added.
/* -*- coding: utf-8 -*-
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
*/
#ifndef TEST_H
#define TEST_H
#include <iostream>
int generate_id(int num){
return 42/num; // warn
}
#endif // TEST_H

tools/codechecker/tests/unit/plist_test_files/gen_plist/test.cpp

  • This file was added.
/* -*- coding: utf-8 -*-
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
*/
#include <test.h>
#include <iostream>
using namespace std;
void test_func(int base){
int id;
id = generate_id(base);
}
char const *p;
void test() {
char const str[] = "string";
p = str; // warn
}
int main(){
int unused; // warn
int base = 0;
test_func(base);
test();
return 0;
}

tools/codechecker/tests/unit/source_suppress_test_files/test_file_1

  • This file was added.
// THIS FILE IS ENCODED IN UTF-8! BE CAREFUL WITH MODIFYING IT
int test_func(int num){ // line 3
cout << "test func" << endl;
return 0;
}
// this is an ordinary comment
int test_func(int num){ // line 9
cout << "test func" << endl;
return 1;
}
// this is another non checker related comment
// codechecker_suppress [all] some comment
void test_func(int num){ // line 16
// suppress all checker results
cout << "test func" << endl;
}
// codechecker_suppress [all] some long
// comment
void test_func(int num){ // line 23
// suppress all checker results
cout << "test func" << endl;
}
// codechecker_suppress [my_checker_1, my_checker_2] some comment
void test_func(int num){ // line 29
// suppress some checker results my_checker_2 and my_checker_2
cout << "test func" << endl;
}
// codechecker_suppress [my.checker_1 my.checker_2] some really
// long comment
void test_func(int num){ // line 36
cout << "test func" << endl;
}
// codechecker_suppress [my.Checker_1, my.Checker_2] some really
// really
// long comment
void test_func(int num){ // line 43
cout << "test func" << endl;
}
// codechecker_suppress [my_checker_1, my_checker_2] some really
// really
void test_func(int num){ // line 50
// wrong formatted suppress comment
cout << "test func" << endl;
}
// codechecker_suppress [my.checker_1, my.checker_2]
// i/';0 (*&^%$#@!)
void test_func(int num){ // line 57
// wrong formatted suppress comment
cout << "test func" << endl;
}
// codechecker_suppress [ my_checker_1 ]
// áúőóüöáé ▬▬▬▬▬▬▬▬▬▬ஜ۩۞۩ஜ▬▬▬▬▬▬▬▬▬▬
void test_func(int num){ // line 64
// wrong formatted suppress comment
cout << "test func" << endl;
}
// codechecker_suppress [ my_checker_1 ]
void test_func(int num){ // line 70
// wrong formatted suppress comment
cout << "test func" << endl;
}

tools/codechecker/tests/unit/source_suppress_test_files/test_file_2

  • This file was added.
// first line non checker comment
int test_func(int num){ // line 2
cout << 'test func' << endl;
return 0;
}

tools/codechecker/tests/unit/test_buildcmd_escaping.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
"""Test the build commands escaping and execution."""
import json
import os
import shutil
import StringIO
import tempfile
import unittest
from contextlib import closing
from libcodechecker import log_parser
from libcodechecker import build_manager
from libcodechecker.analyzers import analyzer_base
class BuildCmdTestNose(unittest.TestCase):
"""
Test the build command escaping and execution.
"""
@classmethod
def setup_class(cls):
"""
Make a temporary directory and generate a source file
which will be built.
"""
cls.tmp_dir = tempfile.mkdtemp()
cls.src_file_name = "main.cpp"
cls.src_file_path = os.path.join(cls.tmp_dir,
cls.src_file_name)
cls.compiler = "g++"
with open(cls.src_file_path, "w") as test_src:
test_src.write("""
#include <iostream>
#ifndef MYPATH
#define MYPATH "/some/path"
#endif
int main(){
std::cout<< MYPATH << std::endl;
return 0;
}""")
@classmethod
def teardown_class(cls):
"""
Clean temporary directory and files.
"""
dir_to_clean = cls.tmp_dir
shutil.rmtree(dir_to_clean)
def __get_cmp_json(self, buildcmd):
"""
Generate a compile command json file.
"""
compile_cmds = []
compile_cmd = {}
compile_cmd["directory"] = self.tmp_dir
compile_cmd["command"] = buildcmd + " -c " + self.src_file_path
compile_cmd["file"] = self.src_file_path
compile_cmds.append(compile_cmd)
return json.dumps(compile_cmds)
def __get_comp_actions(self, compile_cmd):
"""
Generate a compilation command json file and parse it
to return the compilation actions.
"""
comp_cmd_json = self.__get_cmp_json(compile_cmd)
with closing(StringIO.StringIO()) as text:
text.write(comp_cmd_json)
return log_parser.parse_compile_commands_json(text)
def test_buildmgr(self):
"""
Check some simple command to be executed by
the build manager.
"""
cmd = 'cd ' + self.tmp_dir + ' && echo "test"'
print("Running: " + cmd)
ret_val = build_manager.execute_buildcmd(cmd)
self.assertEqual(ret_val, 0)
def test_analyzer_exec_double_quote(self):
"""
Test the process execution by the analyzer,
If the escaping fails the source file will not compile.
"""
compile_cmd = self.compiler + \
' -DDEBUG -DMYPATH="/this/some/path/"'
comp_actions = self.__get_comp_actions(compile_cmd)
for comp_action in comp_actions:
for source in comp_action.sources:
cmd = [self.compiler]
cmd.extend(comp_action.analyzer_options)
cmd.append(str(source))
cwd = comp_action.directory
print(cmd)
print(cwd)
ret_val, stdout, stderr = analyzer_base.SourceAnalyzer \
.run_proc(' '.join(cmd), cwd=cwd)
print(stdout)
print(stderr)
self.assertEqual(ret_val, 0)
def test_analyzer_ansic_double_quote(self):
"""
Test the process execution by the analyzer with ansi-C like
escape characters in it \".
If the escaping fails the source file will not compile.
"""
compile_cmd = self.compiler + ' -DMYPATH=\"/some/other/path\"'
comp_actions = self.__get_comp_actions(compile_cmd)
for comp_action in comp_actions:
for source in comp_action.sources:
cmd = [self.compiler]
cmd.extend(comp_action.analyzer_options)
cmd.append(str(source))
cwd = comp_action.directory
print(cmd)
print(cwd)
ret_val, stdout, stderr = analyzer_base.SourceAnalyzer \
.run_proc(' '.join(cmd), cwd=cwd)
print(stdout)
print(stderr)
self.assertEqual(ret_val, 0)

tools/codechecker/tests/unit/test_plist_parser.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
"""Test the parsing of the plist generated by multiple clang versions."""
import os
import unittest
from libcodechecker import plist_parser
class PlistParserTestCaseNose(unittest.TestCase):
"""Test the parsing of the plist generated by multiple clang versions."""
@classmethod
def setup_class(cls):
"""Initialize test source file."""
# Bugs found by these checkers in the test source files.
cls.__found_checker_names = [
'core.DivideZero',
'core.StackAddressEscape',
'deadcode.DeadStores']
# Previous clang versions do not generate the checker name into the
# plist output.
cls.__not_found_checker_names = ['NOT FOUND']
# Bug id hash values generated by codechecker framework.
cls.__framework_generated_hashes = [
'e9fb5a280e64610cfa82472117c8d0ac',
'9536c46411df42db29014729867146fa',
'b1bc0e8364a255659522055d1e15cd16']
# Bug were found in these test files.
cls.__found_file_names = ['test.cpp', './test.h']
# Already generated plist files for the tests.
cls.__this_dir = os.path.dirname(__file__)
cls.__plist_test_files = os.path.join(
cls.__this_dir, 'plist_test_files')
def __core_div_bug_event(self, bug_event):
"""Check the core.dividezero checker last event position."""
self.assertEqual(bug_event.start_pos.line, 7)
self.assertEqual(bug_event.start_pos.col, 12)
self.assertEqual(bug_event.start_pos.file_path, './test.h')
self.assertEqual(bug_event.end_pos.line, 7)
self.assertEqual(bug_event.end_pos.col, 17)
self.assertEqual(bug_event.end_pos.file_path, './test.h')
def __core_stack_addr_esc_event(self, bug_event):
"""Check the core.StackAddressEscape checker last event position."""
self.assertEqual(bug_event.start_pos.line, 14)
self.assertEqual(bug_event.start_pos.col, 3)
self.assertEqual(bug_event.start_pos.file_path, 'test.cpp')
self.assertEqual(bug_event.end_pos.line, 14)
self.assertEqual(bug_event.end_pos.col, 29)
self.assertEqual(bug_event.end_pos.file_path, 'test.cpp')
def test_empty_file(self):
"""Plist file is empty."""
empty_plist = os.path.join(self.__plist_test_files, 'empty_file')
files, bugs = plist_parser.parse_plist(empty_plist)
self.assertEquals(files, [])
self.assertEquals(bugs, [])
def test_no_bug_file(self):
"""There was no bug in the checked file."""
no_bug_plist = os.path.join(
self.__plist_test_files, 'clang-3.7-noerror.plist')
files, bugs = plist_parser.parse_plist(no_bug_plist)
self.assertEquals(files, [])
self.assertEquals(bugs, [])
def test_clang34_plist(self):
"""
Check plist generated by clang 3.4 checker name is not in the plist
file plist parser tries to find out the name.
"""
clang34_plist = os.path.join(
self.__plist_test_files, 'clang-3.4.plist')
files, bugs = plist_parser.parse_plist(clang34_plist)
self.assertEquals(files, self.__found_file_names)
self.assertEquals(len(bugs), 3)
valid_checker_names = []
valid_checker_names.extend(self.__found_checker_names)
valid_checker_names.extend(self.__not_found_checker_names)
for bug in bugs:
self.assertIn(bug.checker_name, valid_checker_names)
if bug.checker_name == 'core.DivideZero':
self.__core_div_bug_event(bug.get_last_event())
if bug.checker_name == 'NOT FOUND':
self.__core_stack_addr_esc_event(bug.get_last_event())
self.assertIn(bug.hash_value, self.__framework_generated_hashes)
def test_clang35_plist(self):
"""
Check plist generated by clang 3.5 checker name is not in the plist
file plist parser tries to find out the name.
"""
clang35_plist = os.path.join(
self.__plist_test_files, 'clang-3.5.plist')
files, bugs = plist_parser.parse_plist(clang35_plist)
self.assertEquals(files, self.__found_file_names)
self.assertEquals(len(bugs), 3)
valid_checker_names = []
valid_checker_names.extend(self.__found_checker_names)
valid_checker_names.extend(self.__not_found_checker_names)
for bug in bugs:
self.assertIn(bug.checker_name, valid_checker_names)
if bug.checker_name == 'core.DivideZero':
self.__core_div_bug_event(bug.get_last_event())
if bug.checker_name == 'NOT FOUND':
self.__core_stack_addr_esc_event(bug.get_last_event())
self.assertIn(bug.hash_value, self.__framework_generated_hashes)
def test_clang36_plist(self):
"""
Check plist generated by clang 3.6 checker name is not in the plist
file plist parser tries to find out the name.
"""
clang36_plist = os.path.join(
self.__plist_test_files, 'clang-3.6.plist')
files, bugs = plist_parser.parse_plist(clang36_plist)
self.assertEquals(files, self.__found_file_names)
self.assertEquals(len(bugs), 3)
valid_checker_names = []
valid_checker_names.extend(self.__found_checker_names)
valid_checker_names.extend(self.__not_found_checker_names)
for bug in bugs:
self.assertIn(bug.checker_name, valid_checker_names)
if bug.checker_name == 'core.DivideZero':
self.__core_div_bug_event(bug.get_last_event())
if bug.checker_name == 'NOT FOUND':
self.__core_stack_addr_esc_event(bug.get_last_event())
self.assertIn(bug.hash_value, self.__framework_generated_hashes)
def test_clang37_plist(self):
"""
Check plist generated by clang 3.7 checker name should be in the plist
file.
"""
clang37_plist = os.path.join(
self.__plist_test_files, 'clang-3.7.plist')
files, bugs = plist_parser.parse_plist(clang37_plist)
self.assertEquals(files, self.__found_file_names)
self.assertEquals(len(bugs), 3)
valid_checker_names = []
valid_checker_names.extend(self.__found_checker_names)
for bug in bugs:
# Checker name should be in the plist file.
self.assertNotEqual(bug.checker_name, 'NOT FOUND')
self.assertIn(bug.checker_name, valid_checker_names)
if bug.checker_name == 'core.DivideZero':
self.__core_div_bug_event(bug.get_last_event())
if bug.checker_name == 'core.StackAddressEscape':
self.__core_stack_addr_esc_event(bug.get_last_event())
self.assertNotEquals(bug.hash_value, '')
def test_clang38_trunk_plist(self):
"""
Check plist generated by clang 3.8 trunk checker name and bug hash
should be in the plist file.
"""
clang38_plist = os.path.join(
self.__plist_test_files, 'clang-3.8-trunk.plist')
files, bugs = plist_parser.parse_plist(clang38_plist)
self.assertEquals(files, self.__found_file_names)
self.assertEquals(len(bugs), 3)
valid_checker_names = []
valid_checker_names.extend(self.__found_checker_names)
for bug in bugs:
# Checker name should be in the plist file.
self.assertNotEqual(bug.checker_name, 'NOT FOUND')
self.assertIn(bug.checker_name, valid_checker_names)
if bug.checker_name == 'core.DivideZero':
self.__core_div_bug_event(bug.get_last_event())
self.assertEquals(
bug.hash_value, '79e31a6ba028f0b7d9779faf4a6cb9cf')
if bug.checker_name == 'core.StackAddressEscape':
self.__core_stack_addr_esc_event(bug.get_last_event())
self.assertEquals(
bug.hash_value, 'f7b5072d428e890f2d309217f3ead16f')
if bug.checker_name == 'deadcode.DeadStores':
self.assertEquals(
bug.hash_value, '8714f42d8328bc78d5d7bff6ced918cc')

tools/codechecker/tests/unit/test_source_suppress.py

  • This file was added.
# -*- coding: utf-8 -*-
#
# -----------------------------------------------------------------------------
# The CodeChecker Infrastructure
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
# -----------------------------------------------------------------------------
"""Tests for suppressing by comment in source file."""
import os
import unittest
from libcodechecker.suppress_handler import SourceSuppressHandler
class SourceSuppressTestCase(unittest.TestCase):
"""Tests for suppressing by comment in source file."""
@classmethod
def setup_class(cls):
"""Initialize test source file references."""
cls.__test_src_dir = os.path.join(
os.path.dirname(__file__), 'source_suppress_test_files')
cls.__tmp_srcfile_1 = os.path.join(cls.__test_src_dir, 'test_file_1')
cls.__tmp_srcfile_2 = os.path.join(cls.__test_src_dir, 'test_file_2')
def test_suppress_first_line(self):
"""Bug is reported for the first line."""
test_handler = SourceSuppressHandler(self.__tmp_srcfile_1, 3)
res = test_handler.check_source_suppress()
self.assertFalse(res)
self.assertEqual(test_handler.suppressed_checkers(), [])
self.assertIsNone(test_handler.suppress_comment())
def test_no_comment(self):
"""There is no comment above the bug line."""
sp_handler = SourceSuppressHandler(self.__tmp_srcfile_1, 9)
res = sp_handler.check_source_suppress()
self.assertFalse(res)
self.assertEqual(sp_handler.suppressed_checkers(), [])
self.assertIsNone(sp_handler.suppress_comment())
def test_no_suppress_comment(self):
"""There is suppress comment above the bug line."""
sp_handler = SourceSuppressHandler(self.__tmp_srcfile_1, 16)
res = sp_handler.check_source_suppress()
self.assertTrue(res)
self.assertEqual(sp_handler.suppressed_checkers(), ['all'])
self.assertEqual(sp_handler.suppress_comment(), 'some comment')
def test_multi_liner_all(self):
"""There is suppress comment above the bug line."""
sp_handler = SourceSuppressHandler(self.__tmp_srcfile_1, 23)
res = sp_handler.check_source_suppress()
self.assertTrue(res)
self.assertEqual(sp_handler.suppressed_checkers(), ['all'])
self.assertEqual(sp_handler.suppress_comment(), 'some long comment')
def test_one_liner_all(self):
"""There is suppress comment above the bug line."""
sp_handler = SourceSuppressHandler(self.__tmp_srcfile_1, 29)
res = sp_handler.check_source_suppress()
self.assertTrue(res)
self.assertEqual(
sp_handler.suppressed_checkers(), ['my_checker_1', 'my_checker_2'])
self.assertEqual(sp_handler.suppress_comment(), 'some comment')
def test_multi_liner_all_2(self):
"""There is suppress comment above the bug line."""
sp_handler = SourceSuppressHandler(self.__tmp_srcfile_1, 36)
res = sp_handler.check_source_suppress()
self.assertTrue(res)
self.assertEqual(
sp_handler.suppressed_checkers(), ['my.checker_1', 'my.checker_2'])
self.assertEqual(
sp_handler.suppress_comment(), 'some really long comment')
def test_one_liner_some_checkers(self):
"""There is suppress comment above the bug line."""
sp_handler = SourceSuppressHandler(self.__tmp_srcfile_1, 43)
res = sp_handler.check_source_suppress()
self.assertTrue(res)
self.assertEqual(
sp_handler.suppressed_checkers(), ['my.Checker_1', 'my.Checker_2'])
self.assertEqual(
sp_handler.suppress_comment(), 'some really really long comment')
def test_multi_liner_some_checkers(self):
"""There is suppress comment above the bug line."""
sp_handler = SourceSuppressHandler(self.__tmp_srcfile_1, 50)
res = sp_handler.check_source_suppress()
self.assertFalse(res)
self.assertEqual(sp_handler.suppressed_checkers(), [])
self.assertIsNone(sp_handler.suppress_comment())
def test_comment_characters(self):
"""Check for different special comment characters."""
sp_handler = SourceSuppressHandler(self.__tmp_srcfile_1, 57)
res = sp_handler.check_source_suppress()
self.assertTrue(res)
self.assertEqual(
sp_handler.suppressed_checkers(), ['my.checker_1', 'my.checker_2'])
self.assertEqual(sp_handler.suppress_comment(), "i/';0 (*&^%$#@!)")
def test_fancy_comment_characters(self):
"""Check fancy comment."""
sp_handler = SourceSuppressHandler(self.__tmp_srcfile_1, 64)
res = sp_handler.check_source_suppress()
self.assertTrue(res)
self.assertEqual(sp_handler.suppressed_checkers(), ['my_checker_1'])
self.assertEqual(
sp_handler.suppress_comment(),
"áúőóüöáé ▬▬▬▬▬▬▬▬▬▬ஜ۩۞۩ஜ▬▬▬▬▬▬▬▬▬▬")
def test_no_fancy_comment(self):
"""Check no fancy comment."""
sp_handler = SourceSuppressHandler(self.__tmp_srcfile_1, 70)
res = sp_handler.check_source_suppress()
self.assertTrue(res)
self.assertEqual(sp_handler.suppressed_checkers(), ['my_checker_1'])
self.assertEqual(
sp_handler.suppress_comment(),
'WARNING! suppress comment is missing')
def test_malformed_commment_format(self):
"""Check malformed comment."""
sp_handler = SourceSuppressHandler(self.__tmp_srcfile_2, 1)
res = sp_handler.check_source_suppress()
self.assertFalse(res)
self.assertEqual(sp_handler.suppressed_checkers(), [])
self.assertIsNone(sp_handler.suppress_comment())

tools/codechecker/tests/unit/test_tidy_output_converter.py

  • This file was added.
# -*- coding: utf-8 -*-
# The LLVM Compiler Infrastructure
#
# This file is distributed under the University of Illinois Open Source
# License. See LICENSE.TXT for details.
"""
This module tests the correctness of the OutputParser and PListConverter, which
used in sequence transform a Clang Tidy output file to a plist file.
"""
import os
import unittest
import copy
import StringIO
import libcodechecker.tidy_output_converter as tidy_out_conv
def setup_module():
"""Setup the test tidy reprs for the test classes in the module."""
os.chdir(os.path.join(os.path.dirname(__file__), 'tidy_output_test_files'))
# tidy1.out Message/Note representation
tidy1_repr = [
tidy_out_conv.Message(
os.path.abspath('files/test.cpp'),
8, 12,
'Division by zero',
'clang-analyzer-core.DivideZero',
None,
[tidy_out_conv.Note(
os.path.abspath('files/test.cpp'),
8, 12,
'Division by zero')]),
tidy_out_conv.Message(
os.path.abspath('files/test.cpp'),
8, 12,
'remainder by zero is undefined',
'clang-diagnostic-division-by-zero')
]
# tidy2.out Message/Note representation
tidy2_repr = [
tidy_out_conv.Message(
os.path.abspath('files/test2.cpp'),
5, 7,
"unused variable 'y'",
'clang-diagnostic-unused-variable'),
tidy_out_conv.Message(
os.path.abspath('files/test2.cpp'),
13, 12,
'Division by zero',
'clang-analyzer-core.DivideZero',
None,
[
tidy_out_conv.Note(
os.path.abspath('files/test2.cpp'),
9, 7,
"Left side of '||' is false"),
tidy_out_conv.Note(
os.path.abspath('files/test2.cpp'),
9, 3,
'Taking false branch'),
tidy_out_conv.Note(
os.path.abspath('files/test2.cpp'),
13, 12,
'Division by zero')
]),
tidy_out_conv.Message(
os.path.abspath('files/test2.cpp'),
13, 12,
'remainder by zero is undefined',
'clang-diagnostic-division-by-zero'),
]
# tidy3.out Message/Note representation
tidy3_repr = [
tidy_out_conv.Message(
os.path.abspath('files/test3.cpp'),
4, 12,
'use nullptr',
'modernize-use-nullptr',
[tidy_out_conv.Note(
os.path.abspath('files/test3.cpp'),
4, 12,
'nullptr')]),
tidy_out_conv.Message(
os.path.abspath('files/test3.hh'),
6, 6,
"Dereference of null pointer (loaded from variable 'x')",
'clang-analyzer-core.NullDereference',
None,
[
tidy_out_conv.Note(
os.path.abspath('files/test3.cpp'),
4, 3,
"'x' initialized to a null pointer value"),
tidy_out_conv.Note(
os.path.abspath('files/test3.cpp'),
6, 11,
"Assuming 'argc' is > 3"),
tidy_out_conv.Note(
os.path.abspath('files/test3.cpp'),
6, 3,
'Taking true branch'),
tidy_out_conv.Note(
os.path.abspath('files/test3.cpp'),
7, 9,
"Passing null pointer value via 1st parameter 'x'"),
tidy_out_conv.Note(
os.path.abspath('files/test3.cpp'),
7, 5,
"Calling 'bar'"),
tidy_out_conv.Note(
os.path.abspath('files/test3.hh'),
6, 6,
"Dereference of null pointer (loaded from variable 'x')")
])
]
TidyOutputParserTestCase.tidy1_repr = tidy1_repr
TidyOutputParserTestCase.tidy2_repr = tidy2_repr
TidyOutputParserTestCase.tidy3_repr = tidy3_repr
TidyPListConverterTestCase.tidy1_repr = tidy1_repr
TidyPListConverterTestCase.tidy2_repr = tidy2_repr
TidyPListConverterTestCase.tidy3_repr = tidy3_repr
class TidyOutputParserTestCase(unittest.TestCase):
"""
Tests the output of the OutputParser, which converts a Clang Tidy output
file to zero or more tidy_output_converter.Message.
"""
def setUp(self):
"""Setup the OutputParser."""
self.parser = tidy_out_conv.OutputParser()
def test_absolute_path(self):
"""Test for absolute paths in Messages."""
for tfile in ['abs.out', 'tidy1.out']:
messages = self.parser.parse_messages_from_file(tfile)
self.assertNotEqual(len(messages), 0)
for message in messages:
self.assertTrue(os.path.isabs(message.path))
def test_empty1(self):
"""Test an empty ClangTidy output file."""
messages = self.parser.parse_messages_from_file('empty1.out')
self.assertEqual(messages, [])
def test_empty2(self):
"""Test a ClangTidy output file that only contains empty lines."""
messages = self.parser.parse_messages_from_file('empty2.out')
self.assertEqual(messages, [])
def test_tidy1(self):
"""Test the generated Messages of tidy1.out ClangTidy output file."""
messages = self.parser.parse_messages_from_file('tidy1.out')
self.assertEqual(len(messages), len(self.tidy1_repr))
for message in messages:
self.assertIn(message, self.tidy1_repr)
def test_tidy2(self):
"""Test the generated Messages of tidy2.out ClangTidy output file."""
messages = self.parser.parse_messages_from_file('tidy2.out')
self.assertEqual(len(messages), len(self.tidy2_repr))
for message in messages:
self.assertIn(message, self.tidy2_repr)
def test_tidy3(self):
"""Test the generated Messages of tidy3.out ClangTidy output file."""
messages = self.parser.parse_messages_from_file('tidy3.out')
self.assertEqual(len(messages), len(self.tidy3_repr))
for message in messages:
self.assertIn(message, self.tidy3_repr)
class TidyPListConverterTestCase(unittest.TestCase):
"""
Test the output of the PListConverter, which converts Messages to plist
format.
"""
def setUp(self):
"""Setup the PListConverter."""
self.plist_conv = tidy_out_conv.PListConverter()
def test_empty(self):
"""Test for empty Messages."""
orig_plist = copy.deepcopy(self.plist_conv.plist)
self.plist_conv.add_messages([])
self.assertDictEqual(orig_plist, self.plist_conv.plist)
output = StringIO.StringIO()
self.plist_conv.write(output)
with open('empty.plist') as pfile:
exp = pfile.read()
self.assertEqual(exp, output.getvalue())
output.close()
def test_tidy1(self):
"""Test for the tidy1.plist file."""
self.plist_conv.add_messages(self.tidy1_repr)
# use relative path for this test
self.plist_conv.plist['files'][0] = 'files/test.cpp'
output = StringIO.StringIO()
self.plist_conv.write(output)
with open('tidy1.plist') as pfile:
exp = pfile.read()
self.assertEqual(exp, output.getvalue())
output.close()
def test_tidy2(self):
"""Test for the tidy2.plist file."""
self.plist_conv.add_messages(self.tidy2_repr)
# use relative path for this test
self.plist_conv.plist['files'][0] = 'files/test2.cpp'
output = StringIO.StringIO()
self.plist_conv.write(output)
with open('tidy2.plist') as pfile:
exp = pfile.read()
self.assertEqual(exp, output.getvalue())
output.close()
def test_tidy3(self):
"""Test for the tidy3.plist file."""
self.plist_conv.add_messages(self.tidy3_repr)
# use relative path for this test
self.plist_conv.plist['files'][0] = 'files/test3.cpp'
self.plist_conv.plist['files'][1] = 'files/test3.hh'
output = StringIO.StringIO()
self.plist_conv.write(output)
with open('tidy3.plist') as pfile:
exp = pfile.read()
self.assertEqual(exp, output.getvalue())
output.close()

tools/codechecker/tests/unit/tidy_output_test_files/abs.out

  • This file was added.
/something/fake/test.cpp:8:12: warning: Division by zero [clang-analyzer-core.DivideZero]
return x % 0;
^
/something/fake/test.cpp:8:12: note: Division by zero
return x % 0;
^
/something/fake/test.cpp:8:12: warning: remainder by zero is undefined [clang-diagnostic-division-by-zero]
return x % 0;
^

tools/codechecker/tests/unit/tidy_output_test_files/empty.plist

  • This file was added.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>diagnostics</key>
<array>
</array>
<key>files</key>
<array>
</array>
</dict>
</plist>

tools/codechecker/tests/unit/tidy_output_test_files/empty1.out

  • This file was added.
This is an empty file.

tools/codechecker/tests/unit/tidy_output_test_files/empty2.out

  • This file was added.
This is an empty file.

tools/codechecker/tests/unit/tidy_output_test_files/files/test.cpp

  • This file was added.
#include <iostream>
int main() {
int x;
std::cin >> x;
return x % 0;
}

tools/codechecker/tests/unit/tidy_output_test_files/files/test2.cpp

  • This file was added.
#include <iostream>
int main() {
int x;
int y;
std::cin >> x;
if (false || x) {
return 42;
}
return x % 0;
}

tools/codechecker/tests/unit/tidy_output_test_files/files/test3.hh

  • This file was added.
inline bool foo(bool arg) {
return false || arg;
}
inline void bar(int* x) {
*x = 42;
}

tools/codechecker/tests/unit/tidy_output_test_files/files/test3.cpp

  • This file was added.
#include "test3.hh"
int main(int argc, const char** /*argv*/) {
int* x = 0;
if (foo(argc > 3)) {
bar(x);
}
return 0;
}

tools/codechecker/tests/unit/tidy_output_test_files/tidy1.out

  • This file was added.
files/test.cpp:8:12: warning: Division by zero [clang-analyzer-core.DivideZero]
return x % 0;
^
files/test.cpp:8:12: note: Division by zero
return x % 0;
^
files/test.cpp:8:12: warning: remainder by zero is undefined [clang-diagnostic-division-by-zero]
return x % 0;
^

tools/codechecker/tests/unit/tidy_output_test_files/tidy1.plist

  • This file was added.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>diagnostics</key>
<array>
<dict>
<key>category</key>
<string>clang</string>
<key>check_name</key>
<string>clang-analyzer-core.DivideZero</string>
<key>description</key>
<string>Division by zero</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>8</integer>
</dict>
<key>path</key>
<array>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>8</integer>
</dict>
<key>message</key>
<string>Division by zero</string>
</dict>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>8</integer>
</dict>
<key>message</key>
<string>Division by zero</string>
</dict>
<dict>
<key>edges</key>
<array>
</array>
<key>kind</key>
<string>control</string>
</dict>
</array>
<key>type</key>
<string>clang-tidy</string>
</dict>
<dict>
<key>category</key>
<string>clang</string>
<key>check_name</key>
<string>clang-diagnostic-division-by-zero</string>
<key>description</key>
<string>remainder by zero is undefined</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>8</integer>
</dict>
<key>path</key>
<array>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>8</integer>
</dict>
<key>message</key>
<string>remainder by zero is undefined</string>
</dict>
<dict>
<key>edges</key>
<array>
</array>
<key>kind</key>
<string>control</string>
</dict>
</array>
<key>type</key>
<string>clang-tidy</string>
</dict>
</array>
<key>files</key>
<array>
<string>files/test.cpp</string>
</array>
</dict>
</plist>

tools/codechecker/tests/unit/tidy_output_test_files/tidy2.out

  • This file was added.
files/test2.cpp:5:7: warning: unused variable 'y' [clang-diagnostic-unused-variable]
int y;
^
files/test2.cpp:13:12: warning: Division by zero [clang-analyzer-core.DivideZero]
return x % 0;
^
files/test2.cpp:9:7: note: Left side of '||' is false
if (false || x) {
^
files/test2.cpp:9:3: note: Taking false branch
if (false || x) {
^
files/test2.cpp:13:12: note: Division by zero
return x % 0;
^
files/test2.cpp:13:12: warning: remainder by zero is undefined [clang-diagnostic-division-by-zero]
return x % 0;
^

tools/codechecker/tests/unit/tidy_output_test_files/tidy2.plist

  • This file was added.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>diagnostics</key>
<array>
<dict>
<key>category</key>
<string>clang</string>
<key>check_name</key>
<string>clang-diagnostic-unused-variable</string>
<key>description</key>
<string>unused variable 'y'</string>
<key>location</key>
<dict>
<key>col</key>
<integer>7</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>5</integer>
</dict>
<key>path</key>
<array>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>7</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>5</integer>
</dict>
<key>message</key>
<string>unused variable 'y'</string>
</dict>
<dict>
<key>edges</key>
<array>
</array>
<key>kind</key>
<string>control</string>
</dict>
</array>
<key>type</key>
<string>clang-tidy</string>
</dict>
<dict>
<key>category</key>
<string>clang</string>
<key>check_name</key>
<string>clang-analyzer-core.DivideZero</string>
<key>description</key>
<string>Division by zero</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>13</integer>
</dict>
<key>path</key>
<array>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>13</integer>
</dict>
<key>message</key>
<string>Division by zero</string>
</dict>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>7</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>9</integer>
</dict>
<key>message</key>
<string>Left side of '||' is false</string>
</dict>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>9</integer>
</dict>
<key>message</key>
<string>Taking false branch</string>
</dict>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>13</integer>
</dict>
<key>message</key>
<string>Division by zero</string>
</dict>
<dict>
<key>edges</key>
<array>
<dict>
<key>end</key>
<array>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>9</integer>
</dict>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>9</integer>
</dict>
</array>
<key>start</key>
<array>
<dict>
<key>col</key>
<integer>7</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>9</integer>
</dict>
<dict>
<key>col</key>
<integer>7</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>9</integer>
</dict>
</array>
</dict>
<dict>
<key>end</key>
<array>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>13</integer>
</dict>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>13</integer>
</dict>
</array>
<key>start</key>
<array>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>9</integer>
</dict>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>9</integer>
</dict>
</array>
</dict>
</array>
<key>kind</key>
<string>control</string>
</dict>
</array>
<key>type</key>
<string>clang-tidy</string>
</dict>
<dict>
<key>category</key>
<string>clang</string>
<key>check_name</key>
<string>clang-diagnostic-division-by-zero</string>
<key>description</key>
<string>remainder by zero is undefined</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>13</integer>
</dict>
<key>path</key>
<array>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>13</integer>
</dict>
<key>message</key>
<string>remainder by zero is undefined</string>
</dict>
<dict>
<key>edges</key>
<array>
</array>
<key>kind</key>
<string>control</string>
</dict>
</array>
<key>type</key>
<string>clang-tidy</string>
</dict>
</array>
<key>files</key>
<array>
<string>files/test2.cpp</string>
</array>
</dict>
</plist>

tools/codechecker/tests/unit/tidy_output_test_files/tidy3.out

  • This file was added.
files/test3.cpp:4:12: warning: use nullptr [modernize-use-nullptr]
int* x = 0;
^
nullptr
files/test3.hh:6:6: warning: Dereference of null pointer (loaded from variable 'x') [clang-analyzer-core.NullDereference]
*x = 42;
^
files/test3.cpp:4:3: note: 'x' initialized to a null pointer value
int* x = 0;
^
files/test3.cpp:6:11: note: Assuming 'argc' is > 3
if (foo(argc > 3)) {
^
files/test3.cpp:6:3: note: Taking true branch
if (foo(argc > 3)) {
^
files/test3.cpp:7:9: note: Passing null pointer value via 1st parameter 'x'
bar(x);
^
files/test3.cpp:7:5: note: Calling 'bar'
bar(x);
^
files/test3.hh:6:6: note: Dereference of null pointer (loaded from variable 'x')
*x = 42;
^

tools/codechecker/tests/unit/tidy_output_test_files/tidy3.plist

  • This file was added.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>diagnostics</key>
<array>
<dict>
<key>category</key>
<string>modernize</string>
<key>check_name</key>
<string>modernize-use-nullptr</string>
<key>description</key>
<string>use nullptr</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>4</integer>
</dict>
<key>path</key>
<array>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>4</integer>
</dict>
<key>message</key>
<string>use nullptr</string>
</dict>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>12</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>4</integer>
</dict>
<key>message</key>
<string>nullptr (fixit)</string>
</dict>
<dict>
<key>edges</key>
<array>
</array>
<key>kind</key>
<string>control</string>
</dict>
</array>
<key>type</key>
<string>clang-tidy</string>
</dict>
<dict>
<key>category</key>
<string>clang</string>
<key>check_name</key>
<string>clang-analyzer-core.NullDereference</string>
<key>description</key>
<string>Dereference of null pointer (loaded from variable 'x')</string>
<key>location</key>
<dict>
<key>col</key>
<integer>6</integer>
<key>file</key>
<integer>1</integer>
<key>line</key>
<integer>6</integer>
</dict>
<key>path</key>
<array>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>6</integer>
<key>file</key>
<integer>1</integer>
<key>line</key>
<integer>6</integer>
</dict>
<key>message</key>
<string>Dereference of null pointer (loaded from variable 'x')</string>
</dict>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>4</integer>
</dict>
<key>message</key>
<string>'x' initialized to a null pointer value</string>
</dict>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>11</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>6</integer>
</dict>
<key>message</key>
<string>Assuming 'argc' is &gt; 3</string>
</dict>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>6</integer>
</dict>
<key>message</key>
<string>Taking true branch</string>
</dict>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>9</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>7</integer>
</dict>
<key>message</key>
<string>Passing null pointer value via 1st parameter 'x'</string>
</dict>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>5</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>7</integer>
</dict>
<key>message</key>
<string>Calling 'bar'</string>
</dict>
<dict>
<key>depth</key>
<integer>0</integer>
<key>kind</key>
<string>event</string>
<key>location</key>
<dict>
<key>col</key>
<integer>6</integer>
<key>file</key>
<integer>1</integer>
<key>line</key>
<integer>6</integer>
</dict>
<key>message</key>
<string>Dereference of null pointer (loaded from variable 'x')</string>
</dict>
<dict>
<key>edges</key>
<array>
<dict>
<key>end</key>
<array>
<dict>
<key>col</key>
<integer>11</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>6</integer>
</dict>
<dict>
<key>col</key>
<integer>11</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>6</integer>
</dict>
</array>
<key>start</key>
<array>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>4</integer>
</dict>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>4</integer>
</dict>
</array>
</dict>
<dict>
<key>end</key>
<array>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>6</integer>
</dict>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>6</integer>
</dict>
</array>
<key>start</key>
<array>
<dict>
<key>col</key>
<integer>11</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>6</integer>
</dict>
<dict>
<key>col</key>
<integer>11</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>6</integer>
</dict>
</array>
</dict>
<dict>
<key>end</key>
<array>
<dict>
<key>col</key>
<integer>9</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>7</integer>
</dict>
<dict>
<key>col</key>
<integer>9</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>7</integer>
</dict>
</array>
<key>start</key>
<array>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>6</integer>
</dict>
<dict>
<key>col</key>
<integer>3</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>6</integer>
</dict>
</array>
</dict>
<dict>
<key>end</key>
<array>
<dict>
<key>col</key>
<integer>5</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>7</integer>
</dict>
<dict>
<key>col</key>
<integer>5</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>7</integer>
</dict>
</array>
<key>start</key>
<array>
<dict>
<key>col</key>
<integer>9</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>7</integer>
</dict>
<dict>
<key>col</key>
<integer>9</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>7</integer>
</dict>
</array>
</dict>
<dict>
<key>end</key>
<array>
<dict>
<key>col</key>
<integer>6</integer>
<key>file</key>
<integer>1</integer>
<key>line</key>
<integer>6</integer>
</dict>
<dict>
<key>col</key>
<integer>6</integer>
<key>file</key>
<integer>1</integer>
<key>line</key>
<integer>6</integer>
</dict>
</array>
<key>start</key>
<array>
<dict>
<key>col</key>
<integer>5</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>7</integer>
</dict>
<dict>
<key>col</key>
<integer>5</integer>
<key>file</key>
<integer>0</integer>
<key>line</key>
<integer>7</integer>
</dict>
</array>
</dict>
</array>
<key>kind</key>
<string>control</string>
</dict>
</array>
<key>type</key>
<string>clang-tidy</string>
</dict>
</array>
<key>files</key>
<array>
<string>files/test3.cpp</string>
<string>files/test3.hh</string>
</array>
</dict>
</plist>

tools/codechecker/thrift_api/report_storage_server.thrift

  • This file was added.
// -*- coding: utf-8 -*-
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
include "shared.thrift"
namespace py DBThriftAPI
struct NeedFileResult {
1: bool needed;
2: i64 fileId;
}
struct SuppressBugData {
1: string bug_hash,
2: string file_name,
3: string comment
}
typedef list<SuppressBugData> SuppressBugList
// The order of the functions inditaces the order that must be maintained when
// calling into the server.
service CheckerReport {
// store checker run related data to the database
// by default updates the results if name was already found
// using the force flag removes existing analysis results for a run
i64 addCheckerRun(
1: string command,
2: string name,
3: string version,
4: bool force)
throws (1: shared.RequestFailed requestError),
bool replaceConfigInfo(
1: i64 run_id,
2: shared.CheckerConfigList values)
throws (1: shared.RequestFailed requestError),
bool addSuppressBug(
1: i64 run_id,
2: SuppressBugList bugsToSuppress
)
throws (1: shared.RequestFailed requestError),
# remove all suppress information from the database
bool cleanSuppressData(
1: i64 run_id,
)
throws (1: shared.RequestFailed requestError),
# the map contains a path and a comment (can be empty)
bool addSkipPath(
1: i64 run_id,
2: map<string, string> paths)
throws (1: shared.RequestFailed requestError),
// The next few following functions must be called via the same connection.
// =============================================================
i64 addBuildAction(
1: i64 run_id,
2: string build_cmd,
3: string check_cmd,
4: string analyzer_type,
5: string analyzed_source_file)
throws (1: shared.RequestFailed requestError),
i64 addReport(
1: i64 build_action_id,
2: i64 file_id,
3: string bug_hash,
4: string checker_message,
5: shared.BugPath bugpath,
6: shared.BugPathEvents events,
7: string checker_id,
8: string checker_cat,
9: string bug_type,
10: shared.Severity severity,
11: bool suppress)
throws (1: shared.RequestFailed requestError),
bool finishBuildAction(
1: i64 action_id,
2: string failure)
throws (1: shared.RequestFailed requestError),
NeedFileResult needFileContent(
1: i64 run_id,
2: string filepath)
throws (1: shared.RequestFailed requestError),
bool addFileContent(
1: i64 file_id,
2: binary file_content)
throws (1: shared.RequestFailed requestError),
bool finishCheckerRun(1: i64 run_id)
throws (1: shared.RequestFailed requestError),
bool stopServer()
throws (1: shared.RequestFailed requestError)
}

tools/codechecker/thrift_api/report_viewer_server.thrift

  • This file was added.
// -*- coding: utf-8 -*-
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
// =================================================
// !!! Update version information if api changes!!!
// =================================================
// backward incompatible changes should increase major version
// other changes should only increase minor version
include "shared.thrift"
namespace py codeCheckerDBAccess
namespace js codeCheckerDBAccess
namespace cpp cc.service.codechecker
//=================================================
const string API_VERSION = '4.3'
const i64 MAX_QUERY_SIZE = 500
//=================================================
//-----------------------------------------------------------------------------
struct RunData{
1: i64 runId, // unique id of the run
2: string runDate, // date of the run
3: string name, // human readable name of the run
4: i64 duration, // duration of the run; -1 if not finished
5: i64 resultCount, // number of results in the run
6: string runCmd, // the used check command
7: optional bool can_delete // true if codeCheckerDBAccess::removeRunResults()
// is allowed on this run (see issue 151)
}
typedef list<RunData> RunDataList
//-----------------------------------------------------------------------------
struct ReportData{
1: string checkerId, // the qualified id of the checker that reported this
2: string bugHash, // This is unique id of the concrete report.
3: string checkedFile, // this is a filepath
4: string checkerMsg, // description of the bug report
5: i64 reportId, // id of the report in the current run in the db
6: bool suppressed, // true if the bug is suppressed
7: i64 fileId, // unique id of the file the report refers to
8: shared.BugPathEvent lastBugPosition // This contains the range and message of the last item in the symbolic
// execution step list.
9: shared.Severity severity // checker severity
10: optional string suppressComment // suppress commment if report is suppressed
}
typedef list<ReportData> ReportDataList
//-----------------------------------------------------------------------------
/**
* Members of this struct are interpreted in "AND" relation with each other.
* So they need to match a single report at the same time.
*/
struct ReportFilter{
1: optional string filepath, // In the filters a single wildcard can be be used: *
2: optional string checkerMsg,
3: optional shared.Severity severity,
4: optional string checkerId, // should filter in the fully qualified checker id name such as alpha.core.
// the analyzed system. Projects can optionally use this concept.
5: optional bool suppressed = false // if the bug state is suppressed
}
/**
* If there is a list of ReportFilter, there is an OR relation between the list members.
*/
typedef list<ReportFilter> ReportFilterList
//-----------------------------------------------------------------------------
struct ReportDetails{
1: shared.BugPathEvents pathEvents,
2: shared.BugPath executionPath
}
//-----------------------------------------------------------------------------
struct ReportFileData{
1: i64 reportFileId,
2: string reportFileContent
}
//-----------------------------------------------------------------------------
// default sorting of the results
enum SortType {
FILENAME,
CHECKER_NAME,
SEVERITY
}
//-----------------------------------------------------------------------------
struct SourceFileData{
1: i64 fileId,
2: string filePath,
3: optional string fileContent
}
//-----------------------------------------------------------------------------
enum Order {
ASC,
DESC
}
//-----------------------------------------------------------------------------
struct SortMode{
1: SortType type,
2: Order ord
}
//-----------------------------------------------------------------------------
struct ReportDataTypeCount{
1: string checkerId,
2: shared.Severity severity,
3: i64 count
}
typedef list<ReportDataTypeCount> ReportDataTypeCountList
//-----------------------------------------------------------------------------
struct SkipPathData{
1: string path,
2: string comment
}
typedef list<SkipPathData> SkipPathDataList
//-----------------------------------------------------------------------------
// diff result types
enum DiffType {
NEW,
RESOLVED,
UNRESOLVED
}
//-----------------------------------------------------------------------------
service codeCheckerDBAccess {
// get the run Ids and dates from the database to select one run
RunDataList getRunData()
throws (1: shared.RequestFailed requestError),
ReportData getReport(
1: i64 reportId)
throws (1: shared.RequestFailed requestError),
// get the results for one runId
ReportDataList getRunResults(
1: i64 runId,
2: i64 limit,
3: i64 offset,
4: list<SortMode> sortType,
5: ReportFilterList reportFilters)
throws (1: shared.RequestFailed requestError),
// count all the results for one run
i64 getRunResultCount(
1: i64 runId,
2: ReportFilterList reportFilters)
throws (1: shared.RequestFailed requestError),
// gives back the all marked region and message for a report
ReportDetails getReportDetails(
1: i64 reportId)
throws (1: shared.RequestFailed requestError),
// get file informations if fileContent is true the content of the source file
// will be also returned
SourceFileData getSourceFileData(
1: i64 fileId,
2: bool fileContent)
throws (1: shared.RequestFailed requestError),
// get the file id from the database for a filepath, returns -1 if not found
i64 getFileId(1: i64 runId,
2: string path)
throws (1: shared.RequestFailed requestError),
// suppress the bug
bool suppressBug(1: list<i64> runIds,
2: i64 reportId,
3: string comment)
throws (1: shared.RequestFailed requestError),
// unsuppress the bug
bool unSuppressBug(1: list<i64> runIds,
2: i64 reportId)
throws (1: shared.RequestFailed requestError),
// get the md documentation for a checker
string getCheckerDoc(1: string checkerId)
throws (1: shared.RequestFailed requestError),
// compare the results of two runs
ReportDataList getNewResults(1: i64 base_run_id,
2: i64 new_run_id,
3: i64 limit,
4: i64 offset,
5: list<SortMode> sortType,
6: ReportFilterList reportFilters )
throws (1: shared.RequestFailed requestError),
ReportDataList getResolvedResults(1: i64 base_run_id,
2: i64 new_run_id,
3: i64 limit,
4: i64 offset,
5: list<SortMode> sortType,
6: ReportFilterList reportFilters )
throws (1: shared.RequestFailed requestError),
ReportDataList getUnresolvedResults(1: i64 base_run_id,
2: i64 new_run_id,
3: i64 limit,
4: i64 offset,
5: list<SortMode> sortType,
6: ReportFilterList reportFilters )
throws (1: shared.RequestFailed requestError),
// get the checker configuration values
shared.CheckerConfigList getCheckerConfigs(1: i64 runId)
throws (1: shared.RequestFailed requestError),
// get the skip list of paths
SkipPathDataList getSkipPaths(1: i64 runId)
throws (1: shared.RequestFailed requestError),
// gives back the build Actions that generate the given report.
// multiple build actions can belong to a report in a header.
list<string> getBuildActions(1: i64 reportId)
throws (1: shared.RequestFailed requestError),
// get all the results for one runId
// count all results for a checker
ReportDataTypeCountList getRunResultTypes(1: i64 runId,
2: ReportFilterList reportFilters)
throws (1: shared.RequestFailed requestError),
// returns the database access handler api version
string getAPIVersion();
// remove bug results from the database
bool removeRunResults(1: list<i64> runIds)
throws (1: shared.RequestFailed requestError),
// get the suppress file path set by the command line
// returns empty string if not set
string getSuppressFile()
throws (1: shared.RequestFailed requestError),
// count the diff results
i64 getDiffResultCount(1: i64 base_run_id,
2: i64 new_run_id,
3: DiffType diff_type,
4: ReportFilterList reportFilters)
throws (1: shared.RequestFailed requestError),
// count all the diff results for each checker
ReportDataTypeCountList getDiffResultTypes(1: i64 base_run_id,
2: i64 new_run_id,
3: DiffType diff_type,
4: ReportFilterList reportFilters)
throws (1: shared.RequestFailed requestError)
}

tools/codechecker/thrift_api/shared.thrift

  • This file was added.
// -*- coding: utf-8 -*-
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//-----------------------------------------------------------------------------
struct BugPathEvent {
1: i64 startLine,
2: i64 startCol,
3: i64 endLine,
4: i64 endCol,
5: string msg,
6: i64 fileId
7: string filePath
}
typedef list<BugPathEvent> BugPathEvents
//-----------------------------------------------------------------------------
struct BugPathPos {
1: i64 startLine,
2: i64 startCol,
3: i64 endLine,
4: i64 endCol,
5: i64 fileId
6: string filePath
}
typedef list<BugPathPos> BugPath
//-----------------------------------------------------------------------------
struct ConfigValue {
1: string checker_name,
2: string attribute,
3: string value
}
typedef list<ConfigValue> CheckerConfigList
//-----------------------------------------------------------------------------
enum Severity{
UNSPECIFIED = 0,
STYLE = 10,
LOW = 20,
MEDIUM = 30,
HIGH = 40,
CRITICAL = 50
}
//-----------------------------------------------------------------------------
enum ErrorCode{
DATABASE,
IOERROR,
GENERAL
}
//-----------------------------------------------------------------------------
exception RequestFailed {
1: ErrorCode error_code,
2: string message
}

tools/codechecker/thrift_api/thrift_api.md

  • This file was added.
#Thrift APIs
These APIs should be used by the clients using the database to store or to get the results. Any new client should only interact with the database through these APIs.
## Report viewer server API
The report viewer server API should be used by any client to view the check results.
## Report storage server API
The report storage server API is used internally in the package during runtime to store the results to the database.