scan-build: Add --analyzer-target option

[honggyu.kim]

Summary

Currently scan-build cannot pass target triple information to clang.
It makes analysis failure when build script has target specific compiler flags.
This patch adds --analyzer-target option to support analysis for other targets.

--analyzer-target [target triple name for analysis]
--analyzer-target=[target triple name for analysis]

This provides target triple information to clang static analyzer.
It only changes the target for analysis and doesn't change the target of a
real compiler given by --use-cc and --use-c++ options.

[honggyu.kim]

Hi all,

I wrote this patch because I had some problem generating a clang static analysis report for ARM Linux kernel code. Since it uses
some target specific flags for ARM, I got the following error message during analysis.

error: unknown target CPU 'armv7-a'

I have checked scan-build script, but there was no way to pass target triple information to clang, so clang thinks that the source
code can be compiled for host machine, x86_64. This patch adds --triple option to fix this problem.

With this patch, I was able to generate clang static analysis report for ARM Linux kernel code with the following command:
$ scan-build --use-cc=arm-linux-gcc --triple=arm-linux-gnueabi make ARCH=arm CROSS_COMPILE=arm-linux-

Please let me know your opinion.
I appreciate all your comment.

Thanks,
Honggyu

[honggyu.kim]

Hi Anna,

I have updated the patch with full diff as you requested.
But I cannot find Anton Yartsev here in phabricator so I cannot add him as a reviewer.
Can you please add him if you can find him?
Thanks very much for your comment.

Honggyu

[honggyu.kim]

I also tested this patch with the following example.

$ cat -n test.c
     1  int main()
     2  {
     3    int a;
     4  #if __arm__
     5    int b = a;
     6  #endif
     7    return a;
     8  }

As you can see above, the example contains arm specific code region with arm macro in #if statement.
If I run it with scan-build the result doesn't contain the bug for variable b since it is within arm specific code region.

$ scan-build clang test.c
scan-build: Using '/home/hong.gyu.kim/work.hard/clang/install/bin/clang' for static analysis
test.c:7:3: warning: Undefined or garbage value returned to caller
  return a;
  ^~~~~~~~
1 warning generated.
scan-build: 1 bug found.
scan-build: Run 'scan-view /tmp/scan-build-2015-07-31-002115-17505-1' to examine bug reports.

But when I run it with --triple option with arm target triple, it correctly shows the bug from variable b.

$ scan-build --triple=arm-linux-gnueabi clang test.c
scan-build: Using '/home/hong.gyu.kim/work.hard/clang/install/bin/clang' for static analysis
test.c:5:3: warning: Assigned value is garbage or undefined
  int b = a;
  ^~~~~   ~
test.c:5:7: warning: Value stored to 'b' during its initialization is never read
  int b = a;
      ^   ~
2 warnings generated.
scan-build: 2 bugs found.
scan-build: Run 'scan-view /tmp/scan-build-2015-07-31-002128-17518-1' to examine bug reports.

[krememek]

tools/scan-build/scan-build

1167

This is the only real documentation provided to the user to understand what this option does. It is a completely optional option, and I feel it needs a bit more clarification. Perhaps just a second sentence to say "use this when you want to analyze for a different architecture than the default". Also, it isn't clear to the user if this effects both static analysis and compilation, or just compilation. For example, is the expectation that compilation gets the normal flags without this trip, and that static analysis gets this triple?

There's also many cases where this flag won't work. Instead of having a specific flag for target triple, maybe just an "extra flags to pass to the compiler" would be sufficient (and more general).

[honggyu.kim]

I have updated the new patchset with more information in help message and changed the option name to —analyzer-target.
If you think the "extra flags to pass to the compiler” is still needed, I will write another patch for that.
I appreciate your comment and suggestion.

tools/scan-build/scan-build

1167

As you know, the basic sequence of scan-build comprises into analysis and compilation.
To properly analyse the source code, it is required to use the same target for analysis and compilation.
Otherwise, build scripts wouldn’t work in some cases due to target specific flags and some code couldn’t be compiled if the code contains target specific inline assembly code.

Ideally, I hope to automatically extract the target triple from the actual compiler, but it seems that there’s no standard way to extract the target triple from all the existing compilers including clang, gcc, icc, etc.

I added this option explaining target information for analysis has to be provided explicitly.
We can also consider to add --extra-clang-flags option in a general way, but users may not notice that they have to actually provide the target information only for analysis when they do cross compilation. I was also very confused at first.

[krememek]

In D10356#216741, @honggyu.kim wrote:

I have updated the new patchset with more information in help message and changed the option name to —analyzer-target.
If you think the "extra flags to pass to the compiler” is still needed, I will write another patch for that.
I appreciate your comment and suggestion.

What I am getting at is that I’m not convinced this is a good option to add to scan-build. Doing cross-compilation accurately involves many factors beyond just passing the target triple. The build itself probably needs to be setup to do a cross-compile. If that is the case, this really seems like jerry-rigging something into scan-build to facilitate a set of experiments, but not necessarily enable a new workflow for analyzing cross-compiled projects.

[krememek]

In D10356#186078, @honggyu.kim wrote:

Hi all,

I wrote this patch because I had some problem generating a clang static analysis report for ARM Linux kernel code. Since it uses
some target specific flags for ARM, I got the following error message during analysis.

error: unknown target CPU 'armv7-a'

I have checked scan-build script, but there was no way to pass target triple information to clang, so clang thinks that the source
code can be compiled for host machine, x86_64. This patch adds --triple option to fix this problem.

With this patch, I was able to generate clang static analysis report for ARM Linux kernel code with the following command:
$ scan-build --use-cc=arm-linux-gcc --triple=arm-linux-gnueabi make ARCH=arm CROSS_COMPILE=arm-linux-

Please let me know your opinion.
I appreciate all your comment.

Thanks,
Honggyu

Is this an issue that the clang used for static analysis does not understand the target triple used to build Linux (i.e., "'armv7-a")? Shouldn't we just fix that by adding that target triple to Clang?

I also don't understand this point:

I have checked scan-build script, but there was no way to pass target triple information to clang, so clang thinks that the source
code can be compiled for host machine, x86_64. This patch adds --triple option to fix this problem.

Which 'clang' are you talking about? The clang used for static analysis? Are you saying because clang doesn't recognize the target it tries to compile for the host machine instead, and thus the analysis is wrong?

[honggyu.kim]

Hi Ted,

Let's me show you the full command and the error with verbose message.

If I run the following command with previous scan-build (i.e. without --analyzer-target option)
$ scan-build -v -v -v --use-cc=arm-linux-gnueabi-gcc make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-

...
CC      arch/arm/kernel/opcodes.o

First, it does the normal compilation with given compiler, "arm-linux-gnueabi-gcc", as follows:

arm-linux-gnueabi-gcc -Wp,-MD,arch/arm/kernel/.opcodes.o.d -nostdinc -isystem /opt/toolchain/arm-linux-gnueabi-4.8-2014.11-x86_64/bin/../lib/gcc/arm-linux-gnueabi/4.8.3/include -I./arch/arm/include -Iarch/arm/include/generated/uapi -Iarch/arm/include/generated -Iinclude -I./arch/arm/include/uapi -Iarch/arm/include/generated/uapi -I./include/uapi -Iinclude/generated/uapi -include ./include/linux/kconfig.h -DKERNEL -mlittle-endian -Iarch/arm/mach-xxxx/include -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -std=gnu89 -fno-dwarf2-cfi-asm -mabi=aapcs-linux -mno-thumb-interwork -mfpu=vfp -funwind-tables -marm -DLINUX_ARM_ARCH=7 -march=armv7-a -msoft-float -Uarm -fno-delete-null-pointer-checks -O2 --param=allow-store-data-races=0 -Wframe-larger-than=1024 -fno-stack-protector -Wno-unused-but-set-variable -fomit-frame-pointer -fno-var-tracking-assignments -g -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -Werror=implicit-int -Werror=strict-prototypes -DCC_HAVE_ASM_GOTO -DKBUILD_STR(s)=#s -DKBUILD_BASENAME=KBUILD_STR(opcodes) -DKBUILD_MODNAME=KBUILD_STR(opcodes) -c -o arch/arm/kernel/opcodes.o arch/arm/kernel/opcodes.c

Then, it does analysis with clang static analyzer as follows:

[LOCATION]: /home/hong.gyu.kim/work.hard/kernel/clang/linux
#SHELL (cd '/home/hong.gyu.kim/work.hard/kernel/clang/linux' && '/home/hong.gyu.kim/usr/bin/clang' '-cc1' '-triple' 'x86_64-unknown-linux-gnu' '-analyze' '-disable-free' '-disable-llvm-verifier' '-main-file-name' 'opcodes.c' '-analyzer-store=region' '-analyzer-opt-analyze-nested-blocks' '-analyzer-eagerly-assume' '-analyzer-checker=core' '-analyzer-checker=unix' '-analyzer-checker=deadcode' '-analyzer-checker=security.insecureAPI.UncheckedReturn' '-analyzer-checker=security.insecureAPI.getpw' '-analyzer-checker=security.insecureAPI.gets' '-analyzer-checker=security.insecureAPI.mktemp' '-analyzer-checker=security.insecureAPI.mkstemp' '-analyzer-checker=security.insecureAPI.vfork' '-analyzer-output' 'plist' '-w' '-mrelocation-model' 'static' '-mthread-model' 'posix' '-relaxed-aliasing' '-fmath-errno' '-masm-verbose' '-mconstructor-aliases' '-munwind-tables' '-fuse-init-array' '-target-cpu' 'armv7-a' '-no-implicit-float' '-target-linker-version' '2.22' '-momit-leaf-frame-pointer' '-dwarf-column-info' '-nostdsysteminc' '-nobuiltininc' '-resource-dir' '/home/hong.gyu.kim/usr/bin/../lib/clang/3.7.0' '-isystem' '/opt/toolchain/arm-linux-gnueabi-4.8-2014.11-x86_64/bin/../lib/gcc/arm-linux-gnueabi/4.8.3/include' '-include' './include/linux/kconfig.h' '-D' 'KERNEL' '-D' 'LINUX_ARM_ARCH=7' '-U' 'arm' '-D' 'CC_HAVE_ASM_GOTO' '-D' 'KBUILD_STR(s)=#s' '-D' 'KBUILD_BASENAME=KBUILD_STR(opcodes)' '-D' 'KBUILD_MODNAME=KBUILD_STR(opcodes)' '-I' './arch/arm/include' '-I' 'arch/arm/include/generated/uapi' '-I' 'arch/arm/include/generated' '-I' 'include' '-I' './arch/arm/include/uapi' '-I' 'arch/arm/include/generated/uapi' '-I' './include/uapi' '-I' 'include/generated/uapi' '-I' 'arch/arm/mach-xxxx/include' '-O2' '-Wno-trigraphs' '-Wno-format-security' '-Wno-unused-but-set-variable' '-Wno-pointer-sign' '-std=gnu89' '-fdebug-compilation-dir' '/home/hong.gyu.kim/work.hard/kernel/clang/linux' '-ferror-limit' '19' '-fmessage-length' '0' '-fwrapv' '-mstackrealign' '-fobjc-runtime=gcc' '-fno-common' '-fdiagnostics-show-option' '-vectorize-loops' '-vectorize-slp' '-analyzer-display-progress' '-analyzer-output=html' '-o' '/tmp/scan-build-2015-08-04-210454-16108-1' '-x' 'c' 'arch/arm/kernel/opcodes.c')

And this shows an error.

error: unknown target CPU 'armv7-a'

If I just simplify the log message, it looks like this:

1. compilation with arm-linux-gnueabi-gcc

arm-linux-gnueabi-gcc ... -march=armv7-a ... -c -o arch/arm/kernel/opcodes.o arch/arm/kernel/opcodes.c

As you can see the above, there's no problem for compilation.

2. analysis with clang static analyzer

'clang' '-cc1' '-triple' 'x86_64-unknown-linux-gnu' '-analyze' ... '-target-cpu' 'armv7-a' ... 'arch/arm/kernel/opcodes.c'

In analysis step, clang uses the default -triple option "x86_64-unknown-linux-gnu", but still uses the the same arguments including "-target-cpu armv7-a". It makes a conflict and shows "error: unknown target CPU 'armv7-a'".

With --analyzer-target option, we can provide target info to clang static analyzer as follows:

$ scan-build -v -v -v --analyzer-target=arm-linux-gnueabi --use-cc=arm-linux-gnueabi-gcc make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-

...
CC      arch/arm/kernel/opcodes.o

1. compilation works the same as before

arm-linux-gnueabi-gcc ... -march=armv7-a ... -c -o arch/arm/kernel/opcodes.o arch/arm/kernel/opcodes.c

2. analysis uses the target triple that is given by --analyzer-target option and analysis works fine then.

'clang' '-cc1' '-triple' 'armv7--linux-gnueabi' '-analyze' ... '-target-cpu' 'cortex-a8' ... 'arch/arm/kernel/opcodes.c')

So we have to use the same target triple for compilation and analysis by using this option. Please let me know if there's a better way to fix this issue.

[honggyu.kim]

Here's one more example.

$ cat -n test.c
     1  int main(int argc, char** argv) {
     2    int a;
     3  #if __arm__
     4    int *p = 0;
     5    if (argc == 3)
     6      a = *p;
     7  #endif
     8    return a;
     9  }

If we cross compile this code with the following command:

$ scan-build --use-cc=arm-linux-gnueabi-gcc arm-linux-gnueabi-gcc test.c
scan-build: Using '/home/hong.gyu.kim/usr/bin/clang' for static analysis
test.c:8:3: warning: Undefined or garbage value returned to caller
  return a;
  ^~~~~~~~
1 warning generated.
scan-build: 1 bug found.
scan-build: Run 'scan-view /tmp/scan-build-2015-08-04-215933-29482-1' to examine bug reports.

I detected a bug but in #if arm region, null point dereference can happen and there's no way to detect it with current scan-build implementation.
With --analyzer-target option, we can detect those target dependent code region properly as below:

$ scan-build --analyzer-target=arm --use-cc=arm-linux-gnueabi-gcc arm-linux-gnueabi-gcc test.c
scan-build: Using '/home/hong.gyu.kim/usr/bin/clang' for static analysis
test.c:6:9: warning: Dereference of null pointer (loaded from variable 'p')
    a = *p;
        ^~
test.c:8:3: warning: Undefined or garbage value returned to caller
  return a;
  ^~~~~~~~
2 warnings generated.
scan-build: 2 bugs found.
scan-build: Run 'scan-view /tmp/scan-build-2015-08-04-215948-29652-1' to examine bug reports.

Now, "Dereference of null pointer" bug is found.

[krememek]

In D10356#217564, @honggyu.kim wrote:

With --analyzer-target option, we can provide target info to clang static analyzer as follows:

$ scan-build -v -v -v --analyzer-target=arm-linux-gnueabi --use-cc=arm-linux-gnueabi-gcc make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi-

...
CC      arch/arm/kernel/opcodes.o

1. compilation works the same as before

arm-linux-gnueabi-gcc ... -march=armv7-a ... -c -o arch/arm/kernel/opcodes.o arch/arm/kernel/opcodes.c

2. analysis uses the target triple that is given by --analyzer-target option and analysis works fine then.

'clang' '-cc1' '-triple' 'armv7--linux-gnueabi' '-analyze' ... '-target-cpu' 'cortex-a8' ... 'arch/arm/kernel/opcodes.c')

So we have to use the same target triple for compilation and analysis by using this option. Please let me know if there's a better way to fix this issue.

I see. The problem is that the ad hoc interposition that scan-build uses does not handle the case that the underlying compiler is built as a cross-compiler, but does not take a direct target triple. That compiler only builds for the other target, unlike clang which is a cross-compiler.

In that case, I am fine with taking this change. That you for explaining it.

[honggyu.kim]

In D10356#219499, @krememek wrote:

I see. The problem is that the ad hoc interposition that scan-build uses does not handle the case that the underlying compiler is built as a cross-compiler, but does not take a direct target triple. That compiler only builds for the other target, unlike clang which is a cross-compiler.

That's exactly what I was going to explain. I hope that scan-build can automatically extract the target triple information from the given compiler, but there's no way to do it as of yet. That's the reason I made this option to explicitly set the target triple info for analysis.

In that case, I am fine with taking this change. That you for explaining it.

Thanks for the review and comments. If you want to change the option name, --analyzer-target, I'm okay to change it. And I would also like to hear some feedback about the description for the option as well.
So please let me know if you have any other idea on that. I appreciate your help.

[krememek]

Committed r244400

[honggyu.kim]

In D10356#219772, @krememek wrote:

Committed r244400

Thanks for accepting it!

tools/scan-build/ccc-analyzer

sub SearchInPath {
     return 0;
 }
 
 my $Compiler;
 my $Clang;
 my $DefaultCCompiler;
 my $DefaultCXXCompiler;
 my $IsCXX;
+my $AnalyzerTarget;
 
 # If on OSX, use xcrun to determine the SDK root.
 my $UseXCRUN = 0;
 
 if (`uname -a` =~ m/Darwin/) {
   $DefaultCCompiler = 'clang';
   $DefaultCXXCompiler = 'clang++';
   # Older versions of OSX do not have xcrun to
else {
   if (!defined $Compiler || (! -x $Compiler && ! SearchInPath($Compiler))) { $Compiler = $DefaultCCompiler; }
 
   $Clang = $ENV{'CLANG'};
   if (!defined $Clang || ! -x $Clang) { $Clang = 'clang'; }
 
   $IsCXX = 0
 }
 
+$AnalyzerTarget = $ENV{'CLANG_ANALYZER_TARGET'};
+
 ##===----------------------------------------------------------------------===##
 # Cleanup.
 ##===----------------------------------------------------------------------===##
 
 my $ReportFailures = $ENV{'CCC_REPORT_FAILURES'};
 if (!defined $ReportFailures) { $ReportFailures = 1; }
 
 my $CleanupFile;
foreach my $arg (@$AnalyzeArgs) {
       push @Args, "-Xclang", $arg;
     }
 
     # Display Ubiviz graph?
     if (defined $ENV{'CCC_UBI'}) {
       push @Args, "-Xclang", "-analyzer-viz-egraph-ubigraph";
     }
 
+    if (defined $AnalyzerTarget) {
+      push @Args, "-target", $AnalyzerTarget;
+    }
+
     my $AnalysisArgs = GetCCArgs($HtmlDir, "--analyze", \@Args);
     @CmdArgs = @$AnalysisArgs;
   }
 
   my @PrintArgs;
   my $dir;
 
   if ($Verbose) {

tools/scan-build/scan-build

--use-cc=[compiler path]
    Because of the current implementation of interposition, scan-build does not
    know what compiler your project normally uses.  Instead, it simply overrides
    the CC environment variable, and guesses your default compiler.
 
    In the future, this interposition mechanism to be improved, but if you need
    scan-build to use a specific compiler for *compilation* then you can use
    this option to specify a path to that compiler.
 
+   If the given compiler is a cross compiler, you may also need to provide
+   --analyzer-target option to properly analyze the source code because static
+   analyzer runs as if the code is compiled for the host machine by default.
+
  --use-c++ [compiler path]
  --use-c++=[compiler path]
 
-   This is the same as "-use-cc" but for C++ code.
+   This is the same as "--use-cc" but for C++ code.
+
+ --analyzer-target [target triple name for analysis]
+ --analyzer-target=[target triple name for analysis]
+
+   This provides target triple information to clang static analyzer.
+   It only changes the target for analysis but doesn't change the target of a
+   real compiler given by --use-cc and --use-c++ options.
 
  -v
 
    Enable verbose output from scan-build. A second and third '-v' increases
    verbosity.
 
  -V
  --view
 
    View analysis results in a web browser when the build completes.
 
 ADVANCED OPTIONS:
 
if ($arg =~ /^--use-c\+\+(=(.+))?$/) {
     else {
       $cxx = $2;
     }
 
     $ENV{"CCC_CXX"} = $cxx;
     next;
   }
 
+  if ($arg =~ /^--analyzer-target(=(.+))?$/) {
+    shift @ARGV;
+    my $AnalyzerTarget;
+
+    if (!defined $2 || $2 eq "") {
+      if (!@ARGV) {
+        DieDiag("'--analyzer-target' option requires a target triple name.\n");
+      }
+      $AnalyzerTarget = shift @ARGV;
+    }
+    else {
+      $AnalyzerTarget = $2;
+    }
+
+    $ENV{"CLANG_ANALYZER_TARGET"} = $AnalyzerTarget;
+    next;
+  }
+
   if ($arg eq "-v") {
     shift @ARGV;
     $Verbose++;
     next;
   }
 
   if ($arg eq "-V" or $arg eq "--view") {
     shift @ARGV;