[analyzer][CrossTU][NFC] Generalize to external definitions instead of external functions

[r.stahl]

Summary

This is just changing naming and documentation to be general about external definitions that can be imported for cross translation unit analysis. There is at least a plan to add VarDecls: D46421

[xazax.hun]

Some nits inline. Otherwise looks good to me.

include/clang/CrossTU/CrossTranslationUnit.h

32

Unrelated change?

106

These comment changes might better be in the next patch.

[r.stahl]

Thanks for the quick response! Will wait a couple days to see if someone else has something to add.

But I think something is wrong here...

When trying to "arc diff" I got the following error:

 Exception 
Command failed with error #1!
COMMAND
svn propget 'svn:mime-type' '/data/work/commitllvm/llvm/tools/clang/tools/clang-extdef-mapping/ClangExtDefMapGen.cpp'@

STDOUT
(empty)

STDERR
svn: warning: W200017: Property 'svn:mime-type' not found on '/data/work/commitllvm/llvm/tools/clang/tools/clang-extdef-mapping/ClangExtDefMapGen.cpp@'
svn: E200000: A problem occurred; see other errors for details

So I did:

svn propset svn:mime-type text/plain tools/clang-extdef-mapping/ClangExtDefMapGen.cpp

This was on all the new/renamed files.

Now this diff shows some weird "svn:mime-type" changes which are probably unwanted. But only on two of the new files (?).

I only found this online, but weird that no one else is having this issue: https://secure.phabricator.com/T10608

[martong]

Looks good to me (but xazax's comments are valid)! Thank you.

[r.stahl]

addressed review comments

[NoQ]

Hmm, this fails for me. The substitution fails for some reason:

FAIL: Clang :: Analysis/func-mapping-test.cpp (211 of 569)
******************** TEST 'Clang :: Analysis/func-mapping-test.cpp' FAILED ********************
Script:
--
: 'RUN: at line 1';   %clang_extdef_map /Users/adergachev/svn/llvm/tools/clang/test/Analysis/func-mapping-test.cpp -- | /Users/adergachev/svn/release/bin/FileCheck /Users/adergachev/svn/llvm/tools/clang/test/Analysis/func-mapping-test.cpp
--
Exit Code: 2

Command Output (stderr):
--
/Users/adergachev/svn/release/tools/clang/test/Analysis/Output/func-mapping-test.cpp.script: line 1: %clang_extdef_map: command not found
FileCheck error: '-' is empty.
FileCheck command line:  /Users/adergachev/svn/release/bin/FileCheck /Users/adergachev/svn/llvm/tools/clang/test/Analysis/func-mapping-test.cpp

--

********************
Testing Time: 5.12s
********************
Failing Tests (1):
    Clang :: Analysis/func-mapping-test.cpp

I'm not really good with LIT and i don't immediately see any problems here. If it only fails for me, please let me know so that i could see if the problem is on my end or otherwise help investigating the problem :)

[NoQ]

Hmm, it disappeared after a clean rebuild. I think the problem was that i did make clang and then ran tests manually, instead of make, so the tool was not rebuilt and was only present under its old name. Because the tool does get rebuilt under make check-clang-analysis, i guess i was just doing an unsupported thing.

Sry for distraction, pls nvm :)

include/clang/Basic/DiagnosticCrossTUKinds.td

 //==--- DiagnosticCrossTUKinds.td - Cross Translation Unit diagnostics ----===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 
 let Component = "CrossTU" in {
 
 def err_ctu_error_opening : Error<
   "error opening '%0': required by the CrossTU functionality">;
 
-def err_fnmap_parsing : Error<
+def err_extdefmap_parsing : Error<
   "error parsing index file: '%0' line: %1 'UniqueID filename' format "
   "expected">;
 
 def err_multiple_def_index : Error<
   "multiple definitions are found for the same key in index ">;
 
 def warn_ctu_incompat_triple : Warning<
   "imported AST from '%0' had been generated for a different target, "
   "current: %1, imported: %2">, InGroup<CrossTU>;
 }

include/clang/CrossTU/CrossTranslationUnit.h

 
 namespace clang {
 class CompilerInstance;
 class ASTContext;
 class ASTImporter;
 class ASTUnit;
 class DeclContext;
 class FunctionDecl;
 class NamedDecl;
 class TranslationUnitDecl;
 
 namespace cross_tu {
 
 enum class index_error_code {
   unspecified = 1,
   missing_index_file,
   invalid_index_format,

 llvm::Expected<llvm::StringMap<std::string>>
 parseCrossTUIndex(StringRef IndexPath, StringRef CrossTUDir);
 
 std::string createCrossTUIndexString(const llvm::StringMap<std::string> &Index);
 
 /// This class is used for tools that requires cross translation
 ///        unit capability.
 ///
-/// This class can load function definitions from external AST files.
+/// This class can load definitions from external AST files.
 /// The loaded definition will be merged back to the original AST using the
 /// AST Importer.
 /// In order to use this class, an index file is required that describes
-/// the locations of the AST files for each function definition.
+/// the locations of the AST files for each definition.
 ///
 /// Note that this class also implements caching.
 class CrossTranslationUnitContext {
 public:
   CrossTranslationUnitContext(CompilerInstance &CI);
   ~CrossTranslationUnitContext();
 
   /// This function loads a function definition from an external AST
   ///        file and merge it into the original AST.
   ///
   /// This method should only be used on functions that have no definitions in
   /// the current translation unit. A function definition with the same
   /// declaration will be looked up in the index file which should be in the
   /// \p CrossTUDir directory, called \p IndexName. In case the declaration is
   /// found in the index the corresponding AST file will be loaded and the
   /// definition of the function will be merged into the original AST using
   /// the AST Importer.

include/clang/StaticAnalyzer/Core/AnalyzerOptions.def

ANALYZER_OPTION(
     "evaluating it to the integers 0 or 1 respectively. The upside is that "
     "this can increase analysis precision until we have a better way to lazily "
     "evaluate such logic. The downside is that it eagerly bifurcates paths.",
     true)
 
 ANALYZER_OPTION(
     bool, IsNaiveCTUEnabled, "experimental-enable-naive-ctu-analysis",
     "Whether naive cross translation unit analysis is enabled. This is an "
-    "experimental feature to inline functions from another translation units.",
+    "experimental feature to inline functions from other translation units.",
     false)
 
 ANALYZER_OPTION(bool, ShouldDisplayMacroExpansions, "expand-macros",
                 "Whether macros related to the bugpath should be "
                 "expanded and included in the plist output.",
                 false)
 
 ANALYZER_OPTION(bool, DisplayCTUProgress, "display-ctu-progress",

 //===----------------------------------------------------------------------===//
 // String analyzer options.
 //===----------------------------------------------------------------------===//
 
 ANALYZER_OPTION(StringRef, CTUDir, "ctu-dir",
                 "The directory containing the CTU related files.", "")
 
 ANALYZER_OPTION(StringRef, CTUIndexName, "ctu-index-name",
-                "the name of the file containing the CTU index of functions.",
-                "externalFnMap.txt")
+                "the name of the file containing the CTU index of definitions.",
+                "externalDefMap.txt")
 
 ANALYZER_OPTION(
     StringRef, ModelPath, "model-path",
     "The analyzer can inline an alternative implementation written in C at the "
     "call site if the called function's body is not available. This is a path "
     "where to look for those alternative implementations (called models).",
     "")
 

lib/CrossTU/CrossTranslationUnit.cpp

 }
 
 std::error_code IndexError::convertToErrorCode() const {
   return std::error_code(static_cast<int>(Code), *Category);
 }
 
 llvm::Expected<llvm::StringMap<std::string>>
 parseCrossTUIndex(StringRef IndexPath, StringRef CrossTUDir) {
-  std::ifstream ExternalFnMapFile(IndexPath);
-  if (!ExternalFnMapFile)
+  std::ifstream ExternalMapFile(IndexPath);
+  if (!ExternalMapFile)
     return llvm::make_error<IndexError>(index_error_code::missing_index_file,
                                         IndexPath.str());
 
   llvm::StringMap<std::string> Result;
   std::string Line;
   unsigned LineNo = 1;
-  while (std::getline(ExternalFnMapFile, Line)) {
+  while (std::getline(ExternalMapFile, Line)) {
     const size_t Pos = Line.find(" ");
     if (Pos > 0 && Pos != std::string::npos) {
       StringRef LineRef{Line};
-      StringRef FunctionLookupName = LineRef.substr(0, Pos);
-      if (Result.count(FunctionLookupName))
+      StringRef LookupName = LineRef.substr(0, Pos);
+      if (Result.count(LookupName))
         return llvm::make_error<IndexError>(
             index_error_code::multiple_definitions, IndexPath.str(), LineNo);
       StringRef FileName = LineRef.substr(Pos + 1);
       SmallString<256> FilePath = CrossTUDir;
       llvm::sys::path::append(FilePath, FileName);
-      Result[FunctionLookupName] = FilePath.str().str();
+      Result[LookupName] = FilePath.str().str();
     } else
       return llvm::make_error<IndexError>(
           index_error_code::invalid_index_format, IndexPath.str(), LineNo);
     LineNo++;
   }
   return Result;
 }
 

 
 void CrossTranslationUnitContext::emitCrossTUDiagnostics(const IndexError &IE) {
   switch (IE.getCode()) {
   case index_error_code::missing_index_file:
     Context.getDiagnostics().Report(diag::err_ctu_error_opening)
         << IE.getFileName();
     break;
   case index_error_code::invalid_index_format:
-    Context.getDiagnostics().Report(diag::err_fnmap_parsing)
+    Context.getDiagnostics().Report(diag::err_extdefmap_parsing)
         << IE.getFileName() << IE.getLineNum();
     break;
   case index_error_code::multiple_definitions:
     Context.getDiagnostics().Report(diag::err_multiple_def_index)
         << IE.getLineNum();
     break;
   case index_error_code::triple_mismatch:
     Context.getDiagnostics().Report(diag::warn_ctu_incompat_triple)

test/Analysis/Inputs/ctu-other.c.externalDefMap.txt

 c:@F@inlineAsm ctu-other.c.ast
 c:@F@g ctu-other.c.ast
 c:@F@f ctu-other.c.ast
 c:@F@enumCheck ctu-other.c.ast
 c:@F@identImplicit ctu-other.c.ast
 c:@F@structInProto ctu-other.c.ast
 

test/Analysis/Inputs/ctu-other.cpp.externalDefMap.txt

 c:@N@chns@F@chf1#I# ctu-other.cpp.ast
 c:@N@myns@N@embed_ns@F@fens#I# ctu-other.cpp.ast
 c:@F@g#I# ctu-other.cpp.ast
 c:@S@mycls@F@fscl#I#S ctu-other.cpp.ast
 c:@S@mycls@F@fcl#I# ctu-other.cpp.ast
 c:@N@myns@S@embed_cls@F@fecl#I# ctu-other.cpp.ast
 c:@S@mycls@S@embed_cls2@F@fecl2#I# ctu-other.cpp.ast
 c:@F@f#I# ctu-other.cpp.ast
 c:@N@myns@F@fns#I# ctu-other.cpp.ast
 c:@F@h#I# ctu-other.cpp.ast
 c:@F@h_chain#I# ctu-chain.cpp.ast
 c:@N@chns@S@chcls@F@chf4#I# ctu-chain.cpp.ast
 c:@N@chns@F@chf2#I# ctu-chain.cpp.ast
 c:@F@fun_using_anon_struct#I# ctu-other.cpp.ast
 c:@F@other_macro_diag#I# ctu-other.cpp.ast
 

test/Analysis/analyzer-config.c

 // CHECK-NEXT: cfg-implicit-dtors = true
 // CHECK-NEXT: cfg-lifetime = false
 // CHECK-NEXT: cfg-loopexit = false
 // CHECK-NEXT: cfg-rich-constructors = true
 // CHECK-NEXT: cfg-scopes = false
 // CHECK-NEXT: cfg-temporary-dtors = true
 // CHECK-NEXT: crosscheck-with-z3 = false
 // CHECK-NEXT: ctu-dir = ""
-// CHECK-NEXT: ctu-index-name = externalFnMap.txt
+// CHECK-NEXT: ctu-index-name = externalDefMap.txt
 // CHECK-NEXT: display-ctu-progress = false
 // CHECK-NEXT: eagerly-assume = true
 // CHECK-NEXT: elide-constructors = true
 // CHECK-NEXT: expand-macros = false
 // CHECK-NEXT: experimental-enable-naive-ctu-analysis = false
 // CHECK-NEXT: exploration_strategy = unexplored_first_queue
 // CHECK-NEXT: faux-bodies = true
 // CHECK-NEXT: graph-trim-interval = 1000

test/Analysis/ctu-different-triples.cpp

 // RUN: rm -rf %t && mkdir %t
 // RUN: mkdir -p %t/ctudir
 // RUN: %clang_cc1 -triple x86_64-pc-linux-gnu \
 // RUN:   -emit-pch -o %t/ctudir/ctu-other.cpp.ast %S/Inputs/ctu-other.cpp
-// RUN: cp %S/Inputs/ctu-other.cpp.externalFnMap.txt %t/ctudir/externalFnMap.txt
+// RUN: cp %S/Inputs/ctu-other.cpp.externalDefMap.txt %t/ctudir/externalDefMap.txt
 // RUN: %clang_analyze_cc1 -triple powerpc64-montavista-linux-gnu \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
 // RUN:   -analyzer-config ctu-dir=%t/ctudir \
 // RUN:   -Werror=ctu \
 // RUN:   -verify %s
 
 // We expect an error in this file, but without a location.
 // expected-error-re@./ctu-different-triples.cpp:*{{imported AST from {{.*}} had been generated for a different target, current: powerpc64-montavista-linux-gnu, imported: x86_64-pc-linux-gnu}}
 
 int f(int);
 
 int main() {
   return f(5);
 }

test/Analysis/ctu-main.c

 // RUN: rm -rf %t && mkdir %t
 // RUN: mkdir -p %t/ctudir2
 // RUN: %clang_cc1 -triple x86_64-pc-linux-gnu \
 // RUN:   -emit-pch -o %t/ctudir2/ctu-other.c.ast %S/Inputs/ctu-other.c
-// RUN: cp %S/Inputs/ctu-other.c.externalFnMap.txt %t/ctudir2/externalFnMap.txt
+// RUN: cp %S/Inputs/ctu-other.c.externalDefMap.txt %t/ctudir2/externalDefMap.txt
 // RUN: %clang_cc1 -triple x86_64-pc-linux-gnu -fsyntax-only -std=c89 -analyze \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
 // RUN:   -analyzer-config ctu-dir=%t/ctudir2 \
 // RUN:   -verify %s
 
 void clang_analyzer_eval(int);
 

test/Analysis/ctu-main.cpp

 // RUN: rm -rf %t && mkdir %t
 // RUN: mkdir -p %t/ctudir
 // RUN: %clang_cc1 -triple x86_64-pc-linux-gnu \
 // RUN:   -emit-pch -o %t/ctudir/ctu-other.cpp.ast %S/Inputs/ctu-other.cpp
 // RUN: %clang_cc1 -triple x86_64-pc-linux-gnu \
 // RUN:   -emit-pch -o %t/ctudir/ctu-chain.cpp.ast %S/Inputs/ctu-chain.cpp
-// RUN: cp %S/Inputs/ctu-other.cpp.externalFnMap.txt %t/ctudir/externalFnMap.txt
+// RUN: cp %S/Inputs/ctu-other.cpp.externalDefMap.txt %t/ctudir/externalDefMap.txt
 // RUN: %clang_analyze_cc1 -triple x86_64-pc-linux-gnu \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
 // RUN:   -analyzer-config ctu-dir=%t/ctudir \
 // RUN:   -verify %s
 // RUN: %clang_analyze_cc1 -triple x86_64-pc-linux-gnu \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \

test/Analysis/ctu-unknown-parts-in-triples.cpp

 // We do not expect any error when one part of the triple is unknown, but other
 // known parts are equal.
 
 // RUN: rm -rf %t && mkdir %t
 // RUN: mkdir -p %t/ctudir
 // RUN: %clang_cc1 -triple x86_64-pc-linux-gnu \
 // RUN:   -emit-pch -o %t/ctudir/ctu-other.cpp.ast %S/Inputs/ctu-other.cpp
-// RUN: cp %S/Inputs/ctu-other.cpp.externalFnMap.txt %t/ctudir/externalFnMap.txt
+// RUN: cp %S/Inputs/ctu-other.cpp.externalDefMap.txt %t/ctudir/externalDefMap.txt
 // RUN: %clang_analyze_cc1 -triple x86_64-unknown-linux-gnu \
 // RUN:   -analyzer-checker=core,debug.ExprInspection \
 // RUN:   -analyzer-config experimental-enable-naive-ctu-analysis=true \
 // RUN:   -analyzer-config ctu-dir=%t/ctudir \
 // RUN:   -Werror=ctu \
 // RUN:   -verify %s
 
 // expected-no-diagnostics
 
 int f(int);
 
 int main() {
   return f(5);
 }

test/Analysis/func-mapping-test.cpp

-// RUN: %clang_func_map %s -- | FileCheck %s
+// RUN: %clang_extdef_map %s -- | FileCheck %s
 
 int f(int) {
   return 0;
 }
 
 // CHECK: c:@F@f#I#

test/CMakeLists.txt

list(APPEND CLANG_TEST_DEPS
   clang-refactor
   clang-diff
   hmaptool
   )
   
 if(CLANG_ENABLE_STATIC_ANALYZER)
   list(APPEND CLANG_TEST_DEPS
     clang-check
-    clang-func-mapping
+    clang-extdef-mapping
     )
 endif()
 
 if (CLANG_ENABLE_ARCMT)
   list(APPEND CLANG_TEST_DEPS
     arcmt-test
     c-arcmt-test
   )

test/lit.cfg.py

 # the build directory holding that tool.  We explicitly specify the directories
 # to search to ensure that we get the tools just built and not some random
 # tools that might happen to be in the user's PATH.
 tool_dirs = [config.clang_tools_dir, config.llvm_tools_dir]
 
 tools = [
     'c-index-test', 'clang-check', 'clang-diff', 'clang-format', 'clang-tblgen',
     'opt',
-    ToolSubst('%clang_func_map', command=FindTool(
-        'clang-func-mapping'), unresolved='ignore'),
+    ToolSubst('%clang_extdef_map', command=FindTool(
+        'clang-extdef-mapping'), unresolved='ignore'),
 ]
 
 if config.clang_examples:
     config.available_features.add('examples')
     tools.append('clang-interpreter')
 
 llvm_config.add_tool_substitutions(tools, tool_dirs)
 

tools/CMakeLists.txt

 
 if(CLANG_ENABLE_ARCMT)
   add_clang_subdirectory(arcmt-test)
   add_clang_subdirectory(c-arcmt-test)
 endif()
 
 if(CLANG_ENABLE_STATIC_ANALYZER)
   add_clang_subdirectory(clang-check)
-  add_clang_subdirectory(clang-func-mapping)
+  add_clang_subdirectory(clang-extdef-mapping)
   add_clang_subdirectory(scan-build)
   add_clang_subdirectory(scan-view)
 endif()
 
 # We support checking out the clang-tools-extra repository into the 'extra'
 # subdirectory. It contains tools developed as part of the Clang/LLVM project
 # on top of the Clang tooling platform. We keep them in a separate repository
 # to keep the primary Clang repository small and focused.
 # It also may be included by LLVM_EXTERNAL_CLANG_TOOLS_EXTRA_SOURCE_DIR.
 add_llvm_external_project(clang-tools-extra extra)
 
 # libclang may require clang-tidy in clang-tools-extra.
 add_clang_subdirectory(libclang)

tools/clang-extdef-mapping/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   ${LLVM_TARGETS_TO_BUILD}
   support
   )
 
-add_clang_executable(clang-func-mapping
-  ClangFnMapGen.cpp
+add_clang_executable(clang-extdef-mapping
+  ClangExtDefMapGen.cpp
   )
 
-target_link_libraries(clang-func-mapping
+target_link_libraries(clang-extdef-mapping
   PRIVATE
   clangAST
   clangBasic
   clangCrossTU
   clangFrontend
   clangSerialization
   clangTooling
   )
 
-install(TARGETS clang-func-mapping
+install(TARGETS clang-extdef-mapping
   RUNTIME DESTINATION bin)

tools/clang-extdef-mapping/ClangExtDefMapGen.cpp

 #include <sstream>
 #include <string>
 
 using namespace llvm;
 using namespace clang;
 using namespace clang::cross_tu;
 using namespace clang::tooling;
 
-static cl::OptionCategory ClangFnMapGenCategory("clang-fnmapgen options");
+static cl::OptionCategory ClangExtDefMapGenCategory("clang-extdefmapgen options");
 
-class MapFunctionNamesConsumer : public ASTConsumer {
+class MapExtDefNamesConsumer : public ASTConsumer {
 public:
-  MapFunctionNamesConsumer(ASTContext &Context)
+  MapExtDefNamesConsumer(ASTContext &Context)
       : SM(Context.getSourceManager()) {}
 
-  ~MapFunctionNamesConsumer() {
+  ~MapExtDefNamesConsumer() {
     // Flush results to standard output.
     llvm::outs() << createCrossTUIndexString(Index);
   }
 
   void HandleTranslationUnit(ASTContext &Ctx) override {
     handleDecl(Ctx.getTranslationUnitDecl());
   }
 
 private:
   void handleDecl(const Decl *D);
 
   SourceManager &SM;
   llvm::StringMap<std::string> Index;
   std::string CurrentFileName;
 };
 
-void MapFunctionNamesConsumer::handleDecl(const Decl *D) {
+void MapExtDefNamesConsumer::handleDecl(const Decl *D) {
   if (!D)
     return;
 
   if (const auto *FD = dyn_cast<FunctionDecl>(D)) {
     if (FD->isThisDeclarationADefinition()) {
       if (const Stmt *Body = FD->getBody()) {
         if (CurrentFileName.empty()) {
           CurrentFileName =
if (const auto *FD = dyn_cast<FunctionDecl>(D)) {
     }
   }
 
   if (const auto *DC = dyn_cast<DeclContext>(D))
     for (const Decl *D : DC->decls())
       handleDecl(D);
 }
 
-class MapFunctionNamesAction : public ASTFrontendAction {
+class MapExtDefNamesAction : public ASTFrontendAction {
 protected:
   std::unique_ptr<ASTConsumer> CreateASTConsumer(CompilerInstance &CI,
                                                  llvm::StringRef) {
-    return llvm::make_unique<MapFunctionNamesConsumer>(CI.getASTContext());
+    return llvm::make_unique<MapExtDefNamesConsumer>(CI.getASTContext());
   }
 };
 
 static cl::extrahelp CommonHelp(CommonOptionsParser::HelpMessage);
 
 int main(int argc, const char **argv) {
   // Print a stack trace if we signal out.
   sys::PrintStackTraceOnErrorSignal(argv[0], false);
   PrettyStackTraceProgram X(argc, argv);
 
   const char *Overview = "\nThis tool collects the USR name and location "
-                         "of all functions definitions in the source files "
+                         "of external definitions in the source files "
                          "(excluding headers).\n";
-  CommonOptionsParser OptionsParser(argc, argv, ClangFnMapGenCategory,
+  CommonOptionsParser OptionsParser(argc, argv, ClangExtDefMapGenCategory,
                                     cl::ZeroOrMore, Overview);
 
   ClangTool Tool(OptionsParser.getCompilations(),
                  OptionsParser.getSourcePathList());
 
-  return Tool.run(newFrontendActionFactory<MapFunctionNamesAction>().get());
+  return Tool.run(newFrontendActionFactory<MapExtDefNamesAction>().get());
 }

tools/scan-build-py/README.md

To run the CTU analysis, a compilation database file has to be created:
 
     $ intercept-build <your build command>
 
 To run the Clang Static Analyzer against a compilation database
 with CTU analysis enabled, execute:
     
     $ analyze-build --ctu
 
-For CTU analysis an additional (function-definition) collection-phase is required. 
+For CTU analysis an additional (external definition) collection-phase is required. 
 For debugging purposes, it is possible to separately execute the collection 
 and the analysis phase. By doing this, the intermediate files used for 
 the analysis are kept on the disk in `./ctu-dir`.
     
     # Collect and store the data required by the CTU analysis
     $ analyze-build --ctu-collect-only
     
     # Analyze using the previously collected data

tools/scan-build-py/libscanbuild/__init__.py

 import subprocess
 import sys
 
 ENVIRONMENT_KEY = 'INTERCEPT_BUILD'
 
 Execution = collections.namedtuple('Execution', ['pid', 'cwd', 'cmd'])
 
 CtuConfig = collections.namedtuple('CtuConfig', ['collect', 'analyze', 'dir',
-                                                 'func_map_cmd'])
+                                                 'extdef_map_cmd'])
 
 
 def duplicate_check(method):
     """ Predicate to detect duplicated entries.
 
     Unique hash method can be use to detect duplicates. Entries are
     represented as dictionaries, which has no default hash method.
     This implementation uses a set datatype to store the unique hash values.

tools/scan-build-py/libscanbuild/analyze.py

 from libscanbuild.clang import get_version, get_arguments, get_triple_arch
 from libscanbuild.shell import decode
 
 __all__ = ['scan_build', 'analyze_build', 'analyze_compiler_wrapper']
 
 COMPILER_WRAPPER_CC = 'analyze-cc'
 COMPILER_WRAPPER_CXX = 'analyze-c++'
 
-CTU_FUNCTION_MAP_FILENAME = 'externalFnMap.txt'
-CTU_TEMP_FNMAP_FOLDER = 'tmpExternalFnMaps'
+CTU_EXTDEF_MAP_FILENAME = 'externalDefMap.txt'
+CTU_TEMP_DEFMAP_FOLDER = 'tmpExternalDefMaps'
 
 
 @command_entry_point
 def scan_build():
     """ Entry point for scan-build command. """
 
     args = parse_args_for_scan_build()
     # will re-assign the report directory as new output

 
 def get_ctu_config_from_args(args):
     """ CTU configuration is created from the chosen phases and dir. """
 
     return (
         CtuConfig(collect=args.ctu_phases.collect,
                   analyze=args.ctu_phases.analyze,
                   dir=args.ctu_dir,
-                  func_map_cmd=args.func_map_cmd)
+                  extdef_map_cmd=args.extdef_map_cmd)
         if hasattr(args, 'ctu_phases') and hasattr(args.ctu_phases, 'dir')
-        else CtuConfig(collect=False, analyze=False, dir='', func_map_cmd=''))
+        else CtuConfig(collect=False, analyze=False, dir='', extdef_map_cmd=''))
 
 
 def get_ctu_config_from_json(ctu_conf_json):
     """ CTU configuration is created from the chosen phases and dir. """
 
     ctu_config = json.loads(ctu_conf_json)
     # Recover namedtuple from json when coming from analyze-cc or analyze-c++
     return CtuConfig(collect=ctu_config[0],
                      analyze=ctu_config[1],
                      dir=ctu_config[2],
-                     func_map_cmd=ctu_config[3])
+                     extdef_map_cmd=ctu_config[3])
 
 
-def create_global_ctu_function_map(func_map_lines):
-    """ Takes iterator of individual function maps and creates a global map
-    keeping only unique names. We leave conflicting names out of CTU.
+def create_global_ctu_extdef_map(extdef_map_lines):
+    """ Takes iterator of individual external definition maps and creates a
+    global map keeping only unique names. We leave conflicting names out of
+    CTU.
 
-    :param func_map_lines: Contains the id of a function (mangled name) and
+    :param extdef_map_lines: Contains the id of a definition (mangled name) and
     the originating source (the corresponding AST file) name.
-    :type func_map_lines: Iterator of str.
+    :type extdef_map_lines: Iterator of str.
     :returns: Mangled name - AST file pairs.
     :rtype: List of (str, str) tuples.
     """
 
     mangled_to_asts = defaultdict(set)
 
-    for line in func_map_lines:
+    for line in extdef_map_lines:
         mangled_name, ast_file = line.strip().split(' ', 1)
         mangled_to_asts[mangled_name].add(ast_file)
 
     mangled_ast_pairs = []
 
     for mangled_name, ast_files in mangled_to_asts.items():
         if len(ast_files) == 1:
             mangled_ast_pairs.append((mangled_name, next(iter(ast_files))))
 
     return mangled_ast_pairs
 
 
-def merge_ctu_func_maps(ctudir):
-    """ Merge individual function maps into a global one.
+def merge_ctu_extdef_maps(ctudir):
+    """ Merge individual external definition maps into a global one.
 
     As the collect phase runs parallel on multiple threads, all compilation
-    units are separately mapped into a temporary file in CTU_TEMP_FNMAP_FOLDER.
-    These function maps contain the mangled names of functions and the source
-    (AST generated from the source) which had them.
+    units are separately mapped into a temporary file in CTU_TEMP_DEFMAP_FOLDER.
+    These definition maps contain the mangled names and the source
+    (AST generated from the source) which had their definition.
     These files should be merged at the end into a global map file:
-    CTU_FUNCTION_MAP_FILENAME."""
+    CTU_EXTDEF_MAP_FILENAME."""
 
-    def generate_func_map_lines(fnmap_dir):
+    def generate_extdef_map_lines(extdefmap_dir):
         """ Iterate over all lines of input files in a determined order. """
 
-        files = glob.glob(os.path.join(fnmap_dir, '*'))
+        files = glob.glob(os.path.join(extdefmap_dir, '*'))
         files.sort()
         for filename in files:
             with open(filename, 'r') as in_file:
                 for line in in_file:
                     yield line
 
     def write_global_map(arch, mangled_ast_pairs):
-        """ Write (mangled function name, ast file) pairs into final file. """
+        """ Write (mangled name, ast file) pairs into final file. """
 
-        extern_fns_map_file = os.path.join(ctudir, arch,
-                                           CTU_FUNCTION_MAP_FILENAME)
-        with open(extern_fns_map_file, 'w') as out_file:
+        extern_defs_map_file = os.path.join(ctudir, arch,
+                                           CTU_EXTDEF_MAP_FILENAME)
+        with open(extern_defs_map_file, 'w') as out_file:
             for mangled_name, ast_file in mangled_ast_pairs:
                 out_file.write('%s %s\n' % (mangled_name, ast_file))
 
     triple_arches = glob.glob(os.path.join(ctudir, '*'))
     for triple_path in triple_arches:
         if os.path.isdir(triple_path):
             triple_arch = os.path.basename(triple_path)
-            fnmap_dir = os.path.join(ctudir, triple_arch,
-                                     CTU_TEMP_FNMAP_FOLDER)
+            extdefmap_dir = os.path.join(ctudir, triple_arch,
+                                     CTU_TEMP_DEFMAP_FOLDER)
 
-            func_map_lines = generate_func_map_lines(fnmap_dir)
-            mangled_ast_pairs = create_global_ctu_function_map(func_map_lines)
+            extdef_map_lines = generate_extdef_map_lines(extdefmap_dir)
+            mangled_ast_pairs = create_global_ctu_extdef_map(extdef_map_lines)
             write_global_map(triple_arch, mangled_ast_pairs)
 
             # Remove all temporary files
-            shutil.rmtree(fnmap_dir, ignore_errors=True)
+            shutil.rmtree(extdefmap_dir, ignore_errors=True)
 
 
 def run_analyzer_parallel(args):
     """ Runs the analyzer against the given compilation database. """
 
     def exclude(filename):
         """ Return true when any excluded directory prefix the filename. """
         return any(re.match(r'^' + directory, filename)
if ctu_config.collect:
         shutil.rmtree(ctu_config.dir, ignore_errors=True)
 
     # If the user asked for a collect (1st) and analyze (2nd) phase, we do an
     # all-in-one run where we deliberately remove collection data before and
     # also after the run. If the user asks only for a single phase data is
     # left so multiple analyze runs can use the same data gathered by a single
     # collection run.
     if ctu_config.collect and ctu_config.analyze:
-        # CTU strings are coming from args.ctu_dir and func_map_cmd,
+        # CTU strings are coming from args.ctu_dir and extdef_map_cmd,
         # so we can leave it empty
         args.ctu_phases = CtuConfig(collect=True, analyze=False,
-                                    dir='', func_map_cmd='')
+                                    dir='', extdef_map_cmd='')
         run_analyzer_parallel(args)
-        merge_ctu_func_maps(ctu_config.dir)
+        merge_ctu_extdef_maps(ctu_config.dir)
         args.ctu_phases = CtuConfig(collect=False, analyze=True,
-                                    dir='', func_map_cmd='')
+                                    dir='', extdef_map_cmd='')
         run_analyzer_parallel(args)
         shutil.rmtree(ctu_config.dir, ignore_errors=True)
     else:
         # Single runs (collect or analyze) are launched from here.
         run_analyzer_parallel(args)
         if ctu_config.collect:
-            merge_ctu_func_maps(ctu_config.dir)
+            merge_ctu_extdef_maps(ctu_config.dir)
 
 
 def setup_environment(args):
     """ Set up environment for build command to interpose compiler wrapper. """
 
     environment = dict(os.environ)
     environment.update(wrapper_environment(args))
     environment.update({
def run_analyzer(opts, continuation=report_failure):
     except subprocess.CalledProcessError as ex:
         result = {'error_output': ex.output, 'exit_code': ex.returncode}
         if opts.get('output_failures', False):
             opts.update(result)
             continuation(opts)
         return result
 
 
-def func_map_list_src_to_ast(func_src_list):
-    """ Turns textual function map list with source files into a
-    function map list with ast files. """
-
-    func_ast_list = []
-    for fn_src_txt in func_src_list:
-        mangled_name, path = fn_src_txt.split(" ", 1)
+def extdef_map_list_src_to_ast(extdef_src_list):
+    """ Turns textual external definition map list with source files into an
+    external definition map list with ast files. """
+
+    extdef_ast_list = []
+    for extdef_src_txt in extdef_src_list:
+        mangled_name, path = extdef_src_txt.split(" ", 1)
         # Normalize path on windows as well
         path = os.path.splitdrive(path)[1]
         # Make relative path out of absolute
         path = path[1:] if path[0] == os.sep else path
         ast_path = os.path.join("ast", path + ".ast")
-        func_ast_list.append(mangled_name + " " + ast_path)
-    return func_ast_list
+        extdef_ast_list.append(mangled_name + " " + ast_path)
+    return extdef_ast_list
 
 
 @require(['clang', 'directory', 'flags', 'direct_args', 'file', 'ctu'])
 def ctu_collect_phase(opts):
     """ Preprocess source by generating all data needed by CTU analysis. """
 
     def generate_ast(triple_arch):
         """ Generates ASTs for the current compilation command. """
def generate_ast(triple_arch):
         ast_command.extend(args)
         ast_command.append('-w')
         ast_command.append(opts['file'])
         ast_command.append('-o')
         ast_command.append(ast_path)
         logging.debug("Generating AST using '%s'", ast_command)
         run_command(ast_command, cwd=opts['directory'])
 
-    def map_functions(triple_arch):
-        """ Generate function map file for the current source. """
+    def map_extdefs(triple_arch):
+        """ Generate external definition map file for the current source. """
 
         args = opts['direct_args'] + opts['flags']
-        funcmap_command = [opts['ctu'].func_map_cmd]
-        funcmap_command.append(opts['file'])
-        funcmap_command.append('--')
-        funcmap_command.extend(args)
-        logging.debug("Generating function map using '%s'", funcmap_command)
-        func_src_list = run_command(funcmap_command, cwd=opts['directory'])
-        func_ast_list = func_map_list_src_to_ast(func_src_list)
-        extern_fns_map_folder = os.path.join(opts['ctu'].dir, triple_arch,
-                                             CTU_TEMP_FNMAP_FOLDER)
-        if not os.path.isdir(extern_fns_map_folder):
+        extdefmap_command = [opts['ctu'].extdef_map_cmd]
+        extdefmap_command.append(opts['file'])
+        extdefmap_command.append('--')
+        extdefmap_command.extend(args)
+        logging.debug("Generating external definition map using '%s'",
+                      extdefmap_command)
+        extdef_src_list = run_command(extdefmap_command, cwd=opts['directory'])
+        extdef_ast_list = extdef_map_list_src_to_ast(extdef_src_list)
+        extern_defs_map_folder = os.path.join(opts['ctu'].dir, triple_arch,
+                                             CTU_TEMP_DEFMAP_FOLDER)
+        if not os.path.isdir(extern_defs_map_folder):
             try:
-                os.makedirs(extern_fns_map_folder)
+                os.makedirs(extern_defs_map_folder)
             except OSError:
                 # In case an other process already created it.
                 pass
-        if func_ast_list:
+        if extdef_ast_list:
             with tempfile.NamedTemporaryFile(mode='w',
-                                             dir=extern_fns_map_folder,
+                                             dir=extern_defs_map_folder,
                                              delete=False) as out_file:
-                out_file.write("\n".join(func_ast_list) + "\n")
+                out_file.write("\n".join(extdef_ast_list) + "\n")
 
     cwd = opts['directory']
     cmd = [opts['clang'], '--analyze'] + opts['direct_args'] + opts['flags'] \
         + [opts['file']]
     triple_arch = get_triple_arch(cmd, cwd)
     generate_ast(triple_arch)
-    map_functions(triple_arch)
+    map_extdefs(triple_arch)
 
 
 @require(['ctu'])
 def dispatch_ctu(opts, continuation=run_analyzer):
     """ Execute only one phase of 2 phases of CTU if needed. """
 
     ctu_config = opts['ctu']
 

tools/scan-build-py/libscanbuild/arguments.py

def validate_args_for_analyze(parser, args, from_build_command):
 
     # If the user wants CTU mode
     if not from_build_command and hasattr(args, 'ctu_phases') \
             and hasattr(args.ctu_phases, 'dir'):
         # If CTU analyze_only, the input directory should exist
         if args.ctu_phases.analyze and not args.ctu_phases.collect \
                 and not os.path.exists(args.ctu_dir):
             parser.error(message='missing CTU directory')
-        # Check CTU capability via checking clang-func-mapping
-        if not is_ctu_capable(args.func_map_cmd):
+        # Check CTU capability via checking clang-extdef-mapping
+        if not is_ctu_capable(args.extdef_map_cmd):
             parser.error(message="""This version of clang does not support CTU
-            functionality or clang-func-mapping command not found.""")
+            functionality or clang-extdef-mapping command not found.""")
 
 
 def create_intercept_parser():
     """ Creates a parser for command-line arguments to 'intercept'. """
 
     parser = create_default_parser()
     parser_add_cdb(parser)
 
if from_build_command:
             dest='build', nargs=argparse.REMAINDER, help="""Command to run.""")
     else:
         ctu = parser.add_argument_group('cross translation unit analysis')
         ctu_mutex_group = ctu.add_mutually_exclusive_group()
         ctu_mutex_group.add_argument(
             '--ctu',
             action='store_const',
             const=CtuConfig(collect=True, analyze=True,
-                            dir='', func_map_cmd=''),
+                            dir='', extdef_map_cmd=''),
             dest='ctu_phases',
             help="""Perform cross translation unit (ctu) analysis (both collect
             and analyze phases) using default <ctu-dir> for temporary output.
             At the end of the analysis, the temporary directory is removed.""")
         ctu.add_argument(
             '--ctu-dir',
             metavar='<ctu-dir>',
             dest='ctu_dir',
             default='ctu-dir',
             help="""Defines the temporary directory used between ctu
             phases.""")
         ctu_mutex_group.add_argument(
             '--ctu-collect-only',
             action='store_const',
             const=CtuConfig(collect=True, analyze=False,
-                            dir='', func_map_cmd=''),
+                            dir='', extdef_map_cmd=''),
             dest='ctu_phases',
             help="""Perform only the collect phase of ctu.
             Keep <ctu-dir> for further use.""")
         ctu_mutex_group.add_argument(
             '--ctu-analyze-only',
             action='store_const',
             const=CtuConfig(collect=False, analyze=True,
-                            dir='', func_map_cmd=''),
+                            dir='', extdef_map_cmd=''),
             dest='ctu_phases',
             help="""Perform only the analyze phase of ctu. <ctu-dir> should be
             present and will not be removed after analysis.""")
         ctu.add_argument(
-            '--use-func-map-cmd',
+            '--use-extdef-map-cmd',
             metavar='<path>',
-            dest='func_map_cmd',
-            default='clang-func-mapping',
-            help="""'%(prog)s' uses the 'clang-func-mapping' executable
-            relative to itself for generating function maps for static
-            analysis. One can override this behavior with this option by using
-            the 'clang-func-mapping' packaged with Xcode (on OS X) or from the
-            PATH.""")
+            dest='extdef_map_cmd',
+            default='clang-extdef-mapping',
+            help="""'%(prog)s' uses the 'clang-extdef-mapping' executable
+            relative to itself for generating external definition maps for
+            static analysis. One can override this behavior with this option
+            by using the 'clang-extdef-mapping' packaged with Xcode (on OS X)
+            or from the PATH.""")
     return parser
 
 
 def create_default_parser():
     """ Creates command line parser for all build wrapper commands. """
 
     parser = argparse.ArgumentParser(
         formatter_class=argparse.ArgumentDefaultsHelpFormatter)

tools/scan-build-py/libscanbuild/clang.py

checkers = {
         for name, description in parse_checkers(lines)
     }
     if not checkers:
         raise Exception('Could not query Clang for available checkers.')
 
     return checkers
 
 
-def is_ctu_capable(func_map_cmd):
-    """ Detects if the current (or given) clang and function mapping
+def is_ctu_capable(extdef_map_cmd):
+    """ Detects if the current (or given) clang and external definition mapping
     executables are CTU compatible. """
 
     try:
-        run_command([func_map_cmd, '-version'])
+        run_command([extdef_map_cmd, '-version'])
     except (OSError, subprocess.CalledProcessError):
         return False
     return True
 
 
 def get_triple_arch(command, cwd):
     """Returns the architecture part of the target triple for the given
     compilation command. """
 
     cmd = get_arguments(command, cwd)
     try:
         separator = cmd.index("-triple")
         return cmd[separator + 1]
     except (IndexError, ValueError):
         return ""

tools/scan-build-py/tests/unit/test_analyze.py

class PrefixWithTest(unittest.TestCase):
     def test_interleaves_prefix(self):
         res = sut.prefix_with(0, [1, 2, 3])
         self.assertListEqual([0, 1, 0, 2, 0, 3], res)
 
 
 class MergeCtuMapTest(unittest.TestCase):
 
     def test_no_map_gives_empty(self):
-        pairs = sut.create_global_ctu_function_map([])
+        pairs = sut.create_global_ctu_extdef_map([])
         self.assertFalse(pairs)
 
     def test_multiple_maps_merged(self):
         concat_map = ['c:@F@fun1#I# ast/fun1.c.ast',
                       'c:@F@fun2#I# ast/fun2.c.ast',
                       'c:@F@fun3#I# ast/fun3.c.ast']
-        pairs = sut.create_global_ctu_function_map(concat_map)
+        pairs = sut.create_global_ctu_extdef_map(concat_map)
         self.assertTrue(('c:@F@fun1#I#', 'ast/fun1.c.ast') in pairs)
         self.assertTrue(('c:@F@fun2#I#', 'ast/fun2.c.ast') in pairs)
         self.assertTrue(('c:@F@fun3#I#', 'ast/fun3.c.ast') in pairs)
         self.assertEqual(3, len(pairs))
 
     def test_not_unique_func_left_out(self):
         concat_map = ['c:@F@fun1#I# ast/fun1.c.ast',
                       'c:@F@fun2#I# ast/fun2.c.ast',
                       'c:@F@fun1#I# ast/fun7.c.ast']
-        pairs = sut.create_global_ctu_function_map(concat_map)
+        pairs = sut.create_global_ctu_extdef_map(concat_map)
         self.assertFalse(('c:@F@fun1#I#', 'ast/fun1.c.ast') in pairs)
         self.assertFalse(('c:@F@fun1#I#', 'ast/fun7.c.ast') in pairs)
         self.assertTrue(('c:@F@fun2#I#', 'ast/fun2.c.ast') in pairs)
         self.assertEqual(1, len(pairs))
 
     def test_duplicates_are_kept(self):
         concat_map = ['c:@F@fun1#I# ast/fun1.c.ast',
                       'c:@F@fun2#I# ast/fun2.c.ast',
                       'c:@F@fun1#I# ast/fun1.c.ast']
-        pairs = sut.create_global_ctu_function_map(concat_map)
+        pairs = sut.create_global_ctu_extdef_map(concat_map)
         self.assertTrue(('c:@F@fun1#I#', 'ast/fun1.c.ast') in pairs)
         self.assertTrue(('c:@F@fun2#I#', 'ast/fun2.c.ast') in pairs)
         self.assertEqual(2, len(pairs))
 
     def test_space_handled_in_source(self):
         concat_map = ['c:@F@fun1#I# ast/f un.c.ast']
-        pairs = sut.create_global_ctu_function_map(concat_map)
+        pairs = sut.create_global_ctu_extdef_map(concat_map)
         self.assertTrue(('c:@F@fun1#I#', 'ast/f un.c.ast') in pairs)
         self.assertEqual(1, len(pairs))
 
 
-class FuncMapSrcToAstTest(unittest.TestCase):
+class ExtdefMapSrcToAstTest(unittest.TestCase):
 
     def test_empty_gives_empty(self):
-        fun_ast_lst = sut.func_map_list_src_to_ast([])
+        fun_ast_lst = sut.extdef_map_list_src_to_ast([])
         self.assertFalse(fun_ast_lst)
 
     def test_sources_to_asts(self):
         fun_src_lst = ['c:@F@f1#I# ' + os.path.join(os.sep + 'path', 'f1.c'),
                        'c:@F@f2#I# ' + os.path.join(os.sep + 'path', 'f2.c')]
-        fun_ast_lst = sut.func_map_list_src_to_ast(fun_src_lst)
+        fun_ast_lst = sut.extdef_map_list_src_to_ast(fun_src_lst)
         self.assertTrue('c:@F@f1#I# ' +
                         os.path.join('ast', 'path', 'f1.c.ast')
                         in fun_ast_lst)
         self.assertTrue('c:@F@f2#I# ' +
                         os.path.join('ast', 'path', 'f2.c.ast')
                         in fun_ast_lst)
         self.assertEqual(2, len(fun_ast_lst))
 
     def test_spaces_handled(self):
         fun_src_lst = ['c:@F@f1#I# ' + os.path.join(os.sep + 'path', 'f 1.c')]
-        fun_ast_lst = sut.func_map_list_src_to_ast(fun_src_lst)
+        fun_ast_lst = sut.extdef_map_list_src_to_ast(fun_src_lst)
         self.assertTrue('c:@F@f1#I# ' +
                         os.path.join('ast', 'path', 'f 1.c.ast')
                         in fun_ast_lst)
         self.assertEqual(1, len(fun_ast_lst))

tools/scan-build-py/tests/unit/test_clang.py

def test_parse_checkers(self):
         self.assertTrue('checker.one' in result)
         self.assertEqual('Checker One description', result.get('checker.one'))
         self.assertTrue('checker.two' in result)
         self.assertEqual('Checker Two description', result.get('checker.two'))
 
 
 class ClangIsCtuCapableTest(unittest.TestCase):
     def test_ctu_not_found(self):
-        is_ctu = sut.is_ctu_capable('not-found-clang-func-mapping')
+        is_ctu = sut.is_ctu_capable('not-found-clang-extdef-mapping')
         self.assertFalse(is_ctu)
 
 
 class ClangGetTripleArchTest(unittest.TestCase):
     def test_arch_is_not_empty(self):
         arch = sut.get_triple_arch(['clang', '-E', '-'], '.')
         self.assertTrue(len(arch) > 0)