Index: docs/ModularizeUsage.rst =================================================================== --- docs/ModularizeUsage.rst +++ docs/ModularizeUsage.rst @@ -2,16 +2,18 @@ Modularize Usage ================ -``modularize [] [...]`` +``modularize [] +[...]`` ```` is a place-holder for options specific to modularize, which are described below in `Modularize Command Line Options`. -```` specifies the path of a file name for a -file containing the newline-separated list of headers to check -with respect to each other. Lines beginning with '#' and empty -lines are ignored. Header file names followed by a colon and +```` specifies either +the path to a module map to check or for a file containing +a newline-separated list of headers to check with respect to +each other. Lines beginning with '#' and empty lines are ignored. +Header file names followed by a colon and other space-separated file names will include those extra files as dependencies. The file names can be relative or full paths, but must be on the same line. For example:: @@ -40,7 +42,20 @@ By default, headers are assumed to be relative to the header list file directory. Use ``-prefix`` to specify a different directory. -.. option:: -module-map-path= +.. option:: -I + + For the case of the coverage check look for headers in this path. + There can be multiple -I options. + +.. option:: -coverage-check-only + + Only do the coverage check for a module map. + +.. option:: -dump-module-map + + Tells modularize to display the contents of the module map. + +.. option:: -module-map-output-path= Generate a module map and output it to the given file. See the description in :ref:`module-map-generation`. Index: docs/modularize.rst =================================================================== --- docs/modularize.rst +++ docs/modularize.rst @@ -10,12 +10,15 @@ ModularizeUsage :program:`modularize` is a standalone tool that checks whether a set of headers -provides the consistent definitions required to use modules. For example, it -detects whether the same entity (say, a NULL macro or size_t typedef) is -defined in multiple headers or whether a header produces different definitions -under different circumstances. These conditions cause modules built from the -headers to behave poorly, and should be fixed before introducing a module -map. +provides the consistent definitions required to use modules. It also can +check an existing module map for full coverage of the headers in its +directory tree. + +For example, it detects whether the same entity (say, a NULL macro or +size_t typedef) is defined in multiple headers or whether a header produces +different definitions under different circumstances. These conditions cause +modules built from the headers to behave poorly, and should be fixed before +introducing a module map. :program:`modularize` also has an assistant mode option for generating a module map file based on the provided header list. The generated file @@ -56,6 +59,8 @@ * Macro instances, 'defined(macro)', or #if, #elif, #ifdef, #ifndef conditions that evaluate differently in a header * #include directives inside 'extern "C/C++" {}' or 'namespace (name) {}' blocks +* Module map header coverage completeness (in the case of a module map input + only) Modularize will do normal C/C++ parsing, reporting normal errors and warnings, but will also report special error messages like the following:: @@ -107,6 +112,44 @@ ^ The "extern "C" {}" block is here. +.. _module-map-coverage: + +Module Map Coverage Check +========================= + +The coverage check uses the Clang library to read and parse the +module map file. Starting at the module map file directory, or just the +include paths, if specified, it will collect the names of all the files it +considers headers (no extension, .h, or .inc--if you need more, modify the +isHeader function). It then compares the headers against those referenced +in the module map, either explicitly named, or implicitly named via an +umbrella directory or umbrella file, as parsed by the ModuleMap object. +If headers are found which are not referenced or covered by an umbrella +directory or file, warning messages will be produced, and this program +will return an error code of 1. Other errors result in an error code of 2. +If no problems are found, an error code of 0 is returned. + +Note that in the case of umbrella headers, this tool invokes the compiler +to preprocess the file, and uses a callback to collect the header files +included by the umbrella header or any of its nested includes. If any +front end options are needed for these compiler invocations, these +can be included on the command line after the module map file argument. + +Warning message have the form: + + warning: module.modulemap does not account for file: Level3A.h + +Note that for the case of the module map referencing a file that does +not exist, the module map parser in Clang will (at the time of this +writing) display an error message. + +To limit the checks :program:`modularize` does to just the module +map coverage check, use the ``-coverage-check-only option``. + +For example:: + + modularize -coverage-check-only module.modulemap + .. _module-map-generation: Module Map Generation @@ -133,7 +176,7 @@ The following module map will be generated:: // Output/NoProblemsAssistant.txt - // Generated by: modularize -module-map-path=Output/NoProblemsAssistant.txt \ + // Generated by: modularize -module-map-output-path=Output/NoProblemsAssistant.txt \ -root-module=Root NoProblemsAssistant.modularize module SomeTypes { Index: docs/module-map-checker.rst =================================================================== --- docs/module-map-checker.rst +++ docs/module-map-checker.rst @@ -70,7 +70,7 @@ Module-Map-Checker Command Line Options ======================================= -.. option:: -I(include path) +.. option:: -I (include path) Look at headers only in this directory tree. Must be a path relative to the module.map file. Index: modularize/CMakeLists.txt =================================================================== --- modularize/CMakeLists.txt +++ modularize/CMakeLists.txt @@ -5,6 +5,7 @@ add_clang_executable(modularize Modularize.cpp + ModuleMapChecker.cpp ModuleAssistant.cpp PreprocessorTracker.cpp ) Index: modularize/Modularize.cpp =================================================================== --- modularize/Modularize.cpp +++ modularize/Modularize.cpp @@ -7,37 +7,73 @@ // //===----------------------------------------------------------------------===// // +// Introduction +// // This file implements a tool that checks whether a set of headers provides -// the consistent definitions required to use modules. For example, it detects -// whether the same entity (say, a NULL macro or size_t typedef) is defined in -// multiple headers or whether a header produces different definitions under +// the consistent definitions required to use modules. It can also check an +// existing module map for full coverage of the headers in a directory tree. +// +// For example, in examining headers, it detects whether the same entity +// (say, a NULL macro or size_t typedef) is defined in multiple headers +// or whether a header produces different definitions under // different circumstances. These conditions cause modules built from the // headers to behave poorly, and should be fixed before introducing a module // map. // -// Modularize takes as argument a file name for a file containing the -// newline-separated list of headers to check with respect to each other. +// Modularize takes as input either an existing module map (by default, +// "module.modulemap") or a text file list of headers. +// +// In the case of a module map, the module map must be well-formed in +// terms of syntax. Modularize will extract the header file names +// from the map, internally treating file in umbrella directories +// as dependents if any headers are listed in the module too, or as +// top level headers if they are not. +// +// In the case of a file list, the list is a newline-separated list of headers +// to check with respect to each other. // Lines beginning with '#' and empty lines are ignored. // Header file names followed by a colon and other space-separated // file names will include those extra files as dependencies. // The file names can be relative or full paths, but must be on the // same line. // -// Modularize also accepts regular front-end arguments. +// Modularize also accepts regular clang front-end arguments. // -// Usage: modularize [-prefix (optional header path prefix)] -// (include-files_list) [(front-end-options) ...] +// Usage: modularize [(modularize options)] +// [(include-files_list)|(module map)] [(front-end-options) ...] // -// Note that unless a "-prefix (header path)" option is specified, -// non-absolute file paths in the header list file will be relative -// to the header list file directory. Use -prefix to specify a different -// directory. +// Options: +// -prefix (optional header path prefix) +// Note that unless a "-prefix (header path)" option is specified, +// non-absolute file paths in the header list file will be relative +// to the header list file directory. Use -prefix to specify a +// different directory. +// -I (include path) +// For the case of the coverage check look for header in this +// path. +// -module-map-output-path (module map) +// Skip the checks, and instead act as a module.map generation +// assistant, generating a module map file based on the header list. +// An optional "-root-module=(rootName)" argument can specify a root +// module to be created in the generated module.map file. Note that +// you will likely need to edit this file to suit the needs of your +// headers. +// -root-module (root module name) +// Specifies a root module to be created in the generated module.map +// file. +// -coverage-check-only +// Only do the coverage check. +// -dump-module-map +// Tells modularize to display the contents of the module map. // // Note that by default, the underlying Clang front end assumes .h files // contain C source. If your .h files in the file list contain C++ source, // you should append the following to your command lines: -x c++ // -// Modularize will do normal parsing, reporting normal errors and warnings, +// Modularization Issue Checks +// +// In the process of checking headers for modularization issues, modularize +// will do normal parsing, reporting normal errors and warnings, // but will also report special error messages like the following: // // error: '(symbol)' defined at multiple locations: @@ -90,16 +126,51 @@ // // See PreprocessorTracker.cpp for additional details. // -// Modularize also has an option ("-module-map-path=module.map") that will -// skip the checks, and instead act as a module.map generation assistant, -// generating a module map file based on the header list. An optional -// "-root-module=(rootName)" argument can specify a root module to be -// created in the generated module.map file. Note that you will likely -// need to edit this file to suit the needs of your headers. +// Module Map Coverage Check +// +// The coverage check uses the Clang ModuleMap class to read and parse the +// module map file. Starting at the module map file directory, or just the +// include paths, if specified, it will collect the names of all the files it +// considers headers (no extension, .h, or .inc--if you need more, modify the +// isHeader function). It then compares the headers against those referenced +// in the module map, either explicitly named, or implicitly named via an +// umbrella directory or umbrella file, as parsed by the ModuleMap object. +// If headers are found which are not referenced or covered by an umbrella +// directory or file, warning messages will be produced, and this program +// will return an error code of 1. Other errors result in an error code of 2. +// If no problems are found, an error code of 0 is returned. +// +// Note that in the case of umbrella headers, this tool invokes the compiler +// to preprocess the file, and uses a callback to collect the header files +// included by the umbrella header or any of its nested includes. If any +// front end options are needed for these compiler invocations, these +// can be included on the command line after the module map file argument. +// +// Warning message have the form: +// +// warning: module.modulemap does not account for file: Level3A.h +// +// Note that for the case of the module map referencing a file that does +// not exist, the module map parser in Clang will (at the time of this +// writing) display an error message. +// +// Module Map Assistant - Module Map Generation +// +// Modularize has a mode of operation where it can generate a starting +// point module map based on a given header file list. In this mode it +// will skip the checks, and instead generate a module map file based +// on the header list. +// +// An optional "-root-module=(rootName)" argument can specify a root +// module to be created in the generated module.map file. +// +// Note that you will likely need to edit this file to suit the needs of your +// headers. // // An example command line for generating a module.map file: // -// modularize -module-map-path=module.map -root-module=myroot headerlist.txt +// modularize -module-map-output-path=module.modulemap \ +// -root-module=myroot headerlist.txt // // Note that if the headers in the header list have partial paths, sub-modules // will be created for the subdirectires involved, assuming that the @@ -109,30 +180,36 @@ // See the ModuleAssistant.cpp file comments for additional details about the // implementation of the assistant mode. // -// Future directions: +// Future directions/Potential problems: +// +// 1. Might need a better header matching mechanism, or extensions to the +// canonical file format used. // -// Basically, we want to add new checks for whatever we can check with respect -// to checking headers for module'ability. +// 2. It might need to support additional header file extensions. // -// Some ideas: +// 3. Basically, we want to add new checks for whatever we can check with respect +// to checking headers for module'ability. // -// 1. Omit duplicate "not always provided" messages +// 4. Add an option to fix the problems found, writing a new module map. +// Include an extra option to add unaccounted-for headers as excluded. // -// 2. Add options to disable any of the checks, in case -// there is some problem with them, or the messages get too verbose. +// 5. Omit duplicate "not always provided" messages // -// 3. Try to figure out the preprocessor conditional directives that -// contribute to problems and tie them to the inconsistent definitions. +// 6. Add options to disable any of the checks, in case +// there is some problem with them, or the messages get too verbose. // -// 4. There are some legitimate uses of preprocessor macros that -// modularize will flag as errors, such as repeatedly #include'ing -// a file and using interleaving defined/undefined macros -// to change declarations in the included file. Is there a way -// to address this? Maybe have modularize accept a list of macros -// to ignore. Otherwise you can just exclude the file, after checking -// for legitimate errors. +// 7. Try to figure out the preprocessor conditional directives that +// contribute to problems and tie them to the inconsistent definitions. // -// 5. What else? +// 8. There are some legitimate uses of preprocessor macros that +// modularize will flag as errors, such as repeatedly #include'ing +// a file and using interleaving defined/undefined macros +// to change declarations in the included file. Is there a way +// to address this? Maybe have modularize accept a list of macros +// to ignore. Otherwise you can just exclude the file, after checking +// for legitimate errors. +// +// 9. What else? // // General clean-up and refactoring: // @@ -142,8 +219,6 @@ // //===----------------------------------------------------------------------===// -#include "Modularize.h" -#include "PreprocessorTracker.h" #include "clang/AST/ASTConsumer.h" #include "clang/AST/ASTContext.h" #include "clang/AST/RecursiveASTVisitor.h" @@ -167,6 +242,9 @@ #include #include #include +#include "Modularize.h" +#include "ModuleMapChecker.h" +#include "PreprocessorTracker.h" using namespace clang; using namespace clang::driver; @@ -177,17 +255,18 @@ using namespace Modularize; // Option to specify a file name for a list of header files to check. -cl::opt -ListFileName(cl::Positional, - cl::desc("")); +static cl::opt +ModuleMapOrListInputFileName(cl::Positional, + cl::desc( + "")); // Collect all other arguments, which will be passed to the front end. -cl::list +static cl::list CC1Arguments(cl::ConsumeAfter, cl::desc("...")); // Option to specify a prefix to be prepended to the header names. -cl::opt HeaderPrefix( +static cl::opt HeaderPrefix( "prefix", cl::init(""), cl::desc( "Prepend header file paths with this prefix." @@ -196,12 +275,28 @@ // Option for assistant mode, telling modularize to output a module map // based on the headers list, and where to put it. -cl::opt ModuleMapPath( - "module-map-path", cl::init(""), +static cl::opt ModuleMapOutputPath( + "module-map-output-path", cl::init(""), cl::desc("Turn on module map output and specify output path or file name." " If no path is specified and if prefix option is specified," " use prefix for file path.")); +// Option for include paths. +static cl::list +IncludePaths("I", cl::desc("Include path." +" Must be relative to module.modulemap file."), +cl::ZeroOrMore, cl::value_desc("path")); + +// Option for just doing the coverage check. +static cl::opt +CoverageCheckOnly("coverage-check-only", cl::init(false), +cl::desc("Only do the coverage check.")); + +// Option for dumping the parsed module map. +static cl::opt +DumpModuleMap("dump-module-map", cl::init(false), +cl::desc("Dump the parsed module map information.")); + // Option for assistant mode, telling modularize to output a module map // based on the headers list, and where to put it. cl::opt @@ -216,75 +311,20 @@ // Read the header list file and collect the header file names and // optional dependencies. std::error_code -getHeaderFileNames(SmallVectorImpl &HeaderFileNames, - DependencyMap &Dependencies, StringRef ListFileName, - StringRef HeaderPrefix) { - // By default, use the path component of the list file name. - SmallString<256> HeaderDirectory(ListFileName); - sys::path::remove_filename(HeaderDirectory); - SmallString<256> CurrentDirectory; - sys::fs::current_path(CurrentDirectory); - - // Get the prefix if we have one. - if (HeaderPrefix.size() != 0) - HeaderDirectory = HeaderPrefix; - - // Read the header list file into a buffer. - ErrorOr> listBuffer = - MemoryBuffer::getFile(ListFileName); - if (std::error_code EC = listBuffer.getError()) - return EC; - - // Parse the header list into strings. - SmallVector Strings; - listBuffer.get()->getBuffer().split(Strings, "\n", -1, false); - - // Collect the header file names from the string list. - for (SmallVectorImpl::iterator I = Strings.begin(), - E = Strings.end(); - I != E; ++I) { - StringRef Line = I->trim(); - // Ignore comments and empty lines. - if (Line.empty() || (Line[0] == '#')) - continue; - std::pair TargetAndDependents = Line.split(':'); - SmallString<256> HeaderFileName; - // Prepend header file name prefix if it's not absolute. - if (sys::path::is_absolute(TargetAndDependents.first)) - llvm::sys::path::native(TargetAndDependents.first, HeaderFileName); - else { - if (HeaderDirectory.size() != 0) - HeaderFileName = HeaderDirectory; - else - HeaderFileName = CurrentDirectory; - sys::path::append(HeaderFileName, TargetAndDependents.first); - sys::path::native(HeaderFileName); - } - // Handle optional dependencies. - DependentsVector Dependents; - SmallVector DependentsList; - TargetAndDependents.second.split(DependentsList, " ", -1, false); - int Count = DependentsList.size(); - for (int Index = 0; Index < Count; ++Index) { - SmallString<256> Dependent; - if (sys::path::is_absolute(DependentsList[Index])) - Dependent = DependentsList[Index]; - else { - if (HeaderDirectory.size() != 0) - Dependent = HeaderDirectory; - else - Dependent = CurrentDirectory; - sys::path::append(Dependent, DependentsList[Index]); - } - sys::path::native(Dependent); - Dependents.push_back(Dependent.str()); - } - // Save the resulting header file path and dependencies. - HeaderFileNames.push_back(HeaderFileName.str()); - Dependencies[HeaderFileName.str()] = Dependents; +getHeaderFileNames(StringRef ModuleMapOrListInputFileName, + StringRef HeaderPrefix, ModuleMapChecker *checker) { + std::error_code EC; + if (ModuleMapOrListInputFileName.endswith(".modulemap") + || ModuleMapOrListInputFileName.endswith(".map")) { + // Load the input module map. + EC = checker->loadModuleMapAndCollectHeaders(); + } + else { + // Load the input file list and dependencies. + EC = checker->loadHeaderListAndDependencies( + ModuleMapOrListInputFileName, HeaderPrefix); } - - return std::error_code(); + return EC; } // Helper function for finding the input file in an arguments list. @@ -689,30 +729,47 @@ // This causes options to be parsed. cl::ParseCommandLineOptions(Argc, Argv, "modularize.\n"); - // No go if we have no header list file. - if (ListFileName.size() == 0) { + // No go if we have no module map or header list file. + if (ModuleMapOrListInputFileName.size() == 0) { cl::PrintHelpMessage(); return 1; } + // Create checker object. + std::unique_ptr Checker( + ModuleMapChecker::createModuleMapChecker(ModuleMapOrListInputFileName, + IncludePaths, DumpModuleMap, CC1Arguments)); + int HadErrors = 0; + // Get header file names and dependencies. - SmallVector Headers; - DependencyMap Dependencies; - if (std::error_code EC = getHeaderFileNames(Headers, Dependencies, - ListFileName, HeaderPrefix)) { - errs() << Argv[0] << ": error: Unable to get header list '" << ListFileName + if (std::error_code EC = getHeaderFileNames( + ModuleMapOrListInputFileName, HeaderPrefix, Checker.get())) { + errs() << Argv[0] << ": error: Unable to get module map or header list '" + << ModuleMapOrListInputFileName << "': " << EC.message() << '\n'; return 1; } // If we are in assistant mode, output the module map and quit. - if (ModuleMapPath.length() != 0) { - if (!createModuleMap(ModuleMapPath, Headers, Dependencies, HeaderPrefix, - RootModule)) + if (ModuleMapOutputPath.length() != 0) { + if (!createModuleMap( + ModuleMapOutputPath, Checker->HeaderFileNames, Checker->Dependencies, + HeaderPrefix, RootModule)) return 1; // Failed. return 0; // Success - Skip checks in assistant mode. } + // If we're doing a module map. + if (Checker->IsModuleMap) { + // Do coverage check. + if (Checker->doCoverageCheck()) + HadErrors = 1; + } + + // Bail early if only doing the coverage check. + if (CoverageCheckOnly) + return HadErrors; + // Create the compilation database. SmallString<256> PathBuf; sys::fs::current_path(PathBuf); @@ -725,9 +782,9 @@ // Parse all of the headers, detecting duplicates. EntityMap Entities; - ClangTool Tool(*Compilations, Headers); - Tool.appendArgumentsAdjuster(getAddDependenciesAdjuster(Dependencies)); - int HadErrors = 0; + ClangTool Tool(*Compilations, Checker->HeaderFileNames); + Tool.appendArgumentsAdjuster( + getAddDependenciesAdjuster(Checker->Dependencies)); ModularizeFrontendActionFactory Factory(Entities, *PPTracker, HadErrors); HadErrors |= Tool.run(&Factory); @@ -762,7 +819,8 @@ for (EntryBinArray::iterator DI = EntryBins.begin(), DE = EntryBins.end(); DI != DE; ++DI, ++KindIndex) { int ECount = DI->size(); - // If only 1 occurrence of this entity, skip it, as we only report duplicates. + // If only 1 occurrence of this entity, skip it, as we only report + // duplicates. if (ECount <= 1) continue; LocationArray::iterator FI = DI->begin(); Index: modularize/ModuleMapChecker.h =================================================================== --- modularize/ModuleMapChecker.h +++ modularize/ModuleMapChecker.h @@ -0,0 +1,253 @@ +//===-- ModuleMapChecker.h - Common defs for module-map-checker -*- C++ -*-==// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===--------------------------------------------------------------------===// +/// +/// \file +/// \brief ModuleMapChecker class definition. +/// +//===--------------------------------------------------------------------===// + +#ifndef MODULEMAPCHECKER_H +#define MODULEMAPCHECKER_H + +#include "Modularize.h" +#include "clang/Basic/Diagnostic.h" +#include "clang/Basic/FileManager.h" +#include "clang/Basic/LangOptions.h" +#include "clang/Basic/TargetInfo.h" +#include "clang/Basic/TargetOptions.h" +#include "clang/Frontend/TextDiagnosticPrinter.h" +#include "clang/Lex/HeaderSearch.h" +#include "clang/Lex/HeaderSearchOptions.h" +#include "clang/Lex/ModuleMap.h" +#include "clang/Lex/Preprocessor.h" +#include "llvm/ADT/SmallString.h" +#include "llvm/ADT/StringSet.h" +#include "llvm/Support/Host.h" +#include +#include + +namespace Modularize { + +/// Subclass TargetOptions so we can construct it inline with +/// the minimal option, the triple. +class ModuleMapTargetOptions : public clang::TargetOptions { +public: + ModuleMapTargetOptions() { Triple = llvm::sys::getDefaultTargetTriple(); } +}; + +/// Module map checker class. +/// This is the heart of the checker. +/// The doChecks function does the main work. +/// The data members store the options and internally collected data. +class ModuleMapChecker { + // Checker arguments. + +public: + /// The module.map file path. Can be relative or absolute. + llvm::StringRef ModuleMapPath; + /// The include paths to check for files. + /// (Note that other directories above these paths are ignored. + /// To expect all files to be accounted for from the module.map + /// file directory on down, leave this empty.) + std::vector IncludePaths; + /// Flag to dump the module map information during check. + bool DumpModuleMap; + /// The remaining arguments, to be passed to the front end. + llvm::ArrayRef CommandLine; + /// Flag indicating we are processing a module map. + bool IsModuleMap; + + // Supporting objects. + +protected: + /// Options controlling the language variant. + std::shared_ptr LangOpts; + /// Diagnostic IDs. + const llvm::IntrusiveRefCntPtr DiagIDs; + /// Options controlling the diagnostic engine. + llvm::IntrusiveRefCntPtr DiagnosticOpts; + /// Diagnostic consumer. + clang::TextDiagnosticPrinter DC; + /// Diagnostic engine. + llvm::IntrusiveRefCntPtr Diagnostics; + /// Options controlling the target. + std::shared_ptr TargetOpts; + /// Target information. + llvm::IntrusiveRefCntPtr Target; + /// Options controlling the file system manager. + clang::FileSystemOptions FileSystemOpts; + /// File system manager. + llvm::IntrusiveRefCntPtr FileMgr; + /// Source manager. + llvm::IntrusiveRefCntPtr SourceMgr; + /// Options controlling the \#include directive. + llvm::IntrusiveRefCntPtr HeaderSearchOpts; + /// Header search manager. + std::unique_ptr HeaderInfo; + /// The module map. + std::unique_ptr ModMap; + + // Internal data. + + /// Directory containing the module map. + /// Might be relative to the current directory, or absolute. + std::string ModuleMapDirectory; + /// Set of all the headers found in the module map. + llvm::StringSet ModuleMapHeadersSet; + /// All the headers found in the file system starting at the + /// module map, or the union of those from the include paths. + std::vector FileSystemHeaders; + /// Headers found in file system, but not in module map. + std::vector UnaccountedForHeaders; + + // Output data. + +public: + + // List of top-level header files. + llvm::SmallVector HeaderFileNames; + // Map of top-level header file dependencies. + DependencyMap Dependencies; + + // Functions. + + /// Constructor. + /// You can use the static createModuleMapChecker to create an instance + /// of this object. + /// \param ModuleMapPath The module.map file path. + /// Can be relative or absolute. + /// \param IncludePaths The include paths to check for files. + /// (Note that other directories above these paths are ignored. + /// To expect all files to be accounted for from the module.map + /// file directory on down, leave this empty.) + /// \param DumpModuleMap Flag to dump the module map information + /// during check. + ModuleMapChecker(llvm::StringRef ModuleMapPath, + std::vector &IncludePaths, bool DumpModuleMap, + llvm::ArrayRef CommandLine); + + /// Create instance of ModuleMapChecker. + /// \param ModuleMapPath The module.map file path. + /// Can be relative or absolute. + /// \param IncludePaths The include paths to check for files. + /// (Note that other directories above these paths are ignored. + /// To expect all files to be accounted for from the module.map + /// file directory on down, leave this empty.) + /// \param DumpModuleMap Flag to dump the module map information + /// during check. + /// \returns Initialized ModuleMapChecker object. + static ModuleMapChecker *createModuleMapChecker( + llvm::StringRef ModuleMapPath, std::vector &IncludePaths, + bool DumpModuleMap, llvm::ArrayRef CommandLine); + + /// Load header list and dependencies. + /// \param HeaderListInputFileName Header list file name. + /// \param HeaderPrefix Directory prefix to be prepended to header paths. + /// \returns std::error_code. + std::error_code loadHeaderListAndDependencies( + llvm::StringRef HeaderListInputFileName, llvm::StringRef HeaderPrefix); + + /// Load module map and collect headers and dependencies. + /// \returns std::error_code. + std::error_code loadModuleMapAndCollectHeaders(); + + /// Do coverage checks. + /// Starting from the directory of the module.map file, + /// Find all header files, optionally looking only at files + /// covered by the include path options, and compare against + /// the headers referenced by the module.map file. + /// Display warnings for unaccounted-for header files. + /// \returns 0 if there were no errors or warnings, 1 if there + /// were warnings, 2 if any other problem, such as a bad + /// module map path argument was specified. + std::error_code doCoverageCheck(); + + /// Load module map. + /// \returns True if module.map file loaded successfully. + bool loadModuleMap(); + + /// Collect module headers. + /// Walks the modules and collects referenced headers into + /// ModuleMapHeadersSet. + /// \return True if no errors. + bool collectModuleHeaders(); + + /// Collect referenced headers from one module. + /// Collects the headers referenced in the given module into + /// ModuleMapHeadersSet. + /// \param Mod The module reference. + /// \return True if no errors. + bool collectModuleHeaders(const clang::Module &Mod); + + /// Collect headers from an umbrella directory. + /// \param UmbrellaDirName The umbrella directory name. + /// \return True if no errors. + bool collectUmbrellaHeaders(llvm::StringRef UmbrellaDirName, + DependentsVector &Dependents); + + /// Collect headers rferenced from an umbrella file. + /// \param UmbrellaHeaderName The umbrella file path. + /// \return True if no errors. + bool collectUmbrellaHeaderHeaders(llvm::StringRef UmbrellaHeaderName); + + /// Called from ModuleMapCheckerCallbacks to track a header included + /// from an umbrella header. + /// \param HeaderName The header file path. + void collectUmbrellaHeaderHeader(llvm::StringRef HeaderName); + + /// Collect file system header files. + /// This function scans the file system for header files, + /// starting at the directory of the module.map file, + /// optionally filtering out all but the files covered by + /// the include path options. + /// \returns True if no errors. + bool collectFileSystemHeaders(); + + /// Collect file system header files from the given path. + /// This function scans the file system for header files, + /// starting at the given directory, which is assumed to be + /// relative to the directory of the module.map file. + /// \returns True if no errors. + bool collectFileSystemHeaders(llvm::StringRef IncludePath); + + /// Find headers unaccounted-for in module map. + /// This function compares the list of collected header files + /// against those referenced in the module map. Display + /// warnings for unaccounted-for header files. + /// Save unaccounted-for file list for possible. + /// fixing action. + void findUnaccountedForHeaders(); + + // Utility functions. + + /// Get directory path component from file path. + /// \returns the component of the given path, which will be + /// relative if the given path is relative, absolute if the + /// given path is absolute, or "." if the path has no leading + /// path component. + std::string getDirectoryFromPath(llvm::StringRef Path); + + /// Convert header path to canonical form. + /// The canonical form is basically just use forward slashes, + /// and remove "./". + /// \param FilePath The file path. + /// \returns The file path in canonical form. + std::string getCanonicalPath(llvm::StringRef FilePath); + + /// Check for header file extension. + /// If the file extension is .h, .inc, or missing, it's + /// assumed to be a header. + /// \param FileName The file name. Must not be a directory. + /// \returns true if it has a header extension or no extension. + bool isHeader(llvm::StringRef FileName); +}; + +} // end namespace Modularize + +#endif // MODULEMAPCHECKER_H Index: modularize/ModuleMapChecker.cpp =================================================================== --- modularize/ModuleMapChecker.cpp +++ modularize/ModuleMapChecker.cpp @@ -0,0 +1,603 @@ +//===--- extra/module-map-checker/ModuleMapChecker.cpp -------------------===// +// +// The LLVM Compiler Infrastructure +// +// This file is distributed under the University of Illinois Open Source +// License. See LICENSE.TXT for details. +// +//===----------------------------------------------------------------------===// +// +// This file implements a class for loading and validating a module map or +// header list by checking that all headers in the corresponding directories +// are accounted for. +// +//===----------------------------------------------------------------------===// + +#include "clang/AST/ASTConsumer.h" +#include "ModuleMapChecker.h" +#include "clang/AST/ASTContext.h" +#include "clang/AST/RecursiveASTVisitor.h" +#include "clang/Basic/SourceManager.h" +#include "clang/Driver/Options.h" +#include "clang/Frontend/CompilerInstance.h" +#include "clang/Frontend/FrontendActions.h" +#include "clang/Lex/PPCallbacks.h" +#include "clang/Lex/Preprocessor.h" +#include "clang/Tooling/CompilationDatabase.h" +#include "clang/Tooling/Tooling.h" +#include "llvm/Option/Option.h" +#include "llvm/Support/CommandLine.h" +#include "llvm/Support/FileSystem.h" +#include "llvm/Support/MemoryBuffer.h" +#include "llvm/Support/Path.h" +#include "llvm/Support/raw_ostream.h" + +using namespace clang; +using namespace clang::driver; +using namespace clang::driver::options; +using namespace clang::tooling; +using namespace llvm; +using namespace Modularize; +namespace cl = llvm::cl; +namespace sys = llvm::sys; + +// Preprocessor callbacks. +// We basically just collect include files. +class ModuleMapCheckerCallbacks : public PPCallbacks { +public: + ModuleMapCheckerCallbacks(ModuleMapChecker &Checker) : Checker(Checker) {} + ~ModuleMapCheckerCallbacks() {} + + // Include directive callback. + void InclusionDirective(SourceLocation HashLoc, const Token &IncludeTok, + StringRef FileName, bool IsAngled, + CharSourceRange FilenameRange, const FileEntry *File, + StringRef SearchPath, StringRef RelativePath, + const Module *Imported) { + Checker.collectUmbrellaHeaderHeader(File->getName()); + } + +private: + ModuleMapChecker &Checker; +}; + +// Frontend action stuff: + +// Consumer is responsible for setting up the callbacks. +class ModuleMapCheckerConsumer : public ASTConsumer { +public: + ModuleMapCheckerConsumer(ModuleMapChecker &Checker, Preprocessor &PP) { + // PP takes ownership. + PP.addPPCallbacks(llvm::make_unique(Checker)); + } +}; + +class ModuleMapCheckerAction : public SyntaxOnlyAction { +public: + ModuleMapCheckerAction(ModuleMapChecker &Checker) : Checker(Checker) {} + +protected: + std::unique_ptr CreateASTConsumer(CompilerInstance &CI, + StringRef InFile) override { + return llvm::make_unique(Checker, + CI.getPreprocessor()); + } + +private: + ModuleMapChecker &Checker; +}; + +class ModuleMapCheckerFrontendActionFactory : public FrontendActionFactory { +public: + ModuleMapCheckerFrontendActionFactory(ModuleMapChecker &Checker) + : Checker(Checker) {} + + virtual ModuleMapCheckerAction *create() { + return new ModuleMapCheckerAction(Checker); + } + +private: + ModuleMapChecker &Checker; +}; + +// ModuleMapChecker class implementation. + +// Constructor. +ModuleMapChecker::ModuleMapChecker(StringRef ModuleMapPath, + std::vector &IncludePaths, + bool DumpModuleMap, + ArrayRef CommandLine) + : ModuleMapPath(ModuleMapPath), IncludePaths(IncludePaths), + DumpModuleMap(DumpModuleMap), CommandLine(CommandLine), + IsModuleMap(false), + LangOpts(new LangOptions()), DiagIDs(new DiagnosticIDs()), + DiagnosticOpts(new DiagnosticOptions()), + DC(llvm::errs(), DiagnosticOpts.get()), + Diagnostics( + new DiagnosticsEngine(DiagIDs, DiagnosticOpts.get(), &DC, false)), + TargetOpts(new ModuleMapTargetOptions()), + Target(TargetInfo::CreateTargetInfo(*Diagnostics, TargetOpts)), + FileMgr(new FileManager(FileSystemOpts)), + SourceMgr(new SourceManager(*Diagnostics, *FileMgr, false)), + HeaderSearchOpts(new HeaderSearchOptions()), + HeaderInfo(new HeaderSearch(HeaderSearchOpts, *SourceMgr, *Diagnostics, + *LangOpts, Target.get())), + ModMap(new ModuleMap(*SourceMgr, *Diagnostics, *LangOpts, Target.get(), + *HeaderInfo)) {} + +// Create instance of ModuleMapChecker, to simplify setting up +// subordinate objects. +ModuleMapChecker *ModuleMapChecker::createModuleMapChecker( + StringRef ModuleMapPath, std::vector &IncludePaths, + bool DumpModuleMap, ArrayRef CommandLine) { + + return new ModuleMapChecker(ModuleMapPath, IncludePaths, DumpModuleMap, + CommandLine); +} + +// Load header list and dependencies. +std::error_code ModuleMapChecker::loadHeaderListAndDependencies( + StringRef HeaderListInputFileName, StringRef HeaderPrefix) { + + // Not doing a module map. + IsModuleMap = false; + + // By default, use the path component of the list file name. + SmallString<256> HeaderDirectory(HeaderListInputFileName); + llvm::sys::path::remove_filename(HeaderDirectory); + SmallString<256> CurrentDirectory; + llvm::sys::fs::current_path(CurrentDirectory); + + // Get the prefix if we have one. + if (HeaderPrefix.size() != 0) + HeaderDirectory = HeaderPrefix; + + // Read the header list file into a buffer. + ErrorOr> listBuffer = + MemoryBuffer::getFile(HeaderListInputFileName); + if (std::error_code EC = listBuffer.getError()) + return EC; + + // Parse the header list into strings. + SmallVector Strings; + listBuffer.get()->getBuffer().split(Strings, "\n", -1, false); + + // Collect the header file names from the string list. + for (SmallVectorImpl::iterator I = Strings.begin(), + E = Strings.end(); + I != E; ++I) { + StringRef Line = I->trim(); + // Ignore comments and empty lines. + if (Line.empty() || (Line[0] == '#')) + continue; + std::pair TargetAndDependents = Line.split(':'); + SmallString<256> HeaderFileName; + // Prepend header file name prefix if it's not absolute. + if (llvm::sys::path::is_absolute(TargetAndDependents.first)) + llvm::sys::path::native(TargetAndDependents.first, HeaderFileName); + else { + if (HeaderDirectory.size() != 0) + HeaderFileName = HeaderDirectory; + else + HeaderFileName = CurrentDirectory; + llvm::sys::path::append(HeaderFileName, TargetAndDependents.first); + llvm::sys::path::native(HeaderFileName); + } + // Handle optional dependencies. + DependentsVector Dependents; + SmallVector DependentsList; + TargetAndDependents.second.split(DependentsList, " ", -1, false); + int Count = DependentsList.size(); + for (int Index = 0; Index < Count; ++Index) { + SmallString<256> Dependent; + if (llvm::sys::path::is_absolute(DependentsList[Index])) + Dependent = DependentsList[Index]; + else { + if (HeaderDirectory.size() != 0) + Dependent = HeaderDirectory; + else + Dependent = CurrentDirectory; + llvm::sys::path::append(Dependent, DependentsList[Index]); + } + llvm::sys::path::native(Dependent); + Dependents.push_back(Dependent.str()); + } + // Save the resulting header file path and dependencies. + ModuleMapHeadersSet.insert(HeaderFileName); + HeaderFileNames.push_back(HeaderFileName.str()); + Dependencies[HeaderFileName.str()] = Dependents; + } + return std::error_code(); +} + +// Load module map and collect headers. +std::error_code ModuleMapChecker::loadModuleMapAndCollectHeaders() { + // Load the module map. + if (!loadModuleMap()) + return std::error_code(2, std::generic_category()); + + // Collect the headers referenced in the modules. + if (!collectModuleHeaders()) + return std::error_code(2, std::generic_category()); + + return std::error_code(); +} + +// Do coverage checks. +// Starting from the directory of the module.modulemap file, +// Find all header files, optionally looking only at files +// covered by the include path options, and compare against +// the headers referenced by the module.modulemap file. +// Display warnings for unaccounted-for header files. +// Returns error_code of 0 if there were no errors or warnings, 1 if there +// were warnings, 2 if any other problem, such as if a bad +// module map path argument was specified. +std::error_code ModuleMapChecker::doCoverageCheck() { + std::error_code returnValue; + + // If we haven't already loaded the headers, do so now. + if (ModuleMapHeadersSet.size() == 0) { + // Load the module map. + if (!loadModuleMap()) + return std::error_code(2, std::generic_category()); + + // Collect the headers referenced in the modules. + if (!collectModuleHeaders()) + return std::error_code(2, std::generic_category()); + } + + // Collect the file system headers. + if (!collectFileSystemHeaders()) + return std::error_code(2, std::generic_category()); + + // Do the checks. These save the problematic file names. + findUnaccountedForHeaders(); + + // Check for warnings. + if (UnaccountedForHeaders.size()) + returnValue = std::error_code(1, std::generic_category()); + + // Dump module map if requested. + if (DumpModuleMap) { + llvm::errs() << "\nDump of module map:\n\n"; + ModMap->dump(); + } + + return returnValue; +} + +// The following functions are called by doChecks. + +// Load module map. +// Returns true if module.modulemap file loaded successfully. +bool ModuleMapChecker::loadModuleMap() { + // Get file entry for module.modulemap file. + const FileEntry *ModuleMapEntry = + SourceMgr->getFileManager().getFile(ModuleMapPath); + + // return error if not found. + if (!ModuleMapEntry) { + llvm::errs() << "error: File \"" << ModuleMapPath << "\" not found.\n"; + return false; + } + + // Doing a module map. + IsModuleMap = true; + + // Because the module map parser uses a ForwardingDiagnosticConsumer, + // which doesn't forward the BeginSourceFile call, we do it explicitly here. + DC.BeginSourceFile(*LangOpts, nullptr); + + // Figure out the home directory for the module map file. + // FIXME: Add an option to specify this. + const DirectoryEntry *Dir = ModuleMapEntry->getDir(); + StringRef DirName(Dir->getName()); + if (llvm::sys::path::filename(DirName) == "Modules") { + DirName = llvm::sys::path::parent_path(DirName); + if (DirName.endswith(".framework")) + Dir = FileMgr->getDirectory(DirName); + // FIXME: This assert can fail if there's a race between the above check + // and the removal of the directory. + assert(Dir && "parent must exist"); + } + + // Parse module.modulemap file into module map. + if (ModMap->parseModuleMapFile(ModuleMapEntry, false, Dir)) + return false; + + // Do matching end call. + DC.EndSourceFile(); + + return true; +} + +// Collect module headers. +// Walks the modules and collects referenced headers into +// ModuleMapHeadersSet. +bool ModuleMapChecker::collectModuleHeaders() { + for (ModuleMap::module_iterator I = ModMap->module_begin(), + E = ModMap->module_end(); + I != E; ++I) { + if (!collectModuleHeaders(*I->second)) + return false; + } + return true; +} + +// Collect referenced headers from one module. +// Collects the headers referenced in the given module into +// ModuleMapHeadersSet. +bool ModuleMapChecker::collectModuleHeaders(const Module &Mod) { + + // Treat headers in umbrella directory as dependencies. + DependentsVector Dependents; + + // Recursively do submodules. + for (Module::submodule_const_iterator MI = Mod.submodule_begin(), + MIEnd = Mod.submodule_end(); + MI != MIEnd; ++MI) + collectModuleHeaders(**MI); + + if (const FileEntry *UmbrellaHeader = Mod.getUmbrellaHeader()) { + // Collect umbrella header. + ModuleMapHeadersSet.insert(getCanonicalPath(UmbrellaHeader->getName())); + HeaderFileNames.push_back(UmbrellaHeader->getName()); + + // Preprocess umbrella header and collect the headers it references. + if (!collectUmbrellaHeaderHeaders(UmbrellaHeader->getName())) + return false; + } else if (const DirectoryEntry *UmbrellaDir = Mod.getUmbrellaDir()) { + // Collect headers in umbrella directory. + if (!collectUmbrellaHeaders(UmbrellaDir->getName(), Dependents)) + return false; + } + + // Collect the explicit headers. + int HeaderCount = 0; + for (auto &HeaderKind : Mod.Headers) { + for (auto &Header : HeaderKind) { + std::string HeaderPath = getCanonicalPath(Header.Entry->getName()); + ModuleMapHeadersSet.insert(HeaderPath); + HeaderFileNames.push_back(HeaderPath); + HeaderCount++; + + if (Dependents.size() != 0) + Dependencies[HeaderPath] = Dependents; + } + } + + // If there were umbrella files but no headers, treat them all as top-level. + if ((HeaderCount == 0) && (Dependents.size() != 0)) { + for (auto &Header : Dependents) + HeaderFileNames.push_back(Header); + } + + return true; +} + +// Collect headers from an umbrella directory. +bool ModuleMapChecker::collectUmbrellaHeaders(StringRef UmbrellaDirName, + DependentsVector &Dependents) { + // Initialize directory name. + SmallString<256> Directory(ModuleMapDirectory); + if (UmbrellaDirName.size()) + llvm::sys::path::append(Directory, UmbrellaDirName); + if (Directory.size() == 0) + Directory = "."; + // Walk the directory. + std::error_code EC; + llvm::sys::fs::file_status Status; + for (llvm::sys::fs::directory_iterator I(Directory.str(), EC), E; I != E; + I.increment(EC)) { + if (EC) + return false; + std::string File(I->path()); + I->status(Status); + llvm::sys::fs::file_type Type = Status.type(); + // If the file is a directory, ignore the name. + if (Type == llvm::sys::fs::file_type::directory_file) { + if (!collectUmbrellaHeaders(File, Dependents)) + return false; + continue; + } + // If the file does not have a common header extension, ignore it. + if (!isHeader(File)) + continue; + // Save header name. + std::string HeaderPath = getCanonicalPath(File); + ModuleMapHeadersSet.insert(HeaderPath); + Dependents.push_back(HeaderPath); + } + return true; +} + +// Collect headers rferenced from an umbrella file. +bool +ModuleMapChecker::collectUmbrellaHeaderHeaders(StringRef UmbrellaHeaderName) { + + SmallString<256> PathBuf(ModuleMapDirectory); + + // If directory is empty, it's the current directory. + if (ModuleMapDirectory.length() == 0) + llvm::sys::fs::current_path(PathBuf); + + // Create the compilation database. + std::unique_ptr Compilations; + Compilations.reset(new FixedCompilationDatabase(Twine(PathBuf), CommandLine)); + + std::vector HeaderPath; + HeaderPath.push_back(UmbrellaHeaderName); + + // Create the tool and run the compilation. + ClangTool Tool(*Compilations, HeaderPath); + int HadErrors = Tool.run(new ModuleMapCheckerFrontendActionFactory(*this)); + + // If we had errors, exit early. + return HadErrors ? false : true; +} + +// Called from ModuleMapCheckerCallbacks to track a header included +// from an umbrella header. +void ModuleMapChecker::collectUmbrellaHeaderHeader(StringRef HeaderName) { + + SmallString<256> PathBuf(ModuleMapDirectory); + // If directory is empty, it's the current directory. + if (ModuleMapDirectory.length() == 0) + llvm::sys::fs::current_path(PathBuf); + // HeaderName will have an absolute path, so if it's the module map + // directory, we remove it, also skipping trailing separator. + if (HeaderName.startswith(PathBuf)) + HeaderName = HeaderName.substr(PathBuf.size() + 1); + // Save header name. + ModuleMapHeadersSet.insert(getCanonicalPath(HeaderName)); +} + +// Collect file system header files. +// This function scans the file system for header files, +// starting at the directory of the module.modulemap file, +// optionally filtering out all but the files covered by +// the include path options. +// Returns true if no errors. +bool ModuleMapChecker::collectFileSystemHeaders() { + + // Get directory containing the module.modulemap file. + // Might be relative to current directory, absolute, or empty. + ModuleMapDirectory = getDirectoryFromPath(ModuleMapPath); + + // If no include paths specified, we do the whole tree starting + // at the module.modulemap directory. + if (IncludePaths.size() == 0) { + if (!collectFileSystemHeaders(StringRef(""))) + return false; + } else { + // Otherwise we only look at the sub-trees specified by the + // include paths. + for (std::vector::const_iterator I = IncludePaths.begin(), + E = IncludePaths.end(); + I != E; ++I) { + if (!collectFileSystemHeaders(*I)) + return false; + } + } + + // Sort it, because different file systems might order the file differently. + std::sort(FileSystemHeaders.begin(), FileSystemHeaders.end()); + + return true; +} + +// Collect file system header files from the given path. +// This function scans the file system for header files, +// starting at the given directory, which is assumed to be +// relative to the directory of the module.modulemap file. +// \returns True if no errors. +bool ModuleMapChecker::collectFileSystemHeaders(StringRef IncludePath) { + + // Initialize directory name. + SmallString<256> Directory(ModuleMapDirectory); + if (IncludePath.size()) + llvm::sys::path::append(Directory, IncludePath); + if (Directory.size() == 0) + Directory = "."; + if (IncludePath.startswith("/") || IncludePath.startswith("\\") || + ((IncludePath.size() >= 2) && (IncludePath[1] == ':'))) { + llvm::errs() << "error: Include path \"" << IncludePath + << "\" is not relative to the module map file.\n"; + return false; + } + + // Recursively walk the directory tree. + std::error_code EC; + llvm::sys::fs::file_status Status; + int Count = 0; + for (llvm::sys::fs::recursive_directory_iterator I(Directory.str(), EC), E; + I != E; I.increment(EC)) { + if (EC) + return false; + std::string file(I->path()); + I->status(Status); + llvm::sys::fs::file_type type = Status.type(); + // If the file is a directory, ignore the name (but still recurses). + if (type == llvm::sys::fs::file_type::directory_file) + continue; + // If the file does not have a common header extension, ignore it. + if (!isHeader(file)) + continue; + // Save header name. + FileSystemHeaders.push_back(getCanonicalPath(file)); + Count++; + } + if (Count == 0) { + llvm::errs() << "warning: No headers found in include path: \"" + << IncludePath << "\"\n"; + } + return true; +} + +// Find headers unaccounted-for in module map. +// This function compares the list of collected header files +// against those referenced in the module map. Display +// warnings for unaccounted-for header files. +// Save unaccounted-for file list for possible. +// fixing action. +// FIXME: There probably needs to be some canonalization +// of file names so that header path can be correctly +// matched. Also, a map could be used for the headers +// referenced in the module, but +void ModuleMapChecker::findUnaccountedForHeaders() { + // Walk over file system headers. + for (std::vector::const_iterator I = FileSystemHeaders.begin(), + E = FileSystemHeaders.end(); + I != E; ++I) { + // Look for header in module map. + if (ModuleMapHeadersSet.insert(*I).second) { + UnaccountedForHeaders.push_back(*I); + llvm::errs() << "warning: " << ModuleMapPath + << " does not account for file: " << *I << "\n"; + } + } +} + +// Utility functions. + +// Get directory path component from file path. +// \returns the component of the given path, which will be +// relative if the given path is relative, absolute if the +// given path is absolute, or "." if the path has no leading +// path component. +std::string ModuleMapChecker::getDirectoryFromPath(StringRef Path) { + SmallString<256> Directory(Path); + llvm::sys::path::remove_filename(Directory); + if (Directory.size() == 0) + return "."; + return Directory.str(); +} + +// Convert header path to canonical form. +// The canonical form is basically just use forward slashes, and remove "./". +// \param FilePath The file path, relative to the module map directory. +// \returns The file path in canonical form. +std::string ModuleMapChecker::getCanonicalPath(StringRef FilePath) { + std::string Tmp(FilePath); + std::replace(Tmp.begin(), Tmp.end(), '\\', '/'); + StringRef Result(Tmp); + if (Result.startswith("./")) + Result = Result.substr(2); + return Result; +} + +// Check for header file extension. +// If the file extension is .h, .inc, or missing, it's +// assumed to be a header. +// \param FileName The file name. Must not be a directory. +// \returns true if it has a header extension or no extension. +bool ModuleMapChecker::isHeader(StringRef FileName) { + StringRef Extension = llvm::sys::path::extension(FileName); + if (Extension.size() == 0) + return false; + if (Extension.equals_lower(".h")) + return true; + if (Extension.equals_lower(".inc")) + return true; + return false; +} Index: module-map-checker/ModuleMapChecker.cpp =================================================================== --- module-map-checker/ModuleMapChecker.cpp +++ module-map-checker/ModuleMapChecker.cpp @@ -15,8 +15,8 @@ // // Options: // -// -I(include path) Look at headers only in this directory tree. -// Must be a path relative to the module.map file. +// -I (include path) Look at headers only in this directory tree. +// Must be a path relative to the module.modulemap file. // There can be multiple -I options, for when the // module map covers multiple directories, and // excludes higher or sibling directories not @@ -52,7 +52,7 @@ // // Warning message have the form: // -// warning: module.map does not account for file: Level3A.h +// warning: module.modulemap does not account for file: Level3A.h // // Note that for the case of the module map referencing a file that does // not exist, the module map parser in Clang will (at the time of this @@ -100,7 +100,7 @@ // Option for include paths. static cl::list IncludePaths("I", cl::desc("Include path." - " Must be relative to module.map file."), + " Must be relative to module.modulemap file."), cl::ZeroOrMore, cl::value_desc("path")); // Option for dumping the parsed module map. @@ -108,11 +108,11 @@ DumpModuleMap("dump-module-map", cl::init(false), cl::desc("Dump the parsed module map information.")); -// Option for module.map path. +// Option for module.modulemap path. static cl::opt -ModuleMapPath(cl::Positional, cl::init("module.map"), - cl::desc("")); +ModuleMapPath(cl::Positional, cl::init("module.modulemap"), + cl::desc("")); // Collect all other arguments, which will be passed to the front end. static cl::list @@ -235,10 +235,10 @@ } // Do checks. -// Starting from the directory of the module.map file, +// Starting from the directory of the module.modulemap file, // Find all header files, optionally looking only at files // covered by the include path options, and compare against -// the headers referenced by the module.map file. +// the headers referenced by the module.modulemap file. // Display warnings for unaccounted-for header files. // Returns error_code of 0 if there were no errors or warnings, 1 if there // were warnings, 2 if any other problem, such as if a bad @@ -276,9 +276,9 @@ // The following functions are called by doChecks. // Load module map. -// Returns true if module.map file loaded successfully. +// Returns true if module.modulemap file loaded successfully. bool ModuleMapChecker::loadModuleMap() { - // Get file entry for module.map file. + // Get file entry for module.modulemap file. const FileEntry *ModuleMapEntry = SourceMgr->getFileManager().getFile(ModuleMapPath); @@ -305,7 +305,7 @@ assert(Dir && "parent must exist"); } - // Parse module.map file into module map. + // Parse module.modulemap file into module map. if (ModMap->parseModuleMapFile(ModuleMapEntry, false, Dir)) return false; @@ -375,8 +375,11 @@ I->status(Status); sys::fs::file_type Type = Status.type(); // If the file is a directory, ignore the name. - if (Type == sys::fs::file_type::directory_file) + if (Type == sys::fs::file_type::directory_file) { + if (!collectUmbrellaHeaders(File)) + return false; continue; + } // If the file does not have a common header extension, ignore it. if (!isHeader(File)) continue; @@ -429,18 +432,18 @@ // Collect file system header files. // This function scans the file system for header files, -// starting at the directory of the module.map file, +// starting at the directory of the module.modulemap file, // optionally filtering out all but the files covered by // the include path options. // Returns true if no errors. bool ModuleMapChecker::collectFileSystemHeaders() { - // Get directory containing the module.map file. + // Get directory containing the module.modulemap file. // Might be relative to current directory, absolute, or empty. ModuleMapDirectory = getDirectoryFromPath(ModuleMapPath); // If no include paths specified, we do the whole tree starting - // at the module.map directory. + // at the module.modulemap directory. if (IncludePaths.size() == 0) { if (!collectFileSystemHeaders(StringRef(""))) return false; @@ -464,7 +467,7 @@ // Collect file system header files from the given path. // This function scans the file system for header files, // starting at the given directory, which is assumed to be -// relative to the directory of the module.map file. +// relative to the directory of the module.modulemap file. // \returns True if no errors. bool ModuleMapChecker::collectFileSystemHeaders(StringRef IncludePath) { Index: test/modularize/Inputs/CoverageNoProblems/Includes1/Level1A.h =================================================================== --- test/modularize/Inputs/CoverageNoProblems/Includes1/Level1A.h +++ test/modularize/Inputs/CoverageNoProblems/Includes1/Level1A.h @@ -0,0 +1 @@ +#define MACRO_1A 1 Index: test/modularize/Inputs/CoverageNoProblems/Includes2/Level2A.h =================================================================== --- test/modularize/Inputs/CoverageNoProblems/Includes2/Level2A.h +++ test/modularize/Inputs/CoverageNoProblems/Includes2/Level2A.h @@ -0,0 +1 @@ +#define MACRO_2A 1 Index: test/modularize/Inputs/CoverageNoProblems/NonIncludes/Level3A.h =================================================================== --- test/modularize/Inputs/CoverageNoProblems/NonIncludes/Level3A.h +++ test/modularize/Inputs/CoverageNoProblems/NonIncludes/Level3A.h @@ -0,0 +1 @@ +#define MACRO_3A 1 Index: test/modularize/Inputs/CoverageNoProblems/module.modulemap =================================================================== --- test/modularize/Inputs/CoverageNoProblems/module.modulemap +++ test/modularize/Inputs/CoverageNoProblems/module.modulemap @@ -0,0 +1,10 @@ +// module.map + +module Level1A { + header "Includes1/Level1A.h" + export * +} +module Level2A { + header "Includes2/Level2A.h" + export * +} Index: test/modularize/Inputs/CoverageProblems/Level1A.h =================================================================== --- test/modularize/Inputs/CoverageProblems/Level1A.h +++ test/modularize/Inputs/CoverageProblems/Level1A.h @@ -0,0 +1,2 @@ +#include "Level2A.h" +#define MACRO_1A 1 Index: test/modularize/Inputs/CoverageProblems/Level1B.h =================================================================== --- test/modularize/Inputs/CoverageProblems/Level1B.h +++ test/modularize/Inputs/CoverageProblems/Level1B.h @@ -0,0 +1,2 @@ +#include "Level2B.h" +#define MACRO_1B 1 Index: test/modularize/Inputs/CoverageProblems/Level2A.h =================================================================== --- test/modularize/Inputs/CoverageProblems/Level2A.h +++ test/modularize/Inputs/CoverageProblems/Level2A.h @@ -0,0 +1 @@ +#define MACRO_2A 1 Index: test/modularize/Inputs/CoverageProblems/Level2B.h =================================================================== --- test/modularize/Inputs/CoverageProblems/Level2B.h +++ test/modularize/Inputs/CoverageProblems/Level2B.h @@ -0,0 +1 @@ +#define MACRO_2B 1 Index: test/modularize/Inputs/CoverageProblems/Level3A.h =================================================================== --- test/modularize/Inputs/CoverageProblems/Level3A.h +++ test/modularize/Inputs/CoverageProblems/Level3A.h @@ -0,0 +1,2 @@ +#include "Sub/Level3B.h" +#define MACRO_3A 1 Index: test/modularize/Inputs/CoverageProblems/Sub/Level3B.h =================================================================== --- test/modularize/Inputs/CoverageProblems/Sub/Level3B.h +++ test/modularize/Inputs/CoverageProblems/Sub/Level3B.h @@ -0,0 +1 @@ +#define MACRO_3B 1 Index: test/modularize/Inputs/CoverageProblems/UmbrellaFile.h =================================================================== --- test/modularize/Inputs/CoverageProblems/UmbrellaFile.h +++ test/modularize/Inputs/CoverageProblems/UmbrellaFile.h @@ -0,0 +1,3 @@ +#define UMBRELLA_HEADER 1 +#include "UmbrellaInclude1.h" +#include "UmbrellaInclude2.h" Index: test/modularize/Inputs/CoverageProblems/UmbrellaInclude1.h =================================================================== --- test/modularize/Inputs/CoverageProblems/UmbrellaInclude1.h +++ test/modularize/Inputs/CoverageProblems/UmbrellaInclude1.h @@ -0,0 +1 @@ +#define UMBRELLA_INCLUDE_1 1 Index: test/modularize/Inputs/CoverageProblems/UmbrellaInclude2.h =================================================================== --- test/modularize/Inputs/CoverageProblems/UmbrellaInclude2.h +++ test/modularize/Inputs/CoverageProblems/UmbrellaInclude2.h @@ -0,0 +1 @@ +#define UMBRELLA_INCLUDE_2 1 Index: test/modularize/Inputs/CoverageProblems/UmbrellaSub/Umbrell1.h =================================================================== --- test/modularize/Inputs/CoverageProblems/UmbrellaSub/Umbrell1.h +++ test/modularize/Inputs/CoverageProblems/UmbrellaSub/Umbrell1.h @@ -0,0 +1 @@ +#define UMBRELLA_1 1 Index: test/modularize/Inputs/CoverageProblems/UmbrellaSub/Umbrell2.h =================================================================== --- test/modularize/Inputs/CoverageProblems/UmbrellaSub/Umbrell2.h +++ test/modularize/Inputs/CoverageProblems/UmbrellaSub/Umbrell2.h @@ -0,0 +1 @@ +#define UMBRELLA_2 1 Index: test/modularize/Inputs/CoverageProblems/module.modulemap =================================================================== --- test/modularize/Inputs/CoverageProblems/module.modulemap +++ test/modularize/Inputs/CoverageProblems/module.modulemap @@ -0,0 +1,30 @@ +// module.map + +module Level1A { + header "Level1A.h" + export * +} +module Level1B { + header "Level1B.h" + export * + module Level2B { + header "Level2B.h" + export * + } +} +module Level2A { + header "Level2A.h" + export * +} +module UmbrellaDirectoryModule { + umbrella "UmbrellaSub" +} +module UmbrellaHeaderModule { + umbrella header "UmbrellaFile.h" +} +/* +module NoHeader { + header "NoHeader.h" + export * +} +*/ Index: test/modularize/NoProblemsAssistant.modularize =================================================================== --- test/modularize/NoProblemsAssistant.modularize +++ test/modularize/NoProblemsAssistant.modularize @@ -1,4 +1,4 @@ -# RUN: modularize -module-map-path=Output/NoProblemsAssistant.txt -root-module=Root -prefix=%S/Input %s +# RUN: modularize -module-map-output-path=Output/NoProblemsAssistant.txt -root-module=Root -prefix=%S/Input %s # RUN: FileCheck --input-file=%T/NoProblemsAssistant.txt %s SomeTypes.h @@ -10,7 +10,7 @@ SubModule2.h # CHECK: // Output/NoProblemsAssistant.txt -# CHECK-NEXT: // Generated by: modularize -module-map-path=Output/NoProblemsAssistant.txt -root-module=Root -prefix={{.*}}{{[/\\]}}{{.*}} {{.*}}{{[/\\]}}NoProblemsAssistant.modularize +# CHECK-NEXT: // Generated by: modularize -module-map-output-path=Output/NoProblemsAssistant.txt -root-module=Root -prefix={{.*}}{{[/\\]}}{{.*}} {{.*}}{{[/\\]}}NoProblemsAssistant.modularize # CHECK: module Root { # CHECK-NEXT: module SomeTypes { # CHECK-NEXT: header "SomeTypes.h" Index: test/modularize/NoProblemsCoverage.modularize =================================================================== --- test/modularize/NoProblemsCoverage.modularize +++ test/modularize/NoProblemsCoverage.modularize @@ -0,0 +1 @@ +# RUN: modularize -I Includes1 -I Includes2 %S/Inputs/CoverageNoProblems/module.modulemap Index: test/modularize/ProblemsCoverage.modularize =================================================================== --- test/modularize/ProblemsCoverage.modularize +++ test/modularize/ProblemsCoverage.modularize @@ -0,0 +1,4 @@ +# RUN: not modularize %S/Inputs/CoverageProblems/module.modulemap 2>&1 | FileCheck %s + +# CHECK: warning: {{.*}}{{[/\\]}}Inputs/CoverageProblems/module.modulemap does not account for file: {{.*}}{{[/\\]}}Inputs/CoverageProblems/Level3A.h +# CHECK-NEXT: warning: {{.*}}{{[/\\]}}Inputs/CoverageProblems/module.modulemap does not account for file: {{.*}}{{[/\\]}}Inputs/CoverageProblems/Sub/Level3B.h