Install the LLVM header module.modulemaps

[EricWF]

Summary

Currently the LLVM module.modulemap files are not installed along side the headers. This is because these module maps only work during in-tree builds where some of the headers are in the build directory. Once those generated headers are installed with the in-tree headers the module maps no longer work.

This patch adds a new map called module.modulemap.install which correctly handles the LLVM headers in their install configuration. The changes needed to support that are:

1. Build llvm/Support/DataTypes.h as a separate module instead of as part of LLVM_Utils. This is required to avoid a cyclic dependency between the LLVM_C and LLVM_Utils modules.
2. Add the generated Config/*.def headers to the LLVM_Config module.
3. Handle the IR/Attributes.gen and IR/Intrinsics.gen headers so that the LLVM_IR umbrella is complete.

Unfortunately I don't think there is a way to modify the existing module.modulemap file so that it works with both in-tree and installed headers. @rsmith is there a better way to handle this?

[EricWF]

• The foo.gen headers don't need to be textual, so don't mark them as such.

[EricWF]

• Change the foo.gen headers back to being textual. I was incorrect in saying they didn't need to be.

[v.g.vassilev]

Can't we concatenate the modulemap in the source directory with the modulemap in the build directory and publish them in the install folder?

[EricWF]

In D27396#613150, @v.g.vassilev wrote:

Can't we concatenate the modulemap in the source directory with the modulemap in the build directory and publish them in the install folder?

Unfortunately not. The module.modulemap.install file provides *different* definitions for existing modules. For example LLVM_Config is defined in module.modulemap as

module LLVM_Config { requires cplusplus umbrella "Config" module * { export * } }

but module.modulemap.install needs to take the generated headers into account, which leads to the differing definition below:,

module LLVM_Config {
  requires cplusplus
  umbrella "Config"
  module * { export * }
 textual header "Config/AsmParsers.def"
 textual header "Config/AsmPrinters.def"
 textual header "Config/Disassemblers.def"
 textual header "Config/Targets.def"
}

We can't easily achieve this by concatenating the maps. We also can't name these additional headers in the module.modulemap file since they don't exist as part of that modules "umbrella" until after installation.

[v.g.vassilev]

Can we use extern module in that case. It seems that it could help reducing the duplication:

module LLVM_Config_Extended {
 textual header "Config/AsmParsers.def"
 textual header "Config/AsmPrinters.def"
 textual header "Config/Disassemblers.def"
 textual header "Config/Targets.def"
  use "LLVM_Config"
}
extern module "LLVM_Config" "our_installed_src_module.modulemap"

[EricWF]

In D27396#613194, @v.g.vassilev wrote:

Can we use extern module in that case. It seems that it could help reducing the duplication:

IDK, I'll give it a shot! Thanks for the advice.

[EricWF]

• Use extern module instead of an entirely new module map.

[EricWF]

• Fix copy-paste error

[EricWF]

• Fix CMake changes to match existing style.
• Move the IR/Intrinsics.gen header into the LLVM_Intrinsics module instead of its parent LLVM_intrinsics_gen

[EricWF]

• Fix installation directory for module.install.modulemap.

I've tested this patch against a full in-tree build with modules enabled and full out-of-tree clang builds.

[v.g.vassilev]

Thanks for working on this! LGTM.

[EricWF]

After installing the module.modulemap LLVM fails to build because when building the modules it finds both the in-tree map and installed map. I think fixing this will require changing the Clang module.modulemap search behavior.

[v.g.vassilev]

Hm, do we have both -I pointing to the folders of the installed modulemap and the modulemap in the source? Or it is because of the module.extern.modulemap?

[v.g.vassilev]

@EricFW, ping. I was wondering whether there are news about the planned changes.

[EricWF]

So there is a bug tracking the Clang changes needed: https://bugs.llvm.org/show_bug.cgi?id=31905
I've tried to fix the bug myself but I haven't had any luck.

Once that gets fixed these changes are good to go, I still use them every day.

[EricWF]

• Merging with master in case other people want to use this.

It's still waiting on http://llvm.org/PR31905 to be fixed though.

CMakeLists.txt

install(DIRECTORY include/llvm include/llvm-c
     DESTINATION include
     COMPONENT llvm-headers
     FILES_MATCHING
     PATTERN "*.def"
     PATTERN "*.h"
     PATTERN "*.td"
     PATTERN "*.inc"
     PATTERN "LICENSE.TXT"
+    PATTERN "module.modulemap"
     PATTERN ".svn" EXCLUDE
     )
 
   install(DIRECTORY ${LLVM_INCLUDE_DIR}/llvm
     DESTINATION include
     COMPONENT llvm-headers
     FILES_MATCHING
     PATTERN "*.def"
     PATTERN "*.h"
     PATTERN "*.gen"
     PATTERN "*.inc"
     # Exclude include/llvm/CMakeFiles/intrinsics_gen.dir, matched by "*.def"
     PATTERN "CMakeFiles" EXCLUDE
     PATTERN "config.h" EXCLUDE
     PATTERN ".svn" EXCLUDE
     )
 
+  install(FILES include/llvm/module.install.modulemap
+    DESTINATION include/llvm
+    COMPONENT llvm-headers
+    RENAME "module.extern.modulemap"
+    )
+
   # Installing the headers needs to depend on generating any public
   # tablegen'd headers.
   add_custom_target(llvm-headers DEPENDS intrinsics_gen)
 
   if (NOT CMAKE_CONFIGURATION_TYPES)
     add_custom_target(install-llvm-headers
                       DEPENDS llvm-headers
                       COMMAND "${CMAKE_COMMAND}"

include/llvm/module.extern.modulemap

+module LLVM_Extern_Config_Def {}
+module LLVM_Extern_IR_Attributes_Gen {}
+module LLVM_Extern_IR_Intrinsics_Gen {}
+module LLVM_Extern_Utils_DataTypes {}

include/llvm/module.install.modulemap

+
+module LLVM_Extern_Config_Def {
+ textual header "Config/AsmParsers.def"
+ textual header "Config/AsmPrinters.def"
+ textual header "Config/Disassemblers.def"
+ textual header "Config/Targets.def"
+ export *
+}
+
+module LLVM_Extern_IR_Attributes_Gen {
+  textual header "IR/Attributes.gen"
+}
+
+module LLVM_Extern_IR_Intrinsics_Gen {
+  textual header "IR/Intrinsics.gen"
+}
+
+module LLVM_Extern_Utils_DataTypes {
+  header "Support/DataTypes.h"
+  export *
+}

include/llvm/module.modulemap

module Target {
     module * { export * }
 
     // This is intended for (repeated) textual inclusion.
     textual header "Target/TargetOpcodes.def"
   }
 }
 
 module LLVM_Bitcode { requires cplusplus umbrella "Bitcode" module * { export * } }
-module LLVM_Config { requires cplusplus umbrella "Config" module * { export * } }
+module LLVM_Config {
+  requires cplusplus
+  umbrella "Config"
+  extern module LLVM_Extern_Config_Def "module.extern.modulemap"
+  module * { export * }
+}
 
 module LLVM_DebugInfo {
   requires cplusplus
   module DIContext { header "DebugInfo/DIContext.h" export * }
 }
 
 module LLVM_DebugInfo_DWARF {
   requires cplusplus

 module LLVM_intrinsic_gen {
   requires cplusplus
 
   // Delay building the modules containing dependencies to Attributes.h and
   // Intrinsics.h because they need to be generated by tablegen first.
 
   // Attributes.h
   module IR_Argument { header "IR/Argument.h" export * }
-  module IR_Attributes { header "IR/Attributes.h" export * }
+  module IR_Attributes {
+    header "IR/Attributes.h"
+    extern module LLVM_Extern_IR_Attributes_Gen "module.extern.modulemap"
+    export *
+  }
   module IR_CallSite { header "IR/CallSite.h" export * }
   module IR_ConstantFolder { header "IR/ConstantFolder.h" export * }
   module IR_NoFolder { header "IR/NoFolder.h" export * }
   module IR_Module { header "IR/Module.h" export * }
   module IR_ModuleSummaryIndex { header "IR/ModuleSummaryIndex.h" export * }
   module IR_ModuleSummaryIndexYAML { header "IR/ModuleSummaryIndexYAML.h" export * }
   module IR_Function { header "IR/Function.h" export * }
   module IR_InstrTypes { header "IR/InstrTypes.h" export * }
   module IR_Instructions { header "IR/Instructions.h" export * }
 
 
   // Intrinsics.h
   module IR_CFG { header "IR/CFG.h" export * }
   module IR_ConstantRange { header "IR/ConstantRange.h" export * }
   module IR_Dominators { header "IR/Dominators.h" export * }
   module IR_IRBuilder { header "IR/IRBuilder.h" export * }
   module IR_PassManager { header "IR/PassManager.h" export * }
   module IR_PredIteratorCache { header "IR/PredIteratorCache.h" export * }
   module IR_Verifier { header "IR/Verifier.h" export * }
   module IR_InstIterator { header "IR/InstIterator.h" export * }
   module IR_InstVisitor { header "IR/InstVisitor.h" export * }
-  module IR_Intrinsics { header "IR/Intrinsics.h" export * }
+  module IR_Intrinsics {
+    header "IR/Intrinsics.h"
+    extern module LLVM_Extern_IR_Intricsics_Gen "module.extern.modulemap"
+    export *
+  }
   module IR_IntrinsicInst { header "IR/IntrinsicInst.h" export * }
   module IR_PatternMatch { header "IR/PatternMatch.h" export * }
   module IR_Statepoint { header "IR/Statepoint.h" export * }
 
   export *
 }
 
 module LLVM_IR {

 module LLVM_TableGen { requires cplusplus umbrella "TableGen" module * { export * } }
 
 module LLVM_Transforms {
   requires cplusplus
   umbrella "Transforms"
   module * { export * }
 }
 
+extern module LLVM_Extern_Utils_DataTypes "module.extern.modulemap"
+
 // A module covering ADT/ and Support/. These are intertwined and
 // codependent, and notionally form a single module.
 module LLVM_Utils {
   module ADT {
     requires cplusplus
 
     umbrella "ADT"
     module * { export * }