[flang] Add High level Fortran IR dialect

[jeanPerier]

Summary

This patch adds the basic dialect definition of the HLFIR dialect that
was described in https://reviews.llvm.org/D134285.

It adds the definition of the hlfir.expr type and related tests so that
it can be verified that the dialect is properly hooked up by the tools.

Operations will be added as progress is made in the expression
lowering update.

[clementval]

LGTM

[jeanPerier]

Add missing dependencies for shared library builds

[DavidSpickett]

Hi @jeanPerier, one of the flang bots is failing every couple of builds since this change https://lab.llvm.org/buildbot/#/builders/173.

Specific example: https://lab.llvm.org/buildbot/#/builders/173/builds/10304

/home/tcwg-buildbot/worker/flang-aarch64-release/llvm-project/flang/include/flang/Optimizer/HLFIR/HLFIRDialect.h:19:10: fatal error: 'flang/Optimizer/HLFIR/HLFIRDialect.h.inc' file not found
#include "flang/Optimizer/HLFIR/HLFIRDialect.h.inc"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I see you have fixed some of the other builders (thanks!) do you expect this one to be working consistently? Let me know if you need any more details about it. As flang goes, it's one of the simpler build configs.

[jeanPerier]

In D136328#3874697, @DavidSpickett wrote:

Hi @jeanPerier, one of the flang bots is failing every couple of builds since this change https://lab.llvm.org/buildbot/#/builders/173.

Specific example: https://lab.llvm.org/buildbot/#/builders/173/builds/10304

/home/tcwg-buildbot/worker/flang-aarch64-release/llvm-project/flang/include/flang/Optimizer/HLFIR/HLFIRDialect.h:19:10: fatal error: 'flang/Optimizer/HLFIR/HLFIRDialect.h.inc' file not found
#include "flang/Optimizer/HLFIR/HLFIRDialect.h.inc"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I see you have fixed some of the other builders (thanks!) do you expect this one to be working consistently? Let me know if you need any more details about it. As flang goes, it's one of the simpler build configs.

Hi David, thanks for the warning, I missed the mails about these failures. I think I know how to fix this build race condition. I will push a fix ASAP.

[DavidSpickett]

No problem, I think the first build with this commit was green so you wouldn't have seen anything else.

I have moved the bot to silent so you can now find the latest results at https://lab.llvm.org/staging/#/builders/160. So if you're short on time you don't need to rush a fix, it isn't spamming anyone.

[jeanPerier]

In D136328#3874697, @DavidSpickett wrote:

Hi @jeanPerier, one of the flang bots is failing every couple of builds since this change https://lab.llvm.org/buildbot/#/builders/173.

Specific example: https://lab.llvm.org/buildbot/#/builders/173/builds/10304

/home/tcwg-buildbot/worker/flang-aarch64-release/llvm-project/flang/include/flang/Optimizer/HLFIR/HLFIRDialect.h:19:10: fatal error: 'flang/Optimizer/HLFIR/HLFIRDialect.h.inc' file not found
#include "flang/Optimizer/HLFIR/HLFIRDialect.h.inc"
         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I see you have fixed some of the other builders (thanks!) do you expect this one to be working consistently? Let me know if you need any more details about it. As flang goes, it's one of the simpler build configs.

I merged a fix:
https://github.com/llvm/llvm-project/commit/2c0d0566ee2d871c5614b829416bb628ef1892a8

Since it is build race condition, it is hard to reproduce and be sure this fixed all, but from a build logic point of view, I think that was the problem. Thanks again for brining this to my attention.

[DavidSpickett]

Great! We (Linaro that is) will let you know if it comes up again.

flang/include/flang/Optimizer/CMakeLists.txt

 add_subdirectory(CodeGen)
 add_subdirectory(Dialect)
+add_subdirectory(HLFIR)
 add_subdirectory(Transforms)

flang/include/flang/Optimizer/HLFIR/CMakeLists.txt

+set(LLVM_TARGET_DEFINITIONS HLFIROpBase.td)
+mlir_tablegen(HLFIRTypes.h.inc -gen-typedef-decls)
+mlir_tablegen(HLFIRTypes.cpp.inc -gen-typedef-defs)
+mlir_tablegen(HLFIRDialect.h.inc -gen-dialect-decls -dialect=hlfir)
+mlir_tablegen(HLFIRDialect.cpp.inc -gen-dialect-defs -dialect=hlfir)
+mlir_tablegen(HLFIRAttributes.h.inc -gen-attrdef-decls -attrdefs-dialect=hlfir)
+mlir_tablegen(HLFIRAttributes.cpp.inc -gen-attrdef-defs -attrdefs-dialect=hlfir)
+add_public_tablegen_target(HLFIROpsIncGen)

flang/include/flang/Optimizer/HLFIR/HLFIRDialect.h

+//===- HLFIRDialect.h - High Level Fortran IR dialect -----------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file defines the HLFIR dialect that models Fortran expressions and
+// assignments without requiring storage allocation and manipulations.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef FORTRAN_OPTIMIZER_HLFIR_HLFIRDIALECT_H
+#define FORTRAN_OPTIMIZER_HLFIR_HLFIRDIALECT_H
+
+#include "mlir/IR/Dialect.h"
+
+#include "flang/Optimizer/HLFIR/HLFIRDialect.h.inc"
+
+#define GET_TYPEDEF_CLASSES
+#include "flang/Optimizer/HLFIR/HLFIRTypes.h.inc"
+
+#define GET_ATTRDEF_CLASSES
+#include "flang/Optimizer/HLFIR/HLFIRAttributes.h.inc"
+
+#endif // FORTRAN_OPTIMIZER_HLFIR_HLFIRDIALECT_H

flang/include/flang/Optimizer/HLFIR/HLFIROpBase.td

+//===-- HLFIROpBase.td - HLFIR dialect base definitions ----*- tablegen -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// Definition of the HLFIR dialect and core hlfir.expr type
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef FORTRAN_DIALECT_HLFIR_OP_BASE
+#define FORTRAN_DIALECT_HLFIR_OP_BASE
+
+include "mlir/IR/AttrTypeBase.td"
+include "mlir/IR/OpBase.td"
+
+def hlfir_Dialect : Dialect {
+  let name = "hlfir";
+
+  let summary = "High Level Fortran IR.";
+
+  let description = [{
+    This dialect models Fortran expressions and assignments without requiring
+    the allocation and manipulation of temporary storage.
+    It allows running high level optimization passes and is rather
+    straightforward to generate from Fortran expressions and assignments.
+
+    It is not a complete implementation of Fortran, for constructs and lower
+    level operations, FIR should be used directly.
+
+    A bufferization pass transforms hlfir.expr values into FIR temporary in
+    memory, and its translation pass to FIR translates high level operations
+    into sequence of lower level FIR operations operating on memory.
+  }];
+
+  let useDefaultTypePrinterParser = 1;
+  let cppNamespace = "hlfir";
+}
+
+
+def hlfir_ExprType : TypeDef<hlfir_Dialect, "Expr"> {
+  let mnemonic = "expr";
+  let summary = "The type of an array, character, or derived type Fortran expression";
+
+  let description = [{
+    Abstract value type for Fortran arrays, characters and derived types.
+    The rank cannot be assumed, and empty shape means that the expression is a scalar.
+    When the element type is a derived type, the polymorphic flag may be set to true
+    to indicate that the expression dynamic type can differ from its static type.
+  }];
+
+
+  let parameters = (ins
+    ArrayRefParameter<"int64_t", "expression shape">:$shape,
+    "mlir::Type":$elementType,
+    "bool":$polymorphic
+  );
+
+  let extraClassDeclaration = [{
+    using Shape = llvm::SmallVector<int64_t>;
+    mlir::Type getEleTy() const {return getElementType();}
+    bool isArray() const { return !getShape().empty(); }
+    bool isPolymorphic() const { return getPolymorphic(); }
+  }];
+
+  let hasCustomAssemblyFormat = 1;
+
+}
+
+#endif // FORTRAN_DIALECT_HLFIR_OP_BASE

flang/include/flang/Optimizer/Support/InitFIR.h

 //===-- Optimizer/Support/InitFIR.h -----------------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // Coding style: https://mlir.llvm.org/getting_started/DeveloperGuide/
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef FORTRAN_OPTIMIZER_SUPPORT_INITFIR_H
 #define FORTRAN_OPTIMIZER_SUPPORT_INITFIR_H
 
 #include "flang/Optimizer/Dialect/FIRDialect.h"
+#include "flang/Optimizer/HLFIR/HLFIRDialect.h"
 #include "mlir/Conversion/Passes.h"
 #include "mlir/Dialect/Affine/Passes.h"
 #include "mlir/InitAllDialects.h"
 #include "mlir/Pass/Pass.h"
 #include "mlir/Pass/PassRegistry.h"
 #include "mlir/Transforms/LocationSnapshot.h"
 #include "mlir/Transforms/Passes.h"
 
 namespace fir::support {
 
 #define FLANG_NONCODEGEN_DIALECT_LIST                                          \
-  mlir::AffineDialect, FIROpsDialect, mlir::acc::OpenACCDialect,               \
-      mlir::omp::OpenMPDialect, mlir::scf::SCFDialect,                         \
-      mlir::arith::ArithDialect, mlir::cf::ControlFlowDialect,                 \
-      mlir::func::FuncDialect, mlir::vector::VectorDialect,                    \
-      mlir::math::MathDialect
+  mlir::AffineDialect, FIROpsDialect, hlfir::hlfirDialect,                     \
+      mlir::acc::OpenACCDialect, mlir::omp::OpenMPDialect,                     \
+      mlir::scf::SCFDialect, mlir::arith::ArithDialect,                        \
+      mlir::cf::ControlFlowDialect, mlir::func::FuncDialect,                   \
+      mlir::vector::VectorDialect, mlir::math::MathDialect
 
 // The definitive list of dialects used by flang.
 #define FLANG_DIALECT_LIST                                                     \
   FLANG_NONCODEGEN_DIALECT_LIST, FIRCodeGenDialect, mlir::LLVM::LLVMDialect
 
 inline void registerNonCodegenDialects(mlir::DialectRegistry &registry) {
   registry.insert<FLANG_NONCODEGEN_DIALECT_LIST>();
 }

flang/lib/Frontend/CMakeLists.txt

add_flang_library(flangFrontend
   TextDiagnostic.cpp
 
   DEPENDS
   clangBasic
   FIRBuilder
   FIRDialect
   FIRSupport
   FIROptTransformsPassIncGen
+  HLFIRDialect
   MLIRIR
   ${dialect_libs}
 
   LINK_LIBS
   FortranParser
   FortranSemantics
   FortranEvaluate
   FortranCommon
   FortranLower
   clangBasic
   clangDriver
   FIRDialect
   FIRSupport
   FIRBuilder
   FIRCodeGen
   FIRTransforms
+  HLFIRDialect
   MLIRTransforms
   MLIRLLVMToLLVMIRTranslation
   MLIRSCFToControlFlow
   ${dialect_libs}
 
   LINK_COMPONENTS
   Passes
   Analysis
   IRReader
   Option
   Support
   Target
   FrontendOpenACC
   FrontendOpenMP
 )

flang/lib/Optimizer/CMakeLists.txt

 add_subdirectory(Builder)
 add_subdirectory(CodeGen)
 add_subdirectory(Dialect)
+add_subdirectory(HLFIR)
 add_subdirectory(Support)
 add_subdirectory(Transforms)

flang/lib/Optimizer/HLFIR/CMakeLists.txt

+get_property(dialect_libs GLOBAL PROPERTY MLIR_DIALECT_LIBS)
+
+add_flang_library(HLFIRDialect
+  HLFIRDialect.cpp
+
+  DEPENDS
+  FIRDialect
+  HLFIROpsIncGen
+  ${dialect_libs}
+
+  LINK_LIBS
+  FIRDialect
+  MLIRIR
+  ${dialect_libs}
+
+  LINK_COMPONENTS
+  AsmParser
+  AsmPrinter
+  Remarks
+)

flang/lib/Optimizer/HLFIR/HLFIRDialect.cpp

+//===-- HLFIRDialect.cpp --------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Coding style: https://mlir.llvm.org/getting_started/DeveloperGuide/
+//
+//===----------------------------------------------------------------------===//
+
+#include "flang/Optimizer/HLFIR/HLFIRDialect.h"
+#include "mlir/IR/Builders.h"
+#include "mlir/IR/BuiltinTypes.h"
+#include "mlir/IR/DialectImplementation.h"
+#include "mlir/IR/Matchers.h"
+#include "mlir/IR/OpImplementation.h"
+#include "llvm/ADT/TypeSwitch.h"
+
+#include "flang/Optimizer/HLFIR/HLFIRDialect.cpp.inc"
+
+#define GET_TYPEDEF_CLASSES
+#include "flang/Optimizer/HLFIR/HLFIRTypes.cpp.inc"
+
+#define GET_ATTRDEF_CLASSES
+#include "flang/Optimizer/HLFIR/HLFIRAttributes.cpp.inc"
+
+void hlfir::hlfirDialect::initialize() {
+  addTypes<
+#define GET_TYPEDEF_LIST
+#include "flang/Optimizer/HLFIR/HLFIRTypes.cpp.inc"
+      >();
+}
+
+// `expr` `<` `*` | bounds (`x` bounds)* `:` type [`?`] `>`
+// bounds ::= `?` | int-lit
+mlir::Type hlfir::ExprType::parse(mlir::AsmParser &parser) {
+  if (parser.parseLess())
+    return {};
+  ExprType::Shape shape;
+  if (parser.parseOptionalStar()) {
+    if (parser.parseDimensionList(shape, /*allowDynamic=*/true))
+      return {};
+  } else if (parser.parseColon()) {
+    return {};
+  }
+  mlir::Type eleTy;
+  if (parser.parseType(eleTy))
+    return {};
+  const bool polymorphic = mlir::succeeded(parser.parseOptionalQuestion());
+  if (parser.parseGreater())
+    return {};
+  return ExprType::get(parser.getContext(), shape, eleTy, polymorphic);
+}
+
+void hlfir::ExprType::print(mlir::AsmPrinter &printer) const {
+  auto shape = getShape();
+  printer << '<';
+  if (shape.size()) {
+    for (const auto &b : shape) {
+      if (b >= 0)
+        printer << b << 'x';
+      else
+        printer << "?x";
+    }
+  }
+  printer << getEleTy();
+  if (isPolymorphic())
+    printer << '?';
+  printer << '>';
+}

flang/test/HLFIR/expr-type.fir

+// Test the HLFIR Expr type
+// Parse types and check that we can reparse what we print.
+// RUN: fir-opt --split-input-file %s | fir-opt --split-input-file | FileCheck %s
+
+// Scalar expression types
+func.func private @scalar01() -> !hlfir.expr<!fir.char<1,?>>
+func.func private @scalar02() -> !hlfir.expr<!fir.char<2,10>>
+func.func private @scalar03() -> !hlfir.expr<!fir.type<derived{f:f32}>>
+// CHECK-LABEL:  func.func private @scalar01() -> !hlfir.expr<!fir.char<1,?>>
+// CHECK-LABEL:  func.func private @scalar02() -> !hlfir.expr<!fir.char<2,10>>
+// CHECK-LABEL:  func.func private @scalar03() -> !hlfir.expr<!fir.type<derived{f:f32}>>
+
+// Array expression types
+func.func private @array01() -> !hlfir.expr<?x!fir.char<1,?>>
+func.func private @array02() -> !hlfir.expr<10x!fir.char<1,20>>
+func.func private @array03() -> !hlfir.expr<10x?x20x?x30x?x40x?x50x?x60xf128>
+func.func private @array04() -> !hlfir.expr<10x20x!fir.type<derived{f:f32}>>
+func.func private @array05() -> !hlfir.expr<10xi32>
+// CHECK-LABEL:  func.func private @array01() -> !hlfir.expr<?x!fir.char<1,?>>
+// CHECK-LABEL:  func.func private @array02() -> !hlfir.expr<10x!fir.char<1,20>>
+// CHECK-LABEL:  func.func private @array03() -> !hlfir.expr<10x?x20x?x30x?x40x?x50x?x60xf128>
+// CHECK-LABEL:  func.func private @array04() -> !hlfir.expr<10x20x!fir.type<derived{f:f32}>>
+// CHECK-LABEL:  func.func private @array05() -> !hlfir.expr<10xi32>
+
+// Polymorphic expression types
+func.func private @polymorph01() -> !hlfir.expr<!fir.type<derived{f:f32}>?>
+func.func private @polymorph02() -> !hlfir.expr<?x!fir.type<derived{f:f32}>?>
+func.func private @polymorph03() -> !hlfir.expr<10x!fir.type<derived{f:f32}>?>
+// CHECK-LABEL:  func.func private @polymorph01() -> !hlfir.expr<!fir.type<derived{f:f32}>?>
+// CHECK-LABEL:  func.func private @polymorph02() -> !hlfir.expr<?x!fir.type<derived{f:f32}>?>
+// CHECK-LABEL:  func.func private @polymorph03() -> !hlfir.expr<10x!fir.type<derived{f:f32}>?>

flang/tools/bbc/CMakeLists.txt

 
 add_flang_tool(bbc bbc.cpp
 DEPENDS
 FIROptCodeGenPassIncGen
 )
 
 llvm_update_compile_flags(bbc)
 get_property(dialect_libs GLOBAL PROPERTY MLIR_DIALECT_LIBS)
 target_link_libraries(bbc PRIVATE
 FIRDialect
 FIRSupport
 FIRTransforms
 FIRBuilder
+HLFIRDialect
 ${dialect_libs}
 MLIRAffineToStandard
 MLIRSCFToControlFlow
 FortranCommon
 FortranParser
 FortranEvaluate
 FortranSemantics
 FortranLower
 LLVMPasses
 )

flang/tools/fir-opt/CMakeLists.txt

 add_flang_tool(fir-opt fir-opt.cpp)
 llvm_update_compile_flags(fir-opt)
 get_property(dialect_libs GLOBAL PROPERTY MLIR_DIALECT_LIBS)
 
 target_link_libraries(fir-opt PRIVATE
   FIRDialect
   FIRSupport
   FIRTransforms
   FIRCodeGen
+  HLFIRDialect
   ${dialect_libs}
 
   # TODO: these should be transitive dependencies from a target providing
   # "registerFIRPasses()"
   MLIRIR
   MLIRLLVMDialect
   MLIRPass
   MLIRFuncToLLVM

flang/tools/tco/CMakeLists.txt

 add_flang_tool(tco tco.cpp)
 llvm_update_compile_flags(tco)
 get_property(dialect_libs GLOBAL PROPERTY MLIR_DIALECT_LIBS)
 target_link_libraries(tco PRIVATE
   FIRCodeGen
   FIRDialect
   FIRSupport
   FIRTransforms
   FIRBuilder
+  HLFIRDialect
   ${dialect_libs}
   MLIRIR
   MLIRLLVMDialect
   MLIRLLVMToLLVMIRTranslation
   MLIRTargetLLVMIRExport
   MLIRPass
   MLIRFuncToLLVM
   MLIRTransforms
   MLIRAffineToStandard
   MLIRAnalysis
   MLIRSCFToControlFlow
   MLIRParser
   MLIRSupport
   MLIRVectorToLLVM
   LLVMPasses
 )

flang/unittests/Optimizer/CMakeLists.txt

 get_property(dialect_libs GLOBAL PROPERTY MLIR_DIALECT_LIBS)
 
 set(LIBS
   FIRBuilder
   FIRCodeGen
   FIRDialect
   FIRSupport
+  HLFIRDialect
   ${dialect_libs}
 )
 
 add_flang_unittest(FlangOptimizerTests
   Builder/CharacterTest.cpp
   Builder/ComplexTest.cpp
   Builder/DoLoopHelperTest.cpp
   Builder/FIRBuilderTest.cpp