[flang] Lower integer exponentiation into math::IPowI.

[vzakhari]

flang/lib/Optimizer/CodeGen/CodeGen.cpp

3304

I did it this way, because otherwise the pass has to be added into the general pass pipeline(s), and this implies that MathToFuncs dependency has to be added to multiple components (FrontEnd, fir-opt, tco).

[kiranchandramohan]

flang/lib/Optimizer/CodeGen/CodeGen.cpp

3304

These tools share some code in include/flang/Tools/CLOptions.inc. Is it possible to add this pass there?

[vzakhari]

flang/lib/Optimizer/CodeGen/CodeGen.cpp

3304

Yes, this is what I tried, first. Adding the pass there makes all these tools components depend on CMake MathToFuncs component/library.

[jeanPerier]

flang/lib/Optimizer/CodeGen/CodeGen.cpp

3304

Isn't it possible to add the dependency to the FIRSupport library that (from Optimizer/Support) enough ? That is the library that defines registerLLVMTranslation.

[vzakhari]

flang/lib/Optimizer/CodeGen/CodeGen.cpp

3304

Yes, it is possible, but this requires making all dependencies of FIRSupport transitive. This means that any component that links FIRSupport will now link all link-libraries specified for FIRSupport. It may not have huge impact now, though. I will upload the changes to show how it will look like.

Another "inconvenience" is that fir-opt --fir-to-llvm-ir... may fail for FIR with math.ipowi unless --convert-math-to-funcs is passed explicitly. It will complain that math.ipowi cannot be legalized (since the only legal dialect is LLVM after the FIR conversion).

[jeanPerier]

The change inserting the pass in codegen looks good to me.

flang/lib/Optimizer/CodeGen/CodeGen.cpp

3304

Thanks a lot for testing and showing this. I had forgotten that codegen is already in charge of adding the conversion patterns for arith, openMP and other dialects towards LLVM (https://github.com/llvm/llvm-project/blob/367a1fa88992800ee2cc73d3f5a8bc345766cdc8/flang/lib/Optimizer/CodeGen/CodeGen.cpp#L3320-L3329). So your first change makes more sense to me to preserve "fir-opt --fir-to-llvm-ir" behavior.

[kiranchandramohan]

flang/lib/Optimizer/CodeGen/CodeGen.cpp

3304

The other option is to run MathToFunc inside MathToLLVM Conversion. I don't know whether that will be OK. But we can explore that in a separate patch.

[vzakhari]

flang/lib/Optimizer/CodeGen/CodeGen.cpp

3304

Thanks a lot for testing and showing this. I had forgotten that codegen is already in charge of adding the conversion patterns for arith, openMP and other dialects towards LLVM (https://github.com/llvm/llvm-project/blob/367a1fa88992800ee2cc73d3f5a8bc345766cdc8/flang/lib/Optimizer/CodeGen/CodeGen.cpp#L3320-L3329). So your first change makes more sense to me to preserve "fir-opt --fir-to-llvm-ir" behavior.

I also think that satisfying all the output assumptions (i.e. that everything is converted) in FIR-to-LLVM conversion pass is the right thing to do. I will revert to the previous version. Thanks!

3304

The other option is to run MathToFunc inside MathToLLVM Conversion. I don't know whether that will be OK. But we can explore that in a separate patch.

I do not think MathToFuncs can be done inside MathToLLVM conversion, because MathToLLVM is not a Module operation pass, so it is unsafe to modify Module (insert new FuncOps) during MathToLLVM conversion. Also, exposing just the conversion patterns for MathToFuncs and adding them to the patterns set in CodeGen is not safe. We discussed this in D129810: basically, it is not expected that an operation (e.g. IPowI) conversion pattern modifies parent operation(s), so it is not valid to insert new FuncOps in the parent Module. This is why MathToFuncs was implemented as a Module pass that, first, inserts new FuncOps in the parent module and then applies Math operations convert patterns, which only modify the Math operations themselves.

Note that this also means that MathToLibm implementation and our current usage of it should also be redone: it has to be a conversion pass that inserts new FuncOps and then applies Math operations convert patterns; CodeGen has to run MathToLibm the same way I propose running MathToFuncs in this change-set.

I see the following future changes in Math related conversions:

• Support FPowI in MathToFuncs: any FPowI operation will be lowered to a call of an outlined implementation.
• Support FPowI in MathToLLVM: only operations with i32 exponent will be converted to LLVM's PowI. This is quite llvm-backend dependent matter, because llvm backend only supports llvm.powi with i32 exponent: "Generally, the only supported type for the exponent is the one matching with the C type int." As long as MathToLLVM is a partial conversion it should be valid to leave other FPowI's untouched.
• In CodeGen, run:
  • MathToLLVM conversion pass the same way as MathToFuncs.
  • MathToFuncs
  • MathToLibm - the same way as MathToFuncs.

[vzakhari]

Reverted to first version.

[kiranchandramohan]

flang/lib/Optimizer/CodeGen/CodeGen.cpp

3304

I think there is a minor difference,
-> they are all conversion patterns, that should be applied repeatedly.
-> they all convert to LLVM

In the case of MathtoFunc, it is a pass and it converts to Func. So a separate conversion from Func to LLVM is required. I don't know whether this makes much of a difference to be treated separately.

3304

Yes, I recollect that it was moved to a pass since it works on Modules. I was coming more from a user perspective that it might be confusing that -math-to-llvm conversion would leave out conversion of some of the math operations.

[vzakhari]

flang/lib/Optimizer/CodeGen/CodeGen.cpp

3304

Yes, I recollect that it was moved to a pass since it works on Modules. I was coming more from a user perspective that it might be confusing that -math-to-llvm conversion would leave out conversion of some of the math operations.

This is what happens now, because MathToLLVM is a partial conversion and it does not convert all Math operations, so I guess it cannot be more confusing than it is now :)

3304

I think there is a minor difference,
-> they are all conversion patterns, that should be applied repeatedly.
-> they all convert to LLVM

In the case of MathtoFunc, it is a pass and it converts to Func. So a separate conversion from Func to LLVM is required. I don't know whether this makes much of a difference to be treated separately.

Sorry, I am not sure what message you are commenting on.

I believe MathToLibm does not convert to LLVM dialect. It uses Func dialect for the new functions, and Arith dialect for airthmetic, so, yes, further conversion to LLVM is required to satisfy the output requirements for FIR-to-LLVM conversion.

[kiranchandramohan]

flang/lib/Optimizer/CodeGen/CodeGen.cpp

3304

I missed the MathToLibm conversion. All the other ones are conversions to LLVM and they are conversion patterns not passes.

I was basically commenting on @jeanPerier's point that "I had forgotten that codegen is already in charge of adding the conversion patterns for arith, openMP and other dialects towards LLVM (https://github.com/llvm/llvm-project/blob/367a1fa88992800ee2cc73d3f5a8bc345766cdc8/flang/lib/Optimizer/CodeGen/CodeGen.cpp#L3320-L3329)."

[vzakhari]

flang/lib/Optimizer/CodeGen/CodeGen.cpp

3304

Thanks for clarification! And thank you for the final approval!

flang/lib/Lower/IntrinsicCall.cpp

 
 template <int Bits>
 static mlir::FunctionType genF32IntF32FuncType(mlir::MLIRContext *context) {
   auto ftype = mlir::FloatType::getF32(context);
   auto itype = mlir::IntegerType::get(context, Bits);
   return mlir::FunctionType::get(context, {itype, ftype}, {ftype});
 }
 
+template <int Bits>
+static mlir::FunctionType genIntIntIntFuncType(mlir::MLIRContext *context) {
+  auto itype = mlir::IntegerType::get(context, Bits);
+  return mlir::FunctionType::get(context, {itype, itype}, {itype});
+}
+
 /// Callback type for generating lowering for a math operation.
 using MathGeneratorTy = mlir::Value (*)(fir::FirOpBuilder &, mlir::Location,
                                         llvm::StringRef, mlir::FunctionType,
                                         llvm::ArrayRef<mlir::Value>);
 
 struct MathOperation {
   // llvm::StringRef comparison operator are not constexpr, so use string_view.
   using Key = std::string_view;
static mlir::Value genMathOp(fir::FirOpBuilder &builder, mlir::Location loc,
   //           end up transformed into 'strictfp' LLVM attribute (TBD).
   //           Elsewhere, "precise/strict" FP mode should also set
   //           'strictfp' for all user functions and calls so that
   //           LLVM backend does the right job.
   //         * Operations that cannot be reasonably optimized in MLIR
   //           can be also lowered to libm calls for "fast" and "relaxed"
   //           modes.
   mlir::Value result;
-  if (mathRuntimeVersion == preciseVersion) {
+  if (mathRuntimeVersion == preciseVersion &&
+      // Some operations do not have to be lowered as conservative
+      // calls, since they do not affect strict FP behavior.
+      // For example, purely integer operations like exponentiation
+      // with integer operands fall into this class.
+      !mathLibFuncName.empty()) {
     result = genLibCall(builder, loc, mathLibFuncName, mathLibFuncType, args);
   } else {
     LLVM_DEBUG(llvm::dbgs() << "Generating '" << mathLibFuncName
                             << "' operation with type ";
                mathLibFuncType.dump(); llvm::dbgs() << "\n");
     result = builder.create<T>(loc, args);
   }
   LLVM_DEBUG(result.dump(); llvm::dbgs() << "\n");
static constexpr MathOperation mathOperations[] = {
     {"log10", "log10", genF64F64FuncType, genMathOp<mlir::math::Log10Op>},
     {"log_gamma", "lgammaf", genF32F32FuncType, genLibCall},
     {"log_gamma", "lgamma", genF64F64FuncType, genLibCall},
     // llvm.lround behaves the same way as libm's lround.
     {"nint", "llvm.lround.i64.f64", genIntF64FuncType<64>, genLibCall},
     {"nint", "llvm.lround.i64.f32", genIntF32FuncType<64>, genLibCall},
     {"nint", "llvm.lround.i32.f64", genIntF64FuncType<32>, genLibCall},
     {"nint", "llvm.lround.i32.f32", genIntF32FuncType<32>, genLibCall},
+    {"pow", {}, genIntIntIntFuncType<8>, genMathOp<mlir::math::IPowIOp>},
+    {"pow", {}, genIntIntIntFuncType<16>, genMathOp<mlir::math::IPowIOp>},
+    {"pow", {}, genIntIntIntFuncType<32>, genMathOp<mlir::math::IPowIOp>},
+    {"pow", {}, genIntIntIntFuncType<64>, genMathOp<mlir::math::IPowIOp>},
     {"pow", "powf", genF32F32F32FuncType, genMathOp<mlir::math::PowFOp>},
     {"pow", "pow", genF64F64F64FuncType, genMathOp<mlir::math::PowFOp>},
     // TODO: add PowIOp in math and complex dialects.
     {"pow", "llvm.powi.f32.i32", genF32F32IntFuncType<32>, genLibCall},
     {"pow", "llvm.powi.f64.i32", genF64F64IntFuncType<32>, genLibCall},
     {"sign", "copysignf", genF32F32F32FuncType,
      genMathOp<mlir::math::CopySignOp>},
     {"sign", "copysign", genF64F64F64FuncType,

flang/lib/Optimizer/CodeGen/CMakeLists.txt

add_flang_library(FIRCodeGen
   FIRSupport
   FIROptCodeGenPassIncGen
   CGOpsIncGen
 
   LINK_LIBS
   FIRBuilder
   FIRDialect
   FIRSupport
+  MLIRMathToFuncs
   MLIRMathToLLVM
   MLIRMathToLibm
   MLIROpenMPToLLVM
   MLIRLLVMToLLVMIRTranslation
   MLIRTargetLLVMIRExport
 
   LINK_COMPONENTS
   AsmParser
   AsmPrinter
   Remarks
 )

flang/lib/Optimizer/CodeGen/CodeGen.cpp

 #include "flang/Optimizer/Dialect/FIROps.h"
 #include "flang/Optimizer/Support/InternalNames.h"
 #include "flang/Optimizer/Support/TypeCode.h"
 #include "flang/Semantics/runtime-type-info.h"
 #include "mlir/Conversion/ArithmeticToLLVM/ArithmeticToLLVM.h"
 #include "mlir/Conversion/ControlFlowToLLVM/ControlFlowToLLVM.h"
 #include "mlir/Conversion/FuncToLLVM/ConvertFuncToLLVM.h"
 #include "mlir/Conversion/LLVMCommon/Pattern.h"
+#include "mlir/Conversion/MathToFuncs/MathToFuncs.h"
 #include "mlir/Conversion/MathToLLVM/MathToLLVM.h"
 #include "mlir/Conversion/MathToLibm/MathToLibm.h"
 #include "mlir/Conversion/OpenMPToLLVM/ConvertOpenMPToLLVM.h"
 #include "mlir/IR/BuiltinTypes.h"
 #include "mlir/IR/Matchers.h"
 #include "mlir/Pass/Pass.h"
+#include "mlir/Pass/PassManager.h"
 #include "mlir/Target/LLVMIR/ModuleTranslation.h"
 #include "llvm/ADT/ArrayRef.h"
 
 #define DEBUG_TYPE "flang-codegen"
 
 // fir::LLVMTypeConverter for converting to LLVM IR dialect types.
 #include "TypeConverter.h"
 
public:
   FIRToLLVMLowering(fir::FIRToLLVMPassOptions options) : options{options} {}
   mlir::ModuleOp getModule() { return getOperation(); }
 
   void runOnOperation() override final {
     auto mod = getModule();
     if (!forcedTargetTriple.empty())
       fir::setTargetTriple(mod, forcedTargetTriple);
 
+    // Run dynamic pass pipeline for converting Math dialect
+    // operations into other dialects (llvm, func, etc.).
+    // Some conversions of Math operations cannot be done
+    // by just using conversion patterns. This is true for
+    // conversions that affect the ModuleOp, e.g. create new
+    // function operations in it. We have to run such conversions
+    // as passes here.
+    mlir::OpPassManager mathConvertionPM("builtin.module");
+    mathConvertionPM.addPass(mlir::createConvertMathToFuncsPass());
+    if (mlir::failed(runPipeline(mathConvertionPM, mod)))
+      return signalPassFailure();
+
     auto *context = getModule().getContext();
     fir::LLVMTypeConverter typeConverter{getModule()};
     mlir::RewritePatternSet pattern(context);
     pattern.insert<
         AbsentOpConversion, AddcOpConversion, AddrOfOpConversion,
         AllocaOpConversion, AllocMemOpConversion, BoxAddrOpConversion,
         BoxCharLenOpConversion, BoxDimsOpConversion, BoxEleSizeOpConversion,
         BoxIsAllocOpConversion, BoxIsArrayOpConversion, BoxIsPtrOpConversion,

flang/test/Intrinsics/math-codegen.fir

func.func @_QPtest_real8(%arg0: !fir.ref<f64> {fir.bindc_name = "x"}, %arg1: !fir.ref<f64> {fir.bindc_name = "y"}, %arg2: !fir.ref<i16> {fir.bindc_name = "s"}, %arg3: !fir.ref<i32> {fir.bindc_name = "i"}) -> f64 {
   %13 = fir.load %0 : !fir.ref<f64>
   return %13 : f64
 }
 func.func private @llvm.powi.f32.i32(f32, i32) -> f32
 func.func private @powf(f32, f32) -> f32
 func.func private @llvm.powi.f64.i32(f64, i32) -> f64
 func.func private @pow(f64, f64) -> f64
 
+//--- exponentiation_integer.fir
+// RUN: fir-opt %t/exponentiation_integer.fir --fir-to-llvm-ir="target=x86_64-unknown-linux-gnu" | FileCheck %t/exponentiation_integer.fir
+// CHECK: @_QPtest_int4
+// CHECK: llvm.call @__mlir_math_ipowi_i32({{%[A-Za-z0-9._]+}}, {{%[A-Za-z0-9._]+}}) : (i32, i32) -> i32
+
+func.func @_QPtest_int4(%arg0: !fir.ref<i32> {fir.bindc_name = "x"}, %arg1: !fir.ref<i32> {fir.bindc_name = "y"}, %arg2: !fir.ref<i32> {fir.bindc_name = "z"}) {
+  %0 = fir.load %arg0 : !fir.ref<i32>
+  %1 = fir.load %arg1 : !fir.ref<i32>
+  %2 = math.ipowi %0, %1 : i32
+  fir.store %2 to %arg2 : !fir.ref<i32>
+  return
+}
+
 //--- sign_fast.fir
 // RUN: fir-opt %t/sign_fast.fir --fir-to-llvm-ir="target=x86_64-unknown-linux-gnu" | FileCheck %t/sign_fast.fir
 // CHECK: @_QPtest_real4
 // CHECK: {{%[A-Za-z0-9._]+}} = "llvm.intr.copysign"({{%[A-Za-z0-9._]+}}, {{%[A-Za-z0-9._]+}}) : (f32, f32) -> f32
 
 // CHECK: @_QPtest_real8
 // CHECK: {{%[A-Za-z0-9._]+}} = "llvm.intr.copysign"({{%[A-Za-z0-9._]+}}, {{%[A-Za-z0-9._]+}}) : (f64, f64) -> f64
 

flang/test/Lower/power-operator.f90

 subroutine pow_r4_r8(x, y, z)
   real(4) :: x
   real(8) :: z, y
   z = x ** y
   ! CHECK: %{{.*}} = fir.convert %{{.*}} : (f32) -> f64
   ! CHECK: math.powf %{{.*}}, %{{.*}} : f64
 end subroutine
 
+! CHECK-LABEL: pow_i1_i1
+subroutine pow_i1_i1(x, y, z)
+  integer(1) :: x, y, z
+  z = x ** y
+  ! CHECK: math.ipowi %{{.*}}, %{{.*}} : i8
+end subroutine
+
+! CHECK-LABEL: pow_i2_i2
+subroutine pow_i2_i2(x, y, z)
+  integer(2) :: x, y, z
+  z = x ** y
+  ! CHECK: math.ipowi %{{.*}}, %{{.*}} : i16
+end subroutine
+
 ! CHECK-LABEL: pow_i4_i4
 subroutine pow_i4_i4(x, y, z)
   integer(4) :: x, y, z
   z = x ** y
-  ! CHECK: call @__mth_i_ipowi
+  ! CHECK: math.ipowi %{{.*}}, %{{.*}} : i32
 end subroutine
 
 ! CHECK-LABEL: pow_i8_i8
 subroutine pow_i8_i8(x, y, z)
   integer(8) :: x, y, z
   z = x ** y
-  ! CHECK: call @__mth_i_kpowk
+  ! CHECK: math.ipowi %{{.*}}, %{{.*}} : i64
 end subroutine
 
 ! CHECK-LABEL: pow_c4_i4
 subroutine pow_c4_i4(x, y, z)
   complex :: x, z
   integer :: y
   z = x ** y
   ! CHECK: call @__fc_powi_1