[flang] Restore stack after allocas created by TargetRewrite.

[vzakhari]

Summary

This resolves issues with running out of stack on examples like
https://fortran-lang.discourse.group/t/modern-fortran-sample-code/2019/18
reported by @clementval.

When target rewrite creates alloca(s) around a call, we need to insert
stacksave/stackrestore to free the allocated stack. Better performant
code may be achieved by placing the alloca(s) outside of loops,
but the placement has to behave correctly with regards to OpenMP/OpenACC/etc.
dialect operations that have special representation for "private"
objects. This is a concervative fix for correctness issue.

[clementval]

LGTM

flang/include/flang/Optimizer/CodeGen/CGPasses.td

56

I think you only to add the dialect as a dependency if you generate op from this dialect. I might be wrong.

[vzakhari]

Thank you for the review, Valentin!

flang/include/flang/Optimizer/CodeGen/CGPasses.td

56

You are totally right. The fir::factory::getLlvmStackSave/Restore calls below may generate FuncOps.

[jeanPerier]

This looks like a good solution to me.

but the placement has to behave correctly with regards to OpenMP/OpenACC/etc.
dialect operations that have special representation for "private"
objects.

FYI, fir::FirOpBuilder::createTemporary can be used to create fir.alloca that are hoisted as far as allowed by OpenMP using the "mlir::omp::OutlineableOpenMPOpInterface" (@clementval BTW, I think something similar may be needed for OpenACC regions, because createTemporary is massively used in lowering. See the getAllocaBlock() logic: https://github.com/llvm/llvm-project/blob/faa1043144b3f6c3362fe1d9f43e55e741b40386/flang/lib/Optimizer/Builder/FIRBuilder.cpp#L209

createTemporary is however not a generic solution either, because alloca can only be hoisted if their size does not depend on SSA variables local to the loop (which is the case here, but is not true in general).

So, I feel we may hit this issue here and there, and that we will probably want a generic solution at the do loop construct level (and implicit loop from array expressions) to cleanup loop iteration alloca if this becomes a burden.

But in the meantime, the fix here seems just fine to me, thanks!

[clementval]

In D149222#4298118, @jeanPerier wrote:

FYI, fir::FirOpBuilder::createTemporary can be used to create fir.alloca that are hoisted as far as allowed by OpenMP using the "mlir::omp::OutlineableOpenMPOpInterface" (@clementval BTW, I think something similar may be needed for OpenACC regions, because createTemporary is massively used in lowering. See the getAllocaBlock() logic: https://github.com/llvm/llvm-project/blob/faa1043144b3f6c3362fe1d9f43e55e741b40386/flang/lib/Optimizer/Builder/FIRBuilder.cpp#L209

Yeah we will probably need it but since we did not design how we will handle privatization in acc dialect we did not need it until now.

[vzakhari]

Thank you for pointing out fir::FirOpBuilder::createTemporary, Jean! I did not know that it was aware of the OpenMP operations.
To use it generically I suppose we will have to accompany it with some sort of "finalizeTemporary", which we will deal with inserting the stackrestore call in cases where the hoisting was not able to place the alloca outside of the scope where the alloca may be repeatedly executed (e.g. due to non-constant size).
I guess another option is to have a "fixup" pass that will detect alloca that can be repeatedly executed without automatic deallocation (such as in OpenMP/OpenACC operation scope) and will insert stacksave/stackrestore around it based on its lifetime. This may work assuming that conformant user programs cannot produce repeatable alloca, e.g. to deliberately overflow the stack.

flang/include/flang/Optimizer/CodeGen/CGPasses.td

 
 def TargetRewritePass : Pass<"target-rewrite", "mlir::ModuleOp"> {
   let summary = "Rewrite some FIR dialect into target specific forms.";
   let description = [{
       Certain abstractions in the FIR dialect need to be rewritten to reflect
       representations that may differ based on the target machine.
   }];
   let constructor = "::fir::createFirTargetRewritePass()";
-  let dependentDialects = [ "fir::FIROpsDialect" ];
+  let dependentDialects = [ "fir::FIROpsDialect", "mlir::func::FuncDialect" ];
   let options = [
     Option<"forcedTargetTriple", "target", "std::string", /*default=*/"",
            "Override module's target triple.">,
     Option<"noCharacterConversion", "no-character-conversion",
            "bool", /*default=*/"false",
            "Disable target-specific conversion of CHARACTER.">,
     Option<"noComplexConversion", "no-complex-conversion",
            "bool", /*default=*/"false",

flang/lib/Optimizer/CodeGen/TargetRewrite.cpp

void runOnOperation() override final {
     auto mod = getModule();
     if (!forcedTargetTriple.empty())
       fir::setTargetTriple(mod, forcedTargetTriple);
 
     auto specifics = fir::CodeGenSpecifics::get(
         mod.getContext(), fir::getTargetTriple(mod), fir::getKindMapping(mod));
     setMembers(specifics.get(), &rewriter);
 
+    // We may need to call stacksave/stackrestore later, so
+    // create the FuncOps beforehand.
+    fir::FirOpBuilder builder(rewriter, fir::getKindMapping(mod));
+    builder.setInsertionPointToStart(mod.getBody());
+    stackSaveFn = fir::factory::getLlvmStackSave(builder);
+    stackRestoreFn = fir::factory::getLlvmStackRestore(builder);
+
     // Perform type conversion on signatures and call sites.
     if (mlir::failed(convertTypes(mod))) {
       mlir::emitError(mlir::UnknownLoc::get(&context),
                       "error in converting types to target abi");
       signalPassFailure();
     }
 
     // Convert ops in target-specific patterns.
void runOnOperation() override final {
     clearMembers();
   }
 
   mlir::ModuleOp getModule() { return getOperation(); }
 
   template <typename A, typename B, typename C>
   std::optional<std::function<mlir::Value(mlir::Operation *)>>
   rewriteCallComplexResultType(mlir::Location loc, A ty, B &newResTys,
-                               B &newInTys, C &newOpers) {
+                               B &newInTys, C &newOpers,
+                               mlir::Value &savedStackPtr) {
     if (noComplexConversion) {
       newResTys.push_back(ty);
       return std::nullopt;
     }
     auto m = specifics->complexReturnType(loc, ty.getElementType());
     // Currently targets mandate COMPLEX is a single aggregate or packed
     // scalar, including the sret case.
     assert(m.size() == 1 && "target of complex return not supported");
     auto resTy = std::get<mlir::Type>(m[0]);
     auto attr = std::get<fir::CodeGenSpecifics::Attributes>(m[0]);
     if (attr.isSRet()) {
       assert(fir::isa_ref_type(resTy) && "must be a memory reference type");
+      // Save the stack pointer, if it has not been saved for this call yet.
+      // We will need to restore it after the call, because the alloca
+      // needs to be deallocated.
+      if (!savedStackPtr)
+        savedStackPtr = genStackSave(loc);
       mlir::Value stack =
           rewriter->create<fir::AllocaOp>(loc, fir::dyn_cast_ptrEleTy(resTy));
       newInTys.push_back(resTy);
       newOpers.push_back(stack);
       return [=](mlir::Operation *) -> mlir::Value {
         auto memTy = fir::ReferenceType::get(ty);
         auto cast = rewriter->create<fir::ConvertOp>(loc, memTy, stack);
         return rewriter->create<fir::LoadOp>(loc, cast);
       };
     }
     newResTys.push_back(resTy);
-    return [=](mlir::Operation *call) -> mlir::Value {
+    return [=, &savedStackPtr](mlir::Operation *call) -> mlir::Value {
+      // We are going to generate an alloca, so save the stack pointer.
+      if (!savedStackPtr)
+        savedStackPtr = genStackSave(loc);
       auto mem = rewriter->create<fir::AllocaOp>(loc, resTy);
       rewriter->create<fir::StoreOp>(loc, call->getResult(0), mem);
       auto memTy = fir::ReferenceType::get(ty);
       auto cast = rewriter->create<fir::ConvertOp>(loc, memTy, mem);
       return rewriter->create<fir::LoadOp>(loc, cast);
     };
   }
 
   template <typename A, typename B, typename C>
   void rewriteCallComplexInputType(A ty, mlir::Value oper, B &newInTys,
-                                   C &newOpers) {
+                                   C &newOpers, mlir::Value &savedStackPtr) {
     if (noComplexConversion) {
       newInTys.push_back(ty);
       newOpers.push_back(oper);
       return;
     }
 
     auto *ctx = ty.getContext();
     mlir::Location loc = mlir::UnknownLoc::get(ctx);
     if (auto *op = oper.getDefiningOp())
       loc = op->getLoc();
     auto m = specifics->complexArgumentType(loc, ty.getElementType());
     if (m.size() == 1) {
       // COMPLEX is a single aggregate
       auto resTy = std::get<mlir::Type>(m[0]);
       auto attr = std::get<fir::CodeGenSpecifics::Attributes>(m[0]);
       auto oldRefTy = fir::ReferenceType::get(ty);
+      // We are going to generate an alloca, so save the stack pointer.
+      if (!savedStackPtr)
+        savedStackPtr = genStackSave(loc);
       if (attr.isByVal()) {
         auto mem = rewriter->create<fir::AllocaOp>(loc, ty);
         rewriter->create<fir::StoreOp>(loc, oper, mem);
         newOpers.push_back(rewriter->create<fir::ConvertOp>(loc, resTy, mem));
       } else {
         auto mem = rewriter->create<fir::AllocaOp>(loc, resTy);
         auto cast = rewriter->create<fir::ConvertOp>(loc, oldRefTy, mem);
         rewriter->create<fir::StoreOp>(loc, oper, cast);
public:
   template <typename A>
   void convertCallOp(A callOp) {
     auto fnTy = callOp.getFunctionType();
     auto loc = callOp.getLoc();
     rewriter->setInsertionPoint(callOp);
     llvm::SmallVector<mlir::Type> newResTys;
     llvm::SmallVector<mlir::Type> newInTys;
     llvm::SmallVector<mlir::Value> newOpers;
+    mlir::Value savedStackPtr = nullptr;
 
     // If the call is indirect, the first argument must still be the function
     // to call.
     int dropFront = 0;
     if constexpr (std::is_same_v<std::decay_t<A>, fir::CallOp>) {
       if (!callOp.getCallee()) {
         newInTys.push_back(fnTy.getInput(0));
         newOpers.push_back(callOp.getOperand(0));
         dropFront = 1;
       }
     } else {
       dropFront = 1; // First operand is the polymorphic object.
     }
 
     // Determine the rewrite function, `wrap`, for the result value.
     std::optional<std::function<mlir::Value(mlir::Operation *)>> wrap;
     if (fnTy.getResults().size() == 1) {
       mlir::Type ty = fnTy.getResult(0);
       llvm::TypeSwitch<mlir::Type>(ty)
           .template Case<fir::ComplexType>([&](fir::ComplexType cmplx) {
             wrap = rewriteCallComplexResultType(loc, cmplx, newResTys, newInTys,
-                                                newOpers);
+                                                newOpers, savedStackPtr);
           })
           .template Case<mlir::ComplexType>([&](mlir::ComplexType cmplx) {
             wrap = rewriteCallComplexResultType(loc, cmplx, newResTys, newInTys,
-                                                newOpers);
+                                                newOpers, savedStackPtr);
           })
           .Default([&](mlir::Type ty) { newResTys.push_back(ty); });
     } else if (fnTy.getResults().size() > 1) {
       TODO(loc, "multiple results not supported yet");
     }
 
     llvm::SmallVector<mlir::Type> trailingInTys;
     llvm::SmallVector<mlir::Value> trailingOpers;
for (auto e : llvm::enumerate(
                 trailingOpers.push_back(unbox.getResult(idx));
               } else {
                 newInTys.push_back(argTy);
                 newOpers.push_back(unbox.getResult(idx));
               }
             }
           })
           .template Case<fir::ComplexType>([&](fir::ComplexType cmplx) {
-            rewriteCallComplexInputType(cmplx, oper, newInTys, newOpers);
+            rewriteCallComplexInputType(cmplx, oper, newInTys, newOpers,
+                                        savedStackPtr);
           })
           .template Case<mlir::ComplexType>([&](mlir::ComplexType cmplx) {
-            rewriteCallComplexInputType(cmplx, oper, newInTys, newOpers);
+            rewriteCallComplexInputType(cmplx, oper, newInTys, newOpers,
+                                        savedStackPtr);
           })
           .template Case<mlir::TupleType>([&](mlir::TupleType tuple) {
             if (fir::isCharacterProcedureTuple(tuple)) {
               mlir::ModuleOp module = getModule();
               if constexpr (std::is_same_v<std::decay_t<A>, fir::CallOp>) {
                 if (callOp.getCallee()) {
                   llvm::StringRef charProcAttr =
                       fir::getCharacterProcedureDummyAttrName();
for (auto e : llvm::enumerate(
                 passArgShift = newOpers.size() - *callOp.getPassArgPos();
             }
             newInTys.push_back(ty);
             newOpers.push_back(oper);
           });
     }
     newInTys.insert(newInTys.end(), trailingInTys.begin(), trailingInTys.end());
     newOpers.insert(newOpers.end(), trailingOpers.begin(), trailingOpers.end());
+
+    llvm::SmallVector<mlir::Value, 1> newCallResults;
     if constexpr (std::is_same_v<std::decay_t<A>, fir::CallOp>) {
       fir::CallOp newCall;
       if (callOp.getCallee()) {
         newCall =
             rewriter->create<A>(loc, *callOp.getCallee(), newResTys, newOpers);
       } else {
         // Force new type on the input operand.
         newOpers[0].setType(mlir::FunctionType::get(
             callOp.getContext(),
             mlir::TypeRange{newInTys}.drop_front(dropFront), newResTys));
         newCall = rewriter->create<A>(loc, newResTys, newOpers);
       }
       LLVM_DEBUG(llvm::dbgs() << "replacing call with " << newCall << '\n');
       if (wrap)
-        replaceOp(callOp, (*wrap)(newCall.getOperation()));
+        newCallResults.push_back((*wrap)(newCall.getOperation()));
       else
-        replaceOp(callOp, newCall.getResults());
+        newCallResults.append(newCall.result_begin(), newCall.result_end());
     } else {
       fir::DispatchOp dispatchOp = rewriter->create<A>(
           loc, newResTys, rewriter->getStringAttr(callOp.getMethod()),
           callOp.getOperands()[0], newOpers,
           rewriter->getI32IntegerAttr(*callOp.getPassArgPos() + passArgShift));
       if (wrap)
-        replaceOp(callOp, (*wrap)(dispatchOp.getOperation()));
+        newCallResults.push_back((*wrap)(dispatchOp.getOperation()));
       else
-        replaceOp(callOp, dispatchOp.getResults());
+        newCallResults.append(dispatchOp.result_begin(),
+                              dispatchOp.result_end());
+    }
+
+    if (newCallResults.size() <= 1) {
+      if (savedStackPtr) {
+        if (newCallResults.size() == 1) {
+          // We assume that all the allocas are inserted before
+          // the operation that defines the new call result.
+          rewriter->setInsertionPointAfterValue(newCallResults[0]);
+        } else {
+          // If the call does not have results, then insert
+          // stack restore after the original call operation.
+          rewriter->setInsertionPointAfter(callOp);
+        }
+        genStackRestore(loc, savedStackPtr);
+      }
+      replaceOp(callOp, newCallResults);
+    } else {
+      // The TODO is duplicated here to make sure this part
+      // handles the stackrestore insertion properly, if
+      // we add support for multiple call results.
+      TODO(loc, "multiple results not supported yet");
     }
   }
 
   // Result type fixup for fir::ComplexType and mlir::ComplexType
   template <typename A, typename B>
   void lowerComplexSignatureRes(mlir::Location loc, A cmplx, B &newResTys,
                                 B &newInTys) {
     if (noComplexConversion) {
private:
 
   inline void setMembers(fir::CodeGenSpecifics *s, mlir::OpBuilder *r) {
     specifics = s;
     rewriter = r;
   }
 
   inline void clearMembers() { setMembers(nullptr, nullptr); }
 
+  // Inserts a call to llvm.stacksave at the current insertion
+  // point and the given location. Returns the call's result Value.
+  inline mlir::Value genStackSave(mlir::Location loc) {
+    return rewriter->create<fir::CallOp>(loc, stackSaveFn).getResult(0);
+  }
+
+  // Inserts a call to llvm.stackrestore at the current insertion
+  // point and the given location and argument.
+  inline void genStackRestore(mlir::Location loc, mlir::Value sp) {
+    rewriter->create<fir::CallOp>(loc, stackRestoreFn, mlir::ValueRange{sp});
+  }
+
   fir::CodeGenSpecifics *specifics = nullptr;
   mlir::OpBuilder *rewriter = nullptr;
+  mlir::func::FuncOp stackSaveFn = nullptr;
+  mlir::func::FuncOp stackRestoreFn = nullptr;
 }; // namespace
 } // namespace
 
 std::unique_ptr<mlir::OperationPass<mlir::ModuleOp>>
 fir::createFirTargetRewritePass(const fir::TargetRewriteOptions &options) {
   return std::make_unique<TargetRewrite>(options);
 }

flang/test/Fir/target-rewrite-complex16.fir

 // CHECK:           %[[VAL_8:.*]] = fir.insert_value %[[VAL_5]], %[[VAL_7]], [1 : i32] : (!fir.complex<16>, f128) -> !fir.complex<16>
 // CHECK:           %[[VAL_9:.*]] = fir.convert %[[VAL_0]] : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> !fir.ref<!fir.complex<16>>
 // CHECK:           fir.store %[[VAL_8]] to %[[VAL_9]] : !fir.ref<!fir.complex<16>>
 // CHECK:           return
 // CHECK:         }
 // CHECK:         func.func private @paramcomplex16(!fir.ref<tuple<!fir.real<16>, !fir.real<16>>> {llvm.align = 16 : i32, llvm.byval = tuple<!fir.real<16>, !fir.real<16>>})
 
 // CHECK-LABEL:   func.func @callcomplex16() {
-// CHECK:           %[[VAL_0:.*]] = fir.alloca tuple<!fir.real<16>, !fir.real<16>>
-// CHECK:           fir.call @returncomplex16(%[[VAL_0]]) : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> ()
-// CHECK:           %[[VAL_1:.*]] = fir.convert %[[VAL_0]] : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> !fir.ref<!fir.complex<16>>
-// CHECK:           %[[VAL_2:.*]] = fir.load %[[VAL_1]] : !fir.ref<!fir.complex<16>>
-// CHECK:           %[[VAL_3:.*]] = fir.alloca !fir.complex<16>
-// CHECK:           fir.store %[[VAL_2]] to %[[VAL_3]] : !fir.ref<!fir.complex<16>>
-// CHECK:           %[[VAL_4:.*]] = fir.convert %[[VAL_3]] : (!fir.ref<!fir.complex<16>>) -> !fir.ref<tuple<!fir.real<16>, !fir.real<16>>>
-// CHECK:           fir.call @paramcomplex16(%[[VAL_4]]) : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> ()
+// CHECK:           %[[VAL_0:.*]] = fir.call @llvm.stacksave() : () -> !fir.ref<i8>
+// CHECK:           %[[VAL_1:.*]] = fir.alloca tuple<!fir.real<16>, !fir.real<16>>
+// CHECK:           fir.call @returncomplex16(%[[VAL_1]]) : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> ()
+// CHECK:           %[[VAL_2:.*]] = fir.convert %[[VAL_1]] : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> !fir.ref<!fir.complex<16>>
+// CHECK:           %[[VAL_3:.*]] = fir.load %[[VAL_2]] : !fir.ref<!fir.complex<16>>
+// CHECK:           fir.call @llvm.stackrestore(%[[VAL_0]]) : (!fir.ref<i8>) -> ()
+// CHECK:           %[[VAL_4:.*]] = fir.call @llvm.stacksave() : () -> !fir.ref<i8>
+// CHECK:           %[[VAL_5:.*]] = fir.alloca !fir.complex<16>
+// CHECK:           fir.store %[[VAL_3]] to %[[VAL_5]] : !fir.ref<!fir.complex<16>>
+// CHECK:           %[[VAL_6:.*]] = fir.convert %[[VAL_5]] : (!fir.ref<!fir.complex<16>>) -> !fir.ref<tuple<!fir.real<16>, !fir.real<16>>>
+// CHECK:           fir.call @paramcomplex16(%[[VAL_6]]) : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> ()
+// CHECK:           fir.call @llvm.stackrestore(%[[VAL_4]]) : (!fir.ref<i8>) -> ()
 // CHECK:           return
 // CHECK:         }
 // CHECK:         func.func private @calleemultipleparamscomplex16(!fir.ref<tuple<!fir.real<16>, !fir.real<16>>> {llvm.align = 16 : i32, llvm.byval = tuple<!fir.real<16>, !fir.real<16>>}, !fir.ref<tuple<!fir.real<16>, !fir.real<16>>> {llvm.align = 16 : i32, llvm.byval = tuple<!fir.real<16>, !fir.real<16>>}, !fir.ref<tuple<!fir.real<16>, !fir.real<16>>> {llvm.align = 16 : i32, llvm.byval = tuple<!fir.real<16>, !fir.real<16>>})
 
 // CHECK-LABEL:   func.func @multipleparamscomplex16(
-// CHECK-SAME:       %[[VAL_0:.*]]: !fir.ref<tuple<!fir.real<16>, !fir.real<16>>> {llvm.align = 16 : i32, llvm.byval = tuple<!fir.real<16>, !fir.real<16>>}, %[[VAL_1:.*]]: !fir.ref<tuple<!fir.real<16>, !fir.real<16>>> {llvm.align = 16 : i32, llvm.byval = tuple<!fir.real<16>, !fir.real<16>>}, %[[VAL_2:.*]]: !fir.ref<tuple<!fir.real<16>, !fir.real<16>>> {llvm.align = 16 : i32, llvm.byval = tuple<!fir.real<16>, !fir.real<16>>}) {
+// CHECK-SAME:      %[[VAL_0:.*]]: !fir.ref<tuple<!fir.real<16>, !fir.real<16>>> {llvm.align = 16 : i32, llvm.byval = tuple<!fir.real<16>, !fir.real<16>>}, %[[VAL_1:.*]]: !fir.ref<tuple<!fir.real<16>, !fir.real<16>>> {llvm.align = 16 : i32, llvm.byval = tuple<!fir.real<16>, !fir.real<16>>}, %[[VAL_2:.*]]: !fir.ref<tuple<!fir.real<16>, !fir.real<16>>> {llvm.align = 16 : i32, llvm.byval = tuple<!fir.real<16>, !fir.real<16>>}) {
 // CHECK:           %[[VAL_3:.*]] = fir.convert %[[VAL_2]] : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> !fir.ref<!fir.complex<16>>
 // CHECK:           %[[VAL_4:.*]] = fir.load %[[VAL_3]] : !fir.ref<!fir.complex<16>>
 // CHECK:           %[[VAL_5:.*]] = fir.convert %[[VAL_1]] : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> !fir.ref<!fir.complex<16>>
 // CHECK:           %[[VAL_6:.*]] = fir.load %[[VAL_5]] : !fir.ref<!fir.complex<16>>
 // CHECK:           %[[VAL_7:.*]] = fir.convert %[[VAL_0]] : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> !fir.ref<!fir.complex<16>>
 // CHECK:           %[[VAL_8:.*]] = fir.load %[[VAL_7]] : !fir.ref<!fir.complex<16>>
-// CHECK:           %[[VAL_9:.*]] = fir.alloca !fir.complex<16>
-// CHECK:           fir.store %[[VAL_8]] to %[[VAL_9]] : !fir.ref<!fir.complex<16>>
-// CHECK:           %[[VAL_10:.*]] = fir.convert %[[VAL_9]] : (!fir.ref<!fir.complex<16>>) -> !fir.ref<tuple<!fir.real<16>, !fir.real<16>>>
-// CHECK:           %[[VAL_11:.*]] = fir.alloca !fir.complex<16>
-// CHECK:           fir.store %[[VAL_6]] to %[[VAL_11]] : !fir.ref<!fir.complex<16>>
-// CHECK:           %[[VAL_12:.*]] = fir.convert %[[VAL_11]] : (!fir.ref<!fir.complex<16>>) -> !fir.ref<tuple<!fir.real<16>, !fir.real<16>>>
-// CHECK:           %[[VAL_13:.*]] = fir.alloca !fir.complex<16>
-// CHECK:           fir.store %[[VAL_4]] to %[[VAL_13]] : !fir.ref<!fir.complex<16>>
-// CHECK:           %[[VAL_14:.*]] = fir.convert %[[VAL_13]] : (!fir.ref<!fir.complex<16>>) -> !fir.ref<tuple<!fir.real<16>, !fir.real<16>>>
-// CHECK:           fir.call @calleemultipleparamscomplex16(%[[VAL_10]], %[[VAL_12]], %[[VAL_14]]) : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>, !fir.ref<tuple<!fir.real<16>, !fir.real<16>>>, !fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> ()
+// CHECK:           %[[VAL_9:.*]] = fir.call @llvm.stacksave() : () -> !fir.ref<i8>
+// CHECK:           %[[VAL_10:.*]] = fir.alloca !fir.complex<16>
+// CHECK:           fir.store %[[VAL_8]] to %[[VAL_10]] : !fir.ref<!fir.complex<16>>
+// CHECK:           %[[VAL_11:.*]] = fir.convert %[[VAL_10]] : (!fir.ref<!fir.complex<16>>) -> !fir.ref<tuple<!fir.real<16>, !fir.real<16>>>
+// CHECK:           %[[VAL_12:.*]] = fir.alloca !fir.complex<16>
+// CHECK:           fir.store %[[VAL_6]] to %[[VAL_12]] : !fir.ref<!fir.complex<16>>
+// CHECK:           %[[VAL_13:.*]] = fir.convert %[[VAL_12]] : (!fir.ref<!fir.complex<16>>) -> !fir.ref<tuple<!fir.real<16>, !fir.real<16>>>
+// CHECK:           %[[VAL_14:.*]] = fir.alloca !fir.complex<16>
+// CHECK:           fir.store %[[VAL_4]] to %[[VAL_14]] : !fir.ref<!fir.complex<16>>
+// CHECK:           %[[VAL_15:.*]] = fir.convert %[[VAL_14]] : (!fir.ref<!fir.complex<16>>) -> !fir.ref<tuple<!fir.real<16>, !fir.real<16>>>
+// CHECK:           fir.call @calleemultipleparamscomplex16(%[[VAL_11]], %[[VAL_13]], %[[VAL_15]]) : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>, !fir.ref<tuple<!fir.real<16>, !fir.real<16>>>, !fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> ()
+// CHECK:           fir.call @llvm.stackrestore(%[[VAL_9]]) : (!fir.ref<i8>) -> ()
 // CHECK:           return
 // CHECK:         }
 
 // CHECK-LABEL:   func.func private @mlircomplexf128(
-// CHECK-SAME:      %[[VAL_0:.*]]: !fir.ref<tuple<f128, f128>> {llvm.align = 16 : i32, llvm.sret = tuple<f128, f128>}, %[[VAL_1:.*]]: !fir.ref<tuple<f128, f128>> {llvm.align = 16 : i32, llvm.byval = tuple<f128, f128>},  %[[VAL_2:.*]]: !fir.ref<tuple<f128, f128>> {llvm.align = 16 : i32, llvm.byval = tuple<f128, f128>}) {
+// CHECK-SAME:      %[[VAL_0:.*]]: !fir.ref<tuple<f128, f128>> {llvm.align = 16 : i32, llvm.sret = tuple<f128, f128>}, %[[VAL_1:.*]]: !fir.ref<tuple<f128, f128>> {llvm.align = 16 : i32, llvm.byval = tuple<f128, f128>}, %[[VAL_2:.*]]: !fir.ref<tuple<f128, f128>> {llvm.align = 16 : i32, llvm.byval = tuple<f128, f128>}) {
 // CHECK:           %[[VAL_3:.*]] = fir.convert %[[VAL_2]] : (!fir.ref<tuple<f128, f128>>) -> !fir.ref<complex<f128>>
 // CHECK:           %[[VAL_4:.*]] = fir.load %[[VAL_3]] : !fir.ref<complex<f128>>
 // CHECK:           %[[VAL_5:.*]] = fir.convert %[[VAL_1]] : (!fir.ref<tuple<f128, f128>>) -> !fir.ref<complex<f128>>
 // CHECK:           %[[VAL_6:.*]] = fir.load %[[VAL_5]] : !fir.ref<complex<f128>>
-// CHECK:           %[[VAL_7:.*]] = fir.alloca tuple<f128, f128>
-// CHECK:           %[[VAL_8:.*]] = fir.alloca complex<f128>
-// CHECK:           fir.store %[[VAL_6]] to %[[VAL_8]] : !fir.ref<complex<f128>>
-// CHECK:           %[[VAL_9:.*]] = fir.convert %[[VAL_8]] : (!fir.ref<complex<f128>>) -> !fir.ref<tuple<f128, f128>>
-// CHECK:           %[[VAL_10:.*]] = fir.alloca complex<f128>
-// CHECK:           fir.store %[[VAL_4]] to %[[VAL_10]] : !fir.ref<complex<f128>>
-// CHECK:           %[[VAL_11:.*]] = fir.convert %[[VAL_10]] : (!fir.ref<complex<f128>>) -> !fir.ref<tuple<f128, f128>>
-// CHECK:           fir.call @mlircomplexf128(%[[VAL_7]], %[[VAL_9]], %[[VAL_11]]) : (!fir.ref<tuple<f128, f128>>, !fir.ref<tuple<f128, f128>>, !fir.ref<tuple<f128, f128>>) -> ()
-// CHECK:           %[[VAL_12:.*]] = fir.convert %[[VAL_7]] : (!fir.ref<tuple<f128, f128>>) -> !fir.ref<complex<f128>>
-// CHECK:           %[[VAL_13:.*]] = fir.load %[[VAL_12]] : !fir.ref<complex<f128>>
-// CHECK:           %[[VAL_14:.*]] = fir.convert %[[VAL_0]] : (!fir.ref<tuple<f128, f128>>) -> !fir.ref<complex<f128>>
-// CHECK:           fir.store %[[VAL_13]] to %[[VAL_14]] : !fir.ref<complex<f128>>
+// CHECK:           %[[VAL_7:.*]] = fir.call @llvm.stacksave() : () -> !fir.ref<i8>
+// CHECK:           %[[VAL_8:.*]] = fir.alloca tuple<f128, f128>
+// CHECK:           %[[VAL_9:.*]] = fir.alloca complex<f128>
+// CHECK:           fir.store %[[VAL_6]] to %[[VAL_9]] : !fir.ref<complex<f128>>
+// CHECK:           %[[VAL_10:.*]] = fir.convert %[[VAL_9]] : (!fir.ref<complex<f128>>) -> !fir.ref<tuple<f128, f128>>
+// CHECK:           %[[VAL_11:.*]] = fir.alloca complex<f128>
+// CHECK:           fir.store %[[VAL_4]] to %[[VAL_11]] : !fir.ref<complex<f128>>
+// CHECK:           %[[VAL_12:.*]] = fir.convert %[[VAL_11]] : (!fir.ref<complex<f128>>) -> !fir.ref<tuple<f128, f128>>
+// CHECK:           fir.call @mlircomplexf128(%[[VAL_8]], %[[VAL_10]], %[[VAL_12]]) : (!fir.ref<tuple<f128, f128>>, !fir.ref<tuple<f128, f128>>, !fir.ref<tuple<f128, f128>>) -> ()
+// CHECK:           %[[VAL_13:.*]] = fir.convert %[[VAL_8]] : (!fir.ref<tuple<f128, f128>>) -> !fir.ref<complex<f128>>
+// CHECK:           %[[VAL_14:.*]] = fir.load %[[VAL_13]] : !fir.ref<complex<f128>>
+// CHECK:           fir.call @llvm.stackrestore(%[[VAL_7]]) : (!fir.ref<i8>) -> ()
+// CHECK:           %[[VAL_15:.*]] = fir.convert %[[VAL_0]] : (!fir.ref<tuple<f128, f128>>) -> !fir.ref<complex<f128>>
+// CHECK:           fir.store %[[VAL_14]] to %[[VAL_15]] : !fir.ref<complex<f128>>
 // CHECK:           return
 // CHECK:         }
 
 // CHECK-LABEL:   func.func @addrof() {
 // CHECK:           %[[VAL_0:.*]] = fir.address_of(@returncomplex16) : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> ()
 // CHECK:           %[[VAL_1:.*]] = fir.address_of(@paramcomplex16) : (!fir.ref<tuple<!fir.real<16>, !fir.real<16>>>) -> ()
 // CHECK:           return
 // CHECK:         }
+// CHECK:         func.func private @llvm.stacksave() -> !fir.ref<i8>
+// CHECK:         func.func private @llvm.stackrestore(!fir.ref<i8>)

flang/test/Fir/target.fir

 // AARCH64-LABEL: define void @call8()
 func.func @call8() {
   // I32: call void @gen8(ptr %
   // X64: = call { double, double } @gen8()
   // AARCH64: = call { double, double } @gen8()
   // PPC: = call { double, double } @gen8()
   %1 = fir.call @gen8() : () -> !fir.complex<8>
   // I32: call void @sink8(ptr %
-  // X64: call void @sink8(double %4, double %5)
+  // X64: call void @sink8(double %{{[0-9]*}}, double %{{[0-9]*}})
   // AARCH64: call void @sink8([2 x double] %
   // PPC: call void @sink8(double %{{.*}}, double %{{.*}})
   fir.call @sink8(%1) : (!fir.complex<8>) -> ()
   return
 }
 
 // I32-LABEL: define i64 @char1lensum(ptr %0, ptr %1, i32 %2, i32 %3)
 // X64-LABEL: define i64 @char1lensum(ptr %0, ptr %1, i64 %2, i64 %3)