[LoongArch] Add GHC Calling Convention

[lrzlin]

Summary

This is modeled after the RISCV GHC calling convention
and matches the corresponding GHC change.

[MaskRay]

Is GHC buildable now? If not, such changes probably should be deferred until verification can actually be made.

llvm/lib/Target/LoongArch/LoongArchISelLowering.cpp

1636

delete blank line

[lrzlin]

In D137495#3910276, @MaskRay wrote:

Is GHC buildable now? If not, such changes probably should be deferred until verification can actually be made.

Yes, with some modifications, both GHC HEAD and 9.4.2 Release could pass build and testsuite. However, GHC haven't implemented the support of LLVM 14 and higher versions yet, this may lead to a tardive merge in GHC side.

[SixWeining]

Thanks. But I have to admit I know little about GHC. Maybe I have to read some docs before reviewing the change.

llvm/test/CodeGen/LoongArch/ghc-cc.ll

22

fs4?

[lrzlin]

Fix typos of the register mapping and delete empty lines.

[lrzlin]

This is used in Implement LoongArch64 LLVM backend support

[wangleiat]

@lrzlin

I don't quite understand what you mean by` CodeModel=Small` not implemented? The default CodeModel we currently define is Small, which is the same as gcc's mcmodel=normal. Medium only uses pcalau12i+jirl to increase the jumping capability relative to pc when the function is called, and the others remain unchanged, which is also the same as gcc's mcmodel=medium, currently we only provide these two codemodels(D137394 for Medium), gcc also has a mcmodel=extreme(support 64bit addressing capability). For this(extreme), I haven't figured out which CodeModel corresponds to llvm, maybe large will be better. It's going to take a while to support it, because currently no one uses it except jit. The loongson gcc team may(just maybe) also consider the implementation of mcmodel=large (support 52bit addressing capability). so? large = large? or large=extreme? - -|||

[lrzlin]

@wangleiat
I have consulted the RISC-V codemodel here, which CodeModel::Small is represented by addi (lui %hi(sym)) %lo(sym), using for accessing the first 2GiB of address space, I think it could be implemented in LoongArch also, or LoongArch have a different codemodel here?

[wangleiat]

In D137495#3911248, @lrzlin wrote:

@wangleiat
I have consulted the RISC-V codemodel here, which CodeModel::Small is represented by addi (lui %hi(sym)) %lo(sym), using for accessing the first 2GiB of address space, I think it could be implemented in LoongArch also, or LoongArch have a different codemodel here?

RISC-V only distinguishes when relocation-model=static is specified, but it is not required on LoongArch. On LoongArch, regardless of relocation-model=everything, it is sufficient to use pc-relative instructions.

[lrzlin]

In D137495#3911433, @wangleiat wrote:

In D137495#3911248, @lrzlin wrote:

@wangleiat
I have consulted the RISC-V codemodel here, which CodeModel::Small is represented by addi (lui %hi(sym)) %lo(sym), using for accessing the first 2GiB of address space, I think it could be implemented in LoongArch also, or LoongArch have a different codemodel here?

RISC-V only distinguishes when relocation-model=static is specified, but it is not required on LoongArch. On LoongArch, regardless of relocation-model=everything, it is sufficient to use pc-relative instructions.

Without the isPositionIndepent() and code-model check, original getAddr() will map the PseudoLA_GOT pattern in GHC calling convention (in ghc-cc.ll), which is obviously incorrect (should be LA_PCREL). In case that all relocation-models are sufficient to use pc-relative instructions, both CodeModel=Small and CodeModel=Medium could keep being LA_PCREL.

[wangleiat]

In D137495#3911535, @lrzlin wrote:

In D137495#3911433, @wangleiat wrote:

In D137495#3911248, @lrzlin wrote:

@wangleiat
I have consulted the RISC-V codemodel here, which CodeModel::Small is represented by addi (lui %hi(sym)) %lo(sym), using for accessing the first 2GiB of address space, I think it could be implemented in LoongArch also, or LoongArch have a different codemodel here?

RISC-V only distinguishes when relocation-model=static is specified, but it is not required on LoongArch. On LoongArch, regardless of relocation-model=everything, it is sufficient to use pc-relative instructions.

Without the isPositionIndepent() and code-model check, original getAddr() will map the PseudoLA_GOT pattern in GHC calling convention (in ghc-cc.ll), which is obviously incorrect (should be LA_PCREL). In case that all relocation-models are sufficient to use pc-relative instructions, both CodeModel=Small and CodeModel=Medium could keep being LA_PCREL.

Personally I think using GOT or PCREL should have nothing to do with isPositionIndependent . If this test case uses -relocation-model=pic on RISC-V, it should also generate got related instructions.
In addition, I really don't understand the calling convention of ghc, is it not allowed to use got with ghc?

[lrzlin]

@wangleiat
Basically, what GHC calling convention do is mapping the virtual registers of STG Machine (part of the Haskell Runtime System) into real registers, so here in ghc-cc.ll must be pcrel, in short, it isn't a function call, but a simple value load process, using got here will lead to load a symbol twice, which isn't a function symbol, so it will not present in GOT table. More information about this calling convention could find on GHC llvm porting
Also, using -pic or not is determined by GHC itself, and judge by its knowing behavior, isPositionIndependent is needed to avoid generating got patterns while compiling it using llvm backend.

[xen0n]

Thanks for the contribution, I've checked against your GHC-side changes and the register convention looks good. Apart from some minor nits this should be good to go.

llvm/lib/Target/LoongArch/LoongArchISelLowering.cpp

445–446

Is the comment still necessary? The small codemodel case is being implemented right below...

llvm/test/CodeGen/LoongArch/ghc-cc.ll

2

In LoongArch testcases --mattr (double-dash) is used throughout.

84–100

Please convert to opaque pointers.

[lrzlin]

Following @xen0n's advice: Fix --mattr and replace pointer types with opaque pointers in ghc-cc.ll, delete TODO in CodeModel:Small and make it fall through to CodeModel:Medium.

[xen0n]

LGTM, thanks very much for the contribution!

As discussed elsewhere, there's the possibility of making use of $s9 because we don't need a frame pointer, but as the GHC changes are already merged, this can come later with another pair of coordinated changes.

Any opinions on this from other reviewers?

[SixWeining]

1. Could you please rebase the change? I failed to apply it to my codebase.
2. It's better to add -U999999 when generating the change so that we can have more context. See: https://llvm.org/docs/Phabricator.html#requesting-a-review-via-the-web-interface

[lrzlin]

In D137495#3993789, @SixWeining wrote:

1. Could you please rebase the change? I failed to apply it to my codebase.
2. It's better to add -U999999 when generating the change so that we can have more context. See: https://llvm.org/docs/Phabricator.html#requesting-a-review-via-the-web-interface

Rebased and add -U999999 while generating the patch.

[wangleiat]

llvm/lib/Target/LoongArch/LoongArchISelLowering.cpp

471

I think there is no need to judge here. The following modifications(switch codemodel) should also be unnecessary. This modification will change the semantics of dso_preemptable. https://llvm.org/docs/LangRef.html#runtime-preemption-specifiers

llvm/test/CodeGen/LoongArch/ghc-cc.ll

7–26

dso_local can be used.

[lrzlin]

Using dso_local in ghc-cc.ll and revert the changes in getAddr() to keep the semantics of dso_preemtable.

[wangleiat]

LGTM, Thanks.

[xen0n]

This is cleaner than previous revisions (which contained code model related changes) which is good. I've slightly adjusted the patch summary to remove the trivial bits and now inaccurate descriptions.

Thanks very much for the contribution!

[SixWeining]

llvm/lib/Target/LoongArch/LoongArchFrameLowering.cpp

194

Please remove useless spaces.

199

Ditto.

330

Ditto.

335

Ditto.

llvm/lib/Target/LoongArch/LoongArchISelLowering.cpp

2539

Ditto.

2542

Ditto.

llvm/test/CodeGen/LoongArch/ghc-cc.ll

108

Ditto.

[lrzlin]

Delete blank lines as @SixWeining suggest.

[SixWeining]

In D137495#4016701, @lrzlin wrote:

Delete blank lines as @SixWeining suggest.

Thanks. Do you have commit access? If not, I could commit it for you if you leave the name and email here.

[lrzlin]

In D137495#4016708, @SixWeining wrote:

In D137495#4016701, @lrzlin wrote:

Delete blank lines as @SixWeining suggest.

Thanks. Do you have commit access? If not, I could commit it for you if you leave the name and email here.

I do not have commit access, my name is Lin Runze and my email is [e-mail], thanks for your help.

[SixWeining]

Unfortunately it breads the build of buildbot: https://lab.llvm.org/staging/#/builders/236/builds/206. Please fix it as soon as possible, otherewise it would be reverted.

[SixWeining]

In D137495#4016750, @SixWeining wrote:

Unfortunately it breads the build of buildbot: https://lab.llvm.org/staging/#/builders/236/builds/206. Please fix it as soon as possible, otherewise it would be reverted.

D140670 fixs it.

llvm/lib/Target/LoongArch/LoongArchFrameLowering.cpp

   // Debug location must be unknown since the first debug location is used
   // to determine the end of the prologue.
   DebugLoc DL;
   
+  // All calls are tail calls in GHC calling conv, and functions have no
+  // prologue/epilogue.
+  if (MF.getFunction().getCallingConv() == CallingConv::GHC)
+    return;
+  
   // Determine the correct frame layout
   determineFrameLayout(MF);
 

   MachineFrameInfo &MFI = MF.getFrameInfo();
   auto *LoongArchFI = MF.getInfo<LoongArchMachineFunctionInfo>();
   Register SPReg = LoongArch::R3;
   
+  // All calls are tail calls in GHC calling conv, and functions have no
+  // prologue/epilogue.
+  if (MF.getFunction().getCallingConv() == CallingConv::GHC)
+    return;
+  
   MachineBasicBlock::iterator MBBI = MBB.getFirstTerminator();
   DebugLoc DL = MBBI != MBB.end() ? MBBI->getDebugLoc() : DebugLoc();
 

llvm/lib/Target/LoongArch/LoongArchISelLowering.cpp

   SDLoc DL(N);
   EVT Ty = getPointerTy(DAG.getDataLayout());
   SDValue Addr = getTargetNode(N, DL, Ty, DAG, 0);
-  // TODO: Check CodeModel.
-  if (IsLocal)
+  
+  if (isPositionIndependent()) {
+    if (IsLocal)
+      // This generates the pattern (PseudoLA_PCREL sym), which expands to
+      // (addi.w/d (pcalau12i %pc_hi20(sym)) %pc_lo12(sym)).
+      return SDValue(DAG.getMachineNode(LoongArch::PseudoLA_PCREL, DL, Ty, Addr),
+                     0);
+
+    // This generates the pattern (PseudoLA_GOT sym), which expands to (ld.w/d
+    // (pcalau12i %got_pc_hi20(sym)) %got_pc_lo12(sym)).
+    return SDValue(DAG.getMachineNode(LoongArch::PseudoLA_GOT, DL, Ty, Addr), 0);
+  }
+  
+  switch (getTargetMachine().getCodeModel()) {
+  default:
+    report_fatal_error("Unsupported code model for lowering");
+  // TODO: Add Small CodeModel.
+  case CodeModel::Small: {
+    return SDValue(DAG.getMachineNode(LoongArch::PseudoLA_PCREL, DL, Ty, Addr),
+                     0);
+  }
+  case CodeModel::Medium: {
     // This generates the pattern (PseudoLA_PCREL sym), which expands to
     // (addi.w/d (pcalau12i %pc_hi20(sym)) %pc_lo12(sym)).
     return SDValue(DAG.getMachineNode(LoongArch::PseudoLA_PCREL, DL, Ty, Addr),
                      0);
-
-  // This generates the pattern (PseudoLA_GOT sym), which expands to (ld.w/d
-  // (pcalau12i %got_pc_hi20(sym)) %got_pc_lo12(sym)).
-  return SDValue(DAG.getMachineNode(LoongArch::PseudoLA_GOT, DL, Ty, Addr), 0);
+  }
+  }
 }
 
 SDValue LoongArchTargetLowering::lowerBlockAddress(SDValue Op,

   SDValue Addr;
   TLSModel::Model Model = getTargetMachine().getTLSModel(N->getGlobal());
 
+  if (DAG.getMachineFunction().getFunction().getCallingConv() ==
+      CallingConv::GHC)
+    report_fatal_error("In GHC calling convention TLS is not supported");
+
   switch (Model) {
   case TLSModel::GeneralDynamic:
     // In this model, application code calls the dynamic linker function

   return Val;
 }
 
+static bool CC_LoongArch_GHC(unsigned ValNo, MVT ValVT, MVT LocVT,
+                            CCValAssign::LocInfo LocInfo,
+                            ISD::ArgFlagsTy ArgFlags, CCState &State) {
+  if (LocVT == MVT::i32 || LocVT == MVT::i64) {
+    // Pass in STG registers: Base, Sp, Hp, R1, R2, R3, R4, R5, SpLim
+    //                        s0    s1  s2  s3  s4  s5  s6  s7  s8
+    static const MCPhysReg GPRList[] = {
+        LoongArch::R23, LoongArch::R24, LoongArch::R25, LoongArch::R26, LoongArch::R27,
+        LoongArch::R28, LoongArch::R29, LoongArch::R30, LoongArch::R31};
+    if (unsigned Reg = State.AllocateReg(GPRList)) {
+      State.addLoc(CCValAssign::getReg(ValNo, ValVT, Reg, LocVT, LocInfo));
+      return false;
+    }
+  }
+
+  if (LocVT == MVT::f32) {
+    // Pass in STG registers: F1, F2, F3, F4
+    //                        fs0,fs1,fs2,fs3
+    static const MCPhysReg FPR32List[] = {LoongArch::F24, LoongArch::F25,
+                                          LoongArch::F26, LoongArch::F27};
+    if (unsigned Reg = State.AllocateReg(FPR32List)) {
+      State.addLoc(CCValAssign::getReg(ValNo, ValVT, Reg, LocVT, LocInfo));
+      return false;
+    }
+  }
+
+  if (LocVT == MVT::f64) {
+    // Pass in STG registers: D1, D2, D3, D4
+    //                        fs4,fs5,fs6,fs7
+    static const MCPhysReg FPR64List[] = {LoongArch::F28_64, LoongArch::F29_64,
+                                          LoongArch::F30_64, LoongArch::F31_64};
+    if (unsigned Reg = State.AllocateReg(FPR64List)) {
+      State.addLoc(CCValAssign::getReg(ValNo, ValVT, Reg, LocVT, LocInfo));
+      return false;
+    }
+  }
+
+  report_fatal_error("No registers left in GHC calling convention");
+  return true;
+}
+
 // Transform physical registers into virtual registers.
 SDValue LoongArchTargetLowering::LowerFormalArguments(
     SDValue Chain, CallingConv::ID CallConv, bool IsVarArg,

   case CallingConv::C:
   case CallingConv::Fast:
     break;
+  case CallingConv::GHC:
+    if (!MF.getSubtarget().getFeatureBits()[LoongArch::FeatureBasicF] ||
+        !MF.getSubtarget().getFeatureBits()[LoongArch::FeatureBasicD])
+      report_fatal_error(
+        "GHC calling convention requires the F and D extensions");
   }
 
   EVT PtrVT = getPointerTy(DAG.getDataLayout());

   SmallVector<CCValAssign> ArgLocs;
   CCState CCInfo(CallConv, IsVarArg, MF, ArgLocs, *DAG.getContext());
 
-  analyzeInputArgs(MF, CCInfo, Ins, /*IsRet=*/false, CC_LoongArch);
+  if (CallConv == CallingConv::GHC)
+    CCInfo.AnalyzeFormalArguments(Ins, CC_LoongArch_GHC);
+  else
+    analyzeInputArgs(MF, CCInfo, Ins, /*IsRet=*/false, CC_LoongArch);
 
   for (unsigned i = 0, e = ArgLocs.size(); i != e; ++i) {
     CCValAssign &VA = ArgLocs[i];

   SmallVector<CCValAssign> ArgLocs;
   CCState ArgCCInfo(CallConv, IsVarArg, MF, ArgLocs, *DAG.getContext());
 
-  analyzeOutputArgs(MF, ArgCCInfo, Outs, /*IsRet=*/false, &CLI, CC_LoongArch);
+  if (CallConv == CallingConv::GHC)
+    ArgCCInfo.AnalyzeCallOperands(Outs, CC_LoongArch_GHC);
+  else
+    analyzeOutputArgs(MF, ArgCCInfo, Outs, /*IsRet=*/false, &CLI, CC_LoongArch);
 
   // Get a count of how many bytes are to be pushed on the stack.
   unsigned NumBytes = ArgCCInfo.getNextStackOffset();

 
   analyzeOutputArgs(DAG.getMachineFunction(), CCInfo, Outs, /*IsRet=*/true,
                     nullptr, CC_LoongArch);
   
+  if (CallConv == CallingConv::GHC && !RVLocs.empty())
+    report_fatal_error("GHC functions return void only");
+    
   SDValue Glue;
   SmallVector<SDValue, 4> RetOps(1, Chain);
 

llvm/lib/Target/LoongArch/LoongArchRegisterInfo.cpp

 LoongArchRegisterInfo::getCalleeSavedRegs(const MachineFunction *MF) const {
   auto &Subtarget = MF->getSubtarget<LoongArchSubtarget>();
 
+  if (MF->getFunction().getCallingConv() == CallingConv::GHC)
+    return CSR_NoRegs_SaveList;
   switch (Subtarget.getTargetABI()) {
   default:
     llvm_unreachable("Unrecognized ABI");

                                             CallingConv::ID CC) const {
   auto &Subtarget = MF.getSubtarget<LoongArchSubtarget>();
 
+  if (CC == CallingConv::GHC)
+    return CSR_NoRegs_RegMask;
   switch (Subtarget.getTargetABI()) {
   default:
     llvm_unreachable("Unrecognized ABI");

llvm/test/CodeGen/LoongArch/ghc-cc.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc --mtriple=loongarch64 -mattr=+f,+d < %s | FileCheck %s --check-prefix=LA64
+
+; Check the GHC call convention works (la64)
+
+@base  = external global i64 ; assigned to register: s0
+@sp    = external global i64 ; assigned to register: s1
+@hp    = external global i64 ; assigned to register: s2
+@r1    = external global i64 ; assigned to register: s3
+@r2    = external global i64 ; assigned to register: s4
+@r3    = external global i64 ; assigned to register: s5
+@r4    = external global i64 ; assigned to register: s6
+@r5    = external global i64 ; assigned to register: s7
+@splim = external global i64 ; assigned to register: s8
+
+@f1 = external global float  ; assigned to register: fs0
+@f2 = external global float  ; assigned to register: fs1
+@f3 = external global float  ; assigned to register: fs2
+@f4 = external global float  ; assigned to register: fs3
+
+@d1 = external global double ; assigned to register: fs4
+@d2 = external global double ; assigned to register: fs5
+@d3 = external global double ; assigned to register: fs6
+@d4 = external global double ; assigned to register: fs7
+
+define ghccc void @foo() nounwind {
+; LA64-LABEL: foo:
+; LA64:   # %bb.0: # %entry
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(base)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(base)
+; LA64-NEXT:  ld.d  $s0, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(sp)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(sp)
+; LA64-NEXT:  ld.d  $s1, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(hp)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(hp)
+; LA64-NEXT:  ld.d  $s2, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(r1)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(r1)
+; LA64-NEXT:  ld.d  $s3, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(r2)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(r2)
+; LA64-NEXT:  ld.d  $s4, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(r3)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(r3)
+; LA64-NEXT:  ld.d  $s5, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(r4)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(r4)
+; LA64-NEXT:  ld.d  $s6, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(r5)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(r5)
+; LA64-NEXT:  ld.d  $s7, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(splim)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(splim)
+; LA64-NEXT:  ld.d  $s8, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(f1)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(f1)
+; LA64-NEXT:  fld.s $fs0, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(f2)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(f2)
+; LA64-NEXT:  fld.s $fs1, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(f3)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(f3)
+; LA64-NEXT:  fld.s $fs2, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(f4)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(f4)
+; LA64-NEXT:  fld.s $fs3, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(d1)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(d1)
+; LA64-NEXT:  fld.d $fs4, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(d2)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(d2)
+; LA64-NEXT:  fld.d $fs5, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(d3)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(d3)
+; LA64-NEXT:  fld.d $fs6, $a0, 0
+; LA64-NEXT:  pcalau12i $a0, %pc_hi20(d4)
+; LA64-NEXT:  addi.d  $a0, $a0, %pc_lo12(d4)
+; LA64-NEXT:  fld.d $fs7, $a0, 0
+; LA64-NEXT:  bl  %plt(bar)
+; LA64-NEXT:  ret
+
+entry:
+  %0  = load double, double* @d4
+  %1  = load double, double* @d3
+  %2  = load double, double* @d2
+  %3  = load double, double* @d1
+  %4  = load float, float* @f4
+  %5  = load float, float* @f3
+  %6 = load float, float* @f2
+  %7 = load float, float* @f1
+  %8 = load i64, i64* @splim
+  %9 = load i64, i64* @r5
+  %10 = load i64, i64* @r4
+  %11 = load i64, i64* @r3
+  %12 = load i64, i64* @r2
+  %13 = load i64, i64* @r1
+  %14 = load i64, i64* @hp
+  %15 = load i64, i64* @sp
+  %16 = load i64, i64* @base
+  tail call ghccc void @bar(i64 %16, i64 %15, i64 %14, i64 %13, i64 %12,
+                            i64 %11, i64 %10, i64 %9, i64 %8, float %7, float %6,
+                            float %5, float %4, double %3, double %2, double %1, double %0) nounwind
+  ret void
+}
+
+declare ghccc void @bar(i64, i64, i64, i64, i64, i64, i64, i64, i64, 
+                        float, float, float, float,
+                        double, double, double, double)