[llvm][AArch64][SVE] Fold literals into math instructions

[DavidTruby]

Summary

SVE has predicated literal forms of some instructions for specific
literals, which currently are generated correctly when using ACLE
but not when those instructions are generated directly.

This adds the patterns to generate those instructions when
generating from standard LLVM IR instructions.

Co-authored-by: Sander De Smalen <[e-mail]>
Co-authored-by: Cullen Rhodes <[e-mail]>

[david-arm]

llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td

2254

Hi @DavidTruby, this is quite a lot of very similar looking patterns to maintain here. Is there a possible way to condense them somehow, i.e. the A and B patterns are duplicates, so you could invoke a multiclass for A, then a multiclass for B?

[DavidTruby]

Added extra tests for fmax, fmaxnm, fmin, fminnm

[DavidTruby]

Refactored patterns into helper multiclass

[DavidTruby]

llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td

2254

@david-arm hopefully this helps a bit, it at least means that each pattern is only mentioned once so only need changing once if they have to be changed in future

[joechrisellis]

llvm/lib/Target/AArch64/AArch64InstrFormats.td

1209–1219

nit: really small thing but the naming here is inconsistent with fpimm0. I would expect fpimm1 and fpimm2. I suppose the difficulty here is shoehoring fpimm_half into this naming convention. If you can think of a sensible way of doing this, I would prefer we change this. Otherwise, ignore me. :-)

llvm/test/CodeGen/AArch64/sve-fp-immediates-merging.ll

2

General test suggestion -- prefer:

target triple = "aarch64-unknown-linux-gnu"

...

attributes #0 = { "target-features"="+sve" }

... and tag your functions with #0. I personally find this to be a bit more reliable when you're running llc on this file. 😄

Also might be worth adding:

; RUN: ... 2>%t ...
; RUN: FileCheck --check-prefix=WARN --allow-empty %s <%t

3

Please can we add:

; If this check fails please read test/CodeGen/AArch64/README for instructions on how to resolve it.
; WARN-NOT: warning

[DavidTruby]

Moved the zeroing predicates behind the UseExperimentalZeroingPseudos feature
Added additional tests for one of the patterns
Addressed misc reivew comments

[DavidTruby]

ping

[paulwalker-arm]

llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td

411–418

These will need to be split in two because the the UNDEF forms are safe to use by default but the ZERO forms must be protected by UseExperimentalZeroingPseudos. You can see how we've done this for FADD_ZPZZ where there exists two multiclasses, sve_fp_bin_pred_hfd and sve_fp_2op_p_zds_zeroing_hsd.

2254

My immediate thought is whether these (or at least some of these) be rolled into the instruction/pseudo multiclasses. By this I mean can sve_fp_2op_i_p_zds be extended with extra parameters so the necessary intrinsic patterns can be added.

Likewise for the multiclasses used to create the UNDEF and ZERO psuedos, can the multiclasses themselves contain the IR patterns plus the vselect variants.

llvm/test/CodeGen/AArch64/sve-fp-immediates-merging.ll

3

Just a note to say this and the above RUN: ... 2>%t request are no longer necessary because llc will now error in these circumstances.

6–7

Minor point but attributes are normally placed after the function definitions as that where the IR printer would put them.

15–16

Please use update_llc_test_checks.py to generate the CHECK lines.

[DavidTruby]

Separated zeroing forms and regenreated check lines

[DavidTruby]

This comment was removed by DavidTruby.

[DavidTruby]

Fixed accidentally overwriting this with a different patch...

[DavidTruby]

Merge multiclasses into SVEInstrFormats.td

[DavidTruby]

@paulwalker-arm sorry for the long lag on this; is this new update more like what you wanted to see?

[bsmith]

Would be good to be more consistent with the existing classes in here, for example the classes with patterns in are mostly near the top of the file with specific naming SVE_ style.

llvm/lib/Target/AArch64/SVEInstrFormats.td

8157

This doesn't really need to be a separate mutliclass as it is only instantiated once, similar with the other classes.

[DavidTruby]

Merge additional multiclasses

[paulwalker-arm]

llvm/lib/Target/AArch64/SVEInstrFormats.td

1730–1733

Assuming my other comments are valid I believe we won't need a multiclass and can then follow the same idiom as with class SVE_2_Op_Pred... (i.e. add new classes in the SVE pattern match helpers section towards the start of this file.

1734–1737

This pattern seems like a better fit for the sve_fp_2op_i_p_zds multiclass where the relevant instructions are defined. Is that possible?

1738–1741

Is this pattern necessary? I'm thinking that even for the IR ops you can just pass the predicate through and thus we can reuse the first pattern. Doing this means we'll also take advantage of the immediate forms when lowering VLS operations.

1745

There's a bit of clash with naming here. In the past _zx implied zeroing where as we now typically attach element types info for the undef case (i.e. _hfd in this instance) and then zeroing_hfd for the zeroing case.

1769

Perhaps you don't need this parameter. Typically you can reference NAME which is the string after the defm when used. So for example you have:

defm FADD_ZPZI    : sve_fp_2op_i_p_zds_zx_zeroing

so NAME will be FADD_ZPZI and thus you can then do:

defm : sve_intrinsic_compact_fp_immediates_zeroing<NAME # "_ZERO_H",

This suggestion likely applies to sve_intrinsic_compact_fp_immediates also, especially if the second pattern can be removed.

[DavidTruby]

Further cleanup of patterns

[DavidTruby]

Hopefully this is going broadly in the direction you wanted @paulwalker-arm I just had a couple of questions about your comments

llvm/lib/Target/AArch64/SVEInstrFormats.td

1738–1741

I'm not sure I understand fully; I've managed to reuse this pattern for both the op and ir_op but it definitely seems to still be necessary to have the ir_op version there. I might be wrong about what you meant though?

1745

Hopefully the new names are more what you're expecting? Let me know if not!

[DavidTruby]

Further refactor of patterns

[paulwalker-arm]

Structurally this is looking great. There is just a few missing patterns after which I think you're done.

llvm/lib/Target/AArch64/SVEInstrFormats.td

443

Does the inst line need the PPR_3b: and ZPR: parts? I ask because the next class you've added seems to work without them.

1734–1739

For this class, which deals the the normal IR nodes (i.e. not intrinsics), can you add entries for all the legal floating point types (i.e. add entries for nxv2f16, nxv4f16 & nxv2f32) along with suitable tests.

llvm/test/CodeGen/AArch64/sve-fp-immediates-merging.ll

23–34

Sorry to be a pain but can you move the intrinsic tests into a separate file. This is what we've done for some of the other ones, for example there's sve-intrinsics-int-arith.ll and then sve-intrinsics-int-arith-imm.ll. So can you move them into sve-intrinsics-fp-arith-imm.ll leaving this file to just contain the stock IR tests.

[DavidTruby]

Add extra patterns and tests for additional legal types

llvm/lib/Target/AArch64/AArch64InstrFormats.td

def fpimm8 : Operand<i32> {
   let ParserMatchClass = FPImmOperand;
   let PrintMethod = "printFPImmOperand";
 }
 
 def fpimm0 : FPImmLeaf<fAny, [{
   return Imm.isExactlyValue(+0.0);
 }]>;
 
+def fpimm_half : FPImmLeaf<fAny, [{
+  return Imm.isExactlyValue(+0.5);
+}]>;
+
+def fpimm_one : FPImmLeaf<fAny, [{
+  return Imm.isExactlyValue(+1.0);
+}]>;
+
+def fpimm_two : FPImmLeaf<fAny, [{
+  return Imm.isExactlyValue(+2.0);
+}]>;
+
 def gi_fpimm16 : GICustomOperandRenderer<"renderFPImm16">,
   GISDNodeXFormEquiv<fpimm16XForm>;
 def gi_fpimm32 : GICustomOperandRenderer<"renderFPImm32">,
   GISDNodeXFormEquiv<fpimm32XForm>;
 def gi_fpimm64 : GICustomOperandRenderer<"renderFPImm64">,
   GISDNodeXFormEquiv<fpimm64XForm>;
 
 // Vector lane operands

llvm/lib/Target/AArch64/AArch64SVEInstrInfo.td

let Predicates = [HasSVE] in {
   defm SMAX_ZPZZ  : sve_int_bin_pred_bhsd<AArch64smax_p>;
   defm UMAX_ZPZZ  : sve_int_bin_pred_bhsd<AArch64umax_p>;
   defm SMIN_ZPZZ  : sve_int_bin_pred_bhsd<AArch64smin_p>;
   defm UMIN_ZPZZ  : sve_int_bin_pred_bhsd<AArch64umin_p>;
 
   defm FRECPE_ZZ  : sve_fp_2op_u_zd<0b110, "frecpe",  int_aarch64_sve_frecpe_x>;
   defm FRSQRTE_ZZ : sve_fp_2op_u_zd<0b111, "frsqrte", int_aarch64_sve_frsqrte_x>;
 
-  defm FADD_ZPmI    : sve_fp_2op_i_p_zds<0b000, "fadd", sve_fpimm_half_one>;
-  defm FSUB_ZPmI    : sve_fp_2op_i_p_zds<0b001, "fsub", sve_fpimm_half_one>;
-  defm FMUL_ZPmI    : sve_fp_2op_i_p_zds<0b010, "fmul", sve_fpimm_half_two>;
-  defm FSUBR_ZPmI   : sve_fp_2op_i_p_zds<0b011, "fsubr", sve_fpimm_half_one>;
-  defm FMAXNM_ZPmI  : sve_fp_2op_i_p_zds<0b100, "fmaxnm", sve_fpimm_zero_one>;
-  defm FMINNM_ZPmI  : sve_fp_2op_i_p_zds<0b101, "fminnm", sve_fpimm_zero_one>;
-  defm FMAX_ZPmI    : sve_fp_2op_i_p_zds<0b110, "fmax", sve_fpimm_zero_one>;
-  defm FMIN_ZPmI    : sve_fp_2op_i_p_zds<0b111, "fmin", sve_fpimm_zero_one>;
+  defm FADD_ZPmI    : sve_fp_2op_i_p_zds<0b000, "fadd", "FADD_ZPZI", sve_fpimm_half_one>;
+  defm FSUB_ZPmI    : sve_fp_2op_i_p_zds<0b001, "fsub", "FSUB_ZPZI", sve_fpimm_half_one>;
+  defm FMUL_ZPmI    : sve_fp_2op_i_p_zds<0b010, "fmul", "FMUL_ZPZI", sve_fpimm_half_two>;
+  defm FSUBR_ZPmI   : sve_fp_2op_i_p_zds<0b011, "fsubr", "FSUBR_ZPZI", sve_fpimm_half_one>;
+  defm FMAXNM_ZPmI  : sve_fp_2op_i_p_zds<0b100, "fmaxnm", "FMAXNM_ZPZI", sve_fpimm_zero_one>;
+  defm FMINNM_ZPmI  : sve_fp_2op_i_p_zds<0b101, "fminnm", "FMINNM_ZPZI", sve_fpimm_zero_one>;
+  defm FMAX_ZPmI    : sve_fp_2op_i_p_zds<0b110, "fmax", "FMAX_ZPZI", sve_fpimm_zero_one>;
+  defm FMIN_ZPmI    : sve_fp_2op_i_p_zds<0b111, "fmin", "FMIN_ZPZI", sve_fpimm_zero_one>;
+   
+  defm FADD_ZPZI    : sve_fp_2op_i_p_zds_zx<sve_fpimm_half_one>;
+  defm FSUB_ZPZI    : sve_fp_2op_i_p_zds_zx<sve_fpimm_half_one>;
+  defm FMUL_ZPZI    : sve_fp_2op_i_p_zds_zx<sve_fpimm_half_two>;
+  defm FSUBR_ZPZI   : sve_fp_2op_i_p_zds_zx<sve_fpimm_half_one>;
+  defm FMAXNM_ZPZI  : sve_fp_2op_i_p_zds_zx<sve_fpimm_zero_one>;
+  defm FMINNM_ZPZI  : sve_fp_2op_i_p_zds_zx<sve_fpimm_zero_one>;
+  defm FMAX_ZPZI    : sve_fp_2op_i_p_zds_zx<sve_fpimm_zero_one>;
+  defm FMIN_ZPZI    : sve_fp_2op_i_p_zds_zx<sve_fpimm_zero_one>;
+
+  let Predicates = [HasSVE, UseExperimentalZeroingPseudos] in {
+    defm FADD_ZPZI    : sve_fp_2op_i_p_zds_zx_zeroing<sve_fpimm_half_one>;
+    defm FSUB_ZPZI    : sve_fp_2op_i_p_zds_zx_zeroing<sve_fpimm_half_one>;
+    defm FMUL_ZPZI    : sve_fp_2op_i_p_zds_zx_zeroing<sve_fpimm_half_two>;
+    defm FSUBR_ZPZI   : sve_fp_2op_i_p_zds_zx_zeroing<sve_fpimm_half_one>;
+    defm FMAXNM_ZPZI  : sve_fp_2op_i_p_zds_zx_zeroing<sve_fpimm_zero_one>;
+    defm FMINNM_ZPZI  : sve_fp_2op_i_p_zds_zx_zeroing<sve_fpimm_zero_one>;
+    defm FMAX_ZPZI    : sve_fp_2op_i_p_zds_zx_zeroing<sve_fpimm_zero_one>;
+    defm FMIN_ZPZI    : sve_fp_2op_i_p_zds_zx_zeroing<sve_fpimm_zero_one>;
+  }
 
   defm FADD_ZPmZ   : sve_fp_2op_p_zds<0b0000, "fadd", "FADD_ZPZZ", int_aarch64_sve_fadd, DestructiveBinaryComm>;
   defm FSUB_ZPmZ   : sve_fp_2op_p_zds<0b0001, "fsub", "FSUB_ZPZZ", int_aarch64_sve_fsub, DestructiveBinaryCommWithRev, "FSUBR_ZPmZ">;
   defm FMUL_ZPmZ   : sve_fp_2op_p_zds<0b0010, "fmul", "FMUL_ZPZZ", int_aarch64_sve_fmul, DestructiveBinaryComm>;
   defm FSUBR_ZPmZ  : sve_fp_2op_p_zds<0b0011, "fsubr", "FSUBR_ZPZZ", int_aarch64_sve_fsubr, DestructiveBinaryCommWithRev, "FSUB_ZPmZ", /*isReverseInstr*/ 1>;
   defm FMAXNM_ZPmZ : sve_fp_2op_p_zds<0b0100, "fmaxnm", "FMAXNM_ZPZZ", int_aarch64_sve_fmaxnm, DestructiveBinaryComm>;
   defm FMINNM_ZPmZ : sve_fp_2op_p_zds<0b0101, "fminnm", "FMINNM_ZPZZ", int_aarch64_sve_fminnm, DestructiveBinaryComm>;
   defm FMAX_ZPmZ   : sve_fp_2op_p_zds<0b0110, "fmax", "FMAX_ZPZZ", int_aarch64_sve_fmax, DestructiveBinaryComm>;
def : Pat<(nxv4i32 (vector_insert (nxv4i32 ZPR:$vec), GPR32:$src, GPR64:$index)),
                         GPR32:$src)>;
   def : Pat<(nxv2i64 (vector_insert (nxv2i64 ZPR:$vec), GPR64:$src, GPR64:$index)),
             (CPY_ZPmR_D ZPR:$vec,
                         (CMPEQ_PPzZZ_D (PTRUE_D 31),
                                        (INDEX_II_D 0, 1),
                                        (DUP_ZR_D GPR64:$index)),
                         GPR64:$src)>;
 
+  /// Compact single bit fp immediates
+  multiclass intrinsic_compact_fp_immediates_patterns<string I, string IX, string IZ,
+                    ValueType itype, ValueType ftype, ValueType sftype,
+                    SDPatternOperator op, SDPatternOperator ir_op, FPImmLeaf A,
+                    int imm> {
+    def : Pat<(ftype (op (itype PPR_3b:$Pg),
+                           (ftype ZPR:$Zs1),
+                           (ftype (AArch64dup (sftype A))))),
+              (!cast<Instruction>(I) PPR_3b:$Pg, ZPR:$Zs1, imm)>;
+    def : Pat<(ftype (ir_op (itype (AArch64ptrue 31)),
+                              (ftype ZPR:$Zs1),
+                              (ftype (AArch64dup (sftype A))))),
+              (!cast<Instruction>(IX) (PTRUE_S 31), ZPR:$Zs1, imm)>;
+  }
+
+  multiclass intrinsic_compact_fp_immediates<string I, string IZ, string IX,
+                    FPImmLeaf A, FPImmLeaf B,
+                    SDPatternOperator op, SDPatternOperator ir_op = null_frag> {
+    defm : intrinsic_compact_fp_immediates_patterns<I # "_H", IX # "_H", IZ # "_H", nxv8i1, nxv8f16, f16, op, ir_op, A, 0>;
+    defm : intrinsic_compact_fp_immediates_patterns<I # "_H", IX # "_H", IZ # "_H", nxv8i1, nxv8f16, f16, op, ir_op, B, 1>;
+    defm : intrinsic_compact_fp_immediates_patterns<I # "_S", IX # "_S", IZ # "_S", nxv4i1, nxv4f32, f32, op, ir_op, A, 0>;
+    defm : intrinsic_compact_fp_immediates_patterns<I # "_S", IX # "_S", IZ # "_S", nxv4i1, nxv4f32, f32, op, ir_op, B, 1>;
+    defm : intrinsic_compact_fp_immediates_patterns<I # "_D", IX # "_D", IZ # "_D", nxv2i1, nxv2f64, f64, op, ir_op, A, 0>;
+    defm : intrinsic_compact_fp_immediates_patterns<I # "_D", IX # "_D", IZ # "_D", nxv2i1, nxv2f64, f64, op, ir_op, B, 1>;
+  }
+
+  defm : intrinsic_compact_fp_immediates<"FADD_ZPmI",  "FADD_ZPZI_ZERO",  "FADD_ZPZI_UNDEF",   fpimm_half, fpimm_one, int_aarch64_sve_fadd, AArch64fadd_p>;
+  defm : intrinsic_compact_fp_immediates<"FSUB_ZPmI",  "FSUB_ZPZI_ZERO",  "FSUB_ZPZI_UNDEF",   fpimm_half, fpimm_one, int_aarch64_sve_fsub, AArch64fsub_p>;
+  defm : intrinsic_compact_fp_immediates<"FSUBR_ZPmI", "FSUBR_ZPZI_ZERO", "FSUBR_ZPZI_UNDEF",  fpimm_half, fpimm_one, int_aarch64_sve_fsubr>;
+  defm : intrinsic_compact_fp_immediates<"FMUL_ZPmI",  "FMUL_ZPZI_ZERO",  "FMUL_ZPZI_UNDEF",   fpimm_half, fpimm_two, int_aarch64_sve_fmul, AArch64fmul_p>;
+  defm : intrinsic_compact_fp_immediates<"FMAX_ZPmI",  "FMAX_ZPZI_ZERO",  "FMAX_ZPZI_UNDEF",   fpimm0, fpimm_one, int_aarch64_sve_fmax, AArch64fmax_p>;
+  defm : intrinsic_compact_fp_immediates<"FMIN_ZPmI",  "FMIN_ZPZI_ZERO",  "FMIN_ZPZI_UNDEF",   fpimm0, fpimm_one, int_aarch64_sve_fmin, AArch64fmin_p>;
+  defm : intrinsic_compact_fp_immediates<"FMAXNM_ZPmI","FMAXNM_ZPZI_ZERO","FMAXNM_ZPZI_UNDEF", fpimm0, fpimm_one, int_aarch64_sve_fmaxnm, AArch64fmaxnm_p>;
+  defm : intrinsic_compact_fp_immediates<"FMINNM_ZPmI","FMINNM_ZPZI_ZERO","FMINNM_ZPZI_UNDEF", fpimm0, fpimm_one, int_aarch64_sve_fminnm, AArch64fminnm_p>;
+
+  multiclass intrinsic_compact_fp_immediates_patterns_zeroing<string IZ,
+                    ValueType itype, ValueType ftype, ValueType sftype,
+                    SDPatternOperator op, FPImmLeaf A, int imm> {
+    let AddedComplexity = 2 in {
+      def : Pat<(ftype (op itype:$Pg,
+                          (vselect itype:$Pg, ftype:$Zs1, (SVEDup0)),
+                          (ftype (AArch64dup (sftype A))))),
+                (!cast<Instruction>(IZ) $Pg, $Zs1, imm)>;
+    }
+  }
+
+  multiclass intrinsic_compact_fp_immediates_zeroing<string IZ,
+                    FPImmLeaf A, FPImmLeaf B,
+                    SDPatternOperator op> {
+    defm : intrinsic_compact_fp_immediates_patterns_zeroing<IZ # "_H", nxv8i1, nxv8f16, f16, op, A, 0>;
+    defm : intrinsic_compact_fp_immediates_patterns_zeroing<IZ # "_H", nxv8i1, nxv8f16, f16, op, B, 1>;
+    defm : intrinsic_compact_fp_immediates_patterns_zeroing<IZ # "_S", nxv4i1, nxv4f32, f32, op, A, 0>;
+    defm : intrinsic_compact_fp_immediates_patterns_zeroing<IZ # "_S", nxv4i1, nxv4f32, f32, op, B, 1>;
+    defm : intrinsic_compact_fp_immediates_patterns_zeroing<IZ # "_D", nxv2i1, nxv2f64, f64, op, A, 0>;
+    defm : intrinsic_compact_fp_immediates_patterns_zeroing<IZ # "_D", nxv2i1, nxv2f64, f64, op, B, 1>;
+  }
+
+  let Predicates = [UseExperimentalZeroingPseudos] in {
+    defm : intrinsic_compact_fp_immediates_zeroing<"FADD_ZPZI_ZERO", fpimm_half, fpimm_one, int_aarch64_sve_fadd>;
+    defm : intrinsic_compact_fp_immediates_zeroing<"FSUB_ZPZI_ZERO", fpimm_half, fpimm_one, int_aarch64_sve_fsub>;
+    defm : intrinsic_compact_fp_immediates_zeroing<"FSUBR_ZPZI_ZERO", fpimm_half, fpimm_one, int_aarch64_sve_fsubr>;
+    defm : intrinsic_compact_fp_immediates_zeroing<"FMUL_ZPZI_ZERO", fpimm_half, fpimm_two, int_aarch64_sve_fmul>;
+    defm : intrinsic_compact_fp_immediates_zeroing<"FMAX_ZPZI_ZERO", fpimm0, fpimm_one, int_aarch64_sve_fmax>;
+    defm : intrinsic_compact_fp_immediates_zeroing<"FMIN_ZPZI_ZERO", fpimm0, fpimm_one, int_aarch64_sve_fmin>;
+    defm : intrinsic_compact_fp_immediates_zeroing<"FMAXNM_ZPZI_ZERO", fpimm0, fpimm_one, int_aarch64_sve_fmaxnm>;
+    defm : intrinsic_compact_fp_immediates_zeroing<"FMINNM_ZPZI_ZERO", fpimm0, fpimm_one, int_aarch64_sve_fminnm>;
+  }
+
   // Insert FP scalar into vector with scalar index
   def : Pat<(nxv8f16 (vector_insert (nxv8f16 ZPR:$vec), (f16 FPR16:$src), GPR64:$index)),
             (CPY_ZPmV_H ZPR:$vec,
                         (CMPEQ_PPzZZ_H (PTRUE_H 31),
                                        (INDEX_II_H 0, 1),
                                        (DUP_ZR_H (i32 (EXTRACT_SUBREG GPR64:$index, sub_32)))),
                         $src)>;
   def : Pat<(nxv4f32 (vector_insert (nxv4f32 ZPR:$vec), (f32 FPR32:$src), GPR64:$index)),

llvm/lib/Target/AArch64/SVEInstrFormats.td

 : Pat<(vt (op (pt (AArch64ptrue 31)), vt:$Rn, (vt (AArch64dup (it (cast i32:$imm)))))),
       (inst $Rn, i32:$imm)>;
 
 //
 // Pseudo -> Instruction mappings
 //
 def getSVEPseudoMap : InstrMapping {
   let FilterClass = "SVEPseudo2Instr";
   let RowFields = ["PseudoName"];
   let ColFields = ["IsInstr"];
   let KeyCol = ["0"];
   let ValueCols = [["1"]];
 }
 
 class SVEPseudo2Instr<string name, bit instr> {
   string PseudoName = name;
   bit IsInstr = instr;
: I<(outs zprty:$Zdn), (ins PPR3bAny:$Pg, zprty:$_Zdn, imm_ty:$i1),
   let Inst{5}     = i1;
   let Inst{4-0}   = Zdn;
 
   let Constraints = "$Zdn = $_Zdn";
   let DestructiveInstType = DestructiveOther;
   let ElementSize = zprty.ElementSize;
 }
 
-multiclass sve_fp_2op_i_p_zds<bits<3> opc, string asm, Operand imm_ty> {
-  def _H : sve_fp_2op_i_p_zds<0b01, opc, asm, ZPR16, imm_ty>;
-  def _S : sve_fp_2op_i_p_zds<0b10, opc, asm, ZPR32, imm_ty>;
-  def _D : sve_fp_2op_i_p_zds<0b11, opc, asm, ZPR64, imm_ty>;
+multiclass sve_fp_2op_i_p_zds<bits<3> opc, string asm, string Ps, Operand imm_ty> {
+  let DestructiveInstType = DestructiveBinaryImm in {
+  def _H : SVEPseudo2Instr<Ps # _H, 1>, sve_fp_2op_i_p_zds<0b01, opc, asm, ZPR16, imm_ty>;
+  def _S : SVEPseudo2Instr<Ps # _S, 1>, sve_fp_2op_i_p_zds<0b10, opc, asm, ZPR32, imm_ty>;
+  def _D : SVEPseudo2Instr<Ps # _D, 1>, sve_fp_2op_i_p_zds<0b11, opc, asm, ZPR64, imm_ty>;
+  }
 }
 
 class sve_fp_2op_p_zds<bits<2> sz, bits<4> opc, string asm,
                        ZPRRegOp zprty>
 : I<(outs zprty:$Zdn), (ins PPR3bAny:$Pg, zprty:$_Zdn, zprty:$Zm),
   asm, "\t$Zdn, $Pg/m, $_Zdn, $Zm",
   "",
   []>, Sched<[]> {
def : Pat<(nxv8f16 (op (nxv8f16 ZPR16:$Zn), (nxv8f16 ZPR16:$Zm), (i32 imm32_0_7:$imm))),
             (!cast<Instruction>(NAME # _H) ZPR16:$Zn, ZPR16:$Zm, imm32_0_7:$imm)>;
   def : Pat<(nxv4f32 (op (nxv4f32 ZPR32:$Zn), (nxv4f32 ZPR32:$Zm), (i32 imm32_0_7:$imm))),
             (!cast<Instruction>(NAME # _S) ZPR32:$Zn, ZPR32:$Zm, imm32_0_7:$imm)>;
   def : Pat<(nxv2f64 (op (nxv2f64 ZPR64:$Zn), (nxv2f64 ZPR64:$Zm), (i32 imm32_0_7:$imm))),
             (!cast<Instruction>(NAME # _D) ZPR64:$Zn, ZPR64:$Zm, imm32_0_7:$imm)>;
 }
 
 //===----------------------------------------------------------------------===//
 // SVE Floating Point Arithmetic - Unpredicated Group
 //===----------------------------------------------------------------------===//
 
 class sve_fp_3op_u_zd<bits<2> sz, bits<3> opc, string asm, ZPRRegOp zprty>
 : I<(outs zprty:$Zd), (ins  zprty:$Zn, zprty:$Zm),
   asm, "\t$Zd, $Zn, $Zm",
   "",
   []>, Sched<[]> {
   bits<5> Zd;
   bits<5> Zm;
   bits<5> Zn;
   let Inst{31-24} = 0b01100101;
   let Inst{23-22} = sz;
   let Inst{21}    = 0b0;
   let Inst{20-16} = Zm;
   let Inst{15-13} = 0b000;
   let Inst{12-10} = opc;
   let Inst{9-5}   = Zn;
   let Inst{4-0}   = Zd;
 }
 
 multiclass sve_fp_3op_u_zd<bits<3> opc, string asm, SDPatternOperator op,
                            SDPatternOperator predicated_op = null_frag> {
   def _H : sve_fp_3op_u_zd<0b01, opc, asm, ZPR16>;
   def _S : sve_fp_3op_u_zd<0b10, opc, asm, ZPR32>;
   def _D : sve_fp_3op_u_zd<0b11, opc, asm, ZPR64>;
 
   def : SVE_2_Op_Pat<nxv8f16, op, nxv8f16, nxv8f16, !cast<Instruction>(NAME # _H)>;
   def : SVE_2_Op_Pat<nxv4f32, op, nxv4f32, nxv4f32, !cast<Instruction>(NAME # _S)>;
   def : SVE_2_Op_Pat<nxv2f64, op, nxv2f64, nxv2f64, !cast<Instruction>(NAME # _D)>;
 
   def : SVE_2_Op_Pred_All_Active<nxv8f16, predicated_op, nxv8i1, nxv8f16, nxv8f16, !cast<Instruction>(NAME # _H)>;
   def : SVE_2_Op_Pred_All_Active<nxv4f32, predicated_op, nxv4i1, nxv4f32, nxv4f32, !cast<Instruction>(NAME # _S)>;
   def : SVE_2_Op_Pred_All_Active<nxv2f64, predicated_op, nxv2i1, nxv2f64, nxv2f64, !cast<Instruction>(NAME # _D)>;
 }
 
 multiclass sve_fp_3op_u_zd_ftsmul<bits<3> opc, string asm, SDPatternOperator op> {
   def _H : sve_fp_3op_u_zd<0b01, opc, asm, ZPR16>;
   def _S : sve_fp_3op_u_zd<0b10, opc, asm, ZPR32>;
   def _D : sve_fp_3op_u_zd<0b11, opc, asm, ZPR64>;
 
   def : SVE_2_Op_Pat<nxv8f16, op, nxv8f16, nxv8i16, !cast<Instruction>(NAME # _H)>;
   def : SVE_2_Op_Pat<nxv4f32, op, nxv4f32, nxv4i32, !cast<Instruction>(NAME # _S)>;
   def : SVE_2_Op_Pat<nxv2f64, op, nxv2f64, nxv2i64, !cast<Instruction>(NAME # _D)>;
 }
 
 //===----------------------------------------------------------------------===//
 // SVE Floating Point Fused Multiply-Add Group
multiclass sve_int_shift_pred_bhsd<SDPatternOperator op,
   def _UNDEF_S : PredTwoOpImmPseudo<NAME # _S, ZPR32, Operand<i32>, FalseLanesUndef>;
   def _UNDEF_D : PredTwoOpImmPseudo<NAME # _D, ZPR64, Operand<i32>, FalseLanesUndef>;
 
   def : SVE_Shift_DupImm_Pred_Pat<nxv16i8, op, nxv16i1, i32, imm_b, !cast<Instruction>(NAME # _UNDEF_B)>;
   def : SVE_Shift_DupImm_Pred_Pat<nxv8i16, op, nxv8i1,  i32, imm_h, !cast<Instruction>(NAME # _UNDEF_H)>;
   def : SVE_Shift_DupImm_Pred_Pat<nxv4i32, op, nxv4i1,  i32, imm_s, !cast<Instruction>(NAME # _UNDEF_S)>;
   def : SVE_Shift_DupImm_Pred_Pat<nxv2i64, op, nxv2i1,  i64, imm_d, !cast<Instruction>(NAME # _UNDEF_D)>;
 }
+
+multiclass sve_fp_2op_i_p_zds_zx<Operand imm_ty> {
+  def _UNDEF_H : PredTwoOpImmPseudo<NAME # _H, ZPR16, imm_ty, FalseLanesUndef>;
+  def _UNDEF_S : PredTwoOpImmPseudo<NAME # _S, ZPR32, imm_ty, FalseLanesUndef>;
+  def _UNDEF_D : PredTwoOpImmPseudo<NAME # _D, ZPR64, imm_ty, FalseLanesUndef>;
+}
+
+
+multiclass sve_fp_2op_i_p_zds_zx_zeroing<Operand imm_ty> {
+  def _ZERO_H : PredTwoOpImmPseudo<NAME # _H, ZPR16, imm_ty, FalseLanesZero>;
+  def _ZERO_S : PredTwoOpImmPseudo<NAME # _S, ZPR32, imm_ty, FalseLanesZero>;
+  def _ZERO_D : PredTwoOpImmPseudo<NAME # _D, ZPR64, imm_ty, FalseLanesZero>;
+}
 No newline at end of file

llvm/test/CodeGen/AArch64/sve-fp-immediates-merging.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s | FileCheck %s
+
+target triple = "aarch64-unknown-linux-gnu"
+
+;
+; FADD
+;
+
+define <vscale x 8 x half> @fadd_h_immhalf(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fadd_h_immhalf:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fadd z0.h, p0/m, z0.h, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = fadd <vscale x 8 x half> %a, %splat
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fadd_h_immhalf_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fadd_h_immhalf_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fadd z0.h, p0/m, z0.h, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fadd.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fadd_h_immhalf_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fadd_h_immhalf_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fadd z0.h, p0/m, z0.h, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fadd.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a_z,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fadd_h_immone(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fadd_h_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fadd z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = fadd <vscale x 8 x half> %a, %splat
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fadd_h_immone_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fadd_h_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fadd z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fadd.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fadd_h_immone_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fadd_h_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fadd z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fadd.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a_z,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 4 x float> @fadd_s_immhalf(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fadd_s_immhalf:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fadd z0.s, p0/m, z0.s, #0.5
+; CHECK-NEXT:    ret
+  %elt = insertelement <vscale x 4 x float> undef, float 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = fadd <vscale x 4 x float> %a, %splat
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fadd_s_immhalf_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fadd_s_immhalf_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fadd z0.s, p0/m, z0.s, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fadd.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fadd_s_immhalf_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fadd_s_immhalf_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fadd z0.s, p0/m, z0.s, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fadd.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a_z,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fadd_s_immone(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fadd_s_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fadd z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = fadd <vscale x 4 x float> %a, %splat
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fadd_s_immone_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fadd_s_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fadd z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fadd.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fadd_s_immone_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fadd_s_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fadd z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fadd.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a_z,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 2 x double> @fadd_d_immhalf(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fadd_d_immhalf:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fadd z0.d, p0/m, z0.d, #0.5
+; CHECK-NEXT:    ret
+  %elt = insertelement <vscale x 2 x double> undef, double 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = fadd <vscale x 2 x double> %a, %splat
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fadd_d_immhalf_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fadd_d_immhalf_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fadd z0.d, p0/m, z0.d, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fadd.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fadd_d_immhalf_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fadd_d_immhalf_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fadd z0.d, p0/m, z0.d, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fadd.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a_z,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fadd_d_immone(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fadd_d_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fadd z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = fadd <vscale x 2 x double> %a, %splat
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fadd_d_immone_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fadd_d_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fadd z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fadd.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fadd_d_immone_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fadd_d_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fadd z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fadd.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a_z,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+;
+; FMAX
+;
+
+define <vscale x 8 x half> @fmax_h_immzero(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmax_h_immzero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmax z0.h, p0/m, z0.h, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.maximum.nxv8f16(<vscale x 8 x half> %a, <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmax_h_immzero_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmax_h_immzero_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmax z0.h, p0/m, z0.h, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmax.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmax_h_immzero_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fmax_h_immzero_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fmax z0.h, p0/m, z0.h, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmax.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a_z,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmax_h_immone(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmax_h_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmax z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.maximum.nxv8f16(<vscale x 8 x half> %a, <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmax_h_immone_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmax_h_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmax z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmax.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmax_h_immone_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fmax_h_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fmax z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmax.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a_z,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 4 x float> @fmax_s_immzero(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmax_s_immzero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmax z0.s, p0/m, z0.s, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.maximum.nxv4f32(<vscale x 4 x float> %a, <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmax_s_immzero_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmax_s_immzero_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmax z0.s, p0/m, z0.s, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmax.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmax_s_immzero_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fmax_s_immzero_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fmax z0.s, p0/m, z0.s, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmax.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a_z,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmax_s_immone(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmax_s_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmax z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.maximum.nxv4f32(<vscale x 4 x float> %a, <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmax_s_immone_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmax_s_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmax z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmax.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmax_s_immone_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fmax_s_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fmax z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmax.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a_z,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 2 x double> @fmax_d_immzero(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmax_d_immzero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmax z0.d, p0/m, z0.d, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.maximum.nxv2f64(<vscale x 2 x double> %a, <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmax_d_immzero_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmax_d_immzero_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmax z0.d, p0/m, z0.d, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmax.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmax_d_immzero_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fmax_d_immzero_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fmax z0.d, p0/m, z0.d, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmax.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a_z,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmax_d_immone(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmax_d_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmax z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.maximum.nxv2f64(<vscale x 2 x double> %a, <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmax_d_immone_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmax_d_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmax z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmax.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmax_d_immone_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fmax_d_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fmax z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmax.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a_z,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+;
+; FMAXNM
+;
+
+define <vscale x 8 x half> @fmaxnm_h_immzero(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmaxnm_h_immzero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmaxnm z0.h, p0/m, z0.h, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.maxnum.nxv8f16(<vscale x 8 x half> %a, <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmaxnm_h_immzero_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmaxnm_h_immzero_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmaxnm z0.h, p0/m, z0.h, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmaxnm.nxv8f16(<vscale x 8 x i1> %pg,
+                                                              <vscale x 8 x half> %a,
+                                                              <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmaxnm_h_immzero_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fmaxnm_h_immzero_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fmaxnm z0.h, p0/m, z0.h, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmaxnm.nxv8f16(<vscale x 8 x i1> %pg,
+                                                              <vscale x 8 x half> %a_z,
+                                                              <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmaxnm_h_immone(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmaxnm_h_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmaxnm z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.maxnum.nxv8f16(<vscale x 8 x half> %a, <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmaxnm_h_immone_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmaxnm_h_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmaxnm z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmaxnm.nxv8f16(<vscale x 8 x i1> %pg,
+                                                              <vscale x 8 x half> %a,
+                                                              <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmaxnm_h_immone_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fmaxnm_h_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fmaxnm z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmaxnm.nxv8f16(<vscale x 8 x i1> %pg,
+                                                              <vscale x 8 x half> %a_z,
+                                                              <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 4 x float> @fmaxnm_s_immzero(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmaxnm_s_immzero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmaxnm z0.s, p0/m, z0.s, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.maxnum.nxv4f32(<vscale x 4 x float> %a, <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmaxnm_s_immzero_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmaxnm_s_immzero_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmaxnm z0.s, p0/m, z0.s, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmaxnm.nxv4f32(<vscale x 4 x i1> %pg,
+                                                               <vscale x 4 x float> %a,
+                                                               <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmaxnm_s_immzero_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fmaxnm_s_immzero_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fmaxnm z0.s, p0/m, z0.s, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmaxnm.nxv4f32(<vscale x 4 x i1> %pg,
+                                                               <vscale x 4 x float> %a_z,
+                                                               <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmaxnm_s_immone(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmaxnm_s_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmaxnm z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.maxnum.nxv4f32(<vscale x 4 x float> %a, <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmaxnm_s_immone_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmaxnm_s_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmaxnm z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmaxnm.nxv4f32(<vscale x 4 x i1> %pg,
+                                                               <vscale x 4 x float> %a,
+                                                               <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmaxnm_s_immone_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fmaxnm_s_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fmaxnm z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmaxnm.nxv4f32(<vscale x 4 x i1> %pg,
+                                                               <vscale x 4 x float> %a_z,
+                                                               <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 2 x double> @fmaxnm_d_immzero(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmaxnm_d_immzero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmaxnm z0.d, p0/m, z0.d, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.maxnum.nxv2f64(<vscale x 2 x double> %a, <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmaxnm_d_immzero_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmaxnm_d_immzero_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmaxnm z0.d, p0/m, z0.d, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmaxnm.nxv2f64(<vscale x 2 x i1> %pg,
+                                                                <vscale x 2 x double> %a,
+                                                                <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmaxnm_d_immzero_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fmaxnm_d_immzero_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fmaxnm z0.d, p0/m, z0.d, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmaxnm.nxv2f64(<vscale x 2 x i1> %pg,
+                                                                <vscale x 2 x double> %a_z,
+                                                                <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmaxnm_d_immone(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmaxnm_d_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmaxnm z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.maxnum.nxv2f64(<vscale x 2 x double> %a, <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmaxnm_d_immone_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmaxnm_d_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmaxnm z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmaxnm.nxv2f64(<vscale x 2 x i1> %pg,
+                                                                <vscale x 2 x double> %a,
+                                                                <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmaxnm_d_immone_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fmaxnm_d_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fmaxnm z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmaxnm.nxv2f64(<vscale x 2 x i1> %pg,
+                                                                <vscale x 2 x double> %a_z,
+                                                                <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+;
+; FMIN
+;
+
+define <vscale x 8 x half> @fmin_h_immzero(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmin_h_immzero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmin z0.h, p0/m, z0.h, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.minimum.nxv8f16(<vscale x 8 x half> %a, <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmin_h_immzero_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmin_h_immzero_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmin z0.h, p0/m, z0.h, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmin.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmin_h_immzero_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fmin_h_immzero_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fmin z0.h, p0/m, z0.h, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmin.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a_z,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmin_h_immone(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmin_h_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmin z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.minimum.nxv8f16(<vscale x 8 x half> %a, <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmin_h_immone_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmin_h_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmin z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmin.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmin_h_immone_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fmin_h_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fmin z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmin.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a_z,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 4 x float> @fmin_s_immzero(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmin_s_immzero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmin z0.s, p0/m, z0.s, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.minimum.nxv4f32(<vscale x 4 x float> %a, <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmin_s_immzero_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmin_s_immzero_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmin z0.s, p0/m, z0.s, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmin.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmin_s_immzero_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fmin_s_immzero_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fmin z0.s, p0/m, z0.s, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmin.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a_z,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmin_s_immone(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmin_s_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmin z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.minimum.nxv4f32(<vscale x 4 x float> %a, <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmin_s_immone_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmin_s_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmin z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmin.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmin_s_immone_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fmin_s_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fmin z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmin.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a_z,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 2 x double> @fmin_d_immzero(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmin_d_immzero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmin z0.d, p0/m, z0.d, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.minimum.nxv2f64(<vscale x 2 x double> %a, <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmin_d_immzero_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmin_d_immzero_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmin z0.d, p0/m, z0.d, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmin.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmin_d_immzero_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fmin_d_immzero_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fmin z0.d, p0/m, z0.d, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmin.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a_z,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmin_d_immone(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmin_d_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmin z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.minimum.nxv2f64(<vscale x 2 x double> %a, <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmin_d_immone_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmin_d_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmin z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmin.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmin_d_immone_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fmin_d_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fmin z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmin.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a_z,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+;
+; FMINNM
+;
+
+define <vscale x 8 x half> @fminnm_h_immzero(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fminnm_h_immzero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fminnm z0.h, p0/m, z0.h, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.minnum.nxv8f16(<vscale x 8 x half> %a, <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fminnm_h_immzero_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fminnm_h_immzero_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fminnm z0.h, p0/m, z0.h, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fminnm.nxv8f16(<vscale x 8 x i1> %pg,
+                                                              <vscale x 8 x half> %a,
+                                                              <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fminnm_h_immzero_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fminnm_h_immzero_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fminnm z0.h, p0/m, z0.h, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fminnm.nxv8f16(<vscale x 8 x i1> %pg,
+                                                              <vscale x 8 x half> %a_z,
+                                                              <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fminnm_h_immone(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fminnm_h_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fminnm z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.minnum.nxv8f16(<vscale x 8 x half> %a, <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fminnm_h_immone_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fminnm_h_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fminnm z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fminnm.nxv8f16(<vscale x 8 x i1> %pg,
+                                                              <vscale x 8 x half> %a,
+                                                              <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fminnm_h_immone_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fminnm_h_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fminnm z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fminnm.nxv8f16(<vscale x 8 x i1> %pg,
+                                                              <vscale x 8 x half> %a_z,
+                                                              <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 4 x float> @fminnm_s_immzero(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fminnm_s_immzero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fminnm z0.s, p0/m, z0.s, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.minnum.nxv4f32(<vscale x 4 x float> %a, <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fminnm_s_immzero_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fminnm_s_immzero_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fminnm z0.s, p0/m, z0.s, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fminnm.nxv4f32(<vscale x 4 x i1> %pg,
+                                                               <vscale x 4 x float> %a,
+                                                               <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fminnm_s_immzero_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fminnm_s_immzero_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fminnm z0.s, p0/m, z0.s, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fminnm.nxv4f32(<vscale x 4 x i1> %pg,
+                                                               <vscale x 4 x float> %a_z,
+                                                               <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fminnm_s_immone(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fminnm_s_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fminnm z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.minnum.nxv4f32(<vscale x 4 x float> %a, <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fminnm_s_immone_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fminnm_s_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fminnm z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fminnm.nxv4f32(<vscale x 4 x i1> %pg,
+                                                               <vscale x 4 x float> %a,
+                                                               <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fminnm_s_immone_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fminnm_s_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fminnm z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fminnm.nxv4f32(<vscale x 4 x i1> %pg,
+                                                               <vscale x 4 x float> %a_z,
+                                                               <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 2 x double> @fminnm_d_immzero(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fminnm_d_immzero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fminnm z0.d, p0/m, z0.d, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.minnum.nxv2f64(<vscale x 2 x double> %a, <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fminnm_d_immzero_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fminnm_d_immzero_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fminnm z0.d, p0/m, z0.d, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fminnm.nxv2f64(<vscale x 2 x i1> %pg,
+                                                                <vscale x 2 x double> %a,
+                                                                <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fminnm_d_immzero_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fminnm_d_immzero_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fminnm z0.d, p0/m, z0.d, #0.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fminnm.nxv2f64(<vscale x 2 x i1> %pg,
+                                                                <vscale x 2 x double> %a_z,
+                                                                <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fminnm_d_immone(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fminnm_d_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fminnm z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.minnum.nxv2f64(<vscale x 2 x double> %a, <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fminnm_d_immone_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fminnm_d_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fminnm z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fminnm.nxv2f64(<vscale x 2 x i1> %pg,
+                                                                <vscale x 2 x double> %a,
+                                                                <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fminnm_d_immone_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fminnm_d_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fminnm z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fminnm.nxv2f64(<vscale x 2 x i1> %pg,
+                                                                <vscale x 2 x double> %a_z,
+                                                                <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+;
+; FMUL
+;
+
+define <vscale x 8 x half> @fmul_h_immhalf(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmul_h_immhalf:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmul z0.h, p0/m, z0.h, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = fmul <vscale x 8 x half> %a, %splat
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmul_h_immhalf_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmul_h_immhalf_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmul z0.h, p0/m, z0.h, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmul.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmul_h_immhalf_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fmul_h_immhalf_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fmul z0.h, p0/m, z0.h, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmul.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a_z,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmul_h_immtwo(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmul_h_immtwo:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fadd z0.h, z0.h, z0.h
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 2.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = fmul <vscale x 8 x half> %a, %splat
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmul_h_immtwo_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fmul_h_immtwo_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmul z0.h, p0/m, z0.h, #2.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 2.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmul.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fmul_h_immtwo_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fmul_h_immtwo_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fmul z0.h, p0/m, z0.h, #2.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 2.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fmul.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a_z,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 4 x float> @fmul_s_immhalf(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmul_s_immhalf:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmul z0.s, p0/m, z0.s, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = fmul <vscale x 4 x float> %a, %splat
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmul_s_immhalf_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmul_s_immhalf_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmul z0.s, p0/m, z0.s, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmul.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmul_s_immhalf_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fmul_s_immhalf_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fmul z0.s, p0/m, z0.s, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmul.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a_z,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmul_s_immtwo(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmul_s_immtwo:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fadd z0.s, z0.s, z0.s
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 2.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = fmul <vscale x 4 x float> %a, %splat
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmul_s_immtwo_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fmul_s_immtwo_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmul z0.s, p0/m, z0.s, #2.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 2.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmul.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fmul_s_immtwo_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fmul_s_immtwo_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fmul z0.s, p0/m, z0.s, #2.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 2.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fmul.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a_z,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 2 x double> @fmul_d_immhalf(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmul_d_immhalf:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fmul z0.d, p0/m, z0.d, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = fmul <vscale x 2 x double> %a, %splat
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmul_d_immhalf_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmul_d_immhalf_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmul z0.d, p0/m, z0.d, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmul.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmul_d_immhalf_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fmul_d_immhalf_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fmul z0.d, p0/m, z0.d, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmul.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a_z,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmul_d_immtwo(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmul_d_immtwo:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fadd z0.d, z0.d, z0.d
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 2.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = fmul <vscale x 2 x double> %a, %splat
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmul_d_immtwo_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fmul_d_immtwo_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fmul z0.d, p0/m, z0.d, #2.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 2.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmul.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fmul_d_immtwo_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fmul_d_immtwo_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fmul z0.d, p0/m, z0.d, #2.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 2.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fmul.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a_z,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+;
+; FSUB
+;
+
+define <vscale x 8 x half> @fsub_h_immhalf(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fsub_h_immhalf:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fsub z0.h, p0/m, z0.h, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = fsub <vscale x 8 x half> %a, %splat
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fsub_h_immhalf_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fsub_h_immhalf_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fsub z0.h, p0/m, z0.h, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fsub.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fsub_h_immhalf_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fsub_h_immhalf_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fsub z0.h, p0/m, z0.h, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fsub.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a_z,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fsub_h_immone(<vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fsub_h_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fsub z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = fsub <vscale x 8 x half> %a, %splat
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fsub_h_immone_acle(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #0 {
+; CHECK-LABEL: fsub_h_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fsub z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fsub.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fsub_h_immone_acle_zero(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fsub_h_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fsub z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fsub.nxv8f16(<vscale x 8 x i1> %pg,
+                                                            <vscale x 8 x half> %a_z,
+                                                            <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 4 x float> @fsub_s_immhalf(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fsub_s_immhalf:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fsub z0.s, p0/m, z0.s, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = fsub <vscale x 4 x float> %a, %splat
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fsub_s_immhalf_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fsub_s_immhalf_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fsub z0.s, p0/m, z0.s, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fsub.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fsub_s_immhalf_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fsub_s_immhalf_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fsub z0.s, p0/m, z0.s, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fsub.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a_z,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fsub_s_immone(<vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fsub_s_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fsub z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = fsub <vscale x 4 x float> %a, %splat
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fsub_s_immone_acle(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #0 {
+; CHECK-LABEL: fsub_s_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fsub z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fsub.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fsub_s_immone_acle_zero(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fsub_s_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fsub z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fsub.nxv4f32(<vscale x 4 x i1> %pg,
+                                                             <vscale x 4 x float> %a_z,
+                                                             <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 2 x double> @fsub_d_immhalf(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fsub_d_immhalf:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fsub z0.d, p0/m, z0.d, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = fsub <vscale x 2 x double> %a, %splat
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fsub_d_immhalf_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fsub_d_immhalf_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fsub z0.d, p0/m, z0.d, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fsub.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fsub_d_immhalf_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fsub_d_immhalf_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fsub z0.d, p0/m, z0.d, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fsub.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a_z,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fsub_d_immone(<vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fsub_d_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    fsub z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = fsub <vscale x 2 x double> %a, %splat
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fsub_d_immone_acle(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #0 {
+; CHECK-LABEL: fsub_d_immone_acle:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    fsub z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fsub.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fsub_d_immone_acle_zero(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fsub_d_immone_acle_zero:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fsub z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fsub.nxv2f64(<vscale x 2 x i1> %pg,
+                                                              <vscale x 2 x double> %a_z,
+                                                              <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+;
+; FSUBR
+;
+
+define <vscale x 8 x half> @fsubr_h_immhalf(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fsubr_h_immhalf:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fsubr z0.h, p0/m, z0.h, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fsubr.nxv8f16(<vscale x 8 x i1> %pg,
+                                                             <vscale x 8 x half> %a_z,
+                                                             <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 8 x half> @fsubr_h_immone(<vscale x 8 x i1> %pg, <vscale x 8 x half> %a) #1 {
+; CHECK-LABEL: fsubr_h_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.h, p0/z, z0.h
+; CHECK-NEXT:    fsubr z0.h, p0/m, z0.h, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 8 x half> undef, half 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 8 x half> %elt, <vscale x 8 x half> undef, <vscale x 8 x i32> zeroinitializer
+  %a_z = select <vscale x 8 x i1> %pg, <vscale x 8 x half> %a, <vscale x 8 x half> zeroinitializer
+  %out = call <vscale x 8 x half> @llvm.aarch64.sve.fsubr.nxv8f16(<vscale x 8 x i1> %pg,
+                                                             <vscale x 8 x half> %a_z,
+                                                             <vscale x 8 x half> %splat)
+  ret <vscale x 8 x half> %out
+}
+
+define <vscale x 4 x float> @fsubr_s_immhalf(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fsubr_s_immhalf:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fsubr z0.s, p0/m, z0.s, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fsubr.nxv4f32(<vscale x 4 x i1> %pg,
+                                                              <vscale x 4 x float> %a_z,
+                                                              <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 4 x float> @fsubr_s_immone(<vscale x 4 x i1> %pg, <vscale x 4 x float> %a) #1 {
+; CHECK-LABEL: fsubr_s_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.s, p0/z, z0.s
+; CHECK-NEXT:    fsubr z0.s, p0/m, z0.s, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 4 x float> undef, float 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 4 x float> %elt, <vscale x 4 x float> undef, <vscale x 4 x i32> zeroinitializer
+  %a_z = select <vscale x 4 x i1> %pg, <vscale x 4 x float> %a, <vscale x 4 x float> zeroinitializer
+  %out = call <vscale x 4 x float> @llvm.aarch64.sve.fsubr.nxv4f32(<vscale x 4 x i1> %pg,
+                                                              <vscale x 4 x float> %a_z,
+                                                              <vscale x 4 x float> %splat)
+  ret <vscale x 4 x float> %out
+}
+
+define <vscale x 2 x double> @fsubr_d_immhalf(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fsubr_d_immhalf:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fsubr z0.d, p0/m, z0.d, #0.5
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 0.500000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fsubr.nxv2f64(<vscale x 2 x i1> %pg,
+                                                               <vscale x 2 x double> %a_z,
+                                                               <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+define <vscale x 2 x double> @fsubr_d_immone(<vscale x 2 x i1> %pg, <vscale x 2 x double> %a) #1 {
+; CHECK-LABEL: fsubr_d_immone:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    movprfx z0.d, p0/z, z0.d
+; CHECK-NEXT:    fsubr z0.d, p0/m, z0.d, #1.0
+; CHECK-NEXT:    ret
+  %elt   = insertelement <vscale x 2 x double> undef, double 1.000000e+00, i32 0
+  %splat = shufflevector <vscale x 2 x double> %elt, <vscale x 2 x double> undef, <vscale x 2 x i32> zeroinitializer
+  %a_z = select <vscale x 2 x i1> %pg, <vscale x 2 x double> %a, <vscale x 2 x double> zeroinitializer
+  %out = call <vscale x 2 x double> @llvm.aarch64.sve.fsubr.nxv2f64(<vscale x 2 x i1> %pg,
+                                                               <vscale x 2 x double> %a_z,
+                                                               <vscale x 2 x double> %splat)
+  ret <vscale x 2 x double> %out
+}
+
+;; Arithmetic intrinsic declarations
+
+declare <vscale x 8 x half> @llvm.maximum.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>)
+declare <vscale x 4 x float> @llvm.maximum.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.maximum.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>)
+
+declare <vscale x 8 x half> @llvm.maxnum.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>)
+declare <vscale x 4 x float> @llvm.maxnum.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.maxnum.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>)
+
+declare <vscale x 8 x half> @llvm.minimum.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>)
+declare <vscale x 4 x float> @llvm.minimum.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.minimum.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>)
+
+declare <vscale x 8 x half> @llvm.minnum.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>)
+declare <vscale x 4 x float> @llvm.minnum.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.minnum.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>)
+
+declare <vscale x 8 x half> @llvm.aarch64.sve.fadd.nxv8f16(<vscale x 8 x i1>, <vscale x 8 x half>, <vscale x 8 x half>)
+declare <vscale x 4 x float> @llvm.aarch64.sve.fadd.nxv4f32(<vscale x 4 x i1>, <vscale x 4 x float>, <vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.aarch64.sve.fadd.nxv2f64(<vscale x 2 x i1>, <vscale x 2 x double>, <vscale x 2 x double>)
+
+declare <vscale x 8 x half> @llvm.aarch64.sve.fmax.nxv8f16(<vscale x 8 x i1>, <vscale x 8 x half>, <vscale x 8 x half>)
+declare <vscale x 4 x float> @llvm.aarch64.sve.fmax.nxv4f32(<vscale x 4 x i1>, <vscale x 4 x float>, <vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.aarch64.sve.fmax.nxv2f64(<vscale x 2 x i1>, <vscale x 2 x double>, <vscale x 2 x double>)
+
+declare <vscale x 8 x half> @llvm.aarch64.sve.fmaxnm.nxv8f16(<vscale x 8 x i1>, <vscale x 8 x half>, <vscale x 8 x half>)
+declare <vscale x 4 x float> @llvm.aarch64.sve.fmaxnm.nxv4f32(<vscale x 4 x i1>, <vscale x 4 x float>, <vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.aarch64.sve.fmaxnm.nxv2f64(<vscale x 2 x i1>, <vscale x 2 x double>, <vscale x 2 x double>)
+
+declare <vscale x 8 x half> @llvm.aarch64.sve.fmin.nxv8f16(<vscale x 8 x i1>, <vscale x 8 x half>, <vscale x 8 x half>)
+declare <vscale x 4 x float> @llvm.aarch64.sve.fmin.nxv4f32(<vscale x 4 x i1>, <vscale x 4 x float>, <vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.aarch64.sve.fmin.nxv2f64(<vscale x 2 x i1>, <vscale x 2 x double>, <vscale x 2 x double>)
+
+declare <vscale x 8 x half> @llvm.aarch64.sve.fminnm.nxv8f16(<vscale x 8 x i1>, <vscale x 8 x half>, <vscale x 8 x half>)
+declare <vscale x 4 x float> @llvm.aarch64.sve.fminnm.nxv4f32(<vscale x 4 x i1>, <vscale x 4 x float>, <vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.aarch64.sve.fminnm.nxv2f64(<vscale x 2 x i1>, <vscale x 2 x double>, <vscale x 2 x double>)
+
+declare <vscale x 8 x half> @llvm.aarch64.sve.fmul.nxv8f16(<vscale x 8 x i1>, <vscale x 8 x half>, <vscale x 8 x half>)
+declare <vscale x 4 x float> @llvm.aarch64.sve.fmul.nxv4f32(<vscale x 4 x i1>, <vscale x 4 x float>, <vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.aarch64.sve.fmul.nxv2f64(<vscale x 2 x i1>, <vscale x 2 x double>, <vscale x 2 x double>)
+
+declare <vscale x 8 x half> @llvm.aarch64.sve.fsub.nxv8f16(<vscale x 8 x i1>, <vscale x 8 x half>, <vscale x 8 x half>)
+declare <vscale x 4 x float> @llvm.aarch64.sve.fsub.nxv4f32(<vscale x 4 x i1>, <vscale x 4 x float>, <vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.aarch64.sve.fsub.nxv2f64(<vscale x 2 x i1>, <vscale x 2 x double>, <vscale x 2 x double>)
+
+declare <vscale x 8 x half> @llvm.aarch64.sve.fsubr.nxv8f16(<vscale x 8 x i1>, <vscale x 8 x half>, <vscale x 8 x half>)
+declare <vscale x 4 x float> @llvm.aarch64.sve.fsubr.nxv4f32(<vscale x 4 x i1>, <vscale x 4 x float>, <vscale x 4 x float>)
+declare <vscale x 2 x double> @llvm.aarch64.sve.fsubr.nxv2f64(<vscale x 2 x i1>, <vscale x 2 x double>, <vscale x 2 x double>)
+
+attributes #0 = { "target-features"="+sve" }
+attributes #1 = { "target-features"="+sve,+use-experimental-zeroing-pseudos" }