Mark the llvm.vector.insert/extract intrinsics speculatable

[dkreitzer]

Summary

This patch is the follow-up from this discourse topic.

The discourse thread has the explanation for why I believe this change to be correct, and I am hoping reviewers can agree on behalf of AArch64 & RISCV targets.

Another thing I noticed that I'd like to fix in a follow-up patch: The language reference claims that you cannot insert/extract a scalable vector into/from a fixed-width vector, but the verifier doesn't actually check this.

[paulwalker-arm]

I don't see anything contentious here from an AArch64 point of view but please wait for agreement from the RISCV folk.

[frasercrmck]

Sounds fine to me too. It looks like you need to fix https://reviews.llvm.org/harbormaster/unit/view/4419863/ before landing though.

[dkreitzer]

Updated llvmir-intrinsics.mlir to reflect the change in intrinsic attributes.

[frasercrmck]

LGTM

llvm/docs/LangRef.rst

 
 ``idx`` represents the starting element number at which ``subvec`` will be
 inserted. ``idx`` must be a constant multiple of ``subvec``'s known minimum
 vector length. If ``subvec`` is a scalable vector, ``idx`` is first scaled by
 the runtime scaling factor of ``subvec``. The elements of ``vec`` starting at
 ``idx`` are overwritten with ``subvec``. Elements ``idx`` through (``idx`` +
 num_elements(``subvec``) - 1) must be valid ``vec`` indices. If this condition
 cannot be determined statically but is false at runtime, then the result vector
-is undefined.
+is a :ref:`poison value <poisonvalues>`.
 
 
 '``llvm.vector.extract``' Intrinsic
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Syntax:
 """""""
 This is an overloaded intrinsic.

 The ``vec`` is the vector from which we will extract a subvector.
 
 The ``idx`` specifies the starting element number within ``vec`` from which a
 subvector is extracted. ``idx`` must be a constant multiple of the known-minimum
 vector length of the result type. If the result type is a scalable vector,
 ``idx`` is first scaled by the result type's runtime scaling factor. Elements
 ``idx`` through (``idx`` + num_elements(result_type) - 1) must be valid vector
 indices. If this condition cannot be determined statically but is false at
-runtime, then the result vector is undefined. The ``idx`` parameter must be a
-vector index constant type (for most targets this will be an integer pointer
-type).
+runtime, then the result vector is a :ref:`poison value <poisonvalues>`. The
+``idx`` parameter must be a vector index constant type (for most targets this
+will be an integer pointer type).
 
 '``llvm.experimental.vector.reverse``' Intrinsic
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Syntax:
 """""""
 This is an overloaded intrinsic.
 

llvm/include/llvm/IR/Intrinsics.td

def int_experimental_vector_splice : DefaultAttrsIntrinsic<[llvm_anyvector_ty],
                                                            [IntrNoMem, ImmArg<ArgIndex<2>>]>;
 
 //===---------- Intrinsics to query properties of scalable vectors --------===//
 def int_vscale : DefaultAttrsIntrinsic<[llvm_anyint_ty], [], [IntrNoMem]>;
 
 //===---------- Intrinsics to perform subvector insertion/extraction ------===//
 def int_vector_insert : DefaultAttrsIntrinsic<[llvm_anyvector_ty],
                                               [LLVMMatchType<0>, llvm_anyvector_ty, llvm_i64_ty],
-                                              [IntrNoMem, ImmArg<ArgIndex<2>>]>;
+                                              [IntrNoMem, IntrSpeculatable, ImmArg<ArgIndex<2>>]>;
 
 def int_vector_extract : DefaultAttrsIntrinsic<[llvm_anyvector_ty],
                                                [llvm_anyvector_ty, llvm_i64_ty],
-                                               [IntrNoMem, ImmArg<ArgIndex<1>>]>;
+                                               [IntrNoMem, IntrSpeculatable, ImmArg<ArgIndex<1>>]>;
 
 //===----------------- Pointer Authentication Intrinsics ------------------===//
 //
 
 // Sign an unauthenticated pointer using the specified key and discriminator,
 // passed in that order.
 // Returns the first argument, with some known bits replaced with a signature.
 def int_ptrauth_sign : Intrinsic<[llvm_i64_ty],

llvm/test/Transforms/SimplifyCFG/speculate-vector-insert-extract.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt < %s -simplifycfg -S | FileCheck %s
+
+declare <vscale x 4 x i32> @llvm.vector.insert.nxv4i32.v4i32(<vscale x 4 x i32>, <4 x i32>, i64 immarg)
+declare <4 x i32> @llvm.vector.extract.v4i32.nxv4i32(<vscale x 4 x i32>, i64 immarg)
+
+define <vscale x 4 x i32> @speculate_vector_insert(i32 %c, <4 x i32> %v1, <4 x i32> %v2) {
+; CHECK-LABEL: @speculate_vector_insert(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[TOBOOL:%.*]] = icmp eq i32 [[C:%.*]], 0
+; CHECK-NEXT:    [[T1:%.*]] = tail call <vscale x 4 x i32> @llvm.vector.insert.nxv4i32.v4i32(<vscale x 4 x i32> zeroinitializer, <4 x i32> [[V1:%.*]], i64 0)
+; CHECK-NEXT:    [[T2:%.*]] = tail call <vscale x 4 x i32> @llvm.vector.insert.nxv4i32.v4i32(<vscale x 4 x i32> zeroinitializer, <4 x i32> [[V2:%.*]], i64 0)
+; CHECK-NEXT:    [[COND:%.*]] = select i1 [[TOBOOL]], <vscale x 4 x i32> [[T2]], <vscale x 4 x i32> [[T1]]
+; CHECK-NEXT:    ret <vscale x 4 x i32> [[COND]]
+;
+entry:
+  %tobool = icmp eq i32 %c, 0
+  br i1 %tobool, label %cond.else, label %cond.then
+
+cond.then:                                        ; preds = %entry
+  %t1 = tail call <vscale x 4 x i32> @llvm.vector.insert.nxv4i32.v4i32(<vscale x 4 x i32> zeroinitializer, <4 x i32> %v1, i64 0)
+  br label %cond.end
+
+cond.else:                                        ; preds = %entry
+  %t2 = tail call <vscale x 4 x i32> @llvm.vector.insert.nxv4i32.v4i32(<vscale x 4 x i32> zeroinitializer, <4 x i32> %v2, i64 0)
+  br label %cond.end
+
+cond.end:                                         ; preds = %cond.else, %cond.then
+  %cond = phi <vscale x 4 x i32> [ %t1, %cond.then ], [ %t2, %cond.else ]
+  br label %return
+
+return:                                           ; preds = %cond.end
+  ret <vscale x 4 x i32> %cond
+}
+
+define <4 x i32> @speculate_vector_extract(i32 %c, <vscale x 4 x i32> %v1, <vscale x 4 x i32> %v2) {
+; CHECK-LABEL: @speculate_vector_extract(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[TOBOOL:%.*]] = icmp eq i32 [[C:%.*]], 0
+; CHECK-NEXT:    [[T1:%.*]] = tail call <4 x i32> @llvm.vector.extract.v4i32.nxv4i32(<vscale x 4 x i32> [[V1:%.*]], i64 0)
+; CHECK-NEXT:    [[T2:%.*]] = tail call <4 x i32> @llvm.vector.extract.v4i32.nxv4i32(<vscale x 4 x i32> [[V2:%.*]], i64 0)
+; CHECK-NEXT:    [[COND:%.*]] = select i1 [[TOBOOL]], <4 x i32> [[T2]], <4 x i32> [[T1]]
+; CHECK-NEXT:    ret <4 x i32> [[COND]]
+;
+entry:
+  %tobool = icmp eq i32 %c, 0
+  br i1 %tobool, label %cond.else, label %cond.then
+
+cond.then:                                        ; preds = %entry
+  %t1 = tail call <4 x i32> @llvm.vector.extract.v4i32.nxv4i32(<vscale x 4 x i32> %v1, i64 0)
+  br label %cond.end
+
+cond.else:                                        ; preds = %entry
+  %t2 = tail call <4 x i32> @llvm.vector.extract.v4i32.nxv4i32(<vscale x 4 x i32> %v2, i64 0)
+  br label %cond.end
+
+cond.end:                                         ; preds = %cond.else, %cond.then
+  %cond = phi <4 x i32> [ %t1, %cond.then ], [ %t2, %cond.else ]
+  br label %return
+
+return:                                           ; preds = %cond.end
+  ret <4 x i32> %cond
+}

mlir/test/Target/LLVMIR/llvmir-intrinsics.mlir

 // CHECK-DAG: declare <8 x i64> @llvm.vp.zext.v8i64.v8i32(<8 x i32>, <8 x i1>, i32) #2
 // CHECK-DAG: declare <8 x i64> @llvm.vp.sext.v8i64.v8i32(<8 x i32>, <8 x i1>, i32) #2
 // CHECK-DAG: declare <8 x float> @llvm.vp.fptrunc.v8f32.v8f64(<8 x double>, <8 x i1>, i32) #2
 // CHECK-DAG: declare <8 x double> @llvm.vp.fpext.v8f64.v8f32(<8 x float>, <8 x i1>, i32) #2
 // CHECK-DAG: declare <8 x i64> @llvm.vp.fptoui.v8i64.v8f64(<8 x double>, <8 x i1>, i32) #2
 // CHECK-DAG: declare <8 x i64> @llvm.vp.fptosi.v8i64.v8f64(<8 x double>, <8 x i1>, i32) #2
 // CHECK-DAG: declare <8 x i64> @llvm.vp.ptrtoint.v8i64.v8p0(<8 x ptr>, <8 x i1>, i32) #2
 // CHECK-DAG: declare <8 x ptr> @llvm.vp.inttoptr.v8p0.v8i64(<8 x i64>, <8 x i1>, i32) #2
-// CHECK-DAG: declare <vscale x 4 x i32> @llvm.vector.insert.nxv4i32.v8i32(<vscale x 4 x i32>, <8 x i32>, i64 immarg) #2
-// CHECK-DAG: declare <vscale x 4 x i32> @llvm.vector.insert.nxv4i32.v4i32(<vscale x 4 x i32>, <4 x i32>, i64 immarg) #2
-// CHECK-DAG: declare <8 x i32> @llvm.vector.insert.v8i32.v4i32(<8 x i32>, <4 x i32>, i64 immarg) #2
-// CHECK-DAG: declare <8 x i32> @llvm.vector.extract.v8i32.nxv4i32(<vscale x 4 x i32>, i64 immarg) #2
-// CHECK-DAG: declare <4 x i32> @llvm.vector.extract.v4i32.nxv4i32(<vscale x 4 x i32>, i64 immarg) #2
-// CHECK-DAG: declare <2 x i32> @llvm.vector.extract.v2i32.v8i32(<8 x i32>, i64 immarg) #2
+// CHECK-DAG: declare <vscale x 4 x i32> @llvm.vector.insert.nxv4i32.v8i32(<vscale x 4 x i32>, <8 x i32>, i64 immarg) #0
+// CHECK-DAG: declare <vscale x 4 x i32> @llvm.vector.insert.nxv4i32.v4i32(<vscale x 4 x i32>, <4 x i32>, i64 immarg) #0
+// CHECK-DAG: declare <8 x i32> @llvm.vector.insert.v8i32.v4i32(<8 x i32>, <4 x i32>, i64 immarg) #0
+// CHECK-DAG: declare <8 x i32> @llvm.vector.extract.v8i32.nxv4i32(<vscale x 4 x i32>, i64 immarg) #0
+// CHECK-DAG: declare <4 x i32> @llvm.vector.extract.v4i32.nxv4i32(<vscale x 4 x i32>, i64 immarg) #0
+// CHECK-DAG: declare <2 x i32> @llvm.vector.extract.v2i32.v8i32(<8 x i32>, i64 immarg) #0