New intrinsic @llvm.get.active.lane.mask()

[SjoerdMeijer]

Summary

This is split off from D79100 and:

• adds a intrinsic description/definition for @llvm.get.active.lane.mask(), and
• describe its semantics in LangRef.

[simoll]

llvm/docs/LangRef.rst

16174

Since this intrinsic is really only useful (by itself) with scalable types, i guess you should add an example with scalable types here.

16201

I don't like how this is described in terms of the vector hwloop thing. If i am not mistaken, the semantics is simply:

%m[i] = icmp ule (%base + i) <= %n

is equivalent to

%m = @llvm.get.active.lane.mask(%base, %n)

16201

%VIV and %BTC are both integers now (no splat needed).

16217

This example seems a bit convoluted - i understand that this is because this is the code pattern you emit in LV. How about you drop the first two lines and call the intrinsic directly with two unspecified i64 arguments?

[SjoerdMeijer]

Hi Simon, many thanks for the feedback, comments addressed.

[SjoerdMeijer]

Minor change in Examples

[simoll]

llvm/docs/LangRef.rst

16208

You don't need the VIV and back-edge taken count here to describe the semantics of the intrinsic. It makes sense to mention them only in the second paragrah referring to vector hwloops. So, the first paragraph would be about the semantics and the second paragraph below would explain how this is used for hwloops.

16210

To make it clear that overflows cannot occur, you could instead say that the implied %base + i and comparison against %n is performed properly in the integer numbers (and not machine integers).

[SjoerdMeijer]

Agreed, and thanks again, comments addressed.

[simoll]

LGTM. Anybody else?

[efriedma]

llvm/docs/LangRef.rst

16188

The "to accomodate scalable vectors" etc. belongs in the paragraph describing the intended use-case, I think.

llvm/include/llvm/IR/Intrinsics.td

1239

We probably want a verifier check that the result is actually an i1 vector, since we can't express that in TableGen.

[SjoerdMeijer]

Added a Verifier check and test, moved that sentence to the intendend use-case paragraph.

[efriedma]

LGTM

[SjoerdMeijer]

Many thanks for your help with this, and the reviews.

llvm/docs/LangRef.rst

.. code-block:: llvm
 
       %r = call <4 x i32> @llvm.vp.xor.v4i32(<4 x i32> %a, <4 x i32> %b, <4 x i1> %mask, i32 %evl)
       ;; For all lanes below %evl, %r is lane-wise equivalent to %also.r
 
       %t = xor <4 x i32> %a, %b
       %also.r = select <4 x i1> %mask, <4 x i32> %t, <4 x i32> undef
 
 
+.. _int_get_active_lane_mask:
+
+'``llvm.get.active.lane.mask.*``' Intrinsics
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+This is an overloaded intrinsic.
+
+::
+
+      declare <4 x i1> @llvm.get.active.lane.mask.v4i1.i32(i32 %VIV[0], i32 %BTC)
+      declare <8 x i1> @llvm.get.active.lane.mask.v8i1.i64(i64 %VIV[0], i64 %BTC)
+      declare <16 x i1> @llvm.get.active.lane.mask.v16i1.i64.v16i64(i64 %VIV[0], i64 %BTC)
+
+
+Overview:
+"""""""""
+
+Create a mask representing active and inactive vector lanes.
+
+
+Arguments:
+""""""""""
+
+Both operands have the same scalar integer type. The first operand is the first
+element of the Vector Induction Variable (VIV), denoted by ``%VIV[0].`` The second
+operand is the scalar loop Back-edge Taken Count (BTC). The result is a vector
+with the same number of elements as the VIV, but with the i1 element value
+type.
+
+The arguments are scalar types to accomodate scalable vector types, for which
+it is unknown what the the type of the step vector needs to be that enumerate
+its lanes without overflow.
+
+
+Semantics:
+""""""""""
+
+The '``llvm.get.active.lane.mask.*``' intrinsics are semantically equivalent to
+`icmp ule %VIV, (splat %BTC)`, where VIV is the Vector Induction Variable, and
+BTC the back-edge taken count splat into a vector, except when the vector
+induction variable overflows, in which case they return false in the lanes
+where the VIV overflows. Thus, these intrinsics perform an element-wise less
+than or equal comparison of VIV with BTC, producing a mask of true/false values
+representing active/inactive vector lanes. This mask can e.g. be used in the
+masked load/store instructions. These intrinsics provides a hint to the
+backend. I.e., for a vector loop, the back-edge taken count of the original
+scalar loop is explicit as the second argument.
+
+
+Examples:
+"""""""""
+
+.. code-block:: llvm
+
+      %induction = add <4 x i64> %broadcast.splat, <i64 0, i64 1, i64 2, i64 3>
+      %elem0 = extractelement <4 x i32> %induction, i32 0
+      %get.active.lane.mask = call <4 x i1> @llvm.get.active.lane.mask.v4i1.i64(i64 %elem0, i64 429)
+      %wide.masked.load = call <4 x i32> @llvm.masked.load.v4i32.p0v4i32(<4 x i32>* %3, i32 4, <4 x i1> %get.active.lane.mask, <4 x i32> undef)
+
+
 .. _int_mload_mstore:
 
 Masked Vector Load and Store Intrinsics
 ---------------------------------------
 
 LLVM provides intrinsics for predicated vector load and store operations. The predicate is specified by a mask operand, which holds one bit per vector element, switching the associated vector lane on or off. The memory addresses corresponding to the "off" lanes are not accessed. When all bits of the mask are on, the intrinsic is identical to a regular vector load or store. When all bits are off, no memory is accessed.
 
 .. _int_mload:

llvm/include/llvm/IR/Intrinsics.td

let IntrProperties = [IntrNoMem, IntrNoSync, IntrWillReturn] in {
   def int_vp_xor : Intrinsic<[ llvm_anyvector_ty ],
                              [ LLVMMatchType<0>,
                                LLVMMatchType<0>,
                                LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>,
                                llvm_i32_ty]>;
 
 }
 
+def int_get_active_lane_mask:
+  Intrinsic<[llvm_anyvector_ty],
+            [llvm_anyint_ty, LLVMMatchType<1>],
+            [IntrNoMem, IntrNoSync, IntrWillReturn]>;
 
 //===-------------------------- Masked Intrinsics -------------------------===//
 //
 def int_masked_store : Intrinsic<[], [llvm_anyvector_ty,
                                       LLVMAnyPointerType<LLVMMatchType<0>>,
                                       llvm_i32_ty,
                                       LLVMScalarOrSameVectorWidth<0, llvm_i1_ty>],
                                  [IntrArgMemOnly, IntrWillReturn, ImmArg<2>]>;