[LegalizeTypes] Factor in vscale_range when widening insert_subvector

[luke]

Summary

Currently when widening operands for insert_subvector nodes, we check
first that the indices are valid by seeing if the subvector is
statically known to be smaller than or equal to the in-place vector.

However if we're inserting a fixed subvector into a scalable vector we rely on
the minimum vector length of the latter. This patch extends the widening logic
to also take into account the minimum vscale from the vscale_range attribute,
so we can handle more scenarios where we know the scalable vector is large
enough to contain the subvector.

Fixes https://github.com/llvm/llvm-project/issues/63437

[craig.topper]

Is it possible for the widened type to exceed the minimum vscale?

[craig.topper]

llvm/test/CodeGen/RISCV/rvv/insert-subvector.ll

498

This test already passes. v3i64 widens to v4i64 which should be detected as less than or equal to nxv4i64.

[luke]

In D153519#4441316, @craig.topper wrote:

Is it possible for the widened type to exceed the minimum vscale?

Yes, but I think that would still fall under the "If this condition cannot be determined statically but is
false at runtime, then the result vector is undefined." part of insert_subvector's definition. Since we currently already lower an insert_subvector of e.g. v8i32 into nxv4i32

llvm/test/CodeGen/RISCV/rvv/insert-subvector.ll

498

Argh, copy and paste error, should be nxv2i64

[luke]

Fix test vector size

[craig.topper]

In D153519#4441673, @luke wrote:

In D153519#4441316, @craig.topper wrote:

Is it possible for the widened type to exceed the minimum vscale?

Yes, but I think that would still fall under the "If this condition cannot be determined statically but is
false at runtime, then the result vector is undefined." part of insert_subvector's definition. Since we currently already lower an insert_subvector of e.g. v8i32 into nxv4i32

What if vscale min is 3(not possible for RISC-V but this is generic code), and the types are nxv1i64 and v3i64. The operation is defined for those types, but when v3i64 gets widened to v4i64 it becomes undefined.

[luke]

In D153519#4441716, @craig.topper wrote:

In D153519#4441673, @luke wrote:

In D153519#4441316, @craig.topper wrote:

Is it possible for the widened type to exceed the minimum vscale?

Yes, but I think that would still fall under the "If this condition cannot be determined statically but is
false at runtime, then the result vector is undefined." part of insert_subvector's definition. Since we currently already lower an insert_subvector of e.g. v8i32 into nxv4i32

What if vscale min is 3(not possible for RISC-V but this is generic code), and the types are nxv1i64 and v3i64. The operation is defined for those types, but when v3i64 gets widened to v4i64 it becomes undefined.

Right, I see now. Could we then lower it as a series of insert_vector_elt?

[craig.topper]

In D153519#4441792, @luke wrote:

In D153519#4441716, @craig.topper wrote:

In D153519#4441673, @luke wrote:

In D153519#4441316, @craig.topper wrote:

Is it possible for the widened type to exceed the minimum vscale?

Yes, but I think that would still fall under the "If this condition cannot be determined statically but is
false at runtime, then the result vector is undefined." part of insert_subvector's definition. Since we currently already lower an insert_subvector of e.g. v8i32 into nxv4i32

What if vscale min is 3(not possible for RISC-V but this is generic code), and the types are nxv1i64 and v3i64. The operation is defined for those types, but when v3i64 gets widened to v4i64 it becomes undefined.

Right, I see now. Could we then lower it as a series of insert_vector_elt?

Yeah I think the ultimate fallback can be a series of insert_vector_elts. For the original failing case, it looks like we have the vscale_range attribute so we could use the minimum vscale to guarantee safety?

[luke]

Rework to check for min vscale

[luke]

Rebase and fix using the unwidened SubVec VT.

Gentle ping, https://github.com/llvm/llvm-project/issues/63437 still seems
to crash on ToT, and this might be good to fix for the next llvm-17 rc

[craig.topper]

LGTM

llvm/lib/CodeGen/SelectionDAG/LegalizeVectorTypes.cpp

 SDValue DAGTypeLegalizer::WidenVecOp_INSERT_SUBVECTOR(SDNode *N) {
   EVT VT = N->getValueType(0);
   SDValue SubVec = N->getOperand(1);
   SDValue InVec = N->getOperand(0);
 
   if (getTypeAction(SubVec.getValueType()) == TargetLowering::TypeWidenVector)
     SubVec = GetWidenedVector(SubVec);
 
-  if (SubVec.getValueType().knownBitsLE(VT) && InVec.isUndef() &&
-      N->getConstantOperandVal(2) == 0)
+  EVT SubVT = SubVec.getValueType();
+
+  // Whether or not all the elements of the widened SubVec will be inserted into
+  // valid indices of VT.
+  bool IndicesValid = false;
+  // If we statically know that VT can fit SubVT, the indices are valid.
+  if (VT.knownBitsGE(SubVT))
+    IndicesValid = true;
+  else if (VT.isScalableVector() && SubVT.isFixedLengthVector()) {
+    // Otherwise, if we're inserting a fixed vector into a scalable vector and
+    // we know the minimum vscale we can work out if it's valid ourselves.
+    Attribute Attr = DAG.getMachineFunction().getFunction().getFnAttribute(
+        Attribute::VScaleRange);
+    if (Attr.isValid()) {
+      unsigned VScaleMin = Attr.getVScaleRangeMin();
+      if (VT.getSizeInBits().getKnownMinValue() * VScaleMin >=
+          SubVT.getFixedSizeInBits())
+        IndicesValid = true;
+    }
+  }
+
+  // We need to make sure that the indices are still valid, otherwise we might
+  // widen what was previously well-defined to something undefined.
+  if (IndicesValid && InVec.isUndef() && N->getConstantOperandVal(2) == 0)
     return DAG.getNode(ISD::INSERT_SUBVECTOR, SDLoc(N), VT, InVec, SubVec,
                        N->getOperand(2));
 
   report_fatal_error("Don't know how to widen the operands for "
                      "INSERT_SUBVECTOR");
 }
 
 SDValue DAGTypeLegalizer::WidenVecOp_EXTRACT_SUBVECTOR(SDNode *N) {

llvm/test/CodeGen/RISCV/rvv/insert-subvector.ll

 ; CHECK-NEXT:    add a0, a0, a1
 ; CHECK-NEXT:    vs8r.v v8, (a0)
 ; CHECK-NEXT:    ret
   %v = call <vscale x 16 x i64> @llvm.vector.insert.nxv8i64.nxv16i64(<vscale x 16 x i64> undef, <vscale x 8 x i64> %sv0, i64 8)
   store <vscale x 16 x i64> %v, <vscale x 16 x i64>* %out
   ret void
 }
 
+; We should be able to widen the <3 x i64> subvector to a <4 x i64> here because
+; we know that the minimum vscale is 2
+define <vscale x 2 x i64> @insert_nxv2i64_nxv3i64(<3 x i64> %sv) #0 {
+; CHECK-LABEL: insert_nxv2i64_nxv3i64:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    ret
+  %vec = call <vscale x 2 x i64> @llvm.vector.insert.nxv2i64.v3i64(<vscale x 2 x i64> undef, <3 x i64> %sv, i64 0)
+  ret <vscale x 2 x i64> %vec
+}
+
+attributes #0 = { vscale_range(2,1024) }
+
 declare <vscale x 4 x i1> @llvm.vector.insert.nxv1i1.nxv4i1(<vscale x 4 x i1>, <vscale x 1 x i1>, i64)
 declare <vscale x 32 x i1> @llvm.vector.insert.nxv8i1.nxv32i1(<vscale x 32 x i1>, <vscale x 8 x i1>, i64)
 
 declare <vscale x 16 x i8> @llvm.vector.insert.nxv1i8.nxv16i8(<vscale x 16 x i8>, <vscale x 1 x i8>, i64)
 
 declare <vscale x 32 x half> @llvm.vector.insert.nxv1f16.nxv32f16(<vscale x 32 x half>, <vscale x 1 x half>, i64)
 declare <vscale x 32 x half> @llvm.vector.insert.nxv2f16.nxv32f16(<vscale x 32 x half>, <vscale x 2 x half>, i64)
 
 declare <vscale x 4 x i8> @llvm.vector.insert.nxv1i8.nxv4i8(<vscale x 4 x i8>, <vscale x 1 x i8>, i64 %idx)
 
 declare <vscale x 8 x i32> @llvm.vector.insert.nxv2i32.nxv8i32(<vscale x 8 x i32>, <vscale x 2 x i32>, i64 %idx)
 declare <vscale x 8 x i32> @llvm.vector.insert.nxv4i32.nxv8i32(<vscale x 8 x i32>, <vscale x 4 x i32>, i64 %idx)
 
 declare <vscale x 16 x i32> @llvm.vector.insert.nxv1i32.nxv16i32(<vscale x 16 x i32>, <vscale x 1 x i32>, i64 %idx)
 declare <vscale x 16 x i32> @llvm.vector.insert.nxv2i32.nxv16i32(<vscale x 16 x i32>, <vscale x 2 x i32>, i64 %idx)
 declare <vscale x 16 x i32> @llvm.vector.insert.nxv4i32.nxv16i32(<vscale x 16 x i32>, <vscale x 4 x i32>, i64 %idx)
 declare <vscale x 16 x i32> @llvm.vector.insert.nxv8i32.nxv16i32(<vscale x 16 x i32>, <vscale x 8 x i32>, i64 %idx)
+
+declare <vscale x 2 x i64> @llvm.vector.insert.nxv2i64.v3i64(<vscale x 2 x i64>, <3 x i64>, i64 %idx)