[BasicTTI] Set scalarization cost of getCommonMaskedMemoryOpCost to Invalid.

[Jim]

Summary

This fixes crashing on auto *VT = cast<FixedVectorType>(DataTy); when
the type is a scalable vector.

Fix https://github.com/llvm/llvm-project/issues/53599.

[sdesmalen]

Can you add a test for this change?

[Jim]

In D119529#3313680, @sdesmalen wrote:

Can you add a test for this change?

The crash only triggered target is given (specify -mtriple=riscv64) and BasicTTIImplBase::getCommonMaskedMemoryOpCost would be called.
If no target given, NoTTIImpl is used as TTI, and TargetTransformInfoImpl.h::getMemoryOpCost simply returns cost 1 without any crash.

I don't have plan to implement hook getMaskedMemoryOpCost for RISCV.
Do you have any suggest for this kind situation?

[sdesmalen]

In D119529#3321740, @Jim wrote:

In D119529#3313680, @sdesmalen wrote:

Can you add a test for this change?

The crash only triggered target is given (specify -mtriple=riscv64) and BasicTTIImplBase::getCommonMaskedMemoryOpCost would be called.
If no target given, NoTTIImpl is used as TTI, and TargetTransformInfoImpl.h::getMemoryOpCost simply returns cost 1 without any crash.

I don't have plan to implement hook getMaskedMemoryOpCost for RISCV.
Do you have any suggest for this kind situation?

This function assumes fixed-width vectors and cannot be used for scalable vectors. I'm not sure what returning 'Invalid' here would really fix, other than the compiler not crashing for a use-case that should not have occurred in the first place, because an overloaded cost function should have been implemented. Whether the compiler falls into the assert from cast<FixedVectorType>, or whether it returns an Invalid cost, in either case you'll need to implement a memory-op-cost function for your target. And because you'll need to implement a cost-function anyway, returning Invalid doesn't really make a difference, because then this code will never be hit. Otherwise, you would have been able to write a test-case for it.

[liaolucy]

This function assumes fixed-width vectors and cannot be used for scalable vectors. I'm not sure what returning 'Invalid' here would really fix, other than the compiler not crashing for a use-case that should not have occurred in the first place, because an overloaded cost function should have been implemented. Whether the compiler falls into the assert from cast<FixedVectorType>, or whether it returns an Invalid cost, in either case you'll need to implement a memory-op-cost function for your target. And because you'll need to implement a cost-function anyway, returning Invalid doesn't really make a difference, because then this code will never be hit. Otherwise, you would have been able to write a test-case for it.

Many salable vectors use this function and crash . Maybe D121677 is a test-case. GetMaskedMemoryOpCost and getGatherScatterOpCost only return getCommonMaskedMemoryOpCost , so each scalable vector call to these functions will crash. However, getMaskedMemoryOpCost and getGatherScatterOpCost can be implemented by each target, not implemented will it crashes. There seems to be a problem.

llvm/include/llvm/CodeGen/BasicTTIImpl.h

static ISD::MemIndexedMode getISDIndexedMode(TTI::MemIndexedMode M) {
     llvm_unreachable("Unexpected MemIndexedMode");
   }
 
   InstructionCost getCommonMaskedMemoryOpCost(unsigned Opcode, Type *DataTy,
                                               Align Alignment,
                                               bool VariableMask,
                                               bool IsGatherScatter,
                                               TTI::TargetCostKind CostKind) {
+    // Cost for scalable type is Invalid, can't assume any num elements.
+    if (isa<ScalableVectorType>(DataTy))
+      return InstructionCost::getInvalid();
+
     auto *VT = cast<FixedVectorType>(DataTy);
     // Assume the target does not have support for gather/scatter operations
     // and provide a rough estimate.
     //
     // First, compute the cost of the individual memory operations.
     InstructionCost AddrExtractCost =
         IsGatherScatter
             ? getVectorInstrCost(Instruction::ExtractElement,