[TTI] Check that lowered type is floating point before calling isFabsFree

[sbc100]

Summary

Fixes a crash in SLP vectoriser under soft-fp.

In the case of soft-fp (e.g. fp128 under wasm) the result of
getTypeLegalizationCost() can be an integer type even if the input is
floating point (See LegalizeTypeAction::TypeSoftenFloat).

Before calling isFabsFree() (which asserts if given a non-fp
type) we need to check that that result is fp. This is safe since in
fabs is certainly not free in the soft-fp case.

Fixes PR39168

[efriedma]

Testcase?

[sbc100]

I can trigger the same issue by using __float128 types and --target=i686-linux (where fp128 is emulated in software).

[sbc100]

• add test

[sbc100]

This is the smallest test I could create. I would love it to be smaller but it seems hard to tickle this particular path.

[efriedma]

test/Transforms/SLPVectorizer/X86/software-float.ll

6 ↗

(On Diff #168382)

Probably should call FileCheck to make sure the code is actually vectorized the way you expect.

[sbc100]

test/Transforms/SLPVectorizer/X86/software-float.ll

6 ↗

(On Diff #168382)

I actually don't know if it successfully vectorizing. I think we only really care that this doesn't crash here don't we?

[dschuff]

Right but the point is that we don't currently have a way to ensure that the code is doing its job; if the SLP vectorizer stops doing its thing and calling the code that was asserting, then this test won't be testing anything.

[sbc100]

I'm not sure it actually does the vectorization in the end. The bit-code that some out is basically identical.

I get the following stuff in -debug-only=SLP:

SLP: Analyzing blocks in __multc3.
SLP: Trying to vectorize starting at PHIs (1)
SLP: Trying to vectorize a list of length = 2.
SLP: Trying to vectorize starting at PHIs (2)
SLP: Trying to vectorize a list of length = 2.
SLP: Analyzing 2 operations 
SLP: We are able to schedule this bundle.
...
...
SLP: Calculating cost for tree of size 4.
SLP: Adding cost 0 for bundle that starts with   %cmpinf = fcmp oeq fp128 %0, 0xL00000000000000007FFF000000000000.
SLP: Call cost 0 (16-16) for   %0 = tail call fp128 @llvm.fabs.f128(fp128 %a)
SLP: Adding cost 0 for bundle that starts with   %0 = tail call fp128 @llvm.fabs.f128(fp128 %a).
SLP: Adding cost 0 for bundle that starts with fp128 %a.
SLP: Adding cost 0 for bundle that starts with fp128 0xL00000000000000007FFF000000000000.
SLP: #LV: 1 , Looking at   %0 = tail call fp128 @llvm.fabs.f128(fp128 %a)
SLP: Spill Cost = 0.
SLP: Extract Cost = 2.
SLP: Total Cost = 2.
SLP: Trying to vectorize a list of length = 2.
SLP: Trying to vectorize a list of length = 2.

I don't really know how to interpret the debug output, but I don't see any changes made to the bit code.

[arsenm]

test/Transforms/SLPVectorizer/X86/software-float.ll

6 ↗

(On Diff #168382)

No, tests that don't check anything are considered bad form. You can also make this testcase a lot smaller. You may want/need to set the cost threshold to a negative value to force it

[sbc100]

• add filecheck and minimize test

[sbc100]

Thanks for the tip, setting a negative threshold helped, and I was able to reduce the test case too.

[dschuff]

Test looks much nicer now, LGTM

include/llvm/CodeGen/BasicTTIImpl.h

unsigned getIntrinsicInstrCost(
 
     const TargetLoweringBase *TLI = getTLI();
     std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, RetTy);
 
     SmallVector<unsigned, 2> LegalCost;
     SmallVector<unsigned, 2> CustomCost;
     for (unsigned ISD : ISDs) {
       if (TLI->isOperationLegalOrPromote(ISD, LT.second)) {
-        if (IID == Intrinsic::fabs && TLI->isFAbsFree(LT.second)) {
+        if (IID == Intrinsic::fabs && LT.second.isFloatingPoint() &&
+            TLI->isFAbsFree(LT.second)) {
           return 0;
         }
 
         // The operation is legal. Assume it costs 1.
         // If the type is split to multiple registers, assume that there is some
         // overhead to this.
         // TODO: Once we have extract/insert subvector cost we need to use them.
         if (LT.first > 1)