[InstCombine] fold double reverses

[liuz]

Summary

LLVM's auto-vectorizer sometimes generates silly reverse loops, where a value is reversed by a shuffle, followed by some arithmetic, and then get reversed back.

For example, LLVM compiles this code sequence:

do {
  if (*--p == '.') *p = '_';
} while (p != name);

into

%1 = shufflevector <32 x i8> %0, poison, <31, 30, 29, 28, 27, ... 4, 3, 2, 1, 0>
%2 = icmp eq <32 x i8> %1, <46, 46, 46, 46, 46, ... 46, 46, 46, 46, 46>
%3 = shufflevector <32 x i1> %2, poison, <31, 30, 29, 28, 27, ... 4, 3, 2, 1, 0>

Obviously reverse shuffle %1 and %3 are redundant, and the sequence can be optimized into

%1 = icmp eq <32 x i8> %0, <46, 46, 46, 46, 46, ... 46, 46, 46, 46, 46>

This sequence is found in gzip's make_simple_name function.

This patch implements this rule in VectorCombine. Currently it works when %2 is a cmp instruction comparing with an splat constant. I will add other cases if this patch is approved.

The test cases in this patch are validated with Alive2.

This rewrite pattern is found by the Minotaur superoptimizer (https://github.com/minotaur-toolkit/minotaur).

[xbolva00]

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

365

dyn_cast without isa check?

[RKSimon]

Why is the test file called reverse-loop.ll when it doesn't involve loops?

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

361

Why does this need to be a splat? If its constant we can just constant fold with the outer shuffle

llvm/test/Transforms/VectorCombine/reverse-loop.ll

142

Add negative test coverage that checks for length changing shuffles?

[nikic]

As this is not a cost-model driven transform, it should be part of InstCombine, not VectorCombine.

[liuz]

Updated the patch according to comments

[liuz]

In D147373#4238404, @nikic wrote:

As this is not a cost-model driven transform, it should be part of InstCombine, not VectorCombine.

Hi @nikic, thanks for the comment, I moved the code to InstCombine.

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

361

Thanks Simon for the comment, yes this does not need to be a splat, let me update this code to handle any constants.

[liuz]

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

361

Updated the patch to support general form (reverse (cmp (reverse x), y)) -> (cmp x, (reverse y)) and (reverse (cmp y, (reverse x))) -> (cmp (reverse y), x). This is exactly what Simon is suggesting in https://github.com/llvm/llvm-project/issues/48960#issuecomment-981040032 , except that this one takes cmpinst instead of binops, see the updated test cases. I'll add binops and other cases once this patch is approved.

365

fixed. thanks.

llvm/test/Transforms/VectorCombine/reverse-loop.ll

142

added.

[liuz]

In D147373#4238398, @RKSimon wrote:

Why is the test file called reverse-loop.ll when it doesn't involve loops?

Changed the test-cases's name to "vector-double-reverse.ll".

[liuz]

All the new test cases are validated by Alive2.

[liuz]

fixed the failing test case.

[junaire]

llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp

2724 ↗

(On Diff #510279)

FWIW, we have m_ICmp

[liuz]

llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp

2724 ↗

(On Diff #510279)

I avoided m_ICmp because this code will support other type of instructions (Unary, Binary, Intrinsic, etc...) in the future.

[liuz]

ping

[craig.topper]

llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp

2731 ↗

(On Diff #510279)

Can we inline this into the call to CmpInst::Create like we did with CI->getOpcode()?

2736 ↗

(On Diff #510279)

You can do

if ((RevX = dyn_cast<ShuffleVectorInst>(LHS)) &&
    RevX->hasOneUse() && RevX->isReverse())

2751 ↗

(On Diff #510279)

Why do we need to check for Constant here? Builder will constant fold.

2754 ↗

(On Diff #510279)

There's a version of CreateShuffleVector that only takes one Value and creates the Poison vector.

Though I wonder if we should just call Builder.CreateVectorReverse instead of copying the mask.

[liuz]

rebase + solve comments

[liuz]

hey @craig.topper thanks for the comments. I've updated my patch, please kindly have a look again if you have time :) thanks.

[liuz]

rebased

[liuz]

ping

[goldstein.w.n]

llvm/lib/Transforms/InstCombine/InstCombineVectorOps.cpp

2770 ↗

(On Diff #545330)

Is there any different logic you need for binop that doesn't apply for cmp?

[craig.topper]

I was recently wondering if it would make sense for canEvaluateShuffled to detect cases where there's an input shuffle that cancels the starting shuffle.

llvm/lib/Transforms/Vectorize/VectorCombine.cpp

 #include "llvm/ADT/Statistic.h"
 #include "llvm/Analysis/AssumptionCache.h"
 #include "llvm/Analysis/BasicAliasAnalysis.h"
 #include "llvm/Analysis/GlobalsModRef.h"
 #include "llvm/Analysis/Loads.h"
 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/Analysis/ValueTracking.h"
 #include "llvm/Analysis/VectorUtils.h"
+#include "llvm/IR/Constant.h"
 #include "llvm/IR/Dominators.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/IRBuilder.h"
+#include "llvm/IR/InstrTypes.h"
+#include "llvm/IR/Instructions.h"
 #include "llvm/IR/PatternMatch.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/Pass.h"
+#include "llvm/Support/Casting.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Transforms/Utils/Local.h"
 #include "llvm/Transforms/Vectorize.h"
 #include <numeric>
 
 #define DEBUG_TYPE "vector-combine"
 #include "llvm/Transforms/Utils/InstructionWorklist.h"
 
private:
 
   InstructionWorklist Worklist;
 
   // TODO: Direct calls from the top-level "run" loop use a plain "Instruction"
   //       parameter. That should be updated to specific sub-classes because the
   //       run loop was changed to dispatch on opcode.
   bool vectorizeLoadInsert(Instruction &I);
   bool widenSubvectorLoad(Instruction &I);
+  bool foldReverseLoop(Instruction &I);
   ExtractElementInst *getShuffleExtract(ExtractElementInst *Ext0,
                                         ExtractElementInst *Ext1,
                                         unsigned PreferredExtractIndex) const;
   bool isExtractExtractCheap(ExtractElementInst *Ext0, ExtractElementInst *Ext1,
                              const Instruction &I,
                              ExtractElementInst *&ConvertToShuffle,
                              unsigned PreferredExtractIndex);
   void foldExtExtCmp(ExtractElementInst *Ext0, ExtractElementInst *Ext1,
bool VectorCombine::widenSubvectorLoad(Instruction &I) {
   Value *CastedPtr =
       Builder.CreatePointerBitCastOrAddrSpaceCast(SrcPtr, Ty->getPointerTo(AS));
   Value *VecLd = Builder.CreateAlignedLoad(Ty, CastedPtr, Alignment);
   replaceValue(I, *VecLd);
   ++NumVecLoad;
   return true;
 }
 
+// Fold (reverse (icmp (reverse x), c)) -> (icmp x, c), if c is a splat.
+// TODO: (reverse (unaryop (reverse x))) -> (unaryop x)
+// TODO: (reverse (binop (reverse x), c) -> (binop x, c)
+// TODO: (reverse (binop (reverse x), (reverse y))) -> (binop x, y)
+bool VectorCombine::foldReverseLoop(Instruction &I) {
+  ShuffleVectorInst *Rev = cast<ShuffleVectorInst>(&I);
+  if (!Rev->isReverse())
+    return false;
+
+  Instruction *Cmp;
+  if (!match(Rev->getOperand(0), m_OneUse(m_Instruction(Cmp))))
+    return false;
+  if (!isa<CmpInst>(Cmp))
+    return false;
+
+  Value *InnerIst;
+  Constant *C;
+  CmpInst::Predicate Pred;
+  if (!match(Cmp, m_Cmp(Pred, m_OneUse(m_Value(InnerIst)), m_Constant(C))))
+    return false;
+
+  if (!C->getSplatValue())
+    return false;
+
+  if (!isa<ShuffleVectorInst>(InnerIst))
+    return false;
+  ShuffleVectorInst *InnerRev = cast<ShuffleVectorInst>(InnerIst);
+  if (!InnerRev->isReverse())
+    return false;
+
+  Value *NewI = Builder.CreateCmp(Pred, InnerRev->getOperand(0), C);
+  replaceValue(I, *NewI);
+  return true;
+}
+
 /// Determine which, if any, of the inputs should be replaced by a shuffle
 /// followed by extract from a different index.
 ExtractElementInst *VectorCombine::getShuffleExtract(
     ExtractElementInst *Ext0, ExtractElementInst *Ext1,
     unsigned PreferredExtractIndex = InvalidIndex) const {
   auto *Index0C = dyn_cast<ConstantInt>(Ext0->getIndexOperand());
   auto *Index1C = dyn_cast<ConstantInt>(Ext1->getIndexOperand());
   assert(Index0C && Index1C && "Expected constant extract indexes");
auto FoldInst = [this, &MadeChange](Instruction &I) {
     // dispatching to folding functions if there's no chance of matching.
     if (IsFixedVectorType) {
       switch (Opcode) {
       case Instruction::InsertElement:
         MadeChange |= vectorizeLoadInsert(I);
         break;
       case Instruction::ShuffleVector:
         MadeChange |= widenSubvectorLoad(I);
+        MadeChange |= foldReverseLoop(I);
         break;
       case Instruction::Load:
         MadeChange |= scalarizeLoadExtract(I);
         break;
       default:
         break;
       }
     }

llvm/test/Transforms/VectorCombine/reverse-loop.ll

+; RUN: opt -S -passes=vector-combine < %s | FileCheck %s
+
+define <32 x i1> @icmp32_8_eq(<32 x i8> %0) {
+; CHECK-LABEL: @icmp32_8_eq
+; CHECK-NEXT: [[T0:%.*]] = icmp eq <32 x i8> %0, <i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46>
+; CHECK-NEXT: ret <32 x i1> [[T0]]
+  %t1 = shufflevector <32 x i8> %0, <32 x i8> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = icmp eq <32 x i8> %t1, <i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46>
+  %t3 = shufflevector <32 x i1> %t2, <32 x i1> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <32 x i1> %t3
+}
+
+define <8 x i1> @icmp8_8_eq(<8 x i8> %0) {
+; CHECK-LABEL: @icmp8_8_eq
+; CHECK-NEXT: [[T0:%.*]] = icmp eq <8 x i8> %0, <i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46>
+; CHECK-NEXT: ret <8 x i1> [[T0]]
+  %t1 = shufflevector <8 x i8> %0, <8 x i8> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = icmp eq <8 x i8> %t1,  <i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46>
+  %t3 = shufflevector <8 x i1> %t2, <8 x i1> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <8 x i1> %t3
+}
+
+define <32 x i1> @icmp32_8_neq(<32 x i8> %0) {
+; CHECK-LABEL: @icmp32_8_neq
+; CHECK-NEXT: [[T0:%.*]] = icmp ne <32 x i8> %0, <i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46>
+; CHECK-NEXT: ret <32 x i1> [[T0]]
+  %t1 = shufflevector <32 x i8> %0, <32 x i8> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = icmp ne <32 x i8> %t1, <i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46>
+  %t3 = shufflevector <32 x i1> %t2, <32 x i1> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <32 x i1> %t3
+}
+
+define <8 x i1> @icmp8_8_neq(<8 x i8> %0) {
+; CHECK-LABEL: @icmp8_8_neq
+; CHECK-NEXT: [[T0:%.*]] = icmp ne <8 x i8> %0, <i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46>
+; CHECK-NEXT: ret <8 x i1> [[T0]]
+  %t1 = shufflevector <8 x i8> %0, <8 x i8> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = icmp ne <8 x i8> %t1,  <i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46>
+  %t3 = shufflevector <8 x i1> %t2, <8 x i1> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <8 x i1> %t3
+}
+
+define <32 x i1> @icmp32_8_ult(<32 x i8> %0) {
+; CHECK-LABEL: @icmp32_8_ult
+; CHECK-NEXT: [[T0:%.*]] = icmp ult <32 x i8> %0, <i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46>
+; CHECK-NEXT: ret <32 x i1> [[T0]]
+  %t1 = shufflevector <32 x i8> %0, <32 x i8> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = icmp ult <32 x i8> %t1, <i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46>
+  %t3 = shufflevector <32 x i1> %t2, <32 x i1> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <32 x i1> %t3
+}
+
+define <8 x i1> @icmp8_8_ult(<8 x i8> %0) {
+; CHECK-LABEL: @icmp8_8_ult
+; CHECK-NEXT: [[T0:%.*]] = icmp ult <8 x i8> %0, <i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46>
+; CHECK-NEXT: ret <8 x i1> [[T0]]
+  %t1 = shufflevector <8 x i8> %0, <8 x i8> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = icmp ult <8 x i8> %t1,  <i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46, i8 46>
+  %t3 = shufflevector <8 x i1> %t2, <8 x i1> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <8 x i1> %t3
+}
+
+define <32 x i1> @fcmp32_8_oeq(<32 x float> %0) {
+; CHECK-LABEL: @fcmp32_8_oeq
+; CHECK-NEXT: [[T0:%.*]] = fcmp oeq <32 x float> %0, <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+; CHECK-NEXT: ret <32 x i1> [[T0]]
+  %t1 = shufflevector <32 x float> %0, <32 x float> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = fcmp oeq <32 x float> %t1, <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+  %t3 = shufflevector <32 x i1> %t2, <32 x i1> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <32 x i1> %t3
+}
+
+define <8 x i1> @fcmp8_8_oeq(<8 x float> %0) {
+; CHECK-LABEL: @fcmp8_8_oeq
+; CHECK-NEXT: [[T0:%.*]] = fcmp oeq <8 x float> %0, <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+; CHECK-NEXT: ret <8 x i1> [[T0]]
+  %t1 = shufflevector <8 x float> %0, <8 x float> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = fcmp oeq <8 x float> %t1,  <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+  %t3 = shufflevector <8 x i1> %t2, <8 x i1> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <8 x i1> %t3
+}
+
+define <32 x i1> @fcmp32_8_olt(<32 x float> %0) {
+; CHECK-LABEL: @fcmp32_8_olt
+; CHECK-NEXT: [[T0:%.*]] = fcmp olt <32 x float> %0, <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+; CHECK-NEXT: ret <32 x i1> [[T0]]
+  %t1 = shufflevector <32 x float> %0, <32 x float> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = fcmp olt <32 x float> %t1, <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+  %t3 = shufflevector <32 x i1> %t2, <32 x i1> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <32 x i1> %t3
+}
+
+define <8 x i1> @fcmp8_8_olt(<8 x float> %0) {
+; CHECK-LABEL: @fcmp8_8_olt
+; CHECK-NEXT: [[T0:%.*]] = fcmp olt <8 x float> %0, <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+; CHECK-NEXT: ret <8 x i1> [[T0]]
+  %t1 = shufflevector <8 x float> %0, <8 x float> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = fcmp olt <8 x float> %t1,  <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+  %t3 = shufflevector <8 x i1> %t2, <8 x i1> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <8 x i1> %t3
+}
+
+define <32 x i1> @fcmp32_8_uno(<32 x float> %0) {
+; CHECK-LABEL: @fcmp32_8_uno
+; CHECK-NEXT: [[T0:%.*]] = fcmp uno <32 x float> %0, <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+; CHECK-NEXT: ret <32 x i1> [[T0]]
+  %t1 = shufflevector <32 x float> %0, <32 x float> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = fcmp uno <32 x float> %t1, <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+  %t3 = shufflevector <32 x i1> %t2, <32 x i1> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <32 x i1> %t3
+}
+
+define <8 x i1> @fcmp8_8_uno(<8 x float> %0) {
+; CHECK-LABEL: @fcmp8_8_uno
+; CHECK-NEXT: [[T0:%.*]] = fcmp uno <8 x float> %0, <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+; CHECK-NEXT: ret <8 x i1> [[T0]]
+  %t1 = shufflevector <8 x float> %0, <8 x float> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = fcmp uno <8 x float> %t1,  <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+  %t3 = shufflevector <8 x i1> %t2, <8 x i1> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <8 x i1> %t3
+}
+
+define <32 x i1> @fcmp32_8_ult(<32 x float> %0) {
+; CHECK-LABEL: @fcmp32_8_ult
+; CHECK-NEXT: [[T0:%.*]] = fcmp ult <32 x float> %0, <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+; CHECK-NEXT: ret <32 x i1> [[T0]]
+  %t1 = shufflevector <32 x float> %0, <32 x float> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = fcmp ult <32 x float> %t1, <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+  %t3 = shufflevector <32 x i1> %t2, <32 x i1> poison, <32 x i32> <i32 31, i32 30, i32 29, i32 28, i32 27, i32 26, i32 25, i32 24, i32 23, i32 22, i32 21, i32 20, i32 19, i32 18, i32 17, i32 16, i32 15, i32 14, i32 13, i32 12, i32 11, i32 10, i32 9, i32 8, i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <32 x i1> %t3
+}
+
+define <8 x i1> @fcmp8_8_ult(<8 x float> %0) {
+; CHECK-LABEL: @fcmp8_8_ult
+; CHECK-NEXT: [[T0:%.*]] = fcmp ult <8 x float> %0, <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+; CHECK-NEXT: ret <8 x i1> [[T0]]
+  %t1 = shufflevector <8 x float> %0, <8 x float> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  %t2 = fcmp ult <8 x float> %t1,  <float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01, float 4.600000e+01>
+  %t3 = shufflevector <8 x i1> %t2, <8 x i1> poison, <8 x i32> <i32 7, i32 6, i32 5, i32 4, i32 3, i32 2, i32 1, i32 0>
+  ret <8 x i1> %t3
+}