This is an archive of the discontinued LLVM Phabricator instance.

Differential D150143

[X86] Add X86FixupVectorConstantsPass to fold vectors constant loads as broadcasts (WIP)
AbandonedPublic

Authored by RKSimon on May 8 2023, 2:05 PM.

Details

Reviewers
pengfei
goldstein.w.n
craig.topper
andreadb
Summary

This is WIP patch to remove the broadcasting of constants from the DAG and to instead perform this in a later pass, I'd like to hear people's thoughts on the approach while its still in the early stages.

The principal aim is to prevent the premature creation of broadcasts that prevent us folding the loads with another instruction, helping to reduce register pressure.

There's still a lot to be addressed in this early patch including:

  • Subvector Broadcast handling (VBROADCASTF128 etc.).
  • Folding of AVX512 constant loads (including masked loads) to AVX512 broadcasts.
  • Folding of AVX512 instruction with folded constant loads to folded broadcasts.
  • Better use of AVX (fp broadcasts) and SSE3 (movddup) broadcast instructions - the comment printout are a mess of float / integer which we might want to address first?
  • Use of VPMOVZ/VPMOVSX extension load for non-uniform constants that are representable with smaller integers
  • Remove the constant support entirely from lowerBuildVectorAsBroadcast in DAG

Diff Detail

Unit TestsFailed

TimeTest
3,730 msx64 debian > AddressSanitizer-x86_64-linux-dynamic.TestCases/Linux::auto_memory_profile_test.cpp
4,270 msx64 debian > AddressSanitizer-x86_64-linux.TestCases/Linux::auto_memory_profile_test.cpp

Event Timeline

RKSimon created this revision.May 8 2023, 2:05 PM
Herald added a project: Restricted Project. · View Herald TranscriptMay 8 2023, 2:05 PM
Herald added a subscriber: hiraditya. · View Herald Transcript
RKSimon requested review of this revision.May 8 2023, 2:05 PM
Herald added a project: Restricted Project. · View Herald TranscriptMay 8 2023, 2:05 PM
RKSimon updated this revision to Diff 520480.May 8 2023, 2:07 PM

Add as much context as possible without exceeding phab's limit

Harbormaster completed remote builds in B230702: Diff 520480.May 8 2023, 3:18 PM
goldstein.w.n added a comment.EditedMay 8 2023, 4:55 PM

The principal aim is to prevent the premature creation of broadcasts that prevent us folding the loads with another instruction, helping to reduce register pressure.

There are two other problems we have with constant generation that it might also be nice to address with a new pass but that I don't think this approach will be
able to handle.

  1. There are a variety of cases where we could get "better" constants by re-ordering instructions which is very difficult to do at any stage in the current DAG lowering process without creating an infinite loop, but is also something that would be very difficult to do after lowering at the machineinstruction level. I tried to do this for shl; and in D141653 but it ran into an infinite loop when I did more robust testing, and we could imagine also doing it for vector-compares, add/sub, etc...
  1. There are constants like splat(1), splat(mask), splat(-mask), etc... which can often be preferable to build without a load at all (abs(ALL_ONES), shr(ALL_ONES), shl(ALL_ONES), etc...), especially if the fairly common ALL_ONES node dominates.

I think both of these goal lend themselves to a pass before DAG lowering has been completed, but after the common transforms/optimizations.
I was thinking a pass in CodeGenAndEmitDAG after the fourth post-legalization Combine but before DoInstructionSelection would make the
most sense for fixing up constant generation (including the aims of this patch).

pengfei added inline comments.May 8 2023, 9:12 PM
llvm/lib/Target/X86/X86FixupVectorConstants.cpp
208

What's the reason to check hasInt256 rather than hasAVX2?

211

Should avoid FP instruction for integer?

227

Why don't use VBROADCASTSD here?

llvm/test/CodeGen/X86/combine-concatvectors.ll
67–68

Regression?

llvm/test/CodeGen/X86/horizontal-reduce-umin.ll
1170–1171

This looks like regression too.

junaire added a subscriber: junaire.May 9 2023, 12:00 AM
RKSimon planned changes to this revision.May 14 2023, 7:07 AM
In D150143#4328111, @goldstein.w.n wrote:

The principal aim is to prevent the premature creation of broadcasts that prevent us folding the loads with another instruction, helping to reduce register pressure.

There are two other problems we have with constant generation that it might also be nice to address with a new pass but that I don't think this approach will be
able to handle.

  1. There are a variety of cases where we could get "better" constants by re-ordering instructions which is very difficult to do at any stage in the current DAG lowering process without creating an infinite loop, but is also something that would be very difficult to do after lowering at the machineinstruction level. I tried to do this for shl; and in D141653 but it ran into an infinite loop when I did more robust testing, and we could imagine also doing it for vector-compares, add/sub, etc...
  1. There are constants like splat(1), splat(mask), splat(-mask), etc... which can often be preferable to build without a load at all (abs(ALL_ONES), shr(ALL_ONES), shl(ALL_ONES), etc...), especially if the fairly common ALL_ONES node dominates.

I think both of these goal lend themselves to a pass before DAG lowering has been completed, but after the common transforms/optimizations.
I was thinking a pass in CodeGenAndEmitDAG after the fourth post-legalization Combine but before DoInstructionSelection would make the
most sense for fixing up constant generation (including the aims of this patch).

I think we agree that broadcasting the constants in build vector lowering is premature and working on full width constant data is going to be a lot easier.

I have been investigating constant rematerialization as part of this patch and its looking like it should be handled separately (maybe as part of better general instruction rematerialization handling) - your proposal to investigate fitting rematerialization into CodeGenAndEmitDAG makes sense to me.

But I'm seeing most problems with premature constant broadcasting regarding folding of the constants to stack and hoisting, so a great deal of these cases are going to need to be addressed much later than DAG, I think I have this proposed pass in the correct location for this, but I will drop the proposal for it to handle constant rematerialization, this should be a true fixup for poorly lowered vector constant loads, missed AVX512 broadcast folds, etc.

I'll split off another patch.

llvm/lib/Target/X86/X86FixupVectorConstants.cpp
208

There's actually very little consistency between the 2 checks (which are identical), I try to use hasInt256 when I'm identifying cases where we're selecting 256-bit ops between AVX1 and AVX2 for 256-bit integer ops - I can change it to hasAVX2().

211

Interestingly, DAG already uses MOVDDUP in some cases to broadcast integer, but I agree we should avoid it in most cases.

227

xmm VBROADCASTSD doesn't exist - it would have exactly the same behaviour as VMOVDDUP

llvm/test/CodeGen/X86/combine-concatvectors.ll
67–68

Yes, we previously reused the ymm0 splat to avoid a second load

llvm/test/CodeGen/X86/horizontal-reduce-umin.ll
1170–1171

Yes, we have some peepholes in DAG to reuse broadcasts of different widths - I'll address this in a later iteration.

RKSimon edited the summary of this revision. (Show Details)May 14 2023, 7:09 AM
RKSimon mentioned this in D150526: [X86] Add X86FixupVectorConstantsPass to re-fold AVX512 vector load folds as broadcast folds.May 14 2023, 8:20 AM
Matt added a subscriber: Matt.May 22 2023, 2:31 PM
RKSimon mentioned this in rG0b91de5ea32d: [X86] Add X86FixupVectorConstantsPass to re-fold AVX512 vector load folds as….May 23 2023, 3:01 AM
RKSimon mentioned this in rG0f8e0f422880: [X86] lowerBuildVectorAsBroadcast - broadcast Constant of original….May 27 2023, 6:30 AM
RKSimon abandoned this revision.Nov 2 2023, 9:34 AM
Herald added subscribers: wangpc, sunshaoce. · View Herald TranscriptNov 2 2023, 9:34 AM

Large Diff

This large diff affects 255 files. Files without inline comments have been collapsed. Expand All Files

Revision Contents

PathSize
llvm/
lib/
Target/
X86/
1 line
6 lines
255 lines
2 lines
1 line
test/
CodeGen/
X86/
10 lines
4 lines
4 lines
9 lines
2 lines
127 lines
4 lines
68 lines
16 lines
6 lines
18 lines
4 lines
38 lines
18 lines
6 lines
3 lines
21 lines
35 lines
6 lines
63 lines
12 lines
803 lines
4 lines
24 lines
3 lines
10 lines
45 lines
3 lines
18 lines
54 lines
15 lines
38 lines
3 lines
15 lines
57 lines
27 lines
44 lines
8 lines
66 lines
9 lines
3 lines
18 lines
28 lines
17 lines
2 lines
9 lines
6 lines
12 lines
38 lines
1 line
11 lines
131 lines
1 line
9 lines
42 lines
60 lines
3 lines
8 lines
6 lines
25 lines
6 lines
4 lines
4 lines
37 lines
18 lines
18 lines
47 lines
47 lines
6 lines
60 lines
5 lines
28 lines
8 lines
40 lines
3 lines
10 lines
288 lines
445 lines
16 lines
4 lines
528 lines
55 lines
130 lines
18 lines
31 lines
1 line
26 lines
24 lines
56 lines
2 lines
2 lines
20 lines
5 lines
48 lines
54 lines
172 lines
66 lines
33 lines
12 lines
29 lines
6 lines
5 lines
4 lines
6 lines
6 lines
2 lines
27 lines
171 lines
4 lines
182 lines
187 lines
34 lines
53 lines
3 lines
68 lines
56 lines
4 lines
15 lines
33 lines
54 lines
45 lines
42 lines
6 lines
21 lines
31 lines
8 lines
9 lines
50 lines
44 lines
300 lines
5 lines
41 lines
4 lines
8 lines
71 lines
8 lines
8 lines
136 lines
204 lines
8 lines
6 lines
4 lines
37 lines
188 lines
232 lines
14 lines
37 lines
264 lines
20 lines
10 lines
74 lines
164 lines
24 lines
16 lines
45 lines
11 lines
6 lines
3 lines
287 lines
32 lines
20 lines
86 lines
17 lines
6 lines
3 lines
56 lines
62 lines
28 lines
48 lines
54 lines
18 lines
28 lines
929 lines
90 lines
1232 lines
372 lines
1473 lines
5014 lines
46 lines
12 lines
385 lines
19 lines
2844 lines
24 lines
6 lines
6 lines
12 lines
649 lines
655 lines
36 lines
9 lines
9 lines
102 lines
1197 lines
24 lines
18 lines
12 lines
3 lines
999 lines
59 lines
834 lines
10 lines
596 lines
10 lines
14 lines
63 lines
63 lines
43 lines
57 lines
57 lines
2 lines
36 lines
11 lines
6 lines
6 lines
114 lines
34 lines
8 lines
87 lines
9 lines
6 lines
6 lines
6 lines
144 lines
19 lines
29 lines
27 lines
4 lines
49 lines
52 lines
193 lines
197 lines
301 lines
38 lines
334 lines
44 lines
12 lines
118 lines
3 lines
880 lines
16 lines
16 lines
2 lines
2 lines
6 lines
14 lines
109 lines
12 lines
10 lines
10 lines

Diff 520480

llvm/lib/Target/X86/CMakeLists.txt

llvm/lib/Target/X86/X86.h

llvm/lib/Target/X86/X86FixupVectorConstants.cpp

  • This file was added.
//===-- X86FixupVectorConstants.cpp - optimize constant generation -------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
//
// This file examines all full size vector constant pool loads and attempts to
// replace them with smaller constant pool entries, including folding AVX512
// broadcast-able instructions (TODO), broadcasting and extending (TODO) loads.
//
//===----------------------------------------------------------------------===//
#include "X86.h"
#include "X86InstrInfo.h"
#include "X86Subtarget.h"
#include "llvm/ADT/Statistic.h"
#include "llvm/CodeGen/MachineConstantPool.h"
#include "llvm/CodeGen/MachineFunctionPass.h"
#include "llvm/CodeGen/MachineInstrBuilder.h"
#include "llvm/CodeGen/MachineRegisterInfo.h"
using namespace llvm;
#define DEBUG_TYPE "x86-fixup-vector-constants"
STATISTIC(NumInstChanges, "Number of instructions changes");
namespace {
class X86FixupVectorConstantsPass : public MachineFunctionPass {
public:
static char ID;
X86FixupVectorConstantsPass() : MachineFunctionPass(ID) {}
StringRef getPassName() const override {
return "X86 Fixup Vector Constants";
}
bool runOnMachineFunction(MachineFunction &MF) override;
bool processInstruction(MachineFunction &MF, MachineBasicBlock &MBB,
MachineBasicBlock::iterator &I);
// This pass runs after regalloc and doesn't support VReg operands.
MachineFunctionProperties getRequiredProperties() const override {
return MachineFunctionProperties().set(
MachineFunctionProperties::Property::NoVRegs);
}
private:
const X86InstrInfo *TII = nullptr;
const X86Subtarget *ST = nullptr;
const MCSchedModel *SM = nullptr;
};
} // end anonymous namespace
char X86FixupVectorConstantsPass::ID = 0;
INITIALIZE_PASS(X86FixupVectorConstantsPass, DEBUG_TYPE, DEBUG_TYPE, false, false)
FunctionPass *llvm::createX86FixupVectorConstants() {
return new X86FixupVectorConstantsPass();
}
static const Constant *getConstantFromPool(const MachineInstr &MI,
const MachineOperand &Op) {
if (!Op.isCPI() || Op.getOffset() != 0)
return nullptr;
ArrayRef<MachineConstantPoolEntry> Constants =
MI.getParent()->getParent()->getConstantPool()->getConstants();
const MachineConstantPoolEntry &ConstantEntry = Constants[Op.getIndex()];
// Bail if this is a machine constant pool entry, we won't be able to dig out
// anything useful.
if (ConstantEntry.isMachineConstantPoolEntry())
return nullptr;
return ConstantEntry.Val.ConstVal;
}
std::optional<APInt> getConstantBits(const Constant *C) {
unsigned NumBits = C->getType()->getPrimitiveSizeInBits();
if (auto *CInt = dyn_cast<ConstantInt>(C))
return CInt->getValue();
if (auto *CFP = dyn_cast<ConstantFP>(C))
return CFP->getValue().bitcastToAPInt();
if (auto *CV = dyn_cast<ConstantVector>(C))
if (auto *CVSplat = CV->getSplatValue(/*AllowUndefs*/ true))
if (std::optional<APInt> Bits = getConstantBits(CVSplat)) {
assert((NumBits % Bits->getBitWidth()) == 0 && "Illegal splat");
return APInt::getSplat(NumBits, *Bits);
}
if (auto *CDS = dyn_cast<ConstantDataSequential>(C)) {
bool IsInteger = CDS->getElementType()->isIntegerTy();
bool IsFloat = CDS->getElementType()->isHalfTy() ||
CDS->getElementType()->isFloatTy() ||
CDS->getElementType()->isDoubleTy();
if (IsInteger || IsFloat) {
APInt Bits = APInt::getZero(NumBits);
unsigned EltBits = CDS->getElementType()->getPrimitiveSizeInBits();
for (unsigned I = 0, E = CDS->getNumElements(); I != E; ++I) {
if (IsInteger)
Bits.insertBits(CDS->getElementAsAPInt(I), I * EltBits);
else
Bits.insertBits(CDS->getElementAsAPFloat(I).bitcastToAPInt(),
I * EltBits);
}
return Bits;
}
}
return std::nullopt;
}
bool X86FixupVectorConstantsPass::processInstruction(
MachineFunction &MF, MachineBasicBlock &MBB,
MachineBasicBlock::iterator &I) {
MachineInstr &MI = *I;
unsigned Opc = MI.getOpcode();
MachineConstantPool *CP = MI.getParent()->getParent()->getConstantPool();
auto ConvertToBroadcastInt = [&](unsigned OpBcst64, unsigned OpBcst32,
unsigned OpBcst16, unsigned OpBcst8) {
assert(MI.getNumOperands() >= (1 + X86::AddrNumOperands) &&
"Unexpected number of operands!");
MachineOperand &CstOp = MI.getOperand(1 + X86::AddrDisp);
if (auto *C = getConstantFromPool(MI, CstOp)) {
if (std::optional<APInt> Bits = getConstantBits(C)) {
if (OpBcst8 && Bits->isSplat(8)) {
Type *CstTy = Type::getInt8Ty(C->getType()->getContext());
Constant *NewCst = Constant::getIntegerValue(CstTy, Bits->trunc(8));
unsigned NewCPIdx = CP->getConstantPoolIndex(NewCst, Align(1));
MI.setDesc(TII->get(OpBcst8));
CstOp.setIndex(NewCPIdx);
return true;
}
if (OpBcst16 && Bits->isSplat(16)) {
Type *CstTy = Type::getInt16Ty(C->getType()->getContext());
Constant *NewCst = Constant::getIntegerValue(CstTy, Bits->trunc(16));
unsigned NewCPIdx = CP->getConstantPoolIndex(NewCst, Align(2));
MI.setDesc(TII->get(OpBcst16));
CstOp.setIndex(NewCPIdx);
return true;
}
if (OpBcst32 && Bits->isSplat(32)) {
Type *CstTy = Type::getInt32Ty(C->getType()->getContext());
Constant *NewCst = Constant::getIntegerValue(CstTy, Bits->trunc(32));
unsigned NewCPIdx = CP->getConstantPoolIndex(NewCst, Align(4));
MI.setDesc(TII->get(OpBcst32));
CstOp.setIndex(NewCPIdx);
return true;
}
if (OpBcst64 && Bits->isSplat(64)) {
Type *CstTy = Type::getInt64Ty(C->getType()->getContext());
Constant *NewCst = Constant::getIntegerValue(CstTy, Bits->trunc(64));
unsigned NewCPIdx = CP->getConstantPoolIndex(NewCst, Align(8));
MI.setDesc(TII->get(OpBcst64));
CstOp.setIndex(NewCPIdx);
return true;
}
return false;
}
}
return false;
};
auto ConvertToBroadcastFP = [&](unsigned OpBcst64, unsigned OpBcst32) {
assert(MI.getNumOperands() >= (1 + X86::AddrNumOperands) &&
"Unexpected number of operands!");
MachineOperand &CstOp = MI.getOperand(1 + X86::AddrDisp);
if (auto *C = getConstantFromPool(MI, CstOp)) {
if (std::optional<APInt> Bits = getConstantBits(C)) {
if (OpBcst32 && Bits->isSplat(32)) {
APFloat FBits(APFloat::IEEEsingle(), Bits->trunc(32));
Constant *NewCst = ConstantFP::get(C->getType()->getContext(), FBits);
unsigned NewCPIdx = CP->getConstantPoolIndex(NewCst, Align(4));
MI.setDesc(TII->get(OpBcst32));
CstOp.setIndex(NewCPIdx);
return true;
}
if (OpBcst64 && Bits->isSplat(64)) {
APFloat FBits(APFloat::IEEEdouble(), Bits->trunc(64));
Constant *NewCst = ConstantFP::get(C->getType()->getContext(), FBits);
unsigned NewCPIdx = CP->getConstantPoolIndex(NewCst, Align(8));
MI.setDesc(TII->get(OpBcst64));
CstOp.setIndex(NewCPIdx);
return true;
}
return false;
}
}
return false;
};
switch (Opc) {
case X86::VMOVDQArm:
case X86::VMOVDQUrm:
if (ST->hasInt256())
pengfeiUnsubmitted
Not Done
ReplyInline Actions

What's the reason to check hasInt256 rather than hasAVX2?

pengfei: What's the reason to check `hasInt256` rather than `hasAVX2`?
RKSimonAuthorUnsubmitted
Not Done
ReplyInline Actions

There's actually very little consistency between the 2 checks (which are identical), I try to use hasInt256 when I'm identifying cases where we're selecting 256-bit ops between AVX1 and AVX2 for 256-bit integer ops - I can change it to hasAVX2().

RKSimon: There's actually very little consistency between the 2 checks (which are identical), I try to…
return ConvertToBroadcastInt(X86::VPBROADCASTQrm, X86::VPBROADCASTDrm,
X86::VPBROADCASTWrm, X86::VPBROADCASTBrm);
return false; // TODO: Add AVX VMOVDDUP/VBROADCASTSS.
pengfeiUnsubmitted
Not Done
ReplyInline Actions

Should avoid FP instruction for integer?

pengfei: Should avoid FP instruction for integer?
RKSimonAuthorUnsubmitted
Done
ReplyInline Actions

Interestingly, DAG already uses MOVDDUP in some cases to broadcast integer, but I agree we should avoid it in most cases.

RKSimon: Interestingly, DAG already uses MOVDDUP in some cases to broadcast integer, but I agree we…
case X86::VMOVDQAYrm:
case X86::VMOVDQUYrm:
if (ST->hasInt256())
return ConvertToBroadcastInt(X86::VPBROADCASTQYrm, X86::VPBROADCASTDYrm,
X86::VPBROADCASTWYrm, X86::VPBROADCASTBYrm);
return false; // TODO: Add AVX VBROADCASTSD/VBROADCASTSS.
case X86::MOVAPDrm:
case X86::MOVAPSrm:
case X86::MOVUPDrm:
case X86::MOVUPSrm:
return false; // TODO: Add SSE3 MOVDDUP.
case X86::VMOVAPDrm:
case X86::VMOVAPSrm:
case X86::VMOVUPDrm:
case X86::VMOVUPSrm:
return ConvertToBroadcastFP(X86::VMOVDDUPrm, X86::VBROADCASTSSrm);
pengfeiUnsubmitted
Not Done
ReplyInline Actions

Why don't use VBROADCASTSD here?

pengfei: Why don't use `VBROADCASTSD` here?
RKSimonAuthorUnsubmitted
Not Done
ReplyInline Actions

xmm VBROADCASTSD doesn't exist - it would have exactly the same behaviour as VMOVDDUP

RKSimon: xmm VBROADCASTSD doesn't exist - it would have exactly the same behaviour as VMOVDDUP
case X86::VMOVAPDYrm:
case X86::VMOVAPSYrm:
case X86::VMOVUPDYrm:
case X86::VMOVUPSYrm:
return ConvertToBroadcastFP(X86::VBROADCASTSDYrm, X86::VBROADCASTSSYrm);
}
return false;
}
bool X86FixupVectorConstantsPass::runOnMachineFunction(MachineFunction &MF) {
LLVM_DEBUG(dbgs() << "Start X86FixupVectorConstants\n";);
bool Changed = false;
ST = &MF.getSubtarget<X86Subtarget>();
TII = ST->getInstrInfo();
SM = &ST->getSchedModel();
for (MachineBasicBlock &MBB : MF) {
for (MachineBasicBlock::iterator I = MBB.begin(); I != MBB.end(); ++I) {
if (processInstruction(MF, MBB, I)) {
++NumInstChanges;
Changed = true;
}
}
}
LLVM_DEBUG(dbgs() << "End X86FixupVectorConstants\n";);
return Changed;
}

llvm/lib/Target/X86/X86ISelLowering.cpp

llvm/lib/Target/X86/X86TargetMachine.cpp

llvm/test/CodeGen/X86/abdu-vector-128.ll

llvm/test/CodeGen/X86/any_extend_vector_inreg_of_broadcast.ll

llvm/test/CodeGen/X86/any_extend_vector_inreg_of_broadcast_from_memory.ll

llvm/test/CodeGen/X86/avg.ll

llvm/test/CodeGen/X86/avx-basic.ll

llvm/test/CodeGen/X86/avx-logic.ll

llvm/test/CodeGen/X86/avx-vbroadcast.ll

llvm/test/CodeGen/X86/avx-vperm2x128.ll

llvm/test/CodeGen/X86/avx2-arith.ll

llvm/test/CodeGen/X86/avx2-fma-fneg-combine.ll

llvm/test/CodeGen/X86/avx2-intrinsics-x86.ll

llvm/test/CodeGen/X86/avx2-shift.ll

llvm/test/CodeGen/X86/avx2-vbroadcast.ll

llvm/test/CodeGen/X86/avx2-vector-shifts.ll

llvm/test/CodeGen/X86/avx512-arith.ll

llvm/test/CodeGen/X86/avx512-regcall-Mask.ll

llvm/test/CodeGen/X86/bitcast-int-to-vector-bool-sext.ll

llvm/test/CodeGen/X86/bitcast-int-to-vector-bool-zext.ll

llvm/test/CodeGen/X86/bitcast-int-to-vector-bool.ll

llvm/test/CodeGen/X86/bitcast-vector-bool.ll

llvm/test/CodeGen/X86/bool-ext-inc.ll

llvm/test/CodeGen/X86/broadcast-elm-cross-splat-vec.ll

llvm/test/CodeGen/X86/cast-vsel.ll

llvm/test/CodeGen/X86/combine-add.ll

llvm/test/CodeGen/X86/combine-addo.ll

llvm/test/CodeGen/X86/combine-and.ll

llvm/test/CodeGen/X86/combine-bitselect.ll

llvm/test/CodeGen/X86/combine-concatvectors.ll

Show All 22 Lines
; AVX2-LABEL: concat_of_broadcast_v2f64_v4f64: ; AVX2-LABEL: concat_of_broadcast_v2f64_v4f64:
; AVX2: # %bb.0: # %alloca_0 ; AVX2: # %bb.0: # %alloca_0
; AVX2-NEXT: movq qa_@GOTPCREL(%rip), %rax ; AVX2-NEXT: movq qa_@GOTPCREL(%rip), %rax
; AVX2-NEXT: movl $1091567616, 30256(%rax) # imm = 0x41100000 ; AVX2-NEXT: movl $1091567616, 30256(%rax) # imm = 0x41100000
; AVX2-NEXT: movabsq $4294967297, %rcx # imm = 0x100000001 ; AVX2-NEXT: movabsq $4294967297, %rcx # imm = 0x100000001
; AVX2-NEXT: movq %rcx, 46348(%rax) ; AVX2-NEXT: movq %rcx, 46348(%rax)
; AVX2-NEXT: vbroadcastss {{.*#+}} ymm0 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0] ; AVX2-NEXT: vbroadcastss {{.*#+}} ymm0 = [1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0,1.0E+0]
; AVX2-NEXT: vmovups %ymm0, 48296(%rax) ; AVX2-NEXT: vmovups %ymm0, 48296(%rax)
; AVX2-NEXT: vmovlps %xmm0, 47372(%rax) ; AVX2-NEXT: vmovsd {{.*#+}} xmm0 = mem[0],zero
; AVX2-NEXT: vmovsd %xmm0, 47372(%rax)
pengfeiUnsubmitted
Not Done
ReplyInline Actions

Regression?

pengfei: Regression?
RKSimonAuthorUnsubmitted
Not Done
ReplyInline Actions

Yes, we previously reused the ymm0 splat to avoid a second load

RKSimon: Yes, we previously reused the ymm0 splat to avoid a second load
; AVX2-NEXT: vzeroupper ; AVX2-NEXT: vzeroupper
; AVX2-NEXT: retq ; AVX2-NEXT: retq
alloca_0: alloca_0:
store float 9.000000e+00, ptr getelementptr inbounds ([49216 x i8], ptr @qa_, i64 0, i64 30256), align 16 store float 9.000000e+00, ptr getelementptr inbounds ([49216 x i8], ptr @qa_, i64 0, i64 30256), align 16
store <2 x i32> <i32 1, i32 1>, ptr getelementptr inbounds ([49216 x i8], ptr @qa_, i64 0, i64 46348), align 4 store <2 x i32> <i32 1, i32 1>, ptr getelementptr inbounds ([49216 x i8], ptr @qa_, i64 0, i64 46348), align 4
br label %loop.4942 br label %loop.4942
loop.4942: ; preds = %loop.4942, %alloca_0 loop.4942: ; preds = %loop.4942, %alloca_0
Show All 22 Lines

llvm/test/CodeGen/X86/combine-fabs.ll

llvm/test/CodeGen/X86/combine-fcopysign.ll

llvm/test/CodeGen/X86/combine-mul.ll

llvm/test/CodeGen/X86/combine-pavg.ll

llvm/test/CodeGen/X86/combine-pmuldq.ll

llvm/test/CodeGen/X86/combine-rotates.ll

llvm/test/CodeGen/X86/combine-sdiv.ll

llvm/test/CodeGen/X86/combine-shl.ll

llvm/test/CodeGen/X86/combine-smax.ll

llvm/test/CodeGen/X86/combine-smin.ll

llvm/test/CodeGen/X86/combine-srem.ll

llvm/test/CodeGen/X86/combine-srl.ll

llvm/test/CodeGen/X86/combine-sub-usat.ll

llvm/test/CodeGen/X86/combine-udiv.ll

llvm/test/CodeGen/X86/combine-urem.ll

llvm/test/CodeGen/X86/concat-cast.ll

llvm/test/CodeGen/X86/copy-low-subvec-elt-to-high-subvec-elt.ll

llvm/test/CodeGen/X86/exedepsfix-broadcast.ll

llvm/test/CodeGen/X86/expand-vp-fp-intrinsics.ll

llvm/test/CodeGen/X86/expand-vp-int-intrinsics.ll

llvm/test/CodeGen/X86/extractelement-fp.ll

llvm/test/CodeGen/X86/extractelement-from-arg.ll

llvm/test/CodeGen/X86/extractelement-legalization-cycle.ll

llvm/test/CodeGen/X86/extractelement-load.ll

llvm/test/CodeGen/X86/extractelement-shuffle.ll

llvm/test/CodeGen/X86/fma-intrinsics-fast-isel.ll

llvm/test/CodeGen/X86/fma_patterns.ll

llvm/test/CodeGen/X86/fma_patterns_wide.ll

llvm/test/CodeGen/X86/fold-vector-trunc-sitofp.ll

llvm/test/CodeGen/X86/fp-round.ll

llvm/test/CodeGen/X86/freeze-binary.ll

llvm/test/CodeGen/X86/freeze-vector.ll

llvm/test/CodeGen/X86/funnel-shift-rot.ll

llvm/test/CodeGen/X86/gfni-funnel-shifts.ll

llvm/test/CodeGen/X86/gfni-rotates.ll

llvm/test/CodeGen/X86/gfni-shifts.ll

llvm/test/CodeGen/X86/hoist-and-by-const-from-lshr-in-eqcmp-zero.ll

llvm/test/CodeGen/X86/hoist-and-by-const-from-shl-in-eqcmp-zero.ll

llvm/test/CodeGen/X86/horizontal-reduce-umax.ll

llvm/test/CodeGen/X86/horizontal-reduce-umin.ll

Show All 22 Lines
; X86-SSE42-NEXT: pcmpgtq %xmm3, %xmm0 ; X86-SSE42-NEXT: pcmpgtq %xmm3, %xmm0
; X86-SSE42-NEXT: blendvpd %xmm0, %xmm1, %xmm2 ; X86-SSE42-NEXT: blendvpd %xmm0, %xmm1, %xmm2
; X86-SSE42-NEXT: movd %xmm2, %eax ; X86-SSE42-NEXT: movd %xmm2, %eax
; X86-SSE42-NEXT: pextrd $1, %xmm2, %edx ; X86-SSE42-NEXT: pextrd $1, %xmm2, %edx
; X86-SSE42-NEXT: retl ; X86-SSE42-NEXT: retl
; ;
; X86-AVX1-LABEL: test_reduce_v2i64: ; X86-AVX1-LABEL: test_reduce_v2i64:
; X86-AVX1: ## %bb.0: ; X86-AVX1: ## %bb.0:
; X86-AVX1-NEXT: vshufps {{.*#+}} xmm1 = xmm0[2,3,2,3] ; X86-AVX1-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,2,3]
; X86-AVX1-NEXT: vmovddup {{.*#+}} xmm2 = [9223372036854775808,9223372036854775808] ; X86-AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [0,2147483648,0,2147483648]
; X86-AVX1-NEXT: ## xmm2 = mem[0,0] ; X86-AVX1-NEXT: vpxor %xmm2, %xmm0, %xmm3
; X86-AVX1-NEXT: vxorps %xmm2, %xmm0, %xmm3 ; X86-AVX1-NEXT: vpxor %xmm2, %xmm1, %xmm2
; X86-AVX1-NEXT: vxorps %xmm2, %xmm1, %xmm2
; X86-AVX1-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2 ; X86-AVX1-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2
; X86-AVX1-NEXT: vblendvpd %xmm2, %xmm0, %xmm1, %xmm0 ; X86-AVX1-NEXT: vblendvpd %xmm2, %xmm0, %xmm1, %xmm0
; X86-AVX1-NEXT: vmovd %xmm0, %eax ; X86-AVX1-NEXT: vmovd %xmm0, %eax
; X86-AVX1-NEXT: vpextrd $1, %xmm0, %edx ; X86-AVX1-NEXT: vpextrd $1, %xmm0, %edx
; X86-AVX1-NEXT: retl ; X86-AVX1-NEXT: retl
; ;
; X86-AVX2-LABEL: test_reduce_v2i64: ; X86-AVX2-LABEL: test_reduce_v2i64:
; X86-AVX2: ## %bb.0: ; X86-AVX2: ## %bb.0:
▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Lines
; X64-AVX1-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2 ; X64-AVX1-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2
; X64-AVX1-NEXT: vblendvpd %xmm2, %xmm0, %xmm1, %xmm0 ; X64-AVX1-NEXT: vblendvpd %xmm2, %xmm0, %xmm1, %xmm0
; X64-AVX1-NEXT: vmovq %xmm0, %rax ; X64-AVX1-NEXT: vmovq %xmm0, %rax
; X64-AVX1-NEXT: retq ; X64-AVX1-NEXT: retq
; ;
; X64-AVX2-LABEL: test_reduce_v2i64: ; X64-AVX2-LABEL: test_reduce_v2i64:
; X64-AVX2: ## %bb.0: ; X64-AVX2: ## %bb.0:
; X64-AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,2,3] ; X64-AVX2-NEXT: vpshufd {{.*#+}} xmm1 = xmm0[2,3,2,3]
; X64-AVX2-NEXT: vmovdqa {{.*#+}} xmm2 = [9223372036854775808,9223372036854775808] ; X64-AVX2-NEXT: vpbroadcastq {{.*#+}} xmm2 = [9223372036854775808,9223372036854775808]
; X64-AVX2-NEXT: vpxor %xmm2, %xmm0, %xmm3 ; X64-AVX2-NEXT: vpxor %xmm2, %xmm0, %xmm3
; X64-AVX2-NEXT: vpxor %xmm2, %xmm1, %xmm2 ; X64-AVX2-NEXT: vpxor %xmm2, %xmm1, %xmm2
; X64-AVX2-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2 ; X64-AVX2-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2
; X64-AVX2-NEXT: vblendvpd %xmm2, %xmm0, %xmm1, %xmm0 ; X64-AVX2-NEXT: vblendvpd %xmm2, %xmm0, %xmm1, %xmm0
; X64-AVX2-NEXT: vmovq %xmm0, %rax ; X64-AVX2-NEXT: vmovq %xmm0, %rax
; X64-AVX2-NEXT: retq ; X64-AVX2-NEXT: retq
; ;
; X64-AVX512-LABEL: test_reduce_v2i64: ; X64-AVX512-LABEL: test_reduce_v2i64:
▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Lines
; X86-SSE42-NEXT: movdqa %xmm2, %xmm0 ; X86-SSE42-NEXT: movdqa %xmm2, %xmm0
; X86-SSE42-NEXT: blendvpd %xmm0, %xmm1, %xmm3 ; X86-SSE42-NEXT: blendvpd %xmm0, %xmm1, %xmm3
; X86-SSE42-NEXT: movd %xmm3, %eax ; X86-SSE42-NEXT: movd %xmm3, %eax
; X86-SSE42-NEXT: pextrd $1, %xmm3, %edx ; X86-SSE42-NEXT: pextrd $1, %xmm3, %edx
; X86-SSE42-NEXT: retl ; X86-SSE42-NEXT: retl
; ;
; X86-AVX1-LABEL: test_reduce_v4i64: ; X86-AVX1-LABEL: test_reduce_v4i64:
; X86-AVX1: ## %bb.0: ; X86-AVX1: ## %bb.0:
; X86-AVX1-NEXT: vmovddup {{.*#+}} xmm1 = [9223372036854775808,9223372036854775808] ; X86-AVX1-NEXT: vmovdqa {{.*#+}} xmm1 = [0,2147483648,0,2147483648]
; X86-AVX1-NEXT: ## xmm1 = mem[0,0] ; X86-AVX1-NEXT: vpxor %xmm1, %xmm0, %xmm2
; X86-AVX1-NEXT: vxorps %xmm1, %xmm0, %xmm2
; X86-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3 ; X86-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; X86-AVX1-NEXT: vxorps %xmm1, %xmm3, %xmm4 ; X86-AVX1-NEXT: vpxor %xmm1, %xmm3, %xmm4
; X86-AVX1-NEXT: vpcmpgtq %xmm2, %xmm4, %xmm2 ; X86-AVX1-NEXT: vpcmpgtq %xmm2, %xmm4, %xmm2
; X86-AVX1-NEXT: vblendvpd %xmm2, %xmm0, %xmm3, %xmm0 ; X86-AVX1-NEXT: vblendvpd %xmm2, %xmm0, %xmm3, %xmm0
; X86-AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm0[2,3,2,3] ; X86-AVX1-NEXT: vshufps {{.*#+}} xmm2 = xmm0[2,3,2,3]
; X86-AVX1-NEXT: vxorpd %xmm1, %xmm0, %xmm3 ; X86-AVX1-NEXT: vxorpd %xmm1, %xmm0, %xmm3
; X86-AVX1-NEXT: vxorpd %xmm1, %xmm2, %xmm1 ; X86-AVX1-NEXT: vxorpd %xmm1, %xmm2, %xmm1
; X86-AVX1-NEXT: vpcmpgtq %xmm3, %xmm1, %xmm1 ; X86-AVX1-NEXT: vpcmpgtq %xmm3, %xmm1, %xmm1
; X86-AVX1-NEXT: vblendvpd %xmm1, %xmm0, %xmm2, %xmm0 ; X86-AVX1-NEXT: vblendvpd %xmm1, %xmm0, %xmm2, %xmm0
; X86-AVX1-NEXT: vmovd %xmm0, %eax ; X86-AVX1-NEXT: vmovd %xmm0, %eax
▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Lines
; X64-AVX1-NEXT: vblendvpd %xmm1, %xmm0, %xmm2, %xmm0 ; X64-AVX1-NEXT: vblendvpd %xmm1, %xmm0, %xmm2, %xmm0
; X64-AVX1-NEXT: vmovq %xmm0, %rax ; X64-AVX1-NEXT: vmovq %xmm0, %rax
; X64-AVX1-NEXT: vzeroupper ; X64-AVX1-NEXT: vzeroupper
; X64-AVX1-NEXT: retq ; X64-AVX1-NEXT: retq
; ;
; X64-AVX2-LABEL: test_reduce_v4i64: ; X64-AVX2-LABEL: test_reduce_v4i64:
; X64-AVX2: ## %bb.0: ; X64-AVX2: ## %bb.0:
; X64-AVX2-NEXT: vextracti128 $1, %ymm0, %xmm1 ; X64-AVX2-NEXT: vextracti128 $1, %ymm0, %xmm1
; X64-AVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [9223372036854775808,9223372036854775808,9223372036854775808,9223372036854775808] ; X64-AVX2-NEXT: vpbroadcastq {{.*#+}} xmm2 = [9223372036854775808,9223372036854775808]
; X64-AVX2-NEXT: vpxor %xmm2, %xmm0, %xmm3 ; X64-AVX2-NEXT: vpxor %xmm2, %xmm0, %xmm3
; X64-AVX2-NEXT: vpxor %xmm2, %xmm1, %xmm4 ; X64-AVX2-NEXT: vpxor %xmm2, %xmm1, %xmm4
; X64-AVX2-NEXT: vpcmpgtq %xmm3, %xmm4, %xmm3 ; X64-AVX2-NEXT: vpcmpgtq %xmm3, %xmm4, %xmm3
; X64-AVX2-NEXT: vblendvpd %xmm3, %xmm0, %xmm1, %xmm0 ; X64-AVX2-NEXT: vblendvpd %xmm3, %xmm0, %xmm1, %xmm0
; X64-AVX2-NEXT: vshufps {{.*#+}} xmm1 = xmm0[2,3,2,3] ; X64-AVX2-NEXT: vshufps {{.*#+}} xmm1 = xmm0[2,3,2,3]
; X64-AVX2-NEXT: vxorpd %xmm2, %xmm0, %xmm3 ; X64-AVX2-NEXT: vxorpd %xmm2, %xmm0, %xmm3
; X64-AVX2-NEXT: vxorpd %xmm2, %xmm1, %xmm2 ; X64-AVX2-NEXT: vxorpd %xmm2, %xmm1, %xmm2
; X64-AVX2-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2 ; X64-AVX2-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2
▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Lines
; X86-SSE42-NEXT: blendvpd %xmm0, %xmm3, %xmm1 ; X86-SSE42-NEXT: blendvpd %xmm0, %xmm3, %xmm1
; X86-SSE42-NEXT: movd %xmm1, %eax ; X86-SSE42-NEXT: movd %xmm1, %eax
; X86-SSE42-NEXT: pextrd $1, %xmm1, %edx ; X86-SSE42-NEXT: pextrd $1, %xmm1, %edx
; X86-SSE42-NEXT: retl ; X86-SSE42-NEXT: retl
; ;
; X86-AVX1-LABEL: test_reduce_v8i64: ; X86-AVX1-LABEL: test_reduce_v8i64:
; X86-AVX1: ## %bb.0: ; X86-AVX1: ## %bb.0:
; X86-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3 ; X86-AVX1-NEXT: vextractf128 $1, %ymm0, %xmm3
; X86-AVX1-NEXT: vmovddup {{.*#+}} xmm2 = [9223372036854775808,9223372036854775808] ; X86-AVX1-NEXT: vmovdqa {{.*#+}} xmm2 = [0,2147483648,0,2147483648]
; X86-AVX1-NEXT: ## xmm2 = mem[0,0] ; X86-AVX1-NEXT: vpxor %xmm2, %xmm3, %xmm4
; X86-AVX1-NEXT: vxorps %xmm2, %xmm3, %xmm4
; X86-AVX1-NEXT: vextractf128 $1, %ymm1, %xmm5 ; X86-AVX1-NEXT: vextractf128 $1, %ymm1, %xmm5
; X86-AVX1-NEXT: vxorps %xmm2, %xmm5, %xmm6 ; X86-AVX1-NEXT: vpxor %xmm2, %xmm5, %xmm6
; X86-AVX1-NEXT: vpcmpgtq %xmm4, %xmm6, %xmm4 ; X86-AVX1-NEXT: vpcmpgtq %xmm4, %xmm6, %xmm4
; X86-AVX1-NEXT: vblendvpd %xmm4, %xmm3, %xmm5, %xmm3 ; X86-AVX1-NEXT: vblendvpd %xmm4, %xmm3, %xmm5, %xmm3
; X86-AVX1-NEXT: vxorps %xmm2, %xmm0, %xmm4 ; X86-AVX1-NEXT: vpxor %xmm2, %xmm0, %xmm4
; X86-AVX1-NEXT: vxorps %xmm2, %xmm1, %xmm5 ; X86-AVX1-NEXT: vpxor %xmm2, %xmm1, %xmm5
; X86-AVX1-NEXT: vpcmpgtq %xmm4, %xmm5, %xmm4 ; X86-AVX1-NEXT: vpcmpgtq %xmm4, %xmm5, %xmm4
; X86-AVX1-NEXT: vblendvpd %xmm4, %xmm0, %xmm1, %xmm0 ; X86-AVX1-NEXT: vblendvpd %xmm4, %xmm0, %xmm1, %xmm0
; X86-AVX1-NEXT: vxorpd %xmm2, %xmm0, %xmm1 ; X86-AVX1-NEXT: vxorpd %xmm2, %xmm0, %xmm1
; X86-AVX1-NEXT: vxorpd %xmm2, %xmm3, %xmm4 ; X86-AVX1-NEXT: vxorpd %xmm2, %xmm3, %xmm4
; X86-AVX1-NEXT: vpcmpgtq %xmm1, %xmm4, %xmm1 ; X86-AVX1-NEXT: vpcmpgtq %xmm1, %xmm4, %xmm1
; X86-AVX1-NEXT: vblendvpd %xmm1, %xmm0, %xmm3, %xmm0 ; X86-AVX1-NEXT: vblendvpd %xmm1, %xmm0, %xmm3, %xmm0
; X86-AVX1-NEXT: vshufps {{.*#+}} xmm1 = xmm0[2,3,2,3] ; X86-AVX1-NEXT: vshufps {{.*#+}} xmm1 = xmm0[2,3,2,3]
; X86-AVX1-NEXT: vxorpd %xmm2, %xmm0, %xmm3 ; X86-AVX1-NEXT: vxorpd %xmm2, %xmm0, %xmm3
; X86-AVX1-NEXT: vxorpd %xmm2, %xmm1, %xmm2 ; X86-AVX1-NEXT: vxorpd %xmm2, %xmm1, %xmm2
; X86-AVX1-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2 ; X86-AVX1-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2
; X86-AVX1-NEXT: vblendvpd %xmm2, %xmm0, %xmm1, %xmm0 ; X86-AVX1-NEXT: vblendvpd %xmm2, %xmm0, %xmm1, %xmm0
; X86-AVX1-NEXT: vmovd %xmm0, %eax ; X86-AVX1-NEXT: vmovd %xmm0, %eax
; X86-AVX1-NEXT: vpextrd $1, %xmm0, %edx ; X86-AVX1-NEXT: vpextrd $1, %xmm0, %edx
; X86-AVX1-NEXT: vzeroupper ; X86-AVX1-NEXT: vzeroupper
; X86-AVX1-NEXT: retl ; X86-AVX1-NEXT: retl
; ;
; X86-AVX2-LABEL: test_reduce_v8i64: ; X86-AVX2-LABEL: test_reduce_v8i64:
; X86-AVX2: ## %bb.0: ; X86-AVX2: ## %bb.0:
; X86-AVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [9223372036854775808,9223372036854775808,9223372036854775808,9223372036854775808] ; X86-AVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [9223372036854775808,9223372036854775808,9223372036854775808,9223372036854775808]
; X86-AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm3 ; X86-AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm3
; X86-AVX2-NEXT: vpxor %ymm2, %ymm1, %ymm4 ; X86-AVX2-NEXT: vpxor %ymm2, %ymm1, %ymm2
; X86-AVX2-NEXT: vpcmpgtq %ymm3, %ymm4, %ymm3 ; X86-AVX2-NEXT: vpcmpgtq %ymm3, %ymm2, %ymm2
; X86-AVX2-NEXT: vblendvpd %ymm3, %ymm0, %ymm1, %ymm0 ; X86-AVX2-NEXT: vblendvpd %ymm2, %ymm0, %ymm1, %ymm0
; X86-AVX2-NEXT: vextractf128 $1, %ymm0, %xmm1 ; X86-AVX2-NEXT: vextractf128 $1, %ymm0, %xmm1
; X86-AVX2-NEXT: vmovddup {{.*#+}} xmm2 = [9223372036854775808,9223372036854775808]
; X86-AVX2-NEXT: ## xmm2 = mem[0,0]
pengfeiUnsubmitted
Not Done
ReplyInline Actions

This looks like regression too.

pengfei: This looks like regression too.
RKSimonAuthorUnsubmitted
Not Done
ReplyInline Actions

Yes, we have some peepholes in DAG to reuse broadcasts of different widths - I'll address this in a later iteration.

RKSimon: Yes, we have some peepholes in DAG to reuse broadcasts of different widths - I'll address this…
; X86-AVX2-NEXT: vxorpd %xmm2, %xmm0, %xmm3 ; X86-AVX2-NEXT: vxorpd %xmm2, %xmm0, %xmm3
; X86-AVX2-NEXT: vxorpd %xmm2, %xmm1, %xmm4 ; X86-AVX2-NEXT: vxorpd %xmm2, %xmm1, %xmm4
; X86-AVX2-NEXT: vpcmpgtq %xmm3, %xmm4, %xmm3 ; X86-AVX2-NEXT: vpcmpgtq %xmm3, %xmm4, %xmm3
; X86-AVX2-NEXT: vblendvpd %xmm3, %xmm0, %xmm1, %xmm0 ; X86-AVX2-NEXT: vblendvpd %xmm3, %xmm0, %xmm1, %xmm0
; X86-AVX2-NEXT: vshufps {{.*#+}} xmm1 = xmm0[2,3,2,3] ; X86-AVX2-NEXT: vshufps {{.*#+}} xmm1 = xmm0[2,3,2,3]
; X86-AVX2-NEXT: vxorpd %xmm2, %xmm0, %xmm3 ; X86-AVX2-NEXT: vxorpd %xmm2, %xmm0, %xmm3
; X86-AVX2-NEXT: vxorpd %xmm2, %xmm1, %xmm2 ; X86-AVX2-NEXT: vxorpd %xmm2, %xmm1, %xmm2
; X86-AVX2-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2 ; X86-AVX2-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2
▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Lines
; X64-AVX1-NEXT: vmovq %xmm0, %rax ; X64-AVX1-NEXT: vmovq %xmm0, %rax
; X64-AVX1-NEXT: vzeroupper ; X64-AVX1-NEXT: vzeroupper
; X64-AVX1-NEXT: retq ; X64-AVX1-NEXT: retq
; ;
; X64-AVX2-LABEL: test_reduce_v8i64: ; X64-AVX2-LABEL: test_reduce_v8i64:
; X64-AVX2: ## %bb.0: ; X64-AVX2: ## %bb.0:
; X64-AVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [9223372036854775808,9223372036854775808,9223372036854775808,9223372036854775808] ; X64-AVX2-NEXT: vpbroadcastq {{.*#+}} ymm2 = [9223372036854775808,9223372036854775808,9223372036854775808,9223372036854775808]
; X64-AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm3 ; X64-AVX2-NEXT: vpxor %ymm2, %ymm0, %ymm3
; X64-AVX2-NEXT: vpxor %ymm2, %ymm1, %ymm4 ; X64-AVX2-NEXT: vpxor %ymm2, %ymm1, %ymm2
; X64-AVX2-NEXT: vpcmpgtq %ymm3, %ymm4, %ymm3 ; X64-AVX2-NEXT: vpcmpgtq %ymm3, %ymm2, %ymm2
; X64-AVX2-NEXT: vblendvpd %ymm3, %ymm0, %ymm1, %ymm0 ; X64-AVX2-NEXT: vblendvpd %ymm2, %ymm0, %ymm1, %ymm0
; X64-AVX2-NEXT: vextractf128 $1, %ymm0, %xmm1 ; X64-AVX2-NEXT: vextractf128 $1, %ymm0, %xmm1
; X64-AVX2-NEXT: vmovddup {{.*#+}} xmm2 = [9223372036854775808,9223372036854775808]
; X64-AVX2-NEXT: ## xmm2 = mem[0,0]
; X64-AVX2-NEXT: vxorpd %xmm2, %xmm0, %xmm3 ; X64-AVX2-NEXT: vxorpd %xmm2, %xmm0, %xmm3
; X64-AVX2-NEXT: vxorpd %xmm2, %xmm1, %xmm4 ; X64-AVX2-NEXT: vxorpd %xmm2, %xmm1, %xmm4
; X64-AVX2-NEXT: vpcmpgtq %xmm3, %xmm4, %xmm3 ; X64-AVX2-NEXT: vpcmpgtq %xmm3, %xmm4, %xmm3
; X64-AVX2-NEXT: vblendvpd %xmm3, %xmm0, %xmm1, %xmm0 ; X64-AVX2-NEXT: vblendvpd %xmm3, %xmm0, %xmm1, %xmm0
; X64-AVX2-NEXT: vshufps {{.*#+}} xmm1 = xmm0[2,3,2,3] ; X64-AVX2-NEXT: vshufps {{.*#+}} xmm1 = xmm0[2,3,2,3]
; X64-AVX2-NEXT: vxorpd %xmm2, %xmm0, %xmm3 ; X64-AVX2-NEXT: vxorpd %xmm2, %xmm0, %xmm3
; X64-AVX2-NEXT: vxorpd %xmm2, %xmm1, %xmm2 ; X64-AVX2-NEXT: vxorpd %xmm2, %xmm1, %xmm2
; X64-AVX2-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2 ; X64-AVX2-NEXT: vpcmpgtq %xmm3, %xmm2, %xmm2
Show All 22 Lines

llvm/test/CodeGen/X86/i64-to-float.ll

llvm/test/CodeGen/X86/icmp-abs-C-vec.ll

llvm/test/CodeGen/X86/icmp-pow2-diff.ll

llvm/test/CodeGen/X86/insert-into-constant-vector.ll

llvm/test/CodeGen/X86/known-bits-vector.ll

llvm/test/CodeGen/X86/machine-combiner-int-vec.ll

llvm/test/CodeGen/X86/masked_load.ll

llvm/test/CodeGen/X86/masked_store_trunc.ll

llvm/test/CodeGen/X86/masked_store_trunc_ssat.ll

llvm/test/CodeGen/X86/masked_store_trunc_usat.ll

llvm/test/CodeGen/X86/memset-nonzero.ll

llvm/test/CodeGen/X86/merge-store-constants.ll

llvm/test/CodeGen/X86/midpoint-int-vec-128.ll

llvm/test/CodeGen/X86/midpoint-int-vec-256.ll

llvm/test/CodeGen/X86/movmsk-cmp.ll

llvm/test/CodeGen/X86/oddshuffles.ll

llvm/test/CodeGen/X86/omit-urem-of-power-of-two-or-zero-when-comparing-with-zero.ll

llvm/test/CodeGen/X86/opt-pipeline.ll

llvm/test/CodeGen/X86/packss.ll

llvm/test/CodeGen/X86/paddus.ll

llvm/test/CodeGen/X86/pmul.ll

llvm/test/CodeGen/X86/pmulh.ll

llvm/test/CodeGen/X86/pr30290.ll

llvm/test/CodeGen/X86/pr32368.ll

llvm/test/CodeGen/X86/pr38639.ll

llvm/test/CodeGen/X86/prefer-avx256-popcnt.ll

llvm/test/CodeGen/X86/psubus.ll

llvm/test/CodeGen/X86/recip-fastmath.ll

llvm/test/CodeGen/X86/recip-fastmath2.ll

llvm/test/CodeGen/X86/sadd_sat_vec.ll

llvm/test/CodeGen/X86/sar_fold64.ll

llvm/test/CodeGen/X86/sat-add.ll

llvm/test/CodeGen/X86/sdiv-exact.ll

llvm/test/CodeGen/X86/select-of-fp-constants.ll

llvm/test/CodeGen/X86/setcc-non-simple-type.ll

llvm/test/CodeGen/X86/shrink_vmul.ll

llvm/test/CodeGen/X86/shuffle-blendw.ll

llvm/test/CodeGen/X86/shuffle-of-splat-multiuses.ll

llvm/test/CodeGen/X86/shuffle-vs-trunc-256.ll

llvm/test/CodeGen/X86/slow-pmulld.ll

llvm/test/CodeGen/X86/splat-const.ll

llvm/test/CodeGen/X86/splat-for-size.ll

llvm/test/CodeGen/X86/sqrt-fastmath-mir.ll

llvm/test/CodeGen/X86/sqrt-fastmath-tune.ll

llvm/test/CodeGen/X86/sqrt-fastmath.ll

llvm/test/CodeGen/X86/srem-seteq-illegal-types.ll

llvm/test/CodeGen/X86/srem-seteq-vec-nonsplat.ll

llvm/test/CodeGen/X86/srem-seteq-vec-splat.ll

llvm/test/CodeGen/X86/sse2.ll

llvm/test/CodeGen/X86/sshl_sat_vec.ll

llvm/test/CodeGen/X86/ssub_sat_vec.ll

llvm/test/CodeGen/X86/subvector-broadcast.ll

llvm/test/CodeGen/X86/uadd_sat_vec.ll

llvm/test/CodeGen/X86/umax.ll

llvm/test/CodeGen/X86/urem-seteq-vec-nonsplat.ll

llvm/test/CodeGen/X86/urem-seteq-vec-nonzero.ll

llvm/test/CodeGen/X86/urem-seteq-vec-splat.ll

llvm/test/CodeGen/X86/urem-seteq-vec-tautological.ll

llvm/test/CodeGen/X86/urem-seteq.ll

llvm/test/CodeGen/X86/urem-vector-lkk.ll

llvm/test/CodeGen/X86/usub_sat_vec.ll

llvm/test/CodeGen/X86/v8i1-masks.ll

llvm/test/CodeGen/X86/var-permute-128.ll

llvm/test/CodeGen/X86/var-permute-256.ll

llvm/test/CodeGen/X86/vec-strict-fptoint-128.ll

llvm/test/CodeGen/X86/vec-strict-fptoint-256.ll

llvm/test/CodeGen/X86/vec-strict-inttofp-256.ll

llvm/test/CodeGen/X86/vec_cmp_uint-128.ll

llvm/test/CodeGen/X86/vec_fabs.ll

llvm/test/CodeGen/X86/vec_fp_to_int.ll

llvm/test/CodeGen/X86/vec_int_to_fp.ll

llvm/test/CodeGen/X86/vec_minmax_uint.ll

llvm/test/CodeGen/X86/vec_shift6.ll

llvm/test/CodeGen/X86/vec_smulo.ll

llvm/test/CodeGen/X86/vec_uaddo.ll

llvm/test/CodeGen/X86/vec_uint_to_fp-fastmath.ll

llvm/test/CodeGen/X86/vec_uint_to_fp.ll

llvm/test/CodeGen/X86/vec_umulo.ll

llvm/test/CodeGen/X86/vec_usubo.ll

llvm/test/CodeGen/X86/vector-bitreverse.ll

llvm/test/CodeGen/X86/vector-blend.ll

llvm/test/CodeGen/X86/vector-bo-select.ll

llvm/test/CodeGen/X86/vector-constrained-fp-intrinsics.ll

llvm/test/CodeGen/X86/vector-fshl-128.ll

llvm/test/CodeGen/X86/vector-fshl-256.ll

llvm/test/CodeGen/X86/vector-fshl-512.ll

llvm/test/CodeGen/X86/vector-fshl-rot-128.ll

llvm/test/CodeGen/X86/vector-fshl-rot-256.ll

llvm/test/CodeGen/X86/vector-fshl-rot-512.ll

llvm/test/CodeGen/X86/vector-fshl-rot-sub128.ll

llvm/test/CodeGen/X86/vector-fshr-128.ll

llvm/test/CodeGen/X86/vector-fshr-256.ll

llvm/test/CodeGen/X86/vector-fshr-512.ll

llvm/test/CodeGen/X86/vector-fshr-rot-128.ll

llvm/test/CodeGen/X86/vector-fshr-rot-256.ll

llvm/test/CodeGen/X86/vector-fshr-rot-512.ll

llvm/test/CodeGen/X86/vector-fshr-rot-sub128.ll

llvm/test/CodeGen/X86/vector-idiv-sdiv-128.ll

llvm/test/CodeGen/X86/vector-idiv-sdiv-256.ll

llvm/test/CodeGen/X86/vector-idiv-sdiv-512.ll

llvm/test/CodeGen/X86/vector-idiv-udiv-128.ll

llvm/test/CodeGen/X86/vector-idiv-udiv-256.ll

llvm/test/CodeGen/X86/vector-idiv-udiv-512.ll

llvm/test/CodeGen/X86/vector-interleaved-load-i16-stride-5.ll

llvm/test/CodeGen/X86/vector-interleaved-load-i16-stride-7.ll

llvm/test/CodeGen/X86/vector-interleaved-load-i32-stride-3.ll

llvm/test/CodeGen/X86/vector-interleaved-load-i32-stride-4.ll

llvm/test/CodeGen/X86/vector-interleaved-load-i32-stride-5.ll

llvm/test/CodeGen/X86/vector-interleaved-load-i32-stride-6.ll

llvm/test/CodeGen/X86/vector-interleaved-load-i32-stride-7.ll

llvm/test/CodeGen/X86/vector-interleaved-load-i8-stride-2.ll

llvm/test/CodeGen/X86/vector-interleaved-load-i8-stride-3.ll

llvm/test/CodeGen/X86/vector-interleaved-load-i8-stride-4.ll

llvm/test/CodeGen/X86/vector-interleaved-load-i8-stride-5.ll

llvm/test/CodeGen/X86/vector-interleaved-load-i8-stride-7.ll

llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-6.ll

llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-7.ll

llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-8.ll

llvm/test/CodeGen/X86/vector-interleaved-store-i32-stride-3.ll

llvm/test/CodeGen/X86/vector-interleaved-store-i32-stride-6.ll

llvm/test/CodeGen/X86/vector-interleaved-store-i32-stride-7.ll

llvm/test/CodeGen/X86/vector-interleaved-store-i32-stride-8.ll

llvm/test/CodeGen/X86/vector-interleaved-store-i8-stride-3.ll

llvm/test/CodeGen/X86/vector-interleaved-store-i8-stride-5.ll

llvm/test/CodeGen/X86/vector-interleaved-store-i8-stride-7.ll

llvm/test/CodeGen/X86/vector-interleaved-store-i8-stride-8.ll

llvm/test/CodeGen/X86/vector-lzcnt-512.ll

llvm/test/CodeGen/X86/vector-mul.ll

llvm/test/CodeGen/X86/vector-pack-128.ll

llvm/test/CodeGen/X86/vector-pack-256.ll

llvm/test/CodeGen/X86/vector-popcnt-128-ult-ugt.ll

llvm/test/CodeGen/X86/vector-popcnt-128.ll

llvm/test/CodeGen/X86/vector-popcnt-256-ult-ugt.ll

llvm/test/CodeGen/X86/vector-popcnt-256.ll

llvm/test/CodeGen/X86/vector-popcnt-512-ult-ugt.ll

llvm/test/CodeGen/X86/vector-popcnt-512.ll

llvm/test/CodeGen/X86/vector-reduce-add-mask.ll

llvm/test/CodeGen/X86/vector-reduce-and-bool.ll

llvm/test/CodeGen/X86/vector-reduce-or-bool.ll

llvm/test/CodeGen/X86/vector-reduce-or-cmp.ll

llvm/test/CodeGen/X86/vector-reduce-umax.ll

llvm/test/CodeGen/X86/vector-reduce-umin.ll

llvm/test/CodeGen/X86/vector-reduce-xor-bool.ll

llvm/test/CodeGen/X86/vector-rotate-128.ll

llvm/test/CodeGen/X86/vector-rotate-256.ll

llvm/test/CodeGen/X86/vector-rotate-512.ll

llvm/test/CodeGen/X86/vector-sext.ll

llvm/test/CodeGen/X86/vector-shift-ashr-128.ll

llvm/test/CodeGen/X86/vector-shift-ashr-256.ll

llvm/test/CodeGen/X86/vector-shift-ashr-512.ll

llvm/test/CodeGen/X86/vector-shift-ashr-sub128.ll

llvm/test/CodeGen/X86/vector-shift-lshr-256.ll

llvm/test/CodeGen/X86/vector-shift-lshr-512.ll

llvm/test/CodeGen/X86/vector-shift-shl-256.ll

llvm/test/CodeGen/X86/vector-shift-shl-512.ll

llvm/test/CodeGen/X86/vector-shuffle-128-v16.ll

llvm/test/CodeGen/X86/vector-shuffle-256-v16.ll

llvm/test/CodeGen/X86/vector-shuffle-256-v32.ll

llvm/test/CodeGen/X86/vector-shuffle-256-v8.ll

llvm/test/CodeGen/X86/vector-shuffle-avx512.ll

llvm/test/CodeGen/X86/vector-shuffle-combining.ll

llvm/test/CodeGen/X86/vector-trunc-math.ll

llvm/test/CodeGen/X86/vector-trunc-packus.ll

llvm/test/CodeGen/X86/vector-trunc-ssat.ll

llvm/test/CodeGen/X86/vector-trunc-usat.ll

llvm/test/CodeGen/X86/vector-trunc.ll

llvm/test/CodeGen/X86/vector-tzcnt-128.ll

llvm/test/CodeGen/X86/vector-tzcnt-256.ll

llvm/test/CodeGen/X86/vector-tzcnt-512.ll

llvm/test/CodeGen/X86/vector-unsigned-cmp.ll

llvm/test/CodeGen/X86/vector-zext.ll

llvm/test/CodeGen/X86/vector_splat-const-shift-of-constmasked.ll

llvm/test/CodeGen/X86/vselect-avx.ll

llvm/test/CodeGen/X86/vselect-minmax.ll

llvm/test/CodeGen/X86/vselect-pcmp.ll

llvm/test/CodeGen/X86/vselect-post-combine.ll

llvm/test/CodeGen/X86/vselect-zero.ll

llvm/test/CodeGen/X86/win_cst_pool.ll

llvm/test/CodeGen/X86/x86-interleaved-access.ll

llvm/test/CodeGen/X86/zero_extend_vector_inreg.ll

llvm/test/CodeGen/X86/zero_extend_vector_inreg_of_broadcast.ll

llvm/test/CodeGen/X86/zero_extend_vector_inreg_of_broadcast_from_memory.ll