This is an archive of the discontinued LLVM Phabricator instance.

Differential D107923

[X86] X86DAGToDAGISel::matchBitExtract(): support 'num high bits to clear' pattern
ClosedPublic

Authored by lebedev.ri on Aug 11 2021, 12:48 PM.

Details

Reviewers
craig.topper
RKSimon
spatel
pengfei
Commits
rG0852f8706b76: [X86] X86DAGToDAGISel::matchBitExtract(): support 'num high bits to clear'…
Summary

Currently, we only deal with the case where we can match
the number of low bits to be kept, i.e.:

x & ((1 << y) - 1)

will extract low y bits of x.

But what will

x & (-1 >> y)

do?

Logically, it will extract bitwidth(x) - y low bits, i.e.:

x & ~(-1 << (bitwidth(x)-y))

... except we can't do such a transformation in IR in general,
because if we wanted to extract all the bits (-1 >> 0) is fine,
but -1 << bitwidth(x) would be poison: https://alive2.llvm.org/ce/z/BKJZfw,

Yet, here with BMI's BEXTR and BMI2's BZHI we don't have any such problems with edge-cases.
So what we can do is: https://alive2.llvm.org/ce/z/gm5M2B

As briefly discussed with @craig.topper, this appears to be not worse than what we'd end up with currently (a pair of shifts):

Diff Detail

Event Timeline

lebedev.ri created this revision.Aug 11 2021, 12:48 PM
Herald added subscribers: pengfei, hiraditya. · View Herald TranscriptAug 11 2021, 12:48 PM
lebedev.ri requested review of this revision.Aug 11 2021, 12:48 PM
Harbormaster completed remote builds in B119127: Diff 365820.Aug 11 2021, 1:24 PM
lebedev.ri added a comment.Aug 11 2021, 3:24 PM

That being said, i guess we should restrict this to BMI2 BZHI only:
https://godbolt.org/z/zxhPa8Gdh https://godbolt.org/z/KhWhE1eeE
Let me do that..

lebedev.ri updated this revision to Diff 365868.Aug 11 2021, 3:41 PM

Restrict to BMI2 BZHI.

Harbormaster completed remote builds in B119158: Diff 365868.Aug 11 2021, 4:08 PM
lebedev.ri added a comment.Aug 24 2021, 2:22 PM

Bump

lebedev.ri added a reviewer: pengfei.Aug 25 2021, 7:41 AM
RKSimon added inline comments.Aug 25 2021, 7:58 AM
llvm/lib/Target/X86/X86ISelDAGToDAG.cpp
3462

is this a match function anymore? should it canonicalizeShiftAmt?

lebedev.ri updated this revision to Diff 368641.Aug 25 2021, 8:07 AM
lebedev.ri marked an inline comment as done.

@RKSimon thank you for taking a look!
I guess you're right, renamed lambda accordingly.

Harbormaster completed remote builds in B121168: Diff 368641.Aug 25 2021, 8:53 AM
lebedev.ri added a comment.Sep 2 2021, 10:39 AM

bump

lebedev.ri added a comment.Sep 7 2021, 11:06 AM

bump

craig.topper added inline comments.Sep 7 2021, 11:37 AM
llvm/lib/Target/X86/X86ISelDAGToDAG.cpp
3567

NVT is a scalar isn't it? So this could just be getSizeINBits()?

llvm/test/CodeGen/X86/clear-highbits.ll
1189

This doesn't seem better. But maybe it increases parallelism the if the shrxl use wasn't a call?

lebedev.ri updated this revision to Diff 371169.Sep 7 2021, 1:25 PM
lebedev.ri marked an inline comment as done.

Thanks for taking a look!

Yeah, i keep wondering about these one-use checks.
I'm not sure what's right.

llvm/lib/Target/X86/X86ISelDAGToDAG.cpp
3567

Err, right.

lebedev.ri updated this revision to Diff 371185.Sep 7 2021, 2:14 PM

Fix one-use checks - this is hilariously hideous.

lebedev.ri marked an inline comment as done.Sep 7 2021, 2:15 PM
Harbormaster completed remote builds in B122944: Diff 371185.Sep 7 2021, 3:05 PM
craig.topper added inline comments.Sep 7 2021, 9:13 PM
llvm/lib/Target/X86/X86ISelDAGToDAG.cpp
3399

Can we use Optional::getValueOr here?

lebedev.ri updated this revision to Diff 371280.Sep 8 2021, 1:57 AM
lebedev.ri marked an inline comment as done.

Thank you for taking a look!
Use Optional::getValueOr().

Harbormaster completed remote builds in B123013: Diff 371280.Sep 8 2021, 2:27 AM
craig.topper accepted this revision.Sep 8 2021, 9:14 AM

LGTM

This revision is now accepted and ready to land.Sep 8 2021, 9:14 AM
lebedev.ri added a comment.Sep 8 2021, 9:15 AM
In D107923#2989642, @craig.topper wrote:

LGTM

Thank you for the review!

This revision was landed with ongoing or failed builds.Sep 8 2021, 9:27 AM
Closed by commit rG0852f8706b76: [X86] X86DAGToDAGISel::matchBitExtract(): support 'num high bits to clear'… (authored by lebedev.ri). · Explain Why
This revision was automatically updated to reflect the committed changes.
lebedev.ri added a commit: rG0852f8706b76: [X86] X86DAGToDAGISel::matchBitExtract(): support 'num high bits to clear'….

Revision Contents

PathSize
llvm/
lib/
Target/
X86/
69 lines
test/
CodeGen/
X86/
1527 lines

Diff 365820

llvm/lib/Target/X86/X86ISelDAGToDAG.cpp

Show First 20 Lines • Show All 3,381 Lines • ▼ Show 20 Linesbool X86DAGToDAGISel::matchBitExtract(SDNode *Node) {
MVT NVT = Node->getSimpleValueType(0); MVT NVT = Node->getSimpleValueType(0);
// Only supported for 32 and 64 bits. // Only supported for 32 and 64 bits.
if (NVT != MVT::i32 && NVT != MVT::i64) if (NVT != MVT::i32 && NVT != MVT::i64)
return false; return false;
SDValue NBits; SDValue NBits;
bool NegateNBits;
// If we have BMI2's BZHI, we are ok with muti-use patterns. // If we have BMI2's BZHI, we are ok with muti-use patterns.
// Else, if we only have BMI1's BEXTR, we require one-use. // Else, if we only have BMI1's BEXTR, we require one-use.
const bool CanHaveExtraUses = Subtarget->hasBMI2(); const bool CanHaveExtraUses = Subtarget->hasBMI2();
auto checkUses = [CanHaveExtraUses](SDValue Op, unsigned NUses) { auto checkUses = [CanHaveExtraUses](SDValue Op, unsigned NUses) {
return CanHaveExtraUses || return CanHaveExtraUses ||
Op.getNode()->hasNUsesOfValue(NUses, Op.getResNo()); Op.getNode()->hasNUsesOfValue(NUses, Op.getResNo());
}; };
auto checkOneUse = [checkUses](SDValue Op) { return checkUses(Op, 1); }; auto checkOneUse = [checkUses](SDValue Op) { return checkUses(Op, 1); };
craig.topperUnsubmitted
Done
ReplyInline Actions

Can we use Optional::getValueOr here?

craig.topper: Can we use Optional::getValueOr here?
auto checkTwoUse = [checkUses](SDValue Op) { return checkUses(Op, 2); }; auto checkTwoUse = [checkUses](SDValue Op) { return checkUses(Op, 2); };
auto peekThroughOneUseTruncation = [checkOneUse](SDValue V) { auto peekThroughOneUseTruncation = [checkOneUse](SDValue V) {
if (V->getOpcode() == ISD::TRUNCATE && checkOneUse(V)) { if (V->getOpcode() == ISD::TRUNCATE && checkOneUse(V)) {
assert(V.getSimpleValueType() == MVT::i32 && assert(V.getSimpleValueType() == MVT::i32 &&
V.getOperand(0).getSimpleValueType() == MVT::i64 && V.getOperand(0).getSimpleValueType() == MVT::i64 &&
"Expected i64 -> i32 truncation"); "Expected i64 -> i32 truncation");
V = V.getOperand(0); V = V.getOperand(0);
} }
return V; return V;
}; };
// a) x & ((1 << nbits) + (-1)) // a) x & ((1 << nbits) + (-1))
auto matchPatternA = [checkOneUse, peekThroughOneUseTruncation, auto matchPatternA = [checkOneUse, peekThroughOneUseTruncation, &NBits,
&NBits](SDValue Mask) -> bool { &NegateNBits](SDValue Mask) -> bool {
// Match `add`. Must only have one use! // Match `add`. Must only have one use!
if (Mask->getOpcode() != ISD::ADD || !checkOneUse(Mask)) if (Mask->getOpcode() != ISD::ADD || !checkOneUse(Mask))
Lint: Pre-merge checks
Inline Actions

clang-tidy: warning: invalid case style for variable 'matchPatternA' [readability-identifier-naming]
not useful

Lint: Pre-merge checks: clang-tidy: warning: invalid case style for variable 'matchPatternA' [readability-identifier…
return false; return false;
// We should be adding all-ones constant (i.e. subtracting one.) // We should be adding all-ones constant (i.e. subtracting one.)
if (!isAllOnesConstant(Mask->getOperand(1))) if (!isAllOnesConstant(Mask->getOperand(1)))
return false; return false;
// Match `1 << nbits`. Might be truncated. Must only have one use! // Match `1 << nbits`. Might be truncated. Must only have one use!
SDValue M0 = peekThroughOneUseTruncation(Mask->getOperand(0)); SDValue M0 = peekThroughOneUseTruncation(Mask->getOperand(0));
if (M0->getOpcode() != ISD::SHL || !checkOneUse(M0)) if (M0->getOpcode() != ISD::SHL || !checkOneUse(M0))
return false; return false;
if (!isOneConstant(M0->getOperand(0))) if (!isOneConstant(M0->getOperand(0)))
return false; return false;
NBits = M0->getOperand(1); NBits = M0->getOperand(1);
NegateNBits = false;
return true; return true;
}; };
auto isAllOnes = [this, peekThroughOneUseTruncation, NVT](SDValue V) { auto isAllOnes = [this, peekThroughOneUseTruncation, NVT](SDValue V) {
V = peekThroughOneUseTruncation(V); V = peekThroughOneUseTruncation(V);
return CurDAG->MaskedValueIsAllOnes( return CurDAG->MaskedValueIsAllOnes(
V, APInt::getLowBitsSet(V.getSimpleValueType().getSizeInBits(), V, APInt::getLowBitsSet(V.getSimpleValueType().getSizeInBits(),
NVT.getSizeInBits())); NVT.getSizeInBits()));
}; };
// b) x & ~(-1 << nbits) // b) x & ~(-1 << nbits)
auto matchPatternB = [checkOneUse, isAllOnes, peekThroughOneUseTruncation, auto matchPatternB = [checkOneUse, isAllOnes, peekThroughOneUseTruncation,
&NBits](SDValue Mask) -> bool { &NBits, &NegateNBits](SDValue Mask) -> bool {
// Match `~()`. Must only have one use! // Match `~()`. Must only have one use!
if (Mask.getOpcode() != ISD::XOR || !checkOneUse(Mask)) if (Mask.getOpcode() != ISD::XOR || !checkOneUse(Mask))
return false; return false;
// The -1 only has to be all-ones for the final Node's NVT. // The -1 only has to be all-ones for the final Node's NVT.
if (!isAllOnes(Mask->getOperand(1))) if (!isAllOnes(Mask->getOperand(1)))
return false; return false;
// Match `-1 << nbits`. Might be truncated. Must only have one use! // Match `-1 << nbits`. Might be truncated. Must only have one use!
SDValue M0 = peekThroughOneUseTruncation(Mask->getOperand(0)); SDValue M0 = peekThroughOneUseTruncation(Mask->getOperand(0));
if (M0->getOpcode() != ISD::SHL || !checkOneUse(M0)) if (M0->getOpcode() != ISD::SHL || !checkOneUse(M0))
return false; return false;
// The -1 only has to be all-ones for the final Node's NVT. // The -1 only has to be all-ones for the final Node's NVT.
if (!isAllOnes(M0->getOperand(0))) if (!isAllOnes(M0->getOperand(0)))
return false; return false;
NBits = M0->getOperand(1); NBits = M0->getOperand(1);
NegateNBits = false;
return true; return true;
}; };
// Match potentially-truncated (bitwidth - y) // Try to match potentially-truncated shift amount as `(bitwidth - y)`.
auto matchShiftAmt = [checkOneUse, &NBits](SDValue ShiftAmt, auto matchShiftAmt = [&NBits, &NegateNBits](SDValue ShiftAmt,
unsigned Bitwidth) { unsigned Bitwidth) {
RKSimonUnsubmitted
Done
ReplyInline Actions

is this a match function anymore? should it canonicalizeShiftAmt?

RKSimon: is this a match function anymore? should it canonicalizeShiftAmt?
// Skip over a truncate of the shift amount. NBits = ShiftAmt;
if (ShiftAmt.getOpcode() == ISD::TRUNCATE) { NegateNBits = true;
Lint: Pre-merge checks
Inline Actions

clang-tidy: warning: invalid case style for variable 'matchShiftAmt' [readability-identifier-naming]
not useful

Lint: Pre-merge checks: clang-tidy: warning: invalid case style for variable 'matchShiftAmt' [readability-identifier…
ShiftAmt = ShiftAmt.getOperand(0); // Skip over a truncate of the shift amount, if any.
// The trunc should have been the only user of the real shift amount. if (NBits.getOpcode() == ISD::TRUNCATE)
if (!checkOneUse(ShiftAmt)) NBits = NBits.getOperand(0);
return false; // Try to match the shift amount as (bitwidth - y). It should go away, too.
} // If it doesn't match, that's fine, we'll just negate it ourselves.
// Match the shift amount as: (bitwidth - y). It should go away, too. if (NBits.getOpcode() != ISD::SUB)
if (ShiftAmt.getOpcode() != ISD::SUB) return;
return false; auto *V0 = dyn_cast<ConstantSDNode>(NBits.getOperand(0));
auto *V0 = dyn_cast<ConstantSDNode>(ShiftAmt.getOperand(0));
if (!V0 || V0->getZExtValue() != Bitwidth) if (!V0 || V0->getZExtValue() != Bitwidth)
return false; return;
NBits = ShiftAmt.getOperand(1); NBits = NBits.getOperand(1);
return true; NegateNBits = false;
}; };
// c) x & (-1 >> z) but then we'll have to subtract z from bitwidth
// or
// c) x & (-1 >> (32 - y)) // c) x & (-1 >> (32 - y))
auto matchPatternC = [checkOneUse, peekThroughOneUseTruncation, auto matchPatternC = [checkOneUse, peekThroughOneUseTruncation,
matchShiftAmt](SDValue Mask) -> bool { matchShiftAmt](SDValue Mask) -> bool {
// The mask itself may be truncated. // The mask itself may be truncated.
Mask = peekThroughOneUseTruncation(Mask); Mask = peekThroughOneUseTruncation(Mask);
unsigned Bitwidth = Mask.getSimpleValueType().getSizeInBits(); unsigned Bitwidth = Mask.getSimpleValueType().getSizeInBits();
// Match `l>>`. Must only have one use! // Match `l>>`. Must only have one use!
if (Mask.getOpcode() != ISD::SRL || !checkOneUse(Mask)) if (Mask.getOpcode() != ISD::SRL || !checkOneUse(Mask))
return false; return false;
// We should be shifting truly all-ones constant. // We should be shifting truly all-ones constant.
if (!isAllOnesConstant(Mask.getOperand(0))) if (!isAllOnesConstant(Mask.getOperand(0)))
return false; return false;
SDValue M1 = Mask.getOperand(1); SDValue M1 = Mask.getOperand(1);
// The shift amount should not be used externally. // The shift amount should not be used externally.
if (!checkOneUse(M1)) if (!checkOneUse(M1))
return false; return false;
return matchShiftAmt(M1, Bitwidth); matchShiftAmt(M1, Bitwidth);
return true;
}; };
SDValue X; SDValue X;
// d) x << z >> z but then we'll have to subtract z from bitwidth
// or
// d) x << (32 - y) >> (32 - y) // d) x << (32 - y) >> (32 - y)
auto matchPatternD = [checkOneUse, checkTwoUse, matchShiftAmt, auto matchPatternD = [checkOneUse, checkTwoUse, matchShiftAmt,
&X](SDNode *Node) -> bool { &X](SDNode *Node) -> bool {
if (Node->getOpcode() != ISD::SRL) if (Node->getOpcode() != ISD::SRL)
return false; return false;
SDValue N0 = Node->getOperand(0); SDValue N0 = Node->getOperand(0);
if (N0->getOpcode() != ISD::SHL || !checkOneUse(N0)) if (N0->getOpcode() != ISD::SHL || !checkOneUse(N0))
return false; return false;
unsigned Bitwidth = N0.getSimpleValueType().getSizeInBits(); unsigned Bitwidth = N0.getSimpleValueType().getSizeInBits();
SDValue N1 = Node->getOperand(1); SDValue N1 = Node->getOperand(1);
SDValue N01 = N0->getOperand(1); SDValue N01 = N0->getOperand(1);
// Both of the shifts must be by the exact same value. // Both of the shifts must be by the exact same value.
// There should not be any uses of the shift amount outside of the pattern. // There should not be any uses of the shift amount outside of the pattern.
if (N1 != N01 || !checkTwoUse(N1)) if (N1 != N01 || !checkTwoUse(N1))
return false; return false;
if (!matchShiftAmt(N1, Bitwidth)) matchShiftAmt(N1, Bitwidth);
return false;
X = N0->getOperand(0); X = N0->getOperand(0);
return true; return true;
}; };
auto matchLowBitMask = [matchPatternA, matchPatternB, auto matchLowBitMask = [matchPatternA, matchPatternB,
matchPatternC](SDValue Mask) -> bool { matchPatternC](SDValue Mask) -> bool {
return matchPatternA(Mask) || matchPatternB(Mask) || matchPatternC(Mask); return matchPatternA(Mask) || matchPatternB(Mask) || matchPatternC(Mask);
}; };
Show All 21 Linesbool X86DAGToDAGISel::matchBitExtract(SDNode *Node) {
// Insert 8-bit NBits into lowest 8 bits of 32-bit register. // Insert 8-bit NBits into lowest 8 bits of 32-bit register.
// All the other bits are undefined, we do not care about them. // All the other bits are undefined, we do not care about them.
SDValue ImplDef = SDValue( SDValue ImplDef = SDValue(
CurDAG->getMachineNode(TargetOpcode::IMPLICIT_DEF, DL, MVT::i32), 0); CurDAG->getMachineNode(TargetOpcode::IMPLICIT_DEF, DL, MVT::i32), 0);
insertDAGNode(*CurDAG, SDValue(Node, 0), ImplDef); insertDAGNode(*CurDAG, SDValue(Node, 0), ImplDef);
SDValue SRIdxVal = CurDAG->getTargetConstant(X86::sub_8bit, DL, MVT::i32); SDValue SRIdxVal = CurDAG->getTargetConstant(X86::sub_8bit, DL, MVT::i32);
insertDAGNode(*CurDAG, SDValue(Node, 0), SRIdxVal); insertDAGNode(*CurDAG, SDValue(Node, 0), SRIdxVal);
NBits = SDValue( NBits = SDValue(CurDAG->getMachineNode(TargetOpcode::INSERT_SUBREG, DL,
CurDAG->getMachineNode(TargetOpcode::INSERT_SUBREG, DL, MVT::i32, ImplDef, MVT::i32, ImplDef, NBits, SRIdxVal),
NBits, SRIdxVal), 0); 0);
insertDAGNode(*CurDAG, SDValue(Node, 0), NBits); insertDAGNode(*CurDAG, SDValue(Node, 0), NBits);
// We might have matched the amount of high bits to be cleared,
// but we want the amount of low bits to be kept, so negate it then.
if (NegateNBits) {
SDValue BitWidthC =
CurDAG->getConstant(NVT.getScalarSizeInBits(), DL, MVT::i32);
craig.topperUnsubmitted
Done
ReplyInline Actions

NVT is a scalar isn't it? So this could just be getSizeINBits()?

craig.topper: NVT is a scalar isn't it? So this could just be getSizeINBits()?
lebedev.riAuthorUnsubmitted
Done
ReplyInline Actions

Err, right.

lebedev.ri: Err, right.
insertDAGNode(*CurDAG, SDValue(Node, 0), BitWidthC);
NBits = CurDAG->getNode(ISD::SUB, DL, MVT::i32, BitWidthC, NBits);
insertDAGNode(*CurDAG, SDValue(Node, 0), NBits);
}
if (Subtarget->hasBMI2()) { if (Subtarget->hasBMI2()) {
// Great, just emit the the BZHI.. // Great, just emit the the BZHI..
if (NVT != MVT::i32) { if (NVT != MVT::i32) {
// But have to place the bit count into the wide-enough register first. // But have to place the bit count into the wide-enough register first.
NBits = CurDAG->getNode(ISD::ANY_EXTEND, DL, NVT, NBits); NBits = CurDAG->getNode(ISD::ANY_EXTEND, DL, NVT, NBits);
insertDAGNode(*CurDAG, SDValue(Node, 0), NBits); insertDAGNode(*CurDAG, SDValue(Node, 0), NBits);
} }
▲ Show 20 Lines • Show All 2,442 Lines • Show Last 20 Lines

llvm/test/CodeGen/X86/clear-highbits.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=-bmi,-tbm,-bmi2 < %s | FileCheck %s --check-prefixes=X86,X86-NOBMI2,X86-FALLBACK0 ; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=-bmi,-tbm,-bmi2 < %s | FileCheck %s --check-prefixes=X86,X86-BASELINE
; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=+cmov,+bmi,-tbm,-bmi2 < %s | FileCheck %s --check-prefixes=X86,X86-NOBMI2,X86-FALLBACK1 ; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=+cmov,+bmi,-tbm,-bmi2 < %s | FileCheck %s --check-prefixes=X86,X86-BMI
; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=+cmov,+bmi,+tbm,-bmi2 < %s | FileCheck %s --check-prefixes=X86,X86-NOBMI2,X86-FALLBACK2 ; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=+cmov,+bmi,+tbm,-bmi2 < %s | FileCheck %s --check-prefixes=X86,X86-BMI
; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=+cmov,+bmi,+tbm,+bmi2 < %s | FileCheck %s --check-prefixes=X86,X86-BMI2 ; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=+cmov,+bmi,+tbm,+bmi2 < %s | FileCheck %s --check-prefixes=X86,X86-BMI2
; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=+cmov,+bmi,-tbm,+bmi2 < %s | FileCheck %s --check-prefixes=X86,X86-BMI2 ; RUN: llc -mtriple=i686-unknown-linux-gnu -mattr=+cmov,+bmi,-tbm,+bmi2 < %s | FileCheck %s --check-prefixes=X86,X86-BMI2
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -mattr=-bmi,-tbm,-bmi2 < %s | FileCheck %s --check-prefixes=X64,X64-NOBMI2 ; RUN: llc -mtriple=x86_64-unknown-linux-gnu -mattr=-bmi,-tbm,-bmi2 < %s | FileCheck %s --check-prefixes=X64,X64-BASELINE
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -mattr=+bmi,-tbm,-bmi2 < %s | FileCheck %s --check-prefixes=X64,X64-NOBMI2 ; RUN: llc -mtriple=x86_64-unknown-linux-gnu -mattr=+bmi,-tbm,-bmi2 < %s | FileCheck %s --check-prefixes=X64,X64-BMI
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -mattr=+bmi,+tbm,-bmi2 < %s | FileCheck %s --check-prefixes=X64,X64-NOBMI2 ; RUN: llc -mtriple=x86_64-unknown-linux-gnu -mattr=+bmi,+tbm,-bmi2 < %s | FileCheck %s --check-prefixes=X64,X64-BMI
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -mattr=+bmi,+tbm,+bmi2 < %s | FileCheck %s --check-prefixes=X64,X64-BMI2 ; RUN: llc -mtriple=x86_64-unknown-linux-gnu -mattr=+bmi,+tbm,+bmi2 < %s | FileCheck %s --check-prefixes=X64,X64-BMI2
; RUN: llc -mtriple=x86_64-unknown-linux-gnu -mattr=+bmi,-tbm,+bmi2 < %s | FileCheck %s --check-prefixes=X64,X64-BMI2 ; RUN: llc -mtriple=x86_64-unknown-linux-gnu -mattr=+bmi,-tbm,+bmi2 < %s | FileCheck %s --check-prefixes=X64,X64-BMI2
; Patterns: ; Patterns:
; c) x & (-1 >> y) ; c) x & (-1 >> y)
; d) x << y >> y ; d) x << y >> y
; are equivalent, but we prefer the second variant if we have BMI2. ; are equivalent, but we prefer the second variant if we have BMI2.
▲ Show 20 Lines • Show All 73 Lines • ▼ Show 20 Lines; X64-NEXT: retq
ret i8 %masked ret i8 %masked
} }
; ---------------------------------------------------------------------------- ; ; ---------------------------------------------------------------------------- ;
; 16-bit ; 16-bit
; ---------------------------------------------------------------------------- ; ; ---------------------------------------------------------------------------- ;
define i16 @clear_highbits16_c0(i16 %val, i16 %numhighbits) nounwind { define i16 @clear_highbits16_c0(i16 %val, i16 %numhighbits) nounwind {
; X86-NOBMI2-LABEL: clear_highbits16_c0: ; X86-BASELINE-LABEL: clear_highbits16_c0:
; X86-NOBMI2: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-NOBMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-NOBMI2-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NOBMI2-NEXT: shll %cl, %eax ; X86-BASELINE-NEXT: shll %cl, %eax
; X86-NOBMI2-NEXT: movzwl %ax, %eax ; X86-BASELINE-NEXT: movzwl %ax, %eax
; X86-NOBMI2-NEXT: shrl %cl, %eax ; X86-BASELINE-NEXT: shrl %cl, %eax
; X86-NOBMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X86-BASELINE-NEXT: # kill: def $ax killed $ax killed $eax
; X86-NOBMI2-NEXT: retl ; X86-BASELINE-NEXT: retl
;
; X86-BMI-LABEL: clear_highbits16_c0:
; X86-BMI: # %bb.0:
; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-BMI-NEXT: shll %cl, %eax
; X86-BMI-NEXT: movzwl %ax, %eax
; X86-BMI-NEXT: shrl %cl, %eax
; X86-BMI-NEXT: # kill: def $ax killed $ax killed $eax
; X86-BMI-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits16_c0: ; X86-BMI2-LABEL: clear_highbits16_c0:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al
; X86-BMI2-NEXT: shlxl %eax, {{[0-9]+}}(%esp), %ecx ; X86-BMI2-NEXT: shlxl %eax, {{[0-9]+}}(%esp), %ecx
; X86-BMI2-NEXT: movzwl %cx, %ecx ; X86-BMI2-NEXT: movzwl %cx, %ecx
; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax ; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax
; X86-BMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X86-BMI2-NEXT: # kill: def $ax killed $ax killed $eax
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits16_c0: ; X64-BASELINE-LABEL: clear_highbits16_c0:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: shll %cl, %edi ; X64-BASELINE-NEXT: shll %cl, %edi
; X64-NOBMI2-NEXT: movzwl %di, %eax ; X64-BASELINE-NEXT: movzwl %di, %eax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrl %cl, %eax ; X64-BASELINE-NEXT: shrl %cl, %eax
; X64-NOBMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X64-BASELINE-NEXT: # kill: def $ax killed $ax killed $eax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits16_c0:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl %esi, %ecx
; X64-BMI-NEXT: shll %cl, %edi
; X64-BMI-NEXT: movzwl %di, %eax
; X64-BMI-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-BMI-NEXT: shrl %cl, %eax
; X64-BMI-NEXT: # kill: def $ax killed $ax killed $eax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits16_c0: ; X64-BMI2-LABEL: clear_highbits16_c0:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: shlxl %esi, %edi, %eax ; X64-BMI2-NEXT: shlxl %esi, %edi, %eax
; X64-BMI2-NEXT: movzwl %ax, %eax ; X64-BMI2-NEXT: movzwl %ax, %eax
; X64-BMI2-NEXT: shrxl %esi, %eax, %eax ; X64-BMI2-NEXT: shrxl %esi, %eax, %eax
; X64-BMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X64-BMI2-NEXT: # kill: def $ax killed $ax killed $eax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%mask = lshr i16 -1, %numhighbits %mask = lshr i16 -1, %numhighbits
%masked = and i16 %mask, %val %masked = and i16 %mask, %val
ret i16 %masked ret i16 %masked
} }
define i16 @clear_highbits16_c1_indexzext(i16 %val, i8 %numhighbits) nounwind { define i16 @clear_highbits16_c1_indexzext(i16 %val, i8 %numhighbits) nounwind {
; X86-NOBMI2-LABEL: clear_highbits16_c1_indexzext: ; X86-BASELINE-LABEL: clear_highbits16_c1_indexzext:
; X86-NOBMI2: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-NOBMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-NOBMI2-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NOBMI2-NEXT: shll %cl, %eax ; X86-BASELINE-NEXT: shll %cl, %eax
; X86-NOBMI2-NEXT: movzwl %ax, %eax ; X86-BASELINE-NEXT: movzwl %ax, %eax
; X86-NOBMI2-NEXT: shrl %cl, %eax ; X86-BASELINE-NEXT: shrl %cl, %eax
; X86-NOBMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X86-BASELINE-NEXT: # kill: def $ax killed $ax killed $eax
; X86-NOBMI2-NEXT: retl ; X86-BASELINE-NEXT: retl
;
; X86-BMI-LABEL: clear_highbits16_c1_indexzext:
; X86-BMI: # %bb.0:
; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-BMI-NEXT: shll %cl, %eax
; X86-BMI-NEXT: movzwl %ax, %eax
; X86-BMI-NEXT: shrl %cl, %eax
; X86-BMI-NEXT: # kill: def $ax killed $ax killed $eax
; X86-BMI-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits16_c1_indexzext: ; X86-BMI2-LABEL: clear_highbits16_c1_indexzext:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al
; X86-BMI2-NEXT: shlxl %eax, {{[0-9]+}}(%esp), %ecx ; X86-BMI2-NEXT: shlxl %eax, {{[0-9]+}}(%esp), %ecx
; X86-BMI2-NEXT: movzwl %cx, %ecx ; X86-BMI2-NEXT: movzwl %cx, %ecx
; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax ; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax
; X86-BMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X86-BMI2-NEXT: # kill: def $ax killed $ax killed $eax
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits16_c1_indexzext: ; X64-BASELINE-LABEL: clear_highbits16_c1_indexzext:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: shll %cl, %edi ; X64-BASELINE-NEXT: shll %cl, %edi
; X64-NOBMI2-NEXT: movzwl %di, %eax ; X64-BASELINE-NEXT: movzwl %di, %eax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrl %cl, %eax ; X64-BASELINE-NEXT: shrl %cl, %eax
; X64-NOBMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X64-BASELINE-NEXT: # kill: def $ax killed $ax killed $eax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits16_c1_indexzext:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl %esi, %ecx
; X64-BMI-NEXT: shll %cl, %edi
; X64-BMI-NEXT: movzwl %di, %eax
; X64-BMI-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-BMI-NEXT: shrl %cl, %eax
; X64-BMI-NEXT: # kill: def $ax killed $ax killed $eax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits16_c1_indexzext: ; X64-BMI2-LABEL: clear_highbits16_c1_indexzext:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: shlxl %esi, %edi, %eax ; X64-BMI2-NEXT: shlxl %esi, %edi, %eax
; X64-BMI2-NEXT: movzwl %ax, %eax ; X64-BMI2-NEXT: movzwl %ax, %eax
; X64-BMI2-NEXT: shrxl %esi, %eax, %eax ; X64-BMI2-NEXT: shrxl %esi, %eax, %eax
; X64-BMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X64-BMI2-NEXT: # kill: def $ax killed $ax killed $eax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%sh_prom = zext i8 %numhighbits to i16 %sh_prom = zext i8 %numhighbits to i16
%mask = lshr i16 -1, %sh_prom %mask = lshr i16 -1, %sh_prom
%masked = and i16 %mask, %val %masked = and i16 %mask, %val
ret i16 %masked ret i16 %masked
} }
define i16 @clear_highbits16_c2_load(i16* %w, i16 %numhighbits) nounwind { define i16 @clear_highbits16_c2_load(i16* %w, i16 %numhighbits) nounwind {
; X86-NOBMI2-LABEL: clear_highbits16_c2_load: ; X86-BASELINE-LABEL: clear_highbits16_c2_load:
; X86-NOBMI2: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-NOBMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-NOBMI2-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NOBMI2-NEXT: movzwl (%eax), %eax ; X86-BASELINE-NEXT: movzwl (%eax), %eax
; X86-NOBMI2-NEXT: shll %cl, %eax ; X86-BASELINE-NEXT: shll %cl, %eax
; X86-NOBMI2-NEXT: movzwl %ax, %eax ; X86-BASELINE-NEXT: movzwl %ax, %eax
; X86-NOBMI2-NEXT: shrl %cl, %eax ; X86-BASELINE-NEXT: shrl %cl, %eax
; X86-NOBMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X86-BASELINE-NEXT: # kill: def $ax killed $ax killed $eax
; X86-NOBMI2-NEXT: retl ; X86-BASELINE-NEXT: retl
;
; X86-BMI-LABEL: clear_highbits16_c2_load:
; X86-BMI: # %bb.0:
; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-BMI-NEXT: movzwl (%eax), %eax
; X86-BMI-NEXT: shll %cl, %eax
; X86-BMI-NEXT: movzwl %ax, %eax
; X86-BMI-NEXT: shrl %cl, %eax
; X86-BMI-NEXT: # kill: def $ax killed $ax killed $eax
; X86-BMI-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits16_c2_load: ; X86-BMI2-LABEL: clear_highbits16_c2_load:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al
; X86-BMI2-NEXT: movl {{[0-9]+}}(%esp), %ecx ; X86-BMI2-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-BMI2-NEXT: movzwl (%ecx), %ecx ; X86-BMI2-NEXT: movzwl (%ecx), %ecx
; X86-BMI2-NEXT: shlxl %eax, %ecx, %ecx ; X86-BMI2-NEXT: shlxl %eax, %ecx, %ecx
; X86-BMI2-NEXT: movzwl %cx, %ecx ; X86-BMI2-NEXT: movzwl %cx, %ecx
; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax ; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax
; X86-BMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X86-BMI2-NEXT: # kill: def $ax killed $ax killed $eax
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits16_c2_load: ; X64-BASELINE-LABEL: clear_highbits16_c2_load:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: movzwl (%rdi), %eax ; X64-BASELINE-NEXT: movzwl (%rdi), %eax
; X64-NOBMI2-NEXT: shll %cl, %eax ; X64-BASELINE-NEXT: shll %cl, %eax
; X64-NOBMI2-NEXT: movzwl %ax, %eax ; X64-BASELINE-NEXT: movzwl %ax, %eax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrl %cl, %eax ; X64-BASELINE-NEXT: shrl %cl, %eax
; X64-NOBMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X64-BASELINE-NEXT: # kill: def $ax killed $ax killed $eax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits16_c2_load:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl %esi, %ecx
; X64-BMI-NEXT: movzwl (%rdi), %eax
; X64-BMI-NEXT: shll %cl, %eax
; X64-BMI-NEXT: movzwl %ax, %eax
; X64-BMI-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-BMI-NEXT: shrl %cl, %eax
; X64-BMI-NEXT: # kill: def $ax killed $ax killed $eax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits16_c2_load: ; X64-BMI2-LABEL: clear_highbits16_c2_load:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: movzwl (%rdi), %eax ; X64-BMI2-NEXT: movzwl (%rdi), %eax
; X64-BMI2-NEXT: shlxl %esi, %eax, %eax ; X64-BMI2-NEXT: shlxl %esi, %eax, %eax
; X64-BMI2-NEXT: movzwl %ax, %eax ; X64-BMI2-NEXT: movzwl %ax, %eax
; X64-BMI2-NEXT: shrxl %esi, %eax, %eax ; X64-BMI2-NEXT: shrxl %esi, %eax, %eax
; X64-BMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X64-BMI2-NEXT: # kill: def $ax killed $ax killed $eax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%val = load i16, i16* %w %val = load i16, i16* %w
%mask = lshr i16 -1, %numhighbits %mask = lshr i16 -1, %numhighbits
%masked = and i16 %mask, %val %masked = and i16 %mask, %val
ret i16 %masked ret i16 %masked
} }
define i16 @clear_highbits16_c3_load_indexzext(i16* %w, i8 %numhighbits) nounwind { define i16 @clear_highbits16_c3_load_indexzext(i16* %w, i8 %numhighbits) nounwind {
; X86-NOBMI2-LABEL: clear_highbits16_c3_load_indexzext: ; X86-BASELINE-LABEL: clear_highbits16_c3_load_indexzext:
; X86-NOBMI2: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-NOBMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-NOBMI2-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NOBMI2-NEXT: movzwl (%eax), %eax ; X86-BASELINE-NEXT: movzwl (%eax), %eax
; X86-NOBMI2-NEXT: shll %cl, %eax ; X86-BASELINE-NEXT: shll %cl, %eax
; X86-NOBMI2-NEXT: movzwl %ax, %eax ; X86-BASELINE-NEXT: movzwl %ax, %eax
; X86-NOBMI2-NEXT: shrl %cl, %eax ; X86-BASELINE-NEXT: shrl %cl, %eax
; X86-NOBMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X86-BASELINE-NEXT: # kill: def $ax killed $ax killed $eax
; X86-NOBMI2-NEXT: retl ; X86-BASELINE-NEXT: retl
;
; X86-BMI-LABEL: clear_highbits16_c3_load_indexzext:
; X86-BMI: # %bb.0:
; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-BMI-NEXT: movzwl (%eax), %eax
; X86-BMI-NEXT: shll %cl, %eax
; X86-BMI-NEXT: movzwl %ax, %eax
; X86-BMI-NEXT: shrl %cl, %eax
; X86-BMI-NEXT: # kill: def $ax killed $ax killed $eax
; X86-BMI-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits16_c3_load_indexzext: ; X86-BMI2-LABEL: clear_highbits16_c3_load_indexzext:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al
; X86-BMI2-NEXT: movl {{[0-9]+}}(%esp), %ecx ; X86-BMI2-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-BMI2-NEXT: movzwl (%ecx), %ecx ; X86-BMI2-NEXT: movzwl (%ecx), %ecx
; X86-BMI2-NEXT: shlxl %eax, %ecx, %ecx ; X86-BMI2-NEXT: shlxl %eax, %ecx, %ecx
; X86-BMI2-NEXT: movzwl %cx, %ecx ; X86-BMI2-NEXT: movzwl %cx, %ecx
; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax ; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax
; X86-BMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X86-BMI2-NEXT: # kill: def $ax killed $ax killed $eax
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits16_c3_load_indexzext: ; X64-BASELINE-LABEL: clear_highbits16_c3_load_indexzext:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: movzwl (%rdi), %eax ; X64-BASELINE-NEXT: movzwl (%rdi), %eax
; X64-NOBMI2-NEXT: shll %cl, %eax ; X64-BASELINE-NEXT: shll %cl, %eax
; X64-NOBMI2-NEXT: movzwl %ax, %eax ; X64-BASELINE-NEXT: movzwl %ax, %eax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrl %cl, %eax ; X64-BASELINE-NEXT: shrl %cl, %eax
; X64-NOBMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X64-BASELINE-NEXT: # kill: def $ax killed $ax killed $eax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits16_c3_load_indexzext:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl %esi, %ecx
; X64-BMI-NEXT: movzwl (%rdi), %eax
; X64-BMI-NEXT: shll %cl, %eax
; X64-BMI-NEXT: movzwl %ax, %eax
; X64-BMI-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-BMI-NEXT: shrl %cl, %eax
; X64-BMI-NEXT: # kill: def $ax killed $ax killed $eax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits16_c3_load_indexzext: ; X64-BMI2-LABEL: clear_highbits16_c3_load_indexzext:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: movzwl (%rdi), %eax ; X64-BMI2-NEXT: movzwl (%rdi), %eax
; X64-BMI2-NEXT: shlxl %esi, %eax, %eax ; X64-BMI2-NEXT: shlxl %esi, %eax, %eax
; X64-BMI2-NEXT: movzwl %ax, %eax ; X64-BMI2-NEXT: movzwl %ax, %eax
; X64-BMI2-NEXT: shrxl %esi, %eax, %eax ; X64-BMI2-NEXT: shrxl %esi, %eax, %eax
; X64-BMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X64-BMI2-NEXT: # kill: def $ax killed $ax killed $eax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%val = load i16, i16* %w %val = load i16, i16* %w
%sh_prom = zext i8 %numhighbits to i16 %sh_prom = zext i8 %numhighbits to i16
%mask = lshr i16 -1, %sh_prom %mask = lshr i16 -1, %sh_prom
%masked = and i16 %mask, %val %masked = and i16 %mask, %val
ret i16 %masked ret i16 %masked
} }
define i16 @clear_highbits16_c4_commutative(i16 %val, i16 %numhighbits) nounwind { define i16 @clear_highbits16_c4_commutative(i16 %val, i16 %numhighbits) nounwind {
; X86-NOBMI2-LABEL: clear_highbits16_c4_commutative: ; X86-BASELINE-LABEL: clear_highbits16_c4_commutative:
; X86-NOBMI2: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-NOBMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-NOBMI2-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NOBMI2-NEXT: shll %cl, %eax ; X86-BASELINE-NEXT: shll %cl, %eax
; X86-NOBMI2-NEXT: movzwl %ax, %eax ; X86-BASELINE-NEXT: movzwl %ax, %eax
; X86-NOBMI2-NEXT: shrl %cl, %eax ; X86-BASELINE-NEXT: shrl %cl, %eax
; X86-NOBMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X86-BASELINE-NEXT: # kill: def $ax killed $ax killed $eax
; X86-NOBMI2-NEXT: retl ; X86-BASELINE-NEXT: retl
;
; X86-BMI-LABEL: clear_highbits16_c4_commutative:
; X86-BMI: # %bb.0:
; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-BMI-NEXT: shll %cl, %eax
; X86-BMI-NEXT: movzwl %ax, %eax
; X86-BMI-NEXT: shrl %cl, %eax
; X86-BMI-NEXT: # kill: def $ax killed $ax killed $eax
; X86-BMI-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits16_c4_commutative: ; X86-BMI2-LABEL: clear_highbits16_c4_commutative:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al
; X86-BMI2-NEXT: shlxl %eax, {{[0-9]+}}(%esp), %ecx ; X86-BMI2-NEXT: shlxl %eax, {{[0-9]+}}(%esp), %ecx
; X86-BMI2-NEXT: movzwl %cx, %ecx ; X86-BMI2-NEXT: movzwl %cx, %ecx
; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax ; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax
; X86-BMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X86-BMI2-NEXT: # kill: def $ax killed $ax killed $eax
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits16_c4_commutative: ; X64-BASELINE-LABEL: clear_highbits16_c4_commutative:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: shll %cl, %edi ; X64-BASELINE-NEXT: shll %cl, %edi
; X64-NOBMI2-NEXT: movzwl %di, %eax ; X64-BASELINE-NEXT: movzwl %di, %eax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrl %cl, %eax ; X64-BASELINE-NEXT: shrl %cl, %eax
; X64-NOBMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X64-BASELINE-NEXT: # kill: def $ax killed $ax killed $eax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits16_c4_commutative:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl %esi, %ecx
; X64-BMI-NEXT: shll %cl, %edi
; X64-BMI-NEXT: movzwl %di, %eax
; X64-BMI-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-BMI-NEXT: shrl %cl, %eax
; X64-BMI-NEXT: # kill: def $ax killed $ax killed $eax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits16_c4_commutative: ; X64-BMI2-LABEL: clear_highbits16_c4_commutative:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: shlxl %esi, %edi, %eax ; X64-BMI2-NEXT: shlxl %esi, %edi, %eax
; X64-BMI2-NEXT: movzwl %ax, %eax ; X64-BMI2-NEXT: movzwl %ax, %eax
; X64-BMI2-NEXT: shrxl %esi, %eax, %eax ; X64-BMI2-NEXT: shrxl %esi, %eax, %eax
; X64-BMI2-NEXT: # kill: def $ax killed $ax killed $eax ; X64-BMI2-NEXT: # kill: def $ax killed $ax killed $eax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%mask = lshr i16 -1, %numhighbits %mask = lshr i16 -1, %numhighbits
%masked = and i16 %val, %mask ; swapped order %masked = and i16 %val, %mask ; swapped order
ret i16 %masked ret i16 %masked
} }
; ---------------------------------------------------------------------------- ; ; ---------------------------------------------------------------------------- ;
; 32-bit ; 32-bit
; ---------------------------------------------------------------------------- ; ; ---------------------------------------------------------------------------- ;
define i32 @clear_highbits32_c0(i32 %val, i32 %numhighbits) nounwind { define i32 @clear_highbits32_c0(i32 %val, i32 %numhighbits) nounwind {
; X86-NOBMI2-LABEL: clear_highbits32_c0: ; X86-BASELINE-LABEL: clear_highbits32_c0:
; X86-NOBMI2: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-NOBMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-NOBMI2-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NOBMI2-NEXT: shll %cl, %eax ; X86-BASELINE-NEXT: shll %cl, %eax
; X86-NOBMI2-NEXT: shrl %cl, %eax ; X86-BASELINE-NEXT: shrl %cl, %eax
; X86-NOBMI2-NEXT: retl ; X86-BASELINE-NEXT: retl
;
; X86-BMI-LABEL: clear_highbits32_c0:
; X86-BMI: # %bb.0:
; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %al
; X86-BMI-NEXT: movl $32, %ecx
; X86-BMI-NEXT: subl %eax, %ecx
; X86-BMI-NEXT: shll $8, %ecx
; X86-BMI-NEXT: bextrl %ecx, {{[0-9]+}}(%esp), %eax
; X86-BMI-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits32_c0: ; X86-BMI2-LABEL: clear_highbits32_c0:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al
; X86-BMI2-NEXT: shlxl %eax, {{[0-9]+}}(%esp), %ecx ; X86-BMI2-NEXT: movl $32, %ecx
; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax ; X86-BMI2-NEXT: subl %eax, %ecx
; X86-BMI2-NEXT: bzhil %ecx, {{[0-9]+}}(%esp), %eax
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits32_c0: ; X64-BASELINE-LABEL: clear_highbits32_c0:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: movl %edi, %eax ; X64-BASELINE-NEXT: movl %edi, %eax
; X64-NOBMI2-NEXT: shll %cl, %eax ; X64-BASELINE-NEXT: shll %cl, %eax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrl %cl, %eax ; X64-BASELINE-NEXT: shrl %cl, %eax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits32_c0:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl $32, %eax
; X64-BMI-NEXT: subl %esi, %eax
; X64-BMI-NEXT: shll $8, %eax
; X64-BMI-NEXT: bextrl %eax, %edi, %eax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits32_c0: ; X64-BMI2-LABEL: clear_highbits32_c0:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: shlxl %esi, %edi, %eax ; X64-BMI2-NEXT: movl $32, %eax
; X64-BMI2-NEXT: shrxl %esi, %eax, %eax ; X64-BMI2-NEXT: subl %esi, %eax
; X64-BMI2-NEXT: bzhil %eax, %edi, %eax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%mask = lshr i32 -1, %numhighbits %mask = lshr i32 -1, %numhighbits
%masked = and i32 %mask, %val %masked = and i32 %mask, %val
ret i32 %masked ret i32 %masked
} }
define i32 @clear_highbits32_c1_indexzext(i32 %val, i8 %numhighbits) nounwind { define i32 @clear_highbits32_c1_indexzext(i32 %val, i8 %numhighbits) nounwind {
; X86-NOBMI2-LABEL: clear_highbits32_c1_indexzext: ; X86-BASELINE-LABEL: clear_highbits32_c1_indexzext:
; X86-NOBMI2: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-NOBMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-NOBMI2-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NOBMI2-NEXT: shll %cl, %eax ; X86-BASELINE-NEXT: shll %cl, %eax
; X86-NOBMI2-NEXT: shrl %cl, %eax ; X86-BASELINE-NEXT: shrl %cl, %eax
; X86-NOBMI2-NEXT: retl ; X86-BASELINE-NEXT: retl
;
; X86-BMI-LABEL: clear_highbits32_c1_indexzext:
; X86-BMI: # %bb.0:
; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %al
; X86-BMI-NEXT: movl $32, %ecx
; X86-BMI-NEXT: subl %eax, %ecx
; X86-BMI-NEXT: shll $8, %ecx
; X86-BMI-NEXT: bextrl %ecx, {{[0-9]+}}(%esp), %eax
; X86-BMI-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits32_c1_indexzext: ; X86-BMI2-LABEL: clear_highbits32_c1_indexzext:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al
; X86-BMI2-NEXT: shlxl %eax, {{[0-9]+}}(%esp), %ecx ; X86-BMI2-NEXT: movl $32, %ecx
; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax ; X86-BMI2-NEXT: subl %eax, %ecx
; X86-BMI2-NEXT: bzhil %ecx, {{[0-9]+}}(%esp), %eax
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits32_c1_indexzext: ; X64-BASELINE-LABEL: clear_highbits32_c1_indexzext:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: movl %edi, %eax ; X64-BASELINE-NEXT: movl %edi, %eax
; X64-NOBMI2-NEXT: shll %cl, %eax ; X64-BASELINE-NEXT: shll %cl, %eax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrl %cl, %eax ; X64-BASELINE-NEXT: shrl %cl, %eax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits32_c1_indexzext:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl $32, %eax
; X64-BMI-NEXT: subl %esi, %eax
; X64-BMI-NEXT: shll $8, %eax
; X64-BMI-NEXT: bextrl %eax, %edi, %eax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits32_c1_indexzext: ; X64-BMI2-LABEL: clear_highbits32_c1_indexzext:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: shlxl %esi, %edi, %eax ; X64-BMI2-NEXT: movl $32, %eax
; X64-BMI2-NEXT: shrxl %esi, %eax, %eax ; X64-BMI2-NEXT: subl %esi, %eax
; X64-BMI2-NEXT: bzhil %eax, %edi, %eax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%sh_prom = zext i8 %numhighbits to i32 %sh_prom = zext i8 %numhighbits to i32
%mask = lshr i32 -1, %sh_prom %mask = lshr i32 -1, %sh_prom
%masked = and i32 %mask, %val %masked = and i32 %mask, %val
ret i32 %masked ret i32 %masked
} }
define i32 @clear_highbits32_c2_load(i32* %w, i32 %numhighbits) nounwind { define i32 @clear_highbits32_c2_load(i32* %w, i32 %numhighbits) nounwind {
; X86-NOBMI2-LABEL: clear_highbits32_c2_load: ; X86-BASELINE-LABEL: clear_highbits32_c2_load:
; X86-NOBMI2: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-NOBMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-NOBMI2-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NOBMI2-NEXT: movl (%eax), %eax ; X86-BASELINE-NEXT: movl (%eax), %eax
; X86-NOBMI2-NEXT: shll %cl, %eax ; X86-BASELINE-NEXT: shll %cl, %eax
; X86-NOBMI2-NEXT: shrl %cl, %eax ; X86-BASELINE-NEXT: shrl %cl, %eax
; X86-NOBMI2-NEXT: retl ; X86-BASELINE-NEXT: retl
;
; X86-BMI-LABEL: clear_highbits32_c2_load:
; X86-BMI: # %bb.0:
; X86-BMI-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI-NEXT: movl $32, %edx
; X86-BMI-NEXT: subl %ecx, %edx
; X86-BMI-NEXT: shll $8, %edx
; X86-BMI-NEXT: bextrl %edx, (%eax), %eax
; X86-BMI-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits32_c2_load: ; X86-BMI2-LABEL: clear_highbits32_c2_load:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-BMI2-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI2-NEXT: shlxl %ecx, (%eax), %eax ; X86-BMI2-NEXT: movl $32, %edx
; X86-BMI2-NEXT: shrxl %ecx, %eax, %eax ; X86-BMI2-NEXT: subl %ecx, %edx
; X86-BMI2-NEXT: bzhil %edx, (%eax), %eax
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits32_c2_load: ; X64-BASELINE-LABEL: clear_highbits32_c2_load:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: movl (%rdi), %eax ; X64-BASELINE-NEXT: movl (%rdi), %eax
; X64-NOBMI2-NEXT: shll %cl, %eax ; X64-BASELINE-NEXT: shll %cl, %eax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrl %cl, %eax ; X64-BASELINE-NEXT: shrl %cl, %eax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits32_c2_load:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl $32, %eax
; X64-BMI-NEXT: subl %esi, %eax
; X64-BMI-NEXT: shll $8, %eax
; X64-BMI-NEXT: bextrl %eax, (%rdi), %eax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits32_c2_load: ; X64-BMI2-LABEL: clear_highbits32_c2_load:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: shlxl %esi, (%rdi), %eax ; X64-BMI2-NEXT: movl $32, %eax
; X64-BMI2-NEXT: shrxl %esi, %eax, %eax ; X64-BMI2-NEXT: subl %esi, %eax
; X64-BMI2-NEXT: bzhil %eax, (%rdi), %eax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%val = load i32, i32* %w %val = load i32, i32* %w
%mask = lshr i32 -1, %numhighbits %mask = lshr i32 -1, %numhighbits
%masked = and i32 %mask, %val %masked = and i32 %mask, %val
ret i32 %masked ret i32 %masked
} }
define i32 @clear_highbits32_c3_load_indexzext(i32* %w, i8 %numhighbits) nounwind { define i32 @clear_highbits32_c3_load_indexzext(i32* %w, i8 %numhighbits) nounwind {
; X86-NOBMI2-LABEL: clear_highbits32_c3_load_indexzext: ; X86-BASELINE-LABEL: clear_highbits32_c3_load_indexzext:
; X86-NOBMI2: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-NOBMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-NOBMI2-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NOBMI2-NEXT: movl (%eax), %eax ; X86-BASELINE-NEXT: movl (%eax), %eax
; X86-NOBMI2-NEXT: shll %cl, %eax ; X86-BASELINE-NEXT: shll %cl, %eax
; X86-NOBMI2-NEXT: shrl %cl, %eax ; X86-BASELINE-NEXT: shrl %cl, %eax
; X86-NOBMI2-NEXT: retl ; X86-BASELINE-NEXT: retl
;
; X86-BMI-LABEL: clear_highbits32_c3_load_indexzext:
; X86-BMI: # %bb.0:
; X86-BMI-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI-NEXT: movl $32, %edx
; X86-BMI-NEXT: subl %ecx, %edx
; X86-BMI-NEXT: shll $8, %edx
; X86-BMI-NEXT: bextrl %edx, (%eax), %eax
; X86-BMI-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits32_c3_load_indexzext: ; X86-BMI2-LABEL: clear_highbits32_c3_load_indexzext:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-BMI2-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI2-NEXT: shlxl %ecx, (%eax), %eax ; X86-BMI2-NEXT: movl $32, %edx
; X86-BMI2-NEXT: shrxl %ecx, %eax, %eax ; X86-BMI2-NEXT: subl %ecx, %edx
; X86-BMI2-NEXT: bzhil %edx, (%eax), %eax
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits32_c3_load_indexzext: ; X64-BASELINE-LABEL: clear_highbits32_c3_load_indexzext:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: movl (%rdi), %eax ; X64-BASELINE-NEXT: movl (%rdi), %eax
; X64-NOBMI2-NEXT: shll %cl, %eax ; X64-BASELINE-NEXT: shll %cl, %eax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrl %cl, %eax ; X64-BASELINE-NEXT: shrl %cl, %eax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits32_c3_load_indexzext:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl $32, %eax
; X64-BMI-NEXT: subl %esi, %eax
; X64-BMI-NEXT: shll $8, %eax
; X64-BMI-NEXT: bextrl %eax, (%rdi), %eax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits32_c3_load_indexzext: ; X64-BMI2-LABEL: clear_highbits32_c3_load_indexzext:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: shlxl %esi, (%rdi), %eax ; X64-BMI2-NEXT: movl $32, %eax
; X64-BMI2-NEXT: shrxl %esi, %eax, %eax ; X64-BMI2-NEXT: subl %esi, %eax
; X64-BMI2-NEXT: bzhil %eax, (%rdi), %eax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%val = load i32, i32* %w %val = load i32, i32* %w
%sh_prom = zext i8 %numhighbits to i32 %sh_prom = zext i8 %numhighbits to i32
%mask = lshr i32 -1, %sh_prom %mask = lshr i32 -1, %sh_prom
%masked = and i32 %mask, %val %masked = and i32 %mask, %val
ret i32 %masked ret i32 %masked
} }
define i32 @clear_highbits32_c4_commutative(i32 %val, i32 %numhighbits) nounwind { define i32 @clear_highbits32_c4_commutative(i32 %val, i32 %numhighbits) nounwind {
; X86-NOBMI2-LABEL: clear_highbits32_c4_commutative: ; X86-BASELINE-LABEL: clear_highbits32_c4_commutative:
; X86-NOBMI2: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-NOBMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-NOBMI2-NEXT: movl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: movl {{[0-9]+}}(%esp), %eax
; X86-NOBMI2-NEXT: shll %cl, %eax ; X86-BASELINE-NEXT: shll %cl, %eax
; X86-NOBMI2-NEXT: shrl %cl, %eax ; X86-BASELINE-NEXT: shrl %cl, %eax
; X86-NOBMI2-NEXT: retl ; X86-BASELINE-NEXT: retl
;
; X86-BMI-LABEL: clear_highbits32_c4_commutative:
; X86-BMI: # %bb.0:
; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %al
; X86-BMI-NEXT: movl $32, %ecx
; X86-BMI-NEXT: subl %eax, %ecx
; X86-BMI-NEXT: shll $8, %ecx
; X86-BMI-NEXT: bextrl %ecx, {{[0-9]+}}(%esp), %eax
; X86-BMI-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits32_c4_commutative: ; X86-BMI2-LABEL: clear_highbits32_c4_commutative:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al
; X86-BMI2-NEXT: shlxl %eax, {{[0-9]+}}(%esp), %ecx ; X86-BMI2-NEXT: movl $32, %ecx
; X86-BMI2-NEXT: shrxl %eax, %ecx, %eax ; X86-BMI2-NEXT: subl %eax, %ecx
; X86-BMI2-NEXT: bzhil %ecx, {{[0-9]+}}(%esp), %eax
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits32_c4_commutative: ; X64-BASELINE-LABEL: clear_highbits32_c4_commutative:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: movl %edi, %eax ; X64-BASELINE-NEXT: movl %edi, %eax
; X64-NOBMI2-NEXT: shll %cl, %eax ; X64-BASELINE-NEXT: shll %cl, %eax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrl %cl, %eax ; X64-BASELINE-NEXT: shrl %cl, %eax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits32_c4_commutative:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl $32, %eax
; X64-BMI-NEXT: subl %esi, %eax
; X64-BMI-NEXT: shll $8, %eax
; X64-BMI-NEXT: bextrl %eax, %edi, %eax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits32_c4_commutative: ; X64-BMI2-LABEL: clear_highbits32_c4_commutative:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: shlxl %esi, %edi, %eax ; X64-BMI2-NEXT: movl $32, %eax
; X64-BMI2-NEXT: shrxl %esi, %eax, %eax ; X64-BMI2-NEXT: subl %esi, %eax
; X64-BMI2-NEXT: bzhil %eax, %edi, %eax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%mask = lshr i32 -1, %numhighbits %mask = lshr i32 -1, %numhighbits
%masked = and i32 %val, %mask ; swapped order %masked = and i32 %val, %mask ; swapped order
ret i32 %masked ret i32 %masked
} }
; ---------------------------------------------------------------------------- ; ; ---------------------------------------------------------------------------- ;
; 64-bit ; 64-bit
; ---------------------------------------------------------------------------- ; ; ---------------------------------------------------------------------------- ;
define i64 @clear_highbits64_c0(i64 %val, i64 %numhighbits) nounwind { define i64 @clear_highbits64_c0(i64 %val, i64 %numhighbits) nounwind {
; X86-FALLBACK0-LABEL: clear_highbits64_c0: ; X86-BASELINE-LABEL: clear_highbits64_c0:
; X86-FALLBACK0: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-FALLBACK0-NEXT: pushl %esi ; X86-BASELINE-NEXT: pushl %esi
; X86-FALLBACK0-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK0-NEXT: movl $-1, %eax ; X86-BASELINE-NEXT: movl $-1, %eax
; X86-FALLBACK0-NEXT: movl $-1, %esi ; X86-BASELINE-NEXT: movl $-1, %esi
; X86-FALLBACK0-NEXT: shrl %cl, %esi ; X86-BASELINE-NEXT: shrl %cl, %esi
; X86-FALLBACK0-NEXT: xorl %edx, %edx ; X86-BASELINE-NEXT: xorl %edx, %edx
; X86-FALLBACK0-NEXT: testb $32, %cl ; X86-BASELINE-NEXT: testb $32, %cl
; X86-FALLBACK0-NEXT: jne .LBB13_1 ; X86-BASELINE-NEXT: jne .LBB13_1
; X86-FALLBACK0-NEXT: # %bb.2: ; X86-BASELINE-NEXT: # %bb.2:
; X86-FALLBACK0-NEXT: movl %esi, %edx ; X86-BASELINE-NEXT: movl %esi, %edx
; X86-FALLBACK0-NEXT: jmp .LBB13_3 ; X86-BASELINE-NEXT: jmp .LBB13_3
; X86-FALLBACK0-NEXT: .LBB13_1: ; X86-BASELINE-NEXT: .LBB13_1:
; X86-FALLBACK0-NEXT: movl %esi, %eax ; X86-BASELINE-NEXT: movl %esi, %eax
; X86-FALLBACK0-NEXT: .LBB13_3: ; X86-BASELINE-NEXT: .LBB13_3:
; X86-FALLBACK0-NEXT: andl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: andl {{[0-9]+}}(%esp), %eax
; X86-FALLBACK0-NEXT: andl {{[0-9]+}}(%esp), %edx ; X86-BASELINE-NEXT: andl {{[0-9]+}}(%esp), %edx
; X86-FALLBACK0-NEXT: popl %esi ; X86-BASELINE-NEXT: popl %esi
; X86-FALLBACK0-NEXT: retl ; X86-BASELINE-NEXT: retl
; ;
; X86-FALLBACK1-LABEL: clear_highbits64_c0: ; X86-BMI-LABEL: clear_highbits64_c0:
; X86-FALLBACK1: # %bb.0: ; X86-BMI: # %bb.0:
; X86-FALLBACK1-NEXT: pushl %esi ; X86-BMI-NEXT: pushl %esi
; X86-FALLBACK1-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK1-NEXT: movl $-1, %esi ; X86-BMI-NEXT: movl $-1, %esi
; X86-FALLBACK1-NEXT: movl $-1, %eax ; X86-BMI-NEXT: movl $-1, %eax
; X86-FALLBACK1-NEXT: shrl %cl, %eax ; X86-BMI-NEXT: shrl %cl, %eax
; X86-FALLBACK1-NEXT: xorl %edx, %edx ; X86-BMI-NEXT: xorl %edx, %edx
; X86-FALLBACK1-NEXT: testb $32, %cl ; X86-BMI-NEXT: testb $32, %cl
; X86-FALLBACK1-NEXT: cmovel %eax, %edx ; X86-BMI-NEXT: cmovel %eax, %edx
; X86-FALLBACK1-NEXT: cmovel %esi, %eax ; X86-BMI-NEXT: cmovel %esi, %eax
; X86-FALLBACK1-NEXT: andl {{[0-9]+}}(%esp), %eax ; X86-BMI-NEXT: andl {{[0-9]+}}(%esp), %eax
; X86-FALLBACK1-NEXT: andl {{[0-9]+}}(%esp), %edx ; X86-BMI-NEXT: andl {{[0-9]+}}(%esp), %edx
; X86-FALLBACK1-NEXT: popl %esi ; X86-BMI-NEXT: popl %esi
; X86-FALLBACK1-NEXT: retl ; X86-BMI-NEXT: retl
;
; X86-FALLBACK2-LABEL: clear_highbits64_c0:
; X86-FALLBACK2: # %bb.0:
; X86-FALLBACK2-NEXT: pushl %esi
; X86-FALLBACK2-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK2-NEXT: movl $-1, %esi
; X86-FALLBACK2-NEXT: movl $-1, %eax
; X86-FALLBACK2-NEXT: shrl %cl, %eax
; X86-FALLBACK2-NEXT: xorl %edx, %edx
; X86-FALLBACK2-NEXT: testb $32, %cl
; X86-FALLBACK2-NEXT: cmovel %eax, %edx
; X86-FALLBACK2-NEXT: cmovel %esi, %eax
; X86-FALLBACK2-NEXT: andl {{[0-9]+}}(%esp), %eax
; X86-FALLBACK2-NEXT: andl {{[0-9]+}}(%esp), %edx
; X86-FALLBACK2-NEXT: popl %esi
; X86-FALLBACK2-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits64_c0: ; X86-BMI2-LABEL: clear_highbits64_c0:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: pushl %esi ; X86-BMI2-NEXT: pushl %esi
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI2-NEXT: movl $-1, %eax ; X86-BMI2-NEXT: movl $-1, %eax
; X86-BMI2-NEXT: shrxl %ecx, %eax, %esi ; X86-BMI2-NEXT: shrxl %ecx, %eax, %esi
; X86-BMI2-NEXT: xorl %edx, %edx ; X86-BMI2-NEXT: xorl %edx, %edx
; X86-BMI2-NEXT: testb $32, %cl ; X86-BMI2-NEXT: testb $32, %cl
; X86-BMI2-NEXT: cmovel %esi, %edx ; X86-BMI2-NEXT: cmovel %esi, %edx
; X86-BMI2-NEXT: cmovnel %esi, %eax ; X86-BMI2-NEXT: cmovnel %esi, %eax
; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %eax ; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %eax
; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %edx ; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %edx
; X86-BMI2-NEXT: popl %esi ; X86-BMI2-NEXT: popl %esi
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits64_c0: ; X64-BASELINE-LABEL: clear_highbits64_c0:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movq %rsi, %rcx ; X64-BASELINE-NEXT: movq %rsi, %rcx
; X64-NOBMI2-NEXT: movq %rdi, %rax ; X64-BASELINE-NEXT: movq %rdi, %rax
; X64-NOBMI2-NEXT: shlq %cl, %rax ; X64-BASELINE-NEXT: shlq %cl, %rax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $rcx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $rcx
; X64-NOBMI2-NEXT: shrq %cl, %rax ; X64-BASELINE-NEXT: shrq %cl, %rax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits64_c0:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl $64, %eax
; X64-BMI-NEXT: subl %esi, %eax
; X64-BMI-NEXT: shll $8, %eax
; X64-BMI-NEXT: bextrq %rax, %rdi, %rax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits64_c0: ; X64-BMI2-LABEL: clear_highbits64_c0:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: shlxq %rsi, %rdi, %rax ; X64-BMI2-NEXT: movl $64, %eax
; X64-BMI2-NEXT: shrxq %rsi, %rax, %rax ; X64-BMI2-NEXT: subl %esi, %eax
; X64-BMI2-NEXT: bzhiq %rax, %rdi, %rax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%mask = lshr i64 -1, %numhighbits %mask = lshr i64 -1, %numhighbits
%masked = and i64 %mask, %val %masked = and i64 %mask, %val
ret i64 %masked ret i64 %masked
} }
define i64 @clear_highbits64_c1_indexzext(i64 %val, i8 %numhighbits) nounwind { define i64 @clear_highbits64_c1_indexzext(i64 %val, i8 %numhighbits) nounwind {
; X86-FALLBACK0-LABEL: clear_highbits64_c1_indexzext: ; X86-BASELINE-LABEL: clear_highbits64_c1_indexzext:
; X86-FALLBACK0: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-FALLBACK0-NEXT: pushl %esi ; X86-BASELINE-NEXT: pushl %esi
; X86-FALLBACK0-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK0-NEXT: movl $-1, %eax ; X86-BASELINE-NEXT: movl $-1, %eax
; X86-FALLBACK0-NEXT: movl $-1, %esi ; X86-BASELINE-NEXT: movl $-1, %esi
; X86-FALLBACK0-NEXT: shrl %cl, %esi ; X86-BASELINE-NEXT: shrl %cl, %esi
; X86-FALLBACK0-NEXT: xorl %edx, %edx ; X86-BASELINE-NEXT: xorl %edx, %edx
; X86-FALLBACK0-NEXT: testb $32, %cl ; X86-BASELINE-NEXT: testb $32, %cl
; X86-FALLBACK0-NEXT: jne .LBB14_1 ; X86-BASELINE-NEXT: jne .LBB14_1
; X86-FALLBACK0-NEXT: # %bb.2: ; X86-BASELINE-NEXT: # %bb.2:
; X86-FALLBACK0-NEXT: movl %esi, %edx ; X86-BASELINE-NEXT: movl %esi, %edx
; X86-FALLBACK0-NEXT: jmp .LBB14_3 ; X86-BASELINE-NEXT: jmp .LBB14_3
; X86-FALLBACK0-NEXT: .LBB14_1: ; X86-BASELINE-NEXT: .LBB14_1:
; X86-FALLBACK0-NEXT: movl %esi, %eax ; X86-BASELINE-NEXT: movl %esi, %eax
; X86-FALLBACK0-NEXT: .LBB14_3: ; X86-BASELINE-NEXT: .LBB14_3:
; X86-FALLBACK0-NEXT: andl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: andl {{[0-9]+}}(%esp), %eax
; X86-FALLBACK0-NEXT: andl {{[0-9]+}}(%esp), %edx ; X86-BASELINE-NEXT: andl {{[0-9]+}}(%esp), %edx
; X86-FALLBACK0-NEXT: popl %esi ; X86-BASELINE-NEXT: popl %esi
; X86-FALLBACK0-NEXT: retl ; X86-BASELINE-NEXT: retl
; ;
; X86-FALLBACK1-LABEL: clear_highbits64_c1_indexzext: ; X86-BMI-LABEL: clear_highbits64_c1_indexzext:
; X86-FALLBACK1: # %bb.0: ; X86-BMI: # %bb.0:
; X86-FALLBACK1-NEXT: pushl %esi ; X86-BMI-NEXT: pushl %esi
; X86-FALLBACK1-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK1-NEXT: movl $-1, %esi ; X86-BMI-NEXT: movl $-1, %esi
; X86-FALLBACK1-NEXT: movl $-1, %eax ; X86-BMI-NEXT: movl $-1, %eax
; X86-FALLBACK1-NEXT: shrl %cl, %eax ; X86-BMI-NEXT: shrl %cl, %eax
; X86-FALLBACK1-NEXT: xorl %edx, %edx ; X86-BMI-NEXT: xorl %edx, %edx
; X86-FALLBACK1-NEXT: testb $32, %cl ; X86-BMI-NEXT: testb $32, %cl
; X86-FALLBACK1-NEXT: cmovel %eax, %edx ; X86-BMI-NEXT: cmovel %eax, %edx
; X86-FALLBACK1-NEXT: cmovel %esi, %eax ; X86-BMI-NEXT: cmovel %esi, %eax
; X86-FALLBACK1-NEXT: andl {{[0-9]+}}(%esp), %eax ; X86-BMI-NEXT: andl {{[0-9]+}}(%esp), %eax
; X86-FALLBACK1-NEXT: andl {{[0-9]+}}(%esp), %edx ; X86-BMI-NEXT: andl {{[0-9]+}}(%esp), %edx
; X86-FALLBACK1-NEXT: popl %esi ; X86-BMI-NEXT: popl %esi
; X86-FALLBACK1-NEXT: retl ; X86-BMI-NEXT: retl
;
; X86-FALLBACK2-LABEL: clear_highbits64_c1_indexzext:
; X86-FALLBACK2: # %bb.0:
; X86-FALLBACK2-NEXT: pushl %esi
; X86-FALLBACK2-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK2-NEXT: movl $-1, %esi
; X86-FALLBACK2-NEXT: movl $-1, %eax
; X86-FALLBACK2-NEXT: shrl %cl, %eax
; X86-FALLBACK2-NEXT: xorl %edx, %edx
; X86-FALLBACK2-NEXT: testb $32, %cl
; X86-FALLBACK2-NEXT: cmovel %eax, %edx
; X86-FALLBACK2-NEXT: cmovel %esi, %eax
; X86-FALLBACK2-NEXT: andl {{[0-9]+}}(%esp), %eax
; X86-FALLBACK2-NEXT: andl {{[0-9]+}}(%esp), %edx
; X86-FALLBACK2-NEXT: popl %esi
; X86-FALLBACK2-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits64_c1_indexzext: ; X86-BMI2-LABEL: clear_highbits64_c1_indexzext:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: pushl %esi ; X86-BMI2-NEXT: pushl %esi
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI2-NEXT: movl $-1, %eax ; X86-BMI2-NEXT: movl $-1, %eax
; X86-BMI2-NEXT: shrxl %ecx, %eax, %esi ; X86-BMI2-NEXT: shrxl %ecx, %eax, %esi
; X86-BMI2-NEXT: xorl %edx, %edx ; X86-BMI2-NEXT: xorl %edx, %edx
; X86-BMI2-NEXT: testb $32, %cl ; X86-BMI2-NEXT: testb $32, %cl
; X86-BMI2-NEXT: cmovel %esi, %edx ; X86-BMI2-NEXT: cmovel %esi, %edx
; X86-BMI2-NEXT: cmovnel %esi, %eax ; X86-BMI2-NEXT: cmovnel %esi, %eax
; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %eax ; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %eax
; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %edx ; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %edx
; X86-BMI2-NEXT: popl %esi ; X86-BMI2-NEXT: popl %esi
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits64_c1_indexzext: ; X64-BASELINE-LABEL: clear_highbits64_c1_indexzext:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: movq %rdi, %rax ; X64-BASELINE-NEXT: movq %rdi, %rax
; X64-NOBMI2-NEXT: shlq %cl, %rax ; X64-BASELINE-NEXT: shlq %cl, %rax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrq %cl, %rax ; X64-BASELINE-NEXT: shrq %cl, %rax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits64_c1_indexzext:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl $64, %eax
; X64-BMI-NEXT: subl %esi, %eax
; X64-BMI-NEXT: shll $8, %eax
; X64-BMI-NEXT: bextrq %rax, %rdi, %rax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits64_c1_indexzext: ; X64-BMI2-LABEL: clear_highbits64_c1_indexzext:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: # kill: def $esi killed $esi def $rsi ; X64-BMI2-NEXT: movl $64, %eax
; X64-BMI2-NEXT: shlxq %rsi, %rdi, %rax ; X64-BMI2-NEXT: subl %esi, %eax
; X64-BMI2-NEXT: shrxq %rsi, %rax, %rax ; X64-BMI2-NEXT: bzhiq %rax, %rdi, %rax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%sh_prom = zext i8 %numhighbits to i64 %sh_prom = zext i8 %numhighbits to i64
%mask = lshr i64 -1, %sh_prom %mask = lshr i64 -1, %sh_prom
%masked = and i64 %mask, %val %masked = and i64 %mask, %val
ret i64 %masked ret i64 %masked
} }
define i64 @clear_highbits64_c2_load(i64* %w, i64 %numhighbits) nounwind { define i64 @clear_highbits64_c2_load(i64* %w, i64 %numhighbits) nounwind {
; X86-FALLBACK0-LABEL: clear_highbits64_c2_load: ; X86-BASELINE-LABEL: clear_highbits64_c2_load:
; X86-FALLBACK0: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-FALLBACK0-NEXT: pushl %edi ; X86-BASELINE-NEXT: pushl %edi
; X86-FALLBACK0-NEXT: pushl %esi ; X86-BASELINE-NEXT: pushl %esi
; X86-FALLBACK0-NEXT: movl {{[0-9]+}}(%esp), %esi ; X86-BASELINE-NEXT: movl {{[0-9]+}}(%esp), %esi
; X86-FALLBACK0-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK0-NEXT: movl $-1, %eax ; X86-BASELINE-NEXT: movl $-1, %eax
; X86-FALLBACK0-NEXT: movl $-1, %edi ; X86-BASELINE-NEXT: movl $-1, %edi
; X86-FALLBACK0-NEXT: shrl %cl, %edi ; X86-BASELINE-NEXT: shrl %cl, %edi
; X86-FALLBACK0-NEXT: xorl %edx, %edx ; X86-BASELINE-NEXT: xorl %edx, %edx
; X86-FALLBACK0-NEXT: testb $32, %cl ; X86-BASELINE-NEXT: testb $32, %cl
; X86-FALLBACK0-NEXT: jne .LBB15_1 ; X86-BASELINE-NEXT: jne .LBB15_1
; X86-FALLBACK0-NEXT: # %bb.2: ; X86-BASELINE-NEXT: # %bb.2:
; X86-FALLBACK0-NEXT: movl %edi, %edx ; X86-BASELINE-NEXT: movl %edi, %edx
; X86-FALLBACK0-NEXT: jmp .LBB15_3 ; X86-BASELINE-NEXT: jmp .LBB15_3
; X86-FALLBACK0-NEXT: .LBB15_1: ; X86-BASELINE-NEXT: .LBB15_1:
; X86-FALLBACK0-NEXT: movl %edi, %eax ; X86-BASELINE-NEXT: movl %edi, %eax
; X86-FALLBACK0-NEXT: .LBB15_3: ; X86-BASELINE-NEXT: .LBB15_3:
; X86-FALLBACK0-NEXT: andl (%esi), %eax ; X86-BASELINE-NEXT: andl (%esi), %eax
; X86-FALLBACK0-NEXT: andl 4(%esi), %edx ; X86-BASELINE-NEXT: andl 4(%esi), %edx
; X86-FALLBACK0-NEXT: popl %esi ; X86-BASELINE-NEXT: popl %esi
; X86-FALLBACK0-NEXT: popl %edi ; X86-BASELINE-NEXT: popl %edi
; X86-FALLBACK0-NEXT: retl ; X86-BASELINE-NEXT: retl
; ;
; X86-FALLBACK1-LABEL: clear_highbits64_c2_load: ; X86-BMI-LABEL: clear_highbits64_c2_load:
; X86-FALLBACK1: # %bb.0: ; X86-BMI: # %bb.0:
; X86-FALLBACK1-NEXT: pushl %edi ; X86-BMI-NEXT: pushl %edi
; X86-FALLBACK1-NEXT: pushl %esi ; X86-BMI-NEXT: pushl %esi
; X86-FALLBACK1-NEXT: movl {{[0-9]+}}(%esp), %esi ; X86-BMI-NEXT: movl {{[0-9]+}}(%esp), %esi
; X86-FALLBACK1-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK1-NEXT: movl $-1, %edi ; X86-BMI-NEXT: movl $-1, %edi
; X86-FALLBACK1-NEXT: movl $-1, %eax ; X86-BMI-NEXT: movl $-1, %eax
; X86-FALLBACK1-NEXT: shrl %cl, %eax ; X86-BMI-NEXT: shrl %cl, %eax
; X86-FALLBACK1-NEXT: xorl %edx, %edx ; X86-BMI-NEXT: xorl %edx, %edx
; X86-FALLBACK1-NEXT: testb $32, %cl ; X86-BMI-NEXT: testb $32, %cl
; X86-FALLBACK1-NEXT: cmovel %eax, %edx ; X86-BMI-NEXT: cmovel %eax, %edx
; X86-FALLBACK1-NEXT: cmovel %edi, %eax ; X86-BMI-NEXT: cmovel %edi, %eax
; X86-FALLBACK1-NEXT: andl (%esi), %eax ; X86-BMI-NEXT: andl (%esi), %eax
; X86-FALLBACK1-NEXT: andl 4(%esi), %edx ; X86-BMI-NEXT: andl 4(%esi), %edx
; X86-FALLBACK1-NEXT: popl %esi ; X86-BMI-NEXT: popl %esi
; X86-FALLBACK1-NEXT: popl %edi ; X86-BMI-NEXT: popl %edi
; X86-FALLBACK1-NEXT: retl ; X86-BMI-NEXT: retl
;
; X86-FALLBACK2-LABEL: clear_highbits64_c2_load:
; X86-FALLBACK2: # %bb.0:
; X86-FALLBACK2-NEXT: pushl %edi
; X86-FALLBACK2-NEXT: pushl %esi
; X86-FALLBACK2-NEXT: movl {{[0-9]+}}(%esp), %esi
; X86-FALLBACK2-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK2-NEXT: movl $-1, %edi
; X86-FALLBACK2-NEXT: movl $-1, %eax
; X86-FALLBACK2-NEXT: shrl %cl, %eax
; X86-FALLBACK2-NEXT: xorl %edx, %edx
; X86-FALLBACK2-NEXT: testb $32, %cl
; X86-FALLBACK2-NEXT: cmovel %eax, %edx
; X86-FALLBACK2-NEXT: cmovel %edi, %eax
; X86-FALLBACK2-NEXT: andl (%esi), %eax
; X86-FALLBACK2-NEXT: andl 4(%esi), %edx
; X86-FALLBACK2-NEXT: popl %esi
; X86-FALLBACK2-NEXT: popl %edi
; X86-FALLBACK2-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits64_c2_load: ; X86-BMI2-LABEL: clear_highbits64_c2_load:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: pushl %ebx ; X86-BMI2-NEXT: pushl %ebx
; X86-BMI2-NEXT: pushl %esi ; X86-BMI2-NEXT: pushl %esi
; X86-BMI2-NEXT: movl {{[0-9]+}}(%esp), %ecx ; X86-BMI2-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %bl ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %bl
; X86-BMI2-NEXT: movl $-1, %eax ; X86-BMI2-NEXT: movl $-1, %eax
; X86-BMI2-NEXT: shrxl %ebx, %eax, %esi ; X86-BMI2-NEXT: shrxl %ebx, %eax, %esi
; X86-BMI2-NEXT: xorl %edx, %edx ; X86-BMI2-NEXT: xorl %edx, %edx
; X86-BMI2-NEXT: testb $32, %bl ; X86-BMI2-NEXT: testb $32, %bl
; X86-BMI2-NEXT: cmovel %esi, %edx ; X86-BMI2-NEXT: cmovel %esi, %edx
; X86-BMI2-NEXT: cmovnel %esi, %eax ; X86-BMI2-NEXT: cmovnel %esi, %eax
; X86-BMI2-NEXT: andl (%ecx), %eax ; X86-BMI2-NEXT: andl (%ecx), %eax
; X86-BMI2-NEXT: andl 4(%ecx), %edx ; X86-BMI2-NEXT: andl 4(%ecx), %edx
; X86-BMI2-NEXT: popl %esi ; X86-BMI2-NEXT: popl %esi
; X86-BMI2-NEXT: popl %ebx ; X86-BMI2-NEXT: popl %ebx
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits64_c2_load: ; X64-BASELINE-LABEL: clear_highbits64_c2_load:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movq %rsi, %rcx ; X64-BASELINE-NEXT: movq %rsi, %rcx
; X64-NOBMI2-NEXT: movq (%rdi), %rax ; X64-BASELINE-NEXT: movq (%rdi), %rax
; X64-NOBMI2-NEXT: shlq %cl, %rax ; X64-BASELINE-NEXT: shlq %cl, %rax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $rcx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $rcx
; X64-NOBMI2-NEXT: shrq %cl, %rax ; X64-BASELINE-NEXT: shrq %cl, %rax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits64_c2_load:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl $64, %eax
; X64-BMI-NEXT: subl %esi, %eax
; X64-BMI-NEXT: shll $8, %eax
; X64-BMI-NEXT: bextrq %rax, (%rdi), %rax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits64_c2_load: ; X64-BMI2-LABEL: clear_highbits64_c2_load:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: shlxq %rsi, (%rdi), %rax ; X64-BMI2-NEXT: movl $64, %eax
; X64-BMI2-NEXT: shrxq %rsi, %rax, %rax ; X64-BMI2-NEXT: subl %esi, %eax
; X64-BMI2-NEXT: bzhiq %rax, (%rdi), %rax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%val = load i64, i64* %w %val = load i64, i64* %w
%mask = lshr i64 -1, %numhighbits %mask = lshr i64 -1, %numhighbits
%masked = and i64 %mask, %val %masked = and i64 %mask, %val
ret i64 %masked ret i64 %masked
} }
define i64 @clear_highbits64_c3_load_indexzext(i64* %w, i8 %numhighbits) nounwind { define i64 @clear_highbits64_c3_load_indexzext(i64* %w, i8 %numhighbits) nounwind {
; X86-FALLBACK0-LABEL: clear_highbits64_c3_load_indexzext: ; X86-BASELINE-LABEL: clear_highbits64_c3_load_indexzext:
; X86-FALLBACK0: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-FALLBACK0-NEXT: pushl %edi ; X86-BASELINE-NEXT: pushl %edi
; X86-FALLBACK0-NEXT: pushl %esi ; X86-BASELINE-NEXT: pushl %esi
; X86-FALLBACK0-NEXT: movl {{[0-9]+}}(%esp), %esi ; X86-BASELINE-NEXT: movl {{[0-9]+}}(%esp), %esi
; X86-FALLBACK0-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK0-NEXT: movl $-1, %eax ; X86-BASELINE-NEXT: movl $-1, %eax
; X86-FALLBACK0-NEXT: movl $-1, %edi ; X86-BASELINE-NEXT: movl $-1, %edi
; X86-FALLBACK0-NEXT: shrl %cl, %edi ; X86-BASELINE-NEXT: shrl %cl, %edi
; X86-FALLBACK0-NEXT: xorl %edx, %edx ; X86-BASELINE-NEXT: xorl %edx, %edx
; X86-FALLBACK0-NEXT: testb $32, %cl ; X86-BASELINE-NEXT: testb $32, %cl
; X86-FALLBACK0-NEXT: jne .LBB16_1 ; X86-BASELINE-NEXT: jne .LBB16_1
; X86-FALLBACK0-NEXT: # %bb.2: ; X86-BASELINE-NEXT: # %bb.2:
; X86-FALLBACK0-NEXT: movl %edi, %edx ; X86-BASELINE-NEXT: movl %edi, %edx
; X86-FALLBACK0-NEXT: jmp .LBB16_3 ; X86-BASELINE-NEXT: jmp .LBB16_3
; X86-FALLBACK0-NEXT: .LBB16_1: ; X86-BASELINE-NEXT: .LBB16_1:
; X86-FALLBACK0-NEXT: movl %edi, %eax ; X86-BASELINE-NEXT: movl %edi, %eax
; X86-FALLBACK0-NEXT: .LBB16_3: ; X86-BASELINE-NEXT: .LBB16_3:
; X86-FALLBACK0-NEXT: andl (%esi), %eax ; X86-BASELINE-NEXT: andl (%esi), %eax
; X86-FALLBACK0-NEXT: andl 4(%esi), %edx ; X86-BASELINE-NEXT: andl 4(%esi), %edx
; X86-FALLBACK0-NEXT: popl %esi ; X86-BASELINE-NEXT: popl %esi
; X86-FALLBACK0-NEXT: popl %edi ; X86-BASELINE-NEXT: popl %edi
; X86-FALLBACK0-NEXT: retl ; X86-BASELINE-NEXT: retl
; ;
; X86-FALLBACK1-LABEL: clear_highbits64_c3_load_indexzext: ; X86-BMI-LABEL: clear_highbits64_c3_load_indexzext:
; X86-FALLBACK1: # %bb.0: ; X86-BMI: # %bb.0:
; X86-FALLBACK1-NEXT: pushl %edi ; X86-BMI-NEXT: pushl %edi
; X86-FALLBACK1-NEXT: pushl %esi ; X86-BMI-NEXT: pushl %esi
; X86-FALLBACK1-NEXT: movl {{[0-9]+}}(%esp), %esi ; X86-BMI-NEXT: movl {{[0-9]+}}(%esp), %esi
; X86-FALLBACK1-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK1-NEXT: movl $-1, %edi ; X86-BMI-NEXT: movl $-1, %edi
; X86-FALLBACK1-NEXT: movl $-1, %eax ; X86-BMI-NEXT: movl $-1, %eax
; X86-FALLBACK1-NEXT: shrl %cl, %eax ; X86-BMI-NEXT: shrl %cl, %eax
; X86-FALLBACK1-NEXT: xorl %edx, %edx ; X86-BMI-NEXT: xorl %edx, %edx
; X86-FALLBACK1-NEXT: testb $32, %cl ; X86-BMI-NEXT: testb $32, %cl
; X86-FALLBACK1-NEXT: cmovel %eax, %edx ; X86-BMI-NEXT: cmovel %eax, %edx
; X86-FALLBACK1-NEXT: cmovel %edi, %eax ; X86-BMI-NEXT: cmovel %edi, %eax
; X86-FALLBACK1-NEXT: andl (%esi), %eax ; X86-BMI-NEXT: andl (%esi), %eax
; X86-FALLBACK1-NEXT: andl 4(%esi), %edx ; X86-BMI-NEXT: andl 4(%esi), %edx
; X86-FALLBACK1-NEXT: popl %esi ; X86-BMI-NEXT: popl %esi
; X86-FALLBACK1-NEXT: popl %edi ; X86-BMI-NEXT: popl %edi
; X86-FALLBACK1-NEXT: retl ; X86-BMI-NEXT: retl
;
; X86-FALLBACK2-LABEL: clear_highbits64_c3_load_indexzext:
; X86-FALLBACK2: # %bb.0:
; X86-FALLBACK2-NEXT: pushl %edi
; X86-FALLBACK2-NEXT: pushl %esi
; X86-FALLBACK2-NEXT: movl {{[0-9]+}}(%esp), %esi
; X86-FALLBACK2-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK2-NEXT: movl $-1, %edi
; X86-FALLBACK2-NEXT: movl $-1, %eax
; X86-FALLBACK2-NEXT: shrl %cl, %eax
; X86-FALLBACK2-NEXT: xorl %edx, %edx
; X86-FALLBACK2-NEXT: testb $32, %cl
; X86-FALLBACK2-NEXT: cmovel %eax, %edx
; X86-FALLBACK2-NEXT: cmovel %edi, %eax
; X86-FALLBACK2-NEXT: andl (%esi), %eax
; X86-FALLBACK2-NEXT: andl 4(%esi), %edx
; X86-FALLBACK2-NEXT: popl %esi
; X86-FALLBACK2-NEXT: popl %edi
; X86-FALLBACK2-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits64_c3_load_indexzext: ; X86-BMI2-LABEL: clear_highbits64_c3_load_indexzext:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: pushl %ebx ; X86-BMI2-NEXT: pushl %ebx
; X86-BMI2-NEXT: pushl %esi ; X86-BMI2-NEXT: pushl %esi
; X86-BMI2-NEXT: movl {{[0-9]+}}(%esp), %ecx ; X86-BMI2-NEXT: movl {{[0-9]+}}(%esp), %ecx
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %bl ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %bl
; X86-BMI2-NEXT: movl $-1, %eax ; X86-BMI2-NEXT: movl $-1, %eax
; X86-BMI2-NEXT: shrxl %ebx, %eax, %esi ; X86-BMI2-NEXT: shrxl %ebx, %eax, %esi
; X86-BMI2-NEXT: xorl %edx, %edx ; X86-BMI2-NEXT: xorl %edx, %edx
; X86-BMI2-NEXT: testb $32, %bl ; X86-BMI2-NEXT: testb $32, %bl
; X86-BMI2-NEXT: cmovel %esi, %edx ; X86-BMI2-NEXT: cmovel %esi, %edx
; X86-BMI2-NEXT: cmovnel %esi, %eax ; X86-BMI2-NEXT: cmovnel %esi, %eax
; X86-BMI2-NEXT: andl (%ecx), %eax ; X86-BMI2-NEXT: andl (%ecx), %eax
; X86-BMI2-NEXT: andl 4(%ecx), %edx ; X86-BMI2-NEXT: andl 4(%ecx), %edx
; X86-BMI2-NEXT: popl %esi ; X86-BMI2-NEXT: popl %esi
; X86-BMI2-NEXT: popl %ebx ; X86-BMI2-NEXT: popl %ebx
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits64_c3_load_indexzext: ; X64-BASELINE-LABEL: clear_highbits64_c3_load_indexzext:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: movq (%rdi), %rax ; X64-BASELINE-NEXT: movq (%rdi), %rax
; X64-NOBMI2-NEXT: shlq %cl, %rax ; X64-BASELINE-NEXT: shlq %cl, %rax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrq %cl, %rax ; X64-BASELINE-NEXT: shrq %cl, %rax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits64_c3_load_indexzext:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl $64, %eax
; X64-BMI-NEXT: subl %esi, %eax
; X64-BMI-NEXT: shll $8, %eax
; X64-BMI-NEXT: bextrq %rax, (%rdi), %rax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits64_c3_load_indexzext: ; X64-BMI2-LABEL: clear_highbits64_c3_load_indexzext:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: # kill: def $esi killed $esi def $rsi ; X64-BMI2-NEXT: movl $64, %eax
; X64-BMI2-NEXT: shlxq %rsi, (%rdi), %rax ; X64-BMI2-NEXT: subl %esi, %eax
; X64-BMI2-NEXT: shrxq %rsi, %rax, %rax ; X64-BMI2-NEXT: bzhiq %rax, (%rdi), %rax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%val = load i64, i64* %w %val = load i64, i64* %w
%sh_prom = zext i8 %numhighbits to i64 %sh_prom = zext i8 %numhighbits to i64
%mask = lshr i64 -1, %sh_prom %mask = lshr i64 -1, %sh_prom
%masked = and i64 %mask, %val %masked = and i64 %mask, %val
ret i64 %masked ret i64 %masked
} }
define i64 @clear_highbits64_c4_commutative(i64 %val, i64 %numhighbits) nounwind { define i64 @clear_highbits64_c4_commutative(i64 %val, i64 %numhighbits) nounwind {
; X86-FALLBACK0-LABEL: clear_highbits64_c4_commutative: ; X86-BASELINE-LABEL: clear_highbits64_c4_commutative:
; X86-FALLBACK0: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-FALLBACK0-NEXT: pushl %esi ; X86-BASELINE-NEXT: pushl %esi
; X86-FALLBACK0-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK0-NEXT: movl $-1, %eax ; X86-BASELINE-NEXT: movl $-1, %eax
; X86-FALLBACK0-NEXT: movl $-1, %esi ; X86-BASELINE-NEXT: movl $-1, %esi
; X86-FALLBACK0-NEXT: shrl %cl, %esi ; X86-BASELINE-NEXT: shrl %cl, %esi
; X86-FALLBACK0-NEXT: xorl %edx, %edx ; X86-BASELINE-NEXT: xorl %edx, %edx
; X86-FALLBACK0-NEXT: testb $32, %cl ; X86-BASELINE-NEXT: testb $32, %cl
; X86-FALLBACK0-NEXT: jne .LBB17_1 ; X86-BASELINE-NEXT: jne .LBB17_1
; X86-FALLBACK0-NEXT: # %bb.2: ; X86-BASELINE-NEXT: # %bb.2:
; X86-FALLBACK0-NEXT: movl %esi, %edx ; X86-BASELINE-NEXT: movl %esi, %edx
; X86-FALLBACK0-NEXT: jmp .LBB17_3 ; X86-BASELINE-NEXT: jmp .LBB17_3
; X86-FALLBACK0-NEXT: .LBB17_1: ; X86-BASELINE-NEXT: .LBB17_1:
; X86-FALLBACK0-NEXT: movl %esi, %eax ; X86-BASELINE-NEXT: movl %esi, %eax
; X86-FALLBACK0-NEXT: .LBB17_3: ; X86-BASELINE-NEXT: .LBB17_3:
; X86-FALLBACK0-NEXT: andl {{[0-9]+}}(%esp), %eax ; X86-BASELINE-NEXT: andl {{[0-9]+}}(%esp), %eax
; X86-FALLBACK0-NEXT: andl {{[0-9]+}}(%esp), %edx ; X86-BASELINE-NEXT: andl {{[0-9]+}}(%esp), %edx
; X86-FALLBACK0-NEXT: popl %esi ; X86-BASELINE-NEXT: popl %esi
; X86-FALLBACK0-NEXT: retl ; X86-BASELINE-NEXT: retl
; ;
; X86-FALLBACK1-LABEL: clear_highbits64_c4_commutative: ; X86-BMI-LABEL: clear_highbits64_c4_commutative:
; X86-FALLBACK1: # %bb.0: ; X86-BMI: # %bb.0:
; X86-FALLBACK1-NEXT: pushl %esi ; X86-BMI-NEXT: pushl %esi
; X86-FALLBACK1-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK1-NEXT: movl $-1, %esi ; X86-BMI-NEXT: movl $-1, %esi
; X86-FALLBACK1-NEXT: movl $-1, %eax ; X86-BMI-NEXT: movl $-1, %eax
; X86-FALLBACK1-NEXT: shrl %cl, %eax ; X86-BMI-NEXT: shrl %cl, %eax
; X86-FALLBACK1-NEXT: xorl %edx, %edx ; X86-BMI-NEXT: xorl %edx, %edx
; X86-FALLBACK1-NEXT: testb $32, %cl ; X86-BMI-NEXT: testb $32, %cl
; X86-FALLBACK1-NEXT: cmovel %eax, %edx ; X86-BMI-NEXT: cmovel %eax, %edx
; X86-FALLBACK1-NEXT: cmovel %esi, %eax ; X86-BMI-NEXT: cmovel %esi, %eax
; X86-FALLBACK1-NEXT: andl {{[0-9]+}}(%esp), %eax ; X86-BMI-NEXT: andl {{[0-9]+}}(%esp), %eax
; X86-FALLBACK1-NEXT: andl {{[0-9]+}}(%esp), %edx ; X86-BMI-NEXT: andl {{[0-9]+}}(%esp), %edx
; X86-FALLBACK1-NEXT: popl %esi ; X86-BMI-NEXT: popl %esi
; X86-FALLBACK1-NEXT: retl ; X86-BMI-NEXT: retl
;
; X86-FALLBACK2-LABEL: clear_highbits64_c4_commutative:
; X86-FALLBACK2: # %bb.0:
; X86-FALLBACK2-NEXT: pushl %esi
; X86-FALLBACK2-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK2-NEXT: movl $-1, %esi
; X86-FALLBACK2-NEXT: movl $-1, %eax
; X86-FALLBACK2-NEXT: shrl %cl, %eax
; X86-FALLBACK2-NEXT: xorl %edx, %edx
; X86-FALLBACK2-NEXT: testb $32, %cl
; X86-FALLBACK2-NEXT: cmovel %eax, %edx
; X86-FALLBACK2-NEXT: cmovel %esi, %eax
; X86-FALLBACK2-NEXT: andl {{[0-9]+}}(%esp), %eax
; X86-FALLBACK2-NEXT: andl {{[0-9]+}}(%esp), %edx
; X86-FALLBACK2-NEXT: popl %esi
; X86-FALLBACK2-NEXT: retl
; ;
; X86-BMI2-LABEL: clear_highbits64_c4_commutative: ; X86-BMI2-LABEL: clear_highbits64_c4_commutative:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: pushl %esi ; X86-BMI2-NEXT: pushl %esi
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI2-NEXT: movl $-1, %eax ; X86-BMI2-NEXT: movl $-1, %eax
; X86-BMI2-NEXT: shrxl %ecx, %eax, %esi ; X86-BMI2-NEXT: shrxl %ecx, %eax, %esi
; X86-BMI2-NEXT: xorl %edx, %edx ; X86-BMI2-NEXT: xorl %edx, %edx
; X86-BMI2-NEXT: testb $32, %cl ; X86-BMI2-NEXT: testb $32, %cl
; X86-BMI2-NEXT: cmovel %esi, %edx ; X86-BMI2-NEXT: cmovel %esi, %edx
; X86-BMI2-NEXT: cmovnel %esi, %eax ; X86-BMI2-NEXT: cmovnel %esi, %eax
; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %eax ; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %eax
; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %edx ; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %edx
; X86-BMI2-NEXT: popl %esi ; X86-BMI2-NEXT: popl %esi
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: clear_highbits64_c4_commutative: ; X64-BASELINE-LABEL: clear_highbits64_c4_commutative:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: movq %rsi, %rcx ; X64-BASELINE-NEXT: movq %rsi, %rcx
; X64-NOBMI2-NEXT: movq %rdi, %rax ; X64-BASELINE-NEXT: movq %rdi, %rax
; X64-NOBMI2-NEXT: shlq %cl, %rax ; X64-BASELINE-NEXT: shlq %cl, %rax
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $rcx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $rcx
; X64-NOBMI2-NEXT: shrq %cl, %rax ; X64-BASELINE-NEXT: shrq %cl, %rax
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: clear_highbits64_c4_commutative:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: movl $64, %eax
; X64-BMI-NEXT: subl %esi, %eax
; X64-BMI-NEXT: shll $8, %eax
; X64-BMI-NEXT: bextrq %rax, %rdi, %rax
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: clear_highbits64_c4_commutative: ; X64-BMI2-LABEL: clear_highbits64_c4_commutative:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: shlxq %rsi, %rdi, %rax ; X64-BMI2-NEXT: movl $64, %eax
; X64-BMI2-NEXT: shrxq %rsi, %rax, %rax ; X64-BMI2-NEXT: subl %esi, %eax
; X64-BMI2-NEXT: bzhiq %rax, %rdi, %rax
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%mask = lshr i64 -1, %numhighbits %mask = lshr i64 -1, %numhighbits
%masked = and i64 %val, %mask ; swapped order %masked = and i64 %val, %mask ; swapped order
ret i64 %masked ret i64 %masked
} }
; ---------------------------------------------------------------------------- ; ; ---------------------------------------------------------------------------- ;
; Multi-use tests ; Multi-use tests
; ---------------------------------------------------------------------------- ; ; ---------------------------------------------------------------------------- ;
declare void @use32(i32) declare void @use32(i32)
declare void @use64(i64) declare void @use64(i64)
define i32 @oneuse32(i32 %val, i32 %numhighbits) nounwind { define i32 @oneuse32(i32 %val, i32 %numhighbits) nounwind {
; X86-NOBMI2-LABEL: oneuse32: ; X86-BASELINE-LABEL: oneuse32:
; X86-NOBMI2: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-NOBMI2-NEXT: pushl %esi ; X86-BASELINE-NEXT: pushl %esi
; X86-NOBMI2-NEXT: subl $8, %esp ; X86-BASELINE-NEXT: subl $8, %esp
; X86-NOBMI2-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-NOBMI2-NEXT: movl $-1, %esi ; X86-BASELINE-NEXT: movl $-1, %esi
; X86-NOBMI2-NEXT: shrl %cl, %esi ; X86-BASELINE-NEXT: shrl %cl, %esi
; X86-NOBMI2-NEXT: movl %esi, (%esp) ; X86-BASELINE-NEXT: movl %esi, (%esp)
; X86-NOBMI2-NEXT: calll use32@PLT ; X86-BASELINE-NEXT: calll use32@PLT
; X86-NOBMI2-NEXT: andl {{[0-9]+}}(%esp), %esi ; X86-BASELINE-NEXT: andl {{[0-9]+}}(%esp), %esi
; X86-NOBMI2-NEXT: movl %esi, %eax ; X86-BASELINE-NEXT: movl %esi, %eax
; X86-NOBMI2-NEXT: addl $8, %esp ; X86-BASELINE-NEXT: addl $8, %esp
; X86-NOBMI2-NEXT: popl %esi ; X86-BASELINE-NEXT: popl %esi
; X86-NOBMI2-NEXT: retl ; X86-BASELINE-NEXT: retl
;
; X86-BMI-LABEL: oneuse32:
; X86-BMI: # %bb.0:
; X86-BMI-NEXT: pushl %esi
; X86-BMI-NEXT: subl $8, %esp
; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-BMI-NEXT: movl $-1, %esi
; X86-BMI-NEXT: shrl %cl, %esi
; X86-BMI-NEXT: movl %esi, (%esp)
; X86-BMI-NEXT: calll use32@PLT
; X86-BMI-NEXT: andl {{[0-9]+}}(%esp), %esi
; X86-BMI-NEXT: movl %esi, %eax
; X86-BMI-NEXT: addl $8, %esp
; X86-BMI-NEXT: popl %esi
; X86-BMI-NEXT: retl
; ;
; X86-BMI2-LABEL: oneuse32: ; X86-BMI2-LABEL: oneuse32:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: pushl %esi ; X86-BMI2-NEXT: pushl %ebx
; X86-BMI2-NEXT: subl $8, %esp ; X86-BMI2-NEXT: subl $8, %esp
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %bl
; X86-BMI2-NEXT: movl $-1, %ecx ; X86-BMI2-NEXT: movl $-1, %eax
; X86-BMI2-NEXT: shrxl %eax, %ecx, %esi ; X86-BMI2-NEXT: shrxl %ebx, %eax, %eax
; X86-BMI2-NEXT: movl %esi, (%esp) ; X86-BMI2-NEXT: movl %eax, (%esp)
; X86-BMI2-NEXT: calll use32@PLT ; X86-BMI2-NEXT: calll use32@PLT
; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %esi ; X86-BMI2-NEXT: movl $32, %eax
; X86-BMI2-NEXT: movl %esi, %eax ; X86-BMI2-NEXT: subl %ebx, %eax
craig.topperUnsubmitted
Done
ReplyInline Actions

This doesn't seem better. But maybe it increases parallelism the if the shrxl use wasn't a call?

craig.topper: This doesn't seem better. But maybe it increases parallelism the if the shrxl use wasn't a call?
; X86-BMI2-NEXT: bzhil %eax, {{[0-9]+}}(%esp), %eax
; X86-BMI2-NEXT: addl $8, %esp ; X86-BMI2-NEXT: addl $8, %esp
; X86-BMI2-NEXT: popl %esi ; X86-BMI2-NEXT: popl %ebx
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: oneuse32: ; X64-BASELINE-LABEL: oneuse32:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: pushq %rbp ; X64-BASELINE-NEXT: pushq %rbp
; X64-NOBMI2-NEXT: pushq %rbx ; X64-BASELINE-NEXT: pushq %rbx
; X64-NOBMI2-NEXT: pushq %rax ; X64-BASELINE-NEXT: pushq %rax
; X64-NOBMI2-NEXT: movl %esi, %ecx ; X64-BASELINE-NEXT: movl %esi, %ecx
; X64-NOBMI2-NEXT: movl %edi, %ebx ; X64-BASELINE-NEXT: movl %edi, %ebx
; X64-NOBMI2-NEXT: movl $-1, %ebp ; X64-BASELINE-NEXT: movl $-1, %ebp
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $ecx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-NOBMI2-NEXT: shrl %cl, %ebp ; X64-BASELINE-NEXT: shrl %cl, %ebp
; X64-NOBMI2-NEXT: movl %ebp, %edi ; X64-BASELINE-NEXT: movl %ebp, %edi
; X64-NOBMI2-NEXT: callq use32@PLT ; X64-BASELINE-NEXT: callq use32@PLT
; X64-NOBMI2-NEXT: andl %ebx, %ebp ; X64-BASELINE-NEXT: andl %ebx, %ebp
; X64-NOBMI2-NEXT: movl %ebp, %eax ; X64-BASELINE-NEXT: movl %ebp, %eax
; X64-NOBMI2-NEXT: addq $8, %rsp ; X64-BASELINE-NEXT: addq $8, %rsp
; X64-NOBMI2-NEXT: popq %rbx ; X64-BASELINE-NEXT: popq %rbx
; X64-NOBMI2-NEXT: popq %rbp ; X64-BASELINE-NEXT: popq %rbp
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: oneuse32:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: pushq %rbp
; X64-BMI-NEXT: pushq %rbx
; X64-BMI-NEXT: pushq %rax
; X64-BMI-NEXT: movl %esi, %ecx
; X64-BMI-NEXT: movl %edi, %ebx
; X64-BMI-NEXT: movl $-1, %ebp
; X64-BMI-NEXT: # kill: def $cl killed $cl killed $ecx
; X64-BMI-NEXT: shrl %cl, %ebp
; X64-BMI-NEXT: movl %ebp, %edi
; X64-BMI-NEXT: callq use32@PLT
; X64-BMI-NEXT: andl %ebx, %ebp
; X64-BMI-NEXT: movl %ebp, %eax
; X64-BMI-NEXT: addq $8, %rsp
; X64-BMI-NEXT: popq %rbx
; X64-BMI-NEXT: popq %rbp
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: oneuse32: ; X64-BMI2-LABEL: oneuse32:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: pushq %rbp ; X64-BMI2-NEXT: pushq %rbp
; X64-BMI2-NEXT: pushq %rbx ; X64-BMI2-NEXT: pushq %rbx
; X64-BMI2-NEXT: pushq %rax ; X64-BMI2-NEXT: pushq %rax
; X64-BMI2-NEXT: movl %edi, %ebx ; X64-BMI2-NEXT: movl %esi, %ebx
; X64-BMI2-NEXT: movl %edi, %ebp
; X64-BMI2-NEXT: movl $-1, %eax ; X64-BMI2-NEXT: movl $-1, %eax
; X64-BMI2-NEXT: shrxl %esi, %eax, %ebp ; X64-BMI2-NEXT: shrxl %esi, %eax, %edi
; X64-BMI2-NEXT: movl %ebp, %edi
; X64-BMI2-NEXT: callq use32@PLT ; X64-BMI2-NEXT: callq use32@PLT
; X64-BMI2-NEXT: andl %ebx, %ebp ; X64-BMI2-NEXT: movl $32, %eax
; X64-BMI2-NEXT: movl %ebp, %eax ; X64-BMI2-NEXT: subl %ebx, %eax
; X64-BMI2-NEXT: bzhil %eax, %ebp, %eax
; X64-BMI2-NEXT: addq $8, %rsp ; X64-BMI2-NEXT: addq $8, %rsp
; X64-BMI2-NEXT: popq %rbx ; X64-BMI2-NEXT: popq %rbx
; X64-BMI2-NEXT: popq %rbp ; X64-BMI2-NEXT: popq %rbp
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%mask = lshr i32 -1, %numhighbits %mask = lshr i32 -1, %numhighbits
call void @use32(i32 %mask) call void @use32(i32 %mask)
%masked = and i32 %mask, %val %masked = and i32 %mask, %val
ret i32 %masked ret i32 %masked
} }
define i64 @oneuse64(i64 %val, i64 %numhighbits) nounwind { define i64 @oneuse64(i64 %val, i64 %numhighbits) nounwind {
; X86-FALLBACK0-LABEL: oneuse64: ; X86-BASELINE-LABEL: oneuse64:
; X86-FALLBACK0: # %bb.0: ; X86-BASELINE: # %bb.0:
; X86-FALLBACK0-NEXT: pushl %edi ; X86-BASELINE-NEXT: pushl %edi
; X86-FALLBACK0-NEXT: pushl %esi ; X86-BASELINE-NEXT: pushl %esi
; X86-FALLBACK0-NEXT: pushl %eax ; X86-BASELINE-NEXT: pushl %eax
; X86-FALLBACK0-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BASELINE-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK0-NEXT: movl $-1, %esi ; X86-BASELINE-NEXT: movl $-1, %esi
; X86-FALLBACK0-NEXT: movl $-1, %edi ; X86-BASELINE-NEXT: movl $-1, %edi
; X86-FALLBACK0-NEXT: shrl %cl, %edi ; X86-BASELINE-NEXT: shrl %cl, %edi
; X86-FALLBACK0-NEXT: testb $32, %cl ; X86-BASELINE-NEXT: testb $32, %cl
; X86-FALLBACK0-NEXT: je .LBB19_2 ; X86-BASELINE-NEXT: je .LBB19_2
; X86-FALLBACK0-NEXT: # %bb.1: ; X86-BASELINE-NEXT: # %bb.1:
; X86-FALLBACK0-NEXT: movl %edi, %esi ; X86-BASELINE-NEXT: movl %edi, %esi
; X86-FALLBACK0-NEXT: xorl %edi, %edi ; X86-BASELINE-NEXT: xorl %edi, %edi
; X86-FALLBACK0-NEXT: .LBB19_2: ; X86-BASELINE-NEXT: .LBB19_2:
; X86-FALLBACK0-NEXT: subl $8, %esp ; X86-BASELINE-NEXT: subl $8, %esp
; X86-FALLBACK0-NEXT: pushl %edi ; X86-BASELINE-NEXT: pushl %edi
; X86-FALLBACK0-NEXT: pushl %esi ; X86-BASELINE-NEXT: pushl %esi
; X86-FALLBACK0-NEXT: calll use64@PLT ; X86-BASELINE-NEXT: calll use64@PLT
; X86-FALLBACK0-NEXT: addl $16, %esp ; X86-BASELINE-NEXT: addl $16, %esp
; X86-FALLBACK0-NEXT: andl {{[0-9]+}}(%esp), %esi ; X86-BASELINE-NEXT: andl {{[0-9]+}}(%esp), %esi
; X86-FALLBACK0-NEXT: andl {{[0-9]+}}(%esp), %edi ; X86-BASELINE-NEXT: andl {{[0-9]+}}(%esp), %edi
; X86-FALLBACK0-NEXT: movl %esi, %eax ; X86-BASELINE-NEXT: movl %esi, %eax
; X86-FALLBACK0-NEXT: movl %edi, %edx ; X86-BASELINE-NEXT: movl %edi, %edx
; X86-FALLBACK0-NEXT: addl $4, %esp ; X86-BASELINE-NEXT: addl $4, %esp
; X86-FALLBACK0-NEXT: popl %esi ; X86-BASELINE-NEXT: popl %esi
; X86-FALLBACK0-NEXT: popl %edi ; X86-BASELINE-NEXT: popl %edi
; X86-FALLBACK0-NEXT: retl ; X86-BASELINE-NEXT: retl
; ;
; X86-FALLBACK1-LABEL: oneuse64: ; X86-BMI-LABEL: oneuse64:
; X86-FALLBACK1: # %bb.0: ; X86-BMI: # %bb.0:
; X86-FALLBACK1-NEXT: pushl %edi ; X86-BMI-NEXT: pushl %edi
; X86-FALLBACK1-NEXT: pushl %esi ; X86-BMI-NEXT: pushl %esi
; X86-FALLBACK1-NEXT: pushl %eax ; X86-BMI-NEXT: pushl %eax
; X86-FALLBACK1-NEXT: movb {{[0-9]+}}(%esp), %cl ; X86-BMI-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK1-NEXT: movl $-1, %esi ; X86-BMI-NEXT: movl $-1, %esi
; X86-FALLBACK1-NEXT: movl $-1, %eax ; X86-BMI-NEXT: movl $-1, %eax
; X86-FALLBACK1-NEXT: shrl %cl, %eax ; X86-BMI-NEXT: shrl %cl, %eax
; X86-FALLBACK1-NEXT: xorl %edi, %edi ; X86-BMI-NEXT: xorl %edi, %edi
; X86-FALLBACK1-NEXT: testb $32, %cl ; X86-BMI-NEXT: testb $32, %cl
; X86-FALLBACK1-NEXT: cmovnel %eax, %esi ; X86-BMI-NEXT: cmovnel %eax, %esi
; X86-FALLBACK1-NEXT: cmovel %eax, %edi ; X86-BMI-NEXT: cmovel %eax, %edi
; X86-FALLBACK1-NEXT: subl $8, %esp ; X86-BMI-NEXT: subl $8, %esp
; X86-FALLBACK1-NEXT: pushl %edi ; X86-BMI-NEXT: pushl %edi
; X86-FALLBACK1-NEXT: pushl %esi ; X86-BMI-NEXT: pushl %esi
; X86-FALLBACK1-NEXT: calll use64@PLT ; X86-BMI-NEXT: calll use64@PLT
; X86-FALLBACK1-NEXT: addl $16, %esp ; X86-BMI-NEXT: addl $16, %esp
; X86-FALLBACK1-NEXT: andl {{[0-9]+}}(%esp), %esi ; X86-BMI-NEXT: andl {{[0-9]+}}(%esp), %esi
; X86-FALLBACK1-NEXT: andl {{[0-9]+}}(%esp), %edi ; X86-BMI-NEXT: andl {{[0-9]+}}(%esp), %edi
; X86-FALLBACK1-NEXT: movl %esi, %eax ; X86-BMI-NEXT: movl %esi, %eax
; X86-FALLBACK1-NEXT: movl %edi, %edx ; X86-BMI-NEXT: movl %edi, %edx
; X86-FALLBACK1-NEXT: addl $4, %esp ; X86-BMI-NEXT: addl $4, %esp
; X86-FALLBACK1-NEXT: popl %esi ; X86-BMI-NEXT: popl %esi
; X86-FALLBACK1-NEXT: popl %edi ; X86-BMI-NEXT: popl %edi
; X86-FALLBACK1-NEXT: retl ; X86-BMI-NEXT: retl
;
; X86-FALLBACK2-LABEL: oneuse64:
; X86-FALLBACK2: # %bb.0:
; X86-FALLBACK2-NEXT: pushl %edi
; X86-FALLBACK2-NEXT: pushl %esi
; X86-FALLBACK2-NEXT: pushl %eax
; X86-FALLBACK2-NEXT: movb {{[0-9]+}}(%esp), %cl
; X86-FALLBACK2-NEXT: movl $-1, %esi
; X86-FALLBACK2-NEXT: movl $-1, %eax
; X86-FALLBACK2-NEXT: shrl %cl, %eax
; X86-FALLBACK2-NEXT: xorl %edi, %edi
; X86-FALLBACK2-NEXT: testb $32, %cl
; X86-FALLBACK2-NEXT: cmovnel %eax, %esi
; X86-FALLBACK2-NEXT: cmovel %eax, %edi
; X86-FALLBACK2-NEXT: subl $8, %esp
; X86-FALLBACK2-NEXT: pushl %edi
; X86-FALLBACK2-NEXT: pushl %esi
; X86-FALLBACK2-NEXT: calll use64@PLT
; X86-FALLBACK2-NEXT: addl $16, %esp
; X86-FALLBACK2-NEXT: andl {{[0-9]+}}(%esp), %esi
; X86-FALLBACK2-NEXT: andl {{[0-9]+}}(%esp), %edi
; X86-FALLBACK2-NEXT: movl %esi, %eax
; X86-FALLBACK2-NEXT: movl %edi, %edx
; X86-FALLBACK2-NEXT: addl $4, %esp
; X86-FALLBACK2-NEXT: popl %esi
; X86-FALLBACK2-NEXT: popl %edi
; X86-FALLBACK2-NEXT: retl
; ;
; X86-BMI2-LABEL: oneuse64: ; X86-BMI2-LABEL: oneuse64:
; X86-BMI2: # %bb.0: ; X86-BMI2: # %bb.0:
; X86-BMI2-NEXT: pushl %edi ; X86-BMI2-NEXT: pushl %edi
; X86-BMI2-NEXT: pushl %esi ; X86-BMI2-NEXT: pushl %esi
; X86-BMI2-NEXT: pushl %eax ; X86-BMI2-NEXT: pushl %eax
; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al ; X86-BMI2-NEXT: movb {{[0-9]+}}(%esp), %al
; X86-BMI2-NEXT: movl $-1, %esi ; X86-BMI2-NEXT: movl $-1, %esi
Show All 11 Lines
; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %edi ; X86-BMI2-NEXT: andl {{[0-9]+}}(%esp), %edi
; X86-BMI2-NEXT: movl %esi, %eax ; X86-BMI2-NEXT: movl %esi, %eax
; X86-BMI2-NEXT: movl %edi, %edx ; X86-BMI2-NEXT: movl %edi, %edx
; X86-BMI2-NEXT: addl $4, %esp ; X86-BMI2-NEXT: addl $4, %esp
; X86-BMI2-NEXT: popl %esi ; X86-BMI2-NEXT: popl %esi
; X86-BMI2-NEXT: popl %edi ; X86-BMI2-NEXT: popl %edi
; X86-BMI2-NEXT: retl ; X86-BMI2-NEXT: retl
; ;
; X64-NOBMI2-LABEL: oneuse64: ; X64-BASELINE-LABEL: oneuse64:
; X64-NOBMI2: # %bb.0: ; X64-BASELINE: # %bb.0:
; X64-NOBMI2-NEXT: pushq %r14 ; X64-BASELINE-NEXT: pushq %r14
; X64-NOBMI2-NEXT: pushq %rbx ; X64-BASELINE-NEXT: pushq %rbx
; X64-NOBMI2-NEXT: pushq %rax ; X64-BASELINE-NEXT: pushq %rax
; X64-NOBMI2-NEXT: movq %rsi, %rcx ; X64-BASELINE-NEXT: movq %rsi, %rcx
; X64-NOBMI2-NEXT: movq %rdi, %r14 ; X64-BASELINE-NEXT: movq %rdi, %r14
; X64-NOBMI2-NEXT: movq $-1, %rbx ; X64-BASELINE-NEXT: movq $-1, %rbx
; X64-NOBMI2-NEXT: # kill: def $cl killed $cl killed $rcx ; X64-BASELINE-NEXT: # kill: def $cl killed $cl killed $rcx
; X64-NOBMI2-NEXT: shrq %cl, %rbx ; X64-BASELINE-NEXT: shrq %cl, %rbx
; X64-NOBMI2-NEXT: movq %rbx, %rdi ; X64-BASELINE-NEXT: movq %rbx, %rdi
; X64-NOBMI2-NEXT: callq use64@PLT ; X64-BASELINE-NEXT: callq use64@PLT
; X64-NOBMI2-NEXT: andq %r14, %rbx ; X64-BASELINE-NEXT: andq %r14, %rbx
; X64-NOBMI2-NEXT: movq %rbx, %rax ; X64-BASELINE-NEXT: movq %rbx, %rax
; X64-NOBMI2-NEXT: addq $8, %rsp ; X64-BASELINE-NEXT: addq $8, %rsp
; X64-NOBMI2-NEXT: popq %rbx ; X64-BASELINE-NEXT: popq %rbx
; X64-NOBMI2-NEXT: popq %r14 ; X64-BASELINE-NEXT: popq %r14
; X64-NOBMI2-NEXT: retq ; X64-BASELINE-NEXT: retq
;
; X64-BMI-LABEL: oneuse64:
; X64-BMI: # %bb.0:
; X64-BMI-NEXT: pushq %r14
; X64-BMI-NEXT: pushq %rbx
; X64-BMI-NEXT: pushq %rax
; X64-BMI-NEXT: movq %rsi, %rcx
; X64-BMI-NEXT: movq %rdi, %r14
; X64-BMI-NEXT: movq $-1, %rbx
; X64-BMI-NEXT: # kill: def $cl killed $cl killed $rcx
; X64-BMI-NEXT: shrq %cl, %rbx
; X64-BMI-NEXT: movq %rbx, %rdi
; X64-BMI-NEXT: callq use64@PLT
; X64-BMI-NEXT: andq %r14, %rbx
; X64-BMI-NEXT: movq %rbx, %rax
; X64-BMI-NEXT: addq $8, %rsp
; X64-BMI-NEXT: popq %rbx
; X64-BMI-NEXT: popq %r14
; X64-BMI-NEXT: retq
; ;
; X64-BMI2-LABEL: oneuse64: ; X64-BMI2-LABEL: oneuse64:
; X64-BMI2: # %bb.0: ; X64-BMI2: # %bb.0:
; X64-BMI2-NEXT: pushq %r14 ; X64-BMI2-NEXT: pushq %r14
; X64-BMI2-NEXT: pushq %rbx ; X64-BMI2-NEXT: pushq %rbx
; X64-BMI2-NEXT: pushq %rax ; X64-BMI2-NEXT: pushq %rax
; X64-BMI2-NEXT: movq %rsi, %rbx
; X64-BMI2-NEXT: movq %rdi, %r14 ; X64-BMI2-NEXT: movq %rdi, %r14
; X64-BMI2-NEXT: movq $-1, %rax ; X64-BMI2-NEXT: movq $-1, %rax
; X64-BMI2-NEXT: shrxq %rsi, %rax, %rbx ; X64-BMI2-NEXT: shrxq %rsi, %rax, %rdi
; X64-BMI2-NEXT: movq %rbx, %rdi
; X64-BMI2-NEXT: callq use64@PLT ; X64-BMI2-NEXT: callq use64@PLT
; X64-BMI2-NEXT: andq %r14, %rbx ; X64-BMI2-NEXT: movl $64, %eax
; X64-BMI2-NEXT: movq %rbx, %rax ; X64-BMI2-NEXT: subl %ebx, %eax
; X64-BMI2-NEXT: bzhiq %rax, %r14, %rax
; X64-BMI2-NEXT: addq $8, %rsp ; X64-BMI2-NEXT: addq $8, %rsp
; X64-BMI2-NEXT: popq %rbx ; X64-BMI2-NEXT: popq %rbx
; X64-BMI2-NEXT: popq %r14 ; X64-BMI2-NEXT: popq %r14
; X64-BMI2-NEXT: retq ; X64-BMI2-NEXT: retq
%mask = lshr i64 -1, %numhighbits %mask = lshr i64 -1, %numhighbits
call void @use64(i64 %mask) call void @use64(i64 %mask)
%masked = and i64 %mask, %val %masked = and i64 %mask, %val
ret i64 %masked ret i64 %masked
} }