[X86] Extend the integer parameter if the function isn't local linked

[LuoYuanke]

Summary

This patch is based on the D71178.
We have disscussion here:
https://discourse.llvm.org/t/always-extend-the-integer-parameters-of-callee/61319.

We want to make clang behave just like gcc to avoid compatibility issues
between different compilers. We can't remove zeroext/signext on the front-end,
which will cause compatibility issues between the new clang and the previous clang.

There are many test cases waiting to be fixed. I put this patch here to see if
we can do this.

[LuoYuanke]

llvm/lib/Target/X86/X86ISelLowering.cpp

3960

Add comments to explain why we ignore sign/zero attribute here?

[LuoYuanke]

llvm/lib/Target/X86/X86ISelLowering.cpp

3960

Maybe check F.hasLocalLinkage() to respect sign/zero attribute for local linkage functions.

[LiuChen3]

Address Yuanke's comments: do the extension only if the function isn't local linked.

[skan]

llvm/lib/Target/X86/X86ISelLowering.cpp

3956–3957

If you check hasLocalLinkage here, I believe we also need to check hasAddressTaken...

However, I think such optimizations should be done in llvm/lib/Transforms/IPO/*.cpp. instead of here.

[LiuChen3]

llvm/lib/Target/X86/X86ISelLowering.cpp

3956–3957

Sorry that I am not familiar with hasAddressTaken. I have one question, does hasAddressTaken also means hasLocalLinkage ?

[skan]

llvm/lib/Target/X86/X86ISelLowering.cpp

3956–3957

hasLocalLinkage does not implies !hasAddressTaken. If a function’s address is taken, then it could be called indirectly by libraries. You can see an example in llvm/lib/Transforms/IPO/GlobalOpt.cpp.

[LiuChen3]

Address Shenchen's comment

[lebedev.ri]

Still not okay with it. The fix is to fix whatever is emitting the attribute, not completely ignore it.

[LiuChen3]

In D122963#3435619, @lebedev.ri wrote:

Still not okay with it. The fix is to fix whatever is emitting the attribute, not completely ignore it.

I don't quite understand what you mean.
If there is no such attribute, it should not do any extension itself. Regardless of whether this function is only a local linked. The optimization here is only for parameters with zeroext/signext attributes.

[lebedev.ri]

In D122963#3435629, @LiuChen3 wrote:

In D122963#3435619, @lebedev.ri wrote:

Still not okay with it. The fix is to fix whatever is emitting the attribute, not completely ignore it.

I don't quite understand what you mean.
If there is no such attribute, it should not do any extension itself. Regardless of whether this function is only a local linked. The optimization here is only for parameters with zeroext/signext attributes.

If there is an zeroext attribute on the function argument, it tells us that the caller is responsible to ensure that the i8 argument,
which is passed in an i64 register, does not have garbage bits in the upper 64-8 bits, but those bits are zeros, so we don't need to zeroext ourselves.
If the caller forgets to do that, the behavior is undefined. If the attribute is present, but the caller wasn't actually supposed to zeroext (ABI mismatch),
then whoever has put that attribute there is at fault. Sometimes ignoring the attribute and performing zeroext ourselves is a workaround for ABI mismatch.

[jyknight]

Sometimes ignoring the attribute and performing zeroext ourselves is a
workaround for ABI mismatch.

Yes, that's precisely the point of this change -- as discussed already!
Clang has been putting zeroext on all C functions taking 8/16 bit types,
incorrectly, for it's entire existence.

And, thus, now we must extend redundantly on both sides of the call, for
external functions.

We must extend in the caller to support calling old-clang code (which
didn't extend in callee). We must extend in the callee to support being
called by non-clang compilers (which don't always extend in the caller).

[LiuChen3]

It seems we all agree we should extend the parameter both in caller and in callee. This is also the behavior of gcc.
For now, the front-end doesn't have an accurate attribute that describes what we're expecting right now and doesn't know whether the function is external linked. By default clang will do the extension only in the caller. This forced us to do some workarounds on the backend.
We also thought about setting these attributes correctly in the IPO, but is there any possibility that IPO won't do in some cases?
D112942 introduces a new parameter attribute NoExt might help us. But this still means that by default, we will do extension in both caller and callee. And only in some optimization cases, we will do extension in caller or callee. Still not the ideal method that lebedev.ri expected.
Besides, I think that as an ABI issue, the correctness should be guaranteed first, optimization is not always guaranteed.
@lebedev.ri, is there any better approach to fix this issue?

[LiuChen3]

Ping? This bug will cause correctness issues, we need to fix it. I think it is possible to fix in the backend in the current situation.
Is there any other better method, @jyknight @lebedev.ri

[LiuChen3]

rebase

[LiuChen3]

If there is no better solution, I hope this patch merge in. This is a bug that needs to be fixed.

[lebedev.ri]

I'm not really seeing consensus, neither here nor in the RFC, i'm afraid.
I suspect the fix will require keeping the existing attributes intact,
introducing new ones that mean what you want, and using them.

[LuoYuanke]

I think we all agree that this is a bug. Given there is no better solution for it, I'd like check in this patch first. We can revert the patch when there is better solution proposed.

[craig.topper]

We have been incompatible for years, why the urgency now? What level of testing has been done outside of the lit tests?

[rnk]

In D122963#3456350, @LuoYuanke wrote:

I think we all agree that this is a bug. Given there is no better solution for it, I'd like check in this patch first. We can revert the patch when there is better solution proposed.

I don't think we've reached agreement that this is a bug. I am concerned that removing these extension assert nodes will result in performance regressions. Here's the summary table from the RFC:

clang: caller do the zero/sign extension while callee do nothing.
gcc: both caller and callee do the zero/sign extension.
icc: callee do the zero/sign extension while caller do nothong.

The way I read that, GCC and Clang are ABI compatible with each other.

The incompatibility exists between ICC and Clang/LLVM. ICC does not extend parameters, but Clang/LLVM assumes they will have been extended in the caller.

---

I think in cases where we disagree and are blocking each other, that's when we need to resort to flags to make forward progress. What do you think about adding a cl::opt for this feature? At the very least, this will allow folks to do their own performance testing, even if we ultimately default to GCC's behavior. I see the OpenVMS folks have a setting for this as well ("assume clean parameters").

Finally, maybe this should be discussed in the x86_64 psABI issue tracker. At the very least, the psabi docs could be clearer.

[emilio]

In D122963#3457431, @rnk wrote:

gcc: both caller and callee do the zero/sign extension.

The way I read that, GCC and Clang are ABI compatible with each other.

This is not right. GCC doesn't guarantee sign extension on the caller, and it doesn't always do it see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=92821 and https://bugs.llvm.org/show_bug.cgi?id=44228 for example.

[LuoYuanke]

I don't think we've reached agreement that this is a bug. I am concerned that removing these extension assert nodes will result in performance regressions.

The performance concern looks reasonable. We need collect performance data to prove there is no performance regression for typical benchmarks.

I think in cases where we disagree and are blocking each other, that's when we need to resort to flags to make forward progress. What do you think about adding a cl::opt for this feature? At the very least, this will allow folks to do their own performance testing, even if we ultimately default to GCC's behavior. I see the OpenVMS folks have a setting for this as well ("assume clean parameters").

Adding cl::opt looks good to me.

[rjmccall]

From Michael Matz's comments on the GCC bug and some of the linked conversation, it sounds pretty clear that the psABI does not require extension. They could certainly be more explicit about that in the psABI, but they're under no real obligation to: the baseline assumption should always be that things not stated are not guaranteed. If the psABI intended extension to be required, the document would really need to state it positively, rather than leaving it to be inferred. Since it does not, extension is not guaranteed by the psABI.

Of course, that creates a problem, because clang is not following the psABI. Platforms where clang is the system compiler have the option of declaring clang's current behavior to be their platform ABI and documenting the variance, if they like. Otherwise, clang needs to come into compliance with the psABI, but of course it also has to maintain compatibility with code emitted by old versions of clang, which are extending in the caller and assuming extension in the callee. As has been pointed out, those goals are not incompatible: we just need to conservatively extend in the caller and not assume extension in the callee. That won't be free, but it seems like what we have to do in order to make up for years of miscompilation, at least by default.

That said, I agree with the people who say that it's not the backend's responsibility to make this policy choice. The frontend should be emitting IR correctly to get the behavior it wants. So if we accept that the frontend might legitimately want conservative extension, the immediate question is whether we actually have a way to request it. I suspect that we don't, unless we've recently gotten much stricter about treating call-site and declaration attributes as independent.

So to recap:

1. We should add a way for frontends to request conservative extension.
2. clang should request conservative extension on most x86_64 platforms by default.
3. clang can add a flag to control the ABI it uses for clients that are unwilling to pay the cost of conservative extension.
4. Platforms that use clang as a system compiler should decide whether they want to (a) default to conservative extension or (b) document that they use caller extension and thus vary from the psABI.

[LiuChen3]

According to our internal tests, the performance of most of the tests is not affected. The most affected test performance dropped by %0.76. Average performance dropped 0.25%. This matches our expectations

It would be great if it could be fixed on the front end but now no one does. The compromise is to do workaround in the backend. I will fix it after the front end solves this issue.

There is a discussion about sign/zero extend parameter years ago: https://groups.google.com/g/x86-64-abi/c/E8O33onbnGQ. Obviously psABI does not guarantee parameter has been sign/zero extended.

[rjmccall]

If people think that we'll reliably get the right behavior by the frontend adding sext and zext to call sites but not to function arguments, I can walk you through how to make that change in the frontend.

What we can't do in the frontend is just switch to unconditionally not using sext and zext at all on x86_64, because we need to maintain compatibility with old versions of clang. But we can introduce a flag that makes clang generate incompatible code if someone wants to opt in to it for performance.

[LiuChen3]

In D122963#3460934, @rjmccall wrote:

If people think that we'll reliably get the right behavior by the frontend adding sext and zext to call sites but not to function arguments, I can walk you through how to make that change in the frontend.

What we can't do in the frontend is just switch to unconditionally not using sext and zext at all on x86_64, because we need to maintain compatibility with old versions of clang. But we can introduce a flag that makes clang generate incompatible code if someone wants to opt in to it for performance.

Thanks. @rjmccall . I'd like to hear your approach. If the change in FE is not large, I am willing to do it. But if it is too complicated, I may not be able to do that. Firstly I found we might be able to ignore the attribute in void CodeGenModule::ConstructAttributeList in clang/lib/CodeGen/CGCall.cpp , but it is the target independent part, so I gave up.
I agree introducing one new flag.

[rjmccall]

In D122963#3460976, @LiuChen3 wrote:

In D122963#3460934, @rjmccall wrote:

If people think that we'll reliably get the right behavior by the frontend adding sext and zext to call sites but not to function arguments, I can walk you through how to make that change in the frontend.

What we can't do in the frontend is just switch to unconditionally not using sext and zext at all on x86_64, because we need to maintain compatibility with old versions of clang. But we can introduce a flag that makes clang generate incompatible code if someone wants to opt in to it for performance.

Thanks. @rjmccall . I'd like to hear your approach. If the change in FE is not large, I am willing to do it. But if it is too complicated, I may not be able to do that. Firstly I found we might be able to ignore the attribute in void CodeGenModule::ConstructAttributeList in clang/lib/CodeGen/CGCall.cpp , but it is the target independent part, so I gave up.

I think you need to add a new ConservativeExtend kind to ABIArgInfo and make X86_64ABIInfo::classifyArgumentType (and classifyReturnType?) use that instead of Extend.

[LiuChen3]

I think you need to add a new ConservativeExtend kind to ABIArgInfo and make X86_64ABIInfo::classifyArgumentType (and classifyReturnType?) use that instead of Extend.

I will take a look. Thanks!

[lebedev.ri]

This review may be stuck/dead, consider abandoning if no longer relevant.
Removing myself as reviewer in attempt to clean dashboard.

[LuoYuanke]

In D122963#4049503, @lebedev.ri wrote:

This review may be stuck/dead, consider abandoning if no longer relevant.
Removing myself as reviewer in attempt to clean dashboard.

There is another solution implemented in https://reviews.llvm.org/D124435, so I think we can abandon this patch.

llvm/lib/Target/X86/X86ISelLowering.cpp

if (VA.isRegLoc()) {
         else
           llvm_unreachable("Unknown argument type!");
 
         Register Reg = MF.addLiveIn(VA.getLocReg(), RC);
         ArgValue = DAG.getCopyFromReg(Chain, dl, Reg, RegVT);
       }
 
       // If this is an 8 or 16-bit value, it is really passed promoted to 32
-      // bits.  Insert an assert[sz]ext to capture this, then truncate to the
-      // right size.
-      if (VA.getLocInfo() == CCValAssign::SExt)
+      // bits and the function is only local linked. Insert an assert[sz]ext to
+      // capture this, then truncate to the right size.
+      if (VA.getLocInfo() == CCValAssign::SExt && F.hasLocalLinkage() &&
+          !F.hasAddressTaken())
         ArgValue = DAG.getNode(ISD::AssertSext, dl, RegVT, ArgValue,
                                DAG.getValueType(VA.getValVT()));
-      else if (VA.getLocInfo() == CCValAssign::ZExt)
+      else if (VA.getLocInfo() == CCValAssign::ZExt && F.hasLocalLinkage() &&
+               !F.hasAddressTaken())
         ArgValue = DAG.getNode(ISD::AssertZext, dl, RegVT, ArgValue,
                                DAG.getValueType(VA.getValVT()));
       else if (VA.getLocInfo() == CCValAssign::BCvt)
         ArgValue = DAG.getBitcast(VA.getValVT(), ArgValue);
 
       if (VA.isExtInLoc()) {
         // Handle MMX values passed in XMM regs.
         if (RegVT.isVector() && VA.getValVT().getScalarType() != MVT::i1)
           ArgValue = DAG.getNode(X86ISD::MOVDQ2Q, dl, VA.getValVT(), ArgValue);
         else if (VA.getValVT().isVector() &&
                  VA.getValVT().getScalarType() == MVT::i1 &&

llvm/test/CodeGen/X86/2012-08-16-setcc.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown | FileCheck %s
 
 ; rdar://12081007
 
 define i32 @and_1(i8 zeroext %a, i8 zeroext %b, i32 %x) {
 ; CHECK-LABEL: and_1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    andl %esi, %eax
+; CHECK-NEXT:    xorl %eax, %eax
+; CHECK-NEXT:    testb %dil, %sil
 ; CHECK-NEXT:    cmovnel %edx, %eax
 ; CHECK-NEXT:    retq
   %1 = and i8 %b, %a
   %2 = icmp ne i8 %1, 0
   %3 = select i1 %2, i32 %x, i32 0
   ret i32 %3
 }
 
 define zeroext i1 @and_2(i8 zeroext %a, i8 zeroext %b) {
 ; CHECK-LABEL: and_2:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    testl %edi, %esi
+; CHECK-NEXT:    testb %dil, %sil
 ; CHECK-NEXT:    setne %al
 ; CHECK-NEXT:    retq
   %1 = and i8 %b, %a
   %2 = icmp ne i8 %1, 0
   ret i1 %2
 }
 
 define i32 @xor_1(i8 zeroext %a, i8 zeroext %b, i32 %x) {
 ; CHECK-LABEL: xor_1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    xorl %esi, %edi
+; CHECK-NEXT:    xorb %dil, %sil
 ; CHECK-NEXT:    cmovnel %edx, %eax
 ; CHECK-NEXT:    retq
   %1 = xor i8 %b, %a
   %2 = icmp ne i8 %1, 0
   %3 = select i1 %2, i32 %x, i32 0
   ret i32 %3
 }
 
 define zeroext i1 @xor_2(i8 zeroext %a, i8 zeroext %b) {
 ; CHECK-LABEL: xor_2:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    xorl %esi, %edi
+; CHECK-NEXT:    xorb %dil, %sil
 ; CHECK-NEXT:    setne %al
 ; CHECK-NEXT:    retq
   %1 = xor i8 %b, %a
   %2 = icmp ne i8 %1, 0
   ret i1 %2
 }
 

llvm/test/CodeGen/X86/addr-mode-matcher-2.ll

 ; X86-NEXT:    pushl %eax
 ; X86-NEXT:    calll bar@PLT
 ; X86-NEXT:    addl $4, %esp
 ; X86-NEXT:    jmp .LBB0_2
 ;
 ; X64-LABEL: foo:
 ; X64:       # %bb.0:
 ; X64-NEXT:    pushq %rax
-; X64-NEXT:    testl %edi, %edi
+; X64-NEXT:    testb %dil, %dil
 ; X64-NEXT:    je .LBB0_1
 ; X64-NEXT:  # %bb.3:
 ; X64-NEXT:    popq %rax
 ; X64-NEXT:    retq
 ; X64-NEXT:  .LBB0_1: # %.preheader
 ; X64-NEXT:    movl %esi, %eax
 ; X64-NEXT:    .p2align 4, 0x90
 ; X64-NEXT:  .LBB0_2: # =>This Inner Loop Header: Depth=1

llvm/test/CodeGen/X86/and-with-overflow.ll

 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:  .LBB1_2:
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: and_i8_rr:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %esi, %eax
 ; X64-NEXT:    andl %edi, %eax
+; X64-NEXT:    testb %al, %al
 ; X64-NEXT:    cmovel %edi, %eax
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %3 = and i8 %1, %0
   %4 = icmp eq i8 %3, 0
   %5 = select i1 %4, i8 %0, i8 %3
   ret i8 %5
 }

 ; X86-NEXT:  .LBB2_2:
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: and_i16_ri:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    andl $-17, %eax
+; X64-NEXT:    testw %ax, %ax
 ; X64-NEXT:    cmovel %edi, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %3 = and i16 %0, -17
   %4 = icmp eq i16 %3, 0
   %5 = select i1 %4, i16 %0, i16 %3
   ret i16 %5
 }

 ; X86-NEXT:  .LBB3_2:
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: and_i16_rr:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %esi, %eax
 ; X64-NEXT:    andl %edi, %eax
+; X64-NEXT:    testw %ax, %ax
 ; X64-NEXT:    cmovel %edi, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %3 = and i16 %1, %0
   %4 = icmp eq i16 %3, 0
   %5 = select i1 %4, i16 %0, i16 %3
   ret i16 %5
 }

llvm/test/CodeGen/X86/avx512-regcall-NoMask.ll

 ; X32-NEXT:    imull %ebx, %ecx
 ; X32-NEXT:    addl %eax, %ecx
 ; X32-NEXT:    addl {{[-0-9]+}}(%e{{[sb]}}p), %edi # 4-byte Folded Reload
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    addl (%esp), %eax # 4-byte Folded Reload
 ; X32-NEXT:    addl {{[0-9]+}}(%esp), %ebp
 ; X32-NEXT:    imull %ebp, %edi
 ; X32-NEXT:    addl {{[0-9]+}}(%esp), %esi
-; X32-NEXT:    imull {{[0-9]+}}(%esp), %esi
+; X32-NEXT:    imull {{[-0-9]+}}(%e{{[sb]}}p), %esi # 4-byte Folded Reload
 ; X32-NEXT:    addl %esi, %edi
 ; X32-NEXT:    addl {{[0-9]+}}(%esp), %edx
 ; X32-NEXT:    imull %eax, %edx
 ; X32-NEXT:    addl %edx, %edi
 ; X32-NEXT:    addl %ecx, %edi
 ; X32-NEXT:    movl %edi, %eax
 ; X32-NEXT:    addl $12, %esp
 ; X32-NEXT:    popl %ebx

 ; LINUXOSX64-NEXT:    subl %r12d, %edi
 ; LINUXOSX64-NEXT:    movl %eax, %edx
 ; LINUXOSX64-NEXT:    subl %ecx, %edx
 ; LINUXOSX64-NEXT:    imull %edx, %edi
 ; LINUXOSX64-NEXT:    leal (%r13,%r14), %edx
 ; LINUXOSX64-NEXT:    movl %r13d, %ebx
 ; LINUXOSX64-NEXT:    subl %r14d, %ebx
 ; LINUXOSX64-NEXT:    imull %ebp, %ebx
-; LINUXOSX64-NEXT:    movl 24(%rsp), %ebp
+; LINUXOSX64-NEXT:    movl {{[0-9]+}}(%rsp), %ebp
 ; LINUXOSX64-NEXT:    addl %edi, %ebx
 ; LINUXOSX64-NEXT:    movl %r15d, %edi
 ; LINUXOSX64-NEXT:    subl %ebp, %edi
 ; LINUXOSX64-NEXT:    imull %esi, %edi
 ; LINUXOSX64-NEXT:    addl %ebx, %edi
 ; LINUXOSX64-NEXT:    addl %ecx, %eax
 ; LINUXOSX64-NEXT:    imull %r8d, %eax
 ; LINUXOSX64-NEXT:    imull %r10d, %edx

 ; Test regcall when passing/retrieving mixed types
 define dso_local x86_regcallcc i32 @test_argRetMixTypes(double, float, i8 signext, i32, i64, i16 signext, i32*) #0 {
 ; X32-LABEL: test_argRetMixTypes:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushl %ebx
 ; X32-NEXT:    movl {{[0-9]+}}(%esp), %ebx
 ; X32-NEXT:    vcvtss2sd %xmm1, %xmm1, %xmm1
 ; X32-NEXT:    vaddsd %xmm0, %xmm1, %xmm0
+; X32-NEXT:    movsbl %al, %eax
 ; X32-NEXT:    vcvtsi2sd %eax, %xmm2, %xmm1
 ; X32-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; X32-NEXT:    vcvtsi2sd %ecx, %xmm2, %xmm1
 ; X32-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; X32-NEXT:    vmovd %edx, %xmm1
 ; X32-NEXT:    vpinsrd $1, %edi, %xmm1, %xmm1
 ; X32-NEXT:    vcvtqq2pd %ymm1, %ymm1
 ; X32-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
-; X32-NEXT:    vcvtsi2sd %esi, %xmm2, %xmm1
+; X32-NEXT:    movswl %si, %eax
+; X32-NEXT:    vcvtsi2sd %eax, %xmm2, %xmm1
 ; X32-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; X32-NEXT:    vcvtsi2sdl (%ebx), %xmm2, %xmm1
 ; X32-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; X32-NEXT:    vcvttsd2si %xmm0, %eax
 ; X32-NEXT:    popl %ebx
 ; X32-NEXT:    vzeroupper
 ; X32-NEXT:    retl
 ;
 ; WIN64-LABEL: test_argRetMixTypes:
 ; WIN64:       # %bb.0:
 ; WIN64-NEXT:    vcvtss2sd %xmm1, %xmm1, %xmm1
 ; WIN64-NEXT:    vaddsd %xmm0, %xmm1, %xmm0
+; WIN64-NEXT:    movsbl %al, %eax
 ; WIN64-NEXT:    vcvtsi2sd %eax, %xmm2, %xmm1
 ; WIN64-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; WIN64-NEXT:    vcvtsi2sd %ecx, %xmm2, %xmm1
 ; WIN64-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; WIN64-NEXT:    vcvtsi2sd %rdx, %xmm2, %xmm1
 ; WIN64-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
-; WIN64-NEXT:    vcvtsi2sd %edi, %xmm2, %xmm1
+; WIN64-NEXT:    movswl %di, %eax
+; WIN64-NEXT:    vcvtsi2sd %eax, %xmm2, %xmm1
 ; WIN64-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; WIN64-NEXT:    vcvtsi2sdl (%rsi), %xmm2, %xmm1
 ; WIN64-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; WIN64-NEXT:    vcvttsd2si %xmm0, %eax
 ; WIN64-NEXT:    retq
 ;
 ; LINUXOSX64-LABEL: test_argRetMixTypes:
 ; LINUXOSX64:       # %bb.0:
 ; LINUXOSX64-NEXT:    vcvtss2sd %xmm1, %xmm1, %xmm1
 ; LINUXOSX64-NEXT:    vaddsd %xmm0, %xmm1, %xmm0
+; LINUXOSX64-NEXT:    movsbl %al, %eax
 ; LINUXOSX64-NEXT:    vcvtsi2sd %eax, %xmm2, %xmm1
 ; LINUXOSX64-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; LINUXOSX64-NEXT:    vcvtsi2sd %ecx, %xmm2, %xmm1
 ; LINUXOSX64-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; LINUXOSX64-NEXT:    vcvtsi2sd %rdx, %xmm2, %xmm1
 ; LINUXOSX64-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
-; LINUXOSX64-NEXT:    vcvtsi2sd %edi, %xmm2, %xmm1
+; LINUXOSX64-NEXT:    movswl %di, %eax
+; LINUXOSX64-NEXT:    vcvtsi2sd %eax, %xmm2, %xmm1
 ; LINUXOSX64-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; LINUXOSX64-NEXT:    vcvtsi2sdl (%rsi), %xmm2, %xmm1
 ; LINUXOSX64-NEXT:    vaddsd %xmm1, %xmm0, %xmm0
 ; LINUXOSX64-NEXT:    vcvttsd2si %xmm0, %eax
 ; LINUXOSX64-NEXT:    retq
   %8 = fpext float %1 to double
   %9 = fadd double %8, %0
   %10 = sitofp i8 %2 to double

llvm/test/CodeGen/X86/avx512vl-vec-masked-cmp.ll

 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqb_v16i1_v32i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqb %xmm1, %xmm0, %xmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %1 = bitcast <2 x i64> %__b to <16 x i8>
   %2 = icmp eq <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqb_v16i1_v32i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqb (%rsi), %xmm0, %xmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %load = load <2 x i64>, <2 x i64>* %__b
   %1 = bitcast <2 x i64> %load to <16 x i8>
   %2 = icmp eq <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqb_v16i1_v64i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqb %xmm1, %xmm0, %xmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %1 = bitcast <2 x i64> %__b to <16 x i8>
   %2 = icmp eq <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqb_v16i1_v64i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqb (%rsi), %xmm0, %xmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %load = load <2 x i64>, <2 x i64>* %__b
   %1 = bitcast <2 x i64> %load to <16 x i8>
   %2 = icmp eq <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqw_v16i1_v32i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqw %ymm1, %ymm0, %ymm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %1 = bitcast <4 x i64> %__b to <16 x i16>
   %2 = icmp eq <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqw_v16i1_v32i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqw (%rsi), %ymm0, %ymm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %load = load <4 x i64>, <4 x i64>* %__b
   %1 = bitcast <4 x i64> %load to <16 x i16>
   %2 = icmp eq <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqw_v16i1_v64i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqw %ymm1, %ymm0, %ymm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %1 = bitcast <4 x i64> %__b to <16 x i16>
   %2 = icmp eq <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqw_v16i1_v64i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqw (%rsi), %ymm0, %ymm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %load = load <4 x i64>, <4 x i64>* %__b
   %1 = bitcast <4 x i64> %load to <16 x i16>
   %2 = icmp eq <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqd_v16i1_v32i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqd %zmm1, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %1 = bitcast <8 x i64> %__b to <16 x i32>
   %2 = icmp eq <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqd_v16i1_v32i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqd (%rsi), %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load <8 x i64>, <8 x i64>* %__b
   %1 = bitcast <8 x i64> %load to <16 x i32>
   %2 = icmp eq <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqd_v16i1_v32i1_mask_mem_b:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqd (%rsi){1to16}, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load i32, i32* %__b
   %vec = insertelement <16 x i32> undef, i32 %load, i32 0
   %1 = shufflevector <16 x i32> %vec, <16 x i32> undef, <16 x i32> <i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0>
   %2 = icmp eq <16 x i32> %0, %1

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqd_v16i1_v64i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqd %zmm1, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %1 = bitcast <8 x i64> %__b to <16 x i32>
   %2 = icmp eq <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqd_v16i1_v64i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqd (%rsi), %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load <8 x i64>, <8 x i64>* %__b
   %1 = bitcast <8 x i64> %load to <16 x i32>
   %2 = icmp eq <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpeqd_v16i1_v64i1_mask_mem_b:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpeqd (%rsi){1to16}, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load i32, i32* %__b
   %vec = insertelement <16 x i32> undef, i32 %load, i32 0
   %1 = shufflevector <16 x i32> %vec, <16 x i32> undef, <16 x i32> <i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0>
   %2 = icmp eq <16 x i32> %0, %1

 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtb_v16i1_v32i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtb %xmm1, %xmm0, %xmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %1 = bitcast <2 x i64> %__b to <16 x i8>
   %2 = icmp sgt <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtb_v16i1_v32i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtb (%rsi), %xmm0, %xmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %load = load <2 x i64>, <2 x i64>* %__b
   %1 = bitcast <2 x i64> %load to <16 x i8>
   %2 = icmp sgt <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtb_v16i1_v64i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtb %xmm1, %xmm0, %xmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %1 = bitcast <2 x i64> %__b to <16 x i8>
   %2 = icmp sgt <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtb_v16i1_v64i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtb (%rsi), %xmm0, %xmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %load = load <2 x i64>, <2 x i64>* %__b
   %1 = bitcast <2 x i64> %load to <16 x i8>
   %2 = icmp sgt <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtw_v16i1_v32i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtw %ymm1, %ymm0, %ymm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %1 = bitcast <4 x i64> %__b to <16 x i16>
   %2 = icmp sgt <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtw_v16i1_v32i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtw (%rsi), %ymm0, %ymm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %load = load <4 x i64>, <4 x i64>* %__b
   %1 = bitcast <4 x i64> %load to <16 x i16>
   %2 = icmp sgt <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtw_v16i1_v64i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtw %ymm1, %ymm0, %ymm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %1 = bitcast <4 x i64> %__b to <16 x i16>
   %2 = icmp sgt <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtw_v16i1_v64i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtw (%rsi), %ymm0, %ymm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %load = load <4 x i64>, <4 x i64>* %__b
   %1 = bitcast <4 x i64> %load to <16 x i16>
   %2 = icmp sgt <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtd_v16i1_v32i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtd %zmm1, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %1 = bitcast <8 x i64> %__b to <16 x i32>
   %2 = icmp sgt <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtd_v16i1_v32i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtd (%rsi), %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load <8 x i64>, <8 x i64>* %__b
   %1 = bitcast <8 x i64> %load to <16 x i32>
   %2 = icmp sgt <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtd_v16i1_v32i1_mask_mem_b:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtd (%rsi){1to16}, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load i32, i32* %__b
   %vec = insertelement <16 x i32> undef, i32 %load, i32 0
   %1 = shufflevector <16 x i32> %vec, <16 x i32> undef, <16 x i32> <i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0>
   %2 = icmp sgt <16 x i32> %0, %1

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtd_v16i1_v64i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtd %zmm1, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %1 = bitcast <8 x i64> %__b to <16 x i32>
   %2 = icmp sgt <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtd_v16i1_v64i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtd (%rsi), %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load <8 x i64>, <8 x i64>* %__b
   %1 = bitcast <8 x i64> %load to <16 x i32>
   %2 = icmp sgt <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpsgtd_v16i1_v64i1_mask_mem_b:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtd (%rsi){1to16}, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load i32, i32* %__b
   %vec = insertelement <16 x i32> undef, i32 %load, i32 0
   %1 = shufflevector <16 x i32> %vec, <16 x i32> undef, <16 x i32> <i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0>
   %2 = icmp sgt <16 x i32> %0, %1

 ; NoVLX-LABEL: test_masked_vpcmpsgeb_v16i1_v32i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtb %xmm0, %xmm1, %xmm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %1 = bitcast <2 x i64> %__b to <16 x i8>
   %2 = icmp sge <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vmovdqa (%rsi), %xmm1
 ; NoVLX-NEXT:    vpcmpgtb %xmm0, %xmm1, %xmm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %load = load <2 x i64>, <2 x i64>* %__b
   %1 = bitcast <2 x i64> %load to <16 x i8>
   %2 = icmp sge <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; NoVLX-LABEL: test_masked_vpcmpsgeb_v16i1_v64i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtb %xmm0, %xmm1, %xmm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %1 = bitcast <2 x i64> %__b to <16 x i8>
   %2 = icmp sge <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vmovdqa (%rsi), %xmm1
 ; NoVLX-NEXT:    vpcmpgtb %xmm0, %xmm1, %xmm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %load = load <2 x i64>, <2 x i64>* %__b
   %1 = bitcast <2 x i64> %load to <16 x i8>
   %2 = icmp sge <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; NoVLX-LABEL: test_masked_vpcmpsgew_v16i1_v32i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtw %ymm0, %ymm1, %ymm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %1 = bitcast <4 x i64> %__b to <16 x i16>
   %2 = icmp sge <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vmovdqa (%rsi), %ymm1
 ; NoVLX-NEXT:    vpcmpgtw %ymm0, %ymm1, %ymm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %load = load <4 x i64>, <4 x i64>* %__b
   %1 = bitcast <4 x i64> %load to <16 x i16>
   %2 = icmp sge <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; NoVLX-LABEL: test_masked_vpcmpsgew_v16i1_v64i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpgtw %ymm0, %ymm1, %ymm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %1 = bitcast <4 x i64> %__b to <16 x i16>
   %2 = icmp sge <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vmovdqa (%rsi), %ymm1
 ; NoVLX-NEXT:    vpcmpgtw %ymm0, %ymm1, %ymm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %load = load <4 x i64>, <4 x i64>* %__b
   %1 = bitcast <4 x i64> %load to <16 x i16>
   %2 = icmp sge <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpsged_v16i1_v32i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpnltd %zmm1, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %1 = bitcast <8 x i64> %__b to <16 x i32>
   %2 = icmp sge <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpsged_v16i1_v32i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpnltd (%rsi), %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load <8 x i64>, <8 x i64>* %__b
   %1 = bitcast <8 x i64> %load to <16 x i32>
   %2 = icmp sge <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpsged_v16i1_v32i1_mask_mem_b:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpnltd (%rsi){1to16}, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load i32, i32* %__b
   %vec = insertelement <16 x i32> undef, i32 %load, i32 0
   %1 = shufflevector <16 x i32> %vec, <16 x i32> undef, <16 x i32> <i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0>
   %2 = icmp sge <16 x i32> %0, %1

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpsged_v16i1_v64i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpnltd %zmm1, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %1 = bitcast <8 x i64> %__b to <16 x i32>
   %2 = icmp sge <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpsged_v16i1_v64i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpnltd (%rsi), %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load <8 x i64>, <8 x i64>* %__b
   %1 = bitcast <8 x i64> %load to <16 x i32>
   %2 = icmp sge <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpsged_v16i1_v64i1_mask_mem_b:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpnltd (%rsi){1to16}, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load i32, i32* %__b
   %vec = insertelement <16 x i32> undef, i32 %load, i32 0
   %1 = shufflevector <16 x i32> %vec, <16 x i32> undef, <16 x i32> <i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0>
   %2 = icmp sge <16 x i32> %0, %1

 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpmaxub %xmm1, %xmm0, %xmm1
 ; NoVLX-NEXT:    vpcmpeqb %xmm1, %xmm0, %xmm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %1 = bitcast <2 x i64> %__b to <16 x i8>
   %2 = icmp ult <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpmaxub (%rsi), %xmm0, %xmm1
 ; NoVLX-NEXT:    vpcmpeqb %xmm1, %xmm0, %xmm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %load = load <2 x i64>, <2 x i64>* %__b
   %1 = bitcast <2 x i64> %load to <16 x i8>
   %2 = icmp ult <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpmaxub %xmm1, %xmm0, %xmm1
 ; NoVLX-NEXT:    vpcmpeqb %xmm1, %xmm0, %xmm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %1 = bitcast <2 x i64> %__b to <16 x i8>
   %2 = icmp ult <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpmaxub (%rsi), %xmm0, %xmm1
 ; NoVLX-NEXT:    vpcmpeqb %xmm1, %xmm0, %xmm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxbd %xmm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <2 x i64> %__a to <16 x i8>
   %load = load <2 x i64>, <2 x i64>* %__b
   %1 = bitcast <2 x i64> %load to <16 x i8>
   %2 = icmp ult <16 x i8> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpmaxuw %ymm1, %ymm0, %ymm1
 ; NoVLX-NEXT:    vpcmpeqw %ymm1, %ymm0, %ymm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %1 = bitcast <4 x i64> %__b to <16 x i16>
   %2 = icmp ult <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpmaxuw (%rsi), %ymm0, %ymm1
 ; NoVLX-NEXT:    vpcmpeqw %ymm1, %ymm0, %ymm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %load = load <4 x i64>, <4 x i64>* %__b
   %1 = bitcast <4 x i64> %load to <16 x i16>
   %2 = icmp ult <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpmaxuw %ymm1, %ymm0, %ymm1
 ; NoVLX-NEXT:    vpcmpeqw %ymm1, %ymm0, %ymm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %1 = bitcast <4 x i64> %__b to <16 x i16>
   %2 = icmp ult <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpmaxuw (%rsi), %ymm0, %ymm1
 ; NoVLX-NEXT:    vpcmpeqw %ymm1, %ymm0, %ymm0
 ; NoVLX-NEXT:    vpternlogq $15, %zmm0, %zmm0, %zmm0
 ; NoVLX-NEXT:    vpmovsxwd %ymm0, %zmm0
 ; NoVLX-NEXT:    vptestmd %zmm0, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <4 x i64> %__a to <16 x i16>
   %load = load <4 x i64>, <4 x i64>* %__b
   %1 = bitcast <4 x i64> %load to <16 x i16>
   %2 = icmp ult <16 x i16> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpultd_v16i1_v32i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpltud %zmm1, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %1 = bitcast <8 x i64> %__b to <16 x i32>
   %2 = icmp ult <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpultd_v16i1_v32i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpltud (%rsi), %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load <8 x i64>, <8 x i64>* %__b
   %1 = bitcast <8 x i64> %load to <16 x i32>
   %2 = icmp ult <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpultd_v16i1_v32i1_mask_mem_b:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpltud (%rsi){1to16}, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load i32, i32* %__b
   %vec = insertelement <16 x i32> undef, i32 %load, i32 0
   %1 = shufflevector <16 x i32> %vec, <16 x i32> undef, <16 x i32> <i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0>
   %2 = icmp ult <16 x i32> %0, %1

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpultd_v16i1_v64i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpltud %zmm1, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %1 = bitcast <8 x i64> %__b to <16 x i32>
   %2 = icmp ult <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpultd_v16i1_v64i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpltud (%rsi), %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load <8 x i64>, <8 x i64>* %__b
   %1 = bitcast <8 x i64> %load to <16 x i32>
   %2 = icmp ult <16 x i32> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vpcmpultd_v16i1_v64i1_mask_mem_b:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vpcmpltud (%rsi){1to16}, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x i32>
   %load = load i32, i32* %__b
   %vec = insertelement <16 x i32> undef, i32 %load, i32 0
   %1 = shufflevector <16 x i32> %vec, <16 x i32> undef, <16 x i32> <i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0>
   %2 = icmp ult <16 x i32> %0, %1

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vcmpoeqps_v16i1_v32i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vcmpeqps %zmm1, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x float>
   %1 = bitcast <8 x i64> %__b to <16 x float>
   %2 = fcmp oeq <16 x float> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vcmpoeqps_v16i1_v32i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vcmpeqps (%rsi), %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x float>
   %load = load <8 x i64>, <8 x i64>* %__b
   %1 = bitcast <8 x i64> %load to <16 x float>
   %2 = fcmp oeq <16 x float> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vcmpoeqps_v16i1_v32i1_mask_mem_b:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vcmpeqps (%rsi){1to16}, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x float>
   %load = load float, float* %__b
   %vec = insertelement <16 x float> undef, float %load, i32 0
   %1 = shufflevector <16 x float> %vec, <16 x float> undef, <16 x i32> <i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0>
   %2 = fcmp oeq <16 x float> %0, %1

 }
 
 define zeroext i32 @test_masked_vcmpoeqps_v16i1_v32i1_sae_mask(i16 zeroext %__u, <8 x i64> %__a, <8 x i64> %__b) local_unnamed_addr {
 ; VLX-LABEL: test_masked_vcmpoeqps_v16i1_v32i1_sae_mask:
 ; VLX:       # %bb.0: # %entry
 ; VLX-NEXT:    vcmpleps {sae}, %zmm1, %zmm0, %k0
 ; VLX-NEXT:    kmovd %k0, %eax
 ; VLX-NEXT:    andl %edi, %eax
+; VLX-NEXT:    movzwl %ax, %eax
 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vcmpoeqps_v16i1_v32i1_sae_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vcmpleps {sae}, %zmm1, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x float>
   %1 = bitcast <8 x i64> %__b to <16 x float>
   %2 = call <16 x i1> @llvm.x86.avx512.mask.cmp.ps.512(<16 x float> %0, <16 x float> %1, i32 2, <16 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>, i32 8)
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vcmpoeqps_v16i1_v64i1_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vcmpeqps %zmm1, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x float>
   %1 = bitcast <8 x i64> %__b to <16 x float>
   %2 = fcmp oeq <16 x float> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vcmpoeqps_v16i1_v64i1_mask_mem:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vcmpeqps (%rsi), %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x float>
   %load = load <8 x i64>, <8 x i64>* %__b
   %1 = bitcast <8 x i64> %load to <16 x float>
   %2 = fcmp oeq <16 x float> %0, %1
   %3 = bitcast i16 %__u to <16 x i1>

 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vcmpoeqps_v16i1_v64i1_mask_mem_b:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vcmpeqps (%rsi){1to16}, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x float>
   %load = load float, float* %__b
   %vec = insertelement <16 x float> undef, float %load, i32 0
   %1 = shufflevector <16 x float> %vec, <16 x float> undef, <16 x i32> <i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0, i32 0>
   %2 = fcmp oeq <16 x float> %0, %1

 }
 
 define zeroext i64 @test_masked_vcmpoeqps_v16i1_v64i1_sae_mask(i16 zeroext %__u, <8 x i64> %__a, <8 x i64> %__b) local_unnamed_addr {
 ; VLX-LABEL: test_masked_vcmpoeqps_v16i1_v64i1_sae_mask:
 ; VLX:       # %bb.0: # %entry
 ; VLX-NEXT:    vcmpleps {sae}, %zmm1, %zmm0, %k0
 ; VLX-NEXT:    kmovd %k0, %eax
 ; VLX-NEXT:    andl %edi, %eax
+; VLX-NEXT:    movzwl %ax, %eax
 ; VLX-NEXT:    vzeroupper
 ; VLX-NEXT:    retq
 ;
 ; NoVLX-LABEL: test_masked_vcmpoeqps_v16i1_v64i1_sae_mask:
 ; NoVLX:       # %bb.0: # %entry
 ; NoVLX-NEXT:    vcmpleps {sae}, %zmm1, %zmm0, %k0
 ; NoVLX-NEXT:    kmovw %k0, %eax
 ; NoVLX-NEXT:    andl %edi, %eax
+; NoVLX-NEXT:    movzwl %ax, %eax
 ; NoVLX-NEXT:    vzeroupper
 ; NoVLX-NEXT:    retq
 entry:
   %0 = bitcast <8 x i64> %__a to <16 x float>
   %1 = bitcast <8 x i64> %__b to <16 x float>
   %2 = call <16 x i1> @llvm.x86.avx512.mask.cmp.ps.512(<16 x float> %0, <16 x float> %1, i32 2, <16 x i1> <i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true, i1 true>, i32 8)
   %3 = bitcast i16 %__u to <16 x i1>
   %4 = and <16 x i1> %2, %3

llvm/test/CodeGen/X86/bitcast-int-to-vector-bool.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+sse2 | FileCheck %s --check-prefixes=SSE2-SSSE3,SSE2
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+ssse3 | FileCheck %s --check-prefixes=SSE2-SSSE3,SSSE3
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx | FileCheck %s --check-prefixes=AVX12,AVX1
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx2 | FileCheck %s --check-prefixes=AVX12,AVX2
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+avx512f,+avx512vl,+avx512bw | FileCheck %s --check-prefixes=AVX512
 
 define <2 x i1> @bitcast_i2_2i1(i2 zeroext %a0) {
 ; SSE2-SSSE3-LABEL: bitcast_i2_2i1:
 ; SSE2-SSSE3:       # %bb.0:
-; SSE2-SSSE3-NEXT:    movd %edi, %xmm0
+; SSE2-SSSE3-NEXT:    movzbl %dil, %eax
+; SSE2-SSSE3-NEXT:    movd %eax, %xmm0
 ; SSE2-SSSE3-NEXT:    pshufd {{.*#+}} xmm1 = xmm0[0,1,0,1]
 ; SSE2-SSSE3-NEXT:    movdqa {{.*#+}} xmm0 = [1,2]
 ; SSE2-SSSE3-NEXT:    pand %xmm0, %xmm1
 ; SSE2-SSSE3-NEXT:    pcmpeqd %xmm0, %xmm1
 ; SSE2-SSSE3-NEXT:    pshufd {{.*#+}} xmm0 = xmm1[1,0,3,2]
 ; SSE2-SSSE3-NEXT:    pand %xmm1, %xmm0
 ; SSE2-SSSE3-NEXT:    psrlq $63, %xmm0
 ; SSE2-SSSE3-NEXT:    retq
 ;
 ; AVX1-LABEL: bitcast_i2_2i1:
 ; AVX1:       # %bb.0:
-; AVX1-NEXT:    vmovd %edi, %xmm0
+; AVX1-NEXT:    movzbl %dil, %eax
+; AVX1-NEXT:    vmovd %eax, %xmm0
 ; AVX1-NEXT:    vpshufd {{.*#+}} xmm0 = xmm0[0,1,0,1]
 ; AVX1-NEXT:    vmovdqa {{.*#+}} xmm1 = [1,2]
 ; AVX1-NEXT:    vpand %xmm1, %xmm0, %xmm0
 ; AVX1-NEXT:    vpcmpeqq %xmm1, %xmm0, %xmm0
 ; AVX1-NEXT:    vpsrlq $63, %xmm0, %xmm0
 ; AVX1-NEXT:    retq
 ;
 ; AVX2-LABEL: bitcast_i2_2i1:
 ; AVX2:       # %bb.0:
-; AVX2-NEXT:    vmovd %edi, %xmm0
+; AVX2-NEXT:    movzbl %dil, %eax
+; AVX2-NEXT:    vmovd %eax, %xmm0
 ; AVX2-NEXT:    vpbroadcastq %xmm0, %xmm0
 ; AVX2-NEXT:    vmovdqa {{.*#+}} xmm1 = [1,2]
 ; AVX2-NEXT:    vpand %xmm1, %xmm0, %xmm0
 ; AVX2-NEXT:    vpcmpeqq %xmm1, %xmm0, %xmm0
 ; AVX2-NEXT:    vpsrlq $63, %xmm0, %xmm0
 ; AVX2-NEXT:    retq
 ;
 ; AVX512-LABEL: bitcast_i2_2i1:
 ; AVX512:       # %bb.0:
 ; AVX512-NEXT:    kmovd %edi, %k1
 ; AVX512-NEXT:    vpcmpeqd %xmm0, %xmm0, %xmm0
 ; AVX512-NEXT:    vmovdqa64 %xmm0, %xmm0 {%k1} {z}
 ; AVX512-NEXT:    retq
   %1 = bitcast i2 %a0 to <2 x i1>
   ret <2 x i1> %1
 }
 
 define <4 x i1> @bitcast_i4_4i1(i4 zeroext %a0) {
 ; SSE2-SSSE3-LABEL: bitcast_i4_4i1:
 ; SSE2-SSSE3:       # %bb.0:
-; SSE2-SSSE3-NEXT:    movd %edi, %xmm0
+; SSE2-SSSE3-NEXT:    movzbl %dil, %eax
+; SSE2-SSSE3-NEXT:    movd %eax, %xmm0
 ; SSE2-SSSE3-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,0,0,0]
 ; SSE2-SSSE3-NEXT:    movdqa {{.*#+}} xmm1 = [1,2,4,8]
 ; SSE2-SSSE3-NEXT:    pand %xmm1, %xmm0
 ; SSE2-SSSE3-NEXT:    pcmpeqd %xmm1, %xmm0
 ; SSE2-SSSE3-NEXT:    psrld $31, %xmm0
 ; SSE2-SSSE3-NEXT:    retq
 ;
 ; AVX1-LABEL: bitcast_i4_4i1:
 ; AVX1:       # %bb.0:
-; AVX1-NEXT:    vmovd %edi, %xmm0
+; AVX1-NEXT:    movzbl %dil, %eax
+; AVX1-NEXT:    vmovd %eax, %xmm0
 ; AVX1-NEXT:    vpshufd {{.*#+}} xmm0 = xmm0[0,0,0,0]
 ; AVX1-NEXT:    vmovdqa {{.*#+}} xmm1 = [1,2,4,8]
 ; AVX1-NEXT:    vpand %xmm1, %xmm0, %xmm0
 ; AVX1-NEXT:    vpcmpeqd %xmm1, %xmm0, %xmm0
 ; AVX1-NEXT:    vpsrld $31, %xmm0, %xmm0
 ; AVX1-NEXT:    retq
 ;
 ; AVX2-LABEL: bitcast_i4_4i1:
 ; AVX2:       # %bb.0:
-; AVX2-NEXT:    vmovd %edi, %xmm0
+; AVX2-NEXT:    movzbl %dil, %eax
+; AVX2-NEXT:    vmovd %eax, %xmm0
 ; AVX2-NEXT:    vpbroadcastd %xmm0, %xmm0
 ; AVX2-NEXT:    vmovdqa {{.*#+}} xmm1 = [1,2,4,8]
 ; AVX2-NEXT:    vpand %xmm1, %xmm0, %xmm0
 ; AVX2-NEXT:    vpcmpeqd %xmm1, %xmm0, %xmm0
 ; AVX2-NEXT:    vpsrld $31, %xmm0, %xmm0
 ; AVX2-NEXT:    retq
 ;
 ; AVX512-LABEL: bitcast_i4_4i1:

llvm/test/CodeGen/X86/bool-ext-inc.ll

; CHECK-NEXT:    retq
   %b = zext <4 x i1> %c to <4 x i32>
   %s = sub <4 x i32> %x, %b
   ret <4 x i32> %s
 }
 
 define i32 @assertsext_sub_1(i1 signext %cond, i32 %y) {
 ; CHECK-LABEL: assertsext_sub_1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    # kill: def $esi killed $esi def $rsi
-; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
-; CHECK-NEXT:    leal (%rdi,%rsi), %eax
+; CHECK-NEXT:    movl %esi, %eax
+; CHECK-NEXT:    movzbl %dil, %ecx
+; CHECK-NEXT:    andl $1, %ecx
+; CHECK-NEXT:    subl %ecx, %eax
 ; CHECK-NEXT:    retq
   %e = zext i1 %cond to i32
   %r = sub i32 %y, %e
   ret i32 %r
 }
 
 define i32 @assertsext_add_1(i1 signext %cond, i32 %y) {
 ; CHECK-LABEL: assertsext_add_1:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl %esi, %eax
-; CHECK-NEXT:    subl %edi, %eax
+; CHECK-NEXT:    movzbl %dil, %eax
+; CHECK-NEXT:    andl $1, %eax
+; CHECK-NEXT:    addl %esi, %eax
 ; CHECK-NEXT:    retq
   %e = zext i1 %cond to i32
   %r = add i32 %e, %y
   ret i32 %r
 }
 
 define i32 @assertsext_add_1_commute(i1 signext %cond, i32 %y) {
 ; CHECK-LABEL: assertsext_add_1_commute:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl %esi, %eax
-; CHECK-NEXT:    subl %edi, %eax
+; CHECK-NEXT:    movzbl %dil, %eax
+; CHECK-NEXT:    andl $1, %eax
+; CHECK-NEXT:    addl %esi, %eax
 ; CHECK-NEXT:    retq
   %e = zext i1 %cond to i32
   %r = add i32 %y, %e
   ret i32 %r
 }
 

llvm/test/CodeGen/X86/bool-zext.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686-unknown-unknown | FileCheck %s -check-prefix=X32
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown | FileCheck %s -check-prefix=X64
 
-; It's not necessary to zero-extend the arg because it is specified 'zeroext'.
 define void @bar1(i1 zeroext %v1) nounwind ssp {
 ; X32-LABEL: bar1:
 ; X32:       # %bb.0:
 ; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    pushl %eax
 ; X32-NEXT:    calll foo1
 ; X32-NEXT:    addl $4, %esp
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: bar1:
 ; X64:       # %bb.0:
+; X64-NEXT:    movzbl %dil, %edi
 ; X64-NEXT:    xorl %eax, %eax
 ; X64-NEXT:    jmp foo1 # TAILCALL
   %conv = zext i1 %v1 to i32
   %call = tail call i32 (...) @foo1(i32 %conv) nounwind
   ret void
 }
 
 ; Check that on x86-64 the arguments are simply forwarded.
 define void @bar2(i8 zeroext %v1) nounwind ssp {
 ; X32-LABEL: bar2:
 ; X32:       # %bb.0:
 ; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    pushl %eax
 ; X32-NEXT:    calll foo1
 ; X32-NEXT:    addl $4, %esp
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: bar2:
 ; X64:       # %bb.0:
+; X64-NEXT:    movzbl %dil, %edi
 ; X64-NEXT:    xorl %eax, %eax
 ; X64-NEXT:    jmp foo1 # TAILCALL
   %conv = zext i8 %v1 to i32
   %call = tail call i32 (...) @foo1(i32 %conv) nounwind
   ret void
 }
 
 ; Check that i1 return values are not zero-extended.

llvm/test/CodeGen/X86/callee-extend.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; Ignore the zeroext/signext attribute and always do sign/zero extension in the callee.
+; RUN: llc < %s -mtriple=x86_64-unknown-linux | FileCheck %s
+
+define i64 @calleeu8(i8 noundef zeroext %a) nounwind {
+; CHECK-LABEL: calleeu8:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
+; CHECK-NEXT:    movzbl %dil, %eax
+; CHECK-NEXT:    retq
+entry:
+  %conv = zext i8 %a to i64
+  ret i64 %conv
+}
+
+define i64 @calleeu16(i16 noundef zeroext %a) nounwind {
+; CHECK-LABEL: calleeu16:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
+; CHECK-NEXT:    movzwl %di, %eax
+; CHECK-NEXT:    retq
+entry:
+  %conv = zext i16 %a to i64
+  ret i64 %conv
+}
+
+define i64 @callees8(i8 noundef signext %a) nounwind {
+; CHECK-LABEL: callees8:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
+; CHECK-NEXT:    movsbq %dil, %rax
+; CHECK-NEXT:    retq
+entry:
+  %conv = sext i8 %a to i64
+  ret i64 %conv
+}
+
+define i64 @callees16(i16 noundef signext %a) nounwind {
+; CHECK-LABEL: callees16:
+; CHECK:       # %bb.0: # %entry
+; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
+; CHECK-NEXT:    movswq %di, %rax
+; CHECK-NEXT:    retq
+entry:
+  %conv = sext i16 %a to i64
+  ret i64 %conv
+}
+

llvm/test/CodeGen/X86/cmp-bool.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=x86_64 < %s | FileCheck %s
 
 define void @bool_eq(i1 zeroext %a, i1 zeroext %b, void ()* nocapture %c) nounwind {
 ; CHECK-LABEL: bool_eq:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    xorl %esi, %edi
+; CHECK-NEXT:    cmpb %sil, %dil
 ; CHECK-NEXT:    je .LBB0_2
 ; CHECK-NEXT:  # %bb.1: # %if.end
 ; CHECK-NEXT:    retq
 ; CHECK-NEXT:  .LBB0_2: # %if.then
 ; CHECK-NEXT:    jmpq *%rdx # TAILCALL
 entry:
   %0 = xor i1 %a, %b
   br i1 %0, label %if.end, label %if.then

llvm/test/CodeGen/X86/cmp.ll

; CHECK-NEXT:    retq # encoding: [0xc3]
   ret { i64, i64 } %9
 }
 
 ; Make sure we use a 32-bit comparison without an extend based on the input
 ; being pre-sign extended by caller.
 define i32 @pr42189(i16 signext %c) {
 ; CHECK-LABEL: pr42189:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    cmpl $32767, %edi # encoding: [0x81,0xff,0xff,0x7f,0x00,0x00]
+; CHECK-NEXT:    movzwl %di, %eax # encoding: [0x0f,0xb7,0xc7]
+; CHECK-NEXT:    cmpl $32767, %eax # encoding: [0x3d,0xff,0x7f,0x00,0x00]
 ; CHECK-NEXT:    # imm = 0x7FFF
 ; CHECK-NEXT:    jne .LBB45_2 # encoding: [0x75,A]
 ; CHECK-NEXT:    # fixup A - offset: 1, value: .LBB45_2-1, kind: FK_PCRel_1
 ; CHECK-NEXT:  # %bb.1: # %if.then
 ; CHECK-NEXT:    jmp g@PLT # TAILCALL
 ; CHECK-NEXT:    # encoding: [0xeb,A]
 ; CHECK-NEXT:    # fixup A - offset: 1, value: g@PLT-1, kind: FK_PCRel_1
 ; CHECK-NEXT:  .LBB45_2: # %if.end

llvm/test/CodeGen/X86/commute-two-addr.ll

 ; rdar://8762995
 %0 = type { i64, i32 }
 
 define %0 @t3(i32 %lb, i8 zeroext %has_lb, i8 zeroext %lb_inclusive, i32 %ub, i8 zeroext %has_ub, i8 zeroext %ub_inclusive) nounwind {
 entry:
 ; DARWIN-LABEL: t3:
 ; DARWIN: shlq $32, %rcx
 ; DARWIN-NEXT: orq %rcx, %rax
+; DARWIN-NEXT: movzbl  %sil, %ecx
+; DARWIN-NEXT: movzbl  %r8b, %esi
 ; DARWIN-NEXT: shll $8
 ; DARWIN-NOT: leaq
   %tmp21 = zext i32 %lb to i64
   %tmp23 = zext i32 %ub to i64
   %tmp24 = shl i64 %tmp23, 32
   %ins26 = or i64 %tmp24, %tmp21
   %tmp28 = zext i8 %has_lb to i32
   %tmp33 = zext i8 %has_ub to i32

llvm/test/CodeGen/X86/critical-edge-split-2.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown | FileCheck %s
 
 %0 = type <{ %1, %1 }>
 %1 = type { i8, i8, i8, i8 }
 
 @g_2 = dso_local global %0 zeroinitializer
 @g_4 = dso_local global %1 zeroinitializer, align 4
 
 ; PR8642
 define i16 @test1(i1 zeroext %C, i8** nocapture %argv) nounwind ssp {
 ; CHECK-LABEL: test1:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movw $1, %ax
-; CHECK-NEXT:    testl %edi, %edi
+; CHECK-NEXT:    testb %dil, %dil
 ; CHECK-NEXT:    jne .LBB0_2
 ; CHECK-NEXT:  # %bb.1: # %cond.false.i
 ; CHECK-NEXT:    movl $g_2+4, %eax
 ; CHECK-NEXT:    xorl %ecx, %ecx
 ; CHECK-NEXT:    cmpq $g_4, %rax
 ; CHECK-NEXT:    sete %cl
 ; CHECK-NEXT:    movl $1, %eax
 ; CHECK-NEXT:    xorl %edx, %edx

llvm/test/CodeGen/X86/dagcombine-shifts.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-linux-gnu | FileCheck %s
 
 ; fold (shl (zext (lshr (A, X))), X) -> (zext (shl (lshr (A, X)), X))
 
 ; Canolicalize the sequence shl/zext/lshr performing the zeroextend
 ; as the last instruction of the sequence.
 ; This will help DAGCombiner to identify and then fold the sequence
 ; of shifts into a single AND.
 ; This transformation is profitable if the shift amounts are the same
 ; and if there is only one use of the zext.
 
 define i16 @fun1(i8 zeroext %v) {
 ; CHECK-LABEL: fun1:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    andl $-16, %eax
+; CHECK-NEXT:    andl $240, %eax
 ; CHECK-NEXT:    # kill: def $ax killed $ax killed $eax
 ; CHECK-NEXT:    retq
 entry:
   %shr = lshr i8 %v, 4
   %ext = zext i8 %shr to i16
   %shl = shl i16 %ext, 4
   ret i16 %shl
 }
 
 define i32 @fun2(i8 zeroext %v) {
 ; CHECK-LABEL: fun2:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    andl $-16, %eax
+; CHECK-NEXT:    andl $240, %eax
 ; CHECK-NEXT:    retq
 entry:
   %shr = lshr i8 %v, 4
   %ext = zext i8 %shr to i32
   %shl = shl i32 %ext, 4
   ret i32 %shl
 }
 
 define i32 @fun3(i16 zeroext %v) {
 ; CHECK-LABEL: fun3:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    andl $-16, %eax
+; CHECK-NEXT:    andl $65520, %eax # imm = 0xFFF0
 ; CHECK-NEXT:    retq
 entry:
   %shr = lshr i16 %v, 4
   %ext = zext i16 %shr to i32
   %shl = shl i32 %ext, 4
   ret i32 %shl
 }
 
 define i64 @fun4(i8 zeroext %v) {
 ; CHECK-LABEL: fun4:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    andl $-16, %eax
+; CHECK-NEXT:    andl $240, %eax
 ; CHECK-NEXT:    retq
 entry:
   %shr = lshr i8 %v, 4
   %ext = zext i8 %shr to i64
   %shl = shl i64 %ext, 4
   ret i64 %shl
 }
 
 define i64 @fun5(i16 zeroext %v) {
 ; CHECK-LABEL: fun5:
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    andl $-16, %eax
+; CHECK-NEXT:    andl $65520, %eax # imm = 0xFFF0
 ; CHECK-NEXT:    retq
 entry:
   %shr = lshr i16 %v, 4
   %ext = zext i16 %shr to i64
   %shl = shl i64 %ext, 4
   ret i64 %shl
 }
 
entry:
   %shl = shl i64 %ext, 4
   %add = add i64 %shl, %ext
   ret i64 %add
 }
 
 define i64 @fun11(i16 zeroext %v) {
 ; CHECK-LABEL: fun11:
 ; CHECK:       # %bb.0: # %entry
-; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
-; CHECK-NEXT:    shrl $4, %edi
-; CHECK-NEXT:    movq %rdi, %rax
+; CHECK-NEXT:    movzwl %di, %ecx
+; CHECK-NEXT:    shrl $4, %ecx
+; CHECK-NEXT:    movq %rcx, %rax
 ; CHECK-NEXT:    shlq $4, %rax
-; CHECK-NEXT:    addq %rdi, %rax
+; CHECK-NEXT:    addq %rcx, %rax
 ; CHECK-NEXT:    retq
 entry:
   %shr = lshr i16 %v, 4
   %ext = zext i16 %shr to i64
   %shl = shl i64 %ext, 4
   %add = add i64 %shl, %ext
   ret i64 %add
 }

llvm/test/CodeGen/X86/div-rem-pair-recomposition-signed.ll

 ;
 ; X64-LABEL: multiple_bb:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdx, %r9
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    cltd
 ; X64-NEXT:    idivl %esi
 ; X64-NEXT:    movl %eax, (%r9)
-; X64-NEXT:    testl %ecx, %ecx
+; X64-NEXT:    testb %cl, %cl
 ; X64-NEXT:    je .LBB11_2
 ; X64-NEXT:  # %bb.1: # %do_srem
 ; X64-NEXT:    movl %eax, %ecx
 ; X64-NEXT:    imull %esi, %ecx
 ; X64-NEXT:    subl %ecx, %edi
 ; X64-NEXT:    movl %edi, (%r8)
 ; X64-NEXT:  .LBB11_2: # %end
 ; X64-NEXT:    retq

llvm/test/CodeGen/X86/div-rem-pair-recomposition-unsigned.ll

 ;
 ; X64-LABEL: multiple_bb:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdx, %r9
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    xorl %edx, %edx
 ; X64-NEXT:    divl %esi
 ; X64-NEXT:    movl %eax, (%r9)
-; X64-NEXT:    testl %ecx, %ecx
+; X64-NEXT:    testb %cl, %cl
 ; X64-NEXT:    je .LBB11_2
 ; X64-NEXT:  # %bb.1: # %do_urem
 ; X64-NEXT:    movl %eax, %ecx
 ; X64-NEXT:    imull %esi, %ecx
 ; X64-NEXT:    subl %ecx, %edi
 ; X64-NEXT:    movl %edi, (%r8)
 ; X64-NEXT:  .LBB11_2: # %end
 ; X64-NEXT:    retq

llvm/test/CodeGen/X86/divide-by-constant.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=i686-pc-linux-gnu | FileCheck %s --check-prefix=X32
 ; RUN: llc < %s -mtriple=x86_64-pc-linux-gnu | FileCheck %s --check-prefix=X64 --check-prefix=X64-FAST
 ; RUN: llc < %s -mtriple=x86_64-pc-linux-gnu -mattr=idivq-to-divl | FileCheck %s --check-prefix=X64 --check-prefix=X64-SLOW
 
 define zeroext i16 @test1(i16 zeroext %x) nounwind {
 ; X32-LABEL: test1:
 ; X32:       # %bb.0: # %entry
 ; X32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    imull $63551, %eax, %eax # imm = 0xF83F
 ; X32-NEXT:    shrl $21, %eax
 ; X32-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: test1:
 ; X64:       # %bb.0: # %entry
-; X64-NEXT:    imull $63551, %edi, %eax # imm = 0xF83F
+; X64-NEXT:    movzwl %di, %eax
+; X64-NEXT:    imull $63551, %eax, %eax # imm = 0xF83F
 ; X64-NEXT:    shrl $21, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
 entry:
   %div = udiv i16 %x, 33
   ret i16 %div
 }
 
 define zeroext i16 @test2(i8 signext %x, i16 zeroext %c) nounwind readnone ssp noredzone {
 ; X32-LABEL: test2:
 ; X32:       # %bb.0: # %entry
 ; X32-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    imull $43691, %eax, %eax # imm = 0xAAAB
 ; X32-NEXT:    shrl $17, %eax
 ; X32-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: test2:
 ; X64:       # %bb.0: # %entry
-; X64-NEXT:    imull $43691, %esi, %eax # imm = 0xAAAB
+; X64-NEXT:    movzwl %si, %eax
+; X64-NEXT:    imull $43691, %eax, %eax # imm = 0xAAAB
 ; X64-NEXT:    shrl $17, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
 entry:
   %div = udiv i16 %c, 3
   ret i16 %div
 
 }
 
 define zeroext i8 @test3(i8 zeroext %x, i8 zeroext %c) nounwind readnone ssp noredzone {
 ; X32-LABEL: test3:
 ; X32:       # %bb.0: # %entry
 ; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    imull $171, %eax, %eax
 ; X32-NEXT:    shrl $9, %eax
 ; X32-NEXT:    # kill: def $al killed $al killed $eax
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: test3:
 ; X64:       # %bb.0: # %entry
-; X64-NEXT:    imull $171, %esi, %eax
+; X64-NEXT:    movzbl %sil, %eax
+; X64-NEXT:    imull $171, %eax, %eax
 ; X64-NEXT:    shrl $9, %eax
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
 entry:
   %div = udiv i8 %c, 3
   ret i8 %div
 }
 
 define signext i16 @test4(i16 signext %x) nounwind {
 ; X32-LABEL: test4:
 ; X32:       # %bb.0: # %entry
 ; X32-NEXT:    movswl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    imull $1986, %eax, %eax # imm = 0x7C2
 ; X32-NEXT:    movl %eax, %ecx
 ; X32-NEXT:    shrl $31, %ecx
 ; X32-NEXT:    shrl $16, %eax
 ; X32-NEXT:    addl %ecx, %eax
 ; X32-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: test4:
 ; X64:       # %bb.0: # %entry
-; X64-NEXT:    imull $1986, %edi, %eax # imm = 0x7C2
+; X64-NEXT:    movswl %di, %eax
+; X64-NEXT:    imull $1986, %eax, %eax # imm = 0x7C2
 ; X64-NEXT:    movl %eax, %ecx
 ; X64-NEXT:    shrl $31, %ecx
 ; X64-NEXT:    shrl $16, %eax
 ; X64-NEXT:    addl %ecx, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
 entry:
   %div = sdiv i16 %x, 33    ; <i32> [#uses=1]

 ; X32-NEXT:    shrl $31, %ecx
 ; X32-NEXT:    sarl $18, %eax
 ; X32-NEXT:    addl %ecx, %eax
 ; X32-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: test6:
 ; X64:       # %bb.0: # %entry
-; X64-NEXT:    imull $26215, %edi, %eax # imm = 0x6667
+; X64-NEXT:    movswl %di, %eax
+; X64-NEXT:    imull $26215, %eax, %eax # imm = 0x6667
 ; X64-NEXT:    movl %eax, %ecx
 ; X64-NEXT:    shrl $31, %ecx
 ; X64-NEXT:    sarl $18, %eax
 ; X64-NEXT:    addl %ecx, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
 entry:
   %div = sdiv i16 %x, 10

llvm/test/CodeGen/X86/fdiv-combine.ll

; CHECK-NEXT:    retq
   %div2 = fdiv fast double %x, %y
   %ret = fadd fast double %div2, %div1
   ret double %ret
 }
 
 define float @div_select_constant_fold(i1 zeroext %arg) {
 ; CHECK-LABEL: div_select_constant_fold:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    testl %edi, %edi
+; CHECK-NEXT:    testb %dil, %dil
 ; CHECK-NEXT:    jne .LBB6_1
 ; CHECK-NEXT:  # %bb.2:
 ; CHECK-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
 ; CHECK-NEXT:    retq
 ; CHECK-NEXT:  .LBB6_1:
 ; CHECK-NEXT:    movss {{.*#+}} xmm0 = mem[0],zero,zero,zero
 ; CHECK-NEXT:    retq
   %tmp = select i1 %arg, float 5.000000e+00, float 6.000000e+00

llvm/test/CodeGen/X86/fp128-select.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -O2 -mtriple=x86_64-linux-android -mattr=+sse \
 ; RUN:     -enable-legalize-types-checking | FileCheck %s --check-prefix=SSE
 ; RUN: llc < %s -O2 -mtriple=x86_64-linux-gnu -mattr=+sse \
 ; RUN:     -enable-legalize-types-checking | FileCheck %s --check-prefix=SSE
 ; RUN: llc < %s -O2 -mtriple=x86_64-linux-android -mattr=-sse \
 ; RUN:     -enable-legalize-types-checking | FileCheck %s --check-prefix=NOSSE
 ; RUN: llc < %s -O2 -mtriple=x86_64-linux-gnu -mattr=-sse \
 ; RUN:     -enable-legalize-types-checking | FileCheck %s --check-prefix=NOSSE
 
 define void @test_select(fp128* %p, fp128* %q, i1 zeroext %c) {
 ; SSE-LABEL: test_select:
 ; SSE:       # %bb.0:
-; SSE-NEXT:    testl %edx, %edx
+; SSE-NEXT:    testb %dl, %dl
 ; SSE-NEXT:    jne .LBB0_1
 ; SSE-NEXT:  # %bb.2:
 ; SSE-NEXT:    movaps {{\.?LCPI[0-9]+_[0-9]+}}(%rip), %xmm0
 ; SSE-NEXT:    movaps %xmm0, (%rsi)
 ; SSE-NEXT:    retq
 ; SSE-NEXT:  .LBB0_1:
 ; SSE-NEXT:    movups (%rdi), %xmm0
 ; SSE-NEXT:    movaps %xmm0, (%rsi)
 ; SSE-NEXT:    retq
 ;
 ; NOSSE-LABEL: test_select:
 ; NOSSE:       # %bb.0:
 ; NOSSE-NEXT:    xorl %eax, %eax
-; NOSSE-NEXT:    testl %edx, %edx
+; NOSSE-NEXT:    testb %dl, %dl
 ; NOSSE-NEXT:    cmovneq (%rdi), %rax
 ; NOSSE-NEXT:    movabsq $9223231299366420480, %rcx # imm = 0x7FFF800000000000
 ; NOSSE-NEXT:    cmovneq 8(%rdi), %rcx
 ; NOSSE-NEXT:    movq %rcx, 8(%rsi)
 ; NOSSE-NEXT:    movq %rax, (%rsi)
 ; NOSSE-NEXT:    retq
   %a = load fp128, fp128* %p, align 2
   %r = select i1 %c, fp128 %a, fp128 0xL00000000000000007FFF800000000000

llvm/test/CodeGen/X86/illegal-bitfield-loadstore.ll

 ; X86-NEXT:    orl %ecx, %ebx
 ; X86-NEXT:    movw %bx, (%eax)
 ; X86-NEXT:    popl %ebx
 ; X86-NEXT:    .cfi_def_cfa_offset 4
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: i24_insert_bit:
 ; X64:       # %bb.0:
-; X64-NEXT:    movzwl (%rdi), %eax
-; X64-NEXT:    movzbl 2(%rdi), %ecx
-; X64-NEXT:    movb %cl, 2(%rdi)
-; X64-NEXT:    shll $16, %ecx
-; X64-NEXT:    orl %eax, %ecx
-; X64-NEXT:    shll $13, %esi
-; X64-NEXT:    andl $16769023, %ecx # imm = 0xFFDFFF
-; X64-NEXT:    orl %esi, %ecx
-; X64-NEXT:    movw %cx, (%rdi)
+; X64-NEXT:    movzbl %sil, %eax
+; X64-NEXT:    movzwl (%rdi), %ecx
+; X64-NEXT:    movzbl 2(%rdi), %edx
+; X64-NEXT:    movb %dl, 2(%rdi)
+; X64-NEXT:    shll $16, %edx
+; X64-NEXT:    orl %ecx, %edx
+; X64-NEXT:    shll $13, %eax
+; X64-NEXT:    andl $16769023, %edx # imm = 0xFFDFFF
+; X64-NEXT:    orl %eax, %edx
+; X64-NEXT:    movw %dx, (%rdi)
 ; X64-NEXT:    retq
   %extbit = zext i1 %bit to i24
   %b = load i24, i24* %a, align 1
   %extbit.shl = shl nuw nsw i24 %extbit, 13
   %c = and i24 %b, -8193
   %d = or i24 %c, %extbit.shl
   store i24 %d, i24* %a, align 1
   ret void

 ; X86-NEXT:    movl $-8193, %edx # imm = 0xDFFF
 ; X86-NEXT:    andl (%eax), %edx
 ; X86-NEXT:    orl %ecx, %edx
 ; X86-NEXT:    movl %edx, (%eax)
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: i56_insert_bit:
 ; X64:       # %bb.0:
-; X64-NEXT:    movl %esi, %eax
+; X64-NEXT:    movzbl %sil, %eax
 ; X64-NEXT:    movzwl 4(%rdi), %ecx
 ; X64-NEXT:    movzbl 6(%rdi), %edx
 ; X64-NEXT:    movb %dl, 6(%rdi)
 ; X64-NEXT:    # kill: def $edx killed $edx killed $rdx def $rdx
 ; X64-NEXT:    shll $16, %edx
 ; X64-NEXT:    orl %ecx, %edx
 ; X64-NEXT:    shlq $32, %rdx
 ; X64-NEXT:    movl (%rdi), %ecx

llvm/test/CodeGen/X86/known-signbits-vector.ll

 ; X86-NEXT:    vpinsrd $1, %eax, %xmm0, %xmm0
 ; X86-NEXT:    vpinsrd $2, {{[0-9]+}}(%esp), %xmm0, %xmm0
 ; X86-NEXT:    vpinsrd $3, {{[0-9]+}}(%esp), %xmm0, %xmm0
 ; X86-NEXT:    vcvtdq2ps %xmm0, %xmm0
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: signbits_sext_v4i64_sitofp_v4f32:
 ; X64:       # %bb.0:
-; X64-NEXT:    vmovd %edi, %xmm0
-; X64-NEXT:    vpinsrd $1, %esi, %xmm0, %xmm0
+; X64-NEXT:    movswl %si, %eax
+; X64-NEXT:    movsbl %dil, %esi
+; X64-NEXT:    vmovd %esi, %xmm0
+; X64-NEXT:    vpinsrd $1, %eax, %xmm0, %xmm0
 ; X64-NEXT:    vpinsrd $2, %edx, %xmm0, %xmm0
 ; X64-NEXT:    vpinsrd $3, %ecx, %xmm0, %xmm0
 ; X64-NEXT:    vcvtdq2ps %xmm0, %xmm0
 ; X64-NEXT:    retq
   %1 = sext i8 %a0 to i64
   %2 = sext i16 %a1 to i64
   %3 = sext i32 %a2 to i64
   %4 = sext i32 %a3 to i64

llvm/test/CodeGen/X86/mask-negated-bool.ll

; CHECK-NEXT:    retq
   %neg = sub i32 0, %ext
   %and = and i32 %neg, 1
   ret i32 %and
 }
 
 define i32 @mask_negated_zext_bool2(i1 zeroext %x) {
 ; CHECK-LABEL: mask_negated_zext_bool2:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    movzbl %dil, %eax
 ; CHECK-NEXT:    retq
   %ext = zext i1 %x to i32
   %neg = sub i32 0, %ext
   %and = and i32 %neg, 1
   ret i32 %and
 }
 
 define <4 x i32> @mask_negated_zext_bool_vec(<4 x i1> %x) {
; CHECK-NEXT:    retq
   %neg = sub i32 0, %ext
   %and = and i32 %neg, 1
   ret i32 %and
 }
 
 define i32 @mask_negated_sext_bool2(i1 zeroext %x) {
 ; CHECK-LABEL: mask_negated_sext_bool2:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    movzbl %dil, %eax
 ; CHECK-NEXT:    retq
   %ext = sext i1 %x to i32
   %neg = sub i32 0, %ext
   %and = and i32 %neg, 1
   ret i32 %and
 }
 
 define <4 x i32> @mask_negated_sext_bool_vec(<4 x i1> %x) {

llvm/test/CodeGen/X86/musttail-varargs.ll

 ; LINUX-X32-NEXT:    movaps %xmm1, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
 ; LINUX-X32-NEXT:    movaps %xmm0, {{[-0-9]+}}(%e{{[sb]}}p) # 16-byte Spill
 ; LINUX-X32-NEXT:    movb %al, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
 ; LINUX-X32-NEXT:    movq %r9, %r15
 ; LINUX-X32-NEXT:    movq %r8, %r12
 ; LINUX-X32-NEXT:    movq %rcx, %r13
 ; LINUX-X32-NEXT:    movq %rdx, %rbp
 ; LINUX-X32-NEXT:    movq %rsi, %rbx
-; LINUX-X32-NEXT:    movq %rdi, %r14
 ; LINUX-X32-NEXT:    movq %rsi, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    movq %rdx, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    movq %rcx, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    movq %r8, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    movq %r9, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    testb %al, %al
 ; LINUX-X32-NEXT:    je .LBB0_2
 ; LINUX-X32-NEXT:  # %bb.1:
 ; LINUX-X32-NEXT:    movaps %xmm0, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    movaps %xmm1, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    movaps %xmm2, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    movaps %xmm3, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    movaps %xmm4, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    movaps %xmm5, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    movaps %xmm6, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    movaps %xmm7, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:  .LBB0_2:
 ; LINUX-X32-NEXT:    leal {{[0-9]+}}(%rsp), %eax
 ; LINUX-X32-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    leal {{[0-9]+}}(%rsp), %eax
 ; LINUX-X32-NEXT:    movl %eax, {{[0-9]+}}(%esp)
 ; LINUX-X32-NEXT:    movabsq $206158430216, %rax # imm = 0x3000000008
 ; LINUX-X32-NEXT:    movq %rax, {{[0-9]+}}(%esp)
+; LINUX-X32-NEXT:    movl %edi, %r14d
+; LINUX-X32-NEXT:    movq %r14, %rdi
 ; LINUX-X32-NEXT:    callq get_f@PLT
 ; LINUX-X32-NEXT:    movl %eax, %r11d
 ; LINUX-X32-NEXT:    movq %r14, %rdi
 ; LINUX-X32-NEXT:    movq %rbx, %rsi
 ; LINUX-X32-NEXT:    movq %rbp, %rdx
 ; LINUX-X32-NEXT:    movq %r13, %rcx
 ; LINUX-X32-NEXT:    movq %r12, %r8
 ; LINUX-X32-NEXT:    movb {{[-0-9]+}}(%e{{[sb]}}p), %al # 1-byte Reload

 
 define void @g_thunk(i8* %fptr_i8, ...) {
 ; LINUX-LABEL: g_thunk:
 ; LINUX:       # %bb.0:
 ; LINUX-NEXT:    jmpq *%rdi # TAILCALL
 ;
 ; LINUX-X32-LABEL: g_thunk:
 ; LINUX-X32:       # %bb.0:
+; LINUX-X32-NEXT:    movl %edi, %edi
 ; LINUX-X32-NEXT:    jmpq *%rdi # TAILCALL
 ;
 ; WINDOWS-LABEL: g_thunk:
 ; WINDOWS:       # %bb.0:
 ; WINDOWS-NEXT:    rex64 jmpq *%rcx # TAILCALL
 ;
 ; X86-LABEL: g_thunk:
 ; X86:       # %bb.0:

llvm/test/CodeGen/X86/negate-i1.ll

 ; X32-NEXT:    retl
   %b = sext i1 %a to i16
   ret i16 %b
 }
 
 define i16 @select_i16_neg1_or_0_zeroext(i1 zeroext %a) {
 ; X64-LABEL: select_i16_neg1_or_0_zeroext:
 ; X64:       # %bb.0:
-; X64-NEXT:    movl %edi, %eax
+; X64-NEXT:    movzbl %dil, %eax
 ; X64-NEXT:    negl %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: select_i16_neg1_or_0_zeroext:
 ; X32:       # %bb.0:
 ; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    negl %eax

 ; X32-NEXT:    retl
   %b = sext i1 %a to i32
   ret i32 %b
 }
 
 define i32 @select_i32_neg1_or_0_zeroext(i1 zeroext %a) {
 ; X64-LABEL: select_i32_neg1_or_0_zeroext:
 ; X64:       # %bb.0:
-; X64-NEXT:    movl %edi, %eax
+; X64-NEXT:    movzbl %dil, %eax
 ; X64-NEXT:    negl %eax
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: select_i32_neg1_or_0_zeroext:
 ; X32:       # %bb.0:
 ; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    negl %eax
 ; X32-NEXT:    retl

 ; X32-NEXT:    retl
   %b = sext i1 %a to i64
   ret i64 %b
 }
 
 define i64 @select_i64_neg1_or_0_zeroext(i1 zeroext %a) {
 ; X64-LABEL: select_i64_neg1_or_0_zeroext:
 ; X64:       # %bb.0:
-; X64-NEXT:    movl %edi, %eax
+; X64-NEXT:    movzbl %dil, %eax
 ; X64-NEXT:    negq %rax
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: select_i64_neg1_or_0_zeroext:
 ; X32:       # %bb.0:
 ; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    negl %eax
 ; X32-NEXT:    movl %eax, %edx
 ; X32-NEXT:    retl
   %b = sext i1 %a to i64
   ret i64 %b
 }
 

llvm/test/CodeGen/X86/or-with-overflow.ll

 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:  .LBB1_2:
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: or_i8_rr:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %esi, %eax
 ; X64-NEXT:    orl %edi, %eax
+; X64-NEXT:    testb %al, %al
 ; X64-NEXT:    cmovel %edi, %eax
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %3 = or i8 %1, %0
   %4 = icmp eq i8 %3, 0
   %5 = select i1 %4, i8 %0, i8 %3
   ret i8 %5
 }

 ; X86-NEXT:  .LBB3_2:
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: or_i16_rr:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %esi, %eax
 ; X64-NEXT:    orl %edi, %eax
+; X64-NEXT:    testw %ax, %ax
 ; X64-NEXT:    cmovel %edi, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %3 = or i16 %1, %0
   %4 = icmp eq i16 %3, 0
   %5 = select i1 %4, i16 %0, i16 %3
   ret i16 %5
 }

llvm/test/CodeGen/X86/popcnt.ll

 ; X86-NEXT:    addl %eax, %ecx
 ; X86-NEXT:    andl $252645135, %ecx # imm = 0xF0F0F0F
 ; X86-NEXT:    imull $16843009, %ecx, %eax # imm = 0x1010101
 ; X86-NEXT:    shrl $24, %eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: popcount_zext_i32:
 ; X64:       # %bb.0:
-; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    shrl %eax
-; X64-NEXT:    andl $21845, %eax # imm = 0x5555
-; X64-NEXT:    subl %eax, %edi
-; X64-NEXT:    movl %edi, %eax
+; X64-NEXT:    movzwl %di, %eax
+; X64-NEXT:    movl %eax, %ecx
+; X64-NEXT:    shrl %ecx
+; X64-NEXT:    andl $21845, %ecx # imm = 0x5555
+; X64-NEXT:    subl %ecx, %eax
+; X64-NEXT:    movl %eax, %ecx
+; X64-NEXT:    andl $858993459, %ecx # imm = 0x33333333
+; X64-NEXT:    shrl $2, %eax
 ; X64-NEXT:    andl $858993459, %eax # imm = 0x33333333
-; X64-NEXT:    shrl $2, %edi
-; X64-NEXT:    andl $858993459, %edi # imm = 0x33333333
-; X64-NEXT:    addl %eax, %edi
-; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    shrl $4, %eax
-; X64-NEXT:    addl %edi, %eax
-; X64-NEXT:    andl $252645135, %eax # imm = 0xF0F0F0F
-; X64-NEXT:    imull $16843009, %eax, %eax # imm = 0x1010101
+; X64-NEXT:    addl %ecx, %eax
+; X64-NEXT:    movl %eax, %ecx
+; X64-NEXT:    shrl $4, %ecx
+; X64-NEXT:    addl %eax, %ecx
+; X64-NEXT:    andl $252645135, %ecx # imm = 0xF0F0F0F
+; X64-NEXT:    imull $16843009, %ecx, %eax # imm = 0x1010101
 ; X64-NEXT:    shrl $24, %eax
 ; X64-NEXT:    retq
 ;
 ; X86-POPCNT-LABEL: popcount_zext_i32:
 ; X86-POPCNT:       # %bb.0:
 ; X86-POPCNT-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
 ; X86-POPCNT-NEXT:    popcntl %eax, %eax
 ; X86-POPCNT-NEXT:    retl
 ;
 ; X64-POPCNT-LABEL: popcount_zext_i32:
 ; X64-POPCNT:       # %bb.0:
-; X64-POPCNT-NEXT:    popcntl %edi, %eax
+; X64-POPCNT-NEXT:    movzwl %di, %eax
+; X64-POPCNT-NEXT:    popcntl %eax, %eax
 ; X64-POPCNT-NEXT:    retq
   %z = zext i16 %x to i32
   %cnt = tail call i32 @llvm.ctpop.i32(i32 %z)
   ret i32 %cnt
 }
 
 define i32 @popcount_i16_zext(i16 zeroext %x) {
 ; X86-LABEL: popcount_i16_zext:

 ; X86-POPCNT-LABEL: popcount_i16_zext:
 ; X86-POPCNT:       # %bb.0:
 ; X86-POPCNT-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
 ; X86-POPCNT-NEXT:    popcntl %eax, %eax
 ; X86-POPCNT-NEXT:    retl
 ;
 ; X64-POPCNT-LABEL: popcount_i16_zext:
 ; X64-POPCNT:       # %bb.0:
-; X64-POPCNT-NEXT:    popcntl %edi, %eax
+; X64-POPCNT-NEXT:    movzwl %di, %eax
+; X64-POPCNT-NEXT:    popcntl %eax, %eax
 ; X64-POPCNT-NEXT:    retq
   %cnt = tail call i16 @llvm.ctpop.i16(i16 %x)
   %z = zext i16 %cnt to i32
   ret i32 %z
 }
 
 declare i8 @llvm.ctpop.i8(i8) nounwind readnone
 declare i16 @llvm.ctpop.i16(i16) nounwind readnone

llvm/test/CodeGen/X86/pr49028.ll

 ; X86-NEXT:    sete (%ecx)
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: PR49028:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
 ; X64-NEXT:    shrl %eax
+; X64-NEXT:    andl $32767, %eax # imm = 0x7FFF
 ; X64-NEXT:    sete (%rsi)
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %3 = lshr i16 %0, 1
   %4 = icmp eq i16 %3, 0
   %5 = zext i1 %4 to i8
   store i8 %5, i8* %1, align 1
   ret i16 %3
 }

llvm/test/CodeGen/X86/select-ext.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+sse4.1 | FileCheck %s
 
 ; (zext(select c, load1, load2)) -> (select c, zextload1, zextload2)
 define i64 @zext_scalar(i8* %p, i1 zeroext %c) {
 ; CHECK-LABEL: zext_scalar:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movzbl (%rdi), %ecx
 ; CHECK-NEXT:    movzbl 1(%rdi), %eax
-; CHECK-NEXT:    testl %esi, %esi
+; CHECK-NEXT:    testb %sil, %sil
 ; CHECK-NEXT:    cmoveq %rcx, %rax
 ; CHECK-NEXT:    retq
   %ld1 = load volatile i8, i8* %p
   %arrayidx1 = getelementptr inbounds i8, i8* %p, i64 1
   %ld2 = load volatile i8, i8* %arrayidx1
   %cond.v = select i1 %c, i8 %ld2, i8 %ld1
   %cond = zext i8 %cond.v to i64
   ret i64 %cond
 }
 
 define i64 @zext_scalar2(i8* %p, i16* %q, i1 zeroext %c) {
 ; CHECK-LABEL: zext_scalar2:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movzbl (%rdi), %ecx
 ; CHECK-NEXT:    movzwl (%rsi), %eax
-; CHECK-NEXT:    testl %edx, %edx
+; CHECK-NEXT:    testb %dl, %dl
 ; CHECK-NEXT:    cmoveq %rcx, %rax
 ; CHECK-NEXT:    retq
   %ld1 = load volatile i8, i8* %p
   %ext_ld1 = zext i8 %ld1 to i16
   %ld2 = load volatile i16, i16* %q
   %cond.v = select i1 %c, i16 %ld2, i16 %ext_ld1
   %cond = zext i16 %cond.v to i64
   ret i64 %cond
 }
 
 ; Don't fold the ext if there is a load with conflicting ext type.
 define i64 @zext_scalar_neg(i8* %p, i16* %q, i1 zeroext %c) {
 ; CHECK-LABEL: zext_scalar_neg:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movsbl (%rdi), %eax
-; CHECK-NEXT:    testl %edx, %edx
+; CHECK-NEXT:    testb %dl, %dl
 ; CHECK-NEXT:    je .LBB2_2
 ; CHECK-NEXT:  # %bb.1:
 ; CHECK-NEXT:    movzwl (%rsi), %eax
 ; CHECK-NEXT:  .LBB2_2:
 ; CHECK-NEXT:    movzwl %ax, %eax
 ; CHECK-NEXT:    retq
   %ld1 = load volatile i8, i8* %p
   %ext_ld1 = sext i8 %ld1 to i16
   %ld2 = load volatile i16, i16* %q
   %cond.v = select i1 %c, i16 %ld2, i16 %ext_ld1
   %cond = zext i16 %cond.v to i64
   ret i64 %cond
 }
 
 ; (sext(select c, load1, load2)) -> (select c, sextload1, sextload2)
 define i64 @sext_scalar(i8* %p, i1 zeroext %c) {
 ; CHECK-LABEL: sext_scalar:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movsbq (%rdi), %rcx
 ; CHECK-NEXT:    movsbq 1(%rdi), %rax
-; CHECK-NEXT:    testl %esi, %esi
+; CHECK-NEXT:    testb %sil, %sil
 ; CHECK-NEXT:    cmoveq %rcx, %rax
 ; CHECK-NEXT:    retq
   %ld1 = load volatile i8, i8* %p
   %arrayidx1 = getelementptr inbounds i8, i8* %p, i64 1
   %ld2 = load volatile i8, i8* %arrayidx1
   %cond.v = select i1 %c, i8 %ld2, i8 %ld1
   %cond = sext i8 %cond.v to i64
   ret i64 %cond
 }
 
 ; Same as zext_scalar, but operate on vectors.
 define <2 x i64> @zext_vector_i1(<2 x i32>* %p, i1 zeroext %c) {
 ; CHECK-LABEL: zext_vector_i1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pmovzxdq {{.*#+}} xmm1 = mem[0],zero,mem[1],zero
 ; CHECK-NEXT:    pmovzxdq {{.*#+}} xmm0 = mem[0],zero,mem[1],zero
-; CHECK-NEXT:    testl %esi, %esi
+; CHECK-NEXT:    testb %sil, %sil
 ; CHECK-NEXT:    jne .LBB4_2
 ; CHECK-NEXT:  # %bb.1:
 ; CHECK-NEXT:    movdqa %xmm1, %xmm0
 ; CHECK-NEXT:  .LBB4_2:
 ; CHECK-NEXT:    retq
   %ld1 = load volatile <2 x i32>, <2 x i32>* %p
   %arrayidx1 = getelementptr inbounds <2 x i32>, <2 x i32>* %p, i64 1
   %ld2 = load volatile <2 x i32>, <2 x i32>* %arrayidx1

 }
 
 ; Same as sext_scalar, but operate on vectors.
 define <2 x i64> @sext_vector_i1(<2 x i32>* %p, i1 zeroext %c) {
 ; CHECK-LABEL: sext_vector_i1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    pmovsxdq (%rdi), %xmm1
 ; CHECK-NEXT:    pmovsxdq 8(%rdi), %xmm0
-; CHECK-NEXT:    testl %esi, %esi
+; CHECK-NEXT:    testb %sil, %sil
 ; CHECK-NEXT:    jne .LBB6_2
 ; CHECK-NEXT:  # %bb.1:
 ; CHECK-NEXT:    movdqa %xmm1, %xmm0
 ; CHECK-NEXT:  .LBB6_2:
 ; CHECK-NEXT:    retq
   %ld1 = load volatile <2 x i32>, <2 x i32>* %p
   %arrayidx1 = getelementptr inbounds <2 x i32>, <2 x i32>* %p, i64 1
   %ld2 = load volatile <2 x i32>, <2 x i32>* %arrayidx1

llvm/test/CodeGen/X86/select_const.ll

 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 1, i32 0
   ret i32 %sel
 }
 
 define i32 @select_1_or_0_zeroext(i1 zeroext %cond) {
 ; CHECK-LABEL: select_1_or_0_zeroext:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    movzbl %dil, %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 1, i32 0
   ret i32 %sel
 }
 
 define i32 @select_1_or_0_signext(i1 signext %cond) {
 ; CHECK-LABEL: select_1_or_0_signext:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    movzbl %dil, %eax
 ; CHECK-NEXT:    andl $1, %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 1, i32 0
   ret i32 %sel
 }
 
 ; select Cond, 0, -1 --> sext (!Cond)
 
 define i32 @select_0_or_neg1(i1 %cond) {
 ; CHECK-LABEL: select_0_or_neg1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
 ; CHECK-NEXT:    andl $1, %edi
 ; CHECK-NEXT:    leal -1(%rdi), %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 0, i32 -1
   ret i32 %sel
 }
 
 define i32 @select_0_or_neg1_zeroext(i1 zeroext %cond) {
 ; CHECK-LABEL: select_0_or_neg1_zeroext:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
-; CHECK-NEXT:    leal -1(%rdi), %eax
+; CHECK-NEXT:    movzbl %dil, %eax
+; CHECK-NEXT:    decl %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 0, i32 -1
   ret i32 %sel
 }
 
 define i32 @select_0_or_neg1_signext(i1 signext %cond) {
 ; CHECK-LABEL: select_0_or_neg1_signext:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl %edi, %eax
-; CHECK-NEXT:    notl %eax
+; CHECK-NEXT:    movzbl %dil, %eax
+; CHECK-NEXT:    andl $1, %eax
+; CHECK-NEXT:    decl %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 0, i32 -1
   ret i32 %sel
 }
 
 ; select Cond, -1, 0 --> sext (Cond)
 
 define i32 @select_neg1_or_0(i1 %cond) {
 ; CHECK-LABEL: select_neg1_or_0:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    movl %edi, %eax
 ; CHECK-NEXT:    andl $1, %eax
 ; CHECK-NEXT:    negl %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 -1, i32 0
   ret i32 %sel
 }
 
 define i32 @select_neg1_or_0_zeroext(i1 zeroext %cond) {
 ; CHECK-LABEL: select_neg1_or_0_zeroext:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    movzbl %dil, %eax
 ; CHECK-NEXT:    negl %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 -1, i32 0
   ret i32 %sel
 }
 
 define i32 @select_neg1_or_0_signext(i1 signext %cond) {
 ; CHECK-LABEL: select_neg1_or_0_signext:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    movsbl %dil, %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 -1, i32 0
   ret i32 %sel
 }
 
 ; select Cond, C+1, C --> add (zext Cond), C
 
 define i32 @select_Cplus1_C(i1 %cond) {
 ; CHECK-LABEL: select_Cplus1_C:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
 ; CHECK-NEXT:    andl $1, %edi
 ; CHECK-NEXT:    leal 41(%rdi), %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 42, i32 41
   ret i32 %sel
 }
 
 define i32 @select_Cplus1_C_zeroext(i1 zeroext %cond) {
 ; CHECK-LABEL: select_Cplus1_C_zeroext:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
-; CHECK-NEXT:    leal 41(%rdi), %eax
+; CHECK-NEXT:    movzbl %dil, %eax
+; CHECK-NEXT:    addl $41, %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 42, i32 41
   ret i32 %sel
 }
 
 define i32 @select_Cplus1_C_signext(i1 signext %cond) {
 ; CHECK-LABEL: select_Cplus1_C_signext:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    movl $41, %eax
-; CHECK-NEXT:    subl %edi, %eax
+; CHECK-NEXT:    movzbl %dil, %eax
+; CHECK-NEXT:    andl $1, %eax
+; CHECK-NEXT:    addl $41, %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 42, i32 41
   ret i32 %sel
 }
 
 ; select Cond, C, C+1 --> add (sext Cond), C
 
 define i32 @select_C_Cplus1(i1 %cond) {
 ; CHECK-LABEL: select_C_Cplus1:
 ; CHECK:       # %bb.0:
 ; CHECK-NEXT:    andl $1, %edi
 ; CHECK-NEXT:    movl $42, %eax
 ; CHECK-NEXT:    subl %edi, %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 41, i32 42
   ret i32 %sel
 }
 
 define i32 @select_C_Cplus1_zeroext(i1 zeroext %cond) {
 ; CHECK-LABEL: select_C_Cplus1_zeroext:
 ; CHECK:       # %bb.0:
+; CHECK-NEXT:    movzbl %dil, %ecx
 ; CHECK-NEXT:    movl $42, %eax
-; CHECK-NEXT:    subl %edi, %eax
+; CHECK-NEXT:    subl %ecx, %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 41, i32 42
   ret i32 %sel
 }
 
 define i32 @select_C_Cplus1_signext(i1 signext %cond) {
 ; CHECK-LABEL: select_C_Cplus1_signext:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
-; CHECK-NEXT:    leal 42(%rdi), %eax
+; CHECK-NEXT:    movzbl %dil, %ecx
+; CHECK-NEXT:    andl $1, %ecx
+; CHECK-NEXT:    movl $42, %eax
+; CHECK-NEXT:    subl %ecx, %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 41, i32 42
   ret i32 %sel
 }
 
 ; If the constants differ by a small multiplier, use LEA.
 ; select Cond, C1, C2 --> add (mul (zext Cond), C1-C2), C2 --> LEA C2(Cond * (C1-C2))
 

 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i16 7, i16 71
   ret i16 %sel
 }
 
 define i32 @select_pow2_diff_neg(i1 zeroext %cond) {
 ; CHECK-LABEL: select_pow2_diff_neg:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    # kill: def $edi killed $edi def $rdi
-; CHECK-NEXT:    shll $4, %edi
-; CHECK-NEXT:    leal -25(%rdi), %eax
+; CHECK-NEXT:    movzbl %dil, %eax
+; CHECK-NEXT:    shll $4, %eax
+; CHECK-NEXT:    orl $-25, %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 -9, i32 -25
   ret i32 %sel
 }
 
 define i64 @select_pow2_diff_neg_invert(i1 zeroext %cond) {
 ; CHECK-LABEL: select_pow2_diff_neg_invert:
 ; CHECK:       # %bb.0:

 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 421, i32 42
   ret i32 %sel
 }
 
 define i32 @select_C1_C2_zeroext(i1 zeroext %cond) {
 ; CHECK-LABEL: select_C1_C2_zeroext:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    testl %edi, %edi
+; CHECK-NEXT:    testb %dil, %dil
 ; CHECK-NEXT:    movl $421, %ecx # imm = 0x1A5
 ; CHECK-NEXT:    movl $42, %eax
 ; CHECK-NEXT:    cmovnel %ecx, %eax
 ; CHECK-NEXT:    retq
   %sel = select i1 %cond, i32 421, i32 42
   ret i32 %sel
 }
 

llvm/test/CodeGen/X86/setcc-logic.ll

 ; BMI-NEXT:    setne %al
 ; BMI-NEXT:    retq
   %o = or i32 %x, %y
   %c = icmp ne i32 %o, %y
   ret i1 %c
 }
 
 define i1 @or_cmp_eq_i16(i16 zeroext %x, i16 zeroext %y) {
-; NOBMI-LABEL: or_cmp_eq_i16:
-; NOBMI:       # %bb.0:
-; NOBMI-NEXT:    notl %edi
-; NOBMI-NEXT:    testl %esi, %edi
-; NOBMI-NEXT:    sete %al
-; NOBMI-NEXT:    retq
-;
-; BMI-LABEL: or_cmp_eq_i16:
-; BMI:       # %bb.0:
-; BMI-NEXT:    andnl %esi, %edi, %eax
-; BMI-NEXT:    sete %al
-; BMI-NEXT:    retq
+; CHECK-LABEL: or_cmp_eq_i16:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:    notl %edi
+; CHECK-NEXT:    testw %si, %di
+; CHECK-NEXT:    sete %al
+; CHECK-NEXT:    retq
   %o = or i16 %x, %y
   %c = icmp eq i16 %x, %o
   ret i1 %c
 }
 
 define i1 @or_cmp_ne_i8(i8 zeroext %x, i8 zeroext %y) {
 ; CHECK-LABEL: or_cmp_ne_i8:
 ; CHECK:       # %bb.0:

llvm/test/CodeGen/X86/sext-i1.ll

 ; X32-LABEL: select_0_or_1s_zeroext:
 ; X32:       # %bb.0:
 ; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    decl %eax
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: select_0_or_1s_zeroext:
 ; X64:       # %bb.0:
-; X64-NEXT:    # kill: def $edi killed $edi def $rdi
-; X64-NEXT:    leal -1(%rdi), %eax
+; X64-NEXT:    movzbl %dil, %eax
+; X64-NEXT:    decl %eax
 ; X64-NEXT:    retq
   %not = xor i1 %cond, 1
   %sext = sext i1 %not to i32
   ret i32 %sext
 }
 
 ; sext (xor Bool, -1) --> sub (zext Bool), 1
 
 define i32 @select_0_or_1s_signext(i1 signext %cond) {
 ; X32-LABEL: select_0_or_1s_signext:
 ; X32:       # %bb.0:
 ; X32-NEXT:    movzbl {{[0-9]+}}(%esp), %eax
 ; X32-NEXT:    andl $1, %eax
 ; X32-NEXT:    decl %eax
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: select_0_or_1s_signext:
 ; X64:       # %bb.0:
-; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    notl %eax
+; X64-NEXT:    movzbl %dil, %eax
+; X64-NEXT:    andl $1, %eax
+; X64-NEXT:    decl %eax
 ; X64-NEXT:    retq
   %not = xor i1 %cond, 1
   %sext = sext i1 %not to i32
   ret i32 %sext
 }
 
 define i32 @zext_decrement_sext(i8 %x) {
 ; X32-LABEL: zext_decrement_sext:

llvm/test/CodeGen/X86/sibcall.ll

 ; X64-LABEL: t4:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    xorl %edi, %edi
 ; X64-NEXT:    jmpq *%rax # TAILCALL
 ;
 ; X32-LABEL: t4:
 ; X32:       # %bb.0:
-; X32-NEXT:    movq %rdi, %rax
+; X32-NEXT:    movl %edi, %eax
 ; X32-NEXT:    xorl %edi, %edi
 ; X32-NEXT:    jmpq *%rax # TAILCALL
   tail call void %x(i32 0) nounwind
   ret void
 }
 
 define dso_local void @t5(void ()* nocapture %x) nounwind ssp {
 ; X86-LABEL: t5:
 ; X86:       # %bb.0:
 ; X86-NEXT:    jmpl *{{[0-9]+}}(%esp) # TAILCALL
 ;
 ; X64-LABEL: t5:
 ; X64:       # %bb.0:
 ; X64-NEXT:    jmpq *%rdi # TAILCALL
 ;
 ; X32-LABEL: t5:
 ; X32:       # %bb.0:
-; X32-NEXT:    jmpq *%rdi # TAILCALL
+; X32-NEXT:    movl %edi, %eax
+; X32-NEXT:    jmpq *%rax # TAILCALL
   tail call void %x() nounwind
   ret void
 }
 
 ; Basically the same test as t5, except pass the function pointer on the stack
 ; for x86_64.
 
 define dso_local void @t5_x64(i32, i32, i32, i32, i32, i32, void ()* nocapture %x) nounwind ssp {

 ; X64-LABEL: t9:
 ; X64:       # %bb.0: # %entry
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    xorl %edi, %edi
 ; X64-NEXT:    jmpq *%rax # TAILCALL
 ;
 ; X32-LABEL: t9:
 ; X32:       # %bb.0: # %entry
-; X32-NEXT:    movq %rdi, %rax
+; X32-NEXT:    movl %edi, %eax
 ; X32-NEXT:    xorl %edi, %edi
 ; X32-NEXT:    jmpq *%rax # TAILCALL
 entry:
   %0 = bitcast i32 (i32)* %x to i16 (i32)*
   %1 = tail call signext i16 %0(i32 0) nounwind
   ret i16 %1
 }
 

 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: t15:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushq %rbx
 ; X32-NEXT:    movq %rdi, %rbx
+; X32-NEXT:    movl %ebx, %edi
 ; X32-NEXT:    callq f
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    popq %rbx
 ; X32-NEXT:    retq
   tail call fastcc void @f(%struct.foo* noalias sret(%struct.foo) %agg.result) nounwind
   ret void
 }
 

 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: t21_sret_to_sret:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushq %rbx
 ; X32-NEXT:    movq %rdi, %rbx
+; X32-NEXT:    movl %ebx, %edi
 ; X32-NEXT:    callq t21_f_sret
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    popq %rbx
 ; X32-NEXT:    retq
   tail call fastcc void @t21_f_sret(%struct.foo* noalias sret(%struct.foo) %agg.result) nounwind
   ret void
 }
 

 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: t21_sret_to_sret_more_args:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushq %rbx
 ; X32-NEXT:    movq %rdi, %rbx
+; X32-NEXT:    movl %ebx, %edi
 ; X32-NEXT:    callq f_sret@PLT
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    popq %rbx
 ; X32-NEXT:    retq
   tail call fastcc void @f_sret(%struct.foo* noalias sret(%struct.foo) %agg.result, i32 %a, i32 %b) nounwind
   ret void
 }
 

 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: t21_sret_to_sret_second_arg_sret:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushq %rbx
 ; X32-NEXT:    movq %rsi, %rbx
-; X32-NEXT:    movq %rsi, %rdi
+; X32-NEXT:    movl %ebx, %edi
 ; X32-NEXT:    callq t21_f_sret
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    popq %rbx
 ; X32-NEXT:    retq
   tail call fastcc void @t21_f_sret(%struct.foo* noalias sret(%struct.foo) %ret) nounwind
   ret void
 }
 

 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: t21_sret_to_sret_more_args2:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushq %rbx
 ; X32-NEXT:    movl %esi, %eax
 ; X32-NEXT:    movq %rdi, %rbx
+; X32-NEXT:    movl %ebx, %edi
 ; X32-NEXT:    movl %edx, %esi
 ; X32-NEXT:    movl %eax, %edx
 ; X32-NEXT:    callq f_sret@PLT
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    popq %rbx
 ; X32-NEXT:    retq
   tail call fastcc void @f_sret(%struct.foo* noalias sret(%struct.foo) %agg.result, i32 %b, i32 %a) nounwind
   ret void

 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: t21_sret_to_sret_args_mismatch:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushq %rbx
 ; X32-NEXT:    movq %rdi, %rbx
-; X32-NEXT:    movq %rsi, %rdi
+; X32-NEXT:    movl %esi, %edi
 ; X32-NEXT:    callq t21_f_sret
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    popq %rbx
 ; X32-NEXT:    retq
   tail call fastcc void @t21_f_sret(%struct.foo* noalias sret(%struct.foo) %ret) nounwind
   ret void
 }
 

 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: t21_sret_to_sret_args_mismatch2:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushq %rbx
 ; X32-NEXT:    movq %rdi, %rbx
-; X32-NEXT:    movq %rsi, %rdi
+; X32-NEXT:    movl %esi, %edi
 ; X32-NEXT:    callq t21_f_sret
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    popq %rbx
 ; X32-NEXT:    retq
   tail call fastcc void @t21_f_sret(%struct.foo* noalias sret(%struct.foo) %ret) nounwind
   ret void
 }
 

 ; X32-LABEL: t21_sret_to_sret_structs_mismatch:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushq %r14
 ; X32-NEXT:    pushq %rbx
 ; X32-NEXT:    pushq %rax
 ; X32-NEXT:    movq %rsi, %rbx
 ; X32-NEXT:    movq %rdi, %r14
 ; X32-NEXT:    callq ret_struct@PLT
+; X32-NEXT:    movl %ebx, %edi
 ; X32-NEXT:    movl %eax, %esi
-; X32-NEXT:    movq %rbx, %rdi
 ; X32-NEXT:    callq t21_f_sret2
 ; X32-NEXT:    movl %r14d, %eax
 ; X32-NEXT:    addl $8, %esp
 ; X32-NEXT:    popq %rbx
 ; X32-NEXT:    popq %r14
 ; X32-NEXT:    retq
   %b = call fastcc %struct.foo* @ret_struct()
   tail call fastcc void @t21_f_sret2(%struct.foo* noalias sret(%struct.foo) %a, %struct.foo* noalias %b) nounwind

 ; X64-NEXT:    movq %rbx, %rax
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: t21_sret_to_non_sret:
 ; X32:       # %bb.0:
 ; X32-NEXT:    pushq %rbx
 ; X32-NEXT:    movq %rdi, %rbx
+; X32-NEXT:    movl %ebx, %edi
 ; X32-NEXT:    callq t21_f_non_sret
 ; X32-NEXT:    movl %ebx, %eax
 ; X32-NEXT:    popq %rbx
 ; X32-NEXT:    retq
   tail call fastcc void @t21_f_non_sret(%struct.foo* %agg.result) nounwind
   ret void
 }
 
 define ccc void @t22_non_sret_to_sret(%struct.foo* %agg.result) nounwind  {
 ; i686 not tailcallable, as sret is callee-pop here.
 ; X86-LABEL: t22_non_sret_to_sret:
 ; X86:       # %bb.0:
 ; X86-NEXT:    subl $12, %esp
 ; X86-NEXT:    movl {{[0-9]+}}(%esp), %eax
 ; X86-NEXT:    movl %eax, (%esp)
 ; X86-NEXT:    calll t22_f_sret@PLT
 ; X86-NEXT:    addl $8, %esp
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: t22_non_sret_to_sret:
 ; X64:       # %bb.0:
-; X64-NEXT:    jmp  t22_f_sret@PLT                  # TAILCALL
+; X64-NEXT:    jmp t22_f_sret@PLT # TAILCALL
 ;
 ; X32-LABEL: t22_non_sret_to_sret:
 ; X32:       # %bb.0:
-; X32-NEXT:    jmp  t22_f_sret@PLT                  # TAILCALL
+; X32-NEXT:    movl %edi, %edi
+; X32-NEXT:    jmp t22_f_sret@PLT # TAILCALL
   tail call ccc void @t22_f_sret(%struct.foo* noalias sret(%struct.foo) %agg.result) nounwind
   ret void
 }
 
 declare dso_local fastcc void @t21_f_sret(%struct.foo* noalias sret(%struct.foo)) nounwind
 declare dso_local fastcc void @t21_f_sret2(%struct.foo* noalias sret(%struct.foo), %struct.foo* noalias) nounwind
 declare dso_local fastcc void @t21_f_non_sret(%struct.foo*) nounwind
 
 declare ccc void @t22_f_sret(%struct.foo* noalias sret(%struct.foo)) nounwind
 
 declare ccc void @f_sret(%struct.foo* noalias sret(%struct.foo), i32, i32) nounwind

llvm/test/CodeGen/X86/split-store.ll

; CHECK-NEXT:    retq
   ret void
 }
 
 ; getTypeSizeInBits(i14) != getTypeStoreSizeInBits(i14), so store split doesn't kick in.
 
 define void @int7_int7_pair(i7 signext %tmp1, i7 signext %tmp2, i14* %ref.tmp) {
 ; CHECK-LABEL: int7_int7_pair:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    shll $7, %esi
-; CHECK-NEXT:    andl $127, %edi
-; CHECK-NEXT:    orl %esi, %edi
-; CHECK-NEXT:    andl $16383, %edi # imm = 0x3FFF
-; CHECK-NEXT:    movw %di, (%rdx)
+; CHECK-NEXT:    movzbl %sil, %eax
+; CHECK-NEXT:    shll $7, %eax
+; CHECK-NEXT:    movzbl %dil, %ecx
+; CHECK-NEXT:    andl $127, %ecx
+; CHECK-NEXT:    orl %eax, %ecx
+; CHECK-NEXT:    andl $16383, %ecx # imm = 0x3FFF
+; CHECK-NEXT:    movw %cx, (%rdx)
 ; CHECK-NEXT:    retq
   %t1 = zext i7 %tmp2 to i14
   %t2 = shl nuw i14 %t1, 7
   %t3 = zext i7 %tmp1 to i14
   %t4 = or i14 %t2, %t3
   store i14 %t4, i14* %ref.tmp, align 2
   ret void
 }

llvm/test/CodeGen/X86/x86-64-arg.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s | FileCheck %s
 
 ; The input value is already sign extended, don't re-extend it.
 ; This testcase corresponds to:
 ;   int test(short X) { return (int)X; }
 
 target datalayout = "e-p:64:64"
 target triple = "x86_64-apple-darwin8"
 
 
 define i32 @test(i16 signext  %X) {
 ; CHECK-LABEL: test:
 ; CHECK:       ## %bb.0: ## %entry
-; CHECK-NEXT:    movl %edi, %eax
+; CHECK-NEXT:    movswl %di, %eax
 ; CHECK-NEXT:    retq
 entry:
         %tmp12 = sext i16 %X to i32             ; <i32> [#uses=1]
         ret i32 %tmp12
 }
 

llvm/test/CodeGen/X86/xor-icmp.ll

 ; X86-NEXT:    movb {{[0-9]+}}(%esp), %al
 ; X86-NEXT:    xorb {{[0-9]+}}(%esp), %al
 ; X86-NEXT:    xorb $1, %al
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: xor_not_bools:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %edi, %eax
-; X64-NEXT:    xorl %esi, %eax
+; X64-NEXT:    xorb %sil, %al
 ; X64-NEXT:    xorb $1, %al
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %xor = xor i1 %x, %y
   %not = xor i1 %xor, true
   ret i1 %not
 }
 

llvm/test/CodeGen/X86/xor-with-overflow.ll

 ; X86-NEXT:    movl %ecx, %eax
 ; X86-NEXT:  .LBB1_2:
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: xor_i8_rr:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %esi, %eax
 ; X64-NEXT:    xorl %edi, %eax
+; X64-NEXT:    testb %al, %al
 ; X64-NEXT:    cmovel %edi, %eax
 ; X64-NEXT:    # kill: def $al killed $al killed $eax
 ; X64-NEXT:    retq
   %3 = xor i8 %1, %0
   %4 = icmp eq i8 %3, 0
   %5 = select i1 %4, i8 %0, i8 %3
   ret i8 %5
 }

 ; X86-NEXT:  .LBB3_2:
 ; X86-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X86-NEXT:    retl
 ;
 ; X64-LABEL: xor_i16_rr:
 ; X64:       # %bb.0:
 ; X64-NEXT:    movl %esi, %eax
 ; X64-NEXT:    xorl %edi, %eax
+; X64-NEXT:    testw %ax, %ax
 ; X64-NEXT:    cmovel %edi, %eax
 ; X64-NEXT:    # kill: def $ax killed $ax killed $eax
 ; X64-NEXT:    retq
   %3 = xor i16 %1, %0
   %4 = icmp eq i16 %3, 0
   %5 = select i1 %4, i16 %0, i16 %3
   ret i16 %5
 }