[DAG Legalization] Zero-extend the value compared in cmpxchg

[nemanjai]

Summary

Current legalization scheme extends input operands to atomic compare and exchange without specifying whether it's a sign or zero extension. However, this leads to bugs at least in PPC codegen. Of course, this can be fixed in the PPC back end, but it seems like the right thing to do would be to change this in legalization since the underlying comparison is an equality comparison. Operands of equality comparisons (SETCC) are zero-extended as well.

Since this affects other back ends as well, I'll send out an RFC about this to make sure this is safe/desirable for other targets.

[efriedma]

lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

236

The value operand to a store is implicitly truncated. The same thing happens for ATOMIC_LOAD_*. The same thing happens for the third operand of ATOMIC_CMP_SWAP. I think it's more confusing than helpful to add a special case for the compare operand of a compare. Atomic compare-and-swap doesn't zero-extend on all targets, anyway; see getExtendForAtomicOps().

[efriedma]

(It looks like https://reviews.llvm.org/D41798 fixes your testcase?)

[nemanjai]

In D41791#969091, @efriedma wrote:

(It looks like https://reviews.llvm.org/D41798 fixes your testcase?)

Almost. I've added a detailed comment there regarding why it doesn't fix the whole problem.

lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

236

I agree that this seems a bit confusing - i.e. why is one operand zero-extended whereas the other operand is essentially "any-extended". However, I think that the "store" operand of the atomic compare and exchange should be extended according to getExtendForAtomicOps() whereas the "compare" operand should always be zero-extended.
The reason I think this is that the compare operand is essentially used for equality comparison similarly to SETEQ/SETNE for SETCC, whereas the other operand is used to load and store the value - so it should be handled the way a target desires to handle atomics.

[nemanjai]

Abandoning this as at least one target prefers that the compare input be any-extended and each target decide what to do about the high bits that may differ between the two values.

lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

236

I had planned to guard this zero-extension with TLI.getExtendForAtomicOps() == ISD::ZERO_EXTEND. However, since there are objections to doing this in legalization (see D41798), I'll abandon this and fix it in the PPC back end.

lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

SDValue Res = DAG.getAtomicCmpSwap(
         ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS, SDLoc(N), N->getMemoryVT(), VTs,
         N->getChain(), N->getBasePtr(), N->getOperand(2), N->getOperand(3),
         N->getMemOperand());
     ReplaceValueWith(SDValue(N, 0), Res.getValue(0));
     ReplaceValueWith(SDValue(N, 2), Res.getValue(2));
     return Res.getValue(1);
   }
 
-  SDValue Op2 = GetPromotedInteger(N->getOperand(2));
+  SDValue Op2 = ZExtPromotedInteger(N->getOperand(2));
   SDValue Op3 = GetPromotedInteger(N->getOperand(3));
   SDVTList VTs =
       DAG.getVTList(Op2.getValueType(), N->getValueType(1), MVT::Other);
   SDValue Res = DAG.getAtomicCmpSwap(
       N->getOpcode(), SDLoc(N), N->getMemoryVT(), VTs, N->getChain(),
       N->getBasePtr(), Op2, Op3, N->getMemOperand());
   // Update the use to N with the newly created Res.
   for (unsigned i = 1, NumResults = N->getNumValues(); i < NumResults; ++i)

test/CodeGen/ARM/atomic-cmpxchg.ll

 ; RUN: llc < %s -mtriple=arm-linux-gnueabi -asm-verbose=false -verify-machineinstrs | FileCheck %s -check-prefix=CHECK-ARM
 ; RUN: llc < %s -mtriple=thumb-linux-gnueabi -asm-verbose=false -verify-machineinstrs | FileCheck %s -check-prefix=CHECK-THUMB
 
 ; RUN: llc < %s -mtriple=armv6-linux-gnueabi -asm-verbose=false -verify-machineinstrs | FileCheck %s -check-prefix=CHECK-ARMV6
 ; RUN: llc < %s -mtriple=thumbv6-linux-gnueabi -asm-verbose=false -verify-machineinstrs | FileCheck %s -check-prefix=CHECK-THUMBV6
 
 ; RUN: llc < %s -mtriple=armv7-linux-gnueabi -asm-verbose=false -verify-machineinstrs | FileCheck %s -check-prefix=CHECK-ARMV7
 ; RUN: llc < %s -mtriple=thumbv7-linux-gnueabi -asm-verbose=false -verify-machineinstrs | FileCheck %s -check-prefix=CHECK-THUMBV7
 
-define zeroext i1 @test_cmpxchg_res_i8(i8* %addr, i8 %desired, i8 zeroext %new) {
+define zeroext i1 @test_cmpxchg_res_i8(i8* %addr, i8 zeroext %desired, i8 zeroext %new) {
 entry:
   %0 = cmpxchg i8* %addr, i8 %desired, i8 %new monotonic monotonic
   %1 = extractvalue { i8, i1 } %0, 1
   ret i1 %1
 }
 
 ; CHECK-ARM-LABEL: test_cmpxchg_res_i8
 ; CHECK-ARM: bl __sync_val_compare_and_swap_1

 ; CHECK-THUMB: movs r0, #1
 ; CHECK-THUMB: movs [[R2:r[0-9]+]], #0
 ; CHECK-THUMB: cmp [[R1]], {{r[0-9]+}}
 ; CHECK-THUMB: beq
 ; CHECK-THUMB: movs r0, [[R2]]
 
 ; CHECK-ARMV6-LABEL: test_cmpxchg_res_i8:
 ; CHECK-ARMV6-NEXT:  .fnstart
-; CHECK-ARMV6-NEXT: uxtb [[DESIRED:r[0-9]+]], r1
 ; CHECK-ARMV6-NEXT: [[TRY:.LBB[0-9_]+]]:
 ; CHECK-ARMV6-NEXT: ldrexb [[LD:r[0-9]+]], [r0]
-; CHECK-ARMV6-NEXT: cmp [[LD]], [[DESIRED]]
+; CHECK-ARMV6-NEXT: cmp [[LD]], r1
 ; CHECK-ARMV6-NEXT: movne [[RES:r[0-9]+]], #0
 ; CHECK-ARMV6-NEXT: bxne lr
 ; CHECK-ARMV6-NEXT: strexb [[SUCCESS:r[0-9]+]], r2, [r0]
 ; CHECK-ARMV6-NEXT: cmp [[SUCCESS]], #0
 ; CHECK-ARMV6-NEXT: moveq [[RES]], #1
 ; CHECK-ARMV6-NEXT: bxeq lr
 ; CHECK-ARMV6-NEXT: b [[TRY]]
 
 ; CHECK-THUMBV6-LABEL: test_cmpxchg_res_i8:
 ; CHECK-THUMBV6:       mov [[EXPECTED:r[0-9]+]], r1
 ; CHECK-THUMBV6-NEXT:  bl __sync_val_compare_and_swap_1
 ; CHECK-THUMBV6-NEXT:  mov [[RES:r[0-9]+]], r0
 ; CHECK-THUMBV6-NEXT:  movs r0, #1
 ; CHECK-THUMBV6-NEXT:  movs [[ZERO:r[0-9]+]], #0
 ; CHECK-THUMBV6-NEXT:  cmp [[RES]], [[EXPECTED]]
 ; CHECK-THUMBV6-NEXT:  beq [[END:.LBB[0-9_]+]]
 ; CHECK-THUMBV6-NEXT:  mov r0, [[ZERO]]
 ; CHECK-THUMBV6-NEXT: [[END]]:
 ; CHECK-THUMBV6-NEXT:  pop {{.*}}pc}
 
 ; CHECK-ARMV7-LABEL: test_cmpxchg_res_i8:
 ; CHECK-ARMV7-NEXT: .fnstart
-; CHECK-ARMV7-NEXT: uxtb [[DESIRED:r[0-9]+]], r1
 ; CHECK-ARMV7-NEXT: b [[TRY:.LBB[0-9_]+]]
 ; CHECK-ARMV7-NEXT: [[HEAD:.LBB[0-9_]+]]:
 ; CHECK-ARMV7-NEXT: strexb [[SUCCESS:r[0-9]+]], r2, [r0]
 ; CHECK-ARMV7-NEXT: cmp [[SUCCESS]], #0
 ; CHECK-ARMV7-NEXT: moveq r0, #1
 ; CHECK-ARMV7-NEXT: bxeq lr
 ; CHECK-ARMV7-NEXT: [[TRY]]:
 ; CHECK-ARMV7-NEXT: ldrexb [[SUCCESS]], [r0]
 ; CHECK-ARMV7-NEXT: cmp [[SUCCESS]], r1
 ; CHECK-ARMV7-NEXT: beq [[HEAD]]
 ; CHECK-ARMV7-NEXT: mov r0, #0
 ; CHECK-ARMV7-NEXT: clrex
 ; CHECK-ARMV7-NEXT: bx lr
 
 ; CHECK-THUMBV7-LABEL: test_cmpxchg_res_i8:
 ; CHECK-THUMBV7-NEXT: .fnstart
-; CHECK-THUMBV7-NEXT: uxtb [[DESIRED:r[0-9]+]], r1
 ; CHECK-THUMBV7-NEXT: b [[TRYLD:.LBB[0-9_]+]]
 ; CHECK-THUMBV7-NEXT: [[TRYST:.LBB[0-9_]+]]:
 ; CHECK-THUMBV7-NEXT: strexb [[SUCCESS:r[0-9]+]], r2, [r0]
 ; CHECK-THUMBV7-NEXT: cmp [[SUCCESS]], #0
 ; CHECK-THUMBV7-NEXT: itt eq
 ; CHECK-THUMBV7-NEXT: moveq r0, #1
 ; CHECK-THUMBV7-NEXT: bxeq lr
 ; CHECK-THUMBV7-NEXT: [[TRYLD]]:
 ; CHECK-THUMBV7-NEXT: ldrexb [[LD:r[0-9]+]], [r0]
-; CHECK-THUMBV7-NEXT: cmp [[LD]], [[DESIRED]]
+; CHECK-THUMBV7-NEXT: cmp [[LD]], r1
 ; CHECK-THUMBV7-NEXT: beq [[TRYST:.LBB[0-9_]+]]
 ; CHECK-THUMBV7-NEXT: movs r0, #0
 ; CHECK-THUMBV7-NEXT: clrex
 ; CHECK-THUMBV7-NEXT: bx lr

test/CodeGen/ARM/cmpxchg-O0.ll

 ; RUN: llc -verify-machineinstrs -mtriple=armv7-linux-gnu -O0 %s -o - | FileCheck %s
 ; RUN: llc -verify-machineinstrs -mtriple=thumbv8-linux-gnu -O0 %s -o - | FileCheck %s
 ; RUN: llc -verify-machineinstrs -mtriple=thumbv6m-none-eabi -O0 %s -o - | FileCheck %s --check-prefix=CHECK-T1
 
 ; CHECK-T1-NOT: ldrex
 ; CHECK-T1-NOT: strex
 
-define { i8, i1 } @test_cmpxchg_8(i8* %addr, i8 %desired, i8 %new) nounwind {
+define { i8, i1 } @test_cmpxchg_8(i8* %addr, i8 zeroext %desired, i8 %new) nounwind {
 ; CHECK-LABEL: test_cmpxchg_8:
 ; CHECK:     dmb ish
 ; CHECK:     uxtb [[DESIRED:r[0-9]+]], [[DESIRED]]
 ; CHECK: [[RETRY:.LBB[0-9]+_[0-9]+]]:
 ; CHECK:     ldrexb [[OLD:r[0-9]+]], [r0]
 ; CHECK:     cmp [[OLD]], [[DESIRED]]
 ; CHECK:     bne [[DONE:.LBB[0-9]+_[0-9]+]]
 ; CHECK:     strexb [[STATUS:r[0-9]+]], r2, [r0]
 ; CHECK:     cmp{{(\.w)?}} [[STATUS]], #0
 ; CHECK:     bne [[RETRY]]
 ; CHECK: [[DONE]]:
 ; CHECK:     cmp{{(\.w)?}} [[OLD]], [[DESIRED]]
 ; CHECK:     {{moveq|movweq}} {{r[0-9]+}}, #1
 ; CHECK:     dmb ish
   %res = cmpxchg i8* %addr, i8 %desired, i8 %new seq_cst monotonic
   ret { i8, i1 } %res
 }
 
-define { i16, i1 } @test_cmpxchg_16(i16* %addr, i16 %desired, i16 %new) nounwind {
+define { i16, i1 } @test_cmpxchg_16(i16* %addr, i16 zeroext %desired, i16 %new) nounwind {
 ; CHECK-LABEL: test_cmpxchg_16:
 ; CHECK:     dmb ish
 ; CHECK:     uxth [[DESIRED:r[0-9]+]], [[DESIRED]]
 ; CHECK: [[RETRY:.LBB[0-9]+_[0-9]+]]:
 ; CHECK:     ldrexh [[OLD:r[0-9]+]], [r0]
 ; CHECK:     cmp [[OLD]], [[DESIRED]]
 ; CHECK:     bne [[DONE:.LBB[0-9]+_[0-9]+]]
 ; CHECK:     strexh [[STATUS:r[0-9]+]], r2, [r0]

test/CodeGen/Mips/atomic.ll

 
 ; NO-SEB-SEH:    sll     $[[R17:[0-9]+]], $[[R16]], 24
 ; NO-SEB-SEH:    sra     $2, $[[R17]], 24
 
 ; HAS-SEB-SEH:   seb     $2, $[[R16]]
 
 }
 
-define signext i8 @AtomicCmpSwap8(i8 signext %oldval, i8 signext %newval) nounwind {
+define signext i8 @AtomicCmpSwap8(i8 zeroext %oldval, i8 signext %newval) nounwind {
 entry:
   %pair0 = cmpxchg i8* @y, i8 %oldval, i8 %newval monotonic monotonic
   %0 = extractvalue { i8, i1 } %pair0, 0
   ret i8 %0
 
 ; ALL-LABEL: AtomicCmpSwap8:
 
 ; MIPS32-ANY:    lw      $[[R0:[0-9]+]], %got(y)

 ; ALL:           srlv    $[[R17:[0-9]+]], $[[R14]], $[[R5]]
 
 ; NO-SEB-SEH:    sll     $[[R18:[0-9]+]], $[[R17]], 24
 ; NO-SEB-SEH:    sra     $2, $[[R18]], 24
 
 ; HAS-SEB-SEH:   seb     $2, $[[R17]]
 }
 
-define i1 @AtomicCmpSwapRes8(i8* %ptr, i8 signext %oldval, i8 signext %newval) nounwind {
+define i1 @AtomicCmpSwapRes8(i8* %ptr, i8 zeroext %oldval, i8 signext %newval) nounwind {
 entry:
   %0 = cmpxchg i8* %ptr, i8 %oldval, i8 %newval monotonic monotonic
   %1 = extractvalue { i8, i1 } %0, 1
   ret i1 %1
 ; ALL-LABEL: AtomicCmpSwapRes8
 
 ; ALL:           addiu   $[[R1:[0-9]+]], $zero, -4
 ; ALL:           and     $[[R2:[0-9]+]], $4, $[[R1]]

 ; NO-SEB-SEH:    sll     $[[R18:[0-9]+]], $[[R17]], 24
 ; NO-SEB-SEH:    sra     $[[R19:[0-9]+]], $[[R18]], 24
 
 ; FIXME: -march=mips produces a redundant sign extension here...
 ; NO-SEB-SEH:    sll     $[[R20:[0-9]+]], $5, 24
 ; NO-SEB-SEH:    sra     $[[R20]], $[[R20]], 24
 
 ; HAS-SEB-SEH:   seb     $[[R19:[0-9]+]], $[[R17]]
+; HAS-SEB-SEH:   seb     $[[R20:[0-9]+]], $5
 
 ; FIXME: ...Leading to this split check.
 ; NO-SEB-SEH:    xor     $[[R21:[0-9]+]], $[[R19]], $[[R20]]
-; HAS-SEB-SEH:   xor     $[[R21:[0-9]+]], $[[R19]], $5
+; HAS-SEB-SEH:   xor     $[[R21:[0-9]+]], $[[R19]], $[[R20]]
 
 ; ALL: sltiu   $2, $[[R21]], 1
 }
 
 ; Check one i16 so that we cover the seh sign extend
 @z = common global i16 0, align 1
 
 define signext i16 @AtomicLoadAdd16(i16 signext %incr) nounwind {
define {i16, i1} @foo(i16* %addr, i16 %l, i16 %r, i16 %new) {
   %res = cmpxchg i16* %addr, i16 %desired, i16 %new seq_cst seq_cst
   ret {i16, i1} %res
 
 ; ALL-LABEL: foo
 ; MIPSR6:        addu    $[[R2:[0-9]+]], $[[R1:[0-9]+]], $[[R0:[0-9]+]]
 ; NOT-MICROMIPS: addu    $[[R2:[0-9]+]], $[[R1:[0-9]+]], $[[R0:[0-9]+]]
 ; MICROMIPS:     addu16  $[[R2:[0-9]+]], $[[R1:[0-9]+]], $[[R0:[0-9]+]]
 
+; ALL:           andi{{[16 ]*}}  $[[R3:[0-9]+]], $[[R2]], 65535
 ; ALL:           sync
 
-; ALL:           andi    $[[R3:[0-9]+]], $[[R2]], 65535
 ; ALL:       [[BB0:(\$|\.L)[A-Z_0-9]+]]:
 ; ALL:           ll      $[[R4:[0-9]+]], 0($[[R5:[0-9]+]])
 ; ALL:           and     $[[R6:[0-9]+]], $[[R4]], $
 ; ALL:           and     $[[R7:[0-9]+]], $[[R4]], $
 ; ALL:           or      $[[R8:[0-9]+]], $[[R7]], $
 ; ALL:           sc      $[[R8]], 0($[[R5]])
 ; NOT-MICROMIPS: beqz    $[[R8]], [[BB0]]
 ; MICROMIPS:     beqzc   $[[R8]], [[BB0]]
 ; MIPSR6:        beqzc   $[[R8]], [[BB0]]
 
 ; ALL:           srlv    $[[R9:[0-9]+]], $[[R6]], $
 
 ; NO-SEB-SEH:    sll     $[[R10:[0-9]+]], $[[R9]], 16
 ; NO-SEB-SEH:    sra     $[[R11:[0-9]+]], $[[R10]], 16
 
-; NO-SEB-SEH:    sll     $[[R12:[0-9]+]], $[[R2]], 16
+; NO-SEB-SEH:    sll     $[[R12:[0-9]+]], $[[R3]], 16
 ; NO-SEB-SEH:    sra     $[[R13:[0-9]+]], $[[R12]], 16
 
 ; HAS-SEB-SEH:   seh     $[[R11:[0-9]+]], $[[R9]]
-; HAS-SEB-SEH:   seh     $[[R13:[0-9]+]], $[[R2]]
+; HAS-SEB-SEH:   seh     $[[R13:[0-9]+]], $[[R3]]
 
 ; ALL:           xor     $[[R12:[0-9]+]], $[[R11]], $[[R13]]
 ; ALL:           sltiu   $3, $[[R12]], 1
 ; ALL:           sync
 }
 
 @countsint = common global i32 0, align 4
 

test/CodeGen/PowerPC/PR35812-neg-cmpxchg.ll

+; Make sure that a negative value for the compare-and-swap is zero extended
+; from i8/i16 to i32 since it will be compared for equality.
+; RUN: llc -mtriple=powerpc64le-linux-gnu -verify-machineinstrs < %s | FileCheck %s
+@Glob = global i16 -3344, align 2
+@str = private unnamed_addr constant [9 x i8] c"Failure!\00"
+@str.2 = private unnamed_addr constant [9 x i8] c"Success!\00"
+
+; Function Attrs: nounwind
+define signext i32 @main() {
+; CHECK:    lis 4, 0
+; CHECK:    li 6, 222
+; CHECK:    sync
+; CHECK:    ori 5, 4, 62192
+; CHECK:  .LBB0_1: # %entry
+; CHECK:    lharx 4, 0, 3
+; CHECK:    cmpw 5, 4
+; CHECK:    bne 0, .LBB0_3
+; CHECK:    sthcx. 6, 0, 3
+; CHECK:    bne 0, .LBB0_1
+; CHECK:    b .LBB0_4
+; CHECK:  .LBB0_3: # %entry
+; CHECK:    sthcx. 4, 0, 3
+; CHECK:  .LBB0_4: # %entry
+; CHECK:    cmplwi 4, 62192
+; CHECK:    lwsync
+entry:
+  %0 = cmpxchg i16* @Glob, i16 -3344, i16 222 seq_cst seq_cst
+  %1 = extractvalue { i16, i1 } %0, 1
+  br i1 %1, label %if.then, label %if.else
+
+if.then:                                          ; preds = %entry
+  %puts2 = tail call i32 @puts(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @str.2, i64 0, i64 0))
+  br label %if.end
+
+if.else:                                          ; preds = %entry
+  %puts = tail call i32 @puts(i8* getelementptr inbounds ([9 x i8], [9 x i8]* @str, i64 0, i64 0))
+  br label %if.end
+
+if.end:                                           ; preds = %if.else, %if.then
+  ret i32 0
+}
+
+; Function Attrs: nounwind
+declare i32 @puts(i8* nocapture readonly)

test/CodeGen/PowerPC/atomics-regression.ll

 ; PPC64LE-LABEL: test39:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:    blr
   fence syncscope("singlethread") seq_cst
   ret void
 }
 
-define void @test40(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test40(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test40:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    b .LBB40_2
 ; PPC64LE-NEXT:    .p2align 5
 ; PPC64LE-NEXT:  .LBB40_1:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    beqlr 0
 ; PPC64LE-NEXT:  .LBB40_2:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    beq 0, .LBB40_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val monotonic monotonic
   ret void
 }
 
-define void @test41(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test41(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test41:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:  .LBB41_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB41_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB41_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB41_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val acquire monotonic
   ret void
 }
 
-define void @test42(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test42(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test42:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:  .LBB42_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB42_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB42_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB42_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val acquire acquire
   ret void
 }
 
-define void @test43(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test43(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test43:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    b .LBB43_2
 ; PPC64LE-NEXT:    .p2align 5
 ; PPC64LE-NEXT:  .LBB43_1:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    beqlr 0
 ; PPC64LE-NEXT:  .LBB43_2:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    beq 0, .LBB43_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val release monotonic
   ret void
 }
 
-define void @test44(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test44(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test44:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    b .LBB44_2
 ; PPC64LE-NEXT:    .p2align 5
 ; PPC64LE-NEXT:  .LBB44_1:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    beqlr 0
 ; PPC64LE-NEXT:  .LBB44_2:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    beq 0, .LBB44_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val release acquire
   ret void
 }
 
-define void @test45(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test45(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test45:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:  .LBB45_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB45_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB45_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB45_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val acq_rel monotonic
   ret void
 }
 
-define void @test46(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test46(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test46:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:  .LBB46_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB46_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB46_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB46_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val acq_rel acquire
   ret void
 }
 
-define void @test47(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test47(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test47:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:  .LBB47_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB47_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB47_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB47_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val seq_cst monotonic
   ret void
 }
 
-define void @test48(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test48(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test48:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:  .LBB48_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB48_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB48_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB48_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val seq_cst acquire
   ret void
 }
 
-define void @test49(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test49(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test49:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:  .LBB49_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB49_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB49_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB49_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val seq_cst seq_cst
   ret void
 }
 
-define void @test50(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test50(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test50:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    b .LBB50_2
 ; PPC64LE-NEXT:    .p2align 5
 ; PPC64LE-NEXT:  .LBB50_1:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    beqlr 0
 ; PPC64LE-NEXT:  .LBB50_2:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    beq 0, .LBB50_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val monotonic monotonic
   ret void
 }
 
-define void @test51(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test51(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test51:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:  .LBB51_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB51_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB51_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB51_4:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val acquire monotonic
   ret void
 }
 
-define void @test52(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test52(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test52:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:  .LBB52_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB52_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB52_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB52_4:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val acquire acquire
   ret void
 }
 
-define void @test53(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test53(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test53:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    b .LBB53_2
 ; PPC64LE-NEXT:    .p2align 5
 ; PPC64LE-NEXT:  .LBB53_1:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    beqlr 0
 ; PPC64LE-NEXT:  .LBB53_2:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    beq 0, .LBB53_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val release monotonic
   ret void
 }
 
-define void @test54(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test54(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test54:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    b .LBB54_2
 ; PPC64LE-NEXT:    .p2align 5
 ; PPC64LE-NEXT:  .LBB54_1:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    beqlr 0
 ; PPC64LE-NEXT:  .LBB54_2:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    beq 0, .LBB54_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val release acquire
   ret void
 }
 
-define void @test55(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test55(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test55:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:  .LBB55_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB55_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB55_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB55_4:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val acq_rel monotonic
   ret void
 }
 
-define void @test56(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test56(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test56:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:  .LBB56_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB56_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB56_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB56_4:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val acq_rel acquire
   ret void
 }
 
-define void @test57(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test57(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test57:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:  .LBB57_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB57_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB57_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB57_4:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val seq_cst monotonic
   ret void
 }
 
-define void @test58(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test58(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test58:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:  .LBB58_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB58_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB58_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB58_4:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val seq_cst acquire
   ret void
 }
 
-define void @test59(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test59(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test59:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:  .LBB59_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB59_4
 ; PPC64LE-NEXT:  # %bb.2:

 ; PPC64LE-NEXT:  .LBB79_4:
 ; PPC64LE-NEXT:    stdcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i64* %ptr, i64 %cmp, i64 %val seq_cst seq_cst
   ret void
 }
 
-define void @test80(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test80(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test80:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    b .LBB80_2
 ; PPC64LE-NEXT:    .p2align 5
 ; PPC64LE-NEXT:  .LBB80_1:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    beqlr 0
 ; PPC64LE-NEXT:  .LBB80_2:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    beq 0, .LBB80_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") monotonic monotonic
   ret void
 }
 
-define void @test81(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test81(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test81:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:  .LBB81_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB81_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB81_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB81_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") acquire monotonic
   ret void
 }
 
-define void @test82(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test82(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test82:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:  .LBB82_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB82_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB82_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB82_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") acquire acquire
   ret void
 }
 
-define void @test83(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test83(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test83:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    b .LBB83_2
 ; PPC64LE-NEXT:    .p2align 5
 ; PPC64LE-NEXT:  .LBB83_1:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    beqlr 0
 ; PPC64LE-NEXT:  .LBB83_2:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    beq 0, .LBB83_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") release monotonic
   ret void
 }
 
-define void @test84(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test84(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test84:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    b .LBB84_2
 ; PPC64LE-NEXT:    .p2align 5
 ; PPC64LE-NEXT:  .LBB84_1:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    beqlr 0
 ; PPC64LE-NEXT:  .LBB84_2:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    beq 0, .LBB84_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") release acquire
   ret void
 }
 
-define void @test85(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test85(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test85:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:  .LBB85_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB85_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB85_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB85_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") acq_rel monotonic
   ret void
 }
 
-define void @test86(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test86(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test86:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:  .LBB86_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB86_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB86_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB86_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") acq_rel acquire
   ret void
 }
 
-define void @test87(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test87(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test87:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:  .LBB87_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB87_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB87_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB87_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") seq_cst monotonic
   ret void
 }
 
-define void @test88(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test88(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test88:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:  .LBB88_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB88_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB88_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB88_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") seq_cst acquire
   ret void
 }
 
-define void @test89(i8* %ptr, i8 %cmp, i8 %val) {
+define void @test89(i8* %ptr, i8 zeroext %cmp, i8 %val) {
 ; PPC64LE-LABEL: test89:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:  .LBB89_1:
 ; PPC64LE-NEXT:    lbarx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB89_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    stbcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB89_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB89_4:
 ; PPC64LE-NEXT:    stbcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i8* %ptr, i8 %cmp, i8 %val syncscope("singlethread") seq_cst seq_cst
   ret void
 }
 
-define void @test90(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test90(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test90:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    b .LBB90_2
 ; PPC64LE-NEXT:    .p2align 5
 ; PPC64LE-NEXT:  .LBB90_1:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    beqlr 0
 ; PPC64LE-NEXT:  .LBB90_2:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    beq 0, .LBB90_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") monotonic monotonic
   ret void
 }
 
-define void @test91(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test91(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test91:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:  .LBB91_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB91_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB91_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB91_4:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") acquire monotonic
   ret void
 }
 
-define void @test92(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test92(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test92:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:  .LBB92_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB92_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB92_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB92_4:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") acquire acquire
   ret void
 }
 
-define void @test93(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test93(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test93:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    b .LBB93_2
 ; PPC64LE-NEXT:    .p2align 5
 ; PPC64LE-NEXT:  .LBB93_1:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    beqlr 0
 ; PPC64LE-NEXT:  .LBB93_2:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    beq 0, .LBB93_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") release monotonic
   ret void
 }
 
-define void @test94(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test94(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test94:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    b .LBB94_2
 ; PPC64LE-NEXT:    .p2align 5
 ; PPC64LE-NEXT:  .LBB94_1:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    beqlr 0
 ; PPC64LE-NEXT:  .LBB94_2:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    beq 0, .LBB94_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") release acquire
   ret void
 }
 
-define void @test95(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test95(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test95:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:  .LBB95_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB95_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB95_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB95_4:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") acq_rel monotonic
   ret void
 }
 
-define void @test96(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test96(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test96:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:  .LBB96_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB96_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB96_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB96_4:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") acq_rel acquire
   ret void
 }
 
-define void @test97(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test97(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test97:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:  .LBB97_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB97_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB97_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB97_4:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") seq_cst monotonic
   ret void
 }
 
-define void @test98(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test98(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test98:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:  .LBB98_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB98_4
 ; PPC64LE-NEXT:  # %bb.2:
 ; PPC64LE-NEXT:    sthcx. 5, 0, 3
 ; PPC64LE-NEXT:    bne 0, .LBB98_1
 ; PPC64LE-NEXT:  # %bb.3:
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
 ; PPC64LE-NEXT:  .LBB98_4:
 ; PPC64LE-NEXT:    sthcx. 6, 0, 3
 ; PPC64LE-NEXT:    lwsync
 ; PPC64LE-NEXT:    blr
   %res = cmpxchg i16* %ptr, i16 %cmp, i16 %val syncscope("singlethread") seq_cst acquire
   ret void
 }
 
-define void @test99(i16* %ptr, i16 %cmp, i16 %val) {
+define void @test99(i16* %ptr, i16 zeroext %cmp, i16 %val) {
 ; PPC64LE-LABEL: test99:
 ; PPC64LE:       # %bb.0:
 ; PPC64LE-NEXT:    sync
 ; PPC64LE-NEXT:  .LBB99_1:
 ; PPC64LE-NEXT:    lharx 6, 0, 3
 ; PPC64LE-NEXT:    cmpw 4, 6
 ; PPC64LE-NEXT:    bne 0, .LBB99_4
 ; PPC64LE-NEXT:  # %bb.2:

test/CodeGen/SystemZ/cmpxchg-01.ll

 ; Test 8-bit compare and swap.
 ;
 ; RUN: llc < %s -mtriple=s390x-linux-gnu | FileCheck %s -check-prefix=CHECK-MAIN
 ; RUN: llc < %s -mtriple=s390x-linux-gnu | FileCheck %s -check-prefix=CHECK-SHIFT
 
 ; Check compare and swap with a variable.
 ; - CHECK is for the main loop.
 ; - CHECK-SHIFT makes sure that the negated shift count used by the second
 ;   RLL is set up correctly.  The negation is independent of the NILL and L
 ;   tested in CHECK.  CHECK-SHIFT also checks that %r3 is not modified before
 ;   being used in the RISBG (in contrast to things like atomic addition,
 ;   which shift %r3 left so that %b is at the high end of the word).
-define i8 @f1(i8 %dummy, i8 *%src, i8 %cmp, i8 %swap) {
+define i8 @f1(i8 %dummy, i8 *%src, i8 zeroext %cmp, i8 %swap) {
 ; CHECK-MAIN-LABEL: f1:
 ; CHECK-MAIN: risbg [[RISBG:%r[1-9]+]], %r3, 0, 189, 0{{$}}
 ; CHECK-MAIN-DAG: sll %r3, 3
 ; CHECK-MAIN-DAG: l [[OLD:%r[0-9]+]], 0([[RISBG]])
 ; CHECK-MAIN: [[LOOP:\.[^ ]*]]:
 ; CHECK-MAIN: rll %r2, [[OLD]], 8(%r3)
 ; CHECK-MAIN: risbg %r4, %r2, 32, 55, 0
 ; CHECK-MAIN: crjlh %r2, %r4, [[EXIT:\.[^ ]*]]

 ; CHECK-SHIFT: risbg [[SWAP]], {{%r[0-9]+}}, 32, 55, 0
 ; CHECK-SHIFT: br %r14
   %pair = cmpxchg i8 *%src, i8 42, i8 88 seq_cst seq_cst
   %res = extractvalue { i8, i1 } %pair, 0
   ret i8 %res
 }
 
 ; Check generating the comparison result.
-define i32 @f3(i8 %dummy, i8 *%src, i8 %cmp, i8 %swap) {
+define i32 @f3(i8 %dummy, i8 *%src, i8 zeroext %cmp, i8 %swap) {
 ; CHECK-MAIN-LABEL: f3:
 ; CHECK-MAIN: risbg [[RISBG:%r[1-9]+]], %r3, 0, 189, 0{{$}}
 ; CHECK-MAIN-DAG: sll %r3, 3
 ; CHECK-MAIN-DAG: l [[OLD:%r[0-9]+]], 0([[RISBG]])
 ; CHECK-MAIN: [[LOOP:\.[^ ]*]]:
 ; CHECK-MAIN: rll [[TMP:%r[0-9]+]], [[OLD]], 8(%r3)
 ; CHECK-MAIN: risbg %r4, [[TMP]], 32, 55, 0
 ; CHECK-MAIN: cr [[TMP]], %r4

test/CodeGen/SystemZ/cmpxchg-02.ll

 ; Test 16-bit compare and swap.
 ;
 ; RUN: llc < %s -mtriple=s390x-linux-gnu | FileCheck %s -check-prefix=CHECK-MAIN
 ; RUN: llc < %s -mtriple=s390x-linux-gnu | FileCheck %s -check-prefix=CHECK-SHIFT
 
 ; Check compare and swap with a variable.
 ; - CHECK is for the main loop.
 ; - CHECK-SHIFT makes sure that the negated shift count used by the second
 ;   RLL is set up correctly.  The negation is independent of the NILL and L
 ;   tested in CHECK.  CHECK-SHIFT also checks that %r3 is not modified before
 ;   being used in the RISBG (in contrast to things like atomic addition,
 ;   which shift %r3 left so that %b is at the high end of the word).
-define i16 @f1(i16 %dummy, i16 *%src, i16 %cmp, i16 %swap) {
+define i16 @f1(i16 %dummy, i16 *%src, i16 zeroext %cmp, i16 %swap) {
 ; CHECK-MAIN-LABEL: f1:
 ; CHECK-MAIN: risbg [[RISBG:%r[1-9]+]], %r3, 0, 189, 0{{$}}
 ; CHECK-MAIN-DAG: sll %r3, 3
 ; CHECK-MAIN-DAG: l [[OLD:%r[0-9]+]], 0([[RISBG]])
 ; CHECK-MAIN: [[LOOP:\.[^ ]*]]:
 ; CHECK-MAIN: rll %r2, [[OLD]], 16(%r3)
 ; CHECK-MAIN: risbg %r4, %r2, 32, 47, 0
 ; CHECK-MAIN: crjlh %r2, %r4, [[EXIT:\.[^ ]*]]

 ; CHECK-SHIFT: risbg [[SWAP]], {{%r[0-9]+}}, 32, 47, 0
 ; CHECK-SHIFT: br %r14
   %pair = cmpxchg i16 *%src, i16 42, i16 88 seq_cst seq_cst
   %res = extractvalue { i16, i1 } %pair, 0
   ret i16 %res
 }
 
 ; Check generating the comparison result.
-define i32 @f3(i16 %dummy, i16 *%src, i16 %cmp, i16 %swap) {
+define i32 @f3(i16 %dummy, i16 *%src, i16 zeroext %cmp, i16 %swap) {
 ; CHECK-MAIN-LABEL: f3:
 ; CHECK-MAIN: risbg [[RISBG:%r[1-9]+]], %r3, 0, 189, 0{{$}}
 ; CHECK-MAIN-DAG: sll %r3, 3
 ; CHECK-MAIN-DAG: l [[OLD:%r[0-9]+]], 0([[RISBG]])
 ; CHECK-MAIN: [[LOOP:\.[^ ]*]]:
 ; CHECK-MAIN: rll [[TMP:%r[0-9]+]], [[OLD]], 16(%r3)
 ; CHECK-MAIN: risbg %r4, [[TMP]], 32, 47, 0
 ; CHECK-MAIN: cr [[TMP]], %r4