[AArch64] Do not emit an extra zero-extend for i1 argument

[asavonic]

Summary

AAPCS requires i1 argument to be zero-extended to 8-bits by the caller.
Emit a new AArch64ISD::ASSERT_ZEXT_BOOL hint to enable some
optimization opportunities. In particular, when the argument is forwarded
to the callee, we can avoid zero-extension and use it as-is.

[efriedma]

On the SelectionDAG ISel side, I'm not really happy we're working around the usual framework for dealing with known bits. Can we use some sort of target-specific equivalent to AssertZExt to encode the known bits, then use those known bits when we generate the call?

[paquette]

If it's possible to handle this using AssertZExt on the SDAG side, I think it would be nice to handle it similarly on the GISel side using G_ASSERT_ZEXT.

llvm/lib/Target/AArch64/GISel/AArch64CallLowering.cpp

1079

I think that you can handle most of the logic in here using MIPatternMatch:

Register OrigReg;
if (!mi_match(AI.Regs[0], MRI, m_GTrunc(m_Reg(OrigReg))))
   return false;

1088

Maybe this loop can be replaced with llvm::any_of?

[asavonic]

Used an AssertZExt to keep information about zero-extended low bits.
There were a number of places where the compiler failed to optimize this correctly (found by LIT tests), so I fixed them as well.

[efriedma]

This turned out much messier than I was hoping for...

I guess we have the following possibilities:

1. Go back to the original patch.
2. Go with something like this approach, maybe with a few small cleanups.
3. Try to use target-specific nodes to dodge the legalization issues. Instead of using the target-independent nodes, introduce AArch64ISD::AssertZextBool/AArch64::ZERO_EXTEND_BOOL which have an i32 argument/result, and then DAGCombine them away after type legalization cleans up all the extend/truncate ops which make this harder to analyze. Maybe more code, but easier to understand the implications, I think.

You might want to add a testcase where the call isn't in the entry block; I think maybe the original patch handles this?

llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp

3889 ↗

(On Diff #368054)

You can't unconditionally truncate to i1 here, I think; this might be after type legalization.

[asavonic]

Added AArch64ISD::ASSERT_ZEXT_BOOL opcode to avoid legalization issues with AssertZExt.
Added a test case with a call not in the entry block.

[asavonic]

In D107160#2961436, @efriedma wrote:

This turned out much messier than I was hoping for...

I guess we have the following possibilities:

1. Go back to the original patch.
2. Go with something like this approach, maybe with a few small cleanups.
3. Try to use target-specific nodes to dodge the legalization issues. Instead of using the target-independent nodes, introduce AArch64ISD::AssertZextBool/AArch64::ZERO_EXTEND_BOOL which have an i32 argument/result, and then DAGCombine them away after type legalization cleans up all the extend/truncate ops which make this harder to analyze. Maybe more code, but easier to understand the implications, I think.

Thanks a lot! I decided to try the option 3, and it seems to work fine.

You might want to add a testcase where the call isn't in the entry block; I think maybe the original patch handles this?

This case does not work well with SelectionDAG - the hint and the call are in different blocks, and thus not visible.
I noticed that zeroext flag is propagated correctly between blocks, so we might be able to do the same for the assertion.
I guess that ideally we need to propagate known bits instead, but I'm not sure if it is possible to do this.

[efriedma]

For cross-block information, we do some limited tracking in target-independent code. See FunctionLoweringInfo::ComputePHILiveOutRegInfo. We could maybe mess with that code to track more information, or add a target hook to SelectionDAG around GetLiveOutRegInfo(). Actually, we could maybe even just feed the info from ComputePHILiveOutRegInfo directly into ComputeKnownBits; we actually keep all the information around already? (Not sure what the history is here...)

But we can leave that for a followup. The current SelectionDAG changes seem fine. I'm not familiar enough with the globalisel bits to approve that.

[asavonic]

In D107160#2917852, @paquette wrote:

If it's possible to handle this using AssertZExt on the SDAG side, I think it would be nice to handle it similarly on the GISel side using G_ASSERT_ZEXT.

Can you please check if the GISel changes are fine?

[asavonic]

ping

[aemerson]

llvm/lib/Target/AArch64/GISel/AArch64CallLowering.cpp

553

This can be LLT::scalar(8)

587

If these are bool args then we can assume OrigTy.getScalarSizeInBits() == 1 right?

588–589

MIRBuilder's builder methods return a MachineInstrBuilder that can be directly passed into most source operands of other builder methods. You can simplify these lines into something like:

MIRBuilder.buildTrunc(OrigReg, MIRBuilder.buildAssertZExt(WideTy, WideReg, 1));

[asavonic]

llvm/lib/Target/AArch64/GISel/AArch64CallLowering.cpp

553

Done.

587

Right. Added an assert and replaced the call with 1.

588–589

Thank you. Done.

llvm/lib/Target/AArch64/AArch64ISelLowering.h

enum NodeType : unsigned {
   SST1_UXTW_SCALED_PRED,
   SST1_SXTW_SCALED_PRED,
   SST1_IMM_PRED,
 
   // Non-temporal scatter store
   SSTNT1_PRED,
   SSTNT1_INDEX_PRED,
 
+  // Asserts that a function argument (i32) is zero-extended to i8 by
+  // the caller
+  ASSERT_ZEXT_BOOL,
+
   // Strict (exception-raising) floating point comparison
   STRICT_FCMP = ISD::FIRST_TARGET_STRICTFP_OPCODE,
   STRICT_FCMPE,
 
   // NEON Load/Store with post-increment base updates
   LD2post = ISD::FIRST_TARGET_MEMORY_OPCODE,
   LD3post,
   LD4post,

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp

void AArch64TargetLowering::computeKnownBitsForTargetNode(
   case AArch64ISD::LOADgot:
   case AArch64ISD::ADDlow: {
     if (!Subtarget->isTargetILP32())
       break;
     // In ILP32 mode all valid pointers are in the low 4GB of the address-space.
     Known.Zero = APInt::getHighBitsSet(64, 32);
     break;
   }
+  case AArch64ISD::ASSERT_ZEXT_BOOL: {
+    Known = DAG.computeKnownBits(Op->getOperand(0), Depth + 1);
+    Known.Zero |= APInt(Known.getBitWidth(), 0xFE);
+    break;
+  }
   case ISD::INTRINSIC_W_CHAIN: {
     ConstantSDNode *CN = cast<ConstantSDNode>(Op->getOperand(1));
     Intrinsic::ID IntID = static_cast<Intrinsic::ID>(CN->getZExtValue());
     switch (IntID) {
     default: return;
     case Intrinsic::aarch64_ldaxr:
     case Intrinsic::aarch64_ldxr: {
       unsigned BitWidth = Known.getBitWidth();
case AArch64ISD::FIRST_NUMBER:
     MAKE_CASE(AArch64ISD::BITREVERSE_MERGE_PASSTHRU)
     MAKE_CASE(AArch64ISD::BSWAP_MERGE_PASSTHRU)
     MAKE_CASE(AArch64ISD::CTLZ_MERGE_PASSTHRU)
     MAKE_CASE(AArch64ISD::CTPOP_MERGE_PASSTHRU)
     MAKE_CASE(AArch64ISD::DUP_MERGE_PASSTHRU)
     MAKE_CASE(AArch64ISD::INDEX_VECTOR)
     MAKE_CASE(AArch64ISD::UADDLP)
     MAKE_CASE(AArch64ISD::CALL_RVMARKER)
+    MAKE_CASE(AArch64ISD::ASSERT_ZEXT_BOOL)
   }
 #undef MAKE_CASE
   return nullptr;
 }
 
 MachineBasicBlock *
 AArch64TargetLowering::EmitF128CSEL(MachineInstr &MI,
                                     MachineBasicBlock *MBB) const {
if (VA.getLocInfo() == CCValAssign::Indirect) {
           ExtraArgLocs++;
           i++;
         }
       }
     } else {
       if (Subtarget->isTargetILP32() && Ins[i].Flags.isPointer())
         ArgValue = DAG.getNode(ISD::AssertZext, DL, ArgValue.getValueType(),
                                ArgValue, DAG.getValueType(MVT::i32));
+
+      // i1 arguments are zero-extended to i8 by the caller. Emit a
+      // hint to reflect this.
+      if (Ins[i].isOrigArg()) {
+        Argument *OrigArg = MF.getFunction().getArg(Ins[i].getOrigArgIndex());
+        if (OrigArg->getType()->isIntegerTy(1)) {
+          if (!Ins[i].Flags.isZExt()) {
+            ArgValue = DAG.getNode(AArch64ISD::ASSERT_ZEXT_BOOL, DL,
+                                   ArgValue.getValueType(), ArgValue);
+          }
+        }
+      }
+
       InVals.push_back(ArgValue);
     }
   }
   assert((ArgLocs.size() + ExtraArgLocs) == Ins.size());
 
   // varargs
   AArch64FunctionInfo *FuncInfo = MF.getInfo<AArch64FunctionInfo>();
   if (isVarArg) {

 }
 
 bool AArch64TargetLowering::DoesCalleeRestoreStack(CallingConv::ID CallCC,
                                                    bool TailCallOpt) const {
   return (CallCC == CallingConv::Fast && TailCallOpt) ||
          CallCC == CallingConv::Tail || CallCC == CallingConv::SwiftTail;
 }
 
+// Check if the value is zero-extended from i1 to i8
+static bool checkZExtBool(SDValue Arg, const SelectionDAG &DAG) {
+  unsigned SizeInBits = Arg.getValueType().getSizeInBits();
+  if (SizeInBits < 8)
+    return false;
+
+  APInt LowBits(SizeInBits, 0xFF);
+  APInt RequredZero(SizeInBits, 0xFE);
+  KnownBits Bits = DAG.computeKnownBits(Arg, LowBits, 4);
+  bool ZExtBool = (Bits.Zero & RequredZero) == RequredZero;
+  return ZExtBool;
+}
+
 /// LowerCall - Lower a call to a callseq_start + CALL + callseq_end chain,
 /// and add input and output parameter nodes.
 SDValue
 AArch64TargetLowering::LowerCall(CallLoweringInfo &CLI,
                                  SmallVectorImpl<SDValue> &InVals) const {
   SelectionDAG &DAG = CLI.DAG;
   SDLoc &DL = CLI.DL;
   SmallVector<ISD::OutputArg, 32> &Outs = CLI.Outs;
case CCValAssign::SExt:
       Arg = DAG.getNode(ISD::SIGN_EXTEND, DL, VA.getLocVT(), Arg);
       break;
     case CCValAssign::ZExt:
       Arg = DAG.getNode(ISD::ZERO_EXTEND, DL, VA.getLocVT(), Arg);
       break;
     case CCValAssign::AExt:
       if (Outs[i].ArgVT == MVT::i1) {
         // AAPCS requires i1 to be zero-extended to 8-bits by the caller.
+        //
+        // Check if we actually have to do this, because the value may
+        // already be zero-extended.
+        //
+        // We cannot just emit a (zext i8 (trunc (assert-zext i8)))
+        // and rely on DAGCombiner to fold this, because the following
+        // (anyext i32) is combined with (zext i8) in DAG.getNode:
+        //
+        //   (ext (zext x)) -> (zext x)
+        //
+        // This will give us (zext i32), which we cannot remove, so
+        // try to check this beforehand.
+        if (!checkZExtBool(Arg, DAG)) {
           Arg = DAG.getNode(ISD::TRUNCATE, DL, MVT::i1, Arg);
           Arg = DAG.getNode(ISD::ZERO_EXTEND, DL, MVT::i8, Arg);
         }
+      }
       Arg = DAG.getNode(ISD::ANY_EXTEND, DL, VA.getLocVT(), Arg);
       break;
     case CCValAssign::AExtUpper:
       assert(VA.getValVT() == MVT::i32 && "only expect 32 -> 64 upper bits");
       Arg = DAG.getNode(ISD::ANY_EXTEND, DL, VA.getLocVT(), Arg);
       Arg = DAG.getNode(ISD::SHL, DL, VA.getLocVT(), Arg,
                         DAG.getConstant(32, DL, VA.getLocVT()));
       break;

llvm/lib/Target/AArch64/AArch64InstrInfo.td

let Predicates = [HasLS64] in {
   def : ST64BPattern<int_aarch64_st64bv0, ST64BV0>;
 }
 
 let Defs = [X16, X17], mayStore = 1, isCodeGenOnly = 1 in
 def StoreSwiftAsyncContext
       : Pseudo<(outs), (ins GPR64:$ctx, GPR64sp:$base, simm9:$offset),
                []>, Sched<[]>;
 
+def AArch64AssertZExtBool : SDNode<"AArch64ISD::ASSERT_ZEXT_BOOL", SDT_assert>;
+def : Pat<(AArch64AssertZExtBool GPR32:$op),
+          (i32 GPR32:$op)>;
+
 include "AArch64InstrAtomics.td"
 include "AArch64SVEInstrInfo.td"
 include "AArch64SMEInstrInfo.td"
 include "AArch64InstrGISel.td"

llvm/lib/Target/AArch64/GISel/AArch64CallLowering.cpp

bool AArch64CallLowering::lowerFormalArguments(
     MachineIRBuilder &MIRBuilder, const Function &F,
     ArrayRef<ArrayRef<Register>> VRegs, FunctionLoweringInfo &FLI) const {
   MachineFunction &MF = MIRBuilder.getMF();
   MachineBasicBlock &MBB = MIRBuilder.getMBB();
   MachineRegisterInfo &MRI = MF.getRegInfo();
   auto &DL = F.getParent()->getDataLayout();
 
   SmallVector<ArgInfo, 8> SplitArgs;
+  SmallVector<std::pair<Register, Register>> BoolArgs;
   unsigned i = 0;
   for (auto &Arg : F.args()) {
     if (DL.getTypeStoreSize(Arg.getType()).isZero())
       continue;
 
     ArgInfo OrigArg{VRegs[i], Arg, i};
     setArgFlags(OrigArg, i + AttributeList::FirstArgIndex, DL, F);
 
+    // i1 arguments are zero-extended to i8 by the caller. Emit a
+    // hint to reflect this.
+    if (OrigArg.Ty->isIntegerTy(1)) {
+      assert(OrigArg.Regs.size() == 1 &&
+             MRI.getType(OrigArg.Regs[0]).getSizeInBits() == 1 &&
+             "Unexpected registers used for i1 arg");
+
+      if (!OrigArg.Flags[0].isZExt()) {
+        // Lower i1 argument as i8, and insert AssertZExt + Trunc later.
+        Register OrigReg = OrigArg.Regs[0];
+        Register WideReg = MRI.createGenericVirtualRegister(LLT::scalar(8));
+        OrigArg.Regs[0] = WideReg;
+        BoolArgs.push_back({OrigReg, WideReg});
+      }
+    }
+
     if (Arg.hasAttribute(Attribute::SwiftAsync))
       MF.getInfo<AArch64FunctionInfo>()->setHasSwiftAsyncContext(true);
 
     splitToValueTypes(OrigArg, SplitArgs, DL, F.getCallingConv());
     ++i;
   }
 
   if (!MBB.empty())
     MIRBuilder.setInstr(*MBB.begin());
 
   const AArch64TargetLowering &TLI = *getTLI<AArch64TargetLowering>();
   CCAssignFn *AssignFn =
       TLI.CCAssignFnForCall(F.getCallingConv(), /*IsVarArg=*/false);
 
   AArch64IncomingValueAssigner Assigner(AssignFn, AssignFn);
   FormalArgHandler Handler(MIRBuilder, MRI);
   if (!determineAndHandleAssignments(Handler, Assigner, SplitArgs, MIRBuilder,
                                      F.getCallingConv(), F.isVarArg()))
     return false;
 
+  if (!BoolArgs.empty()) {
+    for (auto &KV : BoolArgs) {
+      Register OrigReg = KV.first;
+      Register WideReg = KV.second;
+      LLT WideTy = MRI.getType(WideReg);
+      assert(MRI.getType(OrigReg).getScalarSizeInBits() == 1 &&
+             "Unexpected bit size of a bool arg");
+      MIRBuilder.buildTrunc(
+          OrigReg, MIRBuilder.buildAssertZExt(WideTy, WideReg, 1).getReg(0));
+    }
+  }
+
   AArch64FunctionInfo *FuncInfo = MF.getInfo<AArch64FunctionInfo>();
   uint64_t StackOffset = Assigner.StackOffset;
   if (F.isVarArg()) {
     auto &Subtarget = MF.getSubtarget<AArch64Subtarget>();
     if (!Subtarget.isTargetDarwin()) {
         // FIXME: we need to reimplement saveVarArgsRegisters from
       // AArch64ISelLowering.
       return false;

 bool AArch64CallLowering::lowerCall(MachineIRBuilder &MIRBuilder,
                                     CallLoweringInfo &Info) const {
   MachineFunction &MF = MIRBuilder.getMF();
   const Function &F = MF.getFunction();
   MachineRegisterInfo &MRI = MF.getRegInfo();
   auto &DL = F.getParent()->getDataLayout();
   const AArch64TargetLowering &TLI = *getTLI<AArch64TargetLowering>();
 
   SmallVector<ArgInfo, 8> OutArgs;
   for (auto &OrigArg : Info.OrigArgs) {
     splitToValueTypes(OrigArg, OutArgs, DL, Info.CallConv);
     // AAPCS requires that we zero-extend i1 to 8 bits by the caller.
-    if (OrigArg.Ty->isIntegerTy(1))
-      OutArgs.back().Flags[0].setZExt();
+    if (OrigArg.Ty->isIntegerTy(1)) {
+      ArgInfo &OutArg = OutArgs.back();
+      assert(OutArg.Regs.size() == 1 &&
+             MRI.getType(OutArg.Regs[0]).getSizeInBits() == 1 &&
+             "Unexpected registers used for i1 arg");
+
+      // We cannot use a ZExt ArgInfo flag here, because it will
+      // zero-extend the argument to i32 instead of just i8.
+      OutArg.Regs[0] =
+          MIRBuilder.buildZExt(LLT::scalar(8), OutArg.Regs[0]).getReg(0);
+      LLVMContext &Ctx = MF.getFunction().getContext();
+      OutArg.Ty = Type::getInt8Ty(Ctx);
+    }
   }
 
   SmallVector<ArgInfo, 8> InArgs;
   if (!Info.OrigRet.Ty->isVoidTy())
     splitToValueTypes(Info.OrigRet, InArgs, DL, Info.CallConv);
 
   // If we can lower as a tail call, do that instead.
   bool CanTailCallOpt =

llvm/test/CodeGen/AArch64/GlobalISel/arm64-irtranslator-switch.ll

 }
 
 define i1 @i1_value_cmp_is_signed(i1) {
   ; CHECK-LABEL: name: i1_value_cmp_is_signed
   ; CHECK: bb.1.Entry:
   ; CHECK:   successors: %bb.3(0x40000000), %bb.2(0x40000000)
   ; CHECK:   liveins: $w0
   ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
-  ; CHECK:   [[TRUNC:%[0-9]+]]:_(s1) = G_TRUNC [[COPY]](s32)
+  ; CHECK:   [[TRUNC:%[0-9]+]]:_(s8) = G_TRUNC [[COPY]](s32)
+  ; CHECK:   [[ASSERT_ZEXT:%[0-9]+]]:_(s8) = G_ASSERT_ZEXT [[TRUNC]], 1
+  ; CHECK:   [[TRUNC1:%[0-9]+]]:_(s1) = G_TRUNC [[ASSERT_ZEXT]](s8)
   ; CHECK:   [[C:%[0-9]+]]:_(s1) = G_CONSTANT i1 true
   ; CHECK:   [[C1:%[0-9]+]]:_(s1) = G_CONSTANT i1 false
-  ; CHECK:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(sle), [[TRUNC]](s1), [[C1]]
+  ; CHECK:   [[ICMP:%[0-9]+]]:_(s1) = G_ICMP intpred(sle), [[TRUNC1]](s1), [[C1]]
   ; CHECK:   G_BRCOND [[ICMP]](s1), %bb.3
   ; CHECK:   G_BR %bb.2
   ; CHECK: bb.2.BadValue:
   ; CHECK:   successors:
   ; CHECK:   ADJCALLSTACKDOWN 0, 0, implicit-def $sp, implicit $sp
   ; CHECK:   BL @bar, csr_aarch64_aapcs, implicit-def $lr, implicit $sp
   ; CHECK:   ADJCALLSTACKUP 0, 0, implicit-def $sp, implicit $sp
   ; CHECK: bb.3.OkValue:
-  ; CHECK:   [[ZEXT:%[0-9]+]]:_(s8) = G_ZEXT [[TRUNC]](s1)
+  ; CHECK:   [[ZEXT:%[0-9]+]]:_(s8) = G_ZEXT [[TRUNC1]](s1)
   ; CHECK:   [[ANYEXT:%[0-9]+]]:_(s32) = G_ANYEXT [[ZEXT]](s8)
   ; CHECK:   $w0 = COPY [[ANYEXT]](s32)
   ; CHECK:   RET_ReallyLR implicit $w0
 Entry:
   switch i1 %0, label %BadValue [
     i1 false, label %OkValue
     i1 true, label %OkValue
   ]

llvm/test/CodeGen/AArch64/GlobalISel/arm64-irtranslator.ll

define void @test_insertvalue_agg(%struct.nested* %addr, {i8, i32}* %addr2) {
   %struct = load %struct.nested, %struct.nested* %addr
   %res = insertvalue %struct.nested %struct, {i8, i32} %smallstruct, 1
   store %struct.nested %res, %struct.nested* %addr
   ret void
 }
 
 ; CHECK-LABEL: name: test_select
 ; CHECK: [[TST_C:%[0-9]+]]:_(s32) = COPY $w0
-; CHECK: [[TST:%[0-9]+]]:_(s1) = G_TRUNC [[TST_C]]
+; CHECK: [[TSTEXT:%[0-9]+]]:_(s8) = G_TRUNC [[TST_C]]
 ; CHECK: [[LHS:%[0-9]+]]:_(s32) = COPY $w1
 ; CHECK: [[RHS:%[0-9]+]]:_(s32) = COPY $w2
+; CHECK: [[TSTASSERT:%[0-9]+]]:_(s8) = G_ASSERT_ZEXT [[TSTEXT]], 1
+; CHECK: [[TST:%[0-9]+]]:_(s1) = G_TRUNC [[TSTASSERT]]
 ; CHECK: [[RES:%[0-9]+]]:_(s32) = G_SELECT [[TST]](s1), [[LHS]], [[RHS]]
 ; CHECK: $w0 = COPY [[RES]]
 define i32 @test_select(i1 %tst, i32 %lhs, i32 %rhs) {
   %res = select i1 %tst, i32 %lhs, i32 %rhs
   ret i32 %res
 }
 
 ; CHECK-LABEL: name: test_select_flags
 ; CHECK:   [[COPY:%[0-9]+]]:_(s32) = COPY $w0
-; CHECK:   [[TRUNC:%[0-9]+]]:_(s1) = G_TRUNC [[COPY]](s32)
+; CHECK:   [[TRUNC8:%[0-9]+]]:_(s8) = G_TRUNC [[COPY]]
 ; CHECK:   [[COPY1:%[0-9]+]]:_(s32) = COPY $s0
 ; CHECK:   [[COPY2:%[0-9]+]]:_(s32) = COPY $s1
+; CHECK:   [[TRUNCASSERT:%[0-9]+]]:_(s8) = G_ASSERT_ZEXT [[TRUNC8]], 1
+; CHECK:   [[TRUNC:%[0-9]+]]:_(s1) = G_TRUNC [[TRUNCASSERT]]
 ; CHECK:   [[SELECT:%[0-9]+]]:_(s32) = nnan G_SELECT [[TRUNC]](s1), [[COPY1]], [[COPY2]]
 define float @test_select_flags(i1 %tst, float %lhs, float %rhs) {
   %res = select nnan i1 %tst, float %lhs, float %rhs
   ret float %res
 }
 
 ; Don't take the flags from the compare condition
 ; CHECK-LABEL: name: test_select_cmp_flags
 ; CHECK:   [[COPY0:%[0-9]+]]:_(s32) = COPY $s0
 ; CHECK:   [[COPY1:%[0-9]+]]:_(s32) = COPY $s1
 ; CHECK:   [[COPY2:%[0-9]+]]:_(s32) = COPY $s2
 ; CHECK:   [[COPY3:%[0-9]+]]:_(s32) = COPY $s3
 ; CHECK:   [[CMP:%[0-9]+]]:_(s1) = nsz G_FCMP floatpred(oeq), [[COPY0]](s32), [[COPY1]]
 ; CHECK:   [[SELECT:%[0-9]+]]:_(s32) = G_SELECT [[CMP]](s1), [[COPY2]], [[COPY3]]
 define float @test_select_cmp_flags(float %cmp0, float %cmp1, float %lhs, float %rhs) {
   %tst = fcmp nsz oeq float %cmp0, %cmp1
   %res = select i1 %tst, float %lhs, float %rhs
   ret float %res
 }
 
 ; CHECK-LABEL: name: test_select_ptr
 ; CHECK: [[TST_C:%[0-9]+]]:_(s32) = COPY $w0
-; CHECK: [[TST:%[0-9]+]]:_(s1) = G_TRUNC [[TST_C]]
+; CHECK: [[TSTEXT:%[0-9]+]]:_(s8) = G_TRUNC [[TST_C]]
 ; CHECK: [[LHS:%[0-9]+]]:_(p0) = COPY $x1
 ; CHECK: [[RHS:%[0-9]+]]:_(p0) = COPY $x2
+; CHECK: [[TSTASSERT:%[0-9]+]]:_(s8) = G_ASSERT_ZEXT [[TSTEXT]], 1
+; CHECK: [[TST:%[0-9]+]]:_(s1) = G_TRUNC [[TSTASSERT]]
 ; CHECK: [[RES:%[0-9]+]]:_(p0) = G_SELECT [[TST]](s1), [[LHS]], [[RHS]]
 ; CHECK: $x0 = COPY [[RES]]
 define i8* @test_select_ptr(i1 %tst, i8* %lhs, i8* %rhs) {
   %res = select i1 %tst, i8* %lhs, i8* %rhs
   ret i8* %res
 }
 
 ; CHECK-LABEL: name: test_select_vec
 ; CHECK: [[TST_C:%[0-9]+]]:_(s32) = COPY $w0
-; CHECK: [[TST:%[0-9]+]]:_(s1) = G_TRUNC [[TST_C]]
+; CHECK: [[TSTEXT:%[0-9]+]]:_(s8) = G_TRUNC [[TST_C]]
 ; CHECK: [[LHS:%[0-9]+]]:_(<4 x s32>) = COPY $q0
 ; CHECK: [[RHS:%[0-9]+]]:_(<4 x s32>) = COPY $q1
+; CHECK: [[TSTASSERT:%[0-9]+]]:_(s8) = G_ASSERT_ZEXT [[TSTEXT]], 1
+; CHECK: [[TST:%[0-9]+]]:_(s1) = G_TRUNC [[TSTASSERT]]
 ; CHECK: [[RES:%[0-9]+]]:_(<4 x s32>) = G_SELECT [[TST]](s1), [[LHS]], [[RHS]]
 ; CHECK: $q0 = COPY [[RES]]
 define <4 x i32> @test_select_vec(i1 %tst, <4 x i32> %lhs, <4 x i32> %rhs) {
   %res = select i1 %tst, <4 x i32> %lhs, <4 x i32> %rhs
   ret <4 x i32> %res
 }
 
 ; CHECK-LABEL: name: test_vselect_vec
exit:
   ret i64 %res
 }
 
 define void @test_phi_diamond({ i8, i16, i32 }* %a.ptr, { i8, i16, i32 }* %b.ptr, i1 %selector, { i8, i16, i32 }* %dst) {
 ; CHECK-LABEL: name: test_phi_diamond
 ; CHECK: [[ARG1:%[0-9]+]]:_(p0) = COPY $x0
 ; CHECK: [[ARG2:%[0-9]+]]:_(p0) = COPY $x1
 ; CHECK: [[ARG3:%[0-9]+]]:_(s32) = COPY $w2
-; CHECK: [[TRUNC:%[0-9]+]]:_(s1) = G_TRUNC [[ARG3]](s32)
+; CHECK: [[TRUNC8:%[0-9]+]]:_(s8) = G_TRUNC [[ARG3]]
 ; CHECK: [[ARG4:%[0-9]+]]:_(p0) = COPY $x3
+; CHECK: [[TRUNCASSERT:%[0-9]+]]:_(s8) = G_ASSERT_ZEXT [[TRUNC8]], 1
+; CHECK: [[TRUNC:%[0-9]+]]:_(s1) = G_TRUNC [[TRUNCASSERT]]
 ; CHECK: G_BRCOND [[TRUNC]](s1), %bb.2
 ; CHECK: G_BR %bb.3
 
 ; CHECK: [[LD1:%[0-9]+]]:_(s8) = G_LOAD [[ARG1]](p0) :: (load (s8) from %ir.a.ptr, align 4)
 ; CHECK: [[CST1:%[0-9]+]]:_(s64) = G_CONSTANT i64 2
 ; CHECK: [[GEP1:%[0-9]+]]:_(p0) = G_PTR_ADD [[ARG1]], [[CST1]](s64)
 ; CHECK: [[LD2:%[0-9]+]]:_(s16) = G_LOAD [[GEP1]](p0) :: (load (s16) from %ir.a.ptr + 2)
 ; CHECK: [[CST2:%[0-9]+]]:_(s64) = G_CONSTANT i64 4

 }
 
 declare { <4 x i32>, <4 x i32>, <4 x i32> } @llvm.aarch64.neon.ld3.v4i32.p0i32(i32*) #3
 
 define void @test_i1_arg_zext(void (i1)* %f) {
 ; CHECK-LABEL: name: test_i1_arg_zext
 ; CHECK: [[I1:%[0-9]+]]:_(s1) = G_CONSTANT i1 true
 ; CHECK: [[ZEXT0:%[0-9]+]]:_(s8) = G_ZEXT [[I1]](s1)
-; CHECK: [[ZEXT1:%[0-9]+]]:_(s32) = G_ZEXT [[ZEXT0]](s8)
+; CHECK: [[ZEXT1:%[0-9]+]]:_(s32) = G_ANYEXT [[ZEXT0]](s8)
 ; CHECK: $w0 = COPY [[ZEXT1]](s32)
   call void %f(i1 true)
   ret void
 }
 
 declare i8* @llvm.stacksave()
 declare void @llvm.stackrestore(i8*)
 define void @test_stacksaverestore() {

llvm/test/CodeGen/AArch64/GlobalISel/call-lowering-signext.ll

define i32 @signext_param_stack(i64 %a, i64 %b, i64 %c, i64 %d, i64 %e, i64 %f,
   ; CHECK:   [[COPY5:%[0-9]+]]:_(s64) = COPY $x5
   ; CHECK:   [[COPY6:%[0-9]+]]:_(s64) = COPY $x6
   ; CHECK:   [[COPY7:%[0-9]+]]:_(s64) = COPY $x7
   ; CHECK:   [[FRAME_INDEX:%[0-9]+]]:_(p0) = G_FRAME_INDEX %fixed-stack.1
   ; CHECK:   [[LOAD:%[0-9]+]]:_(s64) = G_LOAD [[FRAME_INDEX]](p0) :: (invariant load (s64) from %fixed-stack.1, align 16)
   ; CHECK:   [[FRAME_INDEX1:%[0-9]+]]:_(p0) = G_FRAME_INDEX %fixed-stack.0
   ; CHECK:   [[SEXTLOAD:%[0-9]+]]:_(s32) = G_SEXTLOAD [[FRAME_INDEX1]](p0) :: (invariant load (s8) from %fixed-stack.0, align 8)
   ; CHECK:   [[ASSERT_SEXT:%[0-9]+]]:_(s32) = G_ASSERT_SEXT [[SEXTLOAD]], 1
-  ; CHECK:   [[TRUNC:%[0-9]+]]:_(s1) = G_TRUNC [[ASSERT_SEXT]](s32)
-  ; CHECK:   [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[TRUNC]](s1)
+  ; CHECK:   [[TRUNC:%[0-9]+]]:_(s8) = G_TRUNC [[ASSERT_SEXT]](s32)
+  ; CHECK:   [[ASSERT_ZEXT:%[0-9]+]]:_(s8) = G_ASSERT_ZEXT [[TRUNC]], 1
+  ; CHECK:   [[TRUNC1:%[0-9]+]]:_(s1) = G_TRUNC [[ASSERT_ZEXT]](s8)
+  ; CHECK:   [[ZEXT:%[0-9]+]]:_(s32) = G_ZEXT [[TRUNC1]](s1)
   ; CHECK:   $w0 = COPY [[ZEXT]](s32)
   ; CHECK:   RET_ReallyLR implicit $w0
                                 i64 %g, i64 %h, i64 %i, i1 signext %j) {
   %v = zext i1 %j to i32
   ret i32 %v
 }
 
 ; The signext parameter is a s32, so there's no extension required.

llvm/test/CodeGen/AArch64/GlobalISel/call-translator.ll

 }
 
 ; CHECK-LABEL: name: test_mem_i1
 ; CHECK: fixedStack:
 ; CHECK-NEXT: - { id: [[SLOT:[0-9]+]], type: default, offset: 0, size: 1, alignment: 16, stack-id: default,
 ; CHECK-NEXT: isImmutable: true,
 ; CHECK: [[ADDR:%[0-9]+]]:_(p0) = G_FRAME_INDEX %fixed-stack.[[SLOT]]
 ; CHECK: [[LOAD:%[0-9]+]]:_(s32) = G_LOAD [[ADDR]](p0) :: (invariant load (s8) from %fixed-stack.[[SLOT]], align 16)
-; CHECK-NEXT: {{%[0-9]+}}:_(s1) = G_TRUNC [[LOAD]]
+; CHECK: [[TRUNC8:%[0-9]+]]:_(s8) = G_TRUNC [[LOAD]]
+; CHECK: [[TRUNCASSERT:%[0-9]+]]:_(s8) = G_ASSERT_ZEXT [[TRUNC8]], 1
+; CHECK: {{%[0-9]+}}:_(s1) = G_TRUNC [[TRUNCASSERT]]
+
 define void @test_mem_i1([8 x i64], i1 %in) {
   ret void
 }
 
 ; CHECK-LABEL: name: test_128bit_struct
 ; CHECK: $x0 = COPY
 ; CHECK: $x1 = COPY
 ; CHECK: $x2 = COPY

llvm/test/CodeGen/AArch64/arm64-aapcs.ll

 define dso_local void @test_stack_slots([8 x i64], i1 %bool, i8 %char, i16 %short,
                                 i32 %int, i64 %long) {
 ; CHECK-LABEL: test_stack_slots:
 ; CHECK-DAG: ldr w[[ext1:[0-9]+]], [sp, #24]
 ; CHECK-DAG: ldrh w[[ext2:[0-9]+]], [sp, #16]
 ; CHECK-DAG: ldrb w[[ext3:[0-9]+]], [sp, #8]
 ; CHECK-DAG: ldr x[[ext4:[0-9]+]], [sp, #32]
 ; CHECK-DAG: ldrb w[[ext5:[0-9]+]], [sp]
-; CHECK-DAG: and x[[ext5]], x[[ext5]], #0x1
 
   %ext_bool = zext i1 %bool to i64
   store volatile i64 %ext_bool, i64* @var64, align 8
 ; CHECK: str x[[ext5]], [{{x[0-9]+}}, :lo12:var64]
 
   %ext_char = zext i8 %char to i64
   store volatile i64 %ext_char, i64* @var64, align 8
 ; CHECK: str x[[ext3]], [{{x[0-9]+}}, :lo12:var64]

llvm/test/CodeGen/AArch64/i1-contents.ll

-; RUN: llc -mtriple=aarch64-linux-gnu -o - %s | FileCheck %s
+; RUN: llc -mtriple=aarch64-linux-gnu -o - %s | FileCheck %s --check-prefixes CHECK,CHECK-SDAG
+; RUN: llc -global-isel -mtriple=aarch64-linux-gnu -o - %s | FileCheck %s --check-prefixes CHECK,CHECK-GISEL
 %big = type i32
 
 @var = dso_local global %big 0
 
 ; AAPCS: low 8 bits of %in (== w0) will be either 0 or 1. Need to extend to
 ; 32-bits.
 define dso_local void @consume_i1_arg(i1 %in) {
 ; CHECK-LABEL: consume_i1_arg:

 ; CHECK: bl consume_i1_arg
   %val = load %big, %big* @var
   %val1 = trunc %big %val to i1
   call void @consume_i1_arg(i1 %val1)
   ret void
 }
 
 
+define dso_local void @forward_i1_arg1(i1 %in) {
+; CHECK-LABEL: forward_i1_arg1:
+; CHECK-NOT: and
+; CHECK: bl consume_i1_arg
+  call void @consume_i1_arg(i1 %in)
+  ret void
+}
+
+define dso_local void @forward_i1_arg2(i1 %in, i1 %cond) {
+; CHECK-LABEL: forward_i1_arg2:
+;
+; The optimization in SelectionDAG currently fails to recognize that
+; %in is already zero-extended to i8 if the call is not in the entry
+; block.
+;
+; CHECK-SDAG: and
+; CHECK-GISEL-NOT: and
+;
+; CHECK: bl consume_i1_arg
+  br i1 %cond, label %true, label %false
+true:
+  call void @consume_i1_arg(i1 %in)
+  ret void
+
+false:
+  ret void
+}
+
 ;define zeroext i1 @foo(i8 %in) {
 ;  %val = trunc i8 %in to i1
 ;  ret i1 %val
 ;}