[GISel]: Fix generation of illegal COPYs (of different sizes) during CallLowering

[aditya_nandakumar]

Summary

We end up creating COPY's that are either truncating/extending and this should be illegal.
Eg %0(s8) = COPY %w0
Instead, we generate TRUNCs/ANY_EXT and let the optimizer deal with it later.
%tmp(s32) = COPY %w0
%0(s8) = G_TRUNC %tmp(s32)

I've updated the AArch64 backend, I'd appreciate it if people more familiar with X86/ARM can update it as well(post feedback).

Looking forward to your comments.

[igorb]

X86 backend part - https://reviews.llvm.org/D37678
Could you please merge into you patch.
Regards.

[qcolombet]

Hi Aditya,

Could you add a check in the verifier too?

Cheers,
-Quentin

[aditya_nandakumar]

Added verifier check for catching simple cases (non subreg indices for now).

[aditya_nandakumar]

Adding the verifier check has resulted in 5, 12, and 43 failures in ARM, AArch64, and X86 respectively which needs to be fixed.

[aditya_nandakumar]

Splitting of the verifier check into https://reviews.llvm.org/D37775.

[qcolombet]

The AArch64 part looks good to me.

include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h

549

You can split the patch for those, when you commit.

670

Ditto.

lib/CodeGen/MachineVerifier.cpp

52 ↗

(On Diff #114664)

This change is problematic because we add a dependency from CodeGen to GlobalISel

953 ↗

(On Diff #114664)

Using the type of the vregs, MachineRegisterInfo::getType, should eliminate the dependency on RegisterBankInfo.

[aditya_nandakumar]

lib/CodeGen/MachineVerifier.cpp

953 ↗

(On Diff #114664)

This part has moved to https://reviews.llvm.org/D37775
MRI->getType would only give me valid types for generic virtual registers.
For cases such as %0(s16) = COPY %W0, I'd still need to get the size of W0. If dependency is the issue, I can certainly do what RegisterBankInfo::getSizeInBits() is doing, but that's duplicating work.

[qcolombet]

lib/CodeGen/MachineVerifier.cpp

953 ↗

(On Diff #114664)

I would push the interesting part of getSizeInBits from RegisterBankInfo to TargetRegisterInfo and have RegisterBankInfo::getSizeInBits uses TargetRegisterInfo::getSizeInBits to avoid the duplication while having the necessary functionality available for the verifier.

[aditya_nandakumar]

lib/CodeGen/MachineVerifier.cpp

953 ↗

(On Diff #114664)

Quentin - I refactored getSizeInBits in the other review.

[aditya_nandakumar]

Merged X86 part from https://reviews.llvm.org/D37678
Thanks Igor.

[aditya_nandakumar]

Updated ARM-irtranslator.ll test to account for extendRegister emitting G_ANYEXT.

Now, all of the backends pass tests - but LowerFormalArguments still generates wrong (truncating) copies. This will likely be an issue when https://reviews.llvm.org/D37775 lands
Could someone familiar with ARM codebase implement that?

[rovka]

In D37640#886384, @aditya_nandakumar wrote:

Could someone familiar with ARM codebase implement that?

I'll look into it.

[rovka]

Hi Aditya,

I uploaded the ARM changes here: https://reviews.llvm.org/differential/diff/118037/

As a sidenote, this caused trouble with some of my integration tests because now the legalize combiner finds all sorts of opportunities to replace trunc + ext with shifts, which weren't implemented on ARM yet (they are now). I don't think the combiner should be introducing instructions without first checking if they are legal, maybe you can consider fixing that in the future.

[aditya_nandakumar]

Thanks for the patch Diana.
Also regarding the side node, I'll be looking into that soon.

[aditya_nandakumar]

Updated with Diana's patch for ARM.

[aditya_nandakumar]

Can I get an LGTM for this if there are no other comments/concerns?

[qcolombet]

LGTM

[aditya_nandakumar]

Committed in r315240

include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h

public:
   /// Build and insert \p Res<def> = COPY Op
   ///
   /// Register-to-register COPY sets \p Res to \p Op.
   ///
   /// \pre setBasicBlock or setMI must have been called.
   ///
   /// \return a MachineInstrBuilder for the newly created instruction.
   MachineInstrBuilder buildCopy(unsigned Res, unsigned Op);
+  template <typename DstType, typename SrcType>
+  MachineInstrBuilder buildCopy(DstType &&Res, SrcType &&Src) {
+    return buildCopy(getDestFromArg(Res), getRegFromArg(Src));
+  }
 
   /// Build and insert `Res<def> = G_LOAD Addr, MMO`.
   ///
   /// Loads the value stored at \p Addr. Puts the result in \p Res.
   ///
   /// \pre setBasicBlock or setMI must have been called.
   /// \pre \p Res must be a generic virtual register.
   /// \pre \p Addr must be a generic virtual register with pointer type.
public:
   ///
   /// \pre setBasicBlock or setMI must have been called.
   /// \pre \p Res must be a generic virtual register with scalar or vector type.
   /// \pre \p Op must be a generic virtual register with scalar or vector type.
   /// \pre \p Res must be smaller than \p Op
   ///
   /// \return The newly created instruction.
   MachineInstrBuilder buildTrunc(unsigned Res, unsigned Op);
+  template <typename DstType, typename SrcType>
+  MachineInstrBuilder buildTrunc(DstType &&Res, SrcType &&Src) {
+    return buildTrunc(getDestFromArg(Res), getRegFromArg(Src));
+  }
 
   /// Build and insert a \p Res = G_ICMP \p Pred, \p Op0, \p Op1
   ///
   /// \pre setBasicBlock or setMI must have been called.
 
   /// \pre \p Res must be a generic virtual register with scalar or
   ///      vector type. Typically this starts as s1 or <N x s1>.
   /// \pre \p Op0 and Op1 must be generic virtual registers with the

lib/CodeGen/GlobalISel/CallLowering.cpp

unsigned CallLowering::ValueHandler::extendRegister(unsigned ValReg,
   LLT LocTy{VA.getLocVT()};
   switch (VA.getLocInfo()) {
   default: break;
   case CCValAssign::Full:
   case CCValAssign::BCvt:
     // FIXME: bitconverting between vector types may or may not be a
     // nop in big-endian situations.
     return ValReg;
-  case CCValAssign::AExt:
+  case CCValAssign::AExt: {
     assert(!VA.getLocVT().isVector() && "unexpected vector extend");
-    // Otherwise, it's a nop.
-    return ValReg;
+    auto MIB = MIRBuilder.buildAnyExt(LocTy, ValReg);
+    return MIB->getOperand(0).getReg();
+  }
   case CCValAssign::SExt: {
     unsigned NewReg = MRI.createGenericVirtualRegister(LocTy);
     MIRBuilder.buildSExt(NewReg, ValReg);
     return NewReg;
   }
   case CCValAssign::ZExt: {
     unsigned NewReg = MRI.createGenericVirtualRegister(LocTy);
     MIRBuilder.buildZExt(NewReg, ValReg);
     return NewReg;
   }
   }
   llvm_unreachable("unable to extend register");
 }

lib/Target/AArch64/AArch64CallLowering.cpp

unsigned getStackAddress(uint64_t Size, int64_t Offset,
     MIRBuilder.buildFrameIndex(AddrReg, FI);
     StackUsed = std::max(StackUsed, Size + Offset);
     return AddrReg;
   }
 
   void assignValueToReg(unsigned ValVReg, unsigned PhysReg,
                         CCValAssign &VA) override {
     markPhysRegUsed(PhysReg);
+    switch (VA.getLocInfo()) {
+    default:
       MIRBuilder.buildCopy(ValVReg, PhysReg);
-    // FIXME: assert extension
+      break;
+    case CCValAssign::LocInfo::SExt:
+    case CCValAssign::LocInfo::ZExt:
+    case CCValAssign::LocInfo::AExt: {
+      auto Copy = MIRBuilder.buildCopy(LLT{VA.getLocVT()}, PhysReg);
+      MIRBuilder.buildTrunc(ValVReg, Copy);
+      break;
+    }
+    }
   }
 
   void assignValueToAddress(unsigned ValVReg, unsigned Addr, uint64_t Size,
                             MachinePointerInfo &MPO, CCValAssign &VA) override {
     auto MMO = MIRBuilder.getMF().getMachineMemOperand(
         MPO, MachineMemOperand::MOLoad | MachineMemOperand::MOInvariant, Size,
         0);
     MIRBuilder.buildLoad(ValVReg, Addr, *MMO);

lib/Target/X86/X86CallLowering.cpp

void assignValueToAddress(unsigned ValVReg, unsigned Addr, uint64_t Size,
                             MachinePointerInfo &MPO, CCValAssign &VA) override {
 
     auto MMO = MIRBuilder.getMF().getMachineMemOperand(
         MPO, MachineMemOperand::MOLoad | MachineMemOperand::MOInvariant, Size,
         0);
     MIRBuilder.buildLoad(ValVReg, Addr, *MMO);
   }
 
+  void assignValueToReg(unsigned ValVReg, unsigned PhysReg,
+                        CCValAssign &VA) override {
+    markPhysRegUsed(PhysReg);
+    switch (VA.getLocInfo()) {
+    default:
+      MIRBuilder.buildCopy(ValVReg, PhysReg);
+      break;
+    case CCValAssign::LocInfo::SExt:
+    case CCValAssign::LocInfo::ZExt:
+    case CCValAssign::LocInfo::AExt: {
+      auto Copy = MIRBuilder.buildCopy(LLT{VA.getLocVT()}, PhysReg);
+      MIRBuilder.buildTrunc(ValVReg, Copy);
+      break;
+    }
+    }
+  }
+
+  /// How the physical register gets marked varies between formal
+  /// parameters (it's a basic-block live-in), and a call instruction
+  /// (it's an implicit-def of the BL).
+  virtual void markPhysRegUsed(unsigned PhysReg) = 0;
+
 protected:
   const DataLayout &DL;
 };
 
 struct FormalArgHandler : public IncomingValueHandler {
   FormalArgHandler(MachineIRBuilder &MIRBuilder, MachineRegisterInfo &MRI,
                    CCAssignFn *AssignFn)
       : IncomingValueHandler(MIRBuilder, MRI, AssignFn) {}
 
-  void assignValueToReg(unsigned ValVReg, unsigned PhysReg,
-                        CCValAssign &VA) override {
+  void markPhysRegUsed(unsigned PhysReg) override {
     MIRBuilder.getMBB().addLiveIn(PhysReg);
-    MIRBuilder.buildCopy(ValVReg, PhysReg);
   }
 };
 
 struct CallReturnHandler : public IncomingValueHandler {
   CallReturnHandler(MachineIRBuilder &MIRBuilder, MachineRegisterInfo &MRI,
                     CCAssignFn *AssignFn, MachineInstrBuilder &MIB)
       : IncomingValueHandler(MIRBuilder, MRI, AssignFn), MIB(MIB) {}
 
-  void assignValueToReg(unsigned ValVReg, unsigned PhysReg,
-                        CCValAssign &VA) override {
+  void markPhysRegUsed(unsigned PhysReg) override {
     MIB.addDef(PhysReg, RegState::Implicit);
-    MIRBuilder.buildCopy(ValVReg, PhysReg);
   }
 
 protected:
   MachineInstrBuilder &MIB;
 };
 
 } // namespace
 

test/CodeGen/AArch64/GlobalISel/arm64-callingconv.ll

 ; CHECK: [[ONE:%[0-9]+]](s16) = G_CONSTANT i16 1
 ; CHECK: [[FOUR:%[0-9]+]](s32) = G_CONSTANT i32 4
 ; CHECK: [[F_ONE:%[0-9]+]](s32) = G_FCONSTANT float 1.000000e+00
 ; CHECK: [[TWO:%[0-9]+]](s64) = G_FCONSTANT double 2.000000e+00
 
 ; CHECK: %w0 = COPY [[ANSWER]]
 ; CHECK: %d0 = COPY [[D_ONE]]
 ; CHECK: %x1 = COPY [[TWELVE]]
-; CHECK: %w2 = COPY [[THREE]](s8)
-; CHECK: %w3 = COPY [[ONE]](s16)
+; CHECK: [[THREE_TMP:%[0-9]+]](s32) = G_ANYEXT [[THREE]]
+; CHECK: %w2 = COPY [[THREE_TMP]](s32)
+; CHECK: [[ONE_TMP:%[0-9]+]](s32) = G_ANYEXT [[ONE]]
+; CHECK: %w3 = COPY [[ONE_TMP]](s32)
 ; CHECK: %w4 = COPY [[FOUR]](s32)
 ; CHECK: %s1 = COPY [[F_ONE]](s32)
 ; CHECK: %d2 = COPY [[TWO]](s64)
 declare void @varargs(i32, double, i64, ...)
 define void @test_varargs() {
   call void(i32, double, i64, ...) @varargs(i32 42, double 1.0, i64 12, i8 3, i16 1, i32 4, float 1.0, double 2.0)
   ret void
 }

test/CodeGen/AArch64/GlobalISel/arm64-fallback.ll

 ; FALLBACK-WITH-REPORT-OUT: fmov d0, #1.0
 define [1 x double] @constant() {
   ret [1 x double] [double 1.0]
 }
 
   ; The key problem here is that we may fail to create an MBB referenced by a
   ; PHI. If so, we cannot complete the G_PHI and mustn't try or bad things
   ; happen.
-; FALLBACK-WITH-REPORT-ERR: remark: <unknown>:0:0: cannot select: G_STORE %vreg5, %vreg2; mem:ST4[%addr] GPR:%vreg5,%vreg2 (in function: pending_phis)
+; FALLBACK-WITH-REPORT-ERR: remark: <unknown>:0:0: cannot select: G_STORE %vreg6, %vreg2; mem:ST4[%addr] GPR:%vreg6,%vreg2 (in function: pending_phis)
 ; FALLBACK-WITH-REPORT-ERR: warning: Instruction selection used fallback path for pending_phis
 ; FALLBACK-WITH-REPORT-OUT-LABEL: pending_phis:
 define i32 @pending_phis(i1 %tst, i32 %val, i32* %addr) {
   br i1 %tst, label %true, label %false
 
 end:
   %res = phi i32 [%val, %true], [42, %false]
   ret i32 %res

test/CodeGen/AArch64/GlobalISel/arm64-irtranslator.ll

define void @test_insertvalue_agg(%struct.nested* %addr, {i8, i32}* %addr2) {
   %smallstruct = load {i8, i32}, {i8, i32}* %addr2
   %struct = load %struct.nested, %struct.nested* %addr
   %res = insertvalue %struct.nested %struct, {i8, i32} %smallstruct, 1
   store %struct.nested %res, %struct.nested* %addr
   ret void
 }
 
 ; CHECK-LABEL: name: test_select
-; CHECK: [[TST:%[0-9]+]](s1) = COPY %w0
+; CHECK: [[TST_C:%[0-9]+]](s32) = COPY %w0
+; CHECK: [[TST:%[0-9]+]](s1) = G_TRUNC [[TST_C]]
 ; CHECK: [[LHS:%[0-9]+]](s32) = COPY %w1
 ; CHECK: [[RHS:%[0-9]+]](s32) = COPY %w2
 ; CHECK: [[RES:%[0-9]+]](s32) = G_SELECT [[TST]](s1), [[LHS]], [[RHS]]
 ; CHECK: %w0 = COPY [[RES]]
 define i32 @test_select(i1 %tst, i32 %lhs, i32 %rhs) {
   %res = select i1 %tst, i32 %lhs, i32 %rhs
   ret i32 %res
 }
 
 ; CHECK-LABEL: name: test_select_ptr
-; CHECK: [[TST:%[0-9]+]](s1) = COPY %w0
+; CHECK: [[TST_C:%[0-9]+]](s32) = COPY %w0
+; CHECK: [[TST:%[0-9]+]](s1) = G_TRUNC [[TST_C]]
 ; CHECK: [[LHS:%[0-9]+]](p0) = COPY %x1
 ; CHECK: [[RHS:%[0-9]+]](p0) = COPY %x2
 ; CHECK: [[RES:%[0-9]+]](p0) = G_SELECT [[TST]](s1), [[LHS]], [[RHS]]
 ; CHECK: %x0 = COPY [[RES]]
 define i8* @test_select_ptr(i1 %tst, i8* %lhs, i8* %rhs) {
   %res = select i1 %tst, i8* %lhs, i8* %rhs
   ret i8* %res
 }
 
 ; CHECK-LABEL: name: test_select_vec
-; CHECK: [[TST:%[0-9]+]](s1) = COPY %w0
+; CHECK: [[TST_C:%[0-9]+]](s32) = COPY %w0
+; CHECK: [[TST:%[0-9]+]](s1) = G_TRUNC [[TST_C]]
 ; CHECK: [[LHS:%[0-9]+]](<4 x s32>) = COPY %q0
 ; CHECK: [[RHS:%[0-9]+]](<4 x s32>) = COPY %q1
 ; CHECK: [[RES:%[0-9]+]](<4 x s32>) = G_SELECT [[TST]](s1), [[LHS]], [[RHS]]
 ; CHECK: %q0 = COPY [[RES]]
 define <4 x i32> @test_select_vec(i1 %tst, <4 x i32> %lhs, <4 x i32> %rhs) {
   %res = select i1 %tst, <4 x i32> %lhs, <4 x i32> %rhs
   ret <4 x i32> %res
 }
; CHECK: BL $memmove, csr_aarch64_aapcs, implicit-def %lr, implicit %sp, implicit %x0, implicit %x1, implicit %x2
   call void @llvm.memmove.p0i8.p0i8.i64(i8* %dst, i8* %src, i64 %size, i32 1, i1 0)
   ret void
 }
 
 declare void @llvm.memset.p0i8.i64(i8*, i8, i64, i32 %align, i1 %volatile)
 define void @test_memset(i8* %dst, i8 %val, i64 %size) {
 ; CHECK-LABEL: name: test_memset
 ; CHECK: [[DST:%[0-9]+]](p0) = COPY %x0
-; CHECK: [[SRC:%[0-9]+]](s8) = COPY %w1
+; CHECK: [[SRC_C:%[0-9]+]](s32) = COPY %w1
+; CHECK: [[SRC:%[0-9]+]](s8) = G_TRUNC [[SRC_C]]
 ; CHECK: [[SIZE:%[0-9]+]](s64) = COPY %x2
 ; CHECK: %x0 = COPY [[DST]]
-; CHECK: %w1 = COPY [[SRC]]
+; CHECK: [[SRC_TMP:%[0-9]+]](s32) = G_ANYEXT [[SRC]]
+; CHECK: %w1 = COPY [[SRC_TMP]]
 ; CHECK: %x2 = COPY [[SIZE]]
 ; CHECK: BL $memset, csr_aarch64_aapcs, implicit-def %lr, implicit %sp, implicit %x0, implicit %w1, implicit %x2
   call void @llvm.memset.p0i8.i64(i8* %dst, i8 %val, i64 %size, i32 1, i1 0)
   ret void
 }
 
 declare i64 @llvm.objectsize.i64(i8*, i1)
 declare i32 @llvm.objectsize.i32(i8*, i1)

test/CodeGen/AArch64/GlobalISel/call-translator.ll

define void @test_multiple_args(i64 %in) {
   call void @multiple_args_callee(i32 42, i64 %in)
   ret void
 }
 
 
 ; CHECK-LABEL: name: test_struct_formal
 ; CHECK: [[DBL:%[0-9]+]](s64) = COPY %d0
 ; CHECK: [[I64:%[0-9]+]](s64) = COPY %x0
-; CHECK: [[I8:%[0-9]+]](s8) = COPY %w1
+; CHECK: [[I8_C:%[0-9]+]](s32) = COPY %w1
+; CHECK: [[I8:%[0-9]+]](s8) = G_TRUNC [[I8_C]]
 ; CHECK: [[ADDR:%[0-9]+]](p0) = COPY %x2
 
 ; CHECK: [[UNDEF:%[0-9]+]](s192) = G_IMPLICIT_DEF
 ; CHECK: [[ARG0:%[0-9]+]](s192) = G_INSERT [[UNDEF]], [[DBL]](s64), 0
 ; CHECK: [[ARG1:%[0-9]+]](s192) = G_INSERT [[ARG0]], [[I64]](s64), 64
 ; CHECK: [[ARG2:%[0-9]+]](s192) = G_INSERT [[ARG1]], [[I8]](s8), 128
 ; CHECK: [[ARG:%[0-9]+]](s192) = COPY [[ARG2]]
 
define i64 @test_arr_call([4 x i64]* %addr) {
   %res = call [4 x i64] @arr_callee([4 x i64] %arg)
   %val = extractvalue [4 x i64] %res, 1
   ret i64 %val
 }
 
 
 ; CHECK-LABEL: name: test_abi_exts_call
 ; CHECK: [[VAL:%[0-9]+]](s8) = G_LOAD
-; CHECK: %w0 = COPY [[VAL]]
+; CHECK: [[VAL_TMP:%[0-9]+]](s32) = G_ANYEXT [[VAL]]
+; CHECK: %w0 = COPY [[VAL_TMP]]
 ; CHECK: BL @take_char, csr_aarch64_aapcs, implicit-def %lr, implicit %sp, implicit %w0
 ; CHECK: [[SVAL:%[0-9]+]](s32) = G_SEXT [[VAL]](s8)
 ; CHECK: %w0 = COPY [[SVAL]](s32)
 ; CHECK: BL @take_char, csr_aarch64_aapcs, implicit-def %lr, implicit %sp, implicit %w0
 ; CHECK: [[ZVAL:%[0-9]+]](s32) = G_ZEXT [[VAL]](s8)
 ; CHECK: %w0 = COPY [[ZVAL]](s32)
 ; CHECK: BL @take_char, csr_aarch64_aapcs, implicit-def %lr, implicit %sp, implicit %w0
 declare void @take_char(i8)

test/CodeGen/ARM/GlobalISel/arm-irtranslator.ll

 }
 
 define i8 @test_add_i8(i8 %x, i8 %y) {
 ; CHECK-LABEL: name: test_add_i8
 ; CHECK: liveins: %r0, %r1
 ; CHECK-DAG: [[VREGX:%[0-9]+]](s8) = COPY %r0
 ; CHECK-DAG: [[VREGY:%[0-9]+]](s8) = COPY %r1
 ; CHECK: [[SUM:%[0-9]+]](s8) = G_ADD [[VREGX]], [[VREGY]]
-; CHECK: %r0 = COPY [[SUM]](s8)
+; CHECK: [[SUM_EXT:%[0-9]+]](s32) = G_ANYEXT [[SUM]]
+; CHECK: %r0 = COPY [[SUM_EXT]](s32)
 ; CHECK: BX_RET 14, _, implicit %r0
 entry:
   %sum = add i8 %x, %y
   ret i8 %sum
 }
 
 define i8 @test_sub_i8(i8 %x, i8 %y) {
 ; CHECK-LABEL: name: test_sub_i8
 ; CHECK: liveins: %r0, %r1
 ; CHECK-DAG: [[VREGX:%[0-9]+]](s8) = COPY %r0
 ; CHECK-DAG: [[VREGY:%[0-9]+]](s8) = COPY %r1
 ; CHECK: [[RES:%[0-9]+]](s8) = G_SUB [[VREGX]], [[VREGY]]
-; CHECK: %r0 = COPY [[RES]](s8)
+; CHECK: [[RES_EXT:%[0-9]+]](s32) = G_ANYEXT [[RES]]
+; CHECK: %r0 = COPY [[RES_EXT]](s32)
 ; CHECK: BX_RET 14, _, implicit %r0
 entry:
   %res = sub i8 %x, %y
   ret i8 %res
 }
 
 define signext i8 @test_return_sext_i8(i8 %x) {
 ; CHECK-LABEL: name: test_return_sext_i8
 ; CHECK: liveins: %r0
 ; CHECK: [[VREG:%[0-9]+]](s8) = COPY %r0
 ; CHECK: [[VREGEXT:%[0-9]+]](s32) = G_SEXT [[VREG]]
 ; CHECK: %r0 = COPY [[VREGEXT]](s32)
 ; CHECK: BX_RET 14, _, implicit %r0
 entry:
   ret i8 %x
 }
 
 define i16 @test_add_i16(i16 %x, i16 %y) {
 ; CHECK-LABEL: name: test_add_i16
 ; CHECK: liveins: %r0, %r1
 ; CHECK-DAG: [[VREGX:%[0-9]+]](s16) = COPY %r0
 ; CHECK-DAG: [[VREGY:%[0-9]+]](s16) = COPY %r1
 ; CHECK: [[SUM:%[0-9]+]](s16) = G_ADD [[VREGX]], [[VREGY]]
-; CHECK: %r0 = COPY [[SUM]](s16)
+; CHECK: [[SUM_EXT:%[0-9]+]](s32) = G_ANYEXT [[SUM]]
+; CHECK: %r0 = COPY [[SUM_EXT]](s32)
 ; CHECK: BX_RET 14, _, implicit %r0
 entry:
   %sum = add i16 %x, %y
   ret i16 %sum
 }
 
 define i16 @test_sub_i16(i16 %x, i16 %y) {
 ; CHECK-LABEL: name: test_sub_i16
 ; CHECK: liveins: %r0, %r1
 ; CHECK-DAG: [[VREGX:%[0-9]+]](s16) = COPY %r0
 ; CHECK-DAG: [[VREGY:%[0-9]+]](s16) = COPY %r1
 ; CHECK: [[RES:%[0-9]+]](s16) = G_SUB [[VREGX]], [[VREGY]]
-; CHECK: %r0 = COPY [[RES]](s16)
+; CHECK: [[RES_EXT:%[0-9]+]](s32) = G_ANYEXT [[RES]]
+; CHECK: %r0 = COPY [[RES_EXT]](s32)
 ; CHECK: BX_RET 14, _, implicit %r0
 entry:
   %res = sub i16 %x, %y
   ret i16 %res
 }
 
 define zeroext i16 @test_return_zext_i16(i16 %x) {
 ; CHECK-LABEL: name: test_return_zext_i16

 ; CHECK-DAG: id: [[P4:[0-9]]]{{.*}}offset: 0{{.*}}size: 1
 ; CHECK-DAG: id: [[P5:[0-9]]]{{.*}}offset: 4{{.*}}size: 2
 ; CHECK: liveins: %r0, %r1, %r2, %r3
 ; CHECK: [[VREGP1:%[0-9]+]](s16) = COPY %r1
 ; CHECK: [[FIP5:%[0-9]+]](p0) = G_FRAME_INDEX %fixed-stack.[[P5]]
 ; CHECK: [[VREGP5EXT:%[0-9]+]](s32) = G_LOAD [[FIP5]](p0){{.*}}load 4
 ; CHECK: [[VREGP5:%[0-9]+]](s16) = G_TRUNC [[VREGP5EXT]]
 ; CHECK: [[SUM:%[0-9]+]](s16) = G_ADD [[VREGP1]], [[VREGP5]]
-; CHECK: %r0 = COPY [[SUM]]
+; CHECK: [[SUM_EXT:%[0-9]+]](s32) = G_ANYEXT [[SUM]]
+; CHECK: %r0 = COPY [[SUM_EXT]](s32)
 ; CHECK: BX_RET 14, _, implicit %r0
 entry:
   %sum = add i16 %p1, %p5
   ret i16 %sum
 }
 
 define i8 @test_stack_args_zeroext(i32 %p0, i16 %p1, i8 %p2, i1 %p3,
                                     i8 zeroext %p4, i16 zeroext %p5) {
 ; CHECK-LABEL: name: test_stack_args_zeroext
 ; CHECK: fixedStack:
 ; CHECK-DAG: id: [[P4:[0-9]]]{{.*}}offset: 0{{.*}}size: 1
 ; CHECK-DAG: id: [[P5:[0-9]]]{{.*}}offset: 4{{.*}}size: 2
 ; CHECK: liveins: %r0, %r1, %r2, %r3
 ; CHECK: [[VREGP2:%[0-9]+]](s8) = COPY %r2
 ; CHECK: [[FIP4:%[0-9]+]](p0) = G_FRAME_INDEX %fixed-stack.[[P4]]
 ; CHECK: [[VREGP4EXT:%[0-9]+]](s32) = G_LOAD [[FIP4]](p0){{.*}}load 4
 ; CHECK: [[VREGP4:%[0-9]+]](s8) = G_TRUNC [[VREGP4EXT]]
 ; CHECK: [[SUM:%[0-9]+]](s8) = G_ADD [[VREGP2]], [[VREGP4]]
-; CHECK: %r0 = COPY [[SUM]]
+; CHECK: [[SUM_EXT:%[0-9]+]](s32) = G_ANYEXT [[SUM]]
+; CHECK: %r0 = COPY [[SUM_EXT]](s32)
 ; CHECK: BX_RET 14, _, implicit %r0
 entry:
   %sum = add i8 %p2, %p4
   ret i8 %sum
 }
 
 define i8 @test_stack_args_noext(i32 %p0, i16 %p1, i8 %p2, i1 %p3,
                                  i8 %p4, i16 %p5) {
 ; CHECK-LABEL: name: test_stack_args_noext
 ; CHECK: fixedStack:
 ; CHECK-DAG: id: [[P4:[0-9]]]{{.*}}offset: 0{{.*}}size: 1
 ; CHECK-DAG: id: [[P5:[0-9]]]{{.*}}offset: 4{{.*}}size: 2
 ; CHECK: liveins: %r0, %r1, %r2, %r3
 ; CHECK: [[VREGP2:%[0-9]+]](s8) = COPY %r2
 ; CHECK: [[FIP4:%[0-9]+]](p0) = G_FRAME_INDEX %fixed-stack.[[P4]]
 ; CHECK: [[VREGP4:%[0-9]+]](s8) = G_LOAD [[FIP4]](p0){{.*}}load 1
 ; CHECK: [[SUM:%[0-9]+]](s8) = G_ADD [[VREGP2]], [[VREGP4]]
-; CHECK: %r0 = COPY [[SUM]]
+; CHECK: [[SUM_EXT:%[0-9]+]](s32) = G_ANYEXT [[SUM]]
+; CHECK: %r0 = COPY [[SUM_EXT]](s32)
 ; CHECK: BX_RET 14, _, implicit %r0
 entry:
   %sum = add i8 %p2, %p4
   ret i8 %sum
 }
 
 define zeroext i16 @test_stack_args_extend_the_extended(i32 %p0, i16 %p1, i8 %p2, i1 %p3,
                                                         i8 signext %p4, i16 signext %p5) {

test/CodeGen/X86/GlobalISel/add-scalar.ll

 ; X32-NEXT:    retl
   %ret = add i32 %arg1, %arg2
   ret i32 %ret
 }
 
 define i16 @test_add_i16(i16 %arg1, i16 %arg2) {
 ; X64-LABEL: test_add_i16:
 ; X64:       # BB#0:
-; X64-NEXT:    # kill: %DI<def> %DI<kill> %RDI<def>
-; X64-NEXT:    # kill: %SI<def> %SI<kill> %RSI<def>
+; X64-NEXT:    # kill: %EDI<def> %EDI<kill> %RDI<def>
+; X64-NEXT:    # kill: %ESI<def> %ESI<kill> %RSI<def>
 ; X64-NEXT:    leal (%rsi,%rdi), %eax
 ; X64-NEXT:    # kill: %AX<def> %AX<kill> %EAX<kill>
 ; X64-NEXT:    retq
 ;
 ; X32-LABEL: test_add_i16:
 ; X32:       # BB#0:
 ; X32-NEXT:    movzwl 8(%esp), %eax
 ; X32-NEXT:    addw 4(%esp), %ax

test/CodeGen/X86/GlobalISel/callingconv.ll

 
 declare void @take_char(i8)
 define void @test_abi_exts_call(i8* %addr) {
 ; X32-LABEL: test_abi_exts_call:
 ; X32:       # BB#0:
 ; X32-NEXT:    pushl %ebx
 ; X32-NEXT:  .Lcfi7:
 ; X32-NEXT:    .cfi_def_cfa_offset 8
-; X32-NEXT:    subl $8, %esp
+; X32-NEXT:    pushl %esi
 ; X32-NEXT:  .Lcfi8:
-; X32-NEXT:    .cfi_def_cfa_offset 16
+; X32-NEXT:    .cfi_def_cfa_offset 12
+; X32-NEXT:    pushl %eax
 ; X32-NEXT:  .Lcfi9:
+; X32-NEXT:    .cfi_def_cfa_offset 16
+; X32-NEXT:  .Lcfi10:
+; X32-NEXT:    .cfi_offset %esi, -12
+; X32-NEXT:  .Lcfi11:
 ; X32-NEXT:    .cfi_offset %ebx, -8
 ; X32-NEXT:    movl 16(%esp), %eax
 ; X32-NEXT:    movb (%eax), %bl
-; X32-NEXT:    movb %bl, (%esp)
+; X32-NEXT:    movzbl %bl, %esi
+; X32-NEXT:    movl %esi, (%esp)
 ; X32-NEXT:    calll take_char
 ; X32-NEXT:    movsbl %bl, %eax
 ; X32-NEXT:    movl %eax, (%esp)
 ; X32-NEXT:    calll take_char
-; X32-NEXT:    movzbl %bl, %eax
-; X32-NEXT:    movl %eax, (%esp)
+; X32-NEXT:    movl %esi, (%esp)
 ; X32-NEXT:    calll take_char
-; X32-NEXT:    addl $8, %esp
+; X32-NEXT:    addl $4, %esp
+; X32-NEXT:    popl %esi
 ; X32-NEXT:    popl %ebx
 ; X32-NEXT:    retl
 ;
 ; X64-LABEL: test_abi_exts_call:
 ; X64:       # BB#0:
 ; X64-NEXT:    pushq %rbx
 ; X64-NEXT:  .Lcfi6:
 ; X64-NEXT:    .cfi_def_cfa_offset 16
 ; X64-NEXT:  .Lcfi7:
 ; X64-NEXT:    .cfi_offset %rbx, -16
-; X64-NEXT:    movb (%rdi), %bl
+; X64-NEXT:    movb (%rdi), %al
+; X64-NEXT:    movzbl %al, %ebx
 ; X64-NEXT:    movl %ebx, %edi
 ; X64-NEXT:    callq take_char
-; X64-NEXT:    movsbl %bl, %ebx
-; X64-NEXT:    movl %ebx, %edi
+; X64-NEXT:    movsbl %bl, %edi
 ; X64-NEXT:    callq take_char
-; X64-NEXT:    movzbl %bl, %edi
+; X64-NEXT:    movl %ebx, %edi
 ; X64-NEXT:    callq take_char
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    retq
   %val = load i8, i8* %addr
   call void @take_char(i8 %val)
   call void @take_char(i8 signext %val)
   call void @take_char(i8 zeroext %val)
  ret void
 }
 
 declare void @variadic_callee(i8*, ...)
 define void @test_variadic_call_1(i8** %addr_ptr, i32* %val_ptr) {
 ; X32-LABEL: test_variadic_call_1:
 ; X32:       # BB#0:
 ; X32-NEXT:    subl $12, %esp
-; X32-NEXT:  .Lcfi10:
+; X32-NEXT:  .Lcfi12:
 ; X32-NEXT:    .cfi_def_cfa_offset 16
 ; X32-NEXT:    movl 16(%esp), %eax
 ; X32-NEXT:    movl 20(%esp), %ecx
 ; X32-NEXT:    movl (%eax), %eax
 ; X32-NEXT:    movl (%ecx), %ecx
 ; X32-NEXT:    movl %eax, (%esp)
 ; X32-NEXT:    movl %ecx, 4(%esp)
 ; X32-NEXT:    calll variadic_callee
; X64-NEXT:    retq
   call void (i8*, ...) @variadic_callee(i8* %addr, i32 %val)
   ret void
 }
 
 define void @test_variadic_call_2(i8** %addr_ptr, double* %val_ptr) {
 ; X32-LABEL: test_variadic_call_2:
 ; X32:       # BB#0:
 ; X32-NEXT:    subl $12, %esp
-; X32-NEXT:  .Lcfi11:
+; X32-NEXT:  .Lcfi13:
 ; X32-NEXT:    .cfi_def_cfa_offset 16
 ; X32-NEXT:    movl 16(%esp), %eax
 ; X32-NEXT:    movl 20(%esp), %ecx
 ; X32-NEXT:    movl (%eax), %eax
 ; X32-NEXT:    movl (%ecx), %edx
 ; X32-NEXT:    movl 4(%ecx), %ecx
 ; X32-NEXT:    movl %eax, (%esp)
 ; X32-NEXT:    movl $4, %eax

test/CodeGen/X86/GlobalISel/ext-x86-64.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=x86_64-linux-gnu    -global-isel -verify-machineinstrs < %s -o - | FileCheck %s --check-prefix=X64
 
 ; TODO merge with ext.ll after i64 sext supported on 32bit platform
 
 define i64 @test_zext_i1(i8 %a) {
 ; X64-LABEL: test_zext_i1:
 ; X64:       # BB#0:
-; X64-NEXT:    # kill: %DIL<def> %DIL<kill> %RDI<def>
+; X64-NEXT:    # kill: %EDI<def> %EDI<kill> %RDI<def>
 ; X64-NEXT:    andq $1, %rdi
 ; X64-NEXT:    movq %rdi, %rax
 ; X64-NEXT:    retq
   %val = trunc i8 %a to i1
   %r = zext i1 %val to i64
   ret i64 %r
 }
 

test/CodeGen/X86/GlobalISel/irtranslator-callingconv.ll

 ; X64: fixedStack:
 ; X64:  id: [[STACK8:[0-9]+]], type: default, offset: 8, size: 1, alignment: 8,
 ; X64-NEXT: isImmutable: true,
 
 ; X64:  id: [[STACK0:[0-9]+]], type: default, offset: 0, size: 1, alignment: 16,
 ; X64-NEXT: isImmutable: true,
 
 ; X64: liveins: %ecx, %edi, %edx, %esi, %r8d, %r9d
-; X64:      [[ARG1:%[0-9]+]](s8) = COPY %edi
-; X64-NEXT: %{{[0-9]+}}(s8) = COPY %esi
-; X64-NEXT: %{{[0-9]+}}(s8) = COPY %edx
-; X64-NEXT: %{{[0-9]+}}(s8) = COPY %ecx
-; X64-NEXT: %{{[0-9]+}}(s8) = COPY %r8d
-; X64-NEXT: %{{[0-9]+}}(s8) = COPY %r9d
+; X64:      [[ARG1_TMP:%[0-9]+]](s32) = COPY %edi
+; X64:      [[ARG1:%[0-9]+]](s8) = G_TRUNC [[ARG1_TMP]](s32)
+; X64-NEXT: %{{[0-9]+}}(s32) = COPY %esi
+; X64-NEXT: %{{[0-9]+}}(s8) = G_TRUNC %{{[0-9]+}}(s32)
+; X64-NEXT: %{{[0-9]+}}(s32) = COPY %edx
+; X64-NEXT: %{{[0-9]+}}(s8) = G_TRUNC %{{[0-9]+}}(s32)
+; X64-NEXT: %{{[0-9]+}}(s32) = COPY %ecx
+; X64-NEXT: %{{[0-9]+}}(s8) = G_TRUNC %{{[0-9]+}}(s32)
+; X64-NEXT: %{{[0-9]+}}(s32) = COPY %r8d
+; X64-NEXT: %{{[0-9]+}}(s8) = G_TRUNC %{{[0-9]+}}(s32)
+; X64-NEXT: %{{[0-9]+}}(s32) = COPY %r9d
+; X64-NEXT: %{{[0-9]+}}(s8) = G_TRUNC %{{[0-9]+}}(s32)
 ; X64-NEXT: [[ARG7_ADDR:%[0-9]+]](p0) = G_FRAME_INDEX %fixed-stack.[[STACK0]]
 ; X64-NEXT: [[ARG7:%[0-9]+]](s8) = G_LOAD [[ARG7_ADDR]](p0) :: (invariant load 1 from %fixed-stack.[[STACK0]], align 0)
 ; X64-NEXT: [[ARG8_ADDR:%[0-9]+]](p0) = G_FRAME_INDEX %fixed-stack.[[STACK8]]
 ; X64-NEXT: [[ARG8:%[0-9]+]](s8) = G_LOAD [[ARG8_ADDR]](p0) :: (invariant load 1 from %fixed-stack.[[STACK8]], align 0)
 
 ; X32: fixedStack:
 ; X32:  id: [[STACK28:[0-9]+]], type: default, offset: 28, size: 1, alignment: 4,
 ; X32-NEXT: isImmutable: true,

 ; X32-NEXT:       isImmutable: true, 
 ; X32:       %1(p0) = G_FRAME_INDEX %fixed-stack.0
 ; X32-NEXT:  %0(p0) = G_LOAD %1(p0) :: (invariant load 4 from %fixed-stack.0, align 0)
 ; X32-NEXT:  %2(s8) = G_LOAD %0(p0) :: (load 1 from %ir.addr)
 ; X32-NEXT:  ADJCALLSTACKDOWN32 4, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
 ; X32-NEXT:  %3(p0) = COPY %esp
 ; X32-NEXT:  %4(s32) = G_CONSTANT i32 0
 ; X32-NEXT:  %5(p0) = G_GEP %3, %4(s32)
-; X32-NEXT:  G_STORE %2(s8), %5(p0) :: (store 4 into stack, align 0)
+; X32-NEXT:  %6(s32) = G_ANYEXT %2(s8)
+; X32-NEXT:  G_STORE %6(s32), %5(p0) :: (store 4 into stack, align 0)
 ; X32-NEXT:  CALLpcrel32 @take_char, csr_32, implicit %esp
 ; X32-NEXT:  ADJCALLSTACKUP32 4, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
 ; X32-NEXT:  ADJCALLSTACKDOWN32 4, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
-; X32-NEXT:  %6(p0) = COPY %esp
-; X32-NEXT:  %7(s32) = G_CONSTANT i32 0
-; X32-NEXT:  %8(p0) = G_GEP %6, %7(s32)
-; X32-NEXT:  %9(s32) = G_SEXT %2(s8)
-; X32-NEXT:  G_STORE %9(s32), %8(p0) :: (store 4 into stack, align 0)
+; X32-NEXT:  %7(p0) = COPY %esp
+; X32-NEXT:  %8(s32) = G_CONSTANT i32 0
+; X32-NEXT:  %9(p0) = G_GEP %7, %8(s32)
+; X32-NEXT:  %10(s32) = G_SEXT %2(s8)
+; X32-NEXT:  G_STORE %10(s32), %9(p0) :: (store 4 into stack, align 0)
 ; X32-NEXT:  CALLpcrel32 @take_char, csr_32, implicit %esp
 ; X32-NEXT:  ADJCALLSTACKUP32 4, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
 ; X32-NEXT:  ADJCALLSTACKDOWN32 4, 0, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
-; X32-NEXT:  %10(p0) = COPY %esp
-; X32-NEXT:  %11(s32) = G_CONSTANT i32 0
-; X32-NEXT:  %12(p0) = G_GEP %10, %11(s32)
-; X32-NEXT:  %13(s32) = G_ZEXT %2(s8)
-; X32-NEXT:  G_STORE %13(s32), %12(p0) :: (store 4 into stack, align 0)
+; X32-NEXT:  %11(p0) = COPY %esp
+; X32-NEXT:  %12(s32) = G_CONSTANT i32 0
+; X32-NEXT:  %13(p0) = G_GEP %11, %12(s32)
+; X32-NEXT:  %14(s32) = G_ZEXT %2(s8)
+; X32-NEXT:  G_STORE %14(s32), %13(p0) :: (store 4 into stack, align 0)
 ; X32-NEXT:  CALLpcrel32 @take_char, csr_32, implicit %esp
 ; X32-NEXT:  ADJCALLSTACKUP32 4, 0, implicit-def %esp, implicit-def %eflags, implicit %esp
 ; X32-NEXT:  RET 0
 
 ; X64:       %0(p0) = COPY %rdi
 ; X64-NEXT:  %1(s8) = G_LOAD %0(p0) :: (load 1 from %ir.addr)
 ; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
-; X64-NEXT:  %edi = COPY %1(s8)
+; X64-NEXT:  %2(s32) = G_ANYEXT %1(s8)
+; X64-NEXT:  %edi = COPY %2(s32)
 ; X64-NEXT:  CALL64pcrel32 @take_char, csr_64, implicit %rsp, implicit %edi
 ; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
 ; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
-; X64-NEXT:  %2(s32) = G_SEXT %1(s8)
-; X64-NEXT:  %edi = COPY %2(s32)
+; X64-NEXT:  %3(s32) = G_SEXT %1(s8)
+; X64-NEXT:  %edi = COPY %3(s32)
 ; X64-NEXT:  CALL64pcrel32 @take_char, csr_64, implicit %rsp, implicit %edi
 ; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
 ; X64-NEXT:  ADJCALLSTACKDOWN64 0, 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
-; X64-NEXT:  %3(s32) = G_ZEXT %1(s8)
-; X64-NEXT:  %edi = COPY %3(s32)
+; X64-NEXT:  %4(s32) = G_ZEXT %1(s8)
+; X64-NEXT:  %edi = COPY %4(s32)
 ; X64-NEXT:  CALL64pcrel32 @take_char, csr_64, implicit %rsp, implicit %edi
 ; X64-NEXT:  ADJCALLSTACKUP64 0, 0, implicit-def %rsp, implicit-def %eflags, implicit %rsp
 ; X64-NEXT:  RET 0
 
   %val = load i8, i8* %addr
   call void @take_char(i8 %val)
   call void @take_char(i8 signext %val)
   call void @take_char(i8 zeroext %val)