Do not generate calls to the 128-bit function __multi3() on 32-bit ARM.

[koutheir]

Summary

The function __multi3() is undefined on 32-bit ARM, so a call to it should never be emitted. Instead, plain instructions need to be generated to perform 128-bit multiplications.

[koutheir]

This patch is intended to solve the bug 20871 for the ARM back-end.

[rengolin]

Doesn't this depend on which runtime library you're using? No Arm32 library implement that?

I remember a patch to add 128-bit arithmetic into Armv7 many years ago (from Android) but I'm not sure it ever got into compiler-rt.

[koutheir]

Doesn't this depend on which runtime library you're using? No Arm32 library implement that?

I remember a patch to add 128-bit arithmetic into Armv7 many years ago (from Android) but I'm not sure it ever got into compiler-rt.

I got linking errors due to undefined symbol __multi3.

Looking at compiler-rt/lib/builtins/multi3.c, I see that __multi3() is implemented only if CRT_HAS_128BIT is defined:

#ifdef CRT_HAS_128BIT
/*...*/
// Returns: a * b

COMPILER_RT_ABI ti_int __multi3(ti_int a, ti_int b) {
/*...*/

The file compiler-rt/lib/builtins/int_types.h does not define CRT_HAS_128BIT on ARM:

#if defined(__LP64__) || defined(__wasm__) || defined(__mips64) ||             \
    defined(__riscv) || defined(_WIN64)
#define CRT_HAS_128BIT
#endif

So, I think the answer to your question is no, __multi3() is not implemented for ARM.

[rengolin]

Right, ok, this is true for compiler-rt, but not necessarily other runtime libraries.

There is an open discussion if we should assume compiler-rt is the default and use what we have, but this is a different issue.

If the call to __multi3 is there by accident (leaked from other target's default), then it's ok to remove it. If not, we may need some more complex solution.

@compnerd @joerg @t.p.northover @srhines, do you know of any Arm32 runtime library that implements 128-bit maths with Clang?

I think Android does, so perhaps we can add a condition there on AndroidEABI?

[srhines]

In D103906#2806084, @rengolin wrote:

Right, ok, this is true for compiler-rt, but not necessarily other runtime libraries.

There is an open discussion if we should assume compiler-rt is the default and use what we have, but this is a different issue.

If the call to __multi3 is there by accident (leaked from other target's default), then it's ok to remove it. If not, we may need some more complex solution.

@compnerd @joerg @t.p.northover @srhines, do you know of any Arm32 runtime library that implements 128-bit maths with Clang?

I think Android does, so perhaps we can add a condition there on AndroidEABI?

We don't define 128-bit operations. https://cs.android.com/search?q=CRT_HAS_128BIT%20-filepath:external%2Fcompiler-rt%20-filepath:external%2Fllvm-project&sq=&ss=android shows that we don't define CRT_HAS_128BIT anywhere, nor do I see any 128-bit symbols in our libclang_rt.builtins-arm-android.a. We do default to compiler-rt these days for builtins, and thus the 128-bit symbols can't be coming from anywhere else either.

[joerg]

We inherited the lack of int128_t support on 32bit platforms from GCC. IMO we should fix that, the only tricky one is division.

[efriedma]

We could change compiler-rt to expose the relevant routines if we wanted to. See https://reviews.llvm.org/D43106 etc.

[rengolin]

Right, I think it's a bit clearer now.

1. We inherited the lack of 128-bit arithmetic from GCC and other targets, so currently, there isn't a default place we can get them from.
2. The current version of compiler-rt doesn't have them, nor does libgcc, so if users want them, they have to link their own libs.
3. We want to implement them, long term, in compiler-rt, which we also want to make the default in LLVM for all supported targets.

Given the time-frame of item (3) and the fact that, today, the compiler is generating code that doesn't link (without users providing the symbols themselves), I think this change is correct for the time being. I'm creating a bug to address this and adding people here to the CC list.

Otherwise, this patch LGTM, thanks!

[koutheir]

@rengolin: I don't have commit access. Can you please commit this patch on my behalf?

[rengolin]

Done. I have run tests locally and they were fine, but please look out for buildbot emails to make sure it didn't break anything we didn't test. Thanks!

[thakis]

This breaks tests everywhere as far as I can tell. Eg http://45.33.8.238/linux/48494/step_12.txt

Please take a look and revert for now if it takes a while to fix.

[thakis]

Reverted in 68a1d9a1f5735ec5a595bbe2fffab540b9fc1710 . More or less all bots on https://lab.llvm.org/buildbot/#/console are red.

[rengolin]

In D103906#2808482, @thakis wrote:

Reverted in 68a1d9a1f5735ec5a595bbe2fffab540b9fc1710 . More or less all bots on https://lab.llvm.org/buildbot/#/console are red.

Thanks Nico! Sorry, I was out for a while. Not sure why I didn't catch this on my build.

@koutheir how did you test this?

[koutheir]

@rengolin I tested the ARM tests that I thought would be affected. This appears to be affecting other tests.

I'm currently running a full check-llvm for the ancestor commit of 64e9aa33020d, after which I'll also perform another check-llvm for the commit 64e9aa33020d. If more tests are failing with the commit 64e9aa33020d, then I'll fix them and update the patch.

[rengolin]

In D103906#2808814, @koutheir wrote:

@rengolin I tested the ARM tests that I thought would be affected. This appears to be affecting other tests.

Right, before submitting a patch, you should run at the very least "ninja check" on the affected components, in your case, LLVM only.

[koutheir]

I'll do that from now on. I'm sorry for the confusion.

[koutheir]

@rengolin: Can you please commit this updated patch on my behalf?

[rengolin]

I'm having trouble building and testing, if anyone else could commit this for the time being, I'd appreciate it. Thanks!

@efriedma @joerg

[rengolin]

In D103906#2810130, @rengolin wrote:

I'm having trouble building and testing, if anyone else could commit this for the time being, I'd appreciate it. Thanks!

Ok, managed to fix my environment, built and tested fine on x86_64. Let me know if bots crash again.

[koutheir]

The commit 789708617d20 caused two failures:

• https://lab.llvm.org/buildbot/#/builders/111/builds/2256
• https://lab.llvm.org/buildbot/#/builders/12/builds/5021

However, I believe these failures are unrelated to these changes. It looks like a problem of selecting the required C++ standard for the LLVM C++ source files.

[koutheir]

The build builders/53/builds/3020 fails due a time out. I don't think this is an actual failure. Maybe someone could increase the time out value for certain long-running tests.

[rengolin]

In D103906#2814737, @koutheir wrote:

The build builders/53/builds/3020 fails due a time out. I don't think this is an actual failure. Maybe someone could increase the time out value for certain long-running tests.

You're right, none of these breakages seem related to your patch. Thanks for checking.

llvm/lib/Target/ARM/ARMISelLowering.cpp

if (Subtarget->isThumb() && Subtarget->hasVFP2Base() &&
       }
     }
   }
 
   // These libcalls are not available in 32-bit.
   setLibcallName(RTLIB::SHL_I128, nullptr);
   setLibcallName(RTLIB::SRL_I128, nullptr);
   setLibcallName(RTLIB::SRA_I128, nullptr);
+  setLibcallName(RTLIB::MUL_I128, nullptr);
 
   // RTLIB
   if (Subtarget->isAAPCS_ABI() &&
       (Subtarget->isTargetAEABI() || Subtarget->isTargetGNUAEABI() ||
        Subtarget->isTargetMuslAEABI() || Subtarget->isTargetAndroid())) {
     static const struct {
       const RTLIB::Libcall Op;
       const char * const Name;

llvm/test/CodeGen/ARM/umulo-128-legalisation-lowering.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=armv6-unknown-linux-gnu | FileCheck %s --check-prefixes=ARMV6
 ; RUN: llc < %s -mtriple=armv7-unknown-linux-gnu | FileCheck %s --check-prefixes=ARMV7
 
 define { i128, i8 } @muloti_test(i128 %l, i128 %r) unnamed_addr #0 {
 ; ARMV6-LABEL: muloti_test:
 ; ARMV6:       @ %bb.0: @ %start
 ; ARMV6-NEXT:    push {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; ARMV6-NEXT:    sub sp, sp, #28
-; ARMV6-NEXT:    mov r9, #0
-; ARMV6-NEXT:    mov r11, r0
+; ARMV6-NEXT:    ldr r7, [sp, #72]
+; ARMV6-NEXT:    mov r6, r0
+; ARMV6-NEXT:    str r0, [sp, #8]            @ 4-byte Spill
+; ARMV6-NEXT:    ldr r4, [sp, #84]
+; ARMV6-NEXT:    umull r1, r0, r2, r7
+; ARMV6-NEXT:    mov lr, r7
+; ARMV6-NEXT:    umull r5, r10, r4, r2
+; ARMV6-NEXT:    str r1, [r6]
+; ARMV6-NEXT:    ldr r6, [sp, #80]
+; ARMV6-NEXT:    umull r1, r7, r3, r6
+; ARMV6-NEXT:    str r7, [sp, #12]           @ 4-byte Spill
+; ARMV6-NEXT:    add r1, r5, r1
+; ARMV6-NEXT:    umull r7, r5, r6, r2
+; ARMV6-NEXT:    mov r6, lr
+; ARMV6-NEXT:    str r7, [sp, #16]           @ 4-byte Spill
+; ARMV6-NEXT:    mov r7, #0
+; ARMV6-NEXT:    adds r1, r5, r1
+; ARMV6-NEXT:    str r1, [sp, #4]            @ 4-byte Spill
+; ARMV6-NEXT:    adc r1, r7, #0
+; ARMV6-NEXT:    str r1, [sp, #24]           @ 4-byte Spill
+; ARMV6-NEXT:    ldr r1, [sp, #64]
 ; ARMV6-NEXT:    ldr r7, [sp, #76]
-; ARMV6-NEXT:    mov r5, r3
-; ARMV6-NEXT:    ldr r10, [sp, #72]
-; ARMV6-NEXT:    mov r1, r3
-; ARMV6-NEXT:    mov r6, r2
-; ARMV6-NEXT:    mov r0, r2
+; ARMV6-NEXT:    ldr r5, [sp, #64]
+; ARMV6-NEXT:    umull r12, r9, r7, r1
+; ARMV6-NEXT:    ldr r1, [sp, #68]
+; ARMV6-NEXT:    umull r11, r8, r1, lr
+; ARMV6-NEXT:    add r12, r11, r12
+; ARMV6-NEXT:    umull r11, lr, r5, lr
+; ARMV6-NEXT:    mov r5, r6
+; ARMV6-NEXT:    mov r6, #0
+; ARMV6-NEXT:    adds r12, lr, r12
+; ARMV6-NEXT:    umull r2, lr, r2, r7
+; ARMV6-NEXT:    adc r6, r6, #0
+; ARMV6-NEXT:    str r6, [sp, #20]           @ 4-byte Spill
+; ARMV6-NEXT:    ldr r6, [sp, #16]           @ 4-byte Reload
+; ARMV6-NEXT:    adds r11, r11, r6
+; ARMV6-NEXT:    ldr r6, [sp, #4]            @ 4-byte Reload
+; ARMV6-NEXT:    adc r6, r12, r6
+; ARMV6-NEXT:    mov r12, #0
+; ARMV6-NEXT:    umlal r0, r12, r3, r5
+; ARMV6-NEXT:    ldr r5, [sp, #8]            @ 4-byte Reload
+; ARMV6-NEXT:    str r6, [sp, #16]           @ 4-byte Spill
+; ARMV6-NEXT:    ldr r6, [sp, #64]
+; ARMV6-NEXT:    adds r0, r2, r0
+; ARMV6-NEXT:    str r0, [r5, #4]
+; ARMV6-NEXT:    adcs r0, r12, lr
 ; ARMV6-NEXT:    mov r2, #0
-; ARMV6-NEXT:    mov r3, #0
-; ARMV6-NEXT:    str r9, [sp, #12]
-; ARMV6-NEXT:    str r9, [sp, #8]
-; ARMV6-NEXT:    str r7, [sp, #4]
-; ARMV6-NEXT:    str r10, [sp]
-; ARMV6-NEXT:    bl __multi3
-; ARMV6-NEXT:    str r3, [sp, #20] @ 4-byte Spill
-; ARMV6-NEXT:    str r2, [sp, #16] @ 4-byte Spill
-; ARMV6-NEXT:    stm r11, {r0, r1}
-; ARMV6-NEXT:    ldr r0, [sp, #84]
-; ARMV6-NEXT:    ldr r3, [sp, #80]
-; ARMV6-NEXT:    ldr r8, [sp, #64]
-; ARMV6-NEXT:    umull r4, r0, r0, r6
-; ARMV6-NEXT:    umull r2, r1, r5, r3
-; ARMV6-NEXT:    add r2, r4, r2
-; ARMV6-NEXT:    umull lr, r4, r3, r6
-; ARMV6-NEXT:    umull r3, r6, r7, r8
-; ARMV6-NEXT:    adds r12, r4, r2
-; ARMV6-NEXT:    adc r2, r9, #0
-; ARMV6-NEXT:    str r2, [sp, #24] @ 4-byte Spill
-; ARMV6-NEXT:    ldr r2, [sp, #68]
-; ARMV6-NEXT:    umull r4, r2, r2, r10
-; ARMV6-NEXT:    add r3, r4, r3
-; ARMV6-NEXT:    umull r4, r10, r8, r10
-; ARMV6-NEXT:    adds r3, r10, r3
-; ARMV6-NEXT:    adc r10, r9, #0
-; ARMV6-NEXT:    adds r4, r4, lr
-; ARMV6-NEXT:    adc r12, r3, r12
-; ARMV6-NEXT:    ldr r3, [sp, #16] @ 4-byte Reload
-; ARMV6-NEXT:    adds r4, r3, r4
-; ARMV6-NEXT:    str r4, [r11, #8]
-; ARMV6-NEXT:    ldr r4, [sp, #20] @ 4-byte Reload
-; ARMV6-NEXT:    adcs r3, r4, r12
-; ARMV6-NEXT:    str r3, [r11, #12]
-; ARMV6-NEXT:    ldr r3, [sp, #84]
-; ARMV6-NEXT:    adc r12, r9, #0
-; ARMV6-NEXT:    cmp r5, #0
-; ARMV6-NEXT:    movne r5, #1
-; ARMV6-NEXT:    cmp r3, #0
-; ARMV6-NEXT:    mov r4, r3
-; ARMV6-NEXT:    movne r4, #1
-; ARMV6-NEXT:    cmp r0, #0
-; ARMV6-NEXT:    movne r0, #1
+; ARMV6-NEXT:    adc r2, r2, #0
+; ARMV6-NEXT:    orrs lr, r6, r1
+; ARMV6-NEXT:    ldr r6, [sp, #80]
+; ARMV6-NEXT:    movne lr, #1
+; ARMV6-NEXT:    umlal r0, r2, r3, r7
+; ARMV6-NEXT:    orrs r12, r6, r4
+; ARMV6-NEXT:    movne r12, #1
+; ARMV6-NEXT:    cmp r9, #0
+; ARMV6-NEXT:    ldr r6, [sp, #12]           @ 4-byte Reload
+; ARMV6-NEXT:    movne r9, #1
+; ARMV6-NEXT:    cmp r8, #0
+; ARMV6-NEXT:    movne r8, #1
+; ARMV6-NEXT:    cmp r6, #0
+; ARMV6-NEXT:    movne r6, #1
+; ARMV6-NEXT:    cmp r10, #0
+; ARMV6-NEXT:    movne r10, #1
 ; ARMV6-NEXT:    cmp r1, #0
-; ARMV6-NEXT:    and r5, r4, r5
 ; ARMV6-NEXT:    movne r1, #1
-; ARMV6-NEXT:    orr r0, r5, r0
-; ARMV6-NEXT:    ldr r5, [sp, #68]
-; ARMV6-NEXT:    orr r0, r0, r1
-; ARMV6-NEXT:    ldr r1, [sp, #24] @ 4-byte Reload
 ; ARMV6-NEXT:    cmp r7, #0
-; ARMV6-NEXT:    orr r0, r0, r1
 ; ARMV6-NEXT:    movne r7, #1
-; ARMV6-NEXT:    cmp r5, #0
-; ARMV6-NEXT:    mov r1, r5
-; ARMV6-NEXT:    movne r1, #1
-; ARMV6-NEXT:    cmp r2, #0
-; ARMV6-NEXT:    movne r2, #1
+; ARMV6-NEXT:    cmp r4, #0
+; ARMV6-NEXT:    movne r4, #1
+; ARMV6-NEXT:    cmp r3, #0
+; ARMV6-NEXT:    movne r3, #1
+; ARMV6-NEXT:    adds r0, r0, r11
+; ARMV6-NEXT:    str r0, [r5, #8]
 ; ARMV6-NEXT:    and r1, r1, r7
+; ARMV6-NEXT:    ldr r0, [sp, #16]           @ 4-byte Reload
+; ARMV6-NEXT:    orr r1, r1, r8
+; ARMV6-NEXT:    orr r1, r1, r9
+; ARMV6-NEXT:    adcs r0, r2, r0
+; ARMV6-NEXT:    str r0, [r5, #12]
+; ARMV6-NEXT:    and r0, r4, r3
+; ARMV6-NEXT:    ldr r2, [sp, #24]           @ 4-byte Reload
+; ARMV6-NEXT:    orr r0, r0, r10
+; ARMV6-NEXT:    orr r0, r0, r6
+; ARMV6-NEXT:    orr r0, r0, r2
+; ARMV6-NEXT:    ldr r2, [sp, #20]           @ 4-byte Reload
 ; ARMV6-NEXT:    orr r1, r1, r2
-; ARMV6-NEXT:    ldr r2, [sp, #80]
-; ARMV6-NEXT:    cmp r6, #0
-; ARMV6-NEXT:    movne r6, #1
-; ARMV6-NEXT:    orrs r2, r2, r3
-; ARMV6-NEXT:    orr r1, r1, r6
-; ARMV6-NEXT:    movne r2, #1
-; ARMV6-NEXT:    orrs r7, r8, r5
-; ARMV6-NEXT:    orr r1, r1, r10
-; ARMV6-NEXT:    movne r7, #1
-; ARMV6-NEXT:    and r2, r7, r2
+; ARMV6-NEXT:    and r2, lr, r12
 ; ARMV6-NEXT:    orr r1, r2, r1
 ; ARMV6-NEXT:    orr r0, r1, r0
-; ARMV6-NEXT:    orr r0, r0, r12
+; ARMV6-NEXT:    mov r1, #0
+; ARMV6-NEXT:    adc r1, r1, #0
+; ARMV6-NEXT:    orr r0, r0, r1
 ; ARMV6-NEXT:    and r0, r0, #1
-; ARMV6-NEXT:    strb r0, [r11, #16]
+; ARMV6-NEXT:    strb r0, [r5, #16]
 ; ARMV6-NEXT:    add sp, sp, #28
 ; ARMV6-NEXT:    pop {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 ;
 ; ARMV7-LABEL: muloti_test:
 ; ARMV7:       @ %bb.0: @ %start
 ; ARMV7-NEXT:    push {r4, r5, r6, r7, r8, r9, r10, r11, lr}
-; ARMV7-NEXT:    sub sp, sp, #44
-; ARMV7-NEXT:    str r0, [sp, #40] @ 4-byte Spill
-; ARMV7-NEXT:    mov r0, #0
-; ARMV7-NEXT:    ldr r8, [sp, #88]
-; ARMV7-NEXT:    mov r5, r3
-; ARMV7-NEXT:    ldr r7, [sp, #92]
-; ARMV7-NEXT:    mov r1, r3
-; ARMV7-NEXT:    mov r6, r2
-; ARMV7-NEXT:    str r0, [sp, #8]
-; ARMV7-NEXT:    str r0, [sp, #12]
-; ARMV7-NEXT:    mov r0, r2
-; ARMV7-NEXT:    mov r2, #0
-; ARMV7-NEXT:    mov r3, #0
-; ARMV7-NEXT:    str r8, [sp]
-; ARMV7-NEXT:    str r7, [sp, #4]
-; ARMV7-NEXT:    bl __multi3
-; ARMV7-NEXT:    str r1, [sp, #28] @ 4-byte Spill
-; ARMV7-NEXT:    ldr r1, [sp, #80]
-; ARMV7-NEXT:    str r2, [sp, #24] @ 4-byte Spill
-; ARMV7-NEXT:    str r3, [sp, #20] @ 4-byte Spill
-; ARMV7-NEXT:    umull r2, r9, r7, r1
-; ARMV7-NEXT:    str r0, [sp, #32] @ 4-byte Spill
-; ARMV7-NEXT:    ldr r4, [sp, #84]
-; ARMV7-NEXT:    ldr r0, [sp, #96]
-; ARMV7-NEXT:    umull r1, r3, r1, r8
-; ARMV7-NEXT:    umull r12, r10, r4, r8
-; ARMV7-NEXT:    str r1, [sp, #16] @ 4-byte Spill
-; ARMV7-NEXT:    umull lr, r1, r5, r0
-; ARMV7-NEXT:    add r2, r12, r2
-; ARMV7-NEXT:    umull r11, r8, r0, r6
-; ARMV7-NEXT:    ldr r0, [sp, #100]
-; ARMV7-NEXT:    adds r2, r3, r2
-; ARMV7-NEXT:    mov r12, #0
-; ARMV7-NEXT:    umull r6, r0, r0, r6
-; ARMV7-NEXT:    adc r3, r12, #0
-; ARMV7-NEXT:    str r3, [sp, #36] @ 4-byte Spill
-; ARMV7-NEXT:    add r3, r6, lr
-; ARMV7-NEXT:    ldr r6, [sp, #16] @ 4-byte Reload
-; ARMV7-NEXT:    adds r3, r8, r3
-; ARMV7-NEXT:    adc lr, r12, #0
-; ARMV7-NEXT:    adds r6, r6, r11
-; ARMV7-NEXT:    adc r2, r2, r3
-; ARMV7-NEXT:    ldr r3, [sp, #24] @ 4-byte Reload
-; ARMV7-NEXT:    mov r12, #0
-; ARMV7-NEXT:    adds r3, r3, r6
-; ARMV7-NEXT:    ldr r6, [sp, #20] @ 4-byte Reload
-; ARMV7-NEXT:    adcs r8, r6, r2
-; ARMV7-NEXT:    ldr r6, [sp, #40] @ 4-byte Reload
-; ARMV7-NEXT:    ldr r2, [sp, #32] @ 4-byte Reload
-; ARMV7-NEXT:    str r2, [r6]
-; ARMV7-NEXT:    ldr r2, [sp, #28] @ 4-byte Reload
-; ARMV7-NEXT:    stmib r6, {r2, r3, r8}
-; ARMV7-NEXT:    adc r8, r12, #0
-; ARMV7-NEXT:    cmp r5, #0
-; ARMV7-NEXT:    ldr r2, [sp, #100]
-; ARMV7-NEXT:    movwne r5, #1
-; ARMV7-NEXT:    cmp r2, #0
-; ARMV7-NEXT:    mov r3, r2
+; ARMV7-NEXT:    sub sp, sp, #36
+; ARMV7-NEXT:    ldr r5, [sp, #84]
+; ARMV7-NEXT:    mov r8, r0
+; ARMV7-NEXT:    ldr r1, [sp, #72]
+; ARMV7-NEXT:    ldr r10, [sp, #80]
+; ARMV7-NEXT:    ldr r9, [sp, #76]
+; ARMV7-NEXT:    umull r4, lr, r5, r1
+; ARMV7-NEXT:    umull r0, r7, r2, r10
+; ARMV7-NEXT:    str r4, [sp, #24]           @ 4-byte Spill
+; ARMV7-NEXT:    ldr r4, [sp, #88]
+; ARMV7-NEXT:    umull r1, r6, r1, r10
+; ARMV7-NEXT:    str r0, [sp, #32]           @ 4-byte Spill
+; ARMV7-NEXT:    umull r11, r0, r2, r5
+; ARMV7-NEXT:    str r6, [sp, #20]           @ 4-byte Spill
+; ARMV7-NEXT:    str r1, [sp, #28]           @ 4-byte Spill
+; ARMV7-NEXT:    umull r6, r12, r3, r4
+; ARMV7-NEXT:    ldr r1, [sp, #92]
+; ARMV7-NEXT:    str r0, [sp, #8]            @ 4-byte Spill
+; ARMV7-NEXT:    mov r0, #0
+; ARMV7-NEXT:    umlal r7, r0, r3, r10
+; ARMV7-NEXT:    str r6, [sp, #16]           @ 4-byte Spill
+; ARMV7-NEXT:    umull r6, r1, r1, r2
+; ARMV7-NEXT:    umull r2, r4, r4, r2
+; ARMV7-NEXT:    str r6, [sp, #4]            @ 4-byte Spill
+; ARMV7-NEXT:    str r2, [sp, #12]           @ 4-byte Spill
+; ARMV7-NEXT:    adds r2, r11, r7
+; ARMV7-NEXT:    ldr r7, [sp, #8]            @ 4-byte Reload
+; ARMV7-NEXT:    mov r11, #0
+; ARMV7-NEXT:    str r4, [sp]                @ 4-byte Spill
+; ARMV7-NEXT:    umull r6, r4, r9, r10
+; ARMV7-NEXT:    adcs r9, r0, r7
+; ARMV7-NEXT:    ldr r0, [sp, #32]           @ 4-byte Reload
+; ARMV7-NEXT:    adc r10, r11, #0
+; ARMV7-NEXT:    stm r8, {r0, r2}
+; ARMV7-NEXT:    ldr r0, [sp, #24]           @ 4-byte Reload
+; ARMV7-NEXT:    umlal r9, r10, r3, r5
+; ARMV7-NEXT:    ldr r2, [sp, #20]           @ 4-byte Reload
+; ARMV7-NEXT:    add r0, r6, r0
+; ARMV7-NEXT:    adds r2, r2, r0
+; ARMV7-NEXT:    ldr r6, [sp, #4]            @ 4-byte Reload
+; ARMV7-NEXT:    adc r0, r11, #0
+; ARMV7-NEXT:    str r0, [sp, #32]           @ 4-byte Spill
+; ARMV7-NEXT:    ldr r0, [sp, #16]           @ 4-byte Reload
+; ARMV7-NEXT:    ldr r7, [sp, #28]           @ 4-byte Reload
+; ARMV7-NEXT:    add r0, r6, r0
+; ARMV7-NEXT:    ldr r6, [sp]                @ 4-byte Reload
+; ARMV7-NEXT:    adds r0, r6, r0
+; ARMV7-NEXT:    ldr r6, [sp, #12]           @ 4-byte Reload
+; ARMV7-NEXT:    adc r11, r11, #0
+; ARMV7-NEXT:    adds r7, r7, r6
+; ARMV7-NEXT:    ldr r6, [sp, #92]
+; ARMV7-NEXT:    adc r0, r2, r0
+; ARMV7-NEXT:    str r0, [sp, #28]           @ 4-byte Spill
+; ARMV7-NEXT:    ldr r0, [sp, #92]
+; ARMV7-NEXT:    cmp r3, #0
 ; ARMV7-NEXT:    movwne r3, #1
+; ARMV7-NEXT:    ldr r2, [sp, #76]
 ; ARMV7-NEXT:    cmp r0, #0
 ; ARMV7-NEXT:    movwne r0, #1
 ; ARMV7-NEXT:    cmp r1, #0
-; ARMV7-NEXT:    and r3, r3, r5
 ; ARMV7-NEXT:    movwne r1, #1
-; ARMV7-NEXT:    orr r0, r3, r0
-; ARMV7-NEXT:    cmp r7, #0
+; ARMV7-NEXT:    cmp r12, #0
+; ARMV7-NEXT:    and r0, r0, r3
+; ARMV7-NEXT:    movwne r12, #1
+; ARMV7-NEXT:    cmp r5, #0
 ; ARMV7-NEXT:    orr r0, r0, r1
-; ARMV7-NEXT:    ldr r1, [sp, #80]
-; ARMV7-NEXT:    movwne r7, #1
+; ARMV7-NEXT:    movwne r5, #1
+; ARMV7-NEXT:    cmp r2, #0
+; ARMV7-NEXT:    mov r1, r2
+; ARMV7-NEXT:    mov r3, r2
+; ARMV7-NEXT:    movwne r1, #1
 ; ARMV7-NEXT:    cmp r4, #0
-; ARMV7-NEXT:    orr r1, r1, r4
+; ARMV7-NEXT:    ldr r2, [sp, #72]
 ; ARMV7-NEXT:    movwne r4, #1
-; ARMV7-NEXT:    cmp r10, #0
-; ARMV7-NEXT:    and r3, r4, r7
-; ARMV7-NEXT:    movwne r10, #1
-; ARMV7-NEXT:    cmp r9, #0
-; ARMV7-NEXT:    orr r3, r3, r10
-; ARMV7-NEXT:    ldr r7, [sp, #36] @ 4-byte Reload
-; ARMV7-NEXT:    movwne r9, #1
-; ARMV7-NEXT:    orr r3, r3, r9
-; ARMV7-NEXT:    orr r3, r3, r7
-; ARMV7-NEXT:    ldr r7, [sp, #96]
-; ARMV7-NEXT:    orr r0, r0, lr
-; ARMV7-NEXT:    orrs r7, r7, r2
-; ARMV7-NEXT:    movwne r7, #1
-; ARMV7-NEXT:    cmp r1, #0
-; ARMV7-NEXT:    movwne r1, #1
-; ARMV7-NEXT:    and r1, r1, r7
-; ARMV7-NEXT:    orr r1, r1, r3
+; ARMV7-NEXT:    cmp lr, #0
+; ARMV7-NEXT:    and r1, r1, r5
+; ARMV7-NEXT:    movwne lr, #1
+; ARMV7-NEXT:    orrs r2, r2, r3
+; ARMV7-NEXT:    ldr r3, [sp, #88]
+; ARMV7-NEXT:    movwne r2, #1
+; ARMV7-NEXT:    orr r1, r1, r4
+; ARMV7-NEXT:    orr r0, r0, r12
+; ARMV7-NEXT:    orrs r3, r3, r6
+; ARMV7-NEXT:    orr r1, r1, lr
+; ARMV7-NEXT:    movwne r3, #1
+; ARMV7-NEXT:    adds r7, r9, r7
+; ARMV7-NEXT:    str r7, [r8, #8]
+; ARMV7-NEXT:    and r2, r2, r3
+; ARMV7-NEXT:    ldr r7, [sp, #28]           @ 4-byte Reload
+; ARMV7-NEXT:    orr r0, r0, r11
+; ARMV7-NEXT:    adcs r7, r10, r7
+; ARMV7-NEXT:    str r7, [r8, #12]
+; ARMV7-NEXT:    ldr r7, [sp, #32]           @ 4-byte Reload
+; ARMV7-NEXT:    orr r1, r1, r7
+; ARMV7-NEXT:    orr r1, r2, r1
 ; ARMV7-NEXT:    orr r0, r1, r0
-; ARMV7-NEXT:    orr r0, r0, r8
+; ARMV7-NEXT:    mov r1, #0
+; ARMV7-NEXT:    adc r1, r1, #0
+; ARMV7-NEXT:    orr r0, r0, r1
 ; ARMV7-NEXT:    and r0, r0, #1
-; ARMV7-NEXT:    strb r0, [r6, #16]
-; ARMV7-NEXT:    add sp, sp, #44
+; ARMV7-NEXT:    strb r0, [r8, #16]
+; ARMV7-NEXT:    add sp, sp, #36
 ; ARMV7-NEXT:    pop {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 start:
   %0 = tail call { i128, i1 } @llvm.umul.with.overflow.i128(i128 %l, i128 %r) #2
   %1 = extractvalue { i128, i1 } %0, 0
   %2 = extractvalue { i128, i1 } %0, 1
   %3 = zext i1 %2 to i8
   %4 = insertvalue { i128, i8 } undef, i128 %1, 0
   %5 = insertvalue { i128, i8 } %4, i8 %3, 1

llvm/test/CodeGen/Thumb/pr35836_2.ll

entry:
   %conv.i30 = zext i64 %5 to i128
   %shl.i = shl nuw i128 %conv.i30, 64
   %or17 = or i128 %shl.i, %conv.i37
   %add = add i128 %or7, %or
   %add18 = add i128 %or17, %or12
   %mul = mul i128 %add18, %add
   ret i128 %mul
 }
-; CHECK: adds r4, r2, r7
-; CHECK: mov  r4, r1
-; CHECK: adcs r4, r6
-; CHECK: ldr  r4, [sp, #20]           @ 4-byte Reload
-; CHECK: adcs r5, r4
-; CHECK: ldr  r4, [sp, #24]           @ 4-byte Reload
-; CHECK: adcs r3, r4
-; CHECK: adds r4, r2, r7
-; CHECK: adcs r1, r6
-; CHECK: str  r4, [sp]
-; CHECK: str  r1, [sp, #4]
-; CHECK: ldr  r2, [r0, #16]
-; CHECK: ldr  r6, [r0, #24]
-; CHECK: adcs r6, r2
-; CHECK: str  r6, [sp, #8]
-; CHECK: ldr  r2, [r0, #20]
-; CHECK: ldr  r0, [r0, #28]
-; CHECK: adcs r0, r2
+; CHECK: adds r5, r1, r7
+; CHECK: mov r5, r4
+; CHECK: adcs r5, r6
+; CHECK: ldr r5, [sp, #12]                   @ 4-byte Reload
+; CHECK: adcs r2, r5
+; CHECK: ldr r5, [sp, #16]                   @ 4-byte Reload
+; CHECK: adcs r3, r5
+; CHECK: adds r5, r1, r7
+; CHECK: adcs r4, r6
+; CHECK: ldr r1, [r0, #20]
+; CHECK: str r1, [sp, #16]                   @ 4-byte Spill
+; CHECK: ldr r6, [r0, #28]
+; CHECK: ldr r1, [r0, #16]
+; CHECK: ldr r7, [r0, #24]
+; CHECK: adcs r7, r1
+; CHECK: ldr r0, [sp, #16]                   @ 4-byte Reload
+; CHECK: adcs r6, r0

llvm/test/CodeGen/Thumb/umulo-128-legalisation-lowering.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=thumb-eabi -mattr=+v6 | FileCheck %s --check-prefixes=THUMBV6
 
 define { i128, i8 } @muloti_test(i128 %l, i128 %r) unnamed_addr #0 {
 ; THUMBV6-LABEL: muloti_test:
 ; THUMBV6:       @ %bb.0: @ %start
 ; THUMBV6-NEXT:    .save {r4, r5, r6, r7, lr}
 ; THUMBV6-NEXT:    push {r4, r5, r6, r7, lr}
-; THUMBV6-NEXT:    .pad #68
-; THUMBV6-NEXT:    sub sp, #68
-; THUMBV6-NEXT:    mov r4, r3
-; THUMBV6-NEXT:    str r2, [sp, #56] @ 4-byte Spill
-; THUMBV6-NEXT:    mov r6, r0
+; THUMBV6-NEXT:    .pad #60
+; THUMBV6-NEXT:    sub sp, #60
+; THUMBV6-NEXT:    mov r6, r3
+; THUMBV6-NEXT:    mov r1, r2
+; THUMBV6-NEXT:    str r2, [sp, #52] @ 4-byte Spill
+; THUMBV6-NEXT:    mov r4, r0
+; THUMBV6-NEXT:    str r0, [sp, #40] @ 4-byte Spill
+; THUMBV6-NEXT:    ldr r2, [sp, #88]
+; THUMBV6-NEXT:    str r2, [sp, #48] @ 4-byte Spill
 ; THUMBV6-NEXT:    movs r5, #0
-; THUMBV6-NEXT:    str r5, [sp, #12]
-; THUMBV6-NEXT:    str r5, [sp, #8]
-; THUMBV6-NEXT:    ldr r0, [sp, #100]
-; THUMBV6-NEXT:    str r0, [sp, #28] @ 4-byte Spill
-; THUMBV6-NEXT:    str r0, [sp, #4]
-; THUMBV6-NEXT:    ldr r0, [sp, #96]
-; THUMBV6-NEXT:    str r0, [sp, #64] @ 4-byte Spill
-; THUMBV6-NEXT:    str r0, [sp]
-; THUMBV6-NEXT:    mov r0, r2
-; THUMBV6-NEXT:    mov r1, r3
-; THUMBV6-NEXT:    mov r2, r5
+; THUMBV6-NEXT:    mov r0, r1
+; THUMBV6-NEXT:    mov r1, r5
 ; THUMBV6-NEXT:    mov r3, r5
-; THUMBV6-NEXT:    bl __multi3
-; THUMBV6-NEXT:    str r2, [sp, #48] @ 4-byte Spill
-; THUMBV6-NEXT:    str r3, [sp, #52] @ 4-byte Spill
-; THUMBV6-NEXT:    str r6, [sp, #44] @ 4-byte Spill
-; THUMBV6-NEXT:    stm r6!, {r0, r1}
-; THUMBV6-NEXT:    ldr r2, [sp, #104]
-; THUMBV6-NEXT:    str r2, [sp, #60] @ 4-byte Spill
-; THUMBV6-NEXT:    mov r0, r4
+; THUMBV6-NEXT:    bl __aeabi_lmul
+; THUMBV6-NEXT:    str r1, [sp, #28] @ 4-byte Spill
+; THUMBV6-NEXT:    str r0, [r4]
+; THUMBV6-NEXT:    ldr r2, [sp, #96]
+; THUMBV6-NEXT:    str r2, [sp, #36] @ 4-byte Spill
+; THUMBV6-NEXT:    mov r4, r6
+; THUMBV6-NEXT:    str r6, [sp, #56] @ 4-byte Spill
+; THUMBV6-NEXT:    mov r0, r6
 ; THUMBV6-NEXT:    mov r1, r5
 ; THUMBV6-NEXT:    mov r3, r5
 ; THUMBV6-NEXT:    bl __aeabi_lmul
-; THUMBV6-NEXT:    str r0, [sp, #36] @ 4-byte Spill
+; THUMBV6-NEXT:    str r0, [sp, #44] @ 4-byte Spill
 ; THUMBV6-NEXT:    mov r7, r1
 ; THUMBV6-NEXT:    subs r0, r1, #1
 ; THUMBV6-NEXT:    sbcs r7, r0
-; THUMBV6-NEXT:    ldr r6, [sp, #108]
-; THUMBV6-NEXT:    mov r0, r6
+; THUMBV6-NEXT:    ldr r0, [sp, #100]
+; THUMBV6-NEXT:    str r0, [sp, #32] @ 4-byte Spill
 ; THUMBV6-NEXT:    mov r1, r5
-; THUMBV6-NEXT:    ldr r2, [sp, #56] @ 4-byte Reload
+; THUMBV6-NEXT:    ldr r6, [sp, #52] @ 4-byte Reload
+; THUMBV6-NEXT:    mov r2, r6
 ; THUMBV6-NEXT:    mov r3, r5
 ; THUMBV6-NEXT:    bl __aeabi_lmul
+; THUMBV6-NEXT:    str r0, [sp, #24] @ 4-byte Spill
 ; THUMBV6-NEXT:    subs r2, r1, #1
 ; THUMBV6-NEXT:    sbcs r1, r2
 ; THUMBV6-NEXT:    subs r2, r4, #1
+; THUMBV6-NEXT:    mov r3, r4
+; THUMBV6-NEXT:    sbcs r3, r2
+; THUMBV6-NEXT:    ldr r4, [sp, #32] @ 4-byte Reload
+; THUMBV6-NEXT:    subs r2, r4, #1
 ; THUMBV6-NEXT:    sbcs r4, r2
-; THUMBV6-NEXT:    str r6, [sp, #40] @ 4-byte Spill
-; THUMBV6-NEXT:    subs r2, r6, #1
-; THUMBV6-NEXT:    sbcs r6, r2
-; THUMBV6-NEXT:    ands r6, r4
-; THUMBV6-NEXT:    orrs r6, r1
-; THUMBV6-NEXT:    orrs r6, r7
-; THUMBV6-NEXT:    ldr r1, [sp, #36] @ 4-byte Reload
-; THUMBV6-NEXT:    adds r4, r0, r1
-; THUMBV6-NEXT:    ldr r0, [sp, #60] @ 4-byte Reload
+; THUMBV6-NEXT:    ands r4, r3
+; THUMBV6-NEXT:    orrs r4, r1
+; THUMBV6-NEXT:    orrs r4, r7
+; THUMBV6-NEXT:    ldr r0, [sp, #44] @ 4-byte Reload
+; THUMBV6-NEXT:    ldr r1, [sp, #24] @ 4-byte Reload
+; THUMBV6-NEXT:    adds r7, r1, r0
+; THUMBV6-NEXT:    ldr r0, [sp, #36] @ 4-byte Reload
 ; THUMBV6-NEXT:    mov r1, r5
-; THUMBV6-NEXT:    ldr r2, [sp, #56] @ 4-byte Reload
+; THUMBV6-NEXT:    mov r2, r6
 ; THUMBV6-NEXT:    mov r3, r5
 ; THUMBV6-NEXT:    bl __aeabi_lmul
-; THUMBV6-NEXT:    str r0, [sp, #36] @ 4-byte Spill
-; THUMBV6-NEXT:    adds r0, r1, r4
-; THUMBV6-NEXT:    str r0, [sp, #32] @ 4-byte Spill
+; THUMBV6-NEXT:    str r0, [sp, #24] @ 4-byte Spill
+; THUMBV6-NEXT:    adds r0, r1, r7
+; THUMBV6-NEXT:    str r0, [sp, #20] @ 4-byte Spill
 ; THUMBV6-NEXT:    mov r0, r5
 ; THUMBV6-NEXT:    adcs r0, r5
-; THUMBV6-NEXT:    orrs r0, r6
-; THUMBV6-NEXT:    str r0, [sp, #24] @ 4-byte Spill
-; THUMBV6-NEXT:    ldr r4, [sp, #88]
-; THUMBV6-NEXT:    ldr r7, [sp, #28] @ 4-byte Reload
-; THUMBV6-NEXT:    mov r0, r7
+; THUMBV6-NEXT:    orrs r0, r4
+; THUMBV6-NEXT:    str r0, [sp, #16] @ 4-byte Spill
+; THUMBV6-NEXT:    ldr r0, [sp, #92]
+; THUMBV6-NEXT:    str r0, [sp, #44] @ 4-byte Spill
+; THUMBV6-NEXT:    ldr r7, [sp, #80]
 ; THUMBV6-NEXT:    mov r1, r5
-; THUMBV6-NEXT:    mov r2, r4
+; THUMBV6-NEXT:    mov r2, r7
 ; THUMBV6-NEXT:    mov r3, r5
 ; THUMBV6-NEXT:    bl __aeabi_lmul
-; THUMBV6-NEXT:    str r0, [sp, #20] @ 4-byte Spill
-; THUMBV6-NEXT:    mov r6, r1
+; THUMBV6-NEXT:    str r0, [sp, #12] @ 4-byte Spill
+; THUMBV6-NEXT:    mov r4, r1
 ; THUMBV6-NEXT:    subs r0, r1, #1
-; THUMBV6-NEXT:    sbcs r6, r0
-; THUMBV6-NEXT:    ldr r0, [sp, #92]
-; THUMBV6-NEXT:    str r0, [sp, #56] @ 4-byte Spill
+; THUMBV6-NEXT:    sbcs r4, r0
+; THUMBV6-NEXT:    ldr r6, [sp, #84]
+; THUMBV6-NEXT:    mov r0, r6
 ; THUMBV6-NEXT:    mov r1, r5
-; THUMBV6-NEXT:    ldr r2, [sp, #64] @ 4-byte Reload
+; THUMBV6-NEXT:    ldr r2, [sp, #48] @ 4-byte Reload
 ; THUMBV6-NEXT:    mov r3, r5
 ; THUMBV6-NEXT:    bl __aeabi_lmul
-; THUMBV6-NEXT:    str r0, [sp, #16] @ 4-byte Spill
+; THUMBV6-NEXT:    str r0, [sp, #4] @ 4-byte Spill
 ; THUMBV6-NEXT:    subs r2, r1, #1
 ; THUMBV6-NEXT:    sbcs r1, r2
-; THUMBV6-NEXT:    subs r2, r7, #1
-; THUMBV6-NEXT:    sbcs r7, r2
-; THUMBV6-NEXT:    mov r3, r7
-; THUMBV6-NEXT:    ldr r7, [sp, #56] @ 4-byte Reload
+; THUMBV6-NEXT:    ldr r3, [sp, #44] @ 4-byte Reload
+; THUMBV6-NEXT:    subs r2, r3, #1
+; THUMBV6-NEXT:    sbcs r3, r2
+; THUMBV6-NEXT:    str r6, [sp, #8] @ 4-byte Spill
+; THUMBV6-NEXT:    subs r2, r6, #1
+; THUMBV6-NEXT:    sbcs r6, r2
+; THUMBV6-NEXT:    ands r6, r3
+; THUMBV6-NEXT:    orrs r6, r1
+; THUMBV6-NEXT:    orrs r6, r4
+; THUMBV6-NEXT:    ldr r0, [sp, #12] @ 4-byte Reload
+; THUMBV6-NEXT:    ldr r1, [sp, #4] @ 4-byte Reload
+; THUMBV6-NEXT:    adds r0, r1, r0
+; THUMBV6-NEXT:    str r0, [sp, #4] @ 4-byte Spill
+; THUMBV6-NEXT:    mov r0, r7
+; THUMBV6-NEXT:    mov r1, r5
+; THUMBV6-NEXT:    ldr r4, [sp, #48] @ 4-byte Reload
+; THUMBV6-NEXT:    mov r2, r4
+; THUMBV6-NEXT:    mov r3, r5
+; THUMBV6-NEXT:    bl __aeabi_lmul
+; THUMBV6-NEXT:    str r0, [sp, #12] @ 4-byte Spill
+; THUMBV6-NEXT:    ldr r0, [sp, #4] @ 4-byte Reload
+; THUMBV6-NEXT:    adds r0, r1, r0
+; THUMBV6-NEXT:    mov r1, r5
+; THUMBV6-NEXT:    adcs r1, r5
+; THUMBV6-NEXT:    orrs r1, r6
+; THUMBV6-NEXT:    ldr r3, [sp, #36] @ 4-byte Reload
+; THUMBV6-NEXT:    ldr r2, [sp, #32] @ 4-byte Reload
+; THUMBV6-NEXT:    orrs r3, r2
+; THUMBV6-NEXT:    subs r2, r3, #1
+; THUMBV6-NEXT:    sbcs r3, r2
+; THUMBV6-NEXT:    ldr r2, [sp, #8] @ 4-byte Reload
+; THUMBV6-NEXT:    orrs r7, r2
 ; THUMBV6-NEXT:    subs r2, r7, #1
 ; THUMBV6-NEXT:    sbcs r7, r2
 ; THUMBV6-NEXT:    ands r7, r3
 ; THUMBV6-NEXT:    orrs r7, r1
-; THUMBV6-NEXT:    orrs r7, r6
-; THUMBV6-NEXT:    ldr r0, [sp, #20] @ 4-byte Reload
 ; THUMBV6-NEXT:    ldr r1, [sp, #16] @ 4-byte Reload
-; THUMBV6-NEXT:    adds r6, r1, r0
-; THUMBV6-NEXT:    mov r0, r4
+; THUMBV6-NEXT:    orrs r7, r1
+; THUMBV6-NEXT:    ldr r1, [sp, #24] @ 4-byte Reload
+; THUMBV6-NEXT:    ldr r2, [sp, #12] @ 4-byte Reload
+; THUMBV6-NEXT:    adds r1, r2, r1
+; THUMBV6-NEXT:    str r1, [sp, #32] @ 4-byte Spill
+; THUMBV6-NEXT:    ldr r1, [sp, #20] @ 4-byte Reload
+; THUMBV6-NEXT:    adcs r0, r1
+; THUMBV6-NEXT:    str r0, [sp, #36] @ 4-byte Spill
+; THUMBV6-NEXT:    ldr r0, [sp, #56] @ 4-byte Reload
 ; THUMBV6-NEXT:    mov r1, r5
-; THUMBV6-NEXT:    ldr r2, [sp, #64] @ 4-byte Reload
+; THUMBV6-NEXT:    mov r2, r4
+; THUMBV6-NEXT:    mov r3, r5
+; THUMBV6-NEXT:    bl __aeabi_lmul
+; THUMBV6-NEXT:    mov r4, r1
+; THUMBV6-NEXT:    ldr r1, [sp, #28] @ 4-byte Reload
+; THUMBV6-NEXT:    adds r6, r0, r1
+; THUMBV6-NEXT:    adcs r4, r5
+; THUMBV6-NEXT:    ldr r0, [sp, #52] @ 4-byte Reload
+; THUMBV6-NEXT:    mov r1, r5
+; THUMBV6-NEXT:    ldr r2, [sp, #44] @ 4-byte Reload
+; THUMBV6-NEXT:    mov r3, r5
+; THUMBV6-NEXT:    bl __aeabi_lmul
+; THUMBV6-NEXT:    adds r0, r0, r6
+; THUMBV6-NEXT:    ldr r2, [sp, #40] @ 4-byte Reload
+; THUMBV6-NEXT:    str r0, [r2, #4]
+; THUMBV6-NEXT:    adcs r1, r5
+; THUMBV6-NEXT:    adds r0, r4, r1
+; THUMBV6-NEXT:    str r0, [sp, #28] @ 4-byte Spill
+; THUMBV6-NEXT:    mov r6, r5
+; THUMBV6-NEXT:    adcs r6, r5
+; THUMBV6-NEXT:    ldr r0, [sp, #56] @ 4-byte Reload
+; THUMBV6-NEXT:    mov r1, r5
+; THUMBV6-NEXT:    ldr r4, [sp, #44] @ 4-byte Reload
+; THUMBV6-NEXT:    mov r2, r4
 ; THUMBV6-NEXT:    mov r3, r5
 ; THUMBV6-NEXT:    bl __aeabi_lmul
-; THUMBV6-NEXT:    adds r1, r1, r6
+; THUMBV6-NEXT:    ldr r2, [sp, #28] @ 4-byte Reload
+; THUMBV6-NEXT:    adds r0, r0, r2
+; THUMBV6-NEXT:    str r0, [sp, #28] @ 4-byte Spill
+; THUMBV6-NEXT:    adcs r1, r6
+; THUMBV6-NEXT:    str r1, [sp, #24] @ 4-byte Spill
+; THUMBV6-NEXT:    ldr r0, [sp, #48] @ 4-byte Reload
+; THUMBV6-NEXT:    mov r1, r4
 ; THUMBV6-NEXT:    mov r2, r5
-; THUMBV6-NEXT:    adcs r2, r5
-; THUMBV6-NEXT:    orrs r2, r7
-; THUMBV6-NEXT:    ldr r6, [sp, #60] @ 4-byte Reload
-; THUMBV6-NEXT:    ldr r3, [sp, #40] @ 4-byte Reload
-; THUMBV6-NEXT:    orrs r6, r3
-; THUMBV6-NEXT:    subs r3, r6, #1
-; THUMBV6-NEXT:    sbcs r6, r3
-; THUMBV6-NEXT:    ldr r3, [sp, #56] @ 4-byte Reload
-; THUMBV6-NEXT:    orrs r4, r3
-; THUMBV6-NEXT:    subs r3, r4, #1
-; THUMBV6-NEXT:    sbcs r4, r3
-; THUMBV6-NEXT:    ands r4, r6
-; THUMBV6-NEXT:    orrs r4, r2
+; THUMBV6-NEXT:    mov r3, r5
+; THUMBV6-NEXT:    bl __aeabi_lmul
+; THUMBV6-NEXT:    mov r6, r0
+; THUMBV6-NEXT:    mov r4, r1
+; THUMBV6-NEXT:    ldr r0, [sp, #52] @ 4-byte Reload
+; THUMBV6-NEXT:    ldr r1, [sp, #56] @ 4-byte Reload
+; THUMBV6-NEXT:    mov r2, r5
+; THUMBV6-NEXT:    mov r3, r5
+; THUMBV6-NEXT:    bl __aeabi_lmul
+; THUMBV6-NEXT:    adds r0, r0, r6
+; THUMBV6-NEXT:    adcs r1, r4
+; THUMBV6-NEXT:    ldr r2, [sp, #28] @ 4-byte Reload
+; THUMBV6-NEXT:    adds r0, r2, r0
 ; THUMBV6-NEXT:    ldr r2, [sp, #24] @ 4-byte Reload
-; THUMBV6-NEXT:    orrs r4, r2
-; THUMBV6-NEXT:    ldr r2, [sp, #36] @ 4-byte Reload
-; THUMBV6-NEXT:    adds r0, r0, r2
-; THUMBV6-NEXT:    ldr r2, [sp, #32] @ 4-byte Reload
 ; THUMBV6-NEXT:    adcs r1, r2
-; THUMBV6-NEXT:    ldr r2, [sp, #48] @ 4-byte Reload
-; THUMBV6-NEXT:    adds r0, r2, r0
-; THUMBV6-NEXT:    ldr r2, [sp, #44] @ 4-byte Reload
+; THUMBV6-NEXT:    ldr r2, [sp, #32] @ 4-byte Reload
+; THUMBV6-NEXT:    adds r0, r0, r2
+; THUMBV6-NEXT:    ldr r2, [sp, #40] @ 4-byte Reload
 ; THUMBV6-NEXT:    str r0, [r2, #8]
-; THUMBV6-NEXT:    ldr r0, [sp, #52] @ 4-byte Reload
+; THUMBV6-NEXT:    ldr r0, [sp, #36] @ 4-byte Reload
 ; THUMBV6-NEXT:    adcs r1, r0
 ; THUMBV6-NEXT:    str r1, [r2, #12]
 ; THUMBV6-NEXT:    adcs r5, r5
-; THUMBV6-NEXT:    orrs r5, r4
+; THUMBV6-NEXT:    orrs r5, r7
 ; THUMBV6-NEXT:    movs r0, #1
 ; THUMBV6-NEXT:    ands r0, r5
 ; THUMBV6-NEXT:    strb r0, [r2, #16]
-; THUMBV6-NEXT:    add sp, #68
+; THUMBV6-NEXT:    add sp, #60
 ; THUMBV6-NEXT:    pop {r4, r5, r6, r7, pc}
 start:
   %0 = tail call { i128, i1 } @llvm.umul.with.overflow.i128(i128 %l, i128 %r) #2
   %1 = extractvalue { i128, i1 } %0, 0
   %2 = extractvalue { i128, i1 } %0, 1
   %3 = zext i1 %2 to i8
   %4 = insertvalue { i128, i8 } undef, i128 %1, 0
   %5 = insertvalue { i128, i8 } %4, i8 %3, 1

llvm/test/CodeGen/Thumb2/umulo-128-legalisation-lowering.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc < %s -mtriple=thumbv7-unknown-none-gnueabi | FileCheck %s --check-prefixes=THUMBV7
 
 define { i128, i8 } @muloti_test(i128 %l, i128 %r) unnamed_addr #0 {
 ; THUMBV7-LABEL: muloti_test:
 ; THUMBV7:       @ %bb.0: @ %start
 ; THUMBV7-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; THUMBV7-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; THUMBV7-NEXT:    .pad #44
 ; THUMBV7-NEXT:    sub sp, #44
+; THUMBV7-NEXT:    ldr.w lr, [sp, #88]
+; THUMBV7-NEXT:    mov r11, r0
+; THUMBV7-NEXT:    ldr r4, [sp, #96]
+; THUMBV7-NEXT:    ldr.w r12, [sp, #80]
+; THUMBV7-NEXT:    umull r1, r5, r2, lr
+; THUMBV7-NEXT:    umull r7, r6, r3, r4
+; THUMBV7-NEXT:    str r1, [sp, #40] @ 4-byte Spill
+; THUMBV7-NEXT:    ldr r1, [sp, #100]
+; THUMBV7-NEXT:    umull r4, r0, r4, r2
+; THUMBV7-NEXT:    str r7, [sp, #32] @ 4-byte Spill
+; THUMBV7-NEXT:    umull r7, r1, r1, r2
+; THUMBV7-NEXT:    str r4, [sp, #24] @ 4-byte Spill
+; THUMBV7-NEXT:    str r0, [sp, #12] @ 4-byte Spill
+; THUMBV7-NEXT:    ldr r0, [sp, #84]
+; THUMBV7-NEXT:    str r7, [sp, #20] @ 4-byte Spill
+; THUMBV7-NEXT:    ldr r7, [sp, #92]
+; THUMBV7-NEXT:    umull r10, r8, r0, lr
+; THUMBV7-NEXT:    umull r4, r9, r7, r12
+; THUMBV7-NEXT:    str r4, [sp, #8] @ 4-byte Spill
+; THUMBV7-NEXT:    umull r4, r0, r12, lr
+; THUMBV7-NEXT:    mov.w r12, #0
+; THUMBV7-NEXT:    umlal r5, r12, r3, lr
+; THUMBV7-NEXT:    str r4, [sp, #16] @ 4-byte Spill
+; THUMBV7-NEXT:    str r0, [sp, #4] @ 4-byte Spill
+; THUMBV7-NEXT:    umull r4, r2, r2, r7
+; THUMBV7-NEXT:    ldr r0, [sp, #40] @ 4-byte Reload
+; THUMBV7-NEXT:    str r4, [sp, #28] @ 4-byte Spill
+; THUMBV7-NEXT:    str r2, [sp, #36] @ 4-byte Spill
+; THUMBV7-NEXT:    str.w r0, [r11]
+; THUMBV7-NEXT:    ldr r0, [sp, #32] @ 4-byte Reload
+; THUMBV7-NEXT:    ldr r2, [sp, #20] @ 4-byte Reload
+; THUMBV7-NEXT:    add r2, r0
+; THUMBV7-NEXT:    ldr r0, [sp, #12] @ 4-byte Reload
+; THUMBV7-NEXT:    adds.w lr, r0, r2
+; THUMBV7-NEXT:    mov.w r2, #0
+; THUMBV7-NEXT:    adc r0, r2, #0
+; THUMBV7-NEXT:    str r0, [sp, #32] @ 4-byte Spill
+; THUMBV7-NEXT:    ldr r0, [sp, #8] @ 4-byte Reload
+; THUMBV7-NEXT:    add.w r4, r10, r0
+; THUMBV7-NEXT:    ldr r0, [sp, #4] @ 4-byte Reload
+; THUMBV7-NEXT:    adds r4, r4, r0
+; THUMBV7-NEXT:    adc r0, r2, #0
 ; THUMBV7-NEXT:    str r0, [sp, #40] @ 4-byte Spill
-; THUMBV7-NEXT:    movs r0, #0
-; THUMBV7-NEXT:    ldrd r4, r7, [sp, #88]
-; THUMBV7-NEXT:    mov r5, r3
-; THUMBV7-NEXT:    strd r0, r0, [sp, #8]
-; THUMBV7-NEXT:    mov r1, r3
-; THUMBV7-NEXT:    mov r6, r2
-; THUMBV7-NEXT:    mov r0, r2
-; THUMBV7-NEXT:    movs r2, #0
-; THUMBV7-NEXT:    movs r3, #0
-; THUMBV7-NEXT:    strd r4, r7, [sp]
-; THUMBV7-NEXT:    bl __multi3
-; THUMBV7-NEXT:    strd r1, r0, [sp, #32] @ 8-byte Folded Spill
-; THUMBV7-NEXT:    strd r3, r2, [sp, #24] @ 8-byte Folded Spill
-; THUMBV7-NEXT:    ldrd r2, r0, [sp, #96]
-; THUMBV7-NEXT:    ldr.w r9, [sp, #80]
-; THUMBV7-NEXT:    umull lr, r0, r0, r6
-; THUMBV7-NEXT:    ldr.w r11, [sp, #84]
-; THUMBV7-NEXT:    umull r3, r1, r5, r2
-; THUMBV7-NEXT:    umull r2, r12, r2, r6
-; THUMBV7-NEXT:    add r3, lr
-; THUMBV7-NEXT:    umull r8, r10, r7, r9
-; THUMBV7-NEXT:    str r2, [sp, #20] @ 4-byte Spill
-; THUMBV7-NEXT:    adds.w lr, r12, r3
-; THUMBV7-NEXT:    umull r6, r9, r9, r4
-; THUMBV7-NEXT:    mov.w r3, #0
-; THUMBV7-NEXT:    adc r12, r3, #0
-; THUMBV7-NEXT:    umull r2, r4, r11, r4
-; THUMBV7-NEXT:    add r2, r8
-; THUMBV7-NEXT:    mov.w r8, #0
-; THUMBV7-NEXT:    adds.w r2, r2, r9
-; THUMBV7-NEXT:    adc r9, r3, #0
-; THUMBV7-NEXT:    ldr r3, [sp, #20] @ 4-byte Reload
-; THUMBV7-NEXT:    adds r3, r3, r6
-; THUMBV7-NEXT:    ldr r6, [sp, #28] @ 4-byte Reload
-; THUMBV7-NEXT:    adc.w r2, r2, lr
-; THUMBV7-NEXT:    adds r3, r3, r6
-; THUMBV7-NEXT:    ldr r6, [sp, #24] @ 4-byte Reload
-; THUMBV7-NEXT:    adcs r2, r6
-; THUMBV7-NEXT:    ldrd r6, lr, [sp, #36] @ 8-byte Folded Reload
-; THUMBV7-NEXT:    str.w r6, [lr]
-; THUMBV7-NEXT:    adc r8, r8, #0
-; THUMBV7-NEXT:    ldr r6, [sp, #32] @ 4-byte Reload
-; THUMBV7-NEXT:    cmp r5, #0
-; THUMBV7-NEXT:    strd r6, r3, [lr, #4]
-; THUMBV7-NEXT:    str.w r2, [lr, #12]
-; THUMBV7-NEXT:    it ne
-; THUMBV7-NEXT:    movne r5, #1
-; THUMBV7-NEXT:    ldr r2, [sp, #100]
-; THUMBV7-NEXT:    cmp r2, #0
-; THUMBV7-NEXT:    mov r3, r2
-; THUMBV7-NEXT:    it ne
-; THUMBV7-NEXT:    movne r3, #1
-; THUMBV7-NEXT:    cmp r0, #0
-; THUMBV7-NEXT:    it ne
-; THUMBV7-NEXT:    movne r0, #1
+; THUMBV7-NEXT:    ldr r0, [sp, #24] @ 4-byte Reload
+; THUMBV7-NEXT:    ldr r2, [sp, #16] @ 4-byte Reload
+; THUMBV7-NEXT:    adds.w r10, r2, r0
+; THUMBV7-NEXT:    mov r2, r3
+; THUMBV7-NEXT:    adc.w r0, r4, lr
+; THUMBV7-NEXT:    ldr.w lr, [sp, #100]
 ; THUMBV7-NEXT:    cmp r1, #0
-; THUMBV7-NEXT:    and.w r3, r3, r5
+; THUMBV7-NEXT:    str r0, [sp, #24] @ 4-byte Spill
 ; THUMBV7-NEXT:    it ne
 ; THUMBV7-NEXT:    movne r1, #1
-; THUMBV7-NEXT:    orrs r0, r3
-; THUMBV7-NEXT:    cmp r7, #0
-; THUMBV7-NEXT:    orr.w r0, r0, r1
+; THUMBV7-NEXT:    cmp r3, #0
+; THUMBV7-NEXT:    mov r0, lr
 ; THUMBV7-NEXT:    it ne
-; THUMBV7-NEXT:    movne r7, #1
-; THUMBV7-NEXT:    cmp.w r11, #0
-; THUMBV7-NEXT:    mov r1, r11
+; THUMBV7-NEXT:    movne r2, #1
+; THUMBV7-NEXT:    cmp.w lr, #0
 ; THUMBV7-NEXT:    it ne
-; THUMBV7-NEXT:    movne r1, #1
-; THUMBV7-NEXT:    cmp r4, #0
-; THUMBV7-NEXT:    ldr r3, [sp, #96]
+; THUMBV7-NEXT:    movne r0, #1
+; THUMBV7-NEXT:    ldr r4, [sp, #28] @ 4-byte Reload
+; THUMBV7-NEXT:    ands r0, r2
+; THUMBV7-NEXT:    orrs r1, r0
+; THUMBV7-NEXT:    adds r5, r5, r4
+; THUMBV7-NEXT:    str.w r5, [r11, #4]
+; THUMBV7-NEXT:    ldr r0, [sp, #36] @ 4-byte Reload
+; THUMBV7-NEXT:    mov.w r5, #0
+; THUMBV7-NEXT:    adcs.w r0, r0, r12
+; THUMBV7-NEXT:    adc r2, r5, #0
+; THUMBV7-NEXT:    cmp r6, #0
+; THUMBV7-NEXT:    it ne
+; THUMBV7-NEXT:    movne r6, #1
+; THUMBV7-NEXT:    orrs r1, r6
+; THUMBV7-NEXT:    ldr r6, [sp, #84]
+; THUMBV7-NEXT:    umlal r0, r2, r3, r7
+; THUMBV7-NEXT:    ldr r3, [sp, #32] @ 4-byte Reload
+; THUMBV7-NEXT:    cmp r7, #0
 ; THUMBV7-NEXT:    it ne
-; THUMBV7-NEXT:    movne r4, #1
-; THUMBV7-NEXT:    cmp.w r10, #0
-; THUMBV7-NEXT:    and.w r1, r1, r7
-; THUMBV7-NEXT:    it ne
-; THUMBV7-NEXT:    movne.w r10, #1
-; THUMBV7-NEXT:    orrs r3, r2
-; THUMBV7-NEXT:    ldr r2, [sp, #80]
-; THUMBV7-NEXT:    orr.w r1, r1, r4
-; THUMBV7-NEXT:    orr.w r1, r1, r10
+; THUMBV7-NEXT:    movne r7, #1
+; THUMBV7-NEXT:    orrs r1, r3
+; THUMBV7-NEXT:    mov r3, r6
+; THUMBV7-NEXT:    cmp r6, #0
 ; THUMBV7-NEXT:    it ne
 ; THUMBV7-NEXT:    movne r3, #1
-; THUMBV7-NEXT:    orrs.w r7, r2, r11
-; THUMBV7-NEXT:    orr.w r1, r1, r9
+; THUMBV7-NEXT:    cmp.w r8, #0
+; THUMBV7-NEXT:    and.w r3, r3, r7
+; THUMBV7-NEXT:    ldr r7, [sp, #80]
+; THUMBV7-NEXT:    it ne
+; THUMBV7-NEXT:    movne.w r8, #1
+; THUMBV7-NEXT:    cmp.w r9, #0
+; THUMBV7-NEXT:    it ne
+; THUMBV7-NEXT:    movne.w r9, #1
+; THUMBV7-NEXT:    orrs r7, r6
+; THUMBV7-NEXT:    ldr r6, [sp, #96]
 ; THUMBV7-NEXT:    it ne
 ; THUMBV7-NEXT:    movne r7, #1
-; THUMBV7-NEXT:    ands r3, r7
-; THUMBV7-NEXT:    orr.w r0, r0, r12
-; THUMBV7-NEXT:    orrs r1, r3
+; THUMBV7-NEXT:    orr.w r3, r3, r8
+; THUMBV7-NEXT:    orrs.w r6, r6, lr
+; THUMBV7-NEXT:    orr.w r3, r3, r9
+; THUMBV7-NEXT:    it ne
+; THUMBV7-NEXT:    movne r6, #1
+; THUMBV7-NEXT:    adds.w r0, r0, r10
+; THUMBV7-NEXT:    str.w r0, [r11, #8]
+; THUMBV7-NEXT:    ldr r0, [sp, #24] @ 4-byte Reload
+; THUMBV7-NEXT:    adcs r0, r2
+; THUMBV7-NEXT:    str.w r0, [r11, #12]
+; THUMBV7-NEXT:    ldr r0, [sp, #40] @ 4-byte Reload
+; THUMBV7-NEXT:    and.w r2, r7, r6
+; THUMBV7-NEXT:    orr.w r0, r0, r3
+; THUMBV7-NEXT:    orr.w r0, r0, r2
+; THUMBV7-NEXT:    orr.w r0, r0, r1
+; THUMBV7-NEXT:    adc r1, r5, #0
 ; THUMBV7-NEXT:    orrs r0, r1
-; THUMBV7-NEXT:    orr.w r0, r0, r8
 ; THUMBV7-NEXT:    and r0, r0, #1
-; THUMBV7-NEXT:    strb.w r0, [lr, #16]
+; THUMBV7-NEXT:    strb.w r0, [r11, #16]
 ; THUMBV7-NEXT:    add sp, #44
 ; THUMBV7-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 start:
   %0 = tail call { i128, i1 } @llvm.umul.with.overflow.i128(i128 %l, i128 %r) #2
   %1 = extractvalue { i128, i1 } %0, 0
   %2 = extractvalue { i128, i1 } %0, 1
   %3 = zext i1 %2 to i8
   %4 = insertvalue { i128, i8 } undef, i128 %1, 0