Allow rematerialization of virtual reg uses

[rampitec]

Summary

Currently isReallyTriviallyReMaterializableGeneric() implementation
prevents rematerialization on any virtual register use on the grounds
that is not a trivial rematerialization and that we do not want to
extend liveranges.

It appears that LRE logic does not attempt to extend a liverange of
a source register for rematerialization so that is not an issue.
That is checked in the LiveRangeEdit::allUsesAvailableAt().

The only non-trivial aspect of it is accounting for tied-defs which
normally represent a read-modify-write operation and not rematerializable.

The test for a tied-def situation already exists in the
/CodeGen/AMDGPU/remat-vop.mir,
test_no_remat_v_cvt_f32_i32_sdwa_dst_unused_preserve.

The change has affected ARM/Thumb, Mips, RISCV, and x86. For the targets
where I more or less understand the asm it seems to reduce spilling
(as expected) or be neutral. However, it needs a review by all targets'
specialists.

[rampitec]

Note, it needs D106396 to prevent rematerialization in coalescer if not all registers are available.

[asb]

From an initial look at the RISC-V test changes, none of them seem concerning to me.

[RKSimon]

llvm/test/CodeGen/X86/licm-regpressure.ll

13

Add checks for the lea?

[jrtc27]

llvm/test/CodeGen/X86/licm-regpressure.ll

13

Or fix the test so it still demonstrates the bug, otherwise this file no longer serves much of a purpose...

[rampitec]

llvm/test/CodeGen/X86/licm-regpressure.ll

13

The question is: should LICM really check register pressure if we can rely on rematerialization instead? I have started this with a very similar situation in AMDGPU, LICM was hoisting instructions out of the loop and we end up spilling in the loop. A proper rematerialization fixes this without burdening the LICM. I.e. this change shall probably fix the bug itself.

[jrtc27]

llvm/test/CodeGen/X86/licm-regpressure.ll

13

If this fixes the bug completely then rewrite the comments to reflect that and tag the revision as closing the bug. If it doesn't fix the bug in all cases then this test needs updating to be a case that's still broken.

[efriedma]

llvm/test/CodeGen/Thumb2/LowOverheadLoops/memcall.ll

8 ↗

(On Diff #360302)

This looks like a regression. Not sure what's happening here; we're saving both r7 and r8, but they aren't used. Maybe something related to the hardware loop instructions?

[rampitec]

Updated test X86/licm-regpressure.ll

[rampitec]

llvm/test/CodeGen/X86/licm-regpressure.ll

13

Looking closely I think we cannot reproduce the bug with the asm inspection because rematerialization mitigates it. However, the issue reported in the PR23143 still exist. This can be explored with MIR inspection. Updated test accordingly.

[rampitec]

llvm/test/CodeGen/Thumb2/LowOverheadLoops/memcall.ll

8 ↗

(On Diff #360302)

Here is what happens, these two instructions are now isTriviallyReMaterializable():

%16:rgpr = t2ADDri %8:rgpr, 15, 14, $noreg, $noreg
%17:rgpr = t2LSRri %16:rgpr, 4, 14, $noreg, $noreg

MachineLICM hoists these out of the loop because of that. RA uses r8 and r8 ends up in the frame setup:

bb.0.entry:
  successors: %bb.1(0x50000000), %bb.2(0x30000000); %bb.1(62.50%), %bb.2(37.50%)
  liveins: $r0, $r1, $r2, $r3, $r4, $r5, $r6, $r7, $r8, $lr
  $sp = frame-setup t2STMDB_UPD $sp(tied-def 0), 14, $noreg, killed $r4, killed $r5, killed $r6, killed $r7, killed $r8, killed $lr
  frame-setup CFI_INSTRUCTION def_cfa_offset 24
  frame-setup CFI_INSTRUCTION offset $lr, -4
  frame-setup CFI_INSTRUCTION offset $r8, -8
  frame-setup CFI_INSTRUCTION offset $r7, -12
  frame-setup CFI_INSTRUCTION offset $r6, -16
  frame-setup CFI_INSTRUCTION offset $r5, -20
  frame-setup CFI_INSTRUCTION offset $r4, -24
  t2CMPri renamable $r2, 1, 14, $noreg, implicit-def $cpsr
  t2Bcc %bb.2, 11, killed $cpsr
  t2B %bb.1, 14, $noreg

bb.1.for.body.preheader:
; predecessors: %bb.0
  successors: %bb.3(0x80000000); %bb.3(100.00%)
  liveins: $r0, $r1, $r2, $r3
  renamable $r12 = nsw t2LSLri renamable $r3, 2, 14, $noreg, $noreg
  renamable $r4 = t2MOVi 0, 14, $noreg, $noreg
  renamable $r7 = t2ADDri renamable $r3, 15, 14, $noreg, $noreg
  renamable $r8 = t2LSRri killed renamable $r7, 4, 14, $noreg, $noreg
  t2B %bb.3, 14, $noreg

But it is eliminated by the ARM Low Overhead Loops pass:

# *** IR Dump After ARM Low Overhead Loops pass (arm-low-overhead-loops) ***:

bb.1.for.body.preheader:
; predecessors: %bb.0
  successors: %bb.2(0x80000000); %bb.2(100.00%)
  liveins: $r0, $r1, $r2, $r3
  renamable $r12 = nsw t2LSLri renamable $r3, 2, 14, $noreg, $noreg
  renamable $r4, dead $cpsr = tMOVi8 0, 14, $noreg
  tB %bb.2, 14, $noreg

Frame setup however already done and not updated.

Maybe pass an extra argument into isTriviallyReMaterializable()? That could return false if any virtual registers are used for the purpose of MachineLICM and CalcSpillWeights and only return true for the regalloc/coalescer itself.

[efriedma]

llvm/test/CodeGen/Thumb2/LowOverheadLoops/memcall.ll

8 ↗

(On Diff #360302)

That's unfortunate.

In general, the hoisting is probably fine. The problem here is that if the instructions are used as input to the low-overhead loop pseudo-instructions, we don't want to hoist them: they're likely to be eliminated by the LowOverheadLoops pass, so it isn't profitable. (The low-overhead loop instructions get formed very late because they have odd restrictions on what branches are allowed.)

[rampitec]

llvm/test/CodeGen/Thumb2/LowOverheadLoops/memcall.ll

8 ↗

(On Diff #360302)

I am not really sure it is fine to hoist in this case because a rematerialization of an instruction with vreg uses is not guaranteed. All used registers must be available at the point of rematerialization and if not it simply will not happen. MachineLICM considers rematerialization as granted, but in this case it is rather opportunistic.

I am looking at the change not to hoist such instructions right now.

[rampitec]

Added argument to isTriviallyReMaterializable() to prevent hoisting of instructions in an expectation these can be later rematerialized. Only opportunistic rematerialization in RA is allowed.

[rampitec]

@arsenm note, we are reporting that instructions with such uses are trivially materializable for a long time in case of VALU moves, and now even with more VALU instructions. We only do not so it for SALU. I think we will need to have the same check for AllowVRegs in our isReallyTriviallyReMaterializable to prevent LICM hoisting it under high pressure.

[rampitec]

In D106408#2897997, @rampitec wrote:

@arsenm note, we are reporting that instructions with such uses are trivially materializable for a long time in case of VALU moves, and now even with more VALU instructions. We only do not so it for SALU. I think we will need to have the same check for AllowVRegs in our isReallyTriviallyReMaterializable to prevent LICM hoisting it under high pressure.

Although that is practically unexploitable because MachineLICM always thinks there is no high pressure, whatever SALU I feed it from out target, and because MachineLoop::isLoopInvariant() always returns false for any VALU because of the exec use. So it always hoists SALU, no matter what, and never hoists VALU. I.e. on practice this change should not pessimize our codegen, because it will not affect LICM for us.

[dmgreen]

Tail predication being awkward aside - do you have performance results for this patch? Something to show it is beneficial to do over a set of benchmarks.

[luismarques]

In D106408#2903678, @dmgreen wrote:

Tail predication being awkward aside - do you have performance results for this patch? Something to show it is beneficial to do over a set of benchmarks.

I had earlier benchmarked this patch for an embedded RISC-V system (Ibex simple system). Not a comprehensive set of results at all, but in case it's useful:

• For CoreMark it had negligible size impact. At Oz and Os it had a small performance benefit (+1.17% and +0.178%, respectively). At O2 and O3 the performance difference was negligible (-0.016% and +0.038%, respectively).
• For Embench it had no performance impact but improved size a bit, overall. The best case was O2, where the size was overall reduced by 440 bytes, or 0.276%. Most programs of that benchmark suite had minor differences, some positive and some negative, but for picojpeg it shaved up to 384 bytes.

(Raw results here, in a confusing format: https://gist.github.com/luismarques/01948685c6a1d16ab1c1a0229252b0f1. Please ignore the absolute performance values).

[rampitec]

In D106408#2904058, @luismarques wrote:

In D106408#2903678, @dmgreen wrote:

Tail predication being awkward aside - do you have performance results for this patch? Something to show it is beneficial to do over a set of benchmarks.

I had earlier benchmarked this patch for an embedded RISC-V system (Ibex simple system). Not a comprehensive set of results at all, but in case it's useful:

• For CoreMark it had negligible size impact. At Oz and Os it had a small performance benefit (+1.17% and +0.178%, respectively). At O2 and O3 the performance difference was negligible (-0.016% and +0.038%, respectively).
• For Embench it had no performance impact but improved size a bit, overall. The best case was O2, where the size was overall reduced by 440 bytes, or 0.276%. Most programs of that benchmark suite had minor differences, some positive and some negative, but for picojpeg it shaved up to 384 bytes.

(Raw results here, in a confusing format: https://gist.github.com/luismarques/01948685c6a1d16ab1c1a0229252b0f1. Please ignore the absolute performance values).

Thanks for measuring this!

For AMDGPU target there is no visible change just yet. It will need at least one more patch to make a difference there. The reason is we are doing this for years now because of the custom isReallyTriviallyReMaterializable implementation which skips the check for virtual registers. That needs to be updated in a way similar to this patch so we skip MachineLICM hoisting on the grounds that an instruction is rematerializable without a check that rematerialization will really happen. Then I have no means to measure other targets.

That said AMDGPU is a target where any spilling is extremely expensive. Not just expensive like on CPU targets, but way more expensive. What I see as a result of this patch is a decrease of spilling across the board, which must be a good thing even for a less sensitive target. Code size might increase though, which is generally a case with any rematerialization.

[rampitec]

Rebased.

[dmgreen]

For AMDGPU target there is no visible change just yet. It will need at least one more patch to make a difference there. The reason is we are doing this for years now because of the custom isReallyTriviallyReMaterializable implementation which skips the check for virtual registers. That needs to be updated in a way similar to this patch so we skip MachineLICM hoisting on the grounds that an instruction is rematerializable without a check that rematerialization will really happen. Then I have no means to measure other targets.

That said AMDGPU is a target where any spilling is extremely expensive. Not just expensive like on CPU targets, but way more expensive. What I see as a result of this patch is a decrease of spilling across the board, which must be a good thing even for a less sensitive target. Code size might increase though, which is generally a case with any rematerialization.

OK. I was hoping you would be able to say this is very important for your performance, and have some data to back that up.

The results I have here look less great.. but looking again, in general it looks OK. Codesize was either flat or smaller by a little, which is good. Performance went up and down depending on the test. The results are not great in places, but looking at the some of the decreases it may just be unlucky, spilling in hot loops where it didn't in the past and behaving a bit chaotically from differences in register allocation. There are some improvements too, to make it generally OK overall.

llvm/test/CodeGen/Thumb2/ldr-str-imm12.ll

103–104

Change this to check for a strd? Or just update the check lines.

[rampitec]

In D106408#2924775, @dmgreen wrote:

OK. I was hoping you would be able to say this is very important for your performance, and have some data to back that up.

I'd love to say that but in fact we have a gordian knot of problems and AMDGPU is the least affected BE by this change:

• We are already doing it for VOP instructions which is the most important kind for AMDGPU, since our implementation of isReallyTriviallyReMaterializable does not check for virtual registers unlike isReallyTriviallyReMaterializableGeneric, so no change here.
• The other kind of rematerializable instructions are SOP, but most of them either cannot be rematerilized at all because of physreg defs or because of subreg defs, so the impact on SOP will not be huge.
• There would be a difference if MachineLICM hoisted VOP instructions and then it would be impossible to rematerialize them, but that simply does not happen because all VOPs have implicit $exec use and it avoids hoisting anyway.
• The latter is very simple to fix with the new isIgnorableUse target callback, but if I do it without this patch it will blindly hoist instructions which will not be remateraialized later.
• In fact MachineLICM needs to have register pressure impact estimate improved, way improved. So if even after all of that it still may have negative impact on performance if such hoisting is enabled.

All of that seems to need a lot of cleanup.

[rampitec]

Updated thumb2/ldr-str-imm12.ll test checks.

[dmgreen]

Thanks. This sounds OK for ARM to me.

llvm/test/CodeGen/Thumb2/ldr-str-imm12.ll

100

On second thoughts, can you just run the update_llc_test_checks on this file?

The strd is just two adjacent str glommed together, so this is checking for more stores than it was before. But there appear to be extra stores in the original, including stm. From the comment at the top of the file it is not very clear what this is trying to test now. Just updating the test checks so that we see the whole function sounds like the best idea.

[rampitec]

llvm/test/CodeGen/Thumb2/ldr-str-imm12.ll

100

Maybe switch to generated checks, precommit and update the test in the review? That way you will better see what has changed. That is more or less what I've been doing looking at it.

JFYI I am not good in thumb isa, so I am guessing what has changed.

[rampitec]

Rebased on top of D107590 and regenerated ldr-str-imm12.ll checks.

[dmgreen]

llvm/test/CodeGen/Thumb2/ldr-str-imm12.ll

100

Yeah, I had taken a look to see how it's changing. It looks fine to me.

Thanks for updating the test checks. It looks like a more useful test now.

[rampitec]

Rebased.

[arsenm]

llvm/include/llvm/CodeGen/TargetInstrInfo.h

120–125

I don't understand why AllowVRegs needs to be a parameter. Why wouldn't it just always true?

120–125

i.e. this is rematerializable, the pressure question is a different heuristic

[rampitec]

llvm/include/llvm/CodeGen/TargetInstrInfo.h

120–125

The problem is licm will hoist instructions if rematerializable even without checking the pressure (not saying its pressure calculation is not great) simply because if we run out of registers it will be easy to rematerialize the instruction. This is not the case with vreg uses, even if possible RA will not do it if it needs extending LR. The rematerialization in RA is opportunistic and not granted. Besides it is not even always possible to rematerialize such instructions even with all checks passed.

[arsenm]

llvm/include/llvm/CodeGen/TargetInstrInfo.h

120–125

But why can't licm check if there are vregs on the instruction itself? Why thread this through the legality check?

[rampitec]

llvm/include/llvm/CodeGen/TargetInstrInfo.h

120–125

A good idea, thanks! I can do it as a separate patch preceding this one and instead of D107677.

[rampitec]

Avoid predication for MachineLICM. MachineLICM itself is updated instead in D107677. Rebased on top of D107677.

[rampitec]

Updated comments on isTriviallyReMaterializable and isReallyTriviallyReMaterializable. These are not exactly true for a long time already at least for AMDGPU and ARM.

[dmgreen]

Have you gone through the uses of isTriviallyReMaterializable to check they line up to the new semantics? It looks like it's used in CalcSpillWeight, LiveRangeEdits (you've fixed recently?), MachineLICM (is getting an updated costmodel), Register Coalescer, shouldRegionSplitForVirtReg, WebAssemblyRegStackifty. Are they all expected to keep working with the change here?

I feel like the new behavior isn't really "Trivial" any more, and it may be worth keeping the old method for trivial cases, which is just the base case plus a virtual reg use check. Essentially MachineLICMBase::isTriviallyReMaterializable from D107677. I'm not sure what to call the new method though, and not sure its worth it if all the uses above are OK as-is.

[rampitec]

In D106408#2946735, @dmgreen wrote:

Have you gone through the uses of isTriviallyReMaterializable to check they line up to the new semantics?

I believe I did:

It looks like it's used in CalcSpillWeight,

Checked, spill weight calculation matches what LiveRangeEdit/RA in general do. Prohibiting vreg uses there did a worse job.

LiveRangeEdits (you've fixed recently?)

LRE handles it for a long time already, it has a check in the allUsesAvailableAt(). AMDGPU allows vreg uses, also ARM for VCTP. This is already the case.

MachineLICM (is getting an updated costmodel),

Yes, D107677.

Register Coalescer,

One corner case was fixed in D106396, otherwise shall work.

shouldRegionSplitForVirtReg,

Part or RA logic, works.

WebAssemblyRegStackifty.

Allows rematerialization of CONST_* opcodes only. These have immediate operands and no register uses at all. Should work and does not have any test changes.

Are they all expected to keep working with the change here?

Overall yes. The only per-requestity is MachineLICM change in the D107677 at this point.

I feel like the new behavior isn't really "Trivial" any more, and it may be worth keeping the old method for trivial cases, which is just the base case plus a virtual reg use check. Essentially MachineLICMBase::isTriviallyReMaterializable from D107677. I'm not sure what to call the new method though, and not sure its worth it if all the uses above are OK as-is.

The uses above are OK as far as I can tell. I can see how that is possible to wrap one method into another, but as usual have problem with names. We already have isRematerializable, isTriviallyRematerializable, and isReallyTriviallyRematerializable. Creating something like isReallyReallyTriviallyRematerializable looks like going down a rabbit hole ;) But I am open to opinions.

[dmgreen]

Are they all expected to keep working with the change here?

Overall yes. The only per-requestity is MachineLICM change in the D107677 at this point.

Nice one

I feel like the new behavior isn't really "Trivial" any more, and it may be worth keeping the old method for trivial cases, which is just the base case plus a virtual reg use check. Essentially MachineLICMBase::isTriviallyReMaterializable from D107677. I'm not sure what to call the new method though, and not sure its worth it if all the uses above are OK as-is.

The uses above are OK as far as I can tell. I can see how that is possible to wrap one method into another, but as usual have problem with names. We already have isRematerializable, isTriviallyRematerializable, and isReallyTriviallyRematerializable. Creating something like isReallyReallyTriviallyRematerializable looks like going down a rabbit hole ;) But I am open to opinions.

I was thinking of something like "isSimpleRematerializable" vs "isTriviallyRematerializable" (but I didn't really like "simple"). If we don't have a user of it (other than Machine LICM which has it's own function), I'd say it is fine as-is.

Thanks for checking. Providing there are not other comment, this SGTM.

[rampitec]

Rebased

[phosek]

We started seeing Clang failure when building our codebase with ASan and bisecting identified this change as a culprit, see PR51516 for details and reproducer. Would it be possible to revert this change until that issue is resolved?

[rampitec]

In D106408#2951212, @phosek wrote:

We started seeing Clang failure when building our codebase with ASan and bisecting identified this change as a culprit, see PR51516 for details and reproducer. Would it be possible to revert this change until that issue is resolved?

Petr, I am AFK for today. Please revert it or I will do it tomorrow morning. Thanks for narrowing down.

[rampitec]

In D106408#2951212, @phosek wrote:

We started seeing Clang failure when building our codebase with ASan and bisecting identified this change as a culprit, see PR51516 for details and reproducer. Would it be possible to revert this change until that issue is resolved?

Thanks Petr. Reproduced the error.

[rampitec]

Proposed fix for https://bugs.llvm.org/show_bug.cgi?id=51516: D108475.
This needs that fix before it can be relanded.

[rampitec]

Rebased to reland.

[thegameg]

This might be related to https://bugs.llvm.org/show_bug.cgi?id=51655. Do you mind checking it out?

[rampitec]

In D106408#2970305, @thegameg wrote:

This might be related to https://bugs.llvm.org/show_bug.cgi?id=51655. Do you mind checking it out?

What happened to it? Lldb is green now. Also do you mind to attach ll to the bug? For some reason I cannot build compiler-rt with sanitizers now, still trying to sort it out. This is all not needed to check RA though, if there is an ll.

[thegameg]

In D106408#2970470, @rampitec wrote:

In D106408#2970305, @thegameg wrote:

This might be related to https://bugs.llvm.org/show_bug.cgi?id=51655. Do you mind checking it out?

What happened to it? Lldb is green now. Also do you mind to attach ll to the bug? For some reason I cannot build compiler-rt with sanitizers now, still trying to sort it out. This is all not needed to check RA though, if there is an ll.

This is an expensive check build that does verify machineinstrs after each pass (I think), so you probably won't see it on the lldb job. I'll try to grab you an ll.

[rampitec]

In D106408#2972416, @thegameg wrote:

In D106408#2970470, @rampitec wrote:

In D106408#2970305, @thegameg wrote:

This might be related to https://bugs.llvm.org/show_bug.cgi?id=51655. Do you mind checking it out?

What happened to it? Lldb is green now. Also do you mind to attach ll to the bug? For some reason I cannot build compiler-rt with sanitizers now, still trying to sort it out. This is all not needed to check RA though, if there is an ll.

This is an expensive check build that does verify machineinstrs after each pass (I think), so you probably won't see it on the lldb job. I'll try to grab you an ll.

Thanks! This object seems to build fine to me with expensive checks, so I am not able to reproduce it myself.

[rampitec]

In D106408#2972416, @thegameg wrote:

What happened to it? Lldb is green now. Also do you mind to attach ll to the bug? For some reason I cannot build compiler-rt with sanitizers now, still trying to sort it out. This is all not needed to check RA though, if there is an ll.

This is an expensive check build that does verify machineinstrs after each pass (I think), so you probably won't see it on the lldb job. I'll try to grab you an ll.

I mean, it seems intermittent. It happened with build 20218, but then 20220 didn't have it, build 20221 was green, and then it has reappeared in the https://green.lab.llvm.org/green/job/clang-stage1-cmake-RA-expensive/

[rampitec]

In D106408#2970305, @thegameg wrote:

This might be related to https://bugs.llvm.org/show_bug.cgi?id=51655. Do you mind checking it out?

This is the proposed fix: D108951

[rampitec]

In D106408#2973408, @rampitec wrote:

In D106408#2970305, @thegameg wrote:

This might be related to https://bugs.llvm.org/show_bug.cgi?id=51655. Do you mind checking it out?

This is the proposed fix: D108951

Fixed by https://reviews.llvm.org/rGd170945bb2b3a0855cea115d31d688b85ddf3dc5

[• zino]

I just want to report that I tried this PR on large internal workloads (for X86) and saw some good performance. The generated code cleaned up nicely, I saw 350K new hits almost all are lea instructions. Observed 3% to 5% reduction in executed load/stores. Can the PR get be approved/landed?

[rampitec]

In D106408#3012954, @zino wrote:

I just want to report that I tried this PR on large internal workloads (for X86) and saw some good performance. The generated code cleaned up nicely, I saw 350K new hits almost all are lea instructions. Observed 3% to 5% reduction in executed load/stores. Can the PR get be approved/landed?

Thanks fort confirming, glad to hear this! This was landed about a month ago.

[wxiao3]

hi,

There is some internal benchmark performance regression after the patch landed.
The story is as follows:
There is a virtual register: %5 which lives across many basic blocks (some are inside loops).
Before this patch is landed, its weight is: 2.299834e-03 as below:

%5 [592r,3248B:0)[3312B,3392B:0)[3456B,3616B:0)[3712B,4368B:0)[4464B,5008B:0)[5072B,5328B:0)[6208B,6480B:0)[6560B,6640B:0)[6720B,6976B:0)[7104B,8224B:0)[8336B,9104B:0)[9184B,9488B:0)[10512B,13904B:0)[14480B,16320B:0)  0@592r weight:2.299834e-03

Its definition is as below:

592B      %5:gr64 = LEA64r %523:gr64, 1, %0:gr64_nosp, 0, $noreg

After this patch is landed, its weight is reduced by half to: 1.149917e-03 as below:

%5 [592r,3248B:0)[3312B,3392B:0)[3456B,3616B:0)[3712B,4368B:0)[4464B,5008B:0)[5072B,5328B:0)[6208B,6480B:0)[6560B,6640B:0)[6720B,6976B:0)[7104B,8224B:0)[8336B,9104B:0)[9184B,9488B:0)[10512B,13904B:0)[14480B,16320B:0)  0@592r weight:1.149917e-03

Finally, %5 is evicted by other VR, which doesn't happen before this patch land.

For this case, treating %5 as ReMaterializable and reducing its weight is very bad.

Any idea to fix the performance regression?

Wei

[rampitec]

In D106408#3014403, @wxiao3 wrote:

hi,

There is some internal benchmark performance regression after the patch landed.
The story is as follows:
There is a virtual register: %5 which lives across many basic blocks (some are inside loops).
Before this patch is landed, its weight is: 2.299834e-03 as below:

%5 [592r,3248B:0)[3312B,3392B:0)[3456B,3616B:0)[3712B,4368B:0)[4464B,5008B:0)[5072B,5328B:0)[6208B,6480B:0)[6560B,6640B:0)[6720B,6976B:0)[7104B,8224B:0)[8336B,9104B:0)[9184B,9488B:0)[10512B,13904B:0)[14480B,16320B:0)  0@592r weight:2.299834e-03

Its definition is as below:

592B      %5:gr64 = LEA64r %523:gr64, 1, %0:gr64_nosp, 0, $noreg

After this patch is landed, its weight is reduced by half to: 1.149917e-03 as below:

%5 [592r,3248B:0)[3312B,3392B:0)[3456B,3616B:0)[3712B,4368B:0)[4464B,5008B:0)[5072B,5328B:0)[6208B,6480B:0)[6560B,6640B:0)[6720B,6976B:0)[7104B,8224B:0)[8336B,9104B:0)[9184B,9488B:0)[10512B,13904B:0)[14480B,16320B:0)  0@592r weight:1.149917e-03

Finally, %5 is evicted by other VR, which doesn't happen before this patch land.

For this case, treating %5 as ReMaterializable and reducing its weight is very bad.

Any idea to fix the performance regression?

Wei

It is now considered rematerializable which has halven the spill weight because it could be rematerialized instead. I guess at the end it was not rematerialized. I'd start with checking what has prevented it, but likely that is because %523 or %0 are not available at the point of rematerialization.

I was experimenting with avoiding the division by 2 at the end of the VirtRegAuxInfo::weightCalcHelper if there are vreg uses, but actually got worse results. Since this is a heuristic calculating weights it might make sense to give it some little extra weight for every vreg use I guess.

[rampitec]

In D106408#3015945, @rampitec wrote:

I was experimenting with avoiding the division by 2 at the end of the VirtRegAuxInfo::weightCalcHelper if there are vreg uses, but actually got worse results. Since this is a heuristic calculating weights it might make sense to give it some little extra weight for every vreg use I guess.

Something like this maybe if you have a good test for it: https://reviews.llvm.org/differential/diff/374324/

Obviously 0.01 there is a pure guesstimate.

[LuoYuanke]

llvm/lib/CodeGen/TargetInstrInfo.cpp

990

The comments looks reasonable to me. The use register's live range is extended if we rematerialize def register. Removing this code is not friendly to RA. Do you have any data to ensure there is no side effect but benefit by removing this code?

[rampitec]

llvm/lib/CodeGen/TargetInstrInfo.cpp

990

The comment is far from reality. RA does not extend use live ranges. Instead it checks that all uses are available at the point of rematerialization and does not rematerialize if not. There is quite a number of cases where we have less spilling now.

[wxiao3]

Obviously 0.01 there is a pure guesstimate.

To fix one of my local performance regressions, I need to set it to 0.1 at least.
But tuning the value seems to be meaningless. Because:
Without your patch, isReallyTriviallyReMaterializableGeneric return true means that the VR is definitely ReMaterializable.
With your patch, isReallyTriviallyReMaterializableGeneric return true means that the VR is probably ReMaterializable.
I don't think the change is consistent with this function's orignal design goal.
Moreover, your patch conflicts with our target implementation: X86InstrInfo::isReallyTriviallyReMaterializable (llvm/lib/Target/X86/X86InstrInfo.cpp). E.g., for most LEA instructions, our implementation will return false. While your patch will always return true if LEA is using virtual registers. Your patch makes our target implementation useless anymore.

it seems to reduce spilling (as expected) or be neutral

I don't observe any data to prove it's good for performance in our internal performance track system running on various hardware. Instead, your patch brings -1.5% drop for CPU2017rerf/557.xz on server side and -2% drop for coremark-pro/core on desktop side. In scenario with big register pressure such as loops with a lot of LiveIntervals, optimistically assuming critical VR (LiveInterval) to be ReMaterializable (but it's not in reality) will make RA tend to spill the critical VR, which is definitely bad for performance.

Could you please revert this change or at least make it not impact our X86 CPU BE? I don't observe the conflicted change bring any benefit but It results at least 2 regressions for X86 CPU BE.

[rampitec]

In D106408#3020762, @wxiao3 wrote:

Obviously 0.01 there is a pure guesstimate.

To fix one of my local performance regressions, I need to set it to 0.1 at least.
But tuning the value seems to be meaningless. Because:
Without your patch, isReallyTriviallyReMaterializableGeneric return true means that the VR is definitely ReMaterializable.
With your patch, isReallyTriviallyReMaterializableGeneric return true means that the VR is probably ReMaterializable.
I don't think the change is consistent with this function's orignal design goal.
Moreover, your patch conflicts with our target implementation: X86InstrInfo::isReallyTriviallyReMaterializable (llvm/lib/Target/X86/X86InstrInfo.cpp). E.g., for most LEA instructions, our implementation will return false. While your patch will always return true if LEA is using virtual registers. Your patch makes our target implementation useless anymore.

it seems to reduce spilling (as expected) or be neutral

I don't observe any data to prove it's good for performance in our internal performance track system running on various hardware. Instead, your patch brings -1.5% drop for CPU2017rerf/557.xz on server side and -2% drop for coremark-pro/core on desktop side. In scenario with big register pressure such as loops with a lot of LiveIntervals, optimistically assuming critical VR (LiveInterval) to be ReMaterializable (but it's not in reality) will make RA tend to spill the critical VR, which is definitely bad for performance.

Could you please revert this change or at least make it not impact our X86 CPU BE? I don't observe the conflicted change bring any benefit but It results at least 2 regressions for X86 CPU BE.

Since this is second perf regression report I will revert it.

[rampitec]

I still consider this a good experiment as it has uncovered couple RA bugs in the process.

[wxiao3]

In D106408#3021110, @rampitec wrote:

I still consider this a good experiment as it has uncovered couple RA bugs in the process.

Yes, It's a great experiment.

Thanks!
Wei

[vporpo]

Just to add that this patch caused a huge compilation time increase in some workloads, with the majority of time spent in eliminateDeadDefs() called by rematerializeAll(). The compilation time regression was gone after this patch was reverted.

[fhahn]

In D106408#3056901, @vporpo wrote:

Just to add that this patch caused a huge compilation time increase in some workloads, with the majority of time spent in eliminateDeadDefs() called by rematerializeAll(). The compilation time regression was gone after this patch was reverted.

Would it be possible to share a reproducer for the issue? Without a reprocducer, it will be hard to track down the compile-time issue.

[Carrot]

@rampitec, any follow up on this patch?

[rampitec]

In D106408#3516718, @Carrot wrote:

@rampitec, any follow up on this patch?

Not planned. It is reverted and AMDGPU specific patch was landed instead. I do not have a capacity to fix all regressions including performance on all targets.

llvm/include/llvm/CodeGen/TargetInstrInfo.h

public:
   /// Given a machine instruction descriptor, returns the register
   /// class constraint for OpNum, or NULL.
   virtual
   const TargetRegisterClass *getRegClass(const MCInstrDesc &MCID, unsigned OpNum,
                                          const TargetRegisterInfo *TRI,
                                          const MachineFunction &MF) const;
 
   /// Return true if the instruction is trivially rematerializable, meaning it
-  /// has no side effects and requires no operands that aren't always available.
-  /// This means the only allowed uses are constants and unallocatable physical
-  /// registers so that the instructions result is independent of the place
-  /// in the function.
+  /// has no side effects. Uses of constants and unallocatable physical
+  /// registers are always trivial to rematerialize so that the instructions
+  /// result is independent of the place in the function. Uses of virtual
+  /// registers are allowed but it is caller's responsility to ensure these
+  /// operands are valid at the point the instruction is beeing moved.
   bool isTriviallyReMaterializable(const MachineInstr &MI,
                                    AAResults *AA = nullptr) const {
     return MI.getOpcode() == TargetOpcode::IMPLICIT_DEF ||
            (MI.getDesc().isRematerializable() &&
             (isReallyTriviallyReMaterializable(MI, AA) ||
              isReallyTriviallyReMaterializableGeneric(MI, AA)));
   }
 
   /// Given \p MO is a PhysReg use return if it can be ignored for the purpose
   /// of instruction rematerialization.
   virtual bool isIgnorableUse(const MachineOperand &MO) const {
     return false;
   }
 
 protected:
   /// For instructions with opcodes for which the M_REMATERIALIZABLE flag is
   /// set, this hook lets the target specify whether the instruction is actually
   /// trivially rematerializable, taking into consideration its operands. This
   /// predicate must return false if the instruction has any side effects other
-  /// than producing a value, or if it requres any address registers that are
-  /// not always available.
+  /// than producing a value.
   /// Requirements must be check as stated in isTriviallyReMaterializable() .
   virtual bool isReallyTriviallyReMaterializable(const MachineInstr &MI,
                                                  AAResults *AA) const {
     return false;
   }
 
   /// This method commutes the operands of the given machine instruction MI.
   /// The operands to be commuted are specified by their indices OpIdx1 and

llvm/lib/CodeGen/TargetInstrInfo.cpp

 }
 
 bool TargetInstrInfo::isReallyTriviallyReMaterializableGeneric(
     const MachineInstr &MI, AAResults *AA) const {
   const MachineFunction &MF = *MI.getMF();
   const MachineRegisterInfo &MRI = MF.getRegInfo();
 
   // Remat clients assume operand 0 is the defined register.
-  if (!MI.getNumOperands() || !MI.getOperand(0).isReg())
+  if (!MI.getNumOperands() || !MI.getOperand(0).isReg() ||
+      MI.getOperand(0).isTied())
     return false;
   Register DefReg = MI.getOperand(0).getReg();
 
   // A sub-register definition can only be rematerialized if the instruction
   // doesn't read the other parts of the register.  Otherwise it is really a
   // read-modify-write operation on the full virtual register which cannot be
   // moved safely.
   if (Register::isVirtualRegister(DefReg) && MI.getOperand(0).getSubReg() &&
if (Register::isPhysicalRegister(Reg)) {
       }
       continue;
     }
 
     // Only allow one virtual-register def.  There may be multiple defs of the
     // same virtual register, though.
     if (MO.isDef() && Reg != DefReg)
       return false;
-
-    // Don't allow any virtual-register uses. Rematting an instruction with
-    // virtual register uses would length the live ranges of the uses, which
-    // is not necessarily a good idea, certainly not "trivial".
-    if (MO.isUse())
-      return false;
   }
 
   // Everything checked out.
   return true;
 }
 
 int TargetInstrInfo::getSPAdjust(const MachineInstr &MI) const {
   const MachineFunction *MF = MI.getMF();

llvm/test/CodeGen/AMDGPU/remat-sop.mir

bb.0:
     %0:sreg_32 = S_MOV_B32 1, implicit $exec
     %1:sreg_32 = S_MOV_B32 2, implicit $exec
     %2:sreg_32 = S_MOV_B32 3, implicit $exec
     S_NOP 0, implicit %0
     S_NOP 0, implicit %1
     S_NOP 0, implicit %2
     S_ENDPGM 0
 ...
+# The liverange of %0 covers a point of rematerialization, source value is
+# availabe.
+---
+name:            test_remat_s_mov_b32_vreg_src_long_lr
+tracksRegLiveness: true
+machineFunctionInfo:
+  stackPtrOffsetReg:  $sgpr32
+body:             |
+  bb.0:
+    ; GCN-LABEL: name: test_remat_s_mov_b32_vreg_src_long_lr
+    ; GCN: renamable $sgpr0 = IMPLICIT_DEF
+    ; GCN: renamable $sgpr1 = S_MOV_B32 renamable $sgpr0
+    ; GCN: S_NOP 0, implicit killed renamable $sgpr1
+    ; GCN: renamable $sgpr1 = S_MOV_B32 renamable $sgpr0
+    ; GCN: S_NOP 0, implicit killed renamable $sgpr1
+    ; GCN: renamable $sgpr1 = S_MOV_B32 renamable $sgpr0
+    ; GCN: S_NOP 0, implicit killed renamable $sgpr1
+    ; GCN: S_NOP 0, implicit killed renamable $sgpr0
+    ; GCN: S_ENDPGM 0
+    %0:sreg_32 = IMPLICIT_DEF
+    %1:sreg_32 = S_MOV_B32 %0:sreg_32
+    %2:sreg_32 = S_MOV_B32 %0:sreg_32
+    %3:sreg_32 = S_MOV_B32 %0:sreg_32
+    S_NOP 0, implicit %1
+    S_NOP 0, implicit %2
+    S_NOP 0, implicit %3
+    S_NOP 0, implicit %0
+    S_ENDPGM 0
+...
+# The liverange of %0 does not cover a point of rematerialization, source value is
+# unavailabe and we do not want to artificially extend the liverange.
+---
+name:            test_no_remat_s_mov_b32_vreg_src_short_lr
+tracksRegLiveness: true
+machineFunctionInfo:
+  stackPtrOffsetReg:  $sgpr32
+body:             |
+  bb.0:
+    ; GCN-LABEL: name: test_no_remat_s_mov_b32_vreg_src_short_lr
+    ; GCN: renamable $sgpr0 = IMPLICIT_DEF
+    ; GCN: renamable $sgpr1 = S_MOV_B32 renamable $sgpr0
+    ; GCN: SI_SPILL_S32_SAVE killed renamable $sgpr1, %stack.1, implicit $exec, implicit $sgpr32 :: (store (s32) into %stack.1, addrspace 5)
+    ; GCN: renamable $sgpr1 = S_MOV_B32 renamable $sgpr0
+    ; GCN: SI_SPILL_S32_SAVE killed renamable $sgpr1, %stack.0, implicit $exec, implicit $sgpr32 :: (store (s32) into %stack.0, addrspace 5)
+    ; GCN: renamable $sgpr0 = S_MOV_B32 killed renamable $sgpr0
+    ; GCN: renamable $sgpr1 = SI_SPILL_S32_RESTORE %stack.1, implicit $exec, implicit $sgpr32 :: (load (s32) from %stack.1, addrspace 5)
+    ; GCN: S_NOP 0, implicit killed renamable $sgpr1
+    ; GCN: renamable $sgpr1 = SI_SPILL_S32_RESTORE %stack.0, implicit $exec, implicit $sgpr32 :: (load (s32) from %stack.0, addrspace 5)
+    ; GCN: S_NOP 0, implicit killed renamable $sgpr1
+    ; GCN: S_NOP 0, implicit killed renamable $sgpr0
+    ; GCN: S_ENDPGM 0
+    %0:sreg_32 = IMPLICIT_DEF
+    %1:sreg_32 = S_MOV_B32 %0:sreg_32
+    %2:sreg_32 = S_MOV_B32 %0:sreg_32
+    %3:sreg_32 = S_MOV_B32 %0:sreg_32
+    S_NOP 0, implicit %1
+    S_NOP 0, implicit %2
+    S_NOP 0, implicit %3
+    S_ENDPGM 0
+...
 ---
 name:            test_remat_s_mov_b64
 tracksRegLiveness: true
 body:             |
   bb.0:
     ; GCN-LABEL: name: test_remat_s_mov_b64
     ; GCN: renamable $sgpr0_sgpr1 = S_MOV_B64 1
     ; GCN: renamable $sgpr2_sgpr3 = S_MOV_B64 2

llvm/test/CodeGen/ARM/arm-shrink-wrapping-linux.ll

 ; ENABLE-NEXT:  @ %bb.1: @ %while.cond.preheader
 ; ENABLE-NEXT:    cmp r1, #0
 ; ENABLE-NEXT:    beq .LBB0_6
 ; ENABLE-NEXT:  @ %bb.2: @ %while.cond.preheader
 ; ENABLE-NEXT:    cmp r0, r2
 ; ENABLE-NEXT:    pophs {r11, pc}
 ; ENABLE-NEXT:  .LBB0_3: @ %while.body.preheader
 ; ENABLE-NEXT:    movw r12, :lower16:skip
-; ENABLE-NEXT:    sub r1, r1, #1
+; ENABLE-NEXT:    sub r3, r1, #1
 ; ENABLE-NEXT:    movt r12, :upper16:skip
 ; ENABLE-NEXT:  .LBB0_4: @ %while.body
 ; ENABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
-; ENABLE-NEXT:    ldrb r3, [r0]
-; ENABLE-NEXT:    ldrb r3, [r12, r3]
-; ENABLE-NEXT:    add r0, r0, r3
-; ENABLE-NEXT:    sub r3, r1, #1
-; ENABLE-NEXT:    cmp r3, r1
+; ENABLE-NEXT:    ldrb r1, [r0]
+; ENABLE-NEXT:    ldrb r1, [r12, r1]
+; ENABLE-NEXT:    add r0, r0, r1
+; ENABLE-NEXT:    sub r1, r3, #1
+; ENABLE-NEXT:    cmp r1, r3
 ; ENABLE-NEXT:    bhs .LBB0_6
 ; ENABLE-NEXT:  @ %bb.5: @ %while.body
 ; ENABLE-NEXT:    @ in Loop: Header=BB0_4 Depth=1
 ; ENABLE-NEXT:    cmp r0, r2
-; ENABLE-NEXT:    mov r1, r3
+; ENABLE-NEXT:    mov r3, r1
 ; ENABLE-NEXT:    blo .LBB0_4
 ; ENABLE-NEXT:  .LBB0_6: @ %if.end29
 ; ENABLE-NEXT:    pop {r11, pc}
 ; ENABLE-NEXT:  .LBB0_7: @ %while.cond2.outer
 ; ENABLE-NEXT:    @ =>This Loop Header: Depth=1
 ; ENABLE-NEXT:    @ Child Loop BB0_8 Depth 2
 ; ENABLE-NEXT:    @ Child Loop BB0_15 Depth 2
 ; ENABLE-NEXT:    mov r3, r0

 ; DISABLE-NEXT:  @ %bb.1: @ %while.cond.preheader
 ; DISABLE-NEXT:    cmp r1, #0
 ; DISABLE-NEXT:    beq .LBB0_6
 ; DISABLE-NEXT:  @ %bb.2: @ %while.cond.preheader
 ; DISABLE-NEXT:    cmp r0, r2
 ; DISABLE-NEXT:    pophs {r11, pc}
 ; DISABLE-NEXT:  .LBB0_3: @ %while.body.preheader
 ; DISABLE-NEXT:    movw r12, :lower16:skip
-; DISABLE-NEXT:    sub r1, r1, #1
+; DISABLE-NEXT:    sub r3, r1, #1
 ; DISABLE-NEXT:    movt r12, :upper16:skip
 ; DISABLE-NEXT:  .LBB0_4: @ %while.body
 ; DISABLE-NEXT:    @ =>This Inner Loop Header: Depth=1
-; DISABLE-NEXT:    ldrb r3, [r0]
-; DISABLE-NEXT:    ldrb r3, [r12, r3]
-; DISABLE-NEXT:    add r0, r0, r3
-; DISABLE-NEXT:    sub r3, r1, #1
-; DISABLE-NEXT:    cmp r3, r1
+; DISABLE-NEXT:    ldrb r1, [r0]
+; DISABLE-NEXT:    ldrb r1, [r12, r1]
+; DISABLE-NEXT:    add r0, r0, r1
+; DISABLE-NEXT:    sub r1, r3, #1
+; DISABLE-NEXT:    cmp r1, r3
 ; DISABLE-NEXT:    bhs .LBB0_6
 ; DISABLE-NEXT:  @ %bb.5: @ %while.body
 ; DISABLE-NEXT:    @ in Loop: Header=BB0_4 Depth=1
 ; DISABLE-NEXT:    cmp r0, r2
-; DISABLE-NEXT:    mov r1, r3
+; DISABLE-NEXT:    mov r3, r1
 ; DISABLE-NEXT:    blo .LBB0_4
 ; DISABLE-NEXT:  .LBB0_6: @ %if.end29
 ; DISABLE-NEXT:    pop {r11, pc}
 ; DISABLE-NEXT:  .LBB0_7: @ %while.cond2.outer
 ; DISABLE-NEXT:    @ =>This Loop Header: Depth=1
 ; DISABLE-NEXT:    @ Child Loop BB0_8 Depth 2
 ; DISABLE-NEXT:    @ Child Loop BB0_15 Depth 2
 ; DISABLE-NEXT:    mov r3, r0

llvm/test/CodeGen/ARM/funnel-shift-rot.ll

 
 define i64 @rotl_i64(i64 %x, i64 %z) {
 ; SCALAR-LABEL: rotl_i64:
 ; SCALAR:       @ %bb.0:
 ; SCALAR-NEXT:    .save {r4, r5, r11, lr}
 ; SCALAR-NEXT:    push {r4, r5, r11, lr}
 ; SCALAR-NEXT:    rsb r3, r2, #0
 ; SCALAR-NEXT:    and r4, r2, #63
-; SCALAR-NEXT:    and lr, r3, #63
-; SCALAR-NEXT:    rsb r3, lr, #32
+; SCALAR-NEXT:    and r12, r3, #63
+; SCALAR-NEXT:    rsb r3, r12, #32
 ; SCALAR-NEXT:    lsl r2, r0, r4
-; SCALAR-NEXT:    lsr r12, r0, lr
-; SCALAR-NEXT:    orr r3, r12, r1, lsl r3
-; SCALAR-NEXT:    subs r12, lr, #32
-; SCALAR-NEXT:    lsrpl r3, r1, r12
+; SCALAR-NEXT:    lsr lr, r0, r12
+; SCALAR-NEXT:    orr r3, lr, r1, lsl r3
+; SCALAR-NEXT:    subs lr, r12, #32
+; SCALAR-NEXT:    lsrpl r3, r1, lr
 ; SCALAR-NEXT:    subs r5, r4, #32
 ; SCALAR-NEXT:    movwpl r2, #0
 ; SCALAR-NEXT:    cmp r5, #0
 ; SCALAR-NEXT:    orr r2, r2, r3
 ; SCALAR-NEXT:    rsb r3, r4, #32
 ; SCALAR-NEXT:    lsr r3, r0, r3
 ; SCALAR-NEXT:    orr r3, r3, r1, lsl r4
 ; SCALAR-NEXT:    lslpl r3, r0, r5
-; SCALAR-NEXT:    lsr r0, r1, lr
-; SCALAR-NEXT:    cmp r12, #0
+; SCALAR-NEXT:    lsr r0, r1, r12
+; SCALAR-NEXT:    cmp lr, #0
 ; SCALAR-NEXT:    movwpl r0, #0
 ; SCALAR-NEXT:    orr r1, r3, r0
 ; SCALAR-NEXT:    mov r0, r2
 ; SCALAR-NEXT:    pop {r4, r5, r11, pc}
 ;
 ; NEON-LABEL: rotl_i64:
 ; NEON:       @ %bb.0:
 ; NEON-NEXT:    .save {r4, r5, r11, lr}
; CHECK-NEXT:    bx lr
   ret i32 %f
 }
 
 define i64 @rotr_i64(i64 %x, i64 %z) {
 ; CHECK-LABEL: rotr_i64:
 ; CHECK:       @ %bb.0:
 ; CHECK-NEXT:    .save {r4, r5, r11, lr}
 ; CHECK-NEXT:    push {r4, r5, r11, lr}
-; CHECK-NEXT:    and lr, r2, #63
+; CHECK-NEXT:    and r12, r2, #63
 ; CHECK-NEXT:    rsb r2, r2, #0
-; CHECK-NEXT:    rsb r3, lr, #32
+; CHECK-NEXT:    rsb r3, r12, #32
 ; CHECK-NEXT:    and r4, r2, #63
-; CHECK-NEXT:    lsr r12, r0, lr
-; CHECK-NEXT:    orr r3, r12, r1, lsl r3
-; CHECK-NEXT:    subs r12, lr, #32
+; CHECK-NEXT:    lsr lr, r0, r12
+; CHECK-NEXT:    orr r3, lr, r1, lsl r3
+; CHECK-NEXT:    subs lr, r12, #32
 ; CHECK-NEXT:    lsl r2, r0, r4
-; CHECK-NEXT:    lsrpl r3, r1, r12
+; CHECK-NEXT:    lsrpl r3, r1, lr
 ; CHECK-NEXT:    subs r5, r4, #32
 ; CHECK-NEXT:    movwpl r2, #0
 ; CHECK-NEXT:    cmp r5, #0
 ; CHECK-NEXT:    orr r2, r3, r2
 ; CHECK-NEXT:    rsb r3, r4, #32
 ; CHECK-NEXT:    lsr r3, r0, r3
 ; CHECK-NEXT:    orr r3, r3, r1, lsl r4
 ; CHECK-NEXT:    lslpl r3, r0, r5
-; CHECK-NEXT:    lsr r0, r1, lr
-; CHECK-NEXT:    cmp r12, #0
+; CHECK-NEXT:    lsr r0, r1, r12
+; CHECK-NEXT:    cmp lr, #0
 ; CHECK-NEXT:    movwpl r0, #0
 ; CHECK-NEXT:    orr r1, r0, r3
 ; CHECK-NEXT:    mov r0, r2
 ; CHECK-NEXT:    pop {r4, r5, r11, pc}
   %f = call i64 @llvm.fshr.i64(i64 %x, i64 %x, i64 %z)
   ret i64 %f
 }
 

llvm/test/CodeGen/ARM/funnel-shift.ll

 ; CHECK-NEXT:    ldr r0, [sp, #32]
 ; CHECK-NEXT:    mov r6, r3
 ; CHECK-NEXT:    ldr r1, [sp, #36]
 ; CHECK-NEXT:    mov r7, r2
 ; CHECK-NEXT:    mov r2, #37
 ; CHECK-NEXT:    mov r3, #0
 ; CHECK-NEXT:    bl __aeabi_uldivmod
 ; CHECK-NEXT:    add r0, r2, #27
-; CHECK-NEXT:    lsl r6, r6, #27
-; CHECK-NEXT:    and r1, r0, #63
 ; CHECK-NEXT:    lsl r2, r7, #27
+; CHECK-NEXT:    and r12, r0, #63
+; CHECK-NEXT:    lsl r6, r6, #27
 ; CHECK-NEXT:    orr r7, r6, r7, lsr #5
+; CHECK-NEXT:    rsb r3, r12, #32
+; CHECK-NEXT:    lsr r2, r2, r12
 ; CHECK-NEXT:    mov r6, #63
-; CHECK-NEXT:    rsb r3, r1, #32
-; CHECK-NEXT:    lsr r2, r2, r1
-; CHECK-NEXT:    subs r12, r1, #32
-; CHECK-NEXT:    bic r6, r6, r0
 ; CHECK-NEXT:    orr r2, r2, r7, lsl r3
+; CHECK-NEXT:    subs r3, r12, #32
+; CHECK-NEXT:    bic r6, r6, r0
 ; CHECK-NEXT:    lsl r5, r9, #1
-; CHECK-NEXT:    lsrpl r2, r7, r12
+; CHECK-NEXT:    lsrpl r2, r7, r3
+; CHECK-NEXT:    subs r1, r6, #32
 ; CHECK-NEXT:    lsl r0, r5, r6
-; CHECK-NEXT:    subs r4, r6, #32
-; CHECK-NEXT:    lsl r3, r8, #1
+; CHECK-NEXT:    lsl r4, r8, #1
 ; CHECK-NEXT:    movwpl r0, #0
-; CHECK-NEXT:    orr r3, r3, r9, lsr #31
+; CHECK-NEXT:    orr r4, r4, r9, lsr #31
 ; CHECK-NEXT:    orr r0, r0, r2
 ; CHECK-NEXT:    rsb r2, r6, #32
-; CHECK-NEXT:    cmp r4, #0
-; CHECK-NEXT:    lsr r1, r7, r1
+; CHECK-NEXT:    cmp r1, #0
 ; CHECK-NEXT:    lsr r2, r5, r2
-; CHECK-NEXT:    orr r2, r2, r3, lsl r6
-; CHECK-NEXT:    lslpl r2, r5, r4
-; CHECK-NEXT:    cmp r12, #0
+; CHECK-NEXT:    orr r2, r2, r4, lsl r6
+; CHECK-NEXT:    lslpl r2, r5, r1
+; CHECK-NEXT:    lsr r1, r7, r12
+; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    movwpl r1, #0
 ; CHECK-NEXT:    orr r1, r2, r1
 ; CHECK-NEXT:    pop {r4, r5, r6, r7, r8, r9, r11, pc}
   %f = call i37 @llvm.fshr.i37(i37 %x, i37 %y, i37 %z)
   ret i37 %f
 }
 
 ; extract(concat(0b1110000, 0b1111111) >> 2) = 0b0011111

llvm/test/CodeGen/ARM/illegal-bitfield-loadstore.ll

 ; LE-NEXT:    ldr r1, [r0]
 ; LE-NEXT:    orr r1, r1, #384
 ; LE-NEXT:    str r1, [r0]
 ; LE-NEXT:    mov pc, lr
 ;
 ; BE-LABEL: i56_or:
 ; BE:       @ %bb.0:
 ; BE-NEXT:    mov r1, r0
-; BE-NEXT:    ldr r12, [r0]
 ; BE-NEXT:    ldrh r2, [r1, #4]!
 ; BE-NEXT:    ldrb r3, [r1, #2]
 ; BE-NEXT:    orr r2, r3, r2, lsl #8
-; BE-NEXT:    orr r2, r2, r12, lsl #24
-; BE-NEXT:    orr r2, r2, #384
-; BE-NEXT:    strb r2, [r1, #2]
-; BE-NEXT:    lsr r3, r2, #8
-; BE-NEXT:    strh r3, [r1]
-; BE-NEXT:    bic r1, r12, #255
-; BE-NEXT:    orr r1, r1, r2, lsr #24
+; BE-NEXT:    ldr r3, [r0]
+; BE-NEXT:    orr r2, r2, r3, lsl #24
+; BE-NEXT:    orr r12, r2, #384
+; BE-NEXT:    strb r12, [r1, #2]
+; BE-NEXT:    lsr r2, r12, #8
+; BE-NEXT:    strh r2, [r1]
+; BE-NEXT:    bic r1, r3, #255
+; BE-NEXT:    orr r1, r1, r12, lsr #24
 ; BE-NEXT:    str r1, [r0]
 ; BE-NEXT:    mov pc, lr
   %aa = load i56, i56* %a
   %b = or i56 %aa, 384
   store i56 %b, i56* %a
   ret void
 }
 

 ; BE-LABEL: i56_and_or:
 ; BE:       @ %bb.0:
 ; BE-NEXT:    mov r1, r0
 ; BE-NEXT:    mov r2, #128
 ; BE-NEXT:    ldrh r12, [r1, #4]!
 ; BE-NEXT:    ldrb r3, [r1, #2]
 ; BE-NEXT:    strb r2, [r1, #2]
 ; BE-NEXT:    orr r2, r3, r12, lsl #8
-; BE-NEXT:    ldr r12, [r0]
-; BE-NEXT:    orr r2, r2, r12, lsl #24
-; BE-NEXT:    orr r2, r2, #384
-; BE-NEXT:    lsr r3, r2, #8
-; BE-NEXT:    strh r3, [r1]
-; BE-NEXT:    bic r1, r12, #255
-; BE-NEXT:    orr r1, r1, r2, lsr #24
+; BE-NEXT:    ldr r3, [r0]
+; BE-NEXT:    orr r2, r2, r3, lsl #24
+; BE-NEXT:    orr r12, r2, #384
+; BE-NEXT:    lsr r2, r12, #8
+; BE-NEXT:    strh r2, [r1]
+; BE-NEXT:    bic r1, r3, #255
+; BE-NEXT:    orr r1, r1, r12, lsr #24
 ; BE-NEXT:    str r1, [r0]
 ; BE-NEXT:    mov pc, lr
 
   %b = load i56, i56* %a, align 1
   %c = and i56 %b, -128
   %d = or i56 %c, 384
   store i56 %d, i56* %a, align 1
   ret void

llvm/test/CodeGen/ARM/neon-copy.ll

 }
 
 define <4 x i16> @test_extracts_inserts_varidx_insert(<8 x i16> %x, i32 %idx) {
 ; CHECK-LABEL: test_extracts_inserts_varidx_insert:
 ; CHECK:       @ %bb.0:
 ; CHECK-NEXT:    .pad #8
 ; CHECK-NEXT:    sub sp, sp, #8
 ; CHECK-NEXT:    vmov.u16 r1, d0[1]
-; CHECK-NEXT:    and r0, r0, #3
+; CHECK-NEXT:    and r12, r0, #3
 ; CHECK-NEXT:    vmov.u16 r2, d0[2]
-; CHECK-NEXT:    mov r3, sp
-; CHECK-NEXT:    vmov.u16 r12, d0[3]
-; CHECK-NEXT:    orr r0, r3, r0, lsl #1
+; CHECK-NEXT:    mov r0, sp
+; CHECK-NEXT:    vmov.u16 r3, d0[3]
+; CHECK-NEXT:    orr r0, r0, r12, lsl #1
 ; CHECK-NEXT:    vst1.16 {d0[0]}, [r0:16]
 ; CHECK-NEXT:    vldr d0, [sp]
 ; CHECK-NEXT:    vmov.16 d0[1], r1
 ; CHECK-NEXT:    vmov.16 d0[2], r2
-; CHECK-NEXT:    vmov.16 d0[3], r12
+; CHECK-NEXT:    vmov.16 d0[3], r3
 ; CHECK-NEXT:    add sp, sp, #8
 ; CHECK-NEXT:    bx lr
   %tmp = extractelement <8 x i16> %x, i32 0
   %tmp2 = insertelement <4 x i16> undef, i16 %tmp, i32 %idx
   %tmp3 = extractelement <8 x i16> %x, i32 1
   %tmp4 = insertelement <4 x i16> %tmp2, i16 %tmp3, i32 1
   %tmp5 = extractelement <8 x i16> %x, i32 2
   %tmp6 = insertelement <4 x i16> %tmp4, i16 %tmp5, i32 2

llvm/test/CodeGen/Mips/llvm-ir/ashr.ll

 ; MMR3-LABEL: ashr_i128:
 ; MMR3:       # %bb.0: # %entry
 ; MMR3-NEXT:    addiusp -48
 ; MMR3-NEXT:    .cfi_def_cfa_offset 48
 ; MMR3-NEXT:    swp $16, 40($sp)
 ; MMR3-NEXT:    .cfi_offset 17, -4
 ; MMR3-NEXT:    .cfi_offset 16, -8
 ; MMR3-NEXT:    move $8, $7
-; MMR3-NEXT:    sw $6, 32($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    sw $5, 36($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    sw $4, 8($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    move $2, $6
+; MMR3-NEXT:    sw $5, 0($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    sw $4, 12($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    lw $16, 76($sp)
-; MMR3-NEXT:    srlv $4, $7, $16
-; MMR3-NEXT:    not16 $3, $16
-; MMR3-NEXT:    sw $3, 24($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    sll16 $2, $6, 1
-; MMR3-NEXT:    sllv $3, $2, $3
-; MMR3-NEXT:    li16 $2, 64
-; MMR3-NEXT:    or16 $3, $4
-; MMR3-NEXT:    srlv $6, $6, $16
-; MMR3-NEXT:    sw $6, 12($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    subu16 $7, $2, $16
+; MMR3-NEXT:    srlv $3, $7, $16
+; MMR3-NEXT:    not16 $6, $16
+; MMR3-NEXT:    sw $6, 24($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    move $4, $2
+; MMR3-NEXT:    sw $2, 32($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    sll16 $2, $2, 1
+; MMR3-NEXT:    sllv $2, $2, $6
+; MMR3-NEXT:    li16 $6, 64
+; MMR3-NEXT:    or16 $2, $3
+; MMR3-NEXT:    srlv $4, $4, $16
+; MMR3-NEXT:    sw $4, 16($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    subu16 $7, $6, $16
 ; MMR3-NEXT:    sllv $9, $5, $7
-; MMR3-NEXT:    andi16 $2, $7, 32
-; MMR3-NEXT:    sw $2, 28($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    andi16 $5, $16, 32
-; MMR3-NEXT:    sw $5, 16($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    move $4, $9
+; MMR3-NEXT:    andi16 $5, $7, 32
+; MMR3-NEXT:    sw $5, 28($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    andi16 $6, $16, 32
+; MMR3-NEXT:    sw $6, 36($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    move $3, $9
 ; MMR3-NEXT:    li16 $17, 0
-; MMR3-NEXT:    movn $4, $17, $2
-; MMR3-NEXT:    movn $3, $6, $5
-; MMR3-NEXT:    addiu $2, $16, -64
-; MMR3-NEXT:    lw $5, 36($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    srlv $5, $5, $2
-; MMR3-NEXT:    sw $5, 20($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    lw $17, 8($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    sll16 $6, $17, 1
-; MMR3-NEXT:    sw $6, 4($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    not16 $5, $2
-; MMR3-NEXT:    sllv $5, $6, $5
-; MMR3-NEXT:    or16 $3, $4
-; MMR3-NEXT:    lw $4, 20($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    or16 $5, $4
-; MMR3-NEXT:    srav $1, $17, $2
-; MMR3-NEXT:    andi16 $2, $2, 32
-; MMR3-NEXT:    sw $2, 20($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    movn $5, $1, $2
-; MMR3-NEXT:    sllv $2, $17, $7
-; MMR3-NEXT:    not16 $4, $7
-; MMR3-NEXT:    lw $7, 36($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    srl16 $6, $7, 1
-; MMR3-NEXT:    srlv $6, $6, $4
+; MMR3-NEXT:    movn $3, $17, $5
+; MMR3-NEXT:    movn $2, $4, $6
+; MMR3-NEXT:    addiu $4, $16, -64
+; MMR3-NEXT:    lw $17, 0($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    srlv $4, $17, $4
+; MMR3-NEXT:    sw $4, 20($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    lw $6, 12($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    sll16 $4, $6, 1
+; MMR3-NEXT:    sw $4, 8($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    addiu $5, $16, -64
+; MMR3-NEXT:    not16 $5, $5
+; MMR3-NEXT:    sllv $5, $4, $5
+; MMR3-NEXT:    or16 $2, $3
+; MMR3-NEXT:    lw $3, 20($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    or16 $5, $3
+; MMR3-NEXT:    addiu $3, $16, -64
+; MMR3-NEXT:    srav $1, $6, $3
+; MMR3-NEXT:    andi16 $3, $3, 32
+; MMR3-NEXT:    sw $3, 20($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    movn $5, $1, $3
+; MMR3-NEXT:    sllv $3, $6, $7
+; MMR3-NEXT:    sw $3, 4($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    not16 $3, $7
+; MMR3-NEXT:    srl16 $4, $17, 1
+; MMR3-NEXT:    srlv $3, $4, $3
 ; MMR3-NEXT:    sltiu $10, $16, 64
-; MMR3-NEXT:    movn $5, $3, $10
-; MMR3-NEXT:    or16 $6, $2
-; MMR3-NEXT:    srlv $2, $7, $16
-; MMR3-NEXT:    lw $3, 24($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    lw $4, 4($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    sllv $3, $4, $3
+; MMR3-NEXT:    movn $5, $2, $10
+; MMR3-NEXT:    lw $2, 4($sp) # 4-byte Folded Reload
 ; MMR3-NEXT:    or16 $3, $2
-; MMR3-NEXT:    srav $11, $17, $16
-; MMR3-NEXT:    lw $4, 16($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $3, $11, $4
-; MMR3-NEXT:    sra $2, $17, 31
+; MMR3-NEXT:    srlv $2, $17, $16
+; MMR3-NEXT:    lw $4, 24($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    lw $7, 8($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    sllv $17, $7, $4
+; MMR3-NEXT:    or16 $17, $2
+; MMR3-NEXT:    srav $11, $6, $16
+; MMR3-NEXT:    lw $2, 36($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    movn $17, $11, $2
+; MMR3-NEXT:    sra $2, $6, 31
 ; MMR3-NEXT:    movz $5, $8, $16
-; MMR3-NEXT:    move $8, $2
-; MMR3-NEXT:    movn $8, $3, $10
-; MMR3-NEXT:    lw $3, 28($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $6, $9, $3
-; MMR3-NEXT:    li16 $3, 0
-; MMR3-NEXT:    lw $7, 12($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $7, $3, $4
-; MMR3-NEXT:    or16 $7, $6
+; MMR3-NEXT:    move $4, $2
+; MMR3-NEXT:    movn $4, $17, $10
+; MMR3-NEXT:    lw $6, 28($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    movn $3, $9, $6
+; MMR3-NEXT:    lw $6, 36($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    li16 $17, 0
+; MMR3-NEXT:    lw $7, 16($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    movn $7, $17, $6
+; MMR3-NEXT:    or16 $7, $3
 ; MMR3-NEXT:    lw $3, 20($sp) # 4-byte Folded Reload
 ; MMR3-NEXT:    movn $1, $2, $3
 ; MMR3-NEXT:    movn $1, $7, $10
 ; MMR3-NEXT:    lw $3, 32($sp) # 4-byte Folded Reload
 ; MMR3-NEXT:    movz $1, $3, $16
-; MMR3-NEXT:    movn $11, $2, $4
+; MMR3-NEXT:    movn $11, $2, $6
 ; MMR3-NEXT:    movn $2, $11, $10
-; MMR3-NEXT:    move $3, $8
+; MMR3-NEXT:    move $3, $4
 ; MMR3-NEXT:    move $4, $1
 ; MMR3-NEXT:    lwp $16, 40($sp)
 ; MMR3-NEXT:    addiusp 48
 ; MMR3-NEXT:    jrc $ra
 ;
 ; MMR6-LABEL: ashr_i128:
 ; MMR6:       # %bb.0: # %entry
 ; MMR6-NEXT:    addiu $sp, $sp, -16
 ; MMR6-NEXT:    .cfi_def_cfa_offset 16
 ; MMR6-NEXT:    sw $17, 12($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    sw $16, 8($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    .cfi_offset 17, -4
 ; MMR6-NEXT:    .cfi_offset 16, -8
-; MMR6-NEXT:    move $1, $7
+; MMR6-NEXT:    move $12, $7
 ; MMR6-NEXT:    lw $3, 44($sp)
 ; MMR6-NEXT:    li16 $2, 64
-; MMR6-NEXT:    subu16 $7, $2, $3
-; MMR6-NEXT:    sllv $8, $5, $7
-; MMR6-NEXT:    andi16 $2, $7, 32
-; MMR6-NEXT:    selnez $9, $8, $2
-; MMR6-NEXT:    sllv $10, $4, $7
-; MMR6-NEXT:    not16 $7, $7
-; MMR6-NEXT:    srl16 $16, $5, 1
-; MMR6-NEXT:    srlv $7, $16, $7
-; MMR6-NEXT:    or $7, $10, $7
-; MMR6-NEXT:    seleqz $7, $7, $2
-; MMR6-NEXT:    or $7, $9, $7
-; MMR6-NEXT:    srlv $9, $1, $3
-; MMR6-NEXT:    not16 $16, $3
-; MMR6-NEXT:    sw $16, 4($sp) # 4-byte Folded Spill
+; MMR6-NEXT:    subu16 $16, $2, $3
+; MMR6-NEXT:    sllv $1, $5, $16
+; MMR6-NEXT:    andi16 $2, $16, 32
+; MMR6-NEXT:    selnez $8, $1, $2
+; MMR6-NEXT:    sllv $9, $4, $16
+; MMR6-NEXT:    not16 $16, $16
+; MMR6-NEXT:    srl16 $17, $5, 1
+; MMR6-NEXT:    srlv $10, $17, $16
+; MMR6-NEXT:    or $9, $9, $10
+; MMR6-NEXT:    seleqz $9, $9, $2
+; MMR6-NEXT:    or $8, $8, $9
+; MMR6-NEXT:    srlv $9, $7, $3
+; MMR6-NEXT:    not16 $7, $3
+; MMR6-NEXT:    sw $7, 4($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    sll16 $17, $6, 1
-; MMR6-NEXT:    sllv $10, $17, $16
+; MMR6-NEXT:    sllv $10, $17, $7
 ; MMR6-NEXT:    or $9, $10, $9
 ; MMR6-NEXT:    andi16 $17, $3, 32
 ; MMR6-NEXT:    seleqz $9, $9, $17
 ; MMR6-NEXT:    srlv $10, $6, $3
 ; MMR6-NEXT:    selnez $11, $10, $17
 ; MMR6-NEXT:    seleqz $10, $10, $17
-; MMR6-NEXT:    or $10, $10, $7
-; MMR6-NEXT:    seleqz $12, $8, $2
-; MMR6-NEXT:    or $8, $11, $9
+; MMR6-NEXT:    or $8, $10, $8
+; MMR6-NEXT:    seleqz $1, $1, $2
+; MMR6-NEXT:    or $9, $11, $9
 ; MMR6-NEXT:    addiu $2, $3, -64
-; MMR6-NEXT:    srlv $9, $5, $2
+; MMR6-NEXT:    srlv $10, $5, $2
 ; MMR6-NEXT:    sll16 $7, $4, 1
 ; MMR6-NEXT:    not16 $16, $2
 ; MMR6-NEXT:    sllv $11, $7, $16
 ; MMR6-NEXT:    sltiu $13, $3, 64
-; MMR6-NEXT:    or $8, $8, $12
-; MMR6-NEXT:    selnez $10, $10, $13
-; MMR6-NEXT:    or $9, $11, $9
-; MMR6-NEXT:    srav $11, $4, $2
+; MMR6-NEXT:    or $1, $9, $1
+; MMR6-NEXT:    selnez $8, $8, $13
+; MMR6-NEXT:    or $9, $11, $10
+; MMR6-NEXT:    srav $10, $4, $2
 ; MMR6-NEXT:    andi16 $2, $2, 32
-; MMR6-NEXT:    seleqz $12, $11, $2
+; MMR6-NEXT:    seleqz $11, $10, $2
 ; MMR6-NEXT:    sra $14, $4, 31
 ; MMR6-NEXT:    selnez $15, $14, $2
 ; MMR6-NEXT:    seleqz $9, $9, $2
-; MMR6-NEXT:    or $12, $15, $12
-; MMR6-NEXT:    seleqz $12, $12, $13
-; MMR6-NEXT:    selnez $2, $11, $2
-; MMR6-NEXT:    seleqz $11, $14, $13
-; MMR6-NEXT:    or $10, $10, $12
-; MMR6-NEXT:    selnez $10, $10, $3
-; MMR6-NEXT:    selnez $8, $8, $13
+; MMR6-NEXT:    or $11, $15, $11
+; MMR6-NEXT:    seleqz $11, $11, $13
+; MMR6-NEXT:    selnez $2, $10, $2
+; MMR6-NEXT:    seleqz $10, $14, $13
+; MMR6-NEXT:    or $8, $8, $11
+; MMR6-NEXT:    selnez $8, $8, $3
+; MMR6-NEXT:    selnez $1, $1, $13
 ; MMR6-NEXT:    or $2, $2, $9
 ; MMR6-NEXT:    srav $9, $4, $3
 ; MMR6-NEXT:    seleqz $4, $9, $17
-; MMR6-NEXT:    selnez $12, $14, $17
-; MMR6-NEXT:    or $4, $12, $4
-; MMR6-NEXT:    selnez $12, $4, $13
+; MMR6-NEXT:    selnez $11, $14, $17
+; MMR6-NEXT:    or $4, $11, $4
+; MMR6-NEXT:    selnez $11, $4, $13
 ; MMR6-NEXT:    seleqz $2, $2, $13
 ; MMR6-NEXT:    seleqz $4, $6, $3
-; MMR6-NEXT:    seleqz $1, $1, $3
-; MMR6-NEXT:    or $2, $8, $2
-; MMR6-NEXT:    selnez $2, $2, $3
+; MMR6-NEXT:    seleqz $6, $12, $3
 ; MMR6-NEXT:    or $1, $1, $2
-; MMR6-NEXT:    or $4, $4, $10
-; MMR6-NEXT:    or $2, $12, $11
-; MMR6-NEXT:    srlv $3, $5, $3
-; MMR6-NEXT:    lw $5, 4($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    sllv $5, $7, $5
-; MMR6-NEXT:    or $3, $5, $3
-; MMR6-NEXT:    seleqz $3, $3, $17
-; MMR6-NEXT:    selnez $5, $9, $17
-; MMR6-NEXT:    or $3, $5, $3
-; MMR6-NEXT:    selnez $3, $3, $13
-; MMR6-NEXT:    or $3, $3, $11
+; MMR6-NEXT:    selnez $1, $1, $3
+; MMR6-NEXT:    or $1, $6, $1
+; MMR6-NEXT:    or $4, $4, $8
+; MMR6-NEXT:    or $6, $11, $10
+; MMR6-NEXT:    srlv $2, $5, $3
+; MMR6-NEXT:    lw $3, 4($sp) # 4-byte Folded Reload
+; MMR6-NEXT:    sllv $3, $7, $3
+; MMR6-NEXT:    or $2, $3, $2
+; MMR6-NEXT:    seleqz $2, $2, $17
+; MMR6-NEXT:    selnez $3, $9, $17
+; MMR6-NEXT:    or $2, $3, $2
+; MMR6-NEXT:    selnez $2, $2, $13
+; MMR6-NEXT:    or $3, $2, $10
+; MMR6-NEXT:    move $2, $6
 ; MMR6-NEXT:    move $5, $1
 ; MMR6-NEXT:    lw $16, 8($sp) # 4-byte Folded Reload
 ; MMR6-NEXT:    lw $17, 12($sp) # 4-byte Folded Reload
 ; MMR6-NEXT:    addiu $sp, $sp, 16
 ; MMR6-NEXT:    jrc $ra
 entry:
   %r = ashr i128 %a, %b
   ret i128 %r
 }

llvm/test/CodeGen/Mips/llvm-ir/lshr.ll

 ; MMR3-LABEL: lshr_i128:
 ; MMR3:       # %bb.0: # %entry
 ; MMR3-NEXT:    addiusp -40
 ; MMR3-NEXT:    .cfi_def_cfa_offset 40
 ; MMR3-NEXT:    swp $16, 32($sp)
 ; MMR3-NEXT:    .cfi_offset 17, -4
 ; MMR3-NEXT:    .cfi_offset 16, -8
 ; MMR3-NEXT:    move $8, $7
-; MMR3-NEXT:    sw $6, 24($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    sw $5, 4($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    sw $4, 28($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    lw $16, 68($sp)
 ; MMR3-NEXT:    li16 $2, 64
-; MMR3-NEXT:    subu16 $7, $2, $16
-; MMR3-NEXT:    sllv $9, $5, $7
-; MMR3-NEXT:    move $17, $5
-; MMR3-NEXT:    sw $5, 0($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    andi16 $3, $7, 32
+; MMR3-NEXT:    subu16 $17, $2, $16
+; MMR3-NEXT:    sllv $9, $5, $17
+; MMR3-NEXT:    andi16 $3, $17, 32
 ; MMR3-NEXT:    sw $3, 20($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    li16 $2, 0
 ; MMR3-NEXT:    move $4, $9
 ; MMR3-NEXT:    movn $4, $2, $3
-; MMR3-NEXT:    srlv $5, $8, $16
+; MMR3-NEXT:    srlv $5, $7, $16
 ; MMR3-NEXT:    not16 $3, $16
 ; MMR3-NEXT:    sw $3, 16($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    sll16 $2, $6, 1
+; MMR3-NEXT:    sw $6, 24($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    sllv $2, $2, $3
 ; MMR3-NEXT:    or16 $2, $5
-; MMR3-NEXT:    srlv $5, $6, $16
-; MMR3-NEXT:    sw $5, 4($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    srlv $7, $6, $16
 ; MMR3-NEXT:    andi16 $3, $16, 32
 ; MMR3-NEXT:    sw $3, 12($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    movn $2, $5, $3
+; MMR3-NEXT:    movn $2, $7, $3
 ; MMR3-NEXT:    addiu $3, $16, -64
 ; MMR3-NEXT:    or16 $2, $4
-; MMR3-NEXT:    srlv $4, $17, $3
-; MMR3-NEXT:    sw $4, 8($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    lw $4, 28($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    sll16 $6, $4, 1
-; MMR3-NEXT:    not16 $5, $3
-; MMR3-NEXT:    sllv $5, $6, $5
-; MMR3-NEXT:    lw $17, 8($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    or16 $5, $17
-; MMR3-NEXT:    srlv $1, $4, $3
-; MMR3-NEXT:    andi16 $3, $3, 32
+; MMR3-NEXT:    lw $6, 4($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    srlv $3, $6, $3
 ; MMR3-NEXT:    sw $3, 8($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    movn $5, $1, $3
+; MMR3-NEXT:    lw $3, 28($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    sll16 $4, $3, 1
+; MMR3-NEXT:    sw $4, 0($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    addiu $5, $16, -64
+; MMR3-NEXT:    not16 $5, $5
+; MMR3-NEXT:    sllv $5, $4, $5
+; MMR3-NEXT:    lw $4, 8($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    or16 $5, $4
+; MMR3-NEXT:    addiu $4, $16, -64
+; MMR3-NEXT:    srlv $1, $3, $4
+; MMR3-NEXT:    andi16 $4, $4, 32
+; MMR3-NEXT:    sw $4, 8($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    movn $5, $1, $4
 ; MMR3-NEXT:    sltiu $10, $16, 64
 ; MMR3-NEXT:    movn $5, $2, $10
-; MMR3-NEXT:    sllv $2, $4, $7
-; MMR3-NEXT:    not16 $3, $7
-; MMR3-NEXT:    lw $7, 0($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    srl16 $4, $7, 1
+; MMR3-NEXT:    sllv $2, $3, $17
+; MMR3-NEXT:    not16 $3, $17
+; MMR3-NEXT:    srl16 $4, $6, 1
 ; MMR3-NEXT:    srlv $4, $4, $3
 ; MMR3-NEXT:    or16 $4, $2
-; MMR3-NEXT:    srlv $2, $7, $16
+; MMR3-NEXT:    srlv $2, $6, $16
 ; MMR3-NEXT:    lw $3, 16($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    lw $6, 0($sp) # 4-byte Folded Reload
 ; MMR3-NEXT:    sllv $3, $6, $3
 ; MMR3-NEXT:    or16 $3, $2
 ; MMR3-NEXT:    lw $2, 28($sp) # 4-byte Folded Reload
 ; MMR3-NEXT:    srlv $2, $2, $16
-; MMR3-NEXT:    lw $17, 12($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $3, $2, $17
+; MMR3-NEXT:    lw $6, 12($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    movn $3, $2, $6
 ; MMR3-NEXT:    movz $5, $8, $16
-; MMR3-NEXT:    li16 $6, 0
-; MMR3-NEXT:    movz $3, $6, $10
-; MMR3-NEXT:    lw $7, 20($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $4, $9, $7
-; MMR3-NEXT:    lw $6, 4($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    li16 $7, 0
-; MMR3-NEXT:    movn $6, $7, $17
-; MMR3-NEXT:    or16 $6, $4
+; MMR3-NEXT:    li16 $17, 0
+; MMR3-NEXT:    movz $3, $17, $10
+; MMR3-NEXT:    lw $17, 20($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    movn $4, $9, $17
+; MMR3-NEXT:    li16 $17, 0
+; MMR3-NEXT:    movn $7, $17, $6
+; MMR3-NEXT:    or16 $7, $4
 ; MMR3-NEXT:    lw $4, 8($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $1, $7, $4
-; MMR3-NEXT:    li16 $7, 0
-; MMR3-NEXT:    movn $1, $6, $10
+; MMR3-NEXT:    movn $1, $17, $4
+; MMR3-NEXT:    li16 $17, 0
+; MMR3-NEXT:    movn $1, $7, $10
 ; MMR3-NEXT:    lw $4, 24($sp) # 4-byte Folded Reload
 ; MMR3-NEXT:    movz $1, $4, $16
-; MMR3-NEXT:    movn $2, $7, $17
+; MMR3-NEXT:    movn $2, $17, $6
 ; MMR3-NEXT:    li16 $4, 0
 ; MMR3-NEXT:    movz $2, $4, $10
 ; MMR3-NEXT:    move $4, $1
 ; MMR3-NEXT:    lwp $16, 32($sp)
 ; MMR3-NEXT:    addiusp 40
 ; MMR3-NEXT:    jrc $ra
 ;
 ; MMR6-LABEL: lshr_i128:
 ; MMR6:       # %bb.0: # %entry
-; MMR6-NEXT:    addiu $sp, $sp, -32
-; MMR6-NEXT:    .cfi_def_cfa_offset 32
-; MMR6-NEXT:    sw $17, 28($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    sw $16, 24($sp) # 4-byte Folded Spill
+; MMR6-NEXT:    addiu $sp, $sp, -24
+; MMR6-NEXT:    .cfi_def_cfa_offset 24
+; MMR6-NEXT:    sw $17, 20($sp) # 4-byte Folded Spill
+; MMR6-NEXT:    sw $16, 16($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    .cfi_offset 17, -4
 ; MMR6-NEXT:    .cfi_offset 16, -8
 ; MMR6-NEXT:    move $1, $7
-; MMR6-NEXT:    move $7, $5
-; MMR6-NEXT:    lw $3, 60($sp)
+; MMR6-NEXT:    move $7, $4
+; MMR6-NEXT:    lw $3, 52($sp)
 ; MMR6-NEXT:    srlv $2, $1, $3
-; MMR6-NEXT:    not16 $5, $3
-; MMR6-NEXT:    sw $5, 12($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    move $17, $6
-; MMR6-NEXT:    sw $6, 16($sp) # 4-byte Folded Spill
+; MMR6-NEXT:    not16 $16, $3
+; MMR6-NEXT:    sw $16, 8($sp) # 4-byte Folded Spill
+; MMR6-NEXT:    move $4, $6
+; MMR6-NEXT:    sw $6, 12($sp) # 4-byte Folded Spill
 ; MMR6-NEXT:    sll16 $6, $6, 1
-; MMR6-NEXT:    sllv $6, $6, $5
+; MMR6-NEXT:    sllv $6, $6, $16
 ; MMR6-NEXT:    or $8, $6, $2
-; MMR6-NEXT:    addiu $5, $3, -64
-; MMR6-NEXT:    srlv $9, $7, $5
-; MMR6-NEXT:    move $6, $4
-; MMR6-NEXT:    sll16 $2, $4, 1
-; MMR6-NEXT:    sw $2, 8($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    not16 $16, $5
+; MMR6-NEXT:    addiu $6, $3, -64
+; MMR6-NEXT:    srlv $9, $5, $6
+; MMR6-NEXT:    sll16 $2, $7, 1
+; MMR6-NEXT:    sw $2, 4($sp) # 4-byte Folded Spill
+; MMR6-NEXT:    not16 $16, $6
 ; MMR6-NEXT:    sllv $10, $2, $16
 ; MMR6-NEXT:    andi16 $16, $3, 32
 ; MMR6-NEXT:    seleqz $8, $8, $16
 ; MMR6-NEXT:    or $9, $10, $9
-; MMR6-NEXT:    srlv $10, $17, $3
+; MMR6-NEXT:    srlv $10, $4, $3
 ; MMR6-NEXT:    selnez $11, $10, $16
 ; MMR6-NEXT:    li16 $17, 64
 ; MMR6-NEXT:    subu16 $2, $17, $3
-; MMR6-NEXT:    sllv $12, $7, $2
-; MMR6-NEXT:    move $17, $7
+; MMR6-NEXT:    sllv $12, $5, $2
 ; MMR6-NEXT:    andi16 $4, $2, 32
-; MMR6-NEXT:    andi16 $7, $5, 32
-; MMR6-NEXT:    sw $7, 20($sp) # 4-byte Folded Spill
-; MMR6-NEXT:    seleqz $9, $9, $7
+; MMR6-NEXT:    andi16 $17, $6, 32
+; MMR6-NEXT:    seleqz $9, $9, $17
 ; MMR6-NEXT:    seleqz $13, $12, $4
 ; MMR6-NEXT:    or $8, $11, $8
 ; MMR6-NEXT:    selnez $11, $12, $4
-; MMR6-NEXT:    sllv $12, $6, $2
-; MMR6-NEXT:    move $7, $6
-; MMR6-NEXT:    sw $6, 4($sp) # 4-byte Folded Spill
+; MMR6-NEXT:    sllv $12, $7, $2
 ; MMR6-NEXT:    not16 $2, $2
-; MMR6-NEXT:    srl16 $6, $17, 1
+; MMR6-NEXT:    srl16 $6, $5, 1
 ; MMR6-NEXT:    srlv $2, $6, $2
 ; MMR6-NEXT:    or $2, $12, $2
 ; MMR6-NEXT:    seleqz $2, $2, $4
-; MMR6-NEXT:    srlv $4, $7, $5
-; MMR6-NEXT:    or $11, $11, $2
-; MMR6-NEXT:    or $5, $8, $13
-; MMR6-NEXT:    srlv $6, $17, $3
-; MMR6-NEXT:    lw $2, 20($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    selnez $7, $4, $2
-; MMR6-NEXT:    sltiu $8, $3, 64
-; MMR6-NEXT:    selnez $12, $5, $8
-; MMR6-NEXT:    or $7, $7, $9
-; MMR6-NEXT:    lw $5, 12($sp) # 4-byte Folded Reload
+; MMR6-NEXT:    addiu $4, $3, -64
+; MMR6-NEXT:    srlv $4, $7, $4
+; MMR6-NEXT:    or $12, $11, $2
+; MMR6-NEXT:    or $6, $8, $13
+; MMR6-NEXT:    srlv $5, $5, $3
+; MMR6-NEXT:    selnez $8, $4, $17
+; MMR6-NEXT:    sltiu $11, $3, 64
+; MMR6-NEXT:    selnez $13, $6, $11
+; MMR6-NEXT:    or $8, $8, $9
 ; MMR6-NEXT:    lw $2, 8($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    sllv $9, $2, $5
+; MMR6-NEXT:    lw $6, 4($sp) # 4-byte Folded Reload
+; MMR6-NEXT:    sllv $9, $6, $2
 ; MMR6-NEXT:    seleqz $10, $10, $16
-; MMR6-NEXT:    li16 $5, 0
-; MMR6-NEXT:    or $10, $10, $11
-; MMR6-NEXT:    or $6, $9, $6
-; MMR6-NEXT:    seleqz $2, $7, $8
-; MMR6-NEXT:    seleqz $7, $5, $8
-; MMR6-NEXT:    lw $5, 4($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    srlv $9, $5, $3
-; MMR6-NEXT:    seleqz $11, $9, $16
-; MMR6-NEXT:    selnez $11, $11, $8
+; MMR6-NEXT:    li16 $2, 0
+; MMR6-NEXT:    or $10, $10, $12
+; MMR6-NEXT:    or $9, $9, $5
+; MMR6-NEXT:    seleqz $5, $8, $11
+; MMR6-NEXT:    seleqz $8, $2, $11
+; MMR6-NEXT:    srlv $7, $7, $3
+; MMR6-NEXT:    seleqz $2, $7, $16
+; MMR6-NEXT:    selnez $2, $2, $11
 ; MMR6-NEXT:    seleqz $1, $1, $3
-; MMR6-NEXT:    or $2, $12, $2
-; MMR6-NEXT:    selnez $2, $2, $3
-; MMR6-NEXT:    or $5, $1, $2
-; MMR6-NEXT:    or $2, $7, $11
-; MMR6-NEXT:    seleqz $1, $6, $16
-; MMR6-NEXT:    selnez $6, $9, $16
-; MMR6-NEXT:    lw $16, 16($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    seleqz $9, $16, $3
-; MMR6-NEXT:    selnez $10, $10, $8
-; MMR6-NEXT:    lw $16, 20($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    seleqz $4, $4, $16
-; MMR6-NEXT:    seleqz $4, $4, $8
-; MMR6-NEXT:    or $4, $10, $4
+; MMR6-NEXT:    or $5, $13, $5
+; MMR6-NEXT:    selnez $5, $5, $3
+; MMR6-NEXT:    or $5, $1, $5
+; MMR6-NEXT:    or $2, $8, $2
+; MMR6-NEXT:    seleqz $1, $9, $16
+; MMR6-NEXT:    selnez $6, $7, $16
+; MMR6-NEXT:    lw $7, 12($sp) # 4-byte Folded Reload
+; MMR6-NEXT:    seleqz $7, $7, $3
+; MMR6-NEXT:    selnez $9, $10, $11
+; MMR6-NEXT:    seleqz $4, $4, $17
+; MMR6-NEXT:    seleqz $4, $4, $11
+; MMR6-NEXT:    or $4, $9, $4
 ; MMR6-NEXT:    selnez $3, $4, $3
-; MMR6-NEXT:    or $4, $9, $3
+; MMR6-NEXT:    or $4, $7, $3
 ; MMR6-NEXT:    or $1, $6, $1
-; MMR6-NEXT:    selnez $1, $1, $8
-; MMR6-NEXT:    or $3, $7, $1
-; MMR6-NEXT:    lw $16, 24($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    lw $17, 28($sp) # 4-byte Folded Reload
-; MMR6-NEXT:    addiu $sp, $sp, 32
+; MMR6-NEXT:    selnez $1, $1, $11
+; MMR6-NEXT:    or $3, $8, $1
+; MMR6-NEXT:    lw $16, 16($sp) # 4-byte Folded Reload
+; MMR6-NEXT:    lw $17, 20($sp) # 4-byte Folded Reload
+; MMR6-NEXT:    addiu $sp, $sp, 24
 ; MMR6-NEXT:    jrc $ra
 entry:
 
 ; o32 shouldn't use TImode helpers.
 ; GP32-NOT:       lw        $25, %call16(__lshrti3)($gp)
 ; MM-NOT:         lw        $25, %call16(__lshrti3)($2)
 
   %r = lshr i128 %a, %b
   ret i128 %r
 }

llvm/test/CodeGen/Mips/llvm-ir/shl.ll

 ;
 ; MMR3-LABEL: shl_i128:
 ; MMR3:       # %bb.0: # %entry
 ; MMR3-NEXT:    addiusp -40
 ; MMR3-NEXT:    .cfi_def_cfa_offset 40
 ; MMR3-NEXT:    swp $16, 32($sp)
 ; MMR3-NEXT:    .cfi_offset 17, -4
 ; MMR3-NEXT:    .cfi_offset 16, -8
-; MMR3-NEXT:    move $17, $7
-; MMR3-NEXT:    sw $7, 4($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    move $7, $6
+; MMR3-NEXT:    sw $7, 8($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    move $17, $6
+; MMR3-NEXT:    sw $5, 28($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    move $1, $4
 ; MMR3-NEXT:    lw $16, 68($sp)
 ; MMR3-NEXT:    li16 $2, 64
 ; MMR3-NEXT:    subu16 $6, $2, $16
-; MMR3-NEXT:    srlv $9, $7, $6
-; MMR3-NEXT:    andi16 $4, $6, 32
-; MMR3-NEXT:    sw $4, 24($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    srlv $9, $17, $6
+; MMR3-NEXT:    andi16 $7, $6, 32
+; MMR3-NEXT:    sw $7, 24($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    li16 $3, 0
-; MMR3-NEXT:    move $2, $9
-; MMR3-NEXT:    movn $2, $3, $4
-; MMR3-NEXT:    sllv $3, $1, $16
-; MMR3-NEXT:    sw $3, 16($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    not16 $4, $16
-; MMR3-NEXT:    sw $4, 20($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    sw $5, 28($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    move $4, $9
+; MMR3-NEXT:    movn $4, $3, $7
+; MMR3-NEXT:    sllv $7, $1, $16
+; MMR3-NEXT:    not16 $2, $16
+; MMR3-NEXT:    sw $2, 20($sp) # 4-byte Folded Spill
 ; MMR3-NEXT:    srl16 $3, $5, 1
-; MMR3-NEXT:    srlv $3, $3, $4
-; MMR3-NEXT:    lw $4, 16($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    or16 $3, $4
+; MMR3-NEXT:    srlv $3, $3, $2
+; MMR3-NEXT:    or16 $3, $7
 ; MMR3-NEXT:    sllv $5, $5, $16
-; MMR3-NEXT:    sw $5, 8($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    andi16 $4, $16, 32
-; MMR3-NEXT:    sw $4, 16($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    movn $3, $5, $4
-; MMR3-NEXT:    addiu $4, $16, -64
-; MMR3-NEXT:    or16 $3, $2
-; MMR3-NEXT:    sllv $2, $7, $4
+; MMR3-NEXT:    sw $5, 4($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    andi16 $2, $16, 32
+; MMR3-NEXT:    sw $2, 16($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    movn $3, $5, $2
+; MMR3-NEXT:    addiu $7, $16, -64
+; MMR3-NEXT:    or16 $3, $4
+; MMR3-NEXT:    sllv $2, $17, $7
 ; MMR3-NEXT:    sw $2, 12($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    srl16 $5, $17, 1
-; MMR3-NEXT:    not16 $2, $4
+; MMR3-NEXT:    lw $4, 8($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    srl16 $5, $4, 1
+; MMR3-NEXT:    not16 $2, $7
 ; MMR3-NEXT:    srlv $2, $5, $2
-; MMR3-NEXT:    lw $17, 12($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    or16 $2, $17
-; MMR3-NEXT:    lw $17, 4($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    sllv $8, $17, $4
-; MMR3-NEXT:    andi16 $4, $4, 32
-; MMR3-NEXT:    sw $4, 12($sp) # 4-byte Folded Spill
-; MMR3-NEXT:    movn $2, $8, $4
+; MMR3-NEXT:    lw $7, 12($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    or16 $2, $7
+; MMR3-NEXT:    addiu $7, $16, -64
+; MMR3-NEXT:    sllv $8, $4, $7
+; MMR3-NEXT:    andi16 $7, $7, 32
+; MMR3-NEXT:    sw $7, 12($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    movn $2, $8, $7
 ; MMR3-NEXT:    sltiu $10, $16, 64
 ; MMR3-NEXT:    movn $2, $3, $10
-; MMR3-NEXT:    srlv $4, $17, $6
+; MMR3-NEXT:    srlv $3, $4, $6
+; MMR3-NEXT:    sw $3, 0($sp) # 4-byte Folded Spill
+; MMR3-NEXT:    move $7, $4
 ; MMR3-NEXT:    not16 $3, $6
-; MMR3-NEXT:    sll16 $6, $7, 1
-; MMR3-NEXT:    sllv $3, $6, $3
+; MMR3-NEXT:    sll16 $4, $17, 1
+; MMR3-NEXT:    sllv $3, $4, $3
+; MMR3-NEXT:    lw $4, 0($sp) # 4-byte Folded Reload
 ; MMR3-NEXT:    or16 $3, $4
-; MMR3-NEXT:    sllv $6, $7, $16
+; MMR3-NEXT:    sllv $6, $17, $16
 ; MMR3-NEXT:    lw $4, 20($sp) # 4-byte Folded Reload
 ; MMR3-NEXT:    srlv $4, $5, $4
 ; MMR3-NEXT:    or16 $4, $6
-; MMR3-NEXT:    sllv $6, $17, $16
-; MMR3-NEXT:    lw $17, 16($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $4, $6, $17
+; MMR3-NEXT:    sllv $6, $7, $16
+; MMR3-NEXT:    lw $7, 16($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    movn $4, $6, $7
 ; MMR3-NEXT:    movz $2, $1, $16
 ; MMR3-NEXT:    li16 $5, 0
 ; MMR3-NEXT:    movz $4, $5, $10
-; MMR3-NEXT:    lw $7, 24($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $3, $9, $7
-; MMR3-NEXT:    lw $5, 8($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    li16 $7, 0
-; MMR3-NEXT:    movn $5, $7, $17
+; MMR3-NEXT:    lw $17, 24($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    movn $3, $9, $17
+; MMR3-NEXT:    lw $5, 4($sp) # 4-byte Folded Reload
+; MMR3-NEXT:    li16 $17, 0
+; MMR3-NEXT:    movn $5, $17, $7
 ; MMR3-NEXT:    or16 $5, $3
 ; MMR3-NEXT:    lw $3, 12($sp) # 4-byte Folded Reload
-; MMR3-NEXT:    movn $8, $7, $3
-; MMR3-NEXT:    li16 $7, 0
+; MMR3-NEXT:    movn $8, $17, $3
+; MMR3-NEXT:    li16 $17, 0
 ; MMR3-NEXT:    movn $8, $5, $10
 ; MMR3-NEXT:    lw $3, 28($sp) # 4-byte Folded Reload
 ; MMR3-NEXT:    movz $8, $3, $16
-; MMR3-NEXT:    movn $6, $7, $17
+; MMR3-NEXT:    movn $6, $17, $7
 ; MMR3-NEXT:    li16 $3, 0
 ; MMR3-NEXT:    movz $6, $3, $10
 ; MMR3-NEXT:    move $3, $8
 ; MMR3-NEXT:    move $5, $6
 ; MMR3-NEXT:    lwp $16, 32($sp)
 ; MMR3-NEXT:    addiusp 40
 ; MMR3-NEXT:    jrc $ra
 ;

llvm/test/CodeGen/Mips/llvm-ir/sub.ll

 ; MMR3: subu16   $[[T15:[0-9]+]], $[[T14]], $[[T11]]
 ; MMR3: subu16   $[[T16:[0-9]+]], $[[T15]], $[[T10]]
 ; MMR3: subu16   $[[T17:[0-9]+]], $6, $[[T1]]
 ; MMR3: subu16   $[[T18:[0-9]+]], $[[T17]], $7
 ; MMR3: lw       $[[T19:[0-9]+]], 8($sp)
 ; MMR3: lw       $[[T20:[0-9]+]], 0($sp)
 ; MMR3: subu16   $5, $[[T19]], $[[T20]]
 
-; MMR6: move     $[[T0:[0-9]+]], $7
-; MMR6: sw       $7, 8($sp)
-; MMR6: move     $[[T1:[0-9]+]], $5
-; MMR6: sw       $4, 12($sp)
+; MMR6: sw       $7, 4($sp)
+; MMR6: sw       $4, 8($sp)
 ; MMR6: lw       $[[T2:[0-9]+]], 48($sp)
 ; MMR6: sltu     $[[T3:[0-9]+]], $6, $[[T2]]
 ; MMR6: xor      $[[T4:[0-9]+]], $6, $[[T2]]
 ; MMR6: sltiu    $[[T5:[0-9]+]], $[[T4]], 1
 ; MMR6: seleqz   $[[T6:[0-9]+]], $[[T3]], $[[T5]]
 ; MMR6: lw       $[[T7:[0-9]+]], 52($sp)
-; MMR6: sltu     $[[T8:[0-9]+]], $[[T0]], $[[T7]]
+; MMR6: sltu     $[[T8:[0-9]+]], $7, $[[T7]]
 ; MMR6: selnez   $[[T9:[0-9]+]], $[[T8]], $[[T5]]
 ; MMR6: or       $[[T10:[0-9]+]], $[[T9]], $[[T6]]
 ; MMR6: lw       $[[T11:[0-9]+]], 44($sp)
-; MMR6: subu16   $[[T12:[0-9]+]], $[[T1]], $[[T11]]
-; MMR6: subu16   $[[T13:[0-9]+]], $[[T12]], $[[T7]]
-; MMR6: sltu     $[[T16:[0-9]+]], $[[T12]], $[[T7]]
-; MMR6: sltu     $[[T17:[0-9]+]], $[[T1]], $[[T11]]
-; MMR6: lw       $[[T18:[0-9]+]], 40($sp)
-; MMR6: lw       $[[T19:[0-9]+]], 12($sp)
-; MMR6: subu16   $[[T20:[0-9]+]], $[[T19]], $[[T18]]
+; MMR6: subu16   $[[T12:[0-9]+]], $5, $[[T11]]
+; MMR6: lw       $[[T1:[0-9]+]], 12($sp)
+; MMR6: subu16   $[[T13:[0-9]+]], $[[T12]], $[[T1]]
+; MMR6: sltu     $[[T16:[0-9]+]], $[[T12]], $[[T1]]
+; MMR6: sltu     $[[T17:[0-9]+]], $5, $[[T11]]
+; MMR6: lw       $[[T19:[0-9]+]], 8($sp)
+; MMR6: subu16   $[[T20:[0-9]+]], $[[T19]], $5
 ; MMR6: subu16   $[[T21:[0-9]+]], $[[T20]], $[[T17]]
 ; MMR6: subu16   $[[T22:[0-9]+]], $[[T21]], $[[T16]]
 ; MMR6: subu16   $[[T23:[0-9]+]], $6, $[[T2]]
-; MMR6: subu16   $4, $[[T23]], $5
-; MMR6: lw       $[[T24:[0-9]+]], 8($sp)
-; MMR6: lw       $[[T25:[0-9]+]], 0($sp)
-; MMR6: subu16   $5, $[[T24]], $[[T25]]
-; MMR6: lw       $3, 4($sp)
+; MMR6: subu16   $4, $[[T23]], $[[T8]]
+; MMR6: lw       $[[T24:[0-9]+]], 4($sp)
+; MMR6: subu16   $5, $[[T24]], $[[T7]]
+; MMR6: lw       $3, 0($sp)
 
 ; FIXME: The sltu, dsll, dsrl pattern here occurs when an i32 is zero
 ;        extended to 64 bits. Fortunately slt(i)(u) actually gives an i1.
 ;        These should be combined away.
 
 ; GP64-NOT-R2: dsubu     $1, $4, $6
 ; GP64-NOT-R2: sltu      $[[T0:[0-9]+]], $5, $7
 ; GP64-NOT-R2: dsll      $[[T1:[0-9]+]], $[[T0]], 32

llvm/test/CodeGen/Mips/tls.ll

 }
 
 @f3.i = internal thread_local unnamed_addr global i32 1, align 4
 
 define dso_preemptable i32 @f3() nounwind {
 entry:
 ; PIC32-LABEL:      f3:
 ; PIC32:   addu    $[[R0:[a-z0-9]+]], $2, $25
-; PIC32:   addiu   $4, $[[R0]], %tlsldm(f3.i)
 ; PIC32:   lw      $25, %call16(__tls_get_addr)($[[R0]])
+; PIC32:   addiu   $4, $[[R0]], %tlsldm(f3.i)
 ; PIC32:   jalr    $25
 ; PIC32:   lui     $[[R0:[0-9]+]], %dtprel_hi(f3.i)
 ; PIC32:   addu    $[[R1:[0-9]+]], $[[R0]], $2
 ; PIC32:   lw      $[[R3:[0-9]+]], %dtprel_lo(f3.i)($[[R1]])
 ; PIC32:   addiu   $[[R3]], $[[R3]], 1
 ; PIC32:   sw      $[[R3]], %dtprel_lo(f3.i)($[[R1]])
 
 ; PIC64-LABEL:      f3:
 ; PIC64:   lui     $[[R0:[a-z0-9]+]], %hi(%neg(%gp_rel(f3)))
 ; PIC64:   daddu   $[[R0]], $[[R0]], $25
 ; PIC64:   daddiu  $[[R1:[a-z0-9]+]], $[[R0]], %lo(%neg(%gp_rel(f3)))
-; PIC64:   daddiu  $4, $[[R1]], %tlsldm(f3.i)
 ; PIC64:   ld      $25, %call16(__tls_get_addr)($[[R1]])
+; PIC64:   daddiu  $4, $[[R1]], %tlsldm(f3.i)
 ; PIC64:   jalr    $25
 ; PIC64:   lui     $[[R0:[0-9]+]], %dtprel_hi(f3.i)
 ; PIC64:   daddu   $[[R1:[0-9]+]], $[[R0]], $2
 ; PIC64:   lw      $[[R2:[0-9]+]], %dtprel_lo(f3.i)($[[R1]])
 ; PIC64:   addiu   $[[R2]], $[[R2]], 1
 ; PIC64:   sw      $[[R2]], %dtprel_lo(f3.i)($[[R1]])
 
 ; MM-LABEL:       f3:

llvm/test/CodeGen/RISCV/atomic-rmw.ll

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s2, a1
-; RV32I-NEXT:    mv s3, a0
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    lhu a1, 0(a0)
 ; RV32I-NEXT:    lui a0, 16
-; RV32I-NEXT:    addi s0, a0, -1
-; RV32I-NEXT:    and s1, s2, s0
+; RV32I-NEXT:    addi s3, a0, -1
+; RV32I-NEXT:    and s0, s2, s3
 ; RV32I-NEXT:    j .LBB100_2
 ; RV32I-NEXT:  .LBB100_1: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB100_2 Depth=1
 ; RV32I-NEXT:    sh a1, 10(sp)
 ; RV32I-NEXT:    addi a1, sp, 10
-; RV32I-NEXT:    mv a0, s3
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    mv a3, zero
 ; RV32I-NEXT:    mv a4, zero
 ; RV32I-NEXT:    call __atomic_compare_exchange_2@plt
 ; RV32I-NEXT:    lh a1, 10(sp)
 ; RV32I-NEXT:    bnez a0, .LBB100_4
 ; RV32I-NEXT:  .LBB100_2: # %atomicrmw.start
 ; RV32I-NEXT:    # =>This Inner Loop Header: Depth=1
-; RV32I-NEXT:    and a0, a1, s0
+; RV32I-NEXT:    and a0, a1, s3
 ; RV32I-NEXT:    mv a2, a1
-; RV32I-NEXT:    bltu s1, a0, .LBB100_1
+; RV32I-NEXT:    bltu s0, a0, .LBB100_1
 ; RV32I-NEXT:  # %bb.3: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB100_2 Depth=1
 ; RV32I-NEXT:    mv a2, s2
 ; RV32I-NEXT:    j .LBB100_1
 ; RV32I-NEXT:  .LBB100_4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a1
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s2, a1
-; RV32I-NEXT:    mv s3, a0
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    lhu a1, 0(a0)
 ; RV32I-NEXT:    lui a0, 16
-; RV32I-NEXT:    addi s0, a0, -1
-; RV32I-NEXT:    and s1, s2, s0
+; RV32I-NEXT:    addi s3, a0, -1
+; RV32I-NEXT:    and s0, s2, s3
 ; RV32I-NEXT:    j .LBB101_2
 ; RV32I-NEXT:  .LBB101_1: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB101_2 Depth=1
 ; RV32I-NEXT:    sh a1, 10(sp)
 ; RV32I-NEXT:    addi a1, sp, 10
 ; RV32I-NEXT:    addi a3, zero, 2
 ; RV32I-NEXT:    addi a4, zero, 2
-; RV32I-NEXT:    mv a0, s3
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    call __atomic_compare_exchange_2@plt
 ; RV32I-NEXT:    lh a1, 10(sp)
 ; RV32I-NEXT:    bnez a0, .LBB101_4
 ; RV32I-NEXT:  .LBB101_2: # %atomicrmw.start
 ; RV32I-NEXT:    # =>This Inner Loop Header: Depth=1
-; RV32I-NEXT:    and a0, a1, s0
+; RV32I-NEXT:    and a0, a1, s3
 ; RV32I-NEXT:    mv a2, a1
-; RV32I-NEXT:    bltu s1, a0, .LBB101_1
+; RV32I-NEXT:    bltu s0, a0, .LBB101_1
 ; RV32I-NEXT:  # %bb.3: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB101_2 Depth=1
 ; RV32I-NEXT:    mv a2, s2
 ; RV32I-NEXT:    j .LBB101_1
 ; RV32I-NEXT:  .LBB101_4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a1
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s2, a1
-; RV32I-NEXT:    mv s3, a0
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    lhu a1, 0(a0)
 ; RV32I-NEXT:    lui a0, 16
-; RV32I-NEXT:    addi s0, a0, -1
-; RV32I-NEXT:    and s1, s2, s0
+; RV32I-NEXT:    addi s3, a0, -1
+; RV32I-NEXT:    and s0, s2, s3
 ; RV32I-NEXT:    j .LBB102_2
 ; RV32I-NEXT:  .LBB102_1: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB102_2 Depth=1
 ; RV32I-NEXT:    sh a1, 10(sp)
 ; RV32I-NEXT:    addi a1, sp, 10
 ; RV32I-NEXT:    addi a3, zero, 3
-; RV32I-NEXT:    mv a0, s3
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    mv a4, zero
 ; RV32I-NEXT:    call __atomic_compare_exchange_2@plt
 ; RV32I-NEXT:    lh a1, 10(sp)
 ; RV32I-NEXT:    bnez a0, .LBB102_4
 ; RV32I-NEXT:  .LBB102_2: # %atomicrmw.start
 ; RV32I-NEXT:    # =>This Inner Loop Header: Depth=1
-; RV32I-NEXT:    and a0, a1, s0
+; RV32I-NEXT:    and a0, a1, s3
 ; RV32I-NEXT:    mv a2, a1
-; RV32I-NEXT:    bltu s1, a0, .LBB102_1
+; RV32I-NEXT:    bltu s0, a0, .LBB102_1
 ; RV32I-NEXT:  # %bb.3: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB102_2 Depth=1
 ; RV32I-NEXT:    mv a2, s2
 ; RV32I-NEXT:    j .LBB102_1
 ; RV32I-NEXT:  .LBB102_4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a1
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s2, a1
-; RV32I-NEXT:    mv s3, a0
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    lhu a1, 0(a0)
 ; RV32I-NEXT:    lui a0, 16
-; RV32I-NEXT:    addi s0, a0, -1
-; RV32I-NEXT:    and s1, s2, s0
+; RV32I-NEXT:    addi s3, a0, -1
+; RV32I-NEXT:    and s0, s2, s3
 ; RV32I-NEXT:    j .LBB103_2
 ; RV32I-NEXT:  .LBB103_1: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB103_2 Depth=1
 ; RV32I-NEXT:    sh a1, 10(sp)
 ; RV32I-NEXT:    addi a1, sp, 10
 ; RV32I-NEXT:    addi a3, zero, 4
 ; RV32I-NEXT:    addi a4, zero, 2
-; RV32I-NEXT:    mv a0, s3
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    call __atomic_compare_exchange_2@plt
 ; RV32I-NEXT:    lh a1, 10(sp)
 ; RV32I-NEXT:    bnez a0, .LBB103_4
 ; RV32I-NEXT:  .LBB103_2: # %atomicrmw.start
 ; RV32I-NEXT:    # =>This Inner Loop Header: Depth=1
-; RV32I-NEXT:    and a0, a1, s0
+; RV32I-NEXT:    and a0, a1, s3
 ; RV32I-NEXT:    mv a2, a1
-; RV32I-NEXT:    bltu s1, a0, .LBB103_1
+; RV32I-NEXT:    bltu s0, a0, .LBB103_1
 ; RV32I-NEXT:  # %bb.3: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB103_2 Depth=1
 ; RV32I-NEXT:    mv a2, s2
 ; RV32I-NEXT:    j .LBB103_1
 ; RV32I-NEXT:  .LBB103_4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a1
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s2, a1
-; RV32I-NEXT:    mv s3, a0
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    lhu a1, 0(a0)
 ; RV32I-NEXT:    lui a0, 16
-; RV32I-NEXT:    addi s0, a0, -1
-; RV32I-NEXT:    and s1, s2, s0
+; RV32I-NEXT:    addi s3, a0, -1
+; RV32I-NEXT:    and s0, s2, s3
 ; RV32I-NEXT:    j .LBB104_2
 ; RV32I-NEXT:  .LBB104_1: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB104_2 Depth=1
 ; RV32I-NEXT:    sh a1, 10(sp)
 ; RV32I-NEXT:    addi a1, sp, 10
 ; RV32I-NEXT:    addi a3, zero, 5
 ; RV32I-NEXT:    addi a4, zero, 5
-; RV32I-NEXT:    mv a0, s3
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    call __atomic_compare_exchange_2@plt
 ; RV32I-NEXT:    lh a1, 10(sp)
 ; RV32I-NEXT:    bnez a0, .LBB104_4
 ; RV32I-NEXT:  .LBB104_2: # %atomicrmw.start
 ; RV32I-NEXT:    # =>This Inner Loop Header: Depth=1
-; RV32I-NEXT:    and a0, a1, s0
+; RV32I-NEXT:    and a0, a1, s3
 ; RV32I-NEXT:    mv a2, a1
-; RV32I-NEXT:    bltu s1, a0, .LBB104_1
+; RV32I-NEXT:    bltu s0, a0, .LBB104_1
 ; RV32I-NEXT:  # %bb.3: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB104_2 Depth=1
 ; RV32I-NEXT:    mv a2, s2
 ; RV32I-NEXT:    j .LBB104_1
 ; RV32I-NEXT:  .LBB104_4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a1
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s2, a1
-; RV32I-NEXT:    mv s3, a0
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    lhu a1, 0(a0)
 ; RV32I-NEXT:    lui a0, 16
-; RV32I-NEXT:    addi s0, a0, -1
-; RV32I-NEXT:    and s1, s2, s0
+; RV32I-NEXT:    addi s3, a0, -1
+; RV32I-NEXT:    and s0, s2, s3
 ; RV32I-NEXT:    j .LBB105_2
 ; RV32I-NEXT:  .LBB105_1: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB105_2 Depth=1
 ; RV32I-NEXT:    sh a1, 10(sp)
 ; RV32I-NEXT:    addi a1, sp, 10
-; RV32I-NEXT:    mv a0, s3
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    mv a3, zero
 ; RV32I-NEXT:    mv a4, zero
 ; RV32I-NEXT:    call __atomic_compare_exchange_2@plt
 ; RV32I-NEXT:    lh a1, 10(sp)
 ; RV32I-NEXT:    bnez a0, .LBB105_4
 ; RV32I-NEXT:  .LBB105_2: # %atomicrmw.start
 ; RV32I-NEXT:    # =>This Inner Loop Header: Depth=1
-; RV32I-NEXT:    and a0, a1, s0
+; RV32I-NEXT:    and a0, a1, s3
 ; RV32I-NEXT:    mv a2, a1
-; RV32I-NEXT:    bgeu s1, a0, .LBB105_1
+; RV32I-NEXT:    bgeu s0, a0, .LBB105_1
 ; RV32I-NEXT:  # %bb.3: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB105_2 Depth=1
 ; RV32I-NEXT:    mv a2, s2
 ; RV32I-NEXT:    j .LBB105_1
 ; RV32I-NEXT:  .LBB105_4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a1
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s2, a1
-; RV32I-NEXT:    mv s3, a0
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    lhu a1, 0(a0)
 ; RV32I-NEXT:    lui a0, 16
-; RV32I-NEXT:    addi s0, a0, -1
-; RV32I-NEXT:    and s1, s2, s0
+; RV32I-NEXT:    addi s3, a0, -1
+; RV32I-NEXT:    and s0, s2, s3
 ; RV32I-NEXT:    j .LBB106_2
 ; RV32I-NEXT:  .LBB106_1: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB106_2 Depth=1
 ; RV32I-NEXT:    sh a1, 10(sp)
 ; RV32I-NEXT:    addi a1, sp, 10
 ; RV32I-NEXT:    addi a3, zero, 2
 ; RV32I-NEXT:    addi a4, zero, 2
-; RV32I-NEXT:    mv a0, s3
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    call __atomic_compare_exchange_2@plt
 ; RV32I-NEXT:    lh a1, 10(sp)
 ; RV32I-NEXT:    bnez a0, .LBB106_4
 ; RV32I-NEXT:  .LBB106_2: # %atomicrmw.start
 ; RV32I-NEXT:    # =>This Inner Loop Header: Depth=1
-; RV32I-NEXT:    and a0, a1, s0
+; RV32I-NEXT:    and a0, a1, s3
 ; RV32I-NEXT:    mv a2, a1
-; RV32I-NEXT:    bgeu s1, a0, .LBB106_1
+; RV32I-NEXT:    bgeu s0, a0, .LBB106_1
 ; RV32I-NEXT:  # %bb.3: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB106_2 Depth=1
 ; RV32I-NEXT:    mv a2, s2
 ; RV32I-NEXT:    j .LBB106_1
 ; RV32I-NEXT:  .LBB106_4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a1
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s2, a1
-; RV32I-NEXT:    mv s3, a0
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    lhu a1, 0(a0)
 ; RV32I-NEXT:    lui a0, 16
-; RV32I-NEXT:    addi s0, a0, -1
-; RV32I-NEXT:    and s1, s2, s0
+; RV32I-NEXT:    addi s3, a0, -1
+; RV32I-NEXT:    and s0, s2, s3
 ; RV32I-NEXT:    j .LBB107_2
 ; RV32I-NEXT:  .LBB107_1: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB107_2 Depth=1
 ; RV32I-NEXT:    sh a1, 10(sp)
 ; RV32I-NEXT:    addi a1, sp, 10
 ; RV32I-NEXT:    addi a3, zero, 3
-; RV32I-NEXT:    mv a0, s3
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    mv a4, zero
 ; RV32I-NEXT:    call __atomic_compare_exchange_2@plt
 ; RV32I-NEXT:    lh a1, 10(sp)
 ; RV32I-NEXT:    bnez a0, .LBB107_4
 ; RV32I-NEXT:  .LBB107_2: # %atomicrmw.start
 ; RV32I-NEXT:    # =>This Inner Loop Header: Depth=1
-; RV32I-NEXT:    and a0, a1, s0
+; RV32I-NEXT:    and a0, a1, s3
 ; RV32I-NEXT:    mv a2, a1
-; RV32I-NEXT:    bgeu s1, a0, .LBB107_1
+; RV32I-NEXT:    bgeu s0, a0, .LBB107_1
 ; RV32I-NEXT:  # %bb.3: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB107_2 Depth=1
 ; RV32I-NEXT:    mv a2, s2
 ; RV32I-NEXT:    j .LBB107_1
 ; RV32I-NEXT:  .LBB107_4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a1
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s2, a1
-; RV32I-NEXT:    mv s3, a0
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    lhu a1, 0(a0)
 ; RV32I-NEXT:    lui a0, 16
-; RV32I-NEXT:    addi s0, a0, -1
-; RV32I-NEXT:    and s1, s2, s0
+; RV32I-NEXT:    addi s3, a0, -1
+; RV32I-NEXT:    and s0, s2, s3
 ; RV32I-NEXT:    j .LBB108_2
 ; RV32I-NEXT:  .LBB108_1: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB108_2 Depth=1
 ; RV32I-NEXT:    sh a1, 10(sp)
 ; RV32I-NEXT:    addi a1, sp, 10
 ; RV32I-NEXT:    addi a3, zero, 4
 ; RV32I-NEXT:    addi a4, zero, 2
-; RV32I-NEXT:    mv a0, s3
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    call __atomic_compare_exchange_2@plt
 ; RV32I-NEXT:    lh a1, 10(sp)
 ; RV32I-NEXT:    bnez a0, .LBB108_4
 ; RV32I-NEXT:  .LBB108_2: # %atomicrmw.start
 ; RV32I-NEXT:    # =>This Inner Loop Header: Depth=1
-; RV32I-NEXT:    and a0, a1, s0
+; RV32I-NEXT:    and a0, a1, s3
 ; RV32I-NEXT:    mv a2, a1
-; RV32I-NEXT:    bgeu s1, a0, .LBB108_1
+; RV32I-NEXT:    bgeu s0, a0, .LBB108_1
 ; RV32I-NEXT:  # %bb.3: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB108_2 Depth=1
 ; RV32I-NEXT:    mv a2, s2
 ; RV32I-NEXT:    j .LBB108_1
 ; RV32I-NEXT:  .LBB108_4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a1
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s2, a1
-; RV32I-NEXT:    mv s3, a0
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    lhu a1, 0(a0)
 ; RV32I-NEXT:    lui a0, 16
-; RV32I-NEXT:    addi s0, a0, -1
-; RV32I-NEXT:    and s1, s2, s0
+; RV32I-NEXT:    addi s3, a0, -1
+; RV32I-NEXT:    and s0, s2, s3
 ; RV32I-NEXT:    j .LBB109_2
 ; RV32I-NEXT:  .LBB109_1: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB109_2 Depth=1
 ; RV32I-NEXT:    sh a1, 10(sp)
 ; RV32I-NEXT:    addi a1, sp, 10
 ; RV32I-NEXT:    addi a3, zero, 5
 ; RV32I-NEXT:    addi a4, zero, 5
-; RV32I-NEXT:    mv a0, s3
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    call __atomic_compare_exchange_2@plt
 ; RV32I-NEXT:    lh a1, 10(sp)
 ; RV32I-NEXT:    bnez a0, .LBB109_4
 ; RV32I-NEXT:  .LBB109_2: # %atomicrmw.start
 ; RV32I-NEXT:    # =>This Inner Loop Header: Depth=1
-; RV32I-NEXT:    and a0, a1, s0
+; RV32I-NEXT:    and a0, a1, s3
 ; RV32I-NEXT:    mv a2, a1
-; RV32I-NEXT:    bgeu s1, a0, .LBB109_1
+; RV32I-NEXT:    bgeu s0, a0, .LBB109_1
 ; RV32I-NEXT:  # %bb.3: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB109_2 Depth=1
 ; RV32I-NEXT:    mv a2, s2
 ; RV32I-NEXT:    j .LBB109_1
 ; RV32I-NEXT:  .LBB109_4: # %atomicrmw.end
 ; RV32I-NEXT:    mv a0, a1
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload

llvm/test/CodeGen/RISCV/atomic-signext.ll

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s2, a1
-; RV32I-NEXT:    mv s3, a0
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    lhu a1, 0(a0)
 ; RV32I-NEXT:    lui a0, 16
-; RV32I-NEXT:    addi s0, a0, -1
-; RV32I-NEXT:    and s1, s2, s0
+; RV32I-NEXT:    addi s3, a0, -1
+; RV32I-NEXT:    and s0, s2, s3
 ; RV32I-NEXT:    j .LBB23_2
 ; RV32I-NEXT:  .LBB23_1: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB23_2 Depth=1
 ; RV32I-NEXT:    sh a1, 10(sp)
 ; RV32I-NEXT:    addi a1, sp, 10
-; RV32I-NEXT:    mv a0, s3
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    mv a3, zero
 ; RV32I-NEXT:    mv a4, zero
 ; RV32I-NEXT:    call __atomic_compare_exchange_2@plt
 ; RV32I-NEXT:    lh a1, 10(sp)
 ; RV32I-NEXT:    bnez a0, .LBB23_4
 ; RV32I-NEXT:  .LBB23_2: # %atomicrmw.start
 ; RV32I-NEXT:    # =>This Inner Loop Header: Depth=1
-; RV32I-NEXT:    and a0, a1, s0
+; RV32I-NEXT:    and a0, a1, s3
 ; RV32I-NEXT:    mv a2, a1
-; RV32I-NEXT:    bltu s1, a0, .LBB23_1
+; RV32I-NEXT:    bltu s0, a0, .LBB23_1
 ; RV32I-NEXT:  # %bb.3: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB23_2 Depth=1
 ; RV32I-NEXT:    mv a2, s2
 ; RV32I-NEXT:    j .LBB23_1
 ; RV32I-NEXT:  .LBB23_4: # %atomicrmw.end
 ; RV32I-NEXT:    slli a0, a1, 16
 ; RV32I-NEXT:    srai a0, a0, 16
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s2, a1
-; RV32I-NEXT:    mv s3, a0
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    lhu a1, 0(a0)
 ; RV32I-NEXT:    lui a0, 16
-; RV32I-NEXT:    addi s0, a0, -1
-; RV32I-NEXT:    and s1, s2, s0
+; RV32I-NEXT:    addi s3, a0, -1
+; RV32I-NEXT:    and s0, s2, s3
 ; RV32I-NEXT:    j .LBB24_2
 ; RV32I-NEXT:  .LBB24_1: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB24_2 Depth=1
 ; RV32I-NEXT:    sh a1, 10(sp)
 ; RV32I-NEXT:    addi a1, sp, 10
-; RV32I-NEXT:    mv a0, s3
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    mv a3, zero
 ; RV32I-NEXT:    mv a4, zero
 ; RV32I-NEXT:    call __atomic_compare_exchange_2@plt
 ; RV32I-NEXT:    lh a1, 10(sp)
 ; RV32I-NEXT:    bnez a0, .LBB24_4
 ; RV32I-NEXT:  .LBB24_2: # %atomicrmw.start
 ; RV32I-NEXT:    # =>This Inner Loop Header: Depth=1
-; RV32I-NEXT:    and a0, a1, s0
+; RV32I-NEXT:    and a0, a1, s3
 ; RV32I-NEXT:    mv a2, a1
-; RV32I-NEXT:    bgeu s1, a0, .LBB24_1
+; RV32I-NEXT:    bgeu s0, a0, .LBB24_1
 ; RV32I-NEXT:  # %bb.3: # %atomicrmw.start
 ; RV32I-NEXT:    # in Loop: Header=BB24_2 Depth=1
 ; RV32I-NEXT:    mv a2, s2
 ; RV32I-NEXT:    j .LBB24_1
 ; RV32I-NEXT:  .LBB24_4: # %atomicrmw.end
 ; RV32I-NEXT:    slli a0, a1, 16
 ; RV32I-NEXT:    srai a0, a0, 16
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload

llvm/test/CodeGen/RISCV/bswap-ctlz-cttz-ctpop.ll

 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s4, 8(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s5, 4(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s6, 0(sp) # 4-byte Folded Spill
-; RV32I-NEXT:    mv s3, a1
-; RV32I-NEXT:    mv s4, a0
+; RV32I-NEXT:    mv s1, a1
+; RV32I-NEXT:    mv s0, a0
 ; RV32I-NEXT:    addi a0, a0, -1
-; RV32I-NEXT:    not a1, s4
+; RV32I-NEXT:    not a1, s0
 ; RV32I-NEXT:    and a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 1
 ; RV32I-NEXT:    lui a2, 349525
-; RV32I-NEXT:    addi s5, a2, 1365
-; RV32I-NEXT:    and a1, a1, s5
+; RV32I-NEXT:    addi s4, a2, 1365
+; RV32I-NEXT:    and a1, a1, s4
 ; RV32I-NEXT:    sub a0, a0, a1
 ; RV32I-NEXT:    lui a1, 209715
-; RV32I-NEXT:    addi s0, a1, 819
-; RV32I-NEXT:    and a1, a0, s0
+; RV32I-NEXT:    addi s5, a1, 819
+; RV32I-NEXT:    and a1, a0, s5
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    and a0, a0, s0
+; RV32I-NEXT:    and a0, a0, s5
 ; RV32I-NEXT:    add a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    add a0, a0, a1
 ; RV32I-NEXT:    lui a1, 61681
 ; RV32I-NEXT:    addi s6, a1, -241
 ; RV32I-NEXT:    and a0, a0, s6
 ; RV32I-NEXT:    lui a1, 4112
-; RV32I-NEXT:    addi s1, a1, 257
-; RV32I-NEXT:    mv a1, s1
+; RV32I-NEXT:    addi s3, a1, 257
+; RV32I-NEXT:    mv a1, s3
 ; RV32I-NEXT:    call __mulsi3@plt
 ; RV32I-NEXT:    mv s2, a0
-; RV32I-NEXT:    addi a0, s3, -1
-; RV32I-NEXT:    not a1, s3
+; RV32I-NEXT:    addi a0, s1, -1
+; RV32I-NEXT:    not a1, s1
 ; RV32I-NEXT:    and a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 1
-; RV32I-NEXT:    and a1, a1, s5
+; RV32I-NEXT:    and a1, a1, s4
 ; RV32I-NEXT:    sub a0, a0, a1
-; RV32I-NEXT:    and a1, a0, s0
+; RV32I-NEXT:    and a1, a0, s5
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    and a0, a0, s0
+; RV32I-NEXT:    and a0, a0, s5
 ; RV32I-NEXT:    add a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    add a0, a0, a1
 ; RV32I-NEXT:    and a0, a0, s6
-; RV32I-NEXT:    mv a1, s1
+; RV32I-NEXT:    mv a1, s3
 ; RV32I-NEXT:    call __mulsi3@plt
-; RV32I-NEXT:    bnez s4, .LBB7_2
+; RV32I-NEXT:    bnez s0, .LBB7_2
 ; RV32I-NEXT:  # %bb.1:
 ; RV32I-NEXT:    srli a0, a0, 24
 ; RV32I-NEXT:    addi a0, a0, 32
 ; RV32I-NEXT:    j .LBB7_3
 ; RV32I-NEXT:  .LBB7_2:
 ; RV32I-NEXT:    srli a0, s2, 24
 ; RV32I-NEXT:  .LBB7_3:
 ; RV32I-NEXT:    mv a1, zero

 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s4, 8(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s5, 4(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s6, 0(sp) # 4-byte Folded Spill
-; RV32I-NEXT:    mv s3, a1
-; RV32I-NEXT:    mv s4, a0
+; RV32I-NEXT:    mv s1, a1
+; RV32I-NEXT:    mv s0, a0
 ; RV32I-NEXT:    addi a0, a0, -1
-; RV32I-NEXT:    not a1, s4
+; RV32I-NEXT:    not a1, s0
 ; RV32I-NEXT:    and a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 1
 ; RV32I-NEXT:    lui a2, 349525
-; RV32I-NEXT:    addi s5, a2, 1365
-; RV32I-NEXT:    and a1, a1, s5
+; RV32I-NEXT:    addi s4, a2, 1365
+; RV32I-NEXT:    and a1, a1, s4
 ; RV32I-NEXT:    sub a0, a0, a1
 ; RV32I-NEXT:    lui a1, 209715
-; RV32I-NEXT:    addi s0, a1, 819
-; RV32I-NEXT:    and a1, a0, s0
+; RV32I-NEXT:    addi s5, a1, 819
+; RV32I-NEXT:    and a1, a0, s5
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    and a0, a0, s0
+; RV32I-NEXT:    and a0, a0, s5
 ; RV32I-NEXT:    add a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    add a0, a0, a1
 ; RV32I-NEXT:    lui a1, 61681
 ; RV32I-NEXT:    addi s6, a1, -241
 ; RV32I-NEXT:    and a0, a0, s6
 ; RV32I-NEXT:    lui a1, 4112
-; RV32I-NEXT:    addi s1, a1, 257
-; RV32I-NEXT:    mv a1, s1
+; RV32I-NEXT:    addi s3, a1, 257
+; RV32I-NEXT:    mv a1, s3
 ; RV32I-NEXT:    call __mulsi3@plt
 ; RV32I-NEXT:    mv s2, a0
-; RV32I-NEXT:    addi a0, s3, -1
-; RV32I-NEXT:    not a1, s3
+; RV32I-NEXT:    addi a0, s1, -1
+; RV32I-NEXT:    not a1, s1
 ; RV32I-NEXT:    and a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 1
-; RV32I-NEXT:    and a1, a1, s5
+; RV32I-NEXT:    and a1, a1, s4
 ; RV32I-NEXT:    sub a0, a0, a1
-; RV32I-NEXT:    and a1, a0, s0
+; RV32I-NEXT:    and a1, a0, s5
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    and a0, a0, s0
+; RV32I-NEXT:    and a0, a0, s5
 ; RV32I-NEXT:    add a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    add a0, a0, a1
 ; RV32I-NEXT:    and a0, a0, s6
-; RV32I-NEXT:    mv a1, s1
+; RV32I-NEXT:    mv a1, s3
 ; RV32I-NEXT:    call __mulsi3@plt
-; RV32I-NEXT:    bnez s4, .LBB11_2
+; RV32I-NEXT:    bnez s0, .LBB11_2
 ; RV32I-NEXT:  # %bb.1:
 ; RV32I-NEXT:    srli a0, a0, 24
 ; RV32I-NEXT:    addi a0, a0, 32
 ; RV32I-NEXT:    j .LBB11_3
 ; RV32I-NEXT:  .LBB11_2:
 ; RV32I-NEXT:    srli a0, s2, 24
 ; RV32I-NEXT:  .LBB11_3:
 ; RV32I-NEXT:    mv a1, zero

 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s4, 8(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s5, 4(sp) # 4-byte Folded Spill
-; RV32I-NEXT:    mv s2, a0
+; RV32I-NEXT:    mv s0, a0
 ; RV32I-NEXT:    srli a0, a1, 1
 ; RV32I-NEXT:    lui a2, 349525
-; RV32I-NEXT:    addi s3, a2, 1365
-; RV32I-NEXT:    and a0, a0, s3
+; RV32I-NEXT:    addi s2, a2, 1365
+; RV32I-NEXT:    and a0, a0, s2
 ; RV32I-NEXT:    sub a0, a1, a0
 ; RV32I-NEXT:    lui a1, 209715
-; RV32I-NEXT:    addi s0, a1, 819
-; RV32I-NEXT:    and a1, a0, s0
+; RV32I-NEXT:    addi s1, a1, 819
+; RV32I-NEXT:    and a1, a0, s1
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    and a0, a0, s0
+; RV32I-NEXT:    and a0, a0, s1
 ; RV32I-NEXT:    add a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    add a0, a0, a1
 ; RV32I-NEXT:    lui a1, 61681
 ; RV32I-NEXT:    addi s4, a1, -241
 ; RV32I-NEXT:    and a0, a0, s4
 ; RV32I-NEXT:    lui a1, 4112
-; RV32I-NEXT:    addi s1, a1, 257
-; RV32I-NEXT:    mv a1, s1
+; RV32I-NEXT:    addi s3, a1, 257
+; RV32I-NEXT:    mv a1, s3
 ; RV32I-NEXT:    call __mulsi3@plt
 ; RV32I-NEXT:    srli s5, a0, 24
-; RV32I-NEXT:    srli a0, s2, 1
-; RV32I-NEXT:    and a0, a0, s3
-; RV32I-NEXT:    sub a0, s2, a0
-; RV32I-NEXT:    and a1, a0, s0
+; RV32I-NEXT:    srli a0, s0, 1
+; RV32I-NEXT:    and a0, a0, s2
+; RV32I-NEXT:    sub a0, s0, a0
+; RV32I-NEXT:    and a1, a0, s1
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    and a0, a0, s0
+; RV32I-NEXT:    and a0, a0, s1
 ; RV32I-NEXT:    add a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    add a0, a0, a1
 ; RV32I-NEXT:    and a0, a0, s4
-; RV32I-NEXT:    mv a1, s1
+; RV32I-NEXT:    mv a1, s3
 ; RV32I-NEXT:    call __mulsi3@plt
 ; RV32I-NEXT:    srli a0, a0, 24
 ; RV32I-NEXT:    add a0, a0, s5
 ; RV32I-NEXT:    mv a1, zero
 ; RV32I-NEXT:    lw s5, 4(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s4, 8(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload

llvm/test/CodeGen/RISCV/rv32i-rv64i-half.ll

 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    addi sp, sp, -16
 ; RV32I-NEXT:    sw ra, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 8(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 4(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 0(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    mv s0, a1
 ; RV32I-NEXT:    lui a1, 16
-; RV32I-NEXT:    addi s1, a1, -1
-; RV32I-NEXT:    and a0, a0, s1
+; RV32I-NEXT:    addi s2, a1, -1
+; RV32I-NEXT:    and a0, a0, s2
 ; RV32I-NEXT:    call __gnu_h2f_ieee@plt
-; RV32I-NEXT:    mv s2, a0
-; RV32I-NEXT:    and a0, s0, s1
+; RV32I-NEXT:    mv s1, a0
+; RV32I-NEXT:    and a0, s0, s2
 ; RV32I-NEXT:    call __gnu_h2f_ieee@plt
 ; RV32I-NEXT:    mv s0, a0
-; RV32I-NEXT:    mv a0, s2
+; RV32I-NEXT:    mv a0, s1
 ; RV32I-NEXT:    mv a1, s0
 ; RV32I-NEXT:    call __addsf3@plt
 ; RV32I-NEXT:    call __gnu_f2h_ieee@plt
-; RV32I-NEXT:    and a0, a0, s1
+; RV32I-NEXT:    and a0, a0, s2
 ; RV32I-NEXT:    call __gnu_h2f_ieee@plt
 ; RV32I-NEXT:    mv a1, s0
 ; RV32I-NEXT:    call __divsf3@plt
 ; RV32I-NEXT:    call __gnu_f2h_ieee@plt
 ; RV32I-NEXT:    lw s2, 0(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s1, 4(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s0, 8(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw ra, 12(sp) # 4-byte Folded Reload

llvm/test/CodeGen/RISCV/rv32zbb-zbp.ll

 ; This test is presented here in case future expansions of the experimental-b
 ; extension introduce instructions suitable for this pattern.
 
 declare i64 @llvm.fshl.i64(i64, i64, i64)
 
 define i64 @rol_i64(i64 %a, i64 %b) nounwind {
 ; RV32I-LABEL: rol_i64:
 ; RV32I:       # %bb.0:
-; RV32I-NEXT:    mv a7, a1
+; RV32I-NEXT:    mv t1, a1
 ; RV32I-NEXT:    andi a1, a2, 63
-; RV32I-NEXT:    addi t0, a1, -32
+; RV32I-NEXT:    addi a7, a1, -32
 ; RV32I-NEXT:    addi a6, zero, 31
-; RV32I-NEXT:    bltz t0, .LBB7_2
+; RV32I-NEXT:    bltz a7, .LBB7_2
 ; RV32I-NEXT:  # %bb.1:
-; RV32I-NEXT:    sll a1, a0, t0
+; RV32I-NEXT:    sll a1, a0, a7
 ; RV32I-NEXT:    j .LBB7_3
 ; RV32I-NEXT:  .LBB7_2:
-; RV32I-NEXT:    sll a3, a7, a2
+; RV32I-NEXT:    sll a4, t1, a2
 ; RV32I-NEXT:    sub a1, a6, a1
-; RV32I-NEXT:    srli a4, a0, 1
-; RV32I-NEXT:    srl a1, a4, a1
-; RV32I-NEXT:    or a1, a3, a1
+; RV32I-NEXT:    srli a5, a0, 1
+; RV32I-NEXT:    srl a1, a5, a1
+; RV32I-NEXT:    or a1, a4, a1
 ; RV32I-NEXT:  .LBB7_3:
 ; RV32I-NEXT:    neg a5, a2
-; RV32I-NEXT:    andi a3, a5, 63
-; RV32I-NEXT:    addi a4, a3, -32
-; RV32I-NEXT:    bltz a4, .LBB7_5
+; RV32I-NEXT:    andi a4, a5, 63
+; RV32I-NEXT:    addi t0, a4, -32
+; RV32I-NEXT:    bltz t0, .LBB7_5
 ; RV32I-NEXT:  # %bb.4:
-; RV32I-NEXT:    srl a3, a7, a4
-; RV32I-NEXT:    bltz t0, .LBB7_6
+; RV32I-NEXT:    srl a3, t1, t0
+; RV32I-NEXT:    bltz a7, .LBB7_6
 ; RV32I-NEXT:    j .LBB7_7
 ; RV32I-NEXT:  .LBB7_5:
-; RV32I-NEXT:    srl a4, a7, a5
-; RV32I-NEXT:    or a1, a1, a4
-; RV32I-NEXT:    srl a4, a0, a5
-; RV32I-NEXT:    sub a3, a6, a3
-; RV32I-NEXT:    slli a5, a7, 1
-; RV32I-NEXT:    sll a3, a5, a3
-; RV32I-NEXT:    or a3, a4, a3
-; RV32I-NEXT:    bgez t0, .LBB7_7
+; RV32I-NEXT:    srl a3, t1, a5
+; RV32I-NEXT:    or a1, a1, a3
+; RV32I-NEXT:    srl a3, a0, a5
+; RV32I-NEXT:    sub a4, a6, a4
+; RV32I-NEXT:    slli a5, t1, 1
+; RV32I-NEXT:    sll a4, a5, a4
+; RV32I-NEXT:    or a3, a3, a4
+; RV32I-NEXT:    bgez a7, .LBB7_7
 ; RV32I-NEXT:  .LBB7_6:
 ; RV32I-NEXT:    sll a0, a0, a2
 ; RV32I-NEXT:    or a3, a3, a0
 ; RV32I-NEXT:  .LBB7_7:
 ; RV32I-NEXT:    mv a0, a3
 ; RV32I-NEXT:    ret
 ;
-; RV32B-LABEL: rol_i64:
-; RV32B:       # %bb.0:
-; RV32B-NEXT:    sll a7, a1, a2
-; RV32B-NEXT:    andi a4, a2, 63
-; RV32B-NEXT:    addi a6, zero, 31
-; RV32B-NEXT:    sub a5, a6, a4
-; RV32B-NEXT:    srli a3, a0, 1
-; RV32B-NEXT:    srl a3, a3, a5
-; RV32B-NEXT:    or a7, a7, a3
-; RV32B-NEXT:    addi t1, a4, -32
-; RV32B-NEXT:    sll a5, a0, t1
-; RV32B-NEXT:    slti a3, t1, 0
-; RV32B-NEXT:    cmov a7, a3, a7, a5
-; RV32B-NEXT:    neg a5, a2
-; RV32B-NEXT:    srl t0, a1, a5
-; RV32B-NEXT:    andi t2, a5, 63
-; RV32B-NEXT:    addi a4, t2, -32
-; RV32B-NEXT:    srai a3, a4, 31
-; RV32B-NEXT:    and a3, a3, t0
-; RV32B-NEXT:    or a7, a7, a3
-; RV32B-NEXT:    srl t0, a0, a5
-; RV32B-NEXT:    sub a5, a6, t2
-; RV32B-NEXT:    slli a3, a1, 1
-; RV32B-NEXT:    sll a3, a3, a5
-; RV32B-NEXT:    or a3, t0, a3
-; RV32B-NEXT:    srl a1, a1, a4
-; RV32B-NEXT:    slti a4, a4, 0
-; RV32B-NEXT:    cmov a1, a4, a3, a1
-; RV32B-NEXT:    sll a0, a0, a2
-; RV32B-NEXT:    srai a2, t1, 31
-; RV32B-NEXT:    and a0, a2, a0
-; RV32B-NEXT:    or a0, a0, a1
-; RV32B-NEXT:    mv a1, a7
-; RV32B-NEXT:    ret
-;
-; RV32ZBB-LABEL: rol_i64:
-; RV32ZBB:       # %bb.0:
-; RV32ZBB-NEXT:    mv a7, a1
-; RV32ZBB-NEXT:    andi a1, a2, 63
-; RV32ZBB-NEXT:    addi t0, a1, -32
-; RV32ZBB-NEXT:    addi a6, zero, 31
-; RV32ZBB-NEXT:    bltz t0, .LBB7_2
-; RV32ZBB-NEXT:  # %bb.1:
-; RV32ZBB-NEXT:    sll a1, a0, t0
-; RV32ZBB-NEXT:    j .LBB7_3
-; RV32ZBB-NEXT:  .LBB7_2:
-; RV32ZBB-NEXT:    sll a3, a7, a2
-; RV32ZBB-NEXT:    sub a1, a6, a1
-; RV32ZBB-NEXT:    srli a4, a0, 1
-; RV32ZBB-NEXT:    srl a1, a4, a1
-; RV32ZBB-NEXT:    or a1, a3, a1
-; RV32ZBB-NEXT:  .LBB7_3:
-; RV32ZBB-NEXT:    neg a5, a2
-; RV32ZBB-NEXT:    andi a3, a5, 63
-; RV32ZBB-NEXT:    addi a4, a3, -32
-; RV32ZBB-NEXT:    bltz a4, .LBB7_5
-; RV32ZBB-NEXT:  # %bb.4:
-; RV32ZBB-NEXT:    srl a3, a7, a4
-; RV32ZBB-NEXT:    bltz t0, .LBB7_6
-; RV32ZBB-NEXT:    j .LBB7_7
-; RV32ZBB-NEXT:  .LBB7_5:
-; RV32ZBB-NEXT:    srl a4, a7, a5
-; RV32ZBB-NEXT:    or a1, a1, a4
-; RV32ZBB-NEXT:    srl a4, a0, a5
-; RV32ZBB-NEXT:    sub a3, a6, a3
-; RV32ZBB-NEXT:    slli a5, a7, 1
-; RV32ZBB-NEXT:    sll a3, a5, a3
-; RV32ZBB-NEXT:    or a3, a4, a3
-; RV32ZBB-NEXT:    bgez t0, .LBB7_7
-; RV32ZBB-NEXT:  .LBB7_6:
-; RV32ZBB-NEXT:    sll a0, a0, a2
-; RV32ZBB-NEXT:    or a3, a3, a0
-; RV32ZBB-NEXT:  .LBB7_7:
-; RV32ZBB-NEXT:    mv a0, a3
-; RV32ZBB-NEXT:    ret
-;
-; RV32ZBP-LABEL: rol_i64:
-; RV32ZBP:       # %bb.0:
-; RV32ZBP-NEXT:    mv a7, a1
-; RV32ZBP-NEXT:    andi a1, a2, 63
-; RV32ZBP-NEXT:    addi t0, a1, -32
-; RV32ZBP-NEXT:    addi a6, zero, 31
-; RV32ZBP-NEXT:    bltz t0, .LBB7_2
-; RV32ZBP-NEXT:  # %bb.1:
-; RV32ZBP-NEXT:    sll a1, a0, t0
-; RV32ZBP-NEXT:    j .LBB7_3
-; RV32ZBP-NEXT:  .LBB7_2:
-; RV32ZBP-NEXT:    sll a3, a7, a2
-; RV32ZBP-NEXT:    sub a1, a6, a1
-; RV32ZBP-NEXT:    srli a4, a0, 1
-; RV32ZBP-NEXT:    srl a1, a4, a1
-; RV32ZBP-NEXT:    or a1, a3, a1
-; RV32ZBP-NEXT:  .LBB7_3:
-; RV32ZBP-NEXT:    neg a5, a2
-; RV32ZBP-NEXT:    andi a3, a5, 63
-; RV32ZBP-NEXT:    addi a4, a3, -32
-; RV32ZBP-NEXT:    bltz a4, .LBB7_5
-; RV32ZBP-NEXT:  # %bb.4:
-; RV32ZBP-NEXT:    srl a3, a7, a4
-; RV32ZBP-NEXT:    bltz t0, .LBB7_6
-; RV32ZBP-NEXT:    j .LBB7_7
-; RV32ZBP-NEXT:  .LBB7_5:
-; RV32ZBP-NEXT:    srl a4, a7, a5
-; RV32ZBP-NEXT:    or a1, a1, a4
-; RV32ZBP-NEXT:    srl a4, a0, a5
-; RV32ZBP-NEXT:    sub a3, a6, a3
-; RV32ZBP-NEXT:    slli a5, a7, 1
-; RV32ZBP-NEXT:    sll a3, a5, a3
-; RV32ZBP-NEXT:    or a3, a4, a3
-; RV32ZBP-NEXT:    bgez t0, .LBB7_7
-; RV32ZBP-NEXT:  .LBB7_6:
-; RV32ZBP-NEXT:    sll a0, a0, a2
-; RV32ZBP-NEXT:    or a3, a3, a0
-; RV32ZBP-NEXT:  .LBB7_7:
-; RV32ZBP-NEXT:    mv a0, a3
-; RV32ZBP-NEXT:    ret
+; RV32IB-LABEL: rol_i64:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    sll a7, a1, a2
+; RV32IB-NEXT:    andi a4, a2, 63
+; RV32IB-NEXT:    addi a6, zero, 31
+; RV32IB-NEXT:    sub a5, a6, a4
+; RV32IB-NEXT:    srli a3, a0, 1
+; RV32IB-NEXT:    srl a3, a3, a5
+; RV32IB-NEXT:    or a3, a7, a3
+; RV32IB-NEXT:    addi a7, a4, -32
+; RV32IB-NEXT:    sll a5, a0, a7
+; RV32IB-NEXT:    slti a4, a7, 0
+; RV32IB-NEXT:    cmov t0, a4, a3, a5
+; RV32IB-NEXT:    neg a4, a2
+; RV32IB-NEXT:    srl t2, a1, a4
+; RV32IB-NEXT:    andi a3, a4, 63
+; RV32IB-NEXT:    addi t1, a3, -32
+; RV32IB-NEXT:    srai a5, t1, 31
+; RV32IB-NEXT:    and a5, a5, t2
+; RV32IB-NEXT:    or t0, t0, a5
+; RV32IB-NEXT:    srl a4, a0, a4
+; RV32IB-NEXT:    sub a3, a6, a3
+; RV32IB-NEXT:    slli a5, a1, 1
+; RV32IB-NEXT:    sll a3, a5, a3
+; RV32IB-NEXT:    or a3, a4, a3
+; RV32IB-NEXT:    srl a1, a1, t1
+; RV32IB-NEXT:    slti a4, t1, 0
+; RV32IB-NEXT:    cmov a1, a4, a3, a1
+; RV32IB-NEXT:    sll a0, a0, a2
+; RV32IB-NEXT:    srai a2, a7, 31
+; RV32IB-NEXT:    and a0, a2, a0
+; RV32IB-NEXT:    or a0, a0, a1
+; RV32IB-NEXT:    mv a1, t0
+; RV32IB-NEXT:    ret
+;
+; RV32IBB-LABEL: rol_i64:
+; RV32IBB:       # %bb.0:
+; RV32IBB-NEXT:    mv t1, a1
+; RV32IBB-NEXT:    andi a1, a2, 63
+; RV32IBB-NEXT:    addi a7, a1, -32
+; RV32IBB-NEXT:    addi a6, zero, 31
+; RV32IBB-NEXT:    bltz a7, .LBB7_2
+; RV32IBB-NEXT:  # %bb.1:
+; RV32IBB-NEXT:    sll a1, a0, a7
+; RV32IBB-NEXT:    j .LBB7_3
+; RV32IBB-NEXT:  .LBB7_2:
+; RV32IBB-NEXT:    sll a4, t1, a2
+; RV32IBB-NEXT:    sub a1, a6, a1
+; RV32IBB-NEXT:    srli a5, a0, 1
+; RV32IBB-NEXT:    srl a1, a5, a1
+; RV32IBB-NEXT:    or a1, a4, a1
+; RV32IBB-NEXT:  .LBB7_3:
+; RV32IBB-NEXT:    neg a5, a2
+; RV32IBB-NEXT:    andi a4, a5, 63
+; RV32IBB-NEXT:    addi t0, a4, -32
+; RV32IBB-NEXT:    bltz t0, .LBB7_5
+; RV32IBB-NEXT:  # %bb.4:
+; RV32IBB-NEXT:    srl a3, t1, t0
+; RV32IBB-NEXT:    bltz a7, .LBB7_6
+; RV32IBB-NEXT:    j .LBB7_7
+; RV32IBB-NEXT:  .LBB7_5:
+; RV32IBB-NEXT:    srl a3, t1, a5
+; RV32IBB-NEXT:    or a1, a1, a3
+; RV32IBB-NEXT:    srl a3, a0, a5
+; RV32IBB-NEXT:    sub a4, a6, a4
+; RV32IBB-NEXT:    slli a5, t1, 1
+; RV32IBB-NEXT:    sll a4, a5, a4
+; RV32IBB-NEXT:    or a3, a3, a4
+; RV32IBB-NEXT:    bgez a7, .LBB7_7
+; RV32IBB-NEXT:  .LBB7_6:
+; RV32IBB-NEXT:    sll a0, a0, a2
+; RV32IBB-NEXT:    or a3, a3, a0
+; RV32IBB-NEXT:  .LBB7_7:
+; RV32IBB-NEXT:    mv a0, a3
+; RV32IBB-NEXT:    ret
+;
+; RV32IBP-LABEL: rol_i64:
+; RV32IBP:       # %bb.0:
+; RV32IBP-NEXT:    mv t1, a1
+; RV32IBP-NEXT:    andi a1, a2, 63
+; RV32IBP-NEXT:    addi a7, a1, -32
+; RV32IBP-NEXT:    addi a6, zero, 31
+; RV32IBP-NEXT:    bltz a7, .LBB7_2
+; RV32IBP-NEXT:  # %bb.1:
+; RV32IBP-NEXT:    sll a1, a0, a7
+; RV32IBP-NEXT:    j .LBB7_3
+; RV32IBP-NEXT:  .LBB7_2:
+; RV32IBP-NEXT:    sll a4, t1, a2
+; RV32IBP-NEXT:    sub a1, a6, a1
+; RV32IBP-NEXT:    srli a5, a0, 1
+; RV32IBP-NEXT:    srl a1, a5, a1
+; RV32IBP-NEXT:    or a1, a4, a1
+; RV32IBP-NEXT:  .LBB7_3:
+; RV32IBP-NEXT:    neg a5, a2
+; RV32IBP-NEXT:    andi a4, a5, 63
+; RV32IBP-NEXT:    addi t0, a4, -32
+; RV32IBP-NEXT:    bltz t0, .LBB7_5
+; RV32IBP-NEXT:  # %bb.4:
+; RV32IBP-NEXT:    srl a3, t1, t0
+; RV32IBP-NEXT:    bltz a7, .LBB7_6
+; RV32IBP-NEXT:    j .LBB7_7
+; RV32IBP-NEXT:  .LBB7_5:
+; RV32IBP-NEXT:    srl a3, t1, a5
+; RV32IBP-NEXT:    or a1, a1, a3
+; RV32IBP-NEXT:    srl a3, a0, a5
+; RV32IBP-NEXT:    sub a4, a6, a4
+; RV32IBP-NEXT:    slli a5, t1, 1
+; RV32IBP-NEXT:    sll a4, a5, a4
+; RV32IBP-NEXT:    or a3, a3, a4
+; RV32IBP-NEXT:    bgez a7, .LBB7_7
+; RV32IBP-NEXT:  .LBB7_6:
+; RV32IBP-NEXT:    sll a0, a0, a2
+; RV32IBP-NEXT:    or a3, a3, a0
+; RV32IBP-NEXT:  .LBB7_7:
+; RV32IBP-NEXT:    mv a0, a3
+; RV32IBP-NEXT:    ret
   %or = tail call i64 @llvm.fshl.i64(i64 %a, i64 %a, i64 %b)
   ret i64 %or
 }
 
 declare i32 @llvm.fshr.i32(i32, i32, i32)
 
 define i32 @ror_i32(i32 %a, i32 %b) nounwind {
 ; RV32I-LABEL: ror_i32:

 ; This test is presented here in case future expansions of the experimental-b
 ; extension introduce instructions suitable for this pattern.
 
 declare i64 @llvm.fshr.i64(i64, i64, i64)
 
 define i64 @ror_i64(i64 %a, i64 %b) nounwind {
 ; RV32I-LABEL: ror_i64:
 ; RV32I:       # %bb.0:
-; RV32I-NEXT:    mv t0, a0
+; RV32I-NEXT:    mv t1, a0
 ; RV32I-NEXT:    andi a0, a2, 63
 ; RV32I-NEXT:    addi a7, a0, -32
 ; RV32I-NEXT:    addi a6, zero, 31
 ; RV32I-NEXT:    bltz a7, .LBB9_2
 ; RV32I-NEXT:  # %bb.1:
 ; RV32I-NEXT:    srl a0, a1, a7
 ; RV32I-NEXT:    j .LBB9_3
 ; RV32I-NEXT:  .LBB9_2:
-; RV32I-NEXT:    srl a3, t0, a2
+; RV32I-NEXT:    srl a4, t1, a2
 ; RV32I-NEXT:    sub a0, a6, a0
-; RV32I-NEXT:    slli a4, a1, 1
-; RV32I-NEXT:    sll a0, a4, a0
-; RV32I-NEXT:    or a0, a3, a0
+; RV32I-NEXT:    slli a5, a1, 1
+; RV32I-NEXT:    sll a0, a5, a0
+; RV32I-NEXT:    or a0, a4, a0
 ; RV32I-NEXT:  .LBB9_3:
 ; RV32I-NEXT:    neg a5, a2
 ; RV32I-NEXT:    andi a4, a5, 63
-; RV32I-NEXT:    addi a3, a4, -32
-; RV32I-NEXT:    bltz a3, .LBB9_5
+; RV32I-NEXT:    addi t0, a4, -32
+; RV32I-NEXT:    bltz t0, .LBB9_5
 ; RV32I-NEXT:  # %bb.4:
-; RV32I-NEXT:    sll a3, t0, a3
+; RV32I-NEXT:    sll a3, t1, t0
 ; RV32I-NEXT:    bltz a7, .LBB9_6
 ; RV32I-NEXT:    j .LBB9_7
 ; RV32I-NEXT:  .LBB9_5:
-; RV32I-NEXT:    sll a3, t0, a5
+; RV32I-NEXT:    sll a3, t1, a5
 ; RV32I-NEXT:    or a0, a0, a3
 ; RV32I-NEXT:    sll a3, a1, a5
 ; RV32I-NEXT:    sub a4, a6, a4
-; RV32I-NEXT:    srli a5, t0, 1
+; RV32I-NEXT:    srli a5, t1, 1
 ; RV32I-NEXT:    srl a4, a5, a4
 ; RV32I-NEXT:    or a3, a3, a4
 ; RV32I-NEXT:    bgez a7, .LBB9_7
 ; RV32I-NEXT:  .LBB9_6:
 ; RV32I-NEXT:    srl a1, a1, a2
 ; RV32I-NEXT:    or a3, a3, a1
 ; RV32I-NEXT:  .LBB9_7:
 ; RV32I-NEXT:    mv a1, a3
 ; RV32I-NEXT:    ret
 ;
-; RV32B-LABEL: ror_i64:
-; RV32B:       # %bb.0:
-; RV32B-NEXT:    srl a7, a0, a2
-; RV32B-NEXT:    andi a4, a2, 63
-; RV32B-NEXT:    addi a6, zero, 31
-; RV32B-NEXT:    sub a5, a6, a4
-; RV32B-NEXT:    slli a3, a1, 1
-; RV32B-NEXT:    sll a3, a3, a5
-; RV32B-NEXT:    or a7, a7, a3
-; RV32B-NEXT:    addi t1, a4, -32
-; RV32B-NEXT:    srl a5, a1, t1
-; RV32B-NEXT:    slti a3, t1, 0
-; RV32B-NEXT:    cmov a7, a3, a7, a5
-; RV32B-NEXT:    neg a5, a2
-; RV32B-NEXT:    sll t0, a0, a5
-; RV32B-NEXT:    andi t2, a5, 63
-; RV32B-NEXT:    addi a4, t2, -32
-; RV32B-NEXT:    srai a3, a4, 31
-; RV32B-NEXT:    and a3, a3, t0
-; RV32B-NEXT:    or a7, a7, a3
-; RV32B-NEXT:    sll t0, a1, a5
-; RV32B-NEXT:    sub a5, a6, t2
-; RV32B-NEXT:    srli a3, a0, 1
-; RV32B-NEXT:    srl a3, a3, a5
-; RV32B-NEXT:    or a3, t0, a3
-; RV32B-NEXT:    sll a0, a0, a4
-; RV32B-NEXT:    slti a4, a4, 0
-; RV32B-NEXT:    cmov a0, a4, a3, a0
-; RV32B-NEXT:    srl a1, a1, a2
-; RV32B-NEXT:    srai a2, t1, 31
-; RV32B-NEXT:    and a1, a2, a1
-; RV32B-NEXT:    or a1, a1, a0
-; RV32B-NEXT:    mv a0, a7
-; RV32B-NEXT:    ret
-;
-; RV32ZBB-LABEL: ror_i64:
-; RV32ZBB:       # %bb.0:
-; RV32ZBB-NEXT:    mv t0, a0
-; RV32ZBB-NEXT:    andi a0, a2, 63
-; RV32ZBB-NEXT:    addi a7, a0, -32
-; RV32ZBB-NEXT:    addi a6, zero, 31
-; RV32ZBB-NEXT:    bltz a7, .LBB9_2
-; RV32ZBB-NEXT:  # %bb.1:
-; RV32ZBB-NEXT:    srl a0, a1, a7
-; RV32ZBB-NEXT:    j .LBB9_3
-; RV32ZBB-NEXT:  .LBB9_2:
-; RV32ZBB-NEXT:    srl a3, t0, a2
-; RV32ZBB-NEXT:    sub a0, a6, a0
-; RV32ZBB-NEXT:    slli a4, a1, 1
-; RV32ZBB-NEXT:    sll a0, a4, a0
-; RV32ZBB-NEXT:    or a0, a3, a0
-; RV32ZBB-NEXT:  .LBB9_3:
-; RV32ZBB-NEXT:    neg a5, a2
-; RV32ZBB-NEXT:    andi a4, a5, 63
-; RV32ZBB-NEXT:    addi a3, a4, -32
-; RV32ZBB-NEXT:    bltz a3, .LBB9_5
-; RV32ZBB-NEXT:  # %bb.4:
-; RV32ZBB-NEXT:    sll a3, t0, a3
-; RV32ZBB-NEXT:    bltz a7, .LBB9_6
-; RV32ZBB-NEXT:    j .LBB9_7
-; RV32ZBB-NEXT:  .LBB9_5:
-; RV32ZBB-NEXT:    sll a3, t0, a5
-; RV32ZBB-NEXT:    or a0, a0, a3
-; RV32ZBB-NEXT:    sll a3, a1, a5
-; RV32ZBB-NEXT:    sub a4, a6, a4
-; RV32ZBB-NEXT:    srli a5, t0, 1
-; RV32ZBB-NEXT:    srl a4, a5, a4
-; RV32ZBB-NEXT:    or a3, a3, a4
-; RV32ZBB-NEXT:    bgez a7, .LBB9_7
-; RV32ZBB-NEXT:  .LBB9_6:
-; RV32ZBB-NEXT:    srl a1, a1, a2
-; RV32ZBB-NEXT:    or a3, a3, a1
-; RV32ZBB-NEXT:  .LBB9_7:
-; RV32ZBB-NEXT:    mv a1, a3
-; RV32ZBB-NEXT:    ret
-;
-; RV32ZBP-LABEL: ror_i64:
-; RV32ZBP:       # %bb.0:
-; RV32ZBP-NEXT:    mv t0, a0
-; RV32ZBP-NEXT:    andi a0, a2, 63
-; RV32ZBP-NEXT:    addi a7, a0, -32
-; RV32ZBP-NEXT:    addi a6, zero, 31
-; RV32ZBP-NEXT:    bltz a7, .LBB9_2
-; RV32ZBP-NEXT:  # %bb.1:
-; RV32ZBP-NEXT:    srl a0, a1, a7
-; RV32ZBP-NEXT:    j .LBB9_3
-; RV32ZBP-NEXT:  .LBB9_2:
-; RV32ZBP-NEXT:    srl a3, t0, a2
-; RV32ZBP-NEXT:    sub a0, a6, a0
-; RV32ZBP-NEXT:    slli a4, a1, 1
-; RV32ZBP-NEXT:    sll a0, a4, a0
-; RV32ZBP-NEXT:    or a0, a3, a0
-; RV32ZBP-NEXT:  .LBB9_3:
-; RV32ZBP-NEXT:    neg a5, a2
-; RV32ZBP-NEXT:    andi a4, a5, 63
-; RV32ZBP-NEXT:    addi a3, a4, -32
-; RV32ZBP-NEXT:    bltz a3, .LBB9_5
-; RV32ZBP-NEXT:  # %bb.4:
-; RV32ZBP-NEXT:    sll a3, t0, a3
-; RV32ZBP-NEXT:    bltz a7, .LBB9_6
-; RV32ZBP-NEXT:    j .LBB9_7
-; RV32ZBP-NEXT:  .LBB9_5:
-; RV32ZBP-NEXT:    sll a3, t0, a5
-; RV32ZBP-NEXT:    or a0, a0, a3
-; RV32ZBP-NEXT:    sll a3, a1, a5
-; RV32ZBP-NEXT:    sub a4, a6, a4
-; RV32ZBP-NEXT:    srli a5, t0, 1
-; RV32ZBP-NEXT:    srl a4, a5, a4
-; RV32ZBP-NEXT:    or a3, a3, a4
-; RV32ZBP-NEXT:    bgez a7, .LBB9_7
-; RV32ZBP-NEXT:  .LBB9_6:
-; RV32ZBP-NEXT:    srl a1, a1, a2
-; RV32ZBP-NEXT:    or a3, a3, a1
-; RV32ZBP-NEXT:  .LBB9_7:
-; RV32ZBP-NEXT:    mv a1, a3
-; RV32ZBP-NEXT:    ret
+; RV32IB-LABEL: ror_i64:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    srl a7, a0, a2
+; RV32IB-NEXT:    andi a4, a2, 63
+; RV32IB-NEXT:    addi a6, zero, 31
+; RV32IB-NEXT:    sub a5, a6, a4
+; RV32IB-NEXT:    slli a3, a1, 1
+; RV32IB-NEXT:    sll a3, a3, a5
+; RV32IB-NEXT:    or a3, a7, a3
+; RV32IB-NEXT:    addi a7, a4, -32
+; RV32IB-NEXT:    srl a5, a1, a7
+; RV32IB-NEXT:    slti a4, a7, 0
+; RV32IB-NEXT:    cmov t0, a4, a3, a5
+; RV32IB-NEXT:    neg a4, a2
+; RV32IB-NEXT:    sll t2, a0, a4
+; RV32IB-NEXT:    andi a3, a4, 63
+; RV32IB-NEXT:    addi t1, a3, -32
+; RV32IB-NEXT:    srai a5, t1, 31
+; RV32IB-NEXT:    and a5, a5, t2
+; RV32IB-NEXT:    or t0, t0, a5
+; RV32IB-NEXT:    sll a4, a1, a4
+; RV32IB-NEXT:    sub a3, a6, a3
+; RV32IB-NEXT:    srli a5, a0, 1
+; RV32IB-NEXT:    srl a3, a5, a3
+; RV32IB-NEXT:    or a3, a4, a3
+; RV32IB-NEXT:    sll a0, a0, t1
+; RV32IB-NEXT:    slti a4, t1, 0
+; RV32IB-NEXT:    cmov a0, a4, a3, a0
+; RV32IB-NEXT:    srl a1, a1, a2
+; RV32IB-NEXT:    srai a2, a7, 31
+; RV32IB-NEXT:    and a1, a2, a1
+; RV32IB-NEXT:    or a1, a1, a0
+; RV32IB-NEXT:    mv a0, t0
+; RV32IB-NEXT:    ret
+;
+; RV32IBB-LABEL: ror_i64:
+; RV32IBB:       # %bb.0:
+; RV32IBB-NEXT:    mv t1, a0
+; RV32IBB-NEXT:    andi a0, a2, 63
+; RV32IBB-NEXT:    addi a7, a0, -32
+; RV32IBB-NEXT:    addi a6, zero, 31
+; RV32IBB-NEXT:    bltz a7, .LBB9_2
+; RV32IBB-NEXT:  # %bb.1:
+; RV32IBB-NEXT:    srl a0, a1, a7
+; RV32IBB-NEXT:    j .LBB9_3
+; RV32IBB-NEXT:  .LBB9_2:
+; RV32IBB-NEXT:    srl a4, t1, a2
+; RV32IBB-NEXT:    sub a0, a6, a0
+; RV32IBB-NEXT:    slli a5, a1, 1
+; RV32IBB-NEXT:    sll a0, a5, a0
+; RV32IBB-NEXT:    or a0, a4, a0
+; RV32IBB-NEXT:  .LBB9_3:
+; RV32IBB-NEXT:    neg a5, a2
+; RV32IBB-NEXT:    andi a4, a5, 63
+; RV32IBB-NEXT:    addi t0, a4, -32
+; RV32IBB-NEXT:    bltz t0, .LBB9_5
+; RV32IBB-NEXT:  # %bb.4:
+; RV32IBB-NEXT:    sll a3, t1, t0
+; RV32IBB-NEXT:    bltz a7, .LBB9_6
+; RV32IBB-NEXT:    j .LBB9_7
+; RV32IBB-NEXT:  .LBB9_5:
+; RV32IBB-NEXT:    sll a3, t1, a5
+; RV32IBB-NEXT:    or a0, a0, a3
+; RV32IBB-NEXT:    sll a3, a1, a5
+; RV32IBB-NEXT:    sub a4, a6, a4
+; RV32IBB-NEXT:    srli a5, t1, 1
+; RV32IBB-NEXT:    srl a4, a5, a4
+; RV32IBB-NEXT:    or a3, a3, a4
+; RV32IBB-NEXT:    bgez a7, .LBB9_7
+; RV32IBB-NEXT:  .LBB9_6:
+; RV32IBB-NEXT:    srl a1, a1, a2
+; RV32IBB-NEXT:    or a3, a3, a1
+; RV32IBB-NEXT:  .LBB9_7:
+; RV32IBB-NEXT:    mv a1, a3
+; RV32IBB-NEXT:    ret
+;
+; RV32IBP-LABEL: ror_i64:
+; RV32IBP:       # %bb.0:
+; RV32IBP-NEXT:    mv t1, a0
+; RV32IBP-NEXT:    andi a0, a2, 63
+; RV32IBP-NEXT:    addi a7, a0, -32
+; RV32IBP-NEXT:    addi a6, zero, 31
+; RV32IBP-NEXT:    bltz a7, .LBB9_2
+; RV32IBP-NEXT:  # %bb.1:
+; RV32IBP-NEXT:    srl a0, a1, a7
+; RV32IBP-NEXT:    j .LBB9_3
+; RV32IBP-NEXT:  .LBB9_2:
+; RV32IBP-NEXT:    srl a4, t1, a2
+; RV32IBP-NEXT:    sub a0, a6, a0
+; RV32IBP-NEXT:    slli a5, a1, 1
+; RV32IBP-NEXT:    sll a0, a5, a0
+; RV32IBP-NEXT:    or a0, a4, a0
+; RV32IBP-NEXT:  .LBB9_3:
+; RV32IBP-NEXT:    neg a5, a2
+; RV32IBP-NEXT:    andi a4, a5, 63
+; RV32IBP-NEXT:    addi t0, a4, -32
+; RV32IBP-NEXT:    bltz t0, .LBB9_5
+; RV32IBP-NEXT:  # %bb.4:
+; RV32IBP-NEXT:    sll a3, t1, t0
+; RV32IBP-NEXT:    bltz a7, .LBB9_6
+; RV32IBP-NEXT:    j .LBB9_7
+; RV32IBP-NEXT:  .LBB9_5:
+; RV32IBP-NEXT:    sll a3, t1, a5
+; RV32IBP-NEXT:    or a0, a0, a3
+; RV32IBP-NEXT:    sll a3, a1, a5
+; RV32IBP-NEXT:    sub a4, a6, a4
+; RV32IBP-NEXT:    srli a5, t1, 1
+; RV32IBP-NEXT:    srl a4, a5, a4
+; RV32IBP-NEXT:    or a3, a3, a4
+; RV32IBP-NEXT:    bgez a7, .LBB9_7
+; RV32IBP-NEXT:  .LBB9_6:
+; RV32IBP-NEXT:    srl a1, a1, a2
+; RV32IBP-NEXT:    or a3, a3, a1
+; RV32IBP-NEXT:  .LBB9_7:
+; RV32IBP-NEXT:    mv a1, a3
+; RV32IBP-NEXT:    ret
   %or = tail call i64 @llvm.fshr.i64(i64 %a, i64 %a, i64 %b)
   ret i64 %or
 }
 
 define i32 @rori_i32_fshl(i32 %a) nounwind {
 ; RV32I-LABEL: rori_i32_fshl:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    srli a1, a0, 1

llvm/test/CodeGen/RISCV/rv32zbb.ll

 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s4, 8(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s5, 4(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s6, 0(sp) # 4-byte Folded Spill
-; RV32I-NEXT:    mv s3, a1
-; RV32I-NEXT:    mv s4, a0
+; RV32I-NEXT:    mv s0, a1
+; RV32I-NEXT:    mv s1, a0
 ; RV32I-NEXT:    srli a0, a1, 1
 ; RV32I-NEXT:    or a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 2
 ; RV32I-NEXT:    or a0, a0, a1
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    or a0, a0, a1
 ; RV32I-NEXT:    srli a1, a0, 8
 ; RV32I-NEXT:    or a0, a0, a1
 ; RV32I-NEXT:    srli a1, a0, 16
 ; RV32I-NEXT:    or a0, a0, a1
 ; RV32I-NEXT:    not a0, a0
 ; RV32I-NEXT:    srli a1, a0, 1
 ; RV32I-NEXT:    lui a2, 349525
-; RV32I-NEXT:    addi s5, a2, 1365
-; RV32I-NEXT:    and a1, a1, s5
+; RV32I-NEXT:    addi s4, a2, 1365
+; RV32I-NEXT:    and a1, a1, s4
 ; RV32I-NEXT:    sub a0, a0, a1
 ; RV32I-NEXT:    lui a1, 209715
-; RV32I-NEXT:    addi s1, a1, 819
-; RV32I-NEXT:    and a1, a0, s1
+; RV32I-NEXT:    addi s5, a1, 819
+; RV32I-NEXT:    and a1, a0, s5
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    and a0, a0, s1
+; RV32I-NEXT:    and a0, a0, s5
 ; RV32I-NEXT:    add a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    add a0, a0, a1
 ; RV32I-NEXT:    lui a1, 61681
 ; RV32I-NEXT:    addi s6, a1, -241
 ; RV32I-NEXT:    and a0, a0, s6
 ; RV32I-NEXT:    lui a1, 4112
-; RV32I-NEXT:    addi s0, a1, 257
-; RV32I-NEXT:    mv a1, s0
+; RV32I-NEXT:    addi s3, a1, 257
+; RV32I-NEXT:    mv a1, s3
 ; RV32I-NEXT:    call __mulsi3@plt
 ; RV32I-NEXT:    mv s2, a0
-; RV32I-NEXT:    srli a0, s4, 1
-; RV32I-NEXT:    or a0, s4, a0
+; RV32I-NEXT:    srli a0, s1, 1
+; RV32I-NEXT:    or a0, s1, a0
 ; RV32I-NEXT:    srli a1, a0, 2
 ; RV32I-NEXT:    or a0, a0, a1
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    or a0, a0, a1
 ; RV32I-NEXT:    srli a1, a0, 8
 ; RV32I-NEXT:    or a0, a0, a1
 ; RV32I-NEXT:    srli a1, a0, 16
 ; RV32I-NEXT:    or a0, a0, a1
 ; RV32I-NEXT:    not a0, a0
 ; RV32I-NEXT:    srli a1, a0, 1
-; RV32I-NEXT:    and a1, a1, s5
+; RV32I-NEXT:    and a1, a1, s4
 ; RV32I-NEXT:    sub a0, a0, a1
-; RV32I-NEXT:    and a1, a0, s1
+; RV32I-NEXT:    and a1, a0, s5
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    and a0, a0, s1
+; RV32I-NEXT:    and a0, a0, s5
 ; RV32I-NEXT:    add a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    add a0, a0, a1
 ; RV32I-NEXT:    and a0, a0, s6
-; RV32I-NEXT:    mv a1, s0
+; RV32I-NEXT:    mv a1, s3
 ; RV32I-NEXT:    call __mulsi3@plt
-; RV32I-NEXT:    bnez s3, .LBB1_2
+; RV32I-NEXT:    bnez s0, .LBB1_2
 ; RV32I-NEXT:  # %bb.1:
 ; RV32I-NEXT:    srli a0, a0, 24
 ; RV32I-NEXT:    addi a0, a0, 32
 ; RV32I-NEXT:    j .LBB1_3
 ; RV32I-NEXT:  .LBB1_2:
 ; RV32I-NEXT:    srli a0, s2, 24
 ; RV32I-NEXT:  .LBB1_3:
 ; RV32I-NEXT:    mv a1, zero

 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s4, 8(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s5, 4(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s6, 0(sp) # 4-byte Folded Spill
-; RV32I-NEXT:    mv s3, a1
-; RV32I-NEXT:    mv s4, a0
+; RV32I-NEXT:    mv s1, a1
+; RV32I-NEXT:    mv s0, a0
 ; RV32I-NEXT:    addi a0, a0, -1
-; RV32I-NEXT:    not a1, s4
+; RV32I-NEXT:    not a1, s0
 ; RV32I-NEXT:    and a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 1
 ; RV32I-NEXT:    lui a2, 349525
-; RV32I-NEXT:    addi s5, a2, 1365
-; RV32I-NEXT:    and a1, a1, s5
+; RV32I-NEXT:    addi s4, a2, 1365
+; RV32I-NEXT:    and a1, a1, s4
 ; RV32I-NEXT:    sub a0, a0, a1
 ; RV32I-NEXT:    lui a1, 209715
-; RV32I-NEXT:    addi s0, a1, 819
-; RV32I-NEXT:    and a1, a0, s0
+; RV32I-NEXT:    addi s5, a1, 819
+; RV32I-NEXT:    and a1, a0, s5
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    and a0, a0, s0
+; RV32I-NEXT:    and a0, a0, s5
 ; RV32I-NEXT:    add a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    add a0, a0, a1
 ; RV32I-NEXT:    lui a1, 61681
 ; RV32I-NEXT:    addi s6, a1, -241
 ; RV32I-NEXT:    and a0, a0, s6
 ; RV32I-NEXT:    lui a1, 4112
-; RV32I-NEXT:    addi s1, a1, 257
-; RV32I-NEXT:    mv a1, s1
+; RV32I-NEXT:    addi s3, a1, 257
+; RV32I-NEXT:    mv a1, s3
 ; RV32I-NEXT:    call __mulsi3@plt
 ; RV32I-NEXT:    mv s2, a0
-; RV32I-NEXT:    addi a0, s3, -1
-; RV32I-NEXT:    not a1, s3
+; RV32I-NEXT:    addi a0, s1, -1
+; RV32I-NEXT:    not a1, s1
 ; RV32I-NEXT:    and a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 1
-; RV32I-NEXT:    and a1, a1, s5
+; RV32I-NEXT:    and a1, a1, s4
 ; RV32I-NEXT:    sub a0, a0, a1
-; RV32I-NEXT:    and a1, a0, s0
+; RV32I-NEXT:    and a1, a0, s5
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    and a0, a0, s0
+; RV32I-NEXT:    and a0, a0, s5
 ; RV32I-NEXT:    add a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    add a0, a0, a1
 ; RV32I-NEXT:    and a0, a0, s6
-; RV32I-NEXT:    mv a1, s1
+; RV32I-NEXT:    mv a1, s3
 ; RV32I-NEXT:    call __mulsi3@plt
-; RV32I-NEXT:    bnez s4, .LBB3_2
+; RV32I-NEXT:    bnez s0, .LBB3_2
 ; RV32I-NEXT:  # %bb.1:
 ; RV32I-NEXT:    srli a0, a0, 24
 ; RV32I-NEXT:    addi a0, a0, 32
 ; RV32I-NEXT:    j .LBB3_3
 ; RV32I-NEXT:  .LBB3_2:
 ; RV32I-NEXT:    srli a0, s2, 24
 ; RV32I-NEXT:  .LBB3_3:
 ; RV32I-NEXT:    mv a1, zero

 ; RV32I-NEXT:    addi sp, sp, -32
 ; RV32I-NEXT:    sw ra, 28(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s0, 24(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s1, 20(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s2, 16(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s3, 12(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s4, 8(sp) # 4-byte Folded Spill
 ; RV32I-NEXT:    sw s5, 4(sp) # 4-byte Folded Spill
-; RV32I-NEXT:    mv s2, a0
+; RV32I-NEXT:    mv s0, a0
 ; RV32I-NEXT:    srli a0, a1, 1
 ; RV32I-NEXT:    lui a2, 349525
-; RV32I-NEXT:    addi s3, a2, 1365
-; RV32I-NEXT:    and a0, a0, s3
+; RV32I-NEXT:    addi s2, a2, 1365
+; RV32I-NEXT:    and a0, a0, s2
 ; RV32I-NEXT:    sub a0, a1, a0
 ; RV32I-NEXT:    lui a1, 209715
-; RV32I-NEXT:    addi s0, a1, 819
-; RV32I-NEXT:    and a1, a0, s0
+; RV32I-NEXT:    addi s1, a1, 819
+; RV32I-NEXT:    and a1, a0, s1
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    and a0, a0, s0
+; RV32I-NEXT:    and a0, a0, s1
 ; RV32I-NEXT:    add a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    add a0, a0, a1
 ; RV32I-NEXT:    lui a1, 61681
 ; RV32I-NEXT:    addi s4, a1, -241
 ; RV32I-NEXT:    and a0, a0, s4
 ; RV32I-NEXT:    lui a1, 4112
-; RV32I-NEXT:    addi s1, a1, 257
-; RV32I-NEXT:    mv a1, s1
+; RV32I-NEXT:    addi s3, a1, 257
+; RV32I-NEXT:    mv a1, s3
 ; RV32I-NEXT:    call __mulsi3@plt
 ; RV32I-NEXT:    srli s5, a0, 24
-; RV32I-NEXT:    srli a0, s2, 1
-; RV32I-NEXT:    and a0, a0, s3
-; RV32I-NEXT:    sub a0, s2, a0
-; RV32I-NEXT:    and a1, a0, s0
+; RV32I-NEXT:    srli a0, s0, 1
+; RV32I-NEXT:    and a0, a0, s2
+; RV32I-NEXT:    sub a0, s0, a0
+; RV32I-NEXT:    and a1, a0, s1
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    and a0, a0, s0
+; RV32I-NEXT:    and a0, a0, s1
 ; RV32I-NEXT:    add a0, a1, a0
 ; RV32I-NEXT:    srli a1, a0, 4
 ; RV32I-NEXT:    add a0, a0, a1
 ; RV32I-NEXT:    and a0, a0, s4
-; RV32I-NEXT:    mv a1, s1
+; RV32I-NEXT:    mv a1, s3
 ; RV32I-NEXT:    call __mulsi3@plt
 ; RV32I-NEXT:    srli a0, a0, 24
 ; RV32I-NEXT:    add a0, a0, s5
 ; RV32I-NEXT:    mv a1, zero
 ; RV32I-NEXT:    lw s5, 4(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s4, 8(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s3, 12(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s2, 16(sp) # 4-byte Folded Reload

llvm/test/CodeGen/RISCV/rv32zbp.ll

 
 ; gorc1, gorc2, gorc1 -> gorc2
 define i64 @gorc3b_i64(i64 %a) nounwind {
 ; RV32I-LABEL: gorc3b_i64:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    slli a2, a0, 1
 ; RV32I-NEXT:    slli a3, a1, 1
 ; RV32I-NEXT:    lui a4, 699051
-; RV32I-NEXT:    addi a4, a4, -1366
-; RV32I-NEXT:    and a6, a3, a4
-; RV32I-NEXT:    and a7, a2, a4
+; RV32I-NEXT:    addi a6, a4, -1366
+; RV32I-NEXT:    and a7, a3, a6
+; RV32I-NEXT:    and a2, a2, a6
 ; RV32I-NEXT:    srli a5, a1, 1
-; RV32I-NEXT:    srli a3, a0, 1
-; RV32I-NEXT:    lui a2, 349525
-; RV32I-NEXT:    addi a2, a2, 1365
-; RV32I-NEXT:    and a3, a3, a2
-; RV32I-NEXT:    and a5, a5, a2
+; RV32I-NEXT:    srli a4, a0, 1
+; RV32I-NEXT:    lui a3, 349525
+; RV32I-NEXT:    addi t0, a3, 1365
+; RV32I-NEXT:    and a4, a4, t0
+; RV32I-NEXT:    and a5, a5, t0
 ; RV32I-NEXT:    or a1, a5, a1
-; RV32I-NEXT:    or a0, a3, a0
-; RV32I-NEXT:    or a0, a0, a7
-; RV32I-NEXT:    or a1, a1, a6
-; RV32I-NEXT:    slli a6, a1, 2
-; RV32I-NEXT:    slli a5, a0, 2
-; RV32I-NEXT:    lui a3, 838861
-; RV32I-NEXT:    addi a3, a3, -820
-; RV32I-NEXT:    and a7, a5, a3
-; RV32I-NEXT:    and a6, a6, a3
-; RV32I-NEXT:    srli t0, a0, 2
+; RV32I-NEXT:    or a0, a4, a0
+; RV32I-NEXT:    or a0, a0, a2
+; RV32I-NEXT:    or a1, a1, a7
+; RV32I-NEXT:    slli a2, a1, 2
+; RV32I-NEXT:    slli a4, a0, 2
+; RV32I-NEXT:    lui a5, 838861
+; RV32I-NEXT:    addi a5, a5, -820
+; RV32I-NEXT:    and a7, a4, a5
+; RV32I-NEXT:    and a2, a2, a5
+; RV32I-NEXT:    srli a5, a0, 2
 ; RV32I-NEXT:    srli a3, a1, 2
-; RV32I-NEXT:    lui a5, 209715
-; RV32I-NEXT:    addi a5, a5, 819
-; RV32I-NEXT:    and a3, a3, a5
-; RV32I-NEXT:    and a5, t0, a5
-; RV32I-NEXT:    or a0, a5, a0
+; RV32I-NEXT:    lui a4, 209715
+; RV32I-NEXT:    addi a4, a4, 819
+; RV32I-NEXT:    and a3, a3, a4
+; RV32I-NEXT:    and a4, a5, a4
+; RV32I-NEXT:    or a0, a4, a0
 ; RV32I-NEXT:    or a1, a3, a1
-; RV32I-NEXT:    or a1, a1, a6
+; RV32I-NEXT:    or a1, a1, a2
 ; RV32I-NEXT:    or a0, a0, a7
-; RV32I-NEXT:    slli a3, a0, 1
-; RV32I-NEXT:    slli a5, a1, 1
-; RV32I-NEXT:    and a6, a5, a4
-; RV32I-NEXT:    and a3, a3, a4
+; RV32I-NEXT:    slli a2, a0, 1
+; RV32I-NEXT:    slli a3, a1, 1
+; RV32I-NEXT:    and a3, a3, a6
+; RV32I-NEXT:    and a2, a2, a6
 ; RV32I-NEXT:    srli a4, a1, 1
 ; RV32I-NEXT:    srli a5, a0, 1
-; RV32I-NEXT:    and a5, a5, a2
-; RV32I-NEXT:    and a2, a4, a2
-; RV32I-NEXT:    or a1, a2, a1
+; RV32I-NEXT:    and a5, a5, t0
+; RV32I-NEXT:    and a4, a4, t0
+; RV32I-NEXT:    or a1, a4, a1
 ; RV32I-NEXT:    or a0, a5, a0
-; RV32I-NEXT:    or a0, a0, a3
-; RV32I-NEXT:    or a1, a1, a6
+; RV32I-NEXT:    or a0, a0, a2
+; RV32I-NEXT:    or a1, a1, a3
 ; RV32I-NEXT:    ret
 ;
 ; RV32B-LABEL: gorc3b_i64:
 ; RV32B:       # %bb.0:
 ; RV32B-NEXT:    orc.n a0, a0
 ; RV32B-NEXT:    orc.n a1, a1
 ; RV32B-NEXT:    ret
 ;

 
 ; grev1, grev2, grev1 -> grev2
 define i64 @grev2b_i64(i64 %a) nounwind {
 ; RV32I-LABEL: grev2b_i64:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    slli a2, a0, 1
 ; RV32I-NEXT:    slli a3, a1, 1
 ; RV32I-NEXT:    lui a4, 699051
-; RV32I-NEXT:    addi a4, a4, -1366
-; RV32I-NEXT:    and a3, a3, a4
-; RV32I-NEXT:    and a2, a2, a4
+; RV32I-NEXT:    addi a6, a4, -1366
+; RV32I-NEXT:    and a3, a3, a6
+; RV32I-NEXT:    and a2, a2, a6
 ; RV32I-NEXT:    srli a0, a0, 1
 ; RV32I-NEXT:    srli a1, a1, 1
 ; RV32I-NEXT:    lui a5, 349525
 ; RV32I-NEXT:    addi a5, a5, 1365
 ; RV32I-NEXT:    and a1, a1, a5
 ; RV32I-NEXT:    and a0, a0, a5
 ; RV32I-NEXT:    or a0, a2, a0
 ; RV32I-NEXT:    or a1, a3, a1
-; RV32I-NEXT:    slli a6, a1, 2
+; RV32I-NEXT:    slli a2, a1, 2
 ; RV32I-NEXT:    slli a3, a0, 2
-; RV32I-NEXT:    lui a2, 838861
-; RV32I-NEXT:    addi a2, a2, -820
-; RV32I-NEXT:    and a7, a3, a2
-; RV32I-NEXT:    and a2, a6, a2
+; RV32I-NEXT:    lui a4, 838861
+; RV32I-NEXT:    addi a4, a4, -820
+; RV32I-NEXT:    and a3, a3, a4
+; RV32I-NEXT:    and a2, a2, a4
 ; RV32I-NEXT:    srli a1, a1, 2
 ; RV32I-NEXT:    srli a0, a0, 2
-; RV32I-NEXT:    lui a3, 209715
-; RV32I-NEXT:    addi a3, a3, 819
-; RV32I-NEXT:    and a0, a0, a3
-; RV32I-NEXT:    and a1, a1, a3
+; RV32I-NEXT:    lui a4, 209715
+; RV32I-NEXT:    addi a4, a4, 819
+; RV32I-NEXT:    and a0, a0, a4
+; RV32I-NEXT:    and a1, a1, a4
 ; RV32I-NEXT:    or a1, a2, a1
-; RV32I-NEXT:    or a0, a7, a0
+; RV32I-NEXT:    or a0, a3, a0
 ; RV32I-NEXT:    slli a2, a0, 1
 ; RV32I-NEXT:    slli a3, a1, 1
-; RV32I-NEXT:    and a3, a3, a4
-; RV32I-NEXT:    and a2, a2, a4
+; RV32I-NEXT:    and a3, a3, a6
+; RV32I-NEXT:    and a2, a2, a6
 ; RV32I-NEXT:    srli a0, a0, 1
 ; RV32I-NEXT:    srli a1, a1, 1
 ; RV32I-NEXT:    and a1, a1, a5
 ; RV32I-NEXT:    and a0, a0, a5
 ; RV32I-NEXT:    or a0, a2, a0
 ; RV32I-NEXT:    or a1, a3, a1
 ; RV32I-NEXT:    ret
 ;

 
 ; grev1, grev2, grev1, grev2 -> identity
 define i64 @grev0_i64(i64 %a) nounwind {
 ; RV32I-LABEL: grev0_i64:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    slli a2, a1, 1
 ; RV32I-NEXT:    slli a3, a0, 1
 ; RV32I-NEXT:    lui a4, 699051
-; RV32I-NEXT:    addi a4, a4, -1366
-; RV32I-NEXT:    and a3, a3, a4
-; RV32I-NEXT:    and a2, a2, a4
+; RV32I-NEXT:    addi a6, a4, -1366
+; RV32I-NEXT:    and a3, a3, a6
+; RV32I-NEXT:    and a2, a2, a6
 ; RV32I-NEXT:    srli a1, a1, 1
 ; RV32I-NEXT:    srli a0, a0, 1
 ; RV32I-NEXT:    lui a5, 349525
-; RV32I-NEXT:    addi a5, a5, 1365
-; RV32I-NEXT:    and a0, a0, a5
-; RV32I-NEXT:    and a1, a1, a5
+; RV32I-NEXT:    addi a7, a5, 1365
+; RV32I-NEXT:    and a0, a0, a7
+; RV32I-NEXT:    and a1, a1, a7
 ; RV32I-NEXT:    or a1, a2, a1
 ; RV32I-NEXT:    or a0, a3, a0
-; RV32I-NEXT:    slli a6, a0, 2
+; RV32I-NEXT:    slli a2, a0, 2
 ; RV32I-NEXT:    slli a3, a1, 2
-; RV32I-NEXT:    lui a2, 838861
-; RV32I-NEXT:    addi a2, a2, -820
-; RV32I-NEXT:    and a7, a3, a2
-; RV32I-NEXT:    and a6, a6, a2
+; RV32I-NEXT:    lui a4, 838861
+; RV32I-NEXT:    addi a4, a4, -820
+; RV32I-NEXT:    and a3, a3, a4
+; RV32I-NEXT:    and a2, a2, a4
 ; RV32I-NEXT:    srli a0, a0, 2
 ; RV32I-NEXT:    srli a1, a1, 2
-; RV32I-NEXT:    lui a3, 209715
-; RV32I-NEXT:    addi a3, a3, 819
-; RV32I-NEXT:    and a1, a1, a3
-; RV32I-NEXT:    and a0, a0, a3
-; RV32I-NEXT:    or t0, a6, a0
-; RV32I-NEXT:    or a1, a7, a1
-; RV32I-NEXT:    slli a6, a1, 1
-; RV32I-NEXT:    slli a0, t0, 1
-; RV32I-NEXT:    and a7, a0, a4
-; RV32I-NEXT:    and a4, a6, a4
-; RV32I-NEXT:    srli a1, a1, 1
-; RV32I-NEXT:    srli a0, t0, 1
-; RV32I-NEXT:    and a0, a0, a5
+; RV32I-NEXT:    lui a5, 209715
+; RV32I-NEXT:    addi a5, a5, 819
 ; RV32I-NEXT:    and a1, a1, a5
-; RV32I-NEXT:    or a1, a4, a1
-; RV32I-NEXT:    or a0, a7, a0
-; RV32I-NEXT:    slli a4, a0, 2
-; RV32I-NEXT:    slli a5, a1, 2
-; RV32I-NEXT:    and a5, a5, a2
-; RV32I-NEXT:    and a2, a4, a2
+; RV32I-NEXT:    and a0, a0, a5
+; RV32I-NEXT:    or a0, a2, a0
+; RV32I-NEXT:    or a1, a3, a1
+; RV32I-NEXT:    slli a2, a1, 1
+; RV32I-NEXT:    slli a3, a0, 1
+; RV32I-NEXT:    and a3, a3, a6
+; RV32I-NEXT:    and a2, a2, a6
+; RV32I-NEXT:    srli a1, a1, 1
+; RV32I-NEXT:    srli a0, a0, 1
+; RV32I-NEXT:    and a0, a0, a7
+; RV32I-NEXT:    and a1, a1, a7
+; RV32I-NEXT:    or a1, a2, a1
+; RV32I-NEXT:    or a0, a3, a0
+; RV32I-NEXT:    slli a2, a0, 2
+; RV32I-NEXT:    slli a3, a1, 2
+; RV32I-NEXT:    and a3, a3, a4
+; RV32I-NEXT:    and a2, a2, a4
 ; RV32I-NEXT:    srli a0, a0, 2
 ; RV32I-NEXT:    srli a1, a1, 2
-; RV32I-NEXT:    and a1, a1, a3
-; RV32I-NEXT:    and a0, a0, a3
+; RV32I-NEXT:    and a1, a1, a5
+; RV32I-NEXT:    and a0, a0, a5
 ; RV32I-NEXT:    or a0, a2, a0
-; RV32I-NEXT:    or a1, a5, a1
+; RV32I-NEXT:    or a1, a3, a1
 ; RV32I-NEXT:    ret
 ;
 ; RV32B-LABEL: grev0_i64:
 ; RV32B:       # %bb.0:
 ; RV32B-NEXT:    ret
 ;
 ; RV32ZBP-LABEL: grev0_i64:
 ; RV32ZBP:       # %bb.0:

 
 declare i64 @llvm.bitreverse.i64(i64)
 
 define i64 @bitreverse_i64(i64 %a) nounwind {
 ; RV32I-LABEL: bitreverse_i64:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    srli a2, a1, 8
 ; RV32I-NEXT:    lui a3, 16
-; RV32I-NEXT:    addi t0, a3, -256
-; RV32I-NEXT:    and a2, a2, t0
+; RV32I-NEXT:    addi a6, a3, -256
+; RV32I-NEXT:    and a2, a2, a6
 ; RV32I-NEXT:    srli a4, a1, 24
 ; RV32I-NEXT:    or a2, a2, a4
 ; RV32I-NEXT:    slli a4, a1, 8
-; RV32I-NEXT:    lui a6, 4080
-; RV32I-NEXT:    and a4, a4, a6
+; RV32I-NEXT:    lui a7, 4080
+; RV32I-NEXT:    and a4, a4, a7
 ; RV32I-NEXT:    slli a1, a1, 24
 ; RV32I-NEXT:    or a1, a1, a4
 ; RV32I-NEXT:    or a1, a1, a2
 ; RV32I-NEXT:    lui a2, 61681
-; RV32I-NEXT:    addi t1, a2, -241
-; RV32I-NEXT:    and a2, a1, t1
+; RV32I-NEXT:    addi t0, a2, -241
+; RV32I-NEXT:    and a2, a1, t0
 ; RV32I-NEXT:    slli a2, a2, 4
-; RV32I-NEXT:    lui a5, 986895
-; RV32I-NEXT:    addi t2, a5, 240
-; RV32I-NEXT:    and a1, a1, t2
+; RV32I-NEXT:    lui a3, 986895
+; RV32I-NEXT:    addi t1, a3, 240
+; RV32I-NEXT:    and a1, a1, t1
 ; RV32I-NEXT:    srli a1, a1, 4
 ; RV32I-NEXT:    or a1, a1, a2
 ; RV32I-NEXT:    lui a2, 209715
-; RV32I-NEXT:    addi t3, a2, 819
-; RV32I-NEXT:    and a3, a1, t3
-; RV32I-NEXT:    slli a3, a3, 2
+; RV32I-NEXT:    addi t2, a2, 819
+; RV32I-NEXT:    and a2, a1, t2
+; RV32I-NEXT:    slli a2, a2, 2
 ; RV32I-NEXT:    lui a4, 838861
-; RV32I-NEXT:    addi a4, a4, -820
-; RV32I-NEXT:    and a1, a1, a4
+; RV32I-NEXT:    addi t3, a4, -820
+; RV32I-NEXT:    and a1, a1, t3
 ; RV32I-NEXT:    srli a1, a1, 2
-; RV32I-NEXT:    or a1, a1, a3
-; RV32I-NEXT:    lui a3, 349525
-; RV32I-NEXT:    addi a3, a3, 1365
-; RV32I-NEXT:    and a5, a1, a3
-; RV32I-NEXT:    slli a5, a5, 1
-; RV32I-NEXT:    lui a2, 699051
-; RV32I-NEXT:    addi a2, a2, -1366
-; RV32I-NEXT:    and a1, a1, a2
+; RV32I-NEXT:    or a1, a1, a2
+; RV32I-NEXT:    lui a2, 349525
+; RV32I-NEXT:    addi a3, a2, 1365
+; RV32I-NEXT:    and a2, a1, a3
+; RV32I-NEXT:    slli a2, a2, 1
+; RV32I-NEXT:    lui a5, 699051
+; RV32I-NEXT:    addi a5, a5, -1366
+; RV32I-NEXT:    and a1, a1, a5
 ; RV32I-NEXT:    srli a1, a1, 1
-; RV32I-NEXT:    or a7, a1, a5
+; RV32I-NEXT:    or a2, a1, a2
 ; RV32I-NEXT:    srli a1, a0, 8
-; RV32I-NEXT:    and a1, a1, t0
-; RV32I-NEXT:    srli a5, a0, 24
-; RV32I-NEXT:    or a1, a1, a5
-; RV32I-NEXT:    slli a5, a0, 8
-; RV32I-NEXT:    and a5, a5, a6
+; RV32I-NEXT:    and a1, a1, a6
+; RV32I-NEXT:    srli a4, a0, 24
+; RV32I-NEXT:    or a1, a1, a4
+; RV32I-NEXT:    slli a4, a0, 8
+; RV32I-NEXT:    and a4, a4, a7
 ; RV32I-NEXT:    slli a0, a0, 24
-; RV32I-NEXT:    or a0, a0, a5
+; RV32I-NEXT:    or a0, a0, a4
 ; RV32I-NEXT:    or a0, a0, a1
-; RV32I-NEXT:    and a1, a0, t1
+; RV32I-NEXT:    and a1, a0, t0
 ; RV32I-NEXT:    slli a1, a1, 4
-; RV32I-NEXT:    and a0, a0, t2
+; RV32I-NEXT:    and a0, a0, t1
 ; RV32I-NEXT:    srli a0, a0, 4
 ; RV32I-NEXT:    or a0, a0, a1
-; RV32I-NEXT:    and a1, a0, t3
+; RV32I-NEXT:    and a1, a0, t2
 ; RV32I-NEXT:    slli a1, a1, 2
-; RV32I-NEXT:    and a0, a0, a4
+; RV32I-NEXT:    and a0, a0, t3
 ; RV32I-NEXT:    srli a0, a0, 2
 ; RV32I-NEXT:    or a0, a0, a1
 ; RV32I-NEXT:    and a1, a0, a3
 ; RV32I-NEXT:    slli a1, a1, 1
-; RV32I-NEXT:    and a0, a0, a2
+; RV32I-NEXT:    and a0, a0, a5
 ; RV32I-NEXT:    srli a0, a0, 1
 ; RV32I-NEXT:    or a1, a0, a1
-; RV32I-NEXT:    mv a0, a7
+; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    ret
 ;
 ; RV32B-LABEL: bitreverse_i64:
 ; RV32B:       # %bb.0:
 ; RV32B-NEXT:    rev a2, a1
 ; RV32B-NEXT:    rev a1, a0
 ; RV32B-NEXT:    mv a0, a2
 ; RV32B-NEXT:    ret
; RV32ZBP-NEXT:    ret
   ret i32 %2
 }
 
 define i64 @bitreverse_bswap_i64(i64 %a) {
 ; RV32I-LABEL: bitreverse_bswap_i64:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    srli a3, a1, 8
 ; RV32I-NEXT:    lui a2, 16
-; RV32I-NEXT:    addi t1, a2, -256
-; RV32I-NEXT:    and a3, a3, t1
+; RV32I-NEXT:    addi t0, a2, -256
+; RV32I-NEXT:    and a3, a3, t0
 ; RV32I-NEXT:    srli a4, a1, 24
 ; RV32I-NEXT:    or a4, a3, a4
 ; RV32I-NEXT:    slli a5, a1, 8
-; RV32I-NEXT:    lui a6, 4080
-; RV32I-NEXT:    and a5, a5, a6
+; RV32I-NEXT:    lui t1, 4080
+; RV32I-NEXT:    and a5, a5, t1
 ; RV32I-NEXT:    slli a1, a1, 24
 ; RV32I-NEXT:    or a1, a1, a5
 ; RV32I-NEXT:    or a1, a1, a4
 ; RV32I-NEXT:    lui a4, 61681
-; RV32I-NEXT:    addi a7, a4, -241
-; RV32I-NEXT:    and a5, a1, a7
+; RV32I-NEXT:    addi a6, a4, -241
+; RV32I-NEXT:    and a5, a1, a6
 ; RV32I-NEXT:    slli a5, a5, 4
-; RV32I-NEXT:    lui a3, 986895
-; RV32I-NEXT:    addi t0, a3, 240
-; RV32I-NEXT:    and a1, a1, t0
+; RV32I-NEXT:    lui a4, 986895
+; RV32I-NEXT:    addi a7, a4, 240
+; RV32I-NEXT:    and a1, a1, a7
 ; RV32I-NEXT:    srli a1, a1, 4
 ; RV32I-NEXT:    or a1, a1, a5
 ; RV32I-NEXT:    lui a5, 209715
 ; RV32I-NEXT:    addi t2, a5, 819
 ; RV32I-NEXT:    and a4, a1, t2
 ; RV32I-NEXT:    slli a4, a4, 2
-; RV32I-NEXT:    lui a3, 838861
-; RV32I-NEXT:    addi t3, a3, -820
+; RV32I-NEXT:    lui a2, 838861
+; RV32I-NEXT:    addi t3, a2, -820
 ; RV32I-NEXT:    and a1, a1, t3
 ; RV32I-NEXT:    srli a1, a1, 2
 ; RV32I-NEXT:    or a1, a1, a4
 ; RV32I-NEXT:    lui a4, 349525
 ; RV32I-NEXT:    addi a4, a4, 1365
-; RV32I-NEXT:    and a2, a1, a4
-; RV32I-NEXT:    slli a2, a2, 1
+; RV32I-NEXT:    and a3, a1, a4
+; RV32I-NEXT:    slli a3, a3, 1
 ; RV32I-NEXT:    lui a5, 699051
 ; RV32I-NEXT:    addi a5, a5, -1366
 ; RV32I-NEXT:    and a1, a1, a5
 ; RV32I-NEXT:    srli a1, a1, 1
-; RV32I-NEXT:    or a1, a1, a2
-; RV32I-NEXT:    srli a2, a0, 8
-; RV32I-NEXT:    and a2, a2, t1
-; RV32I-NEXT:    srli a3, a0, 24
-; RV32I-NEXT:    or a2, a2, a3
+; RV32I-NEXT:    or a1, a1, a3
+; RV32I-NEXT:    srli a3, a0, 8
+; RV32I-NEXT:    and a3, a3, t0
+; RV32I-NEXT:    srli a2, a0, 24
+; RV32I-NEXT:    or a2, a3, a2
 ; RV32I-NEXT:    slli a3, a0, 8
-; RV32I-NEXT:    and a3, a3, a6
+; RV32I-NEXT:    and a3, a3, t1
 ; RV32I-NEXT:    slli a0, a0, 24
 ; RV32I-NEXT:    or a0, a0, a3
 ; RV32I-NEXT:    or a0, a0, a2
-; RV32I-NEXT:    and a2, a0, a7
+; RV32I-NEXT:    and a2, a0, a6
 ; RV32I-NEXT:    slli a2, a2, 4
-; RV32I-NEXT:    and a0, a0, t0
+; RV32I-NEXT:    and a0, a0, a7
 ; RV32I-NEXT:    srli a0, a0, 4
 ; RV32I-NEXT:    or a0, a0, a2
 ; RV32I-NEXT:    and a2, a0, t2
 ; RV32I-NEXT:    slli a2, a2, 2
 ; RV32I-NEXT:    and a0, a0, t3
 ; RV32I-NEXT:    srli a0, a0, 2
 ; RV32I-NEXT:    or a0, a0, a2
 ; RV32I-NEXT:    and a2, a0, a4
 ; RV32I-NEXT:    slli a2, a2, 1
 ; RV32I-NEXT:    and a0, a0, a5
 ; RV32I-NEXT:    srli a0, a0, 1
 ; RV32I-NEXT:    or a0, a0, a2
 ; RV32I-NEXT:    srli a2, a0, 8
-; RV32I-NEXT:    and a2, a2, t1
+; RV32I-NEXT:    and a2, a2, t0
 ; RV32I-NEXT:    srli a3, a0, 24
 ; RV32I-NEXT:    or a2, a2, a3
 ; RV32I-NEXT:    slli a3, a0, 8
-; RV32I-NEXT:    and a3, a3, a6
+; RV32I-NEXT:    and a3, a3, t1
 ; RV32I-NEXT:    slli a0, a0, 24
 ; RV32I-NEXT:    or a0, a0, a3
 ; RV32I-NEXT:    or a0, a0, a2
 ; RV32I-NEXT:    srli a2, a1, 8
-; RV32I-NEXT:    and a2, a2, t1
+; RV32I-NEXT:    and a2, a2, t0
 ; RV32I-NEXT:    srli a3, a1, 24
 ; RV32I-NEXT:    or a2, a2, a3
 ; RV32I-NEXT:    slli a3, a1, 8
-; RV32I-NEXT:    and a3, a3, a6
+; RV32I-NEXT:    and a3, a3, t1
 ; RV32I-NEXT:    slli a1, a1, 24
 ; RV32I-NEXT:    or a1, a1, a3
 ; RV32I-NEXT:    or a1, a1, a2
 ; RV32I-NEXT:    ret
 ;
 ; RV32B-LABEL: bitreverse_bswap_i64:
 ; RV32B:       # %bb.0:
 ; RV32B-NEXT:    rev.b a0, a0

llvm/test/CodeGen/RISCV/rv32zbt.ll

 ; RV32I-NEXT:    j .LBB13_3
 ; RV32I-NEXT:  .LBB13_2:
 ; RV32I-NEXT:    sll t0, a1, a4
 ; RV32I-NEXT:    sub a5, a6, a5
 ; RV32I-NEXT:    srli a1, a0, 1
 ; RV32I-NEXT:    srl a1, a1, a5
 ; RV32I-NEXT:    or a1, t0, a1
 ; RV32I-NEXT:  .LBB13_3:
-; RV32I-NEXT:    not t2, a4
-; RV32I-NEXT:    andi t1, t2, 63
-; RV32I-NEXT:    addi a5, t1, -32
-; RV32I-NEXT:    srli t0, a3, 1
-; RV32I-NEXT:    bltz a5, .LBB13_5
+; RV32I-NEXT:    not t0, a4
+; RV32I-NEXT:    andi t3, t0, 63
+; RV32I-NEXT:    addi t2, t3, -32
+; RV32I-NEXT:    srli t1, a3, 1
+; RV32I-NEXT:    bltz t2, .LBB13_5
 ; RV32I-NEXT:  # %bb.4:
-; RV32I-NEXT:    srl a2, t0, a5
+; RV32I-NEXT:    srl a2, t1, t2
 ; RV32I-NEXT:    bltz a7, .LBB13_6
 ; RV32I-NEXT:    j .LBB13_7
 ; RV32I-NEXT:  .LBB13_5:
-; RV32I-NEXT:    srl a5, t0, t2
+; RV32I-NEXT:    srl a5, t1, t0
 ; RV32I-NEXT:    or a1, a1, a5
 ; RV32I-NEXT:    slli a3, a3, 31
 ; RV32I-NEXT:    srli a2, a2, 1
 ; RV32I-NEXT:    or a2, a2, a3
-; RV32I-NEXT:    srl a2, a2, t2
-; RV32I-NEXT:    sub a3, a6, t1
-; RV32I-NEXT:    slli a5, t0, 1
+; RV32I-NEXT:    srl a2, a2, t0
+; RV32I-NEXT:    sub a3, a6, t3
+; RV32I-NEXT:    slli a5, t1, 1
 ; RV32I-NEXT:    sll a3, a5, a3
 ; RV32I-NEXT:    or a2, a2, a3
 ; RV32I-NEXT:    bgez a7, .LBB13_7
 ; RV32I-NEXT:  .LBB13_6:
 ; RV32I-NEXT:    sll a0, a0, a4
 ; RV32I-NEXT:    or a2, a2, a0
 ; RV32I-NEXT:  .LBB13_7:
 ; RV32I-NEXT:    mv a0, a2
 ; RV32I-NEXT:    ret
 ;
-; RV32B-LABEL: fshl_i64:
-; RV32B:       # %bb.0:
-; RV32B-NEXT:    sll a7, a1, a4
-; RV32B-NEXT:    andi a5, a4, 63
-; RV32B-NEXT:    addi a6, zero, 31
-; RV32B-NEXT:    sub t0, a6, a5
-; RV32B-NEXT:    srli a1, a0, 1
-; RV32B-NEXT:    srl a1, a1, t0
-; RV32B-NEXT:    or a7, a7, a1
-; RV32B-NEXT:    addi t1, a5, -32
-; RV32B-NEXT:    sll t0, a0, t1
-; RV32B-NEXT:    slti a1, t1, 0
-; RV32B-NEXT:    cmov t0, a1, a7, t0
-; RV32B-NEXT:    not a7, a4
-; RV32B-NEXT:    srli t4, a3, 1
-; RV32B-NEXT:    srl t2, t4, a7
-; RV32B-NEXT:    addi a1, zero, 63
-; RV32B-NEXT:    andn t3, a1, a4
-; RV32B-NEXT:    addi a5, t3, -32
-; RV32B-NEXT:    srai a1, a5, 31
-; RV32B-NEXT:    and a1, a1, t2
-; RV32B-NEXT:    or a1, t0, a1
-; RV32B-NEXT:    fsri a2, a2, a3, 1
-; RV32B-NEXT:    srl a7, a2, a7
-; RV32B-NEXT:    sub a3, a6, t3
-; RV32B-NEXT:    slli a2, t4, 1
-; RV32B-NEXT:    sll a2, a2, a3
-; RV32B-NEXT:    or a2, a7, a2
-; RV32B-NEXT:    srl a3, t4, a5
-; RV32B-NEXT:    slti a5, a5, 0
-; RV32B-NEXT:    cmov a2, a5, a2, a3
-; RV32B-NEXT:    sll a0, a0, a4
-; RV32B-NEXT:    srai a3, t1, 31
-; RV32B-NEXT:    and a0, a3, a0
-; RV32B-NEXT:    or a0, a0, a2
-; RV32B-NEXT:    ret
-;
-; RV32ZBT-LABEL: fshl_i64:
-; RV32ZBT:       # %bb.0:
-; RV32ZBT-NEXT:    sll a7, a1, a4
-; RV32ZBT-NEXT:    andi a5, a4, 63
-; RV32ZBT-NEXT:    addi a6, zero, 31
-; RV32ZBT-NEXT:    sub t0, a6, a5
-; RV32ZBT-NEXT:    srli a1, a0, 1
-; RV32ZBT-NEXT:    srl a1, a1, t0
-; RV32ZBT-NEXT:    or a7, a7, a1
-; RV32ZBT-NEXT:    addi t1, a5, -32
-; RV32ZBT-NEXT:    sll t0, a0, t1
-; RV32ZBT-NEXT:    slti a1, t1, 0
-; RV32ZBT-NEXT:    cmov t0, a1, a7, t0
-; RV32ZBT-NEXT:    not a5, a4
-; RV32ZBT-NEXT:    srli a7, a3, 1
-; RV32ZBT-NEXT:    srl t4, a7, a5
-; RV32ZBT-NEXT:    andi t2, a5, 63
-; RV32ZBT-NEXT:    addi t3, t2, -32
-; RV32ZBT-NEXT:    srai a1, t3, 31
-; RV32ZBT-NEXT:    and a1, a1, t4
-; RV32ZBT-NEXT:    or a1, t0, a1
-; RV32ZBT-NEXT:    fsri a2, a2, a3, 1
-; RV32ZBT-NEXT:    srl a2, a2, a5
-; RV32ZBT-NEXT:    sub a3, a6, t2
-; RV32ZBT-NEXT:    slli a5, a7, 1
-; RV32ZBT-NEXT:    sll a3, a5, a3
-; RV32ZBT-NEXT:    or a2, a2, a3
-; RV32ZBT-NEXT:    srl a3, a7, t3
-; RV32ZBT-NEXT:    slti a5, t3, 0
-; RV32ZBT-NEXT:    cmov a2, a5, a2, a3
-; RV32ZBT-NEXT:    sll a0, a0, a4
-; RV32ZBT-NEXT:    srai a3, t1, 31
-; RV32ZBT-NEXT:    and a0, a3, a0
-; RV32ZBT-NEXT:    or a0, a0, a2
-; RV32ZBT-NEXT:    ret
+; RV32IB-LABEL: fshl_i64:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    sll a7, a1, a4
+; RV32IB-NEXT:    andi a5, a4, 63
+; RV32IB-NEXT:    addi a6, zero, 31
+; RV32IB-NEXT:    sub t0, a6, a5
+; RV32IB-NEXT:    srli a1, a0, 1
+; RV32IB-NEXT:    srl a1, a1, t0
+; RV32IB-NEXT:    or t0, a7, a1
+; RV32IB-NEXT:    addi a7, a5, -32
+; RV32IB-NEXT:    sll a5, a0, a7
+; RV32IB-NEXT:    slti a1, a7, 0
+; RV32IB-NEXT:    cmov t1, a1, t0, a5
+; RV32IB-NEXT:    not t0, a4
+; RV32IB-NEXT:    srli a5, a3, 1
+; RV32IB-NEXT:    srl t2, a5, t0
+; RV32IB-NEXT:    addi a1, zero, 63
+; RV32IB-NEXT:    andn t3, a1, a4
+; RV32IB-NEXT:    addi t4, t3, -32
+; RV32IB-NEXT:    srai a1, t4, 31
+; RV32IB-NEXT:    and a1, a1, t2
+; RV32IB-NEXT:    or a1, t1, a1
+; RV32IB-NEXT:    fsri a2, a2, a3, 1
+; RV32IB-NEXT:    srl t0, a2, t0
+; RV32IB-NEXT:    sub a3, a6, t3
+; RV32IB-NEXT:    slli a2, a5, 1
+; RV32IB-NEXT:    sll a2, a2, a3
+; RV32IB-NEXT:    or a2, t0, a2
+; RV32IB-NEXT:    srl a3, a5, t4
+; RV32IB-NEXT:    slti a5, t4, 0
+; RV32IB-NEXT:    cmov a2, a5, a2, a3
+; RV32IB-NEXT:    sll a0, a0, a4
+; RV32IB-NEXT:    srai a3, a7, 31
+; RV32IB-NEXT:    and a0, a3, a0
+; RV32IB-NEXT:    or a0, a0, a2
+; RV32IB-NEXT:    ret
+;
+; RV32IBT-LABEL: fshl_i64:
+; RV32IBT:       # %bb.0:
+; RV32IBT-NEXT:    sll a7, a1, a4
+; RV32IBT-NEXT:    andi a5, a4, 63
+; RV32IBT-NEXT:    addi a6, zero, 31
+; RV32IBT-NEXT:    sub t0, a6, a5
+; RV32IBT-NEXT:    srli a1, a0, 1
+; RV32IBT-NEXT:    srl a1, a1, t0
+; RV32IBT-NEXT:    or t0, a7, a1
+; RV32IBT-NEXT:    addi a7, a5, -32
+; RV32IBT-NEXT:    sll a5, a0, a7
+; RV32IBT-NEXT:    slti a1, a7, 0
+; RV32IBT-NEXT:    cmov t1, a1, t0, a5
+; RV32IBT-NEXT:    not t0, a4
+; RV32IBT-NEXT:    srli a5, a3, 1
+; RV32IBT-NEXT:    srl t4, a5, t0
+; RV32IBT-NEXT:    andi t2, t0, 63
+; RV32IBT-NEXT:    addi t3, t2, -32
+; RV32IBT-NEXT:    srai a1, t3, 31
+; RV32IBT-NEXT:    and a1, a1, t4
+; RV32IBT-NEXT:    or a1, t1, a1
+; RV32IBT-NEXT:    fsri a2, a2, a3, 1
+; RV32IBT-NEXT:    srl t0, a2, t0
+; RV32IBT-NEXT:    sub a3, a6, t2
+; RV32IBT-NEXT:    slli a2, a5, 1
+; RV32IBT-NEXT:    sll a2, a2, a3
+; RV32IBT-NEXT:    or a2, t0, a2
+; RV32IBT-NEXT:    srl a3, a5, t3
+; RV32IBT-NEXT:    slti a5, t3, 0
+; RV32IBT-NEXT:    cmov a2, a5, a2, a3
+; RV32IBT-NEXT:    sll a0, a0, a4
+; RV32IBT-NEXT:    srai a3, a7, 31
+; RV32IBT-NEXT:    and a0, a3, a0
+; RV32IBT-NEXT:    or a0, a0, a2
+; RV32IBT-NEXT:    ret
   %1 = tail call i64 @llvm.fshl.i64(i64 %a, i64 %b, i64 %c)
   ret i64 %1
 }
 
 declare i32 @llvm.fshr.i32(i32, i32, i32)
 
 define i32 @fshr_i32(i32 %a, i32 %b, i32 %c) nounwind {
 ; RV32I-LABEL: fshr_i32:

 ; This test is presented here in case future expansions of the experimental-b
 ; extension introduce instructions that can match more efficiently this pattern.
 
 declare i64 @llvm.fshr.i64(i64, i64, i64)
 
 define i64 @fshr_i64(i64 %a, i64 %b, i64 %c) nounwind {
 ; RV32I-LABEL: fshr_i64:
 ; RV32I:       # %bb.0:
-; RV32I-NEXT:    mv t0, a0
+; RV32I-NEXT:    mv t1, a0
 ; RV32I-NEXT:    andi a0, a4, 63
 ; RV32I-NEXT:    addi a6, a0, -32
 ; RV32I-NEXT:    addi a7, zero, 31
 ; RV32I-NEXT:    bltz a6, .LBB15_2
 ; RV32I-NEXT:  # %bb.1:
 ; RV32I-NEXT:    srl a0, a3, a6
 ; RV32I-NEXT:    j .LBB15_3
 ; RV32I-NEXT:  .LBB15_2:
 ; RV32I-NEXT:    srl a2, a2, a4
 ; RV32I-NEXT:    sub a0, a7, a0
 ; RV32I-NEXT:    slli a5, a3, 1
 ; RV32I-NEXT:    sll a0, a5, a0
 ; RV32I-NEXT:    or a0, a2, a0
 ; RV32I-NEXT:  .LBB15_3:
-; RV32I-NEXT:    not t2, a4
-; RV32I-NEXT:    andi a5, t2, 63
-; RV32I-NEXT:    addi a2, a5, -32
-; RV32I-NEXT:    slli t1, t0, 1
-; RV32I-NEXT:    bltz a2, .LBB15_5
+; RV32I-NEXT:    not t0, a4
+; RV32I-NEXT:    andi a2, t0, 63
+; RV32I-NEXT:    addi t2, a2, -32
+; RV32I-NEXT:    slli a5, t1, 1
+; RV32I-NEXT:    bltz t2, .LBB15_5
 ; RV32I-NEXT:  # %bb.4:
-; RV32I-NEXT:    sll a1, t1, a2
+; RV32I-NEXT:    sll a1, a5, t2
 ; RV32I-NEXT:    bltz a6, .LBB15_6
 ; RV32I-NEXT:    j .LBB15_7
 ; RV32I-NEXT:  .LBB15_5:
-; RV32I-NEXT:    sll a2, t1, t2
-; RV32I-NEXT:    or a0, a0, a2
-; RV32I-NEXT:    lui a2, 524288
-; RV32I-NEXT:    addi a2, a2, -1
-; RV32I-NEXT:    and a2, t0, a2
-; RV32I-NEXT:    sub a5, a7, a5
-; RV32I-NEXT:    srl a2, a2, a5
-; RV32I-NEXT:    srli a5, t0, 31
+; RV32I-NEXT:    sll a5, a5, t0
+; RV32I-NEXT:    or a0, a0, a5
+; RV32I-NEXT:    lui a5, 524288
+; RV32I-NEXT:    addi a5, a5, -1
+; RV32I-NEXT:    and a5, t1, a5
+; RV32I-NEXT:    sub a2, a7, a2
+; RV32I-NEXT:    srl a2, a5, a2
+; RV32I-NEXT:    srli a5, t1, 31
 ; RV32I-NEXT:    slli a1, a1, 1
 ; RV32I-NEXT:    or a1, a1, a5
-; RV32I-NEXT:    sll a1, a1, t2
+; RV32I-NEXT:    sll a1, a1, t0
 ; RV32I-NEXT:    or a1, a1, a2
 ; RV32I-NEXT:    bgez a6, .LBB15_7
 ; RV32I-NEXT:  .LBB15_6:
 ; RV32I-NEXT:    srl a2, a3, a4
 ; RV32I-NEXT:    or a1, a1, a2
 ; RV32I-NEXT:  .LBB15_7:
 ; RV32I-NEXT:    ret
 ;
-; RV32B-LABEL: fshr_i64:
-; RV32B:       # %bb.0:
-; RV32B-NEXT:    srl a7, a2, a4
-; RV32B-NEXT:    andi a5, a4, 63
-; RV32B-NEXT:    addi a6, zero, 31
-; RV32B-NEXT:    sub t0, a6, a5
-; RV32B-NEXT:    slli a2, a3, 1
-; RV32B-NEXT:    sll a2, a2, t0
-; RV32B-NEXT:    or a7, a7, a2
-; RV32B-NEXT:    addi t2, a5, -32
-; RV32B-NEXT:    srl t0, a3, t2
-; RV32B-NEXT:    slti a2, t2, 0
-; RV32B-NEXT:    cmov a7, a2, a7, t0
-; RV32B-NEXT:    not t3, a4
-; RV32B-NEXT:    slli t0, a0, 1
-; RV32B-NEXT:    sll t1, t0, t3
-; RV32B-NEXT:    addi a5, zero, 63
-; RV32B-NEXT:    andn t4, a5, a4
-; RV32B-NEXT:    addi a2, t4, -32
-; RV32B-NEXT:    srai a5, a2, 31
-; RV32B-NEXT:    and a5, a5, t1
-; RV32B-NEXT:    or a7, a5, a7
-; RV32B-NEXT:    fsri a1, a0, a1, 31
-; RV32B-NEXT:    sll a1, a1, t3
-; RV32B-NEXT:    sub a5, a6, t4
-; RV32B-NEXT:    bclri a0, a0, 31
-; RV32B-NEXT:    srl a0, a0, a5
-; RV32B-NEXT:    or a0, a1, a0
-; RV32B-NEXT:    sll a1, t0, a2
-; RV32B-NEXT:    slti a2, a2, 0
-; RV32B-NEXT:    cmov a0, a2, a0, a1
-; RV32B-NEXT:    srl a1, a3, a4
-; RV32B-NEXT:    srai a2, t2, 31
-; RV32B-NEXT:    and a1, a2, a1
-; RV32B-NEXT:    or a1, a0, a1
-; RV32B-NEXT:    mv a0, a7
-; RV32B-NEXT:    ret
-;
-; RV32ZBT-LABEL: fshr_i64:
-; RV32ZBT:       # %bb.0:
-; RV32ZBT-NEXT:    srl a7, a2, a4
-; RV32ZBT-NEXT:    andi a5, a4, 63
-; RV32ZBT-NEXT:    addi a6, zero, 31
-; RV32ZBT-NEXT:    sub t0, a6, a5
-; RV32ZBT-NEXT:    slli a2, a3, 1
-; RV32ZBT-NEXT:    sll a2, a2, t0
-; RV32ZBT-NEXT:    or a7, a7, a2
-; RV32ZBT-NEXT:    addi t2, a5, -32
-; RV32ZBT-NEXT:    srl t0, a3, t2
-; RV32ZBT-NEXT:    slti a2, t2, 0
-; RV32ZBT-NEXT:    cmov a7, a2, a7, t0
-; RV32ZBT-NEXT:    not t4, a4
-; RV32ZBT-NEXT:    slli t0, a0, 1
-; RV32ZBT-NEXT:    sll t1, t0, t4
-; RV32ZBT-NEXT:    andi t3, t4, 63
-; RV32ZBT-NEXT:    addi a5, t3, -32
-; RV32ZBT-NEXT:    srai a2, a5, 31
-; RV32ZBT-NEXT:    and a2, a2, t1
-; RV32ZBT-NEXT:    or a7, a2, a7
-; RV32ZBT-NEXT:    lui a2, 524288
-; RV32ZBT-NEXT:    addi a2, a2, -1
-; RV32ZBT-NEXT:    and t1, a0, a2
-; RV32ZBT-NEXT:    sub a2, a6, t3
-; RV32ZBT-NEXT:    srl a2, t1, a2
-; RV32ZBT-NEXT:    fsri a0, a0, a1, 31
-; RV32ZBT-NEXT:    sll a0, a0, t4
-; RV32ZBT-NEXT:    or a0, a0, a2
-; RV32ZBT-NEXT:    sll a1, t0, a5
-; RV32ZBT-NEXT:    slti a2, a5, 0
-; RV32ZBT-NEXT:    cmov a0, a2, a0, a1
-; RV32ZBT-NEXT:    srl a1, a3, a4
-; RV32ZBT-NEXT:    srai a2, t2, 31
-; RV32ZBT-NEXT:    and a1, a2, a1
-; RV32ZBT-NEXT:    or a1, a0, a1
-; RV32ZBT-NEXT:    mv a0, a7
-; RV32ZBT-NEXT:    ret
+; RV32IB-LABEL: fshr_i64:
+; RV32IB:       # %bb.0:
+; RV32IB-NEXT:    srl a7, a2, a4
+; RV32IB-NEXT:    andi a5, a4, 63
+; RV32IB-NEXT:    addi a6, zero, 31
+; RV32IB-NEXT:    sub t0, a6, a5
+; RV32IB-NEXT:    slli a2, a3, 1
+; RV32IB-NEXT:    sll a2, a2, t0
+; RV32IB-NEXT:    or t0, a7, a2
+; RV32IB-NEXT:    addi a7, a5, -32
+; RV32IB-NEXT:    srl a5, a3, a7
+; RV32IB-NEXT:    slti a2, a7, 0
+; RV32IB-NEXT:    cmov t1, a2, t0, a5
+; RV32IB-NEXT:    not t0, a4
+; RV32IB-NEXT:    slli t4, a0, 1
+; RV32IB-NEXT:    sll t2, t4, t0
+; RV32IB-NEXT:    addi a2, zero, 63
+; RV32IB-NEXT:    andn a2, a2, a4
+; RV32IB-NEXT:    addi t3, a2, -32
+; RV32IB-NEXT:    srai a5, t3, 31
+; RV32IB-NEXT:    and a5, a5, t2
+; RV32IB-NEXT:    or t1, a5, t1
+; RV32IB-NEXT:    fsri a1, a0, a1, 31
+; RV32IB-NEXT:    sll a1, a1, t0
+; RV32IB-NEXT:    sub a2, a6, a2
+; RV32IB-NEXT:    bclri a0, a0, 31
+; RV32IB-NEXT:    srl a0, a0, a2
+; RV32IB-NEXT:    or a0, a1, a0
+; RV32IB-NEXT:    sll a1, t4, t3
+; RV32IB-NEXT:    slti a2, t3, 0
+; RV32IB-NEXT:    cmov a0, a2, a0, a1
+; RV32IB-NEXT:    srl a1, a3, a4
+; RV32IB-NEXT:    srai a2, a7, 31
+; RV32IB-NEXT:    and a1, a2, a1
+; RV32IB-NEXT:    or a1, a0, a1
+; RV32IB-NEXT:    mv a0, t1
+; RV32IB-NEXT:    ret
+;
+; RV32IBT-LABEL: fshr_i64:
+; RV32IBT:       # %bb.0:
+; RV32IBT-NEXT:    srl a7, a2, a4
+; RV32IBT-NEXT:    andi a5, a4, 63
+; RV32IBT-NEXT:    addi a6, zero, 31
+; RV32IBT-NEXT:    sub t0, a6, a5
+; RV32IBT-NEXT:    slli a2, a3, 1
+; RV32IBT-NEXT:    sll a2, a2, t0
+; RV32IBT-NEXT:    or t0, a7, a2
+; RV32IBT-NEXT:    addi a7, a5, -32
+; RV32IBT-NEXT:    srl a5, a3, a7
+; RV32IBT-NEXT:    slti a2, a7, 0
+; RV32IBT-NEXT:    cmov t1, a2, t0, a5
+; RV32IBT-NEXT:    not t0, a4
+; RV32IBT-NEXT:    slli t4, a0, 1
+; RV32IBT-NEXT:    sll t2, t4, t0
+; RV32IBT-NEXT:    andi a2, t0, 63
+; RV32IBT-NEXT:    addi t3, a2, -32
+; RV32IBT-NEXT:    srai a5, t3, 31
+; RV32IBT-NEXT:    and a5, a5, t2
+; RV32IBT-NEXT:    or t1, a5, t1
+; RV32IBT-NEXT:    lui a5, 524288
+; RV32IBT-NEXT:    addi a5, a5, -1
+; RV32IBT-NEXT:    and a5, a0, a5
+; RV32IBT-NEXT:    sub a2, a6, a2
+; RV32IBT-NEXT:    srl a2, a5, a2
+; RV32IBT-NEXT:    fsri a0, a0, a1, 31
+; RV32IBT-NEXT:    sll a0, a0, t0
+; RV32IBT-NEXT:    or a0, a0, a2
+; RV32IBT-NEXT:    sll a1, t4, t3
+; RV32IBT-NEXT:    slti a2, t3, 0
+; RV32IBT-NEXT:    cmov a0, a2, a0, a1
+; RV32IBT-NEXT:    srl a1, a3, a4
+; RV32IBT-NEXT:    srai a2, a7, 31
+; RV32IBT-NEXT:    and a1, a2, a1
+; RV32IBT-NEXT:    or a1, a0, a1
+; RV32IBT-NEXT:    mv a0, t1
+; RV32IBT-NEXT:    ret
   %1 = tail call i64 @llvm.fshr.i64(i64 %a, i64 %b, i64 %c)
   ret i64 %1
 }
 
 define i32 @fshri_i32(i32 %a, i32 %b) nounwind {
 ; RV32I-LABEL: fshri_i32:
 ; RV32I:       # %bb.0:
 ; RV32I-NEXT:    srli a1, a1, 5

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-bitreverse.ll

 ; LMULMAX2-RV64-NEXT:    vsrl.vi v26, v26, 1
 ; LMULMAX2-RV64-NEXT:    vor.vv v26, v26, v28
 ; LMULMAX2-RV64-NEXT:    vse32.v v26, (a0)
 ; LMULMAX2-RV64-NEXT:    ret
 ;
 ; LMULMAX1-RV32-LABEL: bitreverse_v8i32:
 ; LMULMAX1-RV32:       # %bb.0:
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    addi a7, a0, 16
-; LMULMAX1-RV32-NEXT:    vle32.v v25, (a7)
+; LMULMAX1-RV32-NEXT:    addi a6, a0, 16
+; LMULMAX1-RV32-NEXT:    vle32.v v25, (a6)
 ; LMULMAX1-RV32-NEXT:    vle32.v v26, (a0)
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v27, v25, 8
 ; LMULMAX1-RV32-NEXT:    lui a2, 16
-; LMULMAX1-RV32-NEXT:    addi t0, a2, -256
-; LMULMAX1-RV32-NEXT:    vand.vx v27, v27, t0
+; LMULMAX1-RV32-NEXT:    addi a7, a2, -256
+; LMULMAX1-RV32-NEXT:    vand.vx v27, v27, a7
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v28, v25, 24
 ; LMULMAX1-RV32-NEXT:    vor.vv v27, v27, v28
 ; LMULMAX1-RV32-NEXT:    vsll.vi v28, v25, 8
-; LMULMAX1-RV32-NEXT:    lui a6, 4080
-; LMULMAX1-RV32-NEXT:    vand.vx v28, v28, a6
+; LMULMAX1-RV32-NEXT:    lui t0, 4080
+; LMULMAX1-RV32-NEXT:    vand.vx v28, v28, t0
 ; LMULMAX1-RV32-NEXT:    vsll.vi v25, v25, 24
 ; LMULMAX1-RV32-NEXT:    vor.vv v25, v25, v28
 ; LMULMAX1-RV32-NEXT:    vor.vv v25, v25, v27
 ; LMULMAX1-RV32-NEXT:    lui a4, 61681
 ; LMULMAX1-RV32-NEXT:    addi t1, a4, -241
 ; LMULMAX1-RV32-NEXT:    vand.vx v27, v25, t1
 ; LMULMAX1-RV32-NEXT:    vsll.vi v27, v27, 4
 ; LMULMAX1-RV32-NEXT:    lui a5, 986895
 ; LMULMAX1-RV32-NEXT:    addi a5, a5, 240
 ; LMULMAX1-RV32-NEXT:    vand.vx v25, v25, a5
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v25, v25, 4
 ; LMULMAX1-RV32-NEXT:    vor.vv v25, v25, v27
-; LMULMAX1-RV32-NEXT:    lui a3, 209715
-; LMULMAX1-RV32-NEXT:    addi a3, a3, 819
-; LMULMAX1-RV32-NEXT:    vand.vx v27, v25, a3
+; LMULMAX1-RV32-NEXT:    lui a1, 209715
+; LMULMAX1-RV32-NEXT:    addi a1, a1, 819
+; LMULMAX1-RV32-NEXT:    vand.vx v27, v25, a1
 ; LMULMAX1-RV32-NEXT:    vsll.vi v27, v27, 2
-; LMULMAX1-RV32-NEXT:    lui a1, 838861
-; LMULMAX1-RV32-NEXT:    addi a1, a1, -820
-; LMULMAX1-RV32-NEXT:    vand.vx v25, v25, a1
+; LMULMAX1-RV32-NEXT:    lui a2, 838861
+; LMULMAX1-RV32-NEXT:    addi a2, a2, -820
+; LMULMAX1-RV32-NEXT:    vand.vx v25, v25, a2
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v25, v25, 2
 ; LMULMAX1-RV32-NEXT:    vor.vv v25, v25, v27
-; LMULMAX1-RV32-NEXT:    lui a2, 349525
-; LMULMAX1-RV32-NEXT:    addi a2, a2, 1365
-; LMULMAX1-RV32-NEXT:    vand.vx v27, v25, a2
+; LMULMAX1-RV32-NEXT:    lui a3, 349525
+; LMULMAX1-RV32-NEXT:    addi a3, a3, 1365
+; LMULMAX1-RV32-NEXT:    vand.vx v27, v25, a3
 ; LMULMAX1-RV32-NEXT:    vadd.vv v27, v27, v27
 ; LMULMAX1-RV32-NEXT:    lui a4, 699051
 ; LMULMAX1-RV32-NEXT:    addi a4, a4, -1366
 ; LMULMAX1-RV32-NEXT:    vand.vx v25, v25, a4
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v25, v25, 1
 ; LMULMAX1-RV32-NEXT:    vor.vv v25, v25, v27
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v27, v26, 8
-; LMULMAX1-RV32-NEXT:    vand.vx v27, v27, t0
+; LMULMAX1-RV32-NEXT:    vand.vx v27, v27, a7
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v28, v26, 24
 ; LMULMAX1-RV32-NEXT:    vor.vv v27, v27, v28
 ; LMULMAX1-RV32-NEXT:    vsll.vi v28, v26, 8
-; LMULMAX1-RV32-NEXT:    vand.vx v28, v28, a6
+; LMULMAX1-RV32-NEXT:    vand.vx v28, v28, t0
 ; LMULMAX1-RV32-NEXT:    vsll.vi v26, v26, 24
 ; LMULMAX1-RV32-NEXT:    vor.vv v26, v26, v28
 ; LMULMAX1-RV32-NEXT:    vor.vv v26, v26, v27
 ; LMULMAX1-RV32-NEXT:    vand.vx v27, v26, t1
 ; LMULMAX1-RV32-NEXT:    vsll.vi v27, v27, 4
 ; LMULMAX1-RV32-NEXT:    vand.vx v26, v26, a5
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v26, v26, 4
 ; LMULMAX1-RV32-NEXT:    vor.vv v26, v26, v27
-; LMULMAX1-RV32-NEXT:    vand.vx v27, v26, a3
+; LMULMAX1-RV32-NEXT:    vand.vx v27, v26, a1
 ; LMULMAX1-RV32-NEXT:    vsll.vi v27, v27, 2
-; LMULMAX1-RV32-NEXT:    vand.vx v26, v26, a1
+; LMULMAX1-RV32-NEXT:    vand.vx v26, v26, a2
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v26, v26, 2
 ; LMULMAX1-RV32-NEXT:    vor.vv v26, v26, v27
-; LMULMAX1-RV32-NEXT:    vand.vx v27, v26, a2
+; LMULMAX1-RV32-NEXT:    vand.vx v27, v26, a3
 ; LMULMAX1-RV32-NEXT:    vadd.vv v27, v27, v27
 ; LMULMAX1-RV32-NEXT:    vand.vx v26, v26, a4
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v26, v26, 1
 ; LMULMAX1-RV32-NEXT:    vor.vv v26, v26, v27
 ; LMULMAX1-RV32-NEXT:    vse32.v v26, (a0)
-; LMULMAX1-RV32-NEXT:    vse32.v v25, (a7)
+; LMULMAX1-RV32-NEXT:    vse32.v v25, (a6)
 ; LMULMAX1-RV32-NEXT:    ret
 ;
 ; LMULMAX1-RV64-LABEL: bitreverse_v8i32:
 ; LMULMAX1-RV64:       # %bb.0:
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
-; LMULMAX1-RV64-NEXT:    addi a7, a0, 16
-; LMULMAX1-RV64-NEXT:    vle32.v v25, (a7)
+; LMULMAX1-RV64-NEXT:    addi a6, a0, 16
+; LMULMAX1-RV64-NEXT:    vle32.v v25, (a6)
 ; LMULMAX1-RV64-NEXT:    vle32.v v26, (a0)
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v27, v25, 8
 ; LMULMAX1-RV64-NEXT:    lui a2, 16
-; LMULMAX1-RV64-NEXT:    addiw t0, a2, -256
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v27, t0
+; LMULMAX1-RV64-NEXT:    addiw a2, a2, -256
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v27, a2
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v28, v25, 24
 ; LMULMAX1-RV64-NEXT:    vor.vv v27, v27, v28
 ; LMULMAX1-RV64-NEXT:    vsll.vi v28, v25, 8
-; LMULMAX1-RV64-NEXT:    lui a6, 4080
-; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, a6
+; LMULMAX1-RV64-NEXT:    lui a7, 4080
+; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, a7
 ; LMULMAX1-RV64-NEXT:    vsll.vi v25, v25, 24
 ; LMULMAX1-RV64-NEXT:    vor.vv v25, v25, v28
 ; LMULMAX1-RV64-NEXT:    vor.vv v25, v25, v27
 ; LMULMAX1-RV64-NEXT:    lui a4, 61681
-; LMULMAX1-RV64-NEXT:    addiw t1, a4, -241
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v25, t1
+; LMULMAX1-RV64-NEXT:    addiw a4, a4, -241
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v25, a4
 ; LMULMAX1-RV64-NEXT:    vsll.vi v27, v27, 4
 ; LMULMAX1-RV64-NEXT:    lui a5, 241
 ; LMULMAX1-RV64-NEXT:    addiw a5, a5, -241
 ; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
-; LMULMAX1-RV64-NEXT:    addi a5, a5, 240
-; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, a5
+; LMULMAX1-RV64-NEXT:    addi t0, a5, 240
+; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, t0
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v25, v25, 4
 ; LMULMAX1-RV64-NEXT:    vor.vv v25, v25, v27
-; LMULMAX1-RV64-NEXT:    lui a3, 209715
-; LMULMAX1-RV64-NEXT:    addiw a3, a3, 819
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v25, a3
+; LMULMAX1-RV64-NEXT:    lui a1, 209715
+; LMULMAX1-RV64-NEXT:    addiw a1, a1, 819
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v25, a1
 ; LMULMAX1-RV64-NEXT:    vsll.vi v27, v27, 2
-; LMULMAX1-RV64-NEXT:    lui a1, 205
-; LMULMAX1-RV64-NEXT:    addiw a1, a1, -819
-; LMULMAX1-RV64-NEXT:    slli a1, a1, 12
-; LMULMAX1-RV64-NEXT:    addi a1, a1, -820
-; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, a1
+; LMULMAX1-RV64-NEXT:    lui a3, 205
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -819
+; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
+; LMULMAX1-RV64-NEXT:    addi t1, a3, -820
+; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, t1
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v25, v25, 2
 ; LMULMAX1-RV64-NEXT:    vor.vv v25, v25, v27
-; LMULMAX1-RV64-NEXT:    lui a2, 349525
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, 1365
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v25, a2
+; LMULMAX1-RV64-NEXT:    lui a5, 349525
+; LMULMAX1-RV64-NEXT:    addiw a5, a5, 1365
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v25, a5
 ; LMULMAX1-RV64-NEXT:    vadd.vv v27, v27, v27
-; LMULMAX1-RV64-NEXT:    lui a4, 171
-; LMULMAX1-RV64-NEXT:    addiw a4, a4, -1365
-; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
-; LMULMAX1-RV64-NEXT:    addi a4, a4, -1366
-; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, a4
+; LMULMAX1-RV64-NEXT:    lui a3, 171
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -1365
+; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
+; LMULMAX1-RV64-NEXT:    addi a3, a3, -1366
+; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, a3
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v25, v25, 1
 ; LMULMAX1-RV64-NEXT:    vor.vv v25, v25, v27
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v27, v26, 8
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v27, t0
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v27, a2
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v28, v26, 24
 ; LMULMAX1-RV64-NEXT:    vor.vv v27, v27, v28
 ; LMULMAX1-RV64-NEXT:    vsll.vi v28, v26, 8
-; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, a6
+; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, a7
 ; LMULMAX1-RV64-NEXT:    vsll.vi v26, v26, 24
 ; LMULMAX1-RV64-NEXT:    vor.vv v26, v26, v28
 ; LMULMAX1-RV64-NEXT:    vor.vv v26, v26, v27
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v26, t1
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v26, a4
 ; LMULMAX1-RV64-NEXT:    vsll.vi v27, v27, 4
-; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, a5
+; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, t0
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v26, v26, 4
 ; LMULMAX1-RV64-NEXT:    vor.vv v26, v26, v27
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v26, a3
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v26, a1
 ; LMULMAX1-RV64-NEXT:    vsll.vi v27, v27, 2
-; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, a1
+; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, t1
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v26, v26, 2
 ; LMULMAX1-RV64-NEXT:    vor.vv v26, v26, v27
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v26, a2
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v26, a5
 ; LMULMAX1-RV64-NEXT:    vadd.vv v27, v27, v27
-; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, a4
+; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, a3
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v26, v26, 1
 ; LMULMAX1-RV64-NEXT:    vor.vv v26, v26, v27
 ; LMULMAX1-RV64-NEXT:    vse32.v v26, (a0)
-; LMULMAX1-RV64-NEXT:    vse32.v v25, (a7)
+; LMULMAX1-RV64-NEXT:    vse32.v v25, (a6)
 ; LMULMAX1-RV64-NEXT:    ret
   %a = load <8 x i32>, <8 x i32>* %x
   %b = load <8 x i32>, <8 x i32>* %y
   %c = call <8 x i32> @llvm.bitreverse.v8i32(<8 x i32> %a)
   store <8 x i32> %c, <8 x i32>* %x
   ret void
 }
 declare <8 x i32> @llvm.bitreverse.v8i32(<8 x i32>)

 ; LMULMAX2-RV64-NEXT:    vsrl.vi v26, v26, 1
 ; LMULMAX2-RV64-NEXT:    vor.vv v26, v26, v28
 ; LMULMAX2-RV64-NEXT:    vse64.v v26, (a0)
 ; LMULMAX2-RV64-NEXT:    ret
 ;
 ; LMULMAX1-RV32-LABEL: bitreverse_v4i64:
 ; LMULMAX1-RV32:       # %bb.0:
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    addi a1, a0, 16
-; LMULMAX1-RV32-NEXT:    vle64.v v30, (a1)
+; LMULMAX1-RV32-NEXT:    addi a6, a0, 16
+; LMULMAX1-RV32-NEXT:    vle64.v v30, (a6)
 ; LMULMAX1-RV32-NEXT:    vle64.v v25, (a0)
 ; LMULMAX1-RV32-NEXT:    addi a2, zero, 56
 ; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v30, a2
 ; LMULMAX1-RV32-NEXT:    addi a3, zero, 40
 ; LMULMAX1-RV32-NEXT:    vsrl.vx v27, v30, a3
 ; LMULMAX1-RV32-NEXT:    lui a4, 16
 ; LMULMAX1-RV32-NEXT:    addi a4, a4, -256
 ; LMULMAX1-RV32-NEXT:    vand.vx v27, v27, a4
 ; LMULMAX1-RV32-NEXT:    vor.vv v27, v27, v26
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v26, v30, 24
-; LMULMAX1-RV32-NEXT:    lui a6, 4080
-; LMULMAX1-RV32-NEXT:    vand.vx v28, v26, a6
-; LMULMAX1-RV32-NEXT:    addi a5, zero, 5
+; LMULMAX1-RV32-NEXT:    lui a5, 4080
+; LMULMAX1-RV32-NEXT:    vand.vx v28, v26, a5
+; LMULMAX1-RV32-NEXT:    addi a1, zero, 5
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 1, e8, mf8, ta, mu
-; LMULMAX1-RV32-NEXT:    vmv.s.x v0, a5
+; LMULMAX1-RV32-NEXT:    vmv.s.x v0, a1
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vmv.v.i v26, 0
-; LMULMAX1-RV32-NEXT:    lui a5, 1044480
-; LMULMAX1-RV32-NEXT:    vmerge.vxm v26, v26, a5, v0
+; LMULMAX1-RV32-NEXT:    lui a1, 1044480
+; LMULMAX1-RV32-NEXT:    vmerge.vxm v26, v26, a1, v0
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v29, v30, 8
 ; LMULMAX1-RV32-NEXT:    vand.vv v29, v29, v26
 ; LMULMAX1-RV32-NEXT:    vor.vv v28, v29, v28
 ; LMULMAX1-RV32-NEXT:    vor.vv v31, v28, v27
-; LMULMAX1-RV32-NEXT:    addi a5, zero, 255
+; LMULMAX1-RV32-NEXT:    addi a1, zero, 255
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    vmv.v.x v27, a5
+; LMULMAX1-RV32-NEXT:    vmv.v.x v27, a1
 ; LMULMAX1-RV32-NEXT:    vmerge.vim v27, v27, 0, v0
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vsll.vi v28, v30, 8
 ; LMULMAX1-RV32-NEXT:    vand.vv v29, v28, v27
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vmv.v.x v28, a4
 ; LMULMAX1-RV32-NEXT:    vmerge.vim v28, v28, 0, v0
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vsll.vi v8, v30, 24
 ; LMULMAX1-RV32-NEXT:    vand.vv v8, v8, v28
 ; LMULMAX1-RV32-NEXT:    vor.vv v8, v8, v29
 ; LMULMAX1-RV32-NEXT:    vsll.vx v9, v30, a3
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    vmv.v.x v29, a6
+; LMULMAX1-RV32-NEXT:    vmv.v.x v29, a5
 ; LMULMAX1-RV32-NEXT:    vmerge.vim v29, v29, 0, v0
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vand.vv v9, v9, v29
 ; LMULMAX1-RV32-NEXT:    vsll.vx v30, v30, a2
 ; LMULMAX1-RV32-NEXT:    vor.vv v30, v30, v9
 ; LMULMAX1-RV32-NEXT:    vor.vv v30, v30, v8
 ; LMULMAX1-RV32-NEXT:    vor.vv v31, v30, v31
-; LMULMAX1-RV32-NEXT:    lui a5, 61681
-; LMULMAX1-RV32-NEXT:    addi a5, a5, -241
+; LMULMAX1-RV32-NEXT:    lui a1, 61681
+; LMULMAX1-RV32-NEXT:    addi a1, a1, -241
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    vmv.v.x v30, a5
+; LMULMAX1-RV32-NEXT:    vmv.v.x v30, a1
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vand.vv v8, v31, v30
 ; LMULMAX1-RV32-NEXT:    vsll.vi v8, v8, 4
-; LMULMAX1-RV32-NEXT:    lui a5, 986895
-; LMULMAX1-RV32-NEXT:    addi a5, a5, 240
+; LMULMAX1-RV32-NEXT:    lui a1, 986895
+; LMULMAX1-RV32-NEXT:    addi a1, a1, 240
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    vmv.v.x v9, a5
+; LMULMAX1-RV32-NEXT:    vmv.v.x v9, a1
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vand.vv v31, v31, v9
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v31, v31, 4
 ; LMULMAX1-RV32-NEXT:    vor.vv v31, v31, v8
-; LMULMAX1-RV32-NEXT:    lui a5, 209715
-; LMULMAX1-RV32-NEXT:    addi a5, a5, 819
+; LMULMAX1-RV32-NEXT:    lui a1, 209715
+; LMULMAX1-RV32-NEXT:    addi a1, a1, 819
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    vmv.v.x v8, a5
+; LMULMAX1-RV32-NEXT:    vmv.v.x v8, a1
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vand.vv v10, v31, v8
 ; LMULMAX1-RV32-NEXT:    vsll.vi v10, v10, 2
-; LMULMAX1-RV32-NEXT:    lui a5, 838861
-; LMULMAX1-RV32-NEXT:    addi a5, a5, -820
+; LMULMAX1-RV32-NEXT:    lui a1, 838861
+; LMULMAX1-RV32-NEXT:    addi a1, a1, -820
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    vmv.v.x v11, a5
+; LMULMAX1-RV32-NEXT:    vmv.v.x v11, a1
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vand.vv v31, v31, v11
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v31, v31, 2
 ; LMULMAX1-RV32-NEXT:    vor.vv v31, v31, v10
-; LMULMAX1-RV32-NEXT:    lui a5, 349525
-; LMULMAX1-RV32-NEXT:    addi a5, a5, 1365
+; LMULMAX1-RV32-NEXT:    lui a1, 349525
+; LMULMAX1-RV32-NEXT:    addi a1, a1, 1365
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    vmv.v.x v10, a5
+; LMULMAX1-RV32-NEXT:    vmv.v.x v10, a1
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vand.vv v12, v31, v10
 ; LMULMAX1-RV32-NEXT:    vadd.vv v12, v12, v12
-; LMULMAX1-RV32-NEXT:    lui a5, 699051
-; LMULMAX1-RV32-NEXT:    addi a5, a5, -1366
+; LMULMAX1-RV32-NEXT:    lui a1, 699051
+; LMULMAX1-RV32-NEXT:    addi a1, a1, -1366
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    vmv.v.x v13, a5
+; LMULMAX1-RV32-NEXT:    vmv.v.x v13, a1
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vand.vv v31, v31, v13
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v31, v31, 1
 ; LMULMAX1-RV32-NEXT:    vor.vv v31, v31, v12
 ; LMULMAX1-RV32-NEXT:    vsrl.vx v12, v25, a2
 ; LMULMAX1-RV32-NEXT:    vsrl.vx v14, v25, a3
 ; LMULMAX1-RV32-NEXT:    vand.vx v14, v14, a4
 ; LMULMAX1-RV32-NEXT:    vor.vv v12, v14, v12
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v14, v25, 24
-; LMULMAX1-RV32-NEXT:    vand.vx v14, v14, a6
+; LMULMAX1-RV32-NEXT:    vand.vx v14, v14, a5
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v15, v25, 8
 ; LMULMAX1-RV32-NEXT:    vand.vv v26, v15, v26
 ; LMULMAX1-RV32-NEXT:    vor.vv v26, v26, v14
 ; LMULMAX1-RV32-NEXT:    vor.vv v26, v26, v12
 ; LMULMAX1-RV32-NEXT:    vsll.vi v12, v25, 8
 ; LMULMAX1-RV32-NEXT:    vand.vv v27, v12, v27
 ; LMULMAX1-RV32-NEXT:    vsll.vi v12, v25, 24
 ; LMULMAX1-RV32-NEXT:    vand.vv v28, v12, v28

 ; LMULMAX1-RV32-NEXT:    vsrl.vi v25, v25, 2
 ; LMULMAX1-RV32-NEXT:    vor.vv v25, v25, v26
 ; LMULMAX1-RV32-NEXT:    vand.vv v26, v25, v10
 ; LMULMAX1-RV32-NEXT:    vadd.vv v26, v26, v26
 ; LMULMAX1-RV32-NEXT:    vand.vv v25, v25, v13
 ; LMULMAX1-RV32-NEXT:    vsrl.vi v25, v25, 1
 ; LMULMAX1-RV32-NEXT:    vor.vv v25, v25, v26
 ; LMULMAX1-RV32-NEXT:    vse64.v v25, (a0)
-; LMULMAX1-RV32-NEXT:    vse64.v v31, (a1)
+; LMULMAX1-RV32-NEXT:    vse64.v v31, (a6)
 ; LMULMAX1-RV32-NEXT:    ret
 ;
 ; LMULMAX1-RV64-LABEL: bitreverse_v4i64:
 ; LMULMAX1-RV64:       # %bb.0:
 ; LMULMAX1-RV64-NEXT:    addi sp, sp, -16
 ; LMULMAX1-RV64-NEXT:    .cfi_def_cfa_offset 16
 ; LMULMAX1-RV64-NEXT:    sd s0, 8(sp) # 8-byte Folded Spill
 ; LMULMAX1-RV64-NEXT:    .cfi_offset s0, -8
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
-; LMULMAX1-RV64-NEXT:    addi t1, a0, 16
-; LMULMAX1-RV64-NEXT:    vle64.v v26, (t1)
+; LMULMAX1-RV64-NEXT:    addi a6, a0, 16
+; LMULMAX1-RV64-NEXT:    vle64.v v26, (a6)
 ; LMULMAX1-RV64-NEXT:    vle64.v v25, (a0)
-; LMULMAX1-RV64-NEXT:    addi a7, zero, 56
-; LMULMAX1-RV64-NEXT:    vsrl.vx v27, v26, a7
-; LMULMAX1-RV64-NEXT:    addi t0, zero, 40
-; LMULMAX1-RV64-NEXT:    vsrl.vx v28, v26, t0
+; LMULMAX1-RV64-NEXT:    addi t0, zero, 56
+; LMULMAX1-RV64-NEXT:    vsrl.vx v27, v26, t0
+; LMULMAX1-RV64-NEXT:    addi t1, zero, 40
+; LMULMAX1-RV64-NEXT:    vsrl.vx v28, v26, t1
 ; LMULMAX1-RV64-NEXT:    lui a1, 16
-; LMULMAX1-RV64-NEXT:    addiw t2, a1, -256
-; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, t2
+; LMULMAX1-RV64-NEXT:    addiw t4, a1, -256
+; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, t4
 ; LMULMAX1-RV64-NEXT:    vor.vv v27, v28, v27
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v28, v26, 24
-; LMULMAX1-RV64-NEXT:    lui a6, 4080
-; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, a6
+; LMULMAX1-RV64-NEXT:    lui a7, 4080
+; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, a7
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v29, v26, 8
-; LMULMAX1-RV64-NEXT:    addi a1, zero, 255
-; LMULMAX1-RV64-NEXT:    slli t3, a1, 24
-; LMULMAX1-RV64-NEXT:    vand.vx v29, v29, t3
+; LMULMAX1-RV64-NEXT:    addi a3, zero, 255
+; LMULMAX1-RV64-NEXT:    slli a1, a3, 24
+; LMULMAX1-RV64-NEXT:    vand.vx v29, v29, a1
 ; LMULMAX1-RV64-NEXT:    vor.vv v28, v29, v28
 ; LMULMAX1-RV64-NEXT:    vor.vv v27, v28, v27
 ; LMULMAX1-RV64-NEXT:    vsll.vi v28, v26, 8
-; LMULMAX1-RV64-NEXT:    slli t4, a1, 32
-; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, t4
+; LMULMAX1-RV64-NEXT:    slli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, a5
 ; LMULMAX1-RV64-NEXT:    vsll.vi v29, v26, 24
-; LMULMAX1-RV64-NEXT:    slli t5, a1, 40
-; LMULMAX1-RV64-NEXT:    vand.vx v29, v29, t5
+; LMULMAX1-RV64-NEXT:    slli a2, a3, 40
+; LMULMAX1-RV64-NEXT:    vand.vx v29, v29, a2
 ; LMULMAX1-RV64-NEXT:    vor.vv v28, v29, v28
-; LMULMAX1-RV64-NEXT:    vsll.vx v29, v26, a7
-; LMULMAX1-RV64-NEXT:    vsll.vx v26, v26, t0
-; LMULMAX1-RV64-NEXT:    slli t6, a1, 48
-; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, t6
+; LMULMAX1-RV64-NEXT:    vsll.vx v29, v26, t0
+; LMULMAX1-RV64-NEXT:    vsll.vx v26, v26, t1
+; LMULMAX1-RV64-NEXT:    slli a3, a3, 48
+; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, a3
 ; LMULMAX1-RV64-NEXT:    vor.vv v26, v29, v26
 ; LMULMAX1-RV64-NEXT:    vor.vv v26, v26, v28
 ; LMULMAX1-RV64-NEXT:    vor.vv v26, v26, v27
 ; LMULMAX1-RV64-NEXT:    lui a4, 3855
 ; LMULMAX1-RV64-NEXT:    addiw a4, a4, 241
 ; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX1-RV64-NEXT:    addi a4, a4, -241
 ; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX1-RV64-NEXT:    addi a4, a4, 241
 ; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
-; LMULMAX1-RV64-NEXT:    addi a4, a4, -241
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v26, a4
+; LMULMAX1-RV64-NEXT:    addi t2, a4, -241
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v26, t2
 ; LMULMAX1-RV64-NEXT:    vsll.vi v27, v27, 4
-; LMULMAX1-RV64-NEXT:    lui a5, 1044721
-; LMULMAX1-RV64-NEXT:    addiw a5, a5, -241
-; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
-; LMULMAX1-RV64-NEXT:    addi a5, a5, 241
-; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
-; LMULMAX1-RV64-NEXT:    addi a5, a5, -241
-; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
-; LMULMAX1-RV64-NEXT:    addi a5, a5, 240
-; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, a5
+; LMULMAX1-RV64-NEXT:    lui a4, 1044721
+; LMULMAX1-RV64-NEXT:    addiw a4, a4, -241
+; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
+; LMULMAX1-RV64-NEXT:    addi a4, a4, 241
+; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
+; LMULMAX1-RV64-NEXT:    addi a4, a4, -241
+; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
+; LMULMAX1-RV64-NEXT:    addi t3, a4, 240
+; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, t3
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v26, v26, 4
 ; LMULMAX1-RV64-NEXT:    vor.vv v26, v26, v27
-; LMULMAX1-RV64-NEXT:    lui a2, 13107
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, 819
-; LMULMAX1-RV64-NEXT:    slli a2, a2, 12
-; LMULMAX1-RV64-NEXT:    addi a2, a2, 819
-; LMULMAX1-RV64-NEXT:    slli a2, a2, 12
-; LMULMAX1-RV64-NEXT:    addi a2, a2, 819
-; LMULMAX1-RV64-NEXT:    slli a2, a2, 12
-; LMULMAX1-RV64-NEXT:    addi a2, a2, 819
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v26, a2
+; LMULMAX1-RV64-NEXT:    lui a4, 13107
+; LMULMAX1-RV64-NEXT:    addiw a4, a4, 819
+; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
+; LMULMAX1-RV64-NEXT:    addi a4, a4, 819
+; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
+; LMULMAX1-RV64-NEXT:    addi a4, a4, 819
+; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
+; LMULMAX1-RV64-NEXT:    addi t5, a4, 819
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v26, t5
 ; LMULMAX1-RV64-NEXT:    vsll.vi v27, v27, 2
-; LMULMAX1-RV64-NEXT:    lui a3, 1035469
-; LMULMAX1-RV64-NEXT:    addiw a3, a3, -819
-; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
-; LMULMAX1-RV64-NEXT:    addi a3, a3, -819
-; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
-; LMULMAX1-RV64-NEXT:    addi a3, a3, -819
-; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
-; LMULMAX1-RV64-NEXT:    addi a3, a3, -820
-; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, a3
+; LMULMAX1-RV64-NEXT:    lui a4, 1035469
+; LMULMAX1-RV64-NEXT:    addiw a4, a4, -819
+; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
+; LMULMAX1-RV64-NEXT:    addi a4, a4, -819
+; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
+; LMULMAX1-RV64-NEXT:    addi a4, a4, -819
+; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
+; LMULMAX1-RV64-NEXT:    addi t6, a4, -820
+; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, t6
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v26, v26, 2
 ; LMULMAX1-RV64-NEXT:    vor.vv v26, v26, v27
-; LMULMAX1-RV64-NEXT:    lui a1, 21845
-; LMULMAX1-RV64-NEXT:    addiw a1, a1, 1365
-; LMULMAX1-RV64-NEXT:    slli a1, a1, 12
-; LMULMAX1-RV64-NEXT:    addi a1, a1, 1365
-; LMULMAX1-RV64-NEXT:    slli a1, a1, 12
-; LMULMAX1-RV64-NEXT:    addi a1, a1, 1365
-; LMULMAX1-RV64-NEXT:    slli a1, a1, 12
-; LMULMAX1-RV64-NEXT:    addi a1, a1, 1365
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v26, a1
+; LMULMAX1-RV64-NEXT:    lui a4, 21845
+; LMULMAX1-RV64-NEXT:    addiw a4, a4, 1365
+; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
+; LMULMAX1-RV64-NEXT:    addi a4, a4, 1365
+; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
+; LMULMAX1-RV64-NEXT:    addi a4, a4, 1365
+; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
+; LMULMAX1-RV64-NEXT:    addi a4, a4, 1365
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v26, a4
 ; LMULMAX1-RV64-NEXT:    vadd.vv v27, v27, v27
 ; LMULMAX1-RV64-NEXT:    lui s0, 1026731
 ; LMULMAX1-RV64-NEXT:    addiw s0, s0, -1365
 ; LMULMAX1-RV64-NEXT:    slli s0, s0, 12
 ; LMULMAX1-RV64-NEXT:    addi s0, s0, -1365
 ; LMULMAX1-RV64-NEXT:    slli s0, s0, 12
 ; LMULMAX1-RV64-NEXT:    addi s0, s0, -1365
 ; LMULMAX1-RV64-NEXT:    slli s0, s0, 12
 ; LMULMAX1-RV64-NEXT:    addi s0, s0, -1366
 ; LMULMAX1-RV64-NEXT:    vand.vx v26, v26, s0
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v26, v26, 1
 ; LMULMAX1-RV64-NEXT:    vor.vv v26, v26, v27
-; LMULMAX1-RV64-NEXT:    vsrl.vx v27, v25, a7
-; LMULMAX1-RV64-NEXT:    vsrl.vx v28, v25, t0
-; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, t2
+; LMULMAX1-RV64-NEXT:    vsrl.vx v27, v25, t0
+; LMULMAX1-RV64-NEXT:    vsrl.vx v28, v25, t1
+; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, t4
 ; LMULMAX1-RV64-NEXT:    vor.vv v27, v28, v27
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v28, v25, 24
-; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, a6
+; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, a7
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v29, v25, 8
-; LMULMAX1-RV64-NEXT:    vand.vx v29, v29, t3
+; LMULMAX1-RV64-NEXT:    vand.vx v29, v29, a1
 ; LMULMAX1-RV64-NEXT:    vor.vv v28, v29, v28
 ; LMULMAX1-RV64-NEXT:    vor.vv v27, v28, v27
 ; LMULMAX1-RV64-NEXT:    vsll.vi v28, v25, 8
-; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, t4
+; LMULMAX1-RV64-NEXT:    vand.vx v28, v28, a5
 ; LMULMAX1-RV64-NEXT:    vsll.vi v29, v25, 24
-; LMULMAX1-RV64-NEXT:    vand.vx v29, v29, t5
+; LMULMAX1-RV64-NEXT:    vand.vx v29, v29, a2
 ; LMULMAX1-RV64-NEXT:    vor.vv v28, v29, v28
-; LMULMAX1-RV64-NEXT:    vsll.vx v29, v25, a7
-; LMULMAX1-RV64-NEXT:    vsll.vx v25, v25, t0
-; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, t6
+; LMULMAX1-RV64-NEXT:    vsll.vx v29, v25, t0
+; LMULMAX1-RV64-NEXT:    vsll.vx v25, v25, t1
+; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, a3
 ; LMULMAX1-RV64-NEXT:    vor.vv v25, v29, v25
 ; LMULMAX1-RV64-NEXT:    vor.vv v25, v25, v28
 ; LMULMAX1-RV64-NEXT:    vor.vv v25, v25, v27
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v25, a4
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v25, t2
 ; LMULMAX1-RV64-NEXT:    vsll.vi v27, v27, 4
-; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, a5
+; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, t3
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v25, v25, 4
 ; LMULMAX1-RV64-NEXT:    vor.vv v25, v25, v27
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v25, a2
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v25, t5
 ; LMULMAX1-RV64-NEXT:    vsll.vi v27, v27, 2
-; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, a3
+; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, t6
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v25, v25, 2
 ; LMULMAX1-RV64-NEXT:    vor.vv v25, v25, v27
-; LMULMAX1-RV64-NEXT:    vand.vx v27, v25, a1
+; LMULMAX1-RV64-NEXT:    vand.vx v27, v25, a4
 ; LMULMAX1-RV64-NEXT:    vadd.vv v27, v27, v27
 ; LMULMAX1-RV64-NEXT:    vand.vx v25, v25, s0
 ; LMULMAX1-RV64-NEXT:    vsrl.vi v25, v25, 1
 ; LMULMAX1-RV64-NEXT:    vor.vv v25, v25, v27
 ; LMULMAX1-RV64-NEXT:    vse64.v v25, (a0)
-; LMULMAX1-RV64-NEXT:    vse64.v v26, (t1)
+; LMULMAX1-RV64-NEXT:    vse64.v v26, (a6)
 ; LMULMAX1-RV64-NEXT:    ld s0, 8(sp) # 8-byte Folded Reload
 ; LMULMAX1-RV64-NEXT:    addi sp, sp, 16
 ; LMULMAX1-RV64-NEXT:    ret
   %a = load <4 x i64>, <4 x i64>* %x
   %b = load <4 x i64>, <4 x i64>* %y
   %c = call <4 x i64> @llvm.bitreverse.v4i64(<4 x i64> %a)
   store <4 x i64> %c, <4 x i64>* %x
   ret void
 }
 declare <4 x i64> @llvm.bitreverse.v4i64(<4 x i64>)

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-bswap.ll

 ; LMULMAX2-RV32:       # %bb.0:
 ; LMULMAX2-RV32-NEXT:    addi sp, sp, -32
 ; LMULMAX2-RV32-NEXT:    .cfi_def_cfa_offset 32
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX2-RV32-NEXT:    vle64.v v25, (a0)
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
 ; LMULMAX2-RV32-NEXT:    lui a3, 16
-; LMULMAX2-RV32-NEXT:    addi a3, a3, -256
-; LMULMAX2-RV32-NEXT:    and a2, a2, a3
+; LMULMAX2-RV32-NEXT:    addi a6, a3, -256
+; LMULMAX2-RV32-NEXT:    and a2, a2, a6
 ; LMULMAX2-RV32-NEXT:    srli a4, a1, 24
 ; LMULMAX2-RV32-NEXT:    or a2, a2, a4
 ; LMULMAX2-RV32-NEXT:    slli a4, a1, 8
-; LMULMAX2-RV32-NEXT:    lui a6, 4080
-; LMULMAX2-RV32-NEXT:    and a4, a4, a6
+; LMULMAX2-RV32-NEXT:    lui a5, 4080
+; LMULMAX2-RV32-NEXT:    and a4, a4, a5
 ; LMULMAX2-RV32-NEXT:    slli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a4
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    sw a1, 20(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v26, v25, 1
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
 ; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV32-NEXT:    and a2, a2, a3
+; LMULMAX2-RV32-NEXT:    and a2, a2, a6
 ; LMULMAX2-RV32-NEXT:    srli a4, a1, 24
 ; LMULMAX2-RV32-NEXT:    or a2, a2, a4
 ; LMULMAX2-RV32-NEXT:    slli a4, a1, 8
-; LMULMAX2-RV32-NEXT:    and a4, a4, a6
+; LMULMAX2-RV32-NEXT:    and a4, a4, a5
 ; LMULMAX2-RV32-NEXT:    slli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a4
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    sw a1, 28(sp)
 ; LMULMAX2-RV32-NEXT:    addi a1, zero, 32
 ; LMULMAX2-RV32-NEXT:    vsrl.vx v25, v25, a1
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a2, v25
 ; LMULMAX2-RV32-NEXT:    srli a4, a2, 8
-; LMULMAX2-RV32-NEXT:    and a4, a4, a3
-; LMULMAX2-RV32-NEXT:    srli a5, a2, 24
-; LMULMAX2-RV32-NEXT:    or a4, a4, a5
-; LMULMAX2-RV32-NEXT:    slli a5, a2, 8
-; LMULMAX2-RV32-NEXT:    and a5, a5, a6
+; LMULMAX2-RV32-NEXT:    and a4, a4, a6
+; LMULMAX2-RV32-NEXT:    srli a3, a2, 24
+; LMULMAX2-RV32-NEXT:    or a3, a4, a3
+; LMULMAX2-RV32-NEXT:    slli a4, a2, 8
+; LMULMAX2-RV32-NEXT:    and a4, a4, a5
 ; LMULMAX2-RV32-NEXT:    slli a2, a2, 24
-; LMULMAX2-RV32-NEXT:    or a2, a2, a5
 ; LMULMAX2-RV32-NEXT:    or a2, a2, a4
+; LMULMAX2-RV32-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV32-NEXT:    sw a2, 16(sp)
 ; LMULMAX2-RV32-NEXT:    vsrl.vx v25, v26, a1
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV32-NEXT:    and a2, a2, a3
+; LMULMAX2-RV32-NEXT:    and a2, a2, a6
 ; LMULMAX2-RV32-NEXT:    srli a3, a1, 24
 ; LMULMAX2-RV32-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV32-NEXT:    slli a3, a1, 8
-; LMULMAX2-RV32-NEXT:    and a3, a3, a6
+; LMULMAX2-RV32-NEXT:    and a3, a3, a5
 ; LMULMAX2-RV32-NEXT:    slli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a3
 ; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    sw a1, 24(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX2-RV32-NEXT:    addi a1, sp, 16
 ; LMULMAX2-RV32-NEXT:    vle32.v v25, (a1)
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu

 ; LMULMAX1-RV32:       # %bb.0:
 ; LMULMAX1-RV32-NEXT:    addi sp, sp, -32
 ; LMULMAX1-RV32-NEXT:    .cfi_def_cfa_offset 32
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vle64.v v25, (a0)
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
 ; LMULMAX1-RV32-NEXT:    lui a3, 16
-; LMULMAX1-RV32-NEXT:    addi a3, a3, -256
-; LMULMAX1-RV32-NEXT:    and a2, a2, a3
+; LMULMAX1-RV32-NEXT:    addi a6, a3, -256
+; LMULMAX1-RV32-NEXT:    and a2, a2, a6
 ; LMULMAX1-RV32-NEXT:    srli a4, a1, 24
 ; LMULMAX1-RV32-NEXT:    or a2, a2, a4
 ; LMULMAX1-RV32-NEXT:    slli a4, a1, 8
-; LMULMAX1-RV32-NEXT:    lui a6, 4080
-; LMULMAX1-RV32-NEXT:    and a4, a4, a6
+; LMULMAX1-RV32-NEXT:    lui a5, 4080
+; LMULMAX1-RV32-NEXT:    and a4, a4, a5
 ; LMULMAX1-RV32-NEXT:    slli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a4
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    sw a1, 20(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vslidedown.vi v26, v25, 1
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV32-NEXT:    and a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a2, a2, a6
 ; LMULMAX1-RV32-NEXT:    srli a4, a1, 24
 ; LMULMAX1-RV32-NEXT:    or a2, a2, a4
 ; LMULMAX1-RV32-NEXT:    slli a4, a1, 8
-; LMULMAX1-RV32-NEXT:    and a4, a4, a6
+; LMULMAX1-RV32-NEXT:    and a4, a4, a5
 ; LMULMAX1-RV32-NEXT:    slli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a4
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    sw a1, 28(sp)
 ; LMULMAX1-RV32-NEXT:    addi a1, zero, 32
 ; LMULMAX1-RV32-NEXT:    vsrl.vx v25, v25, a1
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v25
 ; LMULMAX1-RV32-NEXT:    srli a4, a2, 8
-; LMULMAX1-RV32-NEXT:    and a4, a4, a3
-; LMULMAX1-RV32-NEXT:    srli a5, a2, 24
-; LMULMAX1-RV32-NEXT:    or a4, a4, a5
-; LMULMAX1-RV32-NEXT:    slli a5, a2, 8
-; LMULMAX1-RV32-NEXT:    and a5, a5, a6
+; LMULMAX1-RV32-NEXT:    and a4, a4, a6
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 24
+; LMULMAX1-RV32-NEXT:    or a3, a4, a3
+; LMULMAX1-RV32-NEXT:    slli a4, a2, 8
+; LMULMAX1-RV32-NEXT:    and a4, a4, a5
 ; LMULMAX1-RV32-NEXT:    slli a2, a2, 24
-; LMULMAX1-RV32-NEXT:    or a2, a2, a5
 ; LMULMAX1-RV32-NEXT:    or a2, a2, a4
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV32-NEXT:    sw a2, 16(sp)
 ; LMULMAX1-RV32-NEXT:    vsrl.vx v25, v26, a1
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV32-NEXT:    and a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a2, a2, a6
 ; LMULMAX1-RV32-NEXT:    srli a3, a1, 24
 ; LMULMAX1-RV32-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV32-NEXT:    slli a3, a1, 8
-; LMULMAX1-RV32-NEXT:    and a3, a3, a6
+; LMULMAX1-RV32-NEXT:    and a3, a3, a5
 ; LMULMAX1-RV32-NEXT:    slli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a3
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    sw a1, 24(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    addi a1, sp, 16
 ; LMULMAX1-RV32-NEXT:    vle32.v v25, (a1)
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu

 ; LMULMAX2-RV32-NEXT:    addi s0, sp, 96
 ; LMULMAX2-RV32-NEXT:    .cfi_def_cfa s0, 0
 ; LMULMAX2-RV32-NEXT:    andi sp, sp, -32
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 4, e64, m2, ta, mu
 ; LMULMAX2-RV32-NEXT:    vle64.v v26, (a0)
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a3, v26
 ; LMULMAX2-RV32-NEXT:    srli a2, a3, 8
 ; LMULMAX2-RV32-NEXT:    lui a1, 16
-; LMULMAX2-RV32-NEXT:    addi a1, a1, -256
-; LMULMAX2-RV32-NEXT:    and a2, a2, a1
+; LMULMAX2-RV32-NEXT:    addi a6, a1, -256
+; LMULMAX2-RV32-NEXT:    and a2, a2, a6
 ; LMULMAX2-RV32-NEXT:    srli a4, a3, 24
 ; LMULMAX2-RV32-NEXT:    or a4, a2, a4
 ; LMULMAX2-RV32-NEXT:    slli a5, a3, 8
-; LMULMAX2-RV32-NEXT:    lui a6, 4080
-; LMULMAX2-RV32-NEXT:    and a5, a5, a6
+; LMULMAX2-RV32-NEXT:    lui a2, 4080
+; LMULMAX2-RV32-NEXT:    and a5, a5, a2
 ; LMULMAX2-RV32-NEXT:    slli a3, a3, 24
 ; LMULMAX2-RV32-NEXT:    or a3, a3, a5
 ; LMULMAX2-RV32-NEXT:    or a3, a3, a4
 ; LMULMAX2-RV32-NEXT:    sw a3, 36(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 1, e64, m2, ta, mu
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v28, v26, 3
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a3, v28
 ; LMULMAX2-RV32-NEXT:    srli a4, a3, 8
-; LMULMAX2-RV32-NEXT:    and a4, a4, a1
+; LMULMAX2-RV32-NEXT:    and a4, a4, a6
 ; LMULMAX2-RV32-NEXT:    srli a5, a3, 24
 ; LMULMAX2-RV32-NEXT:    or a4, a4, a5
 ; LMULMAX2-RV32-NEXT:    slli a5, a3, 8
-; LMULMAX2-RV32-NEXT:    and a5, a5, a6
+; LMULMAX2-RV32-NEXT:    and a5, a5, a2
 ; LMULMAX2-RV32-NEXT:    slli a3, a3, 24
 ; LMULMAX2-RV32-NEXT:    or a3, a3, a5
 ; LMULMAX2-RV32-NEXT:    or a3, a3, a4
 ; LMULMAX2-RV32-NEXT:    sw a3, 60(sp)
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v30, v26, 2
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a3, v30
 ; LMULMAX2-RV32-NEXT:    srli a4, a3, 8
-; LMULMAX2-RV32-NEXT:    and a4, a4, a1
+; LMULMAX2-RV32-NEXT:    and a4, a4, a6
 ; LMULMAX2-RV32-NEXT:    srli a5, a3, 24
 ; LMULMAX2-RV32-NEXT:    or a4, a4, a5
 ; LMULMAX2-RV32-NEXT:    slli a5, a3, 8
-; LMULMAX2-RV32-NEXT:    and a5, a5, a6
+; LMULMAX2-RV32-NEXT:    and a5, a5, a2
 ; LMULMAX2-RV32-NEXT:    slli a3, a3, 24
 ; LMULMAX2-RV32-NEXT:    or a3, a3, a5
 ; LMULMAX2-RV32-NEXT:    or a3, a3, a4
 ; LMULMAX2-RV32-NEXT:    sw a3, 52(sp)
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v8, v26, 1
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a3, v8
 ; LMULMAX2-RV32-NEXT:    srli a4, a3, 8
-; LMULMAX2-RV32-NEXT:    and a4, a4, a1
+; LMULMAX2-RV32-NEXT:    and a4, a4, a6
 ; LMULMAX2-RV32-NEXT:    srli a5, a3, 24
 ; LMULMAX2-RV32-NEXT:    or a4, a4, a5
 ; LMULMAX2-RV32-NEXT:    slli a5, a3, 8
-; LMULMAX2-RV32-NEXT:    and a5, a5, a6
+; LMULMAX2-RV32-NEXT:    and a5, a5, a2
 ; LMULMAX2-RV32-NEXT:    slli a3, a3, 24
 ; LMULMAX2-RV32-NEXT:    or a3, a3, a5
 ; LMULMAX2-RV32-NEXT:    or a3, a3, a4
 ; LMULMAX2-RV32-NEXT:    sw a3, 44(sp)
 ; LMULMAX2-RV32-NEXT:    addi a3, zero, 32
 ; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v26, a3
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a4, v26
 ; LMULMAX2-RV32-NEXT:    srli a5, a4, 8
-; LMULMAX2-RV32-NEXT:    and a5, a5, a1
-; LMULMAX2-RV32-NEXT:    srli a2, a4, 24
-; LMULMAX2-RV32-NEXT:    or a2, a5, a2
-; LMULMAX2-RV32-NEXT:    slli a5, a4, 8
 ; LMULMAX2-RV32-NEXT:    and a5, a5, a6
+; LMULMAX2-RV32-NEXT:    srli a1, a4, 24
+; LMULMAX2-RV32-NEXT:    or a1, a5, a1
+; LMULMAX2-RV32-NEXT:    slli a5, a4, 8
+; LMULMAX2-RV32-NEXT:    and a5, a5, a2
 ; LMULMAX2-RV32-NEXT:    slli a4, a4, 24
 ; LMULMAX2-RV32-NEXT:    or a4, a4, a5
-; LMULMAX2-RV32-NEXT:    or a2, a4, a2
-; LMULMAX2-RV32-NEXT:    sw a2, 32(sp)
+; LMULMAX2-RV32-NEXT:    or a1, a4, a1
+; LMULMAX2-RV32-NEXT:    sw a1, 32(sp)
 ; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v28, a3
-; LMULMAX2-RV32-NEXT:    vmv.x.s a2, v26
-; LMULMAX2-RV32-NEXT:    srli a4, a2, 8
-; LMULMAX2-RV32-NEXT:    and a4, a4, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a2, 24
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX2-RV32-NEXT:    srli a4, a1, 8
+; LMULMAX2-RV32-NEXT:    and a4, a4, a6
+; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
 ; LMULMAX2-RV32-NEXT:    or a4, a4, a5
-; LMULMAX2-RV32-NEXT:    slli a5, a2, 8
-; LMULMAX2-RV32-NEXT:    and a5, a5, a6
-; LMULMAX2-RV32-NEXT:    slli a2, a2, 24
-; LMULMAX2-RV32-NEXT:    or a2, a2, a5
-; LMULMAX2-RV32-NEXT:    or a2, a2, a4
-; LMULMAX2-RV32-NEXT:    sw a2, 56(sp)
+; LMULMAX2-RV32-NEXT:    slli a5, a1, 8
+; LMULMAX2-RV32-NEXT:    and a5, a5, a2
+; LMULMAX2-RV32-NEXT:    slli a1, a1, 24
+; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    or a1, a1, a4
+; LMULMAX2-RV32-NEXT:    sw a1, 56(sp)
 ; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v30, a3
-; LMULMAX2-RV32-NEXT:    vmv.x.s a2, v26
-; LMULMAX2-RV32-NEXT:    srli a4, a2, 8
-; LMULMAX2-RV32-NEXT:    and a4, a4, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a2, 24
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX2-RV32-NEXT:    srli a4, a1, 8
+; LMULMAX2-RV32-NEXT:    and a4, a4, a6
+; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
 ; LMULMAX2-RV32-NEXT:    or a4, a4, a5
-; LMULMAX2-RV32-NEXT:    slli a5, a2, 8
-; LMULMAX2-RV32-NEXT:    and a5, a5, a6
-; LMULMAX2-RV32-NEXT:    slli a2, a2, 24
-; LMULMAX2-RV32-NEXT:    or a2, a2, a5
-; LMULMAX2-RV32-NEXT:    or a2, a2, a4
-; LMULMAX2-RV32-NEXT:    sw a2, 48(sp)
+; LMULMAX2-RV32-NEXT:    slli a5, a1, 8
+; LMULMAX2-RV32-NEXT:    and a5, a5, a2
+; LMULMAX2-RV32-NEXT:    slli a1, a1, 24
+; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    or a1, a1, a4
+; LMULMAX2-RV32-NEXT:    sw a1, 48(sp)
 ; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v8, a3
-; LMULMAX2-RV32-NEXT:    vmv.x.s a2, v26
-; LMULMAX2-RV32-NEXT:    srli a3, a2, 8
-; LMULMAX2-RV32-NEXT:    and a1, a3, a1
-; LMULMAX2-RV32-NEXT:    srli a3, a2, 24
-; LMULMAX2-RV32-NEXT:    or a1, a1, a3
-; LMULMAX2-RV32-NEXT:    slli a3, a2, 8
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX2-RV32-NEXT:    srli a3, a1, 8
 ; LMULMAX2-RV32-NEXT:    and a3, a3, a6
-; LMULMAX2-RV32-NEXT:    slli a2, a2, 24
-; LMULMAX2-RV32-NEXT:    or a2, a2, a3
-; LMULMAX2-RV32-NEXT:    or a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a4, a1, 24
+; LMULMAX2-RV32-NEXT:    or a3, a3, a4
+; LMULMAX2-RV32-NEXT:    slli a4, a1, 8
+; LMULMAX2-RV32-NEXT:    and a2, a4, a2
+; LMULMAX2-RV32-NEXT:    slli a1, a1, 24
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a3
 ; LMULMAX2-RV32-NEXT:    sw a1, 40(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 8, e32, m2, ta, mu
 ; LMULMAX2-RV32-NEXT:    addi a1, sp, 32
 ; LMULMAX2-RV32-NEXT:    vle32.v v26, (a1)
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 4, e64, m2, ta, mu
 ; LMULMAX2-RV32-NEXT:    vse64.v v26, (a0)
 ; LMULMAX2-RV32-NEXT:    addi sp, s0, -96
 ; LMULMAX2-RV32-NEXT:    lw s0, 88(sp) # 4-byte Folded Reload

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-ctlz.ll

 ; LMULMAX2-RV64-LABEL: ctlz_v8i16:
 ; LMULMAX2-RV64:       # %bb.0:
 ; LMULMAX2-RV64-NEXT:    addi sp, sp, -32
 ; LMULMAX2-RV64-NEXT:    .cfi_def_cfa_offset 32
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 8, e16, m1, ta, mu
 ; LMULMAX2-RV64-NEXT:    vle16.v v25, (a0)
 ; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v25
 ; LMULMAX2-RV64-NEXT:    lui a1, 16
-; LMULMAX2-RV64-NEXT:    addiw a6, a1, -1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a6
+; LMULMAX2-RV64-NEXT:    addiw a1, a1, -1
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
 ; LMULMAX2-RV64-NEXT:    srli a3, a2, 1
 ; LMULMAX2-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV64-NEXT:    srli a3, a2, 2
 ; LMULMAX2-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV64-NEXT:    srli a3, a2, 4
 ; LMULMAX2-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV64-NEXT:    srli a3, a2, 8
 ; LMULMAX2-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV64-NEXT:    srli a3, a2, 16
 ; LMULMAX2-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV64-NEXT:    srli a3, a2, 32
 ; LMULMAX2-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV64-NEXT:    not a3, a2
 ; LMULMAX2-RV64-NEXT:    srli a4, a3, 1
 ; LMULMAX2-RV64-NEXT:    lui a2, 21845
 ; LMULMAX2-RV64-NEXT:    addiw a2, a2, 1365
 ; LMULMAX2-RV64-NEXT:    slli a2, a2, 12
 ; LMULMAX2-RV64-NEXT:    addi a2, a2, 1365
 ; LMULMAX2-RV64-NEXT:    slli a2, a2, 12
 ; LMULMAX2-RV64-NEXT:    addi a2, a2, 1365
 ; LMULMAX2-RV64-NEXT:    slli a2, a2, 12
-; LMULMAX2-RV64-NEXT:    addi a7, a2, 1365
-; LMULMAX2-RV64-NEXT:    and a4, a4, a7
+; LMULMAX2-RV64-NEXT:    addi a6, a2, 1365
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
 ; LMULMAX2-RV64-NEXT:    sub a4, a3, a4
 ; LMULMAX2-RV64-NEXT:    lui a3, 13107
 ; LMULMAX2-RV64-NEXT:    addiw a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX2-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX2-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX2-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    and a5, a4, a3
 ; LMULMAX2-RV64-NEXT:    srli a4, a4, 2
 ; LMULMAX2-RV64-NEXT:    and a4, a4, a3
 ; LMULMAX2-RV64-NEXT:    add a4, a5, a4
 ; LMULMAX2-RV64-NEXT:    srli a5, a4, 4
 ; LMULMAX2-RV64-NEXT:    add a5, a4, a5
 ; LMULMAX2-RV64-NEXT:    lui a4, 3855
 ; LMULMAX2-RV64-NEXT:    addiw a4, a4, 241
 ; LMULMAX2-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX2-RV64-NEXT:    addi a4, a4, -241
 ; LMULMAX2-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX2-RV64-NEXT:    addi a4, a4, 241
 ; LMULMAX2-RV64-NEXT:    slli a4, a4, 12
-; LMULMAX2-RV64-NEXT:    addi a4, a4, -241
-; LMULMAX2-RV64-NEXT:    and a1, a5, a4
+; LMULMAX2-RV64-NEXT:    addi a7, a4, -241
+; LMULMAX2-RV64-NEXT:    and a2, a5, a7
 ; LMULMAX2-RV64-NEXT:    lui a5, 4112
 ; LMULMAX2-RV64-NEXT:    addiw a5, a5, 257
 ; LMULMAX2-RV64-NEXT:    slli a5, a5, 16
 ; LMULMAX2-RV64-NEXT:    addi a5, a5, 257
 ; LMULMAX2-RV64-NEXT:    slli a5, a5, 16
 ; LMULMAX2-RV64-NEXT:    addi a5, a5, 257
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 16(sp)
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 16(sp)
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 1, e16, m1, ta, mu
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v26, v25, 7
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 30(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 30(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v26, v25, 6
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 28(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 28(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v26, v25, 5
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 26(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 26(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v26, v25, 4
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 24(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 24(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v26, v25, 3
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 22(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 22(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v26, v25, 2
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 20(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 20(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v25, v25, 1
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v25
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v25
+; LMULMAX2-RV64-NEXT:    and a1, a2, a1
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
 ; LMULMAX2-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
 ; LMULMAX2-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
 ; LMULMAX2-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
 ; LMULMAX2-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
 ; LMULMAX2-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
 ; LMULMAX2-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    not a1, a1
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    and a2, a2, a6
 ; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV64-NEXT:    and a1, a1, a3
 ; LMULMAX2-RV64-NEXT:    add a1, a2, a1
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
 ; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
+; LMULMAX2-RV64-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
 ; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
 ; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
 ; LMULMAX2-RV64-NEXT:    sh a1, 18(sp)
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 8, e16, m1, ta, mu
 ; LMULMAX2-RV64-NEXT:    addi a1, sp, 16
 ; LMULMAX2-RV64-NEXT:    vle16.v v25, (a1)
 ; LMULMAX2-RV64-NEXT:    vse16.v v25, (a0)

 ; LMULMAX1-RV64-LABEL: ctlz_v8i16:
 ; LMULMAX1-RV64:       # %bb.0:
 ; LMULMAX1-RV64-NEXT:    addi sp, sp, -32
 ; LMULMAX1-RV64-NEXT:    .cfi_def_cfa_offset 32
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 8, e16, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    vle16.v v25, (a0)
 ; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v25
 ; LMULMAX1-RV64-NEXT:    lui a1, 16
-; LMULMAX1-RV64-NEXT:    addiw a6, a1, -1
-; LMULMAX1-RV64-NEXT:    and a2, a2, a6
+; LMULMAX1-RV64-NEXT:    addiw a1, a1, -1
+; LMULMAX1-RV64-NEXT:    and a2, a2, a1
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
 ; LMULMAX1-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
 ; LMULMAX1-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
 ; LMULMAX1-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
 ; LMULMAX1-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
 ; LMULMAX1-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
 ; LMULMAX1-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    not a3, a2
 ; LMULMAX1-RV64-NEXT:    srli a4, a3, 1
 ; LMULMAX1-RV64-NEXT:    lui a2, 21845
 ; LMULMAX1-RV64-NEXT:    addiw a2, a2, 1365
 ; LMULMAX1-RV64-NEXT:    slli a2, a2, 12
 ; LMULMAX1-RV64-NEXT:    addi a2, a2, 1365
 ; LMULMAX1-RV64-NEXT:    slli a2, a2, 12
 ; LMULMAX1-RV64-NEXT:    addi a2, a2, 1365
 ; LMULMAX1-RV64-NEXT:    slli a2, a2, 12
-; LMULMAX1-RV64-NEXT:    addi a7, a2, 1365
-; LMULMAX1-RV64-NEXT:    and a4, a4, a7
+; LMULMAX1-RV64-NEXT:    addi a6, a2, 1365
+; LMULMAX1-RV64-NEXT:    and a4, a4, a6
 ; LMULMAX1-RV64-NEXT:    sub a4, a3, a4
 ; LMULMAX1-RV64-NEXT:    lui a3, 13107
 ; LMULMAX1-RV64-NEXT:    addiw a3, a3, 819
 ; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX1-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX1-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX1-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX1-RV64-NEXT:    and a5, a4, a3
 ; LMULMAX1-RV64-NEXT:    srli a4, a4, 2
 ; LMULMAX1-RV64-NEXT:    and a4, a4, a3
 ; LMULMAX1-RV64-NEXT:    add a4, a5, a4
 ; LMULMAX1-RV64-NEXT:    srli a5, a4, 4
 ; LMULMAX1-RV64-NEXT:    add a5, a4, a5
 ; LMULMAX1-RV64-NEXT:    lui a4, 3855
 ; LMULMAX1-RV64-NEXT:    addiw a4, a4, 241
 ; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX1-RV64-NEXT:    addi a4, a4, -241
 ; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX1-RV64-NEXT:    addi a4, a4, 241
 ; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
-; LMULMAX1-RV64-NEXT:    addi a4, a4, -241
-; LMULMAX1-RV64-NEXT:    and a1, a5, a4
+; LMULMAX1-RV64-NEXT:    addi a7, a4, -241
+; LMULMAX1-RV64-NEXT:    and a2, a5, a7
 ; LMULMAX1-RV64-NEXT:    lui a5, 4112
 ; LMULMAX1-RV64-NEXT:    addiw a5, a5, 257
 ; LMULMAX1-RV64-NEXT:    slli a5, a5, 16
 ; LMULMAX1-RV64-NEXT:    addi a5, a5, 257
 ; LMULMAX1-RV64-NEXT:    slli a5, a5, 16
 ; LMULMAX1-RV64-NEXT:    addi a5, a5, 257
-; LMULMAX1-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX1-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX1-RV64-NEXT:    sh a1, 16(sp)
+; LMULMAX1-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX1-RV64-NEXT:    sh a2, 16(sp)
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 1, e16, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 7
-; LMULMAX1-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV64-NEXT:    and a1, a1, a6
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    not a1, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX1-RV64-NEXT:    and a2, a2, a1
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    not a2, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    and a4, a4, a6
+; LMULMAX1-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX1-RV64-NEXT:    and a4, a2, a3
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX1-RV64-NEXT:    and a2, a2, a3
+; LMULMAX1-RV64-NEXT:    add a2, a4, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX1-RV64-NEXT:    add a2, a2, a4
 ; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a2, a1, a3
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a3
-; LMULMAX1-RV64-NEXT:    add a1, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV64-NEXT:    add a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a4
-; LMULMAX1-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX1-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX1-RV64-NEXT:    sh a1, 30(sp)
+; LMULMAX1-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX1-RV64-NEXT:    sh a2, 30(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 6
-; LMULMAX1-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV64-NEXT:    and a1, a1, a6
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    not a1, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX1-RV64-NEXT:    and a2, a2, a1
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    not a2, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    and a4, a4, a6
+; LMULMAX1-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX1-RV64-NEXT:    and a4, a2, a3
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX1-RV64-NEXT:    and a2, a2, a3
+; LMULMAX1-RV64-NEXT:    add a2, a4, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX1-RV64-NEXT:    add a2, a2, a4
 ; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a2, a1, a3
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a3
-; LMULMAX1-RV64-NEXT:    add a1, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV64-NEXT:    add a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a4
-; LMULMAX1-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX1-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX1-RV64-NEXT:    sh a1, 28(sp)
+; LMULMAX1-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX1-RV64-NEXT:    sh a2, 28(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 5
-; LMULMAX1-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV64-NEXT:    and a1, a1, a6
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    not a1, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX1-RV64-NEXT:    and a2, a2, a1
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    not a2, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    and a4, a4, a6
+; LMULMAX1-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX1-RV64-NEXT:    and a4, a2, a3
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX1-RV64-NEXT:    and a2, a2, a3
+; LMULMAX1-RV64-NEXT:    add a2, a4, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX1-RV64-NEXT:    add a2, a2, a4
 ; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a2, a1, a3
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a3
-; LMULMAX1-RV64-NEXT:    add a1, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV64-NEXT:    add a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a4
-; LMULMAX1-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX1-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX1-RV64-NEXT:    sh a1, 26(sp)
+; LMULMAX1-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX1-RV64-NEXT:    sh a2, 26(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 4
-; LMULMAX1-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV64-NEXT:    and a1, a1, a6
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    not a1, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX1-RV64-NEXT:    and a2, a2, a1
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    not a2, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    and a4, a4, a6
+; LMULMAX1-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX1-RV64-NEXT:    and a4, a2, a3
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX1-RV64-NEXT:    and a2, a2, a3
+; LMULMAX1-RV64-NEXT:    add a2, a4, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX1-RV64-NEXT:    add a2, a2, a4
 ; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a2, a1, a3
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a3
-; LMULMAX1-RV64-NEXT:    add a1, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV64-NEXT:    add a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a4
-; LMULMAX1-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX1-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX1-RV64-NEXT:    sh a1, 24(sp)
+; LMULMAX1-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX1-RV64-NEXT:    sh a2, 24(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 3
-; LMULMAX1-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV64-NEXT:    and a1, a1, a6
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    not a1, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX1-RV64-NEXT:    and a2, a2, a1
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    not a2, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    and a4, a4, a6
+; LMULMAX1-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX1-RV64-NEXT:    and a4, a2, a3
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX1-RV64-NEXT:    and a2, a2, a3
+; LMULMAX1-RV64-NEXT:    add a2, a4, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX1-RV64-NEXT:    add a2, a2, a4
 ; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a2, a1, a3
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a3
-; LMULMAX1-RV64-NEXT:    add a1, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV64-NEXT:    add a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a4
-; LMULMAX1-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX1-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX1-RV64-NEXT:    sh a1, 22(sp)
+; LMULMAX1-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX1-RV64-NEXT:    sh a2, 22(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 2
-; LMULMAX1-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV64-NEXT:    and a1, a1, a6
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX1-RV64-NEXT:    or a1, a1, a2
-; LMULMAX1-RV64-NEXT:    not a1, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX1-RV64-NEXT:    and a2, a2, a1
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX1-RV64-NEXT:    or a2, a2, a4
+; LMULMAX1-RV64-NEXT:    not a2, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    and a4, a4, a6
+; LMULMAX1-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX1-RV64-NEXT:    and a4, a2, a3
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX1-RV64-NEXT:    and a2, a2, a3
+; LMULMAX1-RV64-NEXT:    add a2, a4, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX1-RV64-NEXT:    add a2, a2, a4
 ; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a2, a1, a3
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a3
-; LMULMAX1-RV64-NEXT:    add a1, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV64-NEXT:    add a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a4
-; LMULMAX1-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX1-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX1-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX1-RV64-NEXT:    sh a1, 20(sp)
+; LMULMAX1-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX1-RV64-NEXT:    sh a2, 20(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v25, v25, 1
-; LMULMAX1-RV64-NEXT:    vmv.x.s a1, v25
-; LMULMAX1-RV64-NEXT:    and a1, a1, a6
+; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v25
+; LMULMAX1-RV64-NEXT:    and a1, a2, a1
 ; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV64-NEXT:    srli a2, a1, 2
 ; LMULMAX1-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
 ; LMULMAX1-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV64-NEXT:    srli a2, a1, 8
 ; LMULMAX1-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV64-NEXT:    srli a2, a1, 16
 ; LMULMAX1-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV64-NEXT:    srli a2, a1, 32
 ; LMULMAX1-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV64-NEXT:    not a1, a1
 ; LMULMAX1-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
+; LMULMAX1-RV64-NEXT:    and a2, a2, a6
 ; LMULMAX1-RV64-NEXT:    sub a1, a1, a2
 ; LMULMAX1-RV64-NEXT:    and a2, a1, a3
 ; LMULMAX1-RV64-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV64-NEXT:    and a1, a1, a3
 ; LMULMAX1-RV64-NEXT:    add a1, a2, a1
 ; LMULMAX1-RV64-NEXT:    srli a2, a1, 4
 ; LMULMAX1-RV64-NEXT:    add a1, a1, a2
-; LMULMAX1-RV64-NEXT:    and a1, a1, a4
+; LMULMAX1-RV64-NEXT:    and a1, a1, a7
 ; LMULMAX1-RV64-NEXT:    mul a1, a1, a5
 ; LMULMAX1-RV64-NEXT:    srli a1, a1, 56
 ; LMULMAX1-RV64-NEXT:    addiw a1, a1, -48
 ; LMULMAX1-RV64-NEXT:    sh a1, 18(sp)
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 8, e16, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    addi a1, sp, 16
 ; LMULMAX1-RV64-NEXT:    vle16.v v25, (a1)
 ; LMULMAX1-RV64-NEXT:    vse16.v v25, (a0)

 ; LMULMAX2-RV32-LABEL: ctlz_v2i64:
 ; LMULMAX2-RV32:       # %bb.0:
 ; LMULMAX2-RV32-NEXT:    addi sp, sp, -32
 ; LMULMAX2-RV32-NEXT:    .cfi_def_cfa_offset 32
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX2-RV32-NEXT:    vle64.v v25, (a0)
 ; LMULMAX2-RV32-NEXT:    sw zero, 28(sp)
 ; LMULMAX2-RV32-NEXT:    sw zero, 20(sp)
-; LMULMAX2-RV32-NEXT:    addi a6, zero, 32
+; LMULMAX2-RV32-NEXT:    addi a5, zero, 32
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
-; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v25, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v26
-; LMULMAX2-RV32-NEXT:    lui a1, 349525
-; LMULMAX2-RV32-NEXT:    addi a4, a1, 1365
-; LMULMAX2-RV32-NEXT:    lui a1, 209715
-; LMULMAX2-RV32-NEXT:    addi a3, a1, 819
-; LMULMAX2-RV32-NEXT:    lui a1, 61681
-; LMULMAX2-RV32-NEXT:    addi a7, a1, -241
-; LMULMAX2-RV32-NEXT:    lui a1, 4112
-; LMULMAX2-RV32-NEXT:    addi a2, a1, 257
-; LMULMAX2-RV32-NEXT:    bnez a5, .LBB3_2
+; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v25, a5
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX2-RV32-NEXT:    lui a2, 349525
+; LMULMAX2-RV32-NEXT:    addi a4, a2, 1365
+; LMULMAX2-RV32-NEXT:    lui a2, 209715
+; LMULMAX2-RV32-NEXT:    addi a3, a2, 819
+; LMULMAX2-RV32-NEXT:    lui a2, 61681
+; LMULMAX2-RV32-NEXT:    addi a6, a2, -241
+; LMULMAX2-RV32-NEXT:    lui a2, 4112
+; LMULMAX2-RV32-NEXT:    addi a7, a2, 257
+; LMULMAX2-RV32-NEXT:    bnez a1, .LBB3_2
 ; LMULMAX2-RV32-NEXT:  # %bb.1:
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v25
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    and a5, a5, a4
-; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a5, a1, a3
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    and a2, a2, a4
+; LMULMAX2-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
-; LMULMAX2-RV32-NEXT:    add a1, a5, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    add a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a1, a1, a7
-; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
+; LMULMAX2-RV32-NEXT:    add a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    add a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a1, a1, a6
+; LMULMAX2-RV32-NEXT:    mul a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
-; LMULMAX2-RV32-NEXT:    addi a5, a1, 32
+; LMULMAX2-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX2-RV32-NEXT:    j .LBB3_3
 ; LMULMAX2-RV32-NEXT:  .LBB3_2:
-; LMULMAX2-RV32-NEXT:    srli a1, a5, 1
-; LMULMAX2-RV32-NEXT:    or a1, a5, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    and a5, a5, a4
-; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a5, a1, a3
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    and a2, a2, a4
+; LMULMAX2-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
-; LMULMAX2-RV32-NEXT:    add a1, a5, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    add a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a1, a1, a7
-; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
+; LMULMAX2-RV32-NEXT:    add a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    add a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a1, a1, a6
+; LMULMAX2-RV32-NEXT:    mul a1, a1, a7
+; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB3_3:
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v25, v25, 1
-; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v25, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
-; LMULMAX2-RV32-NEXT:    sw a5, 16(sp)
-; LMULMAX2-RV32-NEXT:    bnez a1, .LBB3_5
+; LMULMAX2-RV32-NEXT:    vsrl.vx v26, v25, a5
+; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v26
+; LMULMAX2-RV32-NEXT:    sw a1, 16(sp)
+; LMULMAX2-RV32-NEXT:    bnez a5, .LBB3_5
 ; LMULMAX2-RV32-NEXT:  # %bb.4:
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v25
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    and a4, a5, a4
-; LMULMAX2-RV32-NEXT:    sub a1, a1, a4
-; LMULMAX2-RV32-NEXT:    and a4, a1, a3
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    and a2, a2, a4
+; LMULMAX2-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
-; LMULMAX2-RV32-NEXT:    add a1, a4, a1
-; LMULMAX2-RV32-NEXT:    srli a3, a1, 4
-; LMULMAX2-RV32-NEXT:    add a1, a1, a3
-; LMULMAX2-RV32-NEXT:    and a1, a1, a7
-; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
+; LMULMAX2-RV32-NEXT:    add a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    add a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a1, a1, a6
+; LMULMAX2-RV32-NEXT:    mul a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX2-RV32-NEXT:    j .LBB3_6
 ; LMULMAX2-RV32-NEXT:  .LBB3_5:
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a1, a5, 1
+; LMULMAX2-RV32-NEXT:    or a1, a5, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    and a4, a5, a4
-; LMULMAX2-RV32-NEXT:    sub a1, a1, a4
-; LMULMAX2-RV32-NEXT:    and a4, a1, a3
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    and a2, a2, a4
+; LMULMAX2-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
-; LMULMAX2-RV32-NEXT:    add a1, a4, a1
-; LMULMAX2-RV32-NEXT:    srli a3, a1, 4
-; LMULMAX2-RV32-NEXT:    add a1, a1, a3
-; LMULMAX2-RV32-NEXT:    and a1, a1, a7
-; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
+; LMULMAX2-RV32-NEXT:    add a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    add a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a1, a1, a6
+; LMULMAX2-RV32-NEXT:    mul a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB3_6:
 ; LMULMAX2-RV32-NEXT:    sw a1, 24(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX2-RV32-NEXT:    addi a1, sp, 16
 ; LMULMAX2-RV32-NEXT:    vle32.v v25, (a1)
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX2-RV32-NEXT:    vse64.v v25, (a0)

 ; LMULMAX1-RV32-LABEL: ctlz_v2i64:
 ; LMULMAX1-RV32:       # %bb.0:
 ; LMULMAX1-RV32-NEXT:    addi sp, sp, -32
 ; LMULMAX1-RV32-NEXT:    .cfi_def_cfa_offset 32
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vle64.v v25, (a0)
 ; LMULMAX1-RV32-NEXT:    sw zero, 28(sp)
 ; LMULMAX1-RV32-NEXT:    sw zero, 20(sp)
-; LMULMAX1-RV32-NEXT:    addi a6, zero, 32
+; LMULMAX1-RV32-NEXT:    addi a5, zero, 32
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a6
-; LMULMAX1-RV32-NEXT:    vmv.x.s a5, v26
-; LMULMAX1-RV32-NEXT:    lui a1, 349525
-; LMULMAX1-RV32-NEXT:    addi a4, a1, 1365
-; LMULMAX1-RV32-NEXT:    lui a1, 209715
-; LMULMAX1-RV32-NEXT:    addi a3, a1, 819
-; LMULMAX1-RV32-NEXT:    lui a1, 61681
-; LMULMAX1-RV32-NEXT:    addi a7, a1, -241
-; LMULMAX1-RV32-NEXT:    lui a1, 4112
-; LMULMAX1-RV32-NEXT:    addi a2, a1, 257
-; LMULMAX1-RV32-NEXT:    bnez a5, .LBB3_2
+; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a5
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX1-RV32-NEXT:    lui a2, 349525
+; LMULMAX1-RV32-NEXT:    addi a4, a2, 1365
+; LMULMAX1-RV32-NEXT:    lui a2, 209715
+; LMULMAX1-RV32-NEXT:    addi a3, a2, 819
+; LMULMAX1-RV32-NEXT:    lui a2, 61681
+; LMULMAX1-RV32-NEXT:    addi a6, a2, -241
+; LMULMAX1-RV32-NEXT:    lui a2, 4112
+; LMULMAX1-RV32-NEXT:    addi a7, a2, 257
+; LMULMAX1-RV32-NEXT:    bnez a1, .LBB3_2
 ; LMULMAX1-RV32-NEXT:  # %bb.1:
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    not a1, a1
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX1-RV32-NEXT:    and a5, a5, a4
-; LMULMAX1-RV32-NEXT:    sub a1, a1, a5
-; LMULMAX1-RV32-NEXT:    and a5, a1, a3
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV32-NEXT:    and a2, a2, a4
+; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX1-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a3
-; LMULMAX1-RV32-NEXT:    add a1, a5, a1
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX1-RV32-NEXT:    add a1, a1, a5
-; LMULMAX1-RV32-NEXT:    and a1, a1, a7
-; LMULMAX1-RV32-NEXT:    mul a1, a1, a2
+; LMULMAX1-RV32-NEXT:    add a1, a2, a1
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX1-RV32-NEXT:    add a1, a1, a2
+; LMULMAX1-RV32-NEXT:    and a1, a1, a6
+; LMULMAX1-RV32-NEXT:    mul a1, a1, a7
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
-; LMULMAX1-RV32-NEXT:    addi a5, a1, 32
+; LMULMAX1-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX1-RV32-NEXT:    j .LBB3_3
 ; LMULMAX1-RV32-NEXT:  .LBB3_2:
-; LMULMAX1-RV32-NEXT:    srli a1, a5, 1
-; LMULMAX1-RV32-NEXT:    or a1, a5, a1
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    not a1, a1
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX1-RV32-NEXT:    and a5, a5, a4
-; LMULMAX1-RV32-NEXT:    sub a1, a1, a5
-; LMULMAX1-RV32-NEXT:    and a5, a1, a3
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV32-NEXT:    and a2, a2, a4
+; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX1-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a3
-; LMULMAX1-RV32-NEXT:    add a1, a5, a1
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX1-RV32-NEXT:    add a1, a1, a5
-; LMULMAX1-RV32-NEXT:    and a1, a1, a7
-; LMULMAX1-RV32-NEXT:    mul a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 24
+; LMULMAX1-RV32-NEXT:    add a1, a2, a1
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX1-RV32-NEXT:    add a1, a1, a2
+; LMULMAX1-RV32-NEXT:    and a1, a1, a6
+; LMULMAX1-RV32-NEXT:    mul a1, a1, a7
+; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB3_3:
 ; LMULMAX1-RV32-NEXT:    vslidedown.vi v25, v25, 1
-; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a6
-; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV32-NEXT:    sw a5, 16(sp)
-; LMULMAX1-RV32-NEXT:    bnez a1, .LBB3_5
+; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a5
+; LMULMAX1-RV32-NEXT:    vmv.x.s a5, v26
+; LMULMAX1-RV32-NEXT:    sw a1, 16(sp)
+; LMULMAX1-RV32-NEXT:    bnez a5, .LBB3_5
 ; LMULMAX1-RV32-NEXT:  # %bb.4:
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    not a1, a1
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX1-RV32-NEXT:    and a4, a5, a4
-; LMULMAX1-RV32-NEXT:    sub a1, a1, a4
-; LMULMAX1-RV32-NEXT:    and a4, a1, a3
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV32-NEXT:    and a2, a2, a4
+; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX1-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a3
-; LMULMAX1-RV32-NEXT:    add a1, a4, a1
-; LMULMAX1-RV32-NEXT:    srli a3, a1, 4
-; LMULMAX1-RV32-NEXT:    add a1, a1, a3
-; LMULMAX1-RV32-NEXT:    and a1, a1, a7
-; LMULMAX1-RV32-NEXT:    mul a1, a1, a2
+; LMULMAX1-RV32-NEXT:    add a1, a2, a1
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX1-RV32-NEXT:    add a1, a1, a2
+; LMULMAX1-RV32-NEXT:    and a1, a1, a6
+; LMULMAX1-RV32-NEXT:    mul a1, a1, a7
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX1-RV32-NEXT:    j .LBB3_6
 ; LMULMAX1-RV32-NEXT:  .LBB3_5:
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX1-RV32-NEXT:    or a1, a1, a5
+; LMULMAX1-RV32-NEXT:    srli a1, a5, 1
+; LMULMAX1-RV32-NEXT:    or a1, a5, a1
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    not a1, a1
-; LMULMAX1-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX1-RV32-NEXT:    and a4, a5, a4
-; LMULMAX1-RV32-NEXT:    sub a1, a1, a4
-; LMULMAX1-RV32-NEXT:    and a4, a1, a3
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV32-NEXT:    and a2, a2, a4
+; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX1-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a3
-; LMULMAX1-RV32-NEXT:    add a1, a4, a1
-; LMULMAX1-RV32-NEXT:    srli a3, a1, 4
-; LMULMAX1-RV32-NEXT:    add a1, a1, a3
-; LMULMAX1-RV32-NEXT:    and a1, a1, a7
-; LMULMAX1-RV32-NEXT:    mul a1, a1, a2
+; LMULMAX1-RV32-NEXT:    add a1, a2, a1
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX1-RV32-NEXT:    add a1, a1, a2
+; LMULMAX1-RV32-NEXT:    and a1, a1, a6
+; LMULMAX1-RV32-NEXT:    mul a1, a1, a7
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB3_6:
 ; LMULMAX1-RV32-NEXT:    sw a1, 24(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    addi a1, sp, 16
 ; LMULMAX1-RV32-NEXT:    vle32.v v25, (a1)
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vse64.v v25, (a0)

 ; LMULMAX2-RV64-NEXT:    .cfi_offset s0, -16
 ; LMULMAX2-RV64-NEXT:    addi s0, sp, 96
 ; LMULMAX2-RV64-NEXT:    .cfi_def_cfa s0, 0
 ; LMULMAX2-RV64-NEXT:    andi sp, sp, -32
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 16, e16, m2, ta, mu
 ; LMULMAX2-RV64-NEXT:    vle16.v v26, (a0)
 ; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v26
 ; LMULMAX2-RV64-NEXT:    lui a1, 16
-; LMULMAX2-RV64-NEXT:    addiw a6, a1, -1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a6
+; LMULMAX2-RV64-NEXT:    addiw a1, a1, -1
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
 ; LMULMAX2-RV64-NEXT:    srli a3, a2, 1
 ; LMULMAX2-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV64-NEXT:    srli a3, a2, 2
 ; LMULMAX2-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV64-NEXT:    srli a3, a2, 4
 ; LMULMAX2-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV64-NEXT:    srli a3, a2, 8
 ; LMULMAX2-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV64-NEXT:    srli a3, a2, 16
 ; LMULMAX2-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV64-NEXT:    srli a3, a2, 32
 ; LMULMAX2-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX2-RV64-NEXT:    not a3, a2
 ; LMULMAX2-RV64-NEXT:    srli a4, a3, 1
 ; LMULMAX2-RV64-NEXT:    lui a2, 21845
 ; LMULMAX2-RV64-NEXT:    addiw a2, a2, 1365
 ; LMULMAX2-RV64-NEXT:    slli a2, a2, 12
 ; LMULMAX2-RV64-NEXT:    addi a2, a2, 1365
 ; LMULMAX2-RV64-NEXT:    slli a2, a2, 12
 ; LMULMAX2-RV64-NEXT:    addi a2, a2, 1365
 ; LMULMAX2-RV64-NEXT:    slli a2, a2, 12
-; LMULMAX2-RV64-NEXT:    addi a7, a2, 1365
-; LMULMAX2-RV64-NEXT:    and a4, a4, a7
+; LMULMAX2-RV64-NEXT:    addi a6, a2, 1365
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
 ; LMULMAX2-RV64-NEXT:    sub a4, a3, a4
 ; LMULMAX2-RV64-NEXT:    lui a3, 13107
 ; LMULMAX2-RV64-NEXT:    addiw a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX2-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX2-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX2-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    and a5, a4, a3
 ; LMULMAX2-RV64-NEXT:    srli a4, a4, 2
 ; LMULMAX2-RV64-NEXT:    and a4, a4, a3
 ; LMULMAX2-RV64-NEXT:    add a4, a5, a4
 ; LMULMAX2-RV64-NEXT:    srli a5, a4, 4
 ; LMULMAX2-RV64-NEXT:    add a5, a4, a5
 ; LMULMAX2-RV64-NEXT:    lui a4, 3855
 ; LMULMAX2-RV64-NEXT:    addiw a4, a4, 241
 ; LMULMAX2-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX2-RV64-NEXT:    addi a4, a4, -241
 ; LMULMAX2-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX2-RV64-NEXT:    addi a4, a4, 241
 ; LMULMAX2-RV64-NEXT:    slli a4, a4, 12
-; LMULMAX2-RV64-NEXT:    addi a4, a4, -241
-; LMULMAX2-RV64-NEXT:    and a1, a5, a4
+; LMULMAX2-RV64-NEXT:    addi a7, a4, -241
+; LMULMAX2-RV64-NEXT:    and a2, a5, a7
 ; LMULMAX2-RV64-NEXT:    lui a5, 4112
 ; LMULMAX2-RV64-NEXT:    addiw a5, a5, 257
 ; LMULMAX2-RV64-NEXT:    slli a5, a5, 16
 ; LMULMAX2-RV64-NEXT:    addi a5, a5, 257
 ; LMULMAX2-RV64-NEXT:    slli a5, a5, 16
 ; LMULMAX2-RV64-NEXT:    addi a5, a5, 257
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 32(sp)
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 32(sp)
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 1, e16, m2, ta, mu
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 15
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 62(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 62(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 14
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 60(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 60(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 13
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 58(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 58(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 12
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 56(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 56(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 11
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 54(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 54(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 10
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 52(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 52(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 9
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 50(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 50(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 8
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 48(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 48(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 7
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 46(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 46(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 6
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 44(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 44(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 5
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 42(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 42(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 4
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 40(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 40(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 3
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 38(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 38(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 2
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
-; LMULMAX2-RV64-NEXT:    or a1, a1, a2
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
-; LMULMAX2-RV64-NEXT:    sh a1, 36(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    and a2, a2, a1
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 2
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 8
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 16
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a4
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, -48
+; LMULMAX2-RV64-NEXT:    sh a2, 36(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v26, v26, 1
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX2-RV64-NEXT:    and a1, a1, a6
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX2-RV64-NEXT:    and a1, a2, a1
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
 ; LMULMAX2-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 2
 ; LMULMAX2-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
 ; LMULMAX2-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 8
 ; LMULMAX2-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 16
 ; LMULMAX2-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 32
 ; LMULMAX2-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    not a1, a1
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    and a2, a2, a6
 ; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV64-NEXT:    and a1, a1, a3
 ; LMULMAX2-RV64-NEXT:    add a1, a2, a1
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
 ; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
+; LMULMAX2-RV64-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
 ; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
 ; LMULMAX2-RV64-NEXT:    addiw a1, a1, -48
 ; LMULMAX2-RV64-NEXT:    sh a1, 34(sp)
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 16, e16, m2, ta, mu
 ; LMULMAX2-RV64-NEXT:    addi a1, sp, 32
 ; LMULMAX2-RV64-NEXT:    vle16.v v26, (a1)
 ; LMULMAX2-RV64-NEXT:    vse16.v v26, (a0)

 ; LMULMAX1-RV64-NEXT:    addi sp, sp, -48
 ; LMULMAX1-RV64-NEXT:    .cfi_def_cfa_offset 48
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 8, e16, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    addi a6, a0, 16
 ; LMULMAX1-RV64-NEXT:    vle16.v v26, (a6)
 ; LMULMAX1-RV64-NEXT:    vle16.v v25, (a0)
 ; LMULMAX1-RV64-NEXT:    vmv.x.s a1, v26
 ; LMULMAX1-RV64-NEXT:    lui a2, 16
-; LMULMAX1-RV64-NEXT:    addiw a7, a2, -1
-; LMULMAX1-RV64-NEXT:    and a1, a1, a7
+; LMULMAX1-RV64-NEXT:    addiw a2, a2, -1
+; LMULMAX1-RV64-NEXT:    and a1, a1, a2
 ; LMULMAX1-RV64-NEXT:    srli a3, a1, 1
 ; LMULMAX1-RV64-NEXT:    or a1, a1, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a1, 2
 ; LMULMAX1-RV64-NEXT:    or a1, a1, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a1, 4
 ; LMULMAX1-RV64-NEXT:    or a1, a1, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a1, 8
 ; LMULMAX1-RV64-NEXT:    or a1, a1, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a1, 16
 ; LMULMAX1-RV64-NEXT:    or a1, a1, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a1, 32
 ; LMULMAX1-RV64-NEXT:    or a1, a1, a3
 ; LMULMAX1-RV64-NEXT:    not a1, a1
 ; LMULMAX1-RV64-NEXT:    srli a4, a1, 1
 ; LMULMAX1-RV64-NEXT:    lui a3, 21845
 ; LMULMAX1-RV64-NEXT:    addiw a3, a3, 1365
 ; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX1-RV64-NEXT:    addi a3, a3, 1365
 ; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX1-RV64-NEXT:    addi a3, a3, 1365
 ; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
-; LMULMAX1-RV64-NEXT:    addi t0, a3, 1365
-; LMULMAX1-RV64-NEXT:    and a4, a4, t0
+; LMULMAX1-RV64-NEXT:    addi a7, a3, 1365
+; LMULMAX1-RV64-NEXT:    and a4, a4, a7
 ; LMULMAX1-RV64-NEXT:    sub a1, a1, a4
 ; LMULMAX1-RV64-NEXT:    lui a4, 13107
 ; LMULMAX1-RV64-NEXT:    addiw a4, a4, 819
 ; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX1-RV64-NEXT:    addi a4, a4, 819
 ; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX1-RV64-NEXT:    addi a4, a4, 819
 ; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX1-RV64-NEXT:    addi a4, a4, 819
 ; LMULMAX1-RV64-NEXT:    and a5, a1, a4
 ; LMULMAX1-RV64-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV64-NEXT:    and a1, a1, a4
 ; LMULMAX1-RV64-NEXT:    add a1, a5, a1
 ; LMULMAX1-RV64-NEXT:    srli a5, a1, 4
 ; LMULMAX1-RV64-NEXT:    add a1, a1, a5
 ; LMULMAX1-RV64-NEXT:    lui a5, 3855
 ; LMULMAX1-RV64-NEXT:    addiw a5, a5, 241
 ; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
 ; LMULMAX1-RV64-NEXT:    addi a5, a5, -241
 ; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
 ; LMULMAX1-RV64-NEXT:    addi a5, a5, 241
 ; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
-; LMULMAX1-RV64-NEXT:    addi a5, a5, -241
-; LMULMAX1-RV64-NEXT:    and a2, a1, a5
+; LMULMAX1-RV64-NEXT:    addi t0, a5, -241
+; LMULMAX1-RV64-NEXT:    and a3, a1, t0
 ; LMULMAX1-RV64-NEXT:    lui a1, 4112
 ; LMULMAX1-RV64-NEXT:    addiw a1, a1, 257
 ; LMULMAX1-RV64-NEXT:    slli a1, a1, 16
 ; LMULMAX1-RV64-NEXT:    addi a1, a1, 257
 ; LMULMAX1-RV64-NEXT:    slli a1, a1, 16
 ; LMULMAX1-RV64-NEXT:    addi a1, a1, 257
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 32(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 32(sp)
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 1, e16, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v27, v26, 7
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v27
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v27
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 46(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 46(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v27, v26, 6
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v27
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v27
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 44(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 44(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v27, v26, 5
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v27
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v27
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 42(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 42(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v27, v26, 4
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v27
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v27
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 40(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 40(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v27, v26, 3
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v27
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v27
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 38(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 38(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v27, v26, 2
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v27
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v27
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 36(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 36(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v26, 1
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v26
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 34(sp)
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v25
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 34(sp)
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v25
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 16(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 16(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 7
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v26
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 30(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 30(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 6
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v26
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 28(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 28(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 5
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v26
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 26(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 26(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 4
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v26
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 24(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 24(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 3
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v26
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 22(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 22(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 2
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a3
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v26
+; LMULMAX1-RV64-NEXT:    and a3, a3, a2
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 2
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 8
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 16
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 32
+; LMULMAX1-RV64-NEXT:    or a3, a3, a5
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    addiw a2, a2, -48
-; LMULMAX1-RV64-NEXT:    sh a2, 20(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    addiw a3, a3, -48
+; LMULMAX1-RV64-NEXT:    sh a3, 20(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v25, v25, 1
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v25
-; LMULMAX1-RV64-NEXT:    and a2, a2, a7
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v25
+; LMULMAX1-RV64-NEXT:    and a2, a3, a2
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
 ; LMULMAX1-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 2
 ; LMULMAX1-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
 ; LMULMAX1-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 8
 ; LMULMAX1-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 16
 ; LMULMAX1-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 32
 ; LMULMAX1-RV64-NEXT:    or a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    not a2, a2
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    and a3, a3, t0
+; LMULMAX1-RV64-NEXT:    and a3, a3, a7
 ; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    and a3, a2, a4
 ; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
 ; LMULMAX1-RV64-NEXT:    and a2, a2, a4
 ; LMULMAX1-RV64-NEXT:    add a2, a3, a2
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
 ; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
+; LMULMAX1-RV64-NEXT:    and a2, a2, t0
 ; LMULMAX1-RV64-NEXT:    mul a1, a2, a1
 ; LMULMAX1-RV64-NEXT:    srli a1, a1, 56
 ; LMULMAX1-RV64-NEXT:    addiw a1, a1, -48
 ; LMULMAX1-RV64-NEXT:    sh a1, 18(sp)
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 8, e16, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    addi a1, sp, 16
 ; LMULMAX1-RV64-NEXT:    vle16.v v25, (a1)
 ; LMULMAX1-RV64-NEXT:    addi a1, sp, 32

 ; LMULMAX2-RV32-NEXT:    .cfi_def_cfa s0, 0
 ; LMULMAX2-RV32-NEXT:    andi sp, sp, -32
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 4, e64, m2, ta, mu
 ; LMULMAX2-RV32-NEXT:    vle64.v v26, (a0)
 ; LMULMAX2-RV32-NEXT:    sw zero, 60(sp)
 ; LMULMAX2-RV32-NEXT:    sw zero, 52(sp)
 ; LMULMAX2-RV32-NEXT:    sw zero, 44(sp)
 ; LMULMAX2-RV32-NEXT:    sw zero, 36(sp)
-; LMULMAX2-RV32-NEXT:    addi a6, zero, 32
+; LMULMAX2-RV32-NEXT:    addi a5, zero, 32
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 1, e64, m2, ta, mu
-; LMULMAX2-RV32-NEXT:    vsrl.vx v28, v26, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a5, v28
-; LMULMAX2-RV32-NEXT:    lui a1, 349525
-; LMULMAX2-RV32-NEXT:    addi a4, a1, 1365
-; LMULMAX2-RV32-NEXT:    lui a1, 209715
-; LMULMAX2-RV32-NEXT:    addi a3, a1, 819
-; LMULMAX2-RV32-NEXT:    lui a1, 61681
-; LMULMAX2-RV32-NEXT:    addi a7, a1, -241
-; LMULMAX2-RV32-NEXT:    lui a1, 4112
-; LMULMAX2-RV32-NEXT:    addi a2, a1, 257
-; LMULMAX2-RV32-NEXT:    bnez a5, .LBB7_2
+; LMULMAX2-RV32-NEXT:    vsrl.vx v28, v26, a5
+; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v28
+; LMULMAX2-RV32-NEXT:    lui a2, 349525
+; LMULMAX2-RV32-NEXT:    addi a4, a2, 1365
+; LMULMAX2-RV32-NEXT:    lui a2, 209715
+; LMULMAX2-RV32-NEXT:    addi a3, a2, 819
+; LMULMAX2-RV32-NEXT:    lui a2, 61681
+; LMULMAX2-RV32-NEXT:    addi a6, a2, -241
+; LMULMAX2-RV32-NEXT:    lui a2, 4112
+; LMULMAX2-RV32-NEXT:    addi a7, a2, 257
+; LMULMAX2-RV32-NEXT:    bnez a1, .LBB7_2
 ; LMULMAX2-RV32-NEXT:  # %bb.1:
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    and a5, a5, a4
-; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a5, a1, a3
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    and a2, a2, a4
+; LMULMAX2-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
-; LMULMAX2-RV32-NEXT:    add a1, a5, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    add a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a1, a1, a7
-; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
+; LMULMAX2-RV32-NEXT:    add a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    add a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a1, a1, a6
+; LMULMAX2-RV32-NEXT:    mul a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
-; LMULMAX2-RV32-NEXT:    addi a5, a1, 32
+; LMULMAX2-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX2-RV32-NEXT:    j .LBB7_3
 ; LMULMAX2-RV32-NEXT:  .LBB7_2:
-; LMULMAX2-RV32-NEXT:    srli a1, a5, 1
-; LMULMAX2-RV32-NEXT:    or a1, a5, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    and a5, a5, a4
-; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a5, a1, a3
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    and a2, a2, a4
+; LMULMAX2-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
-; LMULMAX2-RV32-NEXT:    add a1, a5, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    add a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a1, a1, a7
-; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
+; LMULMAX2-RV32-NEXT:    add a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    add a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a1, a1, a6
+; LMULMAX2-RV32-NEXT:    mul a1, a1, a7
+; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB7_3:
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v28, v26, 3
-; LMULMAX2-RV32-NEXT:    vsrl.vx v30, v28, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v30
-; LMULMAX2-RV32-NEXT:    sw a5, 32(sp)
-; LMULMAX2-RV32-NEXT:    bnez a1, .LBB7_5
+; LMULMAX2-RV32-NEXT:    vsrl.vx v30, v28, a5
+; LMULMAX2-RV32-NEXT:    vmv.x.s a2, v30
+; LMULMAX2-RV32-NEXT:    sw a1, 32(sp)
+; LMULMAX2-RV32-NEXT:    bnez a2, .LBB7_5
 ; LMULMAX2-RV32-NEXT:  # %bb.4:
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    and a5, a5, a4
-; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a5, a1, a3
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    and a2, a2, a4
+; LMULMAX2-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
-; LMULMAX2-RV32-NEXT:    add a1, a5, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    add a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a1, a1, a7
-; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
+; LMULMAX2-RV32-NEXT:    add a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    add a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a1, a1, a6
+; LMULMAX2-RV32-NEXT:    mul a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
-; LMULMAX2-RV32-NEXT:    addi a5, a1, 32
+; LMULMAX2-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX2-RV32-NEXT:    j .LBB7_6
 ; LMULMAX2-RV32-NEXT:  .LBB7_5:
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a1, a2, 1
+; LMULMAX2-RV32-NEXT:    or a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    and a5, a5, a4
-; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a5, a1, a3
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    and a2, a2, a4
+; LMULMAX2-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
-; LMULMAX2-RV32-NEXT:    add a1, a5, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    add a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a1, a1, a7
-; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
+; LMULMAX2-RV32-NEXT:    add a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    add a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a1, a1, a6
+; LMULMAX2-RV32-NEXT:    mul a1, a1, a7
+; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB7_6:
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v28, v26, 2
-; LMULMAX2-RV32-NEXT:    vsrl.vx v30, v28, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v30
-; LMULMAX2-RV32-NEXT:    sw a5, 56(sp)
-; LMULMAX2-RV32-NEXT:    bnez a1, .LBB7_8
+; LMULMAX2-RV32-NEXT:    vsrl.vx v30, v28, a5
+; LMULMAX2-RV32-NEXT:    vmv.x.s a2, v30
+; LMULMAX2-RV32-NEXT:    sw a1, 56(sp)
+; LMULMAX2-RV32-NEXT:    bnez a2, .LBB7_8
 ; LMULMAX2-RV32-NEXT:  # %bb.7:
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    and a5, a5, a4
-; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a5, a1, a3
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    and a2, a2, a4
+; LMULMAX2-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
-; LMULMAX2-RV32-NEXT:    add a1, a5, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    add a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a1, a1, a7
-; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
+; LMULMAX2-RV32-NEXT:    add a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    add a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a1, a1, a6
+; LMULMAX2-RV32-NEXT:    mul a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
-; LMULMAX2-RV32-NEXT:    addi a5, a1, 32
+; LMULMAX2-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX2-RV32-NEXT:    j .LBB7_9
 ; LMULMAX2-RV32-NEXT:  .LBB7_8:
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a1, a2, 1
+; LMULMAX2-RV32-NEXT:    or a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    and a5, a5, a4
-; LMULMAX2-RV32-NEXT:    sub a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a5, a1, a3
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    and a2, a2, a4
+; LMULMAX2-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
-; LMULMAX2-RV32-NEXT:    add a1, a5, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    add a1, a1, a5
-; LMULMAX2-RV32-NEXT:    and a1, a1, a7
-; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 24
+; LMULMAX2-RV32-NEXT:    add a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    add a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a1, a1, a6
+; LMULMAX2-RV32-NEXT:    mul a1, a1, a7
+; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB7_9:
 ; LMULMAX2-RV32-NEXT:    vslidedown.vi v26, v26, 1
-; LMULMAX2-RV32-NEXT:    vsrl.vx v28, v26, a6
-; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV32-NEXT:    sw a5, 48(sp)
-; LMULMAX2-RV32-NEXT:    bnez a1, .LBB7_11
+; LMULMAX2-RV32-NEXT:    vsrl.vx v28, v26, a5
+; LMULMAX2-RV32-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV32-NEXT:    sw a1, 48(sp)
+; LMULMAX2-RV32-NEXT:    bnez a2, .LBB7_11
 ; LMULMAX2-RV32-NEXT:  # %bb.10:
 ; LMULMAX2-RV32-NEXT:    vmv.x.s a1, v26
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    and a4, a5, a4
-; LMULMAX2-RV32-NEXT:    sub a1, a1, a4
-; LMULMAX2-RV32-NEXT:    and a4, a1, a3
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    and a2, a2, a4
+; LMULMAX2-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
-; LMULMAX2-RV32-NEXT:    add a1, a4, a1
-; LMULMAX2-RV32-NEXT:    srli a3, a1, 4
-; LMULMAX2-RV32-NEXT:    add a1, a1, a3
-; LMULMAX2-RV32-NEXT:    and a1, a1, a7
-; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
+; LMULMAX2-RV32-NEXT:    add a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    add a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a1, a1, a6
+; LMULMAX2-RV32-NEXT:    mul a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX2-RV32-NEXT:    j .LBB7_12
 ; LMULMAX2-RV32-NEXT:  .LBB7_11:
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 2
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 4
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 8
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 16
-; LMULMAX2-RV32-NEXT:    or a1, a1, a5
+; LMULMAX2-RV32-NEXT:    srli a1, a2, 1
+; LMULMAX2-RV32-NEXT:    or a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 2
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 8
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 16
+; LMULMAX2-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX2-RV32-NEXT:    not a1, a1
-; LMULMAX2-RV32-NEXT:    srli a5, a1, 1
-; LMULMAX2-RV32-NEXT:    and a4, a5, a4
-; LMULMAX2-RV32-NEXT:    sub a1, a1, a4
-; LMULMAX2-RV32-NEXT:    and a4, a1, a3
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV32-NEXT:    and a2, a2, a4
+; LMULMAX2-RV32-NEXT:    sub a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV32-NEXT:    and a1, a1, a3
-; LMULMAX2-RV32-NEXT:    add a1, a4, a1
-; LMULMAX2-RV32-NEXT:    srli a3, a1, 4
-; LMULMAX2-RV32-NEXT:    add a1, a1, a3
-; LMULMAX2-RV32-NEXT:    and a1, a1, a7
-; LMULMAX2-RV32-NEXT:    mul a1, a1, a2
+; LMULMAX2-RV32-NEXT:    add a1, a2, a1
+; LMULMAX2-RV32-NEXT:    srli a2, a1, 4
+; LMULMAX2-RV32-NEXT:    add a1, a1, a2
+; LMULMAX2-RV32-NEXT:    and a1, a1, a6
+; LMULMAX2-RV32-NEXT:    mul a1, a1, a7
 ; LMULMAX2-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX2-RV32-NEXT:  .LBB7_12:
 ; LMULMAX2-RV32-NEXT:    sw a1, 40(sp)
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 8, e32, m2, ta, mu
 ; LMULMAX2-RV32-NEXT:    addi a1, sp, 32
 ; LMULMAX2-RV32-NEXT:    vle32.v v26, (a1)
 ; LMULMAX2-RV32-NEXT:    vsetivli zero, 4, e64, m2, ta, mu
 ; LMULMAX2-RV32-NEXT:    vse64.v v26, (a0)

 ; LMULMAX1-RV32-NEXT:    addi sp, sp, -48
 ; LMULMAX1-RV32-NEXT:    .cfi_def_cfa_offset 48
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vle64.v v25, (a0)
 ; LMULMAX1-RV32-NEXT:    addi a6, a0, 16
 ; LMULMAX1-RV32-NEXT:    vle64.v v26, (a6)
 ; LMULMAX1-RV32-NEXT:    sw zero, 44(sp)
 ; LMULMAX1-RV32-NEXT:    sw zero, 36(sp)
-; LMULMAX1-RV32-NEXT:    addi a7, zero, 32
+; LMULMAX1-RV32-NEXT:    addi a1, zero, 32
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    vsrl.vx v27, v26, a7
-; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v27
-; LMULMAX1-RV32-NEXT:    lui a2, 349525
-; LMULMAX1-RV32-NEXT:    addi a5, a2, 1365
-; LMULMAX1-RV32-NEXT:    lui a2, 209715
-; LMULMAX1-RV32-NEXT:    addi a4, a2, 819
-; LMULMAX1-RV32-NEXT:    lui a2, 61681
-; LMULMAX1-RV32-NEXT:    addi t0, a2, -241
-; LMULMAX1-RV32-NEXT:    lui a2, 4112
-; LMULMAX1-RV32-NEXT:    addi a3, a2, 257
-; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_2
+; LMULMAX1-RV32-NEXT:    vsrl.vx v27, v26, a1
+; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v27
+; LMULMAX1-RV32-NEXT:    lui a3, 349525
+; LMULMAX1-RV32-NEXT:    addi a5, a3, 1365
+; LMULMAX1-RV32-NEXT:    lui a3, 209715
+; LMULMAX1-RV32-NEXT:    addi a4, a3, 819
+; LMULMAX1-RV32-NEXT:    lui a3, 61681
+; LMULMAX1-RV32-NEXT:    addi a7, a3, -241
+; LMULMAX1-RV32-NEXT:    lui a3, 4112
+; LMULMAX1-RV32-NEXT:    addi t0, a3, 257
+; LMULMAX1-RV32-NEXT:    bnez a2, .LBB7_2
 ; LMULMAX1-RV32-NEXT:  # %bb.1:
-; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    not a1, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV32-NEXT:    and a2, a2, a5
-; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a2, a1, a4
-; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
-; LMULMAX1-RV32-NEXT:    and a1, a1, a4
-; LMULMAX1-RV32-NEXT:    add a1, a2, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV32-NEXT:    add a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a1, a1, t0
-; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
-; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
-; LMULMAX1-RV32-NEXT:    addi a1, a1, 32
+; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v26
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 2
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 4
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 8
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 16
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    not a2, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV32-NEXT:    and a3, a3, a5
+; LMULMAX1-RV32-NEXT:    sub a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a3, a2, a4
+; LMULMAX1-RV32-NEXT:    srli a2, a2, 2
+; LMULMAX1-RV32-NEXT:    and a2, a2, a4
+; LMULMAX1-RV32-NEXT:    add a2, a3, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 4
+; LMULMAX1-RV32-NEXT:    add a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a2, a2, a7
+; LMULMAX1-RV32-NEXT:    mul a2, a2, t0
+; LMULMAX1-RV32-NEXT:    srli a2, a2, 24
+; LMULMAX1-RV32-NEXT:    addi a2, a2, 32
 ; LMULMAX1-RV32-NEXT:    j .LBB7_3
 ; LMULMAX1-RV32-NEXT:  .LBB7_2:
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    not a1, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV32-NEXT:    and a2, a2, a5
-; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a2, a1, a4
-; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
-; LMULMAX1-RV32-NEXT:    and a1, a1, a4
-; LMULMAX1-RV32-NEXT:    add a1, a2, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV32-NEXT:    add a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a1, a1, t0
-; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
-; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 2
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 4
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 8
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 16
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    not a2, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV32-NEXT:    and a3, a3, a5
+; LMULMAX1-RV32-NEXT:    sub a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a3, a2, a4
+; LMULMAX1-RV32-NEXT:    srli a2, a2, 2
+; LMULMAX1-RV32-NEXT:    and a2, a2, a4
+; LMULMAX1-RV32-NEXT:    add a2, a3, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 4
+; LMULMAX1-RV32-NEXT:    add a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a2, a2, a7
+; LMULMAX1-RV32-NEXT:    mul a2, a2, t0
+; LMULMAX1-RV32-NEXT:    srli a2, a2, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_3:
 ; LMULMAX1-RV32-NEXT:    vslidedown.vi v26, v26, 1
-; LMULMAX1-RV32-NEXT:    vsrl.vx v27, v26, a7
-; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v27
-; LMULMAX1-RV32-NEXT:    sw a1, 32(sp)
-; LMULMAX1-RV32-NEXT:    bnez a2, .LBB7_5
+; LMULMAX1-RV32-NEXT:    vsrl.vx v27, v26, a1
+; LMULMAX1-RV32-NEXT:    vmv.x.s a3, v27
+; LMULMAX1-RV32-NEXT:    sw a2, 32(sp)
+; LMULMAX1-RV32-NEXT:    bnez a3, .LBB7_5
 ; LMULMAX1-RV32-NEXT:  # %bb.4:
-; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    not a1, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV32-NEXT:    and a2, a2, a5
-; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a2, a1, a4
-; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
-; LMULMAX1-RV32-NEXT:    and a1, a1, a4
-; LMULMAX1-RV32-NEXT:    add a1, a2, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV32-NEXT:    add a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a1, a1, t0
-; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
-; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
-; LMULMAX1-RV32-NEXT:    addi a1, a1, 32
+; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v26
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 2
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 4
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 8
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 16
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    not a2, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV32-NEXT:    and a3, a3, a5
+; LMULMAX1-RV32-NEXT:    sub a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a3, a2, a4
+; LMULMAX1-RV32-NEXT:    srli a2, a2, 2
+; LMULMAX1-RV32-NEXT:    and a2, a2, a4
+; LMULMAX1-RV32-NEXT:    add a2, a3, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 4
+; LMULMAX1-RV32-NEXT:    add a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a2, a2, a7
+; LMULMAX1-RV32-NEXT:    mul a2, a2, t0
+; LMULMAX1-RV32-NEXT:    srli a2, a2, 24
+; LMULMAX1-RV32-NEXT:    addi a2, a2, 32
 ; LMULMAX1-RV32-NEXT:    j .LBB7_6
 ; LMULMAX1-RV32-NEXT:  .LBB7_5:
-; LMULMAX1-RV32-NEXT:    srli a1, a2, 1
-; LMULMAX1-RV32-NEXT:    or a1, a2, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    not a1, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV32-NEXT:    and a2, a2, a5
-; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a2, a1, a4
-; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
-; LMULMAX1-RV32-NEXT:    and a1, a1, a4
-; LMULMAX1-RV32-NEXT:    add a1, a2, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV32-NEXT:    add a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a1, a1, t0
-; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
-; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
+; LMULMAX1-RV32-NEXT:    srli a2, a3, 1
+; LMULMAX1-RV32-NEXT:    or a2, a3, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 2
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 4
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 8
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 16
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    not a2, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV32-NEXT:    and a3, a3, a5
+; LMULMAX1-RV32-NEXT:    sub a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a3, a2, a4
+; LMULMAX1-RV32-NEXT:    srli a2, a2, 2
+; LMULMAX1-RV32-NEXT:    and a2, a2, a4
+; LMULMAX1-RV32-NEXT:    add a2, a3, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 4
+; LMULMAX1-RV32-NEXT:    add a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a2, a2, a7
+; LMULMAX1-RV32-NEXT:    mul a2, a2, t0
+; LMULMAX1-RV32-NEXT:    srli a2, a2, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_6:
-; LMULMAX1-RV32-NEXT:    sw a1, 40(sp)
+; LMULMAX1-RV32-NEXT:    sw a2, 40(sp)
 ; LMULMAX1-RV32-NEXT:    sw zero, 28(sp)
-; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a7
-; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a1
+; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v26
 ; LMULMAX1-RV32-NEXT:    sw zero, 20(sp)
-; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_8
+; LMULMAX1-RV32-NEXT:    bnez a2, .LBB7_8
 ; LMULMAX1-RV32-NEXT:  # %bb.7:
-; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    not a1, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV32-NEXT:    and a2, a2, a5
-; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a2, a1, a4
-; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
-; LMULMAX1-RV32-NEXT:    and a1, a1, a4
-; LMULMAX1-RV32-NEXT:    add a1, a2, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV32-NEXT:    add a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a1, a1, t0
-; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
-; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
-; LMULMAX1-RV32-NEXT:    addi a1, a1, 32
+; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v25
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 2
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 4
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 8
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 16
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    not a2, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV32-NEXT:    and a3, a3, a5
+; LMULMAX1-RV32-NEXT:    sub a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a3, a2, a4
+; LMULMAX1-RV32-NEXT:    srli a2, a2, 2
+; LMULMAX1-RV32-NEXT:    and a2, a2, a4
+; LMULMAX1-RV32-NEXT:    add a2, a3, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 4
+; LMULMAX1-RV32-NEXT:    add a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a2, a2, a7
+; LMULMAX1-RV32-NEXT:    mul a2, a2, t0
+; LMULMAX1-RV32-NEXT:    srli a2, a2, 24
+; LMULMAX1-RV32-NEXT:    addi a2, a2, 32
 ; LMULMAX1-RV32-NEXT:    j .LBB7_9
 ; LMULMAX1-RV32-NEXT:  .LBB7_8:
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
-; LMULMAX1-RV32-NEXT:    or a1, a1, a2
-; LMULMAX1-RV32-NEXT:    not a1, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
-; LMULMAX1-RV32-NEXT:    and a2, a2, a5
-; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a2, a1, a4
-; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
-; LMULMAX1-RV32-NEXT:    and a1, a1, a4
-; LMULMAX1-RV32-NEXT:    add a1, a2, a1
-; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
-; LMULMAX1-RV32-NEXT:    add a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a1, a1, t0
-; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
-; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 2
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 4
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 8
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 16
+; LMULMAX1-RV32-NEXT:    or a2, a2, a3
+; LMULMAX1-RV32-NEXT:    not a2, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV32-NEXT:    and a3, a3, a5
+; LMULMAX1-RV32-NEXT:    sub a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a3, a2, a4
+; LMULMAX1-RV32-NEXT:    srli a2, a2, 2
+; LMULMAX1-RV32-NEXT:    and a2, a2, a4
+; LMULMAX1-RV32-NEXT:    add a2, a3, a2
+; LMULMAX1-RV32-NEXT:    srli a3, a2, 4
+; LMULMAX1-RV32-NEXT:    add a2, a2, a3
+; LMULMAX1-RV32-NEXT:    and a2, a2, a7
+; LMULMAX1-RV32-NEXT:    mul a2, a2, t0
+; LMULMAX1-RV32-NEXT:    srli a2, a2, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_9:
 ; LMULMAX1-RV32-NEXT:    vslidedown.vi v25, v25, 1
-; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a7
-; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV32-NEXT:    sw a1, 16(sp)
-; LMULMAX1-RV32-NEXT:    bnez a2, .LBB7_11
+; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a1
+; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
+; LMULMAX1-RV32-NEXT:    sw a2, 16(sp)
+; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_11
 ; LMULMAX1-RV32-NEXT:  # %bb.10:
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    not a1, a1
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a2, a2, a5
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a2, a1, a4
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a4
 ; LMULMAX1-RV32-NEXT:    add a1, a2, a1
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
 ; LMULMAX1-RV32-NEXT:    add a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a1, a1, t0
-; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
+; LMULMAX1-RV32-NEXT:    and a1, a1, a7
+; LMULMAX1-RV32-NEXT:    mul a1, a1, t0
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:    addi a1, a1, 32
 ; LMULMAX1-RV32-NEXT:    j .LBB7_12
 ; LMULMAX1-RV32-NEXT:  .LBB7_11:
-; LMULMAX1-RV32-NEXT:    srli a1, a2, 1
-; LMULMAX1-RV32-NEXT:    or a1, a2, a1
+; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
+; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 2
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 8
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 16
 ; LMULMAX1-RV32-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    not a1, a1
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a2, a2, a5
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a2, a1, a4
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a4
 ; LMULMAX1-RV32-NEXT:    add a1, a2, a1
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 4
 ; LMULMAX1-RV32-NEXT:    add a1, a1, a2
-; LMULMAX1-RV32-NEXT:    and a1, a1, t0
-; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
+; LMULMAX1-RV32-NEXT:    and a1, a1, a7
+; LMULMAX1-RV32-NEXT:    mul a1, a1, t0
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_12:
 ; LMULMAX1-RV32-NEXT:    sw a1, 24(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    addi a1, sp, 16
 ; LMULMAX1-RV32-NEXT:    vle32.v v25, (a1)
 ; LMULMAX1-RV32-NEXT:    addi a1, sp, 32
 ; LMULMAX1-RV32-NEXT:    vle32.v v26, (a1)

llvm/test/CodeGen/RISCV/rvv/fixed-vectors-cttz.ll

 ; LMULMAX2-RV64-NEXT:    addi sp, sp, -32
 ; LMULMAX2-RV64-NEXT:    .cfi_def_cfa_offset 32
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX2-RV64-NEXT:    vle32.v v25, (a0)
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 1, e32, m1, ta, mu
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v26, v25, 3
 ; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v26
 ; LMULMAX2-RV64-NEXT:    addi a1, zero, 1
-; LMULMAX2-RV64-NEXT:    slli a6, a1, 32
-; LMULMAX2-RV64-NEXT:    or a2, a2, a6
+; LMULMAX2-RV64-NEXT:    slli a1, a1, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a1
 ; LMULMAX2-RV64-NEXT:    addi a3, a2, -1
 ; LMULMAX2-RV64-NEXT:    not a2, a2
 ; LMULMAX2-RV64-NEXT:    and a3, a2, a3
 ; LMULMAX2-RV64-NEXT:    srli a4, a3, 1
 ; LMULMAX2-RV64-NEXT:    lui a2, 21845
 ; LMULMAX2-RV64-NEXT:    addiw a2, a2, 1365
 ; LMULMAX2-RV64-NEXT:    slli a2, a2, 12
 ; LMULMAX2-RV64-NEXT:    addi a2, a2, 1365
 ; LMULMAX2-RV64-NEXT:    slli a2, a2, 12
 ; LMULMAX2-RV64-NEXT:    addi a2, a2, 1365
 ; LMULMAX2-RV64-NEXT:    slli a2, a2, 12
-; LMULMAX2-RV64-NEXT:    addi a7, a2, 1365
-; LMULMAX2-RV64-NEXT:    and a4, a4, a7
+; LMULMAX2-RV64-NEXT:    addi a6, a2, 1365
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
 ; LMULMAX2-RV64-NEXT:    sub a4, a3, a4
 ; LMULMAX2-RV64-NEXT:    lui a3, 13107
 ; LMULMAX2-RV64-NEXT:    addiw a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX2-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX2-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX2-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    and a5, a4, a3
 ; LMULMAX2-RV64-NEXT:    srli a4, a4, 2
 ; LMULMAX2-RV64-NEXT:    and a4, a4, a3
 ; LMULMAX2-RV64-NEXT:    add a4, a5, a4
 ; LMULMAX2-RV64-NEXT:    srli a5, a4, 4
 ; LMULMAX2-RV64-NEXT:    add a4, a4, a5
 ; LMULMAX2-RV64-NEXT:    lui a5, 3855
 ; LMULMAX2-RV64-NEXT:    addiw a5, a5, 241
 ; LMULMAX2-RV64-NEXT:    slli a5, a5, 12
 ; LMULMAX2-RV64-NEXT:    addi a5, a5, -241
 ; LMULMAX2-RV64-NEXT:    slli a5, a5, 12
 ; LMULMAX2-RV64-NEXT:    addi a5, a5, 241
 ; LMULMAX2-RV64-NEXT:    slli a5, a5, 12
-; LMULMAX2-RV64-NEXT:    addi a5, a5, -241
-; LMULMAX2-RV64-NEXT:    and a4, a4, a5
-; LMULMAX2-RV64-NEXT:    lui a1, 4112
-; LMULMAX2-RV64-NEXT:    addiw a1, a1, 257
-; LMULMAX2-RV64-NEXT:    slli a1, a1, 16
-; LMULMAX2-RV64-NEXT:    addi a1, a1, 257
-; LMULMAX2-RV64-NEXT:    slli a1, a1, 16
-; LMULMAX2-RV64-NEXT:    addi a1, a1, 257
-; LMULMAX2-RV64-NEXT:    mul a4, a4, a1
+; LMULMAX2-RV64-NEXT:    addi a7, a5, -241
+; LMULMAX2-RV64-NEXT:    and a4, a4, a7
+; LMULMAX2-RV64-NEXT:    lui a2, 4112
+; LMULMAX2-RV64-NEXT:    addiw a2, a2, 257
+; LMULMAX2-RV64-NEXT:    slli a2, a2, 16
+; LMULMAX2-RV64-NEXT:    addi a2, a2, 257
+; LMULMAX2-RV64-NEXT:    slli a2, a2, 16
+; LMULMAX2-RV64-NEXT:    addi a2, a2, 257
+; LMULMAX2-RV64-NEXT:    mul a4, a4, a2
 ; LMULMAX2-RV64-NEXT:    srli a4, a4, 56
 ; LMULMAX2-RV64-NEXT:    sw a4, 28(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v26, v25, 2
 ; LMULMAX2-RV64-NEXT:    vmv.x.s a4, v26
-; LMULMAX2-RV64-NEXT:    or a4, a4, a6
-; LMULMAX2-RV64-NEXT:    addi a2, a4, -1
+; LMULMAX2-RV64-NEXT:    or a4, a4, a1
+; LMULMAX2-RV64-NEXT:    addi a5, a4, -1
 ; LMULMAX2-RV64-NEXT:    not a4, a4
-; LMULMAX2-RV64-NEXT:    and a2, a4, a2
-; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a5
+; LMULMAX2-RV64-NEXT:    srli a5, a4, 1
+; LMULMAX2-RV64-NEXT:    and a5, a5, a6
+; LMULMAX2-RV64-NEXT:    sub a4, a4, a5
+; LMULMAX2-RV64-NEXT:    and a5, a4, a3
+; LMULMAX2-RV64-NEXT:    srli a4, a4, 2
+; LMULMAX2-RV64-NEXT:    and a4, a4, a3
+; LMULMAX2-RV64-NEXT:    add a4, a5, a4
+; LMULMAX2-RV64-NEXT:    srli a5, a4, 4
+; LMULMAX2-RV64-NEXT:    add a4, a4, a5
 ; LMULMAX2-RV64-NEXT:    and a4, a4, a7
-; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
-; LMULMAX2-RV64-NEXT:    and a4, a2, a3
-; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX2-RV64-NEXT:    and a2, a2, a3
-; LMULMAX2-RV64-NEXT:    add a2, a4, a2
-; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
-; LMULMAX2-RV64-NEXT:    add a2, a2, a4
-; LMULMAX2-RV64-NEXT:    and a2, a2, a5
-; LMULMAX2-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX2-RV64-NEXT:    sw a2, 24(sp)
+; LMULMAX2-RV64-NEXT:    mul a4, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a4, 56
+; LMULMAX2-RV64-NEXT:    sw a4, 24(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v26, v25, 1
-; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX2-RV64-NEXT:    or a2, a2, a6
-; LMULMAX2-RV64-NEXT:    addi a4, a2, -1
-; LMULMAX2-RV64-NEXT:    not a2, a2
-; LMULMAX2-RV64-NEXT:    and a2, a2, a4
-; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
-; LMULMAX2-RV64-NEXT:    and a4, a4, a7
-; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
-; LMULMAX2-RV64-NEXT:    and a4, a2, a3
-; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX2-RV64-NEXT:    and a2, a2, a3
-; LMULMAX2-RV64-NEXT:    add a2, a4, a2
-; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
-; LMULMAX2-RV64-NEXT:    add a2, a2, a4
-; LMULMAX2-RV64-NEXT:    and a2, a2, a5
-; LMULMAX2-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX2-RV64-NEXT:    sw a2, 20(sp)
-; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v25
-; LMULMAX2-RV64-NEXT:    or a2, a2, a6
-; LMULMAX2-RV64-NEXT:    addi a4, a2, -1
-; LMULMAX2-RV64-NEXT:    not a2, a2
-; LMULMAX2-RV64-NEXT:    and a2, a2, a4
-; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    vmv.x.s a4, v26
+; LMULMAX2-RV64-NEXT:    or a4, a4, a1
+; LMULMAX2-RV64-NEXT:    addi a5, a4, -1
+; LMULMAX2-RV64-NEXT:    not a4, a4
+; LMULMAX2-RV64-NEXT:    and a4, a4, a5
+; LMULMAX2-RV64-NEXT:    srli a5, a4, 1
+; LMULMAX2-RV64-NEXT:    and a5, a5, a6
+; LMULMAX2-RV64-NEXT:    sub a4, a4, a5
+; LMULMAX2-RV64-NEXT:    and a5, a4, a3
+; LMULMAX2-RV64-NEXT:    srli a4, a4, 2
+; LMULMAX2-RV64-NEXT:    and a4, a4, a3
+; LMULMAX2-RV64-NEXT:    add a4, a5, a4
+; LMULMAX2-RV64-NEXT:    srli a5, a4, 4
+; LMULMAX2-RV64-NEXT:    add a4, a4, a5
 ; LMULMAX2-RV64-NEXT:    and a4, a4, a7
-; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
-; LMULMAX2-RV64-NEXT:    and a4, a2, a3
-; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX2-RV64-NEXT:    and a2, a2, a3
-; LMULMAX2-RV64-NEXT:    add a2, a4, a2
-; LMULMAX2-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX2-RV64-NEXT:    add a2, a2, a3
-; LMULMAX2-RV64-NEXT:    and a2, a2, a5
-; LMULMAX2-RV64-NEXT:    mul a1, a2, a1
+; LMULMAX2-RV64-NEXT:    mul a4, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a4, 56
+; LMULMAX2-RV64-NEXT:    sw a4, 20(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a4, v25
+; LMULMAX2-RV64-NEXT:    or a1, a4, a1
+; LMULMAX2-RV64-NEXT:    addi a4, a1, -1
+; LMULMAX2-RV64-NEXT:    not a1, a1
+; LMULMAX2-RV64-NEXT:    and a1, a1, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a1, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a1, a1, a4
+; LMULMAX2-RV64-NEXT:    and a4, a1, a3
+; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
+; LMULMAX2-RV64-NEXT:    and a1, a1, a3
+; LMULMAX2-RV64-NEXT:    add a1, a4, a1
+; LMULMAX2-RV64-NEXT:    srli a3, a1, 4
+; LMULMAX2-RV64-NEXT:    add a1, a1, a3
+; LMULMAX2-RV64-NEXT:    and a1, a1, a7
+; LMULMAX2-RV64-NEXT:    mul a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
 ; LMULMAX2-RV64-NEXT:    sw a1, 16(sp)
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX2-RV64-NEXT:    addi a1, sp, 16
 ; LMULMAX2-RV64-NEXT:    vle32.v v25, (a1)
 ; LMULMAX2-RV64-NEXT:    vse32.v v25, (a0)
 ; LMULMAX2-RV64-NEXT:    addi sp, sp, 32
 ; LMULMAX2-RV64-NEXT:    ret

 ; LMULMAX1-RV64-NEXT:    addi sp, sp, -32
 ; LMULMAX1-RV64-NEXT:    .cfi_def_cfa_offset 32
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    vle32.v v25, (a0)
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 1, e32, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 3
 ; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
 ; LMULMAX1-RV64-NEXT:    addi a1, zero, 1
-; LMULMAX1-RV64-NEXT:    slli a6, a1, 32
-; LMULMAX1-RV64-NEXT:    or a2, a2, a6
+; LMULMAX1-RV64-NEXT:    slli a1, a1, 32
+; LMULMAX1-RV64-NEXT:    or a2, a2, a1
 ; LMULMAX1-RV64-NEXT:    addi a3, a2, -1
 ; LMULMAX1-RV64-NEXT:    not a2, a2
 ; LMULMAX1-RV64-NEXT:    and a3, a2, a3
 ; LMULMAX1-RV64-NEXT:    srli a4, a3, 1
 ; LMULMAX1-RV64-NEXT:    lui a2, 21845
 ; LMULMAX1-RV64-NEXT:    addiw a2, a2, 1365
 ; LMULMAX1-RV64-NEXT:    slli a2, a2, 12
 ; LMULMAX1-RV64-NEXT:    addi a2, a2, 1365
 ; LMULMAX1-RV64-NEXT:    slli a2, a2, 12
 ; LMULMAX1-RV64-NEXT:    addi a2, a2, 1365
 ; LMULMAX1-RV64-NEXT:    slli a2, a2, 12
-; LMULMAX1-RV64-NEXT:    addi a7, a2, 1365
-; LMULMAX1-RV64-NEXT:    and a4, a4, a7
+; LMULMAX1-RV64-NEXT:    addi a6, a2, 1365
+; LMULMAX1-RV64-NEXT:    and a4, a4, a6
 ; LMULMAX1-RV64-NEXT:    sub a4, a3, a4
 ; LMULMAX1-RV64-NEXT:    lui a3, 13107
 ; LMULMAX1-RV64-NEXT:    addiw a3, a3, 819
 ; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX1-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX1-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX1-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX1-RV64-NEXT:    and a5, a4, a3
 ; LMULMAX1-RV64-NEXT:    srli a4, a4, 2
 ; LMULMAX1-RV64-NEXT:    and a4, a4, a3
 ; LMULMAX1-RV64-NEXT:    add a4, a5, a4
 ; LMULMAX1-RV64-NEXT:    srli a5, a4, 4
 ; LMULMAX1-RV64-NEXT:    add a4, a4, a5
 ; LMULMAX1-RV64-NEXT:    lui a5, 3855
 ; LMULMAX1-RV64-NEXT:    addiw a5, a5, 241
 ; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
 ; LMULMAX1-RV64-NEXT:    addi a5, a5, -241
 ; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
 ; LMULMAX1-RV64-NEXT:    addi a5, a5, 241
 ; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
-; LMULMAX1-RV64-NEXT:    addi a5, a5, -241
-; LMULMAX1-RV64-NEXT:    and a4, a4, a5
-; LMULMAX1-RV64-NEXT:    lui a1, 4112
-; LMULMAX1-RV64-NEXT:    addiw a1, a1, 257
-; LMULMAX1-RV64-NEXT:    slli a1, a1, 16
-; LMULMAX1-RV64-NEXT:    addi a1, a1, 257
-; LMULMAX1-RV64-NEXT:    slli a1, a1, 16
-; LMULMAX1-RV64-NEXT:    addi a1, a1, 257
-; LMULMAX1-RV64-NEXT:    mul a4, a4, a1
+; LMULMAX1-RV64-NEXT:    addi a7, a5, -241
+; LMULMAX1-RV64-NEXT:    and a4, a4, a7
+; LMULMAX1-RV64-NEXT:    lui a2, 4112
+; LMULMAX1-RV64-NEXT:    addiw a2, a2, 257
+; LMULMAX1-RV64-NEXT:    slli a2, a2, 16
+; LMULMAX1-RV64-NEXT:    addi a2, a2, 257
+; LMULMAX1-RV64-NEXT:    slli a2, a2, 16
+; LMULMAX1-RV64-NEXT:    addi a2, a2, 257
+; LMULMAX1-RV64-NEXT:    mul a4, a4, a2
 ; LMULMAX1-RV64-NEXT:    srli a4, a4, 56
 ; LMULMAX1-RV64-NEXT:    sw a4, 28(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 2
 ; LMULMAX1-RV64-NEXT:    vmv.x.s a4, v26
-; LMULMAX1-RV64-NEXT:    or a4, a4, a6
-; LMULMAX1-RV64-NEXT:    addi a2, a4, -1
+; LMULMAX1-RV64-NEXT:    or a4, a4, a1
+; LMULMAX1-RV64-NEXT:    addi a5, a4, -1
 ; LMULMAX1-RV64-NEXT:    not a4, a4
-; LMULMAX1-RV64-NEXT:    and a2, a4, a2
-; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    and a4, a4, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a4, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a6
+; LMULMAX1-RV64-NEXT:    sub a4, a4, a5
+; LMULMAX1-RV64-NEXT:    and a5, a4, a3
+; LMULMAX1-RV64-NEXT:    srli a4, a4, 2
+; LMULMAX1-RV64-NEXT:    and a4, a4, a3
+; LMULMAX1-RV64-NEXT:    add a4, a5, a4
+; LMULMAX1-RV64-NEXT:    srli a5, a4, 4
+; LMULMAX1-RV64-NEXT:    add a4, a4, a5
 ; LMULMAX1-RV64-NEXT:    and a4, a4, a7
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a4
-; LMULMAX1-RV64-NEXT:    and a4, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a3
-; LMULMAX1-RV64-NEXT:    add a2, a4, a2
-; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a4
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    sw a2, 24(sp)
+; LMULMAX1-RV64-NEXT:    mul a4, a4, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a4, 56
+; LMULMAX1-RV64-NEXT:    sw a4, 24(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 1
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV64-NEXT:    or a2, a2, a6
-; LMULMAX1-RV64-NEXT:    addi a4, a2, -1
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
-; LMULMAX1-RV64-NEXT:    and a4, a4, a7
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a4
-; LMULMAX1-RV64-NEXT:    and a4, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a3
-; LMULMAX1-RV64-NEXT:    add a2, a4, a2
-; LMULMAX1-RV64-NEXT:    srli a4, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a4
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    sw a2, 20(sp)
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v25
-; LMULMAX1-RV64-NEXT:    or a2, a2, a6
-; LMULMAX1-RV64-NEXT:    addi a4, a2, -1
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a4, v26
+; LMULMAX1-RV64-NEXT:    or a4, a4, a1
+; LMULMAX1-RV64-NEXT:    addi a5, a4, -1
+; LMULMAX1-RV64-NEXT:    not a4, a4
+; LMULMAX1-RV64-NEXT:    and a4, a4, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a4, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a6
+; LMULMAX1-RV64-NEXT:    sub a4, a4, a5
+; LMULMAX1-RV64-NEXT:    and a5, a4, a3
+; LMULMAX1-RV64-NEXT:    srli a4, a4, 2
+; LMULMAX1-RV64-NEXT:    and a4, a4, a3
+; LMULMAX1-RV64-NEXT:    add a4, a5, a4
+; LMULMAX1-RV64-NEXT:    srli a5, a4, 4
+; LMULMAX1-RV64-NEXT:    add a4, a4, a5
 ; LMULMAX1-RV64-NEXT:    and a4, a4, a7
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a4
-; LMULMAX1-RV64-NEXT:    and a4, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a3
-; LMULMAX1-RV64-NEXT:    add a2, a4, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a1, a2, a1
+; LMULMAX1-RV64-NEXT:    mul a4, a4, a2
+; LMULMAX1-RV64-NEXT:    srli a4, a4, 56
+; LMULMAX1-RV64-NEXT:    sw a4, 20(sp)
+; LMULMAX1-RV64-NEXT:    vmv.x.s a4, v25
+; LMULMAX1-RV64-NEXT:    or a1, a4, a1
+; LMULMAX1-RV64-NEXT:    addi a4, a1, -1
+; LMULMAX1-RV64-NEXT:    not a1, a1
+; LMULMAX1-RV64-NEXT:    and a1, a1, a4
+; LMULMAX1-RV64-NEXT:    srli a4, a1, 1
+; LMULMAX1-RV64-NEXT:    and a4, a4, a6
+; LMULMAX1-RV64-NEXT:    sub a1, a1, a4
+; LMULMAX1-RV64-NEXT:    and a4, a1, a3
+; LMULMAX1-RV64-NEXT:    srli a1, a1, 2
+; LMULMAX1-RV64-NEXT:    and a1, a1, a3
+; LMULMAX1-RV64-NEXT:    add a1, a4, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a1, 4
+; LMULMAX1-RV64-NEXT:    add a1, a1, a3
+; LMULMAX1-RV64-NEXT:    and a1, a1, a7
+; LMULMAX1-RV64-NEXT:    mul a1, a1, a2
 ; LMULMAX1-RV64-NEXT:    srli a1, a1, 56
 ; LMULMAX1-RV64-NEXT:    sw a1, 16(sp)
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    addi a1, sp, 16
 ; LMULMAX1-RV64-NEXT:    vle32.v v25, (a1)
 ; LMULMAX1-RV64-NEXT:    vse32.v v25, (a0)
 ; LMULMAX1-RV64-NEXT:    addi sp, sp, 32
 ; LMULMAX1-RV64-NEXT:    ret

 ; LMULMAX2-RV64-NEXT:    .cfi_def_cfa s0, 0
 ; LMULMAX2-RV64-NEXT:    andi sp, sp, -32
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 8, e32, m2, ta, mu
 ; LMULMAX2-RV64-NEXT:    vle32.v v26, (a0)
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 1, e32, m2, ta, mu
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 7
 ; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
 ; LMULMAX2-RV64-NEXT:    addi a1, zero, 1
-; LMULMAX2-RV64-NEXT:    slli a6, a1, 32
-; LMULMAX2-RV64-NEXT:    or a2, a2, a6
+; LMULMAX2-RV64-NEXT:    slli a1, a1, 32
+; LMULMAX2-RV64-NEXT:    or a2, a2, a1
 ; LMULMAX2-RV64-NEXT:    addi a3, a2, -1
 ; LMULMAX2-RV64-NEXT:    not a2, a2
 ; LMULMAX2-RV64-NEXT:    and a3, a2, a3
 ; LMULMAX2-RV64-NEXT:    srli a4, a3, 1
 ; LMULMAX2-RV64-NEXT:    lui a2, 21845
 ; LMULMAX2-RV64-NEXT:    addiw a2, a2, 1365
 ; LMULMAX2-RV64-NEXT:    slli a2, a2, 12
 ; LMULMAX2-RV64-NEXT:    addi a2, a2, 1365
 ; LMULMAX2-RV64-NEXT:    slli a2, a2, 12
 ; LMULMAX2-RV64-NEXT:    addi a2, a2, 1365
 ; LMULMAX2-RV64-NEXT:    slli a2, a2, 12
-; LMULMAX2-RV64-NEXT:    addi a7, a2, 1365
-; LMULMAX2-RV64-NEXT:    and a4, a4, a7
+; LMULMAX2-RV64-NEXT:    addi a6, a2, 1365
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
 ; LMULMAX2-RV64-NEXT:    sub a4, a3, a4
 ; LMULMAX2-RV64-NEXT:    lui a3, 13107
 ; LMULMAX2-RV64-NEXT:    addiw a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX2-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX2-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX2-RV64-NEXT:    addi a3, a3, 819
 ; LMULMAX2-RV64-NEXT:    and a5, a4, a3
 ; LMULMAX2-RV64-NEXT:    srli a4, a4, 2
 ; LMULMAX2-RV64-NEXT:    and a4, a4, a3
 ; LMULMAX2-RV64-NEXT:    add a4, a5, a4
 ; LMULMAX2-RV64-NEXT:    srli a5, a4, 4
 ; LMULMAX2-RV64-NEXT:    add a5, a4, a5
 ; LMULMAX2-RV64-NEXT:    lui a4, 3855
 ; LMULMAX2-RV64-NEXT:    addiw a4, a4, 241
 ; LMULMAX2-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX2-RV64-NEXT:    addi a4, a4, -241
 ; LMULMAX2-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX2-RV64-NEXT:    addi a4, a4, 241
 ; LMULMAX2-RV64-NEXT:    slli a4, a4, 12
-; LMULMAX2-RV64-NEXT:    addi a4, a4, -241
-; LMULMAX2-RV64-NEXT:    and a1, a5, a4
+; LMULMAX2-RV64-NEXT:    addi a7, a4, -241
+; LMULMAX2-RV64-NEXT:    and a2, a5, a7
 ; LMULMAX2-RV64-NEXT:    lui a5, 4112
 ; LMULMAX2-RV64-NEXT:    addiw a5, a5, 257
 ; LMULMAX2-RV64-NEXT:    slli a5, a5, 16
 ; LMULMAX2-RV64-NEXT:    addi a5, a5, 257
 ; LMULMAX2-RV64-NEXT:    slli a5, a5, 16
 ; LMULMAX2-RV64-NEXT:    addi a5, a5, 257
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    sw a1, 60(sp)
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    sw a2, 60(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 6
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    or a1, a1, a6
-; LMULMAX2-RV64-NEXT:    addi a2, a1, -1
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    and a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    or a2, a2, a1
+; LMULMAX2-RV64-NEXT:    addi a4, a2, -1
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
 ; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    sw a1, 56(sp)
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    sw a2, 56(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 5
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    or a1, a1, a6
-; LMULMAX2-RV64-NEXT:    addi a2, a1, -1
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    and a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    or a2, a2, a1
+; LMULMAX2-RV64-NEXT:    addi a4, a2, -1
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
 ; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    sw a1, 52(sp)
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    sw a2, 52(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 4
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    or a1, a1, a6
-; LMULMAX2-RV64-NEXT:    addi a2, a1, -1
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    and a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    or a2, a2, a1
+; LMULMAX2-RV64-NEXT:    addi a4, a2, -1
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
 ; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    sw a1, 48(sp)
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    sw a2, 48(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 3
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    or a1, a1, a6
-; LMULMAX2-RV64-NEXT:    addi a2, a1, -1
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    and a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    or a2, a2, a1
+; LMULMAX2-RV64-NEXT:    addi a4, a2, -1
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
 ; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    sw a1, 44(sp)
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    sw a2, 44(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 2
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    or a1, a1, a6
-; LMULMAX2-RV64-NEXT:    addi a2, a1, -1
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    and a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    or a2, a2, a1
+; LMULMAX2-RV64-NEXT:    addi a4, a2, -1
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
 ; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    sw a1, 40(sp)
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    sw a2, 40(sp)
 ; LMULMAX2-RV64-NEXT:    vslidedown.vi v28, v26, 1
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v28
-; LMULMAX2-RV64-NEXT:    or a1, a1, a6
-; LMULMAX2-RV64-NEXT:    addi a2, a1, -1
-; LMULMAX2-RV64-NEXT:    not a1, a1
-; LMULMAX2-RV64-NEXT:    and a1, a1, a2
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v28
+; LMULMAX2-RV64-NEXT:    or a2, a2, a1
+; LMULMAX2-RV64-NEXT:    addi a4, a2, -1
+; LMULMAX2-RV64-NEXT:    not a2, a2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a4
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 1
+; LMULMAX2-RV64-NEXT:    and a4, a4, a6
+; LMULMAX2-RV64-NEXT:    sub a2, a2, a4
+; LMULMAX2-RV64-NEXT:    and a4, a2, a3
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 2
+; LMULMAX2-RV64-NEXT:    and a2, a2, a3
+; LMULMAX2-RV64-NEXT:    add a2, a4, a2
+; LMULMAX2-RV64-NEXT:    srli a4, a2, 4
+; LMULMAX2-RV64-NEXT:    add a2, a2, a4
 ; LMULMAX2-RV64-NEXT:    and a2, a2, a7
-; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a2, a1, a3
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a3
-; LMULMAX2-RV64-NEXT:    add a1, a2, a1
-; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
-; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
-; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
-; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
-; LMULMAX2-RV64-NEXT:    sw a1, 36(sp)
-; LMULMAX2-RV64-NEXT:    vmv.x.s a1, v26
-; LMULMAX2-RV64-NEXT:    or a1, a1, a6
+; LMULMAX2-RV64-NEXT:    mul a2, a2, a5
+; LMULMAX2-RV64-NEXT:    srli a2, a2, 56
+; LMULMAX2-RV64-NEXT:    sw a2, 36(sp)
+; LMULMAX2-RV64-NEXT:    vmv.x.s a2, v26
+; LMULMAX2-RV64-NEXT:    or a1, a2, a1
 ; LMULMAX2-RV64-NEXT:    addi a2, a1, -1
 ; LMULMAX2-RV64-NEXT:    not a1, a1
 ; LMULMAX2-RV64-NEXT:    and a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 1
-; LMULMAX2-RV64-NEXT:    and a2, a2, a7
+; LMULMAX2-RV64-NEXT:    and a2, a2, a6
 ; LMULMAX2-RV64-NEXT:    sub a1, a1, a2
 ; LMULMAX2-RV64-NEXT:    and a2, a1, a3
 ; LMULMAX2-RV64-NEXT:    srli a1, a1, 2
 ; LMULMAX2-RV64-NEXT:    and a1, a1, a3
 ; LMULMAX2-RV64-NEXT:    add a1, a2, a1
 ; LMULMAX2-RV64-NEXT:    srli a2, a1, 4
 ; LMULMAX2-RV64-NEXT:    add a1, a1, a2
-; LMULMAX2-RV64-NEXT:    and a1, a1, a4
+; LMULMAX2-RV64-NEXT:    and a1, a1, a7
 ; LMULMAX2-RV64-NEXT:    mul a1, a1, a5
 ; LMULMAX2-RV64-NEXT:    srli a1, a1, 56
 ; LMULMAX2-RV64-NEXT:    sw a1, 32(sp)
 ; LMULMAX2-RV64-NEXT:    vsetivli zero, 8, e32, m2, ta, mu
 ; LMULMAX2-RV64-NEXT:    addi a1, sp, 32
 ; LMULMAX2-RV64-NEXT:    vle32.v v26, (a1)
 ; LMULMAX2-RV64-NEXT:    vse32.v v26, (a0)
 ; LMULMAX2-RV64-NEXT:    addi sp, s0, -96

 ; LMULMAX1-RV64-NEXT:    addi sp, sp, -48
 ; LMULMAX1-RV64-NEXT:    .cfi_def_cfa_offset 48
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    addi a6, a0, 16
 ; LMULMAX1-RV64-NEXT:    vle32.v v26, (a6)
 ; LMULMAX1-RV64-NEXT:    vle32.v v25, (a0)
 ; LMULMAX1-RV64-NEXT:    vmv.x.s a1, v26
 ; LMULMAX1-RV64-NEXT:    addi a2, zero, 1
-; LMULMAX1-RV64-NEXT:    slli a7, a2, 32
-; LMULMAX1-RV64-NEXT:    or a1, a1, a7
+; LMULMAX1-RV64-NEXT:    slli a2, a2, 32
+; LMULMAX1-RV64-NEXT:    or a1, a1, a2
 ; LMULMAX1-RV64-NEXT:    addi a3, a1, -1
 ; LMULMAX1-RV64-NEXT:    not a1, a1
 ; LMULMAX1-RV64-NEXT:    and a1, a1, a3
 ; LMULMAX1-RV64-NEXT:    srli a4, a1, 1
 ; LMULMAX1-RV64-NEXT:    lui a3, 21845
 ; LMULMAX1-RV64-NEXT:    addiw a3, a3, 1365
 ; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX1-RV64-NEXT:    addi a3, a3, 1365
 ; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
 ; LMULMAX1-RV64-NEXT:    addi a3, a3, 1365
 ; LMULMAX1-RV64-NEXT:    slli a3, a3, 12
-; LMULMAX1-RV64-NEXT:    addi t0, a3, 1365
-; LMULMAX1-RV64-NEXT:    and a4, a4, t0
+; LMULMAX1-RV64-NEXT:    addi a7, a3, 1365
+; LMULMAX1-RV64-NEXT:    and a4, a4, a7
 ; LMULMAX1-RV64-NEXT:    sub a1, a1, a4
 ; LMULMAX1-RV64-NEXT:    lui a4, 13107
 ; LMULMAX1-RV64-NEXT:    addiw a4, a4, 819
 ; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX1-RV64-NEXT:    addi a4, a4, 819
 ; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX1-RV64-NEXT:    addi a4, a4, 819
 ; LMULMAX1-RV64-NEXT:    slli a4, a4, 12
 ; LMULMAX1-RV64-NEXT:    addi a4, a4, 819
 ; LMULMAX1-RV64-NEXT:    and a5, a1, a4
 ; LMULMAX1-RV64-NEXT:    srli a1, a1, 2
 ; LMULMAX1-RV64-NEXT:    and a1, a1, a4
 ; LMULMAX1-RV64-NEXT:    add a1, a5, a1
 ; LMULMAX1-RV64-NEXT:    srli a5, a1, 4
 ; LMULMAX1-RV64-NEXT:    add a1, a1, a5
 ; LMULMAX1-RV64-NEXT:    lui a5, 3855
 ; LMULMAX1-RV64-NEXT:    addiw a5, a5, 241
 ; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
 ; LMULMAX1-RV64-NEXT:    addi a5, a5, -241
 ; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
 ; LMULMAX1-RV64-NEXT:    addi a5, a5, 241
 ; LMULMAX1-RV64-NEXT:    slli a5, a5, 12
-; LMULMAX1-RV64-NEXT:    addi a5, a5, -241
-; LMULMAX1-RV64-NEXT:    and a2, a1, a5
+; LMULMAX1-RV64-NEXT:    addi t0, a5, -241
+; LMULMAX1-RV64-NEXT:    and a3, a1, t0
 ; LMULMAX1-RV64-NEXT:    lui a1, 4112
 ; LMULMAX1-RV64-NEXT:    addiw a1, a1, 257
 ; LMULMAX1-RV64-NEXT:    slli a1, a1, 16
 ; LMULMAX1-RV64-NEXT:    addi a1, a1, 257
 ; LMULMAX1-RV64-NEXT:    slli a1, a1, 16
 ; LMULMAX1-RV64-NEXT:    addi a1, a1, 257
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    sw a2, 32(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    sw a3, 32(sp)
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 1, e32, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v27, v26, 3
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v27
-; LMULMAX1-RV64-NEXT:    or a2, a2, a7
-; LMULMAX1-RV64-NEXT:    addi a3, a2, -1
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v27
+; LMULMAX1-RV64-NEXT:    or a3, a3, a2
+; LMULMAX1-RV64-NEXT:    addi a5, a3, -1
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    and a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    sw a2, 44(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    sw a3, 44(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v27, v26, 2
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v27
-; LMULMAX1-RV64-NEXT:    or a2, a2, a7
-; LMULMAX1-RV64-NEXT:    addi a3, a2, -1
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v27
+; LMULMAX1-RV64-NEXT:    or a3, a3, a2
+; LMULMAX1-RV64-NEXT:    addi a5, a3, -1
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    and a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    sw a2, 40(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    sw a3, 40(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v26, 1
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV64-NEXT:    or a2, a2, a7
-; LMULMAX1-RV64-NEXT:    addi a3, a2, -1
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v26
+; LMULMAX1-RV64-NEXT:    or a3, a3, a2
+; LMULMAX1-RV64-NEXT:    addi a5, a3, -1
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    and a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    sw a2, 36(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    sw a3, 36(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 3
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV64-NEXT:    or a2, a2, a7
-; LMULMAX1-RV64-NEXT:    addi a3, a2, -1
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v26
+; LMULMAX1-RV64-NEXT:    or a3, a3, a2
+; LMULMAX1-RV64-NEXT:    addi a5, a3, -1
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    and a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    sw a2, 28(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    sw a3, 28(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 2
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV64-NEXT:    or a2, a2, a7
-; LMULMAX1-RV64-NEXT:    addi a3, a2, -1
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v26
+; LMULMAX1-RV64-NEXT:    or a3, a3, a2
+; LMULMAX1-RV64-NEXT:    addi a5, a3, -1
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    and a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    sw a2, 24(sp)
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    sw a3, 24(sp)
 ; LMULMAX1-RV64-NEXT:    vslidedown.vi v26, v25, 1
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v26
-; LMULMAX1-RV64-NEXT:    or a2, a2, a7
-; LMULMAX1-RV64-NEXT:    addi a3, a2, -1
-; LMULMAX1-RV64-NEXT:    not a2, a2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a3
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v26
+; LMULMAX1-RV64-NEXT:    or a3, a3, a2
+; LMULMAX1-RV64-NEXT:    addi a5, a3, -1
+; LMULMAX1-RV64-NEXT:    not a3, a3
+; LMULMAX1-RV64-NEXT:    and a3, a3, a5
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 1
+; LMULMAX1-RV64-NEXT:    and a5, a5, a7
+; LMULMAX1-RV64-NEXT:    sub a3, a3, a5
+; LMULMAX1-RV64-NEXT:    and a5, a3, a4
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 2
+; LMULMAX1-RV64-NEXT:    and a3, a3, a4
+; LMULMAX1-RV64-NEXT:    add a3, a5, a3
+; LMULMAX1-RV64-NEXT:    srli a5, a3, 4
+; LMULMAX1-RV64-NEXT:    add a3, a3, a5
 ; LMULMAX1-RV64-NEXT:    and a3, a3, t0
-; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a3, a2, a4
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
-; LMULMAX1-RV64-NEXT:    and a2, a2, a4
-; LMULMAX1-RV64-NEXT:    add a2, a3, a2
-; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
-; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
-; LMULMAX1-RV64-NEXT:    mul a2, a2, a1
-; LMULMAX1-RV64-NEXT:    srli a2, a2, 56
-; LMULMAX1-RV64-NEXT:    sw a2, 20(sp)
-; LMULMAX1-RV64-NEXT:    vmv.x.s a2, v25
-; LMULMAX1-RV64-NEXT:    or a2, a2, a7
+; LMULMAX1-RV64-NEXT:    mul a3, a3, a1
+; LMULMAX1-RV64-NEXT:    srli a3, a3, 56
+; LMULMAX1-RV64-NEXT:    sw a3, 20(sp)
+; LMULMAX1-RV64-NEXT:    vmv.x.s a3, v25
+; LMULMAX1-RV64-NEXT:    or a2, a3, a2
 ; LMULMAX1-RV64-NEXT:    addi a3, a2, -1
 ; LMULMAX1-RV64-NEXT:    not a2, a2
 ; LMULMAX1-RV64-NEXT:    and a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 1
-; LMULMAX1-RV64-NEXT:    and a3, a3, t0
+; LMULMAX1-RV64-NEXT:    and a3, a3, a7
 ; LMULMAX1-RV64-NEXT:    sub a2, a2, a3
 ; LMULMAX1-RV64-NEXT:    and a3, a2, a4
 ; LMULMAX1-RV64-NEXT:    srli a2, a2, 2
 ; LMULMAX1-RV64-NEXT:    and a2, a2, a4
 ; LMULMAX1-RV64-NEXT:    add a2, a3, a2
 ; LMULMAX1-RV64-NEXT:    srli a3, a2, 4
 ; LMULMAX1-RV64-NEXT:    add a2, a2, a3
-; LMULMAX1-RV64-NEXT:    and a2, a2, a5
+; LMULMAX1-RV64-NEXT:    and a2, a2, t0
 ; LMULMAX1-RV64-NEXT:    mul a1, a2, a1
 ; LMULMAX1-RV64-NEXT:    srli a1, a1, 56
 ; LMULMAX1-RV64-NEXT:    sw a1, 16(sp)
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    addi a1, sp, 16
 ; LMULMAX1-RV64-NEXT:    vle32.v v25, (a1)
 ; LMULMAX1-RV64-NEXT:    addi a1, sp, 32
 ; LMULMAX1-RV64-NEXT:    vle32.v v26, (a1)

 ; LMULMAX2-RV64-NEXT:    ret
 ;
 ; LMULMAX1-RV32-LABEL: cttz_v4i64:
 ; LMULMAX1-RV32:       # %bb.0:
 ; LMULMAX1-RV32-NEXT:    addi sp, sp, -48
 ; LMULMAX1-RV32-NEXT:    .cfi_def_cfa_offset 48
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vle64.v v25, (a0)
-; LMULMAX1-RV32-NEXT:    addi a7, a0, 16
-; LMULMAX1-RV32-NEXT:    vle64.v v26, (a7)
+; LMULMAX1-RV32-NEXT:    addi a6, a0, 16
+; LMULMAX1-RV32-NEXT:    vle64.v v26, (a6)
 ; LMULMAX1-RV32-NEXT:    sw zero, 44(sp)
 ; LMULMAX1-RV32-NEXT:    sw zero, 36(sp)
-; LMULMAX1-RV32-NEXT:    addi a6, zero, 32
+; LMULMAX1-RV32-NEXT:    addi a7, zero, 32
 ; LMULMAX1-RV32-NEXT:    lui a1, 349525
 ; LMULMAX1-RV32-NEXT:    addi a5, a1, 1365
 ; LMULMAX1-RV32-NEXT:    lui a1, 209715
 ; LMULMAX1-RV32-NEXT:    addi a4, a1, 819
 ; LMULMAX1-RV32-NEXT:    lui a1, 61681
 ; LMULMAX1-RV32-NEXT:    addi t0, a1, -241
 ; LMULMAX1-RV32-NEXT:    lui a2, 4112
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
 ; LMULMAX1-RV32-NEXT:    addi a3, a2, 257
 ; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_2
 ; LMULMAX1-RV32-NEXT:  # %bb.1:
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
-; LMULMAX1-RV32-NEXT:    vsrl.vx v27, v26, a6
+; LMULMAX1-RV32-NEXT:    vsrl.vx v27, v26, a7
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v27
 ; LMULMAX1-RV32-NEXT:    addi a2, a1, -1
 ; LMULMAX1-RV32-NEXT:    not a1, a1
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a2, a2, a5
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a2, a1, a4

 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_3:
 ; LMULMAX1-RV32-NEXT:    sw a1, 32(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 1, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vslidedown.vi v26, v26, 1
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
 ; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_5
 ; LMULMAX1-RV32-NEXT:  # %bb.4:
-; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v26, a6
+; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v26, a7
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
 ; LMULMAX1-RV32-NEXT:    addi a2, a1, -1
 ; LMULMAX1-RV32-NEXT:    not a1, a1
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a2, a2, a5
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a2, a1, a4

 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_6:
 ; LMULMAX1-RV32-NEXT:    sw a1, 40(sp)
 ; LMULMAX1-RV32-NEXT:    sw zero, 28(sp)
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX1-RV32-NEXT:    sw zero, 20(sp)
 ; LMULMAX1-RV32-NEXT:    bnez a1, .LBB7_8
 ; LMULMAX1-RV32-NEXT:  # %bb.7:
-; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a6
+; LMULMAX1-RV32-NEXT:    vsrl.vx v26, v25, a7
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v26
 ; LMULMAX1-RV32-NEXT:    addi a2, a1, -1
 ; LMULMAX1-RV32-NEXT:    not a1, a1
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a2, a2, a5
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a2, a1, a4

 ; LMULMAX1-RV32-NEXT:    mul a1, a1, a3
 ; LMULMAX1-RV32-NEXT:    srli a1, a1, 24
 ; LMULMAX1-RV32-NEXT:  .LBB7_9:
 ; LMULMAX1-RV32-NEXT:    vslidedown.vi v25, v25, 1
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a2, v25
 ; LMULMAX1-RV32-NEXT:    sw a1, 16(sp)
 ; LMULMAX1-RV32-NEXT:    bnez a2, .LBB7_11
 ; LMULMAX1-RV32-NEXT:  # %bb.10:
-; LMULMAX1-RV32-NEXT:    vsrl.vx v25, v25, a6
+; LMULMAX1-RV32-NEXT:    vsrl.vx v25, v25, a7
 ; LMULMAX1-RV32-NEXT:    vmv.x.s a1, v25
 ; LMULMAX1-RV32-NEXT:    addi a2, a1, -1
 ; LMULMAX1-RV32-NEXT:    not a1, a1
 ; LMULMAX1-RV32-NEXT:    and a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    srli a2, a1, 1
 ; LMULMAX1-RV32-NEXT:    and a2, a2, a5
 ; LMULMAX1-RV32-NEXT:    sub a1, a1, a2
 ; LMULMAX1-RV32-NEXT:    and a2, a1, a4

 ; LMULMAX1-RV32-NEXT:    sw a1, 24(sp)
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 4, e32, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    addi a1, sp, 16
 ; LMULMAX1-RV32-NEXT:    vle32.v v25, (a1)
 ; LMULMAX1-RV32-NEXT:    addi a1, sp, 32
 ; LMULMAX1-RV32-NEXT:    vle32.v v26, (a1)
 ; LMULMAX1-RV32-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV32-NEXT:    vse64.v v25, (a0)
-; LMULMAX1-RV32-NEXT:    vse64.v v26, (a7)
+; LMULMAX1-RV32-NEXT:    vse64.v v26, (a6)
 ; LMULMAX1-RV32-NEXT:    addi sp, sp, 48
 ; LMULMAX1-RV32-NEXT:    ret
 ;
 ; LMULMAX1-RV64-LABEL: cttz_v4i64:
 ; LMULMAX1-RV64:       # %bb.0:
 ; LMULMAX1-RV64-NEXT:    vsetivli zero, 2, e64, m1, ta, mu
 ; LMULMAX1-RV64-NEXT:    addi a6, a0, 16
 ; LMULMAX1-RV64-NEXT:    vle64.v v26, (a6)

llvm/test/CodeGen/RISCV/srem-vector-lkk.ll

 ; RV32I-NEXT:    lw s1, 20(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s0, 24(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw ra, 28(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
 ;
 ; RV32IM-LABEL: fold_srem_vec_2:
 ; RV32IM:       # %bb.0:
-; RV32IM-NEXT:    lh a6, 12(a1)
+; RV32IM-NEXT:    lh a7, 12(a1)
 ; RV32IM-NEXT:    lh a3, 8(a1)
 ; RV32IM-NEXT:    lh a4, 0(a1)
 ; RV32IM-NEXT:    lh a1, 4(a1)
 ; RV32IM-NEXT:    lui a5, 706409
-; RV32IM-NEXT:    addi a5, a5, 389
-; RV32IM-NEXT:    mulh a2, a4, a5
-; RV32IM-NEXT:    add a2, a2, a4
-; RV32IM-NEXT:    srli a7, a2, 31
-; RV32IM-NEXT:    srli a2, a2, 6
-; RV32IM-NEXT:    add a2, a2, a7
-; RV32IM-NEXT:    addi a7, zero, 95
-; RV32IM-NEXT:    mul a2, a2, a7
+; RV32IM-NEXT:    addi a6, a5, 389
+; RV32IM-NEXT:    mulh a5, a4, a6
+; RV32IM-NEXT:    add a5, a5, a4
+; RV32IM-NEXT:    srli a2, a5, 31
+; RV32IM-NEXT:    srli a5, a5, 6
+; RV32IM-NEXT:    add a2, a5, a2
+; RV32IM-NEXT:    addi a5, zero, 95
+; RV32IM-NEXT:    mul a2, a2, a5
 ; RV32IM-NEXT:    sub t0, a4, a2
-; RV32IM-NEXT:    mulh a4, a1, a5
+; RV32IM-NEXT:    mulh a4, a1, a6
 ; RV32IM-NEXT:    add a4, a4, a1
 ; RV32IM-NEXT:    srli a2, a4, 31
 ; RV32IM-NEXT:    srli a4, a4, 6
 ; RV32IM-NEXT:    add a2, a4, a2
-; RV32IM-NEXT:    mul a2, a2, a7
+; RV32IM-NEXT:    mul a2, a2, a5
 ; RV32IM-NEXT:    sub a1, a1, a2
-; RV32IM-NEXT:    mulh a2, a3, a5
+; RV32IM-NEXT:    mulh a2, a3, a6
 ; RV32IM-NEXT:    add a2, a2, a3
 ; RV32IM-NEXT:    srli a4, a2, 31
 ; RV32IM-NEXT:    srli a2, a2, 6
 ; RV32IM-NEXT:    add a2, a2, a4
-; RV32IM-NEXT:    mul a2, a2, a7
+; RV32IM-NEXT:    mul a2, a2, a5
 ; RV32IM-NEXT:    sub a2, a3, a2
-; RV32IM-NEXT:    mulh a3, a6, a5
-; RV32IM-NEXT:    add a3, a3, a6
+; RV32IM-NEXT:    mulh a3, a7, a6
+; RV32IM-NEXT:    add a3, a3, a7
 ; RV32IM-NEXT:    srli a4, a3, 31
 ; RV32IM-NEXT:    srli a3, a3, 6
 ; RV32IM-NEXT:    add a3, a3, a4
-; RV32IM-NEXT:    mul a3, a3, a7
-; RV32IM-NEXT:    sub a3, a6, a3
+; RV32IM-NEXT:    mul a3, a3, a5
+; RV32IM-NEXT:    sub a3, a7, a3
 ; RV32IM-NEXT:    sh a3, 6(a0)
 ; RV32IM-NEXT:    sh a2, 4(a0)
 ; RV32IM-NEXT:    sh a1, 2(a0)
 ; RV32IM-NEXT:    sh t0, 0(a0)
 ; RV32IM-NEXT:    ret
 ;
 ; RV64I-LABEL: fold_srem_vec_2:
 ; RV64I:       # %bb.0:

 ; RV64I-NEXT:    ld s1, 40(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s0, 48(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld ra, 56(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    addi sp, sp, 64
 ; RV64I-NEXT:    ret
 ;
 ; RV64IM-LABEL: fold_srem_vec_2:
 ; RV64IM:       # %bb.0:
-; RV64IM-NEXT:    lh a6, 24(a1)
-; RV64IM-NEXT:    lh a7, 16(a1)
+; RV64IM-NEXT:    lh a7, 24(a1)
+; RV64IM-NEXT:    lh a3, 16(a1)
 ; RV64IM-NEXT:    lh a4, 8(a1)
 ; RV64IM-NEXT:    lh a1, 0(a1)
 ; RV64IM-NEXT:    lui a5, 1045903
 ; RV64IM-NEXT:    addiw a5, a5, -733
 ; RV64IM-NEXT:    slli a5, a5, 15
 ; RV64IM-NEXT:    addi a5, a5, 1035
 ; RV64IM-NEXT:    slli a5, a5, 12
 ; RV64IM-NEXT:    addi a5, a5, -905
 ; RV64IM-NEXT:    slli a5, a5, 12
-; RV64IM-NEXT:    addi a5, a5, -1767
-; RV64IM-NEXT:    mulh a2, a1, a5
-; RV64IM-NEXT:    add a2, a2, a1
-; RV64IM-NEXT:    srli a3, a2, 63
-; RV64IM-NEXT:    srli a2, a2, 6
-; RV64IM-NEXT:    addw a2, a2, a3
-; RV64IM-NEXT:    addi a3, zero, 95
-; RV64IM-NEXT:    mulw a2, a2, a3
+; RV64IM-NEXT:    addi a6, a5, -1767
+; RV64IM-NEXT:    mulh a5, a1, a6
+; RV64IM-NEXT:    add a5, a5, a1
+; RV64IM-NEXT:    srli a2, a5, 63
+; RV64IM-NEXT:    srli a5, a5, 6
+; RV64IM-NEXT:    addw a2, a5, a2
+; RV64IM-NEXT:    addi a5, zero, 95
+; RV64IM-NEXT:    mulw a2, a2, a5
 ; RV64IM-NEXT:    subw t0, a1, a2
-; RV64IM-NEXT:    mulh a2, a4, a5
+; RV64IM-NEXT:    mulh a2, a4, a6
 ; RV64IM-NEXT:    add a2, a2, a4
 ; RV64IM-NEXT:    srli a1, a2, 63
 ; RV64IM-NEXT:    srli a2, a2, 6
 ; RV64IM-NEXT:    addw a1, a2, a1
-; RV64IM-NEXT:    mulw a1, a1, a3
+; RV64IM-NEXT:    mulw a1, a1, a5
 ; RV64IM-NEXT:    subw a1, a4, a1
-; RV64IM-NEXT:    mulh a2, a7, a5
-; RV64IM-NEXT:    add a2, a2, a7
+; RV64IM-NEXT:    mulh a2, a3, a6
+; RV64IM-NEXT:    add a2, a2, a3
 ; RV64IM-NEXT:    srli a4, a2, 63
 ; RV64IM-NEXT:    srli a2, a2, 6
 ; RV64IM-NEXT:    addw a2, a2, a4
-; RV64IM-NEXT:    mulw a2, a2, a3
-; RV64IM-NEXT:    subw a2, a7, a2
-; RV64IM-NEXT:    mulh a4, a6, a5
-; RV64IM-NEXT:    add a4, a4, a6
-; RV64IM-NEXT:    srli a5, a4, 63
-; RV64IM-NEXT:    srli a4, a4, 6
-; RV64IM-NEXT:    addw a4, a4, a5
-; RV64IM-NEXT:    mulw a3, a4, a3
-; RV64IM-NEXT:    subw a3, a6, a3
+; RV64IM-NEXT:    mulw a2, a2, a5
+; RV64IM-NEXT:    subw a2, a3, a2
+; RV64IM-NEXT:    mulh a3, a7, a6
+; RV64IM-NEXT:    add a3, a3, a7
+; RV64IM-NEXT:    srli a4, a3, 63
+; RV64IM-NEXT:    srli a3, a3, 6
+; RV64IM-NEXT:    addw a3, a3, a4
+; RV64IM-NEXT:    mulw a3, a3, a5
+; RV64IM-NEXT:    subw a3, a7, a3
 ; RV64IM-NEXT:    sh a3, 6(a0)
 ; RV64IM-NEXT:    sh a2, 4(a0)
 ; RV64IM-NEXT:    sh a1, 2(a0)
 ; RV64IM-NEXT:    sh t0, 0(a0)
 ; RV64IM-NEXT:    ret
   %1 = srem <4 x i16> %x, <i16 95, i16 95, i16 95, i16 95>
   ret <4 x i16> %1
 }

 ; RV32I-NEXT:    lw s1, 36(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s0, 40(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw ra, 44(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    addi sp, sp, 48
 ; RV32I-NEXT:    ret
 ;
 ; RV32IM-LABEL: combine_srem_sdiv:
 ; RV32IM:       # %bb.0:
-; RV32IM-NEXT:    lh a6, 0(a1)
+; RV32IM-NEXT:    lh a7, 0(a1)
 ; RV32IM-NEXT:    lh a3, 4(a1)
 ; RV32IM-NEXT:    lh a4, 12(a1)
 ; RV32IM-NEXT:    lh a1, 8(a1)
 ; RV32IM-NEXT:    lui a5, 706409
-; RV32IM-NEXT:    addi a5, a5, 389
-; RV32IM-NEXT:    mulh a2, a4, a5
-; RV32IM-NEXT:    add a2, a2, a4
-; RV32IM-NEXT:    srli a7, a2, 31
-; RV32IM-NEXT:    srai a2, a2, 6
-; RV32IM-NEXT:    add t0, a2, a7
-; RV32IM-NEXT:    addi a7, zero, 95
-; RV32IM-NEXT:    mul a2, t0, a7
-; RV32IM-NEXT:    sub t1, a4, a2
-; RV32IM-NEXT:    mulh a4, a1, a5
-; RV32IM-NEXT:    add a4, a4, a1
-; RV32IM-NEXT:    srli a2, a4, 31
-; RV32IM-NEXT:    srai a4, a4, 6
-; RV32IM-NEXT:    add a2, a4, a2
-; RV32IM-NEXT:    mul a4, a2, a7
-; RV32IM-NEXT:    sub t2, a1, a4
-; RV32IM-NEXT:    mulh a4, a3, a5
-; RV32IM-NEXT:    add a4, a4, a3
-; RV32IM-NEXT:    srli a1, a4, 31
-; RV32IM-NEXT:    srai a4, a4, 6
-; RV32IM-NEXT:    add a1, a4, a1
-; RV32IM-NEXT:    mul a4, a1, a7
-; RV32IM-NEXT:    sub a3, a3, a4
-; RV32IM-NEXT:    mulh a4, a6, a5
-; RV32IM-NEXT:    add a4, a4, a6
-; RV32IM-NEXT:    srli a5, a4, 31
-; RV32IM-NEXT:    srai a4, a4, 6
-; RV32IM-NEXT:    add a4, a4, a5
-; RV32IM-NEXT:    mul a5, a4, a7
-; RV32IM-NEXT:    sub a5, a6, a5
+; RV32IM-NEXT:    addi a6, a5, 389
+; RV32IM-NEXT:    mulh a5, a4, a6
+; RV32IM-NEXT:    add a5, a5, a4
+; RV32IM-NEXT:    srli a2, a5, 31
+; RV32IM-NEXT:    srai a5, a5, 6
+; RV32IM-NEXT:    add t3, a5, a2
+; RV32IM-NEXT:    addi t0, zero, 95
+; RV32IM-NEXT:    mul a5, t3, t0
+; RV32IM-NEXT:    sub t1, a4, a5
+; RV32IM-NEXT:    mulh a5, a1, a6
+; RV32IM-NEXT:    add a5, a5, a1
+; RV32IM-NEXT:    srli a4, a5, 31
+; RV32IM-NEXT:    srai a5, a5, 6
 ; RV32IM-NEXT:    add a4, a5, a4
+; RV32IM-NEXT:    mul a5, a4, t0
+; RV32IM-NEXT:    sub t2, a1, a5
+; RV32IM-NEXT:    mulh a5, a3, a6
+; RV32IM-NEXT:    add a5, a5, a3
+; RV32IM-NEXT:    srli a1, a5, 31
+; RV32IM-NEXT:    srai a5, a5, 6
+; RV32IM-NEXT:    add a1, a5, a1
+; RV32IM-NEXT:    mul a5, a1, t0
+; RV32IM-NEXT:    sub a3, a3, a5
+; RV32IM-NEXT:    mulh a5, a7, a6
+; RV32IM-NEXT:    add a5, a5, a7
+; RV32IM-NEXT:    srli a2, a5, 31
+; RV32IM-NEXT:    srai a5, a5, 6
+; RV32IM-NEXT:    add a2, a5, a2
+; RV32IM-NEXT:    mul a5, a2, t0
+; RV32IM-NEXT:    sub a5, a7, a5
+; RV32IM-NEXT:    add a2, a5, a2
 ; RV32IM-NEXT:    add a1, a3, a1
-; RV32IM-NEXT:    add a2, t2, a2
-; RV32IM-NEXT:    add a3, t1, t0
-; RV32IM-NEXT:    sh a3, 6(a0)
-; RV32IM-NEXT:    sh a2, 4(a0)
+; RV32IM-NEXT:    add a3, t2, a4
+; RV32IM-NEXT:    add a4, t1, t3
+; RV32IM-NEXT:    sh a4, 6(a0)
+; RV32IM-NEXT:    sh a3, 4(a0)
 ; RV32IM-NEXT:    sh a1, 2(a0)
-; RV32IM-NEXT:    sh a4, 0(a0)
+; RV32IM-NEXT:    sh a2, 0(a0)
 ; RV32IM-NEXT:    ret
 ;
 ; RV64I-LABEL: combine_srem_sdiv:
 ; RV64I:       # %bb.0:
 ; RV64I-NEXT:    addi sp, sp, -96
 ; RV64I-NEXT:    sd ra, 88(sp) # 8-byte Folded Spill
 ; RV64I-NEXT:    sd s0, 80(sp) # 8-byte Folded Spill
 ; RV64I-NEXT:    sd s1, 72(sp) # 8-byte Folded Spill

 ; RV64I-NEXT:    ld s1, 72(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s0, 80(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld ra, 88(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    addi sp, sp, 96
 ; RV64I-NEXT:    ret
 ;
 ; RV64IM-LABEL: combine_srem_sdiv:
 ; RV64IM:       # %bb.0:
-; RV64IM-NEXT:    lh a6, 0(a1)
-; RV64IM-NEXT:    lh a7, 8(a1)
+; RV64IM-NEXT:    lh a7, 0(a1)
+; RV64IM-NEXT:    lh a3, 8(a1)
 ; RV64IM-NEXT:    lh a4, 16(a1)
 ; RV64IM-NEXT:    lh a1, 24(a1)
 ; RV64IM-NEXT:    lui a5, 1045903
 ; RV64IM-NEXT:    addiw a5, a5, -733
 ; RV64IM-NEXT:    slli a5, a5, 15
 ; RV64IM-NEXT:    addi a5, a5, 1035
 ; RV64IM-NEXT:    slli a5, a5, 12
 ; RV64IM-NEXT:    addi a5, a5, -905
 ; RV64IM-NEXT:    slli a5, a5, 12
-; RV64IM-NEXT:    addi a5, a5, -1767
-; RV64IM-NEXT:    mulh a2, a1, a5
-; RV64IM-NEXT:    add a2, a2, a1
-; RV64IM-NEXT:    srli a3, a2, 63
-; RV64IM-NEXT:    srai a2, a2, 6
-; RV64IM-NEXT:    addw t3, a2, a3
+; RV64IM-NEXT:    addi a6, a5, -1767
+; RV64IM-NEXT:    mulh a5, a1, a6
+; RV64IM-NEXT:    add a5, a5, a1
+; RV64IM-NEXT:    srli a2, a5, 63
+; RV64IM-NEXT:    srai a5, a5, 6
+; RV64IM-NEXT:    addw t3, a5, a2
 ; RV64IM-NEXT:    addi t0, zero, 95
-; RV64IM-NEXT:    mulw a3, t3, t0
-; RV64IM-NEXT:    subw t1, a1, a3
-; RV64IM-NEXT:    mulh a3, a4, a5
-; RV64IM-NEXT:    add a3, a3, a4
-; RV64IM-NEXT:    srli a1, a3, 63
-; RV64IM-NEXT:    srai a3, a3, 6
-; RV64IM-NEXT:    addw a1, a3, a1
-; RV64IM-NEXT:    mulw a3, a1, t0
-; RV64IM-NEXT:    subw t2, a4, a3
-; RV64IM-NEXT:    mulh a4, a7, a5
-; RV64IM-NEXT:    add a4, a4, a7
-; RV64IM-NEXT:    srli a3, a4, 63
-; RV64IM-NEXT:    srai a4, a4, 6
-; RV64IM-NEXT:    addw a3, a4, a3
-; RV64IM-NEXT:    mulw a4, a3, t0
-; RV64IM-NEXT:    subw a4, a7, a4
-; RV64IM-NEXT:    mulh a5, a6, a5
-; RV64IM-NEXT:    add a5, a5, a6
+; RV64IM-NEXT:    mulw a5, t3, t0
+; RV64IM-NEXT:    subw t1, a1, a5
+; RV64IM-NEXT:    mulh a5, a4, a6
+; RV64IM-NEXT:    add a5, a5, a4
+; RV64IM-NEXT:    srli a1, a5, 63
+; RV64IM-NEXT:    srai a5, a5, 6
+; RV64IM-NEXT:    addw a1, a5, a1
+; RV64IM-NEXT:    mulw a5, a1, t0
+; RV64IM-NEXT:    subw t2, a4, a5
+; RV64IM-NEXT:    mulh a5, a3, a6
+; RV64IM-NEXT:    add a5, a5, a3
+; RV64IM-NEXT:    srli a4, a5, 63
+; RV64IM-NEXT:    srai a5, a5, 6
+; RV64IM-NEXT:    addw a4, a5, a4
+; RV64IM-NEXT:    mulw a5, a4, t0
+; RV64IM-NEXT:    subw a3, a3, a5
+; RV64IM-NEXT:    mulh a5, a7, a6
+; RV64IM-NEXT:    add a5, a5, a7
 ; RV64IM-NEXT:    srli a2, a5, 63
 ; RV64IM-NEXT:    srai a5, a5, 6
 ; RV64IM-NEXT:    addw a2, a5, a2
 ; RV64IM-NEXT:    mulw a5, a2, t0
-; RV64IM-NEXT:    subw a5, a6, a5
+; RV64IM-NEXT:    subw a5, a7, a5
 ; RV64IM-NEXT:    addw a2, a5, a2
-; RV64IM-NEXT:    addw a3, a4, a3
+; RV64IM-NEXT:    addw a3, a3, a4
 ; RV64IM-NEXT:    addw a1, t2, a1
 ; RV64IM-NEXT:    addw a4, t1, t3
 ; RV64IM-NEXT:    sh a4, 6(a0)
 ; RV64IM-NEXT:    sh a1, 4(a0)
 ; RV64IM-NEXT:    sh a3, 2(a0)
 ; RV64IM-NEXT:    sh a2, 0(a0)
 ; RV64IM-NEXT:    ret
   %1 = srem <4 x i16> %x, <i16 95, i16 95, i16 95, i16 95>

llvm/test/CodeGen/RISCV/urem-vector-lkk.ll

 ; RV32I-NEXT:    lw s1, 20(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s0, 24(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw ra, 28(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    addi sp, sp, 32
 ; RV32I-NEXT:    ret
 ;
 ; RV32IM-LABEL: fold_urem_vec_2:
 ; RV32IM:       # %bb.0:
-; RV32IM-NEXT:    lhu a6, 12(a1)
-; RV32IM-NEXT:    lhu a7, 8(a1)
+; RV32IM-NEXT:    lhu a7, 12(a1)
+; RV32IM-NEXT:    lhu a3, 8(a1)
 ; RV32IM-NEXT:    lhu a4, 0(a1)
 ; RV32IM-NEXT:    lhu a1, 4(a1)
 ; RV32IM-NEXT:    lui a5, 364242
-; RV32IM-NEXT:    addi a5, a5, 777
-; RV32IM-NEXT:    mulhu a2, a4, a5
-; RV32IM-NEXT:    sub a3, a4, a2
-; RV32IM-NEXT:    srli a3, a3, 1
-; RV32IM-NEXT:    add a2, a3, a2
+; RV32IM-NEXT:    addi a6, a5, 777
+; RV32IM-NEXT:    mulhu a5, a4, a6
+; RV32IM-NEXT:    sub a2, a4, a5
+; RV32IM-NEXT:    srli a2, a2, 1
+; RV32IM-NEXT:    add a2, a2, a5
 ; RV32IM-NEXT:    srli a2, a2, 6
-; RV32IM-NEXT:    addi a3, zero, 95
-; RV32IM-NEXT:    mul a2, a2, a3
+; RV32IM-NEXT:    addi a5, zero, 95
+; RV32IM-NEXT:    mul a2, a2, a5
 ; RV32IM-NEXT:    sub t0, a4, a2
-; RV32IM-NEXT:    mulhu a4, a1, a5
+; RV32IM-NEXT:    mulhu a4, a1, a6
 ; RV32IM-NEXT:    sub a2, a1, a4
 ; RV32IM-NEXT:    srli a2, a2, 1
 ; RV32IM-NEXT:    add a2, a2, a4
 ; RV32IM-NEXT:    srli a2, a2, 6
-; RV32IM-NEXT:    mul a2, a2, a3
+; RV32IM-NEXT:    mul a2, a2, a5
 ; RV32IM-NEXT:    sub a1, a1, a2
-; RV32IM-NEXT:    mulhu a2, a7, a5
-; RV32IM-NEXT:    sub a4, a7, a2
+; RV32IM-NEXT:    mulhu a2, a3, a6
+; RV32IM-NEXT:    sub a4, a3, a2
 ; RV32IM-NEXT:    srli a4, a4, 1
 ; RV32IM-NEXT:    add a2, a4, a2
 ; RV32IM-NEXT:    srli a2, a2, 6
-; RV32IM-NEXT:    mul a2, a2, a3
-; RV32IM-NEXT:    sub a2, a7, a2
-; RV32IM-NEXT:    mulhu a4, a6, a5
-; RV32IM-NEXT:    sub a5, a6, a4
-; RV32IM-NEXT:    srli a5, a5, 1
-; RV32IM-NEXT:    add a4, a5, a4
-; RV32IM-NEXT:    srli a4, a4, 6
-; RV32IM-NEXT:    mul a3, a4, a3
-; RV32IM-NEXT:    sub a3, a6, a3
+; RV32IM-NEXT:    mul a2, a2, a5
+; RV32IM-NEXT:    sub a2, a3, a2
+; RV32IM-NEXT:    mulhu a3, a7, a6
+; RV32IM-NEXT:    sub a4, a7, a3
+; RV32IM-NEXT:    srli a4, a4, 1
+; RV32IM-NEXT:    add a3, a4, a3
+; RV32IM-NEXT:    srli a3, a3, 6
+; RV32IM-NEXT:    mul a3, a3, a5
+; RV32IM-NEXT:    sub a3, a7, a3
 ; RV32IM-NEXT:    sh a3, 6(a0)
 ; RV32IM-NEXT:    sh a2, 4(a0)
 ; RV32IM-NEXT:    sh a1, 2(a0)
 ; RV32IM-NEXT:    sh t0, 0(a0)
 ; RV32IM-NEXT:    ret
 ;
 ; RV64I-LABEL: fold_urem_vec_2:
 ; RV64I:       # %bb.0:

 ; RV64I-NEXT:    ld s1, 40(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s0, 48(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld ra, 56(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    addi sp, sp, 64
 ; RV64I-NEXT:    ret
 ;
 ; RV64IM-LABEL: fold_urem_vec_2:
 ; RV64IM:       # %bb.0:
-; RV64IM-NEXT:    lhu a6, 24(a1)
-; RV64IM-NEXT:    lhu a7, 16(a1)
+; RV64IM-NEXT:    lhu a7, 24(a1)
+; RV64IM-NEXT:    lhu a3, 16(a1)
 ; RV64IM-NEXT:    lhu a4, 8(a1)
 ; RV64IM-NEXT:    lhu a1, 0(a1)
 ; RV64IM-NEXT:    lui a5, 1423
 ; RV64IM-NEXT:    addiw a5, a5, -733
 ; RV64IM-NEXT:    slli a5, a5, 15
 ; RV64IM-NEXT:    addi a5, a5, 1035
 ; RV64IM-NEXT:    slli a5, a5, 13
 ; RV64IM-NEXT:    addi a5, a5, -1811
 ; RV64IM-NEXT:    slli a5, a5, 12
-; RV64IM-NEXT:    addi a5, a5, 561
-; RV64IM-NEXT:    mulhu a2, a1, a5
-; RV64IM-NEXT:    sub a3, a1, a2
-; RV64IM-NEXT:    srli a3, a3, 1
-; RV64IM-NEXT:    add a2, a3, a2
+; RV64IM-NEXT:    addi a6, a5, 561
+; RV64IM-NEXT:    mulhu a5, a1, a6
+; RV64IM-NEXT:    sub a2, a1, a5
+; RV64IM-NEXT:    srli a2, a2, 1
+; RV64IM-NEXT:    add a2, a2, a5
 ; RV64IM-NEXT:    srli a2, a2, 6
-; RV64IM-NEXT:    addi a3, zero, 95
-; RV64IM-NEXT:    mulw a2, a2, a3
+; RV64IM-NEXT:    addi a5, zero, 95
+; RV64IM-NEXT:    mulw a2, a2, a5
 ; RV64IM-NEXT:    subw t0, a1, a2
-; RV64IM-NEXT:    mulhu a2, a4, a5
+; RV64IM-NEXT:    mulhu a2, a4, a6
 ; RV64IM-NEXT:    sub a1, a4, a2
 ; RV64IM-NEXT:    srli a1, a1, 1
 ; RV64IM-NEXT:    add a1, a1, a2
 ; RV64IM-NEXT:    srli a1, a1, 6
-; RV64IM-NEXT:    mulw a1, a1, a3
+; RV64IM-NEXT:    mulw a1, a1, a5
 ; RV64IM-NEXT:    subw a1, a4, a1
-; RV64IM-NEXT:    mulhu a2, a7, a5
-; RV64IM-NEXT:    sub a4, a7, a2
+; RV64IM-NEXT:    mulhu a2, a3, a6
+; RV64IM-NEXT:    sub a4, a3, a2
 ; RV64IM-NEXT:    srli a4, a4, 1
 ; RV64IM-NEXT:    add a2, a4, a2
 ; RV64IM-NEXT:    srli a2, a2, 6
-; RV64IM-NEXT:    mulw a2, a2, a3
-; RV64IM-NEXT:    subw a2, a7, a2
-; RV64IM-NEXT:    mulhu a4, a6, a5
-; RV64IM-NEXT:    sub a5, a6, a4
-; RV64IM-NEXT:    srli a5, a5, 1
-; RV64IM-NEXT:    add a4, a5, a4
-; RV64IM-NEXT:    srli a4, a4, 6
-; RV64IM-NEXT:    mulw a3, a4, a3
-; RV64IM-NEXT:    subw a3, a6, a3
+; RV64IM-NEXT:    mulw a2, a2, a5
+; RV64IM-NEXT:    subw a2, a3, a2
+; RV64IM-NEXT:    mulhu a3, a7, a6
+; RV64IM-NEXT:    sub a4, a7, a3
+; RV64IM-NEXT:    srli a4, a4, 1
+; RV64IM-NEXT:    add a3, a4, a3
+; RV64IM-NEXT:    srli a3, a3, 6
+; RV64IM-NEXT:    mulw a3, a3, a5
+; RV64IM-NEXT:    subw a3, a7, a3
 ; RV64IM-NEXT:    sh a3, 6(a0)
 ; RV64IM-NEXT:    sh a2, 4(a0)
 ; RV64IM-NEXT:    sh a1, 2(a0)
 ; RV64IM-NEXT:    sh t0, 0(a0)
 ; RV64IM-NEXT:    ret
   %1 = urem <4 x i16> %x, <i16 95, i16 95, i16 95, i16 95>
   ret <4 x i16> %1
 }

 ; RV32I-NEXT:    lw s1, 36(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw s0, 40(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    lw ra, 44(sp) # 4-byte Folded Reload
 ; RV32I-NEXT:    addi sp, sp, 48
 ; RV32I-NEXT:    ret
 ;
 ; RV32IM-LABEL: combine_urem_udiv:
 ; RV32IM:       # %bb.0:
-; RV32IM-NEXT:    lhu a6, 0(a1)
-; RV32IM-NEXT:    lhu a7, 4(a1)
+; RV32IM-NEXT:    lhu a7, 0(a1)
+; RV32IM-NEXT:    lhu a3, 4(a1)
 ; RV32IM-NEXT:    lhu a4, 12(a1)
 ; RV32IM-NEXT:    lhu a1, 8(a1)
 ; RV32IM-NEXT:    lui a5, 364242
-; RV32IM-NEXT:    addi a5, a5, 777
-; RV32IM-NEXT:    mulhu a2, a4, a5
-; RV32IM-NEXT:    sub a3, a4, a2
-; RV32IM-NEXT:    srli a3, a3, 1
-; RV32IM-NEXT:    add a2, a3, a2
+; RV32IM-NEXT:    addi a6, a5, 777
+; RV32IM-NEXT:    mulhu a5, a4, a6
+; RV32IM-NEXT:    sub a2, a4, a5
+; RV32IM-NEXT:    srli a2, a2, 1
+; RV32IM-NEXT:    add a2, a2, a5
 ; RV32IM-NEXT:    srli t3, a2, 6
 ; RV32IM-NEXT:    addi t0, zero, 95
-; RV32IM-NEXT:    mul a3, t3, t0
-; RV32IM-NEXT:    sub t1, a4, a3
-; RV32IM-NEXT:    mulhu a4, a1, a5
-; RV32IM-NEXT:    sub a3, a1, a4
-; RV32IM-NEXT:    srli a3, a3, 1
-; RV32IM-NEXT:    add a3, a3, a4
-; RV32IM-NEXT:    srli a3, a3, 6
-; RV32IM-NEXT:    mul a4, a3, t0
-; RV32IM-NEXT:    sub t2, a1, a4
-; RV32IM-NEXT:    mulhu a4, a7, a5
-; RV32IM-NEXT:    sub a1, a7, a4
+; RV32IM-NEXT:    mul a5, t3, t0
+; RV32IM-NEXT:    sub t1, a4, a5
+; RV32IM-NEXT:    mulhu a5, a1, a6
+; RV32IM-NEXT:    sub a4, a1, a5
+; RV32IM-NEXT:    srli a4, a4, 1
+; RV32IM-NEXT:    add a4, a4, a5
+; RV32IM-NEXT:    srli a4, a4, 6
+; RV32IM-NEXT:    mul a5, a4, t0
+; RV32IM-NEXT:    sub t2, a1, a5
+; RV32IM-NEXT:    mulhu a5, a3, a6
+; RV32IM-NEXT:    sub a1, a3, a5
 ; RV32IM-NEXT:    srli a1, a1, 1
-; RV32IM-NEXT:    add a1, a1, a4
+; RV32IM-NEXT:    add a1, a1, a5
 ; RV32IM-NEXT:    srli a1, a1, 6
-; RV32IM-NEXT:    mul a4, a1, t0
-; RV32IM-NEXT:    sub a4, a7, a4
-; RV32IM-NEXT:    mulhu a5, a6, a5
-; RV32IM-NEXT:    sub a2, a6, a5
+; RV32IM-NEXT:    mul a5, a1, t0
+; RV32IM-NEXT:    sub a3, a3, a5
+; RV32IM-NEXT:    mulhu a5, a7, a6
+; RV32IM-NEXT:    sub a2, a7, a5
 ; RV32IM-NEXT:    srli a2, a2, 1
 ; RV32IM-NEXT:    add a2, a2, a5
 ; RV32IM-NEXT:    srli a2, a2, 6
 ; RV32IM-NEXT:    mul a5, a2, t0
-; RV32IM-NEXT:    sub a5, a6, a5
+; RV32IM-NEXT:    sub a5, a7, a5
 ; RV32IM-NEXT:    add a2, a5, a2
-; RV32IM-NEXT:    add a1, a4, a1
-; RV32IM-NEXT:    add a3, t2, a3
+; RV32IM-NEXT:    add a1, a3, a1
+; RV32IM-NEXT:    add a3, t2, a4
 ; RV32IM-NEXT:    add a4, t1, t3
 ; RV32IM-NEXT:    sh a4, 6(a0)
 ; RV32IM-NEXT:    sh a3, 4(a0)
 ; RV32IM-NEXT:    sh a1, 2(a0)
 ; RV32IM-NEXT:    sh a2, 0(a0)
 ; RV32IM-NEXT:    ret
 ;
 ; RV64I-LABEL: combine_urem_udiv:

 ; RV64I-NEXT:    ld s1, 72(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld s0, 80(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    ld ra, 88(sp) # 8-byte Folded Reload
 ; RV64I-NEXT:    addi sp, sp, 96
 ; RV64I-NEXT:    ret
 ;
 ; RV64IM-LABEL: combine_urem_udiv:
 ; RV64IM:       # %bb.0:
-; RV64IM-NEXT:    lhu a6, 0(a1)
-; RV64IM-NEXT:    lhu a7, 8(a1)
+; RV64IM-NEXT:    lhu a7, 0(a1)
+; RV64IM-NEXT:    lhu a3, 8(a1)
 ; RV64IM-NEXT:    lhu a4, 16(a1)
 ; RV64IM-NEXT:    lhu a1, 24(a1)
 ; RV64IM-NEXT:    lui a5, 1423
 ; RV64IM-NEXT:    addiw a5, a5, -733
 ; RV64IM-NEXT:    slli a5, a5, 15
 ; RV64IM-NEXT:    addi a5, a5, 1035
 ; RV64IM-NEXT:    slli a5, a5, 13
 ; RV64IM-NEXT:    addi a5, a5, -1811
 ; RV64IM-NEXT:    slli a5, a5, 12
-; RV64IM-NEXT:    addi a5, a5, 561
-; RV64IM-NEXT:    mulhu a2, a1, a5
-; RV64IM-NEXT:    sub a3, a1, a2
-; RV64IM-NEXT:    srli a3, a3, 1
-; RV64IM-NEXT:    add a2, a3, a2
+; RV64IM-NEXT:    addi a6, a5, 561
+; RV64IM-NEXT:    mulhu a5, a1, a6
+; RV64IM-NEXT:    sub a2, a1, a5
+; RV64IM-NEXT:    srli a2, a2, 1
+; RV64IM-NEXT:    add a2, a2, a5
 ; RV64IM-NEXT:    srli t3, a2, 6
 ; RV64IM-NEXT:    addi t0, zero, 95
-; RV64IM-NEXT:    mulw a3, t3, t0
-; RV64IM-NEXT:    subw t1, a1, a3
-; RV64IM-NEXT:    mulhu a3, a4, a5
-; RV64IM-NEXT:    sub a1, a4, a3
+; RV64IM-NEXT:    mulw a5, t3, t0
+; RV64IM-NEXT:    subw t1, a1, a5
+; RV64IM-NEXT:    mulhu a5, a4, a6
+; RV64IM-NEXT:    sub a1, a4, a5
 ; RV64IM-NEXT:    srli a1, a1, 1
-; RV64IM-NEXT:    add a1, a1, a3
+; RV64IM-NEXT:    add a1, a1, a5
 ; RV64IM-NEXT:    srli a1, a1, 6
-; RV64IM-NEXT:    mulw a3, a1, t0
-; RV64IM-NEXT:    subw t2, a4, a3
-; RV64IM-NEXT:    mulhu a4, a7, a5
-; RV64IM-NEXT:    sub a3, a7, a4
-; RV64IM-NEXT:    srli a3, a3, 1
-; RV64IM-NEXT:    add a3, a3, a4
-; RV64IM-NEXT:    srli a3, a3, 6
-; RV64IM-NEXT:    mulw a4, a3, t0
-; RV64IM-NEXT:    subw a4, a7, a4
-; RV64IM-NEXT:    mulhu a5, a6, a5
-; RV64IM-NEXT:    sub a2, a6, a5
+; RV64IM-NEXT:    mulw a5, a1, t0
+; RV64IM-NEXT:    subw t2, a4, a5
+; RV64IM-NEXT:    mulhu a5, a3, a6
+; RV64IM-NEXT:    sub a4, a3, a5
+; RV64IM-NEXT:    srli a4, a4, 1
+; RV64IM-NEXT:    add a4, a4, a5
+; RV64IM-NEXT:    srli a4, a4, 6
+; RV64IM-NEXT:    mulw a5, a4, t0
+; RV64IM-NEXT:    subw a3, a3, a5
+; RV64IM-NEXT:    mulhu a5, a7, a6
+; RV64IM-NEXT:    sub a2, a7, a5
 ; RV64IM-NEXT:    srli a2, a2, 1
 ; RV64IM-NEXT:    add a2, a2, a5
 ; RV64IM-NEXT:    srli a2, a2, 6
 ; RV64IM-NEXT:    mulw a5, a2, t0
-; RV64IM-NEXT:    subw a5, a6, a5
+; RV64IM-NEXT:    subw a5, a7, a5
 ; RV64IM-NEXT:    addw a2, a5, a2
-; RV64IM-NEXT:    addw a3, a4, a3
+; RV64IM-NEXT:    addw a3, a3, a4
 ; RV64IM-NEXT:    addw a1, t2, a1
 ; RV64IM-NEXT:    addw a4, t1, t3
 ; RV64IM-NEXT:    sh a4, 6(a0)
 ; RV64IM-NEXT:    sh a1, 4(a0)
 ; RV64IM-NEXT:    sh a3, 2(a0)
 ; RV64IM-NEXT:    sh a2, 0(a0)
 ; RV64IM-NEXT:    ret
   %1 = urem <4 x i16> %x, <i16 95, i16 95, i16 95, i16 95>

llvm/test/CodeGen/Thumb/dyn-stackalloc.ll

-; RUN: llc < %s -mtriple=thumb-apple-darwin -disable-cgp-branch-opts -disable-post-ra -verify-machineinstrs | FileCheck %s -check-prefix=CHECK -check-prefix=RA_GREEDY
-; RUN: llc < %s -mtriple=thumb-apple-darwin -disable-cgp-branch-opts -disable-post-ra -regalloc=basic -verify-machineinstrs | FileCheck %s -check-prefix=CHECK -check-prefix=RA_BASIC
+; RUN: llc < %s -mtriple=thumb-apple-darwin -disable-cgp-branch-opts -disable-post-ra -verify-machineinstrs | FileCheck %s -check-prefix=CHECK
+; RUN: llc < %s -mtriple=thumb-apple-darwin -disable-cgp-branch-opts -disable-post-ra -regalloc=basic -verify-machineinstrs | FileCheck %s -check-prefix=CHECK
 
   %struct.state = type { i32, %struct.info*, float**, i32, i32, i32, i32, i32, i32, i32, i32, i32, i64, i64, i64, i64, i64, i64, i8* }
   %struct.info = type { i32, i32, i32, i32, i32, i32, i32, i8* }
 
 define void @t1(%struct.state* %v) {
 ; CHECK-LABEL: t1:
 ; CHECK: push
 ; CHECK: add r7, sp, #12

 
 define void @t2(%struct.comment* %vc, i8* %tag, i8* %contents) {
 ; CHECK-LABEL: t2:
 ; CHECK: push
 ; CHECK: add r7, sp, #12
 ; CHECK: sub sp, #
 ; CHECK: mov r[[R0:[0-9]+]], sp
 ; CHECK: str r{{[0-9+]}}, [r[[R0]]
-; RA_GREEDY: str r{{[0-9+]}}, [r[[R0]]
-; RA_BASIC: stm r[[R0]]!
+; CHECK: str r{{[0-9+]}}, [r[[R0]]
 ; CHECK-NOT: ldr r0, [sp
 ; CHECK: mov r[[R1:[0-9]+]], sp
 ; CHECK: subs r[[R2:[0-9]+]], r[[R1]], r{{[0-9]+}}
 ; CHECK: mov sp, r[[R2]]
 ; CHECK-NOT: ldr r0, [sp
 ; CHECK: bx
   %tmp1 = call i32 @strlen( i8* %tag )
   %tmp3 = call i32 @strlen( i8* %contents )

llvm/test/CodeGen/Thumb2/LowOverheadLoops/tail-pred-disabled-in-loloops.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ;
 ; RUN: llc -mtriple=thumbv8.1m.main -mattr=+mve -tail-predication=enabled \
 ; RUN:     %s -o - --verify-machineinstrs | FileCheck %s --check-prefix=ENABLED
 ;
 ; RUN: llc -mtriple=thumbv8.1m.main -mattr=+mve -tail-predication=enabled \
 ; RUN:     -arm-loloops-disable-tailpred %s -o - --verify-machineinstrs | \
 ; RUN:     FileCheck %s --check-prefix=DISABLED
 
 define dso_local void @check_option(i32* noalias nocapture %A, i32* noalias nocapture readonly %B, i32* noalias nocapture readonly %C, i32 %N) local_unnamed_addr #0 {
 ; ENABLED-LABEL: check_option:
 ; ENABLED:       @ %bb.0: @ %entry
-; ENABLED-NEXT:    push.w {r4, r5, r6, r7, r8, lr}
+; ENABLED-NEXT:    push.w {r4, r5, r6, r7, r8, r9, lr}
 ; ENABLED-NEXT:    cmp r3, #1
 ; ENABLED-NEXT:    blt .LBB0_4
 ; ENABLED-NEXT:  @ %bb.1: @ %vector.ph.preheader
 ; ENABLED-NEXT:  .LBB0_2: @ %vector.ph
 ; ENABLED-NEXT:    @ =>This Loop Header: Depth=1
 ; ENABLED-NEXT:    @ Child Loop BB0_3 Depth 2
 ; ENABLED-NEXT:    mov r12, r0
 ; ENABLED-NEXT:    mov r4, r2
 ; ENABLED-NEXT:    mov r5, r1
 ; ENABLED-NEXT:    mov r6, r3
 ; ENABLED-NEXT:    dlstp.32 lr, r6
 ; ENABLED-NEXT:  .LBB0_3: @ %vector.body
 ; ENABLED-NEXT:    @ Parent Loop BB0_2 Depth=1
 ; ENABLED-NEXT:    @ => This Inner Loop Header: Depth=2
 ; ENABLED-NEXT:    vldrw.u32 q0, [r5], #16
 ; ENABLED-NEXT:    vldrw.u32 q1, [r4], #16
 ; ENABLED-NEXT:    vadd.i32 q0, q1, q0
 ; ENABLED-NEXT:    vstrw.32 q0, [r12], #16
 ; ENABLED-NEXT:    letp lr, .LBB0_3
 ; ENABLED-NEXT:    b .LBB0_2
 ; ENABLED-NEXT:  .LBB0_4: @ %for.cond.cleanup
-; ENABLED-NEXT:    pop.w {r4, r5, r6, r7, r8, pc}
+; ENABLED-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, pc}
 ;
 ; DISABLED-LABEL: check_option:
 ; DISABLED:       @ %bb.0: @ %entry
-; DISABLED-NEXT:    push.w {r4, r5, r6, r7, r8, lr}
+; DISABLED-NEXT:    push.w {r4, r5, r6, r7, r8, r9, lr}
 ; DISABLED-NEXT:    cmp r3, #1
 ; DISABLED-NEXT:    blt .LBB0_4
 ; DISABLED-NEXT:  @ %bb.1: @ %vector.ph.preheader
 ; DISABLED-NEXT:    adds r7, r3, #3
 ; DISABLED-NEXT:    movs r6, #1
 ; DISABLED-NEXT:    bic r7, r7, #3
 ; DISABLED-NEXT:    subs r7, #4
 ; DISABLED-NEXT:    add.w r8, r6, r7, lsr #2
 ; DISABLED-NEXT:  .LBB0_2: @ %vector.ph
 ; DISABLED-NEXT:    @ =>This Loop Header: Depth=1
 ; DISABLED-NEXT:    @ Child Loop BB0_3 Depth 2
-; DISABLED-NEXT:    mov r7, r8
+; DISABLED-NEXT:    mov r9, r8
 ; DISABLED-NEXT:    mov r12, r0
 ; DISABLED-NEXT:    mov r4, r2
 ; DISABLED-NEXT:    mov r5, r1
 ; DISABLED-NEXT:    mov r6, r3
 ; DISABLED-NEXT:    dls lr, r8
 ; DISABLED-NEXT:  .LBB0_3: @ %vector.body
 ; DISABLED-NEXT:    @ Parent Loop BB0_2 Depth=1
 ; DISABLED-NEXT:    @ => This Inner Loop Header: Depth=2
-; DISABLED-NEXT:    mov lr, r7
+; DISABLED-NEXT:    mov lr, r9
 ; DISABLED-NEXT:    vctp.32 r6
-; DISABLED-NEXT:    subs r7, #1
+; DISABLED-NEXT:    sub.w r9, r9, #1
 ; DISABLED-NEXT:    subs r6, #4
 ; DISABLED-NEXT:    vpstt
 ; DISABLED-NEXT:    vldrwt.u32 q0, [r5], #16
 ; DISABLED-NEXT:    vldrwt.u32 q1, [r4], #16
 ; DISABLED-NEXT:    vadd.i32 q0, q1, q0
 ; DISABLED-NEXT:    vpst
 ; DISABLED-NEXT:    vstrwt.32 q0, [r12], #16
 ; DISABLED-NEXT:    le lr, .LBB0_3
 ; DISABLED-NEXT:    b .LBB0_2
 ; DISABLED-NEXT:  .LBB0_4: @ %for.cond.cleanup
-; DISABLED-NEXT:    pop.w {r4, r5, r6, r7, r8, pc}
+; DISABLED-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, pc}
 entry:
   %cmp8 = icmp sgt i32 %N, 0
   %0 = add i32 %N, 3
   %1 = lshr i32 %0, 2
   %2 = shl nuw i32 %1, 2
   %3 = add i32 %2, -4
   %4 = lshr i32 %3, 2
   %5 = add nuw nsw i32 %4, 1

llvm/test/CodeGen/Thumb2/LowOverheadLoops/varying-outer-2d-reduction.ll

 ; RUN:   FileCheck %s --check-prefix=NOREDUCTIONS
 ;
 ; RUN: llc -mtriple=thumbv8.1m.main -mattr=+mve -tail-predication=force-enabled-no-reductions %s -o - | \
 ; RUN:   FileCheck %s --check-prefix=NOREDUCTIONS
 
 define dso_local void @varying_outer_2d_reduction(i16* nocapture readonly %Input, i16* nocapture %Output, i16 signext %Size, i16 signext %N, i16 signext %Scale) local_unnamed_addr {
 ; ENABLED-LABEL: varying_outer_2d_reduction:
 ; ENABLED:       @ %bb.0: @ %entry
-; ENABLED-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, lr}
-; ENABLED-NEXT:    sub sp, #4
+; ENABLED-NEXT:    push.w {r4, r5, r6, r7, r9, r10, r11, lr}
 ; ENABLED-NEXT:    cmp r3, #1
-; ENABLED-NEXT:    str r0, [sp] @ 4-byte Spill
 ; ENABLED-NEXT:    blt .LBB0_8
 ; ENABLED-NEXT:  @ %bb.1: @ %for.body.lr.ph
-; ENABLED-NEXT:    ldr r0, [sp, #36]
-; ENABLED-NEXT:    add.w r12, r2, #3
-; ENABLED-NEXT:    ldr.w r10, [sp] @ 4-byte Reload
-; ENABLED-NEXT:    mov.w r8, #0
-; ENABLED-NEXT:    mov r9, r12
+; ENABLED-NEXT:    mov r11, r0
+; ENABLED-NEXT:    ldr r0, [sp, #32]
+; ENABLED-NEXT:    add.w r9, r2, #3
+; ENABLED-NEXT:    mov.w r12, #0
+; ENABLED-NEXT:    mov r10, r11
 ; ENABLED-NEXT:    uxth r0, r0
 ; ENABLED-NEXT:    rsbs r5, r0, #0
 ; ENABLED-NEXT:    b .LBB0_4
 ; ENABLED-NEXT:  .LBB0_2: @ in Loop: Header=BB0_4 Depth=1
 ; ENABLED-NEXT:    movs r0, #0
 ; ENABLED-NEXT:  .LBB0_3: @ %for.end
 ; ENABLED-NEXT:    @ in Loop: Header=BB0_4 Depth=1
 ; ENABLED-NEXT:    lsrs r0, r0, #16
 ; ENABLED-NEXT:    sub.w r9, r9, #1
-; ENABLED-NEXT:    strh.w r0, [r1, r8, lsl #1]
-; ENABLED-NEXT:    add.w r8, r8, #1
+; ENABLED-NEXT:    strh.w r0, [r1, r12, lsl #1]
+; ENABLED-NEXT:    add.w r12, r12, #1
 ; ENABLED-NEXT:    add.w r10, r10, #2
-; ENABLED-NEXT:    cmp r8, r3
+; ENABLED-NEXT:    cmp r12, r3
 ; ENABLED-NEXT:    beq .LBB0_8
 ; ENABLED-NEXT:  .LBB0_4: @ %for.body
 ; ENABLED-NEXT:    @ =>This Loop Header: Depth=1
 ; ENABLED-NEXT:    @ Child Loop BB0_6 Depth 2
-; ENABLED-NEXT:    cmp r2, r8
+; ENABLED-NEXT:    cmp r2, r12
 ; ENABLED-NEXT:    ble .LBB0_2
 ; ENABLED-NEXT:  @ %bb.5: @ %vector.ph
 ; ENABLED-NEXT:    @ in Loop: Header=BB0_4 Depth=1
 ; ENABLED-NEXT:    bic r0, r9, #3
 ; ENABLED-NEXT:    movs r7, #1
 ; ENABLED-NEXT:    subs r0, #4
-; ENABLED-NEXT:    sub.w r4, r2, r8
+; ENABLED-NEXT:    sub.w r4, r2, r12
 ; ENABLED-NEXT:    vmov.i32 q1, #0x0
 ; ENABLED-NEXT:    add.w r6, r7, r0, lsr #2
-; ENABLED-NEXT:    sub.w r0, r12, r8
+; ENABLED-NEXT:    adds r0, r2, #3
+; ENABLED-NEXT:    sub.w r0, r0, r12
 ; ENABLED-NEXT:    bic r0, r0, #3
 ; ENABLED-NEXT:    subs r0, #4
 ; ENABLED-NEXT:    add.w r0, r7, r0, lsr #2
 ; ENABLED-NEXT:    mov r7, r10
 ; ENABLED-NEXT:    dls lr, r0
-; ENABLED-NEXT:    ldr r0, [sp] @ 4-byte Reload
+; ENABLED-NEXT:    mov r0, r11
 ; ENABLED-NEXT:  .LBB0_6: @ %vector.body
 ; ENABLED-NEXT:    @ Parent Loop BB0_4 Depth=1
 ; ENABLED-NEXT:    @ => This Inner Loop Header: Depth=2
 ; ENABLED-NEXT:    vctp.32 r4
 ; ENABLED-NEXT:    vmov q0, q1
 ; ENABLED-NEXT:    vpstt
 ; ENABLED-NEXT:    vldrht.s32 q1, [r0], #8
 ; ENABLED-NEXT:    vldrht.s32 q2, [r7], #8
 ; ENABLED-NEXT:    mov lr, r6
 ; ENABLED-NEXT:    vmul.i32 q1, q2, q1
 ; ENABLED-NEXT:    subs r6, #1
 ; ENABLED-NEXT:    vshl.s32 q1, r5
 ; ENABLED-NEXT:    subs r4, #4
 ; ENABLED-NEXT:    vadd.i32 q1, q1, q0
 ; ENABLED-NEXT:    le lr, .LBB0_6
 ; ENABLED-NEXT:  @ %bb.7: @ %middle.block
 ; ENABLED-NEXT:    @ in Loop: Header=BB0_4 Depth=1
 ; ENABLED-NEXT:    vpsel q0, q1, q0
 ; ENABLED-NEXT:    vaddv.u32 r0, q0
 ; ENABLED-NEXT:    b .LBB0_3
 ; ENABLED-NEXT:  .LBB0_8: @ %for.end17
-; ENABLED-NEXT:    add sp, #4
-; ENABLED-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, pc}
+; ENABLED-NEXT:    pop.w {r4, r5, r6, r7, r9, r10, r11, pc}
 ;
 ; NOREDUCTIONS-LABEL: varying_outer_2d_reduction:
 ; NOREDUCTIONS:       @ %bb.0: @ %entry
-; NOREDUCTIONS-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, lr}
-; NOREDUCTIONS-NEXT:    sub sp, #4
+; NOREDUCTIONS-NEXT:    push.w {r4, r5, r6, r7, r9, r10, r11, lr}
 ; NOREDUCTIONS-NEXT:    cmp r3, #1
-; NOREDUCTIONS-NEXT:    str r0, [sp] @ 4-byte Spill
 ; NOREDUCTIONS-NEXT:    blt .LBB0_8
 ; NOREDUCTIONS-NEXT:  @ %bb.1: @ %for.body.lr.ph
-; NOREDUCTIONS-NEXT:    ldr r0, [sp, #36]
-; NOREDUCTIONS-NEXT:    add.w r12, r2, #3
-; NOREDUCTIONS-NEXT:    ldr.w r10, [sp] @ 4-byte Reload
-; NOREDUCTIONS-NEXT:    mov.w r8, #0
-; NOREDUCTIONS-NEXT:    mov r9, r12
+; NOREDUCTIONS-NEXT:    mov r11, r0
+; NOREDUCTIONS-NEXT:    ldr r0, [sp, #32]
+; NOREDUCTIONS-NEXT:    add.w r9, r2, #3
+; NOREDUCTIONS-NEXT:    mov.w r12, #0
+; NOREDUCTIONS-NEXT:    mov r10, r11
 ; NOREDUCTIONS-NEXT:    uxth r0, r0
 ; NOREDUCTIONS-NEXT:    rsbs r5, r0, #0
 ; NOREDUCTIONS-NEXT:    b .LBB0_4
 ; NOREDUCTIONS-NEXT:  .LBB0_2: @ in Loop: Header=BB0_4 Depth=1
 ; NOREDUCTIONS-NEXT:    movs r0, #0
 ; NOREDUCTIONS-NEXT:  .LBB0_3: @ %for.end
 ; NOREDUCTIONS-NEXT:    @ in Loop: Header=BB0_4 Depth=1
 ; NOREDUCTIONS-NEXT:    lsrs r0, r0, #16
 ; NOREDUCTIONS-NEXT:    sub.w r9, r9, #1
-; NOREDUCTIONS-NEXT:    strh.w r0, [r1, r8, lsl #1]
-; NOREDUCTIONS-NEXT:    add.w r8, r8, #1
+; NOREDUCTIONS-NEXT:    strh.w r0, [r1, r12, lsl #1]
+; NOREDUCTIONS-NEXT:    add.w r12, r12, #1
 ; NOREDUCTIONS-NEXT:    add.w r10, r10, #2
-; NOREDUCTIONS-NEXT:    cmp r8, r3
+; NOREDUCTIONS-NEXT:    cmp r12, r3
 ; NOREDUCTIONS-NEXT:    beq .LBB0_8
 ; NOREDUCTIONS-NEXT:  .LBB0_4: @ %for.body
 ; NOREDUCTIONS-NEXT:    @ =>This Loop Header: Depth=1
 ; NOREDUCTIONS-NEXT:    @ Child Loop BB0_6 Depth 2
-; NOREDUCTIONS-NEXT:    cmp r2, r8
+; NOREDUCTIONS-NEXT:    cmp r2, r12
 ; NOREDUCTIONS-NEXT:    ble .LBB0_2
 ; NOREDUCTIONS-NEXT:  @ %bb.5: @ %vector.ph
 ; NOREDUCTIONS-NEXT:    @ in Loop: Header=BB0_4 Depth=1
 ; NOREDUCTIONS-NEXT:    bic r0, r9, #3
 ; NOREDUCTIONS-NEXT:    movs r7, #1
 ; NOREDUCTIONS-NEXT:    subs r0, #4
-; NOREDUCTIONS-NEXT:    sub.w r4, r2, r8
+; NOREDUCTIONS-NEXT:    sub.w r4, r2, r12
 ; NOREDUCTIONS-NEXT:    vmov.i32 q1, #0x0
 ; NOREDUCTIONS-NEXT:    add.w r6, r7, r0, lsr #2
-; NOREDUCTIONS-NEXT:    sub.w r0, r12, r8
+; NOREDUCTIONS-NEXT:    adds r0, r2, #3
+; NOREDUCTIONS-NEXT:    sub.w r0, r0, r12
 ; NOREDUCTIONS-NEXT:    bic r0, r0, #3
 ; NOREDUCTIONS-NEXT:    subs r0, #4
 ; NOREDUCTIONS-NEXT:    add.w r0, r7, r0, lsr #2
 ; NOREDUCTIONS-NEXT:    mov r7, r10
 ; NOREDUCTIONS-NEXT:    dls lr, r0
-; NOREDUCTIONS-NEXT:    ldr r0, [sp] @ 4-byte Reload
+; NOREDUCTIONS-NEXT:    mov r0, r11
 ; NOREDUCTIONS-NEXT:  .LBB0_6: @ %vector.body
 ; NOREDUCTIONS-NEXT:    @ Parent Loop BB0_4 Depth=1
 ; NOREDUCTIONS-NEXT:    @ => This Inner Loop Header: Depth=2
 ; NOREDUCTIONS-NEXT:    vctp.32 r4
 ; NOREDUCTIONS-NEXT:    vmov q0, q1
 ; NOREDUCTIONS-NEXT:    vpstt
 ; NOREDUCTIONS-NEXT:    vldrht.s32 q1, [r0], #8
 ; NOREDUCTIONS-NEXT:    vldrht.s32 q2, [r7], #8
 ; NOREDUCTIONS-NEXT:    mov lr, r6
 ; NOREDUCTIONS-NEXT:    vmul.i32 q1, q2, q1
 ; NOREDUCTIONS-NEXT:    subs r6, #1
 ; NOREDUCTIONS-NEXT:    vshl.s32 q1, r5
 ; NOREDUCTIONS-NEXT:    subs r4, #4
 ; NOREDUCTIONS-NEXT:    vadd.i32 q1, q1, q0
 ; NOREDUCTIONS-NEXT:    le lr, .LBB0_6
 ; NOREDUCTIONS-NEXT:  @ %bb.7: @ %middle.block
 ; NOREDUCTIONS-NEXT:    @ in Loop: Header=BB0_4 Depth=1
 ; NOREDUCTIONS-NEXT:    vpsel q0, q1, q0
 ; NOREDUCTIONS-NEXT:    vaddv.u32 r0, q0
 ; NOREDUCTIONS-NEXT:    b .LBB0_3
 ; NOREDUCTIONS-NEXT:  .LBB0_8: @ %for.end17
-; NOREDUCTIONS-NEXT:    add sp, #4
-; NOREDUCTIONS-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, pc}
+; NOREDUCTIONS-NEXT:    pop.w {r4, r5, r6, r7, r9, r10, r11, pc}
 entry:
   %conv = sext i16 %N to i32
   %cmp36 = icmp sgt i16 %N, 0
   br i1 %cmp36, label %for.body.lr.ph, label %for.end17
 
 for.body.lr.ph:                                   ; preds = %entry
   %conv2 = sext i16 %Size to i32
   %conv1032 = zext i16 %Scale to i32

llvm/test/CodeGen/Thumb2/LowOverheadLoops/while-loops.ll

 ; CHECK-LABEL: b:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, lr}
 ; CHECK-NEXT:    push {r4, r5, r6, r7, lr}
 ; CHECK-NEXT:    .setfp r7, sp, #12
 ; CHECK-NEXT:    add r7, sp, #12
 ; CHECK-NEXT:    .save {r8, r9, r10, r11}
 ; CHECK-NEXT:    push.w {r8, r9, r10, r11}
-; CHECK-NEXT:    .pad #12
-; CHECK-NEXT:    sub sp, #12
+; CHECK-NEXT:    .pad #8
+; CHECK-NEXT:    sub sp, #8
 ; CHECK-NEXT:    wls lr, r1, .LBB2_3
 ; CHECK-NEXT:  @ %bb.1: @ %while.body.preheader
-; CHECK-NEXT:    mov r4, r2
-; CHECK-NEXT:    adds r2, r3, #4
+; CHECK-NEXT:    adds r4, r3, #4
 ; CHECK-NEXT:    add.w r9, r0, #4
 ; CHECK-NEXT:    mvn r11, #1
 ; CHECK-NEXT:    @ implicit-def: $r6
 ; CHECK-NEXT:    @ implicit-def: $r12
-; CHECK-NEXT:    str r4, [sp] @ 4-byte Spill
+; CHECK-NEXT:    str r2, [sp] @ 4-byte Spill
 ; CHECK-NEXT:  .LBB2_2: @ %while.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    ldr r1, [r9, #-4]
-; CHECK-NEXT:    ldr.w r10, [r2]
+; CHECK-NEXT:    ldr.w r10, [r4]
 ; CHECK-NEXT:    str r0, [sp, #4] @ 4-byte Spill
 ; CHECK-NEXT:    muls r1, r3, r1
 ; CHECK-NEXT:    adds.w r8, r1, #-2147483648
 ; CHECK-NEXT:    asr.w r5, r1, #31
 ; CHECK-NEXT:    adc r1, r5, #0
 ; CHECK-NEXT:    mul r5, r10, r0
-; CHECK-NEXT:    mov r0, r2
-; CHECK-NEXT:    ldr.w r2, [r11, #4]
-; CHECK-NEXT:    str r2, [sp, #8] @ 4-byte Spill
-; CHECK-NEXT:    ldr r2, [sp, #8] @ 4-byte Reload
+; CHECK-NEXT:    mov r0, r4
 ; CHECK-NEXT:    add.w r5, r5, #-2147483648
 ; CHECK-NEXT:    asrl r8, r1, r5
 ; CHECK-NEXT:    smull r4, r5, r10, r8
 ; CHECK-NEXT:    lsll r4, r5, #30
 ; CHECK-NEXT:    asrs r1, r5, #31
 ; CHECK-NEXT:    mov r4, r5
 ; CHECK-NEXT:    lsll r4, r1, r10
 ; CHECK-NEXT:    lsll r4, r1, #30
-; CHECK-NEXT:    ldr.w r4, [r11]
+; CHECK-NEXT:    ldrd r4, r8, [r11]
 ; CHECK-NEXT:    asrs r5, r1, #31
-; CHECK-NEXT:    mov r8, r1
 ; CHECK-NEXT:    muls r4, r6, r4
-; CHECK-NEXT:    adds r4, #2
-; CHECK-NEXT:    lsll r8, r5, r4
+; CHECK-NEXT:    adds r2, r4, #2
+; CHECK-NEXT:    mov r4, r1
+; CHECK-NEXT:    lsll r4, r5, r2
+; CHECK-NEXT:    add.w r1, r4, #-2147483648
 ; CHECK-NEXT:    ldr r4, [r9], #4
 ; CHECK-NEXT:    asr.w r5, r12, #31
-; CHECK-NEXT:    add.w r8, r8, #-2147483648
 ; CHECK-NEXT:    muls r4, r3, r4
 ; CHECK-NEXT:    adds r3, #4
-; CHECK-NEXT:    adds.w r1, r12, r4
+; CHECK-NEXT:    adds.w r2, r12, r4
 ; CHECK-NEXT:    adc.w r5, r5, r4, asr #31
-; CHECK-NEXT:    smull r6, r4, r2, r6
-; CHECK-NEXT:    adds.w r1, r1, #-2147483648
-; CHECK-NEXT:    adc r1, r5, #0
-; CHECK-NEXT:    mov r2, r0
-; CHECK-NEXT:    asrs r5, r1, #31
-; CHECK-NEXT:    subs r6, r1, r6
+; CHECK-NEXT:    smull r6, r4, r8, r6
+; CHECK-NEXT:    adds.w r2, r2, #-2147483648
+; CHECK-NEXT:    adc r2, r5, #0
+; CHECK-NEXT:    asrs r5, r2, #31
+; CHECK-NEXT:    subs r6, r2, r6
 ; CHECK-NEXT:    sbcs r5, r4
 ; CHECK-NEXT:    adds.w r6, r6, #-2147483648
 ; CHECK-NEXT:    adc r5, r5, #0
-; CHECK-NEXT:    asrl r6, r5, r8
+; CHECK-NEXT:    mov r4, r0
+; CHECK-NEXT:    asrl r6, r5, r1
+; CHECK-NEXT:    movs r1, #2
 ; CHECK-NEXT:    lsrl r6, r5, #2
-; CHECK-NEXT:    movs r5, #2
-; CHECK-NEXT:    str r6, [r5]
-; CHECK-NEXT:    ldr r5, [r11], #-4
-; CHECK-NEXT:    mls r1, r5, r10, r1
+; CHECK-NEXT:    ldr r0, [sp, #4] @ 4-byte Reload
+; CHECK-NEXT:    str r6, [r1]
+; CHECK-NEXT:    ldr r1, [r11], #-4
+; CHECK-NEXT:    adds r0, #4
+; CHECK-NEXT:    mls r1, r1, r10, r2
 ; CHECK-NEXT:    adds.w r12, r1, #-2147483648
-; CHECK-NEXT:    asr.w r4, r1, #31
-; CHECK-NEXT:    adc r1, r4, #0
-; CHECK-NEXT:    ldrd r4, r0, [sp] @ 8-byte Folded Reload
+; CHECK-NEXT:    asr.w r2, r1, #31
+; CHECK-NEXT:    adc r1, r2, #0
+; CHECK-NEXT:    ldr r2, [sp] @ 4-byte Reload
 ; CHECK-NEXT:    lsrl r12, r1, #2
 ; CHECK-NEXT:    rsb.w r1, r12, #0
-; CHECK-NEXT:    adds r0, #4
-; CHECK-NEXT:    str r1, [r4]
-; CHECK-NEXT:    str r1, [r2, #-4]
-; CHECK-NEXT:    adds r2, #4
+; CHECK-NEXT:    str r1, [r2]
+; CHECK-NEXT:    str r1, [r4, #-4]
+; CHECK-NEXT:    adds r4, #4
 ; CHECK-NEXT:    le lr, .LBB2_2
 ; CHECK-NEXT:  .LBB2_3: @ %while.end
-; CHECK-NEXT:    add sp, #12
+; CHECK-NEXT:    add sp, #8
 ; CHECK-NEXT:    pop.w {r8, r9, r10, r11}
 ; CHECK-NEXT:    pop {r4, r5, r6, r7, pc}
 entry:
   %0 = inttoptr i32 %e to i32*
   %tobool.not70 = icmp eq i32 %d, 0
   br i1 %tobool.not70, label %while.end, label %while.body
 
 while.body:                                       ; preds = %entry, %while.body

llvm/test/CodeGen/Thumb2/ldr-str-imm12.ll

 ; CHECK-NEXT:    push {r4, r5, r6, r7, lr}
 ; CHECK-NEXT:    add r7, sp, #12
 ; CHECK-NEXT:    push.w {r8, r10, r11}
 ; CHECK-NEXT:    sub sp, #292
 ; CHECK-NEXT:    add.w r10, r7, #8
 ; CHECK-NEXT:    ldrd r8, lr, [r7, #20]
 ; CHECK-NEXT:    movs r5, #0
 ; CHECK-NEXT:    cmp r5, #0
-; CHECK-NEXT:    ldm.w r10, {r4, r9, r10}
-; CHECK-NEXT:    ldr.w r12, [r7, #28]
+; CHECK-NEXT:    ldm.w r10, {r4, r6, r10}
+; CHECK-NEXT:    ldrd r12, r9, [r7, #28]
 ; CHECK-NEXT:    ittt ne
 ; CHECK-NEXT:    addne sp, #292
 ; CHECK-NEXT:    popne.w {r8, r10, r11}
 ; CHECK-NEXT:    popne {r4, r5, r6, r7, pc}
 ; CHECK-NEXT:  LBB0_1: @ %bb20
 ; CHECK-NEXT:    cmp.w r0, #450
 ; CHECK-NEXT:    bge LBB0_4
 ; CHECK-NEXT:  @ %bb.2: @ %bb20
 ; CHECK-NEXT:    cmp r0, #209
 ; CHECK-NEXT:    ble LBB0_5
 ; CHECK-NEXT:  @ %bb.3: @ %bb420
 ; CHECK-NEXT:    movw r5, :lower16:(L_zz_hold$non_lazy_ptr-(LPC0_0+4))
 ; CHECK-NEXT:    movt r5, :upper16:(L_zz_hold$non_lazy_ptr-(LPC0_0+4))
-; CHECK-NEXT:    movw r11, :lower16:(L_zz_res$non_lazy_ptr-(LPC0_1+4))
 ; CHECK-NEXT:  LPC0_0:
 ; CHECK-NEXT:    add r5, pc
-; CHECK-NEXT:    movt r11, :upper16:(L_zz_res$non_lazy_ptr-(LPC0_1+4))
+; CHECK-NEXT:    ldr.w r11, [r5]
+; CHECK-NEXT:    str.w r11, [sp, #28] @ 4-byte Spill
+; CHECK-NEXT:    movw r5, :lower16:(L_zz_res$non_lazy_ptr-(LPC0_1+4))
+; CHECK-NEXT:    movt r5, :upper16:(L_zz_res$non_lazy_ptr-(LPC0_1+4))
 ; CHECK-NEXT:  LPC0_1:
-; CHECK-NEXT:    add r11, pc
+; CHECK-NEXT:    add r5, pc
 ; CHECK-NEXT:    ldr r5, [r5]
 ; CHECK-NEXT:    str r5, [sp, #32] @ 4-byte Spill
-; CHECK-NEXT:    ldr.w r5, [r11]
-; CHECK-NEXT:    mov.w r11, #0
-; CHECK-NEXT:    str r5, [sp, #28] @ 4-byte Spill
-; CHECK-NEXT:    ldr r5, [sp, #32] @ 4-byte Reload
-; CHECK-NEXT:    str.w r11, [r5]
 ; CHECK-NEXT:    movs r5, #0
-; CHECK-NEXT:    ldr r6, [sp, #28] @ 4-byte Reload
-; CHECK-NEXT:    str r5, [r6]
-; CHECK-NEXT:    ldr r5, [sp, #32] @ 4-byte Reload
+; CHECK-NEXT:    str.w r5, [r11]
+; CHECK-NEXT:    ldr.w r11, [sp, #32] @ 4-byte Reload
+; CHECK-NEXT:    str.w r5, [r11]
+; CHECK-NEXT:    ldr r5, [sp, #28] @ 4-byte Reload
 ; CHECK-NEXT:    str r0, [r5]
-; CHECK-NEXT:    ldr r0, [r7, #32]
-; CHECK-NEXT:    stm.w sp, {r4, r9, r10}
+; CHECK-NEXT:    stm.w sp, {r4, r6, r10}
 ; CHECK-NEXT:    strd r8, lr, [sp, #12]
-; CHECK-NEXT:    str.w r12, [sp, #20]
-; CHECK-NEXT:    str r0, [sp, #24]
+; CHECK-NEXT:    strd r12, r9, [sp, #20]
 ; CHECK-NEXT:    bl _Manifest
 ; CHECK-NEXT:    trap
 ; CHECK-NEXT:  LBB0_4: @ %bb20
 ; CHECK-NEXT:    cmp.w r0, #560
 ; CHECK-NEXT:    itt ge
 ; CHECK-NEXT:    movge r0, #0
 ; CHECK-NEXT:    cmpge r0, #0
 ; CHECK-NEXT:  LBB0_5: @ %bb20
switch i32 undef, label %bb1287 [
     i32 560, label %bb569
     i32 640, label %bb745
     i32 780, label %bb1098
   ]
 
 bb119:                                            ; preds = %bb20, %bb20
   unreachable
 
 bb420:                                            ; preds = %bb20, %bb20
   store volatile %union.rec* null, %union.rec** @zz_hold, align 4
   store %union.rec* null, %union.rec** @zz_res, align 4
   store volatile %union.rec* %x, %union.rec** @zz_hold, align 4
   %0 = call  %union.rec* @Manifest(%union.rec* undef, %union.rec* %env, %struct.STYLE* %style, %union.rec** %bthr, %union.rec** %fthr, %union.rec** %target, %union.rec** %crs, i32 %ok, i32 %need_expand, %union.rec** %enclose, i32 %fcr) nounwind ; <%union.rec*> [#uses=0]
   unreachable
 
 bb438:                                            ; preds = %bb20, %bb20
   unreachable
 

llvm/test/CodeGen/Thumb2/mve-float16regloops.ll

 }
 
 
 define void @fir(%struct.arm_fir_instance_f32* nocapture readonly %S, half* nocapture readonly %pSrc, half* nocapture %pDst, i32 %blockSize) {
 ; CHECK-LABEL: fir:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
-; CHECK-NEXT:    .pad #24
-; CHECK-NEXT:    sub sp, #24
+; CHECK-NEXT:    .pad #20
+; CHECK-NEXT:    sub sp, #20
 ; CHECK-NEXT:    cmp r3, #8
-; CHECK-NEXT:    str r1, [sp, #20] @ 4-byte Spill
+; CHECK-NEXT:    str r1, [sp, #16] @ 4-byte Spill
 ; CHECK-NEXT:    blo.w .LBB16_12
 ; CHECK-NEXT:  @ %bb.1: @ %entry
 ; CHECK-NEXT:    lsrs.w r12, r3, #2
 ; CHECK-NEXT:    beq.w .LBB16_12
 ; CHECK-NEXT:  @ %bb.2: @ %while.body.lr.ph
 ; CHECK-NEXT:    ldrh r4, [r0]
 ; CHECK-NEXT:    movs r1, #1
 ; CHECK-NEXT:    ldrd r5, r3, [r0, #4]
 ; CHECK-NEXT:    sub.w r0, r4, #8
 ; CHECK-NEXT:    add.w r7, r0, r0, lsr #29
 ; CHECK-NEXT:    and r0, r0, #7
 ; CHECK-NEXT:    asrs r6, r7, #3
 ; CHECK-NEXT:    cmp r6, #1
 ; CHECK-NEXT:    it gt
 ; CHECK-NEXT:    asrgt r1, r7, #3
 ; CHECK-NEXT:    add.w r7, r5, r4, lsl #1
 ; CHECK-NEXT:    str r1, [sp] @ 4-byte Spill
 ; CHECK-NEXT:    subs r1, r7, #2
 ; CHECK-NEXT:    rsbs r7, r4, #0
-; CHECK-NEXT:    str r7, [sp, #8] @ 4-byte Spill
-; CHECK-NEXT:    add.w r7, r3, #16
-; CHECK-NEXT:    str r4, [sp, #12] @ 4-byte Spill
+; CHECK-NEXT:    str r4, [sp, #8] @ 4-byte Spill
 ; CHECK-NEXT:    str r7, [sp, #4] @ 4-byte Spill
-; CHECK-NEXT:    str r0, [sp, #16] @ 4-byte Spill
+; CHECK-NEXT:    str r0, [sp, #12] @ 4-byte Spill
 ; CHECK-NEXT:    b .LBB16_5
 ; CHECK-NEXT:  .LBB16_3: @ %for.end
 ; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r0, [sp, #16] @ 4-byte Reload
+; CHECK-NEXT:    ldr r0, [sp, #12] @ 4-byte Reload
 ; CHECK-NEXT:    wls lr, r0, .LBB16_4
 ; CHECK-NEXT:    b .LBB16_9
 ; CHECK-NEXT:  .LBB16_4: @ %while.end
 ; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r0, [sp, #8] @ 4-byte Reload
+; CHECK-NEXT:    ldr r0, [sp, #4] @ 4-byte Reload
 ; CHECK-NEXT:    subs.w r12, r12, #1
 ; CHECK-NEXT:    vstrb.8 q0, [r2], #8
-; CHECK-NEXT:    add.w r0, r5, r0, lsl #1
+; CHECK-NEXT:    add.w r0, r6, r0, lsl #1
 ; CHECK-NEXT:    add.w r5, r0, #8
 ; CHECK-NEXT:    beq.w .LBB16_12
 ; CHECK-NEXT:  .LBB16_5: @ %while.body
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
 ; CHECK-NEXT:    @ Child Loop BB16_7 Depth 2
 ; CHECK-NEXT:    @ Child Loop BB16_10 Depth 2
-; CHECK-NEXT:    ldr r0, [sp, #20] @ 4-byte Reload
+; CHECK-NEXT:    ldr r0, [sp, #16] @ 4-byte Reload
 ; CHECK-NEXT:    ldrh.w lr, [r3, #14]
 ; CHECK-NEXT:    vldrw.u32 q0, [r0], #8
-; CHECK-NEXT:    ldrh.w r8, [r3, #12]
+; CHECK-NEXT:    ldrh.w r10, [r3, #12]
 ; CHECK-NEXT:    ldrh r7, [r3, #10]
 ; CHECK-NEXT:    ldrh r4, [r3, #8]
 ; CHECK-NEXT:    ldrh r6, [r3, #6]
 ; CHECK-NEXT:    ldrh.w r9, [r3, #4]
 ; CHECK-NEXT:    ldrh.w r11, [r3, #2]
-; CHECK-NEXT:    ldrh.w r10, [r3]
+; CHECK-NEXT:    ldrh.w r8, [r3]
 ; CHECK-NEXT:    vstrb.8 q0, [r1], #8
 ; CHECK-NEXT:    vldrw.u32 q0, [r5]
-; CHECK-NEXT:    str r0, [sp, #20] @ 4-byte Spill
+; CHECK-NEXT:    str r0, [sp, #16] @ 4-byte Spill
 ; CHECK-NEXT:    adds r0, r5, #2
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    vmul.f16 q0, q0, r10
+; CHECK-NEXT:    vmul.f16 q0, q0, r8
 ; CHECK-NEXT:    adds r0, r5, #6
 ; CHECK-NEXT:    vfma.f16 q0, q1, r11
 ; CHECK-NEXT:    vldrw.u32 q1, [r5, #4]
 ; CHECK-NEXT:    vfma.f16 q0, q1, r9
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
 ; CHECK-NEXT:    add.w r0, r5, #10
 ; CHECK-NEXT:    vfma.f16 q0, q1, r6
 ; CHECK-NEXT:    vldrw.u32 q1, [r5, #8]
+; CHECK-NEXT:    add.w r6, r5, #16
 ; CHECK-NEXT:    vfma.f16 q0, q1, r4
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
 ; CHECK-NEXT:    add.w r0, r5, #14
 ; CHECK-NEXT:    vfma.f16 q0, q1, r7
 ; CHECK-NEXT:    vldrw.u32 q1, [r5, #12]
-; CHECK-NEXT:    adds r5, #16
-; CHECK-NEXT:    vfma.f16 q0, q1, r8
+; CHECK-NEXT:    vfma.f16 q0, q1, r10
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    ldr r0, [sp, #12] @ 4-byte Reload
+; CHECK-NEXT:    ldr r0, [sp, #8] @ 4-byte Reload
 ; CHECK-NEXT:    vfma.f16 q0, q1, lr
 ; CHECK-NEXT:    cmp r0, #16
 ; CHECK-NEXT:    blo .LBB16_8
 ; CHECK-NEXT:  @ %bb.6: @ %for.body.preheader
 ; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
 ; CHECK-NEXT:    ldr r0, [sp] @ 4-byte Reload
+; CHECK-NEXT:    add.w r5, r3, #16
 ; CHECK-NEXT:    dls lr, r0
-; CHECK-NEXT:    ldr r6, [sp, #4] @ 4-byte Reload
 ; CHECK-NEXT:  .LBB16_7: @ %for.body
 ; CHECK-NEXT:    @ Parent Loop BB16_5 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    ldrh r0, [r6], #16
-; CHECK-NEXT:    vldrw.u32 q1, [r5]
-; CHECK-NEXT:    adds r4, r5, #2
+; CHECK-NEXT:    ldrh r0, [r5], #16
+; CHECK-NEXT:    vldrw.u32 q1, [r6]
+; CHECK-NEXT:    adds r4, r6, #2
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
 ; CHECK-NEXT:    vldrw.u32 q1, [r4]
-; CHECK-NEXT:    ldrh r0, [r6, #-14]
-; CHECK-NEXT:    adds r4, r5, #6
+; CHECK-NEXT:    ldrh r0, [r5, #-14]
+; CHECK-NEXT:    adds r4, r6, #6
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
-; CHECK-NEXT:    ldrh r0, [r6, #-12]
-; CHECK-NEXT:    vldrw.u32 q1, [r5, #4]
+; CHECK-NEXT:    ldrh r0, [r5, #-12]
+; CHECK-NEXT:    vldrw.u32 q1, [r6, #4]
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
 ; CHECK-NEXT:    vldrw.u32 q1, [r4]
-; CHECK-NEXT:    ldrh r0, [r6, #-10]
-; CHECK-NEXT:    add.w r4, r5, #10
+; CHECK-NEXT:    ldrh r0, [r5, #-10]
+; CHECK-NEXT:    add.w r4, r6, #10
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
-; CHECK-NEXT:    ldrh r0, [r6, #-8]
-; CHECK-NEXT:    vldrw.u32 q1, [r5, #8]
+; CHECK-NEXT:    ldrh r0, [r5, #-8]
+; CHECK-NEXT:    vldrw.u32 q1, [r6, #8]
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
 ; CHECK-NEXT:    vldrw.u32 q1, [r4]
-; CHECK-NEXT:    ldrh r0, [r6, #-6]
-; CHECK-NEXT:    ldrh r4, [r6, #-2]
+; CHECK-NEXT:    ldrh r0, [r5, #-6]
+; CHECK-NEXT:    ldrh r4, [r5, #-2]
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
-; CHECK-NEXT:    ldrh r0, [r6, #-4]
-; CHECK-NEXT:    vldrw.u32 q1, [r5, #12]
+; CHECK-NEXT:    ldrh r0, [r5, #-4]
+; CHECK-NEXT:    vldrw.u32 q1, [r6, #12]
 ; CHECK-NEXT:    vfma.f16 q0, q1, r0
-; CHECK-NEXT:    add.w r0, r5, #14
+; CHECK-NEXT:    add.w r0, r6, #14
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    adds r5, #16
+; CHECK-NEXT:    adds r6, #16
 ; CHECK-NEXT:    vfma.f16 q0, q1, r4
 ; CHECK-NEXT:    le lr, .LBB16_7
 ; CHECK-NEXT:    b .LBB16_3
 ; CHECK-NEXT:  .LBB16_8: @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r6, [sp, #4] @ 4-byte Reload
+; CHECK-NEXT:    add.w r5, r3, #16
 ; CHECK-NEXT:    b .LBB16_3
 ; CHECK-NEXT:  .LBB16_9: @ %while.body76.preheader
 ; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    mov r0, r5
+; CHECK-NEXT:    mov r0, r6
 ; CHECK-NEXT:  .LBB16_10: @ %while.body76
 ; CHECK-NEXT:    @ Parent Loop BB16_5 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    ldrh r4, [r6], #2
+; CHECK-NEXT:    ldrh r4, [r5], #2
 ; CHECK-NEXT:    vldrh.u16 q1, [r0], #2
 ; CHECK-NEXT:    vfma.f16 q0, q1, r4
 ; CHECK-NEXT:    le lr, .LBB16_10
 ; CHECK-NEXT:  @ %bb.11: @ %while.end.loopexit
 ; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r0, [sp, #16] @ 4-byte Reload
-; CHECK-NEXT:    add.w r5, r5, r0, lsl #1
+; CHECK-NEXT:    ldr r0, [sp, #12] @ 4-byte Reload
+; CHECK-NEXT:    add.w r6, r6, r0, lsl #1
 ; CHECK-NEXT:    b .LBB16_4
 ; CHECK-NEXT:  .LBB16_12: @ %if.end
-; CHECK-NEXT:    add sp, #24
+; CHECK-NEXT:    add sp, #20
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   %pState1 = getelementptr inbounds %struct.arm_fir_instance_f32, %struct.arm_fir_instance_f32* %S, i32 0, i32 1
   %0 = load half*, half** %pState1, align 4
   %pCoeffs2 = getelementptr inbounds %struct.arm_fir_instance_f32, %struct.arm_fir_instance_f32* %S, i32 0, i32 2
   %1 = load half*, half** %pCoeffs2, align 4
   %numTaps3 = getelementptr inbounds %struct.arm_fir_instance_f32, %struct.arm_fir_instance_f32* %S, i32 0, i32 0
   %2 = load i16, i16* %numTaps3, align 4

llvm/test/CodeGen/Thumb2/mve-float32regloops.ll

 ; CHECK-LABEL: fir:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    .pad #4
 ; CHECK-NEXT:    sub sp, #4
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
-; CHECK-NEXT:    .pad #32
-; CHECK-NEXT:    sub sp, #32
+; CHECK-NEXT:    .pad #24
+; CHECK-NEXT:    sub sp, #24
 ; CHECK-NEXT:    cmp r3, #8
 ; CHECK-NEXT:    blo.w .LBB16_12
 ; CHECK-NEXT:  @ %bb.1: @ %entry
 ; CHECK-NEXT:    lsrs.w r12, r3, #2
 ; CHECK-NEXT:    beq.w .LBB16_12
 ; CHECK-NEXT:  @ %bb.2: @ %while.body.lr.ph
 ; CHECK-NEXT:    ldrh r6, [r0]
-; CHECK-NEXT:    movs r5, #1
-; CHECK-NEXT:    ldrd r4, r10, [r0, #4]
+; CHECK-NEXT:    movs r4, #1
+; CHECK-NEXT:    ldrd r7, r10, [r0, #4]
 ; CHECK-NEXT:    sub.w r0, r6, #8
 ; CHECK-NEXT:    add.w r3, r0, r0, lsr #29
 ; CHECK-NEXT:    and r0, r0, #7
-; CHECK-NEXT:    asrs r7, r3, #3
-; CHECK-NEXT:    cmp r7, #1
+; CHECK-NEXT:    asrs r5, r3, #3
+; CHECK-NEXT:    cmp r5, #1
 ; CHECK-NEXT:    it gt
-; CHECK-NEXT:    asrgt r5, r3, #3
-; CHECK-NEXT:    add.w r3, r4, r6, lsl #2
+; CHECK-NEXT:    asrgt r4, r3, #3
+; CHECK-NEXT:    add.w r3, r7, r6, lsl #2
 ; CHECK-NEXT:    sub.w r9, r3, #4
 ; CHECK-NEXT:    rsbs r3, r6, #0
-; CHECK-NEXT:    str r3, [sp, #12] @ 4-byte Spill
-; CHECK-NEXT:    add.w r3, r10, #32
-; CHECK-NEXT:    str r5, [sp, #4] @ 4-byte Spill
-; CHECK-NEXT:    str r6, [sp, #16] @ 4-byte Spill
-; CHECK-NEXT:    str r3, [sp, #8] @ 4-byte Spill
-; CHECK-NEXT:    str r0, [sp, #20] @ 4-byte Spill
+; CHECK-NEXT:    str r4, [sp] @ 4-byte Spill
+; CHECK-NEXT:    str r6, [sp, #8] @ 4-byte Spill
+; CHECK-NEXT:    str r3, [sp, #4] @ 4-byte Spill
+; CHECK-NEXT:    str r0, [sp, #12] @ 4-byte Spill
 ; CHECK-NEXT:    b .LBB16_5
 ; CHECK-NEXT:  .LBB16_3: @ %for.end
 ; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r1, [sp, #28] @ 4-byte Reload
-; CHECK-NEXT:    ldrd r0, r9, [sp, #20] @ 8-byte Folded Reload
+; CHECK-NEXT:    ldr r1, [sp, #20] @ 4-byte Reload
+; CHECK-NEXT:    ldrd r0, r9, [sp, #12] @ 8-byte Folded Reload
 ; CHECK-NEXT:    wls lr, r0, .LBB16_4
 ; CHECK-NEXT:    b .LBB16_9
 ; CHECK-NEXT:  .LBB16_4: @ %while.end
 ; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r0, [sp, #12] @ 4-byte Reload
+; CHECK-NEXT:    ldr r0, [sp, #4] @ 4-byte Reload
 ; CHECK-NEXT:    subs.w r12, r12, #1
 ; CHECK-NEXT:    vstrb.8 q0, [r2], #16
-; CHECK-NEXT:    add.w r0, r4, r0, lsl #2
-; CHECK-NEXT:    add.w r4, r0, #16
+; CHECK-NEXT:    add.w r0, r7, r0, lsl #2
+; CHECK-NEXT:    add.w r7, r0, #16
 ; CHECK-NEXT:    beq .LBB16_12
 ; CHECK-NEXT:  .LBB16_5: @ %while.body
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
 ; CHECK-NEXT:    @ Child Loop BB16_7 Depth 2
 ; CHECK-NEXT:    @ Child Loop BB16_10 Depth 2
 ; CHECK-NEXT:    add.w lr, r10, #8
 ; CHECK-NEXT:    vldrw.u32 q0, [r1], #16
-; CHECK-NEXT:    ldrd r3, r7, [r10]
+; CHECK-NEXT:    ldrd r3, r4, [r10]
 ; CHECK-NEXT:    ldm.w lr, {r0, r5, r6, lr}
 ; CHECK-NEXT:    ldrd r11, r8, [r10, #24]
 ; CHECK-NEXT:    vstrb.8 q0, [r9], #16
-; CHECK-NEXT:    vldrw.u32 q0, [r4], #32
-; CHECK-NEXT:    strd r9, r1, [sp, #24] @ 8-byte Folded Spill
-; CHECK-NEXT:    vldrw.u32 q1, [r4, #-28]
+; CHECK-NEXT:    vldrw.u32 q0, [r7], #32
+; CHECK-NEXT:    strd r9, r1, [sp, #16] @ 8-byte Folded Spill
+; CHECK-NEXT:    vldrw.u32 q1, [r7, #-28]
 ; CHECK-NEXT:    vmul.f32 q0, q0, r3
-; CHECK-NEXT:    vldrw.u32 q6, [r4, #-24]
-; CHECK-NEXT:    vldrw.u32 q4, [r4, #-20]
-; CHECK-NEXT:    vfma.f32 q0, q1, r7
-; CHECK-NEXT:    vldrw.u32 q5, [r4, #-16]
+; CHECK-NEXT:    vldrw.u32 q6, [r7, #-24]
+; CHECK-NEXT:    vldrw.u32 q4, [r7, #-20]
+; CHECK-NEXT:    vfma.f32 q0, q1, r4
+; CHECK-NEXT:    vldrw.u32 q5, [r7, #-16]
 ; CHECK-NEXT:    vfma.f32 q0, q6, r0
-; CHECK-NEXT:    vldrw.u32 q2, [r4, #-12]
+; CHECK-NEXT:    vldrw.u32 q2, [r7, #-12]
 ; CHECK-NEXT:    vfma.f32 q0, q4, r5
-; CHECK-NEXT:    vldrw.u32 q3, [r4, #-8]
+; CHECK-NEXT:    vldrw.u32 q3, [r7, #-8]
 ; CHECK-NEXT:    vfma.f32 q0, q5, r6
-; CHECK-NEXT:    ldr r0, [sp, #16] @ 4-byte Reload
+; CHECK-NEXT:    ldr r0, [sp, #8] @ 4-byte Reload
 ; CHECK-NEXT:    vfma.f32 q0, q2, lr
-; CHECK-NEXT:    vldrw.u32 q1, [r4, #-4]
+; CHECK-NEXT:    vldrw.u32 q1, [r7, #-4]
 ; CHECK-NEXT:    vfma.f32 q0, q3, r11
 ; CHECK-NEXT:    cmp r0, #16
 ; CHECK-NEXT:    vfma.f32 q0, q1, r8
 ; CHECK-NEXT:    blo .LBB16_8
 ; CHECK-NEXT:  @ %bb.6: @ %for.body.preheader
 ; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r0, [sp, #4] @ 4-byte Reload
+; CHECK-NEXT:    ldr r0, [sp] @ 4-byte Reload
+; CHECK-NEXT:    add.w r4, r10, #32
 ; CHECK-NEXT:    dls lr, r0
-; CHECK-NEXT:    ldr r7, [sp, #8] @ 4-byte Reload
 ; CHECK-NEXT:  .LBB16_7: @ %for.body
 ; CHECK-NEXT:    @ Parent Loop BB16_5 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    ldm.w r7, {r0, r3, r5, r6, r8, r11}
-; CHECK-NEXT:    vldrw.u32 q1, [r4], #32
-; CHECK-NEXT:    vldrw.u32 q6, [r4, #-24]
-; CHECK-NEXT:    vldrw.u32 q4, [r4, #-20]
+; CHECK-NEXT:    ldm.w r4, {r0, r3, r5, r6, r8, r11}
+; CHECK-NEXT:    vldrw.u32 q1, [r7], #32
+; CHECK-NEXT:    vldrw.u32 q6, [r7, #-24]
+; CHECK-NEXT:    vldrw.u32 q4, [r7, #-20]
 ; CHECK-NEXT:    vfma.f32 q0, q1, r0
-; CHECK-NEXT:    vldrw.u32 q1, [r4, #-28]
-; CHECK-NEXT:    vldrw.u32 q5, [r4, #-16]
-; CHECK-NEXT:    vldrw.u32 q2, [r4, #-12]
+; CHECK-NEXT:    vldrw.u32 q1, [r7, #-28]
+; CHECK-NEXT:    vldrw.u32 q5, [r7, #-16]
+; CHECK-NEXT:    vldrw.u32 q2, [r7, #-12]
 ; CHECK-NEXT:    vfma.f32 q0, q1, r3
-; CHECK-NEXT:    ldrd r9, r1, [r7, #24]
+; CHECK-NEXT:    ldrd r9, r1, [r4, #24]
 ; CHECK-NEXT:    vfma.f32 q0, q6, r5
-; CHECK-NEXT:    vldrw.u32 q3, [r4, #-8]
+; CHECK-NEXT:    vldrw.u32 q3, [r7, #-8]
 ; CHECK-NEXT:    vfma.f32 q0, q4, r6
-; CHECK-NEXT:    vldrw.u32 q1, [r4, #-4]
+; CHECK-NEXT:    vldrw.u32 q1, [r7, #-4]
 ; CHECK-NEXT:    vfma.f32 q0, q5, r8
-; CHECK-NEXT:    adds r7, #32
+; CHECK-NEXT:    adds r4, #32
 ; CHECK-NEXT:    vfma.f32 q0, q2, r11
 ; CHECK-NEXT:    vfma.f32 q0, q3, r9
 ; CHECK-NEXT:    vfma.f32 q0, q1, r1
 ; CHECK-NEXT:    le lr, .LBB16_7
 ; CHECK-NEXT:    b .LBB16_3
 ; CHECK-NEXT:  .LBB16_8: @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r7, [sp, #8] @ 4-byte Reload
+; CHECK-NEXT:    add.w r4, r10, #32
 ; CHECK-NEXT:    b .LBB16_3
 ; CHECK-NEXT:  .LBB16_9: @ %while.body76.preheader
 ; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    mov r3, r4
+; CHECK-NEXT:    mov r3, r7
 ; CHECK-NEXT:  .LBB16_10: @ %while.body76
 ; CHECK-NEXT:    @ Parent Loop BB16_5 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    ldr r0, [r7], #4
+; CHECK-NEXT:    ldr r0, [r4], #4
 ; CHECK-NEXT:    vldrw.u32 q1, [r3], #4
 ; CHECK-NEXT:    vfma.f32 q0, q1, r0
 ; CHECK-NEXT:    le lr, .LBB16_10
 ; CHECK-NEXT:  @ %bb.11: @ %while.end.loopexit
 ; CHECK-NEXT:    @ in Loop: Header=BB16_5 Depth=1
-; CHECK-NEXT:    ldr r0, [sp, #20] @ 4-byte Reload
-; CHECK-NEXT:    add.w r4, r4, r0, lsl #2
+; CHECK-NEXT:    ldr r0, [sp, #12] @ 4-byte Reload
+; CHECK-NEXT:    add.w r7, r7, r0, lsl #2
 ; CHECK-NEXT:    b .LBB16_4
 ; CHECK-NEXT:  .LBB16_12: @ %if.end
-; CHECK-NEXT:    add sp, #32
+; CHECK-NEXT:    add sp, #24
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   %pState1 = getelementptr inbounds %struct.arm_fir_instance_f32, %struct.arm_fir_instance_f32* %S, i32 0, i32 1
   %0 = load float*, float** %pState1, align 4
   %pCoeffs2 = getelementptr inbounds %struct.arm_fir_instance_f32, %struct.arm_fir_instance_f32* %S, i32 0, i32 2
   %1 = load float*, float** %pCoeffs2, align 4

llvm/test/CodeGen/Thumb2/mve-postinc-dct.ll

 
 define void @DCT_mve2(%struct.DCT_InstanceTypeDef* nocapture readonly %S, float* nocapture readonly %pIn, float* nocapture %pOut) {
 ; CHECK-LABEL: DCT_mve2:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    .pad #4
 ; CHECK-NEXT:    sub sp, #4
-; CHECK-NEXT:    str r1, [sp] @ 4-byte Spill
-; CHECK-NEXT:    ldr r1, [r0, #4]
-; CHECK-NEXT:    subs r1, #2
-; CHECK-NEXT:    cmp r1, #2
+; CHECK-NEXT:    ldr r3, [r0, #4]
+; CHECK-NEXT:    subs r3, #2
+; CHECK-NEXT:    str r3, [sp] @ 4-byte Spill
+; CHECK-NEXT:    cmp r3, #2
 ; CHECK-NEXT:    blo .LBB1_5
 ; CHECK-NEXT:  @ %bb.1: @ %for.body.preheader
 ; CHECK-NEXT:    ldr.w r12, [r0, #8]
 ; CHECK-NEXT:    movs r4, #1
 ; CHECK-NEXT:    ldr r3, [r0]
 ; CHECK-NEXT:    add.w r11, r3, r12, lsl #2
-; CHECK-NEXT:    add.w r7, r3, r12, lsl #3
-; CHECK-NEXT:    lsl.w r9, r12, #3
+; CHECK-NEXT:    add.w r6, r3, r12, lsl #3
+; CHECK-NEXT:    lsl.w r10, r12, #3
 ; CHECK-NEXT:  .LBB1_2: @ %for.body
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
 ; CHECK-NEXT:    @ Child Loop BB1_3 Depth 2
-; CHECK-NEXT:    ldr r5, [sp] @ 4-byte Reload
 ; CHECK-NEXT:    vmov.i32 q0, #0x0
-; CHECK-NEXT:    add.w r10, r4, #1
+; CHECK-NEXT:    add.w r9, r4, #1
+; CHECK-NEXT:    mov r5, r1
 ; CHECK-NEXT:    mov r3, r11
-; CHECK-NEXT:    mov r0, r7
+; CHECK-NEXT:    mov r0, r6
 ; CHECK-NEXT:    vmov q1, q0
 ; CHECK-NEXT:    dlstp.32 lr, r12
 ; CHECK-NEXT:  .LBB1_3: @ %vector.body
 ; CHECK-NEXT:    @ Parent Loop BB1_2 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    vldrw.u32 q2, [r5], #16
 ; CHECK-NEXT:    vldrw.u32 q3, [r3], #16
 ; CHECK-NEXT:    vfma.f32 q1, q3, q2
 ; CHECK-NEXT:    vldrw.u32 q3, [r0], #16
 ; CHECK-NEXT:    vfma.f32 q0, q3, q2
 ; CHECK-NEXT:    letp lr, .LBB1_3
 ; CHECK-NEXT:  @ %bb.4: @ %middle.block
 ; CHECK-NEXT:    @ in Loop: Header=BB1_2 Depth=1
 ; CHECK-NEXT:    vadd.f32 s2, s2, s3
-; CHECK-NEXT:    add.w r0, r2, r10, lsl #2
+; CHECK-NEXT:    add.w r0, r2, r9, lsl #2
 ; CHECK-NEXT:    vadd.f32 s0, s0, s1
-; CHECK-NEXT:    add r11, r9
+; CHECK-NEXT:    add r11, r10
 ; CHECK-NEXT:    vadd.f32 s6, s6, s7
-; CHECK-NEXT:    add r7, r9
+; CHECK-NEXT:    add r6, r10
 ; CHECK-NEXT:    vadd.f32 s4, s4, s5
 ; CHECK-NEXT:    vadd.f32 s0, s0, s2
 ; CHECK-NEXT:    vadd.f32 s2, s4, s6
 ; CHECK-NEXT:    vstr s0, [r0]
 ; CHECK-NEXT:    add.w r0, r2, r4, lsl #2
 ; CHECK-NEXT:    adds r4, #2
-; CHECK-NEXT:    cmp r4, r1
 ; CHECK-NEXT:    vstr s2, [r0]
+; CHECK-NEXT:    ldr r0, [sp] @ 4-byte Reload
+; CHECK-NEXT:    cmp r4, r0
 ; CHECK-NEXT:    blo .LBB1_2
 ; CHECK-NEXT:  .LBB1_5: @ %for.cond.cleanup
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   %NumInputs = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 2
   %0 = load i32, i32* %NumInputs, align 4
   %NumFilters = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 1

 ; CHECK-LABEL: DCT_mve3:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    .pad #4
 ; CHECK-NEXT:    sub sp, #4
 ; CHECK-NEXT:    .vsave {d8, d9}
 ; CHECK-NEXT:    vpush {d8, d9}
-; CHECK-NEXT:    .pad #24
-; CHECK-NEXT:    sub sp, #24
-; CHECK-NEXT:    str r1, [sp, #16] @ 4-byte Spill
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
+; CHECK-NEXT:    str r1, [sp, #12] @ 4-byte Spill
 ; CHECK-NEXT:    ldr r1, [r0, #4]
-; CHECK-NEXT:    str r2, [sp, #8] @ 4-byte Spill
 ; CHECK-NEXT:    subs r1, #3
-; CHECK-NEXT:    str r1, [sp, #12] @ 4-byte Spill
+; CHECK-NEXT:    str r1, [sp, #8] @ 4-byte Spill
 ; CHECK-NEXT:    cmp r1, #2
 ; CHECK-NEXT:    blo .LBB2_5
 ; CHECK-NEXT:  @ %bb.1: @ %for.body.preheader
-; CHECK-NEXT:    ldr r3, [r0, #8]
+; CHECK-NEXT:    ldr.w r9, [r0, #8]
 ; CHECK-NEXT:    movs r5, #1
 ; CHECK-NEXT:    ldr r1, [r0]
-; CHECK-NEXT:    str r3, [sp, #4] @ 4-byte Spill
-; CHECK-NEXT:    add.w r0, r3, r3, lsl #1
-; CHECK-NEXT:    add.w r9, r1, r3, lsl #2
-; CHECK-NEXT:    add.w r12, r1, r3, lsl #3
-; CHECK-NEXT:    adds r3, #3
-; CHECK-NEXT:    bic r3, r3, #3
-; CHECK-NEXT:    ldr r7, [sp, #4] @ 4-byte Reload
-; CHECK-NEXT:    add.w r10, r1, r0, lsl #2
-; CHECK-NEXT:    subs r3, #4
+; CHECK-NEXT:    add.w r0, r9, r9, lsl #1
+; CHECK-NEXT:    add.w r10, r1, r9, lsl #2
+; CHECK-NEXT:    add.w r12, r1, r9, lsl #3
+; CHECK-NEXT:    add.w r8, r1, r0, lsl #2
+; CHECK-NEXT:    add.w r1, r9, #3
+; CHECK-NEXT:    bic r1, r1, #3
 ; CHECK-NEXT:    lsl.w r11, r0, #2
-; CHECK-NEXT:    add.w r1, r5, r3, lsr #2
-; CHECK-NEXT:    str r1, [sp] @ 4-byte Spill
+; CHECK-NEXT:    subs r1, #4
+; CHECK-NEXT:    add.w r1, r5, r1, lsr #2
+; CHECK-NEXT:    str r1, [sp, #4] @ 4-byte Spill
+; CHECK-NEXT:    ldr r7, [sp, #4] @ 4-byte Reload
 ; CHECK-NEXT:  .LBB2_2: @ %for.body
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
 ; CHECK-NEXT:    @ Child Loop BB2_3 Depth 2
-; CHECK-NEXT:    ldr r6, [sp, #16] @ 4-byte Reload
+; CHECK-NEXT:    ldr r6, [sp, #12] @ 4-byte Reload
 ; CHECK-NEXT:    vmov.i32 q0, #0x0
-; CHECK-NEXT:    ldr r1, [sp] @ 4-byte Reload
-; CHECK-NEXT:    adds r0, r5, #2
-; CHECK-NEXT:    adds r2, r5, #1
-; CHECK-NEXT:    str r0, [sp, #20] @ 4-byte Spill
-; CHECK-NEXT:    mov r3, r9
+; CHECK-NEXT:    mov r3, r10
 ; CHECK-NEXT:    mov r0, r12
-; CHECK-NEXT:    mov r4, r10
+; CHECK-NEXT:    mov r4, r8
 ; CHECK-NEXT:    vmov q2, q0
 ; CHECK-NEXT:    vmov q1, q0
-; CHECK-NEXT:    dlstp.32 lr, r7
+; CHECK-NEXT:    dlstp.32 lr, r9
 ; CHECK-NEXT:  .LBB2_3: @ %vector.body
 ; CHECK-NEXT:    @ Parent Loop BB2_2 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    vldrw.u32 q3, [r6], #16
 ; CHECK-NEXT:    vldrw.u32 q4, [r3], #16
 ; CHECK-NEXT:    vfma.f32 q1, q4, q3
 ; CHECK-NEXT:    vldrw.u32 q4, [r0], #16
 ; CHECK-NEXT:    vfma.f32 q2, q4, q3
 ; CHECK-NEXT:    vldrw.u32 q4, [r4], #16
 ; CHECK-NEXT:    vfma.f32 q0, q4, q3
 ; CHECK-NEXT:    letp lr, .LBB2_3
 ; CHECK-NEXT:  @ %bb.4: @ %middle.block
 ; CHECK-NEXT:    @ in Loop: Header=BB2_2 Depth=1
 ; CHECK-NEXT:    vadd.f32 s10, s10, s11
-; CHECK-NEXT:    ldr r1, [sp, #8] @ 4-byte Reload
+; CHECK-NEXT:    adds r0, r5, #1
 ; CHECK-NEXT:    vadd.f32 s8, s8, s9
-; CHECK-NEXT:    add r9, r11
+; CHECK-NEXT:    add r10, r11
 ; CHECK-NEXT:    vadd.f32 s6, s6, s7
-; CHECK-NEXT:    add.w r0, r1, r2, lsl #2
+; CHECK-NEXT:    add.w r0, r2, r0, lsl #2
 ; CHECK-NEXT:    vadd.f32 s4, s4, s5
 ; CHECK-NEXT:    add r12, r11
 ; CHECK-NEXT:    vadd.f32 s2, s2, s3
-; CHECK-NEXT:    add r10, r11
+; CHECK-NEXT:    add r8, r11
 ; CHECK-NEXT:    vadd.f32 s0, s0, s1
 ; CHECK-NEXT:    vadd.f32 s8, s8, s10
 ; CHECK-NEXT:    vadd.f32 s4, s4, s6
 ; CHECK-NEXT:    vadd.f32 s0, s0, s2
 ; CHECK-NEXT:    vstr s8, [r0]
-; CHECK-NEXT:    add.w r0, r1, r5, lsl #2
-; CHECK-NEXT:    adds r5, #3
+; CHECK-NEXT:    add.w r0, r2, r5, lsl #2
 ; CHECK-NEXT:    vstr s4, [r0]
-; CHECK-NEXT:    ldr r0, [sp, #20] @ 4-byte Reload
-; CHECK-NEXT:    add.w r0, r1, r0, lsl #2
+; CHECK-NEXT:    adds r0, r5, #2
+; CHECK-NEXT:    adds r5, #3
+; CHECK-NEXT:    add.w r0, r2, r0, lsl #2
 ; CHECK-NEXT:    vstr s0, [r0]
-; CHECK-NEXT:    ldr r0, [sp, #12] @ 4-byte Reload
+; CHECK-NEXT:    ldr r0, [sp, #8] @ 4-byte Reload
 ; CHECK-NEXT:    cmp r5, r0
 ; CHECK-NEXT:    blo .LBB2_2
 ; CHECK-NEXT:  .LBB2_5: @ %for.cond.cleanup
-; CHECK-NEXT:    add sp, #24
+; CHECK-NEXT:    add sp, #16
 ; CHECK-NEXT:    vpop {d8, d9}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   %NumInputs = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 2
   %0 = load i32, i32* %NumInputs, align 4
   %NumFilters = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 1
   %1 = load i32, i32* %NumFilters, align 4

 ; CHECK-LABEL: DCT_mve4:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    .pad #4
 ; CHECK-NEXT:    sub sp, #4
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11}
-; CHECK-NEXT:    .pad #40
-; CHECK-NEXT:    sub sp, #40
-; CHECK-NEXT:    str r1, [sp, #24] @ 4-byte Spill
+; CHECK-NEXT:    .pad #24
+; CHECK-NEXT:    sub sp, #24
+; CHECK-NEXT:    str r1, [sp, #20] @ 4-byte Spill
 ; CHECK-NEXT:    ldr r1, [r0, #4]
-; CHECK-NEXT:    str r2, [sp, #16] @ 4-byte Spill
+; CHECK-NEXT:    str r2, [sp, #12] @ 4-byte Spill
 ; CHECK-NEXT:    subs r1, #4
-; CHECK-NEXT:    str r1, [sp, #20] @ 4-byte Spill
+; CHECK-NEXT:    str r1, [sp, #16] @ 4-byte Spill
 ; CHECK-NEXT:    cmp r1, #2
-; CHECK-NEXT:    blo.w .LBB3_5
+; CHECK-NEXT:    blo .LBB3_5
 ; CHECK-NEXT:  @ %bb.1: @ %for.body.preheader
 ; CHECK-NEXT:    ldr r2, [r0, #8]
 ; CHECK-NEXT:    movs r6, #1
 ; CHECK-NEXT:    ldr r1, [r0]
 ; CHECK-NEXT:    add.w r0, r2, r2, lsl #1
 ; CHECK-NEXT:    add.w r12, r1, r2, lsl #2
 ; CHECK-NEXT:    add.w r8, r1, r2, lsl #3
-; CHECK-NEXT:    add.w r9, r1, r2, lsl #4
-; CHECK-NEXT:    add.w r11, r1, r0, lsl #2
+; CHECK-NEXT:    add.w r10, r1, r2, lsl #4
+; CHECK-NEXT:    add.w r9, r1, r0, lsl #2
 ; CHECK-NEXT:    adds r0, r2, #3
 ; CHECK-NEXT:    bic r0, r0, #3
 ; CHECK-NEXT:    subs r0, #4
 ; CHECK-NEXT:    add.w r0, r6, r0, lsr #2
-; CHECK-NEXT:    strd r0, r2, [sp, #8] @ 8-byte Folded Spill
+; CHECK-NEXT:    strd r0, r2, [sp, #4] @ 8-byte Folded Spill
 ; CHECK-NEXT:    lsls r0, r2, #4
-; CHECK-NEXT:    ldrd r2, r7, [sp, #8] @ 8-byte Folded Reload
-; CHECK-NEXT:    str r0, [sp, #4] @ 4-byte Spill
+; CHECK-NEXT:    ldrd r2, r7, [sp, #4] @ 8-byte Folded Reload
+; CHECK-NEXT:    str r0, [sp] @ 4-byte Spill
 ; CHECK-NEXT:  .LBB3_2: @ %for.body
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
 ; CHECK-NEXT:    @ Child Loop BB3_3 Depth 2
-; CHECK-NEXT:    adds r0, r6, #3
-; CHECK-NEXT:    str r0, [sp, #36] @ 4-byte Spill
-; CHECK-NEXT:    adds r0, r6, #2
-; CHECK-NEXT:    ldr r1, [sp, #24] @ 4-byte Reload
+; CHECK-NEXT:    ldr r1, [sp, #20] @ 4-byte Reload
 ; CHECK-NEXT:    vmov.i32 q0, #0x0
-; CHECK-NEXT:    str r0, [sp, #32] @ 4-byte Spill
-; CHECK-NEXT:    adds r0, r6, #1
-; CHECK-NEXT:    str r0, [sp, #28] @ 4-byte Spill
 ; CHECK-NEXT:    mov r3, r12
 ; CHECK-NEXT:    mov r0, r8
-; CHECK-NEXT:    mov r5, r11
-; CHECK-NEXT:    mov r4, r9
+; CHECK-NEXT:    mov r5, r9
+; CHECK-NEXT:    mov r4, r10
 ; CHECK-NEXT:    vmov q1, q0
 ; CHECK-NEXT:    vmov q2, q0
 ; CHECK-NEXT:    vmov q3, q0
 ; CHECK-NEXT:    dlstp.32 lr, r7
 ; CHECK-NEXT:  .LBB3_3: @ %vector.body
 ; CHECK-NEXT:    @ Parent Loop BB3_2 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    vldrw.u32 q4, [r1], #16
 ; CHECK-NEXT:    vldrw.u32 q5, [r0], #16
 ; CHECK-NEXT:    vfma.f32 q3, q5, q4
 ; CHECK-NEXT:    vldrw.u32 q5, [r3], #16
 ; CHECK-NEXT:    vfma.f32 q2, q5, q4
 ; CHECK-NEXT:    vldrw.u32 q5, [r5], #16
 ; CHECK-NEXT:    vfma.f32 q1, q5, q4
 ; CHECK-NEXT:    vldrw.u32 q5, [r4], #16
 ; CHECK-NEXT:    vfma.f32 q0, q5, q4
 ; CHECK-NEXT:    letp lr, .LBB3_3
 ; CHECK-NEXT:  @ %bb.4: @ %middle.block
 ; CHECK-NEXT:    @ in Loop: Header=BB3_2 Depth=1
 ; CHECK-NEXT:    vadd.f32 s14, s14, s15
-; CHECK-NEXT:    ldr r0, [sp, #28] @ 4-byte Reload
+; CHECK-NEXT:    ldr r1, [sp, #12] @ 4-byte Reload
 ; CHECK-NEXT:    vadd.f32 s12, s12, s13
-; CHECK-NEXT:    ldr r1, [sp, #16] @ 4-byte Reload
+; CHECK-NEXT:    adds r0, r6, #1
 ; CHECK-NEXT:    vadd.f32 s10, s10, s11
 ; CHECK-NEXT:    vadd.f32 s8, s8, s9
 ; CHECK-NEXT:    add.w r0, r1, r0, lsl #2
 ; CHECK-NEXT:    vadd.f32 s6, s6, s7
 ; CHECK-NEXT:    vadd.f32 s4, s4, s5
 ; CHECK-NEXT:    vadd.f32 s2, s2, s3
 ; CHECK-NEXT:    vadd.f32 s0, s0, s1
 ; CHECK-NEXT:    vadd.f32 s12, s12, s14
 ; CHECK-NEXT:    vadd.f32 s8, s8, s10
 ; CHECK-NEXT:    vadd.f32 s4, s4, s6
 ; CHECK-NEXT:    vadd.f32 s0, s0, s2
 ; CHECK-NEXT:    vstr s12, [r0]
 ; CHECK-NEXT:    add.w r0, r1, r6, lsl #2
-; CHECK-NEXT:    adds r6, #4
 ; CHECK-NEXT:    vstr s8, [r0]
-; CHECK-NEXT:    ldr r0, [sp, #32] @ 4-byte Reload
+; CHECK-NEXT:    adds r0, r6, #2
 ; CHECK-NEXT:    add.w r0, r1, r0, lsl #2
 ; CHECK-NEXT:    vstr s4, [r0]
-; CHECK-NEXT:    ldr r0, [sp, #36] @ 4-byte Reload
+; CHECK-NEXT:    adds r0, r6, #3
+; CHECK-NEXT:    adds r6, #4
 ; CHECK-NEXT:    add.w r0, r1, r0, lsl #2
 ; CHECK-NEXT:    vstr s0, [r0]
-; CHECK-NEXT:    ldr r0, [sp, #4] @ 4-byte Reload
+; CHECK-NEXT:    ldr r0, [sp] @ 4-byte Reload
 ; CHECK-NEXT:    add r12, r0
 ; CHECK-NEXT:    add r8, r0
-; CHECK-NEXT:    add r11, r0
 ; CHECK-NEXT:    add r9, r0
-; CHECK-NEXT:    ldr r0, [sp, #20] @ 4-byte Reload
+; CHECK-NEXT:    add r10, r0
+; CHECK-NEXT:    ldr r0, [sp, #16] @ 4-byte Reload
 ; CHECK-NEXT:    cmp r6, r0
 ; CHECK-NEXT:    blo .LBB3_2
 ; CHECK-NEXT:  .LBB3_5: @ %for.cond.cleanup
-; CHECK-NEXT:    add sp, #40
+; CHECK-NEXT:    add sp, #24
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   %NumInputs = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 2
   %0 = load i32, i32* %NumInputs, align 4
   %NumFilters = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 1
   %1 = load i32, i32* %NumFilters, align 4

 ; CHECK-LABEL: DCT_mve5:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    .pad #4
 ; CHECK-NEXT:    sub sp, #4
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
-; CHECK-NEXT:    .pad #32
-; CHECK-NEXT:    sub sp, #32
-; CHECK-NEXT:    str r1, [sp, #20] @ 4-byte Spill
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
+; CHECK-NEXT:    str r1, [sp, #12] @ 4-byte Spill
 ; CHECK-NEXT:    ldr r1, [r0, #4]
 ; CHECK-NEXT:    subs r1, #5
-; CHECK-NEXT:    str r1, [sp, #16] @ 4-byte Spill
+; CHECK-NEXT:    str r1, [sp, #8] @ 4-byte Spill
 ; CHECK-NEXT:    cmp r1, #2
 ; CHECK-NEXT:    blo.w .LBB4_5
 ; CHECK-NEXT:  @ %bb.1: @ %for.body.preheader
-; CHECK-NEXT:    ldr r3, [r0, #8]
-; CHECK-NEXT:    ldr r1, [r0]
-; CHECK-NEXT:    adds r0, r3, #3
-; CHECK-NEXT:    str r3, [sp, #12] @ 4-byte Spill
+; CHECK-NEXT:    ldr r1, [r0, #8]
+; CHECK-NEXT:    ldr r3, [r0]
+; CHECK-NEXT:    adds r0, r1, #3
 ; CHECK-NEXT:    bic r0, r0, #3
-; CHECK-NEXT:    add.w r8, r1, r3, lsl #2
-; CHECK-NEXT:    subs r1, r0, #4
+; CHECK-NEXT:    add.w r12, r3, r1, lsl #2
+; CHECK-NEXT:    subs r3, r0, #4
 ; CHECK-NEXT:    movs r0, #1
-; CHECK-NEXT:    lsls r5, r3, #2
-; CHECK-NEXT:    add.w r1, r0, r1, lsr #2
-; CHECK-NEXT:    str r1, [sp, #8] @ 4-byte Spill
-; CHECK-NEXT:    add.w r1, r3, r3, lsl #2
-; CHECK-NEXT:    lsls r1, r1, #2
-; CHECK-NEXT:    str r1, [sp, #4] @ 4-byte Spill
+; CHECK-NEXT:    lsls r5, r1, #2
+; CHECK-NEXT:    add.w r3, r0, r3, lsr #2
+; CHECK-NEXT:    str r3, [sp, #4] @ 4-byte Spill
+; CHECK-NEXT:    add.w r3, r1, r1, lsl #2
+; CHECK-NEXT:    lsls r3, r3, #2
+; CHECK-NEXT:    str r3, [sp] @ 4-byte Spill
 ; CHECK-NEXT:  .LBB4_2: @ %for.body
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
 ; CHECK-NEXT:    @ Child Loop BB4_3 Depth 2
-; CHECK-NEXT:    ldr r7, [sp, #12] @ 4-byte Reload
-; CHECK-NEXT:    adds r1, r0, #4
-; CHECK-NEXT:    ldr r4, [sp, #20] @ 4-byte Reload
+; CHECK-NEXT:    ldr r4, [sp, #12] @ 4-byte Reload
 ; CHECK-NEXT:    vmov.i32 q1, #0x0
-; CHECK-NEXT:    ldr r6, [sp, #8] @ 4-byte Reload
-; CHECK-NEXT:    add.w r10, r0, #2
-; CHECK-NEXT:    str r1, [sp, #28] @ 4-byte Spill
-; CHECK-NEXT:    adds r1, r0, #3
+; CHECK-NEXT:    ldr r7, [sp, #4] @ 4-byte Reload
 ; CHECK-NEXT:    add.w r11, r0, #1
-; CHECK-NEXT:    str r1, [sp, #24] @ 4-byte Spill
-; CHECK-NEXT:    mov r3, r8
+; CHECK-NEXT:    mov r3, r12
 ; CHECK-NEXT:    vmov q0, q1
 ; CHECK-NEXT:    vmov q3, q1
 ; CHECK-NEXT:    vmov q2, q1
 ; CHECK-NEXT:    vmov q4, q1
-; CHECK-NEXT:    dlstp.32 lr, r7
+; CHECK-NEXT:    dlstp.32 lr, r1
 ; CHECK-NEXT:  .LBB4_3: @ %vector.body
 ; CHECK-NEXT:    @ Parent Loop BB4_2 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    add.w r9, r3, r5
 ; CHECK-NEXT:    vldrw.u32 q5, [r4], #16
 ; CHECK-NEXT:    vldrw.u32 q6, [r3], #16
 ; CHECK-NEXT:    vfma.f32 q3, q6, q5
-; CHECK-NEXT:    add.w r12, r9, r5
+; CHECK-NEXT:    add.w r10, r9, r5
 ; CHECK-NEXT:    vldrw.u32 q6, [r9]
 ; CHECK-NEXT:    vfma.f32 q4, q6, q5
-; CHECK-NEXT:    add.w r6, r12, r5
-; CHECK-NEXT:    vldrw.u32 q6, [r12]
+; CHECK-NEXT:    add.w r6, r10, r5
+; CHECK-NEXT:    vldrw.u32 q6, [r10]
 ; CHECK-NEXT:    vfma.f32 q2, q6, q5
 ; CHECK-NEXT:    adds r7, r6, r5
 ; CHECK-NEXT:    vldrw.u32 q6, [r6]
 ; CHECK-NEXT:    vfma.f32 q0, q6, q5
 ; CHECK-NEXT:    vldrw.u32 q6, [r7]
 ; CHECK-NEXT:    vfma.f32 q1, q6, q5
 ; CHECK-NEXT:    letp lr, .LBB4_3
 ; CHECK-NEXT:  @ %bb.4: @ %middle.block
 ; CHECK-NEXT:    @ in Loop: Header=BB4_2 Depth=1
 ; CHECK-NEXT:    vadd.f32 s18, s18, s19
-; CHECK-NEXT:    add.w r1, r2, r11, lsl #2
+; CHECK-NEXT:    add.w r3, r2, r11, lsl #2
 ; CHECK-NEXT:    vadd.f32 s16, s16, s17
 ; CHECK-NEXT:    vadd.f32 s14, s14, s15
 ; CHECK-NEXT:    vadd.f32 s12, s12, s13
 ; CHECK-NEXT:    vadd.f32 s6, s6, s7
 ; CHECK-NEXT:    vadd.f32 s4, s4, s5
 ; CHECK-NEXT:    vadd.f32 s10, s10, s11
 ; CHECK-NEXT:    vadd.f32 s8, s8, s9
 ; CHECK-NEXT:    vadd.f32 s0, s0, s1
 ; CHECK-NEXT:    vadd.f32 s1, s16, s18
-; CHECK-NEXT:    vadd.f32 s2, s2, s3
 ; CHECK-NEXT:    vadd.f32 s12, s12, s14
+; CHECK-NEXT:    vadd.f32 s2, s2, s3
 ; CHECK-NEXT:    vadd.f32 s4, s4, s6
 ; CHECK-NEXT:    vadd.f32 s6, s8, s10
-; CHECK-NEXT:    vstr s1, [r1]
-; CHECK-NEXT:    add.w r1, r2, r0, lsl #2
+; CHECK-NEXT:    vstr s1, [r3]
+; CHECK-NEXT:    add.w r3, r2, r0, lsl #2
+; CHECK-NEXT:    vstr s12, [r3]
+; CHECK-NEXT:    adds r3, r0, #2
 ; CHECK-NEXT:    vadd.f32 s0, s0, s2
+; CHECK-NEXT:    add.w r3, r2, r3, lsl #2
+; CHECK-NEXT:    vstr s6, [r3]
+; CHECK-NEXT:    adds r3, r0, #3
+; CHECK-NEXT:    add.w r3, r2, r3, lsl #2
+; CHECK-NEXT:    vstr s0, [r3]
+; CHECK-NEXT:    adds r3, r0, #4
 ; CHECK-NEXT:    adds r0, #5
-; CHECK-NEXT:    vstr s12, [r1]
-; CHECK-NEXT:    add.w r1, r2, r10, lsl #2
-; CHECK-NEXT:    vstr s6, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #24] @ 4-byte Reload
-; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
-; CHECK-NEXT:    vstr s0, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #28] @ 4-byte Reload
-; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
-; CHECK-NEXT:    vstr s4, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #4] @ 4-byte Reload
-; CHECK-NEXT:    add r8, r1
-; CHECK-NEXT:    ldr r1, [sp, #16] @ 4-byte Reload
-; CHECK-NEXT:    cmp r0, r1
-; CHECK-NEXT:    blo.w .LBB4_2
+; CHECK-NEXT:    add.w r3, r2, r3, lsl #2
+; CHECK-NEXT:    vstr s4, [r3]
+; CHECK-NEXT:    ldr r3, [sp] @ 4-byte Reload
+; CHECK-NEXT:    add r12, r3
+; CHECK-NEXT:    ldr r3, [sp, #8] @ 4-byte Reload
+; CHECK-NEXT:    cmp r0, r3
+; CHECK-NEXT:    blo .LBB4_2
 ; CHECK-NEXT:  .LBB4_5: @ %for.cond.cleanup
-; CHECK-NEXT:    add sp, #32
+; CHECK-NEXT:    add sp, #16
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   %NumInputs = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 2
   %0 = load i32, i32* %NumInputs, align 4
   %NumFilters = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 1
   %1 = load i32, i32* %NumFilters, align 4

 ; CHECK-LABEL: DCT_mve6:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    .pad #4
 ; CHECK-NEXT:    sub sp, #4
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    .pad #32
-; CHECK-NEXT:    sub sp, #32
-; CHECK-NEXT:    str r1, [sp, #16] @ 4-byte Spill
+; CHECK-NEXT:    .pad #16
+; CHECK-NEXT:    sub sp, #16
+; CHECK-NEXT:    str r1, [sp, #12] @ 4-byte Spill
 ; CHECK-NEXT:    ldr r1, [r0, #4]
 ; CHECK-NEXT:    subs r1, #6
-; CHECK-NEXT:    str r1, [sp, #12] @ 4-byte Spill
+; CHECK-NEXT:    str r1, [sp, #8] @ 4-byte Spill
 ; CHECK-NEXT:    cmp r1, #2
 ; CHECK-NEXT:    blo.w .LBB5_5
 ; CHECK-NEXT:  @ %bb.1: @ %for.body.preheader
-; CHECK-NEXT:    ldr r3, [r0, #8]
+; CHECK-NEXT:    ldr.w r9, [r0, #8]
 ; CHECK-NEXT:    ldr r1, [r0]
-; CHECK-NEXT:    adds r0, r3, #3
-; CHECK-NEXT:    str r3, [sp, #8] @ 4-byte Spill
+; CHECK-NEXT:    add.w r0, r9, #3
 ; CHECK-NEXT:    bic r0, r0, #3
-; CHECK-NEXT:    add.w r8, r1, r3, lsl #2
+; CHECK-NEXT:    add.w r12, r1, r9, lsl #2
 ; CHECK-NEXT:    subs r1, r0, #4
 ; CHECK-NEXT:    movs r0, #1
-; CHECK-NEXT:    lsls r5, r3, #2
+; CHECK-NEXT:    lsl.w r5, r9, #2
 ; CHECK-NEXT:    add.w r1, r0, r1, lsr #2
 ; CHECK-NEXT:    str r1, [sp, #4] @ 4-byte Spill
-; CHECK-NEXT:    add.w r1, r3, r3, lsl #1
+; CHECK-NEXT:    add.w r1, r9, r9, lsl #1
 ; CHECK-NEXT:    lsls r1, r1, #3
 ; CHECK-NEXT:    str r1, [sp] @ 4-byte Spill
 ; CHECK-NEXT:  .LBB5_2: @ %for.body
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
 ; CHECK-NEXT:    @ Child Loop BB5_3 Depth 2
-; CHECK-NEXT:    adds r1, r0, #5
-; CHECK-NEXT:    str r1, [sp, #28] @ 4-byte Spill
-; CHECK-NEXT:    adds r1, r0, #4
-; CHECK-NEXT:    str r1, [sp, #24] @ 4-byte Spill
-; CHECK-NEXT:    adds r1, r0, #3
-; CHECK-NEXT:    ldr r7, [sp, #8] @ 4-byte Reload
-; CHECK-NEXT:    str r1, [sp, #20] @ 4-byte Spill
+; CHECK-NEXT:    ldr r1, [sp, #12] @ 4-byte Reload
 ; CHECK-NEXT:    vmov.i32 q1, #0x0
-; CHECK-NEXT:    ldr r1, [sp, #16] @ 4-byte Reload
-; CHECK-NEXT:    add.w r11, r0, #2
-; CHECK-NEXT:    ldr r6, [sp, #4] @ 4-byte Reload
+; CHECK-NEXT:    ldr r7, [sp, #4] @ 4-byte Reload
 ; CHECK-NEXT:    adds r4, r0, #1
-; CHECK-NEXT:    mov r3, r8
+; CHECK-NEXT:    mov r3, r12
 ; CHECK-NEXT:    vmov q3, q1
 ; CHECK-NEXT:    vmov q4, q1
 ; CHECK-NEXT:    vmov q0, q1
 ; CHECK-NEXT:    vmov q5, q1
 ; CHECK-NEXT:    vmov q2, q1
-; CHECK-NEXT:    dlstp.32 lr, r7
+; CHECK-NEXT:    dlstp.32 lr, r9
 ; CHECK-NEXT:  .LBB5_3: @ %vector.body
 ; CHECK-NEXT:    @ Parent Loop BB5_2 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    add.w r12, r3, r5
+; CHECK-NEXT:    add.w r10, r3, r5
 ; CHECK-NEXT:    vldrw.u32 q6, [r1], #16
 ; CHECK-NEXT:    vldrw.u32 q7, [r3], #16
 ; CHECK-NEXT:    vfma.f32 q4, q7, q6
-; CHECK-NEXT:    add.w r10, r12, r5
-; CHECK-NEXT:    vldrw.u32 q7, [r12]
-; CHECK-NEXT:    vfma.f32 q5, q7, q6
-; CHECK-NEXT:    add.w r6, r10, r5
+; CHECK-NEXT:    add.w r11, r10, r5
 ; CHECK-NEXT:    vldrw.u32 q7, [r10]
+; CHECK-NEXT:    vfma.f32 q5, q7, q6
+; CHECK-NEXT:    add.w r6, r11, r5
+; CHECK-NEXT:    vldrw.u32 q7, [r11]
 ; CHECK-NEXT:    vfma.f32 q2, q7, q6
 ; CHECK-NEXT:    adds r7, r6, r5
 ; CHECK-NEXT:    vldrw.u32 q7, [r6]
 ; CHECK-NEXT:    vfma.f32 q0, q7, q6
 ; CHECK-NEXT:    adds r6, r7, r5
 ; CHECK-NEXT:    vldrw.u32 q7, [r7]
 ; CHECK-NEXT:    vfma.f32 q3, q7, q6
 ; CHECK-NEXT:    vldrw.u32 q7, [r6]

 ; CHECK-NEXT:    vadd.f32 s3, s16, s18
 ; CHECK-NEXT:    vadd.f32 s4, s4, s5
 ; CHECK-NEXT:    vadd.f32 s8, s8, s10
 ; CHECK-NEXT:    vadd.f32 s14, s14, s15
 ; CHECK-NEXT:    vadd.f32 s12, s12, s13
 ; CHECK-NEXT:    vstr s1, [r1]
 ; CHECK-NEXT:    add.w r1, r2, r0, lsl #2
 ; CHECK-NEXT:    vadd.f32 s0, s0, s2
-; CHECK-NEXT:    adds r0, #6
 ; CHECK-NEXT:    vstr s3, [r1]
-; CHECK-NEXT:    add.w r1, r2, r11, lsl #2
+; CHECK-NEXT:    adds r1, r0, #2
 ; CHECK-NEXT:    vadd.f32 s4, s4, s6
+; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
 ; CHECK-NEXT:    vstr s8, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #20] @ 4-byte Reload
+; CHECK-NEXT:    adds r1, r0, #3
 ; CHECK-NEXT:    vadd.f32 s6, s12, s14
 ; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
 ; CHECK-NEXT:    vstr s0, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #24] @ 4-byte Reload
+; CHECK-NEXT:    adds r1, r0, #4
 ; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
 ; CHECK-NEXT:    vstr s6, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #28] @ 4-byte Reload
+; CHECK-NEXT:    adds r1, r0, #5
+; CHECK-NEXT:    adds r0, #6
 ; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
 ; CHECK-NEXT:    vstr s4, [r1]
 ; CHECK-NEXT:    ldr r1, [sp] @ 4-byte Reload
-; CHECK-NEXT:    add r8, r1
-; CHECK-NEXT:    ldr r1, [sp, #12] @ 4-byte Reload
+; CHECK-NEXT:    add r12, r1
+; CHECK-NEXT:    ldr r1, [sp, #8] @ 4-byte Reload
 ; CHECK-NEXT:    cmp r0, r1
 ; CHECK-NEXT:    blo.w .LBB5_2
 ; CHECK-NEXT:  .LBB5_5: @ %for.cond.cleanup
-; CHECK-NEXT:    add sp, #32
+; CHECK-NEXT:    add sp, #16
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   %NumInputs = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 2
   %0 = load i32, i32* %NumInputs, align 4
   %NumFilters = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 1
   %1 = load i32, i32* %NumFilters, align 4

 ; CHECK-LABEL: DCT_mve7:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    .pad #4
 ; CHECK-NEXT:    sub sp, #4
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    .pad #72
-; CHECK-NEXT:    sub sp, #72
-; CHECK-NEXT:    str r1, [sp, #20] @ 4-byte Spill
+; CHECK-NEXT:    .pad #48
+; CHECK-NEXT:    sub sp, #48
+; CHECK-NEXT:    str r1, [sp, #12] @ 4-byte Spill
 ; CHECK-NEXT:    ldr r1, [r0, #4]
 ; CHECK-NEXT:    subs r1, #7
-; CHECK-NEXT:    str r1, [sp, #16] @ 4-byte Spill
+; CHECK-NEXT:    str r1, [sp, #8] @ 4-byte Spill
 ; CHECK-NEXT:    cmp r1, #2
 ; CHECK-NEXT:    blo.w .LBB6_5
 ; CHECK-NEXT:  @ %bb.1: @ %for.body.preheader
-; CHECK-NEXT:    ldr r3, [r0, #8]
+; CHECK-NEXT:    ldr.w r10, [r0, #8]
 ; CHECK-NEXT:    ldr r1, [r0]
-; CHECK-NEXT:    adds r0, r3, #3
-; CHECK-NEXT:    str r3, [sp, #12] @ 4-byte Spill
+; CHECK-NEXT:    add.w r0, r10, #3
 ; CHECK-NEXT:    bic r0, r0, #3
-; CHECK-NEXT:    add.w r9, r1, r3, lsl #2
+; CHECK-NEXT:    add.w r8, r1, r10, lsl #2
 ; CHECK-NEXT:    subs r1, r0, #4
 ; CHECK-NEXT:    movs r0, #1
-; CHECK-NEXT:    lsls r5, r3, #2
+; CHECK-NEXT:    lsl.w r5, r10, #2
 ; CHECK-NEXT:    add.w r1, r0, r1, lsr #2
-; CHECK-NEXT:    str r1, [sp, #8] @ 4-byte Spill
-; CHECK-NEXT:    rsb r1, r3, r3, lsl #3
-; CHECK-NEXT:    lsls r1, r1, #2
 ; CHECK-NEXT:    str r1, [sp, #4] @ 4-byte Spill
+; CHECK-NEXT:    rsb r1, r10, r10, lsl #3
+; CHECK-NEXT:    lsls r1, r1, #2
+; CHECK-NEXT:    str r1, [sp] @ 4-byte Spill
 ; CHECK-NEXT:  .LBB6_2: @ %for.body
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
 ; CHECK-NEXT:    @ Child Loop BB6_3 Depth 2
-; CHECK-NEXT:    adds r1, r0, #6
-; CHECK-NEXT:    str r1, [sp, #36] @ 4-byte Spill
-; CHECK-NEXT:    adds r1, r0, #5
-; CHECK-NEXT:    str r1, [sp, #32] @ 4-byte Spill
-; CHECK-NEXT:    adds r1, r0, #4
-; CHECK-NEXT:    str r1, [sp, #28] @ 4-byte Spill
-; CHECK-NEXT:    adds r1, r0, #3
-; CHECK-NEXT:    ldr r7, [sp, #12] @ 4-byte Reload
-; CHECK-NEXT:    str r1, [sp, #24] @ 4-byte Spill
+; CHECK-NEXT:    ldr r1, [sp, #12] @ 4-byte Reload
 ; CHECK-NEXT:    vmov.i32 q2, #0x0
-; CHECK-NEXT:    ldr r1, [sp, #20] @ 4-byte Reload
+; CHECK-NEXT:    ldr r7, [sp, #4] @ 4-byte Reload
 ; CHECK-NEXT:    adds r4, r0, #2
-; CHECK-NEXT:    ldr r6, [sp, #8] @ 4-byte Reload
-; CHECK-NEXT:    add.w r8, r0, #1
-; CHECK-NEXT:    mov r3, r9
+; CHECK-NEXT:    add.w r12, r0, #1
+; CHECK-NEXT:    mov r3, r8
 ; CHECK-NEXT:    vmov q4, q2
 ; CHECK-NEXT:    vmov q5, q2
 ; CHECK-NEXT:    vmov q3, q2
 ; CHECK-NEXT:    vmov q6, q2
 ; CHECK-NEXT:    vmov q1, q2
-; CHECK-NEXT:    mov r12, r7
-; CHECK-NEXT:    vstrw.32 q2, [sp, #56] @ 16-byte Spill
-; CHECK-NEXT:    dls lr, r6
+; CHECK-NEXT:    mov r9, r10
+; CHECK-NEXT:    vstrw.32 q2, [sp, #32] @ 16-byte Spill
+; CHECK-NEXT:    dls lr, r7
 ; CHECK-NEXT:  .LBB6_3: @ %vector.body
 ; CHECK-NEXT:    @ Parent Loop BB6_2 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    add.w r10, r3, r5
-; CHECK-NEXT:    vctp.32 r12
+; CHECK-NEXT:    add.w r11, r3, r5
+; CHECK-NEXT:    vctp.32 r9
 ; CHECK-NEXT:    vpsttt
 ; CHECK-NEXT:    vldrwt.u32 q7, [r1], #16
 ; CHECK-NEXT:    vldrwt.u32 q0, [r3], #16
 ; CHECK-NEXT:    vfmat.f32 q5, q0, q7
-; CHECK-NEXT:    add.w r11, r10, r5
+; CHECK-NEXT:    add.w r6, r11, r5
 ; CHECK-NEXT:    vpstt
-; CHECK-NEXT:    vldrwt.u32 q0, [r10]
+; CHECK-NEXT:    vldrwt.u32 q0, [r11]
 ; CHECK-NEXT:    vfmat.f32 q6, q0, q7
-; CHECK-NEXT:    vstrw.32 q6, [sp, #40] @ 16-byte Spill
+; CHECK-NEXT:    vstrw.32 q6, [sp, #16] @ 16-byte Spill
 ; CHECK-NEXT:    vpstt
-; CHECK-NEXT:    vldrwt.u32 q0, [r11]
+; CHECK-NEXT:    vldrwt.u32 q0, [r6]
 ; CHECK-NEXT:    vfmat.f32 q1, q0, q7
-; CHECK-NEXT:    add.w r6, r11, r5
+; CHECK-NEXT:    adds r7, r6, r5
 ; CHECK-NEXT:    vmov q6, q5
 ; CHECK-NEXT:    vmov q5, q4
 ; CHECK-NEXT:    vmov q4, q3
 ; CHECK-NEXT:    vpst
-; CHECK-NEXT:    vldrwt.u32 q0, [r6]
+; CHECK-NEXT:    vldrwt.u32 q0, [r7]
 ; CHECK-NEXT:    vmov q3, q1
-; CHECK-NEXT:    vldrw.u32 q1, [sp, #56] @ 16-byte Reload
+; CHECK-NEXT:    vldrw.u32 q1, [sp, #32] @ 16-byte Reload
 ; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vfmat.f32 q1, q0, q7
-; CHECK-NEXT:    adds r7, r6, r5
-; CHECK-NEXT:    vstrw.32 q1, [sp, #56] @ 16-byte Spill
+; CHECK-NEXT:    adds r6, r7, r5
+; CHECK-NEXT:    vstrw.32 q1, [sp, #32] @ 16-byte Spill
 ; CHECK-NEXT:    vmov q1, q3
 ; CHECK-NEXT:    vmov q3, q4
 ; CHECK-NEXT:    vmov q4, q5
 ; CHECK-NEXT:    vmov q5, q6
-; CHECK-NEXT:    vldrw.u32 q6, [sp, #40] @ 16-byte Reload
-; CHECK-NEXT:    sub.w r12, r12, #4
-; CHECK-NEXT:    adds r6, r7, r5
+; CHECK-NEXT:    vldrw.u32 q6, [sp, #16] @ 16-byte Reload
+; CHECK-NEXT:    sub.w r9, r9, #4
+; CHECK-NEXT:    adds r7, r6, r5
 ; CHECK-NEXT:    vpstt
-; CHECK-NEXT:    vldrwt.u32 q0, [r7]
+; CHECK-NEXT:    vldrwt.u32 q0, [r6]
 ; CHECK-NEXT:    vfmat.f32 q3, q0, q7
-; CHECK-NEXT:    adds r7, r6, r5
+; CHECK-NEXT:    adds r6, r7, r5
 ; CHECK-NEXT:    vpstttt
-; CHECK-NEXT:    vldrwt.u32 q0, [r6]
-; CHECK-NEXT:    vfmat.f32 q4, q0, q7
 ; CHECK-NEXT:    vldrwt.u32 q0, [r7]
+; CHECK-NEXT:    vfmat.f32 q4, q0, q7
+; CHECK-NEXT:    vldrwt.u32 q0, [r6]
 ; CHECK-NEXT:    vfmat.f32 q2, q0, q7
 ; CHECK-NEXT:    le lr, .LBB6_3
 ; CHECK-NEXT:  @ %bb.4: @ %middle.block
 ; CHECK-NEXT:    @ in Loop: Header=BB6_2 Depth=1
 ; CHECK-NEXT:    vadd.f32 s0, s26, s27
-; CHECK-NEXT:    add.w r1, r2, r8, lsl #2
+; CHECK-NEXT:    add.w r1, r2, r12, lsl #2
 ; CHECK-NEXT:    vadd.f32 s2, s24, s25
 ; CHECK-NEXT:    vadd.f32 s1, s22, s23
 ; CHECK-NEXT:    vadd.f32 s3, s20, s21
 ; CHECK-NEXT:    vadd.f32 s6, s6, s7
 ; CHECK-NEXT:    vadd.f32 s4, s4, s5
 ; CHECK-NEXT:    vadd.f32 s10, s10, s11
 ; CHECK-NEXT:    vadd.f32 s8, s8, s9
-; CHECK-NEXT:    vadd.f32 s0, s2, s0
 ; CHECK-NEXT:    vadd.f32 s9, s18, s19
 ; CHECK-NEXT:    vadd.f32 s11, s16, s17
-; CHECK-NEXT:    vldrw.u32 q4, [sp, #56] @ 16-byte Reload
-; CHECK-NEXT:    vadd.f32 s2, s3, s1
+; CHECK-NEXT:    vldrw.u32 q4, [sp, #32] @ 16-byte Reload
+; CHECK-NEXT:    vadd.f32 s0, s2, s0
 ; CHECK-NEXT:    vadd.f32 s5, s18, s19
 ; CHECK-NEXT:    vadd.f32 s7, s16, s17
+; CHECK-NEXT:    vadd.f32 s2, s3, s1
 ; CHECK-NEXT:    vadd.f32 s4, s4, s6
-; CHECK-NEXT:    vstr s0, [r1]
-; CHECK-NEXT:    add.w r1, r2, r0, lsl #2
 ; CHECK-NEXT:    vadd.f32 s14, s14, s15
-; CHECK-NEXT:    adds r0, #7
 ; CHECK-NEXT:    vadd.f32 s12, s12, s13
-; CHECK-NEXT:    vstr s2, [r1]
-; CHECK-NEXT:    add.w r1, r2, r4, lsl #2
+; CHECK-NEXT:    vstr s0, [r1]
+; CHECK-NEXT:    add.w r1, r2, r0, lsl #2
 ; CHECK-NEXT:    vadd.f32 s8, s8, s10
 ; CHECK-NEXT:    vadd.f32 s6, s7, s5
-; CHECK-NEXT:    vstr s4, [r1]
+; CHECK-NEXT:    vstr s2, [r1]
+; CHECK-NEXT:    add.w r1, r2, r4, lsl #2
 ; CHECK-NEXT:    vadd.f32 s10, s11, s9
-; CHECK-NEXT:    ldr r1, [sp, #24] @ 4-byte Reload
+; CHECK-NEXT:    vstr s4, [r1]
+; CHECK-NEXT:    adds r1, r0, #3
 ; CHECK-NEXT:    vadd.f32 s12, s12, s14
 ; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
 ; CHECK-NEXT:    vstr s6, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #28] @ 4-byte Reload
+; CHECK-NEXT:    adds r1, r0, #4
 ; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
 ; CHECK-NEXT:    vstr s12, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #32] @ 4-byte Reload
+; CHECK-NEXT:    adds r1, r0, #5
 ; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
 ; CHECK-NEXT:    vstr s10, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #36] @ 4-byte Reload
+; CHECK-NEXT:    adds r1, r0, #6
+; CHECK-NEXT:    adds r0, #7
 ; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
 ; CHECK-NEXT:    vstr s8, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #4] @ 4-byte Reload
-; CHECK-NEXT:    add r9, r1
-; CHECK-NEXT:    ldr r1, [sp, #16] @ 4-byte Reload
+; CHECK-NEXT:    ldr r1, [sp] @ 4-byte Reload
+; CHECK-NEXT:    add r8, r1
+; CHECK-NEXT:    ldr r1, [sp, #8] @ 4-byte Reload
 ; CHECK-NEXT:    cmp r0, r1
 ; CHECK-NEXT:    blo.w .LBB6_2
 ; CHECK-NEXT:  .LBB6_5: @ %for.cond.cleanup
-; CHECK-NEXT:    add sp, #72
+; CHECK-NEXT:    add sp, #48
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   %NumInputs = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 2
   %0 = load i32, i32* %NumInputs, align 4
   %NumFilters = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 1
   %1 = load i32, i32* %NumFilters, align 4

 ; CHECK-LABEL: DCT_mve8:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    push.w {r4, r5, r6, r7, r8, r9, r10, r11, lr}
 ; CHECK-NEXT:    .pad #4
 ; CHECK-NEXT:    sub sp, #4
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    .pad #88
-; CHECK-NEXT:    sub sp, #88
-; CHECK-NEXT:    str r1, [sp, #20] @ 4-byte Spill
+; CHECK-NEXT:    .pad #64
+; CHECK-NEXT:    sub sp, #64
+; CHECK-NEXT:    str r1, [sp, #12] @ 4-byte Spill
 ; CHECK-NEXT:    ldr r1, [r0, #4]
 ; CHECK-NEXT:    subs r1, #8
-; CHECK-NEXT:    str r1, [sp, #16] @ 4-byte Spill
+; CHECK-NEXT:    str r1, [sp, #8] @ 4-byte Spill
 ; CHECK-NEXT:    cmp r1, #2
 ; CHECK-NEXT:    blo.w .LBB7_5
 ; CHECK-NEXT:  @ %bb.1: @ %for.body.preheader
-; CHECK-NEXT:    ldr r3, [r0, #8]
+; CHECK-NEXT:    ldr.w r11, [r0, #8]
 ; CHECK-NEXT:    ldr r1, [r0]
-; CHECK-NEXT:    adds r0, r3, #3
-; CHECK-NEXT:    str r3, [sp, #12] @ 4-byte Spill
+; CHECK-NEXT:    add.w r0, r11, #3
 ; CHECK-NEXT:    bic r0, r0, #3
-; CHECK-NEXT:    add.w r12, r1, r3, lsl #2
+; CHECK-NEXT:    add.w r9, r1, r11, lsl #2
 ; CHECK-NEXT:    subs r1, r0, #4
 ; CHECK-NEXT:    movs r0, #1
-; CHECK-NEXT:    lsls r5, r3, #2
+; CHECK-NEXT:    lsl.w r5, r11, #2
 ; CHECK-NEXT:    add.w r1, r0, r1, lsr #2
-; CHECK-NEXT:    str r1, [sp, #8] @ 4-byte Spill
-; CHECK-NEXT:    lsls r1, r3, #5
 ; CHECK-NEXT:    str r1, [sp, #4] @ 4-byte Spill
+; CHECK-NEXT:    lsl.w r1, r11, #5
+; CHECK-NEXT:    str r1, [sp] @ 4-byte Spill
 ; CHECK-NEXT:  .LBB7_2: @ %for.body
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
 ; CHECK-NEXT:    @ Child Loop BB7_3 Depth 2
-; CHECK-NEXT:    adds r1, r0, #7
-; CHECK-NEXT:    str r1, [sp, #36] @ 4-byte Spill
-; CHECK-NEXT:    adds r1, r0, #6
-; CHECK-NEXT:    str r1, [sp, #32] @ 4-byte Spill
-; CHECK-NEXT:    adds r1, r0, #5
-; CHECK-NEXT:    ldr r7, [sp, #12] @ 4-byte Reload
-; CHECK-NEXT:    str r1, [sp, #28] @ 4-byte Spill
-; CHECK-NEXT:    adds r1, r0, #4
-; CHECK-NEXT:    ldr.w r9, [sp, #20] @ 4-byte Reload
+; CHECK-NEXT:    ldr r1, [sp, #12] @ 4-byte Reload
 ; CHECK-NEXT:    vmov.i32 q3, #0x0
-; CHECK-NEXT:    ldr r6, [sp, #8] @ 4-byte Reload
+; CHECK-NEXT:    ldr r7, [sp, #4] @ 4-byte Reload
 ; CHECK-NEXT:    adds r4, r0, #3
-; CHECK-NEXT:    str r1, [sp, #24] @ 4-byte Spill
-; CHECK-NEXT:    add.w r8, r0, #2
-; CHECK-NEXT:    adds r1, r0, #1
-; CHECK-NEXT:    mov r3, r12
+; CHECK-NEXT:    add.w r12, r0, #2
+; CHECK-NEXT:    add.w r8, r0, #1
+; CHECK-NEXT:    mov r3, r9
 ; CHECK-NEXT:    vmov q5, q3
 ; CHECK-NEXT:    vmov q6, q3
 ; CHECK-NEXT:    vmov q4, q3
 ; CHECK-NEXT:    vmov q7, q3
 ; CHECK-NEXT:    vmov q2, q3
-; CHECK-NEXT:    mov r10, r7
-; CHECK-NEXT:    vstrw.32 q3, [sp, #56] @ 16-byte Spill
-; CHECK-NEXT:    vstrw.32 q3, [sp, #72] @ 16-byte Spill
-; CHECK-NEXT:    dls lr, r6
+; CHECK-NEXT:    mov r10, r11
+; CHECK-NEXT:    vstrw.32 q3, [sp, #32] @ 16-byte Spill
+; CHECK-NEXT:    vstrw.32 q3, [sp, #48] @ 16-byte Spill
+; CHECK-NEXT:    dls lr, r7
 ; CHECK-NEXT:  .LBB7_3: @ %vector.body
 ; CHECK-NEXT:    @ Parent Loop BB7_2 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    add.w r11, r3, r5
+; CHECK-NEXT:    adds r6, r3, r5
 ; CHECK-NEXT:    vctp.32 r10
 ; CHECK-NEXT:    vpsttt
-; CHECK-NEXT:    vldrwt.u32 q0, [r9], #16
+; CHECK-NEXT:    vldrwt.u32 q0, [r1], #16
 ; CHECK-NEXT:    vldrwt.u32 q1, [r3], #16
 ; CHECK-NEXT:    vfmat.f32 q6, q1, q0
-; CHECK-NEXT:    vstrw.32 q6, [sp, #40] @ 16-byte Spill
+; CHECK-NEXT:    vstrw.32 q6, [sp, #16] @ 16-byte Spill
 ; CHECK-NEXT:    vpstt
-; CHECK-NEXT:    vldrwt.u32 q1, [r11]
+; CHECK-NEXT:    vldrwt.u32 q1, [r6]
 ; CHECK-NEXT:    vfmat.f32 q7, q1, q0
-; CHECK-NEXT:    add.w r6, r11, r5
+; CHECK-NEXT:    adds r7, r6, r5
 ; CHECK-NEXT:    vmov q6, q5
 ; CHECK-NEXT:    vmov q5, q3
 ; CHECK-NEXT:    vmov q3, q4
 ; CHECK-NEXT:    vpst
-; CHECK-NEXT:    vldrwt.u32 q1, [r6]
+; CHECK-NEXT:    vldrwt.u32 q1, [r7]
 ; CHECK-NEXT:    vmov q4, q2
-; CHECK-NEXT:    vldrw.u32 q2, [sp, #56] @ 16-byte Reload
+; CHECK-NEXT:    vldrw.u32 q2, [sp, #32] @ 16-byte Reload
 ; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vfmat.f32 q2, q1, q0
-; CHECK-NEXT:    vstrw.32 q2, [sp, #56] @ 16-byte Spill
-; CHECK-NEXT:    adds r7, r6, r5
-; CHECK-NEXT:    vpst
-; CHECK-NEXT:    vldrwt.u32 q1, [r7]
-; CHECK-NEXT:    vldrw.u32 q2, [sp, #72] @ 16-byte Reload
+; CHECK-NEXT:    vstrw.32 q2, [sp, #32] @ 16-byte Spill
 ; CHECK-NEXT:    adds r6, r7, r5
 ; CHECK-NEXT:    vpst
+; CHECK-NEXT:    vldrwt.u32 q1, [r6]
+; CHECK-NEXT:    vldrw.u32 q2, [sp, #48] @ 16-byte Reload
+; CHECK-NEXT:    adds r7, r6, r5
+; CHECK-NEXT:    vpst
 ; CHECK-NEXT:    vfmat.f32 q2, q1, q0
-; CHECK-NEXT:    vstrw.32 q2, [sp, #72] @ 16-byte Spill
+; CHECK-NEXT:    vstrw.32 q2, [sp, #48] @ 16-byte Spill
 ; CHECK-NEXT:    vmov q2, q4
 ; CHECK-NEXT:    vmov q4, q3
 ; CHECK-NEXT:    vmov q3, q5
 ; CHECK-NEXT:    vmov q5, q6
-; CHECK-NEXT:    vldrw.u32 q6, [sp, #40] @ 16-byte Reload
-; CHECK-NEXT:    adds r7, r6, r5
+; CHECK-NEXT:    vldrw.u32 q6, [sp, #16] @ 16-byte Reload
+; CHECK-NEXT:    adds r6, r7, r5
 ; CHECK-NEXT:    vpstt
-; CHECK-NEXT:    vldrwt.u32 q1, [r6]
+; CHECK-NEXT:    vldrwt.u32 q1, [r7]
 ; CHECK-NEXT:    vfmat.f32 q2, q1, q0
 ; CHECK-NEXT:    sub.w r10, r10, #4
-; CHECK-NEXT:    adds r6, r7, r5
+; CHECK-NEXT:    adds r7, r6, r5
 ; CHECK-NEXT:    vpstttt
-; CHECK-NEXT:    vldrwt.u32 q1, [r7]
-; CHECK-NEXT:    vfmat.f32 q4, q1, q0
 ; CHECK-NEXT:    vldrwt.u32 q1, [r6]
+; CHECK-NEXT:    vfmat.f32 q4, q1, q0
+; CHECK-NEXT:    vldrwt.u32 q1, [r7]
 ; CHECK-NEXT:    vfmat.f32 q5, q1, q0
-; CHECK-NEXT:    add r6, r5
+; CHECK-NEXT:    adds r6, r7, r5
 ; CHECK-NEXT:    vpstt
 ; CHECK-NEXT:    vldrwt.u32 q1, [r6]
 ; CHECK-NEXT:    vfmat.f32 q3, q1, q0
 ; CHECK-NEXT:    le lr, .LBB7_3
 ; CHECK-NEXT:  @ %bb.4: @ %middle.block
 ; CHECK-NEXT:    @ in Loop: Header=BB7_2 Depth=1
 ; CHECK-NEXT:    vadd.f32 s0, s30, s31
-; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
+; CHECK-NEXT:    add.w r1, r2, r8, lsl #2
 ; CHECK-NEXT:    vadd.f32 s2, s28, s29
 ; CHECK-NEXT:    vadd.f32 s4, s26, s27
 ; CHECK-NEXT:    vadd.f32 s6, s24, s25
 ; CHECK-NEXT:    vadd.f32 s5, s18, s19
 ; CHECK-NEXT:    vadd.f32 s7, s16, s17
-; CHECK-NEXT:    vldrw.u32 q4, [sp, #56] @ 16-byte Reload
+; CHECK-NEXT:    vldrw.u32 q4, [sp, #32] @ 16-byte Reload
 ; CHECK-NEXT:    vadd.f32 s10, s10, s11
 ; CHECK-NEXT:    vadd.f32 s8, s8, s9
 ; CHECK-NEXT:    vadd.f32 s9, s18, s19
 ; CHECK-NEXT:    vadd.f32 s11, s16, s17
-; CHECK-NEXT:    vldrw.u32 q4, [sp, #72] @ 16-byte Reload
+; CHECK-NEXT:    vldrw.u32 q4, [sp, #48] @ 16-byte Reload
 ; CHECK-NEXT:    vadd.f32 s14, s14, s15
 ; CHECK-NEXT:    vadd.f32 s12, s12, s13
 ; CHECK-NEXT:    vadd.f32 s13, s18, s19
 ; CHECK-NEXT:    vadd.f32 s15, s16, s17
 ; CHECK-NEXT:    vadd.f32 s0, s2, s0
 ; CHECK-NEXT:    vadd.f32 s2, s6, s4
 ; CHECK-NEXT:    vadd.f32 s8, s8, s10
 ; CHECK-NEXT:    vadd.f32 s10, s11, s9
 ; CHECK-NEXT:    vadd.f32 s6, s12, s14
 ; CHECK-NEXT:    vadd.f32 s1, s22, s23
 ; CHECK-NEXT:    vadd.f32 s14, s15, s13
 ; CHECK-NEXT:    vstr s0, [r1]
 ; CHECK-NEXT:    add.w r1, r2, r0, lsl #2
 ; CHECK-NEXT:    vadd.f32 s3, s20, s21
-; CHECK-NEXT:    adds r0, #8
 ; CHECK-NEXT:    vstr s2, [r1]
-; CHECK-NEXT:    add.w r1, r2, r8, lsl #2
+; CHECK-NEXT:    add.w r1, r2, r12, lsl #2
 ; CHECK-NEXT:    vadd.f32 s12, s7, s5
 ; CHECK-NEXT:    vstr s10, [r1]
 ; CHECK-NEXT:    add.w r1, r2, r4, lsl #2
 ; CHECK-NEXT:    vstr s14, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #24] @ 4-byte Reload
-; CHECK-NEXT:    vadd.f32 s4, s3, s1
+; CHECK-NEXT:    adds r1, r0, #4
 ; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
+; CHECK-NEXT:    vadd.f32 s4, s3, s1
 ; CHECK-NEXT:    vstr s8, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #28] @ 4-byte Reload
+; CHECK-NEXT:    adds r1, r0, #5
 ; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
 ; CHECK-NEXT:    vstr s12, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #32] @ 4-byte Reload
+; CHECK-NEXT:    adds r1, r0, #6
 ; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
 ; CHECK-NEXT:    vstr s4, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #36] @ 4-byte Reload
+; CHECK-NEXT:    adds r1, r0, #7
+; CHECK-NEXT:    adds r0, #8
 ; CHECK-NEXT:    add.w r1, r2, r1, lsl #2
 ; CHECK-NEXT:    vstr s6, [r1]
-; CHECK-NEXT:    ldr r1, [sp, #4] @ 4-byte Reload
-; CHECK-NEXT:    add r12, r1
-; CHECK-NEXT:    ldr r1, [sp, #16] @ 4-byte Reload
+; CHECK-NEXT:    ldr r1, [sp] @ 4-byte Reload
+; CHECK-NEXT:    add r9, r1
+; CHECK-NEXT:    ldr r1, [sp, #8] @ 4-byte Reload
 ; CHECK-NEXT:    cmp r0, r1
 ; CHECK-NEXT:    blo.w .LBB7_2
 ; CHECK-NEXT:  .LBB7_5: @ %for.cond.cleanup
-; CHECK-NEXT:    add sp, #88
+; CHECK-NEXT:    add sp, #64
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 entry:
   %NumInputs = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 2
   %0 = load i32, i32* %NumInputs, align 4
   %NumFilters = getelementptr inbounds %struct.DCT_InstanceTypeDef, %struct.DCT_InstanceTypeDef* %S, i32 0, i32 1
   %1 = load i32, i32* %NumFilters, align 4

llvm/test/CodeGen/X86/addcarry.ll

 ; CHECK-NEXT:    .cfi_offset %rbx, -24
 ; CHECK-NEXT:    .cfi_offset %r14, -16
 ; CHECK-NEXT:    movq 16(%rdi), %rax
 ; CHECK-NEXT:    leaq (%rax,%rcx), %r10
 ; CHECK-NEXT:    addq %rsi, (%rdi)
 ; CHECK-NEXT:    adcq %rdx, 8(%rdi)
 ; CHECK-NEXT:    movq %rax, %rdx
 ; CHECK-NEXT:    adcq %rcx, %rdx
-; CHECK-NEXT:    movq 24(%rdi), %r11
-; CHECK-NEXT:    leaq (%r8,%r11), %r14
+; CHECK-NEXT:    movq 24(%rdi), %r14
+; CHECK-NEXT:    leaq (%r8,%r14), %r11
 ; CHECK-NEXT:    xorl %ebx, %ebx
 ; CHECK-NEXT:    cmpq %r10, %rdx
 ; CHECK-NEXT:    setb %bl
 ; CHECK-NEXT:    addq %rcx, %rax
-; CHECK-NEXT:    adcq %r14, %rbx
-; CHECK-NEXT:    movq 32(%rdi), %r10
-; CHECK-NEXT:    leaq (%r9,%r10), %rcx
+; CHECK-NEXT:    adcq %r11, %rbx
+; CHECK-NEXT:    movq 32(%rdi), %rcx
+; CHECK-NEXT:    leaq (%r9,%rcx), %r10
 ; CHECK-NEXT:    xorl %esi, %esi
-; CHECK-NEXT:    cmpq %r14, %rbx
+; CHECK-NEXT:    cmpq %r11, %rbx
 ; CHECK-NEXT:    setb %sil
-; CHECK-NEXT:    addq %r11, %r8
-; CHECK-NEXT:    adcq %rcx, %rsi
+; CHECK-NEXT:    addq %r14, %r8
+; CHECK-NEXT:    adcq %r10, %rsi
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    cmpq %rcx, %rsi
+; CHECK-NEXT:    cmpq %r10, %rsi
 ; CHECK-NEXT:    setb %al
-; CHECK-NEXT:    addq %r10, %r9
+; CHECK-NEXT:    addq %rcx, %r9
 ; CHECK-NEXT:    movq %rdx, 16(%rdi)
 ; CHECK-NEXT:    movq %rbx, 24(%rdi)
 ; CHECK-NEXT:    movq %rsi, 32(%rdi)
 ; CHECK-NEXT:    adcl $0, %eax
 ; CHECK-NEXT:    popq %rbx
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    popq %r14
 ; CHECK-NEXT:    .cfi_def_cfa_offset 8

llvm/test/CodeGen/X86/callbr-asm-blockplacement.ll

 ; CHECK-NEXT:    pushq %r12
 ; CHECK-NEXT:    pushq %rbx
 ; CHECK-NEXT:    pushq %rax
 ; CHECK-NEXT:    movabsq $-2305847407260205056, %rbx # imm = 0xDFFFFC0000000000
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    testb %al, %al
 ; CHECK-NEXT:    jne .LBB0_5
 ; CHECK-NEXT:  # %bb.1: # %bb5
-; CHECK-NEXT:    movq %rsi, %r14
+; CHECK-NEXT:    movq %rsi, %r12
 ; CHECK-NEXT:    movslq %edi, %rbp
 ; CHECK-NEXT:    leaq (,%rbp,8), %rax
-; CHECK-NEXT:    leaq global(%rax,%rax,2), %r15
-; CHECK-NEXT:    leaq global+4(%rax,%rax,2), %r12
+; CHECK-NEXT:    leaq global(%rax,%rax,2), %r14
+; CHECK-NEXT:    leaq global+4(%rax,%rax,2), %r15
 ; CHECK-NEXT:    xorl %r13d, %r13d
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB0_2: # %bb8
 ; CHECK-NEXT:    # =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    callq bar@PLT
 ; CHECK-NEXT:    movq %rax, %rbx
 ; CHECK-NEXT:    movq %rax, %rdi
-; CHECK-NEXT:    callq *%r14
-; CHECK-NEXT:    movq %r15, %rdi
+; CHECK-NEXT:    callq *%r12
+; CHECK-NEXT:    movq %r14, %rdi
 ; CHECK-NEXT:    callq hoge@PLT
-; CHECK-NEXT:    movq %r12, %rdi
+; CHECK-NEXT:    movq %r15, %rdi
 ; CHECK-NEXT:    callq hoge@PLT
 ; CHECK-NEXT:    testb %r13b, %r13b
 ; CHECK-NEXT:    jne .LBB0_2
 ; CHECK-NEXT:  # %bb.3: # %bb15
 ; CHECK-NEXT:    leaq (%rbp,%rbp,2), %rax
 ; CHECK-NEXT:    movq %rbx, global+16(,%rax,8)
 ; CHECK-NEXT:    movabsq $-2305847407260205056, %rbx # imm = 0xDFFFFC0000000000
 ; CHECK-NEXT:    #APP

llvm/test/CodeGen/X86/dag-update-nodetomatch.ll

 ; CHECK-NEXT:    notl %eax
 ; CHECK-NEXT:    leaq 8(,%rax,8), %r14
 ; CHECK-NEXT:    imulq %r13, %r14
 ; CHECK-NEXT:    addq %r12, %r14
 ; CHECK-NEXT:    movq x2@GOTPCREL(%rip), %r15
 ; CHECK-NEXT:    movl (%r15), %eax
 ; CHECK-NEXT:    leal 8(,%rcx,8), %ecx
 ; CHECK-NEXT:    movq %rcx, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
-; CHECK-NEXT:    leaq 8(%r12), %rcx
-; CHECK-NEXT:    movq %rcx, {{[-0-9]+}}(%r{{[sb]}}p) # 8-byte Spill
 ; CHECK-NEXT:    leaq 32(%r12), %rbx
 ; CHECK-NEXT:    shlq $3, %r13
 ; CHECK-NEXT:    xorl %esi, %esi
 ; CHECK-NEXT:    movq x0@GOTPCREL(%rip), %rcx
 ; CHECK-NEXT:    movq %r12, %rdi
 ; CHECK-NEXT:    jmp .LBB1_2
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  .LBB1_15: # %for.cond1.for.inc3_crit_edge

 ; CHECK-NEXT:    leaq (%rax,%r9,8), %rax
 ; CHECK-NEXT:    testq %r9, %r9
 ; CHECK-NEXT:    movq $-1, %r11
 ; CHECK-NEXT:    cmovnsq %r9, %r11
 ; CHECK-NEXT:    cmpq %rcx, %rax
 ; CHECK-NEXT:    jae .LBB1_7
 ; CHECK-NEXT:  # %bb.6: # %vector.memcheck
 ; CHECK-NEXT:    # in Loop: Header=BB1_2 Depth=1
-; CHECK-NEXT:    addq {{[-0-9]+}}(%r{{[sb]}}p), %r10 # 8-byte Folded Reload
+; CHECK-NEXT:    leaq 8(%r12), %rax
+; CHECK-NEXT:    addq %rax, %r10
 ; CHECK-NEXT:    leaq (%r10,%r11,8), %rax
 ; CHECK-NEXT:    cmpq %rcx, %rax
 ; CHECK-NEXT:    ja .LBB1_14
 ; CHECK-NEXT:  .LBB1_7: # %vector.body.preheader
 ; CHECK-NEXT:    # in Loop: Header=BB1_2 Depth=1
-; CHECK-NEXT:    leaq -4(%r8), %rax
-; CHECK-NEXT:    movq %rax, %r10
-; CHECK-NEXT:    shrq $2, %r10
-; CHECK-NEXT:    btl $2, %eax
+; CHECK-NEXT:    leaq -4(%r8), %r10
+; CHECK-NEXT:    movq %r10, %rax
+; CHECK-NEXT:    shrq $2, %rax
+; CHECK-NEXT:    btl $2, %r10d
 ; CHECK-NEXT:    jb .LBB1_8
 ; CHECK-NEXT:  # %bb.9: # %vector.body.prol.preheader
 ; CHECK-NEXT:    # in Loop: Header=BB1_2 Depth=1
 ; CHECK-NEXT:    movq {{.*#+}} xmm0 = mem[0],zero
 ; CHECK-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,1,0,1]
 ; CHECK-NEXT:    movdqu %xmm0, (%rdi,%r9,8)
 ; CHECK-NEXT:    movdqu %xmm0, 16(%rdi,%r9,8)
 ; CHECK-NEXT:    movl $4, %r11d
-; CHECK-NEXT:    testq %r10, %r10
+; CHECK-NEXT:    testq %rax, %rax
 ; CHECK-NEXT:    jne .LBB1_11
 ; CHECK-NEXT:    jmp .LBB1_13
 ; CHECK-NEXT:  .LBB1_8: # in Loop: Header=BB1_2 Depth=1
 ; CHECK-NEXT:    xorl %r11d, %r11d
-; CHECK-NEXT:    testq %r10, %r10
+; CHECK-NEXT:    testq %rax, %rax
 ; CHECK-NEXT:    je .LBB1_13
 ; CHECK-NEXT:  .LBB1_11: # %vector.body.preheader.new
 ; CHECK-NEXT:    # in Loop: Header=BB1_2 Depth=1
 ; CHECK-NEXT:    movq {{.*#+}} xmm0 = mem[0],zero
 ; CHECK-NEXT:    pshufd {{.*#+}} xmm0 = xmm0[0,1,0,1]
 ; CHECK-NEXT:    movq %r11, %rax
 ; CHECK-NEXT:    subq %r8, %rax
 ; CHECK-NEXT:    addq %r9, %r11

llvm/test/CodeGen/X86/inalloca-invoke.ll

blah:
   %inalloca.save = call i8* @llvm.stacksave()
   %rev_args = alloca inalloca %frame.reverse, align 4
   %beg = getelementptr %frame.reverse, %frame.reverse* %rev_args, i32 0, i32 0
   %end = getelementptr %frame.reverse, %frame.reverse* %rev_args, i32 0, i32 1
 
 ; CHECK:  pushl   %eax
 ; CHECK:  subl    $20, %esp
 ; CHECK:  movl %esp, %[[beg:[^ ]*]]
-; CHECK:  leal 12(%[[beg]]), %[[end:[^ ]*]]
 
   call void @begin(%Iter* sret(%Iter) %temp.lvalue)
 ; CHECK:  calll _begin
 
   invoke void @plus(%Iter* sret(%Iter) %end, %Iter* %temp.lvalue, i32 4)
           to label %invoke.cont unwind label %lpad
 
 ;  Uses end as sret param.
+; CHECK:  leal 12(%[[beg]]), %[[end:[^ ]*]]
 ; CHECK:  pushl %[[end]]
 ; CHECK:  calll _plus
 
 invoke.cont:
   call void @begin(%Iter* sret(%Iter) %beg)
 
 ; CHECK:  pushl %[[beg]]
 ; CHECK:  calll _begin

llvm/test/CodeGen/X86/licm-regpressure.ll

 ; RUN: llc < %s -mtriple=x86_64-linux | FileCheck %s
-; This tests currently fails as MachineLICM does not compute register pressure
+; RUN: llc < %s -mtriple=x86_64-linux -stop-after=early-machinelicm -o - | FileCheck %s -check-prefix=MIR
+
+; This tests should fail as MachineLICM does not compute register pressure
 ; correctly. More details: llvm.org/PR23143
+
+; It however does not show any spills because leaq is rematerialized instead
+; of spilling.
+
+; Stopping after MachineLICM however exposes all ADD64ri8 instructions
+; to be hoisted which still has to be avoided.
+
 ; XFAIL: *
 
 ; MachineLICM should take register pressure into account.
-; CHECK-NOT: Spill
+; CHECK-LABEL: {{^}}test:
+; CHECK-NOT:     Spill
+; CHECK-COUNT-4: leaq
+; CHECK-NOT:     Spill
+; CHECK:         [[LOOP:\.LBB[0-9_]+]]:
+; CHECK-NOT:     Reload
+; CHECK-COUNT-2: leaq
+; CHECK-NOT:     Reload
+; CHECK:         jne [[LOOP]]
+
+; MIR-LABEL: name: test
+; MIR:         bb.0.entry:
+; MIR-COUNT-4: ADD64ri8
+; MIR:         bb.1.loop-body:
+; MIR-COUNT-2: ADD64ri8
+; MIR:         JCC_1 %bb.1
 
 %struct.A = type { i32, i32, i32, i32, i32, i32, i32 }
 
 define void @test(i1 %b, %struct.A* %a) nounwind {
 entry:
   br label %loop-header
 
 loop-header:

llvm/test/CodeGen/X86/ragreedy-hoist-spill.ll

 ; CHECK-NEXT:  LBB0_9: ## %do.body
 ; CHECK-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    movl $0, (%rax)
 ; CHECK-NEXT:    testb %cl, %cl
 ; CHECK-NEXT:    jne LBB0_9
 ; CHECK-NEXT:  ## %bb.10: ## %do.end
 ; CHECK-NEXT:    movq %rdx, {{[-0-9]+}}(%r{{[sb]}}p) ## 8-byte Spill
 ; CHECK-NEXT:    movq %rbp, {{[-0-9]+}}(%r{{[sb]}}p) ## 8-byte Spill
-; CHECK-NEXT:    xorl %r13d, %r13d
-; CHECK-NEXT:    testb %r13b, %r13b
+; CHECK-NEXT:    xorl %r12d, %r12d
+; CHECK-NEXT:    testb %r12b, %r12b
 ; CHECK-NEXT:    jne LBB0_11
 ; CHECK-NEXT:  ## %bb.12: ## %while.body200.preheader
-; CHECK-NEXT:    xorl %r12d, %r12d
+; CHECK-NEXT:    xorl %ebx, %ebx
 ; CHECK-NEXT:    leaq LJTI0_0(%rip), %rdx
-; CHECK-NEXT:    leaq LJTI0_1(%rip), %rbx
+; CHECK-NEXT:    leaq LJTI0_1(%rip), %r13
 ; CHECK-NEXT:    movl $0, {{[-0-9]+}}(%r{{[sb]}}p) ## 4-byte Folded Spill
 ; CHECK-NEXT:    xorl %r14d, %r14d
 ; CHECK-NEXT:    jmp LBB0_13
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  LBB0_20: ## %sw.bb256
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
-; CHECK-NEXT:    movl %r13d, %r14d
+; CHECK-NEXT:    movl %r12d, %r14d
 ; CHECK-NEXT:  LBB0_21: ## %while.cond197.backedge
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
 ; CHECK-NEXT:    decl %r15d
 ; CHECK-NEXT:    testl %r15d, %r15d
-; CHECK-NEXT:    movl %r14d, %r13d
+; CHECK-NEXT:    movl %r14d, %r12d
 ; CHECK-NEXT:    jle LBB0_22
 ; CHECK-NEXT:  LBB0_13: ## %while.body200
 ; CHECK-NEXT:    ## =>This Loop Header: Depth=1
 ; CHECK-NEXT:    ## Child Loop BB0_29 Depth 2
 ; CHECK-NEXT:    ## Child Loop BB0_38 Depth 2
-; CHECK-NEXT:    leal -268(%r13), %eax
+; CHECK-NEXT:    leal -268(%r12), %eax
 ; CHECK-NEXT:    cmpl $105, %eax
 ; CHECK-NEXT:    ja LBB0_14
 ; CHECK-NEXT:  ## %bb.56: ## %while.body200
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
-; CHECK-NEXT:    movslq (%rbx,%rax,4), %rax
-; CHECK-NEXT:    addq %rbx, %rax
+; CHECK-NEXT:    movslq (%r13,%rax,4), %rax
+; CHECK-NEXT:    addq %r13, %rax
 ; CHECK-NEXT:    jmpq *%rax
 ; CHECK-NEXT:  LBB0_44: ## %while.cond1037.preheader
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
-; CHECK-NEXT:    testb %r12b, %r12b
-; CHECK-NEXT:    movl %r13d, %r14d
+; CHECK-NEXT:    testb %bl, %bl
+; CHECK-NEXT:    movl %r12d, %r14d
 ; CHECK-NEXT:    jne LBB0_21
 ; CHECK-NEXT:    jmp LBB0_55
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  LBB0_14: ## %while.body200
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
-; CHECK-NEXT:    leal 1(%r13), %eax
+; CHECK-NEXT:    leal 1(%r12), %eax
 ; CHECK-NEXT:    cmpl $21, %eax
 ; CHECK-NEXT:    ja LBB0_20
 ; CHECK-NEXT:  ## %bb.15: ## %while.body200
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
 ; CHECK-NEXT:    movl $-1, %r14d
 ; CHECK-NEXT:    movslq (%rdx,%rax,4), %rax
 ; CHECK-NEXT:    addq %rdx, %rax
 ; CHECK-NEXT:    jmpq *%rax
 ; CHECK-NEXT:  LBB0_18: ## %while.cond201.preheader
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
 ; CHECK-NEXT:    movl $1, %r14d
 ; CHECK-NEXT:    jmp LBB0_21
 ; CHECK-NEXT:  LBB0_26: ## %sw.bb474
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
-; CHECK-NEXT:    testb %r12b, %r12b
+; CHECK-NEXT:    testb %bl, %bl
 ; CHECK-NEXT:    ## implicit-def: $rbp
 ; CHECK-NEXT:    jne LBB0_34
 ; CHECK-NEXT:  ## %bb.27: ## %do.body479.preheader
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
-; CHECK-NEXT:    testb %r12b, %r12b
+; CHECK-NEXT:    testb %bl, %bl
 ; CHECK-NEXT:    ## implicit-def: $rbp
 ; CHECK-NEXT:    jne LBB0_34
 ; CHECK-NEXT:  ## %bb.28: ## %land.rhs485.preheader
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
 ; CHECK-NEXT:    ## implicit-def: $rax
 ; CHECK-NEXT:    jmp LBB0_29
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  LBB0_32: ## %do.body479.backedge
 ; CHECK-NEXT:    ## in Loop: Header=BB0_29 Depth=2
 ; CHECK-NEXT:    leaq 1(%rbp), %rax
-; CHECK-NEXT:    testb %r12b, %r12b
+; CHECK-NEXT:    testb %bl, %bl
 ; CHECK-NEXT:    je LBB0_33
 ; CHECK-NEXT:  LBB0_29: ## %land.rhs485
 ; CHECK-NEXT:    ## Parent Loop BB0_13 Depth=1
 ; CHECK-NEXT:    ## => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    testb %al, %al
 ; CHECK-NEXT:    js LBB0_55
 ; CHECK-NEXT:  ## %bb.30: ## %cond.true.i.i2780
 ; CHECK-NEXT:    ## in Loop: Header=BB0_29 Depth=2
 ; CHECK-NEXT:    movq %rax, %rbp
-; CHECK-NEXT:    testb %r12b, %r12b
+; CHECK-NEXT:    testb %bl, %bl
 ; CHECK-NEXT:    jne LBB0_32
 ; CHECK-NEXT:  ## %bb.31: ## %lor.rhs500
 ; CHECK-NEXT:    ## in Loop: Header=BB0_29 Depth=2
 ; CHECK-NEXT:    movl $256, %esi ## imm = 0x100
 ; CHECK-NEXT:    callq ___maskrune
-; CHECK-NEXT:    testb %r12b, %r12b
+; CHECK-NEXT:    testb %bl, %bl
 ; CHECK-NEXT:    jne LBB0_32
 ; CHECK-NEXT:    jmp LBB0_34
 ; CHECK-NEXT:  LBB0_45: ## %sw.bb1134
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
 ; CHECK-NEXT:    leaq {{[0-9]+}}(%rsp), %rax
 ; CHECK-NEXT:    leaq {{[0-9]+}}(%rsp), %rcx
 ; CHECK-NEXT:    cmpq %rax, %rcx
 ; CHECK-NEXT:    jb LBB0_55

 ; CHECK-NEXT:  ## %bb.37: ## %if.then532
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
 ; CHECK-NEXT:    movq _SyFgets.yank@GOTPCREL(%rip), %rax
 ; CHECK-NEXT:    movb $0, (%rax)
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  LBB0_38: ## %for.cond534
 ; CHECK-NEXT:    ## Parent Loop BB0_13 Depth=1
 ; CHECK-NEXT:    ## => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    testb %r12b, %r12b
+; CHECK-NEXT:    testb %bl, %bl
 ; CHECK-NEXT:    jne LBB0_38
 ; CHECK-NEXT:  ## %bb.39: ## %for.cond542.preheader
 ; CHECK-NEXT:    ## in Loop: Header=BB0_13 Depth=1
-; CHECK-NEXT:    testb %r12b, %r12b
+; CHECK-NEXT:    testb %bl, %bl
 ; CHECK-NEXT:    movb $0, (%rbp)
-; CHECK-NEXT:    movl %r13d, %r14d
+; CHECK-NEXT:    movl %r12d, %r14d
 ; CHECK-NEXT:    leaq LJTI0_0(%rip), %rdx
 ; CHECK-NEXT:    jmp LBB0_21
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  LBB0_42: ## %while.cond864
 ; CHECK-NEXT:    ## =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    jmp LBB0_42
 ; CHECK-NEXT:    .p2align 4, 0x90
 ; CHECK-NEXT:  LBB0_43: ## %while.cond962

llvm/test/CodeGen/X86/sdiv_fix.ll

 ; X64-NEXT:    movq %rsi, %r12
 ; X64-NEXT:    sarq $63, %r12
 ; X64-NEXT:    movq %r15, %rdi
 ; X64-NEXT:    movq %rbx, %rsi
 ; X64-NEXT:    movq %r14, %rdx
 ; X64-NEXT:    movq %r12, %rcx
 ; X64-NEXT:    callq __divti3@PLT
 ; X64-NEXT:    movq %rax, %r13
-; X64-NEXT:    decq %rax
-; X64-NEXT:    movq %rax, (%rsp) # 8-byte Spill
 ; X64-NEXT:    testq %rbx, %rbx
 ; X64-NEXT:    sets %al
 ; X64-NEXT:    testq %r12, %r12
 ; X64-NEXT:    sets %bpl
 ; X64-NEXT:    xorb %al, %bpl
 ; X64-NEXT:    movq %r15, %rdi
 ; X64-NEXT:    movq %rbx, %rsi
 ; X64-NEXT:    movq %r14, %rdx
 ; X64-NEXT:    movq %r12, %rcx
 ; X64-NEXT:    callq __modti3@PLT
 ; X64-NEXT:    orq %rax, %rdx
 ; X64-NEXT:    setne %al
 ; X64-NEXT:    testb %bpl, %al
-; X64-NEXT:    cmovneq (%rsp), %r13 # 8-byte Folded Reload
+; X64-NEXT:    leaq -1(%r13), %rax
+; X64-NEXT:    cmovneq %rax, %r13
 ; X64-NEXT:    movq %r13, %rax
 ; X64-NEXT:    addq $8, %rsp
 ; X64-NEXT:    popq %rbx
 ; X64-NEXT:    popq %r12
 ; X64-NEXT:    popq %r13
 ; X64-NEXT:    popq %r14
 ; X64-NEXT:    popq %r15
 ; X64-NEXT:    popq %rbp