This is an archive of the discontinued LLVM Phabricator instance.

Differential D156799

Update generic scheduling to use A510 scheduling model
ClosedPublic

Authored by harviniriawan on Aug 1 2023, 7:47 AM.

Details

Reviewers
dmgreen
SjoerdMeijer
samtebbs
t.p.northover
tmatheson
Summary

Refresh of the generic scheduling model to use A510 instead of A55.
Main benefits are to the little core, and introducing SVE scheduling information.
Changes tested on various OoO cores, no performance degradation is seen.

Diff Detail

Event Timeline

harviniriawan created this revision.Aug 1 2023, 7:47 AM
Herald added a project: Restricted Project. · View Herald TranscriptAug 1 2023, 7:47 AM
Herald added subscribers: ctetreau, armkevincheng, sjarus and 5 others. · View Herald Transcript
harviniriawan requested review of this revision.Aug 1 2023, 7:47 AM
Herald added a project: Restricted Project. · View Herald TranscriptAug 1 2023, 7:47 AM
Herald added a subscriber: llvm-commits. · View Herald Transcript
Harbormaster completed remote builds in B249491: Diff 546061.Aug 1 2023, 7:48 AM
Matt added a subscriber: Matt.Aug 1 2023, 2:28 PM
dmgreen added reviewers: SjoerdMeijer, samtebbs, t.p.northover, tmatheson.Aug 2 2023, 2:37 AM

Sounds great. The change certainly sounds like a good idea.

There are a lot of test updates needed though, and some of them might be better not-auto-generated, or could otherwise do with a bit of a cleanup. I remember when changing to the A55 scheduling model that I just manually updated some of the tests, it might be better to do the same here. I haven't gone through everything, but have marked some of the tests that could do with some cleanup.

llvm/test/CodeGen/AArch64/128bit_load_store.ll
1 ↗(On Diff #546061)

I'm not sure why this needed updating, but the check lines OK here.

llvm/test/CodeGen/AArch64/2s-complement-asm.ll
1 ↗(On Diff #546061)

This one isn't needed, and seems to not have updated the actual tests.

llvm/test/CodeGen/AArch64/GlobalISel/swifterror.ll
1

I think this is one that is best left manually updated.

llvm/test/CodeGen/AArch64/aarch64-2014-08-11-MachineCombinerCrash.ll
1 ↗(On Diff #546061)

Probably not needed.

llvm/test/CodeGen/AArch64/fptoui-sat-vector.ll
2277

There is an extra spill here, but that kind of things is expected given enough code.

llvm/test/CodeGen/AArch64/logical_shifted_reg.ll
261–263

This looks like it no longer decides to ifcvt. Given the test that sound fine I think.

llvm/test/CodeGen/AArch64/memcpy-scoped-aa.ll
137–138

Things like this can be removed too, so long as there are check lines elsewhere.

llvm/test/CodeGen/AArch64/misched-detail-resource-booking-01.mir
1

This doesn't look right. Should it be using -mcpu=cortex-a55 instead?

llvm/test/CodeGen/AArch64/misched-fusion-lit.ll
1

This is likely best checked manually.

llvm/test/CodeGen/AArch64/named-vector-shuffles-neon.ll
114

Some of these have less movs thanks to the lower latencies, which is nice.

llvm/test/CodeGen/AArch64/wineh-bti.ll
1

This has test CHECK line elsewhere in the file.

harviniriawan updated this revision to Diff 546501.Aug 2 2023, 9:09 AM
harviniriawan marked 11 inline comments as done.
Herald added a subscriber: arichardson. · View Herald TranscriptAug 2 2023, 9:09 AM
harviniriawan added inline comments.Aug 2 2023, 9:09 AM
llvm/test/CodeGen/AArch64/misched-detail-resource-booking-01.mir
1

shouldn't it be a510 as it's the default scheduling policy now?

Harbormaster completed remote builds in B249800: Diff 546501.Aug 2 2023, 9:10 AM
harviniriawan updated this revision to Diff 546509.Aug 2 2023, 9:33 AM
Harbormaster completed remote builds in B249807: Diff 546509.Aug 2 2023, 9:34 AM
harviniriawan updated this revision to Diff 546520.Aug 2 2023, 9:48 AM
Harbormaster completed remote builds in B249818: Diff 546520.Aug 2 2023, 9:49 AM
harviniriawan updated this revision to Diff 546739.Aug 3 2023, 1:22 AM
Harbormaster completed remote builds in B249981: Diff 546739.Aug 3 2023, 2:48 AM
dmgreen added a comment.Aug 3 2023, 4:00 AM

Thanks. If you can clean up these tests, then this looks good to me.

llvm/test/CodeGen/AArch64/aarch64-dynamic-stack-layout.ll
86

This looks like it would be better with the manual check lines below.

llvm/test/CodeGen/AArch64/arm64-ld1.ll
1

This doesn't look like it has generated properly.

llvm/test/CodeGen/AArch64/arm64-promote-const-complex-initializers.ll
12

This doesn't look like it is checking everything it did in the past.

llvm/test/CodeGen/AArch64/arm64_32.ll
17

This file might be better manually generated. Some of the check lines and comments are deliberately placed.

llvm/test/CodeGen/AArch64/atomic-ops-lse.ll
36

It looks like ths RUN line for CHECK-REG is trying to check that there are no invalid register formed. You might be able to change it to --check-prefixes=CHECK,CHECK-REG

llvm/test/CodeGen/AArch64/bfis-in-loop.ll
13

This doesn't look right

llvm/test/CodeGen/AArch64/f16-instructions.ll
13–14

This one needs to be done manually.

llvm/test/CodeGen/AArch64/fmlal-loreg.ll
3

I'm not sure why these blank lines were removed? I think they can be kept in.

llvm/test/CodeGen/AArch64/misched-detail-resource-booking-01.mir
29

This file can specify -mcpu=cortex-a55 to keep the test as it was before.

llvm/test/CodeGen/AArch64/nontemporal-load.ll
5

These newlines and blank ; lines look funny.

llvm/test/CodeGen/AArch64/shrink-wrapping-vla.ll
18

Manual update might be better.

llvm/test/CodeGen/AArch64/sme2-intrinsics-min.ll
140–141

This doesn't look right.

llvm/test/CodeGen/AArch64/sme2-intrinsics-rshl.ll
53

This doesn't look right.

llvm/test/CodeGen/AArch64/sme2-intrinsics-sqdmulh.ll
54–55

This doesn't look right.

llvm/test/CodeGen/AArch64/speculation-hardening-loads.ll
5

Manual update might be better.

llvm/test/CodeGen/AArch64/split-vector-insert.ll
13

This was testing the debug output before.

llvm/test/CodeGen/AArch64/stack-guard-sysreg.ll
1

Can you explain this one? Are the two versions now too different to keep the same?

llvm/test/CodeGen/AArch64/sve-fixed-length-build-vector.ll
56

Keep this

llvm/test/CodeGen/AArch64/sve-split-int-pred-reduce.ll
25

This is nice :)

llvm/test/CodeGen/AArch64/swift-return.ll
7

Manually update this one may be best.

llvm/test/CodeGen/AArch64/vec-combine-compare-to-bitmask.ll
10

These need to keep checking the constant pools

llvm/test/CodeGen/AArch64/vec-combine-compare-truncate-store.ll
5

These too.

llvm/test/CodeGen/AArch64/zext-to-tbl.ll
5

These too.

harviniriawan updated this revision to Diff 546884.Aug 3 2023, 8:16 AM
harviniriawan marked 23 inline comments as done.
Herald added a project: Restricted Project. · View Herald TranscriptAug 3 2023, 8:16 AM
Herald added a subscriber: cfe-commits. · View Herald Transcript
harviniriawan added inline comments.Aug 3 2023, 8:16 AM
llvm/test/CodeGen/AArch64/stack-guard-sysreg.ll
1

Yes indeed. the ones with CHECK-ADD annoyingly is scheduled slightly differently due to it being ADD instead of LDR

Harbormaster completed remote builds in B250083: Diff 546884.Aug 3 2023, 10:16 AM
dmgreen accepted this revision.Aug 3 2023, 1:11 PM

Thanks. Looks good. You may need to rebase over new tests, make sure everything is still passing when you do.

LGTM

llvm/test/CodeGen/AArch64/misched-detail-resource-booking-01.mir
1

Sorry I forgot to reply to the old comment. AFAIU this is testing some of the info in the scheduling model, not which model is being used. So pinning it to the A55 makes sense for keeping the old testing in place.

This revision is now accepted and ready to land.Aug 3 2023, 1:11 PM
harviniriawan updated this revision to Diff 551951.Aug 21 2023, 3:08 AM
Harbormaster completed remote builds in B253799: Diff 551951.Aug 21 2023, 3:09 AM
harviniriawan marked an inline comment as done.Aug 21 2023, 3:09 AM
harviniriawan updated this revision to Diff 551953.Aug 21 2023, 3:16 AM
Harbormaster completed remote builds in B253801: Diff 551953.Aug 21 2023, 3:58 AM
harviniriawan closed this revision.Aug 21 2023, 4:27 AM
harviniriawan updated this revision to Diff 551978.
Harbormaster completed remote builds in B253816: Diff 551978.Aug 21 2023, 4:28 AM
aeubanks added a subscriber: aeubanks.Aug 21 2023, 8:50 AM
aeubanks added inline comments.
llvm/test/CodeGen/AArch64/stack-guard-sysreg.ll
1

this test is broken, without the split-file the other files won't exist in a clean build, please fix or revert if it takes time to fix

Large Diff

This large diff affects 583 files. Files without inline comments have been collapsed. Expand All Files

Revision Contents

PathSize
llvm/
lib/
Target/
AArch64/
2 lines
test/
Analysis/
CostModel/
AArch64/
2 lines
CodeGen/
AArch64/
Atomics/
660 lines
560 lines
660 lines
660 lines
660 lines
560 lines
660 lines
600 lines
500 lines
600 lines
600 lines
600 lines
500 lines
600 lines
GlobalISel/
3 lines
86 lines
83 lines
24 lines
156 lines
2 lines
84 lines
74 lines
9 lines
60 lines
72 lines
16 lines
21 lines
40 lines
8 lines
893 lines
2 lines
6 lines
26 lines
2 lines
36 lines
74 lines
12 lines
168 lines
48 lines
62 lines
20 lines
136 lines
34 lines
90 lines
24 lines
77 lines
40 lines
268 lines
4 lines
6 lines
4 lines
52 lines
2 lines
62 lines
160 lines
39 lines
8 lines
26 lines
18 lines
59 lines
20 lines
2 lines
12 lines
4 lines
6 lines
8 lines
116 lines
4 lines
4 lines
348 lines
4 lines
10 lines
18 lines
20 lines
504 lines
96 lines
54 lines
2 lines
70 lines
12 lines
104 lines
2 lines
8 lines
120 lines
542 lines
68 lines
662 lines
96 lines
18 lines
186 lines
10 lines
734 lines
4 lines
6 lines
947 lines
14 lines
8519 lines
56 lines
56 lines
24 lines
186 lines
6 lines
50 lines
42 lines
8 lines
4 lines
10 lines
13 lines
2 lines
4 lines
18 lines
4 lines
24 lines
12 lines
20 lines
48 lines
105 lines
84 lines
84 lines
69 lines
2 lines
4 lines
14 lines
30 lines
2 lines
2 lines
12 lines
30 lines
2 lines
2 lines
12 lines
30 lines
2 lines
20 lines
2 lines
6 lines
2 lines
12 lines
2 lines
80 lines
172 lines
92 lines
104 lines
76 lines
42 lines
28 lines
152 lines
2 lines
6 lines
50 lines
16 lines
34 lines
56 lines
14 lines
44 lines
6 lines
28 lines
28 lines
8 lines
34 lines
12 lines
411 lines
56 lines
282 lines
110 lines
862 lines
16 lines
240 lines
8 lines
50 lines
10 lines
6 lines
24 lines
4 lines
152 lines
6 lines
2 lines
2 lines
34 lines
18 lines
17 lines
4 lines
141 lines
36 lines
12 lines
814 lines
86 lines
1624 lines
64 lines
1287 lines
8 lines
18 lines
144 lines
36 lines
20 lines
10 lines
6 lines
8 lines
70 lines
16 lines
10 lines
188 lines
58 lines
92 lines
16 lines
8 lines
10 lines
137 lines
98 lines
2 lines
38 lines
81 lines
8 lines
12 lines
36 lines
16 lines
16 lines
50 lines
6 lines
26 lines
12 lines
20 lines
208 lines
256 lines
10 lines
2338 lines
1 line
112 lines
2 lines
24 lines
98 lines
3 lines
24 lines
13 lines
8 lines
4 lines
3504 lines
464 lines
6 lines
2 lines
4 lines
6 lines
24 lines
4 lines
2 lines
6 lines
269 lines
38 lines
24 lines
10 lines
30 lines
14 lines
48 lines
2 lines
6 lines
16 lines
50 lines
9 lines
58 lines
308 lines
4 lines
6 lines
14 lines
96 lines
96 lines
574 lines
96 lines
2 lines
16 lines
30 lines
44 lines
66 lines
34 lines
10 lines
146 lines
20 lines
20 lines
4 lines
84 lines
221 lines
44 lines
264 lines
12 lines
126 lines
8 lines
89 lines
87 lines
40 lines
72 lines
30 lines
24 lines
4 lines
20 lines
30 lines
102 lines
4 lines
16 lines
24 lines
12 lines
18 lines
48 lines
64 lines
16 lines
28 lines
32 lines
36 lines
116 lines
497 lines
495 lines
148 lines
48 lines
281 lines
141 lines
16 lines
8 lines
190 lines
155 lines
52 lines
92 lines
232 lines
74 lines
86 lines
20 lines
30 lines
44 lines
66 lines
125 lines
16 lines
2 lines
22 lines
124 lines
112 lines
110 lines
18 lines
16 lines
4 lines
18 lines
70 lines
34 lines
2 lines
36 lines
2 lines
16 lines
44 lines
6 lines
12 lines
9 lines
184 lines
16 lines
26 lines
42 lines
76 lines
66 lines
6 lines
20 lines
6 lines
24 lines
52 lines
42 lines
212 lines
88 lines
6 lines
20 lines
79 lines
236 lines
108 lines
8 lines
376 lines
68 lines
24 lines
48 lines
48 lines
48 lines
560 lines
282 lines
24 lines
84 lines
8 lines
8 lines
60 lines
66 lines
20 lines
292 lines
180 lines
314 lines
98 lines
4 lines
152 lines
48 lines
8 lines
2 lines
30 lines
24 lines
96 lines
80 lines
52 lines
12 lines
30 lines
12 lines
146 lines
4 lines
4 lines
12 lines
12 lines
46 lines
2 lines
753 lines
437 lines
6 lines
116 lines
12 lines
2 lines
23 lines
200 lines
32 lines
34 lines
20 lines
12 lines
30 lines
10 lines
20 lines
4 lines
32 lines
12 lines
42 lines
4 lines
63 lines
8 lines
12 lines
2 lines
4 lines
24 lines
2 lines
2 lines
40 lines
136 lines
440 lines
68 lines
3 lines
2 lines
6 lines
12 lines
292 lines
52 lines
18 lines
2 lines
50 lines
12 lines
6 lines
12 lines
4 lines
12 lines
28 lines
6 lines
92 lines
4 lines
24 lines
10 lines
94 lines
88 lines
24 lines
278 lines
228 lines
106 lines
10 lines
162 lines
40 lines
124 lines
111 lines
126 lines
30 lines
756 lines
48 lines
244 lines
172 lines
68 lines
556 lines
422 lines
16 lines
96 lines
176 lines
164 lines
110 lines
586 lines
44 lines
148 lines
427 lines
58 lines
56 lines
28 lines
78 lines
124 lines
160 lines
59 lines
70 lines
1246 lines
162 lines
14 lines
48 lines
28 lines
10 lines
10 lines
788 lines
114 lines
44 lines
26 lines
212 lines
2 lines
12 lines
22 lines
12 lines
44 lines
42 lines
48 lines
242 lines
32 lines
96 lines
96 lines
440 lines
254 lines
276 lines
230 lines
122 lines
34 lines
20 lines
24 lines
38 lines
82 lines
76 lines
2 lines
24 lines
14 lines
2 lines
112 lines
16 lines
26 lines
24 lines
190 lines
30 lines
36 lines
28 lines
112 lines
64 lines
6 lines
585 lines
293 lines
20 lines
8 lines
98 lines
216 lines
318 lines
4 lines
36 lines
14 lines
32 lines
10 lines
32 lines
10 lines
245 lines
372 lines
4 lines
162 lines
41 lines
588 lines
100 lines
263 lines
16 lines
68 lines
78 lines
2 lines
360 lines
2197 lines
MC/
AArch64/
32 lines
MachineVerifier/
2 lines
Transforms/
CanonicalizeFreezeInLoops/
4 lines
CodeGenPrepare/
AArch64/
30 lines
LoopStrengthReduce/
AArch64/
22 lines
2 lines
4 lines
tools/
UpdateTestChecks/
update_llc_test_checks/
Inputs/
24 lines
20 lines

Diff 546520

llvm/lib/Target/AArch64/AArch64.td

llvm/test/Analysis/CostModel/AArch64/vector-select.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-lse2.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-lse2_lse128.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-outline_atomics.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-rcpc.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-rcpc3.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-v8_1a.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64-atomicrmw-v8a.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-lse2.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-lse2_lse128.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-outline_atomics.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-rcpc.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-rcpc3.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-v8_1a.ll

llvm/test/CodeGen/AArch64/Atomics/aarch64_be-atomicrmw-v8a.ll

llvm/test/CodeGen/AArch64/GlobalISel/aapcs_vararg_frame.ll

llvm/test/CodeGen/AArch64/GlobalISel/arm64-atomic.ll

llvm/test/CodeGen/AArch64/GlobalISel/arm64-pcsections.ll

llvm/test/CodeGen/AArch64/GlobalISel/call-translator-variadic-musttail.ll

llvm/test/CodeGen/AArch64/GlobalISel/combine-udiv.ll

llvm/test/CodeGen/AArch64/GlobalISel/select-bitfield-insert.ll

llvm/test/CodeGen/AArch64/GlobalISel/store-merging.ll

llvm/test/CodeGen/AArch64/GlobalISel/swifterror.ll

Context not available.
; "caller2" is the caller of "foo", it calls "foo" inside a loop. ; "caller2" is the caller of "foo", it calls "foo" inside a loop.
define float @caller2(ptr %error_ref) { define float @caller2(ptr %error_ref) {
; CHECK-LABEL: caller2: ; CHECK-LABEL: caller2:
; CHECK: mov [[ID:x[0-9]+]], x0
; CHECK: fmov [[CMP:s[0-9]+]], #1.0 ; CHECK: fmov [[CMP:s[0-9]+]], #1.0
; CHECK: mov [[ID:x[0-9]+]], x0
; CHECK: mov x21, xzr ; CHECK: mov x21, xzr
; CHECK: bl {{.*}}foo ; CHECK: bl {{.*}}foo
; CHECK: cbnz x21 ; CHECK: cbnz x21
Context not available.
; CHECK-DAG: mov w0, #16 ; CHECK-DAG: mov w0, #16
; CHECK: malloc ; CHECK: malloc
; CHECK: mov [[ID:w[0-9]+]], #1 ; CHECK: mov [[ID:w[0-9]+]], #1
; CHECK: mov x21, x0
; CHECK: strb [[ID]], [x0, #8] ; CHECK: strb [[ID]], [x0, #8]
; CHECK: mov x21, x0
; CHECK: str w{{.*}}, [{{.*}}[[SRET]], #4] ; CHECK: str w{{.*}}, [{{.*}}[[SRET]], #4]
; CHECK-NOT: x21 ; CHECK-NOT: x21
Context not available.
; CHECK: mov w0, #16 ; CHECK: mov w0, #16
; CHECK: malloc ; CHECK: malloc
; CHECK: mov [[ID:w[0-9]+]], #1 ; CHECK: mov [[ID:w[0-9]+]], #1
; CHECK: mov x21, x0
; CHECK-NOT: x21
; CHECK: strb [[ID]], [x0, #8] ; CHECK: strb [[ID]], [x0, #8]
; CHECK: mov x21, x0
; CHECK-NOT: x21 ; CHECK-NOT: x21
; First vararg ; First vararg
Context not available.
; CHECK: str xzr, [sp] ; CHECK: str xzr, [sp]
; CHECK: bl _params_in_reg2 ; CHECK: bl _params_in_reg2
; Restore original arguments for next call. ; Restore original arguments for next call.
; CHECK: ldr x8, [sp, #24]
; CHECK: mov x1, x20 ; CHECK: mov x1, x20
; CHECK: mov x2, x22 ; CHECK: mov x2, x22
; CHECK: mov x3, x23 ; CHECK: mov x3, x23
Context not available.
; CHECK: mov x7, x27 ; CHECK: mov x7, x27
; Restore original swiftself argument and swifterror %err. ; Restore original swiftself argument and swifterror %err.
; CHECK: mov x21, x28 ; CHECK: mov x21, x28
; CHECK: ldr x8, [sp
; CHECK: bl _params_in_reg2 ; CHECK: bl _params_in_reg2
; Restore calle save registers but don't clober swifterror x21. ; Restore calle save registers but don't clober swifterror x21.
; CHECK-NOT: x21 ; CHECK-NOT: x21
; CHECK: ldp x29, x30, [sp ; CHECK: ldp x29, x30, [sp
; CHECK-NOT: x21 ; CHECK-NOT: x21
; CHECK: ldr x28, [sp
; CHECK-NOT: x21
; CHECK: ldp x20, x19, [sp ; CHECK: ldp x20, x19, [sp
; CHECK-NOT: x21 ; CHECK-NOT: x21
; CHECK: ldp x23, x22, [sp ; CHECK: ldp x23, x22, [sp
Context not available.
; CHECK-NOT: x21 ; CHECK-NOT: x21
; CHECK: ldp x27, x26, [sp ; CHECK: ldp x27, x26, [sp
; CHECK-NOT: x21 ; CHECK-NOT: x21
; CHECK: ldr x28, [sp
; CHECK-NOT: x21
; CHECK: ret ; CHECK: ret
define swiftcc void @params_in_reg(i64, i64, i64, i64, i64, i64, i64, i64, ptr, ptr nocapture swifterror %err) { define swiftcc void @params_in_reg(i64, i64, i64, i64, i64, i64, i64, i64, ptr, ptr nocapture swifterror %err) {
%error_ptr_ref = alloca swifterror ptr, align 8 %error_ptr_ref = alloca swifterror ptr, align 8
Context not available.
; CHECK-LABEL: params_and_return_in_reg ; CHECK-LABEL: params_and_return_in_reg
; Store callee saved registers. ; Store callee saved registers.
; CHECK: stp x28, x21, [sp, #16 ; CHECK: stp x28, x0, [sp, #16
; CHECK: stp x27, x26, [sp ; CHECK: stp x27, x26, [sp
; CHECK: stp x25, x24, [sp ; CHECK: stp x25, x24, [sp
; CHECK: stp x23, x22, [sp ; CHECK: stp x23, x22, [sp
Context not available.
; CHECK: mov x21, xzr ; CHECK: mov x21, xzr
; CHECK: bl _params_in_reg2 ; CHECK: bl _params_in_reg2
; Store swifterror %error_ptr_ref. ; Store swifterror %error_ptr_ref.
; CHECK: ldr x0, [sp, #24]
; CHECK: stp {{x[0-9]+}}, x21, [sp] ; CHECK: stp {{x[0-9]+}}, x21, [sp]
; Setup call arguments from original arguments. ; Setup call arguments from original arguments.
; CHECK: mov x0, x19
; CHECK: mov x1, x20 ; CHECK: mov x1, x20
; CHECK: mov x2, x22 ; CHECK: mov x2, x22
; CHECK: mov x3, x23 ; CHECK: mov x3, x23
Context not available.
; CHECK: mov x5, x25 ; CHECK: mov x5, x25
; CHECK: mov x6, x26 ; CHECK: mov x6, x26
; CHECK: mov x7, x27 ; CHECK: mov x7, x27
; CHECK: ldr x21, [sp, #24 ; CHECK: mov x21, x28
; CHECK: bl _params_and_return_in_reg2 ; CHECK: bl _params_and_return_in_reg2
; CHECK: mov x19, x21
; CHECK: ldr x21, [sp, #8
; Store return values. ; Store return values.
; CHECK: mov x19, x0 ; CHECK: mov x20, x0
; CHECK: mov x20, x1 ; CHECK: mov x22, x1
; CHECK: mov x22, x2 ; CHECK: mov x23, x2
; CHECK: mov x23, x3 ; CHECK: mov x24, x3
; CHECK: mov x24, x4 ; CHECK: mov x25, x4
; CHECK: mov x25, x5 ; CHECK: mov x26, x5
; CHECK: mov x26, x6 ; CHECK: mov x27, x6
; CHECK: mov x27, x7 ; CHECK: mov x28, x7
; Save swifterror %err.
; CHECK: mov x28, x21
; Setup call. ; Setup call.
; CHECK: mov w0, #1 ; CHECK: mov w0, #1
; CHECK: mov w1, #2 ; CHECK: mov w1, #2
Context not available.
; CHECK: mov w5, #6 ; CHECK: mov w5, #6
; CHECK: mov w6, #7 ; CHECK: mov w6, #7
; CHECK: mov w7, #8 ; CHECK: mov w7, #8
; ... setup call with swiferror %error_ptr_ref. ; CHECK: str xzr, [sp]
; CHECK: ldr x21, [sp, #8]
; CHECK: bl _params_in_reg2 ; CHECK: bl _params_in_reg2
; Restore return values for return from this function. ; Restore return values for return from this function.
; CHECK: mov x0, x19 ; CHECK: mov x0, x20
; CHECK: mov x1, x20 ; CHECK: mov x1, x22
; CHECK: mov x2, x22 ; CHECK: mov x2, x23
; CHECK: mov x3, x23 ; CHECK: mov x3, x24
; CHECK: mov x4, x24 ; CHECK: mov x4, x25
; CHECK: mov x5, x25 ; CHECK: mov x5, x26
; CHECK: mov x6, x26 ; CHECK: mov x6, x27
; CHECK: mov x7, x27 ; CHECK: mov x21, x19
; CHECK: ldp x29, x30, [sp ; CHECK: mov x7, x28
; CHECK: mov x21, x28 ; CHECK: ldp x29, x30, [sp, #96] ; 16-byte Folded Reload
; Restore callee save registers. ; CHECK: ldr x28, [sp, #16] ; 8-byte Folded Reload
; CHECK: ldp x20, x19, [sp ; CHECK: ldp x20, x19, [sp, #80] ; 16-byte Folded Reload
; CHECK: ldp x23, x22, [sp ; CHECK: ldp x23, x22, [sp, #64] ; 16-byte Folded Reload
; CHECK: ldp x25, x24, [sp ; CHECK: ldp x25, x24, [sp, #48] ; 16-byte Folded Reload
; CHECK: ldp x27, x26, [sp ; CHECK: ldp x27, x26, [sp, #32] ; 16-byte Folded Reload
; CHECK: ldr x28, [sp ; CHECK: add sp, sp, #112
; CHECK: ret ; CHECK: ret
define swiftcc { i64, i64, i64, i64, i64, i64, i64, i64 } @params_and_return_in_reg(i64, i64, i64, i64, i64, i64, i64, i64, ptr , ptr nocapture swifterror %err) { define swiftcc { i64, i64, i64, i64, i64, i64, i64, i64 } @params_and_return_in_reg(i64, i64, i64, i64, i64, i64, i64, i64, ptr , ptr nocapture swifterror %err) {
%error_ptr_ref = alloca swifterror ptr, align 8 %error_ptr_ref = alloca swifterror ptr, align 8
Context not available.

llvm/test/CodeGen/AArch64/a57-csel.ll

llvm/test/CodeGen/AArch64/aarch64-addv.ll

llvm/test/CodeGen/AArch64/aarch64-be-bv.ll

llvm/test/CodeGen/AArch64/aarch64-bit-gen.ll

llvm/test/CodeGen/AArch64/aarch64-combine-add-sub-mul.ll

llvm/test/CodeGen/AArch64/aarch64-dup-ext-scalable.ll

llvm/test/CodeGen/AArch64/aarch64-dup-ext.ll

llvm/test/CodeGen/AArch64/aarch64-dynamic-stack-layout.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc -verify-machineinstrs -mtriple=aarch64-none-linux-gnu -disable-post-ra < %s | FileCheck %s ; RUN: llc -verify-machineinstrs -mtriple=aarch64-none-linux-gnu -disable-post-ra < %s | FileCheck %s
; RUN: llc -verify-machineinstrs -mtriple=arm64-apple-ios -frame-pointer=all -disable-post-ra < %s | FileCheck %s --check-prefix=CHECK-MACHO ; RUN: llc -verify-machineinstrs -mtriple=arm64-apple-ios -frame-pointer=all -disable-post-ra < %s | FileCheck %s --check-prefix=CHECK-MACHO
; This test aims to check basic correctness of frame layout & ; This test aims to check basic correctness of frame layout &
; frame access code. There are 8 functions in this test file, ; frame access code. There are 8 functions in this test file,
; each function implements one element in the cartesian product ; each function implements one element in the cartesian product
Context not available.
; volatile int vla[i1]; ; volatile int vla[i1];
; return i10 + (int)d10 + l1 + vla[0]; ; return i10 + (int)d10 + l1 + vla[0];
;} ;}
define i32 @novla_nodynamicrealign_call(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #0 { define i32 @novla_nodynamicrealign_call(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #0 {
;
; CHECK-LABEL: novla_nodynamicrealign_call:
dmgreenUnsubmitted
Done
ReplyInline Actions

This looks like it would be better with the manual check lines below.

dmgreen: This looks like it would be better with the manual check lines below.
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: sub sp, sp, #32
; CHECK-NEXT: stp x30, x19, [sp, #16] // 16-byte Folded Spill
; CHECK-NEXT: .cfi_def_cfa_offset 32
; CHECK-NEXT: .cfi_offset w19, -8
; CHECK-NEXT: .cfi_offset w30, -16
; CHECK-NEXT: ldr d0, [sp, #56]
; CHECK-NEXT: fcvtzs w8, d0
; CHECK-NEXT: ldr w9, [sp, #40]
; CHECK-NEXT: ldr w10, [sp, #12]
; CHECK-NEXT: add w8, w8, w9
; CHECK-NEXT: orr w19, w8, w10
; CHECK-NEXT: bl g
; CHECK-NEXT: add w0, w19, w0
; CHECK-NEXT: ldp x30, x19, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: add sp, sp, #32
; CHECK-NEXT: ret
;
; CHECK-MACHO-LABEL: novla_nodynamicrealign_call:
; CHECK-MACHO: ; %bb.0: ; %entry
; CHECK-MACHO-NEXT: sub sp, sp, #48
; CHECK-MACHO-NEXT: stp x20, x19, [sp, #16] ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: stp x29, x30, [sp, #32] ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: add x29, sp, #32
; CHECK-MACHO-NEXT: .cfi_def_cfa w29, 16
; CHECK-MACHO-NEXT: .cfi_offset w30, -8
; CHECK-MACHO-NEXT: .cfi_offset w29, -16
; CHECK-MACHO-NEXT: .cfi_offset w19, -24
; CHECK-MACHO-NEXT: .cfi_offset w20, -32
; CHECK-MACHO-NEXT: ldr d0, [x29, #32]
; CHECK-MACHO-NEXT: fcvtzs w8, d0
; CHECK-MACHO-NEXT: ldr w9, [x29, #20]
; CHECK-MACHO-NEXT: ldr w10, [sp, #12]
; CHECK-MACHO-NEXT: add w8, w8, w9
; CHECK-MACHO-NEXT: orr w19, w8, w10
; CHECK-MACHO-NEXT: bl _g
; CHECK-MACHO-NEXT: add w0, w19, w0
; CHECK-MACHO-NEXT: ldp x29, x30, [sp, #32] ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ldp x20, x19, [sp, #16] ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: add sp, sp, #48
; CHECK-MACHO-NEXT: ret
entry: entry:
%l1 = alloca i32, align 4 %l1 = alloca i32, align 4
%conv = fptosi double %d10 to i32 %conv = fptosi double %d10 to i32
Context not available.
%add2 = add nsw i32 %add1, %call %add2 = add nsw i32 %add1, %call
ret i32 %add2 ret i32 %add2
} }
; CHECK-LABEL: novla_nodynamicrealign_call
; CHECK: .cfi_startproc
; Check that used callee-saved registers are saved ; Check that used callee-saved registers are saved
; CHECK: sub sp, sp, #32
; CHECK: stp x30, x19, [sp, #16]
; Check correctness of cfi pseudo-instructions ; Check correctness of cfi pseudo-instructions
; CHECK: .cfi_def_cfa_offset 32
; CHECK: .cfi_offset w19, -8
; CHECK: .cfi_offset w30, -16
; Check correct access to arguments passed on the stack, through stack pointer ; Check correct access to arguments passed on the stack, through stack pointer
; CHECK: ldr d[[DARG:[0-9]+]], [sp, #56]
; CHECK: ldr w[[IARG:[0-9]+]], [sp, #40]
; Check correct access to local variable on the stack, through stack pointer ; Check correct access to local variable on the stack, through stack pointer
; CHECK: ldr w[[ILOC:[0-9]+]], [sp, #12]
; Check epilogue: ; Check epilogue:
; CHECK: ldp x30, x19, [sp, #16]
; CHECK: ret
; CHECK: .cfi_endproc
; CHECK-MACHO-LABEL: _novla_nodynamicrealign_call:
; CHECK-MACHO: .cfi_startproc
; Check that used callee-saved registers are saved ; Check that used callee-saved registers are saved
; CHECK-MACHO: sub sp, sp, #48
; CHECK-MACHO: stp x20, x19, [sp, #16]
; Check that the frame pointer is created: ; Check that the frame pointer is created:
; CHECK-MACHO: stp x29, x30, [sp, #32]
; CHECK-MACHO: add x29, sp, #32
; Check correctness of cfi pseudo-instructions ; Check correctness of cfi pseudo-instructions
; CHECK-MACHO: .cfi_def_cfa w29, 16
; CHECK-MACHO: .cfi_offset w30, -8
; CHECK-MACHO: .cfi_offset w29, -16
; CHECK-MACHO: .cfi_offset w19, -24
; CHECK-MACHO: .cfi_offset w20, -32
; Check correct access to arguments passed on the stack, through frame pointer ; Check correct access to arguments passed on the stack, through frame pointer
; CHECK-MACHO: ldr d[[DARG:[0-9]+]], [x29, #32]
; CHECK-MACHO: ldr w[[IARG:[0-9]+]], [x29, #20]
; Check correct access to local variable on the stack, through stack pointer ; Check correct access to local variable on the stack, through stack pointer
; CHECK-MACHO: ldr w[[ILOC:[0-9]+]], [sp, #12]
; Check epilogue: ; Check epilogue:
; CHECK-MACHO: ldp x29, x30, [sp, #32]
; CHECK-MACHO: ldp x20, x19, [sp, #16]
; CHECK-MACHO: ret
; CHECK-MACHO: .cfi_endproc
declare i32 @g() #0 declare i32 @g() #0
; Function Attrs: nounwind ; Function Attrs: nounwind
define i32 @novla_nodynamicrealign_nocall(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #1 { define i32 @novla_nodynamicrealign_nocall(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #1 {
;
; CHECK-LABEL: novla_nodynamicrealign_nocall:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: sub sp, sp, #16
; CHECK-NEXT: ldr d0, [sp, #40]
; CHECK-NEXT: fcvtzs w8, d0
; CHECK-NEXT: ldr w9, [sp, #24]
; CHECK-NEXT: ldr w10, [sp, #12]
; CHECK-NEXT: add w9, w9, w10
; CHECK-NEXT: add w0, w8, w9
; CHECK-NEXT: add sp, sp, #16
; CHECK-NEXT: ret
;
; CHECK-MACHO-LABEL: novla_nodynamicrealign_nocall:
; CHECK-MACHO: ; %bb.0: ; %entry
; CHECK-MACHO-NEXT: sub sp, sp, #32
; CHECK-MACHO-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: add x29, sp, #16
; CHECK-MACHO-NEXT: ldr d0, [x29, #32]
; CHECK-MACHO-NEXT: fcvtzs w8, d0
; CHECK-MACHO-NEXT: ldr w9, [x29, #20]
; CHECK-MACHO-NEXT: ldur w10, [x29, #-4]
; CHECK-MACHO-NEXT: add w9, w9, w10
; CHECK-MACHO-NEXT: add w0, w8, w9
; CHECK-MACHO-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: add sp, sp, #32
; CHECK-MACHO-NEXT: ret
entry: entry:
%l1 = alloca i32, align 4 %l1 = alloca i32, align 4
%conv = fptosi double %d10 to i32 %conv = fptosi double %d10 to i32
Context not available.
%add1 = add nsw i32 %add, %l1.0.l1.0. %add1 = add nsw i32 %add, %l1.0.l1.0.
ret i32 %add1 ret i32 %add1
} }
; CHECK-LABEL: novla_nodynamicrealign_nocall
; Check that space is reserved for one local variable on the stack. ; Check that space is reserved for one local variable on the stack.
; CHECK: sub sp, sp, #16
; Check correct access to arguments passed on the stack, through stack pointer ; Check correct access to arguments passed on the stack, through stack pointer
; CHECK: ldr d[[DARG:[0-9]+]], [sp, #40]
; CHECK: ldr w[[IARG:[0-9]+]], [sp, #24]
; Check correct access to local variable on the stack, through stack pointer ; Check correct access to local variable on the stack, through stack pointer
; CHECK: ldr w[[ILOC:[0-9]+]], [sp, #12]
; Check epilogue: ; Check epilogue:
; CHECK: add sp, sp, #16
; CHECK: ret
define i32 @novla_dynamicrealign_call(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #0 { define i32 @novla_dynamicrealign_call(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #0 {
;
; CHECK-LABEL: novla_dynamicrealign_call:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: stp x29, x30, [sp, #-32]! // 16-byte Folded Spill
; CHECK-NEXT: str x19, [sp, #16] // 8-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: sub x9, sp, #96
; CHECK-NEXT: and sp, x9, #0xffffffffffffff80
; CHECK-NEXT: .cfi_def_cfa w29, 32
; CHECK-NEXT: .cfi_offset w19, -16
; CHECK-NEXT: .cfi_offset w30, -24
; CHECK-NEXT: .cfi_offset w29, -32
; CHECK-NEXT: ldr d0, [x29, #56]
; CHECK-NEXT: fcvtzs w8, d0
; CHECK-NEXT: ldr w9, [x29, #40]
; CHECK-NEXT: ldr w10, [sp]
; CHECK-NEXT: add w8, w8, w9
; CHECK-NEXT: orr w19, w8, w10
; CHECK-NEXT: bl g
; CHECK-NEXT: add w0, w19, w0
; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: ldr x19, [sp, #16] // 8-byte Folded Reload
; CHECK-NEXT: ldp x29, x30, [sp], #32 // 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-MACHO-LABEL: novla_dynamicrealign_call:
; CHECK-MACHO: ; %bb.0: ; %entry
; CHECK-MACHO-NEXT: stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: add x29, sp, #16
; CHECK-MACHO-NEXT: sub x9, sp, #96
; CHECK-MACHO-NEXT: and sp, x9, #0xffffffffffffff80
; CHECK-MACHO-NEXT: .cfi_def_cfa w29, 16
; CHECK-MACHO-NEXT: .cfi_offset w30, -8
; CHECK-MACHO-NEXT: .cfi_offset w29, -16
; CHECK-MACHO-NEXT: .cfi_offset w19, -24
; CHECK-MACHO-NEXT: .cfi_offset w20, -32
; CHECK-MACHO-NEXT: ldr d0, [x29, #32]
; CHECK-MACHO-NEXT: fcvtzs w8, d0
; CHECK-MACHO-NEXT: ldr w9, [x29, #20]
; CHECK-MACHO-NEXT: ldr w10, [sp]
; CHECK-MACHO-NEXT: add w8, w8, w9
; CHECK-MACHO-NEXT: orr w19, w8, w10
; CHECK-MACHO-NEXT: bl _g
; CHECK-MACHO-NEXT: add w0, w19, w0
; CHECK-MACHO-NEXT: sub sp, x29, #16
; CHECK-MACHO-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ret
entry: entry:
%l1 = alloca i32, align 128 %l1 = alloca i32, align 128
%conv = fptosi double %d10 to i32 %conv = fptosi double %d10 to i32
Context not available.
%add2 = add nsw i32 %add1, %call %add2 = add nsw i32 %add1, %call
ret i32 %add2 ret i32 %add2
} }
; CHECK-LABEL: novla_dynamicrealign_call
; CHECK: .cfi_startproc
; Check that used callee-saved registers are saved ; Check that used callee-saved registers are saved
; CHECK: stp x29, x30, [sp, #-32]!
; Check that the frame pointer is created: ; Check that the frame pointer is created:
; CHECK: str x19, [sp, #16]
; CHECK: mov x29, sp
; Check the dynamic realignment of the stack pointer to a 128-byte boundary ; Check the dynamic realignment of the stack pointer to a 128-byte boundary
; CHECK: sub x9, sp, #96
; CHECK: and sp, x9, #0xffffffffffffff80
; Check correctness of cfi pseudo-instructions ; Check correctness of cfi pseudo-instructions
; CHECK: .cfi_def_cfa w29, 32
; CHECK: .cfi_offset w19, -16
; CHECK: .cfi_offset w30, -24
; CHECK: .cfi_offset w29, -32
; Check correct access to arguments passed on the stack, through frame pointer ; Check correct access to arguments passed on the stack, through frame pointer
; CHECK: ldr d[[DARG:[0-9]+]], [x29, #56]
; CHECK: ldr w[[IARG:[0-9]+]], [x29, #40]
; Check correct access to local variable on the stack, through re-aligned stack pointer ; Check correct access to local variable on the stack, through re-aligned stack pointer
; CHECK: ldr w[[ILOC:[0-9]+]], [sp]
; Check epilogue: ; Check epilogue:
; Check that stack pointer get restored from frame pointer. ; Check that stack pointer get restored from frame pointer.
; CHECK: mov sp, x29
; CHECK: ldr x19, [sp, #16]
; CHECK: ldp x29, x30, [sp], #32
; CHECK: ret
; CHECK: .cfi_endproc
; CHECK-MACHO-LABEL: _novla_dynamicrealign_call:
; CHECK-MACHO: .cfi_startproc
; Check that used callee-saved registers are saved ; Check that used callee-saved registers are saved
; CHECK-MACHO: stp x20, x19, [sp, #-32]!
; Check that the frame pointer is created: ; Check that the frame pointer is created:
; CHECK-MACHO: stp x29, x30, [sp, #16]
; CHECK-MACHO: add x29, sp, #16
; Check the dynamic realignment of the stack pointer to a 128-byte boundary ; Check the dynamic realignment of the stack pointer to a 128-byte boundary
; CHECK-MACHO: sub x9, sp, #96
; CHECK-MACHO: and sp, x9, #0xffffffffffffff80
; Check correctness of cfi pseudo-instructions ; Check correctness of cfi pseudo-instructions
; CHECK-MACHO: .cfi_def_cfa w29, 16
; CHECK-MACHO: .cfi_offset w30, -8
; CHECK-MACHO: .cfi_offset w29, -16
; CHECK-MACHO: .cfi_offset w19, -24
; CHECK-MACHO: .cfi_offset w20, -32
; Check correct access to arguments passed on the stack, through frame pointer ; Check correct access to arguments passed on the stack, through frame pointer
; CHECK-MACHO: ldr d[[DARG:[0-9]+]], [x29, #32]
; CHECK-MACHO: ldr w[[IARG:[0-9]+]], [x29, #20]
; Check correct access to local variable on the stack, through re-aligned stack pointer ; Check correct access to local variable on the stack, through re-aligned stack pointer
; CHECK-MACHO: ldr w[[ILOC:[0-9]+]], [sp]
; Check epilogue: ; Check epilogue:
; Check that stack pointer get restored from frame pointer. ; Check that stack pointer get restored from frame pointer.
; CHECK-MACHO: sub sp, x29, #16
; CHECK-MACHO: ldp x29, x30, [sp, #16]
; CHECK-MACHO: ldp x20, x19, [sp], #32
; CHECK-MACHO: ret
; CHECK-MACHO: .cfi_endproc
; Function Attrs: nounwind ; Function Attrs: nounwind
define i32 @novla_dynamicrealign_nocall(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #1 { define i32 @novla_dynamicrealign_nocall(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #1 {
;
; CHECK-LABEL: novla_dynamicrealign_nocall:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: sub x9, sp, #112
; CHECK-NEXT: and sp, x9, #0xffffffffffffff80
; CHECK-NEXT: ldr d0, [x29, #40]
; CHECK-NEXT: fcvtzs w8, d0
; CHECK-NEXT: ldr w9, [x29, #24]
; CHECK-NEXT: ldr w10, [sp]
; CHECK-NEXT: add w9, w9, w10
; CHECK-NEXT: add w0, w8, w9
; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: ldp x29, x30, [sp], #16 // 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-MACHO-LABEL: novla_dynamicrealign_nocall:
; CHECK-MACHO: ; %bb.0: ; %entry
; CHECK-MACHO-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: mov x29, sp
; CHECK-MACHO-NEXT: sub x9, sp, #112
; CHECK-MACHO-NEXT: and sp, x9, #0xffffffffffffff80
; CHECK-MACHO-NEXT: ldr d0, [x29, #32]
; CHECK-MACHO-NEXT: fcvtzs w8, d0
; CHECK-MACHO-NEXT: ldr w9, [x29, #20]
; CHECK-MACHO-NEXT: ldr w10, [sp]
; CHECK-MACHO-NEXT: add w9, w9, w10
; CHECK-MACHO-NEXT: add w0, w8, w9
; CHECK-MACHO-NEXT: mov sp, x29
; CHECK-MACHO-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ret
entry: entry:
%l1 = alloca i32, align 128 %l1 = alloca i32, align 128
%conv = fptosi double %d10 to i32 %conv = fptosi double %d10 to i32
Context not available.
%add1 = add nsw i32 %add, %l1.0.l1.0. %add1 = add nsw i32 %add, %l1.0.l1.0.
ret i32 %add1 ret i32 %add1
} }
; CHECK-LABEL: novla_dynamicrealign_nocall
; Check that the frame pointer is created: ; Check that the frame pointer is created:
; CHECK: stp x29, x30, [sp, #-16]!
; CHECK: mov x29, sp
; Check the dynamic realignment of the stack pointer to a 128-byte boundary ; Check the dynamic realignment of the stack pointer to a 128-byte boundary
; CHECK: sub x9, sp, #112
; CHECK: and sp, x9, #0xffffffffffffff80
; Check correct access to arguments passed on the stack, through frame pointer ; Check correct access to arguments passed on the stack, through frame pointer
; CHECK: ldr d[[DARG:[0-9]+]], [x29, #40]
; CHECK: ldr w[[IARG:[0-9]+]], [x29, #24]
; Check correct access to local variable on the stack, through re-aligned stack pointer ; Check correct access to local variable on the stack, through re-aligned stack pointer
; CHECK: ldr w[[ILOC:[0-9]+]], [sp]
; Check epilogue: ; Check epilogue:
; Check that stack pointer get restored from frame pointer. ; Check that stack pointer get restored from frame pointer.
; CHECK: mov sp, x29
; CHECK: ldp x29, x30, [sp], #16
; CHECK: ret
define i32 @vla_nodynamicrealign_call(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #0 { define i32 @vla_nodynamicrealign_call(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #0 {
;
; CHECK-LABEL: vla_nodynamicrealign_call:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: stp x29, x30, [sp, #-32]! // 16-byte Folded Spill
; CHECK-NEXT: stp x20, x19, [sp, #16] // 16-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: sub sp, sp, #16
; CHECK-NEXT: .cfi_def_cfa w29, 32
; CHECK-NEXT: .cfi_offset w19, -8
; CHECK-NEXT: .cfi_offset w20, -16
; CHECK-NEXT: .cfi_offset w30, -24
; CHECK-NEXT: .cfi_offset w29, -32
; CHECK-NEXT: // kill: def $w0 killed $w0 def $x0
; CHECK-NEXT: ubfiz x8, x0, #2, #32
; CHECK-NEXT: ldr w9, [x29, #40]
; CHECK-NEXT: ldr d0, [x29, #56]
; CHECK-NEXT: add x8, x8, #15
; CHECK-NEXT: and x8, x8, #0x7fffffff0
; CHECK-NEXT: mov x10, sp
; CHECK-NEXT: sub x19, x10, x8
; CHECK-NEXT: mov sp, x19
; CHECK-NEXT: fcvtzs w8, d0
; CHECK-NEXT: ldur w10, [x29, #-4]
; CHECK-NEXT: add w8, w8, w9
; CHECK-NEXT: orr w20, w8, w10
; CHECK-NEXT: bl g
; CHECK-NEXT: ldr w8, [x19]
; CHECK-NEXT: add w8, w0, w8
; CHECK-NEXT: add w0, w20, w8
; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: ldp x20, x19, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: ldp x29, x30, [sp], #32 // 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-MACHO-LABEL: vla_nodynamicrealign_call:
; CHECK-MACHO: ; %bb.0: ; %entry
; CHECK-MACHO-NEXT: stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: add x29, sp, #16
; CHECK-MACHO-NEXT: sub sp, sp, #16
; CHECK-MACHO-NEXT: .cfi_def_cfa w29, 16
; CHECK-MACHO-NEXT: .cfi_offset w30, -8
; CHECK-MACHO-NEXT: .cfi_offset w29, -16
; CHECK-MACHO-NEXT: .cfi_offset w19, -24
; CHECK-MACHO-NEXT: .cfi_offset w20, -32
; CHECK-MACHO-NEXT: ; kill: def $w0 killed $w0 def $x0
; CHECK-MACHO-NEXT: ubfiz x8, x0, #2, #32
; CHECK-MACHO-NEXT: ldr w9, [x29, #20]
; CHECK-MACHO-NEXT: ldr d0, [x29, #32]
; CHECK-MACHO-NEXT: add x8, x8, #15
; CHECK-MACHO-NEXT: and x8, x8, #0x7fffffff0
; CHECK-MACHO-NEXT: mov x10, sp
; CHECK-MACHO-NEXT: sub x19, x10, x8
; CHECK-MACHO-NEXT: mov sp, x19
; CHECK-MACHO-NEXT: fcvtzs w8, d0
; CHECK-MACHO-NEXT: ldur w10, [x29, #-20]
; CHECK-MACHO-NEXT: add w8, w8, w9
; CHECK-MACHO-NEXT: orr w20, w8, w10
; CHECK-MACHO-NEXT: bl _g
; CHECK-MACHO-NEXT: ldr w8, [x19]
; CHECK-MACHO-NEXT: add w8, w0, w8
; CHECK-MACHO-NEXT: add w0, w20, w8
; CHECK-MACHO-NEXT: sub sp, x29, #16
; CHECK-MACHO-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ret
entry: entry:
%l1 = alloca i32, align 4 %l1 = alloca i32, align 4
%0 = zext i32 %i1 to i64 %0 = zext i32 %i1 to i64
Context not available.
%add3 = add nsw i32 %add2, %1 %add3 = add nsw i32 %add2, %1
ret i32 %add3 ret i32 %add3
} }
; CHECK-LABEL: vla_nodynamicrealign_call
; CHECK: .cfi_startproc
; Check that used callee-saved registers are saved ; Check that used callee-saved registers are saved
; CHECK: stp x29, x30, [sp, #-32]!
; Check that the frame pointer is created: ; Check that the frame pointer is created:
; CHECK: stp x20, x19, [sp, #16]
; CHECK: mov x29, sp
; Check that space is reserved on the stack for the local variable, ; Check that space is reserved on the stack for the local variable,
; rounded up to a multiple of 16 to keep the stack pointer 16-byte aligned. ; rounded up to a multiple of 16 to keep the stack pointer 16-byte aligned.
; CHECK: sub sp, sp, #16
; Check correctness of cfi pseudo-instructions ; Check correctness of cfi pseudo-instructions
; CHECK: .cfi_def_cfa w29, 32
; CHECK: .cfi_offset w19, -8
; CHECK: .cfi_offset w20, -16
; CHECK: .cfi_offset w30, -24
; CHECK: .cfi_offset w29, -32
; Check correct access to arguments passed on the stack, through frame pointer ; Check correct access to arguments passed on the stack, through frame pointer
; CHECK: ldr w[[IARG:[0-9]+]], [x29, #40]
; Check correct reservation of 16-byte aligned VLA (size in w0) on stack ; Check correct reservation of 16-byte aligned VLA (size in w0) on stack
; CHECK: ubfiz x9, x0, #2, #32
; CHECK: add x9, x9, #15
; CHECK: ldr d[[DARG:[0-9]+]], [x29, #56]
; CHECK: and x9, x9, #0x7fffffff0
; CHECK: mov x10, sp
; CHECK: sub x[[VLASPTMP:[0-9]+]], x10, x9
; CHECK: mov sp, x[[VLASPTMP]]
; Check correct access to local variable, through frame pointer ; Check correct access to local variable, through frame pointer
; CHECK: ldur w[[ILOC:[0-9]+]], [x29, #-4]
; Check correct accessing of the VLA variable through the base pointer ; Check correct accessing of the VLA variable through the base pointer
; CHECK: ldr w[[VLA:[0-9]+]], [x[[VLASPTMP]]]
; Check epilogue: ; Check epilogue:
; Check that stack pointer get restored from frame pointer. ; Check that stack pointer get restored from frame pointer.
; CHECK: mov sp, x29
; CHECK: ldp x20, x19, [sp, #16]
; CHECK: ldp x29, x30, [sp], #32
; CHECK: ret
; CHECK: .cfi_endproc
; Function Attrs: nounwind ; Function Attrs: nounwind
define i32 @vla_nodynamicrealign_nocall(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #1 { define i32 @vla_nodynamicrealign_nocall(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #1 {
;
; CHECK-LABEL: vla_nodynamicrealign_nocall:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: sub sp, sp, #16
; CHECK-NEXT: // kill: def $w0 killed $w0 def $x0
; CHECK-NEXT: ubfiz x8, x0, #2, #32
; CHECK-NEXT: ldr w9, [x29, #24]
; CHECK-NEXT: ldr d0, [x29, #40]
; CHECK-NEXT: add x8, x8, #15
; CHECK-NEXT: and x8, x8, #0x7fffffff0
; CHECK-NEXT: mov x10, sp
; CHECK-NEXT: sub x8, x10, x8
; CHECK-NEXT: mov sp, x8
; CHECK-NEXT: fcvtzs w10, d0
; CHECK-NEXT: ldur w11, [x29, #-4]
; CHECK-NEXT: add w9, w9, w11
; CHECK-NEXT: ldr w8, [x8]
; CHECK-NEXT: add w9, w10, w9
; CHECK-NEXT: add w0, w9, w8
; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: ldp x29, x30, [sp], #16 // 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-MACHO-LABEL: vla_nodynamicrealign_nocall:
; CHECK-MACHO: ; %bb.0: ; %entry
; CHECK-MACHO-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: mov x29, sp
; CHECK-MACHO-NEXT: sub sp, sp, #16
; CHECK-MACHO-NEXT: ; kill: def $w0 killed $w0 def $x0
; CHECK-MACHO-NEXT: ubfiz x8, x0, #2, #32
; CHECK-MACHO-NEXT: ldr w9, [x29, #20]
; CHECK-MACHO-NEXT: ldr d0, [x29, #32]
; CHECK-MACHO-NEXT: add x8, x8, #15
; CHECK-MACHO-NEXT: and x8, x8, #0x7fffffff0
; CHECK-MACHO-NEXT: mov x10, sp
; CHECK-MACHO-NEXT: sub x8, x10, x8
; CHECK-MACHO-NEXT: mov sp, x8
; CHECK-MACHO-NEXT: fcvtzs w10, d0
; CHECK-MACHO-NEXT: ldur w11, [x29, #-4]
; CHECK-MACHO-NEXT: add w9, w9, w11
; CHECK-MACHO-NEXT: ldr w8, [x8]
; CHECK-MACHO-NEXT: add w9, w10, w9
; CHECK-MACHO-NEXT: add w0, w9, w8
; CHECK-MACHO-NEXT: mov sp, x29
; CHECK-MACHO-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ret
entry: entry:
%l1 = alloca i32, align 4 %l1 = alloca i32, align 4
%0 = zext i32 %i1 to i64 %0 = zext i32 %i1 to i64
Context not available.
%add2 = add nsw i32 %add1, %1 %add2 = add nsw i32 %add1, %1
ret i32 %add2 ret i32 %add2
} }
; CHECK-LABEL: vla_nodynamicrealign_nocall
; Check that the frame pointer is created: ; Check that the frame pointer is created:
; CHECK: stp x29, x30, [sp, #-16]!
; CHECK: mov x29, sp
; Check that space is reserved on the stack for the local variable, ; Check that space is reserved on the stack for the local variable,
; rounded up to a multiple of 16 to keep the stack pointer 16-byte aligned. ; rounded up to a multiple of 16 to keep the stack pointer 16-byte aligned.
; CHECK: sub sp, sp, #16
; Check correctness of cfi pseudo-instructions ; Check correctness of cfi pseudo-instructions
; Check correct access to arguments passed on the stack, through frame pointer ; Check correct access to arguments passed on the stack, through frame pointer
; CHECK: ldr w[[IARG:[0-9]+]], [x29, #24]
; Check correct reservation of 16-byte aligned VLA (size in w0) on stack ; Check correct reservation of 16-byte aligned VLA (size in w0) on stack
; CHECK: ubfiz x9, x0, #2, #32
; CHECK: add x9, x9, #15
; CHECK: ldr d[[DARG:[0-9]+]], [x29, #40]
; CHECK: and x9, x9, #0x7fffffff0
; CHECK: mov x10, sp
; CHECK: sub x[[VLASPTMP:[0-9]+]], x10, x9
; CHECK: mov sp, x[[VLASPTMP]]
; Check correct access to local variable, through frame pointer ; Check correct access to local variable, through frame pointer
; CHECK: ldur w[[ILOC:[0-9]+]], [x29, #-4]
; Check correct accessing of the VLA variable through the base pointer ; Check correct accessing of the VLA variable through the base pointer
; CHECK: ldr w[[VLA:[0-9]+]], [x[[VLASPTMP]]]
; Check epilogue: ; Check epilogue:
; Check that stack pointer get restored from frame pointer. ; Check that stack pointer get restored from frame pointer.
; CHECK: mov sp, x29
; CHECK: ldp x29, x30, [sp], #16
; CHECK: ret
define i32 @vla_dynamicrealign_call(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #0 { define i32 @vla_dynamicrealign_call(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #0 {
;
; CHECK-LABEL: vla_dynamicrealign_call:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: stp x29, x30, [sp, #-48]! // 16-byte Folded Spill
; CHECK-NEXT: str x21, [sp, #16] // 8-byte Folded Spill
; CHECK-NEXT: stp x20, x19, [sp, #32] // 16-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: sub x9, sp, #80
; CHECK-NEXT: and sp, x9, #0xffffffffffffff80
; CHECK-NEXT: mov x19, sp
; CHECK-NEXT: .cfi_def_cfa w29, 48
; CHECK-NEXT: .cfi_offset w19, -8
; CHECK-NEXT: .cfi_offset w20, -16
; CHECK-NEXT: .cfi_offset w21, -32
; CHECK-NEXT: .cfi_offset w30, -40
; CHECK-NEXT: .cfi_offset w29, -48
; CHECK-NEXT: // kill: def $w0 killed $w0 def $x0
; CHECK-NEXT: ubfiz x8, x0, #2, #32
; CHECK-NEXT: ldr w9, [x29, #56]
; CHECK-NEXT: ldr d0, [x29, #72]
; CHECK-NEXT: add x8, x8, #15
; CHECK-NEXT: and x8, x8, #0x7fffffff0
; CHECK-NEXT: mov x10, sp
; CHECK-NEXT: sub x20, x10, x8
; CHECK-NEXT: mov sp, x20
; CHECK-NEXT: fcvtzs w8, d0
; CHECK-NEXT: ldr w10, [x19]
; CHECK-NEXT: add w8, w8, w9
; CHECK-NEXT: orr w21, w8, w10
; CHECK-NEXT: bl g
; CHECK-NEXT: ldr w8, [x20]
; CHECK-NEXT: add w8, w0, w8
; CHECK-NEXT: add w0, w21, w8
; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: ldp x20, x19, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: ldr x21, [sp, #16] // 8-byte Folded Reload
; CHECK-NEXT: ldp x29, x30, [sp], #48 // 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-MACHO-LABEL: vla_dynamicrealign_call:
; CHECK-MACHO: ; %bb.0: ; %entry
; CHECK-MACHO-NEXT: stp x22, x21, [sp, #-48]! ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: stp x20, x19, [sp, #16] ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: stp x29, x30, [sp, #32] ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: add x29, sp, #32
; CHECK-MACHO-NEXT: sub x9, sp, #80
; CHECK-MACHO-NEXT: and sp, x9, #0xffffffffffffff80
; CHECK-MACHO-NEXT: mov x19, sp
; CHECK-MACHO-NEXT: .cfi_def_cfa w29, 16
; CHECK-MACHO-NEXT: .cfi_offset w30, -8
; CHECK-MACHO-NEXT: .cfi_offset w29, -16
; CHECK-MACHO-NEXT: .cfi_offset w19, -24
; CHECK-MACHO-NEXT: .cfi_offset w20, -32
; CHECK-MACHO-NEXT: .cfi_offset w21, -40
; CHECK-MACHO-NEXT: .cfi_offset w22, -48
; CHECK-MACHO-NEXT: ; kill: def $w0 killed $w0 def $x0
; CHECK-MACHO-NEXT: ubfiz x8, x0, #2, #32
; CHECK-MACHO-NEXT: ldr w9, [x29, #20]
; CHECK-MACHO-NEXT: ldr d0, [x29, #32]
; CHECK-MACHO-NEXT: add x8, x8, #15
; CHECK-MACHO-NEXT: and x8, x8, #0x7fffffff0
; CHECK-MACHO-NEXT: mov x10, sp
; CHECK-MACHO-NEXT: sub x20, x10, x8
; CHECK-MACHO-NEXT: mov sp, x20
; CHECK-MACHO-NEXT: fcvtzs w8, d0
; CHECK-MACHO-NEXT: ldr w10, [x19]
; CHECK-MACHO-NEXT: add w8, w8, w9
; CHECK-MACHO-NEXT: orr w21, w8, w10
; CHECK-MACHO-NEXT: bl _g
; CHECK-MACHO-NEXT: ldr w8, [x20]
; CHECK-MACHO-NEXT: add w8, w0, w8
; CHECK-MACHO-NEXT: add w0, w21, w8
; CHECK-MACHO-NEXT: sub sp, x29, #32
; CHECK-MACHO-NEXT: ldp x29, x30, [sp, #32] ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ldp x20, x19, [sp, #16] ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ldp x22, x21, [sp], #48 ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ret
entry: entry:
%l1 = alloca i32, align 128 %l1 = alloca i32, align 128
%0 = zext i32 %i1 to i64 %0 = zext i32 %i1 to i64
Context not available.
%add3 = add nsw i32 %add2, %1 %add3 = add nsw i32 %add2, %1
ret i32 %add3 ret i32 %add3
} }
; CHECK-LABEL: vla_dynamicrealign_call
; CHECK: .cfi_startproc
; Check that used callee-saved registers are saved ; Check that used callee-saved registers are saved
; CHECK: stp x29, x30, [sp, #-48]!
; CHECK: str x21, [sp, #16]
; CHECK: stp x20, x19, [sp, #32]
; Check that the frame pointer is created: ; Check that the frame pointer is created:
; CHECK: mov x29, sp
; Check that the stack pointer gets re-aligned to 128 ; Check that the stack pointer gets re-aligned to 128
; bytes & the base pointer (x19) gets initialized to ; bytes & the base pointer (x19) gets initialized to
; this 128-byte aligned area for local variables & ; this 128-byte aligned area for local variables &
; spill slots ; spill slots
; CHECK: sub x9, sp, #80
; CHECK: and sp, x9, #0xffffffffffffff80
; CHECK: mov x19, sp
; Check correctness of cfi pseudo-instructions ; Check correctness of cfi pseudo-instructions
; CHECK: .cfi_def_cfa w29, 48
; CHECK: .cfi_offset w19, -8
; CHECK: .cfi_offset w20, -16
; CHECK: .cfi_offset w21, -32
; CHECK: .cfi_offset w30, -40
; CHECK: .cfi_offset w29, -48
; Check correct access to arguments passed on the stack, through frame pointer ; Check correct access to arguments passed on the stack, through frame pointer
; CHECK: ldr w[[IARG:[0-9]+]], [x29, #56]
; Check correct reservation of 16-byte aligned VLA (size in w0) on stack ; Check correct reservation of 16-byte aligned VLA (size in w0) on stack
; and set-up of base pointer (x19). ; and set-up of base pointer (x19).
; CHECK: ubfiz x9, x0, #2, #32
; CHECK: add x9, x9, #15
; CHECK: ldr d[[DARG:[0-9]+]], [x29, #72]
; CHECK: and x9, x9, #0x7fffffff0
; CHECK: mov x10, sp
; CHECK: sub x[[VLASPTMP:[0-9]+]], x10, x9
; CHECK: mov sp, x[[VLASPTMP]]
; Check correct access to local variable, through base pointer ; Check correct access to local variable, through base pointer
; CHECK: ldr w[[ILOC:[0-9]+]], [x19]
; CHECK: ldr w[[VLA:[0-9]+]], [x[[VLASPTMP]]]
; Check epilogue: ; Check epilogue:
; Check that stack pointer get restored from frame pointer. ; Check that stack pointer get restored from frame pointer.
; CHECK: mov sp, x29
; CHECK: ldp x20, x19, [sp, #32]
; CHECK: ldr x21, [sp, #16]
; CHECK: ldp x29, x30, [sp], #48
; CHECK: ret
; CHECK: .cfi_endproc
; CHECK-MACHO-LABEL: _vla_dynamicrealign_call:
; CHECK-MACHO: .cfi_startproc
; Check that used callee-saved registers are saved ; Check that used callee-saved registers are saved
; CHECK-MACHO: stp x22, x21, [sp, #-48]!
; CHECK-MACHO: stp x20, x19, [sp, #16]
; Check that the frame pointer is created: ; Check that the frame pointer is created:
; CHECK-MACHO: stp x29, x30, [sp, #32]
; CHECK-MACHO: add x29, sp, #32
; Check that the stack pointer gets re-aligned to 128 ; Check that the stack pointer gets re-aligned to 128
; bytes & the base pointer (x19) gets initialized to ; bytes & the base pointer (x19) gets initialized to
; this 128-byte aligned area for local variables & ; this 128-byte aligned area for local variables &
; spill slots ; spill slots
; CHECK-MACHO: sub x9, sp, #80
; CHECK-MACHO: and sp, x9, #0xffffffffffffff80
; CHECK-MACHO: mov x19, sp
; Check correctness of cfi pseudo-instructions ; Check correctness of cfi pseudo-instructions
; CHECK-MACHO: .cfi_def_cfa w29, 16
; CHECK-MACHO: .cfi_offset w30, -8
; CHECK-MACHO: .cfi_offset w29, -16
; CHECK-MACHO: .cfi_offset w19, -24
; CHECK-MACHO: .cfi_offset w20, -32
; CHECK-MACHO: .cfi_offset w21, -40
; CHECK-MACHO: .cfi_offset w22, -48
; Check correct access to arguments passed on the stack, through frame pointer ; Check correct access to arguments passed on the stack, through frame pointer
; CHECK-MACHO: ldr w[[IARG:[0-9]+]], [x29, #20]
; Check correct reservation of 16-byte aligned VLA (size in w0) on stack ; Check correct reservation of 16-byte aligned VLA (size in w0) on stack
; and set-up of base pointer (x19). ; and set-up of base pointer (x19).
; CHECK-MACHO: ubfiz x9, x0, #2, #32
; CHECK-MACHO: add x9, x9, #15
; CHECK-MACHO: ldr d[[DARG:[0-9]+]], [x29, #32]
; CHECK-MACHO: and x9, x9, #0x7fffffff0
; CHECK-MACHO: mov x10, sp
; CHECK-MACHO: sub x[[VLASPTMP:[0-9]+]], x10, x9
; CHECK-MACHO: mov sp, x[[VLASPTMP]]
; Check correct access to local variable, through base pointer ; Check correct access to local variable, through base pointer
; CHECK-MACHO: ldr w[[ILOC:[0-9]+]], [x19]
; CHECK-MACHO: ldr w[[VLA:[0-9]+]], [x[[VLASPTMP]]]
; Check epilogue: ; Check epilogue:
; Check that stack pointer get restored from frame pointer. ; Check that stack pointer get restored from frame pointer.
; CHECK-MACHO: sub sp, x29, #32
; CHECK-MACHO: ldp x29, x30, [sp, #32]
; CHECK-MACHO: ldp x20, x19, [sp, #16]
; CHECK-MACHO: ldp x22, x21, [sp], #48
; CHECK-MACHO: ret
; CHECK-MACHO: .cfi_endproc
; Function Attrs: nounwind ; Function Attrs: nounwind
define i32 @vla_dynamicrealign_nocall(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #1 { define i32 @vla_dynamicrealign_nocall(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #1 {
;
; CHECK-LABEL: vla_dynamicrealign_nocall:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: stp x29, x30, [sp, #-32]! // 16-byte Folded Spill
; CHECK-NEXT: str x19, [sp, #16] // 8-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: sub x9, sp, #96
; CHECK-NEXT: and sp, x9, #0xffffffffffffff80
; CHECK-NEXT: mov x19, sp
; CHECK-NEXT: // kill: def $w0 killed $w0 def $x0
; CHECK-NEXT: ubfiz x8, x0, #2, #32
; CHECK-NEXT: ldr w9, [x29, #40]
; CHECK-NEXT: ldr d0, [x29, #56]
; CHECK-NEXT: add x8, x8, #15
; CHECK-NEXT: and x8, x8, #0x7fffffff0
; CHECK-NEXT: mov x10, sp
; CHECK-NEXT: sub x8, x10, x8
; CHECK-NEXT: mov sp, x8
; CHECK-NEXT: fcvtzs w10, d0
; CHECK-NEXT: ldr w11, [x19]
; CHECK-NEXT: add w9, w9, w11
; CHECK-NEXT: ldr w8, [x8]
; CHECK-NEXT: add w9, w10, w9
; CHECK-NEXT: add w0, w9, w8
; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: ldr x19, [sp, #16] // 8-byte Folded Reload
; CHECK-NEXT: ldp x29, x30, [sp], #32 // 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-MACHO-LABEL: vla_dynamicrealign_nocall:
; CHECK-MACHO: ; %bb.0: ; %entry
; CHECK-MACHO-NEXT: stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: add x29, sp, #16
; CHECK-MACHO-NEXT: sub x9, sp, #96
; CHECK-MACHO-NEXT: and sp, x9, #0xffffffffffffff80
; CHECK-MACHO-NEXT: mov x19, sp
; CHECK-MACHO-NEXT: ; kill: def $w0 killed $w0 def $x0
; CHECK-MACHO-NEXT: ubfiz x8, x0, #2, #32
; CHECK-MACHO-NEXT: ldr w9, [x29, #20]
; CHECK-MACHO-NEXT: ldr d0, [x29, #32]
; CHECK-MACHO-NEXT: add x8, x8, #15
; CHECK-MACHO-NEXT: and x8, x8, #0x7fffffff0
; CHECK-MACHO-NEXT: mov x10, sp
; CHECK-MACHO-NEXT: sub x8, x10, x8
; CHECK-MACHO-NEXT: mov sp, x8
; CHECK-MACHO-NEXT: fcvtzs w10, d0
; CHECK-MACHO-NEXT: ldr w11, [x19]
; CHECK-MACHO-NEXT: add w9, w9, w11
; CHECK-MACHO-NEXT: ldr w8, [x8]
; CHECK-MACHO-NEXT: add w9, w10, w9
; CHECK-MACHO-NEXT: add w0, w9, w8
; CHECK-MACHO-NEXT: sub sp, x29, #16
; CHECK-MACHO-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ret
entry: entry:
%l1 = alloca i32, align 128 %l1 = alloca i32, align 128
%0 = zext i32 %i1 to i64 %0 = zext i32 %i1 to i64
Context not available.
%add2 = add nsw i32 %add1, %1 %add2 = add nsw i32 %add1, %1
ret i32 %add2 ret i32 %add2
} }
; CHECK-LABEL: vla_dynamicrealign_nocall
; Check that used callee-saved registers are saved ; Check that used callee-saved registers are saved
; CHECK: stp x29, x30, [sp, #-32]!
; CHECK: str x19, [sp, #16]
; Check that the frame pointer is created: ; Check that the frame pointer is created:
; CHECK: mov x29, sp
; Check that the stack pointer gets re-aligned to 128 ; Check that the stack pointer gets re-aligned to 128
; bytes & the base pointer (x19) gets initialized to ; bytes & the base pointer (x19) gets initialized to
; this 128-byte aligned area for local variables & ; this 128-byte aligned area for local variables &
; spill slots ; spill slots
; CHECK: sub x9, sp, #96
; CHECK: and sp, x9, #0xffffffffffffff80
; CHECK: mov x19, sp
; Check correct access to arguments passed on the stack, through frame pointer ; Check correct access to arguments passed on the stack, through frame pointer
; CHECK: ldr w[[IARG:[0-9]+]], [x29, #40]
; Check correct reservation of 16-byte aligned VLA (size in w0) on stack ; Check correct reservation of 16-byte aligned VLA (size in w0) on stack
; and set-up of base pointer (x19). ; and set-up of base pointer (x19).
; CHECK: ubfiz x9, x0, #2, #32
; CHECK: add x9, x9, #15
; CHECK: ldr d[[DARG:[0-9]+]], [x29, #56]
; CHECK: and x9, x9, #0x7fffffff0
; CHECK: mov x10, sp
; CHECK: sub x[[VLASPTMP:[0-9]+]], x10, x9
; CHECK: mov sp, x[[VLASPTMP]]
; Check correct access to local variable, through base pointer ; Check correct access to local variable, through base pointer
; CHECK: ldr w[[ILOC:[0-9]+]], [x19]
; CHECK: ldr w[[VLA:[0-9]+]], [x[[VLASPTMP]]]
; Check epilogue: ; Check epilogue:
; Check that stack pointer get restored from frame pointer. ; Check that stack pointer get restored from frame pointer.
; CHECK: mov sp, x29
; CHECK: ldr x19, [sp, #16]
; CHECK: ldp x29, x30, [sp], #32
; CHECK: ret
; CHECK-MACHO-LABEL: _vla_dynamicrealign_nocall:
; Check that used callee-saved registers are saved ; Check that used callee-saved registers are saved
; CHECK-MACHO: stp x20, x19, [sp, #-32]!
; Check that the frame pointer is created: ; Check that the frame pointer is created:
; CHECK-MACHO: stp x29, x30, [sp, #16]
; CHECK-MACHO: add x29, sp, #16
; Check that the stack pointer gets re-aligned to 128 ; Check that the stack pointer gets re-aligned to 128
; bytes & the base pointer (x19) gets initialized to ; bytes & the base pointer (x19) gets initialized to
; this 128-byte aligned area for local variables & ; this 128-byte aligned area for local variables &
; spill slots ; spill slots
; CHECK-MACHO: sub x9, sp, #96
; CHECK-MACHO: and sp, x9, #0xffffffffffffff80
; CHECK-MACHO: mov x19, sp
; Check correct access to arguments passed on the stack, through frame pointer ; Check correct access to arguments passed on the stack, through frame pointer
; CHECK-MACHO: ldr w[[IARG:[0-9]+]], [x29, #20]
; Check correct reservation of 16-byte aligned VLA (size in w0) on stack ; Check correct reservation of 16-byte aligned VLA (size in w0) on stack
; and set-up of base pointer (x19). ; and set-up of base pointer (x19).
; CHECK-MACHO: ubfiz x9, x0, #2, #32
; CHECK-MACHO: add x9, x9, #15
; CHECK-MACHO: ldr d[[DARG:[0-9]+]], [x29, #32]
; CHECK-MACHO: and x9, x9, #0x7fffffff0
; CHECK-MACHO: mov x10, sp
; CHECK-MACHO: sub x[[VLASPTMP:[0-9]+]], x10, x9
; CHECK-MACHO: mov sp, x[[VLASPTMP]]
; Check correct access to local variable, through base pointer ; Check correct access to local variable, through base pointer
; CHECK-MACHO: ldr w[[ILOC:[0-9]+]], [x19]
; CHECK-MACHO: ldr w[[VLA:[0-9]+]], [x[[VLASPTMP]]]
; Check epilogue: ; Check epilogue:
; Check that stack pointer get restored from frame pointer. ; Check that stack pointer get restored from frame pointer.
; CHECK-MACHO: sub sp, x29, #16
; CHECK-MACHO: ldp x29, x30, [sp, #16]
; CHECK-MACHO: ldp x20, x19, [sp], #32
; CHECK-MACHO: ret
; Function Attrs: nounwind ; Function Attrs: nounwind
define i32 @vla_dynamicrealign_nocall_large_align(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #1 { define i32 @vla_dynamicrealign_nocall_large_align(i32 %i1, i32 %i2, i32 %i3, i32 %i4, i32 %i5, i32 %i6, i32 %i7, i32 %i8, i32 %i9, i32 %i10, double %d1, double %d2, double %d3, double %d4, double %d5, double %d6, double %d7, double %d8, double %d9, double %d10) #1 {
;
; CHECK-LABEL: vla_dynamicrealign_nocall_large_align:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: stp x29, x30, [sp, #-32]! // 16-byte Folded Spill
; CHECK-NEXT: str x19, [sp, #16] // 8-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: sub x9, sp, #7, lsl #12 // =28672
; CHECK-NEXT: sub x9, x9, #4064
; CHECK-NEXT: and sp, x9, #0xffffffffffff8000
; CHECK-NEXT: mov x19, sp
; CHECK-NEXT: // kill: def $w0 killed $w0 def $x0
; CHECK-NEXT: ubfiz x8, x0, #2, #32
; CHECK-NEXT: ldr w9, [x29, #40]
; CHECK-NEXT: ldr d0, [x29, #56]
; CHECK-NEXT: add x8, x8, #15
; CHECK-NEXT: and x8, x8, #0x7fffffff0
; CHECK-NEXT: mov x10, sp
; CHECK-NEXT: sub x8, x10, x8
; CHECK-NEXT: mov sp, x8
; CHECK-NEXT: fcvtzs w10, d0
; CHECK-NEXT: ldr w11, [x19]
; CHECK-NEXT: add w9, w9, w11
; CHECK-NEXT: ldr w8, [x8]
; CHECK-NEXT: add w9, w10, w9
; CHECK-NEXT: add w0, w9, w8
; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: ldr x19, [sp, #16] // 8-byte Folded Reload
; CHECK-NEXT: ldp x29, x30, [sp], #32 // 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-MACHO-LABEL: vla_dynamicrealign_nocall_large_align:
; CHECK-MACHO: ; %bb.0: ; %entry
; CHECK-MACHO-NEXT: stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: add x29, sp, #16
; CHECK-MACHO-NEXT: sub x9, sp, #7, lsl #12 ; =28672
; CHECK-MACHO-NEXT: sub x9, x9, #4064
; CHECK-MACHO-NEXT: and sp, x9, #0xffffffffffff8000
; CHECK-MACHO-NEXT: mov x19, sp
; CHECK-MACHO-NEXT: ; kill: def $w0 killed $w0 def $x0
; CHECK-MACHO-NEXT: ubfiz x8, x0, #2, #32
; CHECK-MACHO-NEXT: ldr w9, [x29, #20]
; CHECK-MACHO-NEXT: ldr d0, [x29, #32]
; CHECK-MACHO-NEXT: add x8, x8, #15
; CHECK-MACHO-NEXT: and x8, x8, #0x7fffffff0
; CHECK-MACHO-NEXT: mov x10, sp
; CHECK-MACHO-NEXT: sub x8, x10, x8
; CHECK-MACHO-NEXT: mov sp, x8
; CHECK-MACHO-NEXT: fcvtzs w10, d0
; CHECK-MACHO-NEXT: ldr w11, [x19]
; CHECK-MACHO-NEXT: add w9, w9, w11
; CHECK-MACHO-NEXT: ldr w8, [x8]
; CHECK-MACHO-NEXT: add w9, w10, w9
; CHECK-MACHO-NEXT: add w0, w9, w8
; CHECK-MACHO-NEXT: sub sp, x29, #16
; CHECK-MACHO-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ret
entry: entry:
%l1 = alloca i32, align 32768 %l1 = alloca i32, align 32768
%0 = zext i32 %i1 to i64 %0 = zext i32 %i1 to i64
Context not available.
%add2 = add nsw i32 %add1, %1 %add2 = add nsw i32 %add1, %1
ret i32 %add2 ret i32 %add2
} }
; CHECK-LABEL: vla_dynamicrealign_nocall_large_align
; Check that used callee-saved registers are saved ; Check that used callee-saved registers are saved
; CHECK: stp x29, x30, [sp, #-32]!
; CHECK: str x19, [sp, #16]
; Check that the frame pointer is created: ; Check that the frame pointer is created:
; CHECK: mov x29, sp
; Check that the stack pointer gets re-aligned to 128 ; Check that the stack pointer gets re-aligned to 128
; bytes & the base pointer (x19) gets initialized to ; bytes & the base pointer (x19) gets initialized to
; this 128-byte aligned area for local variables & ; this 128-byte aligned area for local variables &
; spill slots ; spill slots
; CHECK: sub x9, sp, #7, lsl #12
; CHECK: and sp, x9, #0xffffffffffff8000
; CHECK: mov x19, sp
; Check correct access to arguments passed on the stack, through frame pointer ; Check correct access to arguments passed on the stack, through frame pointer
; CHECK: ldr w[[IARG:[0-9]+]], [x29, #40]
; Check correct reservation of 16-byte aligned VLA (size in w0) on stack ; Check correct reservation of 16-byte aligned VLA (size in w0) on stack
; and set-up of base pointer (x19). ; and set-up of base pointer (x19).
; CHECK: ubfiz x9, x0, #2, #32
; CHECK: add x9, x9, #15
; CHECK: ldr d[[DARG:[0-9]+]], [x29, #56]
; CHECK: and x9, x9, #0x7fffffff0
; CHECK: mov x10, sp
; CHECK: sub x[[VLASPTMP:[0-9]+]], x10, x9
; CHECK: mov sp, x[[VLASPTMP]]
; Check correct access to local variable, through base pointer ; Check correct access to local variable, through base pointer
; CHECK: ldr w[[ILOC:[0-9]+]], [x19]
; CHECK: ldr w[[VLA:[0-9]+]], [x[[VLASPTMP]]]
; Check epilogue: ; Check epilogue:
; Check that stack pointer get restored from frame pointer. ; Check that stack pointer get restored from frame pointer.
; CHECK: mov sp, x29
; CHECK: ldr x19, [sp, #16]
; CHECK: ldp x29, x30, [sp], #32
; CHECK: ret
; CHECK-MACHO-LABEL: _vla_dynamicrealign_nocall_large_align:
; Check that used callee-saved registers are saved ; Check that used callee-saved registers are saved
; CHECK-MACHO: stp x20, x19, [sp, #-32]!
; Check that the frame pointer is created: ; Check that the frame pointer is created:
; CHECK-MACHO: stp x29, x30, [sp, #16]
; CHECK-MACHO: add x29, sp, #16
; Check that the stack pointer gets re-aligned to 128 ; Check that the stack pointer gets re-aligned to 128
; bytes & the base pointer (x19) gets initialized to ; bytes & the base pointer (x19) gets initialized to
; this 128-byte aligned area for local variables & ; this 128-byte aligned area for local variables &
; spill slots ; spill slots
; CHECK-MACHO: sub x9, sp, #7, lsl #12
; CHECK-MACHO: and sp, x9, #0xffffffffffff8000
; CHECK-MACHO: mov x19, sp
; Check correct access to arguments passed on the stack, through frame pointer ; Check correct access to arguments passed on the stack, through frame pointer
; CHECK-MACHO: ldr w[[IARG:[0-9]+]], [x29, #20]
; Check correct reservation of 16-byte aligned VLA (size in w0) on stack ; Check correct reservation of 16-byte aligned VLA (size in w0) on stack
; and set-up of base pointer (x19). ; and set-up of base pointer (x19).
; CHECK-MACHO: ubfiz x9, x0, #2, #32
; CHECK-MACHO: add x9, x9, #15
; CHECK-MACHO: ldr d[[DARG:[0-9]+]], [x29, #32]
; CHECK-MACHO: and x9, x9, #0x7fffffff0
; CHECK-MACHO: mov x10, sp
; CHECK-MACHO: sub x[[VLASPTMP:[0-9]+]], x10, x9
; CHECK-MACHO: mov sp, x[[VLASPTMP]]
; Check correct access to local variable, through base pointer ; Check correct access to local variable, through base pointer
; CHECK-MACHO: ldr w[[ILOC:[0-9]+]], [x19]
; CHECK-MACHO: ldr w[[VLA:[0-9]+]], [x[[VLASPTMP]]]
; Check epilogue: ; Check epilogue:
; Check that stack pointer get restored from frame pointer. ; Check that stack pointer get restored from frame pointer.
; CHECK-MACHO: sub sp, x29, #16
; CHECK-MACHO: ldp x29, x30, [sp, #16]
; CHECK-MACHO: ldp x20, x19, [sp], #32
; CHECK-MACHO: ret
declare void @use(ptr) declare void @use(ptr)
define void @realign_conditional(i1 %b, ptr %p) { define void @realign_conditional(i1 %b, ptr %p) {
;
; CHECK-LABEL: realign_conditional:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: tbz w0, #0, .LBB9_2
; CHECK-NEXT: // %bb.1: // %bb0
; CHECK-NEXT: stp x29, x30, [sp, #-32]! // 16-byte Folded Spill
; CHECK-NEXT: str x19, [sp, #16] // 8-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: mov x19, sp
; CHECK-NEXT: .cfi_def_cfa w29, 32
; CHECK-NEXT: .cfi_offset w19, -16
; CHECK-NEXT: .cfi_offset w30, -24
; CHECK-NEXT: .cfi_offset w29, -32
; CHECK-NEXT: sub x8, sp, #64
; CHECK-NEXT: and x8, x8, #0xffffffffffffffe0
; CHECK-NEXT: mov sp, x8
; CHECK-NEXT: str x8, [x1]
; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: ldr x19, [sp, #16] // 8-byte Folded Reload
; CHECK-NEXT: ldp x29, x30, [sp], #32 // 16-byte Folded Reload
; CHECK-NEXT: .LBB9_2: // %bb1
; CHECK-NEXT: ret
;
; CHECK-MACHO-LABEL: realign_conditional:
; CHECK-MACHO: ; %bb.0: ; %entry
; CHECK-MACHO-NEXT: tbz w0, #0, LBB9_2
; CHECK-MACHO-NEXT: ; %bb.1: ; %bb0
; CHECK-MACHO-NEXT: stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: add x29, sp, #16
; CHECK-MACHO-NEXT: mov x19, sp
; CHECK-MACHO-NEXT: .cfi_def_cfa w29, 16
; CHECK-MACHO-NEXT: .cfi_offset w30, -8
; CHECK-MACHO-NEXT: .cfi_offset w29, -16
; CHECK-MACHO-NEXT: .cfi_offset w19, -24
; CHECK-MACHO-NEXT: .cfi_offset w20, -32
; CHECK-MACHO-NEXT: sub x8, sp, #64
; CHECK-MACHO-NEXT: and x8, x8, #0xffffffffffffffe0
; CHECK-MACHO-NEXT: mov sp, x8
; CHECK-MACHO-NEXT: str x8, [x1]
; CHECK-MACHO-NEXT: sub sp, x29, #16
; CHECK-MACHO-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: LBB9_2: ; %bb1
; CHECK-MACHO-NEXT: ret
entry: entry:
br i1 %b, label %bb0, label %bb1 br i1 %b, label %bb0, label %bb1
bb0: bb0:
%MyAlloca = alloca i8, i64 64, align 32 %MyAlloca = alloca i8, i64 64, align 32
store ptr %MyAlloca, ptr %p store ptr %MyAlloca, ptr %p
br label %bb1 br label %bb1
bb1: bb1:
ret void ret void
} }
; CHECK-LABEL: realign_conditional
; No realignment in the prologue. ; No realignment in the prologue.
; CHECK-NOT: and
; CHECK-NOT: 0xffffffffffffffe0
; CHECK: tbz {{.*}} .[[LABEL:.*]]
; Stack is realigned in a non-entry BB. ; Stack is realigned in a non-entry BB.
; CHECK: sub [[REG:x[01-9]+]], sp, #64
; CHECK: and [[REG]], [[REG]], #0xffffffffffffffe0
; CHECK: mov sp, [[REG]]
; CHECK: .[[LABEL]]:
; CHECK: ret
define void @realign_conditional2(i1 %b, ptr %p) { define void @realign_conditional2(i1 %b, ptr %p) {
;
; CHECK-LABEL: realign_conditional2:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: tbz w0, #0, .LBB10_2
; CHECK-NEXT: // %bb.1: // %bb0
; CHECK-NEXT: stp x29, x30, [sp, #-32]! // 16-byte Folded Spill
; CHECK-NEXT: str x19, [sp, #16] // 8-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: sub x9, sp, #32
; CHECK-NEXT: and sp, x9, #0xffffffffffffffe0
; CHECK-NEXT: mov x19, sp
; CHECK-NEXT: .cfi_def_cfa w29, 32
; CHECK-NEXT: .cfi_offset w19, -16
; CHECK-NEXT: .cfi_offset w30, -24
; CHECK-NEXT: .cfi_offset w29, -32
; CHECK-NEXT: sub x8, sp, #64
; CHECK-NEXT: and x8, x8, #0xffffffffffffffe0
; CHECK-NEXT: mov sp, x8
; CHECK-NEXT: str x8, [x1]
; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: ldr x19, [sp, #16] // 8-byte Folded Reload
; CHECK-NEXT: ldp x29, x30, [sp], #32 // 16-byte Folded Reload
; CHECK-NEXT: .LBB10_2: // %bb1
; CHECK-NEXT: ret
;
; CHECK-MACHO-LABEL: realign_conditional2:
; CHECK-MACHO: ; %bb.0: ; %entry
; CHECK-MACHO-NEXT: tbz w0, #0, LBB10_2
; CHECK-MACHO-NEXT: ; %bb.1: ; %bb0
; CHECK-MACHO-NEXT: stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-MACHO-NEXT: add x29, sp, #16
; CHECK-MACHO-NEXT: sub x9, sp, #32
; CHECK-MACHO-NEXT: and sp, x9, #0xffffffffffffffe0
; CHECK-MACHO-NEXT: mov x19, sp
; CHECK-MACHO-NEXT: .cfi_def_cfa w29, 16
; CHECK-MACHO-NEXT: .cfi_offset w30, -8
; CHECK-MACHO-NEXT: .cfi_offset w29, -16
; CHECK-MACHO-NEXT: .cfi_offset w19, -24
; CHECK-MACHO-NEXT: .cfi_offset w20, -32
; CHECK-MACHO-NEXT: sub x8, sp, #64
; CHECK-MACHO-NEXT: and x8, x8, #0xffffffffffffffe0
; CHECK-MACHO-NEXT: mov sp, x8
; CHECK-MACHO-NEXT: str x8, [x1]
; CHECK-MACHO-NEXT: sub sp, x29, #16
; CHECK-MACHO-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
; CHECK-MACHO-NEXT: LBB10_2: ; %bb1
; CHECK-MACHO-NEXT: ret
entry: entry:
%tmp = alloca i8, i32 16 %tmp = alloca i8, i32 16
br i1 %b, label %bb0, label %bb1 br i1 %b, label %bb0, label %bb1
bb0: bb0:
%MyAlloca = alloca i8, i64 64, align 32 %MyAlloca = alloca i8, i64 64, align 32
store ptr %MyAlloca, ptr %p store ptr %MyAlloca, ptr %p
br label %bb1 br label %bb1
bb1: bb1:
ret void ret void
} }
; CHECK-LABEL: realign_conditional2
; Extra realignment in the prologue (performance issue). ; Extra realignment in the prologue (performance issue).
; CHECK: tbz {{.*}} .[[LABEL:.*]]
; CHECK: sub x9, sp, #32
; CHECK: and sp, x9, #0xffffffffffffffe0
; CHECK: mov x19, sp
; Stack is realigned in a non-entry BB. ; Stack is realigned in a non-entry BB.
; CHECK: sub [[REG:x[01-9]+]], sp, #64
; CHECK: and [[REG]], [[REG]], #0xffffffffffffffe0
; CHECK: mov sp, [[REG]]
; CHECK: .[[LABEL]]:
; CHECK: ret
attributes #0 = { "less-precise-fpmad"="false" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" } attributes #0 = { "less-precise-fpmad"="false" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" }
attributes #1 = { nounwind "less-precise-fpmad"="false" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" } attributes #1 = { nounwind "less-precise-fpmad"="false" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "unsafe-fp-math"="false" "use-soft-float"="false" }
!1 = !{!2, !2, i64 0} !1 = !{!2, !2, i64 0}
!2 = !{!"int", !3, i64 0} !2 = !{!"int", !3, i64 0}
!3 = !{!"omnipotent char", !4, i64 0} !3 = !{!"omnipotent char", !4, i64 0}
Context not available.

llvm/test/CodeGen/AArch64/aarch64-fixup-statepoint-regs-crash.ll

llvm/test/CodeGen/AArch64/aarch64-fold-lslfast.ll

llvm/test/CodeGen/AArch64/aarch64-interleaved-access-w-undef.ll

llvm/test/CodeGen/AArch64/aarch64-isel-csinc.ll

llvm/test/CodeGen/AArch64/aarch64-load-ext.ll

llvm/test/CodeGen/AArch64/aarch64-matrix-umull-smull.ll

llvm/test/CodeGen/AArch64/aarch64-mops-consecutive.ll

llvm/test/CodeGen/AArch64/aarch64-mops.ll

llvm/test/CodeGen/AArch64/aarch64-mull-masks.ll

llvm/test/CodeGen/AArch64/aarch64-neon-vector-insert-uaddlv.ll

llvm/test/CodeGen/AArch64/aarch64-pmull2.ll

llvm/test/CodeGen/AArch64/aarch64-smull.ll

llvm/test/CodeGen/AArch64/aarch64-uzp1-combine.ll

llvm/test/CodeGen/AArch64/aarch64-wide-mul.ll

llvm/test/CodeGen/AArch64/aarch64_fnmadd.ll

llvm/test/CodeGen/AArch64/aarch64_win64cc_vararg.ll

llvm/test/CodeGen/AArch64/abd-combine.ll

llvm/test/CodeGen/AArch64/active_lane_mask.ll

llvm/test/CodeGen/AArch64/add-extract.ll

llvm/test/CodeGen/AArch64/addcarry-crash.ll

llvm/test/CodeGen/AArch64/addsub-constant-folding.ll

llvm/test/CodeGen/AArch64/addsub.ll

llvm/test/CodeGen/AArch64/align-down.ll

llvm/test/CodeGen/AArch64/and-mask-removal.ll

llvm/test/CodeGen/AArch64/andorbrcompare.ll

llvm/test/CodeGen/AArch64/argument-blocks-array-of-struct.ll

llvm/test/CodeGen/AArch64/arm64-AdvSIMD-Scalar.ll

llvm/test/CodeGen/AArch64/arm64-addr-type-promotion.ll

llvm/test/CodeGen/AArch64/arm64-addrmode.ll

llvm/test/CodeGen/AArch64/arm64-bitfield-extract.ll

llvm/test/CodeGen/AArch64/arm64-convert-v4f64.ll

llvm/test/CodeGen/AArch64/arm64-cse.ll

llvm/test/CodeGen/AArch64/arm64-csel.ll

llvm/test/CodeGen/AArch64/arm64-fcopysign.ll

llvm/test/CodeGen/AArch64/arm64-fmadd.ll

llvm/test/CodeGen/AArch64/arm64-homogeneous-prolog-epilog-no-helper.ll

llvm/test/CodeGen/AArch64/arm64-indexed-vector-ldst.ll

llvm/test/CodeGen/AArch64/arm64-inline-asm.ll

llvm/test/CodeGen/AArch64/arm64-instruction-mix-remarks.ll

llvm/test/CodeGen/AArch64/arm64-ld1.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
dmgreenUnsubmitted
Done
ReplyInline Actions

This doesn't look like it has generated properly.

dmgreen: This doesn't look like it has generated properly.
; RUN: llc < %s -mtriple=arm64-eabi -aarch64-neon-syntax=apple -verify-machineinstrs -asm-verbose=false | FileCheck %s ; RUN: llc < %s -mtriple=arm64-eabi -aarch64-neon-syntax=apple -verify-machineinstrs -asm-verbose=false | FileCheck %s
%struct.__neon_int8x8x2_t = type { <8 x i8>, <8 x i8> } %struct.__neon_int8x8x2_t = type { <8 x i8>, <8 x i8> }
Context not available.
%struct.__neon_int8x8x4_t = type { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> } %struct.__neon_int8x8x4_t = type { <8 x i8>, <8 x i8>, <8 x i8>, <8 x i8> }
define %struct.__neon_int8x8x2_t @ld2_8b(ptr %A) nounwind { define %struct.__neon_int8x8x2_t @ld2_8b(ptr %A) nounwind {
; CHECK-LABEL: ld2_8b
; Make sure we are loading into the results defined by the ABI (i.e., v0, v1) ; Make sure we are loading into the results defined by the ABI (i.e., v0, v1)
; and from the argument of the function also defined by ABI (i.e., x0) ; and from the argument of the function also defined by ABI (i.e., x0)
; CHECK: ld2.8b { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x8x2_t @llvm.aarch64.neon.ld2.v8i8.p0(ptr %A) %tmp2 = call %struct.__neon_int8x8x2_t @llvm.aarch64.neon.ld2.v8i8.p0(ptr %A)
ret %struct.__neon_int8x8x2_t %tmp2 ret %struct.__neon_int8x8x2_t %tmp2
} }
define %struct.__neon_int8x8x3_t @ld3_8b(ptr %A) nounwind { define %struct.__neon_int8x8x3_t @ld3_8b(ptr %A) nounwind {
; CHECK-LABEL: ld3_8b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3.8b { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x8x3_t @llvm.aarch64.neon.ld3.v8i8.p0(ptr %A) %tmp2 = call %struct.__neon_int8x8x3_t @llvm.aarch64.neon.ld3.v8i8.p0(ptr %A)
ret %struct.__neon_int8x8x3_t %tmp2 ret %struct.__neon_int8x8x3_t %tmp2
} }
define %struct.__neon_int8x8x4_t @ld4_8b(ptr %A) nounwind { define %struct.__neon_int8x8x4_t @ld4_8b(ptr %A) nounwind {
; CHECK-LABEL: ld4_8b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4.8b { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x8x4_t @llvm.aarch64.neon.ld4.v8i8.p0(ptr %A) %tmp2 = call %struct.__neon_int8x8x4_t @llvm.aarch64.neon.ld4.v8i8.p0(ptr %A)
ret %struct.__neon_int8x8x4_t %tmp2 ret %struct.__neon_int8x8x4_t %tmp2
} }
Context not available.
%struct.__neon_int8x16x4_t = type { <16 x i8>, <16 x i8>, <16 x i8>, <16 x i8> } %struct.__neon_int8x16x4_t = type { <16 x i8>, <16 x i8>, <16 x i8>, <16 x i8> }
define %struct.__neon_int8x16x2_t @ld2_16b(ptr %A) nounwind { define %struct.__neon_int8x16x2_t @ld2_16b(ptr %A) nounwind {
; CHECK-LABEL: ld2_16b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2.16b { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x16x2_t @llvm.aarch64.neon.ld2.v16i8.p0(ptr %A) %tmp2 = call %struct.__neon_int8x16x2_t @llvm.aarch64.neon.ld2.v16i8.p0(ptr %A)
ret %struct.__neon_int8x16x2_t %tmp2 ret %struct.__neon_int8x16x2_t %tmp2
} }
define %struct.__neon_int8x16x3_t @ld3_16b(ptr %A) nounwind { define %struct.__neon_int8x16x3_t @ld3_16b(ptr %A) nounwind {
; CHECK-LABEL: ld3_16b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3.16b { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x16x3_t @llvm.aarch64.neon.ld3.v16i8.p0(ptr %A) %tmp2 = call %struct.__neon_int8x16x3_t @llvm.aarch64.neon.ld3.v16i8.p0(ptr %A)
ret %struct.__neon_int8x16x3_t %tmp2 ret %struct.__neon_int8x16x3_t %tmp2
} }
define %struct.__neon_int8x16x4_t @ld4_16b(ptr %A) nounwind { define %struct.__neon_int8x16x4_t @ld4_16b(ptr %A) nounwind {
; CHECK-LABEL: ld4_16b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4.16b { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x16x4_t @llvm.aarch64.neon.ld4.v16i8.p0(ptr %A) %tmp2 = call %struct.__neon_int8x16x4_t @llvm.aarch64.neon.ld4.v16i8.p0(ptr %A)
ret %struct.__neon_int8x16x4_t %tmp2 ret %struct.__neon_int8x16x4_t %tmp2
} }
Context not available.
%struct.__neon_int16x4x4_t = type { <4 x i16>, <4 x i16>, <4 x i16>, <4 x i16> } %struct.__neon_int16x4x4_t = type { <4 x i16>, <4 x i16>, <4 x i16>, <4 x i16> }
define %struct.__neon_int16x4x2_t @ld2_4h(ptr %A) nounwind { define %struct.__neon_int16x4x2_t @ld2_4h(ptr %A) nounwind {
; CHECK-LABEL: ld2_4h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2.4h { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x4x2_t @llvm.aarch64.neon.ld2.v4i16.p0(ptr %A) %tmp2 = call %struct.__neon_int16x4x2_t @llvm.aarch64.neon.ld2.v4i16.p0(ptr %A)
ret %struct.__neon_int16x4x2_t %tmp2 ret %struct.__neon_int16x4x2_t %tmp2
} }
define %struct.__neon_int16x4x3_t @ld3_4h(ptr %A) nounwind { define %struct.__neon_int16x4x3_t @ld3_4h(ptr %A) nounwind {
; CHECK-LABEL: ld3_4h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3.4h { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x4x3_t @llvm.aarch64.neon.ld3.v4i16.p0(ptr %A) %tmp2 = call %struct.__neon_int16x4x3_t @llvm.aarch64.neon.ld3.v4i16.p0(ptr %A)
ret %struct.__neon_int16x4x3_t %tmp2 ret %struct.__neon_int16x4x3_t %tmp2
} }
define %struct.__neon_int16x4x4_t @ld4_4h(ptr %A) nounwind { define %struct.__neon_int16x4x4_t @ld4_4h(ptr %A) nounwind {
; CHECK-LABEL: ld4_4h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4.4h { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x4x4_t @llvm.aarch64.neon.ld4.v4i16.p0(ptr %A) %tmp2 = call %struct.__neon_int16x4x4_t @llvm.aarch64.neon.ld4.v4i16.p0(ptr %A)
ret %struct.__neon_int16x4x4_t %tmp2 ret %struct.__neon_int16x4x4_t %tmp2
} }
Context not available.
%struct.__neon_int16x8x4_t = type { <8 x i16>, <8 x i16>, <8 x i16>, <8 x i16> } %struct.__neon_int16x8x4_t = type { <8 x i16>, <8 x i16>, <8 x i16>, <8 x i16> }
define %struct.__neon_int16x8x2_t @ld2_8h(ptr %A) nounwind { define %struct.__neon_int16x8x2_t @ld2_8h(ptr %A) nounwind {
; CHECK-LABEL: ld2_8h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2.8h { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x8x2_t @llvm.aarch64.neon.ld2.v8i16.p0(ptr %A) %tmp2 = call %struct.__neon_int16x8x2_t @llvm.aarch64.neon.ld2.v8i16.p0(ptr %A)
ret %struct.__neon_int16x8x2_t %tmp2 ret %struct.__neon_int16x8x2_t %tmp2
} }
define %struct.__neon_int16x8x3_t @ld3_8h(ptr %A) nounwind { define %struct.__neon_int16x8x3_t @ld3_8h(ptr %A) nounwind {
; CHECK-LABEL: ld3_8h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3.8h { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x8x3_t @llvm.aarch64.neon.ld3.v8i16.p0(ptr %A) %tmp2 = call %struct.__neon_int16x8x3_t @llvm.aarch64.neon.ld3.v8i16.p0(ptr %A)
ret %struct.__neon_int16x8x3_t %tmp2 ret %struct.__neon_int16x8x3_t %tmp2
} }
define %struct.__neon_int16x8x4_t @ld4_8h(ptr %A) nounwind { define %struct.__neon_int16x8x4_t @ld4_8h(ptr %A) nounwind {
; CHECK-LABEL: ld4_8h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4.8h { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x8x4_t @llvm.aarch64.neon.ld4.v8i16.p0(ptr %A) %tmp2 = call %struct.__neon_int16x8x4_t @llvm.aarch64.neon.ld4.v8i16.p0(ptr %A)
ret %struct.__neon_int16x8x4_t %tmp2 ret %struct.__neon_int16x8x4_t %tmp2
} }
Context not available.
%struct.__neon_int32x2x4_t = type { <2 x i32>, <2 x i32>, <2 x i32>, <2 x i32> } %struct.__neon_int32x2x4_t = type { <2 x i32>, <2 x i32>, <2 x i32>, <2 x i32> }
define %struct.__neon_int32x2x2_t @ld2_2s(ptr %A) nounwind { define %struct.__neon_int32x2x2_t @ld2_2s(ptr %A) nounwind {
; CHECK-LABEL: ld2_2s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2.2s { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x2x2_t @llvm.aarch64.neon.ld2.v2i32.p0(ptr %A) %tmp2 = call %struct.__neon_int32x2x2_t @llvm.aarch64.neon.ld2.v2i32.p0(ptr %A)
ret %struct.__neon_int32x2x2_t %tmp2 ret %struct.__neon_int32x2x2_t %tmp2
} }
define %struct.__neon_int32x2x3_t @ld3_2s(ptr %A) nounwind { define %struct.__neon_int32x2x3_t @ld3_2s(ptr %A) nounwind {
; CHECK-LABEL: ld3_2s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3.2s { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x2x3_t @llvm.aarch64.neon.ld3.v2i32.p0(ptr %A) %tmp2 = call %struct.__neon_int32x2x3_t @llvm.aarch64.neon.ld3.v2i32.p0(ptr %A)
ret %struct.__neon_int32x2x3_t %tmp2 ret %struct.__neon_int32x2x3_t %tmp2
} }
define %struct.__neon_int32x2x4_t @ld4_2s(ptr %A) nounwind { define %struct.__neon_int32x2x4_t @ld4_2s(ptr %A) nounwind {
; CHECK-LABEL: ld4_2s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4.2s { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x2x4_t @llvm.aarch64.neon.ld4.v2i32.p0(ptr %A) %tmp2 = call %struct.__neon_int32x2x4_t @llvm.aarch64.neon.ld4.v2i32.p0(ptr %A)
ret %struct.__neon_int32x2x4_t %tmp2 ret %struct.__neon_int32x2x4_t %tmp2
} }
Context not available.
%struct.__neon_int32x4x4_t = type { <4 x i32>, <4 x i32>, <4 x i32>, <4 x i32> } %struct.__neon_int32x4x4_t = type { <4 x i32>, <4 x i32>, <4 x i32>, <4 x i32> }
define %struct.__neon_int32x4x2_t @ld2_4s(ptr %A) nounwind { define %struct.__neon_int32x4x2_t @ld2_4s(ptr %A) nounwind {
; CHECK-LABEL: ld2_4s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2.4s { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x4x2_t @llvm.aarch64.neon.ld2.v4i32.p0(ptr %A) %tmp2 = call %struct.__neon_int32x4x2_t @llvm.aarch64.neon.ld2.v4i32.p0(ptr %A)
ret %struct.__neon_int32x4x2_t %tmp2 ret %struct.__neon_int32x4x2_t %tmp2
} }
define %struct.__neon_int32x4x3_t @ld3_4s(ptr %A) nounwind { define %struct.__neon_int32x4x3_t @ld3_4s(ptr %A) nounwind {
; CHECK-LABEL: ld3_4s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3.4s { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x4x3_t @llvm.aarch64.neon.ld3.v4i32.p0(ptr %A) %tmp2 = call %struct.__neon_int32x4x3_t @llvm.aarch64.neon.ld3.v4i32.p0(ptr %A)
ret %struct.__neon_int32x4x3_t %tmp2 ret %struct.__neon_int32x4x3_t %tmp2
} }
define %struct.__neon_int32x4x4_t @ld4_4s(ptr %A) nounwind { define %struct.__neon_int32x4x4_t @ld4_4s(ptr %A) nounwind {
; CHECK-LABEL: ld4_4s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4.4s { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x4x4_t @llvm.aarch64.neon.ld4.v4i32.p0(ptr %A) %tmp2 = call %struct.__neon_int32x4x4_t @llvm.aarch64.neon.ld4.v4i32.p0(ptr %A)
ret %struct.__neon_int32x4x4_t %tmp2 ret %struct.__neon_int32x4x4_t %tmp2
} }
Context not available.
%struct.__neon_int64x2x4_t = type { <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> } %struct.__neon_int64x2x4_t = type { <2 x i64>, <2 x i64>, <2 x i64>, <2 x i64> }
define %struct.__neon_int64x2x2_t @ld2_2d(ptr %A) nounwind { define %struct.__neon_int64x2x2_t @ld2_2d(ptr %A) nounwind {
; CHECK-LABEL: ld2_2d
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2.2d { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x2x2_t @llvm.aarch64.neon.ld2.v2i64.p0(ptr %A) %tmp2 = call %struct.__neon_int64x2x2_t @llvm.aarch64.neon.ld2.v2i64.p0(ptr %A)
ret %struct.__neon_int64x2x2_t %tmp2 ret %struct.__neon_int64x2x2_t %tmp2
} }
define %struct.__neon_int64x2x3_t @ld3_2d(ptr %A) nounwind { define %struct.__neon_int64x2x3_t @ld3_2d(ptr %A) nounwind {
; CHECK-LABEL: ld3_2d
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3.2d { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x2x3_t @llvm.aarch64.neon.ld3.v2i64.p0(ptr %A) %tmp2 = call %struct.__neon_int64x2x3_t @llvm.aarch64.neon.ld3.v2i64.p0(ptr %A)
ret %struct.__neon_int64x2x3_t %tmp2 ret %struct.__neon_int64x2x3_t %tmp2
} }
define %struct.__neon_int64x2x4_t @ld4_2d(ptr %A) nounwind { define %struct.__neon_int64x2x4_t @ld4_2d(ptr %A) nounwind {
; CHECK-LABEL: ld4_2d
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4.2d { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x2x4_t @llvm.aarch64.neon.ld4.v2i64.p0(ptr %A) %tmp2 = call %struct.__neon_int64x2x4_t @llvm.aarch64.neon.ld4.v2i64.p0(ptr %A)
ret %struct.__neon_int64x2x4_t %tmp2 ret %struct.__neon_int64x2x4_t %tmp2
} }
Context not available.
define %struct.__neon_int64x1x2_t @ld2_1di64(ptr %A) nounwind { define %struct.__neon_int64x1x2_t @ld2_1di64(ptr %A) nounwind {
; CHECK-LABEL: ld2_1di64
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.1d { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x1x2_t @llvm.aarch64.neon.ld2.v1i64.p0(ptr %A) %tmp2 = call %struct.__neon_int64x1x2_t @llvm.aarch64.neon.ld2.v1i64.p0(ptr %A)
ret %struct.__neon_int64x1x2_t %tmp2 ret %struct.__neon_int64x1x2_t %tmp2
} }
define %struct.__neon_int64x1x3_t @ld3_1di64(ptr %A) nounwind { define %struct.__neon_int64x1x3_t @ld3_1di64(ptr %A) nounwind {
; CHECK-LABEL: ld3_1di64
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.1d { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x1x3_t @llvm.aarch64.neon.ld3.v1i64.p0(ptr %A) %tmp2 = call %struct.__neon_int64x1x3_t @llvm.aarch64.neon.ld3.v1i64.p0(ptr %A)
ret %struct.__neon_int64x1x3_t %tmp2 ret %struct.__neon_int64x1x3_t %tmp2
} }
define %struct.__neon_int64x1x4_t @ld4_1di64(ptr %A) nounwind { define %struct.__neon_int64x1x4_t @ld4_1di64(ptr %A) nounwind {
; CHECK-LABEL: ld4_1di64
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.1d { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x1x4_t @llvm.aarch64.neon.ld4.v1i64.p0(ptr %A) %tmp2 = call %struct.__neon_int64x1x4_t @llvm.aarch64.neon.ld4.v1i64.p0(ptr %A)
ret %struct.__neon_int64x1x4_t %tmp2 ret %struct.__neon_int64x1x4_t %tmp2
} }
Context not available.
define %struct.__neon_float64x1x2_t @ld2_1df64(ptr %A) nounwind { define %struct.__neon_float64x1x2_t @ld2_1df64(ptr %A) nounwind {
; CHECK-LABEL: ld2_1df64
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.1d { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_float64x1x2_t @llvm.aarch64.neon.ld2.v1f64.p0(ptr %A) %tmp2 = call %struct.__neon_float64x1x2_t @llvm.aarch64.neon.ld2.v1f64.p0(ptr %A)
ret %struct.__neon_float64x1x2_t %tmp2 ret %struct.__neon_float64x1x2_t %tmp2
} }
define %struct.__neon_float64x1x3_t @ld3_1df64(ptr %A) nounwind { define %struct.__neon_float64x1x3_t @ld3_1df64(ptr %A) nounwind {
; CHECK-LABEL: ld3_1df64
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.1d { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_float64x1x3_t @llvm.aarch64.neon.ld3.v1f64.p0(ptr %A) %tmp2 = call %struct.__neon_float64x1x3_t @llvm.aarch64.neon.ld3.v1f64.p0(ptr %A)
ret %struct.__neon_float64x1x3_t %tmp2 ret %struct.__neon_float64x1x3_t %tmp2
} }
define %struct.__neon_float64x1x4_t @ld4_1df64(ptr %A) nounwind { define %struct.__neon_float64x1x4_t @ld4_1df64(ptr %A) nounwind {
; CHECK-LABEL: ld4_1df64
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.1d { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_float64x1x4_t @llvm.aarch64.neon.ld4.v1f64.p0(ptr %A) %tmp2 = call %struct.__neon_float64x1x4_t @llvm.aarch64.neon.ld4.v1f64.p0(ptr %A)
ret %struct.__neon_float64x1x4_t %tmp2 ret %struct.__neon_float64x1x4_t %tmp2
} }
Context not available.
define %struct.__neon_int8x16x2_t @ld2lane_16b(<16 x i8> %L1, <16 x i8> %L2, ptr %A) nounwind { define %struct.__neon_int8x16x2_t @ld2lane_16b(<16 x i8> %L1, <16 x i8> %L2, ptr %A) nounwind {
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2lane_16b
; CHECK: ld2.b { v0, v1 }[1], [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x16x2_t @llvm.aarch64.neon.ld2lane.v16i8.p0(<16 x i8> %L1, <16 x i8> %L2, i64 1, ptr %A) %tmp2 = call %struct.__neon_int8x16x2_t @llvm.aarch64.neon.ld2lane.v16i8.p0(<16 x i8> %L1, <16 x i8> %L2, i64 1, ptr %A)
ret %struct.__neon_int8x16x2_t %tmp2 ret %struct.__neon_int8x16x2_t %tmp2
} }
define %struct.__neon_int8x16x3_t @ld3lane_16b(<16 x i8> %L1, <16 x i8> %L2, <16 x i8> %L3, ptr %A) nounwind { define %struct.__neon_int8x16x3_t @ld3lane_16b(<16 x i8> %L1, <16 x i8> %L2, <16 x i8> %L3, ptr %A) nounwind {
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3lane_16b
; CHECK: ld3.b { v0, v1, v2 }[1], [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x16x3_t @llvm.aarch64.neon.ld3lane.v16i8.p0(<16 x i8> %L1, <16 x i8> %L2, <16 x i8> %L3, i64 1, ptr %A) %tmp2 = call %struct.__neon_int8x16x3_t @llvm.aarch64.neon.ld3lane.v16i8.p0(<16 x i8> %L1, <16 x i8> %L2, <16 x i8> %L3, i64 1, ptr %A)
ret %struct.__neon_int8x16x3_t %tmp2 ret %struct.__neon_int8x16x3_t %tmp2
} }
define %struct.__neon_int8x16x4_t @ld4lane_16b(<16 x i8> %L1, <16 x i8> %L2, <16 x i8> %L3, <16 x i8> %L4, ptr %A) nounwind { define %struct.__neon_int8x16x4_t @ld4lane_16b(<16 x i8> %L1, <16 x i8> %L2, <16 x i8> %L3, <16 x i8> %L4, ptr %A) nounwind {
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4lane_16b
; CHECK: ld4.b { v0, v1, v2, v3 }[1], [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x16x4_t @llvm.aarch64.neon.ld4lane.v16i8.p0(<16 x i8> %L1, <16 x i8> %L2, <16 x i8> %L3, <16 x i8> %L4, i64 1, ptr %A) %tmp2 = call %struct.__neon_int8x16x4_t @llvm.aarch64.neon.ld4lane.v16i8.p0(<16 x i8> %L1, <16 x i8> %L2, <16 x i8> %L3, <16 x i8> %L4, i64 1, ptr %A)
ret %struct.__neon_int8x16x4_t %tmp2 ret %struct.__neon_int8x16x4_t %tmp2
} }
Context not available.
define %struct.__neon_int16x8x2_t @ld2lane_8h(<8 x i16> %L1, <8 x i16> %L2, ptr %A) nounwind { define %struct.__neon_int16x8x2_t @ld2lane_8h(<8 x i16> %L1, <8 x i16> %L2, ptr %A) nounwind {
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2lane_8h
; CHECK: ld2.h { v0, v1 }[1], [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x8x2_t @llvm.aarch64.neon.ld2lane.v8i16.p0(<8 x i16> %L1, <8 x i16> %L2, i64 1, ptr %A) %tmp2 = call %struct.__neon_int16x8x2_t @llvm.aarch64.neon.ld2lane.v8i16.p0(<8 x i16> %L1, <8 x i16> %L2, i64 1, ptr %A)
ret %struct.__neon_int16x8x2_t %tmp2 ret %struct.__neon_int16x8x2_t %tmp2
} }
define %struct.__neon_int16x8x3_t @ld3lane_8h(<8 x i16> %L1, <8 x i16> %L2, <8 x i16> %L3, ptr %A) nounwind { define %struct.__neon_int16x8x3_t @ld3lane_8h(<8 x i16> %L1, <8 x i16> %L2, <8 x i16> %L3, ptr %A) nounwind {
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3lane_8h
; CHECK: ld3.h { v0, v1, v2 }[1], [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x8x3_t @llvm.aarch64.neon.ld3lane.v8i16.p0(<8 x i16> %L1, <8 x i16> %L2, <8 x i16> %L3, i64 1, ptr %A) %tmp2 = call %struct.__neon_int16x8x3_t @llvm.aarch64.neon.ld3lane.v8i16.p0(<8 x i16> %L1, <8 x i16> %L2, <8 x i16> %L3, i64 1, ptr %A)
ret %struct.__neon_int16x8x3_t %tmp2 ret %struct.__neon_int16x8x3_t %tmp2
} }
define %struct.__neon_int16x8x4_t @ld4lane_8h(<8 x i16> %L1, <8 x i16> %L2, <8 x i16> %L3, <8 x i16> %L4, ptr %A) nounwind { define %struct.__neon_int16x8x4_t @ld4lane_8h(<8 x i16> %L1, <8 x i16> %L2, <8 x i16> %L3, <8 x i16> %L4, ptr %A) nounwind {
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4lane_8h
; CHECK: ld4.h { v0, v1, v2, v3 }[1], [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x8x4_t @llvm.aarch64.neon.ld4lane.v8i16.p0(<8 x i16> %L1, <8 x i16> %L2, <8 x i16> %L3, <8 x i16> %L4, i64 1, ptr %A) %tmp2 = call %struct.__neon_int16x8x4_t @llvm.aarch64.neon.ld4lane.v8i16.p0(<8 x i16> %L1, <8 x i16> %L2, <8 x i16> %L3, <8 x i16> %L4, i64 1, ptr %A)
ret %struct.__neon_int16x8x4_t %tmp2 ret %struct.__neon_int16x8x4_t %tmp2
} }
Context not available.
define %struct.__neon_int32x4x2_t @ld2lane_4s(<4 x i32> %L1, <4 x i32> %L2, ptr %A) nounwind { define %struct.__neon_int32x4x2_t @ld2lane_4s(<4 x i32> %L1, <4 x i32> %L2, ptr %A) nounwind {
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2lane_4s
; CHECK: ld2.s { v0, v1 }[1], [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x4x2_t @llvm.aarch64.neon.ld2lane.v4i32.p0(<4 x i32> %L1, <4 x i32> %L2, i64 1, ptr %A) %tmp2 = call %struct.__neon_int32x4x2_t @llvm.aarch64.neon.ld2lane.v4i32.p0(<4 x i32> %L1, <4 x i32> %L2, i64 1, ptr %A)
ret %struct.__neon_int32x4x2_t %tmp2 ret %struct.__neon_int32x4x2_t %tmp2
} }
define %struct.__neon_int32x4x3_t @ld3lane_4s(<4 x i32> %L1, <4 x i32> %L2, <4 x i32> %L3, ptr %A) nounwind { define %struct.__neon_int32x4x3_t @ld3lane_4s(<4 x i32> %L1, <4 x i32> %L2, <4 x i32> %L3, ptr %A) nounwind {
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3lane_4s
; CHECK: ld3.s { v0, v1, v2 }[1], [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x4x3_t @llvm.aarch64.neon.ld3lane.v4i32.p0(<4 x i32> %L1, <4 x i32> %L2, <4 x i32> %L3, i64 1, ptr %A) %tmp2 = call %struct.__neon_int32x4x3_t @llvm.aarch64.neon.ld3lane.v4i32.p0(<4 x i32> %L1, <4 x i32> %L2, <4 x i32> %L3, i64 1, ptr %A)
ret %struct.__neon_int32x4x3_t %tmp2 ret %struct.__neon_int32x4x3_t %tmp2
} }
define %struct.__neon_int32x4x4_t @ld4lane_4s(<4 x i32> %L1, <4 x i32> %L2, <4 x i32> %L3, <4 x i32> %L4, ptr %A) nounwind { define %struct.__neon_int32x4x4_t @ld4lane_4s(<4 x i32> %L1, <4 x i32> %L2, <4 x i32> %L3, <4 x i32> %L4, ptr %A) nounwind {
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4lane_4s
; CHECK: ld4.s { v0, v1, v2, v3 }[1], [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x4x4_t @llvm.aarch64.neon.ld4lane.v4i32.p0(<4 x i32> %L1, <4 x i32> %L2, <4 x i32> %L3, <4 x i32> %L4, i64 1, ptr %A) %tmp2 = call %struct.__neon_int32x4x4_t @llvm.aarch64.neon.ld4lane.v4i32.p0(<4 x i32> %L1, <4 x i32> %L2, <4 x i32> %L3, <4 x i32> %L4, i64 1, ptr %A)
ret %struct.__neon_int32x4x4_t %tmp2 ret %struct.__neon_int32x4x4_t %tmp2
} }
Context not available.
define %struct.__neon_int64x2x2_t @ld2lane_2d(<2 x i64> %L1, <2 x i64> %L2, ptr %A) nounwind { define %struct.__neon_int64x2x2_t @ld2lane_2d(<2 x i64> %L1, <2 x i64> %L2, ptr %A) nounwind {
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2lane_2d
; CHECK: ld2.d { v0, v1 }[1], [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x2x2_t @llvm.aarch64.neon.ld2lane.v2i64.p0(<2 x i64> %L1, <2 x i64> %L2, i64 1, ptr %A) %tmp2 = call %struct.__neon_int64x2x2_t @llvm.aarch64.neon.ld2lane.v2i64.p0(<2 x i64> %L1, <2 x i64> %L2, i64 1, ptr %A)
ret %struct.__neon_int64x2x2_t %tmp2 ret %struct.__neon_int64x2x2_t %tmp2
} }
define %struct.__neon_int64x2x3_t @ld3lane_2d(<2 x i64> %L1, <2 x i64> %L2, <2 x i64> %L3, ptr %A) nounwind { define %struct.__neon_int64x2x3_t @ld3lane_2d(<2 x i64> %L1, <2 x i64> %L2, <2 x i64> %L3, ptr %A) nounwind {
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3lane_2d
; CHECK: ld3.d { v0, v1, v2 }[1], [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x2x3_t @llvm.aarch64.neon.ld3lane.v2i64.p0(<2 x i64> %L1, <2 x i64> %L2, <2 x i64> %L3, i64 1, ptr %A) %tmp2 = call %struct.__neon_int64x2x3_t @llvm.aarch64.neon.ld3lane.v2i64.p0(<2 x i64> %L1, <2 x i64> %L2, <2 x i64> %L3, i64 1, ptr %A)
ret %struct.__neon_int64x2x3_t %tmp2 ret %struct.__neon_int64x2x3_t %tmp2
} }
define %struct.__neon_int64x2x4_t @ld4lane_2d(<2 x i64> %L1, <2 x i64> %L2, <2 x i64> %L3, <2 x i64> %L4, ptr %A) nounwind { define %struct.__neon_int64x2x4_t @ld4lane_2d(<2 x i64> %L1, <2 x i64> %L2, <2 x i64> %L3, <2 x i64> %L4, ptr %A) nounwind {
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4lane_2d
; CHECK: ld4.d { v0, v1, v2, v3 }[1], [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x2x4_t @llvm.aarch64.neon.ld4lane.v2i64.p0(<2 x i64> %L1, <2 x i64> %L2, <2 x i64> %L3, <2 x i64> %L4, i64 1, ptr %A) %tmp2 = call %struct.__neon_int64x2x4_t @llvm.aarch64.neon.ld4lane.v2i64.p0(<2 x i64> %L1, <2 x i64> %L2, <2 x i64> %L3, <2 x i64> %L4, i64 1, ptr %A)
ret %struct.__neon_int64x2x4_t %tmp2 ret %struct.__neon_int64x2x4_t %tmp2
} }
Context not available.
declare %struct.__neon_int64x2x4_t @llvm.aarch64.neon.ld4lane.v2i64.p0(<2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, i64, ptr) nounwind readonly declare %struct.__neon_int64x2x4_t @llvm.aarch64.neon.ld4lane.v2i64.p0(<2 x i64>, <2 x i64>, <2 x i64>, <2 x i64>, i64, ptr) nounwind readonly
define <8 x i8> @ld1r_8b(ptr %bar) { define <8 x i8> @ld1r_8b(ptr %bar) {
; CHECK: ld1r_8b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.8b { v0 }, [x0]
; CHECK-NEXT: ret
%tmp1 = load i8, ptr %bar %tmp1 = load i8, ptr %bar
%tmp2 = insertelement <8 x i8> <i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef>, i8 %tmp1, i32 0 %tmp2 = insertelement <8 x i8> <i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef>, i8 %tmp1, i32 0
%tmp3 = insertelement <8 x i8> %tmp2, i8 %tmp1, i32 1 %tmp3 = insertelement <8 x i8> %tmp2, i8 %tmp1, i32 1
Context not available.
} }
define <16 x i8> @ld1r_16b(ptr %bar) { define <16 x i8> @ld1r_16b(ptr %bar) {
; CHECK: ld1r_16b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.16b { v0 }, [x0]
; CHECK-NEXT: ret
%tmp1 = load i8, ptr %bar %tmp1 = load i8, ptr %bar
%tmp2 = insertelement <16 x i8> <i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef>, i8 %tmp1, i32 0 %tmp2 = insertelement <16 x i8> <i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef, i8 undef>, i8 %tmp1, i32 0
%tmp3 = insertelement <16 x i8> %tmp2, i8 %tmp1, i32 1 %tmp3 = insertelement <16 x i8> %tmp2, i8 %tmp1, i32 1
Context not available.
} }
define <4 x i16> @ld1r_4h(ptr %bar) { define <4 x i16> @ld1r_4h(ptr %bar) {
; CHECK: ld1r_4h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.4h { v0 }, [x0]
; CHECK-NEXT: ret
%tmp1 = load i16, ptr %bar %tmp1 = load i16, ptr %bar
%tmp2 = insertelement <4 x i16> <i16 undef, i16 undef, i16 undef, i16 undef>, i16 %tmp1, i32 0 %tmp2 = insertelement <4 x i16> <i16 undef, i16 undef, i16 undef, i16 undef>, i16 %tmp1, i32 0
%tmp3 = insertelement <4 x i16> %tmp2, i16 %tmp1, i32 1 %tmp3 = insertelement <4 x i16> %tmp2, i16 %tmp1, i32 1
Context not available.
} }
define <8 x i16> @ld1r_8h(ptr %bar) { define <8 x i16> @ld1r_8h(ptr %bar) {
; CHECK: ld1r_8h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.8h { v0 }, [x0]
; CHECK-NEXT: ret
%tmp1 = load i16, ptr %bar %tmp1 = load i16, ptr %bar
%tmp2 = insertelement <8 x i16> <i16 undef, i16 undef, i16 undef, i16 undef, i16 undef, i16 undef, i16 undef, i16 undef>, i16 %tmp1, i32 0 %tmp2 = insertelement <8 x i16> <i16 undef, i16 undef, i16 undef, i16 undef, i16 undef, i16 undef, i16 undef, i16 undef>, i16 %tmp1, i32 0
%tmp3 = insertelement <8 x i16> %tmp2, i16 %tmp1, i32 1 %tmp3 = insertelement <8 x i16> %tmp2, i16 %tmp1, i32 1
Context not available.
} }
define <2 x i32> @ld1r_2s(ptr %bar) { define <2 x i32> @ld1r_2s(ptr %bar) {
; CHECK: ld1r_2s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.2s { v0 }, [x0]
; CHECK-NEXT: ret
%tmp1 = load i32, ptr %bar %tmp1 = load i32, ptr %bar
%tmp2 = insertelement <2 x i32> <i32 undef, i32 undef>, i32 %tmp1, i32 0 %tmp2 = insertelement <2 x i32> <i32 undef, i32 undef>, i32 %tmp1, i32 0
%tmp3 = insertelement <2 x i32> %tmp2, i32 %tmp1, i32 1 %tmp3 = insertelement <2 x i32> %tmp2, i32 %tmp1, i32 1
Context not available.
} }
define <4 x i32> @ld1r_4s(ptr %bar) { define <4 x i32> @ld1r_4s(ptr %bar) {
; CHECK: ld1r_4s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.4s { v0 }, [x0]
; CHECK-NEXT: ret
%tmp1 = load i32, ptr %bar %tmp1 = load i32, ptr %bar
%tmp2 = insertelement <4 x i32> <i32 undef, i32 undef, i32 undef, i32 undef>, i32 %tmp1, i32 0 %tmp2 = insertelement <4 x i32> <i32 undef, i32 undef, i32 undef, i32 undef>, i32 %tmp1, i32 0
%tmp3 = insertelement <4 x i32> %tmp2, i32 %tmp1, i32 1 %tmp3 = insertelement <4 x i32> %tmp2, i32 %tmp1, i32 1
Context not available.
} }
define <2 x i64> @ld1r_2d(ptr %bar) { define <2 x i64> @ld1r_2d(ptr %bar) {
; CHECK: ld1r_2d
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.2d { v0 }, [x0]
; CHECK-NEXT: ret
%tmp1 = load i64, ptr %bar %tmp1 = load i64, ptr %bar
%tmp2 = insertelement <2 x i64> <i64 undef, i64 undef>, i64 %tmp1, i32 0 %tmp2 = insertelement <2 x i64> <i64 undef, i64 undef>, i64 %tmp1, i32 0
%tmp3 = insertelement <2 x i64> %tmp2, i64 %tmp1, i32 1 %tmp3 = insertelement <2 x i64> %tmp2, i64 %tmp1, i32 1
Context not available.
} }
define %struct.__neon_int8x8x2_t @ld2r_8b(ptr %A) nounwind { define %struct.__neon_int8x8x2_t @ld2r_8b(ptr %A) nounwind {
; CHECK: ld2r_8b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2r.8b { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x8x2_t @llvm.aarch64.neon.ld2r.v8i8.p0(ptr %A) %tmp2 = call %struct.__neon_int8x8x2_t @llvm.aarch64.neon.ld2r.v8i8.p0(ptr %A)
ret %struct.__neon_int8x8x2_t %tmp2 ret %struct.__neon_int8x8x2_t %tmp2
} }
define %struct.__neon_int8x8x3_t @ld3r_8b(ptr %A) nounwind { define %struct.__neon_int8x8x3_t @ld3r_8b(ptr %A) nounwind {
; CHECK: ld3r_8b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3r.8b { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x8x3_t @llvm.aarch64.neon.ld3r.v8i8.p0(ptr %A) %tmp2 = call %struct.__neon_int8x8x3_t @llvm.aarch64.neon.ld3r.v8i8.p0(ptr %A)
ret %struct.__neon_int8x8x3_t %tmp2 ret %struct.__neon_int8x8x3_t %tmp2
} }
define %struct.__neon_int8x8x4_t @ld4r_8b(ptr %A) nounwind { define %struct.__neon_int8x8x4_t @ld4r_8b(ptr %A) nounwind {
; CHECK: ld4r_8b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4r.8b { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x8x4_t @llvm.aarch64.neon.ld4r.v8i8.p0(ptr %A) %tmp2 = call %struct.__neon_int8x8x4_t @llvm.aarch64.neon.ld4r.v8i8.p0(ptr %A)
ret %struct.__neon_int8x8x4_t %tmp2 ret %struct.__neon_int8x8x4_t %tmp2
} }
Context not available.
declare %struct.__neon_int8x8x4_t @llvm.aarch64.neon.ld4r.v8i8.p0(ptr) nounwind readonly declare %struct.__neon_int8x8x4_t @llvm.aarch64.neon.ld4r.v8i8.p0(ptr) nounwind readonly
define %struct.__neon_int8x16x2_t @ld2r_16b(ptr %A) nounwind { define %struct.__neon_int8x16x2_t @ld2r_16b(ptr %A) nounwind {
; CHECK: ld2r_16b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2r.16b { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x16x2_t @llvm.aarch64.neon.ld2r.v16i8.p0(ptr %A) %tmp2 = call %struct.__neon_int8x16x2_t @llvm.aarch64.neon.ld2r.v16i8.p0(ptr %A)
ret %struct.__neon_int8x16x2_t %tmp2 ret %struct.__neon_int8x16x2_t %tmp2
} }
define %struct.__neon_int8x16x3_t @ld3r_16b(ptr %A) nounwind { define %struct.__neon_int8x16x3_t @ld3r_16b(ptr %A) nounwind {
; CHECK: ld3r_16b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3r.16b { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x16x3_t @llvm.aarch64.neon.ld3r.v16i8.p0(ptr %A) %tmp2 = call %struct.__neon_int8x16x3_t @llvm.aarch64.neon.ld3r.v16i8.p0(ptr %A)
ret %struct.__neon_int8x16x3_t %tmp2 ret %struct.__neon_int8x16x3_t %tmp2
} }
define %struct.__neon_int8x16x4_t @ld4r_16b(ptr %A) nounwind { define %struct.__neon_int8x16x4_t @ld4r_16b(ptr %A) nounwind {
; CHECK: ld4r_16b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4r.16b { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int8x16x4_t @llvm.aarch64.neon.ld4r.v16i8.p0(ptr %A) %tmp2 = call %struct.__neon_int8x16x4_t @llvm.aarch64.neon.ld4r.v16i8.p0(ptr %A)
ret %struct.__neon_int8x16x4_t %tmp2 ret %struct.__neon_int8x16x4_t %tmp2
} }
Context not available.
declare %struct.__neon_int8x16x4_t @llvm.aarch64.neon.ld4r.v16i8.p0(ptr) nounwind readonly declare %struct.__neon_int8x16x4_t @llvm.aarch64.neon.ld4r.v16i8.p0(ptr) nounwind readonly
define %struct.__neon_int16x4x2_t @ld2r_4h(ptr %A) nounwind { define %struct.__neon_int16x4x2_t @ld2r_4h(ptr %A) nounwind {
; CHECK: ld2r_4h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2r.4h { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x4x2_t @llvm.aarch64.neon.ld2r.v4i16.p0(ptr %A) %tmp2 = call %struct.__neon_int16x4x2_t @llvm.aarch64.neon.ld2r.v4i16.p0(ptr %A)
ret %struct.__neon_int16x4x2_t %tmp2 ret %struct.__neon_int16x4x2_t %tmp2
} }
define %struct.__neon_int16x4x3_t @ld3r_4h(ptr %A) nounwind { define %struct.__neon_int16x4x3_t @ld3r_4h(ptr %A) nounwind {
; CHECK: ld3r_4h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3r.4h { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x4x3_t @llvm.aarch64.neon.ld3r.v4i16.p0(ptr %A) %tmp2 = call %struct.__neon_int16x4x3_t @llvm.aarch64.neon.ld3r.v4i16.p0(ptr %A)
ret %struct.__neon_int16x4x3_t %tmp2 ret %struct.__neon_int16x4x3_t %tmp2
} }
define %struct.__neon_int16x4x4_t @ld4r_4h(ptr %A) nounwind { define %struct.__neon_int16x4x4_t @ld4r_4h(ptr %A) nounwind {
; CHECK: ld4r_4h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4r.4h { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x4x4_t @llvm.aarch64.neon.ld4r.v4i16.p0(ptr %A) %tmp2 = call %struct.__neon_int16x4x4_t @llvm.aarch64.neon.ld4r.v4i16.p0(ptr %A)
ret %struct.__neon_int16x4x4_t %tmp2 ret %struct.__neon_int16x4x4_t %tmp2
} }
Context not available.
declare %struct.__neon_int16x4x4_t @llvm.aarch64.neon.ld4r.v4i16.p0(ptr) nounwind readonly declare %struct.__neon_int16x4x4_t @llvm.aarch64.neon.ld4r.v4i16.p0(ptr) nounwind readonly
define %struct.__neon_int16x8x2_t @ld2r_8h(ptr %A) nounwind { define %struct.__neon_int16x8x2_t @ld2r_8h(ptr %A) nounwind {
; CHECK: ld2r_8h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2r.8h { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x8x2_t @llvm.aarch64.neon.ld2r.v8i16.p0(ptr %A) %tmp2 = call %struct.__neon_int16x8x2_t @llvm.aarch64.neon.ld2r.v8i16.p0(ptr %A)
ret %struct.__neon_int16x8x2_t %tmp2 ret %struct.__neon_int16x8x2_t %tmp2
} }
define %struct.__neon_int16x8x3_t @ld3r_8h(ptr %A) nounwind { define %struct.__neon_int16x8x3_t @ld3r_8h(ptr %A) nounwind {
; CHECK: ld3r_8h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3r.8h { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x8x3_t @llvm.aarch64.neon.ld3r.v8i16.p0(ptr %A) %tmp2 = call %struct.__neon_int16x8x3_t @llvm.aarch64.neon.ld3r.v8i16.p0(ptr %A)
ret %struct.__neon_int16x8x3_t %tmp2 ret %struct.__neon_int16x8x3_t %tmp2
} }
define %struct.__neon_int16x8x4_t @ld4r_8h(ptr %A) nounwind { define %struct.__neon_int16x8x4_t @ld4r_8h(ptr %A) nounwind {
; CHECK: ld4r_8h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4r.8h { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int16x8x4_t @llvm.aarch64.neon.ld4r.v8i16.p0(ptr %A) %tmp2 = call %struct.__neon_int16x8x4_t @llvm.aarch64.neon.ld4r.v8i16.p0(ptr %A)
ret %struct.__neon_int16x8x4_t %tmp2 ret %struct.__neon_int16x8x4_t %tmp2
} }
Context not available.
declare %struct.__neon_int16x8x4_t @llvm.aarch64.neon.ld4r.v8i16.p0(ptr) nounwind readonly declare %struct.__neon_int16x8x4_t @llvm.aarch64.neon.ld4r.v8i16.p0(ptr) nounwind readonly
define %struct.__neon_int32x2x2_t @ld2r_2s(ptr %A) nounwind { define %struct.__neon_int32x2x2_t @ld2r_2s(ptr %A) nounwind {
; CHECK: ld2r_2s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2r.2s { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x2x2_t @llvm.aarch64.neon.ld2r.v2i32.p0(ptr %A) %tmp2 = call %struct.__neon_int32x2x2_t @llvm.aarch64.neon.ld2r.v2i32.p0(ptr %A)
ret %struct.__neon_int32x2x2_t %tmp2 ret %struct.__neon_int32x2x2_t %tmp2
} }
define %struct.__neon_int32x2x3_t @ld3r_2s(ptr %A) nounwind { define %struct.__neon_int32x2x3_t @ld3r_2s(ptr %A) nounwind {
; CHECK: ld3r_2s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3r.2s { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x2x3_t @llvm.aarch64.neon.ld3r.v2i32.p0(ptr %A) %tmp2 = call %struct.__neon_int32x2x3_t @llvm.aarch64.neon.ld3r.v2i32.p0(ptr %A)
ret %struct.__neon_int32x2x3_t %tmp2 ret %struct.__neon_int32x2x3_t %tmp2
} }
define %struct.__neon_int32x2x4_t @ld4r_2s(ptr %A) nounwind { define %struct.__neon_int32x2x4_t @ld4r_2s(ptr %A) nounwind {
; CHECK: ld4r_2s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4r.2s { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x2x4_t @llvm.aarch64.neon.ld4r.v2i32.p0(ptr %A) %tmp2 = call %struct.__neon_int32x2x4_t @llvm.aarch64.neon.ld4r.v2i32.p0(ptr %A)
ret %struct.__neon_int32x2x4_t %tmp2 ret %struct.__neon_int32x2x4_t %tmp2
} }
Context not available.
declare %struct.__neon_int32x2x4_t @llvm.aarch64.neon.ld4r.v2i32.p0(ptr) nounwind readonly declare %struct.__neon_int32x2x4_t @llvm.aarch64.neon.ld4r.v2i32.p0(ptr) nounwind readonly
define %struct.__neon_int32x4x2_t @ld2r_4s(ptr %A) nounwind { define %struct.__neon_int32x4x2_t @ld2r_4s(ptr %A) nounwind {
; CHECK: ld2r_4s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2r.4s { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x4x2_t @llvm.aarch64.neon.ld2r.v4i32.p0(ptr %A) %tmp2 = call %struct.__neon_int32x4x2_t @llvm.aarch64.neon.ld2r.v4i32.p0(ptr %A)
ret %struct.__neon_int32x4x2_t %tmp2 ret %struct.__neon_int32x4x2_t %tmp2
} }
define %struct.__neon_int32x4x3_t @ld3r_4s(ptr %A) nounwind { define %struct.__neon_int32x4x3_t @ld3r_4s(ptr %A) nounwind {
; CHECK: ld3r_4s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3r.4s { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x4x3_t @llvm.aarch64.neon.ld3r.v4i32.p0(ptr %A) %tmp2 = call %struct.__neon_int32x4x3_t @llvm.aarch64.neon.ld3r.v4i32.p0(ptr %A)
ret %struct.__neon_int32x4x3_t %tmp2 ret %struct.__neon_int32x4x3_t %tmp2
} }
define %struct.__neon_int32x4x4_t @ld4r_4s(ptr %A) nounwind { define %struct.__neon_int32x4x4_t @ld4r_4s(ptr %A) nounwind {
; CHECK: ld4r_4s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4r.4s { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int32x4x4_t @llvm.aarch64.neon.ld4r.v4i32.p0(ptr %A) %tmp2 = call %struct.__neon_int32x4x4_t @llvm.aarch64.neon.ld4r.v4i32.p0(ptr %A)
ret %struct.__neon_int32x4x4_t %tmp2 ret %struct.__neon_int32x4x4_t %tmp2
} }
Context not available.
declare %struct.__neon_int32x4x4_t @llvm.aarch64.neon.ld4r.v4i32.p0(ptr) nounwind readonly declare %struct.__neon_int32x4x4_t @llvm.aarch64.neon.ld4r.v4i32.p0(ptr) nounwind readonly
define %struct.__neon_int64x1x2_t @ld2r_1d(ptr %A) nounwind { define %struct.__neon_int64x1x2_t @ld2r_1d(ptr %A) nounwind {
; CHECK: ld2r_1d
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2r.1d { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x1x2_t @llvm.aarch64.neon.ld2r.v1i64.p0(ptr %A) %tmp2 = call %struct.__neon_int64x1x2_t @llvm.aarch64.neon.ld2r.v1i64.p0(ptr %A)
ret %struct.__neon_int64x1x2_t %tmp2 ret %struct.__neon_int64x1x2_t %tmp2
} }
define %struct.__neon_int64x1x3_t @ld3r_1d(ptr %A) nounwind { define %struct.__neon_int64x1x3_t @ld3r_1d(ptr %A) nounwind {
; CHECK: ld3r_1d
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3r.1d { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x1x3_t @llvm.aarch64.neon.ld3r.v1i64.p0(ptr %A) %tmp2 = call %struct.__neon_int64x1x3_t @llvm.aarch64.neon.ld3r.v1i64.p0(ptr %A)
ret %struct.__neon_int64x1x3_t %tmp2 ret %struct.__neon_int64x1x3_t %tmp2
} }
define %struct.__neon_int64x1x4_t @ld4r_1d(ptr %A) nounwind { define %struct.__neon_int64x1x4_t @ld4r_1d(ptr %A) nounwind {
; CHECK: ld4r_1d
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4r.1d { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x1x4_t @llvm.aarch64.neon.ld4r.v1i64.p0(ptr %A) %tmp2 = call %struct.__neon_int64x1x4_t @llvm.aarch64.neon.ld4r.v1i64.p0(ptr %A)
ret %struct.__neon_int64x1x4_t %tmp2 ret %struct.__neon_int64x1x4_t %tmp2
} }
Context not available.
declare %struct.__neon_int64x1x4_t @llvm.aarch64.neon.ld4r.v1i64.p0(ptr) nounwind readonly declare %struct.__neon_int64x1x4_t @llvm.aarch64.neon.ld4r.v1i64.p0(ptr) nounwind readonly
define %struct.__neon_int64x2x2_t @ld2r_2d(ptr %A) nounwind { define %struct.__neon_int64x2x2_t @ld2r_2d(ptr %A) nounwind {
; CHECK: ld2r_2d
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld2r.2d { v0, v1 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x2x2_t @llvm.aarch64.neon.ld2r.v2i64.p0(ptr %A) %tmp2 = call %struct.__neon_int64x2x2_t @llvm.aarch64.neon.ld2r.v2i64.p0(ptr %A)
ret %struct.__neon_int64x2x2_t %tmp2 ret %struct.__neon_int64x2x2_t %tmp2
} }
define %struct.__neon_int64x2x3_t @ld3r_2d(ptr %A) nounwind { define %struct.__neon_int64x2x3_t @ld3r_2d(ptr %A) nounwind {
; CHECK: ld3r_2d
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld3r.2d { v0, v1, v2 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x2x3_t @llvm.aarch64.neon.ld3r.v2i64.p0(ptr %A) %tmp2 = call %struct.__neon_int64x2x3_t @llvm.aarch64.neon.ld3r.v2i64.p0(ptr %A)
ret %struct.__neon_int64x2x3_t %tmp2 ret %struct.__neon_int64x2x3_t %tmp2
} }
define %struct.__neon_int64x2x4_t @ld4r_2d(ptr %A) nounwind { define %struct.__neon_int64x2x4_t @ld4r_2d(ptr %A) nounwind {
; CHECK: ld4r_2d
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld4r.2d { v0, v1, v2, v3 }, [x0]
; CHECK-NEXT: ret
%tmp2 = call %struct.__neon_int64x2x4_t @llvm.aarch64.neon.ld4r.v2i64.p0(ptr %A) %tmp2 = call %struct.__neon_int64x2x4_t @llvm.aarch64.neon.ld4r.v2i64.p0(ptr %A)
ret %struct.__neon_int64x2x4_t %tmp2 ret %struct.__neon_int64x2x4_t %tmp2
} }
Context not available.
declare %struct.__neon_int64x2x4_t @llvm.aarch64.neon.ld4r.v2i64.p0(ptr) nounwind readonly declare %struct.__neon_int64x2x4_t @llvm.aarch64.neon.ld4r.v2i64.p0(ptr) nounwind readonly
define <16 x i8> @ld1_16b(<16 x i8> %V, ptr %bar) { define <16 x i8> @ld1_16b(<16 x i8> %V, ptr %bar) {
; CHECK-LABEL: ld1_16b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.b { v0 }[0], [x0]
; CHECK-NEXT: ret
%tmp1 = load i8, ptr %bar %tmp1 = load i8, ptr %bar
%tmp2 = insertelement <16 x i8> %V, i8 %tmp1, i32 0 %tmp2 = insertelement <16 x i8> %V, i8 %tmp1, i32 0
ret <16 x i8> %tmp2 ret <16 x i8> %tmp2
} }
define <8 x i16> @ld1_8h(<8 x i16> %V, ptr %bar) { define <8 x i16> @ld1_8h(<8 x i16> %V, ptr %bar) {
; CHECK-LABEL: ld1_8h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.h { v0 }[0], [x0]
; CHECK-NEXT: ret
%tmp1 = load i16, ptr %bar %tmp1 = load i16, ptr %bar
%tmp2 = insertelement <8 x i16> %V, i16 %tmp1, i32 0 %tmp2 = insertelement <8 x i16> %V, i16 %tmp1, i32 0
ret <8 x i16> %tmp2 ret <8 x i16> %tmp2
} }
define <4 x i32> @ld1_4s(<4 x i32> %V, ptr %bar) { define <4 x i32> @ld1_4s(<4 x i32> %V, ptr %bar) {
; CHECK-LABEL: ld1_4s
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.s { v0 }[0], [x0]
; CHECK-NEXT: ret
%tmp1 = load i32, ptr %bar %tmp1 = load i32, ptr %bar
%tmp2 = insertelement <4 x i32> %V, i32 %tmp1, i32 0 %tmp2 = insertelement <4 x i32> %V, i32 %tmp1, i32 0
ret <4 x i32> %tmp2 ret <4 x i32> %tmp2
} }
define <4 x float> @ld1_4s_float(<4 x float> %V, ptr %bar) { define <4 x float> @ld1_4s_float(<4 x float> %V, ptr %bar) {
; CHECK-LABEL: ld1_4s_float:
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.s { v0 }[0], [x0]
; CHECK-NEXT: ret
%tmp1 = load float, ptr %bar %tmp1 = load float, ptr %bar
%tmp2 = insertelement <4 x float> %V, float %tmp1, i32 0 %tmp2 = insertelement <4 x float> %V, float %tmp1, i32 0
ret <4 x float> %tmp2 ret <4 x float> %tmp2
} }
define <2 x i64> @ld1_2d(<2 x i64> %V, ptr %bar) { define <2 x i64> @ld1_2d(<2 x i64> %V, ptr %bar) {
; CHECK-LABEL: ld1_2d
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.d { v0 }[0], [x0]
; CHECK-NEXT: ret
%tmp1 = load i64, ptr %bar %tmp1 = load i64, ptr %bar
%tmp2 = insertelement <2 x i64> %V, i64 %tmp1, i32 0 %tmp2 = insertelement <2 x i64> %V, i64 %tmp1, i32 0
ret <2 x i64> %tmp2 ret <2 x i64> %tmp2
} }
define <2 x double> @ld1_2d_double(<2 x double> %V, ptr %bar) { define <2 x double> @ld1_2d_double(<2 x double> %V, ptr %bar) {
; CHECK-LABEL: ld1_2d_double:
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.d { v0 }[0], [x0]
; CHECK-NEXT: ret
%tmp1 = load double, ptr %bar %tmp1 = load double, ptr %bar
%tmp2 = insertelement <2 x double> %V, double %tmp1, i32 0 %tmp2 = insertelement <2 x double> %V, double %tmp1, i32 0
ret <2 x double> %tmp2 ret <2 x double> %tmp2
} }
define <1 x i64> @ld1_1d(ptr %p) { define <1 x i64> @ld1_1d(ptr %p) {
; CHECK-LABEL: ld1_1d
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ldr [[REG:d[0-9]+]], [x0]
; CHECK-NEXT: ret
%tmp = load <1 x i64>, ptr %p, align 8 %tmp = load <1 x i64>, ptr %p, align 8
ret <1 x i64> %tmp ret <1 x i64> %tmp
} }
define <8 x i8> @ld1_8b(<8 x i8> %V, ptr %bar) { define <8 x i8> @ld1_8b(<8 x i8> %V, ptr %bar) {
; CHECK-LABEL: ld1_8b
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.b { v0 }[0], [x0]
; CHECK-NEXT: ret
%tmp1 = load i8, ptr %bar %tmp1 = load i8, ptr %bar
%tmp2 = insertelement <8 x i8> %V, i8 %tmp1, i32 0 %tmp2 = insertelement <8 x i8> %V, i8 %tmp1, i32 0
ret <8 x i8> %tmp2 ret <8 x i8> %tmp2
} }
define <4 x i16> @ld1_4h(<4 x i16> %V, ptr %bar) { define <4 x i16> @ld1_4h(<4 x i16> %V, ptr %bar) {
; CHECK-LABEL: ld1_4h
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.h { v0 }[0], [x0]
; CHECK-NEXT: ret
%tmp1 = load i16, ptr %bar %tmp1 = load i16, ptr %bar
%tmp2 = insertelement <4 x i16> %V, i16 %tmp1, i32 0 %tmp2 = insertelement <4 x i16> %V, i16 %tmp1, i32 0
ret <4 x i16> %tmp2 ret <4 x i16> %tmp2
} }
define <2 x i32> @ld1_2s(<2 x i32> %V, ptr %bar) { define <2 x i32> @ld1_2s(<2 x i32> %V, ptr %bar) {
; CHECK-LABEL: ld1_2s:
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.s { v0 }[0], [x0]
; CHECK-NEXT: ret
%tmp1 = load i32, ptr %bar %tmp1 = load i32, ptr %bar
%tmp2 = insertelement <2 x i32> %V, i32 %tmp1, i32 0 %tmp2 = insertelement <2 x i32> %V, i32 %tmp1, i32 0
ret <2 x i32> %tmp2 ret <2 x i32> %tmp2
} }
define <2 x float> @ld1_2s_float(<2 x float> %V, ptr %bar) { define <2 x float> @ld1_2s_float(<2 x float> %V, ptr %bar) {
; CHECK-LABEL: ld1_2s_float:
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1.s { v0 }[0], [x0]
; CHECK-NEXT: ret
%tmp1 = load float, ptr %bar %tmp1 = load float, ptr %bar
%tmp2 = insertelement <2 x float> %V, float %tmp1, i32 0 %tmp2 = insertelement <2 x float> %V, float %tmp1, i32 0
ret <2 x float> %tmp2 ret <2 x float> %tmp2
Context not available.
; Add rdar://13098923 test case: vld1_dup_u32 doesn't generate ld1r.2s ; Add rdar://13098923 test case: vld1_dup_u32 doesn't generate ld1r.2s
define void @ld1r_2s_from_dup(ptr nocapture %a, ptr nocapture %b, ptr nocapture %diff) nounwind ssp { define void @ld1r_2s_from_dup(ptr nocapture %a, ptr nocapture %b, ptr nocapture %diff) nounwind ssp {
entry: entry:
; CHECK: ld1r_2s_from_dup
; CHECK: ld1r.2s { [[ARG2:v[0-9]+]] }, [x1]
; CHECK-NEXT: ld1r.2s { [[ARG1:v[0-9]+]] }, [x0]
; CHECK-NEXT: usubl.8h v[[RESREGNUM:[0-9]+]], [[ARG1]], [[ARG2]]
; CHECK-NEXT: str d[[RESREGNUM]], [x2]
; CHECK-NEXT: ret
%tmp1 = load i32, ptr %a, align 4 %tmp1 = load i32, ptr %a, align 4
%tmp2 = insertelement <2 x i32> undef, i32 %tmp1, i32 0 %tmp2 = insertelement <2 x i32> undef, i32 %tmp1, i32 0
%lane = shufflevector <2 x i32> %tmp2, <2 x i32> undef, <2 x i32> zeroinitializer %lane = shufflevector <2 x i32> %tmp2, <2 x i32> undef, <2 x i32> zeroinitializer
Context not available.
; Tests for rdar://11947069: vld1_dup_* and vld1q_dup_* code gen is suboptimal ; Tests for rdar://11947069: vld1_dup_* and vld1q_dup_* code gen is suboptimal
define <4 x float> @ld1r_4s_float(ptr nocapture %x) { define <4 x float> @ld1r_4s_float(ptr nocapture %x) {
entry: entry:
; CHECK-LABEL: ld1r_4s_float
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.4s { v0 }, [x0]
; CHECK-NEXT: ret
%tmp = load float, ptr %x, align 4 %tmp = load float, ptr %x, align 4
%tmp1 = insertelement <4 x float> undef, float %tmp, i32 0 %tmp1 = insertelement <4 x float> undef, float %tmp, i32 0
%tmp2 = insertelement <4 x float> %tmp1, float %tmp, i32 1 %tmp2 = insertelement <4 x float> %tmp1, float %tmp, i32 1
Context not available.
define <2 x float> @ld1r_2s_float(ptr nocapture %x) { define <2 x float> @ld1r_2s_float(ptr nocapture %x) {
entry: entry:
; CHECK-LABEL: ld1r_2s_float
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.2s { v0 }, [x0]
; CHECK-NEXT: ret
%tmp = load float, ptr %x, align 4 %tmp = load float, ptr %x, align 4
%tmp1 = insertelement <2 x float> undef, float %tmp, i32 0 %tmp1 = insertelement <2 x float> undef, float %tmp, i32 0
%tmp2 = insertelement <2 x float> %tmp1, float %tmp, i32 1 %tmp2 = insertelement <2 x float> %tmp1, float %tmp, i32 1
Context not available.
define <2 x double> @ld1r_2d_double(ptr nocapture %x) { define <2 x double> @ld1r_2d_double(ptr nocapture %x) {
entry: entry:
; CHECK-LABEL: ld1r_2d_double
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.2d { v0 }, [x0]
; CHECK-NEXT: ret
%tmp = load double, ptr %x, align 4 %tmp = load double, ptr %x, align 4
%tmp1 = insertelement <2 x double> undef, double %tmp, i32 0 %tmp1 = insertelement <2 x double> undef, double %tmp, i32 0
%tmp2 = insertelement <2 x double> %tmp1, double %tmp, i32 1 %tmp2 = insertelement <2 x double> %tmp1, double %tmp, i32 1
Context not available.
define <1 x double> @ld1r_1d_double(ptr nocapture %x) { define <1 x double> @ld1r_1d_double(ptr nocapture %x) {
entry: entry:
; CHECK-LABEL: ld1r_1d_double
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ldr d0, [x0]
; CHECK-NEXT: ret
%tmp = load double, ptr %x, align 4 %tmp = load double, ptr %x, align 4
%tmp1 = insertelement <1 x double> undef, double %tmp, i32 0 %tmp1 = insertelement <1 x double> undef, double %tmp, i32 0
ret <1 x double> %tmp1 ret <1 x double> %tmp1
Context not available.
define <4 x float> @ld1r_4s_float_shuff(ptr nocapture %x) { define <4 x float> @ld1r_4s_float_shuff(ptr nocapture %x) {
entry: entry:
; CHECK-LABEL: ld1r_4s_float_shuff
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.4s { v0 }, [x0]
; CHECK-NEXT: ret
%tmp = load float, ptr %x, align 4 %tmp = load float, ptr %x, align 4
%tmp1 = insertelement <4 x float> undef, float %tmp, i32 0 %tmp1 = insertelement <4 x float> undef, float %tmp, i32 0
%lane = shufflevector <4 x float> %tmp1, <4 x float> undef, <4 x i32> zeroinitializer %lane = shufflevector <4 x float> %tmp1, <4 x float> undef, <4 x i32> zeroinitializer
Context not available.
define <2 x float> @ld1r_2s_float_shuff(ptr nocapture %x) { define <2 x float> @ld1r_2s_float_shuff(ptr nocapture %x) {
entry: entry:
; CHECK-LABEL: ld1r_2s_float_shuff
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.2s { v0 }, [x0]
; CHECK-NEXT: ret
%tmp = load float, ptr %x, align 4 %tmp = load float, ptr %x, align 4
%tmp1 = insertelement <2 x float> undef, float %tmp, i32 0 %tmp1 = insertelement <2 x float> undef, float %tmp, i32 0
%lane = shufflevector <2 x float> %tmp1, <2 x float> undef, <2 x i32> zeroinitializer %lane = shufflevector <2 x float> %tmp1, <2 x float> undef, <2 x i32> zeroinitializer
Context not available.
define <2 x double> @ld1r_2d_double_shuff(ptr nocapture %x) { define <2 x double> @ld1r_2d_double_shuff(ptr nocapture %x) {
entry: entry:
; CHECK-LABEL: ld1r_2d_double_shuff
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ld1r.2d { v0 }, [x0]
; CHECK-NEXT: ret
%tmp = load double, ptr %x, align 4 %tmp = load double, ptr %x, align 4
%tmp1 = insertelement <2 x double> undef, double %tmp, i32 0 %tmp1 = insertelement <2 x double> undef, double %tmp, i32 0
%lane = shufflevector <2 x double> %tmp1, <2 x double> undef, <2 x i32> zeroinitializer %lane = shufflevector <2 x double> %tmp1, <2 x double> undef, <2 x i32> zeroinitializer
Context not available.
define <1 x double> @ld1r_1d_double_shuff(ptr nocapture %x) { define <1 x double> @ld1r_1d_double_shuff(ptr nocapture %x) {
entry: entry:
; CHECK-LABEL: ld1r_1d_double_shuff
; Make sure we are using the operands defined by the ABI ; Make sure we are using the operands defined by the ABI
; CHECK: ldr d0, [x0]
; CHECK-NEXT: ret
%tmp = load double, ptr %x, align 4 %tmp = load double, ptr %x, align 4
%tmp1 = insertelement <1 x double> undef, double %tmp, i32 0 %tmp1 = insertelement <1 x double> undef, double %tmp, i32 0
%lane = shufflevector <1 x double> %tmp1, <1 x double> undef, <1 x i32> zeroinitializer %lane = shufflevector <1 x double> %tmp1, <1 x double> undef, <1 x i32> zeroinitializer
Context not available.
declare %struct.__neon_float64x1x2_t @llvm.aarch64.neon.ld1x2.v1f64.p0(ptr) nounwind readonly declare %struct.__neon_float64x1x2_t @llvm.aarch64.neon.ld1x2.v1f64.p0(ptr) nounwind readonly
define %struct.__neon_int8x8x2_t @ld1_x2_v8i8(ptr %addr) { define %struct.__neon_int8x8x2_t @ld1_x2_v8i8(ptr %addr) {
; CHECK-LABEL: ld1_x2_v8i8:
; CHECK: ld1.8b { {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int8x8x2_t @llvm.aarch64.neon.ld1x2.v8i8.p0(ptr %addr) %val = call %struct.__neon_int8x8x2_t @llvm.aarch64.neon.ld1x2.v8i8.p0(ptr %addr)
ret %struct.__neon_int8x8x2_t %val ret %struct.__neon_int8x8x2_t %val
} }
define %struct.__neon_int16x4x2_t @ld1_x2_v4i16(ptr %addr) { define %struct.__neon_int16x4x2_t @ld1_x2_v4i16(ptr %addr) {
; CHECK-LABEL: ld1_x2_v4i16:
; CHECK: ld1.4h { {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int16x4x2_t @llvm.aarch64.neon.ld1x2.v4i16.p0(ptr %addr) %val = call %struct.__neon_int16x4x2_t @llvm.aarch64.neon.ld1x2.v4i16.p0(ptr %addr)
ret %struct.__neon_int16x4x2_t %val ret %struct.__neon_int16x4x2_t %val
} }
define %struct.__neon_int32x2x2_t @ld1_x2_v2i32(ptr %addr) { define %struct.__neon_int32x2x2_t @ld1_x2_v2i32(ptr %addr) {
; CHECK-LABEL: ld1_x2_v2i32:
; CHECK: ld1.2s { {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int32x2x2_t @llvm.aarch64.neon.ld1x2.v2i32.p0(ptr %addr) %val = call %struct.__neon_int32x2x2_t @llvm.aarch64.neon.ld1x2.v2i32.p0(ptr %addr)
ret %struct.__neon_int32x2x2_t %val ret %struct.__neon_int32x2x2_t %val
} }
define %struct.__neon_float32x2x2_t @ld1_x2_v2f32(ptr %addr) { define %struct.__neon_float32x2x2_t @ld1_x2_v2f32(ptr %addr) {
; CHECK-LABEL: ld1_x2_v2f32:
; CHECK: ld1.2s { {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_float32x2x2_t @llvm.aarch64.neon.ld1x2.v2f32.p0(ptr %addr) %val = call %struct.__neon_float32x2x2_t @llvm.aarch64.neon.ld1x2.v2f32.p0(ptr %addr)
ret %struct.__neon_float32x2x2_t %val ret %struct.__neon_float32x2x2_t %val
} }
define %struct.__neon_int64x1x2_t @ld1_x2_v1i64(ptr %addr) { define %struct.__neon_int64x1x2_t @ld1_x2_v1i64(ptr %addr) {
; CHECK-LABEL: ld1_x2_v1i64:
; CHECK: ld1.1d { {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int64x1x2_t @llvm.aarch64.neon.ld1x2.v1i64.p0(ptr %addr) %val = call %struct.__neon_int64x1x2_t @llvm.aarch64.neon.ld1x2.v1i64.p0(ptr %addr)
ret %struct.__neon_int64x1x2_t %val ret %struct.__neon_int64x1x2_t %val
} }
define %struct.__neon_float64x1x2_t @ld1_x2_v1f64(ptr %addr) { define %struct.__neon_float64x1x2_t @ld1_x2_v1f64(ptr %addr) {
; CHECK-LABEL: ld1_x2_v1f64:
; CHECK: ld1.1d { {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_float64x1x2_t @llvm.aarch64.neon.ld1x2.v1f64.p0(ptr %addr) %val = call %struct.__neon_float64x1x2_t @llvm.aarch64.neon.ld1x2.v1f64.p0(ptr %addr)
ret %struct.__neon_float64x1x2_t %val ret %struct.__neon_float64x1x2_t %val
} }
Context not available.
declare %struct.__neon_float64x2x2_t @llvm.aarch64.neon.ld1x2.v2f64.p0(ptr) nounwind readonly declare %struct.__neon_float64x2x2_t @llvm.aarch64.neon.ld1x2.v2f64.p0(ptr) nounwind readonly
define %struct.__neon_int8x16x2_t @ld1_x2_v16i8(ptr %addr) { define %struct.__neon_int8x16x2_t @ld1_x2_v16i8(ptr %addr) {
; CHECK-LABEL: ld1_x2_v16i8:
; CHECK: ld1.16b { {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int8x16x2_t @llvm.aarch64.neon.ld1x2.v16i8.p0(ptr %addr) %val = call %struct.__neon_int8x16x2_t @llvm.aarch64.neon.ld1x2.v16i8.p0(ptr %addr)
ret %struct.__neon_int8x16x2_t %val ret %struct.__neon_int8x16x2_t %val
} }
define %struct.__neon_int16x8x2_t @ld1_x2_v8i16(ptr %addr) { define %struct.__neon_int16x8x2_t @ld1_x2_v8i16(ptr %addr) {
; CHECK-LABEL: ld1_x2_v8i16:
; CHECK: ld1.8h { {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int16x8x2_t @llvm.aarch64.neon.ld1x2.v8i16.p0(ptr %addr) %val = call %struct.__neon_int16x8x2_t @llvm.aarch64.neon.ld1x2.v8i16.p0(ptr %addr)
ret %struct.__neon_int16x8x2_t %val ret %struct.__neon_int16x8x2_t %val
} }
define %struct.__neon_int32x4x2_t @ld1_x2_v4i32(ptr %addr) { define %struct.__neon_int32x4x2_t @ld1_x2_v4i32(ptr %addr) {
; CHECK-LABEL: ld1_x2_v4i32:
; CHECK: ld1.4s { {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int32x4x2_t @llvm.aarch64.neon.ld1x2.v4i32.p0(ptr %addr) %val = call %struct.__neon_int32x4x2_t @llvm.aarch64.neon.ld1x2.v4i32.p0(ptr %addr)
ret %struct.__neon_int32x4x2_t %val ret %struct.__neon_int32x4x2_t %val
} }
define %struct.__neon_float32x4x2_t @ld1_x2_v4f32(ptr %addr) { define %struct.__neon_float32x4x2_t @ld1_x2_v4f32(ptr %addr) {
; CHECK-LABEL: ld1_x2_v4f32:
; CHECK: ld1.4s { {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_float32x4x2_t @llvm.aarch64.neon.ld1x2.v4f32.p0(ptr %addr) %val = call %struct.__neon_float32x4x2_t @llvm.aarch64.neon.ld1x2.v4f32.p0(ptr %addr)
ret %struct.__neon_float32x4x2_t %val ret %struct.__neon_float32x4x2_t %val
} }
define %struct.__neon_int64x2x2_t @ld1_x2_v2i64(ptr %addr) { define %struct.__neon_int64x2x2_t @ld1_x2_v2i64(ptr %addr) {
; CHECK-LABEL: ld1_x2_v2i64:
; CHECK: ld1.2d { {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int64x2x2_t @llvm.aarch64.neon.ld1x2.v2i64.p0(ptr %addr) %val = call %struct.__neon_int64x2x2_t @llvm.aarch64.neon.ld1x2.v2i64.p0(ptr %addr)
ret %struct.__neon_int64x2x2_t %val ret %struct.__neon_int64x2x2_t %val
} }
define %struct.__neon_float64x2x2_t @ld1_x2_v2f64(ptr %addr) { define %struct.__neon_float64x2x2_t @ld1_x2_v2f64(ptr %addr) {
; CHECK-LABEL: ld1_x2_v2f64:
; CHECK: ld1.2d { {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_float64x2x2_t @llvm.aarch64.neon.ld1x2.v2f64.p0(ptr %addr) %val = call %struct.__neon_float64x2x2_t @llvm.aarch64.neon.ld1x2.v2f64.p0(ptr %addr)
ret %struct.__neon_float64x2x2_t %val ret %struct.__neon_float64x2x2_t %val
} }
Context not available.
declare %struct.__neon_float64x1x3_t @llvm.aarch64.neon.ld1x3.v1f64.p0(ptr) nounwind readonly declare %struct.__neon_float64x1x3_t @llvm.aarch64.neon.ld1x3.v1f64.p0(ptr) nounwind readonly
define %struct.__neon_int8x8x3_t @ld1_x3_v8i8(ptr %addr) { define %struct.__neon_int8x8x3_t @ld1_x3_v8i8(ptr %addr) {
; CHECK-LABEL: ld1_x3_v8i8:
; CHECK: ld1.8b { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int8x8x3_t @llvm.aarch64.neon.ld1x3.v8i8.p0(ptr %addr) %val = call %struct.__neon_int8x8x3_t @llvm.aarch64.neon.ld1x3.v8i8.p0(ptr %addr)
ret %struct.__neon_int8x8x3_t %val ret %struct.__neon_int8x8x3_t %val
} }
define %struct.__neon_int16x4x3_t @ld1_x3_v4i16(ptr %addr) { define %struct.__neon_int16x4x3_t @ld1_x3_v4i16(ptr %addr) {
; CHECK-LABEL: ld1_x3_v4i16:
; CHECK: ld1.4h { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int16x4x3_t @llvm.aarch64.neon.ld1x3.v4i16.p0(ptr %addr) %val = call %struct.__neon_int16x4x3_t @llvm.aarch64.neon.ld1x3.v4i16.p0(ptr %addr)
ret %struct.__neon_int16x4x3_t %val ret %struct.__neon_int16x4x3_t %val
} }
define %struct.__neon_int32x2x3_t @ld1_x3_v2i32(ptr %addr) { define %struct.__neon_int32x2x3_t @ld1_x3_v2i32(ptr %addr) {
; CHECK-LABEL: ld1_x3_v2i32:
; CHECK: ld1.2s { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int32x2x3_t @llvm.aarch64.neon.ld1x3.v2i32.p0(ptr %addr) %val = call %struct.__neon_int32x2x3_t @llvm.aarch64.neon.ld1x3.v2i32.p0(ptr %addr)
ret %struct.__neon_int32x2x3_t %val ret %struct.__neon_int32x2x3_t %val
} }
define %struct.__neon_float32x2x3_t @ld1_x3_v2f32(ptr %addr) { define %struct.__neon_float32x2x3_t @ld1_x3_v2f32(ptr %addr) {
; CHECK-LABEL: ld1_x3_v2f32:
; CHECK: ld1.2s { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_float32x2x3_t @llvm.aarch64.neon.ld1x3.v2f32.p0(ptr %addr) %val = call %struct.__neon_float32x2x3_t @llvm.aarch64.neon.ld1x3.v2f32.p0(ptr %addr)
ret %struct.__neon_float32x2x3_t %val ret %struct.__neon_float32x2x3_t %val
} }
define %struct.__neon_int64x1x3_t @ld1_x3_v1i64(ptr %addr) { define %struct.__neon_int64x1x3_t @ld1_x3_v1i64(ptr %addr) {
; CHECK-LABEL: ld1_x3_v1i64:
; CHECK: ld1.1d { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int64x1x3_t @llvm.aarch64.neon.ld1x3.v1i64.p0(ptr %addr) %val = call %struct.__neon_int64x1x3_t @llvm.aarch64.neon.ld1x3.v1i64.p0(ptr %addr)
ret %struct.__neon_int64x1x3_t %val ret %struct.__neon_int64x1x3_t %val
} }
define %struct.__neon_float64x1x3_t @ld1_x3_v1f64(ptr %addr) { define %struct.__neon_float64x1x3_t @ld1_x3_v1f64(ptr %addr) {
; CHECK-LABEL: ld1_x3_v1f64:
; CHECK: ld1.1d { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_float64x1x3_t @llvm.aarch64.neon.ld1x3.v1f64.p0(ptr %addr) %val = call %struct.__neon_float64x1x3_t @llvm.aarch64.neon.ld1x3.v1f64.p0(ptr %addr)
ret %struct.__neon_float64x1x3_t %val ret %struct.__neon_float64x1x3_t %val
} }
Context not available.
declare %struct.__neon_float64x2x3_t @llvm.aarch64.neon.ld1x3.v2f64.p0(ptr) nounwind readonly declare %struct.__neon_float64x2x3_t @llvm.aarch64.neon.ld1x3.v2f64.p0(ptr) nounwind readonly
define %struct.__neon_int8x16x3_t @ld1_x3_v16i8(ptr %addr) { define %struct.__neon_int8x16x3_t @ld1_x3_v16i8(ptr %addr) {
; CHECK-LABEL: ld1_x3_v16i8:
; CHECK: ld1.16b { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int8x16x3_t @llvm.aarch64.neon.ld1x3.v16i8.p0(ptr %addr) %val = call %struct.__neon_int8x16x3_t @llvm.aarch64.neon.ld1x3.v16i8.p0(ptr %addr)
ret %struct.__neon_int8x16x3_t %val ret %struct.__neon_int8x16x3_t %val
} }
define %struct.__neon_int16x8x3_t @ld1_x3_v8i16(ptr %addr) { define %struct.__neon_int16x8x3_t @ld1_x3_v8i16(ptr %addr) {
; CHECK-LABEL: ld1_x3_v8i16:
; CHECK: ld1.8h { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int16x8x3_t @llvm.aarch64.neon.ld1x3.v8i16.p0(ptr %addr) %val = call %struct.__neon_int16x8x3_t @llvm.aarch64.neon.ld1x3.v8i16.p0(ptr %addr)
ret %struct.__neon_int16x8x3_t %val ret %struct.__neon_int16x8x3_t %val
} }
define %struct.__neon_int32x4x3_t @ld1_x3_v4i32(ptr %addr) { define %struct.__neon_int32x4x3_t @ld1_x3_v4i32(ptr %addr) {
; CHECK-LABEL: ld1_x3_v4i32:
; CHECK: ld1.4s { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int32x4x3_t @llvm.aarch64.neon.ld1x3.v4i32.p0(ptr %addr) %val = call %struct.__neon_int32x4x3_t @llvm.aarch64.neon.ld1x3.v4i32.p0(ptr %addr)
ret %struct.__neon_int32x4x3_t %val ret %struct.__neon_int32x4x3_t %val
} }
define %struct.__neon_float32x4x3_t @ld1_x3_v4f32(ptr %addr) { define %struct.__neon_float32x4x3_t @ld1_x3_v4f32(ptr %addr) {
; CHECK-LABEL: ld1_x3_v4f32:
; CHECK: ld1.4s { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_float32x4x3_t @llvm.aarch64.neon.ld1x3.v4f32.p0(ptr %addr) %val = call %struct.__neon_float32x4x3_t @llvm.aarch64.neon.ld1x3.v4f32.p0(ptr %addr)
ret %struct.__neon_float32x4x3_t %val ret %struct.__neon_float32x4x3_t %val
} }
define %struct.__neon_int64x2x3_t @ld1_x3_v2i64(ptr %addr) { define %struct.__neon_int64x2x3_t @ld1_x3_v2i64(ptr %addr) {
; CHECK-LABEL: ld1_x3_v2i64:
; CHECK: ld1.2d { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int64x2x3_t @llvm.aarch64.neon.ld1x3.v2i64.p0(ptr %addr) %val = call %struct.__neon_int64x2x3_t @llvm.aarch64.neon.ld1x3.v2i64.p0(ptr %addr)
ret %struct.__neon_int64x2x3_t %val ret %struct.__neon_int64x2x3_t %val
} }
define %struct.__neon_float64x2x3_t @ld1_x3_v2f64(ptr %addr) { define %struct.__neon_float64x2x3_t @ld1_x3_v2f64(ptr %addr) {
; CHECK-LABEL: ld1_x3_v2f64:
; CHECK: ld1.2d { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_float64x2x3_t @llvm.aarch64.neon.ld1x3.v2f64.p0(ptr %addr) %val = call %struct.__neon_float64x2x3_t @llvm.aarch64.neon.ld1x3.v2f64.p0(ptr %addr)
ret %struct.__neon_float64x2x3_t %val ret %struct.__neon_float64x2x3_t %val
} }
Context not available.
declare %struct.__neon_float64x1x4_t @llvm.aarch64.neon.ld1x4.v1f64.p0(ptr) nounwind readonly declare %struct.__neon_float64x1x4_t @llvm.aarch64.neon.ld1x4.v1f64.p0(ptr) nounwind readonly
define %struct.__neon_int8x8x4_t @ld1_x4_v8i8(ptr %addr) { define %struct.__neon_int8x8x4_t @ld1_x4_v8i8(ptr %addr) {
; CHECK-LABEL: ld1_x4_v8i8:
; CHECK: ld1.8b { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int8x8x4_t @llvm.aarch64.neon.ld1x4.v8i8.p0(ptr %addr) %val = call %struct.__neon_int8x8x4_t @llvm.aarch64.neon.ld1x4.v8i8.p0(ptr %addr)
ret %struct.__neon_int8x8x4_t %val ret %struct.__neon_int8x8x4_t %val
} }
define %struct.__neon_int16x4x4_t @ld1_x4_v4i16(ptr %addr) { define %struct.__neon_int16x4x4_t @ld1_x4_v4i16(ptr %addr) {
; CHECK-LABEL: ld1_x4_v4i16:
; CHECK: ld1.4h { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int16x4x4_t @llvm.aarch64.neon.ld1x4.v4i16.p0(ptr %addr) %val = call %struct.__neon_int16x4x4_t @llvm.aarch64.neon.ld1x4.v4i16.p0(ptr %addr)
ret %struct.__neon_int16x4x4_t %val ret %struct.__neon_int16x4x4_t %val
} }
define %struct.__neon_int32x2x4_t @ld1_x4_v2i32(ptr %addr) { define %struct.__neon_int32x2x4_t @ld1_x4_v2i32(ptr %addr) {
; CHECK-LABEL: ld1_x4_v2i32:
; CHECK: ld1.2s { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int32x2x4_t @llvm.aarch64.neon.ld1x4.v2i32.p0(ptr %addr) %val = call %struct.__neon_int32x2x4_t @llvm.aarch64.neon.ld1x4.v2i32.p0(ptr %addr)
ret %struct.__neon_int32x2x4_t %val ret %struct.__neon_int32x2x4_t %val
} }
define %struct.__neon_float32x2x4_t @ld1_x4_v2f32(ptr %addr) { define %struct.__neon_float32x2x4_t @ld1_x4_v2f32(ptr %addr) {
; CHECK-LABEL: ld1_x4_v2f32:
; CHECK: ld1.2s { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_float32x2x4_t @llvm.aarch64.neon.ld1x4.v2f32.p0(ptr %addr) %val = call %struct.__neon_float32x2x4_t @llvm.aarch64.neon.ld1x4.v2f32.p0(ptr %addr)
ret %struct.__neon_float32x2x4_t %val ret %struct.__neon_float32x2x4_t %val
} }
define %struct.__neon_int64x1x4_t @ld1_x4_v1i64(ptr %addr) { define %struct.__neon_int64x1x4_t @ld1_x4_v1i64(ptr %addr) {
; CHECK-LABEL: ld1_x4_v1i64:
; CHECK: ld1.1d { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int64x1x4_t @llvm.aarch64.neon.ld1x4.v1i64.p0(ptr %addr) %val = call %struct.__neon_int64x1x4_t @llvm.aarch64.neon.ld1x4.v1i64.p0(ptr %addr)
ret %struct.__neon_int64x1x4_t %val ret %struct.__neon_int64x1x4_t %val
} }
define %struct.__neon_float64x1x4_t @ld1_x4_v1f64(ptr %addr) { define %struct.__neon_float64x1x4_t @ld1_x4_v1f64(ptr %addr) {
; CHECK-LABEL: ld1_x4_v1f64:
; CHECK: ld1.1d { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_float64x1x4_t @llvm.aarch64.neon.ld1x4.v1f64.p0(ptr %addr) %val = call %struct.__neon_float64x1x4_t @llvm.aarch64.neon.ld1x4.v1f64.p0(ptr %addr)
ret %struct.__neon_float64x1x4_t %val ret %struct.__neon_float64x1x4_t %val
} }
Context not available.
declare %struct.__neon_float64x2x4_t @llvm.aarch64.neon.ld1x4.v2f64.p0(ptr) nounwind readonly declare %struct.__neon_float64x2x4_t @llvm.aarch64.neon.ld1x4.v2f64.p0(ptr) nounwind readonly
define %struct.__neon_int8x16x4_t @ld1_x4_v16i8(ptr %addr) { define %struct.__neon_int8x16x4_t @ld1_x4_v16i8(ptr %addr) {
; CHECK-LABEL: ld1_x4_v16i8:
; CHECK: ld1.16b { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int8x16x4_t @llvm.aarch64.neon.ld1x4.v16i8.p0(ptr %addr) %val = call %struct.__neon_int8x16x4_t @llvm.aarch64.neon.ld1x4.v16i8.p0(ptr %addr)
ret %struct.__neon_int8x16x4_t %val ret %struct.__neon_int8x16x4_t %val
} }
define %struct.__neon_int16x8x4_t @ld1_x4_v8i16(ptr %addr) { define %struct.__neon_int16x8x4_t @ld1_x4_v8i16(ptr %addr) {
; CHECK-LABEL: ld1_x4_v8i16:
; CHECK: ld1.8h { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int16x8x4_t @llvm.aarch64.neon.ld1x4.v8i16.p0(ptr %addr) %val = call %struct.__neon_int16x8x4_t @llvm.aarch64.neon.ld1x4.v8i16.p0(ptr %addr)
ret %struct.__neon_int16x8x4_t %val ret %struct.__neon_int16x8x4_t %val
} }
define %struct.__neon_int32x4x4_t @ld1_x4_v4i32(ptr %addr) { define %struct.__neon_int32x4x4_t @ld1_x4_v4i32(ptr %addr) {
; CHECK-LABEL: ld1_x4_v4i32:
; CHECK: ld1.4s { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int32x4x4_t @llvm.aarch64.neon.ld1x4.v4i32.p0(ptr %addr) %val = call %struct.__neon_int32x4x4_t @llvm.aarch64.neon.ld1x4.v4i32.p0(ptr %addr)
ret %struct.__neon_int32x4x4_t %val ret %struct.__neon_int32x4x4_t %val
} }
define %struct.__neon_float32x4x4_t @ld1_x4_v4f32(ptr %addr) { define %struct.__neon_float32x4x4_t @ld1_x4_v4f32(ptr %addr) {
; CHECK-LABEL: ld1_x4_v4f32:
; CHECK: ld1.4s { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_float32x4x4_t @llvm.aarch64.neon.ld1x4.v4f32.p0(ptr %addr) %val = call %struct.__neon_float32x4x4_t @llvm.aarch64.neon.ld1x4.v4f32.p0(ptr %addr)
ret %struct.__neon_float32x4x4_t %val ret %struct.__neon_float32x4x4_t %val
} }
define %struct.__neon_int64x2x4_t @ld1_x4_v2i64(ptr %addr) { define %struct.__neon_int64x2x4_t @ld1_x4_v2i64(ptr %addr) {
; CHECK-LABEL: ld1_x4_v2i64:
; CHECK: ld1.2d { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_int64x2x4_t @llvm.aarch64.neon.ld1x4.v2i64.p0(ptr %addr) %val = call %struct.__neon_int64x2x4_t @llvm.aarch64.neon.ld1x4.v2i64.p0(ptr %addr)
ret %struct.__neon_int64x2x4_t %val ret %struct.__neon_int64x2x4_t %val
} }
define %struct.__neon_float64x2x4_t @ld1_x4_v2f64(ptr %addr) { define %struct.__neon_float64x2x4_t @ld1_x4_v2f64(ptr %addr) {
; CHECK-LABEL: ld1_x4_v2f64:
; CHECK: ld1.2d { {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}}, {{v[0-9]+}} }, [x0]
%val = call %struct.__neon_float64x2x4_t @llvm.aarch64.neon.ld1x4.v2f64.p0(ptr %addr) %val = call %struct.__neon_float64x2x4_t @llvm.aarch64.neon.ld1x4.v2f64.p0(ptr %addr)
ret %struct.__neon_float64x2x4_t %val ret %struct.__neon_float64x2x4_t %val
} }
;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
; CHECK: {{.*}}
Context not available.

llvm/test/CodeGen/AArch64/arm64-ldp.ll

llvm/test/CodeGen/AArch64/arm64-misaligned-memcpy-inline.ll

llvm/test/CodeGen/AArch64/arm64-mul.ll

llvm/test/CodeGen/AArch64/arm64-neon-3vdiff.ll

llvm/test/CodeGen/AArch64/arm64-neon-mul-div.ll

llvm/test/CodeGen/AArch64/arm64-neon-v8.1a.ll

llvm/test/CodeGen/AArch64/arm64-non-pow2-ldst.ll

llvm/test/CodeGen/AArch64/arm64-nvcast.ll

llvm/test/CodeGen/AArch64/arm64-promote-const-complex-initializers.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc -o - %s | FileCheck %s ; RUN: llc -o - %s | FileCheck %s
; AsmPrinter cannot lower floating point constant expressions in global ; AsmPrinter cannot lower floating point constant expressions in global
Context not available.
target triple = "arm64-apple-ios14.0.0" target triple = "arm64-apple-ios14.0.0"
define [1 x <4 x float>] @test1() { define [1 x <4 x float>] @test1() {
; CHECK-LABEL: .p2align 4, 0x0 ; -- Begin function test1 ; CHECK-LABEL: test1:
dmgreenUnsubmitted
Done
ReplyInline Actions

This doesn't look like it is checking everything it did in the past.

dmgreen: This doesn't look like it is checking everything it did in the past.
; CHECK-NEXT: lCPI0_0: ; CHECK: ; %bb.0:
; CHECK-NEXT: .quad 0 ; 0x0 ; CHECK-NEXT: Lloh0:
; CHECK-NEXT: .quad 4575657221408423936 ; 0x3f80000000000000 ; CHECK-NEXT: adrp x8, lCPI0_0@PAGE
; CHECK-NEXT: .section __TEXT,__text,regular,pure_instructions ; CHECK-NEXT: Lloh1:
; CHECK-NEXT: .globl _test1 ; CHECK-NEXT: ldr q0, [x8, lCPI0_0@PAGEOFF]
; CHECK-NEXT: .p2align 2 ; CHECK-NEXT: ret
; CHECK-NEXT: _test1: ; @test1 ; CHECK-NEXT: .loh AdrpLdr Lloh0, Lloh1
; CHECK-NEXT: .cfi_startproc
; CHECK-NEXT: ; %bb.0:
; CHECK-NEXT: Lloh0:
; CHECK-NEXT: adrp x8, lCPI0_0@PAGE
; CHECK-NEXT: Lloh1:
; CHECK-NEXT: ldr q0, [x8, lCPI0_0@PAGEOFF]
; CHECK-NEXT: ret
ret [1 x <4 x float>] [<4 x float> bitcast (<1 x i128> <i128 84405977732342157929391748327801880576> to <4 x float>)] ret [1 x <4 x float>] [<4 x float> bitcast (<1 x i128> <i128 84405977732342157929391748327801880576> to <4 x float>)]
} }
define [1 x <4 x float>] @test2() { define [1 x <4 x float>] @test2() {
; CHECK-LABEL: .p2align 4, 0x0 ; -- Begin function test2 ; CHECK-LABEL: test2:
; CHECK-NEXT: lCPI1_0: ; CHECK: ; %bb.0:
; CHECK-NEXT: .long 0x00000000 ; float 0 ; CHECK-NEXT: Lloh2:
; CHECK-NEXT: .long 0x00000000 ; float 0 ; CHECK-NEXT: adrp x8, lCPI1_0@PAGE
; CHECK-NEXT: .long 0x00000000 ; float 0 ; CHECK-NEXT: Lloh3:
; CHECK-NEXT: .long 0x3f800000 ; float 1 ; CHECK-NEXT: ldr q1, [x8, lCPI1_0@PAGEOFF]
; CHECK-NEXT: .section __TEXT,__text,regular,pure_instructions ; CHECK-NEXT: mov s2, v1[1]
; CHECK-NEXT: .globl _test2 ; CHECK-NEXT: fneg s0, s1
; CHECK-NEXT: .p2align 2 ; CHECK-NEXT: mov s3, v1[2]
; CHECK-NEXT: _test2: ; @test2 ; CHECK-NEXT: mov s1, v1[3]
; CHECK-NEXT: .cfi_startproc ; CHECK-NEXT: fneg s2, s2
; CHECK-NEXT: ; %bb.0: ; CHECK-NEXT: fneg s3, s3
; CHECK-NEXT: Lloh2: ; CHECK-NEXT: fneg s1, s1
; CHECK-NEXT: adrp x8, lCPI1_0@PAGE ; CHECK-NEXT: mov.s v0[1], v2[0]
; CHECK-NEXT: Lloh3: ; CHECK-NEXT: mov.s v0[2], v3[0]
; CHECK-NEXT: ldr q1, [x8, lCPI1_0@PAGEOFF] ; CHECK-NEXT: mov.s v0[3], v1[0]
; CHECK-NEXT: mov s2, v1[1] ; CHECK-NEXT: ret
; CHECK-NEXT: mov s3, v1[2] ; CHECK-NEXT: .loh AdrpLdr Lloh2, Lloh3
; CHECK-NEXT: fneg s0, s1
; CHECK-NEXT: mov s1, v1[3]
; CHECK-NEXT: fneg s2, s2
; CHECK-NEXT: fneg s3, s3
; CHECK-NEXT: fneg s1, s1
; CHECK-NEXT: mov.s v0[1], v2[0]
; CHECK-NEXT: mov.s v0[2], v3[0]
; CHECK-NEXT: mov.s v0[3], v1[0]
; CHECK-NEXT: ret
;
%constexpr = fneg float extractelement (<4 x float> bitcast (<1 x i128> <i128 84405977732342157929391748327801880576> to <4 x float>), i32 0) %constexpr = fneg float extractelement (<4 x float> bitcast (<1 x i128> <i128 84405977732342157929391748327801880576> to <4 x float>), i32 0)
%constexpr1 = fneg float extractelement (<4 x float> bitcast (<1 x i128> <i128 84405977732342157929391748327801880576> to <4 x float>), i32 1) %constexpr1 = fneg float extractelement (<4 x float> bitcast (<1 x i128> <i128 84405977732342157929391748327801880576> to <4 x float>), i32 1)
%constexpr2 = fneg float extractelement (<4 x float> bitcast (<1 x i128> <i128 84405977732342157929391748327801880576> to <4 x float>), i32 2) %constexpr2 = fneg float extractelement (<4 x float> bitcast (<1 x i128> <i128 84405977732342157929391748327801880576> to <4 x float>), i32 2)
Context not available.

llvm/test/CodeGen/AArch64/arm64-register-pairing.ll

llvm/test/CodeGen/AArch64/arm64-rev.ll

llvm/test/CodeGen/AArch64/arm64-setcc-int-to-fp-combine.ll

llvm/test/CodeGen/AArch64/arm64-setcc-swap-infloop.ll

llvm/test/CodeGen/AArch64/arm64-shrink-wrapping.ll

llvm/test/CodeGen/AArch64/arm64-subvector-extend.ll

llvm/test/CodeGen/AArch64/arm64-tbl.ll

llvm/test/CodeGen/AArch64/arm64-vabs.ll

llvm/test/CodeGen/AArch64/arm64-vector-ldst.ll

llvm/test/CodeGen/AArch64/arm64-vhadd.ll

llvm/test/CodeGen/AArch64/arm64-vmul.ll

llvm/test/CodeGen/AArch64/arm64-vshift.ll

llvm/test/CodeGen/AArch64/arm64-xaluo.ll

llvm/test/CodeGen/AArch64/arm64-zip.ll

llvm/test/CodeGen/AArch64/arm64_32-addrs.ll

llvm/test/CodeGen/AArch64/arm64_32.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc -mtriple=arm64_32-apple-ios7.0 %s -filetype=obj -o - -disable-post-ra -frame-pointer=non-leaf | \ ; RUN: llc -mtriple=arm64_32-apple-ios7.0 %s -filetype=obj -o - -disable-post-ra -frame-pointer=non-leaf | \
; RUN: llvm-objdump --private-headers - | \ ; RUN: llvm-objdump --private-headers - | \
; RUN: FileCheck %s --check-prefix=CHECK-MACHO ; RUN: FileCheck %s --check-prefix=CHECK-MACHO
Context not available.
@var_got = external global i8 @var_got = external global i8
define ptr @test_global_addr() { define ptr @test_global_addr() {
; CHECK-LABEL: test_global_addr: ; CHECK-OPT-LABEL: test_global_addr:
dmgreenUnsubmitted
Done
ReplyInline Actions

This file might be better manually generated. Some of the check lines and comments are deliberately placed.

dmgreen: This file might be better manually generated. Some of the check lines and comments are…
; CHECK: adrp [[PAGE:x[0-9]+]], _var32@PAGE ; CHECK-OPT: ; %bb.0:
; CHECK-OPT: add x0, [[PAGE]], _var32@PAGEOFF ; CHECK-OPT-NEXT: Lloh0:
; CHECK-FAST: add [[TMP:x[0-9]+]], [[PAGE]], _var32@PAGEOFF ; CHECK-OPT-NEXT: adrp x0, _var32@PAGE
; CHECK-FAST: and x0, [[TMP]], #0xffffffff ; CHECK-OPT-NEXT: Lloh1:
; CHECK-OPT-NEXT: add x0, x0, _var32@PAGEOFF
; CHECK-OPT-NEXT: ret
; CHECK-OPT-NEXT: .loh AdrpAdd Lloh0, Lloh1
;
; CHECK-FAST-LABEL: test_global_addr:
; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: Lloh0:
; CHECK-FAST-NEXT: adrp x8, _var32@PAGE
; CHECK-FAST-NEXT: Lloh1:
; CHECK-FAST-NEXT: add x8, x8, _var32@PAGEOFF
; CHECK-FAST-NEXT: and x0, x8, #0xffffffff
; CHECK-FAST-NEXT: ret
; CHECK-FAST-NEXT: .loh AdrpAdd Lloh0, Lloh1
ret ptr @var32 ret ptr @var32
} }
Context not available.
; gets truncated to 32-bits, it's free. No need to zero out higher bits of that ; gets truncated to 32-bits, it's free. No need to zero out higher bits of that
; register. ; register.
define i64 @test_global_addr_extension() { define i64 @test_global_addr_extension() {
; CHECK-LABEL: test_global_addr_extension: ; CHECK-OPT-LABEL: test_global_addr_extension:
; CHECK: adrp [[PAGE:x[0-9]+]], _var32@PAGE ; CHECK-OPT: ; %bb.0:
; CHECK: add x0, [[PAGE]], _var32@PAGEOFF ; CHECK-OPT-NEXT: Lloh2:
; CHECK-NOT: and ; CHECK-OPT-NEXT: adrp x0, _var32@PAGE
; CHECK: ret ; CHECK-OPT-NEXT: Lloh3:
; CHECK-OPT-NEXT: add x0, x0, _var32@PAGEOFF
; CHECK-OPT-NEXT: ret
; CHECK-OPT-NEXT: .loh AdrpAdd Lloh2, Lloh3
;
; CHECK-FAST-LABEL: test_global_addr_extension:
; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: Lloh2:
; CHECK-FAST-NEXT: adrp x8, _var32@PAGE
; CHECK-FAST-NEXT: Lloh3:
; CHECK-FAST-NEXT: add x0, x8, _var32@PAGEOFF
; CHECK-FAST-NEXT: ret
; CHECK-FAST-NEXT: .loh AdrpAdd Lloh2, Lloh3
ret i64 ptrtoint(ptr @var32 to i64) ret i64 ptrtoint(ptr @var32 to i64)
} }
define i32 @test_global_value() { define i32 @test_global_value() {
; CHECK-LABEL: test_global_value: ; CHECK-LABEL: test_global_value:
; CHECK: adrp x[[PAGE:[0-9]+]], _var32@PAGE ; CHECK: ; %bb.0:
; CHECK: ldr w0, [x[[PAGE]], _var32@PAGEOFF] ; CHECK-NEXT: Lloh4:
; CHECK-NEXT: adrp x8, _var32@PAGE
; CHECK-NEXT: Lloh5:
; CHECK-NEXT: ldr w0, [x8, _var32@PAGEOFF]
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh4, Lloh5
%val = load i32, ptr @var32, align 4 %val = load i32, ptr @var32, align 4
ret i32 %val ret i32 %val
} }
Context not available.
; Because the addition may wrap, it is not safe to use "ldr w0, [xN, #32]" here. ; Because the addition may wrap, it is not safe to use "ldr w0, [xN, #32]" here.
define i32 @test_unsafe_indexed_add() { define i32 @test_unsafe_indexed_add() {
; CHECK-LABEL: test_unsafe_indexed_add: ; CHECK-LABEL: test_unsafe_indexed_add:
; CHECK: add x[[VAR32:[0-9]+]], {{x[0-9]+}}, _var32@PAGEOFF ; CHECK: ; %bb.0:
; CHECK: add w[[ADDR:[0-9]+]], w[[VAR32]], #32 ; CHECK-NEXT: Lloh6:
; CHECK: ldr w0, [x[[ADDR]]] ; CHECK-NEXT: adrp x8, _var32@PAGE
; CHECK-NEXT: Lloh7:
; CHECK-NEXT: add x8, x8, _var32@PAGEOFF
; CHECK-NEXT: add w8, w8, #32
; CHECK-NEXT: ldr w0, [x8]
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpAdd Lloh6, Lloh7
%addr_int = ptrtoint ptr @var32 to i32 %addr_int = ptrtoint ptr @var32 to i32
%addr_plus_32 = add i32 %addr_int, 32 %addr_plus_32 = add i32 %addr_int, 32
%addr = inttoptr i32 %addr_plus_32 to ptr %addr = inttoptr i32 %addr_plus_32 to ptr
Context not available.
; 32-bytes below 2^32, and we can use the load this time. ; 32-bytes below 2^32, and we can use the load this time.
define i32 @test_safe_indexed_add() { define i32 @test_safe_indexed_add() {
; CHECK-LABEL: test_safe_indexed_add: ; CHECK-LABEL: test_safe_indexed_add:
; CHECK: add x[[VAR32:[0-9]+]], {{x[0-9]+}}, _var32@PAGEOFF ; CHECK: ; %bb.0:
; CHECK: add w[[ADDR:[0-9]+]], w[[VAR32]], #32 ; CHECK-NEXT: Lloh8:
; CHECK: ldr w0, [x[[ADDR]]] ; CHECK-NEXT: adrp x8, _var32@PAGE
; CHECK-NEXT: Lloh9:
; CHECK-NEXT: add x8, x8, _var32@PAGEOFF
; CHECK-NEXT: add w8, w8, #32
; CHECK-NEXT: ldr w0, [x8]
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpAdd Lloh8, Lloh9
%addr_int = ptrtoint ptr @var32 to i64 %addr_int = ptrtoint ptr @var32 to i64
%addr_plus_32 = add nuw i64 %addr_int, 32 %addr_plus_32 = add nuw i64 %addr_int, 32
%addr = inttoptr i64 %addr_plus_32 to ptr %addr = inttoptr i64 %addr_plus_32 to ptr
Context not available.
define i32 @test_safe_indexed_or(i32 %in) { define i32 @test_safe_indexed_or(i32 %in) {
; CHECK-LABEL: test_safe_indexed_or: ; CHECK-LABEL: test_safe_indexed_or:
; CHECK: and [[TMP:w[0-9]+]], {{w[0-9]+}}, #0xfffffff0 ; CHECK: ; %bb.0:
; CHECK: orr w[[ADDR:[0-9]+]], [[TMP]], #0x4 ; CHECK-NEXT: and w8, w0, #0xfffffff0
; CHECK: ldr w0, [x[[ADDR]]] ; CHECK-NEXT: orr w8, w8, #0x4
; CHECK-NEXT: ldr w0, [x8]
; CHECK-NEXT: ret
%addr_int = and i32 %in, -16 %addr_int = and i32 %in, -16
%addr_plus_4 = or i32 %addr_int, 4 %addr_plus_4 = or i32 %addr_int, 4
%addr = inttoptr i32 %addr_plus_4 to ptr %addr = inttoptr i32 %addr_plus_4 to ptr
Context not available.
; "sext(base) + sext(offset) == base + offset". ; "sext(base) + sext(offset) == base + offset".
define i32 @test_unsafe_nsw_indexed_add() { define i32 @test_unsafe_nsw_indexed_add() {
; CHECK-LABEL: test_unsafe_nsw_indexed_add: ; CHECK-LABEL: test_unsafe_nsw_indexed_add:
; CHECK: add x[[VAR32:[0-9]+]], {{x[0-9]+}}, _var32@PAGEOFF ; CHECK: ; %bb.0:
; CHECK: add w[[ADDR:[0-9]+]], w[[VAR32]], #32 ; CHECK-NEXT: Lloh10:
; CHECK-NOT: ubfx ; CHECK-NEXT: adrp x8, _var32@PAGE
; CHECK: ldr w0, [x[[ADDR]]] ; CHECK-NEXT: Lloh11:
; CHECK-NEXT: add x8, x8, _var32@PAGEOFF
; CHECK-NEXT: add w8, w8, #32
; CHECK-NEXT: ldr w0, [x8]
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpAdd Lloh10, Lloh11
%addr_int = ptrtoint ptr @var32 to i32 %addr_int = ptrtoint ptr @var32 to i32
%addr_plus_32 = add nsw i32 %addr_int, 32 %addr_plus_32 = add nsw i32 %addr_int, 32
%addr = inttoptr i32 %addr_plus_32 to ptr %addr = inttoptr i32 %addr_plus_32 to ptr
Context not available.
; Because the addition may wrap, it is not safe to use "ldr w0, [xN, #32]" here. ; Because the addition may wrap, it is not safe to use "ldr w0, [xN, #32]" here.
define i32 @test_unsafe_unscaled_add() { define i32 @test_unsafe_unscaled_add() {
; CHECK-LABEL: test_unsafe_unscaled_add: ; CHECK-LABEL: test_unsafe_unscaled_add:
; CHECK: add x[[VAR32:[0-9]+]], {{x[0-9]+}}, _var32@PAGEOFF ; CHECK: ; %bb.0:
; CHECK: add w[[ADDR:[0-9]+]], w[[VAR32]], #3 ; CHECK-NEXT: Lloh12:
; CHECK: ldr w0, [x[[ADDR]]] ; CHECK-NEXT: adrp x8, _var32@PAGE
; CHECK-NEXT: Lloh13:
; CHECK-NEXT: add x8, x8, _var32@PAGEOFF
; CHECK-NEXT: add w8, w8, #3
; CHECK-NEXT: ldr w0, [x8]
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpAdd Lloh12, Lloh13
%addr_int = ptrtoint ptr @var32 to i32 %addr_int = ptrtoint ptr @var32 to i32
%addr_plus_3 = add i32 %addr_int, 3 %addr_plus_3 = add i32 %addr_int, 3
%addr = inttoptr i32 %addr_plus_3 to ptr %addr = inttoptr i32 %addr_plus_3 to ptr
Context not available.
; 32-bytes below 2^32, and we can use the load this time. ; 32-bytes below 2^32, and we can use the load this time.
define i32 @test_safe_unscaled_add() { define i32 @test_safe_unscaled_add() {
; CHECK-LABEL: test_safe_unscaled_add: ; CHECK-LABEL: test_safe_unscaled_add:
; CHECK: add x[[VAR32:[0-9]+]], {{x[0-9]+}}, _var32@PAGEOFF ; CHECK: ; %bb.0:
; CHECK: add w[[ADDR:[0-9]+]], w[[VAR32]], #3 ; CHECK-NEXT: Lloh14:
; CHECK: ldr w0, [x[[ADDR]]] ; CHECK-NEXT: adrp x8, _var32@PAGE
; CHECK-NEXT: Lloh15:
; CHECK-NEXT: add x8, x8, _var32@PAGEOFF
; CHECK-NEXT: add w8, w8, #3
; CHECK-NEXT: ldr w0, [x8]
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpAdd Lloh14, Lloh15
%addr_int = ptrtoint ptr @var32 to i32 %addr_int = ptrtoint ptr @var32 to i32
%addr_plus_3 = add nuw i32 %addr_int, 3 %addr_plus_3 = add nuw i32 %addr_int, 3
%addr = inttoptr i32 %addr_plus_3 to ptr %addr = inttoptr i32 %addr_plus_3 to ptr
Context not available.
; "sext(base) + sext(offset) == base + offset". ; "sext(base) + sext(offset) == base + offset".
define i32 @test_unsafe_nsw_unscaled_add() { define i32 @test_unsafe_nsw_unscaled_add() {
; CHECK-LABEL: test_unsafe_nsw_unscaled_add: ; CHECK-LABEL: test_unsafe_nsw_unscaled_add:
; CHECK: add x[[VAR32:[0-9]+]], {{x[0-9]+}}, _var32@PAGEOFF ; CHECK: ; %bb.0:
; CHECK: add w[[ADDR:[0-9]+]], w[[VAR32]], #3 ; CHECK-NEXT: Lloh16:
; CHECK-NOT: ubfx ; CHECK-NEXT: adrp x8, _var32@PAGE
; CHECK: ldr w0, [x[[ADDR]]] ; CHECK-NEXT: Lloh17:
; CHECK-NEXT: add x8, x8, _var32@PAGEOFF
; CHECK-NEXT: add w8, w8, #3
; CHECK-NEXT: ldr w0, [x8]
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpAdd Lloh16, Lloh17
%addr_int = ptrtoint ptr @var32 to i32 %addr_int = ptrtoint ptr @var32 to i32
%addr_plus_3 = add nsw i32 %addr_int, 3 %addr_plus_3 = add nsw i32 %addr_int, 3
%addr = inttoptr i32 %addr_plus_3 to ptr %addr = inttoptr i32 %addr_plus_3 to ptr
Context not available.
; here. ; here.
define i32 @test_unsafe_negative_unscaled_add() { define i32 @test_unsafe_negative_unscaled_add() {
; CHECK-LABEL: test_unsafe_negative_unscaled_add: ; CHECK-LABEL: test_unsafe_negative_unscaled_add:
; CHECK: add x[[VAR32:[0-9]+]], {{x[0-9]+}}, _var32@PAGEOFF ; CHECK: ; %bb.0:
; CHECK: sub w[[ADDR:[0-9]+]], w[[VAR32]], #3 ; CHECK-NEXT: Lloh18:
; CHECK: ldr w0, [x[[ADDR]]] ; CHECK-NEXT: adrp x8, _var32@PAGE
; CHECK-NEXT: Lloh19:
; CHECK-NEXT: add x8, x8, _var32@PAGEOFF
; CHECK-NEXT: sub w8, w8, #3
; CHECK-NEXT: ldr w0, [x8]
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpAdd Lloh18, Lloh19
%addr_int = ptrtoint ptr @var32 to i32 %addr_int = ptrtoint ptr @var32 to i32
%addr_minus_3 = add i32 %addr_int, -3 %addr_minus_3 = add i32 %addr_int, -3
%addr = inttoptr i32 %addr_minus_3 to ptr %addr = inttoptr i32 %addr_minus_3 to ptr
Context not available.
} }
define ptr @test_got_addr() { define ptr @test_got_addr() {
; CHECK-LABEL: test_got_addr: ; CHECK-OPT-LABEL: test_got_addr:
; CHECK: adrp x[[PAGE:[0-9]+]], _var_got@GOTPAGE ; CHECK-OPT: ; %bb.0:
; CHECK-OPT: ldr w0, [x[[PAGE]], _var_got@GOTPAGEOFF] ; CHECK-OPT-NEXT: Lloh20:
; CHECK-FAST: ldr w[[TMP:[0-9]+]], [x[[PAGE]], _var_got@GOTPAGEOFF] ; CHECK-OPT-NEXT: adrp x0, _var_got@GOTPAGE
; CHECK-FAST: and x0, x[[TMP]], #0xffffffff ; CHECK-OPT-NEXT: Lloh21:
; CHECK-OPT-NEXT: ldr w0, [x0, _var_got@GOTPAGEOFF]
; CHECK-OPT-NEXT: ret
; CHECK-OPT-NEXT: .loh AdrpLdrGot Lloh20, Lloh21
;
; CHECK-FAST-LABEL: test_got_addr:
; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: Lloh20:
; CHECK-FAST-NEXT: adrp x8, _var_got@GOTPAGE
; CHECK-FAST-NEXT: Lloh21:
; CHECK-FAST-NEXT: ldr w8, [x8, _var_got@GOTPAGEOFF]
; CHECK-FAST-NEXT: and x0, x8, #0xffffffff
; CHECK-FAST-NEXT: ret
; CHECK-FAST-NEXT: .loh AdrpLdrGot Lloh20, Lloh21
ret ptr @var_got ret ptr @var_got
} }
define float @test_va_arg_f32(ptr %list) { define float @test_va_arg_f32(ptr %list) {
; CHECK-LABEL: test_va_arg_f32: ; CHECK-LABEL: test_va_arg_f32:
; CHECK: ; %bb.0:
; CHECK-NEXT: ldr w8, [x0]
; CHECK-NEXT: add w9, w8, #8
; CHECK-NEXT: str w9, [x0]
; CHECK-NEXT: ldr d0, [x8]
; CHECK-NEXT: fcvt s0, d0
; CHECK-NEXT: ret
; CHECK: ldr w[[START:[0-9]+]], [x0]
; CHECK: add [[AFTER:w[0-9]+]], w[[START]], #8
; CHECK: str [[AFTER]], [x0]
; Floating point arguments get promoted to double as per C99. ; Floating point arguments get promoted to double as per C99.
; CHECK: ldr [[DBL:d[0-9]+]], [x[[START]]]
; CHECK: fcvt s0, [[DBL]]
%res = va_arg ptr %list, float %res = va_arg ptr %list, float
ret float %res ret float %res
} }
Context not available.
; Interesting point is that the slot is 4 bytes. ; Interesting point is that the slot is 4 bytes.
define i8 @test_va_arg_i8(ptr %list) { define i8 @test_va_arg_i8(ptr %list) {
; CHECK-LABEL: test_va_arg_i8: ; CHECK-LABEL: test_va_arg_i8:
; CHECK: ; %bb.0:
; CHECK-NEXT: ldr w8, [x0]
; CHECK-NEXT: add w9, w8, #4
; CHECK-NEXT: str w9, [x0]
; CHECK-NEXT: ldr w0, [x8]
; CHECK-NEXT: ret
; CHECK: ldr w[[START:[0-9]+]], [x0]
; CHECK: add [[AFTER:w[0-9]+]], w[[START]], #4
; CHECK: str [[AFTER]], [x0]
; i8 gets promoted to int (again, as per C99). ; i8 gets promoted to int (again, as per C99).
; CHECK: ldr w0, [x[[START]]]
%res = va_arg ptr %list, i8 %res = va_arg ptr %list, i8
ret i8 %res ret i8 %res
Context not available.
; bytes). ; bytes).
define i64 @test_va_arg_i64(ptr %list) { define i64 @test_va_arg_i64(ptr %list) {
; CHECK-LABEL: test_va_arg_i64: ; CHECK-LABEL: test_va_arg_i64:
; CHECK: ; %bb.0:
; CHECK-NEXT: ldr w8, [x0]
; CHECK-NEXT: add x8, x8, #7
; CHECK-NEXT: and x8, x8, #0x1fffffff8
; CHECK-NEXT: add w9, w8, #8
; CHECK-NEXT: str w9, [x0]
; CHECK-NEXT: ldr x0, [x8]
; CHECK-NEXT: ret
; Update the list for the next user (minimum slot size is 4, but the actual ; Update the list for the next user (minimum slot size is 4, but the actual
; argument is 8 which had better be reflected!) ; argument is 8 which had better be reflected!)
; CHECK: ldr w[[UNALIGNED_START:[0-9]+]], [x0]
; CHECK: add [[ALIGN_TMP:x[0-9]+]], x[[UNALIGNED_START]], #7
; CHECK: and x[[START:[0-9]+]], [[ALIGN_TMP]], #0x1fffffff8
; CHECK: add w[[AFTER:[0-9]+]], w[[START]], #8
; CHECK: str w[[AFTER]], [x0]
; CHECK: ldr x0, [x[[START]]]
%res = va_arg ptr %list, i64 %res = va_arg ptr %list, i64
ret i64 %res ret i64 %res
Context not available.
declare void @bar(...) declare void @bar(...)
define void @test_va_call(i8 %l, i8 %r, float %in, ptr %ptr) { define void @test_va_call(i8 %l, i8 %r, float %in, ptr %ptr) {
; CHECK-LABEL: test_va_call: ; CHECK-OPT-LABEL: test_va_call:
; CHECK: add [[SUM:w[0-9]+]], {{w[0-9]+}}, w1 ; CHECK-OPT: ; %bb.0:
; CHECK-OPT-NEXT: sub sp, sp, #64
; CHECK-OPT-NEXT: stp x29, x30, [sp, #48] ; 16-byte Folded Spill
; CHECK-OPT-NEXT: add x29, sp, #48
; CHECK-OPT-NEXT: .cfi_def_cfa w29, 16
; CHECK-OPT-NEXT: .cfi_offset w30, -8
; CHECK-OPT-NEXT: .cfi_offset w29, -16
; CHECK-OPT-NEXT: add w8, w0, w1
; CHECK-OPT-NEXT: str w2, [sp, #32]
; CHECK-OPT-NEXT: str xzr, [sp, #24]
; CHECK-OPT-NEXT: str s0, [sp, #16]
; CHECK-OPT-NEXT: str xzr, [sp, #8]
; CHECK-OPT-NEXT: str w8, [sp]
; CHECK-OPT-NEXT: bl _bar
; CHECK-OPT-NEXT: ldp x29, x30, [sp, #48] ; 16-byte Folded Reload
; CHECK-OPT-NEXT: add sp, sp, #64
; CHECK-OPT-NEXT: ret
;
; CHECK-FAST-LABEL: test_va_call:
; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: sub sp, sp, #64
; CHECK-FAST-NEXT: stp x29, x30, [sp, #48] ; 16-byte Folded Spill
; CHECK-FAST-NEXT: add x29, sp, #48
; CHECK-FAST-NEXT: .cfi_def_cfa w29, 16
; CHECK-FAST-NEXT: .cfi_offset w30, -8
; CHECK-FAST-NEXT: .cfi_offset w29, -16
; CHECK-FAST-NEXT: sxtb w8, w0
; CHECK-FAST-NEXT: add w8, w8, w1, sxtb
; CHECK-FAST-NEXT: str w2, [sp, #32]
; CHECK-FAST-NEXT: str xzr, [sp, #24]
; CHECK-FAST-NEXT: str s0, [sp, #16]
; CHECK-FAST-NEXT: str xzr, [sp, #8]
; CHECK-FAST-NEXT: str w8, [sp]
; CHECK-FAST-NEXT: bl _bar
; CHECK-FAST-NEXT: ldp x29, x30, [sp, #48] ; 16-byte Folded Reload
; CHECK-FAST-NEXT: add sp, sp, #64
; CHECK-FAST-NEXT: ret
; CHECK-DAG: str w2, [sp, #32]
; CHECK-DAG: str xzr, [sp, #24]
; CHECK-DAG: str s0, [sp, #16]
; CHECK-DAG: str xzr, [sp, #8]
; CHECK-DAG: str [[SUM]], [sp]
; Add them to ensure real promotion occurs. ; Add them to ensure real promotion occurs.
%sum = add i8 %l, %r %sum = add i8 %l, %r
Context not available.
declare ptr @llvm.frameaddress(i32) declare ptr @llvm.frameaddress(i32)
define ptr @test_frameaddr() { define ptr @test_frameaddr() {
; CHECK-LABEL: test_frameaddr: ; CHECK-OPT-LABEL: test_frameaddr:
; CHECK-OPT: ldr x0, [x29] ; CHECK-OPT: ; %bb.0:
; CHECK-FAST: ldr [[TMP:x[0-9]+]], [x29] ; CHECK-OPT-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-FAST: and x0, [[TMP]], #0xffffffff ; CHECK-OPT-NEXT: mov x29, sp
; CHECK-OPT-NEXT: .cfi_def_cfa w29, 16
; CHECK-OPT-NEXT: .cfi_offset w30, -8
; CHECK-OPT-NEXT: .cfi_offset w29, -16
; CHECK-OPT-NEXT: ldr x0, [x29]
; CHECK-OPT-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-OPT-NEXT: ret
;
; CHECK-FAST-LABEL: test_frameaddr:
; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-FAST-NEXT: mov x29, sp
; CHECK-FAST-NEXT: .cfi_def_cfa w29, 16
; CHECK-FAST-NEXT: .cfi_offset w30, -8
; CHECK-FAST-NEXT: .cfi_offset w29, -16
; CHECK-FAST-NEXT: ldr x8, [x29]
; CHECK-FAST-NEXT: and x0, x8, #0xffffffff
; CHECK-FAST-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-FAST-NEXT: ret
%val = call ptr @llvm.frameaddress(i32 1) %val = call ptr @llvm.frameaddress(i32 1)
ret ptr %val ret ptr %val
} }
Context not available.
declare ptr @llvm.returnaddress(i32) declare ptr @llvm.returnaddress(i32)
define ptr @test_toplevel_returnaddr() { define ptr @test_toplevel_returnaddr() {
; CHECK-LABEL: test_toplevel_returnaddr: ; CHECK-OPT-LABEL: test_toplevel_returnaddr:
; CHECK-OPT: mov x0, x30 ; CHECK-OPT: ; %bb.0:
; CHECK-FAST: and x0, x30, #0xffffffff ; CHECK-OPT-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-OPT-NEXT: .cfi_def_cfa_offset 16
; CHECK-OPT-NEXT: .cfi_offset w30, -8
; CHECK-OPT-NEXT: .cfi_offset w29, -16
; CHECK-OPT-NEXT: hint #7
; CHECK-OPT-NEXT: mov x0, x30
; CHECK-OPT-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-OPT-NEXT: ret
;
; CHECK-FAST-LABEL: test_toplevel_returnaddr:
; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-FAST-NEXT: .cfi_def_cfa_offset 16
; CHECK-FAST-NEXT: .cfi_offset w30, -8
; CHECK-FAST-NEXT: .cfi_offset w29, -16
; CHECK-FAST-NEXT: hint #7
; CHECK-FAST-NEXT: and x0, x30, #0xffffffff
; CHECK-FAST-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-FAST-NEXT: ret
%val = call ptr @llvm.returnaddress(i32 0) %val = call ptr @llvm.returnaddress(i32 0)
ret ptr %val ret ptr %val
} }
define ptr @test_deep_returnaddr() { define ptr @test_deep_returnaddr() {
; CHECK-LABEL: test_deep_returnaddr: ; CHECK-OPT-LABEL: test_deep_returnaddr:
; CHECK: ldr x[[FRAME_REC:[0-9]+]], [x29] ; CHECK-OPT: ; %bb.0:
; CHECK-OPT: ldr x30, [x[[FRAME_REC]], #8] ; CHECK-OPT-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-OPT: hint #7 ; CHECK-OPT-NEXT: mov x29, sp
; CHECK-OPT: mov x0, x30 ; CHECK-OPT-NEXT: .cfi_def_cfa w29, 16
; CHECK-FAST: ldr [[TMP:x[0-9]+]], [x[[FRAME_REC]], #8] ; CHECK-OPT-NEXT: .cfi_offset w30, -8
; CHECK-FAST: and x0, [[TMP]], #0xffffffff ; CHECK-OPT-NEXT: .cfi_offset w29, -16
; CHECK-OPT-NEXT: ldr x8, [x29]
; CHECK-OPT-NEXT: ldr x30, [x8, #8]
; CHECK-OPT-NEXT: hint #7
; CHECK-OPT-NEXT: mov x0, x30
; CHECK-OPT-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-OPT-NEXT: ret
;
; CHECK-FAST-LABEL: test_deep_returnaddr:
; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-FAST-NEXT: mov x29, sp
; CHECK-FAST-NEXT: .cfi_def_cfa w29, 16
; CHECK-FAST-NEXT: .cfi_offset w30, -8
; CHECK-FAST-NEXT: .cfi_offset w29, -16
; CHECK-FAST-NEXT: ldr x8, [x29]
; CHECK-FAST-NEXT: ldr x30, [x8, #8]
; CHECK-FAST-NEXT: hint #7
; CHECK-FAST-NEXT: and x0, x30, #0xffffffff
; CHECK-FAST-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-FAST-NEXT: ret
%val = call ptr @llvm.returnaddress(i32 1) %val = call ptr @llvm.returnaddress(i32 1)
ret ptr %val ret ptr %val
} }
define void @test_indirect_call(ptr %func) { define void @test_indirect_call(ptr %func) {
; CHECK-LABEL: test_indirect_call: ; CHECK-LABEL: test_indirect_call:
; CHECK: blr x0 ; CHECK: ; %bb.0:
; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: blr x0
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
call void() %func() call void() %func()
ret void ret void
} }
Context not available.
; Safe to use the unextended address here ; Safe to use the unextended address here
define void @test_indirect_safe_call(ptr %weird_funcs) { define void @test_indirect_safe_call(ptr %weird_funcs) {
; CHECK-LABEL: test_indirect_safe_call: ; CHECK-LABEL: test_indirect_safe_call:
; CHECK: add w[[ADDR32:[0-9]+]], w0, #4 ; CHECK: ; %bb.0:
; CHECK-OPT-NOT: ubfx ; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK: blr x[[ADDR32]] ; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: add w8, w0, #4
; CHECK-NEXT: blr x8
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
%addr = getelementptr i32, ptr %weird_funcs, i32 1 %addr = getelementptr i32, ptr %weird_funcs, i32 1
call void() %addr() call void() %addr()
ret void ret void
Context not available.
declare void @simple() declare void @simple()
define void @test_simple_tail_call() { define void @test_simple_tail_call() {
; CHECK-LABEL: test_simple_tail_call: ; CHECK-LABEL: test_simple_tail_call:
; CHECK: b _simple ; CHECK: ; %bb.0:
; CHECK-NEXT: b _simple
tail call void @simple() tail call void @simple()
ret void ret void
} }
define void @test_indirect_tail_call(ptr %func) { define void @test_indirect_tail_call(ptr %func) {
; CHECK-LABEL: test_indirect_tail_call: ; CHECK-LABEL: test_indirect_tail_call:
; CHECK: br x0 ; CHECK: ; %bb.0:
; CHECK-NEXT: br x0
tail call void() %func() tail call void() %func()
ret void ret void
} }
Context not available.
; Safe to use the unextended address here ; Safe to use the unextended address here
define void @test_indirect_safe_tail_call(ptr %weird_funcs) { define void @test_indirect_safe_tail_call(ptr %weird_funcs) {
; CHECK-LABEL: test_indirect_safe_tail_call: ; CHECK-LABEL: test_indirect_safe_tail_call:
; CHECK: add w[[ADDR32:[0-9]+]], w0, #4 ; CHECK: ; %bb.0:
; CHECK-OPT-NOT: ubfx ; CHECK-NEXT: add w0, w0, #4
; CHECK-OPT: br x[[ADDR32]] ; CHECK-NEXT: br x0
%addr = getelementptr i32, ptr %weird_funcs, i32 1 %addr = getelementptr i32, ptr %weird_funcs, i32 1
tail call void() %addr() tail call void() %addr()
ret void ret void
Context not available.
define i32 @test_in_smallstruct_low([3 x i32] %in) { define i32 @test_in_smallstruct_low([3 x i32] %in) {
; CHECK-LABEL: test_in_smallstruct_low: ; CHECK-LABEL: test_in_smallstruct_low:
; CHECK: mov x0, x1 ; CHECK: ; %bb.0:
; CHECK-NEXT: mov x0, x1
; CHECK-NEXT: ; kill: def $w0 killed $w0 killed $x0
; CHECK-NEXT: ret
%val = extractvalue [3 x i32] %in, 2 %val = extractvalue [3 x i32] %in, 2
ret i32 %val ret i32 %val
} }
define i32 @test_in_smallstruct_high([3 x i32] %in) { define i32 @test_in_smallstruct_high([3 x i32] %in) {
; CHECK-LABEL: test_in_smallstruct_high: ; CHECK-LABEL: test_in_smallstruct_high:
; CHECK: lsr x0, x0, #32 ; CHECK: ; %bb.0:
; CHECK-NEXT: lsr x0, x0, #32
; CHECK-NEXT: ; kill: def $w0 killed $w0 killed $x0
; CHECK-NEXT: ret
%val = extractvalue [3 x i32] %in, 1 %val = extractvalue [3 x i32] %in, 1
ret i32 %val ret i32 %val
} }
Context not available.
; be incompatible with the armv7k ABI. ; be incompatible with the armv7k ABI.
define i32 @test_in_smallstruct_stack([8 x i64], i32, [3 x i32] %in) { define i32 @test_in_smallstruct_stack([8 x i64], i32, [3 x i32] %in) {
; CHECK-LABEL: test_in_smallstruct_stack: ; CHECK-LABEL: test_in_smallstruct_stack:
; CHECK: ldr w0, [sp, #4] ; CHECK: ; %bb.0:
; CHECK-NEXT: ldr w0, [sp, #4]
; CHECK-NEXT: ret
%val = extractvalue [3 x i32] %in, 0 %val = extractvalue [3 x i32] %in, 0
ret i32 %val ret i32 %val
} }
define [2 x i32] @test_ret_smallstruct([3 x i32] %in) { define [2 x i32] @test_ret_smallstruct([3 x i32] %in) {
; CHECK-LABEL: test_ret_smallstruct: ; CHECK-LABEL: test_ret_smallstruct:
; CHECK: mov x0, #1 ; CHECK: ; %bb.0:
; CHECK: movk x0, #2, lsl #32 ; CHECK-NEXT: mov x0, #1 ; =0x1
; CHECK-NEXT: movk x0, #2, lsl #32
; CHECK-NEXT: ret
ret [2 x i32] [i32 1, i32 2] ret [2 x i32] [i32 1, i32 2]
} }
Context not available.
declare void @smallstruct_callee([4 x i32]) declare void @smallstruct_callee([4 x i32])
define void @test_call_smallstruct() { define void @test_call_smallstruct() {
; CHECK-LABEL: test_call_smallstruct: ; CHECK-LABEL: test_call_smallstruct:
; CHECK: mov x0, #1 ; CHECK: ; %bb.0:
; CHECK: movk x0, #2, lsl #32 ; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK: mov x1, #3 ; CHECK-NEXT: mov x29, sp
; CHECK: movk x1, #4, lsl #32 ; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK: bl _smallstruct_callee ; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: mov x0, #1 ; =0x1
; CHECK-NEXT: movk x0, #2, lsl #32
; CHECK-NEXT: mov x1, #3 ; =0x3
; CHECK-NEXT: movk x1, #4, lsl #32
; CHECK-NEXT: bl _smallstruct_callee
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
call void @smallstruct_callee([4 x i32] [i32 1, i32 2, i32 3, i32 4]) call void @smallstruct_callee([4 x i32] [i32 1, i32 2, i32 3, i32 4])
ret void ret void
Context not available.
declare void @smallstruct_callee_stack([8 x i64], i32, [2 x i32]) declare void @smallstruct_callee_stack([8 x i64], i32, [2 x i32])
define void @test_call_smallstruct_stack() { define void @test_call_smallstruct_stack() {
; CHECK-LABEL: test_call_smallstruct_stack: ; CHECK-LABEL: test_call_smallstruct_stack:
; CHECK: mov [[VAL:x[0-9]+]], #1 ; CHECK: ; %bb.0:
; CHECK: movk [[VAL]], #2, lsl #32 ; CHECK-NEXT: sub sp, sp, #32
; CHECK: stur [[VAL]], [sp, #4] ; CHECK-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-NEXT: add x29, sp, #16
; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: mov x8, #1 ; =0x1
; CHECK-NEXT: movk x8, #2, lsl #32
; CHECK-NEXT: stur x8, [sp, #4]
; CHECK-NEXT: bl _smallstruct_callee_stack
; CHECK-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-NEXT: add sp, sp, #32
; CHECK-NEXT: ret
call void @smallstruct_callee_stack([8 x i64] undef, i32 undef, [2 x i32] [i32 1, i32 2]) call void @smallstruct_callee_stack([8 x i64] undef, i32 undef, [2 x i32] [i32 1, i32 2])
ret void ret void
Context not available.
declare [3 x i32] @returns_smallstruct() declare [3 x i32] @returns_smallstruct()
define i32 @test_use_smallstruct_low() { define i32 @test_use_smallstruct_low() {
; CHECK-LABEL: test_use_smallstruct_low: ; CHECK-LABEL: test_use_smallstruct_low:
; CHECK: bl _returns_smallstruct ; CHECK: ; %bb.0:
; CHECK: mov x0, x1 ; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: bl _returns_smallstruct
; CHECK-NEXT: mov x0, x1
; CHECK-NEXT: ; kill: def $w0 killed $w0 killed $x0
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
%struct = call [3 x i32] @returns_smallstruct() %struct = call [3 x i32] @returns_smallstruct()
%val = extractvalue [3 x i32] %struct, 2 %val = extractvalue [3 x i32] %struct, 2
Context not available.
define i32 @test_use_smallstruct_high() { define i32 @test_use_smallstruct_high() {
; CHECK-LABEL: test_use_smallstruct_high: ; CHECK-LABEL: test_use_smallstruct_high:
; CHECK: bl _returns_smallstruct ; CHECK: ; %bb.0:
; CHECK: lsr x0, x0, #32 ; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: bl _returns_smallstruct
; CHECK-NEXT: lsr x0, x0, #32
; CHECK-NEXT: ; kill: def $w0 killed $w0 killed $x0
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
%struct = call [3 x i32] @returns_smallstruct() %struct = call [3 x i32] @returns_smallstruct()
%val = extractvalue [3 x i32] %struct, 1 %val = extractvalue [3 x i32] %struct, 1
Context not available.
; be marked as unavailable and subsequent GPR arguments should also be on the ; be marked as unavailable and subsequent GPR arguments should also be on the
; stack. Obviously the struct itself should be passed entirely on the stack. ; stack. Obviously the struct itself should be passed entirely on the stack.
define i32 @test_smallstruct_padding([7 x i64], [4 x i32] %struct, i32 %in) { define i32 @test_smallstruct_padding([7 x i64], [4 x i32] %struct, i32 %in) {
; CHECK-LABEL: test_smallstruct_padding: ; CHECK-OPT-LABEL: test_smallstruct_padding:
; CHECK-DAG: ldr [[IN:w[0-9]+]], [sp, #16] ; CHECK-OPT: ; %bb.0:
; CHECK-DAG: ldr [[LHS:w[0-9]+]], [sp] ; CHECK-OPT-NEXT: ldr w8, [sp, #16]
; CHECK: add w0, [[LHS]], [[IN]] ; CHECK-OPT-NEXT: ldr w9, [sp]
; CHECK-OPT-NEXT: add w0, w9, w8
; CHECK-OPT-NEXT: ret
;
; CHECK-FAST-LABEL: test_smallstruct_padding:
; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: ldr w8, [sp]
; CHECK-FAST-NEXT: ldr w9, [sp, #16]
; CHECK-FAST-NEXT: add w0, w8, w9
; CHECK-FAST-NEXT: ret
%lhs = extractvalue [4 x i32] %struct, 0 %lhs = extractvalue [4 x i32] %struct, 0
%sum = add i32 %lhs, %in %sum = add i32 %lhs, %in
ret i32 %sum ret i32 %sum
Context not available.
declare void @take_small_smallstruct(i64, [1 x i32]) declare void @take_small_smallstruct(i64, [1 x i32])
define void @test_small_smallstruct() { define void @test_small_smallstruct() {
; CHECK-LABEL: test_small_smallstruct: ; CHECK-LABEL: test_small_smallstruct:
; CHECK-DAG: mov w0, #1 ; CHECK: ; %bb.0:
; CHECK-DAG: mov w1, #2 ; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK: bl _take_small_smallstruct ; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: mov w0, #1 ; =0x1
; CHECK-NEXT: mov w1, #2 ; =0x2
; CHECK-NEXT: bl _take_small_smallstruct
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
call void @take_small_smallstruct(i64 1, [1 x i32] [i32 2]) call void @take_small_smallstruct(i64 1, [1 x i32] [i32 2])
ret void ret void
} }
define void @test_bare_frameaddr(ptr %addr) { define void @test_bare_frameaddr(ptr %addr) {
; CHECK-LABEL: test_bare_frameaddr: ; CHECK-LABEL: test_bare_frameaddr:
; CHECK: add x[[LOCAL:[0-9]+]], sp, #{{[0-9]+}} ; CHECK: ; %bb.0:
; CHECK: str w[[LOCAL]], ; CHECK-NEXT: sub sp, sp, #16
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: add x8, sp, #15
; CHECK-NEXT: str w8, [x0]
; CHECK-NEXT: add sp, sp, #16
; CHECK-NEXT: ret
%ptr = alloca i8 %ptr = alloca i8
store ptr %ptr, ptr %addr, align 4 store ptr %ptr, ptr %addr, align 4
Context not available.
define void @test_sret_use(ptr sret([8 x i64]) %out) { define void @test_sret_use(ptr sret([8 x i64]) %out) {
; CHECK-LABEL: test_sret_use: ; CHECK-LABEL: test_sret_use:
; CHECK: str xzr, [x8] ; CHECK: ; %bb.0:
; CHECK-NEXT: str xzr, [x8]
; CHECK-NEXT: ret
store i64 0, ptr %out store i64 0, ptr %out
ret void ret void
} }
define i64 @test_sret_call() { define i64 @test_sret_call() {
; CHECK-LABEL: test_sret_call: ; CHECK-LABEL: test_sret_call:
; CHECK: mov x8, sp ; CHECK: ; %bb.0:
; CHECK: bl _test_sret_use ; CHECK-NEXT: sub sp, sp, #80
; CHECK-NEXT: stp x29, x30, [sp, #64] ; 16-byte Folded Spill
; CHECK-NEXT: add x29, sp, #64
; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: mov x8, sp
; CHECK-NEXT: bl _test_sret_use
; CHECK-NEXT: ldr x0, [sp]
; CHECK-NEXT: ldp x29, x30, [sp, #64] ; 16-byte Folded Reload
; CHECK-NEXT: add sp, sp, #80
; CHECK-NEXT: ret
%arr = alloca [8 x i64] %arr = alloca [8 x i64]
call void @test_sret_use(ptr sret([8 x i64]) %arr) call void @test_sret_use(ptr sret([8 x i64]) %arr)
Context not available.
define double @test_constpool() { define double @test_constpool() {
; CHECK-LABEL: test_constpool: ; CHECK-LABEL: test_constpool:
; CHECK: adrp x[[PAGE:[0-9]+]], [[POOL:lCPI[0-9]+_[0-9]+]]@PAGE ; CHECK: ; %bb.0:
; CHECK: ldr d0, [x[[PAGE]], [[POOL]]@PAGEOFF] ; CHECK-NEXT: Lloh22:
; CHECK-NEXT: adrp x8, lCPI37_0@PAGE
; CHECK-NEXT: Lloh23:
; CHECK-NEXT: ldr d0, [x8, lCPI37_0@PAGEOFF]
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh22, Lloh23
ret double 1.0e-6 ret double 1.0e-6
} }
define ptr @test_blockaddress() { define ptr @test_blockaddress() {
; CHECK-LABEL: test_blockaddress: ; CHECK-LABEL: test_blockaddress:
; CHECK: [[BLOCK:Ltmp[0-9]+]]: ; CHECK: ; %bb.0:
; CHECK: adrp x[[PAGE:[0-9]+]], lCPI{{[0-9]+_[0-9]+}}@PAGE ; CHECK-NEXT: Ltmp7: ; Block address taken
; CHECK: ldr x0, [x[[PAGE]], lCPI{{[0-9]+_[0-9]+}}@PAGEOFF] ; CHECK-NEXT: ; %bb.1: ; %dest
; CHECK-NEXT: Lloh24:
; CHECK-NEXT: adrp x0, lCPI38_0@PAGE
; CHECK-NEXT: Lloh25:
; CHECK-NEXT: ldr x0, [x0, lCPI38_0@PAGEOFF]
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh24, Lloh25
br label %dest br label %dest
dest: dest:
ret ptr blockaddress(@test_blockaddress, %dest) ret ptr blockaddress(@test_blockaddress, %dest)
Context not available.
define ptr @test_indirectbr(ptr %dest) { define ptr @test_indirectbr(ptr %dest) {
; CHECK-LABEL: test_indirectbr: ; CHECK-LABEL: test_indirectbr:
; CHECK: br x0 ; CHECK: ; %bb.0:
; CHECK-NEXT: br x0
; CHECK-NEXT: Ltmp8: ; Block address taken
; CHECK-NEXT: LBB39_1: ; %true
; CHECK-NEXT: Lloh26:
; CHECK-NEXT: adrp x0, lCPI39_0@PAGE
; CHECK-NEXT: Lloh27:
; CHECK-NEXT: ldr x0, [x0, lCPI39_0@PAGEOFF]
; CHECK-NEXT: ret
; CHECK-NEXT: Ltmp9: ; Block address taken
; CHECK-NEXT: LBB39_2: ; %false
; CHECK-NEXT: Lloh28:
; CHECK-NEXT: adrp x0, lCPI39_1@PAGE
; CHECK-NEXT: Lloh29:
; CHECK-NEXT: ldr x0, [x0, lCPI39_1@PAGEOFF]
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh26, Lloh27
; CHECK-NEXT: .loh AdrpLdr Lloh28, Lloh29
indirectbr ptr %dest, [label %true, label %false] indirectbr ptr %dest, [label %true, label %false]
true: true:
Context not available.
; claim the FI in the process -- it doesn't need extending. ; claim the FI in the process -- it doesn't need extending.
define float @test_frameindex_offset_load() { define float @test_frameindex_offset_load() {
; CHECK-LABEL: test_frameindex_offset_load: ; CHECK-LABEL: test_frameindex_offset_load:
; CHECK: ldr s0, [sp, #4] ; CHECK: ; %bb.0:
; CHECK-NEXT: sub sp, sp, #16
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: ldr s0, [sp, #4]
; CHECK-NEXT: add sp, sp, #16
; CHECK-NEXT: ret
%arr = alloca float, i32 4, align 8 %arr = alloca float, i32 4, align 8
%addr = getelementptr inbounds float, ptr %arr, i32 1 %addr = getelementptr inbounds float, ptr %arr, i32 1
Context not available.
define void @test_unaligned_frameindex_offset_store() { define void @test_unaligned_frameindex_offset_store() {
; CHECK-LABEL: test_unaligned_frameindex_offset_store: ; CHECK-LABEL: test_unaligned_frameindex_offset_store:
; CHECK: mov x[[TMP:[0-9]+]], sp ; CHECK: ; %bb.0:
; CHECK: orr w[[ADDR:[0-9]+]], w[[TMP]], #0x2 ; CHECK-NEXT: sub sp, sp, #16
; CHECK: mov [[VAL:w[0-9]+]], #42 ; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK: str [[VAL]], [x[[ADDR]]] ; CHECK-NEXT: mov x8, sp
; CHECK-NEXT: orr w8, w8, #0x2
; CHECK-NEXT: mov w9, #42 ; =0x2a
; CHECK-NEXT: str w9, [x8]
; CHECK-NEXT: add sp, sp, #16
; CHECK-NEXT: ret
%arr = alloca [4 x i32] %arr = alloca [4 x i32]
%addr.int = ptrtoint ptr %arr to i32 %addr.int = ptrtoint ptr %arr to i32
Context not available.
define {i64, ptr} @test_pre_idx(ptr %addr) { define {i64, ptr} @test_pre_idx(ptr %addr) {
; CHECK-LABEL: test_pre_idx: ; CHECK-LABEL: test_pre_idx:
; CHECK: ; %bb.0:
; CHECK-NEXT: add w1, w0, #8
; CHECK-NEXT: ldr x0, [x1]
; CHECK-NEXT: ret
; CHECK: add w[[ADDR:[0-9]+]], w0, #8
; CHECK: ldr x0, [x[[ADDR]]]
%addr.int = ptrtoint ptr %addr to i32 %addr.int = ptrtoint ptr %addr to i32
%addr.next.int = add nuw i32 %addr.int, 8 %addr.next.int = add nuw i32 %addr.int, 8
%addr.next = inttoptr i32 %addr.next.int to ptr %addr.next = inttoptr i32 %addr.next.int to ptr
Context not available.
; %addr wraps round to 0. ; %addr wraps round to 0.
define {i64, ptr} @test_invalid_pre_idx(ptr %addr) { define {i64, ptr} @test_invalid_pre_idx(ptr %addr) {
; CHECK-LABEL: test_invalid_pre_idx: ; CHECK-LABEL: test_invalid_pre_idx:
; CHECK: add w1, w0, #8 ; CHECK: ; %bb.0:
; CHECK: ldr x0, [x1] ; CHECK-NEXT: add w1, w0, #8
; CHECK-NEXT: ldr x0, [x1]
; CHECK-NEXT: ret
%addr.next = getelementptr i64, ptr %addr, i32 1 %addr.next = getelementptr i64, ptr %addr, i32 1
%val = load i64, ptr %addr.next %val = load i64, ptr %addr.next
Context not available.
declare void @callee(ptr) declare void @callee(ptr)
define void @test_stack_guard() ssp { define void @test_stack_guard() ssp {
; CHECK-LABEL: test_stack_guard: ; CHECK-OPT-LABEL: test_stack_guard:
; CHECK: adrp x[[GUARD_GOTPAGE:[0-9]+]], ___stack_chk_guard@GOTPAGE ; CHECK-OPT: ; %bb.0:
; CHECK: ldr w[[GUARD_ADDR:[0-9]+]], [x[[GUARD_GOTPAGE]], ___stack_chk_guard@GOTPAGEOFF] ; CHECK-OPT-NEXT: sub sp, sp, #64
; CHECK: ldr [[GUARD_VAL:w[0-9]+]], [x[[GUARD_ADDR]]] ; CHECK-OPT-NEXT: stp x29, x30, [sp, #48] ; 16-byte Folded Spill
; CHECK: stur [[GUARD_VAL]], [x29, #[[GUARD_OFFSET:-[0-9]+]]] ; CHECK-OPT-NEXT: add x29, sp, #48
; CHECK-OPT-NEXT: .cfi_def_cfa w29, 16
; CHECK: add x0, sp, #{{[0-9]+}} ; CHECK-OPT-NEXT: .cfi_offset w30, -8
; CHECK: bl _callee ; CHECK-OPT-NEXT: .cfi_offset w29, -16
; CHECK-OPT-NEXT: Lloh30:
; CHECK-OPT: adrp x[[GUARD_GOTPAGE:[0-9]+]], ___stack_chk_guard@GOTPAGE ; CHECK-OPT-NEXT: adrp x8, ___stack_chk_guard@GOTPAGE
; CHECK-OPT: ldr w[[GUARD_ADDR:[0-9]+]], [x[[GUARD_GOTPAGE]], ___stack_chk_guard@GOTPAGEOFF] ; CHECK-OPT-NEXT: Lloh31:
; CHECK-OPT: ldr [[GUARD_VAL:w[0-9]+]], [x[[GUARD_ADDR]]] ; CHECK-OPT-NEXT: ldr w8, [x8, ___stack_chk_guard@GOTPAGEOFF]
; CHECK-OPT: ldur [[NEW_VAL:w[0-9]+]], [x29, #[[GUARD_OFFSET]]] ; CHECK-OPT-NEXT: Lloh32:
; CHECK-OPT: cmp [[GUARD_VAL]], [[NEW_VAL]] ; CHECK-OPT-NEXT: ldr w8, [x8]
; CHECK-OPT: b.ne [[FAIL:LBB[0-9]+_[0-9]+]] ; CHECK-OPT-NEXT: stur w8, [x29, #-4]
; CHECK-OPT-NEXT: add x0, sp, #12
; CHECK-OPT: [[FAIL]]: ; CHECK-OPT-NEXT: bl _callee
; CHECK-OPT-NEXT: bl ___stack_chk_fail ; CHECK-OPT-NEXT: Lloh33:
; CHECK-OPT-NEXT: adrp x8, ___stack_chk_guard@GOTPAGE
; CHECK-OPT-NEXT: Lloh34:
; CHECK-OPT-NEXT: ldr w8, [x8, ___stack_chk_guard@GOTPAGEOFF]
; CHECK-OPT-NEXT: Lloh35:
; CHECK-OPT-NEXT: ldr w8, [x8]
; CHECK-OPT-NEXT: ldur w9, [x29, #-4]
; CHECK-OPT-NEXT: cmp w8, w9
; CHECK-OPT-NEXT: b.ne LBB44_2
; CHECK-OPT-NEXT: ; %bb.1:
; CHECK-OPT-NEXT: ldp x29, x30, [sp, #48] ; 16-byte Folded Reload
; CHECK-OPT-NEXT: add sp, sp, #64
; CHECK-OPT-NEXT: ret
; CHECK-OPT-NEXT: LBB44_2:
; CHECK-OPT-NEXT: bl ___stack_chk_fail
; CHECK-OPT-NEXT: .loh AdrpLdrGotLdr Lloh33, Lloh34, Lloh35
; CHECK-OPT-NEXT: .loh AdrpLdrGotLdr Lloh30, Lloh31, Lloh32
;
; CHECK-FAST-LABEL: test_stack_guard:
; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: sub sp, sp, #64
; CHECK-FAST-NEXT: stp x29, x30, [sp, #48] ; 16-byte Folded Spill
; CHECK-FAST-NEXT: add x29, sp, #48
; CHECK-FAST-NEXT: .cfi_def_cfa w29, 16
; CHECK-FAST-NEXT: .cfi_offset w30, -8
; CHECK-FAST-NEXT: .cfi_offset w29, -16
; CHECK-FAST-NEXT: Lloh30:
; CHECK-FAST-NEXT: adrp x8, ___stack_chk_guard@GOTPAGE
; CHECK-FAST-NEXT: Lloh31:
; CHECK-FAST-NEXT: ldr w8, [x8, ___stack_chk_guard@GOTPAGEOFF]
; CHECK-FAST-NEXT: Lloh32:
; CHECK-FAST-NEXT: ldr w8, [x8]
; CHECK-FAST-NEXT: stur w8, [x29, #-4]
; CHECK-FAST-NEXT: add x0, sp, #12
; CHECK-FAST-NEXT: bl _callee
; CHECK-FAST-NEXT: Lloh33:
; CHECK-FAST-NEXT: adrp x8, ___stack_chk_guard@GOTPAGE
; CHECK-FAST-NEXT: Lloh34:
; CHECK-FAST-NEXT: ldr w8, [x8, ___stack_chk_guard@GOTPAGEOFF]
; CHECK-FAST-NEXT: Lloh35:
; CHECK-FAST-NEXT: ldr w8, [x8]
; CHECK-FAST-NEXT: ldur w9, [x29, #-4]
; CHECK-FAST-NEXT: and x8, x8, #0xffffffff
; CHECK-FAST-NEXT: cmp x8, x9
; CHECK-FAST-NEXT: b.ne LBB44_2
; CHECK-FAST-NEXT: ; %bb.1: ; %SP_return
; CHECK-FAST-NEXT: ldp x29, x30, [sp, #48] ; 16-byte Folded Reload
; CHECK-FAST-NEXT: add sp, sp, #64
; CHECK-FAST-NEXT: ret
; CHECK-FAST-NEXT: LBB44_2: ; %CallStackCheckFailBlk
; CHECK-FAST-NEXT: bl ___stack_chk_fail
; CHECK-FAST-NEXT: .loh AdrpLdrGotLdr Lloh33, Lloh34, Lloh35
; CHECK-FAST-NEXT: .loh AdrpLdrGotLdr Lloh30, Lloh31, Lloh32
%arr = alloca [8 x i32] %arr = alloca [8 x i32]
call void @callee(ptr %arr) call void @callee(ptr %arr)
ret void ret void
Context not available.
@_ZTI8Whatever = external global i8 @_ZTI8Whatever = external global i8
define void @test_landingpad_marshalling() personality ptr @__gxx_personality_v0 { define void @test_landingpad_marshalling() personality ptr @__gxx_personality_v0 {
; CHECK-LABEL: test_landingpad_marshalling: ; CHECK-LABEL: test_landingpad_marshalling:
; CHECK-OPT: mov x2, x1 ; CHECK: Lfunc_begin0:
; CHECK-OPT: mov x1, x0 ; CHECK-NEXT: .cfi_startproc
; CHECK: bl _eat_landingpad_args ; CHECK-NEXT: .cfi_personality 155, ___gxx_personality_v0
; CHECK-NEXT: .cfi_lsda 16, Lexception0
; CHECK-NEXT: ; %bb.0:
; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: Ltmp3:
; CHECK-NEXT: bl _callee
; CHECK-NEXT: Ltmp4:
; CHECK-NEXT: ; %bb.1: ; %done
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
; CHECK-NEXT: LBB45_2: ; %lpad
; CHECK-NEXT: Ltmp5:
; CHECK-NEXT: mov x2, x1
; CHECK-NEXT: mov x1, x0
; CHECK-NEXT: ; kill: def $w2 killed $w2 killed $x2
; CHECK-NEXT: bl _eat_landingpad_args
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
; CHECK-NEXT: Lfunc_end0:
; CHECK-NEXT: .cfi_endproc
; CHECK-NEXT: .section __TEXT,__gcc_except_tab
; CHECK-NEXT: .p2align 2, 0x0
; CHECK-NEXT: GCC_except_table45:
; CHECK-NEXT: Lexception0:
; CHECK-NEXT: .byte 255 ; @LPStart Encoding = omit
; CHECK-NEXT: .byte 155 ; @TType Encoding = indirect pcrel sdata4
; CHECK-NEXT: .uleb128 Lttbase0-Lttbaseref0
; CHECK-NEXT: Lttbaseref0:
; CHECK-NEXT: .byte 1 ; Call site Encoding = uleb128
; CHECK-NEXT: .uleb128 Lcst_end0-Lcst_begin0
; CHECK-NEXT: Lcst_begin0:
; CHECK-NEXT: .uleb128 Ltmp3-Lfunc_begin0 ; >> Call Site 1 <<
; CHECK-NEXT: .uleb128 Ltmp4-Ltmp3 ; Call between Ltmp3 and Ltmp4
; CHECK-NEXT: .uleb128 Ltmp5-Lfunc_begin0 ; jumps to Ltmp5
; CHECK-NEXT: .byte 1 ; On action: 1
; CHECK-NEXT: .uleb128 Ltmp4-Lfunc_begin0 ; >> Call Site 2 <<
; CHECK-NEXT: .uleb128 Lfunc_end0-Ltmp4 ; Call between Ltmp4 and Lfunc_end0
; CHECK-NEXT: .byte 0 ; has no landing pad
; CHECK-NEXT: .byte 0 ; On action: cleanup
; CHECK-NEXT: Lcst_end0:
; CHECK-NEXT: .byte 1 ; >> Action Record 1 <<
; CHECK-NEXT: ; Catch TypeInfo 1
; CHECK-NEXT: .byte 0 ; No further actions
; CHECK-NEXT: .p2align 2, 0x0
; CHECK-NEXT: ; >> Catch TypeInfos <<
; CHECK-NEXT: Ltmp10: ; TypeInfo 1
; CHECK-NEXT: .long __ZTI8Whatever@GOT-Ltmp10
; CHECK-NEXT: Lttbase0:
; CHECK-NEXT: .p2align 2, 0x0
invoke void @callee(ptr undef) to label %done unwind label %lpad invoke void @callee(ptr undef) to label %done unwind label %lpad
lpad: ; preds = %entry lpad: ; preds = %entry
Context not available.
define void @test_dynamic_stackalloc() { define void @test_dynamic_stackalloc() {
; CHECK-LABEL: test_dynamic_stackalloc: ; CHECK-LABEL: test_dynamic_stackalloc:
; CHECK: sub [[REG:x[0-9]+]], sp, #32 ; CHECK: ; %bb.0:
; CHECK: mov sp, [[REG]] ; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-OPT-NOT: ubfx ; CHECK-NEXT: mov x29, sp
; CHECK: bl _callee ; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: sub x0, sp, #32
; CHECK-NEXT: mov sp, x0
; CHECK-NEXT: bl _callee
; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
br label %next br label %next
next: next:
Context not available.
define void @test_asm_memory(ptr %base.addr) { define void @test_asm_memory(ptr %base.addr) {
; CHECK-LABEL: test_asm_memory: ; CHECK-LABEL: test_asm_memory:
; CHECK: add w[[ADDR:[0-9]+]], w0, #4 ; CHECK: ; %bb.0:
; CHECK: str wzr, [x[[ADDR]] ; CHECK-NEXT: add w8, w0, #4
; CHECK-NEXT: ; InlineAsm Start
; CHECK-NEXT: str wzr, [x8]
; CHECK-NEXT: ; InlineAsm End
; CHECK-NEXT: ret
%addr = getelementptr i32, ptr %base.addr, i32 1 %addr = getelementptr i32, ptr %base.addr, i32 1
call void asm sideeffect "str wzr, $0", "*m"(ptr elementtype(i32) %addr) call void asm sideeffect "str wzr, $0", "*m"(ptr elementtype(i32) %addr)
ret void ret void
Context not available.
define void @test_unsafe_asm_memory(i64 %val) { define void @test_unsafe_asm_memory(i64 %val) {
; CHECK-LABEL: test_unsafe_asm_memory: ; CHECK-LABEL: test_unsafe_asm_memory:
; CHECK: and x[[ADDR:[0-9]+]], x0, #0xffffffff ; CHECK: ; %bb.0:
; CHECK: str wzr, [x[[ADDR]]] ; CHECK-NEXT: and x8, x0, #0xffffffff
; CHECK-NEXT: ; InlineAsm Start
; CHECK-NEXT: str wzr, [x8]
; CHECK-NEXT: ; InlineAsm End
; CHECK-NEXT: ret
%addr_int = trunc i64 %val to i32 %addr_int = trunc i64 %val to i32
%addr = inttoptr i32 %addr_int to ptr %addr = inttoptr i32 %addr_int to ptr
call void asm sideeffect "str wzr, $0", "*m"(ptr elementtype(i32) %addr) call void asm sideeffect "str wzr, $0", "*m"(ptr elementtype(i32) %addr)
Context not available.
define [9 x ptr] @test_demoted_return(ptr %in) { define [9 x ptr] @test_demoted_return(ptr %in) {
; CHECK-LABEL: test_demoted_return: ; CHECK-LABEL: test_demoted_return:
; CHECK: str w0, [x8, #32] ; CHECK: ; %bb.0:
; CHECK-NEXT: str w0, [x8, #32]
; CHECK-NEXT: ret
%res = insertvalue [9 x ptr] undef, ptr %in, 8 %res = insertvalue [9 x ptr] undef, ptr %in, 8
ret [9 x ptr] %res ret [9 x ptr] %res
} }
define ptr @test_inttoptr(i64 %in) { define ptr @test_inttoptr(i64 %in) {
; CHECK-LABEL: test_inttoptr: ; CHECK-LABEL: test_inttoptr:
; CHECK: and x0, x0, #0xffffffff ; CHECK: ; %bb.0:
; CHECK-NEXT: and x0, x0, #0xffffffff
; CHECK-NEXT: ret
%res = inttoptr i64 %in to ptr %res = inttoptr i64 %in to ptr
ret ptr %res ret ptr %res
} }
Context not available.
declare i32 @llvm.get.dynamic.area.offset.i32() declare i32 @llvm.get.dynamic.area.offset.i32()
define i32 @test_dynamic_area() { define i32 @test_dynamic_area() {
; CHECK-LABEL: test_dynamic_area: ; CHECK-LABEL: test_dynamic_area:
; CHECK: mov w0, wzr ; CHECK: ; %bb.0:
; CHECK-NEXT: mov w0, wzr
; CHECK-NEXT: ret
%res = call i32 @llvm.get.dynamic.area.offset.i32() %res = call i32 @llvm.get.dynamic.area.offset.i32()
ret i32 %res ret i32 %res
} }
define void @test_pointer_vec_store(ptr %addr) { define void @test_pointer_vec_store(ptr %addr) {
; CHECK-LABEL: test_pointer_vec_store: ; CHECK-LABEL: test_pointer_vec_store:
; CHECK: str xzr, [x0] ; CHECK: ; %bb.0:
; CHECK-NOT: str ; CHECK-NEXT: str xzr, [x0]
; CHECK-NOT: stp ; CHECK-NEXT: ret
store <2 x ptr> zeroinitializer, ptr %addr, align 16 store <2 x ptr> zeroinitializer, ptr %addr, align 16
ret void ret void
Context not available.
define <2 x ptr> @test_pointer_vec_load(ptr %addr) { define <2 x ptr> @test_pointer_vec_load(ptr %addr) {
; CHECK-LABEL: test_pointer_vec_load: ; CHECK-LABEL: test_pointer_vec_load:
; CHECK: ldr d[[TMP:[0-9]+]], [x0] ; CHECK: ; %bb.0:
; CHECK: ushll.2d v0, v[[TMP]], #0 ; CHECK-NEXT: ldr d0, [x0]
; CHECK-NEXT: ushll.2d v0, v0, #0
; CHECK-NEXT: ret
%val = load <2 x ptr>, ptr %addr, align 16 %val = load <2 x ptr>, ptr %addr, align 16
ret <2 x ptr> %val ret <2 x ptr> %val
} }
define void @test_inline_asm_mem_pointer(ptr %in) { define void @test_inline_asm_mem_pointer(ptr %in) {
; CHECK-LABEL: test_inline_asm_mem_pointer: ; CHECK-LABEL: test_inline_asm_mem_pointer:
; CHECK: str w0, ; CHECK: ; %bb.0:
; CHECK-NEXT: sub sp, sp, #16
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: add x8, sp, #12
; CHECK-NEXT: str w0, [sp, #12]
; CHECK-NEXT: ; InlineAsm Start
; CHECK-NEXT: ldr x0, [x8]
; CHECK-NEXT: ; InlineAsm End
; CHECK-NEXT: add sp, sp, #16
; CHECK-NEXT: ret
tail call void asm sideeffect "ldr x0, $0", "rm"(ptr %in) tail call void asm sideeffect "ldr x0, $0", "rm"(ptr %in)
ret void ret void
} }
define void @test_struct_hi(i32 %hi) nounwind { define void @test_struct_hi(i32 %hi) nounwind {
; CHECK-LABEL: test_struct_hi: ; CHECK-OPT-LABEL: test_struct_hi:
; CHECK: mov w[[IN:[0-9]+]], w0 ; CHECK-OPT: ; %bb.0:
; CHECK: bl _get_int ; CHECK-OPT-NEXT: stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
; CHECK-FAST-NEXT: mov w[[DST:[0-9]+]], w0 ; CHECK-OPT-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-FAST-NEXT: orr x0, x[[DST]], x[[IN]], lsl #32 ; CHECK-OPT-NEXT: add x29, sp, #16
; CHECK-OPT-NEXT: bfi x0, x[[IN]], #32, #32 ; CHECK-OPT-NEXT: mov w19, w0
; CHECK-NEXT: bl _take_pair ; CHECK-OPT-NEXT: bl _get_int
; CHECK-OPT-NEXT: bfi x0, x19, #32, #32
; CHECK-OPT-NEXT: bl _take_pair
; CHECK-OPT-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-OPT-NEXT: ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
; CHECK-OPT-NEXT: ret
;
; CHECK-FAST-LABEL: test_struct_hi:
; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: stp x20, x19, [sp, #-32]! ; 16-byte Folded Spill
; CHECK-FAST-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-FAST-NEXT: add x29, sp, #16
; CHECK-FAST-NEXT: mov w19, w0
; CHECK-FAST-NEXT: bl _get_int
; CHECK-FAST-NEXT: mov w8, w0
; CHECK-FAST-NEXT: orr x0, x8, x19, lsl #32
; CHECK-FAST-NEXT: bl _take_pair
; CHECK-FAST-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-FAST-NEXT: ldp x20, x19, [sp], #32 ; 16-byte Folded Reload
; CHECK-FAST-NEXT: ret
%val.64 = call i64 @get_int() %val.64 = call i64 @get_int()
%val.32 = trunc i64 %val.64 to i32 %val.32 = trunc i64 %val.64 to i32
Context not available.
declare i64 @get_int() declare i64 @get_int()
define i1 @test_icmp_ptr(ptr %in) { define i1 @test_icmp_ptr(ptr %in) {
; CHECK-LABEL: test_icmp_ptr ; CHECK-LABEL: test_icmp_ptr:
; CHECK: ubfx x0, x0, #31, #1 ; CHECK: ; %bb.0:
; CHECK-NEXT: ubfx x0, x0, #31, #1
; CHECK-NEXT: ; kill: def $w0 killed $w0 killed $x0
; CHECK-NEXT: ret
%res = icmp slt ptr %in, null %res = icmp slt ptr %in, null
ret i1 %res ret i1 %res
} }
define void @test_multiple_icmp_ptr(ptr %l, ptr %r) { define void @test_multiple_icmp_ptr(ptr %l, ptr %r) {
; CHECK-LABEL: test_multiple_icmp_ptr: ; CHECK-OPT-LABEL: test_multiple_icmp_ptr:
; CHECK: tbnz w0, #31, [[FALSEBB:LBB[0-9]+_[0-9]+]] ; CHECK-OPT: ; %bb.0:
; CHECK: tbnz w1, #31, [[FALSEBB]] ; CHECK-OPT-NEXT: tbnz w0, #31, LBB57_3
; CHECK-OPT-NEXT: ; %bb.1:
; CHECK-OPT-NEXT: tbnz w1, #31, LBB57_3
; CHECK-OPT-NEXT: ; %bb.2: ; %true
; CHECK-OPT-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-OPT-NEXT: mov x29, sp
; CHECK-OPT-NEXT: .cfi_def_cfa w29, 16
; CHECK-OPT-NEXT: .cfi_offset w30, -8
; CHECK-OPT-NEXT: .cfi_offset w29, -16
; CHECK-OPT-NEXT: bl _bar
; CHECK-OPT-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-OPT-NEXT: LBB57_3: ; %false
; CHECK-OPT-NEXT: ret
;
; CHECK-FAST-LABEL: test_multiple_icmp_ptr:
; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: tbnz w0, #31, LBB57_3
; CHECK-FAST-NEXT: ; %bb.1: ; %.cond.split
; CHECK-FAST-NEXT: tbnz w1, #31, LBB57_3
; CHECK-FAST-NEXT: ; %bb.2: ; %true
; CHECK-FAST-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-FAST-NEXT: mov x29, sp
; CHECK-FAST-NEXT: .cfi_def_cfa w29, 16
; CHECK-FAST-NEXT: .cfi_offset w30, -8
; CHECK-FAST-NEXT: .cfi_offset w29, -16
; CHECK-FAST-NEXT: bl _bar
; CHECK-FAST-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-FAST-NEXT: LBB57_3: ; %false
; CHECK-FAST-NEXT: ret
%tst1 = icmp sgt ptr %l, inttoptr (i32 -1 to ptr) %tst1 = icmp sgt ptr %l, inttoptr (i32 -1 to ptr)
%tst2 = icmp sgt ptr %r, inttoptr (i32 -1 to ptr) %tst2 = icmp sgt ptr %r, inttoptr (i32 -1 to ptr)
%tst = and i1 %tst1, %tst2 %tst = and i1 %tst1, %tst2
Context not available.
} }
define void @test_multiple_icmp_ptr_select(ptr %l, ptr %r) { define void @test_multiple_icmp_ptr_select(ptr %l, ptr %r) {
; CHECK-LABEL: test_multiple_icmp_ptr_select: ; CHECK-OPT-LABEL: test_multiple_icmp_ptr_select:
; CHECK: tbnz w0, #31, [[FALSEBB:LBB[0-9]+_[0-9]+]] ; CHECK-OPT: ; %bb.0:
; CHECK: tbnz w1, #31, [[FALSEBB]] ; CHECK-OPT-NEXT: tbnz w0, #31, LBB58_3
; CHECK-OPT-NEXT: ; %bb.1:
; CHECK-OPT-NEXT: tbnz w1, #31, LBB58_3
; CHECK-OPT-NEXT: ; %bb.2: ; %true
; CHECK-OPT-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-OPT-NEXT: mov x29, sp
; CHECK-OPT-NEXT: .cfi_def_cfa w29, 16
; CHECK-OPT-NEXT: .cfi_offset w30, -8
; CHECK-OPT-NEXT: .cfi_offset w29, -16
; CHECK-OPT-NEXT: bl _bar
; CHECK-OPT-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-OPT-NEXT: LBB58_3: ; %false
; CHECK-OPT-NEXT: ret
;
; CHECK-FAST-LABEL: test_multiple_icmp_ptr_select:
; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: tbnz w0, #31, LBB58_3
; CHECK-FAST-NEXT: ; %bb.1: ; %.cond.split
; CHECK-FAST-NEXT: tbnz w1, #31, LBB58_3
; CHECK-FAST-NEXT: ; %bb.2: ; %true
; CHECK-FAST-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-FAST-NEXT: mov x29, sp
; CHECK-FAST-NEXT: .cfi_def_cfa w29, 16
; CHECK-FAST-NEXT: .cfi_offset w30, -8
; CHECK-FAST-NEXT: .cfi_offset w29, -16
; CHECK-FAST-NEXT: bl _bar
; CHECK-FAST-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-FAST-NEXT: LBB58_3: ; %false
; CHECK-FAST-NEXT: ret
%tst1 = icmp sgt ptr %l, inttoptr (i32 -1 to ptr) %tst1 = icmp sgt ptr %l, inttoptr (i32 -1 to ptr)
%tst2 = icmp sgt ptr %r, inttoptr (i32 -1 to ptr) %tst2 = icmp sgt ptr %r, inttoptr (i32 -1 to ptr)
%tst = select i1 %tst1, i1 %tst2, i1 false %tst = select i1 %tst1, i1 %tst2, i1 false
Context not available.
} }
define ptr @test_gep_nonpow2(ptr %a0, i32 %a1) { define ptr @test_gep_nonpow2(ptr %a0, i32 %a1) {
; CHECK-LABEL: test_gep_nonpow2: ; CHECK-OPT-LABEL: test_gep_nonpow2:
; CHECK-OPT: mov w[[SIZE:[0-9]+]], #18 ; CHECK-OPT: ; %bb.0:
; CHECK-OPT-NEXT: smaddl x0, w1, w[[SIZE]], x0 ; CHECK-OPT-NEXT: mov w8, #18 ; =0x12
; CHECK-OPT-NEXT: ret ; CHECK-OPT-NEXT: smaddl x0, w1, w8, x0
; CHECK-OPT-NEXT: ret
; CHECK-FAST: mov w[[SIZE:[0-9]+]], #18 ;
; CHECK-FAST-NEXT: smaddl [[TMP:x[0-9]+]], w1, w[[SIZE]], x0 ; CHECK-FAST-LABEL: test_gep_nonpow2:
; CHECK-FAST-NEXT: and x0, [[TMP]], #0xffffffff ; CHECK-FAST: ; %bb.0:
; CHECK-FAST-NEXT: ret ; CHECK-FAST-NEXT: mov w8, #18 ; =0x12
; CHECK-FAST-NEXT: smaddl x8, w1, w8, x0
; CHECK-FAST-NEXT: and x0, x8, #0xffffffff
; CHECK-FAST-NEXT: ret
%tmp0 = getelementptr inbounds { [18 x i8] }, ptr %a0, i32 %a1 %tmp0 = getelementptr inbounds { [18 x i8] }, ptr %a0, i32 %a1
ret ptr %tmp0 ret ptr %tmp0
} }
define void @test_memset(i64 %in, i8 %value) { define void @test_memset(i64 %in, i8 %value) {
; CHECK-LABEL: test_memset: ; CHECK-LABEL: test_memset:
; CHECK-DAG: and x8, x0, #0xffffffff ; CHECK: ; %bb.0:
; CHECK-DAG: lsr x2, x0, #32 ; CHECK-NEXT: lsr x2, x0, #32
; CHECK-DAG: mov x0, x8 ; CHECK-NEXT: and x0, x0, #0xffffffff
; CHECK: b _memset ; CHECK-NEXT: ; kill: def $w2 killed $w2 killed $x2
; CHECK-NEXT: b _memset
%ptr.i32 = trunc i64 %in to i32 %ptr.i32 = trunc i64 %in to i32
%size.64 = lshr i64 %in, 32 %size.64 = lshr i64 %in, 32
Context not available.
define void @test_bzero(i64 %in) { define void @test_bzero(i64 %in) {
; CHECK-LABEL: test_bzero: ; CHECK-LABEL: test_bzero:
; CHECK-DAG: lsr x1, x0, #32 ; CHECK: ; %bb.0:
; CHECK-DAG: and x0, x0, #0xffffffff ; CHECK-NEXT: lsr x1, x0, #32
; CHECK: b _bzero ; CHECK-NEXT: and x0, x0, #0xffffffff
; CHECK-NEXT: ; kill: def $w1 killed $w1 killed $x1
; CHECK-NEXT: b _bzero
%ptr.i32 = trunc i64 %in to i32 %ptr.i32 = trunc i64 %in to i32
%size.64 = lshr i64 %in, 32 %size.64 = lshr i64 %in, 32
Context not available.

llvm/test/CodeGen/AArch64/arm64ec-reservedregs.ll

llvm/test/CodeGen/AArch64/atomic-ops-lse.ll

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc -mtriple=aarch64-none-linux-gnu -disable-post-ra -verify-machineinstrs -mattr=+lse < %s | FileCheck %s ; RUN: llc -mtriple=aarch64-none-linux-gnu -disable-post-ra -verify-machineinstrs -mattr=+lse < %s | FileCheck %s
; RUN: llc -mtriple=aarch64-none-linux-gnu -disable-post-ra -verify-machineinstrs -mattr=+lse -mattr=+outline-atomics < %s | FileCheck %s ; RUN: llc -mtriple=aarch64-none-linux-gnu -disable-post-ra -verify-machineinstrs -mattr=+lse -mattr=+outline-atomics < %s | FileCheck %s
; RUN: llc -mtriple=aarch64-none-linux-gnu -disable-post-ra -verify-machineinstrs -mattr=+outline-atomics < %s | FileCheck %s --check-prefix=OUTLINE-ATOMICS ; RUN: llc -mtriple=aarch64-none-linux-gnu -disable-post-ra -verify-machineinstrs -mattr=+outline-atomics < %s | FileCheck %s --check-prefix=OUTLINE-ATOMICS
Context not available.
define dso_local i8 @test_atomic_load_add_i8(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_add_i8(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i8: ; CHECK-LABEL: test_atomic_load_add_i8:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldaddalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i8: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i8:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i8:
dmgreenUnsubmitted
Done
ReplyInline Actions

It looks like ths RUN line for CHECK-REG is trying to check that there are no invalid register formed. You might be able to change it to --check-prefixes=CHECK,CHECK-REG

dmgreen: It looks like ths RUN line for CHECK-REG is trying to check that there are no invalid register…
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldaddalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var8, i8 %offset seq_cst %old = atomicrmw add ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_add_i16(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_add_i16(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i16: ; CHECK-LABEL: test_atomic_load_add_i16:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldaddalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i16: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i16:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i16:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldaddalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var16, i16 %offset seq_cst %old = atomicrmw add ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_add_i32(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_add_i32(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i32: ; CHECK-LABEL: test_atomic_load_add_i32:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldaddal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i32:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldaddal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var32, i32 %offset seq_cst %old = atomicrmw add ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_add_i64(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_add_i64(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i64: ; CHECK-LABEL: test_atomic_load_add_i64:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldaddal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i64:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldaddal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var64, i64 %offset seq_cst %old = atomicrmw add ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_add_i32_noret(i32 %offset) nounwind { define dso_local void @test_atomic_load_add_i32_noret(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i32_noret: ; CHECK-LABEL: test_atomic_load_add_i32_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldaddal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i32_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldaddal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw add ptr @var32, i32 %offset seq_cst atomicrmw add ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_add_i64_noret(i64 %offset) nounwind { define dso_local void @test_atomic_load_add_i64_noret(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i64_noret: ; CHECK-LABEL: test_atomic_load_add_i64_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldaddal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i64_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldaddal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw add ptr @var64, i64 %offset seq_cst atomicrmw add ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_or_i8(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_or_i8(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i8: ; CHECK-LABEL: test_atomic_load_or_i8:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsetalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i8: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i8:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i8:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsetalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var8, i8 %offset seq_cst %old = atomicrmw or ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsetalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_or_i16(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_or_i16(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i16: ; CHECK-LABEL: test_atomic_load_or_i16:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsetalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i16: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i16:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i16:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsetalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var16, i16 %offset seq_cst %old = atomicrmw or ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsetalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_or_i32(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_or_i32(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i32: ; CHECK-LABEL: test_atomic_load_or_i32:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsetal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i32:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsetal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var32, i32 %offset seq_cst %old = atomicrmw or ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsetal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_or_i64(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_or_i64(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i64: ; CHECK-LABEL: test_atomic_load_or_i64:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsetal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i64:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsetal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var64, i64 %offset seq_cst %old = atomicrmw or ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsetal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_or_i32_noret(i32 %offset) nounwind { define dso_local void @test_atomic_load_or_i32_noret(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i32_noret: ; CHECK-LABEL: test_atomic_load_or_i32_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsetal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i32_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsetal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw or ptr @var32, i32 %offset seq_cst atomicrmw or ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsetal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_or_i64_noret(i64 %offset) nounwind { define dso_local void @test_atomic_load_or_i64_noret(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i64_noret: ; CHECK-LABEL: test_atomic_load_or_i64_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsetal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i64_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsetal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw or ptr @var64, i64 %offset seq_cst atomicrmw or ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsetal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_xor_i8(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_xor_i8(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i8: ; CHECK-LABEL: test_atomic_load_xor_i8:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldeoralb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i8: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i8:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i8:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldeoralb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var8, i8 %offset seq_cst %old = atomicrmw xor ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldeoralb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_xor_i16(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_xor_i16(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i16: ; CHECK-LABEL: test_atomic_load_xor_i16:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldeoralh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i16: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i16:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i16:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldeoralh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var16, i16 %offset seq_cst %old = atomicrmw xor ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldeoralh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_xor_i32(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_xor_i32(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i32: ; CHECK-LABEL: test_atomic_load_xor_i32:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldeoral w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i32:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldeoral w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var32, i32 %offset seq_cst %old = atomicrmw xor ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldeoral w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_xor_i64(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_xor_i64(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i64: ; CHECK-LABEL: test_atomic_load_xor_i64:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldeoral x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i64:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldeoral x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var64, i64 %offset seq_cst %old = atomicrmw xor ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldeoral x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_xor_i32_noret(i32 %offset) nounwind { define dso_local void @test_atomic_load_xor_i32_noret(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i32_noret: ; CHECK-LABEL: test_atomic_load_xor_i32_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldeoral w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i32_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldeoral w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xor ptr @var32, i32 %offset seq_cst atomicrmw xor ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldeoral w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_xor_i64_noret(i64 %offset) nounwind { define dso_local void @test_atomic_load_xor_i64_noret(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i64_noret: ; CHECK-LABEL: test_atomic_load_xor_i64_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldeoral x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i64_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldeoral x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xor ptr @var64, i64 %offset seq_cst atomicrmw xor ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldeoral x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_min_i8(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_min_i8(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i8: ; CHECK-LABEL: test_atomic_load_min_i8:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsminalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i8: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i8:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: adrp x9, var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB18_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxtb w8, w10 ; OUTLINE-ATOMICS-NEXT: sxtb w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB18_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i8:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsminalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var8, i8 %offset seq_cst %old = atomicrmw min ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsminalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_min_i16(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_min_i16(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i16: ; CHECK-LABEL: test_atomic_load_min_i16:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsminalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i16: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i16:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: adrp x9, var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB19_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxth w8, w10 ; OUTLINE-ATOMICS-NEXT: sxth w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB19_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i16:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsminalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var16, i16 %offset seq_cst %old = atomicrmw min ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsminalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_min_i32(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_min_i32(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i32: ; CHECK-LABEL: test_atomic_load_min_i32:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsminal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB20_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, le
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB20_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i32:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsminal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var32, i32 %offset seq_cst %old = atomicrmw min ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsminal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_min_i64(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_min_i64(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i64: ; CHECK-LABEL: test_atomic_load_min_i64:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsminal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB21_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, le ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, le
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB21_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i64:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsminal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var64, i64 %offset seq_cst %old = atomicrmw min ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsminal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_min_i32_noret(i32 %offset) nounwind { define dso_local void @test_atomic_load_min_i32_noret(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i32_noret: ; CHECK-LABEL: test_atomic_load_min_i32_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsminal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB22_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, le ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, le
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB22_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i32_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsminal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw min ptr @var32, i32 %offset seq_cst atomicrmw min ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsminal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_min_i64_noret(i64 %offset) nounwind { define dso_local void @test_atomic_load_min_i64_noret(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i64_noret: ; CHECK-LABEL: test_atomic_load_min_i64_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsminal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB23_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, le ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, le
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB23_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i64_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsminal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw min ptr @var64, i64 %offset seq_cst atomicrmw min ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsminal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_umin_i8(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_umin_i8(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i8: ; CHECK-LABEL: test_atomic_load_umin_i8:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: lduminalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i8: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i8:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xff ; OUTLINE-ATOMICS-NEXT: adrp x8, var8
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB24_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, ls
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB24_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i8:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: lduminalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var8, i8 %offset seq_cst %old = atomicrmw umin ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: lduminalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_umin_i16(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_umin_i16(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i16: ; CHECK-LABEL: test_atomic_load_umin_i16:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: lduminalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i16: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i16:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xffff ; OUTLINE-ATOMICS-NEXT: adrp x8, var16
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xffff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB25_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, ls
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB25_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i16:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: lduminalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var16, i16 %offset seq_cst %old = atomicrmw umin ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: lduminalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_umin_i32(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_umin_i32(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i32: ; CHECK-LABEL: test_atomic_load_umin_i32:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: lduminal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB26_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB26_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i32:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: lduminal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var32, i32 %offset seq_cst %old = atomicrmw umin ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: lduminal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_umin_i64(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_umin_i64(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i64: ; CHECK-LABEL: test_atomic_load_umin_i64:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: lduminal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB27_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, ls ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB27_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i64:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: lduminal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var64, i64 %offset seq_cst %old = atomicrmw umin ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: lduminal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_umin_i32_noret(i32 %offset) nounwind { define dso_local void @test_atomic_load_umin_i32_noret(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i32_noret: ; CHECK-LABEL: test_atomic_load_umin_i32_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: lduminal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB28_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, ls ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB28_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i32_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: lduminal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umin ptr @var32, i32 %offset seq_cst atomicrmw umin ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: lduminal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_umin_i64_noret(i64 %offset) nounwind { define dso_local void @test_atomic_load_umin_i64_noret(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i64_noret: ; CHECK-LABEL: test_atomic_load_umin_i64_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: lduminal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB29_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, ls ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB29_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i64_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: lduminal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umin ptr @var64, i64 %offset seq_cst atomicrmw umin ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: lduminal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_max_i8(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_max_i8(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i8: ; CHECK-LABEL: test_atomic_load_max_i8:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsmaxalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i8: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i8:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: adrp x9, var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB30_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxtb w8, w10 ; OUTLINE-ATOMICS-NEXT: sxtb w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB30_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i8:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsmaxalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var8, i8 %offset seq_cst %old = atomicrmw max ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsmaxalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_max_i16(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_max_i16(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i16: ; CHECK-LABEL: test_atomic_load_max_i16:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsmaxalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i16: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i16:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: adrp x9, var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB31_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxth w8, w10 ; OUTLINE-ATOMICS-NEXT: sxth w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB31_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i16:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsmaxalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var16, i16 %offset seq_cst %old = atomicrmw max ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsmaxalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_max_i32(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_max_i32(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i32: ; CHECK-LABEL: test_atomic_load_max_i32:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmaxal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB32_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB32_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i32:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmaxal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var32, i32 %offset seq_cst %old = atomicrmw max ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmaxal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_max_i64(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_max_i64(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i64: ; CHECK-LABEL: test_atomic_load_max_i64:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmaxal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB33_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, gt ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB33_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i64:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmaxal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var64, i64 %offset seq_cst %old = atomicrmw max ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmaxal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_max_i32_noret(i32 %offset) nounwind { define dso_local void @test_atomic_load_max_i32_noret(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i32_noret: ; CHECK-LABEL: test_atomic_load_max_i32_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmaxal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB34_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB34_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i32_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmaxal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw max ptr @var32, i32 %offset seq_cst atomicrmw max ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmaxal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_max_i64_noret(i64 %offset) nounwind { define dso_local void @test_atomic_load_max_i64_noret(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i64_noret: ; CHECK-LABEL: test_atomic_load_max_i64_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmaxal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB35_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, gt ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB35_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i64_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmaxal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw max ptr @var64, i64 %offset seq_cst atomicrmw max ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmaxal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_umax_i8(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_umax_i8(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i8: ; CHECK-LABEL: test_atomic_load_umax_i8:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldumaxalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i8: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i8:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xff ; OUTLINE-ATOMICS-NEXT: adrp x8, var8
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB36_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, hi
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB36_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i8:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldumaxalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var8, i8 %offset seq_cst %old = atomicrmw umax ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldumaxalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_umax_i16(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_umax_i16(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i16: ; CHECK-LABEL: test_atomic_load_umax_i16:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldumaxalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i16: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i16:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xffff ; OUTLINE-ATOMICS-NEXT: adrp x8, var16
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xffff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB37_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, hi
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB37_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i16:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldumaxalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var16, i16 %offset seq_cst %old = atomicrmw umax ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldumaxalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_umax_i32(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_umax_i32(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i32: ; CHECK-LABEL: test_atomic_load_umax_i32:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumaxal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB38_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB38_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i32:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumaxal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var32, i32 %offset seq_cst %old = atomicrmw umax ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumaxal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_umax_i64(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_umax_i64(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i64: ; CHECK-LABEL: test_atomic_load_umax_i64:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumaxal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB39_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, hi ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB39_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i64:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumaxal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var64, i64 %offset seq_cst %old = atomicrmw umax ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumaxal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_umax_i32_noret(i32 %offset) nounwind { define dso_local void @test_atomic_load_umax_i32_noret(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i32_noret: ; CHECK-LABEL: test_atomic_load_umax_i32_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumaxal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB40_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, hi ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB40_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i32_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumaxal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umax ptr @var32, i32 %offset seq_cst atomicrmw umax ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumaxal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_umax_i64_noret(i64 %offset) nounwind { define dso_local void @test_atomic_load_umax_i64_noret(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i64_noret: ; CHECK-LABEL: test_atomic_load_umax_i64_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumaxal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB41_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, hi ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB41_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i64_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumaxal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umax ptr @var64, i64 %offset seq_cst atomicrmw umax ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumaxal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_xchg_i8(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_xchg_i8(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i8: ; CHECK-LABEL: test_atomic_load_xchg_i8:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: swpalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i8: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i8:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i8:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: swpalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var8, i8 %offset seq_cst %old = atomicrmw xchg ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: swpalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_xchg_i16(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_xchg_i16(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i16: ; CHECK-LABEL: test_atomic_load_xchg_i16:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: swpalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i16: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i16:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i16:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: swpalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var16, i16 %offset seq_cst %old = atomicrmw xchg ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: swpalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_xchg_i32(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_xchg_i32(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i32: ; CHECK-LABEL: test_atomic_load_xchg_i32:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: swpal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i32:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: swpal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var32, i32 %offset seq_cst %old = atomicrmw xchg ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: swpal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_xchg_i64(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_xchg_i64(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i64: ; CHECK-LABEL: test_atomic_load_xchg_i64:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: swpal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i64:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: swpal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var64, i64 %offset seq_cst %old = atomicrmw xchg ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: swpal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_xchg_i32_noret(i32 %offset) nounwind { define dso_local void @test_atomic_load_xchg_i32_noret(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i32_noret: ; CHECK-LABEL: test_atomic_load_xchg_i32_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: swpal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i32_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: swpal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xchg ptr @var32, i32 %offset seq_cst atomicrmw xchg ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: swpal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_xchg_i64_noret(i64 %offset) nounwind { define dso_local void @test_atomic_load_xchg_i64_noret(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i64_noret: ; CHECK-LABEL: test_atomic_load_xchg_i64_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: swpal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i64_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: swpal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xchg ptr @var64, i64 %offset seq_cst atomicrmw xchg ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: swpal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_cmpxchg_i8(i8 %wanted, i8 %new) nounwind { define dso_local i8 @test_atomic_cmpxchg_i8(i8 %wanted, i8 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i8: ; CHECK-LABEL: test_atomic_cmpxchg_i8:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: casab w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i8: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i8:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas1_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas1_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i8:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: casab w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var8, i8 %wanted, i8 %new acquire acquire %pair = cmpxchg ptr @var8, i8 %wanted, i8 %new acquire acquire
%old = extractvalue { i8, i1 } %pair, 0 %old = extractvalue { i8, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK-NEXT: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK-NEXT: casab w0, w1, [x[[ADDR]]]
; CHECK-NEXT: ret
ret i8 %old ret i8 %old
} }
define dso_local i1 @test_atomic_cmpxchg_i8_1(i8 %wanted, i8 %new) nounwind { define dso_local i1 @test_atomic_cmpxchg_i8_1(i8 %wanted, i8 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i8_1: ; CHECK-LABEL: test_atomic_cmpxchg_i8_1:
; CHECK: // %bb.0:
; CHECK-NEXT: mov w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: casab w8, w1, [x9]
; CHECK-NEXT: cmp w8, w0, uxtb
; CHECK-NEXT: cset w0, eq
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i8_1: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i8_1:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: stp x30, x19, [sp, #-16]! // 16-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: cset w0, eq ; OUTLINE-ATOMICS-NEXT: cset w0, eq
; OUTLINE-ATOMICS-NEXT: ldp x30, x19, [sp], #16 // 16-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldp x30, x19, [sp], #16 // 16-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i8_1:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mov w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: casab w8, w1, [x9]
; CHECK-REG-NEXT: cmp w8, w0, uxtb
; CHECK-REG-NEXT: cset w0, eq
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var8, i8 %wanted, i8 %new acquire acquire %pair = cmpxchg ptr @var8, i8 %wanted, i8 %new acquire acquire
%success = extractvalue { i8, i1 } %pair, 1 %success = extractvalue { i8, i1 } %pair, 1
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: casab w[[NEW:[0-9]+]], w1, [x[[ADDR]]]
; CHECK-NEXT: cmp w[[NEW]], w0, uxtb
; CHECK-NEXT: cset w0, eq
; CHECK-NEXT: ret
ret i1 %success ret i1 %success
} }
define dso_local i16 @test_atomic_cmpxchg_i16(i16 %wanted, i16 %new) nounwind { define dso_local i16 @test_atomic_cmpxchg_i16(i16 %wanted, i16 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i16: ; CHECK-LABEL: test_atomic_cmpxchg_i16:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: casah w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i16: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i16:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas2_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas2_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i16:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: casah w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var16, i16 %wanted, i16 %new acquire acquire %pair = cmpxchg ptr @var16, i16 %wanted, i16 %new acquire acquire
%old = extractvalue { i16, i1 } %pair, 0 %old = extractvalue { i16, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK-NEXT: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK-NEXT: casah w0, w1, [x[[ADDR]]]
; CHECK-NEXT: ret
ret i16 %old ret i16 %old
} }
define dso_local i1 @test_atomic_cmpxchg_i16_1(i16 %wanted, i16 %new) nounwind { define dso_local i1 @test_atomic_cmpxchg_i16_1(i16 %wanted, i16 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i16_1: ; CHECK-LABEL: test_atomic_cmpxchg_i16_1:
; CHECK: // %bb.0:
; CHECK-NEXT: mov w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: casah w8, w1, [x9]
; CHECK-NEXT: cmp w8, w0, uxth
; CHECK-NEXT: cset w0, eq
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i16_1: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i16_1:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: stp x30, x19, [sp, #-16]! // 16-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: stp x30, x19, [sp, #-16]! // 16-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: cset w0, eq ; OUTLINE-ATOMICS-NEXT: cset w0, eq
; OUTLINE-ATOMICS-NEXT: ldp x30, x19, [sp], #16 // 16-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldp x30, x19, [sp], #16 // 16-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i16_1:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mov w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: casah w8, w1, [x9]
; CHECK-REG-NEXT: cmp w8, w0, uxth
; CHECK-REG-NEXT: cset w0, eq
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var16, i16 %wanted, i16 %new acquire acquire %pair = cmpxchg ptr @var16, i16 %wanted, i16 %new acquire acquire
%success = extractvalue { i16, i1 } %pair, 1 %success = extractvalue { i16, i1 } %pair, 1
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK-NEXT: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: casah w[[NEW:[0-9]+]], w1, [x[[ADDR]]]
; CHECK-NEXT: cmp w[[NEW]], w0, uxth
; CHECK-NEXT: cset w0, eq
; CHECK-NEXT: ret
ret i1 %success ret i1 %success
} }
define dso_local i32 @test_atomic_cmpxchg_i32(i32 %wanted, i32 %new) nounwind { define dso_local i32 @test_atomic_cmpxchg_i32(i32 %wanted, i32 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i32: ; CHECK-LABEL: test_atomic_cmpxchg_i32:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: casa w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i32:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: casa w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var32, i32 %wanted, i32 %new acquire acquire %pair = cmpxchg ptr @var32, i32 %wanted, i32 %new acquire acquire
%old = extractvalue { i32, i1 } %pair, 0 %old = extractvalue { i32, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: casa w0, w1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i32 @test_atomic_cmpxchg_i32_monotonic_acquire(i32 %wanted, i32 %new) nounwind { define dso_local i32 @test_atomic_cmpxchg_i32_monotonic_acquire(i32 %wanted, i32 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i32_monotonic_acquire: ; CHECK-LABEL: test_atomic_cmpxchg_i32_monotonic_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: casa w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32_monotonic_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32_monotonic_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i32_monotonic_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: casa w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var32, i32 %wanted, i32 %new monotonic acquire %pair = cmpxchg ptr @var32, i32 %wanted, i32 %new monotonic acquire
%old = extractvalue { i32, i1 } %pair, 0 %old = extractvalue { i32, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: casa w0, w1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_cmpxchg_i64(i64 %wanted, i64 %new) nounwind { define dso_local i64 @test_atomic_cmpxchg_i64(i64 %wanted, i64 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i64: ; CHECK-LABEL: test_atomic_cmpxchg_i64:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: casa x0, x1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i64: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i64:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i64:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: casa x0, x1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var64, i64 %wanted, i64 %new acquire acquire %pair = cmpxchg ptr @var64, i64 %wanted, i64 %new acquire acquire
%old = extractvalue { i64, i1 } %pair, 0 %old = extractvalue { i64, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: casa x0, x1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local i128 @test_atomic_cmpxchg_i128(i128 %wanted, i128 %new) nounwind { define dso_local i128 @test_atomic_cmpxchg_i128(i128 %wanted, i128 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i128: ; CHECK-LABEL: test_atomic_cmpxchg_i128:
; CHECK: // %bb.0:
; CHECK-NEXT: // kill: def $x3 killed $x3 killed $x2_x3 def $x2_x3
; CHECK-NEXT: // kill: def $x1 killed $x1 killed $x0_x1 def $x0_x1
; CHECK-NEXT: // kill: def $x2 killed $x2 killed $x2_x3 def $x2_x3
; CHECK-NEXT: // kill: def $x0 killed $x0 killed $x0_x1 def $x0_x1
; CHECK-NEXT: adrp x8, var128
; CHECK-NEXT: add x8, x8, :lo12:var128
; CHECK-NEXT: caspa x0, x1, x2, x3, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i128: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i128:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas16_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas16_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i128:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: // kill: def $x3 killed $x3 killed $x2_x3 def $x2_x3
; CHECK-REG-NEXT: // kill: def $x1 killed $x1 killed $x0_x1 def $x0_x1
; CHECK-REG-NEXT: // kill: def $x2 killed $x2 killed $x2_x3 def $x2_x3
; CHECK-REG-NEXT: // kill: def $x0 killed $x0 killed $x0_x1 def $x0_x1
; CHECK-REG-NEXT: adrp x8, var128
; CHECK-REG-NEXT: add x8, x8, :lo12:var128
; CHECK-REG-NEXT: caspa x0, x1, x2, x3, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var128, i128 %wanted, i128 %new acquire acquire %pair = cmpxchg ptr @var128, i128 %wanted, i128 %new acquire acquire
%old = extractvalue { i128, i1 } %pair, 0 %old = extractvalue { i128, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var128
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var128
; CHECK: caspa x0, x1, x2, x3, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i128 %old ret i128 %old
} }
define dso_local i128 @test_atomic_cmpxchg_i128_monotonic_seqcst(i128 %wanted, i128 %new) nounwind { define dso_local i128 @test_atomic_cmpxchg_i128_monotonic_seqcst(i128 %wanted, i128 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i128_monotonic_seqcst: ; CHECK-LABEL: test_atomic_cmpxchg_i128_monotonic_seqcst:
; CHECK: // %bb.0:
; CHECK-NEXT: // kill: def $x3 killed $x3 killed $x2_x3 def $x2_x3
; CHECK-NEXT: // kill: def $x1 killed $x1 killed $x0_x1 def $x0_x1
; CHECK-NEXT: // kill: def $x2 killed $x2 killed $x2_x3 def $x2_x3
; CHECK-NEXT: // kill: def $x0 killed $x0 killed $x0_x1 def $x0_x1
; CHECK-NEXT: adrp x8, var128
; CHECK-NEXT: add x8, x8, :lo12:var128
; CHECK-NEXT: caspal x0, x1, x2, x3, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i128_monotonic_seqcst: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i128_monotonic_seqcst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas16_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas16_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i128_monotonic_seqcst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: // kill: def $x3 killed $x3 killed $x2_x3 def $x2_x3
; CHECK-REG-NEXT: // kill: def $x1 killed $x1 killed $x0_x1 def $x0_x1
; CHECK-REG-NEXT: // kill: def $x2 killed $x2 killed $x2_x3 def $x2_x3
; CHECK-REG-NEXT: // kill: def $x0 killed $x0 killed $x0_x1 def $x0_x1
; CHECK-REG-NEXT: adrp x8, var128
; CHECK-REG-NEXT: add x8, x8, :lo12:var128
; CHECK-REG-NEXT: caspal x0, x1, x2, x3, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var128, i128 %wanted, i128 %new monotonic seq_cst %pair = cmpxchg ptr @var128, i128 %wanted, i128 %new monotonic seq_cst
%old = extractvalue { i128, i1 } %pair, 0 %old = extractvalue { i128, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var128
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var128
; CHECK: caspal x0, x1, x2, x3, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i128 %old ret i128 %old
} }
define dso_local i128 @test_atomic_cmpxchg_i128_release_acquire(i128 %wanted, i128 %new) nounwind { define dso_local i128 @test_atomic_cmpxchg_i128_release_acquire(i128 %wanted, i128 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i128_release_acquire: ; CHECK-LABEL: test_atomic_cmpxchg_i128_release_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: // kill: def $x3 killed $x3 killed $x2_x3 def $x2_x3
; CHECK-NEXT: // kill: def $x1 killed $x1 killed $x0_x1 def $x0_x1
; CHECK-NEXT: // kill: def $x2 killed $x2 killed $x2_x3 def $x2_x3
; CHECK-NEXT: // kill: def $x0 killed $x0 killed $x0_x1 def $x0_x1
; CHECK-NEXT: adrp x8, var128
; CHECK-NEXT: add x8, x8, :lo12:var128
; CHECK-NEXT: caspal x0, x1, x2, x3, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i128_release_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i128_release_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas16_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas16_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i128_release_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: // kill: def $x3 killed $x3 killed $x2_x3 def $x2_x3
; CHECK-REG-NEXT: // kill: def $x1 killed $x1 killed $x0_x1 def $x0_x1
; CHECK-REG-NEXT: // kill: def $x2 killed $x2 killed $x2_x3 def $x2_x3
; CHECK-REG-NEXT: // kill: def $x0 killed $x0 killed $x0_x1 def $x0_x1
; CHECK-REG-NEXT: adrp x8, var128
; CHECK-REG-NEXT: add x8, x8, :lo12:var128
; CHECK-REG-NEXT: caspal x0, x1, x2, x3, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var128, i128 %wanted, i128 %new release acquire %pair = cmpxchg ptr @var128, i128 %wanted, i128 %new release acquire
%old = extractvalue { i128, i1 } %pair, 0 %old = extractvalue { i128, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var128
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var128
; CHECK: caspal x0, x1, x2, x3, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i128 %old ret i128 %old
} }
define dso_local i8 @test_atomic_load_sub_i8(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_sub_i8(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i8: ; CHECK-LABEL: test_atomic_load_sub_i8:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldaddalb w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i8:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldaddalb w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var8, i8 %offset seq_cst %old = atomicrmw sub ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddalb w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_sub_i16(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_sub_i16(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i16: ; CHECK-LABEL: test_atomic_load_sub_i16:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldaddalh w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i16:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldaddalh w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var16, i16 %offset seq_cst %old = atomicrmw sub ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddalh w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_sub_i32(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_sub_i32(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32: ; CHECK-LABEL: test_atomic_load_sub_i32:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldaddal w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldaddal w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var32, i32 %offset seq_cst %old = atomicrmw sub ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_sub_i64(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_sub_i64(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64: ; CHECK-LABEL: test_atomic_load_sub_i64:
; CHECK: // %bb.0:
; CHECK-NEXT: neg x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldaddal x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldaddal x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var64, i64 %offset seq_cst %old = atomicrmw sub ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: neg x[[NEG:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x[[NEG]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_sub_i32_noret(i32 %offset) nounwind { define dso_local void @test_atomic_load_sub_i32_noret(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_noret: ; CHECK-LABEL: test_atomic_load_sub_i32_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldaddal w8, w8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldaddal w8, w8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw sub ptr @var32, i32 %offset seq_cst atomicrmw sub ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_sub_i64_noret(i64 %offset) nounwind { define dso_local void @test_atomic_load_sub_i64_noret(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_noret: ; CHECK-LABEL: test_atomic_load_sub_i64_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: neg x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldaddal x8, x8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldaddal x8, x8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw sub ptr @var64, i64 %offset seq_cst atomicrmw sub ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: neg x[[NEG:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x[[NEG]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_sub_i8_neg_imm() nounwind { define dso_local i8 @test_atomic_load_sub_i8_neg_imm() nounwind {
; CHECK-LABEL: test_atomic_load_sub_i8_neg_imm: ; CHECK-LABEL: test_atomic_load_sub_i8_neg_imm:
; CHECK: // %bb.0:
; CHECK-NEXT: mov w8, #1 // =0x1
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldaddalb w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_neg_imm: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_neg_imm:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
; OUTLINE-ATOMICS-NEXT: adrp x1, var8 ; OUTLINE-ATOMICS-NEXT: adrp x1, var8
; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var8
; OUTLINE-ATOMICS-NEXT: mov w0, #1 ; OUTLINE-ATOMICS-NEXT: mov w0, #1 // =0x1
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i8_neg_imm:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mov w8, #1 // =0x1
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldaddalb w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var8, i8 -1 seq_cst %old = atomicrmw sub ptr @var8, i8 -1 seq_cst
; CHECK-NOT: dmb
; CHECK: mov w[[IMM:[0-9]+]], #1
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddalb w[[IMM]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_sub_i16_neg_imm() nounwind { define dso_local i16 @test_atomic_load_sub_i16_neg_imm() nounwind {
; CHECK-LABEL: test_atomic_load_sub_i16_neg_imm: ; CHECK-LABEL: test_atomic_load_sub_i16_neg_imm:
; CHECK: // %bb.0:
; CHECK-NEXT: mov w8, #1 // =0x1
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldaddalh w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_neg_imm: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_neg_imm:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
; OUTLINE-ATOMICS-NEXT: adrp x1, var16 ; OUTLINE-ATOMICS-NEXT: adrp x1, var16
; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var16
; OUTLINE-ATOMICS-NEXT: mov w0, #1 ; OUTLINE-ATOMICS-NEXT: mov w0, #1 // =0x1
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i16_neg_imm:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mov w8, #1 // =0x1
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldaddalh w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var16, i16 -1 seq_cst %old = atomicrmw sub ptr @var16, i16 -1 seq_cst
; CHECK-NOT: dmb
; CHECK: mov w[[IMM:[0-9]+]], #1
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddalh w[[IMM]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_sub_i32_neg_imm() nounwind { define dso_local i32 @test_atomic_load_sub_i32_neg_imm() nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_neg_imm: ; CHECK-LABEL: test_atomic_load_sub_i32_neg_imm:
; CHECK: // %bb.0:
; CHECK-NEXT: mov w8, #1 // =0x1
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldaddal w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_neg_imm: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_neg_imm:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
; OUTLINE-ATOMICS-NEXT: adrp x1, var32 ; OUTLINE-ATOMICS-NEXT: adrp x1, var32
; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var32
; OUTLINE-ATOMICS-NEXT: mov w0, #1 ; OUTLINE-ATOMICS-NEXT: mov w0, #1 // =0x1
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_neg_imm:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mov w8, #1 // =0x1
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldaddal w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var32, i32 -1 seq_cst %old = atomicrmw sub ptr @var32, i32 -1 seq_cst
; CHECK-NOT: dmb
; CHECK: mov w[[IMM:[0-9]+]], #1
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w[[IMM]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_sub_i64_neg_imm() nounwind { define dso_local i64 @test_atomic_load_sub_i64_neg_imm() nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_neg_imm: ; CHECK-LABEL: test_atomic_load_sub_i64_neg_imm:
; CHECK: // %bb.0:
; CHECK-NEXT: mov w8, #1 // =0x1
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldaddal x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_neg_imm: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_neg_imm:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
; OUTLINE-ATOMICS-NEXT: adrp x1, var64 ; OUTLINE-ATOMICS-NEXT: adrp x1, var64
; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var64
; OUTLINE-ATOMICS-NEXT: mov w0, #1 ; OUTLINE-ATOMICS-NEXT: mov w0, #1 // =0x1
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_neg_imm:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mov w8, #1 // =0x1
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldaddal x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var64, i64 -1 seq_cst %old = atomicrmw sub ptr @var64, i64 -1 seq_cst
; CHECK-NOT: dmb
; CHECK: mov w[[IMM:[0-9]+]], #1
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x[[IMM]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local i8 @test_atomic_load_sub_i8_neg_arg(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_sub_i8_neg_arg(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i8_neg_arg: ; CHECK-LABEL: test_atomic_load_sub_i8_neg_arg:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldaddalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_neg_arg: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_neg_arg:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i8_neg_arg:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldaddalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%neg = sub i8 0, %offset %neg = sub i8 0, %offset
%old = atomicrmw sub ptr @var8, i8 %neg seq_cst %old = atomicrmw sub ptr @var8, i8 %neg seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddalb w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_sub_i16_neg_arg(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_sub_i16_neg_arg(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i16_neg_arg: ; CHECK-LABEL: test_atomic_load_sub_i16_neg_arg:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldaddalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_neg_arg: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_neg_arg:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i16_neg_arg:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldaddalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%neg = sub i16 0, %offset %neg = sub i16 0, %offset
%old = atomicrmw sub ptr @var16, i16 %neg seq_cst %old = atomicrmw sub ptr @var16, i16 %neg seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddalh w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_sub_i32_neg_arg(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_sub_i32_neg_arg(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_neg_arg: ; CHECK-LABEL: test_atomic_load_sub_i32_neg_arg:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldaddal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_neg_arg: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_neg_arg:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_neg_arg:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldaddal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%neg = sub i32 0, %offset %neg = sub i32 0, %offset
%old = atomicrmw sub ptr @var32, i32 %neg seq_cst %old = atomicrmw sub ptr @var32, i32 %neg seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_sub_i64_neg_arg(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_sub_i64_neg_arg(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_neg_arg: ; CHECK-LABEL: test_atomic_load_sub_i64_neg_arg:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldaddal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_neg_arg: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_neg_arg:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_neg_arg:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldaddal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%neg = sub i64 0, %offset %neg = sub i64 0, %offset
%old = atomicrmw sub ptr @var64, i64 %neg seq_cst %old = atomicrmw sub ptr @var64, i64 %neg seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local i8 @test_atomic_load_and_i8(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_and_i8(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i8: ; CHECK-LABEL: test_atomic_load_and_i8:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldclralb w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i8:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldclralb w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var8, i8 %offset seq_cst %old = atomicrmw and ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldclralb w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_and_i16(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_and_i16(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i16: ; CHECK-LABEL: test_atomic_load_and_i16:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldclralh w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i16:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldclralh w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var16, i16 %offset seq_cst %old = atomicrmw and ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldclralh w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_and_i32(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_and_i32(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32: ; CHECK-LABEL: test_atomic_load_and_i32:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclral w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclral w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var32, i32 %offset seq_cst %old = atomicrmw and ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclral w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_and_i64(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_and_i64(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64: ; CHECK-LABEL: test_atomic_load_and_i64:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclral x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclral x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var64, i64 %offset seq_cst %old = atomicrmw and ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: mvn x[[NOT:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclral x[[NOT]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local i8 @test_atomic_load_and_i8_inv_imm() nounwind { define dso_local i8 @test_atomic_load_and_i8_inv_imm() nounwind {
; CHECK-LABEL: test_atomic_load_and_i8_inv_imm: ; CHECK-LABEL: test_atomic_load_and_i8_inv_imm:
; CHECK: // %bb.0:
; CHECK-NEXT: mov w8, #1 // =0x1
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldclralb w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_inv_imm: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_inv_imm:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
; OUTLINE-ATOMICS-NEXT: adrp x1, var8 ; OUTLINE-ATOMICS-NEXT: adrp x1, var8
; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var8
; OUTLINE-ATOMICS-NEXT: mov w0, #1 ; OUTLINE-ATOMICS-NEXT: mov w0, #1 // =0x1
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i8_inv_imm:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mov w8, #1 // =0x1
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldclralb w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var8, i8 -2 seq_cst %old = atomicrmw and ptr @var8, i8 -2 seq_cst
; CHECK-NOT: dmb
; CHECK: mov w[[CONST:[0-9]+]], #1
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldclralb w[[CONST]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_and_i16_inv_imm() nounwind { define dso_local i16 @test_atomic_load_and_i16_inv_imm() nounwind {
; CHECK-LABEL: test_atomic_load_and_i16_inv_imm: ; CHECK-LABEL: test_atomic_load_and_i16_inv_imm:
; CHECK: // %bb.0:
; CHECK-NEXT: mov w8, #1 // =0x1
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldclralh w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_inv_imm: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_inv_imm:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
; OUTLINE-ATOMICS-NEXT: adrp x1, var16 ; OUTLINE-ATOMICS-NEXT: adrp x1, var16
; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var16
; OUTLINE-ATOMICS-NEXT: mov w0, #1 ; OUTLINE-ATOMICS-NEXT: mov w0, #1 // =0x1
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i16_inv_imm:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mov w8, #1 // =0x1
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldclralh w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var16, i16 -2 seq_cst %old = atomicrmw and ptr @var16, i16 -2 seq_cst
; CHECK-NOT: dmb
; CHECK: mov w[[CONST:[0-9]+]], #1
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldclralh w[[CONST]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_and_i32_inv_imm() nounwind { define dso_local i32 @test_atomic_load_and_i32_inv_imm() nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_inv_imm: ; CHECK-LABEL: test_atomic_load_and_i32_inv_imm:
; CHECK: // %bb.0:
; CHECK-NEXT: mov w8, #1 // =0x1
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclral w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_inv_imm: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_inv_imm:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
; OUTLINE-ATOMICS-NEXT: adrp x1, var32 ; OUTLINE-ATOMICS-NEXT: adrp x1, var32
; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var32
; OUTLINE-ATOMICS-NEXT: mov w0, #1 ; OUTLINE-ATOMICS-NEXT: mov w0, #1 // =0x1
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_inv_imm:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mov w8, #1 // =0x1
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclral w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var32, i32 -2 seq_cst %old = atomicrmw and ptr @var32, i32 -2 seq_cst
; CHECK-NOT: dmb
; CHECK: mov w[[CONST:[0-9]+]], #1
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclral w[[CONST]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_and_i64_inv_imm() nounwind { define dso_local i64 @test_atomic_load_and_i64_inv_imm() nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_inv_imm: ; CHECK-LABEL: test_atomic_load_and_i64_inv_imm:
; CHECK: // %bb.0:
; CHECK-NEXT: mov w8, #1 // =0x1
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclral x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_inv_imm: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_inv_imm:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
; OUTLINE-ATOMICS-NEXT: adrp x1, var64 ; OUTLINE-ATOMICS-NEXT: adrp x1, var64
; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x1, x1, :lo12:var64
; OUTLINE-ATOMICS-NEXT: mov w0, #1 ; OUTLINE-ATOMICS-NEXT: mov w0, #1 // =0x1
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_inv_imm:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mov w8, #1 // =0x1
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclral x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var64, i64 -2 seq_cst %old = atomicrmw and ptr @var64, i64 -2 seq_cst
; CHECK-NOT: dmb
; CHECK: mov w[[CONST:[0-9]+]], #1
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclral x[[CONST]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local i8 @test_atomic_load_and_i8_inv_arg(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_and_i8_inv_arg(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i8_inv_arg: ; CHECK-LABEL: test_atomic_load_and_i8_inv_arg:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldclralb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_inv_arg: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_inv_arg:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i8_inv_arg:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldclralb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%inv = xor i8 %offset, -1 %inv = xor i8 %offset, -1
%old = atomicrmw and ptr @var8, i8 %inv seq_cst %old = atomicrmw and ptr @var8, i8 %inv seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldclralb w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_and_i16_inv_arg(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_and_i16_inv_arg(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i16_inv_arg: ; CHECK-LABEL: test_atomic_load_and_i16_inv_arg:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldclralh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_inv_arg: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_inv_arg:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i16_inv_arg:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldclralh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%inv = xor i16 %offset, -1 %inv = xor i16 %offset, -1
%old = atomicrmw and ptr @var16, i16 %inv seq_cst %old = atomicrmw and ptr @var16, i16 %inv seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldclralh w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_and_i32_inv_arg(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_and_i32_inv_arg(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_inv_arg: ; CHECK-LABEL: test_atomic_load_and_i32_inv_arg:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldclral w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_inv_arg: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_inv_arg:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_inv_arg:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldclral w0, w0, [x8]
; CHECK-REG-NEXT: ret
%inv = xor i32 %offset, -1 %inv = xor i32 %offset, -1
%old = atomicrmw and ptr @var32, i32 %inv seq_cst %old = atomicrmw and ptr @var32, i32 %inv seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclral w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_and_i64_inv_arg(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_and_i64_inv_arg(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_inv_arg: ; CHECK-LABEL: test_atomic_load_and_i64_inv_arg:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldclral x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_inv_arg: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_inv_arg:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_inv_arg:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldclral x0, x0, [x8]
; CHECK-REG-NEXT: ret
%inv = xor i64 %offset, -1 %inv = xor i64 %offset, -1
%old = atomicrmw and ptr @var64, i64 %inv seq_cst %old = atomicrmw and ptr @var64, i64 %inv seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclral x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_and_i32_noret(i32 %offset) nounwind { define dso_local void @test_atomic_load_and_i32_noret(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_noret: ; CHECK-LABEL: test_atomic_load_and_i32_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclral w8, w8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclral w8, w8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw and ptr @var32, i32 %offset seq_cst atomicrmw and ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclral w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_and_i64_noret(i64 %offset) nounwind { define dso_local void @test_atomic_load_and_i64_noret(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_noret: ; CHECK-LABEL: test_atomic_load_and_i64_noret:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclral x8, x8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_noret: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_noret:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_noret:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclral x8, x8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw and ptr @var64, i64 %offset seq_cst atomicrmw and ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: mvn x[[NOT:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclral x[[NOT]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_add_i8_acq_rel(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_add_i8_acq_rel(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i8_acq_rel: ; CHECK-LABEL: test_atomic_load_add_i8_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldaddalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i8_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i8_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i8_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldaddalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var8, i8 %offset acq_rel %old = atomicrmw add ptr @var8, i8 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_add_i16_acq_rel(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_add_i16_acq_rel(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i16_acq_rel: ; CHECK-LABEL: test_atomic_load_add_i16_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldaddalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i16_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i16_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i16_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldaddalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var16, i16 %offset acq_rel %old = atomicrmw add ptr @var16, i16 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_add_i32_acq_rel(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_add_i32_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i32_acq_rel: ; CHECK-LABEL: test_atomic_load_add_i32_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldaddal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i32_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldaddal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var32, i32 %offset acq_rel %old = atomicrmw add ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_add_i64_acq_rel(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_add_i64_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i64_acq_rel: ; CHECK-LABEL: test_atomic_load_add_i64_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldaddal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i64_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldaddal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var64, i64 %offset acq_rel %old = atomicrmw add ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_add_i32_noret_acq_rel(i32 %offset) nounwind { define dso_local void @test_atomic_load_add_i32_noret_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i32_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_add_i32_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldaddal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i32_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldaddal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw add ptr @var32, i32 %offset acq_rel atomicrmw add ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_add_i64_noret_acq_rel(i64 %offset) nounwind { define dso_local void @test_atomic_load_add_i64_noret_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i64_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_add_i64_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldaddal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i64_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldaddal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw add ptr @var64, i64 %offset acq_rel atomicrmw add ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_add_i8_acquire(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_add_i8_acquire(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i8_acquire: ; CHECK-LABEL: test_atomic_load_add_i8_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldaddab w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i8_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i8_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i8_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldaddab w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var8, i8 %offset acquire %old = atomicrmw add ptr @var8, i8 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddab w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_add_i16_acquire(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_add_i16_acquire(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i16_acquire: ; CHECK-LABEL: test_atomic_load_add_i16_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldaddah w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i16_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i16_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i16_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldaddah w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var16, i16 %offset acquire %old = atomicrmw add ptr @var16, i16 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddah w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_add_i32_acquire(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_add_i32_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i32_acquire: ; CHECK-LABEL: test_atomic_load_add_i32_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldadda w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i32_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldadda w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var32, i32 %offset acquire %old = atomicrmw add ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldadda w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_add_i64_acquire(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_add_i64_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i64_acquire: ; CHECK-LABEL: test_atomic_load_add_i64_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldadda x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i64_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldadda x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var64, i64 %offset acquire %old = atomicrmw add ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldadda x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_add_i32_noret_acquire(i32 %offset) nounwind { define dso_local void @test_atomic_load_add_i32_noret_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i32_noret_acquire: ; CHECK-LABEL: test_atomic_load_add_i32_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldadda w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i32_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldadda w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw add ptr @var32, i32 %offset acquire atomicrmw add ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldadda w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_add_i64_noret_acquire(i64 %offset) nounwind { define dso_local void @test_atomic_load_add_i64_noret_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i64_noret_acquire: ; CHECK-LABEL: test_atomic_load_add_i64_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldadda x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i64_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldadda x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw add ptr @var64, i64 %offset acquire atomicrmw add ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldadda x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_add_i8_monotonic(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_add_i8_monotonic(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i8_monotonic: ; CHECK-LABEL: test_atomic_load_add_i8_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldaddb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i8_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i8_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i8_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldaddb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var8, i8 %offset monotonic %old = atomicrmw add ptr @var8, i8 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_add_i16_monotonic(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_add_i16_monotonic(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i16_monotonic: ; CHECK-LABEL: test_atomic_load_add_i16_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldaddh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i16_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i16_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i16_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldaddh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var16, i16 %offset monotonic %old = atomicrmw add ptr @var16, i16 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_add_i32_monotonic(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_add_i32_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i32_monotonic: ; CHECK-LABEL: test_atomic_load_add_i32_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldadd w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i32_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldadd w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var32, i32 %offset monotonic %old = atomicrmw add ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldadd w[[OLD:[0-9]+]], w[[NEW:[0-9,a-z]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_add_i64_monotonic(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_add_i64_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i64_monotonic: ; CHECK-LABEL: test_atomic_load_add_i64_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldadd x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i64_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldadd x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var64, i64 %offset monotonic %old = atomicrmw add ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldadd x[[OLD:[0-9]+]], x[[NEW:[0-9,a-z]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_add_i32_noret_monotonic(i32 %offset) nounwind { define dso_local void @test_atomic_load_add_i32_noret_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i32_noret_monotonic: ; CHECK-LABEL: test_atomic_load_add_i32_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldadd w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i32_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldadd w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw add ptr @var32, i32 %offset monotonic atomicrmw add ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldadd w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_add_i64_noret_monotonic(i64 %offset) nounwind { define dso_local void @test_atomic_load_add_i64_noret_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i64_noret_monotonic: ; CHECK-LABEL: test_atomic_load_add_i64_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldadd x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i64_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldadd x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw add ptr @var64, i64 %offset monotonic atomicrmw add ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldadd x{{[0-9]}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_add_i8_release(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_add_i8_release(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i8_release: ; CHECK-LABEL: test_atomic_load_add_i8_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldaddlb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i8_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i8_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i8_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldaddlb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var8, i8 %offset release %old = atomicrmw add ptr @var8, i8 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddlb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_add_i16_release(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_add_i16_release(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i16_release: ; CHECK-LABEL: test_atomic_load_add_i16_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldaddlh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i16_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i16_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i16_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldaddlh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var16, i16 %offset release %old = atomicrmw add ptr @var16, i16 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddlh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_add_i32_release(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_add_i32_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i32_release: ; CHECK-LABEL: test_atomic_load_add_i32_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldaddl w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i32_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldaddl w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var32, i32 %offset release %old = atomicrmw add ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddl w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_add_i64_release(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_add_i64_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i64_release: ; CHECK-LABEL: test_atomic_load_add_i64_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldaddl x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i64_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldaddl x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var64, i64 %offset release %old = atomicrmw add ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddl x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_add_i32_noret_release(i32 %offset) nounwind { define dso_local void @test_atomic_load_add_i32_noret_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i32_noret_release: ; CHECK-LABEL: test_atomic_load_add_i32_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldaddl w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i32_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldaddl w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw add ptr @var32, i32 %offset release atomicrmw add ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddl w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_add_i64_noret_release(i64 %offset) nounwind { define dso_local void @test_atomic_load_add_i64_noret_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i64_noret_release: ; CHECK-LABEL: test_atomic_load_add_i64_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldaddl x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i64_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldaddl x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw add ptr @var64, i64 %offset release atomicrmw add ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddl x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_add_i8_seq_cst(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_add_i8_seq_cst(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i8_seq_cst: ; CHECK-LABEL: test_atomic_load_add_i8_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldaddalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i8_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i8_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i8_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldaddalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var8, i8 %offset seq_cst %old = atomicrmw add ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_add_i16_seq_cst(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_add_i16_seq_cst(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i16_seq_cst: ; CHECK-LABEL: test_atomic_load_add_i16_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldaddalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i16_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i16_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i16_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldaddalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var16, i16 %offset seq_cst %old = atomicrmw add ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_add_i32_seq_cst(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_add_i32_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i32_seq_cst: ; CHECK-LABEL: test_atomic_load_add_i32_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldaddal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i32_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldaddal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var32, i32 %offset seq_cst %old = atomicrmw add ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_add_i64_seq_cst(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_add_i64_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i64_seq_cst: ; CHECK-LABEL: test_atomic_load_add_i64_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldaddal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i64_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldaddal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw add ptr @var64, i64 %offset seq_cst %old = atomicrmw add ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_add_i32_noret_seq_cst(i32 %offset) nounwind { define dso_local void @test_atomic_load_add_i32_noret_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i32_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_add_i32_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldaddal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i32_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i32_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldaddal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw add ptr @var32, i32 %offset seq_cst atomicrmw add ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_add_i64_noret_seq_cst(i64 %offset) nounwind { define dso_local void @test_atomic_load_add_i64_noret_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_add_i64_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_add_i64_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldaddal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_add_i64_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_add_i64_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldaddal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw add ptr @var64, i64 %offset seq_cst atomicrmw add ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_and_i8_acq_rel(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_and_i8_acq_rel(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i8_acq_rel: ; CHECK-LABEL: test_atomic_load_and_i8_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldclralb w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i8_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldclralb w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var8, i8 %offset acq_rel %old = atomicrmw and ptr @var8, i8 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldclralb w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_and_i16_acq_rel(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_and_i16_acq_rel(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i16_acq_rel: ; CHECK-LABEL: test_atomic_load_and_i16_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldclralh w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i16_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldclralh w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var16, i16 %offset acq_rel %old = atomicrmw and ptr @var16, i16 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldclralh w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_and_i32_acq_rel(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_and_i32_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_acq_rel: ; CHECK-LABEL: test_atomic_load_and_i32_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclral w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclral w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var32, i32 %offset acq_rel %old = atomicrmw and ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclral w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_and_i64_acq_rel(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_and_i64_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_acq_rel: ; CHECK-LABEL: test_atomic_load_and_i64_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclral x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclral x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var64, i64 %offset acq_rel %old = atomicrmw and ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: mvn x[[NOT:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclral x[[NOT]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_and_i32_noret_acq_rel(i32 %offset) nounwind { define dso_local void @test_atomic_load_and_i32_noret_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_and_i32_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclral w8, w8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclral w8, w8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw and ptr @var32, i32 %offset acq_rel atomicrmw and ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclral w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_and_i64_noret_acq_rel(i64 %offset) nounwind { define dso_local void @test_atomic_load_and_i64_noret_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_and_i64_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclral x8, x8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclral x8, x8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw and ptr @var64, i64 %offset acq_rel atomicrmw and ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: mvn x[[NOT:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclral x[[NOT]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_and_i8_acquire(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_and_i8_acquire(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i8_acquire: ; CHECK-LABEL: test_atomic_load_and_i8_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldclrab w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i8_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldclrab w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var8, i8 %offset acquire %old = atomicrmw and ptr @var8, i8 %offset acquire
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldclrab w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_and_i16_acquire(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_and_i16_acquire(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i16_acquire: ; CHECK-LABEL: test_atomic_load_and_i16_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldclrah w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i16_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldclrah w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var16, i16 %offset acquire %old = atomicrmw and ptr @var16, i16 %offset acquire
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldclrah w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_and_i32_acquire(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_and_i32_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_acquire: ; CHECK-LABEL: test_atomic_load_and_i32_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclra w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclra w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var32, i32 %offset acquire %old = atomicrmw and ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclra w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_and_i64_acquire(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_and_i64_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_acquire: ; CHECK-LABEL: test_atomic_load_and_i64_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclra x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclra x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var64, i64 %offset acquire %old = atomicrmw and ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: mvn x[[NOT:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclra x[[NOT]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_and_i32_noret_acquire(i32 %offset) nounwind { define dso_local void @test_atomic_load_and_i32_noret_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_noret_acquire: ; CHECK-LABEL: test_atomic_load_and_i32_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclra w8, w8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclra w8, w8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw and ptr @var32, i32 %offset acquire atomicrmw and ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclra w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_and_i64_noret_acquire(i64 %offset) nounwind { define dso_local void @test_atomic_load_and_i64_noret_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_noret_acquire: ; CHECK-LABEL: test_atomic_load_and_i64_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclra x8, x8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclra x8, x8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw and ptr @var64, i64 %offset acquire atomicrmw and ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: mvn x[[NOT:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclra x[[NOT]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_and_i8_monotonic(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_and_i8_monotonic(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i8_monotonic: ; CHECK-LABEL: test_atomic_load_and_i8_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldclrb w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i8_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldclrb w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var8, i8 %offset monotonic %old = atomicrmw and ptr @var8, i8 %offset monotonic
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldclrb w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_and_i16_monotonic(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_and_i16_monotonic(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i16_monotonic: ; CHECK-LABEL: test_atomic_load_and_i16_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldclrh w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i16_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldclrh w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var16, i16 %offset monotonic %old = atomicrmw and ptr @var16, i16 %offset monotonic
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldclrh w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_and_i32_monotonic(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_and_i32_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_monotonic: ; CHECK-LABEL: test_atomic_load_and_i32_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclr w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclr w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var32, i32 %offset monotonic %old = atomicrmw and ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclr w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_and_i64_monotonic(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_and_i64_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_monotonic: ; CHECK-LABEL: test_atomic_load_and_i64_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclr x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclr x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var64, i64 %offset monotonic %old = atomicrmw and ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: mvn x[[NOT:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclr x[[NOT]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_and_i32_noret_monotonic(i32 %offset) nounwind { define dso_local void @test_atomic_load_and_i32_noret_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_noret_monotonic: ; CHECK-LABEL: test_atomic_load_and_i32_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclr w8, w8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclr w8, w8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw and ptr @var32, i32 %offset monotonic atomicrmw and ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclr w{{[0-9]+}}, w[[NEW:[1-9][0-9]*]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_and_i64_noret_monotonic(i64 %offset) nounwind { define dso_local void @test_atomic_load_and_i64_noret_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_noret_monotonic: ; CHECK-LABEL: test_atomic_load_and_i64_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclr x8, x8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclr x8, x8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw and ptr @var64, i64 %offset monotonic atomicrmw and ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: mvn x[[NOT:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclr x{{[0-9]+}}, x[[NEW:[1-9][0-9]*]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_and_i8_release(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_and_i8_release(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i8_release: ; CHECK-LABEL: test_atomic_load_and_i8_release:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldclrlb w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i8_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldclrlb w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var8, i8 %offset release %old = atomicrmw and ptr @var8, i8 %offset release
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldclrlb w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_and_i16_release(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_and_i16_release(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i16_release: ; CHECK-LABEL: test_atomic_load_and_i16_release:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldclrlh w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i16_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldclrlh w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var16, i16 %offset release %old = atomicrmw and ptr @var16, i16 %offset release
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldclrlh w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_and_i32_release(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_and_i32_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_release: ; CHECK-LABEL: test_atomic_load_and_i32_release:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclrl w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclrl w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var32, i32 %offset release %old = atomicrmw and ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclrl w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_and_i64_release(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_and_i64_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_release: ; CHECK-LABEL: test_atomic_load_and_i64_release:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclrl x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclrl x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var64, i64 %offset release %old = atomicrmw and ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: mvn x[[NOT:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclrl x[[NOT]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_and_i32_noret_release(i32 %offset) nounwind { define dso_local void @test_atomic_load_and_i32_noret_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_noret_release: ; CHECK-LABEL: test_atomic_load_and_i32_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclrl w8, w8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclrl w8, w8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw and ptr @var32, i32 %offset release atomicrmw and ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclrl w{{[0-9]*}}, w[[NEW:[1-9][0-9]*]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_and_i64_noret_release(i64 %offset) nounwind { define dso_local void @test_atomic_load_and_i64_noret_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_noret_release: ; CHECK-LABEL: test_atomic_load_and_i64_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclrl x8, x8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclrl x8, x8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw and ptr @var64, i64 %offset release atomicrmw and ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: mvn x[[NOT:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclrl x{{[0-9]*}}, x[[NEW:[1-9][0-9]*]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_and_i8_seq_cst(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_and_i8_seq_cst(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i8_seq_cst: ; CHECK-LABEL: test_atomic_load_and_i8_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldclralb w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i8_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i8_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldclralb w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var8, i8 %offset seq_cst %old = atomicrmw and ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldclralb w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_and_i16_seq_cst(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_and_i16_seq_cst(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i16_seq_cst: ; CHECK-LABEL: test_atomic_load_and_i16_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldclralh w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i16_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i16_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldclralh w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var16, i16 %offset seq_cst %old = atomicrmw and ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldclralh w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_and_i32_seq_cst(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_and_i32_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_seq_cst: ; CHECK-LABEL: test_atomic_load_and_i32_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclral w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclral w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var32, i32 %offset seq_cst %old = atomicrmw and ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclral w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_and_i64_seq_cst(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_and_i64_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_seq_cst: ; CHECK-LABEL: test_atomic_load_and_i64_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclral x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclral x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw and ptr @var64, i64 %offset seq_cst %old = atomicrmw and ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: mvn x[[NOT:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclral x[[NOT]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_and_i32_noret_seq_cst(i32 %offset) nounwind { define dso_local void @test_atomic_load_and_i32_noret_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i32_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_and_i32_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldclral w8, w8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i32_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i32_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldclral w8, w8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw and ptr @var32, i32 %offset seq_cst atomicrmw and ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: mvn w[[NOT:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldclral w[[NOT]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_and_i64_noret_seq_cst(i64 %offset) nounwind { define dso_local void @test_atomic_load_and_i64_noret_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_and_i64_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_and_i64_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: mvn x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldclral x8, x8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_and_i64_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldclr8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_and_i64_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mvn x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldclral x8, x8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw and ptr @var64, i64 %offset seq_cst atomicrmw and ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: mvn x[[NOT:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldclral x[[NOT]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_cmpxchg_i8_acquire(i8 %wanted, i8 %new) nounwind { define dso_local i8 @test_atomic_cmpxchg_i8_acquire(i8 %wanted, i8 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i8_acquire: ; CHECK-LABEL: test_atomic_cmpxchg_i8_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: casab w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i8_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i8_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas1_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas1_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i8_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: casab w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var8, i8 %wanted, i8 %new acquire acquire %pair = cmpxchg ptr @var8, i8 %wanted, i8 %new acquire acquire
%old = extractvalue { i8, i1 } %pair, 0 %old = extractvalue { i8, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: casab w[[NEW:[0-9]+]], w[[OLD:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_cmpxchg_i16_acquire(i16 %wanted, i16 %new) nounwind { define dso_local i16 @test_atomic_cmpxchg_i16_acquire(i16 %wanted, i16 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i16_acquire: ; CHECK-LABEL: test_atomic_cmpxchg_i16_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: casah w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i16_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i16_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas2_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas2_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i16_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: casah w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var16, i16 %wanted, i16 %new acquire acquire %pair = cmpxchg ptr @var16, i16 %wanted, i16 %new acquire acquire
%old = extractvalue { i16, i1 } %pair, 0 %old = extractvalue { i16, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: casah w0, w1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_cmpxchg_i32_acquire(i32 %wanted, i32 %new) nounwind { define dso_local i32 @test_atomic_cmpxchg_i32_acquire(i32 %wanted, i32 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i32_acquire: ; CHECK-LABEL: test_atomic_cmpxchg_i32_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: casa w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i32_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: casa w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var32, i32 %wanted, i32 %new acquire acquire %pair = cmpxchg ptr @var32, i32 %wanted, i32 %new acquire acquire
%old = extractvalue { i32, i1 } %pair, 0 %old = extractvalue { i32, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: casa w0, w1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_cmpxchg_i64_acquire(i64 %wanted, i64 %new) nounwind { define dso_local i64 @test_atomic_cmpxchg_i64_acquire(i64 %wanted, i64 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i64_acquire: ; CHECK-LABEL: test_atomic_cmpxchg_i64_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: casa x0, x1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i64_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i64_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i64_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: casa x0, x1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var64, i64 %wanted, i64 %new acquire acquire %pair = cmpxchg ptr @var64, i64 %wanted, i64 %new acquire acquire
%old = extractvalue { i64, i1 } %pair, 0 %old = extractvalue { i64, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: casa x0, x1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local i128 @test_atomic_cmpxchg_i128_acquire(i128 %wanted, i128 %new) nounwind { define dso_local i128 @test_atomic_cmpxchg_i128_acquire(i128 %wanted, i128 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i128_acquire: ; CHECK-LABEL: test_atomic_cmpxchg_i128_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: // kill: def $x3 killed $x3 killed $x2_x3 def $x2_x3
; CHECK-NEXT: // kill: def $x1 killed $x1 killed $x0_x1 def $x0_x1
; CHECK-NEXT: // kill: def $x2 killed $x2 killed $x2_x3 def $x2_x3
; CHECK-NEXT: // kill: def $x0 killed $x0 killed $x0_x1 def $x0_x1
; CHECK-NEXT: adrp x8, var128
; CHECK-NEXT: add x8, x8, :lo12:var128
; CHECK-NEXT: caspa x0, x1, x2, x3, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i128_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i128_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas16_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas16_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i128_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: // kill: def $x3 killed $x3 killed $x2_x3 def $x2_x3
; CHECK-REG-NEXT: // kill: def $x1 killed $x1 killed $x0_x1 def $x0_x1
; CHECK-REG-NEXT: // kill: def $x2 killed $x2 killed $x2_x3 def $x2_x3
; CHECK-REG-NEXT: // kill: def $x0 killed $x0 killed $x0_x1 def $x0_x1
; CHECK-REG-NEXT: adrp x8, var128
; CHECK-REG-NEXT: add x8, x8, :lo12:var128
; CHECK-REG-NEXT: caspa x0, x1, x2, x3, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var128, i128 %wanted, i128 %new acquire acquire %pair = cmpxchg ptr @var128, i128 %wanted, i128 %new acquire acquire
%old = extractvalue { i128, i1 } %pair, 0 %old = extractvalue { i128, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var128
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var128
; CHECK: caspa x0, x1, x2, x3, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i128 %old ret i128 %old
} }
define dso_local i8 @test_atomic_cmpxchg_i8_monotonic(i8 %wanted, i8 %new) nounwind { define dso_local i8 @test_atomic_cmpxchg_i8_monotonic(i8 %wanted, i8 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i8_monotonic: ; CHECK-LABEL: test_atomic_cmpxchg_i8_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: casb w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i8_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i8_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas1_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas1_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i8_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: casb w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var8, i8 %wanted, i8 %new monotonic monotonic %pair = cmpxchg ptr @var8, i8 %wanted, i8 %new monotonic monotonic
%old = extractvalue { i8, i1 } %pair, 0 %old = extractvalue { i8, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: casb w[[NEW:[0-9]+]], w[[OLD:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_cmpxchg_i16_monotonic(i16 %wanted, i16 %new) nounwind { define dso_local i16 @test_atomic_cmpxchg_i16_monotonic(i16 %wanted, i16 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i16_monotonic: ; CHECK-LABEL: test_atomic_cmpxchg_i16_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: cash w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i16_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i16_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas2_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas2_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i16_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: cash w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var16, i16 %wanted, i16 %new monotonic monotonic %pair = cmpxchg ptr @var16, i16 %wanted, i16 %new monotonic monotonic
%old = extractvalue { i16, i1 } %pair, 0 %old = extractvalue { i16, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: cash w0, w1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_cmpxchg_i32_monotonic(i32 %wanted, i32 %new) nounwind { define dso_local i32 @test_atomic_cmpxchg_i32_monotonic(i32 %wanted, i32 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i32_monotonic: ; CHECK-LABEL: test_atomic_cmpxchg_i32_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: cas w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i32_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: cas w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var32, i32 %wanted, i32 %new monotonic monotonic %pair = cmpxchg ptr @var32, i32 %wanted, i32 %new monotonic monotonic
%old = extractvalue { i32, i1 } %pair, 0 %old = extractvalue { i32, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: cas w0, w1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_cmpxchg_i64_monotonic(i64 %wanted, i64 %new) nounwind { define dso_local i64 @test_atomic_cmpxchg_i64_monotonic(i64 %wanted, i64 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i64_monotonic: ; CHECK-LABEL: test_atomic_cmpxchg_i64_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: cas x0, x1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i64_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i64_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i64_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: cas x0, x1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var64, i64 %wanted, i64 %new monotonic monotonic %pair = cmpxchg ptr @var64, i64 %wanted, i64 %new monotonic monotonic
%old = extractvalue { i64, i1 } %pair, 0 %old = extractvalue { i64, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: cas x0, x1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local i128 @test_atomic_cmpxchg_i128_monotonic(i128 %wanted, i128 %new) nounwind { define dso_local i128 @test_atomic_cmpxchg_i128_monotonic(i128 %wanted, i128 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i128_monotonic: ; CHECK-LABEL: test_atomic_cmpxchg_i128_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: // kill: def $x3 killed $x3 killed $x2_x3 def $x2_x3
; CHECK-NEXT: // kill: def $x1 killed $x1 killed $x0_x1 def $x0_x1
; CHECK-NEXT: // kill: def $x2 killed $x2 killed $x2_x3 def $x2_x3
; CHECK-NEXT: // kill: def $x0 killed $x0 killed $x0_x1 def $x0_x1
; CHECK-NEXT: adrp x8, var128
; CHECK-NEXT: add x8, x8, :lo12:var128
; CHECK-NEXT: casp x0, x1, x2, x3, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i128_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i128_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas16_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas16_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i128_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: // kill: def $x3 killed $x3 killed $x2_x3 def $x2_x3
; CHECK-REG-NEXT: // kill: def $x1 killed $x1 killed $x0_x1 def $x0_x1
; CHECK-REG-NEXT: // kill: def $x2 killed $x2 killed $x2_x3 def $x2_x3
; CHECK-REG-NEXT: // kill: def $x0 killed $x0 killed $x0_x1 def $x0_x1
; CHECK-REG-NEXT: adrp x8, var128
; CHECK-REG-NEXT: add x8, x8, :lo12:var128
; CHECK-REG-NEXT: casp x0, x1, x2, x3, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var128, i128 %wanted, i128 %new monotonic monotonic %pair = cmpxchg ptr @var128, i128 %wanted, i128 %new monotonic monotonic
%old = extractvalue { i128, i1 } %pair, 0 %old = extractvalue { i128, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var128
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var128
; CHECK: casp x0, x1, x2, x3, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i128 %old ret i128 %old
} }
define dso_local i8 @test_atomic_cmpxchg_i8_seq_cst(i8 %wanted, i8 %new) nounwind { define dso_local i8 @test_atomic_cmpxchg_i8_seq_cst(i8 %wanted, i8 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i8_seq_cst: ; CHECK-LABEL: test_atomic_cmpxchg_i8_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: casalb w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i8_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i8_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i8_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: casalb w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var8, i8 %wanted, i8 %new seq_cst seq_cst %pair = cmpxchg ptr @var8, i8 %wanted, i8 %new seq_cst seq_cst
%old = extractvalue { i8, i1 } %pair, 0 %old = extractvalue { i8, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: casalb w[[NEW:[0-9]+]], w[[OLD:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_cmpxchg_i16_seq_cst(i16 %wanted, i16 %new) nounwind { define dso_local i16 @test_atomic_cmpxchg_i16_seq_cst(i16 %wanted, i16 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i16_seq_cst: ; CHECK-LABEL: test_atomic_cmpxchg_i16_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: casalh w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i16_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i16_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i16_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: casalh w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var16, i16 %wanted, i16 %new seq_cst seq_cst %pair = cmpxchg ptr @var16, i16 %wanted, i16 %new seq_cst seq_cst
%old = extractvalue { i16, i1 } %pair, 0 %old = extractvalue { i16, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: casalh w0, w1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_cmpxchg_i32_seq_cst(i32 %wanted, i32 %new) nounwind { define dso_local i32 @test_atomic_cmpxchg_i32_seq_cst(i32 %wanted, i32 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i32_seq_cst: ; CHECK-LABEL: test_atomic_cmpxchg_i32_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: casal w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i32_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: casal w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var32, i32 %wanted, i32 %new seq_cst seq_cst %pair = cmpxchg ptr @var32, i32 %wanted, i32 %new seq_cst seq_cst
%old = extractvalue { i32, i1 } %pair, 0 %old = extractvalue { i32, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: casal w0, w1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i32 @test_atomic_cmpxchg_i32_monotonic_seq_cst(i32 %wanted, i32 %new) nounwind { define dso_local i32 @test_atomic_cmpxchg_i32_monotonic_seq_cst(i32 %wanted, i32 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i32_monotonic_seq_cst: ; CHECK-LABEL: test_atomic_cmpxchg_i32_monotonic_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: casal w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32_monotonic_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32_monotonic_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i32_monotonic_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: casal w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var32, i32 %wanted, i32 %new monotonic seq_cst %pair = cmpxchg ptr @var32, i32 %wanted, i32 %new monotonic seq_cst
%old = extractvalue { i32, i1 } %pair, 0 %old = extractvalue { i32, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: casal w0, w1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i32 @test_atomic_cmpxchg_i32_release_acquire(i32 %wanted, i32 %new) nounwind { define dso_local i32 @test_atomic_cmpxchg_i32_release_acquire(i32 %wanted, i32 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i32_release_acquire: ; CHECK-LABEL: test_atomic_cmpxchg_i32_release_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: casal w0, w1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32_release_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i32_release_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i32_release_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: casal w0, w1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var32, i32 %wanted, i32 %new release acquire %pair = cmpxchg ptr @var32, i32 %wanted, i32 %new release acquire
%old = extractvalue { i32, i1 } %pair, 0 %old = extractvalue { i32, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: casal w0, w1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_cmpxchg_i64_seq_cst(i64 %wanted, i64 %new) nounwind { define dso_local i64 @test_atomic_cmpxchg_i64_seq_cst(i64 %wanted, i64 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i64_seq_cst: ; CHECK-LABEL: test_atomic_cmpxchg_i64_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: casal x0, x1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i64_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i64_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i64_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: casal x0, x1, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var64, i64 %wanted, i64 %new seq_cst seq_cst %pair = cmpxchg ptr @var64, i64 %wanted, i64 %new seq_cst seq_cst
%old = extractvalue { i64, i1 } %pair, 0 %old = extractvalue { i64, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: casal x0, x1, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local i128 @test_atomic_cmpxchg_i128_seq_cst(i128 %wanted, i128 %new) nounwind { define dso_local i128 @test_atomic_cmpxchg_i128_seq_cst(i128 %wanted, i128 %new) nounwind {
; CHECK-LABEL: test_atomic_cmpxchg_i128_seq_cst: ; CHECK-LABEL: test_atomic_cmpxchg_i128_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: // kill: def $x3 killed $x3 killed $x2_x3 def $x2_x3
; CHECK-NEXT: // kill: def $x1 killed $x1 killed $x0_x1 def $x0_x1
; CHECK-NEXT: // kill: def $x2 killed $x2 killed $x2_x3 def $x2_x3
; CHECK-NEXT: // kill: def $x0 killed $x0 killed $x0_x1 def $x0_x1
; CHECK-NEXT: adrp x8, var128
; CHECK-NEXT: add x8, x8, :lo12:var128
; CHECK-NEXT: caspal x0, x1, x2, x3, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i128_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_cmpxchg_i128_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas16_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_cas16_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_cmpxchg_i128_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: // kill: def $x3 killed $x3 killed $x2_x3 def $x2_x3
; CHECK-REG-NEXT: // kill: def $x1 killed $x1 killed $x0_x1 def $x0_x1
; CHECK-REG-NEXT: // kill: def $x2 killed $x2 killed $x2_x3 def $x2_x3
; CHECK-REG-NEXT: // kill: def $x0 killed $x0 killed $x0_x1 def $x0_x1
; CHECK-REG-NEXT: adrp x8, var128
; CHECK-REG-NEXT: add x8, x8, :lo12:var128
; CHECK-REG-NEXT: caspal x0, x1, x2, x3, [x8]
; CHECK-REG-NEXT: ret
%pair = cmpxchg ptr @var128, i128 %wanted, i128 %new seq_cst seq_cst %pair = cmpxchg ptr @var128, i128 %wanted, i128 %new seq_cst seq_cst
%old = extractvalue { i128, i1 } %pair, 0 %old = extractvalue { i128, i1 } %pair, 0
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var128
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var128
; CHECK: caspal x0, x1, x2, x3, [x[[ADDR]]]
; CHECK-NOT: dmb
ret i128 %old ret i128 %old
} }
define dso_local i8 @test_atomic_load_max_i8_acq_rel(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_max_i8_acq_rel(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i8_acq_rel: ; CHECK-LABEL: test_atomic_load_max_i8_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsmaxalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i8_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i8_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: adrp x9, var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB163_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxtb w8, w10 ; OUTLINE-ATOMICS-NEXT: sxtb w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB163_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i8_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsmaxalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var8, i8 %offset acq_rel %old = atomicrmw max ptr @var8, i8 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsmaxalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_max_i16_acq_rel(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_max_i16_acq_rel(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i16_acq_rel: ; CHECK-LABEL: test_atomic_load_max_i16_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsmaxalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i16_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i16_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: adrp x9, var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB164_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxth w8, w10 ; OUTLINE-ATOMICS-NEXT: sxth w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB164_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i16_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsmaxalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var16, i16 %offset acq_rel %old = atomicrmw max ptr @var16, i16 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsmaxalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_max_i32_acq_rel(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_max_i32_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i32_acq_rel: ; CHECK-LABEL: test_atomic_load_max_i32_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmaxal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB165_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB165_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i32_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmaxal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var32, i32 %offset acq_rel %old = atomicrmw max ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmaxal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_max_i64_acq_rel(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_max_i64_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i64_acq_rel: ; CHECK-LABEL: test_atomic_load_max_i64_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmaxal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB166_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, gt ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB166_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i64_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmaxal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var64, i64 %offset acq_rel %old = atomicrmw max ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmaxal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_max_i32_noret_acq_rel(i32 %offset) nounwind { define dso_local void @test_atomic_load_max_i32_noret_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i32_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_max_i32_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmaxal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB167_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB167_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i32_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmaxal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw max ptr @var32, i32 %offset acq_rel atomicrmw max ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmaxal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_max_i64_noret_acq_rel(i64 %offset) nounwind { define dso_local void @test_atomic_load_max_i64_noret_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i64_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_max_i64_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmaxal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB168_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, gt ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB168_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i64_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmaxal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw max ptr @var64, i64 %offset acq_rel atomicrmw max ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmaxal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_max_i8_acquire(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_max_i8_acquire(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i8_acquire: ; CHECK-LABEL: test_atomic_load_max_i8_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsmaxab w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i8_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i8_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: adrp x9, var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB169_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxtb w8, w10 ; OUTLINE-ATOMICS-NEXT: sxtb w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt
; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB169_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i8_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsmaxab w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var8, i8 %offset acquire %old = atomicrmw max ptr @var8, i8 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsmaxab w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_max_i16_acquire(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_max_i16_acquire(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i16_acquire: ; CHECK-LABEL: test_atomic_load_max_i16_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsmaxah w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i16_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i16_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: adrp x9, var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB170_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxth w8, w10 ; OUTLINE-ATOMICS-NEXT: sxth w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt
; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB170_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i16_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsmaxah w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var16, i16 %offset acquire %old = atomicrmw max ptr @var16, i16 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsmaxah w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_max_i32_acquire(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_max_i32_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i32_acquire: ; CHECK-LABEL: test_atomic_load_max_i32_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmaxa w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB171_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, gt
; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB171_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i32_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmaxa w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var32, i32 %offset acquire %old = atomicrmw max ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmaxa w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_max_i64_acquire(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_max_i64_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i64_acquire: ; CHECK-LABEL: test_atomic_load_max_i64_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmaxa x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB172_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, gt ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, gt
; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB172_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i64_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmaxa x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var64, i64 %offset acquire %old = atomicrmw max ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmaxa x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_max_i32_noret_acquire(i32 %offset) nounwind { define dso_local void @test_atomic_load_max_i32_noret_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i32_noret_acquire: ; CHECK-LABEL: test_atomic_load_max_i32_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmaxa w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB173_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, gt
; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB173_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i32_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmaxa w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw max ptr @var32, i32 %offset acquire atomicrmw max ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmaxa w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_max_i64_noret_acquire(i64 %offset) nounwind { define dso_local void @test_atomic_load_max_i64_noret_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i64_noret_acquire: ; CHECK-LABEL: test_atomic_load_max_i64_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmaxa x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB174_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, gt ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, gt
; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB174_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i64_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmaxa x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw max ptr @var64, i64 %offset acquire atomicrmw max ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmaxa x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_max_i8_monotonic(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_max_i8_monotonic(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i8_monotonic: ; CHECK-LABEL: test_atomic_load_max_i8_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsmaxb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i8_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i8_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: adrp x9, var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB175_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrb w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrb w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxtb w8, w10 ; OUTLINE-ATOMICS-NEXT: sxtb w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt
; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB175_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i8_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsmaxb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var8, i8 %offset monotonic %old = atomicrmw max ptr @var8, i8 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsmaxb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_max_i16_monotonic(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_max_i16_monotonic(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i16_monotonic: ; CHECK-LABEL: test_atomic_load_max_i16_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsmaxh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i16_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i16_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: adrp x9, var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB176_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrh w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrh w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxth w8, w10 ; OUTLINE-ATOMICS-NEXT: sxth w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt
; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB176_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i16_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsmaxh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var16, i16 %offset monotonic %old = atomicrmw max ptr @var16, i16 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsmaxh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_max_i32_monotonic(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_max_i32_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i32_monotonic: ; CHECK-LABEL: test_atomic_load_max_i32_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmax w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB177_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, gt
; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB177_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i32_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmax w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var32, i32 %offset monotonic %old = atomicrmw max ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmax w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_max_i64_monotonic(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_max_i64_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i64_monotonic: ; CHECK-LABEL: test_atomic_load_max_i64_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmax x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB178_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, gt ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, gt
; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB178_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i64_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmax x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var64, i64 %offset monotonic %old = atomicrmw max ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmax x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_max_i32_noret_monotonic(i32 %offset) nounwind { define dso_local void @test_atomic_load_max_i32_noret_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i32_noret_monotonic: ; CHECK-LABEL: test_atomic_load_max_i32_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmax w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB179_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, gt
; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB179_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i32_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmax w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw max ptr @var32, i32 %offset monotonic atomicrmw max ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmax w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_max_i64_noret_monotonic(i64 %offset) nounwind { define dso_local void @test_atomic_load_max_i64_noret_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i64_noret_monotonic: ; CHECK-LABEL: test_atomic_load_max_i64_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmax x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB180_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, gt ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, gt
; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB180_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i64_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmax x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw max ptr @var64, i64 %offset monotonic atomicrmw max ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmax x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_max_i8_release(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_max_i8_release(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i8_release: ; CHECK-LABEL: test_atomic_load_max_i8_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsmaxlb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i8_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i8_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: adrp x9, var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB181_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrb w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrb w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxtb w8, w10 ; OUTLINE-ATOMICS-NEXT: sxtb w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB181_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i8_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsmaxlb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var8, i8 %offset release %old = atomicrmw max ptr @var8, i8 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsmaxlb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_max_i16_release(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_max_i16_release(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i16_release: ; CHECK-LABEL: test_atomic_load_max_i16_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsmaxlh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i16_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i16_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: adrp x9, var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB182_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrh w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrh w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxth w8, w10 ; OUTLINE-ATOMICS-NEXT: sxth w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB182_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i16_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsmaxlh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var16, i16 %offset release %old = atomicrmw max ptr @var16, i16 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsmaxlh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_max_i32_release(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_max_i32_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i32_release: ; CHECK-LABEL: test_atomic_load_max_i32_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmaxl w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB183_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB183_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i32_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmaxl w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var32, i32 %offset release %old = atomicrmw max ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmaxl w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_max_i64_release(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_max_i64_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i64_release: ; CHECK-LABEL: test_atomic_load_max_i64_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmaxl x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB184_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, gt ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB184_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i64_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmaxl x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var64, i64 %offset release %old = atomicrmw max ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmaxl x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_max_i32_noret_release(i32 %offset) nounwind { define dso_local void @test_atomic_load_max_i32_noret_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i32_noret_release: ; CHECK-LABEL: test_atomic_load_max_i32_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmaxl w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB185_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB185_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i32_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmaxl w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw max ptr @var32, i32 %offset release atomicrmw max ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmaxl w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_max_i64_noret_release(i64 %offset) nounwind { define dso_local void @test_atomic_load_max_i64_noret_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i64_noret_release: ; CHECK-LABEL: test_atomic_load_max_i64_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmaxl x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB186_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, gt ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB186_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i64_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmaxl x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw max ptr @var64, i64 %offset release atomicrmw max ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmaxl x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_max_i8_seq_cst(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_max_i8_seq_cst(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i8_seq_cst: ; CHECK-LABEL: test_atomic_load_max_i8_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsmaxalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i8_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i8_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: adrp x9, var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB187_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxtb w8, w10 ; OUTLINE-ATOMICS-NEXT: sxtb w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB187_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i8_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsmaxalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var8, i8 %offset seq_cst %old = atomicrmw max ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsmaxalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_max_i16_seq_cst(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_max_i16_seq_cst(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i16_seq_cst: ; CHECK-LABEL: test_atomic_load_max_i16_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsmaxalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i16_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i16_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: adrp x9, var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB188_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxth w8, w10 ; OUTLINE-ATOMICS-NEXT: sxth w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB188_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i16_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsmaxalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var16, i16 %offset seq_cst %old = atomicrmw max ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsmaxalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_max_i32_seq_cst(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_max_i32_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i32_seq_cst: ; CHECK-LABEL: test_atomic_load_max_i32_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmaxal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB189_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB189_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i32_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmaxal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var32, i32 %offset seq_cst %old = atomicrmw max ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmaxal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_max_i64_seq_cst(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_max_i64_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i64_seq_cst: ; CHECK-LABEL: test_atomic_load_max_i64_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmaxal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB190_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, gt ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB190_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i64_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmaxal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw max ptr @var64, i64 %offset seq_cst %old = atomicrmw max ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmaxal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_max_i32_noret_seq_cst(i32 %offset) nounwind { define dso_local void @test_atomic_load_max_i32_noret_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i32_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_max_i32_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmaxal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i32_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB191_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, gt ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB191_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i32_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmaxal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw max ptr @var32, i32 %offset seq_cst atomicrmw max ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmaxal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_max_i64_noret_seq_cst(i64 %offset) nounwind { define dso_local void @test_atomic_load_max_i64_noret_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_max_i64_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_max_i64_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmaxal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_max_i64_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB192_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, gt ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, gt
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB192_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_max_i64_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmaxal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw max ptr @var64, i64 %offset seq_cst atomicrmw max ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmaxal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_min_i8_acq_rel(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_min_i8_acq_rel(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i8_acq_rel: ; CHECK-LABEL: test_atomic_load_min_i8_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsminalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i8_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i8_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: adrp x9, var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB193_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxtb w8, w10 ; OUTLINE-ATOMICS-NEXT: sxtb w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB193_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i8_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsminalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var8, i8 %offset acq_rel %old = atomicrmw min ptr @var8, i8 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsminalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_min_i16_acq_rel(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_min_i16_acq_rel(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i16_acq_rel: ; CHECK-LABEL: test_atomic_load_min_i16_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsminalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i16_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i16_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: adrp x9, var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB194_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxth w8, w10 ; OUTLINE-ATOMICS-NEXT: sxth w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB194_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i16_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsminalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var16, i16 %offset acq_rel %old = atomicrmw min ptr @var16, i16 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsminalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_min_i32_acq_rel(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_min_i32_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i32_acq_rel: ; CHECK-LABEL: test_atomic_load_min_i32_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsminal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB195_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, le
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB195_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i32_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsminal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var32, i32 %offset acq_rel %old = atomicrmw min ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsminal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_min_i64_acq_rel(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_min_i64_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i64_acq_rel: ; CHECK-LABEL: test_atomic_load_min_i64_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsminal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB196_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, le ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, le
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB196_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i64_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsminal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var64, i64 %offset acq_rel %old = atomicrmw min ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsminal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_min_i32_noret_acq_rel(i32 %offset) nounwind { define dso_local void @test_atomic_load_min_i32_noret_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i32_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_min_i32_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsminal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB197_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, le ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, le
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB197_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i32_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsminal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw min ptr @var32, i32 %offset acq_rel atomicrmw min ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsminal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_min_i64_noret_acq_rel(i64 %offset) nounwind { define dso_local void @test_atomic_load_min_i64_noret_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i64_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_min_i64_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsminal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB198_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, le ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, le
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB198_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i64_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsminal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw min ptr @var64, i64 %offset acq_rel atomicrmw min ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsminal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_min_i8_acquire(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_min_i8_acquire(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i8_acquire: ; CHECK-LABEL: test_atomic_load_min_i8_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsminab w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i8_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i8_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: adrp x9, var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB199_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxtb w8, w10 ; OUTLINE-ATOMICS-NEXT: sxtb w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le
; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB199_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i8_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsminab w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var8, i8 %offset acquire %old = atomicrmw min ptr @var8, i8 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsminab w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_min_i16_acquire(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_min_i16_acquire(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i16_acquire: ; CHECK-LABEL: test_atomic_load_min_i16_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsminah w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i16_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i16_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: adrp x9, var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB200_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxth w8, w10 ; OUTLINE-ATOMICS-NEXT: sxth w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le
; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB200_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i16_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsminah w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var16, i16 %offset acquire %old = atomicrmw min ptr @var16, i16 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsminah w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_min_i32_acquire(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_min_i32_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i32_acquire: ; CHECK-LABEL: test_atomic_load_min_i32_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmina w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB201_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, le
; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB201_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i32_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmina w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var32, i32 %offset acquire %old = atomicrmw min ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmina w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_min_i64_acquire(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_min_i64_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i64_acquire: ; CHECK-LABEL: test_atomic_load_min_i64_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmina x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB202_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, le ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, le
; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB202_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i64_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmina x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var64, i64 %offset acquire %old = atomicrmw min ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmina x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_min_i32_noret_acquire(i32 %offset) nounwind { define dso_local void @test_atomic_load_min_i32_noret_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i32_noret_acquire: ; CHECK-LABEL: test_atomic_load_min_i32_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmina w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB203_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, le ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, le
; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB203_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i32_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmina w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw min ptr @var32, i32 %offset acquire atomicrmw min ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmina w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_min_i64_noret_acquire(i64 %offset) nounwind { define dso_local void @test_atomic_load_min_i64_noret_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i64_noret_acquire: ; CHECK-LABEL: test_atomic_load_min_i64_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmina x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB204_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, le ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, le
; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB204_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i64_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmina x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw min ptr @var64, i64 %offset acquire atomicrmw min ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmina x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_min_i8_monotonic(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_min_i8_monotonic(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i8_monotonic: ; CHECK-LABEL: test_atomic_load_min_i8_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsminb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i8_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i8_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: adrp x9, var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB205_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrb w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrb w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxtb w8, w10 ; OUTLINE-ATOMICS-NEXT: sxtb w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le
; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB205_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i8_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsminb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var8, i8 %offset monotonic %old = atomicrmw min ptr @var8, i8 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsminb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_min_i16_monotonic(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_min_i16_monotonic(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i16_monotonic: ; CHECK-LABEL: test_atomic_load_min_i16_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsminh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i16_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i16_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: adrp x9, var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB206_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrh w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrh w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxth w8, w10 ; OUTLINE-ATOMICS-NEXT: sxth w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le
; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB206_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i16_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsminh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var16, i16 %offset monotonic %old = atomicrmw min ptr @var16, i16 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsminh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_min_i32_monotonic(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_min_i32_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i32_monotonic: ; CHECK-LABEL: test_atomic_load_min_i32_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmin w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB207_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, le
; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB207_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i32_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmin w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var32, i32 %offset monotonic %old = atomicrmw min ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmin w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_min_i64_monotonic(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_min_i64_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i64_monotonic: ; CHECK-LABEL: test_atomic_load_min_i64_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmin x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB208_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, le ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, le
; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB208_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i64_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmin x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var64, i64 %offset monotonic %old = atomicrmw min ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmin x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_min_i32_noret_monotonic(i32 %offset) nounwind { define dso_local void @test_atomic_load_min_i32_noret_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i32_noret_monotonic: ; CHECK-LABEL: test_atomic_load_min_i32_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsmin w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB209_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, le ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, le
; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB209_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i32_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsmin w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw min ptr @var32, i32 %offset monotonic atomicrmw min ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsmin w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_min_i64_noret_monotonic(i64 %offset) nounwind { define dso_local void @test_atomic_load_min_i64_noret_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i64_noret_monotonic: ; CHECK-LABEL: test_atomic_load_min_i64_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsmin x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB210_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, le ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, le
; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB210_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i64_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsmin x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw min ptr @var64, i64 %offset monotonic atomicrmw min ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsmin x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_min_i8_release(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_min_i8_release(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i8_release: ; CHECK-LABEL: test_atomic_load_min_i8_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsminlb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i8_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i8_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: adrp x9, var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB211_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrb w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrb w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxtb w8, w10 ; OUTLINE-ATOMICS-NEXT: sxtb w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB211_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i8_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsminlb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var8, i8 %offset release %old = atomicrmw min ptr @var8, i8 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsminlb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_min_i16_release(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_min_i16_release(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i16_release: ; CHECK-LABEL: test_atomic_load_min_i16_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsminlh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i16_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i16_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: adrp x9, var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB212_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrh w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrh w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxth w8, w10 ; OUTLINE-ATOMICS-NEXT: sxth w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB212_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i16_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsminlh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var16, i16 %offset release %old = atomicrmw min ptr @var16, i16 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsminlh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_min_i32_release(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_min_i32_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i32_release: ; CHECK-LABEL: test_atomic_load_min_i32_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsminl w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB213_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, le
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB213_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i32_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsminl w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var32, i32 %offset release %old = atomicrmw min ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsminl w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_min_i64_release(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_min_i64_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i64_release: ; CHECK-LABEL: test_atomic_load_min_i64_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsminl x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB214_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, le ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, le
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB214_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i64_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsminl x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var64, i64 %offset release %old = atomicrmw min ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsminl x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_min_i32_noret_release(i32 %offset) nounwind { define dso_local void @test_atomic_load_min_i32_noret_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i32_noret_release: ; CHECK-LABEL: test_atomic_load_min_i32_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsminl w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB215_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, le ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, le
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB215_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i32_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsminl w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw min ptr @var32, i32 %offset release atomicrmw min ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsminl w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_min_i64_noret_release(i64 %offset) nounwind { define dso_local void @test_atomic_load_min_i64_noret_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i64_noret_release: ; CHECK-LABEL: test_atomic_load_min_i64_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsminl x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB216_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, le ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, le
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB216_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i64_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsminl x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw min ptr @var64, i64 %offset release atomicrmw min ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsminl x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_min_i8_seq_cst(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_min_i8_seq_cst(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i8_seq_cst: ; CHECK-LABEL: test_atomic_load_min_i8_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsminalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i8_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i8_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: adrp x9, var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB217_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxtb w8, w10 ; OUTLINE-ATOMICS-NEXT: sxtb w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxtb
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB217_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i8_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsminalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var8, i8 %offset seq_cst %old = atomicrmw min ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsminalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_min_i16_seq_cst(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_min_i16_seq_cst(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i16_seq_cst: ; CHECK-LABEL: test_atomic_load_min_i16_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsminalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i16_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i16_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: adrp x9, var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB218_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w10, [x9]
; OUTLINE-ATOMICS-NEXT: sxth w8, w10 ; OUTLINE-ATOMICS-NEXT: sxth w8, w10
; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth ; OUTLINE-ATOMICS-NEXT: cmp w8, w0, sxth
; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w10, w0, le
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB218_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i16_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsminalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var16, i16 %offset seq_cst %old = atomicrmw min ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsminalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_min_i32_seq_cst(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_min_i32_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i32_seq_cst: ; CHECK-LABEL: test_atomic_load_min_i32_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsminal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB219_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, le ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, le
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB219_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i32_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsminal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var32, i32 %offset seq_cst %old = atomicrmw min ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsminal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_min_i64_seq_cst(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_min_i64_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i64_seq_cst: ; CHECK-LABEL: test_atomic_load_min_i64_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsminal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB220_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, le ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, le
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB220_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i64_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsminal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw min ptr @var64, i64 %offset seq_cst %old = atomicrmw min ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsminal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_min_i32_noret_seq_cst(i32 %offset) nounwind { define dso_local void @test_atomic_load_min_i32_noret_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i32_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_min_i32_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsminal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i32_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB221_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, le ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, le
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB221_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i32_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsminal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw min ptr @var32, i32 %offset seq_cst atomicrmw min ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsminal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_min_i64_noret_seq_cst(i64 %offset) nounwind { define dso_local void @test_atomic_load_min_i64_noret_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_min_i64_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_min_i64_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsminal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_min_i64_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB222_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, le ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, le
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB222_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_min_i64_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsminal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw min ptr @var64, i64 %offset seq_cst atomicrmw min ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsminal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_or_i8_acq_rel(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_or_i8_acq_rel(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i8_acq_rel: ; CHECK-LABEL: test_atomic_load_or_i8_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsetalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i8_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i8_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i8_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsetalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var8, i8 %offset acq_rel %old = atomicrmw or ptr @var8, i8 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsetalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_or_i16_acq_rel(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_or_i16_acq_rel(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i16_acq_rel: ; CHECK-LABEL: test_atomic_load_or_i16_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsetalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i16_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i16_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i16_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsetalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var16, i16 %offset acq_rel %old = atomicrmw or ptr @var16, i16 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsetalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_or_i32_acq_rel(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_or_i32_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i32_acq_rel: ; CHECK-LABEL: test_atomic_load_or_i32_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsetal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i32_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsetal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var32, i32 %offset acq_rel %old = atomicrmw or ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsetal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_or_i64_acq_rel(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_or_i64_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i64_acq_rel: ; CHECK-LABEL: test_atomic_load_or_i64_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsetal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i64_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsetal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var64, i64 %offset acq_rel %old = atomicrmw or ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsetal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_or_i32_noret_acq_rel(i32 %offset) nounwind { define dso_local void @test_atomic_load_or_i32_noret_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i32_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_or_i32_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsetal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i32_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsetal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw or ptr @var32, i32 %offset acq_rel atomicrmw or ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsetal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_or_i64_noret_acq_rel(i64 %offset) nounwind { define dso_local void @test_atomic_load_or_i64_noret_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i64_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_or_i64_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsetal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i64_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsetal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw or ptr @var64, i64 %offset acq_rel atomicrmw or ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsetal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_or_i8_acquire(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_or_i8_acquire(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i8_acquire: ; CHECK-LABEL: test_atomic_load_or_i8_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsetab w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i8_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i8_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset1_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset1_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i8_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsetab w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var8, i8 %offset acquire %old = atomicrmw or ptr @var8, i8 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsetab w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_or_i16_acquire(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_or_i16_acquire(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i16_acquire: ; CHECK-LABEL: test_atomic_load_or_i16_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsetah w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i16_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i16_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset2_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset2_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i16_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsetah w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var16, i16 %offset acquire %old = atomicrmw or ptr @var16, i16 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsetah w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_or_i32_acquire(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_or_i32_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i32_acquire: ; CHECK-LABEL: test_atomic_load_or_i32_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldseta w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i32_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldseta w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var32, i32 %offset acquire %old = atomicrmw or ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldseta w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_or_i64_acquire(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_or_i64_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i64_acquire: ; CHECK-LABEL: test_atomic_load_or_i64_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldseta x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i64_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldseta x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var64, i64 %offset acquire %old = atomicrmw or ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldseta x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_or_i32_noret_acquire(i32 %offset) nounwind { define dso_local void @test_atomic_load_or_i32_noret_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i32_noret_acquire: ; CHECK-LABEL: test_atomic_load_or_i32_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldseta w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i32_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldseta w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw or ptr @var32, i32 %offset acquire atomicrmw or ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldseta w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_or_i64_noret_acquire(i64 %offset) nounwind { define dso_local void @test_atomic_load_or_i64_noret_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i64_noret_acquire: ; CHECK-LABEL: test_atomic_load_or_i64_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldseta x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i64_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldseta x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw or ptr @var64, i64 %offset acquire atomicrmw or ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldseta x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_or_i8_monotonic(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_or_i8_monotonic(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i8_monotonic: ; CHECK-LABEL: test_atomic_load_or_i8_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsetb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i8_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i8_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset1_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset1_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i8_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsetb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var8, i8 %offset monotonic %old = atomicrmw or ptr @var8, i8 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsetb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_or_i16_monotonic(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_or_i16_monotonic(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i16_monotonic: ; CHECK-LABEL: test_atomic_load_or_i16_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldseth w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i16_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i16_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset2_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset2_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i16_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldseth w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var16, i16 %offset monotonic %old = atomicrmw or ptr @var16, i16 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldseth w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_or_i32_monotonic(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_or_i32_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i32_monotonic: ; CHECK-LABEL: test_atomic_load_or_i32_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldset w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i32_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldset w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var32, i32 %offset monotonic %old = atomicrmw or ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldset w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_or_i64_monotonic(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_or_i64_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i64_monotonic: ; CHECK-LABEL: test_atomic_load_or_i64_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldset x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i64_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldset x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var64, i64 %offset monotonic %old = atomicrmw or ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldset x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_or_i32_noret_monotonic(i32 %offset) nounwind { define dso_local void @test_atomic_load_or_i32_noret_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i32_noret_monotonic: ; CHECK-LABEL: test_atomic_load_or_i32_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldset w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i32_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldset w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw or ptr @var32, i32 %offset monotonic atomicrmw or ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldset w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_or_i64_noret_monotonic(i64 %offset) nounwind { define dso_local void @test_atomic_load_or_i64_noret_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i64_noret_monotonic: ; CHECK-LABEL: test_atomic_load_or_i64_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldset x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i64_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldset x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw or ptr @var64, i64 %offset monotonic atomicrmw or ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldset x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_or_i8_release(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_or_i8_release(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i8_release: ; CHECK-LABEL: test_atomic_load_or_i8_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsetlb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i8_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i8_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset1_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset1_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i8_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsetlb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var8, i8 %offset release %old = atomicrmw or ptr @var8, i8 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsetlb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_or_i16_release(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_or_i16_release(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i16_release: ; CHECK-LABEL: test_atomic_load_or_i16_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsetlh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i16_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i16_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset2_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset2_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i16_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsetlh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var16, i16 %offset release %old = atomicrmw or ptr @var16, i16 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsetlh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_or_i32_release(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_or_i32_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i32_release: ; CHECK-LABEL: test_atomic_load_or_i32_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsetl w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i32_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsetl w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var32, i32 %offset release %old = atomicrmw or ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsetl w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_or_i64_release(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_or_i64_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i64_release: ; CHECK-LABEL: test_atomic_load_or_i64_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsetl x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i64_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsetl x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var64, i64 %offset release %old = atomicrmw or ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsetl x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_or_i32_noret_release(i32 %offset) nounwind { define dso_local void @test_atomic_load_or_i32_noret_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i32_noret_release: ; CHECK-LABEL: test_atomic_load_or_i32_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsetl w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i32_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsetl w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw or ptr @var32, i32 %offset release atomicrmw or ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsetl w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_or_i64_noret_release(i64 %offset) nounwind { define dso_local void @test_atomic_load_or_i64_noret_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i64_noret_release: ; CHECK-LABEL: test_atomic_load_or_i64_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsetl x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i64_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsetl x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw or ptr @var64, i64 %offset release atomicrmw or ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsetl x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_or_i8_seq_cst(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_or_i8_seq_cst(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i8_seq_cst: ; CHECK-LABEL: test_atomic_load_or_i8_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldsetalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i8_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i8_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i8_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldsetalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var8, i8 %offset seq_cst %old = atomicrmw or ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldsetalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_or_i16_seq_cst(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_or_i16_seq_cst(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i16_seq_cst: ; CHECK-LABEL: test_atomic_load_or_i16_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldsetalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i16_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i16_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i16_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldsetalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var16, i16 %offset seq_cst %old = atomicrmw or ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldsetalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_or_i32_seq_cst(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_or_i32_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i32_seq_cst: ; CHECK-LABEL: test_atomic_load_or_i32_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsetal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i32_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsetal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var32, i32 %offset seq_cst %old = atomicrmw or ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsetal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_or_i64_seq_cst(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_or_i64_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i64_seq_cst: ; CHECK-LABEL: test_atomic_load_or_i64_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsetal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i64_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsetal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw or ptr @var64, i64 %offset seq_cst %old = atomicrmw or ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsetal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_or_i32_noret_seq_cst(i32 %offset) nounwind { define dso_local void @test_atomic_load_or_i32_noret_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i32_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_or_i32_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldsetal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i32_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i32_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldsetal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw or ptr @var32, i32 %offset seq_cst atomicrmw or ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldsetal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_or_i64_noret_seq_cst(i64 %offset) nounwind { define dso_local void @test_atomic_load_or_i64_noret_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_or_i64_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_or_i64_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldsetal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_or_i64_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldset8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_or_i64_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldsetal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw or ptr @var64, i64 %offset seq_cst atomicrmw or ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldsetal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_sub_i8_acq_rel(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_sub_i8_acq_rel(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i8_acq_rel: ; CHECK-LABEL: test_atomic_load_sub_i8_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldaddalb w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i8_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldaddalb w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var8, i8 %offset acq_rel %old = atomicrmw sub ptr @var8, i8 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddalb w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_sub_i16_acq_rel(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_sub_i16_acq_rel(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i16_acq_rel: ; CHECK-LABEL: test_atomic_load_sub_i16_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldaddalh w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i16_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldaddalh w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var16, i16 %offset acq_rel %old = atomicrmw sub ptr @var16, i16 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddalh w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_sub_i32_acq_rel(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_sub_i32_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_acq_rel: ; CHECK-LABEL: test_atomic_load_sub_i32_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldaddal w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldaddal w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var32, i32 %offset acq_rel %old = atomicrmw sub ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_sub_i64_acq_rel(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_sub_i64_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_acq_rel: ; CHECK-LABEL: test_atomic_load_sub_i64_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: neg x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldaddal x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldaddal x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var64, i64 %offset acq_rel %old = atomicrmw sub ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: neg x[[NEG:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x[[NEG]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_sub_i32_noret_acq_rel(i32 %offset) nounwind { define dso_local void @test_atomic_load_sub_i32_noret_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_sub_i32_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldaddal w8, w8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldaddal w8, w8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw sub ptr @var32, i32 %offset acq_rel atomicrmw sub ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_sub_i64_noret_acq_rel(i64 %offset) nounwind { define dso_local void @test_atomic_load_sub_i64_noret_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_sub_i64_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: neg x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldaddal x8, x8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldaddal x8, x8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw sub ptr @var64, i64 %offset acq_rel atomicrmw sub ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: neg x[[NEG:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x[[NEG]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_sub_i8_acquire(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_sub_i8_acquire(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i8_acquire: ; CHECK-LABEL: test_atomic_load_sub_i8_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldaddab w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i8_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldaddab w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var8, i8 %offset acquire %old = atomicrmw sub ptr @var8, i8 %offset acquire
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddab w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_sub_i16_acquire(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_sub_i16_acquire(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i16_acquire: ; CHECK-LABEL: test_atomic_load_sub_i16_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldaddah w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i16_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldaddah w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var16, i16 %offset acquire %old = atomicrmw sub ptr @var16, i16 %offset acquire
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddah w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_sub_i32_acquire(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_sub_i32_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_acquire: ; CHECK-LABEL: test_atomic_load_sub_i32_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldadda w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldadda w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var32, i32 %offset acquire %old = atomicrmw sub ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldadda w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_sub_i64_acquire(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_sub_i64_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_acquire: ; CHECK-LABEL: test_atomic_load_sub_i64_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: neg x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldadda x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldadda x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var64, i64 %offset acquire %old = atomicrmw sub ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: neg x[[NEG:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldadda x[[NEG]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_sub_i32_noret_acquire(i32 %offset) nounwind { define dso_local void @test_atomic_load_sub_i32_noret_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_noret_acquire: ; CHECK-LABEL: test_atomic_load_sub_i32_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldadda w8, w8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldadda w8, w8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw sub ptr @var32, i32 %offset acquire atomicrmw sub ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldadda w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_sub_i64_noret_acquire(i64 %offset) nounwind { define dso_local void @test_atomic_load_sub_i64_noret_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_noret_acquire: ; CHECK-LABEL: test_atomic_load_sub_i64_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: neg x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldadda x8, x8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldadda x8, x8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw sub ptr @var64, i64 %offset acquire atomicrmw sub ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: neg x[[NEG:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldadda x[[NEG]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_sub_i8_monotonic(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_sub_i8_monotonic(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i8_monotonic: ; CHECK-LABEL: test_atomic_load_sub_i8_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldaddb w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i8_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldaddb w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var8, i8 %offset monotonic %old = atomicrmw sub ptr @var8, i8 %offset monotonic
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddb w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_sub_i16_monotonic(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_sub_i16_monotonic(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i16_monotonic: ; CHECK-LABEL: test_atomic_load_sub_i16_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldaddh w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i16_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldaddh w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var16, i16 %offset monotonic %old = atomicrmw sub ptr @var16, i16 %offset monotonic
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddh w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_sub_i32_monotonic(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_sub_i32_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_monotonic: ; CHECK-LABEL: test_atomic_load_sub_i32_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldadd w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldadd w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var32, i32 %offset monotonic %old = atomicrmw sub ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldadd w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_sub_i64_monotonic(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_sub_i64_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_monotonic: ; CHECK-LABEL: test_atomic_load_sub_i64_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: neg x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldadd x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldadd x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var64, i64 %offset monotonic %old = atomicrmw sub ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: neg x[[NEG:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldadd x[[NEG]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_sub_i32_noret_monotonic(i32 %offset) nounwind { define dso_local void @test_atomic_load_sub_i32_noret_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_noret_monotonic: ; CHECK-LABEL: test_atomic_load_sub_i32_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldadd w8, w8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldadd w8, w8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw sub ptr @var32, i32 %offset monotonic atomicrmw sub ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldadd w{{[0-9]+}}, w[[NEW:[1-9][0-9]*]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_sub_i64_noret_monotonic(i64 %offset) nounwind { define dso_local void @test_atomic_load_sub_i64_noret_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_noret_monotonic: ; CHECK-LABEL: test_atomic_load_sub_i64_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: neg x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldadd x8, x8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldadd x8, x8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw sub ptr @var64, i64 %offset monotonic atomicrmw sub ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: neg x[[NEG:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldadd x{{[0-9]+}}, x[[NEW:[1-9][0-9]*]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_sub_i8_release(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_sub_i8_release(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i8_release: ; CHECK-LABEL: test_atomic_load_sub_i8_release:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldaddlb w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i8_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldaddlb w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var8, i8 %offset release %old = atomicrmw sub ptr @var8, i8 %offset release
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddlb w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_sub_i16_release(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_sub_i16_release(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i16_release: ; CHECK-LABEL: test_atomic_load_sub_i16_release:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldaddlh w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i16_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldaddlh w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var16, i16 %offset release %old = atomicrmw sub ptr @var16, i16 %offset release
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddlh w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_sub_i32_release(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_sub_i32_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_release: ; CHECK-LABEL: test_atomic_load_sub_i32_release:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldaddl w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldaddl w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var32, i32 %offset release %old = atomicrmw sub ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddl w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_sub_i64_release(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_sub_i64_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_release: ; CHECK-LABEL: test_atomic_load_sub_i64_release:
; CHECK: // %bb.0:
; CHECK-NEXT: neg x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldaddl x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldaddl x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var64, i64 %offset release %old = atomicrmw sub ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: neg x[[NEG:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddl x[[NEG]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_sub_i32_noret_release(i32 %offset) nounwind { define dso_local void @test_atomic_load_sub_i32_noret_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_noret_release: ; CHECK-LABEL: test_atomic_load_sub_i32_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldaddl w8, w8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldaddl w8, w8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw sub ptr @var32, i32 %offset release atomicrmw sub ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddl w{{[0-9]*}}, w[[NEW:[1-9][0-9]*]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_sub_i64_noret_release(i64 %offset) nounwind { define dso_local void @test_atomic_load_sub_i64_noret_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_noret_release: ; CHECK-LABEL: test_atomic_load_sub_i64_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: neg x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldaddl x8, x8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldaddl x8, x8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw sub ptr @var64, i64 %offset release atomicrmw sub ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: neg x[[NEG:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddl x{{[0-9]*}}, x[[NEW:[1-9][0-9]*]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_sub_i8_seq_cst(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_sub_i8_seq_cst(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i8_seq_cst: ; CHECK-LABEL: test_atomic_load_sub_i8_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var8
; CHECK-NEXT: add x9, x9, :lo12:var8
; CHECK-NEXT: ldaddalb w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i8_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i8_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var8
; CHECK-REG-NEXT: add x9, x9, :lo12:var8
; CHECK-REG-NEXT: ldaddalb w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var8, i8 %offset seq_cst %old = atomicrmw sub ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldaddalb w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_sub_i16_seq_cst(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_sub_i16_seq_cst(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i16_seq_cst: ; CHECK-LABEL: test_atomic_load_sub_i16_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var16
; CHECK-NEXT: add x9, x9, :lo12:var16
; CHECK-NEXT: ldaddalh w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i16_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i16_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var16
; CHECK-REG-NEXT: add x9, x9, :lo12:var16
; CHECK-REG-NEXT: ldaddalh w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var16, i16 %offset seq_cst %old = atomicrmw sub ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldaddalh w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_sub_i32_seq_cst(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_sub_i32_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_seq_cst: ; CHECK-LABEL: test_atomic_load_sub_i32_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldaddal w8, w0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldaddal w8, w0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var32, i32 %offset seq_cst %old = atomicrmw sub ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_sub_i64_seq_cst(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_sub_i64_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_seq_cst: ; CHECK-LABEL: test_atomic_load_sub_i64_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: neg x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldaddal x8, x0, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldaddal x8, x0, [x9]
; CHECK-REG-NEXT: ret
%old = atomicrmw sub ptr @var64, i64 %offset seq_cst %old = atomicrmw sub ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: neg x[[NEG:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x[[NEG]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_sub_i32_noret_seq_cst(i32 %offset) nounwind { define dso_local void @test_atomic_load_sub_i32_noret_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i32_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_sub_i32_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: neg w8, w0
; CHECK-NEXT: adrp x9, var32
; CHECK-NEXT: add x9, x9, :lo12:var32
; CHECK-NEXT: ldaddal w8, w8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i32_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i32_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg w8, w0
; CHECK-REG-NEXT: adrp x9, var32
; CHECK-REG-NEXT: add x9, x9, :lo12:var32
; CHECK-REG-NEXT: ldaddal w8, w8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw sub ptr @var32, i32 %offset seq_cst atomicrmw sub ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: neg w[[NEG:[0-9]+]], w[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldaddal w[[NEG]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_sub_i64_noret_seq_cst(i64 %offset) nounwind { define dso_local void @test_atomic_load_sub_i64_noret_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_sub_i64_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_sub_i64_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: neg x8, x0
; CHECK-NEXT: adrp x9, var64
; CHECK-NEXT: add x9, x9, :lo12:var64
; CHECK-NEXT: ldaddal x8, x8, [x9]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_sub_i64_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldadd8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_sub_i64_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: neg x8, x0
; CHECK-REG-NEXT: adrp x9, var64
; CHECK-REG-NEXT: add x9, x9, :lo12:var64
; CHECK-REG-NEXT: ldaddal x8, x8, [x9]
; CHECK-REG-NEXT: ret
atomicrmw sub ptr @var64, i64 %offset seq_cst atomicrmw sub ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: neg x[[NEG:[0-9]+]], x[[OLD:[0-9]+]]
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldaddal x[[NEG]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_xchg_i8_acq_rel(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_xchg_i8_acq_rel(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i8_acq_rel: ; CHECK-LABEL: test_atomic_load_xchg_i8_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: swpalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i8_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i8_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i8_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: swpalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var8, i8 %offset acq_rel %old = atomicrmw xchg ptr @var8, i8 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: swpalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_xchg_i16_acq_rel(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_xchg_i16_acq_rel(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i16_acq_rel: ; CHECK-LABEL: test_atomic_load_xchg_i16_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: swpalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i16_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i16_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i16_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: swpalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var16, i16 %offset acq_rel %old = atomicrmw xchg ptr @var16, i16 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: swpalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_xchg_i32_acq_rel(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_xchg_i32_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i32_acq_rel: ; CHECK-LABEL: test_atomic_load_xchg_i32_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: swpal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i32_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: swpal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var32, i32 %offset acq_rel %old = atomicrmw xchg ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: swpal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_xchg_i64_acq_rel(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_xchg_i64_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i64_acq_rel: ; CHECK-LABEL: test_atomic_load_xchg_i64_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: swpal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i64_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: swpal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var64, i64 %offset acq_rel %old = atomicrmw xchg ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: swpal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_xchg_i32_noret_acq_rel(i32 %offset) nounwind { define dso_local void @test_atomic_load_xchg_i32_noret_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i32_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_xchg_i32_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: swpal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i32_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: swpal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xchg ptr @var32, i32 %offset acq_rel atomicrmw xchg ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: swpal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_xchg_i64_noret_acq_rel(i64 %offset) nounwind { define dso_local void @test_atomic_load_xchg_i64_noret_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i64_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_xchg_i64_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: swpal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i64_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: swpal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xchg ptr @var64, i64 %offset acq_rel atomicrmw xchg ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: swpal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_xchg_i8_acquire(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_xchg_i8_acquire(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i8_acquire: ; CHECK-LABEL: test_atomic_load_xchg_i8_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: swpab w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i8_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i8_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp1_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp1_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i8_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: swpab w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var8, i8 %offset acquire %old = atomicrmw xchg ptr @var8, i8 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: swpab w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_xchg_i16_acquire(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_xchg_i16_acquire(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i16_acquire: ; CHECK-LABEL: test_atomic_load_xchg_i16_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: swpah w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i16_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i16_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp2_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp2_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i16_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: swpah w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var16, i16 %offset acquire %old = atomicrmw xchg ptr @var16, i16 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: swpah w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_xchg_i32_acquire(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_xchg_i32_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i32_acquire: ; CHECK-LABEL: test_atomic_load_xchg_i32_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: swpa w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i32_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: swpa w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var32, i32 %offset acquire %old = atomicrmw xchg ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: swpa w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_xchg_i64_acquire(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_xchg_i64_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i64_acquire: ; CHECK-LABEL: test_atomic_load_xchg_i64_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: swpa x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i64_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: swpa x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var64, i64 %offset acquire %old = atomicrmw xchg ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: swpa x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_xchg_i32_noret_acquire(i32 %offset) nounwind { define dso_local void @test_atomic_load_xchg_i32_noret_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i32_noret_acquire: ; CHECK-LABEL: test_atomic_load_xchg_i32_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: swpa w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i32_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: swpa w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xchg ptr @var32, i32 %offset acquire atomicrmw xchg ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: swpa w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_xchg_i64_noret_acquire(i64 %offset) nounwind { define dso_local void @test_atomic_load_xchg_i64_noret_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i64_noret_acquire: ; CHECK-LABEL: test_atomic_load_xchg_i64_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: swpa x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i64_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: swpa x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xchg ptr @var64, i64 %offset acquire atomicrmw xchg ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: swpa x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_xchg_i8_monotonic(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_xchg_i8_monotonic(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i8_monotonic: ; CHECK-LABEL: test_atomic_load_xchg_i8_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: swpb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i8_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i8_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp1_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp1_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i8_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: swpb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var8, i8 %offset monotonic %old = atomicrmw xchg ptr @var8, i8 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: swpb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_xchg_i16_monotonic(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_xchg_i16_monotonic(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i16_monotonic: ; CHECK-LABEL: test_atomic_load_xchg_i16_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: swph w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i16_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i16_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp2_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp2_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i16_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: swph w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var16, i16 %offset monotonic %old = atomicrmw xchg ptr @var16, i16 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: swph w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_xchg_i32_monotonic(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_xchg_i32_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i32_monotonic: ; CHECK-LABEL: test_atomic_load_xchg_i32_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: swp w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i32_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: swp w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var32, i32 %offset monotonic %old = atomicrmw xchg ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: swp w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_xchg_i64_monotonic(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_xchg_i64_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i64_monotonic: ; CHECK-LABEL: test_atomic_load_xchg_i64_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: swp x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i64_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: swp x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var64, i64 %offset monotonic %old = atomicrmw xchg ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: swp x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_xchg_i32_noret_monotonic(i32 %offset) nounwind { define dso_local void @test_atomic_load_xchg_i32_noret_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i32_noret_monotonic: ; CHECK-LABEL: test_atomic_load_xchg_i32_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: swp w0, wzr, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i32_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: swp w0, wzr, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xchg ptr @var32, i32 %offset monotonic atomicrmw xchg ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: swp w[[OLD:[0-9]+]], w[[NEW:[0-9,a-z]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_xchg_i64_noret_monotonic(i64 %offset) nounwind { define dso_local void @test_atomic_load_xchg_i64_noret_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i64_noret_monotonic: ; CHECK-LABEL: test_atomic_load_xchg_i64_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: swp x0, xzr, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i64_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: swp x0, xzr, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xchg ptr @var64, i64 %offset monotonic atomicrmw xchg ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: swp x[[OLD:[0-9]+]], x[[NEW:[0-9,a-z]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_xchg_i8_release(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_xchg_i8_release(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i8_release: ; CHECK-LABEL: test_atomic_load_xchg_i8_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: swplb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i8_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i8_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp1_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp1_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i8_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: swplb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var8, i8 %offset release %old = atomicrmw xchg ptr @var8, i8 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: swplb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_xchg_i16_release(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_xchg_i16_release(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i16_release: ; CHECK-LABEL: test_atomic_load_xchg_i16_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: swplh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i16_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i16_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp2_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp2_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i16_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: swplh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var16, i16 %offset release %old = atomicrmw xchg ptr @var16, i16 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: swplh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_xchg_i32_release(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_xchg_i32_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i32_release: ; CHECK-LABEL: test_atomic_load_xchg_i32_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: swpl w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i32_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: swpl w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var32, i32 %offset release %old = atomicrmw xchg ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: swpl w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_xchg_i64_release(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_xchg_i64_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i64_release: ; CHECK-LABEL: test_atomic_load_xchg_i64_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: swpl x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i64_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: swpl x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var64, i64 %offset release %old = atomicrmw xchg ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: swpl x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_xchg_i32_noret_release(i32 %offset) nounwind { define dso_local void @test_atomic_load_xchg_i32_noret_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i32_noret_release: ; CHECK-LABEL: test_atomic_load_xchg_i32_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: swpl w0, wzr, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i32_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: swpl w0, wzr, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xchg ptr @var32, i32 %offset release atomicrmw xchg ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: swpl w[[OLD:[0-9]+]], w[[NEW:[0-9,a-z]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_xchg_i64_noret_release(i64 %offset) nounwind { define dso_local void @test_atomic_load_xchg_i64_noret_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i64_noret_release: ; CHECK-LABEL: test_atomic_load_xchg_i64_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: swpl x0, xzr, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i64_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: swpl x0, xzr, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xchg ptr @var64, i64 %offset release atomicrmw xchg ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: swpl x[[OLD:[0-9]+]], x[[NEW:[0-9,a-z]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_xchg_i8_seq_cst(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_xchg_i8_seq_cst(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i8_seq_cst: ; CHECK-LABEL: test_atomic_load_xchg_i8_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: swpalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i8_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i8_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i8_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: swpalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var8, i8 %offset seq_cst %old = atomicrmw xchg ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: swpalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_xchg_i16_seq_cst(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_xchg_i16_seq_cst(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i16_seq_cst: ; CHECK-LABEL: test_atomic_load_xchg_i16_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: swpalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i16_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i16_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i16_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: swpalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var16, i16 %offset seq_cst %old = atomicrmw xchg ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: swpalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_xchg_i32_seq_cst(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_xchg_i32_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i32_seq_cst: ; CHECK-LABEL: test_atomic_load_xchg_i32_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: swpal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i32_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: swpal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var32, i32 %offset seq_cst %old = atomicrmw xchg ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: swpal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_xchg_i64_seq_cst(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_xchg_i64_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i64_seq_cst: ; CHECK-LABEL: test_atomic_load_xchg_i64_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: swpal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i64_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: swpal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xchg ptr @var64, i64 %offset seq_cst %old = atomicrmw xchg ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: swpal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_xchg_i32_noret_seq_cst(i32 %offset) nounwind { define dso_local void @test_atomic_load_xchg_i32_noret_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i32_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_xchg_i32_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: swpal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i32_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i32_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: swpal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xchg ptr @var32, i32 %offset seq_cst atomicrmw xchg ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: swpal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_xchg_i64_noret_seq_cst(i64 %offset) nounwind { define dso_local void @test_atomic_load_xchg_i64_noret_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xchg_i64_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_xchg_i64_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: swpal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xchg_i64_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_swp8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xchg_i64_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: swpal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xchg ptr @var64, i64 %offset seq_cst atomicrmw xchg ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: swpal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_umax_i8_acq_rel(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_umax_i8_acq_rel(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i8_acq_rel: ; CHECK-LABEL: test_atomic_load_umax_i8_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldumaxalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i8_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i8_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xff ; OUTLINE-ATOMICS-NEXT: adrp x8, var8
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB313_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, hi
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB313_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i8_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldumaxalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var8, i8 %offset acq_rel %old = atomicrmw umax ptr @var8, i8 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldumaxalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_umax_i16_acq_rel(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_umax_i16_acq_rel(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i16_acq_rel: ; CHECK-LABEL: test_atomic_load_umax_i16_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldumaxalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i16_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i16_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xffff ; OUTLINE-ATOMICS-NEXT: adrp x8, var16
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xffff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB314_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, hi
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB314_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i16_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldumaxalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var16, i16 %offset acq_rel %old = atomicrmw umax ptr @var16, i16 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldumaxalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_umax_i32_acq_rel(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_umax_i32_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i32_acq_rel: ; CHECK-LABEL: test_atomic_load_umax_i32_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumaxal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB315_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB315_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i32_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumaxal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var32, i32 %offset acq_rel %old = atomicrmw umax ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumaxal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_umax_i64_acq_rel(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_umax_i64_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i64_acq_rel: ; CHECK-LABEL: test_atomic_load_umax_i64_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumaxal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB316_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, hi ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB316_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i64_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumaxal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var64, i64 %offset acq_rel %old = atomicrmw umax ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumaxal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_umax_i32_noret_acq_rel(i32 %offset) nounwind { define dso_local void @test_atomic_load_umax_i32_noret_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i32_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_umax_i32_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumaxal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB317_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, hi ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB317_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i32_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumaxal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umax ptr @var32, i32 %offset acq_rel atomicrmw umax ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumaxal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_umax_i64_noret_acq_rel(i64 %offset) nounwind { define dso_local void @test_atomic_load_umax_i64_noret_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i64_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_umax_i64_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumaxal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB318_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, hi ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB318_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i64_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumaxal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umax ptr @var64, i64 %offset acq_rel atomicrmw umax ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumaxal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_umax_i8_acquire(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_umax_i8_acquire(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i8_acquire: ; CHECK-LABEL: test_atomic_load_umax_i8_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldumaxab w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i8_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i8_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xff ; OUTLINE-ATOMICS-NEXT: adrp x8, var8
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB319_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, hi
; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB319_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i8_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldumaxab w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var8, i8 %offset acquire %old = atomicrmw umax ptr @var8, i8 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldumaxab w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_umax_i16_acquire(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_umax_i16_acquire(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i16_acquire: ; CHECK-LABEL: test_atomic_load_umax_i16_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldumaxah w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i16_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i16_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xffff ; OUTLINE-ATOMICS-NEXT: adrp x8, var16
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xffff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB320_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, hi
; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB320_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i16_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldumaxah w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var16, i16 %offset acquire %old = atomicrmw umax ptr @var16, i16 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldumaxah w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_umax_i32_acquire(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_umax_i32_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i32_acquire: ; CHECK-LABEL: test_atomic_load_umax_i32_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumaxa w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB321_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, hi
; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB321_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i32_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumaxa w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var32, i32 %offset acquire %old = atomicrmw umax ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumaxa w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_umax_i64_acquire(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_umax_i64_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i64_acquire: ; CHECK-LABEL: test_atomic_load_umax_i64_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumaxa x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB322_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, hi ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, hi
; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB322_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i64_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumaxa x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var64, i64 %offset acquire %old = atomicrmw umax ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumaxa x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_umax_i32_noret_acquire(i32 %offset) nounwind { define dso_local void @test_atomic_load_umax_i32_noret_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i32_noret_acquire: ; CHECK-LABEL: test_atomic_load_umax_i32_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumaxa w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB323_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, hi ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, hi
; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB323_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i32_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumaxa w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umax ptr @var32, i32 %offset acquire atomicrmw umax ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumaxa w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_umax_i64_noret_acquire(i64 %offset) nounwind { define dso_local void @test_atomic_load_umax_i64_noret_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i64_noret_acquire: ; CHECK-LABEL: test_atomic_load_umax_i64_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumaxa x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB324_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, hi ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, hi
; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB324_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i64_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumaxa x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umax ptr @var64, i64 %offset acquire atomicrmw umax ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumaxa x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_umax_i8_monotonic(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_umax_i8_monotonic(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i8_monotonic: ; CHECK-LABEL: test_atomic_load_umax_i8_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldumaxb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i8_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i8_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xff ; OUTLINE-ATOMICS-NEXT: adrp x8, var8
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB325_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrb w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrb w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, hi
; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB325_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i8_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldumaxb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var8, i8 %offset monotonic %old = atomicrmw umax ptr @var8, i8 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldumaxb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_umax_i16_monotonic(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_umax_i16_monotonic(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i16_monotonic: ; CHECK-LABEL: test_atomic_load_umax_i16_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldumaxh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i16_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i16_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xffff ; OUTLINE-ATOMICS-NEXT: adrp x8, var16
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xffff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB326_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrh w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrh w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, hi
; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB326_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i16_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldumaxh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var16, i16 %offset monotonic %old = atomicrmw umax ptr @var16, i16 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldumaxh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_umax_i32_monotonic(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_umax_i32_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i32_monotonic: ; CHECK-LABEL: test_atomic_load_umax_i32_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumax w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB327_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, hi
; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB327_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i32_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumax w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var32, i32 %offset monotonic %old = atomicrmw umax ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumax w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_umax_i64_monotonic(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_umax_i64_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i64_monotonic: ; CHECK-LABEL: test_atomic_load_umax_i64_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumax x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB328_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, hi ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, hi
; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB328_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i64_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumax x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var64, i64 %offset monotonic %old = atomicrmw umax ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumax x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_umax_i32_noret_monotonic(i32 %offset) nounwind { define dso_local void @test_atomic_load_umax_i32_noret_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i32_noret_monotonic: ; CHECK-LABEL: test_atomic_load_umax_i32_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumax w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB329_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, hi ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, hi
; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB329_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i32_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumax w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umax ptr @var32, i32 %offset monotonic atomicrmw umax ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumax w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_umax_i64_noret_monotonic(i64 %offset) nounwind { define dso_local void @test_atomic_load_umax_i64_noret_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i64_noret_monotonic: ; CHECK-LABEL: test_atomic_load_umax_i64_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumax x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB330_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, hi ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, hi
; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB330_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i64_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumax x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umax ptr @var64, i64 %offset monotonic atomicrmw umax ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumax x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_umax_i8_release(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_umax_i8_release(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i8_release: ; CHECK-LABEL: test_atomic_load_umax_i8_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldumaxlb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i8_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i8_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xff ; OUTLINE-ATOMICS-NEXT: adrp x8, var8
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB331_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrb w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrb w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, hi
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB331_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i8_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldumaxlb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var8, i8 %offset release %old = atomicrmw umax ptr @var8, i8 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldumaxlb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_umax_i16_release(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_umax_i16_release(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i16_release: ; CHECK-LABEL: test_atomic_load_umax_i16_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldumaxlh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i16_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i16_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xffff ; OUTLINE-ATOMICS-NEXT: adrp x8, var16
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xffff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB332_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrh w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrh w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, hi
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB332_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i16_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldumaxlh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var16, i16 %offset release %old = atomicrmw umax ptr @var16, i16 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldumaxlh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_umax_i32_release(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_umax_i32_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i32_release: ; CHECK-LABEL: test_atomic_load_umax_i32_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumaxl w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB333_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB333_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i32_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumaxl w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var32, i32 %offset release %old = atomicrmw umax ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumaxl w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_umax_i64_release(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_umax_i64_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i64_release: ; CHECK-LABEL: test_atomic_load_umax_i64_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumaxl x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB334_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, hi ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB334_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i64_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumaxl x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var64, i64 %offset release %old = atomicrmw umax ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumaxl x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_umax_i32_noret_release(i32 %offset) nounwind { define dso_local void @test_atomic_load_umax_i32_noret_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i32_noret_release: ; CHECK-LABEL: test_atomic_load_umax_i32_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumaxl w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB335_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, hi ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB335_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i32_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumaxl w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umax ptr @var32, i32 %offset release atomicrmw umax ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumaxl w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_umax_i64_noret_release(i64 %offset) nounwind { define dso_local void @test_atomic_load_umax_i64_noret_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i64_noret_release: ; CHECK-LABEL: test_atomic_load_umax_i64_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumaxl x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB336_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, hi ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB336_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i64_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumaxl x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umax ptr @var64, i64 %offset release atomicrmw umax ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumaxl x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_umax_i8_seq_cst(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_umax_i8_seq_cst(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i8_seq_cst: ; CHECK-LABEL: test_atomic_load_umax_i8_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldumaxalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i8_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i8_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xff ; OUTLINE-ATOMICS-NEXT: adrp x8, var8
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB337_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, hi
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB337_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i8_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldumaxalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var8, i8 %offset seq_cst %old = atomicrmw umax ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldumaxalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_umax_i16_seq_cst(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_umax_i16_seq_cst(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i16_seq_cst: ; CHECK-LABEL: test_atomic_load_umax_i16_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldumaxalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i16_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i16_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xffff ; OUTLINE-ATOMICS-NEXT: adrp x8, var16
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xffff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB338_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, hi
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB338_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i16_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldumaxalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var16, i16 %offset seq_cst %old = atomicrmw umax ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldumaxalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_umax_i32_seq_cst(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_umax_i32_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i32_seq_cst: ; CHECK-LABEL: test_atomic_load_umax_i32_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumaxal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB339_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, hi ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB339_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i32_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumaxal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var32, i32 %offset seq_cst %old = atomicrmw umax ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumaxal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_umax_i64_seq_cst(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_umax_i64_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i64_seq_cst: ; CHECK-LABEL: test_atomic_load_umax_i64_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumaxal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB340_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, hi ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB340_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i64_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumaxal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umax ptr @var64, i64 %offset seq_cst %old = atomicrmw umax ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumaxal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_umax_i32_noret_seq_cst(i32 %offset) nounwind { define dso_local void @test_atomic_load_umax_i32_noret_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i32_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_umax_i32_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumaxal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i32_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB341_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, hi ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB341_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i32_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumaxal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umax ptr @var32, i32 %offset seq_cst atomicrmw umax ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumaxal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_umax_i64_noret_seq_cst(i64 %offset) nounwind { define dso_local void @test_atomic_load_umax_i64_noret_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umax_i64_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_umax_i64_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumaxal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umax_i64_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB342_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, hi ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, hi
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB342_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umax_i64_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumaxal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umax ptr @var64, i64 %offset seq_cst atomicrmw umax ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumaxal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_umin_i8_acq_rel(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_umin_i8_acq_rel(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i8_acq_rel: ; CHECK-LABEL: test_atomic_load_umin_i8_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: lduminalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i8_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i8_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xff ; OUTLINE-ATOMICS-NEXT: adrp x8, var8
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB343_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, ls
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB343_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i8_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: lduminalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var8, i8 %offset acq_rel %old = atomicrmw umin ptr @var8, i8 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: lduminalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_umin_i16_acq_rel(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_umin_i16_acq_rel(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i16_acq_rel: ; CHECK-LABEL: test_atomic_load_umin_i16_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: lduminalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i16_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i16_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xffff ; OUTLINE-ATOMICS-NEXT: adrp x8, var16
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xffff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB344_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, ls
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB344_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i16_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: lduminalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var16, i16 %offset acq_rel %old = atomicrmw umin ptr @var16, i16 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: lduminalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_umin_i32_acq_rel(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_umin_i32_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i32_acq_rel: ; CHECK-LABEL: test_atomic_load_umin_i32_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: lduminal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB345_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB345_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i32_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: lduminal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var32, i32 %offset acq_rel %old = atomicrmw umin ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: lduminal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_umin_i64_acq_rel(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_umin_i64_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i64_acq_rel: ; CHECK-LABEL: test_atomic_load_umin_i64_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: lduminal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB346_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, ls ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB346_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i64_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: lduminal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var64, i64 %offset acq_rel %old = atomicrmw umin ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: lduminal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_umin_i32_noret_acq_rel(i32 %offset) nounwind { define dso_local void @test_atomic_load_umin_i32_noret_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i32_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_umin_i32_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: lduminal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB347_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, ls ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB347_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i32_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: lduminal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umin ptr @var32, i32 %offset acq_rel atomicrmw umin ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: lduminal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_umin_i64_noret_acq_rel(i64 %offset) nounwind { define dso_local void @test_atomic_load_umin_i64_noret_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i64_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_umin_i64_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: lduminal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB348_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, ls ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB348_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i64_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: lduminal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umin ptr @var64, i64 %offset acq_rel atomicrmw umin ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: lduminal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_umin_i8_acquire(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_umin_i8_acquire(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i8_acquire: ; CHECK-LABEL: test_atomic_load_umin_i8_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: lduminab w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i8_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i8_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xff ; OUTLINE-ATOMICS-NEXT: adrp x8, var8
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB349_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, ls
; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB349_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i8_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: lduminab w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var8, i8 %offset acquire %old = atomicrmw umin ptr @var8, i8 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: lduminab w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_umin_i16_acquire(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_umin_i16_acquire(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i16_acquire: ; CHECK-LABEL: test_atomic_load_umin_i16_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: lduminah w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i16_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i16_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xffff ; OUTLINE-ATOMICS-NEXT: adrp x8, var16
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xffff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB350_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, ls
; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB350_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i16_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: lduminah w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var16, i16 %offset acquire %old = atomicrmw umin ptr @var16, i16 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: lduminah w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_umin_i32_acquire(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_umin_i32_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i32_acquire: ; CHECK-LABEL: test_atomic_load_umin_i32_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumina w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB351_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, ls
; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB351_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i32_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumina w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var32, i32 %offset acquire %old = atomicrmw umin ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumina w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_umin_i64_acquire(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_umin_i64_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i64_acquire: ; CHECK-LABEL: test_atomic_load_umin_i64_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumina x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB352_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, ls ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, ls
; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB352_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i64_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumina x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var64, i64 %offset acquire %old = atomicrmw umin ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumina x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_umin_i32_noret_acquire(i32 %offset) nounwind { define dso_local void @test_atomic_load_umin_i32_noret_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i32_noret_acquire: ; CHECK-LABEL: test_atomic_load_umin_i32_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumina w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB353_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, ls ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, ls
; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB353_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i32_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumina w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umin ptr @var32, i32 %offset acquire atomicrmw umin ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumina w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_umin_i64_noret_acquire(i64 %offset) nounwind { define dso_local void @test_atomic_load_umin_i64_noret_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i64_noret_acquire: ; CHECK-LABEL: test_atomic_load_umin_i64_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumina x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB354_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, ls ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, ls
; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB354_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i64_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumina x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umin ptr @var64, i64 %offset acquire atomicrmw umin ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumina x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_umin_i8_monotonic(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_umin_i8_monotonic(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i8_monotonic: ; CHECK-LABEL: test_atomic_load_umin_i8_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: lduminb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i8_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i8_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xff ; OUTLINE-ATOMICS-NEXT: adrp x8, var8
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB355_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrb w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrb w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, ls
; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrb w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB355_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i8_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: lduminb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var8, i8 %offset monotonic %old = atomicrmw umin ptr @var8, i8 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: lduminb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_umin_i16_monotonic(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_umin_i16_monotonic(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i16_monotonic: ; CHECK-LABEL: test_atomic_load_umin_i16_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: lduminh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i16_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i16_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xffff ; OUTLINE-ATOMICS-NEXT: adrp x8, var16
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xffff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB356_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrh w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrh w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, ls
; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxrh w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB356_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i16_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: lduminh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var16, i16 %offset monotonic %old = atomicrmw umin ptr @var16, i16 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: lduminh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_umin_i32_monotonic(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_umin_i32_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i32_monotonic: ; CHECK-LABEL: test_atomic_load_umin_i32_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumin w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB357_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, ls
; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB357_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i32_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumin w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var32, i32 %offset monotonic %old = atomicrmw umin ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumin w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_umin_i64_monotonic(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_umin_i64_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i64_monotonic: ; CHECK-LABEL: test_atomic_load_umin_i64_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumin x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB358_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, ls ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, ls
; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB358_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i64_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumin x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var64, i64 %offset monotonic %old = atomicrmw umin ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumin x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_umin_i32_noret_monotonic(i32 %offset) nounwind { define dso_local void @test_atomic_load_umin_i32_noret_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i32_noret_monotonic: ; CHECK-LABEL: test_atomic_load_umin_i32_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldumin w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB359_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, ls ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, ls
; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB359_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i32_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldumin w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umin ptr @var32, i32 %offset monotonic atomicrmw umin ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldumin w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_umin_i64_noret_monotonic(i64 %offset) nounwind { define dso_local void @test_atomic_load_umin_i64_noret_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i64_noret_monotonic: ; CHECK-LABEL: test_atomic_load_umin_i64_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldumin x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB360_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, ls ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, ls
; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB360_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i64_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldumin x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umin ptr @var64, i64 %offset monotonic atomicrmw umin ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldumin x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_umin_i8_release(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_umin_i8_release(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i8_release: ; CHECK-LABEL: test_atomic_load_umin_i8_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: lduminlb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i8_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i8_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xff ; OUTLINE-ATOMICS-NEXT: adrp x8, var8
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB361_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrb w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrb w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, ls
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB361_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i8_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: lduminlb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var8, i8 %offset release %old = atomicrmw umin ptr @var8, i8 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: lduminlb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_umin_i16_release(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_umin_i16_release(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i16_release: ; CHECK-LABEL: test_atomic_load_umin_i16_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: lduminlh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i16_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i16_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xffff ; OUTLINE-ATOMICS-NEXT: adrp x8, var16
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xffff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB362_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxrh w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldxrh w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, ls
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB362_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i16_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: lduminlh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var16, i16 %offset release %old = atomicrmw umin ptr @var16, i16 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: lduminlh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_umin_i32_release(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_umin_i32_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i32_release: ; CHECK-LABEL: test_atomic_load_umin_i32_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: lduminl w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB363_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB363_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i32_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: lduminl w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var32, i32 %offset release %old = atomicrmw umin ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: lduminl w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_umin_i64_release(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_umin_i64_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i64_release: ; CHECK-LABEL: test_atomic_load_umin_i64_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: lduminl x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB364_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, ls ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB364_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i64_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: lduminl x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var64, i64 %offset release %old = atomicrmw umin ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: lduminl x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_umin_i32_noret_release(i32 %offset) nounwind { define dso_local void @test_atomic_load_umin_i32_noret_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i32_noret_release: ; CHECK-LABEL: test_atomic_load_umin_i32_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: lduminl w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB365_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, ls ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB365_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i32_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: lduminl w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umin ptr @var32, i32 %offset release atomicrmw umin ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: lduminl w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_umin_i64_noret_release(i64 %offset) nounwind { define dso_local void @test_atomic_load_umin_i64_noret_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i64_noret_release: ; CHECK-LABEL: test_atomic_load_umin_i64_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: lduminl x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB366_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, ls ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB366_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i64_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: lduminl x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umin ptr @var64, i64 %offset release atomicrmw umin ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: lduminl x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_umin_i8_seq_cst(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_umin_i8_seq_cst(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i8_seq_cst: ; CHECK-LABEL: test_atomic_load_umin_i8_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: lduminalb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i8_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i8_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xff ; OUTLINE-ATOMICS-NEXT: adrp x8, var8
; OUTLINE-ATOMICS-NEXT: adrp x9, var8 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var8
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var8 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB367_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrb w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, ls
; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrb w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB367_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i8_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: lduminalb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var8, i8 %offset seq_cst %old = atomicrmw umin ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: lduminalb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_umin_i16_seq_cst(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_umin_i16_seq_cst(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i16_seq_cst: ; CHECK-LABEL: test_atomic_load_umin_i16_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: lduminalh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i16_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i16_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: and w8, w0, #0xffff ; OUTLINE-ATOMICS-NEXT: adrp x8, var16
; OUTLINE-ATOMICS-NEXT: adrp x9, var16 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var16
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var16 ; OUTLINE-ATOMICS-NEXT: and w9, w0, #0xffff
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB368_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxrh w0, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w0, w8 ; OUTLINE-ATOMICS-NEXT: cmp w0, w9
; OUTLINE-ATOMICS-NEXT: csel w10, w0, w8, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w0, w9, ls
; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxrh w11, w10, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB368_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0 ; OUTLINE-ATOMICS-NEXT: // kill: def $w0 killed $w0 killed $x0
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i16_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: lduminalh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var16, i16 %offset seq_cst %old = atomicrmw umin ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: lduminalh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_umin_i32_seq_cst(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_umin_i32_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i32_seq_cst: ; CHECK-LABEL: test_atomic_load_umin_i32_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: lduminal w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var32 ; OUTLINE-ATOMICS-NEXT: adrp x9, var32
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB369_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr w8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp w8, w0 ; OUTLINE-ATOMICS-NEXT: cmp w8, w0
; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, ls ; OUTLINE-ATOMICS-NEXT: csel w10, w8, w0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, w10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB369_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov w0, w8 ; OUTLINE-ATOMICS-NEXT: mov w0, w8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i32_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: lduminal w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var32, i32 %offset seq_cst %old = atomicrmw umin ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: lduminal w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_umin_i64_seq_cst(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_umin_i64_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i64_seq_cst: ; CHECK-LABEL: test_atomic_load_umin_i64_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: lduminal x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x9, var64 ; OUTLINE-ATOMICS-NEXT: adrp x9, var64
; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x9, x9, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB370_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9] ; OUTLINE-ATOMICS-NEXT: ldaxr x8, [x9]
; OUTLINE-ATOMICS-NEXT: cmp x8, x0 ; OUTLINE-ATOMICS-NEXT: cmp x8, x0
; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, ls ; OUTLINE-ATOMICS-NEXT: csel x10, x8, x0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9] ; OUTLINE-ATOMICS-NEXT: stlxr w11, x10, [x9]
; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w11, .LBB370_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: mov x0, x8 ; OUTLINE-ATOMICS-NEXT: mov x0, x8
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i64_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: lduminal x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw umin ptr @var64, i64 %offset seq_cst %old = atomicrmw umin ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: lduminal x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_umin_i32_noret_seq_cst(i32 %offset) nounwind { define dso_local void @test_atomic_load_umin_i32_noret_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i32_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_umin_i32_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: lduminal w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i32_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var32 ; OUTLINE-ATOMICS-NEXT: adrp x8, var32
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var32
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB371_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr w9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp w9, w0 ; OUTLINE-ATOMICS-NEXT: cmp w9, w0
; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, ls ; OUTLINE-ATOMICS-NEXT: csel w9, w9, w0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, w9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB371_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i32_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: lduminal w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umin ptr @var32, i32 %offset seq_cst atomicrmw umin ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: lduminal w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_umin_i64_noret_seq_cst(i64 %offset) nounwind { define dso_local void @test_atomic_load_umin_i64_noret_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_umin_i64_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_umin_i64_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: lduminal x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_umin_i64_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: adrp x8, var64 ; OUTLINE-ATOMICS-NEXT: adrp x8, var64
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64 ; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var64
; OUTLINE-ATOMICS-NEXT: .LBB[[LOOPSTART:.*]]: // %atomicrmw.start ; OUTLINE-ATOMICS-NEXT: .LBB372_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1 ; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8] ; OUTLINE-ATOMICS-NEXT: ldaxr x9, [x8]
; OUTLINE-ATOMICS-NEXT: cmp x9, x0 ; OUTLINE-ATOMICS-NEXT: cmp x9, x0
; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, ls ; OUTLINE-ATOMICS-NEXT: csel x9, x9, x0, ls
; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8] ; OUTLINE-ATOMICS-NEXT: stlxr w10, x9, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB[[LOOPSTART]] ; OUTLINE-ATOMICS-NEXT: cbnz w10, .LBB372_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end ; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_umin_i64_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: lduminal x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw umin ptr @var64, i64 %offset seq_cst atomicrmw umin ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: lduminal x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_xor_i8_acq_rel(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_xor_i8_acq_rel(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i8_acq_rel: ; CHECK-LABEL: test_atomic_load_xor_i8_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldeoralb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i8_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i8_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i8_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldeoralb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var8, i8 %offset acq_rel %old = atomicrmw xor ptr @var8, i8 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldeoralb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_xor_i16_acq_rel(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_xor_i16_acq_rel(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i16_acq_rel: ; CHECK-LABEL: test_atomic_load_xor_i16_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldeoralh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i16_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i16_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i16_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldeoralh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var16, i16 %offset acq_rel %old = atomicrmw xor ptr @var16, i16 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldeoralh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_xor_i32_acq_rel(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_xor_i32_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i32_acq_rel: ; CHECK-LABEL: test_atomic_load_xor_i32_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldeoral w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i32_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldeoral w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var32, i32 %offset acq_rel %old = atomicrmw xor ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldeoral w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_xor_i64_acq_rel(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_xor_i64_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i64_acq_rel: ; CHECK-LABEL: test_atomic_load_xor_i64_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldeoral x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i64_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldeoral x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var64, i64 %offset acq_rel %old = atomicrmw xor ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldeoral x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_xor_i32_noret_acq_rel(i32 %offset) nounwind { define dso_local void @test_atomic_load_xor_i32_noret_acq_rel(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i32_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_xor_i32_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldeoral w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i32_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldeoral w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xor ptr @var32, i32 %offset acq_rel atomicrmw xor ptr @var32, i32 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldeoral w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_xor_i64_noret_acq_rel(i64 %offset) nounwind { define dso_local void @test_atomic_load_xor_i64_noret_acq_rel(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i64_noret_acq_rel: ; CHECK-LABEL: test_atomic_load_xor_i64_noret_acq_rel:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldeoral x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_noret_acq_rel: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_noret_acq_rel:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i64_noret_acq_rel:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldeoral x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xor ptr @var64, i64 %offset acq_rel atomicrmw xor ptr @var64, i64 %offset acq_rel
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldeoral x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_xor_i8_acquire(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_xor_i8_acquire(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i8_acquire: ; CHECK-LABEL: test_atomic_load_xor_i8_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldeorab w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i8_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i8_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor1_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor1_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i8_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldeorab w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var8, i8 %offset acquire %old = atomicrmw xor ptr @var8, i8 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldeorab w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_xor_i16_acquire(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_xor_i16_acquire(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i16_acquire: ; CHECK-LABEL: test_atomic_load_xor_i16_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldeorah w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i16_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i16_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor2_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor2_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i16_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldeorah w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var16, i16 %offset acquire %old = atomicrmw xor ptr @var16, i16 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldeorah w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_xor_i32_acquire(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_xor_i32_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i32_acquire: ; CHECK-LABEL: test_atomic_load_xor_i32_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldeora w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i32_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldeora w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var32, i32 %offset acquire %old = atomicrmw xor ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldeora w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_xor_i64_acquire(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_xor_i64_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i64_acquire: ; CHECK-LABEL: test_atomic_load_xor_i64_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldeora x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i64_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldeora x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var64, i64 %offset acquire %old = atomicrmw xor ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldeora x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_xor_i32_noret_acquire(i32 %offset) nounwind { define dso_local void @test_atomic_load_xor_i32_noret_acquire(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i32_noret_acquire: ; CHECK-LABEL: test_atomic_load_xor_i32_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldeora w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i32_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldeora w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xor ptr @var32, i32 %offset acquire atomicrmw xor ptr @var32, i32 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldeora w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_xor_i64_noret_acquire(i64 %offset) nounwind { define dso_local void @test_atomic_load_xor_i64_noret_acquire(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i64_noret_acquire: ; CHECK-LABEL: test_atomic_load_xor_i64_noret_acquire:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldeora x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_noret_acquire: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_noret_acquire:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i64_noret_acquire:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldeora x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xor ptr @var64, i64 %offset acquire atomicrmw xor ptr @var64, i64 %offset acquire
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldeora x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_xor_i8_monotonic(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_xor_i8_monotonic(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i8_monotonic: ; CHECK-LABEL: test_atomic_load_xor_i8_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldeorb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i8_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i8_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor1_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor1_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i8_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldeorb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var8, i8 %offset monotonic %old = atomicrmw xor ptr @var8, i8 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldeorb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_xor_i16_monotonic(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_xor_i16_monotonic(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i16_monotonic: ; CHECK-LABEL: test_atomic_load_xor_i16_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldeorh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i16_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i16_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor2_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor2_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i16_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldeorh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var16, i16 %offset monotonic %old = atomicrmw xor ptr @var16, i16 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldeorh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_xor_i32_monotonic(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_xor_i32_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i32_monotonic: ; CHECK-LABEL: test_atomic_load_xor_i32_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldeor w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i32_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldeor w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var32, i32 %offset monotonic %old = atomicrmw xor ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldeor w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_xor_i64_monotonic(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_xor_i64_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i64_monotonic: ; CHECK-LABEL: test_atomic_load_xor_i64_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldeor x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i64_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldeor x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var64, i64 %offset monotonic %old = atomicrmw xor ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldeor x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_xor_i32_noret_monotonic(i32 %offset) nounwind { define dso_local void @test_atomic_load_xor_i32_noret_monotonic(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i32_noret_monotonic: ; CHECK-LABEL: test_atomic_load_xor_i32_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldeor w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i32_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldeor w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xor ptr @var32, i32 %offset monotonic atomicrmw xor ptr @var32, i32 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldeor w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_xor_i64_noret_monotonic(i64 %offset) nounwind { define dso_local void @test_atomic_load_xor_i64_noret_monotonic(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i64_noret_monotonic: ; CHECK-LABEL: test_atomic_load_xor_i64_noret_monotonic:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldeor x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_noret_monotonic: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_noret_monotonic:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_relax ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_relax
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i64_noret_monotonic:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldeor x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xor ptr @var64, i64 %offset monotonic atomicrmw xor ptr @var64, i64 %offset monotonic
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldeor x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_xor_i8_release(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_xor_i8_release(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i8_release: ; CHECK-LABEL: test_atomic_load_xor_i8_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldeorlb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i8_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i8_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor1_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor1_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i8_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldeorlb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var8, i8 %offset release %old = atomicrmw xor ptr @var8, i8 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldeorlb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_xor_i16_release(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_xor_i16_release(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i16_release: ; CHECK-LABEL: test_atomic_load_xor_i16_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldeorlh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i16_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i16_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor2_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor2_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i16_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldeorlh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var16, i16 %offset release %old = atomicrmw xor ptr @var16, i16 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldeorlh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_xor_i32_release(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_xor_i32_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i32_release: ; CHECK-LABEL: test_atomic_load_xor_i32_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldeorl w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i32_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldeorl w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var32, i32 %offset release %old = atomicrmw xor ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldeorl w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_xor_i64_release(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_xor_i64_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i64_release: ; CHECK-LABEL: test_atomic_load_xor_i64_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldeorl x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i64_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldeorl x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var64, i64 %offset release %old = atomicrmw xor ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldeorl x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_xor_i32_noret_release(i32 %offset) nounwind { define dso_local void @test_atomic_load_xor_i32_noret_release(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i32_noret_release: ; CHECK-LABEL: test_atomic_load_xor_i32_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldeorl w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i32_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldeorl w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xor ptr @var32, i32 %offset release atomicrmw xor ptr @var32, i32 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldeorl w{{[0-9]+}}, w{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_xor_i64_noret_release(i64 %offset) nounwind { define dso_local void @test_atomic_load_xor_i64_noret_release(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i64_noret_release: ; CHECK-LABEL: test_atomic_load_xor_i64_noret_release:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldeorl x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_noret_release: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_noret_release:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i64_noret_release:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldeorl x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xor ptr @var64, i64 %offset release atomicrmw xor ptr @var64, i64 %offset release
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldeorl x{{[0-9]+}}, x{{[1-9][0-9]*}}, [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i8 @test_atomic_load_xor_i8_seq_cst(i8 %offset) nounwind { define dso_local i8 @test_atomic_load_xor_i8_seq_cst(i8 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i8_seq_cst: ; CHECK-LABEL: test_atomic_load_xor_i8_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var8
; CHECK-NEXT: add x8, x8, :lo12:var8
; CHECK-NEXT: ldeoralb w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i8_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i8_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor1_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor1_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i8_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var8
; CHECK-REG-NEXT: add x8, x8, :lo12:var8
; CHECK-REG-NEXT: ldeoralb w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var8, i8 %offset seq_cst %old = atomicrmw xor ptr @var8, i8 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var8
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var8
; CHECK: ldeoralb w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i8 %old ret i8 %old
} }
define dso_local i16 @test_atomic_load_xor_i16_seq_cst(i16 %offset) nounwind { define dso_local i16 @test_atomic_load_xor_i16_seq_cst(i16 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i16_seq_cst: ; CHECK-LABEL: test_atomic_load_xor_i16_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var16
; CHECK-NEXT: add x8, x8, :lo12:var16
; CHECK-NEXT: ldeoralh w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i16_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i16_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor2_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor2_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i16_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var16
; CHECK-REG-NEXT: add x8, x8, :lo12:var16
; CHECK-REG-NEXT: ldeoralh w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var16, i16 %offset seq_cst %old = atomicrmw xor ptr @var16, i16 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var16
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var16
; CHECK: ldeoralh w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i16 %old ret i16 %old
} }
define dso_local i32 @test_atomic_load_xor_i32_seq_cst(i32 %offset) nounwind { define dso_local i32 @test_atomic_load_xor_i32_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i32_seq_cst: ; CHECK-LABEL: test_atomic_load_xor_i32_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldeoral w0, w0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i32_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldeoral w0, w0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var32, i32 %offset seq_cst %old = atomicrmw xor ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldeoral w[[OLD:[0-9]+]], w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i32 %old ret i32 %old
} }
define dso_local i64 @test_atomic_load_xor_i64_seq_cst(i64 %offset) nounwind { define dso_local i64 @test_atomic_load_xor_i64_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i64_seq_cst: ; CHECK-LABEL: test_atomic_load_xor_i64_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldeoral x0, x0, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i64_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldeoral x0, x0, [x8]
; CHECK-REG-NEXT: ret
%old = atomicrmw xor ptr @var64, i64 %offset seq_cst %old = atomicrmw xor ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldeoral x[[OLD:[0-9]+]], x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret i64 %old ret i64 %old
} }
define dso_local void @test_atomic_load_xor_i32_noret_seq_cst(i32 %offset) nounwind { define dso_local void @test_atomic_load_xor_i32_noret_seq_cst(i32 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i32_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_xor_i32_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var32
; CHECK-NEXT: add x8, x8, :lo12:var32
; CHECK-NEXT: ldeoral w0, w8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i32_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor4_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i32_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var32
; CHECK-REG-NEXT: add x8, x8, :lo12:var32
; CHECK-REG-NEXT: ldeoral w0, w8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xor ptr @var32, i32 %offset seq_cst atomicrmw xor ptr @var32, i32 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var32
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var32
; CHECK: ldeoral w0, w[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local void @test_atomic_load_xor_i64_noret_seq_cst(i64 %offset) nounwind { define dso_local void @test_atomic_load_xor_i64_noret_seq_cst(i64 %offset) nounwind {
; CHECK-LABEL: test_atomic_load_xor_i64_noret_seq_cst: ; CHECK-LABEL: test_atomic_load_xor_i64_noret_seq_cst:
; CHECK: // %bb.0:
; CHECK-NEXT: adrp x8, var64
; CHECK-NEXT: add x8, x8, :lo12:var64
; CHECK-NEXT: ldeoral x0, x8, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_noret_seq_cst: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_xor_i64_noret_seq_cst:
; OUTLINE-ATOMICS: // %bb.0: ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill ; OUTLINE-ATOMICS-NEXT: str x30, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq_rel ; OUTLINE-ATOMICS-NEXT: bl __aarch64_ldeor8_acq_rel
; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload ; OUTLINE-ATOMICS-NEXT: ldr x30, [sp], #16 // 8-byte Folded Reload
; OUTLINE-ATOMICS-NEXT: ret ; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_xor_i64_noret_seq_cst:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: adrp x8, var64
; CHECK-REG-NEXT: add x8, x8, :lo12:var64
; CHECK-REG-NEXT: ldeoral x0, x8, [x8]
; CHECK-REG-NEXT: ret
atomicrmw xor ptr @var64, i64 %offset seq_cst atomicrmw xor ptr @var64, i64 %offset seq_cst
; CHECK-NOT: dmb
; CHECK: adrp [[TMPADDR:x[0-9]+]], var64
; CHECK: add x[[ADDR:[0-9]+]], [[TMPADDR]], {{#?}}:lo12:var64
; CHECK: ldeoral x0, x[[NEW:[0-9]+]], [x[[ADDR]]]
; CHECK-NOT: dmb
ret void ret void
} }
define dso_local i128 @test_atomic_load_i128() nounwind { define dso_local i128 @test_atomic_load_i128() nounwind {
; CHECK-LABEL: test_atomic_load_i128: ; CHECK-LABEL: test_atomic_load_i128:
; CHECK: casp ; CHECK: // %bb.0:
; CHECK-NEXT: mov x0, xzr
; CHECK-NEXT: mov x1, xzr
; CHECK-NEXT: adrp x8, var128
; CHECK-NEXT: add x8, x8, :lo12:var128
; CHECK-NEXT: casp x0, x1, x0, x1, [x8]
; CHECK-NEXT: ret
;
; OUTLINE-ATOMICS-LABEL: test_atomic_load_i128: ; OUTLINE-ATOMICS-LABEL: test_atomic_load_i128:
; OUTLINE-ATOMICS: ldxp ; OUTLINE-ATOMICS: // %bb.0:
; OUTLINE-ATOMICS: stxp ; OUTLINE-ATOMICS-NEXT: adrp x8, var128
; OUTLINE-ATOMICS-NEXT: add x8, x8, :lo12:var128
; OUTLINE-ATOMICS-NEXT: .LBB403_1: // %atomicrmw.start
; OUTLINE-ATOMICS-NEXT: // =>This Inner Loop Header: Depth=1
; OUTLINE-ATOMICS-NEXT: ldxp x0, x1, [x8]
; OUTLINE-ATOMICS-NEXT: stxp w9, x0, x1, [x8]
; OUTLINE-ATOMICS-NEXT: cbnz w9, .LBB403_1
; OUTLINE-ATOMICS-NEXT: // %bb.2: // %atomicrmw.end
; OUTLINE-ATOMICS-NEXT: ret
;
; CHECK-REG-LABEL: test_atomic_load_i128:
; CHECK-REG: // %bb.0:
; CHECK-REG-NEXT: mov x0, xzr
; CHECK-REG-NEXT: mov x1, xzr
; CHECK-REG-NEXT: adrp x8, var128
; CHECK-REG-NEXT: add x8, x8, :lo12:var128
; CHECK-REG-NEXT: casp x0, x1, x0, x1, [x8]
; CHECK-REG-NEXT: ret
%pair = load atomic i128, ptr @var128 monotonic, align 16 %pair = load atomic i128, ptr @var128 monotonic, align 16
ret i128 %pair ret i128 %pair
} }
Context not available.

llvm/test/CodeGen/AArch64/atomic-ops-msvc.ll

llvm/test/CodeGen/AArch64/atomic-ops.ll

llvm/test/CodeGen/AArch64/bcmp-inline-small.ll

llvm/test/CodeGen/AArch64/bcmp.ll

llvm/test/CodeGen/AArch64/bf16-shuffle.ll

llvm/test/CodeGen/AArch64/bfis-in-loop.ll

Context not available.
@global = external global %struct.bar, align 8 @global = external global %struct.bar, align 8
define i64 @bfis_in_loop_zero() { define i64 @bfis_in_loop_zero() {
dmgreenUnsubmitted
Done
ReplyInline Actions

This doesn't look right

dmgreen: This doesn't look right
; CHECK-LABEL: bfis_in_loop_zero:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: adrp x9, :got:global
; CHECK-NEXT: mov x0, xzr
; CHECK-NEXT: mov w8, wzr
; CHECK-NEXT: ldr x9, [x9, :got_lo12:global]
; CHECK-NEXT: ldr x9, [x9]
; CHECK-NEXT: .LBB0_1: // %midblock
; CHECK-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldrh w10, [x9, #72]
; CHECK-NEXT: cmp w10, #0
; CHECK-NEXT: ubfx x11, x10, #8, #24
; CHECK-NEXT: cset w12, ne
; CHECK-NEXT: csel w8, w8, w11, eq
; CHECK-NEXT: ldr x11, [x9, #8]
; CHECK-NEXT: and x9, x10, #0xff
; CHECK-NEXT: and x10, x0, #0xffffffff00000000
; CHECK-NEXT: orr x9, x9, x8, lsl #8
; CHECK-NEXT: orr x10, x10, x12, lsl #16
; CHECK-NEXT: orr x0, x10, x9
; CHECK-NEXT: ldr x9, [x11, #16]
; CHECK-NEXT: cbnz x11, .LBB0_1
; CHECK-NEXT: // %bb.2: // %exit
; CHECK-NEXT: ret
entry: entry:
%var = load ptr, ptr @global, align 8 %var = load ptr, ptr @global, align 8
br label %preheader br label %preheader
Context not available.
} }
define i64 @bfis_in_loop_undef() { define i64 @bfis_in_loop_undef() {
; CHECK-LABEL: bfis_in_loop_undef:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: adrp x9, :got:global
; CHECK-NEXT: mov w8, wzr
; CHECK-NEXT: // implicit-def: $x0
; CHECK-NEXT: ldr x9, [x9, :got_lo12:global]
; CHECK-NEXT: ldr x9, [x9]
; CHECK-NEXT: .LBB1_1: // %midblock
; CHECK-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldrh w10, [x9, #72]
; CHECK-NEXT: cmp w10, #0
; CHECK-NEXT: ubfx x11, x10, #8, #24
; CHECK-NEXT: cset w12, ne
; CHECK-NEXT: csel w8, w8, w11, eq
; CHECK-NEXT: ldr x11, [x9, #8]
; CHECK-NEXT: and x9, x10, #0xff
; CHECK-NEXT: and x10, x0, #0xffffffff00000000
; CHECK-NEXT: orr x9, x9, x8, lsl #8
; CHECK-NEXT: orr x10, x10, x12, lsl #16
; CHECK-NEXT: orr x0, x10, x9
; CHECK-NEXT: ldr x9, [x11, #16]
; CHECK-NEXT: cbnz x11, .LBB1_1
; CHECK-NEXT: // %bb.2: // %exit
; CHECK-NEXT: ret
entry: entry:
%var = load ptr, ptr @global, align 8 %var = load ptr, ptr @global, align 8
br label %preheader br label %preheader
Context not available.
exit: exit:
ret i64 %var30 ret i64 %var30
} }
;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
; CHECK: {{.*}}
Context not available.

llvm/test/CodeGen/AArch64/bitfield-insert.ll

llvm/test/CodeGen/AArch64/bool-ext-inc.ll

llvm/test/CodeGen/AArch64/branch-relax-alignment.ll

llvm/test/CodeGen/AArch64/branch-relax-bcc.ll

llvm/test/CodeGen/AArch64/build-one-lane.ll

llvm/test/CodeGen/AArch64/build-vector-to-extract-subvec-crash.ll

llvm/test/CodeGen/AArch64/build-vector-two-dup.ll

llvm/test/CodeGen/AArch64/cgp-usubo.ll

llvm/test/CodeGen/AArch64/cmp-chains.ll

llvm/test/CodeGen/AArch64/cmp-select-sign.ll

llvm/test/CodeGen/AArch64/cmpxchg-idioms.ll

llvm/test/CodeGen/AArch64/combine-andintoload.ll

llvm/test/CodeGen/AArch64/combine-comparisons-by-cse.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-add-mull-fixed-contract.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-add-mull-fixed-fast.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-add-mull-scalable-contract.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-add-mull-scalable-fast.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-f16-add-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-f16-add.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-f16-mul-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-f16-mul.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-f32-add-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-f32-add.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-f32-mul-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-f32-mul.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-f64-add-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-f64-add.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-f64-mul-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-f64-mul.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-i16-add-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-i16-mul-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-i32-add-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-i32-mul-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-i64-add-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-i64-mul-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-i8-add-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-mixed-cases.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-multiuses.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-reductions-predicated-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-reductions-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-reductions.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-splat-scalable.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-splat.ll

llvm/test/CodeGen/AArch64/complex-deinterleaving-uniform-cases.ll

llvm/test/CodeGen/AArch64/concat_vector-truncate-combine.ll

llvm/test/CodeGen/AArch64/cond-br-tuning.ll

llvm/test/CodeGen/AArch64/consthoist-gep.ll

llvm/test/CodeGen/AArch64/copyprop.ll

llvm/test/CodeGen/AArch64/ctpop-nonean.ll

llvm/test/CodeGen/AArch64/dag-combine-concat-vectors.ll

llvm/test/CodeGen/AArch64/dag-combine-select.ll

llvm/test/CodeGen/AArch64/dag-combine-setcc.ll

llvm/test/CodeGen/AArch64/dag-numsignbits.ll

llvm/test/CodeGen/AArch64/div-rem-pair-recomposition-signed.ll

llvm/test/CodeGen/AArch64/div-rem-pair-recomposition-unsigned.ll

llvm/test/CodeGen/AArch64/double_reduct.ll

llvm/test/CodeGen/AArch64/expand-select.ll

llvm/test/CodeGen/AArch64/expand-vector-rot.ll

llvm/test/CodeGen/AArch64/extbinopload.ll

llvm/test/CodeGen/AArch64/extend_inreg_of_concat_subvectors.ll

llvm/test/CodeGen/AArch64/extract-bits.ll

llvm/test/CodeGen/AArch64/extract-lowbits.ll

llvm/test/CodeGen/AArch64/f16-instructions.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc < %s -mtriple aarch64-unknown-unknown -aarch64-neon-syntax=apple -asm-verbose=false -disable-post-ra -frame-pointer=non-leaf | FileCheck %s --check-prefix=CHECK-CVT --check-prefix=CHECK-COMMON ; RUN: llc < %s -mtriple aarch64-unknown-unknown -aarch64-neon-syntax=apple -asm-verbose=false -disable-post-ra -frame-pointer=non-leaf | FileCheck %s --check-prefix=CHECK-CVT --check-prefix=CHECK-COMMON
; RUN: llc < %s -mtriple aarch64-unknown-unknown -mattr=+fullfp16 -aarch64-neon-syntax=apple -asm-verbose=false -disable-post-ra -frame-pointer=non-leaf | FileCheck %s --check-prefix=CHECK-COMMON --check-prefix=CHECK-FP16 ; RUN: llc < %s -mtriple aarch64-unknown-unknown -mattr=+fullfp16 -aarch64-neon-syntax=apple -asm-verbose=false -disable-post-ra -frame-pointer=non-leaf | FileCheck %s --check-prefix=CHECK-COMMON --check-prefix=CHECK-FP16
; RUN: llc < %s -mtriple aarch64-unknown-unknown -aarch64-neon-syntax=apple \ ; RUN: llc < %s -mtriple aarch64-unknown-unknown -aarch64-neon-syntax=apple \
; RUN: -asm-verbose=false -disable-post-ra -frame-pointer=non-leaf -global-isel \ ; RUN: -asm-verbose=false -disable-post-ra -frame-pointer=non-leaf -global-isel \
; RUN: -global-isel-abort=2 -pass-remarks-missed=gisel-* 2>&1 | FileCheck %s \ ; RUN: -global-isel-abort=2 -pass-remarks-missed=gisel-* 2>&1 | FileCheck %s \
; RUN: --check-prefixes=FALLBACK,GISEL-CVT,GISEL ; RUN: --check-prefixes=FALLBACK,GISEL-CVT,GISEL
; RUN: llc < %s -mtriple aarch64-unknown-unknown -mattr=+fullfp16 \ ; RUN: llc < %s -mtriple aarch64-unknown-unknown -mattr=+fullfp16 \
; RUN: -aarch64-neon-syntax=apple -asm-verbose=false -disable-post-ra \ ; RUN: -aarch64-neon-syntax=apple -asm-verbose=false -disable-post-ra \
; RUN: -frame-pointer=non-leaf -global-isel -global-isel-abort=2 \ ; RUN: -frame-pointer=non-leaf -global-isel -global-isel-abort=2 \
; RUN: -pass-remarks-missed=gisel-* 2>&1 | FileCheck %s \ ; RUN: -pass-remarks-missed=gisel-* 2>&1 | FileCheck %s \
; RUN: --check-prefixes=FALLBACK-FP16,GISEL-FP16,GISEL ; RUN: --check-prefixes=FALLBACK-FP16,GISEL-FP16,GISEL
target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128" target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
; CHECK-CVT-LABEL: test_fadd:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fadd s0, s0, s1
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fadd:
; CHECK-FP16-NEXT: fadd h0, h0, h1
; CHECK-FP16-NEXT: ret
define half @test_fadd(half %a, half %b) #0 { define half @test_fadd(half %a, half %b) #0 {
dmgreenUnsubmitted
Done
ReplyInline Actions

This one needs to be done manually.

dmgreen: This one needs to be done manually.
%r = fadd half %a, %b %r = fadd half %a, %b
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_fsub:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fsub s0, s0, s1
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fsub:
; CHECK-FP16-NEXT: fsub h0, h0, h1
; CHECK-FP16-NEXT: ret
define half @test_fsub(half %a, half %b) #0 { define half @test_fsub(half %a, half %b) #0 {
%r = fsub half %a, %b %r = fsub half %a, %b
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_fmul:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fmul s0, s0, s1
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fmul:
; CHECK-FP16-NEXT: fmul h0, h0, h1
; CHECK-FP16-NEXT: ret
define half @test_fmul(half %a, half %b) #0 { define half @test_fmul(half %a, half %b) #0 {
%r = fmul half %a, %b %r = fmul half %a, %b
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_fmadd:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fmul s0, s0, s1
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcvt s1, h2
; CHECK-CVT-NEXT: fadd s0, s0, s1
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fmadd:
; CHECK-FP16-NEXT: fmadd h0, h0, h1, h2
; CHECK-FP16-NEXT: ret
define half @test_fmadd(half %a, half %b, half %c) #0 { define half @test_fmadd(half %a, half %b, half %c) #0 {
%mul = fmul fast half %a, %b %mul = fmul fast half %a, %b
%r = fadd fast half %mul, %c %r = fadd fast half %mul, %c
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_fdiv:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fdiv s0, s0, s1
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fdiv:
; CHECK-FP16-NEXT: fdiv h0, h0, h1
; CHECK-FP16-NEXT: ret
define half @test_fdiv(half %a, half %b) #0 { define half @test_fdiv(half %a, half %b) #0 {
%r = fdiv half %a, %b %r = fdiv half %a, %b
ret half %r ret half %r
} }
; CHECK-COMMON-LABEL: test_frem:
; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
; CHECK-COMMON-NEXT: mov x29, sp
; CHECK-COMMON-NEXT: fcvt s0, h0
; CHECK-COMMON-NEXT: fcvt s1, h1
; CHECK-COMMON-NEXT: bl {{_?}}fmodf
; CHECK-COMMON-NEXT: fcvt h0, s0
; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
; CHECK-COMMON-NEXT: ret
define half @test_frem(half %a, half %b) #0 { define half @test_frem(half %a, half %b) #0 {
%r = frem half %a, %b %r = frem half %a, %b
ret half %r ret half %r
} }
; CHECK-COMMON-LABEL: test_store:
; CHECK-COMMON-NEXT: str h0, [x0]
; CHECK-COMMON-NEXT: ret
define void @test_store(half %a, ptr %b) #0 { define void @test_store(half %a, ptr %b) #0 {
store half %a, ptr %b store half %a, ptr %b
ret void ret void
} }
; CHECK-COMMON-LABEL: test_load:
; CHECK-COMMON-NEXT: ldr h0, [x0]
; CHECK-COMMON-NEXT: ret
define half @test_load(ptr %a) #0 { define half @test_load(ptr %a) #0 {
%r = load half, ptr %a %r = load half, ptr %a
ret half %r ret half %r
} }
declare half @test_callee(half %a, half %b) #0 declare half @test_callee(half %a, half %b) #0
; CHECK-COMMON-LABEL: test_call:
; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
; CHECK-COMMON-NEXT: mov x29, sp
; CHECK-COMMON-NEXT: bl {{_?}}test_callee
; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
; CHECK-COMMON-NEXT: ret
define half @test_call(half %a, half %b) #0 { define half @test_call(half %a, half %b) #0 {
%r = call half @test_callee(half %a, half %b) %r = call half @test_callee(half %a, half %b)
ret half %r ret half %r
} }
; CHECK-COMMON-LABEL: test_call_flipped:
; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
; CHECK-COMMON-NEXT: mov x29, sp
; CHECK-COMMON-NEXT: fmov s2, s0
; CHECK-COMMON-NEXT: fmov s0, s1
; CHECK-COMMON-NEXT: fmov s1, s2
; CHECK-COMMON-NEXT: bl {{_?}}test_callee
; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
; CHECK-COMMON-NEXT: ret
define half @test_call_flipped(half %a, half %b) #0 { define half @test_call_flipped(half %a, half %b) #0 {
%r = call half @test_callee(half %b, half %a) %r = call half @test_callee(half %b, half %a)
ret half %r ret half %r
} }
; CHECK-COMMON-LABEL: test_tailcall_flipped:
; CHECK-COMMON-NEXT: fmov s2, s0
; CHECK-COMMON-NEXT: fmov s0, s1
; CHECK-COMMON-NEXT: fmov s1, s2
; CHECK-COMMON-NEXT: b {{_?}}test_callee
define half @test_tailcall_flipped(half %a, half %b) #0 { define half @test_tailcall_flipped(half %a, half %b) #0 {
%r = tail call half @test_callee(half %b, half %a) %r = tail call half @test_callee(half %b, half %a)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_select:
; CHECK-CVT-NEXT: cmp w0, #0
; CHECK-CVT-NEXT: fcsel s0, s0, s1, ne
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_select:
; CHECK-FP16-NEXT: cmp w0, #0
; CHECK-FP16-NEXT: fcsel h0, h0, h1, ne
; CHECK-FP16-NEXT: ret
define half @test_select(half %a, half %b, i1 zeroext %c) #0 { define half @test_select(half %a, half %b, i1 zeroext %c) #0 {
%r = select i1 %c, half %a, half %b %r = select i1 %c, half %a, half %b
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_select_cc:
; CHECK-CVT-DAG: fcvt s3, h3
; CHECK-CVT-DAG: fcvt s2, h2
; CHECK-CVT-DAG: fcmp s2, s3
; CHECK-CVT-NEXT: fcsel s0, s0, s1, ne
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_select_cc:
; CHECK-FP16-NEXT: fcmp h2, h3
; CHECK-FP16-NEXT: fcsel h0, h0, h1, ne
; CHECK-FP16-NEXT: ret
define half @test_select_cc(half %a, half %b, half %c, half %d) #0 { define half @test_select_cc(half %a, half %b, half %c, half %d) #0 {
%cc = fcmp une half %c, %d %cc = fcmp une half %c, %d
%r = select i1 %cc, half %a, half %b %r = select i1 %cc, half %a, half %b
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_select_cc_f32_f16:
; CHECK-CVT-DAG: fcvt s2, h2
; CHECK-CVT-DAG: fcvt s3, h3
; CHECK-CVT-NEXT: fcmp s2, s3
; CHECK-CVT-NEXT: fcsel s0, s0, s1, ne
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_select_cc_f32_f16:
; CHECK-FP16-NEXT: fcmp h2, h3
; CHECK-FP16-NEXT: fcsel s0, s0, s1, ne
; CHECK-FP16-NEXT: ret
define float @test_select_cc_f32_f16(float %a, float %b, half %c, half %d) #0 { define float @test_select_cc_f32_f16(float %a, float %b, half %c, half %d) #0 {
%cc = fcmp une half %c, %d %cc = fcmp une half %c, %d
%r = select i1 %cc, float %a, float %b %r = select i1 %cc, float %a, float %b
ret float %r ret float %r
} }
; CHECK-CVT-LABEL: test_select_cc_f16_f32:
; CHECK-CVT-DAG: fcmp s2, s3
; CHECK-CVT-NEXT: fcsel s0, s0, s1, ne
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_select_cc_f16_f32:
; CHECK-FP16-NEXT: fcmp s2, s3
; CHECK-FP16-NEXT: fcsel h0, h0, h1, ne
; CHECK-FP16-NEXT: ret
define half @test_select_cc_f16_f32(half %a, half %b, float %c, float %d) #0 { define half @test_select_cc_f16_f32(half %a, half %b, float %c, float %d) #0 {
%cc = fcmp une float %c, %d %cc = fcmp une float %c, %d
%r = select i1 %cc, half %a, half %b %r = select i1 %cc, half %a, half %b
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_fcmp_une:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset w0, ne
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_une:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset w0, ne
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_une(half %a, half %b) #0 { define i1 @test_fcmp_une(half %a, half %b) #0 {
%r = fcmp une half %a, %b %r = fcmp une half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_ueq:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset [[TRUE:w[0-9]+]], eq
; CHECK-CVT-NEXT: csinc w0, [[TRUE]], wzr, vc
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_ueq:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset [[TRUE:w[0-9]+]], eq
; CHECK-FP16-NEXT: csinc w0, [[TRUE]], wzr, vc
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_ueq(half %a, half %b) #0 { define i1 @test_fcmp_ueq(half %a, half %b) #0 {
%r = fcmp ueq half %a, %b %r = fcmp ueq half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_ugt:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset w0, hi
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_ugt:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset w0, hi
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_ugt(half %a, half %b) #0 { define i1 @test_fcmp_ugt(half %a, half %b) #0 {
%r = fcmp ugt half %a, %b %r = fcmp ugt half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_uge:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset w0, pl
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_uge:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset w0, pl
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_uge(half %a, half %b) #0 { define i1 @test_fcmp_uge(half %a, half %b) #0 {
%r = fcmp uge half %a, %b %r = fcmp uge half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_ult:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset w0, lt
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_ult:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset w0, lt
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_ult(half %a, half %b) #0 { define i1 @test_fcmp_ult(half %a, half %b) #0 {
%r = fcmp ult half %a, %b %r = fcmp ult half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_ule:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset w0, le
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_ule:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset w0, le
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_ule(half %a, half %b) #0 { define i1 @test_fcmp_ule(half %a, half %b) #0 {
%r = fcmp ule half %a, %b %r = fcmp ule half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_uno:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset w0, vs
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_uno:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset w0, vs
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_uno(half %a, half %b) #0 { define i1 @test_fcmp_uno(half %a, half %b) #0 {
%r = fcmp uno half %a, %b %r = fcmp uno half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_one:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset [[TRUE:w[0-9]+]], mi
; CHECK-CVT-NEXT: csinc w0, [[TRUE]], wzr, le
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_one:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset [[TRUE:w[0-9]+]], mi
; CHECK-FP16-NEXT: csinc w0, [[TRUE]], wzr, le
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_one(half %a, half %b) #0 { define i1 @test_fcmp_one(half %a, half %b) #0 {
%r = fcmp one half %a, %b %r = fcmp one half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_oeq:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset w0, eq
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_oeq:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset w0, eq
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_oeq(half %a, half %b) #0 { define i1 @test_fcmp_oeq(half %a, half %b) #0 {
%r = fcmp oeq half %a, %b %r = fcmp oeq half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_ogt:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset w0, gt
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_ogt:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset w0, gt
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_ogt(half %a, half %b) #0 { define i1 @test_fcmp_ogt(half %a, half %b) #0 {
%r = fcmp ogt half %a, %b %r = fcmp ogt half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_oge:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset w0, ge
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_oge:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset w0, ge
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_oge(half %a, half %b) #0 { define i1 @test_fcmp_oge(half %a, half %b) #0 {
%r = fcmp oge half %a, %b %r = fcmp oge half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_olt:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset w0, mi
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_olt:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset w0, mi
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_olt(half %a, half %b) #0 { define i1 @test_fcmp_olt(half %a, half %b) #0 {
%r = fcmp olt half %a, %b %r = fcmp olt half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_ole:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset w0, ls
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_ole:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset w0, ls
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_ole(half %a, half %b) #0 { define i1 @test_fcmp_ole(half %a, half %b) #0 {
%r = fcmp ole half %a, %b %r = fcmp ole half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-CVT-LABEL: test_fcmp_ord:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: cset w0, vc
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fcmp_ord:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: cset w0, vc
; CHECK-FP16-NEXT: ret
define i1 @test_fcmp_ord(half %a, half %b) #0 { define i1 @test_fcmp_ord(half %a, half %b) #0 {
%r = fcmp ord half %a, %b %r = fcmp ord half %a, %b
ret i1 %r ret i1 %r
} }
; CHECK-COMMON-LABEL: test_fccmp:
; CHECK-CVT: fcvt s1, h0
; CHECK-CVT-NEXT: fmov s2, #5.00000000
; CHECK-CVT-NEXT: fcmp s1, s2
; CHECK-CVT-NEXT: fmov s2, #8.00000000
; CHECK-CVT-NEXT: adrp x8
; CHECK-CVT-NEXT: fccmp s1, s2, #4, mi
; CHECK-CVT-NEXT: ldr h1, [x8,
; CHECK-CVT-NEXT: fcsel s0, s0, s1, gt
; CHECK-CVT-NEXT: str h0, [x0]
; CHECK-CVT-NEXT: ret
; CHECK-FP16: fmov h1, #5.00000000
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: fmov h2, #8.00000000
; CHECK-FP16-NEXT: fccmp h0, h2, #4, mi
; CHECK-FP16-NEXT: fcsel h0, h0, h1, gt
; CHECK-FP16-NEXT: str h0, [x0]
; CHECK-FP16-NEXT: ret
define void @test_fccmp(half %in, ptr %out) { define void @test_fccmp(half %in, ptr %out) {
%cmp1 = fcmp ogt half %in, 0xH4800 %cmp1 = fcmp ogt half %in, 0xH4800
%cmp2 = fcmp olt half %in, 0xH4500 %cmp2 = fcmp olt half %in, 0xH4500
Context not available.
store half %result, ptr %out store half %result, ptr %out
ret void ret void
} }
; CHECK-CVT-LABEL: test_br_cc:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcmp s0, s1
; CHECK-CVT-NEXT: csel x8, x0, x1, pl
; CHECK-CVT-NEXT: str wzr, [x8]
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_br_cc:
; CHECK-FP16-NEXT: fcmp h0, h1
; CHECK-FP16-NEXT: csel x8, x0, x1, pl
; CHECK-FP16-NEXT: str wzr, [x8]
; CHECK-FP16-NEXT: ret
define void @test_br_cc(half %a, half %b, ptr %p1, ptr %p2) #0 { define void @test_br_cc(half %a, half %b, ptr %p1, ptr %p2) #0 {
%c = fcmp uge half %a, %b %c = fcmp uge half %a, %b
br i1 %c, label %then, label %else br i1 %c, label %then, label %else
Context not available.
store i32 0, ptr %p2 store i32 0, ptr %p2
ret void ret void
} }
; CHECK-COMMON-LABEL: test_phi:
; CHECK-COMMON: mov x[[PTR:[0-9]+]], x0
; CHECK-COMMON: ldr h[[AB:[0-9]+]], [x0]
; CHECK-COMMON: [[LOOP:LBB[0-9_]+]]:
; CHECK-COMMON: fmov s[[R:[0-9]+]], s[[AB]]
; CHECK-COMMON: ldr h[[AB]], [x[[PTR]]]
; CHECK-COMMON: mov x0, x[[PTR]]
; CHECK-COMMON: bl {{_?}}test_dummy
; CHECK-COMMON: fmov s0, s[[R]]
; CHECK-COMMON: ret
define half @test_phi(ptr %p1) #0 { define half @test_phi(ptr %p1) #0 {
entry: entry:
%a = load half, ptr %p1 %a = load half, ptr %p1
Context not available.
return: return:
ret half %r ret half %r
} }
declare i1 @test_dummy(ptr %p1) #0 declare i1 @test_dummy(ptr %p1) #0
; CHECK-CVT-LABEL: test_fptosi_i32:
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcvtzs w0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fptosi_i32:
; CHECK-FP16-NEXT: fcvtzs w0, h0
; CHECK-FP16-NEXT: ret
define i32 @test_fptosi_i32(half %a) #0 { define i32 @test_fptosi_i32(half %a) #0 {
%r = fptosi half %a to i32 %r = fptosi half %a to i32
ret i32 %r ret i32 %r
} }
; CHECK-CVT-LABEL: test_fptosi_i64:
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcvtzs x0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fptosi_i64:
; CHECK-FP16-NEXT: fcvtzs x0, h0
; CHECK-FP16-NEXT: ret
define i64 @test_fptosi_i64(half %a) #0 { define i64 @test_fptosi_i64(half %a) #0 {
%r = fptosi half %a to i64 %r = fptosi half %a to i64
ret i64 %r ret i64 %r
} }
; CHECK-CVT-LABEL: test_fptoui_i32:
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcvtzu w0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fptoui_i32:
; CHECK-FP16-NEXT: fcvtzu w0, h0
; CHECK-FP16-NEXT: ret
define i32 @test_fptoui_i32(half %a) #0 { define i32 @test_fptoui_i32(half %a) #0 {
%r = fptoui half %a to i32 %r = fptoui half %a to i32
ret i32 %r ret i32 %r
} }
; CHECK-CVT-LABEL: test_fptoui_i64:
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcvtzu x0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fptoui_i64:
; CHECK-FP16-NEXT: fcvtzu x0, h0
; CHECK-FP16-NEXT: ret
define i64 @test_fptoui_i64(half %a) #0 { define i64 @test_fptoui_i64(half %a) #0 {
%r = fptoui half %a to i64 %r = fptoui half %a to i64
ret i64 %r ret i64 %r
} }
; CHECK-CVT-LABEL: test_uitofp_i32:
; CHECK-CVT-NEXT: ucvtf s0, w0
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_uitofp_i32:
; CHECK-FP16-NEXT: ucvtf h0, w0
; CHECK-FP16-NEXT: ret
define half @test_uitofp_i32(i32 %a) #0 { define half @test_uitofp_i32(i32 %a) #0 {
%r = uitofp i32 %a to half %r = uitofp i32 %a to half
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_uitofp_i64:
; CHECK-CVT-NEXT: ucvtf s0, x0
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_uitofp_i64:
; CHECK-FP16-NEXT: ucvtf h0, x0
; CHECK-FP16-NEXT: ret
define half @test_uitofp_i64(i64 %a) #0 { define half @test_uitofp_i64(i64 %a) #0 {
%r = uitofp i64 %a to half %r = uitofp i64 %a to half
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_sitofp_i32:
; CHECK-CVT-NEXT: scvtf s0, w0
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_sitofp_i32:
; CHECK-FP16-NEXT: scvtf h0, w0
; CHECK-FP16-NEXT: ret
define half @test_sitofp_i32(i32 %a) #0 { define half @test_sitofp_i32(i32 %a) #0 {
%r = sitofp i32 %a to half %r = sitofp i32 %a to half
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_sitofp_i64:
; CHECK-CVT-NEXT: scvtf s0, x0
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_sitofp_i64:
; CHECK-FP16-NEXT: scvtf h0, x0
; CHECK-FP16-NEXT: ret
define half @test_sitofp_i64(i64 %a) #0 { define half @test_sitofp_i64(i64 %a) #0 {
%r = sitofp i64 %a to half %r = sitofp i64 %a to half
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_uitofp_i32_fadd:
; CHECK-CVT-NEXT: ucvtf s1, w0
; CHECK-CVT-NEXT: fcvt h1, s1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fadd s0, s0, s1
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_uitofp_i32_fadd:
; CHECK-FP16-NEXT: ucvtf h1, w0
; CHECK-FP16-NEXT: fadd h0, h0, h1
; CHECK-FP16-NEXT: ret
define half @test_uitofp_i32_fadd(i32 %a, half %b) #0 { define half @test_uitofp_i32_fadd(i32 %a, half %b) #0 {
%c = uitofp i32 %a to half %c = uitofp i32 %a to half
%r = fadd half %b, %c %r = fadd half %b, %c
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_sitofp_i32_fadd:
; CHECK-CVT-NEXT: scvtf s1, w0
; CHECK-CVT-NEXT: fcvt h1, s1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fadd s0, s0, s1
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_sitofp_i32_fadd:
; CHECK-FP16-NEXT: scvtf h1, w0
; CHECK-FP16-NEXT: fadd h0, h0, h1
; CHECK-FP16-NEXT: ret
define half @test_sitofp_i32_fadd(i32 %a, half %b) #0 { define half @test_sitofp_i32_fadd(i32 %a, half %b) #0 {
%c = sitofp i32 %a to half %c = sitofp i32 %a to half
%r = fadd half %b, %c %r = fadd half %b, %c
ret half %r ret half %r
} }
; CHECK-COMMON-LABEL: test_fptrunc_float:
; CHECK-COMMON-NEXT: fcvt h0, s0
; CHECK-COMMON-NEXT: ret
define half @test_fptrunc_float(float %a) #0 { define half @test_fptrunc_float(float %a) #0 {
%r = fptrunc float %a to half %r = fptrunc float %a to half
ret half %r ret half %r
} }
; CHECK-COMMON-LABEL: test_fptrunc_double:
; CHECK-COMMON-NEXT: fcvt h0, d0
; CHECK-COMMON-NEXT: ret
define half @test_fptrunc_double(double %a) #0 { define half @test_fptrunc_double(double %a) #0 {
%r = fptrunc double %a to half %r = fptrunc double %a to half
ret half %r ret half %r
} }
; CHECK-COMMON-LABEL: test_fpext_float:
; CHECK-COMMON-NEXT: fcvt s0, h0
; CHECK-COMMON-NEXT: ret
define float @test_fpext_float(half %a) #0 { define float @test_fpext_float(half %a) #0 {
%r = fpext half %a to float %r = fpext half %a to float
ret float %r ret float %r
} }
; CHECK-COMMON-LABEL: test_fpext_double:
; CHECK-COMMON-NEXT: fcvt d0, h0
; CHECK-COMMON-NEXT: ret
define double @test_fpext_double(half %a) #0 { define double @test_fpext_double(half %a) #0 {
%r = fpext half %a to double %r = fpext half %a to double
ret double %r ret double %r
} }
; CHECK-COMMON-LABEL: test_bitcast_halftoi16:
; CHECK-COMMON-NEXT: fmov w0, s0
; CHECK-COMMON-NEXT: ret
define i16 @test_bitcast_halftoi16(half %a) #0 { define i16 @test_bitcast_halftoi16(half %a) #0 {
%r = bitcast half %a to i16 %r = bitcast half %a to i16
ret i16 %r ret i16 %r
} }
; CHECK-COMMON-LABEL: test_bitcast_i16tohalf:
; CHECK-COMMON-NEXT: fmov s0, w0
; CHECK-COMMON-NEXT: ret
define half @test_bitcast_i16tohalf(i16 %a) #0 { define half @test_bitcast_i16tohalf(i16 %a) #0 {
%r = bitcast i16 %a to half %r = bitcast i16 %a to half
ret half %r ret half %r
} }
declare half @llvm.sqrt.f16(half %a) #0 declare half @llvm.sqrt.f16(half %a) #0
declare half @llvm.powi.f16.i32(half %a, i32 %b) #0 declare half @llvm.powi.f16.i32(half %a, i32 %b) #0
declare half @llvm.sin.f16(half %a) #0 declare half @llvm.sin.f16(half %a) #0
Context not available.
declare half @llvm.round.f16(half %a) #0 declare half @llvm.round.f16(half %a) #0
declare half @llvm.roundeven.f16(half %a) #0 declare half @llvm.roundeven.f16(half %a) #0
declare half @llvm.fmuladd.f16(half %a, half %b, half %c) #0 declare half @llvm.fmuladd.f16(half %a, half %b, half %c) #0
; FALLBACK-NOT: remark:{{.*}}test_sqrt ; FALLBACK-NOT: remark:{{.*}}test_sqrt
; FALLBACK-FP16-NOT: remark:{{.*}}test_sqrt ; FALLBACK-FP16-NOT: remark:{{.*}}test_sqrt
; CHECK-CVT-LABEL: test_sqrt:
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fsqrt s0, s0
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_sqrt:
; CHECK-FP16-NEXT: fsqrt h0, h0
; CHECK-FP16-NEXT: ret
; GISEL-CVT-LABEL: test_sqrt: ; GISEL-CVT-LABEL: test_sqrt:
; GISEL-CVT-NEXT: fcvt s0, h0 ; GISEL-CVT-NEXT: fcvt s0, h0
; GISEL-CVT-NEXT: fsqrt s0, s0 ; GISEL-CVT-NEXT: fsqrt s0, s0
; GISEL-CVT-NEXT: fcvt h0, s0 ; GISEL-CVT-NEXT: fcvt h0, s0
; GISEL-CVT-NEXT: ret ; GISEL-CVT-NEXT: ret
; GISEL-FP16-LABEL: test_sqrt: ; GISEL-FP16-LABEL: test_sqrt:
; GISEL-FP16-NEXT: fsqrt h0, h0 ; GISEL-FP16-NEXT: fsqrt h0, h0
; GISEL-FP16-NEXT: ret ; GISEL-FP16-NEXT: ret
define half @test_sqrt(half %a) #0 { define half @test_sqrt(half %a) #0 {
%r = call half @llvm.sqrt.f16(half %a) %r = call half @llvm.sqrt.f16(half %a)
ret half %r ret half %r
} }
; CHECK-COMMON-LABEL: test_powi:
; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
; CHECK-COMMON-NEXT: mov x29, sp
; CHECK-COMMON-NEXT: fcvt s0, h0
; CHECK-COMMON-NEXT: bl {{_?}}__powisf2
; CHECK-COMMON-NEXT: fcvt h0, s0
; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
; CHECK-COMMON-NEXT: ret
define half @test_powi(half %a, i32 %b) #0 { define half @test_powi(half %a, i32 %b) #0 {
%r = call half @llvm.powi.f16.i32(half %a, i32 %b) %r = call half @llvm.powi.f16.i32(half %a, i32 %b)
ret half %r ret half %r
} }
; FALLBACK-NOT: remark:{{.*}}test_sin ; FALLBACK-NOT: remark:{{.*}}test_sin
; FALLBACK-FP16-NOT: remark:{{.*}}test_sin ; FALLBACK-FP16-NOT: remark:{{.*}}test_sin
; CHECK-COMMON-LABEL: test_sin:
; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
; CHECK-COMMON-NEXT: mov x29, sp
; CHECK-COMMON-NEXT: fcvt s0, h0
; CHECK-COMMON-NEXT: bl {{_?}}sinf
; CHECK-COMMON-NEXT: fcvt h0, s0
; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
; CHECK-COMMON-NEXT: ret
; GISEL-LABEL: test_sin: ; GISEL-LABEL: test_sin:
; GISEL-NEXT: stp x29, x30, [sp, #-16]! ; GISEL-NEXT: stp x29, x30, [sp, #-16]!
; GISEL-NEXT: mov x29, sp ; GISEL-NEXT: mov x29, sp
Context not available.
%r = call half @llvm.sin.f16(half %a) %r = call half @llvm.sin.f16(half %a)
ret half %r ret half %r
} }
; FALLBACK-NOT: remark:{{.*}}test_cos ; FALLBACK-NOT: remark:{{.*}}test_cos
; FALLBACK-FP16-NOT: remark:{{.*}}test_cos ; FALLBACK-FP16-NOT: remark:{{.*}}test_cos
; CHECK-COMMON-LABEL: test_cos:
; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
; CHECK-COMMON-NEXT: mov x29, sp
; CHECK-COMMON-NEXT: fcvt s0, h0
; CHECK-COMMON-NEXT: bl {{_?}}cosf
; CHECK-COMMON-NEXT: fcvt h0, s0
; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
; CHECK-COMMON-NEXT: ret
; GISEL-LABEL: test_cos: ; GISEL-LABEL: test_cos:
; GISEL-NEXT: stp x29, x30, [sp, #-16]! ; GISEL-NEXT: stp x29, x30, [sp, #-16]!
; GISEL-NEXT: mov x29, sp ; GISEL-NEXT: mov x29, sp
Context not available.
%r = call half @llvm.cos.f16(half %a) %r = call half @llvm.cos.f16(half %a)
ret half %r ret half %r
} }
; CHECK-COMMON-LABEL: test_pow:
; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
; CHECK-COMMON-NEXT: mov x29, sp
; CHECK-COMMON-NEXT: fcvt s0, h0
; CHECK-COMMON-NEXT: fcvt s1, h1
; CHECK-COMMON-NEXT: bl {{_?}}powf
; CHECK-COMMON-NEXT: fcvt h0, s0
; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
; CHECK-COMMON-NEXT: ret
define half @test_pow(half %a, half %b) #0 { define half @test_pow(half %a, half %b) #0 {
%r = call half @llvm.pow.f16(half %a, half %b) %r = call half @llvm.pow.f16(half %a, half %b)
ret half %r ret half %r
} }
; FALLBACK-NOT: remark:{{.*}}test_exp ; FALLBACK-NOT: remark:{{.*}}test_exp
; FALLBACK-FP16-NOT: remark:{{.*}}test_exp ; FALLBACK-FP16-NOT: remark:{{.*}}test_exp
; CHECK-COMMON-LABEL: test_exp:
; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
; CHECK-COMMON-NEXT: mov x29, sp
; CHECK-COMMON-NEXT: fcvt s0, h0
; CHECK-COMMON-NEXT: bl {{_?}}expf
; CHECK-COMMON-NEXT: fcvt h0, s0
; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
; CHECK-COMMON-NEXT: ret
; GISEL-LABEL: test_exp: ; GISEL-LABEL: test_exp:
; GISEL-NEXT: stp x29, x30, [sp, #-16]! ; GISEL-NEXT: stp x29, x30, [sp, #-16]!
; GISEL-NEXT: mov x29, sp ; GISEL-NEXT: mov x29, sp
Context not available.
%r = call half @llvm.exp.f16(half %a) %r = call half @llvm.exp.f16(half %a)
ret half %r ret half %r
} }
; CHECK-COMMON-LABEL: test_exp2:
; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
; CHECK-COMMON-NEXT: mov x29, sp
; CHECK-COMMON-NEXT: fcvt s0, h0
; CHECK-COMMON-NEXT: bl {{_?}}exp2f
; CHECK-COMMON-NEXT: fcvt h0, s0
; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
; CHECK-COMMON-NEXT: ret
; GISEL-LABEL: test_exp2: ; GISEL-LABEL: test_exp2:
; GISEL-NEXT: stp x29, x30, [sp, #-16]! ; GISEL-NEXT: stp x29, x30, [sp, #-16]!
; GISEL-NEXT: mov x29, sp ; GISEL-NEXT: mov x29, sp
Context not available.
%r = call half @llvm.exp2.f16(half %a) %r = call half @llvm.exp2.f16(half %a)
ret half %r ret half %r
} }
; FALLBACK-NOT: remark:{{.*}}test_log ; FALLBACK-NOT: remark:{{.*}}test_log
; FALLBACK-FP16-NOT: remark:{{.*}}test_log ; FALLBACK-FP16-NOT: remark:{{.*}}test_log
; CHECK-COMMON-LABEL: test_log:
; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
; CHECK-COMMON-NEXT: mov x29, sp
; CHECK-COMMON-NEXT: fcvt s0, h0
; CHECK-COMMON-NEXT: bl {{_?}}logf
; CHECK-COMMON-NEXT: fcvt h0, s0
; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
; CHECK-COMMON-NEXT: ret
; GISEL-LABEL: test_log: ; GISEL-LABEL: test_log:
; GISEL: stp x29, x30, [sp, #-16]! ; GISEL: stp x29, x30, [sp, #-16]!
; GISEL-NEXT: mov x29, sp ; GISEL-NEXT: mov x29, sp
Context not available.
; GISEL-NEXT: fcvt h0, s0 ; GISEL-NEXT: fcvt h0, s0
; GISEL-NEXT: ldp x29, x30, [sp], #16 ; GISEL-NEXT: ldp x29, x30, [sp], #16
; GISEL-NEXT: ret ; GISEL-NEXT: ret
define half @test_log(half %a) #0 { define half @test_log(half %a) #0 {
%r = call half @llvm.log.f16(half %a) %r = call half @llvm.log.f16(half %a)
ret half %r ret half %r
} }
; FALLBACK-NOT: remark:{{.*}}test_log10 ; FALLBACK-NOT: remark:{{.*}}test_log10
; FALLBACK-FP16-NOT: remark:{{.*}}test_log10 ; FALLBACK-FP16-NOT: remark:{{.*}}test_log10
; CHECK-COMMON-LABEL: test_log10:
; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
; CHECK-COMMON-NEXT: mov x29, sp
; CHECK-COMMON-NEXT: fcvt s0, h0
; CHECK-COMMON-NEXT: bl {{_?}}log10f
; CHECK-COMMON-NEXT: fcvt h0, s0
; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
; CHECK-COMMON-NEXT: ret
; GISEL-LABEL: test_log10: ; GISEL-LABEL: test_log10:
; GISEL-NEXT: stp x29, x30, [sp, #-16]! ; GISEL-NEXT: stp x29, x30, [sp, #-16]!
; GISEL-NEXT: mov x29, sp ; GISEL-NEXT: mov x29, sp
Context not available.
; GISEL-NEXT: fcvt h0, s0 ; GISEL-NEXT: fcvt h0, s0
; GISEL-NEXT: ldp x29, x30, [sp], #16 ; GISEL-NEXT: ldp x29, x30, [sp], #16
; GISEL-NEXT: ret ; GISEL-NEXT: ret
define half @test_log10(half %a) #0 { define half @test_log10(half %a) #0 {
%r = call half @llvm.log10.f16(half %a) %r = call half @llvm.log10.f16(half %a)
ret half %r ret half %r
} }
; FALLBACK-NOT: remark:{{.*}}test_log2 ; FALLBACK-NOT: remark:{{.*}}test_log2
; FALLBACK-FP16-NOT: remark:{{.*}}test_log2 ; FALLBACK-FP16-NOT: remark:{{.*}}test_log2
; CHECK-COMMON-LABEL: test_log2:
; CHECK-COMMON-NEXT: stp x29, x30, [sp, #-16]!
; CHECK-COMMON-NEXT: mov x29, sp
; CHECK-COMMON-NEXT: fcvt s0, h0
; CHECK-COMMON-NEXT: bl {{_?}}log2f
; CHECK-COMMON-NEXT: fcvt h0, s0
; CHECK-COMMON-NEXT: ldp x29, x30, [sp], #16
; CHECK-COMMON-NEXT: ret
; GISEL-LABEL: test_log2: ; GISEL-LABEL: test_log2:
; GISEL-NEXT: stp x29, x30, [sp, #-16]! ; GISEL-NEXT: stp x29, x30, [sp, #-16]!
; GISEL-NEXT: mov x29, sp ; GISEL-NEXT: mov x29, sp
Context not available.
; GISEL-NEXT: fcvt h0, s0 ; GISEL-NEXT: fcvt h0, s0
; GISEL-NEXT: ldp x29, x30, [sp], #16 ; GISEL-NEXT: ldp x29, x30, [sp], #16
; GISEL-NEXT: ret ; GISEL-NEXT: ret
define half @test_log2(half %a) #0 { define half @test_log2(half %a) #0 {
%r = call half @llvm.log2.f16(half %a) %r = call half @llvm.log2.f16(half %a)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_fma:
; CHECK-CVT-NEXT: fcvt s2, h2
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fmadd s0, s0, s1, s2
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fma:
; CHECK-FP16-NEXT: fmadd h0, h0, h1, h2
; CHECK-FP16-NEXT: ret
define half @test_fma(half %a, half %b, half %c) #0 { define half @test_fma(half %a, half %b, half %c) #0 {
%r = call half @llvm.fma.f16(half %a, half %b, half %c) %r = call half @llvm.fma.f16(half %a, half %b, half %c)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_fabs:
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fabs s0, s0
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fabs:
; CHECK-FP16-NEXT: fabs h0, h0
; CHECK-FP16-NEXT: ret
; FALLBACK-NOT: remark:{{.*}}test_fabs ; FALLBACK-NOT: remark:{{.*}}test_fabs
; FALLBACK-FP16-NOT: remark:{{.*}}test_fabs ; FALLBACK-FP16-NOT: remark:{{.*}}test_fabs
; GISEL-CVT-LABEL: test_fabs: ; GISEL-CVT-LABEL: test_fabs:
; GISEL-CVT-NEXT: fcvt s0, h0 ; GISEL-CVT-NEXT: fcvt s0, h0
; GISEL-CVT-NEXT: fabs s0, s0 ; GISEL-CVT-NEXT: fabs s0, s0
; GISEL-CVT-NEXT: fcvt h0, s0 ; GISEL-CVT-NEXT: fcvt h0, s0
; GISEL-CVT-NEXT: ret ; GISEL-CVT-NEXT: ret
; GISEL-FP16-LABEL: test_fabs: ; GISEL-FP16-LABEL: test_fabs:
; GISEL-FP16-NEXT: fabs h0, h0 ; GISEL-FP16-NEXT: fabs h0, h0
; GISEL-FP16-NEXT: ret ; GISEL-FP16-NEXT: ret
define half @test_fabs(half %a) #0 { define half @test_fabs(half %a) #0 {
%r = call half @llvm.fabs.f16(half %a) %r = call half @llvm.fabs.f16(half %a)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_minnum:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fminnm s0, s0, s1
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_minnum:
; CHECK-FP16-NEXT: fminnm h0, h0, h1
; CHECK-FP16-NEXT: ret
define half @test_minnum(half %a, half %b) #0 { define half @test_minnum(half %a, half %b) #0 {
%r = call half @llvm.minnum.f16(half %a, half %b) %r = call half @llvm.minnum.f16(half %a, half %b)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_maxnum:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fmaxnm s0, s0, s1
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_maxnum:
; CHECK-FP16-NEXT: fmaxnm h0, h0, h1
; CHECK-FP16-NEXT: ret
define half @test_maxnum(half %a, half %b) #0 { define half @test_maxnum(half %a, half %b) #0 {
%r = call half @llvm.maxnum.f16(half %a, half %b) %r = call half @llvm.maxnum.f16(half %a, half %b)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_copysign:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: mvni.4s v2, #128, lsl #24
; CHECK-CVT-NEXT: bif.16b v0, v1, v2
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_copysign:
; CHECK-FP16-NEXT: mvni.8h v2, #128, lsl #8
; CHECK-FP16-NEXT: bif.16b v0, v1, v2
; CHECK-FP16-NEXT: ret
define half @test_copysign(half %a, half %b) #0 { define half @test_copysign(half %a, half %b) #0 {
%r = call half @llvm.copysign.f16(half %a, half %b) %r = call half @llvm.copysign.f16(half %a, half %b)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_copysign_f32:
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: mvni.4s v2, #128, lsl #24
; CHECK-CVT-NEXT: bif.16b v0, v1, v2
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_copysign_f32:
; CHECK-FP16-NEXT: fcvt h1, s1
; CHECK-FP16-NEXT: mvni.8h v2, #128, lsl #8
; CHECK-FP16-NEXT: bif.16b v0, v1, v2
; CHECK-FP16-NEXT: ret
define half @test_copysign_f32(half %a, float %b) #0 { define half @test_copysign_f32(half %a, float %b) #0 {
%tb = fptrunc float %b to half %tb = fptrunc float %b to half
%r = call half @llvm.copysign.f16(half %a, half %tb) %r = call half @llvm.copysign.f16(half %a, half %tb)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_copysign_f64:
; CHECK-CVT-NEXT: fcvt s1, d1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: mvni.4s v2, #128, lsl #24
; CHECK-CVT-NEXT: bif.16b v0, v1, v2
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_copysign_f64:
; CHECK-FP16-NEXT: fcvt h1, d1
; CHECK-FP16-NEXT: mvni.8h v2, #128, lsl #8
; CHECK-FP16-NEXT: bif.16b v0, v1, v2
; CHECK-FP16-NEXT: ret
define half @test_copysign_f64(half %a, double %b) #0 { define half @test_copysign_f64(half %a, double %b) #0 {
%tb = fptrunc double %b to half %tb = fptrunc double %b to half
%r = call half @llvm.copysign.f16(half %a, half %tb) %r = call half @llvm.copysign.f16(half %a, half %tb)
ret half %r ret half %r
} }
; Check that the FP promotion will use a truncating FP_ROUND, so we can fold
; away the (fpext (fp_round <result>)) here. ; away the (fpext (fp_round <result>)) here.
; CHECK-CVT-LABEL: test_copysign_extended:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: mvni.4s v2, #128, lsl #24
; CHECK-CVT-NEXT: bif.16b v0, v1, v2
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_copysign_extended:
; CHECK-FP16-NEXT: mvni.8h v2, #128, lsl #8
; CHECK-FP16-NEXT: bif.16b v0, v1, v2
; CHECK-FP16-NEXT: fcvt s0, h0
; CHECK-FP16-NEXT: ret
define float @test_copysign_extended(half %a, half %b) #0 { define float @test_copysign_extended(half %a, half %b) #0 {
%r = call half @llvm.copysign.f16(half %a, half %b) %r = call half @llvm.copysign.f16(half %a, half %b)
%xr = fpext half %r to float %xr = fpext half %r to float
ret float %xr ret float %xr
} }
; CHECK-CVT-LABEL: test_floor:
; CHECK-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
; CHECK-CVT-NEXT: frintm [[INT32:s[0-9]+]], [[FLOAT32]]
; CHECK-CVT-NEXT: fcvt h0, [[INT32]]
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_floor:
; CHECK-FP16-NEXT: frintm h0, h0
; CHECK-FP16-NEXT: ret
; FALLBACK-NOT: remark:{{.*}}test_floor ; FALLBACK-NOT: remark:{{.*}}test_floor
; FALLBACK-FP16-NOT: remark:{{.*}}test_floor ; FALLBACK-FP16-NOT: remark:{{.*}}test_floor
; GISEL-CVT-LABEL: test_floor: ; GISEL-CVT-LABEL: test_floor:
; GISEL-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0 ; GISEL-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
; GISEL-CVT-NEXT: frintm [[INT32:s[0-9]+]], [[FLOAT32]] ; GISEL-CVT-NEXT: frintm [[INT32:s[0-9]+]], [[FLOAT32]]
; GISEL-CVT-NEXT: fcvt h0, [[INT32]] ; GISEL-CVT-NEXT: fcvt h0, [[INT32]]
; GISEL-CVT-NEXT: ret ; GISEL-CVT-NEXT: ret
; GISEL-FP16-LABEL: test_floor: ; GISEL-FP16-LABEL: test_floor:
; GISEL-FP16-NEXT: frintm h0, h0 ; GISEL-FP16-NEXT: frintm h0, h0
; GISEL-FP16-NEXT: ret ; GISEL-FP16-NEXT: ret
define half @test_floor(half %a) #0 { define half @test_floor(half %a) #0 {
%r = call half @llvm.floor.f16(half %a) %r = call half @llvm.floor.f16(half %a)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_ceil:
; CHECK-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
; CHECK-CVT-NEXT: frintp [[INT32:s[0-9]+]], [[FLOAT32]]
; CHECK-CVT-NEXT: fcvt h0, [[INT32]]
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_ceil:
; CHECK-FP16-NEXT: frintp h0, h0
; CHECK-FP16-NEXT: ret
; FALLBACK-NOT: remark:{{.*}}test_ceil ; FALLBACK-NOT: remark:{{.*}}test_ceil
; FALLBACK-FP16-NOT: remark:{{.*}}test_ceil ; FALLBACK-FP16-NOT: remark:{{.*}}test_ceil
; GISEL-CVT-LABEL: test_ceil: ; GISEL-CVT-LABEL: test_ceil:
; GISEL-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0 ; GISEL-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
; GISEL-CVT-NEXT: frintp [[INT32:s[0-9]+]], [[FLOAT32]] ; GISEL-CVT-NEXT: frintp [[INT32:s[0-9]+]], [[FLOAT32]]
; GISEL-CVT-NEXT: fcvt h0, [[INT32]] ; GISEL-CVT-NEXT: fcvt h0, [[INT32]]
; GISEL-CVT-NEXT: ret ; GISEL-CVT-NEXT: ret
; GISEL-FP16-LABEL: test_ceil: ; GISEL-FP16-LABEL: test_ceil:
; GISEL-FP16-NEXT: frintp h0, h0 ; GISEL-FP16-NEXT: frintp h0, h0
; GISEL-FP16-NEXT: ret ; GISEL-FP16-NEXT: ret
Context not available.
%r = call half @llvm.ceil.f16(half %a) %r = call half @llvm.ceil.f16(half %a)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_trunc:
; CHECK-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
; CHECK-CVT-NEXT: frintz [[INT32:s[0-9]+]], [[FLOAT32]]
; CHECK-CVT-NEXT: fcvt h0, [[INT32]]
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_trunc:
; CHECK-FP16-NEXT: frintz h0, h0
; CHECK-FP16-NEXT: ret
define half @test_trunc(half %a) #0 { define half @test_trunc(half %a) #0 {
%r = call half @llvm.trunc.f16(half %a) %r = call half @llvm.trunc.f16(half %a)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_rint:
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: frintx s0, s0
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_rint:
; CHECK-FP16-NEXT: frintx h0, h0
; CHECK-FP16-NEXT: ret
define half @test_rint(half %a) #0 { define half @test_rint(half %a) #0 {
%r = call half @llvm.rint.f16(half %a) %r = call half @llvm.rint.f16(half %a)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_nearbyint:
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: frinti s0, s0
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_nearbyint:
; CHECK-FP16-NEXT: frinti h0, h0
; CHECK-FP16-NEXT: ret
define half @test_nearbyint(half %a) #0 { define half @test_nearbyint(half %a) #0 {
%r = call half @llvm.nearbyint.f16(half %a) %r = call half @llvm.nearbyint.f16(half %a)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_round:
; CHECK-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
; CHECK-CVT-NEXT: frinta [[INT32:s[0-9]+]], [[FLOAT32]]
; CHECK-CVT-NEXT: fcvt h0, [[INT32]]
; CHECK-CVT-NEXT: ret
; GISEL-CVT-LABEL: test_round: ; GISEL-CVT-LABEL: test_round:
; GISEL-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0 ; GISEL-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
; GISEL-CVT-NEXT: frinta [[INT32:s[0-9]+]], [[FLOAT32]] ; GISEL-CVT-NEXT: frinta [[INT32:s[0-9]+]], [[FLOAT32]]
; GISEL-CVT-NEXT: fcvt h0, [[INT32]] ; GISEL-CVT-NEXT: fcvt h0, [[INT32]]
; GISEL-CVT-NEXT: ret ; GISEL-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_round:
; CHECK-FP16-NEXT: frinta h0, h0
; CHECK-FP16-NEXT: ret
; GISEL-FP16-LABEL: test_round: ; GISEL-FP16-LABEL: test_round:
; GISEL-FP16-NEXT: frinta h0, h0 ; GISEL-FP16-NEXT: frinta h0, h0
; GISEL-FP16-NEXT: ret ; GISEL-FP16-NEXT: ret
define half @test_round(half %a) #0 { define half @test_round(half %a) #0 {
%r = call half @llvm.round.f16(half %a) %r = call half @llvm.round.f16(half %a)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_roundeven:
; CHECK-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
; CHECK-CVT-NEXT: frintn [[INT32:s[0-9]+]], [[FLOAT32]]
; CHECK-CVT-NEXT: fcvt h0, [[INT32]]
; CHECK-CVT-NEXT: ret
; GISEL-CVT-LABEL: test_roundeven: ; GISEL-CVT-LABEL: test_roundeven:
; GISEL-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0 ; GISEL-CVT-NEXT: fcvt [[FLOAT32:s[0-9]+]], h0
; GISEL-CVT-NEXT: frintn [[INT32:s[0-9]+]], [[FLOAT32]] ; GISEL-CVT-NEXT: frintn [[INT32:s[0-9]+]], [[FLOAT32]]
; GISEL-CVT-NEXT: fcvt h0, [[INT32]] ; GISEL-CVT-NEXT: fcvt h0, [[INT32]]
; GISEL-CVT-NEXT: ret ; GISEL-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_roundeven:
; CHECK-FP16-NEXT: frintn h0, h0
; CHECK-FP16-NEXT: ret
; GISEL-FP16-LABEL: test_roundeven: ; GISEL-FP16-LABEL: test_roundeven:
; GISEL-FP16-NEXT: frintn h0, h0 ; GISEL-FP16-NEXT: frintn h0, h0
; GISEL-FP16-NEXT: ret ; GISEL-FP16-NEXT: ret
define half @test_roundeven(half %a) #0 { define half @test_roundeven(half %a) #0 {
%r = call half @llvm.roundeven.f16(half %a) %r = call half @llvm.roundeven.f16(half %a)
ret half %r ret half %r
} }
; CHECK-CVT-LABEL: test_fmuladd:
; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fmul s0, s0, s1
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcvt s1, h2
; CHECK-CVT-NEXT: fadd s0, s0, s1
; CHECK-CVT-NEXT: fcvt h0, s0
; CHECK-CVT-NEXT: ret
; CHECK-FP16-LABEL: test_fmuladd:
; CHECK-FP16-NEXT: fmadd h0, h0, h1, h2
; CHECK-FP16-NEXT: ret
define half @test_fmuladd(half %a, half %b, half %c) #0 { define half @test_fmuladd(half %a, half %b, half %c) #0 {
%r = call half @llvm.fmuladd.f16(half %a, half %b, half %c) %r = call half @llvm.fmuladd.f16(half %a, half %b, half %c)
ret half %r ret half %r
} }
attributes #0 = { nounwind } attributes #0 = { nounwind }
;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
; CHECK-COMMON: {{.*}}
; CHECK-CVT: {{.*}}
; CHECK-FP16: {{.*}}
; FALLBACK: {{.*}}
; FALLBACK-FP16: {{.*}}
; GISEL: {{.*}}
; GISEL-CVT: {{.*}}
; GISEL-FP16: {{.*}}
Context not available.

llvm/test/CodeGen/AArch64/fadd-combines.ll

llvm/test/CodeGen/AArch64/faddp-half.ll

llvm/test/CodeGen/AArch64/faddp.ll

llvm/test/CodeGen/AArch64/fast-isel-addressing-modes.ll

llvm/test/CodeGen/AArch64/fast-isel-gep.ll

llvm/test/CodeGen/AArch64/fast-isel-memcpy.ll

llvm/test/CodeGen/AArch64/fast-isel-shift.ll

llvm/test/CodeGen/AArch64/fcopysign.ll

llvm/test/CodeGen/AArch64/fcvt_combine.ll

llvm/test/CodeGen/AArch64/fdiv-combine.ll

llvm/test/CodeGen/AArch64/fixed-vector-deinterleave.ll

llvm/test/CodeGen/AArch64/fixed-vector-interleave.ll

llvm/test/CodeGen/AArch64/flags-multiuse.ll

llvm/test/CodeGen/AArch64/fmaximum-legalization.ll

llvm/test/CodeGen/AArch64/fmlal-loreg.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc -mtriple=aarch64 -mattr=+fp16fml -verify-machineinstrs %s -o - 2>&1 | FileCheck %s ; RUN: llc -mtriple=aarch64 -mattr=+fp16fml -verify-machineinstrs %s -o - 2>&1 | FileCheck %s
; This tests that the fmlal/fmlal2 instructions only accept lo registers for ; This tests that the fmlal/fmlal2 instructions only accept lo registers for
dmgreenUnsubmitted
Done
ReplyInline Actions

I'm not sure why these blank lines were removed? I think they can be kept in.

dmgreen: I'm not sure why these blank lines were removed? I think they can be kept in.
; the index operand, using inline asm to force the available registers. ; the index operand, using inline asm to force the available registers.
define <4 x float> @test(ptr %lhs_panel, ptr %rhs_panel, <4 x float> %a) { define <4 x float> @test(ptr %lhs_panel, ptr %rhs_panel, <4 x float> %a) {
; CHECK-LABEL: test: ; CHECK-LABEL: test:
; CHECK: // %bb.0: // %entry ; CHECK: // %bb.0: // %entry
Context not available.
%z = fadd <4 x float> %vfmlal_low3.i, %vfmlal_high3.i %z = fadd <4 x float> %vfmlal_low3.i, %vfmlal_high3.i
ret <4 x float> %z ret <4 x float> %z
} }
define void @loop(ptr %out_tile, ptr %lhs_panel, ptr %rhs_panel, i32 noundef %K, i32 noundef %flags) { define void @loop(ptr %out_tile, ptr %lhs_panel, ptr %rhs_panel, i32 noundef %K, i32 noundef %flags) {
; CHECK-LABEL: loop: ; CHECK-LABEL: loop:
; CHECK: // %bb.0: // %entry ; CHECK: // %bb.0: // %entry
; CHECK-NEXT: movi v0.2d, #0000000000000000 ; CHECK-NEXT: movi v0.2d, #0000000000000000
; CHECK-NEXT: mov w8, w3
; CHECK-NEXT: movi v1.2d, #0000000000000000 ; CHECK-NEXT: movi v1.2d, #0000000000000000
; CHECK-NEXT: mov w8, w3
; CHECK-NEXT: .LBB1_1: // %for.body ; CHECK-NEXT: .LBB1_1: // %for.body
; CHECK-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldr q2, [x1], #2 ; CHECK-NEXT: ldr q2, [x1], #2
; CHECK-NEXT: ldr q3, [x2], #2
; CHECK-NEXT: subs x8, x8, #1 ; CHECK-NEXT: subs x8, x8, #1
; CHECK-NEXT: ldr q3, [x2], #2
; CHECK-NEXT: fmlal v0.4s, v3.4h, v2.h[0] ; CHECK-NEXT: fmlal v0.4s, v3.4h, v2.h[0]
; CHECK-NEXT: fmlal2 v1.4s, v3.4h, v2.h[0] ; CHECK-NEXT: fmlal2 v1.4s, v3.4h, v2.h[0]
; CHECK-NEXT: b.ne .LBB1_1 ; CHECK-NEXT: b.ne .LBB1_1
Context not available.
entry: entry:
%wide.trip.count = zext i32 %K to i64 %wide.trip.count = zext i32 %K to i64
br label %for.body br label %for.body
for.cond.cleanup: ; preds = %for.body for.cond.cleanup: ; preds = %for.body
store <4 x float> %vfmlal_low3.i, ptr %out_tile, align 4 store <4 x float> %vfmlal_low3.i, ptr %out_tile, align 4
%add.ptr1399 = getelementptr inbounds float, ptr %out_tile, i64 4 %add.ptr1399 = getelementptr inbounds float, ptr %out_tile, i64 4
store <4 x float> %vfmlal_high3.i, ptr %add.ptr1399, align 4 store <4 x float> %vfmlal_high3.i, ptr %add.ptr1399, align 4
ret void ret void
for.body: ; preds = %entry, %for.body for.body: ; preds = %entry, %for.body
%indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.body ] %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.body ]
%acc0.01714 = phi <4 x float> [ zeroinitializer, %entry ], [ %vfmlal_low3.i, %for.body ] %acc0.01714 = phi <4 x float> [ zeroinitializer, %entry ], [ %vfmlal_low3.i, %for.body ]
Context not available.
%exitcond.not = icmp eq i64 %indvars.iv.next, %wide.trip.count %exitcond.not = icmp eq i64 %indvars.iv.next, %wide.trip.count
br i1 %exitcond.not, label %for.cond.cleanup, label %for.body br i1 %exitcond.not, label %for.cond.cleanup, label %for.body
} }
define void @sink(ptr %out_tile, ptr %lhs_panel, ptr %rhs_panel, i32 noundef %K, i32 noundef %flags, <8 x half> noundef %lhs) { define void @sink(ptr %out_tile, ptr %lhs_panel, ptr %rhs_panel, i32 noundef %K, i32 noundef %flags, <8 x half> noundef %lhs) {
; CHECK-LABEL: sink: ; CHECK-LABEL: sink:
; CHECK: // %bb.0: // %entry ; CHECK: // %bb.0: // %entry
; CHECK-NEXT: movi v1.2d, #0000000000000000 ; CHECK-NEXT: movi v1.2d, #0000000000000000
; CHECK-NEXT: mov w8, w3
; CHECK-NEXT: movi v2.2d, #0000000000000000 ; CHECK-NEXT: movi v2.2d, #0000000000000000
; CHECK-NEXT: mov w8, w3
; CHECK-NEXT: .LBB2_1: // %for.body ; CHECK-NEXT: .LBB2_1: // %for.body
; CHECK-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldr q3, [x2], #2 ; CHECK-NEXT: ldr q3, [x2], #2
Context not available.
%vecinit89 = shufflevector <8 x half> %lhs, <8 x half> undef, <8 x i32> zeroinitializer %vecinit89 = shufflevector <8 x half> %lhs, <8 x half> undef, <8 x i32> zeroinitializer
%wide.trip.count = zext i32 %K to i64 %wide.trip.count = zext i32 %K to i64
br label %for.body br label %for.body
for.cond.cleanup: ; preds = %for.body for.cond.cleanup: ; preds = %for.body
store <4 x float> %vfmlal_low3.i, ptr %out_tile, align 4 store <4 x float> %vfmlal_low3.i, ptr %out_tile, align 4
%add.ptr1395 = getelementptr inbounds float, ptr %out_tile, i64 4 %add.ptr1395 = getelementptr inbounds float, ptr %out_tile, i64 4
store <4 x float> %vfmlal_high3.i, ptr %add.ptr1395, align 4 store <4 x float> %vfmlal_high3.i, ptr %add.ptr1395, align 4
ret void ret void
for.body: ; preds = %entry, %for.body for.body: ; preds = %entry, %for.body
%indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.body ] %indvars.iv = phi i64 [ 0, %entry ], [ %indvars.iv.next, %for.body ]
%acc0.01702 = phi <4 x float> [ zeroinitializer, %entry ], [ %vfmlal_low3.i, %for.body ] %acc0.01702 = phi <4 x float> [ zeroinitializer, %entry ], [ %vfmlal_low3.i, %for.body ]
Context not available.
%exitcond.not = icmp eq i64 %indvars.iv.next, %wide.trip.count %exitcond.not = icmp eq i64 %indvars.iv.next, %wide.trip.count
br i1 %exitcond.not, label %for.cond.cleanup, label %for.body br i1 %exitcond.not, label %for.cond.cleanup, label %for.body
} }
declare <4 x float> @llvm.aarch64.neon.fmlal.v4f32.v8f16(<4 x float>, <8 x half>, <8 x half>) #2 declare <4 x float> @llvm.aarch64.neon.fmlal.v4f32.v8f16(<4 x float>, <8 x half>, <8 x half>) #2
declare <4 x float> @llvm.aarch64.neon.fmlal2.v4f32.v8f16(<4 x float>, <8 x half>, <8 x half>) #2 declare <4 x float> @llvm.aarch64.neon.fmlal2.v4f32.v8f16(<4 x float>, <8 x half>, <8 x half>) #2
Context not available.

llvm/test/CodeGen/AArch64/fold-csel-cttz-and.ll

llvm/test/CodeGen/AArch64/fp-conversion-to-tbl.ll

llvm/test/CodeGen/AArch64/fp-intrinsics-vector.ll

llvm/test/CodeGen/AArch64/fp16-v16-instructions.ll

llvm/test/CodeGen/AArch64/fp16-v8-instructions.ll

llvm/test/CodeGen/AArch64/fpclamptosat_vec.ll

llvm/test/CodeGen/AArch64/fptosi-sat-vector.ll

llvm/test/CodeGen/AArch64/fptoui-sat-scalar.ll

llvm/test/CodeGen/AArch64/fptoui-sat-vector.ll

Context not available.
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: mov d1, v0.d[1] ; CHECK-NEXT: mov d1, v0.d[1]
; CHECK-NEXT: fcvtzu w8, d0 ; CHECK-NEXT: fcvtzu w8, d0
; CHECK-NEXT: fcvtzu w9, d1
; CHECK-NEXT: fmov s0, w8 ; CHECK-NEXT: fmov s0, w8
; CHECK-NEXT: fcvtzu w8, d1 ; CHECK-NEXT: mov v0.s[1], w9
; CHECK-NEXT: mov v0.s[1], w8
; CHECK-NEXT: // kill: def $d0 killed $d0 killed $q0 ; CHECK-NEXT: // kill: def $d0 killed $d0 killed $q0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%x = call <2 x i32> @llvm.fptoui.sat.v2f64.v2i32(<2 x double> %f) %x = call <2 x i32> @llvm.fptoui.sat.v2f64.v2i32(<2 x double> %f)
Context not available.
; CHECK-LABEL: test_unsigned_v3f64_v3i32: ; CHECK-LABEL: test_unsigned_v3f64_v3i32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: fcvtzu w8, d0 ; CHECK-NEXT: fcvtzu w8, d0
; CHECK-NEXT: fcvtzu w9, d1
; CHECK-NEXT: fmov s0, w8 ; CHECK-NEXT: fmov s0, w8
; CHECK-NEXT: fcvtzu w8, d1
; CHECK-NEXT: mov v0.s[1], w8
; CHECK-NEXT: fcvtzu w8, d2 ; CHECK-NEXT: fcvtzu w8, d2
; CHECK-NEXT: mov v0.s[1], w9
; CHECK-NEXT: mov v0.s[2], w8 ; CHECK-NEXT: mov v0.s[2], w8
; CHECK-NEXT: fcvtzu w8, d0 ; CHECK-NEXT: fcvtzu w8, d0
; CHECK-NEXT: mov v0.s[3], w8 ; CHECK-NEXT: mov v0.s[3], w8
Context not available.
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: mov d2, v0.d[1] ; CHECK-NEXT: mov d2, v0.d[1]
; CHECK-NEXT: fcvtzu w8, d0 ; CHECK-NEXT: fcvtzu w8, d0
; CHECK-NEXT: fcvtzu w9, d2
; CHECK-NEXT: fmov s0, w8 ; CHECK-NEXT: fmov s0, w8
; CHECK-NEXT: fcvtzu w8, d2
; CHECK-NEXT: mov v0.s[1], w8
; CHECK-NEXT: fcvtzu w8, d1 ; CHECK-NEXT: fcvtzu w8, d1
; CHECK-NEXT: mov d1, v1.d[1] ; CHECK-NEXT: mov d1, v1.d[1]
; CHECK-NEXT: mov v0.s[1], w9
; CHECK-NEXT: mov v0.s[2], w8 ; CHECK-NEXT: mov v0.s[2], w8
; CHECK-NEXT: fcvtzu w8, d1 ; CHECK-NEXT: fcvtzu w8, d1
; CHECK-NEXT: mov v0.s[3], w8 ; CHECK-NEXT: mov v0.s[3], w8
Context not available.
; CHECK-NEXT: adrp x8, .LCPI14_1 ; CHECK-NEXT: adrp x8, .LCPI14_1
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0 ; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI14_1] ; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI14_1]
; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: bl __gttf2 ; CHECK-NEXT: bl __gttf2
; CHECK-NEXT: cmp w0, #0 ; CHECK-NEXT: cmp w0, #0
; CHECK-NEXT: csinv w8, w19, wzr, le ; CHECK-NEXT: csinv w8, w19, wzr, le
Context not available.
; CHECK-NEXT: .cfi_offset w19, -8 ; CHECK-NEXT: .cfi_offset w19, -8
; CHECK-NEXT: .cfi_offset w20, -16 ; CHECK-NEXT: .cfi_offset w20, -16
; CHECK-NEXT: .cfi_offset w30, -32 ; CHECK-NEXT: .cfi_offset w30, -32
; CHECK-NEXT: adrp x8, .LCPI15_0
; CHECK-NEXT: stp q1, q0, [sp, #32] // 32-byte Folded Spill
; CHECK-NEXT: mov v2.16b, v1.16b ; CHECK-NEXT: mov v2.16b, v1.16b
; CHECK-NEXT: mov v0.16b, v2.16b ; CHECK-NEXT: stp q1, q0, [sp, #32] // 32-byte Folded Spill
; CHECK-NEXT: adrp x8, .LCPI15_0
; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI15_0] ; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI15_0]
; CHECK-NEXT: mov v0.16b, v2.16b
; CHECK-NEXT: str q1, [sp, #16] // 16-byte Folded Spill ; CHECK-NEXT: str q1, [sp, #16] // 16-byte Folded Spill
; CHECK-NEXT: bl __getf2 ; CHECK-NEXT: bl __getf2
; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload
Context not available.
; CHECK-NEXT: adrp x8, .LCPI15_1 ; CHECK-NEXT: adrp x8, .LCPI15_1
; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0 ; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI15_1] ; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI15_1]
; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: str q1, [sp] // 16-byte Folded Spill ; CHECK-NEXT: str q1, [sp] // 16-byte Folded Spill
; CHECK-NEXT: bl __gttf2 ; CHECK-NEXT: bl __gttf2
; CHECK-NEXT: ldr q0, [sp, #48] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: cmp w0, #0
; CHECK-NEXT: ldr q1, [sp, #16] // 16-byte Folded Reload ; CHECK-NEXT: ldr q1, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: cmp w0, #0
; CHECK-NEXT: csinv w20, w19, wzr, le ; CHECK-NEXT: csinv w20, w19, wzr, le
; CHECK-NEXT: bl __getf2 ; CHECK-NEXT: bl __getf2
; CHECK-NEXT: ldr q0, [sp, #48] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: mov w19, w0 ; CHECK-NEXT: mov w19, w0
; CHECK-NEXT: bl __fixunstfsi ; CHECK-NEXT: bl __fixunstfsi
; CHECK-NEXT: ldr q0, [sp, #48] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: ldr q1, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q1, [sp] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: csel w19, wzr, w0, lt ; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: bl __gttf2 ; CHECK-NEXT: bl __gttf2
; CHECK-NEXT: cmp w0, #0 ; CHECK-NEXT: cmp w0, #0
Context not available.
; CHECK-NEXT: .cfi_offset w19, -8 ; CHECK-NEXT: .cfi_offset w19, -8
; CHECK-NEXT: .cfi_offset w20, -16 ; CHECK-NEXT: .cfi_offset w20, -16
; CHECK-NEXT: .cfi_offset w30, -32 ; CHECK-NEXT: .cfi_offset w30, -32
; CHECK-NEXT: adrp x8, .LCPI16_0
; CHECK-NEXT: stp q0, q2, [sp, #48] // 32-byte Folded Spill ; CHECK-NEXT: stp q0, q2, [sp, #48] // 32-byte Folded Spill
; CHECK-NEXT: mov v2.16b, v1.16b ; CHECK-NEXT: mov v2.16b, v1.16b
; CHECK-NEXT: adrp x8, .LCPI16_0
; CHECK-NEXT: str q1, [sp] // 16-byte Folded Spill ; CHECK-NEXT: str q1, [sp] // 16-byte Folded Spill
; CHECK-NEXT: mov v0.16b, v2.16b
; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI16_0] ; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI16_0]
; CHECK-NEXT: mov v0.16b, v2.16b
; CHECK-NEXT: str q1, [sp, #32] // 16-byte Folded Spill ; CHECK-NEXT: str q1, [sp, #32] // 16-byte Folded Spill
; CHECK-NEXT: bl __getf2 ; CHECK-NEXT: bl __getf2
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
Context not available.
; CHECK-NEXT: adrp x8, .LCPI16_1 ; CHECK-NEXT: adrp x8, .LCPI16_1
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0 ; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI16_1] ; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI16_1]
; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: str q1, [sp, #16] // 16-byte Folded Spill ; CHECK-NEXT: str q1, [sp, #16] // 16-byte Folded Spill
; CHECK-NEXT: bl __gttf2 ; CHECK-NEXT: bl __gttf2
; CHECK-NEXT: ldp q1, q0, [sp, #32] // 32-byte Folded Reload ; CHECK-NEXT: ldp q1, q0, [sp, #32] // 32-byte Folded Reload
Context not available.
; CHECK-NEXT: mov w19, w0 ; CHECK-NEXT: mov w19, w0
; CHECK-NEXT: bl __fixunstfsi ; CHECK-NEXT: bl __fixunstfsi
; CHECK-NEXT: ldr q0, [sp, #48] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: ldr q1, [sp, #16] // 16-byte Folded Reload ; CHECK-NEXT: ldr q1, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: csel w19, wzr, w0, lt ; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: bl __gttf2 ; CHECK-NEXT: bl __gttf2
; CHECK-NEXT: cmp w0, #0 ; CHECK-NEXT: cmp w0, #0
Context not available.
; CHECK-NEXT: mov w19, w0 ; CHECK-NEXT: mov w19, w0
; CHECK-NEXT: bl __fixunstfsi ; CHECK-NEXT: bl __fixunstfsi
; CHECK-NEXT: ldr q0, [sp, #64] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #64] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: ldr q1, [sp, #16] // 16-byte Folded Reload ; CHECK-NEXT: ldr q1, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: csel w19, wzr, w0, lt ; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: bl __gttf2 ; CHECK-NEXT: bl __gttf2
; CHECK-NEXT: cmp w0, #0 ; CHECK-NEXT: cmp w0, #0
; CHECK-NEXT: ldr q0, [sp, #48] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: csinv w8, w19, wzr, le
; CHECK-NEXT: ldr x30, [sp, #80] // 8-byte Folded Reload ; CHECK-NEXT: ldr x30, [sp, #80] // 8-byte Folded Reload
; CHECK-NEXT: csinv w8, w19, wzr, le
; CHECK-NEXT: ldp x20, x19, [sp, #96] // 16-byte Folded Reload ; CHECK-NEXT: ldp x20, x19, [sp, #96] // 16-byte Folded Reload
; CHECK-NEXT: mov v0.s[2], w8 ; CHECK-NEXT: mov v0.s[2], w8
; CHECK-NEXT: add sp, sp, #112 ; CHECK-NEXT: add sp, sp, #112
Context not available.
; CHECK-NEXT: .cfi_offset w19, -8 ; CHECK-NEXT: .cfi_offset w19, -8
; CHECK-NEXT: .cfi_offset w20, -16 ; CHECK-NEXT: .cfi_offset w20, -16
; CHECK-NEXT: .cfi_offset w30, -32 ; CHECK-NEXT: .cfi_offset w30, -32
; CHECK-NEXT: adrp x8, .LCPI17_0
; CHECK-NEXT: stp q0, q2, [sp, #16] // 32-byte Folded Spill ; CHECK-NEXT: stp q0, q2, [sp, #16] // 32-byte Folded Spill
; CHECK-NEXT: mov v2.16b, v1.16b ; CHECK-NEXT: mov v2.16b, v1.16b
; CHECK-NEXT: adrp x8, .LCPI17_0
; CHECK-NEXT: str q1, [sp] // 16-byte Folded Spill ; CHECK-NEXT: str q1, [sp] // 16-byte Folded Spill
; CHECK-NEXT: mov v0.16b, v2.16b
; CHECK-NEXT: str q3, [sp, #80] // 16-byte Folded Spill
; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI17_0] ; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI17_0]
; CHECK-NEXT: str q3, [sp, #80] // 16-byte Folded Spill
; CHECK-NEXT: mov v0.16b, v2.16b
; CHECK-NEXT: str q1, [sp, #64] // 16-byte Folded Spill ; CHECK-NEXT: str q1, [sp, #64] // 16-byte Folded Spill
; CHECK-NEXT: bl __getf2 ; CHECK-NEXT: bl __getf2
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
Context not available.
; CHECK-NEXT: adrp x8, .LCPI17_1 ; CHECK-NEXT: adrp x8, .LCPI17_1
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0 ; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI17_1] ; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI17_1]
; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: str q1, [sp, #48] // 16-byte Folded Spill ; CHECK-NEXT: str q1, [sp, #48] // 16-byte Folded Spill
; CHECK-NEXT: bl __gttf2 ; CHECK-NEXT: bl __gttf2
; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: cmp w0, #0
; CHECK-NEXT: ldr q1, [sp, #64] // 16-byte Folded Reload ; CHECK-NEXT: ldr q1, [sp, #64] // 16-byte Folded Reload
; CHECK-NEXT: cmp w0, #0
; CHECK-NEXT: csinv w20, w19, wzr, le ; CHECK-NEXT: csinv w20, w19, wzr, le
; CHECK-NEXT: bl __getf2 ; CHECK-NEXT: bl __getf2
; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: mov w19, w0 ; CHECK-NEXT: mov w19, w0
; CHECK-NEXT: bl __fixunstfsi ; CHECK-NEXT: bl __fixunstfsi
; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: ldr q1, [sp, #48] // 16-byte Folded Reload ; CHECK-NEXT: ldr q1, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: csel w19, wzr, w0, lt ; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: bl __gttf2 ; CHECK-NEXT: bl __gttf2
; CHECK-NEXT: cmp w0, #0 ; CHECK-NEXT: cmp w0, #0
Context not available.
; CHECK-NEXT: mov w19, w0 ; CHECK-NEXT: mov w19, w0
; CHECK-NEXT: bl __fixunstfsi ; CHECK-NEXT: bl __fixunstfsi
; CHECK-NEXT: ldr q0, [sp, #80] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #80] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: ldr q1, [sp, #48] // 16-byte Folded Reload ; CHECK-NEXT: ldr q1, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: cmp w19, #0
; CHECK-NEXT: csel w19, wzr, w0, lt ; CHECK-NEXT: csel w19, wzr, w0, lt
; CHECK-NEXT: bl __gttf2 ; CHECK-NEXT: bl __gttf2
; CHECK-NEXT: cmp w0, #0 ; CHECK-NEXT: cmp w0, #0
; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: csinv w8, w19, wzr, le
; CHECK-NEXT: ldr x30, [sp, #96] // 8-byte Folded Reload ; CHECK-NEXT: ldr x30, [sp, #96] // 8-byte Folded Reload
; CHECK-NEXT: csinv w8, w19, wzr, le
; CHECK-NEXT: ldp x20, x19, [sp, #112] // 16-byte Folded Reload ; CHECK-NEXT: ldp x20, x19, [sp, #112] // 16-byte Folded Reload
; CHECK-NEXT: mov v0.s[3], w8 ; CHECK-NEXT: mov v0.s[3], w8
; CHECK-NEXT: add sp, sp, #128 ; CHECK-NEXT: add sp, sp, #128
Context not available.
; CHECK-NEXT: fcvtzu v0.4s, v0.4s ; CHECK-NEXT: fcvtzu v0.4s, v0.4s
; CHECK-NEXT: mov w1, v1.s[1] ; CHECK-NEXT: mov w1, v1.s[1]
; CHECK-NEXT: mov w2, v1.s[2] ; CHECK-NEXT: mov w2, v1.s[2]
; CHECK-NEXT: mov w3, v1.s[3]
; CHECK-NEXT: mov w5, v0.s[1] ; CHECK-NEXT: mov w5, v0.s[1]
; CHECK-NEXT: fmov w0, s1 ; CHECK-NEXT: mov w3, v1.s[3]
; CHECK-NEXT: fmov w4, s0 ; CHECK-NEXT: fmov w4, s0
; CHECK-NEXT: fmov w0, s1
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%x = call <6 x i32> @llvm.fptoui.sat.v6f16.v6i32(<6 x half> %f) %x = call <6 x i32> @llvm.fptoui.sat.v6f16.v6i32(<6 x half> %f)
ret <6 x i32> %x ret <6 x i32> %x
Context not available.
; CHECK-NEXT: // kill: def $d0 killed $d0 def $q0 ; CHECK-NEXT: // kill: def $d0 killed $d0 def $q0
; CHECK-NEXT: mov s1, v0.s[1] ; CHECK-NEXT: mov s1, v0.s[1]
; CHECK-NEXT: fcvtzu x8, s0 ; CHECK-NEXT: fcvtzu x8, s0
; CHECK-NEXT: fcvtzu x9, s1
; CHECK-NEXT: fmov d0, x8 ; CHECK-NEXT: fmov d0, x8
; CHECK-NEXT: fcvtzu x8, s1 ; CHECK-NEXT: mov v0.d[1], x9
; CHECK-NEXT: mov v0.d[1], x8
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%x = call <2 x i64> @llvm.fptoui.sat.v2f32.v2i64(<2 x float> %f) %x = call <2 x i64> @llvm.fptoui.sat.v2f32.v2i64(<2 x float> %f)
ret <2 x i64> %x ret <2 x i64> %x
Context not available.
; CHECK-NEXT: mov w8, #1904214015 // =0x717fffff ; CHECK-NEXT: mov w8, #1904214015 // =0x717fffff
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
; CHECK-NEXT: mov x21, #68719476735 // =0xfffffffff ; CHECK-NEXT: mov x21, #68719476735 // =0xfffffffff
; CHECK-NEXT: fmov s9, w8
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
; CHECK-NEXT: // kill: def $s0 killed $s0 killed $q0 ; CHECK-NEXT: // kill: def $s0 killed $s0 killed $q0
; CHECK-NEXT: fmov s9, w8
; CHECK-NEXT: csel x8, xzr, x0, lt ; CHECK-NEXT: csel x8, xzr, x0, lt
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
Context not available.
; CHECK-NEXT: ldp d9, d8, [sp, #16] // 16-byte Folded Reload ; CHECK-NEXT: ldp d9, d8, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: csinv x8, x8, xzr, le ; CHECK-NEXT: csinv x8, x8, xzr, le
; CHECK-NEXT: csel x1, x21, x9, gt ; CHECK-NEXT: csel x1, x21, x9, gt
; CHECK-NEXT: ldp x30, x21, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: fmov d0, x8 ; CHECK-NEXT: fmov d0, x8
; CHECK-NEXT: ldp x30, x21, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: mov v0.d[1], x1 ; CHECK-NEXT: mov v0.d[1], x1
; CHECK-NEXT: fmov x0, d0 ; CHECK-NEXT: fmov x0, d0
; CHECK-NEXT: add sp, sp, #64 ; CHECK-NEXT: add sp, sp, #64
Context not available.
; CHECK-NEXT: mov w8, #2139095039 // =0x7f7fffff ; CHECK-NEXT: mov w8, #2139095039 // =0x7f7fffff
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
; CHECK-NEXT: // kill: def $s0 killed $s0 killed $q0
; CHECK-NEXT: fmov s9, w8 ; CHECK-NEXT: fmov s9, w8
; CHECK-NEXT: // kill: def $s0 killed $s0 killed $q0
; CHECK-NEXT: csel x8, xzr, x1, lt ; CHECK-NEXT: csel x8, xzr, x1, lt
; CHECK-NEXT: csel x9, xzr, x0, lt ; CHECK-NEXT: csel x9, xzr, x0, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
Context not available.
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
; CHECK-NEXT: mov x2, x19 ; CHECK-NEXT: mov x2, x19
; CHECK-NEXT: mov x3, x20 ; CHECK-NEXT: mov x3, x20
; CHECK-NEXT: ldr x30, [sp, #32] // 8-byte Folded Reload
; CHECK-NEXT: ldp x20, x19, [sp, #48] // 16-byte Folded Reload ; CHECK-NEXT: ldp x20, x19, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: ldr x30, [sp, #32] // 8-byte Folded Reload
; CHECK-NEXT: fcmp s0, #0.0 ; CHECK-NEXT: fcmp s0, #0.0
; CHECK-NEXT: csel x8, xzr, x0, lt ; CHECK-NEXT: csel x8, xzr, x0, lt
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
Context not available.
; CHECK-LABEL: test_unsigned_v4f32_v4i50: ; CHECK-LABEL: test_unsigned_v4f32_v4i50:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: ext v1.16b, v0.16b, v0.16b, #8 ; CHECK-NEXT: ext v1.16b, v0.16b, v0.16b, #8
; CHECK-NEXT: mov x8, #1125899906842623 // =0x3ffffffffffff
; CHECK-NEXT: mov s3, v0.s[1] ; CHECK-NEXT: mov s3, v0.s[1]
; CHECK-NEXT: mov x8, #1125899906842623 // =0x3ffffffffffff
; CHECK-NEXT: fcvtzu x11, s0 ; CHECK-NEXT: fcvtzu x11, s0
; CHECK-NEXT: mov s2, v1.s[1] ; CHECK-NEXT: mov s2, v1.s[1]
; CHECK-NEXT: fcvtzu x9, s1 ; CHECK-NEXT: fcvtzu x9, s1
; CHECK-NEXT: fcvtzu x12, s3 ; CHECK-NEXT: fcvtzu x12, s3
; CHECK-NEXT: cmp x9, x8
; CHECK-NEXT: fcvtzu x10, s2 ; CHECK-NEXT: fcvtzu x10, s2
; CHECK-NEXT: cmp x9, x8
; CHECK-NEXT: csel x2, x9, x8, lo ; CHECK-NEXT: csel x2, x9, x8, lo
; CHECK-NEXT: cmp x10, x8 ; CHECK-NEXT: cmp x10, x8
; CHECK-NEXT: csel x3, x10, x8, lo ; CHECK-NEXT: csel x3, x10, x8, lo
Context not available.
; CHECK-NEXT: fcvtzu x9, s0 ; CHECK-NEXT: fcvtzu x9, s0
; CHECK-NEXT: mov s2, v1.s[1] ; CHECK-NEXT: mov s2, v1.s[1]
; CHECK-NEXT: fcvtzu x8, s1 ; CHECK-NEXT: fcvtzu x8, s1
; CHECK-NEXT: fcvtzu x11, s3
; CHECK-NEXT: fmov d0, x9 ; CHECK-NEXT: fmov d0, x9
; CHECK-NEXT: fcvtzu x9, s3 ; CHECK-NEXT: fcvtzu x10, s2
; CHECK-NEXT: fmov d1, x8 ; CHECK-NEXT: fmov d1, x8
; CHECK-NEXT: fcvtzu x8, s2 ; CHECK-NEXT: mov v0.d[1], x11
; CHECK-NEXT: mov v0.d[1], x9 ; CHECK-NEXT: mov v1.d[1], x10
; CHECK-NEXT: mov v1.d[1], x8
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%x = call <4 x i64> @llvm.fptoui.sat.v4f32.v4i64(<4 x float> %f) %x = call <4 x i64> @llvm.fptoui.sat.v4f32.v4i64(<4 x float> %f)
ret <4 x i64> %x ret <4 x i64> %x
Context not available.
; CHECK-NEXT: fmov s0, s8 ; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
; CHECK-NEXT: mov w8, #1904214015 // =0x717fffff ; CHECK-NEXT: mov w8, #1904214015 // =0x717fffff
; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
; CHECK-NEXT: mov x25, #68719476735 // =0xfffffffff ; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: fmov s9, w8 ; CHECK-NEXT: fmov s9, w8
; CHECK-NEXT: mov x25, #68719476735 // =0xfffffffff
; CHECK-NEXT: ext v0.16b, v0.16b, v0.16b, #8 ; CHECK-NEXT: ext v0.16b, v0.16b, v0.16b, #8
; CHECK-NEXT: csel x8, xzr, x0, lt ; CHECK-NEXT: csel x8, xzr, x0, lt
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
Context not available.
; CHECK-NEXT: fcmp s0, #0.0 ; CHECK-NEXT: fcmp s0, #0.0
; CHECK-NEXT: mov x7, x23 ; CHECK-NEXT: mov x7, x23
; CHECK-NEXT: ldp x20, x19, [sp, #96] // 16-byte Folded Reload ; CHECK-NEXT: ldp x20, x19, [sp, #96] // 16-byte Folded Reload
; CHECK-NEXT: ldp x22, x21, [sp, #80] // 16-byte Folded Reload
; CHECK-NEXT: csel x8, xzr, x0, lt ; CHECK-NEXT: csel x8, xzr, x0, lt
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: fcmp s0, s9 ; CHECK-NEXT: fcmp s0, s9
; CHECK-NEXT: ldp x22, x21, [sp, #80] // 16-byte Folded Reload ; CHECK-NEXT: ldp x24, x23, [sp, #64] // 16-byte Folded Reload
; CHECK-NEXT: ldp d9, d8, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: csinv x8, x8, xzr, le ; CHECK-NEXT: csinv x8, x8, xzr, le
; CHECK-NEXT: csel x1, x25, x9, gt ; CHECK-NEXT: csel x1, x25, x9, gt
; CHECK-NEXT: ldp x24, x23, [sp, #64] // 16-byte Folded Reload
; CHECK-NEXT: fmov d0, x8 ; CHECK-NEXT: fmov d0, x8
; CHECK-NEXT: ldp x30, x25, [sp, #48] // 16-byte Folded Reload ; CHECK-NEXT: ldp x30, x25, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: mov v0.d[1], x1 ; CHECK-NEXT: mov v0.d[1], x1
; CHECK-NEXT: ldp d9, d8, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: fmov x0, d0 ; CHECK-NEXT: fmov x0, d0
; CHECK-NEXT: add sp, sp, #112 ; CHECK-NEXT: add sp, sp, #112
; CHECK-NEXT: ret ; CHECK-NEXT: ret
Context not available.
; CHECK-NEXT: fmov s0, s8 ; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
; CHECK-NEXT: mov w8, #2139095039 // =0x7f7fffff ; CHECK-NEXT: mov w8, #2139095039 // =0x7f7fffff
; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
; CHECK-NEXT: ldr q0, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: fmov s9, w8 ; CHECK-NEXT: fmov s9, w8
; CHECK-NEXT: ext v0.16b, v0.16b, v0.16b, #8 ; CHECK-NEXT: ext v0.16b, v0.16b, v0.16b, #8
; CHECK-NEXT: csel x8, xzr, x1, lt ; CHECK-NEXT: csel x8, xzr, x1, lt
Context not available.
; CHECK-NEXT: mov x6, x23 ; CHECK-NEXT: mov x6, x23
; CHECK-NEXT: fcmp s0, #0.0 ; CHECK-NEXT: fcmp s0, #0.0
; CHECK-NEXT: mov x7, x24 ; CHECK-NEXT: mov x7, x24
; CHECK-NEXT: ldr x30, [sp, #48] // 8-byte Folded Reload
; CHECK-NEXT: ldp x20, x19, [sp, #96] // 16-byte Folded Reload ; CHECK-NEXT: ldp x20, x19, [sp, #96] // 16-byte Folded Reload
; CHECK-NEXT: ldp x22, x21, [sp, #80] // 16-byte Folded Reload
; CHECK-NEXT: csel x8, xzr, x0, lt ; CHECK-NEXT: csel x8, xzr, x0, lt
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: fcmp s0, s9 ; CHECK-NEXT: fcmp s0, s9
; CHECK-NEXT: ldr x30, [sp, #48] // 8-byte Folded Reload ; CHECK-NEXT: ldp x24, x23, [sp, #64] // 16-byte Folded Reload
; CHECK-NEXT: ldp x22, x21, [sp, #80] // 16-byte Folded Reload ; CHECK-NEXT: ldp d9, d8, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: csinv x8, x8, xzr, le ; CHECK-NEXT: csinv x8, x8, xzr, le
; CHECK-NEXT: csinv x1, x9, xzr, le ; CHECK-NEXT: csinv x1, x9, xzr, le
; CHECK-NEXT: ldp x24, x23, [sp, #64] // 16-byte Folded Reload
; CHECK-NEXT: fmov d0, x8 ; CHECK-NEXT: fmov d0, x8
; CHECK-NEXT: ldp d9, d8, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: mov v0.d[1], x1 ; CHECK-NEXT: mov v0.d[1], x1
; CHECK-NEXT: fmov x0, d0 ; CHECK-NEXT: fmov x0, d0
; CHECK-NEXT: add sp, sp, #112 ; CHECK-NEXT: add sp, sp, #112
Context not available.
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: mov d1, v0.d[1] ; CHECK-NEXT: mov d1, v0.d[1]
; CHECK-NEXT: fcvtzu w8, d0 ; CHECK-NEXT: fcvtzu w8, d0
; CHECK-NEXT: fcvtzu w9, d1
; CHECK-NEXT: fmov s0, w8 ; CHECK-NEXT: fmov s0, w8
; CHECK-NEXT: fcvtzu w8, d1 ; CHECK-NEXT: mov v0.s[1], w9
; CHECK-NEXT: mov v0.s[1], w8
; CHECK-NEXT: // kill: def $d0 killed $d0 killed $q0 ; CHECK-NEXT: // kill: def $d0 killed $d0 killed $q0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%x = call <2 x i32> @llvm.fptoui.sat.v2f64.v2i32(<2 x double> %f) %x = call <2 x i32> @llvm.fptoui.sat.v2f64.v2i32(<2 x double> %f)
Context not available.
; CHECK-NEXT: mov x8, #5057542381537067007 // =0x462fffffffffffff ; CHECK-NEXT: mov x8, #5057542381537067007 // =0x462fffffffffffff
; CHECK-NEXT: fcmp d8, #0.0 ; CHECK-NEXT: fcmp d8, #0.0
; CHECK-NEXT: mov x21, #68719476735 // =0xfffffffff ; CHECK-NEXT: mov x21, #68719476735 // =0xfffffffff
; CHECK-NEXT: fmov d9, x8
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
; CHECK-NEXT: // kill: def $d0 killed $d0 killed $q0 ; CHECK-NEXT: // kill: def $d0 killed $d0 killed $q0
; CHECK-NEXT: fmov d9, x8
; CHECK-NEXT: csel x8, xzr, x0, lt ; CHECK-NEXT: csel x8, xzr, x0, lt
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: fcmp d8, d9 ; CHECK-NEXT: fcmp d8, d9
Context not available.
; CHECK-NEXT: ldp d9, d8, [sp, #16] // 16-byte Folded Reload ; CHECK-NEXT: ldp d9, d8, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: csinv x8, x8, xzr, le ; CHECK-NEXT: csinv x8, x8, xzr, le
; CHECK-NEXT: csel x1, x21, x9, gt ; CHECK-NEXT: csel x1, x21, x9, gt
; CHECK-NEXT: ldp x30, x21, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: fmov d0, x8 ; CHECK-NEXT: fmov d0, x8
; CHECK-NEXT: ldp x30, x21, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: mov v0.d[1], x1 ; CHECK-NEXT: mov v0.d[1], x1
; CHECK-NEXT: fmov x0, d0 ; CHECK-NEXT: fmov x0, d0
; CHECK-NEXT: add sp, sp, #64 ; CHECK-NEXT: add sp, sp, #64
Context not available.
; CHECK-NEXT: mov x8, #5183643171103440895 // =0x47efffffffffffff ; CHECK-NEXT: mov x8, #5183643171103440895 // =0x47efffffffffffff
; CHECK-NEXT: fcmp d8, #0.0 ; CHECK-NEXT: fcmp d8, #0.0
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
; CHECK-NEXT: // kill: def $d0 killed $d0 killed $q0
; CHECK-NEXT: fmov d9, x8 ; CHECK-NEXT: fmov d9, x8
; CHECK-NEXT: // kill: def $d0 killed $d0 killed $q0
; CHECK-NEXT: csel x8, xzr, x1, lt ; CHECK-NEXT: csel x8, xzr, x1, lt
; CHECK-NEXT: csel x9, xzr, x0, lt ; CHECK-NEXT: csel x9, xzr, x0, lt
; CHECK-NEXT: fcmp d8, d9 ; CHECK-NEXT: fcmp d8, d9
Context not available.
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
; CHECK-NEXT: mov x2, x19 ; CHECK-NEXT: mov x2, x19
; CHECK-NEXT: mov x3, x20 ; CHECK-NEXT: mov x3, x20
; CHECK-NEXT: ldr x30, [sp, #32] // 8-byte Folded Reload
; CHECK-NEXT: ldp x20, x19, [sp, #48] // 16-byte Folded Reload ; CHECK-NEXT: ldp x20, x19, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: ldr x30, [sp, #32] // 8-byte Folded Reload
; CHECK-NEXT: fcmp d0, #0.0 ; CHECK-NEXT: fcmp d0, #0.0
; CHECK-NEXT: csel x8, xzr, x0, lt ; CHECK-NEXT: csel x8, xzr, x0, lt
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
Context not available.
; CHECK-CVT-NEXT: // kill: def $d0 killed $d0 def $q0 ; CHECK-CVT-NEXT: // kill: def $d0 killed $d0 def $q0
; CHECK-CVT-NEXT: mov h1, v0.h[1] ; CHECK-CVT-NEXT: mov h1, v0.h[1]
; CHECK-CVT-NEXT: mov h2, v0.h[2] ; CHECK-CVT-NEXT: mov h2, v0.h[2]
; CHECK-CVT-NEXT: mov x8, #1125899906842623 // =0x3ffffffffffff
; CHECK-CVT-NEXT: mov h3, v0.h[3] ; CHECK-CVT-NEXT: mov h3, v0.h[3]
; CHECK-CVT-NEXT: fcvt s0, h0 ; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: mov x8, #1125899906842623 // =0x3ffffffffffff
; CHECK-CVT-NEXT: fcvt s1, h1 ; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s2, h2 ; CHECK-CVT-NEXT: fcvt s2, h2
; CHECK-CVT-NEXT: fcvt s3, h3 ; CHECK-CVT-NEXT: fcvt s3, h3
; CHECK-CVT-NEXT: fcvtzu x9, s0 ; CHECK-CVT-NEXT: fcvtzu x9, s0
; CHECK-CVT-NEXT: fcvtzu x10, s1 ; CHECK-CVT-NEXT: fcvtzu x10, s1
; CHECK-CVT-NEXT: fcvtzu x11, s2 ; CHECK-CVT-NEXT: fcvtzu x11, s2
; CHECK-CVT-NEXT: cmp x9, x8
; CHECK-CVT-NEXT: fcvtzu x12, s3 ; CHECK-CVT-NEXT: fcvtzu x12, s3
; CHECK-CVT-NEXT: cmp x9, x8
; CHECK-CVT-NEXT: csel x0, x9, x8, lo ; CHECK-CVT-NEXT: csel x0, x9, x8, lo
; CHECK-CVT-NEXT: cmp x10, x8 ; CHECK-CVT-NEXT: cmp x10, x8
; CHECK-CVT-NEXT: csel x1, x10, x8, lo ; CHECK-CVT-NEXT: csel x1, x10, x8, lo
Context not available.
; CHECK-FP16-NEXT: // kill: def $d0 killed $d0 def $q0 ; CHECK-FP16-NEXT: // kill: def $d0 killed $d0 def $q0
; CHECK-FP16-NEXT: mov h1, v0.h[1] ; CHECK-FP16-NEXT: mov h1, v0.h[1]
; CHECK-FP16-NEXT: mov h2, v0.h[2] ; CHECK-FP16-NEXT: mov h2, v0.h[2]
; CHECK-FP16-NEXT: mov x8, #1125899906842623 // =0x3ffffffffffff
; CHECK-FP16-NEXT: mov h3, v0.h[3] ; CHECK-FP16-NEXT: mov h3, v0.h[3]
; CHECK-FP16-NEXT: fcvtzu x9, h0 ; CHECK-FP16-NEXT: fcvtzu x9, h0
; CHECK-FP16-NEXT: mov x8, #1125899906842623 // =0x3ffffffffffff
; CHECK-FP16-NEXT: fcvtzu x10, h1 ; CHECK-FP16-NEXT: fcvtzu x10, h1
; CHECK-FP16-NEXT: fcvtzu x11, h2 ; CHECK-FP16-NEXT: fcvtzu x11, h2
; CHECK-FP16-NEXT: cmp x9, x8
; CHECK-FP16-NEXT: fcvtzu x12, h3 ; CHECK-FP16-NEXT: fcvtzu x12, h3
; CHECK-FP16-NEXT: cmp x9, x8
; CHECK-FP16-NEXT: csel x0, x9, x8, lo ; CHECK-FP16-NEXT: csel x0, x9, x8, lo
; CHECK-FP16-NEXT: cmp x10, x8 ; CHECK-FP16-NEXT: cmp x10, x8
; CHECK-FP16-NEXT: csel x1, x10, x8, lo ; CHECK-FP16-NEXT: csel x1, x10, x8, lo
Context not available.
; CHECK-CVT-NEXT: // kill: def $d0 killed $d0 def $q0 ; CHECK-CVT-NEXT: // kill: def $d0 killed $d0 def $q0
; CHECK-CVT-NEXT: mov h1, v0.h[2] ; CHECK-CVT-NEXT: mov h1, v0.h[2]
; CHECK-CVT-NEXT: mov h2, v0.h[1] ; CHECK-CVT-NEXT: mov h2, v0.h[1]
; CHECK-CVT-NEXT: fcvt s3, h0 ; CHECK-CVT-NEXT: mov h3, v0.h[3]
; CHECK-CVT-NEXT: mov h0, v0.h[3] ; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: fcvt s1, h1 ; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s2, h2 ; CHECK-CVT-NEXT: fcvt s2, h2
; CHECK-CVT-NEXT: fcvtzu x8, s3 ; CHECK-CVT-NEXT: fcvt s3, h3
; CHECK-CVT-NEXT: fcvt s3, h0 ; CHECK-CVT-NEXT: fcvtzu x8, s0
; CHECK-CVT-NEXT: fcvtzu x9, s1 ; CHECK-CVT-NEXT: fcvtzu x9, s1
; CHECK-CVT-NEXT: fcvtzu x10, s2
; CHECK-CVT-NEXT: fcvtzu x11, s3
; CHECK-CVT-NEXT: fmov d0, x8 ; CHECK-CVT-NEXT: fmov d0, x8
; CHECK-CVT-NEXT: fcvtzu x8, s2
; CHECK-CVT-NEXT: fmov d1, x9 ; CHECK-CVT-NEXT: fmov d1, x9
; CHECK-CVT-NEXT: fcvtzu x9, s3 ; CHECK-CVT-NEXT: mov v0.d[1], x10
; CHECK-CVT-NEXT: mov v0.d[1], x8 ; CHECK-CVT-NEXT: mov v1.d[1], x11
; CHECK-CVT-NEXT: mov v1.d[1], x9
; CHECK-CVT-NEXT: ret ; CHECK-CVT-NEXT: ret
; ;
; CHECK-FP16-LABEL: test_unsigned_v4f16_v4i64: ; CHECK-FP16-LABEL: test_unsigned_v4f16_v4i64:
Context not available.
; CHECK-FP16-NEXT: mov h3, v0.h[3] ; CHECK-FP16-NEXT: mov h3, v0.h[3]
; CHECK-FP16-NEXT: fcvtzu x8, h0 ; CHECK-FP16-NEXT: fcvtzu x8, h0
; CHECK-FP16-NEXT: fcvtzu x9, h1 ; CHECK-FP16-NEXT: fcvtzu x9, h1
; CHECK-FP16-NEXT: fcvtzu x10, h2
; CHECK-FP16-NEXT: fcvtzu x11, h3
; CHECK-FP16-NEXT: fmov d0, x8 ; CHECK-FP16-NEXT: fmov d0, x8
; CHECK-FP16-NEXT: fcvtzu x8, h2
; CHECK-FP16-NEXT: fmov d1, x9 ; CHECK-FP16-NEXT: fmov d1, x9
; CHECK-FP16-NEXT: fcvtzu x9, h3 ; CHECK-FP16-NEXT: mov v0.d[1], x10
; CHECK-FP16-NEXT: mov v0.d[1], x8 ; CHECK-FP16-NEXT: mov v1.d[1], x11
; CHECK-FP16-NEXT: mov v1.d[1], x9
; CHECK-FP16-NEXT: ret ; CHECK-FP16-NEXT: ret
%x = call <4 x i64> @llvm.fptoui.sat.v4f16.v4i64(<4 x half> %f) %x = call <4 x i64> @llvm.fptoui.sat.v4f16.v4i64(<4 x half> %f)
ret <4 x i64> %x ret <4 x i64> %x
Context not available.
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
; CHECK-NEXT: mov w8, #1904214015 // =0x717fffff ; CHECK-NEXT: mov w8, #1904214015 // =0x717fffff
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
; CHECK-NEXT: fmov s9, w8
; CHECK-NEXT: mov x25, #68719476735 // =0xfffffffff ; CHECK-NEXT: mov x25, #68719476735 // =0xfffffffff
; CHECK-NEXT: mov h0, v0.h[1] ; CHECK-NEXT: mov h0, v0.h[1]
; CHECK-NEXT: fmov s9, w8 ; CHECK-NEXT: csel x9, xzr, x0, lt
; CHECK-NEXT: csel x8, xzr, x0, lt ; CHECK-NEXT: csel x8, xzr, x1, lt
; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
; CHECK-NEXT: csel x19, x25, x9, gt ; CHECK-NEXT: csel x19, x25, x8, gt
; CHECK-NEXT: csinv x20, x8, xzr, le ; CHECK-NEXT: csinv x20, x9, xzr, le
; CHECK-NEXT: fmov s0, s8 ; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
Context not available.
; CHECK-NEXT: mov x4, x20 ; CHECK-NEXT: mov x4, x20
; CHECK-NEXT: mov x5, x19 ; CHECK-NEXT: mov x5, x19
; CHECK-NEXT: mov x6, x24 ; CHECK-NEXT: mov x6, x24
; CHECK-NEXT: mov x7, x23
; CHECK-NEXT: ldp x20, x19, [sp, #80] // 16-byte Folded Reload
; CHECK-NEXT: csel x8, xzr, x0, lt ; CHECK-NEXT: csel x8, xzr, x0, lt
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: mov x7, x23 ; CHECK-NEXT: ldp x22, x21, [sp, #64] // 16-byte Folded Reload
; CHECK-NEXT: ldp x20, x19, [sp, #80] // 16-byte Folded Reload ; CHECK-NEXT: ldp x24, x23, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: csinv x8, x8, xzr, le ; CHECK-NEXT: csinv x8, x8, xzr, le
; CHECK-NEXT: csel x1, x25, x9, gt ; CHECK-NEXT: csel x1, x25, x9, gt
; CHECK-NEXT: ldp x22, x21, [sp, #64] // 16-byte Folded Reload
; CHECK-NEXT: fmov d0, x8 ; CHECK-NEXT: fmov d0, x8
; CHECK-NEXT: ldp x24, x23, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: mov v0.d[1], x1
; CHECK-NEXT: ldp x30, x25, [sp, #32] // 16-byte Folded Reload ; CHECK-NEXT: ldp x30, x25, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: ldp d9, d8, [sp, #16] // 16-byte Folded Reload ; CHECK-NEXT: ldp d9, d8, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: mov v0.d[1], x1
; CHECK-NEXT: fmov x0, d0 ; CHECK-NEXT: fmov x0, d0
; CHECK-NEXT: add sp, sp, #96 ; CHECK-NEXT: add sp, sp, #96
; CHECK-NEXT: ret ; CHECK-NEXT: ret
Context not available.
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
; CHECK-NEXT: mov w8, #2139095039 // =0x7f7fffff ; CHECK-NEXT: mov w8, #2139095039 // =0x7f7fffff
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
; CHECK-NEXT: mov h0, v0.h[2]
; CHECK-NEXT: fmov s9, w8 ; CHECK-NEXT: fmov s9, w8
; CHECK-NEXT: csel x8, xzr, x1, lt ; CHECK-NEXT: mov h0, v0.h[2]
; CHECK-NEXT: csel x9, xzr, x0, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: csel x8, xzr, x0, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
; CHECK-NEXT: csinv x19, x9, xzr, le ; CHECK-NEXT: csinv x19, x8, xzr, le
; CHECK-NEXT: csinv x20, x8, xzr, le ; CHECK-NEXT: csinv x20, x9, xzr, le
; CHECK-NEXT: fmov s0, s8 ; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp] // 16-byte Folded Reload
Context not available.
; CHECK-NEXT: mov x4, x21 ; CHECK-NEXT: mov x4, x21
; CHECK-NEXT: mov x5, x22 ; CHECK-NEXT: mov x5, x22
; CHECK-NEXT: mov x6, x23 ; CHECK-NEXT: mov x6, x23
; CHECK-NEXT: mov x7, x24
; CHECK-NEXT: ldp x20, x19, [sp, #80] // 16-byte Folded Reload
; CHECK-NEXT: csel x8, xzr, x0, lt ; CHECK-NEXT: csel x8, xzr, x0, lt
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: mov x7, x24
; CHECK-NEXT: ldp x20, x19, [sp, #80] // 16-byte Folded Reload
; CHECK-NEXT: csinv x8, x8, xzr, le
; CHECK-NEXT: csinv x1, x9, xzr, le
; CHECK-NEXT: ldp x22, x21, [sp, #64] // 16-byte Folded Reload ; CHECK-NEXT: ldp x22, x21, [sp, #64] // 16-byte Folded Reload
; CHECK-NEXT: fmov d0, x8
; CHECK-NEXT: ldr x30, [sp, #32] // 8-byte Folded Reload ; CHECK-NEXT: ldr x30, [sp, #32] // 8-byte Folded Reload
; CHECK-NEXT: ldp x24, x23, [sp, #48] // 16-byte Folded Reload ; CHECK-NEXT: ldp x24, x23, [sp, #48] // 16-byte Folded Reload
; CHECK-NEXT: mov v0.d[1], x1 ; CHECK-NEXT: csinv x8, x8, xzr, le
; CHECK-NEXT: csinv x1, x9, xzr, le
; CHECK-NEXT: fmov d0, x8
; CHECK-NEXT: ldp d9, d8, [sp, #16] // 16-byte Folded Reload ; CHECK-NEXT: ldp d9, d8, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: mov v0.d[1], x1
; CHECK-NEXT: fmov x0, d0 ; CHECK-NEXT: fmov x0, d0
; CHECK-NEXT: add sp, sp, #96 ; CHECK-NEXT: add sp, sp, #96
; CHECK-NEXT: ret ; CHECK-NEXT: ret
Context not available.
; CHECK-CVT-NEXT: mov s2, v1.s[1] ; CHECK-CVT-NEXT: mov s2, v1.s[1]
; CHECK-CVT-NEXT: mov s3, v1.s[2] ; CHECK-CVT-NEXT: mov s3, v1.s[2]
; CHECK-CVT-NEXT: mov s4, v1.s[3] ; CHECK-CVT-NEXT: mov s4, v1.s[3]
; CHECK-CVT-NEXT: mov s5, v0.s[1]
; CHECK-CVT-NEXT: fcvtzu w9, s1 ; CHECK-CVT-NEXT: fcvtzu w9, s1
; CHECK-CVT-NEXT: fcvtzu w10, s0 ; CHECK-CVT-NEXT: fcvtzu w13, s0
; CHECK-CVT-NEXT: mov s1, v0.s[2] ; CHECK-CVT-NEXT: mov s1, v0.s[2]
; CHECK-CVT-NEXT: mov s0, v0.s[3]
; CHECK-CVT-NEXT: fcvtzu w8, s2 ; CHECK-CVT-NEXT: fcvtzu w8, s2
; CHECK-CVT-NEXT: fcvtzu w11, s3 ; CHECK-CVT-NEXT: mov s2, v0.s[1]
; CHECK-CVT-NEXT: fcvtzu w12, s4 ; CHECK-CVT-NEXT: fcvtzu w10, s3
; CHECK-CVT-NEXT: fcvtzu w13, s5 ; CHECK-CVT-NEXT: fcvtzu w11, s4
; CHECK-CVT-NEXT: fcvtzu w14, s1
; CHECK-CVT-NEXT: mov s0, v0.s[3]
; CHECK-CVT-NEXT: cmp w8, #1 ; CHECK-CVT-NEXT: cmp w8, #1
; CHECK-CVT-NEXT: fcvtzu w12, s2
; CHECK-CVT-NEXT: csinc w8, w8, wzr, lo ; CHECK-CVT-NEXT: csinc w8, w8, wzr, lo
; CHECK-CVT-NEXT: cmp w9, #1 ; CHECK-CVT-NEXT: cmp w9, #1
; CHECK-CVT-NEXT: csinc w9, w9, wzr, lo ; CHECK-CVT-NEXT: csinc w9, w9, wzr, lo
; CHECK-CVT-NEXT: cmp w10, #1
; CHECK-CVT-NEXT: csinc w10, w10, wzr, lo
; CHECK-CVT-NEXT: cmp w11, #1 ; CHECK-CVT-NEXT: cmp w11, #1
; CHECK-CVT-NEXT: fmov s1, w9
; CHECK-CVT-NEXT: csinc w11, w11, wzr, lo ; CHECK-CVT-NEXT: csinc w11, w11, wzr, lo
; CHECK-CVT-NEXT: cmp w12, #1 ; CHECK-CVT-NEXT: cmp w12, #1
; CHECK-CVT-NEXT: csinc w12, w12, wzr, lo ; CHECK-CVT-NEXT: csinc w12, w12, wzr, lo
; CHECK-CVT-NEXT: cmp w13, #1 ; CHECK-CVT-NEXT: cmp w13, #1
; CHECK-CVT-NEXT: csinc w13, w13, wzr, lo ; CHECK-CVT-NEXT: csinc w13, w13, wzr, lo
; CHECK-CVT-NEXT: cmp w10, #1 ; CHECK-CVT-NEXT: mov v1.s[1], w8
; CHECK-CVT-NEXT: csinc w10, w10, wzr, lo ; CHECK-CVT-NEXT: cmp w14, #1
; CHECK-CVT-NEXT: fmov s2, w9 ; CHECK-CVT-NEXT: fmov s2, w13
; CHECK-CVT-NEXT: fcvtzu w9, s1 ; CHECK-CVT-NEXT: fcvtzu w8, s0
; CHECK-CVT-NEXT: fmov s3, w10 ; CHECK-CVT-NEXT: csinc w9, w14, wzr, lo
; CHECK-CVT-NEXT: mov v2.s[1], w8 ; CHECK-CVT-NEXT: mov v2.s[1], w12
; CHECK-CVT-NEXT: cmp w9, #1 ; CHECK-CVT-NEXT: mov v1.s[2], w10
; CHECK-CVT-NEXT: csinc w8, w9, wzr, lo ; CHECK-CVT-NEXT: cmp w8, #1
; CHECK-CVT-NEXT: fcvtzu w9, s0 ; CHECK-CVT-NEXT: csinc w8, w8, wzr, lo
; CHECK-CVT-NEXT: mov v3.s[1], w13 ; CHECK-CVT-NEXT: mov v2.s[2], w9
; CHECK-CVT-NEXT: mov v2.s[2], w11 ; CHECK-CVT-NEXT: mov v1.s[3], w11
; CHECK-CVT-NEXT: cmp w9, #1 ; CHECK-CVT-NEXT: mov v2.s[3], w8
; CHECK-CVT-NEXT: mov v3.s[2], w8 ; CHECK-CVT-NEXT: uzp1 v0.8h, v2.8h, v1.8h
; CHECK-CVT-NEXT: csinc w8, w9, wzr, lo
; CHECK-CVT-NEXT: mov v2.s[3], w12
; CHECK-CVT-NEXT: mov v3.s[3], w8
; CHECK-CVT-NEXT: uzp1 v0.8h, v3.8h, v2.8h
; CHECK-CVT-NEXT: xtn v0.8b, v0.8h ; CHECK-CVT-NEXT: xtn v0.8b, v0.8h
; CHECK-CVT-NEXT: ret ; CHECK-CVT-NEXT: ret
; ;
Context not available.
; CHECK-CVT-NEXT: mov s2, v1.s[1] ; CHECK-CVT-NEXT: mov s2, v1.s[1]
; CHECK-CVT-NEXT: mov s3, v1.s[2] ; CHECK-CVT-NEXT: mov s3, v1.s[2]
; CHECK-CVT-NEXT: mov s4, v1.s[3] ; CHECK-CVT-NEXT: mov s4, v1.s[3]
; CHECK-CVT-NEXT: mov s5, v0.s[1]
; CHECK-CVT-NEXT: fcvtzu w10, s1 ; CHECK-CVT-NEXT: fcvtzu w10, s1
; CHECK-CVT-NEXT: fcvtzu w11, s0 ; CHECK-CVT-NEXT: fcvtzu w14, s0
; CHECK-CVT-NEXT: mov s1, v0.s[2] ; CHECK-CVT-NEXT: mov s1, v0.s[2]
; CHECK-CVT-NEXT: mov s0, v0.s[3]
; CHECK-CVT-NEXT: fcvtzu w9, s2 ; CHECK-CVT-NEXT: fcvtzu w9, s2
; CHECK-CVT-NEXT: fcvtzu w12, s3 ; CHECK-CVT-NEXT: mov s2, v0.s[1]
; CHECK-CVT-NEXT: fcvtzu w13, s4 ; CHECK-CVT-NEXT: fcvtzu w11, s3
; CHECK-CVT-NEXT: fcvtzu w14, s5 ; CHECK-CVT-NEXT: fcvtzu w12, s4
; CHECK-CVT-NEXT: fcvtzu w15, s1
; CHECK-CVT-NEXT: mov s0, v0.s[3]
; CHECK-CVT-NEXT: cmp w9, #255 ; CHECK-CVT-NEXT: cmp w9, #255
; CHECK-CVT-NEXT: fcvtzu w13, s2
; CHECK-CVT-NEXT: csel w9, w9, w8, lo ; CHECK-CVT-NEXT: csel w9, w9, w8, lo
; CHECK-CVT-NEXT: cmp w10, #255 ; CHECK-CVT-NEXT: cmp w10, #255
; CHECK-CVT-NEXT: csel w10, w10, w8, lo ; CHECK-CVT-NEXT: csel w10, w10, w8, lo
; CHECK-CVT-NEXT: cmp w11, #255
; CHECK-CVT-NEXT: csel w11, w11, w8, lo
; CHECK-CVT-NEXT: cmp w12, #255 ; CHECK-CVT-NEXT: cmp w12, #255
; CHECK-CVT-NEXT: fmov s1, w10
; CHECK-CVT-NEXT: csel w12, w12, w8, lo ; CHECK-CVT-NEXT: csel w12, w12, w8, lo
; CHECK-CVT-NEXT: cmp w13, #255 ; CHECK-CVT-NEXT: cmp w13, #255
; CHECK-CVT-NEXT: csel w13, w13, w8, lo ; CHECK-CVT-NEXT: csel w13, w13, w8, lo
; CHECK-CVT-NEXT: cmp w14, #255 ; CHECK-CVT-NEXT: cmp w14, #255
; CHECK-CVT-NEXT: csel w14, w14, w8, lo ; CHECK-CVT-NEXT: csel w14, w14, w8, lo
; CHECK-CVT-NEXT: cmp w11, #255 ; CHECK-CVT-NEXT: mov v1.s[1], w9
; CHECK-CVT-NEXT: csel w11, w11, w8, lo ; CHECK-CVT-NEXT: cmp w15, #255
; CHECK-CVT-NEXT: fmov s2, w10 ; CHECK-CVT-NEXT: fmov s2, w14
; CHECK-CVT-NEXT: fcvtzu w10, s1 ; CHECK-CVT-NEXT: fcvtzu w9, s0
; CHECK-CVT-NEXT: fmov s3, w11 ; CHECK-CVT-NEXT: csel w10, w15, w8, lo
; CHECK-CVT-NEXT: mov v2.s[1], w9 ; CHECK-CVT-NEXT: mov v2.s[1], w13
; CHECK-CVT-NEXT: cmp w10, #255 ; CHECK-CVT-NEXT: mov v1.s[2], w11
; CHECK-CVT-NEXT: csel w9, w10, w8, lo ; CHECK-CVT-NEXT: cmp w9, #255
; CHECK-CVT-NEXT: fcvtzu w10, s0 ; CHECK-CVT-NEXT: csel w8, w9, w8, lo
; CHECK-CVT-NEXT: mov v3.s[1], w14 ; CHECK-CVT-NEXT: mov v2.s[2], w10
; CHECK-CVT-NEXT: mov v2.s[2], w12 ; CHECK-CVT-NEXT: mov v1.s[3], w12
; CHECK-CVT-NEXT: cmp w10, #255 ; CHECK-CVT-NEXT: mov v2.s[3], w8
; CHECK-CVT-NEXT: csel w8, w10, w8, lo ; CHECK-CVT-NEXT: uzp1 v0.8h, v2.8h, v1.8h
; CHECK-CVT-NEXT: mov v3.s[2], w9
; CHECK-CVT-NEXT: mov v2.s[3], w13
; CHECK-CVT-NEXT: mov v3.s[3], w8
; CHECK-CVT-NEXT: uzp1 v0.8h, v3.8h, v2.8h
; CHECK-CVT-NEXT: xtn v0.8b, v0.8h ; CHECK-CVT-NEXT: xtn v0.8b, v0.8h
; CHECK-CVT-NEXT: ret ; CHECK-CVT-NEXT: ret
; ;
Context not available.
; CHECK-CVT-NEXT: mov s2, v1.s[1] ; CHECK-CVT-NEXT: mov s2, v1.s[1]
; CHECK-CVT-NEXT: mov s3, v1.s[2] ; CHECK-CVT-NEXT: mov s3, v1.s[2]
; CHECK-CVT-NEXT: mov s4, v1.s[3] ; CHECK-CVT-NEXT: mov s4, v1.s[3]
; CHECK-CVT-NEXT: mov s5, v0.s[1]
; CHECK-CVT-NEXT: fcvtzu w10, s1 ; CHECK-CVT-NEXT: fcvtzu w10, s1
; CHECK-CVT-NEXT: fcvtzu w11, s0 ; CHECK-CVT-NEXT: fcvtzu w14, s0
; CHECK-CVT-NEXT: mov s1, v0.s[2] ; CHECK-CVT-NEXT: mov s1, v0.s[2]
; CHECK-CVT-NEXT: mov s0, v0.s[3]
; CHECK-CVT-NEXT: fcvtzu w9, s2 ; CHECK-CVT-NEXT: fcvtzu w9, s2
; CHECK-CVT-NEXT: fcvtzu w12, s3 ; CHECK-CVT-NEXT: mov s2, v0.s[1]
; CHECK-CVT-NEXT: fcvtzu w13, s4 ; CHECK-CVT-NEXT: fcvtzu w11, s3
; CHECK-CVT-NEXT: fcvtzu w14, s5 ; CHECK-CVT-NEXT: fcvtzu w12, s4
; CHECK-CVT-NEXT: fcvtzu w15, s1
; CHECK-CVT-NEXT: mov s0, v0.s[3]
; CHECK-CVT-NEXT: cmp w9, w8 ; CHECK-CVT-NEXT: cmp w9, w8
; CHECK-CVT-NEXT: fcvtzu w13, s2
; CHECK-CVT-NEXT: csel w9, w9, w8, lo ; CHECK-CVT-NEXT: csel w9, w9, w8, lo
; CHECK-CVT-NEXT: cmp w10, w8 ; CHECK-CVT-NEXT: cmp w10, w8
; CHECK-CVT-NEXT: csel w10, w10, w8, lo ; CHECK-CVT-NEXT: csel w10, w10, w8, lo
; CHECK-CVT-NEXT: cmp w11, w8
; CHECK-CVT-NEXT: csel w11, w11, w8, lo
; CHECK-CVT-NEXT: cmp w12, w8 ; CHECK-CVT-NEXT: cmp w12, w8
; CHECK-CVT-NEXT: fmov s1, w10
; CHECK-CVT-NEXT: csel w12, w12, w8, lo ; CHECK-CVT-NEXT: csel w12, w12, w8, lo
; CHECK-CVT-NEXT: cmp w13, w8 ; CHECK-CVT-NEXT: cmp w13, w8
; CHECK-CVT-NEXT: csel w13, w13, w8, lo ; CHECK-CVT-NEXT: csel w13, w13, w8, lo
; CHECK-CVT-NEXT: cmp w14, w8 ; CHECK-CVT-NEXT: cmp w14, w8
; CHECK-CVT-NEXT: csel w14, w14, w8, lo ; CHECK-CVT-NEXT: csel w14, w14, w8, lo
; CHECK-CVT-NEXT: cmp w11, w8 ; CHECK-CVT-NEXT: mov v1.s[1], w9
; CHECK-CVT-NEXT: csel w11, w11, w8, lo ; CHECK-CVT-NEXT: cmp w15, w8
; CHECK-CVT-NEXT: fmov s2, w10 ; CHECK-CVT-NEXT: fmov s2, w14
; CHECK-CVT-NEXT: fcvtzu w10, s1 ; CHECK-CVT-NEXT: fcvtzu w9, s0
; CHECK-CVT-NEXT: fmov s3, w11 ; CHECK-CVT-NEXT: csel w10, w15, w8, lo
; CHECK-CVT-NEXT: mov v2.s[1], w9 ; CHECK-CVT-NEXT: mov v2.s[1], w13
; CHECK-CVT-NEXT: cmp w10, w8 ; CHECK-CVT-NEXT: mov v1.s[2], w11
; CHECK-CVT-NEXT: csel w9, w10, w8, lo ; CHECK-CVT-NEXT: cmp w9, w8
; CHECK-CVT-NEXT: fcvtzu w10, s0 ; CHECK-CVT-NEXT: csel w8, w9, w8, lo
; CHECK-CVT-NEXT: mov v3.s[1], w14 ; CHECK-CVT-NEXT: mov v2.s[2], w10
; CHECK-CVT-NEXT: mov v2.s[2], w12 ; CHECK-CVT-NEXT: mov v1.s[3], w12
; CHECK-CVT-NEXT: cmp w10, w8 ; CHECK-CVT-NEXT: mov v2.s[3], w8
; CHECK-CVT-NEXT: csel w8, w10, w8, lo ; CHECK-CVT-NEXT: uzp1 v0.8h, v2.8h, v1.8h
; CHECK-CVT-NEXT: mov v3.s[2], w9
; CHECK-CVT-NEXT: mov v2.s[3], w13
; CHECK-CVT-NEXT: mov v3.s[3], w8
; CHECK-CVT-NEXT: uzp1 v0.8h, v3.8h, v2.8h
; CHECK-CVT-NEXT: ret ; CHECK-CVT-NEXT: ret
; ;
; CHECK-FP16-LABEL: test_unsigned_v8f16_v8i13: ; CHECK-FP16-LABEL: test_unsigned_v8f16_v8i13:
Context not available.
; CHECK-CVT-NEXT: mov s2, v1.s[1] ; CHECK-CVT-NEXT: mov s2, v1.s[1]
; CHECK-CVT-NEXT: mov s3, v1.s[2] ; CHECK-CVT-NEXT: mov s3, v1.s[2]
; CHECK-CVT-NEXT: mov s4, v1.s[3] ; CHECK-CVT-NEXT: mov s4, v1.s[3]
; CHECK-CVT-NEXT: mov s5, v0.s[1]
; CHECK-CVT-NEXT: fcvtzu w10, s1 ; CHECK-CVT-NEXT: fcvtzu w10, s1
; CHECK-CVT-NEXT: fcvtzu w11, s0 ; CHECK-CVT-NEXT: fcvtzu w14, s0
; CHECK-CVT-NEXT: mov s1, v0.s[2] ; CHECK-CVT-NEXT: mov s1, v0.s[2]
; CHECK-CVT-NEXT: mov s0, v0.s[3]
; CHECK-CVT-NEXT: fcvtzu w9, s2 ; CHECK-CVT-NEXT: fcvtzu w9, s2
; CHECK-CVT-NEXT: fcvtzu w12, s3 ; CHECK-CVT-NEXT: mov s2, v0.s[1]
; CHECK-CVT-NEXT: fcvtzu w13, s4 ; CHECK-CVT-NEXT: fcvtzu w11, s3
; CHECK-CVT-NEXT: fcvtzu w14, s5 ; CHECK-CVT-NEXT: fcvtzu w12, s4
; CHECK-CVT-NEXT: fcvtzu w15, s1
; CHECK-CVT-NEXT: mov s0, v0.s[3]
; CHECK-CVT-NEXT: cmp w9, w8 ; CHECK-CVT-NEXT: cmp w9, w8
; CHECK-CVT-NEXT: fcvtzu w13, s2
; CHECK-CVT-NEXT: csel w9, w9, w8, lo ; CHECK-CVT-NEXT: csel w9, w9, w8, lo
; CHECK-CVT-NEXT: cmp w10, w8 ; CHECK-CVT-NEXT: cmp w10, w8
; CHECK-CVT-NEXT: csel w10, w10, w8, lo ; CHECK-CVT-NEXT: csel w10, w10, w8, lo
; CHECK-CVT-NEXT: cmp w11, w8
; CHECK-CVT-NEXT: csel w11, w11, w8, lo
; CHECK-CVT-NEXT: cmp w12, w8 ; CHECK-CVT-NEXT: cmp w12, w8
; CHECK-CVT-NEXT: fmov s1, w10
; CHECK-CVT-NEXT: csel w12, w12, w8, lo ; CHECK-CVT-NEXT: csel w12, w12, w8, lo
; CHECK-CVT-NEXT: cmp w13, w8 ; CHECK-CVT-NEXT: cmp w13, w8
; CHECK-CVT-NEXT: csel w13, w13, w8, lo ; CHECK-CVT-NEXT: csel w13, w13, w8, lo
; CHECK-CVT-NEXT: cmp w14, w8 ; CHECK-CVT-NEXT: cmp w14, w8
; CHECK-CVT-NEXT: csel w14, w14, w8, lo ; CHECK-CVT-NEXT: csel w14, w14, w8, lo
; CHECK-CVT-NEXT: cmp w11, w8 ; CHECK-CVT-NEXT: mov v1.s[1], w9
; CHECK-CVT-NEXT: csel w11, w11, w8, lo ; CHECK-CVT-NEXT: cmp w15, w8
; CHECK-CVT-NEXT: fmov s2, w10 ; CHECK-CVT-NEXT: fmov s2, w14
; CHECK-CVT-NEXT: fcvtzu w10, s1 ; CHECK-CVT-NEXT: fcvtzu w9, s0
; CHECK-CVT-NEXT: fmov s3, w11 ; CHECK-CVT-NEXT: csel w10, w15, w8, lo
; CHECK-CVT-NEXT: mov v2.s[1], w9 ; CHECK-CVT-NEXT: mov v2.s[1], w13
; CHECK-CVT-NEXT: cmp w10, w8 ; CHECK-CVT-NEXT: mov v1.s[2], w11
; CHECK-CVT-NEXT: csel w9, w10, w8, lo ; CHECK-CVT-NEXT: cmp w9, w8
; CHECK-CVT-NEXT: fcvtzu w10, s0 ; CHECK-CVT-NEXT: csel w8, w9, w8, lo
; CHECK-CVT-NEXT: mov v3.s[1], w14 ; CHECK-CVT-NEXT: mov v2.s[2], w10
; CHECK-CVT-NEXT: mov v2.s[2], w12 ; CHECK-CVT-NEXT: mov v1.s[3], w12
; CHECK-CVT-NEXT: cmp w10, w8 ; CHECK-CVT-NEXT: mov v2.s[3], w8
; CHECK-CVT-NEXT: csel w8, w10, w8, lo ; CHECK-CVT-NEXT: uzp1 v0.8h, v2.8h, v1.8h
; CHECK-CVT-NEXT: mov v3.s[2], w9
; CHECK-CVT-NEXT: mov v2.s[3], w13
; CHECK-CVT-NEXT: mov v3.s[3], w8
; CHECK-CVT-NEXT: uzp1 v0.8h, v3.8h, v2.8h
; CHECK-CVT-NEXT: ret ; CHECK-CVT-NEXT: ret
; ;
; CHECK-FP16-LABEL: test_unsigned_v8f16_v8i16: ; CHECK-FP16-LABEL: test_unsigned_v8f16_v8i16:
Context not available.
; CHECK-NEXT: umin v0.4s, v0.4s, v1.4s ; CHECK-NEXT: umin v0.4s, v0.4s, v1.4s
; CHECK-NEXT: mov w1, v2.s[1] ; CHECK-NEXT: mov w1, v2.s[1]
; CHECK-NEXT: mov w2, v2.s[2] ; CHECK-NEXT: mov w2, v2.s[2]
; CHECK-NEXT: mov w5, v0.s[1]
; CHECK-NEXT: mov w3, v2.s[3] ; CHECK-NEXT: mov w3, v2.s[3]
; CHECK-NEXT: mov w5, v0.s[1]
; CHECK-NEXT: mov w6, v0.s[2] ; CHECK-NEXT: mov w6, v0.s[2]
; CHECK-NEXT: mov w7, v0.s[3] ; CHECK-NEXT: mov w7, v0.s[3]
; CHECK-NEXT: fmov w4, s0 ; CHECK-NEXT: fmov w4, s0
Context not available.
; CHECK-CVT-LABEL: test_unsigned_v8f16_v8i50: ; CHECK-CVT-LABEL: test_unsigned_v8f16_v8i50:
; CHECK-CVT: // %bb.0: ; CHECK-CVT: // %bb.0:
; CHECK-CVT-NEXT: ext v1.16b, v0.16b, v0.16b, #8 ; CHECK-CVT-NEXT: ext v1.16b, v0.16b, v0.16b, #8
; CHECK-CVT-NEXT: mov h5, v0.h[1]
; CHECK-CVT-NEXT: mov x8, #1125899906842623 // =0x3ffffffffffff ; CHECK-CVT-NEXT: mov x8, #1125899906842623 // =0x3ffffffffffff
; CHECK-CVT-NEXT: mov h2, v0.h[1] ; CHECK-CVT-NEXT: mov h6, v0.h[2]
; CHECK-CVT-NEXT: mov h3, v0.h[2] ; CHECK-CVT-NEXT: mov h7, v0.h[3]
; CHECK-CVT-NEXT: mov h5, v0.h[3]
; CHECK-CVT-NEXT: fcvt s0, h0 ; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: mov h4, v1.h[1] ; CHECK-CVT-NEXT: mov h2, v1.h[1]
; CHECK-CVT-NEXT: mov h6, v1.h[2] ; CHECK-CVT-NEXT: mov h3, v1.h[2]
; CHECK-CVT-NEXT: mov h7, v1.h[3] ; CHECK-CVT-NEXT: mov h4, v1.h[3]
; CHECK-CVT-NEXT: fcvt s1, h1 ; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvtzu x13, s0
; CHECK-CVT-NEXT: fcvt s2, h2 ; CHECK-CVT-NEXT: fcvt s2, h2
; CHECK-CVT-NEXT: fcvt s3, h3 ; CHECK-CVT-NEXT: fcvt s3, h3
; CHECK-CVT-NEXT: fcvtzu x9, s0
; CHECK-CVT-NEXT: fcvt s5, h5
; CHECK-CVT-NEXT: fcvt s4, h4 ; CHECK-CVT-NEXT: fcvt s4, h4
; CHECK-CVT-NEXT: fcvt s6, h6 ; CHECK-CVT-NEXT: fcvtzu x9, s1
; CHECK-CVT-NEXT: fcvt s0, h7 ; CHECK-CVT-NEXT: fcvt s1, h5
; CHECK-CVT-NEXT: fcvtzu x10, s1 ; CHECK-CVT-NEXT: fcvtzu x10, s2
; CHECK-CVT-NEXT: fcvtzu x11, s2 ; CHECK-CVT-NEXT: fcvtzu x11, s3
; CHECK-CVT-NEXT: fcvtzu x12, s3 ; CHECK-CVT-NEXT: fcvt s2, h6
; CHECK-CVT-NEXT: fcvtzu x14, s5 ; CHECK-CVT-NEXT: fcvtzu x12, s4
; CHECK-CVT-NEXT: fcvtzu x13, s4 ; CHECK-CVT-NEXT: fcvt s3, h7
; CHECK-CVT-NEXT: fcvtzu x15, s6
; CHECK-CVT-NEXT: cmp x10, x8
; CHECK-CVT-NEXT: fcvtzu x16, s0
; CHECK-CVT-NEXT: csel x4, x10, x8, lo
; CHECK-CVT-NEXT: cmp x13, x8
; CHECK-CVT-NEXT: csel x5, x13, x8, lo
; CHECK-CVT-NEXT: cmp x15, x8
; CHECK-CVT-NEXT: csel x6, x15, x8, lo
; CHECK-CVT-NEXT: cmp x16, x8
; CHECK-CVT-NEXT: csel x7, x16, x8, lo
; CHECK-CVT-NEXT: cmp x9, x8 ; CHECK-CVT-NEXT: cmp x9, x8
; CHECK-CVT-NEXT: csel x0, x9, x8, lo ; CHECK-CVT-NEXT: fcvtzu x14, s1
; CHECK-CVT-NEXT: csel x4, x9, x8, lo
; CHECK-CVT-NEXT: cmp x10, x8
; CHECK-CVT-NEXT: fcvtzu x9, s2
; CHECK-CVT-NEXT: csel x5, x10, x8, lo
; CHECK-CVT-NEXT: cmp x11, x8 ; CHECK-CVT-NEXT: cmp x11, x8
; CHECK-CVT-NEXT: csel x1, x11, x8, lo ; CHECK-CVT-NEXT: fcvtzu x10, s3
; CHECK-CVT-NEXT: csel x6, x11, x8, lo
; CHECK-CVT-NEXT: cmp x12, x8 ; CHECK-CVT-NEXT: cmp x12, x8
; CHECK-CVT-NEXT: csel x2, x12, x8, lo ; CHECK-CVT-NEXT: csel x7, x12, x8, lo
; CHECK-CVT-NEXT: cmp x13, x8
; CHECK-CVT-NEXT: csel x0, x13, x8, lo
; CHECK-CVT-NEXT: cmp x14, x8 ; CHECK-CVT-NEXT: cmp x14, x8
; CHECK-CVT-NEXT: csel x3, x14, x8, lo ; CHECK-CVT-NEXT: csel x1, x14, x8, lo
; CHECK-CVT-NEXT: cmp x9, x8
; CHECK-CVT-NEXT: csel x2, x9, x8, lo
; CHECK-CVT-NEXT: cmp x10, x8
; CHECK-CVT-NEXT: csel x3, x10, x8, lo
; CHECK-CVT-NEXT: ret ; CHECK-CVT-NEXT: ret
; ;
; CHECK-FP16-LABEL: test_unsigned_v8f16_v8i50: ; CHECK-FP16-LABEL: test_unsigned_v8f16_v8i50:
; CHECK-FP16: // %bb.0: ; CHECK-FP16: // %bb.0:
; CHECK-FP16-NEXT: ext v1.16b, v0.16b, v0.16b, #8 ; CHECK-FP16-NEXT: ext v1.16b, v0.16b, v0.16b, #8
; CHECK-FP16-NEXT: mov x8, #1125899906842623 // =0x3ffffffffffff ; CHECK-FP16-NEXT: mov x8, #1125899906842623 // =0x3ffffffffffff
; CHECK-FP16-NEXT: mov h2, v0.h[1] ; CHECK-FP16-NEXT: fcvtzu x13, h0
; CHECK-FP16-NEXT: mov h3, v0.h[2] ; CHECK-FP16-NEXT: mov h2, v1.h[1]
; CHECK-FP16-NEXT: mov h5, v0.h[3] ; CHECK-FP16-NEXT: mov h3, v1.h[2]
; CHECK-FP16-NEXT: fcvtzu x9, h0 ; CHECK-FP16-NEXT: mov h4, v1.h[3]
; CHECK-FP16-NEXT: mov h4, v1.h[1] ; CHECK-FP16-NEXT: fcvtzu x9, h1
; CHECK-FP16-NEXT: mov h6, v1.h[2] ; CHECK-FP16-NEXT: mov h1, v0.h[1]
; CHECK-FP16-NEXT: mov h0, v1.h[3] ; CHECK-FP16-NEXT: fcvtzu x10, h2
; CHECK-FP16-NEXT: fcvtzu x10, h1 ; CHECK-FP16-NEXT: fcvtzu x11, h3
; CHECK-FP16-NEXT: fcvtzu x11, h2 ; CHECK-FP16-NEXT: mov h2, v0.h[2]
; CHECK-FP16-NEXT: fcvtzu x12, h3 ; CHECK-FP16-NEXT: fcvtzu x12, h4
; CHECK-FP16-NEXT: fcvtzu x14, h5 ; CHECK-FP16-NEXT: mov h3, v0.h[3]
; CHECK-FP16-NEXT: fcvtzu x13, h4
; CHECK-FP16-NEXT: fcvtzu x15, h6
; CHECK-FP16-NEXT: cmp x10, x8
; CHECK-FP16-NEXT: fcvtzu x16, h0
; CHECK-FP16-NEXT: csel x4, x10, x8, lo
; CHECK-FP16-NEXT: cmp x13, x8
; CHECK-FP16-NEXT: csel x5, x13, x8, lo
; CHECK-FP16-NEXT: cmp x15, x8
; CHECK-FP16-NEXT: csel x6, x15, x8, lo
; CHECK-FP16-NEXT: cmp x16, x8
; CHECK-FP16-NEXT: csel x7, x16, x8, lo
; CHECK-FP16-NEXT: cmp x9, x8 ; CHECK-FP16-NEXT: cmp x9, x8
; CHECK-FP16-NEXT: csel x0, x9, x8, lo ; CHECK-FP16-NEXT: fcvtzu x14, h1
; CHECK-FP16-NEXT: csel x4, x9, x8, lo
; CHECK-FP16-NEXT: cmp x10, x8
; CHECK-FP16-NEXT: fcvtzu x9, h2
; CHECK-FP16-NEXT: csel x5, x10, x8, lo
; CHECK-FP16-NEXT: cmp x11, x8 ; CHECK-FP16-NEXT: cmp x11, x8
; CHECK-FP16-NEXT: csel x1, x11, x8, lo ; CHECK-FP16-NEXT: fcvtzu x10, h3
; CHECK-FP16-NEXT: csel x6, x11, x8, lo
; CHECK-FP16-NEXT: cmp x12, x8 ; CHECK-FP16-NEXT: cmp x12, x8
; CHECK-FP16-NEXT: csel x2, x12, x8, lo ; CHECK-FP16-NEXT: csel x7, x12, x8, lo
; CHECK-FP16-NEXT: cmp x13, x8
; CHECK-FP16-NEXT: csel x0, x13, x8, lo
; CHECK-FP16-NEXT: cmp x14, x8 ; CHECK-FP16-NEXT: cmp x14, x8
; CHECK-FP16-NEXT: csel x3, x14, x8, lo ; CHECK-FP16-NEXT: csel x1, x14, x8, lo
; CHECK-FP16-NEXT: cmp x9, x8
; CHECK-FP16-NEXT: csel x2, x9, x8, lo
; CHECK-FP16-NEXT: cmp x10, x8
; CHECK-FP16-NEXT: csel x3, x10, x8, lo
; CHECK-FP16-NEXT: ret ; CHECK-FP16-NEXT: ret
%x = call <8 x i50> @llvm.fptoui.sat.v8f16.v8i50(<8 x half> %f) %x = call <8 x i50> @llvm.fptoui.sat.v8f16.v8i50(<8 x half> %f)
ret <8 x i50> %x ret <8 x i50> %x
Context not available.
; CHECK-CVT-LABEL: test_unsigned_v8f16_v8i64: ; CHECK-CVT-LABEL: test_unsigned_v8f16_v8i64:
; CHECK-CVT: // %bb.0: ; CHECK-CVT: // %bb.0:
; CHECK-CVT-NEXT: ext v1.16b, v0.16b, v0.16b, #8 ; CHECK-CVT-NEXT: ext v1.16b, v0.16b, v0.16b, #8
; CHECK-CVT-NEXT: mov h2, v0.h[2] ; CHECK-CVT-NEXT: mov h4, v0.h[2]
; CHECK-CVT-NEXT: fcvt s3, h0 ; CHECK-CVT-NEXT: mov h3, v0.h[1]
; CHECK-CVT-NEXT: mov h7, v0.h[1] ; CHECK-CVT-NEXT: mov h7, v0.h[3]
; CHECK-CVT-NEXT: mov h0, v0.h[3] ; CHECK-CVT-NEXT: fcvt s0, h0
; CHECK-CVT-NEXT: mov h4, v1.h[1] ; CHECK-CVT-NEXT: mov h2, v1.h[2]
; CHECK-CVT-NEXT: mov h6, v1.h[2] ; CHECK-CVT-NEXT: mov h5, v1.h[1]
; CHECK-CVT-NEXT: fcvt s5, h1 ; CHECK-CVT-NEXT: mov h6, v1.h[3]
; CHECK-CVT-NEXT: mov h1, v1.h[3] ; CHECK-CVT-NEXT: fcvt s1, h1
; CHECK-CVT-NEXT: fcvt s4, h4
; CHECK-CVT-NEXT: fcvt s3, h3
; CHECK-CVT-NEXT: fcvt s7, h7
; CHECK-CVT-NEXT: fcvtzu x9, s0
; CHECK-CVT-NEXT: fcvt s2, h2 ; CHECK-CVT-NEXT: fcvt s2, h2
; CHECK-CVT-NEXT: fcvtzu x8, s3 ; CHECK-CVT-NEXT: fcvt s5, h5
; CHECK-CVT-NEXT: fcvt s3, h4 ; CHECK-CVT-NEXT: fcvt s6, h6
; CHECK-CVT-NEXT: fcvt s4, h6 ; CHECK-CVT-NEXT: fcvtzu x8, s1
; CHECK-CVT-NEXT: fcvtzu x9, s5 ; CHECK-CVT-NEXT: fcvtzu x12, s4
; CHECK-CVT-NEXT: fcvt s5, h7
; CHECK-CVT-NEXT: fcvt s6, h0
; CHECK-CVT-NEXT: fcvt s7, h1
; CHECK-CVT-NEXT: fcvtzu x10, s2
; CHECK-CVT-NEXT: fmov d0, x8
; CHECK-CVT-NEXT: fmov d2, x9
; CHECK-CVT-NEXT: fcvtzu x9, s4
; CHECK-CVT-NEXT: fcvtzu x11, s3 ; CHECK-CVT-NEXT: fcvtzu x11, s3
; CHECK-CVT-NEXT: fcvtzu x8, s5 ; CHECK-CVT-NEXT: fcvtzu x15, s7
; CHECK-CVT-NEXT: fmov d1, x10 ; CHECK-CVT-NEXT: fmov d0, x9
; CHECK-CVT-NEXT: fcvtzu x10, s6 ; CHECK-CVT-NEXT: fcvtzu x10, s2
; CHECK-CVT-NEXT: fmov d3, x9 ; CHECK-CVT-NEXT: fcvtzu x13, s5
; CHECK-CVT-NEXT: fcvtzu x9, s7 ; CHECK-CVT-NEXT: fcvtzu x14, s6
; CHECK-CVT-NEXT: mov v2.d[1], x11 ; CHECK-CVT-NEXT: fmov d2, x8
; CHECK-CVT-NEXT: mov v0.d[1], x8 ; CHECK-CVT-NEXT: fmov d1, x12
; CHECK-CVT-NEXT: mov v1.d[1], x10 ; CHECK-CVT-NEXT: mov v0.d[1], x11
; CHECK-CVT-NEXT: mov v3.d[1], x9 ; CHECK-CVT-NEXT: fmov d3, x10
; CHECK-CVT-NEXT: mov v2.d[1], x13
; CHECK-CVT-NEXT: mov v1.d[1], x15
; CHECK-CVT-NEXT: mov v3.d[1], x14
; CHECK-CVT-NEXT: ret ; CHECK-CVT-NEXT: ret
; ;
; CHECK-FP16-LABEL: test_unsigned_v8f16_v8i64: ; CHECK-FP16-LABEL: test_unsigned_v8f16_v8i64:
; CHECK-FP16: // %bb.0: ; CHECK-FP16: // %bb.0:
; CHECK-FP16-NEXT: ext v1.16b, v0.16b, v0.16b, #8 ; CHECK-FP16-NEXT: ext v1.16b, v0.16b, v0.16b, #8
; CHECK-FP16-NEXT: mov h2, v0.h[2] ; CHECK-FP16-NEXT: mov h4, v0.h[2]
; CHECK-FP16-NEXT: mov h5, v0.h[1] ; CHECK-FP16-NEXT: mov h3, v0.h[1]
; CHECK-FP16-NEXT: mov h6, v0.h[3] ; CHECK-FP16-NEXT: mov h7, v0.h[3]
; CHECK-FP16-NEXT: fcvtzu x8, h0 ; CHECK-FP16-NEXT: fcvtzu x9, h0
; CHECK-FP16-NEXT: mov h4, v1.h[2] ; CHECK-FP16-NEXT: mov h2, v1.h[2]
; CHECK-FP16-NEXT: fcvtzu x9, h1 ; CHECK-FP16-NEXT: mov h5, v1.h[1]
; CHECK-FP16-NEXT: mov h3, v1.h[1] ; CHECK-FP16-NEXT: mov h6, v1.h[3]
; CHECK-FP16-NEXT: mov h7, v1.h[3] ; CHECK-FP16-NEXT: fcvtzu x8, h1
; CHECK-FP16-NEXT: fcvtzu x10, h2 ; CHECK-FP16-NEXT: fcvtzu x12, h4
; CHECK-FP16-NEXT: fmov d0, x8
; CHECK-FP16-NEXT: fmov d2, x9
; CHECK-FP16-NEXT: fcvtzu x8, h5
; CHECK-FP16-NEXT: fcvtzu x9, h4
; CHECK-FP16-NEXT: fcvtzu x11, h3 ; CHECK-FP16-NEXT: fcvtzu x11, h3
; CHECK-FP16-NEXT: fmov d1, x10 ; CHECK-FP16-NEXT: fcvtzu x15, h7
; CHECK-FP16-NEXT: fcvtzu x10, h6 ; CHECK-FP16-NEXT: fmov d0, x9
; CHECK-FP16-NEXT: fmov d3, x9 ; CHECK-FP16-NEXT: fcvtzu x10, h2
; CHECK-FP16-NEXT: fcvtzu x9, h7 ; CHECK-FP16-NEXT: fcvtzu x13, h5
; CHECK-FP16-NEXT: mov v2.d[1], x11 ; CHECK-FP16-NEXT: fcvtzu x14, h6
; CHECK-FP16-NEXT: mov v0.d[1], x8 ; CHECK-FP16-NEXT: fmov d2, x8
; CHECK-FP16-NEXT: mov v1.d[1], x10 ; CHECK-FP16-NEXT: fmov d1, x12
; CHECK-FP16-NEXT: mov v3.d[1], x9 ; CHECK-FP16-NEXT: mov v0.d[1], x11
; CHECK-FP16-NEXT: fmov d3, x10
; CHECK-FP16-NEXT: mov v2.d[1], x13
; CHECK-FP16-NEXT: mov v1.d[1], x15
; CHECK-FP16-NEXT: mov v3.d[1], x14
; CHECK-FP16-NEXT: ret ; CHECK-FP16-NEXT: ret
%x = call <8 x i64> @llvm.fptoui.sat.v8f16.v8i64(<8 x half> %f) %x = call <8 x i64> @llvm.fptoui.sat.v8f16.v8i64(<8 x half> %f)
ret <8 x i64> %x ret <8 x i64> %x
Context not available.
; CHECK-NEXT: .cfi_offset b8, -104 ; CHECK-NEXT: .cfi_offset b8, -104
; CHECK-NEXT: .cfi_offset b9, -112 ; CHECK-NEXT: .cfi_offset b9, -112
; CHECK-NEXT: str q0, [sp, #48] // 16-byte Folded Spill ; CHECK-NEXT: str q0, [sp, #48] // 16-byte Folded Spill
; CHECK-NEXT: mov x19, x8
; CHECK-NEXT: ext v0.16b, v0.16b, v0.16b, #8 ; CHECK-NEXT: ext v0.16b, v0.16b, v0.16b, #8
; CHECK-NEXT: mov x19, x8
; CHECK-NEXT: str q0, [sp, #32] // 16-byte Folded Spill ; CHECK-NEXT: str q0, [sp, #32] // 16-byte Folded Spill
; CHECK-NEXT: mov h0, v0.h[1] ; CHECK-NEXT: mov h0, v0.h[1]
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
Context not available.
; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: mov w8, #1904214015 // =0x717fffff ; CHECK-NEXT: mov w8, #1904214015 // =0x717fffff
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
; CHECK-NEXT: mov x23, #68719476735 // =0xfffffffff
; CHECK-NEXT: mov h0, v0.h[3]
; CHECK-NEXT: fmov s9, w8 ; CHECK-NEXT: fmov s9, w8
; CHECK-NEXT: csel x8, xzr, x0, lt ; CHECK-NEXT: mov x22, #68719476735 // =0xfffffffff
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: mov h0, v0.h[3]
; CHECK-NEXT: csel x9, xzr, x0, lt
; CHECK-NEXT: csel x8, xzr, x1, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
; CHECK-NEXT: csel x9, x23, x9, gt ; CHECK-NEXT: csel x10, x22, x8, gt
; CHECK-NEXT: csinv x8, x8, xzr, le ; CHECK-NEXT: csinv x8, x9, xzr, le
; CHECK-NEXT: stp x8, x10, [sp, #16] // 16-byte Folded Spill
; CHECK-NEXT: fmov s0, s8 ; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: stp x8, x9, [sp, #16] // 16-byte Folded Spill
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload
Context not available.
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
; CHECK-NEXT: csel x9, x23, x9, gt ; CHECK-NEXT: csel x9, x22, x9, gt
; CHECK-NEXT: csinv x24, x8, xzr, le ; CHECK-NEXT: csinv x24, x8, xzr, le
; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: str x9, [sp, #8] // 8-byte Folded Spill ; CHECK-NEXT: str x9, [sp, #8] // 8-byte Folded Spill
; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
Context not available.
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
; CHECK-NEXT: csinv x8, x8, xzr, le ; CHECK-NEXT: csinv x8, x8, xzr, le
; CHECK-NEXT: csel x25, x23, x9, gt ; CHECK-NEXT: csel x25, x22, x9, gt
; CHECK-NEXT: str x8, [sp, #32] // 8-byte Folded Spill ; CHECK-NEXT: str x8, [sp, #32] // 8-byte Folded Spill
; CHECK-NEXT: fmov s0, s8 ; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
Context not available.
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
; CHECK-NEXT: csinv x8, x8, xzr, le ; CHECK-NEXT: csinv x8, x8, xzr, le
; CHECK-NEXT: csel x27, x23, x9, gt ; CHECK-NEXT: csel x26, x22, x9, gt
; CHECK-NEXT: str x8, [sp] // 8-byte Folded Spill ; CHECK-NEXT: str x8, [sp] // 8-byte Folded Spill
; CHECK-NEXT: fmov s0, s8 ; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
Context not available.
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
; CHECK-NEXT: csel x29, x23, x9, gt ; CHECK-NEXT: csel x29, x22, x9, gt
; CHECK-NEXT: csinv x26, x8, xzr, le ; CHECK-NEXT: csinv x27, x8, xzr, le
; CHECK-NEXT: fmov s0, s8 ; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
Context not available.
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
; CHECK-NEXT: csel x28, x23, x9, gt ; CHECK-NEXT: csel x20, x22, x9, gt
; CHECK-NEXT: csinv x20, x8, xzr, le ; CHECK-NEXT: csinv x21, x8, xzr, le
; CHECK-NEXT: fmov s0, s8 ; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
; CHECK-NEXT: ldr q0, [sp, #48] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #48] // 16-byte Folded Reload
Context not available.
; CHECK-NEXT: csel x9, xzr, x1, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
; CHECK-NEXT: csel x21, x23, x9, gt ; CHECK-NEXT: csel x28, x22, x9, gt
; CHECK-NEXT: csinv x22, x8, xzr, le ; CHECK-NEXT: csinv x23, x8, xzr, le
; CHECK-NEXT: fmov s0, s8 ; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
; CHECK-NEXT: ldr x9, [sp] // 8-byte Folded Reload
dmgreenUnsubmitted
Done
ReplyInline Actions

There is an extra spill here, but that kind of things is expected given enough code.

dmgreen: There is an extra spill here, but that kind of things is expected given enough code.
; CHECK-NEXT: extr x8, x20, x21, #28
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
; CHECK-NEXT: extr x8, x28, x20, #28 ; CHECK-NEXT: bfi x28, x27, #36, #28
; CHECK-NEXT: bfi x21, x26, #36, #28
; CHECK-NEXT: extr x9, x29, x26, #28
; CHECK-NEXT: lsr x11, x29, #28 ; CHECK-NEXT: lsr x11, x29, #28
; CHECK-NEXT: str x22, [x19] ; CHECK-NEXT: bfi x26, x24, #36, #28
; CHECK-NEXT: stur x9, [x19, #75]
; CHECK-NEXT: extr x9, x29, x27, #28
; CHECK-NEXT: stur x8, [x19, #41] ; CHECK-NEXT: stur x8, [x19, #41]
; CHECK-NEXT: csel x8, xzr, x0, lt ; CHECK-NEXT: csel x8, xzr, x0, lt
; CHECK-NEXT: csel x10, xzr, x1, lt ; CHECK-NEXT: str x9, [x19, #16]
; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: stp x21, x9, [x19, #8] ; CHECK-NEXT: ldr x10, [sp, #32] // 8-byte Folded Reload
; CHECK-NEXT: lsr x9, x28, #28 ; CHECK-NEXT: stp x23, x28, [x19]
; CHECK-NEXT: strb w11, [x19, #24] ; CHECK-NEXT: strb w11, [x19, #24]
; CHECK-NEXT: bfi x27, x24, #36, #28 ; CHECK-NEXT: stur x10, [x19, #50]
; CHECK-NEXT: csel x10, x23, x10, gt ; CHECK-NEXT: lsr x10, x20, #28
; CHECK-NEXT: csel x9, x22, x9, gt
; CHECK-NEXT: bfi x9, x21, #36, #28
; CHECK-NEXT: csinv x8, x8, xzr, le ; CHECK-NEXT: csinv x8, x8, xzr, le
; CHECK-NEXT: bfi x10, x20, #36, #28 ; CHECK-NEXT: strb w10, [x19, #49]
; CHECK-NEXT: strb w9, [x19, #49] ; CHECK-NEXT: ldr x11, [sp, #8] // 8-byte Folded Reload
; CHECK-NEXT: stur x8, [x19, #25] ; CHECK-NEXT: stur x8, [x19, #25]
; CHECK-NEXT: stur x10, [x19, #33] ; CHECK-NEXT: stur x9, [x19, #33]
; CHECK-NEXT: ldp x9, x12, [sp] // 16-byte Folded Reload ; CHECK-NEXT: extr x10, x11, x24, #28
; CHECK-NEXT: stur x9, [x19, #75] ; CHECK-NEXT: stur x10, [x19, #91]
; CHECK-NEXT: extr x8, x12, x24, #28 ; CHECK-NEXT: ldp x10, x9, [sp, #16] // 16-byte Folded Reload
; CHECK-NEXT: ldr x9, [sp, #32] // 8-byte Folded Reload ; CHECK-NEXT: stur x26, [x19, #83]
; CHECK-NEXT: stur x9, [x19, #50] ; CHECK-NEXT: extr x8, x9, x10, #28
; CHECK-NEXT: ldp x11, x10, [sp, #16] // 16-byte Folded Reload ; CHECK-NEXT: bfi x25, x10, #36, #28
; CHECK-NEXT: stur x8, [x19, #91] ; CHECK-NEXT: lsr x9, x9, #28
; CHECK-NEXT: lsr x8, x12, #28 ; CHECK-NEXT: stur x8, [x19, #66]
; CHECK-NEXT: stur x27, [x19, #83] ; CHECK-NEXT: lsr x8, x11, #28
; CHECK-NEXT: extr x9, x10, x11, #28
; CHECK-NEXT: bfi x25, x11, #36, #28
; CHECK-NEXT: strb w8, [x19, #99]
; CHECK-NEXT: stur x9, [x19, #66]
; CHECK-NEXT: lsr x9, x10, #28
; CHECK-NEXT: stur x25, [x19, #58] ; CHECK-NEXT: stur x25, [x19, #58]
; CHECK-NEXT: strb w8, [x19, #99]
; CHECK-NEXT: strb w9, [x19, #74] ; CHECK-NEXT: strb w9, [x19, #74]
; CHECK-NEXT: ldp x20, x19, [sp, #160] // 16-byte Folded Reload ; CHECK-NEXT: ldp x20, x19, [sp, #160] // 16-byte Folded Reload
; CHECK-NEXT: ldp x22, x21, [sp, #144] // 16-byte Folded Reload ; CHECK-NEXT: ldp x22, x21, [sp, #144] // 16-byte Folded Reload
Context not available.
; CHECK-NEXT: .cfi_offset b8, -104 ; CHECK-NEXT: .cfi_offset b8, -104
; CHECK-NEXT: .cfi_offset b9, -112 ; CHECK-NEXT: .cfi_offset b9, -112
; CHECK-NEXT: str q0, [sp, #48] // 16-byte Folded Spill ; CHECK-NEXT: str q0, [sp, #48] // 16-byte Folded Spill
; CHECK-NEXT: mov x19, x8
; CHECK-NEXT: ext v0.16b, v0.16b, v0.16b, #8 ; CHECK-NEXT: ext v0.16b, v0.16b, v0.16b, #8
; CHECK-NEXT: mov x19, x8
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
; CHECK-NEXT: str q0, [sp, #32] // 16-byte Folded Spill ; CHECK-NEXT: str q0, [sp, #32] // 16-byte Folded Spill
; CHECK-NEXT: fmov s0, s8 ; CHECK-NEXT: fmov s0, s8
Context not available.
; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: mov w8, #2139095039 // =0x7f7fffff ; CHECK-NEXT: mov w8, #2139095039 // =0x7f7fffff
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
; CHECK-NEXT: mov h0, v0.h[1]
; CHECK-NEXT: fmov s9, w8 ; CHECK-NEXT: fmov s9, w8
; CHECK-NEXT: csel x8, xzr, x1, lt ; CHECK-NEXT: mov h0, v0.h[1]
; CHECK-NEXT: csel x9, xzr, x0, lt ; CHECK-NEXT: csel x9, xzr, x1, lt
; CHECK-NEXT: csel x8, xzr, x0, lt
; CHECK-NEXT: fcmp s8, s9 ; CHECK-NEXT: fcmp s8, s9
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
; CHECK-NEXT: csinv x9, x9, xzr, le ; CHECK-NEXT: csinv x10, x8, xzr, le
; CHECK-NEXT: csinv x8, x8, xzr, le ; CHECK-NEXT: csinv x8, x9, xzr, le
; CHECK-NEXT: stp x8, x10, [sp, #16] // 16-byte Folded Spill
; CHECK-NEXT: fmov s0, s8 ; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: stp x8, x9, [sp, #16] // 16-byte Folded Spill
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
Context not available.
; CHECK-NEXT: fcvt s8, h0 ; CHECK-NEXT: fcvt s8, h0
; CHECK-NEXT: csinv x9, x9, xzr, le ; CHECK-NEXT: csinv x9, x9, xzr, le
; CHECK-NEXT: csinv x8, x8, xzr, le ; CHECK-NEXT: csinv x8, x8, xzr, le
; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: stp x8, x9, [sp] // 16-byte Folded Spill ; CHECK-NEXT: stp x8, x9, [sp] // 16-byte Folded Spill
; CHECK-NEXT: fmov s0, s8
; CHECK-NEXT: bl __fixunssfti ; CHECK-NEXT: bl __fixunssfti
; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload ; CHECK-NEXT: ldr q0, [sp, #32] // 16-byte Folded Reload
; CHECK-NEXT: fcmp s8, #0.0 ; CHECK-NEXT: fcmp s8, #0.0
Context not available.
; CHECK-CVT: // %bb.0: ; CHECK-CVT: // %bb.0:
; CHECK-CVT-NEXT: fcvtl2 v2.4s, v1.8h ; CHECK-CVT-NEXT: fcvtl2 v2.4s, v1.8h
; CHECK-CVT-NEXT: fcvtl v1.4s, v1.4h ; CHECK-CVT-NEXT: fcvtl v1.4s, v1.4h
; CHECK-CVT-NEXT: fcvtl2 v5.4s, v0.8h
; CHECK-CVT-NEXT: mov w8, #255 // =0xff ; CHECK-CVT-NEXT: mov w8, #255 // =0xff
; CHECK-CVT-NEXT: fcvtl v0.4s, v0.4h
; CHECK-CVT-NEXT: mov s3, v2.s[1] ; CHECK-CVT-NEXT: mov s3, v2.s[1]
; CHECK-CVT-NEXT: mov s4, v2.s[2] ; CHECK-CVT-NEXT: mov s4, v2.s[2]
; CHECK-CVT-NEXT: fcvtzu w9, s2 ; CHECK-CVT-NEXT: mov s5, v2.s[3]
; CHECK-CVT-NEXT: mov s2, v2.s[3] ; CHECK-CVT-NEXT: fcvtzu w10, s2
; CHECK-CVT-NEXT: fcvtzu w12, s1 ; CHECK-CVT-NEXT: fcvtl2 v2.4s, v0.8h
; CHECK-CVT-NEXT: fcvtzu w16, s5 ; CHECK-CVT-NEXT: fcvtzu w13, s1
; CHECK-CVT-NEXT: fcvtzu w2, s0 ; CHECK-CVT-NEXT: fcvtl v0.4s, v0.4h
; CHECK-CVT-NEXT: fcvtzu w10, s3 ; CHECK-CVT-NEXT: fcvtzu w9, s3
; CHECK-CVT-NEXT: mov s3, v1.s[1] ; CHECK-CVT-NEXT: mov s3, v1.s[1]
; CHECK-CVT-NEXT: fcvtzu w11, s4 ; CHECK-CVT-NEXT: fcvtzu w11, s4
; CHECK-CVT-NEXT: mov s4, v1.s[2] ; CHECK-CVT-NEXT: mov s4, v1.s[2]
; CHECK-CVT-NEXT: fcvtzu w12, s5
; CHECK-CVT-NEXT: mov s1, v1.s[3] ; CHECK-CVT-NEXT: mov s1, v1.s[3]
; CHECK-CVT-NEXT: fcvtzu w13, s2 ; CHECK-CVT-NEXT: fcvtzu w18, s2
; CHECK-CVT-NEXT: cmp w10, #255 ; CHECK-CVT-NEXT: fcvtzu w3, s0
; CHECK-CVT-NEXT: mov s2, v5.s[1]
; CHECK-CVT-NEXT: fcvtzu w14, s3 ; CHECK-CVT-NEXT: fcvtzu w14, s3
; CHECK-CVT-NEXT: csel w10, w10, w8, lo
; CHECK-CVT-NEXT: cmp w9, #255 ; CHECK-CVT-NEXT: cmp w9, #255
; CHECK-CVT-NEXT: fcvtzu w15, s4 ; CHECK-CVT-NEXT: mov s3, v2.s[1]
; CHECK-CVT-NEXT: csel w9, w9, w8, lo ; CHECK-CVT-NEXT: csel w9, w9, w8, lo
; CHECK-CVT-NEXT: cmp w10, #255
; CHECK-CVT-NEXT: fcvtzu w15, s4
; CHECK-CVT-NEXT: csel w10, w10, w8, lo
; CHECK-CVT-NEXT: cmp w11, #255 ; CHECK-CVT-NEXT: cmp w11, #255
; CHECK-CVT-NEXT: mov s4, v2.s[2]
; CHECK-CVT-NEXT: csel w11, w11, w8, lo ; CHECK-CVT-NEXT: csel w11, w11, w8, lo
; CHECK-CVT-NEXT: cmp w13, #255
; CHECK-CVT-NEXT: mov s3, v5.s[2]
; CHECK-CVT-NEXT: fcvtzu w17, s1
; CHECK-CVT-NEXT: csel w13, w13, w8, lo
; CHECK-CVT-NEXT: cmp w14, #255
; CHECK-CVT-NEXT: mov s4, v5.s[3]
; CHECK-CVT-NEXT: fcvtzu w18, s2
; CHECK-CVT-NEXT: csel w14, w14, w8, lo
; CHECK-CVT-NEXT: cmp w12, #255 ; CHECK-CVT-NEXT: cmp w12, #255
; CHECK-CVT-NEXT: mov s1, v0.s[1] ; CHECK-CVT-NEXT: fcvtzu w16, s1
; CHECK-CVT-NEXT: mov s1, v2.s[3]
; CHECK-CVT-NEXT: csel w12, w12, w8, lo ; CHECK-CVT-NEXT: csel w12, w12, w8, lo
; CHECK-CVT-NEXT: cmp w14, #255
; CHECK-CVT-NEXT: fcvtzu w17, s3
; CHECK-CVT-NEXT: mov s3, v0.s[1]
; CHECK-CVT-NEXT: csel w14, w14, w8, lo
; CHECK-CVT-NEXT: cmp w13, #255
; CHECK-CVT-NEXT: fcvtzu w0, s4
; CHECK-CVT-NEXT: fmov s2, w10
; CHECK-CVT-NEXT: csel w13, w13, w8, lo
; CHECK-CVT-NEXT: cmp w15, #255 ; CHECK-CVT-NEXT: cmp w15, #255
; CHECK-CVT-NEXT: fcvtzu w0, s3
; CHECK-CVT-NEXT: csel w15, w15, w8, lo ; CHECK-CVT-NEXT: csel w15, w15, w8, lo
; CHECK-CVT-NEXT: cmp w17, #255
; CHECK-CVT-NEXT: csel w17, w17, w8, lo
; CHECK-CVT-NEXT: cmp w18, #255
; CHECK-CVT-NEXT: fmov s2, w9
; CHECK-CVT-NEXT: csel w9, w18, w8, lo
; CHECK-CVT-NEXT: fcvtzu w18, s4
; CHECK-CVT-NEXT: cmp w16, #255 ; CHECK-CVT-NEXT: cmp w16, #255
; CHECK-CVT-NEXT: fcvtzu w1, s1 ; CHECK-CVT-NEXT: fcvtzu w1, s1
; CHECK-CVT-NEXT: csel w16, w16, w8, lo ; CHECK-CVT-NEXT: csel w16, w16, w8, lo
; CHECK-CVT-NEXT: cmp w0, #255 ; CHECK-CVT-NEXT: cmp w17, #255
; CHECK-CVT-NEXT: fcvtzu w2, s3
; CHECK-CVT-NEXT: csel w17, w17, w8, lo
; CHECK-CVT-NEXT: cmp w18, #255
; CHECK-CVT-NEXT: mov s1, v0.s[2] ; CHECK-CVT-NEXT: mov s1, v0.s[2]
; CHECK-CVT-NEXT: csel w18, w18, w8, lo
; CHECK-CVT-NEXT: cmp w0, #255
; CHECK-CVT-NEXT: mov v2.s[1], w9
; CHECK-CVT-NEXT: csel w0, w0, w8, lo ; CHECK-CVT-NEXT: csel w0, w0, w8, lo
; CHECK-CVT-NEXT: cmp w18, #255
; CHECK-CVT-NEXT: mov v2.s[1], w10
; CHECK-CVT-NEXT: csel w10, w18, w8, lo
; CHECK-CVT-NEXT: cmp w1, #255 ; CHECK-CVT-NEXT: cmp w1, #255
; CHECK-CVT-NEXT: fmov s3, w12 ; CHECK-CVT-NEXT: fmov s3, w18
; CHECK-CVT-NEXT: csel w18, w1, w8, lo ; CHECK-CVT-NEXT: csel w10, w1, w8, lo
; CHECK-CVT-NEXT: cmp w2, #255 ; CHECK-CVT-NEXT: cmp w2, #255
; CHECK-CVT-NEXT: csel w1, w2, w8, lo
; CHECK-CVT-NEXT: fmov s4, w16
; CHECK-CVT-NEXT: mov v2.s[2], w11
; CHECK-CVT-NEXT: fcvtzu w11, s1
; CHECK-CVT-NEXT: mov s0, v0.s[3] ; CHECK-CVT-NEXT: mov s0, v0.s[3]
; CHECK-CVT-NEXT: fmov s1, w1 ; CHECK-CVT-NEXT: csel w9, w2, w8, lo
; CHECK-CVT-NEXT: mov v3.s[1], w14 ; CHECK-CVT-NEXT: cmp w3, #255
; CHECK-CVT-NEXT: cmp w11, #255 ; CHECK-CVT-NEXT: fcvtzu w2, s1
; CHECK-CVT-NEXT: csel w1, w3, w8, lo
; CHECK-CVT-NEXT: fmov s1, w13
; CHECK-CVT-NEXT: mov v3.s[1], w17
; CHECK-CVT-NEXT: fmov s4, w1
; CHECK-CVT-NEXT: mov v2.s[2], w11
; CHECK-CVT-NEXT: mov v1.s[1], w14
; CHECK-CVT-NEXT: cmp w2, #255
; CHECK-CVT-NEXT: mov v4.s[1], w9 ; CHECK-CVT-NEXT: mov v4.s[1], w9
; CHECK-CVT-NEXT: csel w9, w11, w8, lo ; CHECK-CVT-NEXT: fcvtzu w9, s0
; CHECK-CVT-NEXT: mov v1.s[1], w18 ; CHECK-CVT-NEXT: csel w11, w2, w8, lo
; CHECK-CVT-NEXT: fcvtzu w11, s0 ; CHECK-CVT-NEXT: mov v3.s[2], w0
; CHECK-CVT-NEXT: mov v3.s[2], w15 ; CHECK-CVT-NEXT: mov v2.s[3], w12
; CHECK-CVT-NEXT: mov v4.s[2], w0 ; CHECK-CVT-NEXT: mov v1.s[2], w15
; CHECK-CVT-NEXT: mov v1.s[2], w9 ; CHECK-CVT-NEXT: mov v4.s[2], w11
; CHECK-CVT-NEXT: cmp w11, #255 ; CHECK-CVT-NEXT: cmp w9, #255
; CHECK-CVT-NEXT: csel w8, w11, w8, lo ; CHECK-CVT-NEXT: csel w8, w9, w8, lo
; CHECK-CVT-NEXT: mov v2.s[3], w13 ; CHECK-CVT-NEXT: mov v3.s[3], w10
; CHECK-CVT-NEXT: mov v3.s[3], w17 ; CHECK-CVT-NEXT: mov v1.s[3], w16
; CHECK-CVT-NEXT: mov v4.s[3], w10 ; CHECK-CVT-NEXT: mov v4.s[3], w8
; CHECK-CVT-NEXT: mov v1.s[3], w8 ; CHECK-CVT-NEXT: uzp1 v0.8h, v1.8h, v2.8h
; CHECK-CVT-NEXT: uzp1 v0.8h, v3.8h, v2.8h ; CHECK-CVT-NEXT: uzp1 v1.8h, v4.8h, v3.8h
; CHECK-CVT-NEXT: uzp1 v1.8h, v1.8h, v4.8h
; CHECK-CVT-NEXT: uzp1 v0.16b, v1.16b, v0.16b ; CHECK-CVT-NEXT: uzp1 v0.16b, v1.16b, v0.16b
; CHECK-CVT-NEXT: ret ; CHECK-CVT-NEXT: ret
; ;
Context not available.
; CHECK-CVT: // %bb.0: ; CHECK-CVT: // %bb.0:
; CHECK-CVT-NEXT: fcvtl2 v2.4s, v0.8h ; CHECK-CVT-NEXT: fcvtl2 v2.4s, v0.8h
; CHECK-CVT-NEXT: fcvtl v0.4s, v0.4h ; CHECK-CVT-NEXT: fcvtl v0.4s, v0.4h
; CHECK-CVT-NEXT: fcvtl2 v5.4s, v1.8h
; CHECK-CVT-NEXT: mov w8, #65535 // =0xffff ; CHECK-CVT-NEXT: mov w8, #65535 // =0xffff
; CHECK-CVT-NEXT: fcvtl v1.4s, v1.4h
; CHECK-CVT-NEXT: mov s3, v2.s[1] ; CHECK-CVT-NEXT: mov s3, v2.s[1]
; CHECK-CVT-NEXT: mov s4, v2.s[2] ; CHECK-CVT-NEXT: mov s4, v2.s[2]
; CHECK-CVT-NEXT: fcvtzu w9, s2 ; CHECK-CVT-NEXT: mov s5, v2.s[3]
; CHECK-CVT-NEXT: mov s2, v2.s[3] ; CHECK-CVT-NEXT: fcvtzu w10, s2
; CHECK-CVT-NEXT: fcvtzu w12, s0 ; CHECK-CVT-NEXT: fcvtl2 v2.4s, v1.8h
; CHECK-CVT-NEXT: fcvtzu w16, s5 ; CHECK-CVT-NEXT: fcvtzu w13, s0
; CHECK-CVT-NEXT: fcvtzu w2, s1 ; CHECK-CVT-NEXT: fcvtl v1.4s, v1.4h
; CHECK-CVT-NEXT: fcvtzu w10, s3 ; CHECK-CVT-NEXT: fcvtzu w9, s3
; CHECK-CVT-NEXT: mov s3, v0.s[1] ; CHECK-CVT-NEXT: mov s3, v0.s[1]
; CHECK-CVT-NEXT: fcvtzu w11, s4 ; CHECK-CVT-NEXT: fcvtzu w11, s4
; CHECK-CVT-NEXT: mov s4, v0.s[2] ; CHECK-CVT-NEXT: mov s4, v0.s[2]
; CHECK-CVT-NEXT: fcvtzu w12, s5
; CHECK-CVT-NEXT: mov s0, v0.s[3] ; CHECK-CVT-NEXT: mov s0, v0.s[3]
; CHECK-CVT-NEXT: fcvtzu w13, s2 ; CHECK-CVT-NEXT: fcvtzu w18, s2
; CHECK-CVT-NEXT: cmp w10, w8 ; CHECK-CVT-NEXT: fcvtzu w3, s1
; CHECK-CVT-NEXT: mov s2, v5.s[1]
; CHECK-CVT-NEXT: fcvtzu w14, s3 ; CHECK-CVT-NEXT: fcvtzu w14, s3
; CHECK-CVT-NEXT: csel w10, w10, w8, lo
; CHECK-CVT-NEXT: cmp w9, w8 ; CHECK-CVT-NEXT: cmp w9, w8
; CHECK-CVT-NEXT: fcvtzu w15, s4 ; CHECK-CVT-NEXT: mov s3, v2.s[1]
; CHECK-CVT-NEXT: csel w9, w9, w8, lo ; CHECK-CVT-NEXT: csel w9, w9, w8, lo
; CHECK-CVT-NEXT: cmp w10, w8
; CHECK-CVT-NEXT: fcvtzu w15, s4
; CHECK-CVT-NEXT: csel w10, w10, w8, lo
; CHECK-CVT-NEXT: cmp w11, w8 ; CHECK-CVT-NEXT: cmp w11, w8
; CHECK-CVT-NEXT: mov s4, v2.s[2]
; CHECK-CVT-NEXT: csel w11, w11, w8, lo ; CHECK-CVT-NEXT: csel w11, w11, w8, lo
; CHECK-CVT-NEXT: cmp w13, w8
; CHECK-CVT-NEXT: mov s3, v5.s[2]
; CHECK-CVT-NEXT: fcvtzu w17, s0
; CHECK-CVT-NEXT: csel w13, w13, w8, lo
; CHECK-CVT-NEXT: cmp w14, w8
; CHECK-CVT-NEXT: mov s4, v5.s[3]
; CHECK-CVT-NEXT: fcvtzu w18, s2
; CHECK-CVT-NEXT: csel w14, w14, w8, lo
; CHECK-CVT-NEXT: cmp w12, w8 ; CHECK-CVT-NEXT: cmp w12, w8
; CHECK-CVT-NEXT: mov s0, v1.s[1] ; CHECK-CVT-NEXT: fcvtzu w16, s0
; CHECK-CVT-NEXT: mov s0, v2.s[3]
; CHECK-CVT-NEXT: csel w12, w12, w8, lo ; CHECK-CVT-NEXT: csel w12, w12, w8, lo
; CHECK-CVT-NEXT: cmp w14, w8
; CHECK-CVT-NEXT: fcvtzu w17, s3
; CHECK-CVT-NEXT: mov s3, v1.s[1]
; CHECK-CVT-NEXT: csel w14, w14, w8, lo
; CHECK-CVT-NEXT: cmp w13, w8
; CHECK-CVT-NEXT: fcvtzu w0, s4
; CHECK-CVT-NEXT: fmov s2, w10
; CHECK-CVT-NEXT: csel w13, w13, w8, lo
; CHECK-CVT-NEXT: cmp w15, w8 ; CHECK-CVT-NEXT: cmp w15, w8
; CHECK-CVT-NEXT: fcvtzu w0, s3
; CHECK-CVT-NEXT: csel w15, w15, w8, lo ; CHECK-CVT-NEXT: csel w15, w15, w8, lo
; CHECK-CVT-NEXT: cmp w17, w8
; CHECK-CVT-NEXT: csel w17, w17, w8, lo
; CHECK-CVT-NEXT: cmp w18, w8
; CHECK-CVT-NEXT: fmov s2, w9
; CHECK-CVT-NEXT: csel w9, w18, w8, lo
; CHECK-CVT-NEXT: fcvtzu w18, s4
; CHECK-CVT-NEXT: cmp w16, w8 ; CHECK-CVT-NEXT: cmp w16, w8
; CHECK-CVT-NEXT: fcvtzu w1, s0 ; CHECK-CVT-NEXT: fcvtzu w1, s0
; CHECK-CVT-NEXT: csel w16, w16, w8, lo ; CHECK-CVT-NEXT: csel w16, w16, w8, lo
; CHECK-CVT-NEXT: cmp w0, w8 ; CHECK-CVT-NEXT: cmp w17, w8
; CHECK-CVT-NEXT: fcvtzu w2, s3
; CHECK-CVT-NEXT: csel w17, w17, w8, lo
; CHECK-CVT-NEXT: cmp w18, w8
; CHECK-CVT-NEXT: mov s0, v1.s[2] ; CHECK-CVT-NEXT: mov s0, v1.s[2]
; CHECK-CVT-NEXT: csel w18, w18, w8, lo
; CHECK-CVT-NEXT: cmp w0, w8
; CHECK-CVT-NEXT: mov v2.s[1], w9
; CHECK-CVT-NEXT: csel w0, w0, w8, lo ; CHECK-CVT-NEXT: csel w0, w0, w8, lo
; CHECK-CVT-NEXT: cmp w18, w8
; CHECK-CVT-NEXT: mov v2.s[1], w10
; CHECK-CVT-NEXT: csel w10, w18, w8, lo
; CHECK-CVT-NEXT: cmp w1, w8 ; CHECK-CVT-NEXT: cmp w1, w8
; CHECK-CVT-NEXT: fmov s3, w12 ; CHECK-CVT-NEXT: fmov s3, w18
; CHECK-CVT-NEXT: csel w18, w1, w8, lo ; CHECK-CVT-NEXT: csel w10, w1, w8, lo
; CHECK-CVT-NEXT: cmp w2, w8 ; CHECK-CVT-NEXT: cmp w2, w8
; CHECK-CVT-NEXT: csel w1, w2, w8, lo ; CHECK-CVT-NEXT: csel w9, w2, w8, lo
; CHECK-CVT-NEXT: fmov s4, w16 ; CHECK-CVT-NEXT: cmp w3, w8
; CHECK-CVT-NEXT: mov v2.s[2], w11 ; CHECK-CVT-NEXT: fcvtzu w2, s0
; CHECK-CVT-NEXT: fcvtzu w11, s0 ; CHECK-CVT-NEXT: csel w1, w3, w8, lo
; CHECK-CVT-NEXT: mov s0, v1.s[3] ; CHECK-CVT-NEXT: mov s0, v1.s[3]
; CHECK-CVT-NEXT: fmov s5, w1 ; CHECK-CVT-NEXT: fmov s1, w13
; CHECK-CVT-NEXT: mov v3.s[1], w14 ; CHECK-CVT-NEXT: fmov s4, w1
; CHECK-CVT-NEXT: cmp w11, w8 ; CHECK-CVT-NEXT: mov v3.s[1], w17
; CHECK-CVT-NEXT: mov v2.s[2], w11
; CHECK-CVT-NEXT: mov v1.s[1], w14
; CHECK-CVT-NEXT: cmp w2, w8
; CHECK-CVT-NEXT: mov v4.s[1], w9 ; CHECK-CVT-NEXT: mov v4.s[1], w9
; CHECK-CVT-NEXT: csel w9, w11, w8, lo ; CHECK-CVT-NEXT: fcvtzu w9, s0
; CHECK-CVT-NEXT: mov v5.s[1], w18 ; CHECK-CVT-NEXT: csel w11, w2, w8, lo
; CHECK-CVT-NEXT: fcvtzu w11, s0 ; CHECK-CVT-NEXT: mov v3.s[2], w0
; CHECK-CVT-NEXT: mov v3.s[2], w15 ; CHECK-CVT-NEXT: mov v2.s[3], w12
; CHECK-CVT-NEXT: mov v4.s[2], w0 ; CHECK-CVT-NEXT: mov v1.s[2], w15
; CHECK-CVT-NEXT: mov v5.s[2], w9 ; CHECK-CVT-NEXT: mov v4.s[2], w11
; CHECK-CVT-NEXT: cmp w11, w8 ; CHECK-CVT-NEXT: cmp w9, w8
; CHECK-CVT-NEXT: csel w8, w11, w8, lo ; CHECK-CVT-NEXT: csel w8, w9, w8, lo
; CHECK-CVT-NEXT: mov v2.s[3], w13 ; CHECK-CVT-NEXT: mov v3.s[3], w10
; CHECK-CVT-NEXT: mov v3.s[3], w17 ; CHECK-CVT-NEXT: mov v1.s[3], w16
; CHECK-CVT-NEXT: mov v4.s[3], w10 ; CHECK-CVT-NEXT: mov v4.s[3], w8
; CHECK-CVT-NEXT: mov v5.s[3], w8 ; CHECK-CVT-NEXT: uzp1 v0.8h, v1.8h, v2.8h
; CHECK-CVT-NEXT: uzp1 v0.8h, v3.8h, v2.8h ; CHECK-CVT-NEXT: uzp1 v1.8h, v4.8h, v3.8h
; CHECK-CVT-NEXT: uzp1 v1.8h, v5.8h, v4.8h
; CHECK-CVT-NEXT: ret ; CHECK-CVT-NEXT: ret
; ;
; CHECK-FP16-LABEL: test_unsigned_v16f16_v16i16: ; CHECK-FP16-LABEL: test_unsigned_v16f16_v16i16:
Context not available.
; CHECK-LABEL: test_unsigned_v8f64_v8i8: ; CHECK-LABEL: test_unsigned_v8f64_v8i8:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: mov d4, v3.d[1] ; CHECK-NEXT: mov d4, v3.d[1]
; CHECK-NEXT: fcvtzu w10, d3 ; CHECK-NEXT: mov d5, v2.d[1]
; CHECK-NEXT: mov d3, v2.d[1] ; CHECK-NEXT: mov w11, #255 // =0xff
; CHECK-NEXT: mov w8, #255 // =0xff ; CHECK-NEXT: fcvtzu w9, d3
; CHECK-NEXT: mov d3, v1.d[1]
; CHECK-NEXT: fcvtzu w12, d2 ; CHECK-NEXT: fcvtzu w12, d2
; CHECK-NEXT: fcvtzu w13, d1 ; CHECK-NEXT: fcvtzu w14, d1
; CHECK-NEXT: fcvtzu w9, d4 ; CHECK-NEXT: fcvtzu w8, d4
; CHECK-NEXT: mov d4, v1.d[1] ; CHECK-NEXT: mov d4, v0.d[1]
; CHECK-NEXT: fcvtzu w11, d3 ; CHECK-NEXT: fcvtzu w10, d5
; CHECK-NEXT: mov d1, v0.d[1] ; CHECK-NEXT: fcvtzu w13, d3
; CHECK-NEXT: cmp w8, #255
; CHECK-NEXT: fcvtzu w15, d4
; CHECK-NEXT: csel w8, w8, w11, lo
; CHECK-NEXT: cmp w9, #255 ; CHECK-NEXT: cmp w9, #255
; CHECK-NEXT: csel w9, w9, w8, lo ; CHECK-NEXT: csel w9, w9, w11, lo
; CHECK-NEXT: cmp w10, #255 ; CHECK-NEXT: cmp w10, #255
; CHECK-NEXT: csel w10, w10, w8, lo ; CHECK-NEXT: fmov s4, w9
; CHECK-NEXT: cmp w11, #255 ; CHECK-NEXT: csel w9, w10, w11, lo
; CHECK-NEXT: csel w11, w11, w8, lo
; CHECK-NEXT: cmp w12, #255 ; CHECK-NEXT: cmp w12, #255
; CHECK-NEXT: csel w12, w12, w8, lo ; CHECK-NEXT: fcvtzu w10, d0
; CHECK-NEXT: fmov s19, w10 ; CHECK-NEXT: mov v4.s[1], w8
; CHECK-NEXT: fcvtzu w10, d4 ; CHECK-NEXT: csel w8, w12, w11, lo
; CHECK-NEXT: cmp w10, #255
; CHECK-NEXT: mov v19.s[1], w9
; CHECK-NEXT: csel w10, w10, w8, lo
; CHECK-NEXT: cmp w13, #255
; CHECK-NEXT: fmov s18, w12
; CHECK-NEXT: fcvtzu w9, d1
; CHECK-NEXT: csel w12, w13, w8, lo
; CHECK-NEXT: fcvtzu w13, d0
; CHECK-NEXT: mov v18.s[1], w11
; CHECK-NEXT: cmp w9, #255
; CHECK-NEXT: fmov s17, w12
; CHECK-NEXT: csel w9, w9, w8, lo
; CHECK-NEXT: cmp w13, #255 ; CHECK-NEXT: cmp w13, #255
; CHECK-NEXT: csel w8, w13, w8, lo ; CHECK-NEXT: fmov s3, w8
; CHECK-NEXT: mov v17.s[1], w10 ; CHECK-NEXT: csel w8, w13, w11, lo
; CHECK-NEXT: fmov s16, w8 ; CHECK-NEXT: cmp w14, #255
; CHECK-NEXT: mov v3.s[1], w9
; CHECK-NEXT: csel w9, w14, w11, lo
; CHECK-NEXT: cmp w15, #255
; CHECK-NEXT: fmov s2, w9
; CHECK-NEXT: csel w9, w15, w11, lo
; CHECK-NEXT: cmp w10, #255
; CHECK-NEXT: mov v2.s[1], w8
; CHECK-NEXT: csel w8, w10, w11, lo
; CHECK-NEXT: fmov s1, w8
; CHECK-NEXT: adrp x8, .LCPI82_0 ; CHECK-NEXT: adrp x8, .LCPI82_0
; CHECK-NEXT: mov v16.s[1], w9
; CHECK-NEXT: ldr d0, [x8, :lo12:.LCPI82_0] ; CHECK-NEXT: ldr d0, [x8, :lo12:.LCPI82_0]
; CHECK-NEXT: tbl v0.8b, { v16.16b, v17.16b, v18.16b, v19.16b }, v0.8b ; CHECK-NEXT: mov v1.s[1], w9
; CHECK-NEXT: tbl v0.8b, { v1.16b, v2.16b, v3.16b, v4.16b }, v0.8b
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%x = call <8 x i8> @llvm.fptoui.sat.v8f64.v8i8(<8 x double> %f) %x = call <8 x i8> @llvm.fptoui.sat.v8f64.v8i8(<8 x double> %f)
ret <8 x i8> %x ret <8 x i8> %x
Context not available.
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: mov d16, v0.d[1] ; CHECK-NEXT: mov d16, v0.d[1]
; CHECK-NEXT: fcvtzu w10, d0 ; CHECK-NEXT: fcvtzu w10, d0
; CHECK-NEXT: mov d0, v1.d[1]
; CHECK-NEXT: mov w8, #255 // =0xff ; CHECK-NEXT: mov w8, #255 // =0xff
; CHECK-NEXT: fcvtzu w12, d1
; CHECK-NEXT: mov d1, v2.d[1]
; CHECK-NEXT: fcvtzu w9, d16 ; CHECK-NEXT: fcvtzu w9, d16
; CHECK-NEXT: fcvtzu w11, d0 ; CHECK-NEXT: mov d16, v1.d[1]
; CHECK-NEXT: cmp w9, #255 ; CHECK-NEXT: cmp w9, #255
; CHECK-NEXT: csel w9, w9, w8, lo ; CHECK-NEXT: csel w9, w9, w8, lo
; CHECK-NEXT: cmp w10, #255 ; CHECK-NEXT: cmp w10, #255
; CHECK-NEXT: csel w10, w10, w8, lo ; CHECK-NEXT: csel w10, w10, w8, lo
; CHECK-NEXT: cmp w11, #255
; CHECK-NEXT: fmov s0, w10 ; CHECK-NEXT: fmov s0, w10
; CHECK-NEXT: csel w10, w11, w8, lo ; CHECK-NEXT: fcvtzu w10, d16
; CHECK-NEXT: cmp w12, #255 ; CHECK-NEXT: mov d16, v2.d[1]
; CHECK-NEXT: csel w11, w12, w8, lo
; CHECK-NEXT: fcvtzu w12, d2
; CHECK-NEXT: mov v0.s[1], w9 ; CHECK-NEXT: mov v0.s[1], w9
; CHECK-NEXT: fcvtzu w9, d1 ; CHECK-NEXT: fcvtzu w9, d1
; CHECK-NEXT: mov d2, v3.d[1] ; CHECK-NEXT: cmp w10, #255
; CHECK-NEXT: fmov s1, w11 ; CHECK-NEXT: csel w10, w10, w8, lo
; CHECK-NEXT: cmp w9, #255 ; CHECK-NEXT: cmp w9, #255
; CHECK-NEXT: mov w11, v0.s[1] ; CHECK-NEXT: mov w11, v0.s[1]
; CHECK-NEXT: csel w9, w9, w8, lo ; CHECK-NEXT: csel w9, w9, w8, lo
; CHECK-NEXT: cmp w12, #255 ; CHECK-NEXT: fmov s1, w9
; CHECK-NEXT: fcvtzu w9, d16
; CHECK-NEXT: mov d16, v3.d[1]
; CHECK-NEXT: mov v0.b[1], w11
; CHECK-NEXT: mov v1.s[1], w10 ; CHECK-NEXT: mov v1.s[1], w10
; CHECK-NEXT: csel w12, w12, w8, lo
; CHECK-NEXT: fcvtzu w10, d2 ; CHECK-NEXT: fcvtzu w10, d2
; CHECK-NEXT: mov v0.b[1], w11 ; CHECK-NEXT: cmp w9, #255
; CHECK-NEXT: fcvtzu w11, d3 ; CHECK-NEXT: csel w9, w9, w8, lo
; CHECK-NEXT: fmov s2, w12
; CHECK-NEXT: mov w12, v1.s[1]
; CHECK-NEXT: cmp w10, #255 ; CHECK-NEXT: cmp w10, #255
; CHECK-NEXT: mov d3, v4.d[1] ; CHECK-NEXT: mov w11, v1.s[1]
; CHECK-NEXT: csel w10, w10, w8, lo
; CHECK-NEXT: mov v0.b[2], v1.b[0] ; CHECK-NEXT: mov v0.b[2], v1.b[0]
; CHECK-NEXT: cmp w11, #255 ; CHECK-NEXT: csel w10, w10, w8, lo
; CHECK-NEXT: fmov s2, w10
; CHECK-NEXT: fcvtzu w10, d16
; CHECK-NEXT: mov d16, v4.d[1]
; CHECK-NEXT: mov v0.b[3], w11
; CHECK-NEXT: mov v2.s[1], w9 ; CHECK-NEXT: mov v2.s[1], w9
; CHECK-NEXT: csel w11, w11, w8, lo
; CHECK-NEXT: fcvtzu w9, d3 ; CHECK-NEXT: fcvtzu w9, d3
; CHECK-NEXT: mov d3, v5.d[1] ; CHECK-NEXT: cmp w10, #255
; CHECK-NEXT: mov v0.b[3], w12 ; CHECK-NEXT: csel w10, w10, w8, lo
; CHECK-NEXT: fcvtzu w12, d4
; CHECK-NEXT: fmov s4, w11
; CHECK-NEXT: mov w11, v2.s[1]
; CHECK-NEXT: cmp w9, #255 ; CHECK-NEXT: cmp w9, #255
; CHECK-NEXT: csel w9, w9, w8, lo ; CHECK-NEXT: mov w11, v2.s[1]
; CHECK-NEXT: cmp w12, #255
; CHECK-NEXT: mov v0.b[4], v2.b[0] ; CHECK-NEXT: mov v0.b[4], v2.b[0]
; CHECK-NEXT: csel w12, w12, w8, lo ; CHECK-NEXT: csel w9, w9, w8, lo
; CHECK-NEXT: mov v4.s[1], w10 ; CHECK-NEXT: fmov s3, w9
; CHECK-NEXT: fcvtzu w10, d3 ; CHECK-NEXT: fcvtzu w9, d16
; CHECK-NEXT: fmov s3, w12
; CHECK-NEXT: mov v0.b[5], w11 ; CHECK-NEXT: mov v0.b[5], w11
; CHECK-NEXT: fcvtzu w11, d5 ; CHECK-NEXT: mov v3.s[1], w10
; CHECK-NEXT: mov w12, v4.s[1] ; CHECK-NEXT: fcvtzu w10, d4
; CHECK-NEXT: mov d4, v5.d[1]
; CHECK-NEXT: cmp w9, #255
; CHECK-NEXT: csel w9, w9, w8, lo
; CHECK-NEXT: cmp w10, #255 ; CHECK-NEXT: cmp w10, #255
; CHECK-NEXT: csel w10, w10, w8, lo
; CHECK-NEXT: mov d5, v6.d[1]
; CHECK-NEXT: cmp w11, #255
; CHECK-NEXT: mov v0.b[6], v4.b[0]
; CHECK-NEXT: csel w11, w11, w8, lo
; CHECK-NEXT: mov v3.s[1], w9
; CHECK-NEXT: fcvtzu w9, d6
; CHECK-NEXT: mov d6, v7.d[1]
; CHECK-NEXT: mov v0.b[7], w12
; CHECK-NEXT: fcvtzu w12, d5
; CHECK-NEXT: fmov s5, w11
; CHECK-NEXT: mov w11, v3.s[1] ; CHECK-NEXT: mov w11, v3.s[1]
; CHECK-NEXT: cmp w12, #255 ; CHECK-NEXT: mov v0.b[6], v3.b[0]
; CHECK-NEXT: mov v0.b[8], v3.b[0] ; CHECK-NEXT: csel w10, w10, w8, lo
; CHECK-NEXT: csel w12, w12, w8, lo ; CHECK-NEXT: fmov s16, w10
; CHECK-NEXT: fcvtzu w10, d4
; CHECK-NEXT: mov d4, v6.d[1]
; CHECK-NEXT: mov v0.b[7], w11
; CHECK-NEXT: mov v16.s[1], w9
; CHECK-NEXT: fcvtzu w9, d5
; CHECK-NEXT: cmp w10, #255
; CHECK-NEXT: csel w10, w10, w8, lo
; CHECK-NEXT: cmp w9, #255 ; CHECK-NEXT: cmp w9, #255
; CHECK-NEXT: mov v5.s[1], w10 ; CHECK-NEXT: mov w11, v16.s[1]
; CHECK-NEXT: mov v0.b[8], v16.b[0]
; CHECK-NEXT: csel w9, w9, w8, lo ; CHECK-NEXT: csel w9, w9, w8, lo
; CHECK-NEXT: fcvtzu w10, d6 ; CHECK-NEXT: fmov s5, w9
; CHECK-NEXT: fcvtzu w9, d4
; CHECK-NEXT: mov d4, v7.d[1]
; CHECK-NEXT: mov v0.b[9], w11 ; CHECK-NEXT: mov v0.b[9], w11
; CHECK-NEXT: fcvtzu w11, d7 ; CHECK-NEXT: mov v5.s[1], w10
; CHECK-NEXT: fmov s16, w9 ; CHECK-NEXT: fcvtzu w10, d6
; CHECK-NEXT: mov w9, v5.s[1] ; CHECK-NEXT: cmp w9, #255
; CHECK-NEXT: csel w9, w9, w8, lo
; CHECK-NEXT: cmp w10, #255 ; CHECK-NEXT: cmp w10, #255
; CHECK-NEXT: mov v0.b[10], v5.b[0] ; CHECK-NEXT: mov v0.b[10], v5.b[0]
; CHECK-NEXT: mov v16.s[1], w12 ; CHECK-NEXT: mov w11, v5.s[1]
; CHECK-NEXT: mov v0.b[11], w9 ; CHECK-NEXT: csel w10, w10, w8, lo
; CHECK-NEXT: csel w9, w10, w8, lo ; CHECK-NEXT: fmov s6, w10
; CHECK-NEXT: cmp w11, #255 ; CHECK-NEXT: fcvtzu w10, d7
; CHECK-NEXT: mov w10, v16.s[1] ; CHECK-NEXT: mov v0.b[11], w11
; CHECK-NEXT: csel w8, w11, w8, lo
; CHECK-NEXT: mov v0.b[12], v16.b[0]
; CHECK-NEXT: fmov s6, w8
; CHECK-NEXT: mov v0.b[13], w10
; CHECK-NEXT: mov v6.s[1], w9 ; CHECK-NEXT: mov v6.s[1], w9
; CHECK-NEXT: mov v0.b[14], v6.b[0] ; CHECK-NEXT: fcvtzu w9, d4
; CHECK-NEXT: mov w8, v6.s[1] ; CHECK-NEXT: cmp w9, #255
; CHECK-NEXT: mov v0.b[12], v6.b[0]
; CHECK-NEXT: mov w11, v6.s[1]
; CHECK-NEXT: csel w9, w9, w8, lo
; CHECK-NEXT: cmp w10, #255
; CHECK-NEXT: csel w8, w10, w8, lo
; CHECK-NEXT: fmov s4, w8
; CHECK-NEXT: mov v0.b[13], w11
; CHECK-NEXT: mov v4.s[1], w9
; CHECK-NEXT: mov v0.b[14], v4.b[0]
; CHECK-NEXT: mov w8, v4.s[1]
; CHECK-NEXT: mov v0.b[15], w8 ; CHECK-NEXT: mov v0.b[15], w8
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%x = call <16 x i8> @llvm.fptoui.sat.v16f64.v16i8(<16 x double> %f) %x = call <16 x i8> @llvm.fptoui.sat.v16f64.v16i8(<16 x double> %f)
Context not available.
; CHECK-LABEL: test_unsigned_v8f64_v8i16: ; CHECK-LABEL: test_unsigned_v8f64_v8i16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: mov d4, v3.d[1] ; CHECK-NEXT: mov d4, v3.d[1]
; CHECK-NEXT: fcvtzu w10, d3 ; CHECK-NEXT: mov d5, v2.d[1]
; CHECK-NEXT: mov d3, v2.d[1] ; CHECK-NEXT: mov w10, #65535 // =0xffff
; CHECK-NEXT: mov w8, #65535 // =0xffff ; CHECK-NEXT: fcvtzu w9, d3
; CHECK-NEXT: mov d3, v1.d[1]
; CHECK-NEXT: fcvtzu w12, d2 ; CHECK-NEXT: fcvtzu w12, d2
; CHECK-NEXT: fcvtzu w13, d1 ; CHECK-NEXT: fcvtzu w14, d1
; CHECK-NEXT: fcvtzu w9, d4 ; CHECK-NEXT: fcvtzu w8, d4
; CHECK-NEXT: mov d4, v1.d[1] ; CHECK-NEXT: mov d4, v0.d[1]
; CHECK-NEXT: fcvtzu w11, d3 ; CHECK-NEXT: fcvtzu w11, d5
; CHECK-NEXT: mov d1, v0.d[1] ; CHECK-NEXT: fcvtzu w13, d3
; CHECK-NEXT: cmp w9, w8 ; CHECK-NEXT: cmp w8, w10
; CHECK-NEXT: csel w9, w9, w8, lo ; CHECK-NEXT: fcvtzu w15, d4
; CHECK-NEXT: cmp w10, w8 ; CHECK-NEXT: csel w8, w8, w10, lo
; CHECK-NEXT: csel w10, w10, w8, lo ; CHECK-NEXT: cmp w9, w10
; CHECK-NEXT: cmp w11, w8 ; CHECK-NEXT: csel w9, w9, w10, lo
; CHECK-NEXT: csel w11, w11, w8, lo ; CHECK-NEXT: cmp w11, w10
; CHECK-NEXT: cmp w12, w8 ; CHECK-NEXT: fmov s4, w9
; CHECK-NEXT: csel w12, w12, w8, lo ; CHECK-NEXT: csel w9, w11, w10, lo
; CHECK-NEXT: fmov s19, w10 ; CHECK-NEXT: cmp w12, w10
; CHECK-NEXT: fcvtzu w10, d4 ; CHECK-NEXT: fcvtzu w11, d0
; CHECK-NEXT: cmp w10, w8 ; CHECK-NEXT: mov v4.s[1], w8
; CHECK-NEXT: mov v19.s[1], w9 ; CHECK-NEXT: csel w8, w12, w10, lo
; CHECK-NEXT: csel w10, w10, w8, lo ; CHECK-NEXT: cmp w13, w10
; CHECK-NEXT: cmp w13, w8 ; CHECK-NEXT: fmov s3, w8
; CHECK-NEXT: fmov s18, w12 ; CHECK-NEXT: csel w8, w13, w10, lo
; CHECK-NEXT: fcvtzu w9, d1 ; CHECK-NEXT: cmp w14, w10
; CHECK-NEXT: csel w12, w13, w8, lo ; CHECK-NEXT: mov v3.s[1], w9
; CHECK-NEXT: fcvtzu w13, d0 ; CHECK-NEXT: csel w9, w14, w10, lo
; CHECK-NEXT: mov v18.s[1], w11 ; CHECK-NEXT: cmp w15, w10
; CHECK-NEXT: cmp w9, w8 ; CHECK-NEXT: fmov s2, w9
; CHECK-NEXT: fmov s17, w12 ; CHECK-NEXT: csel w9, w15, w10, lo
; CHECK-NEXT: csel w9, w9, w8, lo ; CHECK-NEXT: cmp w11, w10
; CHECK-NEXT: cmp w13, w8 ; CHECK-NEXT: mov v2.s[1], w8
; CHECK-NEXT: csel w8, w13, w8, lo ; CHECK-NEXT: csel w8, w11, w10, lo
; CHECK-NEXT: mov v17.s[1], w10 ; CHECK-NEXT: fmov s1, w8
; CHECK-NEXT: fmov s16, w8
; CHECK-NEXT: adrp x8, .LCPI84_0 ; CHECK-NEXT: adrp x8, .LCPI84_0
; CHECK-NEXT: mov v16.s[1], w9
; CHECK-NEXT: ldr q0, [x8, :lo12:.LCPI84_0] ; CHECK-NEXT: ldr q0, [x8, :lo12:.LCPI84_0]
; CHECK-NEXT: tbl v0.16b, { v16.16b, v17.16b, v18.16b, v19.16b }, v0.16b ; CHECK-NEXT: mov v1.s[1], w9
; CHECK-NEXT: tbl v0.16b, { v1.16b, v2.16b, v3.16b, v4.16b }, v0.16b
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%x = call <8 x i16> @llvm.fptoui.sat.v8f64.v8i16(<8 x double> %f) %x = call <8 x i16> @llvm.fptoui.sat.v8f64.v8i16(<8 x double> %f)
ret <8 x i16> %x ret <8 x i16> %x
Context not available.
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: mov d16, v3.d[1] ; CHECK-NEXT: mov d16, v3.d[1]
; CHECK-NEXT: mov d17, v2.d[1] ; CHECK-NEXT: mov d17, v2.d[1]
; CHECK-NEXT: mov w8, #65535 // =0xffff
; CHECK-NEXT: fcvtzu w9, d3 ; CHECK-NEXT: fcvtzu w9, d3
; CHECK-NEXT: mov d3, v1.d[1] ; CHECK-NEXT: mov d3, v1.d[1]
; CHECK-NEXT: mov w8, #65535 // =0xffff ; CHECK-NEXT: fcvtzu w11, d1
; CHECK-NEXT: fcvtzu w10, d2
; CHECK-NEXT: fcvtzu w12, d1
; CHECK-NEXT: mov d1, v0.d[1] ; CHECK-NEXT: mov d1, v0.d[1]
; CHECK-NEXT: fcvtzu w11, d16 ; CHECK-NEXT: fcvtzu w10, d2
; CHECK-NEXT: fcvtzu w13, d17 ; CHECK-NEXT: fcvtzu w13, d0
; CHECK-NEXT: fcvtzu w14, d3 ; CHECK-NEXT: mov d0, v7.d[1]
; CHECK-NEXT: mov d2, v7.d[1] ; CHECK-NEXT: mov d2, v6.d[1]
; CHECK-NEXT: fcvtzu w17, d6 ; CHECK-NEXT: fcvtzu w15, d7
; CHECK-NEXT: cmp w11, w8 ; CHECK-NEXT: fcvtzu w12, d16
; CHECK-NEXT: fcvtzu w15, d1 ; CHECK-NEXT: fcvtzu w14, d17
; CHECK-NEXT: csel w11, w11, w8, lo ; CHECK-NEXT: fcvtzu w16, d6
; CHECK-NEXT: fcvtzu w17, d3
; CHECK-NEXT: mov d6, v5.d[1]
; CHECK-NEXT: mov d3, v4.d[1]
; CHECK-NEXT: fcvtzu w18, d1
; CHECK-NEXT: cmp w12, w8
; CHECK-NEXT: csel w12, w12, w8, lo
; CHECK-NEXT: cmp w9, w8 ; CHECK-NEXT: cmp w9, w8
; CHECK-NEXT: csel w9, w9, w8, lo ; CHECK-NEXT: csel w9, w9, w8, lo
; CHECK-NEXT: cmp w13, w8
; CHECK-NEXT: csel w13, w13, w8, lo
; CHECK-NEXT: cmp w10, w8
; CHECK-NEXT: csel w10, w10, w8, lo
; CHECK-NEXT: cmp w14, w8 ; CHECK-NEXT: cmp w14, w8
; CHECK-NEXT: fmov s19, w9 ; CHECK-NEXT: fmov s19, w9
; CHECK-NEXT: csel w9, w14, w8, lo ; CHECK-NEXT: csel w9, w14, w8, lo
; CHECK-NEXT: cmp w10, w8
; CHECK-NEXT: fcvtzu w14, d0 ; CHECK-NEXT: fcvtzu w14, d0
; CHECK-NEXT: cmp w12, w8 ; CHECK-NEXT: csel w10, w10, w8, lo
; CHECK-NEXT: fcvtzu w16, d2 ; CHECK-NEXT: cmp w17, w8
; CHECK-NEXT: mov d0, v6.d[1] ; CHECK-NEXT: mov v19.s[1], w12
; CHECK-NEXT: csel w12, w12, w8, lo ; CHECK-NEXT: csel w12, w17, w8, lo
; CHECK-NEXT: cmp w15, w8
; CHECK-NEXT: mov v19.s[1], w11
; CHECK-NEXT: fcvtzu w11, d7
; CHECK-NEXT: fmov s18, w10
; CHECK-NEXT: csel w10, w15, w8, lo
; CHECK-NEXT: cmp w14, w8
; CHECK-NEXT: csel w14, w14, w8, lo
; CHECK-NEXT: cmp w16, w8
; CHECK-NEXT: csel w15, w16, w8, lo
; CHECK-NEXT: cmp w11, w8 ; CHECK-NEXT: cmp w11, w8
; CHECK-NEXT: fcvtzu w16, d0
; CHECK-NEXT: mov d0, v5.d[1]
; CHECK-NEXT: csel w11, w11, w8, lo ; CHECK-NEXT: csel w11, w11, w8, lo
; CHECK-NEXT: mov v18.s[1], w13 ; CHECK-NEXT: cmp w18, w8
; CHECK-NEXT: cmp w16, w8 ; CHECK-NEXT: fmov s18, w10
; CHECK-NEXT: fmov s17, w12 ; CHECK-NEXT: csel w10, w18, w8, lo
; CHECK-NEXT: cmp w13, w8
; CHECK-NEXT: fcvtzu w17, d2
; CHECK-NEXT: csel w13, w13, w8, lo
; CHECK-NEXT: cmp w14, w8
; CHECK-NEXT: fcvtzu w18, d6
; CHECK-NEXT: mov v18.s[1], w9
; CHECK-NEXT: csel w9, w14, w8, lo
; CHECK-NEXT: cmp w15, w8
; CHECK-NEXT: fmov s17, w11
; CHECK-NEXT: csel w11, w15, w8, lo
; CHECK-NEXT: fcvtzu w14, d5
; CHECK-NEXT: fmov s23, w11 ; CHECK-NEXT: fmov s23, w11
; CHECK-NEXT: csel w11, w16, w8, lo
; CHECK-NEXT: cmp w17, w8 ; CHECK-NEXT: cmp w17, w8
; CHECK-NEXT: fcvtzu w16, d0 ; CHECK-NEXT: fcvtzu w15, d3
; CHECK-NEXT: mov d0, v4.d[1] ; CHECK-NEXT: csel w11, w17, w8, lo
; CHECK-NEXT: csel w13, w17, w8, lo
; CHECK-NEXT: fcvtzu w17, d5
; CHECK-NEXT: fcvtzu w12, d4
; CHECK-NEXT: mov v23.s[1], w15
; CHECK-NEXT: cmp w16, w8 ; CHECK-NEXT: cmp w16, w8
; CHECK-NEXT: fmov s22, w13 ; CHECK-NEXT: fcvtzu w17, d4
; CHECK-NEXT: csel w13, w16, w8, lo ; CHECK-NEXT: mov v17.s[1], w12
; CHECK-NEXT: cmp w17, w8 ; CHECK-NEXT: mov v23.s[1], w9
; CHECK-NEXT: fcvtzu w16, d0 ; CHECK-NEXT: csel w9, w16, w8, lo
; CHECK-NEXT: csel w15, w17, w8, lo ; CHECK-NEXT: cmp w18, w8
; CHECK-NEXT: fmov s22, w9
; CHECK-NEXT: csel w9, w18, w8, lo
; CHECK-NEXT: cmp w14, w8
; CHECK-NEXT: fmov s16, w13
; CHECK-NEXT: mov v22.s[1], w11 ; CHECK-NEXT: mov v22.s[1], w11
; CHECK-NEXT: cmp w16, w8 ; CHECK-NEXT: csel w11, w14, w8, lo
; CHECK-NEXT: fmov s21, w15 ; CHECK-NEXT: cmp w15, w8
; CHECK-NEXT: csel w11, w16, w8, lo ; CHECK-NEXT: fmov s21, w11
; CHECK-NEXT: cmp w12, w8 ; CHECK-NEXT: csel w11, w15, w8, lo
; CHECK-NEXT: csel w8, w12, w8, lo ; CHECK-NEXT: cmp w17, w8
; CHECK-NEXT: mov v17.s[1], w9 ; CHECK-NEXT: csel w8, w17, w8, lo
; CHECK-NEXT: adrp x9, .LCPI85_0
; CHECK-NEXT: mov v21.s[1], w13
; CHECK-NEXT: fmov s16, w14
; CHECK-NEXT: fmov s20, w8
; CHECK-NEXT: ldr q1, [x9, :lo12:.LCPI85_0]
; CHECK-NEXT: mov v16.s[1], w10 ; CHECK-NEXT: mov v16.s[1], w10
; CHECK-NEXT: mov v21.s[1], w9
; CHECK-NEXT: fmov s20, w8
; CHECK-NEXT: adrp x8, .LCPI85_0
; CHECK-NEXT: ldr q1, [x8, :lo12:.LCPI85_0]
; CHECK-NEXT: mov v20.s[1], w11 ; CHECK-NEXT: mov v20.s[1], w11
; CHECK-NEXT: tbl v0.16b, { v16.16b, v17.16b, v18.16b, v19.16b }, v1.16b ; CHECK-NEXT: tbl v0.16b, { v16.16b, v17.16b, v18.16b, v19.16b }, v1.16b
; CHECK-NEXT: tbl v1.16b, { v20.16b, v21.16b, v22.16b, v23.16b }, v1.16b ; CHECK-NEXT: tbl v1.16b, { v20.16b, v21.16b, v22.16b, v23.16b }, v1.16b
Context not available.

llvm/test/CodeGen/AArch64/fptrunc.ll

llvm/test/CodeGen/AArch64/funnel-shift-rot.ll

llvm/test/CodeGen/AArch64/funnel-shift.ll

llvm/test/CodeGen/AArch64/global-merge-3.ll

llvm/test/CodeGen/AArch64/gpr_cttz.ll

llvm/test/CodeGen/AArch64/half.ll

llvm/test/CodeGen/AArch64/highextractbitcast.ll

llvm/test/CodeGen/AArch64/hoist-and-by-const-from-lshr-in-eqcmp-zero.ll

llvm/test/CodeGen/AArch64/i128-math.ll

llvm/test/CodeGen/AArch64/i128_volatile_load_store.ll

llvm/test/CodeGen/AArch64/illegal-floating-point-vector-compares.ll

llvm/test/CodeGen/AArch64/insert-extend.ll

llvm/test/CodeGen/AArch64/insert-subvector-res-legalization.ll

llvm/test/CodeGen/AArch64/insert-subvector.ll

llvm/test/CodeGen/AArch64/insertshuffleload.ll

llvm/test/CodeGen/AArch64/isinf.ll

llvm/test/CodeGen/AArch64/known-never-nan.ll

llvm/test/CodeGen/AArch64/llvm-ir-to-intrinsic.ll

llvm/test/CodeGen/AArch64/load-insert-zero.ll

llvm/test/CodeGen/AArch64/logic-reassociate.ll

llvm/test/CodeGen/AArch64/logic-shift.ll

llvm/test/CodeGen/AArch64/logical_shifted_reg.ll

Context not available.
; CHECK-NEXT: ldr w9, [x9] ; CHECK-NEXT: ldr w9, [x9]
; CHECK-NEXT: and w11, w10, w9 ; CHECK-NEXT: and w11, w10, w9
; CHECK-NEXT: bic w12, w10, w9 ; CHECK-NEXT: bic w12, w10, w9
; CHECK-NEXT: orr w13, w10, w9
; CHECK-NEXT: str w11, [x8] ; CHECK-NEXT: str w11, [x8]
; CHECK-NEXT: orn w11, w10, w9 ; CHECK-NEXT: orr w11, w10, w9
; CHECK-NEXT: str w12, [x8] ; CHECK-NEXT: str w12, [x8]
; CHECK-NEXT: eor w12, w10, w9 ; CHECK-NEXT: orn w12, w10, w9
; CHECK-NEXT: str w13, [x8] ; CHECK-NEXT: str w11, [x8]
; CHECK-NEXT: eon w13, w9, w10 ; CHECK-NEXT: eor w11, w10, w9
; CHECK-NEXT: str w12, [x8]
; CHECK-NEXT: eon w12, w9, w10
; CHECK-NEXT: str w11, [x8] ; CHECK-NEXT: str w11, [x8]
; CHECK-NEXT: and w11, w10, w9, lsl #31 ; CHECK-NEXT: and w11, w10, w9, lsl #31
; CHECK-NEXT: str w12, [x8] ; CHECK-NEXT: str w12, [x8]
; CHECK-NEXT: bic w12, w10, w9, lsl #31 ; CHECK-NEXT: bic w12, w10, w9, lsl #31
; CHECK-NEXT: str w13, [x8]
; CHECK-NEXT: orr w13, w10, w9, lsl #31
; CHECK-NEXT: str w11, [x8] ; CHECK-NEXT: str w11, [x8]
; CHECK-NEXT: orn w11, w10, w9, lsl #31 ; CHECK-NEXT: orr w11, w10, w9, lsl #31
; CHECK-NEXT: str w12, [x8]
; CHECK-NEXT: orn w12, w10, w9, lsl #31
; CHECK-NEXT: str w11, [x8]
; CHECK-NEXT: eor w11, w10, w9, lsl #31
; CHECK-NEXT: str w12, [x8] ; CHECK-NEXT: str w12, [x8]
; CHECK-NEXT: eor w12, w10, w9, lsl #31 ; CHECK-NEXT: eon w12, w10, w9, lsl #31
; CHECK-NEXT: str w13, [x8]
; CHECK-NEXT: eon w13, w10, w9, lsl #31
; CHECK-NEXT: str w11, [x8] ; CHECK-NEXT: str w11, [x8]
; CHECK-NEXT: bic w11, w10, w9, asr #10 ; CHECK-NEXT: bic w11, w10, w9, asr #10
; CHECK-NEXT: str w12, [x8] ; CHECK-NEXT: str w12, [x8]
; CHECK-NEXT: eor w12, w10, w9, asr #10 ; CHECK-NEXT: eor w12, w10, w9, asr #10
; CHECK-NEXT: str w13, [x8]
; CHECK-NEXT: orn w13, w10, w9, lsr #1
; CHECK-NEXT: str w11, [x8] ; CHECK-NEXT: str w11, [x8]
; CHECK-NEXT: eor w11, w10, w9, lsr #1 ; CHECK-NEXT: orn w11, w10, w9, lsr #1
; CHECK-NEXT: str w12, [x8] ; CHECK-NEXT: str w12, [x8]
; CHECK-NEXT: eon w12, w10, w9, ror #20 ; CHECK-NEXT: eor w12, w10, w9, lsr #1
; CHECK-NEXT: and w9, w10, w9, ror #20
; CHECK-NEXT: str w13, [x8]
; CHECK-NEXT: str w11, [x8] ; CHECK-NEXT: str w11, [x8]
; CHECK-NEXT: eon w11, w10, w9, ror #20
; CHECK-NEXT: and w9, w10, w9, ror #20
; CHECK-NEXT: str w12, [x8] ; CHECK-NEXT: str w12, [x8]
; CHECK-NEXT: str w11, [x8]
; CHECK-NEXT: str w9, [x8] ; CHECK-NEXT: str w9, [x8]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%val1 = load i32, ptr @var1_32 %val1 = load i32, ptr @var1_32
Context not available.
; CHECK-NEXT: ldr x9, [x9] ; CHECK-NEXT: ldr x9, [x9]
; CHECK-NEXT: and x11, x10, x9 ; CHECK-NEXT: and x11, x10, x9
; CHECK-NEXT: bic x12, x10, x9 ; CHECK-NEXT: bic x12, x10, x9
; CHECK-NEXT: orr x13, x10, x9
; CHECK-NEXT: str x11, [x8] ; CHECK-NEXT: str x11, [x8]
; CHECK-NEXT: orn x11, x10, x9 ; CHECK-NEXT: orr x11, x10, x9
; CHECK-NEXT: str x12, [x8] ; CHECK-NEXT: str x12, [x8]
; CHECK-NEXT: eor x12, x10, x9 ; CHECK-NEXT: orn x12, x10, x9
; CHECK-NEXT: str x13, [x8] ; CHECK-NEXT: str x11, [x8]
; CHECK-NEXT: eon x13, x9, x10 ; CHECK-NEXT: eor x11, x10, x9
; CHECK-NEXT: str x12, [x8]
; CHECK-NEXT: eon x12, x9, x10
; CHECK-NEXT: str x11, [x8] ; CHECK-NEXT: str x11, [x8]
; CHECK-NEXT: and x11, x10, x9, lsl #63 ; CHECK-NEXT: and x11, x10, x9, lsl #63
; CHECK-NEXT: str x12, [x8] ; CHECK-NEXT: str x12, [x8]
; CHECK-NEXT: bic x12, x10, x9, lsl #63 ; CHECK-NEXT: bic x12, x10, x9, lsl #63
; CHECK-NEXT: str x13, [x8]
; CHECK-NEXT: orr x13, x10, x9, lsl #63
; CHECK-NEXT: str x11, [x8] ; CHECK-NEXT: str x11, [x8]
; CHECK-NEXT: orn x11, x10, x9, lsl #63 ; CHECK-NEXT: orr x11, x10, x9, lsl #63
; CHECK-NEXT: str x12, [x8]
; CHECK-NEXT: orn x12, x10, x9, lsl #63
; CHECK-NEXT: str x11, [x8]
; CHECK-NEXT: eor x11, x10, x9, lsl #63
; CHECK-NEXT: str x12, [x8] ; CHECK-NEXT: str x12, [x8]
; CHECK-NEXT: eor x12, x10, x9, lsl #63 ; CHECK-NEXT: eon x12, x10, x9, lsl #63
; CHECK-NEXT: str x13, [x8]
; CHECK-NEXT: eon x13, x10, x9, lsl #63
; CHECK-NEXT: str x11, [x8] ; CHECK-NEXT: str x11, [x8]
; CHECK-NEXT: bic x11, x10, x9, asr #10 ; CHECK-NEXT: bic x11, x10, x9, asr #10
; CHECK-NEXT: str x12, [x8] ; CHECK-NEXT: str x12, [x8]
; CHECK-NEXT: eor x12, x10, x9, asr #10 ; CHECK-NEXT: eor x12, x10, x9, asr #10
; CHECK-NEXT: str x13, [x8]
; CHECK-NEXT: orn x13, x10, x9, lsr #1
; CHECK-NEXT: str x11, [x8] ; CHECK-NEXT: str x11, [x8]
; CHECK-NEXT: eor x11, x10, x9, lsr #1 ; CHECK-NEXT: orn x11, x10, x9, lsr #1
; CHECK-NEXT: str x12, [x8] ; CHECK-NEXT: str x12, [x8]
; CHECK-NEXT: eon x12, x10, x9, ror #20 ; CHECK-NEXT: eor x12, x10, x9, lsr #1
; CHECK-NEXT: and x9, x10, x9, ror #20
; CHECK-NEXT: str x13, [x8]
; CHECK-NEXT: str x11, [x8] ; CHECK-NEXT: str x11, [x8]
; CHECK-NEXT: eon x11, x10, x9, ror #20
; CHECK-NEXT: and x9, x10, x9, ror #20
; CHECK-NEXT: str x12, [x8] ; CHECK-NEXT: str x12, [x8]
; CHECK-NEXT: str x11, [x8]
; CHECK-NEXT: str x9, [x8] ; CHECK-NEXT: str x9, [x8]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%val1 = load i64, ptr @var1_64 %val1 = load i64, ptr @var1_64
Context not available.
; CHECK-NEXT: ldr x9, [x8] ; CHECK-NEXT: ldr x9, [x8]
; CHECK-NEXT: ldr x10, [x10] ; CHECK-NEXT: ldr x10, [x10]
; CHECK-NEXT: tst x9, x10 ; CHECK-NEXT: tst x9, x10
; CHECK-NEXT: b.gt .LBB2_2 ; CHECK-NEXT: b.gt .LBB2_4
; CHECK-NEXT: // %bb.1: // %test2 ; CHECK-NEXT: // %bb.1: // %test2
; CHECK-NEXT: tst x9, x10, lsl #63 ; CHECK-NEXT: tst x9, x10, lsl #63
; CHECK-NEXT: b.lt .LBB2_4
; CHECK-NEXT: // %bb.2: // %test3
; CHECK-NEXT: and x10, x9, x10, asr #12 ; CHECK-NEXT: and x10, x9, x10, asr #12
; CHECK-NEXT: ccmp x10, #1, #0, ge ; CHECK-NEXT: cmp x10, #1
; CHECK-NEXT: b.lt .LBB2_3 ; CHECK-NEXT: b.ge .LBB2_4
; CHECK-NEXT: .LBB2_2: // %common.ret ; CHECK-NEXT: // %bb.3: // %other_exit
dmgreenUnsubmitted
Done
ReplyInline Actions

This looks like it no longer decides to ifcvt. Given the test that sound fine I think.

dmgreen: This looks like it no longer decides to ifcvt. Given the test that sound fine I think.
; CHECK-NEXT: ret
; CHECK-NEXT: .LBB2_3: // %other_exit
; CHECK-NEXT: str x9, [x8] ; CHECK-NEXT: str x9, [x8]
; CHECK-NEXT: .LBB2_4: // %common.ret
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%val1 = load i64, ptr @var1_64 %val1 = load i64, ptr @var1_64
%val2 = load i64, ptr @var2_64 %val2 = load i64, ptr @var2_64
Context not available.

llvm/test/CodeGen/AArch64/lowerMUL-newload.ll

llvm/test/CodeGen/AArch64/machine-combiner-copy.ll

llvm/test/CodeGen/AArch64/machine-combiner-subadd.ll

llvm/test/CodeGen/AArch64/machine-combiner-transient.ll

llvm/test/CodeGen/AArch64/machine-licm-sink-instr.ll

llvm/test/CodeGen/AArch64/machine-licm-sub-loop.ll

llvm/test/CodeGen/AArch64/machine_cse_impdef_killflags.ll

llvm/test/CodeGen/AArch64/madd-combiner.ll

llvm/test/CodeGen/AArch64/memcpy-scoped-aa.ll

Context not available.
; CHECK-LABEL: test_memcpy: ; CHECK-LABEL: test_memcpy:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: ldp w9, w10, [x1] ; CHECK-NEXT: ldp w9, w10, [x1]
; CHECK-NEXT: ldr q0, [x0, #16]
; CHECK-NEXT: mov x8, x0 ; CHECK-NEXT: mov x8, x0
; CHECK-NEXT: add w0, w9, w10 ; CHECK-NEXT: add w0, w9, w10
; CHECK-NEXT: ldr q0, [x8, #16]
; CHECK-NEXT: str q0, [x8] ; CHECK-NEXT: str q0, [x8]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%p0 = bitcast i32* %p to i8* %p0 = bitcast i32* %p to i8*
Context not available.
; CHECK-LABEL: test_memcpy_inline: ; CHECK-LABEL: test_memcpy_inline:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: ldp w9, w10, [x1] ; CHECK-NEXT: ldp w9, w10, [x1]
; CHECK-NEXT: ldr q0, [x0, #16]
; CHECK-NEXT: mov x8, x0 ; CHECK-NEXT: mov x8, x0
; CHECK-NEXT: add w0, w9, w10 ; CHECK-NEXT: add w0, w9, w10
; CHECK-NEXT: ldr q0, [x8, #16]
; CHECK-NEXT: str q0, [x8] ; CHECK-NEXT: str q0, [x8]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%p0 = bitcast i32* %p to i8* %p0 = bitcast i32* %p to i8*
Context not available.
; CHECK-LABEL: test_memmove: ; CHECK-LABEL: test_memmove:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: ldp w9, w10, [x1] ; CHECK-NEXT: ldp w9, w10, [x1]
; CHECK-NEXT: ldr q0, [x0, #16]
; CHECK-NEXT: mov x8, x0 ; CHECK-NEXT: mov x8, x0
; CHECK-NEXT: add w0, w9, w10 ; CHECK-NEXT: add w0, w9, w10
; CHECK-NEXT: ldr q0, [x8, #16]
; CHECK-NEXT: str q0, [x8] ; CHECK-NEXT: str q0, [x8]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%p0 = bitcast i32* %p to i8* %p0 = bitcast i32* %p to i8*
Context not available.
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: ldp w10, w11, [x1] ; CHECK-NEXT: ldp w10, w11, [x1]
; CHECK-NEXT: mov x8, x0 ; CHECK-NEXT: mov x8, x0
; CHECK-NEXT: mov x9, #-6148914691236517206 ; CHECK-NEXT: mov x9, #-6148914691236517206 // =0xaaaaaaaaaaaaaaaa
; CHECK-NEXT: add w0, w10, w11
; CHECK-NEXT: stp x9, x9, [x8] ; CHECK-NEXT: stp x9, x9, [x8]
; CHECK-NEXT: add w0, w10, w11
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%p0 = bitcast i32* %p to i8* %p0 = bitcast i32* %p to i8*
tail call void @llvm.memset.p0i8.i64(i8* noundef nonnull align 4 dereferenceable(16) %p0, i8 170, i64 16, i1 false), !alias.scope !2, !noalias !4 tail call void @llvm.memset.p0i8.i64(i8* noundef nonnull align 4 dereferenceable(16) %p0, i8 170, i64 16, i1 false), !alias.scope !2, !noalias !4
Context not available.
; CHECK-LABEL: test_mempcpy: ; CHECK-LABEL: test_mempcpy:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: ldp w9, w10, [x1] ; CHECK-NEXT: ldp w9, w10, [x1]
; CHECK-NEXT: ldr q0, [x0, #16]
; CHECK-NEXT: mov x8, x0 ; CHECK-NEXT: mov x8, x0
; CHECK-NEXT: add w0, w9, w10 ; CHECK-NEXT: add w0, w9, w10
; CHECK-NEXT: ldr q0, [x8, #16]
; CHECK-NEXT: str q0, [x8] ; CHECK-NEXT: str q0, [x8]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%p0 = bitcast i32* %p to i8* %p0 = bitcast i32* %p to i8*
Context not available.
dmgreenUnsubmitted
Done
ReplyInline Actions

Things like this can be removed too, so long as there are check lines elsewhere.

dmgreen: Things like this can be removed too, so long as there are check lines elsewhere.

llvm/test/CodeGen/AArch64/merge-trunc-store.ll

llvm/test/CodeGen/AArch64/midpoint-int.ll

llvm/test/CodeGen/AArch64/minmax-of-minmax.ll

llvm/test/CodeGen/AArch64/minmax.ll

llvm/test/CodeGen/AArch64/misched-detail-resource-booking-01.mir

Context not available.
source_filename = "../llvm-project/llvm/test/CodeGen/AArch64/aarch64-smull.failing.ll" source_filename = "../llvm-project/llvm/test/CodeGen/AArch64/aarch64-smull.failing.ll"
target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128" target datalayout = "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128"
target triple = "aarch64-none-linux-gnu" target triple = "aarch64-none-linux-gnu"
define <8 x i32> @umull_and_v8i32(<8 x i16> %src1, <8 x i32> %src2) #0 { define <8 x i32> @umull_and_v8i32(<8 x i16> %src1, <8 x i32> %src2) #0 {
entry: entry:
%in1 = zext <8 x i16> %src1 to <8 x i32> %in1 = zext <8 x i16> %src1 to <8 x i32>
Context not available.
%out = mul nsw <8 x i32> %in1, %in2 %out = mul nsw <8 x i32> %in1, %in2
ret <8 x i32> %out ret <8 x i32> %out
} }
attributes #0 = { "target-features"="+neon" } attributes #0 = { "target-features"="+neon" }
... ...
--- ---
name: umull_and_v8i32 name: umull_and_v8i32
dmgreenUnsubmitted
Done
ReplyInline Actions

This file can specify -mcpu=cortex-a55 to keep the test as it was before.

dmgreen: This file can specify -mcpu=cortex-a55 to keep the test as it was before.
alignment: 4 alignment: 4
exposesReturnsTwice: false
legalized: false
regBankSelected: false
selected: false
failedISel: false
tracksRegLiveness: true tracksRegLiveness: true
hasWinCFI: false
callsEHReturn: false
callsUnwindInit: false
hasEHCatchret: false
hasEHScopes: false
hasEHFunclets: false
isOutlined: false
debugInstrRef: false
failsVerification: false
tracksDebugUserValues: false
registers: registers:
- { id: 0, class: fpr128 } - { id: 0, class: fpr128, preferred-register: '' }
- { id: 1, class: fpr128 } - { id: 1, class: fpr128, preferred-register: '' }
- { id: 2, class: fpr128 } - { id: 2, class: fpr128, preferred-register: '' }
- { id: 3, class: fpr128 } - { id: 3, class: fpr128, preferred-register: '' }
- { id: 4, class: fpr64 } - { id: 4, class: fpr64, preferred-register: '' }
- { id: 5, class: fpr64 } - { id: 5, class: fpr64, preferred-register: '' }
- { id: 6, class: fpr128 } - { id: 6, class: fpr128, preferred-register: '' }
- { id: 7, class: fpr128 } - { id: 7, class: fpr128, preferred-register: '' }
- { id: 8, class: fpr128 } - { id: 8, class: fpr128, preferred-register: '' }
- { id: 9, class: fpr64 } - { id: 9, class: fpr64, preferred-register: '' }
- { id: 10, class: fpr128 } - { id: 10, class: fpr128, preferred-register: '' }
- { id: 11, class: fpr64 } - { id: 11, class: fpr64, preferred-register: '' }
- { id: 12, class: fpr128 } - { id: 12, class: fpr128, preferred-register: '' }
liveins: liveins:
- { reg: '$q0', virtual-reg: '%0' } - { reg: '$q0', virtual-reg: '%0' }
- { reg: '$q1', virtual-reg: '%1' } - { reg: '$q1', virtual-reg: '%1' }
- { reg: '$q2', virtual-reg: '%2' } - { reg: '$q2', virtual-reg: '%2' }
frameInfo: frameInfo:
isFrameAddressTaken: false
isReturnAddressTaken: false
hasStackMap: false
hasPatchPoint: false
stackSize: 0
offsetAdjustment: 0
maxAlignment: 1 maxAlignment: 1
adjustsStack: false
hasCalls: false
stackProtector: ''
functionContext: ''
maxCallFrameSize: 0 maxCallFrameSize: 0
cvBytesOfCalleeSavedRegisters: 0
hasOpaqueSPAdjustment: false
hasVAStart: false
hasMustTailInVarArgFunc: false
hasTailCall: false
localFrameSize: 0
savePoint: ''
restorePoint: ''
fixedStack: []
stack: []
entry_values: []
callSites: []
debugValueSubstitutions: []
constants: []
machineFunctionInfo: {} machineFunctionInfo: {}
body: | body: |
bb.0.entry: bb.0.entry:
liveins: $q0, $q1, $q2 liveins: $q0, $q1, $q2
%6:fpr128 = MOVIv2d_ns 17
%2:fpr128 = COPY $q2 %2:fpr128 = COPY $q2
%1:fpr128 = COPY $q1 %1:fpr128 = COPY $q1
%0:fpr128 = COPY $q0
%3:fpr128 = EXTv16i8 %0, %0, 8
%6:fpr128 = MOVIv2d_ns 17
%7:fpr128 = ANDv16i8 %2, %6 %7:fpr128 = ANDv16i8 %2, %6
%8:fpr128 = ANDv16i8 %1, %6 %8:fpr128 = ANDv16i8 %1, %6
%0:fpr128 = COPY $q0
%3:fpr128 = EXTv16i8 %0, %0, 8
%9:fpr64 = XTNv4i16 %8 %9:fpr64 = XTNv4i16 %8
%10:fpr128 = UMULLv4i16_v4i32 %0.dsub, %9
%11:fpr64 = XTNv4i16 %7 %11:fpr64 = XTNv4i16 %7
%10:fpr128 = UMULLv4i16_v4i32 %0.dsub, %9
%12:fpr128 = UMULLv4i16_v4i32 %3.dsub, %11 %12:fpr128 = UMULLv4i16_v4i32 %3.dsub, %11
$q0 = COPY %10 $q0 = COPY %10
$q1 = COPY %12 $q1 = COPY %12
Context not available.
# CHECK-LABEL: Before MISched: # CHECK-LABEL: Before MISched:
# CHECK-NEXT: # Machine code for function umull_and_v8i32: IsSSA, NoPHIs, TracksLiveness # CHECK-NEXT: # Machine code for function umull_and_v8i32: IsSSA, NoPHIs, TracksLiveness
# CHECK-NEXT: Function Live Ins: $q0 in %0, $q1 in %1, $q2 in %2 # CHECK-NEXT: Function Live Ins: $q0 in %0, $q1 in %1, $q2 in %2
# CHECK-EMPTY: # CHECK-EMPTY:
# CHECK-NEXT: bb.0.entry: # CHECK-NEXT: bb.0.entry:
# CHECK-NEXT: liveins: $q0, $q1, $q2 # CHECK-NEXT: liveins: $q0, $q1, $q2
# CHECK-NEXT: %6:fpr128 = MOVIv2d_ns 17
# CHECK-NEXT: %2:fpr128 = COPY $q2 # CHECK-NEXT: %2:fpr128 = COPY $q2
# CHECK-NEXT: %1:fpr128 = COPY $q1 # CHECK-NEXT: %1:fpr128 = COPY $q1
# CHECK-NEXT: %0:fpr128 = COPY $q0
# CHECK-NEXT: %3:fpr128 = EXTv16i8 %0:fpr128, %0:fpr128, 8
# CHECK-NEXT: %6:fpr128 = MOVIv2d_ns 17
# CHECK-NEXT: %7:fpr128 = ANDv16i8 %2:fpr128, %6:fpr128 # CHECK-NEXT: %7:fpr128 = ANDv16i8 %2:fpr128, %6:fpr128
# CHECK-NEXT: %8:fpr128 = ANDv16i8 %1:fpr128, %6:fpr128 # CHECK-NEXT: %8:fpr128 = ANDv16i8 %1:fpr128, %6:fpr128
# CHECK-NEXT: %0:fpr128 = COPY $q0
# CHECK-NEXT: %3:fpr128 = EXTv16i8 %0:fpr128, %0:fpr128, 8
# CHECK-NEXT: %9:fpr64 = XTNv4i16 %8:fpr128 # CHECK-NEXT: %9:fpr64 = XTNv4i16 %8:fpr128
# CHECK-NEXT: %10:fpr128 = UMULLv4i16_v4i32 %0.dsub:fpr128, %9:fpr64
# CHECK-NEXT: %11:fpr64 = XTNv4i16 %7:fpr128 # CHECK-NEXT: %11:fpr64 = XTNv4i16 %7:fpr128
# CHECK-NEXT: %10:fpr128 = UMULLv4i16_v4i32 %0.dsub:fpr128, %9:fpr64
# CHECK-NEXT: %12:fpr128 = UMULLv4i16_v4i32 %3.dsub:fpr128, %11:fpr64 # CHECK-NEXT: %12:fpr128 = UMULLv4i16_v4i32 %3.dsub:fpr128, %11:fpr64
# CHECK-NEXT: $q0 = COPY %10:fpr128 # CHECK-NEXT: $q0 = COPY %10:fpr128
# CHECK-NEXT: $q1 = COPY %12:fpr128 # CHECK-NEXT: $q1 = COPY %12:fpr128
# CHECK-NEXT: RET_ReallyLR implicit $q0, implicit $q1 # CHECK-NEXT: RET_ReallyLR implicit $q0, implicit $q1
# CHECK-EMPTY: # CHECK-EMPTY:
# CHECK-NEXT: # End machine code for function umull_and_v8i32. # CHECK-NEXT: # End machine code for function umull_and_v8i32.
# CHECK-EMPTY: # CHECK-EMPTY:
# CHECK-NEXT: ********** MI Scheduling ********** # CHECK-NEXT: ********** MI Scheduling **********
# CHECK-NEXT: umull_and_v8i32:%bb.0 entry # CHECK-NEXT: umull_and_v8i32:%bb.0 entry
# CHECK-NEXT: From: %2:fpr128 = COPY $q2 # CHECK-NEXT: From: %6:fpr128 = MOVIv2d_ns 17
# CHECK-NEXT: To: RET_ReallyLR implicit $q0, implicit $q1 # CHECK-NEXT: To: RET_ReallyLR implicit $q0, implicit $q1
# CHECK-NEXT: RegionInstrs: 13 # CHECK-NEXT: RegionInstrs: 13
# CHECK-NEXT: ScheduleDAGMILive::schedule starting # CHECK-NEXT: ScheduleDAGMILive::schedule starting
# CHECK-NEXT: GenericScheduler RegionPolicy: ShouldTrackPressure=0 OnlyTopDown=0 OnlyBottomUp=1 # CHECK-NEXT: GenericScheduler RegionPolicy: ShouldTrackPressure=0 OnlyTopDown=0 OnlyBottomUp=1
# CHECK-NEXT: Disabled scoreboard hazard recognizer # CHECK-NEXT: Disabled scoreboard hazard recognizer
# CHECK-NEXT: Disabled scoreboard hazard recognizer # CHECK-NEXT: Disabled scoreboard hazard recognizer
# CHECK-NEXT: SU(0) [TopReadyCycle = 0, BottomReadyCycle = 0]: %2:fpr128 = COPY $q2 # CHECK-NEXT: SU(0) [TopReadyCycle = 0, BottomReadyCycle = 0]: %6:fpr128 = MOVIv2d_ns 17
# CHECK-NEXT: # preds left : 0
# CHECK-NEXT: # succs left : 2
# CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 4
# CHECK-NEXT: Depth : 0
# CHECK-NEXT: Height : 16
# CHECK-NEXT: Successors:
# CHECK-NEXT: SU(4): Data Latency=4 Reg=%6
# CHECK-NEXT: SU(3): Data Latency=4 Reg=%6
# CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: SU(1) [TopReadyCycle = 0, BottomReadyCycle = 0]: %2:fpr128 = COPY $q2
# CHECK-NEXT: # preds left : 0 # CHECK-NEXT: # preds left : 0
# CHECK-NEXT: # succs left : 1 # CHECK-NEXT: # succs left : 1
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 3 # CHECK-NEXT: Latency : 1
# CHECK-NEXT: Depth : 0 # CHECK-NEXT: Depth : 0
# CHECK-NEXT: Height : 13 # CHECK-NEXT: Height : 13
# CHECK-NEXT: Successors: # CHECK-NEXT: Successors:
# CHECK-NEXT: SU(5): Data Latency=3 Reg=%2 # CHECK-NEXT: SU(3): Data Latency=1 Reg=%2
# CHECK-NEXT: Single Issue : false; # CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: SU(1) [TopReadyCycle = 0, BottomReadyCycle = 0]: %1:fpr128 = COPY $q1 # CHECK-NEXT: SU(2) [TopReadyCycle = 0, BottomReadyCycle = 0]: %1:fpr128 = COPY $q1
# CHECK-NEXT: # preds left : 0 # CHECK-NEXT: # preds left : 0
# CHECK-NEXT: # succs left : 2 # CHECK-NEXT: # succs left : 2
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 3 # CHECK-NEXT: Latency : 1
# CHECK-NEXT: Depth : 0 # CHECK-NEXT: Depth : 0
# CHECK-NEXT: Height : 13 # CHECK-NEXT: Height : 13
# CHECK-NEXT: Successors: # CHECK-NEXT: Successors:
# CHECK-NEXT: SU(6): Data Latency=3 Reg=%1 # CHECK-NEXT: SU(4): Data Latency=1 Reg=%1
# CHECK-NEXT: SU(12): Anti Latency=0 # CHECK-NEXT: SU(12): Anti Latency=0
# CHECK-NEXT: Single Issue : false; # CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: SU(2) [TopReadyCycle = 0, BottomReadyCycle = 0]: %0:fpr128 = COPY $q0 # CHECK-NEXT: SU(3) [TopReadyCycle = 0, BottomReadyCycle = 0]: %7:fpr128 = ANDv16i8 %2:fpr128, %6:fpr128
# CHECK-NEXT: # preds left : 0 # CHECK-NEXT: # preds left : 2
# CHECK-NEXT: # succs left : 3 # CHECK-NEXT: # succs left : 1
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 3 # CHECK-NEXT: Latency : 3
# CHECK-NEXT: Depth : 0 # CHECK-NEXT: Depth : 4
# CHECK-NEXT: Height : 14 # CHECK-NEXT: Height : 12
# CHECK-NEXT: Predecessors:
# CHECK-NEXT: SU(1): Data Latency=1 Reg=%2
# CHECK-NEXT: SU(0): Data Latency=4 Reg=%6
# CHECK-NEXT: Successors: # CHECK-NEXT: Successors:
# CHECK-NEXT: SU(8): Data Latency=3 Reg=%0 # CHECK-NEXT: SU(8): Data Latency=3 Reg=%7
# CHECK-NEXT: SU(3): Data Latency=3 Reg=%0
# CHECK-NEXT: SU(11): Anti Latency=0
# CHECK-NEXT: Single Issue : false; # CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: SU(3) [TopReadyCycle = 0, BottomReadyCycle = 0]: %3:fpr128 = EXTv16i8 %0:fpr128, %0:fpr128, 8 # CHECK-NEXT: SU(4) [TopReadyCycle = 0, BottomReadyCycle = 0]: %8:fpr128 = ANDv16i8 %1:fpr128, %6:fpr128
# CHECK-NEXT: # preds left : 1 # CHECK-NEXT: # preds left : 2
# CHECK-NEXT: # succs left : 1 # CHECK-NEXT: # succs left : 1
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 4 # CHECK-NEXT: Latency : 3
# CHECK-NEXT: Depth : 3 # CHECK-NEXT: Depth : 4
# CHECK-NEXT: Height : 11 # CHECK-NEXT: Height : 12
# CHECK-NEXT: Predecessors: # CHECK-NEXT: Predecessors:
# CHECK-NEXT: SU(2): Data Latency=3 Reg=%0 # CHECK-NEXT: SU(2): Data Latency=1 Reg=%1
# CHECK-NEXT: SU(0): Data Latency=4 Reg=%6
# CHECK-NEXT: Successors: # CHECK-NEXT: Successors:
# CHECK-NEXT: SU(10): Data Latency=4 Reg=%3 # CHECK-NEXT: SU(7): Data Latency=3 Reg=%8
# CHECK-NEXT: Single Issue : false; # CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: SU(4) [TopReadyCycle = 0, BottomReadyCycle = 0]: %6:fpr128 = MOVIv2d_ns 17 # CHECK-NEXT: SU(5) [TopReadyCycle = 0, BottomReadyCycle = 0]: %0:fpr128 = COPY $q0
# CHECK-NEXT: # preds left : 0 # CHECK-NEXT: # preds left : 0
# CHECK-NEXT: # succs left : 2 # CHECK-NEXT: # succs left : 3
# CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 4
# CHECK-NEXT: Depth : 0
# CHECK-NEXT: Height : 14
# CHECK-NEXT: Successors:
# CHECK-NEXT: SU(6): Data Latency=4 Reg=%6
# CHECK-NEXT: SU(5): Data Latency=4 Reg=%6
# CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: SU(5) [TopReadyCycle = 0, BottomReadyCycle = 0]: %7:fpr128 = ANDv16i8 %2:fpr128, %6:fpr128
# CHECK-NEXT: # preds left : 2
# CHECK-NEXT: # succs left : 1
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 1 # CHECK-NEXT: Latency : 1
# CHECK-NEXT: Depth : 4 # CHECK-NEXT: Depth : 0
# CHECK-NEXT: Height : 10 # CHECK-NEXT: Height : 10
# CHECK-NEXT: Predecessors:
# CHECK-NEXT: SU(4): Data Latency=4 Reg=%6
# CHECK-NEXT: SU(0): Data Latency=3 Reg=%2
# CHECK-NEXT: Successors: # CHECK-NEXT: Successors:
# CHECK-NEXT: SU(9): Data Latency=1 Reg=%7 # CHECK-NEXT: SU(9): Data Latency=1 Reg=%0
# CHECK-NEXT: SU(6): Data Latency=1 Reg=%0
# CHECK-NEXT: SU(11): Anti Latency=0
# CHECK-NEXT: Single Issue : false; # CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: SU(6) [TopReadyCycle = 0, BottomReadyCycle = 0]: %8:fpr128 = ANDv16i8 %1:fpr128, %6:fpr128 # CHECK-NEXT: SU(6) [TopReadyCycle = 0, BottomReadyCycle = 0]: %3:fpr128 = EXTv16i8 %0:fpr128, %0:fpr128, 8
# CHECK-NEXT: # preds left : 2 # CHECK-NEXT: # preds left : 1
# CHECK-NEXT: # succs left : 1 # CHECK-NEXT: # succs left : 1
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 1 # CHECK-NEXT: Latency : 4
# CHECK-NEXT: Depth : 4 # CHECK-NEXT: Depth : 1
# CHECK-NEXT: Height : 10 # CHECK-NEXT: Height : 9
# CHECK-NEXT: Predecessors: # CHECK-NEXT: Predecessors:
# CHECK-NEXT: SU(4): Data Latency=4 Reg=%6 # CHECK-NEXT: SU(5): Data Latency=1 Reg=%0
# CHECK-NEXT: SU(1): Data Latency=3 Reg=%1
# CHECK-NEXT: Successors: # CHECK-NEXT: Successors:
# CHECK-NEXT: SU(7): Data Latency=1 Reg=%8 # CHECK-NEXT: SU(10): Data Latency=4 Reg=%3
# CHECK-NEXT: Single Issue : false; # CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: SU(7) [TopReadyCycle = 0, BottomReadyCycle = 0]: %9:fpr64 = XTNv4i16 %8:fpr128 # CHECK-NEXT: SU(7) [TopReadyCycle = 0, BottomReadyCycle = 0]: %9:fpr64 = XTNv4i16 %8:fpr128
# CHECK-NEXT: # preds left : 1 # CHECK-NEXT: # preds left : 1
# CHECK-NEXT: # succs left : 1 # CHECK-NEXT: # succs left : 1
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 2 # CHECK-NEXT: Latency : 4
# CHECK-NEXT: Depth : 5 # CHECK-NEXT: Depth : 7
# CHECK-NEXT: Height : 9 # CHECK-NEXT: Height : 9
# CHECK-NEXT: Predecessors: # CHECK-NEXT: Predecessors:
# CHECK-NEXT: SU(6): Data Latency=1 Reg=%8 # CHECK-NEXT: SU(4): Data Latency=3 Reg=%8
# CHECK-NEXT: Successors: # CHECK-NEXT: Successors:
# CHECK-NEXT: SU(8): Data Latency=2 Reg=%9 # CHECK-NEXT: SU(9): Data Latency=4 Reg=%9
# CHECK-NEXT: Single Issue : false; # CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: SU(8) [TopReadyCycle = 0, BottomReadyCycle = 0]: %10:fpr128 = UMULLv4i16_v4i32 %0.dsub:fpr128, %9:fpr64 # CHECK-NEXT: SU(8) [TopReadyCycle = 0, BottomReadyCycle = 0]: %11:fpr64 = XTNv4i16 %7:fpr128
# CHECK-NEXT: # preds left : 2 # CHECK-NEXT: # preds left : 1
# CHECK-NEXT: # succs left : 1 # CHECK-NEXT: # succs left : 1
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 4 # CHECK-NEXT: Latency : 4
# CHECK-NEXT: Depth : 7 # CHECK-NEXT: Depth : 7
# CHECK-NEXT: Height : 7 # CHECK-NEXT: Height : 9
# CHECK-NEXT: Predecessors: # CHECK-NEXT: Predecessors:
# CHECK-NEXT: SU(7): Data Latency=2 Reg=%9 # CHECK-NEXT: SU(3): Data Latency=3 Reg=%7
# CHECK-NEXT: SU(2): Data Latency=3 Reg=%0
# CHECK-NEXT: Successors: # CHECK-NEXT: Successors:
# CHECK-NEXT: SU(11): Data Latency=4 Reg=%10 # CHECK-NEXT: SU(10): Data Latency=4 Reg=%11
# CHECK-NEXT: Single Issue : false; # CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: SU(9) [TopReadyCycle = 0, BottomReadyCycle = 0]: %11:fpr64 = XTNv4i16 %7:fpr128 # CHECK-NEXT: SU(9) [TopReadyCycle = 0, BottomReadyCycle = 0]: %10:fpr128 = UMULLv4i16_v4i32 %0.dsub:fpr128, %9:fpr64
# CHECK-NEXT: # preds left : 1 # CHECK-NEXT: # preds left : 2
# CHECK-NEXT: # succs left : 1 # CHECK-NEXT: # succs left : 1
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 2 # CHECK-NEXT: Latency : 4
# CHECK-NEXT: Depth : 5 # CHECK-NEXT: Depth : 11
# CHECK-NEXT: Height : 9 # CHECK-NEXT: Height : 5
# CHECK-NEXT: Predecessors: # CHECK-NEXT: Predecessors:
# CHECK-NEXT: SU(5): Data Latency=1 Reg=%7 # CHECK-NEXT: SU(7): Data Latency=4 Reg=%9
# CHECK-NEXT: SU(5): Data Latency=1 Reg=%0
# CHECK-NEXT: Successors: # CHECK-NEXT: Successors:
# CHECK-NEXT: SU(10): Data Latency=2 Reg=%11 # CHECK-NEXT: SU(11): Data Latency=4 Reg=%10
# CHECK-NEXT: Single Issue : false; # CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: SU(10) [TopReadyCycle = 0, BottomReadyCycle = 0]: %12:fpr128 = UMULLv4i16_v4i32 %3.dsub:fpr128, %11:fpr64 # CHECK-NEXT: SU(10) [TopReadyCycle = 0, BottomReadyCycle = 0]: %12:fpr128 = UMULLv4i16_v4i32 %3.dsub:fpr128, %11:fpr64
# CHECK-NEXT: # preds left : 2 # CHECK-NEXT: # preds left : 2
# CHECK-NEXT: # succs left : 1 # CHECK-NEXT: # succs left : 1
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 4 # CHECK-NEXT: Latency : 4
# CHECK-NEXT: Depth : 7 # CHECK-NEXT: Depth : 11
# CHECK-NEXT: Height : 7 # CHECK-NEXT: Height : 5
# CHECK-NEXT: Predecessors: # CHECK-NEXT: Predecessors:
# CHECK-NEXT: SU(9): Data Latency=2 Reg=%11 # CHECK-NEXT: SU(8): Data Latency=4 Reg=%11
# CHECK-NEXT: SU(3): Data Latency=4 Reg=%3 # CHECK-NEXT: SU(6): Data Latency=4 Reg=%3
# CHECK-NEXT: Successors: # CHECK-NEXT: Successors:
# CHECK-NEXT: SU(12): Data Latency=4 Reg=%12 # CHECK-NEXT: SU(12): Data Latency=4 Reg=%12
# CHECK-NEXT: Single Issue : false; # CHECK-NEXT: Single Issue : false;
Context not available.
# CHECK-NEXT: # preds left : 2 # CHECK-NEXT: # preds left : 2
# CHECK-NEXT: # succs left : 1 # CHECK-NEXT: # succs left : 1
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 3 # CHECK-NEXT: Latency : 1
# CHECK-NEXT: Depth : 11 # CHECK-NEXT: Depth : 15
# CHECK-NEXT: Height : 3 # CHECK-NEXT: Height : 1
# CHECK-NEXT: Predecessors: # CHECK-NEXT: Predecessors:
# CHECK-NEXT: SU(8): Data Latency=4 Reg=%10 # CHECK-NEXT: SU(9): Data Latency=4 Reg=%10
# CHECK-NEXT: SU(2): Anti Latency=0 # CHECK-NEXT: SU(5): Anti Latency=0
# CHECK-NEXT: Successors: # CHECK-NEXT: Successors:
# CHECK-NEXT: ExitSU: Ord Latency=3 Artificial # CHECK-NEXT: ExitSU: Ord Latency=1 Artificial
# CHECK-NEXT: Single Issue : false; # CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: SU(12) [TopReadyCycle = 0, BottomReadyCycle = 0]: $q1 = COPY %12:fpr128 # CHECK-NEXT: SU(12) [TopReadyCycle = 0, BottomReadyCycle = 0]: $q1 = COPY %12:fpr128
# CHECK-NEXT: # preds left : 2 # CHECK-NEXT: # preds left : 2
# CHECK-NEXT: # succs left : 1 # CHECK-NEXT: # succs left : 1
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 3 # CHECK-NEXT: Latency : 1
# CHECK-NEXT: Depth : 11 # CHECK-NEXT: Depth : 15
# CHECK-NEXT: Height : 3 # CHECK-NEXT: Height : 1
# CHECK-NEXT: Predecessors: # CHECK-NEXT: Predecessors:
# CHECK-NEXT: SU(10): Data Latency=4 Reg=%12 # CHECK-NEXT: SU(10): Data Latency=4 Reg=%12
# CHECK-NEXT: SU(1): Anti Latency=0 # CHECK-NEXT: SU(2): Anti Latency=0
# CHECK-NEXT: Successors: # CHECK-NEXT: Successors:
# CHECK-NEXT: ExitSU: Ord Latency=3 Artificial # CHECK-NEXT: ExitSU: Ord Latency=1 Artificial
# CHECK-NEXT: Single Issue : false; # CHECK-NEXT: Single Issue : false;
# CHECK-NEXT: ExitSU [TopReadyCycle = 0, BottomReadyCycle = 0]: RET_ReallyLR implicit $q0, implicit $q1 # CHECK-NEXT: ExitSU [TopReadyCycle = 0, BottomReadyCycle = 0]: RET_ReallyLR implicit $q0, implicit $q1
# CHECK-NEXT: # preds left : 2 # CHECK-NEXT: # preds left : 2
# CHECK-NEXT: # succs left : 0 # CHECK-NEXT: # succs left : 0
# CHECK-NEXT: # rdefs left : 0 # CHECK-NEXT: # rdefs left : 0
# CHECK-NEXT: Latency : 0 # CHECK-NEXT: Latency : 0
# CHECK-NEXT: Depth : 14 # CHECK-NEXT: Depth : 16
# CHECK-NEXT: Height : 0 # CHECK-NEXT: Height : 0
# CHECK-NEXT: Predecessors: # CHECK-NEXT: Predecessors:
# CHECK-NEXT: SU(12): Ord Latency=3 Artificial # CHECK-NEXT: SU(12): Ord Latency=1 Artificial
# CHECK-NEXT: SU(11): Ord Latency=3 Artificial # CHECK-NEXT: SU(11): Ord Latency=1 Artificial
# CHECK-NEXT: Resource booking (@0c): # CHECK-NEXT: Critical Path(GS-RR ): 16
# CHECK-NEXT: CortexA55UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@0c):
# CHECK-NEXT: Instance 0 available @0c
# CHECK-NEXT: Instance 1 available @0c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @0c
# CHECK-NEXT: Resource booking (@0c):
# CHECK-NEXT: CortexA55UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@0c):
# CHECK-NEXT: Instance 0 available @0c
# CHECK-NEXT: Instance 1 available @0c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @0c
# CHECK-NEXT: Resource booking (@0c):
# CHECK-NEXT: CortexA55UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@0c):
# CHECK-NEXT: Instance 0 available @0c
# CHECK-NEXT: Instance 1 available @0c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @0c
# CHECK-NEXT: Resource booking (@0c):
# CHECK-NEXT: CortexA55UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@0c):
# CHECK-NEXT: Instance 0 available @0c
# CHECK-NEXT: Instance 1 available @0c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @0c
# CHECK-NEXT: Critical Path(GS-RR ): 14
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node # CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Cycle: 3 BotQ.A # CHECK-NEXT: Cycle: 1 BotQ.A
# CHECK-NEXT: Resource booking (@3c): # CHECK-NEXT: Queue BotQ.P:
# CHECK-NEXT: CortexA55UnitALU(0) = 4294967295 # CHECK-NEXT: Queue BotQ.A: 12 11
# CHECK-NEXT: CortexA55UnitALU(1) = 4294967295 # CHECK-NEXT: Cand SU(12) ORDER
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: Pick Bot ORDER
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@3c):
# CHECK-NEXT: Instance 0 available @3c
# CHECK-NEXT: Instance 1 available @3c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @3c
# CHECK-NEXT: Resource booking (@3c):
# CHECK-NEXT: CortexA55UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@3c):
# CHECK-NEXT: Instance 0 available @3c
# CHECK-NEXT: Instance 1 available @3c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @3c
# CHECK-NEXT: Queue BotQ.P:
# CHECK-NEXT: Queue BotQ.A: 12 11
# CHECK-NEXT: Cand SU(12) ORDER
# CHECK-NEXT: Pick Bot ORDER
# CHECK-NEXT: Scheduling SU(12) $q1 = COPY %12:fpr128 # CHECK-NEXT: Scheduling SU(12) $q1 = COPY %12:fpr128
# CHECK-NEXT: Ready @3c # CHECK-NEXT: Ready @1c
# CHECK-NEXT: CortexA55UnitALU +1x1u # CHECK-NEXT: CortexA510UnitALU +1x2u
# CHECK-NEXT: Resource booking (@3c): # CHECK-NEXT: Resource booking (@1c):
# CHECK-NEXT: CortexA55UnitALU(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@3c): # CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: Instance 0 available @3c # CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: Instance 1 available @3c # CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @3c # CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: Resource booking (@3c): # CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 4294967295 # CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295 # CHECK-NEXT: getNextResourceCycle (@1c):
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295 # CHECK-NEXT: Instance 0 available @1c
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: Instance 1 available @1c
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: Instance 2 available @1c
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: selecting CortexA510UnitALU[0] available @1c
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: BotQ.A TopLatency SU(12) 15c
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: BotQ.A BotLatency SU(12) 1c
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: BotQ.A @1c
# CHECK-NEXT: getNextResourceCycle (@3c):
# CHECK-NEXT: Instance 0 available @3c
# CHECK-NEXT: Instance 1 available @3c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @3c
# CHECK-NEXT: BotQ.A TopLatency SU(12) 11c
# CHECK-NEXT: BotQ.A BotLatency SU(12) 3c
# CHECK-NEXT: BotQ.A @3c
# CHECK-NEXT: Retired: 1 # CHECK-NEXT: Retired: 1
# CHECK-NEXT: Executed: 3c # CHECK-NEXT: Executed: 1c
# CHECK-NEXT: Critical: 0c, 1 MOps # CHECK-NEXT: Critical: 0c, 1 MOps
# CHECK-NEXT: ExpectedLatency: 3c # CHECK-NEXT: ExpectedLatency: 1c
# CHECK-NEXT: - Latency limited. # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node # CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Resource booking (@3c): # CHECK-NEXT: Queue BotQ.P: 10
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: Queue BotQ.A: 11
# CHECK-NEXT: CortexA55UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@3c):
# CHECK-NEXT: Instance 0 available @4c
# CHECK-NEXT: Instance 1 available @3c
# CHECK-NEXT: selecting CortexA55UnitALU[1] available @3c
# CHECK-NEXT: Queue BotQ.P: 10
# CHECK-NEXT: Queue BotQ.A: 11
# CHECK-NEXT: Scheduling SU(11) $q0 = COPY %10:fpr128 # CHECK-NEXT: Scheduling SU(11) $q0 = COPY %10:fpr128
# CHECK-NEXT: Ready @3c # CHECK-NEXT: Ready @1c
# CHECK-NEXT: CortexA55UnitALU +1x1u # CHECK-NEXT: CortexA510UnitALU +1x2u
# CHECK-NEXT: Resource booking (@3c): # CHECK-NEXT: Resource booking (@1c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@3c): # CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: Instance 0 available @4c # CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: Instance 1 available @3c # CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: selecting CortexA55UnitALU[1] available @3c # CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: Resource booking (@3c): # CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 4294967295 # CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295 # CHECK-NEXT: getNextResourceCycle (@1c):
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295 # CHECK-NEXT: Instance 0 available @1c
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: Instance 1 available @1c
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: Instance 2 available @1c
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: selecting CortexA510UnitALU[0] available @1c
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: BotQ.A @1c
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@3c):
# CHECK-NEXT: Instance 0 available @4c
# CHECK-NEXT: Instance 1 available @3c
# CHECK-NEXT: selecting CortexA55UnitALU[1] available @3c
# CHECK-NEXT: *** Max MOps 2 at cycle 3
# CHECK-NEXT: Cycle: 4 BotQ.A
# CHECK-NEXT: BotQ.A @4c
# CHECK-NEXT: Retired: 2 # CHECK-NEXT: Retired: 2
# CHECK-NEXT: Executed: 4c # CHECK-NEXT: Executed: 1c
# CHECK-NEXT: Critical: 1c, 2 MOps # CHECK-NEXT: Critical: 0c, 2 MOps
# CHECK-NEXT: ExpectedLatency: 3c # CHECK-NEXT: ExpectedLatency: 1c
# CHECK-NEXT: - Latency limited. # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node # CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Cycle: 7 BotQ.A # CHECK-NEXT: Cycle: 2 BotQ.A
# CHECK-NEXT: Resource booking (@7c): # CHECK-NEXT: Cycle: 5 BotQ.A
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: Queue BotQ.P:
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: Queue BotQ.A: 10 9
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: Cand SU(10) ORDER
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: Pick Bot ORDER
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@7c):
# CHECK-NEXT: Instance 0 available @7c
# CHECK-NEXT: Instance 1 available @7c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @7c
# CHECK-NEXT: Resource booking (@7c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@7c):
# CHECK-NEXT: Instance 0 available @7c
# CHECK-NEXT: Instance 1 available @7c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @7c
# CHECK-NEXT: Queue BotQ.P:
# CHECK-NEXT: Queue BotQ.A: 10 8
# CHECK-NEXT: Cand SU(10) ORDER
# CHECK-NEXT: Pick Bot ORDER
# CHECK-NEXT: Scheduling SU(10) %12:fpr128 = UMULLv4i16_v4i32 %3.dsub:fpr128, %11:fpr64 # CHECK-NEXT: Scheduling SU(10) %12:fpr128 = UMULLv4i16_v4i32 %3.dsub:fpr128, %11:fpr64
# CHECK-NEXT: Ready @7c # CHECK-NEXT: Ready @5c
# CHECK-NEXT: CortexA55UnitFPALU +2x1u # CHECK-NEXT: CortexA510UnitVALU +1x3u
# CHECK-NEXT: Resource booking (@7c): # CHECK-NEXT: Resource booking (@5c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@7c): # CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: Instance 0 available @7c # CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: Instance 1 available @7c # CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @7c # CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: Resource booking (@7c): # CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 4294967295 # CHECK-NEXT: getNextResourceCycle (@5c):
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295 # CHECK-NEXT: Instance 0 available @5c
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: Instance 1 available @5c
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: selecting CortexA510UnitVALU[0] available @5c
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: BotQ.A BotLatency SU(10) 5c
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: BotQ.A @5c
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@7c):
# CHECK-NEXT: Instance 0 available @7c
# CHECK-NEXT: Instance 1 available @7c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @7c
# CHECK-NEXT: BotQ.A BotLatency SU(10) 7c
# CHECK-NEXT: Bump cycle to begin group
# CHECK-NEXT: Cycle: 8 BotQ.A
# CHECK-NEXT: BotQ.A @8c
# CHECK-NEXT: Retired: 3 # CHECK-NEXT: Retired: 3
# CHECK-NEXT: Executed: 8c # CHECK-NEXT: Executed: 5c
# CHECK-NEXT: Critical: 1c, 3 MOps # CHECK-NEXT: Critical: 1c, 3 MOps
# CHECK-NEXT: ExpectedLatency: 7c # CHECK-NEXT: ExpectedLatency: 5c
# CHECK-NEXT: - Latency limited. # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 7 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node # CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Resource booking (@8c): # CHECK-NEXT: Queue BotQ.P: 8 6
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: Queue BotQ.A: 9
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: Scheduling SU(9) %10:fpr128 = UMULLv4i16_v4i32 %0.dsub:fpr128, %9:fpr64
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: Ready @5c
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVALU +1x3u
# CHECK-NEXT: CortexA55UnitFPALU(0) = 7 # CHECK-NEXT: Resource booking (@5c):
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@8c): # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: Instance 0 available @9c # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: Instance 1 available @8c # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: selecting CortexA55UnitFPALU[1] available @8c # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: Queue BotQ.P: 9 3 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: Queue BotQ.A: 8 # CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: Scheduling SU(8) %10:fpr128 = UMULLv4i16_v4i32 %0.dsub:fpr128, %9:fpr64 # CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: Ready @8c # CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU +2x1u # CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: Resource booking (@8c): # CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 7 # CHECK-NEXT: getNextResourceCycle (@5c):
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295 # CHECK-NEXT: Instance 0 available @5c
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: Instance 1 available @5c
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: selecting CortexA510UnitVALU[0] available @5c
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: BotQ.A @5c
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@8c):
# CHECK-NEXT: Instance 0 available @9c
# CHECK-NEXT: Instance 1 available @8c
# CHECK-NEXT: selecting CortexA55UnitFPALU[1] available @8c
# CHECK-NEXT: Resource booking (@8c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 7
# CHECK-NEXT: CortexA55UnitFPALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@8c):
# CHECK-NEXT: Instance 0 available @9c
# CHECK-NEXT: Instance 1 available @8c
# CHECK-NEXT: selecting CortexA55UnitFPALU[1] available @8c
# CHECK-NEXT: BotQ.A TopLatency SU(8) 7c
# CHECK-NEXT: Bump cycle to begin group
# CHECK-NEXT: Cycle: 9 BotQ.A
# CHECK-NEXT: BotQ.A @9c
# CHECK-NEXT: Retired: 4 # CHECK-NEXT: Retired: 4
# CHECK-NEXT: Executed: 9c # CHECK-NEXT: Executed: 5c
# CHECK-NEXT: Critical: 2c, 4 MOps # CHECK-NEXT: Critical: 1c, 4 MOps
# CHECK-NEXT: ExpectedLatency: 7c # CHECK-NEXT: ExpectedLatency: 5c
# CHECK-NEXT: - Latency limited. # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 7 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node # CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Resource booking (@9c): # CHECK-NEXT: Cycle: 6 BotQ.A
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: Cycle: 9 BotQ.A
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: Queue BotQ.P:
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: Queue BotQ.A: 8 7 6
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: Cand SU(8) ORDER
# CHECK-NEXT: CortexA55UnitFPALU(0) = 7 # CHECK-NEXT: Pick Bot ORDER
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8 # CHECK-NEXT: Scheduling SU(8) %11:fpr64 = XTNv4i16 %7:fpr128
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@9c):
# CHECK-NEXT: Instance 0 available @9c
# CHECK-NEXT: Instance 1 available @9c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @9c
# CHECK-NEXT: Resource booking (@9c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 7
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@9c):
# CHECK-NEXT: Instance 0 available @9c
# CHECK-NEXT: Instance 1 available @9c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @9c
# CHECK-NEXT: Queue BotQ.P: 7 3
# CHECK-NEXT: Queue BotQ.A: 9
# CHECK-NEXT: Scheduling SU(9) %11:fpr64 = XTNv4i16 %7:fpr128
# CHECK-NEXT: Ready @9c # CHECK-NEXT: Ready @9c
# CHECK-NEXT: CortexA55UnitFPALU +1x1u # CHECK-NEXT: CortexA510UnitVALU +1x3u
# CHECK-NEXT: Resource booking (@9c): # CHECK-NEXT: Resource booking (@9c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 7 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@9c): # CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@9c):
# CHECK-NEXT: Instance 0 available @9c # CHECK-NEXT: Instance 0 available @9c
# CHECK-NEXT: Instance 1 available @9c # CHECK-NEXT: Instance 1 available @9c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @9c # CHECK-NEXT: selecting CortexA510UnitVALU[0] available @9c
# CHECK-NEXT: Resource booking (@9c): # CHECK-NEXT: BotQ.A BotLatency SU(8) 9c
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: BotQ.A @9c
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: Retired: 5
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: Executed: 9c
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: Critical: 1c, 5 MOps
# CHECK-NEXT: CortexA55UnitFPALU(0) = 7 # CHECK-NEXT: ExpectedLatency: 9c
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8 # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@9c): # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Queue BotQ.P: 3
# CHECK-NEXT: Queue BotQ.A: 6 7
# CHECK-NEXT: Cand SU(6) ORDER
# CHECK-NEXT: Cand SU(7) ORDER
# CHECK-NEXT: Pick Bot ORDER
# CHECK-NEXT: Scheduling SU(7) %9:fpr64 = XTNv4i16 %8:fpr128
# CHECK-NEXT: Ready @9c
# CHECK-NEXT: CortexA510UnitVALU +1x3u
# CHECK-NEXT: Resource booking (@9c):
# CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@9c):
# CHECK-NEXT: Instance 0 available @9c # CHECK-NEXT: Instance 0 available @9c
# CHECK-NEXT: Instance 1 available @9c # CHECK-NEXT: Instance 1 available @9c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @9c # CHECK-NEXT: selecting CortexA510UnitVALU[0] available @9c
# CHECK-NEXT: BotQ.A BotLatency SU(9) 9c
# CHECK-NEXT: BotQ.A @9c # CHECK-NEXT: BotQ.A @9c
# CHECK-NEXT: Retired: 5 # CHECK-NEXT: Retired: 6
# CHECK-NEXT: Executed: 9c # CHECK-NEXT: Executed: 9c
# CHECK-NEXT: Critical: 2c, 5 MOps # CHECK-NEXT: Critical: 2c, 6 MOps
# CHECK-NEXT: ExpectedLatency: 9c # CHECK-NEXT: ExpectedLatency: 9c
# CHECK-NEXT: - Latency limited. # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 9 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node # CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Queue BotQ.P: 3 4
# CHECK-NEXT: Queue BotQ.A: 6
# CHECK-NEXT: Scheduling SU(6) %3:fpr128 = EXTv16i8 %0:fpr128, %0:fpr128, 8
# CHECK-NEXT: Ready @9c
# CHECK-NEXT: CortexA510UnitVALU +1x3u
# CHECK-NEXT: *** Critical resource CortexA510UnitVALU: 2c
# CHECK-NEXT: Resource booking (@9c):
# CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@9c):
# CHECK-NEXT: Instance 0 available @9c
# CHECK-NEXT: Instance 1 available @9c
# CHECK-NEXT: selecting CortexA510UnitVALU[0] available @9c
# CHECK-NEXT: *** Max MOps 3 at cycle 9
# CHECK-NEXT: Cycle: 10 BotQ.A # CHECK-NEXT: Cycle: 10 BotQ.A
# CHECK-NEXT: Resource booking (@10c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 9
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@10c):
# CHECK-NEXT: Instance 0 available @10c
# CHECK-NEXT: Instance 1 available @10c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @10c
# CHECK-NEXT: Resource booking (@10c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 9
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@10c):
# CHECK-NEXT: Instance 0 available @11c
# CHECK-NEXT: Instance 1 available @10c
# CHECK-NEXT: selecting CortexA55UnitFPALU[1] available @10c
# CHECK-NEXT: Queue BotQ.P: 3
# CHECK-NEXT: Queue BotQ.A: 7 5
# CHECK-NEXT: Cand SU(7) ORDER
# CHECK-NEXT: Pick Bot ORDER
# CHECK-NEXT: Scheduling SU(7) %9:fpr64 = XTNv4i16 %8:fpr128
# CHECK-NEXT: Ready @10c
# CHECK-NEXT: CortexA55UnitFPALU +1x1u
# CHECK-NEXT: Resource booking (@10c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 9
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@10c):
# CHECK-NEXT: Instance 0 available @10c
# CHECK-NEXT: Instance 1 available @10c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @10c
# CHECK-NEXT: Resource booking (@10c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 9
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@10c):
# CHECK-NEXT: Instance 0 available @10c
# CHECK-NEXT: Instance 1 available @10c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @10c
# CHECK-NEXT: BotQ.A @10c # CHECK-NEXT: BotQ.A @10c
# CHECK-NEXT: Retired: 6 # CHECK-NEXT: Retired: 7
# CHECK-NEXT: Executed: 10c # CHECK-NEXT: Executed: 10c
# CHECK-NEXT: Critical: 3c, 6 MOps # CHECK-NEXT: Critical: 2c, 5 CortexA510UnitVALU
# CHECK-NEXT: ExpectedLatency: 9c # CHECK-NEXT: ExpectedLatency: 9c
# CHECK-NEXT: - Latency limited. # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 10 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node # CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Resource booking (@10c): # CHECK-NEXT: Queue BotQ.P: 3 4
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: Queue BotQ.A: 5
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: Scheduling SU(5) %0:fpr128 = COPY $q0
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 10
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@10c):
# CHECK-NEXT: Instance 0 available @12c
# CHECK-NEXT: Instance 1 available @10c
# CHECK-NEXT: selecting CortexA55UnitFPALU[1] available @10c
# CHECK-NEXT: Queue BotQ.P: 3 6
# CHECK-NEXT: Queue BotQ.A: 5
# CHECK-NEXT: Scheduling SU(5) %7:fpr128 = ANDv16i8 %2:fpr128, %6:fpr128
# CHECK-NEXT: Ready @10c # CHECK-NEXT: Ready @10c
# CHECK-NEXT: CortexA55UnitFPALU +2x1u # CHECK-NEXT: CortexA510UnitALU +1x2u
# CHECK-NEXT: *** Critical resource CortexA55UnitFPALU: 4c # CHECK-NEXT: Resource booking (@10c):
# CHECK-NEXT: Resource booking (@10c): # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 10 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@10c): # CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: Instance 0 available @12c # CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: Instance 1 available @10c # CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: selecting CortexA55UnitFPALU[1] available @10c # CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: Resource booking (@10c): # CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: getNextResourceCycle (@10c):
# CHECK-NEXT: CortexA55UnitFPALU(0) = 10 # CHECK-NEXT: Instance 0 available @10c
# CHECK-NEXT: CortexA55UnitFPALU(1) = 8
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@10c):
# CHECK-NEXT: Instance 0 available @12c
# CHECK-NEXT: Instance 1 available @10c # CHECK-NEXT: Instance 1 available @10c
# CHECK-NEXT: selecting CortexA55UnitFPALU[1] available @10c # CHECK-NEXT: Instance 2 available @10c
# CHECK-NEXT: selecting CortexA510UnitALU[0] available @10c
# CHECK-NEXT: BotQ.A BotLatency SU(5) 10c # CHECK-NEXT: BotQ.A BotLatency SU(5) 10c
# CHECK-NEXT: Bump cycle to begin group # CHECK-NEXT: BotQ.A @10c
# CHECK-NEXT: Cycle: 11 BotQ.A # CHECK-NEXT: Retired: 8
# CHECK-NEXT: BotQ.A @11c # CHECK-NEXT: Executed: 10c
# CHECK-NEXT: Retired: 7 # CHECK-NEXT: Critical: 2c, 5 CortexA510UnitVALU
# CHECK-NEXT: Executed: 11c
# CHECK-NEXT: Critical: 4c, 8 CortexA55UnitFPALU
# CHECK-NEXT: ExpectedLatency: 10c # CHECK-NEXT: ExpectedLatency: 10c
# CHECK-NEXT: - Latency limited. # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 10 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node # CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Resource booking (@11c): # CHECK-NEXT: Cycle: 11 BotQ.A
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 10
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@11c):
# CHECK-NEXT: Instance 0 available @12c
# CHECK-NEXT: Instance 1 available @12c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @12c
# CHECK-NEXT: SU(3) CortexA55UnitFPALU[0]=12c
# CHECK-NEXT: Resource booking (@11c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 10
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@11c):
# CHECK-NEXT: Instance 0 available @12c
# CHECK-NEXT: Instance 1 available @12c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @12c
# CHECK-NEXT: SU(6) CortexA55UnitFPALU[0]=12c
# CHECK-NEXT: Cycle: 12 BotQ.A # CHECK-NEXT: Cycle: 12 BotQ.A
# CHECK-NEXT: Resource booking (@12c): # CHECK-NEXT: Queue BotQ.P:
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: Queue BotQ.A: 3 4
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: Cand SU(3) ORDER
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: Cand SU(4) ORDER
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: Pick Bot ORDER
# CHECK-NEXT: CortexA55UnitFPALU(0) = 10 # CHECK-NEXT: Scheduling SU(4) %8:fpr128 = ANDv16i8 %1:fpr128, %6:fpr128
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@12c):
# CHECK-NEXT: Instance 0 available @12c
# CHECK-NEXT: Instance 1 available @12c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @12c
# CHECK-NEXT: Resource booking (@12c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 10
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@12c):
# CHECK-NEXT: Instance 0 available @12c
# CHECK-NEXT: Instance 1 available @12c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @12c
# CHECK-NEXT: Queue BotQ.P: 0
# CHECK-NEXT: Queue BotQ.A: 3 6
# CHECK-NEXT: Cand SU(3) ORDER
# CHECK-NEXT: Cand SU(6) ORDER
# CHECK-NEXT: Pick Bot ORDER
# CHECK-NEXT: Scheduling SU(6) %8:fpr128 = ANDv16i8 %1:fpr128, %6:fpr128
# CHECK-NEXT: Ready @12c # CHECK-NEXT: Ready @12c
# CHECK-NEXT: CortexA55UnitFPALU +2x1u # CHECK-NEXT: CortexA510UnitVALU +1x3u
# CHECK-NEXT: Resource booking (@12c): # CHECK-NEXT: Resource booking (@12c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 10 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@12c): # CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@12c):
# CHECK-NEXT: Instance 0 available @12c # CHECK-NEXT: Instance 0 available @12c
# CHECK-NEXT: Instance 1 available @12c # CHECK-NEXT: Instance 1 available @12c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @12c # CHECK-NEXT: selecting CortexA510UnitVALU[0] available @12c
# CHECK-NEXT: Resource booking (@12c): # CHECK-NEXT: BotQ.A BotLatency SU(4) 12c
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: BotQ.A @12c
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: Retired: 9
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: Executed: 12c
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: Critical: 3c, 6 CortexA510UnitVALU
# CHECK-NEXT: CortexA55UnitFPALU(0) = 10 # CHECK-NEXT: ExpectedLatency: 12c
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10 # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@12c): # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Queue BotQ.P: 2
# CHECK-NEXT: Queue BotQ.A: 3
# CHECK-NEXT: Scheduling SU(3) %7:fpr128 = ANDv16i8 %2:fpr128, %6:fpr128
# CHECK-NEXT: Ready @12c
# CHECK-NEXT: CortexA510UnitVALU +1x3u
# CHECK-NEXT: Resource booking (@12c):
# CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@12c):
# CHECK-NEXT: Instance 0 available @12c # CHECK-NEXT: Instance 0 available @12c
# CHECK-NEXT: Instance 1 available @12c # CHECK-NEXT: Instance 1 available @12c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @12c # CHECK-NEXT: selecting CortexA510UnitVALU[0] available @12c
# CHECK-NEXT: BotQ.A TopLatency SU(6) 4c # CHECK-NEXT: BotQ.A @12c
# CHECK-NEXT: Bump cycle to begin group # CHECK-NEXT: Retired: 10
# CHECK-NEXT: Cycle: 13 BotQ.A # CHECK-NEXT: Executed: 12c
# CHECK-NEXT: BotQ.A @13c # CHECK-NEXT: Critical: 3c, 7 CortexA510UnitVALU
# CHECK-NEXT: Retired: 8 # CHECK-NEXT: ExpectedLatency: 12c
# CHECK-NEXT: Executed: 13c
# CHECK-NEXT: Critical: 5c, 10 CortexA55UnitFPALU
# CHECK-NEXT: ExpectedLatency: 10c
# CHECK-NEXT: - Latency limited. # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node # CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Resource booking (@13c): # CHECK-NEXT: Cycle: 13 BotQ.A
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: Queue BotQ.P: 0
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: Queue BotQ.A: 2 1
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: Cand SU(2) ORDER
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: Pick Bot ORDER
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12 # CHECK-NEXT: Scheduling SU(2) %1:fpr128 = COPY $q1
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@13c):
# CHECK-NEXT: Instance 0 available @13c
# CHECK-NEXT: Instance 1 available @13c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @13c
# CHECK-NEXT: Resource booking (@13c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@13c):
# CHECK-NEXT: Instance 0 available @14c
# CHECK-NEXT: Instance 1 available @13c
# CHECK-NEXT: selecting CortexA55UnitFPALU[1] available @13c
# CHECK-NEXT: Resource booking (@13c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@13c):
# CHECK-NEXT: Instance 0 available @13c
# CHECK-NEXT: Instance 1 available @13c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @13c
# CHECK-NEXT: Queue BotQ.P: 1 4
# CHECK-NEXT: Queue BotQ.A: 3 0
# CHECK-NEXT: Cand SU(3) ORDER
# CHECK-NEXT: Pick Bot PHYS-REG
# CHECK-NEXT: Scheduling SU(3) %3:fpr128 = EXTv16i8 %0:fpr128, %0:fpr128, 8
# CHECK-NEXT: Ready @13c # CHECK-NEXT: Ready @13c
# CHECK-NEXT: CortexA55UnitFPALU +2x1u # CHECK-NEXT: CortexA510UnitALU +1x2u
# CHECK-NEXT: Resource booking (@13c): # CHECK-NEXT: Resource booking (@13c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@13c): # CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: Instance 0 available @14c # CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: Instance 1 available @13c # CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: selecting CortexA55UnitFPALU[1] available @13c # CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: Resource booking (@13c): # CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(0) = 3 # CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12 # CHECK-NEXT: getNextResourceCycle (@13c):
# CHECK-NEXT: CortexA55UnitFPALU(1) = 10 # CHECK-NEXT: Instance 0 available @13c
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@13c):
# CHECK-NEXT: Instance 0 available @14c
# CHECK-NEXT: Instance 1 available @13c # CHECK-NEXT: Instance 1 available @13c
# CHECK-NEXT: selecting CortexA55UnitFPALU[1] available @13c # CHECK-NEXT: Instance 2 available @13c
# CHECK-NEXT: BotQ.A BotLatency SU(3) 11c # CHECK-NEXT: selecting CortexA510UnitALU[0] available @13c
# CHECK-NEXT: Bump cycle to begin group # CHECK-NEXT: BotQ.A BotLatency SU(2) 13c
# CHECK-NEXT: Cycle: 14 BotQ.A # CHECK-NEXT: BotQ.A @13c
# CHECK-NEXT: BotQ.A @14c # CHECK-NEXT: Retired: 11
# CHECK-NEXT: Retired: 9 # CHECK-NEXT: Executed: 13c
# CHECK-NEXT: Executed: 14c # CHECK-NEXT: Critical: 3c, 7 CortexA510UnitVALU
# CHECK-NEXT: Critical: 6c, 12 CortexA55UnitFPALU
# CHECK-NEXT: ExpectedLatency: 11c
# CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Resource booking (@14c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@14c):
# CHECK-NEXT: Instance 0 available @14c
# CHECK-NEXT: Instance 1 available @14c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @14c
# CHECK-NEXT: Queue BotQ.P: 1 4 2
# CHECK-NEXT: Queue BotQ.A: 0
# CHECK-NEXT: Scheduling SU(0) %2:fpr128 = COPY $q2
# CHECK-NEXT: Ready @14c
# CHECK-NEXT: CortexA55UnitALU +1x1u
# CHECK-NEXT: Resource booking (@14c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@14c):
# CHECK-NEXT: Instance 0 available @14c
# CHECK-NEXT: Instance 1 available @14c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @14c
# CHECK-NEXT: Resource booking (@14c):
# CHECK-NEXT: CortexA55UnitALU(0) = 3
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@14c):
# CHECK-NEXT: Instance 0 available @14c
# CHECK-NEXT: Instance 1 available @14c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @14c
# CHECK-NEXT: BotQ.A BotLatency SU(0) 13c
# CHECK-NEXT: BotQ.A @14c
# CHECK-NEXT: Retired: 10
# CHECK-NEXT: Executed: 14c
# CHECK-NEXT: Critical: 6c, 12 CortexA55UnitFPALU
# CHECK-NEXT: ExpectedLatency: 13c # CHECK-NEXT: ExpectedLatency: 13c
# CHECK-NEXT: - Latency limited. # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 14 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node # CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Cycle: 15 BotQ.A # CHECK-NEXT: Queue BotQ.P: 0
# CHECK-NEXT: Resource booking (@15c): # CHECK-NEXT: Queue BotQ.A: 1
# CHECK-NEXT: CortexA55UnitALU(0) = 14 # CHECK-NEXT: Scheduling SU(1) %2:fpr128 = COPY $q2
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: Ready @13c
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU +1x2u
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: Resource booking (@13c):
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@15c): # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: Instance 0 available @15c # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: Instance 1 available @15c # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @15c # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: Queue BotQ.P: 2 4 # CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: Queue BotQ.A: 1 # CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: Scheduling SU(1) %1:fpr128 = COPY $q1 # CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: Ready @15c # CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitALU +1x1u # CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: Resource booking (@15c): # CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(0) = 14 # CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: getNextResourceCycle (@13c):
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12 # CHECK-NEXT: Instance 0 available @13c
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13 # CHECK-NEXT: Instance 1 available @13c
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: Instance 2 available @13c
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: selecting CortexA510UnitALU[0] available @13c
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: BotQ.A @13c
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: Retired: 12
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: Executed: 13c
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: Critical: 3c, 7 CortexA510UnitVALU
# CHECK-NEXT: getNextResourceCycle (@15c):
# CHECK-NEXT: Instance 0 available @15c
# CHECK-NEXT: Instance 1 available @15c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @15c
# CHECK-NEXT: Resource booking (@15c):
# CHECK-NEXT: CortexA55UnitALU(0) = 14
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@15c):
# CHECK-NEXT: Instance 0 available @15c
# CHECK-NEXT: Instance 1 available @15c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @15c
# CHECK-NEXT: BotQ.A @15c
# CHECK-NEXT: Retired: 11
# CHECK-NEXT: Executed: 15c
# CHECK-NEXT: Critical: 6c, 12 CortexA55UnitFPALU
# CHECK-NEXT: ExpectedLatency: 13c # CHECK-NEXT: ExpectedLatency: 13c
# CHECK-NEXT: - Latency limited. # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 15 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node # CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Cycle: 14 BotQ.A
# CHECK-NEXT: Cycle: 16 BotQ.A # CHECK-NEXT: Cycle: 16 BotQ.A
# CHECK-NEXT: Resource booking (@16c): # CHECK-NEXT: Queue BotQ.P:
# CHECK-NEXT: CortexA55UnitALU(0) = 15 # CHECK-NEXT: Queue BotQ.A: 0
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: Scheduling SU(0) %6:fpr128 = MOVIv2d_ns 17
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@16c):
# CHECK-NEXT: Instance 0 available @16c
# CHECK-NEXT: Instance 1 available @16c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @16c
# CHECK-NEXT: Resource booking (@16c):
# CHECK-NEXT: CortexA55UnitALU(0) = 15
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@16c):
# CHECK-NEXT: Instance 0 available @16c
# CHECK-NEXT: Instance 1 available @16c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @16c
# CHECK-NEXT: Queue BotQ.P:
# CHECK-NEXT: Queue BotQ.A: 2 4
# CHECK-NEXT: Cand SU(2) ORDER
# CHECK-NEXT: Cand SU(4) PHYS-REG
# CHECK-NEXT: Pick Bot PHYS-REG
# CHECK-NEXT: Scheduling SU(4) %6:fpr128 = MOVIv2d_ns 17
# CHECK-NEXT: Ready @16c # CHECK-NEXT: Ready @16c
# CHECK-NEXT: CortexA55UnitFPALU +2x1u # CHECK-NEXT: CortexA510UnitVALU +1x3u
# CHECK-NEXT: Resource booking (@16c): # CHECK-NEXT: Resource booking (@16c):
# CHECK-NEXT: CortexA55UnitALU(0) = 15 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@16c): # CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@16c):
# CHECK-NEXT: Instance 0 available @16c # CHECK-NEXT: Instance 0 available @16c
# CHECK-NEXT: Instance 1 available @16c # CHECK-NEXT: Instance 1 available @16c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @16c # CHECK-NEXT: selecting CortexA510UnitVALU[0] available @16c
# CHECK-NEXT: Resource booking (@16c): # CHECK-NEXT: BotQ.A BotLatency SU(0) 16c
# CHECK-NEXT: CortexA55UnitALU(0) = 15 # CHECK-NEXT: BotQ.A @16c
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 12
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@16c):
# CHECK-NEXT: Instance 0 available @16c
# CHECK-NEXT: Instance 1 available @16c
# CHECK-NEXT: selecting CortexA55UnitFPALU[0] available @16c
# CHECK-NEXT: BotQ.A BotLatency SU(4) 14c
# CHECK-NEXT: Bump cycle to begin group
# CHECK-NEXT: Cycle: 17 BotQ.A
# CHECK-NEXT: BotQ.A @17c
# CHECK-NEXT: Retired: 12
# CHECK-NEXT: Executed: 17c
# CHECK-NEXT: Critical: 7c, 14 CortexA55UnitFPALU
# CHECK-NEXT: ExpectedLatency: 14c
# CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 15
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 16
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: Resource booking (@17c):
# CHECK-NEXT: CortexA55UnitALU(0) = 15
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 16
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@17c):
# CHECK-NEXT: Instance 0 available @17c
# CHECK-NEXT: Instance 1 available @17c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @17c
# CHECK-NEXT: Queue BotQ.P:
# CHECK-NEXT: Queue BotQ.A: 2
# CHECK-NEXT: Scheduling SU(2) %0:fpr128 = COPY $q0
# CHECK-NEXT: Ready @17c
# CHECK-NEXT: CortexA55UnitALU +1x1u
# CHECK-NEXT: Resource booking (@17c):
# CHECK-NEXT: CortexA55UnitALU(0) = 15
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 16
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@17c):
# CHECK-NEXT: Instance 0 available @17c
# CHECK-NEXT: Instance 1 available @17c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @17c
# CHECK-NEXT: Resource booking (@17c):
# CHECK-NEXT: CortexA55UnitALU(0) = 15
# CHECK-NEXT: CortexA55UnitALU(1) = 3
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 16
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295
# CHECK-NEXT: getNextResourceCycle (@17c):
# CHECK-NEXT: Instance 0 available @17c
# CHECK-NEXT: Instance 1 available @17c
# CHECK-NEXT: selecting CortexA55UnitALU[0] available @17c
# CHECK-NEXT: BotQ.A @17c
# CHECK-NEXT: Retired: 13 # CHECK-NEXT: Retired: 13
# CHECK-NEXT: Executed: 17c # CHECK-NEXT: Executed: 16c
# CHECK-NEXT: Critical: 7c, 14 CortexA55UnitFPALU # CHECK-NEXT: Critical: 4c, 8 CortexA510UnitVALU
# CHECK-NEXT: ExpectedLatency: 14c # CHECK-NEXT: ExpectedLatency: 16c
# CHECK-NEXT: - Latency limited. # CHECK-NEXT: - Latency limited.
# CHECK-NEXT: CortexA55UnitALU(0) = 17 # CHECK-NEXT: CortexA510UnitALU(0) = 4294967295
# CHECK-NEXT: CortexA55UnitALU(1) = 3 # CHECK-NEXT: CortexA510UnitALU(1) = 4294967295
# CHECK-NEXT: CortexA55UnitB(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU(2) = 4294967295
# CHECK-NEXT: CortexA55UnitDiv(0) = 4294967295 # CHECK-NEXT: CortexA510UnitALU0(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(0) = 16 # CHECK-NEXT: CortexA510UnitALU12(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPALU(1) = 13 # CHECK-NEXT: CortexA510UnitALU12(1) = 4294967295
# CHECK-NEXT: CortexA55UnitFPDIV(0) = 4294967295 # CHECK-NEXT: CortexA510UnitB(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitDiv(0) = 4294967295
# CHECK-NEXT: CortexA55UnitFPMAC(1) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(0) = 4294967295
# CHECK-NEXT: CortexA55UnitLd(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd(1) = 4294967295
# CHECK-NEXT: CortexA55UnitMAC(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLd1(0) = 4294967295
# CHECK-NEXT: CortexA55UnitSt(0) = 4294967295 # CHECK-NEXT: CortexA510UnitLdSt(0) = 4294967295
# CHECK-NEXT: CortexA510UnitMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitPAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU0(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVALU1(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(0) = 4294967295
# CHECK-NEXT: CortexA510UnitVMAC(1) = 4294967295
# CHECK-NEXT: CortexA510UnitVMC(0) = 4294967295
# CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node # CHECK-NEXT: ** ScheduleDAGMILive::schedule picking next node
# CHECK-NEXT: *** Final schedule for %bb.0 *** # CHECK-NEXT: *** Final schedule for %bb.0 ***
# CHECK-NEXT: * Schedule table (BottomUp): # CHECK-NEXT: * Schedule table (BottomUp):
# CHECK-NEXT: i: issue # CHECK-NEXT: i: issue
# CHECK-NEXT: x: resource booked # CHECK-NEXT: x: resource booked
# CHECK-NEXT: Cycle | 17 | 16 | 15 | 14 | 13 | 12 | 11 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | # CHECK-NEXT: Cycle | 16 | 15 | 14 | 13 | 12 | 11 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 |
# CHECK-NEXT: SU(2) | i | | | | | | | | | | | | | | | # CHECK-NEXT: SU(0) | i | | | | | | | | | | | | | | | |
# CHECK-NEXT: CortexA55UnitALU | x | | | | | | | | | | | | | | | # CHECK-NEXT: CortexA510UnitVALU | x | | | | | | | | | | | | | | | |
# CHECK-NEXT: SU(4) | | i | | | | | | | | | | | | | | # CHECK-NEXT: SU(1) | | | | i | | | | | | | | | | | | |
# CHECK-NEXT: CortexA55UnitFPALU | | x | x | | | | | | | | | | | | | # CHECK-NEXT: CortexA510UnitALU | | | | x | | | | | | | | | | | | |
# CHECK-NEXT: SU(1) | | | i | | | | | | | | | | | | | # CHECK-NEXT: SU(2) | | | | i | | | | | | | | | | | | |
# CHECK-NEXT: CortexA55UnitALU | | | x | | | | | | | | | | | | | # CHECK-NEXT: CortexA510UnitALU | | | | x | | | | | | | | | | | | |
# CHECK-NEXT: SU(0) | | | | i | | | | | | | | | | | | # CHECK-NEXT: SU(3) | | | | | i | | | | | | | | | | | |
# CHECK-NEXT: CortexA55UnitALU | | | | x | | | | | | | | | | | | # CHECK-NEXT: CortexA510UnitVALU | | | | | x | | | | | | | | | | | |
# CHECK-NEXT: SU(3) | | | | | i | | | | | | | | | | | # CHECK-NEXT: SU(4) | | | | | i | | | | | | | | | | | |
# CHECK-NEXT: CortexA55UnitFPALU | | | | | x | x | | | | | | | | | | # CHECK-NEXT: CortexA510UnitVALU | | | | | x | | | | | | | | | | | |
# CHECK-NEXT: SU(6) | | | | | | i | | | | | | | | | | # CHECK-NEXT: SU(5) | | | | | | | i | | | | | | | | | |
# CHECK-NEXT: CortexA55UnitFPALU | | | | | | x | x | | | | | | | | | # CHECK-NEXT: CortexA510UnitALU | | | | | | | x | | | | | | | | | |
# CHECK-NEXT: SU(5) | | | | | | | | i | | | | | | | | # CHECK-NEXT: SU(6) | | | | | | | | i | | | | | | | | |
# CHECK-NEXT: CortexA55UnitFPALU | | | | | | | | x | x | | | | | | | # CHECK-NEXT: CortexA510UnitVALU | | | | | | | | x | | | | | | | | |
# CHECK-NEXT: SU(7) | | | | | | | | i | | | | | | | | # CHECK-NEXT: SU(7) | | | | | | | | i | | | | | | | | |
# CHECK-NEXT: CortexA55UnitFPALU | | | | | | | | x | | | | | | | | # CHECK-NEXT: CortexA510UnitVALU | | | | | | | | x | | | | | | | | |
# CHECK-NEXT: SU(9) | | | | | | | | | i | | | | | | | # CHECK-NEXT: SU(8) | | | | | | | | i | | | | | | | | |
# CHECK-NEXT: CortexA55UnitFPALU | | | | | | | | | x | | | | | | | # CHECK-NEXT: CortexA510UnitVALU | | | | | | | | x | | | | | | | | |
# CHECK-NEXT: SU(8) | | | | | | | | | | i | | | | | | # CHECK-NEXT: SU(9) | | | | | | | | | | | | i | | | | |
# CHECK-NEXT: CortexA55UnitFPALU | | | | | | | | | | x | x | | | | | # CHECK-NEXT: CortexA510UnitVALU | | | | | | | | | | | | x | | | | |
# CHECK-NEXT: SU(10) | | | | | | | | | | | i | | | | | # CHECK-NEXT: SU(10) | | | | | | | | | | | | i | | | | |
# CHECK-NEXT: CortexA55UnitFPALU | | | | | | | | | | | x | x | | | | # CHECK-NEXT: CortexA510UnitVALU | | | | | | | | | | | | x | | | | |
# CHECK-NEXT: SU(11) | | | | | | | | | | | | | | | i | # CHECK-NEXT: SU(11) | | | | | | | | | | | | | | | | i |
# CHECK-NEXT: CortexA55UnitALU | | | | | | | | | | | | | | | x | # CHECK-NEXT: CortexA510UnitALU | | | | | | | | | | | | | | | | x |
# CHECK-NEXT: SU(12) | | | | | | | | | | | | | | | i | # CHECK-NEXT: SU(12) | | | | | | | | | | | | | | | | i |
# CHECK-NEXT: CortexA55UnitALU | | | | | | | | | | | | | | | x | # CHECK-NEXT: CortexA510UnitALU | | | | | | | | | | | | | | | | x |
# CHECK-NEXT: SU(2) [TopReadyCycle = 0, BottomReadyCycle = 17]: %0:fpr128 = COPY $q0 # CHECK-NEXT: SU(0) [TopReadyCycle = 0, BottomReadyCycle = 16]: %6:fpr128 = MOVIv2d_ns 17
# CHECK-NEXT: SU(4) [TopReadyCycle = 0, BottomReadyCycle = 16]: %6:fpr128 = MOVIv2d_ns 17 # CHECK-NEXT: SU(1) [TopReadyCycle = 0, BottomReadyCycle = 13]: %2:fpr128 = COPY $q2
# CHECK-NEXT: SU(1) [TopReadyCycle = 0, BottomReadyCycle = 15]: %1:fpr128 = COPY $q1 # CHECK-NEXT: SU(2) [TopReadyCycle = 0, BottomReadyCycle = 13]: %1:fpr128 = COPY $q1
# CHECK-NEXT: SU(0) [TopReadyCycle = 0, BottomReadyCycle = 14]: %2:fpr128 = COPY $q2 # CHECK-NEXT: SU(3) [TopReadyCycle = 0, BottomReadyCycle = 12]: %7:fpr128 = ANDv16i8 %2:fpr128, %6:fpr128
# CHECK-NEXT: SU(3) [TopReadyCycle = 0, BottomReadyCycle = 13]: %3:fpr128 = EXTv16i8 %0:fpr128, %0:fpr128, 8 # CHECK-NEXT: SU(4) [TopReadyCycle = 0, BottomReadyCycle = 12]: %8:fpr128 = ANDv16i8 %1:fpr128, %6:fpr128
# CHECK-NEXT: SU(6) [TopReadyCycle = 0, BottomReadyCycle = 12]: %8:fpr128 = ANDv16i8 %1:fpr128, %6:fpr128 # CHECK-NEXT: SU(5) [TopReadyCycle = 0, BottomReadyCycle = 10]: %0:fpr128 = COPY $q0
# CHECK-NEXT: SU(5) [TopReadyCycle = 0, BottomReadyCycle = 10]: %7:fpr128 = ANDv16i8 %2:fpr128, %6:fpr128 # CHECK-NEXT: SU(6) [TopReadyCycle = 0, BottomReadyCycle = 9]: %3:fpr128 = EXTv16i8 %0:fpr128, %0:fpr128, 8
# CHECK-NEXT: SU(7) [TopReadyCycle = 0, BottomReadyCycle = 10]: %9:fpr64 = XTNv4i16 %8:fpr128 # CHECK-NEXT: SU(7) [TopReadyCycle = 0, BottomReadyCycle = 9]: %9:fpr64 = XTNv4i16 %8:fpr128
# CHECK-NEXT: SU(9) [TopReadyCycle = 0, BottomReadyCycle = 9]: %11:fpr64 = XTNv4i16 %7:fpr128 # CHECK-NEXT: SU(8) [TopReadyCycle = 0, BottomReadyCycle = 9]: %11:fpr64 = XTNv4i16 %7:fpr128
# CHECK-NEXT: SU(8) [TopReadyCycle = 0, BottomReadyCycle = 8]: %10:fpr128 = UMULLv4i16_v4i32 %0.dsub:fpr128, %9:fpr64 # CHECK-NEXT: SU(9) [TopReadyCycle = 0, BottomReadyCycle = 5]: %10:fpr128 = UMULLv4i16_v4i32 %0.dsub:fpr128, %9:fpr64
# CHECK-NEXT: SU(10) [TopReadyCycle = 0, BottomReadyCycle = 7]: %12:fpr128 = UMULLv4i16_v4i32 %3.dsub:fpr128, %11:fpr64 # CHECK-NEXT: SU(10) [TopReadyCycle = 0, BottomReadyCycle = 5]: %12:fpr128 = UMULLv4i16_v4i32 %3.dsub:fpr128, %11:fpr64
# CHECK-NEXT: SU(11) [TopReadyCycle = 0, BottomReadyCycle = 3]: $q0 = COPY %10:fpr128 # CHECK-NEXT: SU(11) [TopReadyCycle = 0, BottomReadyCycle = 1]: $q0 = COPY %10:fpr128
# CHECK-NEXT: SU(12) [TopReadyCycle = 0, BottomReadyCycle = 3]: $q1 = COPY %12:fpr128 # CHECK-NEXT: SU(12) [TopReadyCycle = 0, BottomReadyCycle = 1]: $q1 = COPY %12:fpr128
# CHECK-EMPTY: # CHECK-EMPTY:
# CHECK-NEXT: ********** INTERVALS ********** # CHECK-NEXT: ********** INTERVALS **********
# CHECK-NEXT: B0 [0B,48r:0)[192r,224r:1) 0@0B-phi 1@192r # CHECK-NEXT: B0 [0B,96r:0)[192r,224r:1) 0@0B-phi 1@192r
# CHECK-NEXT: B1 [0B,88r:0)[208r,224r:1) 0@0B-phi 1@208r # CHECK-NEXT: B1 [0B,48r:0)[208r,224r:1) 0@0B-phi 1@208r
# CHECK-NEXT: B2 [0B,96r:0) 0@0B-phi # CHECK-NEXT: B2 [0B,32r:0) 0@0B-phi
# CHECK-NEXT: %0 [48r,168r:0) 0@48r weight:0.000000e+00 # CHECK-NEXT: %0 [96r,160r:0) 0@96r weight:0.000000e+00
# CHECK-NEXT: %1 [88r,120r:0) 0@88r weight:0.000000e+00 # CHECK-NEXT: %1 [48r,80r:0) 0@48r weight:0.000000e+00
# CHECK-NEXT: %2 [96r,128r:0) 0@96r weight:0.000000e+00 # CHECK-NEXT: %2 [32r,64r:0) 0@32r weight:0.000000e+00
# CHECK-NEXT: %3 [104r,176r:0) 0@104r weight:0.000000e+00 # CHECK-NEXT: %3 [112r,176r:0) 0@112r weight:0.000000e+00
# CHECK-NEXT: %6 [80r,128r:0) 0@80r weight:0.000000e+00 # CHECK-NEXT: %6 [16r,80r:0) 0@16r weight:0.000000e+00
# CHECK-NEXT: %7 [128r,160r:0) 0@128r weight:0.000000e+00 # CHECK-NEXT: %7 [64r,144r:0) 0@64r weight:0.000000e+00
# CHECK-NEXT: %8 [120r,136r:0) 0@120r weight:0.000000e+00 # CHECK-NEXT: %8 [80r,128r:0) 0@80r weight:0.000000e+00
# CHECK-NEXT: %9 [136r,168r:0) 0@136r weight:0.000000e+00 # CHECK-NEXT: %9 [128r,160r:0) 0@128r weight:0.000000e+00
# CHECK-NEXT: %10 [168r,192r:0) 0@168r weight:0.000000e+00 # CHECK-NEXT: %10 [160r,192r:0) 0@160r weight:0.000000e+00
# CHECK-NEXT: %11 [160r,176r:0) 0@160r weight:0.000000e+00 # CHECK-NEXT: %11 [144r,176r:0) 0@144r weight:0.000000e+00
# CHECK-NEXT: %12 [176r,208r:0) 0@176r weight:0.000000e+00 # CHECK-NEXT: %12 [176r,208r:0) 0@176r weight:0.000000e+00
# CHECK-NEXT: RegMasks: # CHECK-NEXT: RegMasks:
# CHECK-NEXT: ********** MACHINEINSTRS ********** # CHECK-NEXT: ********** MACHINEINSTRS **********
# CHECK-NEXT: # Machine code for function umull_and_v8i32: IsSSA, NoPHIs, TracksLiveness # CHECK-NEXT: # Machine code for function umull_and_v8i32: IsSSA, NoPHIs, TracksLiveness
# CHECK-NEXT: Function Live Ins: $q0 in %0, $q1 in %1, $q2 in %2 # CHECK-NEXT: Function Live Ins: $q0 in %0, $q1 in %1, $q2 in %2
# CHECK-EMPTY: # CHECK-EMPTY:
# CHECK-NEXT: 0B bb.0.entry: # CHECK-NEXT: 0B bb.0.entry:
# CHECK-NEXT: liveins: $q0, $q1, $q2 # CHECK-NEXT: liveins: $q0, $q1, $q2
# CHECK-NEXT: 48B %0:fpr128 = COPY $q0 # CHECK-NEXT: 16B %6:fpr128 = MOVIv2d_ns 17
# CHECK-NEXT: 80B %6:fpr128 = MOVIv2d_ns 17 # CHECK-NEXT: 32B %2:fpr128 = COPY $q2
# CHECK-NEXT: 88B %1:fpr128 = COPY $q1 # CHECK-NEXT: 48B %1:fpr128 = COPY $q1
# CHECK-NEXT: 96B %2:fpr128 = COPY $q2 # CHECK-NEXT: 64B %7:fpr128 = ANDv16i8 %2:fpr128, %6:fpr128
# CHECK-NEXT: 104B %3:fpr128 = EXTv16i8 %0:fpr128, %0:fpr128, 8 # CHECK-NEXT: 80B %8:fpr128 = ANDv16i8 %1:fpr128, %6:fpr128
# CHECK-NEXT: 120B %8:fpr128 = ANDv16i8 %1:fpr128, %6:fpr128 # CHECK-NEXT: 96B %0:fpr128 = COPY $q0
# CHECK-NEXT: 128B %7:fpr128 = ANDv16i8 %2:fpr128, %6:fpr128 # CHECK-NEXT: 112B %3:fpr128 = EXTv16i8 %0:fpr128, %0:fpr128, 8
# CHECK-NEXT: 136B %9:fpr64 = XTNv4i16 %8:fpr128 # CHECK-NEXT: 128B %9:fpr64 = XTNv4i16 %8:fpr128
# CHECK-NEXT: 160B %11:fpr64 = XTNv4i16 %7:fpr128 # CHECK-NEXT: 144B %11:fpr64 = XTNv4i16 %7:fpr128
# CHECK-NEXT: 168B %10:fpr128 = UMULLv4i16_v4i32 %0.dsub:fpr128, %9:fpr64 # CHECK-NEXT: 160B %10:fpr128 = UMULLv4i16_v4i32 %0.dsub:fpr128, %9:fpr64
# CHECK-NEXT: 176B %12:fpr128 = UMULLv4i16_v4i32 %3.dsub:fpr128, %11:fpr64 # CHECK-NEXT: 176B %12:fpr128 = UMULLv4i16_v4i32 %3.dsub:fpr128, %11:fpr64
# CHECK-NEXT: 192B $q0 = COPY %10:fpr128 # CHECK-NEXT: 192B $q0 = COPY %10:fpr128
# CHECK-NEXT: 208B $q1 = COPY %12:fpr128 # CHECK-NEXT: 208B $q1 = COPY %12:fpr128
# CHECK-NEXT: 224B RET_ReallyLR implicit $q0, implicit $q1 # CHECK-NEXT: 224B RET_ReallyLR implicit $q0, implicit $q1
# CHECK-EMPTY: No newline at end of file
# CHECK-NEXT: # End machine code for function umull_and_v8i32.
Context not available.

llvm/test/CodeGen/AArch64/misched-fusion-lit.ll

Context not available.
; CHECK-LABEL: litp_tune_generic: ; CHECK-LABEL: litp_tune_generic:
; CHECK: adrp [[R:x[0-9]+]], litp_tune_generic ; CHECK: adrp [[R:x[0-9]+]], litp_tune_generic
; CHECKDONT: add {{w[0-9]+}}, {{w[0-9]+}}, {{w[0-9]+}}
; CHECK-NEXT: add {{x[0-9]+}}, [[R]], :lo12:litp_tune_generic ; CHECK-NEXT: add {{x[0-9]+}}, [[R]], :lo12:litp_tune_generic
} }
Context not available.

llvm/test/CodeGen/AArch64/mul_pow2.ll

llvm/test/CodeGen/AArch64/mulcmle.ll

llvm/test/CodeGen/AArch64/named-vector-shuffles-neon.ll

Context not available.
define <8 x i32> @splice_v8i32_idx(<8 x i32> %a, <8 x i32> %b) #0 { define <8 x i32> @splice_v8i32_idx(<8 x i32> %a, <8 x i32> %b) #0 {
; CHECK-LABEL: splice_v8i32_idx: ; CHECK-LABEL: splice_v8i32_idx:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: ext v3.16b, v2.16b, v3.16b, #4
; CHECK-NEXT: ext v0.16b, v1.16b, v2.16b, #4 ; CHECK-NEXT: ext v0.16b, v1.16b, v2.16b, #4
; CHECK-NEXT: mov v1.16b, v3.16b ; CHECK-NEXT: ext v1.16b, v2.16b, v3.16b, #4
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <8 x i32> @llvm.experimental.vector.splice.v8i32(<8 x i32> %a, <8 x i32> %b, i32 5) %res = call <8 x i32> @llvm.experimental.vector.splice.v8i32(<8 x i32> %a, <8 x i32> %b, i32 5)
ret <8 x i32> %res ret <8 x i32> %res
Context not available.
define <16 x float> @splice_v16f32_idx(<16 x float> %a, <16 x float> %b) #0 { define <16 x float> @splice_v16f32_idx(<16 x float> %a, <16 x float> %b) #0 {
; CHECK-LABEL: splice_v16f32_idx: ; CHECK-LABEL: splice_v16f32_idx:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: ext v5.16b, v4.16b, v5.16b, #12 ; CHECK-NEXT: ext v6.16b, v3.16b, v4.16b, #12
; CHECK-NEXT: ext v6.16b, v2.16b, v3.16b, #12
; CHECK-NEXT: ext v0.16b, v1.16b, v2.16b, #12 ; CHECK-NEXT: ext v0.16b, v1.16b, v2.16b, #12
; CHECK-NEXT: ext v2.16b, v3.16b, v4.16b, #12 ; CHECK-NEXT: ext v1.16b, v2.16b, v3.16b, #12
; CHECK-NEXT: mov v3.16b, v5.16b ; CHECK-NEXT: ext v3.16b, v4.16b, v5.16b, #12
; CHECK-NEXT: mov v1.16b, v6.16b ; CHECK-NEXT: mov v2.16b, v6.16b
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <16 x float> @llvm.experimental.vector.splice.v16f32(<16 x float> %a, <16 x float> %b, i32 7) %res = call <16 x float> @llvm.experimental.vector.splice.v16f32(<16 x float> %a, <16 x float> %b, i32 7)
ret <16 x float> %res ret <16 x float> %res
Context not available.
define <8 x i32> @splice_v8i32(<8 x i32> %a, <8 x i32> %b) #0 { define <8 x i32> @splice_v8i32(<8 x i32> %a, <8 x i32> %b) #0 {
; CHECK-LABEL: splice_v8i32: ; CHECK-LABEL: splice_v8i32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: ext v3.16b, v2.16b, v3.16b, #4
; CHECK-NEXT: ext v0.16b, v1.16b, v2.16b, #4 ; CHECK-NEXT: ext v0.16b, v1.16b, v2.16b, #4
; CHECK-NEXT: mov v1.16b, v3.16b ; CHECK-NEXT: ext v1.16b, v2.16b, v3.16b, #4
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <8 x i32> @llvm.experimental.vector.splice.v8i32(<8 x i32> %a, <8 x i32> %b, i32 -3) %res = call <8 x i32> @llvm.experimental.vector.splice.v8i32(<8 x i32> %a, <8 x i32> %b, i32 -3)
ret <8 x i32> %res ret <8 x i32> %res
Context not available.
define <16 x float> @splice_v16f32(<16 x float> %a, <16 x float> %b) #0 { define <16 x float> @splice_v16f32(<16 x float> %a, <16 x float> %b) #0 {
; CHECK-LABEL: splice_v16f32: ; CHECK-LABEL: splice_v16f32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: ext v5.16b, v4.16b, v5.16b, #12 ; CHECK-NEXT: ext v6.16b, v3.16b, v4.16b, #12
; CHECK-NEXT: ext v6.16b, v2.16b, v3.16b, #12
; CHECK-NEXT: ext v0.16b, v1.16b, v2.16b, #12 ; CHECK-NEXT: ext v0.16b, v1.16b, v2.16b, #12
; CHECK-NEXT: ext v2.16b, v3.16b, v4.16b, #12 ; CHECK-NEXT: ext v1.16b, v2.16b, v3.16b, #12
; CHECK-NEXT: mov v3.16b, v5.16b ; CHECK-NEXT: ext v3.16b, v4.16b, v5.16b, #12
; CHECK-NEXT: mov v1.16b, v6.16b ; CHECK-NEXT: mov v2.16b, v6.16b
dmgreenUnsubmitted
Done
ReplyInline Actions

Some of these have less movs thanks to the lower latencies, which is nice.

dmgreen: Some of these have less movs thanks to the lower latencies, which is nice.
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call <16 x float> @llvm.experimental.vector.splice.v16f32(<16 x float> %a, <16 x float> %b, i32 -9) %res = call <16 x float> @llvm.experimental.vector.splice.v16f32(<16 x float> %a, <16 x float> %b, i32 -9)
ret <16 x float> %res ret <16 x float> %res
Context not available.

llvm/test/CodeGen/AArch64/named-vector-shuffles-sve.ll

llvm/test/CodeGen/AArch64/neg-imm.ll

llvm/test/CodeGen/AArch64/neon-abd.ll

llvm/test/CodeGen/AArch64/neon-bitwise-instructions.ll

llvm/test/CodeGen/AArch64/neon-compare-instructions.ll

llvm/test/CodeGen/AArch64/neon-dotpattern.ll

llvm/test/CodeGen/AArch64/neon-dotreduce.ll

llvm/test/CodeGen/AArch64/neon-extadd.ll

llvm/test/CodeGen/AArch64/neon-extracttruncate.ll

llvm/test/CodeGen/AArch64/neon-mov.ll

llvm/test/CodeGen/AArch64/neon-reverseshuffle.ll

llvm/test/CodeGen/AArch64/neon-rshrn.ll

llvm/test/CodeGen/AArch64/neon-shift-neg.ll

llvm/test/CodeGen/AArch64/neon-truncstore.ll

llvm/test/CodeGen/AArch64/neon-wide-splat.ll

llvm/test/CodeGen/AArch64/neon-widen-shuffle.ll

llvm/test/CodeGen/AArch64/nontemporal-load.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc --mattr=+sve < %s -mtriple aarch64-apple-darwin | FileCheck %s ; RUN: llc --mattr=+sve < %s -mtriple aarch64-apple-darwin | FileCheck %s
; RUN: llc --mattr=+sve < %s -mtriple aarch64_be-unknown-unknown | FileCheck --check-prefix CHECK-BE %s ; RUN: llc --mattr=+sve < %s -mtriple aarch64_be-unknown-unknown | FileCheck --check-prefix CHECK-BE %s
define <4 x double> @test_ldnp_v4f64(ptr %A) { define <4 x double> @test_ldnp_v4f64(ptr %A) {
;
dmgreenUnsubmitted
Done
ReplyInline Actions

These newlines and blank ; lines look funny.

dmgreen: These newlines and blank ; lines look funny.
; CHECK-LABEL: test_ldnp_v4f64: ; CHECK-LABEL: test_ldnp_v4f64:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <4 x double>, ptr %A, align 8, !nontemporal !0 %lv = load <4 x double>, ptr %A, align 8, !nontemporal !0
ret <4 x double> %lv ret <4 x double> %lv
} }
define <4 x i64> @test_ldnp_v4i64(ptr %A) { define <4 x i64> @test_ldnp_v4i64(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v4i64: ; CHECK-LABEL: test_ldnp_v4i64:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <4 x i64>, ptr %A, align 8, !nontemporal !0 %lv = load <4 x i64>, ptr %A, align 8, !nontemporal !0
ret <4 x i64> %lv ret <4 x i64> %lv
} }
define <8 x i32> @test_ldnp_v8i32(ptr %A) { define <8 x i32> @test_ldnp_v8i32(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v8i32: ; CHECK-LABEL: test_ldnp_v8i32:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <8 x i32>, ptr %A, align 8, !nontemporal !0 %lv = load <8 x i32>, ptr %A, align 8, !nontemporal !0
ret <8 x i32> %lv ret <8 x i32> %lv
} }
define <8 x float> @test_ldnp_v8f32(ptr %A) { define <8 x float> @test_ldnp_v8f32(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v8f32: ; CHECK-LABEL: test_ldnp_v8f32:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <8 x float>, ptr %A, align 8, !nontemporal !0 %lv = load <8 x float>, ptr %A, align 8, !nontemporal !0
ret <8 x float> %lv ret <8 x float> %lv
} }
define <16 x i16> @test_ldnp_v16i16(ptr %A) { define <16 x i16> @test_ldnp_v16i16(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v16i16: ; CHECK-LABEL: test_ldnp_v16i16:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <16 x i16>, ptr %A, align 8, !nontemporal !0 %lv = load <16 x i16>, ptr %A, align 8, !nontemporal !0
ret <16 x i16> %lv ret <16 x i16> %lv
} }
define <16 x half> @test_ldnp_v16f16(ptr %A) { define <16 x half> @test_ldnp_v16f16(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v16f16: ; CHECK-LABEL: test_ldnp_v16f16:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <16 x half>, ptr %A, align 8, !nontemporal !0 %lv = load <16 x half>, ptr %A, align 8, !nontemporal !0
ret <16 x half> %lv ret <16 x half> %lv
} }
define <32 x i8> @test_ldnp_v32i8(ptr %A) { define <32 x i8> @test_ldnp_v32i8(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v32i8: ; CHECK-LABEL: test_ldnp_v32i8:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <32 x i8>, ptr %A, align 8, !nontemporal !0 %lv = load <32 x i8>, ptr %A, align 8, !nontemporal !0
ret <32 x i8> %lv ret <32 x i8> %lv
} }
define <4 x i32> @test_ldnp_v4i32(ptr %A) { define <4 x i32> @test_ldnp_v4i32(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v4i32: ; CHECK-LABEL: test_ldnp_v4i32:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldr q0, [x0] ; CHECK-NEXT: ldr q0, [x0]
Context not available.
%lv = load<4 x i32>, ptr %A, align 8, !nontemporal !0 %lv = load<4 x i32>, ptr %A, align 8, !nontemporal !0
ret <4 x i32> %lv ret <4 x i32> %lv
} }
define <4 x float> @test_ldnp_v4f32(ptr %A) { define <4 x float> @test_ldnp_v4f32(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v4f32: ; CHECK-LABEL: test_ldnp_v4f32:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldr q0, [x0] ; CHECK-NEXT: ldr q0, [x0]
Context not available.
%lv = load<4 x float>, ptr %A, align 8, !nontemporal !0 %lv = load<4 x float>, ptr %A, align 8, !nontemporal !0
ret <4 x float> %lv ret <4 x float> %lv
} }
define <8 x i16> @test_ldnp_v8i16(ptr %A) { define <8 x i16> @test_ldnp_v8i16(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v8i16: ; CHECK-LABEL: test_ldnp_v8i16:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldr q0, [x0] ; CHECK-NEXT: ldr q0, [x0]
Context not available.
%lv = load <8 x i16>, ptr %A, align 8, !nontemporal !0 %lv = load <8 x i16>, ptr %A, align 8, !nontemporal !0
ret <8 x i16> %lv ret <8 x i16> %lv
} }
define <16 x i8> @test_ldnp_v16i8(ptr %A) { define <16 x i8> @test_ldnp_v16i8(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v16i8: ; CHECK-LABEL: test_ldnp_v16i8:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldr q0, [x0] ; CHECK-NEXT: ldr q0, [x0]
Context not available.
ret <16 x i8> %lv ret <16 x i8> %lv
} }
define <2 x double> @test_ldnp_v2f64(ptr %A) { define <2 x double> @test_ldnp_v2f64(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v2f64: ; CHECK-LABEL: test_ldnp_v2f64:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldr q0, [x0] ; CHECK-NEXT: ldr q0, [x0]
Context not available.
%lv = load <2 x double>, ptr %A, align 8, !nontemporal !0 %lv = load <2 x double>, ptr %A, align 8, !nontemporal !0
ret <2 x double> %lv ret <2 x double> %lv
} }
define <2 x i32> @test_ldnp_v2i32(ptr %A) { define <2 x i32> @test_ldnp_v2i32(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v2i32: ; CHECK-LABEL: test_ldnp_v2i32:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldr d0, [x0] ; CHECK-NEXT: ldr d0, [x0]
Context not available.
%lv = load <2 x i32>, ptr %A, align 8, !nontemporal !0 %lv = load <2 x i32>, ptr %A, align 8, !nontemporal !0
ret <2 x i32> %lv ret <2 x i32> %lv
} }
define <2 x float> @test_ldnp_v2f32(ptr %A) { define <2 x float> @test_ldnp_v2f32(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v2f32: ; CHECK-LABEL: test_ldnp_v2f32:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldr d0, [x0] ; CHECK-NEXT: ldr d0, [x0]
Context not available.
%lv = load <2 x float>, ptr %A, align 8, !nontemporal !0 %lv = load <2 x float>, ptr %A, align 8, !nontemporal !0
ret <2 x float> %lv ret <2 x float> %lv
} }
define <4 x i16> @test_ldnp_v4i16(ptr %A) { define <4 x i16> @test_ldnp_v4i16(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v4i16: ; CHECK-LABEL: test_ldnp_v4i16:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldr d0, [x0] ; CHECK-NEXT: ldr d0, [x0]
Context not available.
%lv = load <4 x i16>, ptr %A, align 8, !nontemporal !0 %lv = load <4 x i16>, ptr %A, align 8, !nontemporal !0
ret <4 x i16> %lv ret <4 x i16> %lv
} }
define <8 x i8> @test_ldnp_v8i8(ptr %A) { define <8 x i8> @test_ldnp_v8i8(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v8i8: ; CHECK-LABEL: test_ldnp_v8i8:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldr d0, [x0] ; CHECK-NEXT: ldr d0, [x0]
Context not available.
%lv = load <8 x i8>, ptr %A, align 8, !nontemporal !0 %lv = load <8 x i8>, ptr %A, align 8, !nontemporal !0
ret <8 x i8> %lv ret <8 x i8> %lv
} }
define <1 x double> @test_ldnp_v1f64(ptr %A) { define <1 x double> @test_ldnp_v1f64(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v1f64: ; CHECK-LABEL: test_ldnp_v1f64:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldr d0, [x0] ; CHECK-NEXT: ldr d0, [x0]
Context not available.
%lv = load <1 x double>, ptr %A, align 8, !nontemporal !0 %lv = load <1 x double>, ptr %A, align 8, !nontemporal !0
ret <1 x double> %lv ret <1 x double> %lv
} }
define <1 x i64> @test_ldnp_v1i64(ptr %A) { define <1 x i64> @test_ldnp_v1i64(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v1i64: ; CHECK-LABEL: test_ldnp_v1i64:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldr d0, [x0] ; CHECK-NEXT: ldr d0, [x0]
Context not available.
%lv = load <1 x i64>, ptr %A, align 8, !nontemporal !0 %lv = load <1 x i64>, ptr %A, align 8, !nontemporal !0
ret <1 x i64> %lv ret <1 x i64> %lv
} }
define <32 x i16> @test_ldnp_v32i16(ptr %A) { define <32 x i16> @test_ldnp_v32i16(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v32i16: ; CHECK-LABEL: test_ldnp_v32i16:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <32 x i16>, ptr %A, align 8, !nontemporal !0 %lv = load <32 x i16>, ptr %A, align 8, !nontemporal !0
ret <32 x i16> %lv ret <32 x i16> %lv
} }
define <32 x half> @test_ldnp_v32f16(ptr %A) { define <32 x half> @test_ldnp_v32f16(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v32f16: ; CHECK-LABEL: test_ldnp_v32f16:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <32 x half>, ptr %A, align 8, !nontemporal !0 %lv = load <32 x half>, ptr %A, align 8, !nontemporal !0
ret <32 x half> %lv ret <32 x half> %lv
} }
define <16 x i32> @test_ldnp_v16i32(ptr %A) { define <16 x i32> @test_ldnp_v16i32(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v16i32: ; CHECK-LABEL: test_ldnp_v16i32:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <16 x i32>, ptr %A, align 8, !nontemporal !0 %lv = load <16 x i32>, ptr %A, align 8, !nontemporal !0
ret <16 x i32> %lv ret <16 x i32> %lv
} }
define <16 x float> @test_ldnp_v16f32(ptr %A) { define <16 x float> @test_ldnp_v16f32(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v16f32: ; CHECK-LABEL: test_ldnp_v16f32:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <16 x float>, ptr %A, align 8, !nontemporal !0 %lv = load <16 x float>, ptr %A, align 8, !nontemporal !0
ret <16 x float> %lv ret <16 x float> %lv
} }
define <17 x float> @test_ldnp_v17f32(ptr %A) { define <17 x float> @test_ldnp_v17f32(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v17f32: ; CHECK-LABEL: test_ldnp_v17f32:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0, #32] ; CHECK-NEXT: ldnp q0, q1, [x0, #32]
; CHECK-NEXT: ldnp q2, q3, [x0] ; CHECK-NEXT: ldr s2, [x0, #64]
; CHECK-NEXT: ldr s4, [x0, #64] ; CHECK-NEXT: ldnp q3, q4, [x0]
; CHECK-NEXT: stp q0, q1, [x8, #32] ; CHECK-NEXT: stp q0, q1, [x8, #32]
; CHECK-NEXT: stp q2, q3, [x8] ; CHECK-NEXT: stp q3, q4, [x8]
; CHECK-NEXT: str s4, [x8, #64] ; CHECK-NEXT: str s2, [x8, #64]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; ;
; CHECK-BE-LABEL: test_ldnp_v17f32: ; CHECK-BE-LABEL: test_ldnp_v17f32:
; CHECK-BE: // %bb.0: ; CHECK-BE: // %bb.0:
; CHECK-BE-NEXT: add x9, x0, #32
; CHECK-BE-NEXT: ld1 { v1.4s }, [x0]
; CHECK-BE-NEXT: add x10, x0, #16
; CHECK-BE-NEXT: ldr s2, [x0, #64]
; CHECK-BE-NEXT: ld1 { v0.4s }, [x9]
; CHECK-BE-NEXT: add x9, x0, #48 ; CHECK-BE-NEXT: add x9, x0, #48
; CHECK-BE-NEXT: ld1 { v4.4s }, [x10] ; CHECK-BE-NEXT: ld1 { v0.4s }, [x0]
; CHECK-BE-NEXT: add x10, x8, #32 ; CHECK-BE-NEXT: add x10, x0, #32
; CHECK-BE-NEXT: ld1 { v2.4s }, [x9]
; CHECK-BE-NEXT: add x9, x0, #16
; CHECK-BE-NEXT: ldr s1, [x0, #64]
; CHECK-BE-NEXT: ld1 { v3.4s }, [x9] ; CHECK-BE-NEXT: ld1 { v3.4s }, [x9]
; CHECK-BE-NEXT: ld1 { v4.4s }, [x10]
; CHECK-BE-NEXT: add x9, x8, #48 ; CHECK-BE-NEXT: add x9, x8, #48
; CHECK-BE-NEXT: str s2, [x8, #64] ; CHECK-BE-NEXT: str s1, [x8, #64]
; CHECK-BE-NEXT: st1 { v1.4s }, [x8] ; CHECK-BE-NEXT: add x10, x8, #32
; CHECK-BE-NEXT: st1 { v0.4s }, [x8]
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: st1 { v3.4s }, [x9] ; CHECK-BE-NEXT: st1 { v2.4s }, [x9]
; CHECK-BE-NEXT: st1 { v0.4s }, [x10] ; CHECK-BE-NEXT: st1 { v4.4s }, [x10]
; CHECK-BE-NEXT: st1 { v4.4s }, [x8] ; CHECK-BE-NEXT: st1 { v3.4s }, [x8]
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
%lv = load <17 x float>, ptr %A, align 8, !nontemporal !0 %lv = load <17 x float>, ptr %A, align 8, !nontemporal !0
ret <17 x float> %lv ret <17 x float> %lv
} }
define <33 x double> @test_ldnp_v33f64(ptr %A) { define <33 x double> @test_ldnp_v33f64(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v33f64: ; CHECK-LABEL: test_ldnp_v33f64:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
; CHECK-NEXT: ldr d20, [x0, #256]
; CHECK-NEXT: ldnp q2, q3, [x0, #32] ; CHECK-NEXT: ldnp q2, q3, [x0, #32]
; CHECK-NEXT: ldnp q4, q5, [x0, #64] ; CHECK-NEXT: ldnp q4, q5, [x0, #64]
; CHECK-NEXT: ldnp q6, q7, [x0, #96] ; CHECK-NEXT: ldnp q6, q7, [x0, #96]
; CHECK-NEXT: ldnp q16, q17, [x0, #128] ; CHECK-NEXT: ldnp q16, q17, [x0, #128]
; CHECK-NEXT: ldnp q18, q19, [x0, #224] ; CHECK-NEXT: ldnp q18, q19, [x0, #224]
; CHECK-NEXT: ldnp q20, q21, [x0, #192] ; CHECK-NEXT: ldnp q21, q22, [x0, #160]
; CHECK-NEXT: ldnp q22, q23, [x0, #160] ; CHECK-NEXT: ldnp q23, q24, [x0, #192]
; CHECK-NEXT: ldr d24, [x0, #256]
; CHECK-NEXT: stp q0, q1, [x8] ; CHECK-NEXT: stp q0, q1, [x8]
; CHECK-NEXT: stp q2, q3, [x8, #32] ; CHECK-NEXT: stp q2, q3, [x8, #32]
; CHECK-NEXT: stp q4, q5, [x8, #64] ; CHECK-NEXT: stp q4, q5, [x8, #64]
; CHECK-NEXT: stp q6, q7, [x8, #96] ; CHECK-NEXT: stp q6, q7, [x8, #96]
; CHECK-NEXT: stp q16, q17, [x8, #128] ; CHECK-NEXT: stp q16, q17, [x8, #128]
; CHECK-NEXT: stp q22, q23, [x8, #160] ; CHECK-NEXT: stp q21, q22, [x8, #160]
; CHECK-NEXT: stp q20, q21, [x8, #192] ; CHECK-NEXT: stp q23, q24, [x8, #192]
; CHECK-NEXT: stp q18, q19, [x8, #224] ; CHECK-NEXT: stp q18, q19, [x8, #224]
; CHECK-NEXT: str d24, [x8, #256] ; CHECK-NEXT: str d20, [x8, #256]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; ;
; CHECK-BE-LABEL: test_ldnp_v33f64: ; CHECK-BE-LABEL: test_ldnp_v33f64:
; CHECK-BE: // %bb.0: ; CHECK-BE: // %bb.0:
; CHECK-BE-NEXT: add x9, x0, #16 ; CHECK-BE-NEXT: add x9, x0, #16
; CHECK-BE-NEXT: add x10, x0, #32 ; CHECK-BE-NEXT: add x10, x0, #32
; CHECK-BE-NEXT: ld1 { v21.2d }, [x0] ; CHECK-BE-NEXT: add x11, x0, #48
; CHECK-BE-NEXT: add x11, x8, #208
; CHECK-BE-NEXT: ld1 { v0.2d }, [x9] ; CHECK-BE-NEXT: ld1 { v0.2d }, [x9]
; CHECK-BE-NEXT: add x9, x0, #48 ; CHECK-BE-NEXT: add x9, x0, #64
; CHECK-BE-NEXT: ld1 { v1.2d }, [x10] ; CHECK-BE-NEXT: ld1 { v1.2d }, [x10]
; CHECK-BE-NEXT: add x10, x0, #64 ; CHECK-BE-NEXT: add x10, x0, #80
; CHECK-BE-NEXT: ld1 { v2.2d }, [x9] ; CHECK-BE-NEXT: ld1 { v3.2d }, [x9]
; CHECK-BE-NEXT: add x9, x0, #80
; CHECK-BE-NEXT: ld1 { v3.2d }, [x10]
; CHECK-BE-NEXT: add x10, x0, #96
; CHECK-BE-NEXT: ld1 { v4.2d }, [x9]
; CHECK-BE-NEXT: add x9, x0, #112 ; CHECK-BE-NEXT: add x9, x0, #112
; CHECK-BE-NEXT: ld1 { v5.2d }, [x10] ; CHECK-BE-NEXT: ld1 { v4.2d }, [x10]
; CHECK-BE-NEXT: add x10, x0, #128 ; CHECK-BE-NEXT: add x10, x0, #128
; CHECK-BE-NEXT: ld1 { v6.2d }, [x9] ; CHECK-BE-NEXT: ld1 { v6.2d }, [x9]
; CHECK-BE-NEXT: add x9, x0, #144 ; CHECK-BE-NEXT: add x9, x0, #160
; CHECK-BE-NEXT: ld1 { v7.2d }, [x10] ; CHECK-BE-NEXT: ld1 { v7.2d }, [x10]
; CHECK-BE-NEXT: add x10, x0, #160 ; CHECK-BE-NEXT: add x10, x0, #176
; CHECK-BE-NEXT: ld1 { v16.2d }, [x9] ; CHECK-BE-NEXT: ld1 { v17.2d }, [x9]
; CHECK-BE-NEXT: add x9, x0, #176
; CHECK-BE-NEXT: ld1 { v17.2d }, [x10]
; CHECK-BE-NEXT: add x10, x0, #192
; CHECK-BE-NEXT: ld1 { v18.2d }, [x9]
; CHECK-BE-NEXT: add x9, x0, #224
; CHECK-BE-NEXT: ld1 { v19.2d }, [x10]
; CHECK-BE-NEXT: add x10, x0, #208
; CHECK-BE-NEXT: ld1 { v20.2d }, [x9]
; CHECK-BE-NEXT: add x9, x0, #240 ; CHECK-BE-NEXT: add x9, x0, #240
; CHECK-BE-NEXT: ldr d22, [x0, #256] ; CHECK-BE-NEXT: ld1 { v2.2d }, [x11]
; CHECK-BE-NEXT: add x11, x0, #96
; CHECK-BE-NEXT: ld1 { v18.2d }, [x10]
; CHECK-BE-NEXT: ld1 { v20.2d }, [x0]
; CHECK-BE-NEXT: ld1 { v22.2d }, [x9]
; CHECK-BE-NEXT: add x10, x0, #224
; CHECK-BE-NEXT: ld1 { v5.2d }, [x11]
; CHECK-BE-NEXT: add x11, x0, #144
; CHECK-BE-NEXT: ldr d21, [x0, #256]
; CHECK-BE-NEXT: add x9, x0, #208
; CHECK-BE-NEXT: ld1 { v24.2d }, [x10]
; CHECK-BE-NEXT: ld1 { v16.2d }, [x11]
; CHECK-BE-NEXT: add x11, x0, #192
; CHECK-BE-NEXT: ld1 { v23.2d }, [x9] ; CHECK-BE-NEXT: ld1 { v23.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #240 ; CHECK-BE-NEXT: add x9, x8, #240
; CHECK-BE-NEXT: ld1 { v24.2d }, [x10] ; CHECK-BE-NEXT: ld1 { v19.2d }, [x11]
; CHECK-BE-NEXT: add x10, x8, #224 ; CHECK-BE-NEXT: str d21, [x8, #256]
; CHECK-BE-NEXT: str d22, [x8, #256] ; CHECK-BE-NEXT: st1 { v20.2d }, [x8]
; CHECK-BE-NEXT: st1 { v21.2d }, [x8] ; CHECK-BE-NEXT: st1 { v22.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #224
; CHECK-BE-NEXT: st1 { v24.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #208
; CHECK-BE-NEXT: st1 { v23.2d }, [x9] ; CHECK-BE-NEXT: st1 { v23.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #192 ; CHECK-BE-NEXT: add x9, x8, #192
; CHECK-BE-NEXT: st1 { v20.2d }, [x10]
; CHECK-BE-NEXT: add x10, x8, #176
; CHECK-BE-NEXT: st1 { v24.2d }, [x11]
; CHECK-BE-NEXT: add x11, x8, #160
; CHECK-BE-NEXT: st1 { v19.2d }, [x9] ; CHECK-BE-NEXT: st1 { v19.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #176
; CHECK-BE-NEXT: st1 { v18.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #160
; CHECK-BE-NEXT: st1 { v17.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #144 ; CHECK-BE-NEXT: add x9, x8, #144
; CHECK-BE-NEXT: st1 { v18.2d }, [x10]
; CHECK-BE-NEXT: add x10, x8, #128
; CHECK-BE-NEXT: st1 { v17.2d }, [x11]
; CHECK-BE-NEXT: add x11, x8, #112
; CHECK-BE-NEXT: st1 { v16.2d }, [x9] ; CHECK-BE-NEXT: st1 { v16.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #128
; CHECK-BE-NEXT: st1 { v7.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #112
; CHECK-BE-NEXT: st1 { v6.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #96 ; CHECK-BE-NEXT: add x9, x8, #96
; CHECK-BE-NEXT: st1 { v7.2d }, [x10]
; CHECK-BE-NEXT: add x10, x8, #80
; CHECK-BE-NEXT: st1 { v6.2d }, [x11]
; CHECK-BE-NEXT: add x11, x8, #64
; CHECK-BE-NEXT: st1 { v5.2d }, [x9] ; CHECK-BE-NEXT: st1 { v5.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #80
; CHECK-BE-NEXT: st1 { v4.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #64
; CHECK-BE-NEXT: st1 { v3.2d }, [x9]
; CHECK-BE-NEXT: add x9, x8, #48 ; CHECK-BE-NEXT: add x9, x8, #48
; CHECK-BE-NEXT: st1 { v4.2d }, [x10]
; CHECK-BE-NEXT: add x10, x8, #32
; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: st1 { v3.2d }, [x11]
; CHECK-BE-NEXT: st1 { v2.2d }, [x9] ; CHECK-BE-NEXT: st1 { v2.2d }, [x9]
; CHECK-BE-NEXT: st1 { v1.2d }, [x10] ; CHECK-BE-NEXT: add x9, x8, #32
; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: st1 { v1.2d }, [x9]
; CHECK-BE-NEXT: st1 { v0.2d }, [x8] ; CHECK-BE-NEXT: st1 { v0.2d }, [x8]
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
%lv = load <33 x double>, ptr %A, align 8, !nontemporal !0 %lv = load <33 x double>, ptr %A, align 8, !nontemporal !0
ret <33 x double> %lv ret <33 x double> %lv
} }
define <33 x i8> @test_ldnp_v33i8(ptr %A) { define <33 x i8> @test_ldnp_v33i8(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v33i8: ; CHECK-LABEL: test_ldnp_v33i8:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
; CHECK-BE: // %bb.0: ; CHECK-BE: // %bb.0:
; CHECK-BE-NEXT: add x9, x0, #16 ; CHECK-BE-NEXT: add x9, x0, #16
; CHECK-BE-NEXT: ld1 { v0.16b }, [x0] ; CHECK-BE-NEXT: ld1 { v0.16b }, [x0]
; CHECK-BE-NEXT: add x10, x8, #16 ; CHECK-BE-NEXT: ldrb w10, [x0, #32]
; CHECK-BE-NEXT: ld1 { v1.16b }, [x9] ; CHECK-BE-NEXT: ld1 { v1.16b }, [x9]
; CHECK-BE-NEXT: ldrb w9, [x0, #32] ; CHECK-BE-NEXT: strb w10, [x8, #32]
; CHECK-BE-NEXT: strb w9, [x8, #32]
; CHECK-BE-NEXT: st1 { v0.16b }, [x8] ; CHECK-BE-NEXT: st1 { v0.16b }, [x8]
; CHECK-BE-NEXT: st1 { v1.16b }, [x10] ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: st1 { v1.16b }, [x8]
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
%lv = load<33 x i8>, ptr %A, align 8, !nontemporal !0 %lv = load<33 x i8>, ptr %A, align 8, !nontemporal !0
ret <33 x i8> %lv ret <33 x i8> %lv
} }
define <4 x i65> @test_ldnp_v4i65(ptr %A) { define <4 x i65> @test_ldnp_v4i65(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v4i65: ; CHECK-LABEL: test_ldnp_v4i65:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldp x8, x9, [x0, #8] ; CHECK-NEXT: ldp x8, x9, [x0, #8]
; CHECK-NEXT: ldr d0, [x0] ; CHECK-NEXT: ldr d0, [x0]
; CHECK-NEXT: ldr x10, [x0, #24] ; CHECK-NEXT: ldr x10, [x0, #24]
; CHECK-NEXT: and x1, x8, #0x1
; CHECK-NEXT: ldrb w11, [x0, #32] ; CHECK-NEXT: ldrb w11, [x0, #32]
; CHECK-NEXT: and x1, x8, #0x1
; CHECK-NEXT: extr x2, x9, x8, #1 ; CHECK-NEXT: extr x2, x9, x8, #1
; CHECK-NEXT: extr x4, x10, x9, #2 ; CHECK-NEXT: extr x4, x10, x9, #2
; CHECK-NEXT: mov.d v0[1], x1
; CHECK-NEXT: extr x6, x11, x10, #3 ; CHECK-NEXT: extr x6, x11, x10, #3
; CHECK-NEXT: ubfx x3, x9, #1, #1 ; CHECK-NEXT: ubfx x3, x9, #1, #1
; CHECK-NEXT: mov.d v0[1], x1
; CHECK-NEXT: ubfx x5, x10, #2, #1 ; CHECK-NEXT: ubfx x5, x10, #2, #1
; CHECK-NEXT: ubfx x7, x11, #3, #1 ; CHECK-NEXT: ubfx x7, x11, #3, #1
; CHECK-NEXT: fmov x0, d0 ; CHECK-NEXT: fmov x0, d0
Context not available.
; ;
; CHECK-BE-LABEL: test_ldnp_v4i65: ; CHECK-BE-LABEL: test_ldnp_v4i65:
; CHECK-BE: // %bb.0: ; CHECK-BE: // %bb.0:
; CHECK-BE-NEXT: ldp x10, x9, [x0, #16] ; CHECK-BE-NEXT: ldp x10, x9, [x0]
; CHECK-BE-NEXT: ldp x12, x11, [x0]
; CHECK-BE-NEXT: ldrb w8, [x0, #32] ; CHECK-BE-NEXT: ldrb w8, [x0, #32]
; CHECK-BE-NEXT: ldp x12, x11, [x0, #16]
; CHECK-BE-NEXT: lsr x13, x10, #56 ; CHECK-BE-NEXT: lsr x13, x10, #56
; CHECK-BE-NEXT: lsr x14, x12, #56 ; CHECK-BE-NEXT: orr x7, x8, x11, lsl #8
; CHECK-BE-NEXT: extr x15, x11, x10, #56
; CHECK-BE-NEXT: orr x7, x8, x9, lsl #8
; CHECK-BE-NEXT: extr x8, x10, x9, #56 ; CHECK-BE-NEXT: extr x8, x10, x9, #56
; CHECK-BE-NEXT: extr x9, x12, x11, #56 ; CHECK-BE-NEXT: extr x11, x12, x11, #56
; CHECK-BE-NEXT: lsr x12, x12, #59 ; CHECK-BE-NEXT: lsr x14, x12, #56
; CHECK-BE-NEXT: ubfx x10, x10, #57, #1 ; CHECK-BE-NEXT: extr x15, x9, x12, #56
; CHECK-BE-NEXT: extr x5, x13, x8, #1 ; CHECK-BE-NEXT: lsr x10, x10, #59
; CHECK-BE-NEXT: extr x1, x14, x9, #3 ; CHECK-BE-NEXT: extr x1, x13, x8, #3
; CHECK-BE-NEXT: ubfx x9, x11, #58, #1 ; CHECK-BE-NEXT: lsr x8, x9, #56
; CHECK-BE-NEXT: fmov d0, x12 ; CHECK-BE-NEXT: ubfx x12, x12, #57, #1
; CHECK-BE-NEXT: and x12, x8, #0x1 ; CHECK-BE-NEXT: ubfx x9, x9, #58, #1
; CHECK-BE-NEXT: lsr x11, x11, #56 ; CHECK-BE-NEXT: extr x5, x14, x11, #1
; CHECK-BE-NEXT: fmov d2, x10 ; CHECK-BE-NEXT: and x11, x11, #0x1
; CHECK-BE-NEXT: fmov d0, x10
; CHECK-BE-NEXT: fmov d2, x12
; CHECK-BE-NEXT: fmov d3, x11
; CHECK-BE-NEXT: fmov d1, x9 ; CHECK-BE-NEXT: fmov d1, x9
; CHECK-BE-NEXT: extr x3, x11, x15, #2 ; CHECK-BE-NEXT: extr x3, x8, x15, #2
; CHECK-BE-NEXT: fmov d3, x12
; CHECK-BE-NEXT: mov v0.d[1], x1 ; CHECK-BE-NEXT: mov v0.d[1], x1
; CHECK-BE-NEXT: mov v2.d[1], x5 ; CHECK-BE-NEXT: mov v2.d[1], x5
; CHECK-BE-NEXT: mov v1.d[1], x3
; CHECK-BE-NEXT: mov v3.d[1], x7 ; CHECK-BE-NEXT: mov v3.d[1], x7
; CHECK-BE-NEXT: mov v1.d[1], x3
; CHECK-BE-NEXT: fmov x0, d0 ; CHECK-BE-NEXT: fmov x0, d0
; CHECK-BE-NEXT: fmov x4, d2 ; CHECK-BE-NEXT: fmov x4, d2
; CHECK-BE-NEXT: fmov x2, d1
; CHECK-BE-NEXT: fmov x6, d3 ; CHECK-BE-NEXT: fmov x6, d3
; CHECK-BE-NEXT: fmov x2, d1
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
%lv = load <4 x i65>, ptr %A, align 8, !nontemporal !0 %lv = load <4 x i65>, ptr %A, align 8, !nontemporal !0
ret <4 x i65> %lv ret <4 x i65> %lv
} }
define <4 x i63> @test_ldnp_v4i63(ptr %A) { define <4 x i63> @test_ldnp_v4i63(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v4i63: ; CHECK-LABEL: test_ldnp_v4i63:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldp x8, x9, [x0] ; CHECK-NEXT: ldp x8, x9, [x0]
; CHECK-NEXT: ldp x10, x12, [x0, #16] ; CHECK-NEXT: ldp x10, x11, [x0, #16]
; CHECK-NEXT: extr x11, x9, x8, #63 ; CHECK-NEXT: extr x12, x9, x8, #63
; CHECK-NEXT: and x0, x8, #0x7fffffffffffffff ; CHECK-NEXT: and x0, x8, #0x7fffffffffffffff
; CHECK-NEXT: extr x9, x10, x9, #62 ; CHECK-NEXT: extr x9, x10, x9, #62
; CHECK-NEXT: extr x3, x12, x10, #61 ; CHECK-NEXT: extr x3, x11, x10, #61
; CHECK-NEXT: and x1, x11, #0x7fffffffffffffff ; CHECK-NEXT: and x1, x12, #0x7fffffffffffffff
; CHECK-NEXT: and x2, x9, #0x7fffffffffffffff ; CHECK-NEXT: and x2, x9, #0x7fffffffffffffff
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; ;
; CHECK-BE-LABEL: test_ldnp_v4i63: ; CHECK-BE-LABEL: test_ldnp_v4i63:
; CHECK-BE: // %bb.0: ; CHECK-BE: // %bb.0:
; CHECK-BE-NEXT: ldp x8, x9, [x0, #16] ; CHECK-BE-NEXT: ldp x9, x8, [x0, #8]
; CHECK-BE-NEXT: ldp x11, x10, [x0] ; CHECK-BE-NEXT: ldr x11, [x0, #24]
; CHECK-BE-NEXT: and x3, x9, #0x7fffffffffffffff ; CHECK-BE-NEXT: ldr x10, [x0]
; CHECK-BE-NEXT: extr x12, x10, x8, #62 ; CHECK-BE-NEXT: and x3, x11, #0x7fffffffffffffff
; CHECK-BE-NEXT: extr x8, x8, x9, #63 ; CHECK-BE-NEXT: extr x12, x9, x8, #62
; CHECK-BE-NEXT: extr x0, x11, x10, #61 ; CHECK-BE-NEXT: extr x8, x8, x11, #63
; CHECK-BE-NEXT: extr x0, x10, x9, #61
; CHECK-BE-NEXT: and x1, x12, #0x7fffffffffffffff ; CHECK-BE-NEXT: and x1, x12, #0x7fffffffffffffff
; CHECK-BE-NEXT: and x2, x8, #0x7fffffffffffffff ; CHECK-BE-NEXT: and x2, x8, #0x7fffffffffffffff
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
%lv = load <4 x i63>, ptr %A, align 8, !nontemporal !0 %lv = load <4 x i63>, ptr %A, align 8, !nontemporal !0
ret <4 x i63> %lv ret <4 x i63> %lv
} }
define <5 x double> @test_ldnp_v5f64(ptr %A) { define <5 x double> @test_ldnp_v5f64(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v5f64: ; CHECK-LABEL: test_ldnp_v5f64:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q2, [x0] ; CHECK-NEXT: ldnp q0, q2, [x0]
; CHECK-NEXT: ldr d4, [x0, #32] ; CHECK-NEXT: ldr d4, [x0, #32]
; CHECK-NEXT: ext.16b v1, v0, v0, #8 ; CHECK-NEXT: ext.16b v1, v0, v0, #8
; CHECK-NEXT: ; kill: def $d0 killed $d0 killed $q0
; CHECK-NEXT: ; kill: def $d1 killed $d1 killed $q1
; CHECK-NEXT: ext.16b v3, v2, v2, #8 ; CHECK-NEXT: ext.16b v3, v2, v2, #8
; CHECK-NEXT: ; kill: def $d0 killed $d0 killed $q0
; CHECK-NEXT: ; kill: def $d2 killed $d2 killed $q2 ; CHECK-NEXT: ; kill: def $d2 killed $d2 killed $q2
; CHECK-NEXT: ; kill: def $d1 killed $d1 killed $q1
; CHECK-NEXT: ; kill: def $d3 killed $d3 killed $q3 ; CHECK-NEXT: ; kill: def $d3 killed $d3 killed $q3
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; ;
Context not available.
; CHECK-BE-NEXT: // kill: def $d4 killed $d4 killed $q4 ; CHECK-BE-NEXT: // kill: def $d4 killed $d4 killed $q4
; CHECK-BE-NEXT: ext v1.16b, v0.16b, v0.16b, #8 ; CHECK-BE-NEXT: ext v1.16b, v0.16b, v0.16b, #8
; CHECK-BE-NEXT: // kill: def $d0 killed $d0 killed $q0 ; CHECK-BE-NEXT: // kill: def $d0 killed $d0 killed $q0
; CHECK-BE-NEXT: // kill: def $d1 killed $d1 killed $q1
; CHECK-BE-NEXT: ext v3.16b, v2.16b, v2.16b, #8 ; CHECK-BE-NEXT: ext v3.16b, v2.16b, v2.16b, #8
; CHECK-BE-NEXT: // kill: def $d1 killed $d1 killed $q1
; CHECK-BE-NEXT: // kill: def $d2 killed $d2 killed $q2 ; CHECK-BE-NEXT: // kill: def $d2 killed $d2 killed $q2
; CHECK-BE-NEXT: // kill: def $d3 killed $d3 killed $q3 ; CHECK-BE-NEXT: // kill: def $d3 killed $d3 killed $q3
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
%lv = load<5 x double>, ptr %A, align 8, !nontemporal !0 %lv = load<5 x double>, ptr %A, align 8, !nontemporal !0
ret <5 x double> %lv ret <5 x double> %lv
} }
define <16 x i64> @test_ldnp_v16i64(ptr %A) { define <16 x i64> @test_ldnp_v16i64(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v16i64: ; CHECK-LABEL: test_ldnp_v16i64:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <16 x i64>, ptr %A, align 8, !nontemporal !0 %lv = load <16 x i64>, ptr %A, align 8, !nontemporal !0
ret <16 x i64> %lv ret <16 x i64> %lv
} }
define <16 x double> @test_ldnp_v16f64(ptr %A) { define <16 x double> @test_ldnp_v16f64(ptr %A) {
;
; CHECK-LABEL: test_ldnp_v16f64: ; CHECK-LABEL: test_ldnp_v16f64:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ldnp q0, q1, [x0] ; CHECK-NEXT: ldnp q0, q1, [x0]
Context not available.
%lv = load <16 x double>, ptr %A, align 8, !nontemporal !0 %lv = load <16 x double>, ptr %A, align 8, !nontemporal !0
ret <16 x double> %lv ret <16 x double> %lv
} }
define <vscale x 20 x float> @test_ldnp_v20f32_vscale(<vscale x 20 x float>* %A) { define <vscale x 20 x float> @test_ldnp_v20f32_vscale(<vscale x 20 x float>* %A) {
;
; CHECK-LABEL: test_ldnp_v20f32_vscale: ; CHECK-LABEL: test_ldnp_v20f32_vscale:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: ptrue p0.s ; CHECK-NEXT: ptrue p0.s
Context not available.
%lv = load<vscale x 20 x float>, <vscale x 20 x float>* %A, align 8, !nontemporal !0 %lv = load<vscale x 20 x float>, <vscale x 20 x float>* %A, align 8, !nontemporal !0
ret <vscale x 20 x float> %lv ret <vscale x 20 x float> %lv
} }
!0 = !{i32 1} !0 = !{i32 1}
Context not available.

llvm/test/CodeGen/AArch64/nontemporal.ll

llvm/test/CodeGen/AArch64/nzcv-save.ll

llvm/test/CodeGen/AArch64/overeager_mla_fusing.ll

llvm/test/CodeGen/AArch64/peephole-and-tst.ll

llvm/test/CodeGen/AArch64/pmull-ldr-merge.ll

llvm/test/CodeGen/AArch64/pr-cf624b2.ll

llvm/test/CodeGen/AArch64/pr58350.ll

llvm/test/CodeGen/AArch64/pr58516.ll

llvm/test/CodeGen/AArch64/pr61549.ll

llvm/test/CodeGen/AArch64/predicated-add-sub.ll

llvm/test/CodeGen/AArch64/pull-negations-after-concat-of-truncates.ll

llvm/test/CodeGen/AArch64/ragreedy-csr.ll

llvm/test/CodeGen/AArch64/ragreedy-local-interval-cost.ll

llvm/test/CodeGen/AArch64/rand.ll

llvm/test/CodeGen/AArch64/rax1.ll

llvm/test/CodeGen/AArch64/rcpc3-sve.ll

llvm/test/CodeGen/AArch64/reduce-and.ll

llvm/test/CodeGen/AArch64/reduce-or.ll

llvm/test/CodeGen/AArch64/reduce-shuffle.ll

llvm/test/CodeGen/AArch64/reduce-xor.ll

llvm/test/CodeGen/AArch64/regress-tblgen-chains.ll

llvm/test/CodeGen/AArch64/rotate-extract.ll

llvm/test/CodeGen/AArch64/sadd_sat.ll

llvm/test/CodeGen/AArch64/sadd_sat_plus.ll

llvm/test/CodeGen/AArch64/sadd_sat_vec.ll

llvm/test/CodeGen/AArch64/sat-add.ll

llvm/test/CodeGen/AArch64/select-constant-xor.ll

llvm/test/CodeGen/AArch64/select_const.ll

llvm/test/CodeGen/AArch64/select_fmf.ll

llvm/test/CodeGen/AArch64/selectcc-to-shiftand.ll

llvm/test/CodeGen/AArch64/setcc-type-mismatch.ll

llvm/test/CodeGen/AArch64/settag-merge-order.ll

llvm/test/CodeGen/AArch64/settag-merge.ll

llvm/test/CodeGen/AArch64/settag.ll

llvm/test/CodeGen/AArch64/shift-amount-mod.ll

llvm/test/CodeGen/AArch64/shift-by-signext.ll

llvm/test/CodeGen/AArch64/shift_minsize.ll

llvm/test/CodeGen/AArch64/shrink-wrap-byval-inalloca-preallocated.ll

llvm/test/CodeGen/AArch64/shrink-wrapping-vla.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; Test shrink wrapping placement is correct with respect to calls to llvm.{stacksave,stackrestore} ; Test shrink wrapping placement is correct with respect to calls to llvm.{stacksave,stackrestore}
; void f(int n, int x[]) { ; void f(int n, int x[]) {
; if (n < 0) ; if (n < 0)
; return; ; return;
Context not available.
; } ; }
; ;
; RUN: llc -mtriple aarch64-linux %s -o - | FileCheck %s ; RUN: llc -mtriple aarch64-linux %s -o - | FileCheck %s
define dso_local void @f(i32 %n, ptr nocapture %x) uwtable { define dso_local void @f(i32 %n, ptr nocapture %x) uwtable {
; CHECK-LABEL: f:
dmgreenUnsubmitted
Done
ReplyInline Actions

Manual update might be better.

dmgreen: Manual update might be better.
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: // kill: def $w0 killed $w0 def $x0
; CHECK-NEXT: tbnz w0, #31, .LBB0_7
; CHECK-NEXT: // %bb.1: // %if.end
; CHECK-NEXT: stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: ubfiz x8, x0, #2, #32
; CHECK-NEXT: mov x9, sp
; CHECK-NEXT: mov x2, sp
; CHECK-NEXT: add x8, x8, #15
; CHECK-NEXT: and x8, x8, #0x7fffffff0
; CHECK-NEXT: sub x8, x9, x8
; CHECK-NEXT: mov sp, x8
; CHECK-NEXT: cbz w0, .LBB0_6
; CHECK-NEXT: // %bb.2:
; CHECK-NEXT: mov w9, w0
; CHECK-NEXT: mov x10, x8
; CHECK-NEXT: mov x11, x9
; CHECK-NEXT: .LBB0_3: // %for.body
; CHECK-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-NEXT: subs x11, x11, #1
; CHECK-NEXT: ldr w12, [x1, w11, sxtw #2]
; CHECK-NEXT: str w12, [x10], #4
; CHECK-NEXT: b.ne .LBB0_3
; CHECK-NEXT: // %bb.4: // %for.cond6.preheader
; CHECK-NEXT: cmp w0, #1
; CHECK-NEXT: b.lt .LBB0_6
; CHECK-NEXT: .LBB0_5: // %for.body9
; CHECK-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldr w10, [x8], #4
; CHECK-NEXT: subs x9, x9, #1
; CHECK-NEXT: add w10, w10, #1
; CHECK-NEXT: str w10, [x1], #4
; CHECK-NEXT: b.ne .LBB0_5
; CHECK-NEXT: .LBB0_6: // %for.cond.cleanup8
; CHECK-NEXT: mov sp, x2
; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: .cfi_def_cfa wsp, 16
; CHECK-NEXT: ldp x29, x30, [sp], #16 // 16-byte Folded Reload
; CHECK-NEXT: .cfi_def_cfa_offset 0
; CHECK-NEXT: .cfi_restore w30
; CHECK-NEXT: .cfi_restore w29
; CHECK-NEXT: .LBB0_7: // %return
; CHECK-NEXT: ret
entry: entry:
%cmp = icmp slt i32 %n, 0 %cmp = icmp slt i32 %n, 0
br i1 %cmp, label %return, label %if.end br i1 %cmp, label %return, label %if.end
if.end: ; preds = %entry if.end: ; preds = %entry
%0 = zext i32 %n to i64 %0 = zext i32 %n to i64
%1 = tail call ptr @llvm.stacksave() %1 = tail call ptr @llvm.stacksave()
%vla = alloca i32, i64 %0, align 16 %vla = alloca i32, i64 %0, align 16
%cmp132 = icmp eq i32 %n, 0 %cmp132 = icmp eq i32 %n, 0
br i1 %cmp132, label %for.cond.cleanup8, label %for.body.lr.ph br i1 %cmp132, label %for.cond.cleanup8, label %for.body.lr.ph
for.body.lr.ph: ; preds = %if.end for.body.lr.ph: ; preds = %if.end
%sub = add i32 %n, -1 %sub = add i32 %n, -1
br label %for.body br label %for.body
for.cond6.preheader: ; preds = %for.body for.cond6.preheader: ; preds = %for.body
%cmp730 = icmp sgt i32 %n, 0 %cmp730 = icmp sgt i32 %n, 0
br i1 %cmp730, label %for.body9, label %for.cond.cleanup8 br i1 %cmp730, label %for.body9, label %for.cond.cleanup8
for.body: ; preds = %for.body, %for.body.lr.ph for.body: ; preds = %for.body, %for.body.lr.ph
%indvars.iv34 = phi i64 [ 0, %for.body.lr.ph ], [ %indvars.iv.next35, %for.body ] %indvars.iv34 = phi i64 [ 0, %for.body.lr.ph ], [ %indvars.iv.next35, %for.body ]
%2 = trunc i64 %indvars.iv34 to i32 %2 = trunc i64 %indvars.iv34 to i32
Context not available.
%indvars.iv.next35 = add nuw nsw i64 %indvars.iv34, 1 %indvars.iv.next35 = add nuw nsw i64 %indvars.iv34, 1
%exitcond37 = icmp eq i64 %indvars.iv.next35, %0 %exitcond37 = icmp eq i64 %indvars.iv.next35, %0
br i1 %exitcond37, label %for.cond6.preheader, label %for.body br i1 %exitcond37, label %for.cond6.preheader, label %for.body
for.cond.cleanup8: ; preds = %for.body9, %if.end, %for.cond6.preheader for.cond.cleanup8: ; preds = %for.body9, %if.end, %for.cond6.preheader
tail call void @llvm.stackrestore(ptr %1) tail call void @llvm.stackrestore(ptr %1)
br label %return br label %return
for.body9: ; preds = %for.cond6.preheader, %for.body9 for.body9: ; preds = %for.cond6.preheader, %for.body9
%indvars.iv = phi i64 [ %indvars.iv.next, %for.body9 ], [ 0, %for.cond6.preheader ] %indvars.iv = phi i64 [ %indvars.iv.next, %for.body9 ], [ 0, %for.cond6.preheader ]
%arrayidx11 = getelementptr inbounds i32, ptr %vla, i64 %indvars.iv %arrayidx11 = getelementptr inbounds i32, ptr %vla, i64 %indvars.iv
Context not available.
%indvars.iv.next = add nuw nsw i64 %indvars.iv, 1 %indvars.iv.next = add nuw nsw i64 %indvars.iv, 1
%exitcond = icmp eq i64 %indvars.iv.next, %0 %exitcond = icmp eq i64 %indvars.iv.next, %0
br i1 %exitcond, label %for.cond.cleanup8, label %for.body9 br i1 %exitcond, label %for.cond.cleanup8, label %for.body9
return: ; preds = %entry, %for.cond.cleanup8 return: ; preds = %entry, %for.cond.cleanup8
ret void ret void
} }
; Function Attrs: nounwind ; Function Attrs: nounwind
declare ptr @llvm.stacksave() declare ptr @llvm.stacksave()
; Function Attrs: nounwind ; Function Attrs: nounwind
declare void @llvm.stackrestore(ptr) declare void @llvm.stackrestore(ptr)
; Check that llvm.stackrestore() happens before CSRs are popped off the stack
; CHECK-LABEL: f
; CHECK: stp x29, x30, [sp, #-16]!
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; VLA allocation ; VLA allocation
; CHECK: ubfiz x8, x0, #2, #32
; CHECK: mov x9, sp
; CHECK: add x8, x8, #15
; CHECK: mov [[SAVE:x[0-9]+]], sp
; CHECK: and [[X1:x[0-9]+]], [[X1]], #0x7fffffff0
; Saving the SP via llvm.stacksave() ; Saving the SP via llvm.stacksave()
; CHECK: sub [[X1]], [[X2:x[0-9]+]], [[X1]]
; The next instruction comes from llvm.stackrestore() ; The next instruction comes from llvm.stackrestore()
; CHECK: mov sp, [[SAVE]]
; Epilogue ; Epilogue
; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: .cfi_def_cfa wsp, 16
; CHECK-NEXT: ldp x29, x30, [sp], #16
; CHECK-NEXT: .cfi_def_cfa_offset 0
; CHECK-NEXT: .cfi_restore w30
; CHECK-NEXT: .cfi_restore w29
; CHECK-NEXT: ret
Context not available.

llvm/test/CodeGen/AArch64/shuffle-tbl34.ll

llvm/test/CodeGen/AArch64/shuffles.ll

llvm/test/CodeGen/AArch64/sink-addsub-of-const.ll

llvm/test/CodeGen/AArch64/sinksplat.ll

llvm/test/CodeGen/AArch64/sitofp-fixed-legal.ll

llvm/test/CodeGen/AArch64/sme-aarch64-svcount.ll

llvm/test/CodeGen/AArch64/sme-disable-gisel-fisel.ll

llvm/test/CodeGen/AArch64/sme-intrinsics-loads.ll

llvm/test/CodeGen/AArch64/sme-intrinsics-mova-extract.ll

llvm/test/CodeGen/AArch64/sme-intrinsics-mova-insert.ll

llvm/test/CodeGen/AArch64/sme-intrinsics-stores.ll

llvm/test/CodeGen/AArch64/sme-lazy-save-call.ll

llvm/test/CodeGen/AArch64/sme-shared-za-interface.ll

llvm/test/CodeGen/AArch64/sme-streaming-body.ll

llvm/test/CodeGen/AArch64/sme-streaming-compatible-interface.ll

llvm/test/CodeGen/AArch64/sme-streaming-interface.ll

llvm/test/CodeGen/AArch64/sme2-intrinsics-add.ll

llvm/test/CodeGen/AArch64/sme2-intrinsics-fmlas.ll

llvm/test/CodeGen/AArch64/sme2-intrinsics-fp-dots.ll

llvm/test/CodeGen/AArch64/sme2-intrinsics-insert-mova.ll

llvm/test/CodeGen/AArch64/sme2-intrinsics-int-dots.ll

llvm/test/CodeGen/AArch64/sme2-intrinsics-max.ll

llvm/test/CodeGen/AArch64/sme2-intrinsics-min.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc -mtriple=aarch64-linux-gnu -mattr=+sme2 -verify-machineinstrs < %s | FileCheck %s ; RUN: llc -mtriple=aarch64-linux-gnu -mattr=+sme2 -verify-machineinstrs < %s | FileCheck %s
; SMIN (Single, x2) ; SMIN (Single, x2)
define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_min_single_x2_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) { define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_min_single_x2_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) {
; CHECK-LABEL: multi_vec_min_single_x2_s8: ; CHECK-LABEL: multi_vec_min_single_x2_s8:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.smin.single.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) %res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.smin.single.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm)
ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_min_single_x2_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) { define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_min_single_x2_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) {
; CHECK-LABEL: multi_vec_min_single_x2_s16: ; CHECK-LABEL: multi_vec_min_single_x2_s16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.smin.single.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) %res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.smin.single.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm)
ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_min_single_x2_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) { define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_min_single_x2_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) {
; CHECK-LABEL: multi_vec_min_single_x2_s32: ; CHECK-LABEL: multi_vec_min_single_x2_s32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.smin.single.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) %res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.smin.single.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm)
ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_min_single_x2_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) { define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_min_single_x2_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) {
; CHECK-LABEL: multi_vec_min_single_x2_s64: ; CHECK-LABEL: multi_vec_min_single_x2_s64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.smin.single.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) %res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.smin.single.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm)
ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; UMIN (Single, x2) ; UMIN (Single, x2)
define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_min_single_x2_u8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) { define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_min_single_x2_u8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) {
; CHECK-LABEL: multi_vec_min_single_x2_u8: ; CHECK-LABEL: multi_vec_min_single_x2_u8:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.umin.single.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) %res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.umin.single.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm)
ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_min_single_x2_u16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) { define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_min_single_x2_u16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) {
; CHECK-LABEL: multi_vec_min_single_x2_u16: ; CHECK-LABEL: multi_vec_min_single_x2_u16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.umin.single.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) %res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.umin.single.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm)
ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_min_single_x2_u32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) { define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_min_single_x2_u32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) {
; CHECK-LABEL: multi_vec_min_single_x2_u32: ; CHECK-LABEL: multi_vec_min_single_x2_u32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.umin.single.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) %res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.umin.single.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm)
ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_min_single_x2_u64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) { define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_min_single_x2_u64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) {
; CHECK-LABEL: multi_vec_min_single_x2_u64: ; CHECK-LABEL: multi_vec_min_single_x2_u64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.umin.single.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) %res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.umin.single.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm)
ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; FMIN (Single, x2) ; FMIN (Single, x2)
define { <vscale x 8 x half>, <vscale x 8 x half> } @multi_vec_min_single_x2_f16(<vscale x 8 x half> %unused, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm) { define { <vscale x 8 x half>, <vscale x 8 x half> } @multi_vec_min_single_x2_f16(<vscale x 8 x half> %unused, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm) {
; CHECK-LABEL: multi_vec_min_single_x2_f16: ; CHECK-LABEL: multi_vec_min_single_x2_f16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fmin.single.x2.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm) %res = call { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fmin.single.x2.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm)
ret { <vscale x 8 x half>, <vscale x 8 x half> } %res ret { <vscale x 8 x half>, <vscale x 8 x half> } %res
} }
define { <vscale x 4 x float>, <vscale x 4 x float> } @multi_vec_min_single_x2_f32(<vscale x 4 x float> %unused, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm) { define { <vscale x 4 x float>, <vscale x 4 x float> } @multi_vec_min_single_x2_f32(<vscale x 4 x float> %unused, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm) {
; CHECK-LABEL: multi_vec_min_single_x2_f32: ; CHECK-LABEL: multi_vec_min_single_x2_f32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fmin.single.x2.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm) %res = call { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fmin.single.x2.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm)
ret { <vscale x 4 x float>, <vscale x 4 x float> } %res ret { <vscale x 4 x float>, <vscale x 4 x float> } %res
} }
define { <vscale x 2 x double>, <vscale x 2 x double> } @multi_vec_min_single_x2_f64(<vscale x 2 x double> %unused, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm) { define { <vscale x 2 x double>, <vscale x 2 x double> } @multi_vec_min_single_x2_f64(<vscale x 2 x double> %unused, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm) {
; CHECK-LABEL: multi_vec_min_single_x2_f64: ; CHECK-LABEL: multi_vec_min_single_x2_f64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fmin.single.x2.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm) %res = call { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fmin.single.x2.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm)
ret { <vscale x 2 x double>, <vscale x 2 x double> } %res ret { <vscale x 2 x double>, <vscale x 2 x double> } %res
} }
; SMIN (Single, x4) ; SMIN (Single, x4)
define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@multi_vec_min_single_x4_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) { @multi_vec_min_single_x4_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) {
; CHECK-LABEL: multi_vec_min_single_x4_s8:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: smin { z24.b - z27.b }, { z24.b - z27.b }, z5.b
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
dmgreenUnsubmitted
Done
ReplyInline Actions

This doesn't look right.

dmgreen: This doesn't look right.
@llvm.aarch64.sve.smin.single.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) @llvm.aarch64.sve.smin.single.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm)
ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@multi_vec_min_single_x4_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) { @multi_vec_min_single_x4_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) {
; CHECK-LABEL: multi_vec_min_single_x4_s16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: smin { z24.h - z27.h }, { z24.h - z27.h }, z5.h
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@llvm.aarch64.sve.smin.single.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) @llvm.aarch64.sve.smin.single.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm)
ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@multi_vec_min_single_x4_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) { @multi_vec_min_single_x4_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) {
; CHECK-LABEL: multi_vec_min_single_x4_s32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: smin { z24.s - z27.s }, { z24.s - z27.s }, z5.s
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@llvm.aarch64.sve.smin.single.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) @llvm.aarch64.sve.smin.single.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm)
ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@multi_vec_min_single_x4_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) { @multi_vec_min_single_x4_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) {
; CHECK-LABEL: multi_vec_min_single_x4_s64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: smin { z24.d - z27.d }, { z24.d - z27.d }, z5.d
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.smin.single.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) @llvm.aarch64.sve.smin.single.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm)
ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; UMIN (Single, x4) ; UMIN (Single, x4)
define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@multi_vec_min_single_x4_u8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) { @multi_vec_min_single_x4_u8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) {
; CHECK-LABEL: multi_vec_min_single_x4_u8:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: umin { z24.b - z27.b }, { z24.b - z27.b }, z5.b
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.umin.single.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) @llvm.aarch64.sve.umin.single.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm)
ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@multi_vec_min_single_x4_u16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) { @multi_vec_min_single_x4_u16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) {
; CHECK-LABEL: multi_vec_min_single_x4_u16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: umin { z24.h - z27.h }, { z24.h - z27.h }, z5.h
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@llvm.aarch64.sve.umin.single.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) @llvm.aarch64.sve.umin.single.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm)
ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@multi_vec_min_single_x4_u32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) { @multi_vec_min_single_x4_u32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) {
; CHECK-LABEL: multi_vec_min_single_x4_u32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: umin { z24.s - z27.s }, { z24.s - z27.s }, z5.s
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@llvm.aarch64.sve.umin.single.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) @llvm.aarch64.sve.umin.single.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm)
ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@multi_vec_min_single_x4_u64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) { @multi_vec_min_single_x4_u64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) {
; CHECK-LABEL: multi_vec_min_single_x4_u64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: umin { z24.d - z27.d }, { z24.d - z27.d }, z5.d
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.umin.single.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) @llvm.aarch64.sve.umin.single.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm)
ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; FMIN (SINGLE, x4) ; FMIN (SINGLE, x4)
define { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } define { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> }
@multi_vec_min_single_x4_f16(<vscale x 8 x half> %unused, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, <vscale x 8 x half> %zm) { @multi_vec_min_single_x4_f16(<vscale x 8 x half> %unused, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, <vscale x 8 x half> %zm) {
; CHECK-LABEL: multi_vec_min_single_x4_f16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: fmin { z24.h - z27.h }, { z24.h - z27.h }, z5.h
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } %res = call { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> }
@llvm.aarch64.sve.fmin.single.x4.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, <vscale x 8 x half> %zm) @llvm.aarch64.sve.fmin.single.x4.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, <vscale x 8 x half> %zm)
ret { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } %res ret { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } %res
} }
define { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } define { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> }
@multi_vec_min_single_x4_f32(<vscale x 4 x float> %unused, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, <vscale x 4 x float> %zm) { @multi_vec_min_single_x4_f32(<vscale x 4 x float> %unused, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, <vscale x 4 x float> %zm) {
; CHECK-LABEL: multi_vec_min_single_x4_f32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: fmin { z24.s - z27.s }, { z24.s - z27.s }, z5.s
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } %res = call { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> }
@llvm.aarch64.sve.fmin.single.x4.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, <vscale x 4 x float> %zm) @llvm.aarch64.sve.fmin.single.x4.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, <vscale x 4 x float> %zm)
ret { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } %res ret { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } %res
} }
define { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } define { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> }
@multi_vec_min_single_x4_f64(<vscale x 2 x double> %unused, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, <vscale x 2 x double> %zm) { @multi_vec_min_single_x4_f64(<vscale x 2 x double> %unused, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, <vscale x 2 x double> %zm) {
; CHECK-LABEL: multi_vec_min_single_x4_f64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: fmin { z24.d - z27.d }, { z24.d - z27.d }, z5.d
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } %res = call { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> }
@llvm.aarch64.sve.fmin.single.x4.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, <vscale x 2 x double> %zm) @llvm.aarch64.sve.fmin.single.x4.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, <vscale x 2 x double> %zm)
ret { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } %res ret { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } %res
} }
; SMIN (Multi, x2) ; SMIN (Multi, x2)
define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_min_multi_x2_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) { define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_min_multi_x2_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) {
; CHECK-LABEL: multi_vec_min_multi_x2_s8: ; CHECK-LABEL: multi_vec_min_multi_x2_s8:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.smin.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) %res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.smin.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2)
ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_min_multi_x2_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) { define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_min_multi_x2_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) {
; CHECK-LABEL: multi_vec_min_multi_x2_s16: ; CHECK-LABEL: multi_vec_min_multi_x2_s16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.smin.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) %res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.smin.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2)
ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_min_multi_x2_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) { define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_min_multi_x2_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) {
; CHECK-LABEL: multi_vec_min_multi_x2_s32: ; CHECK-LABEL: multi_vec_min_multi_x2_s32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.smin.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) %res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.smin.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2)
ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_min_multi_x2_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) { define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_min_multi_x2_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) {
; CHECK-LABEL: multi_vec_min_multi_x2_s64: ; CHECK-LABEL: multi_vec_min_multi_x2_s64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.smin.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) %res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.smin.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2)
ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; UMIN (Multi, x2) ; UMIN (Multi, x2)
define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_min_multi_x2_u8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) { define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_min_multi_x2_u8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) {
; CHECK-LABEL: multi_vec_min_multi_x2_u8: ; CHECK-LABEL: multi_vec_min_multi_x2_u8:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.umin.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) %res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.umin.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2)
ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_min_multi_x2_u16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) { define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_min_multi_x2_u16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) {
; CHECK-LABEL: multi_vec_min_multi_x2_u16: ; CHECK-LABEL: multi_vec_min_multi_x2_u16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.umin.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) %res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.umin.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2)
ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_min_multi_x2_u32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) { define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_min_multi_x2_u32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) {
; CHECK-LABEL: multi_vec_min_multi_x2_u32: ; CHECK-LABEL: multi_vec_min_multi_x2_u32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.umin.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) %res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.umin.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2)
ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_min_multi_x2_u64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) { define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_min_multi_x2_u64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) {
; CHECK-LABEL: multi_vec_min_multi_x2_u64: ; CHECK-LABEL: multi_vec_min_multi_x2_u64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.umin.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) %res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.umin.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2)
ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; FMIN (Multi, x2) ; FMIN (Multi, x2)
define { <vscale x 8 x half>, <vscale x 8 x half> } @multi_vec_min_multi_x2_f16(<vscale x 8 x half> %unused, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2) { define { <vscale x 8 x half>, <vscale x 8 x half> } @multi_vec_min_multi_x2_f16(<vscale x 8 x half> %unused, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2) {
; CHECK-LABEL: multi_vec_min_multi_x2_f16: ; CHECK-LABEL: multi_vec_min_multi_x2_f16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fmin.x2.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2) %res = call { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fmin.x2.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2)
ret { <vscale x 8 x half>, <vscale x 8 x half> } %res ret { <vscale x 8 x half>, <vscale x 8 x half> } %res
} }
define { <vscale x 4 x float>, <vscale x 4 x float> } @multi_vec_min_multi_x2_f32(<vscale x 4 x float> %unused, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2) { define { <vscale x 4 x float>, <vscale x 4 x float> } @multi_vec_min_multi_x2_f32(<vscale x 4 x float> %unused, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2) {
; CHECK-LABEL: multi_vec_min_multi_x2_f32: ; CHECK-LABEL: multi_vec_min_multi_x2_f32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fmin.x2.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2) %res = call { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fmin.x2.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2)
ret { <vscale x 4 x float>, <vscale x 4 x float> } %res ret { <vscale x 4 x float>, <vscale x 4 x float> } %res
} }
define { <vscale x 2 x double>, <vscale x 2 x double> } @multi_vec_min_multi_x2_f64(<vscale x 2 x double> %unused, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2) { define { <vscale x 2 x double>, <vscale x 2 x double> } @multi_vec_min_multi_x2_f64(<vscale x 2 x double> %unused, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2) {
; CHECK-LABEL: multi_vec_min_multi_x2_f64: ; CHECK-LABEL: multi_vec_min_multi_x2_f64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fmin.x2.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2) %res = call { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fmin.x2.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2)
ret { <vscale x 2 x double>, <vscale x 2 x double> } %res ret { <vscale x 2 x double>, <vscale x 2 x double> } %res
} }
; SMIN (Multi, x4) ; SMIN (Multi, x4)
define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@multi_vec_min_multi_x4_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, @multi_vec_min_multi_x4_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4,
<vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) { <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) {
; CHECK-LABEL: multi_vec_min_multi_x4_s8:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.b
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1b { z31.b }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: smin { z24.b - z27.b }, { z24.b - z27.b }, { z28.b - z31.b }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.smin.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, @llvm.aarch64.sve.smin.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4,
<vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4)
ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@multi_vec_min_multi_x4_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, @multi_vec_min_multi_x4_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4,
<vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) { <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) {
; CHECK-LABEL: multi_vec_min_multi_x4_s16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.h
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1h { z31.h }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: smin { z24.h - z27.h }, { z24.h - z27.h }, { z28.h - z31.h }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@llvm.aarch64.sve.smin.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, @llvm.aarch64.sve.smin.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4,
<vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4)
ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@multi_vec_min_multi_x4_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, @multi_vec_min_multi_x4_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4,
<vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) { <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) {
; CHECK-LABEL: multi_vec_min_multi_x4_s32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.s
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1w { z31.s }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: smin { z24.s - z27.s }, { z24.s - z27.s }, { z28.s - z31.s }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@llvm.aarch64.sve.smin.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, @llvm.aarch64.sve.smin.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4,
<vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4)
ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@multi_vec_min_multi_x4_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, @multi_vec_min_multi_x4_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4,
<vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) { <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) {
; CHECK-LABEL: multi_vec_min_multi_x4_s64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.d
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1d { z31.d }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: smin { z24.d - z27.d }, { z24.d - z27.d }, { z28.d - z31.d }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.smin.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, @llvm.aarch64.sve.smin.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4,
<vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4)
ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; UMIN (Multi, x4) ; UMIN (Multi, x4)
define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@multi_vec_min_multi_x4_u8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, @multi_vec_min_multi_x4_u8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4,
<vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) { <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) {
; CHECK-LABEL: multi_vec_min_multi_x4_u8:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.b
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1b { z31.b }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: umin { z24.b - z27.b }, { z24.b - z27.b }, { z28.b - z31.b }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.umin.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, @llvm.aarch64.sve.umin.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4,
<vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4)
ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@multi_vec_min_multi_x4_u16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, @multi_vec_min_multi_x4_u16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4,
<vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) { <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) {
; CHECK-LABEL: multi_vec_min_multi_x4_u16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.h
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1h { z31.h }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: umin { z24.h - z27.h }, { z24.h - z27.h }, { z28.h - z31.h }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@llvm.aarch64.sve.umin.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, @llvm.aarch64.sve.umin.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4,
<vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4)
ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@multi_vec_min_multi_x4_u32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, @multi_vec_min_multi_x4_u32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4,
<vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) { <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) {
; CHECK-LABEL: multi_vec_min_multi_x4_u32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.s
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1w { z31.s }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: umin { z24.s - z27.s }, { z24.s - z27.s }, { z28.s - z31.s }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@llvm.aarch64.sve.umin.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, @llvm.aarch64.sve.umin.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4,
<vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4)
ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@multi_vec_min_multi_x4_u64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, @multi_vec_min_multi_x4_u64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4,
<vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) { <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) {
; CHECK-LABEL: multi_vec_min_multi_x4_u64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.d
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1d { z31.d }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: umin { z24.d - z27.d }, { z24.d - z27.d }, { z28.d - z31.d }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.umin.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, @llvm.aarch64.sve.umin.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4,
<vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4)
ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; FMIN (Multi, x4) ; FMIN (Multi, x4)
define { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } define { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> }
@multi_vec_min_multi_x4_f16(<vscale x 8 x half> %unused, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, @multi_vec_min_multi_x4_f16(<vscale x 8 x half> %unused, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4,
<vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2, <vscale x 8 x half> %zm3, <vscale x 8 x half> %zm4) { <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2, <vscale x 8 x half> %zm3, <vscale x 8 x half> %zm4) {
; CHECK-LABEL: multi_vec_min_multi_x4_f16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.h
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1h { z31.h }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: fmin { z24.h - z27.h }, { z24.h - z27.h }, { z28.h - z31.h }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } %res = call { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> }
@llvm.aarch64.sve.fmin.x4.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, @llvm.aarch64.sve.fmin.x4.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4,
<vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2, <vscale x 8 x half> %zm3, <vscale x 8 x half> %zm4) <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2, <vscale x 8 x half> %zm3, <vscale x 8 x half> %zm4)
ret { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } %res ret { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } %res
} }
define { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } define { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> }
@multi_vec_min_multi_x4_f32(<vscale x 4 x float> %unused, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, @multi_vec_min_multi_x4_f32(<vscale x 4 x float> %unused, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4,
<vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2, <vscale x 4 x float> %zm3, <vscale x 4 x float> %zm4) { <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2, <vscale x 4 x float> %zm3, <vscale x 4 x float> %zm4) {
; CHECK-LABEL: multi_vec_min_multi_x4_f32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.s
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1w { z31.s }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: fmin { z24.s - z27.s }, { z24.s - z27.s }, { z28.s - z31.s }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } %res = call { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> }
@llvm.aarch64.sve.fmin.x4.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, @llvm.aarch64.sve.fmin.x4.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4,
<vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2, <vscale x 4 x float> %zm3, <vscale x 4 x float> %zm4) <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2, <vscale x 4 x float> %zm3, <vscale x 4 x float> %zm4)
ret { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } %res ret { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } %res
} }
define { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } define { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> }
@multi_vec_min_multi_x4_f64(<vscale x 2 x double> %unused, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, @multi_vec_min_multi_x4_f64(<vscale x 2 x double> %unused, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4,
<vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2, <vscale x 2 x double> %zm3, <vscale x 2 x double> %zm4) { <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2, <vscale x 2 x double> %zm3, <vscale x 2 x double> %zm4) {
; CHECK-LABEL: multi_vec_min_multi_x4_f64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.d
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1d { z31.d }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: fmin { z24.d - z27.d }, { z24.d - z27.d }, { z28.d - z31.d }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } %res = call { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> }
@llvm.aarch64.sve.fmin.x4.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, @llvm.aarch64.sve.fmin.x4.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4,
<vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2, <vscale x 2 x double> %zm3, <vscale x 2 x double> %zm4) <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2, <vscale x 2 x double> %zm3, <vscale x 2 x double> %zm4)
ret { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } %res ret { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } %res
} }
; FMINNM (Single, x2) ; FMINNM (Single, x2)
define { <vscale x 8 x half>, <vscale x 8 x half> } @multi_vec_minnm_single_x2_f16(<vscale x 8 x half> %dummy, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm) { define { <vscale x 8 x half>, <vscale x 8 x half> } @multi_vec_minnm_single_x2_f16(<vscale x 8 x half> %dummy, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm) {
; CHECK-LABEL: multi_vec_minnm_single_x2_f16: ; CHECK-LABEL: multi_vec_minnm_single_x2_f16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fminnm.single.x2.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm) %res = call { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fminnm.single.x2.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm)
ret { <vscale x 8 x half>, <vscale x 8 x half> } %res ret { <vscale x 8 x half>, <vscale x 8 x half> } %res
} }
define { <vscale x 4 x float>, <vscale x 4 x float> } @multi_vec_minnm_single_x2_f32(<vscale x 8 x half> %dummy, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm) { define { <vscale x 4 x float>, <vscale x 4 x float> } @multi_vec_minnm_single_x2_f32(<vscale x 8 x half> %dummy, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm) {
; CHECK-LABEL: multi_vec_minnm_single_x2_f32: ; CHECK-LABEL: multi_vec_minnm_single_x2_f32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fminnm.single.x2.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm) %res = call { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fminnm.single.x2.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm)
ret { <vscale x 4 x float>, <vscale x 4 x float> } %res ret { <vscale x 4 x float>, <vscale x 4 x float> } %res
} }
define { <vscale x 2 x double>, <vscale x 2 x double> } @multi_vec_minnm_single_x2_f64(<vscale x 8 x half> %dummy, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm) { define { <vscale x 2 x double>, <vscale x 2 x double> } @multi_vec_minnm_single_x2_f64(<vscale x 8 x half> %dummy, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm) {
; CHECK-LABEL: multi_vec_minnm_single_x2_f64: ; CHECK-LABEL: multi_vec_minnm_single_x2_f64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fminnm.single.x2.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm) %res = call { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fminnm.single.x2.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm)
ret { <vscale x 2 x double>, <vscale x 2 x double> } %res ret { <vscale x 2 x double>, <vscale x 2 x double> } %res
} }
; FMINNM (Single, x4) ; FMINNM (Single, x4)
define { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } define { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> }
@multi_vec_minnm_single_x4_f16(<vscale x 8 x half> %dummy, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, <vscale x 8 x half> %zm) { @multi_vec_minnm_single_x4_f16(<vscale x 8 x half> %dummy, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, <vscale x 8 x half> %zm) {
; CHECK-LABEL: multi_vec_minnm_single_x4_f16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: fminnm { z24.h - z27.h }, { z24.h - z27.h }, z5.h
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } %res = call { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> }
@llvm.aarch64.sve.fminnm.single.x4.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, <vscale x 8 x half> %zm) @llvm.aarch64.sve.fminnm.single.x4.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, <vscale x 8 x half> %zm)
ret { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } %res ret { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } %res
} }
define { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } define { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> }
@multi_vec_minnm_single_x4_f32(<vscale x 8 x half> %dummy, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, <vscale x 4 x float> %zm) { @multi_vec_minnm_single_x4_f32(<vscale x 8 x half> %dummy, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, <vscale x 4 x float> %zm) {
; CHECK-LABEL: multi_vec_minnm_single_x4_f32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: fminnm { z24.s - z27.s }, { z24.s - z27.s }, z5.s
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } %res = call { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> }
@llvm.aarch64.sve.fminnm.single.x4.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, <vscale x 4 x float> %zm) @llvm.aarch64.sve.fminnm.single.x4.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, <vscale x 4 x float> %zm)
ret { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } %res ret { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } %res
} }
define { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } define { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> }
@multi_vec_minnm_single_x4_f64(<vscale x 8 x half> %dummy, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, <vscale x 2 x double> %zm) { @multi_vec_minnm_single_x4_f64(<vscale x 8 x half> %dummy, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, <vscale x 2 x double> %zm) {
; CHECK-LABEL: multi_vec_minnm_single_x4_f64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: fminnm { z24.d - z27.d }, { z24.d - z27.d }, z5.d
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } %res = call { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> }
@llvm.aarch64.sve.fminnm.single.x4.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, <vscale x 2 x double> %zm) @llvm.aarch64.sve.fminnm.single.x4.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, <vscale x 2 x double> %zm)
ret { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } %res ret { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } %res
} }
; FMINNM (Multi, x2) ; FMINNM (Multi, x2)
define { <vscale x 8 x half>, <vscale x 8 x half> } @multi_vec_minnm_x2_f16(<vscale x 8 x half> %dummy, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2) { define { <vscale x 8 x half>, <vscale x 8 x half> } @multi_vec_minnm_x2_f16(<vscale x 8 x half> %dummy, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2) {
; CHECK-LABEL: multi_vec_minnm_x2_f16: ; CHECK-LABEL: multi_vec_minnm_x2_f16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fminnm.x2.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2) %res = call { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fminnm.x2.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2)
ret { <vscale x 8 x half>, <vscale x 8 x half> } %res ret { <vscale x 8 x half>, <vscale x 8 x half> } %res
} }
define { <vscale x 4 x float>, <vscale x 4 x float> } @multi_vec_minnm_x2_f32(<vscale x 8 x half> %dummy, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2) { define { <vscale x 4 x float>, <vscale x 4 x float> } @multi_vec_minnm_x2_f32(<vscale x 8 x half> %dummy, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2) {
; CHECK-LABEL: multi_vec_minnm_x2_f32: ; CHECK-LABEL: multi_vec_minnm_x2_f32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fminnm.x2.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2) %res = call { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fminnm.x2.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2)
ret { <vscale x 4 x float>, <vscale x 4 x float> } %res ret { <vscale x 4 x float>, <vscale x 4 x float> } %res
} }
define { <vscale x 2 x double>, <vscale x 2 x double> } @multi_vec_minnm_x2_f64(<vscale x 8 x half> %dummy, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2) { define { <vscale x 2 x double>, <vscale x 2 x double> } @multi_vec_minnm_x2_f64(<vscale x 8 x half> %dummy, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2) {
; CHECK-LABEL: multi_vec_minnm_x2_f64: ; CHECK-LABEL: multi_vec_minnm_x2_f64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fminnm.x2.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2) %res = call { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fminnm.x2.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2)
ret { <vscale x 2 x double>, <vscale x 2 x double> } %res ret { <vscale x 2 x double>, <vscale x 2 x double> } %res
} }
; FMINNM (Multi, x4) ; FMINNM (Multi, x4)
define { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } define { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> }
@multi_vec_minnm_x4_f16(<vscale x 8 x half> %dummy, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2, <vscale x 8 x half> %zm3, <vscale x 8 x half> %zm4) { @multi_vec_minnm_x4_f16(<vscale x 8 x half> %dummy, <vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2, <vscale x 8 x half> %zm3, <vscale x 8 x half> %zm4) {
; CHECK-LABEL: multi_vec_minnm_x4_f16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.h
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1h { z31.h }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: fminnm { z24.h - z27.h }, { z24.h - z27.h }, { z28.h - z31.h }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } %res = call { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> }
@llvm.aarch64.sve.fminnm.x4.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4, @llvm.aarch64.sve.fminnm.x4.nxv8f16(<vscale x 8 x half> %zdn1, <vscale x 8 x half> %zdn2, <vscale x 8 x half> %zdn3, <vscale x 8 x half> %zdn4,
<vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2, <vscale x 8 x half> %zm3, <vscale x 8 x half> %zm4) <vscale x 8 x half> %zm1, <vscale x 8 x half> %zm2, <vscale x 8 x half> %zm3, <vscale x 8 x half> %zm4)
ret { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } %res ret { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } %res
} }
define { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } define { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> }
@multi_vec_minnm_x4_f32(<vscale x 8 x half> %dummy, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2, <vscale x 4 x float> %zm3, <vscale x 4 x float> %zm4) { @multi_vec_minnm_x4_f32(<vscale x 8 x half> %dummy, <vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2, <vscale x 4 x float> %zm3, <vscale x 4 x float> %zm4) {
; CHECK-LABEL: multi_vec_minnm_x4_f32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.s
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1w { z31.s }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: fminnm { z24.s - z27.s }, { z24.s - z27.s }, { z28.s - z31.s }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } %res = call { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> }
@llvm.aarch64.sve.fminnm.x4.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4, @llvm.aarch64.sve.fminnm.x4.nxv4f32(<vscale x 4 x float> %zdn1, <vscale x 4 x float> %zdn2, <vscale x 4 x float> %zdn3, <vscale x 4 x float> %zdn4,
<vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2, <vscale x 4 x float> %zm3, <vscale x 4 x float> %zm4) <vscale x 4 x float> %zm1, <vscale x 4 x float> %zm2, <vscale x 4 x float> %zm3, <vscale x 4 x float> %zm4)
ret { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } %res ret { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } %res
} }
define { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } define { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> }
@multi_vec_minnm_x4_f64(<vscale x 8 x half> %dummy, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2, <vscale x 2 x double> %zm3, <vscale x 2 x double> %zm4) { @multi_vec_minnm_x4_f64(<vscale x 8 x half> %dummy, <vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2, <vscale x 2 x double> %zm3, <vscale x 2 x double> %zm4) {
; CHECK-LABEL: multi_vec_minnm_x4_f64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.d
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1d { z31.d }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: fminnm { z24.d - z27.d }, { z24.d - z27.d }, { z28.d - z31.d }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } %res = call { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> }
@llvm.aarch64.sve.fminnm.x4.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4, @llvm.aarch64.sve.fminnm.x4.nxv2f64(<vscale x 2 x double> %zdn1, <vscale x 2 x double> %zdn2, <vscale x 2 x double> %zdn3, <vscale x 2 x double> %zdn4,
<vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2, <vscale x 2 x double> %zm3, <vscale x 2 x double> %zm4) <vscale x 2 x double> %zm1, <vscale x 2 x double> %zm2, <vscale x 2 x double> %zm3, <vscale x 2 x double> %zm4)
ret { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } %res ret { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } %res
} }
declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.smin.single.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.smin.single.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.smin.single.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>) declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.smin.single.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>)
declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.smin.single.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.smin.single.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.smin.single.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.smin.single.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.umin.single.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.umin.single.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.umin.single.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>) declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.umin.single.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>)
declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.umin.single.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.umin.single.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.umin.single.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.umin.single.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fmin.single.x2.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>) declare { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fmin.single.x2.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>)
declare { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fmin.single.x2.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>) declare { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fmin.single.x2.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>)
declare { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fmin.single.x2.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>) declare { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fmin.single.x2.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.smin.single.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.smin.single.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.smin.single.x4.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>) declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.smin.single.x4.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>)
declare { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.smin.single.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.smin.single.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.smin.single.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.smin.single.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.umin.single.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.umin.single.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.umin.single.x4.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>) declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.umin.single.x4.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>)
declare { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.umin.single.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.umin.single.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.umin.single.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.umin.single.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 8 x half>, <vscale x 8 x half> , <vscale x 8 x half>, <vscale x 8 x half> } declare { <vscale x 8 x half>, <vscale x 8 x half> , <vscale x 8 x half>, <vscale x 8 x half> }
@llvm.aarch64.sve.fmin.single.x4.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>) @llvm.aarch64.sve.fmin.single.x4.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>)
declare { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } declare { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> }
@llvm.aarch64.sve.fmin.single.x4.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>) @llvm.aarch64.sve.fmin.single.x4.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>)
declare { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } declare { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> }
@llvm.aarch64.sve.fmin.single.x4.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>) @llvm.aarch64.sve.fmin.single.x4.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.smin.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.smin.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.smin.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> ) declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.smin.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> )
declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.smin.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.smin.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.smin.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.smin.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.umin.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.umin.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.umin.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> ) declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.umin.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> )
declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.umin.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.umin.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.umin.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.umin.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fmin.x2.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>) declare { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fmin.x2.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>)
declare { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fmin.x2.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>) declare { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fmin.x2.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>)
declare { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fmin.x2.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>) declare { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fmin.x2.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.smin.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) @llvm.aarch64.sve.smin.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
Context not available.
@llvm.aarch64.sve.smin.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) @llvm.aarch64.sve.smin.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.smin.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) @llvm.aarch64.sve.smin.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.umin.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) @llvm.aarch64.sve.umin.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
Context not available.
@llvm.aarch64.sve.umin.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) @llvm.aarch64.sve.umin.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.umin.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) @llvm.aarch64.sve.umin.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 8 x half>, <vscale x 8 x half> , <vscale x 8 x half>, <vscale x 8 x half> } declare { <vscale x 8 x half>, <vscale x 8 x half> , <vscale x 8 x half>, <vscale x 8 x half> }
@llvm.aarch64.sve.fmin.x4.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>) @llvm.aarch64.sve.fmin.x4.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>)
declare { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } declare { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> }
@llvm.aarch64.sve.fmin.x4.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>) @llvm.aarch64.sve.fmin.x4.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>)
declare { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } declare { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> }
@llvm.aarch64.sve.fmin.x4.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>) @llvm.aarch64.sve.fmin.x4.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>)
declare { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fminnm.single.x2.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>) declare { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fminnm.single.x2.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>)
declare { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fminnm.single.x2.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>) declare { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fminnm.single.x2.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>)
declare { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fminnm.single.x2.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>) declare { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fminnm.single.x2.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>)
declare { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } declare { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> }
@llvm.aarch64.sve.fminnm.single.x4.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>) @llvm.aarch64.sve.fminnm.single.x4.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>)
declare { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } declare { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> }
@llvm.aarch64.sve.fminnm.single.x4.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>) @llvm.aarch64.sve.fminnm.single.x4.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>)
declare { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> } declare { <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double> }
@llvm.aarch64.sve.fminnm.single.x4.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>) @llvm.aarch64.sve.fminnm.single.x4.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>)
declare { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fminnm.x2.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>) declare { <vscale x 8 x half>, <vscale x 8 x half> } @llvm.aarch64.sve.fminnm.x2.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>)
declare { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fminnm.x2.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>) declare { <vscale x 4 x float>, <vscale x 4 x float> } @llvm.aarch64.sve.fminnm.x2.nxv4f32(<vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>)
declare { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fminnm.x2.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>) declare { <vscale x 2 x double>, <vscale x 2 x double> } @llvm.aarch64.sve.fminnm.x2.nxv2f64(<vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>, <vscale x 2 x double>)
declare { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> } declare { <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half> }
@llvm.aarch64.sve.fminnm.x4.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>) @llvm.aarch64.sve.fminnm.x4.nxv8f16(<vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>, <vscale x 8 x half>)
declare { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> } declare { <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float>, <vscale x 4 x float> }
Context not available.

llvm/test/CodeGen/AArch64/sme2-intrinsics-mlall.ll

llvm/test/CodeGen/AArch64/sme2-intrinsics-mlals.ll

llvm/test/CodeGen/AArch64/sme2-intrinsics-rshl.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc -mtriple=aarch64-linux-gnu -mattr=+sme2 -verify-machineinstrs < %s | FileCheck %s ; RUN: llc -mtriple=aarch64-linux-gnu -mattr=+sme2 -verify-machineinstrs < %s | FileCheck %s
; SRSHL (Single, x2) ; SRSHL (Single, x2)
define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_rounding_shl_single_x2_s8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) { define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_rounding_shl_single_x2_s8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x2_s8: ; CHECK-LABEL: multi_vec_rounding_shl_single_x2_s8:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.srshl.single.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) %res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.srshl.single.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm)
ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_rounding_shl_single_x2_s16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) { define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_rounding_shl_single_x2_s16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x2_s16: ; CHECK-LABEL: multi_vec_rounding_shl_single_x2_s16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.srshl.single.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) %res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.srshl.single.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm)
ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_rounding_shl_single_x2_s32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) { define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_rounding_shl_single_x2_s32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x2_s32: ; CHECK-LABEL: multi_vec_rounding_shl_single_x2_s32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.srshl.single.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) %res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.srshl.single.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm)
ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_rounding_shl_single_x2_s64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) { define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_rounding_shl_single_x2_s64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x2_s64: ; CHECK-LABEL: multi_vec_rounding_shl_single_x2_s64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.srshl.single.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) %res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.srshl.single.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm)
ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; SRSHL (Single, x4) ; SRSHL (Single, x4)
define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
dmgreenUnsubmitted
Done
ReplyInline Actions

This doesn't look right.

dmgreen: This doesn't look right.
@multi_vec_rounding_shl_single_x4_s8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) { @multi_vec_rounding_shl_single_x4_s8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x4_s8:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: srshl { z24.b - z27.b }, { z24.b - z27.b }, z5.b
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.srshl.single.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) @llvm.aarch64.sve.srshl.single.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm)
ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@multi_vec_rounding_shl_single_x4_s16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) { @multi_vec_rounding_shl_single_x4_s16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x4_s16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: srshl { z24.h - z27.h }, { z24.h - z27.h }, z5.h
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@llvm.aarch64.sve.srshl.single.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) @llvm.aarch64.sve.srshl.single.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm)
ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@multi_vec_rounding_shl_single_x4_s32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) { @multi_vec_rounding_shl_single_x4_s32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x4_s32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: srshl { z24.s - z27.s }, { z24.s - z27.s }, z5.s
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@llvm.aarch64.sve.srshl.single.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) @llvm.aarch64.sve.srshl.single.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm)
ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@multi_vec_rounding_shl_single_x4_s64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) { @multi_vec_rounding_shl_single_x4_s64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x4_s64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: srshl { z24.d - z27.d }, { z24.d - z27.d }, z5.d
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.srshl.single.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) @llvm.aarch64.sve.srshl.single.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm)
ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; URSHL (Single, x2) ; URSHL (Single, x2)
define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_rounding_shl_single_x2_u8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) { define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_rounding_shl_single_x2_u8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x2_u8: ; CHECK-LABEL: multi_vec_rounding_shl_single_x2_u8:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.urshl.single.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) %res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.urshl.single.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm)
ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_rounding_shl_single_x2_u16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) { define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_rounding_shl_single_x2_u16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x2_u16: ; CHECK-LABEL: multi_vec_rounding_shl_single_x2_u16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.urshl.single.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) %res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.urshl.single.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm)
ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_rounding_shl_single_x2_u32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) { define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_rounding_shl_single_x2_u32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x2_u32: ; CHECK-LABEL: multi_vec_rounding_shl_single_x2_u32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.urshl.single.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) %res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.urshl.single.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm)
ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_rounding_shl_single_x2_u64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) { define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_rounding_shl_single_x2_u64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x2_u64: ; CHECK-LABEL: multi_vec_rounding_shl_single_x2_u64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.urshl.single.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) %res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.urshl.single.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm)
ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; URSHL (Single, x4) ; URSHL (Single, x4)
define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@multi_vec_rounding_shl_single_x4_u8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) { @multi_vec_rounding_shl_single_x4_u8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x4_u8:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: urshl { z24.b - z27.b }, { z24.b - z27.b }, z5.b
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.urshl.single.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) @llvm.aarch64.sve.urshl.single.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm)
ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@multi_vec_rounding_shl_single_x4_u16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) { @multi_vec_rounding_shl_single_x4_u16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x4_u16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: urshl { z24.h - z27.h }, { z24.h - z27.h }, z5.h
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@llvm.aarch64.sve.urshl.single.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) @llvm.aarch64.sve.urshl.single.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm)
ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@multi_vec_rounding_shl_single_x4_u32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) { @multi_vec_rounding_shl_single_x4_u32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x4_u32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: urshl { z24.s - z27.s }, { z24.s - z27.s }, z5.s
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@llvm.aarch64.sve.urshl.single.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) @llvm.aarch64.sve.urshl.single.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm)
ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@multi_vec_rounding_shl_single_x4_u64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) { @multi_vec_rounding_shl_single_x4_u64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) {
; CHECK-LABEL: multi_vec_rounding_shl_single_x4_u64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: urshl { z24.d - z27.d }, { z24.d - z27.d }, z5.d
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.urshl.single.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) @llvm.aarch64.sve.urshl.single.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm)
ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; SRSHL (Multi, x2) ; SRSHL (Multi, x2)
define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_rounding_shl_x2_s8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) { define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_rounding_shl_x2_s8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) {
; CHECK-LABEL: multi_vec_rounding_shl_x2_s8: ; CHECK-LABEL: multi_vec_rounding_shl_x2_s8:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.srshl.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) %res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.srshl.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2)
ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_rounding_shl_x2_s16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) { define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_rounding_shl_x2_s16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) {
; CHECK-LABEL: multi_vec_rounding_shl_x2_s16: ; CHECK-LABEL: multi_vec_rounding_shl_x2_s16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.srshl.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) %res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.srshl.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2)
ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_rounding_shl_x2_s32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) { define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_rounding_shl_x2_s32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) {
; CHECK-LABEL: multi_vec_rounding_shl_x2_s32: ; CHECK-LABEL: multi_vec_rounding_shl_x2_s32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.srshl.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) %res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.srshl.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2)
ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_rounding_shl_x2_s64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) { define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_rounding_shl_x2_s64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) {
; CHECK-LABEL: multi_vec_rounding_shl_x2_s64: ; CHECK-LABEL: multi_vec_rounding_shl_x2_s64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.srshl.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) %res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.srshl.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2)
ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; SRSHL (Multi, x4) ; SRSHL (Multi, x4)
define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@multi_vec_rounding_shl_x4_s8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) { @multi_vec_rounding_shl_x4_s8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) {
; CHECK-LABEL: multi_vec_rounding_shl_x4_s8:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.b
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1b { z31.b }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: srshl { z24.b - z27.b }, { z24.b - z27.b }, { z28.b - z31.b }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.srshl.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, @llvm.aarch64.sve.srshl.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4,
<vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4)
ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@multi_vec_rounding_shl_x4_s16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) { @multi_vec_rounding_shl_x4_s16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) {
; CHECK-LABEL: multi_vec_rounding_shl_x4_s16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.h
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1h { z31.h }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: srshl { z24.h - z27.h }, { z24.h - z27.h }, { z28.h - z31.h }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@llvm.aarch64.sve.srshl.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, @llvm.aarch64.sve.srshl.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4,
<vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4)
ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@multi_vec_rounding_shl_x4_s32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) { @multi_vec_rounding_shl_x4_s32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) {
; CHECK-LABEL: multi_vec_rounding_shl_x4_s32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.s
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1w { z31.s }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: srshl { z24.s - z27.s }, { z24.s - z27.s }, { z28.s - z31.s }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@llvm.aarch64.sve.srshl.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, @llvm.aarch64.sve.srshl.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4,
<vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4)
ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@multi_vec_rounding_shl_x4_s64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) { @multi_vec_rounding_shl_x4_s64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) {
; CHECK-LABEL: multi_vec_rounding_shl_x4_s64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.d
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1d { z31.d }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: srshl { z24.d - z27.d }, { z24.d - z27.d }, { z28.d - z31.d }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.srshl.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, @llvm.aarch64.sve.srshl.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4,
<vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4)
ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; URSHL (Multi, x2) ; URSHL (Multi, x2)
define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_rounding_uhl_x2_u8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) { define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_rounding_uhl_x2_u8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) {
; CHECK-LABEL: multi_vec_rounding_uhl_x2_u8: ; CHECK-LABEL: multi_vec_rounding_uhl_x2_u8:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.urshl.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) %res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.urshl.x2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2)
ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_rounding_uhl_x2_u16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) { define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_rounding_uhl_x2_u16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) {
; CHECK-LABEL: multi_vec_rounding_uhl_x2_u16: ; CHECK-LABEL: multi_vec_rounding_uhl_x2_u16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.urshl.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) %res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.urshl.x2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2)
ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_rounding_uhl_x2_u32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) { define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_rounding_uhl_x2_u32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) {
; CHECK-LABEL: multi_vec_rounding_uhl_x2_u32: ; CHECK-LABEL: multi_vec_rounding_uhl_x2_u32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.urshl.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) %res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.urshl.x2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2)
ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_rounding_uhl_x2_u64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) { define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_rounding_uhl_x2_u64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) {
; CHECK-LABEL: multi_vec_rounding_uhl_x2_u64: ; CHECK-LABEL: multi_vec_rounding_uhl_x2_u64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.urshl.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) %res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.urshl.x2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2)
ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; URSHL (Multi, x4) ; URSHL (Multi, x4)
define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@multi_vec_rounding_shl_x4_u8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) { @multi_vec_rounding_shl_x4_u8(<vscale x 16 x i8> %dummy, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) {
; CHECK-LABEL: multi_vec_rounding_shl_x4_u8:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.b
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1b { z31.b }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: urshl { z24.b - z27.b }, { z24.b - z27.b }, { z28.b - z31.b }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.urshl.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, @llvm.aarch64.sve.urshl.x4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4,
<vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4)
ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@multi_vec_rounding_shl_x4_u16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) { @multi_vec_rounding_shl_x4_u16(<vscale x 8 x i16> %dummy, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) {
; CHECK-LABEL: multi_vec_rounding_shl_x4_u16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.h
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1h { z31.h }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: urshl { z24.h - z27.h }, { z24.h - z27.h }, { z28.h - z31.h }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@llvm.aarch64.sve.urshl.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, @llvm.aarch64.sve.urshl.x4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4,
<vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4)
ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@multi_vec_rounding_shl_x4_u32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) { @multi_vec_rounding_shl_x4_u32(<vscale x 4 x i32> %dummy, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) {
; CHECK-LABEL: multi_vec_rounding_shl_x4_u32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.s
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1w { z31.s }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: urshl { z24.s - z27.s }, { z24.s - z27.s }, { z28.s - z31.s }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@llvm.aarch64.sve.urshl.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, @llvm.aarch64.sve.urshl.x4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4,
<vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4)
ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@multi_vec_rounding_shl_x4_u64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) { @multi_vec_rounding_shl_x4_u64(<vscale x 2 x i64> %dummy, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) {
; CHECK-LABEL: multi_vec_rounding_shl_x4_u64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.d
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1d { z31.d }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: urshl { z24.d - z27.d }, { z24.d - z27.d }, { z28.d - z31.d }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.urshl.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, @llvm.aarch64.sve.urshl.x4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4,
<vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4)
ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.srshl.single.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.srshl.single.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.srshl.single.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>) declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.srshl.single.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>)
declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.srshl.single.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.srshl.single.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.srshl.single.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.srshl.single.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.srshl.single.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.srshl.single.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.srshl.single.x4.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>) declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.srshl.single.x4.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>)
declare { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.srshl.single.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.srshl.single.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.srshl.single.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.srshl.single.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.urshl.single.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.urshl.single.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.urshl.single.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>) declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.urshl.single.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>)
declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.urshl.single.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.urshl.single.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.urshl.single.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.urshl.single.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.urshl.single.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.urshl.single.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.urshl.single.x4.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>) declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.urshl.single.x4.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>)
declare { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.urshl.single.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.urshl.single.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.urshl.single.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.urshl.single.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.srshl.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.srshl.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.srshl.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>) declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.srshl.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>)
declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.srshl.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.srshl.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.srshl.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.srshl.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.srshl.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) @llvm.aarch64.sve.srshl.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
Context not available.
@llvm.aarch64.sve.srshl.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) @llvm.aarch64.sve.srshl.x4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.srshl.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> ) @llvm.aarch64.sve.srshl.x4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> )
declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.urshl.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.urshl.x2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.urshl.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>) declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.urshl.x2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>)
declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.urshl.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.urshl.x2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.urshl.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.urshl.x2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.urshl.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) @llvm.aarch64.sve.urshl.x4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
Context not available.

llvm/test/CodeGen/AArch64/sme2-intrinsics-sqdmulh.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc -mtriple=aarch64-linux-gnu -mattr=+sme2 -verify-machineinstrs < %s | FileCheck %s ; RUN: llc -mtriple=aarch64-linux-gnu -mattr=+sme2 -verify-machineinstrs < %s | FileCheck %s
; SQDMULH (Single, x2) ; SQDMULH (Single, x2)
define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_sat_double_mulh_single_x2_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) { define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_sat_double_mulh_single_x2_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) {
; CHECK-LABEL: multi_vec_sat_double_mulh_single_x2_s8: ; CHECK-LABEL: multi_vec_sat_double_mulh_single_x2_s8:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm) %res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm)
ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_sat_double_mulh_single_x2_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) { define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_sat_double_mulh_single_x2_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) {
; CHECK-LABEL: multi_vec_sat_double_mulh_single_x2_s16: ; CHECK-LABEL: multi_vec_sat_double_mulh_single_x2_s16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm) %res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm)
ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_sat_double_mulh_single_x2_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) { define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_sat_double_mulh_single_x2_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) {
; CHECK-LABEL: multi_vec_sat_double_mulh_single_x2_s32: ; CHECK-LABEL: multi_vec_sat_double_mulh_single_x2_s32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm) %res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm)
ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_sat_double_mulh_single_x2_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) { define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_sat_double_mulh_single_x2_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) {
; CHECK-LABEL: multi_vec_sat_double_mulh_single_x2_s64: ; CHECK-LABEL: multi_vec_sat_double_mulh_single_x2_s64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm) %res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm)
ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; SQDMULH (Single, x4) ; SQDMULH (Single, x4)
define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@multi_vec_sat_double_mulh_single_x4_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) { @multi_vec_sat_double_mulh_single_x4_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) {
; CHECK-LABEL: multi_vec_sat_double_mulh_single_x4_s8:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: sqdmulh { z24.b - z27.b }, { z24.b - z27.b }, z5.b
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
dmgreenUnsubmitted
Done
ReplyInline Actions

This doesn't look right.

dmgreen: This doesn't look right.
@llvm.aarch64.sve.sqdmulh.single.vgx4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm) @llvm.aarch64.sve.sqdmulh.single.vgx4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, <vscale x 16 x i8> %zm)
ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@multi_vec_sat_double_mulh_single_x4_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) { @multi_vec_sat_double_mulh_single_x4_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) {
; CHECK-LABEL: multi_vec_sat_double_mulh_single_x4_s16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: sqdmulh { z24.h - z27.h }, { z24.h - z27.h }, z5.h
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@llvm.aarch64.sve.sqdmulh.single.vgx4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm) @llvm.aarch64.sve.sqdmulh.single.vgx4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, <vscale x 8 x i16> %zm)
ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@multi_vec_sat_double_mulh_single_x4_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) { @multi_vec_sat_double_mulh_single_x4_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) {
; CHECK-LABEL: multi_vec_sat_double_mulh_single_x4_s32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: sqdmulh { z24.s - z27.s }, { z24.s - z27.s }, z5.s
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@llvm.aarch64.sve.sqdmulh.single.vgx4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm) @llvm.aarch64.sve.sqdmulh.single.vgx4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, <vscale x 4 x i32> %zm)
ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@multi_vec_sat_double_mulh_single_x4_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) { @multi_vec_sat_double_mulh_single_x4_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) {
; CHECK-LABEL: multi_vec_sat_double_mulh_single_x4_s64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: sqdmulh { z24.d - z27.d }, { z24.d - z27.d }, z5.d
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.sqdmulh.single.vgx4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm) @llvm.aarch64.sve.sqdmulh.single.vgx4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, <vscale x 2 x i64> %zm)
ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; SQDMULH (x2, Multi) ; SQDMULH (x2, Multi)
define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_sat_double_mulh_multi_x2_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) { define { <vscale x 16 x i8>, <vscale x 16 x i8> } @multi_vec_sat_double_mulh_multi_x2_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) {
; CHECK-LABEL: multi_vec_sat_double_mulh_multi_x2_s8: ; CHECK-LABEL: multi_vec_sat_double_mulh_multi_x2_s8:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2) %res = call { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2)
ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_sat_double_mulh_multi_x2_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) { define { <vscale x 8 x i16>, <vscale x 8 x i16> } @multi_vec_sat_double_mulh_multi_x2_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) {
; CHECK-LABEL: multi_vec_sat_double_mulh_multi_x2_s16: ; CHECK-LABEL: multi_vec_sat_double_mulh_multi_x2_s16:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2) %res = call { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2)
ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_sat_double_mulh_multi_x2_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) { define { <vscale x 4 x i32>, <vscale x 4 x i32> } @multi_vec_sat_double_mulh_multi_x2_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) {
; CHECK-LABEL: multi_vec_sat_double_mulh_multi_x2_s32: ; CHECK-LABEL: multi_vec_sat_double_mulh_multi_x2_s32:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2) %res = call { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2)
ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_sat_double_mulh_multi_x2_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) { define { <vscale x 2 x i64>, <vscale x 2 x i64> } @multi_vec_sat_double_mulh_multi_x2_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) {
; CHECK-LABEL: multi_vec_sat_double_mulh_multi_x2_s64: ; CHECK-LABEL: multi_vec_sat_double_mulh_multi_x2_s64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
Context not available.
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2) %res = call { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2)
ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
; SQDMULH (x4, Multi) ; SQDMULH (x4, Multi)
define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } define { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@multi_vec_sat_double_mulh_multi_x4_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, @multi_vec_sat_double_mulh_multi_x4_s8(<vscale x 16 x i8> %unused, <vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4,
<vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) { <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) {
; CHECK-LABEL: multi_vec_sat_double_mulh_multi_x4_s8:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.b
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1b { z31.b }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: sqdmulh { z24.b - z27.b }, { z24.b - z27.b }, { z28.b - z31.b }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res = call { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.sqdmulh.vgx4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4, @llvm.aarch64.sve.sqdmulh.vgx4.nxv16i8(<vscale x 16 x i8> %zdn1, <vscale x 16 x i8> %zdn2, <vscale x 16 x i8> %zdn3, <vscale x 16 x i8> %zdn4,
<vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4) <vscale x 16 x i8> %zm1, <vscale x 16 x i8> %zm2, <vscale x 16 x i8> %zm3, <vscale x 16 x i8> %zm4)
ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res ret { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } %res
} }
define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } define { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@multi_vec_sat_double_mulh_multi_x4_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, @multi_vec_sat_double_mulh_multi_x4_s16(<vscale x 8 x i16> %unused, <vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4,
<vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) { <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) {
; CHECK-LABEL: multi_vec_sat_double_mulh_multi_x4_s16:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.h
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1h { z31.h }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: sqdmulh { z24.h - z27.h }, { z24.h - z27.h }, { z28.h - z31.h }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res = call { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
@llvm.aarch64.sve.sqdmulh.vgx4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4, @llvm.aarch64.sve.sqdmulh.vgx4.nxv8i16(<vscale x 8 x i16> %zdn1, <vscale x 8 x i16> %zdn2, <vscale x 8 x i16> %zdn3, <vscale x 8 x i16> %zdn4,
<vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4) <vscale x 8 x i16> %zm1, <vscale x 8 x i16> %zm2, <vscale x 8 x i16> %zm3, <vscale x 8 x i16> %zm4)
ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res ret { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } %res
} }
define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } define { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@multi_vec_sat_double_mulh_multi_x4_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, @multi_vec_sat_double_mulh_multi_x4_s32(<vscale x 4 x i32> %unused, <vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4,
<vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) { <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) {
; CHECK-LABEL: multi_vec_sat_double_mulh_multi_x4_s32:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.s
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1w { z31.s }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: sqdmulh { z24.s - z27.s }, { z24.s - z27.s }, { z28.s - z31.s }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res = call { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> }
@llvm.aarch64.sve.sqdmulh.vgx4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4, @llvm.aarch64.sve.sqdmulh.vgx4.nxv4i32(<vscale x 4 x i32> %zdn1, <vscale x 4 x i32> %zdn2, <vscale x 4 x i32> %zdn3, <vscale x 4 x i32> %zdn4,
<vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4) <vscale x 4 x i32> %zm1, <vscale x 4 x i32> %zm2, <vscale x 4 x i32> %zm3, <vscale x 4 x i32> %zm4)
ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res ret { <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32> } %res
} }
define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } define { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@multi_vec_sat_double_mulh_multi_x4_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, @multi_vec_sat_double_mulh_multi_x4_s64(<vscale x 2 x i64> %unused, <vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4,
<vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) { <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) {
; CHECK-LABEL: multi_vec_sat_double_mulh_multi_x4_s64:
; CHECK: // %bb.0:
; CHECK-NEXT: mov z30.d, z7.d
; CHECK-NEXT: ptrue p0.d
; CHECK-NEXT: mov z29.d, z6.d
; CHECK-NEXT: mov z27.d, z4.d
; CHECK-NEXT: mov z28.d, z5.d
; CHECK-NEXT: mov z26.d, z3.d
; CHECK-NEXT: ld1d { z31.d }, p0/z, [x0]
; CHECK-NEXT: mov z25.d, z2.d
; CHECK-NEXT: mov z24.d, z1.d
; CHECK-NEXT: sqdmulh { z24.d - z27.d }, { z24.d - z27.d }, { z28.d - z31.d }
; CHECK-NEXT: mov z0.d, z24.d
; CHECK-NEXT: mov z1.d, z25.d
; CHECK-NEXT: mov z2.d, z26.d
; CHECK-NEXT: mov z3.d, z27.d
; CHECK-NEXT: ret
%res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res = call { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.sqdmulh.vgx4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4, @llvm.aarch64.sve.sqdmulh.vgx4.nxv2i64(<vscale x 2 x i64> %zdn1, <vscale x 2 x i64> %zdn2, <vscale x 2 x i64> %zdn3, <vscale x 2 x i64> %zdn4,
<vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4) <vscale x 2 x i64> %zm1, <vscale x 2 x i64> %zm2, <vscale x 2 x i64> %zm3, <vscale x 2 x i64> %zm4)
ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res ret { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } %res
} }
declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>) declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>)
declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.sqdmulh.single.vgx2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.sqdmulh.single.vgx4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) @llvm.aarch64.sve.sqdmulh.single.vgx4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> } declare { <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16> }
Context not available.
@llvm.aarch64.sve.sqdmulh.single.vgx4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) @llvm.aarch64.sve.sqdmulh.single.vgx4.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> } declare { <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64> }
@llvm.aarch64.sve.sqdmulh.single.vgx4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) @llvm.aarch64.sve.sqdmulh.single.vgx4.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) declare { <vscale x 16 x i8>, <vscale x 16 x i8> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>) declare { <vscale x 8 x i16>, <vscale x 8 x i16> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv8i16(<vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>, <vscale x 8 x i16>)
declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>) declare { <vscale x 4 x i32>, <vscale x 4 x i32> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv4i32(<vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>, <vscale x 4 x i32>)
declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>) declare { <vscale x 2 x i64>, <vscale x 2 x i64> } @llvm.aarch64.sve.sqdmulh.vgx2.nxv2i64(<vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>, <vscale x 2 x i64>)
declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> } declare { <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8> }
@llvm.aarch64.sve.sqdmulh.vgx4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, @llvm.aarch64.sve.sqdmulh.vgx4.nxv16i8(<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>,
<vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>) <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>, <vscale x 16 x i8>)
Context not available.

llvm/test/CodeGen/AArch64/sme2-intrinsics-sub.ll

llvm/test/CodeGen/AArch64/sme2-intrinsics-vdot.ll

llvm/test/CodeGen/AArch64/speculation-hardening-loads.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc < %s -verify-machineinstrs -mtriple=aarch64-none-linux-gnu | FileCheck %s ; RUN: llc < %s -verify-machineinstrs -mtriple=aarch64-none-linux-gnu | FileCheck %s
define i128 @ldp_single_csdb(ptr %p) speculative_load_hardening { define i128 @ldp_single_csdb(ptr %p) speculative_load_hardening {
; CHECK-LABEL: ldp_single_csdb:
dmgreenUnsubmitted
Done
ReplyInline Actions

Manual update might be better.

dmgreen: Manual update might be better.
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: ldp x8, x1, [x0]
; CHECK-NEXT: cmp sp, #0
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: and x8, x8, x16
; CHECK-NEXT: and x1, x1, x16
; CHECK-NEXT: csdb
; CHECK-NEXT: mov x2, sp
; CHECK-NEXT: mov x0, x8
; CHECK-NEXT: and x2, x2, x16
; CHECK-NEXT: mov sp, x2
; CHECK-NEXT: ret
entry: entry:
%0 = load i128, ptr %p, align 16 %0 = load i128, ptr %p, align 16
ret i128 %0 ret i128 %0
; CHECK-LABEL: ldp_single_csdb
; CHECK: ldp x8, x1, [x0]
; CHECK-NEXT: cmp sp, #0
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: and x8, x8, x16
; CHECK-NEXT: and x1, x1, x16
; CHECK-NEXT: csdb
; CHECK-NEXT: mov [[TMPREG:x[0-9]+]], sp
; CHECK-NEXT: mov x0, x8
; CHECK-NEXT: and [[TMPREG]], [[TMPREG]], x16
; CHECK-NEXT: mov sp, [[TMPREG]]
; CHECK-NEXT: ret
} }
define double @ld_double(ptr %p) speculative_load_hardening { define double @ld_double(ptr %p) speculative_load_hardening {
; CHECK-LABEL: ld_double:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: cmp sp, #0
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: and x0, x0, x16
; CHECK-NEXT: csdb
; CHECK-NEXT: ldr d0, [x0]
; CHECK-NEXT: mov x0, sp
; CHECK-NEXT: and x0, x0, x16
; CHECK-NEXT: mov sp, x0
; CHECK-NEXT: ret
entry: entry:
%0 = load double, ptr %p, align 8 %0 = load double, ptr %p, align 8
ret double %0 ret double %0
; Checking that the address laoded from is masked for a floating point load. ; Checking that the address laoded from is masked for a floating point load.
; CHECK-LABEL: ld_double
; CHECK: cmp sp, #0
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: and x0, x0, x16
; CHECK-NEXT: csdb
; CHECK-NEXT: ldr d0, [x0]
; CHECK-NEXT: mov [[TMPREG:x[0-9]+]], sp
; CHECK-NEXT: and [[TMPREG]], [[TMPREG]], x16
; CHECK-NEXT: mov sp, [[TMPREG]]
; CHECK-NEXT: ret
} }
define i32 @csdb_emitted_for_subreg_use(ptr %p, i32 %b) speculative_load_hardening { define i32 @csdb_emitted_for_subreg_use(ptr %p, i32 %b) speculative_load_hardening {
; CHECK-LABEL: csdb_emitted_for_subreg_use:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: cmp sp, #0
; CHECK-NEXT: ldr x8, [x0]
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: and x8, x8, x16
; CHECK-NEXT: csdb
; CHECK-NEXT: add w9, w1, w8
; CHECK-NEXT: cmp x8, #0
; CHECK-NEXT: csel w0, w1, w9, eq
; CHECK-NEXT: mov x1, sp
; CHECK-NEXT: and x1, x1, x16
; CHECK-NEXT: mov sp, x1
; CHECK-NEXT: ret
entry: entry:
%X = load i64, ptr %p, align 8 %X = load i64, ptr %p, align 8
%X_trunc = trunc i64 %X to i32 %X_trunc = trunc i64 %X to i32
Context not available.
%ret = select i1 %iszero, i32 %b, i32 %add %ret = select i1 %iszero, i32 %b, i32 %add
ret i32 %ret ret i32 %ret
; Checking that the address laoded from is masked for a floating point load. ; Checking that the address laoded from is masked for a floating point load.
; CHECK-LABEL: csdb_emitted_for_subreg_use
; CHECK: ldr x8, [x0]
; CHECK-NEXT: cmp sp, #0
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: and x8, x8, x16
; csdb instruction must occur before the add instruction with w8 as operand. ; csdb instruction must occur before the add instruction with w8 as operand.
; CHECK-NEXT: csdb
; CHECK-NEXT: add w9, w1, w8
; CHECK-NEXT: cmp x8, #0
; CHECK-NEXT: csel w0, w1, w9, eq
; CHECK-NEXT: mov [[TMPREG:x[0-9]+]], sp
; CHECK-NEXT: and [[TMPREG]], [[TMPREG]], x16
; CHECK-NEXT: mov sp, [[TMPREG]]
; CHECK-NEXT: ret
} }
define i64 @csdb_emitted_for_superreg_use(ptr %p, i64 %b) speculative_load_hardening { define i64 @csdb_emitted_for_superreg_use(ptr %p, i64 %b) speculative_load_hardening {
; CHECK-LABEL: csdb_emitted_for_superreg_use:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: cmp sp, #0
; CHECK-NEXT: ldr w8, [x0]
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: and w8, w8, w16
; CHECK-NEXT: csdb
; CHECK-NEXT: add x9, x1, x8
; CHECK-NEXT: cmp w8, #0
; CHECK-NEXT: csel x0, x1, x9, eq
; CHECK-NEXT: mov x1, sp
; CHECK-NEXT: and x1, x1, x16
; CHECK-NEXT: mov sp, x1
; CHECK-NEXT: ret
entry: entry:
%X = load i32, ptr %p, align 4 %X = load i32, ptr %p, align 4
%X_ext = zext i32 %X to i64 %X_ext = zext i32 %X to i64
Context not available.
%ret = select i1 %iszero, i64 %b, i64 %add %ret = select i1 %iszero, i64 %b, i64 %add
ret i64 %ret ret i64 %ret
; Checking that the address laoded from is masked for a floating point load. ; Checking that the address laoded from is masked for a floating point load.
; CHECK-LABEL: csdb_emitted_for_superreg_use
; CHECK: ldr w8, [x0]
; CHECK-NEXT: cmp sp, #0
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: and w8, w8, w16
; csdb instruction must occur before the add instruction with x8 as operand. ; csdb instruction must occur before the add instruction with x8 as operand.
; CHECK-NEXT: csdb
; CHECK-NEXT: add x9, x1, x8
; CHECK-NEXT: cmp w8, #0
; CHECK-NEXT: csel x0, x1, x9, eq
; CHECK-NEXT: mov [[TMPREG:x[0-9]+]], sp
; CHECK-NEXT: and [[TMPREG]], [[TMPREG]], x16
; CHECK-NEXT: mov sp, [[TMPREG]]
; CHECK-NEXT: ret
} }
define i64 @no_masking_with_full_control_flow_barriers(i64 %a, i64 %b, ptr %p) speculative_load_hardening { define i64 @no_masking_with_full_control_flow_barriers(i64 %a, i64 %b, ptr %p) speculative_load_hardening {
; CHECK-LABEL: no_masking_with_full_control_flow_barriers ; CHECK-LABEL: no_masking_with_full_control_flow_barriers:
; CHECK: dsb sy ; CHECK: // %bb.0: // %entry
; CHECK: isb ; CHECK-NEXT: dsb sy
; CHECK-NEXT: isb
; CHECK-NEXT: ldr x8, [x2]
; CHECK-NEXT: mov x17, x0
; CHECK-NEXT: mov x16, x1
; CHECK-NEXT: //APP
; CHECK-NEXT: hint #12
; CHECK-NEXT: //NO_APP
; CHECK-NEXT: add x0, x8, x17
; CHECK-NEXT: ret
entry: entry:
%0 = tail call i64 asm "hint #12", "={x17},{x16},0"(i64 %b, i64 %a) %0 = tail call i64 asm "hint #12", "={x17},{x16},0"(i64 %b, i64 %a)
%X = load i64, ptr %p, align 8 %X = load i64, ptr %p, align 8
%ret = add i64 %X, %0 %ret = add i64 %X, %0
; CHECK-NOT: csdb
; CHECK-NOT: and
; CHECK: ret
ret i64 %ret ret i64 %ret
} }
define void @f_implicitdef_vector_load(ptr %dst, ptr %src) speculative_load_hardening define void @f_implicitdef_vector_load(ptr %dst, ptr %src) speculative_load_hardening
; CHECK-LABEL: f_implicitdef_vector_load:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: cmp sp, #0
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: and x1, x1, x16
; CHECK-NEXT: csdb
; CHECK-NEXT: ldr d0, [x1]
; CHECK-NEXT: mov v0.d[1], v0.d[0]
; CHECK-NEXT: str q0, [x0]
; CHECK-NEXT: mov x0, sp
; CHECK-NEXT: and x0, x0, x16
; CHECK-NEXT: mov sp, x0
; CHECK-NEXT: ret
{ {
entry: entry:
%0 = load <2 x i32>, ptr %src, align 8 %0 = load <2 x i32>, ptr %src, align 8
%shuffle = shufflevector <2 x i32> %0, <2 x i32> undef, <4 x i32> <i32 0, i32 1, i32 0, i32 1> %shuffle = shufflevector <2 x i32> %0, <2 x i32> undef, <4 x i32> <i32 0, i32 1, i32 0, i32 1>
store <4 x i32> %shuffle, ptr %dst, align 4 store <4 x i32> %shuffle, ptr %dst, align 4
ret void ret void
; CHECK-LABEL: f_implicitdef_vector_load
; CHECK: cmp sp, #0
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: and x1, x1, x16
; CHECK-NEXT: csdb
; CHECK-NEXT: ldr d0, [x1]
; CHECK-NEXT: mov v0.d[1], v0.d[0]
; CHECK-NEXT: str q0, [x0]
; CHECK-NEXT: mov [[TMPREG:x[0-9]+]], sp
; CHECK-NEXT: and [[TMPREG]], [[TMPREG]], x16
; CHECK-NEXT: mov sp, [[TMPREG]]
; CHECK-NEXT: ret
} }
define <2 x double> @f_usedefvectorload(ptr %a, ptr %b) speculative_load_hardening { define <2 x double> @f_usedefvectorload(ptr %a, ptr %b) speculative_load_hardening {
; CHECK-LABEL: f_usedefvectorload:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: cmp sp, #0
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: and x1, x1, x16
; CHECK-NEXT: csdb
; CHECK-NEXT: mov x0, sp
; CHECK-NEXT: ldr d0, [x1]
; CHECK-NEXT: and x0, x0, x16
; CHECK-NEXT: mov sp, x0
; CHECK-NEXT: ret
entry: entry:
; CHECK-LABEL: f_usedefvectorload
; CHECK: cmp sp, #0
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: and x1, x1, x16
; CHECK-NEXT: csdb
; CHECK-NEXT: mov [[TMPREG:x[0-9]+]], sp
; CHECK-NEXT: ldr d0, [x1]
; CHECK-NEXT: and [[TMPREG]], [[TMPREG]], x16
; CHECK-NEXT: mov sp, [[TMPREG]]
; CHECK-NEXT: ret
%0 = load double, ptr %b, align 16 %0 = load double, ptr %b, align 16
%vld1_lane = insertelement <2 x double> <double undef, double 0.000000e+00>, double %0, i32 0 %vld1_lane = insertelement <2 x double> <double undef, double 0.000000e+00>, double %0, i32 0
ret <2 x double> %vld1_lane ret <2 x double> %vld1_lane
} }
define i32 @deadload() speculative_load_hardening uwtable { define i32 @deadload() speculative_load_hardening uwtable {
; CHECK-LABEL: deadload:
; CHECK: // %bb.0: // %entry
; CHECK-NEXT: cmp sp, #0
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: sub sp, sp, #16
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: ldr w8, [sp, #12]
; CHECK-NEXT: add sp, sp, #16
; CHECK-NEXT: .cfi_def_cfa_offset 0
; CHECK-NEXT: mov x0, sp
; CHECK-NEXT: and x0, x0, x16
; CHECK-NEXT: mov sp, x0
; CHECK-NEXT: ret
entry: entry:
; CHECK-LABEL: deadload
; CHECK: cmp sp, #0
; CHECK-NEXT: csetm x16, ne
; CHECK-NEXT: sub sp, sp, #16
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: ldr w8, [sp, #12]
; CHECK-NEXT: add sp, sp, #16
; CHECK-NEXT: .cfi_def_cfa_offset 0
; CHECK-NEXT: mov [[TMPREG:x[0-9]+]], sp
; CHECK-NEXT: and [[TMPREG]], [[TMPREG]], x16
; CHECK-NEXT: mov sp, [[TMPREG]]
; CHECK-NEXT: ret
%a = alloca i32, align 4 %a = alloca i32, align 4
%val = load volatile i32, ptr %a, align 4 %val = load volatile i32, ptr %a, align 4
ret i32 undef ret i32 undef
Context not available.

llvm/test/CodeGen/AArch64/split-vector-insert.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc < %s -debug-only=legalize-types 2>&1 | FileCheck %s --check-prefix=CHECK-LEGALIZATION ; RUN: llc < %s -debug-only=legalize-types 2>&1 | FileCheck %s --check-prefix=CHECK-LEGALIZATION
; RUN: llc < %s | FileCheck %s ; RUN: llc < %s | FileCheck %s
; REQUIRES: asserts ; REQUIRES: asserts
Context not available.
declare <vscale x 2 x double> @llvm.vector.insert.nxv2f64.v8f64(<vscale x 2 x double>, <8 x double>, i64) declare <vscale x 2 x double> @llvm.vector.insert.nxv2f64.v8f64(<vscale x 2 x double>, <8 x double>, i64)
define <vscale x 2 x i64> @test_nxv2i64_v8i64(<vscale x 2 x i64> %a, <8 x i64> %b) #0 { define <vscale x 2 x i64> @test_nxv2i64_v8i64(<vscale x 2 x i64> %a, <8 x i64> %b) #0 {
; CHECK-LEGALIZATION: Legally typed node: [[T1:t[0-9]+]]: nxv2i64 = insert_subvector {{t[0-9]+}}, {{t[0-9]+}}, Constant:i64<0> ; CHECK-LEGALIZATION-LABEL: test_nxv2i64_v8i64:
dmgreenUnsubmitted
Done
ReplyInline Actions

This was testing the debug output before.

dmgreen: This was testing the debug output before.
; CHECK-LEGALIZATION: Legally typed node: [[T2:t[0-9]+]]: nxv2i64 = insert_subvector [[T1]], {{t[0-9]+}}, Constant:i64<2> ; CHECK-LEGALIZATION: // %bb.0:
; CHECK-LEGALIZATION: Legally typed node: [[T3:t[0-9]+]]: nxv2i64 = insert_subvector [[T2]], {{t[0-9]+}}, Constant:i64<4> ; CHECK-LEGALIZATION-NEXT: str x29, [sp, #-16]! // 8-byte Folded Spill
; CHECK-LEGALIZATION: Legally typed node: [[T4:t[0-9]+]]: nxv2i64 = insert_subvector [[T3]], {{t[0-9]+}}, Constant:i64<6> ; CHECK-LEGALIZATION-NEXT: .cfi_def_cfa_offset 16
; CHECK-LEGALIZATION-NEXT: .cfi_offset w29, -16
; CHECK-LEGALIZATION-NEXT: addvl sp, sp, #-3
; CHECK-LEGALIZATION-NEXT: .cfi_escape 0x0f, 0x0c, 0x8f, 0x00, 0x11, 0x10, 0x22, 0x11, 0x18, 0x92, 0x2e, 0x00, 0x1e, 0x22 // sp + 16 + 24 * VG
; CHECK-LEGALIZATION-NEXT: ptrue p1.d, vl2
; CHECK-LEGALIZATION-NEXT: cntd x8
; CHECK-LEGALIZATION-NEXT: mov w9, #2 // =0x2
; CHECK-LEGALIZATION-NEXT: ptrue p0.d
; CHECK-LEGALIZATION-NEXT: sub x8, x8, #2
; CHECK-LEGALIZATION-NEXT: // kill: def $q1 killed $q1 def $z1
; CHECK-LEGALIZATION-NEXT: mov x10, sp
; CHECK-LEGALIZATION-NEXT: cmp x8, #2
; CHECK-LEGALIZATION-NEXT: csel x9, x8, x9, lo
; CHECK-LEGALIZATION-NEXT: cmp x8, #4
; CHECK-LEGALIZATION-NEXT: lsl x9, x9, #3
; CHECK-LEGALIZATION-NEXT: mov z0.d, p1/m, z1.d
; CHECK-LEGALIZATION-NEXT: st1d { z0.d }, p0, [sp]
; CHECK-LEGALIZATION-NEXT: str q2, [x10, x9]
; CHECK-LEGALIZATION-NEXT: mov w9, #4 // =0x4
; CHECK-LEGALIZATION-NEXT: addvl x10, sp, #1
; CHECK-LEGALIZATION-NEXT: ld1d { z0.d }, p0/z, [sp]
; CHECK-LEGALIZATION-NEXT: csel x9, x8, x9, lo
; CHECK-LEGALIZATION-NEXT: cmp x8, #6
; CHECK-LEGALIZATION-NEXT: lsl x9, x9, #3
; CHECK-LEGALIZATION-NEXT: st1d { z0.d }, p0, [sp, #1, mul vl]
; CHECK-LEGALIZATION-NEXT: str q3, [x10, x9]
; CHECK-LEGALIZATION-NEXT: mov w9, #6 // =0x6
; CHECK-LEGALIZATION-NEXT: ld1d { z0.d }, p0/z, [sp, #1, mul vl]
; CHECK-LEGALIZATION-NEXT: csel x8, x8, x9, lo
; CHECK-LEGALIZATION-NEXT: addvl x9, sp, #2
; CHECK-LEGALIZATION-NEXT: lsl x8, x8, #3
; CHECK-LEGALIZATION-NEXT: st1d { z0.d }, p0, [sp, #2, mul vl]
; CHECK-LEGALIZATION-NEXT: str q4, [x9, x8]
; CHECK-LEGALIZATION-NEXT: ld1d { z0.d }, p0/z, [sp, #2, mul vl]
; CHECK-LEGALIZATION-NEXT: addvl sp, sp, #3
; CHECK-LEGALIZATION-NEXT: .cfi_def_cfa wsp, 16
; CHECK-LEGALIZATION-NEXT: ldr x29, [sp], #16 // 8-byte Folded Reload
; CHECK-LEGALIZATION-NEXT: .cfi_def_cfa_offset 0
; CHECK-LEGALIZATION-NEXT: .cfi_restore w29
; CHECK-LEGALIZATION-NEXT: ret
;
; CHECK-LABEL: test_nxv2i64_v8i64: ; CHECK-LABEL: test_nxv2i64_v8i64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: str x29, [sp, #-16]! // 8-byte Folded Spill ; CHECK-NEXT: str x29, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; CHECK-NEXT: .cfi_offset w29, -16 ; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: addvl sp, sp, #-3 ; CHECK-NEXT: addvl sp, sp, #-3
; CHECK-NEXT: .cfi_escape 0x0f, 0x0c, 0x8f, 0x00, 0x11, 0x10, 0x22, 0x11, 0x18, 0x92, 0x2e, 0x00, 0x1e, 0x22 // sp + 16 + 24 * VG ; CHECK-NEXT: .cfi_escape 0x0f, 0x0c, 0x8f, 0x00, 0x11, 0x10, 0x22, 0x11, 0x18, 0x92, 0x2e, 0x00, 0x1e, 0x22 // sp + 16 + 24 * VG
; CHECK-NEXT: ptrue p1.d, vl2
; CHECK-NEXT: cntd x8 ; CHECK-NEXT: cntd x8
; CHECK-NEXT: mov w9, #2 ; CHECK-NEXT: mov w9, #2 // =0x2
; CHECK-NEXT: ptrue p0.d
; CHECK-NEXT: sub x8, x8, #2 ; CHECK-NEXT: sub x8, x8, #2
; CHECK-NEXT: ptrue p0.d, vl2 ; CHECK-NEXT: // kill: def $q1 killed $q1 def $z1
; CHECK-NEXT: cmp x8, #2
; CHECK-NEXT: mov x10, sp ; CHECK-NEXT: mov x10, sp
; CHECK-NEXT: cmp x8, #2
; CHECK-NEXT: csel x9, x8, x9, lo ; CHECK-NEXT: csel x9, x8, x9, lo
; CHECK-NEXT: // kill: def $q1 killed $q1 def $z1 ; CHECK-NEXT: cmp x8, #4
; CHECK-NEXT: mov z0.d, p0/m, z1.d
; CHECK-NEXT: lsl x9, x9, #3 ; CHECK-NEXT: lsl x9, x9, #3
; CHECK-NEXT: ptrue p0.d ; CHECK-NEXT: mov z0.d, p1/m, z1.d
; CHECK-NEXT: st1d { z0.d }, p0, [sp] ; CHECK-NEXT: st1d { z0.d }, p0, [sp]
; CHECK-NEXT: cmp x8, #4
; CHECK-NEXT: str q2, [x10, x9] ; CHECK-NEXT: str q2, [x10, x9]
; CHECK-NEXT: mov w9, #4 ; CHECK-NEXT: mov w9, #4 // =0x4
; CHECK-NEXT: addvl x10, sp, #1
; CHECK-NEXT: ld1d { z0.d }, p0/z, [sp] ; CHECK-NEXT: ld1d { z0.d }, p0/z, [sp]
; CHECK-NEXT: csel x9, x8, x9, lo ; CHECK-NEXT: csel x9, x8, x9, lo
; CHECK-NEXT: lsl x9, x9, #3
; CHECK-NEXT: addvl x10, sp, #1
; CHECK-NEXT: cmp x8, #6 ; CHECK-NEXT: cmp x8, #6
; CHECK-NEXT: lsl x9, x9, #3
; CHECK-NEXT: st1d { z0.d }, p0, [sp, #1, mul vl] ; CHECK-NEXT: st1d { z0.d }, p0, [sp, #1, mul vl]
; CHECK-NEXT: str q3, [x10, x9] ; CHECK-NEXT: str q3, [x10, x9]
; CHECK-NEXT: mov w9, #6 ; CHECK-NEXT: mov w9, #6 // =0x6
; CHECK-NEXT: ld1d { z0.d }, p0/z, [sp, #1, mul vl] ; CHECK-NEXT: ld1d { z0.d }, p0/z, [sp, #1, mul vl]
; CHECK-NEXT: csel x8, x8, x9, lo ; CHECK-NEXT: csel x8, x8, x9, lo
; CHECK-NEXT: addvl x9, sp, #2 ; CHECK-NEXT: addvl x9, sp, #2
Context not available.
; CHECK-NEXT: addvl sp, sp, #3 ; CHECK-NEXT: addvl sp, sp, #3
; CHECK-NEXT: .cfi_def_cfa wsp, 16 ; CHECK-NEXT: .cfi_def_cfa wsp, 16
; CHECK-NEXT: ldr x29, [sp], #16 // 8-byte Folded Reload ; CHECK-NEXT: ldr x29, [sp], #16 // 8-byte Folded Reload
; CHECK-NEXT: .cfi_def_cfa_offset 0 ; CHECK-NEXT: .cfi_def_cfa_offset 0
; CHECK-NEXT: .cfi_restore w29 ; CHECK-NEXT: .cfi_restore w29
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%r = call <vscale x 2 x i64> @llvm.vector.insert.nxv2i64.v8i64(<vscale x 2 x i64> %a, <8 x i64> %b, i64 0) %r = call <vscale x 2 x i64> @llvm.vector.insert.nxv2i64.v8i64(<vscale x 2 x i64> %a, <8 x i64> %b, i64 0)
ret <vscale x 2 x i64> %r ret <vscale x 2 x i64> %r
} }
define <vscale x 2 x double> @test_nxv2f64_v8f64(<vscale x 2 x double> %a, <8 x double> %b) #0 { define <vscale x 2 x double> @test_nxv2f64_v8f64(<vscale x 2 x double> %a, <8 x double> %b) #0 {
; CHECK-LEGALIZATION: Legally typed node: [[T1:t[0-9]+]]: nxv2f64 = insert_subvector {{t[0-9]+}}, {{t[0-9]+}}, Constant:i64<0> ; CHECK-LEGALIZATION-LABEL: test_nxv2f64_v8f64:
; CHECK-LEGALIZATION: Legally typed node: [[T2:t[0-9]+]]: nxv2f64 = insert_subvector [[T1]], {{t[0-9]+}}, Constant:i64<2> ; CHECK-LEGALIZATION: // %bb.0:
; CHECK-LEGALIZATION: Legally typed node: [[T3:t[0-9]+]]: nxv2f64 = insert_subvector [[T2]], {{t[0-9]+}}, Constant:i64<4> ; CHECK-LEGALIZATION-NEXT: str x29, [sp, #-16]! // 8-byte Folded Spill
; CHECK-LEGALIZATION: Legally typed node: [[T4:t[0-9]+]]: nxv2f64 = insert_subvector [[T3]], {{t[0-9]+}}, Constant:i64<6> ; CHECK-LEGALIZATION-NEXT: .cfi_def_cfa_offset 16
; CHECK-LEGALIZATION-NEXT: .cfi_offset w29, -16
; CHECK-LEGALIZATION-NEXT: addvl sp, sp, #-3
; CHECK-LEGALIZATION-NEXT: .cfi_escape 0x0f, 0x0c, 0x8f, 0x00, 0x11, 0x10, 0x22, 0x11, 0x18, 0x92, 0x2e, 0x00, 0x1e, 0x22 // sp + 16 + 24 * VG
; CHECK-LEGALIZATION-NEXT: ptrue p1.d, vl2
; CHECK-LEGALIZATION-NEXT: cntd x8
; CHECK-LEGALIZATION-NEXT: mov w9, #2 // =0x2
; CHECK-LEGALIZATION-NEXT: ptrue p0.d
; CHECK-LEGALIZATION-NEXT: sub x8, x8, #2
; CHECK-LEGALIZATION-NEXT: // kill: def $q1 killed $q1 def $z1
; CHECK-LEGALIZATION-NEXT: mov x10, sp
; CHECK-LEGALIZATION-NEXT: cmp x8, #2
; CHECK-LEGALIZATION-NEXT: csel x9, x8, x9, lo
; CHECK-LEGALIZATION-NEXT: cmp x8, #4
; CHECK-LEGALIZATION-NEXT: lsl x9, x9, #3
; CHECK-LEGALIZATION-NEXT: mov z0.d, p1/m, z1.d
; CHECK-LEGALIZATION-NEXT: st1d { z0.d }, p0, [sp]
; CHECK-LEGALIZATION-NEXT: str q2, [x10, x9]
; CHECK-LEGALIZATION-NEXT: mov w9, #4 // =0x4
; CHECK-LEGALIZATION-NEXT: addvl x10, sp, #1
; CHECK-LEGALIZATION-NEXT: ld1d { z0.d }, p0/z, [sp]
; CHECK-LEGALIZATION-NEXT: csel x9, x8, x9, lo
; CHECK-LEGALIZATION-NEXT: cmp x8, #6
; CHECK-LEGALIZATION-NEXT: lsl x9, x9, #3
; CHECK-LEGALIZATION-NEXT: st1d { z0.d }, p0, [sp, #1, mul vl]
; CHECK-LEGALIZATION-NEXT: str q3, [x10, x9]
; CHECK-LEGALIZATION-NEXT: mov w9, #6 // =0x6
; CHECK-LEGALIZATION-NEXT: ld1d { z0.d }, p0/z, [sp, #1, mul vl]
; CHECK-LEGALIZATION-NEXT: csel x8, x8, x9, lo
; CHECK-LEGALIZATION-NEXT: addvl x9, sp, #2
; CHECK-LEGALIZATION-NEXT: lsl x8, x8, #3
; CHECK-LEGALIZATION-NEXT: st1d { z0.d }, p0, [sp, #2, mul vl]
; CHECK-LEGALIZATION-NEXT: str q4, [x9, x8]
; CHECK-LEGALIZATION-NEXT: ld1d { z0.d }, p0/z, [sp, #2, mul vl]
; CHECK-LEGALIZATION-NEXT: addvl sp, sp, #3
; CHECK-LEGALIZATION-NEXT: .cfi_def_cfa wsp, 16
; CHECK-LEGALIZATION-NEXT: ldr x29, [sp], #16 // 8-byte Folded Reload
; CHECK-LEGALIZATION-NEXT: .cfi_def_cfa_offset 0
; CHECK-LEGALIZATION-NEXT: .cfi_restore w29
; CHECK-LEGALIZATION-NEXT: ret
;
; CHECK-LABEL: test_nxv2f64_v8f64: ; CHECK-LABEL: test_nxv2f64_v8f64:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: str x29, [sp, #-16]! // 8-byte Folded Spill ; CHECK-NEXT: str x29, [sp, #-16]! // 8-byte Folded Spill
Context not available.
; CHECK-NEXT: .cfi_offset w29, -16 ; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: addvl sp, sp, #-3 ; CHECK-NEXT: addvl sp, sp, #-3
; CHECK-NEXT: .cfi_escape 0x0f, 0x0c, 0x8f, 0x00, 0x11, 0x10, 0x22, 0x11, 0x18, 0x92, 0x2e, 0x00, 0x1e, 0x22 // sp + 16 + 24 * VG ; CHECK-NEXT: .cfi_escape 0x0f, 0x0c, 0x8f, 0x00, 0x11, 0x10, 0x22, 0x11, 0x18, 0x92, 0x2e, 0x00, 0x1e, 0x22 // sp + 16 + 24 * VG
; CHECK-NEXT: ptrue p1.d, vl2
; CHECK-NEXT: cntd x8 ; CHECK-NEXT: cntd x8
; CHECK-NEXT: mov w9, #2 ; CHECK-NEXT: mov w9, #2 // =0x2
; CHECK-NEXT: ptrue p0.d
; CHECK-NEXT: sub x8, x8, #2 ; CHECK-NEXT: sub x8, x8, #2
; CHECK-NEXT: ptrue p0.d, vl2 ; CHECK-NEXT: // kill: def $q1 killed $q1 def $z1
; CHECK-NEXT: cmp x8, #2
; CHECK-NEXT: mov x10, sp ; CHECK-NEXT: mov x10, sp
; CHECK-NEXT: cmp x8, #2
; CHECK-NEXT: csel x9, x8, x9, lo ; CHECK-NEXT: csel x9, x8, x9, lo
; CHECK-NEXT: // kill: def $q1 killed $q1 def $z1 ; CHECK-NEXT: cmp x8, #4
; CHECK-NEXT: mov z0.d, p0/m, z1.d
; CHECK-NEXT: lsl x9, x9, #3 ; CHECK-NEXT: lsl x9, x9, #3
; CHECK-NEXT: ptrue p0.d ; CHECK-NEXT: mov z0.d, p1/m, z1.d
; CHECK-NEXT: st1d { z0.d }, p0, [sp] ; CHECK-NEXT: st1d { z0.d }, p0, [sp]
; CHECK-NEXT: cmp x8, #4
; CHECK-NEXT: str q2, [x10, x9] ; CHECK-NEXT: str q2, [x10, x9]
; CHECK-NEXT: mov w9, #4 ; CHECK-NEXT: mov w9, #4 // =0x4
; CHECK-NEXT: addvl x10, sp, #1
; CHECK-NEXT: ld1d { z0.d }, p0/z, [sp] ; CHECK-NEXT: ld1d { z0.d }, p0/z, [sp]
; CHECK-NEXT: csel x9, x8, x9, lo ; CHECK-NEXT: csel x9, x8, x9, lo
; CHECK-NEXT: lsl x9, x9, #3
; CHECK-NEXT: addvl x10, sp, #1
; CHECK-NEXT: cmp x8, #6 ; CHECK-NEXT: cmp x8, #6
; CHECK-NEXT: lsl x9, x9, #3
; CHECK-NEXT: st1d { z0.d }, p0, [sp, #1, mul vl] ; CHECK-NEXT: st1d { z0.d }, p0, [sp, #1, mul vl]
; CHECK-NEXT: str q3, [x10, x9] ; CHECK-NEXT: str q3, [x10, x9]
; CHECK-NEXT: mov w9, #6 ; CHECK-NEXT: mov w9, #6 // =0x6
; CHECK-NEXT: ld1d { z0.d }, p0/z, [sp, #1, mul vl] ; CHECK-NEXT: ld1d { z0.d }, p0/z, [sp, #1, mul vl]
; CHECK-NEXT: csel x8, x8, x9, lo ; CHECK-NEXT: csel x8, x8, x9, lo
; CHECK-NEXT: addvl x9, sp, #2 ; CHECK-NEXT: addvl x9, sp, #2
Context not available.
; CHECK-NEXT: str q4, [x9, x8] ; CHECK-NEXT: str q4, [x9, x8]
; CHECK-NEXT: ld1d { z0.d }, p0/z, [sp, #2, mul vl] ; CHECK-NEXT: ld1d { z0.d }, p0/z, [sp, #2, mul vl]
; CHECK-NEXT: addvl sp, sp, #3 ; CHECK-NEXT: addvl sp, sp, #3
; CHECK-NEXT: .cfi_def_cfa wsp, 16 ; CHECK-NEXT: .cfi_def_cfa wsp, 16
; CHECK-NEXT: ldr x29, [sp], #16 // 8-byte Folded Reload ; CHECK-NEXT: ldr x29, [sp], #16 // 8-byte Folded Reload
; CHECK-NEXT: .cfi_def_cfa_offset 0 ; CHECK-NEXT: .cfi_def_cfa_offset 0
; CHECK-NEXT: .cfi_restore w29 ; CHECK-NEXT: .cfi_restore w29
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%r = call <vscale x 2 x double> @llvm.vector.insert.nxv2f64.v8f64(<vscale x 2 x double> %a, <8 x double> %b, i64 0) %r = call <vscale x 2 x double> @llvm.vector.insert.nxv2f64.v8f64(<vscale x 2 x double> %a, <8 x double> %b, i64 0)
ret <vscale x 2 x double> %r ret <vscale x 2 x double> %r
} }
Context not available.

llvm/test/CodeGen/AArch64/sqrt-fastmath.ll

llvm/test/CodeGen/AArch64/srem-seteq-illegal-types.ll

llvm/test/CodeGen/AArch64/srem-seteq-vec-nonsplat.ll

llvm/test/CodeGen/AArch64/srem-seteq-vec-splat.ll

llvm/test/CodeGen/AArch64/srem-vector-lkk.ll

llvm/test/CodeGen/AArch64/sshl_sat.ll

llvm/test/CodeGen/AArch64/ssub_sat.ll

llvm/test/CodeGen/AArch64/ssub_sat_plus.ll

llvm/test/CodeGen/AArch64/ssub_sat_vec.ll

llvm/test/CodeGen/AArch64/stack-guard-sysreg.ll

; RUN: split-file %s %t
; RUN: cat %t/main.ll %t/a.ll > %t/a2.ll ; RUN: cat %t/main.ll %t/a.ll > %t/a2.ll
dmgreenUnsubmitted
Done
ReplyInline Actions

Can you explain this one? Are the two versions now too different to keep the same?

dmgreen: Can you explain this one? Are the two versions now too different to keep the same?
harviniriawanAuthorUnsubmitted
Done
ReplyInline Actions

Yes indeed. the ones with CHECK-ADD annoyingly is scheduled slightly differently due to it being ADD instead of LDR

harviniriawan: Yes indeed. the ones with CHECK-ADD annoyingly is scheduled slightly differently due to it…
aeubanksUnsubmitted
Not Done
ReplyInline Actions

this test is broken, without the split-file the other files won't exist in a clean build, please fix or revert if it takes time to fix

aeubanks: this test is broken, without the `split-file` the other files won't exist in a clean build…
; RUN: cat %t/main.ll %t/b.ll > %t/b2.ll ; RUN: cat %t/main.ll %t/b.ll > %t/b2.ll
; RUN: cat %t/main.ll %t/c.ll > %t/c2.ll ; RUN: cat %t/main.ll %t/c.ll > %t/c2.ll
Context not available.
; RUN: llc %t/e2.ll -verify-machineinstrs -o - | \ ; RUN: llc %t/e2.ll -verify-machineinstrs -o - | \
; RUN: FileCheck --check-prefix=CHECK --check-prefix=CHECK-NPOT-NEG-OFFSET %s ; RUN: FileCheck --check-prefix=CHECK --check-prefix=CHECK-NPOT-NEG-OFFSET %s
; RUN: llc %t/f2.ll -verify-machineinstrs -o - | \ ; RUN: llc %t/f2.ll -verify-machineinstrs -o - | \
; RUN: FileCheck --check-prefix=CHECK --check-prefix=CHECK-257-OFFSET %s ; RUN: FileCheck --check-prefix=CHECK-ADD --check-prefix=CHECK-257-OFFSET %s
; RUN: llc %t/g2.ll -verify-machineinstrs -o - | \ ; RUN: llc %t/g2.ll -verify-machineinstrs -o - | \
; RUN: FileCheck --check-prefix=CHECK --check-prefix=CHECK-MINUS-257-OFFSET %s ; RUN: FileCheck --check-prefix=CHECK-ADD --check-prefix=CHECK-MINUS-257-OFFSET %s
; XFAIL ; XFAIL
; RUN: not --crash llc %t/h2.ll -o - 2>&1 | \ ; RUN: not --crash llc %t/h2.ll -o - 2>&1 | \
Context not available.
; Verify that we `mrs` from `SP_EL0` twice, rather than load from ; Verify that we `mrs` from `SP_EL0` twice, rather than load from
; __stack_chk_guard. ; __stack_chk_guard.
define dso_local void @foo(i64 %t) local_unnamed_addr #0 { define dso_local void @foo(i64 %t) local_unnamed_addr #0 {
; CHECK-LABEL: foo: ; CHECK-LABEL: foo: // @foo
; CHECK: // %bb.0: // %entry ; CHECK: .cfi_startproc
; CHECK-NEXT: stp x29, x30, [sp, #-16]! // 16-byte Folded Spill ; CHECK: // %bb.0: // %entry
; CHECK-NEXT: stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
; CHECK-NEXT: .cfi_def_cfa_offset 16 ; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: mov x29, sp ; CHECK-NEXT: mov x29, sp
; CHECK-NEXT: .cfi_def_cfa w29, 16 ; CHECK-NEXT: .cfi_def_cfa w29, 16
; CHECK-NEXT: .cfi_offset w30, -8 ; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16 ; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: .cfi_remember_state ; CHECK-NEXT: .cfi_remember_state
; CHECK-NEXT: sub sp, sp, #16 ; CHECK-NEXT: sub sp, sp, #16
; CHECK-NEXT: mrs x8, SP_EL0 ; CHECK-NEXT: mrs x8, SP_EL0
; CHECK-NEXT: lsl x9, x0, #2 ; CHECK-NEXT: lsl x9, x0, #2
; CHECK-NO-OFFSET: ldr x8, [x8] ; CHECK-NO-OFFSET: ldr x8, [x8]
; CHECK-POSITIVE-OFFSET: ldr x8, [x8, #8] ; CHECK-POSITIVE-OFFSET: ldr x8, [x8, #8]
; CHECK-NEGATIVE-OFFSET: ldur x8, [x8, #-8] ; CHECK-NEGATIVE-OFFSET: ldur x8, [x8, #-8]
; CHECK-NPOT-OFFSET: ldur x8, [x8, #1] ; CHECK-NPOT-OFFSET: ldur x8, [x8, #1]
; CHECK-NPOT-NEG-OFFSET: ldur x8, [x8, #-1] ; CHECK-NPOT-NEG-OFFSET: ldur x8, [x8, #-1]
; CHECK-257-OFFSET: add x8, x8, #257 ; CHECK-NEXT: add x9, x9, #15
; CHECK-MINUS-257-OFFSET: sub x8, x8, #257 ; CHECK-NEXT: stur x8, [x29, #-8]
; CHECK-NEXT: add x9, x9, #15 ; CHECK-NEXT mov x8, sp
; CHECK-NEXT: and x9, x9, #0xfffffffffffffff0 ; CHECK-NEXT and x9, x9, #0xfffffffffffffff0
; CHECK-257-OFFSET-NEXT: ldr x8, [x8] ; CHECK-NEXT sub x0, x8, x9
; CHECK-MINUS-257-OFFSET-NEXT: ldr x8, [x8] ; CHECK-NEXT mov sp, x0
; CHECK-NEXT: stur x8, [x29, #-8] ; CHECK-NEXT bl baz
; CHECK-NEXT: mov x8, sp ; CHECK-NEXT mrs x8, SP_EL0
; CHECK-NEXT: sub x0, x8, x9
; CHECK-NEXT: mov sp, x0
; CHECK-NEXT: bl baz
; CHECK-NEXT: mrs x8, SP_EL0
; CHECK-NO-OFFSET: ldr x8, [x8] ; CHECK-NO-OFFSET: ldr x8, [x8]
; CHECK-POSITIVE-OFFSET: ldr x8, [x8, #8] ; CHECK-POSITIVE-OFFSET: ldr x8, [x8, #8]
; CHECK-NEGATIVE-OFFSET: ldur x8, [x8, #-8] ; CHECK-NEGATIVE-OFFSET: ldur x8, [x8, #-8]
; CHECK-NPOT-OFFSET: ldur x8, [x8, #1] ; CHECK-NPOT-OFFSET: ldur x8, [x8, #1]
; CHECK-NPOT-NEG-OFFSET: ldur x8, [x8, #-1] ; CHECK-NPOT-NEG-OFFSET: ldur x8, [x8, #-1]
; CHECK-257-OFFSET: add x8, x8, #257 ; CHECK-NEXT: ldur x9, [x29, #-8]
; CHECK-257-OFFSET-NEXT: ldr x8, [x8] ; CHECK-NEXT: cmp x8, x9
; CHECK-MINUS-257-OFFSET: sub x8, x8, #257 ; CHECK-NEXT: b.ne .LBB0_2
; CHECK-MINUS-257-OFFSET-NEXT: ldr x8, [x8] ; CHECK-NEXT: // %bb.1: // %entry
; CHECK-NEXT: ldur x9, [x29, #-8] ; CHECK-NEXT: mov sp, x29
; CHECK-NEXT: cmp x8, x9 ; CHECK-NEXT: .cfi_def_cfa wsp, 16
; CHECK-NEXT: b.ne .LBB0_2 ; CHECK-NEXT: ldp x29, x30, [sp], #16 // 16-byte Folded Reload
; CHECK-NEXT: // %bb.1: // %entry ; CHECK-NEXT: .cfi_def_cfa_offset 0
; CHECK-NEXT: mov sp, x29 ; CHECK-NEXT: .cfi_restore w30
; CHECK-NEXT: .cfi_def_cfa wsp, 16 ; CHECK-NEXT: .cfi_restore w29
; CHECK-NEXT: ldp x29, x30, [sp], #16 // 16-byte Folded Reload ; CHECK-NEXT: ret
; CHECK-NEXT: .cfi_def_cfa_offset 0 ; CHECK-NEXT: .LBB0_2: // %entry
; CHECK-NEXT: .cfi_restore w30 ; CHECK-NEXT: .cfi_restore_state
; CHECK-NEXT: .cfi_restore w29 ; CHECK-NEXT: bl __stack_chk_fail
; CHECK-NEXT: ret ; CHECK-NEXT: .Lfunc_end0:
; CHECK-NEXT: .LBB0_2: // %entry ; CHECK-NEXT: .size foo, .Lfunc_end0-foo
; CHECK-NEXT: .cfi_restore_state ; CHECK-NEXT: .cfi_endproc
; CHECK-NEXT: bl __stack_chk_fail ; CHECK-NEXT: // -- End function
; CHECK-NOT: __stack_chk_guard ; CHECK-NEXT: .section ".note.GNU-stack","",@progbits
; CHECK-ADD: stp x29, x30, [sp, #-16]! // 16-byte Folded Spill
; CHECK-ADD-NEXT: .cfi_def_cfa_offset 16
; CHECK-ADD-NEXT: mov x29, sp
; CHECK-ADD-NEXT: .cfi_def_cfa w29, 16
; CHECK-ADD-NEXT: .cfi_offset w30, -8
; CHECK-ADD-NEXT: .cfi_offset w29, -16
; CHECK-ADD-NEXT: .cfi_remember_state
; CHECK-ADD-NEXT: sub sp, sp, #16
; CHECK-ADD-NEXT: mrs x8, SP_EL0
; CHECK-ADD-NEXT: lsl x9, x0, #2
; CHECK-MINUS-257-OFFSET: sub x8, x8, #257
; CHECK-257-OFFSET: add x8, x8, #257
; CHECK-ADD-NEXT: ldr x8, [x8]
; CHECK-ADD-NEXT: add x9, x9, #15
; CHECK-ADD-NEXT: and x9, x9, #0xfffffffffffffff0
; CHECK-ADD-NEXT: stur x8, [x29, #-8]
; CHECK-ADD-NEXT: mov x8, sp
; CHECK-ADD-NEXT: sub x0, x8, x9
; CHECK-ADD-NEXT: mov sp, x0
; CHECK-ADD-NEXT: bl baz
; CHECK-ADD-NEXT: mrs x8, SP_EL0
; CHECK-257-OFFSET: add x8, x8, #257
; CHECK-MINUS-257-OFFSET: sub x8, x8, #257
; CHECK-ADD-NEXT: ldr x8, [x8]
; CHECK-ADD-NEXT: ldur x9, [x29, #-8]
; CHECK-ADD-NEXT: cmp x8, x9
; CHECK-ADD-NEXT: b.ne .LBB0_2
; CHECK-ADD-NEXT: // %bb.1: // %entry
; CHECK-ADD-NEXT: mov sp, x29
; CHECK-ADD-NEXT: .cfi_def_cfa wsp, 16
; CHECK-ADD-NEXT: ldp x29, x30, [sp], #16 // 16-byte Folded Reload
; CHECK-ADD-NEXT: .cfi_def_cfa_offset 0
; CHECK-ADD-NEXT: .cfi_restore w30
; CHECK-ADD-NEXT: .cfi_restore w29
; CHECK-ADD-NEXT: ret
; CHECK-ADD-NEXT: .LBB0_2: // %entry
; CHECK-ADD-NEXT: .cfi_restore_state
; CHECK-ADD-NEXT: bl __stack_chk_fail
; CHECK-ADD-NEXT: .Lfunc_end0:
; CHECK-ADD-NEXT: .size foo, .Lfunc_end0-foo
; CHECK-ADD-NEXT: .cfi_endproc
; CHECK-ADD-NEXT: // -- End function
; CHECK-ADD-NEXT: .section ".note.GNU-stack","",@progbits
entry: entry:
%vla = alloca i32, i64 %t, align 4 %vla = alloca i32, i64 %t, align 4
call void @baz(ptr nonnull %vla) call void @baz(ptr nonnull %vla)
Context not available.

llvm/test/CodeGen/AArch64/statepoint-call-lowering.ll

llvm/test/CodeGen/AArch64/sve-abd.ll

llvm/test/CodeGen/AArch64/sve-alloca.ll

llvm/test/CodeGen/AArch64/sve-bitcast.ll

llvm/test/CodeGen/AArch64/sve-breakdown-scalable-vectortype.ll

llvm/test/CodeGen/AArch64/sve-calling-convention-mixed.ll

llvm/test/CodeGen/AArch64/sve-doublereduct.ll

llvm/test/CodeGen/AArch64/sve-expand-div.ll

llvm/test/CodeGen/AArch64/sve-extract-element.ll

llvm/test/CodeGen/AArch64/sve-extract-fixed-from-scalable-vector.ll

llvm/test/CodeGen/AArch64/sve-extract-fixed-vector.ll

llvm/test/CodeGen/AArch64/sve-extract-scalable-vector.ll

llvm/test/CodeGen/AArch64/sve-fcmp.ll

llvm/test/CodeGen/AArch64/sve-fcopysign.ll

llvm/test/CodeGen/AArch64/sve-fix-length-and-combine-512.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-addressing-modes.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-bit-counting.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-bitcast.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-bitselect.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-build-vector.ll

Context not available.
define void @build_vector_minus2_dec32_v4i64(ptr %a) #0 { define void @build_vector_minus2_dec32_v4i64(ptr %a) #0 {
; VBITS_GE_256-LABEL: build_vector_minus2_dec32_v4i64: ; VBITS_GE_256-LABEL: build_vector_minus2_dec32_v4i64:
; VBITS_GE_256: // %bb.0: ; VBITS_GE_256: // %bb.0:
; VBITS_GE_256-NEXT: mov x8, #-32
; VBITS_GE_256-NEXT: ptrue p0.d, vl4 ; VBITS_GE_256-NEXT: ptrue p0.d, vl4
; VBITS_GE_256-NEXT: mov x8, #-32 // =0xffffffffffffffe0
; VBITS_GE_256-NEXT: index z0.d, #-2, x8 ; VBITS_GE_256-NEXT: index z0.d, #-2, x8
; VBITS_GE_256-NEXT: st1d { z0.d }, p0, [x0] ; VBITS_GE_256-NEXT: st1d { z0.d }, p0, [x0]
; VBITS_GE_256-NEXT: ret ; VBITS_GE_256-NEXT: ret
Context not available.
; Constant but not a sequence. ; Constant but not a sequence.
define void @build_vector_no_stride_v4i64(ptr %a) #0 { define void @build_vector_no_stride_v4i64(ptr %a) #0 {
; VBITS_GE_256-LABEL: .LCPI4_0:
dmgreenUnsubmitted
Done
ReplyInline Actions

Keep this

dmgreen: Keep this
; VBITS_GE_256: .xword 0
; VBITS_GE_256-NEXT: .xword 4
; VBITS_GE_256-NEXT: .xword 1
; VBITS_GE_256-NEXT: .xword 8
; VBITS_GE_256-LABEL: build_vector_no_stride_v4i64: ; VBITS_GE_256-LABEL: build_vector_no_stride_v4i64:
; VBITS_GE_256: // %bb.0: ; VBITS_GE_256: // %bb.0:
; VBITS_GE_256-NEXT: ptrue p0.d, vl4
; VBITS_GE_256-NEXT: adrp x8, .LCPI4_0 ; VBITS_GE_256-NEXT: adrp x8, .LCPI4_0
; VBITS_GE_256-NEXT: add x8, x8, :lo12:.LCPI4_0 ; VBITS_GE_256-NEXT: add x8, x8, :lo12:.LCPI4_0
; VBITS_GE_256-NEXT: ptrue p0.d, vl4
; VBITS_GE_256-NEXT: ld1d { z0.d }, p0/z, [x8] ; VBITS_GE_256-NEXT: ld1d { z0.d }, p0/z, [x8]
; VBITS_GE_256-NEXT: st1d { z0.d }, p0, [x0] ; VBITS_GE_256-NEXT: st1d { z0.d }, p0, [x0]
; VBITS_GE_256-NEXT: ret ; VBITS_GE_256-NEXT: ret
Context not available.

llvm/test/CodeGen/AArch64/sve-fixed-length-concat.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-ext-loads.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-extract-subvector.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-extract-vector-elt.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-fcopysign.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-fp-arith.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-fp-compares.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-fp-extend-trunc.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-fp-fma.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-fp-minmax.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-fp-reduce.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-fp-rounding.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-fp-select.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-fp-to-int.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-fp-vselect.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-fp128.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-frame-offests-crash.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-insert-vector-elt.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-arith.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-compares.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-div.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-extends.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-log.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-minmax.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-mulh.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-reduce.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-rem.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-select.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-shifts.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-to-fp.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-int-vselect.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-limit-duplane.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-loads.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-log-reduce.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-mask-opt.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-masked-gather.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-masked-loads.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-masked-scatter.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-masked-stores.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-no-vscale-range.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-permute-rev.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-permute-zip-uzp-trn.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-ptest.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-reshuffle.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-rev.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-sdiv-pow2.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-shuffles.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-splat-vector.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-stores.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-subvector.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-trunc-stores.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-trunc.ll

llvm/test/CodeGen/AArch64/sve-fixed-length-vector-shuffle.ll

llvm/test/CodeGen/AArch64/sve-fold-vscale.ll

llvm/test/CodeGen/AArch64/sve-forward-st-to-ld.ll

llvm/test/CodeGen/AArch64/sve-fp-combine.ll

llvm/test/CodeGen/AArch64/sve-fp-int-min-max.ll

llvm/test/CodeGen/AArch64/sve-fp-reduce-fadda.ll

llvm/test/CodeGen/AArch64/sve-fpext-load.ll

llvm/test/CodeGen/AArch64/sve-fptosi-sat.ll

llvm/test/CodeGen/AArch64/sve-fptoui-sat.ll

llvm/test/CodeGen/AArch64/sve-fptrunc-store.ll

llvm/test/CodeGen/AArch64/sve-gather-scatter-addr-opts.ll

llvm/test/CodeGen/AArch64/sve-gather-scatter-dag-combine.ll

llvm/test/CodeGen/AArch64/sve-gep.ll

llvm/test/CodeGen/AArch64/sve-implicit-zero-filling.ll

llvm/test/CodeGen/AArch64/sve-insert-element.ll

llvm/test/CodeGen/AArch64/sve-insert-vector.ll

llvm/test/CodeGen/AArch64/sve-int-arith-imm.ll

llvm/test/CodeGen/AArch64/sve-int-arith.ll

llvm/test/CodeGen/AArch64/sve-int-reduce.ll

llvm/test/CodeGen/AArch64/sve-intrinsics-counting-elems-i32.ll

llvm/test/CodeGen/AArch64/sve-intrinsics-index.ll

llvm/test/CodeGen/AArch64/sve-intrinsics-int-arith-imm.ll

llvm/test/CodeGen/AArch64/sve-intrinsics-int-arith-merging.ll

llvm/test/CodeGen/AArch64/sve-intrinsics-perm-select.ll

llvm/test/CodeGen/AArch64/sve-intrinsics-stN-reg-imm-addr-mode.ll

llvm/test/CodeGen/AArch64/sve-ld-post-inc.ll

llvm/test/CodeGen/AArch64/sve-ld1-addressing-mode-reg-imm.ll

llvm/test/CodeGen/AArch64/sve-ld1r.ll

llvm/test/CodeGen/AArch64/sve-lsr-scaled-index-addressing-mode.ll

llvm/test/CodeGen/AArch64/sve-masked-gather-legalize.ll

llvm/test/CodeGen/AArch64/sve-masked-ldst-sext.ll

llvm/test/CodeGen/AArch64/sve-masked-ldst-zext.ll

llvm/test/CodeGen/AArch64/sve-masked-scatter-legalize.ll

llvm/test/CodeGen/AArch64/sve-masked-scatter.ll

llvm/test/CodeGen/AArch64/sve-pr62151.ll

llvm/test/CodeGen/AArch64/sve-pred-arith.ll

llvm/test/CodeGen/AArch64/sve-pred-selectop.ll

llvm/test/CodeGen/AArch64/sve-pred-selectop2.ll

llvm/test/CodeGen/AArch64/sve-pred-selectop3.ll

llvm/test/CodeGen/AArch64/sve-ptest-removal-sink.ll

llvm/test/CodeGen/AArch64/sve-redundant-store.ll

llvm/test/CodeGen/AArch64/sve-select.ll

llvm/test/CodeGen/AArch64/sve-sext-zext.ll

llvm/test/CodeGen/AArch64/sve-smulo-sdnode.ll

llvm/test/CodeGen/AArch64/sve-split-extract-elt.ll

llvm/test/CodeGen/AArch64/sve-split-fcvt.ll

llvm/test/CodeGen/AArch64/sve-split-fp-reduce.ll

llvm/test/CodeGen/AArch64/sve-split-insert-elt.ll

llvm/test/CodeGen/AArch64/sve-split-int-pred-reduce.ll

Context not available.
define i1 @andv_nxv64i1(<vscale x 64 x i1> %a) { define i1 @andv_nxv64i1(<vscale x 64 x i1> %a) {
; CHECK-LABEL: andv_nxv64i1: ; CHECK-LABEL: andv_nxv64i1:
; CHECK: // %bb.0: ; CHECK: // %bb.0:
; CHECK-NEXT: str x29, [sp, #-16]! // 8-byte Folded Spill
; CHECK-NEXT: addvl sp, sp, #-1
; CHECK-NEXT: str p4, [sp, #7, mul vl] // 2-byte Folded Spill
; CHECK-NEXT: .cfi_escape 0x0f, 0x0c, 0x8f, 0x00, 0x11, 0x10, 0x22, 0x11, 0x08, 0x92, 0x2e, 0x00, 0x1e, 0x22 // sp + 16 + 8 * VG
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: and p1.b, p1/z, p1.b, p3.b ; CHECK-NEXT: and p1.b, p1/z, p1.b, p3.b
; CHECK-NEXT: and p0.b, p0/z, p0.b, p2.b ; CHECK-NEXT: and p0.b, p0/z, p0.b, p2.b
; CHECK-NEXT: ptrue p4.b
; CHECK-NEXT: and p0.b, p0/z, p0.b, p1.b ; CHECK-NEXT: and p0.b, p0/z, p0.b, p1.b
; CHECK-NEXT: nots p0.b, p4/z, p0.b ; CHECK-NEXT: ptrue p1.b
; CHECK-NEXT: ldr p4, [sp, #7, mul vl] // 2-byte Folded Reload ; CHECK-NEXT: nots p0.b, p1/z, p0.b
dmgreenUnsubmitted
Done
ReplyInline Actions

This is nice :)

dmgreen: This is nice :)
; CHECK-NEXT: cset w0, eq ; CHECK-NEXT: cset w0, eq
; CHECK-NEXT: addvl sp, sp, #1
; CHECK-NEXT: ldr x29, [sp], #16 // 8-byte Folded Reload
; CHECK-NEXT: ret ; CHECK-NEXT: ret
%res = call i1 @llvm.vector.reduce.and.nxv64i1(<vscale x 64 x i1> %a) %res = call i1 @llvm.vector.reduce.and.nxv64i1(<vscale x 64 x i1> %a)
ret i1 %res ret i1 %res
Context not available.

llvm/test/CodeGen/AArch64/sve-split-load.ll

llvm/test/CodeGen/AArch64/sve-split-store.ll

llvm/test/CodeGen/AArch64/sve-srem-combine-loop.ll

llvm/test/CodeGen/AArch64/sve-st1-addressing-mode-reg-imm.ll

llvm/test/CodeGen/AArch64/sve-stepvector.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-and-combine.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-bit-counting.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-bitcast.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-bitselect.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-build-vector.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-concat.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-ext-loads.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-extract-subvector.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fcopysign.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-arith.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-compares.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-convert.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-extend-trunc.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-fma.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-minmax.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-reduce.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-rounding.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-select.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-to-int.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-fp-vselect.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-insert-vector-elt.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-arith.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-compares.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-div.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-extends.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-immediates.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-log.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-minmax.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-mulh.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-reduce.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-rem.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-select.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-shifts.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-to-fp.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-vselect.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-ld2-alloca.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-limit-duplane.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-log-reduce.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-masked-load.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-masked-store.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-optimize-ptrue.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-permute-rev.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-permute-zip-uzp-trn.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-ptest.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-reshuffle.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-rev.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-sdiv-pow2.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-shuffle.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-trunc-stores.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-trunc.ll

llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-vector-shuffle.ll

llvm/test/CodeGen/AArch64/sve-tailcall.ll

llvm/test/CodeGen/AArch64/sve-trunc.ll

llvm/test/CodeGen/AArch64/sve-umulo-sdnode.ll

llvm/test/CodeGen/AArch64/sve-vecreduce-fold.ll

llvm/test/CodeGen/AArch64/sve-vector-deinterleave.ll

llvm/test/CodeGen/AArch64/sve-vector-interleave.ll

llvm/test/CodeGen/AArch64/sve-vector-splat.ll

llvm/test/CodeGen/AArch64/sve-vscale-attr.ll

llvm/test/CodeGen/AArch64/sve-vselect-imm.ll

llvm/test/CodeGen/AArch64/sve2-fcopysign.ll

llvm/test/CodeGen/AArch64/sve2-fixed-length-fcopysign.ll

llvm/test/CodeGen/AArch64/sve2p1-intrinsics-selx2.ll

llvm/test/CodeGen/AArch64/sve2p1-intrinsics-selx4.ll

llvm/test/CodeGen/AArch64/sve2p1-intrinsics-stores.ll

llvm/test/CodeGen/AArch64/swift-return.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc -verify-machineinstrs -mtriple=aarch64-apple-ios -o - %s | FileCheck %s ; RUN: llc -verify-machineinstrs -mtriple=aarch64-apple-ios -o - %s | FileCheck %s
; RUN: llc -O0 -fast-isel -verify-machineinstrs -mtriple=aarch64-apple-ios -o - %s | FileCheck %s --check-prefix=CHECK-O0 ; RUN: llc -O0 -fast-isel -verify-machineinstrs -mtriple=aarch64-apple-ios -o - %s | FileCheck %s --check-prefix=CHECK-O0
; RUN: llc -verify-machineinstrs -mtriple=arm64_32-apple-ios -o - %s | FileCheck %s ; RUN: llc -verify-machineinstrs -mtriple=arm64_32-apple-ios -o - %s | FileCheck %s
; RUN: llc -O0 -fast-isel -verify-machineinstrs -mtriple=arm64_32-apple-ios -o - %s | FileCheck %s --check-prefix=CHECK-O0 ; RUN: llc -O0 -fast-isel -verify-machineinstrs -mtriple=arm64_32-apple-ios -o - %s | FileCheck %s --check-prefix=CHECK-O0
; CHECK-LABEL: test1
; CHECK: bl _gen
; CHECK: sxth [[TMP:w.*]], w0
; CHECK: add w0, [[TMP]], w1, sxtb
; CHECK-O0-LABEL: test1
; CHECK-O0: bl _gen
; CHECK-O0: sxth [[TMP:w.*]], w0
; CHECK-O0: add {{w[0-9]+}}, [[TMP]], w1, sxtb
define i16 @test1(i32) { define i16 @test1(i32) {
;
dmgreenUnsubmitted
Done
ReplyInline Actions

Manually update this one may be best.

dmgreen: Manually update this one may be best.
; CHECK-LABEL: test1:
; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: bl _gen
; CHECK-NEXT: sxth w8, w0
; CHECK-NEXT: add w0, w8, w1, sxtb
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-O0-LABEL: test1:
; CHECK-O0: ; %bb.0: ; %entry
; CHECK-O0-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-O0-NEXT: .cfi_def_cfa_offset 16
; CHECK-O0-NEXT: .cfi_offset w30, -8
; CHECK-O0-NEXT: .cfi_offset w29, -16
; CHECK-O0-NEXT: bl _gen
; CHECK-O0-NEXT: sxth w8, w0
; CHECK-O0-NEXT: add w0, w8, w1, sxtb
; CHECK-O0-NEXT: ; kill: def $w1 killed $w0
; CHECK-O0-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-O0-NEXT: ret
entry: entry:
%call = call swiftcc { i16, i8 } @gen(i32 %0) %call = call swiftcc { i16, i8 } @gen(i32 %0)
%v3 = extractvalue { i16, i8 } %call, 0 %v3 = extractvalue { i16, i8 } %call, 0
Context not available.
%conv = trunc i32 %add to i16 %conv = trunc i32 %add to i16
ret i16 %conv ret i16 %conv
} }
declare swiftcc { i16, i8 } @gen(i32) declare swiftcc { i16, i8 } @gen(i32)
; CHECK-LABEL: test2
; CHECK: bl _gen2
; CHECK: add [[TMP:x.*]], x0, x1
; CHECK: add [[TMP2:x.*]], x2, x3
; CHECK: add [[TMP]], [[TMP]], [[TMP2]]
; CHECK: add x0, [[TMP]], x4
; CHECK-O0-LABEL: test2
; CHECK-O0: bl _gen2
; CHECK-O0: add [[TMP:x.*]], x0, x1
; CHECK-O0: add [[TMP]], [[TMP]], x2
; CHECK-O0: add [[TMP]], [[TMP]], x3
; CHECK-O0: add x0, [[TMP]], x4
define i64 @test2(i64 %key) { define i64 @test2(i64 %key) {
;
; CHECK-LABEL: test2:
; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: sub sp, sp, #32
; CHECK-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-NEXT: .cfi_def_cfa_offset 32
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: str x0, [sp, #8]
; CHECK-NEXT: bl _gen2
; CHECK-NEXT: add x8, x0, x1
; CHECK-NEXT: add x9, x2, x3
; CHECK-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-NEXT: add x8, x8, x9
; CHECK-NEXT: add x0, x8, x4
; CHECK-NEXT: add sp, sp, #32
; CHECK-NEXT: ret
;
; CHECK-O0-LABEL: test2:
; CHECK-O0: ; %bb.0: ; %entry
; CHECK-O0-NEXT: sub sp, sp, #32
; CHECK-O0-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-O0-NEXT: .cfi_def_cfa_offset 32
; CHECK-O0-NEXT: .cfi_offset w30, -8
; CHECK-O0-NEXT: .cfi_offset w29, -16
; CHECK-O0-NEXT: str x0, [sp, #8]
; CHECK-O0-NEXT: ldr x0, [sp, #8]
; CHECK-O0-NEXT: bl _gen2
; CHECK-O0-NEXT: add x8, x0, x1
; CHECK-O0-NEXT: add x8, x8, x2
; CHECK-O0-NEXT: add x8, x8, x3
; CHECK-O0-NEXT: add x0, x8, x4
; CHECK-O0-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-O0-NEXT: add sp, sp, #32
; CHECK-O0-NEXT: ret
entry: entry:
%key.addr = alloca i64, align 4 %key.addr = alloca i64, align 4
store i64 %key, ptr %key.addr, align 4 store i64 %key, ptr %key.addr, align 4
%0 = load i64, ptr %key.addr, align 4 %0 = load i64, ptr %key.addr, align 4
%call = call swiftcc { i64, i64, i64, i64, i64 } @gen2(i64 %0) %call = call swiftcc { i64, i64, i64, i64, i64 } @gen2(i64 %0)
%v3 = extractvalue { i64, i64, i64, i64, i64 } %call, 0 %v3 = extractvalue { i64, i64, i64, i64, i64 } %call, 0
%v5 = extractvalue { i64, i64, i64, i64, i64 } %call, 1 %v5 = extractvalue { i64, i64, i64, i64, i64 } %call, 1
%v6 = extractvalue { i64, i64, i64, i64, i64 } %call, 2 %v6 = extractvalue { i64, i64, i64, i64, i64 } %call, 2
%v7 = extractvalue { i64, i64, i64, i64, i64 } %call, 3 %v7 = extractvalue { i64, i64, i64, i64, i64 } %call, 3
%v8 = extractvalue { i64, i64, i64, i64, i64 } %call, 4 %v8 = extractvalue { i64, i64, i64, i64, i64 } %call, 4
%add = add nsw i64 %v3, %v5 %add = add nsw i64 %v3, %v5
%add1 = add nsw i64 %add, %v6 %add1 = add nsw i64 %add, %v6
%add2 = add nsw i64 %add1, %v7 %add2 = add nsw i64 %add1, %v7
%add3 = add nsw i64 %add2, %v8 %add3 = add nsw i64 %add2, %v8
ret i64 %add3 ret i64 %add3
} }
; CHECK-LABEL: gen2:
; CHECK: mov x1, x0
; CHECK: mov x2, x0
; CHECK: mov x3, x0
; CHECK: mov x4, x0
; CHECK: ret
define swiftcc { i64, i64, i64, i64, i64 } @gen2(i64 %key) { define swiftcc { i64, i64, i64, i64, i64 } @gen2(i64 %key) {
;
; CHECK-LABEL: gen2:
; CHECK: ; %bb.0:
; CHECK-NEXT: mov x1, x0
; CHECK-NEXT: mov x2, x0
; CHECK-NEXT: mov x3, x0
; CHECK-NEXT: mov x4, x0
; CHECK-NEXT: ret
;
; CHECK-O0-LABEL: gen2:
; CHECK-O0: ; %bb.0:
; CHECK-O0-NEXT: mov x4, x0
; CHECK-O0-NEXT: mov x0, x4
; CHECK-O0-NEXT: mov x1, x4
; CHECK-O0-NEXT: mov x2, x4
; CHECK-O0-NEXT: mov x3, x4
; CHECK-O0-NEXT: ret
%Y = insertvalue { i64, i64, i64, i64, i64 } undef, i64 %key, 0 %Y = insertvalue { i64, i64, i64, i64, i64 } undef, i64 %key, 0
%Z = insertvalue { i64, i64, i64, i64, i64 } %Y, i64 %key, 1 %Z = insertvalue { i64, i64, i64, i64, i64 } %Y, i64 %key, 1
%Z2 = insertvalue { i64, i64, i64, i64, i64 } %Z, i64 %key, 2 %Z2 = insertvalue { i64, i64, i64, i64, i64 } %Z, i64 %key, 2
Context not available.
%Z4 = insertvalue { i64, i64, i64, i64, i64 } %Z3, i64 %key, 4 %Z4 = insertvalue { i64, i64, i64, i64, i64 } %Z3, i64 %key, 4
ret { i64, i64, i64, i64, i64 } %Z4 ret { i64, i64, i64, i64, i64 } %Z4
} }
; CHECK-LABEL: test3
; CHECK: bl _gen3
; CHECK: add [[TMP:w.*]], w0, w1
; CHECK: add [[TMP2:w.*]], w2, w3
; CHECK: add w0, [[TMP]], [[TMP2]]
; CHECK-O0-LABEL: test3
; CHECK-O0: bl _gen3
; CHECK-O0: add [[TMP:w.*]], w0, w1
; CHECK-O0: add [[TMP]], [[TMP]], w2
; CHECK-O0: add w0, [[TMP]], w3
define i32 @test3(i32) { define i32 @test3(i32) {
;
; CHECK-LABEL: test3:
; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: bl _gen3
; CHECK-NEXT: add w8, w0, w1
; CHECK-NEXT: add w9, w2, w3
; CHECK-NEXT: add w0, w8, w9
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-O0-LABEL: test3:
; CHECK-O0: ; %bb.0: ; %entry
; CHECK-O0-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-O0-NEXT: .cfi_def_cfa_offset 16
; CHECK-O0-NEXT: .cfi_offset w30, -8
; CHECK-O0-NEXT: .cfi_offset w29, -16
; CHECK-O0-NEXT: bl _gen3
; CHECK-O0-NEXT: add w8, w0, w1
; CHECK-O0-NEXT: add w8, w8, w2
; CHECK-O0-NEXT: add w0, w8, w3
; CHECK-O0-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-O0-NEXT: ret
entry: entry:
%call = call swiftcc { i32, i32, i32, i32 } @gen3(i32 %0) %call = call swiftcc { i32, i32, i32, i32 } @gen3(i32 %0)
%v3 = extractvalue { i32, i32, i32, i32 } %call, 0 %v3 = extractvalue { i32, i32, i32, i32 } %call, 0
%v5 = extractvalue { i32, i32, i32, i32 } %call, 1 %v5 = extractvalue { i32, i32, i32, i32 } %call, 1
%v6 = extractvalue { i32, i32, i32, i32 } %call, 2 %v6 = extractvalue { i32, i32, i32, i32 } %call, 2
%v7 = extractvalue { i32, i32, i32, i32 } %call, 3 %v7 = extractvalue { i32, i32, i32, i32 } %call, 3
%add = add nsw i32 %v3, %v5 %add = add nsw i32 %v3, %v5
%add1 = add nsw i32 %add, %v6 %add1 = add nsw i32 %add, %v6
%add2 = add nsw i32 %add1, %v7 %add2 = add nsw i32 %add1, %v7
ret i32 %add2 ret i32 %add2
} }
declare swiftcc { i32, i32, i32, i32 } @gen3(i32 %key) declare swiftcc { i32, i32, i32, i32 } @gen3(i32 %key)
; CHECK-LABEL: test4
; CHECK: bl _gen4
; CHECK: fadd s0, s0, s1
; CHECK: fadd s0, s0, s2
; CHECK: fadd s0, s0, s3
; CHECK-O0-LABEL: test4
; CHECK-O0: bl _gen4
; CHECK-O0: fadd s0, s0, s1
; CHECK-O0: fadd s0, s0, s2
; CHECK-O0: fadd s0, s0, s3
define float @test4(float) { define float @test4(float) {
;
; CHECK-LABEL: test4:
; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: bl _gen4
; CHECK-NEXT: fadd s0, s0, s1
; CHECK-NEXT: fadd s0, s0, s2
; CHECK-NEXT: fadd s0, s0, s3
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-O0-LABEL: test4:
; CHECK-O0: ; %bb.0: ; %entry
; CHECK-O0-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-O0-NEXT: .cfi_def_cfa_offset 16
; CHECK-O0-NEXT: .cfi_offset w30, -8
; CHECK-O0-NEXT: .cfi_offset w29, -16
; CHECK-O0-NEXT: bl _gen4
; CHECK-O0-NEXT: fadd s0, s0, s1
; CHECK-O0-NEXT: fadd s0, s0, s2
; CHECK-O0-NEXT: fadd s0, s0, s3
; CHECK-O0-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-O0-NEXT: ret
entry: entry:
%call = call swiftcc { float, float, float, float } @gen4(float %0) %call = call swiftcc { float, float, float, float } @gen4(float %0)
%v3 = extractvalue { float, float, float, float } %call, 0 %v3 = extractvalue { float, float, float, float } %call, 0
%v5 = extractvalue { float, float, float, float } %call, 1 %v5 = extractvalue { float, float, float, float } %call, 1
%v6 = extractvalue { float, float, float, float } %call, 2 %v6 = extractvalue { float, float, float, float } %call, 2
%v7 = extractvalue { float, float, float, float } %call, 3 %v7 = extractvalue { float, float, float, float } %call, 3
%add = fadd float %v3, %v5 %add = fadd float %v3, %v5
%add1 = fadd float %add, %v6 %add1 = fadd float %add, %v6
%add2 = fadd float %add1, %v7 %add2 = fadd float %add1, %v7
ret float %add2 ret float %add2
} }
declare swiftcc { float, float, float, float } @gen4(float %key) declare swiftcc { float, float, float, float } @gen4(float %key)
; CHECK-LABEL: test5
; CHECK: bl _gen5
; CHECK: fadd d0, d0, d1
; CHECK: fadd d0, d0, d2
; CHECK: fadd d0, d0, d3
; CHECK-O0-LABEL: test5
; CHECK-O0: bl _gen5
; CHECK-O0: fadd d0, d0, d1
; CHECK-O0: fadd d0, d0, d2
; CHECK-O0: fadd d0, d0, d3
define swiftcc double @test5(){ define swiftcc double @test5(){
;
; CHECK-LABEL: test5:
; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: bl _gen5
; CHECK-NEXT: fadd d0, d0, d1
; CHECK-NEXT: fadd d0, d0, d2
; CHECK-NEXT: fadd d0, d0, d3
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-O0-LABEL: test5:
; CHECK-O0: ; %bb.0: ; %entry
; CHECK-O0-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-O0-NEXT: .cfi_def_cfa_offset 16
; CHECK-O0-NEXT: .cfi_offset w30, -8
; CHECK-O0-NEXT: .cfi_offset w29, -16
; CHECK-O0-NEXT: bl _gen5
; CHECK-O0-NEXT: fadd d0, d0, d1
; CHECK-O0-NEXT: fadd d0, d0, d2
; CHECK-O0-NEXT: fadd d0, d0, d3
; CHECK-O0-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-O0-NEXT: ret
entry: entry:
%call = call swiftcc { double, double, double, double } @gen5() %call = call swiftcc { double, double, double, double } @gen5()
%v3 = extractvalue { double, double, double, double } %call, 0 %v3 = extractvalue { double, double, double, double } %call, 0
%v5 = extractvalue { double, double, double, double } %call, 1 %v5 = extractvalue { double, double, double, double } %call, 1
%v6 = extractvalue { double, double, double, double } %call, 2 %v6 = extractvalue { double, double, double, double } %call, 2
%v7 = extractvalue { double, double, double, double } %call, 3 %v7 = extractvalue { double, double, double, double } %call, 3
%add = fadd double %v3, %v5 %add = fadd double %v3, %v5
%add1 = fadd double %add, %v6 %add1 = fadd double %add, %v6
%add2 = fadd double %add1, %v7 %add2 = fadd double %add1, %v7
ret double %add2 ret double %add2
} }
declare swiftcc { double, double, double, double } @gen5() declare swiftcc { double, double, double, double } @gen5()
; CHECK-LABEL: test6
; CHECK: bl _gen6
; CHECK-DAG: fadd d0, d0, d1
; CHECK-DAG: fadd d0, d0, d2
; CHECK-DAG: fadd d0, d0, d3
; CHECK-DAG: add [[TMP:w.*]], w0, w1
; CHECK-DAG: add [[TMP2:w.*]], w2, w3
; CHECK-DAG: add w0, [[TMP]], [[TMP2]]
; CHECK-O0-LABEL: test6
; CHECK-O0: bl _gen6
; CHECK-O0-DAG: fadd d0, d0, d1
; CHECK-O0-DAG: fadd d0, d0, d2
; CHECK-O0-DAG: fadd d0, d0, d3
; CHECK-O0-DAG: add [[TMP:w.*]], w0, w1
; CHECK-O0-DAG: add [[TMP]], [[TMP]], w2
; CHECK-O0-DAG: add w0, [[TMP]], w3
define swiftcc { double, i32 } @test6() { define swiftcc { double, i32 } @test6() {
;
; CHECK-LABEL: test6:
; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: bl _gen6
; CHECK-NEXT: fadd d0, d0, d1
; CHECK-NEXT: add w8, w0, w1
; CHECK-NEXT: add w9, w2, w3
; CHECK-NEXT: add w0, w8, w9
; CHECK-NEXT: fadd d0, d0, d2
; CHECK-NEXT: fadd d0, d0, d3
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-O0-LABEL: test6:
; CHECK-O0: ; %bb.0: ; %entry
; CHECK-O0-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-O0-NEXT: .cfi_def_cfa_offset 16
; CHECK-O0-NEXT: .cfi_offset w30, -8
; CHECK-O0-NEXT: .cfi_offset w29, -16
; CHECK-O0-NEXT: bl _gen6
; CHECK-O0-NEXT: fadd d0, d0, d1
; CHECK-O0-NEXT: fadd d0, d0, d2
; CHECK-O0-NEXT: fadd d0, d0, d3
; CHECK-O0-NEXT: add w8, w0, w1
; CHECK-O0-NEXT: add w8, w8, w2
; CHECK-O0-NEXT: add w0, w8, w3
; CHECK-O0-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-O0-NEXT: ret
entry: entry:
%call = call swiftcc { double, double, double, double, i32, i32, i32, i32 } @gen6() %call = call swiftcc { double, double, double, double, i32, i32, i32, i32 } @gen6()
%v3 = extractvalue { double, double, double, double, i32, i32, i32, i32 } %call, 0 %v3 = extractvalue { double, double, double, double, i32, i32, i32, i32 } %call, 0
%v5 = extractvalue { double, double, double, double, i32, i32, i32, i32 } %call, 1 %v5 = extractvalue { double, double, double, double, i32, i32, i32, i32 } %call, 1
%v6 = extractvalue { double, double, double, double, i32, i32, i32, i32 } %call, 2 %v6 = extractvalue { double, double, double, double, i32, i32, i32, i32 } %call, 2
Context not available.
%v5.i = extractvalue { double, double, double, double, i32, i32, i32, i32 } %call, 5 %v5.i = extractvalue { double, double, double, double, i32, i32, i32, i32 } %call, 5
%v6.i = extractvalue { double, double, double, double, i32, i32, i32, i32 } %call, 6 %v6.i = extractvalue { double, double, double, double, i32, i32, i32, i32 } %call, 6
%v7.i = extractvalue { double, double, double, double, i32, i32, i32, i32 } %call, 7 %v7.i = extractvalue { double, double, double, double, i32, i32, i32, i32 } %call, 7
%add = fadd double %v3, %v5 %add = fadd double %v3, %v5
%add1 = fadd double %add, %v6 %add1 = fadd double %add, %v6
%add2 = fadd double %add1, %v7 %add2 = fadd double %add1, %v7
%add.i = add nsw i32 %v3.i, %v5.i %add.i = add nsw i32 %v3.i, %v5.i
%add1.i = add nsw i32 %add.i, %v6.i %add1.i = add nsw i32 %add.i, %v6.i
%add2.i = add nsw i32 %add1.i, %v7.i %add2.i = add nsw i32 %add1.i, %v7.i
%Y = insertvalue { double, i32 } undef, double %add2, 0 %Y = insertvalue { double, i32 } undef, double %add2, 0
%Z = insertvalue { double, i32 } %Y, i32 %add2.i, 1 %Z = insertvalue { double, i32 } %Y, i32 %add2.i, 1
ret { double, i32} %Z ret { double, i32} %Z
} }
declare swiftcc { double, double, double, double, i32, i32, i32, i32 } @gen6() declare swiftcc { double, double, double, double, i32, i32, i32, i32 } @gen6()
; CHECK-LABEL: _gen7
; CHECK-DAG: mov w1, w0
; CHECK-DAG: mov w2, w0
; CHECK-DAG: mov w3, w0
; CHECK: ret
; CHECK-O0-LABEL: _gen7
; CHECK-O0: mov w3, w0
; CHECK-O0: mov w0, w3
; CHECK-O0: mov w1, w3
; CHECK-O0: mov w2, w3
define swiftcc { i32, i32, i32, i32 } @gen7(i32 %key) { define swiftcc { i32, i32, i32, i32 } @gen7(i32 %key) {
;
; CHECK-LABEL: gen7:
; CHECK: ; %bb.0:
; CHECK-NEXT: mov w1, w0
; CHECK-NEXT: mov w2, w0
; CHECK-NEXT: mov w3, w0
; CHECK-NEXT: ret
;
; CHECK-O0-LABEL: gen7:
; CHECK-O0: ; %bb.0:
; CHECK-O0-NEXT: mov w3, w0
; CHECK-O0-NEXT: mov w0, w3
; CHECK-O0-NEXT: mov w1, w3
; CHECK-O0-NEXT: mov w2, w3
; CHECK-O0-NEXT: ret
%v0 = insertvalue { i32, i32, i32, i32 } undef, i32 %key, 0 %v0 = insertvalue { i32, i32, i32, i32 } undef, i32 %key, 0
%v1 = insertvalue { i32, i32, i32, i32 } %v0, i32 %key, 1 %v1 = insertvalue { i32, i32, i32, i32 } %v0, i32 %key, 1
%v2 = insertvalue { i32, i32, i32, i32 } %v1, i32 %key, 2 %v2 = insertvalue { i32, i32, i32, i32 } %v1, i32 %key, 2
%v3 = insertvalue { i32, i32, i32, i32 } %v2, i32 %key, 3 %v3 = insertvalue { i32, i32, i32, i32 } %v2, i32 %key, 3
ret { i32, i32, i32, i32 } %v3 ret { i32, i32, i32, i32 } %v3
} }
; CHECK-LABEL: _gen9
; CHECK: mov w1, w0
; CHECK: mov w2, w0
; CHECK: mov w3, w0
; CHECK: ret
; CHECK-O0-LABEL: _gen9
; CHECK-O0: mov w3, w0
; CHECK-O0: mov w0, w3
; CHECK-O0: mov w1, w3
; CHECK-O0: mov w2, w3
define swiftcc { i8, i8, i8, i8 } @gen9(i8 %key) { define swiftcc { i8, i8, i8, i8 } @gen9(i8 %key) {
;
; CHECK-LABEL: gen9:
; CHECK: ; %bb.0:
; CHECK-NEXT: mov w1, w0
; CHECK-NEXT: mov w2, w0
; CHECK-NEXT: mov w3, w0
; CHECK-NEXT: ret
;
; CHECK-O0-LABEL: gen9:
; CHECK-O0: ; %bb.0:
; CHECK-O0-NEXT: mov w3, w0
; CHECK-O0-NEXT: mov w0, w3
; CHECK-O0-NEXT: mov w1, w3
; CHECK-O0-NEXT: mov w2, w3
; CHECK-O0-NEXT: ret
%v0 = insertvalue { i8, i8, i8, i8 } undef, i8 %key, 0 %v0 = insertvalue { i8, i8, i8, i8 } undef, i8 %key, 0
%v1 = insertvalue { i8, i8, i8, i8 } %v0, i8 %key, 1 %v1 = insertvalue { i8, i8, i8, i8 } %v0, i8 %key, 1
%v2 = insertvalue { i8, i8, i8, i8 } %v1, i8 %key, 2 %v2 = insertvalue { i8, i8, i8, i8 } %v1, i8 %key, 2
%v3 = insertvalue { i8, i8, i8, i8 } %v2, i8 %key, 3 %v3 = insertvalue { i8, i8, i8, i8 } %v2, i8 %key, 3
ret { i8, i8, i8, i8 } %v3 ret { i8, i8, i8, i8 } %v3
} }
; CHECK-LABEL: _gen10
; CHECK: fmov d1, d0
; CHECK: fmov d2, d0
; CHECK: fmov d3, d0
; CHECK: mov w1, w0
; CHECK: mov w2, w0
; CHECK: mov w3, w0
; CHECK: ret
define swiftcc { double, double, double, double, i32, i32, i32, i32 } @gen10(double %keyd, i32 %keyi) { define swiftcc { double, double, double, double, i32, i32, i32, i32 } @gen10(double %keyd, i32 %keyi) {
;
; CHECK-LABEL: gen10:
; CHECK: ; %bb.0:
; CHECK-NEXT: fmov d1, d0
; CHECK-NEXT: fmov d2, d0
; CHECK-NEXT: mov w1, w0
; CHECK-NEXT: fmov d3, d0
; CHECK-NEXT: mov w2, w0
; CHECK-NEXT: mov w3, w0
; CHECK-NEXT: ret
;
; CHECK-O0-LABEL: gen10:
; CHECK-O0: ; %bb.0:
; CHECK-O0-NEXT: mov w3, w0
; CHECK-O0-NEXT: fmov d3, d0
; CHECK-O0-NEXT: fmov d0, d3
; CHECK-O0-NEXT: fmov d1, d3
; CHECK-O0-NEXT: fmov d2, d3
; CHECK-O0-NEXT: mov w0, w3
; CHECK-O0-NEXT: mov w1, w3
; CHECK-O0-NEXT: mov w2, w3
; CHECK-O0-NEXT: ret
%v0 = insertvalue { double, double, double, double, i32, i32, i32, i32 } undef, double %keyd, 0 %v0 = insertvalue { double, double, double, double, i32, i32, i32, i32 } undef, double %keyd, 0
%v1 = insertvalue { double, double, double, double, i32, i32, i32, i32 } %v0, double %keyd, 1 %v1 = insertvalue { double, double, double, double, i32, i32, i32, i32 } %v0, double %keyd, 1
%v2 = insertvalue { double, double, double, double, i32, i32, i32, i32 } %v1, double %keyd, 2 %v2 = insertvalue { double, double, double, double, i32, i32, i32, i32 } %v1, double %keyd, 2
Context not available.
%v7 = insertvalue { double, double, double, double, i32, i32, i32, i32 } %v6, i32 %keyi, 7 %v7 = insertvalue { double, double, double, double, i32, i32, i32, i32 } %v6, i32 %keyi, 7
ret { double, double, double, double, i32, i32, i32, i32 } %v7 ret { double, double, double, double, i32, i32, i32, i32 } %v7
} }
; CHECK-LABEL: _test11
; CHECK: bl _gen11
; CHECK: fadd.4s v0, v0, v1
; CHECK: fadd.4s v0, v0, v2
; CHECK: fadd.4s v0, v0, v3
define swiftcc <4 x float> @test11() { define swiftcc <4 x float> @test11() {
;
; CHECK-LABEL: test11:
; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: bl _gen11
; CHECK-NEXT: fadd.4s v0, v0, v1
; CHECK-NEXT: fadd.4s v0, v0, v2
; CHECK-NEXT: fadd.4s v0, v0, v3
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-O0-LABEL: test11:
; CHECK-O0: ; %bb.0: ; %entry
; CHECK-O0-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-O0-NEXT: .cfi_def_cfa_offset 16
; CHECK-O0-NEXT: .cfi_offset w30, -8
; CHECK-O0-NEXT: .cfi_offset w29, -16
; CHECK-O0-NEXT: bl _gen11
; CHECK-O0-NEXT: fadd.4s v0, v0, v1
; CHECK-O0-NEXT: fadd.4s v0, v0, v2
; CHECK-O0-NEXT: fadd.4s v0, v0, v3
; CHECK-O0-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-O0-NEXT: ret
entry: entry:
%call = call swiftcc { <4 x float>, <4 x float>, <4 x float>, <4 x float> } @gen11() %call = call swiftcc { <4 x float>, <4 x float>, <4 x float>, <4 x float> } @gen11()
%v3 = extractvalue { <4 x float>, <4 x float>, <4 x float>, <4 x float> } %call, 0 %v3 = extractvalue { <4 x float>, <4 x float>, <4 x float>, <4 x float> } %call, 0
%v5 = extractvalue { <4 x float>, <4 x float>, <4 x float>, <4 x float> } %call, 1 %v5 = extractvalue { <4 x float>, <4 x float>, <4 x float>, <4 x float> } %call, 1
%v6 = extractvalue { <4 x float>, <4 x float>, <4 x float>, <4 x float> } %call, 2 %v6 = extractvalue { <4 x float>, <4 x float>, <4 x float>, <4 x float> } %call, 2
%v7 = extractvalue { <4 x float>, <4 x float>, <4 x float>, <4 x float> } %call, 3 %v7 = extractvalue { <4 x float>, <4 x float>, <4 x float>, <4 x float> } %call, 3
%add = fadd <4 x float> %v3, %v5 %add = fadd <4 x float> %v3, %v5
%add1 = fadd <4 x float> %add, %v6 %add1 = fadd <4 x float> %add, %v6
%add2 = fadd <4 x float> %add1, %v7 %add2 = fadd <4 x float> %add1, %v7
ret <4 x float> %add2 ret <4 x float> %add2
} }
declare swiftcc { <4 x float>, <4 x float>, <4 x float>, <4 x float> } @gen11() declare swiftcc { <4 x float>, <4 x float>, <4 x float>, <4 x float> } @gen11()
; CHECK-LABEL: _test12
; CHECK: fadd.4s v0, v0, v1
; CHECK: fmov s1, s3
; CHECK: fadd.4s v0, v0, v2
define swiftcc { <4 x float>, float } @test12() #0 { define swiftcc { <4 x float>, float } @test12() #0 {
;
; CHECK-LABEL: test12:
; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: stp x29, x30, [sp, #-16]! ; 16-byte Folded Spill
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: .cfi_offset w30, -8
; CHECK-NEXT: .cfi_offset w29, -16
; CHECK-NEXT: bl _gen12
; CHECK-NEXT: fadd.4s v0, v0, v1
; CHECK-NEXT: fmov s1, s3
; CHECK-NEXT: fadd.4s v0, v0, v2
; CHECK-NEXT: ldp x29, x30, [sp], #16 ; 16-byte Folded Reload
; CHECK-NEXT: ret
;
; CHECK-O0-LABEL: test12:
; CHECK-O0: ; %bb.0: ; %entry
; CHECK-O0-NEXT: sub sp, sp, #32
; CHECK-O0-NEXT: stp x29, x30, [sp, #16] ; 16-byte Folded Spill
; CHECK-O0-NEXT: .cfi_def_cfa_offset 32
; CHECK-O0-NEXT: .cfi_offset w30, -8
; CHECK-O0-NEXT: .cfi_offset w29, -16
; CHECK-O0-NEXT: bl _gen12
; CHECK-O0-NEXT: str q1, [sp] ; 16-byte Folded Spill
; CHECK-O0-NEXT: fmov s1, s3
; CHECK-O0-NEXT: ldr q3, [sp] ; 16-byte Folded Reload
; CHECK-O0-NEXT: fadd.4s v0, v0, v3
; CHECK-O0-NEXT: fadd.4s v0, v0, v2
; CHECK-O0-NEXT: ldp x29, x30, [sp, #16] ; 16-byte Folded Reload
; CHECK-O0-NEXT: add sp, sp, #32
; CHECK-O0-NEXT: ret
entry: entry:
%call = call swiftcc { <4 x float>, <4 x float>, <4 x float>, float } @gen12() %call = call swiftcc { <4 x float>, <4 x float>, <4 x float>, float } @gen12()
%v3 = extractvalue { <4 x float>, <4 x float>, <4 x float>, float } %call, 0 %v3 = extractvalue { <4 x float>, <4 x float>, <4 x float>, float } %call, 0
%v5 = extractvalue { <4 x float>, <4 x float>, <4 x float>, float } %call, 1 %v5 = extractvalue { <4 x float>, <4 x float>, <4 x float>, float } %call, 1
%v6 = extractvalue { <4 x float>, <4 x float>, <4 x float>, float } %call, 2 %v6 = extractvalue { <4 x float>, <4 x float>, <4 x float>, float } %call, 2
%v8 = extractvalue { <4 x float>, <4 x float>, <4 x float>, float } %call, 3 %v8 = extractvalue { <4 x float>, <4 x float>, <4 x float>, float } %call, 3
%add = fadd <4 x float> %v3, %v5 %add = fadd <4 x float> %v3, %v5
%add1 = fadd <4 x float> %add, %v6 %add1 = fadd <4 x float> %add, %v6
%res.0 = insertvalue { <4 x float>, float } undef, <4 x float> %add1, 0 %res.0 = insertvalue { <4 x float>, float } undef, <4 x float> %add1, 0
%res = insertvalue { <4 x float>, float } %res.0, float %v8, 1 %res = insertvalue { <4 x float>, float } %res.0, float %v8, 1
ret { <4 x float>, float } %res ret { <4 x float>, float } %res
} }
declare swiftcc { <4 x float>, <4 x float>, <4 x float>, float } @gen12() declare swiftcc { <4 x float>, <4 x float>, <4 x float>, float } @gen12()
Context not available.

llvm/test/CodeGen/AArch64/swifterror.ll

llvm/test/CodeGen/AArch64/tbl-loops.ll

llvm/test/CodeGen/AArch64/trunc-to-tbl.ll

llvm/test/CodeGen/AArch64/typepromotion-overflow.ll

llvm/test/CodeGen/AArch64/typepromotion-phisret.ll

llvm/test/CodeGen/AArch64/typepromotion-signed.ll

llvm/test/CodeGen/AArch64/uadd_sat.ll

llvm/test/CodeGen/AArch64/uadd_sat_plus.ll

llvm/test/CodeGen/AArch64/uadd_sat_vec.ll

llvm/test/CodeGen/AArch64/umulo-128-legalisation-lowering.ll

llvm/test/CodeGen/AArch64/unfold-masked-merge-scalar-constmask-innerouter.ll

llvm/test/CodeGen/AArch64/unfold-masked-merge-scalar-constmask-interleavedbits.ll

llvm/test/CodeGen/AArch64/unfold-masked-merge-scalar-constmask-interleavedbytehalves.ll

llvm/test/CodeGen/AArch64/unfold-masked-merge-scalar-constmask-lowhigh.ll

llvm/test/CodeGen/AArch64/unfold-masked-merge-scalar-variablemask.ll

llvm/test/CodeGen/AArch64/unfold-masked-merge-vector-variablemask.ll

llvm/test/CodeGen/AArch64/urem-lkk.ll

llvm/test/CodeGen/AArch64/urem-seteq-illegal-types.ll

llvm/test/CodeGen/AArch64/urem-seteq-vec-nonsplat.ll

llvm/test/CodeGen/AArch64/urem-seteq-vec-nonzero.ll

llvm/test/CodeGen/AArch64/urem-seteq-vec-splat.ll

llvm/test/CodeGen/AArch64/urem-seteq-vec-tautological.ll

llvm/test/CodeGen/AArch64/urem-vector-lkk.ll

llvm/test/CodeGen/AArch64/usub_sat_vec.ll

llvm/test/CodeGen/AArch64/vcvt-oversize.ll

llvm/test/CodeGen/AArch64/vec-combine-compare-to-bitmask.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc -mtriple=aarch64-apple-darwin -mattr=+neon -verify-machineinstrs < %s | FileCheck %s ; RUN: llc -mtriple=aarch64-apple-darwin -mattr=+neon -verify-machineinstrs < %s | FileCheck %s
; Basic tests from input vector to bitmask ; Basic tests from input vector to bitmask
Context not available.
define i16 @convert_to_bitmask16(<16 x i8> %vec) { define i16 @convert_to_bitmask16(<16 x i8> %vec) {
; Bits used in mask ; Bits used in mask
; CHECK-LABEL: lCPI0_0 ; CHECK-LABEL: convert_to_bitmask16:
dmgreenUnsubmitted
Done
ReplyInline Actions

These need to keep checking the constant pools

dmgreen: These need to keep checking the constant pools
; CHECK-NEXT: .byte 1 ; CHECK: ; %bb.0:
; CHECK-NEXT: .byte 2 ; CHECK-NEXT: Lloh0:
; CHECK-NEXT: .byte 4 ; CHECK-NEXT: adrp x8, lCPI0_0@PAGE
; CHECK-NEXT: .byte 8 ; CHECK-NEXT: cmeq.16b v0, v0, #0
; CHECK-NEXT: .byte 16 ; CHECK-NEXT: Lloh1:
; CHECK-NEXT: .byte 32 ; CHECK-NEXT: ldr q1, [x8, lCPI0_0@PAGEOFF]
; CHECK-NEXT: .byte 64 ; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: .byte 128 ; CHECK-NEXT: ext.16b v1, v0, v0, #8
; CHECK-NEXT: .byte 1 ; CHECK-NEXT: addv.8b b0, v0
; CHECK-NEXT: .byte 2 ; CHECK-NEXT: addv.8b b1, v1
; CHECK-NEXT: .byte 4 ; CHECK-NEXT: fmov w8, s0
; CHECK-NEXT: .byte 8 ; CHECK-NEXT: fmov w9, s1
; CHECK-NEXT: .byte 16 ; CHECK-NEXT: orr w0, w8, w9, lsl #8
; CHECK-NEXT: .byte 32 ; CHECK-NEXT: ret
; CHECK-NEXT: .byte 64 ; CHECK-NEXT: .loh AdrpLdr Lloh0, Lloh1
; CHECK-NEXT: .byte 128
; Actual conversion ; Actual conversion
; CHECK-LABEL: convert_to_bitmask16
; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh0:
; CHECK-NEXT: adrp x8, lCPI0_0@PAGE
; CHECK-NEXT: cmeq.16b v0, v0, #0
; CHECK-NEXT: Lloh1:
; CHECK-NEXT: ldr q1, [x8, lCPI0_0@PAGEOFF]
; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: ext.16b v1, v0, v0, #8
; CHECK-NEXT: addv.8b b0, v0
; CHECK-NEXT: addv.8b b1, v1
; CHECK-NEXT: fmov w9, s0
; CHECK-NEXT: fmov w8, s1
; CHECK-NEXT: orr w0, w9, w8, lsl #8
; CHECK-NEXT: ret
%cmp_result = icmp ne <16 x i8> %vec, zeroinitializer %cmp_result = icmp ne <16 x i8> %vec, zeroinitializer
%bitmask = bitcast <16 x i1> %cmp_result to i16 %bitmask = bitcast <16 x i1> %cmp_result to i16
Context not available.
} }
define i16 @convert_to_bitmask8(<8 x i16> %vec) { define i16 @convert_to_bitmask8(<8 x i16> %vec) {
; CHECK-LABEL: lCPI1_0: ; CHECK-LABEL: convert_to_bitmask8:
; CHECK-NEXT: .short 1
; CHECK-NEXT: .short 2
; CHECK-NEXT: .short 4
; CHECK-NEXT: .short 8
; CHECK-NEXT: .short 16
; CHECK-NEXT: .short 32
; CHECK-NEXT: .short 64
; CHECK-NEXT: .short 128
; CHECK-LABEL: convert_to_bitmask8
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh2: ; CHECK-NEXT: Lloh2:
; CHECK-NEXT: adrp x8, lCPI1_0@PAGE ; CHECK-NEXT: adrp x8, lCPI1_0@PAGE
; CHECK-NEXT: cmeq.8h v0, v0, #0 ; CHECK-NEXT: cmeq.8h v0, v0, #0
; CHECK-NEXT: Lloh3: ; CHECK-NEXT: Lloh3:
; CHECK-NEXT: ldr q1, [x8, lCPI1_0@PAGEOFF] ; CHECK-NEXT: ldr q1, [x8, lCPI1_0@PAGEOFF]
; CHECK-NEXT: bic.16b v0, v1, v0 ; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: addv.8h h0, v0 ; CHECK-NEXT: addv.8h h0, v0
; CHECK-NEXT: fmov w8, s0 ; CHECK-NEXT: fmov w8, s0
; CHECK-NEXT: and w0, w8, #0xff ; CHECK-NEXT: and w0, w8, #0xff
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh2, Lloh3
%cmp_result = icmp ne <8 x i16> %vec, zeroinitializer %cmp_result = icmp ne <8 x i16> %vec, zeroinitializer
%bitmask = bitcast <8 x i1> %cmp_result to i8 %bitmask = bitcast <8 x i1> %cmp_result to i8
Context not available.
} }
define i4 @convert_to_bitmask4(<4 x i32> %vec) { define i4 @convert_to_bitmask4(<4 x i32> %vec) {
; CHECK-LABEL: lCPI2_0: ; CHECK-LABEL: convert_to_bitmask4:
; CHECK-NEXT: .long 1
; CHECK-NEXT: .long 2
; CHECK-NEXT: .long 4
; CHECK-NEXT: .long 8
; CHECK-LABEL: convert_to_bitmask4
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh4: ; CHECK-NEXT: Lloh4:
; CHECK-NEXT: adrp x8, lCPI2_0@PAGE ; CHECK-NEXT: adrp x8, lCPI2_0@PAGE
; CHECK-NEXT: cmeq.4s v0, v0, #0 ; CHECK-NEXT: cmeq.4s v0, v0, #0
; CHECK-NEXT: Lloh5: ; CHECK-NEXT: Lloh5:
; CHECK-NEXT: ldr q1, [x8, lCPI2_0@PAGEOFF] ; CHECK-NEXT: ldr q1, [x8, lCPI2_0@PAGEOFF]
; CHECK-NEXT: bic.16b v0, v1, v0 ; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: addv.4s s0, v0 ; CHECK-NEXT: addv.4s s0, v0
; CHECK-NEXT: fmov w0, s0 ; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh4, Lloh5
%cmp_result = icmp ne <4 x i32> %vec, zeroinitializer %cmp_result = icmp ne <4 x i32> %vec, zeroinitializer
%bitmask = bitcast <4 x i1> %cmp_result to i4 %bitmask = bitcast <4 x i1> %cmp_result to i4
Context not available.
} }
define i8 @convert_to_bitmask2(<2 x i64> %vec) { define i8 @convert_to_bitmask2(<2 x i64> %vec) {
; CHECK-LABEL: lCPI3_0: ; CHECK-LABEL: convert_to_bitmask2:
; CHECK-NEXT: .quad 1
; CHECK-NEXT: .quad 2
; CHECK-LABEL: convert_to_bitmask2
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh6: ; CHECK-NEXT: Lloh6:
; CHECK-NEXT: adrp x8, lCPI3_0@PAGE ; CHECK-NEXT: adrp x8, lCPI3_0@PAGE
; CHECK-NEXT: cmeq.2d v0, v0, #0 ; CHECK-NEXT: cmeq.2d v0, v0, #0
; CHECK-NEXT: Lloh7: ; CHECK-NEXT: Lloh7:
; CHECK-NEXT: ldr q1, [x8, lCPI3_0@PAGEOFF] ; CHECK-NEXT: ldr q1, [x8, lCPI3_0@PAGEOFF]
; CHECK-NEXT: bic.16b v0, v1, v0 ; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: addp.2d d0, v0 ; CHECK-NEXT: addp.2d d0, v0
; CHECK-NEXT: fmov x8, d0 ; CHECK-NEXT: fmov x8, d0
; CHECK-NEXT: and w0, w8, #0x3 ; CHECK-NEXT: and w0, w8, #0x3
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh6, Lloh7
%cmp_result = icmp ne <2 x i64> %vec, zeroinitializer %cmp_result = icmp ne <2 x i64> %vec, zeroinitializer
%bitmask = bitcast <2 x i1> %cmp_result to i2 %bitmask = bitcast <2 x i1> %cmp_result to i2
Context not available.
; Clang's __builtin_convertvector adds an undef vector concat for vectors with <8 elements. ; Clang's __builtin_convertvector adds an undef vector concat for vectors with <8 elements.
define i8 @clang_builtins_undef_concat_convert_to_bitmask4(<4 x i32> %vec) { define i8 @clang_builtins_undef_concat_convert_to_bitmask4(<4 x i32> %vec) {
; CHECK-LABEL: lCPI4_0: ; CHECK-LABEL: clang_builtins_undef_concat_convert_to_bitmask4:
; CHECK-NEXT: .long 1
; CHECK-NEXT: .long 2
; CHECK-NEXT: .long 4
; CHECK-NEXT: .long 8
; CHECK-LABEL: clang_builtins_undef_concat_convert_to_bitmask4
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh8: ; CHECK-NEXT: Lloh8:
; CHECK-NEXT: adrp x8, lCPI4_0@PAGE ; CHECK-NEXT: adrp x8, lCPI4_0@PAGE
; CHECK-NEXT: cmeq.4s v0, v0, #0 ; CHECK-NEXT: cmeq.4s v0, v0, #0
; CHECK-NEXT: Lloh9: ; CHECK-NEXT: Lloh9:
; CHECK-NEXT: ldr q1, [x8, lCPI4_0@PAGEOFF] ; CHECK-NEXT: ldr q1, [x8, lCPI4_0@PAGEOFF]
; CHECK-NEXT: bic.16b v0, v1, v0 ; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: addv.4s s0, v0 ; CHECK-NEXT: addv.4s s0, v0
; CHECK-NEXT: fmov w0, s0 ; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh8, Lloh9
%cmp_result = icmp ne <4 x i32> %vec, zeroinitializer %cmp_result = icmp ne <4 x i32> %vec, zeroinitializer
%vector_pad = shufflevector <4 x i1> %cmp_result, <4 x i1> poison, <8 x i32> <i32 0, i32 1, i32 2, i32 3, i32 undef, i32 undef, i32 undef, i32 undef> %vector_pad = shufflevector <4 x i1> %cmp_result, <4 x i1> poison, <8 x i32> <i32 0, i32 1, i32 2, i32 3, i32 undef, i32 undef, i32 undef, i32 undef>
Context not available.
define i4 @convert_to_bitmask_no_compare(<4 x i32> %vec1, <4 x i32> %vec2) { define i4 @convert_to_bitmask_no_compare(<4 x i32> %vec1, <4 x i32> %vec2) {
; CHECK-LABEL: lCPI5_0: ; CHECK-LABEL: convert_to_bitmask_no_compare:
; CHECK-NEXT: .long 1
; CHECK-NEXT: .long 2
; CHECK-NEXT: .long 4
; CHECK-NEXT: .long 8
; CHECK-LABEL: convert_to_bitmask_no_compare
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh10: ; CHECK-NEXT: and.16b v0, v0, v1
; CHECK-NEXT: adrp x8, lCPI5_0@PAGE ; CHECK-NEXT: Lloh10:
; CHECK-NEXT: and.16b v0, v0, v1 ; CHECK-NEXT: adrp x8, lCPI5_0@PAGE
; CHECK-NEXT: shl.4s v0, v0, #31 ; CHECK-NEXT: Lloh11:
; CHECK-NEXT: Lloh11: ; CHECK-NEXT: ldr q1, [x8, lCPI5_0@PAGEOFF]
; CHECK-NEXT: ldr q1, [x8, lCPI5_0@PAGEOFF] ; CHECK-NEXT: shl.4s v0, v0, #31
; CHECK-NEXT: cmlt.4s v0, v0, #0 ; CHECK-NEXT: cmlt.4s v0, v0, #0
; CHECK-NEXT: and.16b v0, v0, v1 ; CHECK-NEXT: and.16b v0, v0, v1
; CHECK-NEXT: addv.4s s0, v0 ; CHECK-NEXT: addv.4s s0, v0
; CHECK-NEXT: fmov w0, s0 ; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh10, Lloh11
%cmp = and <4 x i32> %vec1, %vec2 %cmp = and <4 x i32> %vec1, %vec2
%trunc = trunc <4 x i32> %cmp to <4 x i1> %trunc = trunc <4 x i32> %cmp to <4 x i1>
Context not available.
} }
define i4 @convert_to_bitmask_with_compare_chain(<4 x i32> %vec1, <4 x i32> %vec2) { define i4 @convert_to_bitmask_with_compare_chain(<4 x i32> %vec1, <4 x i32> %vec2) {
; CHECK-LABEL: lCPI6_0: ; CHECK-LABEL: convert_to_bitmask_with_compare_chain:
; CHECK-NEXT: .long 1
; CHECK-NEXT: .long 2
; CHECK-NEXT: .long 4
; CHECK-NEXT: .long 8
; CHECK-LABEL: convert_to_bitmask_with_compare_chain
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh12: ; CHECK-NEXT: cmeq.4s v2, v0, #0
; CHECK-NEXT: adrp x8, lCPI6_0@PAGE ; CHECK-NEXT: cmeq.4s v0, v0, v1
; CHECK-NEXT: cmeq.4s v2, v0, #0 ; CHECK-NEXT: Lloh12:
; CHECK-NEXT: cmeq.4s v0, v0, v1 ; CHECK-NEXT: adrp x8, lCPI6_0@PAGE
; CHECK-NEXT: Lloh13: ; CHECK-NEXT: Lloh13:
; CHECK-NEXT: ldr q1, [x8, lCPI6_0@PAGEOFF] ; CHECK-NEXT: ldr q1, [x8, lCPI6_0@PAGEOFF]
; CHECK-NEXT: bic.16b v0, v0, v2 ; CHECK-NEXT: bic.16b v0, v0, v2
; CHECK-NEXT: and.16b v0, v0, v1 ; CHECK-NEXT: and.16b v0, v0, v1
; CHECK-NEXT: addv.4s s0, v0 ; CHECK-NEXT: addv.4s s0, v0
; CHECK-NEXT: fmov w0, s0 ; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh12, Lloh13
%cmp1 = icmp ne <4 x i32> %vec1, zeroinitializer %cmp1 = icmp ne <4 x i32> %vec1, zeroinitializer
%cmp2 = icmp eq <4 x i32> %vec1, %vec2 %cmp2 = icmp eq <4 x i32> %vec1, %vec2
Context not available.
} }
define i4 @convert_to_bitmask_with_trunc_in_chain(<4 x i32> %vec1, <4 x i32> %vec2) { define i4 @convert_to_bitmask_with_trunc_in_chain(<4 x i32> %vec1, <4 x i32> %vec2) {
; CHECK-LABEL: lCPI7_0: ; CHECK-LABEL: convert_to_bitmask_with_trunc_in_chain:
; CHECK-NEXT: .long 1
; CHECK-NEXT: .long 2
; CHECK-NEXT: .long 4
; CHECK-NEXT: .long 8
; CHECK-LABEL: convert_to_bitmask_with_trunc_in_chain
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: cmeq.4s v0, v0, #0 ; CHECK-NEXT: cmeq.4s v0, v0, #0
; CHECK-NEXT: Lloh14: ; CHECK-NEXT: Lloh14:
; CHECK-NEXT: adrp x8, lCPI7_0@PAGE ; CHECK-NEXT: adrp x8, lCPI7_0@PAGE
; CHECK-NEXT: bic.16b v0, v1, v0 ; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: shl.4s v0, v0, #31 ; CHECK-NEXT: Lloh15:
; CHECK-NEXT: Lloh15: ; CHECK-NEXT: ldr q1, [x8, lCPI7_0@PAGEOFF]
; CHECK-NEXT: ldr q1, [x8, lCPI7_0@PAGEOFF] ; CHECK-NEXT: shl.4s v0, v0, #31
; CHECK-NEXT: cmlt.4s v0, v0, #0 ; CHECK-NEXT: cmlt.4s v0, v0, #0
; CHECK-NEXT: and.16b v0, v0, v1 ; CHECK-NEXT: and.16b v0, v0, v1
; CHECK-NEXT: addv.4s s0, v0 ; CHECK-NEXT: addv.4s s0, v0
; CHECK-NEXT: fmov w0, s0 ; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh14, Lloh15
%cmp1 = icmp ne <4 x i32> %vec1, zeroinitializer %cmp1 = icmp ne <4 x i32> %vec1, zeroinitializer
%trunc_vec = trunc <4 x i32> %vec2 to <4 x i1> %trunc_vec = trunc <4 x i32> %vec2 to <4 x i1>
Context not available.
} }
define i4 @convert_to_bitmask_with_unknown_type_in_long_chain(<4 x i32> %vec1, <4 x i32> %vec2) { define i4 @convert_to_bitmask_with_unknown_type_in_long_chain(<4 x i32> %vec1, <4 x i32> %vec2) {
; CHECK-LABEL: lCPI8_0: ; CHECK-LABEL: convert_to_bitmask_with_unknown_type_in_long_chain:
; CHECK-NEXT: .short 1 ; CHECK: ; %bb.0:
; CHECK-NEXT: .short 2 ; CHECK-NEXT: cmeq.4s v0, v0, #0
; CHECK-NEXT: .short 4 ; CHECK-NEXT: cmeq.4s v1, v1, #0
; CHECK-NEXT: .short 8 ; CHECK-NEXT: Lloh16:
; CHECK-NEXT: adrp x8, lCPI8_0@PAGE
; CHECK-LABEL: convert_to_bitmask_with_unknown_type_in_long_chain ; CHECK-NEXT: movi d2, #0x000000ffffffff
; CHECK: ; %bb.0: ; CHECK-NEXT: movi d3, #0x00ffffffffffff
; CHECK-NEXT: cmeq.4s v0, v0, #0 ; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: Lloh16: ; CHECK-NEXT: movi d1, #0xffff0000ffff0000
; CHECK-NEXT: adrp x8, lCPI8_0@PAGE ; CHECK-NEXT: xtn.4h v0, v0
; CHECK-NEXT: cmeq.4s v1, v1, #0 ; CHECK-NEXT: orr.8b v0, v0, v2
; CHECK-NEXT: movi d2, #0x000000ffffffff ; CHECK-NEXT: movi d2, #0x00ffffffff0000
; CHECK-NEXT: bic.16b v0, v1, v0 ; CHECK-NEXT: eor.8b v1, v0, v1
; CHECK-NEXT: movi d1, #0xffff0000ffff0000 ; CHECK-NEXT: eor.8b v0, v0, v2
; CHECK-NEXT: xtn.4h v0, v0 ; CHECK-NEXT: mov.h v1[2], wzr
; CHECK-NEXT: movi d3, #0x00ffffffffffff ; CHECK-NEXT: orr.8b v0, v0, v3
; CHECK-NEXT: orr.8b v0, v0, v2 ; CHECK-NEXT: orr.8b v0, v1, v0
; CHECK-NEXT: movi d2, #0x00ffffffff0000 ; CHECK-NEXT: Lloh17:
; CHECK-NEXT: eor.8b v1, v0, v1 ; CHECK-NEXT: ldr d1, [x8, lCPI8_0@PAGEOFF]
; CHECK-NEXT: mov.h v1[2], wzr ; CHECK-NEXT: shl.4h v0, v0, #15
; CHECK-NEXT: eor.8b v0, v0, v2 ; CHECK-NEXT: cmlt.4h v0, v0, #0
; CHECK-NEXT: orr.8b v0, v0, v3 ; CHECK-NEXT: and.8b v0, v0, v1
; CHECK-NEXT: orr.8b v0, v1, v0 ; CHECK-NEXT: addv.4h h0, v0
; CHECK-NEXT: Lloh17: ; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ldr d1, [x8, lCPI8_0@PAGEOFF] ; CHECK-NEXT: ret
; CHECK-NEXT: shl.4h v0, v0, #15 ; CHECK-NEXT: .loh AdrpLdr Lloh16, Lloh17
; CHECK-NEXT: cmlt.4h v0, v0, #0
; CHECK-NEXT: and.8b v0, v0, v1
; CHECK-NEXT: addv.4h h0, v0
; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret
%cmp1 = icmp ne <4 x i32> %vec1, zeroinitializer %cmp1 = icmp ne <4 x i32> %vec1, zeroinitializer
%cmp2 = icmp eq <4 x i32> %vec2, zeroinitializer %cmp2 = icmp eq <4 x i32> %vec2, zeroinitializer
Context not available.
} }
define i4 @convert_to_bitmask_with_different_types_in_chain(<4 x i16> %vec1, <4 x i32> %vec2) { define i4 @convert_to_bitmask_with_different_types_in_chain(<4 x i16> %vec1, <4 x i32> %vec2) {
; CHECK-LABEL: lCPI9_0: ; CHECK-LABEL: convert_to_bitmask_with_different_types_in_chain:
; CHECK-NEXT: .short 1 ; CHECK: ; %bb.0:
; CHECK-NEXT: .short 2 ; CHECK-NEXT: cmeq.4s v1, v1, #0
; CHECK-NEXT: .short 4 ; CHECK-NEXT: cmeq.4h v0, v0, #0
; CHECK-NEXT: .short 8 ; CHECK-NEXT: Lloh18:
; CHECK-NEXT: adrp x8, lCPI9_0@PAGE
; CHECK-LABEL: convert_to_bitmask_with_different_types_in_chain ; CHECK-NEXT: xtn.4h v1, v1
; CHECK: ; %bb.0: ; CHECK-NEXT: orn.8b v0, v1, v0
; CHECK-NEXT: Lloh18: ; CHECK-NEXT: Lloh19:
; CHECK-NEXT: adrp x8, lCPI9_0@PAGE ; CHECK-NEXT: ldr d1, [x8, lCPI9_0@PAGEOFF]
; CHECK-NEXT: cmeq.4h v0, v0, #0 ; CHECK-NEXT: and.8b v0, v0, v1
; CHECK-NEXT: cmeq.4s v1, v1, #0 ; CHECK-NEXT: addv.4h h0, v0
; CHECK-NEXT: xtn.4h v1, v1 ; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: Lloh19: ; CHECK-NEXT: ret
; CHECK-NEXT: ldr d2, [x8, lCPI9_0@PAGEOFF] ; CHECK-NEXT: .loh AdrpLdr Lloh18, Lloh19
; CHECK-NEXT: orn.8b v0, v1, v0
; CHECK-NEXT: and.8b v0, v0, v2
; CHECK-NEXT: addv.4h h0, v0
; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret
%cmp1 = icmp ne <4 x i16> %vec1, zeroinitializer %cmp1 = icmp ne <4 x i16> %vec1, zeroinitializer
%cmp2 = icmp eq <4 x i32> %vec2, zeroinitializer %cmp2 = icmp eq <4 x i32> %vec2, zeroinitializer
Context not available.
define i16 @convert_to_bitmask_without_knowing_type(<16 x i1> %vec) { define i16 @convert_to_bitmask_without_knowing_type(<16 x i1> %vec) {
; CHECK-LABEL: convert_to_bitmask_without_knowing_type: ; CHECK-LABEL: convert_to_bitmask_without_knowing_type:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: shl.16b v0, v0, #7
; CHECK-NEXT: Lloh20: ; CHECK-NEXT: Lloh20:
; CHECK-NEXT: adrp x8, lCPI10_0@PAGE ; CHECK-NEXT: adrp x8, lCPI10_0@PAGE
; CHECK-NEXT: shl.16b v0, v0, #7
; CHECK-NEXT: cmlt.16b v0, v0, #0
; CHECK-NEXT: Lloh21: ; CHECK-NEXT: Lloh21:
; CHECK-NEXT: ldr q1, [x8, lCPI10_0@PAGEOFF] ; CHECK-NEXT: ldr q1, [x8, lCPI10_0@PAGEOFF]
; CHECK-NEXT: cmlt.16b v0, v0, #0
; CHECK-NEXT: and.16b v0, v0, v1 ; CHECK-NEXT: and.16b v0, v0, v1
; CHECK-NEXT: ext.16b v1, v0, v0, #8 ; CHECK-NEXT: ext.16b v1, v0, v0, #8
; CHECK-NEXT: addv.8b b0, v0 ; CHECK-NEXT: addv.8b b0, v0
; CHECK-NEXT: addv.8b b1, v1 ; CHECK-NEXT: addv.8b b1, v1
; CHECK-NEXT: fmov w9, s0 ; CHECK-NEXT: fmov w8, s0
; CHECK-NEXT: fmov w8, s1 ; CHECK-NEXT: fmov w9, s1
; CHECK-NEXT: orr w0, w9, w8, lsl #8 ; CHECK-NEXT: orr w0, w8, w9, lsl #8
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh20, Lloh21
%bitmask = bitcast <16 x i1> %vec to i16 %bitmask = bitcast <16 x i1> %vec to i16
ret i16 %bitmask ret i16 %bitmask
} }
define i2 @convert_to_bitmask_2xi32(<2 x i32> %vec) { define i2 @convert_to_bitmask_2xi32(<2 x i32> %vec) {
; CHECK-LABEL: convert_to_bitmask_2xi32 ; CHECK-LABEL: convert_to_bitmask_2xi32:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh22: ; CHECK-NEXT: Lloh22:
; CHECK-NEXT: adrp x8, lCPI11_0@PAGE ; CHECK-NEXT: adrp x8, lCPI11_0@PAGE
; CHECK-NEXT: cmeq.2s v0, v0, #0 ; CHECK-NEXT: cmeq.2s v0, v0, #0
; CHECK-NEXT: Lloh23: ; CHECK-NEXT: Lloh23:
; CHECK-NEXT: ldr d1, [x8, lCPI11_0@PAGEOFF] ; CHECK-NEXT: ldr d1, [x8, lCPI11_0@PAGEOFF]
; CHECK-NEXT: bic.8b v0, v1, v0 ; CHECK-NEXT: bic.8b v0, v1, v0
; CHECK-NEXT: addp.2s v0, v0, v0 ; CHECK-NEXT: addp.2s v0, v0, v0
; CHECK-NEXT: fmov w0, s0 ; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh22, Lloh23
%cmp_result = icmp ne <2 x i32> %vec, zeroinitializer %cmp_result = icmp ne <2 x i32> %vec, zeroinitializer
%bitmask = bitcast <2 x i1> %cmp_result to i2 %bitmask = bitcast <2 x i1> %cmp_result to i2
Context not available.
} }
define i4 @convert_to_bitmask_4xi8(<4 x i8> %vec) { define i4 @convert_to_bitmask_4xi8(<4 x i8> %vec) {
; CHECK-LABEL: convert_to_bitmask_4xi8 ; CHECK-LABEL: convert_to_bitmask_4xi8:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: bic.4h v0, #255, lsl #8
; CHECK-NEXT: Lloh24: ; CHECK-NEXT: Lloh24:
; CHECK-NEXT: adrp x8, lCPI12_0@PAGE ; CHECK-NEXT: adrp x8, lCPI12_0@PAGE
; CHECK-NEXT: bic.4h v0, #255, lsl #8
; CHECK-NEXT: cmeq.4h v0, v0, #0
; CHECK-NEXT: Lloh25: ; CHECK-NEXT: Lloh25:
; CHECK-NEXT: ldr d1, [x8, lCPI12_0@PAGEOFF] ; CHECK-NEXT: ldr d1, [x8, lCPI12_0@PAGEOFF]
; CHECK-NEXT: bic.8b v0, v1, v0 ; CHECK-NEXT: cmeq.4h v0, v0, #0
; CHECK-NEXT: addv.4h h0, v0 ; CHECK-NEXT: bic.8b v0, v1, v0
; CHECK-NEXT: fmov w0, s0 ; CHECK-NEXT: addv.4h h0, v0
; CHECK-NEXT: ret ; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh24, Lloh25
%cmp_result = icmp ne <4 x i8> %vec, zeroinitializer %cmp_result = icmp ne <4 x i8> %vec, zeroinitializer
%bitmask = bitcast <4 x i1> %cmp_result to i4 %bitmask = bitcast <4 x i1> %cmp_result to i4
Context not available.
} }
define i8 @convert_to_bitmask_8xi2(<8 x i2> %vec) { define i8 @convert_to_bitmask_8xi2(<8 x i2> %vec) {
; CHECK-LABEL: convert_to_bitmask_8xi2 ; CHECK-LABEL: convert_to_bitmask_8xi2:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: movi.8b v1, #3 ; CHECK-NEXT: movi.8b v1, #3
; CHECK-NEXT: Lloh26: ; CHECK-NEXT: Lloh26:
; CHECK-NEXT: adrp x8, lCPI13_0@PAGE ; CHECK-NEXT: adrp x8, lCPI13_0@PAGE
; CHECK-NEXT: and.8b v0, v0, v1 ; CHECK-NEXT: and.8b v0, v0, v1
; CHECK-NEXT: Lloh27: ; CHECK-NEXT: Lloh27:
; CHECK-NEXT: ldr d1, [x8, lCPI13_0@PAGEOFF] ; CHECK-NEXT: ldr d1, [x8, lCPI13_0@PAGEOFF]
; CHECK-NEXT: cmeq.8b v0, v0, #0 ; CHECK-NEXT: cmeq.8b v0, v0, #0
; CHECK-NEXT: bic.8b v0, v1, v0 ; CHECK-NEXT: bic.8b v0, v1, v0
; CHECK-NEXT: addv.8b b0, v0 ; CHECK-NEXT: addv.8b b0, v0
; CHECK-NEXT: fmov w0, s0 ; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh26, Lloh27
%cmp_result = icmp ne <8 x i2> %vec, zeroinitializer %cmp_result = icmp ne <8 x i2> %vec, zeroinitializer
%bitmask = bitcast <8 x i1> %cmp_result to i8 %bitmask = bitcast <8 x i1> %cmp_result to i8
Context not available.
} }
define i4 @convert_to_bitmask_float(<4 x float> %vec) { define i4 @convert_to_bitmask_float(<4 x float> %vec) {
; CHECK-LABEL: lCPI14_0: ; CHECK-LABEL: convert_to_bitmask_float:
; CHECK-NEXT: .long 1
; CHECK-NEXT: .long 2
; CHECK-NEXT: .long 4
; CHECK-NEXT: .long 8
; CHECK-LABEL: convert_to_bitmask_float
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh28: ; CHECK-NEXT: fcmgt.4s v1, v0, #0.0
; CHECK-NEXT: adrp x8, lCPI14_0@PAGE ; CHECK-NEXT: fcmlt.4s v0, v0, #0.0
; CHECK-NEXT: fcmgt.4s v1, v0, #0.0 ; CHECK-NEXT: Lloh28:
; CHECK-NEXT: fcmlt.4s v0, v0, #0.0 ; CHECK-NEXT: adrp x8, lCPI14_0@PAGE
; CHECK-NEXT: Lloh29: ; CHECK-NEXT: orr.16b v0, v0, v1
; CHECK-NEXT: ldr q2, [x8, lCPI14_0@PAGEOFF] ; CHECK-NEXT: Lloh29:
; CHECK-NEXT: orr.16b v0, v0, v1 ; CHECK-NEXT: ldr q1, [x8, lCPI14_0@PAGEOFF]
; CHECK-NEXT: and.16b v0, v0, v2 ; CHECK-NEXT: and.16b v0, v0, v1
; CHECK-NEXT: addv.4s s0, v0 ; CHECK-NEXT: addv.4s s0, v0
; CHECK-NEXT: fmov w0, s0 ; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh28, Lloh29
%cmp_result = fcmp one <4 x float> %vec, zeroinitializer %cmp_result = fcmp one <4 x float> %vec, zeroinitializer
%bitmask = bitcast <4 x i1> %cmp_result to i4 %bitmask = bitcast <4 x i1> %cmp_result to i4
Context not available.
; Larger vector types don't map directly, but the can be split/truncated and then converted. ; Larger vector types don't map directly, but the can be split/truncated and then converted.
; After the comparison against 0, this is truncated to <8 x i16>, which is valid again. ; After the comparison against 0, this is truncated to <8 x i16>, which is valid again.
define i8 @convert_large_vector(<8 x i32> %vec) { define i8 @convert_large_vector(<8 x i32> %vec) {
; CHECK-LABEL: lCPI15_0:
; CHECK-NEXT: .short 1
; CHECK-NEXT: .short 2
; CHECK-NEXT: .short 4
; CHECK-NEXT: .short 8
; CHECK-NEXT: .short 16
; CHECK-NEXT: .short 32
; CHECK-NEXT: .short 64
; CHECK-NEXT: .short 128
; CHECK-LABEL: convert_large_vector: ; CHECK-LABEL: convert_large_vector:
; CHECK: Lloh30: ; CHECK: ; %bb.0:
; CHECK-NEXT: adrp x8, lCPI15_0@PAGE ; CHECK-NEXT: sub sp, sp, #16
; CHECK-NEXT: cmeq.4s v1, v1, #0 ; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: cmeq.4s v0, v0, #0 ; CHECK-NEXT: cmeq.4s v1, v1, #0
; CHECK-NEXT: uzp1.8h v0, v0, v1 ; CHECK-NEXT: cmeq.4s v0, v0, #0
; CHECK-NEXT: Lloh31: ; CHECK-NEXT: Lloh30:
; CHECK-NEXT: ldr q1, [x8, lCPI15_0@PAGEOFF] ; CHECK-NEXT: adrp x8, lCPI15_0@PAGE
; CHECK-NEXT: bic.16b v0, v1, v0 ; CHECK-NEXT: uzp1.8h v0, v0, v1
; CHECK-NEXT: addv.8h h0, v0 ; CHECK-NEXT: Lloh31:
; CHECK-NEXT: fmov w8, s0 ; CHECK-NEXT: ldr q1, [x8, lCPI15_0@PAGEOFF]
; CHECK-NEXT: and w0, w8, #0xff ; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: add sp, sp, #16 ; CHECK-NEXT: addv.8h h0, v0
; CHECK-NEXT: ret ; CHECK-NEXT: fmov w8, s0
; CHECK-NEXT: and w0, w8, #0xff
; CHECK-NEXT: add sp, sp, #16
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh30, Lloh31
%cmp_result = icmp ne <8 x i32> %vec, zeroinitializer %cmp_result = icmp ne <8 x i32> %vec, zeroinitializer
%bitmask = bitcast <8 x i1> %cmp_result to i8 %bitmask = bitcast <8 x i1> %cmp_result to i8
Context not available.
} }
define i4 @convert_legalized_illegal_element_size(<4 x i22> %vec) { define i4 @convert_legalized_illegal_element_size(<4 x i22> %vec) {
; CHECK-LABEL: convert_legalized_illegal_element_size ; CHECK-LABEL: convert_legalized_illegal_element_size:
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: movi.4s v1, #63, msl #16 ; CHECK-NEXT: movi.4s v1, #63, msl #16
; CHECK-NEXT: Lloh32: ; CHECK-NEXT: Lloh32:
; CHECK-NEXT: adrp x8, lCPI16_0@PAGE ; CHECK-NEXT: adrp x8, lCPI16_0@PAGE
; CHECK-NEXT: cmtst.4s v0, v0, v1 ; CHECK-NEXT: cmtst.4s v0, v0, v1
; CHECK-NEXT: Lloh33: ; CHECK-NEXT: Lloh33:
; CHECK-NEXT: ldr d1, [x8, lCPI16_0@PAGEOFF] ; CHECK-NEXT: ldr d1, [x8, lCPI16_0@PAGEOFF]
; CHECK-NEXT: xtn.4h v0, v0 ; CHECK-NEXT: xtn.4h v0, v0
; CHECK-NEXT: and.8b v0, v0, v1 ; CHECK-NEXT: and.8b v0, v0, v1
; CHECK-NEXT: addv.4h h0, v0 ; CHECK-NEXT: addv.4h h0, v0
; CHECK-NEXT: fmov w0, s0 ; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh32, Lloh33
%cmp_result = icmp ne <4 x i22> %vec, zeroinitializer %cmp_result = icmp ne <4 x i22> %vec, zeroinitializer
%bitmask = bitcast <4 x i1> %cmp_result to i4 %bitmask = bitcast <4 x i1> %cmp_result to i4
Context not available.
; This may still be converted as a v8i8 after the vector concat (but not as v4iX). ; This may still be converted as a v8i8 after the vector concat (but not as v4iX).
define i8 @no_direct_convert_for_bad_concat(<4 x i32> %vec) { define i8 @no_direct_convert_for_bad_concat(<4 x i32> %vec) {
; CHECK-LABEL: no_direct_convert_for_bad_concat: ; CHECK-LABEL: no_direct_convert_for_bad_concat:
; CHECK: cmtst.4s v0, v0, v0 ; CHECK: ; %bb.0:
; CHECK-NOT: addv.4 ; CHECK-NEXT: cmtst.4s v0, v0, v0
; CHECK-NEXT: Lloh34:
; CHECK-NEXT: adrp x8, lCPI17_0@PAGE
; CHECK-NEXT: xtn.4h v0, v0
; CHECK-NEXT: umov.h w9, v0[0]
; CHECK-NEXT: mov.b v1[4], w9
; CHECK-NEXT: umov.h w9, v0[1]
; CHECK-NEXT: mov.b v1[5], w9
; CHECK-NEXT: umov.h w9, v0[2]
; CHECK-NEXT: mov.b v1[6], w9
; CHECK-NEXT: umov.h w9, v0[3]
; CHECK-NEXT: mov.b v1[7], w9
; CHECK-NEXT: shl.8b v0, v1, #7
; CHECK-NEXT: Lloh35:
; CHECK-NEXT: ldr d1, [x8, lCPI17_0@PAGEOFF]
; CHECK-NEXT: cmlt.8b v0, v0, #0
; CHECK-NEXT: and.8b v0, v0, v1
; CHECK-NEXT: addv.8b b0, v0
; CHECK-NEXT: fmov w0, s0
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh34, Lloh35
%cmp_result = icmp ne <4 x i32> %vec, zeroinitializer %cmp_result = icmp ne <4 x i32> %vec, zeroinitializer
%vector_pad = shufflevector <4 x i1> poison, <4 x i1> %cmp_result, <8 x i32> <i32 undef, i32 undef, i32 undef, i32 undef, i32 4, i32 5, i32 6, i32 7> %vector_pad = shufflevector <4 x i1> poison, <4 x i1> %cmp_result, <8 x i32> <i32 undef, i32 undef, i32 undef, i32 undef, i32 4, i32 5, i32 6, i32 7>
Context not available.
define <8 x i1> @no_convert_without_direct_bitcast(<8 x i16> %vec) { define <8 x i1> @no_convert_without_direct_bitcast(<8 x i16> %vec) {
; CHECK-LABEL: no_convert_without_direct_bitcast: ; CHECK-LABEL: no_convert_without_direct_bitcast:
; CHECK: cmtst.8h v0, v0, v0 ; CHECK: ; %bb.0:
; CHECK-NOT: addv.4s s0, v0 ; CHECK-NEXT: cmtst.8h v0, v0, v0
; CHECK-NEXT: xtn.8b v0, v0
; CHECK-NEXT: ret
%cmp_result = icmp ne <8 x i16> %vec, zeroinitializer %cmp_result = icmp ne <8 x i16> %vec, zeroinitializer
ret <8 x i1> %cmp_result ret <8 x i1> %cmp_result
} }
define i6 @no_combine_illegal_num_elements(<6 x i32> %vec) { define i6 @no_combine_illegal_num_elements(<6 x i32> %vec) {
; CHECK-LABEL: no_combine_illegal_num_elements ; CHECK-LABEL: no_combine_illegal_num_elements:
; CHECK-NOT: addv ; CHECK: ; %bb.0:
; CHECK-NEXT: sub sp, sp, #16
; CHECK-NEXT: .cfi_def_cfa_offset 16
; CHECK-NEXT: fmov s0, w0
; CHECK-NEXT: fmov s1, w4
; CHECK-NEXT: mov.s v0[1], w1
; CHECK-NEXT: mov.s v1[1], w5
; CHECK-NEXT: mov.s v0[2], w2
; CHECK-NEXT: cmeq.4s v1, v1, #0
; CHECK-NEXT: mov.s v0[3], w3
; CHECK-NEXT: cmeq.4s v0, v0, #0
; CHECK-NEXT: uzp1.8h v0, v0, v1
; CHECK-NEXT: mvn.16b v0, v0
; CHECK-NEXT: xtn.8b v0, v0
; CHECK-NEXT: umov.b w8, v0[0]
; CHECK-NEXT: umov.b w9, v0[1]
; CHECK-NEXT: umov.b w10, v0[2]
; CHECK-NEXT: and w8, w8, #0x1
; CHECK-NEXT: bfi w8, w9, #1, #1
; CHECK-NEXT: umov.b w9, v0[3]
; CHECK-NEXT: bfi w8, w10, #2, #1
; CHECK-NEXT: umov.b w10, v0[4]
; CHECK-NEXT: bfi w8, w9, #3, #1
; CHECK-NEXT: umov.b w9, v0[5]
; CHECK-NEXT: bfi w8, w10, #4, #1
; CHECK-NEXT: orr w8, w8, w9, lsl #5
; CHECK-NEXT: and w0, w8, #0x3f
; CHECK-NEXT: add sp, sp, #16
; CHECK-NEXT: ret
%cmp_result = icmp ne <6 x i32> %vec, zeroinitializer %cmp_result = icmp ne <6 x i32> %vec, zeroinitializer
%bitmask = bitcast <6 x i1> %cmp_result to i6 %bitmask = bitcast <6 x i1> %cmp_result to i6
Context not available.

llvm/test/CodeGen/AArch64/vec-combine-compare-truncate-store.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
; RUN: llc -mtriple=aarch64-apple-darwin -mattr=+neon -verify-machineinstrs < %s | FileCheck %s ; RUN: llc -mtriple=aarch64-apple-darwin -mattr=+neon -verify-machineinstrs < %s | FileCheck %s
define void @store_16_elements(<16 x i8> %vec, ptr %out) { define void @store_16_elements(<16 x i8> %vec, ptr %out) {
; Bits used in mask ; Bits used in mask
dmgreenUnsubmitted
Done
ReplyInline Actions

These too.

dmgreen: These too.
; CHECK-LABEL: lCPI0_0 ; CHECK-LABEL: store_16_elements:
; CHECK-NEXT: .byte 1
; CHECK-NEXT: .byte 2
; CHECK-NEXT: .byte 4
; CHECK-NEXT: .byte 8
; CHECK-NEXT: .byte 16
; CHECK-NEXT: .byte 32
; CHECK-NEXT: .byte 64
; CHECK-NEXT: .byte 128
; CHECK-NEXT: .byte 1
; CHECK-NEXT: .byte 2
; CHECK-NEXT: .byte 4
; CHECK-NEXT: .byte 8
; CHECK-NEXT: .byte 16
; CHECK-NEXT: .byte 32
; CHECK-NEXT: .byte 64
; CHECK-NEXT: .byte 128
; Actual conversion
; CHECK-LABEL: store_16_elements
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh0: ; CHECK-NEXT: Lloh0:
; CHECK-NEXT: adrp x8, lCPI0_0@PAGE ; CHECK-NEXT: adrp x8, lCPI0_0@PAGE
; CHECK-NEXT: cmeq.16b v0, v0, #0 ; CHECK-NEXT: cmeq.16b v0, v0, #0
; CHECK-NEXT: Lloh1: ; CHECK-NEXT: Lloh1:
; CHECK-NEXT: ldr q1, [x8, lCPI0_0@PAGEOFF] ; CHECK-NEXT: ldr q1, [x8, lCPI0_0@PAGEOFF]
; CHECK-NEXT: bic.16b v0, v1, v0 ; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: ext.16b v1, v0, v0, #8 ; CHECK-NEXT: ext.16b v1, v0, v0, #8
; CHECK-NEXT: addv.8b b0, v0 ; CHECK-NEXT: addv.8b b0, v0
; CHECK-NEXT: addv.8b b1, v1 ; CHECK-NEXT: addv.8b b1, v1
; CHECK-NEXT: fmov w9, s0 ; CHECK-NEXT: fmov w8, s0
; CHECK-NEXT: fmov w8, s1 ; CHECK-NEXT: fmov w9, s1
; CHECK-NEXT: orr w8, w9, w8, lsl #8 ; CHECK-NEXT: orr w8, w8, w9, lsl #8
; CHECK-NEXT: strh w8, [x0] ; CHECK-NEXT: strh w8, [x0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh0, Lloh1
; Actual conversion
%cmp_result = icmp ne <16 x i8> %vec, zeroinitializer %cmp_result = icmp ne <16 x i8> %vec, zeroinitializer
store <16 x i1> %cmp_result, ptr %out store <16 x i1> %cmp_result, ptr %out
Context not available.
} }
define void @store_8_elements(<8 x i16> %vec, ptr %out) { define void @store_8_elements(<8 x i16> %vec, ptr %out) {
; CHECK-LABEL: lCPI1_0: ; CHECK-LABEL: store_8_elements:
; CHECK-NEXT: .short 1
; CHECK-NEXT: .short 2
; CHECK-NEXT: .short 4
; CHECK-NEXT: .short 8
; CHECK-NEXT: .short 16
; CHECK-NEXT: .short 32
; CHECK-NEXT: .short 64
; CHECK-NEXT: .short 128
; CHECK-LABEL: store_8_elements
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh2: ; CHECK-NEXT: Lloh2:
; CHECK-NEXT: adrp x8, lCPI1_0@PAGE ; CHECK-NEXT: adrp x8, lCPI1_0@PAGE
; CHECK-NEXT: cmeq.8h v0, v0, #0 ; CHECK-NEXT: cmeq.8h v0, v0, #0
; CHECK-NEXT: Lloh3: ; CHECK-NEXT: Lloh3:
; CHECK-NEXT: ldr q1, [x8, lCPI1_0@PAGEOFF] ; CHECK-NEXT: ldr q1, [x8, lCPI1_0@PAGEOFF]
; CHECK-NEXT: bic.16b v0, v1, v0 ; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: addv.8h h0, v0 ; CHECK-NEXT: addv.8h h0, v0
; CHECK-NEXT: fmov w8, s0 ; CHECK-NEXT: fmov w8, s0
; CHECK-NEXT: strb w8, [x0] ; CHECK-NEXT: strb w8, [x0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh2, Lloh3
%cmp_result = icmp ne <8 x i16> %vec, zeroinitializer %cmp_result = icmp ne <8 x i16> %vec, zeroinitializer
store <8 x i1> %cmp_result, ptr %out store <8 x i1> %cmp_result, ptr %out
Context not available.
} }
define void @store_4_elements(<4 x i32> %vec, ptr %out) { define void @store_4_elements(<4 x i32> %vec, ptr %out) {
; CHECK-LABEL: lCPI2_0: ; CHECK-LABEL: store_4_elements:
; CHECK-NEXT: .long 1
; CHECK-NEXT: .long 2
; CHECK-NEXT: .long 4
; CHECK-NEXT: .long 8
; CHECK-LABEL: store_4_elements
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh4: ; CHECK-NEXT: Lloh4:
; CHECK-NEXT: adrp x8, lCPI2_0@PAGE ; CHECK-NEXT: adrp x8, lCPI2_0@PAGE
; CHECK-NEXT: cmeq.4s v0, v0, #0 ; CHECK-NEXT: cmeq.4s v0, v0, #0
; CHECK-NEXT: Lloh5: ; CHECK-NEXT: Lloh5:
; CHECK-NEXT: ldr q1, [x8, lCPI2_0@PAGEOFF] ; CHECK-NEXT: ldr q1, [x8, lCPI2_0@PAGEOFF]
; CHECK-NEXT: bic.16b v0, v1, v0 ; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: addv.4s s0, v0 ; CHECK-NEXT: addv.4s s0, v0
; CHECK-NEXT: fmov w8, s0 ; CHECK-NEXT: fmov w8, s0
; CHECK-NEXT: strb w8, [x0] ; CHECK-NEXT: strb w8, [x0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh4, Lloh5
%cmp_result = icmp ne <4 x i32> %vec, zeroinitializer %cmp_result = icmp ne <4 x i32> %vec, zeroinitializer
store <4 x i1> %cmp_result, ptr %out store <4 x i1> %cmp_result, ptr %out
Context not available.
} }
define void @store_2_elements(<2 x i64> %vec, ptr %out) { define void @store_2_elements(<2 x i64> %vec, ptr %out) {
; CHECK-LABEL: lCPI3_0: ; CHECK-LABEL: store_2_elements:
; CHECK-NEXT: .quad 1
; CHECK-NEXT: .quad 2
; CHECK-LABEL: store_2_elements
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh6: ; CHECK-NEXT: Lloh6:
; CHECK-NEXT: adrp x8, lCPI3_0@PAGE ; CHECK-NEXT: adrp x8, lCPI3_0@PAGE
; CHECK-NEXT: cmeq.2d v0, v0, #0 ; CHECK-NEXT: cmeq.2d v0, v0, #0
; CHECK-NEXT: Lloh7: ; CHECK-NEXT: Lloh7:
; CHECK-NEXT: ldr q1, [x8, lCPI3_0@PAGEOFF] ; CHECK-NEXT: ldr q1, [x8, lCPI3_0@PAGEOFF]
; CHECK-NEXT: bic.16b v0, v1, v0 ; CHECK-NEXT: bic.16b v0, v1, v0
; CHECK-NEXT: addp.2d d0, v0 ; CHECK-NEXT: addp.2d d0, v0
; CHECK-NEXT: fmov x8, d0 ; CHECK-NEXT: fmov x8, d0
; CHECK-NEXT: strb w8, [x0] ; CHECK-NEXT: strb w8, [x0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh6, Lloh7
%cmp_result = icmp ne <2 x i64> %vec, zeroinitializer %cmp_result = icmp ne <2 x i64> %vec, zeroinitializer
store <2 x i1> %cmp_result, ptr %out store <2 x i1> %cmp_result, ptr %out
Context not available.
} }
define void @add_trunc_compare_before_store(<4 x i32> %vec, ptr %out) { define void @add_trunc_compare_before_store(<4 x i32> %vec, ptr %out) {
; CHECK-LABEL: lCPI4_0: ; CHECK-LABEL: add_trunc_compare_before_store:
; CHECK-NEXT: .long 1
; CHECK-NEXT: .long 2
; CHECK-NEXT: .long 4
; CHECK-NEXT: .long 8
; CHECK-LABEL: add_trunc_compare_before_store
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: shl.4s v0, v0, #31
; CHECK-NEXT: Lloh8: ; CHECK-NEXT: Lloh8:
; CHECK-NEXT: adrp x8, lCPI4_0@PAGE ; CHECK-NEXT: adrp x8, lCPI4_0@PAGE
; CHECK-NEXT: shl.4s v0, v0, #31
; CHECK-NEXT: cmlt.4s v0, v0, #0
; CHECK-NEXT: Lloh9: ; CHECK-NEXT: Lloh9:
; CHECK-NEXT: ldr q1, [x8, lCPI4_0@PAGEOFF] ; CHECK-NEXT: ldr q1, [x8, lCPI4_0@PAGEOFF]
; CHECK-NEXT: and.16b v0, v0, v1 ; CHECK-NEXT: cmlt.4s v0, v0, #0
; CHECK-NEXT: addv.4s s0, v0 ; CHECK-NEXT: and.16b v0, v0, v1
; CHECK-NEXT: fmov w8, s0 ; CHECK-NEXT: addv.4s s0, v0
; CHECK-NEXT: strb w8, [x0] ; CHECK-NEXT: fmov w8, s0
; CHECK-NEXT: ret ; CHECK-NEXT: strb w8, [x0]
; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh8, Lloh9
%trunc = trunc <4 x i32> %vec to <4 x i1> %trunc = trunc <4 x i32> %vec to <4 x i1>
store <4 x i1> %trunc, ptr %out store <4 x i1> %trunc, ptr %out
Context not available.
} }
define void @add_trunc_mask_unknown_vector_type(<4 x i1> %vec, ptr %out) { define void @add_trunc_mask_unknown_vector_type(<4 x i1> %vec, ptr %out) {
; CHECK-LABEL: lCPI5_0: ; CHECK-LABEL: add_trunc_mask_unknown_vector_type:
; CHECK: .short 1
; CHECK: .short 2
; CHECK: .short 4
; CHECK: .short 8
; CHECK-LABEL: add_trunc_mask_unknown_vector_type
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh10: ; CHECK-NEXT: shl.4h v0, v0, #15
; CHECK-NEXT: adrp x8, lCPI5_0@PAGE ; CHECK-NEXT: Lloh10:
; CHECK-NEXT: shl.4h v0, v0, #15 ; CHECK-NEXT: adrp x8, lCPI5_0@PAGE
; CHECK-NEXT: cmlt.4h v0, v0, #0 ; CHECK-NEXT: Lloh11:
; CHECK-NEXT: Lloh11: ; CHECK-NEXT: ldr d1, [x8, lCPI5_0@PAGEOFF]
; CHECK-NEXT: ldr d1, [x8, lCPI5_0@PAGEOFF] ; CHECK-NEXT: cmlt.4h v0, v0, #0
; CHECK-NEXT: and.8b v0, v0, v1 ; CHECK-NEXT: and.8b v0, v0, v1
; CHECK-NEXT: addv.4h h0, v0 ; CHECK-NEXT: addv.4h h0, v0
; CHECK-NEXT: fmov w8, s0 ; CHECK-NEXT: fmov w8, s0
; CHECK-NEXT: strb w8, [x0] ; CHECK-NEXT: strb w8, [x0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh10, Lloh11
store <4 x i1> %vec, ptr %out store <4 x i1> %vec, ptr %out
ret void ret void
} }
define void @store_8_elements_64_bit_vector(<8 x i8> %vec, ptr %out) { define void @store_8_elements_64_bit_vector(<8 x i8> %vec, ptr %out) {
; CHECK-LABEL: lCPI6_0: ; CHECK-LABEL: store_8_elements_64_bit_vector:
; CHECK-NEXT: .byte 1
; CHECK-NEXT: .byte 2
; CHECK-NEXT: .byte 4
; CHECK-NEXT: .byte 8
; CHECK-NEXT: .byte 16
; CHECK-NEXT: .byte 32
; CHECK-NEXT: .byte 64
; CHECK-NEXT: .byte 128
; CHECK-LABEL: store_8_elements_64_bit_vector
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh12: ; CHECK-NEXT: Lloh12:
; CHECK-NEXT: adrp x8, lCPI6_0@PAGE ; CHECK-NEXT: adrp x8, lCPI6_0@PAGE
; CHECK-NEXT: cmeq.8b v0, v0, #0 ; CHECK-NEXT: cmeq.8b v0, v0, #0
; CHECK-NEXT: Lloh13: ; CHECK-NEXT: Lloh13:
; CHECK-NEXT: ldr d1, [x8, lCPI6_0@PAGEOFF] ; CHECK-NEXT: ldr d1, [x8, lCPI6_0@PAGEOFF]
; CHECK-NEXT: bic.8b v0, v1, v0 ; CHECK-NEXT: bic.8b v0, v1, v0
; CHECK-NEXT: addv.8b b0, v0 ; CHECK-NEXT: addv.8b b0, v0
; CHECK-NEXT: st1.b { v0 }[0], [x0] ; CHECK-NEXT: st1.b { v0 }[0], [x0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh12, Lloh13
%cmp_result = icmp ne <8 x i8> %vec, zeroinitializer %cmp_result = icmp ne <8 x i8> %vec, zeroinitializer
store <8 x i1> %cmp_result, ptr %out store <8 x i1> %cmp_result, ptr %out
Context not available.
} }
define void @store_4_elements_64_bit_vector(<4 x i16> %vec, ptr %out) { define void @store_4_elements_64_bit_vector(<4 x i16> %vec, ptr %out) {
; CHECK-LABEL: lCPI7_0: ; CHECK-LABEL: store_4_elements_64_bit_vector:
; CHECK-NEXT: .short 1
; CHECK-NEXT: .short 2
; CHECK-NEXT: .short 4
; CHECK-NEXT: .short 8
; CHECK-LABEL: store_4_elements_64_bit_vector
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh14: ; CHECK-NEXT: Lloh14:
; CHECK-NEXT: adrp x8, lCPI7_0@PAGE ; CHECK-NEXT: adrp x8, lCPI7_0@PAGE
; CHECK-NEXT: cmeq.4h v0, v0, #0 ; CHECK-NEXT: cmeq.4h v0, v0, #0
; CHECK-NEXT: Lloh15: ; CHECK-NEXT: Lloh15:
; CHECK-NEXT: ldr d1, [x8, lCPI7_0@PAGEOFF] ; CHECK-NEXT: ldr d1, [x8, lCPI7_0@PAGEOFF]
; CHECK-NEXT: bic.8b v0, v1, v0 ; CHECK-NEXT: bic.8b v0, v1, v0
; CHECK-NEXT: addv.4h h0, v0 ; CHECK-NEXT: addv.4h h0, v0
; CHECK-NEXT: fmov w8, s0 ; CHECK-NEXT: fmov w8, s0
; CHECK-NEXT: strb w8, [x0] ; CHECK-NEXT: strb w8, [x0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh14, Lloh15
%cmp_result = icmp ne <4 x i16> %vec, zeroinitializer %cmp_result = icmp ne <4 x i16> %vec, zeroinitializer
store <4 x i1> %cmp_result, ptr %out store <4 x i1> %cmp_result, ptr %out
Context not available.
} }
define void @store_2_elements_64_bit_vector(<2 x i32> %vec, ptr %out) { define void @store_2_elements_64_bit_vector(<2 x i32> %vec, ptr %out) {
; CHECK-LABEL: lCPI8_0: ; CHECK-LABEL: store_2_elements_64_bit_vector:
; CHECK-NEXT: .long 1
; CHECK-NEXT: .long 2
; CHECK-LABEL: store_2_elements_64_bit_vector
; CHECK: ; %bb.0: ; CHECK: ; %bb.0:
; CHECK-NEXT: Lloh16: ; CHECK-NEXT: Lloh16:
; CHECK-NEXT: adrp x8, lCPI8_0@PAGE ; CHECK-NEXT: adrp x8, lCPI8_0@PAGE
; CHECK-NEXT: cmeq.2s v0, v0, #0 ; CHECK-NEXT: cmeq.2s v0, v0, #0
; CHECK-NEXT: Lloh17: ; CHECK-NEXT: Lloh17:
; CHECK-NEXT: ldr d1, [x8, lCPI8_0@PAGEOFF] ; CHECK-NEXT: ldr d1, [x8, lCPI8_0@PAGEOFF]
; CHECK-NEXT: bic.8b v0, v1, v0 ; CHECK-NEXT: bic.8b v0, v1, v0
; CHECK-NEXT: addp.2s v0, v0, v0 ; CHECK-NEXT: addp.2s v0, v0, v0
; CHECK-NEXT: fmov w8, s0 ; CHECK-NEXT: fmov w8, s0
; CHECK-NEXT: strb w8, [x0] ; CHECK-NEXT: strb w8, [x0]
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh16, Lloh17
%cmp_result = icmp ne <2 x i32> %vec, zeroinitializer %cmp_result = icmp ne <2 x i32> %vec, zeroinitializer
store <2 x i1> %cmp_result, ptr %out store <2 x i1> %cmp_result, ptr %out
Context not available.
} }
define void @no_combine_without_truncate(<16 x i8> %vec, ptr %out) { define void @no_combine_without_truncate(<16 x i8> %vec, ptr %out) {
; CHECK-LABEL: no_combine_without_truncate ; CHECK-LABEL: no_combine_without_truncate:
; CHECK: cmtst.16b v0, v0, v0 ; CHECK: ; %bb.0:
; CHECK-NOT: addv.8b b0, v0 ; CHECK-NEXT: cmtst.16b v0, v0, v0
; CHECK-NEXT: str q0, [x0]
; CHECK-NEXT: ret
%cmp_result = icmp ne <16 x i8> %vec, zeroinitializer %cmp_result = icmp ne <16 x i8> %vec, zeroinitializer
%extended_result = sext <16 x i1> %cmp_result to <16 x i8> %extended_result = sext <16 x i1> %cmp_result to <16 x i8>
Context not available.
} }
define void @no_combine_for_non_bool_truncate(<4 x i32> %vec, ptr %out) { define void @no_combine_for_non_bool_truncate(<4 x i32> %vec, ptr %out) {
; CHECK-LABEL: no_combine_for_non_bool_truncate ; CHECK-LABEL: no_combine_for_non_bool_truncate:
; CHECK: xtn.4h v0, v0 ; CHECK: ; %bb.0:
; CHECK-NOT: addv.4s s0, v0 ; CHECK-NEXT: xtn.4h v0, v0
; CHECK-NEXT: xtn.8b v0, v0
; CHECK-NEXT: str s0, [x0]
; CHECK-NEXT: ret
%trunc = trunc <4 x i32> %vec to <4 x i8> %trunc = trunc <4 x i32> %vec to <4 x i8>
store <4 x i8> %trunc, ptr %out store <4 x i8> %trunc, ptr %out
Context not available.
} }
define void @no_combine_for_build_vector(i1 %a, i1 %b, i1 %c, i1 %d, ptr %out) { define void @no_combine_for_build_vector(i1 %a, i1 %b, i1 %c, i1 %d, ptr %out) {
; CHECK-LABEL: no_combine_for_build_vector ; CHECK-LABEL: no_combine_for_build_vector:
; CHECK-NOT: addv ; CHECK: ; %bb.0:
; CHECK-NEXT: orr w8, w0, w1, lsl #1
; CHECK-NEXT: orr w8, w8, w2, lsl #2
; CHECK-NEXT: orr w8, w8, w3, lsl #3
; CHECK-NEXT: strb w8, [x4]
; CHECK-NEXT: ret
%1 = insertelement <4 x i1> undef, i1 %a, i64 0 %1 = insertelement <4 x i1> undef, i1 %a, i64 0
%2 = insertelement <4 x i1> %1, i1 %b, i64 1 %2 = insertelement <4 x i1> %1, i1 %b, i64 1
Context not available.

llvm/test/CodeGen/AArch64/vec-libcalls.ll

llvm/test/CodeGen/AArch64/vec_cttz.ll

llvm/test/CodeGen/AArch64/vec_uaddo.ll

llvm/test/CodeGen/AArch64/vec_umulo.ll

llvm/test/CodeGen/AArch64/vecreduce-add.ll

llvm/test/CodeGen/AArch64/vecreduce-and-legalization.ll

llvm/test/CodeGen/AArch64/vecreduce-fadd-legalization-strict.ll

llvm/test/CodeGen/AArch64/vecreduce-fadd.ll

llvm/test/CodeGen/AArch64/vecreduce-fmax-legalization.ll

llvm/test/CodeGen/AArch64/vecreduce-fmaximum.ll

llvm/test/CodeGen/AArch64/vecreduce-fmin-legalization.ll

llvm/test/CodeGen/AArch64/vecreduce-fminimum.ll

llvm/test/CodeGen/AArch64/vector-fcopysign.ll

llvm/test/CodeGen/AArch64/vector-fcvt.ll

llvm/test/CodeGen/AArch64/vector-gep.ll

llvm/test/CodeGen/AArch64/vldn_shuffle.ll

llvm/test/CodeGen/AArch64/vselect-constants.ll

llvm/test/CodeGen/AArch64/vselect-ext.ll

llvm/test/CodeGen/AArch64/wide-scalar-shift-by-byte-multiple-legalization.ll

llvm/test/CodeGen/AArch64/wide-scalar-shift-legalization.ll

llvm/test/CodeGen/AArch64/win64_vararg.ll

llvm/test/CodeGen/AArch64/win64_vararg_float.ll

llvm/test/CodeGen/AArch64/win64_vararg_float_cc.ll

llvm/test/CodeGen/AArch64/wineh-bti.ll

Context not available.
; CHECK: .LBB0_2: ; CHECK: .LBB0_2:
; CHECK-NEXT: hint #36 ; CHECK-NEXT: hint #36
; CHECK-NEXT: mov w0, #1 ; CHECK: mov w0, #1
; CHECK: .LBB0_3: ; CHECK: .LBB0_3:
; CHECK-NEXT: hint #36 ; CHECK-NEXT: hint #36
Context not available.

llvm/test/CodeGen/AArch64/zero-call-used-regs.ll

llvm/test/CodeGen/AArch64/zext-to-tbl.ll

Context not available.
; RUN: llc -mtriple=arm64-apple-ios -mattr=+sve -o - %s | FileCheck %s ; RUN: llc -mtriple=arm64-apple-ios -mattr=+sve -o - %s | FileCheck %s
; RUN: llc -mtriple=aarch64_be-unknown-linux -mattr=+sve -o - %s | FileCheck --check-prefix=CHECK-BE %s ; RUN: llc -mtriple=aarch64_be-unknown-linux -mattr=+sve -o - %s | FileCheck --check-prefix=CHECK-BE %s
; CHECK-LABEL: lCPI0_0:
dmgreenUnsubmitted
Done
ReplyInline Actions

These too.

dmgreen: These too.
; CHECK-NEXT: .byte 0 ; 0x0
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 1 ; 0x1
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 2 ; 0x2
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 3 ; 0x3
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT:lCPI0_1:
; CHECK-NEXT: .byte 4 ; 0x4
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 5 ; 0x5
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 6 ; 0x6
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 7 ; 0x7
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT:lCPI0_2:
; CHECK-NEXT: .byte 8 ; 0x8
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 9 ; 0x9
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 10 ; 0xa
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 11 ; 0xb
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT:lCPI0_3:
; CHECK-NEXT: .byte 12 ; 0xc
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 13 ; 0xd
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 14 ; 0xe
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 15 ; 0xf
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-BE: .LCPI0_0:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 0 // 0x0
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 1 // 0x1
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 2 // 0x2
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 3 // 0x3
; CHECK-BE-NEXT: .LCPI0_1:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 4 // 0x4
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 5 // 0x5
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 6 // 0x6
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 7 // 0x7
; CHECK-BE-NEXT: .LCPI0_2:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 8 // 0x8
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 9 // 0x9
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 10 // 0xa
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 11 // 0xb
; CHECK-BE-NEXT: .LCPI0_3:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 12 // 0xc
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 13 // 0xd
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 14 // 0xe
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 15 // 0xf
; It's profitable to convert the zext to a shuffle, which in turn will be ; It's profitable to convert the zext to a shuffle, which in turn will be
; lowered to 4 tbl instructions. The masks are materialized outside the loop. ; lowered to 4 tbl instructions. The masks are materialized outside the loop.
define void @zext_v16i8_to_v16i32_in_loop(ptr %src, ptr %dst) { define void @zext_v16i8_to_v16i32_in_loop(ptr %src, ptr %dst) {
; CHECK-LABEL: zext_v16i8_to_v16i32_in_loop: ; CHECK-LABEL: zext_v16i8_to_v16i32_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: Lloh0: ; CHECK-NEXT: Lloh0:
; CHECK-NEXT: adrp x9, lCPI0_0@PAGE ; CHECK-NEXT: adrp x8, lCPI0_0@PAGE
; CHECK-NEXT: Lloh1: ; CHECK-NEXT: Lloh1:
; CHECK-NEXT: adrp x10, lCPI0_1@PAGE ; CHECK-NEXT: adrp x9, lCPI0_1@PAGE
; CHECK-NEXT: Lloh2: ; CHECK-NEXT: Lloh2:
; CHECK-NEXT: adrp x11, lCPI0_2@PAGE ; CHECK-NEXT: adrp x10, lCPI0_2@PAGE
; CHECK-NEXT: Lloh3: ; CHECK-NEXT: Lloh3:
; CHECK-NEXT: adrp x12, lCPI0_3@PAGE ; CHECK-NEXT: ldr q0, [x8, lCPI0_0@PAGEOFF]
; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: Lloh4: ; CHECK-NEXT: Lloh4:
; CHECK-NEXT: ldr q0, [x9, lCPI0_0@PAGEOFF] ; CHECK-NEXT: adrp x8, lCPI0_3@PAGE
; CHECK-NEXT: Lloh5: ; CHECK-NEXT: Lloh5:
; CHECK-NEXT: ldr q1, [x10, lCPI0_1@PAGEOFF] ; CHECK-NEXT: ldr q1, [x9, lCPI0_1@PAGEOFF]
; CHECK-NEXT: Lloh6: ; CHECK-NEXT: Lloh6:
; CHECK-NEXT: ldr q2, [x11, lCPI0_2@PAGEOFF] ; CHECK-NEXT: ldr q2, [x10, lCPI0_2@PAGEOFF]
; CHECK-NEXT: Lloh7: ; CHECK-NEXT: Lloh7:
; CHECK-NEXT: ldr q3, [x12, lCPI0_3@PAGEOFF] ; CHECK-NEXT: ldr q3, [x8, lCPI0_3@PAGEOFF]
; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: LBB0_1: ; %loop ; CHECK-NEXT: LBB0_1: ; %loop
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldr q4, [x0, x8] ; CHECK-NEXT: ldr q4, [x0, x8]
Context not available.
; CHECK-NEXT: b.ne LBB0_1 ; CHECK-NEXT: b.ne LBB0_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh3, Lloh7 ; CHECK-NEXT: .loh AdrpLdr Lloh4, Lloh7
; CHECK-NEXT: .loh AdrpLdr Lloh2, Lloh6 ; CHECK-NEXT: .loh AdrpLdr Lloh2, Lloh6
; CHECK-NEXT: .loh AdrpLdr Lloh1, Lloh5 ; CHECK-NEXT: .loh AdrpLdr Lloh1, Lloh5
; CHECK-NEXT: .loh AdrpLdr Lloh0, Lloh4 ; CHECK-NEXT: .loh AdrpAdrp Lloh0, Lloh4
; CHECK-NEXT: .loh AdrpLdr Lloh0, Lloh3
; ;
; CHECK-BE-LABEL: zext_v16i8_to_v16i32_in_loop: ; CHECK-BE-LABEL: zext_v16i8_to_v16i32_in_loop:
; CHECK-BE: // %bb.0: // %entry ; CHECK-BE: // %bb.0: // %entry
Context not available.
; CHECK-BE-NEXT: .LBB0_1: // %loop ; CHECK-BE-NEXT: .LBB0_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x10, x1, #32
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v4.16b }, [x9] ; CHECK-BE-NEXT: ld1 { v4.16b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #48 ; CHECK-BE-NEXT: add x9, x1, #48
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: tbl v5.16b, { v4.16b }, v3.16b ; CHECK-BE-NEXT: tbl v5.16b, { v4.16b }, v3.16b
; CHECK-BE-NEXT: tbl v6.16b, { v4.16b }, v0.16b ; CHECK-BE-NEXT: tbl v6.16b, { v4.16b }, v2.16b
; CHECK-BE-NEXT: tbl v7.16b, { v4.16b }, v2.16b ; CHECK-BE-NEXT: tbl v7.16b, { v4.16b }, v1.16b
; CHECK-BE-NEXT: tbl v4.16b, { v4.16b }, v1.16b ; CHECK-BE-NEXT: tbl v4.16b, { v4.16b }, v0.16b
; CHECK-BE-NEXT: st1 { v5.16b }, [x9] ; CHECK-BE-NEXT: st1 { v5.16b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #32
; CHECK-BE-NEXT: st1 { v6.16b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #16 ; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: st1 { v6.16b }, [x1] ; CHECK-BE-NEXT: st1 { v4.16b }, [x1]
; CHECK-BE-NEXT: add x1, x1, #64 ; CHECK-BE-NEXT: add x1, x1, #64
; CHECK-BE-NEXT: st1 { v7.16b }, [x10] ; CHECK-BE-NEXT: st1 { v7.16b }, [x9]
; CHECK-BE-NEXT: st1 { v4.16b }, [x9]
; CHECK-BE-NEXT: b.ne .LBB0_1 ; CHECK-BE-NEXT: b.ne .LBB0_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v16i8_to_v16i32_in_loop_not_header(ptr %src, ptr %dst, i1 %c) { define void @zext_v16i8_to_v16i32_in_loop_not_header(ptr %src, ptr %dst, i1 %c) {
;
; CHECK-LABEL: zext_v16i8_to_v16i32_in_loop_not_header: ; CHECK-LABEL: zext_v16i8_to_v16i32_in_loop_not_header:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-BE-NEXT: // in Loop: Header=BB1_2 Depth=1 ; CHECK-BE-NEXT: // in Loop: Header=BB1_2 Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x10, x1, #32 ; CHECK-BE-NEXT: add x10, x1, #32
; CHECK-BE-NEXT: add x11, x1, #16
; CHECK-BE-NEXT: ld1 { v0.16b }, [x9] ; CHECK-BE-NEXT: ld1 { v0.16b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #48 ; CHECK-BE-NEXT: add x9, x1, #48
; CHECK-BE-NEXT: ushll2 v1.8h, v0.16b, #0 ; CHECK-BE-NEXT: ushll2 v1.8h, v0.16b, #0
Context not available.
; CHECK-BE-NEXT: ushll2 v2.4s, v1.8h, #0 ; CHECK-BE-NEXT: ushll2 v2.4s, v1.8h, #0
; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0 ; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0
; CHECK-BE-NEXT: ushll2 v3.4s, v0.8h, #0 ; CHECK-BE-NEXT: ushll2 v3.4s, v0.8h, #0
; CHECK-BE-NEXT: st1 { v2.4s }, [x9]
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0 ; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: st1 { v2.4s }, [x9]
; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: st1 { v1.4s }, [x10] ; CHECK-BE-NEXT: st1 { v1.4s }, [x10]
; CHECK-BE-NEXT: st1 { v3.4s }, [x11] ; CHECK-BE-NEXT: st1 { v3.4s }, [x9]
; CHECK-BE-NEXT: st1 { v0.4s }, [x1] ; CHECK-BE-NEXT: st1 { v0.4s }, [x1]
; CHECK-BE-NEXT: b .LBB1_1 ; CHECK-BE-NEXT: b .LBB1_1
; CHECK-BE-NEXT: .LBB1_4: // %exit ; CHECK-BE-NEXT: .LBB1_4: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop.latch ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop.latch ]
br i1 %c, label %then, label %loop.latch br i1 %c, label %then, label %loop.latch
then: then:
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
%load = load <16 x i8>, ptr %src.gep %load = load <16 x i8>, ptr %src.gep
Context not available.
%dst.gep = getelementptr i32, ptr %dst, i64 %iv %dst.gep = getelementptr i32, ptr %dst, i64 %iv
store <16 x i32> %ext, ptr %dst.gep store <16 x i32> %ext, ptr %dst.gep
br label %loop.latch br label %loop.latch
loop.latch: loop.latch:
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
; Not profitable to use shuffle/tbl, as 4 tbls + materializing the masks ; Not profitable to use shuffle/tbl, as 4 tbls + materializing the masks
; require more instructions than lowering zext directly. ; require more instructions than lowering zext directly.
define void @zext_v16i8_to_v16i32_no_loop(ptr %src, ptr %dst) { define void @zext_v16i8_to_v16i32_no_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v16i8_to_v16i32_no_loop: ; CHECK-LABEL: zext_v16i8_to_v16i32_no_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: ldr q0, [x0] ; CHECK-NEXT: ldr q0, [x0]
Context not available.
; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0 ; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-BE-NEXT: ushll2 v2.4s, v1.8h, #0 ; CHECK-BE-NEXT: ushll2 v2.4s, v1.8h, #0
; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0 ; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0
; CHECK-BE-NEXT: ushll2 v3.4s, v0.8h, #0
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: st1 { v2.4s }, [x8] ; CHECK-BE-NEXT: st1 { v2.4s }, [x8]
; CHECK-BE-NEXT: add x8, x1, #32 ; CHECK-BE-NEXT: add x8, x1, #32
; CHECK-BE-NEXT: st1 { v1.4s }, [x8] ; CHECK-BE-NEXT: st1 { v1.4s }, [x8]
; CHECK-BE-NEXT: add x8, x1, #16 ; CHECK-BE-NEXT: add x8, x1, #16
; CHECK-BE-NEXT: ushll2 v1.4s, v0.8h, #0 ; CHECK-BE-NEXT: st1 { v3.4s }, [x8]
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: st1 { v1.4s }, [x8]
; CHECK-BE-NEXT: st1 { v0.4s }, [x1] ; CHECK-BE-NEXT: st1 { v0.4s }, [x1]
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
Context not available.
store <16 x i32> %ext, ptr %dst store <16 x i32> %ext, ptr %dst
ret void ret void
} }
; Avoid using tbl when optimizing for size. ; Avoid using tbl when optimizing for size.
define void @zext_v16i8_to_v16i32_in_loop_optsize(ptr %src, ptr %dst) optsize { define void @zext_v16i8_to_v16i32_in_loop_optsize(ptr %src, ptr %dst) optsize {
;
; CHECK-LABEL: zext_v16i8_to_v16i32_in_loop_optsize: ; CHECK-LABEL: zext_v16i8_to_v16i32_in_loop_optsize:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-BE-NEXT: .LBB3_1: // %loop ; CHECK-BE-NEXT: .LBB3_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x10, x1, #32
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v0.16b }, [x9] ; CHECK-BE-NEXT: ld1 { v0.16b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #48 ; CHECK-BE-NEXT: add x9, x1, #48
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ushll2 v1.8h, v0.16b, #0 ; CHECK-BE-NEXT: ushll2 v1.8h, v0.16b, #0
; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0 ; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-BE-NEXT: ushll2 v2.4s, v1.8h, #0 ; CHECK-BE-NEXT: ushll2 v2.4s, v1.8h, #0
; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0 ; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0
; CHECK-BE-NEXT: ushll2 v3.4s, v0.8h, #0
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: st1 { v2.4s }, [x9] ; CHECK-BE-NEXT: st1 { v2.4s }, [x9]
; CHECK-BE-NEXT: add x9, x1, #32
; CHECK-BE-NEXT: st1 { v1.4s }, [x9]
; CHECK-BE-NEXT: add x9, x1, #16 ; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: ushll v2.4s, v0.4h, #0 ; CHECK-BE-NEXT: st1 { v0.4s }, [x1]
; CHECK-BE-NEXT: st1 { v1.4s }, [x10]
; CHECK-BE-NEXT: ushll2 v0.4s, v0.8h, #0
; CHECK-BE-NEXT: st1 { v2.4s }, [x1]
; CHECK-BE-NEXT: add x1, x1, #64 ; CHECK-BE-NEXT: add x1, x1, #64
; CHECK-BE-NEXT: st1 { v0.4s }, [x9] ; CHECK-BE-NEXT: st1 { v3.4s }, [x9]
; CHECK-BE-NEXT: b.ne .LBB3_1 ; CHECK-BE-NEXT: b.ne .LBB3_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
; Avoid using tbl when optimizing for size. ; Avoid using tbl when optimizing for size.
define void @zext_v16i8_to_v16i32_in_loop_minsize(ptr %src, ptr %dst) minsize { define void @zext_v16i8_to_v16i32_in_loop_minsize(ptr %src, ptr %dst) minsize {
;
; CHECK-LABEL: zext_v16i8_to_v16i32_in_loop_minsize: ; CHECK-LABEL: zext_v16i8_to_v16i32_in_loop_minsize:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-BE-NEXT: .LBB4_1: // %loop ; CHECK-BE-NEXT: .LBB4_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x10, x1, #32
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v0.16b }, [x9] ; CHECK-BE-NEXT: ld1 { v0.16b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #48 ; CHECK-BE-NEXT: add x9, x1, #48
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ushll2 v1.8h, v0.16b, #0 ; CHECK-BE-NEXT: ushll2 v1.8h, v0.16b, #0
; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0 ; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-BE-NEXT: ushll2 v2.4s, v1.8h, #0 ; CHECK-BE-NEXT: ushll2 v2.4s, v1.8h, #0
; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0 ; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0
; CHECK-BE-NEXT: ushll2 v3.4s, v0.8h, #0
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: st1 { v2.4s }, [x9] ; CHECK-BE-NEXT: st1 { v2.4s }, [x9]
; CHECK-BE-NEXT: add x9, x1, #32
; CHECK-BE-NEXT: st1 { v1.4s }, [x9]
; CHECK-BE-NEXT: add x9, x1, #16 ; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: ushll v2.4s, v0.4h, #0 ; CHECK-BE-NEXT: st1 { v0.4s }, [x1]
; CHECK-BE-NEXT: st1 { v1.4s }, [x10]
; CHECK-BE-NEXT: ushll2 v0.4s, v0.8h, #0
; CHECK-BE-NEXT: st1 { v2.4s }, [x1]
; CHECK-BE-NEXT: add x1, x1, #64 ; CHECK-BE-NEXT: add x1, x1, #64
; CHECK-BE-NEXT: st1 { v0.4s }, [x9] ; CHECK-BE-NEXT: st1 { v3.4s }, [x9]
; CHECK-BE-NEXT: b.ne .LBB4_1 ; CHECK-BE-NEXT: b.ne .LBB4_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v16i8_to_v16i16_in_loop(ptr %src, ptr %dst) { define void @zext_v16i8_to_v16i16_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v16i8_to_v16i16_in_loop: ; CHECK-LABEL: zext_v16i8_to_v16i16_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v0.16b }, [x9] ; CHECK-BE-NEXT: ld1 { v0.16b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #16 ; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: ushll v1.8h, v0.8b, #0 ; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ushll2 v0.8h, v0.16b, #0 ; CHECK-BE-NEXT: ushll2 v1.8h, v0.16b, #0
; CHECK-BE-NEXT: st1 { v1.8h }, [x1] ; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-BE-NEXT: st1 { v0.8h }, [x1]
; CHECK-BE-NEXT: add x1, x1, #32 ; CHECK-BE-NEXT: add x1, x1, #32
; CHECK-BE-NEXT: st1 { v0.8h }, [x9] ; CHECK-BE-NEXT: st1 { v1.8h }, [x9]
; CHECK-BE-NEXT: b.ne .LBB5_1 ; CHECK-BE-NEXT: b.ne .LBB5_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
; CHECK-LABEL: lCPI6_0:
; CHECK-NEXT: .byte 0 ; 0x0
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 1 ; 0x1
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 2 ; 0x2
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 3 ; 0x3
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: lCPI6_1:
; CHECK-NEXT: .byte 4 ; 0x4
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 5 ; 0x5
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 6 ; 0x6
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 7 ; 0x7
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-BE: .LCPI6_0:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 0 // 0x0
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 1 // 0x1
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 2 // 0x2
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 3 // 0x3
; CHECK-BE-NEXT: .LCPI6_1:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 4 // 0x4
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 5 // 0x5
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 6 // 0x6
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 7 // 0x7
define void @zext_v8i8_to_v8i32_in_loop(ptr %src, ptr %dst) { define void @zext_v8i8_to_v8i32_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v8i8_to_v8i32_in_loop: ; CHECK-LABEL: zext_v8i8_to_v8i32_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: Lloh8: ; CHECK-NEXT: Lloh8:
; CHECK-NEXT: adrp x9, lCPI6_0@PAGE ; CHECK-NEXT: adrp x8, lCPI6_0@PAGE
; CHECK-NEXT: Lloh9: ; CHECK-NEXT: Lloh9:
; CHECK-NEXT: adrp x10, lCPI6_1@PAGE ; CHECK-NEXT: adrp x9, lCPI6_1@PAGE
; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: Lloh10: ; CHECK-NEXT: Lloh10:
; CHECK-NEXT: ldr q0, [x9, lCPI6_0@PAGEOFF] ; CHECK-NEXT: ldr q0, [x8, lCPI6_0@PAGEOFF]
; CHECK-NEXT: Lloh11: ; CHECK-NEXT: Lloh11:
; CHECK-NEXT: ldr q1, [x10, lCPI6_1@PAGEOFF] ; CHECK-NEXT: ldr q1, [x9, lCPI6_1@PAGEOFF]
; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: LBB6_1: ; %loop ; CHECK-NEXT: LBB6_1: ; %loop
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldr d2, [x0, x8] ; CHECK-NEXT: ldr d2, [x0, x8]
Context not available.
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v2.8b }, [x9] ; CHECK-BE-NEXT: ld1 { v2.8b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #16 ; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: tbl v3.16b, { v2.16b }, v0.16b ; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: tbl v2.16b, { v2.16b }, v1.16b ; CHECK-BE-NEXT: tbl v3.16b, { v2.16b }, v1.16b
; CHECK-BE-NEXT: st1 { v3.16b }, [x1] ; CHECK-BE-NEXT: tbl v2.16b, { v2.16b }, v0.16b
; CHECK-BE-NEXT: st1 { v2.16b }, [x1]
; CHECK-BE-NEXT: add x1, x1, #64 ; CHECK-BE-NEXT: add x1, x1, #64
; CHECK-BE-NEXT: st1 { v2.16b }, [x9] ; CHECK-BE-NEXT: st1 { v3.16b }, [x9]
; CHECK-BE-NEXT: b.ne .LBB6_1 ; CHECK-BE-NEXT: b.ne .LBB6_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v16i8_to_v16i64_in_loop(ptr %src, ptr %dst) { define void @zext_v16i8_to_v16i64_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v16i8_to_v16i64_in_loop: ; CHECK-LABEL: zext_v16i8_to_v16i64_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-NEXT: ldr q0, [x0, x8] ; CHECK-NEXT: ldr q0, [x0, x8]
; CHECK-NEXT: add x8, x8, #16 ; CHECK-NEXT: add x8, x8, #16
; CHECK-NEXT: cmp x8, #128 ; CHECK-NEXT: cmp x8, #128
; CHECK-NEXT: ushll.8h v1, v0, #0 ; CHECK-NEXT: ushll2.8h v1, v0, #0
; CHECK-NEXT: ushll2.8h v0, v0, #0 ; CHECK-NEXT: ushll.8h v0, v0, #0
; CHECK-NEXT: ushll2.4s v2, v1, #0 ; CHECK-NEXT: ushll2.4s v2, v1, #0
; CHECK-NEXT: ushll2.4s v3, v0, #0 ; CHECK-NEXT: ushll.4s v1, v1, #0
; CHECK-NEXT: ushll2.4s v4, v0, #0
; CHECK-NEXT: ushll.4s v0, v0, #0 ; CHECK-NEXT: ushll.4s v0, v0, #0
; CHECK-NEXT: ushll2.2d v4, v3, #0
; CHECK-NEXT: ushll2.2d v5, v0, #0
; CHECK-NEXT: ushll.2d v0, v0, #0
; CHECK-NEXT: ushll.2d v3, v3, #0
; CHECK-NEXT: stp q0, q5, [x1, #64]
; CHECK-NEXT: ushll.4s v0, v1, #0
; CHECK-NEXT: stp q3, q4, [x1, #96]
; CHECK-NEXT: ushll2.2d v3, v2, #0 ; CHECK-NEXT: ushll2.2d v3, v2, #0
; CHECK-NEXT: ushll.2d v2, v2, #0 ; CHECK-NEXT: ushll.2d v2, v2, #0
; CHECK-NEXT: ushll2.2d v1, v0, #0 ; CHECK-NEXT: ushll2.2d v5, v1, #0
; CHECK-NEXT: ushll.2d v1, v1, #0
; CHECK-NEXT: stp q2, q3, [x1, #96]
; CHECK-NEXT: ushll2.2d v3, v4, #0
; CHECK-NEXT: ushll.2d v2, v4, #0
; CHECK-NEXT: ushll2.2d v4, v0, #0
; CHECK-NEXT: ushll.2d v0, v0, #0 ; CHECK-NEXT: ushll.2d v0, v0, #0
; CHECK-NEXT: stp q1, q5, [x1, #64]
; CHECK-NEXT: stp q2, q3, [x1, #32] ; CHECK-NEXT: stp q2, q3, [x1, #32]
; CHECK-NEXT: stp q0, q1, [x1], #128 ; CHECK-NEXT: stp q0, q4, [x1], #128
; CHECK-NEXT: b.ne LBB7_1 ; CHECK-NEXT: b.ne LBB7_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: ret ; CHECK-NEXT: ret
Context not available.
; CHECK-BE-NEXT: .LBB7_1: // %loop ; CHECK-BE-NEXT: .LBB7_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x10, x1, #96
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v0.16b }, [x9] ; CHECK-BE-NEXT: ld1 { v0.16b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #112 ; CHECK-BE-NEXT: add x9, x1, #112
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ushll2 v1.8h, v0.16b, #0 ; CHECK-BE-NEXT: ushll2 v1.8h, v0.16b, #0
; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0 ; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-BE-NEXT: ushll2 v2.4s, v1.8h, #0 ; CHECK-BE-NEXT: ushll2 v2.4s, v1.8h, #0
; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0 ; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0
; CHECK-BE-NEXT: ushll2 v4.4s, v0.8h, #0
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: ushll2 v3.2d, v2.4s, #0 ; CHECK-BE-NEXT: ushll2 v3.2d, v2.4s, #0
; CHECK-BE-NEXT: ushll v2.2d, v2.2s, #0 ; CHECK-BE-NEXT: ushll v2.2d, v2.2s, #0
; CHECK-BE-NEXT: ushll2 v5.2d, v1.4s, #0
; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0
; CHECK-BE-NEXT: st1 { v3.2d }, [x9] ; CHECK-BE-NEXT: st1 { v3.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #96
; CHECK-BE-NEXT: ushll2 v3.2d, v4.4s, #0
; CHECK-BE-NEXT: st1 { v2.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #80 ; CHECK-BE-NEXT: add x9, x1, #80
; CHECK-BE-NEXT: ushll2 v3.2d, v1.4s, #0 ; CHECK-BE-NEXT: ushll v2.2d, v4.2s, #0
; CHECK-BE-NEXT: st1 { v2.2d }, [x10] ; CHECK-BE-NEXT: st1 { v5.2d }, [x9]
; CHECK-BE-NEXT: ushll2 v2.4s, v0.8h, #0 ; CHECK-BE-NEXT: add x9, x1, #48
; CHECK-BE-NEXT: add x10, x1, #48
; CHECK-BE-NEXT: st1 { v3.2d }, [x9] ; CHECK-BE-NEXT: st1 { v3.2d }, [x9]
; CHECK-BE-NEXT: ushll2 v3.2d, v0.4s, #0
; CHECK-BE-NEXT: ushll v0.2d, v0.2s, #0
; CHECK-BE-NEXT: add x9, x1, #64 ; CHECK-BE-NEXT: add x9, x1, #64
; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: ushll2 v4.2d, v2.4s, #0
; CHECK-BE-NEXT: st1 { v1.2d }, [x9] ; CHECK-BE-NEXT: st1 { v1.2d }, [x9]
; CHECK-BE-NEXT: ushll v1.2d, v0.2s, #0 ; CHECK-BE-NEXT: add x9, x1, #32
; CHECK-BE-NEXT: st1 { v2.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #16 ; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: st1 { v4.2d }, [x10] ; CHECK-BE-NEXT: st1 { v0.2d }, [x1]
; CHECK-BE-NEXT: add x10, x1, #32
; CHECK-BE-NEXT: st1 { v1.2d }, [x1]
; CHECK-BE-NEXT: add x1, x1, #128 ; CHECK-BE-NEXT: add x1, x1, #128
; CHECK-BE-NEXT: ushll2 v0.2d, v0.4s, #0 ; CHECK-BE-NEXT: st1 { v3.2d }, [x9]
; CHECK-BE-NEXT: ushll v2.2d, v2.2s, #0
; CHECK-BE-NEXT: st1 { v0.2d }, [x9]
; CHECK-BE-NEXT: st1 { v2.2d }, [x10]
; CHECK-BE-NEXT: b.ne .LBB7_1 ; CHECK-BE-NEXT: b.ne .LBB7_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v8i8_to_v8i64_in_loop(ptr %src, ptr %dst) { define void @zext_v8i8_to_v8i64_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v8i8_to_v8i64_in_loop: ; CHECK-LABEL: zext_v8i8_to_v8i64_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-BE-NEXT: .LBB8_1: // %loop ; CHECK-BE-NEXT: .LBB8_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x10, x1, #32
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v0.8b }, [x9] ; CHECK-BE-NEXT: ld1 { v0.8b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #48 ; CHECK-BE-NEXT: add x9, x1, #48
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0 ; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-BE-NEXT: ushll2 v1.4s, v0.8h, #0 ; CHECK-BE-NEXT: ushll2 v1.4s, v0.8h, #0
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0 ; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: ushll2 v2.2d, v1.4s, #0 ; CHECK-BE-NEXT: ushll2 v2.2d, v1.4s, #0
; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0 ; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0
; CHECK-BE-NEXT: ushll2 v3.2d, v0.4s, #0
; CHECK-BE-NEXT: ushll v0.2d, v0.2s, #0
; CHECK-BE-NEXT: st1 { v2.2d }, [x9] ; CHECK-BE-NEXT: st1 { v2.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #32
; CHECK-BE-NEXT: st1 { v1.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #16 ; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: ushll v2.2d, v0.2s, #0 ; CHECK-BE-NEXT: st1 { v0.2d }, [x1]
; CHECK-BE-NEXT: st1 { v1.2d }, [x10]
; CHECK-BE-NEXT: ushll2 v0.2d, v0.4s, #0
; CHECK-BE-NEXT: st1 { v2.2d }, [x1]
; CHECK-BE-NEXT: add x1, x1, #128 ; CHECK-BE-NEXT: add x1, x1, #128
; CHECK-BE-NEXT: st1 { v0.2d }, [x9] ; CHECK-BE-NEXT: st1 { v3.2d }, [x9]
; CHECK-BE-NEXT: b.ne .LBB8_1 ; CHECK-BE-NEXT: b.ne .LBB8_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v8i8_to_v8i16_in_loop(ptr %src, ptr %dst) { define void @zext_v8i8_to_v8i16_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v8i8_to_v8i16_in_loop: ; CHECK-LABEL: zext_v8i8_to_v8i16_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v0.8b }, [x9] ; CHECK-BE-NEXT: ld1 { v0.8b }, [x9]
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0 ; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-BE-NEXT: st1 { v0.8h }, [x1] ; CHECK-BE-NEXT: st1 { v0.8h }, [x1]
; CHECK-BE-NEXT: add x1, x1, #32 ; CHECK-BE-NEXT: add x1, x1, #32
; CHECK-BE-NEXT: b.ne .LBB9_1 ; CHECK-BE-NEXT: b.ne .LBB9_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v8i8_to_v8i20_in_loop(ptr %src, ptr %dst) { define void @zext_v8i8_to_v8i20_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v8i8_to_v8i20_in_loop: ; CHECK-LABEL: zext_v8i8_to_v8i20_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-NEXT: ushll.8h v0, v0, #0 ; CHECK-NEXT: ushll.8h v0, v0, #0
; CHECK-NEXT: ushll2.4s v1, v0, #0 ; CHECK-NEXT: ushll2.4s v1, v0, #0
; CHECK-NEXT: ushll.4s v0, v0, #0 ; CHECK-NEXT: ushll.4s v0, v0, #0
; CHECK-NEXT: mov.s w11, v1[1] ; CHECK-NEXT: mov.s w9, v1[1]
; CHECK-NEXT: mov.s w13, v0[1] ; CHECK-NEXT: mov.s w11, v0[1]
; CHECK-NEXT: fmov w12, s1 ; CHECK-NEXT: fmov w10, s1
; CHECK-NEXT: mov.s w14, v1[2] ; CHECK-NEXT: fmov w14, s0
; CHECK-NEXT: fmov w15, s0 ; CHECK-NEXT: mov.s w13, v1[2]
; CHECK-NEXT: mov.s w16, v0[2] ; CHECK-NEXT: mov.s w16, v0[2]
; CHECK-NEXT: mov.s w9, v1[3] ; CHECK-NEXT: mov.s w12, v1[3]
; CHECK-NEXT: mov.s w10, v0[3] ; CHECK-NEXT: mov.s w15, v0[3]
; CHECK-NEXT: orr x11, x12, x11, lsl #20 ; CHECK-NEXT: orr x9, x10, x9, lsl #20
; CHECK-NEXT: orr x12, x15, x13, lsl #20 ; CHECK-NEXT: orr x10, x14, x11, lsl #20
; CHECK-NEXT: orr x11, x11, x14, lsl #40 ; CHECK-NEXT: orr x9, x9, x13, lsl #40
; CHECK-NEXT: orr x12, x12, x16, lsl #40 ; CHECK-NEXT: orr x10, x10, x16, lsl #40
; CHECK-NEXT: lsr w13, w9, #4 ; CHECK-NEXT: lsr w11, w12, #4
; CHECK-NEXT: lsr w14, w10, #4 ; CHECK-NEXT: lsr w13, w15, #4
; CHECK-NEXT: orr x9, x11, x9, lsl #60 ; CHECK-NEXT: orr x9, x9, x12, lsl #60
; CHECK-NEXT: orr x10, x12, x10, lsl #60 ; CHECK-NEXT: orr x10, x10, x15, lsl #60
; CHECK-NEXT: strh w13, [x1, #18] ; CHECK-NEXT: strh w11, [x1, #18]
; CHECK-NEXT: strh w14, [x1, #8] ; CHECK-NEXT: strh w13, [x1, #8]
; CHECK-NEXT: stur x9, [x1, #10] ; CHECK-NEXT: stur x9, [x1, #10]
; CHECK-NEXT: str x10, [x1], #64 ; CHECK-NEXT: str x10, [x1], #64
; CHECK-NEXT: b.ne LBB10_1 ; CHECK-NEXT: b.ne LBB10_1
Context not available.
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v0.8b }, [x9] ; CHECK-BE-NEXT: ld1 { v0.8b }, [x9]
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0 ; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-BE-NEXT: ushll2 v1.4s, v0.8h, #0 ; CHECK-BE-NEXT: ushll2 v1.4s, v0.8h, #0
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0 ; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: mov w9, v1.s[1] ; CHECK-BE-NEXT: mov w9, v1.s[1]
; CHECK-BE-NEXT: mov w11, v1.s[2] ; CHECK-BE-NEXT: mov w10, v0.s[1]
; CHECK-BE-NEXT: fmov w12, s1 ; CHECK-BE-NEXT: fmov w12, s1
; CHECK-BE-NEXT: mov w13, v0.s[1] ; CHECK-BE-NEXT: fmov w14, s0
; CHECK-BE-NEXT: mov w15, v0.s[2] ; CHECK-BE-NEXT: mov w11, v1.s[2]
; CHECK-BE-NEXT: mov w10, v1.s[3] ; CHECK-BE-NEXT: mov w13, v0.s[2]
; CHECK-BE-NEXT: mov w14, v0.s[3] ; CHECK-BE-NEXT: mov w15, v1.s[3]
; CHECK-BE-NEXT: lsl x9, x9, #40 ; CHECK-BE-NEXT: lsl x9, x9, #40
; CHECK-BE-NEXT: lsl x10, x10, #40
; CHECK-BE-NEXT: orr x9, x9, x12, lsl #60 ; CHECK-BE-NEXT: orr x9, x9, x12, lsl #60
; CHECK-BE-NEXT: orr x10, x10, x14, lsl #60
; CHECK-BE-NEXT: lsr x12, x12, #4 ; CHECK-BE-NEXT: lsr x12, x12, #4
; CHECK-BE-NEXT: strh w15, [x1, #18]
; CHECK-BE-NEXT: orr x9, x9, x11, lsl #20 ; CHECK-BE-NEXT: orr x9, x9, x11, lsl #20
; CHECK-BE-NEXT: fmov w11, s0 ; CHECK-BE-NEXT: orr x10, x10, x13, lsl #20
; CHECK-BE-NEXT: lsl x13, x13, #40 ; CHECK-BE-NEXT: mov w11, v0.s[3]
; CHECK-BE-NEXT: lsr x13, x14, #4
; CHECK-BE-NEXT: lsr x9, x9, #16 ; CHECK-BE-NEXT: lsr x9, x9, #16
; CHECK-BE-NEXT: lsr x10, x10, #16
; CHECK-BE-NEXT: bfi x9, x12, #48, #4 ; CHECK-BE-NEXT: bfi x9, x12, #48, #4
; CHECK-BE-NEXT: strh w10, [x1, #18] ; CHECK-BE-NEXT: bfi x10, x13, #48, #4
; CHECK-BE-NEXT: orr x13, x13, x11, lsl #60 ; CHECK-BE-NEXT: strh w11, [x1, #8]
; CHECK-BE-NEXT: lsr x11, x11, #4
; CHECK-BE-NEXT: orr x13, x13, x15, lsl #20
; CHECK-BE-NEXT: strh w14, [x1, #8]
; CHECK-BE-NEXT: lsr x12, x13, #16
; CHECK-BE-NEXT: stur x9, [x1, #10] ; CHECK-BE-NEXT: stur x9, [x1, #10]
; CHECK-BE-NEXT: bfi x12, x11, #48, #4 ; CHECK-BE-NEXT: str x10, [x1], #64
; CHECK-BE-NEXT: str x12, [x1], #64
; CHECK-BE-NEXT: b.ne .LBB10_1 ; CHECK-BE-NEXT: b.ne .LBB10_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v4i8_to_v4i32_in_loop(ptr %src, ptr %dst) { define void @zext_v4i8_to_v4i32_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v4i8_to_v4i32_in_loop: ; CHECK-LABEL: zext_v4i8_to_v4i32_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-BE-NEXT: b.ne .LBB11_1 ; CHECK-BE-NEXT: b.ne .LBB11_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
; CHECK-LABEL: lCPI12_0:
; CHECK-NEXT: .byte 0 ; 0x0
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 1 ; 0x1
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 2 ; 0x2
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 3 ; 0x3
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: lCPI12_1:
; CHECK-NEXT: .byte 4 ; 0x4
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 5 ; 0x5
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 6 ; 0x6
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 7 ; 0x7
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: lCPI12_2:
; CHECK-NEXT: .byte 8 ; 0x8
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 9 ; 0x9
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 10 ; 0xa
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 11 ; 0xb
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-BE-LABEL: .LCPI12_0:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 0 // 0x0
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 1 // 0x1
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 2 // 0x2
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 3 // 0x3
; CHECK-BE-NEXT: .LCPI12_1:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 4 // 0x4
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 5 // 0x5
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 6 // 0x6
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 7 // 0x7
; CHECK-BE-NEXT: .LCPI12_2:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 8 // 0x8
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 9 // 0x9
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 10 // 0xa
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 11 // 0xb
define void @zext_v12i8_to_v12i32_in_loop(ptr %src, ptr %dst) { define void @zext_v12i8_to_v12i32_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v12i8_to_v12i32_in_loop: ; CHECK-LABEL: zext_v12i8_to_v12i32_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: Lloh12: ; CHECK-NEXT: Lloh12:
; CHECK-NEXT: adrp x9, lCPI12_0@PAGE ; CHECK-NEXT: adrp x8, lCPI12_0@PAGE
; CHECK-NEXT: Lloh13: ; CHECK-NEXT: Lloh13:
; CHECK-NEXT: adrp x10, lCPI12_1@PAGE ; CHECK-NEXT: adrp x9, lCPI12_1@PAGE
; CHECK-NEXT: Lloh14: ; CHECK-NEXT: Lloh14:
; CHECK-NEXT: adrp x11, lCPI12_2@PAGE ; CHECK-NEXT: adrp x10, lCPI12_2@PAGE
; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: Lloh15: ; CHECK-NEXT: Lloh15:
; CHECK-NEXT: ldr q0, [x9, lCPI12_0@PAGEOFF] ; CHECK-NEXT: ldr q0, [x8, lCPI12_0@PAGEOFF]
; CHECK-NEXT: Lloh16: ; CHECK-NEXT: Lloh16:
; CHECK-NEXT: ldr q1, [x10, lCPI12_1@PAGEOFF] ; CHECK-NEXT: ldr q1, [x9, lCPI12_1@PAGEOFF]
; CHECK-NEXT: Lloh17: ; CHECK-NEXT: Lloh17:
; CHECK-NEXT: ldr q2, [x11, lCPI12_2@PAGEOFF] ; CHECK-NEXT: ldr q2, [x10, lCPI12_2@PAGEOFF]
; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: LBB12_1: ; %loop ; CHECK-NEXT: LBB12_1: ; %loop
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldr q3, [x0, x8] ; CHECK-NEXT: ldr q3, [x0, x8]
Context not available.
; CHECK-BE-NEXT: .LBB12_1: // %loop ; CHECK-BE-NEXT: .LBB12_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x10, x1, #16
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128 ; CHECK-BE-NEXT: add x10, x1, #16
; CHECK-BE-NEXT: ld1 { v3.16b }, [x9] ; CHECK-BE-NEXT: ld1 { v3.16b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #32 ; CHECK-BE-NEXT: add x9, x1, #32
; CHECK-BE-NEXT: tbl v4.16b, { v3.16b }, v0.16b ; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: tbl v5.16b, { v3.16b }, v2.16b ; CHECK-BE-NEXT: tbl v4.16b, { v3.16b }, v2.16b
; CHECK-BE-NEXT: tbl v3.16b, { v3.16b }, v1.16b ; CHECK-BE-NEXT: tbl v5.16b, { v3.16b }, v1.16b
; CHECK-BE-NEXT: st1 { v4.16b }, [x1] ; CHECK-BE-NEXT: tbl v3.16b, { v3.16b }, v0.16b
; CHECK-BE-NEXT: st1 { v3.16b }, [x1]
; CHECK-BE-NEXT: add x1, x1, #64 ; CHECK-BE-NEXT: add x1, x1, #64
; CHECK-BE-NEXT: st1 { v5.16b }, [x9] ; CHECK-BE-NEXT: st1 { v4.16b }, [x9]
; CHECK-BE-NEXT: st1 { v3.16b }, [x10] ; CHECK-BE-NEXT: st1 { v5.16b }, [x10]
; CHECK-BE-NEXT: b.ne .LBB12_1 ; CHECK-BE-NEXT: b.ne .LBB12_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v16i4_to_v16i32_in_loop(ptr %src, ptr %dst) { define void @zext_v16i4_to_v16i32_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v16i4_to_v16i32_in_loop: ; CHECK-LABEL: zext_v16i4_to_v16i32_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: movi.4s v0, #15 ; CHECK-NEXT: movi.4s v0, #15
Context not available.
; CHECK-NEXT: ldr x9, [x0, x8] ; CHECK-NEXT: ldr x9, [x0, x8]
; CHECK-NEXT: add x8, x8, #16 ; CHECK-NEXT: add x8, x8, #16
; CHECK-NEXT: cmp x8, #128 ; CHECK-NEXT: cmp x8, #128
; CHECK-NEXT: and w10, w9, #0xf ; CHECK-NEXT: and w11, w9, #0xf
; CHECK-NEXT: ubfx w11, w9, #4, #4 ; CHECK-NEXT: ubfx w10, w9, #4, #4
; CHECK-NEXT: fmov s1, w10 ; CHECK-NEXT: fmov s1, w11
; CHECK-NEXT: mov.b v1[1], w10
; CHECK-NEXT: ubfx w10, w9, #8, #4 ; CHECK-NEXT: ubfx w10, w9, #8, #4
; CHECK-NEXT: mov.b v1[1], w11
; CHECK-NEXT: mov.b v1[2], w10 ; CHECK-NEXT: mov.b v1[2], w10
; CHECK-NEXT: ubfx w10, w9, #12, #4 ; CHECK-NEXT: ubfx w10, w9, #12, #4
; CHECK-NEXT: mov.b v1[3], w10 ; CHECK-NEXT: mov.b v1[3], w10
Context not available.
; CHECK-NEXT: ext.16b v2, v1, v1, #8 ; CHECK-NEXT: ext.16b v2, v1, v1, #8
; CHECK-NEXT: zip2.8b v3, v1, v0 ; CHECK-NEXT: zip2.8b v3, v1, v0
; CHECK-NEXT: zip1.8b v1, v1, v0 ; CHECK-NEXT: zip1.8b v1, v1, v0
; CHECK-NEXT: zip1.8b v4, v2, v0 ; CHECK-NEXT: zip2.8b v4, v2, v0
; CHECK-NEXT: zip2.8b v2, v2, v0 ; CHECK-NEXT: zip1.8b v2, v2, v0
; CHECK-NEXT: ushll.4s v3, v3, #0 ; CHECK-NEXT: ushll.4s v3, v3, #0
; CHECK-NEXT: ushll.4s v1, v1, #0 ; CHECK-NEXT: ushll.4s v1, v1, #0
; CHECK-NEXT: and.16b v3, v3, v0 ; CHECK-NEXT: and.16b v3, v3, v0
; CHECK-NEXT: ushll.4s v4, v4, #0
; CHECK-NEXT: ushll.4s v2, v2, #0
; CHECK-NEXT: and.16b v1, v1, v0 ; CHECK-NEXT: and.16b v1, v1, v0
; CHECK-NEXT: stp q1, q3, [x1] ; CHECK-NEXT: and.16b v4, v4, v0
; CHECK-NEXT: ushll.4s v1, v2, #0
; CHECK-NEXT: ushll.4s v2, v4, #0
; CHECK-NEXT: and.16b v1, v1, v0
; CHECK-NEXT: and.16b v2, v2, v0 ; CHECK-NEXT: and.16b v2, v2, v0
; CHECK-NEXT: stp q2, q1, [x1, #32] ; CHECK-NEXT: stp q1, q3, [x1]
; CHECK-NEXT: stp q2, q4, [x1, #32]
; CHECK-NEXT: add x1, x1, #64 ; CHECK-NEXT: add x1, x1, #64
; CHECK-NEXT: b.ne LBB13_1 ; CHECK-NEXT: b.ne LBB13_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
Context not available.
; CHECK-BE-NEXT: fmov s1, w10 ; CHECK-BE-NEXT: fmov s1, w10
; CHECK-BE-NEXT: ubfx x10, x9, #52, #4 ; CHECK-BE-NEXT: ubfx x10, x9, #52, #4
; CHECK-BE-NEXT: mov v1.b[1], w11 ; CHECK-BE-NEXT: mov v1.b[1], w11
; CHECK-BE-NEXT: add x11, x1, #32
; CHECK-BE-NEXT: mov v1.b[2], w10 ; CHECK-BE-NEXT: mov v1.b[2], w10
; CHECK-BE-NEXT: ubfx x10, x9, #48, #4 ; CHECK-BE-NEXT: ubfx x10, x9, #48, #4
; CHECK-BE-NEXT: mov v1.b[3], w10 ; CHECK-BE-NEXT: mov v1.b[3], w10
Context not available.
; CHECK-BE-NEXT: ubfx w10, w9, #4, #4 ; CHECK-BE-NEXT: ubfx w10, w9, #4, #4
; CHECK-BE-NEXT: and w9, w9, #0xf ; CHECK-BE-NEXT: and w9, w9, #0xf
; CHECK-BE-NEXT: mov v1.b[14], w10 ; CHECK-BE-NEXT: mov v1.b[14], w10
; CHECK-BE-NEXT: add x10, x1, #48 ; CHECK-BE-NEXT: add x10, x1, #32
; CHECK-BE-NEXT: mov v1.b[15], w9 ; CHECK-BE-NEXT: mov v1.b[15], w9
; CHECK-BE-NEXT: add x9, x1, #16 ; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: ext v2.16b, v1.16b, v1.16b, #8 ; CHECK-BE-NEXT: ext v2.16b, v1.16b, v1.16b, #8
; CHECK-BE-NEXT: zip2 v3.8b, v1.8b, v0.8b ; CHECK-BE-NEXT: zip2 v3.8b, v1.8b, v0.8b
; CHECK-BE-NEXT: zip1 v1.8b, v1.8b, v0.8b ; CHECK-BE-NEXT: zip1 v1.8b, v1.8b, v0.8b
; CHECK-BE-NEXT: zip1 v4.8b, v2.8b, v0.8b ; CHECK-BE-NEXT: zip2 v4.8b, v2.8b, v0.8b
; CHECK-BE-NEXT: zip2 v2.8b, v2.8b, v0.8b ; CHECK-BE-NEXT: zip1 v2.8b, v2.8b, v0.8b
; CHECK-BE-NEXT: rev16 v1.8b, v1.8b
; CHECK-BE-NEXT: rev16 v3.8b, v3.8b ; CHECK-BE-NEXT: rev16 v3.8b, v3.8b
; CHECK-BE-NEXT: rev16 v1.8b, v1.8b
; CHECK-BE-NEXT: rev16 v4.8b, v4.8b ; CHECK-BE-NEXT: rev16 v4.8b, v4.8b
; CHECK-BE-NEXT: rev16 v2.8b, v2.8b ; CHECK-BE-NEXT: rev16 v2.8b, v2.8b
; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0
; CHECK-BE-NEXT: ushll v3.4s, v3.4h, #0 ; CHECK-BE-NEXT: ushll v3.4s, v3.4h, #0
; CHECK-BE-NEXT: and v1.16b, v1.16b, v0.16b ; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0
; CHECK-BE-NEXT: st1 { v1.4s }, [x1]
; CHECK-BE-NEXT: add x1, x1, #64
; CHECK-BE-NEXT: ushll v1.4s, v2.4h, #0
; CHECK-BE-NEXT: ushll v2.4s, v4.4h, #0
; CHECK-BE-NEXT: and v3.16b, v3.16b, v0.16b ; CHECK-BE-NEXT: and v3.16b, v3.16b, v0.16b
; CHECK-BE-NEXT: ushll v4.4s, v4.4h, #0
; CHECK-BE-NEXT: ushll v2.4s, v2.4h, #0
; CHECK-BE-NEXT: and v1.16b, v1.16b, v0.16b ; CHECK-BE-NEXT: and v1.16b, v1.16b, v0.16b
; CHECK-BE-NEXT: st1 { v3.4s }, [x9] ; CHECK-BE-NEXT: st1 { v3.4s }, [x9]
; CHECK-BE-NEXT: add x9, x1, #48
; CHECK-BE-NEXT: and v4.16b, v4.16b, v0.16b
; CHECK-BE-NEXT: and v2.16b, v2.16b, v0.16b ; CHECK-BE-NEXT: and v2.16b, v2.16b, v0.16b
; CHECK-BE-NEXT: st1 { v1.4s }, [x10] ; CHECK-BE-NEXT: st1 { v1.4s }, [x1]
; CHECK-BE-NEXT: st1 { v2.4s }, [x11] ; CHECK-BE-NEXT: add x1, x1, #64
; CHECK-BE-NEXT: st1 { v4.4s }, [x9]
; CHECK-BE-NEXT: st1 { v2.4s }, [x10]
; CHECK-BE-NEXT: b.ne .LBB13_1 ; CHECK-BE-NEXT: b.ne .LBB13_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i4, ptr %src, i64 %iv %src.gep = getelementptr i4, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v16i16_to_v16i64_in_loop(ptr %src, ptr %dst) { define void @zext_v16i16_to_v16i64_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v16i16_to_v16i64_in_loop: ; CHECK-LABEL: zext_v16i16_to_v16i64_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: add x9, x0, x8 ; CHECK-NEXT: add x9, x0, x8
; CHECK-NEXT: add x8, x8, #32 ; CHECK-NEXT: add x8, x8, #32
; CHECK-NEXT: ldp q1, q0, [x9]
; CHECK-NEXT: cmp x8, #256 ; CHECK-NEXT: cmp x8, #256
; CHECK-NEXT: ldp q0, q1, [x9] ; CHECK-NEXT: ushll2.4s v2, v0, #0
; CHECK-NEXT: ushll.4s v2, v0, #0 ; CHECK-NEXT: ushll2.4s v3, v1, #0
; CHECK-NEXT: ushll2.4s v0, v0, #0 ; CHECK-NEXT: ushll.4s v0, v0, #0
; CHECK-NEXT: ushll.4s v3, v1, #0 ; CHECK-NEXT: ushll.4s v1, v1, #0
; CHECK-NEXT: ushll2.4s v1, v1, #0 ; CHECK-NEXT: ushll2.2d v4, v2, #0
; CHECK-NEXT: ushll2.2d v5, v0, #0 ; CHECK-NEXT: ushll.2d v2, v2, #0
; CHECK-NEXT: ushll2.2d v4, v1, #0 ; CHECK-NEXT: ushll2.2d v5, v3, #0
; CHECK-NEXT: ushll.2d v1, v1, #0
; CHECK-NEXT: ushll.2d v0, v0, #0
; CHECK-NEXT: stp q1, q4, [x1, #96]
; CHECK-NEXT: ushll2.2d v1, v3, #0
; CHECK-NEXT: stp q0, q5, [x1, #32]
; CHECK-NEXT: ushll.2d v3, v3, #0 ; CHECK-NEXT: ushll.2d v3, v3, #0
; CHECK-NEXT: ushll2.2d v0, v2, #0 ; CHECK-NEXT: stp q2, q4, [x1, #96]
; CHECK-NEXT: stp q3, q1, [x1, #64] ; CHECK-NEXT: ushll2.2d v4, v0, #0
; CHECK-NEXT: ushll.2d v1, v2, #0 ; CHECK-NEXT: ushll.2d v0, v0, #0
; CHECK-NEXT: stp q1, q0, [x1], #128 ; CHECK-NEXT: ushll2.2d v2, v1, #0
; CHECK-NEXT: ushll.2d v1, v1, #0
; CHECK-NEXT: stp q3, q5, [x1, #32]
; CHECK-NEXT: stp q0, q4, [x1, #64]
; CHECK-NEXT: stp q1, q2, [x1], #128
; CHECK-NEXT: b.ne LBB14_1 ; CHECK-NEXT: b.ne LBB14_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: ret ; CHECK-NEXT: ret
Context not available.
; CHECK-BE-NEXT: .LBB14_1: // %loop ; CHECK-BE-NEXT: .LBB14_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x10, x1, #48
; CHECK-BE-NEXT: add x8, x8, #32 ; CHECK-BE-NEXT: add x8, x8, #32
; CHECK-BE-NEXT: cmp x8, #256
; CHECK-BE-NEXT: ld1 { v0.8h }, [x9] ; CHECK-BE-NEXT: ld1 { v0.8h }, [x9]
; CHECK-BE-NEXT: add x9, x9, #16 ; CHECK-BE-NEXT: add x9, x9, #16
; CHECK-BE-NEXT: ld1 { v2.8h }, [x9] ; CHECK-BE-NEXT: cmp x8, #256
; CHECK-BE-NEXT: add x9, x1, #32 ; CHECK-BE-NEXT: ld1 { v1.8h }, [x9]
; CHECK-BE-NEXT: ushll2 v1.4s, v0.8h, #0 ; CHECK-BE-NEXT: add x9, x1, #48
; CHECK-BE-NEXT: ushll2 v2.4s, v0.8h, #0
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0 ; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: ushll2 v3.2d, v1.4s, #0 ; CHECK-BE-NEXT: ushll2 v3.4s, v1.8h, #0
; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0 ; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0
; CHECK-BE-NEXT: st1 { v3.2d }, [x10] ; CHECK-BE-NEXT: ushll2 v4.2d, v2.4s, #0
; CHECK-BE-NEXT: add x10, x1, #112 ; CHECK-BE-NEXT: ushll v2.2d, v2.2s, #0
; CHECK-BE-NEXT: st1 { v1.2d }, [x9] ; CHECK-BE-NEXT: ushll2 v5.2d, v0.4s, #0
; CHECK-BE-NEXT: ushll v0.2d, v0.2s, #0
; CHECK-BE-NEXT: st1 { v4.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #32
; CHECK-BE-NEXT: ushll2 v4.2d, v3.4s, #0
; CHECK-BE-NEXT: st1 { v2.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #16 ; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: ushll2 v3.2d, v0.4s, #0 ; CHECK-BE-NEXT: ushll v2.2d, v3.2s, #0
; CHECK-BE-NEXT: ushll2 v1.4s, v2.8h, #0 ; CHECK-BE-NEXT: st1 { v5.2d }, [x9]
; CHECK-BE-NEXT: st1 { v3.2d }, [x9] ; CHECK-BE-NEXT: add x9, x1, #112
; CHECK-BE-NEXT: ushll2 v3.2d, v1.4s, #0
; CHECK-BE-NEXT: st1 { v4.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #96 ; CHECK-BE-NEXT: add x9, x1, #96
; CHECK-BE-NEXT: ushll2 v4.2d, v1.4s, #0
; CHECK-BE-NEXT: ushll v0.2d, v0.2s, #0
; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0 ; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0
; CHECK-BE-NEXT: st1 { v4.2d }, [x10] ; CHECK-BE-NEXT: st1 { v2.2d }, [x9]
; CHECK-BE-NEXT: ushll v2.4s, v2.4h, #0 ; CHECK-BE-NEXT: add x9, x1, #80
; CHECK-BE-NEXT: add x10, x1, #80 ; CHECK-BE-NEXT: st1 { v3.2d }, [x9]
; CHECK-BE-NEXT: st1 { v0.2d }, [x1]
; CHECK-BE-NEXT: st1 { v1.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #64 ; CHECK-BE-NEXT: add x9, x1, #64
; CHECK-BE-NEXT: st1 { v0.2d }, [x1]
; CHECK-BE-NEXT: add x1, x1, #128 ; CHECK-BE-NEXT: add x1, x1, #128
; CHECK-BE-NEXT: ushll v3.2d, v2.2s, #0 ; CHECK-BE-NEXT: st1 { v1.2d }, [x9]
; CHECK-BE-NEXT: ushll2 v2.2d, v2.4s, #0
; CHECK-BE-NEXT: st1 { v3.2d }, [x9]
; CHECK-BE-NEXT: st1 { v2.2d }, [x10]
; CHECK-BE-NEXT: b.ne .LBB14_1 ; CHECK-BE-NEXT: b.ne .LBB14_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i16, ptr %src, i64 %iv %src.gep = getelementptr i16, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v16i32_to_v16i64_in_loop(ptr %src, ptr %dst) { define void @zext_v16i32_to_v16i64_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v16i32_to_v16i64_in_loop: ; CHECK-LABEL: zext_v16i32_to_v16i64_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: add x9, x0, x8 ; CHECK-NEXT: add x9, x0, x8
; CHECK-NEXT: add x8, x8, #64 ; CHECK-NEXT: add x8, x8, #64
; CHECK-NEXT: cmp x8, #512
; CHECK-NEXT: ldp q1, q0, [x9, #32] ; CHECK-NEXT: ldp q1, q0, [x9, #32]
; CHECK-NEXT: ushll2.2d v5, v1, #0 ; CHECK-NEXT: cmp x8, #512
; CHECK-NEXT: ushll.2d v1, v1, #0 ; CHECK-NEXT: ldp q5, q4, [x9]
; CHECK-NEXT: ldp q3, q2, [x9] ; CHECK-NEXT: ushll2.2d v2, v0, #0
; CHECK-NEXT: ushll2.2d v4, v0, #0
; CHECK-NEXT: stp q1, q5, [x1, #64]
; CHECK-NEXT: ushll.2d v0, v0, #0 ; CHECK-NEXT: ushll.2d v0, v0, #0
; CHECK-NEXT: stp q0, q4, [x1, #96] ; CHECK-NEXT: ushll2.2d v3, v1, #0
; CHECK-NEXT: ushll2.2d v1, v3, #0 ; CHECK-NEXT: ushll.2d v1, v1, #0
; CHECK-NEXT: ushll2.2d v0, v2, #0 ; CHECK-NEXT: stp q0, q2, [x1, #96]
; CHECK-NEXT: ushll.2d v2, v2, #0 ; CHECK-NEXT: ushll2.2d v2, v4, #0
; CHECK-NEXT: stp q2, q0, [x1, #32] ; CHECK-NEXT: ushll.2d v0, v4, #0
; CHECK-NEXT: ushll.2d v0, v3, #0 ; CHECK-NEXT: stp q1, q3, [x1, #64]
; CHECK-NEXT: stp q0, q1, [x1], #128 ; CHECK-NEXT: ushll2.2d v3, v5, #0
; CHECK-NEXT: ushll.2d v1, v5, #0
; CHECK-NEXT: stp q0, q2, [x1, #32]
; CHECK-NEXT: stp q1, q3, [x1], #128
; CHECK-NEXT: b.ne LBB15_1 ; CHECK-NEXT: b.ne LBB15_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: ret ; CHECK-NEXT: ret
Context not available.
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x8, x8, #64 ; CHECK-BE-NEXT: add x8, x8, #64
; CHECK-BE-NEXT: ld1 { v0.4s }, [x9]
; CHECK-BE-NEXT: add x10, x9, #48 ; CHECK-BE-NEXT: add x10, x9, #48
; CHECK-BE-NEXT: add x11, x9, #32
; CHECK-BE-NEXT: cmp x8, #512 ; CHECK-BE-NEXT: cmp x8, #512
; CHECK-BE-NEXT: ld1 { v0.4s }, [x9]
; CHECK-BE-NEXT: add x9, x9, #16
; CHECK-BE-NEXT: ld1 { v1.4s }, [x10] ; CHECK-BE-NEXT: ld1 { v1.4s }, [x10]
; CHECK-BE-NEXT: add x10, x1, #16 ; CHECK-BE-NEXT: add x10, x9, #32
; CHECK-BE-NEXT: ld1 { v2.4s }, [x11] ; CHECK-BE-NEXT: add x9, x9, #16
; CHECK-BE-NEXT: ushll2 v3.2d, v0.4s, #0
; CHECK-BE-NEXT: ld1 { v4.4s }, [x9] ; CHECK-BE-NEXT: ld1 { v4.4s }, [x9]
; CHECK-BE-NEXT: add x9, x1, #112 ; CHECK-BE-NEXT: ld1 { v2.4s }, [x10]
; CHECK-BE-NEXT: st1 { v3.2d }, [x10] ; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: ushll2 v3.2d, v0.4s, #0
; CHECK-BE-NEXT: ushll v0.2d, v0.2s, #0
; CHECK-BE-NEXT: add x10, x1, #80 ; CHECK-BE-NEXT: add x10, x1, #80
; CHECK-BE-NEXT: ushll2 v3.2d, v1.4s, #0 ; CHECK-BE-NEXT: ushll2 v5.2d, v1.4s, #0
; CHECK-BE-NEXT: ushll2 v5.2d, v2.4s, #0 ; CHECK-BE-NEXT: ushll2 v6.2d, v2.4s, #0
; CHECK-BE-NEXT: st1 { v3.2d }, [x9] ; CHECK-BE-NEXT: st1 { v3.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #48 ; CHECK-BE-NEXT: ushll2 v3.2d, v4.4s, #0
; CHECK-BE-NEXT: st1 { v5.2d }, [x10] ; CHECK-BE-NEXT: add x9, x1, #112
; CHECK-BE-NEXT: add x10, x1, #96
; CHECK-BE-NEXT: ushll v0.2d, v0.2s, #0
; CHECK-BE-NEXT: ushll v3.2d, v4.2s, #0
; CHECK-BE-NEXT: ushll2 v4.2d, v4.4s, #0
; CHECK-BE-NEXT: st1 { v0.2d }, [x1] ; CHECK-BE-NEXT: st1 { v0.2d }, [x1]
; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0 ; CHECK-BE-NEXT: ushll v0.2d, v1.2s, #0
; CHECK-BE-NEXT: st1 { v4.2d }, [x9] ; CHECK-BE-NEXT: ushll v1.2d, v2.2s, #0
; CHECK-BE-NEXT: st1 { v5.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #48
; CHECK-BE-NEXT: ushll v2.2d, v4.2s, #0
; CHECK-BE-NEXT: st1 { v3.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #64 ; CHECK-BE-NEXT: add x9, x1, #64
; CHECK-BE-NEXT: st1 { v1.2d }, [x10] ; CHECK-BE-NEXT: st1 { v6.2d }, [x10]
; CHECK-BE-NEXT: add x10, x1, #32 ; CHECK-BE-NEXT: add x10, x1, #96
; CHECK-BE-NEXT: st1 { v1.2d }, [x9]
; CHECK-BE-NEXT: add x9, x1, #32
; CHECK-BE-NEXT: add x1, x1, #128 ; CHECK-BE-NEXT: add x1, x1, #128
; CHECK-BE-NEXT: ushll v2.2d, v2.2s, #0 ; CHECK-BE-NEXT: st1 { v0.2d }, [x10]
; CHECK-BE-NEXT: st1 { v3.2d }, [x10]
; CHECK-BE-NEXT: st1 { v2.2d }, [x9] ; CHECK-BE-NEXT: st1 { v2.2d }, [x9]
; CHECK-BE-NEXT: b.ne .LBB15_1 ; CHECK-BE-NEXT: b.ne .LBB15_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i32, ptr %src, i64 %iv %src.gep = getelementptr i32, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v8i8_to_v8i128_in_loop(ptr %src, ptr %dst) { define void @zext_v8i8_to_v8i128_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v8i8_to_v8i128_in_loop: ; CHECK-LABEL: zext_v8i8_to_v8i128_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldr d0, [x0, x8] ; CHECK-NEXT: ldr d0, [x0, x8]
; CHECK-NEXT: add x9, x1, #112 ; CHECK-NEXT: add x9, x1, #112
; CHECK-NEXT: add x10, x1, #80
; CHECK-NEXT: str xzr, [x1, #120]
; CHECK-NEXT: str xzr, [x1, #104]
; CHECK-NEXT: add x8, x8, #16 ; CHECK-NEXT: add x8, x8, #16
; CHECK-NEXT: str xzr, [x1, #88] ; CHECK-NEXT: str xzr, [x1, #120]
; CHECK-NEXT: cmp x8, #128 ; CHECK-NEXT: cmp x8, #128
; CHECK-NEXT: ushll.8h v0, v0, #0 ; CHECK-NEXT: ushll.8h v0, v0, #0
; CHECK-NEXT: str xzr, [x1, #104]
; CHECK-NEXT: str xzr, [x1, #88]
; CHECK-NEXT: str xzr, [x1, #72] ; CHECK-NEXT: str xzr, [x1, #72]
; CHECK-NEXT: str xzr, [x1, #56]
; CHECK-NEXT: ushll2.4s v1, v0, #0 ; CHECK-NEXT: ushll2.4s v1, v0, #0
; CHECK-NEXT: str xzr, [x1, #40]
; CHECK-NEXT: ushll.4s v0, v0, #0 ; CHECK-NEXT: ushll.4s v0, v0, #0
; CHECK-NEXT: str xzr, [x1, #56]
; CHECK-NEXT: str xzr, [x1, #40]
; CHECK-NEXT: str xzr, [x1, #24] ; CHECK-NEXT: str xzr, [x1, #24]
; CHECK-NEXT: ushll2.2d v2, v1, #0 ; CHECK-NEXT: ushll2.2d v2, v1, #0
; CHECK-NEXT: str xzr, [x1, #8]
; CHECK-NEXT: ushll.2d v1, v1, #0 ; CHECK-NEXT: ushll.2d v1, v1, #0
; CHECK-NEXT: ushll2.2d v3, v0, #0
; CHECK-NEXT: ushll.2d v0, v0, #0
; CHECK-NEXT: str xzr, [x1, #8]
; CHECK-NEXT: st1.d { v2 }[1], [x9] ; CHECK-NEXT: st1.d { v2 }[1], [x9]
; CHECK-NEXT: add x9, x1, #80
; CHECK-NEXT: st1.d { v1 }[1], [x9]
; CHECK-NEXT: add x9, x1, #48 ; CHECK-NEXT: add x9, x1, #48
; CHECK-NEXT: str d2, [x1, #96] ; CHECK-NEXT: str d2, [x1, #96]
; CHECK-NEXT: ushll2.2d v2, v0, #0 ; CHECK-NEXT: st1.d { v3 }[1], [x9]
; CHECK-NEXT: st1.d { v1 }[1], [x10] ; CHECK-NEXT: add x9, x1, #16
; CHECK-NEXT: ushll.2d v0, v0, #0
; CHECK-NEXT: str d1, [x1, #64] ; CHECK-NEXT: str d1, [x1, #64]
; CHECK-NEXT: str d2, [x1, #32] ; CHECK-NEXT: str d3, [x1, #32]
; CHECK-NEXT: add x10, x1, #16
; CHECK-NEXT: str d0, [x1] ; CHECK-NEXT: str d0, [x1]
; CHECK-NEXT: add x1, x1, #256 ; CHECK-NEXT: add x1, x1, #256
; CHECK-NEXT: st1.d { v2 }[1], [x9] ; CHECK-NEXT: st1.d { v0 }[1], [x9]
; CHECK-NEXT: st1.d { v0 }[1], [x10]
; CHECK-NEXT: b.ne LBB16_1 ; CHECK-NEXT: b.ne LBB16_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: ret ; CHECK-NEXT: ret
Context not available.
; CHECK-BE-NEXT: .LBB16_1: // %loop ; CHECK-BE-NEXT: .LBB16_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x10, x1, #88
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v0.8b }, [x9] ; CHECK-BE-NEXT: ld1 { v0.8b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #120 ; CHECK-BE-NEXT: add x9, x1, #120
; CHECK-BE-NEXT: str xzr, [x1, #112] ; CHECK-BE-NEXT: str xzr, [x1, #112]
; CHECK-BE-NEXT: str xzr, [x1, #96] ; CHECK-BE-NEXT: str xzr, [x1, #96]
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: str xzr, [x1, #80] ; CHECK-BE-NEXT: str xzr, [x1, #80]
; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0 ; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-BE-NEXT: str xzr, [x1, #64] ; CHECK-BE-NEXT: str xzr, [x1, #64]
; CHECK-BE-NEXT: str xzr, [x1, #48] ; CHECK-BE-NEXT: str xzr, [x1, #48]
; CHECK-BE-NEXT: ushll2 v1.4s, v0.8h, #0
; CHECK-BE-NEXT: str xzr, [x1, #32] ; CHECK-BE-NEXT: str xzr, [x1, #32]
; CHECK-BE-NEXT: ushll2 v1.4s, v0.8h, #0
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0 ; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: str xzr, [x1, #16] ; CHECK-BE-NEXT: str xzr, [x1, #16]
; CHECK-BE-NEXT: ushll2 v2.2d, v1.4s, #0
; CHECK-BE-NEXT: str xzr, [x1] ; CHECK-BE-NEXT: str xzr, [x1]
; CHECK-BE-NEXT: ushll2 v2.2d, v1.4s, #0
; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0 ; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0
; CHECK-BE-NEXT: ushll2 v3.2d, v0.4s, #0
; CHECK-BE-NEXT: ushll v0.2d, v0.2s, #0
; CHECK-BE-NEXT: st1 { v2.d }[1], [x9] ; CHECK-BE-NEXT: st1 { v2.d }[1], [x9]
; CHECK-BE-NEXT: add x9, x1, #88
; CHECK-BE-NEXT: st1 { v1.d }[1], [x9]
; CHECK-BE-NEXT: add x9, x1, #56 ; CHECK-BE-NEXT: add x9, x1, #56
; CHECK-BE-NEXT: str d2, [x1, #104] ; CHECK-BE-NEXT: str d2, [x1, #104]
; CHECK-BE-NEXT: ushll2 v2.2d, v0.4s, #0 ; CHECK-BE-NEXT: st1 { v3.d }[1], [x9]
; CHECK-BE-NEXT: st1 { v1.d }[1], [x10] ; CHECK-BE-NEXT: add x9, x1, #24
; CHECK-BE-NEXT: ushll v0.2d, v0.2s, #0
; CHECK-BE-NEXT: str d1, [x1, #72] ; CHECK-BE-NEXT: str d1, [x1, #72]
; CHECK-BE-NEXT: str d2, [x1, #40] ; CHECK-BE-NEXT: str d3, [x1, #40]
; CHECK-BE-NEXT: add x10, x1, #24
; CHECK-BE-NEXT: str d0, [x1, #8] ; CHECK-BE-NEXT: str d0, [x1, #8]
; CHECK-BE-NEXT: add x1, x1, #256 ; CHECK-BE-NEXT: add x1, x1, #256
; CHECK-BE-NEXT: st1 { v2.d }[1], [x9] ; CHECK-BE-NEXT: st1 { v0.d }[1], [x9]
; CHECK-BE-NEXT: st1 { v0.d }[1], [x10]
; CHECK-BE-NEXT: b.ne .LBB16_1 ; CHECK-BE-NEXT: b.ne .LBB16_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
; multiple back-to-back 'zext' of similar type of vectors combined with arithmetic operations ; multiple back-to-back 'zext' of similar type of vectors combined with arithmetic operations
define void @zext_v8i8_to_v8i64_with_add_in_sequence_in_loop(ptr %src, ptr %dst) { define void @zext_v8i8_to_v8i64_with_add_in_sequence_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v8i8_to_v8i64_with_add_in_sequence_in_loop: ; CHECK-LABEL: zext_v8i8_to_v8i64_with_add_in_sequence_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: Lloh18: ; CHECK-NEXT: Lloh18:
Context not available.
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: Lloh20: ; CHECK-NEXT: Lloh20:
; CHECK-NEXT: ldr q0, [x9, lCPI17_0@PAGEOFF] ; CHECK-NEXT: ldr q0, [x9, lCPI17_0@PAGEOFF]
; CHECK-NEXT: add x9, x0, #8
; CHECK-NEXT: Lloh21: ; CHECK-NEXT: Lloh21:
; CHECK-NEXT: ldr q1, [x10, lCPI17_1@PAGEOFF] ; CHECK-NEXT: ldr q1, [x10, lCPI17_1@PAGEOFF]
; CHECK-NEXT: add x9, x0, #8
; CHECK-NEXT: LBB17_1: ; %loop ; CHECK-NEXT: LBB17_1: ; %loop
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldp d2, d4, [x9, #-8]
; CHECK-NEXT: add x10, x1, x8 ; CHECK-NEXT: add x10, x1, x8
; CHECK-NEXT: ldp q6, q5, [x10, #32]
; CHECK-NEXT: add x8, x8, #128 ; CHECK-NEXT: add x8, x8, #128
; CHECK-NEXT: ldp d2, d3, [x9, #-8] ; CHECK-NEXT: ldp q17, q16, [x10]
; CHECK-NEXT: add x9, x9, #16
; CHECK-NEXT: cmp x8, #1024 ; CHECK-NEXT: cmp x8, #1024
; CHECK-NEXT: ldp q5, q4, [x10, #32] ; CHECK-NEXT: tbl.16b v3, { v2 }, v1
; CHECK-NEXT: tbl.16b v6, { v2 }, v1
; CHECK-NEXT: tbl.16b v2, { v2 }, v0 ; CHECK-NEXT: tbl.16b v2, { v2 }, v0
; CHECK-NEXT: tbl.16b v17, { v3 }, v0 ; CHECK-NEXT: tbl.16b v7, { v4 }, v1
; CHECK-NEXT: tbl.16b v3, { v3 }, v1 ; CHECK-NEXT: tbl.16b v4, { v4 }, v0
; CHECK-NEXT: ldp q16, q7, [x10] ; CHECK-NEXT: add x9, x9, #16
; CHECK-NEXT: uaddw2.2d v4, v4, v6 ; CHECK-NEXT: uaddw2.2d v5, v5, v3
; CHECK-NEXT: uaddw.2d v5, v5, v6 ; CHECK-NEXT: uaddw.2d v3, v6, v3
; CHECK-NEXT: stp q5, q4, [x10, #32] ; CHECK-NEXT: uaddw2.2d v6, v16, v2
; CHECK-NEXT: ldp q19, q18, [x10, #96] ; CHECK-NEXT: ldp q18, q16, [x10, #96]
; CHECK-NEXT: uaddw2.2d v7, v7, v2 ; CHECK-NEXT: uaddw.2d v2, v17, v2
; CHECK-NEXT: uaddw.2d v2, v16, v2 ; CHECK-NEXT: stp q3, q5, [x10, #32]
; CHECK-NEXT: stp q2, q7, [x10] ; CHECK-NEXT: ldp q17, q5, [x10, #64]
; CHECK-NEXT: ldp q6, q20, [x10, #64] ; CHECK-NEXT: uaddw2.2d v16, v16, v7
; CHECK-NEXT: uaddw2.2d v4, v18, v3 ; CHECK-NEXT: uaddw.2d v7, v18, v7
; CHECK-NEXT: uaddw.2d v3, v19, v3 ; CHECK-NEXT: stp q2, q6, [x10]
; CHECK-NEXT: stp q3, q4, [x10, #96] ; CHECK-NEXT: uaddw2.2d v3, v5, v4
; CHECK-NEXT: uaddw2.2d v2, v20, v17 ; CHECK-NEXT: uaddw.2d v4, v17, v4
; CHECK-NEXT: uaddw.2d v4, v6, v17 ; CHECK-NEXT: stp q7, q16, [x10, #96]
; CHECK-NEXT: stp q4, q2, [x10, #64] ; CHECK-NEXT: stp q4, q3, [x10, #64]
; CHECK-NEXT: b.ne LBB17_1 ; CHECK-NEXT: b.ne LBB17_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: ret ; CHECK-NEXT: ret
Context not available.
; CHECK-BE-NEXT: add x9, x0, #8 ; CHECK-BE-NEXT: add x9, x0, #8
; CHECK-BE-NEXT: .LBB17_1: // %loop ; CHECK-BE-NEXT: .LBB17_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: sub x12, x9, #8 ; CHECK-BE-NEXT: sub x10, x9, #8
; CHECK-BE-NEXT: add x10, x1, x8
; CHECK-BE-NEXT: ld1 { v2.8b }, [x9] ; CHECK-BE-NEXT: ld1 { v2.8b }, [x9]
; CHECK-BE-NEXT: add x9, x9, #16
; CHECK-BE-NEXT: ld1 { v3.8b }, [x10]
; CHECK-BE-NEXT: add x10, x1, x8
; CHECK-BE-NEXT: add x8, x8, #128
; CHECK-BE-NEXT: add x15, x10, #96
; CHECK-BE-NEXT: add x11, x10, #32 ; CHECK-BE-NEXT: add x11, x10, #32
; CHECK-BE-NEXT: add x13, x10, #48 ; CHECK-BE-NEXT: add x14, x10, #64
; CHECK-BE-NEXT: add x14, x10, #16 ; CHECK-BE-NEXT: tbl v4.16b, { v2.16b }, v1.16b
; CHECK-BE-NEXT: ld1 { v4.8b }, [x12]
; CHECK-BE-NEXT: add x15, x10, #64
; CHECK-BE-NEXT: ld1 { v3.2d }, [x11]
; CHECK-BE-NEXT: add x12, x10, #96
; CHECK-BE-NEXT: tbl v6.16b, { v2.16b }, v1.16b
; CHECK-BE-NEXT: add x16, x10, #112
; CHECK-BE-NEXT: tbl v2.16b, { v2.16b }, v0.16b ; CHECK-BE-NEXT: tbl v2.16b, { v2.16b }, v0.16b
; CHECK-BE-NEXT: ld1 { v7.2d }, [x13] ; CHECK-BE-NEXT: ld1 { v16.2d }, [x15]
; CHECK-BE-NEXT: tbl v16.16b, { v4.16b }, v0.16b ; CHECK-BE-NEXT: tbl v5.16b, { v3.16b }, v1.16b
; CHECK-BE-NEXT: tbl v3.16b, { v3.16b }, v0.16b
; CHECK-BE-NEXT: ld1 { v6.2d }, [x10]
; CHECK-BE-NEXT: ld1 { v19.2d }, [x14]
; CHECK-BE-NEXT: ld1 { v21.2d }, [x11]
; CHECK-BE-NEXT: add x12, x10, #48
; CHECK-BE-NEXT: add x13, x10, #16
; CHECK-BE-NEXT: add x16, x10, #112
; CHECK-BE-NEXT: add x17, x10, #80 ; CHECK-BE-NEXT: add x17, x10, #80
; CHECK-BE-NEXT: tbl v4.16b, { v4.16b }, v1.16b ; CHECK-BE-NEXT: rev32 v7.8b, v4.8b
; CHECK-BE-NEXT: ld1 { v18.2d }, [x14] ; CHECK-BE-NEXT: ext v4.16b, v4.16b, v4.16b, #8
; CHECK-BE-NEXT: rev32 v17.8b, v6.8b ; CHECK-BE-NEXT: rev32 v17.8b, v2.8b
; CHECK-BE-NEXT: add x8, x8, #128 ; CHECK-BE-NEXT: ext v18.16b, v5.16b, v5.16b, #8
; CHECK-BE-NEXT: ext v6.16b, v6.16b, v6.16b, #8 ; CHECK-BE-NEXT: ext v20.16b, v3.16b, v3.16b, #8
; CHECK-BE-NEXT: ld1 { v5.2d }, [x10] ; CHECK-BE-NEXT: ext v2.16b, v2.16b, v2.16b, #8
; CHECK-BE-NEXT: ext v23.16b, v16.16b, v16.16b, #8 ; CHECK-BE-NEXT: rev32 v5.8b, v5.8b
; CHECK-BE-NEXT: add x9, x9, #16 ; CHECK-BE-NEXT: rev32 v3.8b, v3.8b
; CHECK-BE-NEXT: ext v21.16b, v4.16b, v4.16b, #8
; CHECK-BE-NEXT: ld1 { v20.2d }, [x12]
; CHECK-BE-NEXT: rev32 v4.8b, v4.8b
; CHECK-BE-NEXT: cmp x8, #1024 ; CHECK-BE-NEXT: cmp x8, #1024
; CHECK-BE-NEXT: ext v19.16b, v2.16b, v2.16b, #8 ; CHECK-BE-NEXT: rev32 v4.8b, v4.8b
; CHECK-BE-NEXT: ld1 { v22.2d }, [x15] ; CHECK-BE-NEXT: uaddw v7.2d, v16.2d, v7.2s
; CHECK-BE-NEXT: ld1 { v16.2d }, [x16]
; CHECK-BE-NEXT: rev32 v18.8b, v18.8b
; CHECK-BE-NEXT: rev32 v20.8b, v20.8b
; CHECK-BE-NEXT: rev32 v2.8b, v2.8b ; CHECK-BE-NEXT: rev32 v2.8b, v2.8b
; CHECK-BE-NEXT: rev32 v21.8b, v21.8b ; CHECK-BE-NEXT: uaddw v17.2d, v19.2d, v17.2s
; CHECK-BE-NEXT: ld1 { v24.2d }, [x16] ; CHECK-BE-NEXT: ld1 { v19.2d }, [x12]
; CHECK-BE-NEXT: uaddw v3.2d, v3.2d, v4.2s ; CHECK-BE-NEXT: uaddw v5.2d, v21.2d, v5.2s
; CHECK-BE-NEXT: rev32 v4.8b, v23.8b ; CHECK-BE-NEXT: ld1 { v21.2d }, [x13]
; CHECK-BE-NEXT: ld1 { v23.2d }, [x17] ; CHECK-BE-NEXT: uaddw v3.2d, v6.2d, v3.2s
; CHECK-BE-NEXT: rev32 v16.8b, v16.8b ; CHECK-BE-NEXT: ld1 { v6.2d }, [x17]
; CHECK-BE-NEXT: rev32 v6.8b, v6.8b ; CHECK-BE-NEXT: uaddw v4.2d, v16.2d, v4.2s
; CHECK-BE-NEXT: rev32 v19.8b, v19.8b ; CHECK-BE-NEXT: st1 { v7.2d }, [x15]
; CHECK-BE-NEXT: st1 { v3.2d }, [x11] ; CHECK-BE-NEXT: uaddw v7.2d, v19.2d, v18.2s
; CHECK-BE-NEXT: uaddw v3.2d, v7.2d, v21.2s ; CHECK-BE-NEXT: uaddw v16.2d, v21.2d, v20.2s
; CHECK-BE-NEXT: uaddw v4.2d, v18.2d, v4.2s ; CHECK-BE-NEXT: uaddw v2.2d, v6.2d, v2.2s
; CHECK-BE-NEXT: uaddw v5.2d, v5.2d, v16.2s ; CHECK-BE-NEXT: st1 { v17.2d }, [x14]
; CHECK-BE-NEXT: uaddw v7.2d, v20.2d, v17.2s ; CHECK-BE-NEXT: st1 { v5.2d }, [x11]
; CHECK-BE-NEXT: st1 { v3.2d }, [x13] ; CHECK-BE-NEXT: st1 { v3.2d }, [x10]
; CHECK-BE-NEXT: uaddw v2.2d, v22.2d, v2.2s ; CHECK-BE-NEXT: st1 { v4.2d }, [x16]
; CHECK-BE-NEXT: st1 { v4.2d }, [x14]
; CHECK-BE-NEXT: uaddw v3.2d, v24.2d, v6.2s
; CHECK-BE-NEXT: st1 { v5.2d }, [x10]
; CHECK-BE-NEXT: uaddw v4.2d, v23.2d, v19.2s
; CHECK-BE-NEXT: st1 { v7.2d }, [x12] ; CHECK-BE-NEXT: st1 { v7.2d }, [x12]
; CHECK-BE-NEXT: st1 { v2.2d }, [x15] ; CHECK-BE-NEXT: st1 { v16.2d }, [x13]
; CHECK-BE-NEXT: st1 { v3.2d }, [x16] ; CHECK-BE-NEXT: st1 { v2.2d }, [x17]
; CHECK-BE-NEXT: st1 { v4.2d }, [x17]
; CHECK-BE-NEXT: b.ne .LBB17_1 ; CHECK-BE-NEXT: b.ne .LBB17_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
; multiple back-to-back 'zext' of similar type of vectors ; multiple back-to-back 'zext' of similar type of vectors
define void @zext_v16i8_to_v16i64_in_sequence_in_loop(ptr %src, ptr %dst) { define void @zext_v16i8_to_v16i64_in_sequence_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v16i8_to_v16i64_in_sequence_in_loop: ; CHECK-LABEL: zext_v16i8_to_v16i64_in_sequence_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: add x10, x0, x8 ; CHECK-NEXT: add x10, x0, x8
; CHECK-NEXT: add x8, x8, #16 ; CHECK-NEXT: add x8, x8, #16
; CHECK-NEXT: cmp x8, #128
; CHECK-NEXT: ldp q0, q1, [x10] ; CHECK-NEXT: ldp q0, q1, [x10]
; CHECK-NEXT: ushll.8h v2, v0, #0 ; CHECK-NEXT: cmp x8, #128
; CHECK-NEXT: ushll2.8h v0, v0, #0 ; CHECK-NEXT: ushll2.8h v2, v0, #0
; CHECK-NEXT: ushll.4s v4, v2, #0 ; CHECK-NEXT: ushll.8h v0, v0, #0
; CHECK-NEXT: ushll2.8h v6, v1, #0
; CHECK-NEXT: ushll.8h v1, v1, #0
; CHECK-NEXT: ushll2.4s v3, v2, #0
; CHECK-NEXT: ushll.4s v2, v2, #0
; CHECK-NEXT: ushll2.4s v5, v0, #0 ; CHECK-NEXT: ushll2.4s v5, v0, #0
; CHECK-NEXT: ushll2.4s v2, v2, #0
; CHECK-NEXT: ushll.4s v0, v0, #0 ; CHECK-NEXT: ushll.4s v0, v0, #0
; CHECK-NEXT: ushll2.2d v6, v5, #0 ; CHECK-NEXT: ushll2.2d v4, v3, #0
; CHECK-NEXT: ushll.2d v3, v3, #0
; CHECK-NEXT: ushll2.2d v7, v2, #0
; CHECK-NEXT: ushll.2d v2, v2, #0
; CHECK-NEXT: stp q3, q4, [x9, #-32]
; CHECK-NEXT: ushll2.2d v4, v5, #0
; CHECK-NEXT: ushll2.4s v3, v6, #0
; CHECK-NEXT: ushll.2d v5, v5, #0 ; CHECK-NEXT: ushll.2d v5, v5, #0
; CHECK-NEXT: ushll2.8h v3, v1, #0 ; CHECK-NEXT: stp q2, q7, [x9, #-64]
; CHECK-NEXT: ushll2.2d v7, v0, #0 ; CHECK-NEXT: ushll2.2d v7, v0, #0
; CHECK-NEXT: stp q5, q6, [x9, #-32]
; CHECK-NEXT: ushll.2d v0, v0, #0 ; CHECK-NEXT: ushll.2d v0, v0, #0
; CHECK-NEXT: ushll2.2d v5, v2, #0 ; CHECK-NEXT: ushll.4s v2, v6, #0
; CHECK-NEXT: ushll.2d v2, v2, #0 ; CHECK-NEXT: stp q5, q4, [x9, #-96]
; CHECK-NEXT: stp q0, q7, [x9, #-64] ; CHECK-NEXT: ushll2.2d v4, v3, #0
; CHECK-NEXT: ushll2.2d v0, v4, #0 ; CHECK-NEXT: ushll2.4s v5, v1, #0
; CHECK-NEXT: stp q2, q5, [x9, #-96] ; CHECK-NEXT: ushll.2d v3, v3, #0
; CHECK-NEXT: ushll2.4s v5, v3, #0 ; CHECK-NEXT: stp q0, q7, [x9, #-128]
; CHECK-NEXT: ushll.2d v2, v4, #0 ; CHECK-NEXT: ushll.4s v0, v1, #0
; CHECK-NEXT: ushll2.2d v4, v5, #0 ; CHECK-NEXT: ushll2.2d v6, v2, #0
; CHECK-NEXT: stp q2, q0, [x9, #-128] ; CHECK-NEXT: ushll.2d v1, v2, #0
; CHECK-NEXT: ushll.2d v0, v5, #0 ; CHECK-NEXT: ushll2.2d v2, v5, #0
; CHECK-NEXT: ushll.4s v2, v3, #0 ; CHECK-NEXT: stp q3, q4, [x9, #96]
; CHECK-NEXT: stp q0, q4, [x9, #96] ; CHECK-NEXT: ushll.2d v3, v5, #0
; CHECK-NEXT: ushll.8h v0, v1, #0 ; CHECK-NEXT: ushll2.2d v4, v0, #0
; CHECK-NEXT: ushll2.2d v1, v2, #0
; CHECK-NEXT: ushll.2d v2, v2, #0
; CHECK-NEXT: ushll2.4s v3, v0, #0
; CHECK-NEXT: stp q2, q1, [x9, #64]
; CHECK-NEXT: ushll2.2d v1, v3, #0
; CHECK-NEXT: ushll.2d v2, v3, #0
; CHECK-NEXT: ushll.4s v0, v0, #0
; CHECK-NEXT: stp q2, q1, [x9, #32]
; CHECK-NEXT: ushll2.2d v1, v0, #0
; CHECK-NEXT: ushll.2d v0, v0, #0 ; CHECK-NEXT: ushll.2d v0, v0, #0
; CHECK-NEXT: stp q0, q1, [x9], #128 ; CHECK-NEXT: stp q1, q6, [x9, #64]
; CHECK-NEXT: stp q3, q2, [x9, #32]
; CHECK-NEXT: stp q0, q4, [x9], #128
; CHECK-NEXT: b.ne LBB18_1 ; CHECK-NEXT: b.ne LBB18_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: ret ; CHECK-NEXT: ret
Context not available.
; CHECK-BE-NEXT: .LBB18_1: // %loop ; CHECK-BE-NEXT: .LBB18_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x10, x0, x8 ; CHECK-BE-NEXT: add x10, x0, x8
; CHECK-BE-NEXT: sub x11, x9, #32
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: add x11, x10, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v0.16b }, [x10] ; CHECK-BE-NEXT: ld1 { v0.16b }, [x10]
; CHECK-BE-NEXT: add x10, x10, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v5.16b }, [x10]
; CHECK-BE-NEXT: sub x10, x9, #16 ; CHECK-BE-NEXT: sub x10, x9, #16
; CHECK-BE-NEXT: ld1 { v3.16b }, [x11]
; CHECK-BE-NEXT: sub x11, x9, #32
; CHECK-BE-NEXT: ushll2 v1.8h, v0.16b, #0 ; CHECK-BE-NEXT: ushll2 v1.8h, v0.16b, #0
; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0 ; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-BE-NEXT: ushll2 v2.4s, v1.8h, #0 ; CHECK-BE-NEXT: ushll2 v2.4s, v1.8h, #0
; CHECK-BE-NEXT: ushll2 v3.4s, v0.8h, #0
; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0 ; CHECK-BE-NEXT: ushll v1.4s, v1.4h, #0
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: ushll2 v4.2d, v2.4s, #0 ; CHECK-BE-NEXT: ushll2 v4.2d, v2.4s, #0
; CHECK-BE-NEXT: ushll v2.2d, v2.2s, #0 ; CHECK-BE-NEXT: ushll v2.2d, v2.2s, #0
; CHECK-BE-NEXT: ushll2 v6.2d, v1.4s, #0
; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0
; CHECK-BE-NEXT: st1 { v4.2d }, [x10] ; CHECK-BE-NEXT: st1 { v4.2d }, [x10]
; CHECK-BE-NEXT: sub x10, x9, #48 ; CHECK-BE-NEXT: ushll2 v4.2d, v3.4s, #0
; CHECK-BE-NEXT: ushll2 v4.2d, v1.4s, #0 ; CHECK-BE-NEXT: ushll v3.2d, v3.2s, #0
; CHECK-BE-NEXT: st1 { v2.2d }, [x11] ; CHECK-BE-NEXT: st1 { v2.2d }, [x11]
; CHECK-BE-NEXT: ushll2 v2.4s, v0.8h, #0 ; CHECK-BE-NEXT: ushll2 v2.8h, v5.16b, #0
; CHECK-BE-NEXT: sub x11, x9, #80 ; CHECK-BE-NEXT: sub x11, x9, #80
; CHECK-BE-NEXT: st1 { v4.2d }, [x10] ; CHECK-BE-NEXT: sub x10, x9, #48
; CHECK-BE-NEXT: sub x10, x9, #64 ; CHECK-BE-NEXT: st1 { v4.2d }, [x11]
; CHECK-BE-NEXT: ushll2 v5.2d, v2.4s, #0 ; CHECK-BE-NEXT: ushll v4.8h, v5.8b, #0
; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0 ; CHECK-BE-NEXT: sub x11, x9, #64
; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0 ; CHECK-BE-NEXT: ushll2 v5.4s, v2.8h, #0
; CHECK-BE-NEXT: st1 { v5.2d }, [x11] ; CHECK-BE-NEXT: st1 { v1.2d }, [x11]
; CHECK-BE-NEXT: sub x11, x9, #96 ; CHECK-BE-NEXT: sub x11, x9, #96
; CHECK-BE-NEXT: st1 { v1.2d }, [x10] ; CHECK-BE-NEXT: ushll2 v1.2d, v0.4s, #0
; CHECK-BE-NEXT: ushll v4.8h, v3.8b, #0 ; CHECK-BE-NEXT: ushll v2.4s, v2.4h, #0
; CHECK-BE-NEXT: sub x10, x9, #112 ; CHECK-BE-NEXT: ushll v0.2d, v0.2s, #0
; CHECK-BE-NEXT: ushll2 v3.8h, v3.16b, #0 ; CHECK-BE-NEXT: st1 { v6.2d }, [x10]
; CHECK-BE-NEXT: ushll v5.2d, v0.2s, #0 ; CHECK-BE-NEXT: sub x10, x9, #128
; CHECK-BE-NEXT: ushll2 v0.2d, v0.4s, #0 ; CHECK-BE-NEXT: st1 { v3.2d }, [x11]
; CHECK-BE-NEXT: ushll v2.2d, v2.2s, #0 ; CHECK-BE-NEXT: ushll2 v3.4s, v4.8h, #0
; CHECK-BE-NEXT: ushll v1.4s, v4.4h, #0 ; CHECK-BE-NEXT: ushll2 v6.2d, v5.4s, #0
; CHECK-BE-NEXT: sub x11, x9, #112
; CHECK-BE-NEXT: ushll v5.2d, v5.2s, #0
; CHECK-BE-NEXT: st1 { v0.2d }, [x10] ; CHECK-BE-NEXT: st1 { v0.2d }, [x10]
; CHECK-BE-NEXT: ushll2 v6.4s, v3.8h, #0 ; CHECK-BE-NEXT: st1 { v1.2d }, [x11]
; CHECK-BE-NEXT: st1 { v2.2d }, [x11] ; CHECK-BE-NEXT: ushll2 v1.2d, v2.4s, #0
; CHECK-BE-NEXT: sub x11, x9, #128
; CHECK-BE-NEXT: add x10, x9, #112 ; CHECK-BE-NEXT: add x10, x9, #112
; CHECK-BE-NEXT: ushll v2.2d, v1.2s, #0 ; CHECK-BE-NEXT: ushll v4.4s, v4.4h, #0
; CHECK-BE-NEXT: ushll2 v0.2d, v1.4s, #0 ; CHECK-BE-NEXT: ushll2 v0.2d, v3.4s, #0
; CHECK-BE-NEXT: ushll2 v1.2d, v6.4s, #0 ; CHECK-BE-NEXT: st1 { v6.2d }, [x10]
; CHECK-BE-NEXT: st1 { v5.2d }, [x11] ; CHECK-BE-NEXT: add x10, x9, #96
; CHECK-BE-NEXT: ushll v3.4s, v3.4h, #0 ; CHECK-BE-NEXT: ushll v2.2d, v2.2s, #0
; CHECK-BE-NEXT: add x11, x9, #96 ; CHECK-BE-NEXT: ushll v3.2d, v3.2s, #0
; CHECK-BE-NEXT: st1 { v1.2d }, [x10] ; CHECK-BE-NEXT: st1 { v5.2d }, [x10]
; CHECK-BE-NEXT: add x10, x9, #80 ; CHECK-BE-NEXT: add x10, x9, #80
; CHECK-BE-NEXT: ushll v5.2d, v6.2s, #0 ; CHECK-BE-NEXT: st1 { v1.2d }, [x10]
; CHECK-BE-NEXT: st1 { v2.2d }, [x9] ; CHECK-BE-NEXT: add x10, x9, #48
; CHECK-BE-NEXT: ushll2 v1.4s, v4.8h, #0 ; CHECK-BE-NEXT: ushll2 v1.2d, v4.4s, #0
; CHECK-BE-NEXT: ushll2 v4.2d, v3.4s, #0 ; CHECK-BE-NEXT: st1 { v0.2d }, [x10]
; CHECK-BE-NEXT: st1 { v5.2d }, [x11] ; CHECK-BE-NEXT: ushll v0.2d, v4.2s, #0
; CHECK-BE-NEXT: add x11, x9, #48
; CHECK-BE-NEXT: st1 { v4.2d }, [x10]
; CHECK-BE-NEXT: add x10, x9, #64 ; CHECK-BE-NEXT: add x10, x9, #64
; CHECK-BE-NEXT: ushll2 v5.2d, v1.4s, #0 ; CHECK-BE-NEXT: st1 { v2.2d }, [x10]
; CHECK-BE-NEXT: ushll v3.2d, v3.2s, #0
; CHECK-BE-NEXT: st1 { v5.2d }, [x11]
; CHECK-BE-NEXT: add x11, x9, #16
; CHECK-BE-NEXT: st1 { v3.2d }, [x10]
; CHECK-BE-NEXT: add x10, x9, #32 ; CHECK-BE-NEXT: add x10, x9, #32
; CHECK-BE-NEXT: st1 { v3.2d }, [x10]
; CHECK-BE-NEXT: add x10, x9, #16
; CHECK-BE-NEXT: st1 { v0.2d }, [x9]
; CHECK-BE-NEXT: add x9, x9, #128 ; CHECK-BE-NEXT: add x9, x9, #128
; CHECK-BE-NEXT: ushll v1.2d, v1.2s, #0
; CHECK-BE-NEXT: st1 { v0.2d }, [x11]
; CHECK-BE-NEXT: st1 { v1.2d }, [x10] ; CHECK-BE-NEXT: st1 { v1.2d }, [x10]
; CHECK-BE-NEXT: b.ne .LBB18_1 ; CHECK-BE-NEXT: b.ne .LBB18_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v16i8_to_v16i32_in_loop_scalable_vectors(ptr %src, ptr %dst) { define void @zext_v16i8_to_v16i32_in_loop_scalable_vectors(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v16i8_to_v16i32_in_loop_scalable_vectors: ; CHECK-LABEL: zext_v16i8_to_v16i32_in_loop_scalable_vectors:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: ptrue p0.s ; CHECK-NEXT: ptrue p0.s
; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: LBB19_1: ; %loop ; CHECK-NEXT: LBB19_1: ; %loop
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: add x9, x0, x8
; CHECK-NEXT: ld1b { z0.s }, p0/z, [x0, x8] ; CHECK-NEXT: ld1b { z0.s }, p0/z, [x0, x8]
; CHECK-NEXT: add x9, x0, x8
; CHECK-NEXT: ld1b { z1.s }, p0/z, [x9, #2, mul vl] ; CHECK-NEXT: ld1b { z1.s }, p0/z, [x9, #2, mul vl]
; CHECK-NEXT: ld1b { z2.s }, p0/z, [x9, #3, mul vl] ; CHECK-NEXT: ld1b { z2.s }, p0/z, [x9, #3, mul vl]
; CHECK-NEXT: ld1b { z3.s }, p0/z, [x9, #1, mul vl] ; CHECK-NEXT: ld1b { z3.s }, p0/z, [x9, #1, mul vl]
; CHECK-NEXT: add z0.s, z0.s, z0.s
; CHECK-NEXT: add x9, x1, x8, lsl #2 ; CHECK-NEXT: add x9, x1, x8, lsl #2
; CHECK-NEXT: add z0.s, z0.s, z0.s
; CHECK-NEXT: add z1.s, z1.s, z1.s
; CHECK-NEXT: add z2.s, z2.s, z2.s
; CHECK-NEXT: add z3.s, z3.s, z3.s
; CHECK-NEXT: st1w { z0.s }, p0, [x1, x8, lsl #2] ; CHECK-NEXT: st1w { z0.s }, p0, [x1, x8, lsl #2]
; CHECK-NEXT: add x8, x8, #16 ; CHECK-NEXT: add x8, x8, #16
; CHECK-NEXT: cmp x8, #128 ; CHECK-NEXT: cmp x8, #128
; CHECK-NEXT: add z1.s, z1.s, z1.s
; CHECK-NEXT: add z0.s, z3.s, z3.s
; CHECK-NEXT: add z2.s, z2.s, z2.s
; CHECK-NEXT: st1w { z1.s }, p0, [x9, #2, mul vl] ; CHECK-NEXT: st1w { z1.s }, p0, [x9, #2, mul vl]
; CHECK-NEXT: st1w { z2.s }, p0, [x9, #3, mul vl] ; CHECK-NEXT: st1w { z2.s }, p0, [x9, #3, mul vl]
; CHECK-NEXT: st1w { z0.s }, p0, [x9, #1, mul vl] ; CHECK-NEXT: st1w { z3.s }, p0, [x9, #1, mul vl]
; CHECK-NEXT: b.ne LBB19_1 ; CHECK-NEXT: b.ne LBB19_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; ;
; CHECK-BE-LABEL: zext_v16i8_to_v16i32_in_loop_scalable_vectors: ; CHECK-BE-LABEL: zext_v16i8_to_v16i32_in_loop_scalable_vectors:
; CHECK-BE: // %bb.0: // %entry ; CHECK-BE: // %bb.0: // %entry
; CHECK-BE-NEXT: mov x8, xzr
; CHECK-BE-NEXT: ptrue p0.s ; CHECK-BE-NEXT: ptrue p0.s
; CHECK-BE-NEXT: mov x8, xzr
; CHECK-BE-NEXT: .LBB19_1: // %loop ; CHECK-BE-NEXT: .LBB19_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: ld1b { z0.s }, p0/z, [x0, x8] ; CHECK-BE-NEXT: ld1b { z0.s }, p0/z, [x0, x8]
; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: ld1b { z1.s }, p0/z, [x9, #2, mul vl] ; CHECK-BE-NEXT: ld1b { z1.s }, p0/z, [x9, #2, mul vl]
; CHECK-BE-NEXT: ld1b { z2.s }, p0/z, [x9, #3, mul vl] ; CHECK-BE-NEXT: ld1b { z2.s }, p0/z, [x9, #3, mul vl]
; CHECK-BE-NEXT: ld1b { z3.s }, p0/z, [x9, #1, mul vl] ; CHECK-BE-NEXT: ld1b { z3.s }, p0/z, [x9, #1, mul vl]
; CHECK-BE-NEXT: add z0.s, z0.s, z0.s
; CHECK-BE-NEXT: add x9, x1, x8, lsl #2 ; CHECK-BE-NEXT: add x9, x1, x8, lsl #2
; CHECK-BE-NEXT: add z0.s, z0.s, z0.s
; CHECK-BE-NEXT: add z1.s, z1.s, z1.s
; CHECK-BE-NEXT: add z2.s, z2.s, z2.s
; CHECK-BE-NEXT: add z3.s, z3.s, z3.s
; CHECK-BE-NEXT: st1w { z0.s }, p0, [x1, x8, lsl #2] ; CHECK-BE-NEXT: st1w { z0.s }, p0, [x1, x8, lsl #2]
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128 ; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: add z1.s, z1.s, z1.s
; CHECK-BE-NEXT: add z0.s, z3.s, z3.s
; CHECK-BE-NEXT: add z2.s, z2.s, z2.s
; CHECK-BE-NEXT: st1w { z1.s }, p0, [x9, #2, mul vl] ; CHECK-BE-NEXT: st1w { z1.s }, p0, [x9, #2, mul vl]
; CHECK-BE-NEXT: st1w { z2.s }, p0, [x9, #3, mul vl] ; CHECK-BE-NEXT: st1w { z2.s }, p0, [x9, #3, mul vl]
; CHECK-BE-NEXT: st1w { z0.s }, p0, [x9, #1, mul vl] ; CHECK-BE-NEXT: st1w { z3.s }, p0, [x9, #1, mul vl]
; CHECK-BE-NEXT: b.ne .LBB19_1 ; CHECK-BE-NEXT: b.ne .LBB19_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
; CHECK-LABEL: lCPI20_0:
; CHECK-NEXT: .byte 0 ; 0x0
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 1 ; 0x1
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 2 ; 0x2
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 3 ; 0x3
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 4 ; 0x4
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 5 ; 0x5
; CHECK-NEXT:lCPI20_1:
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 6 ; 0x6
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 7 ; 0x7
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 8 ; 0x8
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 9 ; 0x9
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 10 ; 0xa
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT:lCPI20_2:
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 11 ; 0xb
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 12 ; 0xc
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 13 ; 0xd
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 14 ; 0xe
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 15 ; 0xf
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT:lCPI20_3:
; CHECK-NEXT: .byte 0 ; 0x0
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 1 ; 0x1
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 2 ; 0x2
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 3 ; 0x3
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-BE-LABEL: .LCPI20_0:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 0 // 0x0
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 1 // 0x1
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 2 // 0x2
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 3 // 0x3
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .LCPI20_1:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 0 // 0x0
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 1 // 0x1
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 2 // 0x2
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 3 // 0x3
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 4 // 0x4
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .LCPI20_2:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 5 // 0x5
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 6 // 0x6
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 7 // 0x7
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 8 // 0x8
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 9 // 0x9
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .LCPI20_3:
; CHECK-BE-NEXT: .byte 10 // 0xa
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 11 // 0xb
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 12 // 0xc
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 13 // 0xd
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 14 // 0xe
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 15 // 0xf
define void @zext_v20i8_to_v20i24_in_loop(ptr %src, ptr %dst) { define void @zext_v20i8_to_v20i24_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v20i8_to_v20i24_in_loop: ; CHECK-LABEL: zext_v20i8_to_v20i24_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: Lloh22: ; CHECK-NEXT: Lloh22:
; CHECK-NEXT: adrp x9, lCPI20_0@PAGE ; CHECK-NEXT: adrp x8, lCPI20_0@PAGE
; CHECK-NEXT: Lloh23: ; CHECK-NEXT: Lloh23:
; CHECK-NEXT: adrp x10, lCPI20_1@PAGE ; CHECK-NEXT: adrp x9, lCPI20_1@PAGE
; CHECK-NEXT: Lloh24: ; CHECK-NEXT: Lloh24:
; CHECK-NEXT: adrp x11, lCPI20_2@PAGE ; CHECK-NEXT: adrp x10, lCPI20_2@PAGE
; CHECK-NEXT: Lloh25: ; CHECK-NEXT: Lloh25:
; CHECK-NEXT: adrp x12, lCPI20_3@PAGE ; CHECK-NEXT: ldr q0, [x8, lCPI20_0@PAGEOFF]
; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: Lloh26: ; CHECK-NEXT: Lloh26:
; CHECK-NEXT: ldr q0, [x9, lCPI20_0@PAGEOFF] ; CHECK-NEXT: adrp x8, lCPI20_3@PAGE
; CHECK-NEXT: Lloh27: ; CHECK-NEXT: Lloh27:
; CHECK-NEXT: ldr q1, [x10, lCPI20_1@PAGEOFF] ; CHECK-NEXT: ldr q1, [x9, lCPI20_1@PAGEOFF]
; CHECK-NEXT: Lloh28: ; CHECK-NEXT: Lloh28:
; CHECK-NEXT: ldr q2, [x11, lCPI20_2@PAGEOFF] ; CHECK-NEXT: ldr q2, [x10, lCPI20_2@PAGEOFF]
; CHECK-NEXT: Lloh29: ; CHECK-NEXT: Lloh29:
; CHECK-NEXT: ldr q3, [x12, lCPI20_3@PAGEOFF] ; CHECK-NEXT: ldr q3, [x8, lCPI20_3@PAGEOFF]
; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: LBB20_1: ; %loop ; CHECK-NEXT: LBB20_1: ; %loop
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: add x9, x0, x8 ; CHECK-NEXT: add x9, x0, x8
; CHECK-NEXT: add x8, x8, #16 ; CHECK-NEXT: add x8, x8, #16
; CHECK-NEXT: cmp x8, #128 ; CHECK-NEXT: ldp q5, q4, [x9]
; CHECK-NEXT: ldp q4, q5, [x9]
; CHECK-NEXT: add x9, x1, #56 ; CHECK-NEXT: add x9, x1, #56
; CHECK-NEXT: tbl.16b v6, { v4 }, v2 ; CHECK-NEXT: cmp x8, #128
; CHECK-NEXT: tbl.16b v7, { v4 }, v1 ; CHECK-NEXT: tbl.16b v4, { v4 }, v3
; CHECK-NEXT: tbl.16b v4, { v4 }, v0 ; CHECK-NEXT: tbl.16b v6, { v5 }, v2
; CHECK-NEXT: tbl.16b v5, { v5 }, v3 ; CHECK-NEXT: tbl.16b v7, { v5 }, v1
; CHECK-NEXT: tbl.16b v5, { v5 }, v0
; CHECK-NEXT: stp q7, q6, [x1, #16] ; CHECK-NEXT: stp q7, q6, [x1, #16]
; CHECK-NEXT: str q4, [x1] ; CHECK-NEXT: str q5, [x1]
; CHECK-NEXT: str d5, [x1, #48] ; CHECK-NEXT: str d4, [x1, #48]
; CHECK-NEXT: add x1, x1, #64 ; CHECK-NEXT: add x1, x1, #64
; CHECK-NEXT: st1.s { v5 }[2], [x9] ; CHECK-NEXT: st1.s { v4 }[2], [x9]
; CHECK-NEXT: b.ne LBB20_1 ; CHECK-NEXT: b.ne LBB20_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh25, Lloh29 ; CHECK-NEXT: .loh AdrpLdr Lloh26, Lloh29
; CHECK-NEXT: .loh AdrpLdr Lloh24, Lloh28 ; CHECK-NEXT: .loh AdrpLdr Lloh24, Lloh28
; CHECK-NEXT: .loh AdrpLdr Lloh23, Lloh27 ; CHECK-NEXT: .loh AdrpLdr Lloh23, Lloh27
; CHECK-NEXT: .loh AdrpLdr Lloh22, Lloh26 ; CHECK-NEXT: .loh AdrpAdrp Lloh22, Lloh26
; CHECK-NEXT: .loh AdrpLdr Lloh22, Lloh25
; ;
; CHECK-BE-LABEL: zext_v20i8_to_v20i24_in_loop: ; CHECK-BE-LABEL: zext_v20i8_to_v20i24_in_loop:
; CHECK-BE: // %bb.0: // %entry ; CHECK-BE: // %bb.0: // %entry
Context not available.
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: add x10, x9, #16 ; CHECK-BE-NEXT: add x10, x9, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v5.16b }, [x9] ; CHECK-BE-NEXT: ld1 { v5.16b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #32 ; CHECK-BE-NEXT: add x9, x1, #32
; CHECK-BE-NEXT: ld1 { v4.16b }, [x10] ; CHECK-BE-NEXT: ld1 { v4.16b }, [x10]
; CHECK-BE-NEXT: add x10, x1, #56 ; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: tbl v6.16b, { v5.16b }, v3.16b ; CHECK-BE-NEXT: tbl v6.16b, { v5.16b }, v3.16b
; CHECK-BE-NEXT: tbl v7.16b, { v5.16b }, v1.16b ; CHECK-BE-NEXT: tbl v7.16b, { v5.16b }, v2.16b
; CHECK-BE-NEXT: tbl v5.16b, { v5.16b }, v1.16b
; CHECK-BE-NEXT: tbl v4.16b, { v4.16b }, v0.16b ; CHECK-BE-NEXT: tbl v4.16b, { v4.16b }, v0.16b
; CHECK-BE-NEXT: tbl v5.16b, { v5.16b }, v2.16b
; CHECK-BE-NEXT: st1 { v6.16b }, [x9] ; CHECK-BE-NEXT: st1 { v6.16b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #16 ; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: st1 { v7.16b }, [x1] ; CHECK-BE-NEXT: rev32 v16.16b, v4.16b
; CHECK-BE-NEXT: rev64 v16.16b, v4.16b ; CHECK-BE-NEXT: rev64 v4.16b, v4.16b
; CHECK-BE-NEXT: rev32 v4.16b, v4.16b ; CHECK-BE-NEXT: st1 { v7.16b }, [x9]
; CHECK-BE-NEXT: st1 { v5.16b }, [x9] ; CHECK-BE-NEXT: add x9, x1, #56
; CHECK-BE-NEXT: str d16, [x1, #48] ; CHECK-BE-NEXT: st1 { v5.16b }, [x1]
; CHECK-BE-NEXT: str d4, [x1, #48]
; CHECK-BE-NEXT: add x1, x1, #64 ; CHECK-BE-NEXT: add x1, x1, #64
; CHECK-BE-NEXT: st1 { v4.s }[2], [x10] ; CHECK-BE-NEXT: st1 { v16.s }[2], [x9]
; CHECK-BE-NEXT: b.ne .LBB20_1 ; CHECK-BE-NEXT: b.ne .LBB20_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
; CHECK-LABEL: lCPI21_0:
; CHECK-NEXT: .byte 0 ; 0x0
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 1 ; 0x1
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 2 ; 0x2
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: lCPI21_1:
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 3 ; 0x3
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 4 ; 0x4
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 5 ; 0x5
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: lCPI21_2:
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 6 ; 0x6
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 7 ; 0x7
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: lCPI21_3:
; CHECK-NEXT: .byte 8 ; 0x8
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 9 ; 0x9
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 10 ; 0xa
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: lCPI21_4:
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 11 ; 0xb
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 12 ; 0xc
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 13 ; 0xd
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: lCPI21_5:
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 14 ; 0xe
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 15 ; 0xf
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-NEXT: .byte 255 ; 0xff
; CHECK-BE-LABEL: .LCPI21_0:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 5 // 0x5
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 6 // 0x6
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .LCPI21_1:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 0 // 0x0
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 1 // 0x1
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .LCPI21_2:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 2 // 0x2
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 3 // 0x3
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 4 // 0x4
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .LCPI21_3:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 5 // 0x5
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 6 // 0x6
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 7 // 0x7
; CHECK-BE-NEXT: .LCPI21_4:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 8 // 0x8
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 9 // 0x9
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .LCPI21_5:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 10 // 0xa
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 11 // 0xb
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 12 // 0xc
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .LCPI21_6:
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 13 // 0xd
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 14 // 0xe
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 255 // 0xff
; CHECK-BE-NEXT: .byte 15 // 0xf
define void @zext_v23i8_to_v23i48_in_loop(ptr %src, ptr %dst) { define void @zext_v23i8_to_v23i48_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v23i8_to_v23i48_in_loop: ; CHECK-LABEL: zext_v23i8_to_v23i48_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: Lloh30: ; CHECK-NEXT: Lloh30:
; CHECK-NEXT: adrp x9, lCPI21_0@PAGE ; CHECK-NEXT: adrp x8, lCPI21_0@PAGE
; CHECK-NEXT: Lloh31: ; CHECK-NEXT: Lloh31:
; CHECK-NEXT: adrp x10, lCPI21_1@PAGE ; CHECK-NEXT: adrp x9, lCPI21_1@PAGE
; CHECK-NEXT: Lloh32: ; CHECK-NEXT: Lloh32:
; CHECK-NEXT: adrp x11, lCPI21_2@PAGE ; CHECK-NEXT: adrp x10, lCPI21_2@PAGE
; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: Lloh33: ; CHECK-NEXT: Lloh33:
; CHECK-NEXT: ldr q0, [x9, lCPI21_0@PAGEOFF] ; CHECK-NEXT: ldr q0, [x8, lCPI21_0@PAGEOFF]
; CHECK-NEXT: Lloh34: ; CHECK-NEXT: Lloh34:
; CHECK-NEXT: adrp x9, lCPI21_3@PAGE ; CHECK-NEXT: ldr q1, [x9, lCPI21_1@PAGEOFF]
; CHECK-NEXT: Lloh35: ; CHECK-NEXT: Lloh35:
; CHECK-NEXT: ldr q1, [x10, lCPI21_1@PAGEOFF] ; CHECK-NEXT: ldr q2, [x10, lCPI21_2@PAGEOFF]
; CHECK-NEXT: Lloh36: ; CHECK-NEXT: Lloh36:
; CHECK-NEXT: adrp x10, lCPI21_4@PAGE ; CHECK-NEXT: adrp x8, lCPI21_3@PAGE
; CHECK-NEXT: Lloh37: ; CHECK-NEXT: Lloh37:
; CHECK-NEXT: ldr q2, [x11, lCPI21_2@PAGEOFF] ; CHECK-NEXT: adrp x9, lCPI21_4@PAGE
; CHECK-NEXT: Lloh38: ; CHECK-NEXT: Lloh38:
; CHECK-NEXT: adrp x11, lCPI21_5@PAGE ; CHECK-NEXT: adrp x10, lCPI21_5@PAGE
; CHECK-NEXT: Lloh39: ; CHECK-NEXT: Lloh39:
; CHECK-NEXT: ldr q3, [x9, lCPI21_3@PAGEOFF] ; CHECK-NEXT: ldr q3, [x8, lCPI21_3@PAGEOFF]
; CHECK-NEXT: Lloh40: ; CHECK-NEXT: Lloh40:
; CHECK-NEXT: ldr q4, [x10, lCPI21_4@PAGEOFF] ; CHECK-NEXT: ldr q4, [x9, lCPI21_4@PAGEOFF]
; CHECK-NEXT: Lloh41: ; CHECK-NEXT: Lloh41:
; CHECK-NEXT: ldr q5, [x11, lCPI21_5@PAGEOFF] ; CHECK-NEXT: ldr q5, [x10, lCPI21_5@PAGEOFF]
; CHECK-NEXT: mov x8, xzr
; CHECK-NEXT: LBB21_1: ; %loop ; CHECK-NEXT: LBB21_1: ; %loop
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: add x9, x0, x8 ; CHECK-NEXT: add x9, x0, x8
; CHECK-NEXT: movi.2d v19, #0000000000000000
; CHECK-NEXT: add x8, x8, #16 ; CHECK-NEXT: add x8, x8, #16
; CHECK-NEXT: movi.2d v6, #0000000000000000 ; CHECK-NEXT: ldp q7, q6, [x9]
; CHECK-NEXT: cmp x8, #128 ; CHECK-NEXT: cmp x8, #128
; CHECK-NEXT: ldp q16, q7, [x9]
; CHECK-NEXT: strh wzr, [x1, #136] ; CHECK-NEXT: strh wzr, [x1, #136]
; CHECK-NEXT: tbl.16b v18, { v16 }, v5 ; CHECK-NEXT: tbl.16b v16, { v6 }, v1
; CHECK-NEXT: tbl.16b v19, { v16 }, v4 ; CHECK-NEXT: tbl.16b v17, { v6 }, v0
; CHECK-NEXT: mov.b v6[4], v7[6] ; CHECK-NEXT: mov.b v19[4], v6[6]
; CHECK-NEXT: tbl.16b v18, { v7 }, v5
; CHECK-NEXT: tbl.16b v20, { v7 }, v4
; CHECK-NEXT: tbl.16b v21, { v7 }, v3
; CHECK-NEXT: stp q17, q16, [x1, #96]
; CHECK-NEXT: tbl.16b v16, { v7 }, v2
; CHECK-NEXT: tbl.16b v17, { v7 }, v1 ; CHECK-NEXT: tbl.16b v17, { v7 }, v1
; CHECK-NEXT: tbl.16b v7, { v7 }, v0 ; CHECK-NEXT: tbl.16b v7, { v7 }, v0
; CHECK-NEXT: tbl.16b v20, { v16 }, v3 ; CHECK-NEXT: fmov x9, d19
; CHECK-NEXT: stp q19, q18, [x1, #64] ; CHECK-NEXT: stp q20, q18, [x1, #64]
; CHECK-NEXT: fmov x9, d6 ; CHECK-NEXT: stp q16, q21, [x1, #32]
; CHECK-NEXT: stp q7, q17, [x1, #96] ; CHECK-NEXT: stp q7, q17, [x1]
; CHECK-NEXT: tbl.16b v17, { v16 }, v2
; CHECK-NEXT: tbl.16b v7, { v16 }, v1
; CHECK-NEXT: tbl.16b v16, { v16 }, v0
; CHECK-NEXT: stp q17, q20, [x1, #32]
; CHECK-NEXT: stp q16, q7, [x1]
; CHECK-NEXT: str x9, [x1, #128]! ; CHECK-NEXT: str x9, [x1, #128]!
; CHECK-NEXT: b.ne LBB21_1 ; CHECK-NEXT: b.ne LBB21_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh38, Lloh41 ; CHECK-NEXT: .loh AdrpLdr Lloh38, Lloh41
; CHECK-NEXT: .loh AdrpLdr Lloh36, Lloh40 ; CHECK-NEXT: .loh AdrpLdr Lloh37, Lloh40
; CHECK-NEXT: .loh AdrpLdr Lloh34, Lloh39 ; CHECK-NEXT: .loh AdrpLdr Lloh36, Lloh39
; CHECK-NEXT: .loh AdrpAdrp Lloh32, Lloh38 ; CHECK-NEXT: .loh AdrpAdrp Lloh32, Lloh38
; CHECK-NEXT: .loh AdrpLdr Lloh32, Lloh37 ; CHECK-NEXT: .loh AdrpLdr Lloh32, Lloh35
; CHECK-NEXT: .loh AdrpAdrp Lloh31, Lloh36 ; CHECK-NEXT: .loh AdrpAdrp Lloh31, Lloh37
; CHECK-NEXT: .loh AdrpLdr Lloh31, Lloh35 ; CHECK-NEXT: .loh AdrpLdr Lloh31, Lloh34
; CHECK-NEXT: .loh AdrpAdrp Lloh30, Lloh34 ; CHECK-NEXT: .loh AdrpAdrp Lloh30, Lloh36
; CHECK-NEXT: .loh AdrpLdr Lloh30, Lloh33 ; CHECK-NEXT: .loh AdrpLdr Lloh30, Lloh33
; ;
; CHECK-BE-LABEL: zext_v23i8_to_v23i48_in_loop: ; CHECK-BE-LABEL: zext_v23i8_to_v23i48_in_loop:
Context not available.
; CHECK-BE-NEXT: .LBB21_1: // %loop ; CHECK-BE-NEXT: .LBB21_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x11, x1, #64
; CHECK-BE-NEXT: add x10, x1, #80
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v7.16b }, [x9] ; CHECK-BE-NEXT: ld1 { v7.16b }, [x9]
; CHECK-BE-NEXT: add x9, x9, #16 ; CHECK-BE-NEXT: add x9, x9, #16
; CHECK-BE-NEXT: ld1 { v18.16b }, [x9] ; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: add x9, x1, #48 ; CHECK-BE-NEXT: ld1 { v17.16b }, [x9]
; CHECK-BE-NEXT: tbl v17.16b, { v7.16b }, v5.16b ; CHECK-BE-NEXT: add x9, x1, #80
; CHECK-BE-NEXT: tbl v16.16b, { v7.16b }, v6.16b ; CHECK-BE-NEXT: tbl v16.16b, { v7.16b }, v6.16b
; CHECK-BE-NEXT: tbl v19.16b, { v7.16b }, v3.16b ; CHECK-BE-NEXT: tbl v18.16b, { v7.16b }, v5.16b
; CHECK-BE-NEXT: tbl v20.16b, { v18.16b }, v0.16b ; CHECK-BE-NEXT: tbl v19.16b, { v7.16b }, v4.16b
; CHECK-BE-NEXT: st1 { v17.16b }, [x11] ; CHECK-BE-NEXT: tbl v20.16b, { v7.16b }, v3.16b
; CHECK-BE-NEXT: add x11, x1, #16 ; CHECK-BE-NEXT: tbl v21.16b, { v17.16b }, v0.16b
; CHECK-BE-NEXT: tbl v17.16b, { v7.16b }, v4.16b ; CHECK-BE-NEXT: st1 { v16.16b }, [x9]
; CHECK-BE-NEXT: st1 { v16.16b }, [x10] ; CHECK-BE-NEXT: add x9, x1, #64
; CHECK-BE-NEXT: add x10, x1, #32 ; CHECK-BE-NEXT: tbl v16.16b, { v7.16b }, v2.16b
; CHECK-BE-NEXT: tbl v16.16b, { v7.16b }, v1.16b ; CHECK-BE-NEXT: st1 { v18.16b }, [x9]
; CHECK-BE-NEXT: tbl v7.16b, { v7.16b }, v2.16b ; CHECK-BE-NEXT: add x9, x1, #48
; CHECK-BE-NEXT: tbl v21.16b, { v18.16b }, v1.16b ; CHECK-BE-NEXT: tbl v18.16b, { v17.16b }, v2.16b
; CHECK-BE-NEXT: st1 { v17.16b }, [x9] ; CHECK-BE-NEXT: st1 { v19.16b }, [x9]
; CHECK-BE-NEXT: tbl v17.16b, { v18.16b }, v2.16b ; CHECK-BE-NEXT: add x9, x1, #32
; CHECK-BE-NEXT: tbl v17.16b, { v17.16b }, v1.16b
; CHECK-BE-NEXT: st1 { v20.16b }, [x9]
; CHECK-BE-NEXT: add x9, x1, #16
; CHECK-BE-NEXT: rev64 v19.16b, v21.16b
; CHECK-BE-NEXT: st1 { v16.16b }, [x9]
; CHECK-BE-NEXT: rev16 v16.16b, v21.16b
; CHECK-BE-NEXT: add x9, x1, #112 ; CHECK-BE-NEXT: add x9, x1, #112
; CHECK-BE-NEXT: rev64 v18.16b, v20.16b ; CHECK-BE-NEXT: st1 { v18.16b }, [x9]
; CHECK-BE-NEXT: st1 { v19.16b }, [x10] ; CHECK-BE-NEXT: add x9, x1, #96
; CHECK-BE-NEXT: rev16 v19.16b, v20.16b ; CHECK-BE-NEXT: tbl v7.16b, { v7.16b }, v1.16b
; CHECK-BE-NEXT: add x10, x1, #96
; CHECK-BE-NEXT: st1 { v7.16b }, [x11]
; CHECK-BE-NEXT: add x11, x1, #136
; CHECK-BE-NEXT: st1 { v17.16b }, [x9] ; CHECK-BE-NEXT: st1 { v17.16b }, [x9]
; CHECK-BE-NEXT: fmov x9, d18 ; CHECK-BE-NEXT: add x9, x1, #136
; CHECK-BE-NEXT: st1 { v21.16b }, [x10] ; CHECK-BE-NEXT: st1 { v16.h }[4], [x9]
; CHECK-BE-NEXT: st1 { v19.h }[4], [x11] ; CHECK-BE-NEXT: fmov x9, d19
; CHECK-BE-NEXT: st1 { v16.16b }, [x1] ; CHECK-BE-NEXT: st1 { v7.16b }, [x1]
; CHECK-BE-NEXT: str x9, [x1, #128]! ; CHECK-BE-NEXT: str x9, [x1, #128]!
; CHECK-BE-NEXT: b.ne .LBB21_1 ; CHECK-BE-NEXT: b.ne .LBB21_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
define void @zext_v8i8_to_v8i33_in_loop(ptr %src, ptr %dst) { define void @zext_v8i8_to_v8i33_in_loop(ptr %src, ptr %dst) {
;
; CHECK-LABEL: zext_v8i8_to_v8i33_in_loop: ; CHECK-LABEL: zext_v8i8_to_v8i33_in_loop:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, xzr ; CHECK-NEXT: mov x8, xzr
Context not available.
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldr d0, [x0, x8] ; CHECK-NEXT: ldr d0, [x0, x8]
; CHECK-NEXT: add x8, x8, #16 ; CHECK-NEXT: add x8, x8, #16
; CHECK-NEXT: cmp x8, #128
; CHECK-NEXT: strb wzr, [x1, #32] ; CHECK-NEXT: strb wzr, [x1, #32]
; CHECK-NEXT: cmp x8, #128
; CHECK-NEXT: ushll.8h v0, v0, #0 ; CHECK-NEXT: ushll.8h v0, v0, #0
; CHECK-NEXT: ushll.4s v1, v0, #0 ; CHECK-NEXT: ushll2.4s v1, v0, #0
; CHECK-NEXT: ushll2.4s v0, v0, #0 ; CHECK-NEXT: ushll.4s v0, v0, #0
; CHECK-NEXT: ushll.2d v2, v1, #0 ; CHECK-NEXT: ushll2.2d v2, v1, #0
; CHECK-NEXT: ushll.2d v1, v1, #0
; CHECK-NEXT: ushll2.2d v3, v0, #0 ; CHECK-NEXT: ushll2.2d v3, v0, #0
; CHECK-NEXT: ushll2.2d v1, v1, #0
; CHECK-NEXT: mov.d x9, v3[1]
; CHECK-NEXT: fmov x10, d3
; CHECK-NEXT: mov.d x12, v1[1]
; CHECK-NEXT: ushll.2d v0, v0, #0 ; CHECK-NEXT: ushll.2d v0, v0, #0
; CHECK-NEXT: lsl x9, x9, #39 ; CHECK-NEXT: mov.d x9, v2[1]
; CHECK-NEXT: orr x9, x9, x10, lsl #6 ; CHECK-NEXT: mov.d x10, v1[1]
; CHECK-NEXT: fmov x10, d1
; CHECK-NEXT: mov.d x11, v0[1]
; CHECK-NEXT: lsl x12, x12, #35
; CHECK-NEXT: mov.d x14, v2[1]
; CHECK-NEXT: fmov x13, d0
; CHECK-NEXT: orr x10, x12, x10, lsl #2
; CHECK-NEXT: fmov x12, d2 ; CHECK-NEXT: fmov x12, d2
; CHECK-NEXT: lsl x11, x11, #37 ; CHECK-NEXT: mov.d x11, v3[1]
; CHECK-NEXT: orr x11, x11, x13, lsl #4 ; CHECK-NEXT: mov.d x13, v0[1]
; CHECK-NEXT: orr x12, x12, x14, lsl #33 ; CHECK-NEXT: lsl x9, x9, #39
; CHECK-NEXT: stp x11, x9, [x1, #16] ; CHECK-NEXT: lsl x10, x10, #37
; CHECK-NEXT: stp x12, x10, [x1], #128 ; CHECK-NEXT: lsl x11, x11, #35
; CHECK-NEXT: orr x9, x9, x12, lsl #6
; CHECK-NEXT: fmov x12, d1
; CHECK-NEXT: orr x10, x10, x12, lsl #4
; CHECK-NEXT: fmov x12, d3
; CHECK-NEXT: stp x10, x9, [x1, #16]
; CHECK-NEXT: orr x11, x11, x12, lsl #2
; CHECK-NEXT: fmov x12, d0
; CHECK-NEXT: orr x9, x12, x13, lsl #33
; CHECK-NEXT: stp x9, x11, [x1], #128
; CHECK-NEXT: b.ne LBB22_1 ; CHECK-NEXT: b.ne LBB22_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: ret ; CHECK-NEXT: ret
Context not available.
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: add x9, x0, x8 ; CHECK-BE-NEXT: add x9, x0, x8
; CHECK-BE-NEXT: add x8, x8, #16 ; CHECK-BE-NEXT: add x8, x8, #16
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ld1 { v0.8b }, [x9] ; CHECK-BE-NEXT: ld1 { v0.8b }, [x9]
; CHECK-BE-NEXT: cmp x8, #128
; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0 ; CHECK-BE-NEXT: ushll v0.8h, v0.8b, #0
; CHECK-BE-NEXT: ushll v1.4s, v0.4h, #0 ; CHECK-BE-NEXT: ushll2 v1.4s, v0.8h, #0
; CHECK-BE-NEXT: ushll2 v0.4s, v0.8h, #0 ; CHECK-BE-NEXT: ushll v0.4s, v0.4h, #0
; CHECK-BE-NEXT: ushll v2.2d, v1.2s, #0 ; CHECK-BE-NEXT: ushll v2.2d, v1.2s, #0
; CHECK-BE-NEXT: ushll2 v1.2d, v1.4s, #0
; CHECK-BE-NEXT: ushll2 v3.2d, v0.4s, #0 ; CHECK-BE-NEXT: ushll2 v3.2d, v0.4s, #0
; CHECK-BE-NEXT: ushll v0.2d, v0.2s, #0 ; CHECK-BE-NEXT: ushll v0.2d, v0.2s, #0
; CHECK-BE-NEXT: mov x9, v3.d[1] ; CHECK-BE-NEXT: mov x9, v2.d[1]
; CHECK-BE-NEXT: ushll2 v1.2d, v1.4s, #0 ; CHECK-BE-NEXT: mov x10, v1.d[1]
; CHECK-BE-NEXT: fmov x10, d3 ; CHECK-BE-NEXT: fmov x13, d1
; CHECK-BE-NEXT: mov x11, v0.d[1] ; CHECK-BE-NEXT: mov x11, v3.d[1]
; CHECK-BE-NEXT: mov x12, v0.d[1]
; CHECK-BE-NEXT: fmov x14, d2
; CHECK-BE-NEXT: fmov x15, d3
; CHECK-BE-NEXT: lsl x9, x9, #2
; CHECK-BE-NEXT: orr x13, x10, x13, lsl #33
; CHECK-BE-NEXT: strb w10, [x1, #32]
; CHECK-BE-NEXT: lsl x11, x11, #4
; CHECK-BE-NEXT: lsl x12, x12, #6
; CHECK-BE-NEXT: orr x14, x9, x14, lsl #35
; CHECK-BE-NEXT: extr x9, x9, x13, #8
; CHECK-BE-NEXT: fmov x13, d0 ; CHECK-BE-NEXT: fmov x13, d0
; CHECK-BE-NEXT: mov x12, v1.d[1] ; CHECK-BE-NEXT: orr x15, x11, x15, lsl #37
; CHECK-BE-NEXT: strb w9, [x1, #32] ; CHECK-BE-NEXT: extr x10, x11, x14, #8
; CHECK-BE-NEXT: orr x10, x9, x10, lsl #33 ; CHECK-BE-NEXT: orr x11, x12, x13, lsl #39
; CHECK-BE-NEXT: fmov x15, d1 ; CHECK-BE-NEXT: extr x12, x12, x15, #8
; CHECK-BE-NEXT: mov x14, v2.d[1] ; CHECK-BE-NEXT: stp x10, x9, [x1, #16]
; CHECK-BE-NEXT: lsl x11, x11, #2 ; CHECK-BE-NEXT: lsr x9, x11, #8
; CHECK-BE-NEXT: lsl x12, x12, #4 ; CHECK-BE-NEXT: stp x9, x12, [x1], #128
; CHECK-BE-NEXT: orr x13, x11, x13, lsl #35
; CHECK-BE-NEXT: extr x10, x11, x10, #8
; CHECK-BE-NEXT: fmov x11, d2
; CHECK-BE-NEXT: orr x15, x12, x15, lsl #37
; CHECK-BE-NEXT: lsl x14, x14, #6
; CHECK-BE-NEXT: extr x9, x12, x13, #8
; CHECK-BE-NEXT: orr x11, x14, x11, lsl #39
; CHECK-BE-NEXT: extr x12, x14, x15, #8
; CHECK-BE-NEXT: lsr x11, x11, #8
; CHECK-BE-NEXT: stp x9, x10, [x1, #16]
; CHECK-BE-NEXT: stp x11, x12, [x1], #128
; CHECK-BE-NEXT: b.ne .LBB22_1 ; CHECK-BE-NEXT: b.ne .LBB22_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i64 [ 0, %entry ], [ %iv.next, %loop ]
%src.gep = getelementptr i8, ptr %src, i64 %iv %src.gep = getelementptr i8, ptr %src, i64 %iv
Context not available.
%iv.next = add nuw i64 %iv, 16 %iv.next = add nuw i64 %iv, 16
%ec = icmp eq i64 %iv.next, 128 %ec = icmp eq i64 %iv.next, 128
br i1 %ec, label %exit, label %loop br i1 %ec, label %exit, label %loop
exit: exit:
ret void ret void
} }
; FIXME: Widening instructions should be used instead of tbl. ; FIXME: Widening instructions should be used instead of tbl.
define i32 @test_pr62620_widening_instr(ptr %p1, ptr %p2, i64 %lx, i32 %h) { define i32 @test_pr62620_widening_instr(ptr %p1, ptr %p2, i64 %lx, i32 %h) {
;
; CHECK-LABEL: test_pr62620_widening_instr: ; CHECK-LABEL: test_pr62620_widening_instr:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, x0
; CHECK-NEXT: mov w0, wzr
; CHECK-NEXT: lsl x9, x2, #4 ; CHECK-NEXT: lsl x9, x2, #4
; CHECK-NEXT: mov w8, wzr
; CHECK-NEXT: LBB23_1: ; %loop ; CHECK-NEXT: LBB23_1: ; %loop
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldr q0, [x8, x9] ; CHECK-NEXT: ldr q0, [x0, x9]
; CHECK-NEXT: subs w3, w3, #1
; CHECK-NEXT: ldr q1, [x1, x9] ; CHECK-NEXT: ldr q1, [x1, x9]
; CHECK-NEXT: subs w3, w3, #1
; CHECK-NEXT: uabdl.8h v2, v0, v1 ; CHECK-NEXT: uabdl.8h v2, v0, v1
; CHECK-NEXT: uabal2.8h v2, v0, v1 ; CHECK-NEXT: uabal2.8h v2, v0, v1
; CHECK-NEXT: uaddlv.8h s0, v2 ; CHECK-NEXT: uaddlv.8h s0, v2
; CHECK-NEXT: fmov w10, s0 ; CHECK-NEXT: fmov w10, s0
; CHECK-NEXT: add w0, w10, w0 ; CHECK-NEXT: add w8, w10, w8
; CHECK-NEXT: b.ne LBB23_1 ; CHECK-NEXT: b.ne LBB23_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: mov w0, w8
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; ;
; CHECK-BE-LABEL: test_pr62620_widening_instr: ; CHECK-BE-LABEL: test_pr62620_widening_instr:
; CHECK-BE: // %bb.0: // %entry ; CHECK-BE: // %bb.0: // %entry
; CHECK-BE-NEXT: mov x8, x0
; CHECK-BE-NEXT: lsl x9, x2, #4 ; CHECK-BE-NEXT: lsl x9, x2, #4
; CHECK-BE-NEXT: mov x8, x0
; CHECK-BE-NEXT: mov w0, wzr ; CHECK-BE-NEXT: mov w0, wzr
; CHECK-BE-NEXT: add x8, x8, x9 ; CHECK-BE-NEXT: add x8, x8, x9
; CHECK-BE-NEXT: add x9, x1, x9 ; CHECK-BE-NEXT: add x9, x1, x9
; CHECK-BE-NEXT: .LBB23_1: // %loop ; CHECK-BE-NEXT: .LBB23_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: ld1 { v0.16b }, [x8] ; CHECK-BE-NEXT: ld1 { v0.16b }, [x8]
; CHECK-BE-NEXT: subs w3, w3, #1
; CHECK-BE-NEXT: ld1 { v1.16b }, [x9] ; CHECK-BE-NEXT: ld1 { v1.16b }, [x9]
; CHECK-BE-NEXT: subs w3, w3, #1
; CHECK-BE-NEXT: uabdl v2.8h, v0.8b, v1.8b ; CHECK-BE-NEXT: uabdl v2.8h, v0.8b, v1.8b
; CHECK-BE-NEXT: uabal2 v2.8h, v0.16b, v1.16b ; CHECK-BE-NEXT: uabal2 v2.8h, v0.16b, v1.16b
; CHECK-BE-NEXT: uaddlv s0, v2.8h ; CHECK-BE-NEXT: uaddlv s0, v2.8h
Context not available.
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%s0 = phi i32 [ 0, %entry ], [ %op.rdx, %loop ] %s0 = phi i32 [ 0, %entry ], [ %op.rdx, %loop ]
%j.0261 = phi i32 [ 0, %entry ], [ %inc, %loop ] %j.0261 = phi i32 [ 0, %entry ], [ %inc, %loop ]
Context not available.
%inc = add nuw nsw i32 %j.0261, 1 %inc = add nuw nsw i32 %j.0261, 1
%exitcond.not = icmp eq i32 %inc, %h %exitcond.not = icmp eq i32 %inc, %h
br i1 %exitcond.not, label %exit, label %loop br i1 %exitcond.not, label %exit, label %loop
exit: exit:
%s1 = phi i32 [ %op.rdx, %loop ] %s1 = phi i32 [ %op.rdx, %loop ]
ret i32 %s1 ret i32 %s1
} }
declare <16 x i32> @llvm.abs.v16i32(<16 x i32>, i1 immarg) declare <16 x i32> @llvm.abs.v16i32(<16 x i32>, i1 immarg)
declare i32 @llvm.vector.reduce.add.v16i32(<16 x i32>) declare i32 @llvm.vector.reduce.add.v16i32(<16 x i32>)
define i32 @test_widening_instr_mull(ptr %p1, ptr %p2, i32 %h) { define i32 @test_widening_instr_mull(ptr %p1, ptr %p2, i32 %h) {
;
; CHECK-LABEL: test_widening_instr_mull: ; CHECK-LABEL: test_widening_instr_mull:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: mov x8, x0 ; CHECK-NEXT: mov x8, x0
; CHECK-NEXT: LBB24_1: ; %loop ; CHECK-NEXT: LBB24_1: ; %loop
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldr q0, [x1], #16 ; CHECK-NEXT: ldr q0, [x1], #16
; CHECK-NEXT: ldr q1, [x0] ; CHECK-NEXT: ldr q3, [x0]
; CHECK-NEXT: subs w2, w2, #1
; CHECK-NEXT: ldr q2, [x8, #16]! ; CHECK-NEXT: ldr q2, [x8, #16]!
; CHECK-NEXT: ushll2.8h v3, v0, #0 ; CHECK-NEXT: subs w2, w2, #1
; CHECK-NEXT: ushll2.8h v1, v0, #0
; CHECK-NEXT: ushll.8h v0, v0, #0 ; CHECK-NEXT: ushll.8h v0, v0, #0
; CHECK-NEXT: umull2.4s v4, v2, v3 ; CHECK-NEXT: umull2.4s v4, v2, v1
; CHECK-NEXT: umull.4s v2, v2, v3 ; CHECK-NEXT: umull.4s v1, v2, v1
; CHECK-NEXT: umull.4s v3, v1, v0 ; CHECK-NEXT: umull2.4s v2, v3, v0
; CHECK-NEXT: umull2.4s v0, v1, v0 ; CHECK-NEXT: umull.4s v0, v3, v0
; CHECK-NEXT: stp q2, q4, [x0, #32] ; CHECK-NEXT: stp q1, q4, [x0, #32]
; CHECK-NEXT: str q3, [x0] ; CHECK-NEXT: str q0, [x0]
; CHECK-NEXT: mov x0, x8 ; CHECK-NEXT: mov x0, x8
; CHECK-NEXT: str q0, [x8] ; CHECK-NEXT: str q2, [x8]
; CHECK-NEXT: b.ne LBB24_1 ; CHECK-NEXT: b.ne LBB24_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: mov w0, wzr ; CHECK-NEXT: mov w0, wzr
Context not available.
; CHECK-BE-NEXT: .LBB24_1: // %loop ; CHECK-BE-NEXT: .LBB24_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: ld1 { v0.16b }, [x1] ; CHECK-BE-NEXT: ld1 { v0.16b }, [x1]
; CHECK-BE-NEXT: ld1 { v1.8h }, [x0]
; CHECK-BE-NEXT: add x8, x0, #16 ; CHECK-BE-NEXT: add x8, x0, #16
; CHECK-BE-NEXT: ld1 { v3.8h }, [x8]
; CHECK-BE-NEXT: add x9, x0, #48 ; CHECK-BE-NEXT: add x9, x0, #48
; CHECK-BE-NEXT: add x10, x0, #32 ; CHECK-BE-NEXT: add x10, x0, #32
; CHECK-BE-NEXT: ld1 { v1.8h }, [x0]
; CHECK-BE-NEXT: subs w2, w2, #1 ; CHECK-BE-NEXT: subs w2, w2, #1
; CHECK-BE-NEXT: add x1, x1, #16 ; CHECK-BE-NEXT: add x1, x1, #16
; CHECK-BE-NEXT: ld1 { v4.8h }, [x8]
; CHECK-BE-NEXT: ushll v2.8h, v0.8b, #0 ; CHECK-BE-NEXT: ushll v2.8h, v0.8b, #0
; CHECK-BE-NEXT: ushll2 v0.8h, v0.16b, #0 ; CHECK-BE-NEXT: ushll2 v0.8h, v0.16b, #0
; CHECK-BE-NEXT: umull v3.4s, v1.4h, v2.4h ; CHECK-BE-NEXT: umull v4.4s, v1.4h, v2.4h
; CHECK-BE-NEXT: umull2 v5.4s, v3.8h, v0.8h
; CHECK-BE-NEXT: umull v0.4s, v3.4h, v0.4h
; CHECK-BE-NEXT: umull2 v1.4s, v1.8h, v2.8h ; CHECK-BE-NEXT: umull2 v1.4s, v1.8h, v2.8h
; CHECK-BE-NEXT: umull2 v2.4s, v4.8h, v0.8h ; CHECK-BE-NEXT: st1 { v4.4s }, [x0]
; CHECK-BE-NEXT: umull v0.4s, v4.4h, v0.4h
; CHECK-BE-NEXT: st1 { v3.4s }, [x0]
; CHECK-BE-NEXT: mov x0, x8 ; CHECK-BE-NEXT: mov x0, x8
; CHECK-BE-NEXT: st1 { v1.4s }, [x8] ; CHECK-BE-NEXT: st1 { v5.4s }, [x9]
; CHECK-BE-NEXT: st1 { v2.4s }, [x9]
; CHECK-BE-NEXT: st1 { v0.4s }, [x10] ; CHECK-BE-NEXT: st1 { v0.4s }, [x10]
; CHECK-BE-NEXT: st1 { v1.4s }, [x8]
; CHECK-BE-NEXT: b.ne .LBB24_1 ; CHECK-BE-NEXT: b.ne .LBB24_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: mov w0, wzr ; CHECK-BE-NEXT: mov w0, wzr
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i32 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i32 [ 0, %entry ], [ %iv.next, %loop ]
%gep.1 = getelementptr inbounds <16 x i8>, ptr %p1, i32 %iv %gep.1 = getelementptr inbounds <16 x i8>, ptr %p1, i32 %iv
Context not available.
%iv.next= add nuw nsw i32 %iv, 1 %iv.next= add nuw nsw i32 %iv, 1
%exitcond.not = icmp eq i32 %iv.next, %h %exitcond.not = icmp eq i32 %iv.next, %h
br i1 %exitcond.not, label %exit, label %loop br i1 %exitcond.not, label %exit, label %loop
exit: exit:
ret i32 0 ret i32 0
} }
define i32 @test_widening_instr_mull_64(ptr %p1, ptr %p2, i32 %h) { define i32 @test_widening_instr_mull_64(ptr %p1, ptr %p2, i32 %h) {
;
; CHECK-LABEL: test_widening_instr_mull_64: ; CHECK-LABEL: test_widening_instr_mull_64:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: Lloh42: ; CHECK-NEXT: Lloh42:
Context not available.
; CHECK-NEXT: Lloh43: ; CHECK-NEXT: Lloh43:
; CHECK-NEXT: adrp x9, lCPI25_1@PAGE ; CHECK-NEXT: adrp x9, lCPI25_1@PAGE
; CHECK-NEXT: Lloh44: ; CHECK-NEXT: Lloh44:
; CHECK-NEXT: adrp x10, lCPI25_2@PAGE ; CHECK-NEXT: adrp x10, lCPI25_3@PAGE
; CHECK-NEXT: Lloh45: ; CHECK-NEXT: Lloh45:
; CHECK-NEXT: adrp x11, lCPI25_3@PAGE
; CHECK-NEXT: Lloh46:
; CHECK-NEXT: ldr q0, [x8, lCPI25_0@PAGEOFF] ; CHECK-NEXT: ldr q0, [x8, lCPI25_0@PAGEOFF]
; CHECK-NEXT: mov x8, x1 ; CHECK-NEXT: Lloh46:
; CHECK-NEXT: adrp x8, lCPI25_2@PAGE
; CHECK-NEXT: Lloh47: ; CHECK-NEXT: Lloh47:
; CHECK-NEXT: ldr q1, [x9, lCPI25_1@PAGEOFF] ; CHECK-NEXT: ldr q1, [x9, lCPI25_1@PAGEOFF]
; CHECK-NEXT: Lloh48: ; CHECK-NEXT: Lloh48:
; CHECK-NEXT: ldr q2, [x10, lCPI25_2@PAGEOFF] ; CHECK-NEXT: ldr q2, [x8, lCPI25_2@PAGEOFF]
; CHECK-NEXT: Lloh49: ; CHECK-NEXT: Lloh49:
; CHECK-NEXT: ldr q3, [x11, lCPI25_3@PAGEOFF] ; CHECK-NEXT: ldr q3, [x10, lCPI25_3@PAGEOFF]
; CHECK-NEXT: mov x8, x1
; CHECK-NEXT: LBB25_1: ; %loop ; CHECK-NEXT: LBB25_1: ; %loop
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldr q4, [x0] ; CHECK-NEXT: ldr q4, [x0]
; CHECK-NEXT: ldp q16, q7, [x1, #32]
; CHECK-NEXT: ldr q18, [x8, #16]!
; CHECK-NEXT: subs w2, w2, #1 ; CHECK-NEXT: subs w2, w2, #1
; CHECK-NEXT: ldp q7, q17, [x1, #32] ; CHECK-NEXT: tbl.16b v5, { v4 }, v3
; CHECK-NEXT: tbl.16b v16, { v4 }, v3 ; CHECK-NEXT: tbl.16b v6, { v4 }, v0
; CHECK-NEXT: tbl.16b v18, { v4 }, v0 ; CHECK-NEXT: tbl.16b v17, { v4 }, v2
; CHECK-NEXT: tbl.16b v19, { v4 }, v1 ; CHECK-NEXT: tbl.16b v4, { v4 }, v1
; CHECK-NEXT: tbl.16b v4, { v4 }, v2 ; CHECK-NEXT: umull2.2d v19, v5, v7
; CHECK-NEXT: ldr q5, [x1] ; CHECK-NEXT: umull.2d v5, v5, v7
; CHECK-NEXT: ldr q6, [x8, #16]! ; CHECK-NEXT: ldr q7, [x1]
; CHECK-NEXT: umull2.2d v20, v16, v17 ; CHECK-NEXT: umull2.2d v20, v6, v16
; CHECK-NEXT: umull2.2d v21, v17, v18
; CHECK-NEXT: umull.2d v17, v17, v18
; CHECK-NEXT: umull2.2d v18, v4, v7
; CHECK-NEXT: umull.2d v4, v4, v7
; CHECK-NEXT: mov x1, x8 ; CHECK-NEXT: mov x1, x8
; CHECK-NEXT: umull2.2d v21, v18, v7 ; CHECK-NEXT: stp q5, q19, [x0, #96]
; CHECK-NEXT: umull.2d v16, v16, v17 ; CHECK-NEXT: umull.2d v5, v6, v16
; CHECK-NEXT: umull2.2d v17, v4, v6 ; CHECK-NEXT: str q20, [x0, #80]
; CHECK-NEXT: umull.2d v4, v4, v6 ; CHECK-NEXT: stp q4, q18, [x0]
; CHECK-NEXT: umull2.2d v6, v19, v5 ; CHECK-NEXT: stp q17, q21, [x0, #32]
; CHECK-NEXT: str q21, [x0, #80] ; CHECK-NEXT: str q5, [x0, #64]!
; CHECK-NEXT: umull.2d v5, v19, v5
; CHECK-NEXT: stp q16, q20, [x0, #96]
; CHECK-NEXT: umull.2d v7, v18, v7
; CHECK-NEXT: stp q4, q17, [x0, #32]
; CHECK-NEXT: stp q5, q6, [x0]
; CHECK-NEXT: str q7, [x0, #64]!
; CHECK-NEXT: b.ne LBB25_1 ; CHECK-NEXT: b.ne LBB25_1
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: mov w0, wzr ; CHECK-NEXT: mov w0, wzr
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh45, Lloh49 ; CHECK-NEXT: .loh AdrpLdr Lloh46, Lloh48
; CHECK-NEXT: .loh AdrpLdr Lloh44, Lloh48 ; CHECK-NEXT: .loh AdrpLdr Lloh44, Lloh49
; CHECK-NEXT: .loh AdrpLdr Lloh43, Lloh47 ; CHECK-NEXT: .loh AdrpLdr Lloh43, Lloh47
; CHECK-NEXT: .loh AdrpLdr Lloh42, Lloh46 ; CHECK-NEXT: .loh AdrpAdrp Lloh42, Lloh46
; CHECK-NEXT: .loh AdrpLdr Lloh42, Lloh45
; ;
; CHECK-BE-LABEL: test_widening_instr_mull_64: ; CHECK-BE-LABEL: test_widening_instr_mull_64:
; CHECK-BE: // %bb.0: // %entry ; CHECK-BE: // %bb.0: // %entry
Context not available.
; CHECK-BE-NEXT: .LBB25_1: // %loop ; CHECK-BE-NEXT: .LBB25_1: // %loop
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: ld1 { v4.16b }, [x0] ; CHECK-BE-NEXT: ld1 { v4.16b }, [x0]
; CHECK-BE-NEXT: add x8, x1, #48 ; CHECK-BE-NEXT: add x9, x1, #48
; CHECK-BE-NEXT: add x9, x1, #32 ; CHECK-BE-NEXT: add x8, x1, #32
; CHECK-BE-NEXT: subs w2, w2, #1 ; CHECK-BE-NEXT: ld1 { v18.4s }, [x9]
; CHECK-BE-NEXT: ld1 { v5.4s }, [x1] ; CHECK-BE-NEXT: ld1 { v16.4s }, [x1]
; CHECK-BE-NEXT: add x1, x1, #16 ; CHECK-BE-NEXT: add x1, x1, #16
; CHECK-BE-NEXT: ld1 { v6.4s }, [x8] ; CHECK-BE-NEXT: ld1 { v20.4s }, [x8]
; CHECK-BE-NEXT: ld1 { v22.4s }, [x1]
; CHECK-BE-NEXT: add x8, x0, #96 ; CHECK-BE-NEXT: add x8, x0, #96
; CHECK-BE-NEXT: tbl v7.16b, { v4.16b }, v3.16b ; CHECK-BE-NEXT: tbl v5.16b, { v4.16b }, v3.16b
; CHECK-BE-NEXT: tbl v17.16b, { v4.16b }, v1.16b ; CHECK-BE-NEXT: tbl v6.16b, { v4.16b }, v2.16b
; CHECK-BE-NEXT: ld1 { v20.4s }, [x1] ; CHECK-BE-NEXT: tbl v7.16b, { v4.16b }, v1.16b
; CHECK-BE-NEXT: tbl v16.16b, { v4.16b }, v0.16b ; CHECK-BE-NEXT: tbl v4.16b, { v4.16b }, v0.16b
; CHECK-BE-NEXT: tbl v4.16b, { v4.16b }, v2.16b ; CHECK-BE-NEXT: ext v24.16b, v18.16b, v18.16b, #8
; CHECK-BE-NEXT: ld1 { v18.4s }, [x9]
; CHECK-BE-NEXT: rev32 v19.8b, v7.8b
; CHECK-BE-NEXT: add x9, x0, #32 ; CHECK-BE-NEXT: add x9, x0, #32
; CHECK-BE-NEXT: ext v23.16b, v6.16b, v6.16b, #8 ; CHECK-BE-NEXT: ext v25.16b, v20.16b, v20.16b, #8
; CHECK-BE-NEXT: rev32 v22.8b, v17.8b ; CHECK-BE-NEXT: add x10, x0, #16
; CHECK-BE-NEXT: subs w2, w2, #1
; CHECK-BE-NEXT: ext v17.16b, v5.16b, v5.16b, #8
; CHECK-BE-NEXT: ext v19.16b, v6.16b, v6.16b, #8
; CHECK-BE-NEXT: rev32 v5.8b, v5.8b
; CHECK-BE-NEXT: rev32 v21.8b, v7.8b
; CHECK-BE-NEXT: rev32 v23.8b, v4.8b
; CHECK-BE-NEXT: ext v7.16b, v7.16b, v7.16b, #8 ; CHECK-BE-NEXT: ext v7.16b, v7.16b, v7.16b, #8
; CHECK-BE-NEXT: ext v24.16b, v4.16b, v4.16b, #8 ; CHECK-BE-NEXT: ext v4.16b, v4.16b, v4.16b, #8
; CHECK-BE-NEXT: umull v6.2d, v19.2s, v6.2s ; CHECK-BE-NEXT: rev32 v6.8b, v6.8b
; CHECK-BE-NEXT: umull v19.2d, v22.2s, v20.2s ; CHECK-BE-NEXT: rev32 v17.8b, v17.8b
; CHECK-BE-NEXT: ext v22.16b, v18.16b, v18.16b, #8 ; CHECK-BE-NEXT: rev32 v19.8b, v19.8b
; CHECK-BE-NEXT: ext v21.16b, v5.16b, v5.16b, #8 ; CHECK-BE-NEXT: umull v5.2d, v5.2s, v18.2s
; CHECK-BE-NEXT: st1 { v6.2d }, [x8] ; CHECK-BE-NEXT: umull v18.2d, v21.2s, v22.2s
; CHECK-BE-NEXT: rev32 v6.8b, v7.8b ; CHECK-BE-NEXT: ext v21.16b, v22.16b, v22.16b, #8
; CHECK-BE-NEXT: ext v7.16b, v17.16b, v17.16b, #8 ; CHECK-BE-NEXT: rev32 v7.8b, v7.8b
; CHECK-BE-NEXT: rev32 v17.8b, v16.8b ; CHECK-BE-NEXT: umull v22.2d, v23.2s, v16.2s
; CHECK-BE-NEXT: ext v16.16b, v16.16b, v16.16b, #8 ; CHECK-BE-NEXT: ext v16.16b, v16.16b, v16.16b, #8
; CHECK-BE-NEXT: add x8, x0, #112
; CHECK-BE-NEXT: st1 { v19.2d }, [x9]
; CHECK-BE-NEXT: rev32 v19.8b, v24.8b
; CHECK-BE-NEXT: umull v6.2d, v6.2s, v23.2s
; CHECK-BE-NEXT: rev32 v4.8b, v4.8b ; CHECK-BE-NEXT: rev32 v4.8b, v4.8b
; CHECK-BE-NEXT: umull v5.2d, v17.2s, v5.2s ; CHECK-BE-NEXT: umull v17.2d, v17.2s, v24.2s
; CHECK-BE-NEXT: umull v19.2d, v19.2s, v25.2s
; CHECK-BE-NEXT: st1 { v5.2d }, [x8]
; CHECK-BE-NEXT: umull v5.2d, v6.2s, v20.2s
; CHECK-BE-NEXT: umull v6.2d, v7.2s, v21.2s
; CHECK-BE-NEXT: add x8, x0, #112
; CHECK-BE-NEXT: umull v4.2d, v4.2s, v16.2s
; CHECK-BE-NEXT: st1 { v18.2d }, [x9]
; CHECK-BE-NEXT: add x9, x0, #80 ; CHECK-BE-NEXT: add x9, x0, #80
; CHECK-BE-NEXT: rev32 v7.8b, v7.8b ; CHECK-BE-NEXT: st1 { v22.2d }, [x0]
; CHECK-BE-NEXT: rev32 v16.8b, v16.8b ; CHECK-BE-NEXT: st1 { v17.2d }, [x8]
; CHECK-BE-NEXT: st1 { v6.2d }, [x8] ; CHECK-BE-NEXT: add x8, x0, #64
; CHECK-BE-NEXT: add x8, x0, #48 ; CHECK-BE-NEXT: st1 { v19.2d }, [x9]
; CHECK-BE-NEXT: ext v6.16b, v20.16b, v20.16b, #8 ; CHECK-BE-NEXT: add x9, x0, #48
; CHECK-BE-NEXT: st1 { v5.2d }, [x0] ; CHECK-BE-NEXT: mov x0, x8
; CHECK-BE-NEXT: umull v17.2d, v19.2s, v22.2s
; CHECK-BE-NEXT: umull v4.2d, v4.2s, v18.2s
; CHECK-BE-NEXT: umull v5.2d, v7.2s, v6.2s
; CHECK-BE-NEXT: umull v6.2d, v16.2s, v21.2s
; CHECK-BE-NEXT: st1 { v17.2d }, [x9]
; CHECK-BE-NEXT: add x9, x0, #16
; CHECK-BE-NEXT: add x0, x0, #64
; CHECK-BE-NEXT: st1 { v5.2d }, [x8] ; CHECK-BE-NEXT: st1 { v5.2d }, [x8]
; CHECK-BE-NEXT: st1 { v4.2d }, [x0]
; CHECK-BE-NEXT: st1 { v6.2d }, [x9] ; CHECK-BE-NEXT: st1 { v6.2d }, [x9]
; CHECK-BE-NEXT: st1 { v4.2d }, [x10]
; CHECK-BE-NEXT: b.ne .LBB25_1 ; CHECK-BE-NEXT: b.ne .LBB25_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: mov w0, wzr ; CHECK-BE-NEXT: mov w0, wzr
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i32 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i32 [ 0, %entry ], [ %iv.next, %loop ]
%gep.1 = getelementptr inbounds <16 x i32>, ptr %p1, i32 %iv %gep.1 = getelementptr inbounds <16 x i32>, ptr %p1, i32 %iv
Context not available.
%iv.next= add nuw nsw i32 %iv, 1 %iv.next= add nuw nsw i32 %iv, 1
%exitcond.not = icmp eq i32 %iv.next, %h %exitcond.not = icmp eq i32 %iv.next, %h
br i1 %exitcond.not, label %exit, label %loop br i1 %exitcond.not, label %exit, label %loop
exit: exit:
ret i32 0 ret i32 0
} }
define i32 @test_widening_instr_mull_2(ptr %p1, ptr %p2, i32 %h) { define i32 @test_widening_instr_mull_2(ptr %p1, ptr %p2, i32 %h) {
;
; CHECK-LABEL: test_widening_instr_mull_2: ; CHECK-LABEL: test_widening_instr_mull_2:
; CHECK: ; %bb.0: ; %entry ; CHECK: ; %bb.0: ; %entry
; CHECK-NEXT: Lloh50: ; CHECK-NEXT: Lloh50:
Context not available.
; CHECK-NEXT: Lloh51: ; CHECK-NEXT: Lloh51:
; CHECK-NEXT: adrp x9, lCPI26_1@PAGE ; CHECK-NEXT: adrp x9, lCPI26_1@PAGE
; CHECK-NEXT: Lloh52: ; CHECK-NEXT: Lloh52:
; CHECK-NEXT: adrp x10, lCPI26_2@PAGE ; CHECK-NEXT: adrp x10, lCPI26_3@PAGE
; CHECK-NEXT: Lloh53: ; CHECK-NEXT: Lloh53:
; CHECK-NEXT: adrp x11, lCPI26_3@PAGE
; CHECK-NEXT: Lloh54:
; CHECK-NEXT: ldr q0, [x8, lCPI26_0@PAGEOFF] ; CHECK-NEXT: ldr q0, [x8, lCPI26_0@PAGEOFF]
; CHECK-NEXT: mov x8, x0 ; CHECK-NEXT: Lloh54:
; CHECK-NEXT: adrp x8, lCPI26_2@PAGE
; CHECK-NEXT: Lloh55: ; CHECK-NEXT: Lloh55:
; CHECK-NEXT: ldr q1, [x9, lCPI26_1@PAGEOFF] ; CHECK-NEXT: ldr q1, [x9, lCPI26_1@PAGEOFF]
; CHECK-NEXT: Lloh56: ; CHECK-NEXT: Lloh56:
; CHECK-NEXT: ldr q2, [x10, lCPI26_2@PAGEOFF] ; CHECK-NEXT: ldr q2, [x8, lCPI26_2@PAGEOFF]
; CHECK-NEXT: Lloh57: ; CHECK-NEXT: Lloh57:
; CHECK-NEXT: ldr q3, [x11, lCPI26_3@PAGEOFF] ; CHECK-NEXT: ldr q3, [x10, lCPI26_3@PAGEOFF]
; CHECK-NEXT: mov x8, x0
; CHECK-NEXT: LBB26_1: ; %loop ; CHECK-NEXT: LBB26_1: ; %loop
; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1 ; CHECK-NEXT: ; =>This Inner Loop Header: Depth=1
; CHECK-NEXT: ldr q4, [x1], #16 ; CHECK-NEXT: ldr q4, [x1], #16
; CHECK-NEXT: ldp q5, q6, [x0, #32] ; CHECK-NEXT: ldr q18, [x0]
; CHECK-NEXT: ldp q16, q17, [x0, #32]
; CHECK-NEXT: subs w2, w2, #1 ; CHECK-NEXT: subs w2, w2, #1
; CHECK-NEXT: tbl.16b v16, { v4 }, v0 ; CHECK-NEXT: tbl.16b v5, { v4 }, v0
; CHECK-NEXT: tbl.16b v18, { v4 }, v1 ; CHECK-NEXT: tbl.16b v6, { v4 }, v1
; CHECK-NEXT: tbl.16b v19, { v4 }, v2 ; CHECK-NEXT: tbl.16b v7, { v4 }, v2
; CHECK-NEXT: tbl.16b v4, { v4 }, v3 ; CHECK-NEXT: tbl.16b v4, { v4 }, v3
; CHECK-NEXT: ldr q7, [x0] ; CHECK-NEXT: mul.4s v5, v16, v5
; CHECK-NEXT: ldr q17, [x8, #16]! ; CHECK-NEXT: ldr q16, [x8, #16]!
; CHECK-NEXT: mul.4s v5, v5, v16 ; CHECK-NEXT: mul.4s v6, v17, v6
; CHECK-NEXT: mul.4s v6, v6, v18 ; CHECK-NEXT: mul.4s v7, v18, v7
; CHECK-NEXT: mul.4s v7, v7, v19 ; CHECK-NEXT: mul.4s v4, v16, v4
; CHECK-NEXT: mul.4s v4, v17, v4
; CHECK-NEXT: stp q5, q6, [x0, #32] ; CHECK-NEXT: stp q5, q6, [x0, #32]
; CHECK-NEXT: str q7, [x0] ; CHECK-NEXT: str q7, [x0]
; CHECK-NEXT: mov x0, x8 ; CHECK-NEXT: mov x0, x8
Context not available.
; CHECK-NEXT: ; %bb.2: ; %exit ; CHECK-NEXT: ; %bb.2: ; %exit
; CHECK-NEXT: mov w0, wzr ; CHECK-NEXT: mov w0, wzr
; CHECK-NEXT: ret ; CHECK-NEXT: ret
; CHECK-NEXT: .loh AdrpLdr Lloh53, Lloh57 ; CHECK-NEXT: .loh AdrpLdr Lloh54, Lloh56
; CHECK-NEXT: .loh AdrpLdr Lloh52, Lloh56 ; CHECK-NEXT: .loh AdrpLdr Lloh52, Lloh57
; CHECK-NEXT: .loh AdrpLdr Lloh51, Lloh55 ; CHECK-NEXT: .loh AdrpLdr Lloh51, Lloh55
; CHECK-NEXT: .loh AdrpLdr Lloh50, Lloh54 ; CHECK-NEXT: .loh AdrpAdrp Lloh50, Lloh54
; CHECK-NEXT: .loh AdrpLdr Lloh50, Lloh53
; ;
; CHECK-BE-LABEL: test_widening_instr_mull_2: ; CHECK-BE-LABEL: test_widening_instr_mull_2:
; CHECK-BE: // %bb.0: // %entry ; CHECK-BE: // %bb.0: // %entry
Context not available.
; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1 ; CHECK-BE-NEXT: // =>This Inner Loop Header: Depth=1
; CHECK-BE-NEXT: ld1 { v4.16b }, [x1] ; CHECK-BE-NEXT: ld1 { v4.16b }, [x1]
; CHECK-BE-NEXT: add x8, x0, #32 ; CHECK-BE-NEXT: add x8, x0, #32
; CHECK-BE-NEXT: ld1 { v16.4s }, [x0]
; CHECK-BE-NEXT: add x9, x0, #48 ; CHECK-BE-NEXT: add x9, x0, #48
; CHECK-BE-NEXT: add x10, x0, #16 ; CHECK-BE-NEXT: add x10, x0, #16
; CHECK-BE-NEXT: ld1 { v6.4s }, [x0] ; CHECK-BE-NEXT: ld1 { v17.4s }, [x8]
; CHECK-BE-NEXT: ld1 { v18.4s }, [x9]
; CHECK-BE-NEXT: ld1 { v19.4s }, [x10]
; CHECK-BE-NEXT: subs w2, w2, #1 ; CHECK-BE-NEXT: subs w2, w2, #1
; CHECK-BE-NEXT: add x1, x1, #16
; CHECK-BE-NEXT: ld1 { v16.4s }, [x8]
; CHECK-BE-NEXT: tbl v5.16b, { v4.16b }, v1.16b ; CHECK-BE-NEXT: tbl v5.16b, { v4.16b }, v1.16b
; CHECK-BE-NEXT: tbl v7.16b, { v4.16b }, v0.16b ; CHECK-BE-NEXT: tbl v6.16b, { v4.16b }, v3.16b
; CHECK-BE-NEXT: ld1 { v18.4s }, [x10] ; CHECK-BE-NEXT: tbl v7.16b, { v4.16b }, v2.16b
; CHECK-BE-NEXT: tbl v17.16b, { v4.16b }, v3.16b ; CHECK-BE-NEXT: tbl v4.16b, { v4.16b }, v0.16b
; CHECK-BE-NEXT: tbl v4.16b, { v4.16b }, v2.16b ; CHECK-BE-NEXT: add x1, x1, #16
; CHECK-BE-NEXT: rev32 v5.16b, v5.16b ; CHECK-BE-NEXT: rev32 v5.16b, v5.16b
; CHECK-BE-NEXT: rev32 v6.16b, v6.16b
; CHECK-BE-NEXT: rev32 v7.16b, v7.16b ; CHECK-BE-NEXT: rev32 v7.16b, v7.16b
; CHECK-BE-NEXT: rev32 v17.16b, v17.16b
; CHECK-BE-NEXT: rev32 v4.16b, v4.16b ; CHECK-BE-NEXT: rev32 v4.16b, v4.16b
; CHECK-BE-NEXT: mul v5.4s, v6.4s, v5.4s ; CHECK-BE-NEXT: mul v5.4s, v16.4s, v5.4s
; CHECK-BE-NEXT: ld1 { v6.4s }, [x9] ; CHECK-BE-NEXT: mul v6.4s, v17.4s, v6.4s
; CHECK-BE-NEXT: mul v7.4s, v18.4s, v7.4s ; CHECK-BE-NEXT: mul v7.4s, v18.4s, v7.4s
; CHECK-BE-NEXT: mul v4.4s, v19.4s, v4.4s
; CHECK-BE-NEXT: st1 { v5.4s }, [x0] ; CHECK-BE-NEXT: st1 { v5.4s }, [x0]
; CHECK-BE-NEXT: mov x0, x10 ; CHECK-BE-NEXT: mov x0, x10
; CHECK-BE-NEXT: mul v5.4s, v16.4s, v17.4s ; CHECK-BE-NEXT: st1 { v6.4s }, [x8]
; CHECK-BE-NEXT: st1 { v7.4s }, [x10] ; CHECK-BE-NEXT: st1 { v7.4s }, [x9]
; CHECK-BE-NEXT: mul v4.4s, v6.4s, v4.4s ; CHECK-BE-NEXT: st1 { v4.4s }, [x10]
; CHECK-BE-NEXT: st1 { v5.4s }, [x8]
; CHECK-BE-NEXT: st1 { v4.4s }, [x9]
; CHECK-BE-NEXT: b.ne .LBB26_1 ; CHECK-BE-NEXT: b.ne .LBB26_1
; CHECK-BE-NEXT: // %bb.2: // %exit ; CHECK-BE-NEXT: // %bb.2: // %exit
; CHECK-BE-NEXT: mov w0, wzr ; CHECK-BE-NEXT: mov w0, wzr
; CHECK-BE-NEXT: ret ; CHECK-BE-NEXT: ret
entry: entry:
br label %loop br label %loop
loop: loop:
%iv = phi i32 [ 0, %entry ], [ %iv.next, %loop ] %iv = phi i32 [ 0, %entry ], [ %iv.next, %loop ]
%gep.1 = getelementptr inbounds <16 x i8>, ptr %p1, i32 %iv %gep.1 = getelementptr inbounds <16 x i8>, ptr %p1, i32 %iv
Context not available.
%iv.next= add nuw nsw i32 %iv, 1 %iv.next= add nuw nsw i32 %iv, 1
%exitcond.not = icmp eq i32 %iv.next, %h %exitcond.not = icmp eq i32 %iv.next, %h
br i1 %exitcond.not, label %exit, label %loop br i1 %exitcond.not, label %exit, label %loop
exit: exit:
ret i32 0 ret i32 0
} }
Context not available.

llvm/test/MC/AArch64/elf-globaladdress.ll

llvm/test/MachineVerifier/test_g_concat_vectors.mir

llvm/test/Transforms/CanonicalizeFreezeInLoops/aarch64.ll

llvm/test/Transforms/CodeGenPrepare/AArch64/large-offset-gep.ll

llvm/test/Transforms/LoopStrengthReduce/AArch64/lsr-ldp.ll

llvm/test/Transforms/LoopStrengthReduce/AArch64/pr53625.ll

llvm/test/Transforms/LoopStrengthReduce/AArch64/small-constant.ll

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/aarch64_generated_funcs.ll.generated.expected

llvm/test/tools/UpdateTestChecks/update_llc_test_checks/Inputs/aarch64_generated_funcs.ll.nogenerated.expected