This is an archive of the discontinued LLVM Phabricator instance.

Differential D141048

[SelectionDAG] Add pcsections recursively on SDNode values
ClosedPublic

Authored by martin-fink on Jan 5 2023, 6:12 AM.

Details

Reviewers
melver
Commits
rG1142e6c7c795: [SelectionDAG] Add missing setValue calls in visitIntrinsicCall
rG0c64e1b68f36: [SelectionDAG] Add pcsections recursively on SDNode values
Summary

When adding pcsetions to SDNodes, recursively add them to all values of
the node as well.

[SelectionDAG] Add missing setValue calls in visitIntrinsicCall

This commit introduces missing setValue calls in SelectionDAGBuilder for
mem-transfer intrinsic calls. These setValue calls are required in order to
propagate pcsections metadata from IR to MIR.

Diff Detail

Unit TestsFailed

TimeTest
240 msx64 debian > Profile-x86_64.Linux::counter_promo_for.c
310 msx64 debian > Profile-x86_64.Linux::counter_promo_while.c

Event Timeline

martin-fink created this revision.Jan 5 2023, 6:12 AM
Herald added a project: Restricted Project. · View Herald TranscriptJan 5 2023, 6:12 AM
Herald added subscribers: ecnelises, pengfei, hiraditya. · View Herald Transcript
martin-fink requested review of this revision.Jan 5 2023, 6:12 AM
Herald added a project: Restricted Project. · View Herald TranscriptJan 5 2023, 6:12 AM
Herald added a subscriber: llvm-commits. · View Herald Transcript
martin-fink added a reviewer: melver.Jan 5 2023, 6:22 AM
melver requested changes to this revision.Jan 5 2023, 6:38 AM
melver added inline comments.
llvm/test/CodeGen/AArch64/SelectionDAG/arm64-pcsections-selectiondag.ll
1 ↗(On Diff #486543)

Similar to x86 comment:

I think we want to say -global-isel=0, to ensure GlobalISel isn't used?

Also, just call it similar to the x86 one 'pcsections-memtransfer.ll', i.e. it'll end up in 'CodeGen/AArch64/pcsections-memtransfer.ll'.

87–100 ↗(On Diff #486543)

I see no pcsections, is it still missing?

109–122 ↗(On Diff #486543)

I see no pcsections, is it still missing?

131–144 ↗(On Diff #486543)

I see no pcsections, is it still missing?

llvm/test/CodeGen/X86/SelectionDAG/x64-pcsections-memtransfer.ll
1 ↗(On Diff #486543)

I think creating a SelectionDAG subdir is unnecessary (you're the first to create it). I think we can just dump it in X86/

Also, I think we need to explicitly make sure global-isel isn't used as it might be enabled in future, at which point it doesn't test SelectionDAG anymore.

E.g. see llvm/test/CodeGen/X86/switch-bit-test-unreachable-default.ll

I'd name this test simply 'pcsections-memtransfer.ll'

This revision now requires changes to proceed.Jan 5 2023, 6:38 AM
melver added a comment.Jan 5 2023, 6:39 AM

Also update commit message to say something about "mem transfer" intrinsics.

arsenm added a subscriber: arsenm.Jan 5 2023, 6:47 AM
arsenm added inline comments.
llvm/test/CodeGen/AArch64/SelectionDAG/arm64-pcsections-selectiondag.ll
1 ↗(On Diff #486543)

Just because it's a test for sdag doesn't mean it also shouldn't pass for globalisel. I wouldn't explicitly turn it off

martin-fink added inline comments.Jan 5 2023, 6:56 AM
llvm/test/CodeGen/AArch64/SelectionDAG/arm64-pcsections-selectiondag.ll
87–100 ↗(On Diff #486543)

I think I've overlooked that pcsections are not passed to MIR for the .element.unordered.atomic instrinsics on AArch64.
As the issue for that probably lies somewhere else and not in the scope of this commit (the pcsections are passed to MIR on X86, for example), I would suggest that I (for now) remove these three functions from the test in this commit and have a look at them later on.
What do you think about that?

martin-fink updated this revision to Diff 486573.Jan 5 2023, 7:34 AM
martin-fink edited the summary of this revision. (Show Details)

[SelectionDAG] Add missing setValue calls in visitIntrinsicCall

This commit introduces missing setValue calls in SelectionDAGBuilder for
mem-transfer intrinsic calls. These setValue calls are required in order to
propagate pcsections metadata from IR to MIR.

Harbormaster completed remote builds in B205910: Diff 486573.Jan 5 2023, 9:31 AM
melver added inline comments.Jan 5 2023, 11:11 AM
llvm/test/CodeGen/AArch64/SelectionDAG/arm64-pcsections-selectiondag.ll
87–100 ↗(On Diff #486543)

I think it's somewhere in the AArch64-specific bits of SelectionDAG.

I think if you want this working properly, it should be fixed.

Otherwise, the support for the mem-transfer functions seems incomplete and whatever you intend to use !pcsections for, will be half working only.

martin-fink updated this revision to Diff 489514.Jan 16 2023, 5:17 AM
martin-fink retitled this revision from [SelectionDAG] Add missing setValue calls in visitIntrinsicCall to [SelectionDAG] Add pcsections recursively on SDNode values.
martin-fink edited the summary of this revision. (Show Details)

Recursively set pcsections on SDNodes to make sure pcsections are propagated to all generated instructions.

martin-fink updated this revision to Diff 489516.Jan 16 2023, 5:22 AM

Updating D141048: [SelectionDAG] Add pcsections recursively on SDNode values

Use recursive function instead of lambda

martin-fink updated this revision to Diff 489517.Jan 16 2023, 5:24 AM

Remove unneeded import

Harbormaster completed remote builds in B208028: Diff 489517.Jan 16 2023, 7:33 AM
martin-fink updated this revision to Diff 489741.Jan 17 2023, 2:15 AM
martin-fink edited the summary of this revision. (Show Details)

Rebase main

Harbormaster completed remote builds in B208191: Diff 489741.Jan 17 2023, 3:10 AM
melver added a comment.Jan 25 2023, 3:27 AM

Nice!

Minor comments below. Otherwise LGTM.

llvm/include/llvm/CodeGen/SelectionDAG.h
2283 ↗(On Diff #489741)

This looks like DFS - this is typically called the visited set. So I'd just call it "Visited".

2363 ↗(On Diff #489741)

No need for different name, just "addPCSections" - the fact it has different arguments means it's overloaded and there's no conflict.

Given it's private and has documentation above it, it seems cleaner.

2363 ↗(On Diff #489741)

This is a rather large non-trivial function now, I'd now define it in the header, but move it to the .cpp file (like most functions here).

melver added inline comments.Jan 25 2023, 3:30 AM
llvm/include/llvm/CodeGen/SelectionDAG.h
2363 ↗(On Diff #489741)

... I'd _not_ define it in the header ...

martin-fink updated this revision to Diff 492066.Jan 25 2023, 4:31 AM
martin-fink edited the summary of this revision. (Show Details)

Made the following changes based on @melver's comments:

  • Rename addPcsectionsr
  • Rename Once to Visited
  • Move function out of header file
Harbormaster completed remote builds in B209844: Diff 492066.Jan 25 2023, 6:06 AM
melver accepted this revision.Jan 26 2023, 6:23 AM

Thanks!
I'll land this for you.

This revision is now accepted and ready to land.Jan 26 2023, 6:23 AM
melver added inline comments.Jan 26 2023, 6:39 AM
llvm/include/llvm/CodeGen/SelectionDAG.h
2364 ↗(On Diff #492066)

I think this function wants to receive a reference to the set - I'll fix it up for you.

Closed by commit rG0c64e1b68f36: [SelectionDAG] Add pcsections recursively on SDNode values (authored by martin-fink, committed by melver). · Explain WhyJan 26 2023, 7:15 AM
This revision was automatically updated to reflect the committed changes.
melver added a commit: rG0c64e1b68f36: [SelectionDAG] Add pcsections recursively on SDNode values.
melver added a commit: rG1142e6c7c795: [SelectionDAG] Add missing setValue calls in visitIntrinsicCall.
melver added a reverting change: rG98f0e4f611b4: Revert "[SelectionDAG] Add pcsections recursively on SDNode values".Feb 3 2023, 10:01 AM

Revision Contents

PathSize
llvm/
lib/
CodeGen/
SelectionDAG/
8 lines
test/
CodeGen/
AArch64/
93 lines
X86/
233 lines

Diff 486573

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 5,933 Lines • ▼ Show 20 Lines case Intrinsic::memcpy: {
// FIXME: Support passing different dest/src alignments to the memcpy DAG // FIXME: Support passing different dest/src alignments to the memcpy DAG
// node. // node.
SDValue Root = isVol ? getRoot() : getMemoryRoot(); SDValue Root = isVol ? getRoot() : getMemoryRoot();
SDValue MC = DAG.getMemcpy( SDValue MC = DAG.getMemcpy(
Root, sdl, Op1, Op2, Op3, Alignment, isVol, Root, sdl, Op1, Op2, Op3, Alignment, isVol,
/* AlwaysInline */ false, isTC, MachinePointerInfo(I.getArgOperand(0)), /* AlwaysInline */ false, isTC, MachinePointerInfo(I.getArgOperand(0)),
MachinePointerInfo(I.getArgOperand(1)), I.getAAMetadata(), AA); MachinePointerInfo(I.getArgOperand(1)), I.getAAMetadata(), AA);
updateDAGForMaybeTailCall(MC); updateDAGForMaybeTailCall(MC);
setValue(&I, MC);
return; return;
} }
case Intrinsic::memcpy_inline: { case Intrinsic::memcpy_inline: {
const auto &MCI = cast<MemCpyInlineInst>(I); const auto &MCI = cast<MemCpyInlineInst>(I);
SDValue Dst = getValue(I.getArgOperand(0)); SDValue Dst = getValue(I.getArgOperand(0));
SDValue Src = getValue(I.getArgOperand(1)); SDValue Src = getValue(I.getArgOperand(1));
SDValue Size = getValue(I.getArgOperand(2)); SDValue Size = getValue(I.getArgOperand(2));
assert(isa<ConstantSDNode>(Size) && "memcpy_inline needs constant size"); assert(isa<ConstantSDNode>(Size) && "memcpy_inline needs constant size");
// @llvm.memcpy.inline defines 0 and 1 to both mean no alignment. // @llvm.memcpy.inline defines 0 and 1 to both mean no alignment.
Align DstAlign = MCI.getDestAlign().valueOrOne(); Align DstAlign = MCI.getDestAlign().valueOrOne();
Align SrcAlign = MCI.getSourceAlign().valueOrOne(); Align SrcAlign = MCI.getSourceAlign().valueOrOne();
Align Alignment = std::min(DstAlign, SrcAlign); Align Alignment = std::min(DstAlign, SrcAlign);
bool isVol = MCI.isVolatile(); bool isVol = MCI.isVolatile();
bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget()); bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget());
// FIXME: Support passing different dest/src alignments to the memcpy DAG // FIXME: Support passing different dest/src alignments to the memcpy DAG
// node. // node.
SDValue MC = DAG.getMemcpy( SDValue MC = DAG.getMemcpy(
getRoot(), sdl, Dst, Src, Size, Alignment, isVol, getRoot(), sdl, Dst, Src, Size, Alignment, isVol,
/* AlwaysInline */ true, isTC, MachinePointerInfo(I.getArgOperand(0)), /* AlwaysInline */ true, isTC, MachinePointerInfo(I.getArgOperand(0)),
MachinePointerInfo(I.getArgOperand(1)), I.getAAMetadata(), AA); MachinePointerInfo(I.getArgOperand(1)), I.getAAMetadata(), AA);
updateDAGForMaybeTailCall(MC); updateDAGForMaybeTailCall(MC);
setValue(&I, MC);
return; return;
} }
case Intrinsic::memset: { case Intrinsic::memset: {
const auto &MSI = cast<MemSetInst>(I); const auto &MSI = cast<MemSetInst>(I);
SDValue Op1 = getValue(I.getArgOperand(0)); SDValue Op1 = getValue(I.getArgOperand(0));
SDValue Op2 = getValue(I.getArgOperand(1)); SDValue Op2 = getValue(I.getArgOperand(1));
SDValue Op3 = getValue(I.getArgOperand(2)); SDValue Op3 = getValue(I.getArgOperand(2));
// @llvm.memset defines 0 and 1 to both mean no alignment. // @llvm.memset defines 0 and 1 to both mean no alignment.
Align Alignment = MSI.getDestAlign().valueOrOne(); Align Alignment = MSI.getDestAlign().valueOrOne();
bool isVol = MSI.isVolatile(); bool isVol = MSI.isVolatile();
bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget()); bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget());
SDValue Root = isVol ? getRoot() : getMemoryRoot(); SDValue Root = isVol ? getRoot() : getMemoryRoot();
SDValue MS = DAG.getMemset( SDValue MS = DAG.getMemset(
Root, sdl, Op1, Op2, Op3, Alignment, isVol, /* AlwaysInline */ false, Root, sdl, Op1, Op2, Op3, Alignment, isVol, /* AlwaysInline */ false,
isTC, MachinePointerInfo(I.getArgOperand(0)), I.getAAMetadata()); isTC, MachinePointerInfo(I.getArgOperand(0)), I.getAAMetadata());
updateDAGForMaybeTailCall(MS); updateDAGForMaybeTailCall(MS);
setValue(&I, MS);
return; return;
} }
case Intrinsic::memset_inline: { case Intrinsic::memset_inline: {
const auto &MSII = cast<MemSetInlineInst>(I); const auto &MSII = cast<MemSetInlineInst>(I);
SDValue Dst = getValue(I.getArgOperand(0)); SDValue Dst = getValue(I.getArgOperand(0));
SDValue Value = getValue(I.getArgOperand(1)); SDValue Value = getValue(I.getArgOperand(1));
SDValue Size = getValue(I.getArgOperand(2)); SDValue Size = getValue(I.getArgOperand(2));
assert(isa<ConstantSDNode>(Size) && "memset_inline needs constant size"); assert(isa<ConstantSDNode>(Size) && "memset_inline needs constant size");
// @llvm.memset defines 0 and 1 to both mean no alignment. // @llvm.memset defines 0 and 1 to both mean no alignment.
Align DstAlign = MSII.getDestAlign().valueOrOne(); Align DstAlign = MSII.getDestAlign().valueOrOne();
bool isVol = MSII.isVolatile(); bool isVol = MSII.isVolatile();
bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget()); bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget());
SDValue Root = isVol ? getRoot() : getMemoryRoot(); SDValue Root = isVol ? getRoot() : getMemoryRoot();
SDValue MC = DAG.getMemset(Root, sdl, Dst, Value, Size, DstAlign, isVol, SDValue MC = DAG.getMemset(Root, sdl, Dst, Value, Size, DstAlign, isVol,
/* AlwaysInline */ true, isTC, /* AlwaysInline */ true, isTC,
MachinePointerInfo(I.getArgOperand(0)), MachinePointerInfo(I.getArgOperand(0)),
I.getAAMetadata()); I.getAAMetadata());
updateDAGForMaybeTailCall(MC); updateDAGForMaybeTailCall(MC);
setValue(&I, MC);
return; return;
} }
case Intrinsic::memmove: { case Intrinsic::memmove: {
const auto &MMI = cast<MemMoveInst>(I); const auto &MMI = cast<MemMoveInst>(I);
SDValue Op1 = getValue(I.getArgOperand(0)); SDValue Op1 = getValue(I.getArgOperand(0));
SDValue Op2 = getValue(I.getArgOperand(1)); SDValue Op2 = getValue(I.getArgOperand(1));
SDValue Op3 = getValue(I.getArgOperand(2)); SDValue Op3 = getValue(I.getArgOperand(2));
// @llvm.memmove defines 0 and 1 to both mean no alignment. // @llvm.memmove defines 0 and 1 to both mean no alignment.
Align DstAlign = MMI.getDestAlign().valueOrOne(); Align DstAlign = MMI.getDestAlign().valueOrOne();
Align SrcAlign = MMI.getSourceAlign().valueOrOne(); Align SrcAlign = MMI.getSourceAlign().valueOrOne();
Align Alignment = std::min(DstAlign, SrcAlign); Align Alignment = std::min(DstAlign, SrcAlign);
bool isVol = MMI.isVolatile(); bool isVol = MMI.isVolatile();
bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget()); bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget());
// FIXME: Support passing different dest/src alignments to the memmove DAG // FIXME: Support passing different dest/src alignments to the memmove DAG
// node. // node.
SDValue Root = isVol ? getRoot() : getMemoryRoot(); SDValue Root = isVol ? getRoot() : getMemoryRoot();
SDValue MM = DAG.getMemmove(Root, sdl, Op1, Op2, Op3, Alignment, isVol, SDValue MM = DAG.getMemmove(Root, sdl, Op1, Op2, Op3, Alignment, isVol,
isTC, MachinePointerInfo(I.getArgOperand(0)), isTC, MachinePointerInfo(I.getArgOperand(0)),
MachinePointerInfo(I.getArgOperand(1)), MachinePointerInfo(I.getArgOperand(1)),
I.getAAMetadata(), AA); I.getAAMetadata(), AA);
updateDAGForMaybeTailCall(MM); updateDAGForMaybeTailCall(MM);
setValue(&I, MM);
return; return;
} }
case Intrinsic::memcpy_element_unordered_atomic: { case Intrinsic::memcpy_element_unordered_atomic: {
const AtomicMemCpyInst &MI = cast<AtomicMemCpyInst>(I); const AtomicMemCpyInst &MI = cast<AtomicMemCpyInst>(I);
SDValue Dst = getValue(MI.getRawDest()); SDValue Dst = getValue(MI.getRawDest());
SDValue Src = getValue(MI.getRawSource()); SDValue Src = getValue(MI.getRawSource());
SDValue Length = getValue(MI.getLength()); SDValue Length = getValue(MI.getLength());
Type *LengthTy = MI.getLength()->getType(); Type *LengthTy = MI.getLength()->getType();
unsigned ElemSz = MI.getElementSizeInBytes(); unsigned ElemSz = MI.getElementSizeInBytes();
bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget()); bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget());
SDValue MC = SDValue MC =
DAG.getAtomicMemcpy(getRoot(), sdl, Dst, Src, Length, LengthTy, ElemSz, DAG.getAtomicMemcpy(getRoot(), sdl, Dst, Src, Length, LengthTy, ElemSz,
isTC, MachinePointerInfo(MI.getRawDest()), isTC, MachinePointerInfo(MI.getRawDest()),
MachinePointerInfo(MI.getRawSource())); MachinePointerInfo(MI.getRawSource()));
updateDAGForMaybeTailCall(MC); updateDAGForMaybeTailCall(MC);
setValue(&I, MC);
return; return;
} }
case Intrinsic::memmove_element_unordered_atomic: { case Intrinsic::memmove_element_unordered_atomic: {
auto &MI = cast<AtomicMemMoveInst>(I); auto &MI = cast<AtomicMemMoveInst>(I);
SDValue Dst = getValue(MI.getRawDest()); SDValue Dst = getValue(MI.getRawDest());
SDValue Src = getValue(MI.getRawSource()); SDValue Src = getValue(MI.getRawSource());
SDValue Length = getValue(MI.getLength()); SDValue Length = getValue(MI.getLength());
Type *LengthTy = MI.getLength()->getType(); Type *LengthTy = MI.getLength()->getType();
unsigned ElemSz = MI.getElementSizeInBytes(); unsigned ElemSz = MI.getElementSizeInBytes();
bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget()); bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget());
SDValue MC = SDValue MC =
DAG.getAtomicMemmove(getRoot(), sdl, Dst, Src, Length, LengthTy, ElemSz, DAG.getAtomicMemmove(getRoot(), sdl, Dst, Src, Length, LengthTy, ElemSz,
isTC, MachinePointerInfo(MI.getRawDest()), isTC, MachinePointerInfo(MI.getRawDest()),
MachinePointerInfo(MI.getRawSource())); MachinePointerInfo(MI.getRawSource()));
updateDAGForMaybeTailCall(MC); updateDAGForMaybeTailCall(MC);
setValue(&I, MC);
return; return;
} }
case Intrinsic::memset_element_unordered_atomic: { case Intrinsic::memset_element_unordered_atomic: {
auto &MI = cast<AtomicMemSetInst>(I); auto &MI = cast<AtomicMemSetInst>(I);
SDValue Dst = getValue(MI.getRawDest()); SDValue Dst = getValue(MI.getRawDest());
SDValue Val = getValue(MI.getValue()); SDValue Val = getValue(MI.getValue());
SDValue Length = getValue(MI.getLength()); SDValue Length = getValue(MI.getLength());
Type *LengthTy = MI.getLength()->getType(); Type *LengthTy = MI.getLength()->getType();
unsigned ElemSz = MI.getElementSizeInBytes(); unsigned ElemSz = MI.getElementSizeInBytes();
bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget()); bool isTC = I.isTailCall() && isInTailCallPosition(I, DAG.getTarget());
SDValue MC = SDValue MC =
DAG.getAtomicMemset(getRoot(), sdl, Dst, Val, Length, LengthTy, ElemSz, DAG.getAtomicMemset(getRoot(), sdl, Dst, Val, Length, LengthTy, ElemSz,
isTC, MachinePointerInfo(MI.getRawDest())); isTC, MachinePointerInfo(MI.getRawDest()));
updateDAGForMaybeTailCall(MC); updateDAGForMaybeTailCall(MC);
setValue(&I, MC);
return; return;
} }
case Intrinsic::call_preallocated_setup: { case Intrinsic::call_preallocated_setup: {
const CallBase *PreallocatedCall = FindPreallocatedCall(&I); const CallBase *PreallocatedCall = FindPreallocatedCall(&I);
SDValue SrcValue = DAG.getSrcValue(PreallocatedCall); SDValue SrcValue = DAG.getSrcValue(PreallocatedCall);
SDValue Res = DAG.getNode(ISD::PREALLOCATED_SETUP, sdl, MVT::Other, SDValue Res = DAG.getNode(ISD::PREALLOCATED_SETUP, sdl, MVT::Other,
getRoot(), SrcValue); getRoot(), SrcValue);
setValue(&I, Res); setValue(&I, Res);
▲ Show 20 Lines • Show All 5,522 Lines • Show Last 20 Lines

llvm/test/CodeGen/AArch64/pcsections-memtransfer.ll

  • This file was added.
; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
; RUN: llc < %s -global-isel=0 -mtriple=aarch64-unknown-linux-gnu -stop-after=aarch64-expand-pseudo -verify-machineinstrs | FileCheck %s
define i64 @call_memcpy_intrinsic(ptr %src, ptr %dst) {
; CHECK-LABEL: name: call_memcpy_intrinsic
; CHECK: bb.0 (%ir-block.0):
; CHECK-NEXT: liveins: $x0, $x1
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: renamable $w8 = LDRBBui renamable $x1, 0 :: (volatile load (s8) from %ir.dst)
; CHECK-NEXT: STRBBui killed renamable $w8, killed renamable $x0, 0, pcsections !0 :: (volatile store (s8) into %ir.src)
; CHECK-NEXT: renamable $x0 = LDRXui killed renamable $x1, 0 :: (load (s64) from %ir.dst)
; CHECK-NEXT: RET undef $lr, implicit $x0
call void @llvm.memcpy.p0.p0.i64(ptr %src, ptr %dst, i64 1, i1 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
define i64 @call_memcpy_inline_intrinsic(ptr %src, ptr %dst) {
; CHECK-LABEL: name: call_memcpy_inline_intrinsic
; CHECK: bb.0 (%ir-block.0):
; CHECK-NEXT: liveins: $x0, $x1
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: renamable $w8 = LDRBBui renamable $x1, 0 :: (volatile load (s8) from %ir.dst)
; CHECK-NEXT: STRBBui killed renamable $w8, killed renamable $x0, 0, pcsections !0 :: (volatile store (s8) into %ir.src)
; CHECK-NEXT: renamable $x0 = LDRXui killed renamable $x1, 0 :: (load (s64) from %ir.dst)
; CHECK-NEXT: RET undef $lr, implicit $x0
call void @llvm.memcpy.inline.p0.p0.i64(ptr %src, ptr %dst, i64 1, i1 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
define i64 @call_memmove_intrinsic(ptr %src, ptr %dst) {
; CHECK-LABEL: name: call_memmove_intrinsic
; CHECK: bb.0 (%ir-block.0):
; CHECK-NEXT: liveins: $x0, $x1
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: renamable $w8 = LDRBBui renamable $x1, 0 :: (volatile load (s8) from %ir.dst)
; CHECK-NEXT: STRBBui killed renamable $w8, killed renamable $x0, 0, pcsections !0 :: (volatile store (s8) into %ir.src)
; CHECK-NEXT: renamable $x0 = LDRXui killed renamable $x1, 0 :: (load (s64) from %ir.dst)
; CHECK-NEXT: RET undef $lr, implicit $x0
call void @llvm.memmove.p0.p0.i64(ptr %src, ptr %dst, i64 1, i1 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
define i64 @call_memmove_inline_intrinsic(ptr %src, ptr %dst) {
; CHECK-LABEL: name: call_memmove_inline_intrinsic
; CHECK: bb.0 (%ir-block.0):
; CHECK-NEXT: liveins: $x0, $x1
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: renamable $w8 = LDRBBui renamable $x1, 0 :: (volatile load (s8) from %ir.dst)
; CHECK-NEXT: STRBBui killed renamable $w8, killed renamable $x0, 0, pcsections !0 :: (volatile store (s8) into %ir.src)
; CHECK-NEXT: renamable $x0 = LDRXui killed renamable $x1, 0 :: (load (s64) from %ir.dst)
; CHECK-NEXT: RET undef $lr, implicit $x0
call void @llvm.memmove.inline.p0.p0.i64(ptr %src, ptr %dst, i64 1, i1 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
define i64 @call_memset_intrinsic(ptr %dst) {
; CHECK-LABEL: name: call_memset_intrinsic
; CHECK: bb.0 (%ir-block.0):
; CHECK-NEXT: liveins: $x0
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: STRBBui $wzr, renamable $x0, 0, pcsections !0 :: (volatile store (s8) into %ir.dst)
; CHECK-NEXT: renamable $x0 = LDRXui killed renamable $x0, 0 :: (load (s64) from %ir.dst)
; CHECK-NEXT: RET undef $lr, implicit $x0
call void @llvm.memset.p0.p0.i64(ptr %dst, i8 0, i64 1, i1 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
define i64 @call_memset_inline_intrinsic(ptr %dst) {
; CHECK-LABEL: name: call_memset_inline_intrinsic
; CHECK: bb.0 (%ir-block.0):
; CHECK-NEXT: liveins: $x0
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: STRBBui $wzr, renamable $x0, 0, pcsections !0 :: (volatile store (s8) into %ir.dst)
; CHECK-NEXT: renamable $x0 = LDRXui killed renamable $x0, 0 :: (load (s64) from %ir.dst)
; CHECK-NEXT: RET undef $lr, implicit $x0
call void @llvm.memset.inline.p0.p0.i64(ptr %dst, i8 0, i64 1, i1 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
!0 = !{!"foo"}
declare void @llvm.memcpy.p0.p0.i64(ptr nocapture writeonly, ptr nocapture readonly, i64, i1)
declare void @llvm.memcpy.inline.p0.p0.i64(ptr nocapture writeonly, ptr nocapture readonly, i64, i1)
declare void @llvm.memmove.p0.p0.i64(ptr nocapture, ptr nocapture readonly, i64, i1)
declare void @llvm.memmove.inline.p0.p0.i64(ptr nocapture, ptr nocapture readonly, i64, i1)
declare void @llvm.memset.p0.p0.i64(ptr nocapture, i8, i64, i1)
declare void @llvm.memset.inline.p0.p0.i64(ptr nocapture, i8, i64, i1)

llvm/test/CodeGen/X86/pcsections-memtransfer.ll

  • This file was added.
; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
; RUN: llc < %s -global-isel=0 -mtriple=x86_64-unknown-linux-gnu -stop-after=finalize-isel -verify-machineinstrs | FileCheck %s -check-prefix=X64
; RUN: llc < %s -global-isel=0 -mtriple=i686-unknown-linux-gnu -stop-after=finalize-isel -verify-machineinstrs | FileCheck %s -check-prefix=X32
define i64 @call_memmove_intrinsic(ptr %src, ptr %dst) {
; X64-LABEL: name: call_memmove_intrinsic
; X64: bb.0 (%ir-block.0):
; X64-NEXT: liveins: $rdi, $rsi
; X64-NEXT: {{ $}}
; X64-NEXT: [[COPY:%[0-9]+]]:gr64 = COPY $rsi
; X64-NEXT: [[COPY1:%[0-9]+]]:gr64 = COPY $rdi
; X64-NEXT: [[MOV8rm:%[0-9]+]]:gr8 = MOV8rm [[COPY]], 1, $noreg, 0, $noreg :: (volatile load (s8) from %ir.dst)
; X64-NEXT: MOV8mr [[COPY1]], 1, $noreg, 0, $noreg, killed [[MOV8rm]], pcsections !0 :: (volatile store (s8) into %ir.src)
; X64-NEXT: [[MOV64rm:%[0-9]+]]:gr64 = MOV64rm [[COPY]], 1, $noreg, 0, $noreg :: (load (s64) from %ir.dst)
; X64-NEXT: $rax = COPY [[MOV64rm]]
; X64-NEXT: RET 0, $rax
; X32-LABEL: name: call_memmove_intrinsic
; X32: bb.0 (%ir-block.0):
; X32-NEXT: [[MOV32rm:%[0-9]+]]:gr32 = MOV32rm %fixed-stack.1, 1, $noreg, 0, $noreg :: (load (s32) from %fixed-stack.1, align 16)
; X32-NEXT: [[MOV32rm1:%[0-9]+]]:gr32 = MOV32rm %fixed-stack.0, 1, $noreg, 0, $noreg :: (load (s32) from %fixed-stack.0)
; X32-NEXT: [[MOV8rm:%[0-9]+]]:gr8 = MOV8rm [[MOV32rm1]], 1, $noreg, 0, $noreg :: (volatile load (s8) from %ir.dst)
; X32-NEXT: MOV8mr killed [[MOV32rm]], 1, $noreg, 0, $noreg, killed [[MOV8rm]], pcsections !0 :: (volatile store (s8) into %ir.src)
; X32-NEXT: [[MOV32rm2:%[0-9]+]]:gr32 = MOV32rm [[MOV32rm1]], 1, $noreg, 0, $noreg :: (load (s32) from %ir.dst)
; X32-NEXT: [[MOV32rm3:%[0-9]+]]:gr32 = MOV32rm [[MOV32rm1]], 1, $noreg, 4, $noreg :: (load (s32) from %ir.dst + 4)
; X32-NEXT: $eax = COPY [[MOV32rm2]]
; X32-NEXT: $edx = COPY [[MOV32rm3]]
; X32-NEXT: RET 0, $eax, $edx
call void @llvm.memmove.p0.p0.i64(ptr %src, ptr %dst, i64 1, i1 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
define i64 @call_memmove_inline_intrinsic(ptr %src, ptr %dst) {
; X64-LABEL: name: call_memmove_inline_intrinsic
; X64: bb.0 (%ir-block.0):
; X64-NEXT: liveins: $rdi, $rsi
; X64-NEXT: {{ $}}
; X64-NEXT: [[COPY:%[0-9]+]]:gr64 = COPY $rsi
; X64-NEXT: [[COPY1:%[0-9]+]]:gr64 = COPY $rdi
; X64-NEXT: [[MOV8rm:%[0-9]+]]:gr8 = MOV8rm [[COPY]], 1, $noreg, 0, $noreg :: (volatile load (s8) from %ir.dst)
; X64-NEXT: MOV8mr [[COPY1]], 1, $noreg, 0, $noreg, killed [[MOV8rm]], pcsections !0 :: (volatile store (s8) into %ir.src)
; X64-NEXT: [[MOV64rm:%[0-9]+]]:gr64 = MOV64rm [[COPY]], 1, $noreg, 0, $noreg :: (load (s64) from %ir.dst)
; X64-NEXT: $rax = COPY [[MOV64rm]]
; X64-NEXT: RET 0, $rax
; X32-LABEL: name: call_memmove_inline_intrinsic
; X32: bb.0 (%ir-block.0):
; X32-NEXT: [[MOV32rm:%[0-9]+]]:gr32 = MOV32rm %fixed-stack.1, 1, $noreg, 0, $noreg :: (load (s32) from %fixed-stack.1, align 16)
; X32-NEXT: [[MOV32rm1:%[0-9]+]]:gr32 = MOV32rm %fixed-stack.0, 1, $noreg, 0, $noreg :: (load (s32) from %fixed-stack.0)
; X32-NEXT: [[MOV8rm:%[0-9]+]]:gr8 = MOV8rm [[MOV32rm1]], 1, $noreg, 0, $noreg :: (volatile load (s8) from %ir.dst)
; X32-NEXT: MOV8mr killed [[MOV32rm]], 1, $noreg, 0, $noreg, killed [[MOV8rm]], pcsections !0 :: (volatile store (s8) into %ir.src)
; X32-NEXT: [[MOV32rm2:%[0-9]+]]:gr32 = MOV32rm [[MOV32rm1]], 1, $noreg, 0, $noreg :: (load (s32) from %ir.dst)
; X32-NEXT: [[MOV32rm3:%[0-9]+]]:gr32 = MOV32rm [[MOV32rm1]], 1, $noreg, 4, $noreg :: (load (s32) from %ir.dst + 4)
; X32-NEXT: $eax = COPY [[MOV32rm2]]
; X32-NEXT: $edx = COPY [[MOV32rm3]]
; X32-NEXT: RET 0, $eax, $edx
call void @llvm.memmove.inline.p0.p0.i64(ptr %src, ptr %dst, i64 1, i1 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
define i64 @call_memset_intrinsic(ptr %dst) {
; X64-LABEL: name: call_memset_intrinsic
; X64: bb.0 (%ir-block.0):
; X64-NEXT: liveins: $rdi
; X64-NEXT: {{ $}}
; X64-NEXT: [[COPY:%[0-9]+]]:gr64 = COPY $rdi
; X64-NEXT: MOV8mi [[COPY]], 1, $noreg, 0, $noreg, 0, pcsections !0 :: (volatile store (s8) into %ir.dst)
; X64-NEXT: [[MOV64rm:%[0-9]+]]:gr64 = MOV64rm [[COPY]], 1, $noreg, 0, $noreg :: (load (s64) from %ir.dst)
; X64-NEXT: $rax = COPY [[MOV64rm]]
; X64-NEXT: RET 0, $rax
; X32-LABEL: name: call_memset_intrinsic
; X32: bb.0 (%ir-block.0):
; X32-NEXT: [[MOV32rm:%[0-9]+]]:gr32 = MOV32rm %fixed-stack.0, 1, $noreg, 0, $noreg :: (load (s32) from %fixed-stack.0, align 16)
; X32-NEXT: MOV8mi [[MOV32rm]], 1, $noreg, 0, $noreg, 0, pcsections !0 :: (volatile store (s8) into %ir.dst)
; X32-NEXT: [[MOV32rm1:%[0-9]+]]:gr32 = MOV32rm [[MOV32rm]], 1, $noreg, 4, $noreg :: (load (s32) from %ir.dst + 4)
; X32-NEXT: [[MOV32rm2:%[0-9]+]]:gr32 = MOV32rm [[MOV32rm]], 1, $noreg, 0, $noreg :: (load (s32) from %ir.dst)
; X32-NEXT: $eax = COPY [[MOV32rm2]]
; X32-NEXT: $edx = COPY [[MOV32rm1]]
; X32-NEXT: RET 0, $eax, $edx
call void @llvm.memset.p0.p0.i64(ptr %dst, i8 0, i64 1, i1 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
define i64 @call_memset_inline_intrinsic(ptr %dst) {
; X64-LABEL: name: call_memset_inline_intrinsic
; X64: bb.0 (%ir-block.0):
; X64-NEXT: liveins: $rdi
; X64-NEXT: {{ $}}
; X64-NEXT: [[COPY:%[0-9]+]]:gr64 = COPY $rdi
; X64-NEXT: MOV8mi [[COPY]], 1, $noreg, 0, $noreg, 0, pcsections !0 :: (volatile store (s8) into %ir.dst)
; X64-NEXT: [[MOV64rm:%[0-9]+]]:gr64 = MOV64rm [[COPY]], 1, $noreg, 0, $noreg :: (load (s64) from %ir.dst)
; X64-NEXT: $rax = COPY [[MOV64rm]]
; X64-NEXT: RET 0, $rax
; X32-LABEL: name: call_memset_inline_intrinsic
; X32: bb.0 (%ir-block.0):
; X32-NEXT: [[MOV32rm:%[0-9]+]]:gr32 = MOV32rm %fixed-stack.0, 1, $noreg, 0, $noreg :: (load (s32) from %fixed-stack.0, align 16)
; X32-NEXT: MOV8mi [[MOV32rm]], 1, $noreg, 0, $noreg, 0, pcsections !0 :: (volatile store (s8) into %ir.dst)
; X32-NEXT: [[MOV32rm1:%[0-9]+]]:gr32 = MOV32rm [[MOV32rm]], 1, $noreg, 4, $noreg :: (load (s32) from %ir.dst + 4)
; X32-NEXT: [[MOV32rm2:%[0-9]+]]:gr32 = MOV32rm [[MOV32rm]], 1, $noreg, 0, $noreg :: (load (s32) from %ir.dst)
; X32-NEXT: $eax = COPY [[MOV32rm2]]
; X32-NEXT: $edx = COPY [[MOV32rm1]]
; X32-NEXT: RET 0, $eax, $edx
call void @llvm.memset.inline.p0.p0.i64(ptr %dst, i8 0, i64 1, i1 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
define i64 @call_memcpy_element_unordered_atomic_intrinsic() {
; X64-LABEL: name: call_memcpy_element_unordered_atomic_intrinsic
; X64: bb.0 (%ir-block.0):
; X64-NEXT: ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp
; X64-NEXT: [[LEA64r:%[0-9]+]]:gr64 = LEA64r %stack.0.src, 1, $noreg, 0, $noreg
; X64-NEXT: [[LEA64r1:%[0-9]+]]:gr64 = LEA64r %stack.1.dst, 1, $noreg, 0, $noreg
; X64-NEXT: [[MOV32ri64_:%[0-9]+]]:gr64 = MOV32ri64 1
; X64-NEXT: $rdi = COPY [[LEA64r]]
; X64-NEXT: $rsi = COPY [[LEA64r1]]
; X64-NEXT: $rdx = COPY [[MOV32ri64_]]
; X64-NEXT: CALL64pcrel32 target-flags(x86-plt) &__llvm_memcpy_element_unordered_atomic_1, csr_64, implicit $rsp, implicit $ssp, implicit $rdi, implicit $rsi, implicit $rdx, implicit-def $rsp, implicit-def $ssp
; X64-NEXT: ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp, pcsections !0
; X64-NEXT: [[MOV64rm:%[0-9]+]]:gr64 = MOV64rm %stack.1.dst, 1, $noreg, 0, $noreg :: (load (s64) from %ir.dst)
; X64-NEXT: $rax = COPY [[MOV64rm]]
; X64-NEXT: RET 0, $rax
; X32-LABEL: name: call_memcpy_element_unordered_atomic_intrinsic
; X32: bb.0 (%ir-block.0):
; X32-NEXT: ADJCALLSTACKDOWN32 16, 0, 0, implicit-def dead $esp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $esp, implicit $ssp
; X32-NEXT: [[COPY:%[0-9]+]]:gr32 = COPY $esp
; X32-NEXT: [[LEA32r:%[0-9]+]]:gr32 = LEA32r %stack.1.dst, 1, $noreg, 0, $noreg
; X32-NEXT: MOV32mr [[COPY]], 1, $noreg, 4, $noreg, killed [[LEA32r]] :: (store (s32) into stack + 4)
; X32-NEXT: [[LEA32r1:%[0-9]+]]:gr32 = LEA32r %stack.0.src, 1, $noreg, 0, $noreg
; X32-NEXT: MOV32mr [[COPY]], 1, $noreg, 0, $noreg, killed [[LEA32r1]] :: (store (s32) into stack)
; X32-NEXT: MOV32mi [[COPY]], 1, $noreg, 12, $noreg, 0 :: (store (s32) into stack + 12)
; X32-NEXT: MOV32mi [[COPY]], 1, $noreg, 8, $noreg, 1 :: (store (s32) into stack + 8)
; X32-NEXT: CALLpcrel32 &__llvm_memcpy_element_unordered_atomic_1, csr_32, implicit $esp, implicit $ssp, implicit-def $esp, implicit-def $ssp
; X32-NEXT: ADJCALLSTACKUP32 16, 0, implicit-def dead $esp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $esp, implicit $ssp, pcsections !0
; X32-NEXT: [[MOV32rm:%[0-9]+]]:gr32 = MOV32rm %stack.1.dst, 1, $noreg, 0, $noreg :: (load (s32) from %ir.dst)
; X32-NEXT: [[MOV32rm1:%[0-9]+]]:gr32 = MOV32rm %stack.1.dst, 1, $noreg, 4, $noreg :: (load (s32) from %ir.dst + 4)
; X32-NEXT: $eax = COPY [[MOV32rm]]
; X32-NEXT: $edx = COPY [[MOV32rm1]]
; X32-NEXT: RET 0, $eax, $edx
%src = alloca i32, align 1
%dst = alloca i32, align 1
call void @llvm.memcpy.element.unordered.atomic.p0.p0.i64(ptr align 1 %src, ptr align 1 %dst, i64 1, i32 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
define i64 @call_memmove_element_unordered_atomic_intrinsic() {
; X64-LABEL: name: call_memmove_element_unordered_atomic_intrinsic
; X64: bb.0 (%ir-block.0):
; X64-NEXT: ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp
; X64-NEXT: [[LEA64r:%[0-9]+]]:gr64 = LEA64r %stack.0.src, 1, $noreg, 0, $noreg
; X64-NEXT: [[LEA64r1:%[0-9]+]]:gr64 = LEA64r %stack.1.dst, 1, $noreg, 0, $noreg
; X64-NEXT: [[MOV32ri64_:%[0-9]+]]:gr64 = MOV32ri64 1
; X64-NEXT: $rdi = COPY [[LEA64r]]
; X64-NEXT: $rsi = COPY [[LEA64r1]]
; X64-NEXT: $rdx = COPY [[MOV32ri64_]]
; X64-NEXT: CALL64pcrel32 target-flags(x86-plt) &__llvm_memmove_element_unordered_atomic_1, csr_64, implicit $rsp, implicit $ssp, implicit $rdi, implicit $rsi, implicit $rdx, implicit-def $rsp, implicit-def $ssp
; X64-NEXT: ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp, pcsections !0
; X64-NEXT: [[MOV64rm:%[0-9]+]]:gr64 = MOV64rm %stack.1.dst, 1, $noreg, 0, $noreg :: (load (s64) from %ir.dst)
; X64-NEXT: $rax = COPY [[MOV64rm]]
; X64-NEXT: RET 0, $rax
; X32-LABEL: name: call_memmove_element_unordered_atomic_intrinsic
; X32: bb.0 (%ir-block.0):
; X32-NEXT: ADJCALLSTACKDOWN32 16, 0, 0, implicit-def dead $esp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $esp, implicit $ssp
; X32-NEXT: [[COPY:%[0-9]+]]:gr32 = COPY $esp
; X32-NEXT: [[LEA32r:%[0-9]+]]:gr32 = LEA32r %stack.1.dst, 1, $noreg, 0, $noreg
; X32-NEXT: MOV32mr [[COPY]], 1, $noreg, 4, $noreg, killed [[LEA32r]] :: (store (s32) into stack + 4)
; X32-NEXT: [[LEA32r1:%[0-9]+]]:gr32 = LEA32r %stack.0.src, 1, $noreg, 0, $noreg
; X32-NEXT: MOV32mr [[COPY]], 1, $noreg, 0, $noreg, killed [[LEA32r1]] :: (store (s32) into stack)
; X32-NEXT: MOV32mi [[COPY]], 1, $noreg, 12, $noreg, 0 :: (store (s32) into stack + 12)
; X32-NEXT: MOV32mi [[COPY]], 1, $noreg, 8, $noreg, 1 :: (store (s32) into stack + 8)
; X32-NEXT: CALLpcrel32 &__llvm_memmove_element_unordered_atomic_1, csr_32, implicit $esp, implicit $ssp, implicit-def $esp, implicit-def $ssp
; X32-NEXT: ADJCALLSTACKUP32 16, 0, implicit-def dead $esp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $esp, implicit $ssp, pcsections !0
; X32-NEXT: [[MOV32rm:%[0-9]+]]:gr32 = MOV32rm %stack.1.dst, 1, $noreg, 0, $noreg :: (load (s32) from %ir.dst)
; X32-NEXT: [[MOV32rm1:%[0-9]+]]:gr32 = MOV32rm %stack.1.dst, 1, $noreg, 4, $noreg :: (load (s32) from %ir.dst + 4)
; X32-NEXT: $eax = COPY [[MOV32rm]]
; X32-NEXT: $edx = COPY [[MOV32rm1]]
; X32-NEXT: RET 0, $eax, $edx
%src = alloca i32, align 1
%dst = alloca i32, align 1
call void @llvm.memmove.element.unordered.atomic.p0.p0.i64(ptr align 1 %src, ptr align 1 %dst, i64 1, i32 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
define i64 @call_memset_element_unordered_atomic_intrinsic() {
; X64-LABEL: name: call_memset_element_unordered_atomic_intrinsic
; X64: bb.0 (%ir-block.0):
; X64-NEXT: ADJCALLSTACKDOWN64 0, 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp
; X64-NEXT: [[LEA64r:%[0-9]+]]:gr64 = LEA64r %stack.0.dst, 1, $noreg, 0, $noreg
; X64-NEXT: [[MOV32r0_:%[0-9]+]]:gr32 = MOV32r0 implicit-def dead $eflags
; X64-NEXT: [[MOV32ri64_:%[0-9]+]]:gr64 = MOV32ri64 1
; X64-NEXT: $rdi = COPY [[LEA64r]]
; X64-NEXT: $esi = COPY [[MOV32r0_]]
; X64-NEXT: $rdx = COPY [[MOV32ri64_]]
; X64-NEXT: CALL64pcrel32 target-flags(x86-plt) &__llvm_memset_element_unordered_atomic_1, csr_64, implicit $rsp, implicit $ssp, implicit $rdi, implicit $esi, implicit $rdx, implicit-def $rsp, implicit-def $ssp
; X64-NEXT: ADJCALLSTACKUP64 0, 0, implicit-def dead $rsp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $rsp, implicit $ssp, pcsections !0
; X64-NEXT: [[MOV64rm:%[0-9]+]]:gr64 = MOV64rm %stack.0.dst, 1, $noreg, 0, $noreg :: (load (s64) from %ir.dst)
; X64-NEXT: $rax = COPY [[MOV64rm]]
; X64-NEXT: RET 0, $rax
; X32-LABEL: name: call_memset_element_unordered_atomic_intrinsic
; X32: bb.0 (%ir-block.0):
; X32-NEXT: ADJCALLSTACKDOWN32 16, 0, 0, implicit-def dead $esp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $esp, implicit $ssp
; X32-NEXT: [[COPY:%[0-9]+]]:gr32 = COPY $esp
; X32-NEXT: [[LEA32r:%[0-9]+]]:gr32 = LEA32r %stack.0.dst, 1, $noreg, 0, $noreg
; X32-NEXT: MOV32mr [[COPY]], 1, $noreg, 0, $noreg, killed [[LEA32r]] :: (store (s32) into stack)
; X32-NEXT: MOV32mi [[COPY]], 1, $noreg, 12, $noreg, 0 :: (store (s32) into stack + 12)
; X32-NEXT: MOV32mi [[COPY]], 1, $noreg, 8, $noreg, 1 :: (store (s32) into stack + 8)
; X32-NEXT: MOV32mi [[COPY]], 1, $noreg, 4, $noreg, 0 :: (store (s32) into stack + 4)
; X32-NEXT: CALLpcrel32 &__llvm_memset_element_unordered_atomic_1, csr_32, implicit $esp, implicit $ssp, implicit-def $esp, implicit-def $ssp
; X32-NEXT: ADJCALLSTACKUP32 16, 0, implicit-def dead $esp, implicit-def dead $eflags, implicit-def dead $ssp, implicit $esp, implicit $ssp, pcsections !0
; X32-NEXT: [[MOV32rm:%[0-9]+]]:gr32 = MOV32rm %stack.0.dst, 1, $noreg, 0, $noreg :: (load (s32) from %ir.dst)
; X32-NEXT: [[MOV32rm1:%[0-9]+]]:gr32 = MOV32rm %stack.0.dst, 1, $noreg, 4, $noreg :: (load (s32) from %ir.dst + 4)
; X32-NEXT: $eax = COPY [[MOV32rm]]
; X32-NEXT: $edx = COPY [[MOV32rm1]]
; X32-NEXT: RET 0, $eax, $edx
%dst = alloca i32, align 1
call void @llvm.memset.element.unordered.atomic.p0.p0.i64(ptr align 1%dst, i8 0, i64 1, i32 1), !pcsections !0
%val = load i64, ptr %dst
ret i64 %val
}
!0 = !{!"foo"}
declare void @llvm.memmove.p0.p0.i64(ptr nocapture, ptr nocapture readonly, i64, i1)
declare void @llvm.memmove.inline.p0.p0.i64(ptr nocapture, ptr nocapture readonly, i64, i1)
declare void @llvm.memset.p0.p0.i64(ptr nocapture, i8, i64, i1)
declare void @llvm.memset.inline.p0.p0.i64(ptr nocapture, i8, i64, i1)
declare void @llvm.memcpy.element.unordered.atomic.p0.p0.i64(ptr nocapture writeonly, ptr nocapture readonly, i64, i32)
declare void @llvm.memmove.element.unordered.atomic.p0.p0.i64(ptr nocapture writeonly, ptr nocapture readonly, i64, i32)
declare void @llvm.memset.element.unordered.atomic.p0.p0.i64(ptr nocapture writeonly, i8, i64, i32)