This is an archive of the discontinued LLVM Phabricator instance.

Differential D145441

[AMDGPU] Define data layout entries for buffers
ClosedPublic

Authored by krzysz00 on Mar 6 2023, 3:23 PM.

Details

Reviewers
nhaehnle
arsenm
b-sumner
piotr
sstefan1
jdoerfert
Commits
rGf9c1ede2543b: [AMDGPU] Define data layout entries for buffers
Summary

Per discussion at
https://discourse.llvm.org/t/representing-buffer-descriptors-in-the-amdgpu-target-call-for-suggestions/68798,
we define two new address spaces for AMDGCN targets.

The first is address space 7, a non-integral address space (which was
already in the data layout) that has 160-bit pointers (which are
256-bit aligned) and uses a 32-bit offset. These pointers combine a
128-bit buffer descriptor and a 32-bit offset, and will be usable with
normal LLVM operations (load, store, GEP). However, they will be
rewritten out of existence before code generation.

The second of these is address space 8, the address space for "buffer
resources". These will be used to represent the resource arguments to
buffer instructions, and new buffer intrinsics will be defined that
take them instead of <4 x i32> as resource arguments. ptr
addrspace(8). These pointers are 128-bits long (with the same
alignment). They must not be used as the arguments to getelementptr or
otherwise used in address computations, since they can have
arbitrarily complex inherent addressing semantics that can't be
represented in LLVM. Even though, like their address space 7 cousins,
these pointers have deterministic ptrtoint/inttoptr semantics, they
are defined to be non-integral in order to prevent optimizations that
rely on pointers being a [0, [addr_max]] value from applying to them.

Future work includes:

  • Defining new buffer intrinsics that take ptr addrspace(8) resources.
  • A late rewrite to turn address space 7 operations into buffer

intrinsics and offset computations.

This commit also updates the "fallback address space" for buffer
intrinsics to the buffer resource, and updates the alias analysis
table.

Depends on D143437

Diff Detail

Event Timeline

krzysz00 created this revision.Mar 6 2023, 3:23 PM
Herald added a project: Restricted Project. · View Herald TranscriptMar 6 2023, 3:23 PM
Herald added subscribers: kosarev, jeroen.dobbelaere, foad and 14 others. · View Herald Transcript
krzysz00 requested review of this revision.Mar 6 2023, 3:23 PM
Herald added projects: Restricted Project, Restricted Project. · View Herald TranscriptMar 6 2023, 3:23 PM
Herald added subscribers: llvm-commits, cfe-commits, • pcwang-thead, wdng. · View Herald Transcript
Harbormaster completed remote builds in B217729: Diff 502829.Mar 6 2023, 8:43 PM
foad added a comment.Mar 7 2023, 1:51 AM

Just my 2p: it feels a bit premature to commit patches for this. It feels more like something you could prototype on a branch somewhere and come back when you have more experience with how it all works out in practice.

But I don't actually object to the patch, if the other reviewers are happy and it doens't break anything.

The first is address space 7, a non-integral address space (which was
already in the data layout) that has 160-bit pointers (which are
256-bit aligned)

Any particular reason for choosing 256-bit alignment?

However, they must not be used as the arguments to
getelementptr or otherwise used in address computations

I don't understand what kind of rule this is and how it would be enforced. Is it something that will be written into the IR LangRef?

This commit also updates the "fallback address space" for buffer
intrinsics to the buffer resource,

It's not clear to me that this is any more or less correct, since 7 and 8 behave identically wrt alias analysis don't they?

krzysz00 added a comment.Mar 7 2023, 8:12 AM

@foad I was trying to avoid sending in one mega-patch that has the entire prototype.

As to your comments:

  • Why 256-bit alignment? It's the next power of 2, and using an alignment that isn't a power of 2 causes an assertion failure
  • Re not being allowed to use address space 8 pointers in the usual LLVM operations, I'd call that a target-specific rule about what address space 8 means. I figure this is something that we can check in some sort of AMDGPU-specific validation
  • I changed the fallback address space because the plan is that those intrinsics will be taking address space 8 arguments in the future
nhaehnle added a comment.EditedMar 7 2023, 12:08 PM
In D145441#4175428, @krzysz00 wrote:

@foad I was trying to avoid sending in one mega-patch that has the entire prototype.

The best way to do this is to work on a branch that you rebase regularly. You can push the branch to a personal clone of llvm-project on GitHub and point folks there from e.g. Discourse, and for the purpose of Phabricator you can set up a "Stack" via the "Edit Related Revisions" option.

If you're using arc, then git rebase -i -x "arc diff @^" is one way to update a bunch of patches at once. There are probably other tools that people have written.

These [addrspace(8) pointers] are, however, integral, since inttoptr and ptrtoint behave deterministically and reasonably.

That doesn't make sense to me. I thought "integral" means that

inttoptr(ptrtoint(p) + offset)

is the same (in terms of address, and potentially modulo provenance questions) as

gep i8, p, offset

But that is rather untrue for addrspace(8), isn't it?

llvm/docs/AMDGPUUsage.rst
795

Typo: experimental

805–806

Presumably it is only permitted if the underlying descriptor satisfies all the rules mentioned for addrspace(7).

krzysz00 updated this revision to Diff 507869.Mar 23 2023, 1:47 PM

Fix typo, rebase while making next patch

krzysz00 updated this revision to Diff 507870.Mar 23 2023, 1:50 PM

... actually fix the typo

krzysz00 added a child revision: D146761: [AMDGPU] Make buffer intrinsics pointer-valued.Mar 23 2023, 2:38 PM
krzysz00 updated this revision to Diff 509761.Mar 30 2023, 11:58 AM
krzysz00 edited the summary of this revision. (Show Details)
krzysz00 added a parent revision: D147245: [AMDGPU] Make amdgcn.s.buffer.load a memory-reading intrinsic.

Rebase for splitting s.buffer.load effect change into earlier patch

Harbormaster completed remote builds in B222808: Diff 509761.Mar 30 2023, 1:26 PM
krzysz00 updated this revision to Diff 510056.Mar 31 2023, 9:20 AM
krzysz00 edited the summary of this revision. (Show Details)

Per discussion on the s.buffer.load revision, don't make those changes, and so revert this patch back to what it used to be.

Harbormaster completed remote builds in B223029: Diff 510056.Mar 31 2023, 9:21 AM
krzysz00 updated this revision to Diff 510848.Apr 4 2023, 10:20 AM

Rebase data layout changes

Harbormaster completed remote builds in B223611: Diff 510848.Apr 4 2023, 10:21 AM
krzysz00 updated this revision to Diff 512488.Apr 11 2023, 9:12 AM

Rebase, since we have addrspace 128

Harbormaster completed remote builds in B224802: Diff 512488.Apr 11 2023, 9:13 AM
krzysz00 updated this revision to Diff 514232.Apr 17 2023, 7:50 AM

Fix new merge conflicts

Harbormaster completed remote builds in B226101: Diff 514232.Apr 17 2023, 7:51 AM
krzysz00 removed a parent revision: D147245: [AMDGPU] Make amdgcn.s.buffer.load a memory-reading intrinsic.Apr 21 2023, 8:45 AM
krzysz00 added a child revision: D147547: [AMDGPU] Add buffer intrinsics that take resources as pointers.
krzysz00 updated this revision to Diff 516953.Apr 25 2023, 3:18 PM
krzysz00 edited the summary of this revision. (Show Details)

Make address space 8 non-integral as well because that's a good idea and we'll have an address space cast intrinsic.

arsenm accepted this revision.May 1 2023, 1:26 PM

LGTM. It may not be the end step and may be annoying to make further changes from this point, but on its own this isn't worse than before

llvm/lib/Target/AMDGPU/SIISelLowering.cpp
998

Should this todo be dropped?

This revision is now accepted and ready to land.May 1 2023, 1:26 PM
krzysz00 updated this revision to Diff 518843.May 2 2023, 1:19 PM

Rebase to handle new legalization tests.

Harbormaster completed remote builds in B229517: Diff 518843.May 2 2023, 7:52 PM
This revision was landed with ongoing or failed builds.May 3 2023, 8:26 AM
Closed by commit rGf9c1ede2543b: [AMDGPU] Define data layout entries for buffers (authored by krzysz00). · Explain Why
This revision was automatically updated to reflect the committed changes.
krzysz00 added a commit: rGf9c1ede2543b: [AMDGPU] Define data layout entries for buffers.
jplehr added a subscriber: jplehr.May 3 2023, 9:10 AM

Hey Krzysztof,
I believe this broke the openmp offload buildbot https://lab.llvm.org/buildbot/#/builders/193/builds/30576

krzysz00 added a reverting change: D149758: Revert "[AMDGPU] Define data layout entries for buffers".May 3 2023, 9:10 AM
krzysz00 added a reverting change: rG3f2fbe92d0f4: Revert "[AMDGPU] Define data layout entries for buffers".May 3 2023, 9:11 AM
krzysz00 mentioned this in D149776: Re-land "[AMDGPU] Define data layout entries for buffers"".May 3 2023, 11:52 AM
krzysz00 mentioned this in rGf0415f2a456d: Re-land "[AMDGPU] Define data layout entries for buffers"".May 3 2023, 12:44 PM

Large Diff

This large diff affects 148 files. Files without inline comments have been collapsed. Expand All Files

Revision Contents

PathSize
clang/
lib/
Basic/
Targets/
4 lines
test/
CodeGen/
4 lines
CodeGenOpenCL/
2 lines
llvm/
docs/
21 lines
lib/
IR/
30 lines
Target/
AMDGPU/
5 lines
27 lines
12 lines
3 lines
8 lines
test/
CodeGen/
AMDGPU/
GlobalISel/
16 lines
16 lines
16 lines
16 lines
8 lines
96 lines
168 lines
96 lines
48 lines
8 lines
8 lines
282 lines
16 lines
12 lines
76 lines
32 lines
40 lines
16 lines
10 lines
2 lines
40 lines
20 lines
12 lines
70 lines
44 lines
24 lines
66 lines
32 lines
18 lines
40 lines
16 lines
50 lines
14 lines
10 lines
2 lines
40 lines
24 lines
20 lines
32 lines
20 lines
12 lines
20 lines
24 lines
14 lines
16 lines
20 lines
10 lines
10 lines
10 lines
2 lines
2 lines
16 lines
16 lines
16 lines
16 lines
8 lines
200 lines
2 lines
4 lines
2 lines
10 lines
12 lines
70 lines
22 lines
24 lines
66 lines
16 lines
18 lines
20 lines
50 lines
32 lines
2 lines
2 lines
2 lines
18 lines
15 lines
2 lines
2 lines
2 lines
62 lines
2 lines
2 lines
2 lines
2 lines
MIR/
AMDGPU/
4 lines
Instrumentation/
AddressSanitizer/
AMDGPU/
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
Transforms/
AlignmentFromAssumptions/
2 lines
AtomicExpand/
AMDGPU/
2 lines
EarlyCSE/
AMDGPU/
2 lines
IndVarSimplify/
AMDGPU/
2 lines
InferAddressSpaces/
AMDGPU/
2 lines
2 lines
X86/
2 lines
Inline/
AMDGPU/
2 lines
InstCombine/
AMDGPU/
2 lines
2 lines
LoadStoreVectorizer/
AMDGPU/
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
2 lines
LoopLoadElim/
2 lines
LoopStrengthReduce/
AMDGPU/
2 lines
2 lines
2 lines
2 lines
2 lines
OpenMP/
2 lines
2 lines
2 lines
SLPVectorizer/
AMDGPU/
2 lines
VectorCombine/
AMDGPU/
2 lines
2 lines
unittests/
Bitcode/
34 lines

Diff 519084

clang/lib/Basic/Targets/AMDGPU.cpp

clang/test/CodeGen/target-data.c

clang/test/CodeGenOpenCL/amdgpu-env-amdgcn.cl

llvm/docs/AMDGPUUsage.rst

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 669 Lines • ▼ Show 20 Lines .. table:: AMDGPU Address Spaces
Generic 0 flat flat 64 0x0000000000000000 Generic 0 flat flat 64 0x0000000000000000
Global 1 global global 64 0x0000000000000000 Global 1 global global 64 0x0000000000000000
Region 2 N/A GDS 32 *not implemented for AMDHSA* Region 2 N/A GDS 32 *not implemented for AMDHSA*
Local 3 group LDS 32 0xFFFFFFFF Local 3 group LDS 32 0xFFFFFFFF
Constant 4 constant *same as global* 64 0x0000000000000000 Constant 4 constant *same as global* 64 0x0000000000000000
Private 5 private scratch 32 0xFFFFFFFF Private 5 private scratch 32 0xFFFFFFFF
Constant 32-bit 6 *TODO* 0x00000000 Constant 32-bit 6 *TODO* 0x00000000
Buffer Fat Pointer (experimental) 7 *TODO* Buffer Fat Pointer (experimental) 7 *TODO*
Buffer Resource (experimental) 8 *TODO*
Streamout Registers 128 N/A GS_REGS Streamout Registers 128 N/A GS_REGS
================================= =============== =========== ================ ======= ============================ ================================= =============== =========== ================ ======= ============================
**Generic** **Generic**
The generic address space is supported unless the *Target Properties* column The generic address space is supported unless the *Target Properties* column
of :ref:`amdgpu-processor-table` specifies *Does not support generic address of :ref:`amdgpu-processor-table` specifies *Does not support generic address
space*. space*.
▲ Show 20 Lines • Show All 93 Lines • ▼ Show 20 Lines**Buffer Fat Pointer**
The buffer fat pointer is an experimental address space that is currently The buffer fat pointer is an experimental address space that is currently
unsupported in the backend. It exposes a non-integral pointer that is in unsupported in the backend. It exposes a non-integral pointer that is in
the future intended to support the modelling of 128-bit buffer descriptors the future intended to support the modelling of 128-bit buffer descriptors
plus a 32-bit offset into the buffer (in total encapsulating a 160-bit plus a 32-bit offset into the buffer (in total encapsulating a 160-bit
*pointer*), allowing normal LLVM load/store/atomic operations to be used to *pointer*), allowing normal LLVM load/store/atomic operations to be used to
model the buffer descriptors used heavily in graphics workloads targeting model the buffer descriptors used heavily in graphics workloads targeting
the backend. the backend.
The buffer descriptor used to construct a buffer fat pointer must be *raw*:
the stride must be 0, the "add tid" flag bust be 0, the swizzle enable bits
must be off, and the extent must be measured in bytes. (On subtargets where
bounds checking may be disabled, buffer fat pointers may choose to enable
it or not).
**Buffer Resource**
The buffer resource is an experimental address space that is currently unsupported
nhaehnleUnsubmitted
Not Done
ReplyInline Actions

Typo: experimental

nhaehnle: Typo: experimental
in the backend. It exposes a non-integral pointer that will represent a 128-bit
buffer descriptor resource.
Since, in general, a buffer resource supports complex addressing modes that cannot
be easily represented in LLVM (such as implicit swizzled access to structured
buffers), it is **illegal** to perform non-trivial address computations, such as
``getelementptr`` operations, on buffer resources. They may be passed to
AMDGPU buffer intrinsics, and they may be converted to and from ``i128``.
Casting a buffer resource to a bufer fat pointer is permitted and adds an offset
of 0.
nhaehnleUnsubmitted
Not Done
ReplyInline Actions

Presumably it is only permitted if the underlying descriptor satisfies all the rules mentioned for addrspace(7).

nhaehnle: Presumably it is only permitted if the underlying descriptor satisfies all the rules mentioned…
**Streamout Registers** **Streamout Registers**
Dedicated registers used by the GS NGG Streamout Instructions. The register Dedicated registers used by the GS NGG Streamout Instructions. The register
file is modelled as a memory in a distinct address space because it is indexed file is modelled as a memory in a distinct address space because it is indexed
by an address-like offset in place of named registers, and because register by an address-like offset in place of named registers, and because register
accesses affect LGKMcnt. This is an internal address space used only by the accesses affect LGKMcnt. This is an internal address space used only by the
compiler. Do not use this address space for IR pointers. compiler. Do not use this address space for IR pointers.
.. _amdgpu-memory-scopes: .. _amdgpu-memory-scopes:
▲ Show 20 Lines • Show All 14,402 Lines • Show Last 20 Lines

llvm/lib/IR/AutoUpgrade.cpp

llvm/lib/Target/AMDGPU/AMDGPU.h

llvm/lib/Target/AMDGPU/AMDGPUAliasAnalysis.cpp

llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp

llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

  • This file is larger than 256 KB, so syntax highlighting is disabled by default.
Show First 20 Lines • Show All 989 Lines • ▼ Show 20 Linesbool SITargetLowering::getTgtMemIntrinsic(IntrinsicInfo &Info,
if (const AMDGPU::RsrcIntrinsic *RsrcIntr = if (const AMDGPU::RsrcIntrinsic *RsrcIntr =
AMDGPU::lookupRsrcIntrinsic(IntrID)) { AMDGPU::lookupRsrcIntrinsic(IntrID)) {
AttributeList Attr = Intrinsic::getAttributes(CI.getContext(), AttributeList Attr = Intrinsic::getAttributes(CI.getContext(),
(Intrinsic::ID)IntrID); (Intrinsic::ID)IntrID);
MemoryEffects ME = Attr.getMemoryEffects(); MemoryEffects ME = Attr.getMemoryEffects();
if (ME.doesNotAccessMemory()) if (ME.doesNotAccessMemory())
return false; return false;
// TODO: Should images get their own address space? // TODO: Should images get their own address space?
arsenmUnsubmitted
Not Done
ReplyInline Actions

Should this todo be dropped?

arsenm: Should this todo be dropped?
Info.fallbackAddressSpace = AMDGPUAS::BUFFER_FAT_POINTER; Info.fallbackAddressSpace = AMDGPUAS::BUFFER_RESOURCE;
if (RsrcIntr->IsImage) if (RsrcIntr->IsImage)
Info.align.reset(); Info.align.reset();
Info.flags |= MachineMemOperand::MODereferenceable; Info.flags |= MachineMemOperand::MODereferenceable;
if (ME.onlyReadsMemory()) { if (ME.onlyReadsMemory()) {
unsigned MaxNumLanes = 4; unsigned MaxNumLanes = 4;
▲ Show 20 Lines • Show All 73 Lines • ▼ Show 20 Lines case Intrinsic::amdgcn_ds_fmax: {
if (!Vol->isZero()) if (!Vol->isZero())
Info.flags |= MachineMemOperand::MOVolatile; Info.flags |= MachineMemOperand::MOVolatile;
return true; return true;
} }
case Intrinsic::amdgcn_buffer_atomic_fadd: { case Intrinsic::amdgcn_buffer_atomic_fadd: {
Info.opc = ISD::INTRINSIC_W_CHAIN; Info.opc = ISD::INTRINSIC_W_CHAIN;
Info.memVT = MVT::getVT(CI.getOperand(0)->getType()); Info.memVT = MVT::getVT(CI.getOperand(0)->getType());
Info.fallbackAddressSpace = AMDGPUAS::BUFFER_FAT_POINTER; Info.fallbackAddressSpace = AMDGPUAS::BUFFER_RESOURCE;
Info.align.reset(); Info.align.reset();
Info.flags |= MachineMemOperand::MOLoad | MachineMemOperand::MOStore; Info.flags |= MachineMemOperand::MOLoad | MachineMemOperand::MOStore;
const ConstantInt *Vol = dyn_cast<ConstantInt>(CI.getOperand(4)); const ConstantInt *Vol = dyn_cast<ConstantInt>(CI.getOperand(4));
if (!Vol || !Vol->isZero()) if (!Vol || !Vol->isZero())
Info.flags |= MachineMemOperand::MOVolatile; Info.flags |= MachineMemOperand::MOVolatile;
return true; return true;
Show All 30 Lines Info.flags |= MachineMemOperand::MOLoad |
MachineMemOperand::MOStore | MachineMemOperand::MOStore |
MachineMemOperand::MOVolatile; MachineMemOperand::MOVolatile;
return true; return true;
} }
case Intrinsic::amdgcn_image_bvh_intersect_ray: { case Intrinsic::amdgcn_image_bvh_intersect_ray: {
Info.opc = ISD::INTRINSIC_W_CHAIN; Info.opc = ISD::INTRINSIC_W_CHAIN;
Info.memVT = MVT::getVT(CI.getType()); // XXX: what is correct VT? Info.memVT = MVT::getVT(CI.getType()); // XXX: what is correct VT?
Info.fallbackAddressSpace = AMDGPUAS::BUFFER_FAT_POINTER; Info.fallbackAddressSpace = AMDGPUAS::BUFFER_RESOURCE;
Info.align.reset(); Info.align.reset();
Info.flags |= MachineMemOperand::MOLoad | Info.flags |= MachineMemOperand::MOLoad |
MachineMemOperand::MODereferenceable; MachineMemOperand::MODereferenceable;
return true; return true;
} }
case Intrinsic::amdgcn_global_atomic_fadd: case Intrinsic::amdgcn_global_atomic_fadd:
case Intrinsic::amdgcn_global_atomic_fmin: case Intrinsic::amdgcn_global_atomic_fmin:
case Intrinsic::amdgcn_global_atomic_fmax: case Intrinsic::amdgcn_global_atomic_fmax:
▲ Show 20 Lines • Show All 175 Lines • ▼ Show 20 Linesbool SITargetLowering::isLegalAddressingMode(const DataLayout &DL,
if (AM.BaseGV) if (AM.BaseGV)
return false; return false;
if (AS == AMDGPUAS::GLOBAL_ADDRESS) if (AS == AMDGPUAS::GLOBAL_ADDRESS)
return isLegalGlobalAddressingMode(AM); return isLegalGlobalAddressingMode(AM);
if (AS == AMDGPUAS::CONSTANT_ADDRESS || if (AS == AMDGPUAS::CONSTANT_ADDRESS ||
AS == AMDGPUAS::CONSTANT_ADDRESS_32BIT || AS == AMDGPUAS::CONSTANT_ADDRESS_32BIT ||
AS == AMDGPUAS::BUFFER_FAT_POINTER) { AS == AMDGPUAS::BUFFER_FAT_POINTER || AS == AMDGPUAS::BUFFER_RESOURCE) {
// If the offset isn't a multiple of 4, it probably isn't going to be // If the offset isn't a multiple of 4, it probably isn't going to be
// correctly aligned. // correctly aligned.
// FIXME: Can we get the real alignment here? // FIXME: Can we get the real alignment here?
if (AM.BaseOffs % 4 != 0) if (AM.BaseOffs % 4 != 0)
return isLegalMUBUFAddressingMode(AM); return isLegalMUBUFAddressingMode(AM);
// There are no SMRD extloads, so if we have to do a small type access we // There are no SMRD extloads, so if we have to do a small type access we
// will use a MUBUF load. // will use a MUBUF load.
▲ Show 20 Lines • Show All 12,163 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/GlobalISel/buffer-atomic-fadd.f32-no-rtn.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/buffer-atomic-fadd.f32-rtn.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/buffer-atomic-fadd.f64.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/buffer-atomic-fadd.v2f16-no-rtn.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/buffer-atomic-fadd.v2f16-rtn.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/irtranslator-non-integral-address-spaces.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.atomic.dim.a16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.dim.a16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.load.2d.d16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.load.2d.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.load.2darraymsaa.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.load.3d.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.sample.a16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.sample.d.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.sample.g16.a16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.sample.g16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/legalize-llvm.amdgcn.image.store.2d.d16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.atomic.dim.mir

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.atomic.add.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.atomic.cmpswap.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.atomic.fadd-with-ret.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.atomic.fadd.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.load.format.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.load.format.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.load.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.store.format.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.store.format.f32.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.buffer.store.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.load.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.load.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.store.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.store.i8.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.raw.tbuffer.store.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.atomic.add.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.atomic.cmpswap.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.atomic.fadd-with-ret.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.atomic.fadd.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.load.format.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.load.format.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.load.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.store.format.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.store.format.f32.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.buffer.store.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.tbuffer.load.f16.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.struct.tbuffer.load.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.image.load.1d.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.image.sample.1d.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.raw.buffer.load.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.struct.buffer.load.ll

llvm/test/CodeGen/AMDGPU/GlobalISel/regbankselect-amdgcn.struct.buffer.store.ll

llvm/test/CodeGen/AMDGPU/addrspacecast-captured.ll

llvm/test/CodeGen/AMDGPU/annotate-kernel-features-hsa.ll

llvm/test/CodeGen/AMDGPU/buffer-atomic-fadd.f32-no-rtn.ll

llvm/test/CodeGen/AMDGPU/buffer-atomic-fadd.f32-rtn.ll

llvm/test/CodeGen/AMDGPU/buffer-atomic-fadd.f64.ll

llvm/test/CodeGen/AMDGPU/buffer-atomic-fadd.v2f16-no-rtn.ll

llvm/test/CodeGen/AMDGPU/buffer-atomic-fadd.v2f16-rtn.ll

llvm/test/CodeGen/AMDGPU/buffer-intrinsics-mmo-offsets.ll

llvm/test/CodeGen/AMDGPU/cgp-addressing-modes.ll

llvm/test/CodeGen/AMDGPU/extract_subvector_vec4_vec3.ll

llvm/test/CodeGen/AMDGPU/force-alwaysinline-lds-global-address.ll

llvm/test/CodeGen/AMDGPU/legalize-amdgcn.raw.buffer.load.format.f16.ll

llvm/test/CodeGen/AMDGPU/legalize-amdgcn.raw.buffer.load.format.ll

llvm/test/CodeGen/AMDGPU/legalize-amdgcn.raw.buffer.load.ll

llvm/test/CodeGen/AMDGPU/legalize-amdgcn.raw.buffer.store.format.f16.ll

llvm/test/CodeGen/AMDGPU/legalize-amdgcn.raw.buffer.store.format.f32.ll

llvm/test/CodeGen/AMDGPU/legalize-amdgcn.raw.buffer.store.ll

llvm/test/CodeGen/AMDGPU/legalize-amdgcn.raw.tbuffer.load.f16.ll

llvm/test/CodeGen/AMDGPU/legalize-amdgcn.raw.tbuffer.load.ll

llvm/test/CodeGen/AMDGPU/legalize-amdgcn.raw.tbuffer.store.f16.ll

llvm/test/CodeGen/AMDGPU/legalize-amdgcn.raw.tbuffer.store.ll

llvm/test/CodeGen/AMDGPU/legalize-soffset-mbuf.ll

llvm/test/CodeGen/AMDGPU/loop-idiom.ll

llvm/test/CodeGen/AMDGPU/mdt-preserving-crash.ll

llvm/test/CodeGen/AMDGPU/noop-shader-O0.ll

llvm/test/CodeGen/AMDGPU/nullptr-long-address-spaces.ll

llvm/test/CodeGen/AMDGPU/nullptr.ll

llvm/test/CodeGen/AMDGPU/promote-alloca-lifetime.ll

llvm/test/CodeGen/AMDGPU/promote-alloca-to-lds-select.ll

llvm/test/CodeGen/AMDGPU/sgpr-copy-local-cse.ll

llvm/test/CodeGen/AMDGPU/splitkit-getsubrangeformask.ll

llvm/test/CodeGen/AMDGPU/unroll.ll

llvm/test/CodeGen/AMDGPU/unsupported-image-a16.ll

llvm/test/CodeGen/AMDGPU/unsupported-image-g16.ll

llvm/test/CodeGen/AMDGPU/vgpr-liverange-ir.ll

llvm/test/CodeGen/MIR/AMDGPU/custom-pseudo-source-values.ll

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/adaptive_constant_global_redzones.ll

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/adaptive_global_redzones.ll

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/asan_do_not_instrument_lds.ll

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/asan_do_not_instrument_scratch.ll

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/asan_instrument_constant_address_space.ll

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/asan_instrument_generic_address_space.ll

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/asan_instrument_global_address_space.ll

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/global_metadata_addrspacecasts.ll

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/no_redzones_in_lds_globals.ll

llvm/test/Instrumentation/AddressSanitizer/AMDGPU/no_redzones_in_scratch_globals.ll

llvm/test/Transforms/AlignmentFromAssumptions/amdgpu-crash.ll

llvm/test/Transforms/AtomicExpand/AMDGPU/expand-atomic-i16.ll

llvm/test/Transforms/EarlyCSE/AMDGPU/memrealtime.ll

llvm/test/Transforms/IndVarSimplify/AMDGPU/no-widen-to-i64.ll

llvm/test/Transforms/InferAddressSpaces/AMDGPU/noop-ptrint-pair.ll

llvm/test/Transforms/InferAddressSpaces/AMDGPU/ptrmask.ll

llvm/test/Transforms/InferAddressSpaces/X86/noop-ptrint-pair.ll

llvm/test/Transforms/Inline/AMDGPU/amdgpu-inline-alloca-argument.ll

llvm/test/Transforms/InstCombine/AMDGPU/memcpy-from-constant.ll

llvm/test/Transforms/InstCombine/alloca-in-non-alloca-as.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/aa-metadata.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/adjust-alloca-alignment.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/complex-index.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/extended-index.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/gep-bitcast.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/insertion-point.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/interleaved-mayalias-store.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/invariant-load.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/merge-stores-private.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/merge-stores.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/missing-alignment.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/multiple_tails.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/no-implicit-float.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/optnone.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/pointer-elements.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/selects-inseltpoison.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/selects.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/store_with_aliasing_load.ll

llvm/test/Transforms/LoadStoreVectorizer/AMDGPU/weird-type-accesses.ll

llvm/test/Transforms/LoopLoadElim/pr46854-adress-spaces.ll

llvm/test/Transforms/LoopStrengthReduce/AMDGPU/atomics.ll

llvm/test/Transforms/LoopStrengthReduce/AMDGPU/different-addrspace-addressing-mode-loops.ll

llvm/test/Transforms/LoopStrengthReduce/AMDGPU/lsr-invalid-ptr-extend.ll

llvm/test/Transforms/LoopStrengthReduce/AMDGPU/lsr-postinc-pos-addrspace.ll

llvm/test/Transforms/LoopStrengthReduce/AMDGPU/preserve-addrspace-assert.ll

llvm/test/Transforms/OpenMP/attributor_pointer_offset_crash.ll

llvm/test/Transforms/OpenMP/spmdization_constant_prop.ll

llvm/test/Transforms/OpenMP/values_in_offload_arrays.alloca.ll

llvm/test/Transforms/SLPVectorizer/AMDGPU/address-space-ptr-sze-gep-index-assert.ll

llvm/test/Transforms/VectorCombine/AMDGPU/as-transition-inseltpoison.ll

llvm/test/Transforms/VectorCombine/AMDGPU/as-transition.ll

llvm/unittests/Bitcode/DataLayoutUpgradeTest.cpp