[SPIRV 5/6] Add LegalizerInfo, InstructionSelector and utilities

[iliya-diyachkov]

Summary

The patch adds SPIRVLegalizerInfo, SPIRVInstructionSelector and SPIRV-specific utilities.

[arsenm]

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp

84–85

This should probably be an assert, not report_fatal_error

215–216

should be assert?

277

No const reference

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h

10–13

Why do you need to do this? What are you losing in gMIR that matters?

llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp

205–208

You shouldn't have to construct a fresh new MIRBuilder for every selected instruction

519

I agree there are too many autos

593–610

I think you are just repeating the generic expansion for this. It would be better to just use the default lower action for G_ATOMIC_CMPXCHG_WITH_SUCCESS

596

LLT is shorter and clearer than auto

661

return after else

836–844

It looks like you're repeating work done in the legalizer?

1102

No else after return

1139

Should just use cast<> and remove the check, this should never fail. I believe we disallow null G_GLOBAL_VALUE sources

llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp

178

I thought SPIRV had a fixed set of register types?

203–204

This is illegal MIR, there's no point in checking it

205

Else after return

206

Typo ponter

218–219

You don't need to worry about the cases that are illegal MIR

220

Else after return

234

This has been not true for about 2 years

287

I'm not sure what's meant by an "Id Reg"

302

Are you intentionally discarding the pointer size and address space?

311

Generally code in the legalizer shouldn't bother setting regbanks or classes

333

You're doing a lot of work to convert to the SPIRV types here, which would be more appropriate to do during selection

349–356

it might be cleaner to push this cast to integer into the generic LegalizerHelper

362–366

You're assuming there's only a single use, and your legalize action should only touch the passed instruction

llvm/lib/Target/SPIRV/SPIRVUtils.cpp

133–134

How/why are you not using generic as addrspace 0?

161–170

I dislike this wrapper function. You should have all of these variables available in the context where you are constraining already, and the MF parameter with null check is potentially confusing

[iliya-diyachkov]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.h

777

Ok, "assert((Op.getImm() >> 32) == 0..." will be added.

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h

10–13

SPIR-V annotates each instruction with type IDs. SPIR-V uses high-level types including composites (structs, arrays...) and some others. Most of them can be obtained from LLVM Types but they are not available in gMIR, so we need to save and maintain own type information.

llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp

519

The autos will be cleaned.

836–844

Yes, it looks like I need to fix this.

1139

Ok, I'll check this.

llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp

30

It will be replaced by std::set.

llvm/lib/Target/SPIRV/SPIRVUtils.cpp

133–134

One of the important features of this backend is compatibility with SPIRV-LLVM-Translator. We follow the address spaces agreement announced in https://github.com/KhronosGroup/SPIRV-LLVM-Translator/blob/master/docs/SPIRVRepresentationInLLVM.rst#address-spaces .

[zuban32]

llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp

178

In SPIRV [almost] every value is an ID which is just an integer (usually 32- or 64-bit ), the type of the value with an ID is detemined by the instruction producing it which has 'ResultType' parameter. But we're trying not to enforce that rule on gMIR level so that before the selection all of the types were the ones IRTranslator emitted without anyhting target-specific interfering.

302

Well, yes and no. In fact LLTs do not matter after we generated all the necessary type-assigning pseudo instructions. But LLTs still matter in sence of gMIR being well-formed (passing verification) and not having any significant differences from the original IR (i.e. scalars instead of vectors, vectors of different type, etc) The addrspace of LLT doesn't matter for sure

333

We placed this code here because it's doing some sort of legalization (in old SPIR-V versions pointers can't be compared directly, ptrtoints are required), not sure if it's good to mix things up in the selector.

362–366

The single-use constraint is enforced by the design of our target, every non-void instruction have the only user - type assignment instruction, which in turn may have multiple users just as the original instruction before.

[arsenm]

llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp

178

That's the whole point of the legalizer. There's some notion of fixed supported types implied by the other legalizer rules

362–366

This is not a property the target can decide, the incoming IR can have multiple uses. It's simply wrong to rely on it like this

[zuban32]

llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp

362–366

Why not if we have a target-specific LLVM IR pass making that assumption always true?

[iliya-diyachkov]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

29

Ok, it will be applied to other expanded macros that also return literals.

llvm/lib/Target/SPIRV/SPIRVCallLowering.cpp

132–142

I'm not sure that it is checked by verifier.
I'm sorry, this code fragment is not needed right now (since the series of patches does not include the pass that generates this metadata). I'll exclude it from this patch. We'll verify and correct the fragment by the next time.

172

Thanks, it will be fixed.

llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp

205–208

As I understand you suggest constructing MachineIRBuilder closer to its users, because not all instructions (for selection) are actually use it.

593–610

Yes, I'll fix it.

llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp

178

@arsenm yes, all register types are legal for phis. This rule will be substituted by legalFor(allPtrsScalarsAndVectors).

287

I suppose it means a register which is used in GET_ID pseudo instruction.

302

It's used in creation of new temporary pseudos which are involved in the instruction selection and then are removed. Perhaps it's a bad practice and should be avoided.

311

Probably it would be better to move this to selector.

333

@arsenm Yes, we'll try to move all this work to the selection stage.
@zuban32 I have verified that we don't need this code here (see Khronos repo).

349–356

Ok, I have created a function that generates a single cast.

362–366

Instructions processed here should always have a single use which is a pseudo instruction ASSIGN_TYPE, generated by SPIRV specific pass before the legalizer. We'll add a check that this condition is met. Also it looks like the modification of another instructions may be avoided.
And this fragment works together with another pass that is not presented in this series, so I would postpone it for now.

llvm/lib/Target/SPIRV/SPIRVUtils.cpp

161–170

Ok, we'll get rid of it.

[zuban32]

llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp

362–366

Oh, I see. It's just the pass responsible for making the input IR compliant with what SPIR-V specific GlobalISel stages expect is currently missing from these patches. Maybe we need to extend the series

[iliya-diyachkov]

llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp

362–366

As Renato suggested in the comment (see the first patch), we need a minimum set of patches that does something substantial. Then we'll add passes/features by stand alone patches extending the SPIR-V backend functionality and making more tests passed. I'm not sure that right now we have a reason to increase the series noticeably.

[zuban32]

llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp

362–366

Yep, I understand that. The question is whether initial series should be kind of transitive closure or there might be some logical inconsistency due to some parts of the code not being upstreamed yet.

[iliya-diyachkov]

llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp

362–366

Of course the series should be consistent but I think it doesn't make sense for us to strive for transitive closure at this stage, because it is only the first step towards adding (to LLVM) a full-featured SPIR-V backend, which will be useful for end users. Approaching this goal in accordance with the logic of the LLVM project is the meaning of the current work.

Probably it's worth adding more functionality to SPIRVGlobalTypesAndRegNumPass.cpp so more tests from our test base can initially pass. But as Matt commented we should make changes in the way SPIRVGlobalTypesAndRegNumPass works, I'm going to try it.

[iliya-diyachkov]

The issues found by Renato Golin, Fangrui Song and Matt Arsenault were fixed. The macros in BaseInfo were expanded.

[rengolin]

Too many comments for me to follow and check, I'll let @MaskRay and @arsenm approve this one.

[iliya-diyachkov]

Rebase.

[iliya-diyachkov]

Fix clang-format issue.

[iliya-diyachkov]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp:532:3: warning: default label in switch which covers all enumeration values [-Wcovered-switch-default]

The warning was fixed.
Also addNumImm function was corrected.

[iliya-diyachkov]

Update after the last changes in the 3th, 4th patches.

[arsenm]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

1407

No else after return

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.h

23

SPIRV namespace?

llvm/lib/Target/SPIRV/SPIRVCallLowering.cpp

172

the static_cast to function is also no good since this could be a constexpr cast

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp

151

Use cast here and remove the assert below?

160

no const_casts?

[iliya-diyachkov]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.h

23

Probably you mean

namespace llvm {
namespace SPIRV {
namespace Capability {

[iliya-diyachkov]

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp

160

Module is fixed, but I'm not sure how to cast const Type *Ty to Type * without const_cast.

I see that other targets uses this approach in the similar cases (ARM, Hexagon).

[iliya-diyachkov]

I think the issues are fixed.

[iliya-diyachkov]

The format issue is fixed.

[mpaszkowski]

Gentle ping! We would like to ask for a final "LGTM".

[MaskRay]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.h

26

namespace XXX { enum XXX { ... } } can usually be replaced with enum class XXX {...}`

[iliya-diyachkov]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.h

26

Thanks, @MaskRay. The code is corrected.

[arsenm]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

283

I think these all these switch functions need to move the unknown case to default and use llvm_unreachable after the switch to avoid warnings on all compilers

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.h

521

Don't need llvm::

llvm/lib/Target/SPIRV/SPIRVCallLowering.cpp

37

getSubtarget<SPIRVSubtarget>.I think it's worthwhile to just have this as a permanent member of SPIRVCallLowering

164

This should still return false since multiple registers still can come from valid IR

172

This will still fail on constexpr casts and indirect calls. dyn_cast_or_null and return if you don't want to handle now

176

You should not construct one off MachineIRBuilders. Re-use the one passed in and adjust the insert point. There's additional state (like CSEInfo) you're losing

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp

339

Why all the null checks on Type? I would expect all of these to require non-null inputs

llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp

205–208

No, I mean MachineIRBuilder is a heavy class that should have state set up and maintained. It's not something to construct for a single use. The other selectors mostly just use BuildMI directly during selection.

[iliya-diyachkov]

llvm/lib/Target/SPIRV/SPIRVCallLowering.cpp

37

Well, it looks reasonable.

172

Thanks, it'll be fixed.

176

Sure, this is bad code.

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp

339

Thanks, will fix.

llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp

205–208

Thanks, now I see that only AMDGPU and Mips targets sometimes use MachineIRBuilder. I'll change it in SPIRV.

[iliya-diyachkov]

The issues found by Matt have been resolved.

[iliya-diyachkov]

llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp

205–208

I have left only one (relatively rare) case of using MachineIRBuilder in SPIRVInstructionSelector. I suppose this is acceptable since AMDGPU and Mips do it in a similar way.

[iliya-diyachkov]

Extra headers in SPIRVUtils.h were removed. The changes in SPIRVUtils from the 6th patch were moved to the 5th patch.

@arsenm, @MaskRay, I think all the issues have been fixed. Could you please take the last look at the patches?

[iliya-diyachkov]

Cosmetic changes in SPIRVGlobalRegistry.cpp and SPIRVUtils.h.

[arsenm]

LGTM with nits

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

284

the llvm_unreachable is a bit pointless without a break here

llvm/lib/Target/SPIRV/SPIRVCallLowering.cpp

90–92

getParamAlignment would be nicer

llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp

205–208

I've been meaning to fix those; I wouldn't use that as a good example.

811

Braces

996

Braces. Line wrapping is a bit weird looking

1041

Not much point asserting on things the verifier checks

[iliya-diyachkov]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

284

I see, will change to break.

llvm/lib/Target/SPIRV/SPIRVCallLowering.cpp

90–92

It'll be corrected.

llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp

205–208

Ok, if you don't mind I'll add FIXME for this last case for now and fix it as soon as possible. This fix may require an additional extension in SPIRVGlobalRegistry, in which case I would commit it after this initial series of patches.

1041

Will remove.

[iliya-diyachkov]

The last issues have been fixed.

[iliya-diyachkov]

Fix a warning in SPIRVCallLowering.cpp.

[MaskRay]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

381

Can we use X macros for the long list?

487

Can we use X macros for the long list?

557

Can we use X macros for the long list?

587

Can we use X macros for the long list?

llvm/lib/Target/SPIRV/SPIRVSubtarget.cpp

48–50

make_unique

[iliya-diyachkov]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

381

Initially all these functions were generated using macros. Then Renato suggested getting rid of them and we agreed to use expanded macros for now. Later we'll re-implement all the functions with tablegen (this work is in progress so I expect it to be completed in several months).

I have already fixed a number of issues in these functions after expanding the macros. I'm not sure that now we really need to replace them with new macros.

[MaskRay]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

381

Ah, ok. @rengolin I feel that X macros may be more readable and less error-prone than open coding here.

TableGen can generate such strings as well, but it will add one step, bring a little abstraction code, and is probably not conciser than macros here (AIUI one line for one enumeration value). WDYT?

[MaskRay]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

381

The recent clang-format discussion is also relevant here. I think the current open coding can be perceived as error-prone by some folks:
https://discourse.llvm.org/t/enable-single-line-case-statements-in-llvm-clang-format-style/61062

[iliya-diyachkov]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

381

TableGen can generate such strings as well, but it will add one step, bring a little abstraction code, and is probably not conciser than macros here

As I know we'll have all these functions fully generated by TableGen, so it will be quite concise.

In the first version we had quite complex macros for all BaseInfo entities, and even more complex macros in the original Khronos repository, so we shouldn't return to that implementation.

I assume that you are now suggesting using very simple macros like this:

#define CASE(CLASS, MEMBER) \
  case CLASS::MEMBER: return #MEMBER;
...
StringRef SPIRV::getExecutionModeName(ExecutionMode e) {
  switch (e) {
  CASE(ExecutionMode, Invocations)

I'm not sure whether we need this at the moment, so I set TODO before all the functions for now.

However if you think it's a better solution then we have now, no problem, I'll change all the functions (with this open coding). But we all need to agree on this.

[iliya-diyachkov]

Fix the issues.

[rengolin]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

20

why not use:

namespace llvm {
   namespace SPIRV {

and then avoid prefixing all types with SPIRV?

381

I honestly can't remember what was there before, but this huge list of strings does seem a bit out of hand.

With macros, we'd still have a huge list of calls, but we wouldn't have the opportunity to make typo mistakes (if using X(str) case str: return "#str";).

I'm also not sure why you need all of these strings at this level, unless you want to print all that in your assembly output. Is that the point? Or is it just debug?

[iliya-diyachkov]

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

20

You are right, we can avoid SPIRV prefix.

381

With macros, we'd still have a huge list of calls, but we wouldn't have the opportunity to make typo mistakes (if using X(str) case str: return "#str";).

Ok, I'll change this to X macros.

I'm also not sure why you need all of these strings at this level, unless you want to print all that in your assembly output. Is that the point?

Yes, we need to print all these operands in the assembly output. Anyway the TableGen version is in progress, so this huge list will be removed.

[iliya-diyachkov]

Remove SPIRV prefixing, use "X" macros to avoid typo mistakes in SPIRVBaseInfo.cpp.

[rengolin]

Thanks @iliya-diyachkov, it looks cleaner, even if still very long. A table generated output will hopefully fix this soon.

First, sorry for the confusion if I made you do something only to revert later, but that shows the value of multi-person reviews. So, also thanks others for picking up on that.

I have no further comments, this looks good to me now. Thanks!

[iliya-diyachkov]

Thanks, Renato!

First, sorry for the confusion if I made you do something only to revert later, but that shows the value of multi-person reviews. So, also thanks others for picking up on that.

No problem. The initial version of macros was more complicated, so it was not just a return, but an adaptation of the original code.

Now we have to finish the review of the 6th patch. All the issues were fixed about a week ago, I hope Matt will respond soon.

[nikic]

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp

287

This is incompatible with opaque pointers (https://llvm.org/docs/OpaquePointers.html). Please remove this call or revert the patch if this is non-trivial.

[zuban32]

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp

287

What if we use PointerType::getNonOpaquePointerElementType() instead? IMO that should do the trick while SPIR-V has no support of opaque pointers yet so we're only oging non-opaque way here.

[nikic]

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp

287

You can use getNonOpaquePointerElementType(), but you need to implement a separate code-path for isOpaquePointerTy() if you do so.

[iliya-diyachkov]

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp

287

Sure, we'll implement. Thanks, Nikita.

llvm/lib/Target/SPIRV/CMakeLists.txt

 tablegen(LLVM SPIRVGenRegisterInfo.inc -gen-register-info)
 tablegen(LLVM SPIRVGenSubtargetInfo.inc -gen-subtarget)
 
 add_public_tablegen_target(SPIRVCommonTableGen)
 
 add_llvm_target(SPIRVCodeGen
   SPIRVAsmPrinter.cpp
   SPIRVCallLowering.cpp
+  SPIRVGlobalRegistry.cpp
   SPIRVInstrInfo.cpp
+  SPIRVInstructionSelector.cpp
   SPIRVISelLowering.cpp
+  SPIRVLegalizerInfo.cpp
   SPIRVMCInstLower.cpp
   SPIRVRegisterBankInfo.cpp
   SPIRVRegisterInfo.cpp
   SPIRVSubtarget.cpp
   SPIRVTargetMachine.cpp
+  SPIRVUtils.cpp
 
   LINK_COMPONENTS
   Analysis
   AsmPrinter
   CodeGen
   Core
   GlobalISel
   MC

llvm/lib/Target/SPIRV/MCTargetDesc/CMakeLists.txt

 add_llvm_component_library(LLVMSPIRVDesc
+  SPIRVBaseInfo.cpp
   SPIRVMCAsmInfo.cpp
   SPIRVMCTargetDesc.cpp
   SPIRVTargetStreamer.cpp
   SPIRVAsmBackend.cpp
   SPIRVMCCodeEmitter.cpp
   SPIRVObjectTargetWriter.cpp
   SPIRVInstPrinter.cpp
 
   LINK_COMPONENTS
   MC
   SPIRVInfo
   Support
 
   ADD_TO_COMPONENT
   SPIRV
   )

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.h

+//===-- SPIRVBaseInfo.h -  Top level definitions for SPIRV ------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains small standalone helper functions and enum definitions for
+// the SPIRV target useful for the compiler back-end and the MC libraries.
+// As such, it deliberately does not include references to LLVM core
+// code gen types, passes, etc..
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_TARGET_SPIRV_MCTARGETDESC_SPIRVBASEINFO_H
+#define LLVM_LIB_TARGET_SPIRV_MCTARGETDESC_SPIRVBASEINFO_H
+
+#include "llvm/ADT/StringRef.h"
+#include <string>
+
+namespace llvm {
+namespace SPIRV {
+namespace Capability {
+enum Capability : uint32_t {
+  Matrix = 0,
+  Shader = 1,
+  Geometry = 2,
+  Tessellation = 3,
+  Addresses = 4,
+  Linkage = 5,
+  Kernel = 6,
+  Vector16 = 7,
+  Float16Buffer = 8,
+  Float16 = 9,
+  Float64 = 10,
+  Int64 = 11,
+  Int64Atomics = 12,
+  ImageBasic = 13,
+  ImageReadWrite = 14,
+  ImageMipmap = 15,
+  Pipes = 17,
+  Groups = 18,
+  DeviceEnqueue = 19,
+  LiteralSampler = 20,
+  AtomicStorage = 21,
+  Int16 = 22,
+  TessellationPointSize = 23,
+  GeometryPointSize = 24,
+  ImageGatherExtended = 25,
+  StorageImageMultisample = 27,
+  UniformBufferArrayDynamicIndexing = 28,
+  SampledImageArrayDymnamicIndexing = 29,
+  ClipDistance = 32,
+  CullDistance = 33,
+  ImageCubeArray = 34,
+  SampleRateShading = 35,
+  ImageRect = 36,
+  SampledRect = 37,
+  GenericPointer = 38,
+  Int8 = 39,
+  InputAttachment = 40,
+  SparseResidency = 41,
+  MinLod = 42,
+  Sampled1D = 43,
+  Image1D = 44,
+  SampledCubeArray = 45,
+  SampledBuffer = 46,
+  ImageBuffer = 47,
+  ImageMSArray = 48,
+  StorageImageExtendedFormats = 49,
+  ImageQuery = 50,
+  DerivativeControl = 51,
+  InterpolationFunction = 52,
+  TransformFeedback = 53,
+  GeometryStreams = 54,
+  StorageImageReadWithoutFormat = 55,
+  StorageImageWriteWithoutFormat = 56,
+  MultiViewport = 57,
+  SubgroupDispatch = 58,
+  NamedBarrier = 59,
+  PipeStorage = 60,
+  GroupNonUniform = 61,
+  GroupNonUniformVote = 62,
+  GroupNonUniformArithmetic = 63,
+  GroupNonUniformBallot = 64,
+  GroupNonUniformShuffle = 65,
+  GroupNonUniformShuffleRelative = 66,
+  GroupNonUniformClustered = 67,
+  GroupNonUniformQuad = 68,
+  SubgroupBallotKHR = 4423,
+  DrawParameters = 4427,
+  SubgroupVoteKHR = 4431,
+  StorageBuffer16BitAccess = 4433,
+  StorageUniform16 = 4434,
+  StoragePushConstant16 = 4435,
+  StorageInputOutput16 = 4436,
+  DeviceGroup = 4437,
+  MultiView = 4439,
+  VariablePointersStorageBuffer = 4441,
+  VariablePointers = 4442,
+  AtomicStorageOps = 4445,
+  SampleMaskPostDepthCoverage = 4447,
+  StorageBuffer8BitAccess = 4448,
+  UniformAndStorageBuffer8BitAccess = 4449,
+  StoragePushConstant8 = 4450,
+  DenormPreserve = 4464,
+  DenormFlushToZero = 4465,
+  SignedZeroInfNanPreserve = 4466,
+  RoundingModeRTE = 4467,
+  RoundingModeRTZ = 4468,
+  Float16ImageAMD = 5008,
+  ImageGatherBiasLodAMD = 5009,
+  FragmentMaskAMD = 5010,
+  StencilExportEXT = 5013,
+  ImageReadWriteLodAMD = 5015,
+  SampleMaskOverrideCoverageNV = 5249,
+  GeometryShaderPassthroughNV = 5251,
+  ShaderViewportIndexLayerEXT = 5254,
+  ShaderViewportMaskNV = 5255,
+  ShaderStereoViewNV = 5259,
+  PerViewAttributesNV = 5260,
+  FragmentFullyCoveredEXT = 5265,
+  MeshShadingNV = 5266,
+  ShaderNonUniformEXT = 5301,
+  RuntimeDescriptorArrayEXT = 5302,
+  InputAttachmentArrayDynamicIndexingEXT = 5303,
+  UniformTexelBufferArrayDynamicIndexingEXT = 5304,
+  StorageTexelBufferArrayDynamicIndexingEXT = 5305,
+  UniformBufferArrayNonUniformIndexingEXT = 5306,
+  SampledImageArrayNonUniformIndexingEXT = 5307,
+  StorageBufferArrayNonUniformIndexingEXT = 5308,
+  StorageImageArrayNonUniformIndexingEXT = 5309,
+  InputAttachmentArrayNonUniformIndexingEXT = 5310,
+  UniformTexelBufferArrayNonUniformIndexingEXT = 5311,
+  StorageTexelBufferArrayNonUniformIndexingEXT = 5312,
+  RayTracingNV = 5340,
+  SubgroupShuffleINTEL = 5568,
+  SubgroupBufferBlockIOINTEL = 5569,
+  SubgroupImageBlockIOINTEL = 5570,
+  SubgroupImageMediaBlockIOINTEL = 5579,
+  SubgroupAvcMotionEstimationINTEL = 5696,
+  SubgroupAvcMotionEstimationIntraINTEL = 5697,
+  SubgroupAvcMotionEstimationChromaINTEL = 5698,
+  GroupNonUniformPartitionedNV = 5297,
+  VulkanMemoryModelKHR = 5345,
+  VulkanMemoryModelDeviceScopeKHR = 5346,
+  ImageFootprintNV = 5282,
+  FragmentBarycentricNV = 5284,
+  ComputeDerivativeGroupQuadsNV = 5288,
+  ComputeDerivativeGroupLinearNV = 5350,
+  FragmentDensityEXT = 5291,
+  PhysicalStorageBufferAddressesEXT = 5347,
+  CooperativeMatrixNV = 5357,
+};
+}
+llvm::StringRef getCapabilityName(Capability::Capability e);
+
+namespace SourceLanguage {
+enum SourceLanguage : uint32_t {
+  Unknown = 0,
+  ESSL = 1,
+  GLSL = 2,
+  OpenCL_C = 3,
+  OpenCL_CPP = 4,
+  HLSL = 5,
+};
+}
+llvm::StringRef getSourceLanguageName(SourceLanguage::SourceLanguage e);
+
+namespace AddressingModel {
+enum AddressingModel : uint32_t {
+  Logical = 0,
+  Physical32 = 1,
+  Physical64 = 2,
+  PhysicalStorageBuffer64EXT = 5348,
+};
+}
+llvm::StringRef getAddressingModelName(AddressingModel::AddressingModel e);
+
+namespace ExecutionModel {
+enum ExecutionModel : uint32_t {
+  Vertex = 0,
+  TessellationControl = 1,
+  TessellationEvaluation = 2,
+  Geometry = 3,
+  Fragment = 4,
+  GLCompute = 5,
+  Kernel = 6,
+  TaskNV = 5267,
+  MeshNV = 5268,
+  RayGenerationNV = 5313,
+  IntersectionNV = 5314,
+  AnyHitNV = 5315,
+  ClosestHitNV = 5316,
+  MissNV = 5317,
+  CallableNV = 5318,
+};
+}
+llvm::StringRef getExecutionModelName(ExecutionModel::ExecutionModel e);
+
+namespace MemoryModel {
+enum MemoryModel : uint32_t {
+  Simple = 0,
+  GLSL450 = 1,
+  OpenCL = 2,
+  VulkanKHR = 3,
+};
+}
+llvm::StringRef getMemoryModelName(MemoryModel::MemoryModel e);
+
+namespace ExecutionMode {
+enum ExecutionMode : uint32_t {
+  Invocations = 0,
+  SpacingEqual = 1,
+  SpacingFractionalEven = 2,
+  SpacingFractionalOdd = 3,
+  VertexOrderCw = 4,
+  VertexOrderCcw = 5,
+  PixelCenterInteger = 6,
+  OriginUpperLeft = 7,
+  OriginLowerLeft = 8,
+  EarlyFragmentTests = 9,
+  PointMode = 10,
+  Xfb = 11,
+  DepthReplacing = 12,
+  DepthGreater = 14,
+  DepthLess = 15,
+  DepthUnchanged = 16,
+  LocalSize = 17,
+  LocalSizeHint = 18,
+  InputPoints = 19,
+  InputLines = 20,
+  InputLinesAdjacency = 21,
+  Triangles = 22,
+  InputTrianglesAdjacency = 23,
+  Quads = 24,
+  Isolines = 25,
+  OutputVertices = 26,
+  OutputPoints = 27,
+  OutputLineStrip = 28,
+  OutputTriangleStrip = 29,
+  VecTypeHint = 30,
+  ContractionOff = 31,
+  Initializer = 33,
+  Finalizer = 34,
+  SubgroupSize = 35,
+  SubgroupsPerWorkgroup = 36,
+  SubgroupsPerWorkgroupId = 37,
+  LocalSizeId = 38,
+  LocalSizeHintId = 39,
+  PostDepthCoverage = 4446,
+  DenormPreserve = 4459,
+  DenormFlushToZero = 4460,
+  SignedZeroInfNanPreserve = 4461,
+  RoundingModeRTE = 4462,
+  RoundingModeRTZ = 4463,
+  StencilRefReplacingEXT = 5027,
+  OutputLinesNV = 5269,
+  DerivativeGroupQuadsNV = 5289,
+  DerivativeGroupLinearNV = 5290,
+  OutputTrianglesNV = 5298,
+};
+}
+llvm::StringRef getExecutionModeName(ExecutionMode::ExecutionMode e);
+
+namespace StorageClass {
+enum StorageClass : uint32_t {
+  UniformConstant = 0,
+  Input = 1,
+  Uniform = 2,
+  Output = 3,
+  Workgroup = 4,
+  CrossWorkgroup = 5,
+  Private = 6,
+  Function = 7,
+  Generic = 8,
+  PushConstant = 9,
+  AtomicCounter = 10,
+  Image = 11,
+  StorageBuffer = 12,
+  CallableDataNV = 5328,
+  IncomingCallableDataNV = 5329,
+  RayPayloadNV = 5338,
+  HitAttributeNV = 5339,
+  IncomingRayPayloadNV = 5342,
+  ShaderRecordBufferNV = 5343,
+  PhysicalStorageBufferEXT = 5349,
+};
+}
+llvm::StringRef getStorageClassName(StorageClass::StorageClass e);
+
+namespace Dim {
+enum Dim : uint32_t {
+  DIM_1D = 0,
+  DIM_2D = 1,
+  DIM_3D = 2,
+  DIM_Cube = 3,
+  DIM_Rect = 4,
+  DIM_Buffer = 5,
+  DIM_SubpassData = 6,
+};
+}
+llvm::StringRef getDimName(Dim::Dim e);
+
+namespace SamplerAddressingMode {
+enum SamplerAddressingMode : uint32_t {
+  None = 0,
+  ClampToEdge = 1,
+  Clamp = 2,
+  Repeat = 3,
+  RepeatMirrored = 4,
+};
+}
+llvm::StringRef
+getSamplerAddressingModeName(SamplerAddressingMode::SamplerAddressingMode e);
+
+namespace SamplerFilterMode {
+enum SamplerFilterMode : uint32_t {
+  Nearest = 0,
+  Linear = 1,
+};
+}
+llvm::StringRef
+getSamplerFilterModeName(SamplerFilterMode::SamplerFilterMode e);
+
+namespace ImageFormat {
+enum ImageFormat : uint32_t {
+  Unknown = 0,
+  Rgba32f = 1,
+  Rgba16f = 2,
+  R32f = 3,
+  Rgba8 = 4,
+  Rgba8Snorm = 5,
+  Rg32f = 6,
+  Rg16f = 7,
+  R11fG11fB10f = 8,
+  R16f = 9,
+  Rgba16 = 10,
+  Rgb10A2 = 11,
+  Rg16 = 12,
+  Rg8 = 13,
+  R16 = 14,
+  R8 = 15,
+  Rgba16Snorm = 16,
+  Rg16Snorm = 17,
+  Rg8Snorm = 18,
+  R16Snorm = 19,
+  R8Snorm = 20,
+  Rgba32i = 21,
+  Rgba16i = 22,
+  Rgba8i = 23,
+  R32i = 24,
+  Rg32i = 25,
+  Rg16i = 26,
+  Rg8i = 27,
+  R16i = 28,
+  R8i = 29,
+  Rgba32ui = 30,
+  Rgba16ui = 31,
+  Rgba8ui = 32,
+  R32ui = 33,
+  Rgb10a2ui = 34,
+  Rg32ui = 35,
+  Rg16ui = 36,
+  Rg8ui = 37,
+  R16ui = 38,
+  R8ui = 39,
+};
+}
+llvm::StringRef getImageFormatName(ImageFormat::ImageFormat e);
+
+namespace ImageChannelOrder {
+enum ImageChannelOrder : uint32_t {
+  R = 0,
+  A = 1,
+  RG = 2,
+  RA = 3,
+  RGB = 4,
+  RGBA = 5,
+  BGRA = 6,
+  ARGB = 7,
+  Intensity = 8,
+  Luminance = 9,
+  Rx = 10,
+  RGx = 11,
+  RGBx = 12,
+  Depth = 13,
+  DepthStencil = 14,
+  sRGB = 15,
+  sRGBx = 16,
+  sRGBA = 17,
+  sBGRA = 18,
+  ABGR = 19,
+};
+}
+llvm::StringRef
+getImageChannelOrderName(ImageChannelOrder::ImageChannelOrder e);
+
+namespace ImageChannelDataType {
+enum ImageChannelDataType : uint32_t {
+  SnormInt8 = 0,
+  SnormInt16 = 1,
+  UnormInt8 = 2,
+  UnormInt16 = 3,
+  UnormShort565 = 4,
+  UnormShort555 = 5,
+  UnormInt101010 = 6,
+  SignedInt8 = 7,
+  SignedInt16 = 8,
+  SignedInt32 = 9,
+  UnsignedInt8 = 10,
+  UnsignedInt16 = 11,
+  UnsigendInt32 = 12,
+  HalfFloat = 13,
+  Float = 14,
+  UnormInt24 = 15,
+  UnormInt101010_2 = 16,
+};
+}
+llvm::StringRef
+getImageChannelDataTypeName(ImageChannelDataType::ImageChannelDataType e);
+
+namespace ImageOperand {
+enum ImageOperand : uint32_t {
+  None = 0x0,
+  Bias = 0x1,
+  Lod = 0x2,
+  Grad = 0x4,
+  ConstOffset = 0x8,
+  Offset = 0x10,
+  ConstOffsets = 0x20,
+  Sample = 0x40,
+  MinLod = 0x80,
+  MakeTexelAvailableKHR = 0x100,
+  MakeTexelVisibleKHR = 0x200,
+  NonPrivateTexelKHR = 0x400,
+  VolatileTexelKHR = 0x800,
+  SignExtend = 0x1000,
+  ZeroExtend = 0x2000,
+};
+}
+std::string getImageOperandName(ImageOperand::ImageOperand e);
+
+namespace FPFastMathMode {
+enum FPFastMathMode : uint32_t {
+  None = 0x0,
+  NotNaN = 0x1,
+  NotInf = 0x2,
+  NSZ = 0x4,
+  AllowRecip = 0x8,
+  Fast = 0x10,
+};
+}
+std::string getFPFastMathModeName(FPFastMathMode::FPFastMathMode e);
+
+namespace FPRoundingMode {
+enum FPRoundingMode : uint32_t {
+  RTE = 0,
+  RTZ = 1,
+  RTP = 2,
+  RTN = 3,
+};
+}
+llvm::StringRef getFPRoundingModeName(FPRoundingMode::FPRoundingMode e);
+
+namespace LinkageType {
+enum LinkageType : uint32_t {
+  Export = 0,
+  Import = 1,
+};
+}
+llvm::StringRef getLinkageTypeName(LinkageType::LinkageType e);
+
+namespace AccessQualifier {
+enum AccessQualifier : uint32_t {
+  ReadOnly = 0,
+  WriteOnly = 1,
+  ReadWrite = 2,
+};
+}
+llvm::StringRef getAccessQualifierName(AccessQualifier::AccessQualifier e);
+
+namespace FunctionParameterAttribute {
+enum FunctionParameterAttribute : uint32_t {
+  Zext = 0,
+  Sext = 1,
+  ByVal = 2,
+  Sret = 3,
+  NoAlias = 4,
+  NoCapture = 5,
+  NoWrite = 6,
+  NoReadWrite = 7,
+};
+}
+llvm::StringRef getFunctionParameterAttributeName(
+    FunctionParameterAttribute::FunctionParameterAttribute e);
+
+namespace Decoration {
+enum Decoration : uint32_t {
+  RelaxedPrecision = 0,
+  SpecId = 1,
+  Block = 2,
+  BufferBlock = 3,
+  RowMajor = 4,
+  ColMajor = 5,
+  ArrayStride = 6,
+  MatrixStride = 7,
+  GLSLShared = 8,
+  GLSLPacked = 9,
+  CPacked = 10,
+  BuiltIn = 11,
+  NoPerspective = 13,
+  Flat = 14,
+  Patch = 15,
+  Centroid = 16,
+  Sample = 17,
+  Invariant = 18,
+  Restrict = 19,
+  Aliased = 20,
+  Volatile = 21,
+  Constant = 22,
+  Coherent = 23,
+  NonWritable = 24,
+  NonReadable = 25,
+  Uniform = 26,
+  UniformId = 27,
+  SaturatedConversion = 28,
+  Stream = 29,
+  Location = 30,
+  Component = 31,
+  Index = 32,
+  Binding = 33,
+  DescriptorSet = 34,
+  Offset = 35,
+  XfbBuffer = 36,
+  XfbStride = 37,
+  FuncParamAttr = 38,
+  FPRoundingMode = 39,
+  FPFastMathMode = 40,
+  LinkageAttributes = 41,
+  NoContraction = 42,
+  InputAttachmentIndex = 43,
+  Alignment = 44,
+  MaxByteOffset = 45,
+  AlignmentId = 46,
+  MaxByteOffsetId = 47,
+  NoSignedWrap = 4469,
+  NoUnsignedWrap = 4470,
+  ExplicitInterpAMD = 4999,
+  OverrideCoverageNV = 5248,
+  PassthroughNV = 5250,
+  ViewportRelativeNV = 5252,
+  SecondaryViewportRelativeNV = 5256,
+  PerPrimitiveNV = 5271,
+  PerViewNV = 5272,
+  PerVertexNV = 5273,
+  NonUniformEXT = 5300,
+  CountBuffer = 5634,
+  UserSemantic = 5635,
+  RestrictPointerEXT = 5355,
+  AliasedPointerEXT = 5356,
+};
+}
+llvm::StringRef getDecorationName(Decoration::Decoration e);
+
+namespace BuiltIn {
+enum BuiltIn : uint32_t {
+  Position = 0,
+  PointSize = 1,
+  ClipDistance = 3,
+  CullDistance = 4,
+  VertexId = 5,
+  InstanceId = 6,
+  PrimitiveId = 7,
+  InvocationId = 8,
+  Layer = 9,
+  ViewportIndex = 10,
+  TessLevelOuter = 11,
+  TessLevelInner = 12,
+  TessCoord = 13,
+  PatchVertices = 14,
+  FragCoord = 15,
+  PointCoord = 16,
+  FrontFacing = 17,
+  SampleId = 18,
+  SamplePosition = 19,
+  SampleMask = 20,
+  FragDepth = 22,
+  HelperInvocation = 23,
+  NumWorkgroups = 24,
+  WorkgroupSize = 25,
+  WorkgroupId = 26,
+  LocalInvocationId = 27,
+  GlobalInvocationId = 28,
+  LocalInvocationIndex = 29,
+  WorkDim = 30,
+  GlobalSize = 31,
+  EnqueuedWorkgroupSize = 32,
+  GlobalOffset = 33,
+  GlobalLinearId = 34,
+  SubgroupSize = 36,
+  SubgroupMaxSize = 37,
+  NumSubgroups = 38,
+  NumEnqueuedSubgroups = 39,
+  SubgroupId = 40,
+  SubgroupLocalInvocationId = 41,
+  VertexIndex = 42,
+  InstanceIndex = 43,
+  SubgroupEqMask = 4416,
+  SubgroupGeMask = 4417,
+  SubgroupGtMask = 4418,
+  SubgroupLeMask = 4419,
+  SubgroupLtMask = 4420,
+  BaseVertex = 4424,
+  BaseInstance = 4425,
+  DrawIndex = 4426,
+  DeviceIndex = 4438,
+  ViewIndex = 4440,
+  BaryCoordNoPerspAMD = 4492,
+  BaryCoordNoPerspCentroidAMD = 4493,
+  BaryCoordNoPerspSampleAMD = 4494,
+  BaryCoordSmoothAMD = 4495,
+  BaryCoordSmoothCentroid = 4496,
+  BaryCoordSmoothSample = 4497,
+  BaryCoordPullModel = 4498,
+  FragStencilRefEXT = 5014,
+  ViewportMaskNV = 5253,
+  SecondaryPositionNV = 5257,
+  SecondaryViewportMaskNV = 5258,
+  PositionPerViewNV = 5261,
+  ViewportMaskPerViewNV = 5262,
+  FullyCoveredEXT = 5264,
+  TaskCountNV = 5274,
+  PrimitiveCountNV = 5275,
+  PrimitiveIndicesNV = 5276,
+  ClipDistancePerViewNV = 5277,
+  CullDistancePerViewNV = 5278,
+  LayerPerViewNV = 5279,
+  MeshViewCountNV = 5280,
+  MeshViewIndices = 5281,
+  BaryCoordNV = 5286,
+  BaryCoordNoPerspNV = 5287,
+  FragSizeEXT = 5292,
+  FragInvocationCountEXT = 5293,
+  LaunchIdNV = 5319,
+  LaunchSizeNV = 5320,
+  WorldRayOriginNV = 5321,
+  WorldRayDirectionNV = 5322,
+  ObjectRayOriginNV = 5323,
+  ObjectRayDirectionNV = 5324,
+  RayTminNV = 5325,
+  RayTmaxNV = 5326,
+  InstanceCustomIndexNV = 5327,
+  ObjectToWorldNV = 5330,
+  WorldToObjectNV = 5331,
+  HitTNV = 5332,
+  HitKindNV = 5333,
+  IncomingRayFlagsNV = 5351,
+};
+}
+llvm::StringRef getBuiltInName(BuiltIn::BuiltIn e);
+
+namespace SelectionControl {
+enum SelectionControl : uint32_t {
+  None = 0x0,
+  Flatten = 0x1,
+  DontFlatten = 0x2,
+};
+}
+std::string getSelectionControlName(SelectionControl::SelectionControl e);
+
+namespace LoopControl {
+enum LoopControl : uint32_t {
+  None = 0x0,
+  Unroll = 0x1,
+  DontUnroll = 0x2,
+  DependencyInfinite = 0x4,
+  DependencyLength = 0x8,
+  MinIterations = 0x10,
+  MaxIterations = 0x20,
+  IterationMultiple = 0x40,
+  PeelCount = 0x80,
+  PartialCount = 0x100,
+};
+}
+std::string getLoopControlName(LoopControl::LoopControl e);
+
+namespace FunctionControl {
+enum FunctionControl : uint32_t {
+  None = 0x0,
+  Inline = 0x1,
+  DontInline = 0x2,
+  Pure = 0x4,
+  Const = 0x8,
+};
+}
+std::string getFunctionControlName(FunctionControl::FunctionControl e);
+
+namespace MemorySemantics {
+enum MemorySemantics : uint32_t {
+  None = 0x0,
+  Acquire = 0x2,
+  Release = 0x4,
+  AcquireRelease = 0x8,
+  SequentiallyConsistent = 0x10,
+  UniformMemory = 0x40,
+  SubgroupMemory = 0x80,
+  WorkgroupMemory = 0x100,
+  CrossWorkgroupMemory = 0x200,
+  AtomicCounterMemory = 0x400,
+  ImageMemory = 0x800,
+  OutputMemoryKHR = 0x1000,
+  MakeAvailableKHR = 0x2000,
+  MakeVisibleKHR = 0x4000,
+};
+}
+std::string getMemorySemanticsName(MemorySemantics::MemorySemantics e);
+
+namespace MemoryOperand {
+enum MemoryOperand : uint32_t {
+  None = 0x0,
+  Volatile = 0x1,
+  Aligned = 0x2,
+  Nontemporal = 0x4,
+  MakePointerAvailableKHR = 0x8,
+  MakePointerVisibleKHR = 0x10,
+  NonPrivatePointerKHR = 0x20,
+};
+}
+std::string getMemoryOperandName(MemoryOperand::MemoryOperand e);
+
+namespace Scope {
+enum Scope : uint32_t {
+  CrossDevice = 0,
+  Device = 1,
+  Workgroup = 2,
+  Subgroup = 3,
+  Invocation = 4,
+  QueueFamilyKHR = 5,
+};
+}
+llvm::StringRef getScopeName(Scope::Scope e);
+
+namespace GroupOperation {
+enum GroupOperation : uint32_t {
+  Reduce = 0,
+  InclusiveScan = 1,
+  ExclusiveScan = 2,
+  ClusteredReduce = 3,
+  PartitionedReduceNV = 6,
+  PartitionedInclusiveScanNV = 7,
+  PartitionedExclusiveScanNV = 8,
+};
+}
+llvm::StringRef getGroupOperationName(GroupOperation::GroupOperation e);
+
+namespace KernelEnqueueFlags {
+enum KernelEnqueueFlags : uint32_t {
+  NoWait = 0,
+  WaitKernel = 1,
+  WaitWorkGroup = 2,
+};
+}
+llvm::StringRef
+getKernelEnqueueFlagsName(KernelEnqueueFlags::KernelEnqueueFlags e);
+
+namespace KernelProfilingInfo {
+enum KernelProfilingInfo : uint32_t {
+  None = 0x0,
+  CmdExecTime = 0x1,
+};
+}
+std::string
+getKernelProfilingInfoName(KernelProfilingInfo::KernelProfilingInfo e);
+} // namespace SPIRV
+} // namespace llvm
+
+// Return a string representation of the operands from startIndex onwards.
+// Templated to allow both MachineInstr and MCInst to use the same logic.
+template <class InstType>
+std::string getSPIRVStringOperand(const InstType &MI, unsigned int StartIndex) {
+  std::string s; // Iteratively append to this string
+
+  const unsigned int NumOps = MI.getNumOperands();
+  bool IsFinished = false;
+  for (unsigned int i = StartIndex; i < NumOps && !IsFinished; ++i) {
+    const auto &Op = MI.getOperand(i);
+    if (!Op.isImm()) // Stop if we hit a register operand
+      break;
+    assert((Op.getImm() >> 32) == 0 && "Imm operand should be i32 word");
+    const uint32_t Imm = Op.getImm(); // Each i32 word is up to 4 characters
+    for (unsigned ShiftAmount = 0; ShiftAmount < 32; ShiftAmount += 8) {
+      char c = (Imm >> ShiftAmount) & 0xff;
+      if (c == 0) { // Stop if we hit a null-terminator character
+        IsFinished = true;
+        break;
+      } else {
+        s += c; // Otherwise, append the character to the result string
+      }
+    }
+  }
+  return s;
+}
+
+#endif // LLVM_LIB_TARGET_SPIRV_MCTARGETDESC_SPIRVBASEINFO_H

llvm/lib/Target/SPIRV/MCTargetDesc/SPIRVBaseInfo.cpp

+//===-- SPIRVBaseInfo.cpp -  Top level definitions for SPIRV ----*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains small standalone helper functions and enum definitions for
+// the SPIRV target useful for the compiler back-end and the MC libraries.
+// As such, it deliberately does not include references to LLVM core
+// code gen types, passes, etc..
+//
+//===----------------------------------------------------------------------===//
+
+#include "SPIRVBaseInfo.h"
+
+using namespace llvm;
+using namespace SPIRV;
+// Implement getEnumName(Enum e) helper functions.
+StringRef SPIRV::getCapabilityName(Capability::Capability e) {
+  switch (e) {
+  case Capability::Matrix:
+    return "Matrix";
+  case Capability::Shader:
+    return "Shader";
+  case Capability::Geometry:
+    return "Geometry";
+  case Capability::Tessellation:
+    return "Tessellation";
+  case Capability::Addresses:
+    return "Addresses";
+  case Capability::Linkage:
+    return "Linkage";
+  case Capability::Kernel:
+    return "Kernel";
+  case Capability::Vector16:
+    return "Vector16";
+  case Capability::Float16Buffer:
+    return "Float16Buffer";
+  case Capability::Float16:
+    return "Float16";
+  case Capability::Float64:
+    return "Float64";
+  case Capability::Int64:
+    return "Int64";
+  case Capability::Int64Atomics:
+    return "Int64Atomics";
+  case Capability::ImageBasic:
+    return "ImageBasic";
+  case Capability::ImageReadWrite:
+    return "ImageReadWrite";
+  case Capability::ImageMipmap:
+    return "ImageMipmap";
+  case Capability::Pipes:
+    return "Pipes";
+  case Capability::Groups:
+    return "Groups";
+  case Capability::DeviceEnqueue:
+    return "DeviceEnqueue";
+  case Capability::LiteralSampler:
+    return "LiteralSampler";
+  case Capability::AtomicStorage:
+    return "AtomicStorage";
+  case Capability::Int16:
+    return "Int16";
+  case Capability::TessellationPointSize:
+    return "TessellationPointSize";
+  case Capability::GeometryPointSize:
+    return "GeometryPointSize";
+  case Capability::ImageGatherExtended:
+    return "ImageGatherExtended";
+  case Capability::StorageImageMultisample:
+    return "StorageImageMultisample";
+  case Capability::UniformBufferArrayDynamicIndexing:
+    return "UniformBufferArrayDynamicIndexing";
+  case Capability::SampledImageArrayDymnamicIndexing:
+    return "SampledImageArrayDymnamicIndexing";
+  case Capability::ClipDistance:
+    return "ClipDistance";
+  case Capability::CullDistance:
+    return "CullDistance";
+  case Capability::ImageCubeArray:
+    return "ImageCubeArray";
+  case Capability::SampleRateShading:
+    return "SampleRateShading";
+  case Capability::ImageRect:
+    return "ImageRect";
+  case Capability::SampledRect:
+    return "SampledRect";
+  case Capability::GenericPointer:
+    return "GenericPointer";
+  case Capability::Int8:
+    return "Int8";
+  case Capability::InputAttachment:
+    return "InputAttachment";
+  case Capability::SparseResidency:
+    return "SparseResidency";
+  case Capability::MinLod:
+    return "MinLod";
+  case Capability::Sampled1D:
+    return "Sampled1D";
+  case Capability::Image1D:
+    return "Image1D";
+  case Capability::SampledCubeArray:
+    return "SampledCubeArray";
+  case Capability::SampledBuffer:
+    return "SampledBuffer";
+  case Capability::ImageBuffer:
+    return "ImageBuffer";
+  case Capability::ImageMSArray:
+    return "ImageMSArray";
+  case Capability::StorageImageExtendedFormats:
+    return "StorageImageExtendedFormats";
+  case Capability::ImageQuery:
+    return "ImageQuery";
+  case Capability::DerivativeControl:
+    return "DerivativeControl";
+  case Capability::InterpolationFunction:
+    return "InterpolationFunction";
+  case Capability::TransformFeedback:
+    return "TransformFeedback";
+  case Capability::GeometryStreams:
+    return "GeometryStreams";
+  case Capability::StorageImageReadWithoutFormat:
+    return "StorageImageReadWithoutFormat";
+  case Capability::StorageImageWriteWithoutFormat:
+    return "StorageImageWriteWithoutFormat";
+  case Capability::MultiViewport:
+    return "MultiViewport";
+  case Capability::SubgroupDispatch:
+    return "SubgroupDispatch";
+  case Capability::NamedBarrier:
+    return "NamedBarrier";
+  case Capability::PipeStorage:
+    return "PipeStorage";
+  case Capability::GroupNonUniform:
+    return "GroupNonUniform";
+  case Capability::GroupNonUniformVote:
+    return "GroupNonUniformVote";
+  case Capability::GroupNonUniformArithmetic:
+    return "GroupNonUniformArithmetic";
+  case Capability::GroupNonUniformBallot:
+    return "GroupNonUniformBallot";
+  case Capability::GroupNonUniformShuffle:
+    return "GroupNonUniformShuffle";
+  case Capability::GroupNonUniformShuffleRelative:
+    return "GroupNonUniformShuffleRelative";
+  case Capability::GroupNonUniformClustered:
+    return "GroupNonUniformClustered";
+  case Capability::GroupNonUniformQuad:
+    return "GroupNonUniformQuad";
+  case Capability::SubgroupBallotKHR:
+    return "SubgroupBallotKHR";
+  case Capability::DrawParameters:
+    return "DrawParameters";
+  case Capability::SubgroupVoteKHR:
+    return "SubgroupVoteKHR";
+  case Capability::StorageBuffer16BitAccess:
+    return "SBBA(16)";
+  case Capability::StorageUniform16:
+    return "StorageUniform16";
+  case Capability::StoragePushConstant16:
+    return "StoragePushConstant16";
+  case Capability::StorageInputOutput16:
+    return "StorageInputOutput16";
+  case Capability::DeviceGroup:
+    return "DeviceGroup";
+  case Capability::MultiView:
+    return "MultiView";
+  case Capability::VariablePointersStorageBuffer:
+    return "VAR_PTR_SB";
+  case Capability::VariablePointers:
+    return "VariablePointers";
+  case Capability::AtomicStorageOps:
+    return "AtomicStorageOps";
+  case Capability::SampleMaskPostDepthCoverage:
+    return "SampleMaskPostDepthCoverage";
+  case Capability::StorageBuffer8BitAccess:
+    return "StorageBuffer8BitAccess";
+  case Capability::UniformAndStorageBuffer8BitAccess:
+    return "SB8BA(UniformAnd)";
+  case Capability::StoragePushConstant8:
+    return "StoragePushConstant8";
+  case Capability::DenormPreserve:
+    return "DenormPreserve";
+  case Capability::DenormFlushToZero:
+    return "DenormFlushToZero";
+  case Capability::SignedZeroInfNanPreserve:
+    return "SignedZeroInfNanPreserve";
+  case Capability::RoundingModeRTE:
+    return "RoundingModeRTE";
+  case Capability::RoundingModeRTZ:
+    return "RoundingModeRTZ";
+  case Capability::Float16ImageAMD:
+    return "Float16ImageAMD";
+  case Capability::ImageGatherBiasLodAMD:
+    return "ImageGatherBiasLodAMD";
+  case Capability::FragmentMaskAMD:
+    return "FragmentMaskAMD";
+  case Capability::StencilExportEXT:
+    return "StencilExportEXT";
+  case Capability::ImageReadWriteLodAMD:
+    return "ImageReadWriteLodAMD";
+  case Capability::SampleMaskOverrideCoverageNV:
+    return "SampleMaskOverrideCoverageNV";
+  case Capability::GeometryShaderPassthroughNV:
+    return "GeometryShaderPassthroughNV";
+  case Capability::ShaderViewportIndexLayerEXT:
+    return "ShaderViewportIndexLayerEXT";
+  case Capability::ShaderViewportMaskNV:
+    return "ShaderViewportMaskNV";
+  case Capability::ShaderStereoViewNV:
+    return "ShaderStereoViewNV";
+  case Capability::PerViewAttributesNV:
+    return "PerViewAttributesNV";
+  case Capability::FragmentFullyCoveredEXT:
+    return "FragmentFullyCoveredEXT";
+  case Capability::MeshShadingNV:
+    return "MeshShadingNV";
+  case Capability::ShaderNonUniformEXT:
+    return "ShaderNonUniformEXT";
+  case Capability::RuntimeDescriptorArrayEXT:
+    return "RuntimeDescriptorArrayEXT";
+  case Capability::InputAttachmentArrayDynamicIndexingEXT:
+    return "ADIE(InputAttachment)";
+  case Capability::UniformTexelBufferArrayDynamicIndexingEXT:
+    return "ADIE(UniformTexelBuffer)";
+  case Capability::StorageTexelBufferArrayDynamicIndexingEXT:
+    return "ADIE(StorageTexelBuffer)";
+  case Capability::UniformBufferArrayNonUniformIndexingEXT:
+    return "ANUIE(UniformBuffer)";
+  case Capability::SampledImageArrayNonUniformIndexingEXT:
+    return "ANUIE(SampledImage)";
+  case Capability::StorageBufferArrayNonUniformIndexingEXT:
+    return "ANUIE(StorageBuffer)";
+  case Capability::StorageImageArrayNonUniformIndexingEXT:
+    return "ANUIE(StorageImage)";
+  case Capability::InputAttachmentArrayNonUniformIndexingEXT:
+    return "ANUIE(InputAttachment)";
+  case Capability::UniformTexelBufferArrayNonUniformIndexingEXT:
+    return "ANUIE(UniformTexelBuffer)";
+  case Capability::StorageTexelBufferArrayNonUniformIndexingEXT:
+    return "ANUIE(StorageTexelBuffer)";
+  case Capability::RayTracingNV:
+    return "RayTracingNV";
+  case Capability::SubgroupShuffleINTEL:
+    return "SubgroupShuffleINTEL";
+  case Capability::SubgroupBufferBlockIOINTEL:
+    return "SubgroupBufferBlockIOINTEL";
+  case Capability::SubgroupImageBlockIOINTEL:
+    return "SubgroupImageBlockIOINTEL";
+  case Capability::SubgroupImageMediaBlockIOINTEL:
+    return "SubgroupImageMediaBlockIOINTEL";
+  case Capability::SubgroupAvcMotionEstimationINTEL:
+    return "SubgroupAvcMotionEstimationINTEL";
+  case Capability::SubgroupAvcMotionEstimationIntraINTEL:
+    return "SubgroupAvcMotionEstimationIntraINTEL";
+  case Capability::SubgroupAvcMotionEstimationChromaINTEL:
+    return "SubgroupAvcMotionEstimationChromaINTEL";
+  case Capability::GroupNonUniformPartitionedNV:
+    return "GroupNonUniformPartitionedNV";
+  case Capability::VulkanMemoryModelKHR:
+    return "VulkanMemoryModelKHR";
+  case Capability::VulkanMemoryModelDeviceScopeKHR:
+    return "VulkanMemoryModelDeviceScopeKHR";
+  case Capability::ImageFootprintNV:
+    return "ImageFootprintNV";
+  case Capability::FragmentBarycentricNV:
+    return "FragmentBarycentricNV";
+  case Capability::ComputeDerivativeGroupQuadsNV:
+    return "ComputeDerivativeGroupQuadsNV";
+  case Capability::ComputeDerivativeGroupLinearNV:
+    return "ComputeDerivativeGroupLinearNV";
+  case Capability::FragmentDensityEXT:
+    return "FragmentDensityEXT";
+  case Capability::PhysicalStorageBufferAddressesEXT:
+    return "PhysicalStorageBufferAddressesEXT";
+  case Capability::CooperativeMatrixNV:
+    return "CooperativeMatrixNV";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getSourceLanguageName(SourceLanguage::SourceLanguage e) {
+  switch (e) {
+  case SourceLanguage::Unknown:
+    return "Unknown";
+  case SourceLanguage::ESSL:
+    return "ESSL";
+  case SourceLanguage::GLSL:
+    return "GLSL";
+  case SourceLanguage::OpenCL_C:
+    return "OpenCL_C";
+  case SourceLanguage::OpenCL_CPP:
+    return "OpenCL_CPP";
+  case SourceLanguage::HLSL:
+    return "HLSL";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getExecutionModelName(ExecutionModel::ExecutionModel e) {
+  switch (e) {
+  case ExecutionModel::Vertex:
+    return "Vertex";
+  case ExecutionModel::TessellationControl:
+    return "TessellationControl";
+  case ExecutionModel::TessellationEvaluation:
+    return "TessellationEvaluation";
+  case ExecutionModel::Geometry:
+    return "Geometry";
+  case ExecutionModel::Fragment:
+    return "Fragment";
+  case ExecutionModel::GLCompute:
+    return "GLCompute";
+  case ExecutionModel::Kernel:
+    return "Kernel";
+  case ExecutionModel::TaskNV:
+    return "TaskNV";
+  case ExecutionModel::MeshNV:
+    return "MeshNV";
+  case ExecutionModel::RayGenerationNV:
+    return "RayGenerationNV";
+  case ExecutionModel::IntersectionNV:
+    return "IntersectionNV";
+  case ExecutionModel::AnyHitNV:
+    return "AnyHitNV";
+  case ExecutionModel::ClosestHitNV:
+    return "ClosestHitNV";
+  case ExecutionModel::MissNV:
+    return "MissNV";
+  case ExecutionModel::CallableNV:
+    return "CallableNV";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getAddressingModelName(AddressingModel::AddressingModel e) {
+  switch (e) {
+  case AddressingModel::Logical:
+    return "Logical";
+  case AddressingModel::Physical32:
+    return "Physical32";
+  case AddressingModel::Physical64:
+    return "Physical64";
+  case AddressingModel::PhysicalStorageBuffer64EXT:
+    return "PSB(64EXT)";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getMemoryModelName(MemoryModel::MemoryModel e) {
+  switch (e) {
+  case MemoryModel::Simple:
+    return "Simple";
+  case MemoryModel::GLSL450:
+    return "GLSL450";
+  case MemoryModel::OpenCL:
+    return "OpenCL";
+  case MemoryModel::VulkanKHR:
+    return "VulkanKHR";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getExecutionModeName(ExecutionMode::ExecutionMode e) {
+  switch (e) {
+  case ExecutionMode::Invocations:
+    return "Invocations";
+  case ExecutionMode::SpacingEqual:
+    return "SpacingEqual";
+  case ExecutionMode::SpacingFractionalEven:
+    return "SpacingFractionalEven";
+  case ExecutionMode::SpacingFractionalOdd:
+    return "SpacingFractionalOdd";
+  case ExecutionMode::VertexOrderCw:
+    return "VertexOrderCw";
+  case ExecutionMode::VertexOrderCcw:
+    return "VertexOrderCcw";
+  case ExecutionMode::PixelCenterInteger:
+    return "PixelCenterInteger";
+  case ExecutionMode::OriginUpperLeft:
+    return "OriginUpperLeft";
+  case ExecutionMode::OriginLowerLeft:
+    return "OriginLowerLeft";
+  case ExecutionMode::EarlyFragmentTests:
+    return "EarlyFragmentTests";
+  case ExecutionMode::PointMode:
+    return "PointMode";
+  case ExecutionMode::Xfb:
+    return "Xfb";
+  case ExecutionMode::DepthReplacing:
+    return "DepthReplacing";
+  case ExecutionMode::DepthGreater:
+    return "DepthGreater";
+  case ExecutionMode::DepthLess:
+    return "DepthLess";
+  case ExecutionMode::DepthUnchanged:
+    return "DepthUnchanged";
+  case ExecutionMode::LocalSize:
+    return "LocalSize";
+  case ExecutionMode::LocalSizeHint:
+    return "LocalSizeHint";
+  case ExecutionMode::InputPoints:
+    return "InputPoints";
+  case ExecutionMode::InputLines:
+    return "InputLines";
+  case ExecutionMode::InputLinesAdjacency:
+    return "InputLinesAdjacency";
+  case ExecutionMode::Triangles:
+    return "Triangles";
+  case ExecutionMode::InputTrianglesAdjacency:
+    return "InputTrianglesAdjacency";
+  case ExecutionMode::Quads:
+    return "Quads";
+  case ExecutionMode::Isolines:
+    return "Isolines";
+  case ExecutionMode::OutputVertices:
+    return "OutputVertices";
+  case ExecutionMode::OutputPoints:
+    return "OutputPoints";
+  case ExecutionMode::OutputLineStrip:
+    return "OutputLineStrip";
+  case ExecutionMode::OutputTriangleStrip:
+    return "OutputTriangleStrip";
+  case ExecutionMode::VecTypeHint:
+    return "VecTypeHint";
+  case ExecutionMode::ContractionOff:
+    return "ContractionOff";
+  case ExecutionMode::Initializer:
+    return "Initializer";
+  case ExecutionMode::Finalizer:
+    return "Finalizer";
+  case ExecutionMode::SubgroupSize:
+    return "SubgroupSize";
+  case ExecutionMode::SubgroupsPerWorkgroup:
+    return "SubgroupsPerWorkgroup";
+  case ExecutionMode::SubgroupsPerWorkgroupId:
+    return "SubgroupsPerWorkgroupId";
+  case ExecutionMode::LocalSizeId:
+    return "LocalSizeId";
+  case ExecutionMode::LocalSizeHintId:
+    return "LocalSizeHintId";
+  case ExecutionMode::PostDepthCoverage:
+    return "PostDepthCoverage";
+  case ExecutionMode::DenormPreserve:
+    return "DenormPreserve";
+  case ExecutionMode::DenormFlushToZero:
+    return "DenormFlushToZero";
+  case ExecutionMode::SignedZeroInfNanPreserve:
+    return "SignedZeroInfNanPreserve";
+  case ExecutionMode::RoundingModeRTE:
+    return "RoundingModeRTE";
+  case ExecutionMode::RoundingModeRTZ:
+    return "RoundingModeRTZ";
+  case ExecutionMode::StencilRefReplacingEXT:
+    return "StencilRefReplacingEXT";
+  case ExecutionMode::OutputLinesNV:
+    return "OutputLinesNV";
+  case ExecutionMode::DerivativeGroupQuadsNV:
+    return "DG1(QuadsNV)";
+  case ExecutionMode::DerivativeGroupLinearNV:
+    return "DG1(LinearNV)";
+  case ExecutionMode::OutputTrianglesNV:
+    return "OutputTrianglesNV";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getStorageClassName(StorageClass::StorageClass e) {
+  switch (e) {
+  case StorageClass::UniformConstant:
+    return "UniformConstant";
+  case StorageClass::Input:
+    return "Input";
+  case StorageClass::Uniform:
+    return "Uniform";
+  case StorageClass::Output:
+    return "Output";
+  case StorageClass::Workgroup:
+    return "Workgroup";
+  case StorageClass::CrossWorkgroup:
+    return "CrossWorkgroup";
+  case StorageClass::Private:
+    return "Private";
+  case StorageClass::Function:
+    return "Function";
+  case StorageClass::Generic:
+    return "Generic";
+  case StorageClass::PushConstant:
+    return "PushConstant";
+  case StorageClass::AtomicCounter:
+    return "AtomicCounter";
+  case StorageClass::Image:
+    return "Image";
+  case StorageClass::StorageBuffer:
+    return "StorageBuffer";
+  case StorageClass::CallableDataNV:
+    return "CallableDataNV";
+  case StorageClass::IncomingCallableDataNV:
+    return "IncomingCallableDataNV";
+  case StorageClass::RayPayloadNV:
+    return "RayPayloadNV";
+  case StorageClass::HitAttributeNV:
+    return "HitAttributeNV";
+  case StorageClass::IncomingRayPayloadNV:
+    return "IncomingRayPayloadNV";
+  case StorageClass::ShaderRecordBufferNV:
+    return "ShaderRecordBufferNV";
+  case StorageClass::PhysicalStorageBufferEXT:
+    return "PSB(EXT)";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getDimName(Dim::Dim dim) {
+  switch (dim) {
+  case Dim::DIM_1D:
+    return "1D";
+  case Dim::DIM_2D:
+    return "2D";
+  case Dim::DIM_3D:
+    return "3D";
+  case Dim::DIM_Cube:
+    return "Cube";
+  case Dim::DIM_Rect:
+    return "Rect";
+  case Dim::DIM_Buffer:
+    return "Buffer";
+  case Dim::DIM_SubpassData:
+    return "SubpassData";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getSamplerAddressingModeName(
+    SamplerAddressingMode::SamplerAddressingMode e) {
+  switch (e) {
+  case SamplerAddressingMode::None:
+    return "None";
+  case SamplerAddressingMode::ClampToEdge:
+    return "ClampToEdge";
+  case SamplerAddressingMode::Clamp:
+    return "Clamp";
+  case SamplerAddressingMode::Repeat:
+    return "Repeat";
+  case SamplerAddressingMode::RepeatMirrored:
+    return "RepeatMirrored";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef
+SPIRV::getSamplerFilterModeName(SamplerFilterMode::SamplerFilterMode e) {
+  switch (e) {
+  case SamplerFilterMode::Nearest:
+    return "Nearest";
+  case SamplerFilterMode::Linear:
+    return "Linear";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getImageFormatName(ImageFormat::ImageFormat e) {
+  switch (e) {
+  case ImageFormat::Unknown:
+    return "Unknown";
+  case ImageFormat::Rgba32f:
+    return "Rgba32f";
+  case ImageFormat::Rgba16f:
+    return "Rgba16f";
+  case ImageFormat::R32f:
+    return "R32f";
+  case ImageFormat::Rgba8:
+    return "Rgba8";
+  case ImageFormat::Rgba8Snorm:
+    return "Rgba8Snorm";
+  case ImageFormat::Rg32f:
+    return "Rg32f";
+  case ImageFormat::Rg16f:
+    return "Rg16f";
+  case ImageFormat::R11fG11fB10f:
+    return "R11fG11fB10f";
+  case ImageFormat::R16f:
+    return "R16f";
+  case ImageFormat::Rgba16:
+    return "Rgba16";
+  case ImageFormat::Rgb10A2:
+    return "Rgb10A2";
+  case ImageFormat::Rg16:
+    return "Rg16";
+  case ImageFormat::Rg8:
+    return "Rg8";
+  case ImageFormat::R16:
+    return "R16";
+  case ImageFormat::R8:
+    return "R8";
+  case ImageFormat::Rgba16Snorm:
+    return "Rgba16Snorm";
+  case ImageFormat::Rg16Snorm:
+    return "Rg16Snorm";
+  case ImageFormat::Rg8Snorm:
+    return "Rg8Snorm";
+  case ImageFormat::R16Snorm:
+    return "R16Snorm";
+  case ImageFormat::R8Snorm:
+    return "R8Snorm";
+  case ImageFormat::Rgba32i:
+    return "Rgba32i";
+  case ImageFormat::Rgba16i:
+    return "Rgba16i";
+  case ImageFormat::Rgba8i:
+    return "Rgba8i";
+  case ImageFormat::R32i:
+    return "R32i";
+  case ImageFormat::Rg32i:
+    return "Rg32i";
+  case ImageFormat::Rg16i:
+    return "Rg16i";
+  case ImageFormat::Rg8i:
+    return "Rg8i";
+  case ImageFormat::R16i:
+    return "R16i";
+  case ImageFormat::R8i:
+    return "R8i";
+  case ImageFormat::Rgba32ui:
+    return "Rgba32ui";
+  case ImageFormat::Rgba16ui:
+    return "Rgba16ui";
+  case ImageFormat::Rgba8ui:
+    return "Rgba8ui";
+  case ImageFormat::R32ui:
+    return "R32ui";
+  case ImageFormat::Rgb10a2ui:
+    return "Rgb10a2ui";
+  case ImageFormat::Rg32ui:
+    return "Rg32ui";
+  case ImageFormat::Rg16ui:
+    return "Rg16ui";
+  case ImageFormat::Rg8ui:
+    return "Rg8ui";
+  case ImageFormat::R16ui:
+    return "R16ui";
+  case ImageFormat::R8ui:
+    return "R8ui";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef
+SPIRV::getImageChannelOrderName(ImageChannelOrder::ImageChannelOrder e) {
+  switch (e) {
+  case ImageChannelOrder::R:
+    return "R";
+  case ImageChannelOrder::A:
+    return "A";
+  case ImageChannelOrder::RG:
+    return "RG";
+  case ImageChannelOrder::RA:
+    return "RA";
+  case ImageChannelOrder::RGB:
+    return "RGB";
+  case ImageChannelOrder::RGBA:
+    return "RGBA";
+  case ImageChannelOrder::BGRA:
+    return "BGRA";
+  case ImageChannelOrder::ARGB:
+    return "ARGB";
+  case ImageChannelOrder::Intensity:
+    return "Intensity";
+  case ImageChannelOrder::Luminance:
+    return "Luminance";
+  case ImageChannelOrder::Rx:
+    return "Rx";
+  case ImageChannelOrder::RGx:
+    return "RGx";
+  case ImageChannelOrder::RGBx:
+    return "RGBx";
+  case ImageChannelOrder::Depth:
+    return "Depth";
+  case ImageChannelOrder::DepthStencil:
+    return "DepthStencil";
+  case ImageChannelOrder::sRGB:
+    return "sRGB";
+  case ImageChannelOrder::sRGBx:
+    return "sRGBx";
+  case ImageChannelOrder::sRGBA:
+    return "sRGBA";
+  case ImageChannelOrder::sBGRA:
+    return "sBGRA";
+  case ImageChannelOrder::ABGR:
+    return "ABGR";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getImageChannelDataTypeName(
+    ImageChannelDataType::ImageChannelDataType e) {
+  switch (e) {
+  case ImageChannelDataType::SnormInt8:
+    return "SnormInt8";
+  case ImageChannelDataType::SnormInt16:
+    return "SnormInt16";
+  case ImageChannelDataType::UnormInt8:
+    return "UnormInt8";
+  case ImageChannelDataType::UnormInt16:
+    return "UnormInt16";
+  case ImageChannelDataType::UnormShort565:
+    return "UnormShort565";
+  case ImageChannelDataType::UnormShort555:
+    return "UnormShort555";
+  case ImageChannelDataType::UnormInt101010:
+    return "UnormInt101010";
+  case ImageChannelDataType::SignedInt8:
+    return "SignedInt8";
+  case ImageChannelDataType::SignedInt16:
+    return "SignedInt16";
+  case ImageChannelDataType::SignedInt32:
+    return "SignedInt32";
+  case ImageChannelDataType::UnsignedInt8:
+    return "UnsignedInt8";
+  case ImageChannelDataType::UnsignedInt16:
+    return "UnsignedInt16";
+  case ImageChannelDataType::UnsigendInt32:
+    return "UnsigendInt32";
+  case ImageChannelDataType::HalfFloat:
+    return "HalfFloat";
+  case ImageChannelDataType::Float:
+    return "Float";
+  case ImageChannelDataType::UnormInt24:
+    return "UnormInt24";
+  case ImageChannelDataType::UnormInt101010_2:
+    return "UnormInt101010_2";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+std::string SPIRV::getImageOperandName(ImageOperand::ImageOperand e) {
+  std::string nameString = "";
+  std::string sep = "";
+  if (e == ImageOperand::None)
+    return "None";
+  if ((ImageOperand::None != 0) && (e & ImageOperand::None)) {
+    nameString += sep + "None";
+    sep = "|";
+  }
+  if (e == ImageOperand::Bias)
+    return "Bias";
+  if ((ImageOperand::Bias != 0) && (e & ImageOperand::Bias)) {
+    nameString += sep + "Bias";
+    sep = "|";
+  }
+  if (e == ImageOperand::Lod)
+    return "Lod";
+  if ((ImageOperand::Lod != 0) && (e & ImageOperand::Lod)) {
+    nameString += sep + "Lod";
+    sep = "|";
+  }
+  if (e == ImageOperand::Grad)
+    return "Grad";
+  if ((ImageOperand::Grad != 0) && (e & ImageOperand::Grad)) {
+    nameString += sep + "Grad";
+    sep = "|";
+  }
+  if (e == ImageOperand::ConstOffset)
+    return "ConstOffset";
+  if ((ImageOperand::ConstOffset != 0) && (e & ImageOperand::ConstOffset)) {
+    nameString += sep + "ConstOffset";
+    sep = "|";
+  }
+  if (e == ImageOperand::Offset)
+    return "Offset";
+  if ((ImageOperand::Offset != 0) && (e & ImageOperand::Offset)) {
+    nameString += sep + "Offset";
+    sep = "|";
+  }
+  if (e == ImageOperand::ConstOffsets)
+    return "ConstOffsets";
+  if ((ImageOperand::ConstOffsets != 0) && (e & ImageOperand::ConstOffsets)) {
+    nameString += sep + "ConstOffsets";
+    sep = "|";
+  }
+  if (e == ImageOperand::Sample)
+    return "Sample";
+  if ((ImageOperand::Sample != 0) && (e & ImageOperand::Sample)) {
+    nameString += sep + "Sample";
+    sep = "|";
+  }
+  if (e == ImageOperand::MinLod)
+    return "MinLod";
+  if ((ImageOperand::MinLod != 0) && (e & ImageOperand::MinLod)) {
+    nameString += sep + "MinLod";
+    sep = "|";
+  }
+  if (e == ImageOperand::MakeTexelAvailableKHR)
+    return "MakeTexelAvailableKHR";
+  if ((ImageOperand::MakeTexelAvailableKHR != 0) &&
+      (e & ImageOperand::MakeTexelAvailableKHR)) {
+    nameString += sep + "MakeTexelAvailableKHR";
+    sep = "|";
+  }
+  if (e == ImageOperand::MakeTexelVisibleKHR)
+    return "MakeTexelVisibleKHR";
+  if ((ImageOperand::MakeTexelVisibleKHR != 0) &&
+      (e & ImageOperand::MakeTexelVisibleKHR)) {
+    nameString += sep + "MakeTexelVisibleKHR";
+    sep = "|";
+  }
+  if (e == ImageOperand::NonPrivateTexelKHR)
+    return "NonPrivateTexelKHR";
+  if ((ImageOperand::NonPrivateTexelKHR != 0) &&
+      (e & ImageOperand::NonPrivateTexelKHR)) {
+    nameString += sep + "NonPrivateTexelKHR";
+    sep = "|";
+  }
+  if (e == ImageOperand::VolatileTexelKHR)
+    return "VolatileTexelKHR";
+  if ((ImageOperand::VolatileTexelKHR != 0) &&
+      (e & ImageOperand::VolatileTexelKHR)) {
+    nameString += sep + "VolatileTexelKHR";
+    sep = "|";
+  }
+  if (e == ImageOperand::SignExtend)
+    return "SignExtend";
+  if ((ImageOperand::SignExtend != 0) && (e & ImageOperand::SignExtend)) {
+    nameString += sep + "SignExtend";
+    sep = "|";
+  }
+  if (e == ImageOperand::ZeroExtend)
+    return "ZeroExtend";
+  if ((ImageOperand::ZeroExtend != 0) && (e & ImageOperand::ZeroExtend)) {
+    nameString += sep + "ZeroExtend";
+    sep = "|";
+  };
+  return nameString;
+}
+
+std::string SPIRV::getFPFastMathModeName(FPFastMathMode::FPFastMathMode e) {
+  std::string nameString = "";
+  std::string sep = "";
+  if (e == FPFastMathMode::None)
+    return "None";
+  if ((FPFastMathMode::None != 0) && (e & FPFastMathMode::None)) {
+    nameString += sep + "None";
+    sep = "|";
+  }
+  if (e == FPFastMathMode::NotNaN)
+    return "NotNaN";
+  if ((FPFastMathMode::NotNaN != 0) && (e & FPFastMathMode::NotNaN)) {
+    nameString += sep + "NotNaN";
+    sep = "|";
+  }
+  if (e == FPFastMathMode::NotInf)
+    return "NotInf";
+  if ((FPFastMathMode::NotInf != 0) && (e & FPFastMathMode::NotInf)) {
+    nameString += sep + "NotInf";
+    sep = "|";
+  }
+  if (e == FPFastMathMode::NSZ)
+    return "NSZ";
+  if ((FPFastMathMode::NSZ != 0) && (e & FPFastMathMode::NSZ)) {
+    nameString += sep + "NSZ";
+    sep = "|";
+  }
+  if (e == FPFastMathMode::AllowRecip)
+    return "AllowRecip";
+  if ((FPFastMathMode::AllowRecip != 0) && (e & FPFastMathMode::AllowRecip)) {
+    nameString += sep + "AllowRecip";
+    sep = "|";
+  }
+  if (e == FPFastMathMode::Fast)
+    return "Fast";
+  if ((FPFastMathMode::Fast != 0) && (e & FPFastMathMode::Fast)) {
+    nameString += sep + "Fast";
+    sep = "|";
+  };
+  return nameString;
+}
+
+StringRef SPIRV::getFPRoundingModeName(FPRoundingMode::FPRoundingMode e) {
+  switch (e) {
+  case FPRoundingMode::RTE:
+    return "RTE";
+  case FPRoundingMode::RTZ:
+    return "RTZ";
+  case FPRoundingMode::RTP:
+    return "RTP";
+  case FPRoundingMode::RTN:
+    return "RTN";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getLinkageTypeName(LinkageType::LinkageType e) {
+  switch (e) {
+  case LinkageType::Export:
+    return "Export";
+  case LinkageType::Import:
+    return "Import";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getAccessQualifierName(AccessQualifier::AccessQualifier e) {
+  switch (e) {
+  case AccessQualifier::ReadOnly:
+    return "ReadOnly";
+  case AccessQualifier::WriteOnly:
+    return "WriteOnly";
+  case AccessQualifier::ReadWrite:
+    return "ReadWrite";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getFunctionParameterAttributeName(
+    FunctionParameterAttribute::FunctionParameterAttribute e) {
+  switch (e) {
+  case FunctionParameterAttribute::Zext:
+    return "Zext";
+  case FunctionParameterAttribute::Sext:
+    return "Sext";
+  case FunctionParameterAttribute::ByVal:
+    return "ByVal";
+  case FunctionParameterAttribute::Sret:
+    return "Sret";
+  case FunctionParameterAttribute::NoAlias:
+    return "NoAlias";
+  case FunctionParameterAttribute::NoCapture:
+    return "NoCapture";
+  case FunctionParameterAttribute::NoWrite:
+    return "NoWrite";
+  case FunctionParameterAttribute::NoReadWrite:
+    return "NoReadWrite";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getDecorationName(Decoration::Decoration e) {
+  switch (e) {
+  case Decoration::RelaxedPrecision:
+    return "RelaxedPrecision";
+  case Decoration::SpecId:
+    return "SpecId";
+  case Decoration::Block:
+    return "Block";
+  case Decoration::BufferBlock:
+    return "BufferBlock";
+  case Decoration::RowMajor:
+    return "RowMajor";
+  case Decoration::ColMajor:
+    return "ColMajor";
+  case Decoration::ArrayStride:
+    return "ArrayStride";
+  case Decoration::MatrixStride:
+    return "MatrixStride";
+  case Decoration::GLSLShared:
+    return "GLSLShared";
+  case Decoration::GLSLPacked:
+    return "GLSLPacked";
+  case Decoration::CPacked:
+    return "CPacked";
+  case Decoration::BuiltIn:
+    return "BuiltIn";
+  case Decoration::NoPerspective:
+    return "NoPerspective";
+  case Decoration::Flat:
+    return "Flat";
+  case Decoration::Patch:
+    return "Patch";
+  case Decoration::Centroid:
+    return "Centroid";
+  case Decoration::Sample:
+    return "Sample";
+  case Decoration::Invariant:
+    return "Invariant";
+  case Decoration::Restrict:
+    return "Restrict";
+  case Decoration::Aliased:
+    return "Aliased";
+  case Decoration::Volatile:
+    return "Volatile";
+  case Decoration::Constant:
+    return "Constant";
+  case Decoration::Coherent:
+    return "Coherent";
+  case Decoration::NonWritable:
+    return "NonWritable";
+  case Decoration::NonReadable:
+    return "NonReadable";
+  case Decoration::Uniform:
+    return "Uniform";
+  case Decoration::UniformId:
+    return "UniformId";
+  case Decoration::SaturatedConversion:
+    return "SaturatedConversion";
+  case Decoration::Stream:
+    return "Stream";
+  case Decoration::Location:
+    return "Location";
+  case Decoration::Component:
+    return "Component";
+  case Decoration::Index:
+    return "Index";
+  case Decoration::Binding:
+    return "Binding";
+  case Decoration::DescriptorSet:
+    return "DescriptorSet";
+  case Decoration::Offset:
+    return "Offset";
+  case Decoration::XfbBuffer:
+    return "XfbBuffer";
+  case Decoration::XfbStride:
+    return "XfbStride";
+  case Decoration::FuncParamAttr:
+    return "FuncParamAttr";
+  case Decoration::FPRoundingMode:
+    return "FPRoundingMode";
+  case Decoration::FPFastMathMode:
+    return "FPFastMathMode";
+  case Decoration::LinkageAttributes:
+    return "LinkageAttributes";
+  case Decoration::NoContraction:
+    return "NoContraction";
+  case Decoration::InputAttachmentIndex:
+    return "InputAttachmentIndex";
+  case Decoration::Alignment:
+    return "Alignment";
+  case Decoration::MaxByteOffset:
+    return "MaxByteOffset";
+  case Decoration::AlignmentId:
+    return "AlignmentId";
+  case Decoration::MaxByteOffsetId:
+    return "MaxByteOffsetId";
+  case Decoration::NoSignedWrap:
+    return "NoSignedWrap";
+  case Decoration::NoUnsignedWrap:
+    return "NoUnsignedWrap";
+  case Decoration::ExplicitInterpAMD:
+    return "ExplicitInterpAMD";
+  case Decoration::OverrideCoverageNV:
+    return "OverrideCoverageNV";
+  case Decoration::PassthroughNV:
+    return "PassthroughNV";
+  case Decoration::ViewportRelativeNV:
+    return "ViewportRelativeNV";
+  case Decoration::SecondaryViewportRelativeNV:
+    return "SecondaryViewportRelativeNV";
+  case Decoration::PerPrimitiveNV:
+    return "PerPrimitiveNV";
+  case Decoration::PerViewNV:
+    return "PerViewNV";
+  case Decoration::PerVertexNV:
+    return "PerVertexNV";
+  case Decoration::NonUniformEXT:
+    return "NonUniformEXT";
+  case Decoration::CountBuffer:
+    return "CountBuffer";
+  case Decoration::UserSemantic:
+    return "UserSemantic";
+  case Decoration::RestrictPointerEXT:
+    return "RestrictPointerEXT";
+  case Decoration::AliasedPointerEXT:
+    return "AliasedPointerEXT";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getBuiltInName(BuiltIn::BuiltIn e) {
+  switch (e) {
+  case BuiltIn::Position:
+    return "Position";
+  case BuiltIn::PointSize:
+    return "PointSize";
+  case BuiltIn::ClipDistance:
+    return "ClipDistance";
+  case BuiltIn::CullDistance:
+    return "CullDistance";
+  case BuiltIn::VertexId:
+    return "VertexId";
+  case BuiltIn::InstanceId:
+    return "InstanceId";
+  case BuiltIn::PrimitiveId:
+    return "PrimitiveId";
+  case BuiltIn::InvocationId:
+    return "InvocationId";
+  case BuiltIn::Layer:
+    return "Layer";
+  case BuiltIn::ViewportIndex:
+    return "ViewportIndex";
+  case BuiltIn::TessLevelOuter:
+    return "TessLevelOuter";
+  case BuiltIn::TessLevelInner:
+    return "TessLevelInner";
+  case BuiltIn::TessCoord:
+    return "TessCoord";
+  case BuiltIn::PatchVertices:
+    return "PatchVertices";
+  case BuiltIn::FragCoord:
+    return "FragCoord";
+  case BuiltIn::PointCoord:
+    return "PointCoord";
+  case BuiltIn::FrontFacing:
+    return "FrontFacing";
+  case BuiltIn::SampleId:
+    return "SampleId";
+  case BuiltIn::SamplePosition:
+    return "SamplePosition";
+  case BuiltIn::SampleMask:
+    return "SampleMask";
+  case BuiltIn::FragDepth:
+    return "FragDepth";
+  case BuiltIn::HelperInvocation:
+    return "HelperInvocation";
+  case BuiltIn::NumWorkgroups:
+    return "NumWorkgroups";
+  case BuiltIn::WorkgroupSize:
+    return "WorkgroupSize";
+  case BuiltIn::WorkgroupId:
+    return "WorkgroupId";
+  case BuiltIn::LocalInvocationId:
+    return "LocalInvocationId";
+  case BuiltIn::GlobalInvocationId:
+    return "GlobalInvocationId";
+  case BuiltIn::LocalInvocationIndex:
+    return "LocalInvocationIndex";
+  case BuiltIn::WorkDim:
+    return "WorkDim";
+  case BuiltIn::GlobalSize:
+    return "GlobalSize";
+  case BuiltIn::EnqueuedWorkgroupSize:
+    return "EnqueuedWorkgroupSize";
+  case BuiltIn::GlobalOffset:
+    return "GlobalOffset";
+  case BuiltIn::GlobalLinearId:
+    return "GlobalLinearId";
+  case BuiltIn::SubgroupSize:
+    return "SubgroupSize";
+  case BuiltIn::SubgroupMaxSize:
+    return "SubgroupMaxSize";
+  case BuiltIn::NumSubgroups:
+    return "NumSubgroups";
+  case BuiltIn::NumEnqueuedSubgroups:
+    return "NumEnqueuedSubgroups";
+  case BuiltIn::SubgroupId:
+    return "SubgroupId";
+  case BuiltIn::SubgroupLocalInvocationId:
+    return "SubgroupLocalInvocationId";
+  case BuiltIn::VertexIndex:
+    return "VertexIndex";
+  case BuiltIn::InstanceIndex:
+    return "InstanceIndex";
+  case BuiltIn::SubgroupEqMask:
+    return "SubgroupEqMask";
+  case BuiltIn::SubgroupGeMask:
+    return "SubgroupGeMask";
+  case BuiltIn::SubgroupGtMask:
+    return "SubgroupGtMask";
+  case BuiltIn::SubgroupLeMask:
+    return "SubgroupLeMask";
+  case BuiltIn::SubgroupLtMask:
+    return "SubgroupLtMask";
+  case BuiltIn::BaseVertex:
+    return "BaseVertex";
+  case BuiltIn::BaseInstance:
+    return "BaseInstance";
+  case BuiltIn::DrawIndex:
+    return "DrawIndex";
+  case BuiltIn::DeviceIndex:
+    return "DeviceIndex";
+  case BuiltIn::ViewIndex:
+    return "ViewIndex";
+  case BuiltIn::BaryCoordNoPerspAMD:
+    return "BaryCoordNoPerspAMD";
+  case BuiltIn::BaryCoordNoPerspCentroidAMD:
+    return "BaryCoordNoPerspCentroidAMD";
+  case BuiltIn::BaryCoordNoPerspSampleAMD:
+    return "BaryCoordNoPerspSampleAMD";
+  case BuiltIn::BaryCoordSmoothAMD:
+    return "BaryCoordSmoothAMD";
+  case BuiltIn::BaryCoordSmoothCentroid:
+    return "BaryCoordSmoothCentroid";
+  case BuiltIn::BaryCoordSmoothSample:
+    return "BaryCoordSmoothSample";
+  case BuiltIn::BaryCoordPullModel:
+    return "BaryCoordPullModel";
+  case BuiltIn::FragStencilRefEXT:
+    return "FragStencilRefEXT";
+  case BuiltIn::ViewportMaskNV:
+    return "ViewportMaskNV";
+  case BuiltIn::SecondaryPositionNV:
+    return "SecondaryPositionNV";
+  case BuiltIn::SecondaryViewportMaskNV:
+    return "SecondaryViewportMaskNV";
+  case BuiltIn::PositionPerViewNV:
+    return "PositionPerViewNV";
+  case BuiltIn::ViewportMaskPerViewNV:
+    return "ViewportMaskPerViewNV";
+  case BuiltIn::FullyCoveredEXT:
+    return "FullyCoveredEXT";
+  case BuiltIn::TaskCountNV:
+    return "TaskCountNV";
+  case BuiltIn::PrimitiveCountNV:
+    return "PrimitiveCountNV";
+  case BuiltIn::PrimitiveIndicesNV:
+    return "PrimitiveIndicesNV";
+  case BuiltIn::ClipDistancePerViewNV:
+    return "ClipDistancePerViewNV";
+  case BuiltIn::CullDistancePerViewNV:
+    return "CullDistancePerViewNV";
+  case BuiltIn::LayerPerViewNV:
+    return "LayerPerViewNV";
+  case BuiltIn::MeshViewCountNV:
+    return "MeshViewCountNV";
+  case BuiltIn::MeshViewIndices:
+    return "MeshViewIndices";
+  case BuiltIn::BaryCoordNV:
+    return "BaryCoordNV";
+  case BuiltIn::BaryCoordNoPerspNV:
+    return "BaryCoordNoPerspNV";
+  case BuiltIn::FragSizeEXT:
+    return "FragSizeEXT";
+  case BuiltIn::FragInvocationCountEXT:
+    return "FragInvocationCountEXT";
+  case BuiltIn::LaunchIdNV:
+    return "LaunchIdNV";
+  case BuiltIn::LaunchSizeNV:
+    return "LaunchSizeNV";
+  case BuiltIn::WorldRayOriginNV:
+    return "WorldRayOriginNV";
+  case BuiltIn::WorldRayDirectionNV:
+    return "WorldRayDirectionNV";
+  case BuiltIn::ObjectRayOriginNV:
+    return "ObjectRayOriginNV";
+  case BuiltIn::ObjectRayDirectionNV:
+    return "ObjectRayDirectionNV";
+  case BuiltIn::RayTminNV:
+    return "RayTminNV";
+  case BuiltIn::RayTmaxNV:
+    return "RayTmaxNV";
+  case BuiltIn::InstanceCustomIndexNV:
+    return "InstanceCustomIndexNV";
+  case BuiltIn::ObjectToWorldNV:
+    return "ObjectToWorldNV";
+  case BuiltIn::WorldToObjectNV:
+    return "WorldToObjectNV";
+  case BuiltIn::HitTNV:
+    return "HitTNV";
+  case BuiltIn::HitKindNV:
+    return "HitKindNV";
+  case BuiltIn::IncomingRayFlagsNV:
+    return "IncomingRayFlagsNV";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+std::string
+SPIRV::getSelectionControlName(SelectionControl::SelectionControl e) {
+  std::string nameString = "";
+  std::string sep = "";
+  if (e == SelectionControl::None)
+    return "None";
+  if ((SelectionControl::None != 0) && (e & SelectionControl::None)) {
+    nameString += sep + "None";
+    sep = "|";
+  }
+  if (e == SelectionControl::Flatten)
+    return "Flatten";
+  if ((SelectionControl::Flatten != 0) && (e & SelectionControl::Flatten)) {
+    nameString += sep + "Flatten";
+    sep = "|";
+  }
+  if (e == SelectionControl::DontFlatten)
+    return "DontFlatten";
+  if ((SelectionControl::DontFlatten != 0) &&
+      (e & SelectionControl::DontFlatten)) {
+    nameString += sep + "DontFlatten";
+    sep = "|";
+  };
+  return nameString;
+}
+
+std::string SPIRV::getLoopControlName(LoopControl::LoopControl e) {
+  std::string nameString = "";
+  std::string sep = "";
+  if (e == LoopControl::None)
+    return "None";
+  if ((LoopControl::None != 0) && (e & LoopControl::None)) {
+    nameString += sep + "None";
+    sep = "|";
+  }
+  if (e == LoopControl::Unroll)
+    return "Unroll";
+  if ((LoopControl::Unroll != 0) && (e & LoopControl::Unroll)) {
+    nameString += sep + "Unroll";
+    sep = "|";
+  }
+  if (e == LoopControl::DontUnroll)
+    return "DontUnroll";
+  if ((LoopControl::DontUnroll != 0) && (e & LoopControl::DontUnroll)) {
+    nameString += sep + "DontUnroll";
+    sep = "|";
+  }
+  if (e == LoopControl::DependencyInfinite)
+    return "DependencyInfinite";
+  if ((LoopControl::DependencyInfinite != 0) &&
+      (e & LoopControl::DependencyInfinite)) {
+    nameString += sep + "DependencyInfinite";
+    sep = "|";
+  }
+  if (e == LoopControl::DependencyLength)
+    return "DependencyLength";
+  if ((LoopControl::DependencyLength != 0) &&
+      (e & LoopControl::DependencyLength)) {
+    nameString += sep + "DependencyLength";
+    sep = "|";
+  }
+  if (e == LoopControl::MinIterations)
+    return "MinIterations";
+  if ((LoopControl::MinIterations != 0) && (e & LoopControl::MinIterations)) {
+    nameString += sep + "MinIterations";
+    sep = "|";
+  }
+  if (e == LoopControl::MaxIterations)
+    return "MaxIterations";
+  if ((LoopControl::MaxIterations != 0) && (e & LoopControl::MaxIterations)) {
+    nameString += sep + "MaxIterations";
+    sep = "|";
+  }
+  if (e == LoopControl::IterationMultiple)
+    return "IterationMultiple";
+  if ((LoopControl::IterationMultiple != 0) &&
+      (e & LoopControl::IterationMultiple)) {
+    nameString += sep + "IterationMultiple";
+    sep = "|";
+  }
+  if (e == LoopControl::PeelCount)
+    return "PeelCount";
+  if ((LoopControl::PeelCount != 0) && (e & LoopControl::PeelCount)) {
+    nameString += sep + "PeelCount";
+    sep = "|";
+  }
+  if (e == LoopControl::PartialCount)
+    return "PartialCount";
+  if ((LoopControl::PartialCount != 0) && (e & LoopControl::PartialCount)) {
+    nameString += sep + "PartialCount";
+    sep = "|";
+  };
+  return nameString;
+}
+
+std::string SPIRV::getFunctionControlName(FunctionControl::FunctionControl e) {
+  std::string nameString = "";
+  std::string sep = "";
+  if (e == FunctionControl::None)
+    return "None";
+  if ((FunctionControl::None != 0) && (e & FunctionControl::None)) {
+    nameString += sep + "None";
+    sep = "|";
+  }
+  if (e == FunctionControl::Inline)
+    return "Inline";
+  if ((FunctionControl::Inline != 0) && (e & FunctionControl::Inline)) {
+    nameString += sep + "Inline";
+    sep = "|";
+  }
+  if (e == FunctionControl::DontInline)
+    return "DontInline";
+  if ((FunctionControl::DontInline != 0) && (e & FunctionControl::DontInline)) {
+    nameString += sep + "DontInline";
+    sep = "|";
+  }
+  if (e == FunctionControl::Pure)
+    return "Pure";
+  if ((FunctionControl::Pure != 0) && (e & FunctionControl::Pure)) {
+    nameString += sep + "Pure";
+    sep = "|";
+  }
+  if (e == FunctionControl::Const)
+    return "Const";
+  if ((FunctionControl::Const != 0) && (e & FunctionControl::Const)) {
+    nameString += sep + "Const";
+    sep = "|";
+  };
+  return nameString;
+}
+
+std::string SPIRV::getMemorySemanticsName(MemorySemantics::MemorySemantics e) {
+  std::string nameString = "";
+  std::string sep = "";
+  if (e == MemorySemantics::None)
+    return "None";
+  if ((MemorySemantics::None != 0) && (e & MemorySemantics::None)) {
+    nameString += sep + "None";
+    sep = "|";
+  }
+  if (e == MemorySemantics::Acquire)
+    return "Acquire";
+  if ((MemorySemantics::Acquire != 0) && (e & MemorySemantics::Acquire)) {
+    nameString += sep + "Acquire";
+    sep = "|";
+  }
+  if (e == MemorySemantics::Release)
+    return "Release";
+  if ((MemorySemantics::Release != 0) && (e & MemorySemantics::Release)) {
+    nameString += sep + "Release";
+    sep = "|";
+  }
+  if (e == MemorySemantics::AcquireRelease)
+    return "AcquireRelease";
+  if ((MemorySemantics::AcquireRelease != 0) &&
+      (e & MemorySemantics::AcquireRelease)) {
+    nameString += sep + "AcquireRelease";
+    sep = "|";
+  }
+  if (e == MemorySemantics::SequentiallyConsistent)
+    return "SequentiallyConsistent";
+  if ((MemorySemantics::SequentiallyConsistent != 0) &&
+      (e & MemorySemantics::SequentiallyConsistent)) {
+    nameString += sep + "SequentiallyConsistent";
+    sep = "|";
+  }
+  if (e == MemorySemantics::UniformMemory)
+    return "UniformMemory";
+  if ((MemorySemantics::UniformMemory != 0) &&
+      (e & MemorySemantics::UniformMemory)) {
+    nameString += sep + "UniformMemory";
+    sep = "|";
+  }
+  if (e == MemorySemantics::SubgroupMemory)
+    return "SubgroupMemory";
+  if ((MemorySemantics::SubgroupMemory != 0) &&
+      (e & MemorySemantics::SubgroupMemory)) {
+    nameString += sep + "SubgroupMemory";
+    sep = "|";
+  }
+  if (e == MemorySemantics::WorkgroupMemory)
+    return "WorkgroupMemory";
+  if ((MemorySemantics::WorkgroupMemory != 0) &&
+      (e & MemorySemantics::WorkgroupMemory)) {
+    nameString += sep + "WorkgroupMemory";
+    sep = "|";
+  }
+  if (e == MemorySemantics::CrossWorkgroupMemory)
+    return "CrossWorkgroupMemory";
+  if ((MemorySemantics::CrossWorkgroupMemory != 0) &&
+      (e & MemorySemantics::CrossWorkgroupMemory)) {
+    nameString += sep + "CrossWorkgroupMemory";
+    sep = "|";
+  }
+  if (e == MemorySemantics::AtomicCounterMemory)
+    return "AtomicCounterMemory";
+  if ((MemorySemantics::AtomicCounterMemory != 0) &&
+      (e & MemorySemantics::AtomicCounterMemory)) {
+    nameString += sep + "AtomicCounterMemory";
+    sep = "|";
+  }
+  if (e == MemorySemantics::ImageMemory)
+    return "ImageMemory";
+  if ((MemorySemantics::ImageMemory != 0) &&
+      (e & MemorySemantics::ImageMemory)) {
+    nameString += sep + "ImageMemory";
+    sep = "|";
+  }
+  if (e == MemorySemantics::OutputMemoryKHR)
+    return "OutputMemoryKHR";
+  if ((MemorySemantics::OutputMemoryKHR != 0) &&
+      (e & MemorySemantics::OutputMemoryKHR)) {
+    nameString += sep + "OutputMemoryKHR";
+    sep = "|";
+  }
+  if (e == MemorySemantics::MakeAvailableKHR)
+    return "MakeAvailableKHR";
+  if ((MemorySemantics::MakeAvailableKHR != 0) &&
+      (e & MemorySemantics::MakeAvailableKHR)) {
+    nameString += sep + "MakeAvailableKHR";
+    sep = "|";
+  }
+  if (e == MemorySemantics::MakeVisibleKHR)
+    return "MakeVisibleKHR";
+  if ((MemorySemantics::MakeVisibleKHR != 0) &&
+      (e & MemorySemantics::MakeVisibleKHR)) {
+    nameString += sep + "MakeVisibleKHR";
+    sep = "|";
+  };
+  return nameString;
+}
+
+std::string SPIRV::getMemoryOperandName(MemoryOperand::MemoryOperand e) {
+  std::string nameString = "";
+  std::string sep = "";
+  if (e == MemoryOperand::None)
+    return "None";
+  if ((MemoryOperand::None != 0) && (e & MemoryOperand::None)) {
+    nameString += sep + "None";
+    sep = "|";
+  }
+  if (e == MemoryOperand::Volatile)
+    return "Volatile";
+  if ((MemoryOperand::Volatile != 0) && (e & MemoryOperand::Volatile)) {
+    nameString += sep + "Volatile";
+    sep = "|";
+  }
+  if (e == MemoryOperand::Aligned)
+    return "Aligned";
+  if ((MemoryOperand::Aligned != 0) && (e & MemoryOperand::Aligned)) {
+    nameString += sep + "Aligned";
+    sep = "|";
+  }
+  if (e == MemoryOperand::Nontemporal)
+    return "Nontemporal";
+  if ((MemoryOperand::Nontemporal != 0) && (e & MemoryOperand::Nontemporal)) {
+    nameString += sep + "Nontemporal";
+    sep = "|";
+  }
+  if (e == MemoryOperand::MakePointerAvailableKHR)
+    return "MakePointerAvailableKHR";
+  if ((MemoryOperand::MakePointerAvailableKHR != 0) &&
+      (e & MemoryOperand::MakePointerAvailableKHR)) {
+    nameString += sep + "MakePointerAvailableKHR";
+    sep = "|";
+  }
+  if (e == MemoryOperand::MakePointerVisibleKHR)
+    return "MakePointerVisibleKHR";
+  if ((MemoryOperand::MakePointerVisibleKHR != 0) &&
+      (e & MemoryOperand::MakePointerVisibleKHR)) {
+    nameString += sep + "MakePointerVisibleKHR";
+    sep = "|";
+  }
+  if (e == MemoryOperand::NonPrivatePointerKHR)
+    return "NonPrivatePointerKHR";
+  if ((MemoryOperand::NonPrivatePointerKHR != 0) &&
+      (e & MemoryOperand::NonPrivatePointerKHR)) {
+    nameString += sep + "NonPrivatePointerKHR";
+    sep = "|";
+  };
+  return nameString;
+}
+
+StringRef SPIRV::getScopeName(Scope::Scope e) {
+  switch (e) {
+  case Scope::CrossDevice:
+    return "CrossDevice";
+  case Scope::Device:
+    return "Device";
+  case Scope::Workgroup:
+    return "Workgroup";
+  case Scope::Subgroup:
+    return "Subgroup";
+  case Scope::Invocation:
+    return "Invocation";
+  case Scope::QueueFamilyKHR:
+    return "QueueFamilyKHR";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef SPIRV::getGroupOperationName(GroupOperation::GroupOperation e) {
+  switch (e) {
+  case GroupOperation::Reduce:
+    return "Reduce";
+  case GroupOperation::InclusiveScan:
+    return "InclusiveScan";
+  case GroupOperation::ExclusiveScan:
+    return "ExclusiveScan";
+  case GroupOperation::ClusteredReduce:
+    return "ClusteredReduce";
+  case GroupOperation::PartitionedReduceNV:
+    return "PartitionedReduceNV";
+  case GroupOperation::PartitionedInclusiveScanNV:
+    return "PartitionedInclusiveScanNV";
+  case GroupOperation::PartitionedExclusiveScanNV:
+    return "PartitionedExclusiveScanNV";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+StringRef
+SPIRV::getKernelEnqueueFlagsName(KernelEnqueueFlags::KernelEnqueueFlags e) {
+  switch (e) {
+  case KernelEnqueueFlags::NoWait:
+    return "NoWait";
+  case KernelEnqueueFlags::WaitKernel:
+    return "WaitKernel";
+  case KernelEnqueueFlags::WaitWorkGroup:
+    return "WaitWorkGroup";
+  }
+  return "UNKNOWN_ENUM";
+}
+
+std::string
+SPIRV::getKernelProfilingInfoName(KernelProfilingInfo::KernelProfilingInfo e) {
+  std::string nameString = "";
+  std::string sep = "";
+  if (e == KernelProfilingInfo::None)
+    return "None";
+  if ((KernelProfilingInfo::None != 0) && (e & KernelProfilingInfo::None)) {
+    nameString += sep + "None";
+    sep = "|";
+  }
+  if (e == KernelProfilingInfo::CmdExecTime)
+    return "CmdExecTime";
+  if ((KernelProfilingInfo::CmdExecTime != 0) &&
+      (e & KernelProfilingInfo::CmdExecTime)) {
+    nameString += sep + "CmdExecTime";
+    sep = "|";
+  };
+  return nameString;
+}

llvm/lib/Target/SPIRV/SPIRV.h

 #include "MCTargetDesc/SPIRVMCTargetDesc.h"
 #include "llvm/CodeGen/MachineFunctionPass.h"
 #include "llvm/Target/TargetMachine.h"
 
 namespace llvm {
 class SPIRVTargetMachine;
 class SPIRVRegisterBankInfo;
 class SPIRVSubtarget;
+class InstructionSelector;
+
+InstructionSelector *
+createSPIRVInstructionSelector(const SPIRVTargetMachine &TM,
+                               const SPIRVSubtarget &Subtarget,
+                               const SPIRVRegisterBankInfo &RBI);
 } // namespace llvm
 
 #endif // LLVM_LIB_TARGET_SPIRV_SPIRV_H

llvm/lib/Target/SPIRV/SPIRVCallLowering.h

 //===--- SPIRVCallLowering.h - Call lowering --------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file describes how to lower LLVM calls to machine code calls.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_LIB_TARGET_SPIRV_SPIRVCALLLOWERING_H
 #define LLVM_LIB_TARGET_SPIRV_SPIRVCALLLOWERING_H
 
+#include "SPIRVGlobalRegistry.h"
 #include "llvm/CodeGen/GlobalISel/CallLowering.h"
 
 namespace llvm {
 
 class SPIRVTargetLowering;
 
 class SPIRVCallLowering : public CallLowering {
 private:
+  // Used to create and assign function, argument, and return type information
+  SPIRVGlobalRegistry *GR;
+
 public:
-  SPIRVCallLowering(const SPIRVTargetLowering &TLI);
+  SPIRVCallLowering(const SPIRVTargetLowering &TLI, SPIRVGlobalRegistry *GR);
 
   // Built OpReturn or OpReturnValue
   bool lowerReturn(MachineIRBuilder &MIRBuiler, const Value *Val,
                    ArrayRef<Register> VRegs, FunctionLoweringInfo &FLI,
                    Register SwiftErrorVReg) const override;
 
   // Build OpFunction, OpFunctionParameter, and any EntryPoint or Linkage data
   bool lowerFormalArguments(MachineIRBuilder &MIRBuilder, const Function &F,

llvm/lib/Target/SPIRV/SPIRVCallLowering.cpp

 //===--- SPIRVCallLowering.cpp - Call lowering ------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the lowering of LLVM calls to machine code calls for
 // GlobalISel.
 //
 //===----------------------------------------------------------------------===//
 
 #include "SPIRVCallLowering.h"
+#include "MCTargetDesc/SPIRVBaseInfo.h"
 #include "SPIRV.h"
+#include "SPIRVGlobalRegistry.h"
 #include "SPIRVISelLowering.h"
 #include "SPIRVRegisterInfo.h"
 #include "SPIRVSubtarget.h"
+#include "SPIRVUtils.h"
 #include "llvm/CodeGen/FunctionLoweringInfo.h"
-#include "llvm/CodeGen/GlobalISel/MachineIRBuilder.h"
 
 using namespace llvm;
 
-SPIRVCallLowering::SPIRVCallLowering(const SPIRVTargetLowering &TLI)
-    : CallLowering(&TLI) {}
+SPIRVCallLowering::SPIRVCallLowering(const SPIRVTargetLowering &TLI,
+                                     SPIRVGlobalRegistry *GR)
+    : CallLowering(&TLI), GR(GR) {}
 
 bool SPIRVCallLowering::lowerReturn(MachineIRBuilder &MIRBuilder,
                                     const Value *Val, ArrayRef<Register> VRegs,
                                     FunctionLoweringInfo &FLI,
                                     Register SwiftErrorVReg) const {
   // TODO: handle the case of multiple registers.
   assert(VRegs.size() < 2 && "All return types should use a single register");
   if (Val) {
-    MIRBuilder.buildInstr(SPIRV::OpReturnValue).addUse(VRegs[0]);
-    return true;
+    const auto &STI = MIRBuilder.getMF().getSubtarget();
+    return MIRBuilder.buildInstr(SPIRV::OpReturnValue)
+        .addUse(VRegs[0])
+        .constrainAllUses(MIRBuilder.getTII(), *STI.getRegisterInfo(),
+                          *STI.getRegBankInfo());
   }
   MIRBuilder.buildInstr(SPIRV::OpReturn);
   return true;
 }
 
 // Based on the LLVM function attributes, get a SPIR-V FunctionControl
 static uint32_t getFunctionControl(const Function &F) {
-  uint32_t FuncControl = 0;
+  uint32_t FuncControl = SPIRV::FunctionControl::None;
+  if (F.hasFnAttribute(Attribute::AttrKind::AlwaysInline)) {
+    FuncControl |= SPIRV::FunctionControl::Inline;
+  }
+  if (F.hasFnAttribute(Attribute::AttrKind::ReadNone)) {
+    FuncControl |= SPIRV::FunctionControl::Pure;
+  }
+  if (F.hasFnAttribute(Attribute::AttrKind::ReadOnly)) {
+    FuncControl |= SPIRV::FunctionControl::Const;
+  }
+  if (F.hasFnAttribute(Attribute::AttrKind::NoInline)) {
+    FuncControl |= SPIRV::FunctionControl::DontInline;
+  }
   return FuncControl;
 }
 
 bool SPIRVCallLowering::lowerFormalArguments(MachineIRBuilder &MIRBuilder,
                                              const Function &F,
                                              ArrayRef<ArrayRef<Register>> VRegs,
                                              FunctionLoweringInfo &FLI) const {
-  auto MRI = MIRBuilder.getMRI();
+  assert(GR && "Must initialize the SPIRV type registry before lowering args.");
+
   // Assign types and names to all args, and store their types for later
   SmallVector<Register, 4> ArgTypeVRegs;
   if (VRegs.size() > 0) {
     unsigned int i = 0;
     for (const auto &Arg : F.args()) {
       // TODO: handle the case of multiple registers.
       assert(VRegs[i].size() == 1 && "Formal arg has multiple vregs");
-      ArgTypeVRegs.push_back(
-          MRI->createGenericVirtualRegister(LLT::scalar(32)));
+      auto *SpirvTy =
+          GR->assignTypeToVReg(Arg.getType(), VRegs[i][0], MIRBuilder);
+      ArgTypeVRegs.push_back(GR->getSPIRVTypeID(SpirvTy));
+
+      if (Arg.hasName())
+        buildOpName(VRegs[i][0], Arg.getName(), MIRBuilder);
+      if (Arg.getType()->isPointerTy()) {
+        auto DerefBytes = static_cast<unsigned>(Arg.getDereferenceableBytes());
+        if (DerefBytes != 0)
+          buildOpDecorate(VRegs[i][0], MIRBuilder,
+                          SPIRV::Decoration::MaxByteOffset, {DerefBytes});
+      }
+      if (Arg.hasAttribute(Attribute::Alignment)) {
+        auto Alignment = static_cast<unsigned>(
+            Arg.getAttribute(Attribute::Alignment).getValueAsInt());
+        buildOpDecorate(VRegs[i][0], MIRBuilder, SPIRV::Decoration::Alignment,
+                        {Alignment});
+      }
+      if (Arg.hasAttribute(Attribute::ReadOnly)) {
+        buildOpDecorate(VRegs[i][0], MIRBuilder,
+                        SPIRV::Decoration::FuncParamAttr,
+                        {SPIRV::FunctionParameterAttribute::NoWrite});
+      }
+      if (Arg.hasAttribute(Attribute::ZExt)) {
+        buildOpDecorate(VRegs[i][0], MIRBuilder,
+                        SPIRV::Decoration::FuncParamAttr,
+                        {SPIRV::FunctionParameterAttribute::Zext});
+      }
       ++i;
     }
   }
 
   // Generate a SPIR-V type for the function
+  auto MRI = MIRBuilder.getMRI();
   Register FuncVReg = MRI->createGenericVirtualRegister(LLT::scalar(32));
   MRI->setRegClass(FuncVReg, &SPIRV::IDRegClass);
 
+  auto *FTy = F.getFunctionType();
+  auto FuncTy = GR->assignTypeToVReg(FTy, FuncVReg, MIRBuilder);
+
+  // Build the OpTypeFunction declaring it
+  Register ReturnTypeID = FuncTy->getOperand(1).getReg();
   uint32_t FuncControl = getFunctionControl(F);
 
   MIRBuilder.buildInstr(SPIRV::OpFunction)
       .addDef(FuncVReg)
-      .addUse(MRI->createGenericVirtualRegister(LLT::scalar(32)))
+      .addUse(ReturnTypeID)
       .addImm(FuncControl)
-      .addUse(MRI->createGenericVirtualRegister(LLT::scalar(32)));
+      .addUse(GR->getSPIRVTypeID(FuncTy));
 
   // Add OpFunctionParameters
   const unsigned int NumArgs = ArgTypeVRegs.size();
   for (unsigned int i = 0; i < NumArgs; ++i) {
     assert(VRegs[i].size() == 1 && "Formal arg has multiple vregs");
     MRI->setRegClass(VRegs[i][0], &SPIRV::IDRegClass);
     MIRBuilder.buildInstr(SPIRV::OpFunctionParameter)
         .addDef(VRegs[i][0])
         .addUse(ArgTypeVRegs[i]);
   }
+  // Name the function
+  if (F.hasName())
+    buildOpName(FuncVReg, F.getName(), MIRBuilder);
+
+  // Handle entry points and function linkage
+  if (F.getCallingConv() == CallingConv::SPIR_KERNEL) {
+    auto ExecModel = SPIRV::ExecutionModel::Kernel;
+    auto MIB = MIRBuilder.buildInstr(SPIRV::OpEntryPoint)
+                   .addImm(ExecModel)
+                   .addUse(FuncVReg);
+    addStringImm(F.getName(), MIB);
+  } else if (F.getLinkage() == GlobalValue::LinkageTypes::ExternalLinkage ||
+             F.getLinkage() == GlobalValue::LinkOnceODRLinkage) {
+    auto LnkTy = F.isDeclaration() ? SPIRV::LinkageType::Import
+                                   : SPIRV::LinkageType::Export;
+    buildOpDecorate(FuncVReg, MIRBuilder, SPIRV::Decoration::LinkageAttributes,
+                    {LnkTy}, F.getGlobalIdentifier());
+  }
+
   return true;
 }
 
 bool SPIRVCallLowering::lowerCall(MachineIRBuilder &MIRBuilder,
                                   CallLoweringInfo &Info) const {
+  auto &MF = MIRBuilder.getMF();
   // TODO: handle the case of multiple registers.
   assert(Info.OrigRet.Regs.size() < 2 && "Call returns multiple vregs");
 
   Register ResVReg =
       Info.OrigRet.Regs.empty() ? Register(0) : Info.OrigRet.Regs[0];
+  // Emit a regular OpFunctionCall. If it's an externally declared function,
+  // be sure to emit its type and function declaration here. It will be
+  // hoisted globally later.
+  if (Info.Callee.isGlobal()) {
+    const Function *CF = cast<const Function>(Info.Callee.getGlobal());
+    if (CF->isDeclaration()) {
+      // Emit the type info and forward function declaration to the first MBB
+      // to ensure VReg definition dependencies are valid across all MBBs.
+      MachineIRBuilder FirstBlockBuilder;
+      FirstBlockBuilder.setMF(MF);
+      FirstBlockBuilder.setMBB(*MF.getBlockNumbered(0));
+
+      SmallVector<ArrayRef<Register>, 8> VRegArgs;
+      SmallVector<SmallVector<Register, 1>, 8> ToInsert;
+      for (const Argument &Arg : CF->args()) {
+        if (MIRBuilder.getDataLayout().getTypeStoreSize(Arg.getType()).isZero())
+          continue; // Don't handle zero sized types.
+        ToInsert.push_back({MIRBuilder.getMRI()->createGenericVirtualRegister(
+            LLT::scalar(32))});
+        VRegArgs.push_back(ToInsert.back());
+      }
+      // TODO: Reuse FunctionLoweringInfo
+      FunctionLoweringInfo FuncInfo;
+      lowerFormalArguments(FirstBlockBuilder, *CF, VRegArgs, FuncInfo);
+    }
+  }
+
   // Make sure there's a valid return reg, even for functions returning void
   if (!ResVReg.isValid()) {
     ResVReg = MIRBuilder.getMRI()->createVirtualRegister(&SPIRV::IDRegClass);
   }
+  SPIRVType *RetType =
+      GR->assignTypeToVReg(Info.OrigRet.Ty, ResVReg, MIRBuilder);
+
   // Emit the OpFunctionCall and its args
   auto MIB = MIRBuilder.buildInstr(SPIRV::OpFunctionCall)
                  .addDef(ResVReg)
-                 .addUse(MIRBuilder.getMRI()->createVirtualRegister(
-                     &SPIRV::IDRegClass))
+                 .addUse(GR->getSPIRVTypeID(RetType))
                  .add(Info.Callee);
 
   for (const auto &arg : Info.OrigArgs) {
     assert(arg.Regs.size() == 1 && "Call arg has multiple VRegs");
     MIB.addUse(arg.Regs[0]);
   }
-  return true;
+  const auto &STI = MF.getSubtarget();
+  return MIB.constrainAllUses(MIRBuilder.getTII(), *STI.getRegisterInfo(),
+                              *STI.getRegBankInfo());
 }

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.h

+//===-- SPIRVGlobalRegistry.h - SPIR-V Global Registry ----------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// SPIRVGlobalRegistry is used to maintain rich type information required for
+// SPIR-V even after lowering from LLVM IR to GMIR. It can convert an llvm::Type
+// into an OpTypeXXX instruction, and map it to a virtual register. Also it
+// builds and supports consistency of constants and global variables.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_TARGET_SPIRV_SPIRVTYPEMANAGER_H
+#define LLVM_LIB_TARGET_SPIRV_SPIRVTYPEMANAGER_H
+
+#include "MCTargetDesc/SPIRVBaseInfo.h"
+#include "llvm/CodeGen/GlobalISel/MachineIRBuilder.h"
+
+namespace AQ = llvm::SPIRV::AccessQualifier;
+
+namespace llvm {
+using SPIRVType = const MachineInstr;
+
+class SPIRVGlobalRegistry {
+  // Registers holding values which have types associated with them.
+  // Initialized upon VReg definition in IRTranslator.
+  // Do not confuse this with DuplicatesTracker as DT maps Type* to <MF, Reg>
+  // where Reg = OpType...
+  // while VRegToTypeMap tracks SPIR-V type assigned to other regs (i.e. not
+  // type-declaring ones)
+  DenseMap<MachineFunction *, DenseMap<Register, SPIRVType *>> VRegToTypeMap;
+
+  DenseMap<SPIRVType *, const Type *> SPIRVToLLVMType;
+
+  // Number of bits pointers and size_t integers require.
+  const unsigned int PointerSize;
+
+  // Add a new OpTypeXXX instruction without checking for duplicates
+  SPIRVType *createSPIRVType(const Type *Type, MachineIRBuilder &MIRBuilder,
+                             AQ::AccessQualifier accessQual = AQ::ReadWrite,
+                             bool EmitIR = true);
+
+public:
+  SPIRVGlobalRegistry(unsigned int PointerSize);
+
+  MachineFunction *CurMF;
+
+  // Get or create a SPIR-V type corresponding the given LLVM IR type,
+  // and map it to the given VReg by creating an ASSIGN_TYPE instruction.
+  SPIRVType *assignTypeToVReg(const Type *Type, Register VReg,
+                              MachineIRBuilder &MIRBuilder,
+                              AQ::AccessQualifier AccessQual = AQ::ReadWrite,
+                              bool EmitIR = true);
+
+  // In cases where the SPIR-V type is already known, this function can be
+  // used to map it to the given VReg via an ASSIGN_TYPE instruction.
+  void assignSPIRVTypeToVReg(SPIRVType *Type, Register VReg,
+                             MachineIRBuilder &MIRBuilder);
+
+  // Either generate a new OpTypeXXX instruction or return an existing one
+  // corresponding to the given LLVM IR type.
+  // EmitIR controls if we emit GMIR or SPV constants (e.g. for array sizes)
+  // because this method may be called from InstructionSelector and we don't
+  // want to emit extra IR instructions there
+  SPIRVType *
+  getOrCreateSPIRVType(const Type *Type, MachineIRBuilder &MIRBuilder,
+                       AQ::AccessQualifier accessQual = AQ::ReadWrite,
+                       bool EmitIR = true);
+
+  const Type *getTypeForSPIRVType(const SPIRVType *Ty) const {
+    auto Res = SPIRVToLLVMType.find(Ty);
+    assert(Res != SPIRVToLLVMType.end());
+    return Res->second;
+  }
+
+  // Return the SPIR-V type instruction corresponding to the given VReg, or
+  // nullptr if no such type instruction exists.
+  SPIRVType *getSPIRVTypeForVReg(Register VReg) const;
+
+  // Whether the given VReg has a SPIR-V type mapped to it yet.
+  bool hasSPIRVTypeForVReg(Register VReg) const {
+    return getSPIRVTypeForVReg(VReg) != nullptr;
+  }
+
+  // Return the VReg holding the result of the given OpTypeXXX instruction
+  Register getSPIRVTypeID(const SPIRVType *SpirvType) const {
+    assert(SpirvType && "Attempting to get type id for nullptr type.");
+    return SpirvType->defs().begin()->getReg();
+  }
+
+  void setCurrentFunc(MachineFunction &MF) { CurMF = &MF; }
+
+  // Whether the given VReg has an OpTypeXXX instruction mapped to it with the
+  // given opcode (e.g. OpTypeFloat).
+  bool isScalarOfType(Register VReg, unsigned int TypeOpcode) const;
+
+  // Return true if the given VReg's assigned SPIR-V type is either a scalar
+  // matching the given opcode, or a vector with an element type matching that
+  // opcode (e.g. OpTypeBool, or OpTypeVector %x 4, where %x is OpTypeBool).
+  bool isScalarOrVectorOfType(Register VReg, unsigned int TypeOpcode) const;
+
+  // For vectors or scalars of ints or floats, return the scalar type's bitwidth
+  unsigned getScalarOrVectorBitWidth(const SPIRVType *Type) const;
+
+  // For integer vectors or scalars, return whether the integers are signed
+  bool isScalarOrVectorSigned(const SPIRVType *Type) const;
+
+  // Gets the storage class of the pointer type assigned to this vreg
+  SPIRV::StorageClass::StorageClass getPointerStorageClass(Register VReg) const;
+
+  // Return the number of bits SPIR-V pointers and size_t variables require.
+  unsigned getPointerSize() const { return PointerSize; }
+
+private:
+  SPIRVType *getOpTypeBool(MachineIRBuilder &MIRBuilder);
+
+  SPIRVType *getOpTypeInt(uint32_t Width, MachineIRBuilder &MIRBuilder,
+                          bool IsSigned = false);
+
+  SPIRVType *getOpTypeFloat(uint32_t Width, MachineIRBuilder &MIRBuilder);
+
+  SPIRVType *getOpTypeVoid(MachineIRBuilder &MIRBuilder);
+
+  SPIRVType *getOpTypeVector(uint32_t NumElems, SPIRVType *ElemType,
+                             MachineIRBuilder &MIRBuilder);
+
+  SPIRVType *getOpTypeArray(uint32_t NumElems, SPIRVType *ElemType,
+                            MachineIRBuilder &MIRBuilder, bool EmitIR = true);
+
+  SPIRVType *getOpTypePointer(SPIRV::StorageClass::StorageClass SC,
+                              SPIRVType *ElemType,
+                              MachineIRBuilder &MIRBuilder);
+
+  SPIRVType *getOpTypeFunction(SPIRVType *RetType,
+                               const SmallVectorImpl<SPIRVType *> &ArgTypes,
+                               MachineIRBuilder &MIRBuilder);
+
+public:
+  Register buildConstantInt(uint64_t Val, MachineIRBuilder &MIRBuilder,
+                            SPIRVType *SpvType = nullptr, bool EmitIR = true);
+  Register buildConstantFP(APFloat Val, MachineIRBuilder &MIRBuilder,
+                           SPIRVType *SpvType = nullptr);
+  Register buildGlobalVariable(Register Reg, SPIRVType *BaseType,
+                               StringRef Name, const GlobalValue *GV,
+                               SPIRV::StorageClass::StorageClass Storage,
+                               const MachineInstr *Init, bool IsConst,
+                               bool HasLinkageTy,
+                               SPIRV::LinkageType::LinkageType LinkageType,
+                               MachineIRBuilder &MIRBuilder,
+                               bool IsInstSelector);
+
+  // Convenient helpers for getting types with check for duplicates.
+  SPIRVType *getOrCreateSPIRVIntegerType(unsigned BitWidth,
+                                         MachineIRBuilder &MIRBuilder);
+  SPIRVType *getOrCreateSPIRVBoolType(MachineIRBuilder &MIRBuilder);
+  SPIRVType *getOrCreateSPIRVVectorType(SPIRVType *BaseType,
+                                        unsigned NumElements,
+                                        MachineIRBuilder &MIRBuilder);
+  SPIRVType *getOrCreateSPIRVPointerType(
+      SPIRVType *BaseType, MachineIRBuilder &MIRBuilder,
+      SPIRV::StorageClass::StorageClass SClass = SPIRV::StorageClass::Function);
+};
+} // end namespace llvm
+#endif // LLLVM_LIB_TARGET_SPIRV_SPIRVTYPEMANAGER_H

llvm/lib/Target/SPIRV/SPIRVGlobalRegistry.cpp

+//===-- SPIRVGlobalRegistry.cpp - SPIR-V Global Registry --------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains the implementation of the SPIRVGlobalRegistry class,
+// which is used to maintain rich type information required for SPIR-V even
+// after lowering from LLVM IR to GMIR. It can convert an llvm::Type into
+// an OpTypeXXX instruction, and map it to a virtual register. Also it builds
+// and supports consistency of constants and global variables.
+//
+//===----------------------------------------------------------------------===//
+
+#include "SPIRVGlobalRegistry.h"
+#include "SPIRV.h"
+#include "SPIRVSubtarget.h"
+#include "SPIRVTargetMachine.h"
+#include "SPIRVUtils.h"
+
+using namespace llvm;
+SPIRVGlobalRegistry::SPIRVGlobalRegistry(unsigned int PointerSize)
+    : PointerSize(PointerSize) {}
+
+SPIRVType *SPIRVGlobalRegistry::assignTypeToVReg(const Type *Type,
+                                                 Register VReg,
+                                                 MachineIRBuilder &MIRBuilder,
+                                                 AQ::AccessQualifier AccessQual,
+                                                 bool EmitIR) {
+
+  SPIRVType *SpirvType =
+      getOrCreateSPIRVType(Type, MIRBuilder, AccessQual, EmitIR);
+  assignSPIRVTypeToVReg(SpirvType, VReg, MIRBuilder);
+  return SpirvType;
+}
+
+void SPIRVGlobalRegistry::assignSPIRVTypeToVReg(SPIRVType *SpirvType,
+                                                Register VReg,
+                                                MachineIRBuilder &MIRBuilder) {
+  VRegToTypeMap[&MIRBuilder.getMF()][VReg] = SpirvType;
+}
+
+static Register createTypeVReg(MachineIRBuilder &MIRBuilder) {
+  auto &MRI = MIRBuilder.getMF().getRegInfo();
+  auto Res = MRI.createGenericVirtualRegister(LLT::scalar(32));
+  MRI.setRegClass(Res, &SPIRV::TYPERegClass);
+  return Res;
+}
+
+SPIRVType *SPIRVGlobalRegistry::getOpTypeBool(MachineIRBuilder &MIRBuilder) {
+  return MIRBuilder.buildInstr(SPIRV::OpTypeBool)
+      .addDef(createTypeVReg(MIRBuilder));
+}
+
+SPIRVType *SPIRVGlobalRegistry::getOpTypeInt(uint32_t Width,
+                                             MachineIRBuilder &MIRBuilder,
+                                             bool IsSigned) {
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpTypeInt)
+                 .addDef(createTypeVReg(MIRBuilder))
+                 .addImm(Width)
+                 .addImm(IsSigned ? 1 : 0);
+  return MIB;
+}
+
+SPIRVType *SPIRVGlobalRegistry::getOpTypeFloat(uint32_t Width,
+                                               MachineIRBuilder &MIRBuilder) {
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpTypeFloat)
+                 .addDef(createTypeVReg(MIRBuilder))
+                 .addImm(Width);
+  return MIB;
+}
+
+SPIRVType *SPIRVGlobalRegistry::getOpTypeVoid(MachineIRBuilder &MIRBuilder) {
+  return MIRBuilder.buildInstr(SPIRV::OpTypeVoid)
+      .addDef(createTypeVReg(MIRBuilder));
+}
+
+SPIRVType *SPIRVGlobalRegistry::getOpTypeVector(uint32_t NumElems,
+                                                SPIRVType *ElemType,
+                                                MachineIRBuilder &MIRBuilder) {
+  auto EleOpc = ElemType->getOpcode();
+  assert((EleOpc == SPIRV::OpTypeInt || EleOpc == SPIRV::OpTypeFloat ||
+          EleOpc == SPIRV::OpTypeBool) &&
+         "Invalid vector element type");
+
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpTypeVector)
+                 .addDef(createTypeVReg(MIRBuilder))
+                 .addUse(getSPIRVTypeID(ElemType))
+                 .addImm(NumElems);
+  return MIB;
+}
+
+Register SPIRVGlobalRegistry::buildConstantInt(uint64_t Val,
+                                               MachineIRBuilder &MIRBuilder,
+                                               SPIRVType *SpvType,
+                                               bool EmitIR) {
+  auto &MF = MIRBuilder.getMF();
+  Register Res;
+  const IntegerType *LLVMIntTy;
+  if (SpvType)
+    LLVMIntTy = cast<IntegerType>(getTypeForSPIRVType(SpvType));
+  else
+    LLVMIntTy = IntegerType::getInt32Ty(MF.getFunction().getContext());
+  // Find a constant in DT or build a new one.
+  const auto ConstInt =
+      ConstantInt::get(const_cast<IntegerType *>(LLVMIntTy), Val);
+  unsigned BitWidth = SpvType ? getScalarOrVectorBitWidth(SpvType) : 32;
+  Res = MF.getRegInfo().createGenericVirtualRegister(LLT::scalar(BitWidth));
+  assignTypeToVReg(LLVMIntTy, Res, MIRBuilder);
+  if (EmitIR)
+    MIRBuilder.buildConstant(Res, *ConstInt);
+  else
+    MIRBuilder.buildInstr(SPIRV::OpConstantI)
+        .addDef(Res)
+        .addImm(ConstInt->getSExtValue());
+  return Res;
+}
+
+Register SPIRVGlobalRegistry::buildConstantFP(APFloat Val,
+                                              MachineIRBuilder &MIRBuilder,
+                                              SPIRVType *SpvType) {
+  auto &MF = MIRBuilder.getMF();
+  Register Res;
+  const Type *LLVMFPTy;
+  if (SpvType) {
+    LLVMFPTy = getTypeForSPIRVType(SpvType);
+    assert(LLVMFPTy->isFloatingPointTy());
+  } else {
+    LLVMFPTy = IntegerType::getFloatTy(MF.getFunction().getContext());
+  }
+  // Find a constant in DT or build a new one.
+  const auto ConstFP = ConstantFP::get(LLVMFPTy->getContext(), Val);
+  unsigned BitWidth = SpvType ? getScalarOrVectorBitWidth(SpvType) : 32;
+  Res = MF.getRegInfo().createGenericVirtualRegister(LLT::scalar(BitWidth));
+  assignTypeToVReg(LLVMFPTy, Res, MIRBuilder);
+  MIRBuilder.buildFConstant(Res, *ConstFP);
+  return Res;
+}
+
+Register SPIRVGlobalRegistry::buildGlobalVariable(
+    Register ResVReg, SPIRVType *BaseType, StringRef Name,
+    const GlobalValue *GV, SPIRV::StorageClass::StorageClass Storage,
+    const MachineInstr *Init, bool IsConst, bool HasLinkageTy,
+    SPIRV::LinkageType::LinkageType LinkageType, MachineIRBuilder &MIRBuilder,
+    bool IsInstSelector) {
+  const GlobalVariable *GVar = nullptr;
+  if (GV)
+    GVar = cast<const GlobalVariable>(GV);
+  else {
+    // If GV is not passed explicitly, use the name to find or construct
+    // the global variable.
+    Module *M = MIRBuilder.getMF().getFunction().getParent();
+    GVar = M->getGlobalVariable(Name);
+    if (GVar == nullptr) {
+      const Type *Ty = getTypeForSPIRVType(BaseType); // TODO: check type.
+      GVar = new GlobalVariable(*M, const_cast<Type *>(Ty), false,
+          GlobalValue::ExternalLinkage, nullptr, Twine(Name));
+    }
+    GV = GVar;
+  }
+  Register Reg;
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpVariable)
+                 .addDef(ResVReg)
+                 .addUse(getSPIRVTypeID(BaseType))
+                 .addImm(Storage);
+
+  if (Init != 0) {
+    MIB.addUse(Init->getOperand(0).getReg());
+  }
+
+  // ISel may introduce a new register on this step, so we need to add it to
+  // DT and correct its type avoiding fails on the next stage.
+  if (IsInstSelector) {
+    const auto &Subtarget = CurMF->getSubtarget();
+    constrainSelectedInstRegOperands(*MIB, *Subtarget.getInstrInfo(),
+                                     *Subtarget.getRegisterInfo(),
+                                     *Subtarget.getRegBankInfo());
+  }
+  Reg = MIB->getOperand(0).getReg();
+
+  // Set to Reg the same type as ResVReg has.
+  auto MRI = MIRBuilder.getMRI();
+  assert(MRI->getType(ResVReg).isPointer() && "Pointer type is expected");
+  if (Reg != ResVReg) {
+    LLT RegLLTy = LLT::pointer(MRI->getType(ResVReg).getAddressSpace(), 32);
+    MRI->setType(Reg, RegLLTy);
+    assignSPIRVTypeToVReg(BaseType, Reg, MIRBuilder);
+  }
+
+  // If it's a global variable with name, output OpName for it.
+  if (GVar && GVar->hasName())
+    buildOpName(Reg, GVar->getName(), MIRBuilder);
+
+  // Output decorations for the GV.
+  // TODO: maybe move to GenerateDecorations pass.
+  if (IsConst)
+    buildOpDecorate(Reg, MIRBuilder, SPIRV::Decoration::Constant, {});
+
+  if (GVar && GVar->getAlign().valueOrOne().value() != 1) {
+    unsigned Alignment = (unsigned)GVar->getAlign().valueOrOne().value();
+    buildOpDecorate(Reg, MIRBuilder, SPIRV::Decoration::Alignment, {Alignment});
+  }
+
+  if (HasLinkageTy)
+    buildOpDecorate(Reg, MIRBuilder, SPIRV::Decoration::LinkageAttributes,
+                    {LinkageType}, Name);
+  return Reg;
+}
+
+SPIRVType *SPIRVGlobalRegistry::getOpTypeArray(uint32_t NumElems,
+                                               SPIRVType *ElemType,
+                                               MachineIRBuilder &MIRBuilder,
+                                               bool EmitIR) {
+  assert((ElemType->getOpcode() != SPIRV::OpTypeVoid) &&
+         "Invalid array element type");
+  Register NumElementsVReg =
+      buildConstantInt(NumElems, MIRBuilder, nullptr, EmitIR);
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpTypeArray)
+                 .addDef(createTypeVReg(MIRBuilder))
+                 .addUse(getSPIRVTypeID(ElemType))
+                 .addUse(NumElementsVReg);
+  return MIB;
+}
+
+SPIRVType *
+SPIRVGlobalRegistry::getOpTypePointer(SPIRV::StorageClass::StorageClass SC,
+                                      SPIRVType *ElemType,
+                                      MachineIRBuilder &MIRBuilder) {
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpTypePointer)
+                 .addDef(createTypeVReg(MIRBuilder))
+                 .addImm(SC)
+                 .addUse(getSPIRVTypeID(ElemType));
+  return MIB;
+}
+
+SPIRVType *SPIRVGlobalRegistry::getOpTypeFunction(
+    SPIRVType *RetType, const SmallVectorImpl<SPIRVType *> &ArgTypes,
+    MachineIRBuilder &MIRBuilder) {
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpTypeFunction)
+                 .addDef(createTypeVReg(MIRBuilder))
+                 .addUse(getSPIRVTypeID(RetType));
+  for (const SPIRVType *ArgType : ArgTypes)
+    MIB.addUse(getSPIRVTypeID(ArgType));
+  return MIB;
+}
+
+SPIRVType *SPIRVGlobalRegistry::createSPIRVType(const Type *Ty,
+                                                MachineIRBuilder &MIRBuilder,
+                                                AQ::AccessQualifier AccQual,
+                                                bool EmitIR) {
+  if (auto IType = dyn_cast<IntegerType>(Ty)) {
+    const unsigned int Width = IType->getBitWidth();
+    return Width == 1 ? getOpTypeBool(MIRBuilder)
+                      : getOpTypeInt(Width, MIRBuilder, false);
+  }
+  if (Ty->isFloatingPointTy())
+    return getOpTypeFloat(Ty->getPrimitiveSizeInBits(), MIRBuilder);
+  if (Ty->isVoidTy())
+    return getOpTypeVoid(MIRBuilder);
+  if (Ty->isVectorTy()) {
+    auto El = getOrCreateSPIRVType(cast<FixedVectorType>(Ty)->getElementType(),
+                                   MIRBuilder);
+    return getOpTypeVector(cast<FixedVectorType>(Ty)->getNumElements(), El,
+                           MIRBuilder);
+  }
+  if (Ty->isArrayTy()) {
+    auto *El = getOrCreateSPIRVType(Ty->getArrayElementType(), MIRBuilder);
+    return getOpTypeArray(Ty->getArrayNumElements(), El, MIRBuilder, EmitIR);
+  }
+  assert(!isa<StructType>(Ty) && "Unsupported StructType");
+  if (auto FType = dyn_cast<FunctionType>(Ty)) {
+    SPIRVType *RetTy = getOrCreateSPIRVType(FType->getReturnType(), MIRBuilder);
+    SmallVector<SPIRVType *, 4> ParamTypes;
+    for (const auto &t : FType->params()) {
+      ParamTypes.push_back(getOrCreateSPIRVType(t, MIRBuilder));
+    }
+    return getOpTypeFunction(RetTy, ParamTypes, MIRBuilder);
+  }
+  if (auto PType = dyn_cast<PointerType>(Ty)) {
+    Type *ElemType = PType->getPointerElementType();
+
+    // Some OpenCL and SPIRV builtins like image2d_t are passed in as pointers,
+    // but should be treated as custom types like OpTypeImage
+    assert(!isa<StructType>(ElemType) && "Unsupported StructType pointer");
+
+    // Otherwise, treat it as a regular pointer type
+    auto SC = addressSpaceToStorageClass(PType->getAddressSpace());
+    SPIRVType *SpvElementType =
+        getOrCreateSPIRVType(ElemType, MIRBuilder, AQ::ReadWrite, EmitIR);
+    return getOpTypePointer(SC, SpvElementType, MIRBuilder);
+  }
+  llvm_unreachable("Unable to convert LLVM type to SPIRVType");
+}
+
+SPIRVType *SPIRVGlobalRegistry::getSPIRVTypeForVReg(Register VReg) const {
+  auto t = VRegToTypeMap.find(CurMF);
+  if (t != VRegToTypeMap.end()) {
+    auto tt = t->second.find(VReg);
+    if (tt != t->second.end())
+      return tt->second;
+  }
+  return nullptr;
+}
+
+SPIRVType *SPIRVGlobalRegistry::getOrCreateSPIRVType(
+    const Type *Type, MachineIRBuilder &MIRBuilder,
+    AQ::AccessQualifier AccessQual, bool EmitIR) {
+  Register Reg;
+  SPIRVType *SpirvType = createSPIRVType(Type, MIRBuilder, AccessQual, EmitIR);
+  VRegToTypeMap[&MIRBuilder.getMF()][getSPIRVTypeID(SpirvType)] = SpirvType;
+  SPIRVToLLVMType[SpirvType] = Type;
+  return SpirvType;
+}
+
+bool SPIRVGlobalRegistry::isScalarOfType(Register VReg,
+                                         unsigned int TypeOpcode) const {
+  SPIRVType *Type = getSPIRVTypeForVReg(VReg);
+  assert(Type && "isScalarOfType VReg has no type assigned");
+  return Type->getOpcode() == TypeOpcode;
+}
+
+bool SPIRVGlobalRegistry::isScalarOrVectorOfType(
+    Register VReg, unsigned int TypeOpcode) const {
+  SPIRVType *Type = getSPIRVTypeForVReg(VReg);
+  assert(Type && "isScalarOrVectorOfType VReg has no type assigned");
+  if (Type->getOpcode() == TypeOpcode)
+    return true;
+  if (Type->getOpcode() == SPIRV::OpTypeVector) {
+    Register ScalarTypeVReg = Type->getOperand(1).getReg();
+    SPIRVType *ScalarType = getSPIRVTypeForVReg(ScalarTypeVReg);
+    return ScalarType->getOpcode() == TypeOpcode;
+  }
+  return false;
+}
+
+unsigned
+SPIRVGlobalRegistry::getScalarOrVectorBitWidth(const SPIRVType *Type) const {
+  if (Type && Type->getOpcode() == SPIRV::OpTypeVector) {
+    auto EleTypeReg = Type->getOperand(1).getReg();
+    Type = getSPIRVTypeForVReg(EleTypeReg);
+  }
+  if (Type && (Type->getOpcode() == SPIRV::OpTypeInt ||
+               Type->getOpcode() == SPIRV::OpTypeFloat))
+    return Type->getOperand(1).getImm();
+  if (Type && Type->getOpcode() == SPIRV::OpTypeBool)
+    return 1;
+  llvm_unreachable("Attempting to get bit width of non-integer/float type.");
+}
+
+bool SPIRVGlobalRegistry::isScalarOrVectorSigned(const SPIRVType *Type) const {
+  if (Type && Type->getOpcode() == SPIRV::OpTypeVector) {
+    auto EleTypeReg = Type->getOperand(1).getReg();
+    Type = getSPIRVTypeForVReg(EleTypeReg);
+  }
+  if (Type && Type->getOpcode() == SPIRV::OpTypeInt) {
+    return Type->getOperand(2).getImm() != 0;
+  }
+  llvm_unreachable("Attempting to get sign of non-integer type.");
+}
+
+SPIRV::StorageClass::StorageClass
+SPIRVGlobalRegistry::getPointerStorageClass(Register VReg) const {
+  SPIRVType *Type = getSPIRVTypeForVReg(VReg);
+  if (Type && Type->getOpcode() == SPIRV::OpTypePointer) {
+    auto scOp = Type->getOperand(1).getImm();
+    return static_cast<SPIRV::StorageClass::StorageClass>(scOp);
+  }
+  llvm_unreachable("Attempting to get storage class of non-pointer type.");
+}
+
+SPIRVType *
+SPIRVGlobalRegistry::getOrCreateSPIRVIntegerType(unsigned BitWidth,
+                                                 MachineIRBuilder &MIRBuilder) {
+  return getOrCreateSPIRVType(
+      IntegerType::get(MIRBuilder.getMF().getFunction().getContext(), BitWidth),
+      MIRBuilder);
+}
+
+SPIRVType *
+SPIRVGlobalRegistry::getOrCreateSPIRVBoolType(MachineIRBuilder &MIRBuilder) {
+  return getOrCreateSPIRVType(
+      IntegerType::get(MIRBuilder.getMF().getFunction().getContext(), 1),
+      MIRBuilder);
+}
+
+SPIRVType *SPIRVGlobalRegistry::getOrCreateSPIRVVectorType(
+    SPIRVType *BaseType, unsigned NumElements, MachineIRBuilder &MIRBuilder) {
+  return getOrCreateSPIRVType(
+      FixedVectorType::get(const_cast<Type *>(getTypeForSPIRVType(BaseType)),
+                           NumElements),
+      MIRBuilder);
+}
+
+SPIRVType *SPIRVGlobalRegistry::getOrCreateSPIRVPointerType(
+    SPIRVType *BaseType, MachineIRBuilder &MIRBuilder,
+    SPIRV::StorageClass::StorageClass SClass) {
+  return getOrCreateSPIRVType(
+      PointerType::get(const_cast<Type *>(getTypeForSPIRVType(BaseType)),
+                       storageClassToAddressSpace(SClass)),
+      MIRBuilder);
+}

llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp

+//===- SPIRVInstructionSelector.cpp ------------------------------*- C++ -*-==//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the targeting of the InstructionSelector class for
+// SPIRV.
+// TODO: This should be generated by TableGen.
+//
+//===----------------------------------------------------------------------===//
+
+#include "SPIRV.h"
+#include "SPIRVGlobalRegistry.h"
+#include "SPIRVInstrInfo.h"
+#include "SPIRVRegisterBankInfo.h"
+#include "SPIRVRegisterInfo.h"
+#include "SPIRVTargetMachine.h"
+#include "SPIRVUtils.h"
+#include "llvm/ADT/APFloat.h"
+#include "llvm/CodeGen/GlobalISel/InstructionSelector.h"
+#include "llvm/CodeGen/GlobalISel/InstructionSelectorImpl.h"
+#include "llvm/CodeGen/MachineInstrBuilder.h"
+#include "llvm/CodeGen/MachineRegisterInfo.h"
+#include "llvm/Support/Debug.h"
+
+#define DEBUG_TYPE "spirv-isel"
+
+using namespace llvm;
+
+namespace {
+
+#define GET_GLOBALISEL_PREDICATE_BITSET
+#include "SPIRVGenGlobalISel.inc"
+#undef GET_GLOBALISEL_PREDICATE_BITSET
+
+class SPIRVInstructionSelector : public InstructionSelector {
+  const SPIRVSubtarget &STI;
+  const SPIRVInstrInfo &TII;
+  const SPIRVRegisterInfo &TRI;
+  const SPIRVRegisterBankInfo &RBI;
+  SPIRVGlobalRegistry &GR;
+
+public:
+  SPIRVInstructionSelector(const SPIRVTargetMachine &TM,
+                           const SPIRVSubtarget &ST,
+                           const SPIRVRegisterBankInfo &RBI);
+
+  // Common selection code. Instruction-specific selection occurs in spvSelect()
+  bool select(MachineInstr &I) override;
+  static const char *getName() { return DEBUG_TYPE; }
+
+#define GET_GLOBALISEL_PREDICATES_DECL
+#include "SPIRVGenGlobalISel.inc"
+#undef GET_GLOBALISEL_PREDICATES_DECL
+
+#define GET_GLOBALISEL_TEMPORARIES_DECL
+#include "SPIRVGenGlobalISel.inc"
+#undef GET_GLOBALISEL_TEMPORARIES_DECL
+
+private:
+  // tblgen-erated 'select' implementation, used as the initial selector for
+  // the patterns that don't require complex C++.
+  bool selectImpl(MachineInstr &I, CodeGenCoverage &CoverageInfo) const;
+
+  // All instruction-specific selection that didn't happen in "select()".
+  // Is basically a large Switch/Case delegating to all other select method.
+  bool spvSelect(Register ResVReg, const SPIRVType *ResType,
+                 const MachineInstr &I, MachineIRBuilder &MIRBuilder) const;
+
+  bool selectGlobalValue(Register ResVReg, const MachineInstr &I,
+                         MachineIRBuilder &MIRBuilder,
+                         const MachineInstr *Init = nullptr) const;
+
+  bool selectUnOpWithSrc(Register ResVReg, const SPIRVType *ResType,
+                         const MachineInstr &I, Register SrcReg,
+                         MachineIRBuilder &MIRBuilder, unsigned Opcode) const;
+  bool selectUnOp(Register ResVReg, const SPIRVType *ResType,
+                  const MachineInstr &I, MachineIRBuilder &MIRBuilder,
+                  unsigned Opcode) const;
+
+  bool selectLoad(Register ResVReg, const SPIRVType *ResType,
+                  const MachineInstr &I, MachineIRBuilder &MIRBuilder) const;
+
+  bool selectStore(const MachineInstr &I, MachineIRBuilder &MIRBuilder) const;
+
+  bool selectMemOperation(Register ResVReg, const MachineInstr &I,
+                          MachineIRBuilder &MIRBuilder) const;
+
+  bool selectAtomicRMW(Register ResVReg, const SPIRVType *ResType,
+                       const MachineInstr &I, MachineIRBuilder &MIRBuilder,
+                       unsigned NewOpcode) const;
+
+  bool selectAtomicCmpXchg(Register ResVReg, const SPIRVType *ResType,
+                           const MachineInstr &I,
+                           MachineIRBuilder &MIRBuilder) const;
+
+  bool selectFence(const MachineInstr &I, MachineIRBuilder &MIRBuilder) const;
+
+  bool selectAddrSpaceCast(Register ResVReg, const SPIRVType *ResType,
+                           const MachineInstr &I,
+                           MachineIRBuilder &MIRBuilder) const;
+
+  bool selectBitreverse(Register ResVReg, const SPIRVType *ResType,
+                        const MachineInstr &I,
+                        MachineIRBuilder &MIRBuilder) const;
+
+  bool selectConstVector(Register ResVReg, const SPIRVType *ResType,
+                         const MachineInstr &I,
+                         MachineIRBuilder &MIRBuilder) const;
+
+  bool selectCmp(Register ResVReg, const SPIRVType *ResType,
+                 unsigned comparisonOpcode, const MachineInstr &I,
+                 MachineIRBuilder &MIRBuilder) const;
+
+  bool selectICmp(Register ResVReg, const SPIRVType *ResType,
+                  const MachineInstr &I, MachineIRBuilder &MIRBuilder) const;
+  bool selectFCmp(Register ResVReg, const SPIRVType *ResType,
+                  const MachineInstr &I, MachineIRBuilder &MIRBuilder) const;
+
+  void renderImm32(MachineInstrBuilder &MIB, const MachineInstr &I,
+                   int OpIdx) const;
+  void renderFImm32(MachineInstrBuilder &MIB, const MachineInstr &I,
+                    int OpIdx) const;
+
+  bool selectConst(Register ResVReg, const SPIRVType *ResType, const APInt &Imm,
+                   MachineIRBuilder &MIRBuilder) const;
+
+  bool selectSelect(Register ResVReg, const SPIRVType *ResType,
+                    const MachineInstr &I, bool IsSigned,
+                    MachineIRBuilder &MIRBuilder) const;
+  bool selectIToF(Register ResVReg, const SPIRVType *ResType,
+                  const MachineInstr &I, bool IsSigned,
+                  MachineIRBuilder &MIRBuilder, unsigned Opcode) const;
+  bool selectExt(Register ResVReg, const SPIRVType *ResType,
+                 const MachineInstr &I, bool IsSigned,
+                 MachineIRBuilder &MIRBuilder) const;
+
+  bool selectTrunc(Register ResVReg, const SPIRVType *ResType,
+                   const MachineInstr &I, MachineIRBuilder &MIRBuilder) const;
+
+  bool selectIntToBool(Register IntReg, Register ResVReg,
+                       const SPIRVType *intTy, const SPIRVType *boolTy,
+                       MachineIRBuilder &MIRBuilder) const;
+
+  bool selectOpUndef(Register ResVReg, const SPIRVType *ResType,
+                     MachineIRBuilder &MIRBuilder) const;
+  bool selectIntrinsic(Register ResVReg, const SPIRVType *ResType,
+                       const MachineInstr &I,
+                       MachineIRBuilder &MIRBuilder) const;
+
+  bool selectFrameIndex(Register ResVReg, const SPIRVType *ResType,
+                        MachineIRBuilder &MIRBuilder) const;
+
+  bool selectBranch(const MachineInstr &I, MachineIRBuilder &MIRBuilder) const;
+  bool selectBranchCond(const MachineInstr &I,
+                        MachineIRBuilder &MIRBuilder) const;
+
+  bool selectPhi(Register ResVReg, const SPIRVType *ResType,
+                 const MachineInstr &I, MachineIRBuilder &MIRBuilder) const;
+
+  Register buildI32Constant(uint32_t Val, MachineIRBuilder &MIRBuilder,
+                            const SPIRVType *ResType = nullptr) const;
+
+  Register buildZerosVal(const SPIRVType *ResType,
+                         MachineIRBuilder &MIRBuilder) const;
+  Register buildOnesVal(bool AllOnes, const SPIRVType *ResType,
+                        MachineIRBuilder &MIRBuilder) const;
+};
+
+} // end anonymous namespace
+
+#define GET_GLOBALISEL_IMPL
+#include "SPIRVGenGlobalISel.inc"
+#undef GET_GLOBALISEL_IMPL
+
+SPIRVInstructionSelector::SPIRVInstructionSelector(
+    const SPIRVTargetMachine &TM, const SPIRVSubtarget &ST,
+    const SPIRVRegisterBankInfo &RBI)
+    : InstructionSelector(), STI(ST), TII(*ST.getInstrInfo()),
+      TRI(*ST.getRegisterInfo()), RBI(RBI), GR(*ST.getSPIRVGlobalRegistry()),
+#define GET_GLOBALISEL_PREDICATES_INIT
+#include "SPIRVGenGlobalISel.inc"
+#undef GET_GLOBALISEL_PREDICATES_INIT
+#define GET_GLOBALISEL_TEMPORARIES_INIT
+#include "SPIRVGenGlobalISel.inc"
+#undef GET_GLOBALISEL_TEMPORARIES_INIT
+{
+}
+
+// Defined in SPIRVLegalizerInfo.cpp
+extern bool isTypeFoldingSupported(unsigned Opcode);
+
+bool SPIRVInstructionSelector::select(MachineInstr &I) {
+  assert(I.getParent() && "Instruction should be in a basic block!");
+  assert(I.getParent()->getParent() && "Instruction should be in a function!");
+
+  MachineBasicBlock &MBB = *I.getParent();
+  MachineFunction &MF = *MBB.getParent();
+  GR.setCurrentFunc(MF);
+
+  Register Opcode = I.getOpcode();
+  MachineRegisterInfo &MRI = MF.getRegInfo();
+  // If it's not a GMIR instruction, we've selected it already.
+  if (!isPreISelGenericOpcode(Opcode)) {
+    if (Opcode == SPIRV::ASSIGN_TYPE) { // These pseudos aren't needed any more
+      auto *Def = MRI.getVRegDef(I.getOperand(1).getReg());
+      if (isTypeFoldingSupported(Def->getOpcode())) {
+        auto Res = selectImpl(I, *CoverageInfo);
+        assert(Res || Def->getOpcode() == TargetOpcode::G_CONSTANT);
+        if (Res)
+          return Res;
+      }
+      MRI.replaceRegWith(I.getOperand(1).getReg(), I.getOperand(0).getReg());
+      I.removeFromParent();
+    } else if (I.getNumDefs() == 1) { // Make all vregs 32 bits (for SPIR-V IDs)
+      MRI.setType(I.getOperand(0).getReg(), LLT::scalar(32));
+    }
+    return true;
+  }
+
+  if (I.getNumOperands() != I.getNumExplicitOperands()) {
+    LLVM_DEBUG(errs() << "Generic instr has unexpected implicit operands\n");
+    return false;
+  }
+
+  // Common code for getting return reg+type, and removing selected instr
+  // from parent occurs here. Instr-specific selection happens in spvSelect().
+  bool HasDefs = I.getNumDefs() > 0;
+  Register ResVReg = HasDefs ? I.getOperand(0).getReg() : Register(0);
+  SPIRVType *ResType = HasDefs ? GR.getSPIRVTypeForVReg(ResVReg) : nullptr;
+  assert(!HasDefs || ResType || I.getOpcode() == TargetOpcode::G_GLOBAL_VALUE);
+  MachineIRBuilder MIRBuilder(I);
+  if (spvSelect(ResVReg, ResType, I, MIRBuilder)) {
+    if (HasDefs) { // Make all vregs 32 bits (for SPIR-V IDs)
+      MRI.setType(ResVReg, LLT::scalar(32));
+    }
+    I.removeFromParent();
+    return true;
+  }
+  return false;
+}
+
+bool SPIRVInstructionSelector::spvSelect(Register ResVReg,
+                                         const SPIRVType *ResType,
+                                         const MachineInstr &I,
+                                         MachineIRBuilder &MIRBuilder) const {
+  assert(!isTypeFoldingSupported(I.getOpcode()) ||
+         I.getOpcode() == TargetOpcode::G_CONSTANT);
+
+  const unsigned Opcode = I.getOpcode();
+  switch (Opcode) {
+  case TargetOpcode::G_CONSTANT:
+    return selectConst(ResVReg, ResType, I.getOperand(1).getCImm()->getValue(),
+                       MIRBuilder);
+  case TargetOpcode::G_GLOBAL_VALUE:
+    return selectGlobalValue(ResVReg, I, MIRBuilder);
+  case TargetOpcode::G_IMPLICIT_DEF:
+    return selectOpUndef(ResVReg, ResType, MIRBuilder);
+
+  case TargetOpcode::G_INTRINSIC_W_SIDE_EFFECTS:
+    return selectIntrinsic(ResVReg, ResType, I, MIRBuilder);
+  case TargetOpcode::G_BITREVERSE:
+    return selectBitreverse(ResVReg, ResType, I, MIRBuilder);
+
+  case TargetOpcode::G_BUILD_VECTOR:
+    return selectConstVector(ResVReg, ResType, I, MIRBuilder);
+
+  case TargetOpcode::G_SHUFFLE_VECTOR: {
+    auto MIB = MIRBuilder.buildInstr(SPIRV::OpVectorShuffle)
+                   .addDef(ResVReg)
+                   .addUse(GR.getSPIRVTypeID(ResType))
+                   .addUse(I.getOperand(1).getReg())
+                   .addUse(I.getOperand(2).getReg());
+    for (auto V : I.getOperand(3).getShuffleMask())
+      MIB.addImm(V);
+
+    return MIB.constrainAllUses(TII, TRI, RBI);
+  }
+  case TargetOpcode::G_MEMMOVE:
+  case TargetOpcode::G_MEMCPY:
+    return selectMemOperation(ResVReg, I, MIRBuilder);
+
+  case TargetOpcode::G_ICMP:
+    return selectICmp(ResVReg, ResType, I, MIRBuilder);
+  case TargetOpcode::G_FCMP:
+    return selectFCmp(ResVReg, ResType, I, MIRBuilder);
+
+  case TargetOpcode::G_FRAME_INDEX:
+    return selectFrameIndex(ResVReg, ResType, MIRBuilder);
+
+  case TargetOpcode::G_LOAD:
+    return selectLoad(ResVReg, ResType, I, MIRBuilder);
+  case TargetOpcode::G_STORE:
+    return selectStore(I, MIRBuilder);
+
+  case TargetOpcode::G_BR:
+    return selectBranch(I, MIRBuilder);
+  case TargetOpcode::G_BRCOND:
+    return selectBranchCond(I, MIRBuilder);
+
+  case TargetOpcode::G_PHI:
+    return selectPhi(ResVReg, ResType, I, MIRBuilder);
+
+  case TargetOpcode::G_FPTOSI:
+    return selectUnOp(ResVReg, ResType, I, MIRBuilder, SPIRV::OpConvertFToS);
+  case TargetOpcode::G_FPTOUI:
+    return selectUnOp(ResVReg, ResType, I, MIRBuilder, SPIRV::OpConvertFToU);
+
+  case TargetOpcode::G_SITOFP:
+    return selectIToF(ResVReg, ResType, I, true, MIRBuilder,
+                      SPIRV::OpConvertSToF);
+  case TargetOpcode::G_UITOFP:
+    return selectIToF(ResVReg, ResType, I, false, MIRBuilder,
+                      SPIRV::OpConvertUToF);
+
+  case TargetOpcode::G_CTPOP:
+    return selectUnOp(ResVReg, ResType, I, MIRBuilder, SPIRV::OpBitCount);
+
+  case TargetOpcode::G_SEXT:
+    return selectExt(ResVReg, ResType, I, true, MIRBuilder);
+  case TargetOpcode::G_ANYEXT:
+  case TargetOpcode::G_ZEXT:
+    return selectExt(ResVReg, ResType, I, false, MIRBuilder);
+  case TargetOpcode::G_TRUNC:
+    return selectTrunc(ResVReg, ResType, I, MIRBuilder);
+  case TargetOpcode::G_FPTRUNC:
+  case TargetOpcode::G_FPEXT:
+    return selectUnOp(ResVReg, ResType, I, MIRBuilder, SPIRV::OpFConvert);
+
+  case TargetOpcode::G_PTRTOINT:
+    return selectUnOp(ResVReg, ResType, I, MIRBuilder, SPIRV::OpConvertPtrToU);
+  case TargetOpcode::G_INTTOPTR:
+    return selectUnOp(ResVReg, ResType, I, MIRBuilder, SPIRV::OpConvertUToPtr);
+  case TargetOpcode::G_BITCAST:
+    return selectUnOp(ResVReg, ResType, I, MIRBuilder, SPIRV::OpBitcast);
+  case TargetOpcode::G_ADDRSPACE_CAST:
+    return selectAddrSpaceCast(ResVReg, ResType, I, MIRBuilder);
+
+  case TargetOpcode::G_ATOMICRMW_OR:
+    return selectAtomicRMW(ResVReg, ResType, I, MIRBuilder, SPIRV::OpAtomicOr);
+  case TargetOpcode::G_ATOMICRMW_ADD:
+    return selectAtomicRMW(ResVReg, ResType, I, MIRBuilder,
+                           SPIRV::OpAtomicIAdd);
+  case TargetOpcode::G_ATOMICRMW_AND:
+    return selectAtomicRMW(ResVReg, ResType, I, MIRBuilder, SPIRV::OpAtomicAnd);
+  case TargetOpcode::G_ATOMICRMW_MAX:
+    return selectAtomicRMW(ResVReg, ResType, I, MIRBuilder,
+                           SPIRV::OpAtomicSMax);
+  case TargetOpcode::G_ATOMICRMW_MIN:
+    return selectAtomicRMW(ResVReg, ResType, I, MIRBuilder,
+                           SPIRV::OpAtomicSMin);
+  case TargetOpcode::G_ATOMICRMW_SUB:
+    return selectAtomicRMW(ResVReg, ResType, I, MIRBuilder,
+                           SPIRV::OpAtomicISub);
+  case TargetOpcode::G_ATOMICRMW_XOR:
+    return selectAtomicRMW(ResVReg, ResType, I, MIRBuilder, SPIRV::OpAtomicXor);
+  case TargetOpcode::G_ATOMICRMW_UMAX:
+    return selectAtomicRMW(ResVReg, ResType, I, MIRBuilder,
+                           SPIRV::OpAtomicUMax);
+  case TargetOpcode::G_ATOMICRMW_UMIN:
+    return selectAtomicRMW(ResVReg, ResType, I, MIRBuilder,
+                           SPIRV::OpAtomicUMin);
+  case TargetOpcode::G_ATOMICRMW_XCHG:
+    return selectAtomicRMW(ResVReg, ResType, I, MIRBuilder,
+                           SPIRV::OpAtomicExchange);
+  case TargetOpcode::G_ATOMIC_CMPXCHG:
+    return selectAtomicCmpXchg(ResVReg, ResType, I, MIRBuilder);
+
+  case TargetOpcode::G_FENCE:
+    return selectFence(I, MIRBuilder);
+
+  default:
+    return false;
+  }
+}
+
+bool SPIRVInstructionSelector::selectUnOpWithSrc(
+    Register ResVReg, const SPIRVType *ResType, const MachineInstr &I,
+    Register SrcReg, MachineIRBuilder &MIRBuilder, unsigned Opcode) const {
+  return MIRBuilder.buildInstr(Opcode)
+      .addDef(ResVReg)
+      .addUse(GR.getSPIRVTypeID(ResType))
+      .addUse(SrcReg)
+      .constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectUnOp(Register ResVReg,
+                                          const SPIRVType *ResType,
+                                          const MachineInstr &I,
+                                          MachineIRBuilder &MIRBuilder,
+                                          unsigned Opcode) const {
+  return selectUnOpWithSrc(ResVReg, ResType, I, I.getOperand(1).getReg(),
+                           MIRBuilder, Opcode);
+}
+
+static SPIRV::MemorySemantics::MemorySemantics
+getMemSemantics(AtomicOrdering Ord) {
+  switch (Ord) {
+  case AtomicOrdering::Acquire:
+    return SPIRV::MemorySemantics::Acquire;
+  case AtomicOrdering::Release:
+    return SPIRV::MemorySemantics::Release;
+  case AtomicOrdering::AcquireRelease:
+    return SPIRV::MemorySemantics::AcquireRelease;
+  case AtomicOrdering::SequentiallyConsistent:
+    return SPIRV::MemorySemantics::SequentiallyConsistent;
+  case AtomicOrdering::Unordered:
+  case AtomicOrdering::Monotonic:
+  case AtomicOrdering::NotAtomic:
+  default:
+    return SPIRV::MemorySemantics::None;
+  }
+}
+
+static SPIRV::Scope::Scope getScope(SyncScope::ID Ord) {
+  switch (Ord) {
+  case SyncScope::SingleThread:
+    return SPIRV::Scope::Invocation;
+  case SyncScope::System:
+    return SPIRV::Scope::Device;
+  default:
+    llvm_unreachable("Unsupported synchronization Scope ID.");
+  }
+}
+
+static void addMemoryOperands(MachineMemOperand *MemOp,
+                              MachineInstrBuilder &MIB) {
+  uint32_t SpvMemOp = SPIRV::MemoryOperand::None;
+  if (MemOp->isVolatile())
+    SpvMemOp |= SPIRV::MemoryOperand::Volatile;
+  if (MemOp->isNonTemporal())
+    SpvMemOp |= SPIRV::MemoryOperand::Nontemporal;
+  if (MemOp->getAlign().value())
+    SpvMemOp |= SPIRV::MemoryOperand::Aligned;
+
+  if (SpvMemOp != SPIRV::MemoryOperand::None) {
+    MIB.addImm(SpvMemOp);
+    if (SpvMemOp & SPIRV::MemoryOperand::Aligned)
+      MIB.addImm(MemOp->getAlign().value());
+  }
+}
+
+static void addMemoryOperands(uint64_t Flags, MachineInstrBuilder &MIB) {
+  uint32_t SpvMemOp = SPIRV::MemoryOperand::None;
+  if (Flags & MachineMemOperand::Flags::MOVolatile)
+    SpvMemOp |= SPIRV::MemoryOperand::Volatile;
+  if (Flags & MachineMemOperand::Flags::MONonTemporal)
+    SpvMemOp |= SPIRV::MemoryOperand::Nontemporal;
+
+  if (SpvMemOp != SPIRV::MemoryOperand::None)
+    MIB.addImm(SpvMemOp);
+}
+
+bool SPIRVInstructionSelector::selectLoad(Register ResVReg,
+                                          const SPIRVType *ResType,
+                                          const MachineInstr &I,
+                                          MachineIRBuilder &MIRBuilder) const {
+  unsigned OpOffset =
+      I.getOpcode() == TargetOpcode::G_INTRINSIC_W_SIDE_EFFECTS ? 1 : 0;
+  Register Ptr = I.getOperand(1 + OpOffset).getReg();
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpLoad)
+                 .addDef(ResVReg)
+                 .addUse(GR.getSPIRVTypeID(ResType))
+                 .addUse(Ptr);
+  if (!I.getNumMemOperands()) {
+    assert(I.getOpcode() == TargetOpcode::G_INTRINSIC_W_SIDE_EFFECTS);
+    addMemoryOperands(I.getOperand(2 + OpOffset).getImm(), MIB);
+  } else {
+    addMemoryOperands(*I.memoperands_begin(), MIB);
+  }
+  return MIB.constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectStore(const MachineInstr &I,
+                                           MachineIRBuilder &MIRBuilder) const {
+  unsigned OpOffset =
+      I.getOpcode() == TargetOpcode::G_INTRINSIC_W_SIDE_EFFECTS ? 1 : 0;
+  Register StoreVal = I.getOperand(0 + OpOffset).getReg();
+  Register Ptr = I.getOperand(1 + OpOffset).getReg();
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpStore).addUse(Ptr).addUse(StoreVal);
+  if (!I.getNumMemOperands()) {
+    assert(I.getOpcode() == TargetOpcode::G_INTRINSIC_W_SIDE_EFFECTS);
+    addMemoryOperands(I.getOperand(2 + OpOffset).getImm(), MIB);
+  } else {
+    addMemoryOperands(*I.memoperands_begin(), MIB);
+  }
+  return MIB.constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectMemOperation(
+    Register ResVReg, const MachineInstr &I,
+    MachineIRBuilder &MIRBuilder) const {
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpCopyMemorySized)
+                 .addDef(I.getOperand(0).getReg())
+                 .addUse(I.getOperand(1).getReg())
+                 .addUse(I.getOperand(2).getReg());
+  if (I.getNumMemOperands()) {
+    addMemoryOperands(*I.memoperands_begin(), MIB);
+  }
+  bool Result = MIB.constrainAllUses(TII, TRI, RBI);
+  if (ResVReg != MIB->getOperand(0).getReg()) {
+    MIRBuilder.buildCopy(ResVReg, MIB->getOperand(0).getReg());
+  }
+  return Result;
+}
+
+bool SPIRVInstructionSelector::selectAtomicRMW(Register ResVReg,
+                                               const SPIRVType *ResType,
+                                               const MachineInstr &I,
+                                               MachineIRBuilder &MIRBuilder,
+                                               unsigned NewOpcode) const {
+  assert(I.hasOneMemOperand());
+  const MachineMemOperand *MemOp = *I.memoperands_begin();
+  SPIRV::Scope::Scope Scope = getScope(MemOp->getSyncScopeID());
+  Register ScopeReg = buildI32Constant(Scope, MIRBuilder);
+
+  Register Ptr = I.getOperand(1).getReg();
+  // Changed as it's implemented in the translator. See test/atomicrmw.ll
+  // auto ScSem =
+  // getMemSemanticsForStorageClass(GR.getPointerStorageClass(Ptr));
+
+  SPIRV::MemorySemantics::MemorySemantics MemSem =
+      getMemSemantics(MemOp->getSuccessOrdering());
+  Register MemSemReg = buildI32Constant(MemSem /*| ScSem*/, MIRBuilder);
+
+  return MIRBuilder.buildInstr(NewOpcode)
+      .addDef(ResVReg)
+      .addUse(GR.getSPIRVTypeID(ResType))
+      .addUse(Ptr)
+      .addUse(ScopeReg)
+      .addUse(MemSemReg)
+      .addUse(I.getOperand(2).getReg())
+      .constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectFence(const MachineInstr &I,
+                                           MachineIRBuilder &MIRBuilder) const {
+  SPIRV::MemorySemantics::MemorySemantics MemSem =
+      getMemSemantics(AtomicOrdering(I.getOperand(0).getImm()));
+  Register MemSemReg = buildI32Constant(MemSem, MIRBuilder);
+
+  SPIRV::Scope::Scope Scope = getScope(SyncScope::ID(I.getOperand(1).getImm()));
+  Register ScopeReg = buildI32Constant(Scope, MIRBuilder);
+
+  return MIRBuilder.buildInstr(SPIRV::OpMemoryBarrier)
+      .addUse(ScopeReg)
+      .addUse(MemSemReg)
+      .constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectAtomicCmpXchg(
+    Register ResVReg, const SPIRVType *ResType, const MachineInstr &I,
+    MachineIRBuilder &MIRBuilder) const {
+  assert(I.hasOneMemOperand());
+  const MachineMemOperand *MemOp = *I.memoperands_begin();
+  SPIRV::Scope::Scope Scope = getScope(MemOp->getSyncScopeID());
+  Register ScopeReg = buildI32Constant(Scope, MIRBuilder);
+
+  Register Ptr = I.getOperand(2).getReg();
+  Register Cmp = I.getOperand(3).getReg();
+  Register Val = I.getOperand(4).getReg();
+
+  SPIRVType *SpvValTy = GR.getSPIRVTypeForVReg(Val);
+  unsigned ScSem =
+      getMemSemanticsForStorageClass(GR.getPointerStorageClass(Ptr));
+
+  unsigned MemSemEq = getMemSemantics(MemOp->getSuccessOrdering()) | ScSem;
+  Register MemSemEqReg = buildI32Constant(MemSemEq, MIRBuilder);
+
+  unsigned MemSemNeq = getMemSemantics(MemOp->getFailureOrdering()) | ScSem;
+  Register MemSemNeqReg = MemSemEq == MemSemNeq
+                              ? MemSemEqReg
+                              : buildI32Constant(MemSemNeq, MIRBuilder);
+
+  return MIRBuilder.buildInstr(SPIRV::OpAtomicCompareExchange)
+      .addDef(ResVReg)
+      .addUse(GR.getSPIRVTypeID(SpvValTy))
+      .addUse(Ptr)
+      .addUse(ScopeReg)
+      .addUse(MemSemEqReg)
+      .addUse(MemSemNeqReg)
+      .addUse(Val)
+      .addUse(Cmp)
+      .constrainAllUses(TII, TRI, RBI);
+}
+
+static bool isGenericCastablePtr(SPIRV::StorageClass::StorageClass Sc) {
+  switch (Sc) {
+  case SPIRV::StorageClass::Workgroup:
+  case SPIRV::StorageClass::CrossWorkgroup:
+  case SPIRV::StorageClass::Function:
+    return true;
+  default:
+    return false;
+  }
+}
+
+// In SPIR-V address space casting can only happen to and from the Generic
+// storage class. We can also only case Workgroup, CrossWorkgroup, or Function
+// pointers to and from Generic pointers. As such, we can convert e.g. from
+// Workgroup to Function by going via a Generic pointer as an intermediary. All
+// other combinations can only be done by a bitcast, and are probably not safe.
+bool SPIRVInstructionSelector::selectAddrSpaceCast(
+    Register ResVReg, const SPIRVType *ResType, const MachineInstr &I,
+    MachineIRBuilder &MIRBuilder) const {
+  namespace SC = SPIRV::StorageClass;
+  Register SrcPtr = I.getOperand(1).getReg();
+  SPIRVType *SrcPtrTy = GR.getSPIRVTypeForVReg(SrcPtr);
+  SPIRV::StorageClass::StorageClass SrcSC = GR.getPointerStorageClass(SrcPtr);
+  SPIRV::StorageClass::StorageClass DstSC = GR.getPointerStorageClass(ResVReg);
+
+  if (DstSC == SC::Generic && isGenericCastablePtr(SrcSC))
+    // We're casting from an eligable pointer to Generic
+    return selectUnOp(ResVReg, ResType, I, MIRBuilder,
+                      SPIRV::OpPtrCastToGeneric);
+  if (SrcSC == SC::Generic && isGenericCastablePtr(DstSC))
+    // We're casting from Generic to an eligable pointer
+    return selectUnOp(ResVReg, ResType, I, MIRBuilder,
+                      SPIRV::OpGenericCastToPtr);
+  if (isGenericCastablePtr(SrcSC) && isGenericCastablePtr(DstSC)) {
+    // We're casting between 2 eligable pointers using Generic as an
+    // intermediary
+    Register Tmp =
+        MIRBuilder.getMRI()->createVirtualRegister(&SPIRV::IDRegClass);
+    SPIRVType *GenericPtrTy = GR.getOrCreateSPIRVPointerType(
+        SrcPtrTy, MIRBuilder, SPIRV::StorageClass::Generic);
+    bool Success = MIRBuilder.buildInstr(SPIRV::OpPtrCastToGeneric)
+                       .addDef(Tmp)
+                       .addUse(GR.getSPIRVTypeID(GenericPtrTy))
+                       .addUse(SrcPtr)
+                       .constrainAllUses(TII, TRI, RBI);
+    return Success && MIRBuilder.buildInstr(SPIRV::OpGenericCastToPtr)
+                          .addDef(ResVReg)
+                          .addUse(GR.getSPIRVTypeID(ResType))
+                          .addUse(Tmp)
+                          .constrainAllUses(TII, TRI, RBI);
+  }
+  // TODO Should this case just be disallowed completely?
+  // We're casting 2 other arbitrary address spaces, so have to bitcast
+  return selectUnOp(ResVReg, ResType, I, MIRBuilder, SPIRV::OpBitcast);
+}
+
+static unsigned int getFCmpOpcode(unsigned PredNum) {
+  auto Pred = static_cast<CmpInst::Predicate>(PredNum);
+  switch (Pred) {
+  case CmpInst::FCMP_OEQ:
+    return SPIRV::OpFOrdEqual;
+  case CmpInst::FCMP_OGE:
+    return SPIRV::OpFOrdGreaterThanEqual;
+  case CmpInst::FCMP_OGT:
+    return SPIRV::OpFOrdGreaterThan;
+  case CmpInst::FCMP_OLE:
+    return SPIRV::OpFOrdLessThanEqual;
+  case CmpInst::FCMP_OLT:
+    return SPIRV::OpFOrdLessThan;
+  case CmpInst::FCMP_ONE:
+    return SPIRV::OpFOrdNotEqual;
+  case CmpInst::FCMP_ORD:
+    return SPIRV::OpOrdered;
+  case CmpInst::FCMP_UEQ:
+    return SPIRV::OpFUnordEqual;
+  case CmpInst::FCMP_UGE:
+    return SPIRV::OpFUnordGreaterThanEqual;
+  case CmpInst::FCMP_UGT:
+    return SPIRV::OpFUnordGreaterThan;
+  case CmpInst::FCMP_ULE:
+    return SPIRV::OpFUnordLessThanEqual;
+  case CmpInst::FCMP_ULT:
+    return SPIRV::OpFUnordLessThan;
+  case CmpInst::FCMP_UNE:
+    return SPIRV::OpFUnordNotEqual;
+  case CmpInst::FCMP_UNO:
+    return SPIRV::OpUnordered;
+  default:
+    llvm_unreachable("Unknown predicate type for FCmp");
+  }
+}
+
+static unsigned int getICmpOpcode(unsigned PredNum) {
+  auto Pred = static_cast<CmpInst::Predicate>(PredNum);
+  switch (Pred) {
+  case CmpInst::ICMP_EQ:
+    return SPIRV::OpIEqual;
+  case CmpInst::ICMP_NE:
+    return SPIRV::OpINotEqual;
+  case CmpInst::ICMP_SGE:
+    return SPIRV::OpSGreaterThanEqual;
+  case CmpInst::ICMP_SGT:
+    return SPIRV::OpSGreaterThan;
+  case CmpInst::ICMP_SLE:
+    return SPIRV::OpSLessThanEqual;
+  case CmpInst::ICMP_SLT:
+    return SPIRV::OpSLessThan;
+  case CmpInst::ICMP_UGE:
+    return SPIRV::OpUGreaterThanEqual;
+  case CmpInst::ICMP_UGT:
+    return SPIRV::OpUGreaterThan;
+  case CmpInst::ICMP_ULE:
+    return SPIRV::OpULessThanEqual;
+  case CmpInst::ICMP_ULT:
+    return SPIRV::OpULessThan;
+  default:
+    llvm_unreachable("Unknown predicate type for ICmp");
+  }
+}
+
+static unsigned int getPtrCmpOpcode(unsigned Pred) {
+  switch (static_cast<CmpInst::Predicate>(Pred)) {
+  case CmpInst::ICMP_EQ:
+    return SPIRV::OpPtrEqual;
+  case CmpInst::ICMP_NE:
+    return SPIRV::OpPtrNotEqual;
+  default:
+    llvm_unreachable("Unknown predicate type for pointer comparison");
+  }
+}
+
+// Return the logical operation, or abort if none exists
+static unsigned int getBoolCmpOpcode(unsigned PredNum) {
+  auto Pred = static_cast<CmpInst::Predicate>(PredNum);
+  switch (Pred) {
+  case CmpInst::ICMP_EQ:
+    return SPIRV::OpLogicalEqual;
+  case CmpInst::ICMP_NE:
+    return SPIRV::OpLogicalNotEqual;
+  default:
+    llvm_unreachable("Unknown predicate type for Bool comparison");
+  }
+}
+
+bool SPIRVInstructionSelector::selectBitreverse(
+    Register ResVReg, const SPIRVType *ResType, const MachineInstr &I,
+    MachineIRBuilder &MIRBuilder) const {
+  return MIRBuilder.buildInstr(SPIRV::OpBitReverse)
+      .addDef(ResVReg)
+      .addUse(GR.getSPIRVTypeID(ResType))
+      .addUse(I.getOperand(1).getReg())
+      .constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectConstVector(
+    Register ResVReg, const SPIRVType *ResType, const MachineInstr &I,
+    MachineIRBuilder &MIRBuilder) const {
+  // TODO: only const case is supported for now
+  assert(std::all_of(
+      I.operands_begin(), I.operands_end(),
+      [&MIRBuilder](const MachineOperand &MO) {
+        if (MO.isDef())
+          return true;
+        if (!MO.isReg())
+          return false;
+        SPIRVType *ConstTy = MIRBuilder.getMRI()->getVRegDef(MO.getReg());
+        assert(ConstTy && ConstTy->getOpcode() == SPIRV::ASSIGN_TYPE &&
+               ConstTy->getOperand(1).isReg());
+        const MachineInstr *Const =
+            MIRBuilder.getMRI()->getVRegDef(ConstTy->getOperand(1).getReg());
+        assert(Const);
+        return (Const->getOpcode() == TargetOpcode::G_CONSTANT ||
+                Const->getOpcode() == TargetOpcode::G_FCONSTANT);
+      }));
+
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpConstantComposite)
+                 .addDef(ResVReg)
+                 .addUse(GR.getSPIRVTypeID(ResType));
+  for (unsigned i = I.getNumExplicitDefs(); i < I.getNumExplicitOperands(); ++i)
+    MIB.addUse(I.getOperand(i).getReg());
+  return MIB.constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectCmp(Register ResVReg,
+                                         const SPIRVType *ResType,
+                                         unsigned CmpOpc, const MachineInstr &I,
+                                         MachineIRBuilder &MIRBuilder) const {
+  Register Cmp0 = I.getOperand(2).getReg();
+  Register Cmp1 = I.getOperand(3).getReg();
+  assert(GR.getSPIRVTypeForVReg(Cmp0)->getOpcode() ==
+             GR.getSPIRVTypeForVReg(Cmp1)->getOpcode() &&
+         "CMP operands should have the same type");
+  return MIRBuilder.buildInstr(CmpOpc)
+      .addDef(ResVReg)
+      .addUse(GR.getSPIRVTypeID(ResType))
+      .addUse(Cmp0)
+      .addUse(Cmp1)
+      .constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectICmp(Register ResVReg,
+                                          const SPIRVType *ResType,
+                                          const MachineInstr &I,
+                                          MachineIRBuilder &MIRBuilder) const {
+  auto Pred = I.getOperand(1).getPredicate();
+  unsigned CmpOpc;
+
+  Register CmpOperand = I.getOperand(2).getReg();
+  if (GR.isScalarOfType(CmpOperand, SPIRV::OpTypePointer))
+    CmpOpc = getPtrCmpOpcode(Pred);
+  else if (GR.isScalarOrVectorOfType(CmpOperand, SPIRV::OpTypeBool))
+    CmpOpc = getBoolCmpOpcode(Pred);
+  else
+    CmpOpc = getICmpOpcode(Pred);
+  return selectCmp(ResVReg, ResType, CmpOpc, I, MIRBuilder);
+}
+
+void SPIRVInstructionSelector::renderFImm32(MachineInstrBuilder &MIB,
+                                            const MachineInstr &I,
+                                            int OpIdx) const {
+  assert(I.getOpcode() == TargetOpcode::G_FCONSTANT && OpIdx == -1 &&
+         "Expected G_FCONSTANT");
+  const ConstantFP *FPImm = I.getOperand(1).getFPImm();
+  addNumImm(FPImm->getValueAPF().bitcastToAPInt(), MIB);
+}
+
+void SPIRVInstructionSelector::renderImm32(MachineInstrBuilder &MIB,
+                                           const MachineInstr &I,
+                                           int OpIdx) const {
+  assert(I.getOpcode() == TargetOpcode::G_CONSTANT && OpIdx == -1 &&
+         "Expected G_CONSTANT");
+  addNumImm(I.getOperand(1).getCImm()->getValue(), MIB);
+}
+
+Register
+SPIRVInstructionSelector::buildI32Constant(uint32_t Val,
+                                           MachineIRBuilder &MIRBuilder,
+                                           const SPIRVType *ResType) const {
+  auto MRI = MIRBuilder.getMRI();
+  Type *LLVMTy =
+      IntegerType::get(MIRBuilder.getMF().getFunction().getContext(), 32);
+  const SPIRVType *SpvI32Ty =
+      ResType ? ResType : GR.getOrCreateSPIRVType(LLVMTy, MIRBuilder);
+  Register NewReg;
+  NewReg = MRI->createGenericVirtualRegister(LLT::scalar(32));
+  MachineInstr *MI;
+  if (Val == 0)
+    MI = MIRBuilder.buildInstr(SPIRV::OpConstantNull)
+             .addDef(NewReg)
+             .addUse(GR.getSPIRVTypeID(SpvI32Ty));
+  else
+    MI = MIRBuilder.buildInstr(SPIRV::OpConstantI)
+             .addDef(NewReg)
+             .addUse(GR.getSPIRVTypeID(SpvI32Ty))
+             .addImm(APInt(32, Val).getZExtValue());
+  constrainSelectedInstRegOperands(*MI, TII, TRI, RBI);
+  return NewReg;
+}
+
+bool SPIRVInstructionSelector::selectFCmp(Register ResVReg,
+                                          const SPIRVType *ResType,
+                                          const MachineInstr &I,
+                                          MachineIRBuilder &MIRBuilder) const {
+  unsigned int CmpOp = getFCmpOpcode(I.getOperand(1).getPredicate());
+  return selectCmp(ResVReg, ResType, CmpOp, I, MIRBuilder);
+}
+
+Register
+SPIRVInstructionSelector::buildZerosVal(const SPIRVType *ResType,
+                                        MachineIRBuilder &MIRBuilder) const {
+  return buildI32Constant(0, MIRBuilder, ResType);
+}
+
+Register
+SPIRVInstructionSelector::buildOnesVal(bool AllOnes, const SPIRVType *ResType,
+                                       MachineIRBuilder &MIRBuilder) const {
+  auto MRI = MIRBuilder.getMRI();
+  unsigned BitWidth = GR.getScalarOrVectorBitWidth(ResType);
+  APInt One = AllOnes ? APInt::getAllOnesValue(BitWidth)
+                      : APInt::getOneBitSet(BitWidth, 0);
+  Register OneReg = buildI32Constant(One.getZExtValue(), MIRBuilder, ResType);
+  if (ResType->getOpcode() == SPIRV::OpTypeVector) {
+    const unsigned NumEles = ResType->getOperand(2).getImm();
+    Register OneVec = MRI->createVirtualRegister(&SPIRV::IDRegClass);
+    auto MIB = MIRBuilder.buildInstr(SPIRV::OpConstantComposite)
+                   .addDef(OneVec)
+                   .addUse(GR.getSPIRVTypeID(ResType));
+    for (unsigned i = 0; i < NumEles; ++i) {
+      MIB.addUse(OneReg);
+    }
+    constrainSelectedInstRegOperands(*MIB, TII, TRI, RBI);
+    return OneVec;
+  }
+  return OneReg;
+}
+
+bool SPIRVInstructionSelector::selectSelect(
+    Register ResVReg, const SPIRVType *ResType, const MachineInstr &I,
+    bool IsSigned, MachineIRBuilder &MIRBuilder) const {
+  // To extend a bool, we need to use OpSelect between constants
+  Register ZeroReg = buildZerosVal(ResType, MIRBuilder);
+  Register OneReg = buildOnesVal(IsSigned, ResType, MIRBuilder);
+  bool IsScalarBool =
+      GR.isScalarOfType(I.getOperand(1).getReg(), SPIRV::OpTypeBool);
+  unsigned Opcode =
+      IsScalarBool ? SPIRV::OpSelectSISCond : SPIRV::OpSelectSIVCond;
+  return MIRBuilder.buildInstr(Opcode)
+      .addDef(ResVReg)
+      .addUse(GR.getSPIRVTypeID(ResType))
+      .addUse(I.getOperand(1).getReg())
+      .addUse(OneReg)
+      .addUse(ZeroReg)
+      .constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectIToF(Register ResVReg,
+                                          const SPIRVType *ResType,
+                                          const MachineInstr &I, bool IsSigned,
+                                          MachineIRBuilder &MIRBuilder,
+                                          unsigned Opcode) const {
+  Register SrcReg = I.getOperand(1).getReg();
+  // We can convert bool value directly to float type without OpConvert*ToF,
+  // however the translator generates OpSelect+OpConvert*ToF, so we do the same.
+  if (GR.isScalarOrVectorOfType(I.getOperand(1).getReg(), SPIRV::OpTypeBool)) {
+    unsigned BitWidth = GR.getScalarOrVectorBitWidth(ResType);
+    SPIRVType *TmpType = GR.getOrCreateSPIRVIntegerType(BitWidth, MIRBuilder);
+    if (ResType->getOpcode() == SPIRV::OpTypeVector) {
+      const unsigned NumElts = ResType->getOperand(2).getImm();
+      TmpType = GR.getOrCreateSPIRVVectorType(TmpType, NumElts, MIRBuilder);
+    }
+    auto MRI = MIRBuilder.getMRI();
+    SrcReg = MRI->createVirtualRegister(&SPIRV::IDRegClass);
+    selectSelect(SrcReg, TmpType, I, false, MIRBuilder);
+  }
+  return selectUnOpWithSrc(ResVReg, ResType, I, SrcReg, MIRBuilder, Opcode);
+}
+
+bool SPIRVInstructionSelector::selectExt(Register ResVReg,
+                                         const SPIRVType *ResType,
+                                         const MachineInstr &I, bool IsSigned,
+                                         MachineIRBuilder &MIRBuilder) const {
+  if (GR.isScalarOrVectorOfType(I.getOperand(1).getReg(), SPIRV::OpTypeBool)) {
+    return selectSelect(ResVReg, ResType, I, IsSigned, MIRBuilder);
+  }
+  return selectUnOp(ResVReg, ResType, I, MIRBuilder,
+                    IsSigned ? SPIRV::OpSConvert : SPIRV::OpUConvert);
+}
+
+bool SPIRVInstructionSelector::selectIntToBool(
+    Register IntReg, Register ResVReg, const SPIRVType *IntTy,
+    const SPIRVType *BoolTy, MachineIRBuilder &MIRBuilder) const {
+  // To truncate to a bool, we use OpBitwiseAnd 1 and OpINotEqual to zero
+  auto MRI = MIRBuilder.getMRI();
+  Register BitIntReg = MRI->createVirtualRegister(&SPIRV::IDRegClass);
+  bool IsVectorTy = IntTy->getOpcode() == SPIRV::OpTypeVector;
+  unsigned Opcode = IsVectorTy ? SPIRV::OpBitwiseAndV : SPIRV::OpBitwiseAndS;
+  Register Zero = buildZerosVal(IntTy, MIRBuilder);
+  Register One = buildOnesVal(false, IntTy, MIRBuilder);
+  MIRBuilder.buildInstr(Opcode)
+      .addDef(BitIntReg)
+      .addUse(GR.getSPIRVTypeID(IntTy))
+      .addUse(IntReg)
+      .addUse(One)
+      .constrainAllUses(TII, TRI, RBI);
+  return MIRBuilder.buildInstr(SPIRV::OpINotEqual)
+      .addDef(ResVReg)
+      .addUse(GR.getSPIRVTypeID(BoolTy))
+      .addUse(BitIntReg)
+      .addUse(Zero)
+      .constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectTrunc(Register ResVReg,
+                                           const SPIRVType *ResType,
+                                           const MachineInstr &I,
+                                           MachineIRBuilder &MIRBuilder) const {
+  if (GR.isScalarOrVectorOfType(ResVReg, SPIRV::OpTypeBool)) {
+    Register IntReg = I.getOperand(1).getReg();
+    const SPIRVType *ArgType = GR.getSPIRVTypeForVReg(IntReg);
+    return selectIntToBool(IntReg, ResVReg, ArgType, ResType, MIRBuilder);
+  }
+  bool IsSigned = GR.isScalarOrVectorSigned(ResType);
+  return selectUnOp(ResVReg, ResType, I, MIRBuilder,
+                    IsSigned ? SPIRV::OpSConvert : SPIRV::OpUConvert);
+}
+
+bool SPIRVInstructionSelector::selectConst(Register ResVReg,
+                                           const SPIRVType *ResType,
+                                           const APInt &Imm,
+                                           MachineIRBuilder &MIRBuilder) const {
+  assert(ResType->getOpcode() != SPIRV::OpTypePointer || Imm.isNullValue());
+  if (ResType->getOpcode() == SPIRV::OpTypePointer && Imm.isNullValue())
+    return MIRBuilder.buildInstr(SPIRV::OpConstantNull)
+        .addDef(ResVReg)
+        .addUse(GR.getSPIRVTypeID(ResType))
+        .constrainAllUses(TII, TRI, RBI);
+
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpConstantI)
+                 .addDef(ResVReg)
+                 .addUse(GR.getSPIRVTypeID(ResType));
+  // <=32-bit integers should be caught by the sdag pattern
+  assert(Imm.getBitWidth() > 32);
+  addNumImm(Imm, MIB);
+  return MIB.constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectOpUndef(
+    Register ResVReg, const SPIRVType *ResType,
+    MachineIRBuilder &MIRBuilder) const {
+  return MIRBuilder.buildInstr(SPIRV::OpUndef)
+      .addDef(ResVReg)
+      .addUse(GR.getSPIRVTypeID(ResType))
+      .constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectIntrinsic(
+    Register ResVReg, const SPIRVType *ResType, const MachineInstr &I,
+    MachineIRBuilder &MIRBuilder) const {
+  llvm_unreachable("Intrinsic selection not implemented");
+}
+
+bool SPIRVInstructionSelector::selectFrameIndex(
+    Register ResVReg, const SPIRVType *ResType,
+    MachineIRBuilder &MIRBuilder) const {
+  return MIRBuilder.buildInstr(SPIRV::OpVariable)
+      .addDef(ResVReg)
+      .addUse(GR.getSPIRVTypeID(ResType))
+      .addImm(SPIRV::StorageClass::Function)
+      .constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectBranch(
+    const MachineInstr &I, MachineIRBuilder &MIRBuilder) const {
+  // InstructionSelector walks backwards through the instructions. We can use
+  // both a G_BR and a G_BRCOND to create an OpBranchConditional. We hit G_BR
+  // first, so can generate an OpBranchConditional here. If there is no
+  // G_BRCOND, we just use OpBranch for a regular unconditional branch.
+  const MachineInstr *PrevI = I.getPrevNode();
+  if (PrevI != nullptr && PrevI->getOpcode() == TargetOpcode::G_BRCOND)
+    return MIRBuilder.buildInstr(SPIRV::OpBranchConditional)
+        .addUse(PrevI->getOperand(0).getReg())
+        .addMBB(PrevI->getOperand(1).getMBB())
+        .addMBB(I.getOperand(0).getMBB())
+        .constrainAllUses(TII, TRI, RBI);
+  return MIRBuilder.buildInstr(SPIRV::OpBranch)
+      .addMBB(I.getOperand(0).getMBB())
+      .constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectBranchCond(
+    const MachineInstr &I, MachineIRBuilder &MIRBuilder) const {
+  // InstructionSelector walks backwards through the instructions. For an
+  // explicit conditional branch with no fallthrough, we use both a G_BR and a
+  // G_BRCOND to create an OpBranchConditional. We should hit G_BR first, and
+  // generate the OpBranchConditional in selectBranch above.
+  //
+  // If an OpBranchConditional has been generated, we simply return, as the work
+  // is alread done. If there is no OpBranchConditional, LLVM must be relying on
+  // implicit fallthrough to the next basic block, so we need to create an
+  // OpBranchConditional with an explicit "false" argument pointing to the next
+  // basic block that LLVM would fall through to.
+  const MachineInstr *NextI = I.getNextNode();
+  // Check if this has already been successfully selected
+  if (NextI != nullptr && NextI->getOpcode() == SPIRV::OpBranchConditional)
+    return true;
+  // Must be relying on implicit block fallthrough, so generate an
+  // OpBranchConditional with the "next" basic block as the "false" target.
+  unsigned int NextMBBNum = I.getParent()->getNextNode()->getNumber();
+  MachineBasicBlock *NextMBB = I.getMF()->getBlockNumbered(NextMBBNum);
+  return MIRBuilder.buildInstr(SPIRV::OpBranchConditional)
+      .addUse(I.getOperand(0).getReg())
+      .addMBB(I.getOperand(1).getMBB())
+      .addMBB(NextMBB)
+      .constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectPhi(Register ResVReg,
+                                         const SPIRVType *ResType,
+                                         const MachineInstr &I,
+                                         MachineIRBuilder &MIRBuilder) const {
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpPhi)
+                 .addDef(ResVReg)
+                 .addUse(GR.getSPIRVTypeID(ResType));
+  const unsigned int NumOps = I.getNumOperands();
+  assert((NumOps % 2 == 1) && "Require odd number of operands for G_PHI");
+  for (unsigned int i = 1; i < NumOps; i += 2) {
+    MIB.addUse(I.getOperand(i + 0).getReg());
+    MIB.addMBB(I.getOperand(i + 1).getMBB());
+  }
+  return MIB.constrainAllUses(TII, TRI, RBI);
+}
+
+bool SPIRVInstructionSelector::selectGlobalValue(
+    Register ResVReg, const MachineInstr &I, MachineIRBuilder &MIRBuilder,
+    const MachineInstr *Init) const {
+  const GlobalValue *GV = I.getOperand(1).getGlobal();
+  SPIRVType *ResType =
+      GR.getOrCreateSPIRVType(GV->getType(), MIRBuilder, AQ::ReadWrite, false);
+
+  std::string GlobalIdent = GV->getGlobalIdentifier();
+  // TODO: suport @llvm.global.annotations
+  auto GlobalVar = cast<GlobalVariable>(GV);
+
+  bool HasInit = GlobalVar->hasInitializer() &&
+                 !isa<UndefValue>(GlobalVar->getInitializer());
+  // Skip empty declaration for GVs with initilaizers till we get the decl with
+  // passed initializer.
+  if (HasInit && !Init)
+    return true;
+
+  unsigned AddrSpace = GV->getAddressSpace();
+  SPIRV::StorageClass::StorageClass Storage =
+      addressSpaceToStorageClass(AddrSpace);
+  bool HasLnkTy = GV->getLinkage() != GlobalValue::InternalLinkage &&
+                  Storage != SPIRV::StorageClass::Function;
+  SPIRV::LinkageType::LinkageType LnkType =
+      (GV->isDeclaration() || GV->hasAvailableExternallyLinkage())
+          ? SPIRV::LinkageType::Import
+          : SPIRV::LinkageType::Export;
+
+  Register Reg = GR.buildGlobalVariable(ResVReg, ResType, GlobalIdent, GV,
+                                        Storage, Init, GlobalVar->isConstant(),
+                                        HasLnkTy, LnkType, MIRBuilder, true);
+  return Reg.isValid();
+}
+
+namespace llvm {
+InstructionSelector *
+createSPIRVInstructionSelector(const SPIRVTargetMachine &TM,
+                               const SPIRVSubtarget &Subtarget,
+                               const SPIRVRegisterBankInfo &RBI) {
+  return new SPIRVInstructionSelector(TM, Subtarget, RBI);
+}
+} // namespace llvm

llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.h

+//===- SPIRVLegalizerInfo.h --- SPIR-V Legalization Rules --------*- C++ -*-==//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file declares the targeting of the MachineLegalizer class for SPIR-V.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_TARGET_SPIRV_SPIRVMACHINELEGALIZER_H
+#define LLVM_LIB_TARGET_SPIRV_SPIRVMACHINELEGALIZER_H
+
+#include "SPIRVGlobalRegistry.h"
+#include "llvm/CodeGen/GlobalISel/LegalizerInfo.h"
+
+bool isTypeFoldingSupported(unsigned Opcode);
+
+namespace llvm {
+
+class LLVMContext;
+class SPIRVSubtarget;
+
+// This class provides the information for legalizing SPIR-V instructions.
+class SPIRVLegalizerInfo : public LegalizerInfo {
+  const SPIRVSubtarget *ST;
+  SPIRVGlobalRegistry *GR;
+
+public:
+  bool legalizeCustom(LegalizerHelper &Helper, MachineInstr &MI) const override;
+  SPIRVLegalizerInfo(const SPIRVSubtarget &ST);
+};
+} // namespace llvm
+#endif // LLVM_LIB_TARGET_SPIRV_SPIRVMACHINELEGALIZER_H

llvm/lib/Target/SPIRV/SPIRVLegalizerInfo.cpp

+//===- SPIRVLegalizerInfo.cpp --- SPIR-V Legalization Rules ------*- C++ -*-==//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the targeting of the Machinelegalizer class for SPIR-V.
+//
+//===----------------------------------------------------------------------===//
+
+#include "SPIRVLegalizerInfo.h"
+#include "SPIRV.h"
+#include "SPIRVGlobalRegistry.h"
+#include "SPIRVSubtarget.h"
+#include "llvm/CodeGen/GlobalISel/LegalizerHelper.h"
+#include "llvm/CodeGen/GlobalISel/MachineIRBuilder.h"
+#include "llvm/CodeGen/MachineInstr.h"
+#include "llvm/CodeGen/MachineRegisterInfo.h"
+#include "llvm/CodeGen/TargetOpcodes.h"
+
+//#include <set>
+
+using namespace llvm;
+using namespace llvm::LegalizeActions;
+using namespace llvm::LegalityPredicates;
+
+static const std::set<unsigned> TypeFoldingSupportingOpcs = {
+    TargetOpcode::G_ADD,
+    TargetOpcode::G_FADD,
+    TargetOpcode::G_SUB,
+    TargetOpcode::G_FSUB,
+    TargetOpcode::G_MUL,
+    TargetOpcode::G_FMUL,
+    TargetOpcode::G_SDIV,
+    TargetOpcode::G_UDIV,
+    TargetOpcode::G_FDIV,
+    TargetOpcode::G_SREM,
+    TargetOpcode::G_UREM,
+    TargetOpcode::G_FREM,
+    TargetOpcode::G_FNEG,
+    TargetOpcode::G_CONSTANT,
+    TargetOpcode::G_FCONSTANT,
+    TargetOpcode::G_AND,
+    TargetOpcode::G_OR,
+    TargetOpcode::G_XOR,
+    TargetOpcode::G_SHL,
+    TargetOpcode::G_ASHR,
+    TargetOpcode::G_LSHR,
+    TargetOpcode::G_SELECT,
+    TargetOpcode::G_EXTRACT_VECTOR_ELT,
+};
+
+bool isTypeFoldingSupported(unsigned Opcode) {
+  return TypeFoldingSupportingOpcs.count(Opcode) > 0;
+}
+
+SPIRVLegalizerInfo::SPIRVLegalizerInfo(const SPIRVSubtarget &ST) {
+  using namespace TargetOpcode;
+
+  this->ST = &ST;
+  GR = ST.getSPIRVGlobalRegistry();
+
+  const LLT s1 = LLT::scalar(1);
+  const LLT s8 = LLT::scalar(8);
+  const LLT s16 = LLT::scalar(16);
+  const LLT s32 = LLT::scalar(32);
+  const LLT s64 = LLT::scalar(64);
+
+  const LLT v16s64 = LLT::fixed_vector(16, 64);
+  const LLT v16s32 = LLT::fixed_vector(16, 32);
+  const LLT v16s16 = LLT::fixed_vector(16, 16);
+  const LLT v16s8 = LLT::fixed_vector(16, 8);
+  const LLT v16s1 = LLT::fixed_vector(16, 1);
+
+  const LLT v8s64 = LLT::fixed_vector(8, 64);
+  const LLT v8s32 = LLT::fixed_vector(8, 32);
+  const LLT v8s16 = LLT::fixed_vector(8, 16);
+  const LLT v8s8 = LLT::fixed_vector(8, 8);
+  const LLT v8s1 = LLT::fixed_vector(8, 1);
+
+  const LLT v4s64 = LLT::fixed_vector(4, 64);
+  const LLT v4s32 = LLT::fixed_vector(4, 32);
+  const LLT v4s16 = LLT::fixed_vector(4, 16);
+  const LLT v4s8 = LLT::fixed_vector(4, 8);
+  const LLT v4s1 = LLT::fixed_vector(4, 1);
+
+  const LLT v3s64 = LLT::fixed_vector(3, 64);
+  const LLT v3s32 = LLT::fixed_vector(3, 32);
+  const LLT v3s16 = LLT::fixed_vector(3, 16);
+  const LLT v3s8 = LLT::fixed_vector(3, 8);
+  const LLT v3s1 = LLT::fixed_vector(3, 1);
+
+  const LLT v2s64 = LLT::fixed_vector(2, 64);
+  const LLT v2s32 = LLT::fixed_vector(2, 32);
+  const LLT v2s16 = LLT::fixed_vector(2, 16);
+  const LLT v2s8 = LLT::fixed_vector(2, 8);
+  const LLT v2s1 = LLT::fixed_vector(2, 1);
+
+  const unsigned PSize = ST.getPointerSize();
+  const LLT p0 = LLT::pointer(0, PSize); // Function
+  const LLT p1 = LLT::pointer(1, PSize); // CrossWorkgroup
+  const LLT p2 = LLT::pointer(2, PSize); // UniformConstant
+  const LLT p3 = LLT::pointer(3, PSize); // Workgroup
+  const LLT p4 = LLT::pointer(4, PSize); // Generic
+  const LLT p5 = LLT::pointer(5, PSize); // Input
+
+  // TODO: remove copy-pasting here by using concatenation in some way
+  auto allPtrsScalarsAndVectors = {
+      p0,    p1,    p2,    p3,    p4,    p5,    s1,     s8,     s16,
+      s32,   s64,   v2s1,  v2s8,  v2s16, v2s32, v2s64,  v3s1,   v3s8,
+      v3s16, v3s32, v3s64, v4s1,  v4s8,  v4s16, v4s32,  v4s64,  v8s1,
+      v8s8,  v8s16, v8s32, v8s64, v16s1, v16s8, v16s16, v16s32, v16s64};
+
+  auto allScalarsAndVectors = {
+      s1,   s8,   s16,   s32,   s64,   v2s1,  v2s8,  v2s16,  v2s32,  v2s64,
+      v3s1, v3s8, v3s16, v3s32, v3s64, v4s1,  v4s8,  v4s16,  v4s32,  v4s64,
+      v8s1, v8s8, v8s16, v8s32, v8s64, v16s1, v16s8, v16s16, v16s32, v16s64};
+
+  auto allIntScalarsAndVectors = {s8,    s16,   s32,   s64,    v2s8,   v2s16,
+                                  v2s32, v2s64, v3s8,  v3s16,  v3s32,  v3s64,
+                                  v4s8,  v4s16, v4s32, v4s64,  v8s8,   v8s16,
+                                  v8s32, v8s64, v16s8, v16s16, v16s32, v16s64};
+
+  auto allBoolScalarsAndVectors = {s1, v2s1, v3s1, v4s1, v8s1, v16s1};
+
+  auto allIntScalars = {s8, s16, s32, s64};
+
+  auto allFloatScalarsAndVectors = {
+      s16,   s32,   s64,   v2s16, v2s32, v2s64, v3s16,  v3s32,  v3s64,
+      v4s16, v4s32, v4s64, v8s16, v8s32, v8s64, v16s16, v16s32, v16s64};
+
+  auto allFloatAndIntScalars = allIntScalars;
+
+  auto allPtrs = {p0, p1, p2, p3, p4, p5};
+  auto allWritablePtrs = {p0, p1, p3, p4};
+
+  for (auto Opc : TypeFoldingSupportingOpcs)
+    getActionDefinitionsBuilder(Opc).custom();
+
+  getActionDefinitionsBuilder(G_GLOBAL_VALUE).alwaysLegal();
+
+  // TODO: add proper rules for vectors legalization
+  getActionDefinitionsBuilder({G_BUILD_VECTOR, G_SHUFFLE_VECTOR}).alwaysLegal();
+
+  getActionDefinitionsBuilder({G_MEMCPY, G_MEMMOVE})
+      .legalIf(all(typeInSet(0, allWritablePtrs), typeInSet(1, allPtrs)));
+
+  getActionDefinitionsBuilder(G_ADDRSPACE_CAST)
+      .legalForCartesianProduct(allPtrs, allPtrs);
+
+  getActionDefinitionsBuilder({G_LOAD, G_STORE}).legalIf(typeInSet(1, allPtrs));
+
+  getActionDefinitionsBuilder(G_BITREVERSE).legalFor(allFloatScalarsAndVectors);
+
+  getActionDefinitionsBuilder(G_FMA).legalFor(allFloatScalarsAndVectors);
+
+  getActionDefinitionsBuilder({G_FPTOSI, G_FPTOUI})
+      .legalForCartesianProduct(allIntScalarsAndVectors,
+                                allFloatScalarsAndVectors);
+
+  getActionDefinitionsBuilder({G_SITOFP, G_UITOFP})
+      .legalForCartesianProduct(allFloatScalarsAndVectors,
+                                allScalarsAndVectors);
+
+  getActionDefinitionsBuilder({G_SMIN, G_SMAX, G_UMIN, G_UMAX, G_ABS})
+      .legalFor(allIntScalarsAndVectors);
+
+  getActionDefinitionsBuilder(G_CTPOP).legalForCartesianProduct(
+      allIntScalarsAndVectors, allIntScalarsAndVectors);
+
+  getActionDefinitionsBuilder(G_PHI).legalFor(allPtrsScalarsAndVectors);
+
+  getActionDefinitionsBuilder(G_BITCAST).legalIf(all(
+      typeInSet(0, allPtrsScalarsAndVectors),
+      typeInSet(1, allPtrsScalarsAndVectors),
+      LegalityPredicate(([=](const LegalityQuery &Query) {
+        return Query.Types[0].getSizeInBits() == Query.Types[1].getSizeInBits();
+      }))));
+
+  getActionDefinitionsBuilder(G_IMPLICIT_DEF).alwaysLegal();
+
+  getActionDefinitionsBuilder(G_INTTOPTR)
+      .legalForCartesianProduct(allPtrs, allIntScalars);
+  getActionDefinitionsBuilder(G_PTRTOINT)
+      .legalForCartesianProduct(allIntScalars, allPtrs);
+  getActionDefinitionsBuilder(G_PTR_ADD).legalForCartesianProduct(
+      allPtrs, allIntScalars);
+
+  // ST.canDirectlyComparePointers() for pointer args is supported in
+  // legalizeCustom().
+  getActionDefinitionsBuilder(G_ICMP).customIf(
+      all(typeInSet(0, allBoolScalarsAndVectors),
+          typeInSet(1, allPtrsScalarsAndVectors)));
+
+  getActionDefinitionsBuilder(G_FCMP).legalIf(
+      all(typeInSet(0, allBoolScalarsAndVectors),
+          typeInSet(1, allFloatScalarsAndVectors)));
+
+  getActionDefinitionsBuilder({G_ATOMICRMW_OR, G_ATOMICRMW_ADD, G_ATOMICRMW_AND,
+                               G_ATOMICRMW_MAX, G_ATOMICRMW_MIN,
+                               G_ATOMICRMW_SUB, G_ATOMICRMW_XOR,
+                               G_ATOMICRMW_UMAX, G_ATOMICRMW_UMIN})
+      .legalForCartesianProduct(allIntScalars, allWritablePtrs);
+
+  getActionDefinitionsBuilder(G_ATOMICRMW_XCHG)
+      .legalForCartesianProduct(allFloatAndIntScalars, allWritablePtrs);
+
+  getActionDefinitionsBuilder(G_ATOMIC_CMPXCHG_WITH_SUCCESS).lower();
+  // TODO: add proper legalization rules
+  getActionDefinitionsBuilder(G_ATOMIC_CMPXCHG).alwaysLegal();
+
+  getActionDefinitionsBuilder({G_UADDO, G_USUBO, G_SMULO, G_UMULO})
+      .alwaysLegal();
+
+  // Extensions
+  getActionDefinitionsBuilder({G_TRUNC, G_ZEXT, G_SEXT, G_ANYEXT})
+      .legalForCartesianProduct(allScalarsAndVectors);
+
+  // FP conversions
+  getActionDefinitionsBuilder({G_FPTRUNC, G_FPEXT})
+      .legalForCartesianProduct(allFloatScalarsAndVectors);
+
+  // Pointer-handling
+  getActionDefinitionsBuilder(G_FRAME_INDEX).legalFor({p0});
+
+  // Control-flow
+  getActionDefinitionsBuilder(G_BRCOND).legalFor({s1});
+
+  getActionDefinitionsBuilder({G_FPOW,
+                               G_FEXP,
+                               G_FEXP2,
+                               G_FLOG,
+                               G_FLOG2,
+                               G_FABS,
+                               G_FMINNUM,
+                               G_FMAXNUM,
+                               G_FCEIL,
+                               G_FCOS,
+                               G_FSIN,
+                               G_FSQRT,
+                               G_FFLOOR,
+                               G_FRINT,
+                               G_FNEARBYINT,
+                               G_INTRINSIC_ROUND,
+                               G_INTRINSIC_TRUNC,
+                               G_FMINIMUM,
+                               G_FMAXIMUM,
+                               G_INTRINSIC_ROUNDEVEN})
+      .legalFor(allFloatScalarsAndVectors);
+
+  getActionDefinitionsBuilder(G_FCOPYSIGN)
+      .legalForCartesianProduct(allFloatScalarsAndVectors,
+                                allFloatScalarsAndVectors);
+
+  getActionDefinitionsBuilder(G_FPOWI).legalForCartesianProduct(
+      allFloatScalarsAndVectors, allIntScalarsAndVectors);
+
+  getLegacyLegalizerInfo().computeTables();
+  verify(*ST.getInstrInfo());
+}
+
+static Register convertPtrToInt(Register Reg, LLT ConvTy, SPIRVType *SpirvType,
+                                LegalizerHelper &Helper,
+                                MachineRegisterInfo &MRI,
+                                SPIRVGlobalRegistry *GR) {
+  Register ConvReg = MRI.createGenericVirtualRegister(ConvTy);
+  GR->assignSPIRVTypeToVReg(SpirvType, ConvReg, Helper.MIRBuilder);
+  Helper.MIRBuilder.buildInstr(TargetOpcode::G_PTRTOINT)
+      .addDef(ConvReg)
+      .addUse(Reg);
+  return ConvReg;
+}
+
+bool SPIRVLegalizerInfo::legalizeCustom(LegalizerHelper &Helper,
+                                        MachineInstr &MI) const {
+  auto Opc = MI.getOpcode();
+  MachineRegisterInfo &MRI = MI.getMF()->getRegInfo();
+  if (!isTypeFoldingSupported(Opc)) {
+    assert(Opc == TargetOpcode::G_ICMP);
+    assert(GR->getSPIRVTypeForVReg(MI.getOperand(0).getReg()));
+    auto &Op0 = MI.getOperand(2);
+    auto &Op1 = MI.getOperand(3);
+    Register Reg0 = Op0.getReg();
+    Register Reg1 = Op1.getReg();
+    CmpInst::Predicate Cond =
+        static_cast<CmpInst::Predicate>(MI.getOperand(1).getPredicate());
+    if ((!ST->canDirectlyComparePointers() ||
+         (Cond != CmpInst::ICMP_EQ && Cond != CmpInst::ICMP_NE)) &&
+        MRI.getType(Reg0).isPointer() && MRI.getType(Reg1).isPointer()) {
+      LLT ConvT = LLT::scalar(ST->getPointerSize());
+      Type *LLVMTy = IntegerType::get(MI.getMF()->getFunction().getContext(),
+                                      ST->getPointerSize());
+      SPIRVType *SpirvTy = GR->getOrCreateSPIRVType(LLVMTy, Helper.MIRBuilder);
+      Op0.setReg(convertPtrToInt(Reg0, ConvT, SpirvTy, Helper, MRI, GR));
+      Op1.setReg(convertPtrToInt(Reg1, ConvT, SpirvTy, Helper, MRI, GR));
+    }
+    return true;
+  }
+  // TODO: implement legalization for other opcodes.
+  return true;
+}

llvm/lib/Target/SPIRV/SPIRVSubtarget.h

 #include "llvm/IR/DataLayout.h"
 #include "llvm/Target/TargetMachine.h"
 
 #define GET_SUBTARGETINFO_HEADER
 #include "SPIRVGenSubtargetInfo.inc"
 
 namespace llvm {
 class StringRef;
-
+class SPIRVGlobalRegistry;
 class SPIRVTargetMachine;
 
 class SPIRVSubtarget : public SPIRVGenSubtargetInfo {
 private:
   const unsigned int PointerSize;
   uint32_t SPIRVVersion;
 
+  std::unique_ptr<SPIRVGlobalRegistry> GR;
+
   SPIRVInstrInfo InstrInfo;
   SPIRVFrameLowering FrameLowering;
   SPIRVTargetLowering TLInfo;
 
   // GlobalISel related APIs.
   std::unique_ptr<CallLowering> CallLoweringInfo;
   std::unique_ptr<RegisterBankInfo> RegBankInfo;
+  std::unique_ptr<LegalizerInfo> Legalizer;
+  std::unique_ptr<InstructionSelector> InstSelector;
 
 public:
   // This constructor initializes the data members to match that
   // of the specified triple.
   SPIRVSubtarget(const Triple &TT, const std::string &CPU,
                  const std::string &FS, const SPIRVTargetMachine &TM);
 
   SPIRVSubtarget &initSubtargetDependencies(StringRef CPU, StringRef FS);
 
   // Parses features string setting specified subtarget options.
   //
   // The definition of this function is auto generated by tblgen.
   void ParseSubtargetFeatures(StringRef CPU, StringRef TuneCPU, StringRef FS);
 
   unsigned int getPointerSize() const { return PointerSize; }
   bool canDirectlyComparePointers() const;
 
   uint32_t getSPIRVVersion() const { return SPIRVVersion; };
 
+  SPIRVGlobalRegistry *getSPIRVGlobalRegistry() const { return GR.get(); }
+
   const CallLowering *getCallLowering() const override {
     return CallLoweringInfo.get();
   }
 
   const RegisterBankInfo *getRegBankInfo() const override {
     return RegBankInfo.get();
   }
 
+  const LegalizerInfo *getLegalizerInfo() const override {
+    return Legalizer.get();
+  }
+
+  InstructionSelector *getInstructionSelector() const override {
+    return InstSelector.get();
+  }
+
   const SPIRVInstrInfo *getInstrInfo() const override { return &InstrInfo; }
 
   const SPIRVFrameLowering *getFrameLowering() const override {
     return &FrameLowering;
   }
 
   const SPIRVTargetLowering *getTargetLowering() const override {
     return &TLInfo;

llvm/lib/Target/SPIRV/SPIRVSubtarget.cpp

 //===-- SPIRVSubtarget.cpp - SPIR-V Subtarget Information ------*- C++ -*--===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the SPIR-V specific subclass of TargetSubtargetInfo.
 //
 //===----------------------------------------------------------------------===//
 
 #include "SPIRVSubtarget.h"
 #include "SPIRV.h"
+#include "SPIRVGlobalRegistry.h"
+#include "SPIRVLegalizerInfo.h"
 #include "SPIRVRegisterBankInfo.h"
 #include "SPIRVTargetMachine.h"
 #include "llvm/MC/TargetRegistry.h"
 #include "llvm/Support/Host.h"
 
 using namespace llvm;
 
 #define DEBUG_TYPE "spirv-subtarget"

 }
 
 SPIRVSubtarget::SPIRVSubtarget(const Triple &TT, const std::string &CPU,
                                const std::string &FS,
                                const SPIRVTargetMachine &TM)
     : SPIRVGenSubtargetInfo(TT, CPU, /*TuneCPU=*/CPU, FS),
       PointerSize(computePointerSize(TT)), SPIRVVersion(0), InstrInfo(),
       FrameLowering(initSubtargetDependencies(CPU, FS)), TLInfo(TM, *this) {
-  CallLoweringInfo.reset(new SPIRVCallLowering(TLInfo));
+  GR.reset(new SPIRVGlobalRegistry(PointerSize));
+  CallLoweringInfo.reset(new SPIRVCallLowering(TLInfo, GR.get()));
+  Legalizer.reset(new SPIRVLegalizerInfo(*this));
 
   auto *RBI = new SPIRVRegisterBankInfo();
   RegBankInfo.reset(RBI);
+  InstSelector.reset(createSPIRVInstructionSelector(TM, *this, *RBI));
 }
 
 SPIRVSubtarget &SPIRVSubtarget::initSubtargetDependencies(StringRef CPU,
                                                           StringRef FS) {
   ParseSubtargetFeatures(CPU, /*TuneCPU=*/CPU, FS);
   if (SPIRVVersion == 0)
     SPIRVVersion = 14;
   return *this;
 }
 
 // If the SPIR-V version is >= 1.4 we can call OpPtrEqual and OpPtrNotEqual
 bool SPIRVSubtarget::canDirectlyComparePointers() const {
   return isAtLeastVer(SPIRVVersion, 14);
 }

llvm/lib/Target/SPIRV/SPIRVTargetMachine.cpp

 //===- SPIRVTargetMachine.cpp - Define TargetMachine for SPIR-V -*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // Implements the info about SPIR-V target spec.
 //
 //===----------------------------------------------------------------------===//
 
 #include "SPIRVTargetMachine.h"
 #include "SPIRV.h"
+#include "SPIRVCallLowering.h"
+#include "SPIRVGlobalRegistry.h"
+#include "SPIRVLegalizerInfo.h"
 #include "SPIRVTargetObjectFile.h"
 #include "SPIRVTargetTransformInfo.h"
 #include "TargetInfo/SPIRVTargetInfo.h"
 #include "llvm/CodeGen/GlobalISel/IRTranslator.h"
 #include "llvm/CodeGen/GlobalISel/InstructionSelect.h"
 #include "llvm/CodeGen/GlobalISel/Legalizer.h"
 #include "llvm/CodeGen/GlobalISel/RegBankSelect.h"
 #include "llvm/CodeGen/Passes.h"
 #include "llvm/CodeGen/TargetLoweringObjectFileImpl.h"
 #include "llvm/CodeGen/TargetPassConfig.h"
 #include "llvm/IR/LegacyPassManager.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/MC/TargetRegistry.h"
 #include "llvm/Pass.h"
 #include "llvm/Target/TargetOptions.h"
 
 using namespace llvm;
+InstructionSelector *
+createSPIRVInstructionSelector(const SPIRVTargetMachine &TM,
+                               SPIRVSubtarget &Subtarget,
+                               SPIRVRegisterBankInfo &RBI);
 
 extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeSPIRVTarget() {
   // Register the target.
   RegisterTargetMachine<SPIRVTargetMachine> X(getTheSPIRV32Target());
   RegisterTargetMachine<SPIRVTargetMachine> Y(getTheSPIRV64Target());
+
+  PassRegistry &PR = *PassRegistry::getPassRegistry();
+  initializeGlobalISel(PR);
 }
 
 static std::string computeDataLayout(const Triple &TT) {
   std::string DataLayout = "e-m:e";
 
   const auto Arch = TT.getArch();
   if (Arch == Triple::spirv32)
     DataLayout += "-p:32:32";

 }
 
 // Do not add a RegBankSelect pass, as we only ever need virtual registers
 bool SPIRVPassConfig::addRegBankSelect() {
   disablePass(&RegBankSelect::ID);
   return false;
 }
 
+namespace {
+// A custom subclass of InstructionSelect, which is mostly the same except from
+// not requiring RegBankSelect to occur previously.
+class SPIRVInstructionSelect : public InstructionSelect {
+  // We don't use register banks, so unset the requirement for them
+  MachineFunctionProperties getRequiredProperties() const override {
+    return InstructionSelect::getRequiredProperties().reset(
+        MachineFunctionProperties::Property::RegBankSelected);
+  }
+};
+} // namespace
+
 bool SPIRVPassConfig::addGlobalInstructionSelect() {
-  addPass(new InstructionSelect(getOptLevel()));
+  addPass(new SPIRVInstructionSelect());
   return false;
 }

llvm/lib/Target/SPIRV/SPIRVUtils.h

+//===--- SPIRVUtils.h ---- SPIR-V Utility Functions -------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains miscellaneous utility functions.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_TARGET_SPIRV_SPIRVUTILS_H
+#define LLVM_LIB_TARGET_SPIRV_SPIRVUTILS_H
+
+#include "MCTargetDesc/SPIRVBaseInfo.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/CodeGen/GlobalISel/MachineIRBuilder.h"
+#include "llvm/CodeGen/MachineInstr.h"
+#include "llvm/CodeGen/MachineInstrBuilder.h"
+#include "llvm/IR/IRBuilder.h"
+#include <string>
+
+// Add the given string as a series of integer operand, inserting null
+// terminators and padding to make sure the operands all have 32-bit
+// little-endian words.
+void addStringImm(const llvm::StringRef &Str, llvm::MachineInstrBuilder &MIB);
+void addStringImm(const llvm::StringRef &Str, llvm::IRBuilder<> &B,
+                  std::vector<llvm::Value *> &Args);
+
+// Add the given numerical immediate to MIB.
+void addNumImm(const llvm::APInt &Imm, llvm::MachineInstrBuilder &MIB);
+
+// Add an OpName instruction for the given target register.
+void buildOpName(llvm::Register Target, const llvm::StringRef &Name,
+                 llvm::MachineIRBuilder &MIRBuilder);
+
+// Add an OpDecorate instruction for the given Reg.
+void buildOpDecorate(llvm::Register Reg, llvm::MachineIRBuilder &MIRBuilder,
+                     llvm::SPIRV::Decoration::Decoration Dec,
+                     const std::vector<uint32_t> &DecArgs,
+                     llvm::StringRef StrImm = "");
+
+// Convert a SPIR-V storage class to the corresponding LLVM IR address space.
+unsigned int
+storageClassToAddressSpace(llvm::SPIRV::StorageClass::StorageClass SC);
+
+// Convert an LLVM IR address space to a SPIR-V storage class.
+llvm::SPIRV::StorageClass::StorageClass
+addressSpaceToStorageClass(unsigned int AddrSpace);
+
+llvm::SPIRV::MemorySemantics::MemorySemantics
+getMemSemanticsForStorageClass(llvm::SPIRV::StorageClass::StorageClass sc);
+#endif // LLVM_LIB_TARGET_SPIRV_SPIRVUTILS_H

llvm/lib/Target/SPIRV/SPIRVUtils.cpp

+//===--- SPIRVUtils.cpp ---- SPIR-V Utility Functions -----------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file contains miscellaneous utility functions.
+//
+//===----------------------------------------------------------------------===//
+
+#include "SPIRVUtils.h"
+#include "SPIRV.h"
+
+using namespace llvm;
+
+// The following functions are used to add these string literals as a series of
+// 32-bit integer operands with the correct format, and unpack them if necessary
+// when making string comparisons in compiler passes.
+// SPIR-V requires null-terminated UTF-8 strings padded to 32-bit alignment.
+static uint32_t convertCharsToWord(const StringRef &Str, unsigned i) {
+  uint32_t Word = 0u; // Build up this 32-bit word from 4 8-bit chars
+  for (unsigned WordIndex = 0; WordIndex < 4; ++WordIndex) {
+    unsigned StrIndex = i + WordIndex;
+    uint8_t CharToAdd = 0;       // Initilize char as padding/null
+    if (StrIndex < Str.size()) { // If it's within the string, get a real char
+      CharToAdd = Str[StrIndex];
+    }
+    Word |= (CharToAdd << (WordIndex * 8));
+  }
+  return Word;
+}
+
+// Get length including padding and null terminator.
+static size_t getPaddedLen(const StringRef &Str) {
+  const size_t Len = Str.size() + 1;
+  return (Len % 4 == 0) ? Len : Len + (4 - (Len % 4));
+}
+
+void addStringImm(const StringRef &Str, MachineInstrBuilder &MIB) {
+  const size_t PaddedLen = getPaddedLen(Str);
+  for (unsigned i = 0; i < PaddedLen; i += 4) {
+    // Add an operand for the 32-bits of chars or padding
+    MIB.addImm(convertCharsToWord(Str, i));
+  }
+}
+
+void addStringImm(const StringRef &Str, IRBuilder<> &B,
+                  std::vector<Value *> &Args) {
+  const size_t PaddedLen = getPaddedLen(Str);
+  for (unsigned i = 0; i < PaddedLen; i += 4) {
+    // Add a vector element for the 32-bits of chars or padding
+    Args.push_back(B.getInt32(convertCharsToWord(Str, i)));
+  }
+}
+
+void addNumImm(const APInt &Imm, MachineInstrBuilder &MIB) {
+  const auto Bitwidth = Imm.getBitWidth();
+  switch (Bitwidth) {
+  case 1:
+    break; // Already handled
+  case 8:
+  case 16:
+  case 32:
+    MIB.addImm(Imm.getZExtValue());
+    break;
+  case 64: {
+    uint64_t FullImm = Imm.getZExtValue();
+    uint32_t LowBits = FullImm & 0xffffffff;
+    uint32_t HighBits = (FullImm >> 32) & 0xffffffff;
+    MIB.addImm(LowBits).addImm(HighBits);
+    break;
+  }
+  default:
+    report_fatal_error("Unsupported constant bitwidth");
+  }
+}
+
+void buildOpName(Register Target, const StringRef &Name,
+                 MachineIRBuilder &MIRBuilder) {
+  if (!Name.empty()) {
+    auto MIB = MIRBuilder.buildInstr(SPIRV::OpName).addUse(Target);
+    addStringImm(Name, MIB);
+  }
+}
+
+void buildOpDecorate(Register Reg, MachineIRBuilder &MIRBuilder,
+                     SPIRV::Decoration::Decoration Dec,
+                     const std::vector<uint32_t> &DecArgs, StringRef StrImm) {
+  auto MIB = MIRBuilder.buildInstr(SPIRV::OpDecorate).addUse(Reg).addImm(Dec);
+  if (!StrImm.empty())
+    addStringImm(StrImm, MIB);
+  for (const auto &DecArg : DecArgs)
+    MIB.addImm(DecArg);
+}
+
+// TODO: maybe the following two functions should be handled in the subtarget
+// to allow for different OpenCL vs Vulkan handling.
+unsigned int storageClassToAddressSpace(SPIRV::StorageClass::StorageClass SC) {
+  switch (SC) {
+  case SPIRV::StorageClass::Function:
+    return 0;
+  case SPIRV::StorageClass::CrossWorkgroup:
+    return 1;
+  case SPIRV::StorageClass::UniformConstant:
+    return 2;
+  case SPIRV::StorageClass::Workgroup:
+    return 3;
+  case SPIRV::StorageClass::Generic:
+    return 4;
+  case SPIRV::StorageClass::Input:
+    return 7;
+  default:
+    llvm_unreachable("Unable to get address space id");
+  }
+}
+
+SPIRV::StorageClass::StorageClass
+addressSpaceToStorageClass(unsigned int AddrSpace) {
+  switch (AddrSpace) {
+  case 0:
+    return SPIRV::StorageClass::Function;
+  case 1:
+    return SPIRV::StorageClass::CrossWorkgroup;
+  case 2:
+    return SPIRV::StorageClass::UniformConstant;
+  case 3:
+    return SPIRV::StorageClass::Workgroup;
+  case 4:
+    return SPIRV::StorageClass::Generic;
+  case 7:
+    return SPIRV::StorageClass::Input;
+  default:
+    llvm_unreachable("Unknown address space");
+  }
+}
+
+SPIRV::MemorySemantics::MemorySemantics
+getMemSemanticsForStorageClass(SPIRV::StorageClass::StorageClass SC) {
+  switch (SC) {
+  case SPIRV::StorageClass::StorageBuffer:
+  case SPIRV::StorageClass::Uniform:
+    return SPIRV::MemorySemantics::UniformMemory;
+  case SPIRV::StorageClass::Workgroup:
+    return SPIRV::MemorySemantics::WorkgroupMemory;
+  case SPIRV::StorageClass::CrossWorkgroup:
+    return SPIRV::MemorySemantics::CrossWorkgroupMemory;
+  case SPIRV::StorageClass::AtomicCounter:
+    return SPIRV::MemorySemantics::AtomicCounterMemory;
+  case SPIRV::StorageClass::Image:
+    return SPIRV::MemorySemantics::ImageMemory;
+  default:
+    return SPIRV::MemorySemantics::None;
+  }
+}