AMDGPU : Add trap handler support.

[wdng]

Summary

Add trap handler support.

[wdng]

I am sorry for checking in this code into LLVM repo by mistake. Please let me know if you have any comments for this patch so that I will modify & checkin accordingly. Thanks!

[wdng]

lib/Target/AMDGPU/AMDGPUAnnotateKernelFeatures.cpp

193–194

Matts: "This also needs to handle debug.trap(), and the case when unreachable’s are turned into traps. There should also be tests specifically for the annotator."

test/CodeGen/AMDGPU/trap.ll

8–9

Matt: "Should check for the enabled feature bits in the kernel_code_t. This also doesn’t have anything setting enable_trap_handler"

[wdng]

lib/Target/AMDGPU/AMDGPUAnnotateKernelFeatures.cpp

193–194

How should we handle debug.trap()? For the case of unreachable's, should we invoke debug.trap()?

test/CodeGen/AMDGPU/trap.ll

8–9

I think "@llvm.trap()" will enable_trap_handler, right?

[arsenm]

lib/Target/AMDGPU/AMDGPUAnnotateKernelFeatures.cpp

193–194

There is a separate llvm.debugtrap intrinsic. I looked around SelectionDAG for other places that introduce ISD::TRAP. The expansion for ISD::DEBUGTRAP creates a TRAP. unreachable instructions also emit a trap if the DAG.getTarget().Options.TrapUnreachable is enabled.

test/CodeGen/AMDGPU/trap.ll

8–9

Yes, that is the problem. You need to be setting the trap handler bit

[wdng]

lib/Target/AMDGPU/AMDGPUAnnotateKernelFeatures.cpp

193–194

Yes, ISD::DEBUGTRAP has been replaced with ISD::TRAP at the legalization phase. So, do we still need to map llvm.debugtrap intrinsic with amdgpu-queue-ptr?

[tony-tye]

test/CodeGen/AMDGPU/trap.ll

8–9

I am not sure if we should have the compiler be responsible for setting the enable_trap_handler bit. In general I don't think the compiler can figure this out on a per kernel basis. Once we support function calls and indirect calls how could be know if the closure of all functions include some that need the trap handler?

Also, even if we did set the bit, for compute, the hardware CP microcode cannot do anything with it other than refuse to execute the kernel. Today I believe the CP micro code ignores this bit and always enables the trap handler as it is needed for CWSR (context switching).

So it seems the presence of a trap handler is more a function of the environment that will execute the kernel than the kernel itself.

So perhaps we should simply not define the trap handler bit. The code object is already marked as requiring the HSA environment and perhaps this implies the presence of a trap handler.

I suspect that graphics kernels have their own "environment" they execute in and may not support trap handlers. If they do not then executing an S_TRAP will simply be a NOP which will not halt the shader.

[arsenm]

lib/Target/AMDGPU/AMDGPUAnnotateKernelFeatures.cpp

193–194

Yes. You should add a test which uses this. As it is this will fail because the queue ptr won't be enabled

test/CodeGen/AMDGPU/trap.ll

8–9

I don't think that implies the presence of the trap handler. This is a field in the kernel_code_t, so we should set it. We can figure out a conservative setting in the future whenever indirect calls are needed.

I thought enabling this also required reserving 16 SGPRs?

[wdng]

lib/Target/AMDGPU/AMDGPUAnnotateKernelFeatures.cpp

193–194

To enable or disable queue ptr, it depends whether we have used "-mtriple=amdgcn--amdhsa" or not. Since ISD::DEBUGTRAP has been replaced with ISD::TRAP, I don't think the test will fail if we compile llvm.debugtrap with "-mtriple=amdgcn--amdhsa", correct?

[arsenm]

lib/Target/AMDGPU/AMDGPUAnnotateKernelFeatures.cpp

193–194

That occurs in the DAG. Here you are trying to identify all the situations where ISD::TRAP will be emitted from the IR. Eventually I want to replace how the ABI is lowered so we don't need to do this anymore, but for now you must predict all of the traps.

[wdng]

test/CodeGen/AMDGPU/trap.ll

8–9

So, once compiler detects "llvm.trap()", shall we set up the trap handler bit in the kernel_code_t? I don't know why we need to add the trap handler bit. Does the bit setting will imply the pretense of the trap handler and otherwise it won't?

[wdng]

1. Fixed format issue.
2. Add test case for llvm.debugtrap.
3. Add non-hsa path trap handler.
4. Confirm that 16 reserved SGPRs are not part of the regular SGPRs.
5. Add trap handler bit set in the amd_kernel_code_t.

[wdng]

vim added some indentations by mistake, removed indentations & empty line.

[arsenm]

lib/Target/AMDGPU/SIISelLowering.cpp

276–279

This does not need an else

1794

const DebugLoc &DL

1986

return on next line

lib/Target/AMDGPU/SIMachineFunctionInfo.h

253 ↗

(On Diff #85971)

This should return void and doesn't need HSA in the name

test/CodeGen/AMDGPU/trap.ll

1–3

This should not be checking stderr since the test should pass with HSA

2

This needs a not

9

Check still missing for the enable_trap_handler bit in kernel_code_t

27

GCN should be replaced with ERROR or something like that

34

You should add another test which ends in unreachable. As far as I can tell there is no flag to set TrapUnreachable, and we don't enable it now, but this should catch it if that ever breaks

34

You also add another test which needs to enable the queue ptr for a different feature

[tony-tye]

test/CodeGen/AMDGPU/trap.ll

8–9

It seems that the presence of a trap handler is a property of producing a code object to be executed using the HSA environment. Under ROCM all kernels will be executed with a trap handler and that trap handler will use the HSA ABI. Other environments may have different trap handlers, or no trap handler at all, and the code generated for llvm.trap/debugtrap would change accordingly. The bit in amd_kernel_code_t would then be set according to the demands of environment independent of using traps.

Having a trap handler present does cause an extra 16 SGPRs to be allocated (which should be taken into account when determining wave occupancy), but these are in addition to the SGRs used by the generated code (so no need to reserve them or include them in the SGPR count in the amd_kernel_code_t).

[arsenm]

test/CodeGen/AMDGPU/trap.ll

8–9

Do we still need to make sure we leave 16 unallocated for the implicit use? From the ABI spec it seems clear to not include them in the reported SGPR count, but do we need to ensure reported total + 16 is below the hardware limit?

[tony-tye]

test/CodeGen/AMDGPU/trap.ll

8–9

The hardware limit is the maximum non-privileged SGPRs that can be accessed using the instruction register encoding plus the 16 privileged trap registers. The trap temps are only allocated IF a trap handler is enabled, and are only accessible when executing the trap handler (using special instruction register encoding).

So from the point of view of determining the limit of non-privileged SGPRs that can be allocated the presence of a trap handler can be ignored. The fact they are allocated needs to be considered when determining the number of waves that will fit on a CU.

[wdng]

Address code reviews.

[wdng]

lib/Target/AMDGPU/SIISelLowering.cpp

276–279

I think we need leave the else here. I assume default setting is Legal, but looks like we need to explicitly specify it otherwise it will throw an error.

test/CodeGen/AMDGPU/trap.ll

34

What feature should I test?

34

What kinds of instructiosn will trigger the trap instruction?

[tony-tye]

lib/Target/AMDGPU/SIISelLowering.cpp

276–279

The else implements the non-HSA expansion that generates an end_prm which is used for graphics.

[kzhuravl]

Can you upload a full diff? Thanks

[wdng]

lib/Target/AMDGPU/SIISelLowering.cpp

276–279

yes, so I think we need to keep the else part.

[wdng]

Upload a full diff.

[arsenm]

lib/Target/AMDGPU/SIMachineFunctionInfo.h

265–266 ↗

(On Diff #86133)

Remove Ptr from the names

test/CodeGen/AMDGPU/trap.ll

34

You can do an address space cast from LDS to flat or do something with the queue ptr intrinsic

[tony-tye]

lib/Target/AMDGPU/SIISelLowering.cpp

276

In discussion with @kzhuravl the suggestion was to use a subtarget query to determine the trap abi. Currently there are two ABIs, one for HSA and one for graphics, but in the future there could be others.

lib/Target/AMDGPU/SIMachineFunctionInfo.h

253 ↗

(On Diff #86133)

As mentioned in another comment, I think determining if there is a trap handler should be determined from the environment part of the triple. Check with @kzhuravl where the query should be put.

[kzhuravl]

As @tony-tye mentioned trap handler is more of an environment property (see https://reviews.llvm.org/D26010#inline-251754), so it should be moved into AMDGPUSubtarget or SISubtarget, and we should be setting trap handler bit in pgm rsrc 2 based on the subtarget query.

In order to allow extensibility in the future the suggestion is to introduce an enumeration with available trap handler ABIs. Currently there will be only 2, something along the lines:

enum TrapHandlerAbi {
  TrapHandlerAbiNone = 0,
  TrapHandlerAbiHsa = 1
}

Also, introduce a subtarget query that returns trap handler abi:

TrapHandlerAbi() {
  if (isAmdHsaOs)
    return TrapHandlerAbiHsa;
  return TrapHandlerAbiNone;
}

Every ISD::TRAP is going to be lowered into S_TRAP_PSEUDO regardless of the trap handler ABI.
In order to expand S_TRAP_PSEUDO, we will have to query what kind of trap handler ABI does the subtarget have. If it is an HSA, then move 1 into v0, and queue ptr into s[0:1]. If trap ABI is none, then lower it to s_engpgm.

@tony-tye did I miss anything? @arsenm, @wdng what do you think?

Thanks

lib/Target/AMDGPU/AMDKernelCodeT.h

200–202

This is not right. Trap handler bit is located in amd_compute_pgm_rsrc2_t with the field name enable_trap_handler.

Refer to:

• https://github.com/RadeonOpenCompute/ROCR-Runtime/blob/master/src/inc/amd_hsa_kernel_code.h#L114
• https://github.com/RadeonOpenCompute/ROCm-ComputeABI-Doc/blob/master/AMDGPU-ABI.md#compute-shader-program-settings-2-amd_compute_pgm_rsrc2_t

lib/Target/AMDGPU/SIISelLowering.cpp

278–279

If we go with the approach in my latest comment, custom lowering won't be needed.

1792–1802

If we go with the approach in my latest comment, this should switch on ABI type, for HSA ABI this code will be used. For "none" ABIs s_engpgm should be used (without error or warning).

2495–2511

If we go with the approach in my latest comment, this will be gone.

lib/Target/AMDGPU/SIMachineFunctionInfo.h

253 ↗

(On Diff #86133)

If we go with the approach in my latest comment, this will be moved to subtarget class.

lib/Target/AMDGPU/Utils/AMDKernelCodeTInfo.h

90

enable_trap_handler should be here (see comment with links).

116

There is no is_trap_handler_supported in CODEPROP (see comment with links).

[kzhuravl]

Also, we need to update waves-per-eu calculations.

[wdng]

In D26010#660492, @kzhuravl wrote:

Also, we need to update waves-per-eu calculations.

What kinds of changes should we made for the waves-per-eu?

[wdng]

1. Restructure code to make it extensible based on code reviews.
2. Move trap handler bit in amd_compute_pgm_rsrc2_t.

[wdng]

1. updated patch with complete & full diff
2. Restructure code to make it extensible based on code reviews.
3. Move trap handler bit in amd_compute_pgm_rsrc2_t.

[arsenm]

lib/Target/AMDGPU/AMDGPUSubtarget.h

106

Comment unnecessary, also nothing HSA specific here

271

Return enum type

271–276

Badly formatted. return ternary operator

lib/Target/AMDGPU/SIISelLowering.cpp

1805

Compare to enum value

1817–1821

Formatting of previous code needs to be fixed

1822–1823

Else on previous line. Instead of else you can early return on the error path

1829–1830

Badly formatted. You already have MF and get the Function* above, so you don't need double getParent

lib/Target/AMDGPU/SIMachineFunctionInfo.cpp

136–137 ↗

(On Diff #86456)

This should be removed. The trap handler should be set based on a subtarget feature

[wdng]

lib/Target/AMDGPU/SIMachineFunctionInfo.cpp

136–137 ↗

(On Diff #86456)

S_00B84C_TRAP_HANDLER(MFI->hasTrapHandler()) needs to know whether trap handler has been set. Where should we the based on the sub- target feature?

[kzhuravl]

lib/Target/AMDGPU/SIMachineFunctionInfo.cpp

136–137 ↗

(On Diff #86456)

Set enable_trap_handler bit if Subtarget->getTrapHandlerAbi() != TrapHandlerAbiNone?

lib/Target/AMDGPU/SIMachineFunctionInfo.h

154 ↗

(On Diff #86456)

not needed.

[kzhuravl]

lib/Target/AMDGPU/SIISelLowering.cpp

1819

@tony-tye, do we want to use other code for s_trap for llvm.debugtrap?

[tony-tye]

lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

641

Should this be querying the TrapHandlerAbi using getTrapHandlerAbi() from the subtarget and setting to TRUE if not equal to TrapHandlerAbiNone ?

lib/Target/AMDGPU/AMDGPUSubtarget.h

107

Should this be deleted now as there is getTrapHandlerAbi() instead?

lib/Target/AMDGPU/SIISelLowering.cpp

1819

Should the 0x1 constant be a named enumeration?

Should we use a different values for llvm.trap than for llvm.debugtrap? If so the handling of llvm.debugtrap should not be converted to llvm.trap for AMDGPU.

Have we checked with teh HSA Runtime to see what codes the HSA trap handler is expecting?

1824–1831

Should we be emitting any diagnostic here? If there is no trap handler then doesn't that imply that the environment wants an S_ENDPGM to be generated instead? That is not an error, it is that the environment demands as the implementation. For example, that is what the graphics environment wants.

lib/Target/AMDGPU/SIMachineFunctionInfo.cpp

67 ↗

(On Diff #86456)

Delete too?

136–137 ↗

(On Diff #86456)

Delete too?

lib/Target/AMDGPU/SIMachineFunctionInfo.h

154 ↗

(On Diff #86456)

Should this be deleted? See above.

261–263 ↗

(On Diff #86456)

Delete as now we have the trap ABI query instead.

[tony-tye]

Also need to use the trap handler ABI query to see if there is a trap handler, and if there is add the TRAP_HANDLER_SGPR_COUNT to the number of SGPRs budgeted for the wave in determining the number of waves per EU calculation. TRAP_HANDLER_SGPR_COUNT is 16 for GFX6 onwards.

[wdng]

Address code reviews.

[arsenm]

lib/Target/AMDGPU/AMDGPUSubtarget.h

272–275

Still using return after else. Should be return ternary operator

335–337

This should be a subtarget feature. Returning this is still a constant based purely on the triple. See FeatureUnalignedBufferAccess for an example

test/CodeGen/AMDGPU/trap.ll

1–3

You should have a run line with no subtarget features enabled, and one each explicitly enabling and disabling the trap handler subtarget feature

[wdng]

lib/Target/AMDGPU/SIISelLowering.cpp

1805

Say, if we enable trap handler via subtarget feature, should we need to change the if here?

[wdng]

We enable trap handler support once "-mtriple=amdgcn--amdhsa" is specified. So use the following code to enable trap handler.

bool isTrapHandlerEnabled() const {
  return getTrapHandlerAbi() == TrapHandlerAbiHsa;
}

Not sure why we need to add a subtarget feature. Say if we replace code 328-330 using subtarget feature, do we want to enable trap handler by using both "-mtriple=amdgcn--amdhsa" and "-mattr=+enable-trap-handler"?

def FeatureTrapHandler: SubtargetFeature<"enable-trap-handler",
  "EnableTrapHandler",
  "true",
  "Enable trap handler support"
>;

At SIISelLowering.cpp we enable trap handler based on "mtriple=amdgcn--amdhsa", shall we change to based on the subtarget feature input?

if (Subtarget->getTrapHandlerAbi() == SISubtarget::TrapHandlerAbiHsa) {
 . . . 
}

What do you think?

[kzhuravl]

I do not see updates to waves-per-eu calculations for SGPRs. Did you upload the correct diff?

[wdng]

In D26010#665171, @kzhuravl wrote:

I do not see updates to waves-per-eu calculations for SGPRs. Did you upload the correct diff?

Yes, I did. Introduced TRAP_HANDLER_SGPR_COUNT and add it into ExtraSGPRs.

[tony-tye]

lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

482–484

I do not think enabling the trap handler reduces the number of available SGPRs. The hardware allocates and provides access to theses "in addition" to the regular SGPRs. If this is reserving registers then it should be deleted.

[tony-tye]

lib/Target/AMDGPU/AMDGPUSubtarget.h

73

Add enumeration for S_TRAP codes:

enum TrapCode {

TrapCodeLLVMTrap = 1,
TrapCodeLLVMDebugTrap = 2

};

335–337

Could you explain this more? The trap handler being present is determined by the environment part of the triple. Currently only the HSA environment uses a trap handler.

336

In order to allow additional trap handler ABIs this should be:
getTrapHandlerAbi() != TrapHandlerAbiNone

511

I do not see this being used anywhere. I think it should be used in the waves_per_eu calculation.

lib/Target/AMDGPU/SIISelLowering.cpp

1819

I checked with the HSA Runtime and currently the trap code is not being consulted. So I propose adding an enum for the codes being used for s_trap and use the value here. It would be better to different code for llvm.trap and llvm.debugtrap. Would that require S_TRAP_PSEUDO to have an immediate operand that is set to the trap code used in the S_TRAP instruction.

test/CodeGen/AMDGPU/trap.ll

22

Would like it to be:
; HSA: s_trap 2

[wdng]

Address code reviews.

[arsenm]

lib/Target/AMDGPU/AMDGPU.td

70

Remove the word enable-

[wdng]

1. Separate llvm.debugtrap and llvm.trap.
2. Decrease 16 from available SGPRs once trap handler is enabled.

[arsenm]

lib/Target/AMDGPU/SIISelLowering.cpp

1824

Can we have an enum for these values somewhere instead of hard coding 1?

1833–1838

This duplicates nearly the entire other switch case. They should be consolidated based on the immediate argument and handled with the single pseudo

lib/Target/AMDGPU/SIInstructions.td

117

This conflicts with the VPseudoInst type. You should remove this and change to SPseudoInstSI

121–127

You do not need a second pseudo. You should just add an operand to the other pseudo which is just mimicking the operands of the physical instruction.

lib/Target/AMDGPU/SIRegisterInfo.cpp

1166

Variable naming convention

test/CodeGen/AMDGPU/trap.ll

1

Having FUNC and HSA-FUNC doesn't make sense. Replace these both with just GCN

2

Missing a run line with mesa triple

11–13

HSA:

20–21

The check prefix names should be like HSA-TRAP, HSA-NOTRAP, MESA-TRAP, MESA-NOTRAP

[wdng]

Address code reviews.

[arsenm]

lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

1034–1045

You should not and do not need to touch generic code

lib/Target/AMDGPU/SIISelLowering.cpp

276

DEBUGTRAP should also be legal, you are handling the differences in the selection pattern and custom inserter

lib/Target/AMDGPU/SIInstructions.td

115

You should use i16imm for the operand type to match the instruction (not that it matters much) and name it the same too (which matters more): ( ins i16imm:$simm16)

403

Can you define pseudo-enums for these constants (see for example SRCMODS)

test/CodeGen/AMDGPU/trap.ll

1–4

GCN and HSA are not alternative check prefixes. It doesn't make sense to have it on one of these but not the other. HSA-TRAP for the disabled trap handler feature is broken

3–4

These should both explicitly use the mesa triple, and enable/disable the trap handler in each. The ones which should error are also missing the not.

24–25

These check lines should not be able to coexist

[wdng]

lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

1034–1045

I accidentally touched those lines and have reverted them, thanks!

lib/Target/AMDGPU/SIISelLowering.cpp

276

Yes, I did set DEBUGTRAP to Legal for next pattern matching at isel.

test/CodeGen/AMDGPU/trap.ll

3–4

As per our discussion with @tony-tye last week, looks like we want to issue warning instead of error, correct?

[wdng]

Address code reviews.

[tony-tye]

lib/Target/AMDGPU/SIRegisterInfo.cpp

1166

Is this the right place to useSISubtarget::TRAP_HANDLER_SGPR_COUNT? The presence of a trap handler does not reduce the number of SGPRs addressable for allocation by the register allocator. It only affects the wave occupancy calculation (I did not see any change to that code).

[wdng]

lib/Target/AMDGPU/SIRegisterInfo.cpp

1166

Yes, this is the right place. -mtriple=amdgcn--amdhsa defaults trap handler feature, so ST.isTrapHandlerEnabled() is true, then 16 SGPRs will be deduced otherwise 0 SGPRs will be deducted. @kzhuravl What do you think?

[arsenm]

lib/Target/AMDGPU/AMDKernelCodeT.h

198

This change should be fully reverted

lib/Target/AMDGPU/SIRegisterInfo.cpp

1166

I don't think so. This is the allocated number

test/CodeGen/AMDGPU/trap.ll

5–9

You seem to have removed the HSA enable/disable and do mesa twice?

7

You should also check for the rsrc2 bit is set

[wdng]

test/CodeGen/AMDGPU/trap.ll

5–9

I also do HSA enable/disable twice:
; enable HSA trap handler
; RUN: llc -mtriple=amdgcn--amdhsa -mattr=+trap-handler -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=HSA-TRAP %s
; disable HSA trap handler
; RUN: llc -mtriple=amdgcn--amdhsa -mattr=-trap-handler -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=NO-HSA-TRAP %s
; disable HSA trap handler to catch warning for llvm.debugtrap since llvm.trap doesn't issue any warnings
; RUN: llc -mtriple=amdgcn--amdhsa -mattr=-trap-handler -verify-machineinstrs < %s 2>&1 | FileCheck -check-prefix=GCN -check-prefix=GCN-WARNING %s

[wdng]

Address code reviews.

[arsenm]

lib/Target/AMDGPU/SIISelLowering.cpp

1809

Should have enum (probably in SIDefines.h) for the values put in

[wdng]

lib/Target/AMDGPU/SIISelLowering.cpp

1809

This probably is not an enum. It looks like an interface with HSA is that we need to set v0 to 1:

v0 <- 1

[kzhuravl]

lib/Target/AMDGPU/SIISelLowering.cpp

1809

I think defining it in SIDefines.h for now is OK. I am working on some restructuring to shared header files, and will include it in the restructuring.

[kzhuravl]

lib/Target/AMDGPU/SIRegisterInfo.cpp

1166

I think this should be moved to getMaxNumSGPRs. We still want to report the correct number of addressable SGPRs regardless of trap handler.

[wdng]

lib/Target/AMDGPU/SIRegisterInfo.cpp

1166

I think in terms of functionality it's the same no matter whether we move the code into getMaxNumSGPRs or not. As getMaxNumSGPRs still invokes this function to calculate MaxNumSGPRs and MaxNumAddressableSGPRs. What do you think?

[kzhuravl]

lib/Target/AMDGPU/SIRegisterInfo.cpp

1166

Not really. We need to record addressable number of SGPRs in the metadata for tools such as debugger and profiler. The addressable number of SGPRs should be recorded correctly regardless of trap handler (102 for 8+, 104 for everything else).

https://github.com/llvm-mirror/llvm/blob/master/lib/Target/AMDGPU/MCTargetDesc/AMDGPURuntimeMD.cpp#L422

[wdng]

1. Add rsrc2 trap handler bit check in lit tests.
2. Moved the trap handler SGPRs calculation into getMaxNumSGPRs
3. Add a trap operand enum for the time being to replace a constant value 1.

[kzhuravl]

lib/Target/AMDGPU/SIRegisterInfo.cpp

1166

Can we drop waves-per-eu calculations from this patch? We need to change few things to math, which can be done as a separate patch.

[wdng]

Drop waves-per-eu calculations.

[arsenm]

test/CodeGen/AMDGPU/trap.ll

1–3

Missing GCN check prefix

12

You don't need a check prefix for NO-RSRC2-BIT. That is too specific, the checks should be based on the environment type/options

13

This is just the comment printed. You should check the actual register config register value too

[wdng]

Address code reviews.

[tony-tye]

lib/Target/AMDGPU/AMDGPUSubtarget.h

74–82

After discussion with the HSA runtime team it was decided to use separate S_TRAP codes for each purpose. The current codes would be:

enum TrapCode {
  TrapCodeBreakpoint = 0,
  TrapCodeLLVMTrap = 1,
  TrapCodeLLVMDebugTrap = 2,
  TrapCodeHSADebugTrap = 3
};

These need documenting in the AMDGPU LLVM Spec page.

For now we do not have an intrinsic for the HSA debugtrap (which takes an argument) so remove the TrapRegValues enum.

lib/Target/AMDGPU/SIISelLowering.cpp

1808–1809

Delete this as the traps generated for llvm.trap and llvm.debugtrap will not pass in any value.

[tony-tye]

lib/Target/AMDGPU/SIISelLowering.cpp

1823

Delete this as the traps generated for llvm.trap and llvm.debugtrap will not pass in any value.

[tony-tye]

lib/Target/AMDGPU/AMDGPUSubtarget.h

74–82

The TRAP handler codes can be defined in a new section of:

http://llvm.org/docs/AMDGPUUsage.html

[wdng]

Added enum for better HSA interface and removed mov instruction for V0 as llvm.trap and llvm.debugtrap will not pass in any value.

[wdng]

Upload correct diff file.

[tony-tye]

lib/Target/AMDGPU/SIISelLowering.cpp

1832

Since llvm.debugtrap is not defined as NORETURN then it seems after a warning it should be lowered to a S_NOP not an S_ENDPGM.

llvm.trap is defined as NORETURN so generating S_ENDPGM seems the right choice when there is no trap handler.

[wdng]

Add S_NOP.

[wdng]

Upload full correct diff.

[tony-tye]

lib/Target/AMDGPU/SIISelLowering.cpp

1823

Since TrapType is an enum type should this be a switch with a default: that asserts? Currently the code will silently ignore any new trap types that are added (an assert would avoid that).

[wdng]

Address code reviews.

[arsenm]

lib/Target/AMDGPU/AMDGPUSubtarget.cpp

87

Sort

lib/Target/AMDGPU/AMDGPUSubtarget.h

100

Can you sort this later until after EnableXNACK? I think the unaligned options should stay together

lib/Target/AMDGPU/SIISelLowering.cpp

1825–1827

Asserting !the switch case in the switch case is ugly. There's no reason to special case this, just put unreachable in the default case

1838

addImm on next line.

We don't really need to emit anything here though

1841–1844

Just let it default

1846

llvm_unreachable

[tony-tye]

lib/Target/AMDGPU/SIISelLowering.cpp

1838

Discussed with @arsenm and leaving as a NOP does give some flexibility asa tool may want to patch these points. Since llvm.debugtrap is not being generally used it does not hurt to have a NOP.

[wdng]

Address code reviews.

[wdng]

Discussed with @tony-tye , keep only 2 CASEs that can happen for the pseudo trap is llvmtrap and llvmdebugtrap in switch block.

[arsenm]

LGTM except for minor issues

lib/Target/AMDGPU/SIISelLowering.cpp

1840

Remove dead line

lib/Target/AMDGPU/SIInstructions.td

400

Space after :

401

Ditto

lib/CodeGen/SelectionDAG/LegalizeDAG.cpp

case ISD::READCYCLECOUNTER:
     break;
   case ISD::READ_REGISTER:
   case ISD::WRITE_REGISTER:
     // Named register is legal in the DAG, but blocked by register name
     // selection if not implemented by target (to chose the correct register)
     // They'll be converted to Copy(To/From)Reg.
     Action = TargetLowering::Legal;
     break;
   case ISD::DEBUGTRAP:
     Action = TLI.getOperationAction(Node->getOpcode(), Node->getValueType(0));
     if (Action == TargetLowering::Expand) {
       // replace ISD::DEBUGTRAP with ISD::TRAP
       SDValue NewVal;
+      const SDValue TrapValue = DAG.getTargetConstant(2, SDLoc(Node), MVT::i16);
       NewVal = DAG.getNode(ISD::TRAP, SDLoc(Node), Node->getVTList(),
-                           Node->getOperand(0));
+                           Node->getOperand(0), TrapValue);
       ReplaceNode(Node, NewVal.getNode());
       LegalizeOp(NewVal.getNode());
       return;
     }
     break;
 
   default:
     if (Node->getOpcode() >= ISD::BUILTIN_OP_END) {
       Action = TargetLowering::Legal;
     } else {
       Action = TLI.getOperationAction(Node->getOpcode(), Node->getValueType(0));
     }

lib/Target/AMDGPU/AMDGPU.td

 >;
 
 def FeatureUnalignedBufferAccess : SubtargetFeature<"unaligned-buffer-access",
   "UnalignedBufferAccess",
   "true",
   "Support unaligned global loads and stores"
 >;
 
+def FeatureTrapHandler: SubtargetFeature<"trap-handler",
+  "TrapHandler",
+  "true",
+  "Trap handler support"
+>;
+
 def FeatureUnalignedScratchAccess : SubtargetFeature<"unaligned-scratch-access",
   "UnalignedScratchAccess",
   "true",
   "Support unaligned scratch loads and stores"
 >;
 
 // XNACK is disabled if SH_MEM_CONFIG.ADDRESS_MODE = GPUVM on chips that support
 // XNACK. The current default kernel driver setting is:

lib/Target/AMDGPU/AMDGPUAnnotateKernelFeatures.cpp

static const StringRef IntrinsicToAttr[][2] = {
     // .x omitted
     { "llvm.r600.read.tidig.y", "amdgpu-work-item-id-y" },
     { "llvm.r600.read.tidig.z", "amdgpu-work-item-id-z" }
   };
 
   static const StringRef HSAIntrinsicToAttr[][2] = {
     { "llvm.amdgcn.dispatch.ptr", "amdgpu-dispatch-ptr" },
     { "llvm.amdgcn.queue.ptr", "amdgpu-queue-ptr" },
     { "llvm.amdgcn.dispatch.id", "amdgpu-dispatch-id" },
-  { "llvm.trap", "amdgpu-queue-ptr" }
+    { "llvm.trap", "amdgpu-queue-ptr" },
+    { "llvm.debugtrap", "amdgpu-queue-ptr" }
   };
 
   // TODO: We should not add the attributes if the known compile time workgroup
   // size is 1 for y/z.
 
   // TODO: Intrinsics that require queue ptr.
 
   // We do not need to note the x workitem or workgroup id because they are

lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp

if (STM.getGeneration() >= AMDGPUSubtarget::SOUTHERN_ISLANDS) {
                                     Twine(KernelInfo.DebuggerWavefrontPrivateSegmentOffsetSGPR), false);
         OutStreamer->emitRawComment(" DebuggerPrivateSegmentBufferSGPR: s" +
                                     Twine(KernelInfo.DebuggerPrivateSegmentBufferSGPR), false);
       }
 
       OutStreamer->emitRawComment(" COMPUTE_PGM_RSRC2:USER_SGPR: " +
                                   Twine(G_00B84C_USER_SGPR(KernelInfo.ComputePGMRSrc2)),
                                   false);
+      OutStreamer->emitRawComment(" COMPUTE_PGM_RSRC2:TRAP_HANDLER: " +
+                                  Twine(G_00B84C_TRAP_HANDLER(KernelInfo.ComputePGMRSrc2)),
+                                  false);
       OutStreamer->emitRawComment(" COMPUTE_PGM_RSRC2:TGID_X_EN: " +
                                   Twine(G_00B84C_TGID_X_EN(KernelInfo.ComputePGMRSrc2)),
                                   false);
       OutStreamer->emitRawComment(" COMPUTE_PGM_RSRC2:TGID_Y_EN: " +
                                   Twine(G_00B84C_TGID_Y_EN(KernelInfo.ComputePGMRSrc2)),
                                   false);
       OutStreamer->emitRawComment(" COMPUTE_PGM_RSRC2:TGID_Z_EN: " +
                                   Twine(G_00B84C_TGID_Z_EN(KernelInfo.ComputePGMRSrc2)),
if (STM.getGeneration() < SISubtarget::VOLCANIC_ISLANDS) {
     if (FlatUsed)
       ExtraSGPRs = 4;
   } else {
     if (STM.isXNACKEnabled())
       ExtraSGPRs = 4;
 
     if (FlatUsed)
       ExtraSGPRs = 6;
   }
 
   // Record first reserved register and reserved register count fields, and
   // update max register counts if "amdgpu-debugger-reserve-regs" attribute was
   // requested.
   ProgInfo.ReservedVGPRFirst = STM.debuggerReserveRegs() ? MaxVGPR + 1 : 0;
   ProgInfo.ReservedVGPRCount = RI->getNumDebuggerReservedVGPRs(STM);
 
   // Update DebuggerWavefrontPrivateSegmentOffsetSGPR and
   // DebuggerPrivateSegmentBufferSGPR fields if "amdgpu-debugger-emit-prologue"
   // attribute was requested.
void AMDGPUAsmPrinter::getSIProgramInfo(SIProgramInfo &ProgInfo,
   if (MFI->hasWorkItemIDZ())
     TIDIGCompCnt = 2;
   else if (MFI->hasWorkItemIDY())
     TIDIGCompCnt = 1;
 
   ProgInfo.ComputePGMRSrc2 =
       S_00B84C_SCRATCH_EN(ProgInfo.ScratchBlocks > 0) |
       S_00B84C_USER_SGPR(MFI->getNumUserSGPRs()) |
+      S_00B84C_TRAP_HANDLER(STM.isTrapHandlerEnabled()) |
       S_00B84C_TGID_X_EN(MFI->hasWorkGroupIDX()) |
       S_00B84C_TGID_Y_EN(MFI->hasWorkGroupIDY()) |
       S_00B84C_TGID_Z_EN(MFI->hasWorkGroupIDZ()) |
       S_00B84C_TG_SIZE_EN(MFI->hasWorkGroupInfo()) |
       S_00B84C_TIDIG_COMP_CNT(TIDIGCompCnt) |
       S_00B84C_EXCP_EN_MSB(0) |
       S_00B84C_LDS_SIZE(ProgInfo.LDSBlocks) |
       S_00B84C_EXCP_EN(0);

lib/Target/AMDGPU/AMDGPUSubtarget.h

enum {
     ISAVersion8_0_0,
     ISAVersion8_0_1,
     ISAVersion8_0_2,
     ISAVersion8_0_3,
     ISAVersion8_0_4,
     ISAVersion8_1_0,
   };
 
+  enum TrapHandlerAbi {
+    TrapHandlerAbiNone = 0,
+    TrapHandlerAbiHsa = 1
+  };
+
+  enum TrapCode {
+    TrapCodeLLVMTrap = 1,
+    TrapCodeLLVMDebugTrap = 2
+  };
+
 protected:
   // Basic subtarget description.
   Triple TargetTriple;
   Generation Gen;
   unsigned IsaVersion;
   unsigned WavefrontSize;
   int LocalMemorySize;
   int LDSBankCount;
   unsigned MaxPrivateElementSize;
 
   // Possibly statically set by tablegen, but may want to be overridden.
   bool FastFMAF32;
   bool HalfRate64Ops;
 
   // Dynamially set bits that enable features.
   bool FP32Denormals;
   bool FP64FP16Denormals;
   bool FPExceptions;
   bool FlatForGlobal;
   bool NoAddr64;
   bool UnalignedScratchAccess;
+  bool TrapHandler;
   bool UnalignedBufferAccess;
   bool EnableXNACK;
   bool DebuggerInsertNops;
   bool DebuggerReserveRegs;
   bool DebuggerEmitPrologue;
 
   // Used as options.
   bool EnableVGPRSpilling;
   bool EnablePromoteAlloca;
   bool EnableLoadStoreOpt;
   bool EnableUnsafeDSOffsetFolding;
   bool EnableSIScheduler;
   bool DumpCode;
 
   // Subtarget statically properties set by tablegen
public:
   bool hasBORROW() const {
     return (getGeneration() >= EVERGREEN);
   }
 
   bool hasCaymanISA() const {
     return CaymanISA;
   }
 
+  TrapHandlerAbi getTrapHandlerAbi() const {
+    return isAmdHsaOS() ? TrapHandlerAbiHsa : TrapHandlerAbiNone;
+  }
+
   bool isPromoteAllocaEnabled() const {
     return EnablePromoteAlloca;
   }
 
   bool unsafeDSOffsetFoldingEnabled() const {
     return EnableUnsafeDSOffsetFolding;
   }
 
   bool dumpCode() const {
     return DumpCode;
public:
   bool hasUnalignedBufferAccess() const {
     return UnalignedBufferAccess;
   }
 
   bool hasUnalignedScratchAccess() const {
     return UnalignedScratchAccess;
   }
 
+  bool isTrapHandlerEnabled() const {
+    return TrapHandler;
+  }
+
   bool isXNACKEnabled() const {
     return EnableXNACK;
   }
 
   bool isMesaKernel(const MachineFunction &MF) const {
     return isMesa3DOS() && !AMDGPU::isShader(MF.getFunction()->getCallingConv());
   }
 
   // Covers VS/PS/CS graphics shaders
   bool isMesaGfxShader(const MachineFunction &MF) const {
     return isMesa3DOS() && AMDGPU::isShader(MF.getFunction()->getCallingConv());
   }
 
   bool isAmdCodeObjectV2(const MachineFunction &MF) const {
     return isAmdHsaOS() || isMesaKernel(MF);
public:
   }
 };
 
 class SISubtarget final : public AMDGPUSubtarget {
 public:
   enum {
     // The closed Vulkan driver sets 96, which limits the wave count to 8 but
     // doesn't spill SGPRs as much as when 80 is set.
-    FIXED_SGPR_COUNT_FOR_INIT_BUG = 96
+    FIXED_SGPR_COUNT_FOR_INIT_BUG = 96,
+    TRAP_HANDLER_SGPR_COUNT = 16
   };
 
 private:
   SIInstrInfo InstrInfo;
   SIFrameLowering FrameLowering;
   SITargetLowering TLInfo;
   std::unique_ptr<GISelAccessor> GISel;
 

lib/Target/AMDGPU/AMDGPUSubtarget.cpp

AMDGPUSubtarget::initializeSubtargetDependencies(const Triple &TT,
   // double precision rate, so don't enable by default.
   //
   // We want to be able to turn these off, but making this a subtarget feature
   // for SI has the unhelpful behavior that it unsets everything else if you
   // disable it.
 
   SmallString<256> FullFS("+promote-alloca,+fp64-fp16-denormals,+load-store-opt,");
   if (isAmdHsaOS()) // Turn on FlatForGlobal for HSA.
-    FullFS += "+flat-for-global,+unaligned-buffer-access,";
+    FullFS += "+flat-for-global,+unaligned-buffer-access,+trap-handler,";
 
   FullFS += FS;
 
   ParseSubtargetFeatures(GPU, FullFS);
 
   // FIXME: I don't think think Evergreen has any useful support for
   // denormals, but should be checked. Should we issue a warning somewhere
   // if someone tries to enable these?
: AMDGPUGenSubtargetInfo(TT, GPU, FS),
     HalfRate64Ops(false),
 
     FP32Denormals(false),
     FP64FP16Denormals(false),
     FPExceptions(false),
     FlatForGlobal(false),
     NoAddr64(false),
     UnalignedScratchAccess(false),
+    TrapHandler(false),
     UnalignedBufferAccess(false),
 
     EnableXNACK(false),
     DebuggerInsertNops(false),
     DebuggerReserveRegs(false),
     DebuggerEmitPrologue(false),
 
     EnableVGPRSpilling(false),

lib/Target/AMDGPU/AMDKernelCodeT.h

enum amd_code_property_mask_t {
   AMD_CODE_PROPERTY_IS_DEBUG_SUPPORTED = ((1 << AMD_CODE_PROPERTY_IS_DEBUG_SUPPORTED_WIDTH) - 1) << AMD_CODE_PROPERTY_IS_DEBUG_SUPPORTED_SHIFT,
 
   AMD_CODE_PROPERTY_IS_XNACK_SUPPORTED_SHIFT = 22,
   AMD_CODE_PROPERTY_IS_XNACK_SUPPORTED_WIDTH = 1,
   AMD_CODE_PROPERTY_IS_XNACK_SUPPORTED = ((1 << AMD_CODE_PROPERTY_IS_XNACK_SUPPORTED_WIDTH) - 1) << AMD_CODE_PROPERTY_IS_XNACK_SUPPORTED_SHIFT,
 
   AMD_CODE_PROPERTY_RESERVED2_SHIFT = 23,
   AMD_CODE_PROPERTY_RESERVED2_WIDTH = 9,
-  AMD_CODE_PROPERTY_RESERVED2 = ((1 << AMD_CODE_PROPERTY_RESERVED2_WIDTH) - 1) << AMD_CODE_PROPERTY_RESERVED2_SHIFT
+  AMD_CODE_PROPERTY_RESERVED2 = ((1 << AMD_CODE_PROPERTY_RESERVED2_WIDTH) - 1) << AMD_CODE_PROPERTY_RESERVED2_SHIFT,
 };
 
 /// @brief The hsa_ext_control_directives_t specifies the values for the HSAIL
 /// control directives. These control how the finalizer generates code. This
 /// struct is used both as an argument to hsaFinalizeKernel to specify values for
 /// the control directives, and is used in HsaKernelCode to record the values of
 /// the control directives that the finalize used when generating the code which
 /// either came from the finalizer argument or explicit HSAIL control
 /// directives. See the definition of the control directives in HSA Programmer's
 /// Reference Manual which also defines how the values specified as finalizer
 /// arguments have to agree with the control directives in the HSAIL code.
 typedef struct hsa_ext_control_directives_s {

lib/Target/AMDGPU/SIDefines.h

 
 #define R_00B84C_COMPUTE_PGM_RSRC2                                      0x00B84C
 #define   S_00B84C_SCRATCH_EN(x)                                      (((x) & 0x1) << 0)
 #define   G_00B84C_SCRATCH_EN(x)                                      (((x) >> 0) & 0x1)
 #define   C_00B84C_SCRATCH_EN                                         0xFFFFFFFE
 #define   S_00B84C_USER_SGPR(x)                                       (((x) & 0x1F) << 1)
 #define   G_00B84C_USER_SGPR(x)                                       (((x) >> 1) & 0x1F)
 #define   C_00B84C_USER_SGPR                                          0xFFFFFFC1
+#define   S_00B84C_TRAP_HANDLER(x)                                    (((x) & 0x1) << 6)
+#define   G_00B84C_TRAP_HANDLER(x)                                    (((x) >> 6) & 0x1)
+#define   C_00B84C_TRAP_HANDLER                                       0xFFFFFFBF
 #define   S_00B84C_TGID_X_EN(x)                                       (((x) & 0x1) << 7)
 #define   G_00B84C_TGID_X_EN(x)                                       (((x) >> 7) & 0x1)
 #define   C_00B84C_TGID_X_EN                                          0xFFFFFF7F
 #define   S_00B84C_TGID_Y_EN(x)                                       (((x) & 0x1) << 8)
 #define   G_00B84C_TGID_Y_EN(x)                                       (((x) >> 8) & 0x1)
 #define   C_00B84C_TGID_Y_EN                                          0xFFFFFEFF
 #define   S_00B84C_TGID_Z_EN(x)                                       (((x) & 0x1) << 9)
 #define   G_00B84C_TGID_Z_EN(x)                                       (((x) >> 9) & 0x1)

lib/Target/AMDGPU/SIISelLowering.cpp

SITargetLowering::SITargetLowering(const TargetMachine &TM,
   }
 
   setOperationAction(ISD::BSWAP, MVT::i32, Legal);
   setOperationAction(ISD::BITREVERSE, MVT::i32, Legal);
 
   // On SI this is s_memtime and s_memrealtime on VI.
   setOperationAction(ISD::READCYCLECOUNTER, MVT::i64, Legal);
   setOperationAction(ISD::TRAP, MVT::Other, Legal);
+  setOperationAction(ISD::DEBUGTRAP, MVT::Other, Legal);
 
   setOperationAction(ISD::FMINNUM, MVT::f64, Legal);
   setOperationAction(ISD::FMAXNUM, MVT::f64, Legal);
 
   if (Subtarget->getGeneration() >= SISubtarget::SEA_ISLANDS) {
     setOperationAction(ISD::FTRUNC, MVT::f64, Legal);
     setOperationAction(ISD::FCEIL, MVT::f64, Legal);
     setOperationAction(ISD::FRINT, MVT::f64, Legal);
   }
 
   setOperationAction(ISD::FFLOOR, MVT::f64, Legal);
if (MI.mayLoad())
       Flags |= MachineMemOperand::MOLoad;
 
     auto MMO = MF->getMachineMemOperand(PtrInfo, Flags, 0, 0);
     MI.addMemOperand(*MF, MMO);
     return BB;
   }
 
   switch (MI.getOpcode()) {
-   case AMDGPU::S_TRAP_PSEUDO: {
-  DebugLoc DL = MI.getDebugLoc();
-  BuildMI(*BB, MI, DL, TII->get(AMDGPU::V_MOV_B32_e32), AMDGPU::VGPR0)
+    const DebugLoc &DL = MI.getDebugLoc();
+
+    if (Subtarget->getTrapHandlerAbi() == SISubtarget::TrapHandlerAbiHsa) {
+      BuildMI(*BB, MI, DL, TII->get(AMDGPU::V_MOV_B32_e32), AMDGPU::VGPR0)
         .addImm(1);
 
       MachineFunction *MF = BB->getParent();
       SIMachineFunctionInfo *Info = MF->getInfo<SIMachineFunctionInfo>();
       unsigned UserSGPR = Info->getQueuePtrUserSGPR();
       assert(UserSGPR != AMDGPU::NoRegister);
 
       if (!BB->isLiveIn(UserSGPR))
         BB->addLiveIn(UserSGPR);
 
       BuildMI(*BB, MI, DL, TII->get(AMDGPU::COPY), AMDGPU::SGPR0_SGPR1)
         .addReg(UserSGPR);
-    BuildMI(*BB, MI, DL, TII->get(AMDGPU::S_TRAP)).addImm(0x1)
+      BuildMI(*BB, MI, DL, TII->get(AMDGPU::S_TRAP))
+        .addImm(SISubtarget::TrapCodeLLVMTrap)
         .addReg(AMDGPU::VGPR0, RegState::Implicit)
         .addReg(AMDGPU::SGPR0_SGPR1, RegState::Implicit);
+    } else {
+      BuildMI(*BB, MI, DL, TII->get(AMDGPU::S_ENDPGM));
+    }
 
     MI.eraseFromParent();
     return BB;
   }
 
+  case AMDGPU::S_DEBUGTRAP_PSEUDO: {
+    const DebugLoc &DL = MI.getDebugLoc();
+    MachineFunction *MF = BB->getParent();
+    if (Subtarget->getTrapHandlerAbi() == SISubtarget::TrapHandlerAbiHsa) {
+      BuildMI(*BB, MI, DL, TII->get(AMDGPU::V_MOV_B32_e32), AMDGPU::VGPR0)
+        .addImm(1);
+
+      SIMachineFunctionInfo *Info = MF->getInfo<SIMachineFunctionInfo>();
+      unsigned UserSGPR = Info->getQueuePtrUserSGPR();
+      assert(UserSGPR != AMDGPU::NoRegister);
+
+      if (!BB->isLiveIn(UserSGPR))
+        BB->addLiveIn(UserSGPR);
+
+      BuildMI(*BB, MI, DL, TII->get(AMDGPU::COPY), AMDGPU::SGPR0_SGPR1)
+        .addReg(UserSGPR);
+      BuildMI(*BB, MI, DL, TII->get(AMDGPU::S_TRAP))
+        .addImm(SISubtarget::TrapCodeLLVMDebugTrap)
+        .addReg(AMDGPU::VGPR0, RegState::Implicit)
+        .addReg(AMDGPU::SGPR0_SGPR1, RegState::Implicit);
+    } else {
+      DiagnosticInfoUnsupported NoTrap(*MF->getFunction(),
+                                       "debugtrap handler not supported",
+                                       DL,
+                                       DS_Warning);
+
+      LLVMContext &C = MF->getFunction()->getContext();
+      C.diagnose(NoTrap);
+      BuildMI(*BB, MI, DL, TII->get(AMDGPU::S_ENDPGM));
+    }
+    MI.eraseFromParent();
+    return BB;
+  }
+
   case AMDGPU::SI_INIT_M0:
     BuildMI(*BB, MI.getIterator(), MI.getDebugLoc(),
             TII->get(AMDGPU::S_MOV_B32), AMDGPU::M0)
         .add(MI.getOperand(0));
     MI.eraseFromParent();
     return BB;
 
   case AMDGPU::GET_GROUPSTATICSIZE: {
case MVT::f16:
     return Subtarget->has16BitInsts() && Subtarget->hasFP16Denormals();
   default:
     break;
   }
 
   return false;
 }
 
 //===----------------------------------------------------------------------===//
 // Custom DAG Lowering Operations
 //===----------------------------------------------------------------------===//
 
 SDValue SITargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
   switch (Op.getOpcode()) {
   default: return AMDGPUTargetLowering::LowerOperation(Op, DAG);
   case ISD::BRCOND: return LowerBRCOND(Op, DAG);
   case ISD::LOAD: {
SDValue SITargetLowering::LowerGlobalAddress(AMDGPUMachineFunction *MFI,
   // FIXME: Use a PseudoSourceValue once those can be assigned an address space.
   MachinePointerInfo PtrInfo(UndefValue::get(PtrTy));
 
   return DAG.getLoad(PtrVT, DL, DAG.getEntryNode(), GOTAddr, PtrInfo, Align,
                      MachineMemOperand::MODereferenceable |
                          MachineMemOperand::MOInvariant);
 }
 
 SDValue SITargetLowering::copyToM0(SelectionDAG &DAG, SDValue Chain,
                                    const SDLoc &DL, SDValue V) const {
   // We can't use S_MOV_B32 directly, because there is no way to specify m0 as
   // the destination register.
   //
   // We can't use CopyToReg, because MachineCSE won't combine COPY instructions,
   // so we will end up with redundant moves to m0.
   //
   // We use a pseudo to ensure we emit s_mov_b32 with m0 as the direct result.
 
   // A Null SDValue creates a glue result.
   SDNode *M0 = DAG.getMachineNode(AMDGPU::SI_INIT_M0, DL, MVT::Other, MVT::Glue,
                                   V, Chain);
   return SDValue(M0, 0);
 }
 
 SDValue SITargetLowering::lowerImplicitZextParam(SelectionDAG &DAG,
                                                  SDValue Op,
                                                  MVT VT,
                                                  unsigned Offset) const {
   SDLoc SL(Op);
   SDValue Param = LowerParameter(DAG, MVT::i32, MVT::i32, SL,
                                  DAG.getEntryNode(), Offset, false);
   // The local size values will have the hi 16-bits as zero.
   return DAG.getNode(ISD::AssertZext, SL, MVT::i32, Param,

lib/Target/AMDGPU/SIInstructions.td

 
 // 64-bit vector move instruction.  This is mainly used by the SIFoldOperands
 // pass to enable folding of inline immediates.
 def V_MOV_B64_PSEUDO : VPseudoInstSI <(outs VReg_64:$vdst),
                                       (ins VSrc_b64:$src0)>;
 } // End let hasSideEffects = 0, mayLoad = 0, mayStore = 0, Uses = [EXEC]
 
 def S_TRAP_PSEUDO : VPseudoInstSI <(outs), (ins),
   [(trap)]> {
   let hasSideEffects = 1;
   let SALU = 1;
   let usesCustomInserter = 1;
 }
 
+def S_DEBUGTRAP_PSEUDO : VPseudoInstSI <(outs), (ins),
+  [(debugtrap)]> {
+  let hasSideEffects = 1;
+  let SALU = 1;
+  let usesCustomInserter = 1;
+}
+
 let usesCustomInserter = 1, SALU = 1 in {
 def GET_GROUPSTATICSIZE : PseudoInstSI <(outs SReg_32:$sdst), (ins),
   [(set SReg_32:$sdst, (int_amdgcn_groupstaticsize))]>;
 } // End let usesCustomInserter = 1, SALU = 1
 
 def S_MOV_B64_term : PseudoInstSI<(outs SReg_64:$dst),
    (ins SSrc_b64:$src0)> {
   let SALU = 1;
def SI_PC_ADD_REL_OFFSET : SPseudoInstSI <
   [(set SReg_64:$dst,
    (i64 (SIpc_add_rel_offset (tglobaladdr:$ptr_lo), (tglobaladdr:$ptr_hi))))]> {
   let Defs = [SCC];
 }
 
 } // End SubtargetPredicate = isGCN
 
 let Predicates = [isGCN] in {
 
 def : Pat<
   (int_amdgcn_else i64:$src, bb:$target),
   (SI_ELSE $src, $target, 0)
 >;
 
 def : Pat <
   (int_AMDGPU_kilp),
   (SI_KILL (i32 0xbf800000))
 >;
 
 //===----------------------------------------------------------------------===//

lib/Target/AMDGPU/SIRegisterInfo.cpp

 
 unsigned SIRegisterInfo::getTotalNumSGPRs(const SISubtarget &ST) const {
   if (ST.getGeneration() >= AMDGPUSubtarget::VOLCANIC_ISLANDS)
     return 800;
   return 512;
 }
 
 unsigned SIRegisterInfo::getNumAddressableSGPRs(const SISubtarget &ST) const {
+  int trap_handler_reserved =
+      ST.isTrapHandlerEnabled() ? SISubtarget::TRAP_HANDLER_SGPR_COUNT : 0;
   if (ST.getGeneration() >= AMDGPUSubtarget::VOLCANIC_ISLANDS)
-    return 102;
-  return 104;
+    return 102 - trap_handler_reserved;
+  return 104 - trap_handler_reserved;
 }
 
 unsigned SIRegisterInfo::getNumReservedSGPRs(const SISubtarget &ST,
                                              const SIMachineFunctionInfo &MFI) const {
   if (MFI.hasFlatScratchInit()) {
     if (ST.getGeneration() >= AMDGPUSubtarget::VOLCANIC_ISLANDS)
       return 6; // FLAT_SCRATCH, XNACK, VCC (in that order)
 

lib/Target/AMDGPU/Utils/AMDKernelCodeTInfo.h

 COMPPGM1(priv,                            compute_pgm_rsrc1_priv,           PRIV),
 COMPPGM1(enable_dx10_clamp,               compute_pgm_rsrc1_dx10_clamp,     DX10_CLAMP),
 COMPPGM1(debug_mode,                      compute_pgm_rsrc1_debug_mode,     DEBUG_MODE),
 COMPPGM1(enable_ieee_mode,                compute_pgm_rsrc1_ieee_mode,      IEEE_MODE),
 // TODO: bulky
 // TODO: cdbg_user
 COMPPGM2(enable_sgpr_private_segment_wave_byte_offset, compute_pgm_rsrc2_scratch_en, SCRATCH_EN),
 COMPPGM2(user_sgpr_count,                 compute_pgm_rsrc2_user_sgpr,      USER_SGPR),
-// TODO: enable_trap_handler
+COMPPGM2(enable_trap_handler,             compute_pgm_rsrc2_trap_handler,   TRAP_HANDLER),
 COMPPGM2(enable_sgpr_workgroup_id_x,      compute_pgm_rsrc2_tgid_x_en,      TGID_X_EN),
 COMPPGM2(enable_sgpr_workgroup_id_y,      compute_pgm_rsrc2_tgid_y_en,      TGID_Y_EN),
 COMPPGM2(enable_sgpr_workgroup_id_z,      compute_pgm_rsrc2_tgid_z_en,      TGID_Z_EN),
 COMPPGM2(enable_sgpr_workgroup_info,      compute_pgm_rsrc2_tg_size_en,     TG_SIZE_EN),
 COMPPGM2(enable_vgpr_workitem_id,         compute_pgm_rsrc2_tidig_comp_cnt, TIDIG_COMP_CNT),
 COMPPGM2(enable_exception_msb,            compute_pgm_rsrc2_excp_en_msb,    EXCP_EN_MSB), // TODO: split enable_exception_msb
 COMPPGM2(granulated_lds_size,             compute_pgm_rsrc2_lds_size,       LDS_SIZE),
 COMPPGM2(enable_exception,                compute_pgm_rsrc2_excp_en,        EXCP_EN), // TODO: split enable_exception

 CODEPROP(enable_sgpr_grid_workgroup_count_y,  ENABLE_SGPR_GRID_WORKGROUP_COUNT_Y),
 CODEPROP(enable_sgpr_grid_workgroup_count_z,  ENABLE_SGPR_GRID_WORKGROUP_COUNT_Z),
 CODEPROP(enable_ordered_append_gds,           ENABLE_ORDERED_APPEND_GDS),
 CODEPROP(private_element_size,                PRIVATE_ELEMENT_SIZE),
 CODEPROP(is_ptr64,                            IS_PTR64),
 CODEPROP(is_dynamic_callstack,                IS_DYNAMIC_CALLSTACK),
 CODEPROP(is_debug_enabled,                    IS_DEBUG_SUPPORTED),
 CODEPROP(is_xnack_enabled,                    IS_XNACK_SUPPORTED),
 
 FIELD(workitem_private_segment_byte_size),
 FIELD(workgroup_group_segment_byte_size),
 FIELD(gds_segment_byte_size),
 FIELD(kernarg_segment_byte_size),
 FIELD(workgroup_fbarrier_count),
 FIELD(wavefront_sgpr_count),
 FIELD(workitem_vgpr_count),
 FIELD(reserved_vgpr_first),

test/CodeGen/AMDGPU/trap.ll

-; RUN: llc -mtriple=amdgcn--amdhsa -verify-machineinstrs < %s 2>&1 | FileCheck -check-prefix=GCN %s
+; RUN: llc -mtriple=amdgcn--amdhsa -mattr=+trap-handler -verify-machineinstrs < %s | FileCheck -check-prefix=HSA -check-prefix=HSA-FUNC %s
+; RUN: llc -mtriple=amdgcn--amdhsa -mattr=-trap-handler -verify-machineinstrs < %s | FileCheck -check-prefix=FUNC -check-prefix=HSA-FUNC %s
+; RUN: llc -march=amdgcn -verify-machineinstrs < %s 2>&1 | FileCheck -check-prefix=NONHSA %s
 
 declare void @llvm.trap() #0
+declare void @llvm.debugtrap() #0
 
-; GCN-LABEL: {{^}}trap:
-; GCN: v_mov_b32_e32 v0, 1
-; GCN: s_mov_b64 s[0:1], s[4:5]
-; GCN: s_trap 1
+; HSA-LABEL: {{^}}hsa_trap:
+; HSA: enable_trap_handler = 1
+; FUNC: enable_trap_handler = 0
+; HSA-FUNC: v_mov_b32_e32 v0, 1
+; HSA-FUNC: s_mov_b64 s[0:1], s[4:5]
+; HSA-FUNC: s_trap 1
+define void @hsa_trap() {
+  call void @llvm.trap()
+  ret void
+}
+
+; HSA-LABEL: {{^}}hsa_debugtrap:
+; HSA: enable_trap_handler = 1
+; FUNC: enable_trap_handler = 0
+; HSA-FUNC: v_mov_b32_e32 v0, 1
+; HSA-FUNC: s_mov_b64 s[0:1], s[4:5]
+; HSA-FUNC: s_trap 2
+; NONHSA: <unknown>:0:0: in function hsa_debugtrap void (): debugtrap handler not supported
+; NONHSA: s_endpgm
+define void @hsa_debugtrap() {
+  call void @llvm.debugtrap()
+  ret void
+}
+
+; For non-HSA path
+; NONHSA: s_endpgm
 define void @trap() {
   call void @llvm.trap()
   ret void
 }
 
 attributes #0 = { nounwind noreturn }