[AMDGPU] Add a new intrinsic to control fp_trunc rounding mode

[jpages]

Summary

Add a new intrinsic to precisely control the rounding mode when converting
from f32 to f16.

[dstuttard]

llvm/test/CodeGen/AMDGPU/llvm.experimental.fptrunc.round.ll

38 ↗

(On Diff #375382)

Is there a reason why we couldn't make this a single s_round_mode 0x8 rather than going via s_round_mode 0x0?

[foad]

I'm not sure what level of approval is needed for new experimental intrinsics. It might be worth emailing llvmdev to explain the requirement. In any case they need to be documented in docs/LangRef.rst.

You're using EmitInstrWithCustomInserter to emit s_round_mode instructions, and then adding a new phase to SIModeRegister to remove them. This seems wrong. You should be able to teach SIModeRegister to insert the required s_round_mode instructions.

llvm/include/llvm/IR/Intrinsics.td

905

Needs a comment.

908

Needs a comment.

llvm/lib/Target/AMDGPU/SIInstructions.td

182–183

Can you implement instruction selection for the intrinsics by adding patterns here, instead of writing C++ code?

[jpages]

Rebased based on previous comments.

The pseudo-instructions are now selected in the pass SIModeRegister.

[foad]

llvm/lib/Target/AMDGPU/SIModeRegister.cpp

428

You shouldn't need to add a new pass here to insert s_round_mode instructions. The SIModeRegister pass already knows how to do that. You should just be able to add your new PSEUDOs to the switch in getInstructionMode.

[jpages]

Rebased based on Jay's comments.

The consequence of this change is the use of setreg instead of s_round_mode in the codegen. But it's probably better to not reinvent the wheel as this pass is already optimized to not insert too many setreg.

[foad]

In D110579#3051790, @jpages wrote:

The consequence of this change is the use of setreg instead of s_round_mode in the codegen. But it's probably better to not reinvent the wheel as this pass is already optimized to not insert too many setreg.

Good point. s_round_mode/s_denorm_mode are new in GFX10, so they did not exist when this pass was written. Do you think the pass could be improved to emit s_round_mode/s_denorm_mode instead of s_setreg whenever it only needs to change the rounding/denormal bits of the mode register? That could be a separate patch.

[foad]

llvm/lib/Target/AMDGPU/SIModeRegister.cpp

439

If you arrange for the pseudo to have exactly the same operands as the real instruction, then you don't need to build or delete instructions, all you need to do is change the opcode, which you can do with MI.setDesc().

442

I'm not sure this needs another pass over all instructions in the function. Would it be cleaner to do it in phase 1? Maybe even inside getInstructionMode???

[arsenm]

llvm/include/llvm/IR/Intrinsics.td

914

Would it be better to have a metadata argument for the rounding mode like the constrained intrinsics? The verifier could disallow the unknown mode type

llvm/lib/Target/AMDGPU/AMDGPURegisterBankInfo.cpp

4493–4495

The backend should not directly consume generic intrinsics. These need to be routed through a generic opcode which is subject to normal legalization

[jpages]

llvm/lib/Target/AMDGPU/AMDGPURegisterBankInfo.cpp

4493–4495

What is the reasoning behind this requirement?

I guess I would need something like experimental_fptrunc_round_upward -> amdgpu_experimental_fptrunc_round_upward -> codegen?

llvm/lib/Target/AMDGPU/SIModeRegister.cpp

439

I think this could work. I tried to "replace" my pseudo-instructions in Phase1 but it's not a good idea to add/delete instructions in the middle of an iteration on the basic block.

[jpages]

In D110579#3054424, @foad wrote:

In D110579#3051790, @jpages wrote:

The consequence of this change is the use of setreg instead of s_round_mode in the codegen. But it's probably better to not reinvent the wheel as this pass is already optimized to not insert too many setreg.

Good point. s_round_mode/s_denorm_mode are new in GFX10, so they did not exist when this pass was written. Do you think the pass could be improved to emit s_round_mode/s_denorm_mode instead of s_setreg whenever it only needs to change the rounding/denormal bits of the mode register? That could be a separate patch.

Sure, that's a good suggestion. I'll try to add this in a different patch.

[arsenm]

llvm/lib/Target/AMDGPU/AMDGPURegisterBankInfo.cpp

4493–4495

No, you would just need G_FPTRUNC_ROUND_UPWARD etc. opcodes. We don't have a way to define legalization rules on intrinsics, so if you need type legalization the backend would have to take care of it manually. e.g. we would have to manually scalarize the vector overloads of these intrinsics. With the opcode you can just define legalization rules like normal

[jpages]

Rebased.

Added G_FPTRUNC_ROUND_UPWARD/G_FPTRUNC_ROUND_DOWNWARD opcodes between the intrinsics and the pseudo instructions. Added a custom ISD node for the DAG version.

It may be possible to have a simpler implementation for intrinsics -> G_FPTRUNC_ROUND_UPWARD -> pseudos, I'm open to suggestions.

Arranged the pseudos to have exactly the same operands as the v_cvt instruction like Jay suggested.

[sepavloff]

llvm/include/llvm/IR/Intrinsics.td

910

What is the difference between this function and llvm.ceil? Can the latter be used instead of this one?

914

What is the difference between this function and llvm.floor?

There are also functions llvm.nearbyint and llvm.rint that can be used to make rounding with any mode.

[sepavloff]

llvm/include/llvm/IR/Intrinsics.td

914

Oh, I see, this is conversion between floats.

There is llvm.experimental.constrained.fptrunc, which seems to do the same thing?

[jpages]

llvm/include/llvm/IR/Intrinsics.td

914

Yes, this is indeed a conversion between floats.

The idea of this change is to introduce something simpler than constrained intrinsics in a special case.

According to https://llvm.org/docs/LangRef.html#constrained-floating-point-intrinsics "If any FP operation in a function is constrained then they all must be constrained".

We wanted to do a simpler conversion from f32 -> f16 with a special rounding mode, but without constraining the rest of the function after this operation. Introducing an Intrinsic that would be lowered to 3 instructions seemed to be the best solution.

These 3 instructions are:

• Setting a special rounding mode
• Conversion f32 -> f16
• Restoring the rounding mode to the default value

[sepavloff]

llvm/include/llvm/IR/Intrinsics.td

914

This would be a useful function, which solves a widespread task. On some platforms, like RISC-V, it even can be lowered into a single instruction.

As this is common interface, I would propose you to make rounding mode an argument. It would allows a target to implement the conversion with any supported mode. Besides, it would allow future extensions without changing the interface, for example, to allow other control modes:

lvm.fptrunc.round(%a, !"round.towardzero,denorm.ftz")

What prevents from making it a regular intrinsic, not experimental?

[jpages]

Rebased.

I changed the patch to have only one intrinsic with a rounding mode parameter as suggested. This required many modifications in the frontend parts of LLVM.

Thanks for the comments!

[craig.topper]

llvm/include/llvm/IR/Intrinsics.td

907

Align the brackets with the previous line.

llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp

2261

Use cast instead of dyn_cast if it can't fail. Or check that the dyn_cast didn't return null if it can fail.

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

6337

Why would this intrinsic ever have a chain?

6349

Use cast instead of dyn_cast if it can't fail. Or check that the dyn_cast didn't return null if it can fail.

6384

The code has optionally added a chain, but INTRINSIC_W_CHAIN must always have a chain. Whether there is a chain or not is a property of the ISD opcode.

[craig.topper]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

6337

Err. I guess the intrinsic attributes are defined to always have a Chain. Should it be IntrNoMem instead?

[foad]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

6337

IIUC, if the rounding mode is allowed to be "dynamic" then the intrinsic needs to be IntrInaccessibleMemOnly (just like the constrained fp intrinsics). If we don't allow "dynamic" then it could be IntrNoMem.

Personally I have no interest in supporting "dynamic" so I would vote for disallowing it and making the intrinsic IntrNoMem.

[jpages]

Rebased.

The intrinsic has the IntrNoMem attribute, and is represented by INTRINSIC_WO_CHAIN in the DAG.

[jpages]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

6337

Yes the original version was defined to have a chain, I changed it with IntrNoMem/INTRINSIC_WO_CHAIN as suggested. Thank you for the suggestions.

As a consequence this code is a bit simpler, and the lowering in the backend was modified to reflect that .

[foad]

It seems like people are mostly happy with the design now, so I am being a bit more picky with my review comments!

llvm/docs/LangRef.rst

23858

Add "... with a specified rounding mode". Apart from that I think most of the Overview/Arguments/Semantics text could be copied verbatim from the definition of the fptrunc-to instruction?

23868

Need to disallow "dynamic" if that's what we decided.

23874

Probably need to add the text from fptrunc-to: "This instruction is assumed to execute in the default floating-point environment" (because of the stuff about exceptions) and then say *except* for the rounding mode.

llvm/include/llvm/IR/Intrinsics.td

905

"Truncate"

908

Add IntrWillReturn.

llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp

2257

VRegs isn't used for anything so remove it.

2269

We know it's not void, and has exactly one result, so simplify this.

2273

We know it does not access memory.

llvm/lib/IR/Verifier.cpp

4748

MD could be nullptr here, so you need to handle that.

4750

Check that the rounding mode is not "dynamic"?

llvm/lib/Target/AMDGPU/AMDGPUISelDAGToDAG.cpp

698 ↗

(On Diff #399125)

Can this be done with patterns in the tablegen files instead of C++ code? Then they should work for both SelectionDAG and GlobalISel.

llvm/lib/Target/AMDGPU/AMDGPUInstructionSelector.cpp

3252 ↗

(On Diff #399125)

Can this be done with patterns in the tablegen files instead of C++ code?

llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp

4939

Should this be "return false" to indicate a legalization failure in the normal way?

llvm/lib/Target/AMDGPU/AMDGPURegisterBankInfo.cpp

4600

sgpr seems wrong. This is specifying the mapping for the result value and the input value (not the rounding mode which is no longer an operand). I think this should just use the same mapping as G_FPTRUNC (line 3640).

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

6844

I'm not quite sure but I think you should return SDValue() here to indicate a lowering/legalization failure?

llvm/lib/Target/AMDGPU/SIISelLowering.h

134 ↗

(On Diff #399768)

No longer needed.

[craig.topper]

I think I'd like to see a target independent ISD opcode added for this. We don't use ISD::INTRINSIC_WO_CHAIN for target independent intrinsics.

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

6356

This is always false. And if it isn't you would only push one argument.

6361

I'd probably just use TLI.getPointerTy instead of MVT::i8.

6377

I know this was taken from visitTargetIntrinsic, but I don't really understand it.

6381

I don't think this is needed. AssertZExt isn't useful for an FP result.

6383

I don't think this alignment stuff is needed. The return type is FP so it probably doesn't do anything.

[jpages]

Rebased, as suggested I added a target independent ISD opcode.

Thanks again for all the comments!

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

6377

I removed this line, I'm not sure of what it was doing either.

llvm/lib/Target/AMDGPU/AMDGPUInstructionSelector.cpp

3252 ↗

(On Diff #399125)

Maybe it's possible, but I could not find a way to match both the G_FPTRUNC_ROUND_UPWARD and the selection DAG version.

So I let this one in C++.

llvm/lib/Target/AMDGPU/SIInstructions.td

182–183

Like I said previously, I tried to use pure tablegen for both SelectionDag and global-isel and I could not find a way to do it for both.

[foad]

llvm/lib/Target/AMDGPU/AMDGPUInstructionSelector.cpp

3252 ↗

(On Diff #399125)

You can get some clues about GlobalISel pattern problems by running tablegen with -warn-on-skipped-patterns like this:

$ llvm-tblgen -gen-global-isel -I lib/Target/AMDGPU -I build/include -I include -I lib/Target lib/Target/AMDGPU/AMDGPUGISel.td -warn-on-skipped-patterns
...
lib/Target/AMDGPU/SIInstructions.td:186:1: warning: Skipped pattern: Pattern operator lacks an equivalent Instruction (AMDGPUISD::FPTRUNC_ROUND_UPWARD)
def FPTRUNC_UPWARD_PSEUDO : VPseudoInstSI <(outs VGPR_32:$vdst),
^
...
lib/Target/AMDGPU/SIInstructions.td:190:1: warning: Skipped pattern: Pattern operator lacks an equivalent Instruction (AMDGPUISD::FPTRUNC_ROUND_DOWNWARD)
def FPTRUNC_DOWNWARD_PSEUDO : VPseudoInstSI <(outs VGPR_32:$vdst),
^

This means you need to add some GINodeEquiv lines in AMDGPUGISel.td.

llvm/lib/Target/AMDGPU/SIInstructions.td

188

Just end the line with ; if there is nothing to go inside the {}.

[foad]

Here's what I came up with to get the globalisel pattern matching working, plus a few other suggested tweaks: https://reviews.llvm.org/differential/diff/404879/

[jpages]

Rebased following Jay's comments.

[jpages]

llvm/lib/Target/AMDGPU/AMDGPUInstructionSelector.cpp

3252 ↗

(On Diff #399125)

Thanks again, I didn't know about this command

[jpages]

Updated a failing test on AArch64.

[craig.topper]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

6355

You probably don't need ComputeValueVTs. There shouldn't be any structs or arrays here that need to broken down. I think you can do

EVT VT = TLI.getValueType(I.getType)

And then use VT in place of VTs

[jpages]

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

6355

Indeed it works, thanks!

[foad]

This is looking pretty good, just another round of minor comments.

llvm/docs/LangRef.rst

23870

This seems to be repeating the "must be larger" text from a few lines above.

llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp

2270

return true.

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

6337

You don't really need a SmallVector here, since you know you have exactly two operands, you could just pass them as two separate arguments to the DAG.getNode call. But it's up to you.

6341

Shouldn't this be getArgOperand(1)? How does it work?

6345

This isn't used.

6346

TLI is already available in this function.

6359

All intrinsics returning floating point type are classed as FPMathOperators, so you can do this part unconditionally.

6364

Use sdl here too.

llvm/lib/IR/Verifier.cpp

4748

I am not an expert on verifying metadata but I think you probably need to check that it is an MDString here, otherwise the cast<MDString> below could fail? And really we ought to have some tests in test/Verifier/llvm.fptrunc.round.ll.

4752

I am not sure but I think RoundMode != RoundingMode::Dynamic needs to be RoundMode.getValue() != ... (or *RoundMode != ...).

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

4662

Do you also need to check that the operand is MVT::f32? Maybe add a test where the operand is f64, and check that it fails to legalize?

[jpages]

Rebased following Jay's comments.

Added some tests for the verifier/legalizer.

[foad]

I think this looks good now. Are you happy with it @craig.topper @sepavloff ?

llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp

2258

Should probably use getArgOperand here, though it doesn't make any difference except for bounds checking the index that you pass in.

llvm/lib/IR/Verifier.cpp

4750

isa is a bit more natural than dyn_cast here.

llvm/test/CodeGen/AMDGPU/fail.llvm.fptrunc.round.ll

1

Does this test work in a Release build? If not, you can add a "REQUIRES: asserts" line just after the RUN lines.

[sepavloff]

It looks good to me.

I had a look at generic changes only, someone with expertise in AMDGPU should also approve this patch.

llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp

2268

Flags could be set in the call to MIRBuilder.buildInstr.

[craig.topper]

llvm/docs/LangRef.rst

23879

Should we document that this isn't supported on all targets, and that a particular target may not support all rounding modes?

[jpages]

llvm/docs/LangRef.rst

23879

Good idea, thanks! I added a comment about that

llvm/test/CodeGen/AMDGPU/fail.llvm.fptrunc.round.ll

1

I just checked and it's working as well in Release without that.

[foad]

llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp

2267

You can also do this on the end of the buildInstr line if you prefer, no need for MIB at all.

llvm/docs/LangRef.rst

 <attr_elementtype>` attribute at the call-site. This attribute specifies the
 getelementptr element type.
 
 Semantics:
 """"""""""
 
 The '``llvm.preserve.struct.access.index``' intrinsic produces the same result
 as a getelementptr with base ``base`` and access operands ``{0, gep_index}``.
+
+'``llvm.fptrunc.round``' Intrinsic
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Syntax:
+"""""""
+
+::
+
+      declare <ty2>
+      @llvm.fptrunc.round(<type> <value>, metadata <rounding mode>)
+
+Overview:
+"""""""""
+
+The '``llvm.fptrunc.round``' intrinsic truncates
+:ref:`floating-point <t_floating>` ``value`` to type ``ty2``
+with a specified rounding mode.
+
+Arguments:
+""""""""""
+
+The '``llvm.fptrunc.round``' intrinsic takes a :ref:`floating-point
+<t_floating>` value to cast and a :ref:`floating-point <t_floating>` type
+to cast it to. This argument must be larger in size than the result.
+
+The second argument specifies the rounding mode as described in the constrained
+intrinsics section.
+For this intrinsic, the "round.dynamic" mode is not supported.
+
+Semantics:
+""""""""""
+
+The '``llvm.fptrunc.round``' intrinsic casts a ``value`` from a larger
+:ref:`floating-point <t_floating>` type to a smaller :ref:`floating-point
+<t_floating>` type.
+This instruction is assumed to execute in the default :ref:`floating-point
+environment <floatenv>` *except* for the rounding mode.

llvm/include/llvm/CodeGen/ISDOpcodes.h

enum NodeType {
 
   /// STRICT_FSETCC/STRICT_FSETCCS - Constrained versions of SETCC, used
   /// for floating-point operands only.  STRICT_FSETCC performs a quiet
   /// comparison operation, while STRICT_FSETCCS performs a signaling
   /// comparison operation.
   STRICT_FSETCC,
   STRICT_FSETCCS,
 
+  // FPTRUNC_ROUND - This corresponds to the fptrunc_round intrinsic.
+  FPTRUNC_ROUND,
+
   /// FMA - Perform a * b + c with no intermediate rounding step.
   FMA,
 
   /// FMAD - Perform a * b + c, while getting the same result as the
   /// separately rounded operations.
   FMAD,
 
   /// FCOPYSIGN(X, Y) - Return the value of X with the sign of Y.  NOTE: This

llvm/include/llvm/IR/Intrinsics.td

def int_experimental_constrained_fcmp
                     llvm_metadata_ty, llvm_metadata_ty ]>;
   def int_experimental_constrained_fcmps
       : DefaultAttrsIntrinsic<[ LLVMScalarOrSameVectorWidth<0, llvm_i1_ty> ],
                   [ llvm_anyfloat_ty, LLVMMatchType<0>,
                     llvm_metadata_ty, llvm_metadata_ty ]>;
 }
 // FIXME: Consider maybe adding intrinsics for sitofp, uitofp.
 
+
+// Truncate a floating point number with a specific rounding mode
+def int_fptrunc_round : DefaultAttrsIntrinsic<[ llvm_anyfloat_ty ],
+                                              [ llvm_anyfloat_ty, llvm_metadata_ty ],
+                                              [ IntrNoMem, IntrWillReturn ]>;
+
 //===------------------------- Expect Intrinsics --------------------------===//
 //
 def int_expect : DefaultAttrsIntrinsic<[llvm_anyint_ty],
   [LLVMMatchType<0>, LLVMMatchType<0>], [IntrNoMem, IntrWillReturn]>;
 
 def int_expect_with_probability : DefaultAttrsIntrinsic<[llvm_anyint_ty],
   [LLVMMatchType<0>, LLVMMatchType<0>, llvm_double_ty],
   [IntrNoMem, IntrWillReturn]>;
 
 //===-------------------- Bit Manipulation Intrinsics ---------------------===//
 //
 
 // None of these intrinsics accesses memory at all.

llvm/include/llvm/Support/TargetOpcodes.def

 
 /// Generic bitcast. The source and destination types must be different, or a
 /// COPY is the relevant instruction.
 HANDLE_TARGET_OPCODE(G_BITCAST)
 
 /// Generic freeze.
 HANDLE_TARGET_OPCODE(G_FREEZE)
 
+// INTRINSIC fptrunc_round intrinsic.
+HANDLE_TARGET_OPCODE(G_INTRINSIC_FPTRUNC_ROUND)
+
 /// INTRINSIC trunc intrinsic.
 HANDLE_TARGET_OPCODE(G_INTRINSIC_TRUNC)
 
 /// INTRINSIC round intrinsic.
 HANDLE_TARGET_OPCODE(G_INTRINSIC_ROUND)
 
 /// INTRINSIC round to integer intrinsic.
 HANDLE_TARGET_OPCODE(G_INTRINSIC_LRINT)

llvm/include/llvm/Target/GenericOpcodes.td

def G_FNEARBYINT : GenericInstruction {
   let OutOperandList = (outs type0:$dst);
   let InOperandList = (ins type0:$src1);
   let hasSideEffects = false;
 }
 
 //------------------------------------------------------------------------------
 // Opcodes for LLVM Intrinsics
 //------------------------------------------------------------------------------
+def G_INTRINSIC_FPTRUNC_ROUND : GenericInstruction {
+  let OutOperandList = (outs type0:$dst);
+  let InOperandList = (ins type1:$src1, i32imm:$round_mode);
+  let hasSideEffects = false;
+}
+
 def G_INTRINSIC_TRUNC : GenericInstruction {
   let OutOperandList = (outs type0:$dst);
   let InOperandList = (ins type0:$src1);
   let hasSideEffects = false;
 }
 
 def G_INTRINSIC_ROUND : GenericInstruction {
   let OutOperandList = (outs type0:$dst);

llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp

if (ID == Intrinsic::ubsantrap) {
       Info.OrigArgs.push_back({getOrCreateVRegs(*CI.getArgOperand(0)),
                                CI.getArgOperand(0)->getType(), 0});
     }
     Info.Callee = MachineOperand::CreateES(TrapFuncName.data());
     Info.CB = &CI;
     Info.OrigRet = {Register(), Type::getVoidTy(CI.getContext()), 0};
     return CLI->lowerCall(MIRBuilder, Info);
   }
+  case Intrinsic::fptrunc_round: {
+    unsigned Flags = MachineInstr::copyFlagsFromInstruction(CI);
+
+    // Convert the metadata argument to a constant integer
+    Metadata *MD = cast<MetadataAsValue>(CI.getOperand(1))->getMetadata();
+    Optional<RoundingMode> RoundMode =
+        convertStrToRoundingMode(cast<MDString>(MD)->getString());
+
+    MachineInstrBuilder MIB = MIRBuilder.buildInstr(
+        TargetOpcode::G_INTRINSIC_FPTRUNC_ROUND, {getOrCreateVReg(CI)},
+        {getOrCreateVReg(*CI.getArgOperand(0))});
+
+    // Add the Rounding mode as an integer
+    MIB.addImm((int)RoundMode.getValue());
+    MIB->setFlags(Flags);
+
+    return true;
+  }
 #define INSTRUCTION(NAME, NARG, ROUND_MODE, INTRINSIC)  \
   case Intrinsic::INTRINSIC:
 #include "llvm/IR/ConstrainedOps.def"
     return translateConstrainedFPIntrinsic(cast<ConstrainedFPIntrinsic>(CI),
                                            MIRBuilder);
 
   }
   return false;
 }
 

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

#define INSTRUCTION(NAME, NARG, ROUND_MODE, INTRINSIC)                         \
   case Intrinsic::INTRINSIC:
 #include "llvm/IR/ConstrainedOps.def"
     visitConstrainedFPIntrinsic(cast<ConstrainedFPIntrinsic>(I));
     return;
 #define BEGIN_REGISTER_VP_INTRINSIC(VPID, ...) case Intrinsic::VPID:
 #include "llvm/IR/VPIntrinsics.def"
     visitVectorPredicationIntrinsic(cast<VPIntrinsic>(I));
     return;
+  case Intrinsic::fptrunc_round: {
+    // Get the last argument, the metadata and convert it to an integer in the
+    // call
+    Metadata *MD = cast<MetadataAsValue>(I.getArgOperand(1))->getMetadata();
+    Optional<RoundingMode> RoundMode =
+        convertStrToRoundingMode(cast<MDString>(MD)->getString());
+
+    EVT VT = TLI.getValueType(DAG.getDataLayout(), I.getType());
+
+    // Propagate fast-math-flags from IR to node(s).
+    SDNodeFlags Flags;
+    Flags.copyFMF(*cast<FPMathOperator>(&I));
+    SelectionDAG::FlagInserter FlagsInserter(DAG, Flags);
+
+    SDValue Result;
+    Result = DAG.getNode(
+        ISD::FPTRUNC_ROUND, sdl, VT, getValue(I.getArgOperand(0)),
+        DAG.getTargetConstant((int)RoundMode.getValue(), sdl,
+                              TLI.getPointerTy(DAG.getDataLayout())));
+    setValue(&I, Result);
+
+    return;
+  }
   case Intrinsic::fmuladd: {
     EVT VT = TLI.getValueType(DAG.getDataLayout(), I.getType());
     if (TM.Options.AllowFPOpFusion != FPOpFusion::Strict &&
         TLI.isFMAFasterThanFMulAndFAdd(DAG.getMachineFunction(), VT)) {
       setValue(&I, DAG.getNode(ISD::FMA, sdl,
                                getValue(I.getArgOperand(0)).getValueType(),
                                getValue(I.getArgOperand(0)),
                                getValue(I.getArgOperand(1)),
                                getValue(I.getArgOperand(2)), Flags));
     } else {
       // TODO: Intrinsic calls should have fast-math-flags.
       SDValue Mul = DAG.getNode(
           ISD::FMUL, sdl, getValue(I.getArgOperand(0)).getValueType(),
           getValue(I.getArgOperand(0)), getValue(I.getArgOperand(1)), Flags);
       SDValue Add = DAG.getNode(ISD::FADD, sdl,
                                 getValue(I.getArgOperand(0)).getValueType(),
                                 Mul, getValue(I.getArgOperand(2)), Flags);
       setValue(&I, Add);
     }
     return;
   }
   case Intrinsic::convert_to_fp16:
     setValue(&I, DAG.getNode(ISD::BITCAST, sdl, MVT::i16,
                              DAG.getNode(ISD::FP_ROUND, sdl, MVT::f16,
                                          getValue(I.getArgOperand(0)),
                                          DAG.getTargetConstant(0, sdl,
                                                                MVT::i32))));
     return;
   case Intrinsic::convert_from_fp16:
     setValue(&I, DAG.getNode(ISD::FP_EXTEND, sdl,
                              TLI.getValueType(DAG.getDataLayout(), I.getType()),
                              DAG.getNode(ISD::BITCAST, sdl, MVT::f16,
                                          getValue(I.getArgOperand(0)))));
     return;
   case Intrinsic::fptosi_sat: {

llvm/lib/IR/Verifier.cpp

Assert(GV && GV->isConstant() && GV->hasDefinitiveInitializer(),
            "info argument of llvm.coro.id must refer to an initialized "
            "constant");
     Constant *Init = GV->getInitializer();
     Assert(isa<ConstantStruct>(Init) || isa<ConstantArray>(Init),
            "info argument of llvm.coro.id must refer to either a struct or "
            "an array");
     break;
   }
+  case Intrinsic::fptrunc_round: {
+    // Check the rounding mode
+    Metadata *MD = nullptr;
+    auto *MAV = dyn_cast<MetadataAsValue>(Call.getOperand(1));
+    if (MAV)
+      MD = MAV->getMetadata();
+
+    Assert(MD != nullptr, "missing rounding mode argument", Call);
+
+    Assert(dyn_cast<MDString>(MD),
+           ("invalid value for llvm.fptrunc.round metadata operand"
+            " (the operand should be a string)"),
+           MD);
+
+    Optional<RoundingMode> RoundMode =
+        convertStrToRoundingMode(cast<MDString>(MD)->getString());
+    Assert(RoundMode.hasValue() &&
+               RoundMode.getValue() != RoundingMode::Dynamic,
+           "unsupported rounding mode argument", Call);
+    break;
+  }
 #define INSTRUCTION(NAME, NARGS, ROUND_MODE, INTRINSIC)                        \
   case Intrinsic::INTRINSIC:
 #include "llvm/IR/ConstrainedOps.def"
     visitConstrainedFPIntrinsic(cast<ConstrainedFPIntrinsic>(Call));
     break;
   case Intrinsic::dbg_declare: // llvm.dbg.declare
     Assert(isa<MetadataAsValue>(Call.getArgOperand(0)),
            "invalid llvm.dbg.declare intrinsic call 1", Call);

llvm/lib/Target/AMDGPU/AMDGPUGISel.td

 def : GINodeEquiv<G_AMDGPU_BUFFER_ATOMIC_INC, SIbuffer_atomic_inc>;
 def : GINodeEquiv<G_AMDGPU_BUFFER_ATOMIC_DEC, SIbuffer_atomic_dec>;
 def : GINodeEquiv<G_AMDGPU_BUFFER_ATOMIC_FADD, SIbuffer_atomic_fadd>;
 def : GINodeEquiv<G_AMDGPU_BUFFER_ATOMIC_FMIN, SIbuffer_atomic_fmin>;
 def : GINodeEquiv<G_AMDGPU_BUFFER_ATOMIC_FMAX, SIbuffer_atomic_fmax>;
 def : GINodeEquiv<G_AMDGPU_BUFFER_ATOMIC_CMPSWAP, SIbuffer_atomic_cmpswap>;
 def : GINodeEquiv<G_AMDGPU_S_BUFFER_LOAD, SIsbuffer_load>;
 
+def : GINodeEquiv<G_FPTRUNC_ROUND_UPWARD, SIfptrunc_round_upward>;
+def : GINodeEquiv<G_FPTRUNC_ROUND_DOWNWARD, SIfptrunc_round_downward>;
+
 class GISelSop2Pat <
   SDPatternOperator node,
   Instruction inst,
   ValueType dst_vt,
   ValueType src0_vt = dst_vt, ValueType src1_vt = src0_vt>   : GCNPat <
 
   (dst_vt (node (src0_vt SReg_32:$src0), (src1_vt SReg_32:$src1))),
   (inst src0_vt:$src0, src1_vt:$src1)

llvm/lib/Target/AMDGPU/AMDGPUISelLowering.h

enum NodeType : unsigned {
   /// T1|v.y| | | |
   /// T2|v.z| | | |
   /// T3|v.w| | | |
   BUILD_VERTICAL_VECTOR,
   /// Pointer to the start of the shader's constant data.
   CONST_DATA_PTR,
   PC_ADD_REL_OFFSET,
   LDS,
+  FPTRUNC_ROUND_UPWARD,
+  FPTRUNC_ROUND_DOWNWARD,
+
   DUMMY_CHAIN,
   FIRST_MEM_OPCODE_NUMBER = ISD::FIRST_TARGET_MEMORY_OPCODE,
   LOAD_D16_HI,
   LOAD_D16_LO,
   LOAD_D16_HI_I8,
   LOAD_D16_HI_U8,
   LOAD_D16_LO_I8,
   LOAD_D16_LO_U8,

llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp

const char* AMDGPUTargetLowering::getTargetNodeName(unsigned Opcode) const {
   NODE_NAME_CASE(CVT_PKNORM_U16_F32)
   NODE_NAME_CASE(CVT_PK_I16_I32)
   NODE_NAME_CASE(CVT_PK_U16_U32)
   NODE_NAME_CASE(FP_TO_FP16)
   NODE_NAME_CASE(BUILD_VERTICAL_VECTOR)
   NODE_NAME_CASE(CONST_DATA_PTR)
   NODE_NAME_CASE(PC_ADD_REL_OFFSET)
   NODE_NAME_CASE(LDS)
+  NODE_NAME_CASE(FPTRUNC_ROUND_UPWARD)
+  NODE_NAME_CASE(FPTRUNC_ROUND_DOWNWARD)
   NODE_NAME_CASE(DUMMY_CHAIN)
   case AMDGPUISD::FIRST_MEM_OPCODE_NUMBER: break;
   NODE_NAME_CASE(LOAD_D16_HI)
   NODE_NAME_CASE(LOAD_D16_LO)
   NODE_NAME_CASE(LOAD_D16_HI_I8)
   NODE_NAME_CASE(LOAD_D16_HI_U8)
   NODE_NAME_CASE(LOAD_D16_LO_I8)
   NODE_NAME_CASE(LOAD_D16_LO_U8)

llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.h

public:
   bool legalizeBufferLoad(MachineInstr &MI, MachineRegisterInfo &MRI,
                           MachineIRBuilder &B, bool IsFormat,
                           bool IsTyped) const;
   bool legalizeBufferAtomic(MachineInstr &MI, MachineIRBuilder &B,
                             Intrinsic::ID IID) const;
 
   bool legalizeBVHIntrinsic(MachineInstr &MI, MachineIRBuilder &B) const;
 
+  bool legalizeFPTruncRound(MachineInstr &MI, MachineIRBuilder &B) const;
+
   bool legalizeImageIntrinsic(
       MachineInstr &MI, MachineIRBuilder &B,
       GISelChangeObserver &Observer,
       const AMDGPU::ImageDimIntrinsicInfo *ImageDimIntr) const;
 
   bool legalizeSBufferLoad(LegalizerHelper &Helper, MachineInstr &MI) const;
 
   bool legalizeAtomicIncDec(MachineInstr &MI,  MachineIRBuilder &B,

llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp

AMDGPULegalizerInfo::AMDGPULegalizerInfo(const GCNSubtarget &ST_,
   else
     FPToI.minScalar(1, S32);
 
   FPToI.minScalar(0, S32)
        .widenScalarToNextPow2(0, 32)
        .scalarize(0)
        .lower();
 
+  getActionDefinitionsBuilder(G_INTRINSIC_FPTRUNC_ROUND)
+      .customFor({S16, S32})
+      .scalarize(0)
+      .lower();
+
   // Lower roundeven into G_FRINT
   getActionDefinitionsBuilder({G_INTRINSIC_ROUND, G_INTRINSIC_ROUNDEVEN})
     .scalarize(0)
     .lower();
 
   if (ST.has16BitInsts()) {
     getActionDefinitionsBuilder({G_INTRINSIC_TRUNC, G_FCEIL, G_FRINT})
       .legalFor({S16, S32, S64})
case TargetOpcode::G_FPOW:
     return legalizeFPow(MI, B);
   case TargetOpcode::G_FFLOOR:
     return legalizeFFloor(MI, MRI, B);
   case TargetOpcode::G_BUILD_VECTOR:
     return legalizeBuildVector(MI, MRI, B);
   case TargetOpcode::G_CTLZ:
   case TargetOpcode::G_CTTZ:
     return legalizeCTLZ_CTTZ(MI, MRI, B);
+  case TargetOpcode::G_INTRINSIC_FPTRUNC_ROUND:
+    return legalizeFPTruncRound(MI, B);
   default:
     return false;
   }
 
   llvm_unreachable("expected switch to return");
 }
 
 Register AMDGPULegalizerInfo::getSegmentAperture(
bool AMDGPULegalizerInfo::legalizeBVHIntrinsic(MachineInstr &MI,
   MIB.addUse(TDescr)
      .addImm(IsA16 ? 1 : 0)
      .cloneMemRefs(MI);
 
   MI.eraseFromParent();
   return true;
 }
 
+bool AMDGPULegalizerInfo::legalizeFPTruncRound(MachineInstr &MI,
+                                               MachineIRBuilder &B) const {
+  unsigned Opc;
+  int RoundMode = MI.getOperand(2).getImm();
+
+  if (RoundMode == (int)RoundingMode::TowardPositive)
+    Opc = AMDGPU::G_FPTRUNC_ROUND_UPWARD;
+  else if (RoundMode == (int)RoundingMode::TowardNegative)
+    Opc = AMDGPU::G_FPTRUNC_ROUND_DOWNWARD;
+  else
+    return false;
+
+  B.buildInstr(Opc)
+      .addDef(MI.getOperand(0).getReg())
+      .addUse(MI.getOperand(1).getReg());
+
+  MI.eraseFromParent();
+
+  return true;
+}
+
 bool AMDGPULegalizerInfo::legalizeIntrinsic(LegalizerHelper &Helper,
                                             MachineInstr &MI) const {
   MachineIRBuilder &B = Helper.MIRBuilder;
   MachineRegisterInfo &MRI = *B.getMRI();
 
   // Replace the use G_BRCOND with the exec manipulate and branch pseudos.
   auto IntrID = MI.getIntrinsicID();
   switch (IntrID) {

llvm/lib/Target/AMDGPU/AMDGPURegisterBankInfo.cpp

case AMDGPU::G_INTRINSIC_W_SIDE_EFFECTS: {
     case Intrinsic::amdgcn_ds_gws_sema_p:
     case Intrinsic::amdgcn_ds_gws_sema_release_all: {
       // This must be an SGPR, but accept a VGPR.
       unsigned Bank = getRegBankID(MI.getOperand(1).getReg(), MRI,
                                    AMDGPU::SGPRRegBankID);
       OpdsMapping[1] = AMDGPU::getValueMapping(Bank, 32);
       break;
     }
     default:
       return getInvalidInstructionMapping();
     }
     break;
   }
   case AMDGPU::G_SELECT: {
     unsigned Size = MRI.getType(MI.getOperand(0).getReg()).getSizeInBits();
     unsigned Op2Bank = getRegBankID(MI.getOperand(2).getReg(), MRI,
                                     AMDGPU::SGPRRegBankID);
     unsigned Op3Bank = getRegBankID(MI.getOperand(3).getReg(), MRI,
                                     AMDGPU::SGPRRegBankID);
unsigned Bank = getRegBankID(MI.getOperand(0).getReg(), MRI,
                                  AMDGPU::SGPRRegBankID);
     assert(MRI.getType(MI.getOperand(0).getReg()).getSizeInBits() == 1);
     if (Bank != AMDGPU::SGPRRegBankID)
       Bank = AMDGPU::VCCRegBankID;
 
     OpdsMapping[0] = AMDGPU::getValueMapping(Bank, 1);
     break;
   }
+  case AMDGPU::G_FPTRUNC_ROUND_UPWARD:
+  case AMDGPU::G_FPTRUNC_ROUND_DOWNWARD:
+    return getDefaultMappingVOP(MI);
   }
 
   return getInstructionMapping(/*ID*/1, /*Cost*/1,
                                getOperandsMapping(OpdsMapping),
                                MI.getNumOperands());
 }

llvm/lib/Target/AMDGPU/SIISelLowering.cpp

 //===----------------------------------------------------------------------===//
 
 #include "SIISelLowering.h"
 #include "AMDGPU.h"
 #include "AMDGPUInstrInfo.h"
 #include "AMDGPUTargetMachine.h"
 #include "SIMachineFunctionInfo.h"
 #include "SIRegisterInfo.h"
+#include "llvm/ADT/FloatingPointMode.h"
 #include "llvm/ADT/Statistic.h"
 #include "llvm/Analysis/LegacyDivergenceAnalysis.h"
 #include "llvm/Analysis/OptimizationRemarkEmitter.h"
 #include "llvm/BinaryFormat/ELF.h"
 #include "llvm/CodeGen/Analysis.h"
 #include "llvm/CodeGen/FunctionLoweringInfo.h"
 #include "llvm/CodeGen/GlobalISel/GISelKnownBits.h"
 #include "llvm/CodeGen/MachineFunction.h"
if (Subtarget->has16BitInsts()) {
     setOperationAction(ISD::SINT_TO_FP, MVT::i16, Custom);
     setOperationAction(ISD::UINT_TO_FP, MVT::i16, Custom);
 
     setOperationAction(ISD::FP_TO_SINT, MVT::f16, Promote);
     setOperationAction(ISD::FP_TO_UINT, MVT::f16, Promote);
     setOperationAction(ISD::SINT_TO_FP, MVT::f16, Promote);
     setOperationAction(ISD::UINT_TO_FP, MVT::f16, Promote);
     setOperationAction(ISD::FROUND, MVT::f16, Custom);
+    setOperationAction(ISD::FPTRUNC_ROUND, MVT::f16, Custom);
 
     // F16 - VOP2 Actions.
     setOperationAction(ISD::BR_CC, MVT::f16, Expand);
     setOperationAction(ISD::SELECT_CC, MVT::f16, Expand);
 
     setOperationAction(ISD::FDIV, MVT::f16, Custom);
 
     // F16 - VOP3 Actions.
SDValue SITargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
   case ISD::EXTRACT_VECTOR_ELT:
     return lowerEXTRACT_VECTOR_ELT(Op, DAG);
   case ISD::VECTOR_SHUFFLE:
     return lowerVECTOR_SHUFFLE(Op, DAG);
   case ISD::BUILD_VECTOR:
     return lowerBUILD_VECTOR(Op, DAG);
   case ISD::FP_ROUND:
     return lowerFP_ROUND(Op, DAG);
+  case ISD::FPTRUNC_ROUND: {
+    unsigned Opc;
+    SDLoc DL(Op);
+
+    if (Op.getOperand(0)->getValueType(0) != MVT::f32)
+      return SDValue();
+
+    // Get the rounding mode from the last operand
+    int RoundMode = cast<ConstantSDNode>(Op.getOperand(1))->getZExtValue();
+    if (RoundMode == (int)RoundingMode::TowardPositive)
+      Opc = AMDGPUISD::FPTRUNC_ROUND_UPWARD;
+    else if (RoundMode == (int)RoundingMode::TowardNegative)
+      Opc = AMDGPUISD::FPTRUNC_ROUND_DOWNWARD;
+    else
+      return SDValue();
+
+    return DAG.getNode(Opc, DL, Op.getNode()->getVTList(), Op->getOperand(0));
+  }
   case ISD::TRAP:
     return lowerTRAP(Op, DAG);
   case ISD::DEBUGTRAP:
     return lowerDEBUGTRAP(Op, DAG);
   case ISD::FABS:
   case ISD::FNEG:
   case ISD::FCANONICALIZE:
   case ISD::BSWAP:
return DAG.getNode(AMDGPUISD::FMAD_FTZ, DL, VT, Op.getOperand(1),
                        Op.getOperand(2), Op.getOperand(3));
 
   case Intrinsic::amdgcn_if_break:
     return SDValue(DAG.getMachineNode(AMDGPU::SI_IF_BREAK, DL, VT,
                                       Op->getOperand(1), Op->getOperand(2)), 0);
 
   case Intrinsic::amdgcn_groupstaticsize: {
     Triple::OSType OS = getTargetMachine().getTargetTriple().getOS();
     if (OS == Triple::AMDHSA || OS == Triple::AMDPAL)
       return Op;
 
     const Module *M = MF.getFunction().getParent();
     const GlobalValue *GV =
         M->getNamedValue(Intrinsic::getName(Intrinsic::amdgcn_groupstaticsize));
     SDValue GA = DAG.getTargetGlobalAddress(GV, DL, MVT::i32, 0,
                                             SIInstrInfo::MO_ABS32_LO);
     return {DAG.getMachineNode(AMDGPU::S_MOV_B32, DL, MVT::i32, GA), 0};

llvm/lib/Target/AMDGPU/SIInstrInfo.td

def SIload_d16_hi_i8 : SDNode<"AMDGPUISD::LOAD_D16_HI_I8",
   [SDNPMayLoad, SDNPMemOperand, SDNPHasChain]
 >;
 
 def SIdenorm_mode : SDNode<"AMDGPUISD::DENORM_MODE",
   SDTypeProfile<0 ,1, [SDTCisInt<0>]>,
   [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue]
 >;
 
+def SIfptrunc_round_upward : SDNode<"AMDGPUISD::FPTRUNC_ROUND_UPWARD",
+  SDTFPRoundOp
+>;
+
+def SIfptrunc_round_downward : SDNode<"AMDGPUISD::FPTRUNC_ROUND_DOWNWARD",
+  SDTFPRoundOp
+>;
+
 //===----------------------------------------------------------------------===//
 // ValueType helpers
 //===----------------------------------------------------------------------===//
 
 // Returns 1 if the source arguments have modifiers, 0 if they do not.
 // XXX - do f16 instructions?
 class isFloatType<ValueType SrcVT> {
   bit ret = !or(!eq(SrcVT.Value, f16.Value),

llvm/lib/Target/AMDGPU/SIInstructions.td

 }
 
 def EXIT_STRICT_WQM : SPseudoInstSI <(outs SReg_1:$sdst), (ins SReg_1:$src0)> {
   let hasSideEffects = 0;
   let mayLoad = 0;
   let mayStore = 0;
 }
 
+// Pseudo instructions used for @llvm.fptrunc.round upward
+// and @llvm.fptrunc.round downward.
+// These intrinsics will be legalized to G_FPTRUNC_ROUND_UPWARD
+// and G_FPTRUNC_ROUND_DOWNWARD before being lowered to
+// FPTRUNC_UPWARD_PSEUDO and FPTRUNC_DOWNWARD_PSEUDO.
+// The final codegen is done in the ModeRegister pass.
+let Uses = [MODE, EXEC] in {
+def FPTRUNC_UPWARD_PSEUDO : VPseudoInstSI <(outs VGPR_32:$vdst),
+  (ins VGPR_32:$src0),
+  [(set f16:$vdst, (SIfptrunc_round_upward f32:$src0))]>;
+
+def FPTRUNC_DOWNWARD_PSEUDO : VPseudoInstSI <(outs VGPR_32:$vdst),
+  (ins VGPR_32:$src0),
+  [(set f16:$vdst, (SIfptrunc_round_downward f32:$src0))]>;
+} // End Uses = [MODE, EXEC]
+
 // Invert the exec mask and overwrite the inactive lanes of dst with inactive,
 // restoring it after we're done.
 let Defs = [SCC] in {
 def V_SET_INACTIVE_B32 : VPseudoInstSI <(outs VGPR_32:$vdst),
   (ins VGPR_32: $src, VSrc_b32:$inactive),
   [(set i32:$vdst, (int_amdgcn_set_inactive i32:$src, i32:$inactive))]> {
   let Constraints = "$src = $vdst";
 }
def G_SI_CALL : AMDGPUGenericInstruction {
   let InOperandList = (ins type0:$src0, unknown:$callee);
   let Size = 4;
   let isCall = 1;
   let UseNamedOperandTable = 1;
   let SchedRW = [WriteBranch];
   // TODO: Should really base this on the call target
   let isConvergent = 1;
 }
+
+def G_FPTRUNC_ROUND_UPWARD : AMDGPUGenericInstruction {
+  let OutOperandList = (outs type0:$vdst);
+  let InOperandList = (ins type1:$src0);
+  let hasSideEffects = 0;
+}
+
+def G_FPTRUNC_ROUND_DOWNWARD : AMDGPUGenericInstruction {
+  let OutOperandList = (outs type0:$vdst);
+  let InOperandList = (ins type1:$src0);
+  let hasSideEffects = 0;
+}

llvm/lib/Target/AMDGPU/SIModeRegister.cpp

 FunctionPass *llvm::createSIModeRegisterPass() { return new SIModeRegister(); }
 
 // Determine the Mode register setting required for this instruction.
 // Instructions which don't use the Mode register return a null Status.
 // Note this currently only deals with instructions that use the floating point
 // double precision setting.
 Status SIModeRegister::getInstructionMode(MachineInstr &MI,
                                           const SIInstrInfo *TII) {
-  if (TII->usesFPDPRounding(MI)) {
+  if (TII->usesFPDPRounding(MI) ||
+      MI.getOpcode() == AMDGPU::FPTRUNC_UPWARD_PSEUDO ||
+      MI.getOpcode() == AMDGPU::FPTRUNC_DOWNWARD_PSEUDO) {
     switch (MI.getOpcode()) {
     case AMDGPU::V_INTERP_P1LL_F16:
     case AMDGPU::V_INTERP_P1LV_F16:
     case AMDGPU::V_INTERP_P2_F16:
       // f16 interpolation instructions need double precision round to zero
       return Status(FP_ROUND_MODE_DP(3),
                     FP_ROUND_MODE_DP(FP_ROUND_ROUND_TO_ZERO));
+    case AMDGPU::FPTRUNC_UPWARD_PSEUDO: {
+      // Replacing the pseudo by a real instruction
+      MI.setDesc(TII->get(AMDGPU::V_CVT_F16_F32_e32));
+      return Status(FP_ROUND_MODE_DP(3),
+                    FP_ROUND_MODE_DP(FP_ROUND_ROUND_TO_INF));
+    }
+    case AMDGPU::FPTRUNC_DOWNWARD_PSEUDO: {
+      // Replacing the pseudo by a real instruction
+      MI.setDesc(TII->get(AMDGPU::V_CVT_F16_F32_e32));
+      return Status(FP_ROUND_MODE_DP(3),
+                    FP_ROUND_MODE_DP(FP_ROUND_ROUND_TO_NEGINF));
+    }
     default:
       return DefaultStatus;
     }
   }
   return Status();
 }
 
 // Insert a setreg instruction to update the Mode register.
bool SIModeRegister::runOnMachineFunction(MachineFunction &MF) {
 
   // Phase 1 - determine the initial mode required by each block, and add setreg
   // instructions for intra block requirements.
   for (MachineBasicBlock &BB : MF)
     processBlockPhase1(BB, TII);
 
   // Phase 2 - determine the exit mode from each block. We add all blocks to the
   // list here, but will also add any that need to be revisited during Phase 2
   // processing.
   for (MachineBasicBlock &BB : MF)
     Phase2List.push(&BB);
   while (!Phase2List.empty()) {
     processBlockPhase2(*Phase2List.front(), TII);
     Phase2List.pop();
   }
 
   // Phase 3 - add an initial setreg to each block where the required entry mode
   // is not satisfied by the exit mode of all its predecessors.
   for (MachineBasicBlock &BB : MF)
     processBlockPhase3(BB, TII);
 
   BlockInfo.clear();
 
   return Changed;
 }

llvm/test/CodeGen/AArch64/GlobalISel/legalizer-info-validation.mir

 # DEBUG-NEXT: .. the first uncovered type index: 2, OK
 # DEBUG-NEXT: .. the first uncovered imm index: 0, OK
 #
 # DEBUG-NEXT: G_FREEZE (opcode {{[0-9]+}}): 1 type index, 0 imm indices
 # DEBUG-NEXT: .. opcode {{[0-9]+}} is aliased to {{[0-9]+}}
 # DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected
 # DEBUG-NEXT: .. imm index coverage check SKIPPED: user-defined predicate detected
 #
+# DEBUG-NEXT: G_INTRINSIC_FPTRUNC_ROUND (opcode 71): 2 type indices, 0 imm indices
+# DEBUG-NEXT: .. type index coverage check SKIPPED: no rules defined
+# DEBUG-NEXT: .. imm index coverage check SKIPPED: no rules defined
+#
 # DEBUG-NEXT: G_INTRINSIC_TRUNC (opcode {{[0-9]+}}): 1 type index, 0 imm indices
 # DEBUG-NEXT: .. opcode {{[0-9]+}} is aliased to {{[0-9]+}}
 # DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected
 # DEBUG-NEXT: .. imm index coverage check SKIPPED: user-defined predicate detected
 # DEBUG-NEXT: G_INTRINSIC_ROUND (opcode {{[0-9]+}}): 1 type index, 0 imm indices
 # DEBUG-NEXT: .. opcode {{[0-9]+}} is aliased to {{[0-9]+}}
 # DEBUG-NEXT: .. type index coverage check SKIPPED: user-defined predicate detected
 # DEBUG-NEXT: .. imm index coverage check SKIPPED: user-defined predicate detected

llvm/test/CodeGen/AMDGPU/fail.llvm.fptrunc.round.ll

+; RUN: not --crash llc -march=amdgcn -mcpu=gfx1030 -verify-machineinstrs -o /dev/null %s 2>&1 | FileCheck %s --ignore-case --check-prefix=FAIL
+; RUN: not --crash llc -global-isel -march=amdgcn -mcpu=gfx1030 -verify-machineinstrs -o /dev/null %s 2>&1 | FileCheck %s --ignore-case --check-prefix=FAIL
+
+define amdgpu_gs void @test_fptrunc_round_legalization(double %a, i32 %data0, <4 x i32> %data1, half addrspace(1)* %out) {
+; FAIL: LLVM ERROR: Cannot select
+  %res = call half @llvm.fptrunc.round.f64(double %a, metadata !"round.upward")
+  store half %res, half addrspace(1)* %out, align 4
+  ret void
+}
+
+declare half @llvm.fptrunc.round.f64(double, metadata)

llvm/test/CodeGen/AMDGPU/llvm.fptrunc.round.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -march=amdgcn -mcpu=gfx1030 -verify-machineinstrs < %s | FileCheck %s
+; RUN: llc -march=amdgcn -mcpu=gfx1010 -verify-machineinstrs < %s | FileCheck %s
+; RUN: llc -global-isel -march=amdgcn -mcpu=gfx1030 -verify-machineinstrs < %s | FileCheck %s
+
+define amdgpu_gs void @test_fptrunc_round_upward(float %a, i32 %data0, <4 x i32> %data1, half addrspace(1)* %out) {
+; CHECK-LABEL: test_fptrunc_round_upward:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 1), 1
+; CHECK-NEXT:    v_cvt_f16_f32_e32 v0, v0
+; CHECK-NEXT:    global_store_short v[6:7], v0, off
+; CHECK-NEXT:    s_endpgm
+  %res = call half @llvm.fptrunc.round(float %a, metadata !"round.upward")
+  store half %res, half addrspace(1)* %out, align 4
+  ret void
+}
+
+define amdgpu_gs void @test_fptrunc_round_downward(float %a, i32 %data0, <4 x i32> %data1, half addrspace(1)* %out) {
+; CHECK-LABEL: test_fptrunc_round_downward:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 3, 1), 1
+; CHECK-NEXT:    v_cvt_f16_f32_e32 v0, v0
+; CHECK-NEXT:    global_store_short v[6:7], v0, off
+; CHECK-NEXT:    s_endpgm
+  %res = call half @llvm.fptrunc.round(float %a, metadata !"round.downward")
+  store half %res, half addrspace(1)* %out, align 4
+  ret void
+}
+
+define amdgpu_gs void @test_fptrunc_round_upward_multiple_calls(float %a, float %b, i32 %data0, <4 x i32> %data1, half addrspace(1)* %out) {
+; CHECK-LABEL: test_fptrunc_round_upward_multiple_calls:
+; CHECK:       ; %bb.0:
+; CHECK-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 1), 1
+; CHECK-NEXT:    v_cvt_f16_f32_e32 v0, v0
+; CHECK-NEXT:    v_cvt_f16_f32_e32 v2, v1
+; CHECK-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 2, 2), 2
+; CHECK-NEXT:    v_cvt_f16_f32_e32 v1, v1
+; CHECK-NEXT:    s_setreg_imm32_b32 hwreg(HW_REG_MODE, 3, 1), 0
+; CHECK-NEXT:    v_add_f16_e32 v0, v0, v2
+; CHECK-NEXT:    v_add_f16_e32 v0, v1, v0
+; CHECK-NEXT:    global_store_short v[7:8], v0, off
+; CHECK-NEXT:    s_endpgm
+  %res1 = call half @llvm.fptrunc.round(float %a, metadata !"round.upward")
+  %res2 = call half @llvm.fptrunc.round(float %b, metadata !"round.upward")
+  %res3 = call half @llvm.fptrunc.round(float %b, metadata !"round.downward")
+  %res4 = fadd half %res1, %res2
+  %res5 = fadd half %res3, %res4
+  store half %res5, half addrspace(1)* %out, align 4
+  ret void
+}
+
+declare half @llvm.fptrunc.round(float, metadata)

llvm/test/Verifier/llvm.fptrunc.round.ll

+; RUN: not opt -verify < %s 2>&1 | FileCheck %s
+
+declare half @llvm.fptrunc.round(float, metadata)
+
+define void @test_fptrunc_round_dynamic(float %a) {
+; CHECK: unsupported rounding mode argument
+  %res = call half @llvm.fptrunc.round(float %a, metadata !"round.dynamic")
+; CHECK: unsupported rounding mode argument
+  %res1 = call half @llvm.fptrunc.round(float %a, metadata !"round.test")
+; CHECK: invalid value for llvm.fptrunc.round metadata operand (the operand should be a string)
+  %res2 = call half @llvm.fptrunc.round(float %a, metadata i32 5)
+  ret void
+}