This is an archive of the discontinued LLVM Phabricator instance.

Differential D71358

AMDGPU: Remove denormal subtarget features
ClosedPublic

Authored by arsenm on Dec 11 2019, 6:46 AM.

Details

Reviewers
rampitec
b-sumner
Summary

Switch to using the denormal-fp-math/denormal-fp-math-f32
attributes.

Diff Detail

Event Timeline

arsenm created this revision.Dec 11 2019, 6:46 AM
Herald added a project: Restricted Project. · View Herald TranscriptDec 11 2019, 6:46 AM
Herald added subscribers: hiraditya, t-tye, tpr and 6 others. · View Herald Transcript
rampitec added a reviewer: b-sumner.Dec 11 2019, 9:41 AM

How is it now achieved that we do not support denormals by default before GFX9?

arsenm added a comment.Dec 11 2019, 9:45 AM
In D71358#1780038, @rampitec wrote:

How is it now achieved that we do not support denormals by default before GFX9?

This is a frontend decision. clang emits the attribute based on the subtarget preference

arsenm added a comment.Dec 11 2019, 9:56 AM
In D71358#1780059, @arsenm wrote:
In D71358#1780038, @rampitec wrote:

How is it now achieved that we do not support denormals by default before GFX9?

This is a frontend decision. clang emits the attribute based on the subtarget preference

This is also how it always worked. The backend uniformly treated denormals as off by default

rampitec added a comment.Dec 11 2019, 10:03 AM

The code LGTM. But I want Brian to review it.

b-sumner added a comment.Dec 11 2019, 10:36 AM
In D71358#1780083, @arsenm wrote:
In D71358#1780059, @arsenm wrote:
In D71358#1780038, @rampitec wrote:

How is it now achieved that we do not support denormals by default before GFX9?

This is a frontend decision. clang emits the attribute based on the subtarget preference

This is also how it always worked. The backend uniformly treated denormals as off by default

Where is clang going to get the information it needs after this patch?

nhaehnle added a comment.Dec 13 2019, 5:22 AM

Would it be possible to do this patch in two steps:

  1. First step: Add the parsing of the new function attribute, but keep the subtarget feature.
  2. Second step: Remove the subtarget feature.

This would make the upgrade process easier for out-of-tree users (Mesa, LLPC).

Apart from that this seems like a good change to me.

arsenm added a comment.Dec 13 2019, 7:19 AM
In D71358#1780166, @b-sumner wrote:
In D71358#1780083, @arsenm wrote:
In D71358#1780059, @arsenm wrote:
In D71358#1780038, @rampitec wrote:

How is it now achieved that we do not support denormals by default before GFX9?

This is a frontend decision. clang emits the attribute based on the subtarget preference

This is also how it always worked. The backend uniformly treated denormals as off by default

Where is clang going to get the information it needs after this patch?

Where it already does. This doesn’t change. It just changes what clang produces for the denormal flushing flag/subtarget default

arsenm added a comment.Dec 13 2019, 7:42 AM
In D71358#1783418, @nhaehnle wrote:

Would it be possible to do this patch in two steps:

  1. First step: Add the parsing of the new function attribute, but keep the subtarget feature.
  2. Second step: Remove the subtarget feature.

This would make the upgrade process easier for out-of-tree users (Mesa, LLPC).

Apart from that this seems like a good change to me.

That is basically how it’s split. Once the form of the attributes is finalized, frontends can start emitting the new form and we just won’t look it it until this patch

nhaehnle added a comment.Dec 15 2019, 1:37 PM

Once the form of the attributes is finalized, frontends can start emitting the new form and we just won’t look it it until this patch

Okay, that makes sense.

b-sumner added a comment.Jan 6 2020, 9:44 AM

I am OK with proceeding here.

mareko added a subscriber: mareko.Jan 6 2020, 12:31 PM

Can you push this after (not before) LLVM 10 has been branched?

arsenm updated this revision to Diff 254212.Apr 1 2020, 8:19 AM
arsenm edited the summary of this revision. (Show Details)

Rebase and stop considering the calling convention for the default

Herald added a subscriber: kerbowa. · View Herald TranscriptApr 1 2020, 8:19 AM
rampitec accepted this revision.Apr 2 2020, 12:35 PM

LGTM

This revision is now accepted and ready to land.Apr 2 2020, 12:35 PM
arsenm closed this revision.Apr 2 2020, 2:19 PM

5660bb6bc9ac5ed910d95210e43ed437f155212d

Revision Contents

PathSize
llvm/
lib/
Target/
AMDGPU/
24 lines
5 lines
9 lines
2 lines
14 lines
5 lines
2 lines
25 lines
21 lines
2 lines
4 lines
2 lines
Utils/
7 lines
27 lines
test/
Analysis/
CostModel/
AMDGPU/
4 lines
CodeGen/
AMDGPU/
5 lines
4 lines
6 lines
8 lines
29 lines
8 lines
10 lines
6 lines
12 lines
20 lines
6 lines
4 lines
14 lines
6 lines
8 lines
6 lines
10 lines
16 lines
24 lines
18 lines
10 lines
8 lines
10 lines
8 lines
4 lines
8 lines
12 lines
26 lines
30 lines
10 lines
2 lines
2 lines
4 lines
30 lines
25 lines
2 lines
6 lines
19 lines
8 lines
29 lines
22 lines
18 lines
52 lines
61 lines
22 lines
6 lines
8 lines
10 lines

Diff 233357

llvm/lib/Target/AMDGPU/AMDGPU.td

Show First 20 Lines • Show All 482 Lines • ▼ Show 20 Linesdef FeatureNoDataDepHazard : SubtargetFeature<"no-data-dep-hazard",
"true", "true",
"Does not need SW waitstates" "Does not need SW waitstates"
>; >;
//===------------------------------------------------------------===// //===------------------------------------------------------------===//
// Subtarget Features (options and debugging) // Subtarget Features (options and debugging)
//===------------------------------------------------------------===// //===------------------------------------------------------------===//
// Denormal handling for fp64 and fp16 is controlled by the same
// config register when fp16 supported.
// TODO: Do we need a separate f16 setting when not legal?
def FeatureFP64FP16Denormals : SubtargetFeature<"fp64-fp16-denormals",
"FP64FP16Denormals",
"true",
"Enable double and half precision denormal handling",
[FeatureFP64]
>;
def FeatureFP64Denormals : SubtargetFeature<"fp64-denormals",
"FP64FP16Denormals",
"true",
"Enable double and half precision denormal handling",
[FeatureFP64, FeatureFP64FP16Denormals]
>;
def FeatureFP16Denormals : SubtargetFeature<"fp16-denormals",
"FP64FP16Denormals",
"true",
"Enable half precision denormal handling",
[FeatureFP64FP16Denormals]
>;
def FeatureFPExceptions : SubtargetFeature<"fp-exceptions", def FeatureFPExceptions : SubtargetFeature<"fp-exceptions",
"FPExceptions", "FPExceptions",
"true", "true",
"Enable floating point exceptions" "Enable floating point exceptions"
>; >;
class FeatureMaxPrivateElementSize<int size> : SubtargetFeature< class FeatureMaxPrivateElementSize<int size> : SubtargetFeature<
"max-private-element-size-"#size, "max-private-element-size-"#size,
▲ Show 20 Lines • Show All 651 Lines • Show Last 20 Lines

llvm/lib/Target/AMDGPU/AMDGPUCodeGenPrepare.cpp

Show All 9 Lines
/// This pass does misc. AMDGPU optimizations on IR before instruction /// This pass does misc. AMDGPU optimizations on IR before instruction
/// selection. /// selection.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "AMDGPU.h" #include "AMDGPU.h"
#include "AMDGPUSubtarget.h" #include "AMDGPUSubtarget.h"
#include "AMDGPUTargetMachine.h" #include "AMDGPUTargetMachine.h"
#include "llvm/ADT/FloatingPointMode.h"
#include "llvm/ADT/StringRef.h" #include "llvm/ADT/StringRef.h"
#include "llvm/Analysis/AssumptionCache.h" #include "llvm/Analysis/AssumptionCache.h"
#include "llvm/Analysis/LegacyDivergenceAnalysis.h" #include "llvm/Analysis/LegacyDivergenceAnalysis.h"
#include "llvm/Analysis/Loads.h" #include "llvm/Analysis/Loads.h"
#include "llvm/Analysis/ValueTracking.h" #include "llvm/Analysis/ValueTracking.h"
#include "llvm/CodeGen/Passes.h" #include "llvm/CodeGen/Passes.h"
#include "llvm/CodeGen/TargetPassConfig.h" #include "llvm/CodeGen/TargetPassConfig.h"
#include "llvm/IR/Attributes.h" #include "llvm/IR/Attributes.h"
▲ Show 20 Lines • Show All 1,003 Lines • ▼ Show 20 Linesbool AMDGPUCodeGenPrepare::runOnFunction(Function &F) {
if (!TPC) if (!TPC)
return false; return false;
const AMDGPUTargetMachine &TM = TPC->getTM<AMDGPUTargetMachine>(); const AMDGPUTargetMachine &TM = TPC->getTM<AMDGPUTargetMachine>();
ST = &TM.getSubtarget<GCNSubtarget>(F); ST = &TM.getSubtarget<GCNSubtarget>(F);
AC = &getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F); AC = &getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F);
DA = &getAnalysis<LegacyDivergenceAnalysis>(); DA = &getAnalysis<LegacyDivergenceAnalysis>();
HasUnsafeFPMath = hasUnsafeFPMath(F); HasUnsafeFPMath = hasUnsafeFPMath(F);
HasFP32Denormals = ST->hasFP32Denormals(F);
AMDGPU::SIModeRegisterDefaults Mode(F);
HasFP32Denormals = Mode.allFP32Denormals();
bool MadeChange = false; bool MadeChange = false;
for (BasicBlock &BB : F) { for (BasicBlock &BB : F) {
BasicBlock::iterator Next; BasicBlock::iterator Next;
for (BasicBlock::iterator I = BB.begin(), E = BB.end(); I != E; I = Next) { for (BasicBlock::iterator I = BB.begin(), E = BB.end(); I != E; I = Next) {
Next = std::next(I); Next = std::next(I);
MadeChange |= visit(*I); MadeChange |= visit(*I);
Show All 18 Lines

llvm/lib/Target/AMDGPU/AMDGPUFeatures.td

Show All 12 Lines
>; >;
def FeatureFMA : SubtargetFeature<"fmaf", def FeatureFMA : SubtargetFeature<"fmaf",
"FMA", "FMA",
"true", "true",
"Enable single precision FMA (not as fast as mul+add, but fused)" "Enable single precision FMA (not as fast as mul+add, but fused)"
>; >;
// Some instructions do not support denormals despite this flag. Using
// fp32 denormals also causes instructions to run at the double
// precision rate for the device.
def FeatureFP32Denormals : SubtargetFeature<"fp32-denormals",
"FP32Denormals",
"true",
"Enable single precision denormal handling"
>;
class SubtargetFeatureLocalMemorySize <int Value> : SubtargetFeature< class SubtargetFeatureLocalMemorySize <int Value> : SubtargetFeature<
"localmemorysize"#Value, "localmemorysize"#Value,
"LocalMemorySize", "LocalMemorySize",
!cast<string>(Value), !cast<string>(Value),
"The size of local memory in bytes" "The size of local memory in bytes"
>; >;
def FeatureLocalMemorySize0 : SubtargetFeatureLocalMemorySize<0>; def FeatureLocalMemorySize0 : SubtargetFeatureLocalMemorySize<0>;
Show All 26 Lines

llvm/lib/Target/AMDGPU/AMDGPUISelDAGToDAG.cpp

Show First 20 Lines • Show All 391 Lines • ▼ Show 20 Lines
#ifdef EXPENSIVE_CHECKS #ifdef EXPENSIVE_CHECKS
DominatorTree & DT = getAnalysis<DominatorTreeWrapperPass>().getDomTree(); DominatorTree & DT = getAnalysis<DominatorTreeWrapperPass>().getDomTree();
LoopInfo * LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo(); LoopInfo * LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo();
for (auto &L : LI->getLoopsInPreorder()) { for (auto &L : LI->getLoopsInPreorder()) {
assert(L->isLCSSAForm(DT)); assert(L->isLCSSAForm(DT));
} }
#endif #endif
Subtarget = &MF.getSubtarget<GCNSubtarget>(); Subtarget = &MF.getSubtarget<GCNSubtarget>();
Mode = AMDGPU::SIModeRegisterDefaults(MF.getFunction(), *Subtarget); Mode = AMDGPU::SIModeRegisterDefaults(MF.getFunction());
return SelectionDAGISel::runOnMachineFunction(MF); return SelectionDAGISel::runOnMachineFunction(MF);
} }
bool AMDGPUDAGToDAGISel::matchLoadD16FromBuildVector(SDNode *N) const { bool AMDGPUDAGToDAGISel::matchLoadD16FromBuildVector(SDNode *N) const {
assert(Subtarget->d16PreservesUnusedBits()); assert(Subtarget->d16PreservesUnusedBits());
MVT VT = N->getValueType(0).getSimpleVT(); MVT VT = N->getValueType(0).getSimpleVT();
if (VT != MVT::v2i16 && VT != MVT::v2f16) if (VT != MVT::v2i16 && VT != MVT::v2f16)
return false; return false;
▲ Show 20 Lines • Show All 2,453 Lines • Show Last 20 Lines

llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp

Show First 20 Lines • Show All 1,579 Lines • ▼ Show 20 LinesSDValue AMDGPUTargetLowering::LowerDIVREM24(SDValue Op, SelectionDAG &DAG,
// float fqneg = -fq; // float fqneg = -fq;
SDValue fqneg = DAG.getNode(ISD::FNEG, DL, FltVT, fq); SDValue fqneg = DAG.getNode(ISD::FNEG, DL, FltVT, fq);
MachineFunction &MF = DAG.getMachineFunction(); MachineFunction &MF = DAG.getMachineFunction();
const AMDGPUMachineFunction *MFI = MF.getInfo<AMDGPUMachineFunction>(); const AMDGPUMachineFunction *MFI = MF.getInfo<AMDGPUMachineFunction>();
// float fr = mad(fqneg, fb, fa); // float fr = mad(fqneg, fb, fa);
unsigned OpCode = MFI->getMode().allFP32Denormals() ? unsigned OpCode = !MFI->getMode().allFP32Denormals() ?
(unsigned)AMDGPUISD::FMAD_FTZ : (unsigned)ISD::FMAD :
(unsigned)ISD::FMAD; (unsigned)AMDGPUISD::FMAD_FTZ;
SDValue fr = DAG.getNode(OpCode, DL, FltVT, fqneg, fb, fa); SDValue fr = DAG.getNode(OpCode, DL, FltVT, fqneg, fb, fa);
// int iq = (int)fq; // int iq = (int)fq;
SDValue iq = DAG.getNode(ToInt, DL, IntVT, fq); SDValue iq = DAG.getNode(ToInt, DL, IntVT, fq);
// fr = fabs(fr); // fr = fabs(fr);
fr = DAG.getNode(ISD::FABS, DL, FltVT, fr); fr = DAG.getNode(ISD::FABS, DL, FltVT, fr);
▲ Show 20 Lines • Show All 66 Lines • ▼ Show 20 Lines if (DAG.MaskedValueIsZero(RHS, APInt::getHighBitsSet(64, 32)) &&
return; return;
} }
if (isTypeLegal(MVT::i64)) { if (isTypeLegal(MVT::i64)) {
MachineFunction &MF = DAG.getMachineFunction(); MachineFunction &MF = DAG.getMachineFunction();
const SIMachineFunctionInfo *MFI = MF.getInfo<SIMachineFunctionInfo>(); const SIMachineFunctionInfo *MFI = MF.getInfo<SIMachineFunctionInfo>();
// Compute denominator reciprocal. // Compute denominator reciprocal.
unsigned FMAD = MFI->getMode().allFP32Denormals() ? unsigned FMAD = !MFI->getMode().allFP32Denormals() ?
(unsigned)AMDGPUISD::FMAD_FTZ : (unsigned)ISD::FMAD :
(unsigned)ISD::FMAD; (unsigned)AMDGPUISD::FMAD_FTZ;
SDValue Cvt_Lo = DAG.getNode(ISD::UINT_TO_FP, DL, MVT::f32, RHS_Lo); SDValue Cvt_Lo = DAG.getNode(ISD::UINT_TO_FP, DL, MVT::f32, RHS_Lo);
SDValue Cvt_Hi = DAG.getNode(ISD::UINT_TO_FP, DL, MVT::f32, RHS_Hi); SDValue Cvt_Hi = DAG.getNode(ISD::UINT_TO_FP, DL, MVT::f32, RHS_Hi);
SDValue Mad1 = DAG.getNode(FMAD, DL, MVT::f32, Cvt_Hi, SDValue Mad1 = DAG.getNode(FMAD, DL, MVT::f32, Cvt_Hi,
DAG.getConstantFP(APInt(32, 0x4f800000).bitsToFloat(), DL, MVT::f32), DAG.getConstantFP(APInt(32, 0x4f800000).bitsToFloat(), DL, MVT::f32),
Cvt_Lo); Cvt_Lo);
SDValue Rcp = DAG.getNode(AMDGPUISD::RCP, DL, MVT::f32, Mad1); SDValue Rcp = DAG.getNode(AMDGPUISD::RCP, DL, MVT::f32, Mad1);
SDValue Mul1 = DAG.getNode(ISD::FMUL, DL, MVT::f32, Rcp, SDValue Mul1 = DAG.getNode(ISD::FMUL, DL, MVT::f32, Rcp,
▲ Show 20 Lines • Show All 3,006 Lines • Show Last 20 Lines

llvm/lib/Target/AMDGPU/AMDGPUInstructions.td

Show First 20 Lines • Show All 894 Lines • ▼ Show 20 Linesdef fminnum_like_oneuse : PatFrags<(ops node:$src0, node:$src1),
[(fminnum_ieee_oneuse node:$src0, node:$src1), [(fminnum_ieee_oneuse node:$src0, node:$src1),
(fminnum_oneuse node:$src0, node:$src1)] (fminnum_oneuse node:$src0, node:$src1)]
>; >;
def fmaxnum_like_oneuse : PatFrags<(ops node:$src0, node:$src1), def fmaxnum_like_oneuse : PatFrags<(ops node:$src0, node:$src1),
[(fmaxnum_ieee_oneuse node:$src0, node:$src1), [(fmaxnum_ieee_oneuse node:$src0, node:$src1),
(fmaxnum_oneuse node:$src0, node:$src1)] (fmaxnum_oneuse node:$src0, node:$src1)]
>; >;
def any_fmad : PatFrags<(ops node:$src0, node:$src1, node:$src2),
[(fmad node:$src0, node:$src1, node:$src2),
(AMDGPUfmad_ftz node:$src0, node:$src1, node:$src2)]
>;

llvm/lib/Target/AMDGPU/AMDGPUMachineFunction.cpp

Show All 12 Lines
using namespace llvm; using namespace llvm;
AMDGPUMachineFunction::AMDGPUMachineFunction(const MachineFunction &MF) : AMDGPUMachineFunction::AMDGPUMachineFunction(const MachineFunction &MF) :
MachineFunctionInfo(), MachineFunctionInfo(),
LocalMemoryObjects(), LocalMemoryObjects(),
ExplicitKernArgSize(0), ExplicitKernArgSize(0),
LDSSize(0), LDSSize(0),
Mode(MF.getFunction(), MF.getSubtarget<GCNSubtarget>()), Mode(MF.getFunction()),
IsEntryFunction(AMDGPU::isEntryFunctionCC(MF.getFunction().getCallingConv())), IsEntryFunction(AMDGPU::isEntryFunctionCC(MF.getFunction().getCallingConv())),
NoSignedZerosFPMath(MF.getTarget().Options.NoSignedZerosFPMath), NoSignedZerosFPMath(MF.getTarget().Options.NoSignedZerosFPMath),
MemoryBound(false), MemoryBound(false),
WaveLimiter(false) { WaveLimiter(false) {
const AMDGPUSubtarget &ST = AMDGPUSubtarget::get(MF); const AMDGPUSubtarget &ST = AMDGPUSubtarget::get(MF);
// FIXME: Should initialize KernArgSize based on ExplicitKernelArgOffset, // FIXME: Should initialize KernArgSize based on ExplicitKernelArgOffset,
// except reserved size is not correctly aligned. // except reserved size is not correctly aligned.
Show All 35 Lines

llvm/lib/Target/AMDGPU/AMDGPUSubtarget.h

Show First 20 Lines • Show All 59 Lines • ▼ Show 20 Linespublic:
}; };
private: private:
Triple TargetTriple; Triple TargetTriple;
protected: protected:
bool Has16BitInsts; bool Has16BitInsts;
bool HasMadMixInsts; bool HasMadMixInsts;
bool FP32Denormals;
bool FPExceptions; bool FPExceptions;
bool HasSDWA; bool HasSDWA;
bool HasVOP3PInsts; bool HasVOP3PInsts;
bool HasMulI24; bool HasMulI24;
bool HasMulU24; bool HasMulU24;
bool HasInv2PiInlineImm; bool HasInv2PiInlineImm;
bool HasFminFmaxLegacy; bool HasFminFmaxLegacy;
bool EnablePromoteAlloca; bool EnablePromoteAlloca;
▲ Show 20 Lines • Show All 66 Lines • ▼ Show 20 Linespublic:
bool has16BitInsts() const { bool has16BitInsts() const {
return Has16BitInsts; return Has16BitInsts;
} }
bool hasMadMixInsts() const { bool hasMadMixInsts() const {
return HasMadMixInsts; return HasMadMixInsts;
} }
bool hasFP32Denormals(const Function &F) const {
// FIXME: This should not be a property of the subtarget. This should be a
// property with a default set by the calling convention which can be
// overridden by attributes. For now, use the subtarget feature as a
// placeholder attribute. The function arguments only purpose is to
// discourage use without a function context until this is removed.
return FP32Denormals;
}
bool hasFPExceptions() const { bool hasFPExceptions() const {
return FPExceptions; return FPExceptions;
} }
bool hasSDWA() const { bool hasSDWA() const {
return HasSDWA; return HasSDWA;
} }
▲ Show 20 Lines • Show All 122 Lines • ▼ Show 20 Linesprotected:
int LDSBankCount; int LDSBankCount;
unsigned MaxPrivateElementSize; unsigned MaxPrivateElementSize;
// Possibly statically set by tablegen, but may want to be overridden. // Possibly statically set by tablegen, but may want to be overridden.
bool FastFMAF32; bool FastFMAF32;
bool HalfRate64Ops; bool HalfRate64Ops;
// Dynamially set bits that enable features. // Dynamially set bits that enable features.
bool FP64FP16Denormals;
bool FlatForGlobal; bool FlatForGlobal;
bool AutoWaitcntBeforeBarrier; bool AutoWaitcntBeforeBarrier;
bool CodeObjectV3; bool CodeObjectV3;
bool UnalignedScratchAccess; bool UnalignedScratchAccess;
bool UnalignedBufferAccess; bool UnalignedBufferAccess;
bool HasApertureRegs; bool HasApertureRegs;
bool EnableXNACK; bool EnableXNACK;
bool DoesNotSupportXNACK; bool DoesNotSupportXNACK;
▲ Show 20 Lines • Show All 305 Lines • ▼ Show 20 Lines bool dumpCode() const {
return DumpCode; return DumpCode;
} }
/// Return the amount of LDS that can be used that will not restrict the /// Return the amount of LDS that can be used that will not restrict the
/// occupancy lower than WaveCount. /// occupancy lower than WaveCount.
unsigned getMaxLocalMemSizeWithWaveCount(unsigned WaveCount, unsigned getMaxLocalMemSizeWithWaveCount(unsigned WaveCount,
const Function &) const; const Function &) const;
/// Alias for hasFP64FP16Denormals
bool hasFP16Denormals(const Function &F) const {
return FP64FP16Denormals;
}
/// Alias for hasFP64FP16Denormals
bool hasFP64Denormals(const Function &F) const {
return FP64FP16Denormals;
}
bool hasFP64FP16Denormals(const Function &F) const {
return FP64FP16Denormals;
}
bool supportsMinMaxDenormModes() const { bool supportsMinMaxDenormModes() const {
return getGeneration() >= AMDGPUSubtarget::GFX9; return getGeneration() >= AMDGPUSubtarget::GFX9;
} }
/// \returns If target supports S_DENORM_MODE. /// \returns If target supports S_DENORM_MODE.
bool hasDenormModeInst() const { bool hasDenormModeInst() const {
return getGeneration() >= AMDGPUSubtarget::GFX10; return getGeneration() >= AMDGPUSubtarget::GFX10;
} }
▲ Show 20 Lines • Show All 715 Lines • Show Last 20 Lines

llvm/lib/Target/AMDGPU/AMDGPUSubtarget.cpp

Show First 20 Lines • Show All 48 Lines • ▼ Show 20 Lines
R600Subtarget & R600Subtarget &
R600Subtarget::initializeSubtargetDependencies(const Triple &TT, R600Subtarget::initializeSubtargetDependencies(const Triple &TT,
StringRef GPU, StringRef FS) { StringRef GPU, StringRef FS) {
SmallString<256> FullFS("+promote-alloca,"); SmallString<256> FullFS("+promote-alloca,");
FullFS += FS; FullFS += FS;
ParseSubtargetFeatures(GPU, FullFS); ParseSubtargetFeatures(GPU, FullFS);
// FIXME: I don't think think Evergreen has any useful support for
// denormals, but should be checked. Should we issue a warning somewhere
// if someone tries to enable these?
if (getGeneration() <= AMDGPUSubtarget::NORTHERN_ISLANDS) {
FP32Denormals = false;
}
HasMulU24 = getGeneration() >= EVERGREEN; HasMulU24 = getGeneration() >= EVERGREEN;
HasMulI24 = hasCaymanISA(); HasMulI24 = hasCaymanISA();
return *this; return *this;
} }
GCNSubtarget & GCNSubtarget &
GCNSubtarget::initializeSubtargetDependencies(const Triple &TT, GCNSubtarget::initializeSubtargetDependencies(const Triple &TT,
StringRef GPU, StringRef FS) { StringRef GPU, StringRef FS) {
// Determine default and user-specified characteristics // Determine default and user-specified characteristics
// On SI+, we want FP64 denormals to be on by default. FP32 denormals can be
// enabled, but some instructions do not respect them and they run at the
// double precision rate, so don't enable by default.
// //
// We want to be able to turn these off, but making this a subtarget feature // We want to be able to turn these off, but making this a subtarget feature
// for SI has the unhelpful behavior that it unsets everything else if you // for SI has the unhelpful behavior that it unsets everything else if you
// disable it. // disable it.
// //
// Similarly we want enable-prt-strict-null to be on by default and not to // Similarly we want enable-prt-strict-null to be on by default and not to
// unset everything else if it is disabled // unset everything else if it is disabled
// Assuming ECC is enabled is the conservative default. // Assuming ECC is enabled is the conservative default.
SmallString<256> FullFS("+promote-alloca,+load-store-opt,+sram-ecc,+xnack,"); SmallString<256> FullFS("+promote-alloca,+load-store-opt,+sram-ecc,+xnack,");
if (isAmdHsaOS()) // Turn on FlatForGlobal for HSA. if (isAmdHsaOS()) // Turn on FlatForGlobal for HSA.
FullFS += "+flat-for-global,+unaligned-buffer-access,+trap-handler,"; FullFS += "+flat-for-global,+unaligned-buffer-access,+trap-handler,";
// FIXME: I don't think think Evergreen has any useful support for
// denormals, but should be checked. Should we issue a warning somewhere
// if someone tries to enable these?
if (getGeneration() >= AMDGPUSubtarget::SOUTHERN_ISLANDS) {
FullFS += "+fp64-fp16-denormals,+fp32-denormals,";
} else {
FullFS += "-fp32-denormals,";
}
FullFS += "+enable-prt-strict-null,"; // This is overridden by a disable in FS FullFS += "+enable-prt-strict-null,"; // This is overridden by a disable in FS
// Disable mutually exclusive bits. // Disable mutually exclusive bits.
if (FS.find_lower("+wavefrontsize") != StringRef::npos) { if (FS.find_lower("+wavefrontsize") != StringRef::npos) {
if (FS.find_lower("wavefrontsize16") == StringRef::npos) if (FS.find_lower("wavefrontsize16") == StringRef::npos)
FullFS += "-wavefrontsize16,"; FullFS += "-wavefrontsize16,";
if (FS.find_lower("wavefrontsize32") == StringRef::npos) if (FS.find_lower("wavefrontsize32") == StringRef::npos)
FullFS += "-wavefrontsize32,"; FullFS += "-wavefrontsize32,";
▲ Show 20 Lines • Show All 52 Lines • ▼ Show 20 LinesGCNSubtarget::initializeSubtargetDependencies(const Triple &TT,
return *this; return *this;
} }
AMDGPUSubtarget::AMDGPUSubtarget(const Triple &TT) : AMDGPUSubtarget::AMDGPUSubtarget(const Triple &TT) :
TargetTriple(TT), TargetTriple(TT),
Has16BitInsts(false), Has16BitInsts(false),
HasMadMixInsts(false), HasMadMixInsts(false),
FP32Denormals(false),
FPExceptions(false), FPExceptions(false),
HasSDWA(false), HasSDWA(false),
HasVOP3PInsts(false), HasVOP3PInsts(false),
HasMulI24(true), HasMulI24(true),
HasMulU24(true), HasMulU24(true),
HasInv2PiInlineImm(false), HasInv2PiInlineImm(false),
HasFminFmaxLegacy(true), HasFminFmaxLegacy(true),
EnablePromoteAlloca(false), EnablePromoteAlloca(false),
Show All 11 LinesGCNSubtarget::GCNSubtarget(const Triple &TT, StringRef GPU, StringRef FS,
Gen(TT.getOS() == Triple::AMDHSA ? SEA_ISLANDS : SOUTHERN_ISLANDS), Gen(TT.getOS() == Triple::AMDHSA ? SEA_ISLANDS : SOUTHERN_ISLANDS),
InstrItins(getInstrItineraryForCPU(GPU)), InstrItins(getInstrItineraryForCPU(GPU)),
LDSBankCount(0), LDSBankCount(0),
MaxPrivateElementSize(0), MaxPrivateElementSize(0),
FastFMAF32(false), FastFMAF32(false),
HalfRate64Ops(false), HalfRate64Ops(false),
FP64FP16Denormals(false),
FlatForGlobal(false), FlatForGlobal(false),
AutoWaitcntBeforeBarrier(false), AutoWaitcntBeforeBarrier(false),
CodeObjectV3(false), CodeObjectV3(false),
UnalignedScratchAccess(false), UnalignedScratchAccess(false),
UnalignedBufferAccess(false), UnalignedBufferAccess(false),
HasApertureRegs(false), HasApertureRegs(false),
EnableXNACK(false), EnableXNACK(false),
▲ Show 20 Lines • Show All 696 Lines • Show Last 20 Lines

llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.h

Show First 20 Lines • Show All 127 Lines • ▼ Show 20 Lines
public: public:
explicit GCNTTIImpl(const AMDGPUTargetMachine *TM, const Function &F) explicit GCNTTIImpl(const AMDGPUTargetMachine *TM, const Function &F)
: BaseT(TM, F.getParent()->getDataLayout()), : BaseT(TM, F.getParent()->getDataLayout()),
ST(static_cast<const GCNSubtarget*>(TM->getSubtargetImpl(F))), ST(static_cast<const GCNSubtarget*>(TM->getSubtargetImpl(F))),
TLI(ST->getTargetLowering()), TLI(ST->getTargetLowering()),
CommonTTI(TM, F), CommonTTI(TM, F),
IsGraphicsShader(AMDGPU::isShader(F.getCallingConv())), IsGraphicsShader(AMDGPU::isShader(F.getCallingConv())),
HasFP32Denormals(ST->hasFP32Denormals(F)) { } HasFP32Denormals(AMDGPU::SIModeRegisterDefaults(F).allFP32Denormals()) {}
bool hasBranchDivergence() { return true; } bool hasBranchDivergence() { return true; }
void getUnrollingPreferences(Loop *L, ScalarEvolution &SE, void getUnrollingPreferences(Loop *L, ScalarEvolution &SE,
TTI::UnrollingPreferences &UP); TTI::UnrollingPreferences &UP);
TTI::PopcntSupportKind getPopcntSupport(unsigned TyWidth) { TTI::PopcntSupportKind getPopcntSupport(unsigned TyWidth) {
assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2"); assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2");
▲ Show 20 Lines • Show All 121 Lines • Show Last 20 Lines

llvm/lib/Target/AMDGPU/AMDGPUTargetTransformInfo.cpp

Show First 20 Lines • Show All 684 Lines • ▼ Show 20 Linesbool GCNTTIImpl::areInlineCompatible(const Function *Caller,
FeatureBitset RealCallerBits = CallerBits & ~InlineFeatureIgnoreList; FeatureBitset RealCallerBits = CallerBits & ~InlineFeatureIgnoreList;
FeatureBitset RealCalleeBits = CalleeBits & ~InlineFeatureIgnoreList; FeatureBitset RealCalleeBits = CalleeBits & ~InlineFeatureIgnoreList;
if ((RealCallerBits & RealCalleeBits) != RealCalleeBits) if ((RealCallerBits & RealCalleeBits) != RealCalleeBits)
return false; return false;
// FIXME: dx10_clamp can just take the caller setting, but there seems to be // FIXME: dx10_clamp can just take the caller setting, but there seems to be
// no way to support merge for backend defined attributes. // no way to support merge for backend defined attributes.
AMDGPU::SIModeRegisterDefaults CallerMode(*Caller, *CallerST); AMDGPU::SIModeRegisterDefaults CallerMode(*Caller);
AMDGPU::SIModeRegisterDefaults CalleeMode(*Callee, *CalleeST); AMDGPU::SIModeRegisterDefaults CalleeMode(*Callee);
return CallerMode.isInlineCompatible(CalleeMode); return CallerMode.isInlineCompatible(CalleeMode);
} }
void GCNTTIImpl::getUnrollingPreferences(Loop *L, ScalarEvolution &SE, void GCNTTIImpl::getUnrollingPreferences(Loop *L, ScalarEvolution &SE,
TTI::UnrollingPreferences &UP) { TTI::UnrollingPreferences &UP) {
CommonTTI.getUnrollingPreferences(L, SE, UP); CommonTTI.getUnrollingPreferences(L, SE, UP);
} }
▲ Show 20 Lines • Show All 215 Lines • Show Last 20 Lines

llvm/lib/Target/AMDGPU/R600Instructions.td

Show First 20 Lines • Show All 985 Lines • ▼ Show 20 Lines
class MULADD_Common <bits<5> inst> : R600_3OP < class MULADD_Common <bits<5> inst> : R600_3OP <
inst, "MULADD", inst, "MULADD",
[] []
>; >;
class MULADD_IEEE_Common <bits<5> inst> : R600_3OP < class MULADD_IEEE_Common <bits<5> inst> : R600_3OP <
inst, "MULADD_IEEE", inst, "MULADD_IEEE",
[(set f32:$dst, (fmad f32:$src0, f32:$src1, f32:$src2))] [(set f32:$dst, (any_fmad f32:$src0, f32:$src1, f32:$src2))]
>; >;
class FMA_Common <bits<5> inst> : R600_3OP < class FMA_Common <bits<5> inst> : R600_3OP <
inst, "FMA", inst, "FMA",
[(set f32:$dst, (fma f32:$src0, f32:$src1, f32:$src2))], VecALU [(set f32:$dst, (fma f32:$src0, f32:$src1, f32:$src2))], VecALU
> >
{ {
let OtherPredicates = [FMA]; let OtherPredicates = [FMA];
▲ Show 20 Lines • Show All 780 Lines • Show Last 20 Lines

llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.h

Show First 20 Lines • Show All 687 Lines • ▼ Show 20 Linesstruct SIModeRegisterDefaults {
SIModeRegisterDefaults() : SIModeRegisterDefaults() :
IEEE(true), IEEE(true),
DX10Clamp(true), DX10Clamp(true),
FP32InputDenormals(true), FP32InputDenormals(true),
FP32OutputDenormals(true), FP32OutputDenormals(true),
FP64FP16InputDenormals(true), FP64FP16InputDenormals(true),
FP64FP16OutputDenormals(true) {} FP64FP16OutputDenormals(true) {}
// FIXME: Should not depend on the subtarget SIModeRegisterDefaults(const Function &F);
SIModeRegisterDefaults(const Function &F, const GCNSubtarget &ST);
static SIModeRegisterDefaults getDefaultForCallingConv(CallingConv::ID CC) { static SIModeRegisterDefaults getDefaultForCallingConv(CallingConv::ID CC) {
const bool IsCompute = AMDGPU::isCompute(CC); const bool IsCompute = AMDGPU::isCompute(CC);
SIModeRegisterDefaults Mode; SIModeRegisterDefaults Mode;
Mode.DX10Clamp = true; Mode.DX10Clamp = true;
Mode.IEEE = IsCompute; Mode.IEEE = IsCompute;
Mode.FP32InputDenormals = true; Mode.FP32InputDenormals = IsCompute;
Mode.FP32OutputDenormals = true; Mode.FP32OutputDenormals = IsCompute;
Mode.FP64FP16InputDenormals = true; Mode.FP64FP16InputDenormals = true;
Mode.FP64FP16OutputDenormals = true; Mode.FP64FP16OutputDenormals = true;
return Mode; return Mode;
} }
bool operator ==(const SIModeRegisterDefaults Other) const { bool operator ==(const SIModeRegisterDefaults Other) const {
return IEEE == Other.IEEE && DX10Clamp == Other.DX10Clamp && return IEEE == Other.IEEE && DX10Clamp == Other.DX10Clamp &&
FP32InputDenormals == Other.FP32InputDenormals && FP32InputDenormals == Other.FP32InputDenormals &&
▲ Show 20 Lines • Show All 63 Lines • Show Last 20 Lines

llvm/lib/Target/AMDGPU/Utils/AMDGPUBaseInfo.cpp

Show First 20 Lines • Show All 1,297 Lines • ▼ Show 20 Lines if (Overflow > 0 &&
Subtarget->getGeneration() <= AMDGPUSubtarget::SEA_ISLANDS) Subtarget->getGeneration() <= AMDGPUSubtarget::SEA_ISLANDS)
return false; return false;
ImmOffset = Imm; ImmOffset = Imm;
SOffset = Overflow; SOffset = Overflow;
return true; return true;
} }
SIModeRegisterDefaults::SIModeRegisterDefaults(const Function &F, SIModeRegisterDefaults::SIModeRegisterDefaults(const Function &F) {
const GCNSubtarget &ST) {
*this = getDefaultForCallingConv(F.getCallingConv()); *this = getDefaultForCallingConv(F.getCallingConv());
StringRef IEEEAttr = F.getFnAttribute("amdgpu-ieee").getValueAsString(); StringRef IEEEAttr = F.getFnAttribute("amdgpu-ieee").getValueAsString();
if (!IEEEAttr.empty()) if (!IEEEAttr.empty())
IEEE = IEEEAttr == "true"; IEEE = IEEEAttr == "true";
StringRef DX10ClampAttr StringRef DX10ClampAttr
= F.getFnAttribute("amdgpu-dx10-clamp").getValueAsString(); = F.getFnAttribute("amdgpu-dx10-clamp").getValueAsString();
if (!DX10ClampAttr.empty()) if (!DX10ClampAttr.empty())
DX10Clamp = DX10ClampAttr == "true"; DX10Clamp = DX10ClampAttr == "true";
// FIXME: Split this when denormal-fp-math is used StringRef DenormF32Attr = F.getFnAttribute("denormal-fp-math-f32").getValueAsString();
FP32InputDenormals = ST.hasFP32Denormals(F); if (!DenormF32Attr.empty()) {
FP32OutputDenormals = FP32InputDenormals; DenormalMode DenormMode = parseDenormalFPAttribute(DenormF32Attr);
FP64FP16InputDenormals = ST.hasFP64FP16Denormals(F); FP32InputDenormals = DenormMode.Input == DenormalMode::IEEE;
FP64FP16OutputDenormals = FP64FP16InputDenormals; FP32OutputDenormals = DenormMode.Output == DenormalMode::IEEE;
}
StringRef DenormAttr = F.getFnAttribute("denormal-fp-math").getValueAsString();
if (!DenormAttr.empty()) {
DenormalMode DenormMode = parseDenormalFPAttribute(DenormAttr);
if (DenormF32Attr.empty()) {
FP32InputDenormals = DenormMode.Input == DenormalMode::IEEE;
FP32OutputDenormals = DenormMode.Output == DenormalMode::IEEE;
}
FP64FP16InputDenormals = DenormMode.Input == DenormalMode::IEEE;
FP64FP16OutputDenormals = DenormMode.Output == DenormalMode::IEEE;
}
} }
namespace { namespace {
struct SourceOfDivergence { struct SourceOfDivergence {
unsigned Intr; unsigned Intr;
}; };
const SourceOfDivergence *lookupSourceOfDivergence(unsigned Intr); const SourceOfDivergence *lookupSourceOfDivergence(unsigned Intr);
Show All 30 Lines

llvm/test/Analysis/CostModel/AMDGPU/fdiv.ll

Show First 20 Lines • Show All 272 Lines • ▼ Show 20 Lines
; FP16: estimated cost of 6 for {{.*}} fdiv <2 x half> ; FP16: estimated cost of 6 for {{.*}} fdiv <2 x half>
define amdgpu_kernel void @rcp_v2f16_f32_ftzdaz(<2 x half> addrspace(1)* %out, <2 x half> addrspace(1)* %vaddr) #1 { define amdgpu_kernel void @rcp_v2f16_f32_ftzdaz(<2 x half> addrspace(1)* %out, <2 x half> addrspace(1)* %vaddr) #1 {
%vec = load <2 x half>, <2 x half> addrspace(1)* %vaddr %vec = load <2 x half>, <2 x half> addrspace(1)* %vaddr
%add = fdiv <2 x half> <half 1.0, half 1.0>, %vec %add = fdiv <2 x half> <half 1.0, half 1.0>, %vec
store <2 x half> %add, <2 x half> addrspace(1)* %out store <2 x half> %add, <2 x half> addrspace(1)* %out
ret void ret void
} }
attributes #0 = { nounwind "target-features"="+fp32-denormals" } attributes #0 = { nounwind "denormal-fp-math-f32"="ieee,ieee" }
attributes #1 = { nounwind "target-features"="-fp32-denormals" } attributes #1 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/AMDGPU/amdgcn-ieee.ll

Show First 20 Lines • Show All 144 Lines • ▼ Show 20 Lines
define amdgpu_ps void @ps_ieee_mode_default() #0 { define amdgpu_ps void @ps_ieee_mode_default() #0 {
%val0 = load volatile float, float addrspace(1)* undef %val0 = load volatile float, float addrspace(1)* undef
%val1 = load volatile float, float addrspace(1)* undef %val1 = load volatile float, float addrspace(1)* undef
%min = call float @llvm.minnum.f32(float %val0, float %val1) %min = call float @llvm.minnum.f32(float %val0, float %val1)
store volatile float %min, float addrspace(1)* undef store volatile float %min, float addrspace(1)* undef
ret void ret void
} }
; FIXME: Should have denormals off by default.
; GCN-LABEL: {{^}}ps_ieee_mode_on: ; GCN-LABEL: {{^}}ps_ieee_mode_on:
; GCN: {{buffer|global|flat}}_load_dword [[VAL0:v[0-9]+]] ; GCN: {{buffer|global|flat}}_load_dword [[VAL0:v[0-9]+]]
; GCN-NEXT: {{buffer|global|flat}}_load_dword [[VAL1:v[0-9]+]] ; GCN-NEXT: {{buffer|global|flat}}_load_dword [[VAL1:v[0-9]+]]
; GCN-DAG: v_max_f32_e32 [[QUIET0:v[0-9]+]], [[VAL0]], [[VAL0]] ; GCN-DAG: v_mul_f32_e32 [[QUIET0:v[0-9]+]], 1.0, [[VAL0]]
; GCN-DAG: v_max_f32_e32 [[QUIET1:v[0-9]+]], [[VAL1]], [[VAL1]] ; GCN-DAG: v_mul_f32_e32 [[QUIET1:v[0-9]+]], 1.0, [[VAL1]]
; GCN: v_min_f32_e32 [[MIN:v[0-9]+]], [[QUIET0]], [[QUIET1]] ; GCN: v_min_f32_e32 [[MIN:v[0-9]+]], [[QUIET0]], [[QUIET1]]
; GCN-NOT: v_mul_f32 ; GCN-NOT: v_mul_f32
define amdgpu_ps void @ps_ieee_mode_on() #1 { define amdgpu_ps void @ps_ieee_mode_on() #1 {
%val0 = load volatile float, float addrspace(1)* undef %val0 = load volatile float, float addrspace(1)* undef
%val1 = load volatile float, float addrspace(1)* undef %val1 = load volatile float, float addrspace(1)* undef
%min = call float @llvm.minnum.f32(float %val0, float %val1) %min = call float @llvm.minnum.f32(float %val0, float %val1)
store volatile float %min, float addrspace(1)* undef store volatile float %min, float addrspace(1)* undef
ret void ret void
Show All 23 Lines

llvm/test/CodeGen/AMDGPU/amdgpu-codegenprepare-fdiv.ll

Show First 20 Lines • Show All 208 Lines • ▼ Show 20 Linesdefine amdgpu_kernel void @fdiv_fpmath_f32_denormals(float addrspace(1)* %out, float %a, float %b) #2 {
%arcp.md.25ulp = fdiv arcp float %a, %b, !fpmath !0 %arcp.md.25ulp = fdiv arcp float %a, %b, !fpmath !0
store volatile float %arcp.md.25ulp, float addrspace(1)* %out store volatile float %arcp.md.25ulp, float addrspace(1)* %out
ret void ret void
} }
attributes #0 = { nounwind optnone noinline } attributes #0 = { nounwind optnone noinline }
attributes #1 = { nounwind "target-features"="-fp32-denormals" } attributes #1 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #2 = { nounwind "target-features"="+fp32-denormals" } attributes #2 = { nounwind "denormal-fp-math-f32"="ieee,ieee" }
; CHECK: !0 = !{float 2.500000e+00} ; CHECK: !0 = !{float 2.500000e+00}
; CHECK: !1 = !{float 5.000000e-01} ; CHECK: !1 = !{float 5.000000e-01}
; CHECK: !2 = !{float 1.000000e+00} ; CHECK: !2 = !{float 1.000000e+00}
; CHECK: !3 = !{float 3.000000e+00} ; CHECK: !3 = !{float 3.000000e+00}
!0 = !{float 2.500000e+00} !0 = !{float 2.500000e+00}
!1 = !{float 5.000000e-01} !1 = !{float 5.000000e-01}
!2 = !{float 1.000000e+00} !2 = !{float 1.000000e+00}
!3 = !{float 3.000000e+00} !3 = !{float 3.000000e+00}

llvm/test/CodeGen/AMDGPU/clamp-modifier.ll

Show First 20 Lines • Show All 383 Lines • ▼ Show 20 Lines
declare <2 x half> @llvm.maxnum.v2f16(<2 x half>, <2 x half>) #1 declare <2 x half> @llvm.maxnum.v2f16(<2 x half>, <2 x half>) #1
declare <2 x float> @llvm.minnum.v2f32(<2 x float>, <2 x float>) #1 declare <2 x float> @llvm.minnum.v2f32(<2 x float>, <2 x float>) #1
declare <2 x float> @llvm.maxnum.v2f32(<2 x float>, <2 x float>) #1 declare <2 x float> @llvm.maxnum.v2f32(<2 x float>, <2 x float>) #1
declare <2 x half> @llvm.amdgcn.cvt.pkrtz(float, float) #1 declare <2 x half> @llvm.amdgcn.cvt.pkrtz(float, float) #1
declare void @llvm.dbg.value(metadata, i64, metadata, metadata) #1 declare void @llvm.dbg.value(metadata, i64, metadata, metadata) #1
attributes #0 = { nounwind "target-features"="-fp32-denormals" } attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind readnone } attributes #1 = { nounwind readnone }
attributes #2 = { nounwind "target-features"="+fp32-denormals" } attributes #2 = { nounwind "denormal-fp-math-f32"="ieee.ieee" }
attributes #3 = { nounwind "target-features"="-fp64-fp16-denormals" } attributes #3 = { nounwind "denormal-fp-math-f32"="ieee,ieee" "denormal-fp-math"="preserve-sign,preserve-sign" }
!llvm.dbg.cu = !{!0} !llvm.dbg.cu = !{!0}
!llvm.module.flags = !{!2, !3} !llvm.module.flags = !{!2, !3}
!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, isOptimized: true, runtimeVersion: 0, emissionKind: NoDebug) !0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, isOptimized: true, runtimeVersion: 0, emissionKind: NoDebug)
!1 = !DIFile(filename: "/tmp/foo.cl", directory: "/dev/null") !1 = !DIFile(filename: "/tmp/foo.cl", directory: "/dev/null")
!2 = !{i32 2, !"Dwarf Version", i32 4} !2 = !{i32 2, !"Dwarf Version", i32 4}
!3 = !{i32 2, !"Debug Info Version", i32 3} !3 = !{i32 2, !"Debug Info Version", i32 3}
!4 = !DILocalVariable(name: "add", arg: 1, scope: !5, file: !1, line: 1) !4 = !DILocalVariable(name: "add", arg: 1, scope: !5, file: !1, line: 1)
!5 = distinct !DISubprogram(name: "foo", scope: !1, file: !1, line: 1, type: !6, isLocal: false, isDefinition: true, scopeLine: 2, flags: DIFlagPrototyped, isOptimized: true, unit: !0) !5 = distinct !DISubprogram(name: "foo", scope: !1, file: !1, line: 1, type: !6, isLocal: false, isDefinition: true, scopeLine: 2, flags: DIFlagPrototyped, isOptimized: true, unit: !0)
!6 = !DISubroutineType(types: !7) !6 = !DISubroutineType(types: !7)
!7 = !{null, !8} !7 = !{null, !8}
!8 = !DIBasicType(name: "float", size: 32, align: 32) !8 = !DIBasicType(name: "float", size: 32, align: 32)
!9 = !DIExpression() !9 = !DIExpression()
!10 = !DILocation(line: 1, column: 42, scope: !5) !10 = !DILocation(line: 1, column: 42, scope: !5)

llvm/test/CodeGen/AMDGPU/clamp.ll

Show First 20 Lines • Show All 761 Lines • ▼ Show 20 Lines
declare double @llvm.maxnum.f64(double, double) #1 declare double @llvm.maxnum.f64(double, double) #1
declare half @llvm.fabs.f16(half) #1 declare half @llvm.fabs.f16(half) #1
declare half @llvm.minnum.f16(half, half) #1 declare half @llvm.minnum.f16(half, half) #1
declare half @llvm.maxnum.f16(half, half) #1 declare half @llvm.maxnum.f16(half, half) #1
declare <2 x half> @llvm.fabs.v2f16(<2 x half>) #1 declare <2 x half> @llvm.fabs.v2f16(<2 x half>) #1
declare <2 x half> @llvm.minnum.v2f16(<2 x half>, <2 x half>) #1 declare <2 x half> @llvm.minnum.v2f16(<2 x half>, <2 x half>) #1
declare <2 x half> @llvm.maxnum.v2f16(<2 x half>, <2 x half>) #1 declare <2 x half> @llvm.maxnum.v2f16(<2 x half>, <2 x half>) #1
attributes #0 = { nounwind "target-features"="-fp32-denormals" } attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind readnone } attributes #1 = { nounwind readnone }
attributes #2 = { nounwind "amdgpu-dx10-clamp"="false" "target-features"="-fp32-denormals,-fp-exceptions" "no-nans-fp-math"="false" } attributes #2 = { nounwind "amdgpu-dx10-clamp"="false" "target-features"="-fp-exceptions" "denormal-fp-math-f32"="preserve-sign,preserve-sign" "no-nans-fp-math"="false" }
attributes #3 = { nounwind "amdgpu-dx10-clamp"="true" "target-features"="-fp32-denormals,+fp-exceptions" "no-nans-fp-math"="false" } attributes #3 = { nounwind "amdgpu-dx10-clamp"="true" "target-features"="+fp-exceptions" "denormal-fp-math-f32"="preserve-sign,preserve-sign" "no-nans-fp-math"="false" }
attributes #4 = { nounwind "amdgpu-dx10-clamp"="false" "target-features"="-fp32-denormals,+fp-exceptions" "no-nans-fp-math"="false" } attributes #4 = { nounwind "amdgpu-dx10-clamp"="false" "target-features"="+fp-exceptions" "denormal-fp-math-f32"="preserve-sign,preserve-sign" "no-nans-fp-math"="false" }

llvm/test/CodeGen/AMDGPU/default-fp-mode.ll

Show First 20 Lines • Show All 76 Lines • ▼ Show 20 Lines
; GCN: IeeeMode: 1 ; GCN: IeeeMode: 1
define amdgpu_kernel void @test_f32_f16_f64_denormals(half addrspace(1)* %out0, float addrspace(1)* %out1, double addrspace(1)* %out2) #8 { define amdgpu_kernel void @test_f32_f16_f64_denormals(half addrspace(1)* %out0, float addrspace(1)* %out1, double addrspace(1)* %out2) #8 {
store half 0.0, half addrspace(1)* %out0 store half 0.0, half addrspace(1)* %out0
store float 0.0, float addrspace(1)* %out1 store float 0.0, float addrspace(1)* %out1
store double 0.0, double addrspace(1)* %out2 store double 0.0, double addrspace(1)* %out2
ret void ret void
} }
; FIXME: Denormals should be off by default ; GCN-LABEL: {{^}}test_just_f32_attr_flush
; GCN: FloatMode: 192
; GCN: IeeeMode: 1
define amdgpu_kernel void @test_just_f32_attr_flush(float addrspace(1)* %out0, double addrspace(1)* %out1) #9 {
store float 0.0, float addrspace(1)* %out0
store double 0.0, double addrspace(1)* %out1
ret void
}
; GCN-LABEL: {{^}}kill_gs_const: ; GCN-LABEL: {{^}}kill_gs_const:
; GCN: FloatMode: 240 ; GCN: FloatMode: 192
; GCN: IeeeMode: 0 ; GCN: IeeeMode: 0
define amdgpu_gs void @kill_gs_const() { define amdgpu_gs void @kill_gs_const() {
main_body: main_body:
%cmp0 = icmp ule i32 0, 3 %cmp0 = icmp ule i32 0, 3
call void @llvm.amdgcn.kill(i1 %cmp0) call void @llvm.amdgcn.kill(i1 %cmp0)
%cmp1 = icmp ule i32 3, 0 %cmp1 = icmp ule i32 3, 0
call void @llvm.amdgcn.kill(i1 %cmp1) call void @llvm.amdgcn.kill(i1 %cmp1)
ret void ret void
} }
; GCN-LABEL: {{^}}kill_vcc_implicit_def: ; GCN-LABEL: {{^}}kill_vcc_implicit_def:
; GCN: FloatMode: 240 ; GCN: FloatMode: 192
; GCN: IeeeMode: 0 ; GCN: IeeeMode: 0
define amdgpu_ps float @kill_vcc_implicit_def([6 x <16 x i8>] addrspace(4)* inreg, [17 x <16 x i8>] addrspace(4)* inreg, [17 x <4 x i32>] addrspace(4)* inreg, [34 x <8 x i32>] addrspace(4)* inreg, float inreg, i32 inreg, <2 x i32>, <2 x i32>, <2 x i32>, <3 x i32>, <2 x i32>, <2 x i32>, <2 x i32>, float, float, float, float, float, float, i32, float, float) { define amdgpu_ps float @kill_vcc_implicit_def([6 x <16 x i8>] addrspace(4)* inreg, [17 x <16 x i8>] addrspace(4)* inreg, [17 x <4 x i32>] addrspace(4)* inreg, [34 x <8 x i32>] addrspace(4)* inreg, float inreg, i32 inreg, <2 x i32>, <2 x i32>, <2 x i32>, <3 x i32>, <2 x i32>, <2 x i32>, <2 x i32>, float, float, float, float, float, float, i32, float, float) {
entry: entry:
%tmp0 = fcmp olt float %13, 0.0 %tmp0 = fcmp olt float %13, 0.0
call void @llvm.amdgcn.kill(i1 %tmp0) call void @llvm.amdgcn.kill(i1 %tmp0)
%tmp1 = select i1 %tmp0, float 1.0, float 0.0 %tmp1 = select i1 %tmp0, float 1.0, float 0.0
ret float %tmp1 ret float %tmp1
} }
declare void @llvm.amdgcn.kill(i1) declare void @llvm.amdgcn.kill(i1)
attributes #0 = { nounwind "target-cpu"="tahiti" } attributes #0 = { nounwind "target-cpu"="tahiti" }
attributes #1 = { nounwind "target-cpu"="fiji" } attributes #1 = { nounwind "target-cpu"="fiji" }
attributes #2 = { nounwind "target-features"="+fp64-denormals" } attributes #2 = { nounwind "denormal-fp-math"="ieee,ieee" }
attributes #3 = { nounwind "target-features"="+fp32-denormals" } attributes #3 = { nounwind "denormal-fp-math-f32"="ieee,ieee" }
attributes #4 = { nounwind "target-features"="+fp32-denormals,+fp64-denormals" } attributes #4 = { nounwind "denormal-fp-math"="ieee,ieee" }
attributes #5 = { nounwind "target-features"="-fp32-denormals,-fp64-fp16-denormals" } attributes #5 = { nounwind "denormal-fp-math"="preserve-sign,preserve-sign" }
attributes #6 = { nounwind "target-features"="+fp64-fp16-denormals" } attributes #6 = { nounwind "denormal-fp-math"="ieee,ieee" }
attributes #7 = { nounwind "target-features"="-fp64-fp16-denormals" } attributes #7 = { nounwind "denormal-fp-math-f32"="ieee,ieee" "denormal-fp-math"="preserve-sign,preserve-sign" }
attributes #8 = { nounwind "target-features"="+fp32-denormals,+fp64-fp16-denormals" } attributes #8 = { nounwind "denormal-fp-math"="ieee,ieee" }
attributes #9 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/AMDGPU/fadd-fma-fmul-combine.ll

; RUN: llc -march=amdgcn -mattr=+fast-fmaf,-fp32-denormals -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GCN-FLUSH %s ; RUN: llc -march=amdgcn -mattr=+fast-fmaf -denormal-fp-math-f32=preserve-sign -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GCN-FLUSH %s
; RUN: llc -march=amdgcn -mattr=-fast-fmaf,-fp32-denormals -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GCN-FLUSH %s ; RUN: llc -march=amdgcn -mattr=-fast-fmaf -denormal-fp-math-f32=preserve-sign -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GCN-FLUSH %s
; RUN: llc -march=amdgcn -mattr=+fast-fmaf,+fp32-denormals -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GCN-FASTFMA %s ; RUN: llc -march=amdgcn -mattr=+fast-fmaf -denormal-fp-math-f32=ieee -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GCN-FASTFMA %s
; RUN: llc -march=amdgcn -mattr=-fast-fmaf,+fp32-denormals -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GCN-SLOWFMA %s ; RUN: llc -march=amdgcn -mattr=-fast-fmaf -denormal-fp-math-f32=ieee -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GCN-SLOWFMA %s
; FIXME: This should also fold when fma is actually fast if an FMA ; FIXME: This should also fold when fma is actually fast if an FMA
; exists in the original program. ; exists in the original program.
; (fadd (fma x, y, (fmul u, v), z) -> (fma x, y (fma u, v, z)) ; (fadd (fma x, y, (fmul u, v), z) -> (fma x, y (fma u, v, z))
; GCN-LABEL: {{^}}fast_add_fmuladd_fmul: ; GCN-LABEL: {{^}}fast_add_fmuladd_fmul:
; GCN: buffer_load_dword [[X:v[0-9]+]] ; GCN: buffer_load_dword [[X:v[0-9]+]]
▲ Show 20 Lines • Show All 249 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/fcanonicalize-elimination.ll

; RUN: llc -march=amdgcn -mcpu=gfx801 -verify-machineinstrs -mattr=-fp32-denormals < %s | FileCheck -enable-var-scope -check-prefixes=GCN,VI,VI-FLUSH,GCN-FLUSH,GCN-NOEXCEPT %s ; RUN: llc -march=amdgcn -mcpu=gfx801 -verify-machineinstrs -denormal-fp-math-f32=preserve-sign < %s | FileCheck -enable-var-scope -check-prefixes=GCN,VI,VI-FLUSH,GCN-FLUSH,GCN-NOEXCEPT %s
; RUN: llc -march=amdgcn -mcpu=gfx801 -verify-machineinstrs -mattr=-fp32-denormals,+fp-exceptions < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-EXCEPT,VI,VI-FLUSH,GCN-FLUSH %s ; RUN: llc -march=amdgcn -mcpu=gfx801 -verify-machineinstrs -mattr=+fp-exceptions -denormal-fp-math-f32=preserve-sign < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-EXCEPT,VI,VI-FLUSH,GCN-FLUSH %s
; RUN: llc -march=amdgcn -mcpu=gfx801 -verify-machineinstrs -mattr=+fp32-denormals < %s | FileCheck -enable-var-scope -check-prefixes=GCN,VI,VI-DENORM,GCN-DENORM,GCN-NOEXCEPT %s ; RUN: llc -march=amdgcn -mcpu=gfx801 -verify-machineinstrs -denormal-fp-math-f32=ieee < %s | FileCheck -enable-var-scope -check-prefixes=GCN,VI,VI-DENORM,GCN-DENORM,GCN-NOEXCEPT %s
; RUN: llc -march=amdgcn -mcpu=gfx900 -verify-machineinstrs -mattr=+fp32-denormals < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX9,GFX9-DENORM,GCN-DENORM,GCN-NOEXCEPT %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -verify-machineinstrs -denormal-fp-math-f32=ieee < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX9,GFX9-DENORM,GCN-DENORM,GCN-NOEXCEPT %s
; RUN: llc -march=amdgcn -mcpu=gfx900 -verify-machineinstrs -mattr=-fp32-denormals < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX9,GFX9-FLUSH,GCN-FLUSH,GCN-NOEXCEPT %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -verify-machineinstrs -denormal-fp-math-f32=preserve-sign < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX9,GFX9-FLUSH,GCN-FLUSH,GCN-NOEXCEPT %s
; GCN-LABEL: {{^}}test_no_fold_canonicalize_loaded_value_f32: ; GCN-LABEL: {{^}}test_no_fold_canonicalize_loaded_value_f32:
; GCN-FLUSH: v_mul_f32_e32 v{{[0-9]+}}, 1.0, v{{[0-9]+}} ; GCN-FLUSH: v_mul_f32_e32 v{{[0-9]+}}, 1.0, v{{[0-9]+}}
; GFX9-DENORM: v_max_f32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9-DENORM: v_max_f32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
define amdgpu_kernel void @test_no_fold_canonicalize_loaded_value_f32(float addrspace(1)* %arg) { define amdgpu_kernel void @test_no_fold_canonicalize_loaded_value_f32(float addrspace(1)* %arg) {
%id = tail call i32 @llvm.amdgcn.workitem.id.x() %id = tail call i32 @llvm.amdgcn.workitem.id.x()
%gep = getelementptr inbounds float, float addrspace(1)* %arg, i32 %id %gep = getelementptr inbounds float, float addrspace(1)* %arg, i32 %id
%v = load float, float addrspace(1)* %gep, align 4 %v = load float, float addrspace(1)* %gep, align 4
▲ Show 20 Lines • Show All 899 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/fcanonicalize.f16.ll

Show First 20 Lines • Show All 733 Lines • ▼ Show 20 Linesdefine <4 x half> @v_test_canonicalize_reg_undef_reg_reg_v4f16(half %val0, half %val1, half %val2) #1 {
%vec0 = insertelement <4 x half> undef, half %val0, i32 0 %vec0 = insertelement <4 x half> undef, half %val0, i32 0
%vec1 = insertelement <4 x half> %vec0, half %val1, i32 2 %vec1 = insertelement <4 x half> %vec0, half %val1, i32 2
%vec2 = insertelement <4 x half> %vec1, half %val2, i32 3 %vec2 = insertelement <4 x half> %vec1, half %val2, i32 3
%canonicalized = call <4 x half> @llvm.canonicalize.v4f16(<4 x half> %vec2) %canonicalized = call <4 x half> @llvm.canonicalize.v4f16(<4 x half> %vec2)
ret <4 x half> %canonicalized ret <4 x half> %canonicalized
} }
attributes #0 = { nounwind readnone } attributes #0 = { nounwind readnone }
attributes #1 = { nounwind "target-features"="-fp32-denormals" } attributes #1 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #2 = { nounwind "target-features"="-fp32-denormals,-fp64-fp16-denormals" } attributes #2 = { nounwind "denormal-fp-math"="preserve-sign,preserve-sign" }
attributes #3 = { nounwind "target-features"="-fp32-denormals,+fp64-fp16-denormals" } attributes #3 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/AMDGPU/fcanonicalize.ll

Show First 20 Lines • Show All 619 Lines • ▼ Show 20 Lines
; GCN: v_max_f64 ; GCN: v_max_f64
; GCN: v_max_f64 ; GCN: v_max_f64
define <4 x double> @v_test_canonicalize_v4f64(<4 x double> %arg) #1 { define <4 x double> @v_test_canonicalize_v4f64(<4 x double> %arg) #1 {
%canon = call <4 x double> @llvm.canonicalize.v4f64(<4 x double> %arg) %canon = call <4 x double> @llvm.canonicalize.v4f64(<4 x double> %arg)
ret <4 x double> %canon ret <4 x double> %canon
} }
attributes #0 = { nounwind readnone } attributes #0 = { nounwind readnone }
attributes #1 = { nounwind "target-features"="-fp32-denormals" } attributes #1 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #2 = { nounwind "target-features"="-fp32-denormals,-fp64-fp16-denormals" } attributes #2 = { nounwind "denormal-fp-math"="preserve-sign,preserve-sign" }
attributes #3 = { nounwind "target-features"="+fp32-denormals,+fp64-fp16-denormals" } attributes #3 = { nounwind "denormal-fp-math"="ieee,ieee" }
attributes #4 = { nounwind "target-features"="-fp32-denormals,-fp64-fp16-denormals" "target-cpu"="tonga" } attributes #4 = { nounwind "denormal-fp-math"="preserve-sign,preserve-sign" "target-cpu"="tonga" }
attributes #5 = { nounwind "target-features"="+fp32-denormals,+fp64-fp16-denormals" "target-cpu"="gfx900" } attributes #5 = { nounwind "denormal-fp-math"="ieee,ieee" "target-cpu"="gfx900" }
attributes #6 = { nounwind "target-features"="-fp32-denormals,-fp64-fp16-denormals" "target-cpu"="gfx900" } attributes #6 = { nounwind "denormal-fp-math"="preserve-sign,preserve-sign" "target-cpu"="gfx900" }

llvm/test/CodeGen/AMDGPU/fdiv.f16.ll

; RUN: llc -march=amdgcn -verify-machineinstrs -mattr=-fp32-denormals < %s | FileCheck -check-prefix=GCN -check-prefix=SI %s ; RUN: llc -march=amdgcn -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=SI %s
; RUN: llc -march=amdgcn -mcpu=fiji -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GFX8_9_10 %s ; RUN: llc -march=amdgcn -mcpu=fiji -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GFX8_9_10 %s
; RUN: llc -march=amdgcn -mcpu=fiji -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GFX8_9_10 %s ; RUN: llc -march=amdgcn -mcpu=fiji -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GFX8_9_10 %s
; RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GFX8_9_10 %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GFX8_9_10 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GFX8_9_10 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=GFX8_9_10 %s
; Make sure fdiv is promoted to f32. ; Make sure fdiv is promoted to f32.
; GCN-LABEL: {{^}}v_fdiv_f16 ; GCN-LABEL: {{^}}v_fdiv_f16
; SI: v_cvt_f32_f16 ; SI: v_cvt_f32_f16
; SI: v_cvt_f32_f16 ; SI: v_cvt_f32_f16
▲ Show 20 Lines • Show All 175 Lines • ▼ Show 20 Lines
; GCN-LABEL: {{^}}v_fdiv_f16_unsafe: ; GCN-LABEL: {{^}}v_fdiv_f16_unsafe:
; GFX8_9_10: {{flat|global}}_load_ushort [[LHS:v[0-9]+]] ; GFX8_9_10: {{flat|global}}_load_ushort [[LHS:v[0-9]+]]
; GFX8_9_10: {{flat|global}}_load_ushort [[RHS:v[0-9]+]] ; GFX8_9_10: {{flat|global}}_load_ushort [[RHS:v[0-9]+]]
; GFX8_9_10: v_rcp_f16_e32 [[RCP:v[0-9]+]], [[RHS]] ; GFX8_9_10: v_rcp_f16_e32 [[RCP:v[0-9]+]], [[RHS]]
; GFX8_9_10: v_mul_f16_e32 [[RESULT:v[0-9]+]], [[LHS]], [[RCP]] ; GFX8_9_10: v_mul_f16_e32 [[RESULT:v[0-9]+]], [[LHS]], [[RCP]]
; GFX8_9_10: {{flat|global}}_store_short v{{\[[0-9]+:[0-9]+\]}}, [[RESULT]] ; GFX8_9_10: {{flat|global}}_store_short v{{\[[0-9]+:[0-9]+\]}}, [[RESULT]]
define amdgpu_kernel void @v_fdiv_f16_unsafe(half addrspace(1)* %r, half addrspace(1)* %a, half addrspace(1)* %b) #2 { define amdgpu_kernel void @v_fdiv_f16_unsafe(half addrspace(1)* %r, half addrspace(1)* %a, half addrspace(1)* %b) #1 {
entry: entry:
%tid = call i32 @llvm.amdgcn.workitem.id.x() %tid = call i32 @llvm.amdgcn.workitem.id.x()
%tid.ext = sext i32 %tid to i64 %tid.ext = sext i32 %tid to i64
%gep.a = getelementptr inbounds half, half addrspace(1)* %a, i64 %tid.ext %gep.a = getelementptr inbounds half, half addrspace(1)* %a, i64 %tid.ext
%gep.b = getelementptr inbounds half, half addrspace(1)* %b, i64 %tid.ext %gep.b = getelementptr inbounds half, half addrspace(1)* %b, i64 %tid.ext
%gep.r = getelementptr inbounds half, half addrspace(1)* %r, i64 %tid.ext %gep.r = getelementptr inbounds half, half addrspace(1)* %r, i64 %tid.ext
%a.val = load volatile half, half addrspace(1)* %gep.a %a.val = load volatile half, half addrspace(1)* %gep.a
%b.val = load volatile half, half addrspace(1)* %gep.b %b.val = load volatile half, half addrspace(1)* %gep.b
Show All 33 Lines
; GFX8_9_10: buffer_store_short [[MUL]] ; GFX8_9_10: buffer_store_short [[MUL]]
define amdgpu_kernel void @div_arcp_neg_k_x_pat_f16(half addrspace(1)* %out) #0 { define amdgpu_kernel void @div_arcp_neg_k_x_pat_f16(half addrspace(1)* %out) #0 {
%x = load half, half addrspace(1)* undef %x = load half, half addrspace(1)* undef
%rcp = fdiv arcp half %x, -10.0 %rcp = fdiv arcp half %x, -10.0
store half %rcp, half addrspace(1)* %out, align 4 store half %rcp, half addrspace(1)* %out, align 4
ret void ret void
} }
declare i32 @llvm.amdgcn.workitem.id.x() #1 declare i32 @llvm.amdgcn.workitem.id.x() #2
declare half @llvm.sqrt.f16(half) #1 declare half @llvm.sqrt.f16(half) #2
declare half @llvm.fabs.f16(half) #1 declare half @llvm.fabs.f16(half) #2
attributes #0 = { nounwind } attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind readnone } attributes #1 = { nounwind "unsafe-fp-math"="true" "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #2 = { nounwind "unsafe-fp-math"="true" } attributes #2 = { nounwind readnone speculatable }

llvm/test/CodeGen/AMDGPU/fdiv.ll

Show First 20 Lines • Show All 278 Lines • ▼ Show 20 Linesdefine amdgpu_kernel void @fdiv_v4f32_arcp_math(<4 x float> addrspace(1)* %out, <4 x float> addrspace(1)* %in) #0 {
%b_ptr = getelementptr <4 x float>, <4 x float> addrspace(1)* %in, i32 1 %b_ptr = getelementptr <4 x float>, <4 x float> addrspace(1)* %in, i32 1
%a = load <4 x float>, <4 x float> addrspace(1) * %in %a = load <4 x float>, <4 x float> addrspace(1) * %in
%b = load <4 x float>, <4 x float> addrspace(1) * %b_ptr %b = load <4 x float>, <4 x float> addrspace(1) * %b_ptr
%result = fdiv arcp <4 x float> %a, %b %result = fdiv arcp <4 x float> %a, %b
store <4 x float> %result, <4 x float> addrspace(1)* %out store <4 x float> %result, <4 x float> addrspace(1)* %out
ret void ret void
} }
attributes #0 = { nounwind "enable-unsafe-fp-math"="false" "target-features"="-fp32-denormals,+fp64-fp16-denormals,-flat-for-global" } attributes #0 = { nounwind "enable-unsafe-fp-math"="false" "denormal-fp-math-f32"="preserve-sign,preserve-sign" "target-features"="-flat-for-global" }
attributes #1 = { nounwind "enable-unsafe-fp-math"="true" "target-features"="-fp32-denormals,-flat-for-global" } attributes #1 = { nounwind "enable-unsafe-fp-math"="true" "denormal-fp-math-f32"="preserve-sign,preserve-sign" "target-features"="-flat-for-global" }
attributes #2 = { nounwind "enable-unsafe-fp-math"="false" "target-features"="+fp32-denormals,-flat-for-global" } attributes #2 = { nounwind "enable-unsafe-fp-math"="false" "denormal-fp-math-f32"="ieee,ieee" "target-features"="-flat-for-global" }
!0 = !{float 2.500000e+00} !0 = !{float 2.500000e+00}

llvm/test/CodeGen/AMDGPU/fdiv32-to-rcp-folding.ll

; RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=+fp32-denormals < %s | FileCheck --check-prefixes=GCN,GCN-DENORM %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -denormal-fp-math-f32=ieee < %s | FileCheck --check-prefixes=GCN,GCN-DENORM %s
; RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=-fp32-denormals < %s | FileCheck --check-prefixes=GCN,GCN-FLUSH %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -denormal-fp-math-f32=preserve-sign < %s | FileCheck --check-prefixes=GCN,GCN-FLUSH %s
; GCN-LABEL: {{^}}div_1_by_x_25ulp: ; GCN-LABEL: {{^}}div_1_by_x_25ulp:
; GCN-DENORM-DAG: v_mov_b32_e32 [[L:v[0-9]+]], 0x6f800000 ; GCN-DENORM-DAG: v_mov_b32_e32 [[L:v[0-9]+]], 0x6f800000
; GCN-DENORM-DAG: v_mov_b32_e32 [[S:v[0-9]+]], 0x2f800000 ; GCN-DENORM-DAG: v_mov_b32_e32 [[S:v[0-9]+]], 0x2f800000
; GCN-DAG: s_load_dword [[VAL:s[0-9]+]], s[{{[0-9:]+}}], 0x0{{$}} ; GCN-DAG: s_load_dword [[VAL:s[0-9]+]], s[{{[0-9:]+}}], 0x0{{$}}
; GCN-DENORM-DAG: v_cmp_gt_f32_e64 vcc, |[[VAL]]|, [[L]] ; GCN-DENORM-DAG: v_cmp_gt_f32_e64 vcc, |[[VAL]]|, [[L]]
; GCN-DENORM-DAG: v_cndmask_b32_e32 [[SCALE:v[0-9]+]], 1.0, [[S]], vcc ; GCN-DENORM-DAG: v_cndmask_b32_e32 [[SCALE:v[0-9]+]], 1.0, [[S]], vcc
; GCN-DENORM: v_mul_f32_e32 [[PRESCALED:v[0-9]+]], [[VAL]], [[SCALE]] ; GCN-DENORM: v_mul_f32_e32 [[PRESCALED:v[0-9]+]], [[VAL]], [[SCALE]]
▲ Show 20 Lines • Show All 451 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/fdot2.ll

; RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=-fp32-denormals -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GFX900 ; RUN: llc -march=amdgcn -mcpu=gfx900 -denormal-fp-math-f32=preserve-sign -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GFX900
; RUN: llc -march=amdgcn -mcpu=gfx906 -mattr=-fp32-denormals -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GCN-DL-UNSAFE,GFX906-DL-UNSAFE ; RUN: llc -march=amdgcn -mcpu=gfx906 -denormal-fp-math-f32=preserve-sign -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GCN-DL-UNSAFE,GFX906-DL-UNSAFE
; RUN: llc -march=amdgcn -mcpu=gfx1011 -mattr=-fp32-denormals -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GCN-DL-UNSAFE,GFX10-DL-UNSAFE,GFX10-CONTRACT ; RUN: llc -march=amdgcn -mcpu=gfx1011 -denormal-fp-math-f32=preserve-sign -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GCN-DL-UNSAFE,GFX10-DL-UNSAFE,GFX10-CONTRACT
; RUN: llc -march=amdgcn -mcpu=gfx1012 -mattr=-fp32-denormals -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GCN-DL-UNSAFE,GFX10-DL-UNSAFE,GFX10-CONTRACT ; RUN: llc -march=amdgcn -mcpu=gfx1012 -denormal-fp-math-f32=preserve-sign -enable-unsafe-fp-math -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GCN-DL-UNSAFE,GFX10-DL-UNSAFE,GFX10-CONTRACT
; RUN: llc -march=amdgcn -mcpu=gfx906 -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GFX906 ; RUN: llc -march=amdgcn -mcpu=gfx906 -denormal-fp-math-f32=preserve-sign -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GFX906
; RUN: llc -march=amdgcn -mcpu=gfx906 -mattr=-fp64-fp16-denormals,-fp32-denormals -fp-contract=fast -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GFX906-CONTRACT ; RUN: llc -march=amdgcn -mcpu=gfx906 -denormal-fp-math=preserve-sign -fp-contract=fast -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GFX906-CONTRACT
; RUN: llc -march=amdgcn -mcpu=gfx906 -mattr=+fp64-fp16-denormals,+fp32-denormals -fp-contract=fast -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GFX906-DENORM-CONTRACT ; RUN: llc -march=amdgcn -mcpu=gfx906 -denormal-fp-math=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck %s -check-prefixes=GCN,GFX906-DENORM-CONTRACT
; (fadd (fmul S1.x, S2.x), (fadd (fmul (S1.y, S2.y), z))) -> (fdot2 S1, S2, z) ; (fadd (fmul S1.x, S2.x), (fadd (fmul (S1.y, S2.y), z))) -> (fdot2 S1, S2, z)
; Tests to make sure fdot2 is not generated when vector elements of dot-product expressions ; Tests to make sure fdot2 is not generated when vector elements of dot-product expressions
; are not converted from f16 to f32. ; are not converted from f16 to f32.
; GCN-LABEL: {{^}}dotproduct_f16 ; GCN-LABEL: {{^}}dotproduct_f16
; GFX900: v_fma_f16 ; GFX900: v_fma_f16
; GFX900: v_fma_f16 ; GFX900: v_fma_f16
▲ Show 20 Lines • Show All 222 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/fma-combine.ll

; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -mattr=-fp32-denormals -verify-machineinstrs -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefix=SI-NOFMA -check-prefix=SI-SAFE -check-prefix=SI -check-prefix=FUNC %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -verify-machineinstrs -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefix=SI-NOFMA -check-prefix=SI-SAFE -check-prefix=SI -check-prefix=FUNC %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=verde -mattr=-fp32-denormals -verify-machineinstrs -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefix=SI-NOFMA -check-prefix=SI-SAFE -check-prefix=SI -check-prefix=FUNC %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=verde -denormal-fp-math-f32=preserve-sign -verify-machineinstrs -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefix=SI-NOFMA -check-prefix=SI-SAFE -check-prefix=SI -check-prefix=FUNC %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -verify-machineinstrs -fp-contract=fast -enable-no-infs-fp-math -enable-unsafe-fp-math -mattr=+fp32-denormals < %s | FileCheck -enable-var-scope -check-prefix=SI-FMA -check-prefix=SI-UNSAFE -check-prefix=SI -check-prefix=FUNC %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=ieee -verify-machineinstrs -fp-contract=fast -enable-no-infs-fp-math -enable-unsafe-fp-math < %s | FileCheck -enable-var-scope -check-prefix=SI-FMA -check-prefix=SI-UNSAFE -check-prefix=SI -check-prefix=FUNC %s
; Note: The SI-FMA conversions of type x * (y + 1) --> x * y + x would be ; Note: The SI-FMA conversions of type x * (y + 1) --> x * y + x would be
; beneficial even without fp32 denormals, but they do require no-infs-fp-math ; beneficial even without fp32 denormals, but they do require no-infs-fp-math
; for correctness. ; for correctness.
declare i32 @llvm.amdgcn.workitem.id.x() #0 declare i32 @llvm.amdgcn.workitem.id.x() #0
declare double @llvm.fabs.f64(double) #0 declare double @llvm.fabs.f64(double) #0
declare double @llvm.fma.f64(double, double, double) #0 declare double @llvm.fma.f64(double, double, double) #0
▲ Show 20 Lines • Show All 666 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/fmaxnum.ll

; RUN: llc -march=amdgcn -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN %s ; RUN: llc -march=amdgcn -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN %s
; RUN: llc -march=amdgcn -mcpu=tonga -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN %s ; RUN: llc -march=amdgcn -mcpu=tonga -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN %s
; GCN-LABEL: {{^}}test_fmax_f32_ieee_mode_on: ; GCN-LABEL: {{^}}test_fmax_f32_ieee_mode_on:
; GCN: v_mul_f32_e64 [[QUIET0:v[0-9]+]], 1.0, s{{[0-9]+}} ; GCN: v_mul_f32_e64 [[QUIET0:v[0-9]+]], 1.0, s{{[0-9]+}}
; GCN: v_mul_f32_e64 [[QUIET1:v[0-9]+]], 1.0, s{{[0-9]+}} ; GCN: v_mul_f32_e64 [[QUIET1:v[0-9]+]], 1.0, s{{[0-9]+}}
; GCN: v_max_f32_e32 [[RESULT:v[0-9]+]], [[QUIET1]], [[QUIET0]] ; GCN: v_max_f32_e32 [[RESULT:v[0-9]+]], [[QUIET1]], [[QUIET0]]
; GCN-NOT: [[RESULT]] ; GCN-NOT: [[RESULT]]
; GCN: buffer_store_dword [[RESULT]] ; GCN: buffer_store_dword [[RESULT]]
define amdgpu_kernel void @test_fmax_f32_ieee_mode_on(float addrspace(1)* %out, float %a, float %b) #0 { define amdgpu_kernel void @test_fmax_f32_ieee_mode_on(float addrspace(1)* %out, float %a, float %b) #0 {
▲ Show 20 Lines • Show All 189 Lines • ▼ Show 20 Linesdefine amdgpu_ps float @fmax_literal_var_f32_no_ieee(float inreg %a) #0 {
ret float %val ret float %val
} }
; GCN-LABEL: {{^}}test_func_fmax_v3f32: ; GCN-LABEL: {{^}}test_func_fmax_v3f32:
; GCN: v_max_f32_e32 ; GCN: v_max_f32_e32
; GCN: v_max_f32_e32 ; GCN: v_max_f32_e32
; GCN: v_max_f32_e32 ; GCN: v_max_f32_e32
; GCN-NOT: v_max_f32 ; GCN-NOT: v_max_f32
define <3 x float> @test_func_fmax_v3f32(<3 x float> %a, <3 x float> %b) nounwind { define <3 x float> @test_func_fmax_v3f32(<3 x float> %a, <3 x float> %b) #0 {
%val = call <3 x float> @llvm.maxnum.v3f32(<3 x float> %a, <3 x float> %b) #0 %val = call <3 x float> @llvm.maxnum.v3f32(<3 x float> %a, <3 x float> %b) #0
ret <3 x float> %val ret <3 x float> %val
} }
declare float @llvm.maxnum.f32(float, float) #1 declare float @llvm.maxnum.f32(float, float) #1
declare <2 x float> @llvm.maxnum.v2f32(<2 x float>, <2 x float>) #1 declare <2 x float> @llvm.maxnum.v2f32(<2 x float>, <2 x float>) #1
declare <3 x float> @llvm.maxnum.v3f32(<3 x float>, <3 x float>) #1 declare <3 x float> @llvm.maxnum.v3f32(<3 x float>, <3 x float>) #1
declare <4 x float> @llvm.maxnum.v4f32(<4 x float>, <4 x float>) #1 declare <4 x float> @llvm.maxnum.v4f32(<4 x float>, <4 x float>) #1
declare <8 x float> @llvm.maxnum.v8f32(<8 x float>, <8 x float>) #1 declare <8 x float> @llvm.maxnum.v8f32(<8 x float>, <8 x float>) #1
declare <16 x float> @llvm.maxnum.v16f32(<16 x float>, <16 x float>) #1 declare <16 x float> @llvm.maxnum.v16f32(<16 x float>, <16 x float>) #1
declare double @llvm.maxnum.f64(double, double) declare double @llvm.maxnum.f64(double, double)
attributes #0 = { nounwind } attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind readnone } attributes #1 = { nounwind readnone }

llvm/test/CodeGen/AMDGPU/fminnum.ll

; RUN: llc -march=amdgcn -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN %s ; RUN: llc -march=amdgcn -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN %s
; RUN: llc -march=amdgcn -mcpu=tonga -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN %s ; RUN: llc -march=amdgcn -mcpu=tonga -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN %s
; GCN-LABEL: {{^}}test_fmin_f32_ieee_mode_on: ; GCN-LABEL: {{^}}test_fmin_f32_ieee_mode_on:
; GCN: v_mul_f32_e64 [[QUIET0:v[0-9]+]], 1.0, s{{[0-9]+}} ; GCN: v_mul_f32_e64 [[QUIET0:v[0-9]+]], 1.0, s{{[0-9]+}}
; GCN: v_mul_f32_e64 [[QUIET1:v[0-9]+]], 1.0, s{{[0-9]+}} ; GCN: v_mul_f32_e64 [[QUIET1:v[0-9]+]], 1.0, s{{[0-9]+}}
; GCN: v_min_f32_e32 [[RESULT:v[0-9]+]], [[QUIET1]], [[QUIET0]] ; GCN: v_min_f32_e32 [[RESULT:v[0-9]+]], [[QUIET1]], [[QUIET0]]
; GCN-NOT: [[RESULT]] ; GCN-NOT: [[RESULT]]
; GCN: buffer_store_dword [[RESULT]] ; GCN: buffer_store_dword [[RESULT]]
define amdgpu_kernel void @test_fmin_f32_ieee_mode_on(float addrspace(1)* %out, float %a, float %b) #0 { define amdgpu_kernel void @test_fmin_f32_ieee_mode_on(float addrspace(1)* %out, float %a, float %b) #0 {
▲ Show 20 Lines • Show All 209 Lines • ▼ Show 20 Lines
declare float @llvm.minnum.f32(float, float) #1 declare float @llvm.minnum.f32(float, float) #1
declare <2 x float> @llvm.minnum.v2f32(<2 x float>, <2 x float>) #1 declare <2 x float> @llvm.minnum.v2f32(<2 x float>, <2 x float>) #1
declare <3 x float> @llvm.minnum.v3f32(<3 x float>, <3 x float>) #1 declare <3 x float> @llvm.minnum.v3f32(<3 x float>, <3 x float>) #1
declare <4 x float> @llvm.minnum.v4f32(<4 x float>, <4 x float>) #1 declare <4 x float> @llvm.minnum.v4f32(<4 x float>, <4 x float>) #1
declare <8 x float> @llvm.minnum.v8f32(<8 x float>, <8 x float>) #1 declare <8 x float> @llvm.minnum.v8f32(<8 x float>, <8 x float>) #1
declare <16 x float> @llvm.minnum.v16f32(<16 x float>, <16 x float>) #1 declare <16 x float> @llvm.minnum.v16f32(<16 x float>, <16 x float>) #1
attributes #0 = { nounwind } attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind readnone } attributes #1 = { nounwind readnone }

llvm/test/CodeGen/AMDGPU/fmul-2-combine-multi-use.ll

; XUN: llc -mtriple=amdgcn-amd-amdhsa -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,SI %s ; XUN: llc -mtriple=amdgcn-amd-amdhsa -denormal-fp-math-f32=preserve-sign -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,SI %s
; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=tonga -mattr=-fp32-denormals,+fp64-fp16-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,VI,SIVI,VI-DENORM %s ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=tonga -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,VI,SIVI,VI-DENORM %s
; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=tonga -mattr=-fp32-denormals,-fp64-fp16-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,VI,SIVI,VI-FLUSH %s ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=tonga -denormal-fp-math=preserve-sign -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,VI,SIVI,VI-FLUSH %s
; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx1010 -mattr=-fp32-denormals,+fp64-fp16-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX10,GFX8_10,GFX10-DENORM %s ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx1010 -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX10,GFX8_10,GFX10-DENORM %s
; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx1010 -mattr=-fp32-denormals,-fp64-fp16-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX10,GFX8_10,GFX10-FLUSH %s ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx1010 -denormal-fp-math=preserve-sign -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX10,GFX8_10,GFX10-FLUSH %s
; Make sure (fmul (fadd x, x), c) -> (fmul x, (fmul 2.0, c)) doesn't ; Make sure (fmul (fadd x, x), c) -> (fmul x, (fmul 2.0, c)) doesn't
; make add an instruction if the fadd has more than one use. ; make add an instruction if the fadd has more than one use.
declare half @llvm.fabs.f16(half) #1 declare half @llvm.fabs.f16(half) #1
declare float @llvm.fabs.f32(float) #1 declare float @llvm.fabs.f32(float) #1
; GCN-LABEL: {{^}}multiple_fadd_use_test_f32: ; GCN-LABEL: {{^}}multiple_fadd_use_test_f32:
▲ Show 20 Lines • Show All 241 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/fmuladd.f16.ll

; RUN: llc -march=amdgcn -mcpu=fiji -mattr=-fp64-fp16-denormals -fp-contract=on -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-STRICT,VI-FLUSH,VI %s ; RUN: llc -march=amdgcn -mcpu=fiji -denormal-fp-math=preserve-sign -denormal-fp-math-f32=ieee -fp-contract=on -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-STRICT,VI-FLUSH,VI %s
; RUN: llc -march=amdgcn -mcpu=fiji -mattr=-fp64-fp16-denormals -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-CONTRACT,VI-FLUSH,VI %s ; RUN: llc -march=amdgcn -mcpu=fiji -denormal-fp-math=preserve-sign -denormal-fp-math-f32=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-CONTRACT,VI-FLUSH,VI %s
; RUN: llc -march=amdgcn -mcpu=fiji -mattr=+fp64-fp16-denormals -fp-contract=on -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-STRICT,GCN-DENORM,GCN-DENORM-STRICT,VI-DENORM-STRICT,VI-DENORM,VI %s ; RUN: llc -march=amdgcn -mcpu=fiji -denormal-fp-math=ieee -denormal-fp-math-f32=ieee -fp-contract=on -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-STRICT,GCN-DENORM,GCN-DENORM-STRICT,VI-DENORM-STRICT,VI-DENORM,VI %s
; RUN: llc -march=amdgcn -mcpu=fiji -mattr=+fp64-fp16-denormals -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-CONTRACT,GCN-DENORM,GCN-DENORM-CONTRACT,VI-DENORM-CONTRACT,VI-DENORM,VI %s ; RUN: llc -march=amdgcn -mcpu=fiji -denormal-fp-math=ieee -denormal-fp-math-f32=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-CONTRACT,GCN-DENORM,GCN-DENORM-CONTRACT,VI-DENORM-CONTRACT,VI-DENORM,VI %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-fp64-fp16-denormals -fp-contract=on -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-STRICT,GFX10-FLUSH,GFX10 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=preserve-sign -denormal-fp-math-f32=ieee -fp-contract=on -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-STRICT,GFX10-FLUSH,GFX10 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-fp64-fp16-denormals -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-CONTRACT,GFX10-FLUSH,GFX10 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=preserve-sign -denormal-fp-math-f32=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-CONTRACT,GFX10-FLUSH,GFX10 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=+fp64-fp16-denormals -fp-contract=on -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-STRICT,GCN-DENORM,GCN-DENORM-STRICT,GFX10-DENORM-STRICT,GFX10-DENORM,GFX10 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=ieee -denormal-fp-math-f32=ieee -fp-contract=on -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-STRICT,GCN-DENORM,GCN-DENORM-STRICT,GFX10-DENORM-STRICT,GFX10-DENORM,GFX10 %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=+fp64-fp16-denormals -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-CONTRACT,GCN-DENORM,GCN-DENORM-CONTRACT,GFX10-DENORM-CONTRACT,GFX10-DENORM,GFX10 %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=ieee -denormal-fp-math-f32=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-CONTRACT,GCN-DENORM,GCN-DENORM-CONTRACT,GFX10-DENORM-CONTRACT,GFX10-DENORM,GFX10 %s
declare i32 @llvm.amdgcn.workitem.id.x() #1 declare i32 @llvm.amdgcn.workitem.id.x() #1
declare half @llvm.fmuladd.f16(half, half, half) #1 declare half @llvm.fmuladd.f16(half, half, half) #1
declare half @llvm.fabs.f16(half) #1 declare half @llvm.fabs.f16(half) #1
; GCN-LABEL: {{^}}fmuladd_f16: ; GCN-LABEL: {{^}}fmuladd_f16:
; VI-FLUSH: v_mac_f16_e32 {{v[0-9]+, v[0-9]+, v[0-9]+}} ; VI-FLUSH: v_mac_f16_e32 {{v[0-9]+, v[0-9]+, v[0-9]+}}
▲ Show 20 Lines • Show All 551 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/fmuladd.f32.ll

; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -mattr=-fp32-denormals,+fast-fmaf -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-STRICT,GCN-FLUSH-MAD,GCN-FLUSH-FASTFMA,GCN-FLUSH-FASTFMA-STRICT,SI %s ; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -mattr=+fast-fmaf -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-STRICT,GCN-FLUSH-MAD,GCN-FLUSH-FASTFMA,GCN-FLUSH-FASTFMA-STRICT,SI %s
; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -mattr=+fp32-denormals,+fast-fmaf -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,GCN-DENORM,SI-DENORM,GCN-DENORM-FASTFMA,GCN-DENORM-FASTFMA-STRICT,SI %s ; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -denormal-fp-math-f32=ieee -mattr=+fast-fmaf -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,GCN-DENORM,SI-DENORM,GCN-DENORM-FASTFMA,GCN-DENORM-FASTFMA-STRICT,SI %s
; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -mattr=-fp32-denormals,-fast-fmaf -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-STRICT,GCN-FLUSH-MAD,SI-FLUSH,GCN-FLUSH-SLOWFMA,GCN-FLUSH-SLOWFMA-STRICT,SI %s ; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -denormal-fp-math-f32=preserve-sign -mattr=-fast-fmaf -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-STRICT,GCN-FLUSH-MAD,SI-FLUSH,GCN-FLUSH-SLOWFMA,GCN-FLUSH-SLOWFMA-STRICT,SI %s
; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -mattr=+fp32-denormals,-fast-fmaf -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,GCN-DENORM,SI-DENORM,GCN-DENORM-SLOWFMA,GCN-DENORM-SLOWFMA-STRICT,SI %s ; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -denormal-fp-math-f32=ieee -mattr=-fast-fmaf -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,GCN-DENORM,SI-DENORM,GCN-DENORM-SLOWFMA,GCN-DENORM-SLOWFMA-STRICT,SI %s
; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -mattr=-fp32-denormals,+fast-fmaf -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-CONTRACT,GCN-FLUSH-MAD,SI-FLUSH,GCN-FLUSH-FASTFMA,GCN-FLUSH-FASTFMA-CONTRACT,SI %s ; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -mattr=+fast-fmaf -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-CONTRACT,GCN-FLUSH-MAD,SI-FLUSH,GCN-FLUSH-FASTFMA,GCN-FLUSH-FASTFMA-CONTRACT,SI %s
; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -mattr=+fp32-denormals,+fast-fmaf -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-CONTRACT,GCN-DENORM,SI-DENORM,GCN-DENORM-FASTFMA,GCN-DENORM-FASTFMA-CONTRACT,SI %s ; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=tahiti -denormal-fp-math-f32=ieee -mattr=+fast-fmaf -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-CONTRACT,GCN-DENORM,SI-DENORM,GCN-DENORM-FASTFMA,GCN-DENORM-FASTFMA-CONTRACT,SI %s
; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -mattr=-fp32-denormals,-fast-fmaf -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-CONTRACT,GCN-FLUSH-MAD,SI-FLUSH,GCN-FLUSH-SLOWFMA,GCN-FLUSH-SLOWFMA-CONTRACT,SI %s ; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -denormal-fp-math-f32=preserve-sign -mattr=-fast-fmaf -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-CONTRACT,GCN-FLUSH-MAD,SI-FLUSH,GCN-FLUSH-SLOWFMA,GCN-FLUSH-SLOWFMA-CONTRACT,SI %s
; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -mattr=+fp32-denormals,-fast-fmaf -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-CONTRACT,GCN-DENORM,SI-DENORM,GCN-DENORM-SLOWFMA,GCN-DENORM-SLOWFMA-CONTRACT,SI %s ; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=verde -denormal-fp-math-f32=ieee -mattr=-fast-fmaf -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-CONTRACT,GCN-DENORM,SI-DENORM,GCN-DENORM-SLOWFMA,GCN-DENORM-SLOWFMA-CONTRACT,SI %s
; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx900 -mattr=-fp32-denormals -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-STRICT,GCN-FLUSH-MAD,GFX9-FLUSH,GCN-FLUSH-FASTFMA,GCN-FLUSH-FASTFMA-STRICT,GFX900 %s ; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx900 -denormal-fp-math-f32=preserve-sign -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-STRICT,GCN-FLUSH-MAD,GFX9-FLUSH,GCN-FLUSH-FASTFMA,GCN-FLUSH-FASTFMA-STRICT,GFX900 %s
; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx900 -mattr=+fp32-denormals -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,GCN-DENORM,GFX9-DENORM,GCN-DENORM-FASTFMA,GCN-DENORM-FASTFMA-STRICT,GFX900 %s ; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx900 -denormal-fp-math-f32=ieee -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,GCN-DENORM,GFX9-DENORM,GCN-DENORM-FASTFMA,GCN-DENORM-FASTFMA-STRICT,GFX900 %s
; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx906 -mattr=-fp32-denormals -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-STRICT,GCN-FLUSH-FMAC,GFX9-FLUSH,GCN-FLUSH-FASTFMA,GCN-FLUSH-FASTFMA-STRICT,GFX906 %s ; RUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx906 -denormal-fp-math-f32=preserve-sign -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-FLUSH,GCN-FLUSH-STRICT,GCN-FLUSH-FMAC,GFX9-FLUSH,GCN-FLUSH-FASTFMA,GCN-FLUSH-FASTFMA-STRICT,GFX906 %s
; FIXME: Should probably test this, but sometimes selecting fmac is painful to match. ; FIXME: Should probably test this, but sometimes selecting fmac is painful to match.
; XUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx906 -mattr=+fp32-denormals -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,GCN-DENORM,GFX9-DENORM,GCN-DENORM-FASTFMA,GCN-DENORM-FASTFMA-STRICT,GFX906 %s ; XUN: llc -amdgpu-scalarize-global-loads=false -verify-machineinstrs -mcpu=gfx906 -denormal-fp-math-f32=ieee -fp-contract=on < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GCN-DENORM-STRICT,GCN-DENORM,GFX9-DENORM,GCN-DENORM-FASTFMA,GCN-DENORM-FASTFMA-STRICT,GFX906 %s
; Test all permutations of: fp32 denormals, fast fp contract, fp contract enabled for fmuladd, fmaf fast/slow. ; Test all permutations of: fp32 denormals, fast fp contract, fp contract enabled for fmuladd, fmaf fast/slow.
target triple = "amdgcn--" target triple = "amdgcn--"
declare i32 @llvm.amdgcn.workitem.id.x() #1 declare i32 @llvm.amdgcn.workitem.id.x() #1
▲ Show 20 Lines • Show All 582 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/fmuladd.v2f16.ll

; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -mattr=-fp64-fp16-denormals -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-STRICT,GFX9-FLUSH,GFX9 %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-STRICT,GFX9-FLUSH,GFX9 %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -mattr=-fp64-fp16-denormals -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-STRICT,GFX9-FLUSH,GFX9 %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-STRICT,GFX9-FLUSH,GFX9 %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -mattr=-fp64-fp16-denormals -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-CONTRACT,GFX9-FLUSH,GFX9 %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-CONTRACT,GFX9-FLUSH,GFX9 %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -mattr=-fp64-fp16-denormals -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-CONTRACT,GFX9-FLUSH,GFX9 %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=preserve-sign -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-CONTRACT,GFX9-FLUSH,GFX9 %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -mattr=+fp64-fp16-denormals -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-STRICT,GFX9-DENORM-STRICT,GFX9-DENORM,GFX9 %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-STRICT,GFX9-DENORM-STRICT,GFX9-DENORM,GFX9 %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -mattr=+fp64-fp16-denormals -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-STRICT,GFX9-DENORM-STRICT,GFX9-DENORM,GFX9 %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -fp-contract=on -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-STRICT,GFX9-DENORM-STRICT,GFX9-DENORM,GFX9 %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -mattr=+fp64-fp16-denormals -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-CONTRACT,GFX9-DENORM-CONTRACT,GFX9-DENORM,GFX9 %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-CONTRACT,GFX9-DENORM-CONTRACT,GFX9-DENORM,GFX9 %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -mattr=+fp64-fp16-denormals -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-CONTRACT,GFX9-DENORM-CONTRACT,GFX9-DENORM,GFX9 %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -denormal-fp-math=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GCN-CONTRACT,GFX9-DENORM-CONTRACT,GFX9-DENORM,GFX9 %s
declare i32 @llvm.amdgcn.workitem.id.x() #1 declare i32 @llvm.amdgcn.workitem.id.x() #1
declare <2 x half> @llvm.fmuladd.v2f16(<2 x half>, <2 x half>, <2 x half>) #1 declare <2 x half> @llvm.fmuladd.v2f16(<2 x half>, <2 x half>, <2 x half>) #1
declare <2 x half> @llvm.fabs.v2f16(<2 x half>) #1 declare <2 x half> @llvm.fabs.v2f16(<2 x half>) #1
; GCN-LABEL: {{^}}fmuladd_v2f16: ; GCN-LABEL: {{^}}fmuladd_v2f16:
; GFX9-FLUSH: v_pk_mul_f16 {{v[0-9]+, v[0-9]+, v[0-9]+}} ; GFX9-FLUSH: v_pk_mul_f16 {{v[0-9]+, v[0-9]+, v[0-9]+}}
; GFX9-FLUSH: v_pk_add_f16 {{v[0-9]+, v[0-9]+, v[0-9]+}} ; GFX9-FLUSH: v_pk_add_f16 {{v[0-9]+, v[0-9]+, v[0-9]+}}
▲ Show 20 Lines • Show All 90 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/fneg-combines.ll

; RUN: llc -march=amdgcn -mcpu=hawaii -start-after=sink -mattr=-fp32-denormals,+flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=GCN-SAFE -check-prefix=SI -check-prefix=FUNC %s ; RUN: llc -march=amdgcn -mcpu=hawaii -start-after=sink -mattr=+flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=GCN-SAFE -check-prefix=SI -check-prefix=FUNC %s
; RUN: llc -enable-no-signed-zeros-fp-math -march=amdgcn -mcpu=hawaii -mattr=-fp32-denormals,+flat-for-global -start-after=sink -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=GCN-NSZ -check-prefix=SI -check-prefix=FUNC %s ; RUN: llc -enable-no-signed-zeros-fp-math -march=amdgcn -mcpu=hawaii -mattr=+flat-for-global -start-after=sink -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=GCN-NSZ -check-prefix=SI -check-prefix=FUNC %s
; RUN: llc -march=amdgcn -mcpu=fiji -mattr=-fp32-denormals -start-after=sink --verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=GCN-SAFE -check-prefix=VI -check-prefix=FUNC %s ; RUN: llc -march=amdgcn -mcpu=fiji -start-after=sink --verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=GCN-SAFE -check-prefix=VI -check-prefix=FUNC %s
; RUN: llc -enable-no-signed-zeros-fp-math -march=amdgcn -mcpu=fiji -mattr=-fp32-denormals -start-after=sink -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=GCN-NSZ -check-prefix=VI -check-prefix=FUNC %s ; RUN: llc -enable-no-signed-zeros-fp-math -march=amdgcn -mcpu=fiji -start-after=sink -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=GCN-NSZ -check-prefix=VI -check-prefix=FUNC %s
; -------------------------------------------------------------------------------- ; --------------------------------------------------------------------------------
; fadd tests ; fadd tests
; -------------------------------------------------------------------------------- ; --------------------------------------------------------------------------------
; GCN-LABEL: {{^}}v_fneg_add_f32: ; GCN-LABEL: {{^}}v_fneg_add_f32:
; GCN: {{buffer|flat}}_load_dword [[A:v[0-9]+]] ; GCN: {{buffer|flat}}_load_dword [[A:v[0-9]+]]
; GCN: {{buffer|flat}}_load_dword [[B:v[0-9]+]] ; GCN: {{buffer|flat}}_load_dword [[B:v[0-9]+]]
▲ Show 20 Lines • Show All 2,503 Lines • ▼ Show 20 Lines
declare float @llvm.amdgcn.sin.f32(float) #1 declare float @llvm.amdgcn.sin.f32(float) #1
declare float @llvm.amdgcn.rcp.f32(float) #1 declare float @llvm.amdgcn.rcp.f32(float) #1
declare float @llvm.amdgcn.rcp.legacy(float) #1 declare float @llvm.amdgcn.rcp.legacy(float) #1
declare float @llvm.amdgcn.fmul.legacy(float, float) #1 declare float @llvm.amdgcn.fmul.legacy(float, float) #1
declare float @llvm.amdgcn.interp.p1(float, i32, i32, i32) #0 declare float @llvm.amdgcn.interp.p1(float, i32, i32, i32) #0
declare float @llvm.amdgcn.interp.p2(float, float, i32, i32, i32) #0 declare float @llvm.amdgcn.interp.p2(float, float, i32, i32, i32) #0
attributes #0 = { nounwind } attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind readnone } attributes #1 = { nounwind readnone }

llvm/test/CodeGen/AMDGPU/fpext-free.ll

; RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX89,GFX9,GFX9-F32FLUSH %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -denormal-fp-math-f32=preserve-sign -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX89,GFX9,GFX9-F32FLUSH %s
; RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=+fp32-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX89,GFX9,GFX9-F32DENORM %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -denormal-fp-math-f32=ieee -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX89,GFX9,GFX9-F32DENORM %s
; RUN: llc -march=amdgcn -mcpu=gfx803 -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX89,VI,VI-F32FLUSH %s ; RUN: llc -march=amdgcn -mcpu=gfx803 -denormal-fp-math-f32=preserve-sign -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX89,VI,VI-F32FLUSH %s
; RUN: llc -march=amdgcn -mcpu=gfx803 -mattr=+fp32-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX89,VI,VI-F32DENORM %s ; RUN: llc -march=amdgcn -mcpu=gfx803 -denormal-fp-math-f32=ieee -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GCN,GFX89,VI,VI-F32DENORM %s
; fold (fadd (fpext (fmul x, y)), z) -> (fma (fpext x), (fpext y), z) ; fold (fadd (fpext (fmul x, y)), z) -> (fma (fpext x), (fpext y), z)
; GCN-LABEL: {{^}}fadd_fpext_fmul_f16_to_f32: ; GCN-LABEL: {{^}}fadd_fpext_fmul_f16_to_f32:
; GCN: s_waitcnt ; GCN: s_waitcnt
; GFX9-F32FLUSH-NEXT: v_mad_mix_f32 v0, v0, v1, v2 op_sel_hi:[1,1,0]{{$}} ; GFX9-F32FLUSH-NEXT: v_mad_mix_f32 v0, v0, v1, v2 op_sel_hi:[1,1,0]{{$}}
; GFX9-F32FLUSH-NEXT: s_setpc_b64 ; GFX9-F32FLUSH-NEXT: s_setpc_b64
▲ Show 20 Lines • Show All 372 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/frem.ll

; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -check-prefix=SI -check-prefix=GCN -check-prefix=FUNC %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -verify-machineinstrs < %s | FileCheck -check-prefix=SI -check-prefix=GCN -check-prefix=FUNC %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=bonaire -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -check-prefix=CI -check-prefix=GCN -check-prefix=FUNC %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=bonaire -verify-machineinstrs < %s | FileCheck -check-prefix=CI -check-prefix=GCN -check-prefix=FUNC %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tonga -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=CI -check-prefix=GCN -check-prefix=FUNC %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tonga -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=CI -check-prefix=GCN -check-prefix=FUNC %s
; FUNC-LABEL: {{^}}frem_f32: ; FUNC-LABEL: {{^}}frem_f32:
; GCN-DAG: buffer_load_dword [[X:v[0-9]+]], {{.*$}} ; GCN-DAG: buffer_load_dword [[X:v[0-9]+]], {{.*$}}
; GCN-DAG: buffer_load_dword [[Y:v[0-9]+]], {{.*}} offset:16 ; GCN-DAG: buffer_load_dword [[Y:v[0-9]+]], {{.*}} offset:16
; GCN: v_div_scale_f32 ; GCN: v_div_scale_f32
; GCN: v_rcp_f32_e32 ; GCN: v_rcp_f32_e32
; GCN: v_fma_f32 ; GCN: v_fma_f32
▲ Show 20 Lines • Show All 92 Lines • ▼ Show 20 Linesdefine amdgpu_kernel void @frem_v2f64(<2 x double> addrspace(1)* %out, <2 x double> addrspace(1)* %in1,
%gep2 = getelementptr <2 x double>, <2 x double> addrspace(1)* %in2, i32 4 %gep2 = getelementptr <2 x double>, <2 x double> addrspace(1)* %in2, i32 4
%r0 = load <2 x double>, <2 x double> addrspace(1)* %in1, align 16 %r0 = load <2 x double>, <2 x double> addrspace(1)* %in1, align 16
%r1 = load <2 x double>, <2 x double> addrspace(1)* %gep2, align 16 %r1 = load <2 x double>, <2 x double> addrspace(1)* %gep2, align 16
%r2 = frem <2 x double> %r0, %r1 %r2 = frem <2 x double> %r0, %r1
store <2 x double> %r2, <2 x double> addrspace(1)* %out, align 16 store <2 x double> %r2, <2 x double> addrspace(1)* %out, align 16
ret void ret void
} }
attributes #0 = { nounwind "unsafe-fp-math"="false" } attributes #0 = { nounwind "unsafe-fp-math"="false" "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind "unsafe-fp-math"="true" } attributes #1 = { nounwind "unsafe-fp-math"="true" "denormal-fp-math-f32"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/AMDGPU/hsa-fp-mode.ll

Show First 20 Lines • Show All 86 Lines • ▼ Show 20 Lines
define amdgpu_kernel void @test_no_ieee_mode_no_dx10_clamp_vi(float addrspace(1)* %out0, double addrspace(1)* %out1) #8 { define amdgpu_kernel void @test_no_ieee_mode_no_dx10_clamp_vi(float addrspace(1)* %out0, double addrspace(1)* %out1) #8 {
store float 0.0, float addrspace(1)* %out0 store float 0.0, float addrspace(1)* %out0
store double 0.0, double addrspace(1)* %out1 store double 0.0, double addrspace(1)* %out1
ret void ret void
} }
attributes #0 = { nounwind "target-cpu"="kaveri" "target-features"="-code-object-v3" } attributes #0 = { nounwind "target-cpu"="kaveri" "target-features"="-code-object-v3" }
attributes #1 = { nounwind "target-cpu"="fiji" "target-features"="-code-object-v3" } attributes #1 = { nounwind "target-cpu"="fiji" "target-features"="-code-object-v3" }
attributes #2 = { nounwind "target-features"="-code-object-v3,-fp32-denormals,+fp64-fp16-denormals" } attributes #2 = { nounwind "target-features"="-code-object-v3" "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #3 = { nounwind "target-features"="-code-object-v3,+fp32-denormals,-fp64-fp16-denormals" } attributes #3 = { nounwind "target-features"="-code-object-v3" "denormal-fp-math-f32"="ieee,ieee" "denormal-fp-math"="preserve-sign,preserve-sign" }
attributes #4 = { nounwind "target-features"="-code-object-v3,+fp32-denormals,+fp64-fp16-denormals" } attributes #4 = { nounwind "target-features"="-code-object-v3" "denormal-fp-math"="ieee,ieee" }
attributes #5 = { nounwind "target-features"="-code-object-v3,-fp32-denormals,-fp64-fp16-denormals" } attributes #5 = { nounwind "target-features"="-code-object-v3" "denormal-fp-math"="preserve-sign,preserve-sign" }
attributes #6 = { nounwind "amdgpu-dx10-clamp"="false" "target-cpu"="fiji" "target-features"="-code-object-v3" } attributes #6 = { nounwind "amdgpu-dx10-clamp"="false" "target-cpu"="fiji" "target-features"="-code-object-v3" }
attributes #7 = { nounwind "amdgpu-ieee"="false" "target-cpu"="fiji" "target-features"="-code-object-v3" } attributes #7 = { nounwind "amdgpu-ieee"="false" "target-cpu"="fiji" "target-features"="-code-object-v3" }
attributes #8 = { nounwind "amdgpu-dx10-clamp"="false" "amdgpu-ieee"="false" "target-cpu"="fiji" "target-features"="-code-object-v3" } attributes #8 = { nounwind "amdgpu-dx10-clamp"="false" "amdgpu-ieee"="false" "target-cpu"="fiji" "target-features"="-code-object-v3" }

llvm/test/CodeGen/AMDGPU/known-never-snan.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=fiji -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN %s ; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=fiji -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN %s
; Mostly overlaps with fmed3.ll to stress specific cases of ; Mostly overlaps with fmed3.ll to stress specific cases of
; isKnownNeverSNaN. ; isKnownNeverSNaN.
define float @v_test_known_not_snan_fabs_input_fmed3_r_i_i_f32(float %a) #0 { define float @v_test_known_not_snan_fabs_input_fmed3_r_i_i_f32(float %a) #0 {
; GCN-LABEL: v_test_known_not_snan_fabs_input_fmed3_r_i_i_f32: ; GCN-LABEL: v_test_known_not_snan_fabs_input_fmed3_r_i_i_f32:
; GCN: ; %bb.0: ; GCN: ; %bb.0:
; GCN-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0) ; GCN-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
▲ Show 20 Lines • Show All 651 Lines • ▼ Show 20 Lines
declare float @llvm.amdgcn.fmul.legacy(float, float) #1 declare float @llvm.amdgcn.fmul.legacy(float, float) #1
declare float @llvm.amdgcn.fmed3.f32(float, float, float) #1 declare float @llvm.amdgcn.fmed3.f32(float, float, float) #1
declare float @llvm.amdgcn.frexp.mant.f32(float) #1 declare float @llvm.amdgcn.frexp.mant.f32(float) #1
declare float @llvm.amdgcn.rcp.f32(float) #1 declare float @llvm.amdgcn.rcp.f32(float) #1
declare float @llvm.amdgcn.rsq.f32(float) #1 declare float @llvm.amdgcn.rsq.f32(float) #1
declare float @llvm.amdgcn.fract.f32(float) #1 declare float @llvm.amdgcn.fract.f32(float) #1
declare float @llvm.amdgcn.cubeid(float, float, float) #0 declare float @llvm.amdgcn.cubeid(float, float, float) #0
attributes #0 = { nounwind } attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind readnone speculatable } attributes #1 = { nounwind readnone speculatable }

llvm/test/CodeGen/AMDGPU/llvm.amdgcn.rcp.ll

Show First 20 Lines • Show All 134 Lines • ▼ Show 20 Lines
define amdgpu_kernel void @unsafe_rsq_rcp_pat_f64(double addrspace(1)* %out, double %src) #2 { define amdgpu_kernel void @unsafe_rsq_rcp_pat_f64(double addrspace(1)* %out, double %src) #2 {
%sqrt = call double @llvm.sqrt.f64(double %src) %sqrt = call double @llvm.sqrt.f64(double %src)
%rcp = call double @llvm.amdgcn.rcp.f64(double %sqrt) %rcp = call double @llvm.amdgcn.rcp.f64(double %sqrt)
store double %rcp, double addrspace(1)* %out, align 8 store double %rcp, double addrspace(1)* %out, align 8
ret void ret void
} }
attributes #0 = { nounwind readnone } attributes #0 = { nounwind readnone }
attributes #1 = { nounwind "unsafe-fp-math"="false" "target-features"="-fp32-denormals" } attributes #1 = { nounwind "unsafe-fp-math"="false" "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #2 = { nounwind "unsafe-fp-math"="true" "target-features"="-fp32-denormals" } attributes #2 = { nounwind "unsafe-fp-math"="true" "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #3 = { nounwind "unsafe-fp-math"="false" "target-features"="+fp32-denormals" } attributes #3 = { nounwind "unsafe-fp-math"="false" "denormal-fp-math-f32"="ieee,ieee" }
attributes #4 = { nounwind "unsafe-fp-math"="true" "target-features"="+fp32-denormals" } attributes #4 = { nounwind "unsafe-fp-math"="true" "denormal-fp-math-f32"="ieee,ieee" }

llvm/test/CodeGen/AMDGPU/llvm.fmuladd.f16.ll

; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -mattr=-fp32-denormals,-fp64-fp16-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=SI -check-prefix=SI-FLUSH %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -denormal-fp-math=preserve-sign -denormal-fp-math-f32=preserve-sign -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=SI -check-prefix=SI-FLUSH %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=fiji -mattr=-fp32-denormals,-fp64-fp16-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=VI -check-prefix=VI-FLUSH %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=fiji -denormal-fp-math=preserve-sign -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=VI -check-prefix=VI-FLUSH %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -mattr=-fp32-denormals,+fp64-fp16-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=SI -check-prefix=SI-DENORM %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -denormal-fp-math=ieee -denormal-fp-math-f32=preserve-sign -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=SI -check-prefix=SI-DENORM %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=fiji -mattr=-fp32-denormals,+fp64-fp16-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=VI -check-prefix=VI-DENORM %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=fiji -denormal-fp-math=ieee -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=VI -check-prefix=VI-DENORM %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx1010 -mattr=-fp32-denormals,-fp64-fp16-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=GFX10 -check-prefix=GFX10-FLUSH %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=preserve-sign -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=GFX10 -check-prefix=GFX10-FLUSH %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx1010 -mattr=-fp32-denormals,+fp64-fp16-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=GFX10 -check-prefix=GFX10-DENORM %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx1010 -denormal-fp-math=ieee -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=GCN -check-prefix=GFX10 -check-prefix=GFX10-DENORM %s
declare half @llvm.fmuladd.f16(half %a, half %b, half %c) declare half @llvm.fmuladd.f16(half %a, half %b, half %c)
declare <2 x half> @llvm.fmuladd.v2f16(<2 x half> %a, <2 x half> %b, <2 x half> %c) declare <2 x half> @llvm.fmuladd.v2f16(<2 x half> %a, <2 x half> %b, <2 x half> %c)
; GCN-LABEL: {{^}}fmuladd_f16 ; GCN-LABEL: {{^}}fmuladd_f16
; GCN: buffer_load_ushort v[[A_F16:[0-9]+]] ; GCN: buffer_load_ushort v[[A_F16:[0-9]+]]
; GCN: buffer_load_ushort v[[B_F16:[0-9]+]] ; GCN: buffer_load_ushort v[[B_F16:[0-9]+]]
; GCN: buffer_load_ushort v[[C_F16:[0-9]+]] ; GCN: buffer_load_ushort v[[C_F16:[0-9]+]]
▲ Show 20 Lines • Show All 173 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/llvm.maxnum.f16.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -mtriple=amdgcn-- -mcpu=tahiti -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=SI %s ; RUN: llc -mtriple=amdgcn-- -mcpu=tahiti -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=SI %s
; RUN: llc -mtriple=amdgcn-- -mcpu=fiji -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=VI,SIVI %s ; RUN: llc -mtriple=amdgcn-- -mcpu=fiji -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=VI,SIVI %s
; RUN: llc -mtriple=amdgcn-- -mcpu=gfx900 -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9 %s ; RUN: llc -mtriple=amdgcn-- -mcpu=gfx900 -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9 %s
declare half @llvm.maxnum.f16(half %a, half %b) declare half @llvm.maxnum.f16(half %a, half %b)
declare <2 x half> @llvm.maxnum.v2f16(<2 x half> %a, <2 x half> %b) declare <2 x half> @llvm.maxnum.v2f16(<2 x half> %a, <2 x half> %b)
declare <3 x half> @llvm.maxnum.v3f16(<3 x half> %a, <3 x half> %b) declare <3 x half> @llvm.maxnum.v3f16(<3 x half> %a, <3 x half> %b)
declare <4 x half> @llvm.maxnum.v4f16(<4 x half> %a, <4 x half> %b) declare <4 x half> @llvm.maxnum.v4f16(<4 x half> %a, <4 x half> %b)
define amdgpu_kernel void @maxnum_f16( define amdgpu_kernel void @maxnum_f16(
; SI-LABEL: maxnum_f16: ; SI-LABEL: maxnum_f16:
▲ Show 20 Lines • Show All 70 Lines • ▼ Show 20 Lines
; GFX9-NEXT: v_max_f16_e32 v0, v0, v0 ; GFX9-NEXT: v_max_f16_e32 v0, v0, v0
; GFX9-NEXT: s_waitcnt vmcnt(0) ; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_max_f16_e32 v1, v1, v1 ; GFX9-NEXT: v_max_f16_e32 v1, v1, v1
; GFX9-NEXT: v_max_f16_e32 v0, v0, v1 ; GFX9-NEXT: v_max_f16_e32 v0, v0, v1
; GFX9-NEXT: buffer_store_short v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_short v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
half addrspace(1)* %r, half addrspace(1)* %r,
half addrspace(1)* %a, half addrspace(1)* %a,
half addrspace(1)* %b) { half addrspace(1)* %b) #0 {
entry: entry:
%a.val = load volatile half, half addrspace(1)* %a %a.val = load volatile half, half addrspace(1)* %a
%b.val = load volatile half, half addrspace(1)* %b %b.val = load volatile half, half addrspace(1)* %b
%r.val = call half @llvm.maxnum.f16(half %a.val, half %b.val) %r.val = call half @llvm.maxnum.f16(half %a.val, half %b.val)
store half %r.val, half addrspace(1)* %r store half %r.val, half addrspace(1)* %r
ret void ret void
} }
▲ Show 20 Lines • Show All 52 Lines • ▼ Show 20 Lines
; GFX9-NEXT: s_mov_b32 s7, s3 ; GFX9-NEXT: s_mov_b32 s7, s3
; GFX9-NEXT: buffer_load_ushort v0, off, s[4:7], 0 ; GFX9-NEXT: buffer_load_ushort v0, off, s[4:7], 0
; GFX9-NEXT: s_waitcnt vmcnt(0) ; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_max_f16_e32 v0, v0, v0 ; GFX9-NEXT: v_max_f16_e32 v0, v0, v0
; GFX9-NEXT: v_max_f16_e32 v0, 0x4200, v0 ; GFX9-NEXT: v_max_f16_e32 v0, 0x4200, v0
; GFX9-NEXT: buffer_store_short v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_short v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
half addrspace(1)* %r, half addrspace(1)* %r,
half addrspace(1)* %b) { half addrspace(1)* %b) #0 {
entry: entry:
%b.val = load half, half addrspace(1)* %b %b.val = load half, half addrspace(1)* %b
%r.val = call half @llvm.maxnum.f16(half 3.0, half %b.val) %r.val = call half @llvm.maxnum.f16(half 3.0, half %b.val)
store half %r.val, half addrspace(1)* %r store half %r.val, half addrspace(1)* %r
ret void ret void
} }
define amdgpu_kernel void @maxnum_f16_imm_b( define amdgpu_kernel void @maxnum_f16_imm_b(
▲ Show 20 Lines • Show All 51 Lines • ▼ Show 20 Lines
; GFX9-NEXT: s_mov_b32 s7, s3 ; GFX9-NEXT: s_mov_b32 s7, s3
; GFX9-NEXT: buffer_load_ushort v0, off, s[4:7], 0 ; GFX9-NEXT: buffer_load_ushort v0, off, s[4:7], 0
; GFX9-NEXT: s_waitcnt vmcnt(0) ; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_max_f16_e32 v0, v0, v0 ; GFX9-NEXT: v_max_f16_e32 v0, v0, v0
; GFX9-NEXT: v_max_f16_e32 v0, 4.0, v0 ; GFX9-NEXT: v_max_f16_e32 v0, 4.0, v0
; GFX9-NEXT: buffer_store_short v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_short v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
half addrspace(1)* %r, half addrspace(1)* %r,
half addrspace(1)* %a) { half addrspace(1)* %a) #0 {
entry: entry:
%a.val = load half, half addrspace(1)* %a %a.val = load half, half addrspace(1)* %a
%r.val = call half @llvm.maxnum.f16(half %a.val, half 4.0) %r.val = call half @llvm.maxnum.f16(half %a.val, half 4.0)
store half %r.val, half addrspace(1)* %r store half %r.val, half addrspace(1)* %r
ret void ret void
} }
define amdgpu_kernel void @maxnum_v2f16( define amdgpu_kernel void @maxnum_v2f16(
▲ Show 20 Lines • Show All 66 Lines • ▼ Show 20 Lines
; GFX9-NEXT: s_waitcnt lgkmcnt(0) ; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: v_pk_max_f16 v1, s4, s4 ; GFX9-NEXT: v_pk_max_f16 v1, s4, s4
; GFX9-NEXT: v_pk_max_f16 v0, s5, s5 ; GFX9-NEXT: v_pk_max_f16 v0, s5, s5
; GFX9-NEXT: v_pk_max_f16 v0, v1, v0 ; GFX9-NEXT: v_pk_max_f16 v0, v1, v0
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
<2 x half> addrspace(1)* %r, <2 x half> addrspace(1)* %r,
<2 x half> addrspace(1)* %a, <2 x half> addrspace(1)* %a,
<2 x half> addrspace(1)* %b) { <2 x half> addrspace(1)* %b) #0 {
entry: entry:
%a.val = load <2 x half>, <2 x half> addrspace(1)* %a %a.val = load <2 x half>, <2 x half> addrspace(1)* %a
%b.val = load <2 x half>, <2 x half> addrspace(1)* %b %b.val = load <2 x half>, <2 x half> addrspace(1)* %b
%r.val = call <2 x half> @llvm.maxnum.v2f16(<2 x half> %a.val, <2 x half> %b.val) %r.val = call <2 x half> @llvm.maxnum.v2f16(<2 x half> %a.val, <2 x half> %b.val)
store <2 x half> %r.val, <2 x half> addrspace(1)* %r store <2 x half> %r.val, <2 x half> addrspace(1)* %r
ret void ret void
} }
▲ Show 20 Lines • Show All 51 Lines • ▼ Show 20 Lines
; GFX9-NEXT: s_mov_b32 s1, s5 ; GFX9-NEXT: s_mov_b32 s1, s5
; GFX9-NEXT: s_waitcnt lgkmcnt(0) ; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: v_pk_max_f16 v0, s4, s4 ; GFX9-NEXT: v_pk_max_f16 v0, s4, s4
; GFX9-NEXT: s_mov_b32 s4, 0x44004200 ; GFX9-NEXT: s_mov_b32 s4, 0x44004200
; GFX9-NEXT: v_pk_max_f16 v0, v0, s4 ; GFX9-NEXT: v_pk_max_f16 v0, v0, s4
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
<2 x half> addrspace(1)* %r, <2 x half> addrspace(1)* %r,
<2 x half> addrspace(1)* %b) { <2 x half> addrspace(1)* %b) #0 {
entry: entry:
%b.val = load <2 x half>, <2 x half> addrspace(1)* %b %b.val = load <2 x half>, <2 x half> addrspace(1)* %b
%r.val = call <2 x half> @llvm.maxnum.v2f16(<2 x half> <half 3.0, half 4.0>, <2 x half> %b.val) %r.val = call <2 x half> @llvm.maxnum.v2f16(<2 x half> <half 3.0, half 4.0>, <2 x half> %b.val)
store <2 x half> %r.val, <2 x half> addrspace(1)* %r store <2 x half> %r.val, <2 x half> addrspace(1)* %r
ret void ret void
} }
define amdgpu_kernel void @maxnum_v2f16_imm_b( define amdgpu_kernel void @maxnum_v2f16_imm_b(
▲ Show 20 Lines • Show All 50 Lines • ▼ Show 20 Lines
; GFX9-NEXT: s_mov_b32 s1, s5 ; GFX9-NEXT: s_mov_b32 s1, s5
; GFX9-NEXT: s_waitcnt lgkmcnt(0) ; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: v_pk_max_f16 v0, s4, s4 ; GFX9-NEXT: v_pk_max_f16 v0, s4, s4
; GFX9-NEXT: s_mov_b32 s4, 0x42004400 ; GFX9-NEXT: s_mov_b32 s4, 0x42004400
; GFX9-NEXT: v_pk_max_f16 v0, v0, s4 ; GFX9-NEXT: v_pk_max_f16 v0, v0, s4
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
<2 x half> addrspace(1)* %r, <2 x half> addrspace(1)* %r,
<2 x half> addrspace(1)* %a) { <2 x half> addrspace(1)* %a) #0 {
entry: entry:
%a.val = load <2 x half>, <2 x half> addrspace(1)* %a %a.val = load <2 x half>, <2 x half> addrspace(1)* %a
%r.val = call <2 x half> @llvm.maxnum.v2f16(<2 x half> %a.val, <2 x half> <half 4.0, half 3.0>) %r.val = call <2 x half> @llvm.maxnum.v2f16(<2 x half> %a.val, <2 x half> <half 4.0, half 3.0>)
store <2 x half> %r.val, <2 x half> addrspace(1)* %r store <2 x half> %r.val, <2 x half> addrspace(1)* %r
ret void ret void
} }
; FIXME: Scalarize with undef half ; FIXME: Scalarize with undef half
▲ Show 20 Lines • Show All 82 Lines • ▼ Show 20 Lines
; GFX9-NEXT: v_pk_max_f16 v2, s7, s7 ; GFX9-NEXT: v_pk_max_f16 v2, s7, s7
; GFX9-NEXT: v_pk_max_f16 v1, s5, s5 ; GFX9-NEXT: v_pk_max_f16 v1, s5, s5
; GFX9-NEXT: v_pk_max_f16 v1, v1, v2 ; GFX9-NEXT: v_pk_max_f16 v1, v1, v2
; GFX9-NEXT: buffer_store_short v1, off, s[0:3], 0 offset:4 ; GFX9-NEXT: buffer_store_short v1, off, s[0:3], 0 offset:4
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
<3 x half> addrspace(1)* %r, <3 x half> addrspace(1)* %r,
<3 x half> addrspace(1)* %a, <3 x half> addrspace(1)* %a,
<3 x half> addrspace(1)* %b) { <3 x half> addrspace(1)* %b) #0 {
entry: entry:
%a.val = load <3 x half>, <3 x half> addrspace(1)* %a %a.val = load <3 x half>, <3 x half> addrspace(1)* %a
%b.val = load <3 x half>, <3 x half> addrspace(1)* %b %b.val = load <3 x half>, <3 x half> addrspace(1)* %b
%r.val = call <3 x half> @llvm.maxnum.v3f16(<3 x half> %a.val, <3 x half> %b.val) %r.val = call <3 x half> @llvm.maxnum.v3f16(<3 x half> %a.val, <3 x half> %b.val)
store <3 x half> %r.val, <3 x half> addrspace(1)* %r store <3 x half> %r.val, <3 x half> addrspace(1)* %r
ret void ret void
} }
▲ Show 20 Lines • Show All 95 Lines • ▼ Show 20 Lines
; GFX9-NEXT: v_pk_max_f16 v1, v1, v0 ; GFX9-NEXT: v_pk_max_f16 v1, v1, v0
; GFX9-NEXT: v_pk_max_f16 v2, s6, s6 ; GFX9-NEXT: v_pk_max_f16 v2, s6, s6
; GFX9-NEXT: v_pk_max_f16 v0, s4, s4 ; GFX9-NEXT: v_pk_max_f16 v0, s4, s4
; GFX9-NEXT: v_pk_max_f16 v0, v0, v2 ; GFX9-NEXT: v_pk_max_f16 v0, v0, v2
; GFX9-NEXT: buffer_store_dwordx2 v[0:1], off, s[0:3], 0 ; GFX9-NEXT: buffer_store_dwordx2 v[0:1], off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
<4 x half> addrspace(1)* %r, <4 x half> addrspace(1)* %r,
<4 x half> addrspace(1)* %a, <4 x half> addrspace(1)* %a,
<4 x half> addrspace(1)* %b) { <4 x half> addrspace(1)* %b) #0 {
entry: entry:
%a.val = load <4 x half>, <4 x half> addrspace(1)* %a %a.val = load <4 x half>, <4 x half> addrspace(1)* %a
%b.val = load <4 x half>, <4 x half> addrspace(1)* %b %b.val = load <4 x half>, <4 x half> addrspace(1)* %b
%r.val = call <4 x half> @llvm.maxnum.v4f16(<4 x half> %a.val, <4 x half> %b.val) %r.val = call <4 x half> @llvm.maxnum.v4f16(<4 x half> %a.val, <4 x half> %b.val)
store <4 x half> %r.val, <4 x half> addrspace(1)* %r store <4 x half> %r.val, <4 x half> addrspace(1)* %r
ret void ret void
} }
▲ Show 20 Lines • Show All 74 Lines • ▼ Show 20 Lines
; GFX9-NEXT: s_waitcnt lgkmcnt(0) ; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: v_pk_max_f16 v0, s5, s5 ; GFX9-NEXT: v_pk_max_f16 v0, s5, s5
; GFX9-NEXT: v_pk_max_f16 v2, s4, s4 ; GFX9-NEXT: v_pk_max_f16 v2, s4, s4
; GFX9-NEXT: v_pk_max_f16 v1, v0, s8 ; GFX9-NEXT: v_pk_max_f16 v1, v0, s8
; GFX9-NEXT: v_pk_max_f16 v0, v2, s9 ; GFX9-NEXT: v_pk_max_f16 v0, v2, s9
; GFX9-NEXT: buffer_store_dwordx2 v[0:1], off, s[0:3], 0 ; GFX9-NEXT: buffer_store_dwordx2 v[0:1], off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
<4 x half> addrspace(1)* %r, <4 x half> addrspace(1)* %r,
<4 x half> addrspace(1)* %b) { <4 x half> addrspace(1)* %b) #0 {
entry: entry:
%b.val = load <4 x half>, <4 x half> addrspace(1)* %b %b.val = load <4 x half>, <4 x half> addrspace(1)* %b
%r.val = call <4 x half> @llvm.maxnum.v4f16(<4 x half> <half 8.0, half 2.0, half 3.0, half 4.0>, <4 x half> %b.val) %r.val = call <4 x half> @llvm.maxnum.v4f16(<4 x half> <half 8.0, half 2.0, half 3.0, half 4.0>, <4 x half> %b.val)
store <4 x half> %r.val, <4 x half> addrspace(1)* %r store <4 x half> %r.val, <4 x half> addrspace(1)* %r
ret void ret void
} }
attributes #0 = { "denormal-fp-math-f32"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/AMDGPU/llvm.minnum.f16.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc -mtriple=amdgcn-- -mcpu=tahiti -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=SI %s ; RUN: llc -mtriple=amdgcn-- -mcpu=tahiti -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=SI %s
; RUN: llc -mtriple=amdgcn-- -mcpu=fiji -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=VI %s ; RUN: llc -mtriple=amdgcn-- -mcpu=fiji -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=VI %s
; RUN: llc -mtriple=amdgcn-- -mcpu=gfx900 -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9 %s ; RUN: llc -mtriple=amdgcn-- -mcpu=gfx900 -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9 %s
declare half @llvm.minnum.f16(half %a, half %b) declare half @llvm.minnum.f16(half %a, half %b)
declare <2 x half> @llvm.minnum.v2f16(<2 x half> %a, <2 x half> %b) declare <2 x half> @llvm.minnum.v2f16(<2 x half> %a, <2 x half> %b)
declare <3 x half> @llvm.minnum.v3f16(<3 x half> %a, <3 x half> %b) declare <3 x half> @llvm.minnum.v3f16(<3 x half> %a, <3 x half> %b)
declare <4 x half> @llvm.minnum.v4f16(<4 x half> %a, <4 x half> %b) declare <4 x half> @llvm.minnum.v4f16(<4 x half> %a, <4 x half> %b)
define amdgpu_kernel void @minnum_f16_ieee( define amdgpu_kernel void @minnum_f16_ieee(
; SI-LABEL: minnum_f16_ieee: ; SI-LABEL: minnum_f16_ieee:
▲ Show 20 Lines • Show All 70 Lines • ▼ Show 20 Lines
; GFX9-NEXT: v_max_f16_e32 v0, v0, v0 ; GFX9-NEXT: v_max_f16_e32 v0, v0, v0
; GFX9-NEXT: s_waitcnt vmcnt(0) ; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_max_f16_e32 v1, v1, v1 ; GFX9-NEXT: v_max_f16_e32 v1, v1, v1
; GFX9-NEXT: v_min_f16_e32 v0, v0, v1 ; GFX9-NEXT: v_min_f16_e32 v0, v0, v1
; GFX9-NEXT: buffer_store_short v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_short v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
half addrspace(1)* %r, half addrspace(1)* %r,
half addrspace(1)* %a, half addrspace(1)* %a,
half addrspace(1)* %b) { half addrspace(1)* %b) #0 {
entry: entry:
%a.val = load volatile half, half addrspace(1)* %a %a.val = load volatile half, half addrspace(1)* %a
%b.val = load volatile half, half addrspace(1)* %b %b.val = load volatile half, half addrspace(1)* %b
%r.val = call half @llvm.minnum.f16(half %a.val, half %b.val) %r.val = call half @llvm.minnum.f16(half %a.val, half %b.val)
store half %r.val, half addrspace(1)* %r store half %r.val, half addrspace(1)* %r
ret void ret void
} }
define amdgpu_ps half @minnum_f16_no_ieee(half %a, half %b) { define amdgpu_ps half @minnum_f16_no_ieee(half %a, half %b) #0 {
; SI-LABEL: minnum_f16_no_ieee: ; SI-LABEL: minnum_f16_no_ieee:
; SI: ; %bb.0: ; SI: ; %bb.0:
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1 ; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0 ; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
; SI-NEXT: v_cvt_f32_f16_e32 v1, v1 ; SI-NEXT: v_cvt_f32_f16_e32 v1, v1
; SI-NEXT: v_cvt_f32_f16_e32 v0, v0 ; SI-NEXT: v_cvt_f32_f16_e32 v0, v0
; SI-NEXT: v_min_f32_e32 v0, v0, v1 ; SI-NEXT: v_min_f32_e32 v0, v0, v1
; SI-NEXT: ; return to shader part epilog ; SI-NEXT: ; return to shader part epilog
▲ Show 20 Lines • Show All 66 Lines • ▼ Show 20 Lines
; GFX9-NEXT: s_mov_b32 s7, s3 ; GFX9-NEXT: s_mov_b32 s7, s3
; GFX9-NEXT: buffer_load_ushort v0, off, s[4:7], 0 ; GFX9-NEXT: buffer_load_ushort v0, off, s[4:7], 0
; GFX9-NEXT: s_waitcnt vmcnt(0) ; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_max_f16_e32 v0, v0, v0 ; GFX9-NEXT: v_max_f16_e32 v0, v0, v0
; GFX9-NEXT: v_min_f16_e32 v0, 0x4200, v0 ; GFX9-NEXT: v_min_f16_e32 v0, 0x4200, v0
; GFX9-NEXT: buffer_store_short v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_short v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
half addrspace(1)* %r, half addrspace(1)* %r,
half addrspace(1)* %b) { half addrspace(1)* %b) #0 {
entry: entry:
%b.val = load half, half addrspace(1)* %b %b.val = load half, half addrspace(1)* %b
%r.val = call half @llvm.minnum.f16(half 3.0, half %b.val) %r.val = call half @llvm.minnum.f16(half 3.0, half %b.val)
store half %r.val, half addrspace(1)* %r store half %r.val, half addrspace(1)* %r
ret void ret void
} }
define amdgpu_kernel void @minnum_f16_imm_b( define amdgpu_kernel void @minnum_f16_imm_b(
▲ Show 20 Lines • Show All 51 Lines • ▼ Show 20 Lines
; GFX9-NEXT: s_mov_b32 s7, s3 ; GFX9-NEXT: s_mov_b32 s7, s3
; GFX9-NEXT: buffer_load_ushort v0, off, s[4:7], 0 ; GFX9-NEXT: buffer_load_ushort v0, off, s[4:7], 0
; GFX9-NEXT: s_waitcnt vmcnt(0) ; GFX9-NEXT: s_waitcnt vmcnt(0)
; GFX9-NEXT: v_max_f16_e32 v0, v0, v0 ; GFX9-NEXT: v_max_f16_e32 v0, v0, v0
; GFX9-NEXT: v_min_f16_e32 v0, 4.0, v0 ; GFX9-NEXT: v_min_f16_e32 v0, 4.0, v0
; GFX9-NEXT: buffer_store_short v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_short v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
half addrspace(1)* %r, half addrspace(1)* %r,
half addrspace(1)* %a) { half addrspace(1)* %a) #0 {
entry: entry:
%a.val = load half, half addrspace(1)* %a %a.val = load half, half addrspace(1)* %a
%r.val = call half @llvm.minnum.f16(half %a.val, half 4.0) %r.val = call half @llvm.minnum.f16(half %a.val, half 4.0)
store half %r.val, half addrspace(1)* %r store half %r.val, half addrspace(1)* %r
ret void ret void
} }
define amdgpu_kernel void @minnum_v2f16_ieee( define amdgpu_kernel void @minnum_v2f16_ieee(
▲ Show 20 Lines • Show All 66 Lines • ▼ Show 20 Lines
; GFX9-NEXT: s_waitcnt lgkmcnt(0) ; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: v_pk_max_f16 v1, s4, s4 ; GFX9-NEXT: v_pk_max_f16 v1, s4, s4
; GFX9-NEXT: v_pk_max_f16 v0, s5, s5 ; GFX9-NEXT: v_pk_max_f16 v0, s5, s5
; GFX9-NEXT: v_pk_min_f16 v0, v1, v0 ; GFX9-NEXT: v_pk_min_f16 v0, v1, v0
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
<2 x half> addrspace(1)* %r, <2 x half> addrspace(1)* %r,
<2 x half> addrspace(1)* %a, <2 x half> addrspace(1)* %a,
<2 x half> addrspace(1)* %b) { <2 x half> addrspace(1)* %b) #0 {
entry: entry:
%a.val = load <2 x half>, <2 x half> addrspace(1)* %a %a.val = load <2 x half>, <2 x half> addrspace(1)* %a
%b.val = load <2 x half>, <2 x half> addrspace(1)* %b %b.val = load <2 x half>, <2 x half> addrspace(1)* %b
%r.val = call <2 x half> @llvm.minnum.v2f16(<2 x half> %a.val, <2 x half> %b.val) %r.val = call <2 x half> @llvm.minnum.v2f16(<2 x half> %a.val, <2 x half> %b.val)
store <2 x half> %r.val, <2 x half> addrspace(1)* %r store <2 x half> %r.val, <2 x half> addrspace(1)* %r
ret void ret void
} }
define amdgpu_ps <2 x half> @minnum_v2f16_no_ieee(<2 x half> %a, <2 x half> %b) { define amdgpu_ps <2 x half> @minnum_v2f16_no_ieee(<2 x half> %a, <2 x half> %b) #0 {
; SI-LABEL: minnum_v2f16_no_ieee: ; SI-LABEL: minnum_v2f16_no_ieee:
; SI: ; %bb.0: ; SI: ; %bb.0:
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3 ; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2 ; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0 ; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
; SI-NEXT: v_cvt_f16_f32_e32 v1, v1 ; SI-NEXT: v_cvt_f16_f32_e32 v1, v1
; SI-NEXT: v_cvt_f32_f16_e32 v3, v3 ; SI-NEXT: v_cvt_f32_f16_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v2, v2 ; SI-NEXT: v_cvt_f32_f16_e32 v2, v2
▲ Show 20 Lines • Show All 72 Lines • ▼ Show 20 Lines
; GFX9-NEXT: s_mov_b32 s1, s5 ; GFX9-NEXT: s_mov_b32 s1, s5
; GFX9-NEXT: s_waitcnt lgkmcnt(0) ; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: v_pk_max_f16 v0, s4, s4 ; GFX9-NEXT: v_pk_max_f16 v0, s4, s4
; GFX9-NEXT: s_mov_b32 s4, 0x44004200 ; GFX9-NEXT: s_mov_b32 s4, 0x44004200
; GFX9-NEXT: v_pk_min_f16 v0, v0, s4 ; GFX9-NEXT: v_pk_min_f16 v0, v0, s4
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
<2 x half> addrspace(1)* %r, <2 x half> addrspace(1)* %r,
<2 x half> addrspace(1)* %b) { <2 x half> addrspace(1)* %b) #0 {
entry: entry:
%b.val = load <2 x half>, <2 x half> addrspace(1)* %b %b.val = load <2 x half>, <2 x half> addrspace(1)* %b
%r.val = call <2 x half> @llvm.minnum.v2f16(<2 x half> <half 3.0, half 4.0>, <2 x half> %b.val) %r.val = call <2 x half> @llvm.minnum.v2f16(<2 x half> <half 3.0, half 4.0>, <2 x half> %b.val)
store <2 x half> %r.val, <2 x half> addrspace(1)* %r store <2 x half> %r.val, <2 x half> addrspace(1)* %r
ret void ret void
} }
define amdgpu_kernel void @minnum_v2f16_imm_b( define amdgpu_kernel void @minnum_v2f16_imm_b(
▲ Show 20 Lines • Show All 50 Lines • ▼ Show 20 Lines
; GFX9-NEXT: s_mov_b32 s1, s5 ; GFX9-NEXT: s_mov_b32 s1, s5
; GFX9-NEXT: s_waitcnt lgkmcnt(0) ; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: v_pk_max_f16 v0, s4, s4 ; GFX9-NEXT: v_pk_max_f16 v0, s4, s4
; GFX9-NEXT: s_mov_b32 s4, 0x42004400 ; GFX9-NEXT: s_mov_b32 s4, 0x42004400
; GFX9-NEXT: v_pk_min_f16 v0, v0, s4 ; GFX9-NEXT: v_pk_min_f16 v0, v0, s4
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
<2 x half> addrspace(1)* %r, <2 x half> addrspace(1)* %r,
<2 x half> addrspace(1)* %a) { <2 x half> addrspace(1)* %a) #0 {
entry: entry:
%a.val = load <2 x half>, <2 x half> addrspace(1)* %a %a.val = load <2 x half>, <2 x half> addrspace(1)* %a
%r.val = call <2 x half> @llvm.minnum.v2f16(<2 x half> %a.val, <2 x half> <half 4.0, half 3.0>) %r.val = call <2 x half> @llvm.minnum.v2f16(<2 x half> %a.val, <2 x half> <half 4.0, half 3.0>)
store <2 x half> %r.val, <2 x half> addrspace(1)* %r store <2 x half> %r.val, <2 x half> addrspace(1)* %r
ret void ret void
} }
; FIXME: Scalarize with undef half ; FIXME: Scalarize with undef half
▲ Show 20 Lines • Show All 82 Lines • ▼ Show 20 Lines
; GFX9-NEXT: v_pk_max_f16 v2, s7, s7 ; GFX9-NEXT: v_pk_max_f16 v2, s7, s7
; GFX9-NEXT: v_pk_max_f16 v1, s5, s5 ; GFX9-NEXT: v_pk_max_f16 v1, s5, s5
; GFX9-NEXT: v_pk_min_f16 v1, v1, v2 ; GFX9-NEXT: v_pk_min_f16 v1, v1, v2
; GFX9-NEXT: buffer_store_short v1, off, s[0:3], 0 offset:4 ; GFX9-NEXT: buffer_store_short v1, off, s[0:3], 0 offset:4
; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0 ; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
<3 x half> addrspace(1)* %r, <3 x half> addrspace(1)* %r,
<3 x half> addrspace(1)* %a, <3 x half> addrspace(1)* %a,
<3 x half> addrspace(1)* %b) { <3 x half> addrspace(1)* %b) #0 {
entry: entry:
%a.val = load <3 x half>, <3 x half> addrspace(1)* %a %a.val = load <3 x half>, <3 x half> addrspace(1)* %a
%b.val = load <3 x half>, <3 x half> addrspace(1)* %b %b.val = load <3 x half>, <3 x half> addrspace(1)* %b
%r.val = call <3 x half> @llvm.minnum.v3f16(<3 x half> %a.val, <3 x half> %b.val) %r.val = call <3 x half> @llvm.minnum.v3f16(<3 x half> %a.val, <3 x half> %b.val)
store <3 x half> %r.val, <3 x half> addrspace(1)* %r store <3 x half> %r.val, <3 x half> addrspace(1)* %r
ret void ret void
} }
▲ Show 20 Lines • Show All 95 Lines • ▼ Show 20 Lines
; GFX9-NEXT: v_pk_min_f16 v1, v1, v0 ; GFX9-NEXT: v_pk_min_f16 v1, v1, v0
; GFX9-NEXT: v_pk_max_f16 v2, s6, s6 ; GFX9-NEXT: v_pk_max_f16 v2, s6, s6
; GFX9-NEXT: v_pk_max_f16 v0, s4, s4 ; GFX9-NEXT: v_pk_max_f16 v0, s4, s4
; GFX9-NEXT: v_pk_min_f16 v0, v0, v2 ; GFX9-NEXT: v_pk_min_f16 v0, v0, v2
; GFX9-NEXT: buffer_store_dwordx2 v[0:1], off, s[0:3], 0 ; GFX9-NEXT: buffer_store_dwordx2 v[0:1], off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
<4 x half> addrspace(1)* %r, <4 x half> addrspace(1)* %r,
<4 x half> addrspace(1)* %a, <4 x half> addrspace(1)* %a,
<4 x half> addrspace(1)* %b) { <4 x half> addrspace(1)* %b) #0 {
entry: entry:
%a.val = load <4 x half>, <4 x half> addrspace(1)* %a %a.val = load <4 x half>, <4 x half> addrspace(1)* %a
%b.val = load <4 x half>, <4 x half> addrspace(1)* %b %b.val = load <4 x half>, <4 x half> addrspace(1)* %b
%r.val = call <4 x half> @llvm.minnum.v4f16(<4 x half> %a.val, <4 x half> %b.val) %r.val = call <4 x half> @llvm.minnum.v4f16(<4 x half> %a.val, <4 x half> %b.val)
store <4 x half> %r.val, <4 x half> addrspace(1)* %r store <4 x half> %r.val, <4 x half> addrspace(1)* %r
ret void ret void
} }
▲ Show 20 Lines • Show All 74 Lines • ▼ Show 20 Lines
; GFX9-NEXT: s_waitcnt lgkmcnt(0) ; GFX9-NEXT: s_waitcnt lgkmcnt(0)
; GFX9-NEXT: v_pk_max_f16 v0, s5, s5 ; GFX9-NEXT: v_pk_max_f16 v0, s5, s5
; GFX9-NEXT: v_pk_max_f16 v2, s4, s4 ; GFX9-NEXT: v_pk_max_f16 v2, s4, s4
; GFX9-NEXT: v_pk_min_f16 v1, v0, s8 ; GFX9-NEXT: v_pk_min_f16 v1, v0, s8
; GFX9-NEXT: v_pk_min_f16 v0, v2, s9 ; GFX9-NEXT: v_pk_min_f16 v0, v2, s9
; GFX9-NEXT: buffer_store_dwordx2 v[0:1], off, s[0:3], 0 ; GFX9-NEXT: buffer_store_dwordx2 v[0:1], off, s[0:3], 0
; GFX9-NEXT: s_endpgm ; GFX9-NEXT: s_endpgm
<4 x half> addrspace(1)* %r, <4 x half> addrspace(1)* %r,
<4 x half> addrspace(1)* %b) { <4 x half> addrspace(1)* %b) #0 {
entry: entry:
%b.val = load <4 x half>, <4 x half> addrspace(1)* %b %b.val = load <4 x half>, <4 x half> addrspace(1)* %b
%r.val = call <4 x half> @llvm.minnum.v4f16(<4 x half> <half 8.0, half 2.0, half 3.0, half 4.0>, <4 x half> %b.val) %r.val = call <4 x half> @llvm.minnum.v4f16(<4 x half> <half 8.0, half 2.0, half 3.0, half 4.0>, <4 x half> %b.val)
store <4 x half> %r.val, <4 x half> addrspace(1)* %r store <4 x half> %r.val, <4 x half> addrspace(1)* %r
ret void ret void
} }
attributes #0 = { "denormal-fp-math-f32"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/AMDGPU/mad-combine.ll

; Make sure we still form mad even when unsafe math or fp-contract is allowed instead of fma. ; Make sure we still form mad even when unsafe math or fp-contract is allowed instead of fma.
; RUN: llc -march=amdgcn -mcpu=tahiti -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-STD -check-prefix=SI-STD-SAFE -check-prefix=FUNC %s ; RUN: llc -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-STD -check-prefix=SI-STD-SAFE -check-prefix=FUNC %s
; RUN: llc -march=amdgcn -mcpu=tahiti -mattr=-fp32-denormals -verify-machineinstrs -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-STD -check-prefix=SI-STD-SAFE -check-prefix=FUNC %s ; RUN: llc -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -verify-machineinstrs -fp-contract=fast < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-STD -check-prefix=SI-STD-SAFE -check-prefix=FUNC %s
; RUN: llc -march=amdgcn -mcpu=tahiti -mattr=-fp32-denormals -verify-machineinstrs -enable-unsafe-fp-math < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-STD -check-prefix=SI-STD-UNSAFE -check-prefix=FUNC %s ; RUN: llc -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=preserve-sign -verify-machineinstrs -enable-unsafe-fp-math < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-STD -check-prefix=SI-STD-UNSAFE -check-prefix=FUNC %s
; Make sure we don't form mad with denormals ; Make sure we don't form mad with denormals
; RUN: llc -march=amdgcn -mcpu=tahiti -mattr=+fp32-denormals -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-DENORM -check-prefix=SI-DENORM-FASTFMAF -check-prefix=FUNC %s ; RUN: llc -march=amdgcn -mcpu=tahiti -denormal-fp-math-f32=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-DENORM -check-prefix=SI-DENORM-FASTFMAF -check-prefix=FUNC %s
; RUN: llc -march=amdgcn -mcpu=verde -mattr=+fp32-denormals -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-DENORM -check-prefix=SI-DENORM-SLOWFMAF -check-prefix=FUNC %s ; RUN: llc -march=amdgcn -mcpu=verde -denormal-fp-math-f32=ieee -fp-contract=fast -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefix=SI -check-prefix=SI-DENORM -check-prefix=SI-DENORM-SLOWFMAF -check-prefix=FUNC %s
declare i32 @llvm.amdgcn.workitem.id.x() #0 declare i32 @llvm.amdgcn.workitem.id.x() #0
declare float @llvm.fabs.f32(float) #0 declare float @llvm.fabs.f32(float) #0
declare float @llvm.fma.f32(float, float, float) #0 declare float @llvm.fma.f32(float, float, float) #0
declare float @llvm.fmuladd.f32(float, float, float) #0 declare float @llvm.fmuladd.f32(float, float, float) #0
; (fadd (fmul x, y), z) -> (fma x, y, z) ; (fadd (fmul x, y), z) -> (fma x, y, z)
; FUNC-LABEL: {{^}}combine_to_mad_f32_0: ; FUNC-LABEL: {{^}}combine_to_mad_f32_0:
▲ Show 20 Lines • Show All 561 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/mad-mix-hi.ll

Show First 20 Lines • Show All 137 Lines • ▼ Show 20 Lines
declare half @llvm.minnum.f16(half, half) #1 declare half @llvm.minnum.f16(half, half) #1
declare half @llvm.maxnum.f16(half, half) #1 declare half @llvm.maxnum.f16(half, half) #1
declare float @llvm.minnum.f32(float, float) #1 declare float @llvm.minnum.f32(float, float) #1
declare float @llvm.maxnum.f32(float, float) #1 declare float @llvm.maxnum.f32(float, float) #1
declare float @llvm.fmuladd.f32(float, float, float) #1 declare float @llvm.fmuladd.f32(float, float, float) #1
declare <2 x float> @llvm.fmuladd.v2f32(<2 x float>, <2 x float>, <2 x float>) #1 declare <2 x float> @llvm.fmuladd.v2f32(<2 x float>, <2 x float>, <2 x float>) #1
attributes #0 = { nounwind "target-features"="-fp32-denormals" } attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind readnone speculatable } attributes #1 = { nounwind readnone speculatable }

llvm/test/CodeGen/AMDGPU/mad-mix-lo.ll

Show First 20 Lines • Show All 304 Lines • ▼ Show 20 Lines
declare <3 x float> @llvm.maxnum.v3f32(<3 x float>, <3 x float>) #1 declare <3 x float> @llvm.maxnum.v3f32(<3 x float>, <3 x float>) #1
declare <4 x float> @llvm.maxnum.v4f32(<4 x float>, <4 x float>) #1 declare <4 x float> @llvm.maxnum.v4f32(<4 x float>, <4 x float>) #1
declare float @llvm.fmuladd.f32(float, float, float) #1 declare float @llvm.fmuladd.f32(float, float, float) #1
declare <2 x float> @llvm.fmuladd.v2f32(<2 x float>, <2 x float>, <2 x float>) #1 declare <2 x float> @llvm.fmuladd.v2f32(<2 x float>, <2 x float>, <2 x float>) #1
declare <3 x float> @llvm.fmuladd.v3f32(<3 x float>, <3 x float>, <3 x float>) #1 declare <3 x float> @llvm.fmuladd.v3f32(<3 x float>, <3 x float>, <3 x float>) #1
declare <4 x float> @llvm.fmuladd.v4f32(<4 x float>, <4 x float>, <4 x float>) #1 declare <4 x float> @llvm.fmuladd.v4f32(<4 x float>, <4 x float>, <4 x float>) #1
attributes #0 = { nounwind "target-features"="-fp32-denormals" } attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind readnone speculatable } attributes #1 = { nounwind readnone speculatable }

llvm/test/CodeGen/AMDGPU/mad-mix.ll

Show First 20 Lines • Show All 551 Lines • ▼ Show 20 Lines
declare half @llvm.fabs.f16(half) #2 declare half @llvm.fabs.f16(half) #2
declare <2 x half> @llvm.fabs.v2f16(<2 x half>) #2 declare <2 x half> @llvm.fabs.v2f16(<2 x half>) #2
declare float @llvm.fabs.f32(float) #2 declare float @llvm.fabs.f32(float) #2
declare float @llvm.minnum.f32(float, float) #2 declare float @llvm.minnum.f32(float, float) #2
declare float @llvm.maxnum.f32(float, float) #2 declare float @llvm.maxnum.f32(float, float) #2
declare float @llvm.fmuladd.f32(float, float, float) #2 declare float @llvm.fmuladd.f32(float, float, float) #2
declare <2 x float> @llvm.fmuladd.v2f32(<2 x float>, <2 x float>, <2 x float>) #2 declare <2 x float> @llvm.fmuladd.v2f32(<2 x float>, <2 x float>, <2 x float>) #2
attributes #0 = { nounwind "target-features"="-fp32-denormals" } attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind "target-features"="+fp32-denormals" } attributes #1 = { nounwind "denormal-fp-math-f32"="ieee,ieee" }
attributes #2 = { nounwind readnone speculatable } attributes #2 = { nounwind readnone speculatable }

llvm/test/CodeGen/AMDGPU/madak.ll

; RUN: llc -march=amdgcn -mcpu=tahiti -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX6,GFX6_8_9,MAD %s ; RUN: llc -march=amdgcn -mcpu=tahiti -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX6,GFX6_8_9,MAD %s
; RUN: llc -march=amdgcn -mcpu=tonga -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX8,GFX6_8_9,GFX8_9,GFX8_9_10,MAD %s ; RUN: llc -march=amdgcn -mcpu=tonga -verify-machineinstrs < %s | FileCheck -check-prefixes=GCN,GFX8,GFX6_8_9,GFX8_9,GFX8_9_10,MAD %s
; RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=-fp32-denormals -verify-machineinstrs -amdgpu-enable-global-sgpr-addr < %s | FileCheck -check-prefixes=GCN,GFX9,GFX6_8_9,GFX8_9,GFX8_9_10,MAD %s ; RUN: llc -march=amdgcn -mcpu=gfx900 -verify-machineinstrs -amdgpu-enable-global-sgpr-addr < %s | FileCheck -check-prefixes=GCN,GFX9,GFX6_8_9,GFX8_9,GFX8_9_10,MAD %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-fp32-denormals -verify-machineinstrs -amdgpu-enable-global-sgpr-addr < %s | FileCheck -check-prefixes=GCN,GFX10,GFX8_9_10,GFX10-MAD %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -verify-machineinstrs -amdgpu-enable-global-sgpr-addr < %s | FileCheck -check-prefixes=GCN,GFX10,GFX8_9_10,GFX10-MAD %s
; RUN: llc -march=amdgcn -mcpu=gfx1010 -mattr=-fp32-denormals -verify-machineinstrs -fp-contract=fast -amdgpu-enable-global-sgpr-addr < %s | FileCheck -check-prefixes=GCN,GFX10,GFX8_9_10,FMA %s ; RUN: llc -march=amdgcn -mcpu=gfx1010 -verify-machineinstrs -fp-contract=fast -amdgpu-enable-global-sgpr-addr < %s | FileCheck -check-prefixes=GCN,GFX10,GFX8_9_10,FMA %s
declare i32 @llvm.amdgcn.workitem.id.x() nounwind readnone declare i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
declare float @llvm.fabs.f32(float) nounwind readnone declare float @llvm.fabs.f32(float) nounwind readnone
; GCN-LABEL: {{^}}madak_f32: ; GCN-LABEL: {{^}}madak_f32:
; GFX6: buffer_load_dword [[VA:v[0-9]+]] ; GFX6: buffer_load_dword [[VA:v[0-9]+]]
; GFX6: buffer_load_dword [[VB:v[0-9]+]] ; GFX6: buffer_load_dword [[VB:v[0-9]+]]
; GFX8: {{flat|global}}_load_dword [[VB:v[0-9]+]] ; GFX8: {{flat|global}}_load_dword [[VB:v[0-9]+]]
; GFX8: {{flat|global}}_load_dword [[VA:v[0-9]+]] ; GFX8: {{flat|global}}_load_dword [[VA:v[0-9]+]]
; GFX9: {{flat|global}}_load_dword [[VA:v[0-9]+]] ; GFX9: {{flat|global}}_load_dword [[VA:v[0-9]+]]
; GFX9: {{flat|global}}_load_dword [[VB:v[0-9]+]] ; GFX9: {{flat|global}}_load_dword [[VB:v[0-9]+]]
; GFX10: {{flat|global}}_load_dword [[VA:v[0-9]+]] ; GFX10: {{flat|global}}_load_dword [[VA:v[0-9]+]]
; GFX10: {{flat|global}}_load_dword [[VB:v[0-9]+]] ; GFX10: {{flat|global}}_load_dword [[VB:v[0-9]+]]
; MAD: v_madak_f32 {{v[0-9]+}}, [[VA]], [[VB]], 0x41200000 ; MAD: v_madak_f32 {{v[0-9]+}}, [[VA]], [[VB]], 0x41200000
; GFX10-MAD: v_madak_f32 {{v[0-9]+}}, [[VA]], [[VB]], 0x41200000 ; GFX10-MAD: v_madak_f32 {{v[0-9]+}}, [[VA]], [[VB]], 0x41200000
; FMA: v_fmaak_f32 {{v[0-9]+}}, [[VA]], [[VB]], 0x41200000 ; FMA: v_fmaak_f32 {{v[0-9]+}}, [[VA]], [[VB]], 0x41200000
define amdgpu_kernel void @madak_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float addrspace(1)* noalias %in.b) nounwind { define amdgpu_kernel void @madak_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float addrspace(1)* noalias %in.b) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid %in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid
%in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid %in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%a = load float, float addrspace(1)* %in.a.gep, align 4 %a = load float, float addrspace(1)* %in.a.gep, align 4
%b = load float, float addrspace(1)* %in.b.gep, align 4 %b = load float, float addrspace(1)* %in.b.gep, align 4
Show All 17 Lines
; GFX8_9_10: {{flat|global}}_load_dword [[VC:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}} ; GFX8_9_10: {{flat|global}}_load_dword [[VC:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}
; GFX6-DAG: v_mov_b32_e32 [[VK:v[0-9]+]], 0x41200000 ; GFX6-DAG: v_mov_b32_e32 [[VK:v[0-9]+]], 0x41200000
; GFX6_8_9-DAG: v_madak_f32 {{v[0-9]+}}, [[VA]], [[VB]], 0x41200000 ; GFX6_8_9-DAG: v_madak_f32 {{v[0-9]+}}, [[VA]], [[VB]], 0x41200000
; GFX10-MAD-DAG:v_madak_f32 {{v[0-9]+}}, [[VA]], [[VB]], 0x41200000 ; GFX10-MAD-DAG:v_madak_f32 {{v[0-9]+}}, [[VA]], [[VB]], 0x41200000
; FMA-DAG: v_fmaak_f32 {{v[0-9]+}}, [[VA]], [[VB]], 0x41200000 ; FMA-DAG: v_fmaak_f32 {{v[0-9]+}}, [[VA]], [[VB]], 0x41200000
; MAD-DAG: v_mac_f32_e32 [[VK]], [[VA]], [[VC]] ; MAD-DAG: v_mac_f32_e32 [[VK]], [[VA]], [[VC]]
; FMA-DAG: v_fmac_f32_e32 [[VK]], [[VA]], [[VC]] ; FMA-DAG: v_fmac_f32_e32 [[VK]], [[VA]], [[VC]]
; GCN: s_endpgm ; GCN: s_endpgm
define amdgpu_kernel void @madak_2_use_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @madak_2_use_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%in.gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid %in.gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid
%in.gep.1 = getelementptr float, float addrspace(1)* %in.gep.0, i32 1 %in.gep.1 = getelementptr float, float addrspace(1)* %in.gep.0, i32 1
%in.gep.2 = getelementptr float, float addrspace(1)* %in.gep.0, i32 2 %in.gep.2 = getelementptr float, float addrspace(1)* %in.gep.0, i32 2
%out.gep.0 = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
%out.gep.1 = getelementptr float, float addrspace(1)* %in.gep.0, i32 1 %out.gep.1 = getelementptr float, float addrspace(1)* %in.gep.0, i32 1
Show All 12 Linesdefine amdgpu_kernel void @madak_2_use_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
ret void ret void
} }
; GCN-LABEL: {{^}}madak_m_inline_imm_f32: ; GCN-LABEL: {{^}}madak_m_inline_imm_f32:
; GCN: {{buffer|flat|global}}_load_dword [[VA:v[0-9]+]] ; GCN: {{buffer|flat|global}}_load_dword [[VA:v[0-9]+]]
; MAD: v_madak_f32 {{v[0-9]+}}, 4.0, [[VA]], 0x41200000 ; MAD: v_madak_f32 {{v[0-9]+}}, 4.0, [[VA]], 0x41200000
; GFX10-MAD: v_madak_f32 {{v[0-9]+}}, 4.0, [[VA]], 0x41200000 ; GFX10-MAD: v_madak_f32 {{v[0-9]+}}, 4.0, [[VA]], 0x41200000
; FMA: v_fmaak_f32 {{v[0-9]+}}, 4.0, [[VA]], 0x41200000 ; FMA: v_fmaak_f32 {{v[0-9]+}}, 4.0, [[VA]], 0x41200000
define amdgpu_kernel void @madak_m_inline_imm_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a) nounwind { define amdgpu_kernel void @madak_m_inline_imm_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid %in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%a = load float, float addrspace(1)* %in.a.gep, align 4 %a = load float, float addrspace(1)* %in.a.gep, align 4
%mul = fmul float 4.0, %a %mul = fmul float 4.0, %a
%madak = fadd float %mul, 10.0 %madak = fadd float %mul, 10.0
Show All 11 Lines
; GFX8: {{flat|global}}_load_dword [[VA:v[0-9]+]] ; GFX8: {{flat|global}}_load_dword [[VA:v[0-9]+]]
; GFX9: {{flat|global}}_load_dword [[VA:v[0-9]+]] ; GFX9: {{flat|global}}_load_dword [[VA:v[0-9]+]]
; GFX9: {{flat|global}}_load_dword [[VB:v[0-9]+]] ; GFX9: {{flat|global}}_load_dword [[VB:v[0-9]+]]
; GFX10: {{flat|global}}_load_dword [[VA:v[0-9]+]] ; GFX10: {{flat|global}}_load_dword [[VA:v[0-9]+]]
; GFX10: {{flat|global}}_load_dword [[VB:v[0-9]+]] ; GFX10: {{flat|global}}_load_dword [[VB:v[0-9]+]]
; MAD: v_mad_f32 {{v[0-9]+}}, [[VA]], [[VB]], 4.0 ; MAD: v_mad_f32 {{v[0-9]+}}, [[VA]], [[VB]], 4.0
; GFX10-MAD: v_mad_f32 {{v[0-9]+}}, [[VA]], [[VB]], 4.0 ; GFX10-MAD: v_mad_f32 {{v[0-9]+}}, [[VA]], [[VB]], 4.0
; FMA: v_fma_f32 {{v[0-9]+}}, [[VA]], [[VB]], 4.0 ; FMA: v_fma_f32 {{v[0-9]+}}, [[VA]], [[VB]], 4.0
define amdgpu_kernel void @madak_inline_imm_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float addrspace(1)* noalias %in.b) nounwind { define amdgpu_kernel void @madak_inline_imm_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float addrspace(1)* noalias %in.b) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid %in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid
%in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid %in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%a = load float, float addrspace(1)* %in.a.gep, align 4 %a = load float, float addrspace(1)* %in.a.gep, align 4
%b = load float, float addrspace(1)* %in.b.gep, align 4 %b = load float, float addrspace(1)* %in.b.gep, align 4
%mul = fmul float %a, %b %mul = fmul float %a, %b
%madak = fadd float %mul, 4.0 %madak = fadd float %mul, 4.0
store float %madak, float addrspace(1)* %out.gep, align 4 store float %madak, float addrspace(1)* %out.gep, align 4
ret void ret void
} }
; We can't use an SGPR when forming madak ; We can't use an SGPR when forming madak
; GCN-LABEL: {{^}}s_v_madak_f32: ; GCN-LABEL: {{^}}s_v_madak_f32:
; GCN-DAG: s_load_dword [[SB:s[0-9]+]] ; GCN-DAG: s_load_dword [[SB:s[0-9]+]]
; GFX6_8_9-DAG: v_mov_b32_e32 [[VK:v[0-9]+]], 0x41200000 ; GFX6_8_9-DAG: v_mov_b32_e32 [[VK:v[0-9]+]], 0x41200000
; GCN-DAG: {{buffer|flat|global}}_load_dword{{(_addtid)?}} [[VA:v[0-9]+]] ; GCN-DAG: {{buffer|flat|global}}_load_dword{{(_addtid)?}} [[VA:v[0-9]+]]
; GCN-NOT: v_madak_f32 ; GCN-NOT: v_madak_f32
; GFX6_8_9: v_mac_f32_e32 [[VK]], [[SB]], [[VA]] ; GFX6_8_9: v_mac_f32_e32 [[VK]], [[SB]], [[VA]]
; GFX10-MAD: v_mad_f32 v{{[0-9]+}}, [[VA]], [[SB]], 0x41200000 ; GFX10-MAD: v_mad_f32 v{{[0-9]+}}, [[VA]], [[SB]], 0x41200000
; FMA: v_fma_f32 v{{[0-9]+}}, [[VA]], [[SB]], 0x41200000 ; FMA: v_fma_f32 v{{[0-9]+}}, [[VA]], [[SB]], 0x41200000
define amdgpu_kernel void @s_v_madak_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float %b) nounwind { define amdgpu_kernel void @s_v_madak_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float %b) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid %in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%a = load float, float addrspace(1)* %in.a.gep, align 4 %a = load float, float addrspace(1)* %in.a.gep, align 4
%mul = fmul float %a, %b %mul = fmul float %a, %b
%madak = fadd float %mul, 10.0 %madak = fadd float %mul, 10.0
store float %madak, float addrspace(1)* %out.gep, align 4 store float %madak, float addrspace(1)* %out.gep, align 4
ret void ret void
} }
; GCN-LABEL: @v_s_madak_f32 ; GCN-LABEL: @v_s_madak_f32
; GCN-DAG: s_load_dword [[SB:s[0-9]+]] ; GCN-DAG: s_load_dword [[SB:s[0-9]+]]
; GFX6_8_9-DAG: v_mov_b32_e32 [[VK:v[0-9]+]], 0x41200000 ; GFX6_8_9-DAG: v_mov_b32_e32 [[VK:v[0-9]+]], 0x41200000
; GCN-DAG: {{buffer|flat|global}}_load_dword{{(_addtid)?}} [[VA:v[0-9]+]] ; GCN-DAG: {{buffer|flat|global}}_load_dword{{(_addtid)?}} [[VA:v[0-9]+]]
; GFX6_8_9-NOT: v_madak_f32 ; GFX6_8_9-NOT: v_madak_f32
; GFX6_8_9: v_mac_f32_e32 [[VK]], [[SB]], [[VA]] ; GFX6_8_9: v_mac_f32_e32 [[VK]], [[SB]], [[VA]]
; GFX10-MAD: v_madak_f32 v{{[0-9]+}}, [[SB]], [[VA]], 0x41200000 ; GFX10-MAD: v_madak_f32 v{{[0-9]+}}, [[SB]], [[VA]], 0x41200000
; FMA: v_fmaak_f32 v{{[0-9]+}}, [[SB]], [[VA]], 0x41200000 ; FMA: v_fmaak_f32 v{{[0-9]+}}, [[SB]], [[VA]], 0x41200000
define amdgpu_kernel void @v_s_madak_f32(float addrspace(1)* noalias %out, float %a, float addrspace(1)* noalias %in.b) nounwind { define amdgpu_kernel void @v_s_madak_f32(float addrspace(1)* noalias %out, float %a, float addrspace(1)* noalias %in.b) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid %in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%b = load float, float addrspace(1)* %in.b.gep, align 4 %b = load float, float addrspace(1)* %in.b.gep, align 4
%mul = fmul float %a, %b %mul = fmul float %a, %b
%madak = fadd float %mul, 10.0 %madak = fadd float %mul, 10.0
store float %madak, float addrspace(1)* %out.gep, align 4 store float %madak, float addrspace(1)* %out.gep, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}s_s_madak_f32: ; GCN-LABEL: {{^}}s_s_madak_f32:
; GCN-NOT: v_madak_f32 ; GCN-NOT: v_madak_f32
; GFX8_9: v_mac_f32_e32 {{v[0-9]+}}, {{s[0-9]+}}, {{v[0-9]+}} ; GFX8_9: v_mac_f32_e32 {{v[0-9]+}}, {{s[0-9]+}}, {{v[0-9]+}}
; GFX10-MAD: v_mac_f32_e64 {{v[0-9]+}}, {{s[0-9]+}}, {{s[0-9]+}} ; GFX10-MAD: v_mac_f32_e64 {{v[0-9]+}}, {{s[0-9]+}}, {{s[0-9]+}}
; FMA: v_fmac_f32_e64 {{v[0-9]+}}, {{s[0-9]+}}, {{s[0-9]+}} ; FMA: v_fmac_f32_e64 {{v[0-9]+}}, {{s[0-9]+}}, {{s[0-9]+}}
define amdgpu_kernel void @s_s_madak_f32(float addrspace(1)* %out, float %a, float %b) nounwind { define amdgpu_kernel void @s_s_madak_f32(float addrspace(1)* %out, float %a, float %b) #0 {
%mul = fmul float %a, %b %mul = fmul float %a, %b
%madak = fadd float %mul, 10.0 %madak = fadd float %mul, 10.0
store float %madak, float addrspace(1)* %out, align 4 store float %madak, float addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}no_madak_src0_modifier_f32: ; GCN-LABEL: {{^}}no_madak_src0_modifier_f32:
; GFX6: buffer_load_dword [[VA:v[0-9]+]] ; GFX6: buffer_load_dword [[VA:v[0-9]+]]
; GFX6: buffer_load_dword [[VB:v[0-9]+]] ; GFX6: buffer_load_dword [[VB:v[0-9]+]]
; GFX8_9_10: {{flat|global}}_load_dword [[VB:v[0-9]+]] ; GFX8_9_10: {{flat|global}}_load_dword [[VB:v[0-9]+]]
; GFX8_9_10: {{flat|global}}_load_dword [[VA:v[0-9]+]] ; GFX8_9_10: {{flat|global}}_load_dword [[VA:v[0-9]+]]
; GFX6_8_9: v_mad_f32 {{v[0-9]+}}, |{{v[0-9]+}}|, {{v[0-9]+}}, {{[sv][0-9]+}} ; GFX6_8_9: v_mad_f32 {{v[0-9]+}}, |{{v[0-9]+}}|, {{v[0-9]+}}, {{[sv][0-9]+}}
; GFX10-MAD: v_mad_f32 {{v[0-9]+}}, |{{v[0-9]+}}|, {{v[0-9]+}}, 0x41200000 ; GFX10-MAD: v_mad_f32 {{v[0-9]+}}, |{{v[0-9]+}}|, {{v[0-9]+}}, 0x41200000
; FMA: v_fma_f32 {{v[0-9]+}}, |{{v[0-9]+}}|, {{v[0-9]+}}, 0x41200000 ; FMA: v_fma_f32 {{v[0-9]+}}, |{{v[0-9]+}}|, {{v[0-9]+}}, 0x41200000
; GCN: s_endpgm ; GCN: s_endpgm
define amdgpu_kernel void @no_madak_src0_modifier_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float addrspace(1)* noalias %in.b) nounwind { define amdgpu_kernel void @no_madak_src0_modifier_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float addrspace(1)* noalias %in.b) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid %in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid
%in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid %in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%a = load float, float addrspace(1)* %in.a.gep, align 4 %a = load float, float addrspace(1)* %in.a.gep, align 4
%b = load float, float addrspace(1)* %in.b.gep, align 4 %b = load float, float addrspace(1)* %in.b.gep, align 4
Show All 9 Lines
; GFX6: buffer_load_dword [[VA:v[0-9]+]] ; GFX6: buffer_load_dword [[VA:v[0-9]+]]
; GFX6: buffer_load_dword [[VB:v[0-9]+]] ; GFX6: buffer_load_dword [[VB:v[0-9]+]]
; GFX8_9_10: {{flat|global}}_load_dword [[VB:v[0-9]+]] ; GFX8_9_10: {{flat|global}}_load_dword [[VB:v[0-9]+]]
; GFX8_9_10: {{flat|global}}_load_dword [[VA:v[0-9]+]] ; GFX8_9_10: {{flat|global}}_load_dword [[VA:v[0-9]+]]
; GFX6_8_9: v_mad_f32 {{v[0-9]+}}, {{v[0-9]+}}, |{{v[0-9]+}}|, {{[sv][0-9]+}} ; GFX6_8_9: v_mad_f32 {{v[0-9]+}}, {{v[0-9]+}}, |{{v[0-9]+}}|, {{[sv][0-9]+}}
; GFX10-MAD: v_mad_f32 {{v[0-9]+}}, {{v[0-9]+}}, |{{v[0-9]+}}|, 0x41200000 ; GFX10-MAD: v_mad_f32 {{v[0-9]+}}, {{v[0-9]+}}, |{{v[0-9]+}}|, 0x41200000
; FMA: v_fma_f32 {{v[0-9]+}}, {{v[0-9]+}}, |{{v[0-9]+}}|, 0x41200000 ; FMA: v_fma_f32 {{v[0-9]+}}, {{v[0-9]+}}, |{{v[0-9]+}}|, 0x41200000
; GCN: s_endpgm ; GCN: s_endpgm
define amdgpu_kernel void @no_madak_src1_modifier_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float addrspace(1)* noalias %in.b) nounwind { define amdgpu_kernel void @no_madak_src1_modifier_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in.a, float addrspace(1)* noalias %in.b) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid %in.a.gep = getelementptr float, float addrspace(1)* %in.a, i32 %tid
%in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid %in.b.gep = getelementptr float, float addrspace(1)* %in.b, i32 %tid
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%a = load float, float addrspace(1)* %in.a.gep, align 4 %a = load float, float addrspace(1)* %in.a.gep, align 4
%b = load float, float addrspace(1)* %in.b.gep, align 4 %b = load float, float addrspace(1)* %in.b.gep, align 4
Show All 33 Linesbb4:
%vgpr = load volatile float, float addrspace(1)* undef %vgpr = load volatile float, float addrspace(1)* undef
%tmp0 = fmul float %sgpr0, 0.5 %tmp0 = fmul float %sgpr0, 0.5
%tmp1 = fadd float %tmp0, 42.0 %tmp1 = fadd float %tmp0, 42.0
%tmp2 = fmul float %tmp1, %vgpr %tmp2 = fmul float %tmp1, %vgpr
store volatile float %tmp2, float addrspace(1)* undef, align 4 store volatile float %tmp2, float addrspace(1)* undef, align 4
ret void ret void
} }
attributes #0 = { nounwind} attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/AMDGPU/madmk.ll

; RUN: llc -march=amdgcn -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=SI %s ; RUN: llc -march=amdgcn -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=SI %s
; XUN: llc -march=amdgcn -mcpu=tonga -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=VI %s ; XUN: llc -march=amdgcn -mcpu=tonga -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=VI %s
; FIXME: None of these trigger madmk emission anymore. It is still ; FIXME: None of these trigger madmk emission anymore. It is still
; possible, but requires the correct registers to be used which is ; possible, but requires the correct registers to be used which is
; hard to trigger. ; hard to trigger.
declare i32 @llvm.amdgcn.workitem.id.x() nounwind readnone declare i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
declare float @llvm.fabs.f32(float) nounwind readnone declare float @llvm.fabs.f32(float) nounwind readnone
; GCN-LABEL: {{^}}madmk_f32: ; GCN-LABEL: {{^}}madmk_f32:
; GCN-DAG: buffer_load_dword [[VA:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64{{$}} ; GCN-DAG: buffer_load_dword [[VA:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64{{$}}
; GCN-DAG: buffer_load_dword [[VB:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4 ; GCN-DAG: buffer_load_dword [[VB:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4
; GCN: v_mac_f32_e32 [[VB]], 0x41200000, [[VA]] ; GCN: v_mac_f32_e32 [[VB]], 0x41200000, [[VA]]
define amdgpu_kernel void @madmk_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @madmk_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid %gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid
%gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1 %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%a = load volatile float, float addrspace(1)* %gep.0, align 4 %a = load volatile float, float addrspace(1)* %gep.0, align 4
%b = load volatile float, float addrspace(1)* %gep.1, align 4 %b = load volatile float, float addrspace(1)* %gep.1, align 4
%mul = fmul float %a, 10.0 %mul = fmul float %a, 10.0
%madmk = fadd float %mul, %b %madmk = fadd float %mul, %b
store float %madmk, float addrspace(1)* %out.gep, align 4 store float %madmk, float addrspace(1)* %out.gep, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}madmk_2_use_f32: ; GCN-LABEL: {{^}}madmk_2_use_f32:
; GCN-DAG: buffer_load_dword [[VA:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64{{$}} ; GCN-DAG: buffer_load_dword [[VA:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64{{$}}
; GCN-DAG: buffer_load_dword [[VB:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4 ; GCN-DAG: buffer_load_dword [[VB:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4
; GCN-DAG: buffer_load_dword [[VC:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:8 ; GCN-DAG: buffer_load_dword [[VC:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:8
; GCN-DAG: s_mov_b32 [[SK:s[0-9]+]], 0x41200000 ; GCN-DAG: s_mov_b32 [[SK:s[0-9]+]], 0x41200000
; GCN-DAG: v_mac_f32_e32 [[VB]], [[SK]], [[VA]] ; GCN-DAG: v_mac_f32_e32 [[VB]], [[SK]], [[VA]]
; GCN-DAG: v_mac_f32_e32 [[VC]], [[SK]], [[VA]] ; GCN-DAG: v_mac_f32_e32 [[VC]], [[SK]], [[VA]]
; GCN: s_endpgm ; GCN: s_endpgm
define amdgpu_kernel void @madmk_2_use_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @madmk_2_use_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%in.gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid %in.gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid
%in.gep.1 = getelementptr float, float addrspace(1)* %in.gep.0, i32 1 %in.gep.1 = getelementptr float, float addrspace(1)* %in.gep.0, i32 1
%in.gep.2 = getelementptr float, float addrspace(1)* %in.gep.0, i32 2 %in.gep.2 = getelementptr float, float addrspace(1)* %in.gep.0, i32 2
%out.gep.0 = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep.0 = getelementptr float, float addrspace(1)* %out, i32 %tid
%out.gep.1 = getelementptr float, float addrspace(1)* %in.gep.0, i32 1 %out.gep.1 = getelementptr float, float addrspace(1)* %in.gep.0, i32 1
Show All 12 Linesdefine amdgpu_kernel void @madmk_2_use_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
ret void ret void
} }
; We don't get any benefit if the constant is an inline immediate. ; We don't get any benefit if the constant is an inline immediate.
; GCN-LABEL: {{^}}madmk_inline_imm_f32: ; GCN-LABEL: {{^}}madmk_inline_imm_f32:
; GCN-DAG: buffer_load_dword [[VA:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64{{$}} ; GCN-DAG: buffer_load_dword [[VA:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64{{$}}
; GCN-DAG: buffer_load_dword [[VB:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4 ; GCN-DAG: buffer_load_dword [[VB:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4
; GCN: v_mac_f32_e32 [[VB]], 4.0, [[VA]] ; GCN: v_mac_f32_e32 [[VB]], 4.0, [[VA]]
define amdgpu_kernel void @madmk_inline_imm_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @madmk_inline_imm_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid %gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid
%gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1 %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%a = load volatile float, float addrspace(1)* %gep.0, align 4 %a = load volatile float, float addrspace(1)* %gep.0, align 4
%b = load volatile float, float addrspace(1)* %gep.1, align 4 %b = load volatile float, float addrspace(1)* %gep.1, align 4
%mul = fmul float %a, 4.0 %mul = fmul float %a, 4.0
%madmk = fadd float %mul, %b %madmk = fadd float %mul, %b
store float %madmk, float addrspace(1)* %out.gep, align 4 store float %madmk, float addrspace(1)* %out.gep, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}s_s_madmk_f32: ; GCN-LABEL: {{^}}s_s_madmk_f32:
; GCN-NOT: v_madmk_f32 ; GCN-NOT: v_madmk_f32
; GCN: v_mac_f32_e32 ; GCN: v_mac_f32_e32
; GCN: s_endpgm ; GCN: s_endpgm
define amdgpu_kernel void @s_s_madmk_f32(float addrspace(1)* noalias %out, [8 x i32], float %a, [8 x i32], float %b) nounwind { define amdgpu_kernel void @s_s_madmk_f32(float addrspace(1)* noalias %out, [8 x i32], float %a, [8 x i32], float %b) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%mul = fmul float %a, 10.0 %mul = fmul float %a, 10.0
%madmk = fadd float %mul, %b %madmk = fadd float %mul, %b
store float %madmk, float addrspace(1)* %out.gep, align 4 store float %madmk, float addrspace(1)* %out.gep, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}v_s_madmk_f32: ; GCN-LABEL: {{^}}v_s_madmk_f32:
; GCN: s_load_dword [[SREG:s[0-9]+]] ; GCN: s_load_dword [[SREG:s[0-9]+]]
; GCN: buffer_load_dword [[VREG1:v[0-9]+]] ; GCN: buffer_load_dword [[VREG1:v[0-9]+]]
; GCN: v_mov_b32_e32 [[VREG2:v[0-9]+]], [[SREG]] ; GCN: v_mov_b32_e32 [[VREG2:v[0-9]+]], [[SREG]]
; GCN: v_mac_f32_e32 [[VREG2]], 0x41200000, [[VREG1]] ; GCN: v_mac_f32_e32 [[VREG2]], 0x41200000, [[VREG1]]
; GCN: s_endpgm ; GCN: s_endpgm
define amdgpu_kernel void @v_s_madmk_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in, float %b) nounwind { define amdgpu_kernel void @v_s_madmk_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in, float %b) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid %gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%a = load float, float addrspace(1)* %gep.0, align 4 %a = load float, float addrspace(1)* %gep.0, align 4
%mul = fmul float %a, 10.0 %mul = fmul float %a, 10.0
%madmk = fadd float %mul, %b %madmk = fadd float %mul, %b
store float %madmk, float addrspace(1)* %out.gep, align 4 store float %madmk, float addrspace(1)* %out.gep, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}scalar_vector_madmk_f32: ; GCN-LABEL: {{^}}scalar_vector_madmk_f32:
; GCN-NOT: v_madmk_f32 ; GCN-NOT: v_madmk_f32
; GCN: v_mac_f32_e32 ; GCN: v_mac_f32_e32
; GCN: s_endpgm ; GCN: s_endpgm
define amdgpu_kernel void @scalar_vector_madmk_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in, float %a) nounwind { define amdgpu_kernel void @scalar_vector_madmk_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in, float %a) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid %gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%b = load float, float addrspace(1)* %gep.0, align 4 %b = load float, float addrspace(1)* %gep.0, align 4
%mul = fmul float %a, 10.0 %mul = fmul float %a, 10.0
%madmk = fadd float %mul, %b %madmk = fadd float %mul, %b
store float %madmk, float addrspace(1)* %out.gep, align 4 store float %madmk, float addrspace(1)* %out.gep, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}no_madmk_src0_modifier_f32: ; GCN-LABEL: {{^}}no_madmk_src0_modifier_f32:
; GCN-DAG: buffer_load_dword [[VA:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64{{$}} ; GCN-DAG: buffer_load_dword [[VA:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64{{$}}
; GCN-DAG: buffer_load_dword [[VB:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4 ; GCN-DAG: buffer_load_dword [[VB:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4
; GCN-DAG: s_mov_b32 [[SK:s[0-9]+]], 0x41200000 ; GCN-DAG: s_mov_b32 [[SK:s[0-9]+]], 0x41200000
; GCN: v_mad_f32 {{v[0-9]+}}, |[[VA]]|, [[SK]], [[VB]] ; GCN: v_mad_f32 {{v[0-9]+}}, |[[VA]]|, [[SK]], [[VB]]
define amdgpu_kernel void @no_madmk_src0_modifier_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @no_madmk_src0_modifier_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid %gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid
%gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1 %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%a = load volatile float, float addrspace(1)* %gep.0, align 4 %a = load volatile float, float addrspace(1)* %gep.0, align 4
%b = load volatile float, float addrspace(1)* %gep.1, align 4 %b = load volatile float, float addrspace(1)* %gep.1, align 4
%a.fabs = call float @llvm.fabs.f32(float %a) nounwind readnone %a.fabs = call float @llvm.fabs.f32(float %a) nounwind readnone
%mul = fmul float %a.fabs, 10.0 %mul = fmul float %a.fabs, 10.0
%madmk = fadd float %mul, %b %madmk = fadd float %mul, %b
store float %madmk, float addrspace(1)* %out.gep, align 4 store float %madmk, float addrspace(1)* %out.gep, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}no_madmk_src2_modifier_f32: ; GCN-LABEL: {{^}}no_madmk_src2_modifier_f32:
; GCN-DAG: buffer_load_dword [[VA:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64{{$}} ; GCN-DAG: buffer_load_dword [[VA:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64{{$}}
; GCN-DAG: buffer_load_dword [[VB:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4 ; GCN-DAG: buffer_load_dword [[VB:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4
; GCN: v_mad_f32 {{v[0-9]+}}, {{v[0-9]+}}, {{[sv][0-9]+}}, |{{v[0-9]+}}| ; GCN: v_mad_f32 {{v[0-9]+}}, {{v[0-9]+}}, {{[sv][0-9]+}}, |{{v[0-9]+}}|
define amdgpu_kernel void @no_madmk_src2_modifier_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @no_madmk_src2_modifier_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid %gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid
%gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1 %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%a = load volatile float, float addrspace(1)* %gep.0, align 4 %a = load volatile float, float addrspace(1)* %gep.0, align 4
%b = load volatile float, float addrspace(1)* %gep.1, align 4 %b = load volatile float, float addrspace(1)* %gep.1, align 4
%b.fabs = call float @llvm.fabs.f32(float %b) nounwind readnone %b.fabs = call float @llvm.fabs.f32(float %b) nounwind readnone
%mul = fmul float %a, 10.0 %mul = fmul float %a, 10.0
%madmk = fadd float %mul, %b.fabs %madmk = fadd float %mul, %b.fabs
store float %madmk, float addrspace(1)* %out.gep, align 4 store float %madmk, float addrspace(1)* %out.gep, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}madmk_add_inline_imm_f32: ; GCN-LABEL: {{^}}madmk_add_inline_imm_f32:
; GCN: buffer_load_dword [[A:v[0-9]+]] ; GCN: buffer_load_dword [[A:v[0-9]+]]
; GCN: s_mov_b32 [[SK:s[0-9]+]], 0x41200000 ; GCN: s_mov_b32 [[SK:s[0-9]+]], 0x41200000
; GCN: v_mad_f32 {{v[0-9]+}}, [[A]], [[SK]], 2.0 ; GCN: v_mad_f32 {{v[0-9]+}}, [[A]], [[SK]], 2.0
define amdgpu_kernel void @madmk_add_inline_imm_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @madmk_add_inline_imm_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
%tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = tail call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid %gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%a = load float, float addrspace(1)* %gep.0, align 4 %a = load float, float addrspace(1)* %gep.0, align 4
%mul = fmul float %a, 10.0 %mul = fmul float %a, 10.0
%madmk = fadd float %mul, 2.0 %madmk = fadd float %mul, 2.0
store float %madmk, float addrspace(1)* %out.gep, align 4 store float %madmk, float addrspace(1)* %out.gep, align 4
ret void ret void
} }
; SI-LABEL: {{^}}kill_madmk_verifier_error: ; SI-LABEL: {{^}}kill_madmk_verifier_error:
; SI: s_or_b64 ; SI: s_or_b64
; SI: s_xor_b64 ; SI: s_xor_b64
; SI: v_mac_f32_e32 {{v[0-9]+}}, 0x472aee8c, {{v[0-9]+}} ; SI: v_mac_f32_e32 {{v[0-9]+}}, 0x472aee8c, {{v[0-9]+}}
define amdgpu_kernel void @kill_madmk_verifier_error() nounwind { define amdgpu_kernel void @kill_madmk_verifier_error() #0 {
bb: bb:
br label %bb2 br label %bb2
bb1: ; preds = %bb2 bb1: ; preds = %bb2
ret void ret void
bb2: ; preds = %bb6, %bb bb2: ; preds = %bb6, %bb
%tmp = phi float [ undef, %bb ], [ %tmp8, %bb6 ] %tmp = phi float [ undef, %bb ], [ %tmp8, %bb6 ]
%tid = call i32 @llvm.amdgcn.mbcnt.lo(i32 -1, i32 0) #1 %tid = call i32 @llvm.amdgcn.mbcnt.lo(i32 -1, i32 0) #1
%f_tid = bitcast i32 %tid to float %f_tid = bitcast i32 %tid to float
%tmp3 = fsub float %f_tid, %tmp %tmp3 = fsub float %f_tid, %tmp
%tmp5 = fcmp oeq float %tmp3, 1.000000e+04 %tmp5 = fcmp oeq float %tmp3, 1.000000e+04
br i1 %tmp5, label %bb1, label %bb6 br i1 %tmp5, label %bb1, label %bb6
bb6: ; preds = %bb2 bb6: ; preds = %bb2
%tmp7 = fmul float %tmp, 0x40E55DD180000000 %tmp7 = fmul float %tmp, 0x40E55DD180000000
%tmp8 = fadd float %tmp7, %tmp %tmp8 = fadd float %tmp7, %tmp
br label %bb2 br label %bb2
} }
declare i32 @llvm.amdgcn.mbcnt.lo(i32, i32) #1 declare i32 @llvm.amdgcn.mbcnt.lo(i32, i32) #1
attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind readnone } attributes #1 = { nounwind readnone }

llvm/test/CodeGen/AMDGPU/mul24-pass-ordering.ll

Show First 20 Lines • Show All 251 Lines • ▼ Show 20 Lines; CHECK-NOT: mul i32
call void @foo(i32 %mul2) call void @foo(i32 %mul2)
ret void ret void
} }
declare void @foo(i32) #0 declare void @foo(i32) #0
declare float @llvm.fmuladd.f32(float, float, float) #1 declare float @llvm.fmuladd.f32(float, float, float) #1
attributes #0 = { nounwind willreturn } attributes #0 = { nounwind willreturn "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind readnone speculatable } attributes #1 = { nounwind readnone speculatable }

llvm/test/CodeGen/AMDGPU/omod.ll

Show First 20 Lines • Show All 269 Lines • ▼ Show 20 Lines
declare double @llvm.fabs.f64(double) #1 declare double @llvm.fabs.f64(double) #1
declare double @llvm.minnum.f64(double, double) #1 declare double @llvm.minnum.f64(double, double) #1
declare double @llvm.maxnum.f64(double, double) #1 declare double @llvm.maxnum.f64(double, double) #1
declare half @llvm.fabs.f16(half) #1 declare half @llvm.fabs.f16(half) #1
declare half @llvm.minnum.f16(half, half) #1 declare half @llvm.minnum.f16(half, half) #1
declare half @llvm.maxnum.f16(half, half) #1 declare half @llvm.maxnum.f16(half, half) #1
declare void @llvm.dbg.value(metadata, i64, metadata, metadata) #1 declare void @llvm.dbg.value(metadata, i64, metadata, metadata) #1
attributes #0 = { nounwind "target-features"="-fp32-denormals" "no-signed-zeros-fp-math"="true" } attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" "no-signed-zeros-fp-math"="true" }
attributes #1 = { nounwind readnone } attributes #1 = { nounwind readnone }
attributes #2 = { nounwind "target-features"="+fp32-denormals" "no-signed-zeros-fp-math"="true" } attributes #2 = { nounwind "denormal-fp-math-f32"="ieee,ieee" "no-signed-zeros-fp-math"="true" }
attributes #3 = { nounwind "target-features"="-fp64-fp16-denormals" "no-signed-zeros-fp-math"="true" } attributes #3 = { nounwind "denormal-fp-math"="preserve-sign,preserve-sign" "no-signed-zeros-fp-math"="true" }
attributes #4 = { nounwind "no-signed-zeros-fp-math"="false" } attributes #4 = { nounwind "no-signed-zeros-fp-math"="false" }
!llvm.dbg.cu = !{!0} !llvm.dbg.cu = !{!0}
!llvm.module.flags = !{!2, !3} !llvm.module.flags = !{!2, !3}
!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, isOptimized: true, runtimeVersion: 0, emissionKind: NoDebug) !0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, isOptimized: true, runtimeVersion: 0, emissionKind: NoDebug)
!1 = !DIFile(filename: "/tmp/foo.cl", directory: "/dev/null") !1 = !DIFile(filename: "/tmp/foo.cl", directory: "/dev/null")
!2 = !{i32 2, !"Dwarf Version", i32 4} !2 = !{i32 2, !"Dwarf Version", i32 4}
!3 = !{i32 2, !"Debug Info Version", i32 3} !3 = !{i32 2, !"Debug Info Version", i32 3}
!4 = !DILocalVariable(name: "add", arg: 1, scope: !5, file: !1, line: 1) !4 = !DILocalVariable(name: "add", arg: 1, scope: !5, file: !1, line: 1)
!5 = distinct !DISubprogram(name: "foo", scope: !1, file: !1, line: 1, type: !6, isLocal: false, isDefinition: true, scopeLine: 2, flags: DIFlagPrototyped, isOptimized: true, unit: !0) !5 = distinct !DISubprogram(name: "foo", scope: !1, file: !1, line: 1, type: !6, isLocal: false, isDefinition: true, scopeLine: 2, flags: DIFlagPrototyped, isOptimized: true, unit: !0)
!6 = !DISubroutineType(types: !7) !6 = !DISubroutineType(types: !7)
!7 = !{null, !8} !7 = !{null, !8}
!8 = !DIBasicType(name: "float", size: 32, align: 32) !8 = !DIBasicType(name: "float", size: 32, align: 32)
!9 = !DIExpression() !9 = !DIExpression()
!10 = !DILocation(line: 1, column: 42, scope: !5) !10 = !DILocation(line: 1, column: 42, scope: !5)

llvm/test/CodeGen/AMDGPU/operand-folding.ll

; RUN: llc -march=amdgcn -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck %s ; RUN: llc -march=amdgcn -verify-machineinstrs < %s | FileCheck %s
; CHECK-LABEL: {{^}}fold_sgpr: ; CHECK-LABEL: {{^}}fold_sgpr:
; CHECK: v_add_i32_e32 v{{[0-9]+}}, vcc, s ; CHECK: v_add_i32_e32 v{{[0-9]+}}, vcc, s
define amdgpu_kernel void @fold_sgpr(i32 addrspace(1)* %out, i32 %fold) { define amdgpu_kernel void @fold_sgpr(i32 addrspace(1)* %out, i32 %fold) #1 {
entry: entry:
%tmp0 = icmp ne i32 %fold, 0 %tmp0 = icmp ne i32 %fold, 0
br i1 %tmp0, label %if, label %endif br i1 %tmp0, label %if, label %endif
if: if:
%id = call i32 @llvm.amdgcn.workitem.id.x() %id = call i32 @llvm.amdgcn.workitem.id.x()
%offset = add i32 %fold, %id %offset = add i32 %fold, %id
%tmp1 = getelementptr i32, i32 addrspace(1)* %out, i32 %offset %tmp1 = getelementptr i32, i32 addrspace(1)* %out, i32 %offset
store i32 0, i32 addrspace(1)* %tmp1 store i32 0, i32 addrspace(1)* %tmp1
br label %endif br label %endif
endif: endif:
ret void ret void
} }
; CHECK-LABEL: {{^}}fold_imm: ; CHECK-LABEL: {{^}}fold_imm:
; CHECK: v_or_b32_e32 v{{[0-9]+}}, 5 ; CHECK: v_or_b32_e32 v{{[0-9]+}}, 5
define amdgpu_kernel void @fold_imm(i32 addrspace(1)* %out, i32 %cmp) { define amdgpu_kernel void @fold_imm(i32 addrspace(1)* %out, i32 %cmp) #1 {
entry: entry:
%fold = add i32 3, 2 %fold = add i32 3, 2
%tmp0 = icmp ne i32 %cmp, 0 %tmp0 = icmp ne i32 %cmp, 0
br i1 %tmp0, label %if, label %endif br i1 %tmp0, label %if, label %endif
if: if:
%id = call i32 @llvm.amdgcn.workitem.id.x() %id = call i32 @llvm.amdgcn.workitem.id.x()
%val = or i32 %id, %fold %val = or i32 %id, %fold
Show All 9 Lines
; FIXME: It would be better if we could use v_add here and drop the extra ; FIXME: It would be better if we could use v_add here and drop the extra
; v_mov_b32 instructions. ; v_mov_b32 instructions.
; CHECK-DAG: s_add_u32 [[LO:s[0-9]+]], s{{[0-9]+}}, 1 ; CHECK-DAG: s_add_u32 [[LO:s[0-9]+]], s{{[0-9]+}}, 1
; CHECK-DAG: s_addc_u32 [[HI:s[0-9]+]], s{{[0-9]+}}, 0 ; CHECK-DAG: s_addc_u32 [[HI:s[0-9]+]], s{{[0-9]+}}, 0
; CHECK-DAG: v_mov_b32_e32 v[[VLO:[0-9]+]], [[LO]] ; CHECK-DAG: v_mov_b32_e32 v[[VLO:[0-9]+]], [[LO]]
; CHECK-DAG: v_mov_b32_e32 v[[VHI:[0-9]+]], [[HI]] ; CHECK-DAG: v_mov_b32_e32 v[[VHI:[0-9]+]], [[HI]]
; CHECK: buffer_store_dwordx2 v{{\[}}[[VLO]]:[[VHI]]{{\]}}, ; CHECK: buffer_store_dwordx2 v{{\[}}[[VLO]]:[[VHI]]{{\]}},
define amdgpu_kernel void @fold_64bit_constant_add(i64 addrspace(1)* %out, i32 %cmp, i64 %val) { define amdgpu_kernel void @fold_64bit_constant_add(i64 addrspace(1)* %out, i32 %cmp, i64 %val) #1 {
entry: entry:
%tmp0 = add i64 %val, 1 %tmp0 = add i64 %val, 1
store i64 %tmp0, i64 addrspace(1)* %out store i64 %tmp0, i64 addrspace(1)* %out
ret void ret void
} }
; Inline constants should always be folded. ; Inline constants should always be folded.
; CHECK-LABEL: {{^}}vector_inline: ; CHECK-LABEL: {{^}}vector_inline:
; CHECK: v_xor_b32_e32 v{{[0-9]+}}, 5, v{{[0-9]+}} ; CHECK: v_xor_b32_e32 v{{[0-9]+}}, 5, v{{[0-9]+}}
; CHECK: v_xor_b32_e32 v{{[0-9]+}}, 5, v{{[0-9]+}} ; CHECK: v_xor_b32_e32 v{{[0-9]+}}, 5, v{{[0-9]+}}
; CHECK: v_xor_b32_e32 v{{[0-9]+}}, 5, v{{[0-9]+}} ; CHECK: v_xor_b32_e32 v{{[0-9]+}}, 5, v{{[0-9]+}}
; CHECK: v_xor_b32_e32 v{{[0-9]+}}, 5, v{{[0-9]+}} ; CHECK: v_xor_b32_e32 v{{[0-9]+}}, 5, v{{[0-9]+}}
define amdgpu_kernel void @vector_inline(<4 x i32> addrspace(1)* %out) { define amdgpu_kernel void @vector_inline(<4 x i32> addrspace(1)* %out) #1 {
entry: entry:
%tmp0 = call i32 @llvm.amdgcn.workitem.id.x() %tmp0 = call i32 @llvm.amdgcn.workitem.id.x()
%tmp1 = add i32 %tmp0, 1 %tmp1 = add i32 %tmp0, 1
%tmp2 = add i32 %tmp0, 2 %tmp2 = add i32 %tmp0, 2
%tmp3 = add i32 %tmp0, 3 %tmp3 = add i32 %tmp0, 3
%vec0 = insertelement <4 x i32> undef, i32 %tmp0, i32 0 %vec0 = insertelement <4 x i32> undef, i32 %tmp0, i32 0
%vec1 = insertelement <4 x i32> %vec0, i32 %tmp1, i32 1 %vec1 = insertelement <4 x i32> %vec0, i32 %tmp1, i32 1
%vec2 = insertelement <4 x i32> %vec1, i32 %tmp2, i32 2 %vec2 = insertelement <4 x i32> %vec1, i32 %tmp2, i32 2
%vec3 = insertelement <4 x i32> %vec2, i32 %tmp3, i32 3 %vec3 = insertelement <4 x i32> %vec2, i32 %tmp3, i32 3
%tmp4 = xor <4 x i32> <i32 5, i32 5, i32 5, i32 5>, %vec3 %tmp4 = xor <4 x i32> <i32 5, i32 5, i32 5, i32 5>, %vec3
store <4 x i32> %tmp4, <4 x i32> addrspace(1)* %out store <4 x i32> %tmp4, <4 x i32> addrspace(1)* %out
ret void ret void
} }
; Immediates with one use should be folded ; Immediates with one use should be folded
; CHECK-LABEL: {{^}}imm_one_use: ; CHECK-LABEL: {{^}}imm_one_use:
; CHECK: v_xor_b32_e32 v{{[0-9]+}}, 0x64, v{{[0-9]+}} ; CHECK: v_xor_b32_e32 v{{[0-9]+}}, 0x64, v{{[0-9]+}}
define amdgpu_kernel void @imm_one_use(i32 addrspace(1)* %out) { define amdgpu_kernel void @imm_one_use(i32 addrspace(1)* %out) #1 {
entry: entry:
%tmp0 = call i32 @llvm.amdgcn.workitem.id.x() %tmp0 = call i32 @llvm.amdgcn.workitem.id.x()
%tmp1 = xor i32 %tmp0, 100 %tmp1 = xor i32 %tmp0, 100
store i32 %tmp1, i32 addrspace(1)* %out store i32 %tmp1, i32 addrspace(1)* %out
ret void ret void
} }
; CHECK-LABEL: {{^}}vector_imm: ; CHECK-LABEL: {{^}}vector_imm:
; CHECK: s_movk_i32 [[IMM:s[0-9]+]], 0x64 ; CHECK: s_movk_i32 [[IMM:s[0-9]+]], 0x64
; CHECK: v_xor_b32_e32 v{{[0-9]}}, [[IMM]], v{{[0-9]}} ; CHECK: v_xor_b32_e32 v{{[0-9]}}, [[IMM]], v{{[0-9]}}
; CHECK: v_xor_b32_e32 v{{[0-9]}}, [[IMM]], v{{[0-9]}} ; CHECK: v_xor_b32_e32 v{{[0-9]}}, [[IMM]], v{{[0-9]}}
; CHECK: v_xor_b32_e32 v{{[0-9]}}, [[IMM]], v{{[0-9]}} ; CHECK: v_xor_b32_e32 v{{[0-9]}}, [[IMM]], v{{[0-9]}}
; CHECK: v_xor_b32_e32 v{{[0-9]}}, [[IMM]], v{{[0-9]}} ; CHECK: v_xor_b32_e32 v{{[0-9]}}, [[IMM]], v{{[0-9]}}
define amdgpu_kernel void @vector_imm(<4 x i32> addrspace(1)* %out) { define amdgpu_kernel void @vector_imm(<4 x i32> addrspace(1)* %out) #1 {
entry: entry:
%tmp0 = call i32 @llvm.amdgcn.workitem.id.x() %tmp0 = call i32 @llvm.amdgcn.workitem.id.x()
%tmp1 = add i32 %tmp0, 1 %tmp1 = add i32 %tmp0, 1
%tmp2 = add i32 %tmp0, 2 %tmp2 = add i32 %tmp0, 2
%tmp3 = add i32 %tmp0, 3 %tmp3 = add i32 %tmp0, 3
%vec0 = insertelement <4 x i32> undef, i32 %tmp0, i32 0 %vec0 = insertelement <4 x i32> undef, i32 %tmp0, i32 0
%vec1 = insertelement <4 x i32> %vec0, i32 %tmp1, i32 1 %vec1 = insertelement <4 x i32> %vec0, i32 %tmp1, i32 1
%vec2 = insertelement <4 x i32> %vec1, i32 %tmp2, i32 2 %vec2 = insertelement <4 x i32> %vec1, i32 %tmp2, i32 2
%vec3 = insertelement <4 x i32> %vec2, i32 %tmp3, i32 3 %vec3 = insertelement <4 x i32> %vec2, i32 %tmp3, i32 3
%tmp4 = xor <4 x i32> <i32 100, i32 100, i32 100, i32 100>, %vec3 %tmp4 = xor <4 x i32> <i32 100, i32 100, i32 100, i32 100>, %vec3
store <4 x i32> %tmp4, <4 x i32> addrspace(1)* %out store <4 x i32> %tmp4, <4 x i32> addrspace(1)* %out
ret void ret void
} }
; A subregister use operand should not be tied. ; A subregister use operand should not be tied.
; CHECK-LABEL: {{^}}no_fold_tied_subregister: ; CHECK-LABEL: {{^}}no_fold_tied_subregister:
; CHECK: buffer_load_dwordx2 v{{\[}}[[LO:[0-9]+]]:[[HI:[0-9]+]]{{\]}} ; CHECK: buffer_load_dwordx2 v{{\[}}[[LO:[0-9]+]]:[[HI:[0-9]+]]{{\]}}
; CHECK: v_mac_f32_e32 v[[LO]], 0x41200000, v[[HI]] ; CHECK: v_mac_f32_e32 v[[LO]], 0x41200000, v[[HI]]
; CHECK: buffer_store_dword v[[LO]] ; CHECK: buffer_store_dword v[[LO]]
define amdgpu_kernel void @no_fold_tied_subregister() { define amdgpu_kernel void @no_fold_tied_subregister() #1 {
%tmp1 = load volatile <2 x float>, <2 x float> addrspace(1)* undef %tmp1 = load volatile <2 x float>, <2 x float> addrspace(1)* undef
%tmp2 = extractelement <2 x float> %tmp1, i32 0 %tmp2 = extractelement <2 x float> %tmp1, i32 0
%tmp3 = extractelement <2 x float> %tmp1, i32 1 %tmp3 = extractelement <2 x float> %tmp1, i32 1
%tmp4 = fmul float %tmp3, 10.0 %tmp4 = fmul float %tmp3, 10.0
%tmp5 = fadd float %tmp4, %tmp2 %tmp5 = fadd float %tmp4, %tmp2
store volatile float %tmp5, float addrspace(1)* undef store volatile float %tmp5, float addrspace(1)* undef
ret void ret void
} }
; There should be exact one folding on the same operand. ; There should be exact one folding on the same operand.
; CHECK-LABEL: {{^}}no_extra_fold_on_same_opnd ; CHECK-LABEL: {{^}}no_extra_fold_on_same_opnd
; CHECK: v_xor_b32_e32 v{{[0-9]+}}, 0, v{{[0-9]+}} ; CHECK: v_xor_b32_e32 v{{[0-9]+}}, 0, v{{[0-9]+}}
; CHECK: v_xor_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; CHECK: v_xor_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
define void @no_extra_fold_on_same_opnd() { define void @no_extra_fold_on_same_opnd() #1 {
entry: entry:
%s0 = load i32, i32 addrspace(5)* undef, align 4 %s0 = load i32, i32 addrspace(5)* undef, align 4
%s0.i64= zext i32 %s0 to i64 %s0.i64= zext i32 %s0 to i64
br label %for.body.i.i br label %for.body.i.i
for.body.i.i: for.body.i.i:
%s1 = load i32, i32 addrspace(1)* undef, align 8 %s1 = load i32, i32 addrspace(1)* undef, align 8
%s1.i64 = sext i32 %s1 to i64 %s1.i64 = sext i32 %s1 to i64
%xor = xor i64 %s1.i64, %s0.i64 %xor = xor i64 %s1.i64, %s0.i64
%flag = icmp ult i64 %xor, 8 %flag = icmp ult i64 %xor, 8
br i1 %flag, label %if.then, label %if.else br i1 %flag, label %if.then, label %if.else
if.then: if.then:
unreachable unreachable
if.else: if.else:
unreachable unreachable
} }
declare i32 @llvm.amdgcn.workitem.id.x() #0 declare i32 @llvm.amdgcn.workitem.id.x() #0
attributes #0 = { nounwind readnone } attributes #0 = { nounwind readnone }
attributes #1 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/AMDGPU/rcp-pattern.ll

; RUN: llc -march=amdgcn -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=FUNC %s ; RUN: llc -march=amdgcn -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=FUNC %s
; RUN: llc -march=amdgcn -mcpu=tonga -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=FUNC %s ; RUN: llc -march=amdgcn -mcpu=tonga -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=GCN -check-prefix=FUNC %s
; RUN: llc -march=r600 -mcpu=cypress -verify-machineinstrs < %s | FileCheck -check-prefix=EG -check-prefix=FUNC %s ; RUN: llc -march=r600 -mcpu=cypress -verify-machineinstrs < %s | FileCheck -check-prefix=EG -check-prefix=FUNC %s
; RUN: llc -march=r600 -mcpu=cayman -verify-machineinstrs < %s | FileCheck -check-prefix=EG -check-prefix=FUNC %s ; RUN: llc -march=r600 -mcpu=cayman -verify-machineinstrs < %s | FileCheck -check-prefix=EG -check-prefix=FUNC %s
; FUNC-LABEL: {{^}}rcp_pat_f32: ; FUNC-LABEL: {{^}}rcp_pat_f32:
; GCN: s_load_dword [[SRC:s[0-9]+]] ; GCN: s_load_dword [[SRC:s[0-9]+]]
; GCN: v_rcp_f32_e32 [[RCP:v[0-9]+]], [[SRC]] ; GCN: v_rcp_f32_e32 [[RCP:v[0-9]+]], [[SRC]]
; GCN: buffer_store_dword [[RCP]] ; GCN: buffer_store_dword [[RCP]]
▲ Show 20 Lines • Show All 134 Lines • ▼ Show 20 Linesdefine amdgpu_kernel void @div_arcp_neg_k_x_pat_f32(float addrspace(1)* %out) #0 {
%rcp = fdiv arcp float %x, -10.0 %rcp = fdiv arcp float %x, -10.0
store float %rcp, float addrspace(1)* %out, align 4 store float %rcp, float addrspace(1)* %out, align 4
ret void ret void
} }
declare float @llvm.fabs.f32(float) #1 declare float @llvm.fabs.f32(float) #1
declare float @llvm.sqrt.f32(float) #1 declare float @llvm.sqrt.f32(float) #1
attributes #0 = { nounwind "unsafe-fp-math"="false" } attributes #0 = { nounwind "unsafe-fp-math"="false" "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind readnone } attributes #1 = { nounwind readnone }
attributes #2 = { nounwind "unsafe-fp-math"="true" } attributes #2 = { nounwind "unsafe-fp-math"="true" "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
!0 = !{float 2.500000e+00} !0 = !{float 2.500000e+00}

llvm/test/CodeGen/AMDGPU/rcp_iflag.ll

; RUN: llc -march=amdgcn -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck --check-prefix=GCN %s ; RUN: llc -march=amdgcn -verify-machineinstrs < %s | FileCheck --check-prefix=GCN %s
; GCN-LABEL: {{^}}rcp_uint: ; GCN-LABEL: {{^}}rcp_uint:
; GCN: v_rcp_iflag_f32_e32 ; GCN: v_rcp_iflag_f32_e32
define amdgpu_kernel void @rcp_uint(i32 addrspace(1)* %in, float addrspace(1)* %out) { define amdgpu_kernel void @rcp_uint(i32 addrspace(1)* %in, float addrspace(1)* %out) #0 {
%load = load i32, i32 addrspace(1)* %in, align 4 %load = load i32, i32 addrspace(1)* %in, align 4
%cvt = uitofp i32 %load to float %cvt = uitofp i32 %load to float
%div = fdiv float 1.000000e+00, %cvt %div = fdiv float 1.000000e+00, %cvt
store float %div, float addrspace(1)* %out, align 4 store float %div, float addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}rcp_sint: ; GCN-LABEL: {{^}}rcp_sint:
; GCN: v_rcp_iflag_f32_e32 ; GCN: v_rcp_iflag_f32_e32
define amdgpu_kernel void @rcp_sint(i32 addrspace(1)* %in, float addrspace(1)* %out) { define amdgpu_kernel void @rcp_sint(i32 addrspace(1)* %in, float addrspace(1)* %out) #0 {
%load = load i32, i32 addrspace(1)* %in, align 4 %load = load i32, i32 addrspace(1)* %in, align 4
%cvt = sitofp i32 %load to float %cvt = sitofp i32 %load to float
%div = fdiv float 1.000000e+00, %cvt %div = fdiv float 1.000000e+00, %cvt
store float %div, float addrspace(1)* %out, align 4 store float %div, float addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}rcp_uint_denorm:
; GCN-NOT: v_rcp_iflag_f32
define amdgpu_kernel void @rcp_uint_denorm(i32 addrspace(1)* %in, float addrspace(1)* %out) #1 {
%load = load i32, i32 addrspace(1)* %in, align 4
%cvt = uitofp i32 %load to float
%div = fdiv float 1.000000e+00, %cvt
store float %div, float addrspace(1)* %out, align 4
ret void
}
; GCN-LABEL: {{^}}rcp_sint_denorm:
; GCN-NOT: v_rcp_iflag_f32
define amdgpu_kernel void @rcp_sint_denorm(i32 addrspace(1)* %in, float addrspace(1)* %out) #1 {
%load = load i32, i32 addrspace(1)* %in, align 4
%cvt = sitofp i32 %load to float
%div = fdiv float 1.000000e+00, %cvt
store float %div, float addrspace(1)* %out, align 4
ret void
}
attributes #0 = { "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { "denormal-fp-math-f32"="ieee,ieee" }

llvm/test/CodeGen/AMDGPU/rsq.ll

; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mattr=-fp32-denormals -verify-machineinstrs -enable-unsafe-fp-math < %s | FileCheck -check-prefix=SI-UNSAFE -check-prefix=SI %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -verify-machineinstrs -enable-unsafe-fp-math < %s | FileCheck -check-prefix=SI-UNSAFE -check-prefix=SI %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -check-prefix=SI-SAFE -check-prefix=SI %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -verify-machineinstrs < %s | FileCheck -check-prefix=SI-SAFE -check-prefix=SI %s
declare i32 @llvm.amdgcn.workitem.id.x() nounwind readnone declare i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
declare float @llvm.sqrt.f32(float) nounwind readnone declare float @llvm.sqrt.f32(float) nounwind readnone
declare double @llvm.sqrt.f64(double) nounwind readnone declare double @llvm.sqrt.f64(double) nounwind readnone
; SI-LABEL: {{^}}rsq_f32: ; SI-LABEL: {{^}}rsq_f32:
; SI: v_rsq_f32_e32 ; SI: v_rsq_f32_e32
; SI: s_endpgm ; SI: s_endpgm
define amdgpu_kernel void @rsq_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @rsq_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
%val = load float, float addrspace(1)* %in, align 4 %val = load float, float addrspace(1)* %in, align 4
%sqrt = call float @llvm.sqrt.f32(float %val) nounwind readnone %sqrt = call float @llvm.sqrt.f32(float %val) nounwind readnone
%div = fdiv float 1.0, %sqrt %div = fdiv float 1.0, %sqrt
store float %div, float addrspace(1)* %out, align 4 store float %div, float addrspace(1)* %out, align 4
ret void ret void
} }
; SI-LABEL: {{^}}rsq_f64: ; SI-LABEL: {{^}}rsq_f64:
; SI-UNSAFE: v_rsq_f64_e32 ; SI-UNSAFE: v_rsq_f64_e32
; SI-SAFE: v_sqrt_f64_e32 ; SI-SAFE: v_sqrt_f64_e32
; SI: s_endpgm ; SI: s_endpgm
define amdgpu_kernel void @rsq_f64(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @rsq_f64(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #0 {
%val = load double, double addrspace(1)* %in, align 4 %val = load double, double addrspace(1)* %in, align 4
%sqrt = call double @llvm.sqrt.f64(double %val) nounwind readnone %sqrt = call double @llvm.sqrt.f64(double %val) nounwind readnone
%div = fdiv double 1.0, %sqrt %div = fdiv double 1.0, %sqrt
store double %div, double addrspace(1)* %out, align 4 store double %div, double addrspace(1)* %out, align 4
ret void ret void
} }
; SI-LABEL: {{^}}rsq_f32_sgpr: ; SI-LABEL: {{^}}rsq_f32_sgpr:
; SI: v_rsq_f32_e32 {{v[0-9]+}}, {{s[0-9]+}} ; SI: v_rsq_f32_e32 {{v[0-9]+}}, {{s[0-9]+}}
; SI: s_endpgm ; SI: s_endpgm
define amdgpu_kernel void @rsq_f32_sgpr(float addrspace(1)* noalias %out, float %val) nounwind { define amdgpu_kernel void @rsq_f32_sgpr(float addrspace(1)* noalias %out, float %val) #0 {
%sqrt = call float @llvm.sqrt.f32(float %val) nounwind readnone %sqrt = call float @llvm.sqrt.f32(float %val) nounwind readnone
%div = fdiv float 1.0, %sqrt %div = fdiv float 1.0, %sqrt
store float %div, float addrspace(1)* %out, align 4 store float %div, float addrspace(1)* %out, align 4
ret void ret void
} }
; Recognize that this is rsqrt(a) * rcp(b) * c, ; Recognize that this is rsqrt(a) * rcp(b) * c,
; not 1 / ( 1 / sqrt(a)) * rcp(b) * c. ; not 1 / ( 1 / sqrt(a)) * rcp(b) * c.
; SI-LABEL: @rsqrt_fmul ; SI-LABEL: @rsqrt_fmul
; SI-DAG: buffer_load_dword [[A:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64{{$}} ; SI-DAG: buffer_load_dword [[A:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64{{$}}
; SI-DAG: buffer_load_dword [[B:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4 ; SI-DAG: buffer_load_dword [[B:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:4
; SI-DAG: buffer_load_dword [[C:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:8 ; SI-DAG: buffer_load_dword [[C:v[0-9]+]], {{v\[[0-9]+:[0-9]+\]}}, {{s\[[0-9]+:[0-9]+\]}}, 0 addr64 offset:8
; SI-UNSAFE-DAG: v_rsq_f32_e32 [[RSQA:v[0-9]+]], [[A]] ; SI-UNSAFE-DAG: v_rsq_f32_e32 [[RSQA:v[0-9]+]], [[A]]
; SI-UNSAFE-DAG: v_rcp_f32_e32 [[RCPB:v[0-9]+]], [[B]] ; SI-UNSAFE-DAG: v_rcp_f32_e32 [[RCPB:v[0-9]+]], [[B]]
; SI-UNSAFE-DAG: v_mul_f32_e32 [[TMP:v[0-9]+]], [[RCPB]], [[RSQA]] ; SI-UNSAFE-DAG: v_mul_f32_e32 [[TMP:v[0-9]+]], [[RCPB]], [[RSQA]]
; SI-UNSAFE: v_mul_f32_e32 [[RESULT:v[0-9]+]], [[C]], [[TMP]] ; SI-UNSAFE: v_mul_f32_e32 [[RESULT:v[0-9]+]], [[C]], [[TMP]]
; SI-UNSAFE: buffer_store_dword [[RESULT]] ; SI-UNSAFE: buffer_store_dword [[RESULT]]
; SI-SAFE-NOT: v_rsq_f32 ; SI-SAFE-NOT: v_rsq_f32
; SI: s_endpgm ; SI: s_endpgm
define amdgpu_kernel void @rsqrt_fmul(float addrspace(1)* %out, float addrspace(1)* %in) { define amdgpu_kernel void @rsqrt_fmul(float addrspace(1)* %out, float addrspace(1)* %in) #0 {
%tid = call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone %tid = call i32 @llvm.amdgcn.workitem.id.x() nounwind readnone
%out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid %out.gep = getelementptr float, float addrspace(1)* %out, i32 %tid
%gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid %gep.0 = getelementptr float, float addrspace(1)* %in, i32 %tid
%gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1 %gep.1 = getelementptr float, float addrspace(1)* %gep.0, i32 1
%gep.2 = getelementptr float, float addrspace(1)* %gep.0, i32 2 %gep.2 = getelementptr float, float addrspace(1)* %gep.0, i32 2
%a = load volatile float, float addrspace(1)* %gep.0 %a = load volatile float, float addrspace(1)* %gep.0
%b = load volatile float, float addrspace(1)* %gep.1 %b = load volatile float, float addrspace(1)* %gep.1
Show All 9 Lines
; SI-LABEL: {{^}}neg_rsq_f32: ; SI-LABEL: {{^}}neg_rsq_f32:
; SI-SAFE: v_sqrt_f32_e32 [[SQRT:v[0-9]+]], v{{[0-9]+}} ; SI-SAFE: v_sqrt_f32_e32 [[SQRT:v[0-9]+]], v{{[0-9]+}}
; SI-SAFE: v_rcp_f32_e64 [[RSQ:v[0-9]+]], -[[SQRT]] ; SI-SAFE: v_rcp_f32_e64 [[RSQ:v[0-9]+]], -[[SQRT]]
; SI-SAFE: buffer_store_dword [[RSQ]] ; SI-SAFE: buffer_store_dword [[RSQ]]
; SI-UNSAFE: v_rsq_f32_e32 [[RSQ:v[0-9]+]], v{{[0-9]+}} ; SI-UNSAFE: v_rsq_f32_e32 [[RSQ:v[0-9]+]], v{{[0-9]+}}
; SI-UNSAFE: v_xor_b32_e32 [[NEG_RSQ:v[0-9]+]], 0x80000000, [[RSQ]] ; SI-UNSAFE: v_xor_b32_e32 [[NEG_RSQ:v[0-9]+]], 0x80000000, [[RSQ]]
; SI-UNSAFE: buffer_store_dword [[NEG_RSQ]] ; SI-UNSAFE: buffer_store_dword [[NEG_RSQ]]
define amdgpu_kernel void @neg_rsq_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @neg_rsq_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
%val = load float, float addrspace(1)* %in, align 4 %val = load float, float addrspace(1)* %in, align 4
%sqrt = call float @llvm.sqrt.f32(float %val) %sqrt = call float @llvm.sqrt.f32(float %val)
%div = fdiv float -1.0, %sqrt %div = fdiv float -1.0, %sqrt
store float %div, float addrspace(1)* %out, align 4 store float %div, float addrspace(1)* %out, align 4
ret void ret void
} }
; SI-LABEL: {{^}}neg_rsq_f64: ; SI-LABEL: {{^}}neg_rsq_f64:
; SI-SAFE: v_sqrt_f64_e32 ; SI-SAFE: v_sqrt_f64_e32
; SI-SAFE: v_div_scale_f64 ; SI-SAFE: v_div_scale_f64
; SI-UNSAFE: v_sqrt_f64_e32 [[SQRT:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}} ; SI-UNSAFE: v_sqrt_f64_e32 [[SQRT:v\[[0-9]+:[0-9]+\]]], v{{\[[0-9]+:[0-9]+\]}}
; SI-UNSAFE: v_rcp_f64_e64 [[RCP:v\[[0-9]+:[0-9]+\]]], -[[SQRT]] ; SI-UNSAFE: v_rcp_f64_e64 [[RCP:v\[[0-9]+:[0-9]+\]]], -[[SQRT]]
; SI-UNSAFE: buffer_store_dwordx2 [[RCP]] ; SI-UNSAFE: buffer_store_dwordx2 [[RCP]]
define amdgpu_kernel void @neg_rsq_f64(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @neg_rsq_f64(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #0 {
%val = load double, double addrspace(1)* %in, align 4 %val = load double, double addrspace(1)* %in, align 4
%sqrt = call double @llvm.sqrt.f64(double %val) %sqrt = call double @llvm.sqrt.f64(double %val)
%div = fdiv double -1.0, %sqrt %div = fdiv double -1.0, %sqrt
store double %div, double addrspace(1)* %out, align 4 store double %div, double addrspace(1)* %out, align 4
ret void ret void
} }
; SI-LABEL: {{^}}neg_rsq_neg_f32: ; SI-LABEL: {{^}}neg_rsq_neg_f32:
; SI-SAFE: v_sqrt_f32_e64 [[SQRT:v[0-9]+]], -v{{[0-9]+}} ; SI-SAFE: v_sqrt_f32_e64 [[SQRT:v[0-9]+]], -v{{[0-9]+}}
; SI-SAFE: v_rcp_f32_e64 [[RSQ:v[0-9]+]], -[[SQRT]] ; SI-SAFE: v_rcp_f32_e64 [[RSQ:v[0-9]+]], -[[SQRT]]
; SI-SAFE: buffer_store_dword [[RSQ]] ; SI-SAFE: buffer_store_dword [[RSQ]]
; SI-UNSAFE: v_rsq_f32_e64 [[RSQ:v[0-9]+]], -v{{[0-9]+}} ; SI-UNSAFE: v_rsq_f32_e64 [[RSQ:v[0-9]+]], -v{{[0-9]+}}
; SI-UNSAFE: v_xor_b32_e32 [[NEG_RSQ:v[0-9]+]], 0x80000000, [[RSQ]] ; SI-UNSAFE: v_xor_b32_e32 [[NEG_RSQ:v[0-9]+]], 0x80000000, [[RSQ]]
; SI-UNSAFE: buffer_store_dword [[NEG_RSQ]] ; SI-UNSAFE: buffer_store_dword [[NEG_RSQ]]
define amdgpu_kernel void @neg_rsq_neg_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @neg_rsq_neg_f32(float addrspace(1)* noalias %out, float addrspace(1)* noalias %in) #0 {
%val = load float, float addrspace(1)* %in, align 4 %val = load float, float addrspace(1)* %in, align 4
%val.fneg = fsub float -0.0, %val %val.fneg = fsub float -0.0, %val
%sqrt = call float @llvm.sqrt.f32(float %val.fneg) %sqrt = call float @llvm.sqrt.f32(float %val.fneg)
%div = fdiv float -1.0, %sqrt %div = fdiv float -1.0, %sqrt
store float %div, float addrspace(1)* %out, align 4 store float %div, float addrspace(1)* %out, align 4
ret void ret void
} }
; SI-LABEL: {{^}}neg_rsq_neg_f64: ; SI-LABEL: {{^}}neg_rsq_neg_f64:
; SI-SAFE: v_sqrt_f64_e64 v{{\[[0-9]+:[0-9]+\]}}, -v{{\[[0-9]+:[0-9]+\]}} ; SI-SAFE: v_sqrt_f64_e64 v{{\[[0-9]+:[0-9]+\]}}, -v{{\[[0-9]+:[0-9]+\]}}
; SI-SAFE: v_div_scale_f64 ; SI-SAFE: v_div_scale_f64
; SI-UNSAFE: v_sqrt_f64_e64 [[SQRT:v\[[0-9]+:[0-9]+\]]], -v{{\[[0-9]+:[0-9]+\]}} ; SI-UNSAFE: v_sqrt_f64_e64 [[SQRT:v\[[0-9]+:[0-9]+\]]], -v{{\[[0-9]+:[0-9]+\]}}
; SI-UNSAFE: v_rcp_f64_e64 [[RCP:v\[[0-9]+:[0-9]+\]]], -[[SQRT]] ; SI-UNSAFE: v_rcp_f64_e64 [[RCP:v\[[0-9]+:[0-9]+\]]], -[[SQRT]]
; SI-UNSAFE: buffer_store_dwordx2 [[RCP]] ; SI-UNSAFE: buffer_store_dwordx2 [[RCP]]
define amdgpu_kernel void @neg_rsq_neg_f64(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) nounwind { define amdgpu_kernel void @neg_rsq_neg_f64(double addrspace(1)* noalias %out, double addrspace(1)* noalias %in) #0 {
%val = load double, double addrspace(1)* %in, align 4 %val = load double, double addrspace(1)* %in, align 4
%val.fneg = fsub double -0.0, %val %val.fneg = fsub double -0.0, %val
%sqrt = call double @llvm.sqrt.f64(double %val.fneg) %sqrt = call double @llvm.sqrt.f64(double %val.fneg)
%div = fdiv double -1.0, %sqrt %div = fdiv double -1.0, %sqrt
store double %div, double addrspace(1)* %out, align 4 store double %div, double addrspace(1)* %out, align 4
ret void ret void
} }
attributes #0 = { nounwind "denormal-fp-math-f32"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/AMDGPU/sdivrem64.ll

;RUN: llc -march=amdgcn -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck --check-prefix=SI --check-prefix=GCN --check-prefix=FUNC %s ;RUN: llc -march=amdgcn -verify-machineinstrs < %s | FileCheck --check-prefix=SI --check-prefix=GCN --check-prefix=FUNC %s
;RUN: llc -march=amdgcn -mcpu=tonga -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck --check-prefix=VI --check-prefix=GCN --check-prefix=FUNC %s ;RUN: llc -march=amdgcn -mcpu=tonga -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck --check-prefix=VI --check-prefix=GCN --check-prefix=FUNC %s
;RUN: llc -march=r600 -mcpu=redwood < %s | FileCheck --check-prefix=EG --check-prefix=FUNC %s ;RUN: llc -march=r600 -mcpu=redwood < %s | FileCheck --check-prefix=EG --check-prefix=FUNC %s
;FUNC-LABEL: {{^}}s_test_sdiv: ;FUNC-LABEL: {{^}}s_test_sdiv:
;EG: RECIP_UINT ;EG: RECIP_UINT
;EG: LSHL {{.*}}, 1, ;EG: LSHL {{.*}}, 1,
;EG: BFE_UINT ;EG: BFE_UINT
;EG: BFE_UINT ;EG: BFE_UINT
;EG: BFE_UINT ;EG: BFE_UINT
Show All 27 Lines
;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0x4f800000, ;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0x4f800000,
;GCN: v_rcp_f32_e32 ;GCN: v_rcp_f32_e32
;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x5f7ffffc ;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x5f7ffffc
;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x2f800000 ;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x2f800000
;GCN: v_trunc_f32_e32 ;GCN: v_trunc_f32_e32
;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0xcf800000 ;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0xcf800000
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @s_test_sdiv(i64 addrspace(1)* %out, i64 %x, i64 %y) { define amdgpu_kernel void @s_test_sdiv(i64 addrspace(1)* %out, i64 %x, i64 %y) #0 {
%result = sdiv i64 %x, %y %result = sdiv i64 %x, %y
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
;FUNC-LABEL: {{^}}s_test_srem: ;FUNC-LABEL: {{^}}s_test_srem:
;EG: RECIP_UINT ;EG: RECIP_UINT
;EG: BFE_UINT ;EG: BFE_UINT
Show All 30 Lines
;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0x4f800000, ;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0x4f800000,
;GCN: v_rcp_f32_e32 ;GCN: v_rcp_f32_e32
;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x5f7ffffc ;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x5f7ffffc
;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x2f800000 ;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x2f800000
;GCN: v_trunc_f32_e32 ;GCN: v_trunc_f32_e32
;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0xcf800000 ;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0xcf800000
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @s_test_srem(i64 addrspace(1)* %out, i64 %x, i64 %y) { define amdgpu_kernel void @s_test_srem(i64 addrspace(1)* %out, i64 %x, i64 %y) #0 {
%result = urem i64 %x, %y %result = urem i64 %x, %y
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
;FUNC-LABEL: {{^}}test_sdiv3264: ;FUNC-LABEL: {{^}}test_sdiv3264:
;EG: RECIP_UINT ;EG: RECIP_UINT
;EG-NOT: BFE_UINT ;EG-NOT: BFE_UINT
;GCN-NOT: s_bfe_u32 ;GCN-NOT: s_bfe_u32
;GCN-NOT: v_mad_f32 ;GCN-NOT: v_mad_f32
;SI-NOT: v_lshr_b64 ;SI-NOT: v_lshr_b64
;VI-NOT: v_lshrrev_b64 ;VI-NOT: v_lshrrev_b64
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @test_sdiv3264(i64 addrspace(1)* %out, i64 %x, i64 %y) { define amdgpu_kernel void @test_sdiv3264(i64 addrspace(1)* %out, i64 %x, i64 %y) #0 {
%1 = ashr i64 %x, 33 %1 = ashr i64 %x, 33
%2 = ashr i64 %y, 33 %2 = ashr i64 %y, 33
%result = sdiv i64 %1, %2 %result = sdiv i64 %1, %2
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
;FUNC-LABEL: {{^}}test_srem3264: ;FUNC-LABEL: {{^}}test_srem3264:
;EG: RECIP_UINT ;EG: RECIP_UINT
;EG-NOT: BFE_UINT ;EG-NOT: BFE_UINT
;GCN-NOT: s_bfe_u32 ;GCN-NOT: s_bfe_u32
;GCN-NOT: v_mad_f32 ;GCN-NOT: v_mad_f32
;SI-NOT: v_lshr_b64 ;SI-NOT: v_lshr_b64
;VI-NOT: v_lshrrev_b64 ;VI-NOT: v_lshrrev_b64
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @test_srem3264(i64 addrspace(1)* %out, i64 %x, i64 %y) { define amdgpu_kernel void @test_srem3264(i64 addrspace(1)* %out, i64 %x, i64 %y) #0 {
%1 = ashr i64 %x, 33 %1 = ashr i64 %x, 33
%2 = ashr i64 %y, 33 %2 = ashr i64 %y, 33
%result = srem i64 %1, %2 %result = srem i64 %1, %2
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
;FUNC-LABEL: {{^}}test_sdiv2464: ;FUNC-LABEL: {{^}}test_sdiv2464:
;EG: INT_TO_FLT ;EG: INT_TO_FLT
;EG: INT_TO_FLT ;EG: INT_TO_FLT
;EG: FLT_TO_INT ;EG: FLT_TO_INT
;EG-NOT: RECIP_UINT ;EG-NOT: RECIP_UINT
;EG-NOT: BFE_UINT ;EG-NOT: BFE_UINT
;GCN-NOT: s_bfe_u32 ;GCN-NOT: s_bfe_u32
;GCN: v_mad_f32 ;GCN: v_mad_f32
;SI-NOT: v_lshr_b64 ;SI-NOT: v_lshr_b64
;VI-NOT: v_lshrrev_b64 ;VI-NOT: v_lshrrev_b64
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @test_sdiv2464(i64 addrspace(1)* %out, i64 %x, i64 %y) { define amdgpu_kernel void @test_sdiv2464(i64 addrspace(1)* %out, i64 %x, i64 %y) #0 {
%1 = ashr i64 %x, 40 %1 = ashr i64 %x, 40
%2 = ashr i64 %y, 40 %2 = ashr i64 %y, 40
%result = sdiv i64 %1, %2 %result = sdiv i64 %1, %2
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
;FUNC-LABEL: {{^}}test_srem2464: ;FUNC-LABEL: {{^}}test_srem2464:
;EG: INT_TO_FLT ;EG: INT_TO_FLT
;EG: INT_TO_FLT ;EG: INT_TO_FLT
;EG: FLT_TO_INT ;EG: FLT_TO_INT
;EG-NOT: RECIP_UINT ;EG-NOT: RECIP_UINT
;EG-NOT: BFE_UINT ;EG-NOT: BFE_UINT
;GCN-NOT: s_bfe_u32 ;GCN-NOT: s_bfe_u32
;GCN: v_mad_f32 ;GCN: v_mad_f32
;SI-NOT: v_lshr_b64 ;SI-NOT: v_lshr_b64
;VI-NOT: v_lshrrev_b64 ;VI-NOT: v_lshrrev_b64
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @test_srem2464(i64 addrspace(1)* %out, i64 %x, i64 %y) { define amdgpu_kernel void @test_srem2464(i64 addrspace(1)* %out, i64 %x, i64 %y) #0 {
%1 = ashr i64 %x, 40 %1 = ashr i64 %x, 40
%2 = ashr i64 %y, 40 %2 = ashr i64 %y, 40
%result = srem i64 %1, %2 %result = srem i64 %1, %2
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
attributes #0 = { "denormal-fp-math-f32"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/AMDGPU/sdwa-peephole.ll

; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=fiji -amdgpu-sdwa-peephole=0 -mattr=-fp64-fp16-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=NOSDWA,GCN %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=fiji -amdgpu-sdwa-peephole=0 -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=NOSDWA,GCN %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=fiji -amdgpu-sdwa-peephole -mattr=-fp64-fp16-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=VI,GFX89,SDWA,GCN %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=fiji -amdgpu-sdwa-peephole -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=VI,GFX89,SDWA,GCN %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -amdgpu-sdwa-peephole -mattr=-fp64-fp16-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9,GFX9_10,SDWA,GCN %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx900 -amdgpu-sdwa-peephole -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX9,GFX9_10,SDWA,GCN %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx1010 -amdgpu-sdwa-peephole -mattr=-fp64-fp16-denormals -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX9_10,SDWA,GCN %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=gfx1010 -amdgpu-sdwa-peephole -verify-machineinstrs < %s | FileCheck -enable-var-scope -check-prefixes=GFX10,GFX9_10,SDWA,GCN %s
; GCN-LABEL: {{^}}add_shr_i32: ; GCN-LABEL: {{^}}add_shr_i32:
; NOSDWA: v_lshrrev_b32_e32 v[[DST:[0-9]+]], 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v[[DST:[0-9]+]], 16, v{{[0-9]+}}
; NOSDWA: v_add_u32_e32 v{{[0-9]+}}, vcc, v{{[0-9]+}}, v[[DST]] ; NOSDWA: v_add_u32_e32 v{{[0-9]+}}, vcc, v{{[0-9]+}}, v[[DST]]
; NOSDWA-NOT: v_add_{{(_co)?}}_u32_sdwa ; NOSDWA-NOT: v_add_{{(_co)?}}_u32_sdwa
; VI: v_add_u32_sdwa v{{[0-9]+}}, vcc, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_1 ; VI: v_add_u32_sdwa v{{[0-9]+}}, vcc, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_1
; GFX9: v_add_u32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_1 ; GFX9: v_add_u32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_1
; GFX10: v_add_nc_u32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_1 ; GFX10: v_add_nc_u32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_1
define amdgpu_kernel void @add_shr_i32(i32 addrspace(1)* %out, i32 addrspace(1)* %in) { define amdgpu_kernel void @add_shr_i32(i32 addrspace(1)* %out, i32 addrspace(1)* %in) #0 {
%a = load i32, i32 addrspace(1)* %in, align 4 %a = load i32, i32 addrspace(1)* %in, align 4
%shr = lshr i32 %a, 16 %shr = lshr i32 %a, 16
%add = add i32 %a, %shr %add = add i32 %a, %shr
store i32 %add, i32 addrspace(1)* %out, align 4 store i32 %add, i32 addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}sub_shr_i32: ; GCN-LABEL: {{^}}sub_shr_i32:
; NOSDWA: v_lshrrev_b32_e32 v[[DST:[0-9]+]], 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v[[DST:[0-9]+]], 16, v{{[0-9]+}}
; NOSDWA: v_subrev_u32_e32 v{{[0-9]+}}, vcc, v{{[0-9]+}}, v[[DST]] ; NOSDWA: v_subrev_u32_e32 v{{[0-9]+}}, vcc, v{{[0-9]+}}, v[[DST]]
; NOSDWA-NOT: v_subrev_{{(_co)?}}_u32_sdwa ; NOSDWA-NOT: v_subrev_{{(_co)?}}_u32_sdwa
; VI: v_subrev_u32_sdwa v{{[0-9]+}}, vcc, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_1 ; VI: v_subrev_u32_sdwa v{{[0-9]+}}, vcc, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:WORD_1
; GFX9: v_sub_u32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD ; GFX9: v_sub_u32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX10: v_sub_nc_u32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD ; GFX10: v_sub_nc_u32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
define amdgpu_kernel void @sub_shr_i32(i32 addrspace(1)* %out, i32 addrspace(1)* %in) { define amdgpu_kernel void @sub_shr_i32(i32 addrspace(1)* %out, i32 addrspace(1)* %in) #0 {
%a = load i32, i32 addrspace(1)* %in, align 4 %a = load i32, i32 addrspace(1)* %in, align 4
%shr = lshr i32 %a, 16 %shr = lshr i32 %a, 16
%sub = sub i32 %shr, %a %sub = sub i32 %shr, %a
store i32 %sub, i32 addrspace(1)* %out, align 4 store i32 %sub, i32 addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}mul_shr_i32: ; GCN-LABEL: {{^}}mul_shr_i32:
; NOSDWA: v_lshrrev_b32_e32 v[[DST0:[0-9]+]], 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v[[DST0:[0-9]+]], 16, v{{[0-9]+}}
; NOSDWA: v_lshrrev_b32_e32 v[[DST1:[0-9]+]], 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v[[DST1:[0-9]+]], 16, v{{[0-9]+}}
; NOSDWA: v_mul_u32_u24_e32 v{{[0-9]+}}, v[[DST0]], v[[DST1]] ; NOSDWA: v_mul_u32_u24_e32 v{{[0-9]+}}, v[[DST0]], v[[DST1]]
; NOSDWA-NOT: v_mul_u32_u24_sdwa ; NOSDWA-NOT: v_mul_u32_u24_sdwa
; SDWA: v_mul_u32_u24_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1 ; SDWA: v_mul_u32_u24_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
define amdgpu_kernel void @mul_shr_i32(i32 addrspace(1)* %out, i32 addrspace(1)* %in1, i32 addrspace(1)* %in2) { define amdgpu_kernel void @mul_shr_i32(i32 addrspace(1)* %out, i32 addrspace(1)* %in1, i32 addrspace(1)* %in2) #0 {
%a = load i32, i32 addrspace(1)* %in1, align 4 %a = load i32, i32 addrspace(1)* %in1, align 4
%b = load i32, i32 addrspace(1)* %in2, align 4 %b = load i32, i32 addrspace(1)* %in2, align 4
%shra = lshr i32 %a, 16 %shra = lshr i32 %a, 16
%shrb = lshr i32 %b, 16 %shrb = lshr i32 %b, 16
%mul = mul i32 %shra, %shrb %mul = mul i32 %shra, %shrb
store i32 %mul, i32 addrspace(1)* %out, align 4 store i32 %mul, i32 addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}mul_i16: ; GCN-LABEL: {{^}}mul_i16:
; NOSDWA: v_mul_lo_u32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; NOSDWA: v_mul_lo_u32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; NOSDWA-NOT: v_mul_u32_u24_sdwa ; NOSDWA-NOT: v_mul_u32_u24_sdwa
; GFX89: v_mul_lo_u32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX89: v_mul_lo_u32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; GFX10: v_mul_lo_u32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX10: v_mul_lo_u32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; SDWA-NOT: v_mul_u32_u24_sdwa ; SDWA-NOT: v_mul_u32_u24_sdwa
define amdgpu_kernel void @mul_i16(i16 addrspace(1)* %out, i16 addrspace(1)* %ina, i16 addrspace(1)* %inb) { define amdgpu_kernel void @mul_i16(i16 addrspace(1)* %out, i16 addrspace(1)* %ina, i16 addrspace(1)* %inb) #0 {
entry: entry:
%a = load i16, i16 addrspace(1)* %ina, align 4 %a = load i16, i16 addrspace(1)* %ina, align 4
%b = load i16, i16 addrspace(1)* %inb, align 4 %b = load i16, i16 addrspace(1)* %inb, align 4
%mul = mul i16 %a, %b %mul = mul i16 %a, %b
store i16 %mul, i16 addrspace(1)* %out, align 4 store i16 %mul, i16 addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}mul_v2i16: ; GCN-LABEL: {{^}}mul_v2i16:
; NOSDWA: v_lshrrev_b32_e32 v[[DST0:[0-9]+]], 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v[[DST0:[0-9]+]], 16, v{{[0-9]+}}
; NOSDWA: v_lshrrev_b32_e32 v[[DST1:[0-9]+]], 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v[[DST1:[0-9]+]], 16, v{{[0-9]+}}
; NOSDWA: v_mul_u32_u24_e32 v[[DST_MUL:[0-9]+]], v[[DST0]], v[[DST1]] ; NOSDWA: v_mul_u32_u24_e32 v[[DST_MUL:[0-9]+]], v[[DST0]], v[[DST1]]
; NOSDWA: v_lshlrev_b32_e32 v[[DST_SHL:[0-9]+]], 16, v[[DST_MUL]] ; NOSDWA: v_lshlrev_b32_e32 v[[DST_SHL:[0-9]+]], 16, v[[DST_MUL]]
; NOSDWA: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v[[DST_SHL]] ; NOSDWA: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v[[DST_SHL]]
; NOSDWA-NOT: v_mul_u32_u24_sdwa ; NOSDWA-NOT: v_mul_u32_u24_sdwa
; VI-DAG: v_mul_u32_u24_sdwa v[[DST_MUL_LO:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:WORD_0 ; VI-DAG: v_mul_u32_u24_sdwa v[[DST_MUL_LO:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:WORD_0
; VI-DAG: v_mul_u32_u24_sdwa v[[DST_MUL_HI:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1 ; VI-DAG: v_mul_u32_u24_sdwa v[[DST_MUL_HI:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
; VI: v_or_b32_sdwa v{{[0-9]+}}, v[[DST_MUL_LO]], v[[DST_MUL_HI]] dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD ; VI: v_or_b32_sdwa v{{[0-9]+}}, v[[DST_MUL_LO]], v[[DST_MUL_HI]] dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9_10: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
define amdgpu_kernel void @mul_v2i16(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %ina, <2 x i16> addrspace(1)* %inb) { define amdgpu_kernel void @mul_v2i16(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %ina, <2 x i16> addrspace(1)* %inb) #0 {
entry: entry:
%a = load <2 x i16>, <2 x i16> addrspace(1)* %ina, align 4 %a = load <2 x i16>, <2 x i16> addrspace(1)* %ina, align 4
%b = load <2 x i16>, <2 x i16> addrspace(1)* %inb, align 4 %b = load <2 x i16>, <2 x i16> addrspace(1)* %inb, align 4
%mul = mul <2 x i16> %a, %b %mul = mul <2 x i16> %a, %b
store <2 x i16> %mul, <2 x i16> addrspace(1)* %out, align 4 store <2 x i16> %mul, <2 x i16> addrspace(1)* %out, align 4
ret void ret void
} }
Show All 10 Lines
; VI-DAG: v_mul_u32_u24_sdwa v[[DST_MUL2:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:WORD_0 ; VI-DAG: v_mul_u32_u24_sdwa v[[DST_MUL2:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:WORD_0
; VI-DAG: v_mul_u32_u24_sdwa v[[DST_MUL3:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1 ; VI-DAG: v_mul_u32_u24_sdwa v[[DST_MUL3:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
; VI-DAG: v_or_b32_sdwa v{{[0-9]+}}, v[[DST_MUL2]], v[[DST_MUL3]] dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD ; VI-DAG: v_or_b32_sdwa v{{[0-9]+}}, v[[DST_MUL2]], v[[DST_MUL3]] dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-DAG: v_or_b32_sdwa v{{[0-9]+}}, v[[DST_MUL0]], v[[DST_MUL1]] dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD ; VI-DAG: v_or_b32_sdwa v{{[0-9]+}}, v[[DST_MUL0]], v[[DST_MUL1]] dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9_10-DAG: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10-DAG: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; GFX9_10-DAG: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10-DAG: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
define amdgpu_kernel void @mul_v4i16(<4 x i16> addrspace(1)* %out, <4 x i16> addrspace(1)* %ina, <4 x i16> addrspace(1)* %inb) { define amdgpu_kernel void @mul_v4i16(<4 x i16> addrspace(1)* %out, <4 x i16> addrspace(1)* %ina, <4 x i16> addrspace(1)* %inb) #0 {
entry: entry:
%a = load <4 x i16>, <4 x i16> addrspace(1)* %ina, align 4 %a = load <4 x i16>, <4 x i16> addrspace(1)* %ina, align 4
%b = load <4 x i16>, <4 x i16> addrspace(1)* %inb, align 4 %b = load <4 x i16>, <4 x i16> addrspace(1)* %inb, align 4
%mul = mul <4 x i16> %a, %b %mul = mul <4 x i16> %a, %b
store <4 x i16> %mul, <4 x i16> addrspace(1)* %out, align 4 store <4 x i16> %mul, <4 x i16> addrspace(1)* %out, align 4
ret void ret void
} }
Show All 18 Lines
; VI-DAG: v_or_b32_sdwa v{{[0-9]+}}, v[[DST_MUL2]], v[[DST_MUL3]] dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD ; VI-DAG: v_or_b32_sdwa v{{[0-9]+}}, v[[DST_MUL2]], v[[DST_MUL3]] dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-DAG: v_or_b32_sdwa v{{[0-9]+}}, v[[DST_MUL0]], v[[DST_MUL1]] dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD ; VI-DAG: v_or_b32_sdwa v{{[0-9]+}}, v[[DST_MUL0]], v[[DST_MUL1]] dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; GFX9_10-DAG: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10-DAG: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; GFX9_10-DAG: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10-DAG: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; GFX9_10-DAG: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10-DAG: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; GFX9_10-DAG: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10-DAG: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
define amdgpu_kernel void @mul_v8i16(<8 x i16> addrspace(1)* %out, <8 x i16> addrspace(1)* %ina, <8 x i16> addrspace(1)* %inb) { define amdgpu_kernel void @mul_v8i16(<8 x i16> addrspace(1)* %out, <8 x i16> addrspace(1)* %ina, <8 x i16> addrspace(1)* %inb) #0 {
entry: entry:
%a = load <8 x i16>, <8 x i16> addrspace(1)* %ina, align 4 %a = load <8 x i16>, <8 x i16> addrspace(1)* %ina, align 4
%b = load <8 x i16>, <8 x i16> addrspace(1)* %inb, align 4 %b = load <8 x i16>, <8 x i16> addrspace(1)* %inb, align 4
%mul = mul <8 x i16> %a, %b %mul = mul <8 x i16> %a, %b
store <8 x i16> %mul, <8 x i16> addrspace(1)* %out, align 4 store <8 x i16> %mul, <8 x i16> addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}mul_half: ; GCN-LABEL: {{^}}mul_half:
; NOSDWA: v_mul_f16_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; NOSDWA: v_mul_f16_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; NOSDWA-NOT: v_mul_f16_sdwa ; NOSDWA-NOT: v_mul_f16_sdwa
; SDWA: v_mul_f16_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; SDWA: v_mul_f16_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; SDWA-NOT: v_mul_f16_sdwa ; SDWA-NOT: v_mul_f16_sdwa
define amdgpu_kernel void @mul_half(half addrspace(1)* %out, half addrspace(1)* %ina, half addrspace(1)* %inb) { define amdgpu_kernel void @mul_half(half addrspace(1)* %out, half addrspace(1)* %ina, half addrspace(1)* %inb) #0 {
entry: entry:
%a = load half, half addrspace(1)* %ina, align 4 %a = load half, half addrspace(1)* %ina, align 4
%b = load half, half addrspace(1)* %inb, align 4 %b = load half, half addrspace(1)* %inb, align 4
%mul = fmul half %a, %b %mul = fmul half %a, %b
store half %mul, half addrspace(1)* %out, align 4 store half %mul, half addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}mul_v2half: ; GCN-LABEL: {{^}}mul_v2half:
; NOSDWA: v_lshrrev_b32_e32 v[[DST0:[0-9]+]], 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v[[DST0:[0-9]+]], 16, v{{[0-9]+}}
; NOSDWA: v_lshrrev_b32_e32 v[[DST1:[0-9]+]], 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v[[DST1:[0-9]+]], 16, v{{[0-9]+}}
; NOSDWA: v_mul_f16_e32 v[[DST_MUL:[0-9]+]], v[[DST0]], v[[DST1]] ; NOSDWA: v_mul_f16_e32 v[[DST_MUL:[0-9]+]], v[[DST0]], v[[DST1]]
; NOSDWA: v_lshlrev_b32_e32 v[[DST_SHL:[0-9]+]], 16, v[[DST_MUL]] ; NOSDWA: v_lshlrev_b32_e32 v[[DST_SHL:[0-9]+]], 16, v[[DST_MUL]]
; NOSDWA: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v[[DST_SHL]] ; NOSDWA: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v[[DST_SHL]]
; NOSDWA-NOT: v_mul_f16_sdwa ; NOSDWA-NOT: v_mul_f16_sdwa
; VI-DAG: v_mul_f16_sdwa v[[DST_MUL_HI:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1 ; VI-DAG: v_mul_f16_sdwa v[[DST_MUL_HI:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
; VI-DAG: v_mul_f16_e32 v[[DST_MUL_LO:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} ; VI-DAG: v_mul_f16_e32 v[[DST_MUL_LO:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}}
; VI: v_or_b32_e32 v{{[0-9]+}}, v[[DST_MUL_LO]], v[[DST_MUL_HI]] ; VI: v_or_b32_e32 v{{[0-9]+}}, v[[DST_MUL_LO]], v[[DST_MUL_HI]]
; GFX9_10: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
define amdgpu_kernel void @mul_v2half(<2 x half> addrspace(1)* %out, <2 x half> addrspace(1)* %ina, <2 x half> addrspace(1)* %inb) { define amdgpu_kernel void @mul_v2half(<2 x half> addrspace(1)* %out, <2 x half> addrspace(1)* %ina, <2 x half> addrspace(1)* %inb) #0 {
entry: entry:
%a = load <2 x half>, <2 x half> addrspace(1)* %ina, align 4 %a = load <2 x half>, <2 x half> addrspace(1)* %ina, align 4
%b = load <2 x half>, <2 x half> addrspace(1)* %inb, align 4 %b = load <2 x half>, <2 x half> addrspace(1)* %inb, align 4
%mul = fmul <2 x half> %a, %b %mul = fmul <2 x half> %a, %b
store <2 x half> %mul, <2 x half> addrspace(1)* %out, align 4 store <2 x half> %mul, <2 x half> addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}mul_v4half: ; GCN-LABEL: {{^}}mul_v4half:
; NOSDWA: v_lshrrev_b32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}}
; NOSDWA: v_lshrrev_b32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}}
; NOSDWA: v_mul_f16_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; NOSDWA: v_mul_f16_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; NOSDWA: v_lshlrev_b32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}} ; NOSDWA: v_lshlrev_b32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}}
; NOSDWA: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; NOSDWA: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; NOSDWA-NOT: v_mul_f16_sdwa ; NOSDWA-NOT: v_mul_f16_sdwa
; VI-DAG: v_mul_f16_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1 ; VI-DAG: v_mul_f16_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
; VI-DAG: v_mul_f16_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1 ; VI-DAG: v_mul_f16_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
; VI-DAG: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; VI-DAG: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; VI-DAG: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; VI-DAG: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; GFX9_10-DAG: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10-DAG: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; GFX9_10-DAG: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10-DAG: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
define amdgpu_kernel void @mul_v4half(<4 x half> addrspace(1)* %out, <4 x half> addrspace(1)* %ina, <4 x half> addrspace(1)* %inb) { define amdgpu_kernel void @mul_v4half(<4 x half> addrspace(1)* %out, <4 x half> addrspace(1)* %ina, <4 x half> addrspace(1)* %inb) #0 {
entry: entry:
%a = load <4 x half>, <4 x half> addrspace(1)* %ina, align 4 %a = load <4 x half>, <4 x half> addrspace(1)* %ina, align 4
%b = load <4 x half>, <4 x half> addrspace(1)* %inb, align 4 %b = load <4 x half>, <4 x half> addrspace(1)* %inb, align 4
%mul = fmul <4 x half> %a, %b %mul = fmul <4 x half> %a, %b
store <4 x half> %mul, <4 x half> addrspace(1)* %out, align 4 store <4 x half> %mul, <4 x half> addrspace(1)* %out, align 4
ret void ret void
} }
Show All 14 Lines
; VI-DAG: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; VI-DAG: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; VI-DAG: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; VI-DAG: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; GFX9_10-DAG: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10-DAG: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; GFX9_10-DAG: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10-DAG: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; GFX9_10-DAG: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10-DAG: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; GFX9_10-DAG: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10-DAG: v_pk_mul_f16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
define amdgpu_kernel void @mul_v8half(<8 x half> addrspace(1)* %out, <8 x half> addrspace(1)* %ina, <8 x half> addrspace(1)* %inb) { define amdgpu_kernel void @mul_v8half(<8 x half> addrspace(1)* %out, <8 x half> addrspace(1)* %ina, <8 x half> addrspace(1)* %inb) #0 {
entry: entry:
%a = load <8 x half>, <8 x half> addrspace(1)* %ina, align 4 %a = load <8 x half>, <8 x half> addrspace(1)* %ina, align 4
%b = load <8 x half>, <8 x half> addrspace(1)* %inb, align 4 %b = load <8 x half>, <8 x half> addrspace(1)* %inb, align 4
%mul = fmul <8 x half> %a, %b %mul = fmul <8 x half> %a, %b
store <8 x half> %mul, <8 x half> addrspace(1)* %out, align 4 store <8 x half> %mul, <8 x half> addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}mul_i8: ; GCN-LABEL: {{^}}mul_i8:
; NOSDWA: v_mul_lo_u32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; NOSDWA: v_mul_lo_u32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; NOSDWA-NOT: v_mul_u32_u24_sdwa ; NOSDWA-NOT: v_mul_u32_u24_sdwa
; GFX89: v_mul_lo_u32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX89: v_mul_lo_u32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; GFX10: v_mul_lo_u32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX10: v_mul_lo_u32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; SDWA-NOT: v_mul_u32_u24_sdwa ; SDWA-NOT: v_mul_u32_u24_sdwa
define amdgpu_kernel void @mul_i8(i8 addrspace(1)* %out, i8 addrspace(1)* %ina, i8 addrspace(1)* %inb) { define amdgpu_kernel void @mul_i8(i8 addrspace(1)* %out, i8 addrspace(1)* %ina, i8 addrspace(1)* %inb) #0 {
entry: entry:
%a = load i8, i8 addrspace(1)* %ina, align 4 %a = load i8, i8 addrspace(1)* %ina, align 4
%b = load i8, i8 addrspace(1)* %inb, align 4 %b = load i8, i8 addrspace(1)* %inb, align 4
%mul = mul i8 %a, %b %mul = mul i8 %a, %b
store i8 %mul, i8 addrspace(1)* %out, align 4 store i8 %mul, i8 addrspace(1)* %out, align 4
ret void ret void
} }
Show All 12 Lines
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
; GFX9: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD ; GFX9: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX10: v_lshlrev_b16_e64 v{{[0-9]+}}, 8, v ; GFX10: v_lshlrev_b16_e64 v{{[0-9]+}}, 8, v
; GFX10: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD ; GFX10: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
define amdgpu_kernel void @mul_v2i8(<2 x i8> addrspace(1)* %out, <2 x i8> addrspace(1)* %ina, <2 x i8> addrspace(1)* %inb) { define amdgpu_kernel void @mul_v2i8(<2 x i8> addrspace(1)* %out, <2 x i8> addrspace(1)* %ina, <2 x i8> addrspace(1)* %inb) #0 {
entry: entry:
%a = load <2 x i8>, <2 x i8> addrspace(1)* %ina, align 4 %a = load <2 x i8>, <2 x i8> addrspace(1)* %ina, align 4
%b = load <2 x i8>, <2 x i8> addrspace(1)* %inb, align 4 %b = load <2 x i8>, <2 x i8> addrspace(1)* %inb, align 4
%mul = mul <2 x i8> %a, %b %mul = mul <2 x i8> %a, %b
store <2 x i8> %mul, <2 x i8> addrspace(1)* %out, align 4 store <2 x i8> %mul, <2 x i8> addrspace(1)* %out, align 4
ret void ret void
} }
Show All 13 Lines
; GFX9-DAG: v_mul_lo_u16_sdwa ; GFX9-DAG: v_mul_lo_u16_sdwa
; GFX9-DAG: v_mul_lo_u16_sdwa ; GFX9-DAG: v_mul_lo_u16_sdwa
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
define amdgpu_kernel void @mul_v4i8(<4 x i8> addrspace(1)* %out, <4 x i8> addrspace(1)* %ina, <4 x i8> addrspace(1)* %inb) { define amdgpu_kernel void @mul_v4i8(<4 x i8> addrspace(1)* %out, <4 x i8> addrspace(1)* %ina, <4 x i8> addrspace(1)* %inb) #0 {
entry: entry:
%a = load <4 x i8>, <4 x i8> addrspace(1)* %ina, align 4 %a = load <4 x i8>, <4 x i8> addrspace(1)* %ina, align 4
%b = load <4 x i8>, <4 x i8> addrspace(1)* %inb, align 4 %b = load <4 x i8>, <4 x i8> addrspace(1)* %inb, align 4
%mul = mul <4 x i8> %a, %b %mul = mul <4 x i8> %a, %b
store <4 x i8> %mul, <4 x i8> addrspace(1)* %out, align 4 store <4 x i8> %mul, <4 x i8> addrspace(1)* %out, align 4
ret void ret void
} }
Show All 23 Lines
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
; GFX10-DAG: v_mul_lo_u16_e64 ; GFX10-DAG: v_mul_lo_u16_e64
define amdgpu_kernel void @mul_v8i8(<8 x i8> addrspace(1)* %out, <8 x i8> addrspace(1)* %ina, <8 x i8> addrspace(1)* %inb) { define amdgpu_kernel void @mul_v8i8(<8 x i8> addrspace(1)* %out, <8 x i8> addrspace(1)* %ina, <8 x i8> addrspace(1)* %inb) #0 {
entry: entry:
%a = load <8 x i8>, <8 x i8> addrspace(1)* %ina, align 4 %a = load <8 x i8>, <8 x i8> addrspace(1)* %ina, align 4
%b = load <8 x i8>, <8 x i8> addrspace(1)* %inb, align 4 %b = load <8 x i8>, <8 x i8> addrspace(1)* %inb, align 4
%mul = mul <8 x i8> %a, %b %mul = mul <8 x i8> %a, %b
store <8 x i8> %mul, <8 x i8> addrspace(1)* %out, align 4 store <8 x i8> %mul, <8 x i8> addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}sitofp_v2i16_to_v2f16: ; GCN-LABEL: {{^}}sitofp_v2i16_to_v2f16:
; NOSDWA-DAG: v_bfe_i32 v{{[0-9]+}}, v{{[0-9]+}}, 0, 16 ; NOSDWA-DAG: v_bfe_i32 v{{[0-9]+}}, v{{[0-9]+}}, 0, 16
; NOSDWA-DAG: v_ashrrev_i32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}} ; NOSDWA-DAG: v_ashrrev_i32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}}
; NOSDWA-DAG: v_cvt_f32_i32_e32 v{{[0-9]+}}, v{{[0-9]+}} ; NOSDWA-DAG: v_cvt_f32_i32_e32 v{{[0-9]+}}, v{{[0-9]+}}
; NOSDWA-DAG: v_cvt_f32_i32_e32 v{{[0-9]+}}, v{{[0-9]+}} ; NOSDWA-DAG: v_cvt_f32_i32_e32 v{{[0-9]+}}, v{{[0-9]+}}
; NOSDWA-NOT: v_cvt_f32_i32_sdwa ; NOSDWA-NOT: v_cvt_f32_i32_sdwa
; SDWA-DAG: v_cvt_f32_i32_sdwa v{{[0-9]+}}, sext(v{{[0-9]+}}) dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 ; SDWA-DAG: v_cvt_f32_i32_sdwa v{{[0-9]+}}, sext(v{{[0-9]+}}) dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0
; SDWA-DAG: v_cvt_f32_i32_sdwa v{{[0-9]+}}, sext(v{{[0-9]+}}) dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 ; SDWA-DAG: v_cvt_f32_i32_sdwa v{{[0-9]+}}, sext(v{{[0-9]+}}) dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1
define amdgpu_kernel void @sitofp_v2i16_to_v2f16( define amdgpu_kernel void @sitofp_v2i16_to_v2f16(
<2 x half> addrspace(1)* %r, <2 x half> addrspace(1)* %r,
<2 x i16> addrspace(1)* %a) { <2 x i16> addrspace(1)* %a) #0 {
entry: entry:
%a.val = load <2 x i16>, <2 x i16> addrspace(1)* %a %a.val = load <2 x i16>, <2 x i16> addrspace(1)* %a
%r.val = sitofp <2 x i16> %a.val to <2 x half> %r.val = sitofp <2 x i16> %a.val to <2 x half>
store <2 x half> %r.val, <2 x half> addrspace(1)* %r store <2 x half> %r.val, <2 x half> addrspace(1)* %r
ret void ret void
} }
; GCN-LABEL: {{^}}mac_v2half: ; GCN-LABEL: {{^}}mac_v2half:
; NOSDWA: v_lshrrev_b32_e32 v[[DST0:[0-9]+]], 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v[[DST0:[0-9]+]], 16, v{{[0-9]+}}
; NOSDWA: v_lshrrev_b32_e32 v[[DST1:[0-9]+]], 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v[[DST1:[0-9]+]], 16, v{{[0-9]+}}
; NOSDWA: v_mac_f16_e32 v[[DST_MAC:[0-9]+]], v[[DST0]], v[[DST1]] ; NOSDWA: v_mac_f16_e32 v[[DST_MAC:[0-9]+]], v[[DST0]], v[[DST1]]
; NOSDWA: v_lshlrev_b32_e32 v[[DST_SHL:[0-9]+]], 16, v[[DST_MAC]] ; NOSDWA: v_lshlrev_b32_e32 v[[DST_SHL:[0-9]+]], 16, v[[DST_MAC]]
; NOSDWA: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v[[DST_SHL]] ; NOSDWA: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v[[DST_SHL]]
; NOSDWA-NOT: v_mac_f16_sdwa ; NOSDWA-NOT: v_mac_f16_sdwa
; VI: v_mac_f16_sdwa v[[DST_MAC:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1 ; VI: v_mac_f16_sdwa v[[DST_MAC:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
; VI: v_lshlrev_b32_e32 v[[DST_SHL:[0-9]+]], 16, v[[DST_MAC]] ; VI: v_lshlrev_b32_e32 v[[DST_SHL:[0-9]+]], 16, v[[DST_MAC]]
; GFX9_10: v_pk_mul_f16 v[[DST_MUL:[0-9]+]], v{{[0-9]+}}, v[[SRC:[0-9]+]] ; GFX9_10: v_pk_mul_f16 v[[DST_MUL:[0-9]+]], v{{[0-9]+}}, v[[SRC:[0-9]+]]
; GFX9_10: v_pk_add_f16 v{{[0-9]+}}, v[[DST_MUL]], v[[SRC]] ; GFX9_10: v_pk_add_f16 v{{[0-9]+}}, v[[DST_MUL]], v[[SRC]]
define amdgpu_kernel void @mac_v2half(<2 x half> addrspace(1)* %out, <2 x half> addrspace(1)* %ina, <2 x half> addrspace(1)* %inb) { define amdgpu_kernel void @mac_v2half(<2 x half> addrspace(1)* %out, <2 x half> addrspace(1)* %ina, <2 x half> addrspace(1)* %inb) #0 {
entry: entry:
%a = load <2 x half>, <2 x half> addrspace(1)* %ina, align 4 %a = load <2 x half>, <2 x half> addrspace(1)* %ina, align 4
%b = load <2 x half>, <2 x half> addrspace(1)* %inb, align 4 %b = load <2 x half>, <2 x half> addrspace(1)* %inb, align 4
%mul = fmul <2 x half> %a, %b %mul = fmul <2 x half> %a, %b
%mac = fadd <2 x half> %mul, %b %mac = fadd <2 x half> %mul, %b
store <2 x half> %mac, <2 x half> addrspace(1)* %out, align 4 store <2 x half> %mac, <2 x half> addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}immediate_mul_v2i16: ; GCN-LABEL: {{^}}immediate_mul_v2i16:
; NOSDWA-NOT: v_mul_u32_u24_sdwa ; NOSDWA-NOT: v_mul_u32_u24_sdwa
; VI-DAG: v_mov_b32_e32 v[[M321:[0-9]+]], 0x141 ; VI-DAG: v_mov_b32_e32 v[[M321:[0-9]+]], 0x141
; VI-DAG: v_mov_b32_e32 v[[M123:[0-9]+]], 0x7b ; VI-DAG: v_mov_b32_e32 v[[M123:[0-9]+]], 0x7b
; VI-DAG: v_mul_u32_u24_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v[[M123]] dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD ; VI-DAG: v_mul_u32_u24_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v[[M123]] dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_0 src1_sel:DWORD
; VI-DAG: v_mul_u32_u24_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v[[M321]] dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD ; VI-DAG: v_mul_u32_u24_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v[[M321]] dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9: s_mov_b32 s[[IMM:[0-9]+]], 0x141007b ; GFX9: s_mov_b32 s[[IMM:[0-9]+]], 0x141007b
; GFX9: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, s[[IMM]] ; GFX9: v_pk_mul_lo_u16 v{{[0-9]+}}, v{{[0-9]+}}, s[[IMM]]
; GFX10: v_pk_mul_lo_u16 v{{[0-9]+}}, 0x141007b, v{{[0-9]+}} ; GFX10: v_pk_mul_lo_u16 v{{[0-9]+}}, 0x141007b, v{{[0-9]+}}
define amdgpu_kernel void @immediate_mul_v2i16(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) { define amdgpu_kernel void @immediate_mul_v2i16(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %in) #0 {
entry: entry:
%a = load <2 x i16>, <2 x i16> addrspace(1)* %in, align 4 %a = load <2 x i16>, <2 x i16> addrspace(1)* %in, align 4
%mul = mul <2 x i16> %a, <i16 123, i16 321> %mul = mul <2 x i16> %a, <i16 123, i16 321>
store <2 x i16> %mul, <2 x i16> addrspace(1)* %out, align 4 store <2 x i16> %mul, <2 x i16> addrspace(1)* %out, align 4
ret void ret void
} }
; Double use of same src - should not convert it ; Double use of same src - should not convert it
; GCN-LABEL: {{^}}mulmul_v2i16: ; GCN-LABEL: {{^}}mulmul_v2i16:
; NOSDWA: v_lshrrev_b32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}}
; NOSDWA: v_lshrrev_b32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}} ; NOSDWA: v_lshrrev_b32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}}
; NOSDWA: v_mul_u32_u24_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; NOSDWA: v_mul_u32_u24_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; NOSDWA: v_lshlrev_b32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}} ; NOSDWA: v_lshlrev_b32_e32 v{{[0-9]+}}, 16, v{{[0-9]+}}
; NOSDWA: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; NOSDWA: v_or_b32_e32 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
; NOSDWA-NOT: v_mul_u32_u24_sdwa ; NOSDWA-NOT: v_mul_u32_u24_sdwa
; VI: v_mul_u32_u24_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD ; VI: v_mul_u32_u24_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:DWORD
; GFX9_10: v_pk_mul_lo_u16 v[[DST1:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10: v_pk_mul_lo_u16 v[[DST1:[0-9]+]], v{{[0-9]+}}, v{{[0-9]+}}
; GFX9_10: v_pk_mul_lo_u16 v{{[0-9]+}}, v[[DST1]], v{{[0-9]+}} ; GFX9_10: v_pk_mul_lo_u16 v{{[0-9]+}}, v[[DST1]], v{{[0-9]+}}
define amdgpu_kernel void @mulmul_v2i16(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %ina, <2 x i16> addrspace(1)* %inb) { define amdgpu_kernel void @mulmul_v2i16(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %ina, <2 x i16> addrspace(1)* %inb) #0 {
entry: entry:
%a = load <2 x i16>, <2 x i16> addrspace(1)* %ina, align 4 %a = load <2 x i16>, <2 x i16> addrspace(1)* %ina, align 4
%b = load <2 x i16>, <2 x i16> addrspace(1)* %inb, align 4 %b = load <2 x i16>, <2 x i16> addrspace(1)* %inb, align 4
%mul = mul <2 x i16> %a, %b %mul = mul <2 x i16> %a, %b
%mul2 = mul <2 x i16> %mul, %b %mul2 = mul <2 x i16> %mul, %b
store <2 x i16> %mul2, <2 x i16> addrspace(1)* %out, align 4 store <2 x i16> %mul2, <2 x i16> addrspace(1)* %out, align 4
ret void ret void
} }
; GCN-LABEL: {{^}}add_bb_v2i16: ; GCN-LABEL: {{^}}add_bb_v2i16:
; NOSDWA-NOT: v_add_{{(_co)?}}_u32_sdwa ; NOSDWA-NOT: v_add_{{(_co)?}}_u32_sdwa
; VI: v_add_u32_sdwa v{{[0-9]+}}, vcc, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1 ; VI: v_add_u32_sdwa v{{[0-9]+}}, vcc, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:WORD_1 src1_sel:WORD_1
; GFX9_10: v_pk_add_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} ; GFX9_10: v_pk_add_u16 v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}}
define amdgpu_kernel void @add_bb_v2i16(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %ina, <2 x i16> addrspace(1)* %inb) { define amdgpu_kernel void @add_bb_v2i16(<2 x i16> addrspace(1)* %out, <2 x i16> addrspace(1)* %ina, <2 x i16> addrspace(1)* %inb) #0 {
entry: entry:
%a = load <2 x i16>, <2 x i16> addrspace(1)* %ina, align 4 %a = load <2 x i16>, <2 x i16> addrspace(1)* %ina, align 4
%b = load <2 x i16>, <2 x i16> addrspace(1)* %inb, align 4 %b = load <2 x i16>, <2 x i16> addrspace(1)* %inb, align 4
br label %add_label br label %add_label
add_label: add_label:
%add = add <2 x i16> %a, %b %add = add <2 x i16> %a, %b
br label %store_label br label %store_label
store_label: store_label:
Show All 26 Lines
; ;
; GFX89: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD ; GFX89: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; ;
; GFX10: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD ; GFX10: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX10: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD ; GFX10: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:DWORD dst_unused:UNUSED_PAD src0_sel:BYTE_0 src1_sel:DWORD
; GFX10: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD ; GFX10: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; GFX10: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD ; GFX10: v_or_b32_sdwa v{{[0-9]+}}, v{{[0-9]+}}, v{{[0-9]+}} dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
define amdgpu_kernel void @pulled_out_test(<8 x i8> addrspace(1)* %sourceA, <8 x i8> addrspace(1)* %destValues) { define amdgpu_kernel void @pulled_out_test(<8 x i8> addrspace(1)* %sourceA, <8 x i8> addrspace(1)* %destValues) #0 {
entry: entry:
%idxprom = ashr exact i64 15, 32 %idxprom = ashr exact i64 15, 32
%arrayidx = getelementptr inbounds <8 x i8>, <8 x i8> addrspace(1)* %sourceA, i64 %idxprom %arrayidx = getelementptr inbounds <8 x i8>, <8 x i8> addrspace(1)* %sourceA, i64 %idxprom
%tmp = load <8 x i8>, <8 x i8> addrspace(1)* %arrayidx, align 8 %tmp = load <8 x i8>, <8 x i8> addrspace(1)* %arrayidx, align 8
%tmp1 = extractelement <8 x i8> %tmp, i32 0 %tmp1 = extractelement <8 x i8> %tmp, i32 0
%tmp2 = extractelement <8 x i8> %tmp, i32 1 %tmp2 = extractelement <8 x i8> %tmp, i32 1
%tmp3 = extractelement <8 x i8> %tmp, i32 2 %tmp3 = extractelement <8 x i8> %tmp, i32 2
▲ Show 20 Lines • Show All 44 Lines • ▼ Show 20 Linesbb2: ; preds = %bb1
%tmp6 = insertelement <2 x i32> %tmp, i32 %tmp5, i64 0 %tmp6 = insertelement <2 x i32> %tmp, i32 %tmp5, i64 0
br label %bb11 br label %bb11
bb11: ; preds = %bb10, %bb2 bb11: ; preds = %bb10, %bb2
%tmp12 = phi <2 x i32> [ %tmp6, %bb2 ], [ %tmp, %bb1 ] %tmp12 = phi <2 x i32> [ %tmp6, %bb2 ], [ %tmp, %bb1 ]
store volatile <2 x i32> %tmp12, <2 x i32> addrspace(1)* undef store volatile <2 x i32> %tmp12, <2 x i32> addrspace(1)* undef
br label %bb1 br label %bb1
} }
attributes #0 = { "denormal-fp-math"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/AMDGPU/udivrem24.ll

Show All 15 Linesdefine amdgpu_kernel void @udiv24_i8(i8 addrspace(1)* %out, i8 addrspace(1)* %in) {
%den_ptr = getelementptr i8, i8 addrspace(1)* %in, i8 1 %den_ptr = getelementptr i8, i8 addrspace(1)* %in, i8 1
%num = load i8, i8 addrspace(1) * %in %num = load i8, i8 addrspace(1) * %in
%den = load i8, i8 addrspace(1) * %den_ptr %den = load i8, i8 addrspace(1) * %den_ptr
%result = udiv i8 %num, %den %result = udiv i8 %num, %den
store i8 %result, i8 addrspace(1)* %out store i8 %result, i8 addrspace(1)* %out
ret void ret void
} }
; FUNC-LABEL: {{^}}udiv24_i8_denorm_flush_in_out:
; SI: v_cvt_f32_ubyte
; SI-DAG: v_cvt_f32_ubyte
; SI-DAG: v_rcp_iflag_f32
; SI: v_cvt_u32_f32
; EG: UINT_TO_FLT
; EG-DAG: UINT_TO_FLT
; EG-DAG: RECIP_IEEE
; EG: FLT_TO_UINT
define amdgpu_kernel void @udiv24_i8_denorm_flush_in_out(i8 addrspace(1)* %out, i8 addrspace(1)* %in) #0 {
%den_ptr = getelementptr i8, i8 addrspace(1)* %in, i8 1
%num = load i8, i8 addrspace(1) * %in
%den = load i8, i8 addrspace(1) * %den_ptr
%result = udiv i8 %num, %den
store i8 %result, i8 addrspace(1)* %out
ret void
}
; FUNC-LABEL: {{^}}udiv24_i8_denorm_flush_in:
; SI: v_cvt_f32_ubyte
; SI-DAG: v_cvt_f32_ubyte
; SI-DAG: v_rcp_iflag_f32
; SI: v_cvt_u32_f32
; EG: UINT_TO_FLT
; EG-DAG: UINT_TO_FLT
; EG-DAG: RECIP_IEEE
; EG: FLT_TO_UINT
define amdgpu_kernel void @udiv24_i8_denorm_flush_in(i8 addrspace(1)* %out, i8 addrspace(1)* %in) #1 {
%den_ptr = getelementptr i8, i8 addrspace(1)* %in, i8 1
%num = load i8, i8 addrspace(1) * %in
%den = load i8, i8 addrspace(1) * %den_ptr
%result = udiv i8 %num, %den
store i8 %result, i8 addrspace(1)* %out
ret void
}
; FUNC-LABEL: {{^}}udiv24_i8_denorm_flush_out:
; SI: v_cvt_f32_ubyte
; SI-DAG: v_cvt_f32_ubyte
; SI-DAG: v_rcp_iflag_f32
; SI: v_cvt_u32_f32
; EG: UINT_TO_FLT
; EG-DAG: UINT_TO_FLT
; EG-DAG: RECIP_IEEE
; EG: FLT_TO_UINT
define amdgpu_kernel void @udiv24_i8_denorm_flush_out(i8 addrspace(1)* %out, i8 addrspace(1)* %in) #2 {
%den_ptr = getelementptr i8, i8 addrspace(1)* %in, i8 1
%num = load i8, i8 addrspace(1) * %in
%den = load i8, i8 addrspace(1) * %den_ptr
%result = udiv i8 %num, %den
store i8 %result, i8 addrspace(1)* %out
ret void
}
; FUNC-LABEL: {{^}}udiv24_i16: ; FUNC-LABEL: {{^}}udiv24_i16:
; SI: v_cvt_f32_u32 ; SI: v_cvt_f32_u32
; SI: v_cvt_f32_u32 ; SI: v_cvt_f32_u32
; SI: v_rcp_iflag_f32 ; SI: v_rcp_iflag_f32
; SI: v_cvt_u32_f32 ; SI: v_cvt_u32_f32
; EG: UINT_TO_FLT ; EG: UINT_TO_FLT
; EG-DAG: UINT_TO_FLT ; EG-DAG: UINT_TO_FLT
▲ Show 20 Lines • Show All 288 Lines • ▼ Show 20 Linesdefine amdgpu_kernel void @test_udiv24_u23_u16_i32(i32 addrspace(1)* %out, i32 addrspace(1)* %in) {
%num.i23.0 = shl i32 %num, 9 %num.i23.0 = shl i32 %num, 9
%den.i16.0 = shl i32 %den, 16 %den.i16.0 = shl i32 %den, 16
%num.i23 = lshr i32 %num.i23.0, 9 %num.i23 = lshr i32 %num.i23.0, 9
%den.i16 = lshr i32 %den.i16.0, 16 %den.i16 = lshr i32 %den.i16.0, 16
%result = udiv i32 %num.i23, %den.i16 %result = udiv i32 %num.i23, %den.i16
store i32 %result, i32 addrspace(1)* %out, align 4 store i32 %result, i32 addrspace(1)* %out, align 4
ret void ret void
} }
attributes #0 = { "denormal-fp-math-f32"="preserve-sign,preserve-sign" }
attributes #1 = { "denormal-fp-math-f32"="ieee,preserve-sign" }
attributes #2 = { "denormal-fp-math-f32"="preserve-sign,ieee" }

llvm/test/CodeGen/AMDGPU/udivrem64.ll

;RUN: llc -march=amdgcn -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck --check-prefix=SI --check-prefix=GCN --check-prefix=FUNC %s ;RUN: llc -march=amdgcn -verify-machineinstrs < %s | FileCheck --check-prefix=SI --check-prefix=GCN --check-prefix=FUNC %s
;RUN: llc -march=amdgcn -mcpu=tonga -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck --check-prefix=VI --check-prefix=GCN --check-prefix=FUNC %s ;RUN: llc -march=amdgcn -mcpu=tonga -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck --check-prefix=VI --check-prefix=GCN --check-prefix=FUNC %s
;RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=-fp32-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck --check-prefix=VI --check-prefix=GCN --check-prefix=FUNC %s ;RUN: llc -march=amdgcn -mcpu=gfx900 -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck --check-prefix=VI --check-prefix=GCN --check-prefix=FUNC %s
;RUN: llc -march=r600 -mcpu=redwood < %s | FileCheck --check-prefix=EG --check-prefix=FUNC %s ;RUN: llc -march=r600 -mcpu=redwood < %s | FileCheck --check-prefix=EG --check-prefix=FUNC %s
;FUNC-LABEL: {{^}}test_udiv: ;FUNC-LABEL: {{^}}test_udiv:
;EG: RECIP_UINT ;EG: RECIP_UINT
;EG: LSHL {{.*}}, 1, ;EG: LSHL {{.*}}, 1,
;EG: BFE_UINT ;EG: BFE_UINT
;EG: BFE_UINT ;EG: BFE_UINT
;EG: BFE_UINT ;EG: BFE_UINT
Show All 27 Lines
;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0x4f800000, ;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0x4f800000,
;GCN: v_rcp_f32_e32 ;GCN: v_rcp_f32_e32
;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x5f7ffffc ;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x5f7ffffc
;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x2f800000 ;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x2f800000
;GCN: v_trunc_f32_e32 ;GCN: v_trunc_f32_e32
;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0xcf800000 ;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0xcf800000
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @test_udiv(i64 addrspace(1)* %out, i64 %x, i64 %y) { define amdgpu_kernel void @test_udiv(i64 addrspace(1)* %out, i64 %x, i64 %y) #0 {
%result = udiv i64 %x, %y %result = udiv i64 %x, %y
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
;FUNC-LABEL: {{^}}test_urem: ;FUNC-LABEL: {{^}}test_urem:
;EG: RECIP_UINT ;EG: RECIP_UINT
;EG: BFE_UINT ;EG: BFE_UINT
Show All 30 Lines
;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0x4f800000, ;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0x4f800000,
;GCN: v_rcp_f32_e32 ;GCN: v_rcp_f32_e32
;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x5f7ffffc ;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x5f7ffffc
;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x2f800000 ;GCN: v_mul_f32_e32 v{{[0-9]+}}, 0x2f800000
;GCN: v_trunc_f32_e32 ;GCN: v_trunc_f32_e32
;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0xcf800000 ;GCN: v_mac_f32_e32 v{{[0-9]+}}, 0xcf800000
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @test_urem(i64 addrspace(1)* %out, i64 %x, i64 %y) { define amdgpu_kernel void @test_urem(i64 addrspace(1)* %out, i64 %x, i64 %y) #0 {
%result = urem i64 %x, %y %result = urem i64 %x, %y
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
;FUNC-LABEL: {{^}}test_udiv3264: ;FUNC-LABEL: {{^}}test_udiv3264:
;EG: RECIP_UINT ;EG: RECIP_UINT
;EG-NOT: BFE_UINT ;EG-NOT: BFE_UINT
;GCN-NOT: s_bfe_u32 ;GCN-NOT: s_bfe_u32
;GCN-NOT: v_mad_f32 ;GCN-NOT: v_mad_f32
;SI-NOT: v_lshr_b64 ;SI-NOT: v_lshr_b64
;VI-NOT: v_lshrrev_b64 ;VI-NOT: v_lshrrev_b64
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @test_udiv3264(i64 addrspace(1)* %out, i64 %x, i64 %y) { define amdgpu_kernel void @test_udiv3264(i64 addrspace(1)* %out, i64 %x, i64 %y) #0 {
%1 = lshr i64 %x, 33 %1 = lshr i64 %x, 33
%2 = lshr i64 %y, 33 %2 = lshr i64 %y, 33
%result = udiv i64 %1, %2 %result = udiv i64 %1, %2
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
;FUNC-LABEL: {{^}}test_urem3264: ;FUNC-LABEL: {{^}}test_urem3264:
;EG: RECIP_UINT ;EG: RECIP_UINT
;EG-NOT: BFE_UINT ;EG-NOT: BFE_UINT
;GCN-NOT: s_bfe_u32 ;GCN-NOT: s_bfe_u32
;GCN-NOT: v_mad_f32 ;GCN-NOT: v_mad_f32
;SI-NOT: v_lshr_b64 ;SI-NOT: v_lshr_b64
;VI-NOT: v_lshrrev_b64 ;VI-NOT: v_lshrrev_b64
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @test_urem3264(i64 addrspace(1)* %out, i64 %x, i64 %y) { define amdgpu_kernel void @test_urem3264(i64 addrspace(1)* %out, i64 %x, i64 %y) #0 {
%1 = lshr i64 %x, 33 %1 = lshr i64 %x, 33
%2 = lshr i64 %y, 33 %2 = lshr i64 %y, 33
%result = urem i64 %1, %2 %result = urem i64 %1, %2
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
;FUNC-LABEL: {{^}}test_udiv2364: ;FUNC-LABEL: {{^}}test_udiv2364:
;EG: UINT_TO_FLT ;EG: UINT_TO_FLT
;EG: UINT_TO_FLT ;EG: UINT_TO_FLT
;EG: FLT_TO_UINT ;EG: FLT_TO_UINT
;EG-NOT: RECIP_UINT ;EG-NOT: RECIP_UINT
;EG-NOT: BFE_UINT ;EG-NOT: BFE_UINT
;SI-NOT: v_lshr_b64 ;SI-NOT: v_lshr_b64
;VI-NOT: v_lshrrev_b64 ;VI-NOT: v_lshrrev_b64
;GCN: v_mad_f32 ;GCN: v_mad_f32
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @test_udiv2364(i64 addrspace(1)* %out, i64 %x, i64 %y) { define amdgpu_kernel void @test_udiv2364(i64 addrspace(1)* %out, i64 %x, i64 %y) #0 {
%1 = lshr i64 %x, 41 %1 = lshr i64 %x, 41
%2 = lshr i64 %y, 41 %2 = lshr i64 %y, 41
%result = udiv i64 %1, %2 %result = udiv i64 %1, %2
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
;FUNC-LABEL: {{^}}test_urem2364: ;FUNC-LABEL: {{^}}test_urem2364:
;EG: UINT_TO_FLT ;EG: UINT_TO_FLT
;EG: UINT_TO_FLT ;EG: UINT_TO_FLT
;EG: FLT_TO_UINT ;EG: FLT_TO_UINT
;EG-NOT: RECIP_UINT ;EG-NOT: RECIP_UINT
;EG-NOT: BFE_UINT ;EG-NOT: BFE_UINT
;SI-NOT: v_lshr_b64 ;SI-NOT: v_lshr_b64
;VI-NOT: v_lshrrev_b64 ;VI-NOT: v_lshrrev_b64
;GCN: v_mad_f32 ;GCN: v_mad_f32
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @test_urem2364(i64 addrspace(1)* %out, i64 %x, i64 %y) { define amdgpu_kernel void @test_urem2364(i64 addrspace(1)* %out, i64 %x, i64 %y) #0 {
%1 = lshr i64 %x, 41 %1 = lshr i64 %x, 41
%2 = lshr i64 %y, 41 %2 = lshr i64 %y, 41
%result = urem i64 %1, %2 %result = urem i64 %1, %2
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
;FUNC-LABEL: {{^}}test_udiv_k: ;FUNC-LABEL: {{^}}test_udiv_k:
;GCN: v_mul{{.+}} v{{[0-9]+}}, v{{[0-9]+}}, 24 ;GCN: v_mul{{.+}} v{{[0-9]+}}, v{{[0-9]+}}, 24
;GCN: v_mul{{.+}} v{{[0-9]+}}, v{{[0-9]+}}, 24 ;GCN: v_mul{{.+}} v{{[0-9]+}}, v{{[0-9]+}}, 24
;GCN: v_mul{{.+}} v{{[0-9]+}}, v{{[0-9]+}}, 24 ;GCN: v_mul{{.+}} v{{[0-9]+}}, v{{[0-9]+}}, 24
;GCN: v_add ;GCN: v_add
;GCN: v_addc ;GCN: v_addc
;GCN: v_addc ;GCN: v_addc
;GCN: s_endpgm ;GCN: s_endpgm
define amdgpu_kernel void @test_udiv_k(i64 addrspace(1)* %out, i64 %x) { define amdgpu_kernel void @test_udiv_k(i64 addrspace(1)* %out, i64 %x) #0 {
%result = udiv i64 24, %x %result = udiv i64 24, %x
store i64 %result, i64 addrspace(1)* %out store i64 %result, i64 addrspace(1)* %out
ret void ret void
} }
attributes #0 = { "denormal-fp-math-f32"="preserve-sign,preserve-sign" }

llvm/test/CodeGen/AMDGPU/v_mac.ll

; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mattr=-fp32-denormals -verify-machineinstrs < %s | FileCheck -check-prefix=SI -check-prefix=GCN %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -denormal-fp-math-f32=preserve-sign -verify-machineinstrs < %s | FileCheck -check-prefix=SI -check-prefix=GCN %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tonga -mattr=-fp32-denormals,-fp64-fp16-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=VI -check-prefix=VI-FLUSH -check-prefix=GCN %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tonga -denormal-fp-math=preserve-sign -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=VI -check-prefix=VI-FLUSH -check-prefix=GCN %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tonga -mattr=-fp32-denormals,+fp64-fp16-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=VI -check-prefix=VI-DENORM -check-prefix=GCN %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tonga -denormal-fp-math=ieee -denormal-fp-math-f32=preserve-sign -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -check-prefix=VI -check-prefix=VI-DENORM -check-prefix=GCN %s
; GCN-LABEL: {{^}}mac_vvv: ; GCN-LABEL: {{^}}mac_vvv:
; GCN: buffer_load_dword [[A:v[0-9]+]], off, s[{{[0-9]+:[0-9]+}}], 0{{$}} ; GCN: buffer_load_dword [[A:v[0-9]+]], off, s[{{[0-9]+:[0-9]+}}], 0{{$}}
; GCN: buffer_load_dword [[B:v[0-9]+]], off, s[{{[0-9]+:[0-9]+}}], 0 offset:4 ; GCN: buffer_load_dword [[B:v[0-9]+]], off, s[{{[0-9]+:[0-9]+}}], 0 offset:4
; GCN: buffer_load_dword [[C:v[0-9]+]], off, s[{{[0-9]+:[0-9]+}}], 0 offset:8 ; GCN: buffer_load_dword [[C:v[0-9]+]], off, s[{{[0-9]+:[0-9]+}}], 0 offset:8
; GCN: v_mac_f32_e32 [[C]], [[A]], [[B]] ; GCN: v_mac_f32_e32 [[C]], [[A]], [[B]]
; GCN: buffer_store_dword [[C]] ; GCN: buffer_store_dword [[C]]
define amdgpu_kernel void @mac_vvv(float addrspace(1)* %out, float addrspace(1)* %in) #0 { define amdgpu_kernel void @mac_vvv(float addrspace(1)* %out, float addrspace(1)* %in) #0 {
▲ Show 20 Lines • Show All 276 Lines • Show Last 20 Lines

llvm/test/CodeGen/AMDGPU/v_mac_f16.ll

; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -mattr=-fp32-denormals,-fp64-fp16-denormals -verify-machineinstrs < %s | FileCheck -allow-deprecated-dag-overlap -check-prefix=GCN -check-prefix=SI %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=tahiti -verify-machineinstrs < %s | FileCheck -allow-deprecated-dag-overlap -check-prefix=GCN -check-prefix=SI %s
; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=fiji -mattr=-fp32-denormals,-fp64-fp16-denormals,-flat-for-global -verify-machineinstrs < %s | FileCheck -allow-deprecated-dag-overlap -check-prefix=GCN -check-prefix=VI %s ; RUN: llc -amdgpu-scalarize-global-loads=false -march=amdgcn -mcpu=fiji -mattr=-flat-for-global -verify-machineinstrs < %s | FileCheck -allow-deprecated-dag-overlap -check-prefix=GCN -check-prefix=VI %s
; GCN-LABEL: {{^}}mac_f16: ; GCN-LABEL: {{^}}mac_f16:
; GCN: {{buffer|flat}}_load_ushort v[[A_F16:[0-9]+]] ; GCN: {{buffer|flat}}_load_ushort v[[A_F16:[0-9]+]]
; GCN: {{buffer|flat}}_load_ushort v[[B_F16:[0-9]+]] ; GCN: {{buffer|flat}}_load_ushort v[[B_F16:[0-9]+]]
; GCN: {{buffer|flat}}_load_ushort v[[C_F16:[0-9]+]] ; GCN: {{buffer|flat}}_load_ushort v[[C_F16:[0-9]+]]
; SI: v_cvt_f32_f16_e32 v[[A_F32:[0-9]+]], v[[A_F16]] ; SI: v_cvt_f32_f16_e32 v[[A_F32:[0-9]+]], v[[A_F16]]
; SI: v_cvt_f32_f16_e32 v[[B_F32:[0-9]+]], v[[B_F16]] ; SI: v_cvt_f32_f16_e32 v[[B_F32:[0-9]+]], v[[B_F16]]
; SI: v_cvt_f32_f16_e32 v[[C_F32:[0-9]+]], v[[C_F16]] ; SI: v_cvt_f32_f16_e32 v[[C_F32:[0-9]+]], v[[C_F16]]
▲ Show 20 Lines • Show All 661 Lines • ▼ Show 20 Linesentry:
%r.val = fadd <2 x half> %t.val, %c.neg %r.val = fadd <2 x half> %t.val, %c.neg
store <2 x half> %r.val, <2 x half> addrspace(1)* %r store <2 x half> %r.val, <2 x half> addrspace(1)* %r
ret void ret void
} }
declare void @llvm.amdgcn.s.barrier() #2 declare void @llvm.amdgcn.s.barrier() #2
attributes #0 = { nounwind "no-signed-zeros-fp-math"="false" } attributes #0 = { nounwind "no-signed-zeros-fp-math"="false" "denormal-fp-math"="preserve-sign,preserve-sign" }
attributes #1 = { nounwind "no-signed-zeros-fp-math"="true" } attributes #1 = { nounwind "no-signed-zeros-fp-math"="true" "denormal-fp-math"="preserve-sign,preserve-sign" }
attributes #2 = { nounwind convergent } attributes #2 = { nounwind convergent }

llvm/test/CodeGen/AMDGPU/v_madak_f16.ll

; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
; RUN: llc < %s -mtriple=amdgcn-- -mcpu=tahiti -mattr=-fp32-denormals,-fp64-fp16-denormals -verify-machineinstrs | FileCheck %s -check-prefixes=GCN,SI ; RUN: llc < %s -mtriple=amdgcn-- -mcpu=tahiti -verify-machineinstrs | FileCheck %s -check-prefixes=GCN,SI
; RUN: llc < %s -mtriple=amdgcn-- -mcpu=fiji -mattr=-fp32-denormals,-fp64-fp16-denormals,-flat-for-global -verify-machineinstrs | FileCheck %s -check-prefixes=GCN,VI ; RUN: llc < %s -mtriple=amdgcn-- -mcpu=fiji -mattr=-flat-for-global -verify-machineinstrs | FileCheck %s -check-prefixes=GCN,VI
define amdgpu_kernel void @madak_f16( define amdgpu_kernel void @madak_f16(
; SI-LABEL: madak_f16: ; SI-LABEL: madak_f16:
; SI: ; %bb.0: ; %entry ; SI: ; %bb.0: ; %entry
; SI-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9 ; SI-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x9
; SI-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd ; SI-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0xd
; SI-NEXT: s_mov_b32 s11, 0xf000 ; SI-NEXT: s_mov_b32 s11, 0xf000
; SI-NEXT: s_mov_b32 s10, -1 ; SI-NEXT: s_mov_b32 s10, -1
Show All 35 Lines
; VI-NEXT: buffer_load_ushort v0, off, s[4:7], 0 ; VI-NEXT: buffer_load_ushort v0, off, s[4:7], 0
; VI-NEXT: buffer_load_ushort v1, off, s[8:11], 0 ; VI-NEXT: buffer_load_ushort v1, off, s[8:11], 0
; VI-NEXT: s_waitcnt vmcnt(0) ; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: v_madak_f16 v0, v0, v1, 0x4900 ; VI-NEXT: v_madak_f16 v0, v0, v1, 0x4900
; VI-NEXT: buffer_store_short v0, off, s[0:3], 0 ; VI-NEXT: buffer_store_short v0, off, s[0:3], 0
; VI-NEXT: s_endpgm ; VI-NEXT: s_endpgm
half addrspace(1)* %r, half addrspace(1)* %r,
half addrspace(1)* %a, half addrspace(1)* %a,
half addrspace(1)* %b) { half addrspace(1)* %b) #0 {
entry: entry:
%a.val = load half, half addrspace(1)* %a %a.val = load half, half addrspace(1)* %a
%b.val = load half, half addrspace(1)* %b %b.val = load half, half addrspace(1)* %b
%t.val = fmul half %a.val, %b.val %t.val = fmul half %a.val, %b.val
%r.val = fadd half %t.val, 10.0 %r.val = fadd half %t.val, 10.0
store half %r.val, half addrspace(1)* %r store half %r.val, half addrspace(1)* %r
▲ Show 20 Lines • Show All 67 Lines • ▼ Show 20 Lines
; VI-NEXT: v_mac_f16_e32 v2, v0, v3 ; VI-NEXT: v_mac_f16_e32 v2, v0, v3
; VI-NEXT: buffer_store_short v1, off, s[4:7], 0 ; VI-NEXT: buffer_store_short v1, off, s[4:7], 0
; VI-NEXT: buffer_store_short v2, off, s[0:3], 0 ; VI-NEXT: buffer_store_short v2, off, s[0:3], 0
; VI-NEXT: s_endpgm ; VI-NEXT: s_endpgm
half addrspace(1)* %r0, half addrspace(1)* %r0,
half addrspace(1)* %r1, half addrspace(1)* %r1,
half addrspace(1)* %a, half addrspace(1)* %a,
half addrspace(1)* %b, half addrspace(1)* %b,
half addrspace(1)* %c) { half addrspace(1)* %c) #0 {
entry: entry:
%a.val = load volatile half, half addrspace(1)* %a %a.val = load volatile half, half addrspace(1)* %a
%b.val = load volatile half, half addrspace(1)* %b %b.val = load volatile half, half addrspace(1)* %b
%c.val = load volatile half, half addrspace(1)* %c %c.val = load volatile half, half addrspace(1)* %c
%t0.val = fmul half %a.val, %b.val %t0.val = fmul half %a.val, %b.val
%t1.val = fmul half %a.val, %c.val %t1.val = fmul half %a.val, %c.val
%r0.val = fadd half %t0.val, 10.0 %r0.val = fadd half %t0.val, 10.0
%r1.val = fadd half %t1.val, 10.0 %r1.val = fadd half %t1.val, 10.0
store half %r0.val, half addrspace(1)* %r0 store half %r0.val, half addrspace(1)* %r0
store half %r1.val, half addrspace(1)* %r1 store half %r1.val, half addrspace(1)* %r1
ret void ret void
} }
attributes #0 = { "denormal-fp-math"="preserve-sign,preserve-sign" }