This is an archive of the discontinued LLVM Phabricator instance.

Differential D11741

[TTI] Make the cost APIs in TargetTransformInfo consistently use 'int' rather than 'unsigned' for their costs.
ClosedPublic

Authored by chandlerc on Aug 3 2015, 4:55 PM.

Details

Reviewers
echristo
mcrosier
Commits
rG93205eb966bb: [TTI] Make the cost APIs in TargetTransformInfo consistently use 'int' rather…
rL244080: [TTI] Make the cost APIs in TargetTransformInfo consistently use 'int'
Summary

For something like costs in particular there is a natural "negative"
value, that of savings or saved cost. As a consequence, there is a lot
of code that subtracts or creates negative values based on cost, all of
which is prone to awkwardness or bugs when dealing with an unsigned
type. Similarly, we *never* want these values to wrap, as that would
cause Very Bad code generation (likely percieved as an infinite loop as
we try to emit over 2^32 instructions or some such insanity).

All around 'int' seems a much better fit for these basic metrics.

No functional change intended.

Diff Detail

Event Timeline

chandlerc updated this revision to Diff 31286.Aug 3 2015, 4:55 PM
chandlerc retitled this revision from to [TTI] Make the cost APIs in TargetTransformInfo consistently use 'int' rather than 'unsigned' for their costs..
chandlerc updated this object.
chandlerc added a reviewer: echristo.
chandlerc added a subscriber: llvm-commits.
Herald added a subscriber: jholewinski. · View Herald TranscriptAug 3 2015, 4:55 PM
sanjoy added a subscriber: sanjoy.Aug 3 2015, 5:03 PM

I have two comments:

  • I'd be scared of existing code that returns -1 to mean "very expensive". Now that same value means "cheaper than free". I think these should be fixed, but is there a way to automatically flush these out?
  • Do you intend to add callbacks that actually return negative costs? Or is this meant to make intermediate computation more ergonomic?
chandlerc added a comment.Aug 3 2015, 5:40 PM
In D11741#217335, @sanjoy wrote:

I have two comments:

  • I'd be scared of existing code that returns -1 to mean "very expensive". Now that same value means "cheaper than free". I think these should be fixed, but is there a way to automatically flush these out?

Probably, I think I can add some asserts to check for this. But I've also read through most of this code and don't see anyone doing that. Still, better to check.

  • Do you intend to add callbacks that actually return negative costs? Or is this meant to make intermediate computation more ergonomic?

The latter. In particular, I have seen several cases where we might reasonably wrap up computation or aggregated calls to this API and return a potentially negative number. With aggregation this becomes more more believable. And it is quite awkward to force such code to switch types suddenly.

chandlerc updated this revision to Diff 31292.Aug 3 2015, 6:38 PM

Updated after running clang-format over all the code and adding asserts to
address Sanjoy's concerns.

chandlerc added a comment.Aug 3 2015, 6:38 PM
In D11741#217357, @chandlerc wrote:
In D11741#217335, @sanjoy wrote:

I have two comments:

  • I'd be scared of existing code that returns -1 to mean "very expensive". Now that same value means "cheaper than free". I think these should be fixed, but is there a way to automatically flush these out?

Probably, I think I can add some asserts to check for this. But I've also read through most of this code and don't see anyone doing that. Still, better to check.

Done, now with asserts that we don't misuse -1 inside of TTI.

chandlerc added a comment.Aug 5 2015, 10:52 AM

Ping, should be a simple patch...

mcrosier accepted this revision.Aug 5 2015, 11:03 AM
mcrosier added a reviewer: mcrosier.
mcrosier added a subscriber: mcrosier.

My finger now hurts from scrolling... LGTM.

This revision is now accepted and ready to land.Aug 5 2015, 11:03 AM
chandlerc added a comment.Aug 5 2015, 11:05 AM
In D11741#218034, @mcrosier wrote:

My finger now hurts from scrolling... LGTM.

Awesome, thanks!

Closed by commit rL244080: [TTI] Make the cost APIs in TargetTransformInfo consistently use 'int' (authored by chandlerc). · Explain WhyAug 5 2015, 11:08 AM
This revision was automatically updated to reflect the committed changes.

Revision Contents

Diff 31292

include/llvm/Analysis/TargetTransformInfo.h

Show First 20 Lines • Show All 91 Lines • ▼ Show 20 Linespublic:
/// \name Generic Target Information /// \name Generic Target Information
/// @{ /// @{
/// \brief Underlying constants for 'cost' values in this interface. /// \brief Underlying constants for 'cost' values in this interface.
/// ///
/// Many APIs in this interface return a cost. This enum defines the /// Many APIs in this interface return a cost. This enum defines the
/// fundamental values that should be used to interpret (and produce) those /// fundamental values that should be used to interpret (and produce) those
/// costs. The costs are returned as an unsigned rather than a member of this /// costs. The costs are returned as an int rather than a member of this
/// enumeration because it is expected that the cost of one IR instruction /// enumeration because it is expected that the cost of one IR instruction
/// may have a multiplicative factor to it or otherwise won't fit directly /// may have a multiplicative factor to it or otherwise won't fit directly
/// into the enum. Moreover, it is common to sum or average costs which works /// into the enum. Moreover, it is common to sum or average costs which works
/// better as simple integral values. Thus this enum only provides constants. /// better as simple integral values. Thus this enum only provides constants.
/// Also note that the returned costs are signed integers to make it natural
/// to add, subtract, and test with zero (a common boundary condition). It is
/// not expected that 2^32 is a realistic cost to be modeling at any point.
/// ///
/// Note that these costs should usually reflect the intersection of code-size /// Note that these costs should usually reflect the intersection of code-size
/// cost and execution cost. A free instruction is typically one that folds /// cost and execution cost. A free instruction is typically one that folds
/// into another instruction. For example, reg-to-reg moves can often be /// into another instruction. For example, reg-to-reg moves can often be
/// skipped by renaming the registers in the CPU, but they still are encoded /// skipped by renaming the registers in the CPU, but they still are encoded
/// and thus wouldn't be considered 'free' here. /// and thus wouldn't be considered 'free' here.
enum TargetCostConstants { enum TargetCostConstants {
TCC_Free = 0, ///< Expected to fold away in lowering. TCC_Free = 0, ///< Expected to fold away in lowering.
Show All 10 Linespublic:
/// analyzing a GEP's cost required more information. /// analyzing a GEP's cost required more information.
/// ///
/// Typically only the result type is required, and the operand type can be /// Typically only the result type is required, and the operand type can be
/// omitted. However, if the opcode is one of the cast instructions, the /// omitted. However, if the opcode is one of the cast instructions, the
/// operand type is required. /// operand type is required.
/// ///
/// The returned cost is defined in terms of \c TargetCostConstants, see its /// The returned cost is defined in terms of \c TargetCostConstants, see its
/// comments for a detailed explanation of the cost values. /// comments for a detailed explanation of the cost values.
unsigned getOperationCost(unsigned Opcode, Type *Ty, int getOperationCost(unsigned Opcode, Type *Ty, Type *OpTy = nullptr) const;
Type *OpTy = nullptr) const;
/// \brief Estimate the cost of a GEP operation when lowered. /// \brief Estimate the cost of a GEP operation when lowered.
/// ///
/// The contract for this function is the same as \c getOperationCost except /// The contract for this function is the same as \c getOperationCost except
/// that it supports an interface that provides extra information specific to /// that it supports an interface that provides extra information specific to
/// the GEP operation. /// the GEP operation.
unsigned getGEPCost(Type *PointeeType, const Value *Ptr, int getGEPCost(Type *PointeeType, const Value *Ptr,
ArrayRef<const Value *> Operands) const; ArrayRef<const Value *> Operands) const;
/// \brief Estimate the cost of a function call when lowered. /// \brief Estimate the cost of a function call when lowered.
/// ///
/// The contract for this is the same as \c getOperationCost except that it /// The contract for this is the same as \c getOperationCost except that it
/// supports an interface that provides extra information specific to call /// supports an interface that provides extra information specific to call
/// instructions. /// instructions.
/// ///
/// This is the most basic query for estimating call cost: it only knows the /// This is the most basic query for estimating call cost: it only knows the
/// function type and (potentially) the number of arguments at the call site. /// function type and (potentially) the number of arguments at the call site.
/// The latter is only interesting for varargs function types. /// The latter is only interesting for varargs function types.
unsigned getCallCost(FunctionType *FTy, int NumArgs = -1) const; int getCallCost(FunctionType *FTy, int NumArgs = -1) const;
/// \brief Estimate the cost of calling a specific function when lowered. /// \brief Estimate the cost of calling a specific function when lowered.
/// ///
/// This overload adds the ability to reason about the particular function /// This overload adds the ability to reason about the particular function
/// being called in the event it is a library call with special lowering. /// being called in the event it is a library call with special lowering.
unsigned getCallCost(const Function *F, int NumArgs = -1) const; int getCallCost(const Function *F, int NumArgs = -1) const;
/// \brief Estimate the cost of calling a specific function when lowered. /// \brief Estimate the cost of calling a specific function when lowered.
/// ///
/// This overload allows specifying a set of candidate argument values. /// This overload allows specifying a set of candidate argument values.
unsigned getCallCost(const Function *F, int getCallCost(const Function *F, ArrayRef<const Value *> Arguments) const;
ArrayRef<const Value *> Arguments) const;
/// \brief Estimate the cost of an intrinsic when lowered. /// \brief Estimate the cost of an intrinsic when lowered.
/// ///
/// Mirrors the \c getCallCost method but uses an intrinsic identifier. /// Mirrors the \c getCallCost method but uses an intrinsic identifier.
unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy, int getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<Type *> ParamTys) const; ArrayRef<Type *> ParamTys) const;
/// \brief Estimate the cost of an intrinsic when lowered. /// \brief Estimate the cost of an intrinsic when lowered.
/// ///
/// Mirrors the \c getCallCost method but uses an intrinsic identifier. /// Mirrors the \c getCallCost method but uses an intrinsic identifier.
unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy, int getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<const Value *> Arguments) const; ArrayRef<const Value *> Arguments) const;
/// \brief Estimate the cost of a given IR user when lowered. /// \brief Estimate the cost of a given IR user when lowered.
/// ///
/// This can estimate the cost of either a ConstantExpr or Instruction when /// This can estimate the cost of either a ConstantExpr or Instruction when
/// lowered. It has two primary advantages over the \c getOperationCost and /// lowered. It has two primary advantages over the \c getOperationCost and
/// \c getGEPCost above, and one significant disadvantage: it can only be /// \c getGEPCost above, and one significant disadvantage: it can only be
/// used when the IR construct has already been formed. /// used when the IR construct has already been formed.
/// ///
/// The advantages are that it can inspect the SSA use graph to reason more /// The advantages are that it can inspect the SSA use graph to reason more
/// accurately about the cost. For example, all-constant-GEPs can often be /// accurately about the cost. For example, all-constant-GEPs can often be
/// folded into a load or other instruction, but if they are used in some /// folded into a load or other instruction, but if they are used in some
/// other context they may not be folded. This routine can distinguish such /// other context they may not be folded. This routine can distinguish such
/// cases. /// cases.
/// ///
/// The returned cost is defined in terms of \c TargetCostConstants, see its /// The returned cost is defined in terms of \c TargetCostConstants, see its
/// comments for a detailed explanation of the cost values. /// comments for a detailed explanation of the cost values.
unsigned getUserCost(const User *U) const; int getUserCost(const User *U) const;
/// \brief Return true if branch divergence exists. /// \brief Return true if branch divergence exists.
/// ///
/// Branch divergence has a significantly negative impact on GPU performance /// Branch divergence has a significantly negative impact on GPU performance
/// when threads in the same wavefront take different paths due to conditional /// when threads in the same wavefront take different paths due to conditional
/// branches. /// branches.
bool hasBranchDivergence() const; bool hasBranchDivergence() const;
▲ Show 20 Lines • Show All 158 Lines • ▼ Show 20 Linespublic:
/// \brief Return hardware support for population count. /// \brief Return hardware support for population count.
PopcntSupportKind getPopcntSupport(unsigned IntTyWidthInBit) const; PopcntSupportKind getPopcntSupport(unsigned IntTyWidthInBit) const;
/// \brief Return true if the hardware has a fast square-root instruction. /// \brief Return true if the hardware has a fast square-root instruction.
bool haveFastSqrt(Type *Ty) const; bool haveFastSqrt(Type *Ty) const;
/// \brief Return the expected cost of supporting the floating point operation /// \brief Return the expected cost of supporting the floating point operation
/// of the specified type. /// of the specified type.
unsigned getFPOpCost(Type *Ty) const; int getFPOpCost(Type *Ty) const;
/// \brief Return the expected cost of materializing for the given integer /// \brief Return the expected cost of materializing for the given integer
/// immediate of the specified type. /// immediate of the specified type.
unsigned getIntImmCost(const APInt &Imm, Type *Ty) const; int getIntImmCost(const APInt &Imm, Type *Ty) const;
/// \brief Return the expected cost of materialization for the given integer /// \brief Return the expected cost of materialization for the given integer
/// immediate of the specified type for a given instruction. The cost can be /// immediate of the specified type for a given instruction. The cost can be
/// zero if the immediate can be folded into the specified instruction. /// zero if the immediate can be folded into the specified instruction.
unsigned getIntImmCost(unsigned Opc, unsigned Idx, const APInt &Imm, int getIntImmCost(unsigned Opc, unsigned Idx, const APInt &Imm,
Type *Ty) const; Type *Ty) const;
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm, int getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty) const; Type *Ty) const;
/// @} /// @}
/// \name Vector Target Information /// \name Vector Target Information
/// @{ /// @{
/// \brief The various kinds of shuffle patterns for vector queries. /// \brief The various kinds of shuffle patterns for vector queries.
enum ShuffleKind { enum ShuffleKind {
SK_Broadcast, ///< Broadcast element 0 to all other elements. SK_Broadcast, ///< Broadcast element 0 to all other elements.
Show All 23 Linespublic:
unsigned getRegisterBitWidth(bool Vector) const; unsigned getRegisterBitWidth(bool Vector) const;
/// \return The maximum interleave factor that any transform should try to /// \return The maximum interleave factor that any transform should try to
/// perform for this target. This number depends on the level of parallelism /// perform for this target. This number depends on the level of parallelism
/// and the number of execution units in the CPU. /// and the number of execution units in the CPU.
unsigned getMaxInterleaveFactor(unsigned VF) const; unsigned getMaxInterleaveFactor(unsigned VF) const;
/// \return The expected cost of arithmetic ops, such as mul, xor, fsub, etc. /// \return The expected cost of arithmetic ops, such as mul, xor, fsub, etc.
unsigned int getArithmeticInstrCost(
getArithmeticInstrCost(unsigned Opcode, Type *Ty, unsigned Opcode, Type *Ty, OperandValueKind Opd1Info = OK_AnyValue,
OperandValueKind Opd1Info = OK_AnyValue,
OperandValueKind Opd2Info = OK_AnyValue, OperandValueKind Opd2Info = OK_AnyValue,
OperandValueProperties Opd1PropInfo = OP_None, OperandValueProperties Opd1PropInfo = OP_None,
OperandValueProperties Opd2PropInfo = OP_None) const; OperandValueProperties Opd2PropInfo = OP_None) const;
/// \return The cost of a shuffle instruction of kind Kind and of type Tp. /// \return The cost of a shuffle instruction of kind Kind and of type Tp.
/// The index and subtype parameters are used by the subvector insertion and /// The index and subtype parameters are used by the subvector insertion and
/// extraction shuffle kinds. /// extraction shuffle kinds.
unsigned getShuffleCost(ShuffleKind Kind, Type *Tp, int Index = 0, int getShuffleCost(ShuffleKind Kind, Type *Tp, int Index = 0,
Type *SubTp = nullptr) const; Type *SubTp = nullptr) const;
/// \return The expected cost of cast instructions, such as bitcast, trunc, /// \return The expected cost of cast instructions, such as bitcast, trunc,
/// zext, etc. /// zext, etc.
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) const; int getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) const;
/// \return The expected cost of control-flow related instructions such as /// \return The expected cost of control-flow related instructions such as
/// Phi, Ret, Br. /// Phi, Ret, Br.
unsigned getCFInstrCost(unsigned Opcode) const; int getCFInstrCost(unsigned Opcode) const;
/// \returns The expected cost of compare and select instructions. /// \returns The expected cost of compare and select instructions.
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, int getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy = nullptr) const; Type *CondTy = nullptr) const;
/// \return The expected cost of vector Insert and Extract. /// \return The expected cost of vector Insert and Extract.
/// Use -1 to indicate that there is no information on the index value. /// Use -1 to indicate that there is no information on the index value.
unsigned getVectorInstrCost(unsigned Opcode, Type *Val, int getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index = -1) const;
unsigned Index = -1) const;
/// \return The cost of Load and Store instructions. /// \return The cost of Load and Store instructions.
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, int getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) const; unsigned AddressSpace) const;
/// \return The cost of masked Load and Store instructions. /// \return The cost of masked Load and Store instructions.
unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, int getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) const; unsigned AddressSpace) const;
/// \return The cost of the interleaved memory operation. /// \return The cost of the interleaved memory operation.
/// \p Opcode is the memory operation code /// \p Opcode is the memory operation code
/// \p VecTy is the vector type of the interleaved access. /// \p VecTy is the vector type of the interleaved access.
/// \p Factor is the interleave factor /// \p Factor is the interleave factor
/// \p Indices is the indices for interleaved load members (as interleaved /// \p Indices is the indices for interleaved load members (as interleaved
/// load allows gaps) /// load allows gaps)
/// \p Alignment is the alignment of the memory operation /// \p Alignment is the alignment of the memory operation
/// \p AddressSpace is address space of the pointer. /// \p AddressSpace is address space of the pointer.
unsigned getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy, int getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy, unsigned Factor,
unsigned Factor, ArrayRef<unsigned> Indices, unsigned Alignment,
ArrayRef<unsigned> Indices,
unsigned Alignment,
unsigned AddressSpace) const; unsigned AddressSpace) const;
/// \brief Calculate the cost of performing a vector reduction. /// \brief Calculate the cost of performing a vector reduction.
/// ///
/// This is the cost of reducing the vector value of type \p Ty to a scalar /// This is the cost of reducing the vector value of type \p Ty to a scalar
/// value using the operation denoted by \p Opcode. The form of the reduction /// value using the operation denoted by \p Opcode. The form of the reduction
/// can either be a pairwise reduction or a reduction that splits the vector /// can either be a pairwise reduction or a reduction that splits the vector
/// at every reduction level. /// at every reduction level.
/// ///
/// Pairwise: /// Pairwise:
/// (v0, v1, v2, v3) /// (v0, v1, v2, v3)
/// ((v0+v1), (v2, v3), undef, undef) /// ((v0+v1), (v2, v3), undef, undef)
/// Split: /// Split:
/// (v0, v1, v2, v3) /// (v0, v1, v2, v3)
/// ((v0+v2), (v1+v3), undef, undef) /// ((v0+v2), (v1+v3), undef, undef)
unsigned getReductionCost(unsigned Opcode, Type *Ty, int getReductionCost(unsigned Opcode, Type *Ty, bool IsPairwiseForm) const;
bool IsPairwiseForm) const;
/// \returns The cost of Intrinsic instructions. /// \returns The cost of Intrinsic instructions.
unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy, int getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
ArrayRef<Type *> Tys) const; ArrayRef<Type *> Tys) const;
/// \returns The cost of Call instructions. /// \returns The cost of Call instructions.
unsigned getCallInstrCost(Function *F, Type *RetTy, int getCallInstrCost(Function *F, Type *RetTy, ArrayRef<Type *> Tys) const;
ArrayRef<Type *> Tys) const;
/// \returns The number of pieces into which the provided type must be /// \returns The number of pieces into which the provided type must be
/// split during legalization. Zero is returned when the answer is unknown. /// split during legalization. Zero is returned when the answer is unknown.
unsigned getNumberOfParts(Type *Tp) const; unsigned getNumberOfParts(Type *Tp) const;
/// \returns The cost of the address computation. For most targets this can be /// \returns The cost of the address computation. For most targets this can be
/// merged into the instruction indexing mode. Some targets might want to /// merged into the instruction indexing mode. Some targets might want to
/// distinguish between address computation for memory operations on vector /// distinguish between address computation for memory operations on vector
/// types and scalar types. Such targets should override this function. /// types and scalar types. Such targets should override this function.
/// The 'IsComplex' parameter is a hint that the address computation is likely /// The 'IsComplex' parameter is a hint that the address computation is likely
/// to involve multiple instructions and as such unlikely to be merged into /// to involve multiple instructions and as such unlikely to be merged into
/// the address indexing mode. /// the address indexing mode.
unsigned getAddressComputationCost(Type *Ty, bool IsComplex = false) const; int getAddressComputationCost(Type *Ty, bool IsComplex = false) const;
/// \returns The cost, if any, of keeping values of the given types alive /// \returns The cost, if any, of keeping values of the given types alive
/// over a callsite. /// over a callsite.
/// ///
/// Some types may require the use of register classes that do not have /// Some types may require the use of register classes that do not have
/// any callee-saved registers, so would require a spill and fill. /// any callee-saved registers, so would require a spill and fill.
unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) const; unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) const;
Show All 28 Linesprivate:
std::unique_ptr<Concept> TTIImpl; std::unique_ptr<Concept> TTIImpl;
}; };
class TargetTransformInfo::Concept { class TargetTransformInfo::Concept {
public: public:
virtual ~Concept() = 0; virtual ~Concept() = 0;
virtual const DataLayout &getDataLayout() const = 0; virtual const DataLayout &getDataLayout() const = 0;
virtual unsigned getOperationCost(unsigned Opcode, Type *Ty, Type *OpTy) = 0; virtual int getOperationCost(unsigned Opcode, Type *Ty, Type *OpTy) = 0;
virtual unsigned getGEPCost(Type *PointeeType, const Value *Ptr, virtual int getGEPCost(Type *PointeeType, const Value *Ptr,
ArrayRef<const Value *> Operands) = 0; ArrayRef<const Value *> Operands) = 0;
virtual unsigned getCallCost(FunctionType *FTy, int NumArgs) = 0; virtual int getCallCost(FunctionType *FTy, int NumArgs) = 0;
virtual unsigned getCallCost(const Function *F, int NumArgs) = 0; virtual int getCallCost(const Function *F, int NumArgs) = 0;
virtual unsigned getCallCost(const Function *F, virtual int getCallCost(const Function *F,
ArrayRef<const Value *> Arguments) = 0; ArrayRef<const Value *> Arguments) = 0;
virtual unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy, virtual int getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<Type *> ParamTys) = 0; ArrayRef<Type *> ParamTys) = 0;
virtual unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy, virtual int getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<const Value *> Arguments) = 0; ArrayRef<const Value *> Arguments) = 0;
virtual unsigned getUserCost(const User *U) = 0; virtual int getUserCost(const User *U) = 0;
virtual bool hasBranchDivergence() = 0; virtual bool hasBranchDivergence() = 0;
virtual bool isSourceOfDivergence(const Value *V) = 0; virtual bool isSourceOfDivergence(const Value *V) = 0;
virtual bool isLoweredToCall(const Function *F) = 0; virtual bool isLoweredToCall(const Function *F) = 0;
virtual void getUnrollingPreferences(Loop *L, UnrollingPreferences &UP) = 0; virtual void getUnrollingPreferences(Loop *L, UnrollingPreferences &UP) = 0;
virtual bool isLegalAddImmediate(int64_t Imm) = 0; virtual bool isLegalAddImmediate(int64_t Imm) = 0;
virtual bool isLegalICmpImmediate(int64_t Imm) = 0; virtual bool isLegalICmpImmediate(int64_t Imm) = 0;
virtual bool isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV, virtual bool isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV,
int64_t BaseOffset, bool HasBaseReg, int64_t BaseOffset, bool HasBaseReg,
Show All 9 Linespublic:
virtual bool isProfitableToHoist(Instruction *I) = 0; virtual bool isProfitableToHoist(Instruction *I) = 0;
virtual bool isTypeLegal(Type *Ty) = 0; virtual bool isTypeLegal(Type *Ty) = 0;
virtual unsigned getJumpBufAlignment() = 0; virtual unsigned getJumpBufAlignment() = 0;
virtual unsigned getJumpBufSize() = 0; virtual unsigned getJumpBufSize() = 0;
virtual bool shouldBuildLookupTables() = 0; virtual bool shouldBuildLookupTables() = 0;
virtual bool enableAggressiveInterleaving(bool LoopHasReductions) = 0; virtual bool enableAggressiveInterleaving(bool LoopHasReductions) = 0;
virtual PopcntSupportKind getPopcntSupport(unsigned IntTyWidthInBit) = 0; virtual PopcntSupportKind getPopcntSupport(unsigned IntTyWidthInBit) = 0;
virtual bool haveFastSqrt(Type *Ty) = 0; virtual bool haveFastSqrt(Type *Ty) = 0;
virtual unsigned getFPOpCost(Type *Ty) = 0; virtual int getFPOpCost(Type *Ty) = 0;
virtual unsigned getIntImmCost(const APInt &Imm, Type *Ty) = 0; virtual int getIntImmCost(const APInt &Imm, Type *Ty) = 0;
virtual unsigned getIntImmCost(unsigned Opc, unsigned Idx, const APInt &Imm, virtual int getIntImmCost(unsigned Opc, unsigned Idx, const APInt &Imm,
Type *Ty) = 0;
virtual int getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty) = 0; Type *Ty) = 0;
virtual unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx,
const APInt &Imm, Type *Ty) = 0;
virtual unsigned getNumberOfRegisters(bool Vector) = 0; virtual unsigned getNumberOfRegisters(bool Vector) = 0;
virtual unsigned getRegisterBitWidth(bool Vector) = 0; virtual unsigned getRegisterBitWidth(bool Vector) = 0;
virtual unsigned getMaxInterleaveFactor(unsigned VF) = 0; virtual unsigned getMaxInterleaveFactor(unsigned VF) = 0;
virtual unsigned virtual unsigned
getArithmeticInstrCost(unsigned Opcode, Type *Ty, OperandValueKind Opd1Info, getArithmeticInstrCost(unsigned Opcode, Type *Ty, OperandValueKind Opd1Info,
OperandValueKind Opd2Info, OperandValueKind Opd2Info,
OperandValueProperties Opd1PropInfo, OperandValueProperties Opd1PropInfo,
OperandValueProperties Opd2PropInfo) = 0; OperandValueProperties Opd2PropInfo) = 0;
virtual unsigned getShuffleCost(ShuffleKind Kind, Type *Tp, int Index, virtual int getShuffleCost(ShuffleKind Kind, Type *Tp, int Index,
Type *SubTp) = 0; Type *SubTp) = 0;
virtual unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) = 0; virtual int getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) = 0;
virtual unsigned getCFInstrCost(unsigned Opcode) = 0; virtual int getCFInstrCost(unsigned Opcode) = 0;
virtual unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, virtual int getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy) = 0; Type *CondTy) = 0;
virtual unsigned getVectorInstrCost(unsigned Opcode, Type *Val, virtual int getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index) = 0; unsigned Index) = 0;
virtual unsigned getMemoryOpCost(unsigned Opcode, Type *Src, virtual int getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned Alignment,
unsigned AddressSpace) = 0; unsigned AddressSpace) = 0;
virtual unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src, virtual int getMaskedMemoryOpCost(unsigned Opcode, Type *Src,
unsigned Alignment, unsigned Alignment,
unsigned AddressSpace) = 0; unsigned AddressSpace) = 0;
virtual unsigned getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy, virtual int getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy,
unsigned Factor, unsigned Factor,
ArrayRef<unsigned> Indices, ArrayRef<unsigned> Indices,
unsigned Alignment, unsigned Alignment,
unsigned AddressSpace) = 0; unsigned AddressSpace) = 0;
virtual unsigned getReductionCost(unsigned Opcode, Type *Ty, virtual int getReductionCost(unsigned Opcode, Type *Ty,
bool IsPairwiseForm) = 0; bool IsPairwiseForm) = 0;
virtual unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy, virtual int getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
ArrayRef<Type *> Tys) = 0; ArrayRef<Type *> Tys) = 0;
virtual unsigned getCallInstrCost(Function *F, Type *RetTy, virtual int getCallInstrCost(Function *F, Type *RetTy,
ArrayRef<Type *> Tys) = 0; ArrayRef<Type *> Tys) = 0;
virtual unsigned getNumberOfParts(Type *Tp) = 0; virtual unsigned getNumberOfParts(Type *Tp) = 0;
virtual unsigned getAddressComputationCost(Type *Ty, bool IsComplex) = 0; virtual int getAddressComputationCost(Type *Ty, bool IsComplex) = 0;
virtual unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) = 0; virtual unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) = 0;
virtual bool getTgtMemIntrinsic(IntrinsicInst *Inst, virtual bool getTgtMemIntrinsic(IntrinsicInst *Inst,
MemIntrinsicInfo &Info) = 0; MemIntrinsicInfo &Info) = 0;
virtual Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst, virtual Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst,
Type *ExpectedType) = 0; Type *ExpectedType) = 0;
virtual bool areInlineCompatible(const Function *Caller, virtual bool areInlineCompatible(const Function *Caller,
const Function *Callee) const = 0; const Function *Callee) const = 0;
}; };
template <typename T> template <typename T>
class TargetTransformInfo::Model final : public TargetTransformInfo::Concept { class TargetTransformInfo::Model final : public TargetTransformInfo::Concept {
T Impl; T Impl;
public: public:
Model(T Impl) : Impl(std::move(Impl)) {} Model(T Impl) : Impl(std::move(Impl)) {}
~Model() override {} ~Model() override {}
const DataLayout &getDataLayout() const override { const DataLayout &getDataLayout() const override {
return Impl.getDataLayout(); return Impl.getDataLayout();
} }
unsigned getOperationCost(unsigned Opcode, Type *Ty, Type *OpTy) override { int getOperationCost(unsigned Opcode, Type *Ty, Type *OpTy) override {
return Impl.getOperationCost(Opcode, Ty, OpTy); return Impl.getOperationCost(Opcode, Ty, OpTy);
} }
unsigned getGEPCost(Type *PointeeType, const Value *Ptr, int getGEPCost(Type *PointeeType, const Value *Ptr,
ArrayRef<const Value *> Operands) override { ArrayRef<const Value *> Operands) override {
return Impl.getGEPCost(PointeeType, Ptr, Operands); return Impl.getGEPCost(PointeeType, Ptr, Operands);
} }
unsigned getCallCost(FunctionType *FTy, int NumArgs) override { int getCallCost(FunctionType *FTy, int NumArgs) override {
return Impl.getCallCost(FTy, NumArgs); return Impl.getCallCost(FTy, NumArgs);
} }
unsigned getCallCost(const Function *F, int NumArgs) override { int getCallCost(const Function *F, int NumArgs) override {
return Impl.getCallCost(F, NumArgs); return Impl.getCallCost(F, NumArgs);
} }
unsigned getCallCost(const Function *F, int getCallCost(const Function *F,
ArrayRef<const Value *> Arguments) override { ArrayRef<const Value *> Arguments) override {
return Impl.getCallCost(F, Arguments); return Impl.getCallCost(F, Arguments);
} }
unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy, int getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<Type *> ParamTys) override { ArrayRef<Type *> ParamTys) override {
return Impl.getIntrinsicCost(IID, RetTy, ParamTys); return Impl.getIntrinsicCost(IID, RetTy, ParamTys);
} }
unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy, int getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<const Value *> Arguments) override { ArrayRef<const Value *> Arguments) override {
return Impl.getIntrinsicCost(IID, RetTy, Arguments); return Impl.getIntrinsicCost(IID, RetTy, Arguments);
} }
unsigned getUserCost(const User *U) override { return Impl.getUserCost(U); } int getUserCost(const User *U) override { return Impl.getUserCost(U); }
bool hasBranchDivergence() override { return Impl.hasBranchDivergence(); } bool hasBranchDivergence() override { return Impl.hasBranchDivergence(); }
bool isSourceOfDivergence(const Value *V) override { bool isSourceOfDivergence(const Value *V) override {
return Impl.isSourceOfDivergence(V); return Impl.isSourceOfDivergence(V);
} }
bool isLoweredToCall(const Function *F) override { bool isLoweredToCall(const Function *F) override {
return Impl.isLoweredToCall(F); return Impl.isLoweredToCall(F);
} }
void getUnrollingPreferences(Loop *L, UnrollingPreferences &UP) override { void getUnrollingPreferences(Loop *L, UnrollingPreferences &UP) override {
▲ Show 20 Lines • Show All 41 Lines • ▼ Show 20 Linespublic:
bool enableAggressiveInterleaving(bool LoopHasReductions) override { bool enableAggressiveInterleaving(bool LoopHasReductions) override {
return Impl.enableAggressiveInterleaving(LoopHasReductions); return Impl.enableAggressiveInterleaving(LoopHasReductions);
} }
PopcntSupportKind getPopcntSupport(unsigned IntTyWidthInBit) override { PopcntSupportKind getPopcntSupport(unsigned IntTyWidthInBit) override {
return Impl.getPopcntSupport(IntTyWidthInBit); return Impl.getPopcntSupport(IntTyWidthInBit);
} }
bool haveFastSqrt(Type *Ty) override { return Impl.haveFastSqrt(Ty); } bool haveFastSqrt(Type *Ty) override { return Impl.haveFastSqrt(Ty); }
unsigned getFPOpCost(Type *Ty) override { int getFPOpCost(Type *Ty) override { return Impl.getFPOpCost(Ty); }
return Impl.getFPOpCost(Ty);
}
unsigned getIntImmCost(const APInt &Imm, Type *Ty) override { int getIntImmCost(const APInt &Imm, Type *Ty) override {
return Impl.getIntImmCost(Imm, Ty); return Impl.getIntImmCost(Imm, Ty);
} }
unsigned getIntImmCost(unsigned Opc, unsigned Idx, const APInt &Imm, int getIntImmCost(unsigned Opc, unsigned Idx, const APInt &Imm,
Type *Ty) override { Type *Ty) override {
return Impl.getIntImmCost(Opc, Idx, Imm, Ty); return Impl.getIntImmCost(Opc, Idx, Imm, Ty);
} }
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm, int getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty) override { Type *Ty) override {
return Impl.getIntImmCost(IID, Idx, Imm, Ty); return Impl.getIntImmCost(IID, Idx, Imm, Ty);
} }
unsigned getNumberOfRegisters(bool Vector) override { unsigned getNumberOfRegisters(bool Vector) override {
return Impl.getNumberOfRegisters(Vector); return Impl.getNumberOfRegisters(Vector);
} }
unsigned getRegisterBitWidth(bool Vector) override { unsigned getRegisterBitWidth(bool Vector) override {
return Impl.getRegisterBitWidth(Vector); return Impl.getRegisterBitWidth(Vector);
} }
unsigned getMaxInterleaveFactor(unsigned VF) override { unsigned getMaxInterleaveFactor(unsigned VF) override {
return Impl.getMaxInterleaveFactor(VF); return Impl.getMaxInterleaveFactor(VF);
} }
unsigned unsigned
getArithmeticInstrCost(unsigned Opcode, Type *Ty, OperandValueKind Opd1Info, getArithmeticInstrCost(unsigned Opcode, Type *Ty, OperandValueKind Opd1Info,
OperandValueKind Opd2Info, OperandValueKind Opd2Info,
OperandValueProperties Opd1PropInfo, OperandValueProperties Opd1PropInfo,
OperandValueProperties Opd2PropInfo) override { OperandValueProperties Opd2PropInfo) override {
return Impl.getArithmeticInstrCost(Opcode, Ty, Opd1Info, Opd2Info, return Impl.getArithmeticInstrCost(Opcode, Ty, Opd1Info, Opd2Info,
Opd1PropInfo, Opd2PropInfo); Opd1PropInfo, Opd2PropInfo);
} }
unsigned getShuffleCost(ShuffleKind Kind, Type *Tp, int Index, int getShuffleCost(ShuffleKind Kind, Type *Tp, int Index,
Type *SubTp) override { Type *SubTp) override {
return Impl.getShuffleCost(Kind, Tp, Index, SubTp); return Impl.getShuffleCost(Kind, Tp, Index, SubTp);
} }
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) override { int getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) override {
return Impl.getCastInstrCost(Opcode, Dst, Src); return Impl.getCastInstrCost(Opcode, Dst, Src);
} }
unsigned getCFInstrCost(unsigned Opcode) override { int getCFInstrCost(unsigned Opcode) override {
return Impl.getCFInstrCost(Opcode); return Impl.getCFInstrCost(Opcode);
} }
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, int getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy) override {
Type *CondTy) override {
return Impl.getCmpSelInstrCost(Opcode, ValTy, CondTy); return Impl.getCmpSelInstrCost(Opcode, ValTy, CondTy);
} }
unsigned getVectorInstrCost(unsigned Opcode, Type *Val, int getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index) override {
unsigned Index) override {
return Impl.getVectorInstrCost(Opcode, Val, Index); return Impl.getVectorInstrCost(Opcode, Val, Index);
} }
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, int getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) override { unsigned AddressSpace) override {
return Impl.getMemoryOpCost(Opcode, Src, Alignment, AddressSpace); return Impl.getMemoryOpCost(Opcode, Src, Alignment, AddressSpace);
} }
unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, int getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) override { unsigned AddressSpace) override {
return Impl.getMaskedMemoryOpCost(Opcode, Src, Alignment, AddressSpace); return Impl.getMaskedMemoryOpCost(Opcode, Src, Alignment, AddressSpace);
} }
unsigned getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy, int getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy, unsigned Factor,
unsigned Factor, ArrayRef<unsigned> Indices, unsigned Alignment,
ArrayRef<unsigned> Indices,
unsigned Alignment,
unsigned AddressSpace) override { unsigned AddressSpace) override {
return Impl.getInterleavedMemoryOpCost(Opcode, VecTy, Factor, Indices, return Impl.getInterleavedMemoryOpCost(Opcode, VecTy, Factor, Indices,
Alignment, AddressSpace); Alignment, AddressSpace);
} }
unsigned getReductionCost(unsigned Opcode, Type *Ty, int getReductionCost(unsigned Opcode, Type *Ty,
bool IsPairwiseForm) override { bool IsPairwiseForm) override {
return Impl.getReductionCost(Opcode, Ty, IsPairwiseForm); return Impl.getReductionCost(Opcode, Ty, IsPairwiseForm);
} }
unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy, int getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
ArrayRef<Type *> Tys) override { ArrayRef<Type *> Tys) override {
return Impl.getIntrinsicInstrCost(ID, RetTy, Tys); return Impl.getIntrinsicInstrCost(ID, RetTy, Tys);
} }
unsigned getCallInstrCost(Function *F, Type *RetTy, int getCallInstrCost(Function *F, Type *RetTy,
ArrayRef<Type *> Tys) override { ArrayRef<Type *> Tys) override {
return Impl.getCallInstrCost(F, RetTy, Tys); return Impl.getCallInstrCost(F, RetTy, Tys);
} }
unsigned getNumberOfParts(Type *Tp) override { unsigned getNumberOfParts(Type *Tp) override {
return Impl.getNumberOfParts(Tp); return Impl.getNumberOfParts(Tp);
} }
unsigned getAddressComputationCost(Type *Ty, bool IsComplex) override { int getAddressComputationCost(Type *Ty, bool IsComplex) override {
return Impl.getAddressComputationCost(Ty, IsComplex); return Impl.getAddressComputationCost(Ty, IsComplex);
} }
unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) override { unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) override {
return Impl.getCostOfKeepingLiveOverCall(Tys); return Impl.getCostOfKeepingLiveOverCall(Tys);
} }
bool getTgtMemIntrinsic(IntrinsicInst *Inst, bool getTgtMemIntrinsic(IntrinsicInst *Inst,
MemIntrinsicInfo &Info) override { MemIntrinsicInfo &Info) override {
return Impl.getTgtMemIntrinsic(Inst, Info); return Impl.getTgtMemIntrinsic(Inst, Info);
▲ Show 20 Lines • Show All 116 Lines • Show Last 20 Lines

include/llvm/Target/TargetLowering.h

Show First 20 Lines • Show All 1,011 Lines • ▼ Show 20 Linespublic:
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
/// \name Helpers for TargetTransformInfo implementations /// \name Helpers for TargetTransformInfo implementations
/// @{ /// @{
/// Get the ISD node that corresponds to the Instruction class opcode. /// Get the ISD node that corresponds to the Instruction class opcode.
int InstructionOpcodeToISD(unsigned Opcode) const; int InstructionOpcodeToISD(unsigned Opcode) const;
/// Estimate the cost of type-legalization and the legalized type. /// Estimate the cost of type-legalization and the legalized type.
std::pair<unsigned, MVT> getTypeLegalizationCost(const DataLayout &DL, std::pair<int, MVT> getTypeLegalizationCost(const DataLayout &DL,
Type *Ty) const; Type *Ty) const;
/// @} /// @}
//===--------------------------------------------------------------------===// //===--------------------------------------------------------------------===//
/// \name Helpers for atomic expansion. /// \name Helpers for atomic expansion.
/// @{ /// @{
/// True if AtomicExpandPass should use emitLoadLinked/emitStoreConditional /// True if AtomicExpandPass should use emitLoadLinked/emitStoreConditional
▲ Show 20 Lines • Show All 1,820 Lines • Show Last 20 Lines

lib/Analysis/TargetTransformInfo.cpp

Show All 40 Lines
TargetTransformInfo::TargetTransformInfo(TargetTransformInfo &&Arg) TargetTransformInfo::TargetTransformInfo(TargetTransformInfo &&Arg)
: TTIImpl(std::move(Arg.TTIImpl)) {} : TTIImpl(std::move(Arg.TTIImpl)) {}
TargetTransformInfo &TargetTransformInfo::operator=(TargetTransformInfo &&RHS) { TargetTransformInfo &TargetTransformInfo::operator=(TargetTransformInfo &&RHS) {
TTIImpl = std::move(RHS.TTIImpl); TTIImpl = std::move(RHS.TTIImpl);
return *this; return *this;
} }
unsigned TargetTransformInfo::getOperationCost(unsigned Opcode, Type *Ty, int TargetTransformInfo::getOperationCost(unsigned Opcode, Type *Ty,
Type *OpTy) const { Type *OpTy) const {
return TTIImpl->getOperationCost(Opcode, Ty, OpTy); int Cost = TTIImpl->getOperationCost(Opcode, Ty, OpTy);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getCallCost(FunctionType *FTy, int TargetTransformInfo::getCallCost(FunctionType *FTy, int NumArgs) const {
int NumArgs) const { int Cost = TTIImpl->getCallCost(FTy, NumArgs);
return TTIImpl->getCallCost(FTy, NumArgs); assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned int TargetTransformInfo::getCallCost(const Function *F,
TargetTransformInfo::getCallCost(const Function *F,
ArrayRef<const Value *> Arguments) const {
return TTIImpl->getCallCost(F, Arguments);
}
unsigned
TargetTransformInfo::getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<const Value *> Arguments) const { ArrayRef<const Value *> Arguments) const {
return TTIImpl->getIntrinsicCost(IID, RetTy, Arguments); int Cost = TTIImpl->getCallCost(F, Arguments);
} assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
unsigned TargetTransformInfo::getUserCost(const User *U) const { }
return TTIImpl->getUserCost(U);
int TargetTransformInfo::getIntrinsicCost(
Intrinsic::ID IID, Type *RetTy, ArrayRef<const Value *> Arguments) const {
int Cost = TTIImpl->getIntrinsicCost(IID, RetTy, Arguments);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
}
int TargetTransformInfo::getUserCost(const User *U) const {
int Cost = TTIImpl->getUserCost(U);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
bool TargetTransformInfo::hasBranchDivergence() const { bool TargetTransformInfo::hasBranchDivergence() const {
return TTIImpl->hasBranchDivergence(); return TTIImpl->hasBranchDivergence();
} }
bool TargetTransformInfo::isSourceOfDivergence(const Value *V) const { bool TargetTransformInfo::isSourceOfDivergence(const Value *V) const {
return TTIImpl->isSourceOfDivergence(V); return TTIImpl->isSourceOfDivergence(V);
Show All 35 Linesbool TargetTransformInfo::isLegalMaskedLoad(Type *DataType,
return TTIImpl->isLegalMaskedLoad(DataType, Consecutive); return TTIImpl->isLegalMaskedLoad(DataType, Consecutive);
} }
int TargetTransformInfo::getScalingFactorCost(Type *Ty, GlobalValue *BaseGV, int TargetTransformInfo::getScalingFactorCost(Type *Ty, GlobalValue *BaseGV,
int64_t BaseOffset, int64_t BaseOffset,
bool HasBaseReg, bool HasBaseReg,
int64_t Scale, int64_t Scale,
unsigned AddrSpace) const { unsigned AddrSpace) const {
return TTIImpl->getScalingFactorCost(Ty, BaseGV, BaseOffset, HasBaseReg, int Cost = TTIImpl->getScalingFactorCost(Ty, BaseGV, BaseOffset, HasBaseReg,
Scale, AddrSpace); Scale, AddrSpace);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
bool TargetTransformInfo::isTruncateFree(Type *Ty1, Type *Ty2) const { bool TargetTransformInfo::isTruncateFree(Type *Ty1, Type *Ty2) const {
return TTIImpl->isTruncateFree(Ty1, Ty2); return TTIImpl->isTruncateFree(Ty1, Ty2);
} }
bool TargetTransformInfo::isZExtFree(Type *Ty1, Type *Ty2) const { bool TargetTransformInfo::isZExtFree(Type *Ty1, Type *Ty2) const {
return TTIImpl->isZExtFree(Ty1, Ty2); return TTIImpl->isZExtFree(Ty1, Ty2);
Show All 27 Lines
TargetTransformInfo::getPopcntSupport(unsigned IntTyWidthInBit) const { TargetTransformInfo::getPopcntSupport(unsigned IntTyWidthInBit) const {
return TTIImpl->getPopcntSupport(IntTyWidthInBit); return TTIImpl->getPopcntSupport(IntTyWidthInBit);
} }
bool TargetTransformInfo::haveFastSqrt(Type *Ty) const { bool TargetTransformInfo::haveFastSqrt(Type *Ty) const {
return TTIImpl->haveFastSqrt(Ty); return TTIImpl->haveFastSqrt(Ty);
} }
unsigned TargetTransformInfo::getFPOpCost(Type *Ty) const { int TargetTransformInfo::getFPOpCost(Type *Ty) const {
return TTIImpl->getFPOpCost(Ty); int Cost = TTIImpl->getFPOpCost(Ty);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getIntImmCost(const APInt &Imm, Type *Ty) const { int TargetTransformInfo::getIntImmCost(const APInt &Imm, Type *Ty) const {
return TTIImpl->getIntImmCost(Imm, Ty); int Cost = TTIImpl->getIntImmCost(Imm, Ty);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getIntImmCost(unsigned Opcode, unsigned Idx, int TargetTransformInfo::getIntImmCost(unsigned Opcode, unsigned Idx,
const APInt &Imm, Type *Ty) const { const APInt &Imm, Type *Ty) const {
return TTIImpl->getIntImmCost(Opcode, Idx, Imm, Ty); int Cost = TTIImpl->getIntImmCost(Opcode, Idx, Imm, Ty);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getIntImmCost(Intrinsic::ID IID, unsigned Idx, int TargetTransformInfo::getIntImmCost(Intrinsic::ID IID, unsigned Idx,
const APInt &Imm, Type *Ty) const { const APInt &Imm, Type *Ty) const {
return TTIImpl->getIntImmCost(IID, Idx, Imm, Ty); int Cost = TTIImpl->getIntImmCost(IID, Idx, Imm, Ty);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getNumberOfRegisters(bool Vector) const { unsigned TargetTransformInfo::getNumberOfRegisters(bool Vector) const {
return TTIImpl->getNumberOfRegisters(Vector); return TTIImpl->getNumberOfRegisters(Vector);
} }
unsigned TargetTransformInfo::getRegisterBitWidth(bool Vector) const { unsigned TargetTransformInfo::getRegisterBitWidth(bool Vector) const {
return TTIImpl->getRegisterBitWidth(Vector); return TTIImpl->getRegisterBitWidth(Vector);
} }
unsigned TargetTransformInfo::getMaxInterleaveFactor(unsigned VF) const { unsigned TargetTransformInfo::getMaxInterleaveFactor(unsigned VF) const {
return TTIImpl->getMaxInterleaveFactor(VF); return TTIImpl->getMaxInterleaveFactor(VF);
} }
unsigned TargetTransformInfo::getArithmeticInstrCost( int TargetTransformInfo::getArithmeticInstrCost(
unsigned Opcode, Type *Ty, OperandValueKind Opd1Info, unsigned Opcode, Type *Ty, OperandValueKind Opd1Info,
OperandValueKind Opd2Info, OperandValueProperties Opd1PropInfo, OperandValueKind Opd2Info, OperandValueProperties Opd1PropInfo,
OperandValueProperties Opd2PropInfo) const { OperandValueProperties Opd2PropInfo) const {
return TTIImpl->getArithmeticInstrCost(Opcode, Ty, Opd1Info, Opd2Info, int Cost = TTIImpl->getArithmeticInstrCost(Opcode, Ty, Opd1Info, Opd2Info,
Opd1PropInfo, Opd2PropInfo); Opd1PropInfo, Opd2PropInfo);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getShuffleCost(ShuffleKind Kind, Type *Ty, int TargetTransformInfo::getShuffleCost(ShuffleKind Kind, Type *Ty, int Index,
int Index, Type *SubTp) const { Type *SubTp) const {
return TTIImpl->getShuffleCost(Kind, Ty, Index, SubTp); int Cost = TTIImpl->getShuffleCost(Kind, Ty, Index, SubTp);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getCastInstrCost(unsigned Opcode, Type *Dst, int TargetTransformInfo::getCastInstrCost(unsigned Opcode, Type *Dst,
Type *Src) const { Type *Src) const {
return TTIImpl->getCastInstrCost(Opcode, Dst, Src); int Cost = TTIImpl->getCastInstrCost(Opcode, Dst, Src);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getCFInstrCost(unsigned Opcode) const { int TargetTransformInfo::getCFInstrCost(unsigned Opcode) const {
return TTIImpl->getCFInstrCost(Opcode); int Cost = TTIImpl->getCFInstrCost(Opcode);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, int TargetTransformInfo::getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy) const { Type *CondTy) const {
return TTIImpl->getCmpSelInstrCost(Opcode, ValTy, CondTy); int Cost = TTIImpl->getCmpSelInstrCost(Opcode, ValTy, CondTy);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getVectorInstrCost(unsigned Opcode, Type *Val, int TargetTransformInfo::getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index) const { unsigned Index) const {
return TTIImpl->getVectorInstrCost(Opcode, Val, Index); int Cost = TTIImpl->getVectorInstrCost(Opcode, Val, Index);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getMemoryOpCost(unsigned Opcode, Type *Src, int TargetTransformInfo::getMemoryOpCost(unsigned Opcode, Type *Src,
unsigned Alignment, unsigned Alignment,
unsigned AddressSpace) const { unsigned AddressSpace) const {
return TTIImpl->getMemoryOpCost(Opcode, Src, Alignment, AddressSpace); int Cost = TTIImpl->getMemoryOpCost(Opcode, Src, Alignment, AddressSpace);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned int TargetTransformInfo::getMaskedMemoryOpCost(unsigned Opcode, Type *Src,
TargetTransformInfo::getMaskedMemoryOpCost(unsigned Opcode, Type *Src,
unsigned Alignment, unsigned Alignment,
unsigned AddressSpace) const { unsigned AddressSpace) const {
return TTIImpl->getMaskedMemoryOpCost(Opcode, Src, Alignment, AddressSpace); int Cost =
TTIImpl->getMaskedMemoryOpCost(Opcode, Src, Alignment, AddressSpace);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getInterleavedMemoryOpCost( int TargetTransformInfo::getInterleavedMemoryOpCost(
unsigned Opcode, Type *VecTy, unsigned Factor, ArrayRef<unsigned> Indices, unsigned Opcode, Type *VecTy, unsigned Factor, ArrayRef<unsigned> Indices,
unsigned Alignment, unsigned AddressSpace) const { unsigned Alignment, unsigned AddressSpace) const {
return TTIImpl->getInterleavedMemoryOpCost(Opcode, VecTy, Factor, Indices, int Cost = TTIImpl->getInterleavedMemoryOpCost(Opcode, VecTy, Factor, Indices,
Alignment, AddressSpace); Alignment, AddressSpace);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned int TargetTransformInfo::getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
TargetTransformInfo::getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
ArrayRef<Type *> Tys) const { ArrayRef<Type *> Tys) const {
return TTIImpl->getIntrinsicInstrCost(ID, RetTy, Tys); int Cost = TTIImpl->getIntrinsicInstrCost(ID, RetTy, Tys);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getCallInstrCost(Function *F, Type *RetTy, int TargetTransformInfo::getCallInstrCost(Function *F, Type *RetTy,
ArrayRef<Type *> Tys) const { ArrayRef<Type *> Tys) const {
return TTIImpl->getCallInstrCost(F, RetTy, Tys); int Cost = TTIImpl->getCallInstrCost(F, RetTy, Tys);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getNumberOfParts(Type *Tp) const { unsigned TargetTransformInfo::getNumberOfParts(Type *Tp) const {
return TTIImpl->getNumberOfParts(Tp); return TTIImpl->getNumberOfParts(Tp);
} }
unsigned TargetTransformInfo::getAddressComputationCost(Type *Tp, int TargetTransformInfo::getAddressComputationCost(Type *Tp,
bool IsComplex) const { bool IsComplex) const {
return TTIImpl->getAddressComputationCost(Tp, IsComplex); int Cost = TTIImpl->getAddressComputationCost(Tp, IsComplex);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned TargetTransformInfo::getReductionCost(unsigned Opcode, Type *Ty, int TargetTransformInfo::getReductionCost(unsigned Opcode, Type *Ty,
bool IsPairwiseForm) const { bool IsPairwiseForm) const {
return TTIImpl->getReductionCost(Opcode, Ty, IsPairwiseForm); int Cost = TTIImpl->getReductionCost(Opcode, Ty, IsPairwiseForm);
assert(Cost >= 0 && "TTI should not produce negative costs!");
return Cost;
} }
unsigned unsigned
TargetTransformInfo::getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) const { TargetTransformInfo::getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) const {
return TTIImpl->getCostOfKeepingLiveOverCall(Tys); return TTIImpl->getCostOfKeepingLiveOverCall(Tys);
} }
bool TargetTransformInfo::getTgtMemIntrinsic(IntrinsicInst *Inst, bool TargetTransformInfo::getTgtMemIntrinsic(IntrinsicInst *Inst,
▲ Show 20 Lines • Show All 61 Lines • Show Last 20 Lines

lib/CodeGen/TargetLoweringBase.cpp

Show First 20 Lines • Show All 1,628 Lines • ▼ Show 20 Lines#include "llvm/IR/Instruction.def"
case ExtractValue: return ISD::MERGE_VALUES; case ExtractValue: return ISD::MERGE_VALUES;
case InsertValue: return ISD::MERGE_VALUES; case InsertValue: return ISD::MERGE_VALUES;
case LandingPad: return 0; case LandingPad: return 0;
} }
llvm_unreachable("Unknown instruction type encountered!"); llvm_unreachable("Unknown instruction type encountered!");
} }
std::pair<unsigned, MVT> std::pair<int, MVT>
TargetLoweringBase::getTypeLegalizationCost(const DataLayout &DL, TargetLoweringBase::getTypeLegalizationCost(const DataLayout &DL,
Type *Ty) const { Type *Ty) const {
LLVMContext &C = Ty->getContext(); LLVMContext &C = Ty->getContext();
EVT MTy = getValueType(DL, Ty); EVT MTy = getValueType(DL, Ty);
unsigned Cost = 1; int Cost = 1;
// We keep legalizing the type until we find a legal kind. We assume that // We keep legalizing the type until we find a legal kind. We assume that
// the only operation that costs anything is the split. After splitting // the only operation that costs anything is the split. After splitting
// we need to handle two types. // we need to handle two types.
while (true) { while (true) {
LegalizeKind LK = getTypeConversion(C, MTy); LegalizeKind LK = getTypeConversion(C, MTy);
if (LK.first == TypeLegal) if (LK.first == TypeLegal)
return std::make_pair(Cost, MTy.getSimpleVT()); return std::make_pair(Cost, MTy.getSimpleVT());
▲ Show 20 Lines • Show All 49 Lines • Show Last 20 Lines

lib/Target/AArch64/AArch64TargetTransformInfo.h

Show First 20 Lines • Show All 57 Lines • ▼ Show 20 Linespublic:
AArch64TTIImpl(AArch64TTIImpl &&Arg) AArch64TTIImpl(AArch64TTIImpl &&Arg)
: BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)), : BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)),
TLI(std::move(Arg.TLI)) {} TLI(std::move(Arg.TLI)) {}
/// \name Scalar TTI Implementations /// \name Scalar TTI Implementations
/// @{ /// @{
using BaseT::getIntImmCost; using BaseT::getIntImmCost;
unsigned getIntImmCost(int64_t Val); int getIntImmCost(int64_t Val);
unsigned getIntImmCost(const APInt &Imm, Type *Ty); int getIntImmCost(const APInt &Imm, Type *Ty);
unsigned getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, int getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, Type *Ty);
Type *Ty); int getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty); Type *Ty);
TTI::PopcntSupportKind getPopcntSupport(unsigned TyWidth); TTI::PopcntSupportKind getPopcntSupport(unsigned TyWidth);
/// @} /// @}
/// \name Vector TTI Implementations /// \name Vector TTI Implementations
/// @{ /// @{
unsigned getNumberOfRegisters(bool Vector) { unsigned getNumberOfRegisters(bool Vector) {
Show All 11 Lines if (Vector) {
return 128; return 128;
return 0; return 0;
} }
return 64; return 64;
} }
unsigned getMaxInterleaveFactor(unsigned VF); unsigned getMaxInterleaveFactor(unsigned VF);
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src); int getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src);
unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index); int getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index);
unsigned getArithmeticInstrCost( int getArithmeticInstrCost(
unsigned Opcode, Type *Ty, unsigned Opcode, Type *Ty,
TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue, TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue,
TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue, TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue,
TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None, TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None,
TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None); TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None);
unsigned getAddressComputationCost(Type *Ty, bool IsComplex); int getAddressComputationCost(Type *Ty, bool IsComplex);
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy); int getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy);
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, int getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace); unsigned AddressSpace);
unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys); int getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys);
void getUnrollingPreferences(Loop *L, TTI::UnrollingPreferences &UP); void getUnrollingPreferences(Loop *L, TTI::UnrollingPreferences &UP);
Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst, Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst,
Type *ExpectedType); Type *ExpectedType);
bool getTgtMemIntrinsic(IntrinsicInst *Inst, MemIntrinsicInfo &Info); bool getTgtMemIntrinsic(IntrinsicInst *Inst, MemIntrinsicInfo &Info);
unsigned getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy, int getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy, unsigned Factor,
unsigned Factor, ArrayRef<unsigned> Indices, unsigned Alignment,
ArrayRef<unsigned> Indices,
unsigned Alignment,
unsigned AddressSpace); unsigned AddressSpace);
/// @} /// @}
}; };
} // end namespace llvm } // end namespace llvm
#endif #endif

lib/Target/AArch64/AArch64TargetTransformInfo.cpp

Show All 17 Lines
#include <algorithm> #include <algorithm>
using namespace llvm; using namespace llvm;
#define DEBUG_TYPE "aarch64tti" #define DEBUG_TYPE "aarch64tti"
/// \brief Calculate the cost of materializing a 64-bit value. This helper /// \brief Calculate the cost of materializing a 64-bit value. This helper
/// method might only calculate a fraction of a larger immediate. Therefore it /// method might only calculate a fraction of a larger immediate. Therefore it
/// is valid to return a cost of ZERO. /// is valid to return a cost of ZERO.
unsigned AArch64TTIImpl::getIntImmCost(int64_t Val) { int AArch64TTIImpl::getIntImmCost(int64_t Val) {
// Check if the immediate can be encoded within an instruction. // Check if the immediate can be encoded within an instruction.
if (Val == 0 || AArch64_AM::isLogicalImmediate(Val, 64)) if (Val == 0 || AArch64_AM::isLogicalImmediate(Val, 64))
return 0; return 0;
if (Val < 0) if (Val < 0)
Val = ~Val; Val = ~Val;
// Calculate how many moves we will need to materialize this constant. // Calculate how many moves we will need to materialize this constant.
unsigned LZ = countLeadingZeros((uint64_t)Val); unsigned LZ = countLeadingZeros((uint64_t)Val);
return (64 - LZ + 15) / 16; return (64 - LZ + 15) / 16;
} }
/// \brief Calculate the cost of materializing the given constant. /// \brief Calculate the cost of materializing the given constant.
unsigned AArch64TTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) { int AArch64TTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
if (BitSize == 0) if (BitSize == 0)
return ~0U; return ~0U;
// Sign-extend all constants to a multiple of 64-bit. // Sign-extend all constants to a multiple of 64-bit.
APInt ImmVal = Imm; APInt ImmVal = Imm;
if (BitSize & 0x3f) if (BitSize & 0x3f)
ImmVal = Imm.sext((BitSize + 63) & ~0x3fU); ImmVal = Imm.sext((BitSize + 63) & ~0x3fU);
// Split the constant into 64-bit chunks and calculate the cost for each // Split the constant into 64-bit chunks and calculate the cost for each
// chunk. // chunk.
unsigned Cost = 0; int Cost = 0;
for (unsigned ShiftVal = 0; ShiftVal < BitSize; ShiftVal += 64) { for (unsigned ShiftVal = 0; ShiftVal < BitSize; ShiftVal += 64) {
APInt Tmp = ImmVal.ashr(ShiftVal).sextOrTrunc(64); APInt Tmp = ImmVal.ashr(ShiftVal).sextOrTrunc(64);
int64_t Val = Tmp.getSExtValue(); int64_t Val = Tmp.getSExtValue();
Cost += getIntImmCost(Val); Cost += getIntImmCost(Val);
} }
// We need at least one instruction to materialze the constant. // We need at least one instruction to materialze the constant.
return std::max(1U, Cost); return std::max(1, Cost);
} }
unsigned AArch64TTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx, int AArch64TTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx,
const APInt &Imm, Type *Ty) { const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
// There is no cost model for constants with a bit size of 0. Return TCC_Free // There is no cost model for constants with a bit size of 0. Return TCC_Free
// here, so that constant hoisting will ignore this constant. // here, so that constant hoisting will ignore this constant.
if (BitSize == 0) if (BitSize == 0)
return TTI::TCC_Free; return TTI::TCC_Free;
Show All 39 Linesint AArch64TTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx,
case Instruction::Call: case Instruction::Call:
case Instruction::Select: case Instruction::Select:
case Instruction::Ret: case Instruction::Ret:
case Instruction::Load: case Instruction::Load:
break; break;
} }
if (Idx == ImmIdx) { if (Idx == ImmIdx) {
unsigned NumConstants = (BitSize + 63) / 64; int NumConstants = (BitSize + 63) / 64;
unsigned Cost = AArch64TTIImpl::getIntImmCost(Imm, Ty); int Cost = AArch64TTIImpl::getIntImmCost(Imm, Ty);
return (Cost <= NumConstants * TTI::TCC_Basic) return (Cost <= NumConstants * TTI::TCC_Basic)
? static_cast<unsigned>(TTI::TCC_Free) ? static_cast<int>(TTI::TCC_Free)
: Cost; : Cost;
} }
return AArch64TTIImpl::getIntImmCost(Imm, Ty); return AArch64TTIImpl::getIntImmCost(Imm, Ty);
} }
unsigned AArch64TTIImpl::getIntImmCost(Intrinsic::ID IID, unsigned Idx, int AArch64TTIImpl::getIntImmCost(Intrinsic::ID IID, unsigned Idx,
const APInt &Imm, Type *Ty) { const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
// There is no cost model for constants with a bit size of 0. Return TCC_Free // There is no cost model for constants with a bit size of 0. Return TCC_Free
// here, so that constant hoisting will ignore this constant. // here, so that constant hoisting will ignore this constant.
if (BitSize == 0) if (BitSize == 0)
return TTI::TCC_Free; return TTI::TCC_Free;
switch (IID) { switch (IID) {
default: default:
return TTI::TCC_Free; return TTI::TCC_Free;
case Intrinsic::sadd_with_overflow: case Intrinsic::sadd_with_overflow:
case Intrinsic::uadd_with_overflow: case Intrinsic::uadd_with_overflow:
case Intrinsic::ssub_with_overflow: case Intrinsic::ssub_with_overflow:
case Intrinsic::usub_with_overflow: case Intrinsic::usub_with_overflow:
case Intrinsic::smul_with_overflow: case Intrinsic::smul_with_overflow:
case Intrinsic::umul_with_overflow: case Intrinsic::umul_with_overflow:
if (Idx == 1) { if (Idx == 1) {
unsigned NumConstants = (BitSize + 63) / 64; int NumConstants = (BitSize + 63) / 64;
unsigned Cost = AArch64TTIImpl::getIntImmCost(Imm, Ty); int Cost = AArch64TTIImpl::getIntImmCost(Imm, Ty);
return (Cost <= NumConstants * TTI::TCC_Basic) return (Cost <= NumConstants * TTI::TCC_Basic)
? static_cast<unsigned>(TTI::TCC_Free) ? static_cast<int>(TTI::TCC_Free)
: Cost; : Cost;
} }
break; break;
case Intrinsic::experimental_stackmap: case Intrinsic::experimental_stackmap:
if ((Idx < 2) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue()))) if ((Idx < 2) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue())))
return TTI::TCC_Free; return TTI::TCC_Free;
break; break;
case Intrinsic::experimental_patchpoint_void: case Intrinsic::experimental_patchpoint_void:
Show All 9 Lines
AArch64TTIImpl::getPopcntSupport(unsigned TyWidth) { AArch64TTIImpl::getPopcntSupport(unsigned TyWidth) {
assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2"); assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2");
if (TyWidth == 32 || TyWidth == 64) if (TyWidth == 32 || TyWidth == 64)
return TTI::PSK_FastHardware; return TTI::PSK_FastHardware;
// TODO: AArch64TargetLowering::LowerCTPOP() supports 128bit popcount. // TODO: AArch64TargetLowering::LowerCTPOP() supports 128bit popcount.
return TTI::PSK_Software; return TTI::PSK_Software;
} }
unsigned AArch64TTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, int AArch64TTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) {
Type *Src) {
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
EVT SrcTy = TLI->getValueType(DL, Src); EVT SrcTy = TLI->getValueType(DL, Src);
EVT DstTy = TLI->getValueType(DL, Dst); EVT DstTy = TLI->getValueType(DL, Dst);
if (!SrcTy.isSimple() || !DstTy.isSimple()) if (!SrcTy.isSimple() || !DstTy.isSimple())
return BaseT::getCastInstrCost(Opcode, Dst, Src); return BaseT::getCastInstrCost(Opcode, Dst, Src);
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines int Idx = ConvertCostTableLookup<MVT>(
ConversionTbl, array_lengthof(ConversionTbl), ISD, DstTy.getSimpleVT(), ConversionTbl, array_lengthof(ConversionTbl), ISD, DstTy.getSimpleVT(),
SrcTy.getSimpleVT()); SrcTy.getSimpleVT());
if (Idx != -1) if (Idx != -1)
return ConversionTbl[Idx].Cost; return ConversionTbl[Idx].Cost;
return BaseT::getCastInstrCost(Opcode, Dst, Src); return BaseT::getCastInstrCost(Opcode, Dst, Src);
} }
unsigned AArch64TTIImpl::getVectorInstrCost(unsigned Opcode, Type *Val, int AArch64TTIImpl::getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index) { unsigned Index) {
assert(Val->isVectorTy() && "This must be a vector type"); assert(Val->isVectorTy() && "This must be a vector type");
if (Index != -1U) { if (Index != -1U) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Val); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Val);
// This type is legalized to a scalar type. // This type is legalized to a scalar type.
if (!LT.second.isVector()) if (!LT.second.isVector())
return 0; return 0;
// The type may be split. Normalize the index to the new type. // The type may be split. Normalize the index to the new type.
unsigned Width = LT.second.getVectorNumElements(); unsigned Width = LT.second.getVectorNumElements();
Index = Index % Width; Index = Index % Width;
// The element at index zero is already inside the vector. // The element at index zero is already inside the vector.
if (Index == 0) if (Index == 0)
return 0; return 0;
} }
// All other insert/extracts cost this much. // All other insert/extracts cost this much.
return 2; return 2;
} }
unsigned AArch64TTIImpl::getArithmeticInstrCost( int AArch64TTIImpl::getArithmeticInstrCost(
unsigned Opcode, Type *Ty, TTI::OperandValueKind Opd1Info, unsigned Opcode, Type *Ty, TTI::OperandValueKind Opd1Info,
TTI::OperandValueKind Opd2Info, TTI::OperandValueProperties Opd1PropInfo, TTI::OperandValueKind Opd2Info, TTI::OperandValueProperties Opd1PropInfo,
TTI::OperandValueProperties Opd2PropInfo) { TTI::OperandValueProperties Opd2PropInfo) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Ty); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Ty);
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
if (ISD == ISD::SDIV && if (ISD == ISD::SDIV &&
Opd2Info == TargetTransformInfo::OK_UniformConstantValue && Opd2Info == TargetTransformInfo::OK_UniformConstantValue &&
Opd2PropInfo == TargetTransformInfo::OP_PowerOf2) { Opd2PropInfo == TargetTransformInfo::OP_PowerOf2) {
// On AArch64, scalar signed division by constants power-of-two are // On AArch64, scalar signed division by constants power-of-two are
// normally expanded to the sequence ADD + CMP + SELECT + SRA. // normally expanded to the sequence ADD + CMP + SELECT + SRA.
// The OperandValue properties many not be same as that of previous // The OperandValue properties many not be same as that of previous
// operation; conservatively assume OP_None. // operation; conservatively assume OP_None.
unsigned Cost = int Cost = getArithmeticInstrCost(Instruction::Add, Ty, Opd1Info, Opd2Info,
getArithmeticInstrCost(Instruction::Add, Ty, Opd1Info, Opd2Info,
TargetTransformInfo::OP_None, TargetTransformInfo::OP_None,
TargetTransformInfo::OP_None); TargetTransformInfo::OP_None);
Cost += getArithmeticInstrCost(Instruction::Sub, Ty, Opd1Info, Opd2Info, Cost += getArithmeticInstrCost(Instruction::Sub, Ty, Opd1Info, Opd2Info,
TargetTransformInfo::OP_None, TargetTransformInfo::OP_None,
TargetTransformInfo::OP_None); TargetTransformInfo::OP_None);
Cost += getArithmeticInstrCost(Instruction::Select, Ty, Opd1Info, Opd2Info, Cost += getArithmeticInstrCost(Instruction::Select, Ty, Opd1Info, Opd2Info,
TargetTransformInfo::OP_None, TargetTransformInfo::OP_None,
TargetTransformInfo::OP_None); TargetTransformInfo::OP_None);
Cost += getArithmeticInstrCost(Instruction::AShr, Ty, Opd1Info, Opd2Info, Cost += getArithmeticInstrCost(Instruction::AShr, Ty, Opd1Info, Opd2Info,
TargetTransformInfo::OP_None, TargetTransformInfo::OP_None,
Show All 11 Linesint AArch64TTIImpl::getArithmeticInstrCost(
case ISD::OR: case ISD::OR:
case ISD::AND: case ISD::AND:
// These nodes are marked as 'custom' for combining purposes only. // These nodes are marked as 'custom' for combining purposes only.
// We know that they are legal. See LowerAdd in ISelLowering. // We know that they are legal. See LowerAdd in ISelLowering.
return 1 * LT.first; return 1 * LT.first;
} }
} }
unsigned AArch64TTIImpl::getAddressComputationCost(Type *Ty, bool IsComplex) { int AArch64TTIImpl::getAddressComputationCost(Type *Ty, bool IsComplex) {
// Address computations in vectorized code with non-consecutive addresses will // Address computations in vectorized code with non-consecutive addresses will
// likely result in more instructions compared to scalar code where the // likely result in more instructions compared to scalar code where the
// computation can more often be merged into the index mode. The resulting // computation can more often be merged into the index mode. The resulting
// extra micro-ops can significantly decrease throughput. // extra micro-ops can significantly decrease throughput.
unsigned NumVectorInstToHideOverhead = 10; unsigned NumVectorInstToHideOverhead = 10;
if (Ty->isVectorTy() && IsComplex) if (Ty->isVectorTy() && IsComplex)
return NumVectorInstToHideOverhead; return NumVectorInstToHideOverhead;
// In many cases the address computation is not merged into the instruction // In many cases the address computation is not merged into the instruction
// addressing mode. // addressing mode.
return 1; return 1;
} }
unsigned AArch64TTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, int AArch64TTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy) { Type *CondTy) {
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
// We don't lower vector selects well that are wider than the register width. // We don't lower vector selects well that are wider than the register width.
if (ValTy->isVectorTy() && ISD == ISD::SELECT) { if (ValTy->isVectorTy() && ISD == ISD::SELECT) {
// We would need this many instructions to hide the scalarization happening. // We would need this many instructions to hide the scalarization happening.
const unsigned AmortizationCost = 20; const int AmortizationCost = 20;
static const TypeConversionCostTblEntry<MVT::SimpleValueType> static const TypeConversionCostTblEntry<MVT::SimpleValueType>
VectorSelectTbl[] = { VectorSelectTbl[] = {
{ ISD::SELECT, MVT::v16i1, MVT::v16i16, 16 * AmortizationCost }, { ISD::SELECT, MVT::v16i1, MVT::v16i16, 16 * AmortizationCost },
{ ISD::SELECT, MVT::v8i1, MVT::v8i32, 8 * AmortizationCost }, { ISD::SELECT, MVT::v8i1, MVT::v8i32, 8 * AmortizationCost },
{ ISD::SELECT, MVT::v16i1, MVT::v16i32, 16 * AmortizationCost }, { ISD::SELECT, MVT::v16i1, MVT::v16i32, 16 * AmortizationCost },
{ ISD::SELECT, MVT::v4i1, MVT::v4i64, 4 * AmortizationCost }, { ISD::SELECT, MVT::v4i1, MVT::v4i64, 4 * AmortizationCost },
{ ISD::SELECT, MVT::v8i1, MVT::v8i64, 8 * AmortizationCost }, { ISD::SELECT, MVT::v8i1, MVT::v8i64, 8 * AmortizationCost },
{ ISD::SELECT, MVT::v16i1, MVT::v16i64, 16 * AmortizationCost } { ISD::SELECT, MVT::v16i1, MVT::v16i64, 16 * AmortizationCost }
}; };
EVT SelCondTy = TLI->getValueType(DL, CondTy); EVT SelCondTy = TLI->getValueType(DL, CondTy);
EVT SelValTy = TLI->getValueType(DL, ValTy); EVT SelValTy = TLI->getValueType(DL, ValTy);
if (SelCondTy.isSimple() && SelValTy.isSimple()) { if (SelCondTy.isSimple() && SelValTy.isSimple()) {
int Idx = int Idx =
ConvertCostTableLookup(VectorSelectTbl, ISD, SelCondTy.getSimpleVT(), ConvertCostTableLookup(VectorSelectTbl, ISD, SelCondTy.getSimpleVT(),
SelValTy.getSimpleVT()); SelValTy.getSimpleVT());
if (Idx != -1) if (Idx != -1)
return VectorSelectTbl[Idx].Cost; return VectorSelectTbl[Idx].Cost;
} }
} }
return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy); return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy);
} }
unsigned AArch64TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src, int AArch64TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
unsigned Alignment, unsigned Alignment, unsigned AddressSpace) {
unsigned AddressSpace) { std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Src);
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Src);
if (Opcode == Instruction::Store && Src->isVectorTy() && Alignment != 16 && if (Opcode == Instruction::Store && Src->isVectorTy() && Alignment != 16 &&
Src->getVectorElementType()->isIntegerTy(64)) { Src->getVectorElementType()->isIntegerTy(64)) {
// Unaligned stores are extremely inefficient. We don't split // Unaligned stores are extremely inefficient. We don't split
// unaligned v2i64 stores because the negative impact that has shown in // unaligned v2i64 stores because the negative impact that has shown in
// practice on inlined memcpy code. // practice on inlined memcpy code.
// We make v2i64 stores expensive so that we will only vectorize if there // We make v2i64 stores expensive so that we will only vectorize if there
// are 6 other instructions getting vectorized. // are 6 other instructions getting vectorized.
unsigned AmortizationCost = 6; int AmortizationCost = 6;
return LT.first * 2 * AmortizationCost; return LT.first * 2 * AmortizationCost;
} }
if (Src->isVectorTy() && Src->getVectorElementType()->isIntegerTy(8) && if (Src->isVectorTy() && Src->getVectorElementType()->isIntegerTy(8) &&
Src->getVectorNumElements() < 8) { Src->getVectorNumElements() < 8) {
// We scalarize the loads/stores because there is not v.4b register and we // We scalarize the loads/stores because there is not v.4b register and we
// have to promote the elements to v.4h. // have to promote the elements to v.4h.
unsigned NumVecElts = Src->getVectorNumElements(); unsigned NumVecElts = Src->getVectorNumElements();
unsigned NumVectorizableInstsToAmortize = NumVecElts * 2; unsigned NumVectorizableInstsToAmortize = NumVecElts * 2;
// We generate 2 instructions per vector element. // We generate 2 instructions per vector element.
return NumVectorizableInstsToAmortize * NumVecElts * 2; return NumVectorizableInstsToAmortize * NumVecElts * 2;
} }
return LT.first; return LT.first;
} }
unsigned AArch64TTIImpl::getInterleavedMemoryOpCost( int AArch64TTIImpl::getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy,
unsigned Opcode, Type *VecTy, unsigned Factor, ArrayRef<unsigned> Indices, unsigned Factor,
unsigned Alignment, unsigned AddressSpace) { ArrayRef<unsigned> Indices,
unsigned Alignment,
unsigned AddressSpace) {
assert(Factor >= 2 && "Invalid interleave factor"); assert(Factor >= 2 && "Invalid interleave factor");
assert(isa<VectorType>(VecTy) && "Expect a vector type"); assert(isa<VectorType>(VecTy) && "Expect a vector type");
if (Factor <= TLI->getMaxSupportedInterleaveFactor()) { if (Factor <= TLI->getMaxSupportedInterleaveFactor()) {
unsigned NumElts = VecTy->getVectorNumElements(); unsigned NumElts = VecTy->getVectorNumElements();
Type *SubVecTy = VectorType::get(VecTy->getScalarType(), NumElts / Factor); Type *SubVecTy = VectorType::get(VecTy->getScalarType(), NumElts / Factor);
unsigned SubVecSize = DL.getTypeAllocSizeInBits(SubVecTy); unsigned SubVecSize = DL.getTypeAllocSizeInBits(SubVecTy);
// ldN/stN only support legal vector types of size 64 or 128 in bits. // ldN/stN only support legal vector types of size 64 or 128 in bits.
if (NumElts % Factor == 0 && (SubVecSize == 64 || SubVecSize == 128)) if (NumElts % Factor == 0 && (SubVecSize == 64 || SubVecSize == 128))
return Factor; return Factor;
} }
return BaseT::getInterleavedMemoryOpCost(Opcode, VecTy, Factor, Indices, return BaseT::getInterleavedMemoryOpCost(Opcode, VecTy, Factor, Indices,
Alignment, AddressSpace); Alignment, AddressSpace);
} }
unsigned AArch64TTIImpl::getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) { int AArch64TTIImpl::getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) {
unsigned Cost = 0; int Cost = 0;
for (auto *I : Tys) { for (auto *I : Tys) {
if (!I->isVectorTy()) if (!I->isVectorTy())
continue; continue;
if (I->getScalarSizeInBits() * I->getVectorNumElements() == 128) if (I->getScalarSizeInBits() * I->getVectorNumElements() == 128)
Cost += getMemoryOpCost(Instruction::Store, I, 128, 0) + Cost += getMemoryOpCost(Instruction::Store, I, 128, 0) +
getMemoryOpCost(Instruction::Load, I, 128, 0); getMemoryOpCost(Instruction::Load, I, 128, 0);
} }
return Cost; return Cost;
▲ Show 20 Lines • Show All 102 Lines • Show Last 20 Lines

lib/Target/ARM/ARMTargetTransformInfo.h

Show First 20 Lines • Show All 50 Lines • ▼ Show 20 Linespublic:
ARMTTIImpl(ARMTTIImpl &&Arg) ARMTTIImpl(ARMTTIImpl &&Arg)
: BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)), : BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)),
TLI(std::move(Arg.TLI)) {} TLI(std::move(Arg.TLI)) {}
/// \name Scalar TTI Implementations /// \name Scalar TTI Implementations
/// @{ /// @{
using BaseT::getIntImmCost; using BaseT::getIntImmCost;
unsigned getIntImmCost(const APInt &Imm, Type *Ty); int getIntImmCost(const APInt &Imm, Type *Ty);
/// @} /// @}
/// \name Vector TTI Implementations /// \name Vector TTI Implementations
/// @{ /// @{
unsigned getNumberOfRegisters(bool Vector) { unsigned getNumberOfRegisters(bool Vector) {
if (Vector) { if (Vector) {
Show All 19 Linespublic:
unsigned getMaxInterleaveFactor(unsigned VF) { unsigned getMaxInterleaveFactor(unsigned VF) {
// These are out of order CPUs: // These are out of order CPUs:
if (ST->isCortexA15() || ST->isSwift()) if (ST->isCortexA15() || ST->isSwift())
return 2; return 2;
return 1; return 1;
} }
unsigned getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index, int getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index, Type *SubTp);
Type *SubTp);
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src); int getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src);
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy); int getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy);
unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index); int getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index);
unsigned getAddressComputationCost(Type *Val, bool IsComplex); int getAddressComputationCost(Type *Val, bool IsComplex);
unsigned getFPOpCost(Type *Ty); int getFPOpCost(Type *Ty);
unsigned getArithmeticInstrCost( int getArithmeticInstrCost(
unsigned Opcode, Type *Ty, unsigned Opcode, Type *Ty,
TTI::OperandValueKind Op1Info = TTI::OK_AnyValue, TTI::OperandValueKind Op1Info = TTI::OK_AnyValue,
TTI::OperandValueKind Op2Info = TTI::OK_AnyValue, TTI::OperandValueKind Op2Info = TTI::OK_AnyValue,
TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None, TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None,
TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None); TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None);
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, int getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace); unsigned AddressSpace);
unsigned getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy, int getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy, unsigned Factor,
unsigned Factor, ArrayRef<unsigned> Indices, unsigned Alignment,
ArrayRef<unsigned> Indices,
unsigned Alignment,
unsigned AddressSpace); unsigned AddressSpace);
/// @} /// @}
}; };
} // end namespace llvm } // end namespace llvm
#endif #endif

lib/Target/ARM/ARMTargetTransformInfo.cpp

Show All 9 Lines
#include "ARMTargetTransformInfo.h" #include "ARMTargetTransformInfo.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Target/CostTable.h" #include "llvm/Target/CostTable.h"
#include "llvm/Target/TargetLowering.h" #include "llvm/Target/TargetLowering.h"
using namespace llvm; using namespace llvm;
#define DEBUG_TYPE "armtti" #define DEBUG_TYPE "armtti"
unsigned ARMTTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) { int ARMTTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned Bits = Ty->getPrimitiveSizeInBits(); unsigned Bits = Ty->getPrimitiveSizeInBits();
if (Bits == 0 || Bits > 32) if (Bits == 0 || Bits > 32)
return 4; return 4;
int32_t SImmVal = Imm.getSExtValue(); int32_t SImmVal = Imm.getSExtValue();
uint32_t ZImmVal = Imm.getZExtValue(); uint32_t ZImmVal = Imm.getZExtValue();
Show All 15 Linesint ARMTTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) {
if (SImmVal >= 0 && SImmVal < 256) if (SImmVal >= 0 && SImmVal < 256)
return 1; return 1;
if ((~ZImmVal < 256) || ARM_AM::isThumbImmShiftedVal(ZImmVal)) if ((~ZImmVal < 256) || ARM_AM::isThumbImmShiftedVal(ZImmVal))
return 2; return 2;
// Load from constantpool. // Load from constantpool.
return 3; return 3;
} }
unsigned ARMTTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) { int ARMTTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) {
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
// Single to/from double precision conversions. // Single to/from double precision conversions.
static const CostTblEntry<MVT::SimpleValueType> NEONFltDblTbl[] = { static const CostTblEntry<MVT::SimpleValueType> NEONFltDblTbl[] = {
// Vector fptrunc/fpext conversions. // Vector fptrunc/fpext conversions.
{ ISD::FP_ROUND, MVT::v2f64, 2 }, { ISD::FP_ROUND, MVT::v2f64, 2 },
{ ISD::FP_EXTEND, MVT::v2f32, 2 }, { ISD::FP_EXTEND, MVT::v2f32, 2 },
{ ISD::FP_EXTEND, MVT::v4f32, 4 } { ISD::FP_EXTEND, MVT::v4f32, 4 }
}; };
if (Src->isVectorTy() && ST->hasNEON() && (ISD == ISD::FP_ROUND || if (Src->isVectorTy() && ST->hasNEON() && (ISD == ISD::FP_ROUND ||
ISD == ISD::FP_EXTEND)) { ISD == ISD::FP_EXTEND)) {
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Src); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Src);
int Idx = CostTableLookup(NEONFltDblTbl, ISD, LT.second); int Idx = CostTableLookup(NEONFltDblTbl, ISD, LT.second);
if (Idx != -1) if (Idx != -1)
return LT.first * NEONFltDblTbl[Idx].Cost; return LT.first * NEONFltDblTbl[Idx].Cost;
} }
EVT SrcTy = TLI->getValueType(DL, Src); EVT SrcTy = TLI->getValueType(DL, Src);
EVT DstTy = TLI->getValueType(DL, Dst); EVT DstTy = TLI->getValueType(DL, Dst);
▲ Show 20 Lines • Show All 167 Lines • ▼ Show 20 Lines int Idx = ConvertCostTableLookup(ARMIntegerConversionTbl, ISD,
DstTy.getSimpleVT(), SrcTy.getSimpleVT()); DstTy.getSimpleVT(), SrcTy.getSimpleVT());
if (Idx != -1) if (Idx != -1)
return ARMIntegerConversionTbl[Idx].Cost; return ARMIntegerConversionTbl[Idx].Cost;
} }
return BaseT::getCastInstrCost(Opcode, Dst, Src); return BaseT::getCastInstrCost(Opcode, Dst, Src);
} }
unsigned ARMTTIImpl::getVectorInstrCost(unsigned Opcode, Type *ValTy, int ARMTTIImpl::getVectorInstrCost(unsigned Opcode, Type *ValTy,
unsigned Index) { unsigned Index) {
// Penalize inserting into an D-subregister. We end up with a three times // Penalize inserting into an D-subregister. We end up with a three times
// lower estimated throughput on swift. // lower estimated throughput on swift.
if (ST->isSwift() && if (ST->isSwift() &&
Opcode == Instruction::InsertElement && Opcode == Instruction::InsertElement &&
ValTy->isVectorTy() && ValTy->isVectorTy() &&
ValTy->getScalarSizeInBits() <= 32) ValTy->getScalarSizeInBits() <= 32)
return 3; return 3;
// Cross-class copies are expensive on many microarchitectures, // Cross-class copies are expensive on many microarchitectures,
// so assume they are expensive by default. // so assume they are expensive by default.
if ((Opcode == Instruction::InsertElement || if ((Opcode == Instruction::InsertElement ||
Opcode == Instruction::ExtractElement) && Opcode == Instruction::ExtractElement) &&
ValTy->getVectorElementType()->isIntegerTy()) ValTy->getVectorElementType()->isIntegerTy())
return 3; return 3;
return BaseT::getVectorInstrCost(Opcode, ValTy, Index); return BaseT::getVectorInstrCost(Opcode, ValTy, Index);
} }
unsigned ARMTTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, int ARMTTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy) {
Type *CondTy) {
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
// On NEON a a vector select gets lowered to vbsl. // On NEON a a vector select gets lowered to vbsl.
if (ST->hasNEON() && ValTy->isVectorTy() && ISD == ISD::SELECT) { if (ST->hasNEON() && ValTy->isVectorTy() && ISD == ISD::SELECT) {
// Lowering of some vector selects is currently far from perfect. // Lowering of some vector selects is currently far from perfect.
static const TypeConversionCostTblEntry<MVT::SimpleValueType> static const TypeConversionCostTblEntry<MVT::SimpleValueType>
NEONVectorSelectTbl[] = { NEONVectorSelectTbl[] = {
{ ISD::SELECT, MVT::v16i1, MVT::v16i16, 2*16 + 1 + 3*1 + 4*1 }, { ISD::SELECT, MVT::v16i1, MVT::v16i16, 2*16 + 1 + 3*1 + 4*1 },
Show All 9 Lines if (ST->hasNEON() && ValTy->isVectorTy() && ISD == ISD::SELECT) {
if (SelCondTy.isSimple() && SelValTy.isSimple()) { if (SelCondTy.isSimple() && SelValTy.isSimple()) {
int Idx = ConvertCostTableLookup(NEONVectorSelectTbl, ISD, int Idx = ConvertCostTableLookup(NEONVectorSelectTbl, ISD,
SelCondTy.getSimpleVT(), SelCondTy.getSimpleVT(),
SelValTy.getSimpleVT()); SelValTy.getSimpleVT());
if (Idx != -1) if (Idx != -1)
return NEONVectorSelectTbl[Idx].Cost; return NEONVectorSelectTbl[Idx].Cost;
} }
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, ValTy); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, ValTy);
return LT.first; return LT.first;
} }
return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy); return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy);
} }
unsigned ARMTTIImpl::getAddressComputationCost(Type *Ty, bool IsComplex) { int ARMTTIImpl::getAddressComputationCost(Type *Ty, bool IsComplex) {
// Address computations in vectorized code with non-consecutive addresses will // Address computations in vectorized code with non-consecutive addresses will
// likely result in more instructions compared to scalar code where the // likely result in more instructions compared to scalar code where the
// computation can more often be merged into the index mode. The resulting // computation can more often be merged into the index mode. The resulting
// extra micro-ops can significantly decrease throughput. // extra micro-ops can significantly decrease throughput.
unsigned NumVectorInstToHideOverhead = 10; unsigned NumVectorInstToHideOverhead = 10;
if (Ty->isVectorTy() && IsComplex) if (Ty->isVectorTy() && IsComplex)
return NumVectorInstToHideOverhead; return NumVectorInstToHideOverhead;
// In many cases the address computation is not merged into the instruction // In many cases the address computation is not merged into the instruction
// addressing mode. // addressing mode.
return 1; return 1;
} }
unsigned ARMTTIImpl::getFPOpCost(Type *Ty) { int ARMTTIImpl::getFPOpCost(Type *Ty) {
// Use similar logic that's in ARMISelLowering: // Use similar logic that's in ARMISelLowering:
// Any ARM CPU with VFP2 has floating point, but Thumb1 didn't have access // Any ARM CPU with VFP2 has floating point, but Thumb1 didn't have access
// to VFP. // to VFP.
if (ST->hasVFP2() && !ST->isThumb1Only()) { if (ST->hasVFP2() && !ST->isThumb1Only()) {
if (Ty->isFloatTy()) { if (Ty->isFloatTy()) {
return TargetTransformInfo::TCC_Basic; return TargetTransformInfo::TCC_Basic;
} }
if (Ty->isDoubleTy()) { if (Ty->isDoubleTy()) {
return ST->isFPOnlySP() ? TargetTransformInfo::TCC_Expensive : return ST->isFPOnlySP() ? TargetTransformInfo::TCC_Expensive :
TargetTransformInfo::TCC_Basic; TargetTransformInfo::TCC_Basic;
} }
} }
return TargetTransformInfo::TCC_Expensive; return TargetTransformInfo::TCC_Expensive;
} }
unsigned ARMTTIImpl::getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index, int ARMTTIImpl::getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index,
Type *SubTp) { Type *SubTp) {
// We only handle costs of reverse and alternate shuffles for now. // We only handle costs of reverse and alternate shuffles for now.
if (Kind != TTI::SK_Reverse && Kind != TTI::SK_Alternate) if (Kind != TTI::SK_Reverse && Kind != TTI::SK_Alternate)
return BaseT::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
if (Kind == TTI::SK_Reverse) { if (Kind == TTI::SK_Reverse) {
static const CostTblEntry<MVT::SimpleValueType> NEONShuffleTbl[] = { static const CostTblEntry<MVT::SimpleValueType> NEONShuffleTbl[] = {
// Reverse shuffle cost one instruction if we are shuffling within a // Reverse shuffle cost one instruction if we are shuffling within a
// double word (vrev) or two if we shuffle a quad word (vrev, vext). // double word (vrev) or two if we shuffle a quad word (vrev, vext).
{ISD::VECTOR_SHUFFLE, MVT::v2i32, 1}, {ISD::VECTOR_SHUFFLE, MVT::v2i32, 1},
{ISD::VECTOR_SHUFFLE, MVT::v2f32, 1}, {ISD::VECTOR_SHUFFLE, MVT::v2f32, 1},
{ISD::VECTOR_SHUFFLE, MVT::v2i64, 1}, {ISD::VECTOR_SHUFFLE, MVT::v2i64, 1},
{ISD::VECTOR_SHUFFLE, MVT::v2f64, 1}, {ISD::VECTOR_SHUFFLE, MVT::v2f64, 1},
{ISD::VECTOR_SHUFFLE, MVT::v4i32, 2}, {ISD::VECTOR_SHUFFLE, MVT::v4i32, 2},
{ISD::VECTOR_SHUFFLE, MVT::v4f32, 2}, {ISD::VECTOR_SHUFFLE, MVT::v4f32, 2},
{ISD::VECTOR_SHUFFLE, MVT::v8i16, 2}, {ISD::VECTOR_SHUFFLE, MVT::v8i16, 2},
{ISD::VECTOR_SHUFFLE, MVT::v16i8, 2}}; {ISD::VECTOR_SHUFFLE, MVT::v16i8, 2}};
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Tp); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Tp);
int Idx = CostTableLookup(NEONShuffleTbl, ISD::VECTOR_SHUFFLE, LT.second); int Idx = CostTableLookup(NEONShuffleTbl, ISD::VECTOR_SHUFFLE, LT.second);
if (Idx == -1) if (Idx == -1)
return BaseT::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
return LT.first * NEONShuffleTbl[Idx].Cost; return LT.first * NEONShuffleTbl[Idx].Cost;
} }
if (Kind == TTI::SK_Alternate) { if (Kind == TTI::SK_Alternate) {
Show All 9 Lines static const CostTblEntry<MVT::SimpleValueType> NEONAltShuffleTbl[] = {
{ISD::VECTOR_SHUFFLE, MVT::v4i32, 2}, {ISD::VECTOR_SHUFFLE, MVT::v4i32, 2},
{ISD::VECTOR_SHUFFLE, MVT::v4f32, 2}, {ISD::VECTOR_SHUFFLE, MVT::v4f32, 2},
{ISD::VECTOR_SHUFFLE, MVT::v4i16, 2}, {ISD::VECTOR_SHUFFLE, MVT::v4i16, 2},
{ISD::VECTOR_SHUFFLE, MVT::v8i16, 16}, {ISD::VECTOR_SHUFFLE, MVT::v8i16, 16},
{ISD::VECTOR_SHUFFLE, MVT::v16i8, 32}}; {ISD::VECTOR_SHUFFLE, MVT::v16i8, 32}};
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Tp); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Tp);
int Idx = int Idx =
CostTableLookup(NEONAltShuffleTbl, ISD::VECTOR_SHUFFLE, LT.second); CostTableLookup(NEONAltShuffleTbl, ISD::VECTOR_SHUFFLE, LT.second);
if (Idx == -1) if (Idx == -1)
return BaseT::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
return LT.first * NEONAltShuffleTbl[Idx].Cost; return LT.first * NEONAltShuffleTbl[Idx].Cost;
} }
return BaseT::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
} }
unsigned ARMTTIImpl::getArithmeticInstrCost( int ARMTTIImpl::getArithmeticInstrCost(
unsigned Opcode, Type *Ty, TTI::OperandValueKind Op1Info, unsigned Opcode, Type *Ty, TTI::OperandValueKind Op1Info,
TTI::OperandValueKind Op2Info, TTI::OperandValueProperties Opd1PropInfo, TTI::OperandValueKind Op2Info, TTI::OperandValueProperties Opd1PropInfo,
TTI::OperandValueProperties Opd2PropInfo) { TTI::OperandValueProperties Opd2PropInfo) {
int ISDOpcode = TLI->InstructionOpcodeToISD(Opcode); int ISDOpcode = TLI->InstructionOpcodeToISD(Opcode);
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Ty); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Ty);
const unsigned FunctionCallDivCost = 20; const unsigned FunctionCallDivCost = 20;
const unsigned ReciprocalDivCost = 10; const unsigned ReciprocalDivCost = 10;
static const CostTblEntry<MVT::SimpleValueType> CostTbl[] = { static const CostTblEntry<MVT::SimpleValueType> CostTbl[] = {
// Division. // Division.
// These costs are somewhat random. Choose a cost of 20 to indicate that // These costs are somewhat random. Choose a cost of 20 to indicate that
// vectorizing devision (added function call) is going to be very expensive. // vectorizing devision (added function call) is going to be very expensive.
// Double registers types. // Double registers types.
Show All 36 Linesint ARMTTIImpl::getArithmeticInstrCost(
int Idx = -1; int Idx = -1;
if (ST->hasNEON()) if (ST->hasNEON())
Idx = CostTableLookup(CostTbl, ISDOpcode, LT.second); Idx = CostTableLookup(CostTbl, ISDOpcode, LT.second);
if (Idx != -1) if (Idx != -1)
return LT.first * CostTbl[Idx].Cost; return LT.first * CostTbl[Idx].Cost;
unsigned Cost = BaseT::getArithmeticInstrCost(Opcode, Ty, Op1Info, Op2Info, int Cost = BaseT::getArithmeticInstrCost(Opcode, Ty, Op1Info, Op2Info,
Opd1PropInfo, Opd2PropInfo); Opd1PropInfo, Opd2PropInfo);
// This is somewhat of a hack. The problem that we are facing is that SROA // This is somewhat of a hack. The problem that we are facing is that SROA
// creates a sequence of shift, and, or instructions to construct values. // creates a sequence of shift, and, or instructions to construct values.
// These sequences are recognized by the ISel and have zero-cost. Not so for // These sequences are recognized by the ISel and have zero-cost. Not so for
// the vectorized code. Because we have support for v2i64 but not i64 those // the vectorized code. Because we have support for v2i64 but not i64 those
// sequences look particularly beneficial to vectorize. // sequences look particularly beneficial to vectorize.
// To work around this we increase the cost of v2i64 operations to make them // To work around this we increase the cost of v2i64 operations to make them
// seem less beneficial. // seem less beneficial.
if (LT.second == MVT::v2i64 && if (LT.second == MVT::v2i64 &&
Op2Info == TargetTransformInfo::OK_UniformConstantValue) Op2Info == TargetTransformInfo::OK_UniformConstantValue)
Cost += 4; Cost += 4;
return Cost; return Cost;
} }
unsigned ARMTTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src, int ARMTTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned Alignment,
unsigned AddressSpace) { unsigned AddressSpace) {
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Src); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Src);
if (Src->isVectorTy() && Alignment != 16 && if (Src->isVectorTy() && Alignment != 16 &&
Src->getVectorElementType()->isDoubleTy()) { Src->getVectorElementType()->isDoubleTy()) {
// Unaligned loads/stores are extremely inefficient. // Unaligned loads/stores are extremely inefficient.
// We need 4 uops for vst.1/vld.1 vs 1uop for vldr/vstr. // We need 4 uops for vst.1/vld.1 vs 1uop for vldr/vstr.
return LT.first * 4; return LT.first * 4;
} }
return LT.first; return LT.first;
} }
unsigned ARMTTIImpl::getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy, int ARMTTIImpl::getInterleavedMemoryOpCost(unsigned Opcode, Type *VecTy,
unsigned Factor, unsigned Factor,
ArrayRef<unsigned> Indices, ArrayRef<unsigned> Indices,
unsigned Alignment, unsigned Alignment,
unsigned AddressSpace) { unsigned AddressSpace) {
assert(Factor >= 2 && "Invalid interleave factor"); assert(Factor >= 2 && "Invalid interleave factor");
assert(isa<VectorType>(VecTy) && "Expect a vector type"); assert(isa<VectorType>(VecTy) && "Expect a vector type");
// vldN/vstN doesn't support vector types of i64/f64 element. // vldN/vstN doesn't support vector types of i64/f64 element.
bool EltIs64Bits = DL.getTypeAllocSizeInBits(VecTy->getScalarType()) == 64; bool EltIs64Bits = DL.getTypeAllocSizeInBits(VecTy->getScalarType()) == 64;
if (Factor <= TLI->getMaxSupportedInterleaveFactor() && !EltIs64Bits) { if (Factor <= TLI->getMaxSupportedInterleaveFactor() && !EltIs64Bits) {
unsigned NumElts = VecTy->getVectorNumElements(); unsigned NumElts = VecTy->getVectorNumElements();
Show All 11 Lines

lib/Target/NVPTX/NVPTXTargetTransformInfo.h

Show First 20 Lines • Show All 46 Lines • ▼ Show 20 Linespublic:
NVPTXTTIImpl(NVPTXTTIImpl &&Arg) NVPTXTTIImpl(NVPTXTTIImpl &&Arg)
: BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)), : BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)),
TLI(std::move(Arg.TLI)) {} TLI(std::move(Arg.TLI)) {}
bool hasBranchDivergence() { return true; } bool hasBranchDivergence() { return true; }
bool isSourceOfDivergence(const Value *V); bool isSourceOfDivergence(const Value *V);
unsigned getArithmeticInstrCost( int getArithmeticInstrCost(
unsigned Opcode, Type *Ty, unsigned Opcode, Type *Ty,
TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue, TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue,
TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue, TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue,
TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None, TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None,
TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None); TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None);
void getUnrollingPreferences(Loop *L, TTI::UnrollingPreferences &UP); void getUnrollingPreferences(Loop *L, TTI::UnrollingPreferences &UP);
}; };
} // end namespace llvm } // end namespace llvm
#endif #endif

lib/Target/NVPTX/NVPTXTargetTransformInfo.cpp

Show First 20 Lines • Show All 83 Lines • ▼ Show 20 Lines if (const Instruction *I = dyn_cast<Instruction>(V)) {
// inter-procedural analysis. // inter-procedural analysis.
if (isa<CallInst>(I)) if (isa<CallInst>(I))
return true; return true;
} }
return false; return false;
} }
unsigned NVPTXTTIImpl::getArithmeticInstrCost( int NVPTXTTIImpl::getArithmeticInstrCost(
unsigned Opcode, Type *Ty, TTI::OperandValueKind Opd1Info, unsigned Opcode, Type *Ty, TTI::OperandValueKind Opd1Info,
TTI::OperandValueKind Opd2Info, TTI::OperandValueProperties Opd1PropInfo, TTI::OperandValueKind Opd2Info, TTI::OperandValueProperties Opd1PropInfo,
TTI::OperandValueProperties Opd2PropInfo) { TTI::OperandValueProperties Opd2PropInfo) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Ty); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Ty);
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
switch (ISD) { switch (ISD) {
default: default:
return BaseT::getArithmeticInstrCost(Opcode, Ty, Opd1Info, Opd2Info, return BaseT::getArithmeticInstrCost(Opcode, Ty, Opd1Info, Opd2Info,
Opd1PropInfo, Opd2PropInfo); Opd1PropInfo, Opd2PropInfo);
case ISD::ADD: case ISD::ADD:
Show All 26 Lines

lib/Target/PowerPC/PPCTargetTransformInfo.h

Show First 20 Lines • Show All 46 Lines • ▼ Show 20 Linespublic:
PPCTTIImpl(PPCTTIImpl &&Arg) PPCTTIImpl(PPCTTIImpl &&Arg)
: BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)), : BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)),
TLI(std::move(Arg.TLI)) {} TLI(std::move(Arg.TLI)) {}
/// \name Scalar TTI Implementations /// \name Scalar TTI Implementations
/// @{ /// @{
using BaseT::getIntImmCost; using BaseT::getIntImmCost;
unsigned getIntImmCost(const APInt &Imm, Type *Ty); int getIntImmCost(const APInt &Imm, Type *Ty);
unsigned getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, int getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, Type *Ty);
Type *Ty); int getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty); Type *Ty);
TTI::PopcntSupportKind getPopcntSupport(unsigned TyWidth); TTI::PopcntSupportKind getPopcntSupport(unsigned TyWidth);
void getUnrollingPreferences(Loop *L, TTI::UnrollingPreferences &UP); void getUnrollingPreferences(Loop *L, TTI::UnrollingPreferences &UP);
/// @} /// @}
/// \name Vector TTI Implementations /// \name Vector TTI Implementations
/// @{ /// @{
bool enableAggressiveInterleaving(bool LoopHasReductions); bool enableAggressiveInterleaving(bool LoopHasReductions);
unsigned getNumberOfRegisters(bool Vector); unsigned getNumberOfRegisters(bool Vector);
unsigned getRegisterBitWidth(bool Vector); unsigned getRegisterBitWidth(bool Vector);
unsigned getMaxInterleaveFactor(unsigned VF); unsigned getMaxInterleaveFactor(unsigned VF);
unsigned getArithmeticInstrCost( int getArithmeticInstrCost(
unsigned Opcode, Type *Ty, unsigned Opcode, Type *Ty,
TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue, TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue,
TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue, TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue,
TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None, TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None,
TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None); TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None);
unsigned getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index, int getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index, Type *SubTp);
Type *SubTp); int getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src);
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src); int getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy);
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy); int getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index);
unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index); int getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace); unsigned AddressSpace);
/// @} /// @}
}; };
} // end namespace llvm } // end namespace llvm
#endif #endif

lib/Target/PowerPC/PPCTargetTransformInfo.cpp

Show All 29 Lines
TargetTransformInfo::PopcntSupportKind TargetTransformInfo::PopcntSupportKind
PPCTTIImpl::getPopcntSupport(unsigned TyWidth) { PPCTTIImpl::getPopcntSupport(unsigned TyWidth) {
assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2"); assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2");
if (ST->hasPOPCNTD() && TyWidth <= 64) if (ST->hasPOPCNTD() && TyWidth <= 64)
return TTI::PSK_FastHardware; return TTI::PSK_FastHardware;
return TTI::PSK_Software; return TTI::PSK_Software;
} }
unsigned PPCTTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) { int PPCTTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) {
if (DisablePPCConstHoist) if (DisablePPCConstHoist)
return BaseT::getIntImmCost(Imm, Ty); return BaseT::getIntImmCost(Imm, Ty);
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
if (BitSize == 0) if (BitSize == 0)
return ~0U; return ~0U;
Show All 12 Lines if (isInt<32>(Imm.getSExtValue())) {
return 2 * TTI::TCC_Basic; return 2 * TTI::TCC_Basic;
} }
} }
return 4 * TTI::TCC_Basic; return 4 * TTI::TCC_Basic;
} }
unsigned PPCTTIImpl::getIntImmCost(Intrinsic::ID IID, unsigned Idx, int PPCTTIImpl::getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
const APInt &Imm, Type *Ty) { Type *Ty) {
if (DisablePPCConstHoist) if (DisablePPCConstHoist)
return BaseT::getIntImmCost(IID, Idx, Imm, Ty); return BaseT::getIntImmCost(IID, Idx, Imm, Ty);
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
if (BitSize == 0) if (BitSize == 0)
return ~0U; return ~0U;
Show All 16 Linesint PPCTTIImpl::getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
case Intrinsic::experimental_patchpoint_i64: case Intrinsic::experimental_patchpoint_i64:
if ((Idx < 4) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue()))) if ((Idx < 4) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue())))
return TTI::TCC_Free; return TTI::TCC_Free;
break; break;
} }
return PPCTTIImpl::getIntImmCost(Imm, Ty); return PPCTTIImpl::getIntImmCost(Imm, Ty);
} }
unsigned PPCTTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx, int PPCTTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm,
const APInt &Imm, Type *Ty) { Type *Ty) {
if (DisablePPCConstHoist) if (DisablePPCConstHoist)
return BaseT::getIntImmCost(Opcode, Idx, Imm, Ty); return BaseT::getIntImmCost(Opcode, Idx, Imm, Ty);
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
if (BitSize == 0) if (BitSize == 0)
return ~0U; return ~0U;
▲ Show 20 Lines • Show All 130 Lines • ▼ Show 20 Lines if (Directive == PPC::DIR_PWR7 ||
Directive == PPC::DIR_PWR8) Directive == PPC::DIR_PWR8)
return 12; return 12;
// For most things, modern systems have two execution units (and // For most things, modern systems have two execution units (and
// out-of-order execution). // out-of-order execution).
return 2; return 2;
} }
unsigned PPCTTIImpl::getArithmeticInstrCost( int PPCTTIImpl::getArithmeticInstrCost(
unsigned Opcode, Type *Ty, TTI::OperandValueKind Op1Info, unsigned Opcode, Type *Ty, TTI::OperandValueKind Op1Info,
TTI::OperandValueKind Op2Info, TTI::OperandValueProperties Opd1PropInfo, TTI::OperandValueKind Op2Info, TTI::OperandValueProperties Opd1PropInfo,
TTI::OperandValueProperties Opd2PropInfo) { TTI::OperandValueProperties Opd2PropInfo) {
assert(TLI->InstructionOpcodeToISD(Opcode) && "Invalid opcode"); assert(TLI->InstructionOpcodeToISD(Opcode) && "Invalid opcode");
// Fallback to the default implementation. // Fallback to the default implementation.
return BaseT::getArithmeticInstrCost(Opcode, Ty, Op1Info, Op2Info, return BaseT::getArithmeticInstrCost(Opcode, Ty, Op1Info, Op2Info,
Opd1PropInfo, Opd2PropInfo); Opd1PropInfo, Opd2PropInfo);
} }
unsigned PPCTTIImpl::getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index, int PPCTTIImpl::getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index,
Type *SubTp) { Type *SubTp) {
return BaseT::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
} }
unsigned PPCTTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) { int PPCTTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) {
assert(TLI->InstructionOpcodeToISD(Opcode) && "Invalid opcode"); assert(TLI->InstructionOpcodeToISD(Opcode) && "Invalid opcode");
return BaseT::getCastInstrCost(Opcode, Dst, Src); return BaseT::getCastInstrCost(Opcode, Dst, Src);
} }
unsigned PPCTTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, int PPCTTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy) {
Type *CondTy) {
return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy); return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy);
} }
unsigned PPCTTIImpl::getVectorInstrCost(unsigned Opcode, Type *Val, int PPCTTIImpl::getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index) {
unsigned Index) {
assert(Val->isVectorTy() && "This must be a vector type"); assert(Val->isVectorTy() && "This must be a vector type");
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
if (ST->hasVSX() && Val->getScalarType()->isDoubleTy()) { if (ST->hasVSX() && Val->getScalarType()->isDoubleTy()) {
// Double-precision scalars are already located in index #0. // Double-precision scalars are already located in index #0.
if (Index == 0) if (Index == 0)
Show All 22 Linesint PPCTTIImpl::getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index) {
// these need to be estimated as very costly. // these need to be estimated as very costly.
if (ISD == ISD::EXTRACT_VECTOR_ELT || if (ISD == ISD::EXTRACT_VECTOR_ELT ||
ISD == ISD::INSERT_VECTOR_ELT) ISD == ISD::INSERT_VECTOR_ELT)
return LHSPenalty + BaseT::getVectorInstrCost(Opcode, Val, Index); return LHSPenalty + BaseT::getVectorInstrCost(Opcode, Val, Index);
return BaseT::getVectorInstrCost(Opcode, Val, Index); return BaseT::getVectorInstrCost(Opcode, Val, Index);
} }
unsigned PPCTTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src, int PPCTTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned Alignment,
unsigned AddressSpace) { unsigned AddressSpace) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Src); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Src);
assert((Opcode == Instruction::Load || Opcode == Instruction::Store) && assert((Opcode == Instruction::Load || Opcode == Instruction::Store) &&
"Invalid Opcode"); "Invalid Opcode");
unsigned Cost = BaseT::getMemoryOpCost(Opcode, Src, Alignment, AddressSpace); int Cost = BaseT::getMemoryOpCost(Opcode, Src, Alignment, AddressSpace);
// VSX loads/stores support unaligned access. // VSX loads/stores support unaligned access.
if (ST->hasVSX()) { if (ST->hasVSX()) {
if (LT.second == MVT::v2f64 || LT.second == MVT::v2i64) if (LT.second == MVT::v2f64 || LT.second == MVT::v2i64)
return Cost; return Cost;
} }
bool UnalignedAltivec = bool UnalignedAltivec =
Show All 22 Lines

lib/Target/SystemZ/SystemZTargetTransformInfo.h

Show All 36 Lines SystemZTTIImpl(const SystemZTTIImpl &Arg)
: BaseT(static_cast<const BaseT &>(Arg)), ST(Arg.ST), TLI(Arg.TLI) {} : BaseT(static_cast<const BaseT &>(Arg)), ST(Arg.ST), TLI(Arg.TLI) {}
SystemZTTIImpl(SystemZTTIImpl &&Arg) SystemZTTIImpl(SystemZTTIImpl &&Arg)
: BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)), : BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)),
TLI(std::move(Arg.TLI)) {} TLI(std::move(Arg.TLI)) {}
/// \name Scalar TTI Implementations /// \name Scalar TTI Implementations
/// @{ /// @{
unsigned getIntImmCost(const APInt &Imm, Type *Ty); int getIntImmCost(const APInt &Imm, Type *Ty);
unsigned getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, int getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, Type *Ty);
Type *Ty); int getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty); Type *Ty);
TTI::PopcntSupportKind getPopcntSupport(unsigned TyWidth); TTI::PopcntSupportKind getPopcntSupport(unsigned TyWidth);
/// @} /// @}
/// \name Vector TTI Implementations /// \name Vector TTI Implementations
/// @{ /// @{
Show All 9 Lines

lib/Target/SystemZ/SystemZTargetTransformInfo.cpp

Show All 25 Lines
#define DEBUG_TYPE "systemztti" #define DEBUG_TYPE "systemztti"
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// SystemZ cost model. // SystemZ cost model.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
unsigned SystemZTTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) { int SystemZTTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
// There is no cost model for constants with a bit size of 0. Return TCC_Free // There is no cost model for constants with a bit size of 0. Return TCC_Free
// here, so that constant hoisting will ignore this constant. // here, so that constant hoisting will ignore this constant.
if (BitSize == 0) if (BitSize == 0)
return TTI::TCC_Free; return TTI::TCC_Free;
// No cost model for operations on integers larger than 64 bit implemented yet. // No cost model for operations on integers larger than 64 bit implemented yet.
Show All 15 Lines if ((Imm.getZExtValue() & 0xffffffff) == 0)
return TTI::TCC_Basic; return TTI::TCC_Basic;
return 2 * TTI::TCC_Basic; return 2 * TTI::TCC_Basic;
} }
return 4 * TTI::TCC_Basic; return 4 * TTI::TCC_Basic;
} }
unsigned SystemZTTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx, int SystemZTTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx,
const APInt &Imm, Type *Ty) { const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
// There is no cost model for constants with a bit size of 0. Return TCC_Free // There is no cost model for constants with a bit size of 0. Return TCC_Free
// here, so that constant hoisting will ignore this constant. // here, so that constant hoisting will ignore this constant.
if (BitSize == 0) if (BitSize == 0)
return TTI::TCC_Free; return TTI::TCC_Free;
// No cost model for operations on integers larger than 64 bit implemented yet. // No cost model for operations on integers larger than 64 bit implemented yet.
▲ Show 20 Lines • Show All 100 Lines • ▼ Show 20 Linesint SystemZTTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx,
case Instruction::Ret: case Instruction::Ret:
case Instruction::Load: case Instruction::Load:
break; break;
} }
return SystemZTTIImpl::getIntImmCost(Imm, Ty); return SystemZTTIImpl::getIntImmCost(Imm, Ty);
} }
unsigned SystemZTTIImpl::getIntImmCost(Intrinsic::ID IID, unsigned Idx, int SystemZTTIImpl::getIntImmCost(Intrinsic::ID IID, unsigned Idx,
const APInt &Imm, Type *Ty) { const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
// There is no cost model for constants with a bit size of 0. Return TCC_Free // There is no cost model for constants with a bit size of 0. Return TCC_Free
// here, so that constant hoisting will ignore this constant. // here, so that constant hoisting will ignore this constant.
if (BitSize == 0) if (BitSize == 0)
return TTI::TCC_Free; return TTI::TCC_Free;
// No cost model for operations on integers larger than 64 bit implemented yet. // No cost model for operations on integers larger than 64 bit implemented yet.
▲ Show 20 Lines • Show All 65 Lines • Show Last 20 Lines

lib/Target/X86/X86TargetTransformInfo.h

Show All 27 Lines
class X86TTIImpl : public BasicTTIImplBase<X86TTIImpl> { class X86TTIImpl : public BasicTTIImplBase<X86TTIImpl> {
typedef BasicTTIImplBase<X86TTIImpl> BaseT; typedef BasicTTIImplBase<X86TTIImpl> BaseT;
typedef TargetTransformInfo TTI; typedef TargetTransformInfo TTI;
friend BaseT; friend BaseT;
const X86Subtarget *ST; const X86Subtarget *ST;
const X86TargetLowering *TLI; const X86TargetLowering *TLI;
unsigned getScalarizationOverhead(Type *Ty, bool Insert, bool Extract); int getScalarizationOverhead(Type *Ty, bool Insert, bool Extract);
const X86Subtarget *getST() const { return ST; } const X86Subtarget *getST() const { return ST; }
const X86TargetLowering *getTLI() const { return TLI; } const X86TargetLowering *getTLI() const { return TLI; }
public: public:
explicit X86TTIImpl(const X86TargetMachine *TM, Function &F) explicit X86TTIImpl(const X86TargetMachine *TM, Function &F)
: BaseT(TM, F.getParent()->getDataLayout()), ST(TM->getSubtargetImpl(F)), : BaseT(TM, F.getParent()->getDataLayout()), ST(TM->getSubtargetImpl(F)),
TLI(ST->getTargetLowering()) {} TLI(ST->getTargetLowering()) {}
Show All 12 Linespublic:
/// @} /// @}
/// \name Vector TTI Implementations /// \name Vector TTI Implementations
/// @{ /// @{
unsigned getNumberOfRegisters(bool Vector); unsigned getNumberOfRegisters(bool Vector);
unsigned getRegisterBitWidth(bool Vector); unsigned getRegisterBitWidth(bool Vector);
unsigned getMaxInterleaveFactor(unsigned VF); unsigned getMaxInterleaveFactor(unsigned VF);
unsigned getArithmeticInstrCost( int getArithmeticInstrCost(
unsigned Opcode, Type *Ty, unsigned Opcode, Type *Ty,
TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue, TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue,
TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue, TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue,
TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None, TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None,
TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None); TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None);
unsigned getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index, int getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index, Type *SubTp);
Type *SubTp); int getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src);
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src); int getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy);
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy); int getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index);
unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index); int getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace); unsigned AddressSpace);
unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, int getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace); unsigned AddressSpace);
unsigned getAddressComputationCost(Type *PtrTy, bool IsComplex); int getAddressComputationCost(Type *PtrTy, bool IsComplex);
unsigned getReductionCost(unsigned Opcode, Type *Ty, bool IsPairwiseForm); int getReductionCost(unsigned Opcode, Type *Ty, bool IsPairwiseForm);
unsigned getIntImmCost(int64_t); int getIntImmCost(int64_t);
unsigned getIntImmCost(const APInt &Imm, Type *Ty); int getIntImmCost(const APInt &Imm, Type *Ty);
unsigned getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, int getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, Type *Ty);
Type *Ty); int getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty); Type *Ty);
bool isLegalMaskedLoad(Type *DataType, int Consecutive); bool isLegalMaskedLoad(Type *DataType, int Consecutive);
bool isLegalMaskedStore(Type *DataType, int Consecutive); bool isLegalMaskedStore(Type *DataType, int Consecutive);
bool areInlineCompatible(const Function *Caller, bool areInlineCompatible(const Function *Caller,
const Function *Callee) const; const Function *Callee) const;
/// @} /// @}
}; };
} // end namespace llvm } // end namespace llvm
#endif #endif

lib/Target/X86/X86TargetTransformInfo.cpp

Show First 20 Lines • Show All 78 Lines • ▼ Show 20 Linesunsigned X86TTIImpl::getMaxInterleaveFactor(unsigned VF) {
// Sandybridge and Haswell have multiple execution ports and pipelined // Sandybridge and Haswell have multiple execution ports and pipelined
// vector units. // vector units.
if (ST->hasAVX()) if (ST->hasAVX())
return 4; return 4;
return 2; return 2;
} }
unsigned X86TTIImpl::getArithmeticInstrCost( int X86TTIImpl::getArithmeticInstrCost(
unsigned Opcode, Type *Ty, TTI::OperandValueKind Op1Info, unsigned Opcode, Type *Ty, TTI::OperandValueKind Op1Info,
TTI::OperandValueKind Op2Info, TTI::OperandValueProperties Opd1PropInfo, TTI::OperandValueKind Op2Info, TTI::OperandValueProperties Opd1PropInfo,
TTI::OperandValueProperties Opd2PropInfo) { TTI::OperandValueProperties Opd2PropInfo) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Ty); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Ty);
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
if (ISD == ISD::SDIV && if (ISD == ISD::SDIV &&
Op2Info == TargetTransformInfo::OK_UniformConstantValue && Op2Info == TargetTransformInfo::OK_UniformConstantValue &&
Opd2PropInfo == TargetTransformInfo::OP_PowerOf2) { Opd2PropInfo == TargetTransformInfo::OP_PowerOf2) {
// On X86, vector signed division by constants power-of-two are // On X86, vector signed division by constants power-of-two are
// normally expanded to the sequence SRA + SRL + ADD + SRA. // normally expanded to the sequence SRA + SRL + ADD + SRA.
// The OperandValue properties many not be same as that of previous // The OperandValue properties many not be same as that of previous
// operation;conservatively assume OP_None. // operation;conservatively assume OP_None.
unsigned Cost = int Cost = 2 * getArithmeticInstrCost(Instruction::AShr, Ty, Op1Info,
2 * getArithmeticInstrCost(Instruction::AShr, Ty, Op1Info, Op2Info, Op2Info, TargetTransformInfo::OP_None,
TargetTransformInfo::OP_None,
TargetTransformInfo::OP_None); TargetTransformInfo::OP_None);
Cost += getArithmeticInstrCost(Instruction::LShr, Ty, Op1Info, Op2Info, Cost += getArithmeticInstrCost(Instruction::LShr, Ty, Op1Info, Op2Info,
TargetTransformInfo::OP_None, TargetTransformInfo::OP_None,
TargetTransformInfo::OP_None); TargetTransformInfo::OP_None);
Cost += getArithmeticInstrCost(Instruction::Add, Ty, Op1Info, Op2Info, Cost += getArithmeticInstrCost(Instruction::Add, Ty, Op1Info, Op2Info,
TargetTransformInfo::OP_None, TargetTransformInfo::OP_None,
TargetTransformInfo::OP_None); TargetTransformInfo::OP_None);
return Cost; return Cost;
▲ Show 20 Lines • Show All 228 Lines • ▼ Show 20 Linesint X86TTIImpl::getArithmeticInstrCost(
if (ISD == ISD::MUL && LT.second == MVT::v4i32 && ST->hasSSE2() && if (ISD == ISD::MUL && LT.second == MVT::v4i32 && ST->hasSSE2() &&
!ST->hasSSE41()) !ST->hasSSE41())
return LT.first * 6; return LT.first * 6;
// Fallback to the default implementation. // Fallback to the default implementation.
return BaseT::getArithmeticInstrCost(Opcode, Ty, Op1Info, Op2Info); return BaseT::getArithmeticInstrCost(Opcode, Ty, Op1Info, Op2Info);
} }
unsigned X86TTIImpl::getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index, int X86TTIImpl::getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index,
Type *SubTp) { Type *SubTp) {
// We only estimate the cost of reverse and alternate shuffles. // We only estimate the cost of reverse and alternate shuffles.
if (Kind != TTI::SK_Reverse && Kind != TTI::SK_Alternate) if (Kind != TTI::SK_Reverse && Kind != TTI::SK_Alternate)
return BaseT::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
if (Kind == TTI::SK_Reverse) { if (Kind == TTI::SK_Reverse) {
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Tp); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Tp);
unsigned Cost = 1; int Cost = 1;
if (LT.second.getSizeInBits() > 128) if (LT.second.getSizeInBits() > 128)
Cost = 3; // Extract + insert + copy. Cost = 3; // Extract + insert + copy.
// Multiple by the number of parts. // Multiple by the number of parts.
return Cost * LT.first; return Cost * LT.first;
} }
if (Kind == TTI::SK_Alternate) { if (Kind == TTI::SK_Alternate) {
// 64-bit packed float vectors (v2f32) are widened to type v4f32. // 64-bit packed float vectors (v2f32) are widened to type v4f32.
// 64-bit packed integer vectors (v2i32) are promoted to type v2i64. // 64-bit packed integer vectors (v2i32) are promoted to type v2i64.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Tp); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Tp);
// The backend knows how to generate a single VEX.256 version of // The backend knows how to generate a single VEX.256 version of
// instruction VPBLENDW if the target supports AVX2. // instruction VPBLENDW if the target supports AVX2.
if (ST->hasAVX2() && LT.second == MVT::v16i16) if (ST->hasAVX2() && LT.second == MVT::v16i16)
return LT.first; return LT.first;
static const CostTblEntry<MVT::SimpleValueType> AVXAltShuffleTbl[] = { static const CostTblEntry<MVT::SimpleValueType> AVXAltShuffleTbl[] = {
{ISD::VECTOR_SHUFFLE, MVT::v4i64, 1}, // vblendpd {ISD::VECTOR_SHUFFLE, MVT::v4i64, 1}, // vblendpd
▲ Show 20 Lines • Show All 79 Lines • ▼ Show 20 Lines if (Kind == TTI::SK_Alternate) {
if (Idx != -1) if (Idx != -1)
return LT.first * SSEAltShuffleTbl[Idx].Cost; return LT.first * SSEAltShuffleTbl[Idx].Cost;
return BaseT::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
} }
return BaseT::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
} }
unsigned X86TTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) { int X86TTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) {
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
static const TypeConversionCostTblEntry<MVT::SimpleValueType> static const TypeConversionCostTblEntry<MVT::SimpleValueType>
AVX512ConversionTbl[] = { AVX512ConversionTbl[] = {
{ ISD::FP_EXTEND, MVT::v8f64, MVT::v8f32, 1 }, { ISD::FP_EXTEND, MVT::v8f64, MVT::v8f32, 1 },
{ ISD::FP_EXTEND, MVT::v8f64, MVT::v16f32, 3 }, { ISD::FP_EXTEND, MVT::v8f64, MVT::v16f32, 3 },
{ ISD::FP_ROUND, MVT::v8f32, MVT::v8f64, 1 }, { ISD::FP_ROUND, MVT::v8f32, MVT::v8f64, 1 },
▲ Show 20 Lines • Show All 147 Lines • ▼ Show 20 Lines SSE2ConvTbl[] = {
{ ISD::UINT_TO_FP, MVT::v4f32, MVT::v8i16, 15 }, { ISD::UINT_TO_FP, MVT::v4f32, MVT::v8i16, 15 },
{ ISD::UINT_TO_FP, MVT::v4f32, MVT::v16i8, 8 }, { ISD::UINT_TO_FP, MVT::v4f32, MVT::v16i8, 8 },
{ ISD::SINT_TO_FP, MVT::v4f32, MVT::v2i64, 15 }, { ISD::SINT_TO_FP, MVT::v4f32, MVT::v2i64, 15 },
{ ISD::SINT_TO_FP, MVT::v4f32, MVT::v4i32, 15 }, { ISD::SINT_TO_FP, MVT::v4f32, MVT::v4i32, 15 },
{ ISD::SINT_TO_FP, MVT::v4f32, MVT::v8i16, 15 }, { ISD::SINT_TO_FP, MVT::v4f32, MVT::v8i16, 15 },
{ ISD::SINT_TO_FP, MVT::v4f32, MVT::v16i8, 8 }, { ISD::SINT_TO_FP, MVT::v4f32, MVT::v16i8, 8 },
}; };
std::pair<unsigned, MVT> LTSrc = TLI->getTypeLegalizationCost(DL, Src); std::pair<int, MVT> LTSrc = TLI->getTypeLegalizationCost(DL, Src);
std::pair<unsigned, MVT> LTDest = TLI->getTypeLegalizationCost(DL, Dst); std::pair<int, MVT> LTDest = TLI->getTypeLegalizationCost(DL, Dst);
if (ST->hasSSE2() && !ST->hasAVX()) { if (ST->hasSSE2() && !ST->hasAVX()) {
int Idx = int Idx =
ConvertCostTableLookup(SSE2ConvTbl, ISD, LTDest.second, LTSrc.second); ConvertCostTableLookup(SSE2ConvTbl, ISD, LTDest.second, LTSrc.second);
if (Idx != -1) if (Idx != -1)
return LTSrc.first * SSE2ConvTbl[Idx].Cost; return LTSrc.first * SSE2ConvTbl[Idx].Cost;
} }
Show All 23 Lines int Idx = ConvertCostTableLookup(AVXConversionTbl, ISD, DstTy.getSimpleVT(),
SrcTy.getSimpleVT()); SrcTy.getSimpleVT());
if (Idx != -1) if (Idx != -1)
return AVXConversionTbl[Idx].Cost; return AVXConversionTbl[Idx].Cost;
} }
return BaseT::getCastInstrCost(Opcode, Dst, Src); return BaseT::getCastInstrCost(Opcode, Dst, Src);
} }
unsigned X86TTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, int X86TTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy) {
Type *CondTy) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, ValTy); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, ValTy);
MVT MTy = LT.second; MVT MTy = LT.second;
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
static const CostTblEntry<MVT::SimpleValueType> SSE42CostTbl[] = { static const CostTblEntry<MVT::SimpleValueType> SSE42CostTbl[] = {
{ ISD::SETCC, MVT::v2f64, 1 }, { ISD::SETCC, MVT::v2f64, 1 },
▲ Show 20 Lines • Show All 50 Lines • ▼ Show 20 Lines if (ST->hasSSE42()) {
int Idx = CostTableLookup(SSE42CostTbl, ISD, MTy); int Idx = CostTableLookup(SSE42CostTbl, ISD, MTy);
if (Idx != -1) if (Idx != -1)
return LT.first * SSE42CostTbl[Idx].Cost; return LT.first * SSE42CostTbl[Idx].Cost;
} }
return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy); return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy);
} }
unsigned X86TTIImpl::getVectorInstrCost(unsigned Opcode, Type *Val, int X86TTIImpl::getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index) {
unsigned Index) {
assert(Val->isVectorTy() && "This must be a vector type"); assert(Val->isVectorTy() && "This must be a vector type");
if (Index != -1U) { if (Index != -1U) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Val); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Val);
// This type is legalized to a scalar type. // This type is legalized to a scalar type.
if (!LT.second.isVector()) if (!LT.second.isVector())
return 0; return 0;
// The type may be split. Normalize the index to the new type. // The type may be split. Normalize the index to the new type.
unsigned Width = LT.second.getVectorNumElements(); unsigned Width = LT.second.getVectorNumElements();
Index = Index % Width; Index = Index % Width;
// Floating point scalars are already located in index #0. // Floating point scalars are already located in index #0.
if (Val->getScalarType()->isFloatingPointTy() && Index == 0) if (Val->getScalarType()->isFloatingPointTy() && Index == 0)
return 0; return 0;
} }
return BaseT::getVectorInstrCost(Opcode, Val, Index); return BaseT::getVectorInstrCost(Opcode, Val, Index);
} }
unsigned X86TTIImpl::getScalarizationOverhead(Type *Ty, bool Insert, int X86TTIImpl::getScalarizationOverhead(Type *Ty, bool Insert, bool Extract) {
bool Extract) {
assert (Ty->isVectorTy() && "Can only scalarize vectors"); assert (Ty->isVectorTy() && "Can only scalarize vectors");
unsigned Cost = 0; int Cost = 0;
for (int i = 0, e = Ty->getVectorNumElements(); i < e; ++i) { for (int i = 0, e = Ty->getVectorNumElements(); i < e; ++i) {
if (Insert) if (Insert)
Cost += getVectorInstrCost(Instruction::InsertElement, Ty, i); Cost += getVectorInstrCost(Instruction::InsertElement, Ty, i);
if (Extract) if (Extract)
Cost += getVectorInstrCost(Instruction::ExtractElement, Ty, i); Cost += getVectorInstrCost(Instruction::ExtractElement, Ty, i);
} }
return Cost; return Cost;
} }
unsigned X86TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src, int X86TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned Alignment,
unsigned AddressSpace) { unsigned AddressSpace) {
// Handle non-power-of-two vectors such as <3 x float> // Handle non-power-of-two vectors such as <3 x float>
if (VectorType *VTy = dyn_cast<VectorType>(Src)) { if (VectorType *VTy = dyn_cast<VectorType>(Src)) {
unsigned NumElem = VTy->getVectorNumElements(); unsigned NumElem = VTy->getVectorNumElements();
// Handle a few common cases: // Handle a few common cases:
// <3 x float> // <3 x float>
if (NumElem == 3 && VTy->getScalarSizeInBits() == 32) if (NumElem == 3 && VTy->getScalarSizeInBits() == 32)
// Cost = 64 bit store + extract + 32 bit store. // Cost = 64 bit store + extract + 32 bit store.
return 3; return 3;
// <3 x double> // <3 x double>
if (NumElem == 3 && VTy->getScalarSizeInBits() == 64) if (NumElem == 3 && VTy->getScalarSizeInBits() == 64)
// Cost = 128 bit store + unpack + 64 bit store. // Cost = 128 bit store + unpack + 64 bit store.
return 3; return 3;
// Assume that all other non-power-of-two numbers are scalarized. // Assume that all other non-power-of-two numbers are scalarized.
if (!isPowerOf2_32(NumElem)) { if (!isPowerOf2_32(NumElem)) {
unsigned Cost = BaseT::getMemoryOpCost(Opcode, VTy->getScalarType(), int Cost = BaseT::getMemoryOpCost(Opcode, VTy->getScalarType(), Alignment,
Alignment, AddressSpace); AddressSpace);
unsigned SplitCost = getScalarizationOverhead(Src, int SplitCost = getScalarizationOverhead(Src, Opcode == Instruction::Load,
Opcode == Instruction::Load,
Opcode==Instruction::Store); Opcode == Instruction::Store);
return NumElem * Cost + SplitCost; return NumElem * Cost + SplitCost;
} }
} }
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, Src); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, Src);
assert((Opcode == Instruction::Load || Opcode == Instruction::Store) && assert((Opcode == Instruction::Load || Opcode == Instruction::Store) &&
"Invalid Opcode"); "Invalid Opcode");
// Each load/store unit costs 1. // Each load/store unit costs 1.
unsigned Cost = LT.first * 1; int Cost = LT.first * 1;
// On Sandybridge 256bit load/stores are double pumped // On Sandybridge 256bit load/stores are double pumped
// (but not on Haswell). // (but not on Haswell).
if (LT.second.getSizeInBits() > 128 && !ST->hasAVX2()) if (LT.second.getSizeInBits() > 128 && !ST->hasAVX2())
Cost*=2; Cost*=2;
return Cost; return Cost;
} }
unsigned X86TTIImpl::getMaskedMemoryOpCost(unsigned Opcode, Type *SrcTy, int X86TTIImpl::getMaskedMemoryOpCost(unsigned Opcode, Type *SrcTy,
unsigned Alignment, unsigned Alignment,
unsigned AddressSpace) { unsigned AddressSpace) {
VectorType *SrcVTy = dyn_cast<VectorType>(SrcTy); VectorType *SrcVTy = dyn_cast<VectorType>(SrcTy);
if (!SrcVTy) if (!SrcVTy)
// To calculate scalar take the regular cost, without mask // To calculate scalar take the regular cost, without mask
return getMemoryOpCost(Opcode, SrcTy, Alignment, AddressSpace); return getMemoryOpCost(Opcode, SrcTy, Alignment, AddressSpace);
unsigned NumElem = SrcVTy->getVectorNumElements(); unsigned NumElem = SrcVTy->getVectorNumElements();
VectorType *MaskTy = VectorType *MaskTy =
VectorType::get(Type::getInt8Ty(getGlobalContext()), NumElem); VectorType::get(Type::getInt8Ty(getGlobalContext()), NumElem);
if ((Opcode == Instruction::Load && !isLegalMaskedLoad(SrcVTy, 1)) || if ((Opcode == Instruction::Load && !isLegalMaskedLoad(SrcVTy, 1)) ||
(Opcode == Instruction::Store && !isLegalMaskedStore(SrcVTy, 1)) || (Opcode == Instruction::Store && !isLegalMaskedStore(SrcVTy, 1)) ||
!isPowerOf2_32(NumElem)) { !isPowerOf2_32(NumElem)) {
// Scalarization // Scalarization
unsigned MaskSplitCost = getScalarizationOverhead(MaskTy, false, true); int MaskSplitCost = getScalarizationOverhead(MaskTy, false, true);
unsigned ScalarCompareCost = int ScalarCompareCost = getCmpSelInstrCost(
getCmpSelInstrCost(Instruction::ICmp, Instruction::ICmp, Type::getInt8Ty(getGlobalContext()), NULL);
Type::getInt8Ty(getGlobalContext()), NULL); int BranchCost = getCFInstrCost(Instruction::Br);
unsigned BranchCost = getCFInstrCost(Instruction::Br); int MaskCmpCost = NumElem * (BranchCost + ScalarCompareCost);
unsigned MaskCmpCost = NumElem * (BranchCost + ScalarCompareCost);
int ValueSplitCost = getScalarizationOverhead(
unsigned ValueSplitCost = SrcVTy, Opcode == Instruction::Load, Opcode == Instruction::Store);
getScalarizationOverhead(SrcVTy, Opcode == Instruction::Load, int MemopCost =
Opcode == Instruction::Store);
unsigned MemopCost =
NumElem * BaseT::getMemoryOpCost(Opcode, SrcVTy->getScalarType(), NumElem * BaseT::getMemoryOpCost(Opcode, SrcVTy->getScalarType(),
Alignment, AddressSpace); Alignment, AddressSpace);
return MemopCost + ValueSplitCost + MaskSplitCost + MaskCmpCost; return MemopCost + ValueSplitCost + MaskSplitCost + MaskCmpCost;
} }
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, SrcVTy); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, SrcVTy);
unsigned Cost = 0; int Cost = 0;
if (LT.second != TLI->getValueType(DL, SrcVTy).getSimpleVT() && if (LT.second != TLI->getValueType(DL, SrcVTy).getSimpleVT() &&
LT.second.getVectorNumElements() == NumElem) LT.second.getVectorNumElements() == NumElem)
// Promotion requires expand/truncate for data and a shuffle for mask. // Promotion requires expand/truncate for data and a shuffle for mask.
Cost += getShuffleCost(TTI::SK_Alternate, SrcVTy, 0, 0) + Cost += getShuffleCost(TTI::SK_Alternate, SrcVTy, 0, 0) +
getShuffleCost(TTI::SK_Alternate, MaskTy, 0, 0); getShuffleCost(TTI::SK_Alternate, MaskTy, 0, 0);
else if (LT.second.getVectorNumElements() > NumElem) { else if (LT.second.getVectorNumElements() > NumElem) {
VectorType *NewMaskTy = VectorType::get(MaskTy->getVectorElementType(), VectorType *NewMaskTy = VectorType::get(MaskTy->getVectorElementType(),
LT.second.getVectorNumElements()); LT.second.getVectorNumElements());
// Expanding requires fill mask with zeroes // Expanding requires fill mask with zeroes
Cost += getShuffleCost(TTI::SK_InsertSubvector, NewMaskTy, 0, MaskTy); Cost += getShuffleCost(TTI::SK_InsertSubvector, NewMaskTy, 0, MaskTy);
} }
if (!ST->hasAVX512()) if (!ST->hasAVX512())
return Cost + LT.first*4; // Each maskmov costs 4 return Cost + LT.first*4; // Each maskmov costs 4
// AVX-512 masked load/store is cheapper // AVX-512 masked load/store is cheapper
return Cost+LT.first; return Cost+LT.first;
} }
unsigned X86TTIImpl::getAddressComputationCost(Type *Ty, bool IsComplex) { int X86TTIImpl::getAddressComputationCost(Type *Ty, bool IsComplex) {
// Address computations in vectorized code with non-consecutive addresses will // Address computations in vectorized code with non-consecutive addresses will
// likely result in more instructions compared to scalar code where the // likely result in more instructions compared to scalar code where the
// computation can more often be merged into the index mode. The resulting // computation can more often be merged into the index mode. The resulting
// extra micro-ops can significantly decrease throughput. // extra micro-ops can significantly decrease throughput.
unsigned NumVectorInstToHideOverhead = 10; unsigned NumVectorInstToHideOverhead = 10;
if (Ty->isVectorTy() && IsComplex) if (Ty->isVectorTy() && IsComplex)
return NumVectorInstToHideOverhead; return NumVectorInstToHideOverhead;
return BaseT::getAddressComputationCost(Ty, IsComplex); return BaseT::getAddressComputationCost(Ty, IsComplex);
} }
unsigned X86TTIImpl::getReductionCost(unsigned Opcode, Type *ValTy, int X86TTIImpl::getReductionCost(unsigned Opcode, Type *ValTy,
bool IsPairwise) { bool IsPairwise) {
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(DL, ValTy); std::pair<int, MVT> LT = TLI->getTypeLegalizationCost(DL, ValTy);
MVT MTy = LT.second; MVT MTy = LT.second;
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
// We use the Intel Architecture Code Analyzer(IACA) to measure the throughput // We use the Intel Architecture Code Analyzer(IACA) to measure the throughput
// and make it as the cost. // and make it as the cost.
▲ Show 20 Lines • Show All 63 Lines • ▼ Show 20 Linesint X86TTIImpl::getReductionCost(unsigned Opcode, Type *ValTy,
} }
return BaseT::getReductionCost(Opcode, ValTy, IsPairwise); return BaseT::getReductionCost(Opcode, ValTy, IsPairwise);
} }
/// \brief Calculate the cost of materializing a 64-bit value. This helper /// \brief Calculate the cost of materializing a 64-bit value. This helper
/// method might only calculate a fraction of a larger immediate. Therefore it /// method might only calculate a fraction of a larger immediate. Therefore it
/// is valid to return a cost of ZERO. /// is valid to return a cost of ZERO.
unsigned X86TTIImpl::getIntImmCost(int64_t Val) { int X86TTIImpl::getIntImmCost(int64_t Val) {
if (Val == 0) if (Val == 0)
return TTI::TCC_Free; return TTI::TCC_Free;
if (isInt<32>(Val)) if (isInt<32>(Val))
return TTI::TCC_Basic; return TTI::TCC_Basic;
return 2 * TTI::TCC_Basic; return 2 * TTI::TCC_Basic;
} }
unsigned X86TTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) { int X86TTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
if (BitSize == 0) if (BitSize == 0)
return ~0U; return ~0U;
// Never hoist constants larger than 128bit, because this might lead to // Never hoist constants larger than 128bit, because this might lead to
// incorrect code generation or assertions in codegen. // incorrect code generation or assertions in codegen.
// Fixme: Create a cost model for types larger than i128 once the codegen // Fixme: Create a cost model for types larger than i128 once the codegen
// issues have been fixed. // issues have been fixed.
if (BitSize > 128) if (BitSize > 128)
return TTI::TCC_Free; return TTI::TCC_Free;
if (Imm == 0) if (Imm == 0)
return TTI::TCC_Free; return TTI::TCC_Free;
// Sign-extend all constants to a multiple of 64-bit. // Sign-extend all constants to a multiple of 64-bit.
APInt ImmVal = Imm; APInt ImmVal = Imm;
if (BitSize & 0x3f) if (BitSize & 0x3f)
ImmVal = Imm.sext((BitSize + 63) & ~0x3fU); ImmVal = Imm.sext((BitSize + 63) & ~0x3fU);
// Split the constant into 64-bit chunks and calculate the cost for each // Split the constant into 64-bit chunks and calculate the cost for each
// chunk. // chunk.
unsigned Cost = 0; int Cost = 0;
for (unsigned ShiftVal = 0; ShiftVal < BitSize; ShiftVal += 64) { for (unsigned ShiftVal = 0; ShiftVal < BitSize; ShiftVal += 64) {
APInt Tmp = ImmVal.ashr(ShiftVal).sextOrTrunc(64); APInt Tmp = ImmVal.ashr(ShiftVal).sextOrTrunc(64);
int64_t Val = Tmp.getSExtValue(); int64_t Val = Tmp.getSExtValue();
Cost += getIntImmCost(Val); Cost += getIntImmCost(Val);
} }
// We need at least one instruction to materialze the constant. // We need at least one instruction to materialze the constant.
return std::max(1U, Cost); return std::max(1, Cost);
} }
unsigned X86TTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx, int X86TTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm,
const APInt &Imm, Type *Ty) { Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
// There is no cost model for constants with a bit size of 0. Return TCC_Free // There is no cost model for constants with a bit size of 0. Return TCC_Free
// here, so that constant hoisting will ignore this constant. // here, so that constant hoisting will ignore this constant.
if (BitSize == 0) if (BitSize == 0)
return TTI::TCC_Free; return TTI::TCC_Free;
▲ Show 20 Lines • Show All 41 Lines • ▼ Show 20 Linesint X86TTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm,
case Instruction::Call: case Instruction::Call:
case Instruction::Select: case Instruction::Select:
case Instruction::Ret: case Instruction::Ret:
case Instruction::Load: case Instruction::Load:
break; break;
} }
if (Idx == ImmIdx) { if (Idx == ImmIdx) {
unsigned NumConstants = (BitSize + 63) / 64; int NumConstants = (BitSize + 63) / 64;
unsigned Cost = X86TTIImpl::getIntImmCost(Imm, Ty); int Cost = X86TTIImpl::getIntImmCost(Imm, Ty);
return (Cost <= NumConstants * TTI::TCC_Basic) return (Cost <= NumConstants * TTI::TCC_Basic)
? static_cast<unsigned>(TTI::TCC_Free) ? static_cast<int>(TTI::TCC_Free)
: Cost; : Cost;
} }
return X86TTIImpl::getIntImmCost(Imm, Ty); return X86TTIImpl::getIntImmCost(Imm, Ty);
} }
unsigned X86TTIImpl::getIntImmCost(Intrinsic::ID IID, unsigned Idx, int X86TTIImpl::getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
const APInt &Imm, Type *Ty) { Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
// There is no cost model for constants with a bit size of 0. Return TCC_Free // There is no cost model for constants with a bit size of 0. Return TCC_Free
// here, so that constant hoisting will ignore this constant. // here, so that constant hoisting will ignore this constant.
if (BitSize == 0) if (BitSize == 0)
return TTI::TCC_Free; return TTI::TCC_Free;
▲ Show 20 Lines • Show All 55 Lines • Show Last 20 Lines