This is an archive of the discontinued LLVM Phabricator instance.

Differential D7293

[PM] Change the core design of the TTI analysis to use a polymorphic type erased interface and a single analysis pass rather than an extremely complex analysis group.
ClosedPublic

Authored by chandlerc on Jan 30 2015, 5:36 AM.

Details

Reviewers
echristo
hfinkel
Commits
rG705b185f90a6: [PM] Change the core design of the TTI analysis to use a polymorphic type…
rL227669: [PM] Change the core design of the TTI analysis to use a polymorphic
Summary

The end result is that the TTI analysis can contain a type erased
implementation that supports the polymorphic TTI interface. We can build
one from a target-specific implementation or from a dummy one in the IR.

I've also factored all of the code into "mix-in"-able base classes,
including CRTP base classes to facilitate calling back up to the most
specialized form when delegating horizontally across the surface.

There are a number of reasons for this change, and this particular
design. The first and foremost reason is that an analysis group is
complete overkill, and the chaining delegation strategy was so opaque,
confusing, and high overhead that TTI was suffering greatly for it.
Several of the TTI functions had failed to be implemented in all places
because of the chaining-based delegation making there be no checking of
this. A few other functions were implemented with incorrect delegation.
The message to me was very clear working on this -- the delegation and
analysis group structure was too confusing to be useful here.

The other reason of course is that this is *much* more natural fit for
the new pass manager. This will lay the ground work for a type-erased
per-function info object that can look up the correct subtarget and even
cache it.

Yet another benefit is that this will significantly simplify the
interaction of the pass managers and the TargetMachine. See the future
work below.

There is still a lot more to be done here, but this was the huge chunk
that I couldn't really split things out of because this was the
interface change to TTI. I've tried to minimize all the other parts of
this. The follow up work should include:

  1. Improving the TargetMachine interface by having it directly return a TTI object. Because we have a non-pass object with value semantics and an internal type erasure mechanism, we can narrow the interface of the TargetMachine to *just* do what we need: build and return a TTI object that we can then insert into the pass pipeline.
  2. Make the TTI object be fully specialized for a particular function. This will include splitting off a minimal form of it which is sufficient for the inliner and the old pass manager.
  3. Add a new pass manager analysis which produces TTI objects from the target machine for each function.

Diff Detail

Repository
rL LLVM

Event Timeline

chandlerc updated this revision to Diff 19034.Jan 30 2015, 5:36 AM
chandlerc retitled this revision from to [PM] Change the core design of the TTI analysis to use a polymorphic type erased interface and a single analysis pass rather than an extremely complex analysis group..
chandlerc updated this object.
chandlerc edited the test plan for this revision. (Show Details)
chandlerc added reviewers: hfinkel, echristo.
chandlerc added a subscriber: Unknown Object (MLST).
Herald added a subscriber: jholewinski. · View Herald TranscriptJan 30 2015, 5:36 AM
echristo accepted this revision.Jan 30 2015, 5:03 PM
echristo edited edge metadata.

Looks pretty good to me. It might be nice for TTI to have a class design comment somewhere around it that gives some rough stubbed out directions for how specialization works, but in general no other comments than that.

This revision is now accepted and ready to land.Jan 30 2015, 5:03 PM
echristo added a comment.Jan 30 2015, 5:18 PM

SGTM.

hfinkel accepted this revision.Jan 30 2015, 6:42 PM
hfinkel edited edge metadata.

We had a long discussion about this design on IRC, and I'm relatively happy with this now. Please go ahead.

include/llvm/Analysis/TargetTransformInfo.h
58 ↗(On Diff #19034)

I understand the comment, but I think it can be made less potentially-confusing. How about:

Construct a TTI object using a type implementing the Concept API below.
include/llvm/Analysis/TargetTransformInfoImpl.h
54 ↗(On Diff #19034)

Nevermind. ;)

Closed by commit rL227669: [PM] Change the core design of the TTI analysis to use a polymorphic (authored by chandlerc). · Explain WhyJan 30 2015, 7:45 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents

Diff 19078

llvm/trunk/include/llvm/Analysis/TargetTransformInfo.h

Show First 20 Lines • Show All 47 Lines • ▼ Show 20 Linesstruct MemIntrinsicInfo {
unsigned short MatchingId; unsigned short MatchingId;
int NumMemRefs; int NumMemRefs;
Value *PtrVal; Value *PtrVal;
}; };
/// \brief This pass provides access to the codegen interfaces that are needed /// \brief This pass provides access to the codegen interfaces that are needed
/// for IR-level transformations. /// for IR-level transformations.
class TargetTransformInfo { class TargetTransformInfo {
protected:
/// \brief The TTI instance one level down the stack.
///
/// This is used to implement the default behavior all of the methods which
/// is to delegate up through the stack of TTIs until one can answer the
/// query.
TargetTransformInfo *PrevTTI;
/// \brief The top of the stack of TTI analyses available.
///
/// This is a convenience routine maintained as TTI analyses become available
/// that complements the PrevTTI delegation chain. When one part of an
/// analysis pass wants to query another part of the analysis pass it can use
/// this to start back at the top of the stack.
TargetTransformInfo *TopTTI;
/// All pass subclasses must in their initializePass routine call
/// pushTTIStack with themselves to update the pointers tracking the previous
/// TTI instance in the analysis group's stack, and the top of the analysis
/// group's stack.
void pushTTIStack(Pass *P);
/// All pass subclasses must call TargetTransformInfo::getAnalysisUsage.
virtual void getAnalysisUsage(AnalysisUsage &AU) const;
public: public:
/// This class is intended to be subclassed by real implementations. /// \brief Construct a TTI object using a type implementing the \c Concept
virtual ~TargetTransformInfo() = 0; /// API below.
///
/// This is used by targets to construct a TTI wrapping their target-specific
/// implementaion that encodes appropriate costs for their target.
template <typename T> TargetTransformInfo(T Impl);
// Provide move semantics.
TargetTransformInfo(TargetTransformInfo &&Arg);
TargetTransformInfo &operator=(TargetTransformInfo &&RHS);
// We need to define the destructor out-of-line to define our sub-classes
// out-of-line.
~TargetTransformInfo();
/// \name Generic Target Information /// \name Generic Target Information
/// @{ /// @{
/// \brief Underlying constants for 'cost' values in this interface. /// \brief Underlying constants for 'cost' values in this interface.
/// ///
/// Many APIs in this interface return a cost. This enum defines the /// Many APIs in this interface return a cost. This enum defines the
/// fundamental values that should be used to interpret (and produce) those /// fundamental values that should be used to interpret (and produce) those
Show All 23 Linespublic:
/// analyzing a GEP's cost required more information. /// analyzing a GEP's cost required more information.
/// ///
/// Typically only the result type is required, and the operand type can be /// Typically only the result type is required, and the operand type can be
/// omitted. However, if the opcode is one of the cast instructions, the /// omitted. However, if the opcode is one of the cast instructions, the
/// operand type is required. /// operand type is required.
/// ///
/// The returned cost is defined in terms of \c TargetCostConstants, see its /// The returned cost is defined in terms of \c TargetCostConstants, see its
/// comments for a detailed explanation of the cost values. /// comments for a detailed explanation of the cost values.
virtual unsigned getOperationCost(unsigned Opcode, Type *Ty, unsigned getOperationCost(unsigned Opcode, Type *Ty,
Type *OpTy = nullptr) const; Type *OpTy = nullptr) const;
/// \brief Estimate the cost of a GEP operation when lowered. /// \brief Estimate the cost of a GEP operation when lowered.
/// ///
/// The contract for this function is the same as \c getOperationCost except /// The contract for this function is the same as \c getOperationCost except
/// that it supports an interface that provides extra information specific to /// that it supports an interface that provides extra information specific to
/// the GEP operation. /// the GEP operation.
virtual unsigned getGEPCost(const Value *Ptr, unsigned getGEPCost(const Value *Ptr, ArrayRef<const Value *> Operands) const;
ArrayRef<const Value *> Operands) const;
/// \brief Estimate the cost of a function call when lowered. /// \brief Estimate the cost of a function call when lowered.
/// ///
/// The contract for this is the same as \c getOperationCost except that it /// The contract for this is the same as \c getOperationCost except that it
/// supports an interface that provides extra information specific to call /// supports an interface that provides extra information specific to call
/// instructions. /// instructions.
/// ///
/// This is the most basic query for estimating call cost: it only knows the /// This is the most basic query for estimating call cost: it only knows the
/// function type and (potentially) the number of arguments at the call site. /// function type and (potentially) the number of arguments at the call site.
/// The latter is only interesting for varargs function types. /// The latter is only interesting for varargs function types.
virtual unsigned getCallCost(FunctionType *FTy, int NumArgs = -1) const; unsigned getCallCost(FunctionType *FTy, int NumArgs = -1) const;
/// \brief Estimate the cost of calling a specific function when lowered. /// \brief Estimate the cost of calling a specific function when lowered.
/// ///
/// This overload adds the ability to reason about the particular function /// This overload adds the ability to reason about the particular function
/// being called in the event it is a library call with special lowering. /// being called in the event it is a library call with special lowering.
virtual unsigned getCallCost(const Function *F, int NumArgs = -1) const; unsigned getCallCost(const Function *F, int NumArgs = -1) const;
/// \brief Estimate the cost of calling a specific function when lowered. /// \brief Estimate the cost of calling a specific function when lowered.
/// ///
/// This overload allows specifying a set of candidate argument values. /// This overload allows specifying a set of candidate argument values.
virtual unsigned getCallCost(const Function *F, unsigned getCallCost(const Function *F,
ArrayRef<const Value *> Arguments) const; ArrayRef<const Value *> Arguments) const;
/// \brief Estimate the cost of an intrinsic when lowered. /// \brief Estimate the cost of an intrinsic when lowered.
/// ///
/// Mirrors the \c getCallCost method but uses an intrinsic identifier. /// Mirrors the \c getCallCost method but uses an intrinsic identifier.
virtual unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy, unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<Type *> ParamTys) const; ArrayRef<Type *> ParamTys) const;
/// \brief Estimate the cost of an intrinsic when lowered. /// \brief Estimate the cost of an intrinsic when lowered.
/// ///
/// Mirrors the \c getCallCost method but uses an intrinsic identifier. /// Mirrors the \c getCallCost method but uses an intrinsic identifier.
virtual unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy, unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<const Value *> Arguments) const; ArrayRef<const Value *> Arguments) const;
/// \brief Estimate the cost of a given IR user when lowered. /// \brief Estimate the cost of a given IR user when lowered.
/// ///
/// This can estimate the cost of either a ConstantExpr or Instruction when /// This can estimate the cost of either a ConstantExpr or Instruction when
/// lowered. It has two primary advantages over the \c getOperationCost and /// lowered. It has two primary advantages over the \c getOperationCost and
/// \c getGEPCost above, and one significant disadvantage: it can only be /// \c getGEPCost above, and one significant disadvantage: it can only be
/// used when the IR construct has already been formed. /// used when the IR construct has already been formed.
/// ///
/// The advantages are that it can inspect the SSA use graph to reason more /// The advantages are that it can inspect the SSA use graph to reason more
/// accurately about the cost. For example, all-constant-GEPs can often be /// accurately about the cost. For example, all-constant-GEPs can often be
/// folded into a load or other instruction, but if they are used in some /// folded into a load or other instruction, but if they are used in some
/// other context they may not be folded. This routine can distinguish such /// other context they may not be folded. This routine can distinguish such
/// cases. /// cases.
/// ///
/// The returned cost is defined in terms of \c TargetCostConstants, see its /// The returned cost is defined in terms of \c TargetCostConstants, see its
/// comments for a detailed explanation of the cost values. /// comments for a detailed explanation of the cost values.
virtual unsigned getUserCost(const User *U) const; unsigned getUserCost(const User *U) const;
/// \brief hasBranchDivergence - Return true if branch divergence exists. /// \brief hasBranchDivergence - Return true if branch divergence exists.
/// Branch divergence has a significantly negative impact on GPU performance /// Branch divergence has a significantly negative impact on GPU performance
/// when threads in the same wavefront take different paths due to conditional /// when threads in the same wavefront take different paths due to conditional
/// branches. /// branches.
virtual bool hasBranchDivergence() const; bool hasBranchDivergence() const;
/// \brief Test whether calls to a function lower to actual program function /// \brief Test whether calls to a function lower to actual program function
/// calls. /// calls.
/// ///
/// The idea is to test whether the program is likely to require a 'call' /// The idea is to test whether the program is likely to require a 'call'
/// instruction or equivalent in order to call the given function. /// instruction or equivalent in order to call the given function.
/// ///
/// FIXME: It's not clear that this is a good or useful query API. Client's /// FIXME: It's not clear that this is a good or useful query API. Client's
/// should probably move to simpler cost metrics using the above. /// should probably move to simpler cost metrics using the above.
/// Alternatively, we could split the cost interface into distinct code-size /// Alternatively, we could split the cost interface into distinct code-size
/// and execution-speed costs. This would allow modelling the core of this /// and execution-speed costs. This would allow modelling the core of this
/// query more accurately as a call is a single small instruction, but /// query more accurately as a call is a single small instruction, but
/// incurs significant execution cost. /// incurs significant execution cost.
virtual bool isLoweredToCall(const Function *F) const; bool isLoweredToCall(const Function *F) const;
/// Parameters that control the generic loop unrolling transformation. /// Parameters that control the generic loop unrolling transformation.
struct UnrollingPreferences { struct UnrollingPreferences {
/// The cost threshold for the unrolled loop, compared to /// The cost threshold for the unrolled loop, compared to
/// CodeMetrics.NumInsts aggregated over all basic blocks in the loop body. /// CodeMetrics.NumInsts aggregated over all basic blocks in the loop body.
/// The unrolling factor is set such that the unrolled loop body does not /// The unrolling factor is set such that the unrolled loop body does not
/// exceed this cost. Set this to UINT_MAX to disable the loop body cost /// exceed this cost. Set this to UINT_MAX to disable the loop body cost
/// restriction. /// restriction.
Show All 25 Lines struct UnrollingPreferences {
/// loop body even when the number of loop iterations is not known at /// loop body even when the number of loop iterations is not known at
/// compile time). /// compile time).
bool Runtime; bool Runtime;
}; };
/// \brief Get target-customized preferences for the generic loop unrolling /// \brief Get target-customized preferences for the generic loop unrolling
/// transformation. The caller will initialize UP with the current /// transformation. The caller will initialize UP with the current
/// target-independent defaults. /// target-independent defaults.
virtual void getUnrollingPreferences(const Function *F, Loop *L, void getUnrollingPreferences(const Function *F, Loop *L,
UnrollingPreferences &UP) const; UnrollingPreferences &UP) const;
/// @} /// @}
/// \name Scalar Target Information /// \name Scalar Target Information
/// @{ /// @{
/// \brief Flags indicating the kind of support for population count. /// \brief Flags indicating the kind of support for population count.
/// ///
/// Compared to the SW implementation, HW support is supposed to /// Compared to the SW implementation, HW support is supposed to
/// significantly boost the performance when the population is dense, and it /// significantly boost the performance when the population is dense, and it
/// may or may not degrade performance if the population is sparse. A HW /// may or may not degrade performance if the population is sparse. A HW
/// support is considered as "Fast" if it can outperform, or is on a par /// support is considered as "Fast" if it can outperform, or is on a par
/// with, SW implementation when the population is sparse; otherwise, it is /// with, SW implementation when the population is sparse; otherwise, it is
/// considered as "Slow". /// considered as "Slow".
enum PopcntSupportKind { PSK_Software, PSK_SlowHardware, PSK_FastHardware }; enum PopcntSupportKind { PSK_Software, PSK_SlowHardware, PSK_FastHardware };
/// \brief Return true if the specified immediate is legal add immediate, that /// \brief Return true if the specified immediate is legal add immediate, that
/// is the target has add instructions which can add a register with the /// is the target has add instructions which can add a register with the
/// immediate without having to materialize the immediate into a register. /// immediate without having to materialize the immediate into a register.
virtual bool isLegalAddImmediate(int64_t Imm) const; bool isLegalAddImmediate(int64_t Imm) const;
/// \brief Return true if the specified immediate is legal icmp immediate, /// \brief Return true if the specified immediate is legal icmp immediate,
/// that is the target has icmp instructions which can compare a register /// that is the target has icmp instructions which can compare a register
/// against the immediate without having to materialize the immediate into a /// against the immediate without having to materialize the immediate into a
/// register. /// register.
virtual bool isLegalICmpImmediate(int64_t Imm) const; bool isLegalICmpImmediate(int64_t Imm) const;
/// \brief Return true if the addressing mode represented by AM is legal for /// \brief Return true if the addressing mode represented by AM is legal for
/// this target, for a load/store of the specified type. /// this target, for a load/store of the specified type.
/// The type may be VoidTy, in which case only return true if the addressing /// The type may be VoidTy, in which case only return true if the addressing
/// mode is legal for a load/store of any legal type. /// mode is legal for a load/store of any legal type.
/// TODO: Handle pre/postinc as well. /// TODO: Handle pre/postinc as well.
virtual bool isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV, bool isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV, int64_t BaseOffset,
int64_t BaseOffset, bool HasBaseReg, bool HasBaseReg, int64_t Scale) const;
int64_t Scale) const;
/// \brief Return true if the target works with masked instruction /// \brief Return true if the target works with masked instruction
/// AVX2 allows masks for consecutive load and store for i32 and i64 elements. /// AVX2 allows masks for consecutive load and store for i32 and i64 elements.
/// AVX-512 architecture will also allow masks for non-consecutive memory /// AVX-512 architecture will also allow masks for non-consecutive memory
/// accesses. /// accesses.
virtual bool isLegalMaskedStore(Type *DataType, int Consecutive) const; bool isLegalMaskedStore(Type *DataType, int Consecutive) const;
virtual bool isLegalMaskedLoad(Type *DataType, int Consecutive) const; bool isLegalMaskedLoad(Type *DataType, int Consecutive) const;
/// \brief Return the cost of the scaling factor used in the addressing /// \brief Return the cost of the scaling factor used in the addressing
/// mode represented by AM for this target, for a load/store /// mode represented by AM for this target, for a load/store
/// of the specified type. /// of the specified type.
/// If the AM is supported, the return value must be >= 0. /// If the AM is supported, the return value must be >= 0.
/// If the AM is not supported, it returns a negative value. /// If the AM is not supported, it returns a negative value.
/// TODO: Handle pre/postinc as well. /// TODO: Handle pre/postinc as well.
virtual int getScalingFactorCost(Type *Ty, GlobalValue *BaseGV, int getScalingFactorCost(Type *Ty, GlobalValue *BaseGV, int64_t BaseOffset,
int64_t BaseOffset, bool HasBaseReg, bool HasBaseReg, int64_t Scale) const;
int64_t Scale) const;
/// \brief Return true if it's free to truncate a value of type Ty1 to type /// \brief Return true if it's free to truncate a value of type Ty1 to type
/// Ty2. e.g. On x86 it's free to truncate a i32 value in register EAX to i16 /// Ty2. e.g. On x86 it's free to truncate a i32 value in register EAX to i16
/// by referencing its sub-register AX. /// by referencing its sub-register AX.
virtual bool isTruncateFree(Type *Ty1, Type *Ty2) const; bool isTruncateFree(Type *Ty1, Type *Ty2) const;
/// \brief Return true if this type is legal. /// \brief Return true if this type is legal.
virtual bool isTypeLegal(Type *Ty) const; bool isTypeLegal(Type *Ty) const;
/// \brief Returns the target's jmp_buf alignment in bytes. /// \brief Returns the target's jmp_buf alignment in bytes.
virtual unsigned getJumpBufAlignment() const; unsigned getJumpBufAlignment() const;
/// \brief Returns the target's jmp_buf size in bytes. /// \brief Returns the target's jmp_buf size in bytes.
virtual unsigned getJumpBufSize() const; unsigned getJumpBufSize() const;
/// \brief Return true if switches should be turned into lookup tables for the /// \brief Return true if switches should be turned into lookup tables for the
/// target. /// target.
virtual bool shouldBuildLookupTables() const; bool shouldBuildLookupTables() const;
/// \brief Return hardware support for population count. /// \brief Return hardware support for population count.
virtual PopcntSupportKind getPopcntSupport(unsigned IntTyWidthInBit) const; PopcntSupportKind getPopcntSupport(unsigned IntTyWidthInBit) const;
/// \brief Return true if the hardware has a fast square-root instruction. /// \brief Return true if the hardware has a fast square-root instruction.
virtual bool haveFastSqrt(Type *Ty) const; bool haveFastSqrt(Type *Ty) const;
/// \brief Return the expected cost of materializing for the given integer /// \brief Return the expected cost of materializing for the given integer
/// immediate of the specified type. /// immediate of the specified type.
virtual unsigned getIntImmCost(const APInt &Imm, Type *Ty) const; unsigned getIntImmCost(const APInt &Imm, Type *Ty) const;
/// \brief Return the expected cost of materialization for the given integer /// \brief Return the expected cost of materialization for the given integer
/// immediate of the specified type for a given instruction. The cost can be /// immediate of the specified type for a given instruction. The cost can be
/// zero if the immediate can be folded into the specified instruction. /// zero if the immediate can be folded into the specified instruction.
virtual unsigned getIntImmCost(unsigned Opc, unsigned Idx, const APInt &Imm, unsigned getIntImmCost(unsigned Opc, unsigned Idx, const APInt &Imm,
Type *Ty) const;
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty) const; Type *Ty) const;
virtual unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx,
const APInt &Imm, Type *Ty) const;
/// @} /// @}
/// \name Vector Target Information /// \name Vector Target Information
/// @{ /// @{
/// \brief The various kinds of shuffle patterns for vector queries. /// \brief The various kinds of shuffle patterns for vector queries.
enum ShuffleKind { enum ShuffleKind {
SK_Broadcast, ///< Broadcast element 0 to all other elements. SK_Broadcast, ///< Broadcast element 0 to all other elements.
Show All 12 Linespublic:
}; };
/// \brief Additional properties of an operand's values. /// \brief Additional properties of an operand's values.
enum OperandValueProperties { OP_None = 0, OP_PowerOf2 = 1 }; enum OperandValueProperties { OP_None = 0, OP_PowerOf2 = 1 };
/// \return The number of scalar or vector registers that the target has. /// \return The number of scalar or vector registers that the target has.
/// If 'Vectors' is true, it returns the number of vector registers. If it is /// If 'Vectors' is true, it returns the number of vector registers. If it is
/// set to false, it returns the number of scalar registers. /// set to false, it returns the number of scalar registers.
virtual unsigned getNumberOfRegisters(bool Vector) const; unsigned getNumberOfRegisters(bool Vector) const;
/// \return The width of the largest scalar or vector register type. /// \return The width of the largest scalar or vector register type.
virtual unsigned getRegisterBitWidth(bool Vector) const; unsigned getRegisterBitWidth(bool Vector) const;
/// \return The maximum interleave factor that any transform should try to /// \return The maximum interleave factor that any transform should try to
/// perform for this target. This number depends on the level of parallelism /// perform for this target. This number depends on the level of parallelism
/// and the number of execution units in the CPU. /// and the number of execution units in the CPU.
virtual unsigned getMaxInterleaveFactor() const; unsigned getMaxInterleaveFactor() const;
/// \return The expected cost of arithmetic ops, such as mul, xor, fsub, etc. /// \return The expected cost of arithmetic ops, such as mul, xor, fsub, etc.
virtual unsigned unsigned
getArithmeticInstrCost(unsigned Opcode, Type *Ty, getArithmeticInstrCost(unsigned Opcode, Type *Ty,
OperandValueKind Opd1Info = OK_AnyValue, OperandValueKind Opd1Info = OK_AnyValue,
OperandValueKind Opd2Info = OK_AnyValue, OperandValueKind Opd2Info = OK_AnyValue,
OperandValueProperties Opd1PropInfo = OP_None, OperandValueProperties Opd1PropInfo = OP_None,
OperandValueProperties Opd2PropInfo = OP_None) const; OperandValueProperties Opd2PropInfo = OP_None) const;
/// \return The cost of a shuffle instruction of kind Kind and of type Tp. /// \return The cost of a shuffle instruction of kind Kind and of type Tp.
/// The index and subtype parameters are used by the subvector insertion and /// The index and subtype parameters are used by the subvector insertion and
/// extraction shuffle kinds. /// extraction shuffle kinds.
virtual unsigned getShuffleCost(ShuffleKind Kind, Type *Tp, int Index = 0, unsigned getShuffleCost(ShuffleKind Kind, Type *Tp, int Index = 0,
Type *SubTp = nullptr) const; Type *SubTp = nullptr) const;
/// \return The expected cost of cast instructions, such as bitcast, trunc, /// \return The expected cost of cast instructions, such as bitcast, trunc,
/// zext, etc. /// zext, etc.
virtual unsigned getCastInstrCost(unsigned Opcode, Type *Dst, unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) const;
Type *Src) const;
/// \return The expected cost of control-flow related instructions such as /// \return The expected cost of control-flow related instructions such as
/// Phi, Ret, Br. /// Phi, Ret, Br.
virtual unsigned getCFInstrCost(unsigned Opcode) const; unsigned getCFInstrCost(unsigned Opcode) const;
/// \returns The expected cost of compare and select instructions. /// \returns The expected cost of compare and select instructions.
virtual unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy = nullptr) const; Type *CondTy = nullptr) const;
/// \return The expected cost of vector Insert and Extract. /// \return The expected cost of vector Insert and Extract.
/// Use -1 to indicate that there is no information on the index value. /// Use -1 to indicate that there is no information on the index value.
virtual unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index = -1) const; unsigned Index = -1) const;
/// \return The cost of Load and Store instructions. /// \return The cost of Load and Store instructions.
virtual unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned Alignment,
unsigned AddressSpace) const; unsigned AddressSpace) const;
/// \return The cost of masked Load and Store instructions. /// \return The cost of masked Load and Store instructions.
virtual unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned Alignment,
unsigned AddressSpace) const; unsigned AddressSpace) const;
/// \brief Calculate the cost of performing a vector reduction. /// \brief Calculate the cost of performing a vector reduction.
/// ///
/// This is the cost of reducing the vector value of type \p Ty to a scalar /// This is the cost of reducing the vector value of type \p Ty to a scalar
/// value using the operation denoted by \p Opcode. The form of the reduction /// value using the operation denoted by \p Opcode. The form of the reduction
/// can either be a pairwise reduction or a reduction that splits the vector /// can either be a pairwise reduction or a reduction that splits the vector
/// at every reduction level. /// at every reduction level.
/// ///
/// Pairwise: /// Pairwise:
/// (v0, v1, v2, v3) /// (v0, v1, v2, v3)
/// ((v0+v1), (v2, v3), undef, undef) /// ((v0+v1), (v2, v3), undef, undef)
/// Split: /// Split:
/// (v0, v1, v2, v3) /// (v0, v1, v2, v3)
/// ((v0+v2), (v1+v3), undef, undef) /// ((v0+v2), (v1+v3), undef, undef)
virtual unsigned getReductionCost(unsigned Opcode, Type *Ty, unsigned getReductionCost(unsigned Opcode, Type *Ty,
bool IsPairwiseForm) const; bool IsPairwiseForm) const;
/// \returns The cost of Intrinsic instructions. /// \returns The cost of Intrinsic instructions.
virtual unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy, unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
ArrayRef<Type *> Tys) const; ArrayRef<Type *> Tys) const;
/// \returns The number of pieces into which the provided type must be /// \returns The number of pieces into which the provided type must be
/// split during legalization. Zero is returned when the answer is unknown. /// split during legalization. Zero is returned when the answer is unknown.
virtual unsigned getNumberOfParts(Type *Tp) const; unsigned getNumberOfParts(Type *Tp) const;
/// \returns The cost of the address computation. For most targets this can be /// \returns The cost of the address computation. For most targets this can be
/// merged into the instruction indexing mode. Some targets might want to /// merged into the instruction indexing mode. Some targets might want to
/// distinguish between address computation for memory operations on vector /// distinguish between address computation for memory operations on vector
/// types and scalar types. Such targets should override this function. /// types and scalar types. Such targets should override this function.
/// The 'IsComplex' parameter is a hint that the address computation is likely /// The 'IsComplex' parameter is a hint that the address computation is likely
/// to involve multiple instructions and as such unlikely to be merged into /// to involve multiple instructions and as such unlikely to be merged into
/// the address indexing mode. /// the address indexing mode.
virtual unsigned getAddressComputationCost(Type *Ty, unsigned getAddressComputationCost(Type *Ty, bool IsComplex = false) const;
bool IsComplex = false) const;
/// \returns The cost, if any, of keeping values of the given types alive /// \returns The cost, if any, of keeping values of the given types alive
/// over a callsite. /// over a callsite.
/// ///
/// Some types may require the use of register classes that do not have /// Some types may require the use of register classes that do not have
/// any callee-saved registers, so would require a spill and fill. /// any callee-saved registers, so would require a spill and fill.
virtual unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) const; unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) const;
/// \returns True if the intrinsic is a supported memory intrinsic. Info /// \returns True if the intrinsic is a supported memory intrinsic. Info
/// will contain additional information - whether the intrinsic may write /// will contain additional information - whether the intrinsic may write
/// or read to memory, volatility and the pointer. Info is undefined /// or read to memory, volatility and the pointer. Info is undefined
/// if false is returned. /// if false is returned.
virtual bool getTgtMemIntrinsic(IntrinsicInst *Inst, bool getTgtMemIntrinsic(IntrinsicInst *Inst, MemIntrinsicInfo &Info) const;
MemIntrinsicInfo &Info) const;
/// \returns A value which is the result of the given memory intrinsic. New /// \returns A value which is the result of the given memory intrinsic. New
/// instructions may be created to extract the result from the given intrinsic /// instructions may be created to extract the result from the given intrinsic
/// memory operation. Returns nullptr if the target cannot create a result /// memory operation. Returns nullptr if the target cannot create a result
/// from the given intrinsic. /// from the given intrinsic.
virtual Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst, Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst,
Type *ExpectedType) const; Type *ExpectedType) const;
/// @} /// @}
/// Analysis group identification. private:
/// \brief The abstract base class used to type erase specific TTI
/// implementations.
class Concept;
/// \brief The template model for the base class which wraps a concrete
/// implementation in a type erased interface.
template <typename T> class Model;
std::unique_ptr<Concept> TTIImpl;
};
class TargetTransformInfo::Concept {
public:
virtual ~Concept() = 0;
virtual unsigned getOperationCost(unsigned Opcode, Type *Ty, Type *OpTy) = 0;
virtual unsigned getGEPCost(const Value *Ptr,
ArrayRef<const Value *> Operands) = 0;
virtual unsigned getCallCost(FunctionType *FTy, int NumArgs) = 0;
virtual unsigned getCallCost(const Function *F, int NumArgs) = 0;
virtual unsigned getCallCost(const Function *F,
ArrayRef<const Value *> Arguments) = 0;
virtual unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<Type *> ParamTys) = 0;
virtual unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<const Value *> Arguments) = 0;
virtual unsigned getUserCost(const User *U) = 0;
virtual bool hasBranchDivergence() = 0;
virtual bool isLoweredToCall(const Function *F) = 0;
virtual void getUnrollingPreferences(const Function *F, Loop *L,
UnrollingPreferences &UP) = 0;
virtual bool isLegalAddImmediate(int64_t Imm) = 0;
virtual bool isLegalICmpImmediate(int64_t Imm) = 0;
virtual bool isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV,
int64_t BaseOffset, bool HasBaseReg,
int64_t Scale) = 0;
virtual bool isLegalMaskedStore(Type *DataType, int Consecutive) = 0;
virtual bool isLegalMaskedLoad(Type *DataType, int Consecutive) = 0;
virtual int getScalingFactorCost(Type *Ty, GlobalValue *BaseGV,
int64_t BaseOffset, bool HasBaseReg,
int64_t Scale) = 0;
virtual bool isTruncateFree(Type *Ty1, Type *Ty2) = 0;
virtual bool isTypeLegal(Type *Ty) = 0;
virtual unsigned getJumpBufAlignment() = 0;
virtual unsigned getJumpBufSize() = 0;
virtual bool shouldBuildLookupTables() = 0;
virtual PopcntSupportKind getPopcntSupport(unsigned IntTyWidthInBit) = 0;
virtual bool haveFastSqrt(Type *Ty) = 0;
virtual unsigned getIntImmCost(const APInt &Imm, Type *Ty) = 0;
virtual unsigned getIntImmCost(unsigned Opc, unsigned Idx, const APInt &Imm,
Type *Ty) = 0;
virtual unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx,
const APInt &Imm, Type *Ty) = 0;
virtual unsigned getNumberOfRegisters(bool Vector) = 0;
virtual unsigned getRegisterBitWidth(bool Vector) = 0;
virtual unsigned getMaxInterleaveFactor() = 0;
virtual unsigned
getArithmeticInstrCost(unsigned Opcode, Type *Ty, OperandValueKind Opd1Info,
OperandValueKind Opd2Info,
OperandValueProperties Opd1PropInfo,
OperandValueProperties Opd2PropInfo) = 0;
virtual unsigned getShuffleCost(ShuffleKind Kind, Type *Tp, int Index,
Type *SubTp) = 0;
virtual unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) = 0;
virtual unsigned getCFInstrCost(unsigned Opcode) = 0;
virtual unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy) = 0;
virtual unsigned getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index) = 0;
virtual unsigned getMemoryOpCost(unsigned Opcode, Type *Src,
unsigned Alignment,
unsigned AddressSpace) = 0;
virtual unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src,
unsigned Alignment,
unsigned AddressSpace) = 0;
virtual unsigned getReductionCost(unsigned Opcode, Type *Ty,
bool IsPairwiseForm) = 0;
virtual unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
ArrayRef<Type *> Tys) = 0;
virtual unsigned getNumberOfParts(Type *Tp) = 0;
virtual unsigned getAddressComputationCost(Type *Ty, bool IsComplex) = 0;
virtual unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) = 0;
virtual bool getTgtMemIntrinsic(IntrinsicInst *Inst,
MemIntrinsicInfo &Info) = 0;
virtual Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst,
Type *ExpectedType) = 0;
};
template <typename T>
class TargetTransformInfo::Model final : public TargetTransformInfo::Concept {
T Impl;
public:
Model(T Impl) : Impl(std::move(Impl)) {}
~Model() override {}
unsigned getOperationCost(unsigned Opcode, Type *Ty, Type *OpTy) override {
return Impl.getOperationCost(Opcode, Ty, OpTy);
}
unsigned getGEPCost(const Value *Ptr,
ArrayRef<const Value *> Operands) override {
return Impl.getGEPCost(Ptr, Operands);
}
unsigned getCallCost(FunctionType *FTy, int NumArgs) override {
return Impl.getCallCost(FTy, NumArgs);
}
unsigned getCallCost(const Function *F, int NumArgs) override {
return Impl.getCallCost(F, NumArgs);
}
unsigned getCallCost(const Function *F,
ArrayRef<const Value *> Arguments) override {
return Impl.getCallCost(F, Arguments);
}
unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<Type *> ParamTys) override {
return Impl.getIntrinsicCost(IID, RetTy, ParamTys);
}
unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<const Value *> Arguments) override {
return Impl.getIntrinsicCost(IID, RetTy, Arguments);
}
unsigned getUserCost(const User *U) override { return Impl.getUserCost(U); }
bool hasBranchDivergence() override { return Impl.hasBranchDivergence(); }
bool isLoweredToCall(const Function *F) override {
return Impl.isLoweredToCall(F);
}
void getUnrollingPreferences(const Function *F, Loop *L,
UnrollingPreferences &UP) override {
return Impl.getUnrollingPreferences(F, L, UP);
}
bool isLegalAddImmediate(int64_t Imm) override {
return Impl.isLegalAddImmediate(Imm);
}
bool isLegalICmpImmediate(int64_t Imm) override {
return Impl.isLegalICmpImmediate(Imm);
}
bool isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV, int64_t BaseOffset,
bool HasBaseReg, int64_t Scale) override {
return Impl.isLegalAddressingMode(Ty, BaseGV, BaseOffset, HasBaseReg,
Scale);
}
bool isLegalMaskedStore(Type *DataType, int Consecutive) override {
return Impl.isLegalMaskedStore(DataType, Consecutive);
}
bool isLegalMaskedLoad(Type *DataType, int Consecutive) override {
return Impl.isLegalMaskedLoad(DataType, Consecutive);
}
int getScalingFactorCost(Type *Ty, GlobalValue *BaseGV, int64_t BaseOffset,
bool HasBaseReg, int64_t Scale) override {
return Impl.getScalingFactorCost(Ty, BaseGV, BaseOffset, HasBaseReg, Scale);
}
bool isTruncateFree(Type *Ty1, Type *Ty2) override {
return Impl.isTruncateFree(Ty1, Ty2);
}
bool isTypeLegal(Type *Ty) override { return Impl.isTypeLegal(Ty); }
unsigned getJumpBufAlignment() override { return Impl.getJumpBufAlignment(); }
unsigned getJumpBufSize() override { return Impl.getJumpBufSize(); }
bool shouldBuildLookupTables() override {
return Impl.shouldBuildLookupTables();
}
PopcntSupportKind getPopcntSupport(unsigned IntTyWidthInBit) override {
return Impl.getPopcntSupport(IntTyWidthInBit);
}
bool haveFastSqrt(Type *Ty) override { return Impl.haveFastSqrt(Ty); }
unsigned getIntImmCost(const APInt &Imm, Type *Ty) override {
return Impl.getIntImmCost(Imm, Ty);
}
unsigned getIntImmCost(unsigned Opc, unsigned Idx, const APInt &Imm,
Type *Ty) override {
return Impl.getIntImmCost(Opc, Idx, Imm, Ty);
}
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty) override {
return Impl.getIntImmCost(IID, Idx, Imm, Ty);
}
unsigned getNumberOfRegisters(bool Vector) override {
return Impl.getNumberOfRegisters(Vector);
}
unsigned getRegisterBitWidth(bool Vector) override {
return Impl.getRegisterBitWidth(Vector);
}
unsigned getMaxInterleaveFactor() override {
return Impl.getMaxInterleaveFactor();
}
unsigned
getArithmeticInstrCost(unsigned Opcode, Type *Ty, OperandValueKind Opd1Info,
OperandValueKind Opd2Info,
OperandValueProperties Opd1PropInfo,
OperandValueProperties Opd2PropInfo) override {
return Impl.getArithmeticInstrCost(Opcode, Ty, Opd1Info, Opd2Info,
Opd1PropInfo, Opd2PropInfo);
}
unsigned getShuffleCost(ShuffleKind Kind, Type *Tp, int Index,
Type *SubTp) override {
return Impl.getShuffleCost(Kind, Tp, Index, SubTp);
}
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) override {
return Impl.getCastInstrCost(Opcode, Dst, Src);
}
unsigned getCFInstrCost(unsigned Opcode) override {
return Impl.getCFInstrCost(Opcode);
}
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy) override {
return Impl.getCmpSelInstrCost(Opcode, ValTy, CondTy);
}
unsigned getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index) override {
return Impl.getVectorInstrCost(Opcode, Val, Index);
}
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) override {
return Impl.getMemoryOpCost(Opcode, Src, Alignment, AddressSpace);
}
unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) override {
return Impl.getMaskedMemoryOpCost(Opcode, Src, Alignment, AddressSpace);
}
unsigned getReductionCost(unsigned Opcode, Type *Ty,
bool IsPairwiseForm) override {
return Impl.getReductionCost(Opcode, Ty, IsPairwiseForm);
}
unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
ArrayRef<Type *> Tys) override {
return Impl.getIntrinsicInstrCost(ID, RetTy, Tys);
}
unsigned getNumberOfParts(Type *Tp) override {
return Impl.getNumberOfParts(Tp);
}
unsigned getAddressComputationCost(Type *Ty, bool IsComplex) override {
return Impl.getAddressComputationCost(Ty, IsComplex);
}
unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) override {
return Impl.getCostOfKeepingLiveOverCall(Tys);
}
bool getTgtMemIntrinsic(IntrinsicInst *Inst,
MemIntrinsicInfo &Info) override {
return Impl.getTgtMemIntrinsic(Inst, Info);
}
Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst,
Type *ExpectedType) override {
return Impl.getOrCreateResultFromMemIntrinsic(Inst, ExpectedType);
}
};
template <typename T>
TargetTransformInfo::TargetTransformInfo(T Impl)
: TTIImpl(new Model<T>(Impl)) {}
/// \brief Wrapper pass for TargetTransformInfo.
///
/// This pass can be constructed from a TTI object which it stores internally
/// and is queried by passes.
class TargetTransformInfoWrapperPass : public ImmutablePass {
TargetTransformInfo TTI;
virtual void anchor();
public:
static char ID; static char ID;
/// \brief We must provide a default constructor for the pass but it should
/// never be used.
///
/// Use the constructor below or call one of the creation routines.
TargetTransformInfoWrapperPass();
explicit TargetTransformInfoWrapperPass(TargetTransformInfo TTI);
TargetTransformInfo &getTTI() { return TTI; }
const TargetTransformInfo &getTTI() const { return TTI; }
}; };
/// \brief Create the base case instance of a pass in the TTI analysis group. /// \brief Create the base case instance of a pass in the TTI analysis group.
/// ///
/// This class provides the base case for the stack of TTI analyzes. It doesn't /// This class provides the base case for the stack of TTI analyzes. It doesn't
/// delegate to anything and uses the STTI and VTTI objects passed in to /// delegate to anything and uses the STTI and VTTI objects passed in to
/// satisfy the queries. /// satisfy the queries.
ImmutablePass *createNoTargetTransformInfoPass(); ImmutablePass *createNoTargetTransformInfoPass(const DataLayout *DL);
} // End llvm namespace } // End llvm namespace
#endif #endif

llvm/trunk/include/llvm/Analysis/TargetTransformInfoImpl.h

//===- TargetTransformInfoImpl.h --------------------------------*- C++ -*-===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
/// \file
/// This file provides helpers for the implementation of
/// a TargetTransformInfo-conforming class.
///
//===----------------------------------------------------------------------===//
#ifndef LLVM_ANALYSIS_TARGETTRANSFORMINFOIMPL_H
#define LLVM_ANALYSIS_TARGETTRANSFORMINFOIMPL_H
#include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/IR/CallSite.h"
#include "llvm/IR/DataLayout.h"
#include "llvm/IR/Operator.h"
#include "llvm/IR/Function.h"
#include "llvm/IR/Type.h"
namespace llvm {
/// \brief Base class for use as a mix-in that aids implementing
/// a TargetTransformInfo-compatible class.
class TargetTransformInfoImplBase {
protected:
typedef TargetTransformInfo TTI;
const DataLayout *DL;
explicit TargetTransformInfoImplBase(const DataLayout *DL)
: DL(DL) {}
public:
// Provide value semantics. MSVC requires that we spell all of these out.
TargetTransformInfoImplBase(const TargetTransformInfoImplBase &Arg)
: DL(Arg.DL) {}
TargetTransformInfoImplBase(TargetTransformInfoImplBase &&Arg)
: DL(std::move(Arg.DL)) {}
TargetTransformInfoImplBase &
operator=(const TargetTransformInfoImplBase &RHS) {
DL = RHS.DL;
return *this;
}
TargetTransformInfoImplBase &operator=(TargetTransformInfoImplBase &&RHS) {
DL = std::move(RHS.DL);
return *this;
}
unsigned getOperationCost(unsigned Opcode, Type *Ty, Type *OpTy) {
switch (Opcode) {
default:
// By default, just classify everything as 'basic'.
return TTI::TCC_Basic;
case Instruction::GetElementPtr:
llvm_unreachable("Use getGEPCost for GEP operations!");
case Instruction::BitCast:
assert(OpTy && "Cast instructions must provide the operand type");
if (Ty == OpTy || (Ty->isPointerTy() && OpTy->isPointerTy()))
// Identity and pointer-to-pointer casts are free.
return TTI::TCC_Free;
// Otherwise, the default basic cost is used.
return TTI::TCC_Basic;
case Instruction::IntToPtr: {
if (!DL)
return TTI::TCC_Basic;
// An inttoptr cast is free so long as the input is a legal integer type
// which doesn't contain values outside the range of a pointer.
unsigned OpSize = OpTy->getScalarSizeInBits();
if (DL->isLegalInteger(OpSize) &&
OpSize <= DL->getPointerTypeSizeInBits(Ty))
return TTI::TCC_Free;
// Otherwise it's not a no-op.
return TTI::TCC_Basic;
}
case Instruction::PtrToInt: {
if (!DL)
return TTI::TCC_Basic;
// A ptrtoint cast is free so long as the result is large enough to store
// the pointer, and a legal integer type.
unsigned DestSize = Ty->getScalarSizeInBits();
if (DL->isLegalInteger(DestSize) &&
DestSize >= DL->getPointerTypeSizeInBits(OpTy))
return TTI::TCC_Free;
// Otherwise it's not a no-op.
return TTI::TCC_Basic;
}
case Instruction::Trunc:
// trunc to a native type is free (assuming the target has compare and
// shift-right of the same width).
if (DL && DL->isLegalInteger(DL->getTypeSizeInBits(Ty)))
return TTI::TCC_Free;
return TTI::TCC_Basic;
}
}
unsigned getGEPCost(const Value *Ptr, ArrayRef<const Value *> Operands) {
// In the basic model, we just assume that all-constant GEPs will be folded
// into their uses via addressing modes.
for (unsigned Idx = 0, Size = Operands.size(); Idx != Size; ++Idx)
if (!isa<Constant>(Operands[Idx]))
return TTI::TCC_Basic;
return TTI::TCC_Free;
}
unsigned getCallCost(FunctionType *FTy, int NumArgs) {
assert(FTy && "FunctionType must be provided to this routine.");
// The target-independent implementation just measures the size of the
// function by approximating that each argument will take on average one
// instruction to prepare.
if (NumArgs < 0)
// Set the argument number to the number of explicit arguments in the
// function.
NumArgs = FTy->getNumParams();
return TTI::TCC_Basic * (NumArgs + 1);
}
unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<Type *> ParamTys) {
switch (IID) {
default:
// Intrinsics rarely (if ever) have normal argument setup constraints.
// Model them as having a basic instruction cost.
// FIXME: This is wrong for libc intrinsics.
return TTI::TCC_Basic;
case Intrinsic::annotation:
case Intrinsic::assume:
case Intrinsic::dbg_declare:
case Intrinsic::dbg_value:
case Intrinsic::invariant_start:
case Intrinsic::invariant_end:
case Intrinsic::lifetime_start:
case Intrinsic::lifetime_end:
case Intrinsic::objectsize:
case Intrinsic::ptr_annotation:
case Intrinsic::var_annotation:
case Intrinsic::experimental_gc_result_int:
case Intrinsic::experimental_gc_result_float:
case Intrinsic::experimental_gc_result_ptr:
case Intrinsic::experimental_gc_result:
case Intrinsic::experimental_gc_relocate:
// These intrinsics don't actually represent code after lowering.
return TTI::TCC_Free;
}
}
bool hasBranchDivergence() { return false; }
bool isLoweredToCall(const Function *F) {
// FIXME: These should almost certainly not be handled here, and instead
// handled with the help of TLI or the target itself. This was largely
// ported from existing analysis heuristics here so that such refactorings
// can take place in the future.
if (F->isIntrinsic())
return false;
if (F->hasLocalLinkage() || !F->hasName())
return true;
StringRef Name = F->getName();
// These will all likely lower to a single selection DAG node.
if (Name == "copysign" || Name == "copysignf" || Name == "copysignl" ||
Name == "fabs" || Name == "fabsf" || Name == "fabsl" || Name == "sin" ||
Name == "fmin" || Name == "fminf" || Name == "fminl" ||
Name == "fmax" || Name == "fmaxf" || Name == "fmaxl" ||
Name == "sinf" || Name == "sinl" || Name == "cos" || Name == "cosf" ||
Name == "cosl" || Name == "sqrt" || Name == "sqrtf" || Name == "sqrtl")
return false;
// These are all likely to be optimized into something smaller.
if (Name == "pow" || Name == "powf" || Name == "powl" || Name == "exp2" ||
Name == "exp2l" || Name == "exp2f" || Name == "floor" ||
Name == "floorf" || Name == "ceil" || Name == "round" ||
Name == "ffs" || Name == "ffsl" || Name == "abs" || Name == "labs" ||
Name == "llabs")
return false;
return true;
}
void getUnrollingPreferences(const Function *, Loop *,
TTI::UnrollingPreferences &) {}
bool isLegalAddImmediate(int64_t Imm) { return false; }
bool isLegalICmpImmediate(int64_t Imm) { return false; }
bool isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV, int64_t BaseOffset,
bool HasBaseReg, int64_t Scale) {
// Guess that reg+reg addressing is allowed. This heuristic is taken from
// the implementation of LSR.
return !BaseGV && BaseOffset == 0 && Scale <= 1;
}
bool isLegalMaskedStore(Type *DataType, int Consecutive) { return false; }
bool isLegalMaskedLoad(Type *DataType, int Consecutive) { return false; }
int getScalingFactorCost(Type *Ty, GlobalValue *BaseGV, int64_t BaseOffset,
bool HasBaseReg, int64_t Scale) {
// Guess that all legal addressing mode are free.
if (isLegalAddressingMode(Ty, BaseGV, BaseOffset, HasBaseReg, Scale))
return 0;
return -1;
}
bool isTruncateFree(Type *Ty1, Type *Ty2) { return false; }
bool isTypeLegal(Type *Ty) { return false; }
unsigned getJumpBufAlignment() { return 0; }
unsigned getJumpBufSize() { return 0; }
bool shouldBuildLookupTables() { return true; }
TTI::PopcntSupportKind getPopcntSupport(unsigned IntTyWidthInBit) {
return TTI::PSK_Software;
}
bool haveFastSqrt(Type *Ty) { return false; }
unsigned getIntImmCost(const APInt &Imm, Type *Ty) { return TTI::TCC_Basic; }
unsigned getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm,
Type *Ty) {
return TTI::TCC_Free;
}
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty) {
return TTI::TCC_Free;
}
unsigned getNumberOfRegisters(bool Vector) { return 8; }
unsigned getRegisterBitWidth(bool Vector) { return 32; }
unsigned getMaxInterleaveFactor() { return 1; }
unsigned getArithmeticInstrCost(unsigned Opcode, Type *Ty,
TTI::OperandValueKind Opd1Info,
TTI::OperandValueKind Opd2Info,
TTI::OperandValueProperties Opd1PropInfo,
TTI::OperandValueProperties Opd2PropInfo) {
return 1;
}
unsigned getShuffleCost(TTI::ShuffleKind Kind, Type *Ty, int Index,
Type *SubTp) {
return 1;
}
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) { return 1; }
unsigned getCFInstrCost(unsigned Opcode) { return 1; }
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy) {
return 1;
}
unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index) {
return 1;
}
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) {
return 1;
}
unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) {
return 1;
}
unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
ArrayRef<Type *> Tys) {
return 1;
}
unsigned getNumberOfParts(Type *Tp) { return 0; }
unsigned getAddressComputationCost(Type *Tp, bool) { return 0; }
unsigned getReductionCost(unsigned, Type *, bool) { return 1; }
unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) { return 0; }
bool getTgtMemIntrinsic(IntrinsicInst *Inst, MemIntrinsicInfo &Info) {
return false;
}
Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst,
Type *ExpectedType) {
return nullptr;
}
};
/// \brief CRTP base class for use as a mix-in that aids implementing
/// a TargetTransformInfo-compatible class.
template <typename T>
class TargetTransformInfoImplCRTPBase : public TargetTransformInfoImplBase {
private:
typedef TargetTransformInfoImplBase BaseT;
protected:
explicit TargetTransformInfoImplCRTPBase(const DataLayout *DL)
: BaseT(DL) {}
public:
// Provide value semantics. MSVC requires that we spell all of these out.
TargetTransformInfoImplCRTPBase(const TargetTransformInfoImplCRTPBase &Arg)
: BaseT(static_cast<const BaseT &>(Arg)) {}
TargetTransformInfoImplCRTPBase(TargetTransformInfoImplCRTPBase &&Arg)
: BaseT(std::move(static_cast<BaseT &>(Arg))) {}
TargetTransformInfoImplCRTPBase &
operator=(const TargetTransformInfoImplCRTPBase &RHS) {
BaseT::operator=(static_cast<const BaseT &>(RHS));
return *this;
}
TargetTransformInfoImplCRTPBase &
operator=(TargetTransformInfoImplCRTPBase &&RHS) {
BaseT::operator=(std::move(static_cast<BaseT &>(RHS)));
return *this;
}
using BaseT::getCallCost;
unsigned getCallCost(const Function *F, int NumArgs) {
assert(F && "A concrete function must be provided to this routine.");
if (NumArgs < 0)
// Set the argument number to the number of explicit arguments in the
// function.
NumArgs = F->arg_size();
if (Intrinsic::ID IID = (Intrinsic::ID)F->getIntrinsicID()) {
FunctionType *FTy = F->getFunctionType();
SmallVector<Type *, 8> ParamTys(FTy->param_begin(), FTy->param_end());
return static_cast<T *>(this)
->getIntrinsicCost(IID, FTy->getReturnType(), ParamTys);
}
if (!static_cast<T *>(this)->isLoweredToCall(F))
return TTI::TCC_Basic; // Give a basic cost if it will be lowered
// directly.
return static_cast<T *>(this)->getCallCost(F->getFunctionType(), NumArgs);
}
unsigned getCallCost(const Function *F, ArrayRef<const Value *> Arguments) {
// Simply delegate to generic handling of the call.
// FIXME: We should use instsimplify or something else to catch calls which
// will constant fold with these arguments.
return static_cast<T *>(this)->getCallCost(F, Arguments.size());
}
using BaseT::getIntrinsicCost;
unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<const Value *> Arguments) {
// Delegate to the generic intrinsic handling code. This mostly provides an
// opportunity for targets to (for example) special case the cost of
// certain intrinsics based on constants used as arguments.
SmallVector<Type *, 8> ParamTys;
ParamTys.reserve(Arguments.size());
for (unsigned Idx = 0, Size = Arguments.size(); Idx != Size; ++Idx)
ParamTys.push_back(Arguments[Idx]->getType());
return static_cast<T *>(this)->getIntrinsicCost(IID, RetTy, ParamTys);
}
unsigned getUserCost(const User *U) {
if (isa<PHINode>(U))
return TTI::TCC_Free; // Model all PHI nodes as free.
if (const GEPOperator *GEP = dyn_cast<GEPOperator>(U)) {
SmallVector<const Value *, 4> Indices(GEP->idx_begin(), GEP->idx_end());
return static_cast<T *>(this)
->getGEPCost(GEP->getPointerOperand(), Indices);
}
if (ImmutableCallSite CS = U) {
const Function *F = CS.getCalledFunction();
if (!F) {
// Just use the called value type.
Type *FTy = CS.getCalledValue()->getType()->getPointerElementType();
return static_cast<T *>(this)
->getCallCost(cast<FunctionType>(FTy), CS.arg_size());
}
SmallVector<const Value *, 8> Arguments(CS.arg_begin(), CS.arg_end());
return static_cast<T *>(this)->getCallCost(F, Arguments);
}
if (const CastInst *CI = dyn_cast<CastInst>(U)) {
// Result of a cmp instruction is often extended (to be used by other
// cmp instructions, logical or return instructions). These are usually
// nop on most sane targets.
if (isa<CmpInst>(CI->getOperand(0)))
return TTI::TCC_Free;
}
// Otherwise delegate to the fully generic implementations.
return getOperationCost(
Operator::getOpcode(U), U->getType(),
U->getNumOperands() == 1 ? U->getOperand(0)->getType() : nullptr);
}
};
}
#endif

llvm/trunk/include/llvm/CodeGen/BasicTTIImpl.h

//===- BasicTTIImpl.h -------------------------------------------*- C++ -*-===//
//
// The LLVM Compiler Infrastructure
//
// This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details.
//
//===----------------------------------------------------------------------===//
/// \file
/// This file provides a helper that implements much of the TTI interface in
/// terms of the target-independent code generator and TargetLowering
/// interfaces.
///
//===----------------------------------------------------------------------===//
#ifndef LLVM_CODEGEN_BASICTTIIMPL_H
#define LLVM_CODEGEN_BASICTTIIMPL_H
#include "llvm/Analysis/LoopInfo.h"
#include "llvm/Analysis/TargetTransformInfoImpl.h"
#include "llvm/Support/CommandLine.h"
#include "llvm/Target/TargetLowering.h"
#include "llvm/Target/TargetSubtargetInfo.h"
namespace llvm {
extern cl::opt<unsigned> PartialUnrollingThreshold;
template <typename T>
class BasicTTIImplBase : public TargetTransformInfoImplCRTPBase<T> {
private:
typedef TargetTransformInfoImplCRTPBase<T> BaseT;
typedef TargetTransformInfo TTI;
/// Estimate the overhead of scalarizing an instruction. Insert and Extract
/// are set if the result needs to be inserted and/or extracted from vectors.
unsigned getScalarizationOverhead(Type *Ty, bool Insert, bool Extract) {
assert(Ty->isVectorTy() && "Can only scalarize vectors");
unsigned Cost = 0;
for (int i = 0, e = Ty->getVectorNumElements(); i < e; ++i) {
if (Insert)
Cost += static_cast<T *>(this)
->getVectorInstrCost(Instruction::InsertElement, Ty, i);
if (Extract)
Cost += static_cast<T *>(this)
->getVectorInstrCost(Instruction::ExtractElement, Ty, i);
}
return Cost;
}
/// Estimate the cost overhead of SK_Alternate shuffle.
unsigned getAltShuffleOverhead(Type *Ty) {
assert(Ty->isVectorTy() && "Can only shuffle vectors");
unsigned Cost = 0;
// Shuffle cost is equal to the cost of extracting element from its argument
// plus the cost of inserting them onto the result vector.
// e.g. <4 x float> has a mask of <0,5,2,7> i.e we need to extract from
// index 0 of first vector, index 1 of second vector,index 2 of first
// vector and finally index 3 of second vector and insert them at index
// <0,1,2,3> of result vector.
for (int i = 0, e = Ty->getVectorNumElements(); i < e; ++i) {
Cost += static_cast<T *>(this)
->getVectorInstrCost(Instruction::InsertElement, Ty, i);
Cost += static_cast<T *>(this)
->getVectorInstrCost(Instruction::ExtractElement, Ty, i);
}
return Cost;
}
const TargetLoweringBase *getTLI() const {
return TM->getSubtargetImpl()->getTargetLowering();
}
protected:
const TargetMachine *TM;
explicit BasicTTIImplBase(const TargetMachine *TM = nullptr)
: BaseT(TM ? TM->getDataLayout() : nullptr), TM(TM) {}
public:
// Provide value semantics. MSVC requires that we spell all of these out.
BasicTTIImplBase(const BasicTTIImplBase &Arg)
: BaseT(static_cast<const BaseT &>(Arg)), TM(Arg.TM) {}
BasicTTIImplBase(BasicTTIImplBase &&Arg)
: BaseT(std::move(static_cast<BaseT &>(Arg))), TM(std::move(Arg.TM)) {}
BasicTTIImplBase &operator=(const BasicTTIImplBase &RHS) {
BaseT::operator=(static_cast<const BaseT &>(RHS));
TM = RHS.TM;
return *this;
}
BasicTTIImplBase &operator=(BasicTTIImplBase &&RHS) {
BaseT::operator=(std::move(static_cast<BaseT &>(RHS)));
TM = std::move(RHS.TM);
return *this;
}
/// \name Scalar TTI Implementations
/// @{
bool hasBranchDivergence() { return false; }
bool isLegalAddImmediate(int64_t imm) {
return getTLI()->isLegalAddImmediate(imm);
}
bool isLegalICmpImmediate(int64_t imm) {
return getTLI()->isLegalICmpImmediate(imm);
}
bool isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV, int64_t BaseOffset,
bool HasBaseReg, int64_t Scale) {
TargetLoweringBase::AddrMode AM;
AM.BaseGV = BaseGV;
AM.BaseOffs = BaseOffset;
AM.HasBaseReg = HasBaseReg;
AM.Scale = Scale;
return getTLI()->isLegalAddressingMode(AM, Ty);
}
int getScalingFactorCost(Type *Ty, GlobalValue *BaseGV, int64_t BaseOffset,
bool HasBaseReg, int64_t Scale) {
TargetLoweringBase::AddrMode AM;
AM.BaseGV = BaseGV;
AM.BaseOffs = BaseOffset;
AM.HasBaseReg = HasBaseReg;
AM.Scale = Scale;
return getTLI()->getScalingFactorCost(AM, Ty);
}
bool isTruncateFree(Type *Ty1, Type *Ty2) {
return getTLI()->isTruncateFree(Ty1, Ty2);
}
bool isTypeLegal(Type *Ty) {
EVT VT = getTLI()->getValueType(Ty);
return getTLI()->isTypeLegal(VT);
}
unsigned getJumpBufAlignment() { return getTLI()->getJumpBufAlignment(); }
unsigned getJumpBufSize() { return getTLI()->getJumpBufSize(); }
bool shouldBuildLookupTables() {
const TargetLoweringBase *TLI = getTLI();
return TLI->isOperationLegalOrCustom(ISD::BR_JT, MVT::Other) ||
TLI->isOperationLegalOrCustom(ISD::BRIND, MVT::Other);
}
bool haveFastSqrt(Type *Ty) {
const TargetLoweringBase *TLI = getTLI();
EVT VT = TLI->getValueType(Ty);
return TLI->isTypeLegal(VT) &&
TLI->isOperationLegalOrCustom(ISD::FSQRT, VT);
}
void getUnrollingPreferences(const Function *F, Loop *L,
TTI::UnrollingPreferences &UP) {
// This unrolling functionality is target independent, but to provide some
// motivation for its intended use, for x86:
// According to the Intel 64 and IA-32 Architectures Optimization Reference
// Manual, Intel Core models and later have a loop stream detector (and
// associated uop queue) that can benefit from partial unrolling.
// The relevant requirements are:
// - The loop must have no more than 4 (8 for Nehalem and later) branches
// taken, and none of them may be calls.
// - The loop can have no more than 18 (28 for Nehalem and later) uops.
// According to the Software Optimization Guide for AMD Family 15h
// Processors, models 30h-4fh (Steamroller and later) have a loop predictor
// and loop buffer which can benefit from partial unrolling.
// The relevant requirements are:
// - The loop must have fewer than 16 branches
// - The loop must have less than 40 uops in all executed loop branches
// The number of taken branches in a loop is hard to estimate here, and
// benchmarking has revealed that it is better not to be conservative when
// estimating the branch count. As a result, we'll ignore the branch limits
// until someone finds a case where it matters in practice.
unsigned MaxOps;
const TargetSubtargetInfo *ST = TM->getSubtargetImpl(*F);
if (PartialUnrollingThreshold.getNumOccurrences() > 0)
MaxOps = PartialUnrollingThreshold;
else if (ST->getSchedModel().LoopMicroOpBufferSize > 0)
MaxOps = ST->getSchedModel().LoopMicroOpBufferSize;
else
return;
// Scan the loop: don't unroll loops with calls.
for (Loop::block_iterator I = L->block_begin(), E = L->block_end(); I != E;
++I) {
BasicBlock *BB = *I;
for (BasicBlock::iterator J = BB->begin(), JE = BB->end(); J != JE; ++J)
if (isa<CallInst>(J) || isa<InvokeInst>(J)) {
ImmutableCallSite CS(J);
if (const Function *F = CS.getCalledFunction()) {
if (!static_cast<T *>(this)->isLoweredToCall(F))
continue;
}
return;
}
}
// Enable runtime and partial unrolling up to the specified size.
UP.Partial = UP.Runtime = true;
UP.PartialThreshold = UP.PartialOptSizeThreshold = MaxOps;
}
/// @}
/// \name Vector TTI Implementations
/// @{
unsigned getNumberOfRegisters(bool Vector) { return 1; }
unsigned getRegisterBitWidth(bool Vector) { return 32; }
unsigned getMaxInterleaveFactor() { return 1; }
unsigned getArithmeticInstrCost(
unsigned Opcode, Type *Ty,
TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue,
TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue,
TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None,
TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None) {
// Check if any of the operands are vector operands.
const TargetLoweringBase *TLI = getTLI();
int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode");
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Ty);
bool IsFloat = Ty->getScalarType()->isFloatingPointTy();
// Assume that floating point arithmetic operations cost twice as much as
// integer operations.
unsigned OpCost = (IsFloat ? 2 : 1);
if (TLI->isOperationLegalOrPromote(ISD, LT.second)) {
// The operation is legal. Assume it costs 1.
// If the type is split to multiple registers, assume that there is some
// overhead to this.
// TODO: Once we have extract/insert subvector cost we need to use them.
if (LT.first > 1)
return LT.first * 2 * OpCost;
return LT.first * 1 * OpCost;
}
if (!TLI->isOperationExpand(ISD, LT.second)) {
// If the operation is custom lowered then assume
// thare the code is twice as expensive.
return LT.first * 2 * OpCost;
}
// Else, assume that we need to scalarize this op.
if (Ty->isVectorTy()) {
unsigned Num = Ty->getVectorNumElements();
unsigned Cost = static_cast<T *>(this)
->getArithmeticInstrCost(Opcode, Ty->getScalarType());
// return the cost of multiple scalar invocation plus the cost of
// inserting
// and extracting the values.
return getScalarizationOverhead(Ty, true, true) + Num * Cost;
}
// We don't know anything about this scalar instruction.
return OpCost;
}
unsigned getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index,
Type *SubTp) {
if (Kind == TTI::SK_Alternate) {
return getAltShuffleOverhead(Tp);
}
return 1;
}
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) {
const TargetLoweringBase *TLI = getTLI();
int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode");
std::pair<unsigned, MVT> SrcLT = TLI->getTypeLegalizationCost(Src);
std::pair<unsigned, MVT> DstLT = TLI->getTypeLegalizationCost(Dst);
// Check for NOOP conversions.
if (SrcLT.first == DstLT.first &&
SrcLT.second.getSizeInBits() == DstLT.second.getSizeInBits()) {
// Bitcast between types that are legalized to the same type are free.
if (Opcode == Instruction::BitCast || Opcode == Instruction::Trunc)
return 0;
}
if (Opcode == Instruction::Trunc &&
TLI->isTruncateFree(SrcLT.second, DstLT.second))
return 0;
if (Opcode == Instruction::ZExt &&
TLI->isZExtFree(SrcLT.second, DstLT.second))
return 0;
// If the cast is marked as legal (or promote) then assume low cost.
if (SrcLT.first == DstLT.first &&
TLI->isOperationLegalOrPromote(ISD, DstLT.second))
return 1;
// Handle scalar conversions.
if (!Src->isVectorTy() && !Dst->isVectorTy()) {
// Scalar bitcasts are usually free.
if (Opcode == Instruction::BitCast)
return 0;
// Just check the op cost. If the operation is legal then assume it costs
// 1.
if (!TLI->isOperationExpand(ISD, DstLT.second))
return 1;
// Assume that illegal scalar instruction are expensive.
return 4;
}
// Check vector-to-vector casts.
if (Dst->isVectorTy() && Src->isVectorTy()) {
// If the cast is between same-sized registers, then the check is simple.
if (SrcLT.first == DstLT.first &&
SrcLT.second.getSizeInBits() == DstLT.second.getSizeInBits()) {
// Assume that Zext is done using AND.
if (Opcode == Instruction::ZExt)
return 1;
// Assume that sext is done using SHL and SRA.
if (Opcode == Instruction::SExt)
return 2;
// Just check the op cost. If the operation is legal then assume it
// costs
// 1 and multiply by the type-legalization overhead.
if (!TLI->isOperationExpand(ISD, DstLT.second))
return SrcLT.first * 1;
}
// If we are converting vectors and the operation is illegal, or
// if the vectors are legalized to different types, estimate the
// scalarization costs.
unsigned Num = Dst->getVectorNumElements();
unsigned Cost = static_cast<T *>(this)->getCastInstrCost(
Opcode, Dst->getScalarType(), Src->getScalarType());
// Return the cost of multiple scalar invocation plus the cost of
// inserting and extracting the values.
return getScalarizationOverhead(Dst, true, true) + Num * Cost;
}
// We already handled vector-to-vector and scalar-to-scalar conversions.
// This
// is where we handle bitcast between vectors and scalars. We need to assume
// that the conversion is scalarized in one way or another.
if (Opcode == Instruction::BitCast)
// Illegal bitcasts are done by storing and loading from a stack slot.
return (Src->isVectorTy() ? getScalarizationOverhead(Src, false, true)
: 0) +
(Dst->isVectorTy() ? getScalarizationOverhead(Dst, true, false)
: 0);
llvm_unreachable("Unhandled cast");
}
unsigned getCFInstrCost(unsigned Opcode) {
// Branches are assumed to be predicted.
return 0;
}
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy) {
const TargetLoweringBase *TLI = getTLI();
int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode");
// Selects on vectors are actually vector selects.
if (ISD == ISD::SELECT) {
assert(CondTy && "CondTy must exist");
if (CondTy->isVectorTy())
ISD = ISD::VSELECT;
}
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(ValTy);
if (!(ValTy->isVectorTy() && !LT.second.isVector()) &&
!TLI->isOperationExpand(ISD, LT.second)) {
// The operation is legal. Assume it costs 1. Multiply
// by the type-legalization overhead.
return LT.first * 1;
}
// Otherwise, assume that the cast is scalarized.
if (ValTy->isVectorTy()) {
unsigned Num = ValTy->getVectorNumElements();
if (CondTy)
CondTy = CondTy->getScalarType();
unsigned Cost = static_cast<T *>(this)->getCmpSelInstrCost(
Opcode, ValTy->getScalarType(), CondTy);
// Return the cost of multiple scalar invocation plus the cost of
// inserting
// and extracting the values.
return getScalarizationOverhead(ValTy, true, false) + Num * Cost;
}
// Unknown scalar opcode.
return 1;
}
unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index) {
std::pair<unsigned, MVT> LT =
getTLI()->getTypeLegalizationCost(Val->getScalarType());
return LT.first;
}
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) {
assert(!Src->isVoidTy() && "Invalid type");
std::pair<unsigned, MVT> LT = getTLI()->getTypeLegalizationCost(Src);
// Assuming that all loads of legal types cost 1.
unsigned Cost = LT.first;
if (Src->isVectorTy() &&
Src->getPrimitiveSizeInBits() < LT.second.getSizeInBits()) {
// This is a vector load that legalizes to a larger type than the vector
// itself. Unless the corresponding extending load or truncating store is
// legal, then this will scalarize.
TargetLowering::LegalizeAction LA = TargetLowering::Expand;
EVT MemVT = getTLI()->getValueType(Src, true);
if (MemVT.isSimple() && MemVT != MVT::Other) {
if (Opcode == Instruction::Store)
LA = getTLI()->getTruncStoreAction(LT.second, MemVT.getSimpleVT());
else
LA = getTLI()->getLoadExtAction(ISD::EXTLOAD, LT.second, MemVT);
}
if (LA != TargetLowering::Legal && LA != TargetLowering::Custom) {
// This is a vector load/store for some illegal type that is scalarized.
// We must account for the cost of building or decomposing the vector.
Cost += getScalarizationOverhead(Src, Opcode != Instruction::Store,
Opcode == Instruction::Store);
}
}
return Cost;
}
unsigned getIntrinsicInstrCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<Type *> Tys) {
unsigned ISD = 0;
switch (IID) {
default: {
// Assume that we need to scalarize this intrinsic.
unsigned ScalarizationCost = 0;
unsigned ScalarCalls = 1;
if (RetTy->isVectorTy()) {
ScalarizationCost = getScalarizationOverhead(RetTy, true, false);
ScalarCalls = std::max(ScalarCalls, RetTy->getVectorNumElements());
}
for (unsigned i = 0, ie = Tys.size(); i != ie; ++i) {
if (Tys[i]->isVectorTy()) {
ScalarizationCost += getScalarizationOverhead(Tys[i], false, true);
ScalarCalls = std::max(ScalarCalls, RetTy->getVectorNumElements());
}
}
return ScalarCalls + ScalarizationCost;
}
// Look for intrinsics that can be lowered directly or turned into a scalar
// intrinsic call.
case Intrinsic::sqrt:
ISD = ISD::FSQRT;
break;
case Intrinsic::sin:
ISD = ISD::FSIN;
break;
case Intrinsic::cos:
ISD = ISD::FCOS;
break;
case Intrinsic::exp:
ISD = ISD::FEXP;
break;
case Intrinsic::exp2:
ISD = ISD::FEXP2;
break;
case Intrinsic::log:
ISD = ISD::FLOG;
break;
case Intrinsic::log10:
ISD = ISD::FLOG10;
break;
case Intrinsic::log2:
ISD = ISD::FLOG2;
break;
case Intrinsic::fabs:
ISD = ISD::FABS;
break;
case Intrinsic::minnum:
ISD = ISD::FMINNUM;
break;
case Intrinsic::maxnum:
ISD = ISD::FMAXNUM;
break;
case Intrinsic::copysign:
ISD = ISD::FCOPYSIGN;
break;
case Intrinsic::floor:
ISD = ISD::FFLOOR;
break;
case Intrinsic::ceil:
ISD = ISD::FCEIL;
break;
case Intrinsic::trunc:
ISD = ISD::FTRUNC;
break;
case Intrinsic::nearbyint:
ISD = ISD::FNEARBYINT;
break;
case Intrinsic::rint:
ISD = ISD::FRINT;
break;
case Intrinsic::round:
ISD = ISD::FROUND;
break;
case Intrinsic::pow:
ISD = ISD::FPOW;
break;
case Intrinsic::fma:
ISD = ISD::FMA;
break;
case Intrinsic::fmuladd:
ISD = ISD::FMA;
break;
// FIXME: We should return 0 whenever getIntrinsicCost == TCC_Free.
case Intrinsic::lifetime_start:
case Intrinsic::lifetime_end:
return 0;
case Intrinsic::masked_store:
return static_cast<T *>(this)
->getMaskedMemoryOpCost(Instruction::Store, Tys[0], 0, 0);
case Intrinsic::masked_load:
return static_cast<T *>(this)
->getMaskedMemoryOpCost(Instruction::Load, RetTy, 0, 0);
}
const TargetLoweringBase *TLI = getTLI();
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(RetTy);
if (TLI->isOperationLegalOrPromote(ISD, LT.second)) {
// The operation is legal. Assume it costs 1.
// If the type is split to multiple registers, assume that there is some
// overhead to this.
// TODO: Once we have extract/insert subvector cost we need to use them.
if (LT.first > 1)
return LT.first * 2;
return LT.first * 1;
}
if (!TLI->isOperationExpand(ISD, LT.second)) {
// If the operation is custom lowered then assume
// thare the code is twice as expensive.
return LT.first * 2;
}
// If we can't lower fmuladd into an FMA estimate the cost as a floating
// point mul followed by an add.
if (IID == Intrinsic::fmuladd)
return static_cast<T *>(this)
->getArithmeticInstrCost(BinaryOperator::FMul, RetTy) +
static_cast<T *>(this)
->getArithmeticInstrCost(BinaryOperator::FAdd, RetTy);
// Else, assume that we need to scalarize this intrinsic. For math builtins
// this will emit a costly libcall, adding call overhead and spills. Make it
// very expensive.
if (RetTy->isVectorTy()) {
unsigned Num = RetTy->getVectorNumElements();
unsigned Cost = static_cast<T *>(this)->getIntrinsicInstrCost(
IID, RetTy->getScalarType(), Tys);
return 10 * Cost * Num;
}
// This is going to be turned into a library call, make it expensive.
return 10;
}
unsigned getNumberOfParts(Type *Tp) {
std::pair<unsigned, MVT> LT = getTLI()->getTypeLegalizationCost(Tp);
return LT.first;
}
unsigned getAddressComputationCost(Type *Ty, bool IsComplex) { return 0; }
unsigned getReductionCost(unsigned Opcode, Type *Ty, bool IsPairwise) {
assert(Ty->isVectorTy() && "Expect a vector type");
unsigned NumVecElts = Ty->getVectorNumElements();
unsigned NumReduxLevels = Log2_32(NumVecElts);
unsigned ArithCost =
NumReduxLevels *
static_cast<T *>(this)->getArithmeticInstrCost(Opcode, Ty);
// Assume the pairwise shuffles add a cost.
unsigned ShuffleCost =
NumReduxLevels * (IsPairwise + 1) *
static_cast<T *>(this)
->getShuffleCost(TTI::SK_ExtractSubvector, Ty, NumVecElts / 2, Ty);
return ShuffleCost + ArithCost + getScalarizationOverhead(Ty, false, true);
}
/// @}
};
}
#endif

llvm/trunk/include/llvm/InitializePasses.h

Show First 20 Lines • Show All 71 Lines • ▼ Show 20 Lines
void initializeAlwaysInlinerPass(PassRegistry&); void initializeAlwaysInlinerPass(PassRegistry&);
void initializeArgPromotionPass(PassRegistry&); void initializeArgPromotionPass(PassRegistry&);
void initializeAtomicExpandPass(PassRegistry&); void initializeAtomicExpandPass(PassRegistry&);
void initializeSampleProfileLoaderPass(PassRegistry&); void initializeSampleProfileLoaderPass(PassRegistry&);
void initializeAlignmentFromAssumptionsPass(PassRegistry&); void initializeAlignmentFromAssumptionsPass(PassRegistry&);
void initializeBarrierNoopPass(PassRegistry&); void initializeBarrierNoopPass(PassRegistry&);
void initializeBasicAliasAnalysisPass(PassRegistry&); void initializeBasicAliasAnalysisPass(PassRegistry&);
void initializeCallGraphWrapperPassPass(PassRegistry &); void initializeCallGraphWrapperPassPass(PassRegistry &);
void initializeBasicTTIPass(PassRegistry&);
void initializeBlockExtractorPassPass(PassRegistry&); void initializeBlockExtractorPassPass(PassRegistry&);
void initializeBlockFrequencyInfoPass(PassRegistry&); void initializeBlockFrequencyInfoPass(PassRegistry&);
void initializeBoundsCheckingPass(PassRegistry&); void initializeBoundsCheckingPass(PassRegistry&);
void initializeBranchFolderPassPass(PassRegistry&); void initializeBranchFolderPassPass(PassRegistry&);
void initializeBranchProbabilityInfoPass(PassRegistry&); void initializeBranchProbabilityInfoPass(PassRegistry&);
void initializeBreakCriticalEdgesPass(PassRegistry&); void initializeBreakCriticalEdgesPass(PassRegistry&);
void initializeCallGraphPrinterPass(PassRegistry&); void initializeCallGraphPrinterPass(PassRegistry&);
void initializeCallGraphViewerPass(PassRegistry&); void initializeCallGraphViewerPass(PassRegistry&);
▲ Show 20 Lines • Show All 170 Lines • ▼ Show 20 Lines
void initializeStripDeadPrototypesPassPass(PassRegistry&); void initializeStripDeadPrototypesPassPass(PassRegistry&);
void initializeStripDebugDeclarePass(PassRegistry&); void initializeStripDebugDeclarePass(PassRegistry&);
void initializeStripNonDebugSymbolsPass(PassRegistry&); void initializeStripNonDebugSymbolsPass(PassRegistry&);
void initializeStripSymbolsPass(PassRegistry&); void initializeStripSymbolsPass(PassRegistry&);
void initializeTailCallElimPass(PassRegistry&); void initializeTailCallElimPass(PassRegistry&);
void initializeTailDuplicatePassPass(PassRegistry&); void initializeTailDuplicatePassPass(PassRegistry&);
void initializeTargetPassConfigPass(PassRegistry&); void initializeTargetPassConfigPass(PassRegistry&);
void initializeDataLayoutPassPass(PassRegistry &); void initializeDataLayoutPassPass(PassRegistry &);
void initializeTargetTransformInfoAnalysisGroup(PassRegistry&); void initializeTargetTransformInfoWrapperPassPass(PassRegistry &);
void initializeFunctionTargetTransformInfoPass(PassRegistry &); void initializeFunctionTargetTransformInfoPass(PassRegistry &);
void initializeNoTTIPass(PassRegistry&);
void initializeTargetLibraryInfoWrapperPassPass(PassRegistry &); void initializeTargetLibraryInfoWrapperPassPass(PassRegistry &);
void initializeAssumptionCacheTrackerPass(PassRegistry &); void initializeAssumptionCacheTrackerPass(PassRegistry &);
void initializeTwoAddressInstructionPassPass(PassRegistry&); void initializeTwoAddressInstructionPassPass(PassRegistry&);
void initializeTypeBasedAliasAnalysisPass(PassRegistry&); void initializeTypeBasedAliasAnalysisPass(PassRegistry&);
void initializeScopedNoAliasAAPass(PassRegistry&); void initializeScopedNoAliasAAPass(PassRegistry&);
void initializeUnifyFunctionExitNodesPass(PassRegistry&); void initializeUnifyFunctionExitNodesPass(PassRegistry&);
void initializeUnreachableBlockElimPass(PassRegistry&); void initializeUnreachableBlockElimPass(PassRegistry&);
void initializeUnreachableMachineBlockElimPass(PassRegistry&); void initializeUnreachableMachineBlockElimPass(PassRegistry&);
Show All 18 Lines

llvm/trunk/include/llvm/Target/TargetMachine.h

Show First 20 Lines • Show All 181 Lines • ▼ Show 20 Linespublic:
/// setDataSections - Set if the data are emit into separate sections. /// setDataSections - Set if the data are emit into separate sections.
void setDataSections(bool); void setDataSections(bool);
/// setFunctionSections - Set if the functions are emit into separate /// setFunctionSections - Set if the functions are emit into separate
/// sections. /// sections.
void setFunctionSections(bool); void setFunctionSections(bool);
/// \brief Register analysis passes for this target with a pass manager. /// \brief Register analysis passes for this target with a pass manager.
virtual void addAnalysisPasses(PassManagerBase &) {} virtual void addAnalysisPasses(PassManagerBase &);
/// CodeGenFileType - These enums are meant to be passed into /// CodeGenFileType - These enums are meant to be passed into
/// addPassesToEmitFile to indicate what type of file to emit, and returned by /// addPassesToEmitFile to indicate what type of file to emit, and returned by
/// it to indicate what type of file could actually be made. /// it to indicate what type of file could actually be made.
enum CodeGenFileType { enum CodeGenFileType {
CGFT_AssemblyFile, CGFT_AssemblyFile,
CGFT_ObjectFile, CGFT_ObjectFile,
CGFT_Null // Do not emit any output. CGFT_Null // Do not emit any output.
▲ Show 20 Lines • Show All 73 Lines • Show Last 20 Lines

llvm/trunk/lib/Analysis/Analysis.cpp

Show First 20 Lines • Show All 59 Lines • ▼ Show 20 Linesvoid llvm::initializeAnalysis(PassRegistry &Registry) {
initializePostDominatorTreePass(Registry); initializePostDominatorTreePass(Registry);
initializeRegionInfoPassPass(Registry); initializeRegionInfoPassPass(Registry);
initializeRegionViewerPass(Registry); initializeRegionViewerPass(Registry);
initializeRegionPrinterPass(Registry); initializeRegionPrinterPass(Registry);
initializeRegionOnlyViewerPass(Registry); initializeRegionOnlyViewerPass(Registry);
initializeRegionOnlyPrinterPass(Registry); initializeRegionOnlyPrinterPass(Registry);
initializeScalarEvolutionPass(Registry); initializeScalarEvolutionPass(Registry);
initializeScalarEvolutionAliasAnalysisPass(Registry); initializeScalarEvolutionAliasAnalysisPass(Registry);
initializeTargetTransformInfoAnalysisGroup(Registry); initializeTargetTransformInfoWrapperPassPass(Registry);
initializeTypeBasedAliasAnalysisPass(Registry); initializeTypeBasedAliasAnalysisPass(Registry);
initializeScopedNoAliasAAPass(Registry); initializeScopedNoAliasAAPass(Registry);
} }
void LLVMInitializeAnalysis(LLVMPassRegistryRef R) { void LLVMInitializeAnalysis(LLVMPassRegistryRef R) {
initializeAnalysis(*unwrap(R)); initializeAnalysis(*unwrap(R));
} }
▲ Show 20 Lines • Show All 41 Lines • Show Last 20 Lines

llvm/trunk/lib/Analysis/CostModel.cpp

Show First 20 Lines • Show All 77 Lines • ▼ Show 20 Lines
void void
CostModelAnalysis::getAnalysisUsage(AnalysisUsage &AU) const { CostModelAnalysis::getAnalysisUsage(AnalysisUsage &AU) const {
AU.setPreservesAll(); AU.setPreservesAll();
} }
bool bool
CostModelAnalysis::runOnFunction(Function &F) { CostModelAnalysis::runOnFunction(Function &F) {
this->F = &F; this->F = &F;
TTI = getAnalysisIfAvailable<TargetTransformInfo>(); auto *TTIWP = getAnalysisIfAvailable<TargetTransformInfoWrapperPass>();
TTI = TTIWP ? &TTIWP->getTTI() : nullptr;
return false; return false;
} }
static bool isReverseVectorMask(SmallVectorImpl<int> &Mask) { static bool isReverseVectorMask(SmallVectorImpl<int> &Mask) {
for (unsigned i = 0, MaskSize = Mask.size(); i < MaskSize; ++i) for (unsigned i = 0, MaskSize = Mask.size(); i < MaskSize; ++i)
if (Mask[i] > 0 && Mask[i] != (int)(MaskSize - 1 - i)) if (Mask[i] > 0 && Mask[i] != (int)(MaskSize - 1 - i))
return false; return false;
▲ Show 20 Lines • Show All 443 Lines • Show Last 20 Lines

llvm/trunk/lib/Analysis/FunctionTargetTransformInfo.cpp

Show All 15 Lines
#include "llvm/InitializePasses.h" #include "llvm/InitializePasses.h"
#include "llvm/Analysis/FunctionTargetTransformInfo.h" #include "llvm/Analysis/FunctionTargetTransformInfo.h"
using namespace llvm; using namespace llvm;
#define DEBUG_TYPE "function-tti" #define DEBUG_TYPE "function-tti"
static const char ftti_name[] = "Function TargetTransformInfo"; static const char ftti_name[] = "Function TargetTransformInfo";
INITIALIZE_PASS_BEGIN(FunctionTargetTransformInfo, "function_tti", ftti_name, false, true) INITIALIZE_PASS_BEGIN(FunctionTargetTransformInfo, "function_tti", ftti_name, false, true)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_END(FunctionTargetTransformInfo, "function_tti", ftti_name, false, true) INITIALIZE_PASS_END(FunctionTargetTransformInfo, "function_tti", ftti_name, false, true)
char FunctionTargetTransformInfo::ID = 0; char FunctionTargetTransformInfo::ID = 0;
namespace llvm { namespace llvm {
FunctionPass *createFunctionTargetTransformInfoPass() { FunctionPass *createFunctionTargetTransformInfoPass() {
return new FunctionTargetTransformInfo(); return new FunctionTargetTransformInfo();
} }
} }
FunctionTargetTransformInfo::FunctionTargetTransformInfo() FunctionTargetTransformInfo::FunctionTargetTransformInfo()
: FunctionPass(ID), Fn(nullptr), TTI(nullptr) { : FunctionPass(ID), Fn(nullptr), TTI(nullptr) {
initializeFunctionTargetTransformInfoPass(*PassRegistry::getPassRegistry()); initializeFunctionTargetTransformInfoPass(*PassRegistry::getPassRegistry());
} }
void FunctionTargetTransformInfo::getAnalysisUsage(AnalysisUsage &AU) const { void FunctionTargetTransformInfo::getAnalysisUsage(AnalysisUsage &AU) const {
AU.setPreservesAll(); AU.setPreservesAll();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
} }
void FunctionTargetTransformInfo::releaseMemory() {} void FunctionTargetTransformInfo::releaseMemory() {}
bool FunctionTargetTransformInfo::runOnFunction(Function &F) { bool FunctionTargetTransformInfo::runOnFunction(Function &F) {
Fn = &F; Fn = &F;
TTI = &getAnalysis<TargetTransformInfo>(); TTI = &getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
return false; return false;
} }

llvm/trunk/lib/Analysis/IPA/InlineCost.cpp

Show First 20 Lines • Show All 1,226 Lines • ▼ Show 20 Lines#define DEBUG_PRINT_STAT(x) dbgs() << " " #x ": " << x << "\n"
DEBUG_PRINT_STAT(Threshold); DEBUG_PRINT_STAT(Threshold);
DEBUG_PRINT_STAT(VectorBonus); DEBUG_PRINT_STAT(VectorBonus);
#undef DEBUG_PRINT_STAT #undef DEBUG_PRINT_STAT
} }
#endif #endif
INITIALIZE_PASS_BEGIN(InlineCostAnalysis, "inline-cost", "Inline Cost Analysis", INITIALIZE_PASS_BEGIN(InlineCostAnalysis, "inline-cost", "Inline Cost Analysis",
true, true) true, true)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker) INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker)
INITIALIZE_PASS_END(InlineCostAnalysis, "inline-cost", "Inline Cost Analysis", INITIALIZE_PASS_END(InlineCostAnalysis, "inline-cost", "Inline Cost Analysis",
true, true) true, true)
char InlineCostAnalysis::ID = 0; char InlineCostAnalysis::ID = 0;
InlineCostAnalysis::InlineCostAnalysis() : CallGraphSCCPass(ID) {} InlineCostAnalysis::InlineCostAnalysis() : CallGraphSCCPass(ID) {}
InlineCostAnalysis::~InlineCostAnalysis() {} InlineCostAnalysis::~InlineCostAnalysis() {}
void InlineCostAnalysis::getAnalysisUsage(AnalysisUsage &AU) const { void InlineCostAnalysis::getAnalysisUsage(AnalysisUsage &AU) const {
AU.setPreservesAll(); AU.setPreservesAll();
AU.addRequired<AssumptionCacheTracker>(); AU.addRequired<AssumptionCacheTracker>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
CallGraphSCCPass::getAnalysisUsage(AU); CallGraphSCCPass::getAnalysisUsage(AU);
} }
bool InlineCostAnalysis::runOnSCC(CallGraphSCC &SCC) { bool InlineCostAnalysis::runOnSCC(CallGraphSCC &SCC) {
TTI = &getAnalysis<TargetTransformInfo>(); TTI = &getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
ACT = &getAnalysis<AssumptionCacheTracker>(); ACT = &getAnalysis<AssumptionCacheTracker>();
return false; return false;
} }
InlineCost InlineCostAnalysis::getInlineCost(CallSite CS, int Threshold) { InlineCost InlineCostAnalysis::getInlineCost(CallSite CS, int Threshold) {
return getInlineCost(CS, CS.getCalledFunction(), Threshold); return getInlineCost(CS, CS.getCalledFunction(), Threshold);
} }
▲ Show 20 Lines • Show All 94 Lines • Show Last 20 Lines

llvm/trunk/lib/Analysis/TargetTransformInfo.cpp

//===- llvm/Analysis/TargetTransformInfo.cpp ------------------------------===// //===- llvm/Analysis/TargetTransformInfo.cpp ------------------------------===//
// //
// The LLVM Compiler Infrastructure // The LLVM Compiler Infrastructure
// //
// This file is distributed under the University of Illinois Open Source // This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details. // License. See LICENSE.TXT for details.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "llvm/Analysis/TargetTransformInfo.h" #include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/Analysis/TargetTransformInfoImpl.h"
#include "llvm/IR/CallSite.h" #include "llvm/IR/CallSite.h"
#include "llvm/IR/DataLayout.h" #include "llvm/IR/DataLayout.h"
#include "llvm/IR/Instruction.h" #include "llvm/IR/Instruction.h"
#include "llvm/IR/Instructions.h" #include "llvm/IR/Instructions.h"
#include "llvm/IR/IntrinsicInst.h" #include "llvm/IR/IntrinsicInst.h"
#include "llvm/IR/Operator.h" #include "llvm/IR/Operator.h"
#include "llvm/Support/ErrorHandling.h" #include "llvm/Support/ErrorHandling.h"
using namespace llvm; using namespace llvm;
#define DEBUG_TYPE "tti" #define DEBUG_TYPE "tti"
// Setup the analysis group to manage the TargetTransformInfo passes. TargetTransformInfo::~TargetTransformInfo() {}
INITIALIZE_ANALYSIS_GROUP(TargetTransformInfo, "Target Information", NoTTI)
char TargetTransformInfo::ID = 0;
TargetTransformInfo::~TargetTransformInfo() { TargetTransformInfo::TargetTransformInfo(TargetTransformInfo &&Arg)
} : TTIImpl(std::move(Arg.TTIImpl)) {}
void TargetTransformInfo::pushTTIStack(Pass *P) {
TopTTI = this;
PrevTTI = &P->getAnalysis<TargetTransformInfo>();
// Walk up the chain and update the top TTI pointer.
for (TargetTransformInfo *PTTI = PrevTTI; PTTI; PTTI = PTTI->PrevTTI)
PTTI->TopTTI = this;
}
void TargetTransformInfo::getAnalysisUsage(AnalysisUsage &AU) const { TargetTransformInfo &TargetTransformInfo::operator=(TargetTransformInfo &&RHS) {
AU.addRequired<TargetTransformInfo>(); TTIImpl = std::move(RHS.TTIImpl);
return *this;
} }
unsigned TargetTransformInfo::getOperationCost(unsigned Opcode, Type *Ty, unsigned TargetTransformInfo::getOperationCost(unsigned Opcode, Type *Ty,
Type *OpTy) const { Type *OpTy) const {
return PrevTTI->getOperationCost(Opcode, Ty, OpTy); return TTIImpl->getOperationCost(Opcode, Ty, OpTy);
}
unsigned TargetTransformInfo::getGEPCost(
const Value *Ptr, ArrayRef<const Value *> Operands) const {
return PrevTTI->getGEPCost(Ptr, Operands);
} }
unsigned TargetTransformInfo::getCallCost(FunctionType *FTy, unsigned TargetTransformInfo::getCallCost(FunctionType *FTy,
int NumArgs) const { int NumArgs) const {
return PrevTTI->getCallCost(FTy, NumArgs); return TTIImpl->getCallCost(FTy, NumArgs);
}
unsigned TargetTransformInfo::getCallCost(const Function *F,
int NumArgs) const {
return PrevTTI->getCallCost(F, NumArgs);
}
unsigned TargetTransformInfo::getCallCost(
const Function *F, ArrayRef<const Value *> Arguments) const {
return PrevTTI->getCallCost(F, Arguments);
} }
unsigned TargetTransformInfo::getIntrinsicCost( unsigned
Intrinsic::ID IID, Type *RetTy, ArrayRef<Type *> ParamTys) const { TargetTransformInfo::getCallCost(const Function *F,
return PrevTTI->getIntrinsicCost(IID, RetTy, ParamTys); ArrayRef<const Value *> Arguments) const {
return TTIImpl->getCallCost(F, Arguments);
} }
unsigned TargetTransformInfo::getIntrinsicCost( unsigned
Intrinsic::ID IID, Type *RetTy, ArrayRef<const Value *> Arguments) const { TargetTransformInfo::getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
return PrevTTI->getIntrinsicCost(IID, RetTy, Arguments); ArrayRef<const Value *> Arguments) const {
return TTIImpl->getIntrinsicCost(IID, RetTy, Arguments);
} }
unsigned TargetTransformInfo::getUserCost(const User *U) const { unsigned TargetTransformInfo::getUserCost(const User *U) const {
return PrevTTI->getUserCost(U); return TTIImpl->getUserCost(U);
} }
bool TargetTransformInfo::hasBranchDivergence() const { bool TargetTransformInfo::hasBranchDivergence() const {
return PrevTTI->hasBranchDivergence(); return TTIImpl->hasBranchDivergence();
} }
bool TargetTransformInfo::isLoweredToCall(const Function *F) const { bool TargetTransformInfo::isLoweredToCall(const Function *F) const {
return PrevTTI->isLoweredToCall(F); return TTIImpl->isLoweredToCall(F);
} }
void void TargetTransformInfo::getUnrollingPreferences(
TargetTransformInfo::getUnrollingPreferences(const Function *F, Loop *L, const Function *F, Loop *L, UnrollingPreferences &UP) const {
UnrollingPreferences &UP) const { return TTIImpl->getUnrollingPreferences(F, L, UP);
PrevTTI->getUnrollingPreferences(F, L, UP);
} }
bool TargetTransformInfo::isLegalAddImmediate(int64_t Imm) const { bool TargetTransformInfo::isLegalAddImmediate(int64_t Imm) const {
return PrevTTI->isLegalAddImmediate(Imm); return TTIImpl->isLegalAddImmediate(Imm);
} }
bool TargetTransformInfo::isLegalICmpImmediate(int64_t Imm) const { bool TargetTransformInfo::isLegalICmpImmediate(int64_t Imm) const {
return PrevTTI->isLegalICmpImmediate(Imm); return TTIImpl->isLegalICmpImmediate(Imm);
} }
bool TargetTransformInfo::isLegalMaskedLoad(Type *DataType, bool TargetTransformInfo::isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV,
int Consecutive) const { int64_t BaseOffset,
return false; bool HasBaseReg,
int64_t Scale) const {
return TTIImpl->isLegalAddressingMode(Ty, BaseGV, BaseOffset, HasBaseReg,
Scale);
} }
bool TargetTransformInfo::isLegalMaskedStore(Type *DataType, bool TargetTransformInfo::isLegalMaskedStore(Type *DataType,
int Consecutive) const { int Consecutive) const {
return false; return TTIImpl->isLegalMaskedStore(DataType, Consecutive);
} }
bool TargetTransformInfo::isLegalMaskedLoad(Type *DataType,
bool TargetTransformInfo::isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV, int Consecutive) const {
int64_t BaseOffset, return TTIImpl->isLegalMaskedLoad(DataType, Consecutive);
bool HasBaseReg,
int64_t Scale) const {
return PrevTTI->isLegalAddressingMode(Ty, BaseGV, BaseOffset, HasBaseReg,
Scale);
} }
int TargetTransformInfo::getScalingFactorCost(Type *Ty, GlobalValue *BaseGV, int TargetTransformInfo::getScalingFactorCost(Type *Ty, GlobalValue *BaseGV,
int64_t BaseOffset, int64_t BaseOffset,
bool HasBaseReg, bool HasBaseReg,
int64_t Scale) const { int64_t Scale) const {
return PrevTTI->getScalingFactorCost(Ty, BaseGV, BaseOffset, HasBaseReg, return TTIImpl->getScalingFactorCost(Ty, BaseGV, BaseOffset, HasBaseReg,
Scale); Scale);
} }
bool TargetTransformInfo::isTruncateFree(Type *Ty1, Type *Ty2) const { bool TargetTransformInfo::isTruncateFree(Type *Ty1, Type *Ty2) const {
return PrevTTI->isTruncateFree(Ty1, Ty2); return TTIImpl->isTruncateFree(Ty1, Ty2);
} }
bool TargetTransformInfo::isTypeLegal(Type *Ty) const { bool TargetTransformInfo::isTypeLegal(Type *Ty) const {
return PrevTTI->isTypeLegal(Ty); return TTIImpl->isTypeLegal(Ty);
} }
unsigned TargetTransformInfo::getJumpBufAlignment() const { unsigned TargetTransformInfo::getJumpBufAlignment() const {
return PrevTTI->getJumpBufAlignment(); return TTIImpl->getJumpBufAlignment();
} }
unsigned TargetTransformInfo::getJumpBufSize() const { unsigned TargetTransformInfo::getJumpBufSize() const {
return PrevTTI->getJumpBufSize(); return TTIImpl->getJumpBufSize();
} }
bool TargetTransformInfo::shouldBuildLookupTables() const { bool TargetTransformInfo::shouldBuildLookupTables() const {
return PrevTTI->shouldBuildLookupTables(); return TTIImpl->shouldBuildLookupTables();
} }
TargetTransformInfo::PopcntSupportKind TargetTransformInfo::PopcntSupportKind
TargetTransformInfo::getPopcntSupport(unsigned IntTyWidthInBit) const { TargetTransformInfo::getPopcntSupport(unsigned IntTyWidthInBit) const {
return PrevTTI->getPopcntSupport(IntTyWidthInBit); return TTIImpl->getPopcntSupport(IntTyWidthInBit);
} }
bool TargetTransformInfo::haveFastSqrt(Type *Ty) const { bool TargetTransformInfo::haveFastSqrt(Type *Ty) const {
return PrevTTI->haveFastSqrt(Ty); return TTIImpl->haveFastSqrt(Ty);
} }
unsigned TargetTransformInfo::getIntImmCost(const APInt &Imm, Type *Ty) const { unsigned TargetTransformInfo::getIntImmCost(const APInt &Imm, Type *Ty) const {
return PrevTTI->getIntImmCost(Imm, Ty); return TTIImpl->getIntImmCost(Imm, Ty);
} }
unsigned TargetTransformInfo::getIntImmCost(unsigned Opc, unsigned Idx, unsigned TargetTransformInfo::getIntImmCost(unsigned Opcode, unsigned Idx,
const APInt &Imm, Type *Ty) const { const APInt &Imm, Type *Ty) const {
return PrevTTI->getIntImmCost(Opc, Idx, Imm, Ty); return TTIImpl->getIntImmCost(Opcode, Idx, Imm, Ty);
} }
unsigned TargetTransformInfo::getIntImmCost(Intrinsic::ID IID, unsigned Idx, unsigned TargetTransformInfo::getIntImmCost(Intrinsic::ID IID, unsigned Idx,
const APInt &Imm, Type *Ty) const { const APInt &Imm, Type *Ty) const {
return PrevTTI->getIntImmCost(IID, Idx, Imm, Ty); return TTIImpl->getIntImmCost(IID, Idx, Imm, Ty);
} }
unsigned TargetTransformInfo::getNumberOfRegisters(bool Vector) const { unsigned TargetTransformInfo::getNumberOfRegisters(bool Vector) const {
return PrevTTI->getNumberOfRegisters(Vector); return TTIImpl->getNumberOfRegisters(Vector);
} }
unsigned TargetTransformInfo::getRegisterBitWidth(bool Vector) const { unsigned TargetTransformInfo::getRegisterBitWidth(bool Vector) const {
return PrevTTI->getRegisterBitWidth(Vector); return TTIImpl->getRegisterBitWidth(Vector);
} }
unsigned TargetTransformInfo::getMaxInterleaveFactor() const { unsigned TargetTransformInfo::getMaxInterleaveFactor() const {
return PrevTTI->getMaxInterleaveFactor(); return TTIImpl->getMaxInterleaveFactor();
} }
unsigned TargetTransformInfo::getArithmeticInstrCost( unsigned TargetTransformInfo::getArithmeticInstrCost(
unsigned Opcode, Type *Ty, OperandValueKind Op1Info, unsigned Opcode, Type *Ty, OperandValueKind Opd1Info,
OperandValueKind Op2Info, OperandValueProperties Opd1PropInfo, OperandValueKind Opd2Info, OperandValueProperties Opd1PropInfo,
OperandValueProperties Opd2PropInfo) const { OperandValueProperties Opd2PropInfo) const {
return PrevTTI->getArithmeticInstrCost(Opcode, Ty, Op1Info, Op2Info, return TTIImpl->getArithmeticInstrCost(Opcode, Ty, Opd1Info, Opd2Info,
Opd1PropInfo, Opd2PropInfo); Opd1PropInfo, Opd2PropInfo);
} }
unsigned TargetTransformInfo::getShuffleCost(ShuffleKind Kind, Type *Tp, unsigned TargetTransformInfo::getShuffleCost(ShuffleKind Kind, Type *Ty,
int Index, Type *SubTp) const { int Index, Type *SubTp) const {
return PrevTTI->getShuffleCost(Kind, Tp, Index, SubTp); return TTIImpl->getShuffleCost(Kind, Ty, Index, SubTp);
} }
unsigned TargetTransformInfo::getCastInstrCost(unsigned Opcode, Type *Dst, unsigned TargetTransformInfo::getCastInstrCost(unsigned Opcode, Type *Dst,
Type *Src) const { Type *Src) const {
return PrevTTI->getCastInstrCost(Opcode, Dst, Src); return TTIImpl->getCastInstrCost(Opcode, Dst, Src);
} }
unsigned TargetTransformInfo::getCFInstrCost(unsigned Opcode) const { unsigned TargetTransformInfo::getCFInstrCost(unsigned Opcode) const {
return PrevTTI->getCFInstrCost(Opcode); return TTIImpl->getCFInstrCost(Opcode);
} }
unsigned TargetTransformInfo::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, unsigned TargetTransformInfo::getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy) const { Type *CondTy) const {
return PrevTTI->getCmpSelInstrCost(Opcode, ValTy, CondTy); return TTIImpl->getCmpSelInstrCost(Opcode, ValTy, CondTy);
} }
unsigned TargetTransformInfo::getVectorInstrCost(unsigned Opcode, Type *Val, unsigned TargetTransformInfo::getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index) const { unsigned Index) const {
return PrevTTI->getVectorInstrCost(Opcode, Val, Index); return TTIImpl->getVectorInstrCost(Opcode, Val, Index);
} }
unsigned TargetTransformInfo::getMemoryOpCost(unsigned Opcode, Type *Src, unsigned TargetTransformInfo::getMemoryOpCost(unsigned Opcode, Type *Src,
unsigned Alignment, unsigned Alignment,
unsigned AddressSpace) const { unsigned AddressSpace) const {
return PrevTTI->getMemoryOpCost(Opcode, Src, Alignment, AddressSpace); return TTIImpl->getMemoryOpCost(Opcode, Src, Alignment, AddressSpace);
} }
unsigned unsigned
TargetTransformInfo::getMaskedMemoryOpCost(unsigned Opcode, Type *Src, TargetTransformInfo::getMaskedMemoryOpCost(unsigned Opcode, Type *Src,
unsigned Alignment, unsigned Alignment,
unsigned AddressSpace) const { unsigned AddressSpace) const {
return PrevTTI->getMaskedMemoryOpCost(Opcode, Src, Alignment, AddressSpace); return TTIImpl->getMaskedMemoryOpCost(Opcode, Src, Alignment, AddressSpace);
} }
unsigned unsigned
TargetTransformInfo::getIntrinsicInstrCost(Intrinsic::ID ID, TargetTransformInfo::getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
Type *RetTy,
ArrayRef<Type *> Tys) const { ArrayRef<Type *> Tys) const {
return PrevTTI->getIntrinsicInstrCost(ID, RetTy, Tys); return TTIImpl->getIntrinsicInstrCost(ID, RetTy, Tys);
} }
unsigned TargetTransformInfo::getNumberOfParts(Type *Tp) const { unsigned TargetTransformInfo::getNumberOfParts(Type *Tp) const {
return PrevTTI->getNumberOfParts(Tp); return TTIImpl->getNumberOfParts(Tp);
} }
unsigned TargetTransformInfo::getAddressComputationCost(Type *Tp, unsigned TargetTransformInfo::getAddressComputationCost(Type *Tp,
bool IsComplex) const { bool IsComplex) const {
return PrevTTI->getAddressComputationCost(Tp, IsComplex); return TTIImpl->getAddressComputationCost(Tp, IsComplex);
} }
unsigned TargetTransformInfo::getReductionCost(unsigned Opcode, Type *Ty, unsigned TargetTransformInfo::getReductionCost(unsigned Opcode, Type *Ty,
bool IsPairwise) const { bool IsPairwiseForm) const {
return PrevTTI->getReductionCost(Opcode, Ty, IsPairwise); return TTIImpl->getReductionCost(Opcode, Ty, IsPairwiseForm);
} }
unsigned TargetTransformInfo::getCostOfKeepingLiveOverCall(ArrayRef<Type*> Tys) unsigned
const { TargetTransformInfo::getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) const {
return PrevTTI->getCostOfKeepingLiveOverCall(Tys); return TTIImpl->getCostOfKeepingLiveOverCall(Tys);
}
Value *TargetTransformInfo::getOrCreateResultFromMemIntrinsic(
IntrinsicInst *Inst, Type *ExpectedType) const {
return PrevTTI->getOrCreateResultFromMemIntrinsic(Inst, ExpectedType);
} }
bool TargetTransformInfo::getTgtMemIntrinsic(IntrinsicInst *Inst, bool TargetTransformInfo::getTgtMemIntrinsic(IntrinsicInst *Inst,
MemIntrinsicInfo &Info) const { MemIntrinsicInfo &Info) const {
return PrevTTI->getTgtMemIntrinsic(Inst, Info); return TTIImpl->getTgtMemIntrinsic(Inst, Info);
}
namespace {
struct NoTTI final : ImmutablePass, TargetTransformInfo {
const DataLayout *DL;
NoTTI() : ImmutablePass(ID), DL(nullptr) {
initializeNoTTIPass(*PassRegistry::getPassRegistry());
}
void initializePass() override {
// Note that this subclass is special, and must *not* call initializeTTI as
// it does not chain.
TopTTI = this;
PrevTTI = nullptr;
DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>();
DL = DLP ? &DLP->getDataLayout() : nullptr;
}
void getAnalysisUsage(AnalysisUsage &AU) const override {
// Note that this subclass is special, and must *not* call
// TTI::getAnalysisUsage as it breaks the recursion.
}
/// Pass identification.
static char ID;
/// Provide necessary pointer adjustments for the two base classes.
void *getAdjustedAnalysisPointer(const void *ID) override {
if (ID == &TargetTransformInfo::ID)
return (TargetTransformInfo*)this;
return this;
}
unsigned getOperationCost(unsigned Opcode, Type *Ty,
Type *OpTy) const override {
switch (Opcode) {
default:
// By default, just classify everything as 'basic'.
return TCC_Basic;
case Instruction::GetElementPtr:
llvm_unreachable("Use getGEPCost for GEP operations!");
case Instruction::BitCast:
assert(OpTy && "Cast instructions must provide the operand type");
if (Ty == OpTy || (Ty->isPointerTy() && OpTy->isPointerTy()))
// Identity and pointer-to-pointer casts are free.
return TCC_Free;
// Otherwise, the default basic cost is used.
return TCC_Basic;
case Instruction::IntToPtr: {
if (!DL)
return TCC_Basic;
// An inttoptr cast is free so long as the input is a legal integer type
// which doesn't contain values outside the range of a pointer.
unsigned OpSize = OpTy->getScalarSizeInBits();
if (DL->isLegalInteger(OpSize) &&
OpSize <= DL->getPointerTypeSizeInBits(Ty))
return TCC_Free;
// Otherwise it's not a no-op.
return TCC_Basic;
}
case Instruction::PtrToInt: {
if (!DL)
return TCC_Basic;
// A ptrtoint cast is free so long as the result is large enough to store
// the pointer, and a legal integer type.
unsigned DestSize = Ty->getScalarSizeInBits();
if (DL->isLegalInteger(DestSize) &&
DestSize >= DL->getPointerTypeSizeInBits(OpTy))
return TCC_Free;
// Otherwise it's not a no-op.
return TCC_Basic;
}
case Instruction::Trunc:
// trunc to a native type is free (assuming the target has compare and
// shift-right of the same width).
if (DL && DL->isLegalInteger(DL->getTypeSizeInBits(Ty)))
return TCC_Free;
return TCC_Basic;
}
}
unsigned getGEPCost(const Value *Ptr,
ArrayRef<const Value *> Operands) const override {
// In the basic model, we just assume that all-constant GEPs will be folded
// into their uses via addressing modes.
for (unsigned Idx = 0, Size = Operands.size(); Idx != Size; ++Idx)
if (!isa<Constant>(Operands[Idx]))
return TCC_Basic;
return TCC_Free;
}
unsigned getCallCost(FunctionType *FTy, int NumArgs = -1) const override
{
assert(FTy && "FunctionType must be provided to this routine.");
// The target-independent implementation just measures the size of the
// function by approximating that each argument will take on average one
// instruction to prepare.
if (NumArgs < 0)
// Set the argument number to the number of explicit arguments in the
// function.
NumArgs = FTy->getNumParams();
return TCC_Basic * (NumArgs + 1);
}
unsigned getCallCost(const Function *F, int NumArgs = -1) const override
{
assert(F && "A concrete function must be provided to this routine.");
if (NumArgs < 0)
// Set the argument number to the number of explicit arguments in the
// function.
NumArgs = F->arg_size();
if (Intrinsic::ID IID = (Intrinsic::ID)F->getIntrinsicID()) {
FunctionType *FTy = F->getFunctionType();
SmallVector<Type *, 8> ParamTys(FTy->param_begin(), FTy->param_end());
return TopTTI->getIntrinsicCost(IID, FTy->getReturnType(), ParamTys);
}
if (!TopTTI->isLoweredToCall(F))
return TCC_Basic; // Give a basic cost if it will be lowered directly.
return TopTTI->getCallCost(F->getFunctionType(), NumArgs);
}
unsigned getCallCost(const Function *F,
ArrayRef<const Value *> Arguments) const override {
// Simply delegate to generic handling of the call.
// FIXME: We should use instsimplify or something else to catch calls which
// will constant fold with these arguments.
return TopTTI->getCallCost(F, Arguments.size());
}
unsigned getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<Type *> ParamTys) const override {
switch (IID) {
default:
// Intrinsics rarely (if ever) have normal argument setup constraints.
// Model them as having a basic instruction cost.
// FIXME: This is wrong for libc intrinsics.
return TCC_Basic;
case Intrinsic::annotation:
case Intrinsic::assume:
case Intrinsic::dbg_declare:
case Intrinsic::dbg_value:
case Intrinsic::invariant_start:
case Intrinsic::invariant_end:
case Intrinsic::lifetime_start:
case Intrinsic::lifetime_end:
case Intrinsic::objectsize:
case Intrinsic::ptr_annotation:
case Intrinsic::var_annotation:
case Intrinsic::experimental_gc_result_int:
case Intrinsic::experimental_gc_result_float:
case Intrinsic::experimental_gc_result_ptr:
case Intrinsic::experimental_gc_result:
case Intrinsic::experimental_gc_relocate:
// These intrinsics don't actually represent code after lowering.
return TCC_Free;
}
}
unsigned
getIntrinsicCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<const Value *> Arguments) const override {
// Delegate to the generic intrinsic handling code. This mostly provides an
// opportunity for targets to (for example) special case the cost of
// certain intrinsics based on constants used as arguments.
SmallVector<Type *, 8> ParamTys;
ParamTys.reserve(Arguments.size());
for (unsigned Idx = 0, Size = Arguments.size(); Idx != Size; ++Idx)
ParamTys.push_back(Arguments[Idx]->getType());
return TopTTI->getIntrinsicCost(IID, RetTy, ParamTys);
} }
unsigned getUserCost(const User *U) const override { Value *TargetTransformInfo::getOrCreateResultFromMemIntrinsic(
if (isa<PHINode>(U)) IntrinsicInst *Inst, Type *ExpectedType) const {
return TCC_Free; // Model all PHI nodes as free. return TTIImpl->getOrCreateResultFromMemIntrinsic(Inst, ExpectedType);
if (const GEPOperator *GEP = dyn_cast<GEPOperator>(U)) {
SmallVector<const Value *, 4> Indices(GEP->idx_begin(), GEP->idx_end());
return TopTTI->getGEPCost(GEP->getPointerOperand(), Indices);
}
if (ImmutableCallSite CS = U) {
const Function *F = CS.getCalledFunction();
if (!F) {
// Just use the called value type.
Type *FTy = CS.getCalledValue()->getType()->getPointerElementType();
return TopTTI->getCallCost(cast<FunctionType>(FTy), CS.arg_size());
}
SmallVector<const Value *, 8> Arguments(CS.arg_begin(), CS.arg_end());
return TopTTI->getCallCost(F, Arguments);
}
if (const CastInst *CI = dyn_cast<CastInst>(U)) {
// Result of a cmp instruction is often extended (to be used by other
// cmp instructions, logical or return instructions). These are usually
// nop on most sane targets.
if (isa<CmpInst>(CI->getOperand(0)))
return TCC_Free;
}
// Otherwise delegate to the fully generic implementations.
return getOperationCost(Operator::getOpcode(U), U->getType(),
U->getNumOperands() == 1 ?
U->getOperand(0)->getType() : nullptr);
}
bool hasBranchDivergence() const override { return false; }
bool isLoweredToCall(const Function *F) const override {
// FIXME: These should almost certainly not be handled here, and instead
// handled with the help of TLI or the target itself. This was largely
// ported from existing analysis heuristics here so that such refactorings
// can take place in the future.
if (F->isIntrinsic())
return false;
if (F->hasLocalLinkage() || !F->hasName())
return true;
StringRef Name = F->getName();
// These will all likely lower to a single selection DAG node.
if (Name == "copysign" || Name == "copysignf" || Name == "copysignl" ||
Name == "fabs" || Name == "fabsf" || Name == "fabsl" || Name == "sin" ||
Name == "fmin" || Name == "fminf" || Name == "fminl" ||
Name == "fmax" || Name == "fmaxf" || Name == "fmaxl" ||
Name == "sinf" || Name == "sinl" || Name == "cos" || Name == "cosf" ||
Name == "cosl" || Name == "sqrt" || Name == "sqrtf" || Name == "sqrtl")
return false;
// These are all likely to be optimized into something smaller.
if (Name == "pow" || Name == "powf" || Name == "powl" || Name == "exp2" ||
Name == "exp2l" || Name == "exp2f" || Name == "floor" || Name ==
"floorf" || Name == "ceil" || Name == "round" || Name == "ffs" ||
Name == "ffsl" || Name == "abs" || Name == "labs" || Name == "llabs")
return false;
return true;
}
void getUnrollingPreferences(const Function *, Loop *,
UnrollingPreferences &) const override {}
bool isLegalAddImmediate(int64_t Imm) const override {
return false;
}
bool isLegalICmpImmediate(int64_t Imm) const override {
return false;
}
bool isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV, int64_t BaseOffset,
bool HasBaseReg, int64_t Scale) const override
{
// Guess that reg+reg addressing is allowed. This heuristic is taken from
// the implementation of LSR.
return !BaseGV && BaseOffset == 0 && Scale <= 1;
}
int getScalingFactorCost(Type *Ty, GlobalValue *BaseGV, int64_t BaseOffset,
bool HasBaseReg, int64_t Scale) const override {
// Guess that all legal addressing mode are free.
if(isLegalAddressingMode(Ty, BaseGV, BaseOffset, HasBaseReg, Scale))
return 0;
return -1;
}
bool isTruncateFree(Type *Ty1, Type *Ty2) const override {
return false;
}
bool isTypeLegal(Type *Ty) const override {
return false;
}
unsigned getJumpBufAlignment() const override {
return 0;
}
unsigned getJumpBufSize() const override {
return 0;
}
bool shouldBuildLookupTables() const override {
return true;
}
PopcntSupportKind
getPopcntSupport(unsigned IntTyWidthInBit) const override {
return PSK_Software;
}
bool haveFastSqrt(Type *Ty) const override {
return false;
}
unsigned getIntImmCost(const APInt &Imm, Type *Ty) const override {
return TCC_Basic;
}
unsigned getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm,
Type *Ty) const override {
return TCC_Free;
}
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty) const override {
return TCC_Free;
}
unsigned getNumberOfRegisters(bool Vector) const override {
return 8;
}
unsigned getRegisterBitWidth(bool Vector) const override {
return 32;
}
unsigned getMaxInterleaveFactor() const override {
return 1;
}
unsigned getArithmeticInstrCost(unsigned Opcode, Type *Ty, OperandValueKind,
OperandValueKind, OperandValueProperties,
OperandValueProperties) const override {
return 1;
}
unsigned getShuffleCost(ShuffleKind Kind, Type *Ty,
int Index = 0, Type *SubTp = nullptr) const override {
return 1;
}
unsigned getCastInstrCost(unsigned Opcode, Type *Dst,
Type *Src) const override {
return 1;
}
unsigned getCFInstrCost(unsigned Opcode) const override {
return 1;
}
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy = nullptr) const override {
return 1;
}
unsigned getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index = -1) const override {
return 1;
}
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) const override {
return 1;
}
unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) const override {
return 1;
}
unsigned getIntrinsicInstrCost(Intrinsic::ID ID, Type *RetTy,
ArrayRef<Type*> Tys) const override {
return 1;
} }
unsigned getNumberOfParts(Type *Tp) const override { TargetTransformInfo::Concept::~Concept() {}
return 0;
}
unsigned getAddressComputationCost(Type *Tp, bool) const override { namespace {
return 0; /// \brief No-op implementation of the TTI interface using the utility base
/// classes.
///
/// This is used when no target specific information is available.
struct NoTTIImpl : TargetTransformInfoImplCRTPBase<NoTTIImpl> {
explicit NoTTIImpl(const DataLayout *DL)
: TargetTransformInfoImplCRTPBase<NoTTIImpl>(DL) {}
};
} }
unsigned getReductionCost(unsigned, Type *, bool) const override { // Register the basic pass.
return 1; INITIALIZE_PASS(TargetTransformInfoWrapperPass, "tti",
} "Target Transform Information", false, true)
char TargetTransformInfoWrapperPass::ID = 0;
unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type*> Tys) const override { void TargetTransformInfoWrapperPass::anchor() {}
return 0;
}
bool getTgtMemIntrinsic(IntrinsicInst *Inst, TargetTransformInfoWrapperPass::TargetTransformInfoWrapperPass()
MemIntrinsicInfo &Info) const override { : ImmutablePass(ID), TTI(NoTTIImpl(/*DataLayout*/ nullptr)) {
return false; initializeTargetTransformInfoWrapperPassPass(
*PassRegistry::getPassRegistry());
} }
Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst, TargetTransformInfoWrapperPass::TargetTransformInfoWrapperPass(
Type *ExpectedType) const override { TargetTransformInfo TTI)
return nullptr; : ImmutablePass(ID), TTI(std::move(TTI)) {
initializeTargetTransformInfoWrapperPassPass(
*PassRegistry::getPassRegistry());
} }
};
} // end anonymous namespace
INITIALIZE_AG_PASS(NoTTI, TargetTransformInfo, "notti",
"No target information", true, true, true)
char NoTTI::ID = 0;
ImmutablePass *llvm::createNoTargetTransformInfoPass() { ImmutablePass *llvm::createNoTargetTransformInfoPass(const DataLayout *DL) {
return new NoTTI(); return new TargetTransformInfoWrapperPass(NoTTIImpl(DL));
} }

llvm/trunk/lib/CodeGen/BasicTargetTransformInfo.cpp

Show All 9 Lines
/// This file provides the implementation of a basic TargetTransformInfo pass /// This file provides the implementation of a basic TargetTransformInfo pass
/// predicated on the target abstractions present in the target independent /// predicated on the target abstractions present in the target independent
/// code generator. It uses these (primarily TargetLowering) to model as much /// code generator. It uses these (primarily TargetLowering) to model as much
/// of the TTI query interface as possible. It is included by most targets so /// of the TTI query interface as possible. It is included by most targets so
/// that they can specialize only a small subset of the query space. /// that they can specialize only a small subset of the query space.
/// ///
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "llvm/CodeGen/BasicTTIImpl.h"
#include "llvm/CodeGen/Passes.h" #include "llvm/CodeGen/Passes.h"
#include "llvm/Analysis/LoopInfo.h" #include "llvm/Analysis/LoopInfo.h"
#include "llvm/Analysis/TargetTransformInfo.h" #include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/Analysis/TargetTransformInfoImpl.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
#include "llvm/Target/TargetLowering.h"
#include "llvm/Target/TargetSubtargetInfo.h"
#include <utility> #include <utility>
using namespace llvm; using namespace llvm;
static cl::opt<unsigned> cl::opt<unsigned>
PartialUnrollingThreshold("partial-unrolling-threshold", cl::init(0), llvm::PartialUnrollingThreshold("partial-unrolling-threshold", cl::init(0),
cl::desc("Threshold for partial unrolling"), cl::Hidden); cl::desc("Threshold for partial unrolling"),
cl::Hidden);
#define DEBUG_TYPE "basictti" #define DEBUG_TYPE "basictti"
namespace { namespace {
class BasicTTIImpl : public BasicTTIImplBase<BasicTTIImpl> {
class BasicTTI final : public ImmutablePass, public TargetTransformInfo { typedef BasicTTIImplBase<BasicTTIImpl> BaseT;
const TargetMachine *TM;
/// Estimate the overhead of scalarizing an instruction. Insert and Extract
/// are set if the result needs to be inserted and/or extracted from vectors.
unsigned getScalarizationOverhead(Type *Ty, bool Insert, bool Extract) const;
/// Estimate the cost overhead of SK_Alternate shuffle.
unsigned getAltShuffleOverhead(Type *Ty) const;
const TargetLoweringBase *getTLI() const {
return TM->getSubtargetImpl()->getTargetLowering();
}
public: public:
BasicTTI() : ImmutablePass(ID), TM(nullptr) { explicit BasicTTIImpl(const TargetMachine *TM = nullptr) : BaseT(TM) {}
llvm_unreachable("This pass cannot be directly constructed");
}
BasicTTI(const TargetMachine *TM) : ImmutablePass(ID), TM(TM) {
initializeBasicTTIPass(*PassRegistry::getPassRegistry());
}
void initializePass() override {
pushTTIStack(this);
}
void getAnalysisUsage(AnalysisUsage &AU) const override { // Provide value semantics. MSVC requires that we spell all of these out.
TargetTransformInfo::getAnalysisUsage(AU); BasicTTIImpl(const BasicTTIImpl &Arg)
: BaseT(static_cast<const BaseT &>(Arg)) {}
BasicTTIImpl(BasicTTIImpl &&Arg)
: BaseT(std::move(static_cast<BaseT &>(Arg))) {}
BasicTTIImpl &operator=(const BasicTTIImpl &RHS) {
BaseT::operator=(static_cast<const BaseT &>(RHS));
return *this;
}
BasicTTIImpl &operator=(BasicTTIImpl &&RHS) {
BaseT::operator=(std::move(static_cast<BaseT &>(RHS)));
return *this;
} }
/// Pass identification.
static char ID;
/// Provide necessary pointer adjustments for the two base classes.
void *getAdjustedAnalysisPointer(const void *ID) override {
if (ID == &TargetTransformInfo::ID)
return (TargetTransformInfo*)this;
return this;
}
bool hasBranchDivergence() const override;
/// \name Scalar TTI Implementations
/// @{
bool isLegalAddImmediate(int64_t imm) const override;
bool isLegalICmpImmediate(int64_t imm) const override;
bool isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV,
int64_t BaseOffset, bool HasBaseReg,
int64_t Scale) const override;
int getScalingFactorCost(Type *Ty, GlobalValue *BaseGV,
int64_t BaseOffset, bool HasBaseReg,
int64_t Scale) const override;
bool isTruncateFree(Type *Ty1, Type *Ty2) const override;
bool isTypeLegal(Type *Ty) const override;
unsigned getJumpBufAlignment() const override;
unsigned getJumpBufSize() const override;
bool shouldBuildLookupTables() const override;
bool haveFastSqrt(Type *Ty) const override;
void getUnrollingPreferences(const Function *F, Loop *L,
UnrollingPreferences &UP) const override;
/// @}
/// \name Vector TTI Implementations
/// @{
unsigned getNumberOfRegisters(bool Vector) const override;
unsigned getMaxInterleaveFactor() const override;
unsigned getRegisterBitWidth(bool Vector) const override;
unsigned getArithmeticInstrCost(unsigned Opcode, Type *Ty, OperandValueKind,
OperandValueKind, OperandValueProperties,
OperandValueProperties) const override;
unsigned getShuffleCost(ShuffleKind Kind, Type *Tp,
int Index, Type *SubTp) const override;
unsigned getCastInstrCost(unsigned Opcode, Type *Dst,
Type *Src) const override;
unsigned getCFInstrCost(unsigned Opcode) const override;
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy) const override;
unsigned getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index) const override;
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) const override;
unsigned getIntrinsicInstrCost(Intrinsic::ID, Type *RetTy,
ArrayRef<Type*> Tys) const override;
unsigned getNumberOfParts(Type *Tp) const override;
unsigned getAddressComputationCost( Type *Ty, bool IsComplex) const override;
unsigned getReductionCost(unsigned Opcode, Type *Ty,
bool IsPairwise) const override;
/// @}
}; };
} }
INITIALIZE_AG_PASS(BasicTTI, TargetTransformInfo, "basictti",
"Target independent code generator's TTI", true, true, false)
char BasicTTI::ID = 0;
ImmutablePass * ImmutablePass *
llvm::createBasicTargetTransformInfoPass(const TargetMachine *TM) { llvm::createBasicTargetTransformInfoPass(const TargetMachine *TM) {
return new BasicTTI(TM); return new TargetTransformInfoWrapperPass(BasicTTIImpl(TM));
}
bool BasicTTI::hasBranchDivergence() const { return false; }
bool BasicTTI::isLegalAddImmediate(int64_t imm) const {
return getTLI()->isLegalAddImmediate(imm);
}
bool BasicTTI::isLegalICmpImmediate(int64_t imm) const {
return getTLI()->isLegalICmpImmediate(imm);
}
bool BasicTTI::isLegalAddressingMode(Type *Ty, GlobalValue *BaseGV,
int64_t BaseOffset, bool HasBaseReg,
int64_t Scale) const {
TargetLoweringBase::AddrMode AM;
AM.BaseGV = BaseGV;
AM.BaseOffs = BaseOffset;
AM.HasBaseReg = HasBaseReg;
AM.Scale = Scale;
return getTLI()->isLegalAddressingMode(AM, Ty);
}
int BasicTTI::getScalingFactorCost(Type *Ty, GlobalValue *BaseGV,
int64_t BaseOffset, bool HasBaseReg,
int64_t Scale) const {
TargetLoweringBase::AddrMode AM;
AM.BaseGV = BaseGV;
AM.BaseOffs = BaseOffset;
AM.HasBaseReg = HasBaseReg;
AM.Scale = Scale;
return getTLI()->getScalingFactorCost(AM, Ty);
}
bool BasicTTI::isTruncateFree(Type *Ty1, Type *Ty2) const {
return getTLI()->isTruncateFree(Ty1, Ty2);
}
bool BasicTTI::isTypeLegal(Type *Ty) const {
EVT T = getTLI()->getValueType(Ty);
return getTLI()->isTypeLegal(T);
}
unsigned BasicTTI::getJumpBufAlignment() const {
return getTLI()->getJumpBufAlignment();
}
unsigned BasicTTI::getJumpBufSize() const {
return getTLI()->getJumpBufSize();
} }
bool BasicTTI::shouldBuildLookupTables() const {
const TargetLoweringBase *TLI = getTLI();
return TLI->isOperationLegalOrCustom(ISD::BR_JT, MVT::Other) ||
TLI->isOperationLegalOrCustom(ISD::BRIND, MVT::Other);
}
bool BasicTTI::haveFastSqrt(Type *Ty) const {
const TargetLoweringBase *TLI = getTLI();
EVT VT = TLI->getValueType(Ty);
return TLI->isTypeLegal(VT) && TLI->isOperationLegalOrCustom(ISD::FSQRT, VT);
}
void BasicTTI::getUnrollingPreferences(const Function *F, Loop *L,
UnrollingPreferences &UP) const {
// This unrolling functionality is target independent, but to provide some
// motivation for its intended use, for x86:
// According to the Intel 64 and IA-32 Architectures Optimization Reference
// Manual, Intel Core models and later have a loop stream detector
// (and associated uop queue) that can benefit from partial unrolling.
// The relevant requirements are:
// - The loop must have no more than 4 (8 for Nehalem and later) branches
// taken, and none of them may be calls.
// - The loop can have no more than 18 (28 for Nehalem and later) uops.
// According to the Software Optimization Guide for AMD Family 15h Processors,
// models 30h-4fh (Steamroller and later) have a loop predictor and loop
// buffer which can benefit from partial unrolling.
// The relevant requirements are:
// - The loop must have fewer than 16 branches
// - The loop must have less than 40 uops in all executed loop branches
// The number of taken branches in a loop is hard to estimate here, and
// benchmarking has revealed that it is better not to be conservative when
// estimating the branch count. As a result, we'll ignore the branch limits
// until someone finds a case where it matters in practice.
unsigned MaxOps;
const TargetSubtargetInfo *ST = TM->getSubtargetImpl(*F);
if (PartialUnrollingThreshold.getNumOccurrences() > 0)
MaxOps = PartialUnrollingThreshold;
else if (ST->getSchedModel().LoopMicroOpBufferSize > 0)
MaxOps = ST->getSchedModel().LoopMicroOpBufferSize;
else
return;
// Scan the loop: don't unroll loops with calls.
for (Loop::block_iterator I = L->block_begin(), E = L->block_end();
I != E; ++I) {
BasicBlock *BB = *I;
for (BasicBlock::iterator J = BB->begin(), JE = BB->end(); J != JE; ++J)
if (isa<CallInst>(J) || isa<InvokeInst>(J)) {
ImmutableCallSite CS(J);
if (const Function *F = CS.getCalledFunction()) {
if (!TopTTI->isLoweredToCall(F))
continue;
}
return;
}
}
// Enable runtime and partial unrolling up to the specified size.
UP.Partial = UP.Runtime = true;
UP.PartialThreshold = UP.PartialOptSizeThreshold = MaxOps;
}
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// Calls used by the vectorizers. // Calls used by the vectorizers.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
unsigned BasicTTI::getScalarizationOverhead(Type *Ty, bool Insert,
bool Extract) const {
assert (Ty->isVectorTy() && "Can only scalarize vectors");
unsigned Cost = 0;
for (int i = 0, e = Ty->getVectorNumElements(); i < e; ++i) {
if (Insert)
Cost += TopTTI->getVectorInstrCost(Instruction::InsertElement, Ty, i);
if (Extract)
Cost += TopTTI->getVectorInstrCost(Instruction::ExtractElement, Ty, i);
}
return Cost;
}
unsigned BasicTTI::getNumberOfRegisters(bool Vector) const {
return 1;
}
unsigned BasicTTI::getRegisterBitWidth(bool Vector) const {
return 32;
}
unsigned BasicTTI::getMaxInterleaveFactor() const {
return 1;
}
unsigned BasicTTI::getArithmeticInstrCost(unsigned Opcode, Type *Ty,
OperandValueKind, OperandValueKind,
OperandValueProperties,
OperandValueProperties) const {
// Check if any of the operands are vector operands.
const TargetLoweringBase *TLI = getTLI();
int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode");
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Ty);
bool IsFloat = Ty->getScalarType()->isFloatingPointTy();
// Assume that floating point arithmetic operations cost twice as much as
// integer operations.
unsigned OpCost = (IsFloat ? 2 : 1);
if (TLI->isOperationLegalOrPromote(ISD, LT.second)) {
// The operation is legal. Assume it costs 1.
// If the type is split to multiple registers, assume that there is some
// overhead to this.
// TODO: Once we have extract/insert subvector cost we need to use them.
if (LT.first > 1)
return LT.first * 2 * OpCost;
return LT.first * 1 * OpCost;
}
if (!TLI->isOperationExpand(ISD, LT.second)) {
// If the operation is custom lowered then assume
// thare the code is twice as expensive.
return LT.first * 2 * OpCost;
}
// Else, assume that we need to scalarize this op.
if (Ty->isVectorTy()) {
unsigned Num = Ty->getVectorNumElements();
unsigned Cost = TopTTI->getArithmeticInstrCost(Opcode, Ty->getScalarType());
// return the cost of multiple scalar invocation plus the cost of inserting
// and extracting the values.
return getScalarizationOverhead(Ty, true, true) + Num * Cost;
}
// We don't know anything about this scalar instruction.
return OpCost;
}
unsigned BasicTTI::getAltShuffleOverhead(Type *Ty) const {
assert(Ty->isVectorTy() && "Can only shuffle vectors");
unsigned Cost = 0;
// Shuffle cost is equal to the cost of extracting element from its argument
// plus the cost of inserting them onto the result vector.
// e.g. <4 x float> has a mask of <0,5,2,7> i.e we need to extract from index
// 0 of first vector, index 1 of second vector,index 2 of first vector and
// finally index 3 of second vector and insert them at index <0,1,2,3> of
// result vector.
for (int i = 0, e = Ty->getVectorNumElements(); i < e; ++i) {
Cost += TopTTI->getVectorInstrCost(Instruction::InsertElement, Ty, i);
Cost += TopTTI->getVectorInstrCost(Instruction::ExtractElement, Ty, i);
}
return Cost;
}
unsigned BasicTTI::getShuffleCost(ShuffleKind Kind, Type *Tp, int Index,
Type *SubTp) const {
if (Kind == SK_Alternate) {
return getAltShuffleOverhead(Tp);
}
return 1;
}
unsigned BasicTTI::getCastInstrCost(unsigned Opcode, Type *Dst,
Type *Src) const {
const TargetLoweringBase *TLI = getTLI();
int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode");
std::pair<unsigned, MVT> SrcLT = TLI->getTypeLegalizationCost(Src);
std::pair<unsigned, MVT> DstLT = TLI->getTypeLegalizationCost(Dst);
// Check for NOOP conversions.
if (SrcLT.first == DstLT.first &&
SrcLT.second.getSizeInBits() == DstLT.second.getSizeInBits()) {
// Bitcast between types that are legalized to the same type are free.
if (Opcode == Instruction::BitCast || Opcode == Instruction::Trunc)
return 0;
}
if (Opcode == Instruction::Trunc &&
TLI->isTruncateFree(SrcLT.second, DstLT.second))
return 0;
if (Opcode == Instruction::ZExt &&
TLI->isZExtFree(SrcLT.second, DstLT.second))
return 0;
// If the cast is marked as legal (or promote) then assume low cost.
if (SrcLT.first == DstLT.first &&
TLI->isOperationLegalOrPromote(ISD, DstLT.second))
return 1;
// Handle scalar conversions.
if (!Src->isVectorTy() && !Dst->isVectorTy()) {
// Scalar bitcasts are usually free.
if (Opcode == Instruction::BitCast)
return 0;
// Just check the op cost. If the operation is legal then assume it costs 1.
if (!TLI->isOperationExpand(ISD, DstLT.second))
return 1;
// Assume that illegal scalar instruction are expensive.
return 4;
}
// Check vector-to-vector casts.
if (Dst->isVectorTy() && Src->isVectorTy()) {
// If the cast is between same-sized registers, then the check is simple.
if (SrcLT.first == DstLT.first &&
SrcLT.second.getSizeInBits() == DstLT.second.getSizeInBits()) {
// Assume that Zext is done using AND.
if (Opcode == Instruction::ZExt)
return 1;
// Assume that sext is done using SHL and SRA.
if (Opcode == Instruction::SExt)
return 2;
// Just check the op cost. If the operation is legal then assume it costs
// 1 and multiply by the type-legalization overhead.
if (!TLI->isOperationExpand(ISD, DstLT.second))
return SrcLT.first * 1;
}
// If we are converting vectors and the operation is illegal, or
// if the vectors are legalized to different types, estimate the
// scalarization costs.
unsigned Num = Dst->getVectorNumElements();
unsigned Cost = TopTTI->getCastInstrCost(Opcode, Dst->getScalarType(),
Src->getScalarType());
// Return the cost of multiple scalar invocation plus the cost of
// inserting and extracting the values.
return getScalarizationOverhead(Dst, true, true) + Num * Cost;
}
// We already handled vector-to-vector and scalar-to-scalar conversions. This
// is where we handle bitcast between vectors and scalars. We need to assume
// that the conversion is scalarized in one way or another.
if (Opcode == Instruction::BitCast)
// Illegal bitcasts are done by storing and loading from a stack slot.
return (Src->isVectorTy()? getScalarizationOverhead(Src, false, true):0) +
(Dst->isVectorTy()? getScalarizationOverhead(Dst, true, false):0);
llvm_unreachable("Unhandled cast");
}
unsigned BasicTTI::getCFInstrCost(unsigned Opcode) const {
// Branches are assumed to be predicted.
return 0;
}
unsigned BasicTTI::getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy) const {
const TargetLoweringBase *TLI = getTLI();
int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode");
// Selects on vectors are actually vector selects.
if (ISD == ISD::SELECT) {
assert(CondTy && "CondTy must exist");
if (CondTy->isVectorTy())
ISD = ISD::VSELECT;
}
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(ValTy);
if (!(ValTy->isVectorTy() && !LT.second.isVector()) &&
!TLI->isOperationExpand(ISD, LT.second)) {
// The operation is legal. Assume it costs 1. Multiply
// by the type-legalization overhead.
return LT.first * 1;
}
// Otherwise, assume that the cast is scalarized.
if (ValTy->isVectorTy()) {
unsigned Num = ValTy->getVectorNumElements();
if (CondTy)
CondTy = CondTy->getScalarType();
unsigned Cost = TopTTI->getCmpSelInstrCost(Opcode, ValTy->getScalarType(),
CondTy);
// Return the cost of multiple scalar invocation plus the cost of inserting
// and extracting the values.
return getScalarizationOverhead(ValTy, true, false) + Num * Cost;
}
// Unknown scalar opcode.
return 1;
}
unsigned BasicTTI::getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index) const {
std::pair<unsigned, MVT> LT = getTLI()->getTypeLegalizationCost(Val->getScalarType());
return LT.first;
}
unsigned BasicTTI::getMemoryOpCost(unsigned Opcode, Type *Src,
unsigned Alignment,
unsigned AddressSpace) const {
assert(!Src->isVoidTy() && "Invalid type");
std::pair<unsigned, MVT> LT = getTLI()->getTypeLegalizationCost(Src);
// Assuming that all loads of legal types cost 1.
unsigned Cost = LT.first;
if (Src->isVectorTy() &&
Src->getPrimitiveSizeInBits() < LT.second.getSizeInBits()) {
// This is a vector load that legalizes to a larger type than the vector
// itself. Unless the corresponding extending load or truncating store is
// legal, then this will scalarize.
TargetLowering::LegalizeAction LA = TargetLowering::Expand;
EVT MemVT = getTLI()->getValueType(Src, true);
if (MemVT.isSimple() && MemVT != MVT::Other) {
if (Opcode == Instruction::Store)
LA = getTLI()->getTruncStoreAction(LT.second, MemVT.getSimpleVT());
else
LA = getTLI()->getLoadExtAction(ISD::EXTLOAD, LT.second, MemVT);
}
if (LA != TargetLowering::Legal && LA != TargetLowering::Custom) {
// This is a vector load/store for some illegal type that is scalarized.
// We must account for the cost of building or decomposing the vector.
Cost += getScalarizationOverhead(Src, Opcode != Instruction::Store,
Opcode == Instruction::Store);
}
}
return Cost;
}
unsigned BasicTTI::getIntrinsicInstrCost(Intrinsic::ID IID, Type *RetTy,
ArrayRef<Type *> Tys) const {
unsigned ISD = 0;
switch (IID) {
default: {
// Assume that we need to scalarize this intrinsic.
unsigned ScalarizationCost = 0;
unsigned ScalarCalls = 1;
if (RetTy->isVectorTy()) {
ScalarizationCost = getScalarizationOverhead(RetTy, true, false);
ScalarCalls = std::max(ScalarCalls, RetTy->getVectorNumElements());
}
for (unsigned i = 0, ie = Tys.size(); i != ie; ++i) {
if (Tys[i]->isVectorTy()) {
ScalarizationCost += getScalarizationOverhead(Tys[i], false, true);
ScalarCalls = std::max(ScalarCalls, RetTy->getVectorNumElements());
}
}
return ScalarCalls + ScalarizationCost;
}
// Look for intrinsics that can be lowered directly or turned into a scalar
// intrinsic call.
case Intrinsic::sqrt: ISD = ISD::FSQRT; break;
case Intrinsic::sin: ISD = ISD::FSIN; break;
case Intrinsic::cos: ISD = ISD::FCOS; break;
case Intrinsic::exp: ISD = ISD::FEXP; break;
case Intrinsic::exp2: ISD = ISD::FEXP2; break;
case Intrinsic::log: ISD = ISD::FLOG; break;
case Intrinsic::log10: ISD = ISD::FLOG10; break;
case Intrinsic::log2: ISD = ISD::FLOG2; break;
case Intrinsic::fabs: ISD = ISD::FABS; break;
case Intrinsic::minnum: ISD = ISD::FMINNUM; break;
case Intrinsic::maxnum: ISD = ISD::FMAXNUM; break;
case Intrinsic::copysign: ISD = ISD::FCOPYSIGN; break;
case Intrinsic::floor: ISD = ISD::FFLOOR; break;
case Intrinsic::ceil: ISD = ISD::FCEIL; break;
case Intrinsic::trunc: ISD = ISD::FTRUNC; break;
case Intrinsic::nearbyint:
ISD = ISD::FNEARBYINT; break;
case Intrinsic::rint: ISD = ISD::FRINT; break;
case Intrinsic::round: ISD = ISD::FROUND; break;
case Intrinsic::pow: ISD = ISD::FPOW; break;
case Intrinsic::fma: ISD = ISD::FMA; break;
case Intrinsic::fmuladd: ISD = ISD::FMA; break;
// FIXME: We should return 0 whenever getIntrinsicCost == TCC_Free.
case Intrinsic::lifetime_start:
case Intrinsic::lifetime_end:
return 0;
case Intrinsic::masked_store:
return TopTTI->getMaskedMemoryOpCost(Instruction::Store, Tys[0], 0, 0);
case Intrinsic::masked_load:
return TopTTI->getMaskedMemoryOpCost(Instruction::Load, RetTy, 0, 0);
}
const TargetLoweringBase *TLI = getTLI();
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(RetTy);
if (TLI->isOperationLegalOrPromote(ISD, LT.second)) {
// The operation is legal. Assume it costs 1.
// If the type is split to multiple registers, assume that there is some
// overhead to this.
// TODO: Once we have extract/insert subvector cost we need to use them.
if (LT.first > 1)
return LT.first * 2;
return LT.first * 1;
}
if (!TLI->isOperationExpand(ISD, LT.second)) {
// If the operation is custom lowered then assume
// thare the code is twice as expensive.
return LT.first * 2;
}
// If we can't lower fmuladd into an FMA estimate the cost as a floating
// point mul followed by an add.
if (IID == Intrinsic::fmuladd)
return TopTTI->getArithmeticInstrCost(BinaryOperator::FMul, RetTy) +
TopTTI->getArithmeticInstrCost(BinaryOperator::FAdd, RetTy);
// Else, assume that we need to scalarize this intrinsic. For math builtins
// this will emit a costly libcall, adding call overhead and spills. Make it
// very expensive.
if (RetTy->isVectorTy()) {
unsigned Num = RetTy->getVectorNumElements();
unsigned Cost = TopTTI->getIntrinsicInstrCost(IID, RetTy->getScalarType(),
Tys);
return 10 * Cost * Num;
}
// This is going to be turned into a library call, make it expensive.
return 10;
}
unsigned BasicTTI::getNumberOfParts(Type *Tp) const {
std::pair<unsigned, MVT> LT = getTLI()->getTypeLegalizationCost(Tp);
return LT.first;
}
unsigned BasicTTI::getAddressComputationCost(Type *Ty, bool IsComplex) const {
return 0;
}
unsigned BasicTTI::getReductionCost(unsigned Opcode, Type *Ty,
bool IsPairwise) const {
assert(Ty->isVectorTy() && "Expect a vector type");
unsigned NumVecElts = Ty->getVectorNumElements();
unsigned NumReduxLevels = Log2_32(NumVecElts);
unsigned ArithCost = NumReduxLevels *
TopTTI->getArithmeticInstrCost(Opcode, Ty);
// Assume the pairwise shuffles add a cost.
unsigned ShuffleCost =
NumReduxLevels * (IsPairwise + 1) *
TopTTI->getShuffleCost(SK_ExtractSubvector, Ty, NumVecElts / 2, Ty);
return ShuffleCost + ArithCost + getScalarizationOverhead(Ty, false, true);
}

llvm/trunk/lib/CodeGen/CodeGen.cpp

Show All 15 Lines
#include "llvm-c/Initialization.h" #include "llvm-c/Initialization.h"
#include "llvm/PassRegistry.h" #include "llvm/PassRegistry.h"
using namespace llvm; using namespace llvm;
/// initializeCodeGen - Initialize all passes linked into the CodeGen library. /// initializeCodeGen - Initialize all passes linked into the CodeGen library.
void llvm::initializeCodeGen(PassRegistry &Registry) { void llvm::initializeCodeGen(PassRegistry &Registry) {
initializeAtomicExpandPass(Registry); initializeAtomicExpandPass(Registry);
initializeBasicTTIPass(Registry);
initializeBranchFolderPassPass(Registry); initializeBranchFolderPassPass(Registry);
initializeCodeGenPreparePass(Registry); initializeCodeGenPreparePass(Registry);
initializeDeadMachineInstructionElimPass(Registry); initializeDeadMachineInstructionElimPass(Registry);
initializeEarlyIfConverterPass(Registry); initializeEarlyIfConverterPass(Registry);
initializeExpandPostRAPass(Registry); initializeExpandPostRAPass(Registry);
initializeExpandISelPseudosPass(Registry); initializeExpandISelPseudosPass(Registry);
initializeFinalizeMachineBundlesPass(Registry); initializeFinalizeMachineBundlesPass(Registry);
initializeGCMachineCodeAnalysisPass(Registry); initializeGCMachineCodeAnalysisPass(Registry);
▲ Show 20 Lines • Show All 49 Lines • Show Last 20 Lines

llvm/trunk/lib/CodeGen/CodeGenPrepare.cpp

Show First 20 Lines • Show All 156 Lines • ▼ Show 20 Lines explicit CodeGenPrepare(const TargetMachine *TM = nullptr)
} }
bool runOnFunction(Function &F) override; bool runOnFunction(Function &F) override;
const char *getPassName() const override { return "CodeGen Prepare"; } const char *getPassName() const override { return "CodeGen Prepare"; }
void getAnalysisUsage(AnalysisUsage &AU) const override { void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addPreserved<DominatorTreeWrapperPass>(); AU.addPreserved<DominatorTreeWrapperPass>();
AU.addRequired<TargetLibraryInfoWrapperPass>(); AU.addRequired<TargetLibraryInfoWrapperPass>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
} }
private: private:
bool EliminateFallThrough(Function &F); bool EliminateFallThrough(Function &F);
bool EliminateMostlyEmptyBlocks(Function &F); bool EliminateMostlyEmptyBlocks(Function &F);
bool CanMergeBlocks(const BasicBlock *BB, const BasicBlock *DestBB) const; bool CanMergeBlocks(const BasicBlock *BB, const BasicBlock *DestBB) const;
void EliminateMostlyEmptyBlock(BasicBlock *BB); void EliminateMostlyEmptyBlock(BasicBlock *BB);
bool OptimizeBlock(BasicBlock &BB, bool& ModifiedDT); bool OptimizeBlock(BasicBlock &BB, bool& ModifiedDT);
Show All 34 Linesbool CodeGenPrepare::runOnFunction(Function &F) {
// Clear per function information. // Clear per function information.
InsertedTruncsSet.clear(); InsertedTruncsSet.clear();
PromotedInsts.clear(); PromotedInsts.clear();
ModifiedDT = false; ModifiedDT = false;
if (TM) if (TM)
TLI = TM->getSubtargetImpl(F)->getTargetLowering(); TLI = TM->getSubtargetImpl(F)->getTargetLowering();
TLInfo = &getAnalysis<TargetLibraryInfoWrapperPass>().getTLI(); TLInfo = &getAnalysis<TargetLibraryInfoWrapperPass>().getTLI();
TTI = &getAnalysis<TargetTransformInfo>(); TTI = &getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
DominatorTreeWrapperPass *DTWP = DominatorTreeWrapperPass *DTWP =
getAnalysisIfAvailable<DominatorTreeWrapperPass>(); getAnalysisIfAvailable<DominatorTreeWrapperPass>();
DT = DTWP ? &DTWP->getDomTree() : nullptr; DT = DTWP ? &DTWP->getDomTree() : nullptr;
OptSize = F.getAttributes().hasAttribute(AttributeSet::FunctionIndex, OptSize = F.getAttributes().hasAttribute(AttributeSet::FunctionIndex,
Attribute::OptimizeForSize); Attribute::OptimizeForSize);
/// This optimization identifies DIV instructions that can be /// This optimization identifies DIV instructions that can be
/// profitably bypassed and carried out with a shorter, faster divide. /// profitably bypassed and carried out with a shorter, faster divide.
▲ Show 20 Lines • Show All 4,578 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/AArch64/AArch64TargetMachine.cpp

Show First 20 Lines • Show All 190 Lines • ▼ Show 20 Linespublic:
void addPreRegAlloc() override; void addPreRegAlloc() override;
void addPostRegAlloc() override; void addPostRegAlloc() override;
void addPreSched2() override; void addPreSched2() override;
void addPreEmitPass() override; void addPreEmitPass() override;
}; };
} // namespace } // namespace
void AArch64TargetMachine::addAnalysisPasses(PassManagerBase &PM) { void AArch64TargetMachine::addAnalysisPasses(PassManagerBase &PM) {
// Add first the target-independent BasicTTI pass, then our AArch64 pass. This
// allows the AArch64 pass to delegate to the target independent layer when
// appropriate.
PM.add(createBasicTargetTransformInfoPass(this));
PM.add(createAArch64TargetTransformInfoPass(this)); PM.add(createAArch64TargetTransformInfoPass(this));
} }
TargetPassConfig *AArch64TargetMachine::createPassConfig(PassManagerBase &PM) { TargetPassConfig *AArch64TargetMachine::createPassConfig(PassManagerBase &PM) {
return new AArch64PassConfig(this, PM); return new AArch64PassConfig(this, PM);
} }
void AArch64PassConfig::addIRPasses() { void AArch64PassConfig::addIRPasses() {
▲ Show 20 Lines • Show All 106 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/AArch64/AArch64TargetTransformInfo.cpp

Show All 12 Lines
/// independent and default TTI implementations handle the rest. /// independent and default TTI implementations handle the rest.
/// ///
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "AArch64.h" #include "AArch64.h"
#include "AArch64TargetMachine.h" #include "AArch64TargetMachine.h"
#include "MCTargetDesc/AArch64AddressingModes.h" #include "MCTargetDesc/AArch64AddressingModes.h"
#include "llvm/Analysis/TargetTransformInfo.h" #include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/CodeGen/BasicTTIImpl.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Target/CostTable.h" #include "llvm/Target/CostTable.h"
#include "llvm/Target/TargetLowering.h" #include "llvm/Target/TargetLowering.h"
#include <algorithm> #include <algorithm>
using namespace llvm; using namespace llvm;
#define DEBUG_TYPE "aarch64tti" #define DEBUG_TYPE "aarch64tti"
// Declare the pass initialization routine locally as target-specific passes
// don't have a target-wide initialization entry point, and so we rely on the
// pass constructor initialization.
namespace llvm {
void initializeAArch64TTIPass(PassRegistry &);
}
namespace { namespace {
class AArch64TTI final : public ImmutablePass, public TargetTransformInfo { class AArch64TTIImpl : public BasicTTIImplBase<AArch64TTIImpl> {
const AArch64TargetMachine *TM; typedef BasicTTIImplBase<AArch64TTIImpl> BaseT;
typedef TargetTransformInfo TTI;
const AArch64Subtarget *ST; const AArch64Subtarget *ST;
const AArch64TargetLowering *TLI; const AArch64TargetLowering *TLI;
/// Estimate the overhead of scalarizing an instruction. Insert and Extract /// Estimate the overhead of scalarizing an instruction. Insert and Extract
/// are set if the result needs to be inserted and/or extracted from vectors. /// are set if the result needs to be inserted and/or extracted from vectors.
unsigned getScalarizationOverhead(Type *Ty, bool Insert, bool Extract) const; unsigned getScalarizationOverhead(Type *Ty, bool Insert, bool Extract);
enum MemIntrinsicType { enum MemIntrinsicType {
VECTOR_LDST_TWO_ELEMENTS, VECTOR_LDST_TWO_ELEMENTS,
VECTOR_LDST_THREE_ELEMENTS, VECTOR_LDST_THREE_ELEMENTS,
VECTOR_LDST_FOUR_ELEMENTS VECTOR_LDST_FOUR_ELEMENTS
}; };
public: public:
AArch64TTI() : ImmutablePass(ID), TM(nullptr), ST(nullptr), TLI(nullptr) { explicit AArch64TTIImpl(const AArch64TargetMachine *TM = nullptr)
llvm_unreachable("This pass cannot be directly constructed"); : BaseT(TM), ST(TM ? TM->getSubtargetImpl() : nullptr),
} TLI(ST ? ST->getTargetLowering() : nullptr) {}
AArch64TTI(const AArch64TargetMachine *TM) // Provide value semantics. MSVC requires that we spell all of these out.
: ImmutablePass(ID), TM(TM), ST(TM->getSubtargetImpl()), AArch64TTIImpl(const AArch64TTIImpl &Arg)
TLI(TM->getSubtargetImpl()->getTargetLowering()) { : BaseT(static_cast<const BaseT &>(Arg)), ST(Arg.ST), TLI(Arg.TLI) {}
initializeAArch64TTIPass(*PassRegistry::getPassRegistry()); AArch64TTIImpl(AArch64TTIImpl &&Arg)
} : BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)),
TLI(std::move(Arg.TLI)) {}
void initializePass() override { pushTTIStack(this); } AArch64TTIImpl &operator=(const AArch64TTIImpl &RHS) {
BaseT::operator=(static_cast<const BaseT &>(RHS));
void getAnalysisUsage(AnalysisUsage &AU) const override { ST = RHS.ST;
TargetTransformInfo::getAnalysisUsage(AU); TLI = RHS.TLI;
} return *this;
}
/// Pass identification. AArch64TTIImpl &operator=(AArch64TTIImpl &&RHS) {
static char ID; BaseT::operator=(std::move(static_cast<BaseT &>(RHS)));
ST = std::move(RHS.ST);
/// Provide necessary pointer adjustments for the two base classes. TLI = std::move(RHS.TLI);
void *getAdjustedAnalysisPointer(const void *ID) override { return *this;
if (ID == &TargetTransformInfo::ID)
return (TargetTransformInfo *)this;
return this;
} }
/// \name Scalar TTI Implementations /// \name Scalar TTI Implementations
/// @{ /// @{
unsigned getIntImmCost(int64_t Val) const;
unsigned getIntImmCost(const APInt &Imm, Type *Ty) const override; using BaseT::getIntImmCost;
unsigned getIntImmCost(int64_t Val);
unsigned getIntImmCost(const APInt &Imm, Type *Ty);
unsigned getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, unsigned getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm,
Type *Ty) const override; Type *Ty);
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm, unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty) const override; Type *Ty);
PopcntSupportKind getPopcntSupport(unsigned TyWidth) const override; TTI::PopcntSupportKind getPopcntSupport(unsigned TyWidth);
/// @} /// @}
/// \name Vector TTI Implementations /// \name Vector TTI Implementations
/// @{ /// @{
unsigned getNumberOfRegisters(bool Vector) const override { unsigned getNumberOfRegisters(bool Vector) {
if (Vector) { if (Vector) {
if (ST->hasNEON()) if (ST->hasNEON())
return 32; return 32;
return 0; return 0;
} }
return 31; return 31;
} }
unsigned getRegisterBitWidth(bool Vector) const override { unsigned getRegisterBitWidth(bool Vector) {
if (Vector) { if (Vector) {
if (ST->hasNEON()) if (ST->hasNEON())
return 128; return 128;
return 0; return 0;
} }
return 64; return 64;
} }
unsigned getMaxInterleaveFactor() const override; unsigned getMaxInterleaveFactor();
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) const unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src);
override;
unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index) const unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index);
override;
unsigned getArithmeticInstrCost( unsigned getArithmeticInstrCost(
unsigned Opcode, Type *Ty, OperandValueKind Opd1Info = OK_AnyValue, unsigned Opcode, Type *Ty,
OperandValueKind Opd2Info = OK_AnyValue, TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue,
OperandValueProperties Opd1PropInfo = OP_None, TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue,
OperandValueProperties Opd2PropInfo = OP_None) const override; TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None,
TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None);
unsigned getAddressComputationCost(Type *Ty, bool IsComplex) const override; unsigned getAddressComputationCost(Type *Ty, bool IsComplex);
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy) const unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy);
override;
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) const override; unsigned AddressSpace);
unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type*> Tys) const override; unsigned getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys);
void getUnrollingPreferences(const Function *F, Loop *L, void getUnrollingPreferences(const Function *F, Loop *L,
UnrollingPreferences &UP) const override; TTI::UnrollingPreferences &UP);
Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst, Value *getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst,
Type *ExpectedType) const override; Type *ExpectedType);
bool getTgtMemIntrinsic(IntrinsicInst *Inst, bool getTgtMemIntrinsic(IntrinsicInst *Inst, MemIntrinsicInfo &Info);
MemIntrinsicInfo &Info) const override;
/// @} /// @}
}; };
} // end anonymous namespace } // end anonymous namespace
INITIALIZE_AG_PASS(AArch64TTI, TargetTransformInfo, "aarch64tti",
"AArch64 Target Transform Info", true, true, false)
char AArch64TTI::ID = 0;
ImmutablePass * ImmutablePass *
llvm::createAArch64TargetTransformInfoPass(const AArch64TargetMachine *TM) { llvm::createAArch64TargetTransformInfoPass(const AArch64TargetMachine *TM) {
return new AArch64TTI(TM); return new TargetTransformInfoWrapperPass(AArch64TTIImpl(TM));
} }
/// \brief Calculate the cost of materializing a 64-bit value. This helper /// \brief Calculate the cost of materializing a 64-bit value. This helper
/// method might only calculate a fraction of a larger immediate. Therefore it /// method might only calculate a fraction of a larger immediate. Therefore it
/// is valid to return a cost of ZERO. /// is valid to return a cost of ZERO.
unsigned AArch64TTI::getIntImmCost(int64_t Val) const { unsigned AArch64TTIImpl::getIntImmCost(int64_t Val) {
// Check if the immediate can be encoded within an instruction. // Check if the immediate can be encoded within an instruction.
if (Val == 0 || AArch64_AM::isLogicalImmediate(Val, 64)) if (Val == 0 || AArch64_AM::isLogicalImmediate(Val, 64))
return 0; return 0;
if (Val < 0) if (Val < 0)
Val = ~Val; Val = ~Val;
// Calculate how many moves we will need to materialize this constant. // Calculate how many moves we will need to materialize this constant.
unsigned LZ = countLeadingZeros((uint64_t)Val); unsigned LZ = countLeadingZeros((uint64_t)Val);
return (64 - LZ + 15) / 16; return (64 - LZ + 15) / 16;
} }
/// \brief Calculate the cost of materializing the given constant. /// \brief Calculate the cost of materializing the given constant.
unsigned AArch64TTI::getIntImmCost(const APInt &Imm, Type *Ty) const { unsigned AArch64TTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
if (BitSize == 0) if (BitSize == 0)
return ~0U; return ~0U;
// Sign-extend all constants to a multiple of 64-bit. // Sign-extend all constants to a multiple of 64-bit.
APInt ImmVal = Imm; APInt ImmVal = Imm;
if (BitSize & 0x3f) if (BitSize & 0x3f)
ImmVal = Imm.sext((BitSize + 63) & ~0x3fU); ImmVal = Imm.sext((BitSize + 63) & ~0x3fU);
// Split the constant into 64-bit chunks and calculate the cost for each // Split the constant into 64-bit chunks and calculate the cost for each
// chunk. // chunk.
unsigned Cost = 0; unsigned Cost = 0;
for (unsigned ShiftVal = 0; ShiftVal < BitSize; ShiftVal += 64) { for (unsigned ShiftVal = 0; ShiftVal < BitSize; ShiftVal += 64) {
APInt Tmp = ImmVal.ashr(ShiftVal).sextOrTrunc(64); APInt Tmp = ImmVal.ashr(ShiftVal).sextOrTrunc(64);
int64_t Val = Tmp.getSExtValue(); int64_t Val = Tmp.getSExtValue();
Cost += getIntImmCost(Val); Cost += getIntImmCost(Val);
} }
// We need at least one instruction to materialze the constant. // We need at least one instruction to materialze the constant.
return std::max(1U, Cost); return std::max(1U, Cost);
} }
unsigned AArch64TTI::getIntImmCost(unsigned Opcode, unsigned Idx, unsigned AArch64TTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx,
const APInt &Imm, Type *Ty) const { const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
// There is no cost model for constants with a bit size of 0. Return TCC_Free // There is no cost model for constants with a bit size of 0. Return TCC_Free
// here, so that constant hoisting will ignore this constant. // here, so that constant hoisting will ignore this constant.
if (BitSize == 0) if (BitSize == 0)
return TCC_Free; return TTI::TCC_Free;
unsigned ImmIdx = ~0U; unsigned ImmIdx = ~0U;
switch (Opcode) { switch (Opcode) {
default: default:
return TCC_Free; return TTI::TCC_Free;
case Instruction::GetElementPtr: case Instruction::GetElementPtr:
// Always hoist the base address of a GetElementPtr. // Always hoist the base address of a GetElementPtr.
if (Idx == 0) if (Idx == 0)
return 2 * TCC_Basic; return 2 * TTI::TCC_Basic;
return TCC_Free; return TTI::TCC_Free;
case Instruction::Store: case Instruction::Store:
ImmIdx = 0; ImmIdx = 0;
break; break;
case Instruction::Add: case Instruction::Add:
case Instruction::Sub: case Instruction::Sub:
case Instruction::Mul: case Instruction::Mul:
case Instruction::UDiv: case Instruction::UDiv:
case Instruction::SDiv: case Instruction::SDiv:
case Instruction::URem: case Instruction::URem:
case Instruction::SRem: case Instruction::SRem:
case Instruction::And: case Instruction::And:
case Instruction::Or: case Instruction::Or:
case Instruction::Xor: case Instruction::Xor:
case Instruction::ICmp: case Instruction::ICmp:
ImmIdx = 1; ImmIdx = 1;
break; break;
// Always return TCC_Free for the shift value of a shift instruction. // Always return TCC_Free for the shift value of a shift instruction.
case Instruction::Shl: case Instruction::Shl:
case Instruction::LShr: case Instruction::LShr:
case Instruction::AShr: case Instruction::AShr:
if (Idx == 1) if (Idx == 1)
return TCC_Free; return TTI::TCC_Free;
break; break;
case Instruction::Trunc: case Instruction::Trunc:
case Instruction::ZExt: case Instruction::ZExt:
case Instruction::SExt: case Instruction::SExt:
case Instruction::IntToPtr: case Instruction::IntToPtr:
case Instruction::PtrToInt: case Instruction::PtrToInt:
case Instruction::BitCast: case Instruction::BitCast:
case Instruction::PHI: case Instruction::PHI:
case Instruction::Call: case Instruction::Call:
case Instruction::Select: case Instruction::Select:
case Instruction::Ret: case Instruction::Ret:
case Instruction::Load: case Instruction::Load:
break; break;
} }
if (Idx == ImmIdx) { if (Idx == ImmIdx) {
unsigned NumConstants = (BitSize + 63) / 64; unsigned NumConstants = (BitSize + 63) / 64;
unsigned Cost = AArch64TTI::getIntImmCost(Imm, Ty); unsigned Cost = AArch64TTIImpl::getIntImmCost(Imm, Ty);
return (Cost <= NumConstants * TCC_Basic) return (Cost <= NumConstants * TTI::TCC_Basic)
? static_cast<unsigned>(TCC_Free) : Cost; ? static_cast<unsigned>(TTI::TCC_Free)
: Cost;
} }
return AArch64TTI::getIntImmCost(Imm, Ty); return AArch64TTIImpl::getIntImmCost(Imm, Ty);
} }
unsigned AArch64TTI::getIntImmCost(Intrinsic::ID IID, unsigned Idx, unsigned AArch64TTIImpl::getIntImmCost(Intrinsic::ID IID, unsigned Idx,
const APInt &Imm, Type *Ty) const { const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
// There is no cost model for constants with a bit size of 0. Return TCC_Free // There is no cost model for constants with a bit size of 0. Return TCC_Free
// here, so that constant hoisting will ignore this constant. // here, so that constant hoisting will ignore this constant.
if (BitSize == 0) if (BitSize == 0)
return TCC_Free; return TTI::TCC_Free;
switch (IID) { switch (IID) {
default: default:
return TCC_Free; return TTI::TCC_Free;
case Intrinsic::sadd_with_overflow: case Intrinsic::sadd_with_overflow:
case Intrinsic::uadd_with_overflow: case Intrinsic::uadd_with_overflow:
case Intrinsic::ssub_with_overflow: case Intrinsic::ssub_with_overflow:
case Intrinsic::usub_with_overflow: case Intrinsic::usub_with_overflow:
case Intrinsic::smul_with_overflow: case Intrinsic::smul_with_overflow:
case Intrinsic::umul_with_overflow: case Intrinsic::umul_with_overflow:
if (Idx == 1) { if (Idx == 1) {
unsigned NumConstants = (BitSize + 63) / 64; unsigned NumConstants = (BitSize + 63) / 64;
unsigned Cost = AArch64TTI::getIntImmCost(Imm, Ty); unsigned Cost = AArch64TTIImpl::getIntImmCost(Imm, Ty);
return (Cost <= NumConstants * TCC_Basic) return (Cost <= NumConstants * TTI::TCC_Basic)
? static_cast<unsigned>(TCC_Free) : Cost; ? static_cast<unsigned>(TTI::TCC_Free)
: Cost;
} }
break; break;
case Intrinsic::experimental_stackmap: case Intrinsic::experimental_stackmap:
if ((Idx < 2) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue()))) if ((Idx < 2) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue())))
return TCC_Free; return TTI::TCC_Free;
break; break;
case Intrinsic::experimental_patchpoint_void: case Intrinsic::experimental_patchpoint_void:
case Intrinsic::experimental_patchpoint_i64: case Intrinsic::experimental_patchpoint_i64:
if ((Idx < 4) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue()))) if ((Idx < 4) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue())))
return TCC_Free; return TTI::TCC_Free;
break; break;
} }
return AArch64TTI::getIntImmCost(Imm, Ty); return AArch64TTIImpl::getIntImmCost(Imm, Ty);
} }
AArch64TTI::PopcntSupportKind TargetTransformInfo::PopcntSupportKind
AArch64TTI::getPopcntSupport(unsigned TyWidth) const { AArch64TTIImpl::getPopcntSupport(unsigned TyWidth) {
assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2"); assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2");
if (TyWidth == 32 || TyWidth == 64) if (TyWidth == 32 || TyWidth == 64)
return PSK_FastHardware; return TTI::PSK_FastHardware;
// TODO: AArch64TargetLowering::LowerCTPOP() supports 128bit popcount. // TODO: AArch64TargetLowering::LowerCTPOP() supports 128bit popcount.
return PSK_Software; return TTI::PSK_Software;
} }
unsigned AArch64TTI::getCastInstrCost(unsigned Opcode, Type *Dst, unsigned AArch64TTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst,
Type *Src) const { Type *Src) {
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
EVT SrcTy = TLI->getValueType(Src); EVT SrcTy = TLI->getValueType(Src);
EVT DstTy = TLI->getValueType(Dst); EVT DstTy = TLI->getValueType(Dst);
if (!SrcTy.isSimple() || !DstTy.isSimple()) if (!SrcTy.isSimple() || !DstTy.isSimple())
return TargetTransformInfo::getCastInstrCost(Opcode, Dst, Src); return BaseT::getCastInstrCost(Opcode, Dst, Src);
static const TypeConversionCostTblEntry<MVT> ConversionTbl[] = { static const TypeConversionCostTblEntry<MVT> ConversionTbl[] = {
// LowerVectorINT_TO_FP: // LowerVectorINT_TO_FP:
{ ISD::SINT_TO_FP, MVT::v2f32, MVT::v2i32, 1 }, { ISD::SINT_TO_FP, MVT::v2f32, MVT::v2i32, 1 },
{ ISD::SINT_TO_FP, MVT::v4f32, MVT::v4i32, 1 }, { ISD::SINT_TO_FP, MVT::v4f32, MVT::v4i32, 1 },
{ ISD::SINT_TO_FP, MVT::v2f64, MVT::v2i64, 1 }, { ISD::SINT_TO_FP, MVT::v2f64, MVT::v2i64, 1 },
{ ISD::UINT_TO_FP, MVT::v2f32, MVT::v2i32, 1 }, { ISD::UINT_TO_FP, MVT::v2f32, MVT::v2i32, 1 },
{ ISD::UINT_TO_FP, MVT::v4f32, MVT::v4i32, 1 }, { ISD::UINT_TO_FP, MVT::v4f32, MVT::v4i32, 1 },
▲ Show 20 Lines • Show All 54 Lines • ▼ Show 20 Linesunsigned AArch64TTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst,
}; };
int Idx = ConvertCostTableLookup<MVT>( int Idx = ConvertCostTableLookup<MVT>(
ConversionTbl, array_lengthof(ConversionTbl), ISD, DstTy.getSimpleVT(), ConversionTbl, array_lengthof(ConversionTbl), ISD, DstTy.getSimpleVT(),
SrcTy.getSimpleVT()); SrcTy.getSimpleVT());
if (Idx != -1) if (Idx != -1)
return ConversionTbl[Idx].Cost; return ConversionTbl[Idx].Cost;
return TargetTransformInfo::getCastInstrCost(Opcode, Dst, Src); return BaseT::getCastInstrCost(Opcode, Dst, Src);
} }
unsigned AArch64TTI::getVectorInstrCost(unsigned Opcode, Type *Val, unsigned AArch64TTIImpl::getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index) const { unsigned Index) {
assert(Val->isVectorTy() && "This must be a vector type"); assert(Val->isVectorTy() && "This must be a vector type");
if (Index != -1U) { if (Index != -1U) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Val); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Val);
// This type is legalized to a scalar type. // This type is legalized to a scalar type.
if (!LT.second.isVector()) if (!LT.second.isVector())
return 0; return 0;
// The type may be split. Normalize the index to the new type. // The type may be split. Normalize the index to the new type.
unsigned Width = LT.second.getVectorNumElements(); unsigned Width = LT.second.getVectorNumElements();
Index = Index % Width; Index = Index % Width;
// The element at index zero is already inside the vector. // The element at index zero is already inside the vector.
if (Index == 0) if (Index == 0)
return 0; return 0;
} }
// All other insert/extracts cost this much. // All other insert/extracts cost this much.
return 2; return 2;
} }
unsigned AArch64TTI::getArithmeticInstrCost( unsigned AArch64TTIImpl::getArithmeticInstrCost(
unsigned Opcode, Type *Ty, OperandValueKind Opd1Info, unsigned Opcode, Type *Ty, TTI::OperandValueKind Opd1Info,
OperandValueKind Opd2Info, OperandValueProperties Opd1PropInfo, TTI::OperandValueKind Opd2Info, TTI::OperandValueProperties Opd1PropInfo,
OperandValueProperties Opd2PropInfo) const { TTI::OperandValueProperties Opd2PropInfo) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Ty); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Ty);
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
if (ISD == ISD::SDIV && if (ISD == ISD::SDIV &&
Opd2Info == TargetTransformInfo::OK_UniformConstantValue && Opd2Info == TargetTransformInfo::OK_UniformConstantValue &&
Opd2PropInfo == TargetTransformInfo::OP_PowerOf2) { Opd2PropInfo == TargetTransformInfo::OP_PowerOf2) {
Show All 14 Lines if (ISD == ISD::SDIV &&
Cost += getArithmeticInstrCost(Instruction::AShr, Ty, Opd1Info, Opd2Info, Cost += getArithmeticInstrCost(Instruction::AShr, Ty, Opd1Info, Opd2Info,
TargetTransformInfo::OP_None, TargetTransformInfo::OP_None,
TargetTransformInfo::OP_None); TargetTransformInfo::OP_None);
return Cost; return Cost;
} }
switch (ISD) { switch (ISD) {
default: default:
return TargetTransformInfo::getArithmeticInstrCost( return BaseT::getArithmeticInstrCost(Opcode, Ty, Opd1Info, Opd2Info,
Opcode, Ty, Opd1Info, Opd2Info, Opd1PropInfo, Opd2PropInfo); Opd1PropInfo, Opd2PropInfo);
case ISD::ADD: case ISD::ADD:
case ISD::MUL: case ISD::MUL:
case ISD::XOR: case ISD::XOR:
case ISD::OR: case ISD::OR:
case ISD::AND: case ISD::AND:
// These nodes are marked as 'custom' for combining purposes only. // These nodes are marked as 'custom' for combining purposes only.
// We know that they are legal. See LowerAdd in ISelLowering. // We know that they are legal. See LowerAdd in ISelLowering.
return 1 * LT.first; return 1 * LT.first;
} }
} }
unsigned AArch64TTI::getAddressComputationCost(Type *Ty, bool IsComplex) const { unsigned AArch64TTIImpl::getAddressComputationCost(Type *Ty, bool IsComplex) {
// Address computations in vectorized code with non-consecutive addresses will // Address computations in vectorized code with non-consecutive addresses will
// likely result in more instructions compared to scalar code where the // likely result in more instructions compared to scalar code where the
// computation can more often be merged into the index mode. The resulting // computation can more often be merged into the index mode. The resulting
// extra micro-ops can significantly decrease throughput. // extra micro-ops can significantly decrease throughput.
unsigned NumVectorInstToHideOverhead = 10; unsigned NumVectorInstToHideOverhead = 10;
if (Ty->isVectorTy() && IsComplex) if (Ty->isVectorTy() && IsComplex)
return NumVectorInstToHideOverhead; return NumVectorInstToHideOverhead;
// In many cases the address computation is not merged into the instruction // In many cases the address computation is not merged into the instruction
// addressing mode. // addressing mode.
return 1; return 1;
} }
unsigned AArch64TTI::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, unsigned AArch64TTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy) const { Type *CondTy) {
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
// We don't lower vector selects well that are wider than the register width. // We don't lower vector selects well that are wider than the register width.
if (ValTy->isVectorTy() && ISD == ISD::SELECT) { if (ValTy->isVectorTy() && ISD == ISD::SELECT) {
// We would need this many instructions to hide the scalarization happening. // We would need this many instructions to hide the scalarization happening.
unsigned AmortizationCost = 20; unsigned AmortizationCost = 20;
static const TypeConversionCostTblEntry<MVT::SimpleValueType> static const TypeConversionCostTblEntry<MVT::SimpleValueType>
VectorSelectTbl[] = { VectorSelectTbl[] = {
Show All 10 Lines if (ValTy->isVectorTy() && ISD == ISD::SELECT) {
if (SelCondTy.isSimple() && SelValTy.isSimple()) { if (SelCondTy.isSimple() && SelValTy.isSimple()) {
int Idx = int Idx =
ConvertCostTableLookup(VectorSelectTbl, ISD, SelCondTy.getSimpleVT(), ConvertCostTableLookup(VectorSelectTbl, ISD, SelCondTy.getSimpleVT(),
SelValTy.getSimpleVT()); SelValTy.getSimpleVT());
if (Idx != -1) if (Idx != -1)
return VectorSelectTbl[Idx].Cost; return VectorSelectTbl[Idx].Cost;
} }
} }
return TargetTransformInfo::getCmpSelInstrCost(Opcode, ValTy, CondTy); return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy);
} }
unsigned AArch64TTI::getMemoryOpCost(unsigned Opcode, Type *Src, unsigned AArch64TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
unsigned Alignment, unsigned Alignment,
unsigned AddressSpace) const { unsigned AddressSpace) {
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Src); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Src);
if (Opcode == Instruction::Store && Src->isVectorTy() && Alignment != 16 && if (Opcode == Instruction::Store && Src->isVectorTy() && Alignment != 16 &&
Src->getVectorElementType()->isIntegerTy(64)) { Src->getVectorElementType()->isIntegerTy(64)) {
// Unaligned stores are extremely inefficient. We don't split // Unaligned stores are extremely inefficient. We don't split
// unaligned v2i64 stores because the negative impact that has shown in // unaligned v2i64 stores because the negative impact that has shown in
// practice on inlined memcpy code. // practice on inlined memcpy code.
// We make v2i64 stores expensive so that we will only vectorize if there // We make v2i64 stores expensive so that we will only vectorize if there
Show All 11 Lines if (Src->isVectorTy() && Src->getVectorElementType()->isIntegerTy(8) &&
unsigned NumVectorizableInstsToAmortize = NumVecElts * 2; unsigned NumVectorizableInstsToAmortize = NumVecElts * 2;
// We generate 2 instructions per vector element. // We generate 2 instructions per vector element.
return NumVectorizableInstsToAmortize * NumVecElts * 2; return NumVectorizableInstsToAmortize * NumVecElts * 2;
} }
return LT.first; return LT.first;
} }
unsigned AArch64TTI::getCostOfKeepingLiveOverCall(ArrayRef<Type*> Tys) const { unsigned AArch64TTIImpl::getCostOfKeepingLiveOverCall(ArrayRef<Type *> Tys) {
unsigned Cost = 0; unsigned Cost = 0;
for (auto *I : Tys) { for (auto *I : Tys) {
if (!I->isVectorTy()) if (!I->isVectorTy())
continue; continue;
if (I->getScalarSizeInBits() * I->getVectorNumElements() == 128) if (I->getScalarSizeInBits() * I->getVectorNumElements() == 128)
Cost += getMemoryOpCost(Instruction::Store, I, 128, 0) + Cost += getMemoryOpCost(Instruction::Store, I, 128, 0) +
getMemoryOpCost(Instruction::Load, I, 128, 0); getMemoryOpCost(Instruction::Load, I, 128, 0);
} }
return Cost; return Cost;
} }
unsigned AArch64TTI::getMaxInterleaveFactor() const { unsigned AArch64TTIImpl::getMaxInterleaveFactor() {
if (ST->isCortexA57()) if (ST->isCortexA57())
return 4; return 4;
return 2; return 2;
} }
void AArch64TTI::getUnrollingPreferences(const Function *F, Loop *L, void AArch64TTIImpl::getUnrollingPreferences(const Function *F, Loop *L,
UnrollingPreferences &UP) const { TTI::UnrollingPreferences &UP) {
// Disable partial & runtime unrolling on -Os. // Disable partial & runtime unrolling on -Os.
UP.PartialOptSizeThreshold = 0; UP.PartialOptSizeThreshold = 0;
} }
Value *AArch64TTI::getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst, Value *AArch64TTIImpl::getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst,
Type *ExpectedType) const { Type *ExpectedType) {
switch (Inst->getIntrinsicID()) { switch (Inst->getIntrinsicID()) {
default: default:
return nullptr; return nullptr;
case Intrinsic::aarch64_neon_st2: case Intrinsic::aarch64_neon_st2:
case Intrinsic::aarch64_neon_st3: case Intrinsic::aarch64_neon_st3:
case Intrinsic::aarch64_neon_st4: { case Intrinsic::aarch64_neon_st4: {
// Create a struct type // Create a struct type
StructType *ST = dyn_cast<StructType>(ExpectedType); StructType *ST = dyn_cast<StructType>(ExpectedType);
Show All 18 LinesValue *AArch64TTIImpl::getOrCreateResultFromMemIntrinsic(IntrinsicInst *Inst,
case Intrinsic::aarch64_neon_ld3: case Intrinsic::aarch64_neon_ld3:
case Intrinsic::aarch64_neon_ld4: case Intrinsic::aarch64_neon_ld4:
if (Inst->getType() == ExpectedType) if (Inst->getType() == ExpectedType)
return Inst; return Inst;
return nullptr; return nullptr;
} }
} }
bool AArch64TTI::getTgtMemIntrinsic(IntrinsicInst *Inst, bool AArch64TTIImpl::getTgtMemIntrinsic(IntrinsicInst *Inst,
MemIntrinsicInfo &Info) const { MemIntrinsicInfo &Info) {
switch (Inst->getIntrinsicID()) { switch (Inst->getIntrinsicID()) {
default: default:
break; break;
case Intrinsic::aarch64_neon_ld2: case Intrinsic::aarch64_neon_ld2:
case Intrinsic::aarch64_neon_ld3: case Intrinsic::aarch64_neon_ld3:
case Intrinsic::aarch64_neon_ld4: case Intrinsic::aarch64_neon_ld4:
Info.ReadMem = true; Info.ReadMem = true;
Info.WriteMem = false; Info.WriteMem = false;
Show All 33 Lines

llvm/trunk/lib/Target/ARM/ARMTargetMachine.cpp

Show First 20 Lines • Show All 210 Lines • ▼ Show 20 Lines if (!I) {
// function that reside in TargetOptions. // function that reside in TargetOptions.
resetTargetOptions(F); resetTargetOptions(F);
I = llvm::make_unique<ARMSubtarget>(TargetTriple, CPU, FS, *this, isLittle); I = llvm::make_unique<ARMSubtarget>(TargetTriple, CPU, FS, *this, isLittle);
} }
return I.get(); return I.get();
} }
void ARMBaseTargetMachine::addAnalysisPasses(PassManagerBase &PM) { void ARMBaseTargetMachine::addAnalysisPasses(PassManagerBase &PM) {
// Add first the target-independent BasicTTI pass, then our ARM pass. This
// allows the ARM pass to delegate to the target independent layer when
// appropriate.
PM.add(createBasicTargetTransformInfoPass(this));
PM.add(createARMTargetTransformInfoPass(this)); PM.add(createARMTargetTransformInfoPass(this));
} }
void ARMTargetMachine::anchor() { } void ARMTargetMachine::anchor() { }
ARMTargetMachine::ARMTargetMachine(const Target &T, StringRef TT, StringRef CPU, ARMTargetMachine::ARMTargetMachine(const Target &T, StringRef TT, StringRef CPU,
StringRef FS, const TargetOptions &Options, StringRef FS, const TargetOptions &Options,
▲ Show 20 Lines • Show All 169 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/ARM/ARMTargetTransformInfo.cpp

Show All 11 Lines
/// more precise answers to certain TTI queries, while letting the target /// more precise answers to certain TTI queries, while letting the target
/// independent and default TTI implementations handle the rest. /// independent and default TTI implementations handle the rest.
/// ///
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "ARM.h" #include "ARM.h"
#include "ARMTargetMachine.h" #include "ARMTargetMachine.h"
#include "llvm/Analysis/TargetTransformInfo.h" #include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/CodeGen/BasicTTIImpl.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Target/CostTable.h" #include "llvm/Target/CostTable.h"
#include "llvm/Target/TargetLowering.h" #include "llvm/Target/TargetLowering.h"
using namespace llvm; using namespace llvm;
#define DEBUG_TYPE "armtti" #define DEBUG_TYPE "armtti"
// Declare the pass initialization routine locally as target-specific passes
// don't have a target-wide initialization entry point, and so we rely on the
// pass constructor initialization.
namespace llvm {
void initializeARMTTIPass(PassRegistry &);
}
namespace { namespace {
class ARMTTI final : public ImmutablePass, public TargetTransformInfo { class ARMTTIImpl : public BasicTTIImplBase<ARMTTIImpl> {
const ARMBaseTargetMachine *TM; typedef BasicTTIImplBase<ARMTTIImpl> BaseT;
typedef TargetTransformInfo TTI;
const ARMSubtarget *ST; const ARMSubtarget *ST;
const ARMTargetLowering *TLI; const ARMTargetLowering *TLI;
/// Estimate the overhead of scalarizing an instruction. Insert and Extract /// Estimate the overhead of scalarizing an instruction. Insert and Extract
/// are set if the result needs to be inserted and/or extracted from vectors. /// are set if the result needs to be inserted and/or extracted from vectors.
unsigned getScalarizationOverhead(Type *Ty, bool Insert, bool Extract) const; unsigned getScalarizationOverhead(Type *Ty, bool Insert, bool Extract);
public: public:
ARMTTI() : ImmutablePass(ID), TM(nullptr), ST(nullptr), TLI(nullptr) { explicit ARMTTIImpl(const ARMBaseTargetMachine *TM = nullptr)
llvm_unreachable("This pass cannot be directly constructed"); : BaseT(TM), ST(TM ? TM->getSubtargetImpl() : nullptr),
} TLI(ST ? ST->getTargetLowering() : nullptr) {}
ARMTTI(const ARMBaseTargetMachine *TM) // Provide value semantics. MSVC requires that we spell all of these out.
: ImmutablePass(ID), TM(TM), ST(TM->getSubtargetImpl()), ARMTTIImpl(const ARMTTIImpl &Arg)
TLI(TM->getSubtargetImpl()->getTargetLowering()) { : BaseT(static_cast<const BaseT &>(Arg)), ST(Arg.ST), TLI(Arg.TLI) {}
initializeARMTTIPass(*PassRegistry::getPassRegistry()); ARMTTIImpl(ARMTTIImpl &&Arg)
} : BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)),
TLI(std::move(Arg.TLI)) {}
void initializePass() override { ARMTTIImpl &operator=(const ARMTTIImpl &RHS) {
pushTTIStack(this); BaseT::operator=(static_cast<const BaseT &>(RHS));
} ST = RHS.ST;
TLI = RHS.TLI;
void getAnalysisUsage(AnalysisUsage &AU) const override { return *this;
TargetTransformInfo::getAnalysisUsage(AU); }
} ARMTTIImpl &operator=(ARMTTIImpl &&RHS) {
BaseT::operator=(std::move(static_cast<BaseT &>(RHS)));
/// Pass identification. ST = std::move(RHS.ST);
static char ID; TLI = std::move(RHS.TLI);
return *this;
/// Provide necessary pointer adjustments for the two base classes.
void *getAdjustedAnalysisPointer(const void *ID) override {
if (ID == &TargetTransformInfo::ID)
return (TargetTransformInfo*)this;
return this;
} }
/// \name Scalar TTI Implementations /// \name Scalar TTI Implementations
/// @{ /// @{
using TargetTransformInfo::getIntImmCost;
unsigned getIntImmCost(const APInt &Imm, Type *Ty) const override; using BaseT::getIntImmCost;
unsigned getIntImmCost(const APInt &Imm, Type *Ty);
/// @} /// @}
/// \name Vector TTI Implementations /// \name Vector TTI Implementations
/// @{ /// @{
unsigned getNumberOfRegisters(bool Vector) const override { unsigned getNumberOfRegisters(bool Vector) {
if (Vector) { if (Vector) {
if (ST->hasNEON()) if (ST->hasNEON())
return 16; return 16;
return 0; return 0;
} }
if (ST->isThumb1Only()) if (ST->isThumb1Only())
return 8; return 8;
return 13; return 13;
} }
unsigned getRegisterBitWidth(bool Vector) const override { unsigned getRegisterBitWidth(bool Vector) {
if (Vector) { if (Vector) {
if (ST->hasNEON()) if (ST->hasNEON())
return 128; return 128;
return 0; return 0;
} }
return 32; return 32;
} }
unsigned getMaxInterleaveFactor() const override { unsigned getMaxInterleaveFactor() {
// These are out of order CPUs: // These are out of order CPUs:
if (ST->isCortexA15() || ST->isSwift()) if (ST->isCortexA15() || ST->isSwift())
return 2; return 2;
return 1; return 1;
} }
unsigned getShuffleCost(ShuffleKind Kind, Type *Tp, unsigned getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index,
int Index, Type *SubTp) const override; Type *SubTp);
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src);
Type *Src) const override;
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy);
Type *CondTy) const override;
unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index);
unsigned Index) const override;
unsigned getAddressComputationCost(Type *Val, unsigned getAddressComputationCost(Type *Val, bool IsComplex);
bool IsComplex) const override;
unsigned getArithmeticInstrCost( unsigned getArithmeticInstrCost(
unsigned Opcode, Type *Ty, OperandValueKind Op1Info = OK_AnyValue, unsigned Opcode, Type *Ty,
OperandValueKind Op2Info = OK_AnyValue, TTI::OperandValueKind Op1Info = TTI::OK_AnyValue,
OperandValueProperties Opd1PropInfo = OP_None, TTI::OperandValueKind Op2Info = TTI::OK_AnyValue,
OperandValueProperties Opd2PropInfo = OP_None) const override; TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None,
TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None);
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) const override; unsigned AddressSpace);
/// @} /// @}
}; };
} // end anonymous namespace } // end anonymous namespace
INITIALIZE_AG_PASS(ARMTTI, TargetTransformInfo, "armtti",
"ARM Target Transform Info", true, true, false)
char ARMTTI::ID = 0;
ImmutablePass * ImmutablePass *
llvm::createARMTargetTransformInfoPass(const ARMBaseTargetMachine *TM) { llvm::createARMTargetTransformInfoPass(const ARMBaseTargetMachine *TM) {
return new ARMTTI(TM); return new TargetTransformInfoWrapperPass(ARMTTIImpl(TM));
} }
unsigned ARMTTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) {
unsigned ARMTTI::getIntImmCost(const APInt &Imm, Type *Ty) const {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned Bits = Ty->getPrimitiveSizeInBits(); unsigned Bits = Ty->getPrimitiveSizeInBits();
if (Bits == 0 || Bits > 32) if (Bits == 0 || Bits > 32)
return 4; return 4;
int32_t SImmVal = Imm.getSExtValue(); int32_t SImmVal = Imm.getSExtValue();
uint32_t ZImmVal = Imm.getZExtValue(); uint32_t ZImmVal = Imm.getZExtValue();
Show All 15 Linesunsigned ARMTTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) {
if (SImmVal >= 0 && SImmVal < 256) if (SImmVal >= 0 && SImmVal < 256)
return 1; return 1;
if ((~ZImmVal < 256) || ARM_AM::isThumbImmShiftedVal(ZImmVal)) if ((~ZImmVal < 256) || ARM_AM::isThumbImmShiftedVal(ZImmVal))
return 2; return 2;
// Load from constantpool. // Load from constantpool.
return 3; return 3;
} }
unsigned ARMTTI::getCastInstrCost(unsigned Opcode, Type *Dst, unsigned ARMTTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) {
Type *Src) const {
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
// Single to/from double precision conversions. // Single to/from double precision conversions.
static const CostTblEntry<MVT::SimpleValueType> NEONFltDblTbl[] = { static const CostTblEntry<MVT::SimpleValueType> NEONFltDblTbl[] = {
// Vector fptrunc/fpext conversions. // Vector fptrunc/fpext conversions.
{ ISD::FP_ROUND, MVT::v2f64, 2 }, { ISD::FP_ROUND, MVT::v2f64, 2 },
{ ISD::FP_EXTEND, MVT::v2f32, 2 }, { ISD::FP_EXTEND, MVT::v2f32, 2 },
{ ISD::FP_EXTEND, MVT::v4f32, 4 } { ISD::FP_EXTEND, MVT::v4f32, 4 }
}; };
if (Src->isVectorTy() && ST->hasNEON() && (ISD == ISD::FP_ROUND || if (Src->isVectorTy() && ST->hasNEON() && (ISD == ISD::FP_ROUND ||
ISD == ISD::FP_EXTEND)) { ISD == ISD::FP_EXTEND)) {
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Src); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Src);
int Idx = CostTableLookup(NEONFltDblTbl, ISD, LT.second); int Idx = CostTableLookup(NEONFltDblTbl, ISD, LT.second);
if (Idx != -1) if (Idx != -1)
return LT.first * NEONFltDblTbl[Idx].Cost; return LT.first * NEONFltDblTbl[Idx].Cost;
} }
EVT SrcTy = TLI->getValueType(Src); EVT SrcTy = TLI->getValueType(Src);
EVT DstTy = TLI->getValueType(Dst); EVT DstTy = TLI->getValueType(Dst);
if (!SrcTy.isSimple() || !DstTy.isSimple()) if (!SrcTy.isSimple() || !DstTy.isSimple())
return TargetTransformInfo::getCastInstrCost(Opcode, Dst, Src); return BaseT::getCastInstrCost(Opcode, Dst, Src);
// Some arithmetic, load and store operations have specific instructions // Some arithmetic, load and store operations have specific instructions
// to cast up/down their types automatically at no extra cost. // to cast up/down their types automatically at no extra cost.
// TODO: Get these tables to know at least what the related operations are. // TODO: Get these tables to know at least what the related operations are.
static const TypeConversionCostTblEntry<MVT::SimpleValueType> static const TypeConversionCostTblEntry<MVT::SimpleValueType>
NEONVectorConversionTbl[] = { NEONVectorConversionTbl[] = {
{ ISD::SIGN_EXTEND, MVT::v4i32, MVT::v4i16, 0 }, { ISD::SIGN_EXTEND, MVT::v4i32, MVT::v4i16, 0 },
{ ISD::ZERO_EXTEND, MVT::v4i32, MVT::v4i16, 0 }, { ISD::ZERO_EXTEND, MVT::v4i32, MVT::v4i16, 0 },
▲ Show 20 Lines • Show All 154 Lines • ▼ Show 20 Linesunsigned ARMTTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) {
if (SrcTy.isInteger()) { if (SrcTy.isInteger()) {
int Idx = ConvertCostTableLookup(ARMIntegerConversionTbl, ISD, int Idx = ConvertCostTableLookup(ARMIntegerConversionTbl, ISD,
DstTy.getSimpleVT(), SrcTy.getSimpleVT()); DstTy.getSimpleVT(), SrcTy.getSimpleVT());
if (Idx != -1) if (Idx != -1)
return ARMIntegerConversionTbl[Idx].Cost; return ARMIntegerConversionTbl[Idx].Cost;
} }
return TargetTransformInfo::getCastInstrCost(Opcode, Dst, Src); return BaseT::getCastInstrCost(Opcode, Dst, Src);
} }
unsigned ARMTTI::getVectorInstrCost(unsigned Opcode, Type *ValTy, unsigned ARMTTIImpl::getVectorInstrCost(unsigned Opcode, Type *ValTy,
unsigned Index) const { unsigned Index) {
// Penalize inserting into an D-subregister. We end up with a three times // Penalize inserting into an D-subregister. We end up with a three times
// lower estimated throughput on swift. // lower estimated throughput on swift.
if (ST->isSwift() && if (ST->isSwift() &&
Opcode == Instruction::InsertElement && Opcode == Instruction::InsertElement &&
ValTy->isVectorTy() && ValTy->isVectorTy() &&
ValTy->getScalarSizeInBits() <= 32) ValTy->getScalarSizeInBits() <= 32)
return 3; return 3;
// Cross-class copies are expensive on many microarchitectures, // Cross-class copies are expensive on many microarchitectures,
// so assume they are expensive by default. // so assume they are expensive by default.
if ((Opcode == Instruction::InsertElement || if ((Opcode == Instruction::InsertElement ||
Opcode == Instruction::ExtractElement) && Opcode == Instruction::ExtractElement) &&
ValTy->getVectorElementType()->isIntegerTy()) ValTy->getVectorElementType()->isIntegerTy())
return 3; return 3;
return TargetTransformInfo::getVectorInstrCost(Opcode, ValTy, Index); return BaseT::getVectorInstrCost(Opcode, ValTy, Index);
} }
unsigned ARMTTI::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, unsigned ARMTTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy) const { Type *CondTy) {
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
// On NEON a a vector select gets lowered to vbsl. // On NEON a a vector select gets lowered to vbsl.
if (ST->hasNEON() && ValTy->isVectorTy() && ISD == ISD::SELECT) { if (ST->hasNEON() && ValTy->isVectorTy() && ISD == ISD::SELECT) {
// Lowering of some vector selects is currently far from perfect. // Lowering of some vector selects is currently far from perfect.
static const TypeConversionCostTblEntry<MVT::SimpleValueType> static const TypeConversionCostTblEntry<MVT::SimpleValueType>
NEONVectorSelectTbl[] = { NEONVectorSelectTbl[] = {
{ ISD::SELECT, MVT::v16i1, MVT::v16i16, 2*16 + 1 + 3*1 + 4*1 }, { ISD::SELECT, MVT::v16i1, MVT::v16i16, 2*16 + 1 + 3*1 + 4*1 },
Show All 13 Lines if (SelCondTy.isSimple() && SelValTy.isSimple()) {
if (Idx != -1) if (Idx != -1)
return NEONVectorSelectTbl[Idx].Cost; return NEONVectorSelectTbl[Idx].Cost;
} }
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(ValTy); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(ValTy);
return LT.first; return LT.first;
} }
return TargetTransformInfo::getCmpSelInstrCost(Opcode, ValTy, CondTy); return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy);
} }
unsigned ARMTTI::getAddressComputationCost(Type *Ty, bool IsComplex) const { unsigned ARMTTIImpl::getAddressComputationCost(Type *Ty, bool IsComplex) {
// Address computations in vectorized code with non-consecutive addresses will // Address computations in vectorized code with non-consecutive addresses will
// likely result in more instructions compared to scalar code where the // likely result in more instructions compared to scalar code where the
// computation can more often be merged into the index mode. The resulting // computation can more often be merged into the index mode. The resulting
// extra micro-ops can significantly decrease throughput. // extra micro-ops can significantly decrease throughput.
unsigned NumVectorInstToHideOverhead = 10; unsigned NumVectorInstToHideOverhead = 10;
if (Ty->isVectorTy() && IsComplex) if (Ty->isVectorTy() && IsComplex)
return NumVectorInstToHideOverhead; return NumVectorInstToHideOverhead;
// In many cases the address computation is not merged into the instruction // In many cases the address computation is not merged into the instruction
// addressing mode. // addressing mode.
return 1; return 1;
} }
unsigned ARMTTI::getShuffleCost(ShuffleKind Kind, Type *Tp, int Index, unsigned ARMTTIImpl::getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index,
Type *SubTp) const { Type *SubTp) {
// We only handle costs of reverse and alternate shuffles for now. // We only handle costs of reverse and alternate shuffles for now.
if (Kind != SK_Reverse && Kind != SK_Alternate) if (Kind != TTI::SK_Reverse && Kind != TTI::SK_Alternate)
return TargetTransformInfo::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
if (Kind == SK_Reverse) { if (Kind == TTI::SK_Reverse) {
static const CostTblEntry<MVT::SimpleValueType> NEONShuffleTbl[] = { static const CostTblEntry<MVT::SimpleValueType> NEONShuffleTbl[] = {
// Reverse shuffle cost one instruction if we are shuffling within a // Reverse shuffle cost one instruction if we are shuffling within a
// double word (vrev) or two if we shuffle a quad word (vrev, vext). // double word (vrev) or two if we shuffle a quad word (vrev, vext).
{ISD::VECTOR_SHUFFLE, MVT::v2i32, 1}, {ISD::VECTOR_SHUFFLE, MVT::v2i32, 1},
{ISD::VECTOR_SHUFFLE, MVT::v2f32, 1}, {ISD::VECTOR_SHUFFLE, MVT::v2f32, 1},
{ISD::VECTOR_SHUFFLE, MVT::v2i64, 1}, {ISD::VECTOR_SHUFFLE, MVT::v2i64, 1},
{ISD::VECTOR_SHUFFLE, MVT::v2f64, 1}, {ISD::VECTOR_SHUFFLE, MVT::v2f64, 1},
{ISD::VECTOR_SHUFFLE, MVT::v4i32, 2}, {ISD::VECTOR_SHUFFLE, MVT::v4i32, 2},
{ISD::VECTOR_SHUFFLE, MVT::v4f32, 2}, {ISD::VECTOR_SHUFFLE, MVT::v4f32, 2},
{ISD::VECTOR_SHUFFLE, MVT::v8i16, 2}, {ISD::VECTOR_SHUFFLE, MVT::v8i16, 2},
{ISD::VECTOR_SHUFFLE, MVT::v16i8, 2}}; {ISD::VECTOR_SHUFFLE, MVT::v16i8, 2}};
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Tp); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Tp);
int Idx = CostTableLookup(NEONShuffleTbl, ISD::VECTOR_SHUFFLE, LT.second); int Idx = CostTableLookup(NEONShuffleTbl, ISD::VECTOR_SHUFFLE, LT.second);
if (Idx == -1) if (Idx == -1)
return TargetTransformInfo::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
return LT.first * NEONShuffleTbl[Idx].Cost; return LT.first * NEONShuffleTbl[Idx].Cost;
} }
if (Kind == SK_Alternate) { if (Kind == TTI::SK_Alternate) {
static const CostTblEntry<MVT::SimpleValueType> NEONAltShuffleTbl[] = { static const CostTblEntry<MVT::SimpleValueType> NEONAltShuffleTbl[] = {
// Alt shuffle cost table for ARM. Cost is the number of instructions // Alt shuffle cost table for ARM. Cost is the number of instructions
// required to create the shuffled vector. // required to create the shuffled vector.
{ISD::VECTOR_SHUFFLE, MVT::v2f32, 1}, {ISD::VECTOR_SHUFFLE, MVT::v2f32, 1},
{ISD::VECTOR_SHUFFLE, MVT::v2i64, 1}, {ISD::VECTOR_SHUFFLE, MVT::v2i64, 1},
{ISD::VECTOR_SHUFFLE, MVT::v2f64, 1}, {ISD::VECTOR_SHUFFLE, MVT::v2f64, 1},
{ISD::VECTOR_SHUFFLE, MVT::v2i32, 1}, {ISD::VECTOR_SHUFFLE, MVT::v2i32, 1},
{ISD::VECTOR_SHUFFLE, MVT::v4i32, 2}, {ISD::VECTOR_SHUFFLE, MVT::v4i32, 2},
{ISD::VECTOR_SHUFFLE, MVT::v4f32, 2}, {ISD::VECTOR_SHUFFLE, MVT::v4f32, 2},
{ISD::VECTOR_SHUFFLE, MVT::v4i16, 2}, {ISD::VECTOR_SHUFFLE, MVT::v4i16, 2},
{ISD::VECTOR_SHUFFLE, MVT::v8i16, 16}, {ISD::VECTOR_SHUFFLE, MVT::v8i16, 16},
{ISD::VECTOR_SHUFFLE, MVT::v16i8, 32}}; {ISD::VECTOR_SHUFFLE, MVT::v16i8, 32}};
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Tp); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Tp);
int Idx = int Idx =
CostTableLookup(NEONAltShuffleTbl, ISD::VECTOR_SHUFFLE, LT.second); CostTableLookup(NEONAltShuffleTbl, ISD::VECTOR_SHUFFLE, LT.second);
if (Idx == -1) if (Idx == -1)
return TargetTransformInfo::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
return LT.first * NEONAltShuffleTbl[Idx].Cost; return LT.first * NEONAltShuffleTbl[Idx].Cost;
} }
return TargetTransformInfo::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
} }
unsigned ARMTTI::getArithmeticInstrCost( unsigned ARMTTIImpl::getArithmeticInstrCost(
unsigned Opcode, Type *Ty, OperandValueKind Op1Info, unsigned Opcode, Type *Ty, TTI::OperandValueKind Op1Info,
OperandValueKind Op2Info, OperandValueProperties Opd1PropInfo, TTI::OperandValueKind Op2Info, TTI::OperandValueProperties Opd1PropInfo,
OperandValueProperties Opd2PropInfo) const { TTI::OperandValueProperties Opd2PropInfo) {
int ISDOpcode = TLI->InstructionOpcodeToISD(Opcode); int ISDOpcode = TLI->InstructionOpcodeToISD(Opcode);
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Ty); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Ty);
const unsigned FunctionCallDivCost = 20; const unsigned FunctionCallDivCost = 20;
const unsigned ReciprocalDivCost = 10; const unsigned ReciprocalDivCost = 10;
static const CostTblEntry<MVT::SimpleValueType> CostTbl[] = { static const CostTblEntry<MVT::SimpleValueType> CostTbl[] = {
// Division. // Division.
Show All 39 Linesunsigned ARMTTIImpl::getArithmeticInstrCost(
int Idx = -1; int Idx = -1;
if (ST->hasNEON()) if (ST->hasNEON())
Idx = CostTableLookup(CostTbl, ISDOpcode, LT.second); Idx = CostTableLookup(CostTbl, ISDOpcode, LT.second);
if (Idx != -1) if (Idx != -1)
return LT.first * CostTbl[Idx].Cost; return LT.first * CostTbl[Idx].Cost;
unsigned Cost = TargetTransformInfo::getArithmeticInstrCost( unsigned Cost = BaseT::getArithmeticInstrCost(Opcode, Ty, Op1Info, Op2Info,
Opcode, Ty, Op1Info, Op2Info, Opd1PropInfo, Opd2PropInfo); Opd1PropInfo, Opd2PropInfo);
// This is somewhat of a hack. The problem that we are facing is that SROA // This is somewhat of a hack. The problem that we are facing is that SROA
// creates a sequence of shift, and, or instructions to construct values. // creates a sequence of shift, and, or instructions to construct values.
// These sequences are recognized by the ISel and have zero-cost. Not so for // These sequences are recognized by the ISel and have zero-cost. Not so for
// the vectorized code. Because we have support for v2i64 but not i64 those // the vectorized code. Because we have support for v2i64 but not i64 those
// sequences look particularly beneficial to vectorize. // sequences look particularly beneficial to vectorize.
// To work around this we increase the cost of v2i64 operations to make them // To work around this we increase the cost of v2i64 operations to make them
// seem less beneficial. // seem less beneficial.
if (LT.second == MVT::v2i64 && if (LT.second == MVT::v2i64 &&
Op2Info == TargetTransformInfo::OK_UniformConstantValue) Op2Info == TargetTransformInfo::OK_UniformConstantValue)
Cost += 4; Cost += 4;
return Cost; return Cost;
} }
unsigned ARMTTI::getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, unsigned ARMTTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
unsigned AddressSpace) const { unsigned Alignment,
unsigned AddressSpace) {
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Src); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Src);
if (Src->isVectorTy() && Alignment != 16 && if (Src->isVectorTy() && Alignment != 16 &&
Src->getVectorElementType()->isDoubleTy()) { Src->getVectorElementType()->isDoubleTy()) {
// Unaligned loads/stores are extremely inefficient. // Unaligned loads/stores are extremely inefficient.
// We need 4 uops for vst.1/vld.1 vs 1uop for vldr/vstr. // We need 4 uops for vst.1/vld.1 vs 1uop for vldr/vstr.
return LT.first * 4; return LT.first * 4;
} }
return LT.first; return LT.first;
} }

llvm/trunk/lib/Target/Mips/MipsTargetMachine.cpp

Show First 20 Lines • Show All 239 Lines • ▼ Show 20 Lines
void MipsTargetMachine::addAnalysisPasses(PassManagerBase &PM) { void MipsTargetMachine::addAnalysisPasses(PassManagerBase &PM) {
if (Subtarget->allowMixed16_32()) { if (Subtarget->allowMixed16_32()) {
DEBUG(errs() << "No "); DEBUG(errs() << "No ");
//FIXME: The Basic Target Transform Info //FIXME: The Basic Target Transform Info
// pass needs to become a function pass instead of // pass needs to become a function pass instead of
// being an immutable pass and then this method as it exists now // being an immutable pass and then this method as it exists now
// would be unnecessary. // would be unnecessary.
PM.add(createNoTargetTransformInfoPass()); PM.add(createNoTargetTransformInfoPass(getDataLayout()));
} else } else
LLVMTargetMachine::addAnalysisPasses(PM); LLVMTargetMachine::addAnalysisPasses(PM);
DEBUG(errs() << "Target Transform Info Pass Added\n"); DEBUG(errs() << "Target Transform Info Pass Added\n");
} }
// Implemented by targets that want to run passes immediately before // Implemented by targets that want to run passes immediately before
// machine code is emitted. return true if -print-machineinstrs should // machine code is emitted. return true if -print-machineinstrs should
// print out the code after the passes. // print out the code after the passes.
void MipsPassConfig::addPreEmitPass() { void MipsPassConfig::addPreEmitPass() {
MipsTargetMachine &TM = getMipsTargetMachine(); MipsTargetMachine &TM = getMipsTargetMachine();
addPass(createMipsDelaySlotFillerPass(TM)); addPass(createMipsDelaySlotFillerPass(TM));
addPass(createMipsLongBranchPass(TM)); addPass(createMipsLongBranchPass(TM));
addPass(createMipsConstantIslandPass(TM)); addPass(createMipsConstantIslandPass(TM));
} }

llvm/trunk/lib/Target/NVPTX/NVPTXTargetMachine.cpp

Show First 20 Lines • Show All 131 Lines • ▼ Show 20 Lines
} // end anonymous namespace } // end anonymous namespace
TargetPassConfig *NVPTXTargetMachine::createPassConfig(PassManagerBase &PM) { TargetPassConfig *NVPTXTargetMachine::createPassConfig(PassManagerBase &PM) {
NVPTXPassConfig *PassConfig = new NVPTXPassConfig(this, PM); NVPTXPassConfig *PassConfig = new NVPTXPassConfig(this, PM);
return PassConfig; return PassConfig;
} }
void NVPTXTargetMachine::addAnalysisPasses(PassManagerBase &PM) { void NVPTXTargetMachine::addAnalysisPasses(PassManagerBase &PM) {
// Add first the target-independent BasicTTI pass, then our NVPTX pass. This
// allows the NVPTX pass to delegate to the target independent layer when
// appropriate.
PM.add(createBasicTargetTransformInfoPass(this));
PM.add(createNVPTXTargetTransformInfoPass(this)); PM.add(createNVPTXTargetTransformInfoPass(this));
} }
void NVPTXPassConfig::addIRPasses() { void NVPTXPassConfig::addIRPasses() {
// The following passes are known to not play well with virtual regs hanging // The following passes are known to not play well with virtual regs hanging
// around after register allocation (which in our case, is *all* registers). // around after register allocation (which in our case, is *all* registers).
// We explicitly disable them here. We do, however, need some functionality // We explicitly disable them here. We do, however, need some functionality
// of the PrologEpilogCodeInserter pass, so we emulate that behavior in the // of the PrologEpilogCodeInserter pass, so we emulate that behavior in the
▲ Show 20 Lines • Show All 122 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/NVPTX/NVPTXTargetTransformInfo.cpp

Show All 13 Lines
// independent and default TTI implementations handle the rest. // independent and default TTI implementations handle the rest.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "NVPTXTargetMachine.h" #include "NVPTXTargetMachine.h"
#include "llvm/Analysis/LoopInfo.h" #include "llvm/Analysis/LoopInfo.h"
#include "llvm/Analysis/TargetTransformInfo.h" #include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/Analysis/ValueTracking.h" #include "llvm/Analysis/ValueTracking.h"
#include "llvm/CodeGen/BasicTTIImpl.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Target/CostTable.h" #include "llvm/Target/CostTable.h"
#include "llvm/Target/TargetLowering.h" #include "llvm/Target/TargetLowering.h"
using namespace llvm; using namespace llvm;
#define DEBUG_TYPE "NVPTXtti" #define DEBUG_TYPE "NVPTXtti"
// Declare the pass initialization routine locally as target-specific passes
// don't have a target-wide initialization entry point, and so we rely on the
// pass constructor initialization.
namespace llvm {
void initializeNVPTXTTIPass(PassRegistry &);
}
namespace { namespace {
class NVPTXTTI final : public ImmutablePass, public TargetTransformInfo { class NVPTXTTIImpl : public BasicTTIImplBase<NVPTXTTIImpl> {
const NVPTXTargetLowering *TLI; typedef BasicTTIImplBase<NVPTXTTIImpl> BaseT;
public: typedef TargetTransformInfo TTI;
NVPTXTTI() : ImmutablePass(ID), TLI(nullptr) {
llvm_unreachable("This pass cannot be directly constructed");
}
NVPTXTTI(const NVPTXTargetMachine *TM)
: ImmutablePass(ID), TLI(TM->getSubtargetImpl()->getTargetLowering()) {
initializeNVPTXTTIPass(*PassRegistry::getPassRegistry());
}
void initializePass() override { pushTTIStack(this); }
void getAnalysisUsage(AnalysisUsage &AU) const override {
TargetTransformInfo::getAnalysisUsage(AU);
}
/// Pass identification. const NVPTXTargetLowering *TLI;
static char ID;
/// Provide necessary pointer adjustments for the two base classes. public:
void *getAdjustedAnalysisPointer(const void *ID) override { explicit NVPTXTTIImpl(const NVPTXTargetMachine *TM = nullptr)
if (ID == &TargetTransformInfo::ID) : BaseT(TM),
return (TargetTransformInfo *)this; TLI(TM ? TM->getSubtargetImpl()->getTargetLowering() : nullptr) {}
return this;
// Provide value semantics. MSVC requires that we spell all of these out.
NVPTXTTIImpl(const NVPTXTTIImpl &Arg)
: BaseT(static_cast<const BaseT &>(Arg)), TLI(Arg.TLI) {}
NVPTXTTIImpl(NVPTXTTIImpl &&Arg)
: BaseT(std::move(static_cast<BaseT &>(Arg))), TLI(std::move(Arg.TLI)) {}
NVPTXTTIImpl &operator=(const NVPTXTTIImpl &RHS) {
BaseT::operator=(static_cast<const BaseT &>(RHS));
TLI = RHS.TLI;
return *this;
}
NVPTXTTIImpl &operator=(NVPTXTTIImpl &&RHS) {
BaseT::operator=(std::move(static_cast<BaseT &>(RHS)));
TLI = std::move(RHS.TLI);
return *this;
} }
bool hasBranchDivergence() const override; bool hasBranchDivergence() { return true; }
unsigned getArithmeticInstrCost( unsigned getArithmeticInstrCost(
unsigned Opcode, Type *Ty, OperandValueKind Opd1Info = OK_AnyValue, unsigned Opcode, Type *Ty,
OperandValueKind Opd2Info = OK_AnyValue, TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue,
OperandValueProperties Opd1PropInfo = OP_None, TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue,
OperandValueProperties Opd2PropInfo = OP_None) const override; TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None,
TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None);
}; };
} // end anonymous namespace } // end anonymous namespace
INITIALIZE_AG_PASS(NVPTXTTI, TargetTransformInfo, "NVPTXtti",
"NVPTX Target Transform Info", true, true, false)
char NVPTXTTI::ID = 0;
ImmutablePass * ImmutablePass *
llvm::createNVPTXTargetTransformInfoPass(const NVPTXTargetMachine *TM) { llvm::createNVPTXTargetTransformInfoPass(const NVPTXTargetMachine *TM) {
return new NVPTXTTI(TM); return new TargetTransformInfoWrapperPass(NVPTXTTIImpl(TM));
} }
bool NVPTXTTI::hasBranchDivergence() const { return true; } unsigned NVPTXTTIImpl::getArithmeticInstrCost(
unsigned Opcode, Type *Ty, TTI::OperandValueKind Opd1Info,
unsigned NVPTXTTI::getArithmeticInstrCost( TTI::OperandValueKind Opd2Info, TTI::OperandValueProperties Opd1PropInfo,
unsigned Opcode, Type *Ty, OperandValueKind Opd1Info, TTI::OperandValueProperties Opd2PropInfo) {
OperandValueKind Opd2Info, OperandValueProperties Opd1PropInfo,
OperandValueProperties Opd2PropInfo) const {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Ty); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Ty);
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
switch (ISD) { switch (ISD) {
default: default:
return TargetTransformInfo::getArithmeticInstrCost( return BaseT::getArithmeticInstrCost(Opcode, Ty, Opd1Info, Opd2Info,
Opcode, Ty, Opd1Info, Opd2Info, Opd1PropInfo, Opd2PropInfo); Opd1PropInfo, Opd2PropInfo);
case ISD::ADD: case ISD::ADD:
case ISD::MUL: case ISD::MUL:
case ISD::XOR: case ISD::XOR:
case ISD::OR: case ISD::OR:
case ISD::AND: case ISD::AND:
// The machine code (SASS) simulates an i64 with two i32. Therefore, we // The machine code (SASS) simulates an i64 with two i32. Therefore, we
// estimate that arithmetic operations on i64 are twice as expensive as // estimate that arithmetic operations on i64 are twice as expensive as
// those on types that can fit into one machine register. // those on types that can fit into one machine register.
if (LT.second.SimpleTy == MVT::i64) if (LT.second.SimpleTy == MVT::i64)
return 2 * LT.first; return 2 * LT.first;
// Delegate other cases to the basic TTI. // Delegate other cases to the basic TTI.
return TargetTransformInfo::getArithmeticInstrCost( return BaseT::getArithmeticInstrCost(Opcode, Ty, Opd1Info, Opd2Info,
Opcode, Ty, Opd1Info, Opd2Info, Opd1PropInfo, Opd2PropInfo); Opd1PropInfo, Opd2PropInfo);
} }
} }

llvm/trunk/lib/Target/PowerPC/PPCTargetMachine.cpp

Show First 20 Lines • Show All 269 Lines • ▼ Show 20 Lines
void PPCPassConfig::addPreEmitPass() { void PPCPassConfig::addPreEmitPass() {
if (getOptLevel() != CodeGenOpt::None) if (getOptLevel() != CodeGenOpt::None)
addPass(createPPCEarlyReturnPass(), false); addPass(createPPCEarlyReturnPass(), false);
// Must run branch selection immediately preceding the asm printer. // Must run branch selection immediately preceding the asm printer.
addPass(createPPCBranchSelectionPass(), false); addPass(createPPCBranchSelectionPass(), false);
} }
void PPCTargetMachine::addAnalysisPasses(PassManagerBase &PM) { void PPCTargetMachine::addAnalysisPasses(PassManagerBase &PM) {
// Add first the target-independent BasicTTI pass, then our PPC pass. This
// allows the PPC pass to delegate to the target independent layer when
// appropriate.
PM.add(createBasicTargetTransformInfoPass(this));
PM.add(createPPCTargetTransformInfoPass(this)); PM.add(createPPCTargetTransformInfoPass(this));
} }

llvm/trunk/lib/Target/PowerPC/PPCTargetTransformInfo.cpp

Show All 11 Lines
/// more precise answers to certain TTI queries, while letting the target /// more precise answers to certain TTI queries, while letting the target
/// independent and default TTI implementations handle the rest. /// independent and default TTI implementations handle the rest.
/// ///
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "PPC.h" #include "PPC.h"
#include "PPCTargetMachine.h" #include "PPCTargetMachine.h"
#include "llvm/Analysis/TargetTransformInfo.h" #include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/CodeGen/BasicTTIImpl.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Target/CostTable.h" #include "llvm/Target/CostTable.h"
#include "llvm/Target/TargetLowering.h" #include "llvm/Target/TargetLowering.h"
using namespace llvm; using namespace llvm;
#define DEBUG_TYPE "ppctti" #define DEBUG_TYPE "ppctti"
static cl::opt<bool> DisablePPCConstHoist("disable-ppc-constant-hoisting", static cl::opt<bool> DisablePPCConstHoist("disable-ppc-constant-hoisting",
cl::desc("disable constant hoisting on PPC"), cl::init(false), cl::Hidden); cl::desc("disable constant hoisting on PPC"), cl::init(false), cl::Hidden);
// Declare the pass initialization routine locally as target-specific passes
// don't have a target-wide initialization entry point, and so we rely on the
// pass constructor initialization.
namespace llvm {
void initializePPCTTIPass(PassRegistry &);
}
namespace { namespace {
class PPCTTI final : public ImmutablePass, public TargetTransformInfo { class PPCTTIImpl : public BasicTTIImplBase<PPCTTIImpl> {
const TargetMachine *TM; typedef BasicTTIImplBase<PPCTTIImpl> BaseT;
typedef TargetTransformInfo TTI;
const PPCSubtarget *ST; const PPCSubtarget *ST;
const PPCTargetLowering *TLI; const PPCTargetLowering *TLI;
public: public:
PPCTTI() : ImmutablePass(ID), ST(nullptr), TLI(nullptr) { explicit PPCTTIImpl(const PPCTargetMachine *TM = nullptr)
llvm_unreachable("This pass cannot be directly constructed"); : BaseT(TM), ST(TM->getSubtargetImpl()), TLI(ST->getTargetLowering()) {}
}
PPCTTI(const PPCTargetMachine *TM)
: ImmutablePass(ID), TM(TM), ST(TM->getSubtargetImpl()),
TLI(TM->getSubtargetImpl()->getTargetLowering()) {
initializePPCTTIPass(*PassRegistry::getPassRegistry());
}
void initializePass() override {
pushTTIStack(this);
}
void getAnalysisUsage(AnalysisUsage &AU) const override {
TargetTransformInfo::getAnalysisUsage(AU);
}
/// Pass identification.
static char ID;
/// Provide necessary pointer adjustments for the two base classes. // Provide value semantics. MSVC requires that we spell all of these out.
void *getAdjustedAnalysisPointer(const void *ID) override { PPCTTIImpl(const PPCTTIImpl &Arg)
if (ID == &TargetTransformInfo::ID) : BaseT(static_cast<const BaseT &>(Arg)), ST(Arg.ST), TLI(Arg.TLI) {}
return (TargetTransformInfo*)this; PPCTTIImpl(PPCTTIImpl &&Arg)
return this; : BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)),
TLI(std::move(Arg.TLI)) {}
PPCTTIImpl &operator=(const PPCTTIImpl &RHS) {
BaseT::operator=(static_cast<const BaseT &>(RHS));
ST = RHS.ST;
TLI = RHS.TLI;
return *this;
}
PPCTTIImpl &operator=(PPCTTIImpl &&RHS) {
BaseT::operator=(std::move(static_cast<BaseT &>(RHS)));
ST = std::move(RHS.ST);
TLI = std::move(RHS.TLI);
return *this;
} }
/// \name Scalar TTI Implementations /// \name Scalar TTI Implementations
/// @{ /// @{
unsigned getIntImmCost(const APInt &Imm, Type *Ty) const override;
using BaseT::getIntImmCost;
unsigned getIntImmCost(const APInt &Imm, Type *Ty);
unsigned getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, unsigned getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm,
Type *Ty) const override; Type *Ty);
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm, unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty) const override; Type *Ty);
PopcntSupportKind getPopcntSupport(unsigned TyWidth) const override; TTI::PopcntSupportKind getPopcntSupport(unsigned TyWidth);
void getUnrollingPreferences(const Function *F, Loop *L, void getUnrollingPreferences(const Function *F, Loop *L,
UnrollingPreferences &UP) const override; TTI::UnrollingPreferences &UP);
/// @} /// @}
/// \name Vector TTI Implementations /// \name Vector TTI Implementations
/// @{ /// @{
unsigned getNumberOfRegisters(bool Vector) const override; unsigned getNumberOfRegisters(bool Vector);
unsigned getRegisterBitWidth(bool Vector) const override; unsigned getRegisterBitWidth(bool Vector);
unsigned getMaxInterleaveFactor() const override; unsigned getMaxInterleaveFactor();
unsigned getArithmeticInstrCost(unsigned Opcode, Type *Ty, OperandValueKind, unsigned getArithmeticInstrCost(
OperandValueKind, OperandValueProperties, unsigned Opcode, Type *Ty,
OperandValueProperties) const override; TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue,
unsigned getShuffleCost(ShuffleKind Kind, Type *Tp, TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue,
int Index, Type *SubTp) const override; TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None,
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None);
Type *Src) const override; unsigned getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index,
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *SubTp);
Type *CondTy) const override; unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src);
unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy);
unsigned Index) const override; unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index);
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) const override; unsigned AddressSpace);
/// @} /// @}
}; };
} // end anonymous namespace } // end anonymous namespace
INITIALIZE_AG_PASS(PPCTTI, TargetTransformInfo, "ppctti",
"PPC Target Transform Info", true, true, false)
char PPCTTI::ID = 0;
ImmutablePass * ImmutablePass *
llvm::createPPCTargetTransformInfoPass(const PPCTargetMachine *TM) { llvm::createPPCTargetTransformInfoPass(const PPCTargetMachine *TM) {
return new PPCTTI(TM); return new TargetTransformInfoWrapperPass(PPCTTIImpl(TM));
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// PPC cost model. // PPC cost model.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
PPCTTI::PopcntSupportKind PPCTTI::getPopcntSupport(unsigned TyWidth) const { TargetTransformInfo::PopcntSupportKind
PPCTTIImpl::getPopcntSupport(unsigned TyWidth) {
assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2"); assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2");
if (ST->hasPOPCNTD() && TyWidth <= 64) if (ST->hasPOPCNTD() && TyWidth <= 64)
return PSK_FastHardware; return TTI::PSK_FastHardware;
return PSK_Software; return TTI::PSK_Software;
} }
unsigned PPCTTI::getIntImmCost(const APInt &Imm, Type *Ty) const { unsigned PPCTTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) {
if (DisablePPCConstHoist) if (DisablePPCConstHoist)
return TargetTransformInfo::getIntImmCost(Imm, Ty); return BaseT::getIntImmCost(Imm, Ty);
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
if (BitSize == 0) if (BitSize == 0)
return ~0U; return ~0U;
if (Imm == 0) if (Imm == 0)
return TCC_Free; return TTI::TCC_Free;
if (Imm.getBitWidth() <= 64) { if (Imm.getBitWidth() <= 64) {
if (isInt<16>(Imm.getSExtValue())) if (isInt<16>(Imm.getSExtValue()))
return TCC_Basic; return TTI::TCC_Basic;
if (isInt<32>(Imm.getSExtValue())) { if (isInt<32>(Imm.getSExtValue())) {
// A constant that can be materialized using lis. // A constant that can be materialized using lis.
if ((Imm.getZExtValue() & 0xFFFF) == 0) if ((Imm.getZExtValue() & 0xFFFF) == 0)
return TCC_Basic; return TTI::TCC_Basic;
return 2 * TCC_Basic; return 2 * TTI::TCC_Basic;
} }
} }
return 4 * TCC_Basic; return 4 * TTI::TCC_Basic;
} }
unsigned PPCTTI::getIntImmCost(Intrinsic::ID IID, unsigned Idx, unsigned PPCTTIImpl::getIntImmCost(Intrinsic::ID IID, unsigned Idx,
const APInt &Imm, Type *Ty) const { const APInt &Imm, Type *Ty) {
if (DisablePPCConstHoist) if (DisablePPCConstHoist)
return TargetTransformInfo::getIntImmCost(IID, Idx, Imm, Ty); return BaseT::getIntImmCost(IID, Idx, Imm, Ty);
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
if (BitSize == 0) if (BitSize == 0)
return ~0U; return ~0U;
switch (IID) { switch (IID) {
default: return TCC_Free; default:
return TTI::TCC_Free;
case Intrinsic::sadd_with_overflow: case Intrinsic::sadd_with_overflow:
case Intrinsic::uadd_with_overflow: case Intrinsic::uadd_with_overflow:
case Intrinsic::ssub_with_overflow: case Intrinsic::ssub_with_overflow:
case Intrinsic::usub_with_overflow: case Intrinsic::usub_with_overflow:
if ((Idx == 1) && Imm.getBitWidth() <= 64 && isInt<16>(Imm.getSExtValue())) if ((Idx == 1) && Imm.getBitWidth() <= 64 && isInt<16>(Imm.getSExtValue()))
return TCC_Free; return TTI::TCC_Free;
break; break;
case Intrinsic::experimental_stackmap: case Intrinsic::experimental_stackmap:
if ((Idx < 2) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue()))) if ((Idx < 2) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue())))
return TCC_Free; return TTI::TCC_Free;
break; break;
case Intrinsic::experimental_patchpoint_void: case Intrinsic::experimental_patchpoint_void:
case Intrinsic::experimental_patchpoint_i64: case Intrinsic::experimental_patchpoint_i64:
if ((Idx < 4) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue()))) if ((Idx < 4) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue())))
return TCC_Free; return TTI::TCC_Free;
break; break;
} }
return PPCTTI::getIntImmCost(Imm, Ty); return PPCTTIImpl::getIntImmCost(Imm, Ty);
} }
unsigned PPCTTI::getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, unsigned PPCTTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx,
Type *Ty) const { const APInt &Imm, Type *Ty) {
if (DisablePPCConstHoist) if (DisablePPCConstHoist)
return TargetTransformInfo::getIntImmCost(Opcode, Idx, Imm, Ty); return BaseT::getIntImmCost(Opcode, Idx, Imm, Ty);
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
if (BitSize == 0) if (BitSize == 0)
return ~0U; return ~0U;
unsigned ImmIdx = ~0U; unsigned ImmIdx = ~0U;
bool ShiftedFree = false, RunFree = false, UnsignedFree = false, bool ShiftedFree = false, RunFree = false, UnsignedFree = false,
ZeroFree = false; ZeroFree = false;
switch (Opcode) { switch (Opcode) {
default: return TCC_Free; default:
return TTI::TCC_Free;
case Instruction::GetElementPtr: case Instruction::GetElementPtr:
// Always hoist the base address of a GetElementPtr. This prevents the // Always hoist the base address of a GetElementPtr. This prevents the
// creation of new constants for every base constant that gets constant // creation of new constants for every base constant that gets constant
// folded with the offset. // folded with the offset.
if (Idx == 0) if (Idx == 0)
return 2 * TCC_Basic; return 2 * TTI::TCC_Basic;
return TCC_Free; return TTI::TCC_Free;
case Instruction::And: case Instruction::And:
RunFree = true; // (for the rotate-and-mask instructions) RunFree = true; // (for the rotate-and-mask instructions)
// Fallthrough... // Fallthrough...
case Instruction::Add: case Instruction::Add:
case Instruction::Or: case Instruction::Or:
case Instruction::Xor: case Instruction::Xor:
ShiftedFree = true; ShiftedFree = true;
// Fallthrough... // Fallthrough...
Show All 15 Linesunsigned PPCTTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx,
case Instruction::Call: case Instruction::Call:
case Instruction::Ret: case Instruction::Ret:
case Instruction::Load: case Instruction::Load:
case Instruction::Store: case Instruction::Store:
break; break;
} }
if (ZeroFree && Imm == 0) if (ZeroFree && Imm == 0)
return TCC_Free; return TTI::TCC_Free;
if (Idx == ImmIdx && Imm.getBitWidth() <= 64) { if (Idx == ImmIdx && Imm.getBitWidth() <= 64) {
if (isInt<16>(Imm.getSExtValue())) if (isInt<16>(Imm.getSExtValue()))
return TCC_Free; return TTI::TCC_Free;
if (RunFree) { if (RunFree) {
if (Imm.getBitWidth() <= 32 && if (Imm.getBitWidth() <= 32 &&
(isShiftedMask_32(Imm.getZExtValue()) || (isShiftedMask_32(Imm.getZExtValue()) ||
isShiftedMask_32(~Imm.getZExtValue()))) isShiftedMask_32(~Imm.getZExtValue())))
return TCC_Free; return TTI::TCC_Free;
if (ST->isPPC64() && if (ST->isPPC64() &&
(isShiftedMask_64(Imm.getZExtValue()) || (isShiftedMask_64(Imm.getZExtValue()) ||
isShiftedMask_64(~Imm.getZExtValue()))) isShiftedMask_64(~Imm.getZExtValue())))
return TCC_Free; return TTI::TCC_Free;
} }
if (UnsignedFree && isUInt<16>(Imm.getZExtValue())) if (UnsignedFree && isUInt<16>(Imm.getZExtValue()))
return TCC_Free; return TTI::TCC_Free;
if (ShiftedFree && (Imm.getZExtValue() & 0xFFFF) == 0) if (ShiftedFree && (Imm.getZExtValue() & 0xFFFF) == 0)
return TCC_Free; return TTI::TCC_Free;
} }
return PPCTTI::getIntImmCost(Imm, Ty); return PPCTTIImpl::getIntImmCost(Imm, Ty);
} }
void PPCTTI::getUnrollingPreferences(const Function *F, Loop *L, void PPCTTIImpl::getUnrollingPreferences(const Function *F, Loop *L,
UnrollingPreferences &UP) const { TTI::UnrollingPreferences &UP) {
if (TM->getSubtarget<PPCSubtarget>(F).getDarwinDirective() == PPC::DIR_A2) { if (TM->getSubtarget<PPCSubtarget>(F).getDarwinDirective() == PPC::DIR_A2) {
// The A2 is in-order with a deep pipeline, and concatenation unrolling // The A2 is in-order with a deep pipeline, and concatenation unrolling
// helps expose latency-hiding opportunities to the instruction scheduler. // helps expose latency-hiding opportunities to the instruction scheduler.
UP.Partial = UP.Runtime = true; UP.Partial = UP.Runtime = true;
} }
TargetTransformInfo::getUnrollingPreferences(F, L, UP); BaseT::getUnrollingPreferences(F, L, UP);
} }
unsigned PPCTTI::getNumberOfRegisters(bool Vector) const { unsigned PPCTTIImpl::getNumberOfRegisters(bool Vector) {
if (Vector && !ST->hasAltivec()) if (Vector && !ST->hasAltivec())
return 0; return 0;
return ST->hasVSX() ? 64 : 32; return ST->hasVSX() ? 64 : 32;
} }
unsigned PPCTTI::getRegisterBitWidth(bool Vector) const { unsigned PPCTTIImpl::getRegisterBitWidth(bool Vector) {
if (Vector) { if (Vector) {
if (ST->hasAltivec()) return 128; if (ST->hasAltivec()) return 128;
return 0; return 0;
} }
if (ST->isPPC64()) if (ST->isPPC64())
return 64; return 64;
return 32; return 32;
} }
unsigned PPCTTI::getMaxInterleaveFactor() const { unsigned PPCTTIImpl::getMaxInterleaveFactor() {
unsigned Directive = ST->getDarwinDirective(); unsigned Directive = ST->getDarwinDirective();
// The 440 has no SIMD support, but floating-point instructions // The 440 has no SIMD support, but floating-point instructions
// have a 5-cycle latency, so unroll by 5x for latency hiding. // have a 5-cycle latency, so unroll by 5x for latency hiding.
if (Directive == PPC::DIR_440) if (Directive == PPC::DIR_440)
return 5; return 5;
// The A2 has no SIMD support, but floating-point instructions // The A2 has no SIMD support, but floating-point instructions
// have a 6-cycle latency, so unroll by 6x for latency hiding. // have a 6-cycle latency, so unroll by 6x for latency hiding.
if (Directive == PPC::DIR_A2) if (Directive == PPC::DIR_A2)
return 6; return 6;
// FIXME: For lack of any better information, do no harm... // FIXME: For lack of any better information, do no harm...
if (Directive == PPC::DIR_E500mc || Directive == PPC::DIR_E5500) if (Directive == PPC::DIR_E500mc || Directive == PPC::DIR_E5500)
return 1; return 1;
// For most things, modern systems have two execution units (and // For most things, modern systems have two execution units (and
// out-of-order execution). // out-of-order execution).
return 2; return 2;
} }
unsigned PPCTTI::getArithmeticInstrCost( unsigned PPCTTIImpl::getArithmeticInstrCost(
unsigned Opcode, Type *Ty, OperandValueKind Op1Info, unsigned Opcode, Type *Ty, TTI::OperandValueKind Op1Info,
OperandValueKind Op2Info, OperandValueProperties Opd1PropInfo, TTI::OperandValueKind Op2Info, TTI::OperandValueProperties Opd1PropInfo,
OperandValueProperties Opd2PropInfo) const { TTI::OperandValueProperties Opd2PropInfo) {
assert(TLI->InstructionOpcodeToISD(Opcode) && "Invalid opcode"); assert(TLI->InstructionOpcodeToISD(Opcode) && "Invalid opcode");
// Fallback to the default implementation. // Fallback to the default implementation.
return TargetTransformInfo::getArithmeticInstrCost( return BaseT::getArithmeticInstrCost(Opcode, Ty, Op1Info, Op2Info,
Opcode, Ty, Op1Info, Op2Info, Opd1PropInfo, Opd2PropInfo); Opd1PropInfo, Opd2PropInfo);
} }
unsigned PPCTTI::getShuffleCost(ShuffleKind Kind, Type *Tp, int Index, unsigned PPCTTIImpl::getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index,
Type *SubTp) const { Type *SubTp) {
return TargetTransformInfo::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
} }
unsigned PPCTTI::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) const { unsigned PPCTTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) {
assert(TLI->InstructionOpcodeToISD(Opcode) && "Invalid opcode"); assert(TLI->InstructionOpcodeToISD(Opcode) && "Invalid opcode");
return TargetTransformInfo::getCastInstrCost(Opcode, Dst, Src); return BaseT::getCastInstrCost(Opcode, Dst, Src);
} }
unsigned PPCTTI::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, unsigned PPCTTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy) const { Type *CondTy) {
return TargetTransformInfo::getCmpSelInstrCost(Opcode, ValTy, CondTy); return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy);
} }
unsigned PPCTTI::getVectorInstrCost(unsigned Opcode, Type *Val, unsigned PPCTTIImpl::getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index) const { unsigned Index) {
assert(Val->isVectorTy() && "This must be a vector type"); assert(Val->isVectorTy() && "This must be a vector type");
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
if (ST->hasVSX() && Val->getScalarType()->isDoubleTy()) { if (ST->hasVSX() && Val->getScalarType()->isDoubleTy()) {
// Double-precision scalars are already located in index #0. // Double-precision scalars are already located in index #0.
if (Index == 0) if (Index == 0)
return 0; return 0;
return TargetTransformInfo::getVectorInstrCost(Opcode, Val, Index); return BaseT::getVectorInstrCost(Opcode, Val, Index);
} }
// Estimated cost of a load-hit-store delay. This was obtained // Estimated cost of a load-hit-store delay. This was obtained
// experimentally as a minimum needed to prevent unprofitable // experimentally as a minimum needed to prevent unprofitable
// vectorization for the paq8p benchmark. It may need to be // vectorization for the paq8p benchmark. It may need to be
// raised further if other unprofitable cases remain. // raised further if other unprofitable cases remain.
unsigned LHSPenalty = 2; unsigned LHSPenalty = 2;
if (ISD == ISD::INSERT_VECTOR_ELT) if (ISD == ISD::INSERT_VECTOR_ELT)
LHSPenalty += 7; LHSPenalty += 7;
// Vector element insert/extract with Altivec is very expensive, // Vector element insert/extract with Altivec is very expensive,
// because they require store and reload with the attendant // because they require store and reload with the attendant
// processor stall for load-hit-store. Until VSX is available, // processor stall for load-hit-store. Until VSX is available,
// these need to be estimated as very costly. // these need to be estimated as very costly.
if (ISD == ISD::EXTRACT_VECTOR_ELT || if (ISD == ISD::EXTRACT_VECTOR_ELT ||
ISD == ISD::INSERT_VECTOR_ELT) ISD == ISD::INSERT_VECTOR_ELT)
return LHSPenalty + return LHSPenalty + BaseT::getVectorInstrCost(Opcode, Val, Index);
TargetTransformInfo::getVectorInstrCost(Opcode, Val, Index);
return TargetTransformInfo::getVectorInstrCost(Opcode, Val, Index); return BaseT::getVectorInstrCost(Opcode, Val, Index);
} }
unsigned PPCTTI::getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, unsigned PPCTTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
unsigned AddressSpace) const { unsigned Alignment,
unsigned AddressSpace) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Src); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Src);
assert((Opcode == Instruction::Load || Opcode == Instruction::Store) && assert((Opcode == Instruction::Load || Opcode == Instruction::Store) &&
"Invalid Opcode"); "Invalid Opcode");
unsigned Cost = unsigned Cost = BaseT::getMemoryOpCost(Opcode, Src, Alignment, AddressSpace);
TargetTransformInfo::getMemoryOpCost(Opcode, Src, Alignment, AddressSpace);
// VSX loads/stores support unaligned access. // VSX loads/stores support unaligned access.
if (ST->hasVSX()) { if (ST->hasVSX()) {
if (LT.second == MVT::v2f64 || LT.second == MVT::v2i64) if (LT.second == MVT::v2f64 || LT.second == MVT::v2i64)
return Cost; return Cost;
} }
bool UnalignedAltivec = bool UnalignedAltivec =
Show All 22 Lines

llvm/trunk/lib/Target/R600/AMDGPUTargetMachine.cpp

Show First 20 Lines • Show All 114 Lines • ▼ Show 20 LinesTargetPassConfig *AMDGPUTargetMachine::createPassConfig(PassManagerBase &PM) {
return new AMDGPUPassConfig(this, PM); return new AMDGPUPassConfig(this, PM);
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// AMDGPU Analysis Pass Setup // AMDGPU Analysis Pass Setup
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
void AMDGPUTargetMachine::addAnalysisPasses(PassManagerBase &PM) { void AMDGPUTargetMachine::addAnalysisPasses(PassManagerBase &PM) {
// Add first the target-independent BasicTTI pass, then our AMDGPU pass. This
// allows the AMDGPU pass to delegate to the target independent layer when
// appropriate.
PM.add(createBasicTargetTransformInfoPass(this));
PM.add(createAMDGPUTargetTransformInfoPass(this)); PM.add(createAMDGPUTargetTransformInfoPass(this));
} }
void AMDGPUPassConfig::addIRPasses() { void AMDGPUPassConfig::addIRPasses() {
// Function calls are not supported, so make sure we inline everything. // Function calls are not supported, so make sure we inline everything.
addPass(createAMDGPUAlwaysInlinePass()); addPass(createAMDGPUAlwaysInlinePass());
addPass(createAlwaysInlinerPass()); addPass(createAlwaysInlinerPass());
// We need to add the barrier noop pass, otherwise adding the function // We need to add the barrier noop pass, otherwise adding the function
▲ Show 20 Lines • Show All 114 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/R600/AMDGPUTargetTransformInfo.cpp

Show All 14 Lines
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "AMDGPU.h" #include "AMDGPU.h"
#include "AMDGPUTargetMachine.h" #include "AMDGPUTargetMachine.h"
#include "llvm/Analysis/LoopInfo.h" #include "llvm/Analysis/LoopInfo.h"
#include "llvm/Analysis/TargetTransformInfo.h" #include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/Analysis/ValueTracking.h" #include "llvm/Analysis/ValueTracking.h"
#include "llvm/CodeGen/BasicTTIImpl.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Target/CostTable.h" #include "llvm/Target/CostTable.h"
#include "llvm/Target/TargetLowering.h" #include "llvm/Target/TargetLowering.h"
using namespace llvm; using namespace llvm;
#define DEBUG_TYPE "AMDGPUtti" #define DEBUG_TYPE "AMDGPUtti"
// Declare the pass initialization routine locally as target-specific passes
// don't have a target-wide initialization entry point, and so we rely on the
// pass constructor initialization.
namespace llvm {
void initializeAMDGPUTTIPass(PassRegistry &);
}
namespace { namespace {
class AMDGPUTTI final : public ImmutablePass, public TargetTransformInfo { class AMDGPUTTIImpl : public BasicTTIImplBase<AMDGPUTTIImpl> {
const AMDGPUTargetMachine *TM; typedef BasicTTIImplBase<AMDGPUTTIImpl> BaseT;
const AMDGPUSubtarget *ST; typedef TargetTransformInfo TTI;
const AMDGPUTargetLowering *TLI;
/// Estimate the overhead of scalarizing an instruction. Insert and Extract const AMDGPUSubtarget *ST;
/// are set if the result needs to be inserted and/or extracted from vectors.
unsigned getScalarizationOverhead(Type *Ty, bool Insert, bool Extract) const;
public: public:
AMDGPUTTI() : ImmutablePass(ID), TM(nullptr), ST(nullptr), TLI(nullptr) { explicit AMDGPUTTIImpl(const AMDGPUTargetMachine *TM = nullptr)
llvm_unreachable("This pass cannot be directly constructed"); : BaseT(TM), ST(TM->getSubtargetImpl()) {}
}
AMDGPUTTI(const AMDGPUTargetMachine *TM) // Provide value semantics. MSVC requires that we spell all of these out.
: ImmutablePass(ID), TM(TM), ST(TM->getSubtargetImpl()), AMDGPUTTIImpl(const AMDGPUTTIImpl &Arg)
TLI(TM->getSubtargetImpl()->getTargetLowering()) { : BaseT(static_cast<const BaseT &>(Arg)), ST(Arg.ST) {}
initializeAMDGPUTTIPass(*PassRegistry::getPassRegistry()); AMDGPUTTIImpl(AMDGPUTTIImpl &&Arg)
: BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)) {}
AMDGPUTTIImpl &operator=(const AMDGPUTTIImpl &RHS) {
BaseT::operator=(static_cast<const BaseT &>(RHS));
ST = RHS.ST;
return *this;
}
AMDGPUTTIImpl &operator=(AMDGPUTTIImpl &&RHS) {
BaseT::operator=(std::move(static_cast<BaseT &>(RHS)));
ST = std::move(RHS.ST);
return *this;
} }
void initializePass() override { pushTTIStack(this); } bool hasBranchDivergence() { return true; }
void getAnalysisUsage(AnalysisUsage &AU) const override {
TargetTransformInfo::getAnalysisUsage(AU);
}
/// Pass identification.
static char ID;
/// Provide necessary pointer adjustments for the two base classes.
void *getAdjustedAnalysisPointer(const void *ID) override {
if (ID == &TargetTransformInfo::ID)
return (TargetTransformInfo *)this;
return this;
}
bool hasBranchDivergence() const override;
void getUnrollingPreferences(const Function *F, Loop *L, void getUnrollingPreferences(const Function *F, Loop *L,
UnrollingPreferences &UP) const override; TTI::UnrollingPreferences &UP);
PopcntSupportKind getPopcntSupport(unsigned IntTyWidthInBit) const override; TTI::PopcntSupportKind getPopcntSupport(unsigned TyWidth) {
assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2");
return ST->hasBCNT(TyWidth) ? TTI::PSK_FastHardware : TTI::PSK_Software;
}
unsigned getNumberOfRegisters(bool Vector) const override; unsigned getNumberOfRegisters(bool Vector);
unsigned getRegisterBitWidth(bool Vector) const override; unsigned getRegisterBitWidth(bool Vector);
unsigned getMaxInterleaveFactor() const override; unsigned getMaxInterleaveFactor();
}; };
} // end anonymous namespace } // end anonymous namespace
INITIALIZE_AG_PASS(AMDGPUTTI, TargetTransformInfo, "AMDGPUtti",
"AMDGPU Target Transform Info", true, true, false)
char AMDGPUTTI::ID = 0;
ImmutablePass * ImmutablePass *
llvm::createAMDGPUTargetTransformInfoPass(const AMDGPUTargetMachine *TM) { llvm::createAMDGPUTargetTransformInfoPass(const AMDGPUTargetMachine *TM) {
return new AMDGPUTTI(TM); return new TargetTransformInfoWrapperPass(AMDGPUTTIImpl(TM));
} }
bool AMDGPUTTI::hasBranchDivergence() const { return true; } void AMDGPUTTIImpl::getUnrollingPreferences(const Function *, Loop *L,
TTI::UnrollingPreferences &UP) {
void AMDGPUTTI::getUnrollingPreferences(const Function *, Loop *L,
UnrollingPreferences &UP) const {
UP.Threshold = 300; // Twice the default. UP.Threshold = 300; // Twice the default.
UP.Count = UINT_MAX; UP.Count = UINT_MAX;
UP.Partial = true; UP.Partial = true;
// TODO: Do we want runtime unrolling? // TODO: Do we want runtime unrolling?
for (const BasicBlock *BB : L->getBlocks()) { for (const BasicBlock *BB : L->getBlocks()) {
for (const Instruction &I : *BB) { for (const Instruction &I : *BB) {
Show All 15 Lines for (const Instruction &I : *BB) {
// Don't use the maximum allowed value here as it will make some // Don't use the maximum allowed value here as it will make some
// programs way too big. // programs way too big.
UP.Threshold = 800; UP.Threshold = 800;
} }
} }
} }
} }
AMDGPUTTI::PopcntSupportKind unsigned AMDGPUTTIImpl::getNumberOfRegisters(bool Vec) {
AMDGPUTTI::getPopcntSupport(unsigned TyWidth) const {
assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2");
return ST->hasBCNT(TyWidth) ? PSK_FastHardware : PSK_Software;
}
unsigned AMDGPUTTI::getNumberOfRegisters(bool Vec) const {
if (Vec) if (Vec)
return 0; return 0;
// Number of VGPRs on SI. // Number of VGPRs on SI.
if (ST->getGeneration() >= AMDGPUSubtarget::SOUTHERN_ISLANDS) if (ST->getGeneration() >= AMDGPUSubtarget::SOUTHERN_ISLANDS)
return 256; return 256;
return 4 * 128; // XXX - 4 channels. Should these count as vector instead? return 4 * 128; // XXX - 4 channels. Should these count as vector instead?
} }
unsigned AMDGPUTTI::getRegisterBitWidth(bool) const { unsigned AMDGPUTTIImpl::getRegisterBitWidth(bool) { return 32; }
return 32;
}
unsigned AMDGPUTTI::getMaxInterleaveFactor() const { unsigned AMDGPUTTIImpl::getMaxInterleaveFactor() {
// Semi-arbitrary large amount. // Semi-arbitrary large amount.
return 64; return 64;
} }

llvm/trunk/lib/Target/Target.cpp

Show All 30 Lines
inline LLVMTargetLibraryInfoRef wrap(const TargetLibraryInfoImpl *P) { inline LLVMTargetLibraryInfoRef wrap(const TargetLibraryInfoImpl *P) {
TargetLibraryInfoImpl *X = const_cast<TargetLibraryInfoImpl*>(P); TargetLibraryInfoImpl *X = const_cast<TargetLibraryInfoImpl*>(P);
return reinterpret_cast<LLVMTargetLibraryInfoRef>(X); return reinterpret_cast<LLVMTargetLibraryInfoRef>(X);
} }
void llvm::initializeTarget(PassRegistry &Registry) { void llvm::initializeTarget(PassRegistry &Registry) {
initializeDataLayoutPassPass(Registry); initializeDataLayoutPassPass(Registry);
initializeTargetLibraryInfoWrapperPassPass(Registry); initializeTargetLibraryInfoWrapperPassPass(Registry);
initializeTargetTransformInfoWrapperPassPass(Registry);
} }
void LLVMInitializeTarget(LLVMPassRegistryRef R) { void LLVMInitializeTarget(LLVMPassRegistryRef R) {
initializeTarget(*unwrap(R)); initializeTarget(*unwrap(R));
} }
LLVMTargetDataRef LLVMCreateTargetData(const char *StringRep) { LLVMTargetDataRef LLVMCreateTargetData(const char *StringRep) {
return wrap(new DataLayout(StringRep)); return wrap(new DataLayout(StringRep));
▲ Show 20 Lines • Show All 90 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/TargetMachine.cpp

//===-- TargetMachine.cpp - General Target Information ---------------------==// //===-- TargetMachine.cpp - General Target Information ---------------------==//
// //
// The LLVM Compiler Infrastructure // The LLVM Compiler Infrastructure
// //
// This file is distributed under the University of Illinois Open Source // This file is distributed under the University of Illinois Open Source
// License. See LICENSE.TXT for details. // License. See LICENSE.TXT for details.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// This file describes the general parts of a Target machine. // This file describes the general parts of a Target machine.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "llvm/Target/TargetMachine.h" #include "llvm/Target/TargetMachine.h"
#include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/CodeGen/MachineFunction.h" #include "llvm/CodeGen/MachineFunction.h"
#include "llvm/IR/Function.h" #include "llvm/IR/Function.h"
#include "llvm/IR/GlobalAlias.h" #include "llvm/IR/GlobalAlias.h"
#include "llvm/IR/GlobalValue.h" #include "llvm/IR/GlobalValue.h"
#include "llvm/IR/GlobalVariable.h" #include "llvm/IR/GlobalVariable.h"
#include "llvm/IR/Mangler.h" #include "llvm/IR/Mangler.h"
#include "llvm/MC/MCAsmInfo.h" #include "llvm/MC/MCAsmInfo.h"
#include "llvm/MC/MCCodeGenInfo.h" #include "llvm/MC/MCCodeGenInfo.h"
#include "llvm/MC/MCContext.h" #include "llvm/MC/MCContext.h"
#include "llvm/MC/MCSectionMachO.h" #include "llvm/MC/MCSectionMachO.h"
#include "llvm/MC/MCTargetOptions.h" #include "llvm/MC/MCTargetOptions.h"
#include "llvm/MC/SectionKind.h" #include "llvm/MC/SectionKind.h"
#include "llvm/PassManager.h"
#include "llvm/Support/CommandLine.h" #include "llvm/Support/CommandLine.h"
#include "llvm/Target/TargetLowering.h" #include "llvm/Target/TargetLowering.h"
#include "llvm/Target/TargetLoweringObjectFile.h" #include "llvm/Target/TargetLoweringObjectFile.h"
#include "llvm/Target/TargetSubtargetInfo.h" #include "llvm/Target/TargetSubtargetInfo.h"
using namespace llvm; using namespace llvm;
//--------------------------------------------------------------------------- //---------------------------------------------------------------------------
// TargetMachine Class // TargetMachine Class
▲ Show 20 Lines • Show All 130 Lines • ▼ Show 20 Lines
void TargetMachine::setFunctionSections(bool V) { void TargetMachine::setFunctionSections(bool V) {
Options.FunctionSections = V; Options.FunctionSections = V;
} }
void TargetMachine::setDataSections(bool V) { void TargetMachine::setDataSections(bool V) {
Options.DataSections = V; Options.DataSections = V;
} }
void TargetMachine::addAnalysisPasses(PassManagerBase &PM) {
PM.add(createNoTargetTransformInfoPass(getDataLayout()));
}
static bool canUsePrivateLabel(const MCAsmInfo &AsmInfo, static bool canUsePrivateLabel(const MCAsmInfo &AsmInfo,
const MCSection &Section) { const MCSection &Section) {
if (!AsmInfo.isSectionAtomizableBySymbols(Section)) if (!AsmInfo.isSectionAtomizableBySymbols(Section))
return true; return true;
// If it is not dead stripped, it is safe to use private labels. // If it is not dead stripped, it is safe to use private labels.
const MCSectionMachO &SMO = cast<MCSectionMachO>(Section); const MCSectionMachO &SMO = cast<MCSectionMachO>(Section);
if (SMO.hasAttribute(MachO::S_ATTR_NO_DEAD_STRIP)) if (SMO.hasAttribute(MachO::S_ATTR_NO_DEAD_STRIP))
Show All 29 Lines

llvm/trunk/lib/Target/X86/X86TargetMachine.cpp

Show First 20 Lines • Show All 159 Lines • ▼ Show 20 LinesUseVZeroUpper("x86-use-vzeroupper", cl::Hidden,
cl::desc("Minimize AVX to SSE transition penalty"), cl::desc("Minimize AVX to SSE transition penalty"),
cl::init(true)); cl::init(true));
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// X86 Analysis Pass Setup // X86 Analysis Pass Setup
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
void X86TargetMachine::addAnalysisPasses(PassManagerBase &PM) { void X86TargetMachine::addAnalysisPasses(PassManagerBase &PM) {
// Add first the target-independent BasicTTI pass, then our X86 pass. This
// allows the X86 pass to delegate to the target independent layer when
// appropriate.
PM.add(createBasicTargetTransformInfoPass(this));
PM.add(createX86TargetTransformInfoPass(this)); PM.add(createX86TargetTransformInfoPass(this));
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// Pass Pipeline Configuration // Pass Pipeline Configuration
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
▲ Show 20 Lines • Show All 67 Lines • Show Last 20 Lines

llvm/trunk/lib/Target/X86/X86TargetTransformInfo.cpp

Show All 11 Lines
/// more precise answers to certain TTI queries, while letting the target /// more precise answers to certain TTI queries, while letting the target
/// independent and default TTI implementations handle the rest. /// independent and default TTI implementations handle the rest.
/// ///
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "X86.h" #include "X86.h"
#include "X86TargetMachine.h" #include "X86TargetMachine.h"
#include "llvm/Analysis/TargetTransformInfo.h" #include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/CodeGen/BasicTTIImpl.h"
#include "llvm/IR/IntrinsicInst.h" #include "llvm/IR/IntrinsicInst.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Target/CostTable.h" #include "llvm/Target/CostTable.h"
#include "llvm/Target/TargetLowering.h" #include "llvm/Target/TargetLowering.h"
using namespace llvm; using namespace llvm;
#define DEBUG_TYPE "x86tti" #define DEBUG_TYPE "x86tti"
// Declare the pass initialization routine locally as target-specific passes
// don't have a target-wide initialization entry point, and so we rely on the
// pass constructor initialization.
namespace llvm {
void initializeX86TTIPass(PassRegistry &);
}
namespace { namespace {
class X86TTI final : public ImmutablePass, public TargetTransformInfo { class X86TTIImpl : public BasicTTIImplBase<X86TTIImpl> {
typedef BasicTTIImplBase<X86TTIImpl> BaseT;
typedef TargetTransformInfo TTI;
const X86Subtarget *ST; const X86Subtarget *ST;
const X86TargetLowering *TLI; const X86TargetLowering *TLI;
/// Estimate the overhead of scalarizing an instruction. Insert and Extract unsigned getScalarizationOverhead(Type *Ty, bool Insert, bool Extract);
/// are set if the result needs to be inserted and/or extracted from vectors.
unsigned getScalarizationOverhead(Type *Ty, bool Insert, bool Extract) const;
public: public:
X86TTI() : ImmutablePass(ID), ST(nullptr), TLI(nullptr) { explicit X86TTIImpl(const X86TargetMachine *TM = nullptr)
llvm_unreachable("This pass cannot be directly constructed"); : BaseT(TM), ST(TM ? TM->getSubtargetImpl() : nullptr),
} TLI(ST ? ST->getTargetLowering() : nullptr) {}
X86TTI(const X86TargetMachine *TM) // Provide value semantics. MSVC requires that we spell all of these out.
: ImmutablePass(ID), ST(TM->getSubtargetImpl()), X86TTIImpl(const X86TTIImpl &Arg)
TLI(TM->getSubtargetImpl()->getTargetLowering()) { : BaseT(static_cast<const BaseT &>(Arg)), ST(Arg.ST), TLI(Arg.TLI) {}
initializeX86TTIPass(*PassRegistry::getPassRegistry()); X86TTIImpl(X86TTIImpl &&Arg)
} : BaseT(std::move(static_cast<BaseT &>(Arg))), ST(std::move(Arg.ST)),
TLI(std::move(Arg.TLI)) {}
void initializePass() override { X86TTIImpl &operator=(const X86TTIImpl &RHS) {
pushTTIStack(this); BaseT::operator=(static_cast<const BaseT &>(RHS));
} ST = RHS.ST;
TLI = RHS.TLI;
void getAnalysisUsage(AnalysisUsage &AU) const override { return *this;
TargetTransformInfo::getAnalysisUsage(AU); }
} X86TTIImpl &operator=(X86TTIImpl &&RHS) {
BaseT::operator=(std::move(static_cast<BaseT &>(RHS)));
/// Pass identification. ST = std::move(RHS.ST);
static char ID; TLI = std::move(RHS.TLI);
return *this;
/// Provide necessary pointer adjustments for the two base classes.
void *getAdjustedAnalysisPointer(const void *ID) override {
if (ID == &TargetTransformInfo::ID)
return (TargetTransformInfo*)this;
return this;
} }
/// \name Scalar TTI Implementations /// \name Scalar TTI Implementations
/// @{ /// @{
PopcntSupportKind getPopcntSupport(unsigned TyWidth) const override; TTI::PopcntSupportKind getPopcntSupport(unsigned TyWidth);
/// @} /// @}
/// \name Vector TTI Implementations /// \name Vector TTI Implementations
/// @{ /// @{
unsigned getNumberOfRegisters(bool Vector) const override; unsigned getNumberOfRegisters(bool Vector);
unsigned getRegisterBitWidth(bool Vector) const override; unsigned getRegisterBitWidth(bool Vector);
unsigned getMaxInterleaveFactor() const override; unsigned getMaxInterleaveFactor();
unsigned getArithmeticInstrCost(unsigned Opcode, Type *Ty, OperandValueKind, unsigned getArithmeticInstrCost(
OperandValueKind, OperandValueProperties, unsigned Opcode, Type *Ty,
OperandValueProperties) const override; TTI::OperandValueKind Opd1Info = TTI::OK_AnyValue,
unsigned getShuffleCost(ShuffleKind Kind, Type *Tp, TTI::OperandValueKind Opd2Info = TTI::OK_AnyValue,
int Index, Type *SubTp) const override; TTI::OperandValueProperties Opd1PropInfo = TTI::OP_None,
unsigned getCastInstrCost(unsigned Opcode, Type *Dst, TTI::OperandValueProperties Opd2PropInfo = TTI::OP_None);
Type *Src) const override; unsigned getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index,
unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *SubTp);
Type *CondTy) const override; unsigned getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src);
unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy);
unsigned Index) const override; unsigned getVectorInstrCost(unsigned Opcode, Type *Val, unsigned Index);
unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, unsigned getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned AddressSpace) const override; unsigned AddressSpace);
unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned getMaskedMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
unsigned Alignment, unsigned AddressSpace);
unsigned AddressSpace) const override;
unsigned getAddressComputationCost(Type *PtrTy, unsigned getAddressComputationCost(Type *PtrTy, bool IsComplex);
bool IsComplex) const override;
unsigned getReductionCost(unsigned Opcode, Type *Ty, unsigned getReductionCost(unsigned Opcode, Type *Ty, bool IsPairwiseForm);
bool IsPairwiseForm) const override;
unsigned getIntImmCost(int64_t) const; unsigned getIntImmCost(int64_t);
unsigned getIntImmCost(const APInt &Imm, Type *Ty) const override; unsigned getIntImmCost(const APInt &Imm, Type *Ty);
unsigned getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, unsigned getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm,
Type *Ty) const override; Type *Ty);
unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm, unsigned getIntImmCost(Intrinsic::ID IID, unsigned Idx, const APInt &Imm,
Type *Ty) const override; Type *Ty);
bool isLegalMaskedLoad (Type *DataType, int Consecutive) const override; bool isLegalMaskedLoad(Type *DataType, int Consecutive);
bool isLegalMaskedStore(Type *DataType, int Consecutive) const override; bool isLegalMaskedStore(Type *DataType, int Consecutive);
/// @} /// @}
}; };
} // end anonymous namespace } // end anonymous namespace
INITIALIZE_AG_PASS(X86TTI, TargetTransformInfo, "x86tti",
"X86 Target Transform Info", true, true, false)
char X86TTI::ID = 0;
ImmutablePass * ImmutablePass *
llvm::createX86TargetTransformInfoPass(const X86TargetMachine *TM) { llvm::createX86TargetTransformInfoPass(const X86TargetMachine *TM) {
return new X86TTI(TM); return new TargetTransformInfoWrapperPass(X86TTIImpl(TM));
} }
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
// //
// X86 cost model. // X86 cost model.
// //
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
X86TTI::PopcntSupportKind X86TTI::getPopcntSupport(unsigned TyWidth) const { TargetTransformInfo::PopcntSupportKind
X86TTIImpl::getPopcntSupport(unsigned TyWidth) {
assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2"); assert(isPowerOf2_32(TyWidth) && "Ty width must be power of 2");
// TODO: Currently the __builtin_popcount() implementation using SSE3 // TODO: Currently the __builtin_popcount() implementation using SSE3
// instructions is inefficient. Once the problem is fixed, we should // instructions is inefficient. Once the problem is fixed, we should
// call ST->hasSSE3() instead of ST->hasPOPCNT(). // call ST->hasSSE3() instead of ST->hasPOPCNT().
return ST->hasPOPCNT() ? PSK_FastHardware : PSK_Software; return ST->hasPOPCNT() ? TTI::PSK_FastHardware : TTI::PSK_Software;
} }
unsigned X86TTI::getNumberOfRegisters(bool Vector) const { unsigned X86TTIImpl::getNumberOfRegisters(bool Vector) {
if (Vector && !ST->hasSSE1()) if (Vector && !ST->hasSSE1())
return 0; return 0;
if (ST->is64Bit()) { if (ST->is64Bit()) {
if (Vector && ST->hasAVX512()) if (Vector && ST->hasAVX512())
return 32; return 32;
return 16; return 16;
} }
return 8; return 8;
} }
unsigned X86TTI::getRegisterBitWidth(bool Vector) const { unsigned X86TTIImpl::getRegisterBitWidth(bool Vector) {
if (Vector) { if (Vector) {
if (ST->hasAVX512()) return 512; if (ST->hasAVX512()) return 512;
if (ST->hasAVX()) return 256; if (ST->hasAVX()) return 256;
if (ST->hasSSE1()) return 128; if (ST->hasSSE1()) return 128;
return 0; return 0;
} }
if (ST->is64Bit()) if (ST->is64Bit())
return 64; return 64;
return 32; return 32;
} }
unsigned X86TTI::getMaxInterleaveFactor() const { unsigned X86TTIImpl::getMaxInterleaveFactor() {
if (ST->isAtom()) if (ST->isAtom())
return 1; return 1;
// Sandybridge and Haswell have multiple execution ports and pipelined // Sandybridge and Haswell have multiple execution ports and pipelined
// vector units. // vector units.
if (ST->hasAVX()) if (ST->hasAVX())
return 4; return 4;
return 2; return 2;
} }
unsigned X86TTI::getArithmeticInstrCost( unsigned X86TTIImpl::getArithmeticInstrCost(
unsigned Opcode, Type *Ty, OperandValueKind Op1Info, unsigned Opcode, Type *Ty, TTI::OperandValueKind Op1Info,
OperandValueKind Op2Info, OperandValueProperties Opd1PropInfo, TTI::OperandValueKind Op2Info, TTI::OperandValueProperties Opd1PropInfo,
OperandValueProperties Opd2PropInfo) const { TTI::OperandValueProperties Opd2PropInfo) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Ty); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Ty);
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
if (ISD == ISD::SDIV && if (ISD == ISD::SDIV &&
Op2Info == TargetTransformInfo::OK_UniformConstantValue && Op2Info == TargetTransformInfo::OK_UniformConstantValue &&
▲ Show 20 Lines • Show All 238 Lines • ▼ Show 20 Linesunsigned X86TTIImpl::getArithmeticInstrCost(
// Special lowering of v4i32 mul on sse2, sse3: Lower v4i32 mul as 2x shuffle, // Special lowering of v4i32 mul on sse2, sse3: Lower v4i32 mul as 2x shuffle,
// 2x pmuludq, 2x shuffle. // 2x pmuludq, 2x shuffle.
if (ISD == ISD::MUL && LT.second == MVT::v4i32 && ST->hasSSE2() && if (ISD == ISD::MUL && LT.second == MVT::v4i32 && ST->hasSSE2() &&
!ST->hasSSE41()) !ST->hasSSE41())
return LT.first * 6; return LT.first * 6;
// Fallback to the default implementation. // Fallback to the default implementation.
return TargetTransformInfo::getArithmeticInstrCost(Opcode, Ty, Op1Info, return BaseT::getArithmeticInstrCost(Opcode, Ty, Op1Info, Op2Info);
Op2Info);
} }
unsigned X86TTI::getShuffleCost(ShuffleKind Kind, Type *Tp, int Index, unsigned X86TTIImpl::getShuffleCost(TTI::ShuffleKind Kind, Type *Tp, int Index,
Type *SubTp) const { Type *SubTp) {
// We only estimate the cost of reverse and alternate shuffles. // We only estimate the cost of reverse and alternate shuffles.
if (Kind != SK_Reverse && Kind != SK_Alternate) if (Kind != TTI::SK_Reverse && Kind != TTI::SK_Alternate)
return TargetTransformInfo::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
if (Kind == SK_Reverse) { if (Kind == TTI::SK_Reverse) {
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Tp); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Tp);
unsigned Cost = 1; unsigned Cost = 1;
if (LT.second.getSizeInBits() > 128) if (LT.second.getSizeInBits() > 128)
Cost = 3; // Extract + insert + copy. Cost = 3; // Extract + insert + copy.
// Multiple by the number of parts. // Multiple by the number of parts.
return Cost * LT.first; return Cost * LT.first;
} }
if (Kind == SK_Alternate) { if (Kind == TTI::SK_Alternate) {
// 64-bit packed float vectors (v2f32) are widened to type v4f32. // 64-bit packed float vectors (v2f32) are widened to type v4f32.
// 64-bit packed integer vectors (v2i32) are promoted to type v2i64. // 64-bit packed integer vectors (v2i32) are promoted to type v2i64.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Tp); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Tp);
// The backend knows how to generate a single VEX.256 version of // The backend knows how to generate a single VEX.256 version of
// instruction VPBLENDW if the target supports AVX2. // instruction VPBLENDW if the target supports AVX2.
if (ST->hasAVX2() && LT.second == MVT::v16i16) if (ST->hasAVX2() && LT.second == MVT::v16i16)
return LT.first; return LT.first;
▲ Show 20 Lines • Show All 76 Lines • ▼ Show 20 Lines static const CostTblEntry<MVT::SimpleValueType> SSEAltShuffleTbl[] = {
// 8 x (pinsrw + pextrw + and + movb + movzb + or) // 8 x (pinsrw + pextrw + and + movb + movzb + or)
{ISD::VECTOR_SHUFFLE, MVT::v16i8, 48} {ISD::VECTOR_SHUFFLE, MVT::v16i8, 48}
}; };
// Fall-back (SSE3 and SSE2). // Fall-back (SSE3 and SSE2).
int Idx = CostTableLookup(SSEAltShuffleTbl, ISD::VECTOR_SHUFFLE, LT.second); int Idx = CostTableLookup(SSEAltShuffleTbl, ISD::VECTOR_SHUFFLE, LT.second);
if (Idx != -1) if (Idx != -1)
return LT.first * SSEAltShuffleTbl[Idx].Cost; return LT.first * SSEAltShuffleTbl[Idx].Cost;
return TargetTransformInfo::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
} }
return TargetTransformInfo::getShuffleCost(Kind, Tp, Index, SubTp); return BaseT::getShuffleCost(Kind, Tp, Index, SubTp);
} }
unsigned X86TTI::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) const { unsigned X86TTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) {
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
std::pair<unsigned, MVT> LTSrc = TLI->getTypeLegalizationCost(Src); std::pair<unsigned, MVT> LTSrc = TLI->getTypeLegalizationCost(Src);
std::pair<unsigned, MVT> LTDest = TLI->getTypeLegalizationCost(Dst); std::pair<unsigned, MVT> LTDest = TLI->getTypeLegalizationCost(Dst);
static const TypeConversionCostTblEntry<MVT::SimpleValueType> static const TypeConversionCostTblEntry<MVT::SimpleValueType>
SSE2ConvTbl[] = { SSE2ConvTbl[] = {
▲ Show 20 Lines • Show All 65 Lines • ▼ Show 20 Lines if (ST->hasAVX512()) {
if (Idx != -1) if (Idx != -1)
return AVX512ConversionTbl[Idx].Cost; return AVX512ConversionTbl[Idx].Cost;
} }
EVT SrcTy = TLI->getValueType(Src); EVT SrcTy = TLI->getValueType(Src);
EVT DstTy = TLI->getValueType(Dst); EVT DstTy = TLI->getValueType(Dst);
// The function getSimpleVT only handles simple value types. // The function getSimpleVT only handles simple value types.
if (!SrcTy.isSimple() || !DstTy.isSimple()) if (!SrcTy.isSimple() || !DstTy.isSimple())
return TargetTransformInfo::getCastInstrCost(Opcode, Dst, Src); return BaseT::getCastInstrCost(Opcode, Dst, Src);
static const TypeConversionCostTblEntry<MVT::SimpleValueType> static const TypeConversionCostTblEntry<MVT::SimpleValueType>
AVX2ConversionTbl[] = { AVX2ConversionTbl[] = {
{ ISD::SIGN_EXTEND, MVT::v16i16, MVT::v16i8, 1 }, { ISD::SIGN_EXTEND, MVT::v16i16, MVT::v16i8, 1 },
{ ISD::ZERO_EXTEND, MVT::v16i16, MVT::v16i8, 1 }, { ISD::ZERO_EXTEND, MVT::v16i16, MVT::v16i8, 1 },
{ ISD::SIGN_EXTEND, MVT::v8i32, MVT::v8i1, 3 }, { ISD::SIGN_EXTEND, MVT::v8i32, MVT::v8i1, 3 },
{ ISD::ZERO_EXTEND, MVT::v8i32, MVT::v8i1, 3 }, { ISD::ZERO_EXTEND, MVT::v8i32, MVT::v8i1, 3 },
{ ISD::SIGN_EXTEND, MVT::v8i32, MVT::v8i8, 3 }, { ISD::SIGN_EXTEND, MVT::v8i32, MVT::v8i8, 3 },
▲ Show 20 Lines • Show All 102 Lines • ▼ Show 20 Linesunsigned X86TTIImpl::getCastInstrCost(unsigned Opcode, Type *Dst, Type *Src) {
if (ST->hasAVX()) { if (ST->hasAVX()) {
int Idx = ConvertCostTableLookup(AVXConversionTbl, ISD, DstTy.getSimpleVT(), int Idx = ConvertCostTableLookup(AVXConversionTbl, ISD, DstTy.getSimpleVT(),
SrcTy.getSimpleVT()); SrcTy.getSimpleVT());
if (Idx != -1) if (Idx != -1)
return AVXConversionTbl[Idx].Cost; return AVXConversionTbl[Idx].Cost;
} }
return TargetTransformInfo::getCastInstrCost(Opcode, Dst, Src); return BaseT::getCastInstrCost(Opcode, Dst, Src);
} }
unsigned X86TTI::getCmpSelInstrCost(unsigned Opcode, Type *ValTy, unsigned X86TTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
Type *CondTy) const { Type *CondTy) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(ValTy); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(ValTy);
MVT MTy = LT.second; MVT MTy = LT.second;
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
▲ Show 20 Lines • Show All 49 Lines • ▼ Show 20 Linesunsigned X86TTIImpl::getCmpSelInstrCost(unsigned Opcode, Type *ValTy,
} }
if (ST->hasSSE42()) { if (ST->hasSSE42()) {
int Idx = CostTableLookup(SSE42CostTbl, ISD, MTy); int Idx = CostTableLookup(SSE42CostTbl, ISD, MTy);
if (Idx != -1) if (Idx != -1)
return LT.first * SSE42CostTbl[Idx].Cost; return LT.first * SSE42CostTbl[Idx].Cost;
} }
return TargetTransformInfo::getCmpSelInstrCost(Opcode, ValTy, CondTy); return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy);
} }
unsigned X86TTI::getVectorInstrCost(unsigned Opcode, Type *Val, unsigned X86TTIImpl::getVectorInstrCost(unsigned Opcode, Type *Val,
unsigned Index) const { unsigned Index) {
assert(Val->isVectorTy() && "This must be a vector type"); assert(Val->isVectorTy() && "This must be a vector type");
if (Index != -1U) { if (Index != -1U) {
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Val); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Val);
// This type is legalized to a scalar type. // This type is legalized to a scalar type.
if (!LT.second.isVector()) if (!LT.second.isVector())
return 0; return 0;
// The type may be split. Normalize the index to the new type. // The type may be split. Normalize the index to the new type.
unsigned Width = LT.second.getVectorNumElements(); unsigned Width = LT.second.getVectorNumElements();
Index = Index % Width; Index = Index % Width;
// Floating point scalars are already located in index #0. // Floating point scalars are already located in index #0.
if (Val->getScalarType()->isFloatingPointTy() && Index == 0) if (Val->getScalarType()->isFloatingPointTy() && Index == 0)
return 0; return 0;
} }
return TargetTransformInfo::getVectorInstrCost(Opcode, Val, Index); return BaseT::getVectorInstrCost(Opcode, Val, Index);
} }
unsigned X86TTI::getScalarizationOverhead(Type *Ty, bool Insert, unsigned X86TTIImpl::getScalarizationOverhead(Type *Ty, bool Insert,
bool Extract) const { bool Extract) {
assert (Ty->isVectorTy() && "Can only scalarize vectors"); assert (Ty->isVectorTy() && "Can only scalarize vectors");
unsigned Cost = 0; unsigned Cost = 0;
for (int i = 0, e = Ty->getVectorNumElements(); i < e; ++i) { for (int i = 0, e = Ty->getVectorNumElements(); i < e; ++i) {
if (Insert) if (Insert)
Cost += TopTTI->getVectorInstrCost(Instruction::InsertElement, Ty, i); Cost += getVectorInstrCost(Instruction::InsertElement, Ty, i);
if (Extract) if (Extract)
Cost += TopTTI->getVectorInstrCost(Instruction::ExtractElement, Ty, i); Cost += getVectorInstrCost(Instruction::ExtractElement, Ty, i);
} }
return Cost; return Cost;
} }
unsigned X86TTI::getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment, unsigned X86TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Src,
unsigned AddressSpace) const { unsigned Alignment,
unsigned AddressSpace) {
// Handle non-power-of-two vectors such as <3 x float> // Handle non-power-of-two vectors such as <3 x float>
if (VectorType *VTy = dyn_cast<VectorType>(Src)) { if (VectorType *VTy = dyn_cast<VectorType>(Src)) {
unsigned NumElem = VTy->getVectorNumElements(); unsigned NumElem = VTy->getVectorNumElements();
// Handle a few common cases: // Handle a few common cases:
// <3 x float> // <3 x float>
if (NumElem == 3 && VTy->getScalarSizeInBits() == 32) if (NumElem == 3 && VTy->getScalarSizeInBits() == 32)
// Cost = 64 bit store + extract + 32 bit store. // Cost = 64 bit store + extract + 32 bit store.
return 3; return 3;
// <3 x double> // <3 x double>
if (NumElem == 3 && VTy->getScalarSizeInBits() == 64) if (NumElem == 3 && VTy->getScalarSizeInBits() == 64)
// Cost = 128 bit store + unpack + 64 bit store. // Cost = 128 bit store + unpack + 64 bit store.
return 3; return 3;
// Assume that all other non-power-of-two numbers are scalarized. // Assume that all other non-power-of-two numbers are scalarized.
if (!isPowerOf2_32(NumElem)) { if (!isPowerOf2_32(NumElem)) {
unsigned Cost = TargetTransformInfo::getMemoryOpCost(Opcode, unsigned Cost = BaseT::getMemoryOpCost(Opcode, VTy->getScalarType(),
VTy->getScalarType(), Alignment, AddressSpace);
Alignment,
AddressSpace);
unsigned SplitCost = getScalarizationOverhead(Src, unsigned SplitCost = getScalarizationOverhead(Src,
Opcode == Instruction::Load, Opcode == Instruction::Load,
Opcode==Instruction::Store); Opcode==Instruction::Store);
return NumElem * Cost + SplitCost; return NumElem * Cost + SplitCost;
} }
} }
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Src); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(Src);
assert((Opcode == Instruction::Load || Opcode == Instruction::Store) && assert((Opcode == Instruction::Load || Opcode == Instruction::Store) &&
"Invalid Opcode"); "Invalid Opcode");
// Each load/store unit costs 1. // Each load/store unit costs 1.
unsigned Cost = LT.first * 1; unsigned Cost = LT.first * 1;
// On Sandybridge 256bit load/stores are double pumped // On Sandybridge 256bit load/stores are double pumped
// (but not on Haswell). // (but not on Haswell).
if (LT.second.getSizeInBits() > 128 && !ST->hasAVX2()) if (LT.second.getSizeInBits() > 128 && !ST->hasAVX2())
Cost*=2; Cost*=2;
return Cost; return Cost;
} }
unsigned X86TTI::getMaskedMemoryOpCost(unsigned Opcode, Type *SrcTy, unsigned X86TTIImpl::getMaskedMemoryOpCost(unsigned Opcode, Type *SrcTy,
unsigned Alignment, unsigned Alignment,
unsigned AddressSpace) const { unsigned AddressSpace) {
VectorType *SrcVTy = dyn_cast<VectorType>(SrcTy); VectorType *SrcVTy = dyn_cast<VectorType>(SrcTy);
if (!SrcVTy) if (!SrcVTy)
// To calculate scalar take the regular cost, without mask // To calculate scalar take the regular cost, without mask
return getMemoryOpCost(Opcode, SrcTy, Alignment, AddressSpace); return getMemoryOpCost(Opcode, SrcTy, Alignment, AddressSpace);
unsigned NumElem = SrcVTy->getVectorNumElements(); unsigned NumElem = SrcVTy->getVectorNumElements();
VectorType *MaskTy = VectorType *MaskTy =
VectorType::get(Type::getInt8Ty(getGlobalContext()), NumElem); VectorType::get(Type::getInt8Ty(getGlobalContext()), NumElem);
if ((Opcode == Instruction::Load && !isLegalMaskedLoad(SrcVTy, 1)) || if ((Opcode == Instruction::Load && !isLegalMaskedLoad(SrcVTy, 1)) ||
(Opcode == Instruction::Store && !isLegalMaskedStore(SrcVTy, 1)) || (Opcode == Instruction::Store && !isLegalMaskedStore(SrcVTy, 1)) ||
!isPowerOf2_32(NumElem)) { !isPowerOf2_32(NumElem)) {
// Scalarization // Scalarization
unsigned MaskSplitCost = getScalarizationOverhead(MaskTy, false, true); unsigned MaskSplitCost = getScalarizationOverhead(MaskTy, false, true);
unsigned ScalarCompareCost = unsigned ScalarCompareCost =
getCmpSelInstrCost(Instruction::ICmp, getCmpSelInstrCost(Instruction::ICmp,
Type::getInt8Ty(getGlobalContext()), NULL); Type::getInt8Ty(getGlobalContext()), NULL);
unsigned BranchCost = getCFInstrCost(Instruction::Br); unsigned BranchCost = getCFInstrCost(Instruction::Br);
unsigned MaskCmpCost = NumElem * (BranchCost + ScalarCompareCost); unsigned MaskCmpCost = NumElem * (BranchCost + ScalarCompareCost);
unsigned ValueSplitCost = unsigned ValueSplitCost =
getScalarizationOverhead(SrcVTy, Opcode == Instruction::Load, getScalarizationOverhead(SrcVTy, Opcode == Instruction::Load,
Opcode == Instruction::Store); Opcode == Instruction::Store);
unsigned MemopCost = NumElem * unsigned MemopCost =
TargetTransformInfo::getMemoryOpCost(Opcode, SrcVTy->getScalarType(), NumElem * BaseT::getMemoryOpCost(Opcode, SrcVTy->getScalarType(),
Alignment, AddressSpace); Alignment, AddressSpace);
return MemopCost + ValueSplitCost + MaskSplitCost + MaskCmpCost; return MemopCost + ValueSplitCost + MaskSplitCost + MaskCmpCost;
} }
// Legalize the type. // Legalize the type.
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(SrcVTy); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(SrcVTy);
unsigned Cost = 0; unsigned Cost = 0;
if (LT.second != TLI->getValueType(SrcVTy).getSimpleVT() && if (LT.second != TLI->getValueType(SrcVTy).getSimpleVT() &&
LT.second.getVectorNumElements() == NumElem) LT.second.getVectorNumElements() == NumElem)
// Promotion requires expand/truncate for data and a shuffle for mask. // Promotion requires expand/truncate for data and a shuffle for mask.
Cost += getShuffleCost(TargetTransformInfo::SK_Alternate, SrcVTy, 0, 0) + Cost += getShuffleCost(TTI::SK_Alternate, SrcVTy, 0, 0) +
getShuffleCost(TargetTransformInfo::SK_Alternate, MaskTy, 0, 0); getShuffleCost(TTI::SK_Alternate, MaskTy, 0, 0);
else if (LT.second.getVectorNumElements() > NumElem) { else if (LT.second.getVectorNumElements() > NumElem) {
VectorType *NewMaskTy = VectorType::get(MaskTy->getVectorElementType(), VectorType *NewMaskTy = VectorType::get(MaskTy->getVectorElementType(),
LT.second.getVectorNumElements()); LT.second.getVectorNumElements());
// Expanding requires fill mask with zeroes // Expanding requires fill mask with zeroes
Cost += getShuffleCost(TargetTransformInfo::SK_InsertSubvector, Cost += getShuffleCost(TTI::SK_InsertSubvector, NewMaskTy, 0, MaskTy);
NewMaskTy, 0, MaskTy);
} }
if (!ST->hasAVX512()) if (!ST->hasAVX512())
return Cost + LT.first*4; // Each maskmov costs 4 return Cost + LT.first*4; // Each maskmov costs 4
// AVX-512 masked load/store is cheapper // AVX-512 masked load/store is cheapper
return Cost+LT.first; return Cost+LT.first;
} }
unsigned X86TTI::getAddressComputationCost(Type *Ty, bool IsComplex) const { unsigned X86TTIImpl::getAddressComputationCost(Type *Ty, bool IsComplex) {
// Address computations in vectorized code with non-consecutive addresses will // Address computations in vectorized code with non-consecutive addresses will
// likely result in more instructions compared to scalar code where the // likely result in more instructions compared to scalar code where the
// computation can more often be merged into the index mode. The resulting // computation can more often be merged into the index mode. The resulting
// extra micro-ops can significantly decrease throughput. // extra micro-ops can significantly decrease throughput.
unsigned NumVectorInstToHideOverhead = 10; unsigned NumVectorInstToHideOverhead = 10;
if (Ty->isVectorTy() && IsComplex) if (Ty->isVectorTy() && IsComplex)
return NumVectorInstToHideOverhead; return NumVectorInstToHideOverhead;
return TargetTransformInfo::getAddressComputationCost(Ty, IsComplex); return BaseT::getAddressComputationCost(Ty, IsComplex);
} }
unsigned X86TTI::getReductionCost(unsigned Opcode, Type *ValTy, unsigned X86TTIImpl::getReductionCost(unsigned Opcode, Type *ValTy,
bool IsPairwise) const { bool IsPairwise) {
std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(ValTy); std::pair<unsigned, MVT> LT = TLI->getTypeLegalizationCost(ValTy);
MVT MTy = LT.second; MVT MTy = LT.second;
int ISD = TLI->InstructionOpcodeToISD(Opcode); int ISD = TLI->InstructionOpcodeToISD(Opcode);
assert(ISD && "Invalid opcode"); assert(ISD && "Invalid opcode");
▲ Show 20 Lines • Show All 59 Lines • ▼ Show 20 Lines if (IsPairwise) {
if (ST->hasSSE42()) { if (ST->hasSSE42()) {
int Idx = CostTableLookup(SSE42CostTblNoPairWise, ISD, MTy); int Idx = CostTableLookup(SSE42CostTblNoPairWise, ISD, MTy);
if (Idx != -1) if (Idx != -1)
return LT.first * SSE42CostTblNoPairWise[Idx].Cost; return LT.first * SSE42CostTblNoPairWise[Idx].Cost;
} }
} }
return TargetTransformInfo::getReductionCost(Opcode, ValTy, IsPairwise); return BaseT::getReductionCost(Opcode, ValTy, IsPairwise);
} }
/// \brief Calculate the cost of materializing a 64-bit value. This helper /// \brief Calculate the cost of materializing a 64-bit value. This helper
/// method might only calculate a fraction of a larger immediate. Therefore it /// method might only calculate a fraction of a larger immediate. Therefore it
/// is valid to return a cost of ZERO. /// is valid to return a cost of ZERO.
unsigned X86TTI::getIntImmCost(int64_t Val) const { unsigned X86TTIImpl::getIntImmCost(int64_t Val) {
if (Val == 0) if (Val == 0)
return TCC_Free; return TTI::TCC_Free;
if (isInt<32>(Val)) if (isInt<32>(Val))
return TCC_Basic; return TTI::TCC_Basic;
return 2 * TCC_Basic; return 2 * TTI::TCC_Basic;
} }
unsigned X86TTI::getIntImmCost(const APInt &Imm, Type *Ty) const { unsigned X86TTIImpl::getIntImmCost(const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
if (BitSize == 0) if (BitSize == 0)
return ~0U; return ~0U;
// Never hoist constants larger than 128bit, because this might lead to // Never hoist constants larger than 128bit, because this might lead to
// incorrect code generation or assertions in codegen. // incorrect code generation or assertions in codegen.
// Fixme: Create a cost model for types larger than i128 once the codegen // Fixme: Create a cost model for types larger than i128 once the codegen
// issues have been fixed. // issues have been fixed.
if (BitSize > 128) if (BitSize > 128)
return TCC_Free; return TTI::TCC_Free;
if (Imm == 0) if (Imm == 0)
return TCC_Free; return TTI::TCC_Free;
// Sign-extend all constants to a multiple of 64-bit. // Sign-extend all constants to a multiple of 64-bit.
APInt ImmVal = Imm; APInt ImmVal = Imm;
if (BitSize & 0x3f) if (BitSize & 0x3f)
ImmVal = Imm.sext((BitSize + 63) & ~0x3fU); ImmVal = Imm.sext((BitSize + 63) & ~0x3fU);
// Split the constant into 64-bit chunks and calculate the cost for each // Split the constant into 64-bit chunks and calculate the cost for each
// chunk. // chunk.
unsigned Cost = 0; unsigned Cost = 0;
for (unsigned ShiftVal = 0; ShiftVal < BitSize; ShiftVal += 64) { for (unsigned ShiftVal = 0; ShiftVal < BitSize; ShiftVal += 64) {
APInt Tmp = ImmVal.ashr(ShiftVal).sextOrTrunc(64); APInt Tmp = ImmVal.ashr(ShiftVal).sextOrTrunc(64);
int64_t Val = Tmp.getSExtValue(); int64_t Val = Tmp.getSExtValue();
Cost += getIntImmCost(Val); Cost += getIntImmCost(Val);
} }
// We need at least one instruction to materialze the constant. // We need at least one instruction to materialze the constant.
return std::max(1U, Cost); return std::max(1U, Cost);
} }
unsigned X86TTI::getIntImmCost(unsigned Opcode, unsigned Idx, const APInt &Imm, unsigned X86TTIImpl::getIntImmCost(unsigned Opcode, unsigned Idx,
Type *Ty) const { const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
// There is no cost model for constants with a bit size of 0. Return TCC_Free // There is no cost model for constants with a bit size of 0. Return TCC_Free
// here, so that constant hoisting will ignore this constant. // here, so that constant hoisting will ignore this constant.
if (BitSize == 0) if (BitSize == 0)
return TCC_Free; return TTI::TCC_Free;
unsigned ImmIdx = ~0U; unsigned ImmIdx = ~0U;
switch (Opcode) { switch (Opcode) {
default: return TCC_Free; default:
return TTI::TCC_Free;
case Instruction::GetElementPtr: case Instruction::GetElementPtr:
// Always hoist the base address of a GetElementPtr. This prevents the // Always hoist the base address of a GetElementPtr. This prevents the
// creation of new constants for every base constant that gets constant // creation of new constants for every base constant that gets constant
// folded with the offset. // folded with the offset.
if (Idx == 0) if (Idx == 0)
return 2 * TCC_Basic; return 2 * TTI::TCC_Basic;
return TCC_Free; return TTI::TCC_Free;
case Instruction::Store: case Instruction::Store:
ImmIdx = 0; ImmIdx = 0;
break; break;
case Instruction::Add: case Instruction::Add:
case Instruction::Sub: case Instruction::Sub:
case Instruction::Mul: case Instruction::Mul:
case Instruction::UDiv: case Instruction::UDiv:
case Instruction::SDiv: case Instruction::SDiv:
case Instruction::URem: case Instruction::URem:
case Instruction::SRem: case Instruction::SRem:
case Instruction::And: case Instruction::And:
case Instruction::Or: case Instruction::Or:
case Instruction::Xor: case Instruction::Xor:
case Instruction::ICmp: case Instruction::ICmp:
ImmIdx = 1; ImmIdx = 1;
break; break;
// Always return TCC_Free for the shift value of a shift instruction. // Always return TCC_Free for the shift value of a shift instruction.
case Instruction::Shl: case Instruction::Shl:
case Instruction::LShr: case Instruction::LShr:
case Instruction::AShr: case Instruction::AShr:
if (Idx == 1) if (Idx == 1)
return TCC_Free; return TTI::TCC_Free;
break; break;
case Instruction::Trunc: case Instruction::Trunc:
case Instruction::ZExt: case Instruction::ZExt:
case Instruction::SExt: case Instruction::SExt:
case Instruction::IntToPtr: case Instruction::IntToPtr:
case Instruction::PtrToInt: case Instruction::PtrToInt:
case Instruction::BitCast: case Instruction::BitCast:
case Instruction::PHI: case Instruction::PHI:
case Instruction::Call: case Instruction::Call:
case Instruction::Select: case Instruction::Select:
case Instruction::Ret: case Instruction::Ret:
case Instruction::Load: case Instruction::Load:
break; break;
} }
if (Idx == ImmIdx) { if (Idx == ImmIdx) {
unsigned NumConstants = (BitSize + 63) / 64; unsigned NumConstants = (BitSize + 63) / 64;
unsigned Cost = X86TTI::getIntImmCost(Imm, Ty); unsigned Cost = X86TTIImpl::getIntImmCost(Imm, Ty);
return (Cost <= NumConstants * TCC_Basic) return (Cost <= NumConstants * TTI::TCC_Basic)
? static_cast<unsigned>(TCC_Free) ? static_cast<unsigned>(TTI::TCC_Free)
: Cost; : Cost;
} }
return X86TTI::getIntImmCost(Imm, Ty); return X86TTIImpl::getIntImmCost(Imm, Ty);
} }
unsigned X86TTI::getIntImmCost(Intrinsic::ID IID, unsigned Idx, unsigned X86TTIImpl::getIntImmCost(Intrinsic::ID IID, unsigned Idx,
const APInt &Imm, Type *Ty) const { const APInt &Imm, Type *Ty) {
assert(Ty->isIntegerTy()); assert(Ty->isIntegerTy());
unsigned BitSize = Ty->getPrimitiveSizeInBits(); unsigned BitSize = Ty->getPrimitiveSizeInBits();
// There is no cost model for constants with a bit size of 0. Return TCC_Free // There is no cost model for constants with a bit size of 0. Return TCC_Free
// here, so that constant hoisting will ignore this constant. // here, so that constant hoisting will ignore this constant.
if (BitSize == 0) if (BitSize == 0)
return TCC_Free; return TTI::TCC_Free;
switch (IID) { switch (IID) {
default: return TCC_Free; default:
return TTI::TCC_Free;
case Intrinsic::sadd_with_overflow: case Intrinsic::sadd_with_overflow:
case Intrinsic::uadd_with_overflow: case Intrinsic::uadd_with_overflow:
case Intrinsic::ssub_with_overflow: case Intrinsic::ssub_with_overflow:
case Intrinsic::usub_with_overflow: case Intrinsic::usub_with_overflow:
case Intrinsic::smul_with_overflow: case Intrinsic::smul_with_overflow:
case Intrinsic::umul_with_overflow: case Intrinsic::umul_with_overflow:
if ((Idx == 1) && Imm.getBitWidth() <= 64 && isInt<32>(Imm.getSExtValue())) if ((Idx == 1) && Imm.getBitWidth() <= 64 && isInt<32>(Imm.getSExtValue()))
return TCC_Free; return TTI::TCC_Free;
break; break;
case Intrinsic::experimental_stackmap: case Intrinsic::experimental_stackmap:
if ((Idx < 2) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue()))) if ((Idx < 2) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue())))
return TCC_Free; return TTI::TCC_Free;
break; break;
case Intrinsic::experimental_patchpoint_void: case Intrinsic::experimental_patchpoint_void:
case Intrinsic::experimental_patchpoint_i64: case Intrinsic::experimental_patchpoint_i64:
if ((Idx < 4) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue()))) if ((Idx < 4) || (Imm.getBitWidth() <= 64 && isInt<64>(Imm.getSExtValue())))
return TCC_Free; return TTI::TCC_Free;
break; break;
} }
return X86TTI::getIntImmCost(Imm, Ty); return X86TTIImpl::getIntImmCost(Imm, Ty);
} }
bool X86TTI::isLegalMaskedLoad(Type *DataTy, int Consecutive) const { bool X86TTIImpl::isLegalMaskedLoad(Type *DataTy, int Consecutive) {
int DataWidth = DataTy->getPrimitiveSizeInBits(); int DataWidth = DataTy->getPrimitiveSizeInBits();
// Todo: AVX512 allows gather/scatter, works with strided and random as well // Todo: AVX512 allows gather/scatter, works with strided and random as well
if ((DataWidth < 32) || (Consecutive == 0)) if ((DataWidth < 32) || (Consecutive == 0))
return false; return false;
if (ST->hasAVX512() || ST->hasAVX2()) if (ST->hasAVX512() || ST->hasAVX2())
return true; return true;
return false; return false;
} }
bool X86TTI::isLegalMaskedStore(Type *DataType, int Consecutive) const { bool X86TTIImpl::isLegalMaskedStore(Type *DataType, int Consecutive) {
return isLegalMaskedLoad(DataType, Consecutive); return isLegalMaskedLoad(DataType, Consecutive);
} }

llvm/trunk/lib/Target/XCore/XCoreTargetMachine.cpp

Show First 20 Lines • Show All 77 Lines • ▼ Show 20 Lines
} }
// Force static initialization. // Force static initialization.
extern "C" void LLVMInitializeXCoreTarget() { extern "C" void LLVMInitializeXCoreTarget() {
RegisterTargetMachine<XCoreTargetMachine> X(TheXCoreTarget); RegisterTargetMachine<XCoreTargetMachine> X(TheXCoreTarget);
} }
void XCoreTargetMachine::addAnalysisPasses(PassManagerBase &PM) { void XCoreTargetMachine::addAnalysisPasses(PassManagerBase &PM) {
// Add first the target-independent BasicTTI pass, then our XCore pass. This
// allows the XCore pass to delegate to the target independent layer when
// appropriate.
PM.add(createBasicTargetTransformInfoPass(this));
PM.add(createXCoreTargetTransformInfoPass(this)); PM.add(createXCoreTargetTransformInfoPass(this));
} }

llvm/trunk/lib/Target/XCore/XCoreTargetTransformInfo.cpp

Show All 9 Lines
/// This file implements a TargetTransformInfo analysis pass specific to the /// This file implements a TargetTransformInfo analysis pass specific to the
/// XCore target machine. It uses the target's detailed information to provide /// XCore target machine. It uses the target's detailed information to provide
/// more precise answers to certain TTI queries, while letting the target /// more precise answers to certain TTI queries, while letting the target
/// independent and default TTI implementations handle the rest. /// independent and default TTI implementations handle the rest.
/// ///
//===----------------------------------------------------------------------===// //===----------------------------------------------------------------------===//
#include "XCore.h" #include "XCore.h"
#include "XCoreTargetMachine.h"
#include "llvm/Analysis/TargetTransformInfo.h" #include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/CodeGen/BasicTTIImpl.h"
#include "llvm/Support/Debug.h" #include "llvm/Support/Debug.h"
#include "llvm/Target/CostTable.h" #include "llvm/Target/CostTable.h"
#include "llvm/Target/TargetLowering.h" #include "llvm/Target/TargetLowering.h"
using namespace llvm; using namespace llvm;
#define DEBUG_TYPE "xcoretti" #define DEBUG_TYPE "xcoretti"
// Declare the pass initialization routine locally as target-specific passes
// don't have a target-wide initialization entry point, and so we rely on the
// pass constructor initialization.
namespace llvm {
void initializeXCoreTTIPass(PassRegistry &);
}
namespace { namespace {
class XCoreTTI final : public ImmutablePass, public TargetTransformInfo { class XCoreTTIImpl : public BasicTTIImplBase<XCoreTTIImpl> {
public: typedef BasicTTIImplBase<XCoreTTIImpl> BaseT;
XCoreTTI() : ImmutablePass(ID) { typedef TargetTransformInfo TTI;
llvm_unreachable("This pass cannot be directly constructed");
}
XCoreTTI(const XCoreTargetMachine *TM) public:
: ImmutablePass(ID) { explicit XCoreTTIImpl(const XCoreTargetMachine *TM = nullptr) : BaseT(TM) {}
initializeXCoreTTIPass(*PassRegistry::getPassRegistry());
}
void initializePass() override {
pushTTIStack(this);
}
void getAnalysisUsage(AnalysisUsage &AU) const override {
TargetTransformInfo::getAnalysisUsage(AU);
}
static char ID;
void *getAdjustedAnalysisPointer(const void *ID) override { // Provide value semantics. MSVC requires that we spell all of these out.
if (ID == &TargetTransformInfo::ID) XCoreTTIImpl(const XCoreTTIImpl &Arg)
return (TargetTransformInfo*)this; : BaseT(static_cast<const BaseT &>(Arg)) {}
return this; XCoreTTIImpl(XCoreTTIImpl &&Arg)
: BaseT(std::move(static_cast<BaseT &>(Arg))) {}
XCoreTTIImpl &operator=(const XCoreTTIImpl &RHS) {
BaseT::operator=(static_cast<const BaseT &>(RHS));
return *this;
}
XCoreTTIImpl &operator=(XCoreTTIImpl &&RHS) {
BaseT::operator=(std::move(static_cast<BaseT &>(RHS)));
return *this;
} }
unsigned getNumberOfRegisters(bool Vector) const override { unsigned getNumberOfRegisters(bool Vector) {
if (Vector) { if (Vector) {
return 0; return 0;
} }
return 12; return 12;
} }
}; };
} // end anonymous namespace } // end anonymous namespace
INITIALIZE_AG_PASS(XCoreTTI, TargetTransformInfo, "xcoretti",
"XCore Target Transform Info", true, true, false)
char XCoreTTI::ID = 0;
ImmutablePass * ImmutablePass *
llvm::createXCoreTargetTransformInfoPass(const XCoreTargetMachine *TM) { llvm::createXCoreTargetTransformInfoPass(const XCoreTargetMachine *TM) {
return new XCoreTTI(TM); return new TargetTransformInfoWrapperPass(XCoreTTIImpl(TM));
} }

llvm/trunk/lib/Transforms/Scalar/ConstantHoisting.cpp

Show First 20 Lines • Show All 125 Lines • ▼ Show 20 Linespublic:
bool runOnFunction(Function &Fn) override; bool runOnFunction(Function &Fn) override;
const char *getPassName() const override { return "Constant Hoisting"; } const char *getPassName() const override { return "Constant Hoisting"; }
void getAnalysisUsage(AnalysisUsage &AU) const override { void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.setPreservesCFG(); AU.setPreservesCFG();
AU.addRequired<DominatorTreeWrapperPass>(); AU.addRequired<DominatorTreeWrapperPass>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
} }
private: private:
/// \brief Initialize the pass. /// \brief Initialize the pass.
void setup(Function &Fn) { void setup(Function &Fn) {
DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree(); DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree();
TTI = &getAnalysis<TargetTransformInfo>(); TTI = &getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
Entry = &Fn.getEntryBlock(); Entry = &Fn.getEntryBlock();
} }
/// \brief Cleanup. /// \brief Cleanup.
void cleanup() { void cleanup() {
ConstantVec.clear(); ConstantVec.clear();
ClonedCastMap.clear(); ClonedCastMap.clear();
ConstCandVec.clear(); ConstCandVec.clear();
Show All 21 Linesprivate:
bool optimizeConstants(Function &Fn); bool optimizeConstants(Function &Fn);
}; };
} }
char ConstantHoisting::ID = 0; char ConstantHoisting::ID = 0;
INITIALIZE_PASS_BEGIN(ConstantHoisting, "consthoist", "Constant Hoisting", INITIALIZE_PASS_BEGIN(ConstantHoisting, "consthoist", "Constant Hoisting",
false, false) false, false)
INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass) INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_END(ConstantHoisting, "consthoist", "Constant Hoisting", INITIALIZE_PASS_END(ConstantHoisting, "consthoist", "Constant Hoisting",
false, false) false, false)
FunctionPass *llvm::createConstantHoistingPass() { FunctionPass *llvm::createConstantHoistingPass() {
return new ConstantHoisting(); return new ConstantHoisting();
} }
/// \brief Perform the constant hoisting optimization for the given function. /// \brief Perform the constant hoisting optimization for the given function.
▲ Show 20 Lines • Show All 415 Lines • Show Last 20 Lines

llvm/trunk/lib/Transforms/Scalar/EarlyCSE.cpp

Show First 20 Lines • Show All 706 Lines • ▼ Show 20 Linespublic:
bool runOnFunction(Function &F) override { bool runOnFunction(Function &F) override {
if (skipOptnoneFunction(F)) if (skipOptnoneFunction(F))
return false; return false;
DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>(); DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>();
auto *DL = DLP ? &DLP->getDataLayout() : nullptr; auto *DL = DLP ? &DLP->getDataLayout() : nullptr;
auto &TLI = getAnalysis<TargetLibraryInfoWrapperPass>().getTLI(); auto &TLI = getAnalysis<TargetLibraryInfoWrapperPass>().getTLI();
auto &TTI = getAnalysis<TargetTransformInfo>(); auto &TTI = getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
auto &DT = getAnalysis<DominatorTreeWrapperPass>().getDomTree(); auto &DT = getAnalysis<DominatorTreeWrapperPass>().getDomTree();
auto &AC = getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F); auto &AC = getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F);
EarlyCSE CSE(F, DL, TLI, TTI, DT, AC); EarlyCSE CSE(F, DL, TLI, TTI, DT, AC);
return CSE.run(); return CSE.run();
} }
void getAnalysisUsage(AnalysisUsage &AU) const override { void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addRequired<AssumptionCacheTracker>(); AU.addRequired<AssumptionCacheTracker>();
AU.addRequired<DominatorTreeWrapperPass>(); AU.addRequired<DominatorTreeWrapperPass>();
AU.addRequired<TargetLibraryInfoWrapperPass>(); AU.addRequired<TargetLibraryInfoWrapperPass>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
AU.setPreservesCFG(); AU.setPreservesCFG();
} }
}; };
} }
char EarlyCSELegacyPass::ID = 0; char EarlyCSELegacyPass::ID = 0;
FunctionPass *llvm::createEarlyCSEPass() { return new EarlyCSELegacyPass(); } FunctionPass *llvm::createEarlyCSEPass() { return new EarlyCSELegacyPass(); }
INITIALIZE_PASS_BEGIN(EarlyCSELegacyPass, "early-cse", "Early CSE", false, INITIALIZE_PASS_BEGIN(EarlyCSELegacyPass, "early-cse", "Early CSE", false,
false) false)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker) INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker)
INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass) INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)
INITIALIZE_PASS_DEPENDENCY(TargetLibraryInfoWrapperPass) INITIALIZE_PASS_DEPENDENCY(TargetLibraryInfoWrapperPass)
INITIALIZE_PASS_END(EarlyCSELegacyPass, "early-cse", "Early CSE", false, false) INITIALIZE_PASS_END(EarlyCSELegacyPass, "early-cse", "Early CSE", false, false)

llvm/trunk/lib/Transforms/Scalar/IndVarSimplify.cpp

Show First 20 Lines • Show All 1,930 Lines • ▼ Show 20 Linesbool IndVarSimplify::runOnLoop(Loop *L, LPPassManager &LPM) {
LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo(); LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo();
SE = &getAnalysis<ScalarEvolution>(); SE = &getAnalysis<ScalarEvolution>();
DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree(); DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree();
DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>(); DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>();
DL = DLP ? &DLP->getDataLayout() : nullptr; DL = DLP ? &DLP->getDataLayout() : nullptr;
auto *TLIP = getAnalysisIfAvailable<TargetLibraryInfoWrapperPass>(); auto *TLIP = getAnalysisIfAvailable<TargetLibraryInfoWrapperPass>();
TLI = TLIP ? &TLIP->getTLI() : nullptr; TLI = TLIP ? &TLIP->getTLI() : nullptr;
TTI = getAnalysisIfAvailable<TargetTransformInfo>(); auto *TTIP = getAnalysisIfAvailable<TargetTransformInfoWrapperPass>();
TTI = TTIP ? &TTIP->getTTI() : nullptr;
DeadInsts.clear(); DeadInsts.clear();
Changed = false; Changed = false;
// If there are any floating-point recurrences, attempt to // If there are any floating-point recurrences, attempt to
// transform them to use integer recurrences. // transform them to use integer recurrences.
RewriteNonIntegerIVs(L); RewriteNonIntegerIVs(L);
▲ Show 20 Lines • Show All 91 Lines • Show Last 20 Lines

llvm/trunk/lib/Transforms/Scalar/LoopIdiomRecognize.cpp

Show First 20 Lines • Show All 170 Lines • ▼ Show 20 Lines void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addPreservedID(LCSSAID); AU.addPreservedID(LCSSAID);
AU.addRequired<AliasAnalysis>(); AU.addRequired<AliasAnalysis>();
AU.addPreserved<AliasAnalysis>(); AU.addPreserved<AliasAnalysis>();
AU.addRequired<ScalarEvolution>(); AU.addRequired<ScalarEvolution>();
AU.addPreserved<ScalarEvolution>(); AU.addPreserved<ScalarEvolution>();
AU.addPreserved<DominatorTreeWrapperPass>(); AU.addPreserved<DominatorTreeWrapperPass>();
AU.addRequired<DominatorTreeWrapperPass>(); AU.addRequired<DominatorTreeWrapperPass>();
AU.addRequired<TargetLibraryInfoWrapperPass>(); AU.addRequired<TargetLibraryInfoWrapperPass>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
} }
const DataLayout *getDataLayout() { const DataLayout *getDataLayout() {
if (DL) if (DL)
return DL; return DL;
DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>(); DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>();
DL = DLP ? &DLP->getDataLayout() : nullptr; DL = DLP ? &DLP->getDataLayout() : nullptr;
return DL; return DL;
Show All 11 Lines public:
TargetLibraryInfo *getTargetLibraryInfo() { TargetLibraryInfo *getTargetLibraryInfo() {
if (!TLI) if (!TLI)
TLI = &getAnalysis<TargetLibraryInfoWrapperPass>().getTLI(); TLI = &getAnalysis<TargetLibraryInfoWrapperPass>().getTLI();
return TLI; return TLI;
} }
const TargetTransformInfo *getTargetTransformInfo() { const TargetTransformInfo *getTargetTransformInfo() {
return TTI ? TTI : (TTI = &getAnalysis<TargetTransformInfo>()); return TTI ? TTI : (TTI = &getAnalysis<TargetTransformInfoWrapperPass>()
.getTTI());
} }
Loop *getLoop() const { return CurLoop; } Loop *getLoop() const { return CurLoop; }
private: private:
bool runOnNoncountableLoop(); bool runOnNoncountableLoop();
bool runOnCountableLoop(); bool runOnCountableLoop();
}; };
} }
char LoopIdiomRecognize::ID = 0; char LoopIdiomRecognize::ID = 0;
INITIALIZE_PASS_BEGIN(LoopIdiomRecognize, "loop-idiom", "Recognize loop idioms", INITIALIZE_PASS_BEGIN(LoopIdiomRecognize, "loop-idiom", "Recognize loop idioms",
false, false) false, false)
INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass) INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass) INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)
INITIALIZE_PASS_DEPENDENCY(LoopSimplify) INITIALIZE_PASS_DEPENDENCY(LoopSimplify)
INITIALIZE_PASS_DEPENDENCY(LCSSA) INITIALIZE_PASS_DEPENDENCY(LCSSA)
INITIALIZE_PASS_DEPENDENCY(ScalarEvolution) INITIALIZE_PASS_DEPENDENCY(ScalarEvolution)
INITIALIZE_PASS_DEPENDENCY(TargetLibraryInfoWrapperPass) INITIALIZE_PASS_DEPENDENCY(TargetLibraryInfoWrapperPass)
INITIALIZE_AG_DEPENDENCY(AliasAnalysis) INITIALIZE_AG_DEPENDENCY(AliasAnalysis)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_END(LoopIdiomRecognize, "loop-idiom", "Recognize loop idioms", INITIALIZE_PASS_END(LoopIdiomRecognize, "loop-idiom", "Recognize loop idioms",
false, false) false, false)
Pass *llvm::createLoopIdiomPass() { return new LoopIdiomRecognize(); } Pass *llvm::createLoopIdiomPass() { return new LoopIdiomRecognize(); }
/// deleteDeadInstruction - Delete this instruction. Before we do, go through /// deleteDeadInstruction - Delete this instruction. Before we do, go through
/// and zero out all the operands of this instruction. If any of them become /// and zero out all the operands of this instruction. If any of them become
/// dead, delete them and the computation tree that feeds them. /// dead, delete them and the computation tree that feeds them.
▲ Show 20 Lines • Show All 916 Lines • Show Last 20 Lines

llvm/trunk/lib/Transforms/Scalar/LoopRotation.cpp

Show First 20 Lines • Show All 57 Lines • ▼ Show 20 Lines void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addPreserved<DominatorTreeWrapperPass>(); AU.addPreserved<DominatorTreeWrapperPass>();
AU.addRequired<LoopInfoWrapperPass>(); AU.addRequired<LoopInfoWrapperPass>();
AU.addPreserved<LoopInfoWrapperPass>(); AU.addPreserved<LoopInfoWrapperPass>();
AU.addRequiredID(LoopSimplifyID); AU.addRequiredID(LoopSimplifyID);
AU.addPreservedID(LoopSimplifyID); AU.addPreservedID(LoopSimplifyID);
AU.addRequiredID(LCSSAID); AU.addRequiredID(LCSSAID);
AU.addPreservedID(LCSSAID); AU.addPreservedID(LCSSAID);
AU.addPreserved<ScalarEvolution>(); AU.addPreserved<ScalarEvolution>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
} }
bool runOnLoop(Loop *L, LPPassManager &LPM) override; bool runOnLoop(Loop *L, LPPassManager &LPM) override;
bool simplifyLoopLatch(Loop *L); bool simplifyLoopLatch(Loop *L);
bool rotateLoop(Loop *L, bool SimplifiedLatch); bool rotateLoop(Loop *L, bool SimplifiedLatch);
private: private:
unsigned MaxHeaderSize; unsigned MaxHeaderSize;
LoopInfo *LI; LoopInfo *LI;
const TargetTransformInfo *TTI; const TargetTransformInfo *TTI;
AssumptionCache *AC; AssumptionCache *AC;
DominatorTree *DT; DominatorTree *DT;
}; };
} }
char LoopRotate::ID = 0; char LoopRotate::ID = 0;
INITIALIZE_PASS_BEGIN(LoopRotate, "loop-rotate", "Rotate Loops", false, false) INITIALIZE_PASS_BEGIN(LoopRotate, "loop-rotate", "Rotate Loops", false, false)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker) INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker)
INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass) INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(LoopSimplify) INITIALIZE_PASS_DEPENDENCY(LoopSimplify)
INITIALIZE_PASS_DEPENDENCY(LCSSA) INITIALIZE_PASS_DEPENDENCY(LCSSA)
INITIALIZE_PASS_END(LoopRotate, "loop-rotate", "Rotate Loops", false, false) INITIALIZE_PASS_END(LoopRotate, "loop-rotate", "Rotate Loops", false, false)
Pass *llvm::createLoopRotatePass(int MaxHeaderSize) { Pass *llvm::createLoopRotatePass(int MaxHeaderSize) {
return new LoopRotate(MaxHeaderSize); return new LoopRotate(MaxHeaderSize);
} }
/// Rotate Loop L as many times as possible. Return true if /// Rotate Loop L as many times as possible. Return true if
/// the loop is rotated at least once. /// the loop is rotated at least once.
bool LoopRotate::runOnLoop(Loop *L, LPPassManager &LPM) { bool LoopRotate::runOnLoop(Loop *L, LPPassManager &LPM) {
if (skipOptnoneFunction(L)) if (skipOptnoneFunction(L))
return false; return false;
// Save the loop metadata. // Save the loop metadata.
MDNode *LoopMD = L->getLoopID(); MDNode *LoopMD = L->getLoopID();
LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo(); LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo();
TTI = &getAnalysis<TargetTransformInfo>(); TTI = &getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
AC = &getAnalysis<AssumptionCacheTracker>().getAssumptionCache( AC = &getAnalysis<AssumptionCacheTracker>().getAssumptionCache(
*L->getHeader()->getParent()); *L->getHeader()->getParent());
auto *DTWP = getAnalysisIfAvailable<DominatorTreeWrapperPass>(); auto *DTWP = getAnalysisIfAvailable<DominatorTreeWrapperPass>();
DT = DTWP ? &DTWP->getDomTree() : nullptr; DT = DTWP ? &DTWP->getDomTree() : nullptr;
// Simplify the loop latch before attempting to rotate the header // Simplify the loop latch before attempting to rotate the header
// upward. Rotation may not be needed if the loop tail can be folded into the // upward. Rotation may not be needed if the loop tail can be folded into the
// loop exit. // loop exit.
▲ Show 20 Lines • Show All 489 Lines • Show Last 20 Lines

llvm/trunk/lib/Transforms/Scalar/LoopStrengthReduce.cpp

Show First 20 Lines • Show All 4,860 Lines • ▼ Show 20 Lines#endif
Changed |= DeleteTriviallyDeadInstructions(DeadInsts); Changed |= DeleteTriviallyDeadInstructions(DeadInsts);
} }
LSRInstance::LSRInstance(Loop *L, Pass *P) LSRInstance::LSRInstance(Loop *L, Pass *P)
: IU(P->getAnalysis<IVUsers>()), SE(P->getAnalysis<ScalarEvolution>()), : IU(P->getAnalysis<IVUsers>()), SE(P->getAnalysis<ScalarEvolution>()),
DT(P->getAnalysis<DominatorTreeWrapperPass>().getDomTree()), DT(P->getAnalysis<DominatorTreeWrapperPass>().getDomTree()),
LI(P->getAnalysis<LoopInfoWrapperPass>().getLoopInfo()), LI(P->getAnalysis<LoopInfoWrapperPass>().getLoopInfo()),
TTI(P->getAnalysis<TargetTransformInfo>()), L(L), Changed(false), TTI(P->getAnalysis<TargetTransformInfoWrapperPass>().getTTI()), L(L),
IVIncInsertPos(nullptr) { Changed(false), IVIncInsertPos(nullptr) {
// If LoopSimplify form is not available, stay out of trouble. // If LoopSimplify form is not available, stay out of trouble.
if (!L->isLoopSimplifyForm()) if (!L->isLoopSimplifyForm())
return; return;
// If there's no interesting work to be done, bail early. // If there's no interesting work to be done, bail early.
if (IU.empty()) return; if (IU.empty()) return;
// If there's too much analysis to be done, bail early. We won't be able to // If there's too much analysis to be done, bail early. We won't be able to
▲ Show 20 Lines • Show All 159 Lines • ▼ Show 20 Linesprivate:
void getAnalysisUsage(AnalysisUsage &AU) const override; void getAnalysisUsage(AnalysisUsage &AU) const override;
}; };
} }
char LoopStrengthReduce::ID = 0; char LoopStrengthReduce::ID = 0;
INITIALIZE_PASS_BEGIN(LoopStrengthReduce, "loop-reduce", INITIALIZE_PASS_BEGIN(LoopStrengthReduce, "loop-reduce",
"Loop Strength Reduction", false, false) "Loop Strength Reduction", false, false)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass) INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)
INITIALIZE_PASS_DEPENDENCY(ScalarEvolution) INITIALIZE_PASS_DEPENDENCY(ScalarEvolution)
INITIALIZE_PASS_DEPENDENCY(IVUsers) INITIALIZE_PASS_DEPENDENCY(IVUsers)
INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass) INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(LoopSimplify) INITIALIZE_PASS_DEPENDENCY(LoopSimplify)
INITIALIZE_PASS_END(LoopStrengthReduce, "loop-reduce", INITIALIZE_PASS_END(LoopStrengthReduce, "loop-reduce",
"Loop Strength Reduction", false, false) "Loop Strength Reduction", false, false)
Show All 18 Linesvoid LoopStrengthReduce::getAnalysisUsage(AnalysisUsage &AU) const {
AU.addPreserved<DominatorTreeWrapperPass>(); AU.addPreserved<DominatorTreeWrapperPass>();
AU.addRequired<ScalarEvolution>(); AU.addRequired<ScalarEvolution>();
AU.addPreserved<ScalarEvolution>(); AU.addPreserved<ScalarEvolution>();
// Requiring LoopSimplify a second time here prevents IVUsers from running // Requiring LoopSimplify a second time here prevents IVUsers from running
// twice, since LoopSimplify was invalidated by running ScalarEvolution. // twice, since LoopSimplify was invalidated by running ScalarEvolution.
AU.addRequiredID(LoopSimplifyID); AU.addRequiredID(LoopSimplifyID);
AU.addRequired<IVUsers>(); AU.addRequired<IVUsers>();
AU.addPreserved<IVUsers>(); AU.addPreserved<IVUsers>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
} }
bool LoopStrengthReduce::runOnLoop(Loop *L, LPPassManager & /*LPM*/) { bool LoopStrengthReduce::runOnLoop(Loop *L, LPPassManager & /*LPM*/) {
if (skipOptnoneFunction(L)) if (skipOptnoneFunction(L))
return false; return false;
bool Changed = false; bool Changed = false;
// Run the main LSR transformation. // Run the main LSR transformation.
Changed |= LSRInstance(L, this).getChanged(); Changed |= LSRInstance(L, this).getChanged();
// Remove any extra phis created by processing inner loops. // Remove any extra phis created by processing inner loops.
Changed |= DeleteDeadPHIs(L->getHeader()); Changed |= DeleteDeadPHIs(L->getHeader());
if (EnablePhiElim && L->isLoopSimplifyForm()) { if (EnablePhiElim && L->isLoopSimplifyForm()) {
SmallVector<WeakVH, 16> DeadInsts; SmallVector<WeakVH, 16> DeadInsts;
SCEVExpander Rewriter(getAnalysis<ScalarEvolution>(), "lsr"); SCEVExpander Rewriter(getAnalysis<ScalarEvolution>(), "lsr");
#ifndef NDEBUG #ifndef NDEBUG
Rewriter.setDebugType(DEBUG_TYPE); Rewriter.setDebugType(DEBUG_TYPE);
#endif #endif
unsigned numFolded = Rewriter.replaceCongruentIVs( unsigned numFolded = Rewriter.replaceCongruentIVs(
L, &getAnalysis<DominatorTreeWrapperPass>().getDomTree(), DeadInsts, L, &getAnalysis<DominatorTreeWrapperPass>().getDomTree(), DeadInsts,
&getAnalysis<TargetTransformInfo>()); &getAnalysis<TargetTransformInfoWrapperPass>().getTTI());
if (numFolded) { if (numFolded) {
Changed = true; Changed = true;
DeleteTriviallyDeadInstructions(DeadInsts); DeleteTriviallyDeadInstructions(DeadInsts);
DeleteDeadPHIs(L->getHeader()); DeleteDeadPHIs(L->getHeader());
} }
} }
return Changed; return Changed;
} }

llvm/trunk/lib/Transforms/Scalar/LoopUnrollPass.cpp

Show First 20 Lines • Show All 107 Lines • ▼ Show 20 Lines void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addRequired<LoopInfoWrapperPass>(); AU.addRequired<LoopInfoWrapperPass>();
AU.addPreserved<LoopInfoWrapperPass>(); AU.addPreserved<LoopInfoWrapperPass>();
AU.addRequiredID(LoopSimplifyID); AU.addRequiredID(LoopSimplifyID);
AU.addPreservedID(LoopSimplifyID); AU.addPreservedID(LoopSimplifyID);
AU.addRequiredID(LCSSAID); AU.addRequiredID(LCSSAID);
AU.addPreservedID(LCSSAID); AU.addPreservedID(LCSSAID);
AU.addRequired<ScalarEvolution>(); AU.addRequired<ScalarEvolution>();
AU.addPreserved<ScalarEvolution>(); AU.addPreserved<ScalarEvolution>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
AU.addRequired<FunctionTargetTransformInfo>(); AU.addRequired<FunctionTargetTransformInfo>();
// FIXME: Loop unroll requires LCSSA. And LCSSA requires dom info. // FIXME: Loop unroll requires LCSSA. And LCSSA requires dom info.
// If loop unroll does not preserve dom info then LCSSA pass on next // If loop unroll does not preserve dom info then LCSSA pass on next
// loop will receive invalid dom info. // loop will receive invalid dom info.
// For now, recreate dom info, if loop is unrolled. // For now, recreate dom info, if loop is unrolled.
AU.addPreserved<DominatorTreeWrapperPass>(); AU.addPreserved<DominatorTreeWrapperPass>();
} }
▲ Show 20 Lines • Show All 55 Lines • ▼ Show 20 Lines void selectThresholds(const Loop *L, bool HasPragma,
std::max<unsigned>(PartialThreshold, PragmaUnrollThreshold); std::max<unsigned>(PartialThreshold, PragmaUnrollThreshold);
} }
} }
}; };
} }
char LoopUnroll::ID = 0; char LoopUnroll::ID = 0;
INITIALIZE_PASS_BEGIN(LoopUnroll, "loop-unroll", "Unroll loops", false, false) INITIALIZE_PASS_BEGIN(LoopUnroll, "loop-unroll", "Unroll loops", false, false)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker) INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker)
INITIALIZE_PASS_DEPENDENCY(FunctionTargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(FunctionTargetTransformInfo)
INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass) INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(LoopSimplify) INITIALIZE_PASS_DEPENDENCY(LoopSimplify)
INITIALIZE_PASS_DEPENDENCY(LCSSA) INITIALIZE_PASS_DEPENDENCY(LCSSA)
INITIALIZE_PASS_DEPENDENCY(ScalarEvolution) INITIALIZE_PASS_DEPENDENCY(ScalarEvolution)
INITIALIZE_PASS_END(LoopUnroll, "loop-unroll", "Unroll loops", false, false) INITIALIZE_PASS_END(LoopUnroll, "loop-unroll", "Unroll loops", false, false)
▲ Show 20 Lines • Show All 163 Lines • ▼ Show 20 Lines
} }
bool LoopUnroll::runOnLoop(Loop *L, LPPassManager &LPM) { bool LoopUnroll::runOnLoop(Loop *L, LPPassManager &LPM) {
if (skipOptnoneFunction(L)) if (skipOptnoneFunction(L))
return false; return false;
LoopInfo *LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo(); LoopInfo *LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo();
ScalarEvolution *SE = &getAnalysis<ScalarEvolution>(); ScalarEvolution *SE = &getAnalysis<ScalarEvolution>();
const TargetTransformInfo &TTI = getAnalysis<TargetTransformInfo>(); const TargetTransformInfo &TTI =
getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
const FunctionTargetTransformInfo &FTTI = const FunctionTargetTransformInfo &FTTI =
getAnalysis<FunctionTargetTransformInfo>(); getAnalysis<FunctionTargetTransformInfo>();
auto &AC = getAnalysis<AssumptionCacheTracker>().getAssumptionCache( auto &AC = getAnalysis<AssumptionCacheTracker>().getAssumptionCache(
*L->getHeader()->getParent()); *L->getHeader()->getParent());
BasicBlock *Header = L->getHeader(); BasicBlock *Header = L->getHeader();
DEBUG(dbgs() << "Loop Unroll: F[" << Header->getParent()->getName() DEBUG(dbgs() << "Loop Unroll: F[" << Header->getParent()->getName()
<< "] Loop %" << Header->getName() << "\n"); << "] Loop %" << Header->getName() << "\n");
▲ Show 20 Lines • Show All 150 Lines • Show Last 20 Lines

llvm/trunk/lib/Transforms/Scalar/LoopUnswitch.cpp

Show First 20 Lines • Show All 170 Lines • ▼ Show 20 Lines void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addRequiredID(LoopSimplifyID); AU.addRequiredID(LoopSimplifyID);
AU.addPreservedID(LoopSimplifyID); AU.addPreservedID(LoopSimplifyID);
AU.addRequired<LoopInfoWrapperPass>(); AU.addRequired<LoopInfoWrapperPass>();
AU.addPreserved<LoopInfoWrapperPass>(); AU.addPreserved<LoopInfoWrapperPass>();
AU.addRequiredID(LCSSAID); AU.addRequiredID(LCSSAID);
AU.addPreservedID(LCSSAID); AU.addPreservedID(LCSSAID);
AU.addPreserved<DominatorTreeWrapperPass>(); AU.addPreserved<DominatorTreeWrapperPass>();
AU.addPreserved<ScalarEvolution>(); AU.addPreserved<ScalarEvolution>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
} }
private: private:
void releaseMemory() override { void releaseMemory() override {
BranchesInfo.forgetLoop(currentLoop); BranchesInfo.forgetLoop(currentLoop);
} }
▲ Show 20 Lines • Show All 140 Lines • ▼ Show 20 Lines for (UnswitchedValsIt I = Insts.begin(); I != Insts.end(); ++I) {
NewLoopProps.UnswitchedVals[NewInst] = OldLoopProps.UnswitchedVals[OldInst]; NewLoopProps.UnswitchedVals[NewInst] = OldLoopProps.UnswitchedVals[OldInst];
} }
} }
char LoopUnswitch::ID = 0; char LoopUnswitch::ID = 0;
INITIALIZE_PASS_BEGIN(LoopUnswitch, "loop-unswitch", "Unswitch loops", INITIALIZE_PASS_BEGIN(LoopUnswitch, "loop-unswitch", "Unswitch loops",
false, false) false, false)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker) INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker)
INITIALIZE_PASS_DEPENDENCY(LoopSimplify) INITIALIZE_PASS_DEPENDENCY(LoopSimplify)
INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass) INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(LCSSA) INITIALIZE_PASS_DEPENDENCY(LCSSA)
INITIALIZE_PASS_END(LoopUnswitch, "loop-unswitch", "Unswitch loops", INITIALIZE_PASS_END(LoopUnswitch, "loop-unswitch", "Unswitch loops",
false, false) false, false)
Pass *llvm::createLoopUnswitchPass(bool Os) { Pass *llvm::createLoopUnswitchPass(bool Os) {
▲ Show 20 Lines • Show All 82 Lines • ▼ Show 20 Linesbool LoopUnswitch::processCurrentLoop() {
// Without dedicated exits, splitting the exit edge may fail. // Without dedicated exits, splitting the exit edge may fail.
if (!currentLoop->hasDedicatedExits()) if (!currentLoop->hasDedicatedExits())
return false; return false;
LLVMContext &Context = loopHeader->getContext(); LLVMContext &Context = loopHeader->getContext();
// Probably we reach the quota of branches for this loop. If so // Probably we reach the quota of branches for this loop. If so
// stop unswitching. // stop unswitching.
if (!BranchesInfo.countLoop(currentLoop, getAnalysis<TargetTransformInfo>(), if (!BranchesInfo.countLoop(
currentLoop, getAnalysis<TargetTransformInfoWrapperPass>().getTTI(),
AC)) AC))
return false; return false;
// Loop over all of the basic blocks in the loop. If we find an interior // Loop over all of the basic blocks in the loop. If we find an interior
// block that is branching on a loop-invariant condition, we can unswitch this // block that is branching on a loop-invariant condition, we can unswitch this
// loop. // loop.
for (Loop::block_iterator I = currentLoop->block_begin(), for (Loop::block_iterator I = currentLoop->block_begin(),
E = currentLoop->block_end(); I != E; ++I) { E = currentLoop->block_end(); I != E; ++I) {
TerminatorInst *TI = (*I)->getTerminator(); TerminatorInst *TI = (*I)->getTerminator();
▲ Show 20 Lines • Show All 709 Lines • Show Last 20 Lines

llvm/trunk/lib/Transforms/Scalar/PartiallyInlineLibCalls.cpp

Show First 20 Lines • Show All 47 Lines • ▼ Show 20 Linesnamespace {
char PartiallyInlineLibCalls::ID = 0; char PartiallyInlineLibCalls::ID = 0;
} }
INITIALIZE_PASS(PartiallyInlineLibCalls, "partially-inline-libcalls", INITIALIZE_PASS(PartiallyInlineLibCalls, "partially-inline-libcalls",
"Partially inline calls to library functions", false, false) "Partially inline calls to library functions", false, false)
void PartiallyInlineLibCalls::getAnalysisUsage(AnalysisUsage &AU) const { void PartiallyInlineLibCalls::getAnalysisUsage(AnalysisUsage &AU) const {
AU.addRequired<TargetLibraryInfoWrapperPass>(); AU.addRequired<TargetLibraryInfoWrapperPass>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
FunctionPass::getAnalysisUsage(AU); FunctionPass::getAnalysisUsage(AU);
} }
bool PartiallyInlineLibCalls::runOnFunction(Function &F) { bool PartiallyInlineLibCalls::runOnFunction(Function &F) {
bool Changed = false; bool Changed = false;
Function::iterator CurrBB; Function::iterator CurrBB;
TargetLibraryInfo *TLI = TargetLibraryInfo *TLI =
&getAnalysis<TargetLibraryInfoWrapperPass>().getTLI(); &getAnalysis<TargetLibraryInfoWrapperPass>().getTLI();
const TargetTransformInfo *TTI = &getAnalysis<TargetTransformInfo>(); const TargetTransformInfo *TTI =
&getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
for (Function::iterator BB = F.begin(), BE = F.end(); BB != BE;) { for (Function::iterator BB = F.begin(), BE = F.end(); BB != BE;) {
CurrBB = BB++; CurrBB = BB++;
for (BasicBlock::iterator II = CurrBB->begin(), IE = CurrBB->end(); for (BasicBlock::iterator II = CurrBB->begin(), IE = CurrBB->end();
II != IE; ++II) { II != IE; ++II) {
CallInst *Call = dyn_cast<CallInst>(&*II); CallInst *Call = dyn_cast<CallInst>(&*II);
Function *CalledFunc; Function *CalledFunc;
▲ Show 20 Lines • Show All 89 Lines • Show Last 20 Lines

llvm/trunk/lib/Transforms/Scalar/SeparateConstOffsetFromGEP.cpp

Show First 20 Lines • Show All 307 Lines • ▼ Show 20 Lines public:
SeparateConstOffsetFromGEP(const TargetMachine *TM = nullptr, SeparateConstOffsetFromGEP(const TargetMachine *TM = nullptr,
bool LowerGEP = false) bool LowerGEP = false)
: FunctionPass(ID), TM(TM), LowerGEP(LowerGEP) { : FunctionPass(ID), TM(TM), LowerGEP(LowerGEP) {
initializeSeparateConstOffsetFromGEPPass(*PassRegistry::getPassRegistry()); initializeSeparateConstOffsetFromGEPPass(*PassRegistry::getPassRegistry());
} }
void getAnalysisUsage(AnalysisUsage &AU) const override { void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addRequired<DataLayoutPass>(); AU.addRequired<DataLayoutPass>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
} }
bool doInitialization(Module &M) override { bool doInitialization(Module &M) override {
DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>(); DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>();
if (DLP == nullptr) if (DLP == nullptr)
report_fatal_error("data layout missing"); report_fatal_error("data layout missing");
DL = &DLP->getDataLayout(); DL = &DLP->getDataLayout();
return false; return false;
▲ Show 20 Lines • Show All 54 Lines • ▼ Show 20 Lines
}; };
} // anonymous namespace } // anonymous namespace
char SeparateConstOffsetFromGEP::ID = 0; char SeparateConstOffsetFromGEP::ID = 0;
INITIALIZE_PASS_BEGIN( INITIALIZE_PASS_BEGIN(
SeparateConstOffsetFromGEP, "separate-const-offset-from-gep", SeparateConstOffsetFromGEP, "separate-const-offset-from-gep",
"Split GEPs to a variadic base and a constant offset for better CSE", false, "Split GEPs to a variadic base and a constant offset for better CSE", false,
false) false)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(DataLayoutPass) INITIALIZE_PASS_DEPENDENCY(DataLayoutPass)
INITIALIZE_PASS_END( INITIALIZE_PASS_END(
SeparateConstOffsetFromGEP, "separate-const-offset-from-gep", SeparateConstOffsetFromGEP, "separate-const-offset-from-gep",
"Split GEPs to a variadic base and a constant offset for better CSE", false, "Split GEPs to a variadic base and a constant offset for better CSE", false,
false) false)
FunctionPass * FunctionPass *
llvm::createSeparateConstOffsetFromGEPPass(const TargetMachine *TM, llvm::createSeparateConstOffsetFromGEPPass(const TargetMachine *TM,
▲ Show 20 Lines • Show All 456 Lines • ▼ Show 20 Linesbool SeparateConstOffsetFromGEP::splitGEP(GetElementPtrInst *GEP) {
// If LowerGEP is disabled, before really splitting the GEP, check whether the // If LowerGEP is disabled, before really splitting the GEP, check whether the
// backend supports the addressing mode we are about to produce. If no, this // backend supports the addressing mode we are about to produce. If no, this
// splitting probably won't be beneficial. // splitting probably won't be beneficial.
// If LowerGEP is enabled, even the extracted constant offset can not match // If LowerGEP is enabled, even the extracted constant offset can not match
// the addressing mode, we can still do optimizations to other lowered parts // the addressing mode, we can still do optimizations to other lowered parts
// of variable indices. Therefore, we don't check for addressing modes in that // of variable indices. Therefore, we don't check for addressing modes in that
// case. // case.
if (!LowerGEP) { if (!LowerGEP) {
TargetTransformInfo &TTI = getAnalysis<TargetTransformInfo>(); TargetTransformInfo &TTI =
getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
if (!TTI.isLegalAddressingMode(GEP->getType()->getElementType(), if (!TTI.isLegalAddressingMode(GEP->getType()->getElementType(),
/*BaseGV=*/nullptr, AccumulativeByteOffset, /*BaseGV=*/nullptr, AccumulativeByteOffset,
/*HasBaseReg=*/true, /*Scale=*/0)) { /*HasBaseReg=*/true, /*Scale=*/0)) {
return Changed; return Changed;
} }
} }
// Remove the constant offset in each sequential index. The resultant GEP // Remove the constant offset in each sequential index. The resultant GEP
▲ Show 20 Lines • Show All 145 Lines • Show Last 20 Lines

llvm/trunk/lib/Transforms/Scalar/SimplifyCFGPass.cpp

Show First 20 Lines • Show All 53 Lines • ▼ Show 20 Linesstruct CFGSimplifyPass : public FunctionPass {
CFGSimplifyPass(int T = -1) : FunctionPass(ID) { CFGSimplifyPass(int T = -1) : FunctionPass(ID) {
BonusInstThreshold = (T == -1) ? UserBonusInstThreshold : unsigned(T); BonusInstThreshold = (T == -1) ? UserBonusInstThreshold : unsigned(T);
initializeCFGSimplifyPassPass(*PassRegistry::getPassRegistry()); initializeCFGSimplifyPassPass(*PassRegistry::getPassRegistry());
} }
bool runOnFunction(Function &F) override; bool runOnFunction(Function &F) override;
void getAnalysisUsage(AnalysisUsage &AU) const override { void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addRequired<AssumptionCacheTracker>(); AU.addRequired<AssumptionCacheTracker>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
} }
}; };
} }
char CFGSimplifyPass::ID = 0; char CFGSimplifyPass::ID = 0;
INITIALIZE_PASS_BEGIN(CFGSimplifyPass, "simplifycfg", "Simplify the CFG", false, INITIALIZE_PASS_BEGIN(CFGSimplifyPass, "simplifycfg", "Simplify the CFG", false,
false) false)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker) INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker)
INITIALIZE_PASS_END(CFGSimplifyPass, "simplifycfg", "Simplify the CFG", false, INITIALIZE_PASS_END(CFGSimplifyPass, "simplifycfg", "Simplify the CFG", false,
false) false)
// Public interface to the CFGSimplification pass // Public interface to the CFGSimplification pass
FunctionPass *llvm::createCFGSimplificationPass(int Threshold) { FunctionPass *llvm::createCFGSimplificationPass(int Threshold) {
return new CFGSimplifyPass(Threshold); return new CFGSimplifyPass(Threshold);
} }
▲ Show 20 Lines • Show All 101 Lines • ▼ Show 20 Lines
// simplify the CFG. // simplify the CFG.
// //
bool CFGSimplifyPass::runOnFunction(Function &F) { bool CFGSimplifyPass::runOnFunction(Function &F) {
if (skipOptnoneFunction(F)) if (skipOptnoneFunction(F))
return false; return false;
AssumptionCache *AC = AssumptionCache *AC =
&getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F); &getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F);
const TargetTransformInfo &TTI = getAnalysis<TargetTransformInfo>(); const TargetTransformInfo &TTI =
getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>(); DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>();
const DataLayout *DL = DLP ? &DLP->getDataLayout() : nullptr; const DataLayout *DL = DLP ? &DLP->getDataLayout() : nullptr;
bool EverChanged = removeUnreachableBlocks(F); bool EverChanged = removeUnreachableBlocks(F);
EverChanged |= mergeEmptyReturnBlocks(F); EverChanged |= mergeEmptyReturnBlocks(F);
EverChanged |= iterativelySimplifyCFG(F, TTI, DL, AC, BonusInstThreshold); EverChanged |= iterativelySimplifyCFG(F, TTI, DL, AC, BonusInstThreshold);
// If neither pass changed anything, we're done. // If neither pass changed anything, we're done.
if (!EverChanged) return false; if (!EverChanged) return false;
Show All 16 Lines

llvm/trunk/lib/Transforms/Scalar/TailRecursionElimination.cpp

Show First 20 Lines • Show All 120 Lines • ▼ Show 20 Lines private:
bool CanMoveAboveCall(Instruction *I, CallInst *CI); bool CanMoveAboveCall(Instruction *I, CallInst *CI);
Value *CanTransformAccumulatorRecursion(Instruction *I, CallInst *CI); Value *CanTransformAccumulatorRecursion(Instruction *I, CallInst *CI);
}; };
} }
char TailCallElim::ID = 0; char TailCallElim::ID = 0;
INITIALIZE_PASS_BEGIN(TailCallElim, "tailcallelim", INITIALIZE_PASS_BEGIN(TailCallElim, "tailcallelim",
"Tail Call Elimination", false, false) "Tail Call Elimination", false, false)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_END(TailCallElim, "tailcallelim", INITIALIZE_PASS_END(TailCallElim, "tailcallelim",
"Tail Call Elimination", false, false) "Tail Call Elimination", false, false)
// Public interface to the TailCallElimination pass // Public interface to the TailCallElimination pass
FunctionPass *llvm::createTailCallEliminationPass() { FunctionPass *llvm::createTailCallEliminationPass() {
return new TailCallElim(); return new TailCallElim();
} }
void TailCallElim::getAnalysisUsage(AnalysisUsage &AU) const { void TailCallElim::getAnalysisUsage(AnalysisUsage &AU) const {
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
} }
/// \brief Scan the specified function for alloca instructions. /// \brief Scan the specified function for alloca instructions.
/// If it contains any dynamic allocas, returns false. /// If it contains any dynamic allocas, returns false.
static bool CanTRE(Function &F) { static bool CanTRE(Function &F) {
// Because of PR962, we don't TRE dynamic allocas. // Because of PR962, we don't TRE dynamic allocas.
for (auto &BB : F) { for (auto &BB : F) {
for (auto &I : BB) { for (auto &I : BB) {
▲ Show 20 Lines • Show All 233 Lines • ▼ Show 20 Linesbool TailCallElim::markTails(Function &F, bool &AllCallsAreTailCalls) {
return Modified; return Modified;
} }
bool TailCallElim::runTRE(Function &F) { bool TailCallElim::runTRE(Function &F) {
// If this function is a varargs function, we won't be able to PHI the args // If this function is a varargs function, we won't be able to PHI the args
// right, so don't even try to convert it... // right, so don't even try to convert it...
if (F.getFunctionType()->isVarArg()) return false; if (F.getFunctionType()->isVarArg()) return false;
TTI = &getAnalysis<TargetTransformInfo>(); TTI = &getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
BasicBlock *OldEntry = nullptr; BasicBlock *OldEntry = nullptr;
bool TailCallsAreMarkedTail = false; bool TailCallsAreMarkedTail = false;
SmallVector<PHINode*, 8> ArgumentPHIs; SmallVector<PHINode*, 8> ArgumentPHIs;
bool MadeChange = false; bool MadeChange = false;
// CanTRETailMarkedCall - If false, we cannot perform TRE on tail calls // CanTRETailMarkedCall - If false, we cannot perform TRE on tail calls
// marked with the 'tail' attribute, because doing so would cause the stack // marked with the 'tail' attribute, because doing so would cause the stack
// size to increase (real TRE would deallocate variable sized allocas, TRE // size to increase (real TRE would deallocate variable sized allocas, TRE
▲ Show 20 Lines • Show All 474 Lines • Show Last 20 Lines

llvm/trunk/lib/Transforms/Vectorize/BBVectorize.cpp

Show First 20 Lines • Show All 202 Lines • ▼ Show 20 Lines struct BBVectorize : public BasicBlockPass {
BBVectorize(Pass *P, const VectorizeConfig &C) BBVectorize(Pass *P, const VectorizeConfig &C)
: BasicBlockPass(ID), Config(C) { : BasicBlockPass(ID), Config(C) {
AA = &P->getAnalysis<AliasAnalysis>(); AA = &P->getAnalysis<AliasAnalysis>();
DT = &P->getAnalysis<DominatorTreeWrapperPass>().getDomTree(); DT = &P->getAnalysis<DominatorTreeWrapperPass>().getDomTree();
SE = &P->getAnalysis<ScalarEvolution>(); SE = &P->getAnalysis<ScalarEvolution>();
DataLayoutPass *DLP = P->getAnalysisIfAvailable<DataLayoutPass>(); DataLayoutPass *DLP = P->getAnalysisIfAvailable<DataLayoutPass>();
DL = DLP ? &DLP->getDataLayout() : nullptr; DL = DLP ? &DLP->getDataLayout() : nullptr;
TTI = IgnoreTargetInfo ? nullptr : &P->getAnalysis<TargetTransformInfo>(); TTI = IgnoreTargetInfo
? nullptr
: &P->getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
} }
typedef std::pair<Value *, Value *> ValuePair; typedef std::pair<Value *, Value *> ValuePair;
typedef std::pair<ValuePair, int> ValuePairWithCost; typedef std::pair<ValuePair, int> ValuePairWithCost;
typedef std::pair<ValuePair, size_t> ValuePairWithDepth; typedef std::pair<ValuePair, size_t> ValuePairWithDepth;
typedef std::pair<ValuePair, ValuePair> VPPair; // A ValuePair pair typedef std::pair<ValuePair, ValuePair> VPPair; // A ValuePair pair
typedef std::pair<VPPair, unsigned> VPPairWithType; typedef std::pair<VPPair, unsigned> VPPairWithType;
▲ Show 20 Lines • Show All 217 Lines • ▼ Show 20 Lines void computePairsConnectedTo(
bool runOnBasicBlock(BasicBlock &BB) override { bool runOnBasicBlock(BasicBlock &BB) override {
// OptimizeNone check deferred to vectorizeBB(). // OptimizeNone check deferred to vectorizeBB().
AA = &getAnalysis<AliasAnalysis>(); AA = &getAnalysis<AliasAnalysis>();
DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree(); DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree();
SE = &getAnalysis<ScalarEvolution>(); SE = &getAnalysis<ScalarEvolution>();
DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>(); DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>();
DL = DLP ? &DLP->getDataLayout() : nullptr; DL = DLP ? &DLP->getDataLayout() : nullptr;
TTI = IgnoreTargetInfo ? nullptr : &getAnalysis<TargetTransformInfo>(); TTI = IgnoreTargetInfo
? nullptr
: &getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
return vectorizeBB(BB); return vectorizeBB(BB);
} }
void getAnalysisUsage(AnalysisUsage &AU) const override { void getAnalysisUsage(AnalysisUsage &AU) const override {
BasicBlockPass::getAnalysisUsage(AU); BasicBlockPass::getAnalysisUsage(AU);
AU.addRequired<AliasAnalysis>(); AU.addRequired<AliasAnalysis>();
AU.addRequired<DominatorTreeWrapperPass>(); AU.addRequired<DominatorTreeWrapperPass>();
AU.addRequired<ScalarEvolution>(); AU.addRequired<ScalarEvolution>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
AU.addPreserved<AliasAnalysis>(); AU.addPreserved<AliasAnalysis>();
AU.addPreserved<DominatorTreeWrapperPass>(); AU.addPreserved<DominatorTreeWrapperPass>();
AU.addPreserved<ScalarEvolution>(); AU.addPreserved<ScalarEvolution>();
AU.setPreservesCFG(); AU.setPreservesCFG();
} }
static inline VectorType *getVecTypeForPair(Type *ElemTy, Type *Elem2Ty) { static inline VectorType *getVecTypeForPair(Type *ElemTy, Type *Elem2Ty) {
assert(ElemTy->getScalarType() == Elem2Ty->getScalarType() && assert(ElemTy->getScalarType() == Elem2Ty->getScalarType() &&
▲ Show 20 Lines • Show All 2,723 Lines • ▼ Show 20 Lines void BBVectorize::fuseChosenPairs(BasicBlock &BB,
DEBUG(dbgs() << "BBV: final: \n" << BB << "\n"); DEBUG(dbgs() << "BBV: final: \n" << BB << "\n");
} }
} }
char BBVectorize::ID = 0; char BBVectorize::ID = 0;
static const char bb_vectorize_name[] = "Basic-Block Vectorization"; static const char bb_vectorize_name[] = "Basic-Block Vectorization";
INITIALIZE_PASS_BEGIN(BBVectorize, BBV_NAME, bb_vectorize_name, false, false) INITIALIZE_PASS_BEGIN(BBVectorize, BBV_NAME, bb_vectorize_name, false, false)
INITIALIZE_AG_DEPENDENCY(AliasAnalysis) INITIALIZE_AG_DEPENDENCY(AliasAnalysis)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass) INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)
INITIALIZE_PASS_DEPENDENCY(ScalarEvolution) INITIALIZE_PASS_DEPENDENCY(ScalarEvolution)
INITIALIZE_PASS_END(BBVectorize, BBV_NAME, bb_vectorize_name, false, false) INITIALIZE_PASS_END(BBVectorize, BBV_NAME, bb_vectorize_name, false, false)
BasicBlockPass *llvm::createBBVectorizePass(const VectorizeConfig &C) { BasicBlockPass *llvm::createBBVectorizePass(const VectorizeConfig &C) {
return new BBVectorize(C); return new BBVectorize(C);
} }
Show All 33 Lines

llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp

Show First 20 Lines • Show All 1,324 Lines • ▼ Show 20 Linesstruct LoopVectorize : public FunctionPass {
BlockFrequency ColdEntryFreq; BlockFrequency ColdEntryFreq;
bool runOnFunction(Function &F) override { bool runOnFunction(Function &F) override {
SE = &getAnalysis<ScalarEvolution>(); SE = &getAnalysis<ScalarEvolution>();
DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>(); DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>();
DL = DLP ? &DLP->getDataLayout() : nullptr; DL = DLP ? &DLP->getDataLayout() : nullptr;
LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo(); LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo();
TTI = &getAnalysis<TargetTransformInfo>(); TTI = &getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree(); DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree();
BFI = &getAnalysis<BlockFrequencyInfo>(); BFI = &getAnalysis<BlockFrequencyInfo>();
auto *TLIP = getAnalysisIfAvailable<TargetLibraryInfoWrapperPass>(); auto *TLIP = getAnalysisIfAvailable<TargetLibraryInfoWrapperPass>();
TLI = TLIP ? &TLIP->getTLI() : nullptr; TLI = TLIP ? &TLIP->getTLI() : nullptr;
AA = &getAnalysis<AliasAnalysis>(); AA = &getAnalysis<AliasAnalysis>();
AC = &getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F); AC = &getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F);
// Compute some weights outside of the loop over the loops. Compute this // Compute some weights outside of the loop over the loops. Compute this
▲ Show 20 Lines • Show All 203 Lines • ▼ Show 20 Lines#endif /* NDEBUG */
void getAnalysisUsage(AnalysisUsage &AU) const override { void getAnalysisUsage(AnalysisUsage &AU) const override {
AU.addRequired<AssumptionCacheTracker>(); AU.addRequired<AssumptionCacheTracker>();
AU.addRequiredID(LoopSimplifyID); AU.addRequiredID(LoopSimplifyID);
AU.addRequiredID(LCSSAID); AU.addRequiredID(LCSSAID);
AU.addRequired<BlockFrequencyInfo>(); AU.addRequired<BlockFrequencyInfo>();
AU.addRequired<DominatorTreeWrapperPass>(); AU.addRequired<DominatorTreeWrapperPass>();
AU.addRequired<LoopInfoWrapperPass>(); AU.addRequired<LoopInfoWrapperPass>();
AU.addRequired<ScalarEvolution>(); AU.addRequired<ScalarEvolution>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
AU.addRequired<AliasAnalysis>(); AU.addRequired<AliasAnalysis>();
AU.addPreserved<LoopInfoWrapperPass>(); AU.addPreserved<LoopInfoWrapperPass>();
AU.addPreserved<DominatorTreeWrapperPass>(); AU.addPreserved<DominatorTreeWrapperPass>();
AU.addPreserved<AliasAnalysis>(); AU.addPreserved<AliasAnalysis>();
} }
}; };
▲ Show 20 Lines • Show All 4,585 Lines • ▼ Show 20 LinesType* LoopVectorizationCostModel::ToVectorTy(Type *Scalar, unsigned VF) {
if (Scalar->isVoidTy() || VF == 1) if (Scalar->isVoidTy() || VF == 1)
return Scalar; return Scalar;
return VectorType::get(Scalar, VF); return VectorType::get(Scalar, VF);
} }
char LoopVectorize::ID = 0; char LoopVectorize::ID = 0;
static const char lv_name[] = "Loop Vectorization"; static const char lv_name[] = "Loop Vectorization";
INITIALIZE_PASS_BEGIN(LoopVectorize, LV_NAME, lv_name, false, false) INITIALIZE_PASS_BEGIN(LoopVectorize, LV_NAME, lv_name, false, false)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_AG_DEPENDENCY(AliasAnalysis) INITIALIZE_AG_DEPENDENCY(AliasAnalysis)
INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker) INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker)
INITIALIZE_PASS_DEPENDENCY(BlockFrequencyInfo) INITIALIZE_PASS_DEPENDENCY(BlockFrequencyInfo)
INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass) INITIALIZE_PASS_DEPENDENCY(DominatorTreeWrapperPass)
INITIALIZE_PASS_DEPENDENCY(ScalarEvolution) INITIALIZE_PASS_DEPENDENCY(ScalarEvolution)
INITIALIZE_PASS_DEPENDENCY(LCSSA) INITIALIZE_PASS_DEPENDENCY(LCSSA)
INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass) INITIALIZE_PASS_DEPENDENCY(LoopInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(LoopSimplify) INITIALIZE_PASS_DEPENDENCY(LoopSimplify)
▲ Show 20 Lines • Show All 154 Lines • Show Last 20 Lines

llvm/trunk/lib/Transforms/Vectorize/SLPVectorizer.cpp

Show First 20 Lines • Show All 3,046 Lines • ▼ Show 20 Linesstruct SLPVectorizer : public FunctionPass {
bool runOnFunction(Function &F) override { bool runOnFunction(Function &F) override {
if (skipOptnoneFunction(F)) if (skipOptnoneFunction(F))
return false; return false;
SE = &getAnalysis<ScalarEvolution>(); SE = &getAnalysis<ScalarEvolution>();
DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>(); DataLayoutPass *DLP = getAnalysisIfAvailable<DataLayoutPass>();
DL = DLP ? &DLP->getDataLayout() : nullptr; DL = DLP ? &DLP->getDataLayout() : nullptr;
TTI = &getAnalysis<TargetTransformInfo>(); TTI = &getAnalysis<TargetTransformInfoWrapperPass>().getTTI();
auto *TLIP = getAnalysisIfAvailable<TargetLibraryInfoWrapperPass>(); auto *TLIP = getAnalysisIfAvailable<TargetLibraryInfoWrapperPass>();
TLI = TLIP ? &TLIP->getTLI() : nullptr; TLI = TLIP ? &TLIP->getTLI() : nullptr;
AA = &getAnalysis<AliasAnalysis>(); AA = &getAnalysis<AliasAnalysis>();
LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo(); LI = &getAnalysis<LoopInfoWrapperPass>().getLoopInfo();
DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree(); DT = &getAnalysis<DominatorTreeWrapperPass>().getDomTree();
AC = &getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F); AC = &getAnalysis<AssumptionCacheTracker>().getAssumptionCache(F);
StoreRefs.clear(); StoreRefs.clear();
▲ Show 20 Lines • Show All 45 Lines • ▼ Show 20 Lines bool runOnFunction(Function &F) override {
return Changed; return Changed;
} }
void getAnalysisUsage(AnalysisUsage &AU) const override { void getAnalysisUsage(AnalysisUsage &AU) const override {
FunctionPass::getAnalysisUsage(AU); FunctionPass::getAnalysisUsage(AU);
AU.addRequired<AssumptionCacheTracker>(); AU.addRequired<AssumptionCacheTracker>();
AU.addRequired<ScalarEvolution>(); AU.addRequired<ScalarEvolution>();
AU.addRequired<AliasAnalysis>(); AU.addRequired<AliasAnalysis>();
AU.addRequired<TargetTransformInfo>(); AU.addRequired<TargetTransformInfoWrapperPass>();
AU.addRequired<LoopInfoWrapperPass>(); AU.addRequired<LoopInfoWrapperPass>();
AU.addRequired<DominatorTreeWrapperPass>(); AU.addRequired<DominatorTreeWrapperPass>();
AU.addPreserved<LoopInfoWrapperPass>(); AU.addPreserved<LoopInfoWrapperPass>();
AU.addPreserved<DominatorTreeWrapperPass>(); AU.addPreserved<DominatorTreeWrapperPass>();
AU.setPreservesCFG(); AU.setPreservesCFG();
} }
private: private:
▲ Show 20 Lines • Show All 868 Lines • ▼ Show 20 Lines
} }
} // end anonymous namespace } // end anonymous namespace
char SLPVectorizer::ID = 0; char SLPVectorizer::ID = 0;
static const char lv_name[] = "SLP Vectorizer"; static const char lv_name[] = "SLP Vectorizer";
INITIALIZE_PASS_BEGIN(SLPVectorizer, SV_NAME, lv_name, false, false) INITIALIZE_PASS_BEGIN(SLPVectorizer, SV_NAME, lv_name, false, false)
INITIALIZE_AG_DEPENDENCY(AliasAnalysis) INITIALIZE_AG_DEPENDENCY(AliasAnalysis)
INITIALIZE_AG_DEPENDENCY(TargetTransformInfo) INITIALIZE_PASS_DEPENDENCY(TargetTransformInfoWrapperPass)
INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker) INITIALIZE_PASS_DEPENDENCY(AssumptionCacheTracker)
INITIALIZE_PASS_DEPENDENCY(ScalarEvolution) INITIALIZE_PASS_DEPENDENCY(ScalarEvolution)
INITIALIZE_PASS_DEPENDENCY(LoopSimplify) INITIALIZE_PASS_DEPENDENCY(LoopSimplify)
INITIALIZE_PASS_END(SLPVectorizer, SV_NAME, lv_name, false, false) INITIALIZE_PASS_END(SLPVectorizer, SV_NAME, lv_name, false, false)
namespace llvm { namespace llvm {
Pass *createSLPVectorizerPass() { return new SLPVectorizer(); } Pass *createSLPVectorizerPass() { return new SLPVectorizer(); }
} }

llvm/trunk/test/Analysis/CostModel/no_info.ll

; RUN: opt < %s -cost-model -analyze | FileCheck %s ; RUN: opt < %s -cost-model -analyze | FileCheck %s
; The cost model does not have any target information so it can't make a decision. ; The cost model does not have any target information so it just makes boring
; assumptions.
; -- No triple in this module -- ; -- No triple in this module --
;CHECK: Unknown cost {{.*}} add ;CHECK: cost of 1 {{.*}} add
;CHECK: Unknown cost {{.*}} ret ;CHECK: cost of 1 {{.*}} ret
define i32 @no_info(i32 %arg) { define i32 @no_info(i32 %arg) {
%e = add i32 %arg, %arg %e = add i32 %arg, %arg
ret i32 %e ret i32 %e
} }

llvm/trunk/tools/opt/opt.cpp

Show All 15 Lines
#include "NewPMDriver.h" #include "NewPMDriver.h"
#include "PassPrinters.h" #include "PassPrinters.h"
#include "llvm/ADT/Triple.h" #include "llvm/ADT/Triple.h"
#include "llvm/Analysis/CallGraph.h" #include "llvm/Analysis/CallGraph.h"
#include "llvm/Analysis/CallGraphSCCPass.h" #include "llvm/Analysis/CallGraphSCCPass.h"
#include "llvm/Analysis/LoopPass.h" #include "llvm/Analysis/LoopPass.h"
#include "llvm/Analysis/RegionPass.h" #include "llvm/Analysis/RegionPass.h"
#include "llvm/Analysis/TargetLibraryInfo.h" #include "llvm/Analysis/TargetLibraryInfo.h"
#include "llvm/Analysis/TargetTransformInfo.h"
#include "llvm/Bitcode/BitcodeWriterPass.h" #include "llvm/Bitcode/BitcodeWriterPass.h"
#include "llvm/CodeGen/CommandFlags.h" #include "llvm/CodeGen/CommandFlags.h"
#include "llvm/IR/DataLayout.h" #include "llvm/IR/DataLayout.h"
#include "llvm/IR/IRPrintingPasses.h" #include "llvm/IR/IRPrintingPasses.h"
#include "llvm/IR/LLVMContext.h" #include "llvm/IR/LLVMContext.h"
#include "llvm/IR/LegacyPassNameParser.h" #include "llvm/IR/LegacyPassNameParser.h"
#include "llvm/IR/Module.h" #include "llvm/IR/Module.h"
#include "llvm/IR/Verifier.h" #include "llvm/IR/Verifier.h"
▲ Show 20 Lines • Show All 391 Lines • ▼ Show 20 Lines#endif
TargetMachine *Machine = nullptr; TargetMachine *Machine = nullptr;
if (ModuleTriple.getArch()) if (ModuleTriple.getArch())
Machine = GetTargetMachine(Triple(ModuleTriple)); Machine = GetTargetMachine(Triple(ModuleTriple));
std::unique_ptr<TargetMachine> TM(Machine); std::unique_ptr<TargetMachine> TM(Machine);
// Add internal analysis passes from the target machine. // Add internal analysis passes from the target machine.
if (TM) if (TM)
TM->addAnalysisPasses(Passes); TM->addAnalysisPasses(Passes);
else
Passes.add(createNoTargetTransformInfoPass(DL));
std::unique_ptr<FunctionPassManager> FPasses; std::unique_ptr<FunctionPassManager> FPasses;
if (OptLevelO1 || OptLevelO2 || OptLevelOs || OptLevelOz || OptLevelO3) { if (OptLevelO1 || OptLevelO2 || OptLevelOs || OptLevelOz || OptLevelO3) {
FPasses.reset(new FunctionPassManager(M.get())); FPasses.reset(new FunctionPassManager(M.get()));
if (DL) if (DL)
FPasses->add(new DataLayoutPass()); FPasses->add(new DataLayoutPass());
if (TM) if (TM)
TM->addAnalysisPasses(*FPasses); TM->addAnalysisPasses(*FPasses);
else
FPasses->add(createNoTargetTransformInfoPass(DL));
} }
if (PrintBreakpoints) { if (PrintBreakpoints) {
// Default to standard output. // Default to standard output.
if (!Out) { if (!Out) {
if (OutputFilename.empty()) if (OutputFilename.empty())
OutputFilename = "-"; OutputFilename = "-";
▲ Show 20 Lines • Show All 144 Lines • Show Last 20 Lines