[TTI] Added TTI pass queries for max load/store-per-memory-intrinsic.

[hussainjk]

Summary

Added these query functions to the TargetTransformInfo pass and related support classes:
getMaxStoresPerMemset, getMaxStoresPerMemcpy, getMaxGluedStoresPerMemcpy, getMaxExpandSizeMemcmp, getMaxStoresPerMemmove;
they all wrap a call to the corresponding function in TargetLoweringBase.

I will be using getMaxStoresPerMemcpy and getMaxStoresPerMemset to condition a loop optimization that unrolls a memcpy or memset-patterned loop of unknown but bounded length, up to a maximum threshold on the bound which will be determined by these values. I included the rest for the sake of completion.

[arsenm]

llvm/include/llvm/Analysis/TargetTransformInfo.h

887

Needs an address space

895

Needs source and dest address spaces. The alignment is also a consideration

899–901

I don't understand what I means by glue here. Like SelectionDAG glue? I don't think this is a useful concept to expose to the IR

[hussainjk]

llvm/include/llvm/Analysis/TargetTransformInfo.h

887

The underlying function in TargetLoweringBase -- which this wraps -- doesn't take addressing arguments.

895

Likewise.

899–901

AFAIK this refers to groups of stores getting chained together within the memcpy expansion, with there being no guarantees otherwise about consecutive scheduling.

[hussainjk]

ping

[arsenm]

llvm/include/llvm/Analysis/TargetTransformInfo.h

887

That's another problem that needs to be solved

[nemanjai]

I realize that without any uses of these functions, we can't really add a test case. However, can we at least get a description of why this is useful in the middle end if this patch won't add any users of these functions?

Generally, I am a bit apprehensive about adding functions without callers if we don't have a clear roadmap to when/how these will be used.

[hussainjk]

In D70108#1802747, @nemanjai wrote:

I realize that without any uses of these functions, we can't really add a test case. However, can we at least get a description of why this is useful in the middle end if this patch won't add any users of these functions?

Generally, I am a bit apprehensive about adding functions without callers if we don't have a clear roadmap to when/how these will be used.

I have a pending loop optimization where I want to use some of these values (the memset and memcpy ones) for cost modeling. I included the others because they are lumped together, and it seemed like it would be incomplete to only include the two I need.

[arsenm]

Still needs address space parameters

llvm/include/llvm/Analysis/TargetTransformInfo.h

899–901

This should avoid referring to SelectionDAG specifics

[hussainjk]

llvm/include/llvm/Analysis/TargetTransformInfo.h

887

I consulted with @nemanjai on the best way to resolve this; I will be adding address space arguments with default 0 value to the underlying functions in a separate patch after this gets merged, since it's nontrivial and out of scope for this patch (it involves updating function declarations in several different targets at least). I would then add the address space arguments to these new functions in that patch.

[hussainjk]

@nemanjai I amended the description with some detail on the work I will be using this in, as discussed. LMK if it should be more detailed.

llvm/include/llvm/Analysis/TargetTransformInfo.h

public:
   /// i.e. prefetch with any stride.
   unsigned getMinPrefetchStride() const;
 
   /// \return The maximum number of iterations to prefetch ahead.  If
   /// the required number of iterations is more than this number, no
   /// prefetching is performed.
   unsigned getMaxPrefetchIterationsAhead() const;
 
+  /// \brief Get maximum # of store operations permitted for llvm.memset
+  ///
+  /// This function returns the maximum number of store operations permitted
+  /// to replace a call to llvm.memset. The value is set by the target at the
+  /// performance threshold for such a replacement. If OptSize is true,
+  /// return the limit for functions that have OptSize attribute.
+  unsigned getMaxStoresPerMemset(bool OptSize) const;
+
+  /// \brief Get maximum # of store operations permitted for llvm.memcpy
+  ///
+  /// This function returns the maximum number of store operations permitted
+  /// to replace a call to llvm.memcpy. The value is set by the target at the
+  /// performance threshold for such a replacement. If OptSize is true,
+  /// return the limit for functions that have OptSize attribute.
+  unsigned getMaxStoresPerMemcpy(bool OptSize) const;
+
+  /// \brief Get maximum # of store operations to be glued together
+  ///
+  /// This function returns the maximum number of store operations permitted
+  /// to glue together during lowering of llvm.memcpy. The value is set by
+  //  the target at the performance threshold for such a replacement.
+  unsigned getMaxGluedStoresPerMemcpy() const;
+
+  /// \brief Get maximum # of load operations permitted for memcmp
+  ///
+  /// This function returns the maximum number of load operations permitted
+  /// to replace a call to memcmp. The value is set by the target at the
+  /// performance threshold for such a replacement. If OptSize is true,
+  /// return the limit for functions that have OptSize attribute.
+  unsigned getMaxExpandSizeMemcmp(bool OptSize) const;
+
+  /// \brief Get maximum # of store operations permitted for llvm.memmove
+  ///
+  /// This function returns the maximum number of store operations permitted
+  /// to replace a call to llvm.memmove. The value is set by the target at the
+  /// performance threshold for such a replacement. If OptSize is true,
+  /// return the limit for functions that have OptSize attribute.
+  unsigned getMaxStoresPerMemmove(bool OptSize) const;
+
   /// \return The maximum interleave factor that any transform should try to
   /// perform for this target. This number depends on the level of parallelism
   /// and the number of execution units in the CPU.
   unsigned getMaxInterleaveFactor(unsigned VF) const;
 
   /// Collect properties of V used in cost analysis, e.g. OP_PowerOf2.
   static OperandValueKind getOperandInfo(Value *V,
                                          OperandValueProperties &OpProps);
public:
   /// i.e. prefetch with any stride.
   virtual unsigned getMinPrefetchStride() const = 0;
 
   /// \return The maximum number of iterations to prefetch ahead.  If
   /// the required number of iterations is more than this number, no
   /// prefetching is performed.
   virtual unsigned getMaxPrefetchIterationsAhead() const = 0;
 
+  virtual unsigned getMaxStoresPerMemset(bool OptSize) const = 0;
+  virtual unsigned getMaxStoresPerMemcpy(bool OptSize) const = 0;
+  virtual unsigned getMaxGluedStoresPerMemcpy() const = 0;
+  virtual unsigned getMaxExpandSizeMemcmp(bool OptSize) const = 0;
+  virtual unsigned getMaxStoresPerMemmove(bool OptSize) const = 0;
+
   virtual unsigned getMaxInterleaveFactor(unsigned VF) = 0;
   virtual unsigned
   getArithmeticInstrCost(unsigned Opcode, Type *Ty, OperandValueKind Opd1Info,
                          OperandValueKind Opd2Info,
                          OperandValueProperties Opd1PropInfo,
                          OperandValueProperties Opd2PropInfo,
                          ArrayRef<const Value *> Args) = 0;
   virtual int getShuffleCost(ShuffleKind Kind, Type *Tp, int Index,
public:
 
   /// Return the maximum prefetch distance in terms of loop
   /// iterations.
   ///
   unsigned getMaxPrefetchIterationsAhead() const override {
     return Impl.getMaxPrefetchIterationsAhead();
   }
 
+  unsigned getMaxStoresPerMemset(bool OptSize) const override {
+    return Impl.getMaxStoresPerMemset(OptSize);
+  }
+
+  unsigned getMaxStoresPerMemcpy(bool OptSize) const override {
+    return Impl.getMaxStoresPerMemcpy(OptSize);
+  }
+
+  unsigned getMaxGluedStoresPerMemcpy() const override {
+    return Impl.getMaxGluedStoresPerMemcpy();
+  }
+
+  unsigned getMaxExpandSizeMemcmp(bool OptSize) const override {
+    return Impl.getMaxExpandSizeMemcmp(OptSize);
+  }
+
+  unsigned getMaxStoresPerMemmove(bool OptSize) const override {
+    return Impl.getMaxStoresPerMemmove(OptSize);
+  }
+
   unsigned getMaxInterleaveFactor(unsigned VF) override {
     return Impl.getMaxInterleaveFactor(VF);
   }
   unsigned getEstimatedNumberOfCaseClusters(const SwitchInst &SI,
                                             unsigned &JTSize,
                                             ProfileSummaryInfo *PSI,
                                             BlockFrequencyInfo *BFI) override {
     return Impl.getEstimatedNumberOfCaseClusters(SI, JTSize, PSI, BFI);

llvm/include/llvm/Analysis/TargetTransformInfoImpl.h

llvm::Optional<unsigned> getCacheAssociativity(
 
     llvm_unreachable("Unknown TargetTransformInfo::CacheLevel");
   }
 
   unsigned getPrefetchDistance() const { return 0; }
   unsigned getMinPrefetchStride() const { return 1; }
   unsigned getMaxPrefetchIterationsAhead() const { return UINT_MAX; }
 
+  unsigned getMaxStoresPerMemset(bool OptSize) const { return 0; }
+  unsigned getMaxStoresPerMemcpy(bool OptSize) const { return 0; }
+  unsigned getMaxGluedStoresPerMemcpy() const { return 0; }
+  unsigned getMaxExpandSizeMemcmp(bool OptSize) const { return 0; }
+  unsigned getMaxStoresPerMemmove(bool OptSize) const { return 0; }
+
   unsigned getMaxInterleaveFactor(unsigned VF) { return 1; }
 
   unsigned getArithmeticInstrCost(unsigned Opcode, Type *Ty,
                                   TTI::OperandValueKind Opd1Info,
                                   TTI::OperandValueKind Opd2Info,
                                   TTI::OperandValueProperties Opd1PropInfo,
                                   TTI::OperandValueProperties Opd2PropInfo,
                                   ArrayRef<const Value *> Args) {

llvm/include/llvm/CodeGen/BasicTTIImpl.h

public:
   virtual unsigned getMinPrefetchStride() const {
     return getST()->getMinPrefetchStride();
   }
 
   virtual unsigned getMaxPrefetchIterationsAhead() const {
     return getST()->getMaxPrefetchIterationsAhead();
   }
 
+  virtual unsigned getMaxStoresPerMemset(bool OptSize) const {
+    return getTLI()->getMaxStoresPerMemset(OptSize);
+  }
+
+  virtual unsigned getMaxStoresPerMemcpy(bool OptSize) const {
+    return getTLI()->getMaxStoresPerMemcpy(OptSize);
+  }
+
+  virtual unsigned getMaxGluedStoresPerMemcpy() const {
+    return getTLI()->getMaxGluedStoresPerMemcpy();
+  }
+
+  virtual unsigned getMaxExpandSizeMemcmp(bool OptSize) const {
+    return getTLI()->getMaxExpandSizeMemcmp(OptSize);
+  }
+
+  virtual unsigned getMaxStoresPerMemmove(bool OptSize) const {
+    return getTLI()->getMaxStoresPerMemmove(OptSize);
+  }
+
   /// @}
 
   /// \name Vector TTI Implementations
   /// @{
 
   unsigned getRegisterBitWidth(bool Vector) const { return 32; }
 
   /// Estimate the overhead of scalarizing an instruction. Insert and Extract

llvm/lib/Analysis/TargetTransformInfo.cpp

 unsigned TargetTransformInfo::getMinPrefetchStride() const {
   return TTIImpl->getMinPrefetchStride();
 }
 
 unsigned TargetTransformInfo::getMaxPrefetchIterationsAhead() const {
   return TTIImpl->getMaxPrefetchIterationsAhead();
 }
 
+unsigned TargetTransformInfo::getMaxStoresPerMemset(bool OptSize) const {
+  return TTIImpl->getMaxStoresPerMemset(OptSize);
+}
+
+unsigned TargetTransformInfo::getMaxStoresPerMemcpy(bool OptSize) const {
+  return TTIImpl->getMaxStoresPerMemcpy(OptSize);
+}
+
+unsigned TargetTransformInfo::getMaxGluedStoresPerMemcpy() const {
+  return TTIImpl->getMaxGluedStoresPerMemcpy();
+}
+
+unsigned TargetTransformInfo::getMaxExpandSizeMemcmp(bool OptSize) const {
+  return TTIImpl->getMaxExpandSizeMemcmp(OptSize);
+}
+
+unsigned TargetTransformInfo::getMaxStoresPerMemmove(bool OptSize) const {
+  return TTIImpl->getMaxStoresPerMemmove(OptSize);
+}
+
 unsigned TargetTransformInfo::getMaxInterleaveFactor(unsigned VF) const {
   return TTIImpl->getMaxInterleaveFactor(VF);
 }
 
 TargetTransformInfo::OperandValueKind
 TargetTransformInfo::getOperandInfo(Value *V, OperandValueProperties &OpProps) {
   OperandValueKind OpInfo = OK_AnyValue;
   OpProps = OP_None;