ArgumentPromotion: Drop sret attribute on functions that are only called directly.

[pcc]

Summary

If the first argument to a function is a 'this' argument and the second
has the sret attribute, the ArgumentPromotion pass may promote the 'this'
argument to more than one argument, violating the IR constraint that 'sret'
may only be applied to the first or second argument.

Although this IR constraint is arguably unnecessary, it highlighted the fact
that ArgPromotion does not need to preserve this attribute. Dropping the
attribute reduces register pressure in the backend by avoiding the register
copy required by sret. Because sret implies noalias, we also replace the
former with the latter.

[rnk]

First, the limitation that sret be parameter 1 or 2 is artificial. At the time, I didn't see how it could appear on any other parameter and I tried to make the most minimal change. Now we have a reason to generalize to other parameters. That said, all the backends handle sret appearing on any parameter, because the first parameter can be split up into multiple MVTs. Consider passing i64 as the first parameter on a 32-bit platform. So, we could lift the verifier limitation.

Second, could argpromotion just drop sret in this case? sret does two things: it's equivalent to noalias, and it ensures that the sret pointer is returned in eax. If all uses of the function are internal, dropping sret is conservatively correct and reduces register pressure. We don't leverage that eax return because it would be an ABI break with old LLVMs, so we can replace sret with noalias without losing any power.

Finally, if you like having argpromotion canonicalize sret to be the first parameter, I guess that's also kind of nice, so we could continue on this way. :)

lib/IR/Function.cpp

157–158 ↗

(On Diff #27423)

Oops, that's just a bug.

[pcc]

Second, could argpromotion just drop sret in this case? sret does two things: it's equivalent to noalias, and it ensures that the sret pointer is returned in eax. If all uses of the function are internal, dropping sret is conservatively correct and reduces register pressure. We don't leverage that eax return because it would be an ABI break with old LLVMs, so we can replace sret with noalias without losing any power.

I agree, that seems like a much better solution.

[pcc]

New patch

[rnk]

lgtm

This might be an interesting change, since we'll basically remove sret from all directly called functions in LTO builds, even if we weren't going to do arg promotion. I'm pretty confident it's safe, though.

lib/Transforms/IPO/ArgumentPromotion.cpp

258–262 ↗

(On Diff #27462)

I think this is inefficient because we end up with lots of temporary attribute sets, but whatever. It's kind of an API deficiency. =/

The alternative would be to compute the set of parameters that are losing sret and feed that into DoPromotion, which is ugly.

llvm/trunk/lib/IR/Function.cpp

bool Argument::hasNoCaptureAttr() const {
   return getParent()->getAttributes().
     hasAttribute(getArgNo()+1, Attribute::NoCapture);
 }
 
 /// hasSRetAttr - Return true if this argument has the sret attribute on
 /// it in its containing function.
 bool Argument::hasStructRetAttr() const {
   if (!getType()->isPointerTy()) return false;
-  if (this != getParent()->arg_begin())
-    return false; // StructRet param must be first param
   return getParent()->getAttributes().
-    hasAttribute(1, Attribute::StructRet);
+    hasAttribute(getArgNo()+1, Attribute::StructRet);
 }
 
 /// hasReturnedAttr - Return true if this argument has the returned attribute on
 /// it in its containing function.
 bool Argument::hasReturnedAttr() const {
   return getParent()->getAttributes().
     hasAttribute(getArgNo()+1, Attribute::Returned);
 }

llvm/trunk/lib/Transforms/IPO/ArgumentPromotion.cpp

CallGraphNode *ArgPromotion::PromoteArguments(CallGraphNode *CGN) {
   // Check to see which arguments are promotable.  If an argument is promotable,
   // add it to ArgsToPromote.
   SmallPtrSet<Argument*, 8> ArgsToPromote;
   SmallPtrSet<Argument*, 8> ByValArgsToTransform;
   for (unsigned i = 0, e = PointerArgs.size(); i != e; ++i) {
     Argument *PtrArg = PointerArgs[i];
     Type *AgTy = cast<PointerType>(PtrArg->getType())->getElementType();
 
+    // Replace sret attribute with noalias. This reduces register pressure by
+    // avoiding a register copy.
+    if (PtrArg->hasStructRetAttr()) {
+      unsigned ArgNo = PtrArg->getArgNo();
+      F->setAttributes(
+          F->getAttributes()
+              .removeAttribute(F->getContext(), ArgNo + 1, Attribute::StructRet)
+              .addAttribute(F->getContext(), ArgNo + 1, Attribute::NoAlias));
+      for (Use &U : F->uses()) {
+        CallSite CS(U.getUser());
+        CS.setAttributes(
+            CS.getAttributes()
+                .removeAttribute(F->getContext(), ArgNo + 1,
+                                 Attribute::StructRet)
+                .addAttribute(F->getContext(), ArgNo + 1, Attribute::NoAlias));
+      }
+    }
+
     // If this is a byval argument, and if the aggregate type is small, just
     // pass the elements, which is always safe, if the passed value is densely
     // packed or if we can prove the padding bytes are never accessed. This does
     // not apply to inalloca.
     bool isSafeToPromote =
         PtrArg->hasByValAttr() &&
         (isDenselyPacked(AgTy, DL) || !canPaddingBeAccessed(PtrArg));
     if (isSafeToPromote) {

llvm/trunk/test/Transforms/ArgumentPromotion/sret.ll

+; RUN: opt < %s -argpromotion -S | FileCheck %s
+
+target datalayout = "e-m:w-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-pc-windows-msvc"
+
+; CHECK: define internal void @add(i32 %[[THIS1:.*]], i32 %[[THIS2:.*]], i32* noalias %[[SR:.*]])
+define internal void @add({i32, i32}* %this, i32* sret %r) {
+  %ap = getelementptr {i32, i32}, {i32, i32}* %this, i32 0, i32 0
+  %bp = getelementptr {i32, i32}, {i32, i32}* %this, i32 0, i32 1
+  %a = load i32, i32* %ap
+  %b = load i32, i32* %bp
+  ; CHECK: %[[AB:.*]] = add i32 %[[THIS1]], %[[THIS2]]
+  %ab = add i32 %a, %b
+  ; CHECK: store i32 %[[AB]], i32* %[[SR]]
+  store i32 %ab, i32* %r
+  ret void
+}
+
+; CHECK: define void @f()
+define void @f() {
+  ; CHECK: %[[R:.*]] = alloca i32
+  %r = alloca i32
+  %pair = alloca {i32, i32}
+
+  ; CHECK: call void @add(i32 %{{.*}}, i32 %{{.*}}, i32* noalias %[[R]])
+  call void @add({i32, i32}* %pair, i32* sret %r)
+  ret void
+}