[Attributor] Function signature rewrite infrastructure

[jdoerfert]

Summary

As part of the Attributor manifest we want to change the signature of
functions. This patch introduces a fairly generic interface to do so.
As a first, very simple, use case, we remove unused arguments. A second
use case, pointer privatization, will be committed with this patch as
well.

A lot of the code and ideas are taken from argument promotion.

[jdoerfert]

Lessons learned from running *all* argument promotion tests, patches will follow

[jdoerfert]

Reuse some argument promotion tests with dead arguments

[jdoerfert]

Update tests

[jdoerfert]

Remove spurios changes to tests

[jdoerfert]

ping

[uenoku]

Sorry for the delay. Basically it looks good to me regarding the logic.
I think you need to separate diff for argument promotion test changes or wait for reaching on an agreement in D69748.

llvm/lib/Transforms/IPO/Attributor.cpp

5586–5605

Replaced/Replacement is a bit confusing. Could you change to Old/New or more appropriate words?

[jdoerfert]

In D68765#1738212, @uenoku wrote:

Sorry for the delay. Basically it looks good to me regarding the logic.

Let me know if there is anything else. I would update this for the final version based on the smaller changes you requested.

I think you need to separate diff for argument promotion test changes or wait for reaching on an agreement in D69748.

I'll copy the tests over into a new Attributor test folder.

llvm/lib/Transforms/IPO/Attributor.cpp

5586–5605

Will do.

[jdoerfert]

Rebase, addressed comment

[merge_guards_bot]

clang-format: pass.

Build artifacts: console-log.txt, diff.json

[merge_guards_bot]

clang-format: pass.

Build artifacts: console-log.txt, diff.json

[uenoku]

llvm/test/Transforms/Attributor/ArgumentPromotion/reserve-tbaa.ll

1

Could you explain what occurs in this test?

[jdoerfert]

llvm/test/Transforms/Attributor/ArgumentPromotion/reserve-tbaa.ll

1

So the intent was to make sure we keep tbaa metadata properly attached to the new loads and stores.
I changed it because it was not using one of the arguments (transitively). As a result, we would
deleted the argument with this patch but that is not what we want to test. So I added a transitive user to ensure tbaa metadata are still tested (once we actually promote the arguments).

[uenoku]

LGTM

llvm/lib/Transforms/IPO/Attributor.cpp

5684

nit: Rewrite

llvm/test/Transforms/Attributor/ArgumentPromotion/reserve-tbaa.ll

1

It makes sense.

llvm/include/llvm/Transforms/IPO/Attributor.h

struct Attributor {
 
   /// Check \p Pred on all (transitive) uses of \p V.
   ///
   /// This method will evaluate \p Pred on all (transitive) uses of the
   /// associated value and return true if \p Pred holds every time.
   bool checkForAllUses(const function_ref<bool(const Use &, bool &)> &Pred,
                        const AbstractAttribute &QueryingAA, const Value &V);
 
+  /// Helper struct used in the communication between an abstract attribute (AA)
+  /// that wants to change the signature of a function and the Attributor which
+  /// applies the changes. The struct is partially initialized with the
+  /// information from the AA (see the constructor). All other members are
+  /// provided by the Attributor prior to invoking any callbacks.
+  struct ArgumentReplacementInfo {
+    /// Callee repair callback type
+    ///
+    /// The function repair callback is invoked once to rewire the replacement
+    /// arguments in the body of the new function. The argument replacement info
+    /// is passed, as build from the registerFunctionSignatureRewrite call, as
+    /// well as the replacement function and an iteratore to the first
+    /// replacement argument.
+    using CalleeRepairCBTy = std::function<void(
+        const ArgumentReplacementInfo &, Function &, Function::arg_iterator)>;
+
+    /// Abstract call site (ACS) repair callback type
+    ///
+    /// The abstract call site repair callback is invoked once on every abstract
+    /// call site of the replaced function (\see ReplacedFn). The callback needs
+    /// to provide the operands for the call to the new replacement function.
+    /// The number and type of the operands appended to the provided vector
+    /// (second argument) is defined by the number and types determined through
+    /// the replacement type vector (\see ReplacementTypes). The first argument
+    /// is the ArgumentReplacementInfo object registered with the Attributor
+    /// through the registerFunctionSignatureRewrite call.
+    using ACSRepairCBTy =
+        std::function<void(const ArgumentReplacementInfo &, AbstractCallSite,
+                           SmallVectorImpl<Value *> &)>;
+
+    /// Simple getters, see the corresponding members for details.
+    ///{
+
+    Attributor &getAttributor() const { return A; }
+    const Function &getReplacedFn() const { return ReplacedFn; }
+    const Argument &getReplacedArg() const { return ReplacedArg; }
+    unsigned getNumReplacementArgs() const { return ReplacementTypes.size(); }
+    const SmallVectorImpl<Type *> &getReplacementTypes() const {
+      return ReplacementTypes;
+    }
+
+    ///}
+
+  private:
+    /// Constructor that takes the argument to be replaced, the types of
+    /// the replacement arguments, as well as callbacks to repair the call sites
+    /// and new function after the replacement happened.
+    ArgumentReplacementInfo(Attributor &A, Argument &Arg,
+                            ArrayRef<Type *> ReplacementTypes,
+                            CalleeRepairCBTy &&CalleeRepairCB,
+                            ACSRepairCBTy &&ACSRepairCB)
+        : A(A), ReplacedFn(*Arg.getParent()), ReplacedArg(Arg),
+          ReplacementTypes(ReplacementTypes.begin(), ReplacementTypes.end()),
+          CalleeRepairCB(std::move(CalleeRepairCB)),
+          ACSRepairCB(std::move(ACSRepairCB)) {}
+
+    /// Reference to the attributor to allow access from the callbacks.
+    Attributor &A;
+
+    /// The "old" function replaced by ReplacementFn.
+    const Function &ReplacedFn;
+
+    /// The "old" argument replaced by new ones defined via ReplacementTypes.
+    const Argument &ReplacedArg;
+
+    /// The types of the arguments replacing ReplacedArg.
+    const SmallVector<Type *, 8> ReplacementTypes;
+
+    /// Callee repair callback, see CalleeRepairCBTy.
+    const CalleeRepairCBTy CalleeRepairCB;
+
+    /// Abstract call site (ACS) repair callback, see ACSRepairCBTy.
+    const ACSRepairCBTy ACSRepairCB;
+
+    /// Allow access to the private members from the Attributor.
+    friend struct Attributor;
+  };
+
+  /// Register a rewrite for a function signature.
+  ///
+  /// The argument \p Arg is replaced with new ones defined by the number,
+  /// order, and types in \p ReplacementTypes. The rewiring at the call sites is
+  /// done through \p ACSRepairCB and at the callee site through
+  /// \p CalleeRepairCB.
+  ///
+  /// \returns True, if the replacement was registered, false otherwise.
+  bool registerFunctionSignatureRewrite(
+      Argument &Arg, ArrayRef<Type *> ReplacementTypes,
+      ArgumentReplacementInfo::CalleeRepairCBTy &&CalleeRepairCB,
+      ArgumentReplacementInfo::ACSRepairCBTy &&ACSRepairCB);
+
   /// Check \p Pred on all function call sites.
   ///
   /// This method will evaluate \p Pred on call sites and return
   /// true if \p Pred holds in every call sites. However, this is only possible
   /// all call sites are known, hence the function has internal linkage.
   bool checkForAllCallSites(const function_ref<bool(AbstractCallSite)> &Pred,
                             const AbstractAttribute &QueryingAA,
                             bool RequireAllCallSites);
if (AAType *AA = static_cast<AAType *>(
       if (TrackDependence && AA->getState().isValidState())
         recordDependence(*AA, const_cast<AbstractAttribute &>(*QueryingAA),
                          DepClass);
       return AA;
     }
     return nullptr;
   }
 
+  /// Apply all requested function signature rewrites
+  /// (\see registerFunctionSignatureRewrite) and return Changed if the module
+  /// was altered.
+  ChangeStatus rewriteFunctionSignatures();
+
   /// The set of all abstract attributes.
   ///{
   using AAVector = SmallVector<AbstractAttribute *, 64>;
   AAVector AllAbstractAttributes;
   ///}
 
   /// A nested map to lookup abstract attributes based on the argument position
   /// on the outer level, and the addresses of the static member (AAType::ID) on
struct QueryMapValueTy {
     /// Set of abstract attributes which were used and which were necessarily
     /// required for any potential optimistic state.
     SetVector<AbstractAttribute *> RequiredAAs;
   };
   using QueryMapTy = MapVector<const AbstractAttribute *, QueryMapValueTy>;
   QueryMapTy QueryMap;
   ///}
 
+  /// Map to remember all requested signature changes (= argument replacements).
+  DenseMap<Function *, SmallVector<ArgumentReplacementInfo *, 8>>
+      ArgumentReplacementMap;
+
   /// The information cache that holds pre-processed (LLVM-IR) information.
   InformationCache &InfoCache;
 
   /// Set if the attribute currently updated did query a non-fix attribute.
   bool QueriedNonFixAA;
 
   /// Number of iterations until the dependences between abstract attributes are
   /// recomputed.

llvm/lib/Transforms/IPO/Attributor.cpp

struct AAIsDeadArgument : public AAIsDeadFloating {
   AAIsDeadArgument(const IRPosition &IRP) : AAIsDeadFloating(IRP) {}
 
   /// See AbstractAttribute::initialize(...).
   void initialize(Attributor &A) override {
     if (!getAssociatedFunction()->hasExactDefinition())
       indicatePessimisticFixpoint();
   }
 
+  /// See AbstractAttribute::manifest(...).
+  ChangeStatus manifest(Attributor &A) override {
+    ChangeStatus Changed = AAIsDeadFloating::manifest(A);
+    Argument &Arg = *getAssociatedArgument();
+    if (Arg.getParent()->hasLocalLinkage())
+      if (A.registerFunctionSignatureRewrite(
+              Arg, /* ReplacementTypes */ {},
+              Attributor::ArgumentReplacementInfo::CalleeRepairCBTy{},
+              Attributor::ArgumentReplacementInfo::ACSRepairCBTy{}))
+        return ChangeStatus::CHANGED;
+    return Changed;
+  }
+
   /// See AbstractAttribute::trackStatistics()
   void trackStatistics() const override { STATS_DECLTRACK_ARG_ATTR(IsDead) }
 };
 
 struct AAIsDeadCallSiteArgument : public AAIsDeadValueImpl {
   AAIsDeadCallSiteArgument(const IRPosition &IRP) : AAIsDeadValueImpl(IRP) {}
 
   /// See AbstractAttribute::initialize(...).
while (FoundDeadFn) {
         F->replaceAllUsesWith(UndefValue::get(F->getType()));
         F->eraseFromParent();
         InternalFns[u] = nullptr;
         FoundDeadFn = true;
       }
     }
   }
 
+  // Rewrite the functions as requested during manifest.
+  ManifestChange = ManifestChange | rewriteFunctionSignatures();
+
   if (VerifyMaxFixpointIterations &&
       IterationCounter != MaxFixpointIterations) {
     errs() << "\n[Attributor] Fixpoint iteration done after: "
            << IterationCounter << "/" << MaxFixpointIterations
            << " iterations\n";
     llvm_unreachable("The fixpoint was not reached with exactly the number of "
                      "specified iterations!");
   }
 
   return ManifestChange;
 }
 
+bool Attributor::registerFunctionSignatureRewrite(
+    Argument &Arg, ArrayRef<Type *> ReplacementTypes,
+    ArgumentReplacementInfo::CalleeRepairCBTy &&CalleeRepairCB,
+    ArgumentReplacementInfo::ACSRepairCBTy &&ACSRepairCB) {
+
+  auto CallSiteCanBeChanged = [](AbstractCallSite ACS) {
+    // Forbid must-tail calls for now.
+    return !ACS.isCallbackCall() && !ACS.getCallSite().isMustTailCall();
+  };
+
+  Function *Fn = Arg.getParent();
+  // Avoid var-arg functions for now.
+  if (Fn->isVarArg()) {
+    LLVM_DEBUG(dbgs() << "[Attributor] Cannot rewrite var-args functions\n");
+    return false;
+  }
+
+  // Avoid functions with complicated argument passing semantics.
+  AttributeList FnAttributeList = Fn->getAttributes();
+  if (FnAttributeList.hasAttrSomewhere(Attribute::Nest) ||
+      FnAttributeList.hasAttrSomewhere(Attribute::StructRet) ||
+      FnAttributeList.hasAttrSomewhere(Attribute::InAlloca)) {
+    LLVM_DEBUG(
+        dbgs() << "[Attributor] Cannot rewrite due to complex attribute\n");
+    return false;
+  }
+
+  // Avoid callbacks for now.
+  if (!checkForAllCallSites(CallSiteCanBeChanged, *Fn, true, nullptr)) {
+    LLVM_DEBUG(dbgs() << "[Attributor] Cannot rewrite all call sites\n");
+    return false;
+  }
+
+  auto InstPred = [](Instruction &I) {
+    if (auto *CI = dyn_cast<CallInst>(&I))
+      return !CI->isMustTailCall();
+    return true;
+  };
+
+  // Forbid must-tail calls for now.
+  // TODO:
+  bool AnyDead;
+  auto &OpcodeInstMap = InfoCache.getOpcodeInstMapForFunction(*Fn);
+  if (!checkForAllInstructionsImpl(OpcodeInstMap, InstPred, nullptr, AnyDead,
+                                   {Instruction::Call})) {
+    LLVM_DEBUG(dbgs() << "[Attributor] Cannot rewrite due to instructions\n");
+    return false;
+  }
+
+  SmallVectorImpl<ArgumentReplacementInfo *> &ARIs = ArgumentReplacementMap[Fn];
+  if (ARIs.size() == 0)
+    ARIs.resize(Fn->arg_size());
+
+  // If we have a replacement already with less than or equal new arguments,
+  // ignore this request.
+  ArgumentReplacementInfo *&ARI = ARIs[Arg.getArgNo()];
+  if (ARI && ARI->getNumReplacementArgs() <= ReplacementTypes.size()) {
+    LLVM_DEBUG(dbgs() << "[Attributor] Existing rewrite is preferred\n");
+    return false;
+  }
+
+  // If we have a replacement already but we like the new one better, delete
+  // the old.
+  if (ARI)
+    delete ARI;
+
+  // Remember the replacement.
+  ARI = new ArgumentReplacementInfo(*this, Arg, ReplacementTypes,
+                                    std::move(CalleeRepairCB),
+                                    std::move(ACSRepairCB));
+
+  return true;
+}
+
+ChangeStatus Attributor::rewriteFunctionSignatures() {
+  ChangeStatus Changed = ChangeStatus::UNCHANGED;
+
+  for (auto &It : ArgumentReplacementMap) {
+    Function *OldFn = It.getFirst();
+
+    // Deleted functions do not require rewrites.
+    if (ToBeDeletedFunctions.count(OldFn))
+      continue;
+
+    const SmallVectorImpl<ArgumentReplacementInfo *> &ARIs = It.getSecond();
+    assert(ARIs.size() == OldFn->arg_size() && "Inconsistent state!");
+
+    SmallVector<Type *, 16> NewArgumentTypes;
+    SmallVector<AttributeSet, 16> NewArgumentAttributes;
+
+    // Collect replacement argument types and copy over existing attributes.
+    AttributeList OldFnAttributeList = OldFn->getAttributes();
+    for (Argument &Arg : OldFn->args()) {
+      if (ArgumentReplacementInfo *ARI = ARIs[Arg.getArgNo()]) {
+        NewArgumentTypes.append(ARI->ReplacementTypes.begin(),
+                                ARI->ReplacementTypes.end());
+        NewArgumentAttributes.append(ARI->getNumReplacementArgs(),
+                                     AttributeSet());
+      } else {
+        NewArgumentTypes.push_back(Arg.getType());
+        NewArgumentAttributes.push_back(
+            OldFnAttributeList.getParamAttributes(Arg.getArgNo()));
+      }
+    }
+
+    FunctionType *OldFnTy = OldFn->getFunctionType();
+    Type *RetTy = OldFnTy->getReturnType();
+
+    // Construct the new function type using the new arguments types.
+    FunctionType *NewFnTy =
+        FunctionType::get(RetTy, NewArgumentTypes, OldFnTy->isVarArg());
+
+    LLVM_DEBUG(dbgs() << "[Attributor] Function rewrite '" << OldFn->getName()
+                      << "' from " << *OldFn->getFunctionType() << " to "
+                      << *NewFnTy << "\n");
+
+    // Create the new function body and insert it into the module.
+    Function *NewFn = Function::Create(NewFnTy, OldFn->getLinkage(),
+                                       OldFn->getAddressSpace(), "");
+    OldFn->getParent()->getFunctionList().insert(OldFn->getIterator(), NewFn);
+    NewFn->takeName(OldFn);
+    NewFn->copyAttributesFrom(OldFn);
+
+    // Patch the pointer to LLVM function in debug info descriptor.
+    NewFn->setSubprogram(OldFn->getSubprogram());
+    OldFn->setSubprogram(nullptr);
+
+    // Recompute the parameter attributes list based on the new arguments for
+    // the function.
+    LLVMContext &Ctx = OldFn->getContext();
+    NewFn->setAttributes(AttributeList::get(
+        Ctx, OldFnAttributeList.getFnAttributes(),
+        OldFnAttributeList.getRetAttributes(), NewArgumentAttributes));
+
+    // Since we have now created the new function, splice the body of the old
+    // function right into the new function, leaving the old rotting hulk of the
+    // function empty.
+    NewFn->getBasicBlockList().splice(NewFn->begin(),
+                                      OldFn->getBasicBlockList());
+
+    // Set of all "call-like" instructions that invoke the old function.
+    SmallPtrSet<Instruction *, 8> OldCallSites;
+
+    // Callback to create a new "call-like" instruction for a given one.
+    auto CallSiteReplacementCreator = [&](AbstractCallSite ACS) {
+      CallBase *OldCB = cast<CallBase>(ACS.getInstruction());
+      const AttributeList &OldCallAttributeList = OldCB->getAttributes();
+
+      // Collect the new argument operands for the replacement call site.
+      SmallVector<Value *, 16> NewArgOperands;
+      SmallVector<AttributeSet, 16> NewArgOperandAttributes;
+      for (unsigned OldArgNum = 0; OldArgNum < ARIs.size(); ++OldArgNum) {
+        unsigned NewFirstArgNum = NewArgOperands.size();
+        if (ArgumentReplacementInfo *ARI = ARIs[OldArgNum]) {
+          if (ARI->ACSRepairCB)
+            ARI->ACSRepairCB(*ARI, ACS, NewArgOperands);
+          assert(ARI->getNumReplacementArgs() + NewFirstArgNum ==
+                     NewArgOperands.size() &&
+                 "ACS repair callback did not provide as many operand as new "
+                 "types were registered!");
+          // TODO: Exose the attribute set to the ACS repair callback
+          NewArgOperandAttributes.append(ARI->ReplacementTypes.size(),
+                                         AttributeSet());
+        } else {
+          NewArgOperands.push_back(ACS.getCallArgOperand(OldArgNum));
+          NewArgOperandAttributes.push_back(
+              OldCallAttributeList.getParamAttributes(OldArgNum));
+        }
+      }
+
+      assert(NewArgOperands.size() == NewArgOperandAttributes.size() &&
+             "Mismatch # argument operands vs. # argument operand attributes!");
+      assert(NewArgOperands.size() == NewFn->arg_size() &&
+             "Mismatch # argument operands vs. # function arguments!");
+
+      SmallVector<OperandBundleDef, 4> OperandBundleDefs;
+      OldCB->getOperandBundlesAsDefs(OperandBundleDefs);
+
+      // Create a new call or invoke instruction to replace the old one.
+      CallBase *NewCB;
+      if (InvokeInst *II = dyn_cast<InvokeInst>(OldCB)) {
+        NewCB =
+            InvokeInst::Create(NewFn, II->getNormalDest(), II->getUnwindDest(),
+                               NewArgOperands, OperandBundleDefs, "", OldCB);
+      } else {
+        auto *NewCI = CallInst::Create(NewFn, NewArgOperands, OperandBundleDefs,
+                                       "", OldCB);
+        NewCI->setTailCallKind(cast<CallInst>(OldCB)->getTailCallKind());
+        NewCB = NewCI;
+      }
+
+      // Copy over various properties and the new attributes.
+      OldCB->replaceAllUsesWith(NewCB);
+      uint64_t W;
+      if (OldCB->extractProfTotalWeight(W))
+        NewCB->setProfWeight(W);
+      NewCB->setCallingConv(OldCB->getCallingConv());
+      NewCB->setDebugLoc(OldCB->getDebugLoc());
+      NewCB->takeName(OldCB);
+      NewCB->setAttributes(AttributeList::get(
+          Ctx, OldCallAttributeList.getFnAttributes(),
+          OldCallAttributeList.getRetAttributes(), NewArgOperandAttributes));
+
+      bool Inserted = OldCallSites.insert(OldCB).second;
+      assert(Inserted && "Call site was old twice!");
+      (void)Inserted;
+
+      return true;
+    };
+
+    // Use the CallSiteReplacementCreator to create replacement call sites.
+    bool Success =
+        checkForAllCallSites(CallSiteReplacementCreator, *OldFn, true, nullptr);
+    assert(Success && "Assumed call site replacement to succeed!");
+
+    // Rewire the arguments.
+    auto OldFnArgIt = OldFn->arg_begin();
+    auto NewFnArgIt = NewFn->arg_begin();
+    for (unsigned OldArgNum = 0; OldArgNum < ARIs.size();
+         ++OldArgNum, ++OldFnArgIt) {
+      if (ArgumentReplacementInfo *ARI = ARIs[OldArgNum]) {
+        if (ARI->CalleeRepairCB)
+          ARI->CalleeRepairCB(*ARI, *NewFn, NewFnArgIt);
+        NewFnArgIt += ARI->ReplacementTypes.size();
+      } else {
+        NewFnArgIt->takeName(&*OldFnArgIt);
+        OldFnArgIt->replaceAllUsesWith(&*NewFnArgIt);
+        ++NewFnArgIt;
+      }
+    }
+
+    // Eliminate the instructions *after* we visited all of them.
+    for (Instruction *OldCallSite : OldCallSites)
+      OldCallSite->eraseFromParent();
+
+    assert(OldFn->getNumUses() == 0 && "Unexpected leftover uses!");
+    OldFn->eraseFromParent();
+    Changed = ChangeStatus::CHANGED;
+  }
+
+  return Changed;
+}
+
 void Attributor::initializeInformationCache(Function &F) {
 
   // Walk all instructions to find interesting instructions that might be
   // queried by abstract attributes during their initialization or update.
   // This has to happen before we create attributes.
   auto &ReadOrWriteInsts = InfoCache.FuncRWInstsMap[&F];
   auto &InstOpcodeMap = InfoCache.FuncInstOpcodeMap[&F];
 

llvm/test/Transforms/Attributor/ArgumentPromotion/control-flow.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --function-signature --scrub-attributes
-; RUN: opt -S -passes='attributor' -aa-pipeline='basic-aa' -attributor-disable=false -attributor-max-iterations-verify -attributor-max-iterations=2 < %s | FileCheck %s
+; RUN: opt -S -passes='attributor' -aa-pipeline='basic-aa' -attributor-disable=false -attributor-max-iterations-verify -attributor-max-iterations=3 < %s | FileCheck %s
+
+; FIXME: %C should be removed in callee
+; FIXME: %P should be removed in callee2
 
 ; Don't promote around control flow.
 define internal i32 @callee(i1 %C, i32* %P) {
 ; CHECK-LABEL: define {{[^@]+}}@callee
-; CHECK-SAME: (i1 [[C:%.*]], i32* noalias nocapture nofree readnone [[P:%.*]])
+; CHECK-SAME: (i1 [[C:%.*]])
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br i1 true, label [[T:%.*]], label [[F:%.*]]
 ; CHECK:       T:
 ; CHECK-NEXT:    ret i32 17
 ; CHECK:       F:
 ; CHECK-NEXT:    unreachable
 ;
 entry:
   br i1 %C, label %T, label %F
 
 T:
   ret i32 17
 
 F:
   %X = load i32, i32* %P
   ret i32 %X
 }
 
 define i32 @foo() {
 ; CHECK-LABEL: define {{[^@]+}}@foo()
 ; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[X:%.*]] = call i32 @callee(i1 true, i32* noalias nofree undef)
+; CHECK-NEXT:    [[X:%.*]] = call i32 @callee(i1 true)
 ; CHECK-NEXT:    ret i32 17
 ;
 entry:
   %X = call i32 @callee(i1 true, i32* null)
   ret i32 %X
 }
 
+; Don't promote around control flow.
+define internal i32 @callee2(i1 %C, i32* %P) {
+; CHECK-LABEL: define {{[^@]+}}@callee2
+; CHECK-SAME: (i1 [[C:%.*]], i32* noalias nocapture nofree readonly [[P:%.*]])
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    br i1 [[C]], label [[T:%.*]], label [[F:%.*]]
+; CHECK:       T:
+; CHECK-NEXT:    ret i32 17
+; CHECK:       F:
+; CHECK-NEXT:    [[X:%.*]] = load i32, i32* null
+; CHECK-NEXT:    ret i32 [[X]]
+;
+entry:
+  br i1 %C, label %T, label %F
+
+T:
+  ret i32 17
+
+F:
+  %X = load i32, i32* %P
+  ret i32 %X
+}
+
+define i32 @foo2(i1 %c) {
+; CHECK-LABEL: define {{[^@]+}}@foo2
+; CHECK-SAME: (i1 [[C:%.*]])
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[X:%.*]] = call i32 @callee2(i1 [[C]], i32* noalias nofree null)
+; CHECK-NEXT:    ret i32 [[X]]
+;
+entry:
+  %X = call i32 @callee2(i1 %c, i32* null)
+  ret i32 %X
+}
+
+; Don't promote around control flow.
+define internal i32 @callee3(i1 %C, i32* %P) {
+; CHECK-LABEL: define {{[^@]+}}@callee3
+; CHECK-SAME: (i1 [[C:%.*]], i32* nocapture nofree readonly [[P:%.*]])
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    br i1 [[C]], label [[T:%.*]], label [[F:%.*]]
+; CHECK:       T:
+; CHECK-NEXT:    ret i32 17
+; CHECK:       F:
+; CHECK-NEXT:    [[X:%.*]] = load i32, i32* [[P]]
+; CHECK-NEXT:    ret i32 [[X]]
+;
+entry:
+  br i1 %C, label %T, label %F
+
+T:
+  ret i32 17
+
+F:
+  %X = load i32, i32* %P
+  ret i32 %X
+}
+
+define i32 @foo3(i1 %c, i32* %p) {
+; CHECK-LABEL: define {{[^@]+}}@foo3
+; CHECK-SAME: (i1 [[C:%.*]], i32* nocapture nofree readonly [[P:%.*]])
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[X:%.*]] = call i32 @callee3(i1 [[C]], i32* nocapture nofree readonly [[P]])
+; CHECK-NEXT:    ret i32 [[X]]
+;
+entry:
+  %X = call i32 @callee3(i1 %c, i32* %p)
+  ret i32 %X
+}

llvm/test/Transforms/Attributor/ArgumentPromotion/dbg.ll

;
   %2 = load i32, i32* %1, align 8
   call void @sink(i32 %2)
   ret void
 }
 
 %struct.pair = type { i32, i32 }
 
 define internal void @test_byval(%struct.pair* byval %P) {
-; CHECK-LABEL: define {{[^@]+}}@test_byval
-; CHECK-SAME: (%struct.pair* nocapture nofree readnone byval [[P:%.*]])
+; CHECK-LABEL: define {{[^@]+}}@test_byval()
 ; CHECK-NEXT:    ret void
 ;
   ret void
 }
 
-define void @caller(i32** %Y, %struct.pair* %P) {
+define internal i32 @test_byval_used(%struct.pair* byval %P) {
+; CHECK-LABEL: define {{[^@]+}}@test_byval_used
+; CHECK-SAME: (%struct.pair* nocapture nofree nonnull readonly byval align 8 dereferenceable(4) [[P:%.*]])
+; CHECK-NEXT:    [[C:%.*]] = bitcast %struct.pair* [[P]] to i32*
+; CHECK-NEXT:    [[V:%.*]] = load i32, i32* [[C]], align 8
+; CHECK-NEXT:    ret i32 [[V]]
+;
+  %c = bitcast %struct.pair* %P to i32*
+  %v = load i32, i32* %c, align 8
+  ret i32 %v
+}
+
+define i32 @caller(i32** %Y, %struct.pair* %P) {
 ; CHECK-LABEL: define {{[^@]+}}@caller
 ; CHECK-SAME: (i32** nocapture readonly [[Y:%.*]], %struct.pair* nocapture nofree readonly [[P:%.*]])
 ; CHECK-NEXT:    call void @test(i32** nocapture readonly [[Y]]), !dbg !4
-; CHECK-NEXT:    call void @test_byval(%struct.pair* nocapture nofree readonly undef), !dbg !5
-; CHECK-NEXT:    ret void
+; CHECK-NEXT:    call void @test_byval(), !dbg !5
+; CHECK-NEXT:    [[V:%.*]] = call i32 @test_byval_used(%struct.pair* nocapture nofree readonly [[P]]), !dbg !5
+; CHECK-NEXT:    ret i32 [[V]]
 ;
   call void @test(i32** %Y), !dbg !1
 
   call void @test_byval(%struct.pair* %P), !dbg !6
-  ret void
+  %v = call i32 @test_byval_used(%struct.pair* %P), !dbg !6
+  ret i32 %v
 }
 
 
 !llvm.module.flags = !{!0}
 !llvm.dbg.cu = !{!3}
 
 !0 = !{i32 2, !"Debug Info Version", i32 3}
 !1 = !DILocation(line: 8, scope: !2)
 !2 = distinct !DISubprogram(name: "test", file: !5, line: 3, isLocal: true, isDefinition: true, virtualIndex: 6, flags: DIFlagPrototyped, isOptimized: false, unit: !3, scopeLine: 3, scope: null)
 !3 = distinct !DICompileUnit(language: DW_LANG_C_plus_plus, producer: "clang version 3.5.0 ", isOptimized: false, emissionKind: LineTablesOnly, file: !5)
 !5 = !DIFile(filename: "test.c", directory: "")
 !6 = !DILocation(line: 9, scope: !2)

llvm/test/Transforms/Attributor/ArgumentPromotion/nonzero-address-spaces.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --function-signature --scrub-attributes
 ; RUN: opt -S -passes=attributor -aa-pipeline='basic-aa' -attributor-disable=false -attributor-max-iterations-verify -attributor-max-iterations=1 < %s | FileCheck %s
 
 ; ArgumentPromotion should preserve the default function address space
 ; from the data layout.
 
 target datalayout = "e-P1-p:16:8-i8:8-i16:8-i32:8-i64:8-f32:8-f64:8-n8-a:8"
 
 @g = common global i32 0, align 4
 
 define i32 @bar() {
 ; CHECK-LABEL: define {{[^@]+}}@bar() addrspace(1)
 ; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[CALL:%.*]] = call addrspace(1) i32 @foo(i32* nofree nonnull align 4 dereferenceable(4) undef)
+; CHECK-NEXT:    [[CALL:%.*]] = call addrspace(1) i32 @foo()
 ; CHECK-NEXT:    unreachable
 ; CHECK:       entry.split:
 ; CHECK-NEXT:    unreachable
 ;
 
 entry:
   %call = call i32 @foo(i32* @g)
   ret i32 %call
 }
 
 define internal i32 @foo(i32*) {
-; CHECK-LABEL: define {{[^@]+}}@foo
-; CHECK-SAME: (i32* nocapture nofree nonnull readnone align 4 dereferenceable(4) [[TMP0:%.*]]) addrspace(1)
+; CHECK-LABEL: define {{[^@]+}}@foo() addrspace(1)
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[RETVAL:%.*]] = alloca i32, align 4
 ; CHECK-NEXT:    call addrspace(0) void asm sideeffect "ldr r0, [r0] \0Abx lr \0A", ""()
 ; CHECK-NEXT:    unreachable
 ;
 entry:
   %retval = alloca i32, align 4
   call void asm sideeffect "ldr r0, [r0] \0Abx lr        \0A", ""()
   unreachable
 }
 

llvm/test/Transforms/Attributor/ArgumentPromotion/pr27568.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --function-signature --scrub-attributes
 ; RUN: opt -S -passes=attributor -aa-pipeline='basic-aa' -attributor-disable=false -attributor-max-iterations-verify -attributor-max-iterations=1 < %s | FileCheck %s
 target triple = "x86_64-pc-windows-msvc"
 
 define internal void @callee(i8*) {
-; CHECK-LABEL: define {{[^@]+}}@callee
-; CHECK-SAME: (i8* noalias nocapture nofree readnone [[TMP0:%.*]])
+; CHECK-LABEL: define {{[^@]+}}@callee()
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    call void @thunk()
 ; CHECK-NEXT:    ret void
 ;
 entry:
   call void @thunk()
   ret void
 }
 
 define void @test1() personality i32 (...)* @__CxxFrameHandler3 {
 ; CHECK-LABEL: define {{[^@]+}}@test1() personality i32 (...)* @__CxxFrameHandler3
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    invoke void @thunk()
 ; CHECK-NEXT:    to label [[OUT:%.*]] unwind label [[CPAD:%.*]]
 ; CHECK:       out:
 ; CHECK-NEXT:    ret void
 ; CHECK:       cpad:
 ; CHECK-NEXT:    [[PAD:%.*]] = cleanuppad within none []
-; CHECK-NEXT:    call void @callee(i8* noalias nofree undef) [ "funclet"(token [[PAD]]) ]
+; CHECK-NEXT:    call void @callee() [ "funclet"(token [[PAD]]) ]
 ; CHECK-NEXT:    cleanupret from [[PAD]] unwind to caller
 ;
 entry:
   invoke void @thunk()
   to label %out unwind label %cpad
 
 out:
   ret void

llvm/test/Transforms/Attributor/ArgumentPromotion/reserve-tbaa.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --function-signature --scrub-attributes
 ; RUN: opt -S -passes='attributor' -aa-pipeline='basic-aa' -attributor-disable=false -attributor-max-iterations-verify -attributor-max-iterations=2 < %s | FileCheck %s
 
 ; PR17906
 ; When we promote two arguments in a single function with different types,
 ; before the fix, we used the same tag for the newly-created two loads.
 ; This testing case makes sure that we correctly transfer the tbaa tags from the
 ; original loads to the newly-created loads when promoting pointer arguments.
 
 @a = global i32* null, align 8
 @e = global i32** @a, align 8
 @g = global i32 0, align 4
 @c = global i64 0, align 8
-@d = global i8 0, align 1
+@d0 = global i32 0, align 1
+@d1 = global i8 0, align 1
 
 define internal fastcc void @fn(i32* nocapture readonly %p1, i64* nocapture readonly %p2) {
 ; CHECK-LABEL: define {{[^@]+}}@fn
 ; CHECK-SAME: (i32* nocapture nofree nonnull readonly align 4 dereferenceable(4) [[P1:%.*]], i64* nocapture nofree nonnull readonly align 8 dereferenceable(8) [[P2:%.*]])
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[TMP0:%.*]] = load i64, i64* @c, align 8, !tbaa !0
 ; CHECK-NEXT:    [[CONV:%.*]] = trunc i64 [[TMP0]] to i32
+; CHECK-NEXT:    store i32 [[CONV]], i32* @d0, align 1
 ; CHECK-NEXT:    [[TMP1:%.*]] = load i32, i32* @g, align 4, !tbaa !4
 ; CHECK-NEXT:    [[CONV1:%.*]] = trunc i32 [[TMP1]] to i8
-; CHECK-NEXT:    store i8 [[CONV1]], i8* @d, align 1, !tbaa !6
+; CHECK-NEXT:    store i8 [[CONV1]], i8* @d1, align 1, !tbaa !6
 ; CHECK-NEXT:    ret void
 ;
 entry:
   %0 = load i64, i64* %p2, align 8, !tbaa !1
   %conv = trunc i64 %0 to i32
+  store i32 %conv, i32* @d0, align 1
   %1 = load i32, i32* %p1, align 4, !tbaa !5
   %conv1 = trunc i32 %1 to i8
-  store i8 %conv1, i8* @d, align 1, !tbaa !7
+  store i8 %conv1, i8* @d1, align 1, !tbaa !7
   ret void
 }
 
 define i32 @main() {
 ; CHECK-LABEL: define {{[^@]+}}@main()
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[TMP0:%.*]] = load i32**, i32*** @e, align 8, !tbaa !7
 ; CHECK-NEXT:    store i32* @g, i32** [[TMP0]], align 8, !tbaa !7
 ; CHECK-NEXT:    [[TMP1:%.*]] = load i32*, i32** @a, align 8, !tbaa !7
 ; CHECK-NEXT:    store i32 1, i32* [[TMP1]], align 4, !tbaa !4
-; CHECK-NEXT:    call fastcc void @fn(i32* nofree nonnull align 4 dereferenceable(4) @g, i64* nofree nonnull align 8 dereferenceable(8) undef)
+; CHECK-NEXT:    call fastcc void @fn(i32* nofree nonnull align 4 dereferenceable(4) @g, i64* nofree nonnull align 8 dereferenceable(8) @c)
 ; CHECK-NEXT:    ret i32 0
 ;
 entry:
   %0 = load i32**, i32*** @e, align 8, !tbaa !8
   store i32* @g, i32** %0, align 8, !tbaa !8
   %1 = load i32*, i32** @a, align 8, !tbaa !8
   store i32 1, i32* %1, align 4, !tbaa !5
   call fastcc void @fn(i32* @g, i64* @c)

llvm/test/Transforms/Attributor/liveness.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --function-signature --scrub-attributes
-; RUN: opt -attributor --attributor-disable=false -attributor-max-iterations-verify -attributor-annotate-decl-cs -attributor-max-iterations=4 -S < %s | FileCheck %s
+; RUN: opt -attributor --attributor-disable=false -attributor-max-iterations-verify -attributor-annotate-decl-cs -attributor-max-iterations=4 -S < %s | FileCheck %s --check-prefixes=CHECK,OLDPM
+; RUN: opt -passes=attributor --attributor-disable=false -attributor-max-iterations-verify -attributor-annotate-decl-cs -attributor-max-iterations=4 -S < %s | FileCheck %s --check-prefixes=CHECK,NEWPM
 ; UTC_ARGS: --turn off
 
 ; CHECK: @dead_with_blockaddress_users.l = constant [2 x i8*] [i8* inttoptr (i32 1 to i8*), i8* inttoptr (i32 1 to i8*)]
 @dead_with_blockaddress_users.l = constant [2 x i8*] [i8* blockaddress(@dead_with_blockaddress_users, %lab0), i8* blockaddress(@dead_with_blockaddress_users, %end)]
 
 declare void @no_return_call() nofree noreturn nounwind readnone
 
 declare void @normal_call() readnone

 ; CHECK: define internal void @non_dead_d8()
 ; CHECK: define internal void @non_dead_d9()
 ; CHECK: define internal void @non_dead_d10()
 ; CHECK: define internal void @non_dead_d11()
 ; CHECK: define internal void @non_dead_d12()
 ; CHECK: define internal void @non_dead_d13()
 ; CHECK: define internal void @non_dead_d14()
 ; Verify we actually deduce information for these functions.
-; CHECK: Function Attrs: nofree nosync nounwind readnone willreturn
+; OLDPM: Function Attrs: nofree nosync nounwind readnone willreturn
+; NEWPM: Function Attrs: nofree norecurse nosync nounwind readnone willreturn
 ; CHECK-NEXT: define internal void @non_dead_d15()
 ; CHECK-NOT: define internal void @dead_e
 
 
 declare void @blowup() noreturn
 define void @live_with_dead_entry() personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
 ; CHECK:      define void @live_with_dead_entry(
 ; CHECK-NEXT: entry:
lp1:
   invoke void @blowup() to label %live_with_dead_entry unwind label %lp2
 lp2:
   %0 = landingpad { i8*, i32 } catch i8* null
   br label %live_with_dead_entry
 live_with_dead_entry:
   ret void
 }
 
-; CHECK: define internal void @useless_arg_sink(i32* nocapture nofree readnone %a)
+; CHECK: define internal void @useless_arg_sink()
 define internal void @useless_arg_sink(i32* %a) {
   ret void
 }
 
-; CHECK: define internal void @useless_arg_almost_sink(i32* nocapture nofree readnone %a)
+; CHECK: define internal void @useless_arg_almost_sink()
 define internal void @useless_arg_almost_sink(i32* %a) {
-; CHECK: call void @useless_arg_sink(i32* nofree undef)
+; CHECK: call void @useless_arg_sink()
   call void @useless_arg_sink(i32* %a)
   ret void
 }
 
 ; Check we do not annotate the function interface of this weak function.
 ; CHECK: define weak_odr void @useless_arg_ext(i32* %a)
 define weak_odr void @useless_arg_ext(i32* %a) {
-; CHECK: call void @useless_arg_almost_sink(i32* nofree undef)
+; CHECK: call void @useless_arg_almost_sink()
   call void @useless_arg_almost_sink(i32* %a)
   ret void
 }
 
 ; CHECK: define internal void @useless_arg_ext_int(i32* %a)
 define internal void @useless_arg_ext_int(i32* %a) {
 ; CHECK: call void @useless_arg_ext(i32* %a)
   call void @useless_arg_ext(i32* %a)

llvm/test/Transforms/Attributor/noalias.ll

4:                                                ; preds = %1
   br label %5
 
 5:                                                ; preds = %1, %4
   ret i8* %2
 }
 
 ; TEST 9
 ; Simple Argument Test
-define internal void @test9(i8* %a, i8* %b) {
-; CHECK: define internal void @test9(i8* noalias nocapture nofree readnone %a, i8* nocapture nofree readnone %b)
+declare void @use_i8(i8* nocapture) readnone
+define internal void @test9a(i8* %a, i8* %b) {
+; CHECK: define internal void @test9a()
+  ret void
+}
+define internal void @test9b(i8* %a, i8* %b) {
+; CHECK: define internal void @test9b(i8* noalias nocapture readnone %a, i8* nocapture readnone %b)
+  call void @use_i8(i8* %a)
+  call void @use_i8(i8* %b)
   ret void
 }
 define void @test9_helper(i8* %a, i8* %b) {
-  tail call void @test9(i8* noalias %a, i8* %b)
-  tail call void @test9(i8* noalias %b, i8* noalias %a)
+; CHECK: define void @test9_helper(i8* nocapture readnone %a, i8* nocapture readnone %b)
+; CHECK:  tail call void @test9a()
+; CHECK:  tail call void @test9a()
+; CHECK:  tail call void @test9b(i8* noalias nocapture %a, i8* nocapture %b)
+; CHECK:  tail call void @test9b(i8* noalias nocapture %b, i8* noalias nocapture %a)
+  tail call void @test9a(i8* noalias %a, i8* %b)
+  tail call void @test9a(i8* noalias %b, i8* noalias %a)
+  tail call void @test9b(i8* noalias %a, i8* %b)
+  tail call void @test9b(i8* noalias %b, i8* noalias %a)
   ret void
 }
 
 
 ; TEST 10
 ; Simple CallSite Test
 
 declare void @test10_helper_1(i8* %a)

llvm/test/Transforms/Attributor/nonnull.ll

 declare i8* @unknown()
 define void @test13_helper() {
   %nonnullptr = tail call i8* @ret_nonnull()
   %maybenullptr = tail call i8* @unknown()
   tail call void @test13(i8* %nonnullptr, i8* %nonnullptr, i8* %maybenullptr)
   tail call void @test13(i8* %nonnullptr, i8* %maybenullptr, i8* %nonnullptr)
   ret void
 }
+declare void @use_i8_ptr(i8* nofree) readnone nounwind
 define internal void @test13(i8* %a, i8* %b, i8* %c) {
 ; ATTRIBUTOR: define internal void @test13(i8* nocapture nofree nonnull readnone %a, i8* nocapture nofree readnone %b, i8* nocapture nofree readnone %c)
+  call void @use_i8_ptr(i8* %a)
+  call void @use_i8_ptr(i8* %b)
+  call void @use_i8_ptr(i8* %c)
   ret void
 }
 
 declare nonnull i8* @nonnull()
 
 ; TEST 14
 ; Complex propagation
 ; Argument of f1, f2, f3 can be marked with nonnull.