[ObjCARC] Require the function argument in the clang.arc.attachedcall bundle.

[ab]

Summary

Currently, the clang.arc.attachedcall bundle takes an optional function
argument. Depending on whether the argument is present, calls with this
bundle have the following semantics:

• on x86, with the argument present, the call is lowered to:

call _target
mov rax, rdi
call _objc_retainAutoreleasedReturnValue

• on AArch64, without the argument, the call is lowered to:

bl _target
mov x29, x29

and the objc runtime call is expected to be emitted separately.

That's because, on x86, the objc runtime checks for both the mov and
the call on x86, and treats the combination as the ARC autorelease elision
marker.

But on AArch64, it only checks for the dedicated NOP marker, as that's
historically been sufficiently unique. Thanks to that, the runtime call
wasn't required to be adjacent to the NOP marker, so it wasn't emitted
as part of the bundle sequence.

This patch unifies both architectures: on AArch64, we now emit all
3 instructions for the bundle. This guarantees that the runtime call
is adjacent to the marker in the sequence, and that's information the
runtime can use to further optimize this.

This helps simplify some of the handling, in particular
BundledRetainClaimRVs, which no longer needs to know whether the bundle
is sufficient or not: it now always should be.

Note that this includes an AutoUpgrade that simply drops
clang.arc.attachedcall bundles that don't have an operand: if they
don't, we know that the semantically required call is already a separate
instruction, and the bundle is merely a NOP.
It's not ideal to drop it, but it's okay.

[fhahn]

LGTM, thanks! For the Transforms/ObjCARC stuff it would be good for Akira to also take a quick look.

llvm/lib/IR/AutoUpgrade.cpp

4633

Any reason not to use a SmallVector?

4634

nit: stray new line?

[ahatanak]

The ObjC ARC changes look good.

Are there any users who need the changes in autoupgrade? clang.arc.attachedcall bundles without operands are emitted only after ARC contract (ARC optimizer does't remove the operand), so it seems safe not to upgrade the IR.

[ab]

In D118214#3274486, @ahatanak wrote:

Are there any users who need the changes in autoupgrade? clang.arc.attachedcall bundles without operands are emitted only after ARC contract (ARC optimizer does't remove the operand), so it seems safe not to upgrade the IR.

That's an interesting point. I looked for where the pass was inserted in the pipeline, and I thought it did run in the frontend when doing LTO, but digging a bit more, it sounds like it really only is before the backend, with or without LTO. So I think you're right, we don't need this after all! (and hopefully there are no non-clang users of the bundle with different expectations).

[phosek]

@ab We started seeing crashes in Flutter Engine on iOS after the latest Clang roll. We did a bisect and found this change to be a culprit, see https://github.com/flutter/flutter/issues/99488 for more details. Would it be possible to take a look?

[phosek]

In D118214#3393603, @phosek wrote:

@ab We started seeing crashes in Flutter Engine on iOS after the latest Clang roll. We did a bisect and found this change to be a culprit, see https://github.com/flutter/flutter/issues/99488 for more details. Would it be possible to take a look?

Gentle ping, would it be fine if reverted this change in the meantime?

[ab]

In D118214#3393603, @phosek wrote:

@ab We started seeing crashes in Flutter Engine on iOS after the latest Clang roll. We did a bisect and found this change to be a culprit, see https://github.com/flutter/flutter/issues/99488 for more details. Would it be possible to take a look?

Hm, there's not much to go on, is there any additional information aside from the partial stacktrace? Happy to look into this if y'all can reduce this to something manageable. Thanks!

[ab]

Ah, further down, there's a different crash signature in bisect6.txt, in dyld strlen: that one does ring a bell. I assume it's handling a corrupt selector, but there's not much context to know for sure. I have a later change that may take care of that, hopefully that resolves your original crash as well!

[phosek]

In D118214#3403398, @ab wrote:

Ah, further down, there's a different crash signature in bisect6.txt, in dyld strlen: that one does ring a bell. I assume it's handling a corrupt selector, but there's not much context to know for sure. I have a later change that may take care of that, hopefully that resolves your original crash as well!

Thanks, which change is that? We can give it a try to see if it resolves the issue.

[glandium]

FWIW, We're seeing Firefox crashes on aarch64 macos that have been bisected to this change as well. Top frames look like:

Thread 94 VideoCapture (crashed)
 0  libsystem_platform.dylib!_platform_strlen + 0x4
      x0 = 0x00000000018963bf     x1 = 0x00000000018963b0
      x2 = 0x0000000000000001     x3 = 0x000000005650a7e3
      x4 = 0x0000000000000000     x5 = 0x0000000000000000 
      x6 = 0x0000000000000065     x7 = 0x0000000000000000
      x8 = 0x00000001e2a220c8     x9 = 0x00000000593e20c8
     x10 = 0x00000001f405db98    x11 = 0x0000000189494000
     x12 = 0x0000000217e4c000    x13 = 0x0000000100000000
     x14 = 0x000000000000000c    x15 = 0x000000018963bca1
     x16 = 0x00000001897c2fa0    x17 = 0x00000001ef3c5dc0
     x18 = 0x0000000000000000    x19 = 0x00000000018963bf
     x20 = 0x00000001e2a220c8    x21 = 0x0000000000000001
     x22 = 0x00000000018963bf    x23 = 0x00000001f3a6d898
     x24 = 0x00000001dfb46c13    x25 = 0x0000000000000000
     x26 = 0x00000001dfb46c42    x27 = 0x00000000ffffffff
     x28 = 0x0000000000000000     fp = 0x0000000172aba2a0
      lr = 0x000000018978586c     sp = 0x0000000172aba280
      pc = 0x00000001897c2fa4
    Found by: given as instruction pointer in context
 1  libdyld.dylib!objc_opt::objc_stringhash_t::getIndex(char const*) const + 0x20
      sp = 0x0000000172aba2b0     pc = 0x000000018978586c
    Found by: previous frame's frame pointer
 2  libdyld.dylib!_dyld_get_objc_selector + 0x3c
      sp = 0x0000000172aba2d0     pc = 0x00000001897857dc
    Found by: previous frame's frame pointer
 3  libobjc.A.dylib!__sel_registerName(char const*, bool, bool) + 0x2c
      sp = 0x0000000172aba320     pc = 0x00000001896367e8
    Found by: previous frame's frame pointer
 4  CoreFoundation!___forwarding___ + 0x54c
      sp = 0x0000000172aba490     pc = 0x0000000189858130
    Found by: previous frame's frame pointer
 5  CoreFoundation!__forwarding_prep_0___ + 0x5c
      sp = 0x0000000172aba570     pc = 0x0000000189857b30
    Found by: previous frame's frame pointer
 6  XUL!-[DeviceInfoIosObjC configureObservers] [device_info_objc.mm:6475d6ae523de8e02c1e5c20ddaab4593cc16495 : 131 + 0x14]
      sp = 0x0000000172aba650     pc = 0x00000001057860a0
    Found by: previous frame's frame pointer
 7  XUL!webrtc::videocapturemodule::DeviceInfoIos::Init() [device_info.mm:6475d6ae523de8e02c1e5c20ddaab4593cc16495 : 42 + 0x8]
     x19 = 0x000000014abdfdf0    x20 = 0x000000014a4457c0
     x21 = 0xdb95bce6c9860006    x22 = 0x000000010ab71c30
     x23 = 0x000000014a9e4d40    x24 = 0x00000000000041b7
     x25 = 0x00000001057856ac    x26 = 0x0000000172aba680
     x27 = 0x000000014a9e4d40    x28 = 0x000000014a9e4d40
      fp = 0x0000000172abaa70     sp = 0x0000000172aba690
      pc = 0x0000000105784a08
    Found by: call frame info
 8  XUL!webrtc::videocapturemodule::VideoCaptureImpl::CreateDeviceInfo() [device_info.mm:6475d6ae523de8e02c1e5c20ddaab4593cc16495 : 35 + 0x44]
     x19 = 0x000000014abdfdf0    x20 = 0x000000014a4457c0
     x21 = 0x000000014abdfe80    x22 = 0x000000010a41723c

[ab]

D'oh, this fell through the cracks, sorry folks! I committed the fix I had in mind (cfa4fe7c5187). Long story short: this change exposed an old bug in the LOH pass: it wasn't honoring regmasks in bundled MIs, but since we used to split the bundle and emit the attached call separately, the (unbundled) latter was still visible to the pass. That's not true as of this change. I think for the sort of code that uses bundles on aarch64 it's usually selectors that get corrupted in a later msgSend, and that ends up exploding in dyld the way we're seeing in both cases.

[glandium]

In D118214#3446233, @ab wrote:

D'oh, this fell through the cracks, sorry folks! I committed the fix I had in mind (cfa4fe7c5187).

I can confirm this fixes the issue for Firefox. I guess it should be queued for 14.0.1?

[ab]

Sure, filed 54914 to cherry-pick into 14.0.2

llvm/docs/LangRef.rst

 :ref:`stackmap entry <statepoint-stackmap-format>`.  See the intrinsic description
 for further details.
 
 ObjC ARC Attached Call Operand Bundles
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 A ``"clang.arc.attachedcall"`` operand bundle on a call indicates the call is
 implicitly followed by a marker instruction and a call to an ObjC runtime
-function that uses the result of the call. The operand bundle takes either the
+function that uses the result of the call. The operand bundle takes a mandatory
 pointer to the runtime function (``@objc_retainAutoreleasedReturnValue`` or
-``@objc_unsafeClaimAutoreleasedReturnValue``) or no arguments. If the bundle
-doesn't take any arguments, only the marker instruction has to be emitted after
-the call; the runtime function calls don't have to be emitted since they already
-have been emitted. The return value of a call with this bundle is used by a call
-to ``@llvm.objc.clang.arc.noop.use`` unless the called function's return type is
+``@objc_unsafeClaimAutoreleasedReturnValue``).
+The return value of a call with this bundle is used by a call to
+``@llvm.objc.clang.arc.noop.use`` unless the called function's return type is
 void, in which case the operand bundle is ignored.
 
 .. code-block:: llvm
 
    ; The marker instruction and a runtime function call are inserted after the call
    ; to @foo.
    call i8* @foo() [ "clang.arc.attachedcall"(i8* (i8*)* @objc_retainAutoreleasedReturnValue) ]
    call i8* @foo() [ "clang.arc.attachedcall"(i8* (i8*)* @objc_unsafeClaimAutoreleasedReturnValue) ]
 
-   ; Only the marker instruction is inserted after the call to @foo.
-   call i8* @foo() [ "clang.arc.attachedcall"() ]
-
 The operand bundle is needed to ensure the call is immediately followed by the
-marker instruction or the ObjC runtime call in the final output.
+marker instruction and the ObjC runtime call in the final output.
 
 .. _moduleasm:
 
 Module-Level Inline Assembly
 ----------------------------
 
 Modules may contain "module-level inline asm" blocks, which corresponds
 to the GCC "file scope inline asm" blocks. These blocks are internally

llvm/include/llvm/Analysis/ObjCARCUtil.h

return !CB->getFunctionType()->getReturnType()->isVoidTy() &&
          CB->getOperandBundle(LLVMContext::OB_clang_arc_attachedcall)
              .hasValue();
 }
 
 /// This function returns operand bundle clang_arc_attachedcall's argument,
 /// which is the address of the ARC runtime function.
 inline Optional<Function *> getAttachedARCFunction(const CallBase *CB) {
   auto B = CB->getOperandBundle(LLVMContext::OB_clang_arc_attachedcall);
-  if (!B.hasValue() || B->Inputs.size() == 0)
+  if (!B)
     return None;
 
   return cast<Function>(B->Inputs[0]);
 }
 
 /// Check whether the function is retainRV/unsafeClaimRV.
 inline bool isRetainOrClaimRV(ARCInstKind Kind) {
   return Kind == ARCInstKind::RetainRV || Kind == ARCInstKind::UnsafeClaimRV;

llvm/include/llvm/IR/AutoUpgrade.h

 //===- AutoUpgrade.h - AutoUpgrade Helpers ----------------------*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 //  These functions are implemented by lib/IR/AutoUpgrade.cpp.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_IR_AUTOUPGRADE_H
 #define LLVM_IR_AUTOUPGRADE_H
 
 #include "llvm/ADT/StringRef.h"
+#include <vector>
 
 namespace llvm {
   class AttrBuilder;
   class CallInst;
   class Constant;
   class Function;
+  class GlobalVariable;
   class Instruction;
   class MDNode;
   class Module;
-  class GlobalVariable;
   class Type;
   class Value;
 
+  template <typename T> class OperandBundleDefT;
+  using OperandBundleDef = OperandBundleDefT<Value *>;
+
   /// This is a more granular function that simply checks an intrinsic function
   /// for upgrading, and returns true if it requires upgrading. It may return
   /// null in NewFn if the all calls to the original intrinsic function
   /// should be transformed to non-function-call instructions.
   bool UpgradeIntrinsicFunction(Function *F, Function *&NewFn);
 
   /// This is the complement to the above, replacing a specific call to an
   /// intrinsic function with a call to the specified new function.
namespace llvm {
 
   /// Upgrade the datalayout string by adding a section for address space
   /// pointers.
   std::string UpgradeDataLayoutString(StringRef DL, StringRef Triple);
 
   /// Upgrade attributes that changed format or kind.
   void UpgradeAttributes(AttrBuilder &B);
 
+  /// Upgrade operand bundles (without knowing about their user instruction).
+  void UpgradeOperandBundles(std::vector<OperandBundleDef> &OperandBundles);
+
 } // End llvm namespace
 
 #endif

llvm/lib/Bitcode/Reader/BitcodeReader.cpp

case bitc::FUNC_CODE_INST_INVOKE: {
           Value *Op;
           if (getValueTypePair(Record, OpNum, NextValueNo, Op))
             return error("Invalid record");
           Ops.push_back(Op);
           ArgsTys.push_back(Op->getType());
         }
       }
 
+      // Upgrade the bundles if needed.
+      if (!OperandBundles.empty())
+        UpgradeOperandBundles(OperandBundles);
+
       I = InvokeInst::Create(FTy, Callee, NormalBB, UnwindBB, Ops,
                              OperandBundles);
       OperandBundles.clear();
       InstructionList.push_back(I);
       cast<InvokeInst>(I)->setCallingConv(
           static_cast<CallingConv::ID>(CallingConv::MaxID & CCInfo));
       cast<InvokeInst>(I)->setAttributes(PAL);
       propagateAttributeTypes(cast<CallBase>(I), ArgsTys);
case bitc::FUNC_CODE_INST_CALLBR: {
           Value *Op;
           if (getValueTypePair(Record, OpNum, NextValueNo, Op))
             return error("Invalid record");
           Args.push_back(Op);
           ArgsTys.push_back(Op->getType());
         }
       }
 
+      // Upgrade the bundles if needed.
+      if (!OperandBundles.empty())
+        UpgradeOperandBundles(OperandBundles);
+
       I = CallBrInst::Create(FTy, Callee, DefaultDest, IndirectDests, Args,
                              OperandBundles);
       OperandBundles.clear();
       InstructionList.push_back(I);
       cast<CallBrInst>(I)->setCallingConv(
           static_cast<CallingConv::ID>((0x7ff & CCInfo) >> bitc::CALL_CCONV));
       cast<CallBrInst>(I)->setAttributes(PAL);
       propagateAttributeTypes(cast<CallBase>(I), ArgsTys);
case bitc::FUNC_CODE_INST_CALL: {
           Value *Op;
           if (getValueTypePair(Record, OpNum, NextValueNo, Op))
             return error("Invalid record");
           Args.push_back(Op);
           ArgsTys.push_back(Op->getType());
         }
       }
 
+      // Upgrade the bundles if needed.
+      if (!OperandBundles.empty())
+        UpgradeOperandBundles(OperandBundles);
+
       I = CallInst::Create(FTy, Callee, Args, OperandBundles);
       OperandBundles.clear();
       InstructionList.push_back(I);
       cast<CallInst>(I)->setCallingConv(
           static_cast<CallingConv::ID>((0x7ff & CCInfo) >> bitc::CALL_CCONV));
       CallInst::TailCallKind TCK = CallInst::TCK_None;
       if (CCInfo & 1 << bitc::CALL_TAIL)
         TCK = CallInst::TCK_Tail;

llvm/lib/IR/AutoUpgrade.cpp

void llvm::UpgradeAttributes(AttrBuilder &B) {
   if (A.isValid()) {
     // The value can be "true" or "false".
     bool NullPointerIsValid = A.getValueAsString() == "true";
     B.removeAttribute("null-pointer-is-valid");
     if (NullPointerIsValid)
       B.addAttribute(Attribute::NullPointerIsValid);
   }
 }
+
+void llvm::UpgradeOperandBundles(std::vector<OperandBundleDef> &Bundles) {
+
+  // clang.arc.attachedcall bundles are now required to have an operand.
+  // If they don't, it's okay to drop them entirely: when there is an operand,
+  // the "attachedcall" is meaningful and required, but without an operand,
+  // it's just a marker NOP.  Dropping it merely prevents an optimization.
+  erase_if(Bundles, [&](OperandBundleDef &OBD) {
+    return OBD.getTag() == "clang.arc.attachedcall" &&
+           OBD.inputs().empty();
+  });
+}

llvm/lib/IR/Verifier.cpp

void Verifier::verifyAttachedCallBundle(const CallBase &Call,
 
   Assert((FTy->getReturnType()->isPointerTy() ||
           (Call.doesNotReturn() && FTy->getReturnType()->isVoidTy())),
          "a call with operand bundle \"clang.arc.attachedcall\" must call a "
          "function returning a pointer or a non-returning function that has a "
          "void return type",
          Call);
 
-  Assert((BU.Inputs.empty() ||
-          (BU.Inputs.size() == 1 && isa<Function>(BU.Inputs.front()))),
-         "operand bundle \"clang.arc.attachedcall\" can take either no "
-         "arguments or one function as an argument",
+  Assert(BU.Inputs.size() == 1 && isa<Function>(BU.Inputs.front()),
+         "operand bundle \"clang.arc.attachedcall\" requires one function as "
+         "an argument",
          Call);
 
-  if (BU.Inputs.empty())
-    return;
-
   auto *Fn = cast<Function>(BU.Inputs.front());
   Intrinsic::ID IID = Fn->getIntrinsicID();
 
   if (IID) {
     Assert((IID == Intrinsic::objc_retainAutoreleasedReturnValue ||
             IID == Intrinsic::objc_unsafeClaimAutoreleasedReturnValue),
            "invalid function argument", Call);
   } else {

llvm/lib/Target/AArch64/AArch64ExpandPseudoInsts.cpp

BuildMI(MBB, MBBI, MI.getDebugLoc(), TII->get(Opc))
         .addImm(ImmOffset);
   }
   MI.eraseFromParent();
   return true;
 }
 
 bool AArch64ExpandPseudo::expandCALL_RVMARKER(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI) {
-  // Expand CALL_RVMARKER pseudo to a branch, followed by the special `mov x29,
-  // x29` marker. Mark the sequence as bundle, to avoid passes moving other code
-  // in between.
+  // Expand CALL_RVMARKER pseudo to:
+  // - a branch to the call target, followed by
+  // - the special `mov x29, x29` marker, and
+  // - another branch, to the runtime function
+  // Mark the sequence as bundle, to avoid passes moving other code in between.
   MachineInstr &MI = *MBBI;
 
   MachineInstr *OriginalCall;
-  MachineOperand &CallTarget = MI.getOperand(0);
+  MachineOperand &RVTarget = MI.getOperand(0);
+  MachineOperand &CallTarget = MI.getOperand(1);
   assert((CallTarget.isGlobal() || CallTarget.isReg()) &&
          "invalid operand for regular call");
+  assert(RVTarget.isGlobal() && "invalid operand for attached call");
   unsigned Opc = CallTarget.isGlobal() ? AArch64::BL : AArch64::BLR;
   OriginalCall = BuildMI(MBB, MBBI, MI.getDebugLoc(), TII->get(Opc)).getInstr();
   OriginalCall->addOperand(CallTarget);
 
-  unsigned RegMaskStartIdx = 1;
+  unsigned RegMaskStartIdx = 2;
   // Skip register arguments. Those are added during ISel, but are not
   // needed for the concrete branch.
   while (!MI.getOperand(RegMaskStartIdx).isRegMask()) {
     auto MOP = MI.getOperand(RegMaskStartIdx);
     assert(MOP.isReg() && "can only add register operands");
     OriginalCall->addOperand(MachineOperand::CreateReg(
         MOP.getReg(), /*Def=*/false, /*Implicit=*/true));
     RegMaskStartIdx++;
   }
   for (const MachineOperand &MO :
        llvm::drop_begin(MI.operands(), RegMaskStartIdx))
     OriginalCall->addOperand(MO);
 
-  auto *Marker = BuildMI(MBB, MBBI, MI.getDebugLoc(), TII->get(AArch64::ORRXrs))
+  BuildMI(MBB, MBBI, MI.getDebugLoc(), TII->get(AArch64::ORRXrs))
                      .addReg(AArch64::FP, RegState::Define)
                      .addReg(AArch64::XZR)
                      .addReg(AArch64::FP)
-                     .addImm(0)
+                     .addImm(0);
+
+  auto *RVCall = BuildMI(MBB, MBBI, MI.getDebugLoc(), TII->get(AArch64::BL))
+                     .add(RVTarget)
                      .getInstr();
+
   if (MI.shouldUpdateCallSiteInfo())
-    MBB.getParent()->moveCallSiteInfo(&MI, Marker);
+    MBB.getParent()->moveCallSiteInfo(&MI, OriginalCall);
+
   MI.eraseFromParent();
   finalizeBundle(MBB, OriginalCall->getIterator(),
-                 std::next(Marker->getIterator()));
+                 std::next(RVCall->getIterator()));
   return true;
 }
 
 bool AArch64ExpandPseudo::expandStoreSwiftAsyncContext(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI) {
   Register CtxReg = MBBI->getOperand(0).getReg();
   Register BaseReg = MBBI->getOperand(1).getReg();
   int Offset = MBBI->getOperand(2).getImm();

llvm/lib/Target/AArch64/AArch64ISelLowering.cpp

if (IsTailCall) {
     MF.getFrameInfo().setHasTailCall();
     SDValue Ret = DAG.getNode(AArch64ISD::TC_RETURN, DL, NodeTys, Ops);
     DAG.addCallSiteInfo(Ret.getNode(), std::move(CSInfo));
     return Ret;
   }
 
   unsigned CallOpc = AArch64ISD::CALL;
   // Calls with operand bundle "clang.arc.attachedcall" are special. They should
-  // be expanded to the call, directly followed by a special marker sequence.
-  // Use the CALL_RVMARKER to do that.
+  // be expanded to the call, directly followed by a special marker sequence and
+  // a call to an ObjC library function.  Use CALL_RVMARKER to do that.
   if (CLI.CB && objcarc::hasAttachedCallOpBundle(CLI.CB)) {
     assert(!IsTailCall &&
            "tail calls cannot be marked with clang.arc.attachedcall");
     CallOpc = AArch64ISD::CALL_RVMARKER;
+
+    // Add a target global address for the retainRV/claimRV runtime function
+    // just before the call target.
+    Function *ARCFn = *objcarc::getAttachedARCFunction(CLI.CB);
+    auto GA = DAG.getTargetGlobalAddress(ARCFn, DL, PtrVT);
+    Ops.insert(Ops.begin() + 1, GA);
   }
 
   // Returns a chain and a flag for retval copy to use.
   Chain = DAG.getNode(CallOpc, DL, NodeTys, Ops);
   DAG.addNoMergeSiteInfo(Chain.getNode(), CLI.NoMerge);
   InFlag = Chain.getValue(1);
   DAG.addCallSiteInfo(Chain.getNode(), std::move(CSInfo));
 

llvm/lib/Target/AArch64/AArch64InstrInfo.td

 
 def : Pat<(AArch64call GPR64:$Rn),
           (BLR GPR64:$Rn)>,
       Requires<[NoSLSBLRMitigation]>;
 def : Pat<(AArch64call GPR64noip:$Rn),
           (BLRNoIP GPR64noip:$Rn)>,
       Requires<[SLSBLRMitigation]>;
 
-def : Pat<(AArch64call_rvmarker GPR64:$Rn),
-          (BLR_RVMARKER GPR64:$Rn)>,
+def : Pat<(AArch64call_rvmarker (i64 tglobaladdr:$rvfunc), GPR64:$Rn),
+          (BLR_RVMARKER tglobaladdr:$rvfunc, GPR64:$Rn)>,
       Requires<[NoSLSBLRMitigation]>;
 
 let isBranch = 1, isTerminator = 1, isBarrier = 1, isIndirectBranch = 1 in {
 def BR  : BranchReg<0b0000, "br", [(brind GPR64:$Rn)]>;
 } // isBranch, isTerminator, isBarrier, isIndirectBranch
 
 // Create a separate pseudo-instruction for codegen to use so that we don't
 // flag lr as used in every function. It'll be restored before the RET by the

llvm/lib/Transforms/ObjCARC/ObjCARC.h

 /// going to be removed from the IR before WinEHPrepare.
 CallInst *createCallInstWithColors(
     FunctionCallee Func, ArrayRef<Value *> Args, const Twine &NameStr,
     Instruction *InsertBefore,
     const DenseMap<BasicBlock *, ColorVector> &BlockColors);
 
 class BundledRetainClaimRVs {
 public:
-  BundledRetainClaimRVs(bool ContractPass, bool UseMarker)
-      : ContractPass(ContractPass), UseMarker(UseMarker) {}
+  BundledRetainClaimRVs(bool ContractPass)
+      : ContractPass(ContractPass) {}
   ~BundledRetainClaimRVs();
 
   /// Insert a retainRV/claimRV call to the normal destination blocks of invokes
   /// with operand bundle "clang.arc.attachedcall". If the edge to the normal
   /// destination block is a critical edge, split it.
   std::pair<bool, bool> insertAfterInvokes(Function &F, DominatorTree *DT);
 
   /// Insert a retainRV/claimRV call.
void eraseInst(CallInst *CI) {
     EraseInstruction(CI);
   }
 
 private:
   /// A map of inserted retainRV/claimRV calls to annotated calls/invokes.
   DenseMap<CallInst *, CallBase *> RVCalls;
 
   bool ContractPass;
-
-  /// Indicates whether the target uses a special inline-asm marker.
-  bool UseMarker;
 };
 
 } // end namespace objcarc
 } // end namespace llvm
 
 #endif

llvm/lib/Transforms/ObjCARC/ObjCARC.cpp

for (auto P : RVCalls) {
     if (ContractPass) {
       CallBase *CB = P.second;
       // At this point, we know that the annotated calls can't be tail calls
       // as they are followed by marker instructions and retainRV/claimRV
       // calls. Mark them as notail so that the backend knows these calls
       // can't be tail calls.
       if (auto *CI = dyn_cast<CallInst>(CB))
         CI->setTailCallKind(CallInst::TCK_NoTail);
-
-      if (UseMarker) {
-        // Remove the retainRV/claimRV function operand from the operand bundle
-        // to reflect the fact that the backend is responsible for emitting only
-        // the marker instruction, but not the retainRV/claimRV call.
-        OperandBundleDef OB("clang.arc.attachedcall", None);
-        auto *NewCB = CallBase::Create(CB, OB, CB);
-        CB->replaceAllUsesWith(NewCB);
-        CB->eraseFromParent();
-      }
     }
 
-    if (!ContractPass || !UseMarker)
-      EraseInstruction(P.first);
   }
 
   RVCalls.clear();
 }

llvm/lib/Transforms/ObjCARC/ObjCARCContract.cpp

case ARCInstKind::Retain:
     // Attempt to convert retains to retainrvs if they are next to function
     // calls.
     if (!optimizeRetainCall(F, Inst))
       return false;
     // If we succeed in our optimization, fall through.
     LLVM_FALLTHROUGH;
   case ARCInstKind::RetainRV:
   case ARCInstKind::UnsafeClaimRV: {
-    bool IsInstContainedInBundle = BundledInsts->contains(Inst);
-
-    // Return now if the target doesn't need a special inline-asm marker. Return
-    // true if this is a bundled retainRV/claimRV call, which is going to be
-    // erased at the end of this pass, to avoid undoing objc-arc-expand and
+    // Return true if this is a bundled retainRV/claimRV call, which is always
+    // redundant with the attachedcall in the bundle, and is going to be erased
+    // at the end of this pass.  This avoids undoing objc-arc-expand and
     // replacing uses of the retainRV/claimRV call's argument with its result.
-    if (!RVInstMarker)
-      return IsInstContainedInBundle;
-
-    // The target needs a special inline-asm marker.
+    if (BundledInsts->contains(Inst))
+      return true;
 
-    // We don't have to emit the marker if this is a bundled call since the
-    // backend is responsible for emitting it. Return false to undo
-    // objc-arc-expand.
-    if (IsInstContainedInBundle)
+    // If this isn't a bundled call, and the target doesn't need a special
+    // inline-asm marker, we're done: return now, and undo objc-arc-expand.
+    if (!RVInstMarker)
       return false;
 
+    // The target needs a special inline-asm marker.  Insert it.
+
     BasicBlock::iterator BBI = Inst->getIterator();
     BasicBlock *InstParent = Inst->getParent();
 
     // Step up to see if the call immediately precedes the RV call.
     // If it's an invoke, we have to cross a block boundary. And we have
     // to carefully dodge no-op instructions.
     do {
       if (BBI == InstParent->begin()) {

 bool ObjCARCContract::run(Function &F, AAResults *A, DominatorTree *D) {
   if (!EnableARCOpts)
     return false;
 
   Changed = CFGChanged = false;
   AA = A;
   DT = D;
   PA.setAA(A);
-  BundledRetainClaimRVs BRV(true, RVInstMarker);
+  BundledRetainClaimRVs BRV(/*ContractPass=*/true);
   BundledInsts = &BRV;
 
   std::pair<bool, bool> R = BundledInsts->insertAfterInvokes(F, DT);
   Changed |= R.first;
   CFGChanged |= R.second;
 
   DenseMap<BasicBlock *, ColorVector> BlockColors;
   if (F.hasPersonalityFn() &&

llvm/lib/Transforms/ObjCARC/ObjCARCOpts.cpp

void ObjCARCOpt::init(Module &M) {
   EP.init(&M);
 }
 
 bool ObjCARCOpt::run(Function &F, AAResults &AA) {
   if (!EnableARCOpts)
     return false;
 
   Changed = CFGChanged = false;
-  BundledRetainClaimRVs BRV(false, objcarc::getRVInstMarker(*F.getParent()));
+  BundledRetainClaimRVs BRV(/*ContractPass=*/false);
   BundledInsts = &BRV;
 
   LLVM_DEBUG(dbgs() << "<<< ObjCARCOpt: Visiting Function: " << F.getName()
                     << " >>>"
                        "\n");
 
   std::pair<bool, bool> R = BundledInsts->insertAfterInvokes(F, nullptr);
   Changed |= R.first;

llvm/test/Bitcode/upgrade-arc-attachedcall-bundle.ll

+; Test that nullary clang.arc.attachedcall operand bundles are "upgraded".
+
+; RUN: llvm-dis %s.bc -o - | FileCheck %s
+; RUN: verify-uselistorder %s.bc
+
+define i8* @invalid() {
+; CHECK-LABEL: define i8* @invalid() {
+; CHECK-NEXT:   %tmp0 = call i8* @foo(){{$}}
+; CHECK-NEXT:   ret i8* %tmp0
+  %tmp0 = call i8* @foo() [ "clang.arc.attachedcall"() ]
+  ret i8* %tmp0
+}
+
+define i8* @valid() {
+; CHECK-LABEL: define i8* @valid() {
+; CHECK-NEXT:   %tmp0 = call i8* @foo() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.retainAutoreleasedReturnValue) ]
+; CHECK-NEXT:   ret i8* %tmp0
+  %tmp0 = call i8* @foo() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.retainAutoreleasedReturnValue) ]
+  ret i8* %tmp0
+}
+
+declare i8* @foo()
+declare i8* @llvm.objc.retainAutoreleasedReturnValue(i8*)

llvm/test/CodeGen/AArch64/call-rv-marker.ll

 declare void @llvm.lifetime.end.p0i8(i64 immarg, i8* nocapture)
 
 
 %struct.S = type { i8 }
 
 @g = dso_local global i8* null, align 8
 @fptr = dso_local global i8* ()* null, align 8
 
-define dso_local i8* @rv_marker_1() {
-; CHECK-LABEL:    rv_marker_1:
+define dso_local i8* @rv_marker_1_retain() {
+; CHECK-LABEL:    rv_marker_1_retain:
 ; CHECK:           .cfi_offset w30, -16
 ; CHECK-NEXT:      bl foo1
 ; SELDAG-NEXT:     mov x29, x29
+; SELDAG-NEXT:     bl objc_retainAutoreleasedReturnValue
 ; GISEL-NOT:       mov x29, x29
 ;
 entry:
-  %call = call i8* @foo1() [ "clang.arc.attachedcall"() ]
+  %call = call i8* @foo1() [ "clang.arc.attachedcall"(i8* (i8*)* @objc_retainAutoreleasedReturnValue) ]
+  ret i8* %call
+}
+
+define dso_local i8* @rv_marker_1_unsafeClaim() {
+; CHECK-LABEL:    rv_marker_1_unsafeClaim:
+; CHECK:           .cfi_offset w30, -16
+; CHECK-NEXT:      bl foo1
+; SELDAG-NEXT:     mov x29, x29
+; SELDAG-NEXT:     bl objc_unsafeClaimAutoreleasedReturnValue
+; GISEL-NOT:       mov x29, x29
+;
+entry:
+  %call = call i8* @foo1() [ "clang.arc.attachedcall"(i8* (i8*)* @objc_unsafeClaimAutoreleasedReturnValue) ]
   ret i8* %call
 }
 
 define dso_local void @rv_marker_2_select(i32 %c) {
 ; CHECK-LABEL: rv_marker_2_select:
 ; SELDAG:        cinc  w0, w8, eq
 ; GISEL:         csinc w0, w8, wzr, eq
 ; CHECK-NEXT:    bl  foo0
 ; SELDAG-NEXT:   mov x29, x29
+; SELDAG-NEXT:   bl objc_retainAutoreleasedReturnValue
 ; CHECK-NEXT:    ldr x30, [sp], #16
 ; CHECK-NEXT:    b  foo2
 ;
 entry:
   %tobool.not = icmp eq i32 %c, 0
   %.sink = select i1 %tobool.not, i32 2, i32 1
-  %call1 = call i8* @foo0(i32 %.sink) [ "clang.arc.attachedcall"() ]
+  %call1 = call i8* @foo0(i32 %.sink) [ "clang.arc.attachedcall"(i8* (i8*)* @objc_retainAutoreleasedReturnValue) ]
   tail call void @foo2(i8* %call1)
   ret void
 }
 
 define dso_local void @rv_marker_3() personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
 ; CHECK-LABEL: rv_marker_3
 ; CHECK:         .cfi_offset w30, -32
 ; CHECK-NEXT:    bl  foo1
 ; SELDAG-NEXT:   mov x29, x29
+; SELDAG-NEXT:   bl objc_retainAutoreleasedReturnValue
 ;
 entry:
-  %call = call i8* @foo1() [ "clang.arc.attachedcall"() ]
+  %call = call i8* @foo1() [ "clang.arc.attachedcall"(i8* (i8*)* @objc_retainAutoreleasedReturnValue) ]
   invoke void @objc_object(i8* %call) #5
           to label %invoke.cont unwind label %lpad
 
 invoke.cont:                                      ; preds = %entry
   tail call void @llvm.objc.release(i8* %call)
   ret void
 
 lpad:                                             ; preds = %entry
   %0 = landingpad { i8*, i32 }
           cleanup
   tail call void @llvm.objc.release(i8* %call)
   resume { i8*, i32 } %0
 }
 
 define dso_local void @rv_marker_4() personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
 ; CHECK-LABEL: rv_marker_4
 ; CHECK:       .Ltmp3:
 ; CHECK-NEXT:     bl  foo1
 ; SELDAG-NEXT:    mov x29, x29
+; SELDAG-NEXT:    bl objc_retainAutoreleasedReturnValue
 ; CHECK-NEXT: .Ltmp4:
 ;
 entry:
   %s = alloca %struct.S, align 1
   %0 = getelementptr inbounds %struct.S, %struct.S* %s, i64 0, i32 0
   call void @llvm.lifetime.start.p0i8(i64 1, i8* nonnull %0) #2
-  %call = invoke i8* @foo1() [ "clang.arc.attachedcall"() ]
+  %call = invoke i8* @foo1() [ "clang.arc.attachedcall"(i8* (i8*)* @objc_retainAutoreleasedReturnValue) ]
           to label %invoke.cont unwind label %lpad
 
 invoke.cont:                                      ; preds = %entry
   invoke void @objc_object(i8* %call) #5
           to label %invoke.cont2 unwind label %lpad1
 
 invoke.cont2:                                     ; preds = %invoke.cont
   tail call void @llvm.objc.release(i8* %call)
ehcleanup:                                        ; preds = %lpad1, %lpad
   call void @llvm.lifetime.end.p0i8(i64 1, i8* nonnull %0)
   resume { i8*, i32 } %.pn
 }
 
 define dso_local i8* @rv_marker_5_indirect_call() {
 ; CHECK-LABEL: rv_marker_5_indirect_call
 ; CHECK:         ldr [[ADDR:x[0-9]+]], [
 ; CHECK-NEXT:    blr [[ADDR]]
-; SLEDAG-NEXT:   mov x29, x29
+; SELDAG-NEXT:   mov x29, x29
+; SELDAG-NEXT:   bl objc_retainAutoreleasedReturnValue
 ; GISEL-NOT:     mov x29, x29
 ;
 entry:
   %0 = load i8* ()*, i8* ()** @fptr, align 8
-  %call = call i8* %0() [ "clang.arc.attachedcall"() ]
+  %call = call i8* %0() [ "clang.arc.attachedcall"(i8* (i8*)* @objc_retainAutoreleasedReturnValue) ]
   tail call void @foo2(i8* %call)
   ret i8* %call
 }
 
 declare i8* @foo(i64, i64, i64)
 
 define dso_local void @rv_marker_multiarg(i64 %a, i64 %b, i64 %c) {
 ; CHECK-LABEL: rv_marker_multiarg
 ; CHECK:        mov [[TMP:x[0-9]+]], x0
 ; CHECK-NEXT:   mov x0, x2
 ; CHECK-NEXT:   mov x2, [[TMP]]
 ; CHECK-NEXT:   bl  foo
 ; SELDAG-NEXT:  mov x29, x29
+; SELDAG-NEXT:  bl objc_retainAutoreleasedReturnValue
 ; GISEL-NOT:    mov x29, x29
-  call i8* @foo(i64 %c, i64 %b, i64 %a) [ "clang.arc.attachedcall"() ]
+  call i8* @foo(i64 %c, i64 %b, i64 %a) [ "clang.arc.attachedcall"(i8* (i8*)* @objc_retainAutoreleasedReturnValue) ]
   ret void
 }
 
+declare i8* @objc_retainAutoreleasedReturnValue(i8*)
+declare i8* @objc_unsafeClaimAutoreleasedReturnValue(i8*)
 declare i32 @__gxx_personality_v0(...)

llvm/test/CodeGen/AArch64/expand-blr-rvmarker-pseudo.mir

--- |
   define void @test_blr_pass_w0_w1_args() {
     ret void
   }
 
   define void @foo(i32 %a) {
     ret void
   }
 
+  declare i8* @attachedcall()
 ...
 ---
 
 # CHECK-LABEL: : test_1_callsite_info
 # CHECK:       bb.0:
 # CHECK-NEXT:    BUNDLE implicit-def $lr, implicit-def $w30, implicit-def $sp, implicit-def $wsp, implicit-def dead $x0, implicit-def $fp, implicit-def $w29, implicit $x0, implicit $sp, implicit $xzr, implicit $fp {
 # CHECK-NEXT:      BLR $x0, csr_aarch64_aapcs, implicit-def $lr, implicit $sp, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def dead $x0
 # CHECK-NEXT:      ORRXrs $xzr, $fp, 0
+# CHECK-NEXT:      BL @attachedcall, implicit-def $lr, implicit internal $sp
 # CHECK-NEXT:   }
 # CHECK-NEXT:    RET undef $lr, implicit killed $w0
 #
 name: test_1_callsite_info
 callSites:
   - {bb: 0, offset: 0, fwdArgRegs:
     - { arg: 0, reg: '$x0' } }
 body:             |
     bb.0:
         liveins: $lr, $x0
 
-        BLR_RVMARKER $x0, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def dead $x0
+        BLR_RVMARKER @attachedcall, $x0, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def dead $x0
         RET_ReallyLR implicit killed $w0
 ...
 
 # CHECK-LABEL: : test_bl_pass_x0_arg
 # CHECK:       bb.0:
 # CHECK-NEXT:     BUNDLE implicit-def $lr, implicit-def $w30, implicit-def $sp, implicit-def $wsp, implicit-def dead $x0, implicit-def $fp, implicit-def $w29, implicit $sp, implicit $x0, implicit $xzr, implicit $fp {
 # CHECK-NEXT:      BL @foo, csr_aarch64_aapcs, implicit-def $lr, implicit $sp, implicit $x0, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def dead $x0
 # CHECK-NEXT:      $fp = ORRXrs $xzr, $fp, 0
+# CHECK-NEXT:      BL @attachedcall, implicit-def $lr, implicit internal $sp
 # CHECK-NEXT:    }
 # CHECK-NEXT:    RET undef $lr, implicit killed $w0
 #
 name: test_bl_pass_x0_arg
 body:             |
     bb.0:
         liveins: $lr, $x0
 
-        BLR_RVMARKER @foo, $x0, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def dead $x0
+        BLR_RVMARKER @attachedcall, @foo, $x0, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def dead $x0
         RET_ReallyLR implicit killed $w0
 ...
 
 # CHECK-LABEL: : test_bl_pass_x0_x1_x2_args
 # CHECK:       bb.0:
 # CHECK-NEXT:     BUNDLE implicit-def $lr, implicit-def $w30, implicit-def $sp, implicit-def $wsp, implicit-def $x0, implicit-def $w0, implicit-def $fp, implicit-def $w29, implicit $sp, implicit $x0, implicit $x1, implicit $x2, implicit $xzr, implicit $fp {
 # CHECK-NEXT:      BL @foo, csr_aarch64_aapcs, implicit-def $lr, implicit $sp, implicit $x0, implicit $x1, implicit $x2, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def $x0
 # CHECK-NEXT:      $fp = ORRXrs $xzr, $fp, 0
+# CHECK-NEXT:      BL @attachedcall, implicit-def $lr, implicit internal $sp
 # CHECK-NEXT:    }
 # CHECK-NEXT:    RET undef $lr
 #
 name: test_bl_pass_x0_x1_x2_args
 body:             |
     bb.0:
         liveins: $lr, $x0, $x1, $x2
 
-        BLR_RVMARKER @foo, $x0, $x1, $x2, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def $x0
+        BLR_RVMARKER @attachedcall, @foo, $x0, $x1, $x2, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def $x0
         RET_ReallyLR
 ...
 
 # CHECK-LABEL: : test_bl_pass_w0_w1_args
 # CHECK:       bb.0:
 # CHECK-NEXT:     BUNDLE implicit-def $lr, implicit-def $w30, implicit-def $sp, implicit-def $wsp, implicit-def dead $x0, implicit-def $fp, implicit-def $w29, implicit $sp, implicit $w0, implicit $w1, implicit $xzr, implicit $fp {
 # CHECK-NEXT:      BL @foo, csr_aarch64_aapcs, implicit-def $lr, implicit $sp, implicit $w0, implicit $w1, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def dead $x0
 # CHECK-NEXT:      $fp = ORRXrs $xzr, $fp, 0
+# CHECK-NEXT:      BL @attachedcall, implicit-def $lr, implicit internal $sp
 # CHECK-NEXT:    }
 # CHECK-NEXT:    RET undef $lr, implicit killed $w0
 #
 name: test_bl_pass_w0_w1_args
 body:             |
     bb.0:
         liveins: $lr, $w0, $w1
 
-        BLR_RVMARKER @foo, $w0, $w1, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def dead $x0
+        BLR_RVMARKER @attachedcall, @foo, $w0, $w1, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def dead $x0
         RET_ReallyLR implicit killed $w0
 ...
 
 
 # CHECK-LABEL: : test_blr_pass_w0_w1_args
 # CHECK:       bb.0:
 # CHECK-NEXT:     BUNDLE implicit-def $lr, implicit-def $w30, implicit-def $sp, implicit-def $wsp, implicit-def dead $x0, implicit-def $fp, implicit-def $w29, implicit $x8, implicit $sp, implicit $w0, implicit $w1, implicit $xzr, implicit $fp {
 # CHECK-NEXT:      BLR $x8, csr_aarch64_aapcs, implicit-def $lr, implicit $sp, implicit $w0, implicit $w1, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def dead $x0
 # CHECK-NEXT:      $fp = ORRXrs $xzr, $fp, 0
+# CHECK-NEXT:      BL @attachedcall, implicit-def $lr, implicit internal $sp
 # CHECK-NEXT:    }
 # CHECK-NEXT:    RET undef $lr, implicit killed $w0
 #
 name: test_blr_pass_w0_w1_args
 body:             |
     bb.0:
         liveins: $lr, $x8, $w0, $w1
 
-        BLR_RVMARKER $x8, $w0, $w1, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def dead $x0
+        BLR_RVMARKER @attachedcall, $x8, $w0, $w1, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def dead $x0
         RET_ReallyLR implicit killed $w0
 ...

llvm/test/CodeGen/AArch64/rvmarker-pseudo-expansion-and-outlining.mir

 # RUN: llc -enable-machine-outliner -start-before=aarch64-expand-pseudo -mtriple=arm64-apple-ios -o - %s | FileCheck %s
 #
 #  The calls to _cb1 & _cb2 should be followed by the marker instruction
-#  'mov x29, x29'. The marker should not get outlined.
+#  'mov x29, x29' and the attached call. Neither should get outlined.
 #
 # CHECK-LABEL: _fn1:
 # CHECK:       bb.0:
 # CHECK:        bl _cb1
 # CHECK-NEXT:   mov x29, x29
+# CHECK-NEXT:   bl _attachedcall
 # CHECK:        b _OUTLINED_FUNCTION_0
 #
 # CHECK-LABEL: _fn2:
 # CHECK:       bb.0:
 # CHECK:        bl _cb2
 # CHECK-NEXT:   mov x29, x29
+# CHECK-NEXT:   bl _attachedcall
 # CHECK:        b _OUTLINED_FUNCTION_0
 #
 # CHECK-LABEL: _OUTLINED_FUNCTION_0:
 # CHECK:       bb.0:
 # CHECK-NEXT:    orr w12, wzr, #0x3
 # CHECK-NEXT:    orr w12, wzr, #0x3
 # CHECK-NEXT:    orr w12, wzr, #0x3
 # CHECK-NEXT:    orr w12, wzr, #0x3
--- |
 
   define void @fn2() {
     ret void
   }
 
   declare void @cb1()
 
   declare void @cb2()
+
+  declare i8* @attachedcall()
 ...
 ---
 name:            fn1
 tracksRegLiveness: true
 machineFunctionInfo:
   hasRedZone:      false
 body:             |
   bb.0:
     liveins: $lr
 
-    BLR_RVMARKER @cb1, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def $x0
+    BLR_RVMARKER @attachedcall, @cb1, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def $x0
     $w12 = ORRWri $wzr, 1
     $w12 = ORRWri $wzr, 1
     $w12 = ORRWri $wzr, 1
     $w12 = ORRWri $wzr, 1
     $w12 = ORRWri $wzr, 1
     $lr = ORRXri $xzr, 1
 
     RET undef $lr
 ...
 ---
 name:            fn2
 tracksRegLiveness: true
 machineFunctionInfo:
   hasRedZone:      false
 body:             |
   bb.0:
     liveins: $lr, $x19, $x20, $lr
 
-    BLR_RVMARKER @cb2, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def $x0
+    BLR_RVMARKER @attachedcall, @cb2, csr_aarch64_aapcs, implicit-def dead $lr, implicit $sp, implicit-def $sp, implicit-def $x0
     $w12 = ORRWri $wzr, 1
     $w12 = ORRWri $wzr, 1
     $w12 = ORRWri $wzr, 1
     $w12 = ORRWri $wzr, 1
     $w12 = ORRWri $wzr, 1
     $lr = ORRXri $xzr, 1
 
     RET undef $lr
 ...

llvm/test/CodeGen/X86/call-rv-marker.ll

 ; CHECK-NEXT:    popq    %rcx
 ; CHECK-NEXT:    retq
 ;
 entry:
   %call = call i8* @foo1() [ "clang.arc.attachedcall"(i8* (i8*)* @objc_retainAutoreleasedReturnValue) ]
   ret i8* %call
 }
 
-define i8* @rv_marker_1_claim() {
-; CHECK-LABEL:  rv_marker_1_claim:
+define i8* @rv_marker_1_unsafeClaim() {
+; CHECK-LABEL:  rv_marker_1_unsafeClaim:
 ; CHECK:         pushq %rax
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
 ; CHECK-NEXT:    callq   _foo1
 ; CHECK-NEXT:    movq    %rax, %rdi
 ; CHECK-NEXT:    callq   _objc_unsafeClaimAutoreleasedReturnValue
 ; CHECK-NEXT:    popq    %rcx
 ; CHECK-NEXT:    retq
 ;

llvm/test/Transforms/ObjCARC/contract-marker-funclet.ll

 
 invoke.cont:                                      ; preds = %entry
   %4 = call i8* @llvm.objc.retainAutoreleasedReturnValue(i8* %call)
   call void @llvm.objc.release(i8* %4)
   ret void
 }
 
 ; CHECK-LABEL: define dso_local void @"?test_attr_claimRV@@YAXXZ"()
-; CHECK: %[[CALL4:.*]] = notail call i8* @"?noexcept_func@@YAPAUobjc_object@@XZ"() [ "clang.arc.attachedcall"() ]
-; CHECK: call i8* @llvm.objc.unsafeClaimAutoreleasedReturnValue(i8* %[[CALL4]])
+; CHECK: %[[CALL4:.*]] = notail call i8* @"?noexcept_func@@YAPAUobjc_object@@XZ"() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.unsafeClaimAutoreleasedReturnValue) ]
+; CHECK-NEXT: ret void
 
 ; CHECK: %[[V1:.*]] = cleanuppad
-; CHECK: %[[CALL:.*]] = notail call i8* @"?noexcept_func@@YAPAUobjc_object@@XZ"() [ "funclet"(token %[[V1]]), "clang.arc.attachedcall"() ]
-; CHECK: call i8* @llvm.objc.unsafeClaimAutoreleasedReturnValue(i8* %[[CALL]]) [ "funclet"(token %[[V1]]) ]
+; CHECK: %[[CALL:.*]] = notail call i8* @"?noexcept_func@@YAPAUobjc_object@@XZ"() [ "funclet"(token %[[V1]]), "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.unsafeClaimAutoreleasedReturnValue) ]
+; CHECK-NEXT: cleanupret from %[[V1]] unwind to caller
 
 define dso_local void @"?test_attr_claimRV@@YAXXZ"() local_unnamed_addr #0 personality i8* bitcast (i32 (...)* @__CxxFrameHandler3 to i8*) {
 entry:
   invoke void @"?foo@@YAXXZ"()
           to label %invoke.cont unwind label %ehcleanup
 
 invoke.cont:                                      ; preds = %entry
   %call.i4 = tail call i8* @"?noexcept_func@@YAPAUobjc_object@@XZ"() #2 [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.unsafeClaimAutoreleasedReturnValue) ]

llvm/test/Transforms/ObjCARC/contract-rv-attr.ll

 ; RUN: opt -objc-arc-contract -S < %s | FileCheck %s
 ; RUN: opt -passes=objc-arc-contract -S < %s | FileCheck %s
 
 ; CHECK-LABEL: define void @test0() {
-; CHECK: %[[CALL:.*]] = notail call i8* @foo() [ "clang.arc.attachedcall"() ]
-; CHECK: call i8* @llvm.objc.retainAutoreleasedReturnValue(i8* %[[CALL]])
+; CHECK: %[[CALL:.*]] = notail call i8* @foo() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.retainAutoreleasedReturnValue) ]
+; CHECK-NOT: call i8* @llvm.objc.retainAutoreleasedReturnValue(
 
 define void @test0() {
   %call1 = call i8* @foo() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.retainAutoreleasedReturnValue) ]
   ret void
 }
 
 ; CHECK-LABEL: define void @test1() {
-; CHECK: %[[CALL:.*]] = notail call i8* @foo() [ "clang.arc.attachedcall"() ]
-; CHECK: call i8* @llvm.objc.unsafeClaimAutoreleasedReturnValue(i8* %[[CALL]])
+; CHECK: %[[CALL:.*]] = notail call i8* @foo() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.unsafeClaimAutoreleasedReturnValue) ]
+; CHECK-NOT: call i8* @llvm.objc.unsafeClaimAutoreleasedReturnValue(
 
 define void @test1() {
   %call1 = call i8* @foo() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.unsafeClaimAutoreleasedReturnValue) ]
   ret void
 }
 
 ; CHECK-LABEL:define i8* @test2(
-; CHECK: %[[CALL1:.*]] = invoke i8* @foo() [ "clang.arc.attachedcall"() ]
+; CHECK: %[[V0:.*]] = invoke i8* @foo() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.retainAutoreleasedReturnValue) ]
 
-; CHECK: %[[V0:.*]] = call i8* @llvm.objc.retainAutoreleasedReturnValue(i8* %[[CALL1]])
-; CHECK-NEXT: br
+; CHECK-NOT: = call i8* @llvm.objc.retainAutoreleasedReturnValue(
+; CHECK: br
 
-; CHECK: %[[CALL3:.*]] = invoke i8* @foo() [ "clang.arc.attachedcall"() ]
+; CHECK: %[[V2:.*]] = invoke i8* @foo() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.retainAutoreleasedReturnValue) ]
 
-; CHECK: %[[V2:.*]] = call i8* @llvm.objc.retainAutoreleasedReturnValue(i8* %[[CALL3]])
-; CHECK-NEXT: br
+; CHECK-NOT: = call i8* @llvm.objc.retainAutoreleasedReturnValue(
+; CHECK: br
 
 ; CHECK: %[[RETVAL:.*]] = phi i8* [ %[[V0]], {{.*}} ], [ %[[V2]], {{.*}} ]
 ; CHECK: ret i8* %[[RETVAL]]
 
 define i8* @test2(i1 zeroext %b) personality i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*) {
 entry:
   br i1 %b, label %if.then, label %if.end
 

llvm/test/Verifier/operand-bundles.ll

entry:
   %l = load i32, i32* %ptr
   call void @g() [ "gc-transition"(i32 42, i64 100, i32 %x), "gc-transition"(float 0.0, i64 100, i32 %l) ]
   call void @g() [ "gc-transition"(i32 42, i64 120) ]  ;; The verifier should not complain about this one
   %x = add i32 42, 1
   ret void
 }
 
 define void @f_clang_arc_attachedcall() {
-; CHECK: Multiple "clang.arc.attachedcall" operand bundles
+; CHECK: requires one function as an argument
+; CHECK-NEXT: call %0* @foo0() [ "clang.arc.attachedcall"() ]
+; CHECK-NEXT: Multiple "clang.arc.attachedcall" operand bundles
 ; CHECK-NEXT: call %0* @foo0() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.retainAutoreleasedReturnValue), "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.retainAutoreleasedReturnValue) ]
 ; CHECK-NEXT: must call a function returning a pointer
 ; CHECK-NEXT: call i8 @foo1() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.retainAutoreleasedReturnValue) ]
 ; CHECK-NEXT: or a non-returning function
 ; CHECK-NEXT: call void @g() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.retainAutoreleasedReturnValue) ]
-; CHECK-NEXT: can take either no arguments
+; CHECK-NEXT: requires one function as an argument
 ; CHECK-NEXT: call %0* @foo0() [ "clang.arc.attachedcall"(i8* (i8*)* null) ]
-; CHECK-NEXT: can take either no arguments
+; CHECK-NEXT: requires one function as an argument
 ; CHECK-NEXT: call %0* @foo0() [ "clang.arc.attachedcall"(i64 0) ]
 ; CHECK-NEXT: invalid function argument
 ; CHECK-NEXT: call %0* @foo0() [ "clang.arc.attachedcall"(i8 ()* @foo1) ]
 ; CHECK-NEXT: invalid function argument
 ; CHECK-NEXT: call %0* @foo0() [ "clang.arc.attachedcall"(void (i1)* @llvm.assume) ]
 
   call %0* @foo0() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.retainAutoreleasedReturnValue) ]
   call %0* @foo0() [ "clang.arc.attachedcall"(i8* (i8*)* @llvm.objc.unsafeClaimAutoreleasedReturnValue) ]