[IR] Define "ptrauth" operand bundle.

[ab]

Summary

Building on D90868, this introduces a new "ptrauth" operand bundle to be used in call/invoke. At the IR level, it's semantically equivalent to an @llvm.ptrauth.auth followed by an indirect call, but it additionally provides hardening guarantees, by preventing the intermediate raw pointer from being exposed.

This mostly adds the IR definition, verifier checks, and support in a couple of general helper functions. clang IRGen and backend support will come separately.

Note that we'll eventually want to support this bundle in indirectbr as well, for similar reasons. indirectbr currently doesn't allow bundles at all, and the IR data structures need some rejiggering.

[bruno]

Given context from further patches in the set, I believe ptrauth operand bundle has been a good way to represent this and works well. LGTM

[kristof.beyls]

I just have 2 bike-sheddy comments on the documentation text.
My comments should not let you delay in getting this committed if they do not make sense to you.

llvm/docs/PointerAuth.md

234–235

If you allow me to bikeshed on this sentence: "guaranteeing an intermediate call target is never attackable" seems like a very bold claim to me.
I wonder if it would (a) be better, and (b) be possible to more concretely define exactly what is guaranteed in more detail.
Is what is currently guaranteed "it guarantees that the intermediate call target is kept in a register and never stored to memory, e.g. by being spilled"?
If we want to relate this guarantee to reduced attackability, a sentence could be added saying something like "Not storing and reloading the unauthenticated pointer to/from memory removes an attack surface where an attacker would overwrite the unauthenticated pointer in memory".

I'm assuming that no other guarantees are implemented?
If in the future it becomes clear that more guarantees need to be implemented, the documentation here can be extended to list further guarantees?

259

similarly to my other comment "are never attackable" seems a bit strong. Would "are never stored and reloaded to/from memory" be a more exact description of the added guarantee?

[ab]

Thanks!

I committed a modified description, happy to make more changes:

However, that exposes the intermediate, unauthenticated pointer, e.g., if it
gets spilled to the stack. An attacker can then overwrite the pointer in
memory, negating the security benefit provided by pointer authentication.
To prevent that, the `ptrauth` operand bundle may be used: it guarantees that
the intermediate call target is kept in a register and never stored to memory.
This hardening benefit is similar to that provided by
[`llvm.ptrauth.resign`](#llvm-ptrauth-resign)).

llvm/docs/PointerAuth.md

234–235

That's fair, I reworded it based on your comment, to specifically mention the main hardening measure: preventing spills.

But for context: there is indeed more hardening provided here: for all the combined "hardened" ptrauth operations we have, we use only specific registers for "critical" values (e.g., the unauthenticated pointer result of the auth here), and the darwin kernel provides additional runtime guarantees on these registers (so that they aren't corruptible even across context switches, etc.)
Another layer is the trapping behavior on auth failures: for several of these operations (including these calls) we rely on the hardware checks, but when we can't, we have the software check/trap sequence, so that these can't be bruteforced.

Either way, that belongs somewhere else in the document, given that it's a property common to most of these operations.

llvm/docs/LangRef.rst

.. code-block:: llvm
    ; The marker instruction and a runtime function call are inserted after the call
    ; to @foo.
    call i8* @foo() [ "clang.arc.attachedcall"(i8* (i8*)* @objc_retainAutoreleasedReturnValue) ]
    call i8* @foo() [ "clang.arc.attachedcall"(i8* (i8*)* @objc_unsafeClaimAutoreleasedReturnValue) ]
 
 The operand bundle is needed to ensure the call is immediately followed by the
 marker instruction and the ObjC runtime call in the final output.
 
+.. _ob_ptrauth:
+
+Pointer Authentication Operand Bundles
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Pointer Authentication operand bundles are characterized by the
+``"ptrauth"`` operand bundle tag.  They are described in the
+`Pointer Authentication <PointerAuth.html#operand-bundle>`_ document.
+
 .. _moduleasm:
 
 Module-Level Inline Assembly
 ----------------------------
 
 Modules may contain "module-level inline asm" blocks, which corresponds
 to the GCC "file scope inline asm" blocks. These blocks are internally
 concatenated by LLVM and treated as a single unit, but may be separated

llvm/docs/PointerAuth.md

 # Pointer Authentication
 
 ## Introduction
 
 Pointer Authentication is a mechanism by which certain pointers are signed.
 When a pointer gets signed, a cryptographic hash of its value and other values
 (pepper and salt) is stored in unused bits of that pointer.
 
 Before the pointer is used, it needs to be authenticated, i.e., have its
 signature checked.  This prevents pointer values of unknown origin from being
 used to replace the signed pointer value.
 
-At the IR level, it is represented using a [set of intrinsics](#intrinsics)
-(to sign/authenticate pointers).
+At the IR level, it is represented using:
+
+* a [set of intrinsics](#intrinsics) (to sign/authenticate pointers)
+* a [call operand bundle](#operand-bundle) (to authenticate called pointers)
 
 The current implementation leverages the
 [Armv8.3-A PAuth/Pointer Authentication Code](#armv8-3-a-pauth-pointer-authentication-code)
 instructions in the [AArch64 backend](#aarch64-support).
 This support is used to implement the Darwin arm64e ABI, as well as the
 [PAuth ABI Extension to ELF](https://github.com/ARM-software/abi-aa/blob/main/pauthabielf64/pauthabielf64.rst).
 
 

 
 ##### Semantics:
 
 The '``llvm.ptrauth.blend``' intrinsic combines a small integer discriminator
 with a pointer address discriminator, in a way that is specified by the target
 implementation.
 
 
+### Operand Bundle
+
+Function pointers used as indirect call targets can be signed when materialized,
+and authenticated before calls.  This can be accomplished with the
+[``llvm.ptrauth.auth``](#llvm-ptrauth-auth) intrinsic, feeding its result to
+an indirect call.
+
+However, that exposes the intermediate, unauthenticated pointer, e.g., if it
+gets spilled to the stack.  An attacker can then overwrite the pointer in
+memory, negating the security benefit provided by pointer authentication.
+To prevent that, the ``ptrauth`` operand bundle may be used: it guarantees that
+the intermediate call target is kept in a register and never stored to memory.
+This hardening benefit is similar to that provided by
+[``llvm.ptrauth.resign``](#llvm-ptrauth-resign)).
+
+Concretely:
+
+```llvm
+define void @f(void ()* %fp) {
+  call void %fp() [ "ptrauth"(i32 <key>, i64 <data>) ]
+  ret void
+}
+```
+
+is functionally equivalent to:
+
+```llvm
+define void @f(void ()* %fp) {
+  %fp_i = ptrtoint void ()* %fp to i64
+  %fp_auth = call i64 @llvm.ptrauth.auth(i64 %fp_i, i32 <key>, i64 <data>)
+  %fp_auth_p = inttoptr i64 %fp_auth to void ()*
+  call void %fp_auth_p()
+  ret void
+}
+```
+
+but with the added guarantee that ``%fp_i``, ``%fp_auth``, and ``%fp_auth_p``
+are not stored to (and reloaded from) memory.
+
+
 ## AArch64 Support
 
 AArch64 is currently the only architecture with full support of the pointer
 authentication primitives, based on Armv8.3-A instructions.
 
 ### Armv8.3-A PAuth Pointer Authentication Code
 
 The Armv8.3-A architecture extension defines the PAuth feature, which provides

llvm/include/llvm/IR/InstrTypes.h

public:
   /// may read from the heap.
   bool hasReadingOperandBundles() const;
 
   /// Return true if this operand bundle user has operand bundles that
   /// may write to the heap.
   bool hasClobberingOperandBundles() const {
     for (auto &BOI : bundle_op_infos()) {
       if (BOI.Tag->second == LLVMContext::OB_deopt ||
-          BOI.Tag->second == LLVMContext::OB_funclet)
+          BOI.Tag->second == LLVMContext::OB_funclet ||
+          BOI.Tag->second == LLVMContext::OB_ptrauth)
         continue;
 
       // This instruction has an operand bundle that is not known to us.
       // Assume the worst.
       return true;
     }
 
     return false;

llvm/include/llvm/IR/LLVMContext.h

#undef LLVM_FIXED_MD_KIND
   enum : unsigned {
     OB_deopt = 0,                  // "deopt"
     OB_funclet = 1,                // "funclet"
     OB_gc_transition = 2,          // "gc-transition"
     OB_cfguardtarget = 3,          // "cfguardtarget"
     OB_preallocated = 4,           // "preallocated"
     OB_gc_live = 5,                // "gc-live"
     OB_clang_arc_attachedcall = 6, // "clang.arc.attachedcall"
+    OB_ptrauth = 7,                // "ptrauth"
   };
 
   /// getMDKindID - Return a unique non-zero ID for the specified metadata kind.
   /// This ID is uniqued across modules in the current LLVMContext.
   unsigned getMDKindID(StringRef Name) const;
 
   /// getMDKindNames - Populate client supplied SmallVector with the name for
   /// custom metadata IDs registered in this LLVMContext.

llvm/lib/IR/Instructions.cpp

for (unsigned I = 0, E = CB->getNumOperandBundles(); I != E; ++I) {
     Bundles.emplace_back(Bundle);
   }
 
   return CreateNew ? Create(CB, Bundles, InsertPt) : CB;
 }
 
 bool CallBase::hasReadingOperandBundles() const {
   // Implementation note: this is a conservative implementation of operand
-  // bundle semantics, where *any* non-assume operand bundle forces a callsite
-  // to be at least readonly.
-  return hasOperandBundles() && getIntrinsicID() != Intrinsic::assume;
+  // bundle semantics, where *any* non-assume operand bundle (other than
+  // ptrauth) forces a callsite to be at least readonly.
+  return hasOperandBundlesOtherThan(LLVMContext::OB_ptrauth) &&
+         getIntrinsicID() != Intrinsic::assume;
 }
 
 //===----------------------------------------------------------------------===//
 //                        CallInst Implementation
 //===----------------------------------------------------------------------===//
 
 void CallInst::init(FunctionType *FTy, Value *Func, ArrayRef<Value *> Args,
                     ArrayRef<OperandBundleDef> Bundles, const Twine &NameStr) {

llvm/lib/IR/LLVMContext.cpp

#undef LLVM_FIXED_MD_KIND
   (void)GCLiveEntry;
 
   auto *ClangAttachedCall =
       pImpl->getOrInsertBundleTag("clang.arc.attachedcall");
   assert(ClangAttachedCall->second == LLVMContext::OB_clang_arc_attachedcall &&
          "clang.arc.attachedcall operand bundle id drifted!");
   (void)ClangAttachedCall;
 
+  auto *PtrauthEntry = pImpl->getOrInsertBundleTag("ptrauth");
+  assert(PtrauthEntry->second == LLVMContext::OB_ptrauth &&
+         "ptrauth operand bundle id drifted!");
+  (void)PtrauthEntry;
+
   SyncScope::ID SingleThreadSSID =
       pImpl->getOrInsertSyncScopeID("singlethread");
   assert(SingleThreadSSID == SyncScope::SingleThread &&
          "singlethread synchronization scope ID drifted!");
   (void)SingleThreadSSID;
 
   SyncScope::ID SystemSSID =
       pImpl->getOrInsertSyncScopeID("");

llvm/lib/IR/Verifier.cpp

Assert(!FTy->getReturnType()->isX86_AMXTy(),
            "Return type cannot be x86_amx for indirect call!");
   }
 
   if (Function *F = Call.getCalledFunction())
     if (Intrinsic::ID ID = (Intrinsic::ID)F->getIntrinsicID())
       visitIntrinsicCall(ID, Call);
 
   // Verify that a callsite has at most one "deopt", at most one "funclet", at
-  // most one "gc-transition", at most one "cfguardtarget",
-  // and at most one "preallocated" operand bundle.
+  // most one "gc-transition", at most one "cfguardtarget", at most one
+  // "preallocated" operand bundle, and at most one "ptrauth" operand bundle.
   bool FoundDeoptBundle = false, FoundFuncletBundle = false,
        FoundGCTransitionBundle = false, FoundCFGuardTargetBundle = false,
        FoundPreallocatedBundle = false, FoundGCLiveBundle = false,
+       FoundPtrauthBundle = false,
        FoundAttachedCallBundle = false;
   for (unsigned i = 0, e = Call.getNumOperandBundles(); i < e; ++i) {
     OperandBundleUse BU = Call.getOperandBundleAt(i);
     uint32_t Tag = BU.getTagID();
     if (Tag == LLVMContext::OB_deopt) {
       Assert(!FoundDeoptBundle, "Multiple deopt operand bundles", Call);
       FoundDeoptBundle = true;
     } else if (Tag == LLVMContext::OB_gc_transition) {
if (Tag == LLVMContext::OB_deopt) {
              "Funclet bundle operands should correspond to a FuncletPadInst",
              Call);
     } else if (Tag == LLVMContext::OB_cfguardtarget) {
       Assert(!FoundCFGuardTargetBundle,
              "Multiple CFGuardTarget operand bundles", Call);
       FoundCFGuardTargetBundle = true;
       Assert(BU.Inputs.size() == 1,
              "Expected exactly one cfguardtarget bundle operand", Call);
+    } else if (Tag == LLVMContext::OB_ptrauth) {
+      Assert(!FoundPtrauthBundle, "Multiple ptrauth operand bundles", Call);
+      FoundPtrauthBundle = true;
+      Assert(BU.Inputs.size() == 2,
+             "Expected exactly two ptrauth bundle operands", Call);
+      Assert(isa<ConstantInt>(BU.Inputs[0]) &&
+             BU.Inputs[0]->getType()->isIntegerTy(32),
+             "Ptrauth bundle key operand must be an i32 constant", Call);
+      Assert(BU.Inputs[1]->getType()->isIntegerTy(64),
+             "Ptrauth bundle discriminator operand must be an i64", Call);
     } else if (Tag == LLVMContext::OB_preallocated) {
       Assert(!FoundPreallocatedBundle, "Multiple preallocated operand bundles",
              Call);
       FoundPreallocatedBundle = true;
       Assert(BU.Inputs.size() == 1,
              "Expected exactly one preallocated bundle operand", Call);
       auto Input = dyn_cast<IntrinsicInst>(BU.Inputs.front());
       Assert(Input &&
                  Input->getIntrinsicID() == Intrinsic::call_preallocated_setup,
              "\"preallocated\" argument must be a token from "
              "llvm.call.preallocated.setup",
              Call);
     } else if (Tag == LLVMContext::OB_gc_live) {
       Assert(!FoundGCLiveBundle, "Multiple gc-live operand bundles",
              Call);
       FoundGCLiveBundle = true;
     } else if (Tag == LLVMContext::OB_clang_arc_attachedcall) {
       Assert(!FoundAttachedCallBundle,
              "Multiple \"clang.arc.attachedcall\" operand bundles", Call);
       FoundAttachedCallBundle = true;
       verifyAttachedCallBundle(Call, BU);
     }
   }
 
+  // Verify that callee and callsite agree on whether to use pointer auth.
+  Assert(!(Call.getCalledFunction() && FoundPtrauthBundle),
+         "Direct call cannot have a ptrauth bundle", Call);
+
   // Verify that each inlinable callsite of a debug-info-bearing function in a
   // debug-info-bearing function has a debug location attached to it. Failure to
   // do so causes assertion failures when the inliner sets up inline scope info.
   if (Call.getFunction()->getSubprogram() && Call.getCalledFunction() &&
       Call.getCalledFunction()->getSubprogram())
     AssertDI(Call.getDebugLoc(),
              "inlinable function call in a function with "
              "debug info must have a !dbg location",

llvm/lib/Transforms/Scalar/TailRecursionElimination.cpp

for (auto &I : *BB) {
       CallInst *CI = dyn_cast<CallInst>(&I);
       // A PseudoProbeInst has the IntrInaccessibleMemOnly tag hence it is
       // considered accessing memory and will be marked as a tail call if we
       // don't bail out here.
       if (!CI || CI->isTailCall() || isa<DbgInfoIntrinsic>(&I) ||
           isa<PseudoProbeInst>(&I))
         continue;
 
-      // Special-case operand bundle "clang.arc.attachedcall".
+      // Special-case operand bundles "clang.arc.attachedcall" and "ptrauth".
       bool IsNoTail =
           CI->isNoTailCall() || CI->hasOperandBundlesOtherThan(
-                                    LLVMContext::OB_clang_arc_attachedcall);
+            {LLVMContext::OB_clang_arc_attachedcall, LLVMContext::OB_ptrauth});
 
       if (!IsNoTail && CI->doesNotAccessMemory()) {
         // A call to a readnone function whose arguments are all things computed
         // outside this function can be marked tail. Even if you stored the
         // alloca address into a global, a readnone function can't load the
         // global anyhow.
         //
         // Note that this runs whether we know an alloca has escaped or not. If

llvm/test/Bitcode/operand-bundles-bc-analyzer.ll

 ; RUN: llvm-as < %s | llvm-bcanalyzer -dump -disable-histogram | FileCheck %s
 
 ; CHECK:  <OPERAND_BUNDLE_TAGS_BLOCK
 ; CHECK-NEXT:    <OPERAND_BUNDLE_TAG
 ; CHECK-NEXT:    <OPERAND_BUNDLE_TAG
 ; CHECK-NEXT:    <OPERAND_BUNDLE_TAG
 ; CHECK-NEXT:    <OPERAND_BUNDLE_TAG
 ; CHECK-NEXT:    <OPERAND_BUNDLE_TAG
 ; CHECK-NEXT:    <OPERAND_BUNDLE_TAG
 ; CHECK-NEXT:    <OPERAND_BUNDLE_TAG
 ; CHECK-NEXT:    <OPERAND_BUNDLE_TAG
 ; CHECK-NEXT:    <OPERAND_BUNDLE_TAG
+; CHECK-NEXT:    <OPERAND_BUNDLE_TAG
 ; CHECK-NEXT:  </OPERAND_BUNDLE_TAGS_BLOCK
 
 ; CHECK:   <FUNCTION_BLOCK
 ; CHECK:    <OPERAND_BUNDLE
 ; CHECK:    <OPERAND_BUNDLE
 ; CHECK-NOT: <OPERAND_BUNDLE
 ; CHECK:  </FUNCTION_BLOCK
 

llvm/test/Transforms/TailCallElim/ptrauth-bundle.ll

+; RUN: opt < %s -tailcallelim -verify-dom-info -S | FileCheck %s
+; Check that the "ptrauth" operand bundle doesn't prevent tail calls.
+
+define i64 @f_1(i64 %x, i64(i64)* %f_0) {
+; CHECK-LABEL: @f_1(
+entry:
+; CHECK: tail call i64 %f_0(i64 %x) [ "ptrauth"(i32 42, i64 %x) ]
+  %tmp = call i64 %f_0(i64 %x) [ "ptrauth"(i32 42, i64 %x) ]
+  ret i64 0
+}

llvm/test/Verifier/ptrauth-operand-bundles.ll

+; RUN: not opt -verify < %s 2>&1 | FileCheck %s
+
+declare void @g()
+
+define void @test_ptrauth_bundle(i64 %arg0, i32 %arg1, void()* %arg2) {
+
+; CHECK: Multiple ptrauth operand bundles
+; CHECK-NEXT: call void %arg2() [ "ptrauth"(i32 42, i64 100), "ptrauth"(i32 42, i64 %arg0) ]
+  call void %arg2() [ "ptrauth"(i32 42, i64 100), "ptrauth"(i32 42, i64 %arg0) ]
+
+; CHECK: Ptrauth bundle key operand must be an i32 constant
+; CHECK-NEXT: call void %arg2() [ "ptrauth"(i32 %arg1, i64 120) ]
+  call void %arg2() [ "ptrauth"(i32 %arg1, i64 120) ]
+
+; CHECK: Ptrauth bundle key operand must be an i32 constant
+; CHECK-NEXT: call void %arg2() [ "ptrauth"(i64 42, i64 120) ]
+  call void %arg2() [ "ptrauth"(i64 42, i64 120) ]
+
+; CHECK: Ptrauth bundle discriminator operand must be an i64
+; CHECK-NEXT: call void %arg2() [ "ptrauth"(i32 42, i32 120) ]
+  call void %arg2() [ "ptrauth"(i32 42, i32 120) ]
+
+; CHECK: Direct call cannot have a ptrauth bundle
+; CHECK-NEXT: call void @g() [ "ptrauth"(i32 42, i64 120) ]
+  call void @g() [ "ptrauth"(i32 42, i64 120) ]
+
+; CHECK-NOT: call
+  call void %arg2() [ "ptrauth"(i32 42, i64 120) ]   ; OK
+  call void %arg2() [ "ptrauth"(i32 42, i64 %arg0) ] ; OK
+  ret void
+}