Allow invokable sub-classes of IntrinsicInst

[reames]

Summary

It used to be that all of our intrinsics were call instructions, but over time, we've added more and more invokable intrinsics. According to the verifier, we're up to 8 right now. As IntrinsicInst is a sub-class of CallInst, this puts us in an awkward spot where the idiomatic means to check for intrinsic has a false negative if the intrinsic is invoked.

This change switches IntrinsicInst from being a sub-class of CallInst to being a subclass of CallBase. This allows invoked intrinsics to be instances of IntrinsicInst, at the cost of requiring a few more casts to CallInst in places where the intrinsic really is known to be a call, not an invoke.

This is admittedly a debatable stylistic issue. To me, allowing invokeable intrinsics seems cleaner, but I also spent a lot of my time dealing with those intrinsics. Do others agree?

At the moment, this patch only builds on x86. I'm sure there will be analogous changes in the other targets. I'll do them if folks are okay with the overall direction.

After this lands (if it does), planned cleanups:

• Make GCStatepointInst a IntrinsicInst subclass.
• Merge intrinsic handling in InstCombine and use idiomatic visitIntrinsicInst entry point for InstVisitor.
• Do the same in SelectionDAG.
• Do the same in FastISEL.

[reames]

Rebase over cleanups - realized several of the things I was going to do to reduce diffs were generally useful code improvements so just went ahead and did them.

[reames]

ping

[nikic]

LGTM. The number of necessary casts seems to be pretty small, so I think this is a reasonable tradeoff.

llvm/include/llvm/IR/InstVisitor.h

313

non-CallBase

[echristo]

I guess my question is "why? what does this make easier/harder/etc?"

I see you added a bunch of people and then committed, but I'd still like to hear the answers.

[reames]

In D99976#2703175, @echristo wrote:

I guess my question is "why? what does this make easier/harder/etc?"

A couple of pieces here.

1. We no longer have the chance of a nasty false negative bug. Before, we could fail to handle some intrinsics because they would not match IntrinsicInst.
2. As listed in the potential cleanups in the commit, we have multiple places in code which handled intrinsics generically, but then had to add extra casing for the invokable ones. We'll be able to clean that up.
3. For intrinsics which really are calls - which, admittedly, is many - this does complicate the pattern matching as you have to both match the intrinsicinst, and then cast to CallInst for interfaces which expect calls. (A quick survey shows many of those interfaces could reasonably take CallBase, but not all.) We could explore having two base classes - one for all intrinsics and one for call only intrinsics - not quite sure if the multiple inheritance works out there. It didn't seem worthwhile to justify exploration to me, if you disagree, let me know.

I see you added a bunch of people and then committed, but I'd still like to hear the answers.

Er, did you intentionally leave out the "waited for LGTM" part? Your wording seems to imply bad faith which I don't think is called for.

If you think this is rushed and needs further discussion, I'm happy to revert while that discussion happens.

[reames]

(For anyone following along, discussion has moved to email on review thread which phabricator isn't catching.)

[aeubanks]

This slightly regressed compile times while it was in
http://llvm-compile-time-tracker.com/compare.php?from=080d48f279e2b70e4e9fc65bc4947a53ffc982f3&to=d87b9b81ccb95217181ce75515c6c68bbb408ca4&stat=instructions

[nikic]

In D99976#2703736, @aeubanks wrote:

This slightly regressed compile times while it was in
http://llvm-compile-time-tracker.com/compare.php?from=080d48f279e2b70e4e9fc65bc4947a53ffc982f3&to=d87b9b81ccb95217181ce75515c6c68bbb408ca4&stat=instructions

Interesting, I didn't expect that. Presumably this is because most places working with IntrinsicInst now have to perform two branches rather than one, and apparently it gets used a lot. Most likely this overestimates actual impact (assuming predicted branches), but still, better to avoid it if we can.

Could it make sense to have both IntrinsicBase and IntrinsicInst, similar to how we have CallBase and CallInst?

[reames]

Tweak the initial approach to optimize the non-intrinsic dispatch and involve fewer checked casts. I'm hoping this reduces the compile time impact.

One behavior change in this patch: unknown intrinsics (e.g. "llvm.unrecognized_name") now get passed to visitIntrinsic whereas there didn't previously. I think this is probably the right behavior anyways, and it simplifies the code a bit.

If this doesn't work, I'll look at optimize the intrinsic dispatch path, but that's a bit more involved.

[echristo]

In D99976#2706447, @nikic wrote:

In D99976#2703736, @aeubanks wrote:

This slightly regressed compile times while it was in
http://llvm-compile-time-tracker.com/compare.php?from=080d48f279e2b70e4e9fc65bc4947a53ffc982f3&to=d87b9b81ccb95217181ce75515c6c68bbb408ca4&stat=instructions

Interesting, I didn't expect that. Presumably this is because most places working with IntrinsicInst now have to perform two branches rather than one, and apparently it gets used a lot. Most likely this overestimates actual impact (assuming predicted branches), but still, better to avoid it if we can.

FWIW we also saw this in a build with predicted branches. :)

Could it make sense to have both IntrinsicBase and IntrinsicInst, similar to how we have CallBase and CallInst?

Perhaps? I think some of the cases you've got like CallInst *findIntrinsicBlah(...) could return CallBase and that might make sense since an intrinsic "found" could be invokeable. Perhaps name it something neither Call or Invoke to encompass that it can be used for both? (Maybe Callable? I don't have any good ideas and the thesaurus is full of hilarious, but unlikely to be useful synonyms :)

Thoughts?

[reames]

Closing review I'm not currently working on, and am unlikely to get back to in near future. Will reopen if priorities change.

llvm/include/llvm/Analysis/AssumeBundleQueries.h

 //===- AssumeBundleQueries.h - utilities to query assume bundles *- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file contain tools to query into assume bundles. assume bundles can be
 // built using utilities from Transform/Utils/AssumeBundleBuilder.h
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_ANALYSIS_ASSUMEBUNDLEQUERIES_H
 #define LLVM_ANALYSIS_ASSUMEBUNDLEQUERIES_H
 
 #include "llvm/IR/Attributes.h"
 #include "llvm/IR/Instructions.h"
+#include "llvm/IR/IntrinsicInst.h"
 #include "llvm/ADT/DenseMap.h"
 
 namespace llvm {
 class IntrinsicInst;
 class AssumptionCache;
 class DominatorTree;
 
 /// Index of elements in the operand bundle.
 /// If the element exist it is guaranteed to be what is specified in this enum

 bool hasAttributeInAssume(CallInst &AssumeCI, Value *IsOn, StringRef AttrName,
                           uint64_t *ArgVal = nullptr);
 inline bool hasAttributeInAssume(CallInst &AssumeCI, Value *IsOn,
                                  Attribute::AttrKind Kind,
                                  uint64_t *ArgVal = nullptr) {
   return hasAttributeInAssume(AssumeCI, IsOn,
                               Attribute::getNameFromAttrKind(Kind), ArgVal);
 }
+inline bool hasAttributeInAssume(IntrinsicInst &AssumeCI, Value *IsOn, StringRef AttrName,
+                          uint64_t *ArgVal = nullptr) {
+  return hasAttributeInAssume(cast<CallInst>(AssumeCI), IsOn, AttrName, ArgVal);
+}
+  
+inline bool hasAttributeInAssume(IntrinsicInst &AssumeCI, Value *IsOn,
+                                 Attribute::AttrKind Kind,
+                                 uint64_t *ArgVal = nullptr) {
+  return hasAttributeInAssume(cast<CallInst>(AssumeCI), IsOn, Kind,
+                              ArgVal);
+}
+
 
 template<> struct DenseMapInfo<Attribute::AttrKind> {
   static Attribute::AttrKind getEmptyKey() {
     return Attribute::EmptyKey;
   }
   static Attribute::AttrKind getTombstoneKey() {
     return Attribute::TombstoneKey;
   }
   static unsigned getHashValue(Attribute::AttrKind AK) {
using RetainedKnowledgeMap =
     DenseMap<RetainedKnowledgeKey, Assume2KnowledgeMap>;
 
 /// Insert into the map all the informations contained in the operand bundles of
 /// the llvm.assume. This should be used instead of hasAttributeInAssume when
 /// many queries are going to be made on the same llvm.assume.
 /// String attributes are not inserted in the map.
 /// If the IR changes the map will be outdated.
 void fillMapFromAssume(CallInst &AssumeCI, RetainedKnowledgeMap &Result);
+inline void fillMapFromAssume(IntrinsicInst &AssumeCI, RetainedKnowledgeMap &Result) {
+  return fillMapFromAssume(cast<CallInst>(AssumeCI), Result);
+}
 
 /// Represent one information held inside an operand bundle of an llvm.assume.
 /// AttrKind is the property that holds.
 /// WasOn if not null is that Value for which AttrKind holds.
 /// ArgValue is optionally an argument of the attribute.
 /// For example if we know that %P has an alignment of at least four:
 ///  - AttrKind will be Attribute::Alignment.
 ///  - WasOn will be %P.

 /// contain any valuable information. This is true when:
 ///  - The operand bundle is empty
 ///  - The operand bundle only contains information about dropped values or
 ///    constant folded values.
 ///
 /// the argument to the call of llvm.assume may still be useful even if the
 /// function returned true.
 bool isAssumeWithEmptyBundle(CallInst &Assume);
+  inline bool isAssumeWithEmptyBundle(IntrinsicInst &Assume) {
+    return isAssumeWithEmptyBundle(cast<CallInst>(Assume));
+  }
 
 /// Return a valid Knowledge associated to the Use U if its Attribute kind is
 /// in AttrKinds.
 RetainedKnowledge getKnowledgeFromUse(const Use *U,
                                       ArrayRef<Attribute::AttrKind> AttrKinds);
 
 /// Return a valid Knowledge associated to the Value V if its Attribute kind is
 /// in AttrKinds and it matches the Filter.
RetainedKnowledge getKnowledgeValidInContext(
     const Value *V, ArrayRef<Attribute::AttrKind> AttrKinds,
     const Instruction *CtxI, const DominatorTree *DT = nullptr,
     AssumptionCache *AC = nullptr);
 
 /// This extracts the Knowledge from an element of an operand bundle.
 /// This is mostly for use in the assume builder.
 RetainedKnowledge getKnowledgeFromBundle(CallInst &Assume,
                                          const CallBase::BundleOpInfo &BOI);
+inline RetainedKnowledge getKnowledgeFromBundle(IntrinsicInst &Assume,
+                                                const CallBase::BundleOpInfo &BOI) {
+  return getKnowledgeFromBundle(cast<CallInst>(Assume), BOI);
+}
+
 
 } // namespace llvm
 
 #endif

llvm/include/llvm/Analysis/AssumptionCache.h

 
 #ifndef LLVM_ANALYSIS_ASSUMPTIONCACHE_H
 #define LLVM_ANALYSIS_ASSUMPTIONCACHE_H
 
 #include "llvm/ADT/ArrayRef.h"
 #include "llvm/ADT/DenseMap.h"
 #include "llvm/ADT/DenseMapInfo.h"
 #include "llvm/ADT/SmallVector.h"
+#include "llvm/IR/IntrinsicInst.h"
 #include "llvm/IR/PassManager.h"
 #include "llvm/IR/ValueHandle.h"
 #include "llvm/Pass.h"
 #include <memory>
 
 namespace llvm {
 
 class CallInst;
bool invalidate(Function &, const PreservedAnalyses &,
     return false;
   }
 
   /// Add an \@llvm.assume intrinsic to this function's cache.
   ///
   /// The call passed in must be an instruction within this function and must
   /// not already be in the cache.
   void registerAssumption(CallInst *CI);
+  void registerAssumption(IntrinsicInst *II) {
+    registerAssumption(cast<CallInst>(II));
+  }
 
   /// Remove an \@llvm.assume intrinsic from this function's cache if it has
   /// been added to the cache earlier.
   void unregisterAssumption(CallInst *CI);
+  void unregisterAssumption(IntrinsicInst *II) {
+    unregisterAssumption(cast<CallInst>(II));
+  }
 
   /// Update the cache of values being affected by this assumption (i.e.
   /// the values about which this assumption provides information).
   void updateAffectedValues(CallInst *CI);
+  void updateAffectedValues(IntrinsicInst *II) {
+    updateAffectedValues(cast<CallInst>(II));
+  }
+
 
   /// Clear the cache of \@llvm.assume intrinsics for a function.
   ///
   /// It will be re-scanned the next time it is requested.
   void clear() {
     AssumeHandles.clear();
     AffectedValues.clear();
     Scanned = false;
   }

llvm/include/llvm/IR/InstVisitor.h

#include "llvm/IR/Instruction.def"
   // handle SPECIFIC instructions.  These functions automatically define
   // visitMul to proxy to visitBinaryOperator for instance in case the user does
   // not need this generality.
   //
   // These functions can also implement fan-out, when a single opcode and
   // instruction have multiple more specific Instruction subclasses. The Call
   // instruction currently supports this. We implement that by redirecting that
   // instruction to a special delegation helper.
 #define HANDLE_INST(NUM, OPCODE, CLASS) \
     RetTy visit##OPCODE(CLASS &I) { \
-      if (NUM == Instruction::Call) \
-        return delegateCallInst(I); \
+      if (NUM == Instruction::Call || NUM == Instruction::Invoke) \
+        return delegateCallBase(I); \
       else \
         DELEGATE(CLASS); \
     }
 #include "llvm/IR/Instruction.def"
 
   // Specific Instruction type classes... note that all of the casts are
   // necessary because we use the instruction classes as opaque types...
   //
#include "llvm/IR/Instruction.def"
   RetTy visitMemSetInst(MemSetInst &I)            { DELEGATE(MemIntrinsic); }
   RetTy visitMemCpyInst(MemCpyInst &I)            { DELEGATE(MemTransferInst); }
   RetTy visitMemMoveInst(MemMoveInst &I)          { DELEGATE(MemTransferInst); }
   RetTy visitMemTransferInst(MemTransferInst &I)  { DELEGATE(MemIntrinsic); }
   RetTy visitMemIntrinsic(MemIntrinsic &I)        { DELEGATE(IntrinsicInst); }
   RetTy visitVAStartInst(VAStartInst &I)          { DELEGATE(IntrinsicInst); }
   RetTy visitVAEndInst(VAEndInst &I)              { DELEGATE(IntrinsicInst); }
   RetTy visitVACopyInst(VACopyInst &I)            { DELEGATE(IntrinsicInst); }
-  RetTy visitIntrinsicInst(IntrinsicInst &I)      { DELEGATE(CallInst); }
+  RetTy visitIntrinsicInst(IntrinsicInst &I)      {
+    if (isa<CallInst>(I))
+      return static_cast<SubClass*>(this)->visitCallInst(cast<CallInst>(I));
+    else
+      return static_cast<SubClass*>(this)->visitInvokeInst(cast<InvokeInst>(I));
+  }
   RetTy visitCallInst(CallInst &I)                { DELEGATE(CallBase); }
   RetTy visitInvokeInst(InvokeInst &I)            { DELEGATE(CallBase); }
   RetTy visitCallBrInst(CallBrInst &I)            { DELEGATE(CallBase); }
 
   // While terminators don't have a distinct type modeling them, we support
   // intercepting them with dedicated a visitor callback.
   RetTy visitReturnInst(ReturnInst &I) {
     return static_cast<SubClass *>(this)->visitTerminator(I);
#include "llvm/IR/Instruction.def"
   // this instruction just gets ignored.
   //
   // Note that you MUST override this function if your return type is not void.
   //
   void visitInstruction(Instruction &I) {}  // Ignore unhandled instructions
 
 private:
   // Special helper function to delegate to CallInst subclass visitors.
-  RetTy delegateCallInst(CallInst &I) {
+  RetTy delegateCallBase(CallBase &I) {
     if (const Function *F = I.getCalledFunction()) {
       switch (F->getIntrinsicID()) {
       default:                     DELEGATE(IntrinsicInst);
       case Intrinsic::dbg_declare: DELEGATE(DbgDeclareInst);
       case Intrinsic::dbg_value:   DELEGATE(DbgValueInst);
       case Intrinsic::dbg_label:   DELEGATE(DbgLabelInst);
       case Intrinsic::memcpy:      DELEGATE(MemCpyInst);
       case Intrinsic::memmove:     DELEGATE(MemMoveInst);
       case Intrinsic::memset:      DELEGATE(MemSetInst);
       case Intrinsic::vastart:     DELEGATE(VAStartInst);
       case Intrinsic::vaend:       DELEGATE(VAEndInst);
       case Intrinsic::vacopy:      DELEGATE(VACopyInst);
       case Intrinsic::not_intrinsic: break;
       }
     }
-    DELEGATE(CallInst);
+    if (isa<CallInst>(I))
+      return static_cast<SubClass*>(this)->visitCallInst(cast<CallInst>(I));
+    else
+      return static_cast<SubClass*>(this)->visitInvokeInst(cast<InvokeInst>(I));
   }
 
   // An overload that will never actually be called, it is used only from dead
   // code in the dispatching from opcodes to instruction subclasses.
-  RetTy delegateCallInst(Instruction &I) {
-    llvm_unreachable("delegateCallInst called for non-CallInst");
+  RetTy delegateCallBase(Instruction &I) {
+    llvm_unreachable("delegateCallBase called for non-CallInst");
   }
 };
 
 #undef DELEGATE
 
 } // End llvm namespace
 
 #endif

llvm/include/llvm/IR/IntrinsicInst.h

 //===-- llvm/IntrinsicInst.h - Intrinsic Instruction Wrappers ---*- C++ -*-===//
 //
 // Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 // See https://llvm.org/LICENSE.txt for license information.
 // SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 //
 //===----------------------------------------------------------------------===//
 //
 // This file defines classes that make it really easy to deal with intrinsic
 // functions with the isa/dyncast family of functions.  In particular, this
 // allows you to do things like:
 //
 //     if (MemCpyInst *MCI = dyn_cast<MemCpyInst>(Inst))
 //        ... MCI->getDest() ... MCI->getSource() ...
 //
-// All intrinsic function calls are instances of the call instruction, so these
-// are all subclasses of the CallInst class.  Note that none of these classes
-// has state or virtual methods, which is an important part of this gross/neat
-// hack working.
+// All intrinsic function calls are instances of the call or invoke instruction,
+// so these are all subclasses of the CallBase class.  Note that none of these
+// classes has state or virtual methods, which is an important part of this
+// gross/neat hack working.
 //
 //===----------------------------------------------------------------------===//
 
 #ifndef LLVM_IR_INTRINSICINST_H
 #define LLVM_IR_INTRINSICINST_H
 
 #include "llvm/IR/Constants.h"
 #include "llvm/IR/DebugInfoMetadata.h"

 #include <cassert>
 #include <cstdint>
 
 namespace llvm {
 
 /// A wrapper class for inspecting calls to intrinsic functions.
 /// This allows the standard isa/dyncast/cast functionality to work with calls
 /// to intrinsic functions.
-class IntrinsicInst : public CallInst {
+class IntrinsicInst : public CallBase {
 public:
   IntrinsicInst() = delete;
   IntrinsicInst(const IntrinsicInst &) = delete;
   IntrinsicInst &operator=(const IntrinsicInst &) = delete;
 
   /// Return the intrinsic ID of this intrinsic.
   Intrinsic::ID getIntrinsicID() const {
     return getCalledFunction()->getIntrinsicID();
bool isAssumeLikeIntrinsic() const {
     case Intrinsic::ptr_annotation:
     case Intrinsic::var_annotation:
       return true;
     }
     return false;
   }
 
   // Methods for support type inquiry through isa, cast, and dyn_cast:
-  static bool classof(const CallInst *I) {
+  static bool classof(const CallBase *I) {
     if (const Function *CF = I->getCalledFunction())
       return CF->isIntrinsic();
     return false;
   }
   static bool classof(const Value *V) {
-    return isa<CallInst>(V) && classof(cast<CallInst>(V));
+    return isa<CallBase>(V) && classof(cast<CallBase>(V));
   }
 };
 
 /// Check if \p ID corresponds to a debug info intrinsic.
 static inline bool isDbgInfoIntrinsic(Intrinsic::ID ID) {
   switch (ID) {
   case Intrinsic::dbg_declare:
   case Intrinsic::dbg_value:

llvm/include/llvm/IR/Statepoint.h

enum class StatepointFlags {
   MaskAll = 3 ///< A bitmask that includes all valid flags.
 };
 
 // These two are defined in IntrinsicInst since they're part of the
 // IntrinsicInst class hierarchy.
 class GCRelocateInst;
 class GCResultInst;
 
-/// Represents a gc.statepoint intrinsic call.  This extends directly from
-/// CallBase as the IntrinsicInst only supports calls and gc.statepoint is
-/// invokable.
-class GCStatepointInst : public CallBase {
+/// Represents a gc.statepoint intrinsic call.
+class GCStatepointInst : public IntrinsicInst {
 public:
   GCStatepointInst() = delete;
   GCStatepointInst(const GCStatepointInst &) = delete;
   GCStatepointInst &operator=(const GCStatepointInst &) = delete;
 
-  static bool classof(const CallBase *I) {
-    if (const Function *CF = I->getCalledFunction())
-      return CF->getIntrinsicID() == Intrinsic::experimental_gc_statepoint;
-    return false;
+  static bool classof(const IntrinsicInst *I) {
+    return I->getIntrinsicID() == Intrinsic::experimental_gc_statepoint;
   }
 
   static bool classof(const Value *V) {
-    return isa<CallBase>(V) && classof(cast<CallBase>(V));
+    return isa<IntrinsicInst>(V) && classof(cast<IntrinsicInst>(V));
   }
 
   enum {
     IDPos = 0,
     NumPatchBytesPos = 1,
     CalledFunctionPos = 2,
     NumCallArgsPos = 3,
     FlagsPos = 4,

llvm/lib/Analysis/AssumeBundleQueries.cpp

assert((ArgVal == nullptr || Attribute::doesAttrKindHaveArgument(
          "requested value for an attribute that has no argument");
   if (Assume.bundle_op_infos().empty())
     return false;
 
   for (auto &BOI : Assume.bundle_op_infos()) {
     if (BOI.Tag->getKey() != AttrName)
       continue;
     if (IsOn && (BOI.End - BOI.Begin <= ABA_WasOn ||
-                 IsOn != getValueFromBundleOpInfo(Assume, BOI, ABA_WasOn)))
+                 IsOn != getValueFromBundleOpInfo(AssumeCI, BOI, ABA_WasOn)))
       continue;
     if (ArgVal) {
       assert(BOI.End - BOI.Begin > ABA_Argument);
       *ArgVal =
-          cast<ConstantInt>(getValueFromBundleOpInfo(Assume, BOI, ABA_Argument))
+          cast<ConstantInt>(getValueFromBundleOpInfo(AssumeCI, BOI,
+                                                     ABA_Argument))
               ->getZExtValue();
     }
     return true;
   }
   return false;
 }
 
 void llvm::fillMapFromAssume(CallInst &AssumeCI, RetainedKnowledgeMap &Result) {
   IntrinsicInst &Assume = cast<IntrinsicInst>(AssumeCI);
   assert(Assume.getIntrinsicID() == Intrinsic::assume &&
          "this function is intended to be used on llvm.assume");
   for (auto &Bundles : Assume.bundle_op_infos()) {
     std::pair<Value *, Attribute::AttrKind> Key{
         nullptr, Attribute::getAttrKindFromName(Bundles.Tag->getKey())};
     if (bundleHasArgument(Bundles, ABA_WasOn))
-      Key.first = getValueFromBundleOpInfo(Assume, Bundles, ABA_WasOn);
+      Key.first = getValueFromBundleOpInfo(AssumeCI, Bundles, ABA_WasOn);
 
     if (Key.first == nullptr && Key.second == Attribute::None)
       continue;
     if (!bundleHasArgument(Bundles, ABA_Argument)) {
       Result[Key][&Assume] = {0, 0};
       continue;
     }
     auto *CI = dyn_cast<ConstantInt>(
-        getValueFromBundleOpInfo(Assume, Bundles, ABA_Argument));
+        getValueFromBundleOpInfo(AssumeCI, Bundles, ABA_Argument));
     if (!CI)
       continue;
     unsigned Val = CI->getZExtValue();
     auto Lookup = Result.find(Key);
     if (Lookup == Result.end() || !Lookup->second.count(&Assume)) {
       Result[Key][&Assume] = {Val, Val};
       continue;
     }
llvm::getKnowledgeForValue(const Value *V,
   if (!DebugCounter::shouldExecute(AssumeQueryCounter))
     return RetainedKnowledge::none();
   if (AC) {
     for (AssumptionCache::ResultElem &Elem : AC->assumptionsFor(V)) {
       IntrinsicInst *II = cast_or_null<IntrinsicInst>(Elem.Assume);
       if (!II || Elem.Index == AssumptionCache::ExprResultIdx)
         continue;
       if (RetainedKnowledge RK = getKnowledgeFromBundle(
-              *II, II->bundle_op_info_begin()[Elem.Index])) {
+            cast<CallInst>(*II), II->bundle_op_info_begin()[Elem.Index])) {
         if (V != RK.WasOn)
           continue;
         if (is_contained(AttrKinds, RK.AttrKind) &&
             Filter(RK, II, &II->bundle_op_info_begin()[Elem.Index])) {
           NumUsefullAssumeQueries++;
           return RK;
         }
       }

llvm/lib/CodeGen/SelectionDAG/FastISel.cpp

bool FastISel::selectIntrinsicCall(const IntrinsicInst *II) {
   case Intrinsic::expect: {
     Register ResultReg = getRegForValue(II->getArgOperand(0));
     if (!ResultReg)
       return false;
     updateValueMap(II, ResultReg);
     return true;
   }
   case Intrinsic::experimental_stackmap:
-    return selectStackmap(II);
+    return selectStackmap(cast<CallInst>(II));
   case Intrinsic::experimental_patchpoint_void:
   case Intrinsic::experimental_patchpoint_i64:
-    return selectPatchpoint(II);
+    return selectPatchpoint(cast<CallInst>(II));
 
   case Intrinsic::xray_customevent:
-    return selectXRayCustomEvent(II);
+    return selectXRayCustomEvent(cast<CallInst>(II));
   case Intrinsic::xray_typedevent:
-    return selectXRayTypedEvent(II);
+    return selectXRayTypedEvent(cast<CallInst>(II));
   }
 
   return fastLowerIntrinsicCall(II);
 }
 
 bool FastISel::selectCast(const User *I, unsigned Opcode) {
   EVT SrcVT = TLI.getValueType(DL, I->getOperand(0)->getType());
   EVT DstVT = TLI.getValueType(DL, I->getType());

llvm/lib/CodeGen/SelectionDAG/StatepointLowering.cpp

SDValue SelectionDAGBuilder::LowerAsSTATEPOINT(
   // Clear state
   StatepointLowering.startNewStatepoint(*this);
   assert(SI.Bases.size() == SI.Ptrs.size());
 
   LLVM_DEBUG(dbgs() << "Lowering statepoint " << *SI.StatepointInstr << "\n");
 #ifndef NDEBUG
   for (auto *Reloc : SI.GCRelocates)
     if (Reloc->getParent() == SI.StatepointInstr->getParent())
-      StatepointLowering.scheduleRelocCall(*Reloc);
+      StatepointLowering.scheduleRelocCall(cast<CallInst>(*Reloc));
 #endif
 
   // Lower statepoint vmstate and gcstate arguments
 
   // All lowered meta args.
   SmallVector<SDValue, 10> LoweredMetaArgs;
   // Lowered GC pointers (subset of above).
   SmallVector<SDValue, 16> LoweredGCArgs;

 
 void SelectionDAGBuilder::visitGCRelocate(const GCRelocateInst &Relocate) {
 #ifndef NDEBUG
   // Consistency check
   // We skip this check for relocates not in the same basic block as their
   // statepoint. It would be too expensive to preserve validation info through
   // different basic blocks.
   if (Relocate.getStatepoint()->getParent() == Relocate.getParent())
-    StatepointLowering.relocCallVisited(Relocate);
+    StatepointLowering.relocCallVisited(cast<CallInst>(Relocate));
 
   auto *Ty = Relocate.getType()->getScalarType();
   if (auto IsManaged = GFI->getStrategy().isGCManagedPointer(Ty))
     assert(*IsManaged && "Non gc managed pointer relocated!");
 #endif
 
   const Value *DerivedPtr = Relocate.getDerivedPtr();
   auto &RelocationMap =

llvm/lib/CodeGen/ShadowStackGCLowering.cpp

void ShadowStackGCLowering::CollectRoots(Function &F) {
   SmallVector<std::pair<CallInst *, AllocaInst *>, 16> MetaRoots;
 
   for (BasicBlock &BB : F)
     for (BasicBlock::iterator II = BB.begin(), E = BB.end(); II != E;)
       if (IntrinsicInst *CI = dyn_cast<IntrinsicInst>(II++))
         if (Function *F = CI->getCalledFunction())
           if (F->getIntrinsicID() == Intrinsic::gcroot) {
             std::pair<CallInst *, AllocaInst *> Pair = std::make_pair(
-                CI,
+                                                                      cast<CallInst>(CI),
                 cast<AllocaInst>(CI->getArgOperand(0)->stripPointerCasts()));
             if (IsNullValue(CI->getArgOperand(1)))
               Roots.push_back(Pair);
             else
               MetaRoots.push_back(Pair);
           }
 
   // Number roots with metadata (usually empty) at the beginning, so that the

llvm/lib/CodeGen/StackProtector.cpp

 
 /// Search for the first call to the llvm.stackprotector intrinsic and return it
 /// if present.
 static const CallInst *findStackProtectorIntrinsic(Function &F) {
   for (const BasicBlock &BB : F)
     for (const Instruction &I : BB)
       if (const auto *II = dyn_cast<IntrinsicInst>(&I))
         if (II->getIntrinsicID() == Intrinsic::stackprotector)
-          return II;
+          return cast<CallInst>(II);
   return nullptr;
 }
 
 /// Check whether or not this function needs a stack protector based
 /// upon the stack protector level.
 ///
 /// We use two heuristics: a standard (ssp) and strong (sspstrong).
 /// The standard heuristic which will add a guard variable to functions that

llvm/lib/ExecutionEngine/Interpreter/Execution.cpp

void Interpreter::visitIntrinsicInst(IntrinsicInst &I) {
   // If it is an unknown intrinsic function, use the intrinsic lowering
   // class to transform it into hopefully tasty LLVM code.
   //
   BasicBlock::iterator Me(&I);
   BasicBlock *Parent = I.getParent();
   bool atBegin(Parent->begin() == Me);
   if (!atBegin)
     --Me;
-  IL->LowerIntrinsicCall(&I);
+  IL->LowerIntrinsicCall(cast<CallInst>(&I));
 
   // Restore the CurInst pointer to the first instruction newly inserted, if
   // any.
   if (atBegin) {
     SF.CurInst = Parent->begin();
   } else {
     SF.CurInst = Me;
     ++SF.CurInst;

llvm/lib/Target/X86/X86FastISel.cpp

case Intrinsic::memcpy: {
 
     unsigned SizeWidth = Subtarget->is64Bit() ? 64 : 32;
     if (!MCI->getLength()->getType()->isIntegerTy(SizeWidth))
       return false;
 
     if (MCI->getSourceAddressSpace() > 255 || MCI->getDestAddressSpace() > 255)
       return false;
 
-    return lowerCallTo(II, "memcpy", II->getNumArgOperands() - 1);
+    return lowerCallTo(cast<CallInst>(II), "memcpy", II->getNumArgOperands() - 1);
   }
   case Intrinsic::memset: {
     const MemSetInst *MSI = cast<MemSetInst>(II);
 
     if (MSI->isVolatile())
       return false;
 
     unsigned SizeWidth = Subtarget->is64Bit() ? 64 : 32;
     if (!MSI->getLength()->getType()->isIntegerTy(SizeWidth))
       return false;
 
     if (MSI->getDestAddressSpace() > 255)
       return false;
 
-    return lowerCallTo(II, "memset", II->getNumArgOperands() - 1);
+    return lowerCallTo(cast<CallInst>(II), "memset", II->getNumArgOperands() - 1);
   }
   case Intrinsic::stackprotector: {
     // Emit code to store the stack guard onto the stack.
     EVT PtrTy = TLI.getPointerTy(DL);
 
     const Value *Op1 = II->getArgOperand(0); // The guard's value.
     const AllocaInst *Slot = cast<AllocaInst>(II->getArgOperand(1));
 

llvm/lib/Transforms/InstCombine/InstCombineAndOrXor.cpp

if (!ShAmt) {
     ShAmt = matchShiftAmount(ShAmt1, ShAmt0, Width);
     IsFshl = false; // Sub on SHL.
   }
   if (!ShAmt)
     return nullptr;
 
   Intrinsic::ID IID = IsFshl ? Intrinsic::fshl : Intrinsic::fshr;
   Function *F = Intrinsic::getDeclaration(Or.getModule(), IID, Or.getType());
-  return IntrinsicInst::Create(F, {ShVal0, ShVal1, ShAmt});
+  return CallInst::Create(F, {ShVal0, ShVal1, ShAmt});
 }
 
 /// Attempt to combine or(zext(x),shl(zext(y),bw/2) concat packing patterns.
 static Instruction *matchOrConcat(Instruction &Or,
                                   InstCombiner::BuilderTy &Builder) {
   assert(Or.getOpcode() == Instruction::Or && "bswap requires an 'or'");
   Value *Op0 = Or.getOperand(0), *Op1 = Or.getOperand(1);
   Type *Ty = Or.getType();

llvm/lib/Transforms/InstCombine/InstCombineCasts.cpp

Instruction *InstCombinerImpl::narrowFunnelShift(TruncInst &Trunc) {
   // llvm.fshl.i8(trunc(ShVal), trunc(ShVal), trunc(ShAmt))
   Value *NarrowShAmt = Builder.CreateTrunc(ShAmt, DestTy);
   Value *X, *Y;
   X = Y = Builder.CreateTrunc(ShVal0, DestTy);
   if (ShVal0 != ShVal1)
     Y = Builder.CreateTrunc(ShVal1, DestTy);
   Intrinsic::ID IID = IsFshl ? Intrinsic::fshl : Intrinsic::fshr;
   Function *F = Intrinsic::getDeclaration(Trunc.getModule(), IID, DestTy);
-  return IntrinsicInst::Create(F, {X, Y, NarrowShAmt});
+  return CallInst::Create(F, {X, Y, NarrowShAmt});
 }
 
 /// Try to narrow the width of math or bitwise logic instructions by pulling a
 /// truncate ahead of binary operators.
 /// TODO: Transforms for truncated shifts should be moved into here.
 Instruction *InstCombinerImpl::narrowBinOp(TruncInst &Trunc) {
   Type *SrcTy = Trunc.getSrcTy();
   Type *DestTy = Trunc.getType();
if (DestTy->isIntegerTy() &&
         SrcTy->getScalarSizeInBits() == 8 &&
         ShufElts.getKnownMinValue() % 2 == 0 && Shuf->hasOneUse() &&
         Shuf->isReverse()) {
       assert(ShufOp0->getType() == SrcTy && "Unexpected shuffle mask");
       assert(isa<UndefValue>(ShufOp1) && "Unexpected shuffle op");
       Function *Bswap =
           Intrinsic::getDeclaration(CI.getModule(), Intrinsic::bswap, DestTy);
       Value *ScalarX = Builder.CreateBitCast(ShufOp0, DestTy);
-      return IntrinsicInst::Create(Bswap, { ScalarX });
+      return CallInst::Create(Bswap, { ScalarX });
     }
   }
 
   // Handle the A->B->A cast, and there is an intervening PHI node.
   if (PHINode *PN = dyn_cast<PHINode>(Src))
     if (Instruction *I = optimizeBitCastFromPhi(CI, PN))
       return I;
 

llvm/lib/Transforms/InstCombine/InstCombineInternal.h

public:
   Instruction *visitUIToFP(CastInst &CI);
   Instruction *visitSIToFP(CastInst &CI);
   Instruction *visitPtrToInt(PtrToIntInst &CI);
   Instruction *visitIntToPtr(IntToPtrInst &CI);
   Instruction *visitBitCast(BitCastInst &CI);
   Instruction *visitAddrSpaceCast(AddrSpaceCastInst &CI);
   Instruction *foldItoFPtoI(CastInst &FI);
   Instruction *visitSelectInst(SelectInst &SI);
+  Instruction *visitCallBase(CallBase &Call);
   Instruction *visitCallInst(CallInst &CI);
   Instruction *visitInvokeInst(InvokeInst &II);
   Instruction *visitCallBrInst(CallBrInst &CBI);
 
   Instruction *SliceUpIllegalIntegerPHI(PHINode &PN);
   Instruction *visitPHINode(PHINode &PN);
   Instruction *visitGetElementPtrInst(GetElementPtrInst &GEP);
   Instruction *visitAllocaInst(AllocaInst &AI);
private:
   /// operation in OperationResult and result of the overflow check in
   /// OverflowResult, and return true.  If no simplification is possible,
   /// returns false.
   bool OptimizeOverflowCheck(Instruction::BinaryOps BinaryOp, bool IsSigned,
                              Value *LHS, Value *RHS,
                              Instruction &CtxI, Value *&OperationResult,
                              Constant *&OverflowResult);
 
-  Instruction *visitCallBase(CallBase &Call);
   Instruction *tryOptimizeCall(CallInst *CI);
   bool transformConstExprCastCall(CallBase &Call);
   Instruction *transformCallThroughTrampoline(CallBase &Call,
                                               IntrinsicInst &Tramp);
 
   Value *simplifyMaskedLoad(IntrinsicInst &II);
   Instruction *simplifyMaskedStore(IntrinsicInst &II);
   Instruction *simplifyMaskedGather(IntrinsicInst &II);

llvm/lib/Transforms/InstCombine/InstCombineSelect.cpp

else if (!IsFshl && !llvm::isGuaranteedNotToBePoison(SV0))
       SV0 = Builder.CreateFreeze(SV0);
   }
 
   // This is a funnel/rotate that avoids shift-by-bitwidth UB in a suboptimal way.
   // Convert to funnel shift intrinsic.
   Intrinsic::ID IID = IsFshl ? Intrinsic::fshl : Intrinsic::fshr;
   Function *F = Intrinsic::getDeclaration(Sel.getModule(), IID, Sel.getType());
   ShAmt = Builder.CreateZExt(ShAmt, Sel.getType());
-  return IntrinsicInst::Create(F, { SV0, SV1, ShAmt });
+  return CallInst::Create(F, { SV0, SV1, ShAmt });
 }
 
 static Instruction *foldSelectToCopysign(SelectInst &Sel,
                                          InstCombiner::BuilderTy &Builder) {
   Value *Cond = Sel.getCondition();
   Value *TVal = Sel.getTrueValue();
   Value *FVal = Sel.getFalseValue();
   Type *SelType = Sel.getType();
static Instruction *foldSelectToCopysign(SelectInst &Sel,
   if (IsTrueIfSignSet ^ TC->isNegative())
     X = Builder.CreateFNegFMF(X, &Sel);
 
   // Canonicalize the magnitude argument as the positive constant since we do
   // not care about its sign.
   Value *MagArg = TC->isNegative() ? FVal : TVal;
   Function *F = Intrinsic::getDeclaration(Sel.getModule(), Intrinsic::copysign,
                                           Sel.getType());
-  Instruction *CopySign = IntrinsicInst::Create(F, { MagArg, X });
+  Instruction *CopySign = CallInst::Create(F, { MagArg, X });
   CopySign->setFastMathFlags(Sel.getFastMathFlags());
   return CopySign;
 }
 
 Instruction *InstCombinerImpl::foldVectorSelect(SelectInst &Sel) {
   auto *VecTy = dyn_cast<FixedVectorType>(Sel.getType());
   if (!VecTy)
     return nullptr;

llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp

struct VarArgAMD64Helper : public VarArgHelper {
   unsigned AMD64FpEndOffset;
   Function &F;
   MemorySanitizer &MS;
   MemorySanitizerVisitor &MSV;
   Value *VAArgTLSCopy = nullptr;
   Value *VAArgTLSOriginCopy = nullptr;
   Value *VAArgOverflowSize = nullptr;
 
-  SmallVector<CallInst*, 16> VAStartInstrumentationList;
+  SmallVector<IntrinsicInst*, 16> VAStartInstrumentationList;
 
   enum ArgKind { AK_GeneralPurpose, AK_FloatingPoint, AK_Memory };
 
   VarArgAMD64Helper(Function &F, MemorySanitizer &MS,
                     MemorySanitizerVisitor &MSV)
       : F(F), MS(MS), MSV(MSV) {
     AMD64FpEndOffset = AMD64FpEndOffsetSSE;
     for (const auto &Attr : F.getAttributes().getFnAttributes()) {
if (!VAStartInstrumentationList.empty()) {
         IRB.CreateMemCpy(VAArgTLSOriginCopy, Align(8), MS.VAArgOriginTLS,
                          Align(8), CopySize);
       }
     }
 
     // Instrument va_start.
     // Copy va_list shadow from the backup copy of the TLS contents.
     for (size_t i = 0, n = VAStartInstrumentationList.size(); i < n; i++) {
-      CallInst *OrigInst = VAStartInstrumentationList[i];
+      auto *OrigInst = VAStartInstrumentationList[i];
       IRBuilder<> IRB(OrigInst->getNextNode());
       Value *VAListTag = OrigInst->getArgOperand(0);
 
       Type *RegSaveAreaPtrTy = Type::getInt64PtrTy(*MS.C);
       Value *RegSaveAreaPtrPtr = IRB.CreateIntToPtr(
           IRB.CreateAdd(IRB.CreatePtrToInt(VAListTag, MS.IntptrTy),
                         ConstantInt::get(MS.IntptrTy, 16)),
           PointerType::get(RegSaveAreaPtrTy, 0));

 /// MIPS64-specific implementation of VarArgHelper.
 struct VarArgMIPS64Helper : public VarArgHelper {
   Function &F;
   MemorySanitizer &MS;
   MemorySanitizerVisitor &MSV;
   Value *VAArgTLSCopy = nullptr;
   Value *VAArgSize = nullptr;
 
-  SmallVector<CallInst*, 16> VAStartInstrumentationList;
+  SmallVector<IntrinsicInst*, 16> VAStartInstrumentationList;
 
   VarArgMIPS64Helper(Function &F, MemorySanitizer &MS,
                     MemorySanitizerVisitor &MSV) : F(F), MS(MS), MSV(MSV) {}
 
   void visitCallBase(CallBase &CB, IRBuilder<> &IRB) override {
     unsigned VAArgOffset = 0;
     const DataLayout &DL = F.getParent()->getDataLayout();
     for (auto ArgIt = CB.arg_begin() + CB.getFunctionType()->getNumParams(),
if (!VAStartInstrumentationList.empty()) {
       // va_arg_tls somewhere in the function entry block.
       VAArgTLSCopy = IRB.CreateAlloca(Type::getInt8Ty(*MS.C), CopySize);
       IRB.CreateMemCpy(VAArgTLSCopy, Align(8), MS.VAArgTLS, Align(8), CopySize);
     }
 
     // Instrument va_start.
     // Copy va_list shadow from the backup copy of the TLS contents.
     for (size_t i = 0, n = VAStartInstrumentationList.size(); i < n; i++) {
-      CallInst *OrigInst = VAStartInstrumentationList[i];
+      auto *OrigInst = VAStartInstrumentationList[i];
       IRBuilder<> IRB(OrigInst->getNextNode());
       Value *VAListTag = OrigInst->getArgOperand(0);
       Type *RegSaveAreaPtrTy = Type::getInt64PtrTy(*MS.C);
       Value *RegSaveAreaPtrPtr =
           IRB.CreateIntToPtr(IRB.CreatePtrToInt(VAListTag, MS.IntptrTy),
                              PointerType::get(RegSaveAreaPtrTy, 0));
       Value *RegSaveAreaPtr =
           IRB.CreateLoad(RegSaveAreaPtrTy, RegSaveAreaPtrPtr);
struct VarArgAArch64Helper : public VarArgHelper {
   static const unsigned AArch64VAEndOffset = AArch64VrEndOffset;
 
   Function &F;
   MemorySanitizer &MS;
   MemorySanitizerVisitor &MSV;
   Value *VAArgTLSCopy = nullptr;
   Value *VAArgOverflowSize = nullptr;
 
-  SmallVector<CallInst*, 16> VAStartInstrumentationList;
+  SmallVector<IntrinsicInst*, 16> VAStartInstrumentationList;
 
   enum ArgKind { AK_GeneralPurpose, AK_FloatingPoint, AK_Memory };
 
   VarArgAArch64Helper(Function &F, MemorySanitizer &MS,
                     MemorySanitizerVisitor &MSV) : F(F), MS(MS), MSV(MSV) {}
 
   ArgKind classifyArgument(Value* arg) {
     Type *T = arg->getType();
void finalizeInstrumentation() override {
     }
 
     Value *GrArgSize = ConstantInt::get(MS.IntptrTy, kAArch64GrArgSize);
     Value *VrArgSize = ConstantInt::get(MS.IntptrTy, kAArch64VrArgSize);
 
     // Instrument va_start, copy va_list shadow from the backup copy of
     // the TLS contents.
     for (size_t i = 0, n = VAStartInstrumentationList.size(); i < n; i++) {
-      CallInst *OrigInst = VAStartInstrumentationList[i];
+      auto *OrigInst = VAStartInstrumentationList[i];
       IRBuilder<> IRB(OrigInst->getNextNode());
 
       Value *VAListTag = OrigInst->getArgOperand(0);
 
       // The variadic ABI for AArch64 creates two areas to save the incoming
       // argument registers (one for 64-bit general register xn-x7 and another
       // for 128-bit FP/SIMD vn-v7).
       // We need then to propagate the shadow arguments on both regions

 /// PowerPC64-specific implementation of VarArgHelper.
 struct VarArgPowerPC64Helper : public VarArgHelper {
   Function &F;
   MemorySanitizer &MS;
   MemorySanitizerVisitor &MSV;
   Value *VAArgTLSCopy = nullptr;
   Value *VAArgSize = nullptr;
 
-  SmallVector<CallInst*, 16> VAStartInstrumentationList;
+  SmallVector<IntrinsicInst*, 16> VAStartInstrumentationList;
 
   VarArgPowerPC64Helper(Function &F, MemorySanitizer &MS,
                     MemorySanitizerVisitor &MSV) : F(F), MS(MS), MSV(MSV) {}
 
   void visitCallBase(CallBase &CB, IRBuilder<> &IRB) override {
     // For PowerPC, we need to deal with alignment of stack arguments -
     // they are mostly aligned to 8 bytes, but vectors and i128 arrays
     // are aligned to 16 bytes, byvals can be aligned to 8 or 16 bytes,
if (!VAStartInstrumentationList.empty()) {
       // va_arg_tls somewhere in the function entry block.
       VAArgTLSCopy = IRB.CreateAlloca(Type::getInt8Ty(*MS.C), CopySize);
       IRB.CreateMemCpy(VAArgTLSCopy, Align(8), MS.VAArgTLS, Align(8), CopySize);
     }
 
     // Instrument va_start.
     // Copy va_list shadow from the backup copy of the TLS contents.
     for (size_t i = 0, n = VAStartInstrumentationList.size(); i < n; i++) {
-      CallInst *OrigInst = VAStartInstrumentationList[i];
+      auto *OrigInst = VAStartInstrumentationList[i];
       IRBuilder<> IRB(OrigInst->getNextNode());
       Value *VAListTag = OrigInst->getArgOperand(0);
       Type *RegSaveAreaPtrTy = Type::getInt64PtrTy(*MS.C);
       Value *RegSaveAreaPtrPtr =
           IRB.CreateIntToPtr(IRB.CreatePtrToInt(VAListTag, MS.IntptrTy),
                              PointerType::get(RegSaveAreaPtrTy, 0));
       Value *RegSaveAreaPtr =
           IRB.CreateLoad(RegSaveAreaPtrTy, RegSaveAreaPtrPtr);
struct VarArgSystemZHelper : public VarArgHelper {
 
   Function &F;
   MemorySanitizer &MS;
   MemorySanitizerVisitor &MSV;
   Value *VAArgTLSCopy = nullptr;
   Value *VAArgTLSOriginCopy = nullptr;
   Value *VAArgOverflowSize = nullptr;
 
-  SmallVector<CallInst *, 16> VAStartInstrumentationList;
+  SmallVector<IntrinsicInst *, 16> VAStartInstrumentationList;
 
   enum class ArgKind {
     GeneralPurpose,
     FloatingPoint,
     Vector,
     Memory,
     Indirect,
   };
if (!VAStartInstrumentationList.empty()) {
                          Align(8), CopySize);
       }
     }
 
     // Instrument va_start.
     // Copy va_list shadow from the backup copy of the TLS contents.
     for (size_t VaStartNo = 0, VaStartNum = VAStartInstrumentationList.size();
          VaStartNo < VaStartNum; VaStartNo++) {
-      CallInst *OrigInst = VAStartInstrumentationList[VaStartNo];
+      auto *OrigInst = VAStartInstrumentationList[VaStartNo];
       IRBuilder<> IRB(OrigInst->getNextNode());
       Value *VAListTag = OrigInst->getArgOperand(0);
       copyRegSaveArea(IRB, VAListTag);
       copyOverflowArea(IRB, VAListTag);
     }
   }
 };
 

llvm/unittests/Analysis/AssumeBundleQueriesTest.cpp

if (count > 0) {
       HasArg[i] = true;
     }
     if (count > 1)
       Args.push_back(ConstantInt::get(Type::getInt32Ty(C), value));
 
     OpBundle.push_back(OperandBundleDef{ss.str().c_str(), std::move(Args)});
   }
 
-  auto *Assume = cast<IntrinsicInst>(IntrinsicInst::Create(
+  auto *Assume = cast<IntrinsicInst>(CallInst::Create(
       FnAssume, ArrayRef<Value *>({ConstantInt::getTrue(C)}), OpBundle));
   Assume->insertBefore(&F->begin()->front());
   RetainedKnowledgeMap Map;
   fillMapFromAssume(*Assume, Map);
   for (int i = 0; i < (Size * 2); i++) {
     if (!HasArg[i])
       continue;
     RetainedKnowledge K =