[IRSim][IROutliner] Allowing Intrinsic Calls to be Used in Similarity Matching and Outlined Regions

[AndrewLitteken]

Summary

Due to some complications with lifetime, and assume-like intrinsics, intrinsics were not included as outlinable instructions. This patch opens up most intrinsics, excluding lifetime and assume-like intrinsics, to be outlined. For similarity, it is required that the intrinsic IDs, and the intrinsics names match exactly, as well as the function type. This puts intrinsics in a different class than normal call instructions (https://reviews.llvm.org/D109448), where the name will no longer have to match.

This also adds an additional command line flag debug option to disable outlining intrinsics.

[paquette]

llvm/include/llvm/Analysis/IRSimilarityIdentifier.h

255–279

can you add some comments?

540

add comments for why these are disabled?

574

this comment can probably be simplified

[AndrewLitteken]

Adding comments.

[paquette]

llvm/include/llvm/Analysis/IRSimilarityIdentifier.h

255–279

this always returns, right?

why not just make it an if?

275

this can also just be an if?

544

can you expand on the last sentence here?

llvm/lib/Analysis/IRSimilarityIdentifier.cpp

49

change passive voice?

"Don't match or outline intrinsics."

[AndrewLitteken]

Updating comments

[jroelofs]

LGTM

[uabelho]

Hello,

The following fails with this commit:

opt -passes='require<ir-similarity>' -o /dev/null bbi-65410.ll

Result:

opt: ../lib/Analysis/IRSimilarityIdentifier.cpp:998: void llvm::IRSimilarity::IRSimilarityCandidate::createCanonicalRelationFrom(llvm::IRSimilarity::IRSimilarityCandidate &, DenseMap<unsigned int, DenseSet<unsigned int> > &, DenseMap<unsigned int, DenseSet<unsigned int> > &): Assertion `Found && "Could not find matching value for source GVN"' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.	Program arguments: build-all/bin/opt -passes=require<ir-similarity> -o /dev/null bbi-65410.ll
 #0 0x0000000002cbb993 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (build-all/bin/opt+0x2cbb993)
 #1 0x0000000002cb960e llvm::sys::RunSignalHandlers() (build-all/bin/opt+0x2cb960e)
 #2 0x0000000002cbbd16 SignalHandler(int) Signals.cpp:0:0
 #3 0x00007fcc4be0e630 __restore_rt sigaction.c:0:0
 #4 0x00007fcc49541387 raise (/lib64/libc.so.6+0x36387)
 #5 0x00007fcc49542a78 abort (/lib64/libc.so.6+0x37a78)
 #6 0x00007fcc4953a1a6 __assert_fail_base (/lib64/libc.so.6+0x2f1a6)
 #7 0x00007fcc4953a252 (/lib64/libc.so.6+0x2f252)
 #8 0x0000000001b4f5aa llvm::IRSimilarity::IRSimilarityCandidate::createCanonicalRelationFrom(llvm::IRSimilarity::IRSimilarityCandidate&, llvm::DenseMap<unsigned int, llvm::DenseSet<unsigned int, llvm::DenseMapInfo<unsigned int, void> >, llvm::DenseMapInfo<unsigned int, void>, llvm::detail::DenseMapPair<unsigned int, llvm::DenseSet<unsigned int, llvm::DenseMapInfo<unsigned int, void> > > >&, llvm::DenseMap<unsigned int, llvm::DenseSet<unsigned int, llvm::DenseMapInfo<unsigned int, void> >, llvm::DenseMapInfo<unsigned int, void>, llvm::detail::DenseMapPair<unsigned int, llvm::DenseSet<unsigned int, llvm::DenseMapInfo<unsigned int, void> > > >&) (build-all/bin/opt+0x1b4f5aa)
 #9 0x0000000001b504d1 llvm::IRSimilarity::IRSimilarityIdentifier::findCandidates(std::vector<llvm::IRSimilarity::IRInstructionData*, std::allocator<llvm::IRSimilarity::IRInstructionData*> >&, std::vector<unsigned int, std::allocator<unsigned int> >&) (build-all/bin/opt+0x1b504d1)
#10 0x0000000001b512ba llvm::IRSimilarity::IRSimilarityIdentifier::findSimilarity(llvm::Module&) (build-all/bin/opt+0x1b512ba)
#11 0x0000000001b5185a llvm::IRSimilarityAnalysis::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (build-all/bin/opt+0x1b5185a)
#12 0x0000000002fc43a0 llvm::detail::AnalysisPassModel<llvm::Module, llvm::IRSimilarityAnalysis, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Module>::Invalidator>::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) crtstuff.c:0:0
#13 0x000000000241321e llvm::AnalysisManager<llvm::Module>::getResultImpl(llvm::AnalysisKey*, llvm::Module&) (build-all/bin/opt+0x241321e)
#14 0x0000000002fd0e88 llvm::RequireAnalysisPass<llvm::IRSimilarityAnalysis, llvm::Module, llvm::AnalysisManager<llvm::Module> >::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) crtstuff.c:0:0
#15 0x0000000002fd0ccd llvm::detail::PassModel<llvm::Module, llvm::RequireAnalysisPass<llvm::IRSimilarityAnalysis, llvm::Module, llvm::AnalysisManager<llvm::Module> >, llvm::PreservedAnalyses, llvm::AnalysisManager<llvm::Module> >::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) crtstuff.c:0:0
#16 0x0000000002410a98 llvm::PassManager<llvm::Module, llvm::AnalysisManager<llvm::Module> >::run(llvm::Module&, llvm::AnalysisManager<llvm::Module>&) (build-all/bin/opt+0x2410a98)
#17 0x000000000079e0a2 llvm::runPassPipeline(llvm::StringRef, llvm::Module&, llvm::TargetMachine*, llvm::TargetLibraryInfoImpl*, llvm::ToolOutputFile*, llvm::ToolOutputFile*, llvm::ToolOutputFile*, llvm::StringRef, llvm::ArrayRef<llvm::StringRef>, llvm::opt_tool::OutputKind, llvm::opt_tool::VerifierKind, bool, bool, bool, bool, bool) (build-all/bin/opt+0x79e0a2)
#18 0x00000000007b0c74 main (build-all/bin/opt+0x7b0c74)
#19 0x00007fcc4952d555 __libc_start_main (/lib64/libc.so.6+0x22555)
#20 0x000000000079952c _start (build-all/bin/opt+0x79952c)
Abort

bbi-65410.ll643 BDownload

[uabelho]

In D109450#3283855, @uabelho wrote:

Hello,

The following fails with this commit:

opt -passes='require<ir-similarity>' -o /dev/null bbi-65410.ll

I wrote an issue about this crash:
https://github.com/llvm/llvm-project/issues/53537

llvm/include/llvm/Analysis/IRSimilarityIdentifier.h

for (Value *V : ID.OperVals)
       OperTypes.push_back(V->getType());
 
     if (isa<CmpInst>(ID.Inst))
       return llvm::hash_combine(
           llvm::hash_value(ID.Inst->getOpcode()),
           llvm::hash_value(ID.Inst->getType()),
           llvm::hash_value(ID.getPredicate()),
           llvm::hash_combine_range(OperTypes.begin(), OperTypes.end()));
-    else if (CallInst *CI = dyn_cast<CallInst>(ID.Inst)) {
+
+    if (IntrinsicInst *II = dyn_cast<IntrinsicInst>(ID.Inst)) {
+      // To hash intrinsics, we use the opcode, and types like the other
+      // instructions, but also, the Intrinsic ID, and the Name of the
+      // intrinsic.
+      Intrinsic::ID IntrinsicID = II->getIntrinsicID();
+      FunctionType *FT = II->getFunctionType();
+      std::string Name;
+      // If there is an overloaded name, we have to use the complex version
+      // of getName to get the entire string.
+      if (Intrinsic::isOverloaded(IntrinsicID))
+        Name =
+            Intrinsic::getName(IntrinsicID, FT->params(), II->getModule(), FT);
+      // If there is not an overloaded name, we only need to use this version.
+      else
+        Name = Intrinsic::getName(IntrinsicID).str();
+      return llvm::hash_combine(
+          llvm::hash_value(ID.Inst->getOpcode()),
+          llvm::hash_value(ID.Inst->getType()), llvm::hash_value(IntrinsicID),
+          llvm::hash_value(Name),
+          llvm::hash_combine_range(OperTypes.begin(), OperTypes.end()));
+    }
+
+    if (isa<CallInst>(ID.Inst)) {
       std::string FunctionName = *ID.CalleeName;
       return llvm::hash_combine(
           llvm::hash_value(ID.Inst->getOpcode()),
           llvm::hash_value(ID.Inst->getType()),
           llvm::hash_value(ID.Inst->getType()), llvm::hash_value(FunctionName),
           llvm::hash_combine_range(OperTypes.begin(), OperTypes.end()));
     }
+
     return llvm::hash_combine(
         llvm::hash_value(ID.Inst->getOpcode()),
         llvm::hash_value(ID.Inst->getType()),
         llvm::hash_combine_range(OperTypes.begin(), OperTypes.end()));
   }
 
   IRInstructionDataList *IDL = nullptr;
 };
struct InstructionClassification
     // We exclude all exception handling cases since they are so context
     // dependent.
     InstrType visitLandingPadInst(LandingPadInst &LPI) { return Illegal; }
     InstrType visitFuncletPadInst(FuncletPadInst &FPI) { return Illegal; }
     // DebugInfo should be included in the regions, but should not be
     // analyzed for similarity as it has no bearing on the outcome of the
     // program.
     InstrType visitDbgInfoIntrinsic(DbgInfoIntrinsic &DII) { return Invisible; }
-    // TODO: Handle specific intrinsics.
-    InstrType visitIntrinsicInst(IntrinsicInst &II) { return Illegal; }
+    InstrType visitIntrinsicInst(IntrinsicInst &II) {
+      // These are disabled due to complications in the CodeExtractor when
+      // outlining these instructions.  For instance, It is unclear what we
+      // should do when moving only the start or end lifetime instruction into
+      // an outlined function. Also, assume-like intrinsics could be removed
+      // from the region, removing arguments, causing discrepencies in the
+      // number of inputs between different regions.
+      if (II.isLifetimeStartOrEnd() || II.isAssumeLikeIntrinsic())
+        return Illegal;
+      return EnableIntrinsics ? Legal : Illegal;
+    }
     // We only allow call instructions where the function has a name and
     // is not an indirect call.
     InstrType visitCallInst(CallInst &CI) {
       Function *F = CI.getCalledFunction();
       bool IsIndirectCall = CI.isIndirectCall();
       if (IsIndirectCall && !EnableIndirectCalls)
         return Illegal;
       if (!F && !IsIndirectCall)
         return Illegal;
       return Legal;
     }
     // TODO: We do not current handle similarity that changes the control flow.
     InstrType visitInvokeInst(InvokeInst &II) { return Illegal; }
     // TODO: We do not current handle similarity that changes the control flow.
     InstrType visitCallBrInst(CallBrInst &CBI) { return Illegal; }
     // TODO: Handle interblock similarity.
     InstrType visitTerminator(Instruction &I) { return Illegal; }
     InstrType visitInstruction(Instruction &I) { return Legal; }
 
     // The flag variable that lets the classifier know whether we should
     // allow branches to be checked for similarity.
     bool EnableBranches = false;
 
     // The flag variable that lets the classifier know whether we should
     // allow indirect calls to be considered legal instructions.
     bool EnableIndirectCalls = false;
+
+    // Flag that lets the classifier know whether we should allow intrinsics to
+    // be checked for similarity.
+    bool EnableIntrinsics = false;
   };
 
   /// Maps an Instruction to a member of InstrType.
   InstructionClassification InstClassifier;
 };
 
 /// This is a class that wraps a range of IRInstructionData from one point to
 /// another in the vector of IRInstructionData, which is a region of the

 /// A, B, D             C
 ///
 /// A list of the different similarity groups is then returned after
 /// analyzing the module.
 class IRSimilarityIdentifier {
 public:
   IRSimilarityIdentifier(bool MatchBranches = true,
                          bool MatchIndirectCalls = true,
-                         bool MatchCallsWithName = false)
+                         bool MatchCallsWithName = false,
+                         bool MatchIntrinsics = true)
       : Mapper(&InstDataAllocator, &InstDataListAllocator),
         EnableBranches(MatchBranches), EnableIndirectCalls(MatchIndirectCalls),
-        EnableMatchingCallsByName(MatchCallsWithName) {}
+        EnableMatchingCallsByName(MatchCallsWithName),
+        EnableIntrinsics(MatchIntrinsics) {}
 
 private:
   /// Map the instructions in the module to unsigned integers, using mapping
   /// already present in the Mapper if possible.
   ///
   /// \param [in] M Module - To map to integers.
   /// \param [in,out] InstrList - The vector to append IRInstructionData to.
   /// \param [in,out] IntegerMapping - The vector to append integers to.
private:
   /// for similarity, or exclude them as a legal instruction.
   bool EnableIndirectCalls = true;
 
   /// The flag variable that marks whether we allow calls to be marked as
   /// similar if they do not have the same name, only the same calling
   /// convention, attributes and type signature.
   bool EnableMatchingCallsByName = true;
 
+  /// The flag variable that marks whether we should check intrinsics for
+  /// similarity.
+  bool EnableIntrinsics = true;
+
   /// The SimilarityGroups found with the most recent run of \ref
   /// findSimilarity. None if there is no recent run.
   Optional<SimilarityGroupList> SimilarityCandidates;
 };
 
 } // end namespace IRSimilarity
 
 /// An analysis pass based on legacy pass manager that runs and returns

llvm/include/llvm/Transforms/IPO/IROutliner.h

struct InstructionAllowed : public InstVisitor<InstructionAllowed, bool> {
     bool visitLandingPadInst(LandingPadInst &LPI) { return false; }
     bool visitFuncletPadInst(FuncletPadInst &FPI) { return false; }
     // DebugInfo should be included in the regions, but should not be
     // analyzed for similarity as it has no bearing on the outcome of the
     // program.
     bool visitDbgInfoIntrinsic(DbgInfoIntrinsic &DII) { return true; }
     // TODO: Handle specific intrinsics individually from those that can be
     // handled.
-    bool IntrinsicInst(IntrinsicInst &II) { return false; }
+    bool IntrinsicInst(IntrinsicInst &II) { return EnableIntrinsics; }
     // We only handle CallInsts that are not indirect, since we cannot guarantee
     // that they have a name in these cases.
     bool visitCallInst(CallInst &CI) {
       Function *F = CI.getCalledFunction();
       bool IsIndirectCall = CI.isIndirectCall();
       if (IsIndirectCall && !EnableIndirectCalls)
         return false;
       if (!F && !IsIndirectCall)
struct InstructionAllowed : public InstVisitor<InstructionAllowed, bool> {
 
     // The flag variable that marks whether we should allow branch instructions
     // to be outlined.
     bool EnableBranches = false;
 
     // The flag variable that marks whether we should allow indirect calls
     // to be outlined.
     bool EnableIndirectCalls = true;
+
+    // The flag variable that marks whether we should allow intrinsics
+    // instructions to be outlined.
+    bool EnableIntrinsics = false;
   };
 
   /// A InstVisitor used to exclude certain instructions from being outlined.
   InstructionAllowed InstructionClassifier;
 };
 
 /// Pass to outline similar regions.
 class IROutlinerPass : public PassInfoMixin<IROutlinerPass> {
 public:
   PreservedAnalyses run(Module &M, ModuleAnalysisManager &AM);
 };
 
 } // end namespace llvm
 
 #endif // LLVM_TRANSFORMS_IPO_IROUTLINER_H

llvm/lib/Analysis/IRSimilarityIdentifier.cpp

 using namespace IRSimilarity;
 
 namespace llvm {
 cl::opt<bool>
     DisableBranches("no-ir-sim-branch-matching", cl::init(false),
                     cl::ReallyHidden,
                     cl::desc("disable similarity matching, and outlining, "
                              "across branches for debugging purposes."));
-} // namespace llvm
 
 cl::opt<bool>
     DisableIndirectCalls("no-ir-sim-indirect-calls", cl::init(false),
                          cl::ReallyHidden,
                          cl::desc("disable outlining indirect calls."));
 
 cl::opt<bool>
     MatchCallsByName("ir-sim-calls-by-name", cl::init(false), cl::ReallyHidden,
                      cl::desc("only allow matching call instructions if the "
                               "name and type signature match."));
 
+cl::opt<bool>
+    DisableIntrinsics("no-ir-sim-intrinsics", cl::init(false), cl::ReallyHidden,
+                      cl::desc("Don't match or outline intrinsics"));
 
 IRInstructionData::IRInstructionData(Instruction &I, bool Legality,
                                      IRInstructionDataList &IDList)
     : Inst(&I), Legal(Legality), IDL(&IDList) {
   initializeInstruction();
 }
 
+} // namespace llvm
+
 void IRInstructionData::initializeInstruction() {
   // We check for whether we have a comparison instruction.  If it is, we
   // find the "less than" version of the predicate for consistency for
   // comparison instructions throught the program.
   if (CmpInst *C = dyn_cast<CmpInst>(Inst)) {
     CmpInst::Predicate Predicate = predicateForConsistency(C);
     if (Predicate != C->getPredicate())
       RevisedPredicate = Predicate;
SimilarityGroupList &IRSimilarityIdentifier::findSimilarity(
     ArrayRef<std::unique_ptr<Module>> Modules) {
   resetSimilarityCandidates();
 
   std::vector<IRInstructionData *> InstrList;
   std::vector<unsigned> IntegerMapping;
   Mapper.InstClassifier.EnableBranches = this->EnableBranches;
   Mapper.InstClassifier.EnableIndirectCalls = EnableIndirectCalls;
   Mapper.EnableMatchCallsByName = EnableMatchingCallsByName;
+  Mapper.InstClassifier.EnableIntrinsics = EnableIntrinsics;
 
   populateMapper(Modules, InstrList, IntegerMapping);
   findCandidates(InstrList, IntegerMapping);
 
   return SimilarityCandidates.getValue();
 }
 
 SimilarityGroupList &IRSimilarityIdentifier::findSimilarity(Module &M) {
   resetSimilarityCandidates();
   Mapper.InstClassifier.EnableBranches = this->EnableBranches;
   Mapper.InstClassifier.EnableIndirectCalls = EnableIndirectCalls;
   Mapper.EnableMatchCallsByName = EnableMatchingCallsByName;
+  Mapper.InstClassifier.EnableIntrinsics = EnableIntrinsics;
 
   std::vector<IRInstructionData *> InstrList;
   std::vector<unsigned> IntegerMapping;
 
   populateMapper(M, InstrList, IntegerMapping);
   findCandidates(InstrList, IntegerMapping);
 
   return SimilarityCandidates.getValue();
 }
 
 INITIALIZE_PASS(IRSimilarityIdentifierWrapperPass, "ir-similarity-identifier",
                 "ir-similarity-identifier", false, true)
 
 IRSimilarityIdentifierWrapperPass::IRSimilarityIdentifierWrapperPass()
     : ModulePass(ID) {
   initializeIRSimilarityIdentifierWrapperPassPass(
       *PassRegistry::getPassRegistry());
 }
 
 bool IRSimilarityIdentifierWrapperPass::doInitialization(Module &M) {
   IRSI.reset(new IRSimilarityIdentifier(!DisableBranches, !DisableIndirectCalls,
-                                        MatchCallsByName));
+                                        MatchCallsByName, !DisableIntrinsics));
   return false;
 }
 
 bool IRSimilarityIdentifierWrapperPass::doFinalization(Module &M) {
   IRSI.reset();
   return false;
 }
 
 bool IRSimilarityIdentifierWrapperPass::runOnModule(Module &M) {
   IRSI->findSimilarity(M);
   return false;
 }
 
 AnalysisKey IRSimilarityAnalysis::Key;
 IRSimilarityIdentifier IRSimilarityAnalysis::run(Module &M,
                                                  ModuleAnalysisManager &) {
-
   auto IRSI = IRSimilarityIdentifier(!DisableBranches, !DisableIndirectCalls,
-                                     MatchCallsByName);
+                                     MatchCallsByName, !DisableIntrinsics);
   IRSI.findSimilarity(M);
   return IRSI;
 }
 
 PreservedAnalyses
 IRSimilarityAnalysisPrinterPass::run(Module &M, ModuleAnalysisManager &AM) {
   IRSimilarityIdentifier &IRSI = AM.getResult<IRSimilarityAnalysis>(M);
   Optional<SimilarityGroupList> &SimilarityCandidatesOpt = IRSI.getSimilarity();

llvm/lib/Transforms/IPO/IROutliner.cpp

 
 using namespace llvm;
 using namespace IRSimilarity;
 
 // A command flag to be used for debugging to exclude branches from similarity
 // matching and outlining.
 namespace llvm {
 extern cl::opt<bool> DisableBranches;
-} // namespace llvm
 
 // A command flag to be used for debugging to indirect calls from similarity
 // matching and outlining.
 extern cl::opt<bool> DisableIndirectCalls;
 
+// A command flag to be used for debugging to exclude intrinsics from similarity
+// matching and outlining.
+extern cl::opt<bool> DisableIntrinsics;
+
+} // namespace llvm
+
 // Set to true if the user wants the ir outliner to run on linkonceodr linkage
 // functions. This is false by default because the linker can dedupe linkonceodr
 // functions. Since the outliner is confined to a single module (modulo LTO),
 // this is off by default. It should, however, be the default behavior in
 // LTO.
 static cl::opt<bool> EnableLinkOnceODRIROutlining(
     "enable-linkonceodr-ir-outlining", cl::Hidden,
     cl::desc("Enable the IR outliner on linkonceodr functions"),
bool IROutliner::extractSection(OutlinableRegion &Region) {
   Region.reattachCandidate();
   return true;
 }
 
 unsigned IROutliner::doOutline(Module &M) {
   // Find the possible similarity sections.
   InstructionClassifier.EnableBranches = !DisableBranches;
   InstructionClassifier.EnableIndirectCalls = !DisableIndirectCalls;
+  InstructionClassifier.EnableIntrinsics = !DisableIntrinsics;
+
   IRSimilarityIdentifier &Identifier = getIRSI(M);
   SimilarityGroupList &SimilarityCandidates = *Identifier.getSimilarity();
 
   // Sort them by size of extracted sections
   unsigned OutlinedFunctionNum = 0;
   // If we only have one SimilarityGroup in SimilarityCandidates, we do not have
   // to sort them by the potential number of instructions to be outlined
   if (SimilarityCandidates.size() > 1)

llvm/test/Transforms/IROutliner/different-intrinsics.ll

-; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --include-generated-funcs
 ; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost < %s | FileCheck %s
 
-; This test checks that we do not outline memcpy intrinsics since it may require
-; extra address space checks.
+; This test checks that we do not outline different intrinsics as the same
+; function or as a value like we would for non-intrinsic functions.
 
+declare void @llvm.memcpy.p0i8.p0i8.i64(i8* nocapture writeonly, i8* nocapture readonly, i64, i1)
 declare void @llvm.memmove.p0i8.p0i8.i64(i8* nocapture writeonly, i8* nocapture readonly, i64, i1)
 
 define i8 @function1(i8* noalias %s, i8* noalias %d, i64 %len) {
+entry:
+  %a = load i8, i8* %s
+  %b = load i8, i8* %d
+  call void @llvm.memcpy.p0i8.p0i8.i64(i8* %d, i8* %s, i64 %len, i1 false)
+  %c = add i8 %a, %b
+  %ret = load i8, i8* %s
+  ret i8 %ret
+}
+
+define i8 @function2(i8* noalias %s, i8* noalias %d, i64 %len) {
+entry:
+  %a = load i8, i8* %s
+  %b = load i8, i8* %d
+  call void @llvm.memmove.p0i8.p0i8.i64(i8* %d, i8* %s, i64 %len, i1 false)
+  %c = add i8 %a, %b
+  %ret = load i8, i8* %s
+  ret i8 %ret
+}
 ; CHECK-LABEL: @function1(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[B_LOC:%.*]] = alloca i8, align 1
 ; CHECK-NEXT:    [[A_LOC:%.*]] = alloca i8, align 1
 ; CHECK-NEXT:    [[RET_LOC:%.*]] = alloca i8, align 1
 ; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 -1, i8* [[A_LOC]])
 ; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 -1, i8* [[B_LOC]])
 ; CHECK-NEXT:    call void @outlined_ir_func_1(i8* [[S:%.*]], i8* [[D:%.*]], i8* [[A_LOC]], i8* [[B_LOC]])
 ; CHECK-NEXT:    [[A_RELOAD:%.*]] = load i8, i8* [[A_LOC]], align 1
 ; CHECK-NEXT:    [[B_RELOAD:%.*]] = load i8, i8* [[B_LOC]], align 1
 ; CHECK-NEXT:    call void @llvm.lifetime.end.p0i8(i64 -1, i8* [[A_LOC]])
 ; CHECK-NEXT:    call void @llvm.lifetime.end.p0i8(i64 -1, i8* [[B_LOC]])
-; CHECK-NEXT:    call void @llvm.memmove.p0i8.p0i8.i64(i8* [[D]], i8* [[S]], i64 [[LEN:%.*]], i1 false)
+; CHECK-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* [[D]], i8* [[S]], i64 [[LEN:%.*]], i1 false)
 ; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 -1, i8* [[RET_LOC]])
 ; CHECK-NEXT:    call void @outlined_ir_func_0(i8 [[A_RELOAD]], i8 [[B_RELOAD]], i8* [[S]], i8* [[RET_LOC]])
 ; CHECK-NEXT:    [[RET_RELOAD:%.*]] = load i8, i8* [[RET_LOC]], align 1
 ; CHECK-NEXT:    call void @llvm.lifetime.end.p0i8(i64 -1, i8* [[RET_LOC]])
 ; CHECK-NEXT:    ret i8 [[RET_RELOAD]]
 ;
-entry:
-  %a = load i8, i8* %s
-  %b = load i8, i8* %d
-  call void @llvm.memmove.p0i8.p0i8.i64(i8* %d, i8* %s, i64 %len, i1 false)
-  %c = add i8 %a, %b
-  %ret = load i8, i8* %s
-  ret i8 %ret
-}
-
-define i8 @function2(i8* noalias %s, i8* noalias %d, i64 %len) {
+;
 ; CHECK-LABEL: @function2(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[B_LOC:%.*]] = alloca i8, align 1
 ; CHECK-NEXT:    [[A_LOC:%.*]] = alloca i8, align 1
 ; CHECK-NEXT:    [[RET_LOC:%.*]] = alloca i8, align 1
 ; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 -1, i8* [[A_LOC]])
 ; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 -1, i8* [[B_LOC]])
 ; CHECK-NEXT:    call void @outlined_ir_func_1(i8* [[S:%.*]], i8* [[D:%.*]], i8* [[A_LOC]], i8* [[B_LOC]])
 ; CHECK-NEXT:    [[A_RELOAD:%.*]] = load i8, i8* [[A_LOC]], align 1
 ; CHECK-NEXT:    [[B_RELOAD:%.*]] = load i8, i8* [[B_LOC]], align 1
 ; CHECK-NEXT:    call void @llvm.lifetime.end.p0i8(i64 -1, i8* [[A_LOC]])
 ; CHECK-NEXT:    call void @llvm.lifetime.end.p0i8(i64 -1, i8* [[B_LOC]])
 ; CHECK-NEXT:    call void @llvm.memmove.p0i8.p0i8.i64(i8* [[D]], i8* [[S]], i64 [[LEN:%.*]], i1 false)
 ; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 -1, i8* [[RET_LOC]])
 ; CHECK-NEXT:    call void @outlined_ir_func_0(i8 [[A_RELOAD]], i8 [[B_RELOAD]], i8* [[S]], i8* [[RET_LOC]])
 ; CHECK-NEXT:    [[RET_RELOAD:%.*]] = load i8, i8* [[RET_LOC]], align 1
 ; CHECK-NEXT:    call void @llvm.lifetime.end.p0i8(i64 -1, i8* [[RET_LOC]])
 ; CHECK-NEXT:    ret i8 [[RET_RELOAD]]
 ;
-entry:
-  %a = load i8, i8* %s
-  %b = load i8, i8* %d
-  call void @llvm.memmove.p0i8.p0i8.i64(i8* %d, i8* %s, i64 %len, i1 false)
-  %c = add i8 %a, %b
-  %ret = load i8, i8* %s
-  ret i8 %ret
-}
+;
+; CHECK-LABEL: define internal void @outlined_ir_func_0(
+; CHECK-NEXT:  newFuncRoot:
+; CHECK-NEXT:    br label [[ENTRY_TO_OUTLINE:%.*]]
+; CHECK:       entry_to_outline:
+; CHECK-NEXT:    [[C:%.*]] = add i8 [[TMP0:%.*]], [[TMP1:%.*]]
+; CHECK-NEXT:    [[RET:%.*]] = load i8, i8* [[TMP2:%.*]], align 1
+; CHECK-NEXT:    br label [[ENTRY_AFTER_OUTLINE_EXITSTUB:%.*]]
+; CHECK:       entry_after_outline.exitStub:
+; CHECK-NEXT:    store i8 [[RET]], i8* [[TMP3:%.*]], align 1
+; CHECK-NEXT:    ret void
+;
+;
+; CHECK-LABEL: define internal void @outlined_ir_func_1(
+; CHECK-NEXT:  newFuncRoot:
+; CHECK-NEXT:    br label [[ENTRY_TO_OUTLINE:%.*]]
+; CHECK:       entry_to_outline:
+; CHECK-NEXT:    [[A:%.*]] = load i8, i8* [[TMP0:%.*]], align 1
+; CHECK-NEXT:    [[B:%.*]] = load i8, i8* [[TMP1:%.*]], align 1
+; CHECK-NEXT:    br label [[ENTRY_AFTER_OUTLINE_EXITSTUB:%.*]]
+; CHECK:       entry_after_outline.exitStub:
+; CHECK-NEXT:    store i8 [[A]], i8* [[TMP2:%.*]], align 1
+; CHECK-NEXT:    store i8 [[B]], i8* [[TMP3:%.*]], align 1
+; CHECK-NEXT:    ret void
+;

llvm/test/Transforms/IROutliner/illegal-memcpy.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost < %s | FileCheck %s
+; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost -no-ir-sim-intrinsics < %s | FileCheck %s
 
 ; This test checks that we do not outline memcpy intrinsics since it may require
 ; extra address space checks.
 
 declare void @llvm.memcpy.p0i8.p0i8.i64(i8* nocapture writeonly, i8* nocapture readonly, i64, i1)
 
 define i8 @function1(i8* noalias %s, i8* noalias %d, i64 %len) {
 ; CHECK-LABEL: @function1(

llvm/test/Transforms/IROutliner/illegal-memmove.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost < %s | FileCheck %s
+; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost -no-ir-sim-intrinsics < %s | FileCheck %s
 
 ; This test checks that we do not outline memcpy intrinsics since it may require
 ; extra address space checks.
 
 declare void @llvm.memmove.p0i8.p0i8.i64(i8* nocapture writeonly, i8* nocapture readonly, i64, i1)
 
 define i8 @function1(i8* noalias %s, i8* noalias %d, i64 %len) {
 ; CHECK-LABEL: @function1(

llvm/test/Transforms/IROutliner/illegal-memset.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost < %s | FileCheck %s
+; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost -no-ir-sim-intrinsics < %s | FileCheck %s
 
 ; This test checks that we do not outline memset intrinsics since it requires
 ; extra address space checks.
 
 declare void @llvm.memset.p0i8.i64(i8* nocapture writeonly, i8, i64, i32, i1)
 
 define i64 @function1(i64 %x, i64 %z, i64 %n) {
 ; CHECK-LABEL: @function1(

llvm/test/Transforms/IROutliner/illegal-vaarg.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost < %s | FileCheck %s
+; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost -no-ir-sim-intrinsics < %s | FileCheck %s
 
 ; This test ensures that we do not outline vararg instructions or intrinsics, as
 ; they may cause inconsistencies when outlining.
 
 declare void @llvm.va_start(i8*)
 declare void @llvm.va_copy(i8*, i8*)
 declare void @llvm.va_end(i8*)
 

llvm/test/Transforms/IROutliner/outline-memcpy.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --include-generated-funcs
+; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost < %s | FileCheck %s
+
+; This test checks that we successfully outline identical memcpy instructions.
+
+declare void @llvm.memcpy.p0i8.p0i8.i64(i8* nocapture writeonly, i8* nocapture readonly, i64, i1)
+
+define i8 @function1(i8* noalias %s, i8* noalias %d, i64 %len) {
+entry:
+  %a = load i8, i8* %s
+  %b = load i8, i8* %d
+  call void @llvm.memcpy.p0i8.p0i8.i64(i8* %d, i8* %s, i64 %len, i1 false)
+  %c = add i8 %a, %b
+  %ret = load i8, i8* %s
+  ret i8 %ret
+}
+
+define i8 @function2(i8* noalias %s, i8* noalias %d, i64 %len) {
+entry:
+  %a = load i8, i8* %s
+  %b = load i8, i8* %d
+  call void @llvm.memcpy.p0i8.p0i8.i64(i8* %d, i8* %s, i64 %len, i1 false)
+  %c = add i8 %a, %b
+  %ret = load i8, i8* %s
+  ret i8 %ret
+}
+; CHECK-LABEL: @function1(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[RET_LOC:%.*]] = alloca i8, align 1
+; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 -1, i8* [[RET_LOC]])
+; CHECK-NEXT:    call void @outlined_ir_func_0(i8* [[S:%.*]], i8* [[D:%.*]], i64 [[LEN:%.*]], i8* [[RET_LOC]])
+; CHECK-NEXT:    [[RET_RELOAD:%.*]] = load i8, i8* [[RET_LOC]], align 1
+; CHECK-NEXT:    call void @llvm.lifetime.end.p0i8(i64 -1, i8* [[RET_LOC]])
+; CHECK-NEXT:    ret i8 [[RET_RELOAD]]
+;
+;
+; CHECK-LABEL: @function2(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[RET_LOC:%.*]] = alloca i8, align 1
+; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 -1, i8* [[RET_LOC]])
+; CHECK-NEXT:    call void @outlined_ir_func_0(i8* [[S:%.*]], i8* [[D:%.*]], i64 [[LEN:%.*]], i8* [[RET_LOC]])
+; CHECK-NEXT:    [[RET_RELOAD:%.*]] = load i8, i8* [[RET_LOC]], align 1
+; CHECK-NEXT:    call void @llvm.lifetime.end.p0i8(i64 -1, i8* [[RET_LOC]])
+; CHECK-NEXT:    ret i8 [[RET_RELOAD]]
+;
+;
+; CHECK: define internal void @outlined_ir_func_0(
+; CHECK-NEXT:  newFuncRoot:
+; CHECK-NEXT:    br label [[ENTRY_TO_OUTLINE:%.*]]
+; CHECK:       entry_to_outline:
+; CHECK-NEXT:    [[A:%.*]] = load i8, i8* [[TMP0:%.*]], align 1
+; CHECK-NEXT:    [[B:%.*]] = load i8, i8* [[TMP1:%.*]], align 1
+; CHECK-NEXT:    call void @llvm.memcpy.p0i8.p0i8.i64(i8* [[TMP1]], i8* [[TMP0]], i64 [[TMP2:%.*]], i1 false)
+; CHECK-NEXT:    [[C:%.*]] = add i8 [[A]], [[B]]
+; CHECK-NEXT:    [[RET:%.*]] = load i8, i8* [[TMP0]], align 1
+; CHECK-NEXT:    br label [[ENTRY_AFTER_OUTLINE_EXITSTUB:%.*]]
+; CHECK:       entry_after_outline.exitStub:
+; CHECK-NEXT:    store i8 [[RET]], i8* [[TMP3:%.*]], align 1
+; CHECK-NEXT:    ret void
+;

llvm/test/Transforms/IROutliner/outline-memmove.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --include-generated-funcs
+; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost < %s | FileCheck %s
+
+; This test checks that we sucecssfully outline identical memmove instructions.
+
+declare void @llvm.memmove.p0i8.p0i8.i64(i8* nocapture writeonly, i8* nocapture readonly, i64, i1)
+
+define i8 @function1(i8* noalias %s, i8* noalias %d, i64 %len) {
+entry:
+  %a = load i8, i8* %s
+  %b = load i8, i8* %d
+  call void @llvm.memmove.p0i8.p0i8.i64(i8* %d, i8* %s, i64 %len, i1 false)
+  %c = add i8 %a, %b
+  %ret = load i8, i8* %s
+  ret i8 %ret
+}
+
+define i8 @function2(i8* noalias %s, i8* noalias %d, i64 %len) {
+entry:
+  %a = load i8, i8* %s
+  %b = load i8, i8* %d
+  call void @llvm.memmove.p0i8.p0i8.i64(i8* %d, i8* %s, i64 %len, i1 false)
+  %c = add i8 %a, %b
+  %ret = load i8, i8* %s
+  ret i8 %ret
+}
+; CHECK-LABEL: @function1(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[RET_LOC:%.*]] = alloca i8, align 1
+; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 -1, i8* [[RET_LOC]])
+; CHECK-NEXT:    call void @outlined_ir_func_0(i8* [[S:%.*]], i8* [[D:%.*]], i64 [[LEN:%.*]], i8* [[RET_LOC]])
+; CHECK-NEXT:    [[RET_RELOAD:%.*]] = load i8, i8* [[RET_LOC]], align 1
+; CHECK-NEXT:    call void @llvm.lifetime.end.p0i8(i64 -1, i8* [[RET_LOC]])
+; CHECK-NEXT:    ret i8 [[RET_RELOAD]]
+;
+;
+; CHECK-LABEL: @function2(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[RET_LOC:%.*]] = alloca i8, align 1
+; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 -1, i8* [[RET_LOC]])
+; CHECK-NEXT:    call void @outlined_ir_func_0(i8* [[S:%.*]], i8* [[D:%.*]], i64 [[LEN:%.*]], i8* [[RET_LOC]])
+; CHECK-NEXT:    [[RET_RELOAD:%.*]] = load i8, i8* [[RET_LOC]], align 1
+; CHECK-NEXT:    call void @llvm.lifetime.end.p0i8(i64 -1, i8* [[RET_LOC]])
+; CHECK-NEXT:    ret i8 [[RET_RELOAD]]
+;
+;
+; CHECK: define internal void @outlined_ir_func_0(
+; CHECK-NEXT:  newFuncRoot:
+; CHECK-NEXT:    br label [[ENTRY_TO_OUTLINE:%.*]]
+; CHECK:       entry_to_outline:
+; CHECK-NEXT:    [[A:%.*]] = load i8, i8* [[TMP0:%.*]], align 1
+; CHECK-NEXT:    [[B:%.*]] = load i8, i8* [[TMP1:%.*]], align 1
+; CHECK-NEXT:    call void @llvm.memmove.p0i8.p0i8.i64(i8* [[TMP1]], i8* [[TMP0]], i64 [[TMP2:%.*]], i1 false)
+; CHECK-NEXT:    [[C:%.*]] = add i8 [[A]], [[B]]
+; CHECK-NEXT:    [[RET:%.*]] = load i8, i8* [[TMP0]], align 1
+; CHECK-NEXT:    br label [[ENTRY_AFTER_OUTLINE_EXITSTUB:%.*]]
+; CHECK:       entry_after_outline.exitStub:
+; CHECK-NEXT:    store i8 [[RET]], i8* [[TMP3:%.*]], align 1
+; CHECK-NEXT:    ret void
+;

llvm/test/Transforms/IROutliner/outline-memset.ll

-; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --include-generated-funcs
 ; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost < %s | FileCheck %s
 
-; This test checks that we do not outline memset intrinsics since it requires
-; extra address space checks.
+; This test checks that we successfully outline identical memset instructions.
 
 declare void @llvm.memset.p0i8.i64(i8* nocapture writeonly, i8, i64, i32, i1)
 
 define i64 @function1(i64 %x, i64 %z, i64 %n) {
-; CHECK-LABEL: @function1(
-; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[POOL:%.*]] = alloca [59 x i64], align 4
-; CHECK-NEXT:    [[TMP:%.*]] = bitcast [59 x i64]* [[POOL]] to i8*
-; CHECK-NEXT:    call void @llvm.memset.p0i8.i64(i8* align 4 [[TMP]], i8 0, i64 236, i1 false)
-; CHECK-NEXT:    call void @outlined_ir_func_0(i64 [[N:%.*]], i64 [[X:%.*]], i64 [[Z:%.*]])
-; CHECK-NEXT:    ret i64 0
-;
 entry:
   %pool = alloca [59 x i64], align 4
   %tmp = bitcast [59 x i64]* %pool to i8*
   call void @llvm.memset.p0i8.i64(i8* nonnull %tmp, i8 0, i64 236, i32 4, i1 false)
   %cmp3 = icmp eq i64 %n, 0
   %a = add i64 %x, %z
   %c = add i64 %x, %z
   ret i64 0
 }
 
 define i64 @function2(i64 %x, i64 %z, i64 %n) {
-; CHECK-LABEL: @function2(
-; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[POOL:%.*]] = alloca [59 x i64], align 4
-; CHECK-NEXT:    [[TMP:%.*]] = bitcast [59 x i64]* [[POOL]] to i8*
-; CHECK-NEXT:    call void @llvm.memset.p0i8.i64(i8* align 4 [[TMP]], i8 0, i64 236, i1 false)
-; CHECK-NEXT:    call void @outlined_ir_func_0(i64 [[N:%.*]], i64 [[X:%.*]], i64 [[Z:%.*]])
-; CHECK-NEXT:    ret i64 0
-;
 entry:
   %pool = alloca [59 x i64], align 4
   %tmp = bitcast [59 x i64]* %pool to i8*
   call void @llvm.memset.p0i8.i64(i8* nonnull %tmp, i8 0, i64 236, i32 4, i1 false)
   %cmp3 = icmp eq i64 %n, 0
   %a = add i64 %x, %z
   %c = add i64 %x, %z
   ret i64 0
 }
+; CHECK-LABEL: @function1(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[POOL:%.*]] = alloca [59 x i64], align 4
+; CHECK-NEXT:    call void @outlined_ir_func_0([59 x i64]* [[POOL]], i64 [[N:%.*]], i64 [[X:%.*]], i64 [[Z:%.*]])
+; CHECK-NEXT:    ret i64 0
+;
+;
+; CHECK-LABEL: @function2(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[POOL:%.*]] = alloca [59 x i64], align 4
+; CHECK-NEXT:    call void @outlined_ir_func_0([59 x i64]* [[POOL]], i64 [[N:%.*]], i64 [[X:%.*]], i64 [[Z:%.*]])
+; CHECK-NEXT:    ret i64 0
+;
+;
+; CHECK: define internal void @outlined_ir_func_0(
+; CHECK-NEXT:  newFuncRoot:
+; CHECK-NEXT:    br label [[ENTRY_TO_OUTLINE:%.*]]
+; CHECK:       entry_to_outline:
+; CHECK-NEXT:    [[TMP:%.*]] = bitcast [59 x i64]* [[TMP0:%.*]] to i8*
+; CHECK-NEXT:    call void @llvm.memset.p0i8.i64(i8* align 4 [[TMP]], i8 0, i64 236, i1 false)
+; CHECK-NEXT:    [[CMP3:%.*]] = icmp eq i64 [[TMP1:%.*]], 0
+; CHECK-NEXT:    [[A:%.*]] = add i64 [[TMP2:%.*]], [[TMP3:%.*]]
+; CHECK-NEXT:    [[C:%.*]] = add i64 [[TMP2]], [[TMP3]]
+; CHECK-NEXT:    br label [[ENTRY_AFTER_OUTLINE_EXITSTUB:%.*]]
+; CHECK:       entry_after_outline.exitStub:
+; CHECK-NEXT:    ret void
+;

llvm/test/Transforms/IROutliner/outline-vaarg-intrinsic.ll

-; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --include-generated-funcs
 ; RUN: opt -S -verify -iroutliner -ir-outlining-no-cost < %s | FileCheck %s
 
-; This test ensures that we do not outline vararg instructions or intrinsics, as
-; they may cause inconsistencies when outlining.
+; This test checks that we sucessfully outline identical memcpy var arg
+; intrinsics, but not the var arg instruction itself.
 
 declare void @llvm.va_start(i8*)
 declare void @llvm.va_copy(i8*, i8*)
 declare void @llvm.va_end(i8*)
 
 define i32 @func1(i32 %a, double %b, i8* %v, ...) nounwind {
-; CHECK-LABEL: @func1(
-; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[AP1_LOC:%.*]] = alloca i8*, align 8
-; CHECK-NEXT:    [[A_ADDR:%.*]] = alloca i32, align 4
-; CHECK-NEXT:    [[B_ADDR:%.*]] = alloca double, align 8
-; CHECK-NEXT:    [[AP:%.*]] = alloca i8*, align 4
-; CHECK-NEXT:    [[C:%.*]] = alloca i32, align 4
-; CHECK-NEXT:    [[LT_CAST:%.*]] = bitcast i8** [[AP1_LOC]] to i8*
-; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 -1, i8* [[LT_CAST]])
-; CHECK-NEXT:    call void @outlined_ir_func_0(i32 [[A:%.*]], i32* [[A_ADDR]], double [[B:%.*]], double* [[B_ADDR]], i8** [[AP]], i8** [[AP1_LOC]])
-; CHECK-NEXT:    [[AP1_RELOAD:%.*]] = load i8*, i8** [[AP1_LOC]], align 8
-; CHECK-NEXT:    call void @llvm.lifetime.end.p0i8(i64 -1, i8* [[LT_CAST]])
-; CHECK-NEXT:    call void @llvm.va_start(i8* [[AP1_RELOAD]])
-; CHECK-NEXT:    [[TMP0:%.*]] = va_arg i8** [[AP]], i32
-; CHECK-NEXT:    call void @llvm.va_copy(i8* [[V:%.*]], i8* [[AP1_RELOAD]])
-; CHECK-NEXT:    call void @llvm.va_end(i8* [[AP1_RELOAD]])
-; CHECK-NEXT:    store i32 [[TMP0]], i32* [[C]], align 4
-; CHECK-NEXT:    [[TMP:%.*]] = load i32, i32* [[C]], align 4
-; CHECK-NEXT:    ret i32 [[TMP]]
-;
 entry:
   %a.addr = alloca i32, align 4
   %b.addr = alloca double, align 8
   %ap = alloca i8*, align 4
   %c = alloca i32, align 4
   store i32 %a, i32* %a.addr, align 4
   store double %b, double* %b.addr, align 8
   %ap1 = bitcast i8** %ap to i8*
   call void @llvm.va_start(i8* %ap1)
   %0 = va_arg i8** %ap, i32
   call void @llvm.va_copy(i8* %v, i8* %ap1)
   call void @llvm.va_end(i8* %ap1)
   store i32 %0, i32* %c, align 4
   %tmp = load i32, i32* %c, align 4
   ret i32 %tmp
 }
 
 define i32 @func2(i32 %a, double %b, i8* %v, ...) nounwind {
-; CHECK-LABEL: @func2(
-; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[AP1_LOC:%.*]] = alloca i8*, align 8
-; CHECK-NEXT:    [[A_ADDR:%.*]] = alloca i32, align 4
-; CHECK-NEXT:    [[B_ADDR:%.*]] = alloca double, align 8
-; CHECK-NEXT:    [[AP:%.*]] = alloca i8*, align 4
-; CHECK-NEXT:    [[C:%.*]] = alloca i32, align 4
-; CHECK-NEXT:    [[LT_CAST:%.*]] = bitcast i8** [[AP1_LOC]] to i8*
-; CHECK-NEXT:    call void @llvm.lifetime.start.p0i8(i64 -1, i8* [[LT_CAST]])
-; CHECK-NEXT:    call void @outlined_ir_func_0(i32 [[A:%.*]], i32* [[A_ADDR]], double [[B:%.*]], double* [[B_ADDR]], i8** [[AP]], i8** [[AP1_LOC]])
-; CHECK-NEXT:    [[AP1_RELOAD:%.*]] = load i8*, i8** [[AP1_LOC]], align 8
-; CHECK-NEXT:    call void @llvm.lifetime.end.p0i8(i64 -1, i8* [[LT_CAST]])
-; CHECK-NEXT:    call void @llvm.va_start(i8* [[AP1_RELOAD]])
-; CHECK-NEXT:    [[TMP0:%.*]] = va_arg i8** [[AP]], i32
-; CHECK-NEXT:    call void @llvm.va_copy(i8* [[V:%.*]], i8* [[AP1_RELOAD]])
-; CHECK-NEXT:    call void @llvm.va_end(i8* [[AP1_RELOAD]])
-; CHECK-NEXT:    store i32 [[TMP0]], i32* [[C]], align 4
-; CHECK-NEXT:    [[AP2:%.*]] = bitcast i8** [[AP]] to i8*
-; CHECK-NEXT:    [[TMP:%.*]] = load i32, i32* [[C]], align 4
-; CHECK-NEXT:    ret i32 [[TMP]]
-;
 entry:
   %a.addr = alloca i32, align 4
   %b.addr = alloca double, align 8
   %ap = alloca i8*, align 4
   %c = alloca i32, align 4
   store i32 %a, i32* %a.addr, align 4
   store double %b, double* %b.addr, align 8
   %ap1 = bitcast i8** %ap to i8*
   call void @llvm.va_start(i8* %ap1)
   %0 = va_arg i8** %ap, i32
   call void @llvm.va_copy(i8* %v, i8* %ap1)
   call void @llvm.va_end(i8* %ap1)
   store i32 %0, i32* %c, align 4
   %ap2 = bitcast i8** %ap to i8*
   %tmp = load i32, i32* %c, align 4
   ret i32 %tmp
 }
+; CHECK-LABEL: @func1(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[A_ADDR:%.*]] = alloca i32, align 4
+; CHECK-NEXT:    [[B_ADDR:%.*]] = alloca double, align 8
+; CHECK-NEXT:    [[AP:%.*]] = alloca i8*, align 4
+; CHECK-NEXT:    [[C:%.*]] = alloca i32, align 4
+; CHECK-NEXT:    store i32 [[A:%.*]], i32* [[A_ADDR]], align 4
+; CHECK-NEXT:    store double [[B:%.*]], double* [[B_ADDR]], align 8
+; CHECK-NEXT:    [[AP1:%.*]] = bitcast i8** [[AP]] to i8*
+; CHECK-NEXT:    call void @llvm.va_start(i8* [[AP1]])
+; CHECK-NEXT:    [[TMP0:%.*]] = va_arg i8** [[AP]], i32
+; CHECK-NEXT:    call void @outlined_ir_func_0(i8* [[V:%.*]], i8* [[AP1]], i32 [[TMP0]], i32* [[C]])
+; CHECK-NEXT:    [[TMP:%.*]] = load i32, i32* [[C]], align 4
+; CHECK-NEXT:    ret i32 [[TMP]]
+;
+;
+; CHECK-LABEL: @func2(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[A_ADDR:%.*]] = alloca i32, align 4
+; CHECK-NEXT:    [[B_ADDR:%.*]] = alloca double, align 8
+; CHECK-NEXT:    [[AP:%.*]] = alloca i8*, align 4
+; CHECK-NEXT:    [[C:%.*]] = alloca i32, align 4
+; CHECK-NEXT:    store i32 [[A:%.*]], i32* [[A_ADDR]], align 4
+; CHECK-NEXT:    store double [[B:%.*]], double* [[B_ADDR]], align 8
+; CHECK-NEXT:    [[AP1:%.*]] = bitcast i8** [[AP]] to i8*
+; CHECK-NEXT:    call void @llvm.va_start(i8* [[AP1]])
+; CHECK-NEXT:    [[TMP0:%.*]] = va_arg i8** [[AP]], i32
+; CHECK-NEXT:    call void @outlined_ir_func_0(i8* [[V:%.*]], i8* [[AP1]], i32 [[TMP0]], i32* [[C]])
+; CHECK-NEXT:    [[AP2:%.*]] = bitcast i8** [[AP]] to i8*
+; CHECK-NEXT:    [[TMP:%.*]] = load i32, i32* [[C]], align 4
+; CHECK-NEXT:    ret i32 [[TMP]]
+;
+;
+; CHECK: define internal void @outlined_ir_func_0(
+; CHECK-NEXT:  newFuncRoot:
+; CHECK-NEXT:    br label [[ENTRY_TO_OUTLINE:%.*]]
+; CHECK:       entry_to_outline:
+; CHECK-NEXT:    call void @llvm.va_copy(i8* [[TMP0:%.*]], i8* [[TMP1:%.*]])
+; CHECK-NEXT:    call void @llvm.va_end(i8* [[TMP1]])
+; CHECK-NEXT:    store i32 [[TMP2:%.*]], i32* [[TMP3:%.*]], align 4
+; CHECK-NEXT:    br label [[ENTRY_AFTER_OUTLINE_EXITSTUB:%.*]]
+; CHECK:       entry_after_outline.exitStub:
+; CHECK-NEXT:    ret void
+;

llvm/unittests/Analysis/IRSimilarityIdentifierTest.cpp

TEST(IRInstructionMapper, CleanuppadIllegal) {
   ASSERT_EQ(InstrList.size(), UnsignedVec.size());
   ASSERT_EQ(UnsignedVec.size(), static_cast<unsigned>(0));
 }
 
 // The following three instructions are memory transfer and setting based, which
 // are considered illegal since is extra checking needed to handle the address
 // space checking.
 
-// Checks that a memset instruction is mapped to an illegal value.
+// Checks that a memset instruction is mapped to an illegal value when
+// specified.
 TEST(IRInstructionMapper, MemSetIllegal) {
   StringRef ModuleString = R"(
   declare void @llvm.memset.p0i8.i64(i8* nocapture writeonly, i8, i64, i32, i1)
 
   define i64 @function(i64 %x, i64 %z, i64 %n) {
   entry:
     %pool = alloca [59 x i64], align 4
     %tmp = bitcast [59 x i64]* %pool to i8*
     call void @llvm.memset.p0i8.i64(i8* nonnull %tmp, i8 0, i64 236, i32 4, i1 false)
     %cmp3 = icmp eq i64 %n, 0
     %a = add i64 %x, %z
     %c = add i64 %x, %z
     ret i64 0
   })";
   LLVMContext Context;
   std::unique_ptr<Module> M = makeLLVMModule(Context, ModuleString);
 
   std::vector<IRInstructionData *> InstrList;
   std::vector<unsigned> UnsignedVec;
 
   SpecificBumpPtrAllocator<IRInstructionData> InstDataAllocator;
   SpecificBumpPtrAllocator<IRInstructionDataList> IDLAllocator;
   IRInstructionMapper Mapper(&InstDataAllocator, &IDLAllocator);
+  Mapper.InstClassifier.EnableIntrinsics = false;
   getVectors(*M, Mapper, InstrList, UnsignedVec);
 
   ASSERT_EQ(InstrList.size(), UnsignedVec.size());
   ASSERT_EQ(UnsignedVec.size(), static_cast<unsigned>(7));
   ASSERT_TRUE(UnsignedVec[2] < UnsignedVec[0]);
 }
 
-// Checks that a memcpy instruction is mapped to an illegal value.
+// Checks that a memcpy instruction is mapped to an illegal value  when
+// specified.
 TEST(IRInstructionMapper, MemCpyIllegal) {
   StringRef ModuleString = R"(
   declare void @llvm.memcpy.p0i8.i64(i8* nocapture writeonly, i8, i64, i32, i1)
 
   define i64 @function(i64 %x, i64 %z, i64 %n) {
   entry:
     %pool = alloca [59 x i64], align 4
     %tmp = bitcast [59 x i64]* %pool to i8*
     call void @llvm.memcpy.p0i8.i64(i8* nonnull %tmp, i8 0, i64 236, i32 4, i1 false)
     %cmp3 = icmp eq i64 %n, 0
     %a = add i64 %x, %z
     %c = add i64 %x, %z
     ret i64 0
   })";
   LLVMContext Context;
   std::unique_ptr<Module> M = makeLLVMModule(Context, ModuleString);
 
   std::vector<IRInstructionData *> InstrList;
   std::vector<unsigned> UnsignedVec;
 
   SpecificBumpPtrAllocator<IRInstructionData> InstDataAllocator;
   SpecificBumpPtrAllocator<IRInstructionDataList> IDLAllocator;
   IRInstructionMapper Mapper(&InstDataAllocator, &IDLAllocator);
+  Mapper.InstClassifier.EnableIntrinsics = false;
   getVectors(*M, Mapper, InstrList, UnsignedVec);
 
   ASSERT_EQ(InstrList.size(), UnsignedVec.size());
   ASSERT_EQ(UnsignedVec.size(), static_cast<unsigned>(7));
   ASSERT_GT(UnsignedVec[2], UnsignedVec[3]);
   ASSERT_LT(UnsignedVec[2], UnsignedVec[0]);
 }
 
-// Checks that a memmove instruction is mapped to an illegal value.
+// Checks that a memmove instruction is mapped to an illegal value  when
+// specified.
 TEST(IRInstructionMapper, MemMoveIllegal) {
   StringRef ModuleString = R"(
   declare void @llvm.memmove.p0i8.i64(i8* nocapture writeonly, i8, i64, i32, i1)
 
   define i64 @function(i64 %x, i64 %z, i64 %n) {
   entry:
     %pool = alloca [59 x i64], align 4
     %tmp = bitcast [59 x i64]* %pool to i8*
     call void @llvm.memmove.p0i8.i64(i8* nonnull %tmp, i8 0, i64 236, i32 4, i1 false)
     %cmp3 = icmp eq i64 %n, 0
     %a = add i64 %x, %z
     %c = add i64 %x, %z
     ret i64 0
   })";
   LLVMContext Context;
   std::unique_ptr<Module> M = makeLLVMModule(Context, ModuleString);
 
   std::vector<IRInstructionData *> InstrList;
   std::vector<unsigned> UnsignedVec;
 
   SpecificBumpPtrAllocator<IRInstructionData> InstDataAllocator;
   SpecificBumpPtrAllocator<IRInstructionDataList> IDLAllocator;
   IRInstructionMapper Mapper(&InstDataAllocator, &IDLAllocator);
+  Mapper.InstClassifier.EnableIntrinsics = false;
   getVectors(*M, Mapper, InstrList, UnsignedVec);
 
   ASSERT_EQ(InstrList.size(), UnsignedVec.size());
   ASSERT_EQ(UnsignedVec.size(), static_cast<unsigned>(7));
   ASSERT_LT(UnsignedVec[2], UnsignedVec[0]);
 }
 
+// Checks that mem* instructions are mapped to an legal value when not
+// specified, and that all the intrinsics are marked differently.
+TEST(IRInstructionMapper, MemOpsLegal) {
+  StringRef ModuleString = R"(
+  declare void @llvm.memmove.p0i8.i64(i8* nocapture writeonly, i8, i64, i32, i1)
+  declare void @llvm.memcpy.p0i8.i64(i8* nocapture writeonly, i8, i64, i32, i1)
+  declare void @llvm.memset.p0i8.i64(i8* nocapture writeonly, i8, i64, i32, i1)
+
+  define i64 @function(i64 %x, i64 %z, i64 %n) {
+  entry:
+    %pool = alloca [59 x i64], align 4
+    %tmp = bitcast [59 x i64]* %pool to i8*
+    call void @llvm.memmove.p0i8.i64(i8* nonnull %tmp, i8 0, i64 236, i32 4, i1 false)
+    call void @llvm.memcpy.p0i8.i64(i8* nonnull %tmp, i8 0, i64 236, i32 4, i1 false)
+    call void @llvm.memset.p0i8.i64(i8* nonnull %tmp, i8 0, i64 236, i32 4, i1 false)
+    %cmp3 = icmp eq i64 %n, 0
+    %a = add i64 %x, %z
+    %c = add i64 %x, %z
+    ret i64 0
+  })";
+  LLVMContext Context;
+  std::unique_ptr<Module> M = makeLLVMModule(Context, ModuleString);
+
+  std::vector<IRInstructionData *> InstrList;
+  std::vector<unsigned> UnsignedVec;
+
+  SpecificBumpPtrAllocator<IRInstructionData> InstDataAllocator;
+  SpecificBumpPtrAllocator<IRInstructionDataList> IDLAllocator;
+  IRInstructionMapper Mapper(&InstDataAllocator, &IDLAllocator);
+  Mapper.InstClassifier.EnableIntrinsics = true;
+  getVectors(*M, Mapper, InstrList, UnsignedVec);
+
+  ASSERT_EQ(InstrList.size(), UnsignedVec.size());
+  ASSERT_EQ(UnsignedVec.size(), static_cast<unsigned>(9));
+  ASSERT_LT(UnsignedVec[2], UnsignedVec[3]);
+  ASSERT_LT(UnsignedVec[3], UnsignedVec[4]);
+  ASSERT_LT(UnsignedVec[4], UnsignedVec[5]);
+}
+
 // Checks that a variable argument instructions are mapped to an illegal value.
 // We exclude variable argument instructions since variable arguments
 // requires extra checking of the argument list.
 TEST(IRInstructionMapper, VarArgsIllegal) {
   StringRef ModuleString = R"(
   declare void @llvm.va_start(i8*)
   declare void @llvm.va_copy(i8*, i8*)
   declare void @llvm.va_end(i8*)
TEST(IRInstructionMapper, VarArgsIllegal) {
   std::unique_ptr<Module> M = makeLLVMModule(Context, ModuleString);
 
   std::vector<IRInstructionData *> InstrList;
   std::vector<unsigned> UnsignedVec;
 
   SpecificBumpPtrAllocator<IRInstructionData> InstDataAllocator;
   SpecificBumpPtrAllocator<IRInstructionDataList> IDLAllocator;
   IRInstructionMapper Mapper(&InstDataAllocator, &IDLAllocator);
+  Mapper.InstClassifier.EnableIntrinsics = false;
   getVectors(*M, Mapper, InstrList, UnsignedVec);
 
   ASSERT_EQ(InstrList.size(), UnsignedVec.size());
   ASSERT_EQ(UnsignedVec.size(), static_cast<unsigned>(17));
   ASSERT_TRUE(UnsignedVec[7] < UnsignedVec[0]);
   ASSERT_TRUE(UnsignedVec[13] < UnsignedVec[10]);
   ASSERT_TRUE(UnsignedVec[16] < UnsignedVec[13]);
 }