Add a pass to lower is.constant and objectsize intrinsics

[joerg]

Summary

This pass lowers the remaining is.constant and objectsize intrinsics. This moves the lowering from the codegen-prepare pass and the fallback in the SDAG/GlobalISel/FastISel layers.

The API for replaceAndRecursivelySimplify is extended to provide a list of un-modified instructions. Those are scanned for conditional branches with now invariant condition.

[joerg]

Avoid goto. Create new BranchInst instead of modifying in-place. Update tests to reflect changes. Move most of the x86 is-constant test to generic.

[echristo]

Couple of random comments, haven't looked at most of it yet.

include/llvm/Analysis/InstructionSimplify.h

258 ↗

(On Diff #212070)

Really would like to avoid boolean arguments like this.

lib/Analysis/InstructionSimplify.cpp

5047 ↗

(On Diff #212070)

This seems to return nullptr everywhere?

[arsenm]

lib/Transforms/Scalar/LowerIsConstantIntrinsic.cpp

51 ↗

(On Diff #212070)

Should early exit if there are is no is_constant declaration, or just visit uses of it rather than search the whole function for a rarely used intrinsic?

[void]

Feel free to hijack my revision. :-) I may not get back to my change soon.

[joerg]

include/llvm/Analysis/InstructionSimplify.h

258 ↗

(On Diff #212070)

Right, this is the one big part of the change I am absolutely not sure about. I was considering passing a pointer to a vector down to replaceAndRecursivelySimplify. The contract would be to return the work list of all modified uses. The caller can then iterate the list and see if any branch instructions are affected.

lib/Analysis/InstructionSimplify.cpp

5047 ↗

(On Diff #212070)

It does, trying to preserve the style of the other functions here. The question might become irrelevant when going with the work list idea.

lib/Transforms/Scalar/LowerIsConstantIntrinsic.cpp

51 ↗

(On Diff #212070)

As mentioned on IRC, llvm.is.constant is a family of intrinsics depending on the type of the argument. There doesn't seem to be an easy way to check if a given template-like intrinsic is used.

[joerg]

Replace boolean argument to replaceAndRecursivelySimplify with optional vector of un-modified instructions. This simplifies the API change significantly and allows other potential use cases. Redo the restart handling. After a successful simplification step, restart the current BB only, but always do another full pass of the function.

[jyknight]

I think the pass needs to handle the removal of any remaining llvm.objectsize, as well, so that llvm.is.constant(llvm.objectsize(...)) continues to return true -- even if the object size cannot be determined.

[jyknight]

With this new pass, I think we should be able to remove the code handling the objectsize and is.constant instrinsics in CodegenPrepare, FastISel, SelectionDAGBuilder.cpp, and IRTranslator. Which sure is a nice cleanup.

test/Transforms/LowerIsConstant/is-constant.ll

1 ↗

(On Diff #212341)

While this is a more targeted run, the old asserts are checking what _actually_ needs to continue to work.

E.g. I'd be more confident that my suggested removal of the handling code elsewhere was correct if this test still ran the original 4 tests.

[joerg]

Update PHI nodes in disconnected block.

[joerg]

Generalize slightly to also cover llvm.objectsize which has very similar constraints.

[chandlerc]

Generally, I do like the approach. Two high level comments:

First, I don't think we can crash if these things reach lower layers. I think we should retain the logic to just fold them to a constant if somehow they show up. Anything else makes things weird -- the IR is valid, passes the verifier, but crashes? We want to support fuzzing and bisecting and such and so it should be possible to rip this pass out and still have things lower successfully. This pass should just be a much more *advanced* way of lowering these intrinsics. And yes, I understand that in some bizarre cases failing to do this "optimization" will result in inline assembly that cannot be emitted or some such. While I find that really frustrating as well, I still think it is better than making even more boring uses of these intrinsics hostile to things like fuzzers.

Second, I don't think you need the fancy approach you're taking to find the branches that need simplification. How about the following approach:

1. Walk every instruction and if it is one of these instrinsics, fold it and then recursively simplify.

2. If we simplify any intrinsics, but after simplifying *all* of them found in the function, walk every terminator in every block and if it is a conditional branch with a constant condition, fold the branch.

I don't think this will be too wasteful, and seems much simpler.

[joerg]

For the first part, I was actually asked to do that. I don't mind either way.

For the second part, it seems to lead to inconsistent behavior. If we want to go down that road, it seems to me that this behavior should either be a separate branch or be folded into the existing unreachable block removal pass.

[joerg]

Looking a bit more into the details. Chandler, you've originally suggested going with the LowerAtomic route and that actually does create code that fails the SDAG lowering if the pass is skipped, e.g. on ARM.
The second part is currently overlapping with the CodeGenPrepare pass. I can cleanup the implementation somewhat by reusing the same functionality as that pass is using OR I could factor out a minimal branch for doing the constant folding optimization from CodeGenPrepare as a general branch that is included in the pass chain for -O0, e.g. instead of the more general CodeGenPrepare pass. The main difference is that the non-optimized build would not get any recursive folding from PHI-simplification, but I think that's fine for the original use case. It would also not get the block merging, but again, that seems to be fine for the constraints.

[chandlerc]

In D65280#1614752, @joerg wrote:

Looking a bit more into the details. Chandler, you've originally suggested going with the LowerAtomic route and that actually does create code that fails the SDAG lowering if the pass is skipped, e.g. on ARM.

Sorry, I had forgotten about that discussion. Thank you for digging it up and reminding me to be consistent with my past self.

If this pass is going into the llc pipeline like the lower atomic pass, you're right it totally makes sense to fail hard in SDAG lowering. Sorry for the confusion again.

The second part is currently overlapping with the CodeGenPrepare pass. I can cleanup the implementation somewhat by reusing the same functionality as that pass is using OR I could factor out a minimal branch for doing the constant folding optimization from CodeGenPrepare as a general branch that is included in the pass chain for -O0, e.g. instead of the more general CodeGenPrepare pass. The main difference is that the non-optimized build would not get any recursive folding from PHI-simplification, but I think that's fine for the original use case. It would also not get the block merging, but again, that seems to be fine for the constraints.

I'm not really sure I understand the problem?

This pass needs to run (at least) in the same general area as the atomics lowering for the same reasons (it would be part of SDAG but it can't be due to CFG mutation). It might optionally be run in other places of course.

I think you have all the code you need in this patch already. I'm just suggesting a different strategy for applying it to the IR that I think will be simpler (and not require any changes to instsimplify). Maybe my suggestion isn't clear? If so I can try to clarify....

[void]

This PR is possibly related to the original issue: https://bugs.llvm.org/show_bug.cgi?id=42956

[chandlerc]

Thanks to Joerg for some useful discussion on IRC -- there was a concern I hadn't thought about that is exactly right: we somewhat want this pass to minimally disrupt things but also to be reasonably self contained.

Based on the discussion, I think the patch can still be simplified a bit though (although not as much as I had originally suggested). Notably, the additional API for inst simplify makes a lot more sense to me now.

I'd suggest thinking about this in three phases.

First, simplify the instructions to constants. Here, you would keep the new API to extract a list of potentially relevant instructions. After each simplification, scan this and append any branches or switches to a list of terminators needing an update. (May want it to be a set vector so you don't duplicate.) But don't actually update the CFG in any way during this phase.

Second, rewrite the branches and switches collected in the first phase, collecting basic blocks that might be made unreachable in another SetVector. This just mutates the CFG but doesn't delete any blocks and so doesn't make any of the instructions invalid to visit. You should be able to collect a batch of domtree updates during this phase as well without applying them eagerly.

Third, apply the domtree updates (so you have precise reachability) and remove any blocks from the list that are still reachable. Hand the remaining blocks to the DeleteBasicBlocks utility to completely remove them. This will remove the implicit dependence between this pass and the later codegen passes which I think is much cleaner.

I think that will still remove the need to iterate, and will enhance this to actually fully remove the blocks orphaned by the folding of these branches (but *only* those blocks). Thoughts?

[joerg]

Simplify the pass logic. First round will update the predecessor links and note if any block is orphaned, a second round will remove unreachable blocks if necessary.
The pass can be further refined to use and update DominatorTree and AssumptionCache incrementally, but this should be functionally complete now. It will not handle more complex cases like orphaned loops, but I don't think those are commonly used with is.constant or objectsize conditions either.
The pass will now scan every BB once, but fall back to the start of a BB of recursive removal invalidates the iterator. This seems to be the strictest form I can manage.

[joerg]

Hook up a second instance of the pass after the Float To Int pass for optimized builds. This is after the initial loop transforms, so it can profit from some unrolling, but it is before vectorization. The late run of the pass should is kept for now and ensures any potentially added variants are still dropped before SDAG.

[joerg]

Chandler, are you OK with getting the InstructionSimplify.h part in now, so that it can be merged into 9.0 and the rest follow separately?

[chandlerc]

In D65280#1647026, @joerg wrote:

Chandler, are you OK with getting the InstructionSimplify.h part in now, so that it can be merged into 9.0 and the rest follow separately?

Can we not get the entire thing merged? I'd really like that... I think the patch is actually really close. I have a bunch of comments below but they're all pretty boring in reality.

include/llvm/Analysis/InstructionSimplify.h

265–266 ↗

(On Diff #217160)

I'd call this UnsimplifiedUsers.

lib/Analysis/InstructionSimplify.cpp

5225–5226 ↗

(On Diff #217160)

I'd make this a bit more precise...

"Recursively visited users which do not themselves simplify are ..."

5235 ↗

(On Diff #217160)

If this is what clang-format did with this... I am sad. If not, clang-format?

lib/Transforms/Scalar/LowerConstantIntrinsics.cpp

41 ↗

(On Diff #217160)

No #include after usings and statistics and such please.

45–51 ↗

(On Diff #217160)

Seems simpler to write as:

return isa<Constant>(Op) ? ConstantInt::getTrue(II->getType())
                         : ConstantInt::getFalse(II->getType());

54–55 ↗

(On Diff #217160)

clang-format please.

I'd also call this replaceConditionalBranchesOnConstant.

113–117 ↗

(On Diff #217160)

As I tried to indicate before, this can be even simpler AFAICT.

I think you should keep a single worklist of unsimplified users. Then you should just do the recursive inst simplify here using the same worklist each time. You'll want to change the loop to go in RPO over the function rather than in the naive order.

Then below this loop you will have a single worklist of unsimplified users that you can walk with the exact code you have above AFAICT.

[hans]

Can we not get the entire thing merged? I'd really like that... I think the patch is actually really close. I have a bunch of comments below but they're all pretty boring in reality.

It's really late in the process -- as in I'm only really waiting for this and one other bug -- so I'm hesitant to take a large change, but I also don't really understand exactly what's at stake here.

[jyknight]

In D65280#1649062, @hans wrote:

Can we not get the entire thing merged? I'd really like that... I think the patch is actually really close. I have a bunch of comments below but they're all pretty boring in reality.

It's really late in the process -- as in I'm only really waiting for this and one other bug -- so I'm hesitant to take a large change, but I also don't really understand exactly what's at stake here.

I'm also against putting it into 9.0, which is supposed to have a final release Real Soon Now. This is not a obviously-correct change, and it should bake in trunk for at least a couple weeks before going into the release, to shake out any unexpected problems.

I think we should target this for the 9.0.1 patch though, which is why joerg wants to merge the API change to 9.0 now.

[hans]

I'm also against putting it into 9.0, which is supposed to have a final release Real Soon Now. This is not a obviously-correct change, and it should bake in trunk for at least a couple weeks before going into the release, to shake out any unexpected problems.

I think we should target this for the 9.0.1 patch though, which is why joerg wants to merge the API change to 9.0 now.

This sounds good to me. From my point of view, ideally the API change would be broken out and landed on trunk as soon as possible, and then I'd merge that change to the 9 branch.

[joerg]

Apply feedback.

[joerg]

lib/Transforms/Scalar/LowerConstantIntrinsics.cpp

113–117 ↗

(On Diff #217160)

Inside a BB, we have to use forward order for the processing so that the intrinsic calls are removed with the expected results. This means that the recursive simplification can remove the next instruction (InstPtr) or unhook it (no parent). Both cases are covered by the test cases. I don't see how a different iteration order inside the BB can work or avoid the problem.

Updating the CFG directly can help in those cases were PHIs are removed by removePredecessor. That seems generally desirable as property. So the question is which iteration order for the BBs to use. BB iteration order can change the result, but I'm not sure computing RPO is worth the hassle here? I'm leaving the actual removing of the dead block to removeUnreachableBlocks, especially since I don't want to open-code recursive removal and other edge cases.

[hans]

I've merged the InstructionSimplify.h part of this (r370355) to release_90 in r370447.

[joerg]

Switch the pass to use two rounds. The first round will collect all relevant intrinsics in RPO, the second one will translate them accordingly.

[joerg]

Ping

[joerg]

2nd ping. Chandler, care to check this please?

[chandlerc]

(Tried to get this out last weekend, but was blocked by the Phab down time... Sorry about that ...)

Mostly nits around the exact code here. The approach looks really nice now (and sorry it took so many iterations to get there).

lib/Passes/PassRegistry.def

188 ↗

(On Diff #219444)

Maybe lower-constant-intrinsics as a name? (Since it handles objectsize as well.

lib/Transforms/Scalar/LowerConstantIntrinsics.cpp

93 ↗

(On Diff #219444)

Use an early continue to reduce indentation?

96 ↗

(On Diff #219444)

Odd to continue here but break below. Doesn't matter in this case of course, but just seemed surprising.

105–106 ↗

(On Diff #219444)

FWIW, this doesn't skip anything, the loop has the same behavior.

112–117 ↗

(On Diff #219444)

For both the II thing and the default case -- do we really expect these to ever fail?

I would expect either the VH to be null, or for it to definitively be one of the two intrinsics we added. Maybe switch to cast_or_null above with VN.get() or some such, and llvm_unreachable on the default case.

[joerg]

Adjust based on comments.

[joerg]

lib/Transforms/Scalar/LowerConstantIntrinsics.cpp

105–106 ↗

(On Diff #219444)

It was primarily to get the correct return value, but I'm changing it to push the check to the final return.

112–117 ↗

(On Diff #219444)

Yes, the same concerns as with the earlier version still apply. The recursive simplification can change the instruction type in place or remove it. The logic is still simpler since no new instructions can appear.

[chandlerc]

FWIW, the adjustments I'm suggesting around tightening the logic can easily be in a follow-up patch if you like. I think generally the code LGTM and I'd just like us to pin down exactly what changes we expect to happen w/ the handles as much as possible to avoid subtle latent bugs creeping in and never getting noticed.

The other two are trivial, feel free to land w/ those fixed.

lib/Transforms/Scalar/LowerConstantIntrinsics.cpp

112–117 ↗

(On Diff #219444)

I'm really surprised that it can *change* the value handle in this way.

I guess because we're using a tracking value handle (is that really necessary?) they may be moved onto the constant, but IMO that'd be more cleanly handled by checking for the value handle being either null or a non-instruction value. If its an instruction, it should really only be one of these two intrinsics or something deeply wrong has happened elsewhere, no?

I'm mostly suggesting we assert on that to track down the strange behavior and make sure the overall logic is actually still correct if it comes up rather than potentially hiding a deeper bug.

test/Transforms/LowerConstantIntrinsics/crash-on-large-allocas.ll

1 ↗

(On Diff #224563)

Probable just one - is fine?

test/Transforms/LowerConstantIntrinsics/objectsize_basic.ll

1 ↗

(On Diff #224563)

Probably just one - is fine?

[gribozavr]

Sorry, this change broke the build (http://lab.llvm.org:8011/builders/clang-x86_64-debian-fast/builds/19218) and I reverted it in r374768.

[chill]

FYI: https://bugs.llvm.org/show_bug.cgi?id=48472

llvm/bindings/ocaml/transforms/scalar_opts/llvm_scalar_opts.mli

external add_early_cse
   : [< Llvm.PassManager.any ] Llvm.PassManager.t -> unit
   = "llvm_add_early_cse"
 
 (** See the [llvm::createLowerExpectIntrinsicPass] function. *)
 external add_lower_expect_intrinsic
   : [< Llvm.PassManager.any ] Llvm.PassManager.t -> unit
   = "llvm_add_lower_expect_intrinsic"
 
+(** See the [llvm::createLowerConstantIntrinsicsPass] function. *)
+external add_lower_constant_intrinsics
+  : [< Llvm.PassManager.any ] Llvm.PassManager.t -> unit
+  = "llvm_add_lower_constant_intrinsics"
+
 (** See the [llvm::createTypeBasedAliasAnalysisPass] function. *)
 external add_type_based_alias_analysis
   : [< Llvm.PassManager.any ] Llvm.PassManager.t -> unit
   = "llvm_add_type_based_alias_analysis"
 
 (** See the [llvm::createScopedNoAliasAAPass] function. *)
 external add_scoped_no_alias_alias_analysis
   : [< Llvm.PassManager.any ] Llvm.PassManager.t -> unit

llvm/bindings/ocaml/transforms/scalar_opts/scalar_opts_ocaml.c

 
 /* [<Llvm.PassManager.any] Llvm.PassManager.t -> unit */
 CAMLprim value llvm_add_lower_expect_intrinsic(LLVMPassManagerRef PM) {
   LLVMAddLowerExpectIntrinsicPass(PM);
   return Val_unit;
 }
 
 /* [<Llvm.PassManager.any] Llvm.PassManager.t -> unit */
+CAMLprim value llvm_add_lower_constant_intrinsics(LLVMPassManagerRef PM) {
+  LLVMAddLowerConstantIntrinsicsPass(PM);
+  return Val_unit;
+}
+
+/* [<Llvm.PassManager.any] Llvm.PassManager.t -> unit */
 CAMLprim value llvm_add_type_based_alias_analysis(LLVMPassManagerRef PM) {
   LLVMAddTypeBasedAliasAnalysisPass(PM);
   return Val_unit;
 }
 
 /* [<Llvm.PassManager.any] Llvm.PassManager.t -> unit */
 CAMLprim value llvm_add_scoped_no_alias_aa(LLVMPassManagerRef PM) {
   LLVMAddScopedNoAliasAAPass(PM);

llvm/include/llvm-c/Transforms/Scalar.h

 void LLVMAddEarlyCSEPass(LLVMPassManagerRef PM);
 
 /** See llvm::createEarlyCSEPass function */
 void LLVMAddEarlyCSEMemSSAPass(LLVMPassManagerRef PM);
 
 /** See llvm::createLowerExpectIntrinsicPass function */
 void LLVMAddLowerExpectIntrinsicPass(LLVMPassManagerRef PM);
 
+/** See llvm::createLowerConstantIntrinsicsPass function */
+void LLVMAddLowerConstantIntrinsicsPass(LLVMPassManagerRef PM);
+
 /** See llvm::createTypeBasedAliasAnalysisPass function */
 void LLVMAddTypeBasedAliasAnalysisPass(LLVMPassManagerRef PM);
 
 /** See llvm::createScopedNoAliasAAPass function */
 void LLVMAddScopedNoAliasAAPass(LLVMPassManagerRef PM);
 
 /** See llvm::createBasicAliasAnalysisPass function */
 void LLVMAddBasicAliasAnalysisPass(LLVMPassManagerRef PM);

llvm/include/llvm/InitializePasses.h

 void initializeLoopStrengthReducePass(PassRegistry&);
 void initializeLoopUnrollAndJamPass(PassRegistry&);
 void initializeLoopUnrollPass(PassRegistry&);
 void initializeLoopUnswitchPass(PassRegistry&);
 void initializeLoopVectorizePass(PassRegistry&);
 void initializeLoopVersioningLICMPass(PassRegistry&);
 void initializeLoopVersioningPassPass(PassRegistry&);
 void initializeLowerAtomicLegacyPassPass(PassRegistry&);
+void initializeLowerConstantIntrinsicsPass(PassRegistry&);
 void initializeLowerEmuTLSPass(PassRegistry&);
 void initializeLowerExpectIntrinsicPass(PassRegistry&);
 void initializeLowerGuardIntrinsicLegacyPassPass(PassRegistry&);
 void initializeLowerWidenableConditionLegacyPassPass(PassRegistry&);
 void initializeLowerIntrinsicsPass(PassRegistry&);
 void initializeLowerInvokeLegacyPassPass(PassRegistry&);
 void initializeLowerSwitchPass(PassRegistry&);
 void initializeLowerTypeTestsPass(PassRegistry&);

llvm/include/llvm/LinkAllPasses.h

ForcePassLinking() {
       (void) llvm::createLoopStrengthReducePass();
       (void) llvm::createLoopRerollPass();
       (void) llvm::createLoopUnrollPass();
       (void) llvm::createLoopUnrollAndJamPass();
       (void) llvm::createLoopUnswitchPass();
       (void) llvm::createLoopVersioningLICMPass();
       (void) llvm::createLoopIdiomPass();
       (void) llvm::createLoopRotatePass();
+      (void) llvm::createLowerConstantIntrinsicsPass();
       (void) llvm::createLowerExpectIntrinsicPass();
       (void) llvm::createLowerInvokePass();
       (void) llvm::createLowerSwitchPass();
       (void) llvm::createNaryReassociatePass();
       (void) llvm::createObjCARCAAWrapperPass();
       (void) llvm::createObjCARCAPElimPass();
       (void) llvm::createObjCARCExpandPass();
       (void) llvm::createObjCARCContractPass();

llvm/include/llvm/Transforms/Scalar.h

 //===----------------------------------------------------------------------===//
 //
 // LowerExpectIntrinsics - Removes llvm.expect intrinsics and creates
 // "block_weights" metadata.
 FunctionPass *createLowerExpectIntrinsicPass();
 
 //===----------------------------------------------------------------------===//
 //
+// LowerConstantIntrinsicss - Expand any remaining llvm.objectsize and
+// llvm.is.constant intrinsic calls, even for the unknown cases.
+//
+FunctionPass *createLowerConstantIntrinsicsPass();
+
+//===----------------------------------------------------------------------===//
+//
 // PartiallyInlineLibCalls - Tries to inline the fast path of library
 // calls such as sqrt.
 //
 FunctionPass *createPartiallyInlineLibCallsPass();
 
 //===----------------------------------------------------------------------===//
 //
 // SeparateConstOffsetFromGEP - Split GEPs for better CSE

llvm/include/llvm/Transforms/Scalar/LowerConstantIntrinsics.h

+//===- LowerConstantIntrinsics.h - Lower constant int. pass -*- C++ -*-========//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+/// \file
+///
+/// The header file for the LowerConstantIntrinsics pass as used by the new pass
+/// manager.
+///
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_TRANSFORMS_SCALAR_LOWERCONSTANTINTRINSICS_H
+#define LLVM_TRANSFORMS_SCALAR_LOWERCONSTANTINTRINSICS_H
+
+#include "llvm/IR/Function.h"
+#include "llvm/IR/PassManager.h"
+
+namespace llvm {
+
+struct LowerConstantIntrinsicsPass :
+    PassInfoMixin<LowerConstantIntrinsicsPass> {
+public:
+  explicit LowerConstantIntrinsicsPass() {}
+
+  /// Run the pass over the function.
+  ///
+  /// This will lower all remaining 'objectsize' and 'is.constant'`
+  /// intrinsic calls in this function, even when the argument has no known
+  /// size or is not a constant respectively. The resulting constant is
+  /// propagated and conditional branches are resolved where possible.
+  /// This complements the Instruction Simplification and
+  /// Instruction Combination passes of the optimized pass chain.
+  PreservedAnalyses run(Function &F, FunctionAnalysisManager &);
+};
+
+}
+
+#endif

llvm/lib/CodeGen/CodeGenPrepare.cpp

case Intrinsic::experimental_widenable_condition: {
         return true;
       }
       Constant *RetVal = ConstantInt::getTrue(II->getContext());
       resetIteratorIfInvalidatedWhileCalling(BB, [&]() {
         replaceAndRecursivelySimplify(CI, RetVal, TLInfo, nullptr);
       });
       return true;
     }
-    case Intrinsic::objectsize: {
-      // Lower all uses of llvm.objectsize.*
-      Value *RetVal =
-          lowerObjectSizeCall(II, *DL, TLInfo, /*MustSucceed=*/true);
-
-      resetIteratorIfInvalidatedWhileCalling(BB, [&]() {
-        replaceAndRecursivelySimplify(CI, RetVal, TLInfo, nullptr);
-      });
-      return true;
-    }
-    case Intrinsic::is_constant: {
-      // If is_constant hasn't folded away yet, lower it to false now.
-      Constant *RetVal = ConstantInt::get(II->getType(), 0);
-      resetIteratorIfInvalidatedWhileCalling(BB, [&]() {
-        replaceAndRecursivelySimplify(CI, RetVal, TLInfo, nullptr);
-      });
-      return true;
-    }
+    case Intrinsic::objectsize:
+      llvm_unreachable("llvm.objectsize.* should have been lowered already");
+    case Intrinsic::is_constant:
+      llvm_unreachable("llvm.is.constant.* should have been lowered already");
     case Intrinsic::aarch64_stlxr:
     case Intrinsic::aarch64_stxr: {
       ZExtInst *ExtVal = dyn_cast<ZExtInst>(CI->getArgOperand(0));
       if (!ExtVal || !ExtVal->hasOneUse() ||
           ExtVal->getParent() == CI->getParent())
         return false;
       // Sink a zext feeding stlxr/stxr before it, so it can be folded into it.
       ExtVal->moveBefore(CI);

llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp

case Intrinsic::memset:
     return translateMemFunc(CI, MIRBuilder, ID);
   case Intrinsic::eh_typeid_for: {
     GlobalValue *GV = ExtractTypeInfo(CI.getArgOperand(0));
     Register Reg = getOrCreateVReg(CI);
     unsigned TypeID = MF->getTypeIDFor(GV);
     MIRBuilder.buildConstant(Reg, TypeID);
     return true;
   }
-  case Intrinsic::objectsize: {
-    // If we don't know by now, we're never going to know.
-    const ConstantInt *Min = cast<ConstantInt>(CI.getArgOperand(1));
+  case Intrinsic::objectsize:
+    llvm_unreachable("llvm.objectsize.* should have been lowered already");
 
-    MIRBuilder.buildConstant(getOrCreateVReg(CI), Min->isZero() ? -1ULL : 0);
-    return true;
-  }
   case Intrinsic::is_constant:
-    // If this wasn't constant-folded away by now, then it's not a
-    // constant.
-    MIRBuilder.buildConstant(getOrCreateVReg(CI), 0);
-    return true;
+    llvm_unreachable("llvm.is.constant.* should have been lowered already");
+
   case Intrinsic::stackguard:
     getStackGuard(getOrCreateVReg(CI), MIRBuilder);
     return true;
   case Intrinsic::stackprotector: {
     LLT PtrTy = getLLTForType(*CI.getArgOperand(0)->getType(), *DL);
     Register GuardVal = MRI->createGenericVirtualRegister(PtrTy);
     getStackGuard(GuardVal, MIRBuilder);
 

llvm/lib/CodeGen/SelectionDAG/FastISel.cpp

if (!FuncInfo.MF->getMMI().hasDebugInfo()) {
       LLVM_DEBUG(dbgs() << "Dropping debug info for " << *DI << "\n");
       return true;
     }
 
     BuildMI(*FuncInfo.MBB, FuncInfo.InsertPt, DbgLoc,
             TII.get(TargetOpcode::DBG_LABEL)).addMetadata(DI->getLabel());
     return true;
   }
-  case Intrinsic::objectsize: {
-    ConstantInt *CI = cast<ConstantInt>(II->getArgOperand(1));
-    unsigned long long Res = CI->isZero() ? -1ULL : 0;
-    Constant *ResCI = ConstantInt::get(II->getType(), Res);
-    unsigned ResultReg = getRegForValue(ResCI);
-    if (!ResultReg)
-      return false;
-    updateValueMap(II, ResultReg);
-    return true;
-  }
-  case Intrinsic::is_constant: {
-    Constant *ResCI = ConstantInt::get(II->getType(), 0);
-    unsigned ResultReg = getRegForValue(ResCI);
-    if (!ResultReg)
-      return false;
-    updateValueMap(II, ResultReg);
-    return true;
-  }
+  case Intrinsic::objectsize:
+    llvm_unreachable("llvm.objectsize.* should have been lowered already");
+
+  case Intrinsic::is_constant:
+    llvm_unreachable("llvm.is.constant.* should have been lowered already");
+
   case Intrinsic::launder_invariant_group:
   case Intrinsic::strip_invariant_group:
   case Intrinsic::expect: {
     unsigned ResultReg = getRegForValue(II->getArgOperand(0));
     if (!ResultReg)
       return false;
     updateValueMap(II, ResultReg);
     return true;

llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp

case Intrinsic::stackprotector: {
     // Store the stack protector onto the stack.
     Res = DAG.getStore(Chain, sdl, Src, FIN, MachinePointerInfo::getFixedStack(
                                                  DAG.getMachineFunction(), FI),
                        /* Alignment = */ 0, MachineMemOperand::MOVolatile);
     setValue(&I, Res);
     DAG.setRoot(Res);
     return;
   }
-  case Intrinsic::objectsize: {
-    // If we don't know by now, we're never going to know.
-    ConstantInt *CI = dyn_cast<ConstantInt>(I.getArgOperand(1));
-
-    assert(CI && "Non-constant type in __builtin_object_size?");
-
-    SDValue Arg = getValue(I.getCalledValue());
-    EVT Ty = Arg.getValueType();
-
-    if (CI->isZero())
-      Res = DAG.getConstant(-1ULL, sdl, Ty);
-    else
-      Res = DAG.getConstant(0, sdl, Ty);
-
-    setValue(&I, Res);
-    return;
-  }
+  case Intrinsic::objectsize:
+    llvm_unreachable("llvm.objectsize.* should have been lowered already");
 
   case Intrinsic::is_constant:
-    // If this wasn't constant-folded away by now, then it's not a
-    // constant.
-    setValue(&I, DAG.getConstant(0, sdl, MVT::i1));
-    return;
+    llvm_unreachable("llvm.is.constant.* should have been lowered already");
 
   case Intrinsic::annotation:
   case Intrinsic::ptr_annotation:
   case Intrinsic::launder_invariant_group:
   case Intrinsic::strip_invariant_group:
     // Drop the intrinsic, but forward the value
     setValue(&I, getValue(I.getOperand(0)));
     return;

llvm/lib/CodeGen/TargetPassConfig.cpp

if (!DisableMergeICmps)
       addPass(createMergeICmpsLegacyPass());
     addPass(createExpandMemCmpPass());
   }
 
   // Run GC lowering passes for builtin collectors
   // TODO: add a pass insertion point here
   addPass(createGCLoweringPass());
   addPass(createShadowStackGCLoweringPass());
+  addPass(createLowerConstantIntrinsicsPass());
 
   // Make sure that no unreachable blocks are instruction selected.
   addPass(createUnreachableBlockEliminationPass());
 
   // Prepare expensive constants for SelectionDAG.
   if (getOptLevel() != CodeGenOpt::None && !DisableConstantHoisting)
     addPass(createConstantHoistingPass());
 

llvm/lib/Passes/PassBuilder.cpp

 #include "llvm/Transforms/Scalar/LoopPredication.h"
 #include "llvm/Transforms/Scalar/LoopRotation.h"
 #include "llvm/Transforms/Scalar/LoopSimplifyCFG.h"
 #include "llvm/Transforms/Scalar/LoopSink.h"
 #include "llvm/Transforms/Scalar/LoopStrengthReduce.h"
 #include "llvm/Transforms/Scalar/LoopUnrollAndJamPass.h"
 #include "llvm/Transforms/Scalar/LoopUnrollPass.h"
 #include "llvm/Transforms/Scalar/LowerAtomic.h"
+#include "llvm/Transforms/Scalar/LowerConstantIntrinsics.h"
 #include "llvm/Transforms/Scalar/LowerExpectIntrinsic.h"
 #include "llvm/Transforms/Scalar/LowerGuardIntrinsic.h"
 #include "llvm/Transforms/Scalar/LowerWidenableCondition.h"
 #include "llvm/Transforms/Scalar/MakeGuardsExplicit.h"
 #include "llvm/Transforms/Scalar/MemCpyOptimizer.h"
 #include "llvm/Transforms/Scalar/MergeICmps.h"
 #include "llvm/Transforms/Scalar/MergedLoadStoreMotion.h"
 #include "llvm/Transforms/Scalar/NaryReassociate.h"
ModulePassManager PassBuilder::buildModuleOptimizationPipeline(
   // and richly annotated call graph. By computing aliasing and mod/ref
   // information for all local globals here, the late loop passes and notably
   // the vectorizer will be able to use them to help recognize vectorizable
   // memory operations.
   MPM.addPass(RequireAnalysisPass<GlobalsAA, Module>());
 
   FunctionPassManager OptimizePM(DebugLogging);
   OptimizePM.addPass(Float2IntPass());
+  OptimizePM.addPass(LowerConstantIntrinsicsPass());
+
   // FIXME: We need to run some loop optimizations to re-rotate loops after
   // simplify-cfg and others undo their rotation.
 
   // Optimize the loop execution. These passes operate on entire loop nests
   // rather than on each loop in an inside-out manner, and so they are actually
   // function passes.
 
   for (auto &C : VectorizerStartEPCallbacks)

llvm/lib/Passes/PassRegistry.def

 FUNCTION_PASS("instsimplify", InstSimplifyPass())
 FUNCTION_PASS("invalidate<all>", InvalidateAllAnalysesPass())
 FUNCTION_PASS("float2int", Float2IntPass())
 FUNCTION_PASS("no-op-function", NoOpFunctionPass())
 FUNCTION_PASS("libcalls-shrinkwrap", LibCallsShrinkWrapPass())
 FUNCTION_PASS("loweratomic", LowerAtomicPass())
 FUNCTION_PASS("lower-expect", LowerExpectIntrinsicPass())
 FUNCTION_PASS("lower-guard-intrinsic", LowerGuardIntrinsicPass())
+FUNCTION_PASS("lower-constant-intrinsics", LowerConstantIntrinsicsPass())
 FUNCTION_PASS("lower-widenable-condition", LowerWidenableConditionPass())
 FUNCTION_PASS("guard-widening", GuardWideningPass())
 FUNCTION_PASS("gvn", GVN())
 FUNCTION_PASS("load-store-vectorizer", LoadStoreVectorizerPass())
 FUNCTION_PASS("loop-simplify", LoopSimplifyPass())
 FUNCTION_PASS("loop-sink", LoopSinkPass())
 FUNCTION_PASS("lowerinvoke", LowerInvokePass())
 FUNCTION_PASS("mem2reg", PromotePass())

llvm/lib/Transforms/IPO/PassManagerBuilder.cpp

void PassManagerBuilder::populateModulePassManager(
   // long as the first function pass doesn't invalidate the module analysis.
   // Thus both Float2Int and LoopRotate have to preserve AliasAnalysis for
   // this to work. Fortunately, it is trivial to preserve AliasAnalysis
   // (doing nothing preserves it as it is required to be conservatively
   // correct in the face of IR changes).
   MPM.add(createGlobalsAAWrapperPass());
 
   MPM.add(createFloat2IntPass());
+  MPM.add(createLowerConstantIntrinsicsPass());
 
   addExtensionsToPM(EP_VectorizerStart, MPM);
 
   // Re-rotate loops in all our loop nests. These may have fallout out of
   // rotated form due to GVN or other transformations, and the vectorizer relies
   // on the rotated form. Disable header duplication at -Oz.
   MPM.add(createLoopRotatePass(SizeLevel == 2 ? 0 : -1));
 

llvm/lib/Transforms/Scalar/CMakeLists.txt

add_llvm_library(LLVMScalarOpts
   LoopRotation.cpp
   LoopSimplifyCFG.cpp
   LoopStrengthReduce.cpp
   LoopUnrollPass.cpp
   LoopUnrollAndJamPass.cpp
   LoopUnswitch.cpp
   LoopVersioningLICM.cpp
   LowerAtomic.cpp
+  LowerConstantIntrinsics.cpp
   LowerExpectIntrinsic.cpp
   LowerGuardIntrinsic.cpp
   LowerWidenableCondition.cpp
   MakeGuardsExplicit.cpp
   MemCpyOptimizer.cpp
   MergeICmps.cpp
   MergedLoadStoreMotion.cpp
   NaryReassociate.cpp

llvm/lib/Transforms/Scalar/LowerConstantIntrinsics.cpp

+//===- LowerConstantIntrinsics.cpp - Lower constant intrinsic calls -------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This pass lowers all remaining 'objectsize' 'is.constant' intrinsic calls
+// and provides constant propagation and basic CFG cleanup on the result.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Transforms/Scalar/LowerConstantIntrinsics.h"
+#include "llvm/ADT/PostOrderIterator.h"
+#include "llvm/ADT/Statistic.h"
+#include "llvm/Analysis/InstructionSimplify.h"
+#include "llvm/Analysis/MemoryBuiltins.h"
+#include "llvm/Analysis/TargetLibraryInfo.h"
+#include "llvm/IR/BasicBlock.h"
+#include "llvm/IR/Constants.h"
+#include "llvm/IR/Function.h"
+#include "llvm/IR/Instructions.h"
+#include "llvm/IR/IntrinsicInst.h"
+#include "llvm/IR/Intrinsics.h"
+#include "llvm/IR/PatternMatch.h"
+#include "llvm/Pass.h"
+#include "llvm/Support/Debug.h"
+#include "llvm/Transforms/Scalar.h"
+#include "llvm/Transforms/Utils/Local.h"
+
+using namespace llvm;
+using namespace llvm::PatternMatch;
+
+#define DEBUG_TYPE "lower-is-constant-intrinsic"
+
+STATISTIC(IsConstantIntrinsicsHandled,
+          "Number of 'is.constant' intrinsic calls handled");
+STATISTIC(ObjectSizeIntrinsicsHandled,
+          "Number of 'objectsize' intrinsic calls handled");
+
+static Value *lowerIsConstantIntrinsic(IntrinsicInst *II) {
+  Value *Op = II->getOperand(0);
+
+  return isa<Constant>(Op) ? ConstantInt::getTrue(II->getType())
+                           : ConstantInt::getFalse(II->getType());
+}
+
+static bool replaceConditionalBranchesOnConstant(Instruction *II,
+                                                 Value *NewValue) {
+  bool HasDeadBlocks = false;
+  SmallSetVector<Instruction *, 8> Worklist;
+  replaceAndRecursivelySimplify(II, NewValue, nullptr, nullptr, nullptr,
+                                &Worklist);
+  for (auto I : Worklist) {
+    BranchInst *BI = dyn_cast<BranchInst>(I);
+    if (!BI)
+      continue;
+    if (BI->isUnconditional())
+      continue;
+
+    BasicBlock *Target, *Other;
+    if (match(BI->getOperand(0), m_Zero())) {
+      Target = BI->getSuccessor(1);
+      Other = BI->getSuccessor(0);
+    } else if (match(BI->getOperand(0), m_One())) {
+      Target = BI->getSuccessor(0);
+      Other = BI->getSuccessor(1);
+    } else {
+      Target = nullptr;
+      Other = nullptr;
+    }
+    if (Target && Target != Other) {
+      BasicBlock *Source = BI->getParent();
+      Other->removePredecessor(Source);
+      BI->eraseFromParent();
+      BranchInst::Create(Target, Source);
+      if (pred_begin(Other) == pred_end(Other))
+        HasDeadBlocks = true;
+    }
+  }
+  return HasDeadBlocks;
+}
+
+static bool lowerConstantIntrinsics(Function &F, const TargetLibraryInfo *TLI) {
+  bool HasDeadBlocks = false;
+  const auto &DL = F.getParent()->getDataLayout();
+  SmallVector<WeakTrackingVH, 8> Worklist;
+
+  ReversePostOrderTraversal<Function *> RPOT(&F);
+  for (BasicBlock *BB : RPOT) {
+    for (Instruction &I: *BB) {
+      IntrinsicInst *II = dyn_cast<IntrinsicInst>(&I);
+      if (!II)
+        continue;
+      switch (II->getIntrinsicID()) {
+      default:
+        break;
+      case Intrinsic::is_constant:
+      case Intrinsic::objectsize:
+        Worklist.push_back(WeakTrackingVH(&I));
+        break;
+      }
+    }
+  }
+  for (WeakTrackingVH &VH: Worklist) {
+    // Items on the worklist can be mutated by earlier recursive replaces.
+    // This can remove the intrinsic as dead (VH == null), but also replace
+    // the intrinsic in place.
+    if (!VH)
+      continue;
+    IntrinsicInst *II = dyn_cast<IntrinsicInst>(&*VH);
+    if (!II)
+      continue;
+    Value *NewValue;
+    switch (II->getIntrinsicID()) {
+    default:
+      continue;
+    case Intrinsic::is_constant:
+      NewValue = lowerIsConstantIntrinsic(II);
+      IsConstantIntrinsicsHandled++;
+      break;
+    case Intrinsic::objectsize:
+      NewValue = lowerObjectSizeCall(II, DL, TLI, true);
+      ObjectSizeIntrinsicsHandled++;
+      break;
+    }
+    HasDeadBlocks |= replaceConditionalBranchesOnConstant(II, NewValue);
+  }
+  if (HasDeadBlocks)
+    removeUnreachableBlocks(F);
+  return !Worklist.empty();
+}
+
+PreservedAnalyses
+LowerConstantIntrinsicsPass::run(Function &F, FunctionAnalysisManager &AM) {
+  if (lowerConstantIntrinsics(F, AM.getCachedResult<TargetLibraryAnalysis>(F)))
+    return PreservedAnalyses::none();
+
+  return PreservedAnalyses::all();
+}
+
+namespace {
+/// Legacy pass for lowering is.constant intrinsics out of the IR.
+///
+/// When this pass is run over a function it converts is.constant intrinsics
+/// into 'true' or 'false'. This is completements the normal constand folding
+/// to 'true' as part of Instruction Simplify passes.
+class LowerConstantIntrinsics : public FunctionPass {
+public:
+  static char ID;
+  LowerConstantIntrinsics() : FunctionPass(ID) {
+    initializeLowerConstantIntrinsicsPass(*PassRegistry::getPassRegistry());
+  }
+
+  bool runOnFunction(Function &F) override {
+    auto *TLIP = getAnalysisIfAvailable<TargetLibraryInfoWrapperPass>();
+    const TargetLibraryInfo *TLI = TLIP ? &TLIP->getTLI(F) : nullptr;
+    return lowerConstantIntrinsics(F, TLI);
+  }
+};
+} // namespace
+
+char LowerConstantIntrinsics::ID = 0;
+INITIALIZE_PASS(LowerConstantIntrinsics, "lower-constant-intrinsics",
+                "Lower constant intrinsics", false, false)
+
+FunctionPass *llvm::createLowerConstantIntrinsicsPass() {
+  return new LowerConstantIntrinsics();
+}

llvm/lib/Transforms/Scalar/Scalar.cpp

void llvm::initializeScalarOpts(PassRegistry &Registry) {
   initializeLoopRerollPass(Registry);
   initializeLoopUnrollPass(Registry);
   initializeLoopUnrollAndJamPass(Registry);
   initializeLoopUnswitchPass(Registry);
   initializeWarnMissedTransformationsLegacyPass(Registry);
   initializeLoopVersioningLICMPass(Registry);
   initializeLoopIdiomRecognizeLegacyPassPass(Registry);
   initializeLowerAtomicLegacyPassPass(Registry);
+  initializeLowerConstantIntrinsicsPass(Registry);
   initializeLowerExpectIntrinsicPass(Registry);
   initializeLowerGuardIntrinsicLegacyPassPass(Registry);
   initializeLowerWidenableConditionLegacyPassPass(Registry);
   initializeMemCpyOptLegacyPassPass(Registry);
   initializeMergeICmpsLegacyPassPass(Registry);
   initializeMergedLoadStoreMotionLegacyPassPass(Registry);
   initializeNaryReassociateLegacyPassPass(Registry);
   initializePartiallyInlineLibCallsLegacyPassPass(Registry);

 void LLVMAddScopedNoAliasAAPass(LLVMPassManagerRef PM) {
   unwrap(PM)->add(createScopedNoAliasAAWrapperPass());
 }
 
 void LLVMAddBasicAliasAnalysisPass(LLVMPassManagerRef PM) {
   unwrap(PM)->add(createBasicAAWrapperPass());
 }
 
+void LLVMAddLowerConstantIntrinsicsPass(LLVMPassManagerRef PM) {
+  unwrap(PM)->add(createLowerConstantIntrinsicsPass());
+}
+
 void LLVMAddLowerExpectIntrinsicPass(LLVMPassManagerRef PM) {
   unwrap(PM)->add(createLowerExpectIntrinsicPass());
 }
 
 void LLVMAddUnifyFunctionExitNodesPass(LLVMPassManagerRef PM) {
   unwrap(PM)->add(createUnifyFunctionExitNodesPass());
 }

llvm/test/CodeGen/AArch64/GlobalISel/arm64-irtranslator.ll

 ; CHECK: [[SRC_C:%[0-9]+]]:_(s32) = COPY $w1
 ; CHECK: [[SRC:%[0-9]+]]:_(s8) = G_TRUNC [[SRC_C]]
 ; CHECK: [[SIZE:%[0-9]+]]:_(s64) = COPY $x2
 ; CHECK: G_INTRINSIC_W_SIDE_EFFECTS intrinsic(@llvm.memset), [[DST]](p0), [[SRC]](s8), [[SIZE]](s64), 0 :: (store 1 into %ir.dst)
   call void @llvm.memset.p0i8.i64(i8* %dst, i8 %val, i64 %size, i1 0)
   ret void
 }
 
-declare i64 @llvm.objectsize.i64(i8*, i1)
-declare i32 @llvm.objectsize.i32(i8*, i1)
-define void @test_objectsize(i8* %addr0, i8* %addr1) {
-; CHECK-LABEL: name: test_objectsize
-; CHECK: [[ADDR0:%[0-9]+]]:_(p0) = COPY $x0
-; CHECK: [[ADDR1:%[0-9]+]]:_(p0) = COPY $x1
-; CHECK: {{%[0-9]+}}:_(s64) = G_CONSTANT i64 -1
-; CHECK: {{%[0-9]+}}:_(s64) = G_CONSTANT i64 0
-; CHECK: {{%[0-9]+}}:_(s32) = G_CONSTANT i32 -1
-; CHECK: {{%[0-9]+}}:_(s32) = G_CONSTANT i32 0
-  %size64.0 = call i64 @llvm.objectsize.i64(i8* %addr0, i1 0)
-  %size64.intmin = call i64 @llvm.objectsize.i64(i8* %addr0, i1 1)
-  %size32.0 = call i32 @llvm.objectsize.i32(i8* %addr0, i1 0)
-  %size32.intmin = call i32 @llvm.objectsize.i32(i8* %addr0, i1 1)
-  ret void
-}
-
 define void @test_large_const(i128* %addr) {
 ; CHECK-LABEL: name: test_large_const
 ; CHECK: [[ADDR:%[0-9]+]]:_(p0) = COPY $x0
 ; CHECK: [[VAL:%[0-9]+]]:_(s128) = G_CONSTANT i128 42
 ; CHECK: G_STORE [[VAL]](s128), [[ADDR]](p0)
   store i128 42, i128* %addr
   ret void
 }

llvm/test/CodeGen/AArch64/O0-pipeline.ll

 ; CHECK-NEXT:     Pre-ISel Intrinsic Lowering
 ; CHECK-NEXT:     FunctionPass Manager
 ; CHECK-NEXT:       Expand Atomic instructions
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Basic Alias Analysis (stateless AA impl)
 ; CHECK-NEXT:       Module Verifier
 ; CHECK-NEXT:       Lower Garbage Collection Instructions
 ; CHECK-NEXT:       Shadow Stack GC Lowering
+; CHECK-NEXT:       Lower constant intrinsics
 ; CHECK-NEXT:       Remove unreachable blocks from the CFG
 ; CHECK-NEXT:       Instrument function entry/exit with calls to e.g. mcount() (post inlining)
 ; CHECK-NEXT:       Scalarize Masked Memory Intrinsics
 ; CHECK-NEXT:       Expand reduction intrinsics
 ; CHECK-NEXT:       AArch64 Stack Tagging
 ; CHECK-NEXT:     Rewrite Symbols
 ; CHECK-NEXT:     FunctionPass Manager
 ; CHECK-NEXT:       Dominator Tree Construction

llvm/test/CodeGen/AArch64/O3-pipeline.ll

 ; CHECK-NEXT:         Induction Variable Users
 ; CHECK-NEXT:         Loop Strength Reduction
 ; CHECK-NEXT:       Basic Alias Analysis (stateless AA impl)
 ; CHECK-NEXT:         Function Alias Analysis Results
 ; CHECK-NEXT:       Merge contiguous icmps into a memcmp
 ; CHECK-NEXT:       Expand memcmp() to load/stores
 ; CHECK-NEXT:       Lower Garbage Collection Instructions
 ; CHECK-NEXT:       Shadow Stack GC Lowering
+; CHECK-NEXT:       Lower constant intrinsics
 ; CHECK-NEXT:       Remove unreachable blocks from the CFG
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       Branch Probability Analysis
 ; CHECK-NEXT:       Block Frequency Analysis
 ; CHECK-NEXT:       Constant Hoisting
 ; CHECK-NEXT:       Partially inline calls to library functions
 ; CHECK-NEXT:       Instrument function entry/exit with calls to e.g. mcount() (post inlining)

llvm/test/CodeGen/ARM/O3-pipeline.ll

 ; CHECK-NEXT:        Induction Variable Users
 ; CHECK-NEXT:        Loop Strength Reduction
 ; CHECK-NEXT:      Basic Alias Analysis (stateless AA impl)
 ; CHECK-NEXT:      Function Alias Analysis Results
 ; CHECK-NEXT:      Merge contiguous icmps into a memcmp
 ; CHECK-NEXT:      Expand memcmp() to load/stores
 ; CHECK-NEXT:      Lower Garbage Collection Instructions
 ; CHECK-NEXT:      Shadow Stack GC Lowering
+; CHECK-NEXT:      Lower constant intrinsics
 ; CHECK-NEXT:      Remove unreachable blocks from the CFG
 ; CHECK-NEXT:      Dominator Tree Construction
 ; CHECK-NEXT:      Natural Loop Information
 ; CHECK-NEXT:      Branch Probability Analysis
 ; CHECK-NEXT:      Block Frequency Analysis
 ; CHECK-NEXT:      Constant Hoisting
 ; CHECK-NEXT:      Partially inline calls to library functions
 ; CHECK-NEXT:      Instrument function entry/exit with calls to e.g. mcount() (post inlining)

llvm/test/CodeGen/Generic/is-constant.ll

-; RUN: opt -O2 -S < %s  | FileCheck %s
-; RUN: llc -o /dev/null 2>&1 < %s
-; RUN: llc -O0 -o /dev/null 2>&1 < %s
-
-;; The llc runs above are just to ensure it doesn't blow up upon
-;; seeing an is_constant intrinsic.
-
-declare i1 @llvm.is.constant.i32(i32 %a)
-declare i1 @llvm.is.constant.i64(i64 %a)
-declare i1 @llvm.is.constant.i256(i256 %a)
-declare i1 @llvm.is.constant.v2i64(<2 x i64> %a)
-declare i1 @llvm.is.constant.f32(float %a)
-declare i1 @llvm.is.constant.sl_i32i32s({i32, i32} %a)
-declare i1 @llvm.is.constant.a2i64([2 x i64] %a)
-declare i1 @llvm.is.constant.p0i64(i64* %a)
-
-;; Basic test that optimization folds away the is.constant when given
-;; a constant.
-define i1 @test_constant() #0 {
-; CHECK-LABEL: @test_constant(
-; CHECK-NOT: llvm.is.constant
-; CHECK: ret i1 true
-%y = call i1 @llvm.is.constant.i32(i32 44)
-  ret i1 %y
-}
-
-;; And test that the intrinsic sticks around when given a
-;; non-constant.
-define i1 @test_nonconstant(i32 %x) #0 {
-; CHECK-LABEL: @test_nonconstant(
-; CHECK: @llvm.is.constant
-  %y = call i1 @llvm.is.constant.i32(i32 %x)
-  ret i1 %y
-}
-
-;; Ensure that nested is.constants fold.
-define i32 @test_nested() #0 {
-; CHECK-LABEL: @test_nested(
-; CHECK-NOT: llvm.is.constant
-; CHECK: ret i32 13
-  %val1 = call i1 @llvm.is.constant.i32(i32 27)
-  %val2 = zext i1 %val1 to i32
-  %val3 = add i32 %val2, 12
-  %1 = call i1 @llvm.is.constant.i32(i32 %val3)
-  %2 = zext i1 %1 to i32
-  %3 = add i32 %2, 12
-  ret i32 %3
-}
-
-@G = global [2 x i64] zeroinitializer
-define i1 @test_global() #0 {
-; CHECK-LABEL: @test_global(
-; CHECK: llvm.is.constant
-  %ret = call i1 @llvm.is.constant.p0i64(i64* getelementptr ([2 x i64], [2 x i64]* @G, i32 0, i32 0))
-  ret i1 %ret
-}
-
-define i1 @test_diff() #0 {
-; CHECK-LABEL: @test_diff(
-  %ret = call i1 @llvm.is.constant.i64(i64 sub (
-      i64 ptrtoint (i64* getelementptr inbounds ([2 x i64], [2 x i64]* @G, i64 0, i64 1) to i64),
-      i64 ptrtoint ([2 x i64]* @G to i64)))
-  ret i1 %ret
-}
-
-define i1 @test_various_types(i256 %int, float %float, <2 x i64> %vec, {i32, i32} %struct, [2 x i64] %arr, i64* %ptr) #0 {
-; CHECK-LABEL: @test_various_types(
-; CHECK: llvm.is.constant
-; CHECK: llvm.is.constant
-; CHECK: llvm.is.constant
-; CHECK: llvm.is.constant
-; CHECK: llvm.is.constant
-; CHECK: llvm.is.constant
-; CHECK-NOT: llvm.is.constant
-  %v1 = call i1 @llvm.is.constant.i256(i256 %int)
-  %v2 = call i1 @llvm.is.constant.f32(float %float)
-  %v3 = call i1 @llvm.is.constant.v2i64(<2 x i64> %vec)
-  %v4 = call i1 @llvm.is.constant.sl_i32i32s({i32, i32} %struct)
-  %v5 = call i1 @llvm.is.constant.a2i64([2 x i64] %arr)
-  %v6 = call i1 @llvm.is.constant.p0i64(i64* %ptr)
-
-  %c1 = call i1 @llvm.is.constant.i256(i256 -1)
-  %c2 = call i1 @llvm.is.constant.f32(float 17.0)
-  %c3 = call i1 @llvm.is.constant.v2i64(<2 x i64> <i64 -1, i64 44>)
-  %c4 = call i1 @llvm.is.constant.sl_i32i32s({i32, i32} {i32 -1, i32 32})
-  %c5 = call i1 @llvm.is.constant.a2i64([2 x i64] [i64 -1, i64 32])
-  %c6 = call i1 @llvm.is.constant.p0i64(i64* inttoptr (i32 42 to i64*))
-
-  %x1 = add i1 %v1, %c1
-  %x2 = add i1 %v2, %c2
-  %x3 = add i1 %v3, %c3
-  %x4 = add i1 %v4, %c4
-  %x5 = add i1 %v5, %c5
-  %x6 = add i1 %v6, %c6
-
-  %res2 = add i1 %x1, %x2
-  %res3 = add i1 %res2, %x3
-  %res4 = add i1 %res3, %x4
-  %res5 = add i1 %res4, %x5
-  %res6 = add i1 %res5, %x6
-
-  ret i1 %res6
-}
-
-define i1 @test_various_types2() #0 {
-; CHECK-LABEL: @test_various_types2(
-; CHECK: ret i1 false
-  %r = call i1 @test_various_types(i256 -1, float 22.0, <2 x i64> <i64 -1, i64 44>,
-                     {i32, i32} {i32 -1, i32 55}, [2 x i64] [i64 -1, i64 55],
-         i64* inttoptr (i64 42 to i64*))
-  ret i1 %r
-}
-
-attributes #0 = { nounwind uwtable }

llvm/test/CodeGen/X86/O0-pipeline.ll

 ; CHECK-NEXT:     Pre-ISel Intrinsic Lowering
 ; CHECK-NEXT:     FunctionPass Manager
 ; CHECK-NEXT:       Expand Atomic instructions
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Basic Alias Analysis (stateless AA impl)
 ; CHECK-NEXT:       Module Verifier
 ; CHECK-NEXT:       Lower Garbage Collection Instructions
 ; CHECK-NEXT:       Shadow Stack GC Lowering
+; CHECK-NEXT:       Lower constant intrinsics
 ; CHECK-NEXT:       Remove unreachable blocks from the CFG
 ; CHECK-NEXT:       Instrument function entry/exit with calls to e.g. mcount() (post inlining)
 ; CHECK-NEXT:       Scalarize Masked Memory Intrinsics
 ; CHECK-NEXT:       Expand reduction intrinsics
 ; CHECK-NEXT:       Expand indirectbr instructions
 ; CHECK-NEXT:     Rewrite Symbols
 ; CHECK-NEXT:     FunctionPass Manager
 ; CHECK-NEXT:       Dominator Tree Construction

llvm/test/CodeGen/X86/O3-pipeline.ll

 ; CHECK-NEXT:         Induction Variable Users
 ; CHECK-NEXT:         Loop Strength Reduction
 ; CHECK-NEXT:       Basic Alias Analysis (stateless AA impl)
 ; CHECK-NEXT:         Function Alias Analysis Results
 ; CHECK-NEXT:       Merge contiguous icmps into a memcmp
 ; CHECK-NEXT:       Expand memcmp() to load/stores
 ; CHECK-NEXT:       Lower Garbage Collection Instructions
 ; CHECK-NEXT:       Shadow Stack GC Lowering
+; CHECK-NEXT:       Lower constant intrinsics
 ; CHECK-NEXT:       Remove unreachable blocks from the CFG
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       Branch Probability Analysis
 ; CHECK-NEXT:       Block Frequency Analysis
 ; CHECK-NEXT:       Constant Hoisting
 ; CHECK-NEXT:       Partially inline calls to library functions
 ; CHECK-NEXT:       Instrument function entry/exit with calls to e.g. mcount() (post inlining)

llvm/test/CodeGen/X86/is-constant.ll

-; RUN: llc -O2 < %s | FileCheck %s --check-prefix=CHECK-O2 --check-prefix=CHECK
-; RUN: llc -O0 -fast-isel < %s | FileCheck %s --check-prefix=CHECK-O0 --check-prefix=CHECK
-; RUN: llc -O0 -fast-isel=0 < %s | FileCheck %s --check-prefix=CHECK-O0 --check-prefix=CHECK
-; RUN: llc -O0 -global-isel < %s | FileCheck %s --check-prefix=CHECK-O0 --check-prefix=CHECK
-
-;; Ensure that an unfoldable is.constant gets lowered reasonably in
-;; optimized codegen, in particular, that the "true" branch is
-;; eliminated.
-;;
-;; This isn't asserting any specific output from non-optimized runs,
-;; (e.g., currently the not-taken branch does not get eliminated). But
-;; it does ensure that lowering succeeds in all 3 codegen paths.
-
-target triple = "x86_64-unknown-linux-gnu"
-
-declare i1 @llvm.is.constant.i32(i32 %a) nounwind readnone
-declare i1 @llvm.is.constant.i64(i64 %a) nounwind readnone
-declare i64 @llvm.objectsize.i64.p0i8(i8*, i1, i1, i1) nounwind readnone
-
-declare i32 @subfun_1()
-declare i32 @subfun_2()
-
-define i32 @test_branch(i32 %in) nounwind {
-; CHECK-LABEL:    test_branch:
-; CHECK-O2:       %bb.0:
-; CHECK-O2-NEXT:  jmp subfun_2
-  %v = call i1 @llvm.is.constant.i32(i32 %in)
-  br i1 %v, label %True, label %False
-
-True:
-  %call1 = tail call i32 @subfun_1()
-  ret i32 %call1
-
-False:
-  %call2 = tail call i32 @subfun_2()
-  ret i32 %call2
-}
-
-;; llvm.objectsize is another tricky case which gets folded to -1 very
-;; late in the game. We'd like to ensure that llvm.is.constant of
-;; llvm.objectsize is true.
-define i1 @test_objectsize(i8* %obj) nounwind {
-; CHECK-LABEL:    test_objectsize:
-; CHECK-O2:       %bb.0:
-; CHECK-O2:       movb $1, %al
-; CHECK-O2-NEXT:  retq
-  %os = call i64 @llvm.objectsize.i64.p0i8(i8* %obj, i1 false, i1 false, i1 false)
-  %v = call i1 @llvm.is.constant.i64(i64 %os)
-  ret i1 %v
-}

llvm/test/CodeGen/X86/object-size.ll

-; RUN: llc -O0 < %s | FileCheck %s
-
-; ModuleID = 'ts.c'
-target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
-target triple = "x86_64-apple-darwin10.0"
-
-@p = common global i8* null, align 8              ; <i8**> [#uses=4]
-@.str = private constant [3 x i8] c"Hi\00"        ; <[3 x i8]*> [#uses=1]
-
-define void @bar() nounwind ssp {
-entry:
-  %tmp = load i8*, i8** @p                             ; <i8*> [#uses=1]
-  %0 = call i64 @llvm.objectsize.i64.p0i8(i8* %tmp, i1 0) ; <i64> [#uses=1]
-  %cmp = icmp ne i64 %0, -1                       ; <i1> [#uses=1]
-; CHECK: movq $-1, [[RAX:%r..]]
-; CHECK: cmpq $-1, [[RAX]]
-  br i1 %cmp, label %cond.true, label %cond.false
-
-cond.true:                                        ; preds = %entry
-  %tmp1 = load i8*, i8** @p                            ; <i8*> [#uses=1]
-  %tmp2 = load i8*, i8** @p                            ; <i8*> [#uses=1]
-  %1 = call i64 @llvm.objectsize.i64.p0i8(i8* %tmp2, i1 1) ; <i64> [#uses=1]
-  %call = call i8* @__strcpy_chk(i8* %tmp1, i8* getelementptr inbounds ([3 x i8], [3 x i8]* @.str, i32 0, i32 0), i64 %1) ssp ; <i8*> [#uses=1]
-  br label %cond.end
-
-cond.false:                                       ; preds = %entry
-  %tmp3 = load i8*, i8** @p                            ; <i8*> [#uses=1]
-  %call4 = call i8* @__inline_strcpy_chk(i8* %tmp3, i8* getelementptr inbounds ([3 x i8], [3 x i8]* @.str, i32 0, i32 0)) ssp ; <i8*> [#uses=1]
-  br label %cond.end
-
-cond.end:                                         ; preds = %cond.false, %cond.true
-  %cond = phi i8* [ %call, %cond.true ], [ %call4, %cond.false ] ; <i8*> [#uses=0]
-  ret void
-}
-
-declare i64 @llvm.objectsize.i64.p0i8(i8*, i1) nounwind readonly
-
-declare i8* @__strcpy_chk(i8*, i8*, i64) ssp
-
-define internal i8* @__inline_strcpy_chk(i8* %__dest, i8* %__src) nounwind ssp {
-entry:
-  %retval = alloca i8*                            ; <i8**> [#uses=2]
-  %__dest.addr = alloca i8*                       ; <i8**> [#uses=3]
-  %__src.addr = alloca i8*                        ; <i8**> [#uses=2]
-  store i8* %__dest, i8** %__dest.addr
-  store i8* %__src, i8** %__src.addr
-  %tmp = load i8*, i8** %__dest.addr                   ; <i8*> [#uses=1]
-  %tmp1 = load i8*, i8** %__src.addr                   ; <i8*> [#uses=1]
-  %tmp2 = load i8*, i8** %__dest.addr                  ; <i8*> [#uses=1]
-  %0 = call i64 @llvm.objectsize.i64.p0i8(i8* %tmp2, i1 1) ; <i64> [#uses=1]
-  %call = call i8* @__strcpy_chk(i8* %tmp, i8* %tmp1, i64 %0) ssp ; <i8*> [#uses=1]
-  store i8* %call, i8** %retval
-  %1 = load i8*, i8** %retval                          ; <i8*> [#uses=1]
-  ret i8* %1
-}

llvm/test/Other/new-pm-defaults.ll

 ; CHECK-O-NEXT: Running pass: GlobalOptPass
 ; CHECK-O-NEXT: Running pass: GlobalDCEPass
 ; CHECK-O2-LTO-NOT: Running pass: EliminateAvailableExternallyPass
 ; CHECK-O: Running pass: ReversePostOrderFunctionAttrsPass
 ; CHECK-O-NEXT: Running pass: RequireAnalysisPass<{{.*}}GlobalsAA
 ; CHECK-O-NEXT: Running pass: ModuleToFunctionPassAdaptor<{{.*}}PassManager{{.*}}>
 ; CHECK-O-NEXT: Starting llvm::Function pass manager run.
 ; CHECK-O-NEXT: Running pass: Float2IntPass
+; CHECK-O-NEXT: Running pass: LowerConstantIntrinsicsPass on foo
 ; CHECK-EP-VECTORIZER-START-NEXT: Running pass: NoOpFunctionPass
 ; CHECK-O-NEXT: Running pass: FunctionToLoopPassAdaptor<{{.*}}LoopRotatePass
 ; CHECK-O-NEXT: Starting llvm::Function pass manager run.
 ; CHECK-O-NEXT: Running pass: LoopSimplifyPass
 ; CHECK-O-NEXT: Running pass: LCSSAPass
 ; CHECK-O-NEXT: Finished llvm::Function pass manager run.
 ; CHECK-O-NEXT: Running pass: LoopDistributePass
 ; CHECK-O-NEXT: Running pass: LoopVectorizePass

llvm/test/Other/new-pm-thinlto-defaults.ll

 ; CHECK-POSTLINK-O-NEXT: Running pass: GlobalOptPass
 ; CHECK-POSTLINK-O-NEXT: Running pass: GlobalDCEPass
 ; CHECK-POSTLINK-O-NEXT: Running pass: EliminateAvailableExternallyPass
 ; CHECK-POSTLINK-O-NEXT: Running pass: ReversePostOrderFunctionAttrsPass
 ; CHECK-POSTLINK-O-NEXT: Running pass: RequireAnalysisPass<{{.*}}GlobalsAA
 ; CHECK-POSTLINK-O-NEXT: Running pass: ModuleToFunctionPassAdaptor<{{.*}}PassManager{{.*}}>
 ; CHECK-POSTLINK-O-NEXT: Starting llvm::Function pass manager run.
 ; CHECK-POSTLINK-O-NEXT: Running pass: Float2IntPass
+; CHECK-POSTLINK-O-NEXT: Running pass: LowerConstantIntrinsicsPass
 ; CHECK-POSTLINK-O-NEXT: Running pass: FunctionToLoopPassAdaptor<{{.*}}LoopRotatePass
 ; CHECK-POSTLINK-O-NEXT: Starting llvm::Function pass manager run
 ; CHECK-POSTLINK-O-NEXT: Running pass: LoopSimplifyPass
 ; CHECK-POSTLINK-O-NEXT: Running pass: LCSSAPass
 ; CHECK-POSTLINK-O-NEXT: Finished llvm::Function pass manager run
 ; CHECK-POSTLINK-O-NEXT: Running pass: LoopDistributePass
 ; CHECK-POSTLINK-O-NEXT: Running pass: LoopVectorizePass
 ; CHECK-POSTLINK-O-NEXT: Running analysis: BlockFrequencyAnalysis

llvm/test/Other/opt-O2-pipeline.ll

 ; CHECK-NEXT:     Global Variable Optimizer
 ; CHECK-NEXT:       Unnamed pass: implement Pass::getPassName()
 ; CHECK-NEXT:     Dead Global Elimination
 ; CHECK-NEXT:     CallGraph Construction
 ; CHECK-NEXT:     Globals Alias Analysis
 ; CHECK-NEXT:     FunctionPass Manager
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Float to int
+; CHECK-NEXT:       Lower constant intrinsics
+; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Basic Alias Analysis (stateless AA impl)
 ; CHECK-NEXT:       Function Alias Analysis Results
 ; CHECK-NEXT:       Memory SSA
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       Canonicalize natural loops
 ; CHECK-NEXT:       LCSSA Verifier
 ; CHECK-NEXT:       Loop-Closed SSA Form Pass
 ; CHECK-NEXT:       Scalar Evolution Analysis

llvm/test/Other/opt-O3-pipeline.ll

 ; CHECK-NEXT:     Global Variable Optimizer
 ; CHECK-NEXT:       Unnamed pass: implement Pass::getPassName()
 ; CHECK-NEXT:     Dead Global Elimination
 ; CHECK-NEXT:     CallGraph Construction
 ; CHECK-NEXT:     Globals Alias Analysis
 ; CHECK-NEXT:     FunctionPass Manager
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Float to int
+; CHECK-NEXT:       Lower constant intrinsics
+; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Basic Alias Analysis (stateless AA impl)
 ; CHECK-NEXT:       Function Alias Analysis Results
 ; CHECK-NEXT:       Memory SSA
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       Canonicalize natural loops
 ; CHECK-NEXT:       LCSSA Verifier
 ; CHECK-NEXT:       Loop-Closed SSA Form Pass
 ; CHECK-NEXT:       Scalar Evolution Analysis

llvm/test/Other/opt-Os-pipeline.ll

 ; CHECK-NEXT:     Global Variable Optimizer
 ; CHECK-NEXT:       Unnamed pass: implement Pass::getPassName()
 ; CHECK-NEXT:     Dead Global Elimination
 ; CHECK-NEXT:     CallGraph Construction
 ; CHECK-NEXT:     Globals Alias Analysis
 ; CHECK-NEXT:     FunctionPass Manager
 ; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Float to int
+; CHECK-NEXT:       Lower constant intrinsics
+; CHECK-NEXT:       Dominator Tree Construction
 ; CHECK-NEXT:       Basic Alias Analysis (stateless AA impl)
 ; CHECK-NEXT:       Function Alias Analysis Results
 ; CHECK-NEXT:       Memory SSA
 ; CHECK-NEXT:       Natural Loop Information
 ; CHECK-NEXT:       Canonicalize natural loops
 ; CHECK-NEXT:       LCSSA Verifier
 ; CHECK-NEXT:       Loop-Closed SSA Form Pass
 ; CHECK-NEXT:       Scalar Evolution Analysis

llvm/test/Transforms/CodeGenPrepare/X86/overflow-intrinsics.ll

true:
   %svalue = add i64 %key, -1
   store i64 %svalue, i64* %p64
   br label %exit
 
 exit:
   ret void
 }
 
-; This was crashing when trying to delay instruction removal/deletion.
-
-declare i64 @llvm.objectsize.i64.p0i8(i8*, i1 immarg, i1 immarg, i1 immarg) #0
-
-define hidden fastcc void @crash() {
-; CHECK-LABEL: @crash(
-; CHECK-NEXT:    [[TMP1:%.*]] = call { i64, i1 } @llvm.uadd.with.overflow.i64(i64 undef, i64 undef)
-; CHECK-NEXT:    [[MATH:%.*]] = extractvalue { i64, i1 } [[TMP1]], 0
-; CHECK-NEXT:    [[OV:%.*]] = extractvalue { i64, i1 } [[TMP1]], 1
-; CHECK-NEXT:    [[T2:%.*]] = select i1 undef, i1 undef, i1 [[OV]]
-; CHECK-NEXT:    unreachable
-;
-  %t0 = add i64 undef, undef
-  %t1 = icmp ult i64 %t0, undef
-  %t2 = select i1 undef, i1 undef, i1 %t1
-  %t3 = call i64 @llvm.objectsize.i64.p0i8(i8* nonnull undef, i1 false, i1 false, i1 false)
-  %t4 = icmp ugt i64 %t3, 7
-  unreachable
-}
-
 ; Check that every instruction inserted by -codegenprepare has a debug location.
 ; DEBUG: CheckModuleDebugify: PASS
 

llvm/test/Transforms/CodeGenPrepare/builtin-condition.ll

-; RUN: opt -codegenprepare -S  < %s | FileCheck %s
-
-; Ensure we act sanely on overflow.
-; CHECK-LABEL: define i32 @bar
-define i32 @bar() {
-entry:
-  ; CHECK: ret i32 -1
-  %az = alloca [2147483649 x i32], align 16
-  %a = alloca i8*, align 8
-  %arraydecay = getelementptr inbounds [2147483649 x i32], [2147483649 x i32]* %az, i32 0, i32 0
-  %0 = bitcast i32* %arraydecay to i8*
-  store i8* %0, i8** %a, align 8
-  %1 = load i8*, i8** %a, align 8
-  %2 = call i32 @llvm.objectsize.i32.p0i8(i8* %1, i1 false)
-  ret i32 %2
-}
-
-; CHECK-LABEL: define i32 @baz
-define i32 @baz(i32 %n) {
-entry:
-  ; CHECK: ret i32 -1
-  %az = alloca [1 x i32], align 16
-  %bz = alloca [4294967297 x i32], align 16
-  %tobool = icmp ne i32 %n, 0
-  %arraydecay = getelementptr inbounds [1 x i32], [1 x i32]* %az, i64 0, i64 0
-  %arraydecay1 = getelementptr inbounds [4294967297 x i32], [4294967297 x i32]* %bz, i64 0, i64 0
-  %cond = select i1 %tobool, i32* %arraydecay, i32* %arraydecay1
-  %0 = bitcast i32* %cond to i8*
-  %1 = call i32 @llvm.objectsize.i32.p0i8(i8* %0, i1 false)
-  ret i32 %1
-}
-
-declare i32 @llvm.objectsize.i32.p0i8(i8*, i1)
-
-; The following tests were generated by:
-; #include<stdlib.h>
-; #define STATIC_BUF_SIZE 10
-; #define LARGER_BUF_SIZE 30
-;
-; size_t foo1(int flag) {
-;   char *cptr;
-;   char chararray[LARGER_BUF_SIZE];
-;   char chararray2[STATIC_BUF_SIZE];
-;   if(flag)
-;     cptr = chararray2;
-;    else
-;     cptr = chararray;
-;
-;   return  __builtin_object_size(cptr, 2);
-; }
-;
-; size_t foo2(int n) {
-;   char Small[10];
-;   char Large[20];
-;   char *Ptr = n ? Small : Large + 19;
-;   return __builtin_object_size(Ptr, 0);
-; }
-;
-; void foo() {
-;   size_t ret;
-;   size_t ret1;
-;   ret = foo1(0);
-;   ret1 = foo2(0);
-;   printf("\n%d %d\n", ret, ret1);
-; }
-
-target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
-target triple = "x86_64-unknown-linux-gnu"
-
-@.str = private unnamed_addr constant [8 x i8] c"\0A%d %d\0A\00", align 1
-
-define i64 @foo1(i32 %flag) {
-entry:
-  %chararray = alloca [30 x i8], align 16
-  %chararray2 = alloca [10 x i8], align 1
-  %0 = getelementptr inbounds [30 x i8], [30 x i8]* %chararray, i64 0, i64 0
-  call void @llvm.lifetime.start.p0i8(i64 30, i8* %0)
-  %1 = getelementptr inbounds [10 x i8], [10 x i8]* %chararray2, i64 0, i64 0
-  call void @llvm.lifetime.start.p0i8(i64 10, i8* %1)
-  %tobool = icmp eq i32 %flag, 0
-  %cptr.0 = select i1 %tobool, i8* %0, i8* %1
-  %2 = call i64 @llvm.objectsize.i64.p0i8(i8* %cptr.0, i1 true)
-  call void @llvm.lifetime.end.p0i8(i64 10, i8* %1)
-  call void @llvm.lifetime.end.p0i8(i64 30, i8* %0)
-  ret i64 %2
-; CHECK-LABEL: foo1
-; CHECK:  ret i64 10
-}
-
-declare void @llvm.lifetime.start.p0i8(i64, i8* nocapture)
-
-declare i64 @llvm.objectsize.i64.p0i8(i8*, i1)
-
-declare void @llvm.lifetime.end.p0i8(i64, i8* nocapture)
-
-define i64 @foo2(i32 %n) {
-entry:
-  %Small = alloca [10 x i8], align 1
-  %Large = alloca [20 x i8], align 16
-  %0 = getelementptr inbounds [10 x i8], [10 x i8]* %Small, i64 0, i64 0
-  call void @llvm.lifetime.start.p0i8(i64 10, i8* %0)
-  %1 = getelementptr inbounds [20 x i8], [20 x i8]* %Large, i64 0, i64 0
-  call void @llvm.lifetime.start.p0i8(i64 20, i8* %1)
-  %tobool = icmp ne i32 %n, 0
-  %add.ptr = getelementptr inbounds [20 x i8], [20 x i8]* %Large, i64 0, i64 19
-  %cond = select i1 %tobool, i8* %0, i8* %add.ptr
-  %2 = call i64 @llvm.objectsize.i64.p0i8(i8* %cond, i1 false)
-  call void @llvm.lifetime.end.p0i8(i64 20, i8* %1)
-  call void @llvm.lifetime.end.p0i8(i64 10, i8* %0)
-  ret i64 %2
-; CHECK-LABEL: foo2
-; CHECK:  ret i64 10
-}
-
-define void @foo() {
-entry:
-  %call = tail call i64 @foo1(i32 0)
-  %call1 = tail call i64 @foo2(i32 0)
-  %call2 = tail call i32 (i8*, ...) @printf(i8* getelementptr inbounds ([8 x i8], [8 x i8]* @.str, i64 0, i64 0), i64 %call, i64 %call1)
-  ret void
-}
-
-declare i32 @printf(i8* nocapture readonly, ...)

llvm/test/Transforms/LowerConstantIntrinsics/constant-intrinsics.ll

+; RUN: opt -lower-constant-intrinsics -S < %s | FileCheck %s
+
+;; Ensure that an unfoldable is.constant gets lowered reasonably in
+;; optimized codegen, in particular, that the "true" branch is
+;; eliminated.
+
+;; Also ensure that any unfoldable objectsize is resolved in order.
+
+;; CHECK-NOT: tail call i32 @subfun_1()
+;; CHECK:     tail call i32 @subfun_2()
+;; CHECK-NOT: tail call i32 @subfun_1()
+
+declare i1 @llvm.is.constant.i32(i32 %a) nounwind readnone
+declare i1 @llvm.is.constant.i64(i64 %a) nounwind readnone
+declare i1 @llvm.is.constant.i256(i256 %a) nounwind readnone
+declare i1 @llvm.is.constant.v2i64(<2 x i64> %a) nounwind readnone
+declare i1 @llvm.is.constant.f32(float %a) nounwind readnone
+declare i1 @llvm.is.constant.sl_i32i32s({i32, i32} %a) nounwind readnone
+declare i1 @llvm.is.constant.a2i64([2 x i64] %a) nounwind readnone
+declare i1 @llvm.is.constant.p0i64(i64* %a) nounwind readnone
+
+declare i64 @llvm.objectsize.i64.p0i8(i8*, i1, i1, i1) nounwind readnone
+
+declare i32 @subfun_1()
+declare i32 @subfun_2()
+
+define i32 @test_branch(i32 %in) nounwind {
+  %v = call i1 @llvm.is.constant.i32(i32 %in)
+  br i1 %v, label %True, label %False
+
+True:
+  %call1 = tail call i32 @subfun_1()
+  ret i32 %call1
+
+False:
+  %call2 = tail call i32 @subfun_2()
+  ret i32 %call2
+}
+
+;; llvm.objectsize is another tricky case which gets folded to -1 very
+;; late in the game. We'd like to ensure that llvm.is.constant of
+;; llvm.objectsize is true.
+define i1 @test_objectsize(i8* %obj) nounwind {
+;; CHECK-LABEL:    test_objectsize
+;; CHECK-NOT:      llvm.objectsize
+;; CHECK-NOT:      llvm.is.constant
+;; CHECK:          ret i1 true
+  %os = call i64 @llvm.objectsize.i64.p0i8(i8* %obj, i1 false, i1 false, i1 false)
+  %os1 = add i64 %os, 1
+  %v = call i1 @llvm.is.constant.i64(i64 %os1)
+  ret i1 %v
+}
+
+@test_phi_a = dso_local global i32 0, align 4
+declare dso_local i32 @test_phi_b(...)
+
+; Function Attrs: nounwind uwtable
+define dso_local i32 @test_phi() {
+entry:
+  %0 = load i32, i32* @test_phi_a, align 4
+  %1 = tail call i1 @llvm.is.constant.i32(i32 %0)
+  br i1 %1, label %cond.end, label %cond.false
+
+cond.false:                                       ; preds = %entry
+  %call = tail call i32 bitcast (i32 (...)* @test_phi_b to i32 ()*)() #3
+  %.pre = load i32, i32* @test_phi_a, align 4
+  br label %cond.end
+
+cond.end:                                         ; preds = %entry, %cond.false
+  %2 = phi i32 [ %.pre, %cond.false ], [ %0, %entry ]
+  %cond = phi i32 [ %call, %cond.false ], [ 1, %entry ]
+  %cmp = icmp eq i32 %cond, %2
+  br i1 %cmp, label %cond.true1, label %cond.end4
+
+cond.true1:                                       ; preds = %cond.end
+  %call2 = tail call i32 bitcast (i32 (...)* @test_phi_b to i32 ()*)() #3
+  br label %cond.end4
+
+cond.end4:                                        ; preds = %cond.end, %cond.true1
+  ret i32 undef
+}
+
+define i1 @test_various_types(i256 %int, float %float, <2 x i64> %vec, {i32, i32} %struct, [2 x i64] %arr, i64* %ptr) #0 {
+; CHECK-LABEL: @test_various_types(
+; CHECK-NOT: llvm.is.constant
+  %v1 = call i1 @llvm.is.constant.i256(i256 %int)
+  %v2 = call i1 @llvm.is.constant.f32(float %float)
+  %v3 = call i1 @llvm.is.constant.v2i64(<2 x i64> %vec)
+  %v4 = call i1 @llvm.is.constant.sl_i32i32s({i32, i32} %struct)
+  %v5 = call i1 @llvm.is.constant.a2i64([2 x i64] %arr)
+  %v6 = call i1 @llvm.is.constant.p0i64(i64* %ptr)
+
+  %c1 = call i1 @llvm.is.constant.i256(i256 -1)
+  %c2 = call i1 @llvm.is.constant.f32(float 17.0)
+  %c3 = call i1 @llvm.is.constant.v2i64(<2 x i64> <i64 -1, i64 44>)
+  %c4 = call i1 @llvm.is.constant.sl_i32i32s({i32, i32} {i32 -1, i32 32})
+  %c5 = call i1 @llvm.is.constant.a2i64([2 x i64] [i64 -1, i64 32])
+  %c6 = call i1 @llvm.is.constant.p0i64(i64* inttoptr (i32 42 to i64*))
+
+  %x1 = add i1 %v1, %c1
+  %x2 = add i1 %v2, %c2
+  %x3 = add i1 %v3, %c3
+  %x4 = add i1 %v4, %c4
+  %x5 = add i1 %v5, %c5
+  %x6 = add i1 %v6, %c6
+
+  %res2 = add i1 %x1, %x2
+  %res3 = add i1 %res2, %x3
+  %res4 = add i1 %res3, %x4
+  %res5 = add i1 %res4, %x5
+  %res6 = add i1 %res5, %x6
+
+  ret i1 %res6
+}

llvm/test/Transforms/LowerConstantIntrinsics/crash-on-large-allocas.ll

-; RUN: opt -S -codegenprepare %s -o - | FileCheck %s
+; RUN: opt -S -lower-constant-intrinsics %s -o - | FileCheck %s
 ;
 ; Ensure that we don't {crash,return a bad value} when given an alloca larger
 ; than what a pointer can represent.
 
 target datalayout = "p:16:16"
 
 ; CHECK-LABEL: @alloca_overflow_is_unknown(
 define i16 @alloca_overflow_is_unknown() {
   %i = alloca i8, i32 65537
   %j = call i16 @llvm.objectsize.i16.p0i8(i8* %i, i1 false, i1 false, i1 false)
   ; CHECK: ret i16 -1
   ret i16 %j
 }
 
 declare i16 @llvm.objectsize.i16.p0i8(i8*, i1, i1, i1)

llvm/test/Transforms/LowerConstantIntrinsics/objectsize_basic.ll

-; RUN: opt -codegenprepare -S < %s | FileCheck %s
+; RUN: opt -lower-constant-intrinsics -S < %s | FileCheck %s
 
 target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
 target triple = "x86_64-apple-darwin10.0.0"
 
+declare i64 @llvm.objectsize.i64(i8*, i1, i1, i1) nounwind readonly
+declare i64 @llvm.objectsize.i64.p1i8(i8 addrspace(1)*, i1, i1, i1) nounwind readonly
+declare void @llvm.trap() nounwind
+
 ; CHECK-LABEL: @test1(
 ; objectsize should fold to a constant, which causes the branch to fold to an
-; uncond branch. Next, we fold the control flow alltogether.
-; rdar://8785296
+; uncond branch.
 define i32 @test1(i8* %ptr) nounwind ssp noredzone align 2 {
 entry:
   %0 = tail call i64 @llvm.objectsize.i64(i8* %ptr, i1 false, i1 false, i1 false)
   %1 = icmp ugt i64 %0, 3
   br i1 %1, label %T, label %trap
 
 ; CHECK: entry:
-; CHECK-NOT: br label %
+; CHECK-NOT: label %trap
 
 trap:                                             ; preds = %0, %entry
   tail call void @llvm.trap() noreturn nounwind
   unreachable
 
 T:
 ; CHECK: ret i32 4
   ret i32 4

 ; CHECK-LABEL: @test_objectsize_null_known_flag_min_noas0
 define i64 @test_objectsize_null_known_flag_min_noas0() {
 entry:
   ; CHECK: ret i64 0
   %0 = tail call i64 @llvm.objectsize.i64.p1i8(i8 addrspace(1)* null, i1 true,
                                                i1 false, i1 false)
   ret i64 %0
 }
-
-
-declare i64 @llvm.objectsize.i64(i8*, i1, i1, i1) nounwind readonly
-declare i64 @llvm.objectsize.i64.p1i8(i8 addrspace(1)*, i1, i1, i1) nounwind readonly
-
-declare void @llvm.trap() nounwind

llvm/utils/gn/secondary/llvm/lib/Transforms/Scalar/BUILD.gn

sources = [
     "LoopStrengthReduce.cpp",
     "LoopUnrollAndJamPass.cpp",
     "LoopUnrollPass.cpp",
     "LoopUnswitch.cpp",
     "LoopVersioningLICM.cpp",
     "LowerAtomic.cpp",
     "LowerExpectIntrinsic.cpp",
     "LowerGuardIntrinsic.cpp",
+    "LowerConstantIntrinsics.cpp",
     "LowerWidenableCondition.cpp",
     "MakeGuardsExplicit.cpp",
     "MemCpyOptimizer.cpp",
     "MergeICmps.cpp",
     "MergedLoadStoreMotion.cpp",
     "NaryReassociate.cpp",
     "NewGVN.cpp",
     "PartiallyInlineLibCalls.cpp",