[OpaquePtr] Forbid mixing typed and opaque pointers

[nikic]

Summary

Currently, opaque pointers are supported in two forms: The -force-opaque-pointers mode, where all pointers are opaque and typed pointers do not exist. And as a simple ptr type that can coexist with typed pointers.

I would like to propose that we remove support for the mixed mode. You either get typed pointers, or you get opaque pointers, but not both. In the (current) default mode, using ptr is forbidden. In -opaque-pointers mode, all pointers are opaque.

The motivation here is that the mixed mode introduces additional issues that don't exist in fully opaque mode. D105155 is an example of a design problem. Looking at D109259, it would probably need additional work to support mixed mode (e.g. to generate GEPs for typed base but opaque result). Mixed mode will also end up inserting many casts between i8* and ptr, which would require significant additional work to consistently avoid.

I don't think the mixed mode is particularly valuable, as it doesn't align with our end goal. The only thing I've found it to be moderately useful for is adding some opaque pointer tests in between typed pointer tests, but I think we can live without that.

[nikic]

Rebase, fix more codegen tests.

[nikic]

Add LLVMContext::enableOpaquePointers() to allow enabling opaque pointers in unit tests without having to touch CLI params.

[aeubanks]

I think this makes sense, but this may affect how we'll mass-update tests so we should figure out how we're going to transition tests to opaque pointers.

We could make copies of most test but with opaque pointers. I don't like this because if people have to update tests, they'll be updating twice as many. And keeping the copies in sync with the originals will be all but impossible.
We could slowly start adding --opaque-pointers to RUN lines and update tests. Then only when we have different code paths for opaque vs typed pointers do we have duplicate tests. My idea is that if things work with opaque pointers, they should probably also work with typed pointers. @dblaikie wasn't a fan of this due to us potentially losing test coverage for typed pointers.
We could have a bot running all tests with --opaque-pointers and continuously gather crashes. We won't be able to use the number of test failures since CHECK lines will still be wrong. Then mass-migrate all tests in the same change where we flip --opaque-pointers. Personally I don't like a humongous change, even if it's automatable. We'd also have to update a bunch of non-llvm tests in the same change, like clang tests.

Thoughts?

[dblaikie]

In D109290#2988234, @aeubanks wrote:

I think this makes sense, but this may affect how we'll mass-update tests so we should figure out how we're going to transition tests to opaque pointers.

We could make copies of most test but with opaque pointers. I don't like this because if people have to update tests, they'll be updating twice as many. And keeping the copies in sync with the originals will be all but impossible.
We could slowly start adding --opaque-pointers to RUN lines and update tests. Then only when we have different code paths for opaque vs typed pointers do we have duplicate tests. My idea is that if things work with opaque pointers, they should probably also work with typed pointers. @dblaikie wasn't a fan of this due to us potentially losing test coverage for typed pointers.
We could have a bot running all tests with --opaque-pointers and continuously gather crashes. We won't be able to use the number of test failures since CHECK lines will still be wrong. Then mass-migrate all tests in the same change where we flip --opaque-pointers. Personally I don't like a humongous change, even if it's automatable. We'd also have to update a bunch of non-llvm tests in the same change, like clang tests.

Thoughts?

My preference was, hopefully, to do one major test transition before the opaque pointer flag-flip to make the tests opaque-agnostic (using regex matches for pointer types so they could match typed or untyped pointers - wouldn't even be averse to makiyng some kind of pre-canned regex in FileCheck for this purpose given the size of the effort involved) - under the theory that we are going to have to do a big test cleanup one way or another, and this way we could avoid losing production test coverage pre-flip, and avoid the flip requiring a huge amount of change. Depends how much more expensive it is to transition tests to be typed-agnostic, compared to just transitioning them from typed to opaque explicitly. If it's not a /lot/ more work, I think it's worth it to keep the production test coverage in the interim.

But if folks (llvm-dev thread may be needed) are generally OK with losing production test coverage in the interim, just migrating to opaque pointer-based testing might be the way to go despite my aversion.

[nikic]

For most codegen tests (that produce assembly), output for opaque pointers is the same, so that's an easy case :) For IR tests that contain pointers but don't do anything opaque pointer specific, an idea I had was to use these two run lines:

; RUN: opt -S -passes=xxx -opaque-pointers < %s | FileCheck %s
; RUN: opt -S -passes=xxx < %s | opt -S -opaque-pointers | FileCheck %s

What the second line does is run the test with typed pointers and then strip pointer types afterwards, which should usually give it the same output as working with opaque pointers from the start. This way we won't be testing the exact output under typed pointers anymore, but we do test that all pointer types are consistent (due to verifier after first opt invocation), and I think that's all that really matters.

This should allow us to gradually enable opaque pointer testing while ensuring that we're not missing any typed pointer bitcasts.

[dblaikie]

In D109290#2988806, @nikic wrote:

For most codegen tests (that produce assembly), output for opaque pointers is the same, so that's an easy case :) For IR tests that contain pointers but don't do anything opaque pointer specific, an idea I had was to use these two run lines:

; RUN: opt -S -passes=xxx -opaque-pointers < %s | FileCheck %s
; RUN: opt -S -passes=xxx < %s | opt -S -opaque-pointers | FileCheck %s

What the second line does is run the test with typed pointers and then strip pointer types afterwards, which should usually give it the same output as working with opaque pointers from the start. This way we won't be testing the exact output under typed pointers anymore, but we do test that all pointer types are consistent (due to verifier after first opt invocation), and I think that's all that really matters.

This should allow us to gradually enable opaque pointer testing while ensuring that we're not missing any typed pointer bitcasts.

triples the number of tool invocations, though - which seems a bit concerning. Maybe if we built the option to do the -S -opaque-pointers dance into opt that'd be better for test execution performance?

& having both opaque and non-opaque testing on for everyone (2 x commands for every test) might be a bit much - I was thinking more likely a buildbot/mode where the tests could be run in opaque mode instead, maybe with an "known good" list/marker/indicator of some kind (I guess maybe whatever opt "opacify typed pointers on output" flag could also be seen as an indicator that the test was also acceptable to run in true opaque pointers mod).

[aeubanks]

In D109290#2990260, @dblaikie wrote:

In D109290#2988806, @nikic wrote:

For most codegen tests (that produce assembly), output for opaque pointers is the same, so that's an easy case :) For IR tests that contain pointers but don't do anything opaque pointer specific, an idea I had was to use these two run lines:

; RUN: opt -S -passes=xxx -opaque-pointers < %s | FileCheck %s
; RUN: opt -S -passes=xxx < %s | opt -S -opaque-pointers | FileCheck %s

What the second line does is run the test with typed pointers and then strip pointer types afterwards, which should usually give it the same output as working with opaque pointers from the start. This way we won't be testing the exact output under typed pointers anymore, but we do test that all pointer types are consistent (due to verifier after first opt invocation), and I think that's all that really matters.

This should allow us to gradually enable opaque pointer testing while ensuring that we're not missing any typed pointer bitcasts.

triples the number of tool invocations, though - which seems a bit concerning. Maybe if we built the option to do the -S -opaque-pointers dance into opt that'd be better for test execution performance?

& having both opaque and non-opaque testing on for everyone (2 x commands for every test) might be a bit much - I was thinking more likely a buildbot/mode where the tests could be run in opaque mode instead, maybe with an "known good" list/marker/indicator of some kind (I guess maybe whatever opt "opacify typed pointers on output" flag could also be seen as an indicator that the test was also acceptable to run in true opaque pointers mod).

We could even have a flag that runs the pipeline twice on the module, once with opaque pointers, another time with typed pointers and a opaquify pass at the end. But with this approach are we going to have the same IR both ways? e.g. an opaque pointer run won't have no-op bitcasts, but an opaque pointer version of the output of a typed pointer run will still have no-op bitcasts from ptr to ptr. We could remove ptr to ptr bitcasts in the opaquify pass, but not sure if there are other differences like this. I think a good amount of tests would have differing IR for the two types of opaque pointer pipeline runs.

I don't think should start mass-updating tests until we're fairly sure that opaque pointers work everywhere. Otherwise people unfamiliar with opaque pointers will have to start seeing them in tests when the opaque pointer transition could be far from done. And at that point we'd definitely have to make a post to llvm-dev to keep people up to speed.

For a buildbot/mode where we worry about opaque pointers, doesn't -(force-)opaque-pointers do that job? Then we run check-llvm and scan for crashes (we can't find incorrect IR like this). We could have a buildbot that runs this and reports the number of tests with crashes, but I think running that locally every couple days is probably the same.

Anyway I think this change is good.

llvm/include/llvm/AsmParser/LLParser.h

private:
 
     /// Restore the internal name and slot mappings using the mappings that
     /// were created at an earlier parsing stage.
     void restoreParsingState(const SlotMapping *Slots);
 
     /// getGlobalVal - Get a value with the specified name or ID, creating a
     /// forward reference record if needed.  This can return null if the value
     /// exists but does not have the right type.
-    GlobalValue *getGlobalVal(const std::string &N, Type *Ty, LocTy Loc,
-                              bool IsCall);
-    GlobalValue *getGlobalVal(unsigned ID, Type *Ty, LocTy Loc, bool IsCall);
+    GlobalValue *getGlobalVal(const std::string &N, Type *Ty, LocTy Loc);
+    GlobalValue *getGlobalVal(unsigned ID, Type *Ty, LocTy Loc);
 
     /// Get a Comdat with the specified name, creating a forward reference
     /// record if needed.
     Comdat *getComdat(const std::string &Name, LocTy Loc);
 
     // Helper Routines.
     bool parseToken(lltok::Kind T, const char *ErrMsg);
     bool EatIfPresent(lltok::Kind T) {
public:
 
       Function &getFunction() const { return F; }
 
       bool finishFunction();
 
       /// GetVal - Get a value with the specified name or ID, creating a
       /// forward reference record if needed.  This can return null if the value
       /// exists but does not have the right type.
-      Value *getVal(const std::string &Name, Type *Ty, LocTy Loc, bool IsCall);
-      Value *getVal(unsigned ID, Type *Ty, LocTy Loc, bool IsCall);
+      Value *getVal(const std::string &Name, Type *Ty, LocTy Loc);
+      Value *getVal(unsigned ID, Type *Ty, LocTy Loc);
 
       /// setInstName - After an instruction is parsed and inserted into its
       /// basic block, this installs its name.
       bool setInstName(int NameID, const std::string &NameStr, LocTy NameLoc,
                        Instruction *Inst);
 
       /// GetBB - Get a basic block with the specified name or ID, creating a
       /// forward reference record if needed.  This can return null if the value
       /// is not a BasicBlock.
       BasicBlock *getBB(const std::string &Name, LocTy Loc);
       BasicBlock *getBB(unsigned ID, LocTy Loc);
 
       /// DefineBB - Define the specified basic block, which is either named or
       /// unnamed.  If there is an error, this returns null otherwise it returns
       /// the block being defined.
       BasicBlock *defineBB(const std::string &Name, int NameID, LocTy Loc);
 
       bool resolveForwardRefBlockAddresses();
     };
 
     bool convertValIDToValue(Type *Ty, ValID &ID, Value *&V,
-                             PerFunctionState *PFS, bool IsCall);
+                             PerFunctionState *PFS);
 
     Value *checkValidVariableType(LocTy Loc, const Twine &Name, Type *Ty,
-                                  Value *Val, bool IsCall);
+                                  Value *Val);
 
     bool parseConstantValue(Type *Ty, Constant *&C);
     bool parseValue(Type *Ty, Value *&V, PerFunctionState *PFS);
     bool parseValue(Type *Ty, Value *&V, PerFunctionState &PFS) {
       return parseValue(Ty, V, &PFS);
     }
 
     bool parseValue(Type *Ty, Value *&V, LocTy &Loc, PerFunctionState &PFS) {

llvm/lib/AsmParser/LLLexer.cpp

#define TYPEKEYWORD(STR, LLVMTY)                                               \
   TYPEKEYWORD("x86_fp80",  Type::getX86_FP80Ty(Context));
   TYPEKEYWORD("fp128",     Type::getFP128Ty(Context));
   TYPEKEYWORD("ppc_fp128", Type::getPPC_FP128Ty(Context));
   TYPEKEYWORD("label",     Type::getLabelTy(Context));
   TYPEKEYWORD("metadata",  Type::getMetadataTy(Context));
   TYPEKEYWORD("x86_mmx",   Type::getX86_MMXTy(Context));
   TYPEKEYWORD("x86_amx",   Type::getX86_AMXTy(Context));
   TYPEKEYWORD("token",     Type::getTokenTy(Context));
-  TYPEKEYWORD("ptr", PointerType::getUnqual(Context));
+
+  if (Keyword == "ptr") {
+    if (Context.supportsTypedPointers()) {
+      Warning("ptr type is only supported in -opaque-pointers mode");
+      return lltok::Error;
+    }
+    TyVal = PointerType::getUnqual(Context);
+    return lltok::Type;
+  }
 
 #undef TYPEKEYWORD
 
   // Keywords for instructions.
 #define INSTKEYWORD(STR, Enum)                                                 \
   do {                                                                         \
     if (Keyword == #STR) {                                                     \
       UIntVal = Instruction::Enum;                                             \

llvm/lib/AsmParser/LLParser.cpp

static inline GlobalValue *createGlobalFwdRef(Module *M, PointerType *PTy) {
   else
     return new GlobalVariable(*M, PTy->getPointerElementType(), false,
                               GlobalValue::ExternalWeakLinkage, nullptr, "",
                               nullptr, GlobalVariable::NotThreadLocal,
                               PTy->getAddressSpace());
 }
 
 Value *LLParser::checkValidVariableType(LocTy Loc, const Twine &Name, Type *Ty,
-                                        Value *Val, bool IsCall) {
+                                        Value *Val) {
   Type *ValTy = Val->getType();
   if (ValTy == Ty)
     return Val;
-  // For calls, we also allow opaque pointers.
-  if (IsCall && ValTy == PointerType::get(Ty->getContext(),
-                                          Ty->getPointerAddressSpace()))
-    return Val;
   if (Ty->isLabelTy())
     error(Loc, "'" + Name + "' is not a basic block");
   else
     error(Loc, "'" + Name + "' defined with type '" +
                    getTypeString(Val->getType()) + "' but expected '" +
                    getTypeString(Ty) + "'");
   return nullptr;
 }
 
 /// getGlobalVal - Get a value with the specified name or ID, creating a
 /// forward reference record if needed.  This can return null if the value
 /// exists but does not have the right type.
 GlobalValue *LLParser::getGlobalVal(const std::string &Name, Type *Ty,
-                                    LocTy Loc, bool IsCall) {
+                                    LocTy Loc) {
   PointerType *PTy = dyn_cast<PointerType>(Ty);
   if (!PTy) {
     error(Loc, "global variable reference must have pointer type");
     return nullptr;
   }
 
   // Look this name up in the normal function symbol table.
   GlobalValue *Val =
     cast_or_null<GlobalValue>(M->getValueSymbolTable().lookup(Name));
 
   // If this is a forward reference for the value, see if we already created a
   // forward ref record.
   if (!Val) {
     auto I = ForwardRefVals.find(Name);
     if (I != ForwardRefVals.end())
       Val = I->second.first;
   }
 
   // If we have the value in the symbol table or fwd-ref table, return it.
   if (Val)
     return cast_or_null<GlobalValue>(
-        checkValidVariableType(Loc, "@" + Name, Ty, Val, IsCall));
+        checkValidVariableType(Loc, "@" + Name, Ty, Val));
 
   // Otherwise, create a new forward reference for this value and remember it.
   GlobalValue *FwdVal = createGlobalFwdRef(M, PTy);
   ForwardRefVals[Name] = std::make_pair(FwdVal, Loc);
   return FwdVal;
 }
 
-GlobalValue *LLParser::getGlobalVal(unsigned ID, Type *Ty, LocTy Loc,
-                                    bool IsCall) {
+GlobalValue *LLParser::getGlobalVal(unsigned ID, Type *Ty, LocTy Loc) {
   PointerType *PTy = dyn_cast<PointerType>(Ty);
   if (!PTy) {
     error(Loc, "global variable reference must have pointer type");
     return nullptr;
   }
 
   GlobalValue *Val = ID < NumberedVals.size() ? NumberedVals[ID] : nullptr;
 
   // If this is a forward reference for the value, see if we already created a
   // forward ref record.
   if (!Val) {
     auto I = ForwardRefValIDs.find(ID);
     if (I != ForwardRefValIDs.end())
       Val = I->second.first;
   }
 
   // If we have the value in the symbol table or fwd-ref table, return it.
   if (Val)
     return cast_or_null<GlobalValue>(
-        checkValidVariableType(Loc, "@" + Twine(ID), Ty, Val, IsCall));
+        checkValidVariableType(Loc, "@" + Twine(ID), Ty, Val));
 
   // Otherwise, create a new forward reference for this value and remember it.
   GlobalValue *FwdVal = createGlobalFwdRef(M, PTy);
   ForwardRefValIDs[ID] = std::make_pair(FwdVal, Loc);
   return FwdVal;
 }
 
 //===----------------------------------------------------------------------===//
bool LLParser::parseType(Type *&Result, const Twine &Msg, bool AllowVoid) {
   switch (Lex.getKind()) {
   default:
     return tokError(Msg);
   case lltok::Type:
     // Type ::= 'float' | 'void' (etc)
     Result = Lex.getTyVal();
     Lex.Lex();
 
-    // Handle (explicit) opaque pointer types (not --force-opaque-pointers).
+    // Handle "ptr" opaque pointer type.
     //
     // Type ::= ptr ('addrspace' '(' uint32 ')')?
     if (Result->isOpaquePointerTy()) {
       unsigned AddrSpace;
       if (parseOptionalAddrSpace(AddrSpace))
         return true;
       Result = PointerType::get(getContext(), AddrSpace);
 
return P.error(ForwardRefValIDs.begin()->second.second,
                        Twine(ForwardRefValIDs.begin()->first) + "'");
   return false;
 }
 
 /// getVal - Get a value with the specified name or ID, creating a
 /// forward reference record if needed.  This can return null if the value
 /// exists but does not have the right type.
 Value *LLParser::PerFunctionState::getVal(const std::string &Name, Type *Ty,
-                                          LocTy Loc, bool IsCall) {
+                                          LocTy Loc) {
   // Look this name up in the normal function symbol table.
   Value *Val = F.getValueSymbolTable()->lookup(Name);
 
   // If this is a forward reference for the value, see if we already created a
   // forward ref record.
   if (!Val) {
     auto I = ForwardRefVals.find(Name);
     if (I != ForwardRefVals.end())
       Val = I->second.first;
   }
 
   // If we have the value in the symbol table or fwd-ref table, return it.
   if (Val)
-    return P.checkValidVariableType(Loc, "%" + Name, Ty, Val, IsCall);
+    return P.checkValidVariableType(Loc, "%" + Name, Ty, Val);
 
   // Don't make placeholders with invalid type.
   if (!Ty->isFirstClassType()) {
     P.error(Loc, "invalid use of a non-first-class type");
     return nullptr;
   }
 
   // Otherwise, create a new forward reference for this value and remember it.
   Value *FwdVal;
   if (Ty->isLabelTy()) {
     FwdVal = BasicBlock::Create(F.getContext(), Name, &F);
   } else {
     FwdVal = new Argument(Ty, Name);
   }
 
   ForwardRefVals[Name] = std::make_pair(FwdVal, Loc);
   return FwdVal;
 }
 
-Value *LLParser::PerFunctionState::getVal(unsigned ID, Type *Ty, LocTy Loc,
-                                          bool IsCall) {
+Value *LLParser::PerFunctionState::getVal(unsigned ID, Type *Ty, LocTy Loc) {
   // Look this name up in the normal function symbol table.
   Value *Val = ID < NumberedVals.size() ? NumberedVals[ID] : nullptr;
 
   // If this is a forward reference for the value, see if we already created a
   // forward ref record.
   if (!Val) {
     auto I = ForwardRefValIDs.find(ID);
     if (I != ForwardRefValIDs.end())
       Val = I->second.first;
   }
 
   // If we have the value in the symbol table or fwd-ref table, return it.
   if (Val)
-    return P.checkValidVariableType(Loc, "%" + Twine(ID), Ty, Val, IsCall);
+    return P.checkValidVariableType(Loc, "%" + Twine(ID), Ty, Val);
 
   if (!Ty->isFirstClassType()) {
     P.error(Loc, "invalid use of a non-first-class type");
     return nullptr;
   }
 
   // Otherwise, create a new forward reference for this value and remember it.
   Value *FwdVal;
bool LLParser::PerFunctionState::setInstName(int NameID,
   return false;
 }
 
 /// getBB - Get a basic block with the specified name or ID, creating a
 /// forward reference record if needed.
 BasicBlock *LLParser::PerFunctionState::getBB(const std::string &Name,
                                               LocTy Loc) {
   return dyn_cast_or_null<BasicBlock>(
-      getVal(Name, Type::getLabelTy(F.getContext()), Loc, /*IsCall=*/false));
+      getVal(Name, Type::getLabelTy(F.getContext()), Loc));
 }
 
 BasicBlock *LLParser::PerFunctionState::getBB(unsigned ID, LocTy Loc) {
   return dyn_cast_or_null<BasicBlock>(
-      getVal(ID, Type::getLabelTy(F.getContext()), Loc, /*IsCall=*/false));
+      getVal(ID, Type::getLabelTy(F.getContext()), Loc));
 }
 
 /// defineBB - Define the specified basic block, which is either named or
 /// unnamed.  If there is an error, this returns null otherwise it returns
 /// the block being defined.
 BasicBlock *LLParser::PerFunctionState::defineBB(const std::string &Name,
                                                  int NameID, LocTy Loc) {
   BasicBlock *BB;

 }
 
 /// parseGlobalValue - parse a global value with the specified type.
 bool LLParser::parseGlobalValue(Type *Ty, Constant *&C) {
   C = nullptr;
   ValID ID;
   Value *V = nullptr;
   bool Parsed = parseValID(ID, /*PFS=*/nullptr, Ty) ||
-                convertValIDToValue(Ty, ID, V, nullptr, /*IsCall=*/false);
+                convertValIDToValue(Ty, ID, V, nullptr);
   if (V && !(C = dyn_cast<Constant>(V)))
     return error(ID.Loc, "global values must be constants");
   return Parsed;
 }
 
 bool LLParser::parseGlobalTypeAndValue(Constant *&V) {
   Type *Ty = nullptr;
   return parseType(Ty) || parseGlobalValue(Ty, V);
bool LLParser::parseMetadata(Metadata *&MD, PerFunctionState *PFS) {
   return false;
 }
 
 //===----------------------------------------------------------------------===//
 // Function Parsing.
 //===----------------------------------------------------------------------===//
 
 bool LLParser::convertValIDToValue(Type *Ty, ValID &ID, Value *&V,
-                                   PerFunctionState *PFS, bool IsCall) {
+                                   PerFunctionState *PFS) {
   if (Ty->isFunctionTy())
     return error(ID.Loc, "functions are not values, refer to them as pointers");
 
   switch (ID.Kind) {
   case ValID::t_LocalID:
     if (!PFS)
       return error(ID.Loc, "invalid use of function-local name");
-    V = PFS->getVal(ID.UIntVal, Ty, ID.Loc, IsCall);
+    V = PFS->getVal(ID.UIntVal, Ty, ID.Loc);
     return V == nullptr;
   case ValID::t_LocalName:
     if (!PFS)
       return error(ID.Loc, "invalid use of function-local name");
-    V = PFS->getVal(ID.StrVal, Ty, ID.Loc, IsCall);
+    V = PFS->getVal(ID.StrVal, Ty, ID.Loc);
     return V == nullptr;
   case ValID::t_InlineAsm: {
     if (!ID.FTy || !InlineAsm::Verify(ID.FTy, ID.StrVal2))
       return error(ID.Loc, "invalid type for inline asm constraint string");
     V = InlineAsm::get(
         ID.FTy, ID.StrVal, ID.StrVal2, ID.UIntVal & 1, (ID.UIntVal >> 1) & 1,
         InlineAsm::AsmDialect((ID.UIntVal >> 2) & 1), (ID.UIntVal >> 3) & 1);
     return false;
   }
   case ValID::t_GlobalName:
-    V = getGlobalVal(ID.StrVal, Ty, ID.Loc, IsCall);
+    V = getGlobalVal(ID.StrVal, Ty, ID.Loc);
     return V == nullptr;
   case ValID::t_GlobalID:
-    V = getGlobalVal(ID.UIntVal, Ty, ID.Loc, IsCall);
+    V = getGlobalVal(ID.UIntVal, Ty, ID.Loc);
     return V == nullptr;
   case ValID::t_APSInt:
     if (!Ty->isIntegerTy())
       return error(ID.Loc, "integer constant must have integer type");
     ID.APSIntVal = ID.APSIntVal.extOrTrunc(Ty->getPrimitiveSizeInBits());
     V = ConstantInt::get(Context, ID.APSIntVal);
     return false;
   case ValID::t_APFloat:
bool LLParser::parseConstantValue(Type *Ty, Constant *&C) {
   switch (ID.Kind) {
   case ValID::t_APSInt:
   case ValID::t_APFloat:
   case ValID::t_Undef:
   case ValID::t_Constant:
   case ValID::t_ConstantStruct:
   case ValID::t_PackedConstantStruct: {
     Value *V;
-    if (convertValIDToValue(Ty, ID, V, /*PFS=*/nullptr, /*IsCall=*/false))
+    if (convertValIDToValue(Ty, ID, V, /*PFS=*/nullptr))
       return true;
     assert(isa<Constant>(V) && "Expected a constant value");
     C = cast<Constant>(V);
     return false;
   }
   case ValID::t_Null:
     C = Constant::getNullValue(Ty);
     return false;
   default:
     return error(Loc, "expected a constant value");
   }
 }
 
 bool LLParser::parseValue(Type *Ty, Value *&V, PerFunctionState *PFS) {
   V = nullptr;
   ValID ID;
   return parseValID(ID, PFS, Ty) ||
-         convertValIDToValue(Ty, ID, V, PFS, /*IsCall=*/false);
+         convertValIDToValue(Ty, ID, V, PFS);
 }
 
 bool LLParser::parseTypeAndValue(Value *&V, PerFunctionState *PFS) {
   Type *Ty = nullptr;
   return parseType(Ty) || parseValue(Ty, V, PFS);
 }
 
 bool LLParser::parseTypeAndBasicBlock(BasicBlock *&BB, LocTy &Loc,
if (BBID.Kind == ValID::t_LocalName)
       BB = getBB(BBID.StrVal, BBID.Loc);
     else
       BB = getBB(BBID.UIntVal, BBID.Loc);
     if (!BB)
       return P.error(BBID.Loc, "referenced value is not a basic block");
 
     Value *ResolvedVal = BlockAddress::get(&F, BB);
     ResolvedVal = P.checkValidVariableType(BBID.Loc, BBID.StrVal, GV->getType(),
-                                           ResolvedVal, false);
+                                           ResolvedVal);
     if (!ResolvedVal)
       return true;
     GV->replaceAllUsesWith(ResolvedVal);
     GV->eraseFromParent();
   }
 
   P.ForwardRefBlockAddresses.erase(Blocks);
   return false;
if (!Ty) {
     Ty = FunctionType::get(RetType, ParamTypes, false);
   }
 
   CalleeID.FTy = Ty;
 
   // Look up the callee.
   Value *Callee;
   if (convertValIDToValue(PointerType::get(Ty, InvokeAddrSpace), CalleeID,
-                          Callee, &PFS, /*IsCall=*/true))
+                          Callee, &PFS))
     return true;
 
   // Set up the Attribute for the function.
   SmallVector<Value *, 8> Args;
   SmallVector<AttributeSet, 8> ArgAttrs;
 
   // Loop through FunctionType's arguments and ensure they are specified
   // correctly.  Also, gather any parameter attributes.
if (!Ty) {
 
     Ty = FunctionType::get(RetType, ParamTypes, false);
   }
 
   CalleeID.FTy = Ty;
 
   // Look up the callee.
   Value *Callee;
-  if (convertValIDToValue(PointerType::getUnqual(Ty), CalleeID, Callee, &PFS,
-                          /*IsCall=*/true))
+  if (convertValIDToValue(PointerType::getUnqual(Ty), CalleeID, Callee, &PFS))
     return true;
 
   // Set up the Attribute for the function.
   SmallVector<Value *, 8> Args;
   SmallVector<AttributeSet, 8> ArgAttrs;
 
   // Loop through FunctionType's arguments and ensure they are specified
   // correctly.  Also, gather any parameter attributes.
if (!Ty) {
     Ty = FunctionType::get(RetType, ParamTypes, false);
   }
 
   CalleeID.FTy = Ty;
 
   // Look up the callee.
   Value *Callee;
   if (convertValIDToValue(PointerType::get(Ty, CallAddrSpace), CalleeID, Callee,
-                          &PFS, /*IsCall=*/true))
+                          &PFS))
     return true;
 
   // Set up the Attribute for the function.
   SmallVector<AttributeSet, 8> Attrs;
 
   SmallVector<Value*, 8> Args;
 
   // Loop through FunctionType's arguments and ensure they are specified

llvm/lib/Bitcode/Reader/BitcodeReader.cpp

case bitc::TYPE_CODE_POINTER: { // POINTER: [pointee type] or
           !PointerType::isValidElementType(ResultTy))
         return error("Invalid type");
       ResultTy = PointerType::get(ResultTy, AddressSpace);
       break;
     }
     case bitc::TYPE_CODE_OPAQUE_POINTER: { // OPAQUE_POINTER: [addrspace]
       if (Record.size() != 1)
         return error("Invalid record");
+      if (Context.supportsTypedPointers())
+        return error(
+            "Opaque pointers are only supported in -opaque-pointers mode");
       unsigned AddressSpace = Record[0];
       ResultTy = PointerType::get(Context, AddressSpace);
       break;
     }
     case bitc::TYPE_CODE_FUNCTION_OLD: {
       // Deprecated, but still needed to read old bitcode files.
       // FUNCTION: [vararg, attrid, retty, paramty x N]
       if (Record.size() < 3)

llvm/lib/IR/Function.cpp

switch (D.Kind) {
     }
     case IITDescriptor::Pointer: {
       PointerType *PT = dyn_cast<PointerType>(Ty);
       if (!PT || PT->getAddressSpace() != D.Pointer_AddressSpace)
         return true;
       if (!PT->isOpaque())
         return matchIntrinsicType(PT->getElementType(), Infos, ArgTys,
                                   DeferredChecks, IsDeferredCheck);
-      // If typed pointers are supported, do not allow using opaque pointer in
-      // place of fixed pointer type. This would make the intrinsic signature
-      // non-unique.
-      if (Ty->getContext().supportsTypedPointers())
-        return true;
       // Consume IIT descriptors relating to the pointer element type.
       while (Infos.front().Kind == IITDescriptor::Pointer)
         Infos = Infos.slice(1);
       Infos = Infos.slice(1);
       return false;
     }
 
     case IITDescriptor::Struct: {
case IITDescriptor::PtrToElt: {
       if (D.getArgumentNumber() >= ArgTys.size())
         return IsDeferredCheck || DeferCheck(Ty);
       VectorType * ReferenceType =
         dyn_cast<VectorType> (ArgTys[D.getArgumentNumber()]);
       PointerType *ThisArgType = dyn_cast<PointerType>(Ty);
 
       if (!ThisArgType || !ReferenceType)
         return true;
-      if (!ThisArgType->isOpaque())
-        return ThisArgType->getElementType() != ReferenceType->getElementType();
-      // If typed pointers are supported, do not allow opaque pointer to ensure
-      // uniqueness.
-      return Ty->getContext().supportsTypedPointers();
+      return !ThisArgType->isOpaqueOrPointeeTypeMatches(
+          ReferenceType->getElementType());
     }
     case IITDescriptor::VecOfAnyPtrsToElt: {
       unsigned RefArgNumber = D.getRefArgNumber();
       if (RefArgNumber >= ArgTys.size()) {
         if (IsDeferredCheck)
           return true;
         // If forward referencing, already add the pointer-vector type and
         // defer the checks for later.

llvm/lib/IR/LLVMContext.cpp

const DiagnosticHandler *LLVMContext::getDiagHandlerPtr() const {
   return pImpl->DiagHandler.get();
 }
 
 std::unique_ptr<DiagnosticHandler> LLVMContext::getDiagnosticHandler() {
   return std::move(pImpl->DiagHandler);
 }
 
 bool LLVMContext::supportsTypedPointers() const {
-  return !pImpl->ForceOpaquePointers;
+  return !pImpl->OpaquePointers;
 }

llvm/lib/IR/LLVMContextImpl.h

#include "llvm/IR/Metadata.def"
   StructTypeSet AnonStructTypes;
   StringMap<StructType*> NamedStructTypes;
   unsigned NamedStructTypesUniqueID = 0;
 
   DenseMap<std::pair<Type *, uint64_t>, ArrayType*> ArrayTypes;
   DenseMap<std::pair<Type *, ElementCount>, VectorType*> VectorTypes;
   // TODO: clean up the following after we no longer support non-opaque pointer
   // types.
-  bool ForceOpaquePointers;
+  bool OpaquePointers;
   DenseMap<Type*, PointerType*> PointerTypes;  // Pointers in AddrSpace = 0
   DenseMap<std::pair<Type*, unsigned>, PointerType*> ASPointerTypes;
 
   /// ValueHandles - This map keeps track of all of the value handles that are
   /// watching a Value*.  The Value::HasValueHandle bit is used to know
   /// whether or not a value has an entry in this map.
   using ValueHandlesTy = DenseMap<Value *, ValueHandleBase *>;
   ValueHandlesTy ValueHandles;

llvm/lib/IR/LLVMContextImpl.cpp

 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/ManagedStatic.h"
 #include <cassert>
 #include <utility>
 
 using namespace llvm;
 
 static cl::opt<bool>
-    ForceOpaquePointersCL("force-opaque-pointers",
-                          cl::desc("Force all pointers to be opaque pointers"),
-                          cl::init(false));
+    OpaquePointersCL("opaque-pointers", cl::desc("Use opaque pointers"),
 
 LLVMContextImpl::LLVMContextImpl(LLVMContext &C)
     : DiagHandler(std::make_unique<DiagnosticHandler>()),
       VoidTy(C, Type::VoidTyID), LabelTy(C, Type::LabelTyID),
       HalfTy(C, Type::HalfTyID), BFloatTy(C, Type::BFloatTyID),
       FloatTy(C, Type::FloatTyID), DoubleTy(C, Type::DoubleTyID),
       MetadataTy(C, Type::MetadataTyID), TokenTy(C, Type::TokenTyID),
       X86_FP80Ty(C, Type::X86_FP80TyID), FP128Ty(C, Type::FP128TyID),
       PPC_FP128Ty(C, Type::PPC_FP128TyID), X86_MMXTy(C, Type::X86_MMXTyID),
       X86_AMXTy(C, Type::X86_AMXTyID), Int1Ty(C, 1), Int8Ty(C, 8),
       Int16Ty(C, 16), Int32Ty(C, 32), Int64Ty(C, 64), Int128Ty(C, 128),
-      ForceOpaquePointers(ForceOpaquePointersCL) {}
+      OpaquePointers(OpaquePointersCL) {}
 
 LLVMContextImpl::~LLVMContextImpl() {
   // NOTE: We need to delete the contents of OwnedModules, but Module's dtor
   // will call LLVMContextImpl::removeModule, thus invalidating iterators into
   // the container. Avoid iterators during this operation:
   while (!OwnedModules.empty())
     delete *OwnedModules.begin();
 

llvm/lib/IR/Type.cpp

 //===----------------------------------------------------------------------===//
 
 PointerType *PointerType::get(Type *EltTy, unsigned AddressSpace) {
   assert(EltTy && "Can't get a pointer to <null> type!");
   assert(isValidElementType(EltTy) && "Invalid type for pointer element!");
 
   LLVMContextImpl *CImpl = EltTy->getContext().pImpl;
 
-  // Create opaque pointer for pointer to opaque pointer.
-  if (CImpl->ForceOpaquePointers || EltTy->isOpaquePointerTy())
+  // Automatically convert typed pointers to opaque pointers.
+  if (CImpl->OpaquePointers)
     return get(EltTy->getContext(), AddressSpace);
 
   // Since AddressSpace #0 is the common case, we special case it.
   PointerType *&Entry = AddressSpace == 0 ? CImpl->PointerTypes[EltTy]
      : CImpl->ASPointerTypes[std::make_pair(EltTy, AddressSpace)];
 
   if (!Entry)
     Entry = new (CImpl->Alloc) PointerType(EltTy, AddressSpace);
   return Entry;
 }
 
 PointerType *PointerType::get(LLVMContext &C, unsigned AddressSpace) {
   LLVMContextImpl *CImpl = C.pImpl;
+  assert(CImpl->OpaquePointers &&
+         "Can only create opaque pointers in opaque pointer mode");
 
   // Since AddressSpace #0 is the common case, we special case it.
   PointerType *&Entry =
       AddressSpace == 0
           ? CImpl->PointerTypes[nullptr]
           : CImpl->ASPointerTypes[std::make_pair(nullptr, AddressSpace)];
 
   if (!Entry)

llvm/test/Assembler/invalid-opaque-ptr-addrspace.ll

-; RUN: not llvm-as < %s -disable-output 2>&1 | FileCheck %s
+; RUN: not llvm-as < %s -opaque-pointers -disable-output 2>&1 | FileCheck %s
 
 ; CHECK: ptr* is invalid - use ptr instead
 define void @f(ptr addrspace(3) %a) {
     %b = bitcast ptr addrspace(3) %a to ptr addrspace(3)*
     ret void
 }

llvm/test/Assembler/invalid-opaque-ptr-double-addrspace.ll

-; RUN: not llvm-as < %s -disable-output 2>&1 | FileCheck %s
+; RUN: not llvm-as < %s -opaque-pointers -disable-output 2>&1 | FileCheck %s
 
 ; CHECK: expected top-level entity
 @g1 = external global ptr addrspace(3) addrspace(4)

llvm/test/Assembler/invalid-opaque-ptr.ll

-; RUN: not llvm-as < %s -disable-output 2>&1 | FileCheck %s
+; RUN: not llvm-as < %s -opaque-pointers -disable-output 2>&1 | FileCheck %s
 
 ; CHECK: ptr* is invalid - use ptr instead
 define void @f(ptr %a) {
     %b = bitcast ptr %a to ptr*
     ret void
 }

llvm/test/Assembler/opaque-ptr-cmpxchg.ll

-; RUN: not llvm-as < %s 2>&1 | FileCheck %s
+; RUN: not llvm-as -opaque-pointers < %s 2>&1 | FileCheck %s
 
 ; CHECK: compare value and new value type do not match
 define void @cmpxchg(ptr %p, i32 %a, i64 %b) {
     %val_success = cmpxchg ptr %p, i32 %a, i64 %b acq_rel monotonic
     ret void
 }

llvm/test/Assembler/opaque-ptr.ll

-; RUN: llvm-as < %s | llvm-dis | llvm-as | llvm-dis | FileCheck %s
-; RUN: verify-uselistorder %s
+; RUN: llvm-as -opaque-pointers < %s | llvm-dis -opaque-pointers | llvm-as -opaque-pointers | llvm-dis -opaque-pointers | FileCheck %s
+; RUN: verify-uselistorder -opaque-pointers %s
 
 ; CHECK: @global = external global ptr
 @global = external global ptr
 
-; CHECK: @fptr1 = external global ptr ()*
-; CHECK: @fptr2 = external global ptr () addrspace(1)*
-; CHECK: @fptr3 = external global ptr () addrspace(1)* addrspace(2)*
+; CHECK: @fptr1 = external global ptr
+; CHECK: @fptr2 = external global ptr addrspace(1)
+; CHECK: @fptr3 = external global ptr addrspace(2)
 @fptr1 = external global ptr ()*
 @fptr2 = external global ptr () addrspace(1)*
 @fptr3 = external global ptr () addrspace(1)* addrspace(2)*
 
 ; CHECK: define ptr @f(ptr %a) {
 ; CHECK:     %b = bitcast ptr %a to ptr
 ; CHECK:     ret ptr %b
 define ptr @f(ptr %a) {

 ; CHECK: define void @call_arg(ptr %p, i32 %a)
 ; CHECK:     call void %p(i32 %a)
 ; CHECK:     ret void
 define void @call_arg(ptr %p, i32 %a) {
   call void %p(i32 %a)
   ret void
 }
 
-; CHECK: define void @invoke(ptr %p) personality void ()* @personality {
+; CHECK: define void @invoke(ptr %p) personality ptr @personality {
 ; CHECK:   invoke void %p()
 ; CHECK:     to label %continue unwind label %cleanup
 declare void @personality()
 define void @invoke(ptr %p) personality void ()* @personality {
   invoke void %p()
     to label %continue unwind label %cleanup
 
 continue:

llvm/test/Assembler/ptr-outside-opaque-pointers-mode.ll

+; RUN: not llvm-as < %s -disable-output 2>&1 | FileCheck %s
+
+; CHECK: warning: ptr type is only supported in -opaque-pointers mode
+; CHECK: error: expected type
+define void @f(ptr %a) {
+    ret void
+}

llvm/test/Assembler/remangle-intrinsic-opaque-ptr.ll

-; RUN: not llvm-as < %s -disable-output 2>&1 | FileCheck %s --check-prefix=TYPED
-; RUN: llvm-as --force-opaque-pointers < %s | llvm-dis --force-opaque-pointers | FileCheck %s --check-prefix=OPAQUE
-
-; An opaque pointer type should not be accepted for an intrinsic that
-; specifies a fixed pointer type, outside of --force-opaque-pointers mode.
-
-define void @test() {
-; TYPED: Intrinsic has incorrect return type!
-; OPAQUE: call ptr @llvm.stacksave()
-  call ptr @llvm.stacksave()
-
-; TYPED: Intrinsic has incorrect argument type!
-; OPAQUE: call <2 x i64> @llvm.masked.expandload.v2i64(ptr null, <2 x i1> zeroinitializer, <2 x i64> zeroinitializer)
-  call <2 x i64> @llvm.masked.expandload.v2i64(ptr null, <2 x i1> zeroinitializer, <2 x i64> zeroinitializer)
-
-  ret void
-}
-
-declare ptr @llvm.stacksave()
-declare <2 x i64> @llvm.masked.expandload.v2i64(ptr, <2 x i1>, <2 x i64>)

llvm/test/CodeGen/AArch64/sve-vscale.ll

 ; RUN: llc -mtriple aarch64 -mattr=+sve -asm-verbose=0 < %s | FileCheck %s
+; RUN: llc -mtriple aarch64 -mattr=+sve -asm-verbose=0 -opaque-pointers < %s | FileCheck %s
 ; RUN: opt -mtriple=aarch64 -codegenprepare -S < %s | llc -mtriple=aarch64 -mattr=+sve -asm-verbose=0 | FileCheck %s
 
 ;
 ; RDVL
 ;
 
 ; CHECK-LABEL: rdvl_i8:
 ; CHECK:       rdvl x0, #1

 
 ; CHECK-LABEL: rdvl_const:
 ; CHECK:       rdvl x0, #1
 ; CHECK-NEXT:  ret
 define i32 @rdvl_const() nounwind {
   ret i32 mul nsw (i32 ptrtoint (<vscale x 1 x i8>* getelementptr (<vscale x 1 x i8>, <vscale x 1 x i8>* null, i64 1) to i32), i32 16)
 }
 
-; CHECK-LABEL: rdvl_const_opaque_ptr:
-; CHECK:       rdvl x0, #1
-; CHECK-NEXT:  ret
-define i32 @rdvl_const_opaque_ptr() nounwind {
-  ret i32 mul nsw (i32 ptrtoint (ptr getelementptr (<vscale x 1 x i8>, ptr null, i64 1) to i32), i32 16)
-}
-
 define i32 @vscale_1() nounwind {
 ; CHECK-LABEL: vscale_1:
 ; CHECK:       rdvl [[TMP:x[0-9]+]], #1
 ; CHECK-NEXT:  lsr  x0, [[TMP]], #4
 ; CHECK-NEXT:  ret
   %vscale = call i32 @llvm.vscale.i32()
   ret i32 %vscale
 }

llvm/test/CodeGen/BPF/CORE/intrinsic-array.ll

 ; RUN: opt -O2 %s | llvm-dis > %t1
 ; RUN: llc -filetype=asm -o - %t1 | FileCheck %s
 ; RUN: llc -mattr=+alu32 -filetype=asm -o - %t1 | FileCheck %s
-; RUN: llc -filetype=asm -force-opaque-pointers -o - %t1 | FileCheck %s
-; RUN: llc -mattr=+alu32 -filetype=asm -force-opaque-pointers -o - %t1 | FileCheck %s
+; RUN: llc -filetype=asm -opaque-pointers -o - %t1 | FileCheck %s
+; RUN: llc -mattr=+alu32 -filetype=asm -opaque-pointers -o - %t1 | FileCheck %s
 ;
 ; Source code:
 ;   #define _(x) (__builtin_preserve_access_index(x))
 ;   struct s { int a; int b; };
 ;   int get_value(const void *addr);
 ;   int test(struct s *arg) { return get_value(_(&arg[2].b)); }
 ; Compiler flag to generate IR:
 ;   clang -target bpf -S -O2 -g -emit-llvm -Xclang -disable-llvm-passes test.c

llvm/test/CodeGen/Hexagon/atomic-opaque-basic.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc -march=hexagon -force-opaque-pointers < %s | FileCheck %s
+; RUN: llc -march=hexagon -opaque-pointers < %s | FileCheck %s
 
 %s.0 = type { i8 }
 @g0 = internal global i8 0, align 1
 
 define void @f0() #0 {
 ; CHECK-LABEL: f0:
 ; CHECK:         .cfi_startproc
 ; CHECK-NEXT:  // %bb.0:

llvm/test/CodeGen/Thumb2/mve-gather-ind32-scaled.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp %s -o - | FileCheck %s
+; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -opaque-pointers %s -o - | FileCheck %s
 
 define arm_aapcs_vfpcc <4 x i32> @zext_scaled_i16_i32(i16* %base, <4 x i32>* %offptr) {
 ; CHECK-LABEL: zext_scaled_i16_i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrw.u32 q1, [r1]
 ; CHECK-NEXT:    vldrh.u32 q0, [r0, q1, uxtw #1]
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x i32>, <4 x i32>* %offptr, align 4
   %ptrs = getelementptr inbounds i16, i16* %base, <4 x i32> %offs
   %gather = call <4 x i16> @llvm.masked.gather.v4i16.v4p0i16(<4 x i16*> %ptrs, i32 2, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i16> undef)
   %gather.zext = zext <4 x i16> %gather to <4 x i32>
   ret <4 x i32> %gather.zext
 }
 
-define arm_aapcs_vfpcc <4 x i32> @zext_scaled_i16_i32_opaque(ptr %base, ptr %offptr) {
-; CHECK-LABEL: zext_scaled_i16_i32_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vldrw.u32 q1, [r1]
-; CHECK-NEXT:    vldrh.u32 q0, [r0, q1, uxtw #1]
-; CHECK-NEXT:    bx lr
-entry:
-  %offs = load <4 x i32>, ptr %offptr, align 4
-  %ptrs = getelementptr inbounds i16, ptr %base, <4 x i32> %offs
-  %gather = call <4 x i16> @llvm.masked.gather.v4i16.v4p0(<4 x ptr> %ptrs, i32 2, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i16> undef)
-  %gather.zext = zext <4 x i16> %gather to <4 x i32>
-  ret <4 x i32> %gather.zext
-}
-
 define arm_aapcs_vfpcc <4 x i32> @sext_scaled_i16_i32(i16* %base, <4 x i32>* %offptr) {
 ; CHECK-LABEL: sext_scaled_i16_i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrw.u32 q1, [r1]
 ; CHECK-NEXT:    vldrh.s32 q0, [r0, q1, uxtw #1]
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x i32>, <4 x i32>* %offptr, align 4

 entry:
   %offs = load <4 x i32>, <4 x i32>* %offptr, align 4
   %i32_ptrs = getelementptr inbounds i32, i32* %base, <4 x i32> %offs
   %ptrs = bitcast <4 x i32*> %i32_ptrs to <4 x float*>
   %gather = call <4 x float> @llvm.masked.gather.v4f32.v4p0f32(<4 x float*> %ptrs, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x float> undef)
   ret <4 x float> %gather
 }
 
-define arm_aapcs_vfpcc <4 x float> @scaled_f32_i32_opaque(ptr %base, ptr %offptr) {
-; CHECK-LABEL: scaled_f32_i32_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vldrw.u32 q1, [r1]
-; CHECK-NEXT:    vldrw.u32 q0, [r0, q1, uxtw #2]
-; CHECK-NEXT:    bx lr
-entry:
-  %offs = load <4 x i32>, ptr %offptr, align 4
-  %i32_ptrs = getelementptr inbounds i32, ptr %base, <4 x i32> %offs
-  %ptrs = bitcast <4 x ptr> %i32_ptrs to <4 x ptr>
-  %gather = call <4 x float> @llvm.masked.gather.v4f32.v4p0(<4 x ptr> %ptrs, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x float> undef)
-  ret <4 x float> %gather
-}
-
 define arm_aapcs_vfpcc <4 x i32> @unsigned_scaled_b_i32_i16(i32* %base, <4 x i16>* %offptr) {
 ; CHECK-LABEL: unsigned_scaled_b_i32_i16:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrh.u32 q1, [r1]
 ; CHECK-NEXT:    vldrw.u32 q0, [r0, q1, uxtw #2]
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x i16>, <4 x i16>* %offptr, align 2

 entry:
   %offs = load <4 x i16>, <4 x i16>* %offptr, align 2
   %offs.sext = sext <4 x i16> %offs to <4 x i32>
   %ptrs = getelementptr inbounds i32, i32* %base, <4 x i32> %offs.sext
   %gather = call <4 x i32> @llvm.masked.gather.v4i32.v4p0i32(<4 x i32*> %ptrs, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i32> undef)
   ret <4 x i32> %gather
 }
 
-define arm_aapcs_vfpcc <4 x i32> @unsigned_scaled_b_i32_i16_opaque(ptr %base, ptr %offptr) {
-; CHECK-LABEL: unsigned_scaled_b_i32_i16_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vldrh.u32 q1, [r1]
-; CHECK-NEXT:    vldrw.u32 q0, [r0, q1, uxtw #2]
-; CHECK-NEXT:    bx lr
-entry:
-  %offs = load <4 x i16>, ptr %offptr, align 2
-  %offs.zext = zext <4 x i16> %offs to <4 x i32>
-  %ptrs = getelementptr inbounds i32, ptr %base, <4 x i32> %offs.zext
-  %gather = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %ptrs, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i32> undef)
-  ret <4 x i32> %gather
-}
-
-define arm_aapcs_vfpcc <4 x i32> @signed_scaled_i32_i16_opaque(ptr %base, ptr %offptr) {
-; CHECK-LABEL: signed_scaled_i32_i16_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vldrh.s32 q1, [r1]
-; CHECK-NEXT:    vldrw.u32 q0, [r0, q1, uxtw #2]
-; CHECK-NEXT:    bx lr
-entry:
-  %offs = load <4 x i16>, ptr %offptr, align 2
-  %offs.sext = sext <4 x i16> %offs to <4 x i32>
-  %ptrs = getelementptr inbounds i32, ptr %base, <4 x i32> %offs.sext
-  %gather = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %ptrs, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i32> undef)
-  ret <4 x i32> %gather
-}
-
 define arm_aapcs_vfpcc <4 x float> @a_unsigned_scaled_f32_i16(i32* %base, <4 x i16>* %offptr) {
 ; CHECK-LABEL: a_unsigned_scaled_f32_i16:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrh.u32 q1, [r1]
 ; CHECK-NEXT:    vldrw.u32 q0, [r0, q1, uxtw #2]
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x i16>, <4 x i16>* %offptr, align 2
entry:
   %ptrs2 = getelementptr inbounds i32, <4 x i32*> %ptrs, i32 5
   %gather = call <4 x i32> @llvm.masked.gather.v4i32.v4p0i32(<4 x i32*> %ptrs2, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i32> undef)
   ret <4 x i32> %gather
 }
 
 define arm_aapcs_vfpcc <4 x i32> @scaled_i32_i32_2gep2(i32* %base) {
 ; CHECK-LABEL: scaled_i32_i32_2gep2:
 ; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    adr r1, .LCPI25_0
+; CHECK-NEXT:    adr r1, .LCPI21_0
 ; CHECK-NEXT:    vldrw.u32 q1, [r1]
 ; CHECK-NEXT:    vldrw.u32 q0, [r0, q1, uxtw #2]
 ; CHECK-NEXT:    bx lr
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.1:
-; CHECK-NEXT:  .LCPI25_0:
+; CHECK-NEXT:  .LCPI21_0:
 ; CHECK-NEXT:    .long 5 @ 0x5
 ; CHECK-NEXT:    .long 8 @ 0x8
 ; CHECK-NEXT:    .long 11 @ 0xb
 ; CHECK-NEXT:    .long 14 @ 0xe
 entry:
   %ptrs = getelementptr inbounds i32, i32* %base, <4 x i32> <i32 0, i32 3, i32 6, i32 9>
   %ptrs2 = getelementptr inbounds i32, <4 x i32*> %ptrs, i32 5
   %gather = call <4 x i32> @llvm.masked.gather.v4i32.v4p0i32(<4 x i32*> %ptrs2, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i32> undef)
   ret <4 x i32> %gather
 }
 
-define arm_aapcs_vfpcc <4 x i32> @scaled_i32_i32_2gep_opaque(ptr %base, ptr %offptr) {
-; CHECK-LABEL: scaled_i32_i32_2gep_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vldrw.u32 q1, [r1]
-; CHECK-NEXT:    vmov.i32 q0, #0x14
-; CHECK-NEXT:    vshl.i32 q1, q1, #2
-; CHECK-NEXT:    vadd.i32 q1, q1, r0
-; CHECK-NEXT:    vadd.i32 q1, q1, q0
-; CHECK-NEXT:    vldrw.u32 q0, [q1]
-; CHECK-NEXT:    bx lr
-entry:
-  %offs = load <4 x i32>, ptr %offptr, align 4
-  %ptrs = getelementptr inbounds i32, ptr %base, <4 x i32> %offs
-  %ptrs2 = getelementptr inbounds i32, <4 x ptr> %ptrs, i32 5
-  %gather = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %ptrs2, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i32> undef)
-  ret <4 x i32> %gather
-}
-
-define arm_aapcs_vfpcc <4 x i32> @scaled_i32_i32_2gep2_opaque(ptr %base) {
-; CHECK-LABEL: scaled_i32_i32_2gep2_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    adr r1, .LCPI27_0
-; CHECK-NEXT:    vldrw.u32 q1, [r1]
-; CHECK-NEXT:    vldrw.u32 q0, [r0, q1, uxtw #2]
-; CHECK-NEXT:    bx lr
-; CHECK-NEXT:    .p2align 4
-; CHECK-NEXT:  @ %bb.1:
-; CHECK-NEXT:  .LCPI27_0:
-; CHECK-NEXT:    .long 5 @ 0x5
-; CHECK-NEXT:    .long 8 @ 0x8
-; CHECK-NEXT:    .long 11 @ 0xb
-; CHECK-NEXT:    .long 14 @ 0xe
-entry:
-  %ptrs = getelementptr inbounds i32, ptr %base, <4 x i32> <i32 0, i32 3, i32 6, i32 9>
-  %ptrs2 = getelementptr inbounds i32, <4 x ptr> %ptrs, i32 5
-  %gather = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %ptrs2, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i32> undef)
-  ret <4 x i32> %gather
-}
-
 declare <4 x i8>  @llvm.masked.gather.v4i8.v4p0i8(<4 x i8*>, i32, <4 x i1>, <4 x i8>)
 declare <4 x i16> @llvm.masked.gather.v4i16.v4p0i16(<4 x i16*>, i32, <4 x i1>, <4 x i16>)
-declare <4 x i16> @llvm.masked.gather.v4i16.v4p0(<4 x ptr>, i32, <4 x i1>, <4 x i16>)
 declare <4 x i32> @llvm.masked.gather.v4i32.v4p0i32(<4 x i32*>, i32, <4 x i1>, <4 x i32>)
-declare <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr>, i32, <4 x i1>, <4 x i32>)
 declare <4 x half> @llvm.masked.gather.v4f16.v4p0f16(<4 x half*>, i32, <4 x i1>, <4 x half>)
 declare <4 x float> @llvm.masked.gather.v4f32.v4p0f32(<4 x float*>, i32, <4 x i1>, <4 x float>)
-declare <4 x float> @llvm.masked.gather.v4f32.v4p0(<4 x ptr>, i32, <4 x i1>, <4 x float>)

llvm/test/CodeGen/Thumb2/mve-gather-ind32-unscaled.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp %s -o - | FileCheck %s
+; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -opaque-pointers %s -o - | FileCheck %s
 
 define arm_aapcs_vfpcc <4 x i32> @zext_unscaled_i8_i32(i8* %base, <4 x i32>* %offptr) {
 ; CHECK-LABEL: zext_unscaled_i8_i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrw.u32 q1, [r1]
 ; CHECK-NEXT:    vldrb.u32 q0, [r0, q1]
 ; CHECK-NEXT:    bx lr
 entry:

 entry:
   %offs = load <4 x i32>, <4 x i32>* %offptr, align 4
   %ptrs = getelementptr inbounds i8, i8* %base, <4 x i32> %offs
   %gather = call <4 x i8> @llvm.masked.gather.v4i8.v4p0i8(<4 x i8*> %ptrs, i32 1, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i8> undef)
   %gather.sext = sext <4 x i8> %gather to <4 x i32>
   ret <4 x i32> %gather.sext
 }
 
-define arm_aapcs_vfpcc <4 x i32> @sext_unscaled_i8_i32_opaque(ptr %base, ptr %offptr) {
-; CHECK-LABEL: sext_unscaled_i8_i32_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vldrw.u32 q1, [r1]
-; CHECK-NEXT:    vldrb.s32 q0, [r0, q1]
-; CHECK-NEXT:    bx lr
-entry:
-  %offs = load <4 x i32>, ptr %offptr, align 4
-  %ptrs = getelementptr inbounds i8, ptr %base, <4 x i32> %offs
-  %gather = call <4 x i8> @llvm.masked.gather.v4i8.v4p0(<4 x ptr> %ptrs, i32 1, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i8> undef)
-  %gather.sext = sext <4 x i8> %gather to <4 x i32>
-  ret <4 x i32> %gather.sext
-}
-
 define arm_aapcs_vfpcc <4 x i32> @zext_unscaled_i16_i32(i8* %base, <4 x i32>* %offptr) {
 ; CHECK-LABEL: zext_unscaled_i16_i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrw.u32 q1, [r1]
 ; CHECK-NEXT:    vldrh.u32 q0, [r0, q1]
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x i32>, <4 x i32>* %offptr, align 4
entry:
   %offs = load <4 x i8>, <4 x i8>* %offptr, align 1
   %offs.zext = zext <4 x i8> %offs to <4 x i32>
   %ptrs = getelementptr inbounds i8, i8* %base, <4 x i32> %offs.zext
   %gather = call <4 x i8> @llvm.masked.gather.v4i8.v4p0i8(<4 x i8*> %ptrs, i32 1, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i8> undef)
   %gather.sext = sext <4 x i8> %gather to <4 x i32>
   ret <4 x i32> %gather.sext
 }
 
-define arm_aapcs_vfpcc <4 x i32> @sext_unsigned_unscaled_i8_i8_opaque(ptr %base, ptr %offptr) {
-; CHECK-LABEL: sext_unsigned_unscaled_i8_i8_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vldrb.u32 q1, [r1]
-; CHECK-NEXT:    vldrb.s32 q0, [r0, q1]
-; CHECK-NEXT:    bx lr
-entry:
-  %offs = load <4 x i8>, ptr %offptr, align 1
-  %offs.zext = zext <4 x i8> %offs to <4 x i32>
-  %ptrs = getelementptr inbounds i8, ptr %base, <4 x i32> %offs.zext
-  %gather = call <4 x i8> @llvm.masked.gather.v4i8.v4p0(<4 x ptr> %ptrs, i32 1, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i8> undef)
-  %gather.sext = sext <4 x i8> %gather to <4 x i32>
-  ret <4 x i32> %gather.sext
-}
-
 ; VLDRW.u32 Qd, [P, 4]
 define arm_aapcs_vfpcc <4 x i32> @qi4(<4 x i32*> %p) {
 ; CHECK-LABEL: qi4:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov.i32 q1, #0x10
 ; CHECK-NEXT:    vadd.i32 q1, q0, q1
 ; CHECK-NEXT:    vldrw.u32 q0, [q1]
 ; CHECK-NEXT:    bx lr
entry:
   ret <4 x i32> %gather
 }
 
 declare <4 x i8>  @llvm.masked.gather.v4i8.v4p0i8(<4 x i8*>, i32, <4 x i1>, <4 x i8>)
 declare <4 x i16> @llvm.masked.gather.v4i16.v4p0i16(<4 x i16*>, i32, <4 x i1>, <4 x i16>)
 declare <4 x i32> @llvm.masked.gather.v4i32.v4p0i32(<4 x i32*>, i32, <4 x i1>, <4 x i32>)
 declare <4 x half> @llvm.masked.gather.v4f16.v4p0f16(<4 x half*>, i32, <4 x i1>, <4 x half>)
 declare <4 x float> @llvm.masked.gather.v4f32.v4p0f32(<4 x float*>, i32, <4 x i1>, <4 x float>)
-
-declare <4 x i8>  @llvm.masked.gather.v4i8.v4p0(<4 x ptr>, i32, <4 x i1>, <4 x i8>)

llvm/test/CodeGen/Thumb2/mve-gather-ptrs.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -enable-arm-maskedldst %s -o - | FileCheck %s
+; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -enable-arm-maskedldst -opaque-pointers %s -o - | FileCheck %s
 
 ; i32
 
 define arm_aapcs_vfpcc <2 x i32> @ptr_v2i32(<2 x i32*>* %offptr) {
 ; CHECK-LABEL: ptr_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    ldrd r1, r0, [r0]
 ; CHECK-NEXT:    ldr r0, [r0]

 ; CHECK-NEXT:    vldrw.u32 q0, [q1]
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x i32*>, <4 x i32*>* %offptr, align 4
   %gather = call <4 x i32> @llvm.masked.gather.v4i32.v4p0i32(<4 x i32*> %offs, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i32> undef)
   ret <4 x i32> %gather
 }
 
-define arm_aapcs_vfpcc <4 x i32> @ptr_v4i32_opaque(<4 x ptr>* %offptr) {
-; CHECK-LABEL: ptr_v4i32_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    vldrw.u32 q0, [q1]
-; CHECK-NEXT:    bx lr
-entry:
-  %offs = load <4 x ptr>, <4 x ptr>* %offptr, align 4
-  %gather = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %offs, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i32> undef)
-  ret <4 x i32> %gather
-}
-
 define arm_aapcs_vfpcc <8 x i32> @ptr_v8i32(<8 x i32*>* %offptr) {
 ; CHECK-LABEL: ptr_v8i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r6, r7, lr}
 ; CHECK-NEXT:    push {r4, r5, r6, r7, lr}
 ; CHECK-NEXT:    vldrw.u32 q0, [r0, #16]
 ; CHECK-NEXT:    vmov r1, r2, d1
 ; CHECK-NEXT:    vmov r3, r12, d0

 ; CHECK-NEXT:    vldrw.u32 q0, [q1]
 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x float*>, <4 x float*>* %offptr, align 4
   %gather = call <4 x float> @llvm.masked.gather.v4f32.v4p0f32(<4 x float*> %offs, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x float> undef)
   ret <4 x float> %gather
 }
 
-define arm_aapcs_vfpcc <4 x float> @ptr_v4f32_opaque(<4 x ptr>* %offptr) {
-; CHECK-LABEL: ptr_v4f32_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    vldrw.u32 q0, [q1]
-; CHECK-NEXT:    bx lr
-entry:
-  %offs = load <4 x ptr>, <4 x ptr>* %offptr, align 4
-  %gather = call <4 x float> @llvm.masked.gather.v4f32.v4p0(<4 x ptr> %offs, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x float> undef)
-  ret <4 x float> %gather
-}
-
 define arm_aapcs_vfpcc <8 x float> @ptr_v8f32(<8 x float*>* %offptr) {
 ; CHECK-LABEL: ptr_v8f32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r4, r5, r7, lr}
 ; CHECK-NEXT:    push {r4, r5, r7, lr}
 ; CHECK-NEXT:    vldrw.u32 q0, [r0]
 ; CHECK-NEXT:    vmov r12, r2, d1
 ; CHECK-NEXT:    vmov lr, r1, d0

 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x i16*>, <4 x i16*>* %offptr, align 4
   %gather = call <4 x i16> @llvm.masked.gather.v4i16.v4p0i16(<4 x i16*> %offs, i32 2, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i16> undef)
   %ext = zext <4 x i16> %gather to <4 x i32>
   ret <4 x i32> %ext
 }
 
-define arm_aapcs_vfpcc <4 x i32> @ptr_v4i16_sext_opaque(<4 x ptr>* %offptr) {
-; CHECK-LABEL: ptr_v4i16_sext_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    movs r1, #0
-; CHECK-NEXT:    vldrh.s32 q0, [r1, q1]
-; CHECK-NEXT:    bx lr
-entry:
-  %offs = load <4 x ptr>, <4 x ptr>* %offptr, align 4
-  %gather = call <4 x i16> @llvm.masked.gather.v4i16.v4p0(<4 x ptr> %offs, i32 2, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i16> undef)
-  %ext = sext <4 x i16> %gather to <4 x i32>
-  ret <4 x i32> %ext
-}
-
-define arm_aapcs_vfpcc <4 x i32> @ptr_v4i16_zext_opaque(<4 x ptr>* %offptr) {
-; CHECK-LABEL: ptr_v4i16_zext_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    movs r1, #0
-; CHECK-NEXT:    vldrh.u32 q0, [r1, q1]
-; CHECK-NEXT:    bx lr
-entry:
-  %offs = load <4 x ptr>, <4 x ptr>* %offptr, align 4
-  %gather = call <4 x i16> @llvm.masked.gather.v4i16.v4p0(<4 x ptr> %offs, i32 2, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i16> undef)
-  %ext = zext <4 x i16> %gather to <4 x i32>
-  ret <4 x i32> %ext
-}
-
 define arm_aapcs_vfpcc <4 x i16> @ptr_v4i16(<4 x i16*>* %offptr) {
 ; CHECK-LABEL: ptr_v4i16:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vldrw.u32 q1, [r0]
 ; CHECK-NEXT:    movs r1, #0
 ; CHECK-NEXT:    vldrh.u32 q0, [r1, q1]
 ; CHECK-NEXT:    bx lr
 entry:

 ; CHECK-LABEL: foo_ptr_p_int32_t:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    bic r2, r2, #15
 ; CHECK-NEXT:    cmp r2, #1
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    poplt {r7, pc}
-; CHECK-NEXT:  .LBB30_1: @ %vector.body.preheader
+; CHECK-NEXT:  .LBB26_1: @ %vector.body.preheader
 ; CHECK-NEXT:    subs r2, #4
 ; CHECK-NEXT:    movs r3, #1
 ; CHECK-NEXT:    add.w lr, r3, r2, lsr #2
-; CHECK-NEXT:  .LBB30_2: @ %vector.body
+; CHECK-NEXT:  .LBB26_2: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q0, [r1], #16
 ; CHECK-NEXT:    vptt.i32 ne, q0, zr
 ; CHECK-NEXT:    vldrwt.u32 q1, [q0]
 ; CHECK-NEXT:    vstrwt.32 q1, [r0], #16
-; CHECK-NEXT:    le lr, .LBB30_2
+; CHECK-NEXT:    le lr, .LBB26_2
 ; CHECK-NEXT:  @ %bb.3: @ %for.end
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %and = and i32 %n, -16
   %cmp11 = icmp sgt i32 %and, 0
   br i1 %cmp11, label %vector.body, label %for.end
 
 vector.body:                                      ; preds = %entry, %vector.body
vector.body:                                      ; preds = %entry, %vector.body
   %index.next = add i32 %index, 4
   %5 = icmp eq i32 %index.next, %and
   br i1 %5, label %for.end, label %vector.body
 
 for.end:                                          ; preds = %vector.body, %entry
   ret void
 }
 
-define void @foo_ptr_p_int32_t_opaque(ptr %dest, ptr %src, i32 %n) {
-; CHECK-LABEL: foo_ptr_p_int32_t_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r7, lr}
-; CHECK-NEXT:    push {r7, lr}
-; CHECK-NEXT:    bic r2, r2, #15
-; CHECK-NEXT:    cmp r2, #1
-; CHECK-NEXT:    it lt
-; CHECK-NEXT:    poplt {r7, pc}
-; CHECK-NEXT:  .LBB31_1: @ %vector.body.preheader
-; CHECK-NEXT:    subs r2, #4
-; CHECK-NEXT:    movs r3, #1
-; CHECK-NEXT:    add.w lr, r3, r2, lsr #2
-; CHECK-NEXT:  .LBB31_2: @ %vector.body
-; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    vldrw.u32 q0, [r1], #16
-; CHECK-NEXT:    vptt.i32 ne, q0, zr
-; CHECK-NEXT:    vldrwt.u32 q1, [q0]
-; CHECK-NEXT:    vstrwt.32 q1, [r0], #16
-; CHECK-NEXT:    le lr, .LBB31_2
-; CHECK-NEXT:  @ %bb.3: @ %for.end
-; CHECK-NEXT:    pop {r7, pc}
-entry:
-  %and = and i32 %n, -16
-  %cmp11 = icmp sgt i32 %and, 0
-  br i1 %cmp11, label %vector.body, label %for.end
-
-vector.body:                                      ; preds = %entry, %vector.body
-  %index = phi i32 [ %index.next, %vector.body ], [ 0, %entry ]
-  %0 = getelementptr inbounds ptr, ptr %src, i32 %index
-  %1 = bitcast ptr %0 to ptr
-  %wide.load = load <4 x ptr>, ptr %1, align 4
-  %2 = icmp ne <4 x ptr> %wide.load, zeroinitializer
-  %wide.masked.gather = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %wide.load, i32 4, <4 x i1> %2, <4 x i32> undef)
-  %3 = getelementptr inbounds i32, ptr %dest, i32 %index
-  %4 = bitcast ptr %3 to ptr
-  call void @llvm.masked.store.v4i32.p0(<4 x i32> %wide.masked.gather, ptr %4, i32 4, <4 x i1> %2)
-  %index.next = add i32 %index, 4
-  %5 = icmp eq i32 %index.next, %and
-  br i1 %5, label %for.end, label %vector.body
-
-for.end:                                          ; preds = %vector.body, %entry
-  ret void
-}
-
 define void @foo_ptr_p_float(float* %dest, float** %src, i32 %n) {
 ; CHECK-LABEL: foo_ptr_p_float:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    .save {r7, lr}
 ; CHECK-NEXT:    push {r7, lr}
 ; CHECK-NEXT:    bic r2, r2, #15
 ; CHECK-NEXT:    cmp r2, #1
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    poplt {r7, pc}
-; CHECK-NEXT:  .LBB32_1: @ %vector.body.preheader
+; CHECK-NEXT:  .LBB27_1: @ %vector.body.preheader
 ; CHECK-NEXT:    subs r2, #4
 ; CHECK-NEXT:    movs r3, #1
 ; CHECK-NEXT:    add.w lr, r3, r2, lsr #2
-; CHECK-NEXT:  .LBB32_2: @ %vector.body
+; CHECK-NEXT:  .LBB27_2: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q0, [r1], #16
 ; CHECK-NEXT:    vptt.i32 ne, q0, zr
 ; CHECK-NEXT:    vldrwt.u32 q1, [q0]
 ; CHECK-NEXT:    vstrwt.32 q1, [r0], #16
-; CHECK-NEXT:    le lr, .LBB32_2
+; CHECK-NEXT:    le lr, .LBB27_2
 ; CHECK-NEXT:  @ %bb.3: @ %for.end
 ; CHECK-NEXT:    pop {r7, pc}
 entry:
   %and = and i32 %n, -16
   %cmp11 = icmp sgt i32 %and, 0
   br i1 %cmp11, label %vector.body, label %for.end
 
 vector.body:                                      ; preds = %entry, %vector.body

 declare <4 x half> @llvm.masked.gather.v4f16.v4p0f16(<4 x half*>, i32, <4 x i1>, <4 x half>)
 declare <8 x half> @llvm.masked.gather.v8f16.v8p0f16(<8 x half*>, i32, <8 x i1>, <8 x half>)
 declare <16 x half> @llvm.masked.gather.v16f16.v16p0f16(<16 x half*>, i32, <16 x i1>, <16 x half>)
 declare <4 x i8> @llvm.masked.gather.v4i8.v4p0i8(<4 x i8*>, i32, <4 x i1>, <4 x i8>)
 declare <8 x i8> @llvm.masked.gather.v8i8.v8p0i8(<8 x i8*>, i32, <8 x i1>, <8 x i8>)
 declare <16 x i8> @llvm.masked.gather.v16i8.v16p0i8(<16 x i8*>, i32, <16 x i1>, <16 x i8>)
 declare <32 x i8> @llvm.masked.gather.v32i8.v32p0i8(<32 x i8*>, i32, <32 x i1>, <32 x i8>)
 declare void @llvm.masked.store.v4i32.p0v4i32(<4 x i32>, <4 x i32>*, i32, <4 x i1>)
-
-declare <4 x float> @llvm.masked.gather.v4f32.v4p0(<4 x ptr>, i32, <4 x i1>, <4 x float>)
-declare <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr>, i32, <4 x i1>, <4 x i32>)
-declare <4 x i16> @llvm.masked.gather.v4i16.v4p0(<4 x ptr>, i32, <4 x i1>, <4 x i16>)
-declare void @llvm.masked.store.v4i32.p0(<4 x i32>, ptr, i32, <4 x i1>)

llvm/test/CodeGen/Thumb2/mve-gather-scatter-optimisation.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp --arm-memtransfer-tploop=allow %s -o - | FileCheck %s
+; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp --arm-memtransfer-tploop=allow -opaque-pointers %s -o - | FileCheck %s
 
 !0 = !{i32 1, !"wchar_size", i32 4}
 !1 = !{i32 1, !"min_enum_size", i32 4}
 !2 = !{!"clang version 11.0.0 ([e-mail]:llvm/llvm-project.git 26f04d01a39a33d73fd23165c208b215bf5c350d)"}
 !3 = !{!4, !4, i64 0}
 !4 = !{!"int", !5, i64 0}
 !5 = !{!"omnipotent char", !6, i64 0}
 !6 = !{!"Simple C/C++ TBAA"}
vector.body:                                      ; preds = %vector.body, %vector.ph
   %vec.ind.next = add <4 x i32> %vec.ind, <i32 8, i32 8, i32 8, i32 8>
   %5 = icmp eq i32 %index.next, %n.vec
   br i1 %5, label %end, label %vector.body
 
 end:
   ret void;
 }
 
-define arm_aapcs_vfpcc void @push_out_mul_add_gather_opaque(ptr noalias nocapture readonly %data, i32* noalias nocapture %dst, i32 %n.vec) {
-; CHECK-LABEL: push_out_mul_add_gather_opaque:
-; CHECK:       @ %bb.0: @ %vector.ph
-; CHECK-NEXT:    adr r3, .LCPI3_0
-; CHECK-NEXT:    vldrw.u32 q0, [r3]
-; CHECK-NEXT:    vadd.i32 q0, q0, r0
-; CHECK-NEXT:  .LBB3_1: @ %vector.body
-; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
-; CHECK-NEXT:    vldrw.u32 q1, [q0, #96]!
-; CHECK-NEXT:    subs r2, #4
-; CHECK-NEXT:    vstrb.8 q1, [r1], #16
-; CHECK-NEXT:    bne .LBB3_1
-; CHECK-NEXT:  @ %bb.2: @ %end
-; CHECK-NEXT:    bx lr
-; CHECK-NEXT:    .p2align 4
-; CHECK-NEXT:  @ %bb.3:
-; CHECK-NEXT:  .LCPI3_0:
-; CHECK-NEXT:    .long 4294967224 @ 0xffffffb8
-; CHECK-NEXT:    .long 4294967248 @ 0xffffffd0
-; CHECK-NEXT:    .long 4294967272 @ 0xffffffe8
-; CHECK-NEXT:    .long 0 @ 0x0
-
-vector.ph:                                        ; preds = %for.body.preheader
-  br label %vector.body
-
-vector.body:                                      ; preds = %vector.body, %vector.ph
-  %index = phi i32 [ 0, %vector.ph ], [ %index.next, %vector.body ]
-  %vec.ind = phi <4 x i32> [ <i32 0, i32 2, i32 4, i32 6>, %vector.ph ], [ %vec.ind.next, %vector.body ]
-  %0 = mul <4 x i32> %vec.ind, <i32 3, i32 3, i32 3, i32 3>
-  %1 = add <4 x i32> %0, <i32 6, i32 6, i32 6, i32 6>
-  %2 = getelementptr inbounds i32, ptr %data, <4 x i32> %1
-  %wide.masked.gather = call <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr> %2, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>, <4 x i32> undef)
-  %3 = getelementptr inbounds i32, i32* %dst, i32 %index
-  %4 = bitcast i32* %3 to <4 x i32>*
-  store <4 x i32> %wide.masked.gather, <4 x i32>* %4, align 4
-  %index.next = add i32 %index, 4
-  %vec.ind.next = add <4 x i32> %vec.ind, <i32 8, i32 8, i32 8, i32 8>
-  %5 = icmp eq i32 %index.next, %n.vec
-  br i1 %5, label %end, label %vector.body
-
-end:
-  ret void;
-}
-
 define arm_aapcs_vfpcc void @push_out_mul_scatter(i32* noalias nocapture readonly %data,
 ; CHECK-LABEL: push_out_mul_scatter:
 ; CHECK:       @ %bb.0: @ %vector.ph
-; CHECK-NEXT:    adr r1, .LCPI4_0
+; CHECK-NEXT:    adr r1, .LCPI3_0
 ; CHECK-NEXT:    vldrw.u32 q1, [r1]
 ; CHECK-NEXT:    vadd.i32 q1, q1, r0
-; CHECK-NEXT:  .LBB4_1: @ %vector.body
+; CHECK-NEXT:  .LBB3_1: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    subs r2, #4
 ; CHECK-NEXT:    vstrw.32 q0, [q1, #96]!
-; CHECK-NEXT:    bne .LBB4_1
+; CHECK-NEXT:    bne .LBB3_1
 ; CHECK-NEXT:  @ %bb.2: @ %end
 ; CHECK-NEXT:    bx lr
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.3:
-; CHECK-NEXT:  .LCPI4_0:
+; CHECK-NEXT:  .LCPI3_0:
 ; CHECK-NEXT:    .long 4294967200 @ 0xffffffa0
 ; CHECK-NEXT:    .long 4294967224 @ 0xffffffb8
 ; CHECK-NEXT:    .long 4294967248 @ 0xffffffd0
 ; CHECK-NEXT:    .long 4294967272 @ 0xffffffe8
                                                   i32* noalias nocapture %dst, i32 %n.vec,
                                                   <4 x i32> %to.store) {
 
 vector.ph:                                        ; preds = %for.body.preheader

 
 end:
   ret void;
 }
 
 define arm_aapcs_vfpcc void @push_out_add_scatter(i32* noalias nocapture readonly %data,
 ; CHECK-LABEL: push_out_add_scatter:
 ; CHECK:       @ %bb.0: @ %vector.ph
-; CHECK-NEXT:    adr r1, .LCPI5_0
+; CHECK-NEXT:    adr r1, .LCPI4_0
 ; CHECK-NEXT:    vldrw.u32 q1, [r1]
 ; CHECK-NEXT:    vadd.i32 q1, q1, r0
-; CHECK-NEXT:  .LBB5_1: @ %vector.body
+; CHECK-NEXT:  .LBB4_1: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    subs r2, #4
 ; CHECK-NEXT:    vstrw.32 q0, [q1, #32]!
-; CHECK-NEXT:    bne .LBB5_1
+; CHECK-NEXT:    bne .LBB4_1
 ; CHECK-NEXT:  @ %bb.2: @ %end
 ; CHECK-NEXT:    bx lr
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.3:
-; CHECK-NEXT:  .LCPI5_0:
+; CHECK-NEXT:  .LCPI4_0:
 ; CHECK-NEXT:    .long 4294967288 @ 0xfffffff8
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 8 @ 0x8
 ; CHECK-NEXT:    .long 16 @ 0x10
                                                   i32* noalias nocapture %dst, i32 %n.vec,
                                                   <4 x i32> %to.store) {
 
 vector.ph:                                        ; preds = %for.body.preheader

 
 end:
   ret void;
 }
 
 define arm_aapcs_vfpcc void @push_out_mul_gather_scatter(i32* noalias nocapture readonly %data,
 ; CHECK-LABEL: push_out_mul_gather_scatter:
 ; CHECK:       @ %bb.0: @ %vector.ph
-; CHECK-NEXT:    adr r1, .LCPI6_0
+; CHECK-NEXT:    adr r1, .LCPI5_0
 ; CHECK-NEXT:    vmov.i32 q0, #0x18
 ; CHECK-NEXT:    vldrw.u32 q1, [r1]
-; CHECK-NEXT:  .LBB6_1: @ %vector.body
+; CHECK-NEXT:  .LBB5_1: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q2, [r0, q1, uxtw #2]
 ; CHECK-NEXT:    vadd.i32 q3, q1, q0
 ; CHECK-NEXT:    subs r2, #4
 ; CHECK-NEXT:    vstrw.32 q2, [r0, q1, uxtw #2]
 ; CHECK-NEXT:    vmov q1, q3
-; CHECK-NEXT:    bne .LBB6_1
+; CHECK-NEXT:    bne .LBB5_1
 ; CHECK-NEXT:  @ %bb.2: @ %end
 ; CHECK-NEXT:    bx lr
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.3:
-; CHECK-NEXT:  .LCPI6_0:
+; CHECK-NEXT:  .LCPI5_0:
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 6 @ 0x6
 ; CHECK-NEXT:    .long 12 @ 0xc
 ; CHECK-NEXT:    .long 18 @ 0x12
                                                          i32* noalias nocapture %dst, i32 %n.vec) {
 
 vector.ph:                                        ; preds = %for.body.preheader
   br label %vector.body

 
 end:
   ret void;
 }
 
 define arm_aapcs_vfpcc void @push_out_add_sub_block(i32* noalias nocapture readonly %data, i32* noalias nocapture %dst, i32 %n.vec) {
 ; CHECK-LABEL: push_out_add_sub_block:
 ; CHECK:       @ %bb.0: @ %vector.ph
-; CHECK-NEXT:    adr r3, .LCPI7_0
+; CHECK-NEXT:    adr r3, .LCPI6_0
 ; CHECK-NEXT:    vldrw.u32 q0, [r3]
 ; CHECK-NEXT:    vadd.i32 q0, q0, r0
-; CHECK-NEXT:  .LBB7_1: @ %vector.body
+; CHECK-NEXT:  .LBB6_1: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q1, [q0, #32]!
 ; CHECK-NEXT:    subs r2, #4
 ; CHECK-NEXT:    vstrb.8 q1, [r1], #16
-; CHECK-NEXT:    bne .LBB7_1
+; CHECK-NEXT:    bne .LBB6_1
 ; CHECK-NEXT:  @ %bb.2: @ %end
 ; CHECK-NEXT:    bx lr
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.3:
-; CHECK-NEXT:  .LCPI7_0:
+; CHECK-NEXT:  .LCPI6_0:
 ; CHECK-NEXT:    .long 4294967288 @ 0xfffffff8
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 8 @ 0x8
 ; CHECK-NEXT:    .long 16 @ 0x10
 
 vector.ph:                                        ; preds = %for.body.preheader
   br label %vector.body
 
end:
   ret void;
 }
 
 define arm_aapcs_vfpcc void @non_gatscat_use1(i32* noalias nocapture readonly %data, i32* noalias nocapture %dst, i32 %n.vec, <4 x i32>* %x) {
 ; CHECK-LABEL: non_gatscat_use1:
 ; CHECK:       @ %bb.0: @ %vector.ph
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
-; CHECK-NEXT:    adr.w r12, .LCPI8_0
+; CHECK-NEXT:    adr.w r12, .LCPI7_0
 ; CHECK-NEXT:    vmov.i32 q0, #0x9
 ; CHECK-NEXT:    vldrw.u32 q3, [r12]
 ; CHECK-NEXT:    vmov.i32 q1, #0xc
 ; CHECK-NEXT:    vmov.i32 q2, #0x8
-; CHECK-NEXT:  .LBB8_1: @ %vector.body
+; CHECK-NEXT:  .LBB7_1: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vadd.i32 q4, q3, q2
 ; CHECK-NEXT:    vmul.i32 q5, q3, q0
 ; CHECK-NEXT:    vmlas.u32 q3, q1, r0
 ; CHECK-NEXT:    subs r2, #4
 ; CHECK-NEXT:    vldrw.u32 q6, [q3, #24]
 ; CHECK-NEXT:    vmov q3, q4
 ; CHECK-NEXT:    vstrw.32 q5, [r3]
 ; CHECK-NEXT:    vstrb.8 q6, [r1], #16
-; CHECK-NEXT:    bne .LBB8_1
+; CHECK-NEXT:    bne .LBB7_1
 ; CHECK-NEXT:  @ %bb.2: @ %end
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13}
 ; CHECK-NEXT:    bx lr
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.3:
-; CHECK-NEXT:  .LCPI8_0:
+; CHECK-NEXT:  .LCPI7_0:
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 2 @ 0x2
 ; CHECK-NEXT:    .long 4 @ 0x4
 ; CHECK-NEXT:    .long 6 @ 0x6
 
 vector.ph:                                        ; preds = %for.body.preheader
   br label %vector.body
 
end:
   ret void;
 }
 
 define arm_aapcs_vfpcc void @non_gatscat_use2(i32* noalias nocapture readonly %data, i32* noalias nocapture %dst, i32 %n.vec, <4 x i32>* %x) {
 ; CHECK-LABEL: non_gatscat_use2:
 ; CHECK:       @ %bb.0: @ %vector.ph
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    adr.w r12, .LCPI9_0
+; CHECK-NEXT:    adr.w r12, .LCPI8_0
 ; CHECK-NEXT:    vmov.i32 q0, #0x12
 ; CHECK-NEXT:    vldrw.u32 q4, [r12]
 ; CHECK-NEXT:    vmov.i32 q1, #0x9
 ; CHECK-NEXT:    vmov.i32 q2, #0x8
 ; CHECK-NEXT:    vmov.i32 q3, #0xc
-; CHECK-NEXT:  .LBB9_1: @ %vector.body
+; CHECK-NEXT:  .LBB8_1: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vadd.i32 q5, q4, q2
 ; CHECK-NEXT:    vmul.i32 q6, q4, q1
 ; CHECK-NEXT:    vmlas.u32 q4, q3, r0
 ; CHECK-NEXT:    subs r2, #4
 ; CHECK-NEXT:    vldrw.u32 q7, [q4, #24]
 ; CHECK-NEXT:    vadd.i32 q4, q6, q0
 ; CHECK-NEXT:    vstrw.32 q4, [r3]
 ; CHECK-NEXT:    vmov q4, q5
 ; CHECK-NEXT:    vstrb.8 q7, [r1], #16
-; CHECK-NEXT:    bne .LBB9_1
+; CHECK-NEXT:    bne .LBB8_1
 ; CHECK-NEXT:  @ %bb.2: @ %end
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    bx lr
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.3:
-; CHECK-NEXT:  .LCPI9_0:
+; CHECK-NEXT:  .LCPI8_0:
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 2 @ 0x2
 ; CHECK-NEXT:    .long 4 @ 0x4
 ; CHECK-NEXT:    .long 6 @ 0x6
 
 vector.ph:                                        ; preds = %for.body.preheader
   br label %vector.body
 

 ; CHECK-NEXT:    movs r6, #1
 ; CHECK-NEXT:    mov.w r8, #0
 ; CHECK-NEXT:    add.w r7, r6, r7, lsr #1
 ; CHECK-NEXT:    vdup.32 q1, r9
 ; CHECK-NEXT:    bic r7, r7, #3
 ; CHECK-NEXT:    vshl.i32 q3, q1, #3
 ; CHECK-NEXT:    subs r7, #4
 ; CHECK-NEXT:    add.w r10, r6, r7, lsr #2
-; CHECK-NEXT:    adr r7, .LCPI10_0
-; CHECK-NEXT:    adr r6, .LCPI10_1
+; CHECK-NEXT:    adr r7, .LCPI9_0
+; CHECK-NEXT:    adr r6, .LCPI9_1
 ; CHECK-NEXT:    vldrw.u32 q2, [r7]
 ; CHECK-NEXT:    vldrw.u32 q0, [r6]
 ; CHECK-NEXT:    vstrw.32 q0, [sp] @ 16-byte Spill
-; CHECK-NEXT:  .LBB10_1: @ %for.cond8.preheader.us.us.preheader
+; CHECK-NEXT:  .LBB9_1: @ %for.cond8.preheader.us.us.preheader
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
-; CHECK-NEXT:    @ Child Loop BB10_2 Depth 2
-; CHECK-NEXT:    @ Child Loop BB10_3 Depth 3
+; CHECK-NEXT:    @ Child Loop BB9_2 Depth 2
+; CHECK-NEXT:    @ Child Loop BB9_3 Depth 3
 ; CHECK-NEXT:    mul r11, r8, r9
 ; CHECK-NEXT:    movs r5, #0
 ; CHECK-NEXT:    mul r7, r8, r12
-; CHECK-NEXT:  .LBB10_2: @ %vector.ph
-; CHECK-NEXT:    @ Parent Loop BB10_1 Depth=1
+; CHECK-NEXT:  .LBB9_2: @ %vector.ph
+; CHECK-NEXT:    @ Parent Loop BB9_1 Depth=1
 ; CHECK-NEXT:    @ => This Loop Header: Depth=2
-; CHECK-NEXT:    @ Child Loop BB10_3 Depth 3
+; CHECK-NEXT:    @ Child Loop BB9_3 Depth 3
 ; CHECK-NEXT:    vdup.32 q5, r7
 ; CHECK-NEXT:    vldrw.u32 q0, [sp] @ 16-byte Reload
 ; CHECK-NEXT:    vshl.i32 q5, q5, #2
 ; CHECK-NEXT:    vmov q6, q1
 ; CHECK-NEXT:    vadd.i32 q5, q5, r0
 ; CHECK-NEXT:    dls lr, r10
 ; CHECK-NEXT:    vmov.i32 q4, #0x0
 ; CHECK-NEXT:    vadd.i32 q5, q5, q0
 ; CHECK-NEXT:    vmlas.u32 q6, q2, r5
-; CHECK-NEXT:  .LBB10_3: @ %vector.body
-; CHECK-NEXT:    @ Parent Loop BB10_1 Depth=1
-; CHECK-NEXT:    @ Parent Loop BB10_2 Depth=2
+; CHECK-NEXT:  .LBB9_3: @ %vector.body
+; CHECK-NEXT:    @ Parent Loop BB9_1 Depth=1
+; CHECK-NEXT:    @ Parent Loop BB9_2 Depth=2
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=3
 ; CHECK-NEXT:    vadd.i32 q7, q6, q3
 ; CHECK-NEXT:    vldrw.u32 q0, [r1, q6, uxtw #2]
 ; CHECK-NEXT:    vldrw.u32 q6, [q5, #32]!
 ; CHECK-NEXT:    vmul.i32 q0, q0, q6
 ; CHECK-NEXT:    vmov q6, q7
 ; CHECK-NEXT:    vadd.i32 q4, q0, q4
-; CHECK-NEXT:    le lr, .LBB10_3
+; CHECK-NEXT:    le lr, .LBB9_3
 ; CHECK-NEXT:  @ %bb.4: @ %middle.block
-; CHECK-NEXT:    @ in Loop: Header=BB10_2 Depth=2
+; CHECK-NEXT:    @ in Loop: Header=BB9_2 Depth=2
 ; CHECK-NEXT:    add.w r4, r5, r11
 ; CHECK-NEXT:    adds r5, #1
 ; CHECK-NEXT:    vaddv.u32 r6, q4
 ; CHECK-NEXT:    cmp r5, r9
 ; CHECK-NEXT:    str.w r6, [r2, r4, lsl #2]
-; CHECK-NEXT:    bne .LBB10_2
+; CHECK-NEXT:    bne .LBB9_2
 ; CHECK-NEXT:  @ %bb.5: @ %for.cond4.for.cond.cleanup6_crit_edge.us
-; CHECK-NEXT:    @ in Loop: Header=BB10_1 Depth=1
+; CHECK-NEXT:    @ in Loop: Header=BB9_1 Depth=1
 ; CHECK-NEXT:    add.w r8, r8, #1
 ; CHECK-NEXT:    cmp r8, r3
-; CHECK-NEXT:    bne .LBB10_1
+; CHECK-NEXT:    bne .LBB9_1
 ; CHECK-NEXT:  @ %bb.6: @ %for.end25
 ; CHECK-NEXT:    add sp, #16
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.7:
-; CHECK-NEXT:  .LCPI10_0:
+; CHECK-NEXT:  .LCPI9_0:
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 2 @ 0x2
 ; CHECK-NEXT:    .long 4 @ 0x4
 ; CHECK-NEXT:    .long 6 @ 0x6
-; CHECK-NEXT:  .LCPI10_1:
+; CHECK-NEXT:  .LCPI9_1:
 ; CHECK-NEXT:    .long 4294967264 @ 0xffffffe0
 ; CHECK-NEXT:    .long 4294967272 @ 0xffffffe8
 ; CHECK-NEXT:    .long 4294967280 @ 0xfffffff0
 ; CHECK-NEXT:    .long 4294967288 @ 0xfffffff8
 
 for.cond8.preheader.us.us.preheader.preheader:    ; preds = %entry
   %0 = add i32 %l, -1
   %1 = lshr i32 %0, 1

 ; CHECK-NEXT:    sub sp, #32
 ; CHECK-NEXT:    strd r0, r2, [sp, #24] @ 8-byte Folded Spill
 ; CHECK-NEXT:    cmp r3, #0
 ; CHECK-NEXT:    str r3, [sp, #8] @ 4-byte Spill
 ; CHECK-NEXT:    mov r0, r3
 ; CHECK-NEXT:    itt ne
 ; CHECK-NEXT:    ldrne r0, [sp, #136]
 ; CHECK-NEXT:    cmpne r0, #0
-; CHECK-NEXT:    bne .LBB11_2
-; CHECK-NEXT:  .LBB11_1: @ %for.cond.cleanup
+; CHECK-NEXT:    bne .LBB10_2
+; CHECK-NEXT:  .LBB10_1: @ %for.cond.cleanup
 ; CHECK-NEXT:    add sp, #32
 ; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
 ; CHECK-NEXT:    add sp, #4
 ; CHECK-NEXT:    pop.w {r4, r5, r6, r7, r8, r9, r10, r11, pc}
-; CHECK-NEXT:  .LBB11_2: @ %for.cond1.preheader.us.preheader
+; CHECK-NEXT:  .LBB10_2: @ %for.cond1.preheader.us.preheader
 ; CHECK-NEXT:    ldr.w r12, [sp, #140]
 ; CHECK-NEXT:    movs r7, #1
 ; CHECK-NEXT:    mov.w r11, #0
 ; CHECK-NEXT:    vmov.i32 q0, #0x0
 ; CHECK-NEXT:    bic r2, r12, #3
 ; CHECK-NEXT:    subs r3, r2, #4
 ; CHECK-NEXT:    add.w r0, r7, r3, lsr #2
 ; CHECK-NEXT:    ldr r7, [sp, #136]
-; CHECK-NEXT:    adr r3, .LCPI11_0
+; CHECK-NEXT:    adr r3, .LCPI10_0
 ; CHECK-NEXT:    str r0, [sp, #16] @ 4-byte Spill
 ; CHECK-NEXT:    lsl.w r0, r12, #1
 ; CHECK-NEXT:    vdup.32 q1, r7
 ; CHECK-NEXT:    vldrw.u32 q2, [r3]
 ; CHECK-NEXT:    str r0, [sp, #4] @ 4-byte Spill
 ; CHECK-NEXT:    ldr r0, [sp, #24] @ 4-byte Reload
 ; CHECK-NEXT:    lsls r6, r7, #1
 ; CHECK-NEXT:    vshl.i32 q3, q1, #2
 ; CHECK-NEXT:    movs r3, #0
 ; CHECK-NEXT:    str r0, [sp, #20] @ 4-byte Spill
-; CHECK-NEXT:    b .LBB11_5
-; CHECK-NEXT:  .LBB11_3: @ %for.cond5.preheader.us73.preheader
-; CHECK-NEXT:    @ in Loop: Header=BB11_5 Depth=1
+; CHECK-NEXT:    b .LBB10_5
+; CHECK-NEXT:  .LBB10_3: @ %for.cond5.preheader.us73.preheader
+; CHECK-NEXT:    @ in Loop: Header=BB10_5 Depth=1
 ; CHECK-NEXT:    ldr r0, [sp, #28] @ 4-byte Reload
 ; CHECK-NEXT:    add.w r3, r0, r5, lsl #1
-; CHECK-NEXT:    wlstp.8 lr, r6, .LBB11_4
-; CHECK-NEXT:    b .LBB11_15
-; CHECK-NEXT:  .LBB11_4: @ %for.cond1.for.cond.cleanup3_crit_edge.us
-; CHECK-NEXT:    @ in Loop: Header=BB11_5 Depth=1
+; CHECK-NEXT:    wlstp.8 lr, r6, .LBB10_4
+; CHECK-NEXT:    b .LBB10_15
+; CHECK-NEXT:  .LBB10_4: @ %for.cond1.for.cond.cleanup3_crit_edge.us
+; CHECK-NEXT:    @ in Loop: Header=BB10_5 Depth=1
 ; CHECK-NEXT:    ldr r0, [sp, #4] @ 4-byte Reload
 ; CHECK-NEXT:    add r11, r12
 ; CHECK-NEXT:    ldr r3, [sp, #20] @ 4-byte Reload
 ; CHECK-NEXT:    add r3, r0
 ; CHECK-NEXT:    str r3, [sp, #20] @ 4-byte Spill
 ; CHECK-NEXT:    ldr r3, [sp, #12] @ 4-byte Reload
 ; CHECK-NEXT:    ldr r0, [sp, #8] @ 4-byte Reload
 ; CHECK-NEXT:    adds r3, #1
 ; CHECK-NEXT:    cmp r3, r0
-; CHECK-NEXT:    beq .LBB11_1
-; CHECK-NEXT:  .LBB11_5: @ %for.cond1.preheader.us
+; CHECK-NEXT:    beq .LBB10_1
+; CHECK-NEXT:  .LBB10_5: @ %for.cond1.preheader.us
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
-; CHECK-NEXT:    @ Child Loop BB11_8 Depth 2
-; CHECK-NEXT:    @ Child Loop BB11_11 Depth 3
-; CHECK-NEXT:    @ Child Loop BB11_14 Depth 3
-; CHECK-NEXT:    @ Child Loop BB11_15 Depth 2
+; CHECK-NEXT:    @ Child Loop BB10_8 Depth 2
+; CHECK-NEXT:    @ Child Loop BB10_11 Depth 3
+; CHECK-NEXT:    @ Child Loop BB10_14 Depth 3
+; CHECK-NEXT:    @ Child Loop BB10_15 Depth 2
 ; CHECK-NEXT:    mul r5, r3, r7
 ; CHECK-NEXT:    cmp.w r12, #0
 ; CHECK-NEXT:    str r3, [sp, #12] @ 4-byte Spill
-; CHECK-NEXT:    beq .LBB11_3
+; CHECK-NEXT:    beq .LBB10_3
 ; CHECK-NEXT:  @ %bb.6: @ %for.cond5.preheader.us.us.preheader
-; CHECK-NEXT:    @ in Loop: Header=BB11_5 Depth=1
+; CHECK-NEXT:    @ in Loop: Header=BB10_5 Depth=1
 ; CHECK-NEXT:    mov.w r8, #0
-; CHECK-NEXT:    b .LBB11_8
-; CHECK-NEXT:  .LBB11_7: @ %for.cond5.for.cond.cleanup7_crit_edge.us.us
-; CHECK-NEXT:    @ in Loop: Header=BB11_8 Depth=2
+; CHECK-NEXT:    b .LBB10_8
+; CHECK-NEXT:  .LBB10_7: @ %for.cond5.for.cond.cleanup7_crit_edge.us.us
+; CHECK-NEXT:    @ in Loop: Header=BB10_8 Depth=2
 ; CHECK-NEXT:    ldr r3, [sp, #28] @ 4-byte Reload
 ; CHECK-NEXT:    add.w r0, r8, r5
 ; CHECK-NEXT:    add.w r8, r8, #1
 ; CHECK-NEXT:    cmp r8, r7
 ; CHECK-NEXT:    strh.w r10, [r3, r0, lsl #1]
-; CHECK-NEXT:    beq .LBB11_4
-; CHECK-NEXT:  .LBB11_8: @ %for.cond5.preheader.us.us
-; CHECK-NEXT:    @ Parent Loop BB11_5 Depth=1
+; CHECK-NEXT:    beq .LBB10_4
+; CHECK-NEXT:  .LBB10_8: @ %for.cond5.preheader.us.us
+; CHECK-NEXT:    @ Parent Loop BB10_5 Depth=1
 ; CHECK-NEXT:    @ => This Loop Header: Depth=2
-; CHECK-NEXT:    @ Child Loop BB11_11 Depth 3
-; CHECK-NEXT:    @ Child Loop BB11_14 Depth 3
+; CHECK-NEXT:    @ Child Loop BB10_11 Depth 3
+; CHECK-NEXT:    @ Child Loop BB10_14 Depth 3
 ; CHECK-NEXT:    cmp.w r12, #3
-; CHECK-NEXT:    bhi .LBB11_10
-; CHECK-NEXT:  @ %bb.9: @ in Loop: Header=BB11_8 Depth=2
+; CHECK-NEXT:    bhi .LBB10_10
+; CHECK-NEXT:  @ %bb.9: @ in Loop: Header=BB10_8 Depth=2
 ; CHECK-NEXT:    movs r4, #0
 ; CHECK-NEXT:    mov.w r10, #0
-; CHECK-NEXT:    b .LBB11_13
-; CHECK-NEXT:  .LBB11_10: @ %vector.ph
-; CHECK-NEXT:    @ in Loop: Header=BB11_8 Depth=2
+; CHECK-NEXT:    b .LBB10_13
+; CHECK-NEXT:  .LBB10_10: @ %vector.ph
+; CHECK-NEXT:    @ in Loop: Header=BB10_8 Depth=2
 ; CHECK-NEXT:    ldr r0, [sp, #16] @ 4-byte Reload
 ; CHECK-NEXT:    vmov q5, q1
 ; CHECK-NEXT:    vmov.i32 q4, #0x0
 ; CHECK-NEXT:    vmlas.u32 q5, q2, r8
 ; CHECK-NEXT:    dls lr, r0
 ; CHECK-NEXT:    ldr r3, [sp, #20] @ 4-byte Reload
-; CHECK-NEXT:  .LBB11_11: @ %vector.body
-; CHECK-NEXT:    @ Parent Loop BB11_5 Depth=1
-; CHECK-NEXT:    @ Parent Loop BB11_8 Depth=2
+; CHECK-NEXT:  .LBB10_11: @ %vector.body
+; CHECK-NEXT:    @ Parent Loop BB10_5 Depth=1
+; CHECK-NEXT:    @ Parent Loop BB10_8 Depth=2
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=3
 ; CHECK-NEXT:    vadd.i32 q6, q5, q3
 ; CHECK-NEXT:    vldrh.s32 q7, [r1, q5, uxtw #1]
 ; CHECK-NEXT:    vldrh.s32 q5, [r3], #8
 ; CHECK-NEXT:    vmul.i32 q5, q7, q5
 ; CHECK-NEXT:    vadd.i32 q4, q5, q4
 ; CHECK-NEXT:    vmov q5, q6
-; CHECK-NEXT:    le lr, .LBB11_11
+; CHECK-NEXT:    le lr, .LBB10_11
 ; CHECK-NEXT:  @ %bb.12: @ %middle.block
-; CHECK-NEXT:    @ in Loop: Header=BB11_8 Depth=2
+; CHECK-NEXT:    @ in Loop: Header=BB10_8 Depth=2
 ; CHECK-NEXT:    vaddv.u32 r10, q4
 ; CHECK-NEXT:    cmp r2, r12
 ; CHECK-NEXT:    mov r4, r2
-; CHECK-NEXT:    beq .LBB11_7
-; CHECK-NEXT:  .LBB11_13: @ %for.body8.us.us.preheader
-; CHECK-NEXT:    @ in Loop: Header=BB11_8 Depth=2
+; CHECK-NEXT:    beq .LBB10_7
+; CHECK-NEXT:  .LBB10_13: @ %for.body8.us.us.preheader
+; CHECK-NEXT:    @ in Loop: Header=BB10_8 Depth=2
 ; CHECK-NEXT:    mla r3, r7, r4, r8
 ; CHECK-NEXT:    add.w r0, r11, r4
 ; CHECK-NEXT:    ldr r7, [sp, #24] @ 4-byte Reload
 ; CHECK-NEXT:    sub.w lr, r12, r4
 ; CHECK-NEXT:    add.w r9, r7, r0, lsl #1
 ; CHECK-NEXT:    ldr r7, [sp, #136]
 ; CHECK-NEXT:    add.w r3, r1, r3, lsl #1
-; CHECK-NEXT:  .LBB11_14: @ %for.body8.us.us
-; CHECK-NEXT:    @ Parent Loop BB11_5 Depth=1
-; CHECK-NEXT:    @ Parent Loop BB11_8 Depth=2
+; CHECK-NEXT:  .LBB10_14: @ %for.body8.us.us
+; CHECK-NEXT:    @ Parent Loop BB10_5 Depth=1
+; CHECK-NEXT:    @ Parent Loop BB10_8 Depth=2
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=3
 ; CHECK-NEXT:    ldrsh.w r4, [r3]
 ; CHECK-NEXT:    add r3, r6
 ; CHECK-NEXT:    ldrsh r0, [r9], #2
 ; CHECK-NEXT:    smlabb r10, r4, r0, r10
-; CHECK-NEXT:    le lr, .LBB11_14
-; CHECK-NEXT:    b .LBB11_7
-; CHECK-NEXT:  .LBB11_15: @ Parent Loop BB11_5 Depth=1
+; CHECK-NEXT:    le lr, .LBB10_14
+; CHECK-NEXT:    b .LBB10_7
+; CHECK-NEXT:  .LBB10_15: @ Parent Loop BB10_5 Depth=1
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
 ; CHECK-NEXT:    vstrb.8 q0, [r3], #16
-; CHECK-NEXT:    letp lr, .LBB11_15
-; CHECK-NEXT:    b .LBB11_4
+; CHECK-NEXT:    letp lr, .LBB10_15
+; CHECK-NEXT:    b .LBB10_4
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.16:
-; CHECK-NEXT:  .LCPI11_0:
+; CHECK-NEXT:  .LCPI10_0:
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 1 @ 0x1
 ; CHECK-NEXT:    .long 2 @ 0x2
 ; CHECK-NEXT:    .long 3 @ 0x3
 entry:
   %cmp48 = icmp eq i32 %n, 0
   br i1 %cmp48, label %for.cond.cleanup, label %for.cond1.preheader.lr.ph
 

 ; CHECK-NEXT:    .pad #4
 ; CHECK-NEXT:    sub sp, #4
 ; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13}
 ; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13}
 ; CHECK-NEXT:    .pad #8
 ; CHECK-NEXT:    sub sp, #8
 ; CHECK-NEXT:    ldrd r2, r7, [sp, #104]
 ; CHECK-NEXT:    add.w r8, r7, #10
-; CHECK-NEXT:    adr r7, .LCPI12_0
+; CHECK-NEXT:    adr r7, .LCPI11_0
 ; CHECK-NEXT:    ldr r1, [sp, #96]
 ; CHECK-NEXT:    vdup.32 q0, r2
 ; CHECK-NEXT:    vldrw.u32 q1, [r7]
 ; CHECK-NEXT:    mov.w r10, #0
 ; CHECK-NEXT:    mov.w r9, #6
 ; CHECK-NEXT:    movs r6, #11
 ; CHECK-NEXT:    vshl.i32 q0, q0, #2
 ; CHECK-NEXT:    movs r5, #0
-; CHECK-NEXT:  .LBB12_1: @ %for.body10.i
+; CHECK-NEXT:  .LBB11_1: @ %for.body10.i
 ; CHECK-NEXT:    @ =>This Loop Header: Depth=1
-; CHECK-NEXT:    @ Child Loop BB12_2 Depth 2
-; CHECK-NEXT:    @ Child Loop BB12_3 Depth 3
-; CHECK-NEXT:    @ Child Loop BB12_4 Depth 4
-; CHECK-NEXT:    @ Child Loop BB12_5 Depth 5
+; CHECK-NEXT:    @ Child Loop BB11_2 Depth 2
+; CHECK-NEXT:    @ Child Loop BB11_3 Depth 3
+; CHECK-NEXT:    @ Child Loop BB11_4 Depth 4
+; CHECK-NEXT:    @ Child Loop BB11_5 Depth 5
 ; CHECK-NEXT:    movs r7, #0
 ; CHECK-NEXT:    str r5, [sp, #4] @ 4-byte Spill
-; CHECK-NEXT:  .LBB12_2: @ %for.cond22.preheader.i
-; CHECK-NEXT:    @ Parent Loop BB12_1 Depth=1
+; CHECK-NEXT:  .LBB11_2: @ %for.cond22.preheader.i
+; CHECK-NEXT:    @ Parent Loop BB11_1 Depth=1
 ; CHECK-NEXT:    @ => This Loop Header: Depth=2
-; CHECK-NEXT:    @ Child Loop BB12_3 Depth 3
-; CHECK-NEXT:    @ Child Loop BB12_4 Depth 4
-; CHECK-NEXT:    @ Child Loop BB12_5 Depth 5
+; CHECK-NEXT:    @ Child Loop BB11_3 Depth 3
+; CHECK-NEXT:    @ Child Loop BB11_4 Depth 4
+; CHECK-NEXT:    @ Child Loop BB11_5 Depth 5
 ; CHECK-NEXT:    movs r5, #0
-; CHECK-NEXT:  .LBB12_3: @ %for.body27.i
-; CHECK-NEXT:    @ Parent Loop BB12_1 Depth=1
-; CHECK-NEXT:    @ Parent Loop BB12_2 Depth=2
+; CHECK-NEXT:  .LBB11_3: @ %for.body27.i
+; CHECK-NEXT:    @ Parent Loop BB11_1 Depth=1
+; CHECK-NEXT:    @ Parent Loop BB11_2 Depth=2
 ; CHECK-NEXT:    @ => This Loop Header: Depth=3
-; CHECK-NEXT:    @ Child Loop BB12_4 Depth 4
-; CHECK-NEXT:    @ Child Loop BB12_5 Depth 5
+; CHECK-NEXT:    @ Child Loop BB11_4 Depth 4
+; CHECK-NEXT:    @ Child Loop BB11_5 Depth 5
 ; CHECK-NEXT:    dls lr, r9
 ; CHECK-NEXT:    mov.w r12, #0
 ; CHECK-NEXT:    mov.w r11, #4
-; CHECK-NEXT:  .LBB12_4: @ %for.body78.us.i
-; CHECK-NEXT:    @ Parent Loop BB12_1 Depth=1
-; CHECK-NEXT:    @ Parent Loop BB12_2 Depth=2
-; CHECK-NEXT:    @ Parent Loop BB12_3 Depth=3
+; CHECK-NEXT:  .LBB11_4: @ %for.body78.us.i
+; CHECK-NEXT:    @ Parent Loop BB11_1 Depth=1
+; CHECK-NEXT:    @ Parent Loop BB11_2 Depth=2
+; CHECK-NEXT:    @ Parent Loop BB11_3 Depth=3
 ; CHECK-NEXT:    @ => This Loop Header: Depth=4
-; CHECK-NEXT:    @ Child Loop BB12_5 Depth 5
+; CHECK-NEXT:    @ Child Loop BB11_5 Depth 5
 ; CHECK-NEXT:    mul r4, r11, r6
 ; CHECK-NEXT:    vdup.32 q3, r5
 ; CHECK-NEXT:    vdup.32 q2, r7
 ; CHECK-NEXT:    vadd.i32 q4, q1, r4
 ; CHECK-NEXT:    vmla.u32 q3, q4, r2
 ; CHECK-NEXT:    adds r4, #113
 ; CHECK-NEXT:    vadd.i32 q4, q1, r4
 ; CHECK-NEXT:    mov r4, r8
 ; CHECK-NEXT:    vmla.u32 q2, q4, r2
-; CHECK-NEXT:  .LBB12_5: @ %vector.body
-; CHECK-NEXT:    @ Parent Loop BB12_1 Depth=1
-; CHECK-NEXT:    @ Parent Loop BB12_2 Depth=2
-; CHECK-NEXT:    @ Parent Loop BB12_3 Depth=3
-; CHECK-NEXT:    @ Parent Loop BB12_4 Depth=4
+; CHECK-NEXT:  .LBB11_5: @ %vector.body
+; CHECK-NEXT:    @ Parent Loop BB11_1 Depth=1
+; CHECK-NEXT:    @ Parent Loop BB11_2 Depth=2
+; CHECK-NEXT:    @ Parent Loop BB11_3 Depth=3
+; CHECK-NEXT:    @ Parent Loop BB11_4 Depth=4
 ; CHECK-NEXT:    @ => This Inner Loop Header: Depth=5
 ; CHECK-NEXT:    vldrb.s32 q6, [r0, q2]
 ; CHECK-NEXT:    vadd.i32 q5, q2, q0
 ; CHECK-NEXT:    vadd.i32 q4, q3, q0
 ; CHECK-NEXT:    subs r4, #4
 ; CHECK-NEXT:    vadd.i32 q2, q6, r2
 ; CHECK-NEXT:    vldrb.s32 q6, [r1, q3]
 ; CHECK-NEXT:    vmov q3, q4
 ; CHECK-NEXT:    vmlava.u32 r12, q2, q6
 ; CHECK-NEXT:    vmov q2, q5
-; CHECK-NEXT:    bne .LBB12_5
+; CHECK-NEXT:    bne .LBB11_5
 ; CHECK-NEXT:  @ %bb.6: @ %middle.block
-; CHECK-NEXT:    @ in Loop: Header=BB12_4 Depth=4
+; CHECK-NEXT:    @ in Loop: Header=BB11_4 Depth=4
 ; CHECK-NEXT:    add.w r11, r11, #1
-; CHECK-NEXT:    le lr, .LBB12_4
+; CHECK-NEXT:    le lr, .LBB11_4
 ; CHECK-NEXT:  @ %bb.7: @ %for.cond.cleanup77.i
-; CHECK-NEXT:    @ in Loop: Header=BB12_3 Depth=3
+; CHECK-NEXT:    @ in Loop: Header=BB11_3 Depth=3
 ; CHECK-NEXT:    adds r5, #1
 ; CHECK-NEXT:    add.w r10, r10, #1
 ; CHECK-NEXT:    cmp r5, r2
-; CHECK-NEXT:    bne .LBB12_3
+; CHECK-NEXT:    bne .LBB11_3
 ; CHECK-NEXT:  @ %bb.8: @ %for.cond.cleanup26.i
-; CHECK-NEXT:    @ in Loop: Header=BB12_2 Depth=2
+; CHECK-NEXT:    @ in Loop: Header=BB11_2 Depth=2
 ; CHECK-NEXT:    adds r7, #1
 ; CHECK-NEXT:    cmp r7, r3
-; CHECK-NEXT:    bne .LBB12_2
+; CHECK-NEXT:    bne .LBB11_2
 ; CHECK-NEXT:  @ %bb.9: @ %for.cond.cleanup20.i
-; CHECK-NEXT:    @ in Loop: Header=BB12_1 Depth=1
+; CHECK-NEXT:    @ in Loop: Header=BB11_1 Depth=1
 ; CHECK-NEXT:    ldr r5, [sp, #4] @ 4-byte Reload
 ; CHECK-NEXT:    ldr r7, [sp, #148]
 ; CHECK-NEXT:    adds r5, #1
 ; CHECK-NEXT:    cmp r5, r7
 ; CHECK-NEXT:    it eq
 ; CHECK-NEXT:    moveq r5, #0
-; CHECK-NEXT:    b .LBB12_1
+; CHECK-NEXT:    b .LBB11_1
 ; CHECK-NEXT:    .p2align 4
 ; CHECK-NEXT:  @ %bb.10:
-; CHECK-NEXT:  .LCPI12_0:
+; CHECK-NEXT:  .LCPI11_0:
 ; CHECK-NEXT:    .long 0 @ 0x0
 ; CHECK-NEXT:    .long 1 @ 0x1
 ; CHECK-NEXT:    .long 2 @ 0x2
 ; CHECK-NEXT:    .long 3 @ 0x3
 entry:
   %conv = zext i16 %ch_mult to i32
   %conv6.i = zext i16 %output_x to i32
   %conv17.i = zext i16 %input_ch to i32
for.cond.cleanup77.i:                             ; preds = %middle.block
   %exitcond.i50 = icmp eq i32 %inc131.i, %conv
   br i1 %exitcond.i50, label %for.cond.cleanup26.i, label %for.body27.i
 
 if.end:                                           ; preds = %for.cond.cleanup9.i, %entry, %for.cond.cleanup7.i, %if.then
   ret i32 0
 }
 
 declare <4 x i32> @llvm.masked.gather.v4i32.v4p0i32(<4 x i32*>, i32, <4 x i1>, <4 x i32>)
-declare <4 x i32> @llvm.masked.gather.v4i32.v4p0(<4 x ptr>, i32, <4 x i1>, <4 x i32>)
 declare <4 x i16> @llvm.masked.gather.v4i16.v4p0i16(<4 x i16*>, i32, <4 x i1>, <4 x i16>)
 declare <4 x i8> @llvm.masked.gather.v4i8.v4p0i8(<4 x i8*>, i32 immarg, <4 x i1>, <4 x i8>) #3
 
 declare i32 @llvm.vector.reduce.add.v4i32(<4 x i32>)
 declare void @llvm.memset.p0i8.i32(i8* align 2, i8, i32, i1)
 
 declare void @llvm.masked.scatter.v4i32.v4p0i32(<4 x i32>, <4 x i32*>, i32, <4 x i1>)

llvm/test/CodeGen/Thumb2/mve-scatter-increment.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -enable-arm-maskedldst %s -o - | FileCheck %s
+; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -enable-arm-maskedldst -opaque-pointers %s -o - | FileCheck %s
 
 
 define arm_aapcs_vfpcc void @scatter_inc_minipred_4i32(<4 x i32> %data, i32* %dst, <4 x i32> %offs) {
 ; CHECK-LABEL: scatter_inc_minipred_4i32:
 ; CHECK:       @ %bb.0:
 ; CHECK-NEXT:    movw r1, #3855
 ; CHECK-NEXT:    vmov.i32 q2, #0x4
 ; CHECK-NEXT:    vadd.i32 q1, q1, q2

 middle.block:                                     ; preds = %vector.body
   %cmp.n = icmp eq i32 %n.vec, %n
   br i1 %cmp.n, label %for.cond.cleanup, label %vector.ph
 
 for.cond.cleanup:                                 ; preds = %for.body, %middle.block, %entry
   ret void
 }
 
-define arm_aapcs_vfpcc void @scatter_inc_v4i32_complex_opaque(<4 x i32> %data1, <4 x i32> %data2, <4 x i32> %data3, ptr %dst, i32 %n) {
-; CHECK-LABEL: scatter_inc_v4i32_complex_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    .save {r4, lr}
-; CHECK-NEXT:    push {r4, lr}
-; CHECK-NEXT:    .vsave {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    vpush {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    .pad #16
-; CHECK-NEXT:    sub sp, #16
-; CHECK-NEXT:    cmp r1, #1
-; CHECK-NEXT:    blt .LBB4_5
-; CHECK-NEXT:  @ %bb.1: @ %vector.ph.preheader
-; CHECK-NEXT:    adr r4, .LCPI4_2
-; CHECK-NEXT:    bic r2, r1, #3
-; CHECK-NEXT:    vldrw.u32 q3, [r4]
-; CHECK-NEXT:    sub.w r12, r2, #4
-; CHECK-NEXT:    adr.w lr, .LCPI4_1
-; CHECK-NEXT:    movs r3, #1
-; CHECK-NEXT:    vadd.i32 q3, q3, r0
-; CHECK-NEXT:    add.w r3, r3, r12, lsr #2
-; CHECK-NEXT:    vstrw.32 q3, [sp] @ 16-byte Spill
-; CHECK-NEXT:    vldrw.u32 q3, [lr]
-; CHECK-NEXT:    adr.w r12, .LCPI4_0
-; CHECK-NEXT:    vadd.i32 q4, q3, r0
-; CHECK-NEXT:    vldrw.u32 q3, [r12]
-; CHECK-NEXT:    vadd.i32 q3, q3, r0
-; CHECK-NEXT:  .LBB4_2: @ %vector.ph
-; CHECK-NEXT:    @ =>This Loop Header: Depth=1
-; CHECK-NEXT:    @ Child Loop BB4_3 Depth 2
-; CHECK-NEXT:    dls lr, r3
-; CHECK-NEXT:    vmov q6, q4
-; CHECK-NEXT:    vldrw.u32 q7, [sp] @ 16-byte Reload
-; CHECK-NEXT:    vmov q5, q3
-; CHECK-NEXT:  .LBB4_3: @ %vector.body
-; CHECK-NEXT:    @ Parent Loop BB4_2 Depth=1
-; CHECK-NEXT:    @ => This Inner Loop Header: Depth=2
-; CHECK-NEXT:    vstrw.32 q0, [q5, #48]!
-; CHECK-NEXT:    vstrw.32 q1, [q6, #48]!
-; CHECK-NEXT:    vstrw.32 q2, [q7, #48]!
-; CHECK-NEXT:    le lr, .LBB4_3
-; CHECK-NEXT:  @ %bb.4: @ %middle.block
-; CHECK-NEXT:    @ in Loop: Header=BB4_2 Depth=1
-; CHECK-NEXT:    cmp r2, r1
-; CHECK-NEXT:    bne .LBB4_2
-; CHECK-NEXT:  .LBB4_5: @ %for.cond.cleanup
-; CHECK-NEXT:    add sp, #16
-; CHECK-NEXT:    vpop {d8, d9, d10, d11, d12, d13, d14, d15}
-; CHECK-NEXT:    pop {r4, pc}
-; CHECK-NEXT:    .p2align 4
-; CHECK-NEXT:  @ %bb.6:
-; CHECK-NEXT:  .LCPI4_0:
-; CHECK-NEXT:    .long 4294967248 @ 0xffffffd0
-; CHECK-NEXT:    .long 4294967260 @ 0xffffffdc
-; CHECK-NEXT:    .long 4294967272 @ 0xffffffe8
-; CHECK-NEXT:    .long 4294967284 @ 0xfffffff4
-; CHECK-NEXT:  .LCPI4_1:
-; CHECK-NEXT:    .long 4294967252 @ 0xffffffd4
-; CHECK-NEXT:    .long 4294967264 @ 0xffffffe0
-; CHECK-NEXT:    .long 4294967276 @ 0xffffffec
-; CHECK-NEXT:    .long 4294967288 @ 0xfffffff8
-; CHECK-NEXT:  .LCPI4_2:
-; CHECK-NEXT:    .long 4294967256 @ 0xffffffd8
-; CHECK-NEXT:    .long 4294967268 @ 0xffffffe4
-; CHECK-NEXT:    .long 4294967280 @ 0xfffffff0
-; CHECK-NEXT:    .long 4294967292 @ 0xfffffffc
-entry:
-  %cmp22 = icmp sgt i32 %n, 0
-  br i1 %cmp22, label %vector.ph, label %for.cond.cleanup
-
-vector.ph:                                        ; preds = %for.body.preheader
-  %n.vec = and i32 %n, -4
-  br label %vector.body
-
-vector.body:                                      ; preds = %vector.body, %vector.ph
-  %index = phi i32 [ 0, %vector.ph ], [ %index.next, %vector.body ]
-  %vec.ind = phi <4 x i32> [ <i32 0, i32 1, i32 2, i32 3>, %vector.ph ], [ %vec.ind.next, %vector.body ]
-  %0 = mul nuw nsw <4 x i32> %vec.ind, <i32 3, i32 3, i32 3, i32 3>
-  %1 = getelementptr inbounds i32, ptr %dst, <4 x i32> %0
-  call void @llvm.masked.scatter.v4i32.v4p0(<4 x i32> %data1, <4 x ptr> %1, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>)
-  %2 = add nuw nsw <4 x i32> %0, <i32 1, i32 1, i32 1, i32 1>
-  %3 = getelementptr inbounds i32, ptr %dst, <4 x i32> %2
-  call void @llvm.masked.scatter.v4i32.v4p0(<4 x i32> %data2, <4 x ptr> %3, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>)
-  %4 = add nuw nsw <4 x i32> %0, <i32 2, i32 2, i32 2, i32 2>
-  %5 = getelementptr inbounds i32, ptr %dst, <4 x i32> %4
-  call void @llvm.masked.scatter.v4i32.v4p0(<4 x i32> %data3, <4 x ptr> %5, i32 4, <4 x i1> <i1 true, i1 true, i1 true, i1 true>)
-  %index.next = add i32 %index, 4
-  %vec.ind.next = add <4 x i32> %vec.ind, <i32 4, i32 4, i32 4, i32 4>
-  %6 = icmp eq i32 %index.next, %n.vec
-  br i1 %6, label %middle.block, label %vector.body
-
-middle.block:                                     ; preds = %vector.body
-  %cmp.n = icmp eq i32 %n.vec, %n
-  br i1 %cmp.n, label %for.cond.cleanup, label %vector.ph
-
-for.cond.cleanup:                                 ; preds = %for.body, %middle.block, %entry
-  ret void
-}
-
 declare void @llvm.masked.scatter.v8i8.v8p0i8(<8 x i8>, <8 x i8*>, i32, <8 x i1>)
 declare void @llvm.masked.scatter.v8i16.v8p0i16(<8 x i16>, <8 x i16*>, i32, <8 x i1>)
 declare void @llvm.masked.scatter.v8f16.v8p0f16(<8 x half>, <8 x half*>, i32, <8 x i1>)
 declare void @llvm.masked.scatter.v16i8.v16p0i8(<16 x i8>, <16 x i8*>, i32, <16 x i1>)
 declare void @llvm.masked.scatter.v4i8.v4p0i8(<4 x i8>, <4 x i8*>, i32, <4 x i1>)
 declare void @llvm.masked.scatter.v4i16.v4p0i16(<4 x i16>, <4 x i16*>, i32, <4 x i1>)
 declare void @llvm.masked.scatter.v4f16.v4p0f16(<4 x half>, <4 x half*>, i32, <4 x i1>)
 declare void @llvm.masked.scatter.v4i32.v4p0i32(<4 x i32>, <4 x i32*>, i32, <4 x i1>)
-declare void @llvm.masked.scatter.v4i32.v4p0(<4 x i32>, <4 x ptr>, i32, <4 x i1>)
 declare void @llvm.masked.scatter.v4f32.v4p0f32(<4 x float>, <4 x float*>, i32, <4 x i1>)

llvm/test/CodeGen/Thumb2/mve-scatter-ptrs.ll

 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
 ; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -enable-arm-maskedldst %s -o - | FileCheck %s
+; RUN: llc -mtriple=thumbv8.1m.main-none-none-eabi -mattr=+mve.fp -enable-arm-maskedldst -opaque-pointers %s -o - | FileCheck %s
 
 ; i32
 
 ; Expand
 define arm_aapcs_vfpcc void @ptr_v2i32(<2 x i32> %v, <2 x i32*>* %offptr) {
 ; CHECK-LABEL: ptr_v2i32:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vmov r2, s0

 ; CHECK-NEXT:    bx lr
 entry:
   %offs = load <4 x i16*>, <4 x i16*>* %offptr, align 4
   %ext = trunc <4 x i32> %v to <4 x i16>
   call void @llvm.masked.scatter.v4i16.v4p0i16(<4 x i16> %ext, <4 x i16*> %offs, i32 2, <4 x i1> <i1 true, i1 true, i1 true, i1 true>)
   ret void
 }
 
-define arm_aapcs_vfpcc void @ptr_v4i16_trunc_opaque(<4 x i32> %v, ptr %offptr) {
-; CHECK-LABEL: ptr_v4i16_trunc_opaque:
-; CHECK:       @ %bb.0: @ %entry
-; CHECK-NEXT:    vldrw.u32 q1, [r0]
-; CHECK-NEXT:    movs r0, #0
-; CHECK-NEXT:    vstrh.32 q0, [r0, q1]
-; CHECK-NEXT:    bx lr
-entry:
-  %offs = load <4 x ptr>, ptr %offptr, align 4
-  %ext = trunc <4 x i32> %v to <4 x i16>
-  call void @llvm.masked.scatter.v4i16.v4p0(<4 x i16> %ext, <4 x ptr> %offs, i32 2, <4 x i1> <i1 true, i1 true, i1 true, i1 true>)
-  ret void
-}
-
 define arm_aapcs_vfpcc void @ptr_v4i16_dup(i32 %v, <4 x i16*> %offs) {
 ; CHECK-LABEL: ptr_v4i16_dup:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    vdup.32 q1, r0
 ; CHECK-NEXT:    movs r1, #0
 ; CHECK-NEXT:    vmovlb.u16 q1, q1
 ; CHECK-NEXT:    vstrh.32 q1, [r1, q0]
 ; CHECK-NEXT:    bx lr

 
 define void @foo_ptr_p_int32_t(i32* %dest, i32** %src, i32 %n) {
 ; CHECK-LABEL: foo_ptr_p_int32_t:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    bic r3, r2, #15
 ; CHECK-NEXT:    cmp r3, #1
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    bxlt lr
-; CHECK-NEXT:  .LBB20_1: @ %vector.body
+; CHECK-NEXT:  .LBB19_1: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q0, [r1], #16
 ; CHECK-NEXT:    subs r2, #4
 ; CHECK-NEXT:    vptt.i32 ne, q0, zr
 ; CHECK-NEXT:    vldrwt.u32 q1, [r0], #16
 ; CHECK-NEXT:    vstrwt.32 q1, [q0]
-; CHECK-NEXT:    bne .LBB20_1
+; CHECK-NEXT:    bne .LBB19_1
 ; CHECK-NEXT:  @ %bb.2: @ %for.end
 ; CHECK-NEXT:    bx lr
 entry:
   %and = and i32 %n, -16
   %cmp11 = icmp sgt i32 %and, 0
   br i1 %cmp11, label %vector.body, label %for.end
 
 vector.body:                                      ; preds = %entry, %vector.body

 
 define void @foo_ptr_p_float(float* %dest, float** %src, i32 %n) {
 ; CHECK-LABEL: foo_ptr_p_float:
 ; CHECK:       @ %bb.0: @ %entry
 ; CHECK-NEXT:    bic r3, r2, #15
 ; CHECK-NEXT:    cmp r3, #1
 ; CHECK-NEXT:    it lt
 ; CHECK-NEXT:    bxlt lr
-; CHECK-NEXT:  .LBB21_1: @ %vector.body
+; CHECK-NEXT:  .LBB20_1: @ %vector.body
 ; CHECK-NEXT:    @ =>This Inner Loop Header: Depth=1
 ; CHECK-NEXT:    vldrw.u32 q0, [r1], #16
 ; CHECK-NEXT:    subs r2, #4
 ; CHECK-NEXT:    vptt.i32 ne, q0, zr
 ; CHECK-NEXT:    vldrwt.u32 q1, [r0], #16
 ; CHECK-NEXT:    vstrwt.32 q1, [q0]
-; CHECK-NEXT:    bne .LBB21_1
+; CHECK-NEXT:    bne .LBB20_1
 ; CHECK-NEXT:  @ %bb.2: @ %for.end
 ; CHECK-NEXT:    bx lr
 entry:
   %and = and i32 %n, -16
   %cmp11 = icmp sgt i32 %and, 0
   br i1 %cmp11, label %vector.body, label %for.end
 
 vector.body:                                      ; preds = %entry, %vector.body
entry:
   ret void
 }
 
 declare void @llvm.masked.scatter.v2i16.v2p0i16(<2 x i16>, <2 x i16*>, i32, <2 x i1>)
 declare void @llvm.masked.scatter.v2i32.v2p0i32(<2 x i32>, <2 x i32*>, i32, <2 x i1>)
 declare void @llvm.masked.scatter.v2f32.v2p0f32(<2 x float>, <2 x float*>, i32, <2 x i1>)
 declare void @llvm.masked.scatter.v4i8.v4p0i8(<4 x i8>, <4 x i8*>, i32, <4 x i1>)
 declare void @llvm.masked.scatter.v4i16.v4p0i16(<4 x i16>, <4 x i16*>, i32, <4 x i1>)
-declare void @llvm.masked.scatter.v4i16.v4p0(<4 x i16>, <4 x ptr>, i32, <4 x i1>)
 declare void @llvm.masked.scatter.v4f16.v4p0f16(<4 x half>, <4 x half*>, i32, <4 x i1>)
 declare void @llvm.masked.scatter.v4i32.v4p0i32(<4 x i32>, <4 x i32*>, i32, <4 x i1>)
 declare void @llvm.masked.scatter.v4f32.v4p0f32(<4 x float>, <4 x float*>, i32, <4 x i1>)
 declare void @llvm.masked.scatter.v8i8.v8p0i8(<8 x i8>, <8 x i8*>, i32, <8 x i1>)
 declare void @llvm.masked.scatter.v8i16.v8p0i16(<8 x i16>, <8 x i16*>, i32, <8 x i1>)
 declare void @llvm.masked.scatter.v8f16.v8p0f16(<8 x half>, <8 x half*>, i32, <8 x i1>)
 declare void @llvm.masked.scatter.v8i32.v8p0i32(<8 x i32>, <8 x i32*>, i32, <8 x i1>)
 declare void @llvm.masked.scatter.v8f32.v8p0f32(<8 x float>, <8 x float*>, i32, <8 x i1>)
 declare void @llvm.masked.scatter.v16i8.v16p0i8(<16 x i8>, <16 x i8*>, i32, <16 x i1>)
 declare void @llvm.masked.scatter.v16i32.v16p0i32(<16 x i32>, <16 x i32*>, i32, <16 x i1>)
 declare <4 x i32> @llvm.masked.load.v4i32.v4p0i32(<4 x i32>*, i32, <4 x i1>, <4 x i32>)

llvm/test/CodeGen/WebAssembly/add-prototypes-opaque-ptrs.ll

-; RUN: opt -S -wasm-add-missing-prototypes -force-opaque-pointers %s | FileCheck %s
+; RUN: opt -S -wasm-add-missing-prototypes -opaque-pointers %s | FileCheck %s
 
 target datalayout = "e-m:e-p:32:32-i64:64-n32:64-S128"
 target triple = "wasm32-unknown-unknown"
 
 ; CHECK: @foo_addr = global ptr @foo, align 8
 @foo_addr = global i64 (i32)* bitcast (i64 (...)* @foo to i64 (i32)*), align 8
 
 ; CHECK: @foo_addr_i8 = global ptr @foo, align 8

llvm/test/CodeGen/WebAssembly/function-bitcasts.ll

 ; RUN: llc < %s -asm-verbose=false -wasm-disable-explicit-locals -wasm-keep-registers -enable-emscripten-cxx-exceptions | FileCheck %s --check-prefixes=CHECK,TYPED
-; RUN: llc < %s -asm-verbose=false -wasm-disable-explicit-locals -wasm-keep-registers -enable-emscripten-cxx-exceptions -force-opaque-pointers | FileCheck %s --check-prefixes=CHECK,OPAQUE
+; RUN: llc < %s -asm-verbose=false -wasm-disable-explicit-locals -wasm-keep-registers -enable-emscripten-cxx-exceptions -opaque-pointers | FileCheck %s --check-prefixes=CHECK,OPAQUE
 
 ; Test that function pointer casts are replaced with wrappers.
 
 ; The TYPED and OPAQUE prefixes only differ in function ordering.
 
 target triple = "wasm32-unknown-unknown"
 
 define void @has_i32_arg(i32) {

llvm/test/CodeGen/WebAssembly/main-declaration.ll

 ; RUN: llc < %s -asm-verbose=false | FileCheck %s
-; RUN: llc < %s -asm-verbose=false -force-opaque-pointers | FileCheck %s
+; RUN: llc < %s -asm-verbose=false -opaque-pointers | FileCheck %s
 
 ; Test main functions with alternate signatures.
 
 target triple = "wasm32-unknown-unknown"
 
 declare i32 @main()
 
 define i32 @foo() {
   %t = call i32 @main()
   ret i32 %t
 }
 
 ; CHECK-LABEL: foo:
 ; CHECK-NEXT:    .functype foo () -> (i32)
 ; CHECK-NEXT:    call __original_main
 ; CHECK-NEXT:    end_function

llvm/test/Instrumentation/SanitizerCoverage/opaque-ptr.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --check-globals
-; RUN: opt < %s -passes='module(sancov-module)' -sanitizer-coverage-level=1 -force-opaque-pointers -S | FileCheck %s
+; RUN: opt < %s -passes='module(sancov-module)' -sanitizer-coverage-level=1 -opaque-pointers -S | FileCheck %s
 
 ;.
 ; CHECK: @[[__SANCOV_LOWEST_STACK:[a-zA-Z0-9_$"\\.-]+]] = external thread_local(initialexec) global i64
 ; CHECK: @[[__SANCOV_GEN_:[a-zA-Z0-9_$"\\.-]+]] = private global [1 x i32] zeroinitializer, section "__sancov_guards", comdat($foo), align 4
 ; CHECK: @[[__START___SANCOV_GUARDS:[a-zA-Z0-9_$"\\.-]+]] = extern_weak hidden global i32
 ; CHECK: @[[__STOP___SANCOV_GUARDS:[a-zA-Z0-9_$"\\.-]+]] = extern_weak hidden global i32
 ; CHECK: @[[LLVM_GLOBAL_CTORS:[a-zA-Z0-9_$"\\.-]+]] = appending global [1 x { i32, ptr, ptr }] [{ i32, ptr, ptr } { i32 2, ptr @sancov.module_ctor_trace_pc_guard, ptr @sancov.module_ctor_trace_pc_guard }]
 ; CHECK: @[[LLVM_COMPILER_USED:[a-zA-Z0-9_$"\\.-]+]] = appending global [1 x ptr] [ptr @__sancov_gen_], section "llvm.metadata"

llvm/test/Instrumentation/SanitizerCoverage/stack-depth-variable-declared-by-user.ll

 ; Ensure that we terminate with a useful error message (instead of crash) if the
 ; user declares `__sancov_lowest_stack` with an unexpected type.
 ; RUN: not opt < %s -passes='module(sancov-module)' -sanitizer-coverage-level=1 \
 ; RUN:         -sanitizer-coverage-stack-depth -S 2>&1 | FileCheck %s
 ; RUN: not opt < %s -passes='module(sancov-module)' -sanitizer-coverage-level=1 \
-; RUN:         -sanitizer-coverage-stack-depth -force-opaque-pointers -S 2>&1 | FileCheck %s
+; RUN:         -sanitizer-coverage-stack-depth -opaque-pointers -S 2>&1 | FileCheck %s
 
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 ; Wrong type: i32 instead of expected i64
 @__sancov_lowest_stack = thread_local global i32 0
 
 ; CHECK: error: '__sancov_lowest_stack' should not be declared by the user

llvm/test/Other/force-opaque-ptrs-typed-dis.ll

-; RUN: llvm-as --force-opaque-pointers < %s | llvm-dis | FileCheck %s
+; RUN: llvm-as --opaque-pointers < %s | not llvm-dis 2>&1 | FileCheck %s
+
+; CHECK: error: Opaque pointers are only supported in -opaque-pointers mode
 
-; CHECK: @g = external global i16
 @g = external global i16
 
 define void @f(i32* %p) {
-; CHECK-LABEL: @f(
-; CHECK-NEXT:    [[A:%.*]] = alloca i17, align 4
-; CHECK-NEXT:    ret void
-;
   %a = alloca i17
   ret void
 }

llvm/test/Other/force-opaque-ptrs.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --function-signature
-; RUN: llvm-as --force-opaque-pointers < %s | llvm-dis --force-opaque-pointers | FileCheck %s
-; RUN: llvm-as < %s | llvm-dis --force-opaque-pointers | FileCheck %s
-; RUN: opt --force-opaque-pointers < %s -S | FileCheck %s
-; RUN: verify-uselistorder --force-opaque-pointers < %s
+; RUN: llvm-as --opaque-pointers < %s | llvm-dis --opaque-pointers | FileCheck %s
+; RUN: llvm-as < %s | llvm-dis --opaque-pointers | FileCheck %s
+; RUN: opt --opaque-pointers < %s -S | FileCheck %s
+; RUN: verify-uselistorder --opaque-pointers < %s
 
 %ty = type i32*
 
 ; CHECK: @g = external global i16
 @g = external global i16
 
 ; CHECK: @llvm.global_ctors = appending global [1 x { i32, ptr, ptr }] [{ i32, ptr, ptr } { i32 1, ptr null, ptr null }]
 @llvm.global_ctors = appending global [1 x { i32, void ()*, i8* }] [{ i32, void ()*, i8* } { i32 1, void ()* null, i8* null }]

llvm/test/Transforms/DeadStoreElimination/OverwriteStoreBegin.ll

entry:
   %arrayidx0 = getelementptr inbounds i32, i32* %p, i64 1
   %p3 = bitcast i32* %arrayidx0 to i8*
   call void @llvm.memset.p0i8.i64(i8* align 4 %p3, i8 0, i64 28, i1 false)
   %arrayidx1 = getelementptr inbounds i32, i32* %p, i64 1
   store i32 1, i32* %arrayidx1, align 4
   ret void
 }
 
-define void @write4to7_opaque_ptr(ptr nocapture %p) {
-; CHECK-LABEL: @write4to7_opaque_ptr(
-; CHECK-NEXT:  entry:
-; CHECK-NEXT:    [[ARRAYIDX0:%.*]] = getelementptr inbounds i32, ptr [[P:%.*]], i64 1
-; CHECK-NEXT:    [[TMP0:%.*]] = bitcast ptr [[ARRAYIDX0]] to i8*
-; CHECK-NEXT:    [[TMP1:%.*]] = getelementptr inbounds i8, i8* [[TMP0]], i64 4
-; CHECK-NEXT:    [[TMP2:%.*]] = bitcast i8* [[TMP1]] to ptr
-; CHECK-NEXT:    call void @llvm.memset.p0.i64(ptr align 4 [[TMP2]], i8 0, i64 24, i1 false)
-; CHECK-NEXT:    [[ARRAYIDX1:%.*]] = getelementptr inbounds i32, ptr [[P]], i64 1
-; CHECK-NEXT:    store i32 1, ptr [[ARRAYIDX1]], align 4
-; CHECK-NEXT:    ret void
-;
-entry:
-  %arrayidx0 = getelementptr inbounds i32, ptr %p, i64 1
-  call void @llvm.memset.p0.i64(ptr align 4 %arrayidx0, i8 0, i64 28, i1 false)
-  %arrayidx1 = getelementptr inbounds i32, ptr %p, i64 1
-  store i32 1, ptr %arrayidx1, align 4
-  ret void
-}
-
 define void @write4to7_weird_element_type(i32* nocapture %p) {
 ; CHECK-LABEL: @write4to7_weird_element_type(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[ARRAYIDX0:%.*]] = getelementptr inbounds i32, i32* [[P:%.*]], i64 1
 ; CHECK-NEXT:    [[TMP0:%.*]] = bitcast i32* [[ARRAYIDX0]] to i8*
 ; CHECK-NEXT:    [[TMP1:%.*]] = getelementptr inbounds i8, i8* [[TMP0]], i64 4
 ; CHECK-NEXT:    [[TMP2:%.*]] = bitcast i8* [[TMP1]] to i32*
 ; CHECK-NEXT:    call void @llvm.memset.p0i32.i64(i32* align 4 [[TMP2]], i8 0, i64 24, i1 false)
entry:
 
   store i64 1, i64* %base64_1, align 8
   store atomic i64 2, i64* %base64_0 unordered, align 8
   ret void
 }
 
 declare void @llvm.memset.p0i8.i64(i8* nocapture, i8, i64, i1) nounwind
 declare void @llvm.memset.p0i32.i64(i32* nocapture, i8, i64, i1) nounwind
-declare void @llvm.memset.p0.i64(ptr nocapture, i8, i64, i1) nounwind
 declare void @llvm.memset.p1i8.i64(i8 addrspace(1)* nocapture, i8, i64, i1) nounwind
 declare void @llvm.memset.element.unordered.atomic.p0i8.i64(i8* nocapture, i8, i64, i32) nounwind
 
 define void @ow_begin_align1(i8* nocapture %p) {
 ; CHECK-LABEL: @ow_begin_align1(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[P1:%.*]] = getelementptr inbounds i8, i8* [[P:%.*]], i64 1
 ; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr inbounds i8, i8* [[P1]], i64 7

llvm/test/Transforms/DeadStoreElimination/opaque-ptr.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt < %s -dse -opaque-pointers -S | FileCheck %s
+
+define void @write4to7_opaque_ptr(ptr nocapture %p) {
+; CHECK-LABEL: @write4to7_opaque_ptr(
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    [[ARRAYIDX0:%.*]] = getelementptr inbounds i32, ptr [[P:%.*]], i64 1
+; CHECK-NEXT:    [[TMP0:%.*]] = getelementptr inbounds i8, ptr [[ARRAYIDX0]], i64 4
+; CHECK-NEXT:    call void @llvm.memset.p0.i64(ptr align 4 [[TMP0]], i8 0, i64 24, i1 false)
+; CHECK-NEXT:    [[ARRAYIDX1:%.*]] = getelementptr inbounds i32, ptr [[P]], i64 1
+; CHECK-NEXT:    store i32 1, ptr [[ARRAYIDX1]], align 4
+; CHECK-NEXT:    ret void
+;
+entry:
+  %arrayidx0 = getelementptr inbounds i32, ptr %p, i64 1
+  call void @llvm.memset.p0.i64(ptr align 4 %arrayidx0, i8 0, i64 28, i1 false)
+  %arrayidx1 = getelementptr inbounds i32, ptr %p, i64 1
+  store i32 1, ptr %arrayidx1, align 4
+  ret void
+}
+
+declare void @llvm.memset.p0.i64(ptr nocapture, i8, i64, i1) nounwind

llvm/test/Transforms/InstCombine/force-opaque-ptr.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -S -instcombine -force-opaque-pointers < %s | FileCheck %s
+; RUN: opt -S -instcombine -opaque-pointers < %s | FileCheck %s
 
 @g = global [16 x i16] zeroinitializer
 
 define ptr @gep_constexpr_gv_1() {
 ; CHECK-LABEL: @gep_constexpr_gv_1(
 ; CHECK-NEXT:    ret ptr getelementptr inbounds ([16 x i16], ptr @g, i64 0, i64 10)
 ;
   ret ptr getelementptr([16 x i16], ptr @g, i64 0, i64 10)

llvm/test/Transforms/InstCombine/opaque-ptr.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -S -instcombine < %s | FileCheck %s
+; RUN: opt -S -instcombine -opaque-pointers < %s | FileCheck %s
 
 define ptr @bitcast_opaque_to_opaque(ptr %a) {
 ; CHECK-LABEL: @bitcast_opaque_to_opaque(
 ; CHECK-NEXT:    ret ptr [[A:%.*]]
 ;
   %b = bitcast ptr %a to ptr
   ret ptr %b
 }
 
 define ptr @bitcast_typed_to_opaque(i8* %a) {
 ; CHECK-LABEL: @bitcast_typed_to_opaque(
-; CHECK-NEXT:    [[B:%.*]] = bitcast i8* [[A:%.*]] to ptr
-; CHECK-NEXT:    ret ptr [[B]]
+; CHECK-NEXT:    ret ptr [[A:%.*]]
 ;
   %b = bitcast i8* %a to ptr
   ret ptr %b
 }
 
 define i8* @bitcast_opaque_to_typed(ptr %a) {
 ; CHECK-LABEL: @bitcast_opaque_to_typed(
-; CHECK-NEXT:    [[B:%.*]] = bitcast ptr [[A:%.*]] to i8*
-; CHECK-NEXT:    ret i8* [[B]]
+; CHECK-NEXT:    ret ptr [[A:%.*]]
 ;
   %b = bitcast ptr %a to i8*
   ret i8* %b
 }
 
 @g = global i8 0
 define ptr @bitcast_typed_to_opaque_constexpr() {
 ; CHECK-LABEL: @bitcast_typed_to_opaque_constexpr(
-; CHECK-NEXT:    ret ptr bitcast (i8* @g to ptr)
+; CHECK-NEXT:    ret ptr @g
 ;
   ret ptr bitcast (i8* @g to ptr)
 }
 
 define ptr @addrspacecast_opaque_to_opaque(ptr addrspace(1) %a) {
 ; CHECK-LABEL: @addrspacecast_opaque_to_opaque(
 ; CHECK-NEXT:    [[B:%.*]] = addrspacecast ptr addrspace(1) [[A:%.*]] to ptr
 ; CHECK-NEXT:    ret ptr [[B]]
 ;
   %b = addrspacecast ptr addrspace(1) %a to ptr
   ret ptr %b
 }
 
 define ptr @addrspacecast_typed_to_opaque(i8 addrspace(1)* %a) {
 ; CHECK-LABEL: @addrspacecast_typed_to_opaque(
-; CHECK-NEXT:    [[TMP1:%.*]] = bitcast i8 addrspace(1)* [[A:%.*]] to ptr addrspace(1)
-; CHECK-NEXT:    [[B:%.*]] = addrspacecast ptr addrspace(1) [[TMP1]] to ptr
+; CHECK-NEXT:    [[B:%.*]] = addrspacecast ptr addrspace(1) [[A:%.*]] to ptr
 ; CHECK-NEXT:    ret ptr [[B]]
 ;
   %b = addrspacecast i8 addrspace(1)* %a to ptr
   ret ptr %b
 }
 
 define i8* @addrspacecast_opaque_to_typed(ptr addrspace(1) %a) {
 ; CHECK-LABEL: @addrspacecast_opaque_to_typed(
-; CHECK-NEXT:    [[TMP1:%.*]] = bitcast ptr addrspace(1) [[A:%.*]] to i8 addrspace(1)*
-; CHECK-NEXT:    [[B:%.*]] = addrspacecast i8 addrspace(1)* [[TMP1]] to i8*
-; CHECK-NEXT:    ret i8* [[B]]
+; CHECK-NEXT:    [[B:%.*]] = addrspacecast ptr addrspace(1) [[A:%.*]] to ptr
+; CHECK-NEXT:    ret ptr [[B]]
 ;
   %b = addrspacecast ptr addrspace(1) %a to i8*
   ret i8* %b
 }
 
 define ptr addrspace(1) @bitcast_and_addrspacecast_eliminable(ptr %a) {
 ; CHECK-LABEL: @bitcast_and_addrspacecast_eliminable(
 ; CHECK-NEXT:    [[C:%.*]] = addrspacecast ptr [[A:%.*]] to ptr addrspace(1)
 ; CHECK-NEXT:    ret ptr addrspace(1) [[C]]
 ;
   %b = bitcast ptr %a to i8*
   %c = addrspacecast i8* %b to ptr addrspace(1)
   ret ptr addrspace(1) %c
 }
 
 define ptr addrspace(1) @addrspacecast_typed_to_opaque_constexpr() {
 ; CHECK-LABEL: @addrspacecast_typed_to_opaque_constexpr(
-; CHECK-NEXT:    ret ptr addrspace(1) addrspacecast (ptr bitcast (i8* @g to ptr) to ptr addrspace(1))
+; CHECK-NEXT:    ret ptr addrspace(1) addrspacecast (ptr @g to ptr addrspace(1))
 ;
   ret ptr addrspace(1) addrspacecast (i8* @g to ptr addrspace(1))
 }
 
 define ptr @gep_constexpr_1(ptr %a) {
 ; CHECK-LABEL: @gep_constexpr_1(
 ; CHECK-NEXT:    ret ptr inttoptr (i64 6 to ptr)
 ;
   ret ptr getelementptr (i16, ptr null, i32 3)
 }
 
 define ptr @gep_constexpr_2(ptr %a) {
 ; CHECK-LABEL: @gep_constexpr_2(
-; CHECK-NEXT:    ret ptr bitcast (i8* getelementptr (i8, i8* @g, i64 3) to ptr)
+; CHECK-NEXT:    ret ptr getelementptr (i8, ptr @g, i64 3)
 ;
   ret ptr getelementptr (i8, ptr bitcast (i8* @g to ptr), i32 3)
 }
 
 define ptr @load_bitcast_1(ptr %a) {
 ; CHECK-LABEL: @load_bitcast_1(
 ; CHECK-NEXT:    [[B1:%.*]] = load ptr, ptr [[A:%.*]], align 8
 ; CHECK-NEXT:    ret ptr [[B1]]

 ;
   call void %a()
   ret void
 }
 
 declare void @varargs(...)
 define void @varargs_cast_typed_to_opaque_same_type(i32* %a) {
 ; CHECK-LABEL: @varargs_cast_typed_to_opaque_same_type(
-; CHECK-NEXT:    call void (...) @varargs(i32* byval(i32) [[A:%.*]])
+; CHECK-NEXT:    call void (...) @varargs(ptr byval(i32) [[A:%.*]])
 ; CHECK-NEXT:    ret void
 ;
   %b = bitcast i32* %a to ptr
   call void (...) @varargs(ptr byval(i32) %b)
   ret void
 }
 
 define void @varargs_cast_typed_to_opaque_different_type(i32* %a) {
 ; CHECK-LABEL: @varargs_cast_typed_to_opaque_different_type(
-; CHECK-NEXT:    call void (...) @varargs(i32* byval(i32) [[A:%.*]])
+; CHECK-NEXT:    call void (...) @varargs(ptr byval(float) [[A:%.*]])
 ; CHECK-NEXT:    ret void
 ;
   %b = bitcast i32* %a to ptr
   call void (...) @varargs(ptr byval(float) %b)
   ret void
 }
 
 define void @varargs_cast_typed_to_opaque_different_size(i32* %a) {
 ; CHECK-LABEL: @varargs_cast_typed_to_opaque_different_size(
-; CHECK-NEXT:    [[B:%.*]] = bitcast i32* [[A:%.*]] to ptr
-; CHECK-NEXT:    call void (...) @varargs(ptr byval(i64) [[B]])
+; CHECK-NEXT:    call void (...) @varargs(ptr byval(i64) [[A:%.*]])
 ; CHECK-NEXT:    ret void
 ;
   %b = bitcast i32* %a to ptr
   call void (...) @varargs(ptr byval(i64) %b)
   ret void
 }
 
 define void @varargs_cast_opaque_to_typed(ptr %a) {
 ; CHECK-LABEL: @varargs_cast_opaque_to_typed(
 ; CHECK-NEXT:    call void (...) @varargs(ptr byval(i8) [[A:%.*]])
 ; CHECK-NEXT:    ret void
 ;
   %b = bitcast ptr %a to i8*
   call void (...) @varargs(i8* byval(i8) %b)
   ret void
 }

llvm/test/Transforms/LoadStoreVectorizer/X86/opaque-ptr.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -mtriple=x86_64-unknown-linux-gnu -load-store-vectorizer -S < %s | FileCheck %s
+; RUN: opt -mtriple=x86_64-unknown-linux-gnu -load-store-vectorizer -opaque-pointers -S < %s | FileCheck %s
 
 define void @test(ptr %ptr) {
 ; CHECK-LABEL: @test(
-; CHECK-NEXT:    [[TMP1:%.*]] = bitcast ptr [[PTR:%.*]] to <2 x i32>*
-; CHECK-NEXT:    [[TMP2:%.*]] = load <2 x i32>, <2 x i32>* [[TMP1]], align 4
-; CHECK-NEXT:    [[L11:%.*]] = extractelement <2 x i32> [[TMP2]], i32 0
-; CHECK-NEXT:    [[L22:%.*]] = extractelement <2 x i32> [[TMP2]], i32 1
-; CHECK-NEXT:    [[TMP3:%.*]] = bitcast ptr [[PTR]] to <2 x i32>*
-; CHECK-NEXT:    store <2 x i32> zeroinitializer, <2 x i32>* [[TMP3]], align 4
+; CHECK-NEXT:    [[TMP1:%.*]] = load <2 x i32>, ptr [[PTR:%.*]], align 4
+; CHECK-NEXT:    [[L11:%.*]] = extractelement <2 x i32> [[TMP1]], i32 0
+; CHECK-NEXT:    [[L22:%.*]] = extractelement <2 x i32> [[TMP1]], i32 1
+; CHECK-NEXT:    store <2 x i32> zeroinitializer, ptr [[PTR]], align 4
 ; CHECK-NEXT:    ret void
 ;
   %ptr2 = getelementptr i32, ptr %ptr, i64 1
   %l1 = load i32, ptr %ptr, align 4
   %l2 = load i32, ptr %ptr2, align 4
   store i32 0, ptr %ptr, align 4
   store i32 0, ptr %ptr2, align 4
   ret void
 }
 

llvm/test/Transforms/LoopStrengthReduce/opaque-ptr.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -S -loop-reduce < %s | FileCheck %s
+; RUN: opt -S -loop-reduce -opaque-pointers < %s | FileCheck %s
 
 target datalayout = "e-p:64:64:64-n32:64"
 
 define void @test1(ptr %p.start, i64 %len) {
 ; CHECK-LABEL: @test1(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[UGLYGEP:%.*]] = getelementptr i8, ptr [[P_START:%.*]], i64 4
 ; CHECK-NEXT:    br label [[LOOP:%.*]]

llvm/test/Transforms/LoopVectorize/opaque-ptr.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -S -loop-vectorize -force-vector-width=2 < %s | FileCheck %s
+; RUN: opt -S -loop-vectorize -force-vector-width=2 -opaque-pointers < %s | FileCheck %s
 
 ; TODO: This still crashes with inbounds on the GEPs.
 define void @test(ptr %p1.start, ptr %p2.start, ptr %p1.end) {
 ; CHECK-LABEL: @test(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    br label [[LOOP:%.*]]
 ; CHECK:       loop:
 ; CHECK-NEXT:    [[P1:%.*]] = phi ptr [ [[P1_START:%.*]], [[ENTRY:%.*]] ], [ [[P1_NEXT:%.*]], [[LOOP]] ]

llvm/test/Transforms/MemCpyOpt/memset-memcpy-redundant-memset.ll

 ;
   %gep1 = getelementptr i8, i8* %dst, i64 16
   call void @llvm.memset.p0i8.i64(i8* %gep1, i8 %c, i64 %dst_size, i1 false)
   %gep2 = getelementptr i8, i8* %dst, i64 16
   call void @llvm.memcpy.p0i8.p0i8.i64(i8* %gep2, i8* %src, i64 %src_size, i1 false)
   ret void
 }
 
-define void @test_opaque_ptrs(ptr %src, i64 %src_size, ptr noalias %dst, i64 %dst_size, i8 %c) {
-; CHECK-LABEL: @test_opaque_ptrs(
-; CHECK-NEXT:    [[TMP1:%.*]] = icmp ule i64 [[DST_SIZE:%.*]], [[SRC_SIZE:%.*]]
-; CHECK-NEXT:    [[TMP2:%.*]] = sub i64 [[DST_SIZE]], [[SRC_SIZE]]
-; CHECK-NEXT:    [[TMP3:%.*]] = select i1 [[TMP1]], i64 0, i64 [[TMP2]]
-; CHECK-NEXT:    [[TMP4:%.*]] = bitcast ptr [[DST:%.*]] to i8*
-; CHECK-NEXT:    [[TMP5:%.*]] = getelementptr i8, i8* [[TMP4]], i64 [[SRC_SIZE]]
-; CHECK-NEXT:    call void @llvm.memset.p0i8.i64(i8* align 1 [[TMP5]], i8 [[C:%.*]], i64 [[TMP3]], i1 false)
-; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i64(ptr [[DST]], ptr [[SRC:%.*]], i64 [[SRC_SIZE]], i1 false)
-; CHECK-NEXT:    ret void
-;
-  call void @llvm.memset.p0.i64(ptr %dst, i8 %c, i64 %dst_size, i1 false)
-  call void @llvm.memcpy.p0.p0.i64(ptr %dst, ptr %src, i64 %src_size, i1 false)
-  ret void
-}
-
 define void @test_weird_element_type(i16* %src, i64 %src_size, i16* noalias %dst, i64 %dst_size, i8 %c) {
 ; CHECK-LABEL: @test_weird_element_type(
 ; CHECK-NEXT:    [[TMP1:%.*]] = icmp ule i64 [[DST_SIZE:%.*]], [[SRC_SIZE:%.*]]
 ; CHECK-NEXT:    [[TMP2:%.*]] = sub i64 [[DST_SIZE]], [[SRC_SIZE]]
 ; CHECK-NEXT:    [[TMP3:%.*]] = select i1 [[TMP1]], i64 0, i64 [[TMP2]]
 ; CHECK-NEXT:    [[TMP4:%.*]] = bitcast i16* [[DST:%.*]] to i8*
 ; CHECK-NEXT:    [[TMP5:%.*]] = getelementptr i8, i8* [[TMP4]], i64 [[SRC_SIZE]]
 ; CHECK-NEXT:    call void @llvm.memset.p0i8.i64(i8* align 1 [[TMP5]], i8 [[C:%.*]], i64 [[TMP3]], i1 false)

 }
 
 declare void @llvm.memset.p0i8.i64(i8* nocapture, i8, i64, i1)
 declare void @llvm.memcpy.p0i8.p0i8.i64(i8* nocapture, i8* nocapture readonly, i64, i1)
 declare void @llvm.memset.p0i8.i32(i8* nocapture, i8, i32, i1)
 declare void @llvm.memcpy.p0i8.p0i8.i32(i8* nocapture, i8* nocapture readonly, i32, i1)
 declare void @llvm.memset.p0i8.i128(i8* nocapture, i8, i128, i1)
 declare void @llvm.memcpy.p0i8.p0i8.i128(i8* nocapture, i8* nocapture readonly, i128, i1)
-declare void @llvm.memset.p0.i64(ptr nocapture, i8, i64, i1)
-declare void @llvm.memcpy.p0.p0.i64(ptr nocapture, ptr nocapture readonly, i64, i1)
 declare void @llvm.memset.p0i16.i64(i16* nocapture, i8, i64, i1)
 declare void @llvm.memcpy.p0i16.p0i16.i64(i16* nocapture, i16* nocapture readonly, i64, i1)
 declare void @llvm.memset.p1i8.i64(i8 addrspace(1)* nocapture, i8, i64, i1)
 declare void @llvm.memcpy.p1i8.p1i8.i64(i8 addrspace(1)* nocapture, i8 addrspace(1)* nocapture readonly, i64, i1)
 declare void @call()

llvm/test/Transforms/MemCpyOpt/no-libcalls.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -S -memcpyopt < %s | FileCheck %s --check-prefixes=CHECK,LIBCALLS
-; RUN: opt -S -memcpyopt -mtriple=amdgcn-- < %s | FileCheck %s --check-prefixes=CHECK,NO-LIBCALLS
-; RUN: opt -S -memcpyopt -mtriple=amdgcn-- -enable-memcpyopt-without-libcalls < %s \
+; RUN: opt -S -memcpyopt -opaque-pointers < %s | FileCheck %s --check-prefixes=CHECK,LIBCALLS
+; RUN: opt -S -memcpyopt -mtriple=amdgcn-- -opaque-pointers < %s | FileCheck %s --check-prefixes=CHECK,NO-LIBCALLS
+; RUN: opt -S -memcpyopt -mtriple=amdgcn-- -enable-memcpyopt-without-libcalls -opaque-pointers < %s \
 ; RUN:     | FileCheck %s --check-prefixes=CHECK,LIBCALLS
 
 ; REQUIRES: amdgpu-registered-target
 
 define void @dont_create_memset(ptr %p) {
 ; LIBCALLS-LABEL: @dont_create_memset(
 ; LIBCALLS-NEXT:    [[P1:%.*]] = getelementptr i32, ptr [[P:%.*]], i64 1
 ; LIBCALLS-NEXT:    [[P2:%.*]] = getelementptr i32, ptr [[P]], i64 2

llvm/test/Transforms/MemCpyOpt/opaque-ptr.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt -basic-aa -memcpyopt -S %s -verify-memoryssa -opaque-pointers | FileCheck %s
+
+define void @test_memset_memcpy(ptr %src, i64 %src_size, ptr noalias %dst, i64 %dst_size, i8 %c) {
+; CHECK-LABEL: @test_memset_memcpy(
+; CHECK-NEXT:    [[TMP1:%.*]] = icmp ule i64 [[DST_SIZE:%.*]], [[SRC_SIZE:%.*]]
+; CHECK-NEXT:    [[TMP2:%.*]] = sub i64 [[DST_SIZE]], [[SRC_SIZE]]
+; CHECK-NEXT:    [[TMP3:%.*]] = select i1 [[TMP1]], i64 0, i64 [[TMP2]]
+; CHECK-NEXT:    [[TMP4:%.*]] = getelementptr i8, ptr [[DST:%.*]], i64 [[SRC_SIZE]]
+; CHECK-NEXT:    call void @llvm.memset.p0.i64(ptr align 1 [[TMP4]], i8 [[C:%.*]], i64 [[TMP3]], i1 false)
+; CHECK-NEXT:    call void @llvm.memcpy.p0.p0.i64(ptr [[DST]], ptr [[SRC:%.*]], i64 [[SRC_SIZE]], i1 false)
+; CHECK-NEXT:    ret void
+;
+  call void @llvm.memset.p0.i64(ptr %dst, i8 %c, i64 %dst_size, i1 false)
+  call void @llvm.memcpy.p0.p0.i64(ptr %dst, ptr %src, i64 %src_size, i1 false)
+  ret void
+}
+
+declare void @llvm.memset.p0.i64(ptr nocapture, i8, i64, i1)
+declare void @llvm.memcpy.p0.p0.i64(ptr nocapture, ptr nocapture readonly, i64, i1)

llvm/test/Transforms/SLPVectorizer/X86/opaque-ptr.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -S -slp-vectorizer -mtriple=x86_64-apple-macosx -mcpu=haswell < %s | FileCheck %s
+; RUN: opt -S -slp-vectorizer -mtriple=x86_64-apple-macosx -mcpu=haswell -opaque-pointers < %s | FileCheck %s
 
 define void @test(ptr %r, ptr %p, ptr %q) #0 {
 ; CHECK-LABEL: @test(
 ; CHECK-NEXT:    [[P0:%.*]] = getelementptr inbounds i64, ptr [[P:%.*]], i64 0
 ; CHECK-NEXT:    [[P1:%.*]] = getelementptr inbounds i64, ptr [[P]], i64 1
 ; CHECK-NEXT:    [[P2:%.*]] = getelementptr inbounds i64, ptr [[P]], i64 2
 ; CHECK-NEXT:    [[P3:%.*]] = getelementptr inbounds i64, ptr [[P]], i64 3
 ; CHECK-NEXT:    [[Q0:%.*]] = getelementptr inbounds i64, ptr [[Q:%.*]], i64 0
 ; CHECK-NEXT:    [[Q1:%.*]] = getelementptr inbounds i64, ptr [[Q]], i64 1
 ; CHECK-NEXT:    [[Q2:%.*]] = getelementptr inbounds i64, ptr [[Q]], i64 2
 ; CHECK-NEXT:    [[Q3:%.*]] = getelementptr inbounds i64, ptr [[Q]], i64 3
-; CHECK-NEXT:    [[TMP1:%.*]] = bitcast ptr [[P0]] to <4 x i64>*
-; CHECK-NEXT:    [[TMP2:%.*]] = load <4 x i64>, <4 x i64>* [[TMP1]], align 2
-; CHECK-NEXT:    [[TMP3:%.*]] = bitcast ptr [[Q0]] to <4 x i64>*
-; CHECK-NEXT:    [[TMP4:%.*]] = load <4 x i64>, <4 x i64>* [[TMP3]], align 2
-; CHECK-NEXT:    [[TMP5:%.*]] = sub nsw <4 x i64> [[TMP2]], [[TMP4]]
-; CHECK-NEXT:    [[TMP6:%.*]] = extractelement <4 x i64> [[TMP5]], i32 0
-; CHECK-NEXT:    [[G0:%.*]] = getelementptr inbounds i32, ptr [[R:%.*]], i64 [[TMP6]]
-; CHECK-NEXT:    [[TMP7:%.*]] = extractelement <4 x i64> [[TMP5]], i32 1
-; CHECK-NEXT:    [[G1:%.*]] = getelementptr inbounds i32, ptr [[R]], i64 [[TMP7]]
-; CHECK-NEXT:    [[TMP8:%.*]] = extractelement <4 x i64> [[TMP5]], i32 2
-; CHECK-NEXT:    [[G2:%.*]] = getelementptr inbounds i32, ptr [[R]], i64 [[TMP8]]
-; CHECK-NEXT:    [[TMP9:%.*]] = extractelement <4 x i64> [[TMP5]], i32 3
-; CHECK-NEXT:    [[G3:%.*]] = getelementptr inbounds i32, ptr [[R]], i64 [[TMP9]]
+; CHECK-NEXT:    [[TMP1:%.*]] = load <4 x i64>, ptr [[P0]], align 2
+; CHECK-NEXT:    [[TMP2:%.*]] = load <4 x i64>, ptr [[Q0]], align 2
+; CHECK-NEXT:    [[TMP3:%.*]] = sub nsw <4 x i64> [[TMP1]], [[TMP2]]
+; CHECK-NEXT:    [[TMP4:%.*]] = extractelement <4 x i64> [[TMP3]], i32 0
+; CHECK-NEXT:    [[G0:%.*]] = getelementptr inbounds i32, ptr [[R:%.*]], i64 [[TMP4]]
+; CHECK-NEXT:    [[TMP5:%.*]] = extractelement <4 x i64> [[TMP3]], i32 1
+; CHECK-NEXT:    [[G1:%.*]] = getelementptr inbounds i32, ptr [[R]], i64 [[TMP5]]
+; CHECK-NEXT:    [[TMP6:%.*]] = extractelement <4 x i64> [[TMP3]], i32 2
+; CHECK-NEXT:    [[G2:%.*]] = getelementptr inbounds i32, ptr [[R]], i64 [[TMP6]]
+; CHECK-NEXT:    [[TMP7:%.*]] = extractelement <4 x i64> [[TMP3]], i32 3
+; CHECK-NEXT:    [[G3:%.*]] = getelementptr inbounds i32, ptr [[R]], i64 [[TMP7]]
 ; CHECK-NEXT:    ret void
 ;
   %p0 = getelementptr inbounds i64, ptr %p, i64 0
   %p1 = getelementptr inbounds i64, ptr %p, i64 1
   %p2 = getelementptr inbounds i64, ptr %p, i64 2
   %p3 = getelementptr inbounds i64, ptr %p, i64 3
 
   %q0 = getelementptr inbounds i64, ptr %q, i64 0

llvm/test/Transforms/SimplifyCFG/speculate-store-opaque-pointer.ll

+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
+; RUN: opt -simplifycfg -simplifycfg-require-and-preserve-domtree=1 -opaque-pointers -S < %s | FileCheck %s
+
+declare void @unknown_fun()
+
+define void @different_type(ptr %ptr, i1 %cmp) {
+; CHECK-LABEL: @different_type(
+; CHECK-NEXT:    store i32 0, ptr [[PTR:%.*]], align 4
+; CHECK-NEXT:    br i1 [[CMP:%.*]], label [[IF_THEN:%.*]], label [[RET_END:%.*]]
+; CHECK:       if.then:
+; CHECK-NEXT:    store i64 1, ptr [[PTR]], align 4
+; CHECK-NEXT:    br label [[RET_END]]
+; CHECK:       ret.end:
+; CHECK-NEXT:    ret void
+;
+  store i32 0, ptr %ptr
+  br i1 %cmp, label %if.then, label %ret.end
+
+if.then:
+  store i64 1, ptr %ptr
+  br label %ret.end
+
+ret.end:
+  ret void
+}
+
+define void @readonly_call(ptr %ptr, i1 %cmp) {
+; CHECK-LABEL: @readonly_call(
+; CHECK-NEXT:  ret.end:
+; CHECK-NEXT:    store i32 0, ptr [[PTR:%.*]], align 4
+; CHECK-NEXT:    call void @unknown_fun() #[[ATTR0:[0-9]+]]
+; CHECK-NEXT:    [[SPEC_STORE_SELECT:%.*]] = select i1 [[CMP:%.*]], i32 1, i32 0
+; CHECK-NEXT:    store i32 [[SPEC_STORE_SELECT]], ptr [[PTR]], align 4
+; CHECK-NEXT:    ret void
+;
+  store i32 0, ptr %ptr
+  call void @unknown_fun() readonly
+  br i1 %cmp, label %if.then, label %ret.end
+
+if.then:
+  store i32 1, ptr %ptr
+  br label %ret.end
+
+ret.end:
+  ret void
+}
+
+define void @atomic_and_simple(ptr %ptr, i1 %cmp) {
+; CHECK-LABEL: @atomic_and_simple(
+; CHECK-NEXT:    store atomic i32 0, ptr [[PTR:%.*]] seq_cst, align 4
+; CHECK-NEXT:    br i1 [[CMP:%.*]], label [[IF_THEN:%.*]], label [[RET_END:%.*]]
+; CHECK:       if.then:
+; CHECK-NEXT:    store i32 1, ptr [[PTR]], align 4
+; CHECK-NEXT:    br label [[RET_END]]
+; CHECK:       ret.end:
+; CHECK-NEXT:    ret void
+;
+  store atomic i32 0, ptr %ptr seq_cst, align 4
+  br i1 %cmp, label %if.then, label %ret.end
+
+if.then:
+  store i32 1, ptr %ptr
+  br label %ret.end
+
+ret.end:
+  ret void
+}

llvm/test/Transforms/SimplifyCFG/speculate-store.ll

 if.then:
   store volatile i32 %C, i32* %A
   br label %ret.end
 
 ret.end:
   ret void
 }
 
-define void @different_type(ptr %ptr, i1 %cmp) {
-; CHECK-LABEL: @different_type(
-; CHECK-NEXT:    store i32 0, ptr [[PTR:%.*]], align 4
-; CHECK-NEXT:    br i1 [[CMP:%.*]], label [[IF_THEN:%.*]], label [[RET_END:%.*]]
-; CHECK:       if.then:
-; CHECK-NEXT:    store i64 1, ptr [[PTR]], align 4
-; CHECK-NEXT:    br label [[RET_END]]
-; CHECK:       ret.end:
-; CHECK-NEXT:    ret void
-;
-  store i32 0, ptr %ptr
-  br i1 %cmp, label %if.then, label %ret.end
-
-if.then:
-  store i64 1, ptr %ptr
-  br label %ret.end
-
-ret.end:
-  ret void
-}
-
-define void @readonly_call(ptr %ptr, i1 %cmp) {
-; CHECK-LABEL: @readonly_call(
-; CHECK-NEXT:  ret.end:
-; CHECK-NEXT:    store i32 0, ptr [[PTR:%.*]], align 4
-; CHECK-NEXT:    call void @unknown_fun() #[[ATTR0:[0-9]+]]
-; CHECK-NEXT:    [[SPEC_STORE_SELECT:%.*]] = select i1 [[CMP:%.*]], i32 1, i32 0
-; CHECK-NEXT:    store i32 [[SPEC_STORE_SELECT]], ptr [[PTR]], align 4
-; CHECK-NEXT:    ret void
-;
-  store i32 0, ptr %ptr
-  call void @unknown_fun() readonly
-  br i1 %cmp, label %if.then, label %ret.end
-
-if.then:
-  store i32 1, ptr %ptr
-  br label %ret.end
-
-ret.end:
-  ret void
-}
-
-define void @atomic_and_simple(ptr %ptr, i1 %cmp) {
-; CHECK-LABEL: @atomic_and_simple(
-; CHECK-NEXT:    store atomic i32 0, ptr [[PTR:%.*]] seq_cst, align 4
-; CHECK-NEXT:    br i1 [[CMP:%.*]], label [[IF_THEN:%.*]], label [[RET_END:%.*]]
-; CHECK:       if.then:
-; CHECK-NEXT:    store i32 1, ptr [[PTR]], align 4
-; CHECK-NEXT:    br label [[RET_END]]
-; CHECK:       ret.end:
-; CHECK-NEXT:    ret void
-;
-  store atomic i32 0, ptr %ptr seq_cst, align 4
-  br i1 %cmp, label %if.then, label %ret.end
-
-if.then:
-  store i32 1, ptr %ptr
-  br label %ret.end
-
-ret.end:
-  ret void
-}
 
 ;; Speculate a store, preceded by a local, non-escaping load
 define i32 @load_before_store_noescape(i64 %i, i32 %b)  {
 ; CHECK-LABEL: @load_before_store_noescape(
 ; CHECK-NEXT:  entry:
 ; CHECK-NEXT:    [[A:%.*]] = alloca [2 x i32], align 8
 ; CHECK-NEXT:    [[TMP0:%.*]] = bitcast [2 x i32]* [[A]] to i64*
 ; CHECK-NEXT:    store i64 4294967296, i64* [[TMP0]], align 8

llvm/test/Verifier/force-opaque-ptr.ll

-; RUN: not opt -passes=verify -force-opaque-pointers -S < %s 2>&1 | FileCheck %s
+; RUN: not opt -passes=verify -opaque-pointers -S < %s 2>&1 | FileCheck %s
 
 declare i32 @llvm.umax.i32(i32, i32)
 
 define void @intrinsic_signature_mismatch() {
 ; CHECK: Intrinsic called with incompatible signature
   call i32 @llvm.umax.i32(i32 0)
   ret void
 }

llvm/test/Verifier/musttail-invalid.ll

 ; RUN: not llvm-as %s -o /dev/null 2>&1 | FileCheck %s
+; RUN: not llvm-as %s -opaque-pointers -o /dev/null 2>&1 | FileCheck %s
 
 ; Each musttail call should fail to validate.
 
 declare x86_stdcallcc void @cc_mismatch_callee()
 define void @cc_mismatch() {
 ; CHECK: mismatched calling conv
   musttail call x86_stdcallcc void @cc_mismatch_callee()
   ret void

 
 declare void @mismatched_byval_callee({ i32 }*)
 define void @mismatched_byval({ i32 }* byval({ i32 }) %a) {
 ; CHECK: mismatched ABI impacting function attributes
   musttail call void @mismatched_byval_callee({ i32 }* %a)
   ret void
 }
 
-declare void @mismatched_byval_callee2(ptr byval(i32))
-define void @mismatched_byval2(ptr byval(i64) %a) {
-; CHECK: mismatched ABI impacting function attributes
-  musttail call void @mismatched_byval_callee2(ptr byval(i32) %a)
-  ret void
-}
-
 declare void @mismatched_inreg_callee(i32 inreg)
 define void @mismatched_inreg(i32 %a) {
 ; CHECK: mismatched ABI impacting function attributes
   musttail call void @mismatched_inreg_callee(i32 inreg %a)
   ret void
 }
 
 declare void @mismatched_sret_callee(i32* sret(i32))

llvm/test/Verifier/opaque-ptr-invalid.ll

-; RUN: not opt -verify < %s 2>&1 | FileCheck %s
+; RUN: not opt -verify -opaque-pointers < %s 2>&1 | FileCheck %s
 
 ; CHECK: Attribute 'inalloca' does not support unsized types!
-; CHECK-NEXT: void (ptr)* @f
+; CHECK-NEXT: ptr @f
 define void @f(ptr inalloca(token)) {
     ret void
 }

llvm/test/Verifier/opaque-ptr.ll

 ; NOTE: Assertions have been autogenerated by utils/update_test_checks.py
-; RUN: opt -passes=verify -S < %s | FileCheck %s
+; RUN: opt -passes=verify -opaque-pointers -S < %s | FileCheck %s
 
 define i32 @load(ptr %a) {
 ; CHECK-LABEL: @load(
 ; CHECK-NEXT:    [[I:%.*]] = load i32, ptr [[A:%.*]], align 4
 ; CHECK-NEXT:    ret i32 [[I]]
 ;
   %i = load i32, ptr %a
   ret i32 %i