Introduce GlobalSplit pass.

[pcc]

Summary

This pass splits globals into elements using inbounds annotations on
getelementptr indices.

Depends on D22793

[hfinkel]

docs/LangRef.rst

4904

I find the specification of the semantics somewhat unclear. What I think this is trying to say is that the splitpoint metadata modifies the definition of 'inbounds' on GEPs, so that instead after:

The in bounds addresses for an allocated object are all the addresses that point into the object, plus the address one byte past the end.

We add something like this:

If the object is a global partitioned with one or more !splitpoint metadata, then in bounds addresses are all of the addresses that point into the same partition of the object as the base address. The address one byte past the end of the object is part of the last partition.

Maybe this is only true if the first GEP index value is a constant?

Alternatively, we might specify that the optimizer is free to add an arbitrary amount of padding at the split points. I'm not sure this is strong enough because the later partitions might actually end up with addresses before the first partition.

[pcc]

docs/LangRef.rst

4904

Is that enough, though? It would also be possible to calculate an offseted pointer with 2's complement arithmetic with a non-inbounds getelementptr, or with ptrtoint/add/inttoptr. In both of those cases, per the as-if rule, I'd expect those calculations to produce addresses within the global variable when crossing a split point without language to the contrary.

What I was thinking was that we provide a definition of "object" such that "splitpoint" metadata would allow a global variable to be represented using multiple objects.

[pcc]

• Improve docs

[hfinkel]

docs/LangRef.rst

4904

Fair enough; I'm not sure how this is different in practice.

[eli.friedman]

The semantics you've implemented seem extremely dangerous. You're making some very strong assumptions about how exactly the optimizer manipulates GEPs. I'm not sure whether the optimizer actually breaks this particular construct, but it easily could. The optimizer can and will dig through a ConstantExpr and transform the use into something different.

You could even run into issues with the optimizer manipulating non-ConstantExpr GEPs: for example, the optimizer could transform a series of virtual calls into a loop, then index that loop by the vtable pointer; at that point, splitting the global using the obvious algorithm makes the end pointer point at the wrong global. (I don't think we do this particular kind of loop re-rolling at the moment, and LSR probably runs after the global splitting pass, but the point is that there are side-effects which have nothing to do with globals.)

---

If you want GEPs with special rules, you should attach metadata to the GEPs or something like that. A stronger version of inbounds seem more generally useful, anyway; you could use it for other things like alias analysis.

[pcc]

Thanks Eli. I had similar concerns, but they seemed largely theoretical, as I was unable to come up with a good example of a optimization pass that could reasonably break this, but your example of a loop re-roller seems at least plausible.

I will think about whether there's some reasonable IR extension we can make here. I'm thinking either a constexpr extension, or maybe something with aliases representing the individual objects.

[pcc]

Here's the extension I had in mind: allow the "inbounds" keyword to appear before any index operand of a getelementptr constant. That keyword causes any derived pointer outside of the bounds of the element referred to by that index, other than the pointer one past the end of the pointer, to be treated as a poison value. For example:

i8** getelementptr inbounds ({[4 x i8*], [4 x i8*]}, {[4 x i8*], [4 x i8*]}* @_ZTVfoo, i32 0, inbounds i32 1, i32 2)

This is a reference to the address point of the second element of _ZTVfoo, a virtual table group with two virtual tables, such that any pointer extending beyond the bounds of the second virtual table is a poison value.

Please let me know what you think.

[pcc]

Use new IR extension: D22793

[eugenis]

lib/Transforms/IPO/GlobalSplit.cpp

56

Btw would you like to add a paragraph on !type metadata to http://llvm.org/docs/LangRef.html ?

66

maybe give the new global a human-readable IR name

111

avoid creating the new undef value if there are no remaining uses

[eli.friedman]

lib/Transforms/IPO/GlobalSplit.cpp

52

Don't you also need to check the users of the GEP? The result of getInBoundsIndex() only has a semantic effect for loads and stores.

[pcc]

• Refresh, address review comments

lib/Transforms/IPO/GlobalSplit.cpp

52

If each user of a global is an inrange getelementptr constant (and nothing else can take its address, i.e. it has local linkage), it follows that any loads from or stores to that global must use a pointer derived from an inrange getelementptr constant, which is sufficient to allow us to apply the splitting transform.

Added a comment documenting this.

56

r276817

[eli.friedman]

lib/Transforms/IPO/GlobalSplit.cpp

53

My point was that loads and stores aren't the only way to use a pointer; what about pointer value comparisons?

[pcc]

lib/Transforms/IPO/GlobalSplit.cpp

53

Ah, yes. I think we can address that with language in the langref specifying that the result of a comparison is undefined if either operand was derived from an inrange GEP, with the exception of comparisons where both operands were derived from a GEP which was evaluated with identical operands up to and including the inrange operand. I'll update my other patch.

[pcc]

• Require at least local_unnamed_addr on globals to be split

[eli.friedman]

I don't see how unnamed_addr is relevant here. It means that the code won't compare the address of the global in question to the address of a different global. It doesn't have anything to do with comparisons involving different addresses within the same global.

[hfinkel]

In D22295#497141, @eli.friedman wrote:

I don't see how unnamed_addr is relevant here. It means that the code won't compare the address of the global in question to the address of a different global. It doesn't have anything to do with comparisons involving different addresses within the same global.

I thought about that too, but isn't a split global just like a collection of several globals?

[eli.friedman]

I'll try to outline a scenario where this misbehaves...

Suppose you have an unnamed_addr global, call it "const int a[2][5]", and a loop which sums up all the elements of a[0]. You can express the sum something like this: int* p = a[0]; while (p != a[1]) { sum += *p; ++p; }. Both a[0] and a[1] can be marked inrange: a[0] because all the loads are in range, a[1] because there aren't any memory operations over it. The loop explodes for obvious reasons if you split a[1] into a separate global.

The obvious way to make this illegal is to make marking a[1] inrange illegal (the "comparisons are undefined" rule). There isn't any marking you can put on the global which will help because the optimizer can insert the pointer comparison without knowing a global is even involved.

[pcc]

Thanks Eli. Apologies for the delayed response, I just returned from a long vacation.

Having given this a little more thought, the "comparisons are undefined" rule, while in my opinion perhaps too conservative, does seem like a reasonable starting point. I'll update this patch and the other one.

[pcc]

• Revert "Require at least local_unnamed_addr on globals to be split"

[pcc]

Refresh

[pcc]

Ping

[pcc]

This is unblocked now.

[eugenis]

What happens to !dbg metadata when a global is split?

llvm/lib/Transforms/IPO/GlobalSplit.cpp

82 ↗

(On Diff #75492)

This is N^2.
N is probably very small in practice, but I guess there can be outliers. Consider switching to an NlogN implementation?

103 ↗

(On Diff #75492)

SmallVector?

[pcc]

• Use SmallVector

[pcc]

In D22295#592417, @eugenis wrote:

What happens to !dbg metadata when a global is split?

It gets discarded. This isn't important for the vtable use case (vtables don't normally have debug info of their own). I think we may be able to represent this using DW_OP_piece, but that can be done separately.

llvm/lib/Transforms/IPO/GlobalSplit.cpp

82 ↗

(On Diff #75492)

On my machine this pass takes 0.4 seconds to run during an LTO link of chrome. So I don't think it's worth making major changes for performance here.

[eugenis]

LGTM
Please add a FIXME to preserve/update debug metadata.

llvm/lib/Transforms/IPO/GlobalSplit.cpp

82 ↗

(On Diff #75492)

OK, that's reasonable.

docs/LangRef.rst

.. code-block:: llvm
    declare void @foo(i8*)
    declare i8* @getPointer(i8*)
    declare i8* @llvm.invariant.group.barrier(i8*)
    
    !0 = !{!"magic ptr"}
    !1 = !{!"other ptr"}
 
 
+'``type``' Metadata
+^^^^^^^^^^^^^^^^^^^
+
+See :doc:`TypeMetadata`.
+
+'``splitpoint``' Metadata
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This metadata can be used to annotate global variables with split points.
+Each split point specifies a byte offset within the global, and acts as a
+promise that no pointer derived from a constant-offset reference to the global
+will cross the boundary of a split point. The metadata allows the optimizer
+to split a global variable at split points if possible and beneficial.
+
+Under the Itanium C++ ABI, Clang attaches split points at each boundary
+between virtual tables in a virtual table group, because the virtual tables
+are independent entities as far as a user of a virtual table within an object
+is concerned, provided that the dynamic type is unknown. A compiler cannot
+legally generate code to adjust a virtual table pointer to point to another
+virtual table in the virtual table group, as the primary virtual table will
+have an unknown number of virtual functions.
+
+A virtual table split is possible if it has local linkage. This guarantees
+that nothing outside of the module can reference the virtual table group
+directly. Typically, when LTO is used, the linker is able to internalize
+most virtual tables.
+
+A split is beneficial if either of the whole-program devirtualization or
+control flow integrity features are being used. In the former case, under
+virtual constant propagation we are able to place propagated constants
+directly in front of virtual tables of classes with multiple bases. In the
+latter case, we can arrange virtual tables with multiple bases in a more
+hierarchical order, which reduces the required amount of runtime data and
+simplifies the required checks.
 
 Module Flags Metadata
 =====================
 
 Information about the module as a whole is difficult to convey to LLVM's
 subsystems. The LLVM IR isn't sufficient to transmit this information.
 The ``llvm.module.flags`` named metadata exists in order to facilitate
 this. These flags are in the form of key / value pairs --- much like a

include/llvm/IR/LLVMContext.h

enum {
     MD_dereferenceable = 12,          // "dereferenceable"
     MD_dereferenceable_or_null = 13,  // "dereferenceable_or_null"
     MD_make_implicit = 14,            // "make.implicit"
     MD_unpredictable = 15,            // "unpredictable"
     MD_invariant_group = 16,          // "invariant.group"
     MD_align = 17,                    // "align"
     MD_loop = 18,                     // "llvm.loop"
     MD_type = 19,                     // "type"
+    MD_splitpoint = 20,               // "splitpoint"
   };
 
   /// Known operand bundle tag IDs, which always have the same value.  All
   /// operand bundle tags that LLVM has special knowledge of are listed here.
   /// Additionally, this scheme allows LLVM to efficiently check for specific
   /// operand bundle tags without comparing strings.
   enum {
     OB_deopt = 0,         // "deopt"

include/llvm/InitializePasses.h

 void initializeFunctionImportPassPass(PassRegistry &);
 void initializeGCMachineCodeAnalysisPass(PassRegistry&);
 void initializeGCModuleInfoPass(PassRegistry&);
 void initializeGCOVProfilerLegacyPassPass(PassRegistry&);
 void initializeGVNLegacyPassPass(PassRegistry&);
 void initializeGlobalDCELegacyPassPass(PassRegistry&);
 void initializeGlobalMergePass(PassRegistry&);
 void initializeGlobalOptLegacyPassPass(PassRegistry&);
+void initializeGlobalSplitPass(PassRegistry&);
 void initializeGlobalsAAWrapperPassPass(PassRegistry&);
 void initializeGuardWideningLegacyPassPass(PassRegistry&);
 void initializeIPCPPass(PassRegistry&);
 void initializeIPSCCPLegacyPassPass(PassRegistry &);
 void initializeIRTranslatorPass(PassRegistry &);
 void initializeIVUsersPass(PassRegistry&);
 void initializeIfConverterPass(PassRegistry&);
 void initializeImplicitNullChecksPass(PassRegistry&);

include/llvm/Transforms/IPO.h

 
 /// \brief This pass export CFI checks for use by external modules.
 ModulePass *createCrossDSOCFIPass();
 
 /// \brief This pass implements whole-program devirtualization using type
 /// metadata.
 ModulePass *createWholeProgramDevirtPass();
 
+/// This pass splits globals into pieces for the benefit of whole-program
+/// devirtualization and control-flow integrity.
+ModulePass *createGlobalSplitPass();
+
 //===----------------------------------------------------------------------===//
 // SampleProfilePass - Loads sample profile data from disk and generates
 // IR metadata to reflect the profile.
 ModulePass *createSampleProfileLoaderPass();
 ModulePass *createSampleProfileLoaderPass(StringRef Name);
 
 } // End llvm namespace
 
 #endif

lib/IR/LLVMContext.cpp

LLVMContext::LLVMContext() : pImpl(new LLVMContextImpl(*this)) {
   unsigned LoopID = getMDKindID("llvm.loop");
   assert(LoopID == MD_loop && "llvm.loop kind id drifted");
   (void)LoopID;
 
   unsigned TypeID = getMDKindID("type");
   assert(TypeID == MD_type && "type kind id drifted");
   (void)TypeID;
 
+  unsigned SplitPointID = getMDKindID("splitpoint");
+  assert(SplitPointID == MD_splitpoint && "splitpoint kind id drifted");
+  (void)SplitPointID;
+
   auto *DeoptEntry = pImpl->getOrInsertBundleTag("deopt");
   assert(DeoptEntry->second == LLVMContext::OB_deopt &&
          "deopt operand bundle id drifted!");
   (void)DeoptEntry;
 
   auto *FuncletEntry = pImpl->getOrInsertBundleTag("funclet");
   assert(FuncletEntry->second == LLVMContext::OB_funclet &&
          "funclet operand bundle id drifted!");

lib/Transforms/IPO/CMakeLists.txt

 add_llvm_library(LLVMipo
   ArgumentPromotion.cpp
   BarrierNoopPass.cpp
   ConstantMerge.cpp
   CrossDSOCFI.cpp
   DeadArgumentElimination.cpp
   ElimAvailExtern.cpp
   ExtractGV.cpp
   ForceFunctionAttrs.cpp
   FunctionAttrs.cpp
   FunctionImport.cpp
   GlobalDCE.cpp
   GlobalOpt.cpp
+  GlobalSplit.cpp
   IPConstantPropagation.cpp
   IPO.cpp
   InferFunctionAttrs.cpp
   InlineAlways.cpp
   InlineSimple.cpp
   Inliner.cpp
   Internalize.cpp
   LoopExtractor.cpp

lib/Transforms/IPO/GlobalSplit.cpp

+//===- GlobalSplit.cpp - global variable splitter -------------------------===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This pass uses !splitpoint metadata to split globals where beneficial. Clang
+// currently attaches this metadata to virtual table globals under the Itanium
+// ABI for the benefit of the whole-program virtual call optimization and
+// control flow integrity passes.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/Transforms/IPO.h"
+#include "llvm/IR/Constants.h"
+#include "llvm/IR/GlobalVariable.h"
+#include "llvm/IR/Intrinsics.h"
+#include "llvm/IR/Module.h"
+#include "llvm/IR/Operator.h"
+#include "llvm/Pass.h"
+
+#include <set>
+
+using namespace llvm;
+
+namespace {
+
+bool splitGlobal(GlobalVariable &GV) {
+  // If the address of the global is taken outside of the module, we cannot
+  // apply this transformation.
+  if (!GV.hasLocalLinkage())
+    return false;
+
+  // We currently only know how to split ConstantArrays.
+  auto *Init = dyn_cast_or_null<ConstantArray>(GV.getInitializer());
+  if (!Init)
+    return false;
+
+  const DataLayout &DL = GV.getParent()->getDataLayout();
+
+  Type *ElemTy = Init->getType()->getElementType();
+  uint64_t ElemSize = DL.getTypeAllocSize(ElemTy);
+
+  SmallVector<MDNode *, 2> SplitPointMD, Types;
+  GV.getMetadata(LLVMContext::MD_splitpoint, SplitPointMD);
+  GV.getMetadata(LLVMContext::MD_type, Types);
+
+  // Build the set of offsets at which we need to split.
+  std::set<uint64_t> SplitPoints;
+  for (MDNode *SplitPoint : SplitPointMD) {
+    uint64_t ByteOffset =
+        cast<ConstantInt>(
+            cast<ConstantAsMetadata>(SplitPoint->getOperand(0))->getValue())
+            ->getZExtValue();
+    if (ByteOffset % ElemSize)
+      return false;
+    size_t Index = ByteOffset / ElemSize;
+    if (Index >= Init->getNumOperands())
+      return false;
+    SplitPoints.insert(Index);
+  }
+
+  if (SplitPoints.empty())
+    return false;
+
+  IntegerType *Int8Ty = Type::getInt8Ty(GV.getContext());
+  PointerType *Int8PtrTy = Type::getInt8PtrTy(GV.getContext());
+  IntegerType *Int32Ty = Type::getInt32Ty(GV.getContext());
+
+  // Each element of SplitPoints corresponds to the last operand of a "piece"
+  // of the original initializer. This element terminates the final piece.
+  SplitPoints.insert(Init->getNumOperands());
+
+  std::map<uint64_t, GlobalVariable *> SplitGlobals;
+  uint64_t LastSplitPoint = 0;
+  for (auto SplitPoint : SplitPoints) {
+    // Build a global representing this split piece.
+    std::vector<Constant *> Ops;
+    for (unsigned Op = LastSplitPoint; Op != SplitPoint; ++Op)
+      Ops.push_back(Init->getOperand(Op));
+    ArrayType *Ty = ArrayType::get(ElemTy, Ops.size());
+    Constant *NewInit = ConstantArray::get(Ty, Ops);
+    auto *SplitGV = new GlobalVariable(*GV.getParent(), Ty, GV.isConstant(),
+                                       GlobalValue::PrivateLinkage, NewInit);
+    SplitGlobals[LastSplitPoint] = SplitGV;
+
+    // Rebuild type metadata, adjusting by the split offset.
+    for (MDNode *Type : Types) {
+      uint64_t ByteOffset = cast<ConstantInt>(
+              cast<ConstantAsMetadata>(Type->getOperand(0))->getValue())
+              ->getZExtValue();
+      if (ByteOffset < LastSplitPoint * ElemSize ||
+          ByteOffset >= SplitPoint * ElemSize)
+        continue;
+      SplitGV->addMetadata(
+          LLVMContext::MD_type,
+          *MDNode::get(GV.getContext(),
+                       {ConstantAsMetadata::get(ConstantInt::get(
+                            Int32Ty, ByteOffset - (LastSplitPoint * ElemSize))),
+                        Type->getOperand(1)}));
+    }
+
+    LastSplitPoint = SplitPoint;
+  }
+
+  // Now, replace all uses of the original global with references to the split
+  // globals. This is a little tricky because we need to walk use lists in order
+  // to calculate offsets.
+
+  std::function<void(Constant * C, uint64_t Offset)> ApplyReplacements;
+  ApplyReplacements = [&](Constant *C, uint64_t Offset) {
+    // Replace all users first in order to ensure that we remain valid.
+    for (User *U : C->users()) {
+      // We only handle constant offsets references here. Any non-constant
+      // offsets are presumed to refer to the same global.
+      ConstantExpr *CE = dyn_cast<ConstantExpr>(U);
+      if (!CE)
+        continue;
+
+      switch (CE->getOpcode()) {
+      case Instruction::BitCast:
+        ApplyReplacements(CE, Offset);
+        break;
+
+      case Instruction::GetElementPtr: {
+        APInt APOffset(DL.getPointerSizeInBits(), 0);
+        cast<GEPOperator>(CE)->accumulateConstantOffset(DL, APOffset);
+        ApplyReplacements(CE, Offset + APOffset.getZExtValue());
+        break;
+      }
+      }
+    }
+
+    // Find the global corresponding to the offset calculated so far.
+    auto I = SplitGlobals.upper_bound(Offset / ElemSize);
+    --I;
+
+    // Build a offseted reference to the global adjusting for the calculated
+    // offset and the split offset.
+    Constant *Replacement = ConstantExpr::getBitCast(
+        ConstantExpr::getGetElementPtr(
+            Int8Ty, ConstantExpr::getBitCast(I->second, Int8PtrTy),
+            ArrayRef<Constant *>{
+                ConstantInt::get(Int32Ty, Offset - I->first * ElemSize)}),
+        C->getType());
+
+    // Finally, perform the replacement.
+    C->replaceAllUsesWith(Replacement);
+  };
+
+  ApplyReplacements(&GV, 0);
+
+  // Finally, remove the original global.
+  GV.eraseFromParent();
+  return true;
+}
+
+bool splitGlobals(Module &M) {
+  // First, see if the module uses either of the llvm.type.test or
+  // llvm.type.checked.load intrinsics, which indicates that splitting globals
+  // may be beneficial.
+  Function *TypeTestFunc =
+      M.getFunction(Intrinsic::getName(Intrinsic::type_test));
+  Function *TypeCheckedLoadFunc =
+      M.getFunction(Intrinsic::getName(Intrinsic::type_checked_load));
+  if ((!TypeTestFunc || TypeTestFunc->use_empty()) &&
+      (!TypeCheckedLoadFunc || TypeCheckedLoadFunc->use_empty()))
+    return false;
+
+  bool Changed = false;
+  for (auto I = M.global_begin(); I != M.global_end();) {
+    GlobalVariable &GV = *I;
+    ++I;
+    Changed |= splitGlobal(GV);
+  }
+  return Changed;
+}
+
+struct GlobalSplit : public ModulePass {
+  static char ID;
+  GlobalSplit() : ModulePass(ID) {
+    initializeGlobalSplitPass(*PassRegistry::getPassRegistry());
+  }
+  bool runOnModule(Module &M) {
+    if (skipModule(M))
+      return false;
+
+    return splitGlobals(M);
+  }
+};
+
+}
+
+INITIALIZE_PASS(GlobalSplit, "globalsplit", "Global splitter", false, false)
+char GlobalSplit::ID = 0;
+
+ModulePass *llvm::createGlobalSplitPass() {
+  return new GlobalSplit;
+}

lib/Transforms/IPO/IPO.cpp

void llvm::initializeIPO(PassRegistry &Registry) {
   initializeArgPromotionPass(Registry);
   initializeConstantMergeLegacyPassPass(Registry);
   initializeCrossDSOCFIPass(Registry);
   initializeDAEPass(Registry);
   initializeDAHPass(Registry);
   initializeForceFunctionAttrsLegacyPassPass(Registry);
   initializeGlobalDCELegacyPassPass(Registry);
   initializeGlobalOptLegacyPassPass(Registry);
+  initializeGlobalSplitPass(Registry);
   initializeIPCPPass(Registry);
   initializeAlwaysInlinerPass(Registry);
   initializeSimpleInlinerPass(Registry);
   initializeInferFunctionAttrsLegacyPassPass(Registry);
   initializeInternalizeLegacyPassPass(Registry);
   initializeLoopExtractorPass(Registry);
   initializeBlockExtractorPassPass(Registry);
   initializeSingleLoopExtractorPass(Registry);

lib/Transforms/IPO/PassManagerBuilder.cpp

if (OptLevel > 1) {
     PM.add(createIPSCCPPass());
   }
 
   // Infer attributes about definitions. The readnone attribute in particular is
   // required for virtual constant propagation.
   PM.add(createPostOrderFunctionAttrsLegacyPass());
   PM.add(createReversePostOrderFunctionAttrsPass());
 
+  // Split globals using !splitpoint metadata. This can help improve the quality
+  // of generated code when virtual constant propagation or control flow
+  // integrity are enabled.
+  PM.add(createGlobalSplitPass());
+
   // Apply whole-program devirtualization and virtual constant propagation.
   PM.add(createWholeProgramDevirtPass());
 
   // That's all we need at opt level 1.
   if (OptLevel == 1)
     return;
 
   // Now that we internalized some globals, see if we can hack on them!

test/Transforms/GlobalSplit/basic.ll

+; RUN: opt -S -globalsplit %s | FileCheck %s
+
+target datalayout = "e-p:64:64"
+target triple = "x86_64-unknown-linux-gnu"
+
+; CHECK: @vtt = constant [3 x i8*] [i8* bitcast ([2 x i8* ()*]* @0 to i8*), i8* getelementptr (i8, i8* bitcast ([2 x i8* ()*]* @0 to i8*), i32 8), i8* bitcast ([1 x i8* ()*]* @1 to i8*)]
+@vtt = constant [3 x i8*] [
+  i8* bitcast (i8* ()** getelementptr ([3 x i8* ()*], [3 x i8* ()*]* @global, i32 0, i32 0) to i8*),
+  i8* bitcast (i8* ()** getelementptr ([3 x i8* ()*], [3 x i8* ()*]* @global, i32 0, i32 1) to i8*),
+  i8* bitcast (i8* ()** getelementptr ([3 x i8* ()*], [3 x i8* ()*]* @global, i32 0, i32 2) to i8*)
+]
+
+; CHECK-NOT: @global
+; CHECK: @0 = private constant [2 x i8* ()*] [i8* ()* @f, i8* ()* @g], !type [[T1:![0-9]+$]]
+; CHECK: @1 = private constant [1 x i8* ()*] [i8* ()* @h], !type [[T2:![0-9]+$]]
+; CHECK-NOT: @global
+@global = internal constant [3 x i8* ()*] [i8* ()* @f, i8* ()* @g, i8* ()* @h], !type !0, !type !1, !splitpoint !2
+
+; CHECK: define i8* @f()
+define i8* @f() {
+  ; CHECK-NEXT: ret i8* bitcast ([2 x i8* ()*]* @0 to i8*)
+  ret i8* bitcast (i8* ()** getelementptr ([3 x i8* ()*], [3 x i8* ()*]* @global, i32 0, i32 0) to i8*)
+}
+
+; CHECK: define i8* @g()
+define i8* @g() {
+  ; CHECK-NEXT: ret i8* getelementptr (i8, i8* bitcast ([2 x i8* ()*]* @0 to i8*), i32 8)
+  ret i8* bitcast (i8* ()** getelementptr ([3 x i8* ()*], [3 x i8* ()*]* @global, i32 0, i32 1) to i8*)
+}
+
+; CHECK: define i8* @h()
+define i8* @h() {
+  ; CHECK-NEXT: ret i8* bitcast ([1 x i8* ()*]* @1 to i8*)
+  ret i8* bitcast (i8* ()** getelementptr ([3 x i8* ()*], [3 x i8* ()*]* @global, i32 0, i32 2) to i8*)
+}
+
+define void @foo() {
+  %p = call i1 @llvm.type.test(i8* null, metadata !"")
+  ret void
+}
+
+declare i1 @llvm.type.test(i8*, metadata) nounwind readnone
+
+; CHECK: [[T1]] = !{i32 8, !"foo"}
+; CHECK: [[T2]] = !{i32 0, !"bar"}
+!0 = !{i32 8, !"foo"}
+!1 = !{i32 16, !"bar"}
+!2 = !{i32 16}

test/Transforms/GlobalSplit/invalid-offset.ll

+; RUN: opt -S -globalsplit %s | FileCheck %s
+
+target datalayout = "e-p:64:64"
+target triple = "x86_64-unknown-linux-gnu"
+
+; CHECK: @global1 =
+@global1 = constant [3 x i8* ()*] [i8* ()* @f, i8* ()* @g, i8* ()* @h], !splitpoint !0
+
+; CHECK: @global2 =
+@global2 = constant [3 x i8* ()*] [i8* ()* @f, i8* ()* @g, i8* ()* @h], !splitpoint !1
+
+define i8* @f() {
+  ret i8* null
+}
+
+define i8* @g() {
+  ret i8* null
+}
+
+define i8* @h() {
+  ret i8* null
+}
+
+define void @foo() {
+  %p = call i1 @llvm.type.test(i8* null, metadata !"")
+  ret void
+}
+
+declare i1 @llvm.type.test(i8*, metadata) nounwind readnone
+
+!0 = !{i32 3}
+!1 = !{i32 48}

test/Transforms/GlobalSplit/non-beneficial.ll

+; RUN: opt -S -globalsplit %s | FileCheck %s
+
+target datalayout = "e-p:64:64"
+target triple = "x86_64-unknown-linux-gnu"
+
+; CHECK: @global =
+@global = constant [3 x i8* ()*] [i8* ()* @f, i8* ()* @g, i8* ()* @h], !splitpoint !0
+
+define i8* @f() {
+  ret i8* null
+}
+
+define i8* @g() {
+  ret i8* null
+}
+
+define i8* @h() {
+  ret i8* null
+}
+
+!0 = !{i32 16}

test/Transforms/GlobalSplit/nonlocal.ll

+; RUN: opt -S -globalsplit %s | FileCheck %s
+
+target datalayout = "e-p:64:64"
+target triple = "x86_64-unknown-linux-gnu"
+
+; CHECK: @global =
+@global = constant [3 x i8* ()*] [i8* ()* @f, i8* ()* @g, i8* ()* @h], !splitpoint !0
+
+define i8* @f() {
+  ret i8* null
+}
+
+define i8* @g() {
+  ret i8* null
+}
+
+define i8* @h() {
+  ret i8* null
+}
+
+define void @foo() {
+  %p = call i1 @llvm.type.test(i8* null, metadata !"")
+  ret void
+}
+
+declare i1 @llvm.type.test(i8*, metadata) nounwind readnone
+
+!0 = !{i32 16}