Extend the statepoint intrinsic to allow statepoints to be marked as transitions from GC-aware code to code that is not GC-aware.

[pgavlin]

Summary

This changes the shape of the statepoint intrinsic from:

@llvm.experimental.gc.statepoint(anyptr target, i32 # call args, i32 unused, ...call args, i32 # deopt args, ...deopt args, ...gc args)

to:

@llvm.experimental.gc.statepoint(anyptr target, i32 # call args, i32 flags, ...call args, i32 # transition args, ...transition args, i32 # deopt args, ...deopt args, ...gc args)

This extension offers the backend the opportunity to insert (somewhat) arbitrary code to manage the transition from GC-aware code to code that is not GC-aware and back.

In order to support the injection of transition code, this extension wraps the STATEPOINT ISD node generated by the usual lowering lowering with two additional nodes: GC_TRANSITION_START and GC_TRANSITION_END. The transition arguments that were passed passed to the intrinsic (if any) are lowered and provided as operands to these nodes and may be used by the backend during code generation.

Eventually, the lowering of the GC_TRANSITION_{START,END} nodes should be informed by the GC strategy in use for the function containing the intrinsic call; for now, these nodes are instead replaced with no-ops.

[reames]

General approach seems reasonable. The documentation needs major improvement though before this is ready to go in. If we hadn't talked about this offline, I would have been very confused about what this patch was trying to accomplish.

docs/Statepoints.rst

253

Which bit of the flags is used for this? From the current documentation, it's not clear how to use these.

264

What does "normally" mean?

268

oeprands -> operands

271

Meta point: you never really define transition sequence or transition. Add section to the documentation with the background (preferably with an example) and reference it appropriately.

As part of this, justify why the gc attributes isn't enough. Your function pointer case is fine, just mention it since the reader might be wondering.

485

If you feel like it, you can delete the deopt args from this example. If not, I'll do it at some point.

include/llvm/IR/Statepoint.h

27

Consider using a class enum here. Possibly:
enum class StatepointFlags {
....
}

76

I'd prefer to see a getFlags function added and this become, getFlags() & statepoint::GCTransition.

120

Isn't this just return call_args_end()

It looks like you're defining the length argument to be part of the gc_transition_args range? At first, I thought you were breaking with the convention, but then I noticed you copied this from vm_state_begin and friends. That really doesn't seem like the right approach. Having the length being consider part of the arg bundle just doesn't seem right.

I'm okay with this landing as is for the moment, but I'll likely fix all of these in the not too distant future.

lib/CodeGen/SelectionDAG/StatepointLowering.cpp

600

You're changes here collide with a review Sanjoy has up for review. Please talk with him to separate the shared naming/refactoring pieces. Otherwise, whoever commits second will have a nasty merge conflict to fix.

620

What's the format here? Are what are the source values used for? This should probably be documented somewhere as well. At least in comment form, probably as actual documentation.

675

Doesn't this need to be Chain since you just added a new node?

692

You've got name colision with the Ops at the outer scope. Given the structure of the code, it most likely makes sense to either sink the statepoint operands into a scoped block of their own, or use another name here.

lib/IR/Verifier.cpp

1534

By converting to a 32 bit value, I believe you have silent truncation here. This should probably be a uint64_t.

1535

A more extensible way to express this would to add a MaskAllSet to the enum, then write this as:
Flags & ~MaskAllSet == 0

1538

Adding a check that says "if the callee has a gc attribute that is not the same as the caller, the transition bit should be set" would seem reasonable. Is there a reason not to do this? Or the inverse check for when the flag is set?

Actually, such a check probably *doesn't* belong in the Verifier. You could have unreachable code that gets exposed during optimization and the resulting IR would fail that check, but still be semantically valid IR. It should be a lint rule and probably an instcombine rule to replace a mismatch with undef. :) A follow up patch or filed bug is fine.

lib/Target/X86/X86ISelLowering.cpp

17418

A bit of context would be good here. Readers of the code need to know why these are here.

17423

Just use getGluedNode?

17427

This looks slightly suspicious to me. I wouldn't necessarily expect a noop node to take a control and glue. Are you sure the rest of the backend will expect this?

[sanjoy]

lib/CodeGen/SelectionDAG/StatepointLowering.cpp

600

The latest version of my change (http://reviews.llvm.org/D9480) should not conflict with this one.

[pgavlin]

Thanks for the feedback! I'll post a new version of the code with the requested changes later today.

docs/Statepoints.rst

253

The first bit. Should I add a table for the flags bits here, or out-of-line?

264

I intended this to mean that transition arguments are not lowered in the same fashion as deopt args or GC args (e.g. no spilling is done). Instead, they are lowered however ISel would usually lower them.

268

Yup. Thanks.

271

Good point. I'll add the suggested background.

485

I will leave the deopt args for now.

include/llvm/IR/Statepoint.h

27

Will do.

76

Will do.

120

I thought that this pattern was a bit weird myself. The value I saw here was the assertion, but it's probably acceptable to write the assertion as

assert((call_args_end() - StatepointCS.args_begin()) <= (int)StatepointCS.arg_size());

However, I'd rather be consistent for now and fix all of these in a followup change.

lib/CodeGen/SelectionDAG/StatepointLowering.cpp

600

I don't mind dealing with any merge pain. It does look like Sanjoy is correct, however, and things should be pretty clean.

620

The SrcValues are used so that later lowering can form MachinePointerInfos. As suggested, I'll document the format. Should I just add a section to the existing statepoint docs?

675

Yes, good catch.

692

The name hiding is deliberate, and IMHO more reliable: we shouldn't be accessing the operands to the STATEPOINT node itself here.

lib/IR/Verifier.cpp

1534

Yes, you're correct. This pattern is common throughout statepoint validation--should I fix the other instances now, or file a bug and follow up with another patch?

1535

I agree. I had originally intended to do just that, but could not think of a terribly robust way of defining MaskAllSet.

1538

I'll file a bug for this one :)

lib/Target/X86/X86ISelLowering.cpp

17418

Will add.

17423

Using getGluedNode would change the code to something like:

SDNode *GluedNode = Op->getGluedNode();
if (GluedNode)

Ops.push_back(SDValue(GluedNode, GluedNode->getNumValues() - 1);

I don't feel terribly strongly one way or the other: either way we're following a convention (glue is the terminal {operand,value}). FWIW, the code that is already present is a pretty common pattern.

17427

I tested this by running all of the statepoint lowering tests with the GC transition flag set to '1'. There were no failures (or even asm diffs, as the NOOP instructions are removed by dead machine instruction elimination), which seems like a reasonable indication that the backend can tolerate this.

[pgavlin]

Expand the documentation for GC transitions and address CR feedback.

[reames]

Responding to questions in comments, not yet actually reviewing most recent code.

docs/Statepoints.rst

253

No strong preference. Do what you think creates more readable docs.

include/llvm/IR/Statepoint.h

120

Sounds reasonable. Any chance I can convince you to do the cleanup? If not, I'll try to get to it myself.

lib/CodeGen/SelectionDAG/StatepointLowering.cpp

620

Yep.

692

I generally dislike any code where you can disable a declaration and silently get wrong behaviour.

Please change this. I will not sign off on this with the name collision. I don't really care how you resolve the collision, but I do care you resolve it.

lib/IR/Verifier.cpp

1534

File a bug, fix later. I can't ask you to fix all of my bugs in one go. :)

1535

In the enum declaration:
MaskAllSet = GCTransition | NextFlag | NextFlag,

lib/Target/X86/X86ISelLowering.cpp

17423

Eh, I don't really care here. I'll defer to your preference.

17427

Agreed.

[reames]

This is looking really close to ready to land. Not quite there yet, but probably only one more round of updates.

I did notice that there are no tests specifically for the lowering of statepoints with transition arguments. You should be able to write tests using statepoint-example, even if they're not the most useful tests in the world. At minimum, we can make sure the backend doesn't crash when presented with such a statepoint. The challenge in testing this really makes me think we should have the CLR GC strategy be included as a builtin GC.

docs/Statepoints.rst

211

This paragraph gets confusing. I think you'd be better off introducing a hypothetical GC, describe briefly how you might add support for such a GC, and then show the resulting code. Just make it clear that such a GC is hypothetical. :)

You might also mention where this is actually used. (i.e. the CLR GC support.)

I'd be tempted to have you guys actually add your strategy so that we can have the actual support conditional on the gc name and just use that for our example.

257

LSB? MSB? What order of the bits are you using?

262

Currently unused, but required to be unset to allow possible future extensions.

lib/CodeGen/SelectionDAG/StatepointLowering.cpp

664

This shift should be define in StackMaps.h for the StatepointOperands class with a more clear name.

The other option is to just split this operand into 2. I have no idea why I masked them to start with, that's just unnecessary complexity. Might be better to do this as a separate change though.

695

You could shorten this comment if desired by just referencing the format already defined for GC_TRANSITION_START.

[pgavlin]

Expand documentation and address further CR feedback.

[pgavlin]

docs/Statepoints.rst

211

Noted. I'll see if we can get a GC strategy for CoreCLR added in the very near future.

include/llvm/IR/Statepoint.h

120

Sure, I'm happy to do the cleanup.

lib/CodeGen/SelectionDAG/StatepointLowering.cpp

664

I like the sound of the latter option. Do you mind if I do this in a separate change?

692

That's reasonable enough.

695

Done.

[reames]

Still missing the requested test case, but otherwise LGTM. Once that's in place, let me know, and I'll commit. (Assuming you haven't gotten commit access yet.)

[reames]

lib/CodeGen/SelectionDAG/StatepointLowering.cpp

664

I'd prefer that actually. Given the future change, this portion is fine for submission.

[pgavlin]

Added a test for GC transition lowering.

[pgavlin]

I do have commit access now, so I can commit this once you've signed off.

[reames]

LGTM w/the comment and test fixes mentioned.

test/CodeGen/X86/statepoint-gctransition-call-lowering.ll

2 ↗

(On Diff #25121)

Stale comment.

16 ↗

(On Diff #25121)

Please add at least one test which includes transition arguments which get lowered, but dropped due to the noops. I want to exercise the newly added code.

docs/Statepoints.rst

 This example was taken from the tests for the :ref:`RewriteStatepointsForGC` utility pass.  As such, it's full StackMap can be easily examined with the following command.
 
 .. code-block:: bash
 
   opt -rewrite-statepoints-for-gc test/Transforms/RewriteStatepointsForGC/basics.ll -S | llc -debug-only=stackmaps
 
 
 
 
 
 Intrinsics
 ===========
 
 'llvm.experimental.gc.statepoint' Intrinsic
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Syntax:
 """""""
 
 ::
 
       declare i32
         @llvm.experimental.gc.statepoint(func_type <target>, 
-                       i64 <#call args>. i64 <unused>, 
+                       i64 <#call args>. i64 <flags>,
                        ... (call parameters),
+                       i64 <# transition args>, ... (transition parameters),
                        i64 <# deopt args>, ... (deopt parameters),
                        ... (gc parameters))
 
 Overview:
 """""""""
 
 The statepoint intrinsic represents a call which is parse-able by the
 runtime.
 
 Operands:
 """""""""
 
 The 'target' operand is the function actually being called.  The
 target can be specified as either a symbolic LLVM function, or as an
 arbitrary Value of appropriate function type.  Note that the function
 type must match the signature of the callee and the types of the 'call
 parameters' arguments.
 
 The '#call args' operand is the number of arguments to the actual
 call.  It must exactly match the number of arguments passed in the
 'call parameters' variable length section.
 
-The 'unused' operand is unused and likely to be removed.  Please do
-not use.
+The 'flags' operand is used to specify extra information about the
+statepoint. This is currently only used to mark certain statepoints
+as transitions from GC-aware code to code that is not GC-aware.
 
 The 'call parameters' arguments are simply the arguments which need to
 be passed to the call target.  They will be lowered according to the
 specified calling convention and otherwise handled like a normal call
 instruction.  The number of arguments must exactly match what is
 specified in '# call args'.  The types must match the signature of
 'target'.
 
+The 'transition parameters' arguments contain an arbitrary list of
+Values which need to be passed to GC transition code. They will be
+lowered normally, and will be made available as operands to the
+appropriate nodes in the selection DAG. It is assumed that these
+arguments must be available before and after (but not necessarily
+during) the execution of the callee. The '# transition args' field
+indicates how many oeprands are to be interpreted as 'transition
+parameters'.
+
 The 'deopt parameters' arguments contain an arbitrary list of Values
 which is meaningful to the runtime.  The runtime may read any of these
 values, but is assumed not to modify them.  If the garbage collector
 might need to modify one of these values, it must also be listed in
 the 'gc pointer' argument list.  The '# deopt args' field indicates
 how many operands are to be interpreted as 'deopt parameters'.
 
 The 'gc parameters' arguments contain every pointer to a garbage
 collector object which potentially needs to be updated by the garbage

 distinguish between GC references and non-GC references in IR it is given.
 
 As an example, given this code:
 
 .. code-block:: llvm
 
   define i8 addrspace(1)* @test1(i8 addrspace(1)* %obj) 
          gc "statepoint-example" {
-    call i32 (void ()*, i32, i32, ...)* @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+    call i32 (void ()*, i32, i32, ...)* @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
     ret i8 addrspace(1)* %obj
   }
 
 The pass would produce this IR:
 
 .. code-block:: llvm
 
   define i8 addrspace(1)* @test1(i8 addrspace(1)* %obj) 
          gc "statepoint-example" {
-    %0 = call i32 (void ()*, i32, i32, ...)* @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i8 addrspace(1)* %obj)
+    %0 = call i32 (void ()*, i32, i32, ...)* @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i8 addrspace(1)* %obj)
     %obj.relocated = call coldcc i8 addrspace(1)* @llvm.experimental.gc.relocate.p1i8(i32 %0, i32 9, i32 9)
     ret i8 addrspace(1)* %obj.relocated
   }
 
 In the above examples, the addrspace(1) marker on the pointers is the mechanism
 that the ``statepoint-example`` GC strategy uses to distinguish references from
 non references.  Address space 1 is not globally reserved for this purpose.
 
.. code-block:: llvm
   }
 
 
 This pass would produce the following IR:
 
 .. code-block:: llvm
 
   define void @test() gc "statepoint-example" {
-    %safepoint_token = call i32 (void ()*, i32, i32, ...)* @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0)
-    %safepoint_token1 = call i32 (void ()*, i32, i32, ...)* @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+    %safepoint_token = call i32 (void ()*, i32, i32, ...)* @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0, i32 0)
+    %safepoint_token1 = call i32 (void ()*, i32, i32, ...)* @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
     ret void
   }
 
 In this case, we've added an (unconditional) entry safepoint poll and converted the call into a ``gc.statepoint``.  Note that despite appearances, the entry poll is not necessarily redundant.  We'd have to know that ``foo`` and ``test`` were not mutually recursive for the poll to be redundant.  In practice, you'd probably want to your poll definition to contain a conditional branch of some form.
 
 
 At the moment, PlaceSafepoints can insert safepoint polls at method entry and 
 loop backedges locations.  Extending this to work with return polls would be 

include/llvm/CodeGen/ISDOpcodes.h

enum NodeType {
     // random addresses with additional mask operand that prevents memory
     // accesses to the masked-off lanes.
     MGATHER, MSCATTER,
 
     /// This corresponds to the llvm.lifetime.* intrinsics. The first operand
     /// is the chain and the second operand is the alloca pointer.
     LIFETIME_START, LIFETIME_END,
 
+    /// GC_TRANSITION_START/GC_TRANSITION_END - These operators mark the
+    /// beginning and end of GC transition  sequence, and carry arbitrary
+    /// information that target might need for lowering.  The first operand is
+    /// a chain, the rest are specified by the target and not touched by the DAG
+    /// optimizers. GC_TRANSITION_START..GC_TRANSITION_END pairs may not be
+    /// nested.
+    GC_TRANSITION_START,
+    GC_TRANSITION_END,
+
     /// BUILTIN_OP_END - This must be the last enum value in this list.
     /// The target-specific pre-isel opcode values start here.
     BUILTIN_OP_END
   };
 
   /// FIRST_TARGET_MEMORY_OPCODE - Target-specific pre-isel operations
   /// which do not reference a specific memory location should be less than
   /// this value. Those that do must not be less than this value, and can

include/llvm/IR/Statepoint.h

 #include "llvm/ADT/iterator_range.h"
 #include "llvm/IR/BasicBlock.h"
 #include "llvm/IR/CallSite.h"
 #include "llvm/IR/Instructions.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/Support/Compiler.h"
 
 namespace llvm {
+namespace statepoint {
+/// The statepoint intrinsic accepts a set of flags as its third argument.
+/// Valid values come out of this set.
+enum Flags {
+  None = 0,
+  GCTransition = 1, ///< Indicates that this statepoint is a transition from
+                    ///< GC-aware code to code that is not GC-aware.
+};
+}
 
 class GCRelocateOperands;
 class ImmutableStatepoint;
 
 bool isStatepoint(const ImmutableCallSite &CS);
 bool isStatepoint(const Value *inst);
 bool isStatepoint(const Value &inst);
 
protected:
   }
   explicit StatepointBase(CallSiteTy CS) : StatepointCS(CS) {
     assert(isStatepoint(CS));
   }
 
  public:
   typedef typename CallSiteTy::arg_iterator arg_iterator;
 
-  /// Return the underlying CallSite.
-  CallSiteTy getCallSite() {
-    return StatepointCS;
+  bool isGCTransition() {
+    return cast<ConstantInt>(StatepointCS.getArgument(2))->getZExtValue() &
+        statepoint::GCTransition;
   }
 
+  /// Return the underlying CallSite.
+  CallSiteTy getCallSite() { return StatepointCS; }
+
   /// Return the value actually being called or invoked.
   ValueTy *actualCallee() {
     return StatepointCS.getArgument(0);
   }
   /// Number of arguments to be passed to the actual callee.
   int numCallArgs() {
     return cast<ConstantInt>(StatepointCS.getArgument(1))->getZExtValue();
   }
-  /// Number of additional arguments excluding those intended
-  /// for garbage collection.
+  /// Number of GC transition args.
+  int numTotalGCTransitionArgs() {
+    return cast<ConstantInt>(*gc_transition_args_begin())->getZExtValue();
+  }
+  /// Number of vm state args.
   int numTotalVMSArgs() {
-    return cast<ConstantInt>(StatepointCS.getArgument(3 + numCallArgs()))->getZExtValue();
+    return cast<ConstantInt>(*vm_state_begin())->getZExtValue();
   }
 
   typename CallSiteTy::arg_iterator call_args_begin() {
-    // 3 = callTarget, #callArgs, flag
+    // 3 = callTarget, #callArgs, flags
     int Offset = 3;
     assert(Offset <= (int)StatepointCS.arg_size());
     return StatepointCS.arg_begin() + Offset;
   }
   typename CallSiteTy::arg_iterator call_args_end() {
     int Offset = 3 + numCallArgs();
     assert(Offset <= (int)StatepointCS.arg_size());
     return StatepointCS.arg_begin() + Offset;
   }
 
   /// range adapter for call arguments
   iterator_range<arg_iterator> call_args() {
     return iterator_range<arg_iterator>(call_args_begin(), call_args_end());
   }
 
+  typename CallSiteTy::arg_iterator gc_transition_args_begin() {
+    int Offset = call_args_end() - StatepointCS.arg_begin();
+    assert(Offset <= (int)StatepointCS.arg_size());
+    return StatepointCS.arg_begin() + Offset;
+  }
+  typename CallSiteTy::arg_iterator gc_transition_args_end() {
+    int Offset = (gc_transition_args_begin() + 1 + numTotalGCTransitionArgs()) -
+                 StatepointCS.arg_begin();
+    assert(Offset <= (int)StatepointCS.arg_size());
+    return StatepointCS.arg_begin() + Offset;
+  }
+
+  /// range adapter for GC transition arguments
+  iterator_range<arg_iterator> gc_transition_args() {
+    return iterator_range<arg_iterator>(gc_transition_args_begin(),
+                                        gc_transition_args_end());
+  }
+
   typename CallSiteTy::arg_iterator vm_state_begin() {
-    return call_args_end();
+    return gc_transition_args_end();
   }
   typename CallSiteTy::arg_iterator vm_state_end() {
-    int Offset = 3 + numCallArgs() + 1 + numTotalVMSArgs();
+    int Offset = (gc_transition_args_end() + 1 + numTotalVMSArgs()) -
+                 StatepointCS.arg_begin();
     assert(Offset <= (int)StatepointCS.arg_size());
     return StatepointCS.arg_begin() + Offset;
   }
 
   /// range adapter for vm state arguments
   iterator_range<arg_iterator> vm_state_args() {
     return iterator_range<arg_iterator>(vm_state_begin(), vm_state_end());
   }
#ifndef NDEBUG
   /// illegal values for parameters.
   void verify() {
     assert(numCallArgs() >= 0 &&
            "number of arguments to actually callee can't be negative");
 
     // The internal asserts in the iterator accessors do the rest.
     (void)call_args_begin();
     (void)call_args_end();
+    (void)gc_transition_args_begin();
+    (void)gc_transition_args_end();
     (void)vm_state_begin();
     (void)vm_state_end();
     (void)gc_args_begin();
     (void)gc_args_end();
   }
 #endif
 };
 

lib/CodeGen/SelectionDAG/SelectionDAGDumper.cpp

#endif
   case ISD::EXTRACT_ELEMENT:            return "extract_element";
   case ISD::BUILD_PAIR:                 return "build_pair";
   case ISD::STACKSAVE:                  return "stacksave";
   case ISD::STACKRESTORE:               return "stackrestore";
   case ISD::TRAP:                       return "trap";
   case ISD::DEBUGTRAP:                  return "debugtrap";
   case ISD::LIFETIME_START:             return "lifetime.start";
   case ISD::LIFETIME_END:               return "lifetime.end";
+  case ISD::GC_TRANSITION_START:        return "gc_transition.start";
+  case ISD::GC_TRANSITION_END:          return "gc_transition.end";
 
   // Bit manipulation
   case ISD::BSWAP:                      return "bswap";
   case ISD::CTPOP:                      return "ctpop";
   case ISD::CTTZ:                       return "cttz";
   case ISD::CTTZ_ZERO_UNDEF:            return "cttz_zero_undef";
   case ISD::CTLZ:                       return "ctlz";
   case ISD::CTLZ_ZERO_UNDEF:            return "ctlz_zero_undef";

lib/CodeGen/SelectionDAG/StatepointLowering.cpp

#endif
 
   // Lower statepoint vmstate and gcstate arguments
   SmallVector<SDValue, 10> LoweredArgs;
   lowerStatepointMetaArgs(LoweredArgs, ISP, *this);
 
   // Get call node, we will replace it later with statepoint
   SDNode *CallNode = lowerCallFromStatepoint(ISP, LandingPad, *this);
 
-  // Construct the actual STATEPOINT node with all the appropriate arguments
-  // and return values.
+  // Construct the actual GC_TRANSITION_START, STATEPOINT, and GC_TRANSITION_END
+  // nodes with all the appropriate arguments and return values.
 
   // TODO: Currently, all of these operands are being marked as read/write in
   // PrologEpilougeInserter.cpp, we should special case the VMState arguments
   // and flags to be read-only.
   SmallVector<SDValue, 40> Ops;
 
-  // Calculate and push starting position of vmstate arguments
   // Call Node: Chain, Target, {Args}, RegMask, [Glue]
+  SDValue Chain = CallNode->getOperand(0);
+
   SDValue Glue;
-  if (CallNode->getGluedNode()) {
+  bool CallHasIncomingGlue = CallNode->getGluedNode();
+  if (CallHasIncomingGlue) {
     // Glue is always last operand
     Glue = CallNode->getOperand(CallNode->getNumOperands() - 1);
   }
+
+  // Build the GC_TRANSITION_START node if necessary.
+  if (ISP.isGCTransition()) {
+    SmallVector<SDValue, 8> Ops;
+
+    // Add chain
+    Ops.push_back(Chain);
+
+    // Add GC transition arguments
+    for (auto I = ISP.gc_transition_args_begin() + 1,
+              E = ISP.gc_transition_args_end();
+         I != E; ++I) {
+      Ops.push_back(getValue(*I));
+      if ((*I)->getType()->isPointerTy())
+        Ops.push_back(DAG.getSrcValue(*I));
+    }
+
+    // Add glue if necessary
+    if (CallHasIncomingGlue)
+      Ops.push_back(Glue);
+
+    SDVTList NodeTys = DAG.getVTList(MVT::Other, MVT::Glue);
+
+    SDValue GCTransitionStart =
+        DAG.getNode(ISD::GC_TRANSITION_START, getCurSDLoc(), NodeTys, Ops);
+
+    Chain = GCTransitionStart.getValue(0);
+    Glue = GCTransitionStart.getValue(1);
+  }
+
+  // Calculate and push starting position of vmstate arguments
   // Get number of arguments incoming directly into call node
   unsigned NumCallRegArgs =
-      CallNode->getNumOperands() - (Glue.getNode() ? 4 : 3);
+      CallNode->getNumOperands() - (CallHasIncomingGlue ? 4 : 3);
   Ops.push_back(DAG.getTargetConstant(NumCallRegArgs, getCurSDLoc(), MVT::i32));
 
   // Add call target
   SDValue CallTarget = SDValue(CallNode->getOperand(1).getNode(), 0);
   Ops.push_back(CallTarget);
 
   // Add call arguments
   // Get position of register mask in the call
   SDNode::op_iterator RegMaskIt;
-  if (Glue.getNode())
+  if (CallHasIncomingGlue)
     RegMaskIt = CallNode->op_end() - 2;
   else
     RegMaskIt = CallNode->op_end() - 1;
   Ops.insert(Ops.end(), CallNode->op_begin() + 2, RegMaskIt);
 
   // Add a leading constant argument with the Flags and the calling convention
   // masked together
   CallingConv::ID CallConv = CS.getCallingConv();
   int Flags = cast<ConstantInt>(CS.getArgument(2))->getZExtValue();
-  assert(Flags == 0 && "not expected to be used");
+  assert(Flags == statepoint::None || Flags == statepoint::GCTransition &&
+         "unknown flag used");
   Ops.push_back(DAG.getTargetConstant(StackMaps::ConstantOp, getCurSDLoc(),
                                       MVT::i64));
   Ops.push_back(DAG.getTargetConstant(Flags | ((unsigned)CallConv << 1),
                                       getCurSDLoc(), MVT::i64));
 
   // Insert all vmstate and gcstate arguments
   Ops.insert(Ops.end(), LoweredArgs.begin(), LoweredArgs.end());
 
   // Add register mask from call node
   Ops.push_back(*RegMaskIt);
 
   // Add chain
   Ops.push_back(CallNode->getOperand(0));
 
   // Same for the glue, but we add it only if original call had it
   if (Glue.getNode())
     Ops.push_back(Glue);
 
   // Compute return values.  Provide a glue output since we consume one as
   // input.  This allows someone else to chain off us as needed.
   SDVTList NodeTys = DAG.getVTList(MVT::Other, MVT::Glue);
 
   SDNode *StatepointMCNode =
       DAG.getMachineNode(TargetOpcode::STATEPOINT, getCurSDLoc(), NodeTys, Ops);
 
+  SDNode *SinkNode = StatepointMCNode;
+
+  // Build the GC_TRANSITION_END node if necessary.
+  if (ISP.isGCTransition()) {
+    SmallVector<SDValue, 8> Ops;
+
+    // Add chain
+    Ops.push_back(SDValue(StatepointMCNode, 0));
+
+    // Add GC transition arguments
+    for (auto I = ISP.gc_transition_args_begin() + 1,
+              E = ISP.gc_transition_args_end();
+         I != E; ++I) {
+      Ops.push_back(getValue(*I));
+      if ((*I)->getType()->isPointerTy())
+        Ops.push_back(DAG.getSrcValue(*I));
+    }
+
+    // Add glue
+    Ops.push_back(SDValue(StatepointMCNode, 1));
+
+    SDVTList NodeTys = DAG.getVTList(MVT::Other, MVT::Glue);
+
+    SDValue GCTransitionStart =
+        DAG.getNode(ISD::GC_TRANSITION_END, getCurSDLoc(), NodeTys, Ops);
+
+    SinkNode = GCTransitionStart.getNode();
+  }
+
   // Replace original call
-  DAG.ReplaceAllUsesWith(CallNode, StatepointMCNode); // This may update Root
+  DAG.ReplaceAllUsesWith(CallNode, SinkNode); // This may update Root
   // Remove originall call node
   DAG.DeleteNode(CallNode);
 
   // DON'T set the root - under the assumption that it's already set past the
   // inserted node we created.
 
   // TODO: A better future implementation would be to emit a single variable
   // argument, variable return value STATEPOINT node here and then hookup the

lib/IR/IRBuilder.cpp

 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/IR/Function.h"
 #include "llvm/IR/GlobalVariable.h"
 #include "llvm/IR/IRBuilder.h"
 #include "llvm/IR/Intrinsics.h"
 #include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Statepoint.h"
 using namespace llvm;
 
 /// CreateGlobalString - Make a new global variable with an initializer that
 /// has array of i8 type filled in with the nul terminated string value
 /// specified.  If Name is specified, it is the name of the global variable
 /// created.
 GlobalVariable *IRBuilderBase::CreateGlobalString(StringRef Str,
                                                   const Twine &Name) {

  Type *ArgTypes[] = { FuncPtrType };
  Function *FnStatepoint =
    Intrinsic::getDeclaration(M, Intrinsic::experimental_gc_statepoint,
                              ArgTypes);
 
  std::vector<llvm::Value *> args;
  args.push_back(ActualCallee);
  args.push_back(getInt32(CallArgs.size()));
- args.push_back(getInt32(0 /*unused*/));
+ args.push_back(getInt32(statepoint::None));
  args.insert(args.end(), CallArgs.begin(), CallArgs.end());
+ args.push_back(getInt32(0 /* no transition args */));
  args.push_back(getInt32(DeoptArgs.size()));
  args.insert(args.end(), DeoptArgs.begin(), DeoptArgs.end());
  args.insert(args.end(), GCArgs.begin(), GCArgs.end());
 
  return createCallHelper(FnStatepoint, args, this, Name);
 }
 
 CallInst *IRBuilderBase::CreateGCStatepoint(Value *ActualCallee,

lib/IR/Verifier.cpp

if (TargetFuncType->isVarArg()) {
     Assert(TargetFuncType->getReturnType()->isVoidTy(),
            "gc.statepoint doesn't support wrapping non-void "
            "vararg functions yet",
            &CI);
   } else
     Assert(NumCallArgs == NumParams,
            "gc.statepoint mismatch in number of call args", &CI);
 
-  const Value *Unused = CS.getArgument(2);
-  Assert(isa<ConstantInt>(Unused) && cast<ConstantInt>(Unused)->isNullValue(),
-         "gc.statepoint parameter #3 must be zero", &CI);
+  const Value *FlagsV = CS.getArgument(2);
+  Assert(isa<ConstantInt>(FlagsV),
+         "gc.statepoint flags must be constant integer", &CI);
+  const int Flags = cast<ConstantInt>(FlagsV)->getZExtValue();
+  Assert(Flags == statepoint::None || Flags == statepoint::GCTransition,
+         "unknown flag used in gc.statepoint flags argument", &CI);
 
   // Verify that the types of the call parameter arguments match
   // the type of the wrapped callee.
   for (int i = 0; i < NumParams; i++) {
     Type *ParamType = TargetFuncType->getParamType(i);
     Type *ArgType = CS.getArgument(3+i)->getType();
     Assert(ArgType == ParamType,
            "gc.statepoint call argument does not match wrapped "
            "function type",
            &CI);
   }
   const int EndCallArgsInx = 2+NumCallArgs;
-  const Value *NumDeoptArgsV = CS.getArgument(EndCallArgsInx+1);
+
+  const Value *NumTransitionArgsV = CS.getArgument(EndCallArgsInx+1);
+  Assert(isa<ConstantInt>(NumTransitionArgsV),
+         "gc.statepoint number of transition arguments "
+         "must be constant integer",
+         &CI);
+  const int NumTransitionArgs =
+      cast<ConstantInt>(NumTransitionArgsV)->getZExtValue();
+  Assert(NumTransitionArgs >= 0,
+         "gc.statepoint number of transition arguments must be positive", &CI);
+  const int EndTransitionArgsInx = EndCallArgsInx + 1 + NumTransitionArgs;
+
+  const Value *NumDeoptArgsV = CS.getArgument(EndTransitionArgsInx+1);
   Assert(isa<ConstantInt>(NumDeoptArgsV),
          "gc.statepoint number of deoptimization arguments "
          "must be constant integer",
          &CI);
   const int NumDeoptArgs = cast<ConstantInt>(NumDeoptArgsV)->getZExtValue();
   Assert(NumDeoptArgs >= 0, "gc.statepoint number of deoptimization arguments "
                             "must be positive",
          &CI);
 
-  Assert(4 + NumCallArgs + NumDeoptArgs <= (int)CS.arg_size(),
+  const int ExpectedNumArgs =
+      5 + NumCallArgs + NumTransitionArgs + NumDeoptArgs;
+  Assert(ExpectedNumArgs <= (int)CS.arg_size(),
          "gc.statepoint too few arguments according to length fields", &CI);
 
   // Check that the only uses of this gc.statepoint are gc.result or 
   // gc.relocate calls which are tied to this statepoint and thus part
   // of the same statepoint sequence
   for (const User *U : CI.users()) {
     const CallInst *Call = dyn_cast<const CallInst>(U);
     Assert(Call, "illegal use of statepoint token", &CI, U);
Assert(StatepointCS.arg_size() > 0,
            "gc.statepoint: insufficient arguments");
     Assert(isa<ConstantInt>(StatepointCS.getArgument(1)),
            "gc.statement: number of call arguments must be constant integer");
     const unsigned NumCallArgs =
       cast<ConstantInt>(StatepointCS.getArgument(1))->getZExtValue();
     Assert(StatepointCS.arg_size() > NumCallArgs+3,
            "gc.statepoint: mismatch in number of call arguments");
     Assert(isa<ConstantInt>(StatepointCS.getArgument(NumCallArgs+3)),
+           "gc.statepoint: number of transition arguments must be "
+           "a constant integer");
+    const int NumTransitionArgs =
+      cast<ConstantInt>(StatepointCS.getArgument(NumCallArgs + 3))->getZExtValue();
+    const int DeoptArgsStart = 2 + NumCallArgs + 1 + NumTransitionArgs + 1;
+    Assert(isa<ConstantInt>(StatepointCS.getArgument(DeoptArgsStart)),
            "gc.statepoint: number of deoptimization arguments must be "
            "a constant integer");
     const int NumDeoptArgs =
-      cast<ConstantInt>(StatepointCS.getArgument(NumCallArgs + 3))->getZExtValue();
-    const int GCParamArgsStart = NumCallArgs + NumDeoptArgs + 4;
+      cast<ConstantInt>(StatepointCS.getArgument(DeoptArgsStart))->getZExtValue();
+    const int GCParamArgsStart = DeoptArgsStart + 1 + NumDeoptArgs;
     const int GCParamArgsEnd = StatepointCS.arg_size();
     Assert(GCParamArgsStart <= BaseIndex && BaseIndex < GCParamArgsEnd,
            "gc.relocate: statepoint base index doesn't fall within the "
            "'gc parameters' section of the statepoint call",
            &CI);
     Assert(GCParamArgsStart <= DerivedIndex && DerivedIndex < GCParamArgsEnd,
            "gc.relocate: statepoint derived index doesn't fall within the "
            "'gc parameters' section of the statepoint call",

lib/Target/X86/X86ISelLowering.h

private:
     SDValue LowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFRAME_TO_ARGS_OFFSET(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerEH_RETURN(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerEH_SJLJ_SETJMP(SDValue Op, SelectionDAG &DAG) const;
     SDValue lowerEH_SJLJ_LONGJMP(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerINIT_TRAMPOLINE(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerFLT_ROUNDS_(SDValue Op, SelectionDAG &DAG) const;
     SDValue LowerWin64_i128OP(SDValue Op, SelectionDAG &DAG) const;
+    SDValue LowerGC_TRANSITION_START(SDValue Op, SelectionDAG &DAG) const;
+    SDValue LowerGC_TRANSITION_END(SDValue Op, SelectionDAG &DAG) const;
 
     SDValue
       LowerFormalArguments(SDValue Chain,
                            CallingConv::ID CallConv, bool isVarArg,
                            const SmallVectorImpl<ISD::InputArg> &Ins,
                            SDLoc dl, SelectionDAG &DAG,
                            SmallVectorImpl<SDValue> &InVals) const override;
     SDValue LowerCall(CallLoweringInfo &CLI,

lib/Target/X86/X86ISelLowering.cpp

if (Subtarget->is64Bit() && !Subtarget->isTargetWin64()) {
     setOperationAction(ISD::VACOPY          , MVT::Other, Expand);
   }
 
   setOperationAction(ISD::STACKSAVE,          MVT::Other, Expand);
   setOperationAction(ISD::STACKRESTORE,       MVT::Other, Expand);
 
   setOperationAction(ISD::DYNAMIC_STACKALLOC, getPointerTy(), Custom);
 
+  // GC_TRANSITION_START and GC_TRANSITION_END need custom lowering.
+  setOperationAction(ISD::GC_TRANSITION_START, MVT::Other, Custom);
+  setOperationAction(ISD::GC_TRANSITION_END, MVT::Other, Custom);
+
   if (!TM.Options.UseSoftFloat && X86ScalarSSEf64) {
     // f32 and f64 use SSE.
     // Set up the FP register classes.
     addRegisterClass(MVT::f32, &X86::FR32RegClass);
     addRegisterClass(MVT::f64, &X86::FR64RegClass);
 
     // Use ANDPD to simulate FABS.
     setOperationAction(ISD::FABS , MVT::f64, Custom);
if (!Subtarget->hasVLX() && !VT.is512BitVector() &&
     Index = DAG.getNode(ISD::SIGN_EXTEND, dl, MVT::v8i64, Index);
     SDValue Ops[] = { N->getOperand(0), N->getOperand(1),  N->getOperand(2),
                       N->getOperand(3), Index };
     DAG.UpdateNodeOperands(N, Ops);
   }
   return Op;
 }
 
+SDValue X86TargetLowering::LowerGC_TRANSITION_START(SDValue Op,
+                                                    SelectionDAG &DAG) const {
+  // Replace with a NOOP for now.
+  SmallVector<SDValue, 2> Ops;
+
+  Ops.push_back(Op.getOperand(0));
+  if (Op->getGluedNode())
+    Ops.push_back(Op->getOperand(Op->getNumOperands() - 1));
+
+  SDLoc OpDL(Op);
+  SDVTList VTs = DAG.getVTList(MVT::Other, MVT::Glue);
+  SDValue NOOP(DAG.getMachineNode(X86::NOOP, SDLoc(Op), VTs, Ops), 0);
+
+  return NOOP;
+}
+
+SDValue X86TargetLowering::LowerGC_TRANSITION_END(SDValue Op,
+                                                  SelectionDAG &DAG) const {
+  // Replace with a NOOP for now.
+  SmallVector<SDValue, 2> Ops;
+
+  Ops.push_back(Op.getOperand(0));
+  if (Op->getGluedNode())
+    Ops.push_back(Op->getOperand(Op->getNumOperands() - 1));
+
+  SDLoc OpDL(Op);
+  SDVTList VTs = DAG.getVTList(MVT::Other, MVT::Glue);
+  SDValue NOOP(DAG.getMachineNode(X86::NOOP, SDLoc(Op), VTs, Ops), 0);
+
+  return NOOP;
+}
+
 /// LowerOperation - Provide custom lowering hooks for some operations.
 ///
 SDValue X86TargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
   switch (Op.getOpcode()) {
   default: llvm_unreachable("Should not custom lower this!");
   case ISD::ATOMIC_FENCE:       return LowerATOMIC_FENCE(Op, Subtarget, DAG);
   case ISD::ATOMIC_CMP_SWAP_WITH_SUCCESS:
     return LowerCMP_SWAP(Op, Subtarget, DAG);
SDValue X86TargetLowering::LowerOperation(SDValue Op, SelectionDAG &DAG) const {
   case ISD::ADDE:
   case ISD::SUBC:
   case ISD::SUBE:               return LowerADDC_ADDE_SUBC_SUBE(Op, DAG);
   case ISD::ADD:                return LowerADD(Op, DAG);
   case ISD::SUB:                return LowerSUB(Op, DAG);
   case ISD::FSINCOS:            return LowerFSINCOS(Op, Subtarget, DAG);
   case ISD::MGATHER:            return LowerMGATHER(Op, Subtarget, DAG);
   case ISD::MSCATTER:           return LowerMSCATTER(Op, Subtarget, DAG);
+  case ISD::GC_TRANSITION_START:
+                                return LowerGC_TRANSITION_START(Op, DAG);
+  case ISD::GC_TRANSITION_END:  return LowerGC_TRANSITION_END(Op, DAG);
   }
 }
 
 /// ReplaceNodeResults - Replace a node with an illegal result type
 /// with a new node built out of custom code.
 void X86TargetLowering::ReplaceNodeResults(SDNode *N,
                                            SmallVectorImpl<SDValue>&Results,
                                            SelectionDAG &DAG) const {

lib/Transforms/Scalar/PlaceSafepoints.cpp

if (CS.isCall()) {
     std::vector<llvm::Value *> args;
     // target, #call args, unused, ... call parameters, #deopt args, ... deopt
     // parameters, ... gc parameters
     Value *Target = CS.getCalledValue();
     args.push_back(Target);
     int callArgSize = CS.arg_size();
     // #call args
     args.push_back(Builder.getInt32(callArgSize));
-    // unused
+    // flags
     args.push_back(Builder.getInt32(0));
     // call parameters
     args.insert(args.end(), CS.arg_begin(), CS.arg_end());
+    // #transition args: 0
+    args.push_back(Builder.getInt32(0));
     // #deopt args: 0
     args.push_back(Builder.getInt32(0));
 
     InvokeInst *toReplace = cast<InvokeInst>(CS.getInstruction());
 
     // Insert the new invoke into the old block.  We'll remove the old one in a
     // moment at which point this will become the new terminator for the
     // original block.

test/Analysis/ValueTracking/memory-dereferenceable.ll

 ; CHECK: %relocate
 ; CHECK-NOT: %nparam
 entry:
     %globalptr = getelementptr inbounds [6 x i8], [6 x i8]* @globalstr, i32 0, i32 0
     %load1 = load i8, i8* %globalptr
     %alloca = alloca i1
     %load2 = load i1, i1* %alloca
     %load3 = load i32, i32 addrspace(1)* %dparam
-    %tok = tail call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 addrspace(1)* %dparam)
-    %relocate = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %tok, i32 4, i32 4)
+    %tok = tail call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, i32 addrspace(1)* %dparam)
+    %relocate = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %tok, i32 5, i32 5)
     %load4 = load i32, i32 addrspace(1)* %relocate
     %nparam = getelementptr i32, i32 addrspace(1)* %dparam, i32 5
     %load5 = load i32, i32 addrspace(1)* %nparam
     ret void
 }
 
 declare i32 @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()*, i32, i32, ...)
 declare i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32, i32, i32)

test/CodeGen/Generic/overloaded-intrinsic-name.ll

 ; RUN: opt -verify -S < %s
 
 ; Tests the name mangling performed by the codepath following
 ; getMangledTypeStr(). Only tests that code with the various manglings
 ; run fine: doesn't actually test the mangling with the type of the
 ; arguments. Meant to serve as an example-document on how the user
 ; should do name manglings.
 
 ; Exercise the most general case, llvm_anyptr_type, using gc.relocate
 ; and gc.statepoint. Note that it has nothing to do with gc.*
 ; functions specifically: any function that accepts llvm_anyptr_type
 ; will serve the purpose.
 
 ; function and integer
 define i32* @test_iAny(i32* %v) gc "statepoint-example" {
-       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32* %v)
-       %v-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 4)
+       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, i32* %v)
+       %v-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok,  i32 5, i32 5)
        ret i32* %v-new
 }
 
 ; float
 define float* @test_fAny(float* %v) gc "statepoint-example" {
-       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, float* %v)
-       %v-new = call float* @llvm.experimental.gc.relocate.p0f32(i32 %tok, i32 4, i32 4)
+       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, float* %v)
+       %v-new = call float* @llvm.experimental.gc.relocate.p0f32(i32 %tok,  i32 5, i32 5)
        ret float* %v-new
 }
 
 ; array of integers
 define [3 x i32]* @test_aAny([3 x i32]* %v) gc "statepoint-example" {
-       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, [3 x i32]* %v)
-       %v-new = call [3 x i32]* @llvm.experimental.gc.relocate.p0a3i32(i32 %tok, i32 4, i32 4)
+       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, [3 x i32]* %v)
+       %v-new = call [3 x i32]* @llvm.experimental.gc.relocate.p0a3i32(i32 %tok,  i32 5, i32 5)
        ret [3 x i32]* %v-new
 }
 
 ; vector of integers
 define <3 x i32>* @test_vAny(<3 x i32>* %v) gc "statepoint-example" {
-       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, <3 x i32>* %v)
-       %v-new = call <3 x i32>* @llvm.experimental.gc.relocate.p0v3i32(i32 %tok, i32 4, i32 4)
+       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, <3 x i32>* %v)
+       %v-new = call <3 x i32>* @llvm.experimental.gc.relocate.p0v3i32(i32 %tok,  i32 5, i32 5)
        ret <3 x i32>* %v-new
 }
 
 %struct.test = type { i32, i1 }
 
 ; struct
 define %struct.test* @test_struct(%struct.test* %v) gc "statepoint-example" {
-       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, %struct.test* %v)
-       %v-new = call %struct.test* @llvm.experimental.gc.relocate.p0struct.test(i32 %tok, i32 4, i32 4)
+       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, %struct.test* %v)
+       %v-new = call %struct.test* @llvm.experimental.gc.relocate.p0struct.test(i32 %tok,  i32 5, i32 5)
        ret %struct.test* %v-new
 }
 
 declare zeroext i1 @return_i1()
 declare i32 @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()*, i32, i32, ...)
 declare i32* @llvm.experimental.gc.relocate.p0i32(i32, i32, i32)
 declare float* @llvm.experimental.gc.relocate.p0f32(i32, i32, i32)
 declare [3 x i32]* @llvm.experimental.gc.relocate.p0a3i32(i32, i32, i32)
 declare <3 x i32>* @llvm.experimental.gc.relocate.p0v3i32(i32, i32, i32)
 declare %struct.test* @llvm.experimental.gc.relocate.p0struct.test(i32, i32, i32)

test/CodeGen/X86/statepoint-allocas.ll

 ; CHECK: movq   %rdi, (%rsp)
 ; CHECK: callq return_i1
 ; CHECK: movq   (%rsp), %rax
 ; CHECK: popq   %rdx
 ; CHECK: retq
 entry:
   %alloca = alloca i32 addrspace(1)*, align 8
   store i32 addrspace(1)* %ptr, i32 addrspace(1)** %alloca
-  call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 addrspace(1)** %alloca)
+  call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, i32 addrspace(1)** %alloca)
   %rel = load i32 addrspace(1)*, i32 addrspace(1)** %alloca
   ret i32 addrspace(1)* %rel
 }
 
 ; Can we handle an alloca as a deopt value?  
 define i32 addrspace(1)* @test2(i32 addrspace(1)* %ptr) gc "statepoint-example" {
 ; CHECK-LABEL: test2
 ; CHECK: pushq  %rax
 ; CHECK: movq   %rdi, (%rsp)
 ; CHECK: callq return_i1
 ; CHECK: xorl   %eax, %eax
 ; CHECK: popq   %rdx
 ; CHECK: retq
 entry:
   %alloca = alloca i32 addrspace(1)*, align 8
   store i32 addrspace(1)* %ptr, i32 addrspace(1)** %alloca
-  call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 1, i32 addrspace(1)** %alloca)
+  call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 1, i32 addrspace(1)** %alloca)
   ret i32 addrspace(1)* null
 }
 
 declare i32 @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()*, i32, i32, ...)
 
 
 ; CHECK-LABEL: .section .llvm_stackmaps
 ; CHECK-NEXT:  __LLVM_StackMaps:

test/CodeGen/X86/statepoint-call-lowering.ll

 ; CHECK-LABEL: test_i1_return
 ; This is just checking that a i1 gets lowered normally when there's no extra
 ; state arguments to the statepoint
 ; CHECK: pushq %rax
 ; CHECK: callq return_i1
 ; CHECK: popq %rdx
 ; CHECK: retq
 entry:
-  %safepoint_token = tail call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0)
+  %safepoint_token = tail call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0)
   %call1 = call zeroext i1 @llvm.experimental.gc.result.i1(i32 %safepoint_token)
   ret i1 %call1
 }
 
 define i32 @test_i32_return() gc "statepoint-example" {
 ; CHECK-LABEL: test_i32_return
 ; CHECK: pushq %rax
 ; CHECK: callq return_i32
 ; CHECK: popq %rdx
 ; CHECK: retq
 entry:
-  %safepoint_token = tail call i32 (i32 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i32f(i32 ()* @return_i32, i32 0, i32 0, i32 0)
+  %safepoint_token = tail call i32 (i32 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i32f(i32 ()* @return_i32, i32 0, i32 0, i32 0, i32 0)
   %call1 = call zeroext i32 @llvm.experimental.gc.result.i32(i32 %safepoint_token)
   ret i32 %call1
 }
 
 define i32* @test_i32ptr_return() gc "statepoint-example" {
 ; CHECK-LABEL: test_i32ptr_return
 ; CHECK: pushq %rax
 ; CHECK: callq return_i32ptr
 ; CHECK: popq %rdx
 ; CHECK: retq
 entry:
-  %safepoint_token = tail call i32 (i32* ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p0i32f(i32* ()* @return_i32ptr, i32 0, i32 0, i32 0)
+  %safepoint_token = tail call i32 (i32* ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p0i32f(i32* ()* @return_i32ptr, i32 0, i32 0, i32 0, i32 0)
   %call1 = call i32* @llvm.experimental.gc.result.p0i32(i32 %safepoint_token)
   ret i32* %call1
 }
 
 define float @test_float_return() gc "statepoint-example" {
 ; CHECK-LABEL: test_float_return
 ; CHECK: pushq %rax
 ; CHECK: callq return_float
 ; CHECK: popq %rax
 ; CHECK: retq
 entry:
-  %safepoint_token = tail call i32 (float ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_f32f(float ()* @return_float, i32 0, i32 0, i32 0)
+  %safepoint_token = tail call i32 (float ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_f32f(float ()* @return_float, i32 0, i32 0, i32 0, i32 0)
   %call1 = call float @llvm.experimental.gc.result.f32(i32 %safepoint_token)
   ret float %call1
 }
 
 define i1 @test_relocate(i32 addrspace(1)* %a) gc "statepoint-example" {
 ; CHECK-LABEL: test_relocate
 ; Check that an ununsed relocate has no code-generation impact
 ; CHECK: pushq %rax
 ; CHECK: callq return_i1
 ; CHECK-NEXT: .Ltmp9:
 ; CHECK-NEXT: popq %rdx
 ; CHECK-NEXT: retq
 entry:
-  %safepoint_token = tail call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 addrspace(1)* %a)
-  %call1 = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 4, i32 4)
+  %safepoint_token = tail call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, i32 addrspace(1)* %a)
+  %call1 = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token,  i32 5, i32 5)
   %call2 = call zeroext i1 @llvm.experimental.gc.result.i1(i32 %safepoint_token)
   ret i1 %call2
 }
 
 define void @test_void_vararg() gc "statepoint-example" {
 ; CHECK-LABEL: test_void_vararg
 ; Check a statepoint wrapping a *void* returning vararg function works
 ; CHECK: callq varargf
 entry:
-  %safepoint_token = tail call i32 (void (i32, ...)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidi32varargf(void (i32, ...)* @varargf, i32 2, i32 0, i32 42, i32 43, i32 0)
+  %safepoint_token = tail call i32 (void (i32, ...)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidi32varargf(void (i32, ...)* @varargf, i32 2, i32 0, i32 42, i32 43, i32 0, i32 0)
   ;; if we try to use the result from a statepoint wrapping a
   ;; non-void-returning varargf, we will experience a crash.
   ret void
 }
 
 declare i32 @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()*, i32, i32, ...)
 declare i1 @llvm.experimental.gc.result.i1(i32)
 

test/CodeGen/X86/statepoint-forward.ll

 ;; Forwarding the value of a pointer load is invalid since it may have
 ;; changed at the safepoint.  Forwarding a non-gc pointer value would 
 ;; be valid, but is not currently implemented.
 define i1 @test_load_forward(i32 addrspace(1)* addrspace(1)* %p) gc "statepoint-example" {
 entry:
   %before = load i32 addrspace(1)*, i32 addrspace(1)* addrspace(1)* %p
   %cmp1 = call i1 @f(i32 addrspace(1)* %before)
   call void @llvm.assume(i1 %cmp1)
-  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 addrspace(1)* addrspace(1)* %p)
-  %pnew = call i32 addrspace(1)* addrspace(1)* @llvm.experimental.gc.relocate.p1p1i32(i32 %safepoint_token, i32 4, i32 4)
+  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 0, i32 addrspace(1)* addrspace(1)* %p)
+  %pnew = call i32 addrspace(1)* addrspace(1)* @llvm.experimental.gc.relocate.p1p1i32(i32 %safepoint_token,  i32 5, i32 5)
   %after = load i32 addrspace(1)*, i32 addrspace(1)* addrspace(1)* %pnew
   %cmp2 = call i1 @f(i32 addrspace(1)* %after)
   ret i1 %cmp2
 
 ; CHECK-OPT-LABEL: test_load_forward
 ; CHECK-OPT: ret i1 %cmp2
 ; CHECK-LLC-LABEL: test_load_forward
 ; CHECK-LLC: callq f
 }
 
 ;; Same as above, but forwarding from a store
 define i1 @test_store_forward(i32 addrspace(1)* addrspace(1)* %p,
                               i32 addrspace(1)* %v) gc "statepoint-example" {
 entry:
   %cmp1 = call i1 @f(i32 addrspace(1)* %v)
   call void @llvm.assume(i1 %cmp1)
   store i32 addrspace(1)* %v, i32 addrspace(1)* addrspace(1)* %p
-  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 addrspace(1)* addrspace(1)* %p)
-  %pnew = call i32 addrspace(1)* addrspace(1)* @llvm.experimental.gc.relocate.p1p1i32(i32 %safepoint_token, i32 4, i32 4)
+  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 0, i32 addrspace(1)* addrspace(1)* %p)
+  %pnew = call i32 addrspace(1)* addrspace(1)* @llvm.experimental.gc.relocate.p1p1i32(i32 %safepoint_token,  i32 5, i32 5)
   %after = load i32 addrspace(1)*, i32 addrspace(1)* addrspace(1)* %pnew
   %cmp2 = call i1 @f(i32 addrspace(1)* %after)
   ret i1 %cmp2
 
 ; CHECK-OPT-LABEL: test_store_forward
 ; CHECK-OPT: ret i1 %cmp2
 ; CHECK-LLC-LABEL: test_store_forward
 ; CHECK-LLC: callq f

 ; that is not itself GC managed.  The GC may have an external mechanism
 ; to know about and update that value at a safepoint.  Note that the 
 ; statepoint does not provide the collector with this root.
 define i1 @test_load_forward_nongc_heap(i32 addrspace(1)** %p) gc "statepoint-example" {
 entry:
   %before = load i32 addrspace(1)*, i32 addrspace(1)** %p
   %cmp1 = call i1 @f(i32 addrspace(1)* %before)
   call void @llvm.assume(i1 %cmp1)
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 0)
   %after = load i32 addrspace(1)*, i32 addrspace(1)** %p
   %cmp2 = call i1 @f(i32 addrspace(1)* %after)
   ret i1 %cmp2
 
 ; CHECK-OPT-LABEL: test_load_forward_nongc_heap
 ; CHECK-OPT: ret i1 %cmp2
 ; CHECK-LLC-LABEL: test_load_forward_nongc_heap
 ; CHECK-LLC: callq f
 }
 
 ;; Same as above, but forwarding from a store
 define i1 @test_store_forward_nongc_heap(i32 addrspace(1)** %p,
                                          i32 addrspace(1)* %v) gc "statepoint-example" {
 entry:
   %cmp1 = call i1 @f(i32 addrspace(1)* %v)
   call void @llvm.assume(i1 %cmp1)
   store i32 addrspace(1)* %v, i32 addrspace(1)** %p
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 0)
   %after = load i32 addrspace(1)*, i32 addrspace(1)** %p
   %cmp2 = call i1 @f(i32 addrspace(1)* %after)
   ret i1 %cmp2
 
 ; CHECK-OPT-LABEL: test_store_forward_nongc_heap
 ; CHECK-OPT: ret i1 %cmp2
 ; CHECK-LLC-LABEL: test_store_forward_nongc_heap
 ; CHECK-LLC: callq f
 }
 
 declare void @llvm.assume(i1)
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)
 declare i32 addrspace(1)* addrspace(1)* @llvm.experimental.gc.relocate.p1p1i32(i32, i32, i32) #3
 

test/CodeGen/X86/statepoint-invoke.ll

 ; RUN: llc < %s 2>&1 | FileCheck %s
 
 target triple = "x86_64-pc-linux-gnu"
 
 declare i64 addrspace(1)* @"some_other_call"(i64 addrspace(1)*)
 
 declare i32 @"personality_function"()
 
 define i64 addrspace(1)* @test_result(i64 addrspace(1)* %obj, 
                                       i64 addrspace(1)* %obj1)
   gc "statepoint-example" {
 entry:
   ; CHECK: .Ltmp{{[0-9]+}}:
   ; CHECK: callq some_other_call
   ; CHECK: .Ltmp{{[0-9]+}}:
-  %0 = invoke i32 (i64 addrspace(1)* (i64 addrspace(1)*)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p1i64p1i64f(i64 addrspace(1)* (i64 addrspace(1)*)* @some_other_call, i32 1, i32 0, i64 addrspace(1)* %obj, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i64 addrspace(1)* %obj, i64 addrspace(1)* %obj1)
+  %0 = invoke i32 (i64 addrspace(1)* (i64 addrspace(1)*)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p1i64p1i64f(i64 addrspace(1)* (i64 addrspace(1)*)* @some_other_call, i32 1, i32 0, i64 addrspace(1)* %obj, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i64 addrspace(1)* %obj, i64 addrspace(1)* %obj1)
           to label %normal_return unwind label %exceptional_return
 
 normal_return:
   ; CHECK: popq 
   ; CHECK: retq
   %ret_val = call i64 addrspace(1)* @llvm.experimental.gc.result.p1i64(i32 %0)
   ret i64 addrspace(1)* %ret_val
 

test/CodeGen/X86/statepoint-stack-usage.ll

 ; RUN: llc < %s | FileCheck %s
 
 target datalayout = "e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-pc-linux-gnu"
 
 ; This test is checking to make sure that we reuse the same stack slots
 ; for GC values spilled over two different call sites.  Since the order
 ; of GC arguments differ, niave lowering code would insert loads and 
 ; stores to rearrange items on the stack.  We need to make sure (for
 ; performance) that this doesn't happen.
 define i32 @back_to_back_calls(i32 addrspace(1)* %a, i32 addrspace(1)* %b, i32 addrspace(1)* %c) #1 gc "statepoint-example" {
 ; CHECK-LABEL: back_to_back_calls
 ; The exact stores don't matter, but there need to be three stack slots created
 ; CHECK: movq %rdx, 16(%rsp)
 ; CHECK: movq %rdi, 8(%rsp)
 ; CHECK: movq %rsi, (%rsp)
-  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i32 addrspace(1)* %a, i32 addrspace(1)* %b, i32 addrspace(1)* %c)
-  %a1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 9, i32 9)
-  %b1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 9, i32 10)
-  %c1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 9, i32 11)
+  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i32 addrspace(1)* %a, i32 addrspace(1)* %b, i32 addrspace(1)* %c)
+  %a1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 10, i32 10)
+  %b1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 10, i32 11)
+  %c1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 10, i32 12)
 ; CHECK: callq
 ; This is the key check.  There should NOT be any memory moves here
 ; CHECK-NOT: movq
-  %safepoint_token2 = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i32 addrspace(1)* %c1, i32 addrspace(1)* %b1, i32 addrspace(1)* %a1)
-  %a2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token2, i32 9, i32 11)
-  %b2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token2, i32 9, i32 10)
-  %c2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token2, i32 9, i32 9)
+  %safepoint_token2 = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i32 addrspace(1)* %c1, i32 addrspace(1)* %b1, i32 addrspace(1)* %a1)
+  %a2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token2, i32 10, i32 12)
+  %b2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token2, i32 10, i32 11)
+  %c2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token2, i32 10, i32 10)
 ; CHECK: callq
   ret i32 1
 }
 
 ; This test simply checks that minor changes in vm state don't prevent slots
 ; being reused for gc values.  
 define i32 @reserve_first(i32 addrspace(1)* %a, i32 addrspace(1)* %b, i32 addrspace(1)* %c) #1 gc "statepoint-example" {
 ; CHECK-LABEL: reserve_first
 ; The exact stores don't matter, but there need to be three stack slots created
 ; CHECK: movq %rdx, 16(%rsp)
 ; CHECK: movq %rdi, 8(%rsp)
 ; CHECK: movq %rsi, (%rsp)
-  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i32 addrspace(1)* %a, i32 addrspace(1)* %b, i32 addrspace(1)* %c)
-  %a1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 9, i32 9)
-  %b1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 9, i32 10)
-  %c1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 9, i32 11)
+  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i32 addrspace(1)* %a, i32 addrspace(1)* %b, i32 addrspace(1)* %c)
+  %a1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 10, i32 10)
+  %b1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 10, i32 11)
+  %c1 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 10, i32 12)
 ; CHECK: callq
 ; This is the key check.  There should NOT be any memory moves here
 ; CHECK-NOT: movq
-  %safepoint_token2 = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 5, i32 addrspace(1)* %a1, i32 0, i32 addrspace(1)* %c1, i32 0, i32 0, i32 addrspace(1)* %c1, i32 addrspace(1)* %b1, i32 addrspace(1)* %a1)
-  %a2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token2, i32 9, i32 11)
-  %b2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token2, i32 9, i32 10)
-  %c2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token2, i32 9, i32 9)
+  %safepoint_token2 = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 addrspace(1)* %a1, i32 0, i32 addrspace(1)* %c1, i32 0, i32 0, i32 addrspace(1)* %c1, i32 addrspace(1)* %b1, i32 addrspace(1)* %a1)
+  %a2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token2, i32 10, i32 12)
+  %b2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token2, i32 10, i32 11)
+  %c2 = tail call coldcc i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token2, i32 10, i32 10)
 ; CHECK: callq
   ret i32 1
 }
 
 ; Function Attrs: nounwind
 declare i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32, i32, i32) #3
 
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)
 
 attributes #1 = { uwtable }

test/CodeGen/X86/statepoint-stackmap-format.ll

 ; CHECK: movq $0, 8(%rsp)
 ; CHECK: movq %rdi, (%rsp)
 ; CHECK: callq return_i1
 ; CHECK: addq $24, %rsp
 ; CHECK: retq
 entry:
   %metadata1 = alloca i32 addrspace(1)*, i32 2, align 8
   store i32 addrspace(1)* null, i32 addrspace(1)** %metadata1
-  %safepoint_token = tail call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 2, i32 addrspace(1)* %ptr, i32 addrspace(1)* null, i32 addrspace(1)* %ptr, i32 addrspace(1)* null)
+  %safepoint_token = tail call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 2, i32 addrspace(1)* %ptr, i32 addrspace(1)* null, i32 addrspace(1)* %ptr, i32 addrspace(1)* null)
   %call1 = call zeroext i1 @llvm.experimental.gc.result.i1(i32 %safepoint_token)
-  %a = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 6, i32 6)
-  %b = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 7, i32 7)
+  %a = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 7, i32 7)
+  %b = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 8, i32 8)
 ; 
   ret i1 %call1
 }
 
 declare i32 @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()*, i32, i32, ...)
 declare i1 @llvm.experimental.gc.result.i1(i32)
 declare i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32, i32, i32) #3
 

test/Transforms/CodeGenPrepare/statepoint-relocate.ll

 ; RUN: opt -codegenprepare -S < %s | FileCheck %s
 
 target datalayout = "e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-pc-linux-gnu"
 
 declare zeroext i1 @return_i1()
 
 define i32 @test_sor_basic(i32* %base) gc "statepoint-example" {
 ; CHECK: getelementptr i32, i32* %base, i32 15
 ; CHECK: getelementptr i32, i32* %base-new, i32 15
 entry:
        %ptr = getelementptr i32, i32* %base, i32 15
-       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32* %base, i32* %ptr)
-       %base-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 4)
-       %ptr-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 5)
+       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, i32* %base, i32* %ptr)
+       %base-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok,  i32 5, i32 5)
+       %ptr-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 5, i32 6)
        %ret = load i32, i32* %ptr-new
        ret i32 %ret
 }
 
 define i32 @test_sor_two_derived(i32* %base) gc "statepoint-example" {
 ; CHECK: getelementptr i32, i32* %base, i32 15
 ; CHECK: getelementptr i32, i32* %base, i32 12
-; CHECK: getelementptr i32, i32* %base-new, i32 15
 ; CHECK: getelementptr i32, i32* %base-new, i32 12
+; CHECK: getelementptr i32, i32* %base-new, i32 15
 entry:
        %ptr = getelementptr i32, i32* %base, i32 15
        %ptr2 = getelementptr i32, i32* %base, i32 12
-       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32* %base, i32* %ptr, i32* %ptr2)
-       %base-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 4)
-       %ptr-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 5)
-       %ptr2-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 6)
+       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, i32* %base, i32* %ptr, i32* %ptr2)
+       %base-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok,  i32 5, i32 5)
+       %ptr-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 5, i32 6)
+       %ptr2-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 5, i32 7)
        %ret = load i32, i32* %ptr-new
        ret i32 %ret
 }
 
 define i32 @test_sor_ooo(i32* %base) gc "statepoint-example" {
 ; CHECK: getelementptr i32, i32* %base, i32 15
 ; CHECK: getelementptr i32, i32* %base-new, i32 15
 entry:
        %ptr = getelementptr i32, i32* %base, i32 15
-       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32* %base, i32* %ptr)
-       %ptr-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 5)
-       %base-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 4)
+       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, i32* %base, i32* %ptr)
+       %ptr-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 5, i32 6)
+       %base-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok,  i32 5, i32 5)
        %ret = load i32, i32* %ptr-new
        ret i32 %ret
 }
 
 define i32 @test_sor_gep_smallint([3 x i32]* %base) gc "statepoint-example" {
 ; CHECK: getelementptr [3 x i32], [3 x i32]* %base, i32 0, i32 2
 ; CHECK: getelementptr [3 x i32], [3 x i32]* %base-new, i32 0, i32 2
 entry:
        %ptr = getelementptr [3 x i32], [3 x i32]* %base, i32 0, i32 2
-       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, [3 x i32]* %base, i32* %ptr)
-       %base-new = call [3 x i32]* @llvm.experimental.gc.relocate.p0a3i32(i32 %tok, i32 4, i32 4)
-       %ptr-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 5)
+       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, [3 x i32]* %base, i32* %ptr)
+       %base-new = call [3 x i32]* @llvm.experimental.gc.relocate.p0a3i32(i32 %tok,  i32 5, i32 5)
+       %ptr-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 5, i32 6)
        %ret = load i32, i32* %ptr-new
        ret i32 %ret
 }
 
 define i32 @test_sor_gep_largeint([3 x i32]* %base) gc "statepoint-example" {
 ; CHECK: getelementptr [3 x i32], [3 x i32]* %base, i32 0, i32 21
 ; CHECK-NOT: getelementptr [3 x i32], [3 x i32]* %base-new, i32 0, i32 21
 entry:
        %ptr = getelementptr [3 x i32], [3 x i32]* %base, i32 0, i32 21
-       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, [3 x i32]* %base, i32* %ptr)
-       %base-new = call [3 x i32]* @llvm.experimental.gc.relocate.p0a3i32(i32 %tok, i32 4, i32 4)
-       %ptr-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 5)
+       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, [3 x i32]* %base, i32* %ptr)
+       %base-new = call [3 x i32]* @llvm.experimental.gc.relocate.p0a3i32(i32 %tok,  i32 5, i32 5)
+       %ptr-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 5, i32 6)
        %ret = load i32, i32* %ptr-new
        ret i32 %ret
 }
 
 define i32 @test_sor_noop(i32* %base) gc "statepoint-example" {
 ; CHECK: getelementptr i32, i32* %base, i32 15
-; CHECK: call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 5)
-; CHECK: call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 6)
+; CHECK: call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 5, i32 6)
+; CHECK: call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 5, i32 7)
 entry:
        %ptr = getelementptr i32, i32* %base, i32 15
        %ptr2 = getelementptr i32, i32* %base, i32 12
-       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32* %base, i32* %ptr, i32* %ptr2)
-       %ptr-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 5)
-       %ptr2-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 4, i32 6)
+       %tok = call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, i32* %base, i32* %ptr, i32* %ptr2)
+       %ptr-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 5, i32 6)
+       %ptr2-new = call i32* @llvm.experimental.gc.relocate.p0i32(i32 %tok, i32 5, i32 7)
        %ret = load i32, i32* %ptr-new
        ret i32 %ret
 }
 
 declare i32 @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()*, i32, i32, ...)
 declare i32* @llvm.experimental.gc.relocate.p0i32(i32, i32, i32)
 declare [3 x i32]* @llvm.experimental.gc.relocate.p0a3i32(i32, i32, i32)

test/Transforms/InstCombine/gc.relocate.ll

 declare i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32, i32, i32)
 
 define i32 addrspace(1)* @deref(i32 addrspace(1)* dereferenceable(8) %dparam) gc "statepoint-example" {
 ; Checks that a dereferenceabler pointer
 ; CHECK-LABEL: @deref
 ; CHECK: call dereferenceable(8)
 entry:
     %load = load i32, i32 addrspace(1)* %dparam
-    %tok = tail call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 addrspace(1)* %dparam)
-    %relocate = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %tok, i32 4, i32 4)
+    %tok = tail call i32 (i1 ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i1f(i1 ()* @return_i1, i32 0, i32 0, i32 0, i32 0, i32 addrspace(1)* %dparam)
+    %relocate = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %tok,  i32 5, i32 5)
     ret i32 addrspace(1)* %relocate
 }

test/Transforms/InstCombine/statepoint.ll

 ; RUN: opt < %s -instcombine -S | FileCheck %s
 ; These tests check the optimizations specific to
 ; pointers being relocated at a statepoint.
 
 
 declare void @func()
 
 define i1 @test_negative(i32 addrspace(1)* %p) gc "statepoint-example" {
 entry:
-  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 addrspace(1)* %p)
-  %pnew = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 4, i32 4)
+  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 0, i32 addrspace(1)* %p)
+  %pnew = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token,  i32 5, i32 5)
   %cmp = icmp eq i32 addrspace(1)* %pnew, null
   ret i1 %cmp
 ; CHECK-LABEL: test_negative
 ; CHECK: %pnew = call i32 addrspace(1)*
 ; CHECK: ret i1 %cmp
 }
 
 define i1 @test_nonnull(i32 addrspace(1)* nonnull %p) gc "statepoint-example" {
 entry:
-  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 addrspace(1)* %p)
-  %pnew = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 4, i32 4)
+  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 0, i32 addrspace(1)* %p)
+  %pnew = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token,  i32 5, i32 5)
   %cmp = icmp eq i32 addrspace(1)* %pnew, null
   ret i1 %cmp
 ; CHECK-LABEL: test_nonnull
 ; CHECK: ret i1 false
 }
 
 define i1 @test_null() gc "statepoint-example" {
 entry:
-  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 addrspace(1)* null)
-  %pnew = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 4, i32 4)
+  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 0, i32 addrspace(1)* null)
+  %pnew = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token,  i32 5, i32 5)
   %cmp = icmp eq i32 addrspace(1)* %pnew, null
   ret i1 %cmp
 ; CHECK-LABEL: test_null
 ; CHECK-NOT: %pnew
 ; CHECK: ret i1 true
 }
 
 define i1 @test_undef() gc "statepoint-example" {
 entry:
-  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 addrspace(1)* undef)
-  %pnew = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token, i32 4, i32 4)
+  %safepoint_token = tail call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @func, i32 0, i32 0, i32 0, i32 0, i32 addrspace(1)* undef)
+  %pnew = call i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32 %safepoint_token,  i32 5, i32 5)
   %cmp = icmp eq i32 addrspace(1)* %pnew, null
   ret i1 %cmp
 ; CHECK-LABEL: test_undef
 ; CHECK-NOT: %pnew
 ; CHECK: ret i1 undef
 }
 
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)
 declare i32 addrspace(1)* @llvm.experimental.gc.relocate.p1i32(i32, i32, i32) #3

test/Transforms/PlaceSafepoints/basic.ll

 declare zeroext i1 @i1_return_i1(i1)
 
 define i1 @test_call_with_result() gc "statepoint-example" {
 ; CHECK-LABEL: test_call_with_result
 ; This is checking that a statepoint_poll + statepoint + result is
 ; inserted for a function that takes 1 argument.
 ; CHECK: gc.statepoint.p0f_isVoidf
 ; CHECK: gc.statepoint.p0f_i1i1f
-; CHECK: (i1 (i1)* @i1_return_i1, i32 1, i32 0, i1 false, i32 0)
+; CHECK: (i1 (i1)* @i1_return_i1, i32 1, i32 0, i1 false, i32 0, i32 0)
 ; CHECK: %call12 = call i1 @llvm.experimental.gc.result.i1
 entry:
   %call1 = tail call i1 (i1) @i1_return_i1(i1 false)
   ret i1 %call1
 }
 
 ; This function is inlined when inserting a poll.  To avoid recursive 
 ; issues, make sure we don't place safepoints in it.

test/Transforms/RewriteStatepointsForGC/base-pointers-1.ll

 there:
   %y = getelementptr i64, i64 addrspace(1)* %base_obj_y, i32 1
   br label %merge
 
 merge:
 ; CHECK-LABEL: merge:
 ; CHECK:   %base_phi = phi i64 addrspace(1)* [ %base_obj_x, %here ], [ %base_obj_y, %there ]
   %merged_value = phi i64 addrspace(1)* [ %x, %here ], [ %y, %there ]
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @site_for_call_safpeoint, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @site_for_call_safpeoint, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   ret i64 addrspace(1)* %merged_value
 }
 
 declare void @foo()
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)

test/Transforms/RewriteStatepointsForGC/base-pointers-10.ll

 true:
   br label %merge
 
 false:
   br label %merge
 
 merge:
   %next = phi i64 addrspace(1)* [ %next_x, %true ], [ %next_y, %false ]
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   br label %loop
 }
 
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)

test/Transforms/RewriteStatepointsForGC/base-pointers-11.ll

 ; CHECK: phi i64 addrspace(1)*
 ; CHECK-DAG:  [ %base_obj.relocated, %loop ] 
 ; CHECK-DAG:  [ %base_obj, %entry ]
 ; CHECK:  %current = phi i64 addrspace(1)* 
 ; CHECK-DAG:  [ %obj, %entry ]
 ; CHECK-DAG:  [ %next.relocated, %loop ]
   %current = phi i64 addrspace(1)* [ %obj, %entry ], [ %next, %loop ]
   %next = getelementptr i64, i64 addrspace(1)* %current, i32 1
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   br label %loop
 }
 
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)

test/Transforms/RewriteStatepointsForGC/base-pointers-2.ll

 ; RUN: opt %s -rewrite-statepoints-for-gc -spp-print-base-pointers -S 2>&1 | FileCheck %s
 
 ; CHECK: derived %merged_value base %base_obj
 
 
 define i64 addrspace(1)* @test(i64 addrspace(1)* %base_obj, i1 %runtime_condition) gc "statepoint-example" {
 entry:
   br i1 %runtime_condition, label %merge, label %there
 
 there:
   %derived_obj = getelementptr i64, i64 addrspace(1)* %base_obj, i32 1
   br label %merge
 
 merge:
   %merged_value = phi i64 addrspace(1)* [ %base_obj, %entry ], [ %derived_obj, %there ]
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   ret i64 addrspace(1)* %merged_value
 }
 
 declare void @foo()
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)

test/Transforms/RewriteStatepointsForGC/base-pointers-3.ll

 ; RUN: opt %s -rewrite-statepoints-for-gc -spp-print-base-pointers -S 2>&1 | FileCheck %s
 
 ; CHECK: derived %next.i64 base %base_obj
 
 define void @test(i64 addrspace(1)* %base_obj) gc "statepoint-example" {
 entry:
   %obj = getelementptr i64, i64 addrspace(1)* %base_obj, i32 1
   br label %loop
 
 loop:
   %current = phi i64 addrspace(1)* [ %obj, %entry ], [ %next.i64, %loop ]
   %current.i32 = bitcast i64 addrspace(1)* %current to i32 addrspace(1)*
   %next.i32 = getelementptr i32, i32 addrspace(1)* %current.i32, i32 1
   %next.i64 = bitcast i32 addrspace(1)* %next.i32 to i64 addrspace(1)*
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   br label %loop
 }
 
 declare void @do_safepoint()
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)

test/Transforms/RewriteStatepointsForGC/base-pointers-4.ll

 ; RUN: opt %s -rewrite-statepoints-for-gc -spp-print-base-pointers -S 2>&1 | FileCheck %s
 
 
 ; CHECK: derived %obj_to_consume base %base_phi
 
 declare void @foo()
 declare i64 addrspace(1)* @generate_obj()
 declare void @consume_obj(i64 addrspace(1)*)
 
 define void @test(i32 %condition) gc "statepoint-example" {
 entry:
   br label %loop
 
 loop:
 ; CHECK: loop:
-; CHECK:  %safepoint_token1 = call i32 (i64 addrspace(1)* ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p1i64f(i64 addrspace(1)* ()* @generate_obj, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i3
+; CHECK:  %safepoint_token1 = call i32 (i64 addrspace(1)* ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p1i64f(i64 addrspace(1)* ()* @generate_obj, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i3
 ; CHECK-NEXT:  %obj2 = call i64 addrspace(1)* @llvm.experimental.gc.result
-  %safepoint_token1 = call i32 (i64 addrspace(1)* ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p1i64f(i64 addrspace(1)* ()* @generate_obj, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token1 = call i32 (i64 addrspace(1)* ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p1i64f(i64 addrspace(1)* ()* @generate_obj, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   %obj2 = call i64 addrspace(1)* @llvm.experimental.gc.result.p1i64(i32 %safepoint_token1)
   switch i32 %condition, label %dest_a [
     i32 0, label %dest_b
     i32 1, label %dest_c
   ]
 
 dest_a:
   br label %merge
 
 dest_b:
   br label %merge
 
 dest_c:
   br label %merge
 
 merge:
 ; CHECK: merge:
 ; CHECK:  %base_phi = phi i64 addrspace(1)* [ %obj2, %dest_a ], [ null, %dest_b ], [ null, %dest_c ]
 ; CHECK:  %obj_to_consume = phi i64 addrspace(1)* [ %obj2, %dest_a ], [ null, %dest_b ], [ null, %dest_c ]
 
   %obj_to_consume = phi i64 addrspace(1)* [ %obj2, %dest_a ], [ null, %dest_b ], [ null, %dest_c ]
-  %safepoint_token3 = call i32 (void (i64 addrspace(1)*)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidp1i64f(void (i64 addrspace(1)*)* @consume_obj, i32 1, i32 0, i64 addrspace(1)* %obj_to_consume, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token3 = call i32 (void (i64 addrspace(1)*)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidp1i64f(void (i64 addrspace(1)*)* @consume_obj, i32 1, i32 0, i64 addrspace(1)* %obj_to_consume, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   br label %merge.split
 
 merge.split:                                      ; preds = %merge
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   br label %loop
 }
 
 
 ; Function Attrs: nounwind
 declare i64 addrspace(1)* @llvm.experimental.gc.result.p1i64(i32) #0
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)
 declare i32 @llvm.experimental.gc.statepoint.p0f_p1i64f(i64 addrspace(1)* ()*, i32, i32, ...)
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidp1i64f(void (i64 addrspace(1)*)*, i32, i32, ...)

test/Transforms/RewriteStatepointsForGC/base-pointers-5.ll

there:
   %y = getelementptr i64, i64 addrspace(1)* %base_obj_y, i32 1
   br label %merge
 
 merge:
 ; CHECK: merge:
 ; CHECK:  %base_phi = phi i64 addrspace(1)* [ %base_obj_x, %bump ], [ %base_obj_y, %there ]
 ; CHECK-NEXT:  %merged_value = phi i64 addrspace(1)* [ %base_obj_x, %bump ], [ %y, %there ]  
   %merged_value = phi i64 addrspace(1)* [ %base_obj_x, %bump ], [ %y, %there ]
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   ret i64 addrspace(1)* %merged_value
 }
 
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)
 

test/Transforms/RewriteStatepointsForGC/base-pointers-6.ll

there:
   %y = getelementptr i64, i64 addrspace(1)* %base_obj_y, i32 1
   br label %merge
 
 merge:
 ; CHECK: merge:
 ; CHECK:  %base_phi = phi i64 addrspace(1)* [ %base_obj_x, %merge_here ], [ %base_obj_y, %there ]
 ; CHECK-NEXT:  %merged_value = phi i64 addrspace(1)* [ %x, %merge_here ], [ %y, %there ]  
   %merged_value = phi i64 addrspace(1)* [ %x, %merge_here ], [ %y, %there ]
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @site_for_call_safpeoint, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @site_for_call_safpeoint, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   ret i64 addrspace(1)* %merged_value
 }
 
 declare void @do_safepoint()
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)

test/Transforms/RewriteStatepointsForGC/base-pointers-7.ll

 ; CHECK: merge:
 ; CHECK-DAG:  %base_phi1
 ; CHECK-DAG: phi i64 addrspace(1)*
 ; CHECK-DAG: %merge_here
 ; CHECK-DAG: [ %base_obj_y, %there ]
 ; CHECK:  %merged_value = phi i64 addrspace(1)* [ %x, %merge_here ], [ %y, %there ]  
   %merged_value = phi i64 addrspace(1)* [ %x, %merge_here ], [ %y, %there ]
 
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @site_for_call_safpeoint, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @site_for_call_safpeoint, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   ret i64 addrspace(1)* %merged_value
 }
 
 declare void @do_safepoint()
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)
 

test/Transforms/RewriteStatepointsForGC/base-pointers-8.ll

 check_for_null:
   %current_element = load i64 addrspace(1)*, i64 addrspace(1)* addrspace(1)* %current_element_ptr
   %is_null = icmp eq i64 addrspace(1)* %current_element, null
   br i1 %is_null, label %found, label %loop_back
 
 loop_back:
   %next_element_ptr = getelementptr i64 addrspace(1)*, i64 addrspace(1)* addrspace(1)* %current_element_ptr, i32 1
   %next_index = add i32 %index, 1
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   br label %loop_check
 
 not_found:
   ret i32 -1
 
 found:
   ret i32 %index
 }
 
 declare void @do_safepoint()
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)

test/Transforms/RewriteStatepointsForGC/base-pointers-9.ll

 ; RUN: opt %s -rewrite-statepoints-for-gc -spp-print-base-pointers -S  2>&1 | FileCheck %s
 
 ; CHECK: derived %next base %base_obj
 
 declare i1 @runtime_value()
 
 define void @maybe_GEP(i64 addrspace(1)* %base_obj) gc "statepoint-example" {
 entry:
   br label %loop
 
 loop:
   %current = phi i64 addrspace(1)* [ %base_obj, %entry ], [ %next, %loop ]
   %condition = call i1 @runtime_value()
   %maybe_next = getelementptr i64, i64 addrspace(1)* %current, i32 1
   %next = select i1 %condition, i64 addrspace(1)* %maybe_next, i64 addrspace(1)* %current
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   br label %loop
 }
 
 declare void @do_safepoint()
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)

test/Transforms/RewriteStatepointsForGC/base-pointers.ll

entry:
   %obj = call i64 addrspace(1)* @generate_obj()
   br label %loop
 
 loop:
 ; CHECK: phi i64 addrspace(1)* 
 ; CHECK-DAG: [ %obj.relocated, %loop ]
 ; CHECK-DAG: [ %obj, %entry ]
   call void @use_obj(i64 addrspace(1)* %obj)
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   br label %loop
 }
 
 declare void @do_safepoint()
 
 declare void @parse_point(i64 addrspace(1)*)
 
 define i64 addrspace(1)* @test1(i32 %caller, i8 addrspace(1)* %a, i8 addrspace(1)* %b, i32 %unknown) gc "statepoint-example" {

 ; CHECK: right:
 ; CHECK-NEXT: %b.cast = bitcast i8 addrspace(1)* %b to i64 addrspace(1)*
 ; CHECK-NEXT: [[CAST_R:%.*]] = bitcast i8 addrspace(1)* %b to i64 addrspace(1)*
 
  merge:
 ; CHECK: merge:
 ; CHECK-NEXT: %base_phi = phi i64 addrspace(1)* [ [[CAST_L]], %left ], [ [[CAST_L]], %left ], [ [[CAST_L]], %left ], [ [[CAST_R]], %right ], !is_base_value !0
   %value = phi i64 addrspace(1)* [ %a.cast, %left], [ %a.cast, %left], [ %a.cast, %left], [ %b.cast, %right]
-  %safepoint_token = call i32 (void (i64 addrspace(1)*)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidp1i64f(void (i64 addrspace(1)*)* @parse_point, i32 1, i32 0, i64 addrspace(1)* %value, i32 5, i32 0, i32 0, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void (i64 addrspace(1)*)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidp1i64f(void (i64 addrspace(1)*)* @parse_point, i32 1, i32 0, i64 addrspace(1)* %value, i32 0, i32 5, i32 0, i32 0, i32 0, i32 0, i32 0)
 
   ret i64 addrspace(1)* %value
 }
 
 ;; The purpose of this test is to ensure that when two live values share a
 ;;  base defining value with inherent conflicts, we end up with a *single*
 ;;  base phi/select per such node.  This is testing an optimization, not a
 ;;  fundemental correctness criteria

 ; CHECK: base_select
 ; CHECK: statepoint
 ;; Both 'next' and 'extra2' are live across the backedge safepoint...
   %current = phi i64 addrspace(1)* [ %obj, %entry ], [ %next, %loop ]
   %extra = phi i64 addrspace(1)* [ %obj, %entry ], [ %extra2, %loop ]
   %nexta = getelementptr i64, i64 addrspace(1)* %current, i32 1
   %next = select i1 %cnd, i64 addrspace(1)* %nexta, i64 addrspace(1)* %base_arg2
   %extra2 = select i1 %cnd, i64 addrspace(1)* %nexta, i64 addrspace(1)* %base_arg2
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   br label %loop
 }
 
 declare void @foo()
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidp1i64f(void (i64 addrspace(1)*)*, i32, i32, ...)

test/Transforms/RewriteStatepointsForGC/basics.ll

 ; This is a collection of really basic tests for gc.statepoint rewriting.
 ; RUN:  opt %s -rewrite-statepoints-for-gc -S | FileCheck %s
 
 declare void @foo()
 
 ; Trivial relocation over a single call
 define i8 addrspace(1)* @test1(i8 addrspace(1)* %obj) gc "statepoint-example" {
 ; CHECK-LABEL: @test1
 ; CHECK-LABEL: entry:
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: %obj.relocated = call coldcc i8 addrspace(1)*
 entry:
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   ret i8 addrspace(1)* %obj
 }
 
 ; Two safepoints in a row (i.e. consistent liveness)
 define i8 addrspace(1)* @test2(i8 addrspace(1)* %obj) gc "statepoint-example" {
 ; CHECK-LABEL: @test2
 ; CHECK-LABEL: entry:
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: %obj.relocated = call coldcc i8 addrspace(1)*
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: %obj.relocated1 = call coldcc i8 addrspace(1)*
 entry:
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   ret i8 addrspace(1)* %obj
 }
 
 ; A simple derived pointer
 define i8 @test3(i8 addrspace(1)* %obj) gc "statepoint-example" {
 ; CHECK-LABEL: entry:
 ; CHECK-NEXT: getelementptr
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: %derived.relocated = call coldcc i8 addrspace(1)*
 ; CHECK-NEXT: %obj.relocated = call coldcc i8 addrspace(1)*
 ; CHECK-NEXT: load i8, i8 addrspace(1)* %derived.relocated
 ; CHECK-NEXT: load i8, i8 addrspace(1)* %obj.relocated
 entry:
   %derived = getelementptr i8, i8 addrspace(1)* %obj, i64 10
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
 
   %a = load i8, i8 addrspace(1)* %derived
   %b = load i8, i8 addrspace(1)* %obj
   %c = sub i8 %a, %b
   ret i8 %c
 }
 
 ; Tests to make sure we visit both the taken and untaken predeccessor 
 ; of merge.  This was a bug in the dataflow liveness at one point.
 define i8 addrspace(1)* @test4(i1 %cmp, i8 addrspace(1)* %obj) gc "statepoint-example" {
 entry:
   br i1 %cmp, label %taken, label %untaken
 
 taken:
 ; CHECK-LABEL: taken:
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: %obj.relocated = call coldcc i8 addrspace(1)*
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   br label %merge
 
 untaken:
 ; CHECK-LABEL: untaken:
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: %obj.relocated1 = call coldcc i8 addrspace(1)*
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   br label %merge
 
 merge:
 ; CHECK-LABEL: merge:
 ; CHECK-NEXT: %.0 = phi i8 addrspace(1)* [ %obj.relocated, %taken ], [ %obj.relocated1, %untaken ]
 ; CHECK-NEXT: ret i8 addrspace(1)* %.0
   ret i8 addrspace(1)* %obj
 }
 
 ; When run over a function which doesn't opt in, should do nothing!
 define i8 addrspace(1)* @test5(i8 addrspace(1)* %obj) gc "ocaml" {
 ; CHECK-LABEL: @test5
 ; CHECK-LABEL: entry:
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NOT: %obj.relocated = call coldcc i8 addrspace(1)*
 entry:
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   ret i8 addrspace(1)* %obj
 }
 
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)

test/Transforms/RewriteStatepointsForGC/constants.ll

 ; RUN: opt -S -rewrite-statepoints-for-gc %s | FileCheck %s
 
 declare void @foo()
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)
 
 ; constants don't get relocated.
 define i8 @test() gc "statepoint-example" {
 ; CHECK-LABEL: @test
 ; CHECK: gc.statepoint
 ; CHECK-NEXT: load i8, i8 addrspace(1)* inttoptr (i64 15 to i8 addrspace(1)*)
 entry:
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   %res = load i8, i8 addrspace(1)* inttoptr (i64 15 to i8 addrspace(1)*)
   ret i8 %res
 }
 
 
 ; Mostly just here to show reasonable code test can come from.  
 define i8 @test2(i8 addrspace(1)* %p) gc "statepoint-example" {
 ; CHECK-LABEL: @test2
 ; CHECK: gc.statepoint
 ; CHECK-NEXT: gc.relocate
 ; CHECK-NEXT: icmp
 entry:
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   %cmp = icmp eq i8 addrspace(1)* %p, null
   br i1 %cmp, label %taken, label %not_taken
 
 taken:
   ret i8 0
 
 not_taken:
   %cmp2 = icmp ne i8 addrspace(1)* %p, null

 @G = addrspace(1) global i8 5
 
 ; Globals don't move and thus don't get relocated
 define i8 @test3(i1 %always_true) gc "statepoint-example" {
 ; CHECK-LABEL: @test3
 ; CHECK: gc.statepoint
 ; CHECK-NEXT: load i8, i8 addrspace(1)* @G
 entry:
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   %res = load i8, i8 addrspace(1)* @G, align 1
   ret i8 %res
 }
 
 
 

test/Transforms/RewriteStatepointsForGC/live-vector.ll

 ; Test that we can correctly handle vectors of pointers in statepoint 
 ; rewriting.  Currently, we scalarize, but that's an implementation detail.
 ; RUN: opt %s -rewrite-statepoints-for-gc -S | FileCheck  %s
 
 ; A non-vector relocation for comparison
 define i64 addrspace(1)* @test(i64 addrspace(1)* %obj) gc "statepoint-example" {
 ; CHECK-LABEL: test
 ; CHECK: gc.statepoint
 ; CHECK-NEXT: gc.relocate
 ; CHECK-NEXT: ret i64 addrspace(1)* %obj.relocated
 entry:
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0, i32 0)
   ret i64 addrspace(1)* %obj
 }
 
 ; A base vector from a argument
 define <2 x i64 addrspace(1)*> @test2(<2 x i64 addrspace(1)*> %obj) gc "statepoint-example" {
 ; CHECK-LABEL: test2
 ; CHECK: extractelement
 ; CHECK-NEXT: extractelement
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: gc.relocate
 ; CHECK-NEXT: gc.relocate
 ; CHECK-NEXT: insertelement
 ; CHECK-NEXT: insertelement
 ; CHECK-NEXT: ret <2 x i64 addrspace(1)*> %5
 entry:
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0, i32 0)
   ret <2 x i64 addrspace(1)*> %obj
 }
 
 ; A base vector from a load
 define <2 x i64 addrspace(1)*> @test3(<2 x i64 addrspace(1)*>* %ptr) gc "statepoint-example" {
 ; CHECK-LABEL: test3
 ; CHECK: load
 ; CHECK-NEXT: extractelement
 ; CHECK-NEXT: extractelement
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: gc.relocate
 ; CHECK-NEXT: gc.relocate
 ; CHECK-NEXT: insertelement
 ; CHECK-NEXT: insertelement
 ; CHECK-NEXT: ret <2 x i64 addrspace(1)*> %5
 entry:
   %obj = load <2 x i64 addrspace(1)*>, <2 x i64 addrspace(1)*>* %ptr
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0, i32 0)
   ret <2 x i64 addrspace(1)*> %obj
 }
 
 declare i32 @fake_personality_function()
 
 ; When a statepoint is an invoke rather than a call
 define <2 x i64 addrspace(1)*> @test4(<2 x i64 addrspace(1)*>* %ptr) gc "statepoint-example" {
 ; CHECK-LABEL: test4
 ; CHECK: load
 ; CHECK-NEXT: extractelement
 ; CHECK-NEXT: extractelement
 ; CHECK-NEXT: gc.statepoint
 entry:
   %obj = load <2 x i64 addrspace(1)*>, <2 x i64 addrspace(1)*>* %ptr
-  invoke i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0)
+  invoke i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0, i32 0)
           to label %normal_return unwind label %exceptional_return
 
 ; CHECK-LABEL: normal_return:
 ; CHECK: gc.relocate
 ; CHECK-NEXT: gc.relocate
 ; CHECK-NEXT: insertelement
 ; CHECK-NEXT: insertelement
 ; CHECK-NEXT: ret <2 x i64 addrspace(1)*> %6

test/Transforms/RewriteStatepointsForGC/liveness-basics.ll

 entry:
   br i1 %cmp, label %taken, label %untaken
 
 taken:
 ; CHECK-LABEL: taken:
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: %obj.relocated = call coldcc i64 addrspace(1)*
 ; CHECK-NEXT: br label %merge
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   br label %merge
 
 untaken:
 ; CHECK-LABEL: untaken:
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: %obj.relocated1 = call coldcc i64 addrspace(1)*
 ; CHECK-NEXT: br label %merge
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   br label %merge
 
 merge:
 ; CHECK-LABEL: merge:
 ; CHECK-NEXT: %.0 = phi i64 addrspace(1)* [ %obj.relocated, %taken ], [ %obj.relocated1, %untaken ]
 ; CHECK-NEXT: ret i64 addrspace(1)* %.0
   ret i64 addrspace(1)* %obj
 }
 
 ; A local kill should not effect liveness in predecessor block
 define i64 addrspace(1)* @test2(i1 %cmp, i64 addrspace(1)** %loc) gc "statepoint-example" {
 ; CHECK-LABEL: @test2
 entry:
 ; CHECK-LABEL: entry:
 ; CHECK-NEXT:  gc.statepoint
 ; CHECK-NEXT:  br
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   br i1 %cmp, label %taken, label %untaken
 
 taken:
 ; CHECK-LABEL: taken:
 ; CHECK-NEXT:  %obj = load
 ; CHECK-NEXT:  gc.statepoint
 ; CHECK-NEXT:  gc.relocate
 ; CHECK-NEXT:  ret i64 addrspace(1)* %obj.relocated
 
   %obj = load i64 addrspace(1)*, i64 addrspace(1)** %loc
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   ret i64 addrspace(1)* %obj
 
 untaken:
   ret i64 addrspace(1)* null
 }
 
 ; A local kill should effect values live from a successor phi.  Also, we
 ; should only propagate liveness from a phi to the appropriate predecessors.
 define i64 addrspace(1)* @test3(i1 %cmp, i64 addrspace(1)** %loc) gc "statepoint-example" {
 ; CHECK-LABEL: @test3
 entry:
   br i1 %cmp, label %taken, label %untaken
 
 taken:
 ; CHECK-LABEL: taken:
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: %obj = load
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: %obj.relocated = call coldcc i64 addrspace(1)*
 ; CHECK-NEXT: br label %merge
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   %obj = load i64 addrspace(1)*, i64 addrspace(1)** %loc
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   br label %merge
 
 untaken:
 ; CHECK-LABEL: taken:
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: br label %merge
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   br label %merge
 
 merge:
   %phi = phi i64 addrspace(1)* [ %obj, %taken ], [ null, %untaken ]
   ret i64 addrspace(1)* %phi
 }
 
 ; A base pointer must be live if it is needed at a later statepoint,
 ; even if the base pointer is otherwise unused.
 define i64 addrspace(1)* @test4(i1 %cmp, i64 addrspace(1)* %obj) gc "statepoint-example" {
 ; CHECK-LABEL: @test4
 entry:
 ; CHECK-LABEL: entry:
 ; CHECK-NEXT:  %derived = getelementptr
 ; CHECK-NEXT:  gc.statepoint
 ; CHECK-NEXT:  %derived.relocated = 
 ; CHECK-NEXT:  %obj.relocated =
 ; CHECK-NEXT:  gc.statepoint
 ; CHECK-NEXT:  %derived.relocated1 = 
 ; Note: It's legal to relocate obj again, but not strictly needed
 ; CHECK-NEXT:  %obj.relocated2 =
 ; CHECK-NEXT:  ret i64 addrspace(1)* %derived.relocated1
 ; 
   %derived = getelementptr i64, i64 addrspace(1)* %obj, i64 8
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
 
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   ret i64 addrspace(1)* %derived
 }
 
 declare void @consume(...) readonly
 
 ; Make sure that a phi def visited during iteration is considered a kill.
 ; Also, liveness after base pointer analysis can change based on new uses,
 ; not just new defs.
 define i64 addrspace(1)* @test5(i1 %cmp, i64 addrspace(1)* %obj) gc "statepoint-example" {
 ; CHECK-LABEL: @test5
 entry:
   br i1 %cmp, label %taken, label %untaken
 
 taken:
 ; CHECK-LABEL: taken:
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: %obj.relocated = call coldcc i64 addrspace(1)*
 ; CHECK-NEXT: br label %merge
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   br label %merge
 
 untaken:
 ; CHECK-LABEL: untaken:
 ; CHECK-NEXT: br label %merge
   br label %merge
 
 merge:

test/Transforms/RewriteStatepointsForGC/preprocess.ll

 
 next:
 ; CHECK-LABEL: next:
 ; CHECK-NEXT: gc.statepoint
 ; CHECK-NEXT: gc.relocate
 ; CHECK-NEXT: @consume(i64 addrspace(1)* %obj.relocated)
 ; CHECK-NEXT: @consume(i64 addrspace(1)* %obj.relocated)
   %obj2 = phi i64 addrspace(1)* [ %obj, %entry ]
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   call void (...) @consume(i64 addrspace(1)* %obj2)
   call void (...) @consume(i64 addrspace(1)* %obj)
   ret void
 }
 
 declare void @some_call(i64 addrspace(1)*)
 
 ; Need to delete unreachable gc.statepoint call
 define void @test7() gc "statepoint-example" {
 ; CHECK-LABEL: test7
 ; CHECK-NOT: gc.statepoint
   ret void
 
 unreached:
   %obj = phi i64 addrspace(1)* [null, %unreached]
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   call void (...) @consume(i64 addrspace(1)* %obj)
   br label %unreached
 }
 
 ; Need to delete unreachable gc.statepoint invoke - tested seperately given
 ; a correct implementation could only remove the instructions, not the block
 define void @test8() gc "statepoint-example" {
 ; CHECK-LABEL: test8
 ; CHECK-NOT: gc.statepoint
   ret void
 
 unreached:
-  invoke i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  invoke i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
           to label %normal_return unwind label %exceptional_return
 
 normal_return:                                    ; preds = %entry
   ret void
 
 exceptional_return:                               ; preds = %entry
   %landing_pad4 = landingpad { i8*, i32 } personality i32 ()* undef
           cleanup

test/Transforms/RewriteStatepointsForGC/relocate_invoke_result.ll

unwind_dest:
   %lpad = landingpad { i8*, i32 } personality i32* ()* @fake_personality_function
           cleanup
   resume { i8*, i32 } undef
 
 normal_dest:
 ;; CHECK-LABEL: normal_dest:
 ;; CHECK-NEXT: gc.statepoint
 ;; CHECK-NEXT: %obj.relocated = call coldcc i64* addrspace(1)*                                      
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @gc_call, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @gc_call, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   ret i64* addrspace(1)* %obj
 }
 
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)
 
 

test/Transforms/RewriteStatepointsForGC/relocation.ll

 ; RUN: opt %s -rewrite-statepoints-for-gc -S 2>&1 | FileCheck %s
 
 
 declare void @foo()
 declare void @use(...)
 
 define i64 addrspace(1)* @test1(i64 addrspace(1)* %obj, i64 addrspace(1)* %obj2, i1 %condition) gc "statepoint-example" {
 entry:
 ; CHECK-LABEL: @test1
 ; CHECK-DAG: %obj.relocated
 ; CHECK-DAG: %obj2.relocated
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   br label %joint
 
 joint:
 ; CHECK-LABEL: joint:
 ; CHECK: %phi1 = phi i64 addrspace(1)* [ %obj.relocated, %entry ], [ %obj3, %joint2 ]
   %phi1 = phi i64 addrspace(1)* [ %obj, %entry ], [ %obj3, %joint2 ]
   br i1 %condition, label %use, label %joint2
 
; CHECK-NOT: %location = getelementptr i64, i64 addrspace(1)* %obj, i32 %index
   %index.inc = add i32 %index, 1
   %condition = call i1 @rt()
   br i1 %condition, label %loop_x, label %loop_y
 
 loop_x:
   br label %loop.backedge
 
 loop.backedge:
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @do_safepoint, i32 0, i32 0, i32 0, i32 0)
   br label %loop
 
 loop_y:
   br label %loop.backedge
 }
 
 declare void @some_call(i8 addrspace(1)*)
 
 define void @relocate_merge(i1 %cnd, i8 addrspace(1)* %arg) gc "statepoint-example" {
 ; CHECK-LABEL: @relocate_merge
 bci_0:
   br i1 %cnd, label %if_branch, label %else_branch
 
 if_branch:
 ; CHECK-LABEL: if_branch:
 ; CHECK: gc.statepoint
 ; CHECK: gc.relocate
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   br label %join
 
 else_branch:
 ; CHECK-LABEL: else_branch:
 ; CHECK: gc.statepoint
 ; CHECK: gc.relocate
-  %safepoint_token1 = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0)
+  %safepoint_token1 = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 0)
   br label %join
 
 join:
 ; We need to end up with a single relocation phi updated from both paths 
 ; CHECK-LABEL: join:
 ; CHECK: phi i8 addrspace(1)*
 ; CHECK-DAG: [ %arg.relocated, %if_branch ]
 ; CHECK-DAG: [ %arg.relocated4, %else_branch ]
 ; CHECK-NOT: phi
   call void (i8 addrspace(1)*) @some_call(i8 addrspace(1)* %arg)
   ret void
 }
 
 ; Make sure a use in a statepoint gets properly relocated at a previous one.  
 ; This is basically just making sure that statepoints aren't accidentally 
 ; treated specially.
 define void @test3(i64 addrspace(1)* %obj) gc "statepoint-example" {
 entry:
 ; CHECK-LABEL: @test3
 ; CHECK: gc.statepoint
 ; CHECK-NEXT: gc.relocate
 ; CHECK-NEXT: gc.statepoint
-  %safepoint_token = call i32 (void (i64)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidi64f(void (i64)* undef, i32 1, i32 0, i64 undef, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
-  %safepoint_token1 = call i32 (i32 (i64 addrspace(1)*)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i32p1i64f(i32 (i64 addrspace(1)*)* undef, i32 1, i32 0, i64 addrspace(1)* %obj, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void (i64)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidi64f(void (i64)* undef, i32 1, i32 0, i64 undef, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token1 = call i32 (i32 (i64 addrspace(1)*)*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_i32p1i64f(i32 (i64 addrspace(1)*)* undef, i32 1, i32 0, i64 addrspace(1)* %obj, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   ret void
 }
 
 ; Check specifically for the case where the result of a statepoint needs to 
 ; be relocated itself
 define void @test4() gc "statepoint-example" {
 ; CHECK-LABEL: @test4
 ; CHECK: gc.statepoint
 ; CHECK: gc.result
 ; CHECK: gc.statepoint
 ; CHECK: gc.relocate
 ; CHECK: @use(i8 addrspace(1)* %res.relocated)
-  %safepoint_token2 = tail call i32 (i8 addrspace(1)* ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p1i8f(i8 addrspace(1)* ()* undef, i32 0, i32 0, i32 0)
+  %safepoint_token2 = tail call i32 (i8 addrspace(1)* ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p1i8f(i8 addrspace(1)* ()* undef, i32 0, i32 0, i32 0, i32 0)
   %res = call i8 addrspace(1)* @llvm.experimental.gc.result.ptr.p1i8(i32 %safepoint_token2)
-  call i32 (i8 addrspace(1)* ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p1i8f(i8 addrspace(1)* ()* undef, i32 0, i32 0, i32 0)
+  call i32 (i8 addrspace(1)* ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p1i8f(i8 addrspace(1)* ()* undef, i32 0, i32 0, i32 0, i32 0)
   call void (...) @use(i8 addrspace(1)* %res)
   unreachable
 }
 
 
 ; Test updating a phi where not all inputs are live to begin with
 define void @test5(i8 addrspace(1)* %arg) gc "statepoint-example" {
 ; CHECK-LABEL: test5
 entry:
-  call i32 (i8 addrspace(1)* ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p1i8f(i8 addrspace(1)* ()* undef, i32 0, i32 0, i32 0)
+  call i32 (i8 addrspace(1)* ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_p1i8f(i8 addrspace(1)* ()* undef, i32 0, i32 0, i32 0, i32 0)
   switch i32 undef, label %kill [
     i32 10, label %merge
     i32 13, label %merge
   ]
 
 kill:
   br label %merge
 
entry:
   br i1 undef, label %gc.safepoint_poll.exit2, label %do_safepoint
 
 do_safepoint:
 ; CHECK-LABEL: do_safepoint:
 ; CHECK: gc.statepoint
 ; CHECK: arg1.relocated = 
 ; CHECK: arg2.relocated = 
 ; CHECK: arg3.relocated = 
-  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 3, i8 addrspace(1)* %arg1, i8 addrspace(1)* %arg2, i8 addrspace(1)* %arg3)
+  call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 3, i8 addrspace(1)* %arg1, i8 addrspace(1)* %arg2, i8 addrspace(1)* %arg3)
   br label %gc.safepoint_poll.exit2
 
 gc.safepoint_poll.exit2:
 ; CHECK-LABEL: gc.safepoint_poll.exit2:
 ; CHECK: phi i8 addrspace(1)*
 ; CHECK-DAG: [ %arg3, %entry ]
 ; CHECK-DAG: [ %arg3.relocated, %do_safepoint ]
 ; CHECK: phi i8 addrspace(1)*

 
 inner-loop:
   br i1 %cmp, label %inner-loop, label %outer-inc
 
 outer-inc:
 ; CHECK-LABEL: outer-inc:
 ; CHECK: %arg1.relocated
 ; CHECK: %arg2.relocated
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 2, i8 addrspace(1)* %arg1, i8 addrspace(1)* %arg2)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 2, i8 addrspace(1)* %arg1, i8 addrspace(1)* %arg2)
   br label %outer-loop
 }
 
 ; Check that both inner and outer loops get phis when relocation is in
 ;  inner loop
 define void @test_inner_loop(i8 addrspace(1)* %arg1, i8 addrspace(1)* %arg2, 
                   i1 %cmp) gc "statepoint-example" {
 ; CHECK-LABEL: @test_inner_loop

 ; CHECK-DAG: %outer-loop ]
 ; CHECK-DAG: [ %arg2.relocated, %inner-loop ]
 ; CHECKL phi i8 addrspace(1)* 
 ; CHECK-DAG: %outer-loop ]
 ; CHECK-DAG: [ %arg1.relocated, %inner-loop ]
 ; CHECK: gc.statepoint
 ; CHECK: %arg1.relocated
 ; CHECK: %arg2.relocated
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 2, i8 addrspace(1)* %arg1, i8 addrspace(1)* %arg2)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 2, i8 addrspace(1)* %arg1, i8 addrspace(1)* %arg2)
   br i1 %cmp, label %inner-loop, label %outer-inc
 
 outer-inc:
 ; CHECK-LABEL: outer-inc:
   br label %outer-loop
 }
 
 
 ; This test shows why updating just those uses of the original value being
 ; relocated dominated by the inserted relocation is not always sufficient.
 define i64 addrspace(1)* @test7(i64 addrspace(1)* %obj, i64 addrspace(1)* %obj2, i1 %condition) gc "statepoint-example" {
 ; CHECK-LABEL: @test7
 entry:
   br i1 %condition, label %branch2, label %join
 
 branch2:
   br i1 %condition, label %callbb, label %join2
 
 callbb:
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* @foo, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0)
   br label %join
 
 join:
 ; CHECK-LABEL: join:
 ; CHECK: phi i64 addrspace(1)* [ %obj.relocated, %callbb ], [ %obj, %entry ]
 ; CHECK: phi i64 addrspace(1)* 
 ; CHECK-DAG: [ %obj, %entry ]
 ; CHECK-DAG: [ %obj2.relocated, %callbb ]

test/Verifier/invalid-statepoint2.ll

 ; RUN: not opt -S < %s -verify 2>&1 | FileCheck %s
 
 ; CHECK: gc.statepoint: number of deoptimization arguments must be a constant integer
 
 declare void @use(...)
 declare i8 addrspace(1)* @llvm.experimental.gc.relocate.p1i8(i32, i32, i32)
 declare i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(i32, i32, i32)
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)
 declare i32 @"personality_function"()
 
 ;; Basic usage
 define i64 addrspace(1)* @test1(i8 addrspace(1)* %arg, i32 %val) gc "statepoint-example" {
 entry:
   %cast = bitcast i8 addrspace(1)* %arg to i64 addrspace(1)*
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 %val, i32 0, i32 0, i32 0, i32 10, i32 0, i8 addrspace(1)* %arg, i64 addrspace(1)* %cast, i8 addrspace(1)* %arg, i8 addrspace(1)* %arg)
-  %reloc = call i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(i32 %safepoint_token, i32 9, i32 10)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 0, i32 %val, i32 0, i32 0, i32 0, i32 10, i32 0, i8 addrspace(1)* %arg, i64 addrspace(1)* %cast, i8 addrspace(1)* %arg, i8 addrspace(1)* %arg)
+  %reloc = call i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(i32 %safepoint_token, i32 10, i32 11)
   ret i64 addrspace(1)* %reloc
 }
 

test/Verifier/statepoint.ll

 ; RUN: opt -S %s -verify | FileCheck %s
 
 declare void @use(...)
 declare i8 addrspace(1)* @llvm.experimental.gc.relocate.p1i8(i32, i32, i32)
 declare i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(i32, i32, i32)
 declare i32 @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()*, i32, i32, ...)
 declare i32 @"personality_function"()
 
 ;; Basic usage
 define i64 addrspace(1)* @test1(i8 addrspace(1)* %arg) gc "statepoint-example" {
 entry:
   %cast = bitcast i8 addrspace(1)* %arg to i64 addrspace(1)*
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 5, i32 0, i32 0, i32 0, i32 10, i32 0, i8 addrspace(1)* %arg, i64 addrspace(1)* %cast, i8 addrspace(1)* %arg, i8 addrspace(1)* %arg)
-  %reloc = call i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(i32 %safepoint_token, i32 9, i32 10)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 0, i32 0, i32 10, i32 0, i8 addrspace(1)* %arg, i64 addrspace(1)* %cast, i8 addrspace(1)* %arg, i8 addrspace(1)* %arg)
+  %reloc = call i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(i32 %safepoint_token, i32 10, i32 11)
   ;; It is perfectly legal to relocate the same value multiple times...
-  %reloc2 = call i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(i32 %safepoint_token, i32 9, i32 10)
-  %reloc3 = call i8 addrspace(1)* @llvm.experimental.gc.relocate.p1i8(i32 %safepoint_token, i32 10, i32 9)
+  %reloc2 = call i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(i32 %safepoint_token, i32 10, i32 11)
+  %reloc3 = call i8 addrspace(1)* @llvm.experimental.gc.relocate.p1i8(i32 %safepoint_token, i32 11, i32 10)
   ret i64 addrspace(1)* %reloc
 ; CHECK-LABEL: test1
 ; CHECK: statepoint
 ; CHECK: gc.relocate
 ; CHECK: gc.relocate
 ; CHECK: gc.relocate
 ; CHECK: ret i64 addrspace(1)* %reloc
 }
 
 ; This test catches two cases where the verifier was too strict:
 ; 1) A base doesn't need to be relocated if it's never used again
 ; 2) A value can be replaced by one which is known equal.  This
 ; means a potentially derived pointer can be known base and that
 ; we can't check that derived pointer are never bases.
 define void @test2(i8 addrspace(1)* %arg, i64 addrspace(1)* %arg2) gc "statepoint-example" {
 entry:
   %cast = bitcast i8 addrspace(1)* %arg to i64 addrspace(1)*
   %c = icmp eq i64 addrspace(1)* %cast,  %arg2
   br i1 %c, label %equal, label %notequal
 
 notequal:
   ret void
 
 equal:
-  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 5, i32 0, i32 0, i32 0, i32 10, i32 0, i8 addrspace(1)* %arg, i64 addrspace(1)* %cast, i8 addrspace(1)* %arg, i8 addrspace(1)* %arg)
-  %reloc = call i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(i32 %safepoint_token, i32 9, i32 10)
+  %safepoint_token = call i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 0, i32 0, i32 10, i32 0, i8 addrspace(1)* %arg, i64 addrspace(1)* %cast, i8 addrspace(1)* %arg, i8 addrspace(1)* %arg)
+  %reloc = call i64 addrspace(1)* @llvm.experimental.gc.relocate.p1i64(i32 %safepoint_token, i32 10, i32 11)
   call void undef(i64 addrspace(1)* %reloc)
   ret void
 ; CHECK-LABEL: test2
 ; CHECK-LABEL: equal
 ; CHECK: statepoint
 ; CHECK-NEXT: %reloc = call 
 ; CHECK-NEXT: call
 ; CHECK-NEXT: ret voi
 }
 
 ; Basic test for invoke statepoints
 define i8 addrspace(1)* @test3(i8 addrspace(1)* %obj, i8 addrspace(1)* %obj1) gc "statepoint-example" {
 ; CHECK-LABEL: test3
 entry:
   ; CHECK-LABEL: entry
   ; CHECK: statepoint
-  %0 = invoke i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i8 addrspace(1)* %obj, i8 addrspace(1)* %obj1)
+  %0 = invoke i32 (void ()*, i32, i32, ...) @llvm.experimental.gc.statepoint.p0f_isVoidf(void ()* undef, i32 0, i32 0, i32 0, i32 5, i32 0, i32 -1, i32 0, i32 0, i32 0, i8 addrspace(1)* %obj, i8 addrspace(1)* %obj1)
           to label %normal_dest unwind label %exceptional_return
 
 normal_dest:
   ; CHECK-LABEL: normal_dest:
   ; CHECK: gc.relocate
   ; CHECK: gc.relocate
   ; CHECK: ret
-  %obj.relocated = call coldcc i8 addrspace(1)* @llvm.experimental.gc.relocate.p1i8(i32 %0, i32 9, i32 9)
+  %obj.relocated = call coldcc i8 addrspace(1)* @llvm.experimental.gc.relocate.p1i8(i32 %0, i32 10, i32 10)
   %obj1.relocated = call coldcc i8 addrspace(1)* @llvm.experimental.gc.relocate.p1i8(i32 %0, i32 10, i32 10)
   ret i8 addrspace(1)* %obj.relocated
 
 exceptional_return:
   ; CHECK-LABEL: exceptional_return
   ; CHECK: gc.relocate
   ; CHECK: gc.relocate
   %landing_pad = landingpad { i8*, i32 } personality i32 ()* @"personality_function"
           cleanup
   %relocate_token = extractvalue { i8*, i32 } %landing_pad, 1
-  %obj.relocated1 = call coldcc i8 addrspace(1)* @llvm.experimental.gc.relocate.p1i8(i32 %relocate_token, i32 9, i32 9)
+  %obj.relocated1 = call coldcc i8 addrspace(1)* @llvm.experimental.gc.relocate.p1i8(i32 %relocate_token, i32 10, i32 10)
   %obj1.relocated1 = call coldcc i8 addrspace(1)* @llvm.experimental.gc.relocate.p1i8(i32 %relocate_token, i32 10, i32 10)
   ret i8 addrspace(1)* %obj1.relocated1
 }