GlobalISel: first outline of legalization interface.

[t.p.northover]

Summary

This implements a preliminary interface for targets to specify how a given operation should be legalized, with a fewproof of concept legalizations for integer addition.

ISelDAG seems to have overloaded "Expand" to mean quite a few different possible legalization strategies, which I'd like to avoid for GlobalISel. So at the moment, the possible generic options are:

• Legal (nothing to do)
• NarrowScalar (e.g. i64 -> i32 twice, <4 x i64> -> <4 x i32>).
• WidenScalar (e.g. i8 -> i32, <8 x i1> -> <8 x i8>).
• FewerElements (e.g. <4 x i64> -> <2 x i64> twice).
• MoreElements (e.g. <2 x i8> -> <8 x i8>).
• Libcall (self-explanatory).
• Custom (as with ISelDAG, target will handle it).

I am hoping scalarization can be considered a special case of "FewerElements" where the legal number of elements is 1 since the distinction between "<1 x %type>" and "%type" that exists now is a bit sketchy.

For an operation that is illegal on a vector type, we first legalize the underlying scalar (with a separate "setScalarInVectorAction" interface specifying how because, for example, "G_ADD <8 x i8>" may be fine but "G_ADD i8" only representable by promoting to i64). After that we know that the operation is legal for *some* number of vector elements and the second step is to find that.

Some extension will be needed to handle operations with more than one type (for example "i1 = G_ICMP i8 %a, %b"). In this and some other cases what should happen to the result is independent of the operands. I have some vague ideas on this, but nothing concrete.

One other thought, not on the wider picture but the specific G_ADD case in this patch was: just how many G_ADDs do we need/want? For now I've stuck with ADD/ADDC/ADDE, but I'm increasingly convinced that the last two are redundant (ADDC is just "ADDE a, b, 0") and I'm starting to think that it might be better to just have ADDE at the MIR level (with G_ADD being "res, thing<dead> = G_ADDE a, b, 0"). We could still have convenience queries and MIRBuilder methods to produce simple adds.

Any bright ideas or other comments welcome!

Cheers.

Tim.

[arsenm]

include/llvm/CodeGen/GlobalISel/MachineLegalizeInfo.h

25–28

Alphabetize

33–40

These need descriptive comments. Can we also add an unsupported / error on option?

45

Typo ancillary

include/llvm/Target/GenericOpcodes.td

24

Isn't this the default?

30

I think this should be trunc to match the IR instruction name

66

0 indexed src name? Should named operand tables somehow work with the generic opcodes?

75

EXTRACT_SUBREG?

97

REG_SEQUENCE?

[t.p.northover]

All good suggestions (the ones that I understood), Matt. Thanks! I'll
wait around for more comments and fold them into the next iteration.

But I didn't quite get this one:

Should named operand tables somehow work with the generic opcodes?

Do you mean the MCInstrDesc information? Otherwise I'm in the dark, I'm afraid.

Tim.

[arsenm]

In D21534#462684, @t.p.northover wrote:

All good suggestions (the ones that I understood), Matt. Thanks! I'll
wait around for more comments and fold them into the next iteration.

But I didn't quite get this one:

Should named operand tables somehow work with the generic opcodes?

Do you mean the MCInstrDesc information? Otherwise I'm in the dark, I'm afraid.

Tim.

I mean UseNamedOperandTable, to generate enums for accessing the instruction with the TII::getNamedOperandIdx(opcode, Foo::OpName::srcN)

[MatzeB]

ISelDAG seems to have overloaded "Expand" to mean quite a few different possible legalization strategies, which I'd like to avoid for GlobalISel. So at the moment, the possible generic options are:

Legal (nothing to do)
NarrowScalar (e.g. i64 -> i32 twice, <4 x i64> -> <4 x i32>).
WidenScalar (e.g. i8 -> i32, <8 x i1> -> <8 x i8>).
FewerElements (e.g. <4 x i64> -> <2 x i64> twice).
MoreElements (e.g. <2 x i8> -> <8 x i8>).
Libcall (self-explanatory).
Custom (as with ISelDAG, target will handle it).

I always found the SelectionDAG interfaces that basically force you to create a table indexed by type and operation and only give you N pre-selected choices for the table entries. The amount of code we have in our *ISelLowering.cpp files in the custom lowering parts may be an indication that it is...

So how about a radical change here and simply let the targets handle this all in code. This doesn't need to become too tedious in the targets when we provide the right set of convenience functions in generic codegen code. Some pseudocode of what I imagine:

In generic CodeGen/GlobalISel code:

/* LegalizeHelper provides a set of "standard" legalizations in the form of different methods: */
void LegalizeHelper::narrowScalar(MI), LegalizeHelper::libcall(MI) ...
/* LegalizeHelper provides a set of standard query function to categorize operations into different "classes" that are often handled uniformly: */
boolean isIntArithmetic() { return isScalarIntType() && (is_Add || is_Sub || isMul) }, similar things like isVectorArithmetic(), isLoadStore(), ...

In target specific code:

/* No need to inialize a big table with legalization actions in the constructor, just do the case-by-case decisions inside legalize(). The custom legalization code also becomes part of legalize(): */
MyTargetLegalizer::legalize(MI) {
  if (isIntArithmetic(MI)) {
    if (MVTSize < 32)
      widenScalar(MI);
    else if (MVTSize > 32)
      splitScalar(MI, 32);
  }
  // ...
}

While we possible waste some extra cycles in this scheme because of extra if() checks compared to a table lookup. However this scheme:

• Frees us from arguing semantics of the legalization actions enum; Just see what the function call is doing, if it doesn't suit you, add a new function or add a parameter to the legalization function for variations...
• No artificial split between custom and "normal" lowering anymore.
• Adding, renaming, adding parameters to the shared legalization functions keeps us more flexible in the future.

[eli.friedman]

I'm not sure ScalarInVectorActions covers all the possibilities for legalization... for example, x86 prefers to legalize <4 x i16> to <4 x i32> rather than <8 x i16>.

It isn't clear how you plan on dealing with soft-float; for certain operations, like FABS, you probably want a separate "soften" action (so you can convert f64 FABS -> i64 AND -> i32 AND).

You really need to work out what to do with operations with multiple types sooner rather than later, given how common they are. The existing legalization infrastructure basically splits it into multiple steps: first legalize the result type, then legalize each operand type.

For <1 x i64> vs. i64, the distinction is a bit fuzzy... but I'm not sure we can really afford to throw it away. We're currently pretty comfortable with the distinction that <1 x i64> uses vector registers if possible, and i64 doesn't. Doing something different gets very complicated on x86-32 and other similar architectures because we would need to "legalize" i64 operations after register bank assignment. Maybe it's worth doing so we can put values into vector registers more aggressively, but it's a lot more complicated.

You need some legalization action equivalent to "LegalizeDAG", I think. Do you have something planned?

[qcolombet]

Hi Tim,

Thanks for working on this!

I believe this is work in progress so I did not go into great details to comment on each part of the patch. For instance, for a detailed review, I would have expected the patch to be split into smaller one.

First some general comment:

• I would split the file with the legalization APIs in two:
  • one for the machine pass
  • one for the helper

The rationale is to be able to include them independently.

• I would split the LegalizeHelper in two:
  • One with all the methods you added (narrow, widen, etc), but legalize
  • One with the legalize API and the set action stuff

Put differently, I would have the legalizer info being the actual legalizer. We have one per sub target and it knows how to legalize stuff. It uses the LegalizeHelper for the generic action.

• Replace the Type/EVT by something lower level (Cf. my email)

Basically, we do not need to know what are the types, we only need to know their size and there number of lanes, i.e., I would go with a different types representation for the MI level. It is a much bigger change than just for legalization and we might postpone it, but it is something to keep in mind.

• Add unit test cases for the legalize helper.

I think this is it for the high level comments.

More comments below.

Cheers,
-Quentin

include/llvm/CodeGen/GlobalISel/LegalizeMachineIR.h

39

I think we need to say something about what happens to the definition and arguments of MI.
My goal was to have transformations that take legal mir and produce legal mir. For that to be possible, we need to preserve the definition and arguments (with what I called the extract and build_sequence operations in the talk.)
We need to say that somewhere. Moreover, we need to say what happen to the extract and build_sequence operations that are created. Do we legalize them now, do we return a list of them, etc.?
I think the latter is preferable but maybe we want to also provide an option for the former.

Also, please comment on what is the boolean supposed to mean.

include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h

199

I can see how this method will be useful to handle vectors.
Just one comment on the extract instruction. In my mind, the extract instruction was more a extract bitfield like instruction and could produce several definitions.
In other words, the instruction supports the following generic syntax:
<def1>, …, <defN> = extract <use>, <startBitIdx1>, <lengthDef1>, …, <startBitIdxN>, <lengthDefN>

The only requirement, again in my mind, was that the extracted bits do not overlap. The reason why I bring that here is because I think it is easier for the optimizers to reason about only one instruction for things that work on the same input (regbankselect in mind), e.g., gpr, gpr = movrrd fpr naturally fit with this semantic of extract instruction and vice versa. Therefore if we try to legalize only “half” of the extract, we may miss that this pattern is available.

To give something more concret with respect to legalization, say we have:
def1 = extract val, ...
def2 = extract val, …

Let say this is illegal and needs to be lowered in loads and stores. We will likely have:
store val
def1 = load @offset1
store val ; <— redundant store
def2 = load @offset2

And things needs to be cleaned up later.
Now, we the other representation:
def1, def2 = extract val, …
we get
store val
def1 = load @offset1 ; <— loads may be combined
def2 = load @offset2

include/llvm/CodeGen/GlobalISel/MachineLegalizeInfo.h

30

Copy paste :)

51

We’ll actually need more setAction kind of thing.
E.g., right now, there is no way to specify that "fptoint float to i32" is legal whereas "fptoint float to i64" is not, same for truncate, extensions, etc.
What I am saying is that some instructions need both their input and output types to set an action.
Note: This is something broken in SDISel and we need to fix that while we are here. I believe some of the custom lowering happens because of that.

lib/CodeGen/GlobalISel/IRTranslator.cpp

1

Unrelated change.

lib/CodeGen/GlobalISel/MachineIRBuilder.cpp

69

The fact that we were not moving the cursor was actually by design.
If we change that we need to check what is the impact on the existing code.

lib/CodeGen/LLVMTargetMachine.cpp

163

I’d say we need a addPreLegalize hook as well.

[qcolombet]

Hi Matthias,

[arsenm]

include/llvm/CodeGen/GlobalISel/MachineLegalizeInfo.h

51

For memory operations, we really need to be able to specify legality per address space

[t.p.northover]

Hi Quentin,

[t.p.northover]

OK, I've split off the what-should-I-do decision class into a separate review D22074 (as suggested by Quentin) and marked all comments relating to it as done in this review. Feel free to raise them again there if they're not adequately addressed.

Tim.

[t.p.northover]

I'm not uploading an actual diff yet (I'll want to split out more bits into separate reviews I think). But some comment replies:

include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h

199

OK, I'll give it a go. For now I'll limit it to a single (implicit) size since otherwise we'll have to deal with multiple types.

include/llvm/Target/GenericOpcodes.td

66

Existing instructions are indexed by 1. Could do that in a separate refactoring though, I don't particularly feel strongly either way.

lib/CodeGen/GlobalISel/MachineIRBuilder.cpp

69

Well, all the tests still pass. ;)

I think we definitely need to do something so that the default use doesn't end up inserting instructions in reverse order. Maybe make setInstr default to Before = true instead?

[t.p.northover]

I'll upload more modular patches for what's in this diff.

include/llvm/CodeGen/GlobalISel/GISelAccessor.h

 //
 //===----------------------------------------------------------------------===/
 
 #ifndef LLVM_CODEGEN_GLOBALISEL_GISELACCESSOR_H
 #define LLVM_CODEGEN_GLOBALISEL_GISELACCESSOR_H
 
 namespace llvm {
 class CallLowering;
+class MachineLegalizeInfo;
 class RegisterBankInfo;
 
 /// The goal of this helper class is to gather the accessor to all
 /// the APIs related to GlobalISel.
 /// It should be derived to feature an actual accessor to the GISel APIs.
 /// The reason why this is not simply done into the subtarget is to avoid
 /// spreading ifdefs around.
 struct GISelAccessor {
   virtual ~GISelAccessor() {}
   virtual const CallLowering *getCallLowering() const { return nullptr;}
+  virtual const MachineLegalizeInfo *getMachineLegalizeInfo() const {
+    return nullptr;
+  }
   virtual const RegisterBankInfo *getRegBankInfo() const { return nullptr;}
 };
 } // End namespace llvm;
 #endif

include/llvm/CodeGen/GlobalISel/LegalizeMachineIR.h

+//== llvm/CodeGen/GlobalISel/LegalizeMachineIR.h --------------- -*- C++ -*-==//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+/// \file A pass to convert the target-illegal operations created by IR -> MIR
+/// translation into ones the target expects to be able to select. This may
+/// occur in multiple phases, for example G_ADD <2 x i8> -> G_ADD <2 x i16> ->
+/// G_ADD <4 x i16>.
+///
+/// The LegalizeHelper class is where most of the work happens, and is designed
+/// to be callable from other passes that find themselves with an illegal
+/// instruction.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CODEGEN_GLOBALISEL_LEGALIZEMACHINEIR_H
+#define LLVM_CODEGEN_GLOBALISEL_LEGALIZEMACHINEIR_H
+
+#include "llvm/CodeGen/GlobalISel/MachineIRBuilder.h"
+#include "llvm/CodeGen/MachineFunctionPass.h"
+
+namespace llvm {
+// Forward declarations.
+class MachineLegalizeInfo;
+class MachineRegisterInfo;
+
+class LegalizeHelper {
+public:
+  LegalizeHelper(MachineFunction &MF);
+
+  /// Replace MI by a sequence of legal instructions that can implement the same
+  /// operation. Note that this means MI may be deleted, so any iterator steps
+  /// should be performed before calling this function.
+  bool legalizeInstr(MachineInstr &MI);
+
+  /// Legalize an instruction by reducing the width of the underlying scalar
+  /// type.
+  void narrowScalar(MachineInstr &MI, Type *NarrowTy);
+
+  /// Legalize an instruction by performing the operation on a wider scalar type
+  /// (for example a 16-bit addition can be safely performed at 32-bits
+  /// precision, ignoring the unused bits).
+  void widenScalar(MachineInstr &MI, Type *WideTy);
+
+  /// Legalize a vector instruction by splitting into multiple components, each
+  /// acting on the same scalar type as the original but with fewer elements.
+  void fewerElementsVector(MachineInstr &MI, Type *NarrowTy);
+
+  /// Legalize a vector instruction by increasing the number of vector elements
+  /// involved and ignoring the added elements later.
+  void moreElementsVector(MachineInstr &MI, Type *WideTy);
+
+private:
+
+  /// Helper function to split a wide generic register into bitwise blocks with
+  /// the given Type (which implies the number of blocks needed). The generic
+  /// registers created are appended to Ops, starting at bit 0 of Reg.
+  void extractParts(unsigned Reg, Type *Ty, int NumParts,
+                    SmallVectorImpl<unsigned> &Ops);
+
+  MachineIRBuilder MIRBuilder;
+  const MachineLegalizeInfo &MLI;
+  MachineRegisterInfo &MRI;
+};
+
+class LegalizeMachineIR : public MachineFunctionPass {
+public:
+  static char ID;
+
+private:
+
+  /// Initialize the field members using \p MF.
+  void init(MachineFunction &MF);
+
+public:
+  // Ctor, nothing fancy.
+  LegalizeMachineIR();
+
+  const char *getPassName() const override {
+    return "LegalizeMachineIR";
+  }
+
+  bool runOnMachineFunction(MachineFunction &MF) override;
+};
+} // End namespace llvm.
+
+#endif

include/llvm/CodeGen/GlobalISel/MachineIRBuilder.h

public:
 
   /// Build and insert <empty> = \p Opcode <empty>.
   ///
   /// \pre setBasicBlock or setMI must have been called.
   /// \pre not isPreISelGenericOpcode(\p Opcode)
   ///
   /// \return The newly created instruction.
   MachineInstr *buildInstr(unsigned Opcode);
+
+  /// Build and insert \p Res<def> = G_ADD \p Ty \p Op0, \p Op1
+  ///
+  /// G_ADD sets \p Res to the sum of integer parameters \p Op0 and \p Op1,
+  /// truncated to their width.
+  ///
+  /// \pre setBasicBlock or setMI must have been called.
+  ///
+  /// \return The newly created instruction.
+  MachineInstr *buildAdd(Type *Ty, unsigned Res, unsigned Op0, unsigned Op1);
+
+  /// Build and insert \p Res<def>, \p CarryOut = G_ADDC \p Ty \p Op0, \p Op1
+  ///
+  /// G_ADDC sets \p Res to \p Op0 + \p Op1 (truncated to the bit width) and
+  /// sets \p CarryOut to 1 if the result overflowed in 2s-complement
+  /// arithmetic.
+  ///
+  /// \pre setBasicBlock or setMI must have been called.
+  ///
+  /// \return The newly created instruction.
+  MachineInstr *buildAddc(Type *Ty, unsigned Res, unsigned CarryOut,
+                          unsigned Op0, unsigned Op1);
+
+  /// Build and insert \p Res<def>, \p CarryOut = G_ADDE \p Ty \p Op0, \p Op1,
+  /// \p CarryIn
+  ///
+  /// G_ADDE sets \p Res to \p Op0 + \p Op1 + \p CarryIn (truncated to the bit
+  /// width) and sets \p CarryOut to 1 if the result overflowed in 2s-complement
+  /// arithmetic.
+  ///
+  /// \pre setBasicBlock or setMI must have been called.
+  ///
+  /// \return The newly created instruction.
+  MachineInstr *buildAdde(Type *Ty, unsigned Res, unsigned CarryOut,
+                          unsigned Op0, unsigned Op1, unsigned CarryIn);
+
+  /// Build and insert \p Res<def> = G_ANYEXTEND \p Ty \p Op0
+  ///
+  /// G_ANYEXTEND produces a register of the specified width, with bits 0 to
+  /// sizeof(\p Ty) * 8 set to \p Op. The remaining bits are unspecified
+  /// (i.e. this is neither zero nor sign-extension). For a vector register,
+  /// each element is extended individually.
+  ///
+  /// \pre setBasicBlock or setMI must have been called.
+  ///
+  /// \return The newly created instruction.
+  MachineInstr *buildAnyExtend(Type *Ty, unsigned Res, unsigned Op);
+
+  /// Build and insert \p Res<def> = G_EXTRACT \p Ty \p Op0, \p Idx
+  ///
+  /// If \p Ty has size N bits, G_EXTRACT sets \p Res to bits
+  /// [Idx * N, Idx * (N+1)) of \p Op.
+  ///
+  /// \pre setBasicBlock or setMI must have been called.
+  ///
+  /// \return The newly created instruction.
+  MachineInstr *buildExtract(Type *Ty, unsigned Res, unsigned Op, int Idx);
+
+  /// Build and insert \p Res<def> = G_SEQUENCE \p Ty \p Ops[0], ...
+  ///
+  /// G_SEQUENCE concatenates each element in Ops into a single register, where
+  /// Ops[0] starts at bit 0 of \p Res.
+  ///
+  /// \pre setBasicBlock or setMI must have been called.
+  ///
+  /// \return The newly created instruction.
+  MachineInstr *buildSequence(Type *Ty, unsigned Res,
+                              SmallVectorImpl<unsigned> &Ops);
+
+  MachineInstr *buildTruncate(Type *Ty, unsigned Res, unsigned Op);
 };
 
 } // End namespace llvm.
 #endif // LLVM_CODEGEN_GLOBALISEL_MACHINEIRBUILDER_H

include/llvm/CodeGen/GlobalISel/MachineLegalizeInfo.h

+//==-- llvm/CodeGen/GlobalISel/MachineLegalizeInfo.h -------------*- C++ -*-==//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+/// Interface for Targets to specify which operations they can successfully
+/// select and how the others should be expanded most efficiently.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CODEGEN_GLOBALISEL_MACHINELEGALIZEINFO_H
+#define LLVM_CODEGEN_GLOBALISEL_MACHINELEGALIZEINFO_H
+
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/CodeGen/ValueTypes.h"
+
+#include <cstdint>
+#include <functional>
+
+namespace llvm {
+class Type;
+class VectorType;
+class MachineInstr;
+class LLVMContext;
+
+/// Holds all the information related to register banks.
+class MachineLegalizeInfo {
+public:
+  enum LegalizeAction : std::uint8_t {
+    Legal,
+    NarrowScalar,
+    WidenScalar,
+    FewerElements,
+    MoreElements,
+    Libcall,
+    Custom,
+  };
+
+  MachineLegalizeInfo(LLVMContext &Ctx);
+
+  /// Compute any ancilliary tables needed to quickly decide how an operation
+  /// should be handled. This must be called after all "set*Action"methods but
+  /// before any query is made or incorrect results may be returned.
+  void computeTables();
+
+  /// Set how a specific G_* operation should be legalized on a given type.
+  void setAction(unsigned Opcode, Type *Ty, LegalizeAction Action) {
+    Actions[std::make_pair(Opcode, Ty)] = Action;
+  }
+
+  /// More friendly way to set an action for common types that have an EVT
+  /// representation.
+  void setAction(unsigned Opcode, EVT Ty, LegalizeAction Action) {
+    setAction(Opcode, Ty.getTypeForEVT(Ctx), Action);
+  }
+
+  /// If an operation on a given vector type (say <M x iN>) isn't explicitly
+  /// specified, we proceed in 2 stages. First we legalize the underlying scalar
+  /// (so that there's at least one legal vector with that scalar), then we
+  /// adjust the number of elements in the vector so that it is legal. The
+  /// desired action in the first step is controlled by this function.
+  void setScalarInVectorAction(unsigned Opcode, Type *ScalarTy,
+                               LegalizeAction Action) {
+    ScalarInVectorActions[std::make_pair(Opcode, ScalarTy)] = Action;
+  }
+
+  /// Helper function because EVTs are easier to work with.
+  void setScalarInVectorAction(unsigned Opcode, EVT Ty, LegalizeAction Action) {
+    setScalarInVectorAction(Opcode, Ty.getTypeForEVT(Ctx), Action);
+  }
+
+  std::pair<LegalizeAction, Type *> getAction(unsigned Opcode, Type *Ty) const;
+  std::pair<LegalizeAction, Type *> getAction(MachineInstr &MI) const;
+
+  /// Iterate the given function (typically something like doubling the width)
+  /// on Ty until we find a legal type for this operation.
+  Type *findLegalType(unsigned Opcode, Type *Ty,
+                      std::function<Type *(Type *)> NextType) const {
+    LegalizeAction Action;
+    do {
+      Ty = NextType(Ty);
+      auto ActionIt = Actions.find(std::make_pair(Opcode, Ty));
+      if (ActionIt == Actions.end())
+        Action = DefaultActions.find(Opcode)->second;
+      else
+        Action = ActionIt->second;
+    } while(Action != Legal);
+    return Ty;
+  }
+
+  /// Find what type it's actually OK to perform the given operation on, given
+  /// the general approach we've decided to take.
+  Type *findLegalType(unsigned Opcode, Type *Ty, LegalizeAction Action) const;
+
+  bool isLegal(MachineInstr &MI) const;
+
+protected:
+  // Subclasses may want a context to construct fully general llvm::Type
+  // instances for complex situations.
+  LLVMContext &Ctx;
+
+private:
+  typedef DenseMap<std::pair<unsigned, Type *>, LegalizeAction> ActionMap;
+
+  ActionMap Actions;
+  ActionMap ScalarInVectorActions;
+  DenseMap<std::pair<unsigned, Type *>, unsigned> MaxLegalVectorElts;
+  DenseMap<unsigned, LegalizeAction> DefaultActions;
+};
+
+} // End namespace llvm.
+
+#endif

include/llvm/CodeGen/TargetPassConfig.h

public:
   virtual bool addInstSelector() {
     return true;
   }
 
   /// This method should install an IR translator pass, which converts from
   /// LLVM code to machine instructions with possibly generic opcodes.
   virtual bool addIRTranslator() { return true; }
 
+  virtual bool addLegalizeMachineIR() { return true; }
+
   /// This method may be implemented by targets that want to run passes
   /// immediately before the register bank selection.
   virtual void addPreRegBankSelect() {}
 
   /// This method should install a register bank selector pass, which
   /// assigns register banks to virtual registers without a register
   /// class or register banks.
   virtual bool addRegBankSelect() { return true; }

include/llvm/InitializePasses.h

 void initializeLoopInterchangePass(PassRegistry &);
 void initializeLoopLoadEliminationPass(PassRegistry&);
 void initializeLoopPassPass(PassRegistry&);
 void initializeLoopRerollPass(PassRegistry&);
 void initializeLoopRotateLegacyPassPass(PassRegistry&);
 void initializeLoopSimplifyCFGLegacyPassPass(PassRegistry&);
 void initializeLoopSimplifyPass(PassRegistry&);
 void initializeLoopStrengthReducePass(PassRegistry&);
+void initializeGlobalMergePass(PassRegistry&);
+void initializeLegalizeMachineIRPass(PassRegistry&);
+void initializeLoopRerollPass(PassRegistry&);
 void initializeLoopUnrollPass(PassRegistry&);
 void initializeLoopUnswitchPass(PassRegistry&);
 void initializeLoopVectorizePass(PassRegistry&);
 void initializeLoopVersioningLICMPass(PassRegistry&);
 void initializeLoopVersioningPassPass(PassRegistry &);
 void initializeLowerAtomicLegacyPassPass(PassRegistry &);
 void initializeLowerBitSetsPass(PassRegistry&);
 void initializeLowerEmuTLSPass(PassRegistry&);

include/llvm/Target/GenericOpcodes.td

 //===-- GenericOpcodes.td - Opcodes used with GlobalISel ---*- tablegen -*-===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file defines the generic opcodes used with GlobalISel.
 // After instruction selection, these opcodes should not appear.
 //
 //===----------------------------------------------------------------------===//
 
 //------------------------------------------------------------------------------
+// Unary ops.
+//------------------------------------------------------------------------------
+
+// Extend the underlying scalar type of an operation, leaving the high bits
+// unspecified.
+def G_ANYEXTEND : Instruction {
+  let OutOperandList = (outs unknown:$dst);
+  let InOperandList = (ins unknown:$src);
+  let AsmString = "";
+  let hasSideEffects = 0;
+}
+
+// Truncate the underlying scalar type of an operation. This is equivalent to
+// G_EXTRACT for scalar types, but acts elementwise on vectors.
+def G_TRUNCATE : Instruction {
+  let OutOperandList = (outs unknown:$dst);
+  let InOperandList = (ins unknown:$src);
+  let AsmString = "";
+  let hasSideEffects = 0;
+}
+
+//------------------------------------------------------------------------------
 // Binary ops.
 //------------------------------------------------------------------------------
+
 // Generic addition.
 def G_ADD : Instruction {
   let OutOperandList = (outs unknown:$dst);
   let InOperandList = (ins unknown:$src1, unknown:$src2);
   let AsmString = "";
   let hasSideEffects = 0;
   let isCommutable = 1;
 }
 
+// Generic addition producing a carry flag.
+def G_ADDC : Instruction {
+  let OutOperandList = (outs unknown:$dst, unknown:$carry);
+  let InOperandList = (ins unknown:$src1, unknown:$src2);
+  let AsmString = "";
+  let hasSideEffects = 0;
+  let isCommutable = 1;
+}
+
+// Generic addition consuming and producing a carry flag.
+
+// FIXME: I think I'm going to merge this with G_ADDC (so what is now "G_ADDC a,
+// b" would instead be "G_ADDC a, b, 0"). That needs G_IMMEDIATE or similar
+// though and would bloat the initial patch.
+def G_ADDE : Instruction {
+  let OutOperandList = (outs unknown:$dst, unknown:$carry_out);
+  let InOperandList = (ins unknown:$src1, unknown:$src2, unknown:$carry_in);
+  let AsmString = "";
+  let hasSideEffects = 0;
+  let isCommutable = 1;
+}
+
+// Extract bits of the specified size, starting from block given by $idx. This
+// will almost certainly be mapped to a sub-register COPY after register banks
+// have been selected.
+def G_EXTRACT : Instruction {
+  let OutOperandList = (outs unknown:$dst);
+  let InOperandList = (ins unknown:$src, unknown:$idx);
+  let AsmString = "";
+  let hasSideEffects = 0;
+}
+
 // Generic bitwise or.
 def G_OR : Instruction {
   let OutOperandList = (outs unknown:$dst);
   let InOperandList = (ins unknown:$src1, unknown:$src2);
   let AsmString = "";
   let hasSideEffects = 0;
   let isCommutable = 1;
 }
 
 //------------------------------------------------------------------------------
+// Variadic ops
+//------------------------------------------------------------------------------
+
+// Combine a sequence of generic vregs into a single larger value (starting at
+// bit 0).
+def G_SEQUENCE : Instruction {
+  let OutOperandList = (outs unknown:$dst);
+  let InOperandList = (ins variable_ops);
+  let AsmString = "";
+  let hasSideEffects = 0;
+}
+
+
+//------------------------------------------------------------------------------
 // Branches.
 //------------------------------------------------------------------------------
+
 // Generic unconditional branch.
 def G_BR : Instruction {
   let OutOperandList = (outs);
   let InOperandList = (ins unknown:$src1);
   let AsmString = "";
   let hasSideEffects = 0;
   let isBranch = 1;
   let isTerminator = 1;
 }
 
 // TODO: Add the other generic opcodes.

include/llvm/Target/TargetOpcodes.def

 /// The following generic opcodes are not supposed to appear after ISel.
 /// This is something we might want to relax, but for now, this is convenient
 /// to produce diagnostics.
 
 /// Generic ADD instruction. This is an integer add.
 HANDLE_TARGET_OPCODE(G_ADD, 24)
 HANDLE_TARGET_OPCODE_MARKER(PRE_ISEL_GENERIC_OPCODE_START, G_ADD)
 
-/// Generic Bitwise-OR instruction.
-HANDLE_TARGET_OPCODE(G_OR, 25)
+/// Generic ADD instruction, producing two results: the sum and a carry flag.
+HANDLE_TARGET_OPCODE(G_ADDC, 25)
+
+/// Generic ADD instruction, consuming the normal operands plus a carry flag,
+/// and similarly producing the result and a carry flag.
+HANDLE_TARGET_OPCODE(G_ADDE, 26)
 
 /// Generic BRANCH instruction. This is an unconditional branch.
-HANDLE_TARGET_OPCODE(G_BR, 26)
+HANDLE_TARGET_OPCODE(G_BR, 27)
+
+/// Generic Bitwise-OR instruction.
+HANDLE_TARGET_OPCODE(G_OR, 28)
+
+/// Generic extend instruction, where the high bits introduced take unspecified
+/// values (any_ext).
+HANDLE_TARGET_OPCODE(G_ANYEXTEND, 29)
+
+/// Generic instruction to extract a block of bits from the register given
+/// (typically a sub-register COPY after instruction selection).
+HANDLE_TARGET_OPCODE(G_EXTRACT, 30)
+
+/// Generic instruction to paste a variable number of components together into a
+/// larger register.
+HANDLE_TARGET_OPCODE(G_SEQUENCE, 31)
+
+/// Generic instruction to discard the high bits of a register. This differs
+/// from (G_EXTRACT val, 0) on its action on vectors: G_TRUNCATE will truncate
+/// each element individually, G_EXTRACT will typically discard the high
+/// elements of the vector.
+HANDLE_TARGET_OPCODE(G_TRUNCATE, 32)
 
 // TODO: Add more generic opcodes as we move along.
 
 /// Marker for the end of the generic opcode.
 /// This is used to check if an opcode is in the range of the
 /// generic opcodes.
-HANDLE_TARGET_OPCODE_MARKER(PRE_ISEL_GENERIC_OPCODE_END, G_BR)
+HANDLE_TARGET_OPCODE_MARKER(PRE_ISEL_GENERIC_OPCODE_END, G_TRUNCATE)
 
 /// BUILTIN_OP_END - This must be the last enum value in this list.
 /// The target-specific post-isel opcode values start here.
 HANDLE_TARGET_OPCODE_MARKER(GENERIC_OP_END, PRE_ISEL_GENERIC_OPCODE_END)

include/llvm/Target/TargetSubtargetInfo.h

 #include <vector>
 
 namespace llvm {
 
 class CallLowering;
 class DataLayout;
 class MachineFunction;
 class MachineInstr;
+class MachineLegalizeInfo;
 class RegisterBankInfo;
 class SDep;
 class SUnit;
 class TargetFrameLowering;
 class TargetInstrInfo;
 class TargetLowering;
 class TargetRegisterClass;
 class TargetRegisterInfo;
public:
   }
   virtual const CallLowering *getCallLowering() const { return nullptr; }
   /// Target can subclass this hook to select a different DAG scheduler.
   virtual RegisterScheduler::FunctionPassCtor
       getDAGScheduler(CodeGenOpt::Level) const {
     return nullptr;
   }
 
+  virtual const MachineLegalizeInfo *getMachineLegalizeInfo() const {
+    return nullptr;
+  }
+
   /// getRegisterInfo - If register information is available, return it.  If
   /// not, return null.
   ///
   virtual const TargetRegisterInfo *getRegisterInfo() const { return nullptr; }
 
   /// If the information for the register banks is available, return it.
   /// Otherwise return nullptr.
   virtual const RegisterBankInfo *getRegBankInfo() const { return nullptr; }

lib/CodeGen/GlobalISel/CMakeLists.txt

 # List of all GlobalISel files.
 set(GLOBAL_ISEL_FILES
       IRTranslator.cpp
+      LegalizeMachineIR.cpp
       MachineIRBuilder.cpp
+      MachineLegalizeInfo.cpp
       RegBankSelect.cpp
       RegisterBank.cpp
       RegisterBankInfo.cpp
       )
 
 # Add GlobalISel files to the dependencies if the user wants to build it.
 if(LLVM_BUILD_GLOBAL_ISEL)
   set(GLOBAL_ISEL_BUILD_FILES ${GLOBAL_ISEL_FILES})

lib/CodeGen/GlobalISel/GlobalISel.cpp

 
 void llvm::initializeGlobalISel(PassRegistry &Registry) {
 }
 
 #else
 
 void llvm::initializeGlobalISel(PassRegistry &Registry) {
   initializeIRTranslatorPass(Registry);
+  initializeLegalizeMachineIRPass(Registry);
   initializeRegBankSelectPass(Registry);
 }
 #endif // LLVM_BUILD_GLOBAL_ISEL

lib/CodeGen/GlobalISel/IRTranslator.cpp

-//===-- llvm/CodeGen/GlobalISel/IRTranslator.cpp - IRTranslator --*- C++ -*-==//
+//===--- lib/CodeGen/GlobalISel/IRTranslator.cpp - IRTranslator --*- C++ -*-==//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 /// \file

lib/CodeGen/GlobalISel/LegalizeMachineIR.cpp

+//===-- llvm/CodeGen/GlobalISel/LegalizeMachineIR.cpp - LegalizeMachineIR -===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+/// \file This file implements the LegalizeHelper class to legalize individual
+/// instructions and the LegalizeMachineIR wrapper pass for the primary
+/// legalization.
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/CodeGen/MachineRegisterInfo.h"
+#include "llvm/CodeGen/GlobalISel/LegalizeMachineIR.h"
+#include "llvm/CodeGen/GlobalISel/MachineLegalizeInfo.h"
+#include "llvm/Support/Debug.h"
+#include "llvm/Target/TargetSubtargetInfo.h"
+
+#define DEBUG_TYPE "legalize-mir"
+
+using namespace llvm;
+
+LegalizeHelper::LegalizeHelper(MachineFunction &MF)
+    : MLI(*MF.getSubtarget().getMachineLegalizeInfo()), MRI(MF.getRegInfo()) {
+  MIRBuilder.setMF(MF);
+}
+
+void LegalizeHelper::extractParts(unsigned Reg, Type *Ty, int NumParts,
+                                  SmallVectorImpl<unsigned> &VRegs) {
+  unsigned Size = Ty->getPrimitiveSizeInBits();
+  for (int i = 0; i < NumParts; ++i) {
+    unsigned PartReg = MRI.createGenericVirtualRegister(Size);
+    MIRBuilder.buildExtract(Ty, PartReg, Reg, i);
+    VRegs.push_back(PartReg);
+  }
+}
+
+bool LegalizeHelper::legalizeInstr(MachineInstr &MI) {
+  auto Action = MLI.getAction(MI);
+  switch (Action.first) {
+  case MachineLegalizeInfo::Legal:
+    return false;
+  case MachineLegalizeInfo::NarrowScalar:
+    narrowScalar(MI, Action.second);
+    break;
+  case MachineLegalizeInfo::WidenScalar:
+    widenScalar(MI, Action.second);
+    break;
+  case MachineLegalizeInfo::FewerElements:
+    fewerElementsVector(MI, Action.second);
+    break;
+  default:
+    llvm_unreachable("don't know how to do that yet");
+  }
+
+  return true;
+}
+
+void LegalizeHelper::narrowScalar(MachineInstr &MI, Type *NarrowTy) {
+  switch (MI.getOpcode()) {
+  default:
+    llvm_unreachable("Cannot legalize this instruction");
+  case TargetOpcode::G_ADD: {
+    // Expand in terms of carry-setting/consuming G_ADDC/G_ADDE instructions.
+    unsigned NarrowSize = NarrowTy->getPrimitiveSizeInBits();
+    int NumParts = MI.getType()->getPrimitiveSizeInBits() / NarrowSize;
+
+    MIRBuilder.setInstr(MI, false);
+
+    SmallVector<unsigned, 2> Src1Regs, Src2Regs, DstRegs;
+    extractParts(MI.getOperand(1).getReg(), NarrowTy, NumParts, Src1Regs);
+    extractParts(MI.getOperand(2).getReg(), NarrowTy, NumParts, Src2Regs);
+
+    unsigned DstReg = MRI.createGenericVirtualRegister(NarrowSize);
+    unsigned CarryIn =  MRI.createGenericVirtualRegister(1);
+    MIRBuilder.buildAddc(NarrowTy, DstReg, CarryIn, Src1Regs[0], Src2Regs[0]);
+    DstRegs.push_back(DstReg);
+
+    for (int i = 1; i < NumParts; ++i) {
+      unsigned DstReg = MRI.createGenericVirtualRegister(NarrowSize);
+      unsigned CarryOut =  MRI.createGenericVirtualRegister(1);
+
+      MIRBuilder.buildAdde(NarrowTy, DstReg, CarryOut,
+                           Src1Regs[i], Src2Regs[i], CarryIn);
+
+      DstRegs.push_back(DstReg);
+      CarryIn = CarryOut;
+    }
+    MIRBuilder.buildSequence(MI.getType(), MI.getOperand(0).getReg(), DstRegs);
+    MI.eraseFromParent();
+    break;
+  }
+  }
+}
+
+void LegalizeHelper::widenScalar(MachineInstr &MI, Type *WideTy) {
+  switch (MI.getOpcode()) {
+  default:
+    llvm_unreachable("Cannot legalize this instruction");
+  case TargetOpcode::G_ADD: {
+    // Perform operation at larger width (any extension is fine here, high bits
+    // don't affect the result) and then truncate the result back to the
+    // original type.
+    unsigned WideSize = WideTy->getPrimitiveSizeInBits();
+
+    MIRBuilder.setInstr(MI, false);
+
+    unsigned Src1Ext = MRI.createGenericVirtualRegister(WideSize);
+    unsigned Src2Ext = MRI.createGenericVirtualRegister(WideSize);
+    MIRBuilder.buildAnyExtend(WideTy, Src1Ext, MI.getOperand(1).getReg());
+    MIRBuilder.buildAnyExtend(WideTy, Src2Ext, MI.getOperand(2).getReg());
+
+    unsigned DstExt = MRI.createGenericVirtualRegister(WideSize);
+    MIRBuilder.buildAdd(WideTy, DstExt, Src1Ext, Src2Ext);
+
+    MIRBuilder.buildTruncate(MI.getType(), MI.getOperand(0).getReg(), DstExt);
+    MI.eraseFromParent();
+    break;
+  }
+  }
+}
+
+void LegalizeHelper::fewerElementsVector(MachineInstr &MI, Type *NarrowTy) {
+  switch (MI.getOpcode()) {
+  default:
+    llvm_unreachable("Cannot legalize this instruction");
+  case TargetOpcode::G_ADD: {
+    unsigned NarrowSize = NarrowTy->getPrimitiveSizeInBits();
+    int NumParts = MI.getType()->getPrimitiveSizeInBits() / NarrowSize;
+
+    MIRBuilder.setInstr(MI, false);
+
+    SmallVector<unsigned, 2> Src1Regs, Src2Regs, DstRegs;
+    extractParts(MI.getOperand(1).getReg(), NarrowTy, NumParts, Src1Regs);
+    extractParts(MI.getOperand(2).getReg(), NarrowTy, NumParts, Src2Regs);
+
+    for (int i = 0; i < NumParts; ++i) {
+      unsigned DstReg = MRI.createGenericVirtualRegister(NarrowSize);
+      MIRBuilder.buildAdd(NarrowTy, DstReg, Src1Regs[i], Src2Regs[i]);
+      DstRegs.push_back(DstReg);
+    }
+
+    MIRBuilder.buildSequence(MI.getType(), MI.getOperand(0).getReg(), DstRegs);
+    MI.eraseFromParent();
+    break;
+  }
+  }
+}
+
+char LegalizeMachineIR::ID = 0;
+INITIALIZE_PASS(LegalizeMachineIR, "legalize-mir",
+                "Legalize the Machine IR a function's Machine IR", false,
+                false);
+
+LegalizeMachineIR::LegalizeMachineIR() : MachineFunctionPass(ID) {
+  initializeLegalizeMachineIRPass(*PassRegistry::getPassRegistry());
+}
+
+void LegalizeMachineIR::init(MachineFunction &MF) {
+}
+
+bool LegalizeMachineIR::runOnMachineFunction(MachineFunction &MF) {
+  DEBUG(dbgs() << "Legalize Machine IR for: " << MF.getName() << '\n');
+  init(MF);
+  LegalizeHelper Helper(MF);
+
+  // FIXME: an instruction may need more than one pass before it is legal. For
+  // example on most architectures <3 x i3> is doubly-illegal. It would
+  // typically proceed along a path like: <3 x i3> -> <3 x i8> -> <8 x i8>. We
+  // probably want a worklist of instructions rather than naive iterate until
+  // convergence for performance reasons.
+  bool Changed = false;
+  MachineBasicBlock::iterator NextMI;
+  for (auto &MBB : MF)
+    for (auto MI = MBB.begin(); MI != MBB.end(); MI = NextMI) {
+      // Get the next Instruction before we try to legalize, because there's a
+      // good chance MI will be deleted.
+      NextMI = std::next(MI);
+      Changed |= Helper.legalizeInstr(*MI);
+    }
+
+  return Changed;
+}

lib/CodeGen/GlobalISel/MachineIRBuilder.cpp

MachineInstr *MachineIRBuilder::buildInstr(unsigned Opcode, Type *Ty) {
   if (Ty) {
     assert(isPreISelGenericOpcode(Opcode) &&
            "Only generic instruction can have a type");
     NewMI->setType(Ty);
   } else
     assert(!isPreISelGenericOpcode(Opcode) &&
            "Generic instruction must have a type");
   getMBB().insert(getInsertPt(), NewMI);
+  setInstr(*NewMI, false);
   return NewMI;
 }
 
 MachineInstr *MachineIRBuilder::buildInstr(unsigned Opcode, unsigned Res,
                                            unsigned Op0, unsigned Op1) {
   return buildInstr(Opcode, nullptr, Res, Op0, Op1);
 }
 

 }
 
 MachineInstr *MachineIRBuilder::buildInstr(unsigned Opcode, Type *Ty,
                                            MachineBasicBlock &BB) {
   MachineInstr *NewMI = buildInstr(Opcode, Ty);
   MachineInstrBuilder(getMF(), NewMI).addMBB(&BB);
   return NewMI;
 }
+
+MachineInstr *MachineIRBuilder::buildAdd(Type *Ty, unsigned Res, unsigned Op0,
+                                         unsigned Op1) {
+  return buildInstr(TargetOpcode::G_ADD, Ty, Res, Op0, Op1);
+}
+
+MachineInstr *MachineIRBuilder::buildAddc(Type *Ty, unsigned Res,
+                                          unsigned CarryOut, unsigned Op0,
+                                          unsigned Op1) {
+  MachineInstr *NewMI = buildInstr(TargetOpcode::G_ADDC, Ty);
+  MachineInstrBuilder(getMF(), NewMI)
+      .addReg(Res, RegState::Define)
+      .addReg(CarryOut, RegState::Define)
+      .addReg(Op0)
+      .addReg(Op1);
+  return NewMI;
+}
+
+MachineInstr *MachineIRBuilder::buildAdde(Type *Ty, unsigned Res,
+                                          unsigned CarryOut, unsigned Op0,
+                                          unsigned Op1, unsigned CarryIn) {
+  MachineInstr *NewMI = buildInstr(TargetOpcode::G_ADDE, Ty);
+  MachineInstrBuilder(getMF(), NewMI)
+      .addReg(Res, RegState::Define)
+      .addReg(CarryOut, RegState::Define)
+      .addReg(Op0)
+      .addReg(Op1)
+      .addReg(CarryIn);
+  return NewMI;
+}
+
+MachineInstr *MachineIRBuilder::buildAnyExtend(Type *Ty, unsigned Res,
+                                            unsigned Op) {
+  MachineInstr *NewMI = buildInstr(TargetOpcode::G_ANYEXTEND, Ty);
+  MachineInstrBuilder(getMF(), NewMI)
+      .addReg(Res, RegState::Define)
+      .addReg(Op);
+  return NewMI;
+}
+
+MachineInstr *MachineIRBuilder::buildExtract(Type *Ty, unsigned Res,
+                                             unsigned Op, int Idx) {
+  MachineInstr *NewMI = buildInstr(TargetOpcode::G_EXTRACT, Ty);
+  MachineInstrBuilder(getMF(), NewMI)
+      .addReg(Res, RegState::Define)
+      .addReg(Op)
+      .addImm(Idx);
+  return NewMI;
+}
+
+MachineInstr *MachineIRBuilder::buildSequence(Type *Ty, unsigned Res,
+                                              SmallVectorImpl<unsigned> &Ops) {
+  MachineInstr *NewMI = buildInstr(TargetOpcode::G_SEQUENCE, Ty);
+  auto MIB = MachineInstrBuilder(getMF(), NewMI);
+  MIB.addReg(Res, RegState::Define);
+  for (auto Op : Ops)
+    MIB.addReg(Op);
+  return NewMI;
+}
+
+MachineInstr *MachineIRBuilder::buildTruncate(Type *Ty, unsigned Res,
+                                              unsigned Op) {
+  MachineInstr *NewMI = buildInstr(TargetOpcode::G_TRUNCATE, Ty);
+  MachineInstrBuilder(getMF(), NewMI)
+      .addReg(Res, RegState::Define)
+      .addReg(Op);
+  return NewMI;
+}

lib/CodeGen/GlobalISel/MachineLegalizeInfo.cpp

+//===---- lib/CodeGen/GlobalISel/MachineLegalizeInfo.cpp - IRTranslator ----==//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// Implement an interface to specify and query how an illegal operation on a
+// given type should be expanded.
+//
+// Issues to be resolved:
+//   + Make it fast.
+//   + Support weird types like i3, <7 x i3>, ...
+//   + Operations with more than one type (ICMP, CMPXCHG, intrinsics, ...)
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/CodeGen/MachineInstr.h"
+#include "llvm/CodeGen/ValueTypes.h"
+#include "llvm/CodeGen/GlobalISel/MachineLegalizeInfo.h"
+#include "llvm/IR/Type.h"
+#include "llvm/Target/TargetOpcodes.h"
+using namespace llvm;
+
+MachineLegalizeInfo::MachineLegalizeInfo(LLVMContext &Ctx) : Ctx(Ctx) {
+  // FIXME: these two can be legalized to the fundamental load/store Jakob
+  // proposed. Once loads & stores are supported.
+  DefaultActions[TargetOpcode::G_ANYEXTEND] = Legal;
+  DefaultActions[TargetOpcode::G_TRUNCATE] = Legal;
+
+  DefaultActions[TargetOpcode::G_ADD] = NarrowScalar;
+}
+
+void MachineLegalizeInfo::computeTables() {
+  for (auto &Op : Actions) {
+    VectorType *Ty = dyn_cast<VectorType>(Op.first.second);
+    if (!Ty)
+      continue;
+
+    auto &Entry = MaxLegalVectorElts[std::make_pair(Op.first.first,
+                                                    Ty->getElementType())];
+    Entry = std::max(Entry, Ty->getNumElements());
+  }
+}
+
+// FIXME: inefficient implementation for now. Without ComputeValueVTs we're
+// probably going to need specialized lookup structures for various types before
+// we have any hope of doing well with something like <13 x i3>. Even the common
+// cases should do better than what we have now.
+std::pair<MachineLegalizeInfo::LegalizeAction, Type *>
+MachineLegalizeInfo::getAction(unsigned Opcode, Type *Ty) const {
+  // These *have* to be implemented for now, they're the fundamental basis of
+  // how everything else is transformed.
+
+  // FIXME: the long-term plan calls for expansion in terms of load/store (if
+  // they're not legal).
+  if (Opcode == TargetOpcode::G_SEQUENCE || Opcode == TargetOpcode::G_EXTRACT)
+    return std::make_pair(Legal, Ty);
+
+  auto ActionIt = Actions.find(std::make_pair(Opcode, Ty));
+  if (ActionIt != Actions.end())
+    return std::make_pair(ActionIt->second,
+                          findLegalType(Opcode, Ty, ActionIt->second));
+  else if (!Ty->isVectorTy()) {
+    auto DefaultAction = DefaultActions.find(Opcode);
+    if (DefaultAction != DefaultActions.end() && DefaultAction->second == Legal)
+      return std::make_pair(Legal, Ty);
+
+    assert(DefaultAction->second == NarrowScalar && "unexpected default");
+    return std::make_pair(NarrowScalar,
+                          findLegalType(Opcode, Ty, NarrowScalar));
+  }
+
+  VectorType *VTy = cast<VectorType>(Ty);
+  Type *EltTy = VTy->getElementType();
+  unsigned NumElts = VTy->getNumElements();
+
+  auto ScalarAction = ScalarInVectorActions.find(std::make_pair(Opcode, EltTy));
+  if (ScalarAction != ScalarInVectorActions.end() &&
+      ScalarAction->second != Legal)
+    return std::make_pair(ScalarAction->second,
+                          findLegalType(Opcode, EltTy, ScalarAction->second));
+  else if (ScalarAction != ScalarInVectorActions.end()) {
+    assert(ScalarAction->second == FewerElements);
+    return std::make_pair(FewerElements,
+                          findLegalType(Opcode, VTy, FewerElements));
+  }
+
+  // The element type is legal in principle, but the number of elements is
+  // wrong.
+  auto MaxLegalElts = MaxLegalVectorElts.lookup(std::make_pair(Opcode, EltTy));
+  if (MaxLegalElts > NumElts)
+    return std::make_pair(MoreElements, VectorType::get(EltTy, MaxLegalElts));
+  else if (MaxLegalElts == 0)
+    return std::make_pair(FewerElements, EltTy);
+
+  return std::make_pair(FewerElements, findLegalType(Opcode, Ty, FewerElements));
+}
+
+std::pair<MachineLegalizeInfo::LegalizeAction, Type *>
+MachineLegalizeInfo::getAction(MachineInstr &MI) const {
+  return getAction(MI.getOpcode(), MI.getType());
+}
+
+bool MachineLegalizeInfo::isLegal(MachineInstr &MI) const {
+  return getAction(MI).first == Legal;
+}
+
+Type *MachineLegalizeInfo::findLegalType(unsigned Opcode, Type *Ty,
+                                         LegalizeAction Action) const {
+  switch(Action) {
+  default:
+    llvm_unreachable("Cannot find legal type");
+  case Legal:
+    return Ty;
+  case NarrowScalar: {
+    assert(Ty->isIntegerTy());
+    unsigned Width = Ty->getPrimitiveSizeInBits();
+    return findLegalType(Opcode, Ty, [&](Type *Ty) -> Type * {
+      Width /= 2;
+      return IntegerType::get(Ctx, Width);
+    });
+  }
+  case WidenScalar: {
+    assert(Ty->isIntegerTy());
+    unsigned Width = Ty->getPrimitiveSizeInBits();
+    return findLegalType(Opcode, Ty, [&](Type *Ty) -> Type * {
+      Width *= 2;
+      return IntegerType::get(Ctx, Width);
+    });
+  }
+  case FewerElements: {
+    Type *EltTy = Ty->getVectorElementType();
+    unsigned NumElts = Ty->getVectorNumElements();
+    return findLegalType(Opcode, Ty, [&](Type *Ty) -> Type * {
+      NumElts /= 2;
+      return VectorType::get(EltTy, NumElts);
+    });
+  }
+  case MoreElements: {
+    Type *EltTy = Ty->getVectorElementType();
+    unsigned NumElts = Ty->getVectorNumElements();
+    return findLegalType(Opcode, Ty, [&](Type *Ty) -> Type * {
+      NumElts *= 2;
+      return VectorType::get(EltTy, NumElts);
+    });
+  }
+  }
+}

lib/CodeGen/LLVMTargetMachine.cpp

if (EnableFastISelOption == cl::BOU_TRUE ||
        TM->getO0WantsFastISel()))
     TM->setFastISel(true);
 
   // Ask the target for an isel.
   if (LLVM_UNLIKELY(EnableGlobalISel)) {
     if (PassConfig->addIRTranslator())
       return nullptr;
 
+    PassConfig->addLegalizeMachineIR();
+
     // Before running the register bank selector, ask the target if it
     // wants to run some passes.
     PassConfig->addPreRegBankSelect();
 
     if (PassConfig->addRegBankSelect())
       return nullptr;
 
   } else if (PassConfig->addInstSelector())

lib/Target/AArch64/AArch64MachineLegalizeInfo.h

+//===- AArch64MachineLegalizeInfo --------------------------------*- C++ -*-==//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+/// \file
+/// This file declares the targeting of the MachineLegalizeInfo class for
+/// AArch64.
+/// \todo This should be generated by TableGen.
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_TARGET_AARCH64_AARCH64MACHINELEGALIZEINFO_H
+#define LLVM_LIB_TARGET_AARCH64_AARCH64MACHINELEGALIZEINFO_H
+
+#include "llvm/CodeGen/GlobalISel/MachineLegalizeInfo.h"
+
+namespace llvm {
+
+class LLVMContext;
+
+/// This class provides the information for the target register banks.
+class AArch64MachineLegalizeInfo : public MachineLegalizeInfo {
+public:
+  AArch64MachineLegalizeInfo(LLVMContext &Ctx);
+};
+} // End llvm namespace.
+#endif

lib/Target/AArch64/AArch64MachineLegalizeInfo.cpp

+//===- AArch64MachineLegalizeInfo.cpp -------------------------------*- C++ -*-==//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+/// \file
+/// This file implements the targeting of the MachineLegalizeInfo class for
+/// AArch64.
+/// \todo This should be generated by TableGen.
+//===----------------------------------------------------------------------===//
+
+#include "AArch64MachineLegalizeInfo.h"
+#include "llvm/CodeGen/ValueTypes.h"
+#include "llvm/IR/Type.h"
+#include "llvm/IR/DerivedTypes.h"
+#include "llvm/Target/TargetOpcodes.h"
+
+using namespace llvm;
+
+#ifndef LLVM_BUILD_GLOBAL_ISEL
+#error "You shouldn't build this"
+#endif
+
+AArch64MachineLegalizeInfo::AArch64MachineLegalizeInfo(LLVMContext &Ctx)
+    : MachineLegalizeInfo(Ctx) {
+  // We have 32-bit and 64-bit scalar adds...
+  setAction(TargetOpcode::G_ADD, MVT::i8, WidenScalar);
+  setAction(TargetOpcode::G_ADD, MVT::i16, WidenScalar);
+  setAction(TargetOpcode::G_ADD, MVT::i32, Legal);
+  setAction(TargetOpcode::G_ADD, MVT::i64, Legal);
+
+  // ... and
+  setScalarInVectorAction(TargetOpcode::G_ADD, MVT::i8, Legal);
+  setScalarInVectorAction(TargetOpcode::G_ADD, MVT::i16, Legal);
+  setScalarInVectorAction(TargetOpcode::G_ADD, MVT::i32, Legal);
+
+  setAction(TargetOpcode::G_ADD, MVT::v2i64, Legal);
+  computeTables();
+}

lib/Target/AArch64/AArch64Subtarget.h

public:
   const AArch64TargetLowering *getTargetLowering() const override {
     return &TLInfo;
   }
   const AArch64InstrInfo *getInstrInfo() const override { return &InstrInfo; }
   const AArch64RegisterInfo *getRegisterInfo() const override {
     return &getInstrInfo()->getRegisterInfo();
   }
   const CallLowering *getCallLowering() const override;
+  const MachineLegalizeInfo *getMachineLegalizeInfo() const override;
   const RegisterBankInfo *getRegBankInfo() const override;
   const Triple &getTargetTriple() const { return TargetTriple; }
   bool enableMachineScheduler() const override { return true; }
   bool enablePostRAScheduler() const override {
     return UsePostRAScheduler;
   }
 
   /// Returns ARM processor family.

lib/Target/AArch64/AArch64Subtarget.cpp

: AArch64GenSubtargetInfo(TT, CPU, FS), ReserveX18(TT.isOSDarwin()),
       InstrInfo(initializeSubtargetDependencies(FS)), TSInfo(),
       TLInfo(TM, *this), GISel() {}
 
 const CallLowering *AArch64Subtarget::getCallLowering() const {
   assert(GISel && "Access to GlobalISel APIs not set");
   return GISel->getCallLowering();
 }
 
+const MachineLegalizeInfo *AArch64Subtarget::getMachineLegalizeInfo() const {
+  assert(GISel && "Access to GlobalISel APIs not set");
+  return GISel->getMachineLegalizeInfo();
+}
+
 const RegisterBankInfo *AArch64Subtarget::getRegBankInfo() const {
   assert(GISel && "Access to GlobalISel APIs not set");
   return GISel->getRegBankInfo();
 }
 
 /// Find the target operand flags that describe how a global value should be
 /// referenced for the current subtarget.
 unsigned char

lib/Target/AArch64/AArch64TargetMachine.cpp

 //===-- AArch64TargetMachine.cpp - Define TargetMachine for AArch64 -------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 //
 //===----------------------------------------------------------------------===//
 
 #include "AArch64.h"
 #include "AArch64CallLowering.h"
+#include "AArch64MachineLegalizeInfo.h"
 #include "AArch64RegisterBankInfo.h"
 #include "AArch64TargetMachine.h"
 #include "AArch64TargetObjectFile.h"
 #include "AArch64TargetTransformInfo.h"
 #include "llvm/CodeGen/GlobalISel/IRTranslator.h"
+#include "llvm/CodeGen/GlobalISel/LegalizeMachineIR.h"
 #include "llvm/CodeGen/GlobalISel/RegBankSelect.h"
 #include "llvm/CodeGen/Passes.h"
 #include "llvm/CodeGen/RegAllocRegistry.h"
 #include "llvm/CodeGen/TargetPassConfig.h"
 #include "llvm/IR/Function.h"
 #include "llvm/IR/LegacyPassManager.h"
 #include "llvm/InitializePasses.h"
 #include "llvm/Support/CommandLine.h"

 }
 
 AArch64TargetMachine::~AArch64TargetMachine() {}
 
 #ifdef LLVM_BUILD_GLOBAL_ISEL
 namespace {
 struct AArch64GISelActualAccessor : public GISelAccessor {
   std::unique_ptr<CallLowering> CallLoweringInfo;
+  std::unique_ptr<MachineLegalizeInfo> MachineLegalizeInfo;
   std::unique_ptr<RegisterBankInfo> RegBankInfo;
   const CallLowering *getCallLowering() const override {
     return CallLoweringInfo.get();
   }
+  const class MachineLegalizeInfo *getMachineLegalizeInfo() const override {
+    return MachineLegalizeInfo.get();
+  }
   const RegisterBankInfo *getRegBankInfo() const override {
     return RegBankInfo.get();
   }
 };
 } // End anonymous namespace.
 #endif
 
 const AArch64Subtarget *
I = llvm::make_unique<AArch64Subtarget>(TargetTriple, CPU, FS, *this,
                                             Subtarget.isLittleEndian());
 #ifndef LLVM_BUILD_GLOBAL_ISEL
    GISelAccessor *GISel = new GISelAccessor();
 #else
     AArch64GISelActualAccessor *GISel =
         new AArch64GISelActualAccessor();
     GISel->CallLoweringInfo.reset(
         new AArch64CallLowering(*I->getTargetLowering()));
+    GISel->MachineLegalizeInfo.reset(
+        new AArch64MachineLegalizeInfo(F.getContext()));
     GISel->RegBankInfo.reset(
         new AArch64RegisterBankInfo(*I->getRegisterInfo()));
 #endif
     I->setGISelAccessor(*GISel);
   }
   return I.get();
 }
 
AArch64TargetMachine &getAArch64TargetMachine() const {
     return getTM<AArch64TargetMachine>();
   }
 
   void addIRPasses()  override;
   bool addPreISel() override;
   bool addInstSelector() override;
 #ifdef LLVM_BUILD_GLOBAL_ISEL
   bool addIRTranslator() override;
+  bool addLegalizeMachineIR() override;
   bool addRegBankSelect() override;
 #endif
   bool addILPOpts() override;
   void addPreRegAlloc() override;
   void addPostRegAlloc() override;
   void addPreSched2() override;
   void addPreEmitPass() override;
 };
bool AArch64PassConfig::addInstSelector() {
   return false;
 }
 
 #ifdef LLVM_BUILD_GLOBAL_ISEL
 bool AArch64PassConfig::addIRTranslator() {
   addPass(new IRTranslator());
   return false;
 }
+bool AArch64PassConfig::addLegalizeMachineIR() {
+  addPass(new LegalizeMachineIR());
+  return false;
+}
 bool AArch64PassConfig::addRegBankSelect() {
   addPass(new RegBankSelect());
   return false;
 }
 #endif
 
 bool AArch64PassConfig::addILPOpts() {
   if (EnableCondOpt)

lib/Target/AArch64/CMakeLists.txt

 tablegen(LLVM AArch64GenCallingConv.inc -gen-callingconv)
 tablegen(LLVM AArch64GenSubtargetInfo.inc -gen-subtarget)
 tablegen(LLVM AArch64GenDisassemblerTables.inc -gen-disassembler)
 add_public_tablegen_target(AArch64CommonTableGen)
 
 # List of all GlobalISel files.
 set(GLOBAL_ISEL_FILES
       AArch64CallLowering.cpp
+      AArch64MachineLegalizeInfo.cpp
       AArch64RegisterBankInfo.cpp
       )
 
 # Add GlobalISel files to the dependencies if the user wants to build it.
 if(LLVM_BUILD_GLOBAL_ISEL)
   set(GLOBAL_ISEL_BUILD_FILES ${GLOBAL_ISEL_FILES})
 else()
   set(GLOBAL_ISEL_BUILD_FILES"")

test/CodeGen/AArch64/GlobalISel/legalize-add.mir

+# RUN: llc -O0 -run-pass=legalize-mir -global-isel %s -o - 2>&1 | FileCheck %s
+# REQUIRES: global-isel
+
+--- |
+  ; ModuleID = 'generic-virtual-registers-type-error.mir'
+  target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
+  target triple = "aarch64-apple-ios"
+  define void @test_scalar_add_big() {
+  entry:
+    ret void
+  }
+  define void @test_scalar_add_small() {
+  entry:
+    ret void
+  }
+  define void @test_vector_add() {
+  entry:
+    ret void
+  }
+...
+
+---
+# Check that we assign a relevant register bank for %0.
+# Based on the type i32, this should be gpr.
+name:            test_scalar_add_big
+isSSA:           true
+registers:
+  - { id: 0, class: _ }
+  - { id: 1, class: _ }
+  - { id: 2, class: _ }
+body: |
+  bb.0.entry:
+    liveins: %x0, %x1, %x2, %x3
+    ; CHECK-LABEL: name: test_scalar_add_big
+    ; CHECK-DAG: [[LHS_LO:%.*]](64) = G_EXTRACT i64 %0, 0
+    ; CHECK-DAG: [[LHS_HI:%.*]](64) = G_EXTRACT i64 %0, 1
+    ; CHECK-DAG: [[RHS_LO:%.*]](64) = G_EXTRACT i64 %1, 0
+    ; CHECK-DAG: [[RHS_HI:%.*]](64) = G_EXTRACT i64 %1, 1
+    ; CHECK: [[RES_LO:%.*]](64), [[CARRY:%.*]](1) = G_ADDC i64 [[LHS_LO]], [[RHS_LO]]
+    ; CHECK: [[RES_HI:%.*]](64), {{%.*}}(1) = G_ADDE i64 [[LHS_HI]], [[RHS_HI]], [[CARRY]]
+    ; CHECK: %2(128) = G_SEQUENCE i128 [[RES_LO]], [[RES_HI]]
+
+    %0(128) = G_SEQUENCE i128 %x0, %x1
+    %1(128) = G_SEQUENCE i128 %x2, %x3
+    %2(128) = G_ADD i128 %0, %1
+    %x0 = G_EXTRACT i64 %2, 0
+    %x1 = G_EXTRACT i64 %2, 1
+...
+
+---
+# Check that we assign a relevant register bank for %0.
+# Based on the type i32, this should be gpr.
+name:            test_scalar_add_small
+isSSA:           true
+registers:
+  - { id: 0, class: _ }
+  - { id: 1, class: _ }
+  - { id: 2, class: _ }
+body: |
+  bb.0.entry:
+    liveins: %x0, %x1, %x2, %x3
+    ; CHECK-LABEL: name: test_scalar_add_small
+    ; CHECK-DAG: [[LHS:%.*]](32) = G_ANYEXTEND i32 %0
+    ; CHECK-DAG: [[RHS:%.*]](32) = G_ANYEXTEND i32 %1
+    ; CHECK: [[RES:%.*]](32) = G_ADD i32 [[LHS]], [[RHS]]
+    ; CHECK: %2(8) = G_TRUNCATE i8 [[RES]]
+
+    %0(8) = G_TRUNCATE i8 %x0
+    %1(8) = G_TRUNCATE i8 %x1
+    %2(8) = G_ADD i8 %0, %1
+    %x0 = G_ANYEXTEND i64 %2
+...
+
+---
+# Check that we assign a relevant register bank for %0.
+# Based on the type i32, this should be gpr.
+name:            test_vector_add
+isSSA:           true
+registers:
+  - { id: 0, class: _ }
+  - { id: 1, class: _ }
+  - { id: 2, class: _ }
+body: |
+  bb.0.entry:
+    liveins: %q0, %q1, %q2, %q3
+    ; CHECK-DAG: name: test_vector_add
+    ; CHECK-DAG: [[LHS_LO:%.*]](128) = G_EXTRACT <2 x i64> %0, 0
+    ; CHECK-DAG: [[LHS_HI:%.*]](128) = G_EXTRACT <2 x i64> %0, 1
+    ; CHECK-DAG: [[RHS_LO:%.*]](128) = G_EXTRACT <2 x i64> %1, 0
+    ; CHECK-DAG: [[RHS_HI:%.*]](128) = G_EXTRACT <2 x i64> %1, 1
+    ; CHECK: [[RES_LO:%.*]](128) = G_ADD <2 x i64> [[LHS_LO]], [[RHS_LO]]
+    ; CHECK: [[RES_HI:%.*]](128) = G_ADD <2 x i64> [[LHS_HI]], [[RHS_HI]]
+    ; CHECK: %2(256) = G_SEQUENCE <4 x i64> [[RES_LO]], [[RES_HI]]
+
+    %0(256) = G_SEQUENCE <4 x i64> %q0, %q1
+    %1(256) = G_SEQUENCE <4 x i64> %q2, %q3
+    %2(256) = G_ADD <4 x i64> %0, %1
+    %q0 = G_EXTRACT <2 x i64> %2, 0
+    %q1 = G_EXTRACT <2 x i64> %2, 1
+...

test/TableGen/trydecode-emission.td

def InstB : TestInstruction {
   let AsmString = "InstB";
   let DecoderMethod = "DecodeInstB";
   let hasCompleteDecoder = 0;
 }
 
 // CHECK:      /* 0 */       MCD::OPC_ExtractField, 4, 4,  // Inst{7-4} ...
 // CHECK-NEXT: /* 3 */       MCD::OPC_FilterValue, 0, 14, 0, // Skip to: 21
 // CHECK-NEXT: /* 7 */       MCD::OPC_CheckField, 2, 2, 0, 5, 0, // Skip to: 18
-// CHECK-NEXT: /* 13 */      MCD::OPC_TryDecode, 28, 0, 0, 0, // Opcode: InstB, skip to: 18
-// CHECK-NEXT: /* 18 */      MCD::OPC_Decode, 27, 1, // Opcode: InstA
+// CHECK-NEXT: /* 13 */      MCD::OPC_TryDecode, {{[0-9]+}}, 0, 0, 0, // Opcode: InstB, skip to: 18
+// CHECK-NEXT: /* 18 */      MCD::OPC_Decode, {{[0-9]+}}, 1, // Opcode: InstA
 // CHECK-NEXT: /* 21 */      MCD::OPC_Fail,
 
 // CHECK: if (DecodeInstB(MI, insn, Address, Decoder) == MCDisassembler::Fail) { DecodeComplete = false; return MCDisassembler::Fail; }

test/TableGen/trydecode-emission2.td

def InstB : TestInstruction {
   let hasCompleteDecoder = 0;
 }
 
 // CHECK:      /* 0 */       MCD::OPC_ExtractField, 2, 1,  // Inst{2} ...
 // CHECK-NEXT: /* 3 */       MCD::OPC_FilterValue, 0, 29, 0, // Skip to: 36
 // CHECK-NEXT: /* 7 */       MCD::OPC_ExtractField, 5, 3,  // Inst{7-5} ...
 // CHECK-NEXT: /* 10 */      MCD::OPC_FilterValue, 0, 22, 0, // Skip to: 36
 // CHECK-NEXT: /* 14 */      MCD::OPC_CheckField, 0, 2, 3, 5, 0, // Skip to: 25
-// CHECK-NEXT: /* 20 */      MCD::OPC_TryDecode, 28, 0, 0, 0, // Opcode: InstB, skip to: 25
+// CHECK-NEXT: /* 20 */      MCD::OPC_TryDecode, {{[0-9]+}}, 0, 0, 0, // Opcode: InstB, skip to: 25
 // CHECK-NEXT: /* 25 */      MCD::OPC_CheckField, 3, 2, 0, 5, 0, // Skip to: 36
-// CHECK-NEXT: /* 31 */      MCD::OPC_TryDecode, 27, 1, 0, 0, // Opcode: InstA, skip to: 36
+// CHECK-NEXT: /* 31 */      MCD::OPC_TryDecode, {{[0-9]+}}, 1, 0, 0, // Opcode: InstA, skip to: 36
 // CHECK-NEXT: /* 36 */      MCD::OPC_Fail,
 
 // CHECK: if (DecodeInstB(MI, insn, Address, Decoder) == MCDisassembler::Fail) { DecodeComplete = false; return MCDisassembler::Fail; }
 // CHECK: if (DecodeInstA(MI, insn, Address, Decoder) == MCDisassembler::Fail) { DecodeComplete = false; return MCDisassembler::Fail; }

test/TableGen/trydecode-emission3.td

def InstB : TestInstruction {
   let Inst{1-0} = op;
   let OutOperandList = (outs InstBOp:$op);
   let AsmString = "InstB";
 }
 
 // CHECK:      /* 0 */       MCD::OPC_ExtractField, 4, 4,  // Inst{7-4} ...
 // CHECK-NEXT: /* 3 */       MCD::OPC_FilterValue, 0, 14, 0, // Skip to: 21
 // CHECK-NEXT: /* 7 */       MCD::OPC_CheckField, 2, 2, 0, 5, 0, // Skip to: 18
-// CHECK-NEXT: /* 13 */      MCD::OPC_TryDecode, 28, 0, 0, 0, // Opcode: InstB, skip to: 18
-// CHECK-NEXT: /* 18 */      MCD::OPC_Decode, 27, 1, // Opcode: InstA
+// CHECK-NEXT: /* 13 */      MCD::OPC_TryDecode, {{[0-9]+}}, 0, 0, 0, // Opcode: InstB, skip to: 18
+// CHECK-NEXT: /* 18 */      MCD::OPC_Decode, {{[0-9]+}}, 1, // Opcode: InstA
 // CHECK-NEXT: /* 21 */      MCD::OPC_Fail,
 
 // CHECK: if (DecodeInstBOp(MI, tmp, Address, Decoder) == MCDisassembler::Fail) { DecodeComplete = false; return MCDisassembler::Fail; }