[X86][MC][NFC] Refine code in X86MCCodeEmitter.cpp about opcode prefix

[skan]

Summary

1. Make code clearer by separating the logic of setting bits from the logic of how a prefix is encoded
2. Extract common code into functions to avoid code duplication
3. Return a enum rather a boolean to ehance scalability and uniform the behavior of functions

[craig.topper]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

222

Woudln't this be better as (W | R | X | B) ? REX : None?

224

Similar here.

272–273

clang-format

829–831

The ?: can just be combined into the return here. No need for a variable.

[skan]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

222

Good idea!

224

Will do.

829–831

Good point!

[craig.topper]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

1268

I don't know that it makes sense to make returning the bool for HasREX worse by spreading it to more functions. None of the other prefixes are needed and I'm not sure there's sufficient evidence they ever will be. And if they were needed would what type of prefix be enough information?

A bool &HasREX passed to emitOpcodePrefix and emitREXPrefix feels cleaner to me. I never liked the returned bool for this.

[skan]

Address review comments

[skan]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

1268

I know a little about it. When X86 introduces a bunch of new instructions, it usually extends the prefix. For example,
when moved from ia32 to ia32e, REX was defined. Similarly, when we moved from SSE to AVX, VEX prefix was defined.

And when we introduce new instructions, a new relocation may be needed. Here is the example for REX
https://groups.google.com/g/x86-64-abi/c/n9AWHogmVY0

Linker can do relocation optimization based on the relocation. From my understanding, if no new relocation were not added for the new instructions, the optimization could be done in an incorrect way silently.

Only a bool HasREX was defined here b/c the current interested instructions are only REX-encoded.
But we should allow more possibility here. And returning a enum here is almost as cheap as bool.

[skan]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

1268

Got your idea. Let me to pass the enum by reference rather than a return value.

[craig.topper]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

1268

REX is older than the VEX, XOP, EVEX prefixes so it doesn’t seem common to need the new relocation.

It was cheaper to compute I’d say we could determine if it has REX in the fixup code but we have to inspect multiple operands.

If we need to know about a prefix for a future location that one might be cheaper to compute.

This feels like we’re trying to solve a future problem that might never exist. Is it really worth it?

1268

*relocation

[craig.topper]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

1268

It’s also hard to add new relocations anyway since you would need a new linker that understands it. Or a command line to opt into it.

[skan]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

1268

I just gave a try. But passing the enum as argument increases the length of arglists for four functions: emitREXPrefix, emitVEXOpcodePrefix, emitOpcodePrefix and emitPrefixImpl longer, which made me nervous. I don't even know which order I should use for the parameters for the functions.

I remembered that we once refined the code in this file to reduce the parameters, so I prefer the "return value" version.
AFAICS, whether the prefix is REX, VEX, or EVEX should be enough information for the other code in emitInstructiion.

[skan]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

1268

Yes, I was talking about that if we know more info about prefix, we can easily extend the relocation type in X86MCCodeEmitter::emitMemModRMByte.

[Yikarus]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

771–772

Need rewrite the comment

[skan]

Address review comments

[skan]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

1268

I think it's worthy b/c we uniform the behavior of functions emit*Prefix*, and at the same time, there is almost no extra cost.

[craig.topper]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

1137

The names of these functions is confusing. getKind followed by setKind looks odd. I would naively expect setKind to set the thing that getKind returns. So getKind followed by setKind looks redundant.

[skan]

Address review comments

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

1137

You're right. I re-designed the interface to setLowerBound, determineOptimalKind.

[pengfei]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

774–776

Format.

855

Should be better to move below logic into X86OpcodePrefix ? Then we can simplify the function to:

X86OpcodePrefix Prefix(*Ctx.getRegisterInfo(), TSFlags);
Prefix.emit(OS);
return Prefix.getKind();

860

Should be better to use setEncoding?

[skan]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

774–776

Will do

855

We can not b/c we still need to read the operands the instruction before the final emit.
We shouldn't b/c the first principle mentioned in the summary:

"Make code clearer by separating the logic of setting bits from the logic of how a prefix is encoded"

860

I think setLowerBound is more clear than setEncoding here b/c VEX and REX are determined at the last stage. e.g
Prefix.setEncoding(VEX2) here is counterintuitive.

[skan]

Clang format

[pengfei]

LGTM. Please wait for some days for other reviewers.

[craig.topper]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

863

explicit*

[craig.topper]

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

140

I'm not sure if it makes sense to have this class understand MRI and MI. I kind of think it should only receive the register encoding.

Especially when you look at funcions like setRR2 that now end up looking up the encoding twice because it nests calls to setR and setR2.

[craig.topper]

Thinking.... What if we took this further and had an "encodeable instruction" object that contains the prefix fields and the modrm fields that we build by walking the operands and format once. Then we would only have 1 switch on format instead of the 3 we have now. Then we use that to emit the prefix, the opcode, and the modrm byte.

[skan]

Fix typo and format code

[skan]

In D143471#4114625, @craig.topper wrote:

Thinking.... What if we took this further and had an "encodeable instruction" object that contains the prefix fields and the modrm fields that we build by walking the operands and format once. Then we would only have 1 switch on format instead of the 3 we have now. Then we use that to emit the prefix, the opcode, and the modrm byte.

This sounds good! Let me give a try.

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

140

I agree the name X86OpcodePrefix doesn't sound like much to understand MI, we should have a "Helper" suffix.

Encoding twice is not a big issue. We can easily extract a function "getX86RegEncoding" to avoid this.

I am thinking your new proposal "encodeable instruction"

[skan]

Rename class name and avoid duplicated function call

[skan]

In D143471#4114638, @skan wrote:

In D143471#4114625, @craig.topper wrote:

Thinking.... What if we took this further and had an "encodable instruction" object that contains the prefix fields and the modrm fields that we build by walking the operands and format once. Then we would only have 1 switch on format instead of the 3 we have now. Then we use that to emit the prefix, the opcode, and the modrm byte.

This sounds good! Let me give a try.

@craig.topper I think it's doable with some space cost. e.g, we need to reserve some bits in "encodable instruction" to represent the "REG", "MOD", "RM" and "SIB" and It's even true when we only would like to emitPrefix.
The code change for encodable instruction is quite a big and can share some common code of this patch. I suggest we land this first and possibly implement "encodable instruction" based on it.

[craig.topper]

LGTM

[uabelho]

Hi,

I noticed that if I run lit tests with ubsan built binaries with this patch there are many many tests failing like

../lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp:222:19: runtime error: left shift of negative value -1

[skan]

In D143471#4117637, @uabelho wrote:

Hi,

I noticed that if I run lit tests with ubsan built binaries with this patch there are many many tests failing like

../lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp:222:19: runtime error: left shift of negative value -1

I think it's false positive?

~R << 7

R is an unsigned, so ~R is unsigned too. Although the value of ~R may be 0xffffffff. But I remember the left shift operation on unsigned is always well defined as long as the shift not exceed the size of unsigned.

[pengfei]

In D143471#4118036, @skan wrote:

In D143471#4117637, @uabelho wrote:

Hi,

I noticed that if I run lit tests with ubsan built binaries with this patch there are many many tests failing like

../lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp:222:19: runtime error: left shift of negative value -1

I think it's false positive?

~R << 7

R is an unsigned, so ~R is unsigned too. Although the value of ~R may be 0xffffffff. But I remember the left shift operation on unsigned is always well defined as long as the shift not exceed the size of unsigned.

I remember the << operator is defined to shift signed int in C language. There's no relationship with the type of shifted value.

[skan]

In D143471#4118152, @pengfei wrote:

In D143471#4118036, @skan wrote:

In D143471#4117637, @uabelho wrote:

Hi,

I noticed that if I run lit tests with ubsan built binaries with this patch there are many many tests failing like

../lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp:222:19: runtime error: left shift of negative value -1

I think it's false positive?

~R << 7

R is an unsigned, so ~R is unsigned too. Although the value of ~R may be 0xffffffff. But I remember the left shift operation on unsigned is always well defined as long as the shift not exceed the size of unsigned.

I remember the << operator is defined to shift signed int in C language. There's no relationship with the type of shifted value.

I am not an expert in latest C standard. But we are in C++

https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4713.pdf

8.5.7 Shift operators [expr.shift]
1 The shift operators << and >> group left-to-right.
shift-expression:
additive-expression
shift-expression << additive-expression
shift-expression >> additive-expression
The operands shall be of integral or unscoped enumeration type and integral promotions are performed. The
type of the result is that of the promoted left operand. The behavior is undefined if the right operand is
negative, or greater than or equal to the length in bits of the promoted left operand.
2 The value of E1 << E2 is E1 left-shifted E2 bit positions; vacated bits are zero-filled. If E1 has an unsigned
type, the value of the result is E1 × 2
E2, reduced modulo one more than the maximum value representable in
the result type. Otherwise, if E1 has a signed type and non-negative value, and E1 × 2
E2 is representable
in the corresponding unsigned type of the result type, then that value, converted to the result type, is the
resulting value; otherwise, the behavior is undefined.

According to the rule, unsigned does not need to be promoted and the shift operation is still on a unsigned.

[uabelho]

I think there is promotion to int due to the ~
https://stackoverflow.com/questions/32529080/should-bit-fields-less-than-int-in-size-be-the-subject-of-integral-promotion

[augusto2112]

In D143471#4117637, @uabelho wrote:

Hi,

I noticed that if I run lit tests with ubsan built binaries with this patch there are many many tests failing like

../lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp:222:19: runtime error: left shift of negative value -1

Hi, the LLDB sanitized bots are also failing with this error https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake-sanitized/3335/console

[augusto2112]

I think uabelho is right about the promotion. According to the standard: "The operand of ~ shall have integral or unscoped enumeration type; the result is the ones’ complement of its operand. Integral promotions are performed."

[craig.topper]

I just pushed d37a31cf237cb4f8a18b12c91a9204feca5900ef to fix the usbsan issue

[skan]

In D143471#4118938, @craig.topper wrote:

I just pushed d37a31cf237cb4f8a18b12c91a9204feca5900ef to fix the usbsan issue

Thank you @craig.topper !

llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp

 #include <cstdlib>
 
 using namespace llvm;
 
 #define DEBUG_TYPE "mccodeemitter"
 
 namespace {
 
+enum PrefixKind { None, REX, XOP, VEX2, VEX3, EVEX };
+
+static void emitByte(uint8_t C, raw_ostream &OS) { OS << static_cast<char>(C); }
+
+class X86OpcodePrefix {
+  // REX (1 byte)
+  // +-----+ +------+
+  // | 40H | | WRXB |
+  // +-----+ +------+
+
+  // XOP (3-byte)
+  // +-----+ +--------------+ +-------------------+
+  // | 8Fh | | RXB | m-mmmm | | W | vvvv | L | pp |
+  // +-----+ +--------------+ +-------------------+
+
+  // VEX2 (2 bytes)
+  // +-----+ +-------------------+
+  // | C5h | | R | vvvv | L | pp |
+  // +-----+ +-------------------+
+
+  // VEX3 (3 bytes)
+  // +-----+ +--------------+ +-------------------+
+  // | C4h | | RXB | m-mmmm | | W | vvvv | L | pp |
+  // +-----+ +--------------+ +-------------------+
+
+  // VEX_R: opcode externsion equivalent to REX.R in
+  // 1's complement (inverted) form
+  //
+  //  1: Same as REX_R=0 (must be 1 in 32-bit mode)
+  //  0: Same as REX_R=1 (64 bit mode only)
+
+  // VEX_X: equivalent to REX.X, only used when a
+  // register is used for index in SIB Byte.
+  //
+  //  1: Same as REX.X=0 (must be 1 in 32-bit mode)
+  //  0: Same as REX.X=1 (64-bit mode only)
+
+  // VEX_B:
+  //  1: Same as REX_B=0 (ignored in 32-bit mode)
+  //  0: Same as REX_B=1 (64 bit mode only)
+
+  // VEX_W: opcode specific (use like REX.W, or used for
+  // opcode extension, or ignored, depending on the opcode byte)
+
+  // VEX_5M (VEX m-mmmmm field):
+  //
+  //  0b00000: Reserved for future use
+  //  0b00001: implied 0F leading opcode
+  //  0b00010: implied 0F 38 leading opcode bytes
+  //  0b00011: implied 0F 3A leading opcode bytes
+  //  0b00100: Reserved for future use
+  //  0b00101: VEX MAP5
+  //  0b00110: VEX MAP6
+  //  0b00111-0b11111: Reserved for future use
+  //  0b01000: XOP map select - 08h instructions with imm byte
+  //  0b01001: XOP map select - 09h instructions with no imm byte
+  //  0b01010: XOP map select - 0Ah instructions with imm dword
+
+  // VEX_4V (VEX vvvv field): a register specifier
+  // (in 1's complement form) or 1111 if unused.
+
+  // VEX_PP: opcode extension providing equivalent
+  // functionality of a SIMD prefix
+  //  0b00: None
+  //  0b01: 66
+  //  0b10: F3
+  //  0b11: F2
+
+  // EVEX (4 bytes)
+  // +-----+ +--------------+ +-------------------+ +------------------------+
+  // | 62h | | RXBR' | 00mm | | W | vvvv | 1 | pp | | z | L'L | b | v' | aaa |
+  // +-----+ +--------------+ +-------------------+ +------------------------+
+
+  // EVEX_L2/VEX_L (Vector Length):
+  // L2 L
+  //  0 0: scalar or 128-bit vector
+  //  0 1: 256-bit vector
+  //  1 0: 512-bit vector
+
+private:
+  unsigned W : 1;
+  unsigned R : 1;
+  unsigned X : 1;
+  unsigned B : 1;
+  unsigned VEX_4V : 4;
+  unsigned VEX_L : 1;
+  unsigned VEX_PP : 2;
+  unsigned VEX_5M : 5;
+  unsigned EVEX_R2 : 1;
+  unsigned EVEX_z : 1;
+  unsigned EVEX_L2 : 1;
+  unsigned EVEX_b : 1;
+  unsigned EVEX_V2 : 1;
+  unsigned EVEX_aaa : 3;
+  PrefixKind Kind = None;
+  const MCRegisterInfo &MRI;
+
+public:
+  void setW(bool V) { W = V; }
+  void setR(const MCInst &MI, unsigned OpNum) {
+    R = MRI.getEncodingValue(MI.getOperand(OpNum).getReg()) >> 3 & 1;
+  }
+  void setX(const MCInst &MI, unsigned OpNum, unsigned Shift = 3) {
+    X = MRI.getEncodingValue(MI.getOperand(OpNum).getReg()) >> Shift & 1;
+  }
+  void setB(const MCInst &MI, unsigned OpNum) {
+    B = MRI.getEncodingValue(MI.getOperand(OpNum).getReg()) >> 3 & 1;
+  }
+  void set4V(const MCInst &MI, unsigned OpNum) {
+    VEX_4V = MRI.getEncodingValue(MI.getOperand(OpNum).getReg()) & 0xf;
+  }
+  void setL(bool V) { VEX_L = V; }
+  void setPP(unsigned V) { VEX_PP = V; }
+  void set5M(unsigned V) { VEX_5M = V; }
+  void setR2(const MCInst &MI, unsigned OpNum) {
+    EVEX_R2 = MRI.getEncodingValue(MI.getOperand(OpNum).getReg()) >> 4 & 1;
+  }
+  void setRR2(const MCInst &MI, unsigned OpNum) {
+    setR(MI, OpNum);
+    setR2(MI, OpNum);
+  }
+  void setZ(bool V) { EVEX_z = V; }
+  void setL2(bool V) { EVEX_L2 = V; }
+  void setEVEX_b(bool V) { EVEX_b = V; }
+  void setV2(const MCInst &MI, unsigned OpNum) {
+    EVEX_V2 = MRI.getEncodingValue(MI.getOperand(OpNum).getReg()) >> 4 & 1;
+  }
+  void set4VV2(const MCInst &MI, unsigned OpNum) {
+    set4V(MI, OpNum);
+    setV2(MI, OpNum);
+  }
+  void setAAA(const MCInst &MI, unsigned OpNum) {
+    EVEX_aaa = MRI.getEncodingValue(MI.getOperand(OpNum).getReg());
+  }
+
+  X86OpcodePrefix(const MCRegisterInfo &MRI)
+      : W(0), R(0), X(0), B(0), VEX_4V(0), VEX_L(0), VEX_PP(0), VEX_5M(0),
+        EVEX_R2(0), EVEX_z(0), EVEX_L2(0), EVEX_b(0), EVEX_V2(0), EVEX_aaa(0),
+        MRI(MRI) {}
+
+  void setLowerBound(PrefixKind K) { Kind = K; }
+
+  PrefixKind determineOptimalKind() {
+    switch (Kind) {
+    case None:
+      Kind = (W | R | X | B) ? REX : None;
+      break;
+    case REX:
+    case XOP:
+    case VEX3:
+    case EVEX:
+      break;
+    case VEX2:
+      Kind = (W | X | B | (VEX_5M != 1)) ? VEX3 : VEX2;
+      break;
+    }
+    return Kind;
+  }
+
+  void emit(raw_ostream &OS) const {
+    uint8_t FirstPayload =
+        ((~R) & 0x1) << 7 | ((~X) & 0x1) << 6 | ((~B) & 0x1) << 5;
+    uint8_t LastPayload = ((~VEX_4V) & 0xf) << 3 | VEX_L << 2 | VEX_PP;
+    switch (Kind) {
+    case None:
+      return;
+    case REX:
+      emitByte(0x40 | W << 3 | R << 2 | X << 1 | B, OS);
+      return;
+    case VEX2:
+      emitByte(0xC5, OS);
+      emitByte(~R << 7 | LastPayload, OS);
+      return;
+    case VEX3:
+    case XOP:
+      emitByte(Kind == VEX3 ? 0xC4 : 0x8F, OS);
+      emitByte(FirstPayload | VEX_5M, OS);
+      emitByte(W << 7 | LastPayload, OS);
+      return;
+    case EVEX:
+      assert(VEX_5M & 0x7 &&
+             "More than 3 significant bits in VEX.m-mmmm fields for EVEX!");
+      emitByte(0x62, OS);
+      emitByte(FirstPayload | ((~EVEX_R2) & 0x1) << 4 | VEX_5M, OS);
+      emitByte(W << 7 | ((~VEX_4V) & 0xf) << 3 | 1 << 2 | VEX_PP, OS);
+      emitByte(EVEX_z << 7 | EVEX_L2 << 6 | VEX_L << 5 | EVEX_b << 4 |
+                   ((~EVEX_V2) & 0x1) << 3 | EVEX_aaa,
+               OS);
+      return;
+    }
+  }
+};
+
 class X86MCCodeEmitter : public MCCodeEmitter {
   const MCInstrInfo &MCII;
   MCContext &Ctx;
 
 public:
   X86MCCodeEmitter(const MCInstrInfo &mcii, MCContext &ctx)
       : MCII(mcii), Ctx(ctx) {}
   X86MCCodeEmitter(const X86MCCodeEmitter &) = delete;
   X86MCCodeEmitter &operator=(const X86MCCodeEmitter &) = delete;
   ~X86MCCodeEmitter() override = default;
 
   void emitPrefix(const MCInst &MI, raw_ostream &OS,
                   const MCSubtargetInfo &STI) const override;
 
   void encodeInstruction(const MCInst &MI, raw_ostream &OS,
                          SmallVectorImpl<MCFixup> &Fixups,
                          const MCSubtargetInfo &STI) const override;
 
 private:
   unsigned getX86RegNum(const MCOperand &MO) const;
 
   unsigned getX86RegEncoding(const MCInst &MI, unsigned OpNum) const;
 
-  /// \param MI a single low-level machine instruction.
-  /// \param OpNum the operand #.
-  /// \returns true if the OpNumth operand of MI  require a bit to be set in
-  /// REX prefix.
-  bool isREXExtendedReg(const MCInst &MI, unsigned OpNum) const;
-
   void emitImmediate(const MCOperand &Disp, SMLoc Loc, unsigned ImmSize,
                      MCFixupKind FixupKind, uint64_t StartByte, raw_ostream &OS,
                      SmallVectorImpl<MCFixup> &Fixups, int ImmOffset = 0) const;
 
   void emitRegModRMByte(const MCOperand &ModRMReg, unsigned RegOpcodeFld,
                         raw_ostream &OS) const;
 
   void emitSIBByte(unsigned SS, unsigned Index, unsigned Base,
                    raw_ostream &OS) const;
 
   void emitMemModRMByte(const MCInst &MI, unsigned Op, unsigned RegOpcodeField,
-                        uint64_t TSFlags, bool HasREX, uint64_t StartByte,
+                        uint64_t TSFlags, PrefixKind Kind, uint64_t StartByte,
                         raw_ostream &OS, SmallVectorImpl<MCFixup> &Fixups,
                         const MCSubtargetInfo &STI,
                         bool ForceSIB = false) const;
 
-  bool emitPrefixImpl(unsigned &CurOp, const MCInst &MI,
+  PrefixKind emitPrefixImpl(unsigned &CurOp, const MCInst &MI,
                             const MCSubtargetInfo &STI, raw_ostream &OS) const;
 
-  void emitVEXOpcodePrefix(int MemOperand, const MCInst &MI,
+  PrefixKind emitVEXOpcodePrefix(int MemOperand, const MCInst &MI,
                                  raw_ostream &OS) const;
 
   void emitSegmentOverridePrefix(unsigned SegOperand, const MCInst &MI,
                                  raw_ostream &OS) const;
 
-  bool emitOpcodePrefix(int MemOperand, const MCInst &MI,
-                        const MCSubtargetInfo &STI, raw_ostream &OS) const;
+  PrefixKind emitOpcodePrefix(int MemOperand, const MCInst &MI,
+                              const MCSubtargetInfo &STI,
+                              raw_ostream &OS) const;
 
-  bool emitREXPrefix(int MemOperand, const MCInst &MI,
+  PrefixKind emitREXPrefix(int MemOperand, const MCInst &MI,
                            const MCSubtargetInfo &STI, raw_ostream &OS) const;
 };
 
 } // end anonymous namespace
 
 static uint8_t modRMByte(unsigned Mod, unsigned RegOpcode, unsigned RM) {
   assert(Mod < 4 && RegOpcode < 8 && RM < 8 && "ModRM Fields out of range!");
   return RM | (RegOpcode << 3) | (Mod << 6);
 }
 
-static void emitByte(uint8_t C, raw_ostream &OS) { OS << static_cast<char>(C); }
-
 static void emitConstant(uint64_t Val, unsigned Size, raw_ostream &OS) {
   // Output the constant in little endian byte order.
   for (unsigned i = 0; i != Size; ++i) {
     emitByte(Val & 255, OS);
     Val >>= 8;
   }
 }
 
unsigned X86MCCodeEmitter::getX86RegNum(const MCOperand &MO) const {
   return Ctx.getRegisterInfo()->getEncodingValue(MO.getReg()) & 0x7;
 }
 
 unsigned X86MCCodeEmitter::getX86RegEncoding(const MCInst &MI,
                                              unsigned OpNum) const {
   return Ctx.getRegisterInfo()->getEncodingValue(MI.getOperand(OpNum).getReg());
 }
 
-/// \param MI a single low-level machine instruction.
-/// \param OpNum the operand #.
-/// \returns true if the OpNumth operand of MI  require a bit to be set in
-/// REX prefix.
-bool X86MCCodeEmitter::isREXExtendedReg(const MCInst &MI,
-                                        unsigned OpNum) const {
-  return (getX86RegEncoding(MI, OpNum) >> 3) & 1;
-}
-
 void X86MCCodeEmitter::emitImmediate(const MCOperand &DispOp, SMLoc Loc,
                                      unsigned Size, MCFixupKind FixupKind,
                                      uint64_t StartByte, raw_ostream &OS,
                                      SmallVectorImpl<MCFixup> &Fixups,
                                      int ImmOffset) const {
   const MCExpr *Expr = nullptr;
   if (DispOp.isImm()) {
     // If this is a simple integer displacement that doesn't require a

 void X86MCCodeEmitter::emitSIBByte(unsigned SS, unsigned Index, unsigned Base,
                                    raw_ostream &OS) const {
   // SIB byte is in the same format as the modRMByte.
   emitByte(modRMByte(SS, Index, Base), OS);
 }
 
 void X86MCCodeEmitter::emitMemModRMByte(const MCInst &MI, unsigned Op,
                                         unsigned RegOpcodeField,
-                                        uint64_t TSFlags, bool HasREX,
+                                        uint64_t TSFlags, PrefixKind Kind,
                                         uint64_t StartByte, raw_ostream &OS,
                                         SmallVectorImpl<MCFixup> &Fixups,
                                         const MCSubtargetInfo &STI,
                                         bool ForceSIB) const {
   const MCOperand &Disp = MI.getOperand(Op + X86::AddrDisp);
   const MCOperand &Base = MI.getOperand(Op + X86::AddrBaseReg);
   const MCOperand &Scale = MI.getOperand(Op + X86::AddrScaleAmt);
   const MCOperand &IndexReg = MI.getOperand(Op + X86::AddrIndexReg);
unsigned FixupKind = [&]() {
       // directly if the symbol ends up in the same linkage unit.
       switch (Opcode) {
       default:
         return X86::reloc_riprel_4byte;
       case X86::MOV64rm:
         // movq loads is a subset of reloc_riprel_4byte_relax_rex. It is a
         // special case because COFF and Mach-O don't support ELF's more
         // flexible R_X86_64_REX_GOTPCRELX relaxation.
-        assert(HasREX);
+        assert(Kind == REX);
         return X86::reloc_riprel_4byte_movq_load;
       case X86::ADC32rm:
       case X86::ADD32rm:
       case X86::AND32rm:
       case X86::CMP32rm:
       case X86::MOV32rm:
       case X86::OR32rm:
       case X86::SBB32rm:
       case X86::SUB32rm:
       case X86::TEST32mr:
       case X86::XOR32rm:
       case X86::CALL64m:
       case X86::JMP64m:
       case X86::TAILJMPm64:
       case X86::TEST64mr:
       case X86::ADC64rm:
       case X86::ADD64rm:
       case X86::AND64rm:
       case X86::CMP64rm:
       case X86::OR64rm:
       case X86::SBB64rm:
       case X86::SUB64rm:
       case X86::XOR64rm:
-        return HasREX ? X86::reloc_riprel_4byte_relax_rex
+        return Kind == REX ? X86::reloc_riprel_4byte_relax_rex
                            : X86::reloc_riprel_4byte_relax;
       }
     }();
 
     // rip-relative addressing is actually relative to the *next* instruction.
     // Since an immediate can follow the mod/rm byte for an instruction, this
     // means that we need to bias the displacement field of the instruction with
     // the size of the immediate field. If we have this case, add it into the
     // expression to emit.
if (ForceDisp8)
     emitImmediate(Disp, MI.getLoc(), 1, FK_Data_1, StartByte, OS, Fixups,
                   ImmOffset);
   else if (ForceDisp32)
     emitImmediate(Disp, MI.getLoc(), 4, MCFixupKind(X86::reloc_signed_4byte),
                   StartByte, OS, Fixups);
 }
 
 /// Emit all instruction prefixes.
 ///
-/// \returns true if REX prefix is used, otherwise returns false.
-bool X86MCCodeEmitter::emitPrefixImpl(unsigned &CurOp, const MCInst &MI,
+/// \returns one of the REX, XOP, VEX2, VEX3, EVEX if any of them is used,
+/// otherwise returns None.
+PrefixKind X86MCCodeEmitter::emitPrefixImpl(unsigned &CurOp, const MCInst &MI,
                                       const MCSubtargetInfo &STI,
                                       raw_ostream &OS) const {
   uint64_t TSFlags = MCII.get(MI.getOpcode()).TSFlags;
   // Determine where the memory operand starts, if present.
   int MemoryOperand = X86II::getMemoryOperandNo(TSFlags);
   // Emit segment override opcode prefix as needed.
   if (MemoryOperand != -1) {
     MemoryOperand += CurOp;
     emitSegmentOverridePrefix(MemoryOperand + X86::AddrSegmentReg, MI, OS);
   }
case X86II::RawFrmMemOffs: {
     // Emit segment override opcode prefix as needed.
     emitSegmentOverridePrefix(1, MI, OS);
     break;
   }
   }
 
   // REX prefix is optional, but if used must be immediately before the opcode
   // Encoding type for this instruction.
-  uint64_t Encoding = TSFlags & X86II::EncodingMask;
-  bool HasREX = false;
-  if (Encoding)
-    emitVEXOpcodePrefix(MemoryOperand, MI, OS);
-  else
-    HasREX = emitOpcodePrefix(MemoryOperand, MI, STI, OS);
-
-  return HasREX;
+  return (TSFlags & X86II::EncodingMask)
+             ? emitVEXOpcodePrefix(MemoryOperand, MI, OS)
+             : emitOpcodePrefix(MemoryOperand, MI, STI, OS);
 }
 
-/// AVX instructions are encoded using a opcode prefix called VEX.
-void X86MCCodeEmitter::emitVEXOpcodePrefix(int MemOperand, const MCInst &MI,
+// AVX instructions are encoded using an encoding scheme that combines
+// prefix bytes, opcode extension field, operand encoding fields, and vector
+// length encoding capability into a new prefix, referred to as VEX.
+
+// The majority of the AVX-512 family of instructions (operating on
+// 512/256/128-bit vector register operands) are encoded using a new prefix
+// (called EVEX).
+
+// XOP is a revised subset of what was originally intended as SSE5. It was
+// changed to be similar but not overlapping with AVX.
+
+/// Emit XOP, VEX2, VEX3 or EVEX prefix.
+/// \returns the used prefix.
+PrefixKind X86MCCodeEmitter::emitVEXOpcodePrefix(int MemOperand,
+                                                 const MCInst &MI,
                                                  raw_ostream &OS) const {
   const MCInstrDesc &Desc = MCII.get(MI.getOpcode());
   uint64_t TSFlags = Desc.TSFlags;
 
   assert(!(TSFlags & X86II::LOCK) && "Can't have LOCK VEX.");
 
-  uint64_t Encoding = TSFlags & X86II::EncodingMask;
+  X86OpcodePrefix Prefix(*Ctx.getRegisterInfo());
+  switch (TSFlags & X86II::EncodingMask) {
+  default:
+    break;
+  case X86II::XOP:
+    Prefix.setLowerBound(XOP);
+    break;
+  case X86II::VEX:
+    // VEX can be 2 byte or 3 byte, not determined yet if not expilict
+    Prefix.setLowerBound(MI.getFlags() & X86::IP_USE_VEX3 ? VEX3 : VEX2);
+    break;
+  case X86II::EVEX:
+    Prefix.setLowerBound(EVEX);
+    break;
+  }
+
+  Prefix.setW(TSFlags & X86II::VEX_W);
+
   bool HasEVEX_K = TSFlags & X86II::EVEX_K;
   bool HasVEX_4V = TSFlags & X86II::VEX_4V;
   bool HasEVEX_RC = TSFlags & X86II::EVEX_RC;
 
-  // VEX_R: opcode externsion equivalent to REX.R in
-  // 1's complement (inverted) form
-  //
-  //  1: Same as REX_R=0 (must be 1 in 32-bit mode)
-  //  0: Same as REX_R=1 (64 bit mode only)
-  //
-  uint8_t VEX_R = 0x1;
-  uint8_t EVEX_R2 = 0x1;
-
-  // VEX_X: equivalent to REX.X, only used when a
-  // register is used for index in SIB Byte.
-  //
-  //  1: Same as REX.X=0 (must be 1 in 32-bit mode)
-  //  0: Same as REX.X=1 (64-bit mode only)
-  uint8_t VEX_X = 0x1;
-
-  // VEX_B:
-  //
-  //  1: Same as REX_B=0 (ignored in 32-bit mode)
-  //  0: Same as REX_B=1 (64 bit mode only)
-  //
-  uint8_t VEX_B = 0x1;
-
-  // VEX_W: opcode specific (use like REX.W, or used for
-  // opcode extension, or ignored, depending on the opcode byte)
-  uint8_t VEX_W = (TSFlags & X86II::VEX_W) ? 1 : 0;
-
-  // VEX_5M (VEX m-mmmmm field):
-  //
-  //  0b00000: Reserved for future use
-  //  0b00001: implied 0F leading opcode
-  //  0b00010: implied 0F 38 leading opcode bytes
-  //  0b00011: implied 0F 3A leading opcode bytes
-  //  0b00100: Reserved for future use
-  //  0b00101: VEX MAP5
-  //  0b00110: VEX MAP6
-  //  0b00111-0b11111: Reserved for future use
-  //  0b01000: XOP map select - 08h instructions with imm byte
-  //  0b01001: XOP map select - 09h instructions with no imm byte
-  //  0b01010: XOP map select - 0Ah instructions with imm dword
-  uint8_t VEX_5M;
   switch (TSFlags & X86II::OpMapMask) {
   default:
     llvm_unreachable("Invalid prefix!");
   case X86II::TB:
-    VEX_5M = 0x1;
-    break; // 0F
+    Prefix.set5M(0x1); // 0F
+    break;
   case X86II::T8:
-    VEX_5M = 0x2;
-    break; // 0F 38
+    Prefix.set5M(0x2); // 0F 38
+    break;
   case X86II::TA:
-    VEX_5M = 0x3;
-    break; // 0F 3A
+    Prefix.set5M(0x3); // 0F 3A
+    break;
   case X86II::XOP8:
-    VEX_5M = 0x8;
+    Prefix.set5M(0x8);
     break;
   case X86II::XOP9:
-    VEX_5M = 0x9;
+    Prefix.set5M(0x9);
     break;
   case X86II::XOPA:
-    VEX_5M = 0xA;
+    Prefix.set5M(0xA);
     break;
   case X86II::T_MAP5:
-    VEX_5M = 0x5;
+    Prefix.set5M(0x5);
     break;
   case X86II::T_MAP6:
-    VEX_5M = 0x6;
+    Prefix.set5M(0x6);
     break;
   }
 
-  // VEX_4V (VEX vvvv field): a register specifier
-  // (in 1's complement form) or 1111 if unused.
-  uint8_t VEX_4V = 0xf;
-  uint8_t EVEX_V2 = 0x1;
-
-  // EVEX_L2/VEX_L (Vector Length):
-  //
-  // L2 L
-  //  0 0: scalar or 128-bit vector
-  //  0 1: 256-bit vector
-  //  1 0: 512-bit vector
-  //
-  uint8_t VEX_L = (TSFlags & X86II::VEX_L) ? 1 : 0;
-  uint8_t EVEX_L2 = (TSFlags & X86II::EVEX_L2) ? 1 : 0;
-
-  // VEX_PP: opcode extension providing equivalent
-  // functionality of a SIMD prefix
-  //
-  //  0b00: None
-  //  0b01: 66
-  //  0b10: F3
-  //  0b11: F2
-  //
-  uint8_t VEX_PP = 0;
+  Prefix.setL(TSFlags & X86II::VEX_L);
+  Prefix.setL2(TSFlags & X86II::EVEX_L2);
   switch (TSFlags & X86II::OpPrefixMask) {
   case X86II::PD:
-    VEX_PP = 0x1;
-    break; // 66
+    Prefix.setPP(0x1); // 66
+    break;
   case X86II::XS:
-    VEX_PP = 0x2;
-    break; // F3
+    Prefix.setPP(0x2); // F3
+    break;
   case X86II::XD:
-    VEX_PP = 0x3;
-    break; // F2
+    Prefix.setPP(0x3); // F2
+    break;
   }
 
-  // EVEX_U
-  uint8_t EVEX_U = 1; // Always '1' so far
-
-  // EVEX_z
-  uint8_t EVEX_z = (HasEVEX_K && (TSFlags & X86II::EVEX_Z)) ? 1 : 0;
-
-  // EVEX_b
-  uint8_t EVEX_b = (TSFlags & X86II::EVEX_B) ? 1 : 0;
-
-  // EVEX_rc
-  uint8_t EVEX_rc = 0;
-
-  // EVEX_aaa
-  uint8_t EVEX_aaa = 0;
+  Prefix.setZ(HasEVEX_K && (TSFlags & X86II::EVEX_Z));
+  Prefix.setEVEX_b(TSFlags & X86II::EVEX_B);
 
   bool EncodeRC = false;
-
-  // Classify VEX_B, VEX_4V, VEX_R, VEX_X
-  unsigned NumOps = Desc.getNumOperands();
+  uint8_t EVEX_rc = 0;
   unsigned CurOp = X86II::getOperandBias(Desc);
 
   switch (TSFlags & X86II::FormMask) {
   default:
     llvm_unreachable("Unexpected form in emitVEXOpcodePrefix!");
   case X86II::MRMDestMem4VOp3CC: {
     //  MemAddr, src1(ModR/M), src2(VEX_4V)
-    unsigned BaseRegEnc = getX86RegEncoding(MI, MemOperand + X86::AddrBaseReg);
-    VEX_B = ~(BaseRegEnc >> 3) & 1;
-    unsigned IndexRegEnc =
-        getX86RegEncoding(MI, MemOperand + X86::AddrIndexReg);
-    VEX_X = ~(IndexRegEnc >> 3) & 1;
-
+    Prefix.setB(MI, MemOperand + X86::AddrBaseReg);
+    Prefix.setX(MI, MemOperand + X86::AddrIndexReg);
     CurOp += X86::AddrNumOperands;
-
-    unsigned RegEnc = getX86RegEncoding(MI, ++CurOp);
-    VEX_R = ~(RegEnc >> 3) & 1;
-
-    unsigned VRegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_4V = ~VRegEnc & 0xf;
+    Prefix.setR(MI, ++CurOp);
+    Prefix.set4V(MI, CurOp++);
     break;
   }
   case X86II::MRM_C0:
   case X86II::RawFrm:
   case X86II::PrefixByte:
     break;
   case X86II::MRMDestMemFSIB:
   case X86II::MRMDestMem: {
     // MRMDestMem instructions forms:
     //  MemAddr, src1(ModR/M)
     //  MemAddr, src1(VEX_4V), src2(ModR/M)
     //  MemAddr, src1(ModR/M), imm8
     //
-    unsigned BaseRegEnc = getX86RegEncoding(MI, MemOperand + X86::AddrBaseReg);
-    VEX_B = ~(BaseRegEnc >> 3) & 1;
-    unsigned IndexRegEnc =
-        getX86RegEncoding(MI, MemOperand + X86::AddrIndexReg);
-    VEX_X = ~(IndexRegEnc >> 3) & 1;
+    Prefix.setB(MI, MemOperand + X86::AddrBaseReg);
+    Prefix.setX(MI, MemOperand + X86::AddrIndexReg);
     if (!HasVEX_4V) // Only needed with VSIB which don't use VVVV.
-      EVEX_V2 = ~(IndexRegEnc >> 4) & 1;
+      Prefix.setV2(MI, MemOperand + X86::AddrIndexReg);
 
     CurOp += X86::AddrNumOperands;
 
     if (HasEVEX_K)
-      EVEX_aaa = getX86RegEncoding(MI, CurOp++);
+      Prefix.setAAA(MI, CurOp++);
 
-    if (HasVEX_4V) {
-      unsigned VRegEnc = getX86RegEncoding(MI, CurOp++);
-      VEX_4V = ~VRegEnc & 0xf;
-      EVEX_V2 = ~(VRegEnc >> 4) & 1;
-    }
+    if (HasVEX_4V)
+      Prefix.set4VV2(MI, CurOp++);
 
-    unsigned RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_R = ~(RegEnc >> 3) & 1;
-    EVEX_R2 = ~(RegEnc >> 4) & 1;
+    Prefix.setRR2(MI, CurOp++);
     break;
   }
   case X86II::MRMSrcMemFSIB:
   case X86II::MRMSrcMem: {
     // MRMSrcMem instructions forms:
     //  src1(ModR/M), MemAddr
     //  src1(ModR/M), src2(VEX_4V), MemAddr
     //  src1(ModR/M), MemAddr, imm8
     //  src1(ModR/M), MemAddr, src2(Imm[7:4])
     //
     //  FMA4:
     //  dst(ModR/M.reg), src1(VEX_4V), src2(ModR/M), src3(Imm[7:4])
-    unsigned RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_R = ~(RegEnc >> 3) & 1;
-    EVEX_R2 = ~(RegEnc >> 4) & 1;
+    Prefix.setRR2(MI, CurOp++);
 
     if (HasEVEX_K)
-      EVEX_aaa = getX86RegEncoding(MI, CurOp++);
+      Prefix.setAAA(MI, CurOp++);
 
-    if (HasVEX_4V) {
-      unsigned VRegEnc = getX86RegEncoding(MI, CurOp++);
-      VEX_4V = ~VRegEnc & 0xf;
-      EVEX_V2 = ~(VRegEnc >> 4) & 1;
-    }
+    if (HasVEX_4V)
+      Prefix.set4VV2(MI, CurOp++);
 
-    unsigned BaseRegEnc = getX86RegEncoding(MI, MemOperand + X86::AddrBaseReg);
-    VEX_B = ~(BaseRegEnc >> 3) & 1;
-    unsigned IndexRegEnc =
-        getX86RegEncoding(MI, MemOperand + X86::AddrIndexReg);
-    VEX_X = ~(IndexRegEnc >> 3) & 1;
+    Prefix.setB(MI, MemOperand + X86::AddrBaseReg);
+    Prefix.setX(MI, MemOperand + X86::AddrIndexReg);
     if (!HasVEX_4V) // Only needed with VSIB which don't use VVVV.
-      EVEX_V2 = ~(IndexRegEnc >> 4) & 1;
+      Prefix.setV2(MI, MemOperand + X86::AddrIndexReg);
 
     break;
   }
   case X86II::MRMSrcMem4VOp3: {
     // Instruction format for 4VOp3:
     //   src1(ModR/M), MemAddr, src3(VEX_4V)
-    unsigned RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_R = ~(RegEnc >> 3) & 1;
-
-    unsigned BaseRegEnc = getX86RegEncoding(MI, MemOperand + X86::AddrBaseReg);
-    VEX_B = ~(BaseRegEnc >> 3) & 1;
-    unsigned IndexRegEnc =
-        getX86RegEncoding(MI, MemOperand + X86::AddrIndexReg);
-    VEX_X = ~(IndexRegEnc >> 3) & 1;
-
-    VEX_4V = ~getX86RegEncoding(MI, CurOp + X86::AddrNumOperands) & 0xf;
+    Prefix.setR(MI, CurOp++);
+    Prefix.setB(MI, MemOperand + X86::AddrBaseReg);
+    Prefix.setX(MI, MemOperand + X86::AddrIndexReg);
+    Prefix.set4V(MI, CurOp + X86::AddrNumOperands);
     break;
   }
   case X86II::MRMSrcMemOp4: {
     //  dst(ModR/M.reg), src1(VEX_4V), src2(Imm[7:4]), src3(ModR/M),
-    unsigned RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_R = ~(RegEnc >> 3) & 1;
-
-    unsigned VRegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_4V = ~VRegEnc & 0xf;
-
-    unsigned BaseRegEnc = getX86RegEncoding(MI, MemOperand + X86::AddrBaseReg);
-    VEX_B = ~(BaseRegEnc >> 3) & 1;
-    unsigned IndexRegEnc =
-        getX86RegEncoding(MI, MemOperand + X86::AddrIndexReg);
-    VEX_X = ~(IndexRegEnc >> 3) & 1;
+    Prefix.setR(MI, CurOp++);
+    Prefix.set4V(MI, CurOp++);
+    Prefix.setB(MI, MemOperand + X86::AddrBaseReg);
+    Prefix.setX(MI, MemOperand + X86::AddrIndexReg);
     break;
   }
   case X86II::MRM0m:
   case X86II::MRM1m:
   case X86II::MRM2m:
   case X86II::MRM3m:
   case X86II::MRM4m:
   case X86II::MRM5m:
   case X86II::MRM6m:
   case X86II::MRM7m: {
     // MRM[0-9]m instructions forms:
     //  MemAddr
     //  src1(VEX_4V), MemAddr
-    if (HasVEX_4V) {
-      unsigned VRegEnc = getX86RegEncoding(MI, CurOp++);
-      VEX_4V = ~VRegEnc & 0xf;
-      EVEX_V2 = ~(VRegEnc >> 4) & 1;
-    }
+    if (HasVEX_4V)
+      Prefix.set4VV2(MI, CurOp++);
 
     if (HasEVEX_K)
-      EVEX_aaa = getX86RegEncoding(MI, CurOp++);
+      Prefix.setAAA(MI, CurOp++);
 
-    unsigned BaseRegEnc = getX86RegEncoding(MI, MemOperand + X86::AddrBaseReg);
-    VEX_B = ~(BaseRegEnc >> 3) & 1;
-    unsigned IndexRegEnc =
-        getX86RegEncoding(MI, MemOperand + X86::AddrIndexReg);
-    VEX_X = ~(IndexRegEnc >> 3) & 1;
+    Prefix.setB(MI, MemOperand + X86::AddrBaseReg);
+    Prefix.setX(MI, MemOperand + X86::AddrIndexReg);
     if (!HasVEX_4V) // Only needed with VSIB which don't use VVVV.
-      EVEX_V2 = ~(IndexRegEnc >> 4) & 1;
+      Prefix.setV2(MI, MemOperand + X86::AddrIndexReg);
 
     break;
   }
   case X86II::MRMSrcReg: {
     // MRMSrcReg instructions forms:
     //  dst(ModR/M), src1(VEX_4V), src2(ModR/M), src3(Imm[7:4])
     //  dst(ModR/M), src1(ModR/M)
     //  dst(ModR/M), src1(ModR/M), imm8
     //
     //  FMA4:
     //  dst(ModR/M.reg), src1(VEX_4V), src2(Imm[7:4]), src3(ModR/M),
-    unsigned RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_R = ~(RegEnc >> 3) & 1;
-    EVEX_R2 = ~(RegEnc >> 4) & 1;
+    Prefix.setRR2(MI, CurOp++);
 
     if (HasEVEX_K)
-      EVEX_aaa = getX86RegEncoding(MI, CurOp++);
+      Prefix.setAAA(MI, CurOp++);
 
-    if (HasVEX_4V) {
-      unsigned VRegEnc = getX86RegEncoding(MI, CurOp++);
-      VEX_4V = ~VRegEnc & 0xf;
-      EVEX_V2 = ~(VRegEnc >> 4) & 1;
-    }
+    if (HasVEX_4V)
+      Prefix.set4VV2(MI, CurOp++);
 
-    RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_B = ~(RegEnc >> 3) & 1;
-    VEX_X = ~(RegEnc >> 4) & 1;
+    Prefix.setB(MI, CurOp);
+    Prefix.setX(MI, CurOp, 4);
+    ++CurOp;
 
-    if (EVEX_b) {
+    if (TSFlags & X86II::EVEX_B) {
       if (HasEVEX_RC) {
+        unsigned NumOps = Desc.getNumOperands();
         unsigned RcOperand = NumOps - 1;
         assert(RcOperand >= CurOp);
         EVEX_rc = MI.getOperand(RcOperand).getImm();
         assert(EVEX_rc <= 3 && "Invalid rounding control!");
       }
       EncodeRC = true;
     }
     break;
   }
   case X86II::MRMSrcReg4VOp3: {
     // Instruction format for 4VOp3:
     //   src1(ModR/M), src2(ModR/M), src3(VEX_4V)
-    unsigned RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_R = ~(RegEnc >> 3) & 1;
-
-    RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_B = ~(RegEnc >> 3) & 1;
-
-    VEX_4V = ~getX86RegEncoding(MI, CurOp++) & 0xf;
+    Prefix.setR(MI, CurOp++);
+    Prefix.setB(MI, CurOp++);
+    Prefix.set4V(MI, CurOp++);
     break;
   }
   case X86II::MRMSrcRegOp4: {
     //  dst(ModR/M.reg), src1(VEX_4V), src2(Imm[7:4]), src3(ModR/M),
-    unsigned RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_R = ~(RegEnc >> 3) & 1;
-
-    unsigned VRegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_4V = ~VRegEnc & 0xf;
-
+    Prefix.setR(MI, CurOp++);
+    Prefix.set4V(MI, CurOp++);
     // Skip second register source (encoded in Imm[7:4])
     ++CurOp;
 
-    RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_B = ~(RegEnc >> 3) & 1;
-    VEX_X = ~(RegEnc >> 4) & 1;
+    Prefix.setB(MI, CurOp);
+    Prefix.setX(MI, CurOp, 4);
+    ++CurOp;
     break;
   }
   case X86II::MRMDestReg: {
     // MRMDestReg instructions forms:
     //  dst(ModR/M), src(ModR/M)
     //  dst(ModR/M), src(ModR/M), imm8
     //  dst(ModR/M), src1(VEX_4V), src2(ModR/M)
-    unsigned RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_B = ~(RegEnc >> 3) & 1;
-    VEX_X = ~(RegEnc >> 4) & 1;
+    Prefix.setB(MI, CurOp);
+    Prefix.setX(MI, CurOp, 4);
+    ++CurOp;
 
     if (HasEVEX_K)
-      EVEX_aaa = getX86RegEncoding(MI, CurOp++);
+      Prefix.setAAA(MI, CurOp++);
 
-    if (HasVEX_4V) {
-      unsigned VRegEnc = getX86RegEncoding(MI, CurOp++);
-      VEX_4V = ~VRegEnc & 0xf;
-      EVEX_V2 = ~(VRegEnc >> 4) & 1;
-    }
+    if (HasVEX_4V)
+      Prefix.set4VV2(MI, CurOp++);
 
-    RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_R = ~(RegEnc >> 3) & 1;
-    EVEX_R2 = ~(RegEnc >> 4) & 1;
-    if (EVEX_b)
+    Prefix.setRR2(MI, CurOp++);
+    if (TSFlags & X86II::EVEX_B)
       EncodeRC = true;
     break;
   }
   case X86II::MRMr0: {
     // MRMr0 instructions forms:
     //  11:rrr:000
     //  dst(ModR/M)
-    unsigned RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_R = ~(RegEnc >> 3) & 1;
-    EVEX_R2 = ~(RegEnc >> 4) & 1;
+    Prefix.setRR2(MI, CurOp++);
     break;
   }
   case X86II::MRM0r:
   case X86II::MRM1r:
   case X86II::MRM2r:
   case X86II::MRM3r:
   case X86II::MRM4r:
   case X86II::MRM5r:
   case X86II::MRM6r:
   case X86II::MRM7r: {
     // MRM0r-MRM7r instructions forms:
     //  dst(VEX_4V), src(ModR/M), imm8
-    if (HasVEX_4V) {
-      unsigned VRegEnc = getX86RegEncoding(MI, CurOp++);
-      VEX_4V = ~VRegEnc & 0xf;
-      EVEX_V2 = ~(VRegEnc >> 4) & 1;
-    }
+    if (HasVEX_4V)
+      Prefix.set4VV2(MI, CurOp++);
+
     if (HasEVEX_K)
-      EVEX_aaa = getX86RegEncoding(MI, CurOp++);
+      Prefix.setAAA(MI, CurOp++);
 
-    unsigned RegEnc = getX86RegEncoding(MI, CurOp++);
-    VEX_B = ~(RegEnc >> 3) & 1;
-    VEX_X = ~(RegEnc >> 4) & 1;
+    Prefix.setB(MI, CurOp);
+    Prefix.setX(MI, CurOp, 4);
+    ++CurOp;
     break;
   }
   }
-
-  if (Encoding == X86II::VEX || Encoding == X86II::XOP) {
-    // VEX opcode prefix can have 2 or 3 bytes
-    //
-    //  3 bytes:
-    //    +-----+ +--------------+ +-------------------+
-    //    | C4h | | RXB | m-mmmm | | W | vvvv | L | pp |
-    //    +-----+ +--------------+ +-------------------+
-    //  2 bytes:
-    //    +-----+ +-------------------+
-    //    | C5h | | R | vvvv | L | pp |
-    //    +-----+ +-------------------+
-    //
-    //  XOP uses a similar prefix:
-    //    +-----+ +--------------+ +-------------------+
-    //    | 8Fh | | RXB | m-mmmm | | W | vvvv | L | pp |
-    //    +-----+ +--------------+ +-------------------+
-    uint8_t LastByte = VEX_PP | (VEX_L << 2) | (VEX_4V << 3);
-
-    // Can we use the 2 byte VEX prefix?
-    if (!(MI.getFlags() & X86::IP_USE_VEX3) && Encoding == X86II::VEX &&
-        VEX_B && VEX_X && !VEX_W && (VEX_5M == 1)) {
-      emitByte(0xC5, OS);
-      emitByte(LastByte | (VEX_R << 7), OS);
-      return;
-    }
-
-    // 3 byte VEX prefix
-    emitByte(Encoding == X86II::XOP ? 0x8F : 0xC4, OS);
-    emitByte(VEX_R << 7 | VEX_X << 6 | VEX_B << 5 | VEX_5M, OS);
-    emitByte(LastByte | (VEX_W << 7), OS);
-  } else {
-    assert(Encoding == X86II::EVEX && "unknown encoding!");
-    // EVEX opcode prefix can have 4 bytes
-    //
-    // +-----+ +--------------+ +-------------------+ +------------------------+
-    // | 62h | | RXBR' | 0mmm | | W | vvvv | U | pp | | z | L'L | b | v' | aaa |
-    // +-----+ +--------------+ +-------------------+ +------------------------+
-    assert((VEX_5M & 0x7) == VEX_5M &&
-           "More than 3 significant bits in VEX.m-mmmm fields for EVEX!");
-
-    emitByte(0x62, OS);
-    emitByte((VEX_R << 7) | (VEX_X << 6) | (VEX_B << 5) | (EVEX_R2 << 4) |
-                 VEX_5M,
-             OS);
-    emitByte((VEX_W << 7) | (VEX_4V << 3) | (EVEX_U << 2) | VEX_PP, OS);
-    if (EncodeRC)
-      emitByte((EVEX_z << 7) | (EVEX_rc << 5) | (EVEX_b << 4) | (EVEX_V2 << 3) |
-                   EVEX_aaa,
-               OS);
-    else
-      emitByte((EVEX_z << 7) | (EVEX_L2 << 6) | (VEX_L << 5) | (EVEX_b << 4) |
-                   (EVEX_V2 << 3) | EVEX_aaa,
-               OS);
-  }
+  if (EncodeRC) {
+    Prefix.setL(EVEX_rc & 0x1);
+    Prefix.setL2(EVEX_rc & 0x2);
+  }
+  PrefixKind Kind = Prefix.determineOptimalKind();
+  Prefix.emit(OS);
+  return Kind;
 }
 
 /// Emit REX prefix which specifies
 ///   1) 64-bit instructions,
 ///   2) non-default operand size, and
 ///   3) use of X86-64 extended registers.
 ///
-/// \returns true if REX prefix is used, otherwise returns false.
-bool X86MCCodeEmitter::emitREXPrefix(int MemOperand, const MCInst &MI,
+/// \returns the used prefix (REX or None).
+PrefixKind X86MCCodeEmitter::emitREXPrefix(int MemOperand, const MCInst &MI,
                                      const MCSubtargetInfo &STI,
                                      raw_ostream &OS) const {
-  uint8_t REX = [&, MemOperand]() {
-    uint8_t REX = 0;
-    bool UsesHighByteReg = false;
-
-    const MCInstrDesc &Desc = MCII.get(MI.getOpcode());
-    uint64_t TSFlags = Desc.TSFlags;
-
-    if (TSFlags & X86II::REX_W)
-      REX |= 1 << 3; // set REX.W
-
-    if (MI.getNumOperands() == 0)
-      return REX;
-
-    unsigned NumOps = MI.getNumOperands();
-    unsigned CurOp = X86II::getOperandBias(Desc);
-
-    // If it accesses SPL, BPL, SIL, or DIL, then it requires a 0x40 REX prefix.
-    for (unsigned i = CurOp; i != NumOps; ++i) {
-      const MCOperand &MO = MI.getOperand(i);
-      if (MO.isReg()) {
-        unsigned Reg = MO.getReg();
-        if (Reg == X86::AH || Reg == X86::BH || Reg == X86::CH ||
-            Reg == X86::DH)
-          UsesHighByteReg = true;
-        if (X86II::isX86_64NonExtLowByteReg(Reg))
-          // FIXME: The caller of determineREXPrefix slaps this prefix onto
-          // anything that returns non-zero.
-          REX |= 0x40; // REX fixed encoding prefix
-      } else if (MO.isExpr() && STI.getTargetTriple().isX32()) {
-        // GOTTPOFF and TLSDESC relocations require a REX prefix to allow
-        // linker optimizations: even if the instructions we see may not require
-        // any prefix, they may be replaced by instructions that do. This is
-        // handled as a special case here so that it also works for hand-written
-        // assembly without the user needing to write REX, as with GNU as.
-        const auto *Ref = dyn_cast<MCSymbolRefExpr>(MO.getExpr());
-        if (Ref && (Ref->getKind() == MCSymbolRefExpr::VK_GOTTPOFF ||
-                    Ref->getKind() == MCSymbolRefExpr::VK_TLSDESC)) {
-          REX |= 0x40; // REX fixed encoding prefix
-        }
-      }
-    }
-
-    switch (TSFlags & X86II::FormMask) {
-    case X86II::AddRegFrm:
-      REX |= isREXExtendedReg(MI, CurOp++) << 0; // REX.B
-      break;
-    case X86II::MRMSrcReg:
-    case X86II::MRMSrcRegCC:
-      REX |= isREXExtendedReg(MI, CurOp++) << 2; // REX.R
-      REX |= isREXExtendedReg(MI, CurOp++) << 0; // REX.B
-      break;
-    case X86II::MRMSrcMem:
-    case X86II::MRMSrcMemCC:
-      REX |= isREXExtendedReg(MI, CurOp++) << 2;                        // REX.R
-      REX |= isREXExtendedReg(MI, MemOperand + X86::AddrBaseReg) << 0;  // REX.B
-      REX |= isREXExtendedReg(MI, MemOperand + X86::AddrIndexReg) << 1; // REX.X
-      CurOp += X86::AddrNumOperands;
-      break;
-    case X86II::MRMDestReg:
-      REX |= isREXExtendedReg(MI, CurOp++) << 0; // REX.B
-      REX |= isREXExtendedReg(MI, CurOp++) << 2; // REX.R
-      break;
-    case X86II::MRMDestMem:
-      REX |= isREXExtendedReg(MI, MemOperand + X86::AddrBaseReg) << 0;  // REX.B
-      REX |= isREXExtendedReg(MI, MemOperand + X86::AddrIndexReg) << 1; // REX.X
-      CurOp += X86::AddrNumOperands;
-      REX |= isREXExtendedReg(MI, CurOp++) << 2; // REX.R
-      break;
-    case X86II::MRMXmCC:
-    case X86II::MRMXm:
-    case X86II::MRM0m:
-    case X86II::MRM1m:
-    case X86II::MRM2m:
-    case X86II::MRM3m:
-    case X86II::MRM4m:
-    case X86II::MRM5m:
-    case X86II::MRM6m:
-    case X86II::MRM7m:
-      REX |= isREXExtendedReg(MI, MemOperand + X86::AddrBaseReg) << 0;  // REX.B
-      REX |= isREXExtendedReg(MI, MemOperand + X86::AddrIndexReg) << 1; // REX.X
-      break;
-    case X86II::MRMXrCC:
-    case X86II::MRMXr:
-    case X86II::MRM0r:
-    case X86II::MRM1r:
-    case X86II::MRM2r:
-    case X86II::MRM3r:
-    case X86II::MRM4r:
-    case X86II::MRM5r:
-    case X86II::MRM6r:
-    case X86II::MRM7r:
-      REX |= isREXExtendedReg(MI, CurOp++) << 0; // REX.B
-      break;
-    case X86II::MRMr0:
-      REX |= isREXExtendedReg(MI, CurOp++) << 2; // REX.R
-      break;
-    case X86II::MRMDestMemFSIB:
-      llvm_unreachable("FSIB format never need REX prefix!");
-    }
-    if (REX && UsesHighByteReg)
-      report_fatal_error(
-          "Cannot encode high byte register in REX-prefixed instruction");
-    return REX;
-  }();
-
-  if (!REX)
-    return false;
-
-  emitByte(0x40 | REX, OS);
-  return true;
+  if (!STI.hasFeature(X86::Is64Bit))
+    return None;
+  X86OpcodePrefix Prefix(*Ctx.getRegisterInfo());
+  Prefix.setW(TSFlags & X86II::REX_W);
+  if (!NumOps) {
+    PrefixKind Kind = Prefix.determineOptimalKind();
+    Prefix.emit(OS);
+    return Kind;
+  }
+      // If it accesses SPL, BPL, SIL, or DIL, then it requires a REX prefix.
+        Prefix.setLowerBound(REX);
+        Prefix.setLowerBound(REX);
+    Prefix.setB(MI, CurOp++);
+    Prefix.setR(MI, CurOp++);
+    Prefix.setB(MI, CurOp++);
+    Prefix.setR(MI, CurOp++);
+    Prefix.setB(MI, MemOperand + X86::AddrBaseReg);
+    Prefix.setX(MI, MemOperand + X86::AddrIndexReg);
+    Prefix.setB(MI, CurOp++);
+    Prefix.setR(MI, CurOp++);
+    Prefix.setB(MI, MemOperand + X86::AddrBaseReg);
+    Prefix.setX(MI, MemOperand + X86::AddrIndexReg);
+    Prefix.setR(MI, CurOp++);
+    Prefix.setB(MI, MemOperand + X86::AddrBaseReg);
+    Prefix.setX(MI, MemOperand + X86::AddrIndexReg);
+    Prefix.setB(MI, CurOp++);
+    Prefix.setR(MI, CurOp++);
+  PrefixKind Kind = Prefix.determineOptimalKind();
+  if (Kind && UsesHighByteReg)
+  Prefix.emit(OS);
+  return Kind;
 }
 
 /// Emit segment override opcode prefix as needed.
 void X86MCCodeEmitter::emitSegmentOverridePrefix(unsigned SegOperand,
                                                  const MCInst &MI,
                                                  raw_ostream &OS) const {
   // Check for explicit segment override on memory operand.
   if (unsigned Reg = MI.getOperand(SegOperand).getReg())
     emitByte(X86::getSegmentOverridePrefixForReg(Reg), OS);
 }
 
 /// Emit all instruction prefixes prior to the opcode.
 ///
 /// \param MemOperand the operand # of the start of a memory operand if present.
 /// If not present, it is -1.
 ///
-/// \returns true if REX prefix is used, otherwise returns false.
-bool X86MCCodeEmitter::emitOpcodePrefix(int MemOperand, const MCInst &MI,
+/// \returns the used prefix (REX or None).
+PrefixKind X86MCCodeEmitter::emitOpcodePrefix(int MemOperand, const MCInst &MI,
                                         const MCSubtargetInfo &STI,
                                         raw_ostream &OS) const {
   const MCInstrDesc &Desc = MCII.get(MI.getOpcode());
   uint64_t TSFlags = Desc.TSFlags;
 
   // Emit the operand size opcode prefix as needed.
   if ((TSFlags & X86II::OpSizeMask) ==
       (STI.hasFeature(X86::Is16Bit) ? X86II::OpSize32 : X86II::OpSize16))
PrefixKind X86MCCodeEmitter::emitOpcodePrefix(int MemOperand, const MCInst &MI,
   case X86II::XD: // F2
     emitByte(0xF2, OS);
     break;
   }
 
   // Handle REX prefix.
   assert((STI.hasFeature(X86::Is64Bit) || !(TSFlags & X86II::REX_W)) &&
          "REX.W requires 64bit mode.");
-  bool HasREX = STI.hasFeature(X86::Is64Bit)
-                    ? emitREXPrefix(MemOperand, MI, STI, OS)
-                    : false;
+  PrefixKind Kind = emitREXPrefix(MemOperand, MI, STI, OS);
 
   // 0x0F escape code must be emitted just before the opcode.
   switch (TSFlags & X86II::OpMapMask) {
   case X86II::TB:        // Two-byte opcode map
   case X86II::T8:        // 0F 38
   case X86II::TA:        // 0F 3A
   case X86II::ThreeDNow: // 0F 0F, second 0F emitted by caller.
     emitByte(0x0F, OS);
     break;
   }
 
   switch (TSFlags & X86II::OpMapMask) {
   case X86II::T8: // 0F 38
     emitByte(0x38, OS);
     break;
   case X86II::TA: // 0F 3A
     emitByte(0x3A, OS);
     break;
   }
 
-  return HasREX;
+  return Kind;
 }
 
 void X86MCCodeEmitter::emitPrefix(const MCInst &MI, raw_ostream &OS,
                                   const MCSubtargetInfo &STI) const {
   unsigned Opcode = MI.getOpcode();
   const MCInstrDesc &Desc = MCII.get(Opcode);
   uint64_t TSFlags = Desc.TSFlags;
 
void X86MCCodeEmitter::encodeInstruction(const MCInst &MI, raw_ostream &OS,
   if (X86II::isPseudo(TSFlags))
     return;
 
   unsigned NumOps = Desc.getNumOperands();
   unsigned CurOp = X86II::getOperandBias(Desc);
 
   uint64_t StartByte = OS.tell();
 
-  bool HasREX = emitPrefixImpl(CurOp, MI, STI, OS);
+  PrefixKind Kind = emitPrefixImpl(CurOp, MI, STI, OS);
 
   // It uses the VEX.VVVV field?
   bool HasVEX_4V = TSFlags & X86II::VEX_4V;
   bool HasVEX_I8Reg = (TSFlags & X86II::ImmMask) == X86II::Imm8Reg;
 
   // It uses the EVEX.aaa field?
   bool HasEVEX_K = TSFlags & X86II::EVEX_K;
   bool HasEVEX_RC = TSFlags & X86II::EVEX_RC;
case X86II::MRMDestReg: {
     CurOp = SrcRegNum + 1;
     break;
   }
   case X86II::MRMDestMem4VOp3CC: {
     unsigned CC = MI.getOperand(8).getImm();
     emitByte(BaseOpcode + CC, OS);
     unsigned SrcRegNum = CurOp + X86::AddrNumOperands;
     emitMemModRMByte(MI, CurOp + 1, getX86RegNum(MI.getOperand(0)), TSFlags,
-                    HasREX, StartByte, OS, Fixups, STI, false);
+                     Kind, StartByte, OS, Fixups, STI, false);
     CurOp = SrcRegNum + 3; // skip reg, VEX_V4 and CC
     break;
   }
   case X86II::MRMDestMemFSIB:
   case X86II::MRMDestMem: {
     emitByte(BaseOpcode, OS);
     unsigned SrcRegNum = CurOp + X86::AddrNumOperands;
 
     if (HasEVEX_K) // Skip writemask
       ++SrcRegNum;
 
     if (HasVEX_4V) // Skip 1st src (which is encoded in VEX_VVVV)
       ++SrcRegNum;
 
     bool ForceSIB = (Form == X86II::MRMDestMemFSIB);
     emitMemModRMByte(MI, CurOp, getX86RegNum(MI.getOperand(SrcRegNum)), TSFlags,
-                     HasREX, StartByte, OS, Fixups, STI, ForceSIB);
+                     Kind, StartByte, OS, Fixups, STI, ForceSIB);
     CurOp = SrcRegNum + 1;
     break;
   }
   case X86II::MRMSrcReg: {
     emitByte(BaseOpcode, OS);
     unsigned SrcRegNum = CurOp + 1;
 
     if (HasEVEX_K) // Skip writemask
case X86II::MRMSrcMem: {
 
     if (HasVEX_4V)
       ++FirstMemOp; // Skip the register source (which is encoded in VEX_VVVV).
 
     emitByte(BaseOpcode, OS);
 
     bool ForceSIB = (Form == X86II::MRMSrcMemFSIB);
     emitMemModRMByte(MI, FirstMemOp, getX86RegNum(MI.getOperand(CurOp)),
-                     TSFlags, HasREX, StartByte, OS, Fixups, STI, ForceSIB);
+                     TSFlags, Kind, StartByte, OS, Fixups, STI, ForceSIB);
     CurOp = FirstMemOp + X86::AddrNumOperands;
     if (HasVEX_I8Reg)
       I8RegNum = getX86RegEncoding(MI, CurOp++);
     break;
   }
   case X86II::MRMSrcMem4VOp3: {
     unsigned FirstMemOp = CurOp + 1;
 
     emitByte(BaseOpcode, OS);
 
     emitMemModRMByte(MI, FirstMemOp, getX86RegNum(MI.getOperand(CurOp)),
-                     TSFlags, HasREX, StartByte, OS, Fixups, STI);
+                     TSFlags, Kind, StartByte, OS, Fixups, STI);
     CurOp = FirstMemOp + X86::AddrNumOperands;
     ++CurOp; // Encoded in VEX.VVVV.
     break;
   }
   case X86II::MRMSrcMemOp4: {
     unsigned FirstMemOp = CurOp + 1;
 
     ++FirstMemOp; // Skip the register source (which is encoded in VEX_VVVV).
 
     // Capture second register source (encoded in Imm[7:4])
     assert(HasVEX_I8Reg && "MRMSrcRegOp4 should imply VEX_I8Reg");
     I8RegNum = getX86RegEncoding(MI, FirstMemOp++);
 
     emitByte(BaseOpcode, OS);
 
     emitMemModRMByte(MI, FirstMemOp, getX86RegNum(MI.getOperand(CurOp)),
-                     TSFlags, HasREX, StartByte, OS, Fixups, STI);
+                     TSFlags, Kind, StartByte, OS, Fixups, STI);
     CurOp = FirstMemOp + X86::AddrNumOperands;
     break;
   }
   case X86II::MRMSrcMemCC: {
     unsigned RegOp = CurOp++;
     unsigned FirstMemOp = CurOp;
     CurOp = FirstMemOp + X86::AddrNumOperands;
 
     unsigned CC = MI.getOperand(CurOp++).getImm();
     emitByte(BaseOpcode + CC, OS);
 
     emitMemModRMByte(MI, FirstMemOp, getX86RegNum(MI.getOperand(RegOp)),
-                     TSFlags, HasREX, StartByte, OS, Fixups, STI);
+                     TSFlags, Kind, StartByte, OS, Fixups, STI);
     break;
   }
 
   case X86II::MRMXrCC: {
     unsigned RegOp = CurOp++;
 
     unsigned CC = MI.getOperand(CurOp++).getImm();
     emitByte(BaseOpcode + CC, OS);
void X86MCCodeEmitter::encodeInstruction(const MCInst &MI, raw_ostream &OS,
 
   case X86II::MRMXmCC: {
     unsigned FirstMemOp = CurOp;
     CurOp = FirstMemOp + X86::AddrNumOperands;
 
     unsigned CC = MI.getOperand(CurOp++).getImm();
     emitByte(BaseOpcode + CC, OS);
 
-    emitMemModRMByte(MI, FirstMemOp, 0, TSFlags, HasREX, StartByte, OS, Fixups,
+    emitMemModRMByte(MI, FirstMemOp, 0, TSFlags, Kind, StartByte, OS, Fixups,
                      STI);
     break;
   }
 
   case X86II::MRMXm:
   case X86II::MRM0m:
   case X86II::MRM1m:
   case X86II::MRM2m:
   case X86II::MRM3m:
   case X86II::MRM4m:
   case X86II::MRM5m:
   case X86II::MRM6m:
   case X86II::MRM7m:
     if (HasVEX_4V) // Skip the register dst (which is encoded in VEX_VVVV).
       ++CurOp;
     if (HasEVEX_K) // Skip writemask
       ++CurOp;
     emitByte(BaseOpcode, OS);
     emitMemModRMByte(MI, CurOp,
                      (Form == X86II::MRMXm) ? 0 : Form - X86II::MRM0m, TSFlags,
-                     HasREX, StartByte, OS, Fixups, STI);
+                     Kind, StartByte, OS, Fixups, STI);
     CurOp += X86::AddrNumOperands;
     break;
 
   case X86II::MRM0X:
   case X86II::MRM1X:
   case X86II::MRM2X:
   case X86II::MRM3X:
   case X86II::MRM4X: