[X86] Limit prefix padding w/target specific padding amount

[reames]

Summary

This change defines the maximum number of prefixes safe to use for padding in a target specific way. As noted in the comments, different processor generations behave very differently w.r.t. redundant prefixes. Some processors - mostly, but not entirely older ones - encounter serious stalls if too many prefixes are used.

Note: Originally, this patch also enabled the prefix padding by default. That's currently on hold.

[reames]

I need to restructure this. As I noted over in https://reviews.llvm.org/D76176, we have a problem for the assembler use case w/inserting prefixes between previous data encoded prefixes and the instruction. This isn't relevant for the compiler (since we have the suppression mechanism), but we need to structure this such that only the compiler enables this feature. I'm very unhappy about how ugly that is, but I don't see a way around it.

[skan]

D76285 really reveals a bug when we want to enable prefix padding. I don't believe we can enable prefix padding of relaxable instructions by default without fixing that.

[reames]

Restructure patch not to enable by default. This allows the target specific logic to be tested in tree while avoiding the optimized assembler issue for now.

@skan - Will reply to your point separately via email. I disagree and think we need a broader discussion on this point.

[reames]

Add an extra test to cover the target specific limits. Test structure is analogous to x86_long_nop.s.

[LuoYuanke]

llvm/lib/Target/X86/MCTargetDesc/X86AsmBackend.cpp

844

I'm not sure the long nop feature can determine the prefix size. I'd leave it to Craig. :)

[craig.topper]

llvm/lib/Target/X86/MCTargetDesc/X86AsmBackend.cpp

820

manditory->mandatory

846

What does conservatively handled mean here?

[reames]

llvm/lib/Target/X86/MCTargetDesc/X86AsmBackend.cpp

846

Basically, do all targets have the right setting for the number of prefixes allowed (proxied by nop length atm) and that escape bytes are counted where appropriate. (e.g. atom can decode three prefixes, but silvermont only gets three of either prefix or escape bytes.)

I haven't dug through all the code involved to figure out escape bytes are represented. In some cases, we appear to treat them as prefixes, but I'm not sure this handling is universal. This why I'm disabling prefix padding on targets which need to care about presence of escape bytes.

[skan]

llvm/lib/Target/X86/MCTargetDesc/X86AsmBackend.cpp

865

Use a C++ style cast here.

865

Since the bug fix patch D76285 was landed, I think we can allow all targets to decide how many prefixes can be added by default,
and user can pass option -x86-pad-max-prefix-size to override the default value.

llvm/test/MC/X86/prefix-padding-length.s

11

From my knowledge, atom can decode 5 prefixes without penalty.

llvm/lib/Target/X86/MCTargetDesc/X86AsmBackend.cpp

 }
 
 
 static bool shouldAddPrefix(const MCInst &Inst, const MCInstrInfo &MCII) {
   // Linker may rewrite the instruction with variant symbol operand.
   return !hasVariantSymbol(Inst);
 }
 
+/// Return the number of prefixes which can be added to an instruction without
+/// encountering decoder delays for the given target.  Note that the result is
+/// inclusive of escape bytes and manditory prefixes.
+static unsigned maximumPrefixPadding(const MCSubtargetInfo &STI) {
+  // Summary of decoder behavior (per Agner's guide)
+  // AMD
+  //   K8 and K10 - three prefixes per instruction per cycle (i.e. no stall)
+  //   Bulldozer,Piledriver, Excavator, Steamroller - three prefixes per
+  //     instruction or 10-15 cycle stall
+  //   Ryzen - any number of prefixes in a single cycle
+  //   Bobcat, Jaquar - no penalty for multiple prefixes
+  // Intel
+  //   SandyBridge and later - can decode any number of prefixes in a single
+  //     cycle.
+  //   Atom - Up to three prefix can be decoded in a single cycle.  (Severe
+  //     delay for more)
+  //   Silvermont - can decode three prefixes *and escape sequences* in a
+  //     single cycle (severe stall)
+  //   Goldmont - can decode (most combinations of) four prefixes in a single
+  //     cycle.
+  //   Knights Landing - can decode three prefixes and escape bytes in a single
+  //     cycle (5-6 cycle stall if exceeded)
+
+  // We use maximum nop size as a proxy for the number of prefixes we can
+  // safely execute.  Since nops longer than 8 are formed by adding redundant
+  // prefxes, this seems like a reasonable proxy. 
+  if (STI.getFeatureBits()[X86::FeatureFast7ByteNOP])
+    // TODO: This should probably be '3' once we validate that escape sequences
+    // are conservatively handled.
+    return 0;
+  else if (STI.getFeatureBits()[X86::FeatureFast15ByteNOP])
+    return 15;
+  else if (STI.getFeatureBits()[X86::FeatureFast11ByteNOP])
+    return 3;
+  return 2;
+}
+
 static unsigned getRemainingPrefixSize(const MCInst &Inst,
                                        const MCSubtargetInfo &STI,
                                        MCCodeEmitter &Emitter) {
   SmallString<256> Code;
   raw_svector_ostream VecOS(Code);
   Emitter.emitPrefix(Inst, VecOS, STI);
   assert(Code.size() < 15 && "The number of prefixes must be less than 15.");
 
-  // TODO: It turns out we need a decent amount of plumbing for the target
-  // specific bits to determine number of prefixes its safe to add.  Various
-  // targets (older chips mostly, but also Atom family) encounter decoder
-  // stalls with too many prefixes.  For testing purposes, we set the value
-  // externally for the moment.
-  unsigned ExistingPrefixSize = Code.size();
-  unsigned TargetPrefixMax = X86PadMaxPrefixSize;
+  const unsigned ExistingPrefixSize = Code.size();
+  const unsigned TargetPrefixMax =
+    std::min(maximumPrefixPadding(STI), (unsigned)X86PadMaxPrefixSize);
+ 
   if (TargetPrefixMax <= ExistingPrefixSize)
     return 0;
   return TargetPrefixMax - ExistingPrefixSize;
 }
 
 bool X86AsmBackend::padInstructionViaPrefix(MCRelaxableFragment &RF,
                                             MCCodeEmitter &Emitter,
                                             unsigned &RemainingSize) const {

llvm/test/MC/X86/prefix-padding-32.s

 # RUN: llvm-mc -filetype=obj -triple i386-pc-linux-gnu %s -x86-pad-max-prefix-size=15 | llvm-objdump -d --section=.text - | FileCheck %s
 
 # Check prefix padding generation for all cases on 32 bit x86.
 
-# CHECK:  1: 3e 3e 3e 3e 3e 3e 3e 3e 3e 81 e1 01 00 00 00   andl  $1, %ecx
-# CHECK: 10: 3e 3e 3e 3e 3e 3e 3e 3e 3e 81 21 01 00 00 00   andl  $1, %ds:(%ecx)
-# CHECK: 1f: 2e 2e 2e 2e 2e 2e 2e 2e 2e 81 21 01 00 00 00   andl  $1, %cs:(%ecx)
-# CHECK: 2e: 3e 3e 3e 3e 3e 3e 3e 3e 3e 81 21 01 00 00 00   andl  $1, %ds:(%ecx)
-# CHECK: 3d: 26 26 26 26 26 26 26 26 26 81 21 01 00 00 00   andl  $1, %es:(%ecx)
-# CHECK: 4c: 64 64 64 64 64 64 64 64 64 81 21 01 00 00 00   andl  $1, %fs:(%ecx)
-# CHECK: 5b: 65 65 65 65 65 65 65 65 65 81 21 01 00 00 00   andl  $1, %gs:(%ecx)
-# CHECK: 6a: 36 36 36 36 36 36 36 36 36 81 21 01 00 00 00   andl  $1, %ss:(%ecx)
-# CHECK: 79: 3e 3e 3e 3e 3e 81 a1 00 00 00 00 01 00 00 00   andl  $1, %ds:(%ecx)
-# CHECK: 88: 3e 3e 3e 3e 3e 81 a1 00 00 00 00 01 00 00 00   andl  $1, %ds:(%ecx)
-# CHECK: 97: 36 36 36 36 36 36 36 36 81 24 24 01 00 00 00   andl  $1, %ss:(%esp)
-# CHECK: a6: 65 65 65 65 65 65 65 65 81 24 24 01 00 00 00   andl  $1, %gs:(%esp)
-# CHECK: b5: 36 36 36 36 81 a4 24 00 00 00 00 01 00 00 00   andl  $1, %ss:(%esp)
-# CHECK: c4: 36 36 36 36 36 36 36 36 81 65 00 01 00 00 00   andl  $1, %ss:(%ebp)
-# CHECK: d3: 65 65 65 65 65 65 65 65 81 65 00 01 00 00 00   andl  $1, %gs:(%ebp)
-# CHECK: e2: 36 36 36 36 36 81 a5 00 00 00 00 01 00 00 00   andl  $1, %ss:(%ebp)
+# CHECK:  1: 3e 3e 81 e1 01 00 00 00                  andl  $1, %ecx
+# CHECK:  9: 3e 3e 81 21 01 00 00 00                  andl  $1, %ds:(%ecx)
+# CHECK: 11: 2e 2e 81 21 01 00 00 00                  andl  $1, %cs:(%ecx)
+# CHECK: 19: 3e 3e 81 21 01 00 00 00                  andl  $1, %ds:(%ecx)
+# CHECK: 21: 26 26 81 21 01 00 00 00                  andl  $1, %es:(%ecx)
+# CHECK: 29: 64 64 81 21 01 00 00 00                  andl  $1, %fs:(%ecx)
+# CHECK: 31: 65 65 81 21 01 00 00 00                  andl  $1, %gs:(%ecx)
+# CHECK: 39: 36 36 81 21 01 00 00 00                  andl  $1, %ss:(%ecx)
+# CHECK: 41: 3e 3e 81 a1 00 00 00 00 01 00 00 00      andl  $1, %ds:(%ecx)
+# CHECK: 4d: 3e 3e 81 a1 00 00 00 00 01 00 00 00      andl  $1, %ds:(%ecx)
+# CHECK: 59: 36 36 81 24 24 01 00 00 00               andl  $1, %ss:(%esp)
+# CHECK: 62: 65 65 81 24 24 01 00 00 00               andl  $1, %gs:(%esp)
+# CHECK: 6b: 36 36 81 a4 24 00 00 00 00 01 00 00 00   andl  $1, %ss:(%esp)
+# CHECK: 78: 36 36 81 65 00 01 00 00 00               andl  $1, %ss:(%ebp)
+# CHECK: 81: 65 65 81 65 00 01 00 00 00               andl  $1, %gs:(%ebp)
+# CHECK: 8a: 36 36 81 a5 00 00 00 00 01 00 00 00      andl  $1, %ss:(%ebp)
   .text
   .section  .text
   .p2align 8
 bar:
   int3  
 foo:
   # non-memory
   andl $foo, %ecx

llvm/test/MC/X86/prefix-padding-length.s

+# Default tunings (conservative)
+# RUN: llvm-mc -filetype=obj -triple i386-pc-linux-gnu %s -x86-pad-max-prefix-size=15 | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK32-PREFIX2
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX2
+# Intel Processors
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=sandybridge | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX15
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=ivybridge | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX15
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=haswell | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX15
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=broadwell | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX15
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=skylake | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX15
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=skx | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX15
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=atom | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX2
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=silvermont | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX0
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=goldmont | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX2
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=knl | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX2
+# AMD processors
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=bdver1 | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX3
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=bdver2 | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX3
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=btver1 | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX15
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=btver2 | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX15
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=znver1 | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX15
+# RUN: llvm-mc -filetype=obj -triple x86_64-pc-linux-gnu %s -x86-pad-max-prefix-size=15 -mcpu=znver2 | llvm-objdump -d --section=.text - | FileCheck %s --check-prefix=CHECK64-PREFIX15
+
+# Check target specific maximum prefix length cases
+
+# CHECK32-PREFIX2:  0: 3e 3e 81 e1 00 00 00 00     andl $0, %ecx
+# CHECK64-PREFIX0:  0:          81 e1 00 00 00 00  andl $0, %ecx
+# CHECK64-PREFIX2:  0:    2e 2e 81 e1 00 00 00 00  andl $0, %ecx
+# CHECK64-PREFIX3:  0: 2e 2e 2e 81 e1 00 00 00 00  andl $0, %ecx
+# CHECK64-PREFIX15: 0: 2e 2e 2e 2e 2e 2e 2e 2e 2e 81 e1 00 00 00 00  andl $0, %ecx
+  .text
+  .section  .text
+  .p2align 4
+foo:
+  # non-memory
+  andl $foo, %ecx
+  .p2align 4
+  int3