[X86] Describe wbnoinvd instruction

[GBuella]

Summary

Similar to the wbinvd instruction, except this
one does not invalidate caches. Ring 0 only.
The encoding matches a wbinvd instruction with
an F3 prefix.

[craig.topper]

Can you add a disassembler test as well?

lib/Support/Host.cpp

1220–1221

Can you line up the '=' in the clzero line above? This way it's consistent with the HasExtLeaf1 and HasLeaf7 code where everything is nicely lined up.

lib/Target/X86/X86.td

824

Isn't this instruction only on icelake server? Not icelake desktop/mobile. We may need to separate a icelake server like we do skylake and skylake-avx512.

[GBuella]

Removed wbinvd related code.
Added Ice Lake Server architecture.
Added disassembly tests.

[craig.topper]

LGTM other than that one comment.

include/llvm/IR/IntrinsicsX86.td

6426

This comment is misleading. WBNOINVD writes back the entire cache hierarchy not just a cache line.

[GBuella]

Fixed the comment on "cache line write back"

[GBuella]

Rebased the patch.
Added x86-16 tests.

[craig.topper]

lib/Target/X86/X86InstrSystem.td

488

Do you know if 0xf3 0x0f 0x09 behaves as wbinvd on older processors. Or was 0xf3 0x0f 0x09 a #UD fault on previous CPUs? Wondering whether WBINVD should have TB or PS. I'm guess since wbinvd is super old the prefixes have always been ignored, but I'm not sure.

[GBuella]

lib/Target/X86/X86InstrSystem.td

488

Well, the ISA says about the REP prefix:
"Use these prefixes only with string and I/O instructions (MOVS, CMPS, SCAS, LODS, STOS, INS, and OUTS). Use of repeat prefixes and/or undefined opcodes with other Intel 64 or IA-32 instructions is reserved; such use may cause unpredictable behavior."

So, one should not use the 0xf3 prefix with winvd.

On the other hand, this still doesn't tell us whether it would be #UD on older CPUs, or ignored.

Since the disassembler I think doesn't use flags like -mwbnoinvd, I guess we can modify wbinvd's description to mention PS.

[GBuella]

Disallow 0xf3 prefix with wbinvd.

[craig.topper]

Based on the discussion here https://patchwork.kernel.org/patch/10175159/ I think 0xf3 + wbinvd was previously equivalent to wbinvd so I think TB was correct.

[GBuella]

In D43816#1064590, @craig.topper wrote:

Based on the discussion here https://patchwork.kernel.org/patch/10175159/ I think 0xf3 + wbinvd was previously equivalent to wbinvd so I think TB was correct.

Right, but the disassembler is still going to recognize it as wbnoinvd.

[craig.topper]

Changing it from TB to PS will affect the behavior of 0xf2 0x0f 0x09 and 0x66 0x0f 0x09. Which I think we still want to disassemble to just wbinvd.

[GBuella]

In D43816#1064625, @craig.topper wrote:

Changing it from TB to PS will affect the behavior of 0xf2 0x0f 0x09 and 0x66 0x0f 0x09. Which I think we still want to disassemble to just wbinvd.

True. I reverted it to TB.

[craig.topper]

LGTM

include/llvm/IR/IntrinsicsX86.td

 }
 
 //===----------------------------------------------------------------------===//
 // Cache-line zero
 let TargetPrefix = "x86" in {
   def int_x86_clzero : GCCBuiltin<"__builtin_ia32_clzero">,
       Intrinsic<[], [llvm_ptr_ty], []>;
 }
+
+//===----------------------------------------------------------------------===//
+// Cache line write back intrinsics
+
+// Write back no-invalidate
+let TargetPrefix = "x86" in {
+  def int_x86_wbnoinvd : GCCBuiltin<"__builtin_ia32_wbnoinvd">,
+      Intrinsic<[], [], []>;
+}

lib/Support/Host.cpp

bool sys::getHostCPUFeatures(StringMap<bool> &Features) {
   Features["sse4a"]  = HasExtLeaf1 && ((ECX >>  6) & 1);
   Features["prfchw"] = HasExtLeaf1 && ((ECX >>  8) & 1);
   Features["xop"]    = HasExtLeaf1 && ((ECX >> 11) & 1) && HasAVXSave;
   Features["lwp"]    = HasExtLeaf1 && ((ECX >> 15) & 1);
   Features["fma4"]   = HasExtLeaf1 && ((ECX >> 16) & 1) && HasAVXSave;
   Features["tbm"]    = HasExtLeaf1 && ((ECX >> 21) & 1);
   Features["mwaitx"] = HasExtLeaf1 && ((ECX >> 29) & 1);
 
+  // Miscellaneous memory related features, detected by
+  // using the 0x80000008 leaf of the CPUID instruction
   bool HasExtLeaf8 = MaxExtLevel >= 0x80000008 &&
                      !getX86CpuIDAndInfo(0x80000008, &EAX, &EBX, &ECX, &EDX);
-  Features["clzero"] = HasExtLeaf8 && ((EBX >> 0) & 1);
+  Features["clzero"]   = HasExtLeaf8 && ((EBX >> 0) & 1);
+  Features["wbnoinvd"] = HasExtLeaf8 && ((EBX >> 9) & 1);
 
   bool HasLeaf7 =
       MaxLevel >= 7 && !getX86CpuIDAndInfoEx(0x7, 0x0, &EAX, &EBX, &ECX, &EDX);
 
   Features["fsgsbase"]   = HasLeaf7 && ((EBX >>  0) & 1);
   Features["sgx"]        = HasLeaf7 && ((EBX >>  2) & 1);
   Features["bmi"]        = HasLeaf7 && ((EBX >>  3) & 1);
   // AVX2 is only supported if we have the OS save support from AVX.

lib/Target/X86/X86.td

def FeaturePadShortFunctions : SubtargetFeature<"pad-short-functions",
                                      "PadShortFunctions", "true",
                                      "Pad short functions">;
 def FeatureSGX     : SubtargetFeature<"sgx", "HasSGX", "true",
                                       "Enable Software Guard Extensions">;
 def FeatureCLFLUSHOPT : SubtargetFeature<"clflushopt", "HasCLFLUSHOPT", "true",
                                       "Flush A Cache Line Optimized">;
 def FeatureCLWB    : SubtargetFeature<"clwb", "HasCLWB", "true",
                                       "Cache Line Write Back">;
+def FeatureWBNOINVD    : SubtargetFeature<"wbnoinvd", "HasWBNOINVD", "true",
+                                      "Write Back No Invalidate">;
 def FeatureRDPID : SubtargetFeature<"rdpid", "HasRDPID", "true",
                                     "Support RDPID instructions">;
 // On some processors, instructions that implicitly take two memory operands are
 // slow. In practice, this means that CALL, PUSH, and POP with memory operands
 // should be avoided in favor of a MOV + register CALL/PUSH/POP.
 def FeatureSlowTwoMemOps : SubtargetFeature<"slow-two-mem-ops",
                                      "SlowTwoMemOps", "true",
                                      "Two memory operand instructions are slow">;
def ICLFeatures : ProcessorFeatures<CNLFeatures.Value, [
   FeatureCLWB,
   FeatureRDPID
 ]>;
 
 class IcelakeClientProc<string Name> : ProcModel<Name, SkylakeServerModel,
                                                  ICLFeatures.Value, [
   ProcIntelICL,
   FeatureHasFastGather
 ]>;
 def : IcelakeClientProc<"icelake-client">;
 
 class IcelakeServerProc<string Name> : ProcModel<Name, SkylakeServerModel,
                                                  ICLFeatures.Value, [
   ProcIntelICX,
+  FeatureWBNOINVD,
   FeatureHasFastGather
 ]>;
 def : IcelakeServerProc<"icelake-server">;
 
 // AMD CPUs.
 
 def : Proc<"k6",              [FeatureX87, FeatureSlowUAMem16, FeatureMMX]>;
 def : Proc<"k6-2",            [FeatureX87, FeatureSlowUAMem16, Feature3DNow]>;

lib/Target/X86/X86InstrInfo.td

 def HasCLZERO    : Predicate<"Subtarget->hasCLZERO()">;
 def FPStackf32   : Predicate<"!Subtarget->hasSSE1()">;
 def FPStackf64   : Predicate<"!Subtarget->hasSSE2()">;
 def HasMPX       : Predicate<"Subtarget->hasMPX()">;
 def HasSHSTK     : Predicate<"Subtarget->hasSHSTK()">;
 def HasIBT       : Predicate<"Subtarget->hasIBT()">;
 def HasCLFLUSHOPT : Predicate<"Subtarget->hasCLFLUSHOPT()">;
 def HasCLWB      : Predicate<"Subtarget->hasCLWB()">;
+def HasWBNOINVD  : Predicate<"Subtarget->hasWBNOINVD()">;
 def HasRDPID     : Predicate<"Subtarget->hasRDPID()">;
 def HasCmpxchg16b: Predicate<"Subtarget->hasCmpxchg16b()">;
 def Not64BitMode : Predicate<"!Subtarget->is64Bit()">,
                              AssemblerPredicate<"!Mode64Bit", "Not 64-bit mode">;
 def In64BitMode  : Predicate<"Subtarget->is64Bit()">,
                              AssemblerPredicate<"Mode64Bit", "64-bit mode">;
 def IsLP64  : Predicate<"Subtarget->isTarget64BitLP64()">;
 def NotLP64 : Predicate<"!Subtarget->isTarget64BitLP64()">;

lib/Target/X86/X86InstrSystem.td

let Defs = [EAX, EBX, ECX, EDX], Uses = [EAX, ECX] in
   def CPUID : I<0xA2, RawFrm, (outs), (ins), "cpuid", [], IIC_CPUID>, TB;
 } // SchedRW
 
 //===----------------------------------------------------------------------===//
 // Cache instructions
 let SchedRW = [WriteSystem] in {
 def INVD : I<0x08, RawFrm, (outs), (ins), "invd", [], IIC_INVD>, TB;
 def WBINVD : I<0x09, RawFrm, (outs), (ins), "wbinvd", [], IIC_INVD>, TB;
+
+// wbnoinvd is like wbinvd, except without invalidation
+// encoding: like wbinvd + an 0xF3 prefix
+def WBNOINVD : I<0x09, RawFrm, (outs), (ins), "wbnoinvd",
+                 [(int_x86_wbnoinvd)], IIC_INVD>, XS,
+                 Requires<[HasWBNOINVD]>;
 } // SchedRW
 
 //===----------------------------------------------------------------------===//
 // CET instructions
 let SchedRW = [WriteSystem], Predicates = [HasSHSTK]  in{
   let Uses = [SSP] in {
     let Defs = [SSP] in {
       def INCSSPD : I<0xAE, MRM5r, (outs), (ins GR32:$src), "incsspd\t$src",

lib/Target/X86/X86Subtarget.h

protected:
   bool HasSGX;
 
   /// Processor supports Flush Cache Line instruction
   bool HasCLFLUSHOPT;
 
   /// Processor supports Cache Line Write Back instruction
   bool HasCLWB;
 
+  /// Processor supports Write Back No Invalidate instruction
+  bool HasWBNOINVD;
+
   /// Processor support RDPID instruction
   bool HasRDPID;
 
   /// Use a retpoline thunk rather than indirect calls to block speculative
   /// execution.
   bool UseRetpoline;
 
   /// When using a retpoline thunk, call an externally provided thunk rather
public:
   bool hasPKU() const { return HasPKU; }
   bool hasVNNI() const { return HasVNNI; }
   bool hasBITALG() const { return HasBITALG; }
   bool hasMPX() const { return HasMPX; }
   bool hasSHSTK() const { return HasSHSTK; }
   bool hasIBT() const { return HasIBT; }
   bool hasCLFLUSHOPT() const { return HasCLFLUSHOPT; }
   bool hasCLWB() const { return HasCLWB; }
+  bool hasWBNOINVD() const { return HasWBNOINVD; }
   bool hasRDPID() const { return HasRDPID; }
   bool useRetpoline() const { return UseRetpoline; }
   bool useRetpolineExternalThunk() const { return UseRetpolineExternalThunk; }
 
   unsigned getPreferVectorWidth() const { return PreferVectorWidth; }
   unsigned getRequiredVectorWidth() const { return RequiredVectorWidth; }
 
   // Helper functions to determine when we should allow widening to 512-bit

lib/Target/X86/X86Subtarget.cpp

void X86Subtarget::initializeEnvironment() {
   HasMWAITX = false;
   HasCLZERO = false;
   HasMPX = false;
   HasSHSTK = false;
   HasIBT = false;
   HasSGX = false;
   HasCLFLUSHOPT = false;
   HasCLWB = false;
+  HasWBNOINVD = false;
   HasRDPID = false;
   UseRetpoline = false;
   UseRetpolineExternalThunk = false;
   IsPMULLDSlow = false;
   IsSHLDSlow = false;
   IsUAMem16Slow = false;
   IsUAMem32Slow = false;
   HasSSEUnalignedMem = false;

test/CodeGen/X86/wbnoinvd-intrinsic.ll

+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc < %s -mtriple=i686-unknown-unknown -mattr=+wbnoinvd | FileCheck %s -check-prefix=CHECK32
+; RUN: llc < %s -mtriple=x86_64-unknown-unknown -mattr=+wbnoinvd | FileCheck %s -check-prefix=CHECK64
+
+define void @wbnoinvd() nounwind {
+; CHECK32-LABEL: wbnoinvd:
+; CHECK32:       # %bb.0:
+; CHECK32-NEXT:    wbnoinvd
+; CHECK32-NEXT:    retl
+;
+; CHECK64-LABEL: wbnoinvd:
+; CHECK64:       # %bb.0:
+; CHECK64-NEXT:    wbnoinvd
+; CHECK64-NEXT:    retq
+  tail call void @llvm.x86.wbnoinvd()
+  ret void
+}
+declare void @llvm.x86.wbnoinvd() nounwind

test/MC/Disassembler/X86/x86-16.txt

 # CHECK: lretw
 0xcb
 
 # CHECK: lretl
 0x66 0xcb
 
 # CHECK: callw  -1
 0xe8 0xff 0xff
+
+# CHECK: wbnoinvd
+0xf3 0x0f 0x09

test/MC/Disassembler/X86/x86-32.txt

 # CHECK: rdpid %eax
 0xf3 0x0f 0xc7 0xf8
 
 # CHECK: ptwritel -559038737(%ebx,%ecx,8)
 0xf3 0x0f 0xae 0xa4 0xcb 0xef 0xbe 0xad 0xde
 
 # CHECK: ptwritel %eax
 0xf3 0x0f 0xae 0xe0
+
+# CHECK: wbnoinvd
+0xf3 0x0f 0x09

test/MC/Disassembler/X86/x86-64.txt

 # CHECK: ptwritel %eax
 0xf3 0x0f 0xae 0xe0
 
 # CHECK: ptwriteq -559038737(%rbx,%rcx,8)
 0xf3 0x48 0x0f 0xae 0xa4 0xcb 0xef 0xbe 0xad 0xde
 
 # CHECK: ptwriteq %rax
 0xf3 0x48 0x0f 0xae 0xe0
+
+# CHECK: wbnoinvd
+0xf3 0x0f 0x09

test/MC/X86/x86-16.s

 // CHECK: encoding: [0x66]
 data32
 
 // CHECK: data32
 // CHECK: encoding: [0x66]
 // CHECK: lgdtw 4(%eax)
 // CHECK:  encoding: [0x67,0x0f,0x01,0x50,0x04]
 data32 lgdt 4(%eax)
+
+// CHECK: wbnoinvd
+// CHECK:  encoding: [0xf3,0x0f,0x09]
+wbnoinvd

test/MC/X86/x86-32-coverage.s

 // CHECK: invd
 // CHECK:  encoding: [0x0f,0x08]
           invd
 
 // CHECK: wbinvd
 // CHECK:  encoding: [0x0f,0x09]
           wbinvd
 
+// CHECK: wbnoinvd
+// CHECK:  encoding: [0xf3,0x0f,0x09]
+          wbnoinvd
+
 // CHECK: cpuid
 // CHECK:  encoding: [0x0f,0xa2]
           cpuid
 
 // CHECK: wrmsr
 // CHECK:  encoding: [0x0f,0x30]
           wrmsr
 

test/MC/X86/x86-64.s

 // CHECK: ptwriteq 3735928559(%rbx,%rcx,8)
 // CHECK:  encoding: [0xf3,0x48,0x0f,0xae,0xa4,0xcb,0xef,0xbe,0xad,0xde]
 ptwriteq 0xdeadbeef(%rbx,%rcx,8)
 
 // CHECK: ptwriteq %rax
 // CHECK:  encoding: [0xf3,0x48,0x0f,0xae,0xe0]
 ptwriteq %rax
 
+// CHECK: wbnoinvd
+// CHECK:  encoding: [0xf3,0x0f,0x09]
+wbnoinvd
+
 //  __asm __volatile(
 //    "pushf        \n\t"
 //    "popf       \n\t"
 //    "rep        \n\t"
 //    ".byte  0x0f, 0xa7, 0xd0"
 //  );
 // CHECK: pushfq
 // CHECK-NEXT: popfq