[AArch64] Adding "armv8.8-a" BC instruction.

[tmatheson]

Summary

This instruction is described in the Arm A64 Instruction Set Architecture documentation available here:
https://developer.arm.com/documentation/ddi0596/2021-12/Base-Instructions/BC-cond--Branch-Consistent-conditionally-?lang=en

FEAT_HBC "Hinted Conditional Branches" is listed in the 2021 A-Profile Architecture Extensions: https://developer.arm.com/architectures/cpu-architecture/a-profile/exploration-tools/feature-names-for-a-profile

'BC.cc', where 'cc' is any ordinary condition code, is an instruction
that looks exactly like B.cc (the normal conditional branch), except
that bit 4 of the encoding is 1 rather than 0, which hints something
to the branch predictor (specifically, that this branch is expected to
be highly consistent, even though _which way_ it will consistently go
is not known at compile time).

This commit introduces a special subtarget feature for HBC, which is a
dependency of the top-level 8.8-A feature, and uses that to enable the
new BC instruction.

Patch by Simon Tatham and Tomas Matheson

[nickdesaulniers]

Can you please site the relevant section of the Arm ARM for these extensions in the commit message?

My copy ver ARM DDI 0487G.b (ID072021) doesn't seem to mention these. (Checked § C4.1.3 and § A2).

Since it doesn't seem to mention 8.8, so I can't personally verify the encodings. But that's fine if another reviewer perhaps can?

llvm/lib/Target/AArch64/AArch64.td

420

Should this be hinted branch conditional to match HBC, or does the ARM ARM refer to it as such?

llvm/test/MC/AArch64/armv8.8a-hbc.s

4–5

Why do you need a temporary file (%t) here? Can't you just use a pipe as above?

23

Do you need to define lbl somewhere?

[tmatheson]

Tweak description and test case.

[tmatheson]

llvm/lib/Target/AArch64/AArch64.td

420

No, the name of the feature is "Hinted conditional branches" but it is referred to as FEAT_HBC.

https://developer.arm.com/architectures/cpu-architecture/a-profile/exploration-tools/feature-names-for-a-profile

However, I've updated the description to match the format used for the existing extensions.

I don't think there is a version of the Arm ARM with these extensions released yet, but I've added links to the A64 ISA docs to the description.

llvm/test/MC/AArch64/armv8.8a-hbc.s

23

Apparently not, but I've added it as a label for clarity.

[nickdesaulniers]

llvm/test/MC/Disassembler/AArch64/armv8.8a-hbc.txt

4–5

Same question about the temporary here; prefer a pipe if possible. Sorry I missed that on my first look.

[tmatheson]

Use pipe in second test

[tmatheson]

Thanks for the comments

llvm/lib/Target/AArch64/AArch64.td

def FeatureWFxT : SubtargetFeature<"wfxt", "HasWFxT",
     "true", "Enable Armv8.7-A WFET and WFIT instruction">;
 
 def FeatureHCX : SubtargetFeature<
     "hcx", "HasHCX", "true", "Enable Armv8.7-A HCRX_EL2 system register">;
 
 def FeatureLS64 : SubtargetFeature<"ls64", "HasLS64",
     "true", "Enable Armv8.7-A LD64B/ST64B Accelerator Extension">;
 
+def FeatureHBC : SubtargetFeature<"hbc", "HasHBC",
+    "true", "Enable Armv8.8-A Hinted Conditional Branches Extension">;
+
 def FeatureBRBE : SubtargetFeature<"brbe", "HasBRBE",
     "true", "Enable Branch Record Buffer Extension">;
 
 def FeatureSPE_EEF : SubtargetFeature<"spe-eef", "HasSPE_EEF",
     "true", "Enable extra register in the Statistical Profiling Extension">;
 
 def FeatureFineGrainedTraps : SubtargetFeature<"fgt", "HasFineGrainedTraps",
     "true", "Enable fine grained virtualization traps extension">;
def HasV8_6aOps : SubtargetFeature<
    FeatureEnhancedCounterVirtualization, FeatureMatMulInt8]>;
 
 def HasV8_7aOps : SubtargetFeature<
   "v8.7a", "HasV8_7aOps", "true", "Support ARM v8.7a instructions",
   [HasV8_6aOps, FeatureXS, FeatureWFxT, FeatureHCX]>;
 
 def HasV8_8aOps : SubtargetFeature<
   "v8.8a", "HasV8_8aOps", "true", "Support ARM v8.8a instructions",
-  [HasV8_7aOps]>;
+  [HasV8_7aOps, FeatureHBC]>;
 
 def HasV9_0aOps : SubtargetFeature<
   "v9a", "HasV9_0aOps", "true", "Support ARM v9a instructions",
   [HasV8_5aOps, FeatureSVE2]>;
 
 def HasV9_1aOps : SubtargetFeature<
   "v9.1a", "HasV9_1aOps", "true", "Support ARM v9.1a instructions",
   [HasV8_6aOps, HasV9_0aOps]>;

llvm/lib/Target/AArch64/AArch64InstrFormats.td

 def am_brcond : Operand<OtherVT> {
   let EncoderMethod = "getCondBranchTargetOpValue";
   let DecoderMethod = "DecodePCRelLabel19";
   let PrintMethod = "printAlignedLabel";
   let ParserMatchClass = PCRelLabel19Operand;
   let OperandType = "OPERAND_PCREL";
 }
 
-class BranchCond : I<(outs), (ins ccode:$cond, am_brcond:$target),
-                     "b", ".$cond\t$target", "",
-                     [(AArch64brcond bb:$target, imm:$cond, NZCV)]>,
-                   Sched<[WriteBr]> {
+class BranchCond<bit bit4, string mnemonic>
+   : I<(outs), (ins ccode:$cond, am_brcond:$target),
+       mnemonic, ".$cond\t$target", "",
+       [(AArch64brcond bb:$target, imm:$cond, NZCV)]>, Sched<[WriteBr]> {
   let isBranch = 1;
   let isTerminator = 1;
   let Uses = [NZCV];
 
   bits<4> cond;
   bits<19> target;
   let Inst{31-24} = 0b01010100;
   let Inst{23-5} = target;
-  let Inst{4} = 0;
+  let Inst{4} = bit4;
   let Inst{3-0} = cond;
 }
 
 //---
 // Compare-and-branch instructions.
 //---
 class BaseCmpBranch<RegisterClass regtype, bit op, string asm, SDNode node>
     : I<(outs), (ins regtype:$Rt, am_brcond:$target),

llvm/lib/Target/AArch64/AArch64InstrInfo.td

 def HasWFxT          : Predicate<"Subtarget->hasWFxT()">,
                        AssemblerPredicate<(all_of FeatureWFxT), "wfxt">;
 def HasLS64          : Predicate<"Subtarget->hasLS64()">,
                        AssemblerPredicate<(all_of FeatureLS64), "ls64">;
 def HasBRBE          : Predicate<"Subtarget->hasBRBE()">,
                        AssemblerPredicate<(all_of FeatureBRBE), "brbe">;
 def HasSPE_EEF       : Predicate<"Subtarget->hasSPE_EEF()">,
                        AssemblerPredicate<(all_of FeatureSPE_EEF), "spe-eef">;
+def HasHBC            : Predicate<"Subtarget->hasHBC()">,
+                       AssemblerPredicate<(all_of FeatureHBC), "hbc">;
 def IsLE             : Predicate<"Subtarget->isLittleEndian()">;
 def IsBE             : Predicate<"!Subtarget->isLittleEndian()">;
 def IsWindows        : Predicate<"Subtarget->isTargetWindows()">;
 def UseExperimentalZeroingPseudos
     : Predicate<"Subtarget->useExperimentalZeroingPseudos()">;
 def UseAlternateSExtLoadCVTF32
     : Predicate<"Subtarget->useAlternateSExtLoadCVTF32Pattern()">;
 
: Pseudo<(outs), (ins i64imm:$sym),
              [(AArch64tlsdesc_callseq tglobaltlsaddr:$sym)]>,
       Sched<[WriteI, WriteLD, WriteI, WriteBrReg]>;
 def : Pat<(AArch64tlsdesc_callseq texternalsym:$sym),
           (TLSDESC_CALLSEQ texternalsym:$sym)>;
 
 //===----------------------------------------------------------------------===//
 // Conditional branch (immediate) instruction.
 //===----------------------------------------------------------------------===//
-def Bcc : BranchCond;
+def Bcc : BranchCond<0, "b">;
+
+// Armv8.8-A variant form which hints to the branch predictor that
+// this branch is very likely to go the same way nearly all the time
+// (even though it is not known at compile time _which_ way that is).
+def BCcc : BranchCond<1, "bc">, Requires<[HasHBC]>;
 
 //===----------------------------------------------------------------------===//
 // Compare-and-branch instructions.
 //===----------------------------------------------------------------------===//
 defm CBZ  : CmpBranch<0, "cbz", AArch64cbz>;
 defm CBNZ : CmpBranch<1, "cbnz", AArch64cbnz>;
 
 //===----------------------------------------------------------------------===//

llvm/lib/Target/AArch64/AArch64Subtarget.h

protected:
   bool HasEnhancedCounterVirtualization = false;
 
   // Armv8.7-A Extensions
   bool HasXS = false;
   bool HasWFxT = false;
   bool HasHCX = false;
   bool HasLS64 = false;
 
+  // Armv8.8-A Extensions
+  bool HasHBC = false;
+
   // Arm SVE2 extensions
   bool HasSVE2 = false;
   bool HasSVE2AES = false;
   bool HasSVE2SM4 = false;
   bool HasSVE2SHA3 = false;
   bool HasSVE2BitPerm = false;
 
   // Armv9-A Extensions
public:
   bool hasHCX() const { return HasHCX; }
   bool hasLS64() const { return HasLS64; }
   bool hasSEL2() const { return HasSEL2; }
   bool hasTLB_RMI() const { return HasTLB_RMI; }
   bool hasFlagM() const { return HasFlagM; }
   bool hasRCPC_IMMO() const { return HasRCPC_IMMO; }
   bool hasEL2VMSA() const { return HasEL2VMSA; }
   bool hasEL3() const { return HasEL3; }
+  bool hasHBC() const { return HasHBC; }
 
   bool fixCortexA53_835769() const { return FixCortexA53_835769; }
 
   bool addrSinkUsingGEPs() const override {
     // Keeping GEPs inbounds is important for exploiting AArch64
     // addressing-modes in ILP32 mode.
     return useAA() || isTargetILP32();
   }

llvm/lib/Target/AArch64/AsmParser/AArch64AsmParser.cpp

bool AArch64AsmParser::ParseInstruction(ParseInstructionInfo &Info,
   if (Head == "ic" || Head == "dc" || Head == "at" || Head == "tlbi" ||
       Head == "cfp" || Head == "dvp" || Head == "cpp")
     return parseSysAlias(Head, NameLoc, Operands);
 
   Operands.push_back(AArch64Operand::CreateToken(Head, NameLoc, getContext()));
   Mnemonic = Head;
 
   // Handle condition codes for a branch mnemonic
-  if (Head == "b" && Next != StringRef::npos) {
+  if ((Head == "b" || Head == "bc") && Next != StringRef::npos) {
     Start = Next;
     Next = Name.find('.', Start + 1);
     Head = Name.slice(Start + 1, Next);
 
     SMLoc SuffixLoc = SMLoc::getFromPointer(NameLoc.getPointer() +
                                             (Head.data() - Name.data()));
     AArch64CC::CondCode CC = parseCondCodeString(Head);
     if (CC == AArch64CC::Invalid)

llvm/test/MC/AArch64/armv8.8a-hbc.s

+// RUN: llvm-mc -triple aarch64-none-linux-gnu -show-encoding -mattr=+hbc < %s | FileCheck %s
+// RUN: llvm-mc -triple aarch64-none-linux-gnu -show-encoding -mattr=+v8.8a < %s | FileCheck %s
+// RUN: not llvm-mc -triple aarch64-none-linux-gnu < %s 2>&1 | FileCheck --check-prefix=CHECK-NO-HBC-ERR %s
+
+lbl:
+        bc.eq lbl
+        bc.ne lbl
+        bc.cs lbl
+        bc.hs lbl
+        bc.lo lbl
+        bc.cc lbl
+        bc.mi lbl
+        bc.pl lbl
+        bc.vs lbl
+        bc.vc lbl
+        bc.hi lbl
+        bc.ls lbl
+        bc.ge lbl
+        bc.lt lbl
+        bc.gt lbl
+        bc.le lbl
+        bc.al lbl
+
+// CHECK: bc.eq lbl                    // encoding: [0bAAA10000,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.ne lbl                    // encoding: [0bAAA10001,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.hs lbl                    // encoding: [0bAAA10010,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.hs lbl                    // encoding: [0bAAA10010,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.lo lbl                    // encoding: [0bAAA10011,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.lo lbl                    // encoding: [0bAAA10011,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.mi lbl                    // encoding: [0bAAA10100,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.pl lbl                    // encoding: [0bAAA10101,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.vs lbl                    // encoding: [0bAAA10110,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.vc lbl                    // encoding: [0bAAA10111,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.hi lbl                    // encoding: [0bAAA11000,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.ls lbl                    // encoding: [0bAAA11001,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.ge lbl                    // encoding: [0bAAA11010,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.lt lbl                    // encoding: [0bAAA11011,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.gt lbl                    // encoding: [0bAAA11100,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.le lbl                    // encoding: [0bAAA11101,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+// CHECK: bc.al lbl                    // encoding: [0bAAA11110,A,A,0x54]
+// CHECK:                              //   fixup A - offset: 0, value: lbl, kind: fixup_aarch64_pcrel_branch19
+
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc
+// CHECK-NO-HBC-ERR: [[@LINE-53]]:9: error: instruction requires: hbc

llvm/test/MC/Disassembler/AArch64/armv8.8a-hbc.txt

+# RUN: llvm-mc -triple=aarch64 -mattr=+hbc -disassemble %s | FileCheck %s
+# RUN: llvm-mc -triple=aarch64 -mattr=+v8.8a -disassemble %s | FileCheck %s
+# RUN: not llvm-mc -triple=aarch64 -disassemble %s 2>&1 | FileCheck --check-prefix=ERROR-NO-HBC %s
+
+[0x30,0x00,0x00,0x54]
+# CHECK: bc.eq #4
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x51,0x00,0x00,0x54]
+# CHECK: bc.ne #8
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x92,0x00,0x00,0x54]
+# CHECK: bc.hs #16
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x13,0x01,0x00,0x54]
+# CHECK: bc.lo #32
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x14,0x02,0x00,0x54]
+# CHECK: bc.mi #64
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x15,0x04,0x00,0x54]
+# CHECK: bc.pl #128
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x16,0x08,0x00,0x54]
+# CHECK: bc.vs #256
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x17,0x10,0x00,0x54]
+# CHECK: bc.vc #512
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x18,0x20,0x00,0x54]
+# CHECK: bc.hi #1024
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x19,0x40,0x00,0x54]
+# CHECK: bc.ls #2048
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x1a,0x80,0x00,0x54]
+# CHECK: bc.ge #4096
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x1b,0x00,0x01,0x54]
+# CHECK: bc.lt #8192
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x1c,0x00,0x02,0x54]
+# CHECK: bc.gt #16384
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x1d,0x00,0x04,0x54]
+# CHECK: bc.le #32768
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding
+[0x1e,0x00,0x08,0x54]
+# CHECK: bc.al #65536
+# ERROR-NO-HBC: [[@LINE-2]]:2: warning: invalid instruction encoding