Skip to content

Commit

Permalink
[X86] Added support for nocf_check attribute for indirect Branch Trac…
Browse files Browse the repository at this point in the history
…king

X86 Supports Indirect Branch Tracking (IBT) as part of Control-Flow Enforcement Technology (CET).
IBT instruments ENDBR instructions used to specify valid targets of indirect call / jmp.
The `nocf_check` attribute has two roles in the context of X86 IBT technology:
	1. Appertains to a function - do not add ENDBR instruction at the beginning of the function.
	2. Appertains to a function pointer - do not track the target function of this pointer by adding nocf_check prefix to the indirect-call instruction.

This patch implements `nocf_check` context for Indirect Branch Tracking.
It also auto generates `nocf_check` prefixes before indirect branchs to jump tables that are guarded by range checks.

Differential Revision: https://reviews.llvm.org/D41879

llvm-svn: 327767
  • Loading branch information
Oren Ben Simhon committed Mar 17, 2018
1 parent dbcf1bf commit fdd72fd
Showing 34 changed files with 302 additions and 76 deletions.
1 change: 1 addition & 0 deletions llvm/bindings/go/llvm/ir_test.go
Original file line number Diff line number Diff line change
@@ -89,6 +89,7 @@ func TestAttributes(t *testing.T) {
"uwtable",
"zeroext",
"cold",
"nocf_check"
}

for _, name := range attrTests {
1 change: 1 addition & 0 deletions llvm/docs/BitCodeFormat.rst
Original file line number Diff line number Diff line change
@@ -1054,6 +1054,7 @@ The integer codes are mapped to well-known attributes as follows.
* code 53: ``speculatable``
* code 54: ``strictfp``
* code 55: ``sanitize_hwaddress``
* code 56: ``nocf_check``

.. note::
The ``allocsize`` attribute has a special encoding for its arguments. Its two
6 changes: 6 additions & 0 deletions llvm/docs/LangRef.rst
Original file line number Diff line number Diff line change
@@ -1699,6 +1699,12 @@ example:
show that no exceptions passes by it. This is normally the case for
the ELF x86-64 abi, but it can be disabled for some compilation
units.
``nocf_check``
This attribute indicates that no control-flow check will be perfomed on
the attributed entity. It disables -fcf-protection=<> for a specific
entity to fine grain the HW control flow protection mechanism. The flag
is target independant and currently appertains to a function or function
pointer.

.. _glattrs:

1 change: 1 addition & 0 deletions llvm/include/llvm/Bitcode/LLVMBitCodes.h
Original file line number Diff line number Diff line change
@@ -587,6 +587,7 @@ enum AttributeKindCodes {
ATTR_KIND_SPECULATABLE = 53,
ATTR_KIND_STRICT_FP = 54,
ATTR_KIND_SANITIZE_HWADDRESS = 55,
ATTR_KIND_NOCF_CHECK = 56,
};

enum ComdatSelectionKindCodes {
7 changes: 7 additions & 0 deletions llvm/include/llvm/CodeGen/TargetLowering.h
Original file line number Diff line number Diff line change
@@ -3584,6 +3584,13 @@ class TargetLowering : public TargetLoweringBase {
virtual SDValue LowerToTLSEmulatedModel(const GlobalAddressSDNode *GA,
SelectionDAG &DAG) const;

/// Expands target specific indirect branch for the case of JumpTable
/// expanasion.
virtual SDValue expandIndirectJTBranch(const SDLoc& dl, SDValue Value, SDValue Addr,
SelectionDAG &DAG) const {
return DAG.getNode(ISD::BRIND, dl, MVT::Other, Value, Addr);
}

// seteq(x, 0) -> truncate(srl(ctlz(zext(x)), log2(#bits)))
// If we're comparing for equality to zero and isCtlzFast is true, expose the
// fact that this can be implemented as a ctlz/srl pair, so that the dag
3 changes: 3 additions & 0 deletions llvm/include/llvm/IR/Attributes.td
Original file line number Diff line number Diff line change
@@ -106,6 +106,9 @@ def NoRedZone : EnumAttr<"noredzone">;
/// Mark the function as not returning.
def NoReturn : EnumAttr<"noreturn">;

/// Disable Indirect Branch Tracking.
def NoCfCheck : EnumAttr<"nocf_check">;

/// Function doesn't unwind stack.
def NoUnwind : EnumAttr<"nounwind">;

3 changes: 3 additions & 0 deletions llvm/include/llvm/IR/Function.h
Original file line number Diff line number Diff line change
@@ -494,6 +494,9 @@ class Function : public GlobalObject, public ilist_node<Function> {
addFnAttr(Attribute::NoReturn);
}

/// Determine if the function should not perform indirect branch tracking.
bool doesNoCfCheck() const { return hasFnAttribute(Attribute::NoCfCheck); }

/// @brief Determine if the function cannot unwind.
bool doesNotThrow() const {
return hasFnAttribute(Attribute::NoUnwind);
12 changes: 11 additions & 1 deletion llvm/include/llvm/IR/Instructions.h
Original file line number Diff line number Diff line change
@@ -1795,6 +1795,9 @@ class CallBase : public CallBaseParent<InstTy>::type,
addAttribute(AttributeList::FunctionIndex, Attribute::NoReturn);
}

/// Determine if the call should not perform indirect branch tracking.
bool doesNoCfCheck() const { return hasFnAttr(Attribute::NoCfCheck); }

/// Determine if the call cannot unwind.
bool doesNotThrow() const { return hasFnAttr(Attribute::NoUnwind); }
void setDoesNotThrow() {
@@ -3830,8 +3833,15 @@ class InvokeInst : public CallBase<InvokeInst> {
static InvokeInst *Create(InvokeInst *II, ArrayRef<OperandBundleDef> Bundles,
Instruction *InsertPt = nullptr);

/// Determine if the call should not perform indirect branch tracking.
bool doesNoCfCheck() const { return hasFnAttr(Attribute::NoCfCheck); }


/// Determine if the call cannot unwind.
bool doesNotThrow() const { return hasFnAttr(Attribute::NoUnwind); }
void setDoesNotThrow() {
addAttribute(AttributeList::FunctionIndex, Attribute::NoUnwind);
}

/// Return the function called, or null if this is an
/// indirect function invocation.
///
1 change: 1 addition & 0 deletions llvm/lib/AsmParser/LLLexer.cpp
Original file line number Diff line number Diff line change
@@ -648,6 +648,7 @@ lltok::Kind LLLexer::LexIdentifier() {
KEYWORD(nonnull);
KEYWORD(noredzone);
KEYWORD(noreturn);
KEYWORD(nocf_check);
KEYWORD(nounwind);
KEYWORD(optnone);
KEYWORD(optsize);
3 changes: 3 additions & 0 deletions llvm/lib/AsmParser/LLParser.cpp
Original file line number Diff line number Diff line change
@@ -1131,6 +1131,7 @@ bool LLParser::ParseFnAttributeValuePairs(AttrBuilder &B,
case lltok::kw_nonlazybind: B.addAttribute(Attribute::NonLazyBind); break;
case lltok::kw_noredzone: B.addAttribute(Attribute::NoRedZone); break;
case lltok::kw_noreturn: B.addAttribute(Attribute::NoReturn); break;
case lltok::kw_nocf_check: B.addAttribute(Attribute::NoCfCheck); break;
case lltok::kw_norecurse: B.addAttribute(Attribute::NoRecurse); break;
case lltok::kw_nounwind: B.addAttribute(Attribute::NoUnwind); break;
case lltok::kw_optnone: B.addAttribute(Attribute::OptimizeNone); break;
@@ -1468,6 +1469,7 @@ bool LLParser::ParseOptionalParamAttrs(AttrBuilder &B) {
case lltok::kw_nonlazybind:
case lltok::kw_noredzone:
case lltok::kw_noreturn:
case lltok::kw_nocf_check:
case lltok::kw_nounwind:
case lltok::kw_optnone:
case lltok::kw_optsize:
@@ -1561,6 +1563,7 @@ bool LLParser::ParseOptionalReturnAttrs(AttrBuilder &B) {
case lltok::kw_nonlazybind:
case lltok::kw_noredzone:
case lltok::kw_noreturn:
case lltok::kw_nocf_check:
case lltok::kw_nounwind:
case lltok::kw_optnone:
case lltok::kw_optsize:
1 change: 1 addition & 0 deletions llvm/lib/AsmParser/LLToken.h
Original file line number Diff line number Diff line change
@@ -199,6 +199,7 @@ enum Kind {
kw_nonnull,
kw_noredzone,
kw_noreturn,
kw_nocf_check,
kw_nounwind,
kw_optnone,
kw_optsize,
3 changes: 3 additions & 0 deletions llvm/lib/Bitcode/Reader/BitcodeReader.cpp
Original file line number Diff line number Diff line change
@@ -1160,6 +1160,7 @@ static uint64_t getRawAttributeMask(Attribute::AttrKind Val) {
case Attribute::Speculatable: return 1ULL << 54;
case Attribute::StrictFP: return 1ULL << 55;
case Attribute::SanitizeHWAddress: return 1ULL << 56;
case Attribute::NoCfCheck: return 1ULL << 57;
case Attribute::Dereferenceable:
llvm_unreachable("dereferenceable attribute not supported in raw format");
break;
@@ -1338,6 +1339,8 @@ static Attribute::AttrKind getAttrFromCode(uint64_t Code) {
return Attribute::NoRedZone;
case bitc::ATTR_KIND_NO_RETURN:
return Attribute::NoReturn;
case bitc::ATTR_KIND_NOCF_CHECK:
return Attribute::NoCfCheck;
case bitc::ATTR_KIND_NO_UNWIND:
return Attribute::NoUnwind;
case bitc::ATTR_KIND_OPTIMIZE_FOR_SIZE:
2 changes: 2 additions & 0 deletions llvm/lib/Bitcode/Writer/BitcodeWriter.cpp
Original file line number Diff line number Diff line change
@@ -638,6 +638,8 @@ static uint64_t getAttrKindEncoding(Attribute::AttrKind Kind) {
return bitc::ATTR_KIND_NO_RED_ZONE;
case Attribute::NoReturn:
return bitc::ATTR_KIND_NO_RETURN;
case Attribute::NoCfCheck:
return bitc::ATTR_KIND_NOCF_CHECK;
case Attribute::NoUnwind:
return bitc::ATTR_KIND_NO_UNWIND;
case Attribute::OptimizeForSize:
3 changes: 2 additions & 1 deletion llvm/lib/CodeGen/SelectionDAG/LegalizeDAG.cpp
Original file line number Diff line number Diff line change
@@ -3706,7 +3706,8 @@ bool SelectionDAGLegalize::ExpandNode(SDNode *Node) {
Addr = DAG.getNode(ISD::ADD, dl, PTy, Addr,
TLI.getPICJumpTableRelocBase(Table, DAG));
}
Tmp1 = DAG.getNode(ISD::BRIND, dl, MVT::Other, LD.getValue(1), Addr);

Tmp1 = TLI.expandIndirectJTBranch(dl, LD.getValue(1), Addr, DAG);
Results.push_back(Tmp1);
break;
}
2 changes: 2 additions & 0 deletions llvm/lib/IR/Attributes.cpp
Original file line number Diff line number Diff line change
@@ -299,6 +299,8 @@ std::string Attribute::getAsString(bool InAttrGrp) const {
return "noredzone";
if (hasAttribute(Attribute::NoReturn))
return "noreturn";
if (hasAttribute(Attribute::NoCfCheck))
return "nocf_check";
if (hasAttribute(Attribute::NoRecurse))
return "norecurse";
if (hasAttribute(Attribute::NoUnwind))
1 change: 1 addition & 0 deletions llvm/lib/IR/Verifier.cpp
Original file line number Diff line number Diff line change
@@ -1404,6 +1404,7 @@ Verifier::visitModuleFlag(const MDNode *Op,
static bool isFuncOnlyAttr(Attribute::AttrKind Kind) {
switch (Kind) {
case Attribute::NoReturn:
case Attribute::NoCfCheck:
case Attribute::NoUnwind:
case Attribute::NoInline:
case Attribute::AlwaysInline:
7 changes: 4 additions & 3 deletions llvm/lib/Target/X86/AsmParser/X86AsmParser.cpp
Original file line number Diff line number Diff line change
@@ -2353,21 +2353,22 @@ bool X86AsmParser::ParseInstruction(ParseInstructionInfo &Info, StringRef Name,
.Cases("acquire", "release", isParsingIntelSyntax())
.Default(false);

auto isLockRepeatPrefix = [](StringRef N) {
auto isLockRepeatNtPrefix = [](StringRef N) {
return StringSwitch<bool>(N)
.Cases("lock", "rep", "repe", "repz", "repne", "repnz", true)
.Cases("lock", "rep", "repe", "repz", "repne", "repnz", "notrack", true)
.Default(false);
};

bool CurlyAsEndOfStatement = false;

unsigned Flags = X86::IP_NO_PREFIX;
while (isLockRepeatPrefix(Name.lower())) {
while (isLockRepeatNtPrefix(Name.lower())) {
unsigned Prefix =
StringSwitch<unsigned>(Name)
.Cases("lock", "lock", X86::IP_HAS_LOCK)
.Cases("rep", "repe", "repz", X86::IP_HAS_REPEAT)
.Cases("repne", "repnz", X86::IP_HAS_REPEAT_NE)
.Cases("notrack", "notrack", X86::IP_HAS_NOTRACK)
.Default(X86::IP_NO_PREFIX); // Invalid prefix (impossible)
Flags |= Prefix;
if (getLexer().is(AsmToken::EndOfStatement)) {
3 changes: 3 additions & 0 deletions llvm/lib/Target/X86/InstPrinter/X86ATTInstPrinter.cpp
Original file line number Diff line number Diff line change
@@ -54,6 +54,9 @@ void X86ATTInstPrinter::printInst(const MCInst *MI, raw_ostream &OS,
if ((TSFlags & X86II::LOCK) || (Flags & X86::IP_HAS_LOCK))
OS << "\tlock\t";

if ((TSFlags & X86II::NOTRACK) || (Flags & X86::IP_HAS_NOTRACK))
OS << "\tnotrack\t";

if (Flags & X86::IP_HAS_REPEAT_NE)
OS << "\trepne\t";
else if (Flags & X86::IP_HAS_REPEAT)
3 changes: 3 additions & 0 deletions llvm/lib/Target/X86/InstPrinter/X86IntelInstPrinter.cpp
Original file line number Diff line number Diff line change
@@ -49,6 +49,9 @@ void X86IntelInstPrinter::printInst(const MCInst *MI, raw_ostream &OS,
else if (Flags & X86::IP_HAS_REPEAT)
OS << "\trep\t";

if ((TSFlags & X86II::NOTRACK) || (Flags & X86::IP_HAS_NOTRACK))
OS << "\tnotrack\t";

printInstruction(MI, OS);

// Next always print the annotation.
11 changes: 8 additions & 3 deletions llvm/lib/Target/X86/MCTargetDesc/X86BaseInfo.h
Original file line number Diff line number Diff line change
@@ -60,8 +60,9 @@ namespace X86 {
IP_HAS_REPEAT_NE = 4,
IP_HAS_REPEAT = 8,
IP_HAS_LOCK = 16,
NO_SCHED_INFO = 32 // Don't add sched comment to the current instr because
// it was already added
NO_SCHED_INFO = 32, // Don't add sched comment to the current instr because
// it was already added
IP_HAS_NOTRACK = 64
};
} // end namespace X86;

@@ -572,7 +573,11 @@ namespace X86II {

/// Explicitly specified rounding control
EVEX_RCShift = Has3DNow0F0FOpcodeShift + 1,
EVEX_RC = 1ULL << EVEX_RCShift
EVEX_RC = 1ULL << EVEX_RCShift,

// NOTRACK prefix
NoTrackShift = EVEX_RCShift + 1,
NOTRACK = 1ULL << NoTrackShift
};

// getBaseOpcodeFor - This function returns the "base" X86 opcode for the
4 changes: 4 additions & 0 deletions llvm/lib/Target/X86/MCTargetDesc/X86MCCodeEmitter.cpp
Original file line number Diff line number Diff line change
@@ -1134,6 +1134,10 @@ bool X86MCCodeEmitter::emitOpcodePrefix(uint64_t TSFlags, unsigned &CurByte,
if (TSFlags & X86II::LOCK || MI.getFlags() & X86::IP_HAS_LOCK)
EmitByte(0xF0, CurByte, OS);

// Emit the NOTRACK opcode prefix.
if (TSFlags & X86II::NOTRACK || MI.getFlags() & X86::IP_HAS_NOTRACK)
EmitByte(0x3E, CurByte, OS);

switch (TSFlags & X86II::OpPrefixMask) {
case X86II::PD: // 66
EmitByte(0x66, CurByte, OS);
7 changes: 7 additions & 0 deletions llvm/lib/Target/X86/X86FastISel.cpp
Original file line number Diff line number Diff line change
@@ -3174,6 +3174,13 @@ bool X86FastISel::fastLowerCall(CallLoweringInfo &CLI) {
CLI.CS ? dyn_cast<CallInst>(CLI.CS->getInstruction()) : nullptr;
const Function *CalledFn = CI ? CI->getCalledFunction() : nullptr;

// Call / invoke instructions with NoCfCheck attribute require special
// handling.
const auto *II =
CLI.CS ? dyn_cast<InvokeInst>(CLI.CS->getInstruction()) : nullptr;
if ((CI && CI->doesNoCfCheck()) || (II && II->doesNoCfCheck()))
return false;

// Functions with no_caller_saved_registers that need special handling.
if ((CI && CI->hasFnAttr("no_caller_saved_registers")) ||
(CalledFn && CalledFn->hasFnAttribute("no_caller_saved_registers")))
29 changes: 28 additions & 1 deletion llvm/lib/Target/X86/X86ISelLowering.cpp
Original file line number Diff line number Diff line change
@@ -3415,6 +3415,11 @@ X86TargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
const Function *Fn = CI ? CI->getCalledFunction() : nullptr;
bool HasNCSR = (CI && CI->hasFnAttr("no_caller_saved_registers")) ||
(Fn && Fn->hasFnAttribute("no_caller_saved_registers"));
const auto *II = dyn_cast_or_null<InvokeInst>(CLI.CS.getInstruction());
bool HasNoCfCheck =
(CI && CI->doesNoCfCheck()) || (II && II->doesNoCfCheck());
const Module *M = MF.getMMI().getModule();
Metadata *IsCFProtectionSupported = M->getModuleFlag("cf-protection-branch");

if (CallConv == CallingConv::X86_INTR)
report_fatal_error("X86 interrupts may not be called directly");
@@ -3898,7 +3903,11 @@ X86TargetLowering::LowerCall(TargetLowering::CallLoweringInfo &CLI,
return DAG.getNode(X86ISD::TC_RETURN, dl, NodeTys, Ops);
}

Chain = DAG.getNode(X86ISD::CALL, dl, NodeTys, Ops);
if (HasNoCfCheck && IsCFProtectionSupported) {
Chain = DAG.getNode(X86ISD::NT_CALL, dl, NodeTys, Ops);
} else {
Chain = DAG.getNode(X86ISD::CALL, dl, NodeTys, Ops);
}
InFlag = Chain.getValue(1);

// Create the CALLSEQ_END node.
@@ -25852,6 +25861,8 @@ const char *X86TargetLowering::getTargetNodeName(unsigned Opcode) const {
case X86ISD::GF2P8MULB: return "X86ISD::GF2P8MULB";
case X86ISD::GF2P8AFFINEQB: return "X86ISD::GF2P8AFFINEQB";
case X86ISD::GF2P8AFFINEINVQB: return "X86ISD::GF2P8AFFINEINVQB";
case X86ISD::NT_CALL: return "X86ISD::NT_CALL";
case X86ISD::NT_BRIND: return "X86ISD::NT_BRIND";
}
return nullptr;
}
@@ -38709,6 +38720,22 @@ void X86TargetLowering::finalizeLowering(MachineFunction &MF) const {
TargetLoweringBase::finalizeLowering(MF);
}

SDValue X86TargetLowering::expandIndirectJTBranch(const SDLoc& dl,
SDValue Value, SDValue Addr,
SelectionDAG &DAG) const {
const Module *M = DAG.getMachineFunction().getMMI().getModule();
Metadata *IsCFProtectionSupported = M->getModuleFlag("cf-protection-branch");
if (IsCFProtectionSupported) {
// In case control-flow branch protection is enabled, we need to add
// notrack prefix to the indirect branch.
// In order to do that we create NT_BRIND SDNode.
// Upon ISEL, the pattern will convert it to jmp with NoTrack prefix.
return DAG.getNode(X86ISD::NT_BRIND, dl, MVT::Other, Value, Addr);
}

return TargetLowering::expandIndirectJTBranch(dl, Value, Addr, DAG);
}

/// This method query the target whether it is beneficial for dag combiner to
/// promote the specified node. If true, it should return the desired promotion
/// type by reference.
12 changes: 11 additions & 1 deletion llvm/lib/Target/X86/X86ISelLowering.h
Original file line number Diff line number Diff line change
@@ -75,6 +75,9 @@ namespace llvm {
///
CALL,

/// Same as call except it adds the NoTrack prefix.
NT_CALL,

/// This operation implements the lowering for readcyclecounter.
RDTSC_DAG,

@@ -122,6 +125,10 @@ namespace llvm {
/// or TEST instruction.
BRCOND,

/// BRIND node with NoTrack prefix. Operand 0 is the chain operand and
/// operand 1 is the target address.
NT_BRIND,

/// Return with a flag operand. Operand 0 is the chain operand, operand
/// 1 is the number of bytes of stack to pop.
RET_FLAG,
@@ -1114,9 +1121,12 @@ namespace llvm {
bool lowerInterleavedStore(StoreInst *SI, ShuffleVectorInst *SVI,
unsigned Factor) const override;


void finalizeLowering(MachineFunction &MF) const override;

SDValue expandIndirectJTBranch(const SDLoc& dl, SDValue Value,
SDValue Addr, SelectionDAG &DAG)
const override;

protected:
std::pair<const TargetRegisterClass *, uint8_t>
findRepresentativeClass(const TargetRegisterInfo *TRI,
Loading

0 comments on commit fdd72fd

Please sign in to comment.