Add Forward-Edge Control-Flow Integrity support

[tmroeder]

Summary

This patch adds a new pass that can inject checks before indirect calls to make sure that these calls target known locations. It supports three types of checks and, at compile time, it can take the name of a custom function to call when an indirect call check fails.

This foward CFI implementation depends on the recent jumptable attribute, as well as http://reviews.llvm.org/D4128, which adds a flag to cause all address-taken functions to get the attributes unnamed_addr and jumptable.

This CFI implementation can also easily be augmented with ExternalFunctionAnalysis (http://reviews.llvm.org/D2873); this allows the rewriter to skip rewriting for functions that are known at compile time to take functions defined outside the current module. Another possibility for integration with EFA is to call a different failure function for sites that use known external function pointers than for the other indirect call check sites.

See http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-February/070210 for the initial discussion of this feature on llvmdev.

This pass incidentally moves the function JumpInstrTables::transformType from private to public and makes it static (with a new argument that specifies the table type to use); this is so that the CFI code can transform function types at call sites to determine which jump-instruction table to use for the check at that site.

[tmroeder]

This update to forward-edge control-flow integrity changes the jump-table-all flag to fcfi and uses it to set the JumpTableAll flag in JumpInstrTables. This also makes the FCFI pass optional based on the setting of TargetOptions.FCFI, which is controlled by the fcfi flag.

This change also removes the .globl attribute from jumptable entries and from the generated __llvm_cfi_pointer_warning function. This prevents symbol conflicts between two objects compiled with FCFI.

[tmroeder]

This changes the FCFI code to drop all mention of JumpTableAll, as per the discussion on D4128 on llvm-commits; it will be up to the front end to set jumptable and unnamed_addr on functions that should be transformed by CFI.

[nicholas]

include/llvm/CodeGen/CommandFlags.h

240

Typo: funcion -> function
Grammar, "... pass a the violation ..."

include/llvm/CodeGen/ForwardControlFlowIntegrity.h

52

IIRC, this is a debug-ish string, used for things like -ftime-report output. Feel free to add spaces between the words (a quick survey shows that we do it both with spaces and with StudyCaps in LLVM).

59–64

C++ please. :) Just "struct CFIConstants { ... };" will do.

include/llvm/Target/TargetOptions.h

76–80

I was going to point out that you could use CFIFuncName() instead, but then I noticed TrapFuncName(""). Feel free to fix both of them.

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

903

Could you explain this change a little more? And the deletion of:
507 OutStreamer.EmitSymbolAttribute(FunSym, MCSA_Global);
?

Does this cause us to emit an unnecessary section switch even when jump tables are off?

lib/CodeGen/ForwardControlFlowIntegrity.cpp

119–122

for (BasicBlock &BB : F) {
  for (Instruction &I : BB) {

? (Just use &I in place of I for the rest of this function.)

234–235

PointerType *CharPtrTy = Type::getInt8PtrTy(M.getContext());

which leaves CharTy dead and you can remove it.

test/CodeGen/X86/jump_tables.ll

91

Why the change in fill byte?

[kcc]

I've made a very quick glance. Before the second round I'd like to see tests that use opt (IR=>IR) instead of (or in addition to) llc tests (IR=>ASM).

Maybe in a separate patch I'd like to see end-to-end tests (like we have in compiler-rt/test).

off-topic: are you going to handle RET instructions?

lib/CodeGen/ForwardControlFlowIntegrity.cpp

69

Please use 'static' instead of anon namespace.
From http://llvm.org/docs/CodingStandards.html#anonymous-namespaces
make anonymous namespaces as small as possible, and only use them for class declarations.

test/CodeGen/X86/cfi_non_default_function.ll

11

add
CHECK-LABEL: @m

test/CodeGen/X86/cfi_simple_indirect_call.ll

13

use SUB-LABEL, etc

[kcc]

Ah, I see you can't use opt tests because you are actually running as part of llc, sorry.

[kcc]

some more comments.

include/llvm/CodeGen/ForwardControlFlowIntegrity.h

57

Can this be SmallVector?

lib/CodeGen/ForwardControlFlowIntegrity.cpp

106

maybe inline this in the header?

lib/CodeGen/JumpInstrTables.cpp

50

why did you remove .?

test/CodeGen/X86/cfi_simple_indirect_call.ll

13

also add a test with invoke

[tmroeder]

Comments on the current set of comments. I'm also about to update this with a patch that handles the other comments.

include/llvm/CodeGen/ForwardControlFlowIntegrity.h

57

I don't know SmallVector well, so I'm not sure. It's true that I don't need efficient search over this data structure, since it's just filled and iterated, so at least it could be a vector. Does SmallVector suffer if it gets large?

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

903

I've found that sometimes the jumptable code was getting emitted in a section that was not .text. It might be possible to fix this by emitting the code somewhere else, but it looked to me like each chunk of output was emitting a section switch if it needed to make sure it was in a given section. This shouldn't add an unnecessary section switch if there's no jumptable, since it's guarded by the condition

if (JITI && !JITI->getTables().empty())

There's no way to turn jumptable off, though, since we agreed in the original jumptable code review that the mere presence of the jumptable attribute should be enough to trigger the inclusion of a jumptable.

As for removing the MCSA_Global emission, this fixes a problem I noticed recently with separate compilation. As it stands, if you compile two different modules with jumptable annotations then try to link them, the link will fail, since both will have, e.g., __llvm_jump_instr_table_0_1 as global symbols. But that symbol doesn't need to be global, since it can only be referenced by the current module. Removing the MCSA_Global annotation allows multiple modules to be built with jumptable and to link correctly.

Note that pointers to __llvm_jump_instr_table_0_1 can be handed to other modules, but other modules can't directly refer to this pointer in their code; this was part of the whole problem with &f != <received pointer to f> that led to the requirement for unnamed_addr on all jumptable-annotated functions.

lib/CodeGen/ForwardControlFlowIntegrity.cpp

106

I was asked in a review for a different class to take the destructor out of the header and put it in the cc

[tmroeder]

This patch addresses the latest set of comments on the FCFI code.

[kcc]

include/llvm/CodeGen/ForwardControlFlowIntegrity.h

57

SmallVector becomes just std::vector when it becomes large.

[jfb]

include/llvm/CodeGen/CommandFlags.h

245

Could you clarify that the above two options are linked? Or just have cfi-func-name, implying a call to abort otherwise? It kind of seems like the default cfi-func could just call abort?

Also, what's the function's expected signature?

What happens if the cfi-func itself gets CFI instrumented, and it fails the check? *That* should probably call abort directly, not recurse :)

include/llvm/Target/TargetOptions.h

54

Use enum class.

lib/CodeGen/ForwardControlFlowIntegrity.cpp

147

When does this happen?

152

Same.

159

nullptr.

199

Could you fix this?

245

Shouldn't this be a fatal error?

279

I'd use a switch here, so LLVM warns if a new CFIntegrity is added and unhandled.

298

Shouldn't you get the pointer size from the Target here too?

355

nullptr.

lib/CodeGen/JumpInstrTables.cpp

264

nullptr is already the default for this function, no need to specify it.

[tmroeder]

include/llvm/CodeGen/CommandFlags.h

245

Is there an abort function that's guaranteed to be present for any IR? Right now, my "abort" is to execute a trap instruction.

lib/CodeGen/ForwardControlFlowIntegrity.cpp

152

I think you're right here and in the previous comment: this shouldn't ever happen.

199

I'd like to fix this, but I don't know where I can get this information in a principled manner. This requires the size of a jumptable entry, and that depends on the size of the jump instruction in bytes. In principle, this is available from MCInstrDesc::getSize(), but that returns 0 when I've tried it for JMP_4 on X86.

I can certainly encode it myself in the Targets, but that seems like dangerous duplication of functionality with the descriptions in the instruction tables.

298

It's not a question of pointer size but rather the size of an unconditional jump instruction. I haven't yet figured out where to get that information.

[jfb]

include/llvm/CodeGen/CommandFlags.h

245

Good point, there may not be a C library... Could you default to abort, and fall back to @llvm.trap if it doesn't exits?

lib/CodeGen/ForwardControlFlowIntegrity.cpp

199

Oh I see.

It's not just a single instruction on each target though? If so then the content of jumptable entries does seem pretty target specific, so it kind of sounds like something that would belong in the target. Size is kind of icky though, it would be nice to factor out the target's instruction pattern from their actual encoding.

[tmroeder]

lib/CodeGen/ForwardControlFlowIntegrity.cpp

199

Well, the instruction itself is already generated in the Target; in the jumptable patch, I added a pair of virtual methods to include/llvm/Target/TargetInstrInfo.h and instantiated it for X86 and ARM in their Target directories.

It does happen to be a single instruction on each architecture I've tried so far (X86, ARM, PPC), since it's just an unconditional branch to the target function. Maybe I should just add another method to TargetInstrInfo? But like I said, I don't know any clean way to get this from the encoding...I hope that's just my ignorance of the details of how Targets manage their instruction encodings: surely there must be a way to get the upper bound of the length of an unconditional jump instruction?

[tmroeder]

This version of the patch adds another virtual call to TargetInstrInfo to get a bound on the size of a jumptable entry, as discussed on llvmdev. This is used stored in JumpInstrTableInfo and used by ForwardControlFlowIntegrity to generate masks for jumps to jumptable entries. PTAL.

[jfb]

include/llvm/Analysis/JumpInstrTableInfo.h

41 ↗

(On Diff #11959)

I'm not a fan of default constructor parameters, but it seems like the right thing here.

The power-of-two invariant should also be specified here. Also, you should name the variable ByteAlign (it's pretty obvious it's not BitAlign, but I like explicit), as well as all the other Align.

61 ↗

(On Diff #11959)

This invariant isn't actually enforced in the code, instead the code tries to round up on use. I think it would be better to enforce the invariant on construction.

66 ↗

(On Diff #11959)

It's not clear what Bound is, it would be good to summarize TargetInstrInfo's explanation, or refer to it.

include/llvm/CodeGen/ForwardControlFlowIntegrity.h

83

What's the default function that ignores violations?

87

Power-of-2 restriction?

include/llvm/Target/TargetInstrInfo.h

342 ↗

(On Diff #11959)

The maximum of uncond branch and trap?

include/llvm/Target/TargetOptions.h

243

It looks like someone thought size was important above, since bitfields are used for bools. I'm not sure whether you should follow that lead (it would require reshuffling members around).

lib/Target/ARM/ARMBaseInstrInfo.cpp

4364 ↗

(On Diff #11959)

This comment (and the others below and in other files) isn't clear: what has to be kept in sync exactly?

4391 ↗

(On Diff #11959)

Thumb instructions are 2 or 4 bytes, and ARM instructions are always 4. The largest immediate on a direct branch has 26 bits, so that's insufficient for a lot of cases, you'll therefore need a PC-relative load followed by an indirect branch, and a location to store the 32-bit constant pool entry for the address. Assuming you do:

ldr rX, [PC, +#8]
bx rX
0xdeadbeef ; The address

Then you need 12 bytes, which I think is the worst case. Note that the constant pool entry could be elsewhere, preferably *not* in executable memory, but that would require another register and more infrastructure.

Note that you may want to use blx instead, if you intend on balancing the CPU's call/return stack: blx would be for calls, see the following page for returns

http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.ddi0438i/BABGEAEF.html

[tmroeder]

include/llvm/Analysis/JumpInstrTableInfo.h

61 ↗

(On Diff #11959)

Hmm...I thought I did the rounding in the createJumpInstrTableInfo; but I think you're right that it should be moved to the constructor.

include/llvm/Target/TargetInstrInfo.h

342 ↗

(On Diff #11959)

Right.

lib/Target/ARM/ARMBaseInstrInfo.cpp

4364 ↗

(On Diff #11959)

See my comment after getJumpInstrTableEntryBound: the bound in that function must be a bound on the possible instruction lengths returned by these two functions.

4391 ↗

(On Diff #11959)

I don't understand, but that's probably because I don't know ARM well. The goal here is to do an direct, unconditional branch using the code that's already checked in in the function getUnconditionalBranch. Are you saying that there are circumstances in which that code doesn't work?

If not, then are you saying that there are circumstances in which that code will generate this 12 byte sequence? The bound code needs to provide an upper bound for the sequences produced by getUnconditionalBranch and getTrap.

With respect to b vs blx, the goal is to perform an unconditional branch without touching any stacks at all; does b fail this condition? I'm trying to do the equivalent of the X86 jmp.

[jfb]

include/llvm/Target/TargetInstrInfo.h

342 ↗

(On Diff #11959)

Could you clarify the comment for this?

lib/Target/ARM/ARMBaseInstrInfo.cpp

4364 ↗

(On Diff #11959)

What I meant is that simply reading this comment doesn't provide the relevant information to understand it. It makes sense in this patch, but once checked in I'd have a WTF moment reading just this one comment on its own. You should probably refer the reader to the base class' getJumpInstrTableEntryBound.

4391 ↗

(On Diff #11959)

ARM has a fixed-width instruction encoding, and immediates are either encoded in the instruction itself or they have to be loaded if they overflow the storage available inside a fixed-width instruction. ARM doesn't have variable-bit-width instructions with very big immediates like x86 does.

26 bits is the biggest PC-relative immediate you can encode in a direct branch (for ARM, it's smaller for Thumb and Thumb2), so branches that go out of bounds from this must be indirect branches (or you can sprinkle the code with trampolines, ew).

So yes, for ARM the upper bounds would be 12 bytes if the sequence I suggest is used. For Thumb1 and Thumb2 it would be 8 (since LDR and BX can fit in 2 bytes each, and the immediate in 4), though for Thumb2 that'll depend on the register allocator using one of the first 8 registers (out of 16) for the LDR instruction and I'm not sure if LLVM will guarantee this.

On ARM branches:

• B is a direct branch, with a limited PC-relative immediate.
• BL and BLX are the same direct branches, but they change the link register (aka LR aka r14) meaning that the branch is a call (and LR is set to the address that needs to be returned to, which is the one right after the BL[X] instruction). The "X" means that the instruction set should be eXchanged (if it's currently ARM then go to Thumb, if it's Thumb then go to ARM) it's generally not something you'd want to do!
• BX is an indirect branch, and so is BLX (same name as the above BLX, but with a register operand).

All of these can be conditionalized, either straight in the instruction in ARM mode or with an IT block in Thumb mode. They can also be unconditional.

I recommend looking at the "ARM Architecture Reference Manual ARMv7-A and ARMv7-R edition". It's available from ARM's site behind a registration (or you can get it from work at arm-eng). See section A8.8.18 and later.

The point I'm trying to make about call/return stack is unrelated to the stack where values are spilled: it's a non-architectural stack the CPU keeps internally of all previous calls, and it uses it to predict return locations for indirect branches to speculatively start executing instructions before knowing where it'll actually jump. If you imbalance that stack then you'll shed performance, sometimes 5%-15%, so keeping it balanced matters!

I'm not sure if that helps?

[tmroeder]

I just realized that there are two independent concerns here, and I'd like to separate them.

The first is the core implementation of CFI; there are some remaining comments to address, and then I'd like to be able to commit that part with x86 backend support.

The second is the comments and questions about ARM performance of jumptables and the exact details of the instructions needed to implement ARM backend support. I'm currently looking into this, based on the comments and questions from JF, and I will continue doing that. But I'd like to not block the core CFI work and the x86 backend on those questions.

Are the reviewers amenable to me splitting this into two pieces: one that deals with CFI and the x86 backend code, and one that deals with ARM?

[jfb]

Are the reviewers amenable to me splitting this into two pieces: one that deals with CFI and the x86 backend code, and one that deals with ARM?

Yes, as long as we figure out a nice way to point out that ARM support isn't there at the moment.

[tmroeder]

In D4167#35, @jfb wrote:

Are the reviewers amenable to me splitting this into two pieces: one that deals with CFI and the x86 backend code, and one that deals with ARM?

Yes, as long as we figure out a nice way to point out that ARM support isn't there at the moment.

How about: "ARM support for the jumptable attribute is currently limited to code with small numbers of indirect calls".

Or CFI/x86 commit could temporarily remove ARM support pending performance investigations. The default in TargetInstrInfo is to fail the compile with a message that says the backend doesn't support the jumptable attribute.

[jfb]

How about: "ARM support for the jumptable attribute is currently limited to code with small numbers of indirect calls".

Or CFI/x86 commit could temporarily remove ARM support pending performance investigations. The default in TargetInstrInfo is to fail the compile with a message that says the backend doesn't support the jumptable attribute.

I think the later would be better.

[tmroeder]

Here's a diff that fixes the outstanding comments and removes ARM support for now, as discussed in previous comments.

[jfb]

Looks good overall.

include/llvm/CodeGen/ForwardControlFlowIntegrity.h

46

override

68

You can drop struct here.

test/CodeGen/X86/cfi_enforcing.ll

2

Could you test x86-32 as well here? No triple below, pass it to the command line instead.

15

Add the registers here, to make sure the masks are on the right thing. You can use FileCheck's capture capabilities to avoid hard-coding the register.

28

Could you explain why?

[tmroeder]

This patch addresses the most recent set of comments.

[tmroeder]

Forgot to remove "XFAIL win32" in the previous patch.

[nlewycky]

This looks right about ready to land to me.

include/llvm/CodeGen/CommandFlags.h

256–257

So there's nothing wrong with this as is, but I'd really rather see something in the ubsan style. It turns out that including all the strings for the names of all the functions may be expensive (string data plus relocations) in terms of file size, and may be redundant with getting the same information in other ways (like debug info). In any event, this can wait for a future patch, when it's needed.

include/llvm/Target/TargetInstrInfo.h

431–433 ↗

(On Diff #15510)

What if we're on an architecture where the instruction requires an alignment?

lib/CodeGen/ForwardControlFlowIntegrity.cpp

42–43

You don't appear to use these?

120

Just checking "CS" in bool context is equivalent to "CS.isCall() || CS.isInvoke". That makes this

if (!(CS && isIndirectCall(CS)))

236

Does

FunctionType *WarningFunTy =

​ FunctionType::get(Type::getVoidTy(M.getContext()), {CharPtrTy, CharPtrTy}, false);
work?

241–242

I think this is a case for report_fatal_error. Unreachable will be deleted by the compiler entirely in an optimized build, it should only be used when it is logically impossible. Instead, I think this can be reached through choice of command line options.

244–246

In the sanitizers, we have an option -fsanitize-recover which controls whether the "invariant violated" function will return or not. Apparently having 'noreturn' on the function declaration is a big deal for overall performance for them, probably due to the fact that it gives the register allocator less to worry about. Does this matter to you? And if so, would you use the same flag, or make it always noreturn?

374

"ParentFun->getName()" --> "ParentName" as declared one line above?

381–385

IRBuilder's CreateGlobalString/CreateGlobalStringPtr knows more tricks, like making it "unnamed_addr" so the string can be merged with duplicates. You don't need to give it an insertion point so long as you promise not to create instructions with it. Creating globals is fine.

test/CodeGen/X86/cfi_invoke.ll

35 ↗

(On Diff #15510)

Newline at end of file please!

[tmroeder]

This patch addresses the most recent comments in the review.

[tmroeder]

include/llvm/Target/TargetInstrInfo.h

431–433 ↗

(On Diff #15510)

This value is used to compute the necessary alignment of the jumptable entry. So, if the instruction requires alignment, then that should be taken into account in specifying this bound. I'll add that information to this comment.

lib/CodeGen/ForwardControlFlowIntegrity.cpp

42–43

Ah, right. this is left over from before I switched to SmallVector etc.

236

I tried it, and that doesn't compile; it complains about the initializer list not being convertible to ArrayRef.

241–242

Yes, you're right that this can be reached through command-line flags, so I'll changed it.

244–246

I haven't experimented with the noreturn flag for the "invariant violated" function, but I think that's an interesting and important point and one that should be explored wrt performance here. Can I take that up in a future commit?

381–385

Thanks! This lets me simplify this significantly.

test/CodeGen/X86/cfi_enforcing.ll

28

I think this is an artifact of copy-paste from an earlier LTO test. I don't think this should be here.

[nlewycky]

LGTM

[tmroeder]

Committed in r221708

include/llvm/CodeGen/CommandFlags.h

                       " functions"),
              cl::init(false));
 
+cl::opt<llvm::CFIntegrity::CFIntegrityType>
+CFIType("cfi-type",
+        cl::desc("Choose the type of Control-Flow Integrity check to add"),
+        cl::init(CFIntegrity::Sub),
+        cl::values(
+            clEnumValN(CFIntegrity::Sub, "sub",
+                       "Subtract the pointer from the table base, then mask."),
+            clEnumValN(CFIntegrity::Ror, "ror",
+                       "Use rotate to check the offset from a table base."),
+            clEnumValN(CFIntegrity::Add, "add",
+                       "Mask out the high bits and add to an aligned base."),
+            clEnumValEnd));
+
+cl::opt<bool>
+CFIEnforcing("cfi-enforcing",
+             cl::desc("Enforce CFI or pass a the violation to a funcion."),
+             cl::init(false));
+
+cl::opt<std::string>
+CFIFuncName("cfi-func-name", cl::desc("The name of the CFI function to call"),
+            cl::init(""));
+
 // Common utility function tightly tied to the options listed here. Initializes
 // a TargetOptions object with CodeGen flags and returns it.
 static inline TargetOptions InitTargetOptionsFromCodeGenFlags() {

   Options.MCOptions = InitMCTargetOptionsFromFlags();
   Options.JTType = JTableType;
   Options.JumpTableAll = JumpTableAll;
+  Options.CFIType = CFIType;
+  Options.CFIEnforcing = CFIEnforcing;
+  Options.CFIFuncName = CFIFuncName;
 
   return Options;
 }

include/llvm/CodeGen/ForwardControlFlowIntegrity.h

+//===-- ForwardControlFlowIntegrity.h: Forward-Edge CFI ---------*- C++ -*-===//
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+//
+// This pass instruments indirect calls with checks to ensure that these calls
+// pass through the appropriate jump-instruction table generated by
+// JumpInstrTables.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CODEGEN_FORWARDCONTROLFLOWINTEGRITY_H
+#define LLVM_CODEGEN_FORWARDCONTROLFLOWINTEGRITY_H
+
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/ADT/DenseSet.h"
+#include "llvm/Pass.h"
+#include "llvm/Target/TargetOptions.h"
+
+#include <string>
+
+namespace llvm {
+
+class AnalysisUsage;
+class BasicBlock;
+class Constant;
+class Function;
+class Instruction;
+class Module;
+class Value;
+
+/// ForwardControlFlowIntegrity uses the information from JumpInstrTableInfo to
+/// prepend checks to indirect calls to make sure that these calls target valid
+/// locations.
+class ForwardControlFlowIntegrity : public ModulePass {
+public:
+  static char ID;
+
+  ForwardControlFlowIntegrity();
+  ForwardControlFlowIntegrity(JumpTable::JumpTableType JTT,
+                              CFIntegrity::CFIntegrityType CFIType,
+                              bool CFIEnforcing, std::string CFIFuncName);
+  virtual ~ForwardControlFlowIntegrity();
+
+  /// Runs the CFI pass on a given module. This works best if the module in
+  /// question is the result of link-time optimization (see lib/LTO).
+  bool runOnModule(Module &M) override;
+  const char *getPassName() const override {
+    return "ForwardControlFlowIntegrity";
+  }
+  void getAnalysisUsage(AnalysisUsage &AU) const override;
+
+private:
+  typedef DenseSet<Instruction *> CallSet;
+
+  /// A structure that is used to keep track of constant table information.
+  typedef struct CFIConstants {
+    Constant *StartValue;
+    Constant *MaskValue;
+    Constant *Size;
+  } CFIConstants;
+
+  /// A map from function type to the base of the table for this type and a mask
+  /// for the table
+  typedef DenseMap<FunctionType *, CFIConstants> CFITables;
+
+  CallSet IndirectCalls;
+
+  /// The type of jumptable implementation.
+  JumpTable::JumpTableType JTType;
+
+  /// The type of CFI check to add before each indirect call.
+  CFIntegrity::CFIntegrityType CFIType;
+
+  /// A value that controls whether or not CFI violations cause a halt.
+  bool CFIEnforcing;
+
+  /// The name of the function to call in case of a CFI violation when
+  /// CFIEnforcing is false. There is a default function that ignores
+  /// violations.
+  std::string CFIFuncName;
+
+  /// Adds checks to each indirect call site to make sure that it is calling a
+  /// function in our jump table.
+  void updateIndirectCalls(Module &M, CFITables &CFIT);
+
+  /// Walks the instructions to find all the indirect calls.
+  void getIndirectCalls(Module &M);
+
+  /// Adds a function that handles violations in non-enforcing mode
+  /// (!CFIEnforcing). The default warning function simply returns, since the
+  /// exact details of how to handle CFI violations depend on the application.
+  void addWarningFunction(Module &M);
+
+  /// Rewrites a function pointer in a call/invoke instruction to force it into
+  /// a table.
+  void rewriteFunctionPointer(Module &M, Instruction *I, Value *FunPtr,
+                              Constant *JumpTableStart, Constant *JumpTableMask,
+                              Constant *JumpTableSize);
+
+  /// Inserts a check and a call to a warning function at a given instruction
+  /// that must be an indirect call.
+  void insertWarning(Module &M, BasicBlock *Block, Instruction *I,
+                     Value *FunPtr);
+};
+
+ModulePass *
+createForwardControlFlowIntegrityPass(JumpTable::JumpTableType JTT,
+                                      CFIntegrity::CFIntegrityType CFIType,
+                                      bool CFIEnforcing, StringRef CFIFuncName);
+}
+
+#endif // LLVM_CODEGEN_FORWARDCONTROLFLOWINTEGRITY_H

include/llvm/CodeGen/JumpInstrTables.h

   /// Checks to see if there is already a table for the given FunctionType.
   bool hasTable(FunctionType *FunTy);
 
+  /// Maps the function into a subset of function types, depending on the
+  /// jump-instruction table style selected from JumpTableTypes in
+  /// JumpInstrTables.cpp. The choice of mapping determines the number of
+  /// jump-instruction tables generated by this pass. E.g., the simplest mapping
+  /// converts every function type into void f(); so, all functions end up in a
+  /// single table.
+  static FunctionType *transformType(JumpTable::JumpTableType JTT,
+                                     FunctionType *FunTy);
+
 private:
   /// The metadata used while a jump table is being built
   struct TableMeta {

 
   typedef DenseMap<FunctionType *, struct TableMeta> JumpMap;
 
-  /// Maps the function into a subset of function types, depending on the
-  /// jump-instruction table style selected from JumpTableTypes in
-  /// JumpInstrTables.cpp. The choice of mapping determines the number of
-  /// jump-instruction tables generated by this pass. E.g., the simplest mapping
-  /// converts every function type into void f(); so, all functions end up in a
-  /// single table.
-  FunctionType *transformType(FunctionType *FunTy);
-
   /// The current state of functions and jump entries in the table(s).
   JumpMap Metadata;
 

include/llvm/CodeGen/Passes.h

 
   /// createJumpInstrTables - This pass creates jump-instruction tables.
   ModulePass *createJumpInstrTablesPass();
+
+  /// createForwardControlFlowIntegrityPass - This pass adds control-flow
+  /// integrity.
+  ModulePass *createForwardControlFlowIntegrityPass();
 } // End llvm namespace
 
 #endif

include/llvm/InitializePasses.h

 void initializeCFGOnlyViewerPass(PassRegistry&);
 void initializeCFGPrinterPass(PassRegistry&);
 void initializeCFGSimplifyPassPass(PassRegistry&);
+void initializeForwardControlFlowIntegrityPass(PassRegistry&);
 void initializeFlattenCFGPassPass(PassRegistry&);
 void initializeStructurizeCFGPass(PassRegistry&);
 void initializeCFGViewerPass(PassRegistry&);

include/llvm/Target/TargetOptions.h

     };
   }
 
+  namespace CFIntegrity {
+    enum CFIntegrityType {
+      None,            // Don't add control-flow integrity checks.
+      Sub,             // Use subtraction-based checks.
+      Ror,             // Use rotation-based checks.
+      Add              // Use addition-based checks. This depends on having
+                       // sufficient alignment in the code and is usually not
+                       // feasible.
+    };
+  }
+
   class TargetOptions {
   public:
     TargetOptions()

           DataSections(false), TrapUnreachable(false), TrapFuncName(""),
           FloatABIType(FloatABI::Default),
           AllowFPOpFusion(FPOpFusion::Standard), JTType(JumpTable::Single),
-          JumpTableAll(false) {}
+          JumpTableAll(false), CFIType(CFIntegrity::None), CFIEnforcing(false),
+          CFIFuncName("") {}
 
     /// PrintMachineCode - This flag is enabled when the -print-machineinstrs
     /// option is specified on the command line, and should enable debugging

     /// automatically applied to all address-taken functions.
     bool JumpTableAll;
 
+    /// CFIType - This flag specifies the type of control-flow integrity check
+    /// to add as a preamble to indirect calls.
+    CFIntegrity::CFIntegrityType CFIType;
+
+    /// CFIEnforcing - This flags controls whether or not CFI violations cause
+    /// the program to halt.
+    bool CFIEnforcing;
+
+    /// getCFIFuncName - If this returns a non-empty string, then this is the
+    /// name of the function that will be called for each CFI violation in
+    /// non-enforcing mode.
+    std::string CFIFuncName;
+    StringRef getCFIFuncName() const;
+
     /// Machine level options.
     MCTargetOptions MCOptions;
   };

lib/CodeGen/AsmPrinter/AsmPrinter.cpp

     bool IsThumb = (Arch == Triple::thumb || Arch == Triple::thumbeb);
     MCInst TrapInst;
     TM.getInstrInfo()->getTrap(TrapInst);
+
+    // Emit the right section for these functions. If the tables are not empty,
+    // then we are guaranteed that the first entry of any table is also not
+    // empty. So, we can use it to get the right section for functions.
+    Function *BaseFun = JITI->getTables().begin()->second.begin()->first;
+    OutStreamer.SwitchSection(
+      getObjFileLowering().SectionForGlobal(BaseFun, *Mang, TM));
     for (const auto &KV : JITI->getTables()) {
       uint64_t Count = 0;
       for (const auto &FunPair : KV.second) {

lib/CodeGen/CMakeLists.txt

   ExecutionDepsFix.cpp
   ExpandISelPseudos.cpp
   ExpandPostRAPseudos.cpp
+  ForwardControlFlowIntegrity.cpp
   GCMetadata.cpp
   GCMetadataPrinter.cpp
   GCStrategy.cpp

lib/CodeGen/ForwardControlFlowIntegrity.cpp

+//===-- ForwardControlFlowIntegrity.cpp: Forward-Edge CFI -----------------===//
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+///
+/// \file
+/// \brief A pass that instruments code with fast checks for indirect calls and
+/// hooks for a function to check violations.
+///
+//===----------------------------------------------------------------------===//
+
+#define DEBUG_TYPE "cfi"
+
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/ADT/Statistic.h"
+#include "llvm/Analysis/JumpInstrTableInfo.h"
+#include "llvm/Analysis/Verifier.h"
+#include "llvm/CodeGen/ForwardControlFlowIntegrity.h"
+#include "llvm/CodeGen/JumpInstrTables.h"
+#include "llvm/CodeGen/Passes.h"
+#include "llvm/IR/Attributes.h"
+#include "llvm/IR/CallSite.h"
+#include "llvm/IR/Constants.h"
+#include "llvm/IR/DerivedTypes.h"
+#include "llvm/IR/Function.h"
+#include "llvm/IR/GlobalValue.h"
+#include "llvm/IR/Instructions.h"
+#include "llvm/IR/InlineAsm.h"
+#include "llvm/IR/IRBuilder.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Module.h"
+#include "llvm/IR/Operator.h"
+#include "llvm/IR/Type.h"
+#include "llvm/Pass.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/Debug.h"
+#include "llvm/Support/raw_ostream.h"
+
+#include <list>
+#include <vector>
+
+using namespace llvm;
+
+STATISTIC(NumCFIIndirectCalls,
+          "Number of indirect call sites rewritten by the CFI pass");
+
+char ForwardControlFlowIntegrity::ID = 0;
+INITIALIZE_PASS_BEGIN(ForwardControlFlowIntegrity, "forward-cfi",
+                      "Control-Flow Integrity", true, true)
+INITIALIZE_PASS_DEPENDENCY(JumpInstrTableInfo);
+INITIALIZE_PASS_DEPENDENCY(JumpInstrTables);
+INITIALIZE_PASS_END(ForwardControlFlowIntegrity, "forward-cfi",
+                    "Control-Flow Integrity", true, true)
+
+ModulePass *llvm::createForwardControlFlowIntegrityPass() {
+  return new ForwardControlFlowIntegrity();
+}
+
+ModulePass *llvm::createForwardControlFlowIntegrityPass(
+    JumpTable::JumpTableType JTT, CFIntegrity::CFIntegrityType CFIType,
+    bool CFIEnforcing, StringRef CFIFuncName) {
+  return new ForwardControlFlowIntegrity(JTT, CFIType, CFIEnforcing,
+                                         CFIFuncName);
+}
+
+namespace {
+// Checks to see if a given CallSite is making an indirect call, including
+// cases where the indirect call is made through a bitcast.
+bool isIndirectCall(CallSite &CS) {
+  if (CS.getCalledFunction())
+    return false;
+
+  // Check the value to see if it is merely a bitcast of a function. In
+  // this case, it will translate to a direct function call in the resulting
+  // assembly, so we won't treat it as an indirect call here.
+  const Value *V = CS.getCalledValue();
+  if (const ConstantExpr *CE = dyn_cast<ConstantExpr>(V)) {
+    return !(CE->isCast() && isa<Function>(CE->getOperand(0)));
+  }
+
+  // Otherwise, since we know it's a call, it must be an indirect call
+  return true;
+}
+
+const char cfi_failure_func_name[] = "__llvm_cfi_pointer_warning";
+const char cfi_func_name_prefix[] = "__llvm_cfi_function_";
+} // end anonymous namespace
+
+ForwardControlFlowIntegrity::ForwardControlFlowIntegrity()
+    : ModulePass(ID), IndirectCalls(), JTType(JumpTable::Single),
+      CFIType(CFIntegrity::None), CFIEnforcing(false), CFIFuncName("") {
+  initializeForwardControlFlowIntegrityPass(*PassRegistry::getPassRegistry());
+}
+
+ForwardControlFlowIntegrity::ForwardControlFlowIntegrity(
+    JumpTable::JumpTableType JTT, CFIntegrity::CFIntegrityType CFIType,
+    bool CFIEnforcing, std::string CFIFuncName)
+    : ModulePass(ID), IndirectCalls(), JTType(JTT), CFIType(CFIType),
+      CFIEnforcing(CFIEnforcing), CFIFuncName(CFIFuncName) {
+  initializeForwardControlFlowIntegrityPass(*PassRegistry::getPassRegistry());
+}
+
+ForwardControlFlowIntegrity::~ForwardControlFlowIntegrity() {}
+
+void ForwardControlFlowIntegrity::getAnalysisUsage(AnalysisUsage &AU) const {
+  AU.addRequired<JumpInstrTableInfo>();
+  AU.addRequired<JumpInstrTables>();
+}
+
+void ForwardControlFlowIntegrity::getIndirectCalls(Module &M) {
+  // To get the indirect calls, we iterate over all functions and iterate over
+  // the list of basic blocks in each. We extract a total list of indirect calls
+  // before modifying any of them, since our modifications will modify the list
+  // of basic blocks.
+  for (Function &F : M) {
+    for (auto FI = F.begin(), FE = F.end(); FI != FE; ++FI) {
+      BasicBlock *BB = FI;
+      for (auto BI = BB->begin(), BE = BB->end(); BI != BE; ++BI) {
+        Instruction *I = BI;
+
+        CallSite CS(I);
+        if (!(CS.isCall() && isIndirectCall(CS)))
+          continue;
+
+        Value *CalledValue = CS.getCalledValue();
+
+        // Don't rewrite this instruction if the indirect call is actually just
+        // inline assembly, since our transformation will generate an invalid
+        // module in that case.
+        if (isa<InlineAsm>(CalledValue))
+          continue;
+
+        IndirectCalls.insert(I);
+      }
+    }
+  }
+}
+
+void ForwardControlFlowIntegrity::updateIndirectCalls(Module &M,
+                                                      CFITables &CFIT) {
+  Type *Int64Ty = Type::getInt64Ty(M.getContext());
+  for (Instruction *I : IndirectCalls) {
+    CallSite CS(I);
+    Value *CalledValue = CS.getCalledValue();
+
+    // Get the function type for this call and look it up in the tables.
+    Type *VTy = CalledValue->getType();
+    PointerType *PTy = dyn_cast<PointerType>(VTy);
+    if (!PTy)
+      continue;
+
+    Type *EltTy = PTy->getElementType();
+    FunctionType *FunTy = dyn_cast<FunctionType>(EltTy);
+    if (!FunTy)
+      continue;
+
+    FunctionType *TransformedTy = JumpInstrTables::transformType(JTType, FunTy);
+    ++NumCFIIndirectCalls;
+    Constant *JumpTableStart = NULL;
+    Constant *JumpTableMask = NULL;
+    Constant *JumpTableSize = NULL;
+
+    // Some call sites have function types that don't correspond to any
+    // address-taken function in the module. This happens when function pointers
+    // are passed in from external code.
+    auto it = CFIT.find(TransformedTy);
+    if (it == CFIT.end()) {
+      // In this case, make sure that the function pointer will change by
+      // setting the mask and the start to be 0 so that the transformed
+      // function is 0.
+      JumpTableStart = ConstantInt::get(Int64Ty, 0);
+      JumpTableMask = ConstantInt::get(Int64Ty, 0);
+      JumpTableSize = ConstantInt::get(Int64Ty, 0);
+    } else {
+      JumpTableStart = it->second.StartValue;
+      JumpTableMask = it->second.MaskValue;
+      JumpTableSize = it->second.Size;
+    }
+
+    rewriteFunctionPointer(M, I, CalledValue, JumpTableStart, JumpTableMask,
+                           JumpTableSize);
+  }
+
+  return;
+}
+
+bool ForwardControlFlowIntegrity::runOnModule(Module &M) {
+  if (CFIType == CFIntegrity::None)
+    return false;
+
+  JumpInstrTableInfo *JITI = &getAnalysis<JumpInstrTableInfo>();
+  Type *Int64Ty = Type::getInt64Ty(M.getContext());
+  Type *VoidPtrTy = Type::getInt8PtrTy(M.getContext());
+
+  // Set up tables for control-flow integrity based on information about the
+  // jump-instruction tables.
+  CFITables CFIT;
+  for (const auto &KV : JITI->getTables()) {
+    uint64_t Size = static_cast<uint64_t>(KV.second.size());
+    uint64_t TableSize = NextPowerOf2(Size);
+
+    // This computation assumes that each entry takes up 8 bytes. This should
+    // really get this information from the Target in some way.
+    int64_t MaskValue = ((TableSize << 3) - 1) & -8;
+    Constant *JumpTableMaskValue = ConstantInt::get(Int64Ty, MaskValue);
+    Constant *JumpTableSize = ConstantInt::get(Int64Ty, Size);
+
+    // The base of the table is defined to be the first jumptable function in
+    // the table.
+    Function *First = KV.second.begin()->second;
+    Constant *JumpTableStartValue = ConstantExpr::getBitCast(First, VoidPtrTy);
+    CFIT[KV.first].StartValue = JumpTableStartValue;
+    CFIT[KV.first].MaskValue = JumpTableMaskValue;
+    CFIT[KV.first].Size = JumpTableSize;
+  }
+
+  if (CFIT.empty())
+    return false;
+
+  getIndirectCalls(M);
+
+  if (!CFIEnforcing) {
+    addWarningFunction(M);
+  }
+
+  // Update the instructions with the check and the indirect jump through our
+  // table.
+  updateIndirectCalls(M, CFIT);
+
+  return true;
+}
+
+void ForwardControlFlowIntegrity::addWarningFunction(Module &M) {
+  Type *CharTy = Type::getInt8Ty(M.getContext());
+  PointerType *CharPtrTy = PointerType::getUnqual(CharTy);
+
+  // Get the type of the Warning Function: void (i8*, i8*),
+  // where the first argument is the name of the function in which the violation
+  // occurs, and the second is the function pointer that violates CFI.
+  SmallVector<Type *, 2> WarningFunArgs;
+  WarningFunArgs.push_back(CharPtrTy);
+  WarningFunArgs.push_back(CharPtrTy);
+  FunctionType *WarningFunTy =
+      FunctionType::get(Type::getVoidTy(M.getContext()), WarningFunArgs, false);
+
+  if (!CFIFuncName.empty()) {
+    // If this function is not defined in the module, then it will have to be
+    // defined at link time.
+    Constant *FailureFun = M.getOrInsertFunction(CFIFuncName, WarningFunTy);
+    assert(FailureFun && "Could not get or insert the function specified by"
+                         " -cfi-func-name");
+  } else {
+    // The default warning function swallows the warning and lets the call
+    // continue, since there's no generic way for it to print out this
+    // information.
+    Function *WarningFun = M.getFunction(cfi_failure_func_name);
+    if (!WarningFun) {
+      WarningFun = Function::Create(WarningFunTy, GlobalValue::ExternalLinkage,
+                                    cfi_failure_func_name, &M);
+    }
+
+    BasicBlock *Entry =
+        BasicBlock::Create(M.getContext(), "entry", WarningFun, 0);
+    ReturnInst::Create(M.getContext(), Entry);
+  }
+}
+
+void ForwardControlFlowIntegrity::rewriteFunctionPointer(
+    Module &M, Instruction *I, Value *FunPtr, Constant *JumpTableStart,
+    Constant *JumpTableMask, Constant *JumpTableSize) {
+  IRBuilder<> TempBuilder(I);
+
+  Type *OrigFunType = FunPtr->getType();
+
+  BasicBlock *CurBB = cast<BasicBlock>(I->getParent());
+  Function *CurF = cast<Function>(CurBB->getParent());
+  Type *Int64Ty = Type::getInt64Ty(M.getContext());
+
+  Value *TI = TempBuilder.CreatePtrToInt(FunPtr, Int64Ty);
+  Value *TStartInt = TempBuilder.CreatePtrToInt(JumpTableStart, Int64Ty);
+
+  Value *NewFunPtr = NULL;
+  Value *Check = NULL;
+  assert(CFIType != CFIntegrity::None && "Can't apply null CFI checks");
+  if (CFIType == CFIntegrity::Sub) {
+    // This is the subtract, mask, and add version.
+    // Subtract from the base.
+    Value *Sub = TempBuilder.CreateSub(TI, TStartInt);
+
+    // Mask the difference to force this to be a table offset.
+    Value *And = TempBuilder.CreateAnd(Sub, JumpTableMask);
+
+    // Add it back to the base.
+    Value *Result = TempBuilder.CreateAdd(And, TStartInt);
+
+    // Convert it back into a function pointer that we can call.
+    NewFunPtr = TempBuilder.CreateIntToPtr(Result, OrigFunType);
+  } else if (CFIType == CFIntegrity::Ror) {
+    // This is the subtract and rotate version.
+    // Rotate right by the alignment value. The optimizer should recognize
+    // this sequence as a rotation.
+    // TODO: these values should come from the JumpInstrTableInfo.
+    Constant *RightShift = ConstantInt::get(Int64Ty, 3);
+    Constant *LeftShift = ConstantInt::get(Int64Ty, 64 - 3);
+
+    // Subtract from the base.
+    Value *Sub = TempBuilder.CreateSub(TI, TStartInt);
+
+    // Create the equivalent of a rotate-right instruction.
+    Value *Shr = TempBuilder.CreateLShr(Sub, RightShift);
+    Value *Shl = TempBuilder.CreateShl(Sub, LeftShift);
+    Value *Or = TempBuilder.CreateOr(Shr, Shl);
+
+    // Perform unsigned comparison to check for inclusion in the table.
+    Check = TempBuilder.CreateICmpULT(Or, JumpTableSize);
+    NewFunPtr = FunPtr;
+  } else if (CFIType == CFIntegrity::Add) {
+    // This is the mask and add version.
+    // Mask the function pointer to turn it into an offset into the table.
+    Value *And = TempBuilder.CreateAnd(TI, JumpTableMask);
+
+    // Then or this offset to the base and get the pointer value.
+    Value *Result = TempBuilder.CreateAdd(And, TStartInt);
+
+    // Convert it back into a function pointer that we can call.
+    NewFunPtr = TempBuilder.CreateIntToPtr(Result, OrigFunType);
+  }
+
+  if (!CFIEnforcing) {
+    // If a check hasn't been added (in the rotation version), then check to see
+    // if it's the same as the original function. This check determines whether
+    // or not we call the CFI failure function.
+    if (!Check)
+      Check = TempBuilder.CreateICmpEQ(NewFunPtr, FunPtr);
+    BasicBlock *InvalidPtrBlock =
+        BasicBlock::Create(M.getContext(), "invalid.ptr", CurF, 0);
+    BasicBlock *ContinuationBB = CurBB->splitBasicBlock(I);
+
+    // Remove the unconditional branch that connects the two blocks.
+    TerminatorInst *TermInst = CurBB->getTerminator();
+    TermInst->eraseFromParent();
+
+    // Add a conditional branch that depends on the Check above.
+    BranchInst::Create(ContinuationBB, InvalidPtrBlock, Check, CurBB);
+
+    // Call the warning function for this pointer, then continue.
+    Instruction *BI = BranchInst::Create(ContinuationBB, InvalidPtrBlock);
+    insertWarning(M, InvalidPtrBlock, BI, FunPtr);
+  } else {
+    // Modify the instruction to call this value.
+    CallSite CS(I);
+    CS.setCalledFunction(NewFunPtr);
+  }
+}
+
+void ForwardControlFlowIntegrity::insertWarning(Module &M, BasicBlock *Block,
+                                                Instruction *I, Value *FunPtr) {
+  Function *ParentFun = cast<Function>(Block->getParent());
+
+  // Get the function to call right before the instruction.
+  Function *WarningFun = NULL;
+  if (CFIFuncName.empty()) {
+    WarningFun = M.getFunction(cfi_failure_func_name);
+  } else {
+    WarningFun = M.getFunction(CFIFuncName);
+  }
+
+  assert(WarningFun && "Could not find the CFI failure function");
+
+  // Look up or create a GlobalVariable
+  // __llvm_cfi_function_ParentName containing this name.
+  StringRef ParentName(ParentFun->getName());
+  std::string GVName =
+      Twine(cfi_func_name_prefix).concat(ParentFun->getName()).str();
+  GlobalVariable *ParentNameGV = M.getNamedGlobal(GVName);
+  if (!ParentNameGV) {
+    Type *CharTy = Type::getInt8Ty(M.getContext());
+    ArrayType *ParentNameStringTy =
+        ArrayType::get(CharTy, ParentName.size() + 1);
+
+    ParentNameGV = new GlobalVariable(M, ParentNameStringTy, true,
+                                      GlobalValue::PrivateLinkage, 0, ".str");
+    Constant *ParentNameStrConst =
+        ConstantDataArray::getString(M.getContext(), ParentName, true);
+    ParentNameGV->setInitializer(ParentNameStrConst);
+  }
+
+  Type *VoidPtrTy = Type::getInt8PtrTy(M.getContext());
+
+  IRBuilder<> WarningInserter(I);
+  Value *ParentNamePtr = WarningInserter.CreateBitCast(ParentNameGV, VoidPtrTy);
+  Value *FunVoidPtr = WarningInserter.CreateBitCast(FunPtr, VoidPtrTy);
+  WarningInserter.CreateCall2(WarningFun, ParentNamePtr, FunVoidPtr);
+}

lib/CodeGen/JumpInstrTables.cpp

 
 Function *JumpInstrTables::insertEntry(Module &M, Function *Target) {
   FunctionType *OrigFunTy = Target->getFunctionType();
-  FunctionType *FunTy = transformType(OrigFunTy);
+  FunctionType *FunTy = transformType(JTType, OrigFunTy);
 
   JumpMap::iterator it = Metadata.find(FunTy);
   if (Metadata.end() == it) {

 }
 
 bool JumpInstrTables::hasTable(FunctionType *FunTy) {
-  FunctionType *TransTy = transformType(FunTy);
+  FunctionType *TransTy = transformType(JTType, FunTy);
   return Metadata.end() != Metadata.find(TransTy);
 }
 
-FunctionType *JumpInstrTables::transformType(FunctionType *FunTy) {
+FunctionType *JumpInstrTables::transformType(JumpTable::JumpTableType JTT,
+                                             FunctionType *FunTy) {
   // Returning nullptr forces all types into the same table, since all types map
   // to the same type
   Type *VoidPtrTy = Type::getInt8PtrTy(FunTy->getContext());

   Type *Int32Ty = Type::getInt32Ty(FunTy->getContext());
   FunctionType *VoidFnTy = FunctionType::get(
       Type::getVoidTy(FunTy->getContext()), EmptyParams, false);
-  switch (JTType) {
+  switch (JTT) {
   case JumpTable::Single:
 
     return FunctionType::get(RetTy, EmptyParams, false);

lib/CodeGen/LLVMTargetMachine.cpp

 
 #include "llvm/Analysis/Passes.h"
 #include "llvm/CodeGen/AsmPrinter.h"
+#include "llvm/CodeGen/ForwardControlFlowIntegrity.h"
 #include "llvm/CodeGen/JumpInstrTables.h"
 #include "llvm/CodeGen/MachineFunctionAnalysis.h"
 #include "llvm/CodeGen/MachineModuleInfo.h"

   // JIT time, so we don't add them directly to addPassesToGenerateCode.
   PM.add(createJumpInstrTableInfoPass());
   PM.add(createJumpInstrTablesPass(Options.JTType, Options.JumpTableAll));
+  PM.add(createForwardControlFlowIntegrityPass(Options.JTType, Options.CFIType,
+                                               Options.CFIEnforcing,
+                                               Options.getCFIFuncName()));
 
   // Add common CodeGen passes.
   MCContext *Context = addPassesToGenerateCode(this, PM, DisableVerify,

lib/CodeGen/TargetOptionsImpl.cpp

 StringRef TargetOptions::getTrapFunctionName() const {
   return TrapFuncName;
 }
+
+/// getCFIFuncName - If this returns a non-empty string, then it is the name of
+/// the function that gets called on CFI violations in CFI non-enforcing mode
+/// (!TargetOptions::CFIEnforcing).
+StringRef TargetOptions::getCFIFuncName() const {
+  return CFIFuncName;
+}

test/CodeGen/Generic/stop-after.ll

 ; STOP: Loop Strength Reduction
 ; STOP-NEXT: Machine Function Analysis
 
-; START: -machine-branch-prob -jump-instr-tables -gc-lowering
+; START: -machine-branch-prob -jump-instr-tables -forward-cfi -gc-lowering
 ; START: FunctionPass Manager
 ; START-NEXT: Lower Garbage Collection Instructions

test/CodeGen/X86/cfi_enforcing.ll

+; RUN: llc -jump-table-all -cfi-enforcing <%s | FileCheck %s
+
+target triple = "x86_64-unknown-linux-gnu"
+define void @indirect_fun() {
+  ret void
+}
+
+define i32 @m(void ()* %fun) {
+  call void ()* %fun()
+; CHECK: subl
+; CHECK: andq    $8,
+; CHECK: leaq    __llvm_jump_instr_table_0_1(%r
+; CHECK-NOT: callq __llvm_cfi_pointer_warning
+; CHECK: callq   *%r
+  ret i32 0
+}
+
+define void ()* @get_fun() {
+  ret void ()* @indirect_fun
+}
+
+define i32 @main(i32 %argc, i8** %argv) {
+  %f = call void ()* ()* @get_fun()
+  %a = call i32 @m(void ()* %f)
+  ret i32 %a
+}
+; XFAIL: win32
+
+; CHECK: .globl __llvm_jump_instr_table_0_1
+; CHECK: .align 8
+; CHECK: __llvm_jump_instr_table_0_1:
+; CHECK: jmp indirect_fun@PLT

test/CodeGen/X86/cfi_non_default_function.ll

+; RUN: llc -jump-table-all -cfi-func-name=cfi_new_failure <%s | FileCheck %s
+
+target triple = "x86_64-unknown-linux-gnu"
+define void @indirect_fun() {
+  ret void
+}
+
+define i32 @m(void ()* %fun) {
+  call void ()* %fun()
+; CHECK: callq cfi_new_failure
+  ret i32 0
+}
+
+define void ()* @get_fun() {
+  ret void ()* @indirect_fun
+}
+
+define i32 @main(i32 %argc, i8** %argv) {
+  %f = call void ()* ()* @get_fun()
+  %a = call i32 @m(void ()* %f)
+  ret i32 %a
+}
+; XFAIL: win32
+
+; CHECK: .globl __llvm_jump_instr_table_0_1
+; CHECK: .align 8
+; CHECK: __llvm_jump_instr_table_0_1:
+; CHECK: jmp indirect_fun@PLT

test/CodeGen/X86/cfi_simple_indirect_call.ll

+; RUN: llc -jump-table-all -cfi-type=sub <%s | FileCheck --check-prefix=SUB %s
+; RUN: llc -jump-table-all -cfi-type=add <%s | FileCheck --check-prefix=ADD %s
+; RUN: llc -jump-table-all -cfi-type=ror <%s | FileCheck --check-prefix=ROR %s
+
+target triple = "x86_64-unknown-linux-gnu"
+
+define void @indirect_fun() {
+  ret void
+}
+
+define i32 @m(void ()* %fun) {
+  call void ()* %fun()
+; SUB: subl    
+; SUB: andq    $8
+; SUB: leaq    __llvm_jump_instr_table_0_1
+; SUB: callq   __llvm_cfi_pointer_warning
+
+; ROR: subq
+; ROR: rolq    $61
+; ROR: testq
+; ROR: callq   __llvm_cfi_pointer_warning
+
+; ADD: andq    $8
+; ADD: leaq    __llvm_jump_instr_table_0_1
+; ADD: cmpq
+; ADD: callq   __llvm_cfi_pointer_warning
+ret i32 0
+}
+
+define void ()* @get_fun() {
+  ret void ()* @indirect_fun
+}
+
+define i32 @main(i32 %argc, i8** %argv) {
+  %f = call void ()* ()* @get_fun()
+  %a = call i32 @m(void ()* %f)
+  ret i32 %a
+}
+; XFAIL: win32
+; SUB: .text
+; SUB: .globl __llvm_jump_instr_table_0_1
+; SUB: .align 8
+; SUB: .type __llvm_jump_instr_table_0_1,@function
+; SUB:__llvm_jump_instr_table_0_1:
+; SUB: jmp indirect_fun@PLT

test/CodeGen/X86/jump_table_alias.ll

 ; because they both alias the same value.
 
 ; CHECK:         .globl  __llvm_jump_instr_table_0_1
-; CHECK:         .align  8, 0x90
+; CHECK:         .align  8
 ; CHECK:         .type   __llvm_jump_instr_table_0_1,@function
 ; CHECK: __llvm_jump_instr_table_0_1:
 ; CHECK:         jmp     f@PLT

test/CodeGen/X86/jump_table_all.ll

 ; This table should be created even without the jumptable annotation
 ; because llc is called with -jump-table-all, and @f has its address taken
 ; CHECK:         .globl  __llvm_jump_instr_table_0_1
-; CHECK:         .align  8, 0x90
+; CHECK:         .align  8
 ; CHECK:         .type   __llvm_jump_instr_table_0_1,@function
 ; CHECK: __llvm_jump_instr_table_0_1:
 ; CHECK:         jmp     f@PLT

test/CodeGen/X86/jump_table_bitcast.ll

 define i32 @main() {
   %g = alloca i32 (...)*, align 8
   store i32 (...)* bitcast (i32 ()* @f to i32 (...)*), i32 (...)** %g, align 8
-; CHECK: movq    $__llvm_jump_instr_table_0_[[ENTRY:1|2|3]], (%rsp)
-; CHECK: movl    $__llvm_jump_instr_table_0_[[ENTRY]], %ecx
+; CHECK: movq    $__llvm_jump_instr_table_0_[[ENTRY:1|2|3]], 8(%rsp)
+; CHECK: movl    $__llvm_jump_instr_table_0_[[ENTRY]],
   %1 = load i32 (...)** %g, align 8
   %call = call i32 (...)* %1()
   call void (void ()*)* @h(void ()* bitcast (void (void ()*)* @h to void ()*))
-; CHECK: movl    $__llvm_jump_instr_table_0_{{1|2|3}}, %edi
+; CHECK: movl    $__llvm_jump_instr_table_0_{{1|2|3}},
 ; CHECK: callq   h
 
   %a = call i32 (i32*)* bitcast (i32 (i8*)* @g to i32(i32*)*)(i32* null)

 }
 
 ; CHECK:         .globl  __llvm_jump_instr_table_0_1
-; CHECK:         .align  8, 0x90
+; CHECK:         .align  8
 ; CHECK:         .type   __llvm_jump_instr_table_0_1,@function
 ; CHECK: __llvm_jump_instr_table_0_1:
 ; CHECK:         jmp     {{f|g|h}}@PLT
 ; CHECK:         .globl  __llvm_jump_instr_table_0_2
-; CHECK:         .align  8, 0x90
+; CHECK:         .align  8
 ; CHECK:         .type   __llvm_jump_instr_table_0_2,@function
 ; CHECK: __llvm_jump_instr_table_0_2:
 ; CHECK:         jmp     {{f|g|h}}@PLT
 ; CHECK:         .globl  __llvm_jump_instr_table_0_3
-; CHECK:         .align  8, 0x90
+; CHECK:         .align  8
 ; CHECK:         .type   __llvm_jump_instr_table_0_3,@function
 ; CHECK: __llvm_jump_instr_table_0_3:
 ; CHECK:         jmp     {{f|g|h}}@PLT

test/CodeGen/X86/jump_tables.ll

 }
 
 ; SINGLE-DAG:         .globl  __llvm_jump_instr_table_0_1
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         .type   __llvm_jump_instr_table_0_1,@function
 ; SINGLE-DAG: __llvm_jump_instr_table_0_1:
 ; SINGLE-DAG:         jmp     indirect_fun_array@PLT
 ; SINGLE-DAG:         .globl  __llvm_jump_instr_table_0_2
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         .type   __llvm_jump_instr_table_0_2,@function
 ; SINGLE-DAG: __llvm_jump_instr_table_0_2:
 ; SINGLE-DAG:         jmp     indirect_fun_i32_2@PLT
 ; SINGLE-DAG:         .globl  __llvm_jump_instr_table_0_3
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         .type   __llvm_jump_instr_table_0_3,@function
 ; SINGLE-DAG: __llvm_jump_instr_table_0_3:
 ; SINGLE-DAG:         jmp     indirect_fun_vec_2@PLT
 ; SINGLE-DAG:         .globl  __llvm_jump_instr_table_0_4
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         .type   __llvm_jump_instr_table_0_4,@function
 ; SINGLE-DAG: __llvm_jump_instr_table_0_4:
 ; SINGLE-DAG:         jmp     indirect_fun_i32S_2@PLT
 ; SINGLE-DAG:         .globl  __llvm_jump_instr_table_0_5
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         .type   __llvm_jump_instr_table_0_5,@function
 ; SINGLE-DAG: __llvm_jump_instr_table_0_5:
 ; SINGLE-DAG:         jmp     indirect_fun_struct@PLT
 ; SINGLE-DAG:         .globl  __llvm_jump_instr_table_0_6
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         .type   __llvm_jump_instr_table_0_6,@function
 ; SINGLE-DAG: __llvm_jump_instr_table_0_6:
 ; SINGLE-DAG:         jmp     indirect_fun_i32_1@PLT
 ; SINGLE-DAG:         .globl  __llvm_jump_instr_table_0_7
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         .type   __llvm_jump_instr_table_0_7,@function
 ; SINGLE-DAG: __llvm_jump_instr_table_0_7:
 ; SINGLE-DAG:         jmp     indirect_fun_i32@PLT
 ; SINGLE-DAG:         .globl  __llvm_jump_instr_table_0_8
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         .type   __llvm_jump_instr_table_0_8,@function
 ; SINGLE-DAG: __llvm_jump_instr_table_0_8:
 ; SINGLE-DAG:         jmp     indirect_fun_fun@PLT
 ; SINGLE-DAG:         .globl  __llvm_jump_instr_table_0_9
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         .type   __llvm_jump_instr_table_0_9,@function
 ; SINGLE-DAG: __llvm_jump_instr_table_0_9:
 ; SINGLE-DAG:         jmp     indirect_fun_fun_ret@PLT
 ; SINGLE-DAG:         .globl  __llvm_jump_instr_table_0_10
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         .type   __llvm_jump_instr_table_0_10,@function
 ; SINGLE-DAG: __llvm_jump_instr_table_0_10:
 ; SINGLE-DAG:         jmp     indirect_fun@PLT
 ; SINGLE-DAG:         .globl  __llvm_jump_instr_table_0_11
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         .type   __llvm_jump_instr_table_0_11,@function
 ; SINGLE-DAG: __llvm_jump_instr_table_0_11:
 ; SINGLE-DAG:         jmp     indirect_fun_match@PLT
 ; SINGLE-DAG:         .globl  __llvm_jump_instr_table_0_12
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         .type   __llvm_jump_instr_table_0_12,@function
 ; SINGLE-DAG: __llvm_jump_instr_table_0_12:
 ; SINGLE-DAG:         jmp     indirect_fun_vec@PLT
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         ud2
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         ud2
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         ud2
-; SINGLE-DAG:         .align  8, 0x90
+; SINGLE-DAG:         .align  8
 ; SINGLE-DAG:         ud2
 
 
 ; ARITY-DAG:         .globl  __llvm_jump_instr_table_2_1
-; ARITY-DAG:         .align  8, 0x90
+; ARITY-DAG:         .align  8
 ; ARITY-DAG:         .type   __llvm_jump_instr_table_2_1,@function
 ; ARITY-DAG: __llvm_jump_instr_table_2_1:
 ; ARITY-DAG:         jmp     indirect_fun{{.*}}@PLT
-; ARITY-DAG:         .align  8, 0x90
+; ARITY-DAG:         .align  8
 ; ARITY-DAG:         ud2
 ; ARITY-DAG:         .globl  __llvm_jump_instr_table_0_1
-; ARITY-DAG:         .align  8, 0x90
+; ARITY-DAG:         .align  8
 ; ARITY-DAG:         .type   __llvm_jump_instr_table_0_1,@function
 ; ARITY-DAG: __llvm_jump_instr_table_0_1:
 ; ARITY-DAG:         jmp     indirect_fun{{.*}}@PLT
 ; ARITY-DAG:         .globl  __llvm_jump_instr_table_1_1
-; ARITY-DAG:         .align  8, 0x90
+; ARITY-DAG:         .align  8
 ; ARITY-DAG:         .type   __llvm_jump_instr_table_1_1,@function
 ; ARITY-DAG: __llvm_jump_instr_table_1_1:
 ; ARITY-DAG:         jmp     indirect_fun{{.*}}@PLT
 
 ; SIMPL-DAG:         .globl  __llvm_jump_instr_table_2_1
-; SIMPL-DAG:         .align  8, 0x90
+; SIMPL-DAG:         .align  8
 ; SIMPL-DAG:         .type   __llvm_jump_instr_table_2_1,@function
 ; SIMPL-DAG: __llvm_jump_instr_table_2_1:
 ; SIMPL-DAG:         jmp     indirect_fun{{.*}}@PLT
-; SIMPL-DAG:         .align  8, 0x90
+; SIMPL-DAG:         .align  8
 ; SIMPL-DAG:         ud2
 ; SIMPL-DAG:         .globl  __llvm_jump_instr_table_0_1
-; SIMPL-DAG:         .align  8, 0x90
+; SIMPL-DAG:         .align  8
 ; SIMPL-DAG:         .type   __llvm_jump_instr_table_0_1,@function
 ; SIMPL-DAG: __llvm_jump_instr_table_0_1:
 ; SIMPL-DAG:         jmp     indirect_fun{{.*}}@PLT
 ; SIMPL-DAG:         .globl  __llvm_jump_instr_table_1_1
-; SIMPL-DAG:         .align  8, 0x90
+; SIMPL-DAG:         .align  8
 ; SIMPL-DAG:         .type   __llvm_jump_instr_table_1_1,@function
 ; SIMPL-DAG: __llvm_jump_instr_table_1_1:
 ; SIMPL-DAG:         jmp     indirect_fun{{.*}}@PLT
 ; SIMPL-DAG:         .globl  __llvm_jump_instr_table_3_1
-; SIMPL-DAG:         .align  8, 0x90
+; SIMPL-DAG:         .align  8
 ; SIMPL-DAG:         .type   __llvm_jump_instr_table_3_1,@function
 ; SIMPL-DAG: __llvm_jump_instr_table_3_1:
 ; SIMPL-DAG:         jmp     indirect_fun{{.*}}@PLT
 ; SIMPL-DAG:         .globl  __llvm_jump_instr_table_4_1
-; SIMPL-DAG:         .align  8, 0x90
+; SIMPL-DAG:         .align  8
 ; SIMPL-DAG:         .type   __llvm_jump_instr_table_4_1,@function
 ; SIMPL-DAG: __llvm_jump_instr_table_4_1:
 ; SIMPL-DAG:         jmp     indirect_fun{{.*}}@PLT
 
 
 ; FULL-DAG:        .globl  __llvm_jump_instr_table_10_1
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        .type   __llvm_jump_instr_table_10_1,@function
 ; FULL-DAG:__llvm_jump_instr_table_10_1:
 ; FULL-DAG:        jmp     indirect_fun_i32_1@PLT
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        ud2
 ; FULL-DAG:        .globl  __llvm_jump_instr_table_9_1
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        .type   __llvm_jump_instr_table_9_1,@function
 ; FULL-DAG:__llvm_jump_instr_table_9_1:
 ; FULL-DAG:        jmp     indirect_fun_i32_2@PLT
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        ud2
 ; FULL-DAG:        .globl  __llvm_jump_instr_table_7_1
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        .type   __llvm_jump_instr_table_7_1,@function
 ; FULL-DAG:__llvm_jump_instr_table_7_1:
 ; FULL-DAG:        jmp     indirect_fun_i32S_2@PLT
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        ud2
 ; FULL-DAG:        .globl  __llvm_jump_instr_table_3_1
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        .type   __llvm_jump_instr_table_3_1,@function
 ; FULL-DAG:__llvm_jump_instr_table_3_1:
 ; FULL-DAG:        jmp     indirect_fun_vec_2@PLT
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        ud2
 ; FULL-DAG:        .globl  __llvm_jump_instr_table_2_1
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        .type   __llvm_jump_instr_table_2_1,@function
 ; FULL-DAG:__llvm_jump_instr_table_2_1:
 ; FULL-DAG:        jmp     indirect_fun@PLT
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        ud2
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        ud2
 ; FULL-DAG:        .globl  __llvm_jump_instr_table_8_1
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        .type   __llvm_jump_instr_table_8_1,@function
 ; FULL-DAG:__llvm_jump_instr_table_8_1:
 ; FULL-DAG:        jmp     indirect_fun_i32@PLT
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        ud2
 ; FULL-DAG:        .globl  __llvm_jump_instr_table_1_1
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        .type   __llvm_jump_instr_table_1_1,@function
 ; FULL-DAG:__llvm_jump_instr_table_1_1:
 ; FULL-DAG:        jmp     indirect_fun_array@PLT
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        ud2
 ; FULL-DAG:        .globl  __llvm_jump_instr_table_0_1
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        .type   __llvm_jump_instr_table_0_1,@function
 ; FULL-DAG:__llvm_jump_instr_table_0_1:
 ; FULL-DAG:        jmp     indirect_fun_vec@PLT
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        ud2
 ; FULL-DAG:        .globl  __llvm_jump_instr_table_6_1
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        .type   __llvm_jump_instr_table_6_1,@function
 ; FULL-DAG:__llvm_jump_instr_table_6_1:
 ; FULL-DAG:        jmp     indirect_fun_struct@PLT
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        ud2
 ; FULL-DAG:        .globl  __llvm_jump_instr_table_5_1
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        .type   __llvm_jump_instr_table_5_1,@function
 ; FULL-DAG:__llvm_jump_instr_table_5_1:
 ; FULL-DAG:        jmp     indirect_fun_fun@PLT
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        ud2
 ; FULL-DAG:        .globl  __llvm_jump_instr_table_4_1
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        .type   __llvm_jump_instr_table_4_1,@function
 ; FULL-DAG:__llvm_jump_instr_table_4_1:
 ; FULL-DAG:        jmp     indirect_fun_fun_ret@PLT
-; FULL-DAG:        .align  8, 0x90
+; FULL-DAG:        .align  8
 ; FULL-DAG:        ud2