Support -fstack-clash-protection for x86

[serge-sans-paille]

Summary

Implement protection against the stack clash attack [0] through inline stack probing.

Probe stack allocation every PAGE_SIZE during frame lowering or dynamic allocation to make sure the page guard, if any, is touched when touching the stack, in a similar manner to GCC[1].

This extends the existing probe-stack mechanism with a special value inline-asm. Technically the former uses function call before stack allocation while this patch provides inlined stack probes and chunk allocation.

Only implemented for x86.

[0] https://www.qualys.com/2017/06/19/stack-clash/stack-clash.txt
[1] https://gcc.gnu.org/ml/gcc-patches/2017-07/msg00556.html

[efriedma]

Is there some reason this isn't using the existing stack-probe-size attribute?

[efriedma]

Sorry, I meant the "probe-stack" attribute.

[serge-sans-paille]

@efriedma : there's indeed an intersection with the probe-stack attribute. The probe-stack attribute (a) forces a function call, and (b) this function call only happens before the stack gets expanded.

(a) is probably a performance issue in several cases, plus it requires an extra register (that's mentioned in https://reviews.llvm.org/D9653)
(b) is an issue, as pointed out in https://lwn.net/Articles/726587/ (grep for valgrind) : from valgrind point of view, accessing un-allocated stack memory triggers error, and we probably want to please valgrind

Doing the call *after* the stack allocation is also not an option, as a signal could be raised between the stack allocation and the stack probing, escaping the stack probe if a custom signal handler is executed.

That being said, I do think it would be a good thing to have a special value for probe-stack, say probe-stack=inline-asm, that would trigger generation of inlined assembly as I do. That way we have all the pieces in one place, with different strategies. And we would have clang set the attribute for each function when -fstack-clash-protection is given.

[serge-sans-paille]

Move to stack-probe compatibility, using a dedicated name to trigger inline assembly. It looks better to me because

1. it leverage existing mechanics
2. it has a finer grain

[serge-sans-paille]

@efriedma alos compared to probe-stack with a function, this version has the ability to use existing MOV operations to avoid generating probes, which looks like a big plus to me.

[efriedma]

(b) is an issue, as pointed out in https://lwn.net/Articles/726587/ (grep for valgrind) : from valgrind point of view, accessing un-allocated stack memory triggers error, and we probably want to please valgrind

Doing the call *after* the stack allocation is also not an option, as a signal could be raised between the stack allocation and the stack probing, escaping the stack probe if a custom signal handler is executed.

I'm not sure I follow. How are you solving this problem in your patch? By limiting the amount you adjust the stack at a time? What limit is sufficient to avoid this issue?

---

Can you give a complete assembly listing for small examples of static and dynamic stack probing?

llvm/lib/Target/X86/X86FrameLowering.cpp

427

Why is this code in a different location from the stack probing code that generates a call?

435

This algorithm needs some documentation; it isn't at all obvious what it's doing. Particularly the interaction with "free" stack probes.

Should we generate a loop if the stack frame is large?

[xbolva00]

Please add info about this new feature to release notes

[serge-sans-paille]

Added documentation + release note entry

[serge-sans-paille]

@efriedma the free probe algorithm requires more testing, and I'd like to take into account memset and memcpy as free probes too. To showcase this algorithm, consider the following LLVM bitcode:

define i32 @foo() local_unnamed_addr {
  %a = alloca i32, i64 2000, align 16
  %b = getelementptr inbounds i32, i32* %a, i64 1198
  store volatile i32 1, i32* %b
  %c = load volatile i32, i32* %a
  ret i32 %c
}

when compiled with llc it outputs the following assembly:

foo:                                    # @foo
    subq    $7880, %rsp             # imm = 0x1EC8
    movl    $1, 4664(%rsp)
    movl    -128(%rsp), %eax
    addq    $7880, %rsp             # imm = 0x1EC8
    retq

When probe-stack is set to inline-asm it outputs

foo:                                    # @foo
	subq	$4096, %rsp             # imm = 0x1000
	movl	$1, 880(%rsp)
	subq	$3784, %rsp             # imm = 0xEC8
	movq	$0, (%rsp)
	movl	-128(%rsp), %eax
	addq	$7880, %rsp             # imm = 0x1EC8
	retq

The stack allocation is split in two, but only one MOV is added, the first one is what I call a free probe. Turns out we could only use natural probes here, I need to implement that :-)

As a comparison, setting probe-stack to a random function name like __probe_stack outputs the following:

foo:                                    # @foo
	movl	$8008, %eax             # imm = 0x1F48
	callq	__probe_stack
	subq	%rax, %rsp
	movl	$1, 4792(%rsp)
	movl	(%rsp), %eax
	addq	$8008, %rsp             # imm = 0x1F48
	retq

which requires runtime support (to provide __stack_probe), and a function call overhead, while ideally just an extra sub %rsp would be needed.

[serge-sans-paille]

llvm/lib/Target/X86/X86FrameLowering.cpp

427

Because BuildStackAdjustment has more callsites than just emitPrologue and we want to capture all stack manipulation.

435

Should we generate a loop if the stack frame is large?

That's an option. it's more complex to make looping compatible with free probes, but that's possible.

435

This algorithm needs some documentation

Done

[serge-sans-paille]

Added test case, statistics and refactor interactions with existing stack probing mechanism.

[serge-sans-paille]

Some early stats: on the sqlite amalgamation [0], the free probe reuse allows to skip 123 out of the 474 probes needed during frame lowering.

[0] https://www.sqlite.org/download.html

[xbolva00]

llvm/lib/Target/X86/X86FrameLowering.cpp

451

nit: llvm::any_of

[efriedma]

llvm/lib/Target/X86/X86FrameLowering.cpp

443

Each probe has to have an offset of at most PageSize bytes from the previous probe. If each probe is exactly PageSize bytes away from the previous probe, that's fine. But it looks like you don't enforce the distance between free probes correctly?

453

There are instructions that don't refer to any FI, but are still relevant. For example, calls.

[tstellar]

clang/test/CodeGen/stack-clash-protection.c

4

There were concerns[1] raised recently about adding clang tests that were codegen dependent. Is something being tested here that can't be tested with an IR test? If you only need to test that the frontend option work, I think checking the IR for the necessary function attributes might be enough.

[1] http://lists.llvm.org/pipermail/cfe-dev/2019-September/063309.html

[serge-sans-paille]

Ensure the distance between two probes is at max PAGE_SIZE.
Use Calls as free probes.
Fix alignment for dynamic alloca

This passes the llvm-test suite, and thanks to the use of calls, no inserted probe are needed to compile sqlite!

[rnk]

For maintenance reasons, I'd really prefer it if we could find a way to reuse the existing code that calls an external stack probe function. What do you think about taking a look at X86RetpolineThunks.cpp and doing something similar to that? Basically, when the user sets -fstack-clash-protection, LLVM will emit a small comdat+weak function into every object file that has the same ABI as the existing stack probe mechanism. For other prior art, you can also look at how __clang_call_terminate works.

llvm/lib/Target/X86/X86FrameLowering.cpp

373

Please no dynamically initialized globals. The LLVM-y thing would be to use a switch.

427

If you care about those cases, we should have tests for all of them. These are all the cases:

1. TailCallReturnAddrDelta: When doing guaranteed TCO for a callee with >4K of argument memory
2. eliminateCallFramePseudoInstr: When doing stack adjustments to set up a call with more than 4K of arguments

Both of these cases involve setting up arguments to calls, and I think we can guarantee that the compiler will emit writes to every argument stack slot. So, to set up one of these cases, we'd have this code:

struct EightK { int a[2048]; };
void passEightK(EightK);
void foo() {
  EightK x;
  memset(&x, 0, sizeof(x));
  passEightK(x); // some targets will pass indirect, 32-bit probably uses byval.
}

In this case, LLVM happens to use rep;movsl to copy the argument memory. It could use memcpy, but either way, it will probe all those bytes.

I think that means it's safe to move to emitSPUpdate. I would greatly prefer that BuildStackAdjustment remain a simple primitive that generates a single instruction.

llvm/test/CodeGen/X86/stack-clash-dynamic-alloca.ll

10–12

This seems like a good use case for update_llc_test_checks.py. I'd want to see the body of the loop, the cmp, the jl, etc.

[serge-sans-paille]

Some benchmarks, running the python performance suite from recompiled cpython build, one built with -O2 (baseline) and one built with -O2 -fstack-clash-protection (protection). Surprisingly enough, the stack protection makes the code faster in various scenario, but none of these changes present significant regression, according to me.

baseline.json
=============

Performance version: 0.9.1
Report on Linux-3.10.0-891.el7.x86_64-x86_64-with-redhat-7.5-Maipo
Number of logical CPUs: 8
Start date: 2019-10-15 11:16:28.425344
End date: 2019-10-15 11:37:55.064599

protection.json
===============

Performance version: 0.9.1
Report on Linux-3.10.0-891.el7.x86_64-x86_64-with-redhat-7.5-Maipo
Number of logical CPUs: 8
Start date: 2019-10-15 10:55:24.270166
End date: 2019-10-15 11:16:27.195366

+-------------------------+---------------+-----------------+--------------+-----------------------+
| Benchmark               | baseline.json | protection.json | Change       | Significance          |
+=========================+===============+=================+==============+=======================+
| 2to3                    | 435 ms        | 448 ms          | 1.03x slower | Significant (t=-4.26) |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| chameleon               | 15.0 ms       | 14.7 ms         | 1.02x faster | Significant (t=4.76)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| chaos                   | 176 ms        | 174 ms          | 1.02x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| crypto_pyaes            | 153 ms        | 150 ms          | 1.02x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| deltablue               | 11.9 ms       | 11.9 ms         | 1.00x slower | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| django_template         | 210 ms        | 223 ms          | 1.06x slower | Significant (t=-3.84) |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| dulwich_log             | 96.1 ms       | 102 ms          | 1.06x slower | Significant (t=-8.42) |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| fannkuch                | 703 ms        | 698 ms          | 1.01x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| float                   | 161 ms        | 160 ms          | 1.01x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| genshi_text             | 45.5 ms       | 45.5 ms         | 1.00x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| genshi_xml              | 95.3 ms       | 95.2 ms         | 1.00x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| go                      | 392 ms        | 382 ms          | 1.03x faster | Significant (t=5.95)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| hexiom                  | 16.0 ms       | 15.9 ms         | 1.01x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| html5lib                | 137 ms        | 135 ms          | 1.01x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| json_dumps              | 18.8 ms       | 17.9 ms         | 1.05x faster | Significant (t=4.81)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| json_loads              | 41.3 us       | 40.2 us         | 1.03x faster | Significant (t=2.53)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| logging_format          | 17.4 us       | 17.5 us         | 1.00x slower | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| logging_silent          | 509 ns        | 500 ns          | 1.02x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| logging_simple          | 14.3 us       | 14.8 us         | 1.04x slower | Significant (t=-5.64) |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| mako                    | 30.6 ms       | 30.4 ms         | 1.01x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| meteor_contest          | 134 ms        | 130 ms          | 1.03x faster | Significant (t=4.07)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| nbody                   | 186 ms        | 180 ms          | 1.03x faster | Significant (t=4.87)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| nqueens                 | 152 ms        | 145 ms          | 1.05x faster | Significant (t=5.26)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| pathlib                 | 29.5 ms       | 27.7 ms         | 1.07x faster | Significant (t=4.33)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| pickle                  | 15.7 us       | 15.6 us         | 1.01x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| pickle_dict             | 45.1 us       | 43.2 us         | 1.04x faster | Significant (t=34.35) |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| pickle_list             | 6.05 us       | 5.67 us         | 1.07x faster | Significant (t=5.15)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| pickle_pure_python      | 766 us        | 786 us          | 1.03x slower | Significant (t=-3.19) |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| pidigits                | 206 ms        | 208 ms          | 1.01x slower | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| python_startup          | 12.4 ms       | 12.7 ms         | 1.03x slower | Significant (t=-8.03) |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| python_startup_no_site  | 7.62 ms       | 7.75 ms         | 1.02x slower | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| raytrace                | 830 ms        | 805 ms          | 1.03x faster | Significant (t=15.48) |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| regex_compile           | 264 ms        | 261 ms          | 1.01x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| regex_dna               | 195 ms        | 195 ms          | 1.00x slower | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| regex_effbot            | 3.59 ms       | 3.50 ms         | 1.02x faster | Significant (t=7.90)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| regex_v8                | 30.0 ms       | 29.3 ms         | 1.02x faster | Significant (t=7.10)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| richards                | 122 ms        | 120 ms          | 1.02x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| scimark_fft             | 466 ms        | 462 ms          | 1.01x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| scimark_lu              | 358 ms        | 334 ms          | 1.07x faster | Significant (t=5.46)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| scimark_monte_carlo     | 167 ms        | 155 ms          | 1.08x faster | Significant (t=4.68)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| scimark_sor             | 346 ms        | 328 ms          | 1.05x faster | Significant (t=7.55)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| scimark_sparse_mat_mult | 5.49 ms       | 5.14 ms         | 1.07x faster | Significant (t=13.23) |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| spectral_norm           | 209 ms        | 199 ms          | 1.05x faster | Significant (t=4.10)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| sympy_expand            | 650 ms        | 638 ms          | 1.02x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| sympy_integrate         | 30.8 ms       | 30.2 ms         | 1.02x faster | Significant (t=9.94)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| sympy_str               | 425 ms        | 407 ms          | 1.04x faster | Significant (t=4.26)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| sympy_sum               | 248 ms        | 248 ms          | 1.00x slower | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| telco                   | 9.63 ms       | 9.37 ms         | 1.03x faster | Significant (t=9.26)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| tornado_http            | 371 ms        | 369 ms          | 1.00x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| unpack_sequence         | 60.1 ns       | 59.0 ns         | 1.02x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| unpickle                | 19.3 us       | 18.9 us         | 1.02x faster | Significant (t=8.65)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| unpickle_list           | 4.72 us       | 4.67 us         | 1.01x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| unpickle_pure_python    | 576 us        | 562 us          | 1.02x faster | Significant (t=7.27)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| xml_etree_generate      | 156 ms        | 153 ms          | 1.02x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| xml_etree_iterparse     | 157 ms        | 146 ms          | 1.08x faster | Significant (t=6.27)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| xml_etree_parse         | 187 ms        | 183 ms          | 1.02x faster | Significant (t=4.14)  |
+-------------------------+---------------+-----------------+--------------+-----------------------+
| xml_etree_process       | 126 ms        | 123 ms          | 1.02x faster | Not significant       |
+-------------------------+---------------+-----------------+--------------+-----------------------+

[serge-sans-paille]

For maintenance reasons, I'd really prefer it if we could find a way to reuse the existing code that calls an external stack probe function. What do you think about taking a look at X86RetpolineThunks.cpp and doing something similar to that? Basically, when the user sets -fstack-clash-protection, LLVM will emit a small comdat+weak function into every object file that has the same ABI as the existing stack probe mechanism. For other prior art, you can also look at how __clang_call_terminate works.

@rnk I 100% understand the reasoning, and will have a look to the files you're pointing at. AFAIU the approach you're suggesting is incompatible with the « free probe » algorithm that moves instruction in between stack allocations, but let me confirm that first.

[serge-sans-paille]

Another test run on an x264 encoder (source: https://openbenchmarking.org/test/pts/x264)

Compiled with -O2 and with or without -fstack-clash-protection; Run without threads (x265 --pools 1 -F 1 ./Bosphorus_1920x1080_120fps_420_8bit_YUV.y4m /dev/null)

Clang

baseline:  318.60s 
protection:  317.72s

So no performance impact beyond noise.

The compilation inserts 44 inline probes in 9 functions.

gcc

Out of comparison, with gcc 8.2 (yeah, it's a bit old), I get (same flags & setup)

baseline : 417.53 sec
protected : 412.6 sec

The compilations inserts inline probes in 22 functions.
So gcc inserts more probes, the impact on performance is equally surprising. I need to gather more points to get some statistical informations there.

[serge-sans-paille]

Get rid of static mapping + update test cases

[serge-sans-paille]

Moved the implementation to a specialization of emitStackProbeInline that used to be Windows-centric. Provide a generic implementation that generates inline assembly instead. that way we don't clutter existing functions, and only add a new case to emitSPUpdate.

Handle large stack allocation with a loop instead of a (large) basic block.

@rnk: this is far less intrusive and more integrated to the existing structure, thanks a lot for hinting in that direction.

[serge-sans-paille]

Extra note: an older version of the patch has been tested by the firefox team without much performance impact, (and no test failure), see https://bugzilla.mozilla.org/show_bug.cgi?id=1588710

[serge-sans-paille]

Explicilty prefer existing mechanism for windows.
Split loop/block allocation strategy to different functions.

[sylvestre.ledru]

You should also probably document the arg in
clang/docs/ClangCommandLineReference.rst
we have -fstack-protector-strong & co in the doc

[serge-sans-paille]

Better integration with MachineInstr description
Handle stacks with multiple stack objects
More test case

[serge-sans-paille]

Update documentation as suggested by @sylvestre.ledru
Corner case when a write was touching memory beyond the allocated stack.

[serge-sans-paille]

Some benchmark / instrumentation report: due to the way memory moves are ordered in the entry block, there tend to be relatively few free probes between two stack growth within a function, and a large number after the last stack growth.

When recompiling llc, I only get 18 free probes between stack growth, and a large amount (several thousands) after the last stack growth.
When recompiling cpython, I only get 4 free probes between stack growth.

This may be improved by rescheduling instructions / changing stack object layout but that's probably not worth the effort (there's not that much functions with stack space > a page)

[serge-sans-paille]

Temporarily comment out call support as free probe, everything else passes validation but a call may have some stack effect I don't handle (yet).

[sylvestre.ledru]

clang/docs/ClangCommandLineReference.rst

1922

Maybe add that it is Linux only? :)

[serge-sans-paille]

Moved to a simpler, easier to review, less error prone implementation. Validates just fines on llvm bootstrap an cpython code base. @sylvestre.ledru that's the patch you can play with.

[serge-sans-paille]

@rnk what's your take on that version?

[serge-sans-paille]

@sylvestre.ledru : quick backport for llvm9: https://sergesanspaille.fedorapeople.org/0001-Stack-clash-mir-attempt.patch

[serge-sans-paille]

@sylvestre.ledru did the testing and benchmarking on firefox (see https://bugzilla.mozilla.org/show_bug.cgi?id=1588710#c12), everything seems ok, let's move forward?

[sylvestre.ledru]

In D68720#1733839, @serge-sans-paille wrote:

@sylvestre.ledru did the testing and benchmarking on firefox (see https://bugzilla.mozilla.org/show_bug.cgi?id=1588710#c12), everything seems ok, let's move forward?

Actually, not sure about the benchmark anymore. I am checking with experts. Sorry

[serge-sans-paille]

@rnk : up ?

[xbolva00]

https://reviews.llvm.org/rG397fa687691876de9ff0fbaaf0def3ac5a48899c

Commited?

[serge-sans-paille]

Added extra test for arch support (warns if the flag is used on unsupported targets, and checks it's effectively unused), cc @sylvestre.ledru

[serge-sans-paille]

In D68720#1755546, @xbolva00 wrote:

https://reviews.llvm.org/rG397fa687691876de9ff0fbaaf0def3ac5a48899c

Commited?

No, still waiting for @rnk feedback. This commit is just me using the wrong remote when pushing for validation.

[serge-sans-paille]

For the record : this validates fine on OSX, Linux and Windows, using the github actions setup by @tstellar

https://github.com/serge-sans-paille/llvm-project/pull/3

[serge-sans-paille]

@sylvestre.ledru backport for release/9.x branch: https://sergesanspaille.fedorapeople.org/0001-Stack-clash-mir-attempt.patch

[serge-sans-paille]

@rnk up? the mozilla folks confirmed there's no significant regression (see https://bugzilla.mozilla.org/show_bug.cgi?id=1588710#c12).

[craig.topper]

clang/docs/ClangCommandLineReference.rst

1920

Probably need a -fno-stack-class-protection as well. Looks like gcc has it. You'll need to update the handling in the driver to make sure the last polarity wins. For cc1 you might be able to support just the positive variant. But try to see what we usually do.

clang/lib/Basic/Targets/X86.h

169

What about 32-bit mode where the register name is "esp"?

clang/lib/CodeGen/CGStmt.cpp

2255

Why is this in the frontend diagnostic list?

clang/lib/Driver/ToolChains/Clang.cpp

3012

Can we use EffectiveTriple.isX86() that was just introduced yesterday?

3021

Seems like this should just be an if? Or maybe use Args.filtered or args getLastArg?

[tstellar]

clang/lib/Driver/ToolChains/Clang.cpp

3014

Do we need to warn (or error) here for arches that don't implement -fstack-clash-protection?

[serge-sans-paille]

Take review into account.

[serge-sans-paille]

clang/lib/CodeGen/CGStmt.cpp

2255

As far as I understand, that's the only place where line/col accurate diagnostics are emitted.

clang/lib/Driver/ToolChains/Clang.cpp

3014

That's done indirectly because the argument is not claimed on unsupported platform.

[sylvestre.ledru]

@craig.topper do you think there is a chance that this change could be part of clang-10 ?
Thanks

[craig.topper]

clang/lib/CodeGen/CGStmt.cpp

2255

I was really asking about why we have to use FrontendDiagnostic.h and DriverDiagnostic.h? Is that what we normally use for diagnostics in Codegen files? Seems like a possible layering issue for modules build

[serge-sans-paille]

clang/lib/CodeGen/CGStmt.cpp

2255

Oh, I understand now, let me have a look, your remark totally makes sense.

[serge-sans-paille]

• update warn_fe_stack_clash_protection_inline_asm location.

[serge-sans-paille]

@craig.topper up?

[craig.topper]

clang/include/clang/Basic/DiagnosticCommonKinds.td

243

Remove "fe_" from this? Looks like thats only there because this used to be DiagnosticFrontendKinds?

clang/lib/Basic/Targets/X86.h

169

This probably doesn't work correctly with X32 which is 64-bit but uses 32-bit pointers. Maybe we change this to "bool isSPReg(StringRef Reg)" and pass the register name to it so you can just check both strings always.

clang/lib/Driver/ToolChains/Clang.cpp

3015

I think this can use Args.hasFlag(options::OPT_fstack_clash_protection, options::OPT_fnostack_clash_protection, false) and then you don't need the 'if'

llvm/lib/Target/X86/X86ISelLowering.cpp

31213

This uses physSPReg, but doesn't match the condition for when physSPReg is a 64-bit register. Same at several places below.

llvm/lib/Target/X86/X86InstrCompiler.td

130

Why is this In64BitMode, but above is NotLP64. Shouldn't they be opposites? Looks like this was just copied from SEG_ALLOCA above?

[serge-sans-paille]

Take into account some of the reviews

[serge-sans-paille]

clang/lib/Driver/ToolChains/Clang.cpp

3015

I wasn't aware of that API, thanks for pointing this out o/

llvm/lib/Target/X86/X86ISelLowering.cpp

31213

Sorry, I don't understand your remark. Can you elaborate?

llvm/lib/Target/X86/X86InstrCompiler.td

130

Yeah, these intrinsics are just the same as the one above, except they have a different name. And they get lowered differently. I'm not familiar with that part of LLVM, can you suggest a better approach?

[merge_guards_bot]

Unit tests: fail. 61960 tests passed, 3 failed and 783 were skipped.

failed: Clang.CodeGen/stack-clash-protection.c
failed: Clang.Driver/stack-clash-protection.c
failed: Clang.Misc/warning-flags.c

clang-tidy: unknown.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[craig.topper]

llvm/lib/Target/X86/X86ISelLowering.cpp

31213

If physSPReg is RSP then you need to use SUB64ri32 and if its ESP you need to use SUB32ri. The condition you're using here is "IsLP64", but the condition for phySPReg to be RSP is "IsLP64 || Subtarget.isTargetNaCl64()". So there's a mismatch when IsLP64 is false and Subtarget.isTargetNaCl64() is true.

[serge-sans-paille]

• Harmonize esp/rsp usage as hinted by @craig.topper
• Fix argument detection

[merge_guards_bot]

Unit tests: fail. 62035 tests passed, 2 failed and 783 were skipped.

failed: Clang.CodeGen/stack-clash-protection.c
failed: Clang.Misc/warning-flags.c

clang-tidy: unknown.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[serge-sans-paille]

Update warning category

[merge_guards_bot]

Unit tests: fail. 62059 tests passed, 1 failed and 783 were skipped.

failed: Clang.CodeGen/stack-clash-protection.c

clang-tidy: unknown.

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

[serge-sans-paille]

Make some tests more portable.

[merge_guards_bot]

Unit tests: pass. 62269 tests passed, 0 failed and 827 were skipped.

clang-tidy: fail. clang-tidy found 0 errors and 16 warnings. 0 of them are added as review comments below (why?).

clang-format: fail. Please format your changes with clang-format by running git-clang-format HEAD^ or applying this patch.

Build artifacts: diff.json, clang-tidy.txt, clang-format.patch, CMakeCache.txt, console-log.txt, test-results.xml

Pre-merge checks is in beta. Report issue. Please join beta or enable it for your project.

[serge-sans-paille]

@craig.topper up :-)

[craig.topper]

LGTM

[xbolva00]

Any plans to merge this feature to 10 release?

[sylvestre.ledru]

In D68720#1862050, @xbolva00 wrote:

Any plans to merge this feature to 10 release?

I would love to have it in -10 for Firefox !

[serge-sans-paille]

I would love too, @hans what do you think?

[hans]

This seems to be breaking the buildbots, e.g.
http://lab.llvm.org:8011/builders/clang-cmake-x86_64-sde-avx512-linux/builds/33475/steps/ninja%20check%201/logs/FAIL%3A%20Clang%3A%3Astack-clash-protection.c
http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-ubuntu-fast/builds/62154/steps/test-check-all/logs/stdio
http://45.33.8.238/win/7706/step_7.txt

[thakis]

This breaks check-clang on mac and win: http://45.33.8.238/mac/7485/step_7.txt http://45.33.8.238/win/7753/step_7.txt

Please revert and then investigate asynchronously, unless the fix is obvious.

clang/test/CodeGen/stack-clash-protection.c

2

Tests should compile binaries and then run them if at all possible. This is impossible in cross-build scenarios and in general we try hard to have unit tests, not integration tests. Check that this produces the IR you expect, and have an llvm-level test that the IR produces the assembly you expect, and if you must, an lld level test that checks that the generated elf file looks like you think.

[thakis]

Reverted in b03c3d8c620 for now.

[vvereschaka]

Also got failed clang tests on Arm/Aarch64 builders

http://lab.llvm.org:8011/builders/llvm-clang-win-x-armv7l/builds/4295
http://lab.llvm.org:8011/builders/llvm-clang-win-x-aarch64/builds/4277

• FAIL: Clang::stack-clash-protection.c
• FAIL: Clang::stack-clash-protection.c

error: unable to create target: 'No available targets are compatible with triple "x86_64"'

looks like REQUIRES: x86-registered-target keyword is needed.

[RKSimon]

@serge-sans-paille Please can you fix the issues with EXPENSIVE_CHECKS builds: http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-debian/builds/2604

[serge-sans-paille]

For the record: tests have been updated, option handling cleaned up and expansive check failure fixed in commit e67cbac81211d40332a79d98c9d5953624cc1202.

[sberg]

clang/include/clang/Driver/Options.td

1778

Should this rather spell "fno-stack-clash-protection"? The above change to clang/docs/ClangCommandLineReference.rst (but which got overwritten by https://github.com/llvm/llvm-project/commit/9624beb38a46037f69362650a52e06d8be4fd006 "[docs] Regenerate ClangCommandLineReference.rst") mentions -fno-stack-clash-protection, and also GCC calls it like that. (Though the below clang/test/Driver/stack-clash-protection.c does use -fnostack-clash-protection.)

[xbolva00]

clang/include/clang/Driver/Options.td

1778

And the fix should be backported to release branch.

@hans

[sberg]

I filed https://bugs.llvm.org/show_bug.cgi?id=47139 "Misspelled -fnostack-clash-protection" now.

[kees]

Sorry if I missed something here, but why is this marked as "Closed"? It seems like the feature has still not landed (i.e. it got reverted).

[efriedma]

It was reverted, then re-landed. It's in trunk now.

[kees]

Ah! Yes, I see it now. Thanks and sorry for the noise!

[emaste]

clang/lib/Driver/ToolChains/Clang.cpp

3009–3010

Is there anything OS-dependent here?

I plan to add EffectiveTriple.isOSFreeBSD() to FreeBSD-s in-tree Clang
https://reviews.freebsd.org/D27366
and intend to upstream the change (with a test), but should we just remove the OS test completely?

[serge-sans-paille]

clang/lib/Driver/ToolChains/Clang.cpp

3009–3010

According to https://blog.qualys.com/vulnerabilities-research/2017/06/19/the-stack-clash, stack clash impacts

Linux, OpenBSD, NetBSD, FreeBSD and Solaris

We should probably enlarge the test to these platforms.

clang/docs/ClangCommandLineReference.rst

 .. option:: -fstack-protector-strong
 
 Use a strong heuristic to apply stack protectors to functions
 
 .. option:: -fstack-size-section, -fno-stack-size-section
 
 Emit section containing metadata on function stack sizes
 
+.. option:: -fstack-clash-protection, -fno-stack-clash-protection
+
+Instrument stack allocation to prevent stack clash attacks (x86, non-Windows only).
+
 .. option:: -fstandalone-debug, -fno-limit-debug-info, -fno-standalone-debug
 
 Emit full debug info for all types used by the program
 
 .. option:: -fstrict-aliasing, -fno-strict-aliasing
 
 .. option:: -fstrict-enums, -fno-strict-enums
 

clang/docs/ReleaseNotes.rst

 Non-comprehensive list of changes in this release
 -------------------------------------------------
 
 
 New Compiler Flags
 ------------------
 
 
+- -fstack-clash-protection will provide a protection against the stack clash
+  attack for x86 architecture through automatic probing of each page of
+  allocated stack.
+
 Deprecated Compiler Flags
 -------------------------
 
 The following options are deprecated and ignored. They will be removed in
 future versions of Clang.
 
 - ...
 

clang/include/clang/Basic/CodeGenOptions.def

CODEGENOPT(NoDwarfDirectoryAsm , 1, 0) ///< Set when -fno-dwarf-directory-asm is
                                        ///< enabled.
 CODEGENOPT(NoExecStack       , 1, 0) ///< Set when -Wa,--noexecstack is enabled.
 CODEGENOPT(FatalWarnings     , 1, 0) ///< Set when -Wa,--fatal-warnings is
                                      ///< enabled.
 CODEGENOPT(NoWarn            , 1, 0) ///< Set when -Wa,--no-warn is enabled.
 CODEGENOPT(EnableSegmentedStacks , 1, 0) ///< Set when -fsplit-stack is enabled.
 CODEGENOPT(NoInlineLineTables, 1, 0) ///< Whether debug info should contain
                                      ///< inline line tables.
+CODEGENOPT(StackClashProtector, 1, 0) ///< Set when -fstack-clash-protection is enabled.
 CODEGENOPT(NoImplicitFloat   , 1, 0) ///< Set when -mno-implicit-float is enabled.
 CODEGENOPT(NoInfsFPMath      , 1, 0) ///< Assume FP arguments, results not +-Inf.
 CODEGENOPT(NoSignedZeros     , 1, 0) ///< Allow ignoring the signedness of FP zero
 CODEGENOPT(NullPointerIsValid , 1, 0) ///< Assume Null pointer deference is defined.
 CODEGENOPT(Reassociate       , 1, 0) ///< Allow reassociation of FP math ops
 CODEGENOPT(ReciprocalMath    , 1, 0) ///< Allow FP divisions to be reassociated.
 CODEGENOPT(NoTrappingMath    , 1, 0) ///< Set when -fno-trapping-math is enabled.
 CODEGENOPT(NoNaNsFPMath      , 1, 0) ///< Assume FP arguments, results not NaN.

clang/include/clang/Basic/DiagnosticCommonKinds.td

 // Sema && AST
 def note_invalid_subexpr_in_const_expr : Note<
   "subexpression not valid in a constant expression">;
 
 // Sema && Frontend
 let CategoryName = "Inline Assembly Issue" in {
   def err_asm_invalid_type_in_input : Error<
     "invalid type %0 in asm input for constraint '%1'">;
+
+  def warn_stack_clash_protection_inline_asm : Warning<
+    "Unable to protect inline asm that clobbers stack pointer against stack clash">,
+    InGroup<DiagGroup<"stack-protector">>;
 }
 
 // Sema && Serialization
 def warn_dup_category_def : Warning<
   "duplicate definition of category %1 on interface %0">;
 
 // Targets
 

clang/include/clang/Basic/TargetInfo.h

public:
   /// Returns the "normalized" GCC register name.
   ///
   /// ReturnCannonical true will return the register name without any additions
   /// such as "{}" or "%" in it's canonical form, for example:
   /// ReturnCanonical = true and Name = "rax", will return "ax".
   StringRef getNormalizedGCCRegisterName(StringRef Name,
                                          bool ReturnCanonical = false) const;
 
+  virtual bool isSPRegName(StringRef) const { return false; }
+
   /// Extracts a register from the passed constraint (if it is a
   /// single-register constraint) and the asm label expression related to a
   /// variable in the input or output list of an inline asm statement.
   ///
   /// This function is used by Sema in order to diagnose conflicts between
   /// the clobber list and the input/output lists.
   virtual StringRef getConstraintRegister(StringRef Constraint,
                                           StringRef Expression) const {

clang/include/clang/Driver/CC1Options.td

 def error_on_deserialized_pch_decl_EQ : Joined<["-"], "error-on-deserialized-decl=">,
   Alias<error_on_deserialized_pch_decl>;
 def static_define : Flag<["-"], "static-define">,
   HelpText<"Should __STATIC__ be defined">;
 def stack_protector : Separate<["-"], "stack-protector">,
   HelpText<"Enable stack protectors">;
 def stack_protector_buffer_size : Separate<["-"], "stack-protector-buffer-size">,
   HelpText<"Lower bound for a buffer to be considered for stack protection">;
+def stack_clash_protection : Separate<["-"], "stack-clash-protection">,
+  HelpText<"Enable stack clash protection">;
 def fvisibility : Separate<["-"], "fvisibility">,
   HelpText<"Default type and symbol visibility">;
 def ftype_visibility : Separate<["-"], "ftype-visibility">,
   HelpText<"Default type visibility">;
 def fapply_global_visibility_to_externs : Flag<["-"], "fapply-global-visibility-to-externs">,
   HelpText<"Apply global symbol visibility to external declarations without an explicit visibility">;
 def ftemplate_depth : Separate<["-"], "ftemplate-depth">,
   HelpText<"Maximum depth of recursive template instantiation">;

clang/include/clang/Driver/Options.td

 def fspell_checking_limit_EQ : Joined<["-"], "fspell-checking-limit=">, Group<f_Group>;
 def fsigned_bitfields : Flag<["-"], "fsigned-bitfields">, Group<f_Group>;
 def fsigned_char : Flag<["-"], "fsigned-char">, Group<f_Group>;
 def fno_signed_char : Flag<["-"], "fno-signed-char">, Group<f_Group>,
     Flags<[CC1Option]>, HelpText<"Char is unsigned">;
 def fsplit_stack : Flag<["-"], "fsplit-stack">, Group<f_Group>;
 def fstack_protector_all : Flag<["-"], "fstack-protector-all">, Group<f_Group>,
   HelpText<"Enable stack protectors for all functions">;
+def fstack_clash_protection : Flag<["-"], "fstack-clash-protection">, Group<f_Group>,
+  HelpText<"Enable stack clash protection">;
+def fnostack_clash_protection : Flag<["-"], "fnostack-clash-protection">, Group<f_Group>,
+  HelpText<"Disable stack clash protection">;
 def fstack_protector_strong : Flag<["-"], "fstack-protector-strong">, Group<f_Group>,
   HelpText<"Enable stack protectors for some functions vulnerable to stack smashing. "
            "Compared to -fstack-protector, this uses a stronger heuristic "
            "that includes functions containing arrays of any size (and any type), "
            "as well as any calls to alloca or the taking of an address from a local variable">;
 def fstack_protector : Flag<["-"], "fstack-protector">, Group<f_Group>,
   HelpText<"Enable stack protectors for some functions vulnerable to stack smashing. "
            "This uses a loose heuristic which considers functions vulnerable "

clang/lib/Basic/Targets/X86.h

public:
   ArrayRef<const char *> getGCCRegNames() const override;
 
   ArrayRef<TargetInfo::GCCRegAlias> getGCCRegAliases() const override {
     return None;
   }
 
   ArrayRef<TargetInfo::AddlRegName> getGCCAddlRegNames() const override;
 
+  bool isSPRegName(StringRef RegName) const override {
+    return RegName.equals("esp") || RegName.equals("rsp");
+  }
+
   bool validateCpuSupports(StringRef Name) const override;
 
   bool validateCpuIs(StringRef Name) const override;
 
   bool validateCPUSpecificCPUDispatch(StringRef Name) const override;
 
   char CPUSpecificManglingCharacter(StringRef Name) const override;
 

clang/lib/CodeGen/CGStmt.cpp

void CodeGenFunction::EmitAsmStmt(const AsmStmt &S) {
   }
 
   // Clobbers
   for (unsigned i = 0, e = S.getNumClobbers(); i != e; i++) {
     StringRef Clobber = S.getClobber(i);
 
     if (Clobber == "memory")
       ReadOnly = ReadNone = false;
-    else if (Clobber != "cc")
+    else if (Clobber != "cc") {
       Clobber = getTarget().getNormalizedGCCRegisterName(Clobber);
+      if (CGM.getCodeGenOpts().StackClashProtector &&
+          getTarget().isSPRegName(Clobber)) {
+        CGM.getDiags().Report(S.getAsmLoc(),
+                              diag::warn_stack_clash_protection_inline_asm);
+      }
+    }
 
     if (!Constraints.empty())
       Constraints += ',';
 
     Constraints += "~{";
     Constraints += Clobber;
     Constraints += '}';
   }

clang/lib/CodeGen/CodeGenModule.cpp

 
 void CodeGenModule::SetLLVMFunctionAttributesForDefinition(const Decl *D,
                                                            llvm::Function *F) {
   llvm::AttrBuilder B;
 
   if (CodeGenOpts.UnwindTables)
     B.addAttribute(llvm::Attribute::UWTable);
 
+  if (CodeGenOpts.StackClashProtector)
+    B.addAttribute("probe-stack", "inline-asm");
+
   if (!hasUnwindExceptions(LangOpts))
     B.addAttribute(llvm::Attribute::NoUnwind);
 
   if (!D || !D->hasAttr<NoStackProtectorAttr>()) {
     if (LangOpts.getStackProtector() == LangOptions::SSPOn)
       B.addAttribute(llvm::Attribute::StackProtect);
     else if (LangOpts.getStackProtector() == LangOptions::SSPStrong)
       B.addAttribute(llvm::Attribute::StackProtectStrong);

clang/lib/Driver/ToolChains/Clang.cpp

if (Str.startswith("ssp-buffer-size=")) {
         // FIXME: Verify the argument is a valid integer.
         CmdArgs.push_back(Args.MakeArgString(Str.drop_front(16)));
       }
       A->claim();
     }
   }
 }
 
+static void RenderSCPOptions(const ToolChain &TC, const ArgList &Args,
+                             ArgStringList &CmdArgs) {
+  const llvm::Triple &EffectiveTriple = TC.getEffectiveTriple();
+
+  if (!EffectiveTriple.isOSLinux())
+    return;
+
+  if (!EffectiveTriple.isX86())
+    return;
+
+  if (Args.hasFlag(options::OPT_fstack_clash_protection,
+                   options::OPT_fnostack_clash_protection, false))
+    CmdArgs.push_back("-stack-clash-protection");
+}
+
 static void RenderTrivialAutoVarInitOptions(const Driver &D,
                                             const ToolChain &TC,
                                             const ArgList &Args,
                                             ArgStringList &CmdArgs) {
   auto DefaultTrivialAutoVarInit = TC.GetDefaultTrivialAutoVarInit();
   StringRef TrivialAutoVarInit = "";
 
   for (const Arg *A : Args) {
     switch (A->getOption().getID()) {
     default:
#endif
 
   Args.AddLastArg(CmdArgs, options::OPT_pthread);
 
   if (Args.hasFlag(options::OPT_mspeculative_load_hardening, options::OPT_mno_speculative_load_hardening,
                    false))
     CmdArgs.push_back(Args.MakeArgString("-mspeculative-load-hardening"));
 
   RenderSSPOptions(TC, Args, CmdArgs, KernelOrKext);
+  RenderSCPOptions(TC, Args, CmdArgs);
   RenderTrivialAutoVarInitOptions(D, TC, Args, CmdArgs);
 
   // Translate -mstackrealign
   if (Args.hasFlag(options::OPT_mstackrealign, options::OPT_mno_stackrealign,
                    false))
     CmdArgs.push_back(Args.MakeArgString("-mstackrealign"));
 
   if (Args.hasArg(options::OPT_mstack_alignment)) {

clang/lib/Frontend/CompilerInvocation.cpp

if (Arg *A = Args.getLastArg(OPT_mstack_probe_size)) {
     StringRef Val = A->getValue();
     unsigned StackProbeSize = Opts.StackProbeSize;
     Val.getAsInteger(0, StackProbeSize);
     Opts.StackProbeSize = StackProbeSize;
   }
 
   Opts.NoStackArgProbe = Args.hasArg(OPT_mno_stack_arg_probe);
 
+  Opts.StackClashProtector = Args.hasArg(OPT_stack_clash_protection);
+
   if (Arg *A = Args.getLastArg(OPT_fobjc_dispatch_method_EQ)) {
     StringRef Name = A->getValue();
     unsigned Method = llvm::StringSwitch<unsigned>(Name)
       .Case("legacy", CodeGenOptions::Legacy)
       .Case("non-legacy", CodeGenOptions::NonLegacy)
       .Case("mixed", CodeGenOptions::Mixed)
       .Default(~0U);
     if (Method == ~0U) {

clang/test/CodeGen/stack-clash-protection.c

+// check interaction between -fstack-clash-protection and dynamic allocation schemes
+// RUN: %clang -target x86_64 -O0 -o %t.out %s -fstack-clash-protection && %t.out
+
+int large_stack() __attribute__((noinline));
+
+int large_stack() {
+  int stack[20000], i;
+  for (i = 0; i < sizeof(stack) / sizeof(int); ++i)
+    stack[i] = i;
+  return stack[1];
+}
+
+int main(int argc, char **argv) {
+  int volatile static_mem[8000];
+  for (unsigned i = 0; i < argc * sizeof(static_mem) / sizeof(static_mem[0]); ++i)
+    static_mem[i] = argc * i;
+
+  int vla[argc];
+  __builtin_memset(&vla[0], 0, argc);
+
+  int index = large_stack();
+
+  // also check allocation of 0 size
+  volatile void *mem = __builtin_alloca(argc - 1);
+
+  int volatile *dyn_mem = __builtin_alloca(sizeof(static_mem) * argc);
+  for (unsigned i = 0; i < argc * sizeof(static_mem) / sizeof(static_mem[0]); ++i)
+    dyn_mem[i] = argc * i;
+
+  return static_mem[(7999 * argc) / 2] - dyn_mem[(7999 * argc) / 2] + vla[argc - index];
+}

clang/test/Driver/stack-clash-protection.c

+// RUN: %clang -target i386-unknown-linux -fstack-clash-protection -### %s 2>&1 | FileCheck %s -check-prefix=SCP-i386
+// RUN: %clang -target i386-unknown-linux -fnostack-clash-protection -fstack-clash-protection -### %s 2>&1 | FileCheck %s -check-prefix=SCP-i386
+// RUN: %clang -target i386-unknown-linux -fstack-clash-protection -fnostack-clash-protection -### %s 2>&1 | FileCheck %s -check-prefix=SCP-i386-NO
+// SCP-i386: "-stack-clash-protection"
+// SCP-i386-NO-NOT: "-stack-clash-protection"
+
+// RUN: %clang -target x86_64-scei-linux -fstack-clash-protection -### %s 2>&1 | FileCheck %s -check-prefix=SCP-x86
+// SCP-x86: "-stack-clash-protection"
+
+// RUN: %clang -target armv7k-apple-linux -fstack-clash-protection -### %s 2>&1 | FileCheck %s -check-prefix=SCP-armv7
+// SCP-armv7-NOT: "-stack-clash-protection"
+// SCP-armv7: argument unused during compilation: '-fstack-clash-protection'
+
+// RUN: %clang -target x86_64-unknown-linux -fstack-clash-protection -c %s 2>&1 | FileCheck %s -check-prefix=SCP-warn
+// SCP-warn: warning: Unable to protect inline asm that clobbers stack pointer against stack clash
+
+// RUN: %clang -target x86_64-pc-unknown-linux -fstack-clash-protection -S -emit-llvm -o- %s | FileCheck %s -check-prefix=SCP-ll-linux64
+// SCP-ll-linux64: attributes {{.*}} "probe-stack"="inline-asm"
+
+// RUN: %clang -target x86_64-pc-windows-msvc -fstack-clash-protection -S -emit-llvm -o- %s 2>&1 | FileCheck %s -check-prefix=SCP-ll-win64
+// SCP-ll-win64-NOT: attributes {{.*}} "probe-stack"="inline-asm"
+// SCP-ll-win64: argument unused during compilation: '-fstack-clash-protection'
+
+int foo(int c) {
+  int r;
+  __asm__("sub %0, %%rsp"
+          :
+          : "rm"(c)
+          : "rsp");
+  __asm__("mov %%rsp, %0"
+          : "=rm"(r)::);
+  return r;
+}

llvm/docs/ReleaseNotes.rst

 During this release ...
 
 Changes to the X86 Target
 -------------------------
 
 During this release ...
 
 
+* Functions with the probe-stack attribute set to "inline-asm" are now protected
+  against stack clash without the need of a third-party probing function and
+  with limited impact on performance.
+
 Changes to the AMDGPU Target
 -----------------------------
 
 Changes to the AVR Target
 -----------------------------
 
 During this release ...
 

llvm/include/llvm/CodeGen/TargetLowering.h

Value *getDefaultSafeStackPointerLocation(IRBuilder<> &IRB,
                                             bool UseTLS) const;
 
 public:
   /// Returns the target-specific address of the unsafe stack pointer.
   virtual Value *getSafeStackPointerLocation(IRBuilder<> &IRB) const;
 
   /// Returns the name of the symbol used to emit stack probes or the empty
   /// string if not applicable.
+  virtual bool hasStackProbeSymbol(MachineFunction &MF) const { return false; }
+
+  virtual bool hasInlineStackProbe(MachineFunction &MF) const { return false; }
+
   virtual StringRef getStackProbeSymbolName(MachineFunction &MF) const {
     return "";
   }
 
   /// Returns true if a cast between SrcAS and DestAS is a noop.
   virtual bool isNoopAddrSpaceCast(unsigned SrcAS, unsigned DestAS) const {
     return false;
   }

llvm/lib/Target/X86/X86CallFrameOptimization.cpp

bool X86CallFrameOptimization::isLegal(MachineFunction &MF) {
   // the same block, and, for good measure, that there are no nested frames.
   //
   // If any call allocates more argument stack memory than the stack
   // probe size, don't do this optimization. Otherwise, this pass
   // would need to synthesize additional stack probe calls to allocate
   // memory for arguments.
   unsigned FrameSetupOpcode = TII->getCallFrameSetupOpcode();
   unsigned FrameDestroyOpcode = TII->getCallFrameDestroyOpcode();
-  bool UseStackProbe =
-      !STI->getTargetLowering()->getStackProbeSymbolName(MF).empty();
+  bool EmitStackProbeCall = STI->getTargetLowering()->hasStackProbeSymbol(MF);
   unsigned StackProbeSize = STI->getTargetLowering()->getStackProbeSize(MF);
   for (MachineBasicBlock &BB : MF) {
     bool InsideFrameSequence = false;
     for (MachineInstr &MI : BB) {
       if (MI.getOpcode() == FrameSetupOpcode) {
-        if (TII->getFrameSize(MI) >= StackProbeSize && UseStackProbe)
+        if (TII->getFrameSize(MI) >= StackProbeSize && EmitStackProbeCall)
           return false;
         if (InsideFrameSequence)
           return false;
         InsideFrameSequence = true;
       } else if (MI.getOpcode() == FrameDestroyOpcode) {
         if (!InsideFrameSequence)
           return false;
         InsideFrameSequence = false;

llvm/lib/Target/X86/X86FrameLowering.h

private:
   void emitStackProbeCall(MachineFunction &MF, MachineBasicBlock &MBB,
                           MachineBasicBlock::iterator MBBI, const DebugLoc &DL,
                           bool InProlog) const;
 
   /// Emit target stack probe as an inline sequence.
   void emitStackProbeInline(MachineFunction &MF, MachineBasicBlock &MBB,
                             MachineBasicBlock::iterator MBBI,
                             const DebugLoc &DL, bool InProlog) const;
+  void emitStackProbeInlineWindowsCoreCLR64(MachineFunction &MF,
+                                            MachineBasicBlock &MBB,
+                                            MachineBasicBlock::iterator MBBI,
+                                            const DebugLoc &DL,
+                                            bool InProlog) const;
+  void emitStackProbeInlineGeneric(MachineFunction &MF, MachineBasicBlock &MBB,
+                                   MachineBasicBlock::iterator MBBI,
+                                   const DebugLoc &DL, bool InProlog) const;
+
+  void emitStackProbeInlineGenericBlock(MachineFunction &MF,
+                                        MachineBasicBlock &MBB,
+                                        MachineBasicBlock::iterator MBBI,
+                                        const DebugLoc &DL,
+                                        uint64_t Offset) const;
+
+  void emitStackProbeInlineGenericLoop(MachineFunction &MF,
+                                       MachineBasicBlock &MBB,
+                                       MachineBasicBlock::iterator MBBI,
+                                       const DebugLoc &DL,
+                                       uint64_t Offset) const;
 
   /// Emit a stub to later inline the target stack probe.
-  void emitStackProbeInlineStub(MachineFunction &MF, MachineBasicBlock &MBB,
+  MachineInstr *emitStackProbeInlineStub(MachineFunction &MF,
+                                         MachineBasicBlock &MBB,
                                          MachineBasicBlock::iterator MBBI,
-                                const DebugLoc &DL, bool InProlog) const;
+                                         const DebugLoc &DL,
+                                         bool InProlog) const;
 
   /// Aligns the stack pointer by ANDing it with -MaxAlign.
   void BuildStackAlignAND(MachineBasicBlock &MBB,
                           MachineBasicBlock::iterator MBBI, const DebugLoc &DL,
                           unsigned Reg, uint64_t MaxAlign) const;
 
   /// Make small positive stack adjustments using POPs.
   bool adjustStackWithPops(MachineBasicBlock &MBB,

llvm/lib/Target/X86/X86FrameLowering.cpp

 
 #include "X86FrameLowering.h"
 #include "X86InstrBuilder.h"
 #include "X86InstrInfo.h"
 #include "X86MachineFunctionInfo.h"
 #include "X86Subtarget.h"
 #include "X86TargetMachine.h"
 #include "llvm/ADT/SmallSet.h"
+#include "llvm/ADT/Statistic.h"
 #include "llvm/Analysis/EHPersonalities.h"
 #include "llvm/CodeGen/MachineFrameInfo.h"
 #include "llvm/CodeGen/MachineFunction.h"
 #include "llvm/CodeGen/MachineInstrBuilder.h"
 #include "llvm/CodeGen/MachineModuleInfo.h"
 #include "llvm/CodeGen/MachineRegisterInfo.h"
 #include "llvm/CodeGen/WinEHFuncInfo.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/Function.h"
 #include "llvm/MC/MCAsmInfo.h"
 #include "llvm/MC/MCSymbol.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Target/TargetOptions.h"
 #include <cstdlib>
 
+#define DEBUG_TYPE "x86-fl"
+
+STATISTIC(NumFrameLoopProbe, "Number of loop stack probes used in prologue");
+STATISTIC(NumFrameExtraProbe,
+          "Number of extra stack probes generated in prologue");
+
 using namespace llvm;
 
 X86FrameLowering::X86FrameLowering(const X86Subtarget &STI,
                                    MaybeAlign StackAlignOverride)
     : TargetFrameLowering(StackGrowsDown, StackAlignOverride.valueOrOne(),
                           STI.is64Bit() ? -8 : -4),
       STI(STI), TII(*STI.getInstrInfo()), TRI(STI.getRegisterInfo()) {
   // Cache a bunch of frame-related predicates for this subtarget.
void X86FrameLowering::emitSPUpdate(MachineBasicBlock &MBB,
                                     int64_t NumBytes, bool InEpilogue) const {
   bool isSub = NumBytes < 0;
   uint64_t Offset = isSub ? -NumBytes : NumBytes;
   MachineInstr::MIFlag Flag =
       isSub ? MachineInstr::FrameSetup : MachineInstr::FrameDestroy;
 
   uint64_t Chunk = (1LL << 31) - 1;
 
-  if (Offset > Chunk) {
+  MachineFunction &MF = *MBB.getParent();
+  const X86Subtarget &STI = MF.getSubtarget<X86Subtarget>();
+  const X86TargetLowering &TLI = *STI.getTargetLowering();
+  const bool EmitInlineStackProbe = TLI.hasInlineStackProbe(MF);
+
+  // It's ok to not take into account large chunks when probing, as the
+  // allocation is split in smaller chunks anyway.
+  if (EmitInlineStackProbe && !InEpilogue) {
+
+    // stack probing may involve looping, and control flow generations is
+    // disallowed at this point. Rely to later processing through
+    // `inlineStackProbe`.
+    MachineInstr *Stub = emitStackProbeInlineStub(MF, MBB, MBBI, DL, true);
+
+    // Encode the static offset as a metadata attached to the stub.
+    LLVMContext &Context = MF.getFunction().getContext();
+    MachineInstrBuilder(MF, Stub).addMetadata(
+        MDTuple::get(Context, {ConstantAsMetadata::get(ConstantInt::get(
+                                  IntegerType::get(Context, 64), Offset))}));
+    return;
+  } else if (Offset > Chunk) {
     // Rather than emit a long series of instructions for large offsets,
     // load the offset into a register and do one sub/add
     unsigned Reg = 0;
     unsigned Rax = (unsigned)(Is64Bit ? X86::RAX : X86::EAX);
 
     if (isSub && !isEAXLiveIn(MBB))
       Reg = Rax;
     else
while (Offset) {
 
     BuildStackAdjustment(MBB, MBBI, DL, isSub ? -ThisVal : ThisVal, InEpilogue)
         .setMIFlag(Flag);
 
     Offset -= ThisVal;
   }
 }
 
 MachineInstrBuilder X86FrameLowering::BuildStackAdjustment(
     MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI,
     const DebugLoc &DL, int64_t Offset, bool InEpilogue) const {
   assert(Offset != 0 && "zero offset stack adjustment requested");
 
   // On Atom, using LEA to adjust SP is preferred, but using it in the epilogue
   // is tricky.
   bool UseLEA;
   if (!InEpilogue) {
MachineInstrBuilder X86FrameLowering::BuildStackAdjustment(
   if (UseLEA) {
     MI = addRegOffset(BuildMI(MBB, MBBI, DL,
                               TII.get(getLEArOpcode(Uses64BitFramePtr)),
                               StackPtr),
                       StackPtr, false, Offset);
   } else {
     bool IsSub = Offset < 0;
     uint64_t AbsOffset = IsSub ? -Offset : Offset;
-    unsigned Opc = IsSub ? getSUBriOpcode(Uses64BitFramePtr, AbsOffset)
+    const unsigned Opc = IsSub ? getSUBriOpcode(Uses64BitFramePtr, AbsOffset)
                                : getADDriOpcode(Uses64BitFramePtr, AbsOffset);
     MI = BuildMI(MBB, MBBI, DL, TII.get(Opc), StackPtr)
              .addReg(StackPtr)
              .addImm(AbsOffset);
     MI->getOperand(3).setIsDead(); // The EFLAGS implicit def is dead.
   }
   return MI;
 }
 
 int X86FrameLowering::mergeSPUpdates(MachineBasicBlock &MBB,
                                      MachineBasicBlock::iterator &MBBI,
                                      bool doMergeWithPrevious) const {
   if ((doMergeWithPrevious && MBBI == MBB.begin()) ||
       (!doMergeWithPrevious && MBBI == MBB.end()))
     return 0;
 
   MachineBasicBlock::iterator PI = doMergeWithPrevious ? std::prev(MBBI) : MBBI;
 
   PI = skipDebugInstructionsBackward(PI, MBB.begin());
   // It is assumed that ADD/SUB/LEA instruction is succeded by one CFI
   // instruction, and that there are no DBG_VALUE or other instructions between
   // ADD/SUB/LEA and its corresponding CFI instruction.
   /* TODO: Add support for the case where there are multiple CFI instructions
     below the ADD/SUB/LEA, e.g.:
     ...
     add
     cfi_def_cfa_offset
     cfi_offset
     ...
   */
   if (doMergeWithPrevious && PI != MBB.begin() && PI->isCFIInstruction())
     PI = std::prev(PI);
 
   unsigned Opc = PI->getOpcode();
   int Offset = 0;
 
   if ((Opc == X86::ADD64ri32 || Opc == X86::ADD64ri8 ||
        Opc == X86::ADD32ri || Opc == X86::ADD32ri8) &&
       PI->getOperand(0).getReg() == StackPtr){
     assert(PI->getOperand(1).getReg() == StackPtr);
     Offset = PI->getOperand(2).getImm();
   } else if ((Opc == X86::LEA32r || Opc == X86::LEA64_32r) &&
              PI->getOperand(0).getReg() == StackPtr &&
              PI->getOperand(1).getReg() == StackPtr &&
              PI->getOperand(2).getImm() == 1 &&
              PI->getOperand(3).getReg() == X86::NoRegister &&
              PI->getOperand(5).getReg() == X86::NoRegister) {
     // For LEAs we have: def = lea SP, FI, noreg, Offset, noreg.
     Offset = PI->getOperand(4).getImm();
   } else if ((Opc == X86::SUB64ri32 || Opc == X86::SUB64ri8 ||

 }
 
 void X86FrameLowering::emitStackProbeInline(MachineFunction &MF,
                                             MachineBasicBlock &MBB,
                                             MachineBasicBlock::iterator MBBI,
                                             const DebugLoc &DL,
                                             bool InProlog) const {
   const X86Subtarget &STI = MF.getSubtarget<X86Subtarget>();
+  if (STI.isTargetWindowsCoreCLR() && STI.is64Bit())
+    emitStackProbeInlineWindowsCoreCLR64(MF, MBB, MBBI, DL, InProlog);
+  else
+    emitStackProbeInlineGeneric(MF, MBB, MBBI, DL, InProlog);
+}
+
+void X86FrameLowering::emitStackProbeInlineGeneric(
+    MachineFunction &MF, MachineBasicBlock &MBB,
+    MachineBasicBlock::iterator MBBI, const DebugLoc &DL, bool InProlog) const {
+  MachineInstr &CallToInline = *std::prev(MBBI);
+  assert(CallToInline.getOperand(1).isMetadata() &&
+         "no metadata attached to that probe");
+  uint64_t Offset =
+      cast<ConstantInt>(
+          cast<ConstantAsMetadata>(
+              cast<MDTuple>(CallToInline.getOperand(1).getMetadata())
+                  ->getOperand(0))
+              ->getValue())
+          ->getZExtValue();
+
+  const X86Subtarget &STI = MF.getSubtarget<X86Subtarget>();
+  const X86TargetLowering &TLI = *STI.getTargetLowering();
+  assert(!(STI.is64Bit() && STI.isTargetWindowsCoreCLR()) &&
+         "different expansion expected for CoreCLR 64 bit");
+
+  const uint64_t StackProbeSize = TLI.getStackProbeSize(MF);
+  uint64_t ProbeChunk = StackProbeSize * 8;
+
+  // Synthesize a loop or unroll it, depending on the number of iterations.
+  if (Offset > ProbeChunk) {
+    emitStackProbeInlineGenericLoop(MF, MBB, MBBI, DL, Offset);
+  } else {
+    emitStackProbeInlineGenericBlock(MF, MBB, MBBI, DL, Offset);
+  }
+}
+
+void X86FrameLowering::emitStackProbeInlineGenericBlock(
+    MachineFunction &MF, MachineBasicBlock &MBB,
+    MachineBasicBlock::iterator MBBI, const DebugLoc &DL,
+    uint64_t Offset) const {
+
+  const X86Subtarget &STI = MF.getSubtarget<X86Subtarget>();
+  const X86TargetLowering &TLI = *STI.getTargetLowering();
+  const unsigned Opc = getSUBriOpcode(Uses64BitFramePtr, Offset);
+  const unsigned MovMIOpc = Is64Bit ? X86::MOV64mi32 : X86::MOV32mi;
+  const uint64_t StackProbeSize = TLI.getStackProbeSize(MF);
+  uint64_t CurrentOffset = 0;
+  // 0 Thanks to return address being saved on the stack
+  uint64_t CurrentProbeOffset = 0;
+
+  // For the first N - 1 pages, just probe. I tried to take advantage of
+  // natural probes but it implies much more logic and there was very few
+  // interesting natural probes to interleave.
+  while (CurrentOffset + StackProbeSize < Offset) {
+    MachineInstr *MI = BuildMI(MBB, MBBI, DL, TII.get(Opc), StackPtr)
+                           .addReg(StackPtr)
+                           .addImm(StackProbeSize)
+                           .setMIFlag(MachineInstr::FrameSetup);
+    MI->getOperand(3).setIsDead(); // The EFLAGS implicit def is dead.
+
+
+    addRegOffset(BuildMI(MBB, MBBI, DL, TII.get(MovMIOpc))
+                     .setMIFlag(MachineInstr::FrameSetup),
+                 StackPtr, false, 0)
+        .addImm(0)
+        .setMIFlag(MachineInstr::FrameSetup);
+    NumFrameExtraProbe++;
+    CurrentOffset += StackProbeSize;
+    CurrentProbeOffset += StackProbeSize;
+  }
+
+  uint64_t ChunkSize = Offset - CurrentOffset;
+  MachineInstr *MI = BuildMI(MBB, MBBI, DL, TII.get(Opc), StackPtr)
+                         .addReg(StackPtr)
+                         .addImm(ChunkSize)
+                         .setMIFlag(MachineInstr::FrameSetup);
+  MI->getOperand(3).setIsDead(); // The EFLAGS implicit def is dead.
+}
+
+void X86FrameLowering::emitStackProbeInlineGenericLoop(
+    MachineFunction &MF, MachineBasicBlock &MBB,
+    MachineBasicBlock::iterator MBBI, const DebugLoc &DL,
+    uint64_t Offset) const {
+
+  const X86Subtarget &STI = MF.getSubtarget<X86Subtarget>();
+  const X86TargetLowering &TLI = *STI.getTargetLowering();
+  const unsigned MovMIOpc = Is64Bit ? X86::MOV64mi32 : X86::MOV32mi;
+  const uint64_t StackProbeSize = TLI.getStackProbeSize(MF);
+
+  // Synthesize a loop
+  NumFrameLoopProbe++;
+  const BasicBlock *LLVM_BB = MBB.getBasicBlock();
+
+  MachineBasicBlock *testMBB = MF.CreateMachineBasicBlock(LLVM_BB);
+  MachineBasicBlock *tailMBB = MF.CreateMachineBasicBlock(LLVM_BB);
+
+  MachineFunction::iterator MBBIter = ++MBB.getIterator();
+  MF.insert(MBBIter, testMBB);
+  MF.insert(MBBIter, tailMBB);
+
+  unsigned FinalStackPtr = Uses64BitFramePtr ? X86::R11 : X86::R11D;
+  BuildMI(MBB, MBBI, DL, TII.get(X86::MOV64rr), FinalStackPtr)
+      .addReg(StackPtr)
+      .setMIFlag(MachineInstr::FrameSetup);
+
+  // save loop bound
+  {
+    const unsigned Opc = getSUBriOpcode(Uses64BitFramePtr, Offset);
+    BuildMI(MBB, MBBI, DL, TII.get(Opc), FinalStackPtr)
+        .addReg(FinalStackPtr)
+        .addImm(Offset / StackProbeSize * StackProbeSize)
+        .setMIFlag(MachineInstr::FrameSetup);
+  }
+
+  // allocate a page
+  {
+    const unsigned Opc = getSUBriOpcode(Uses64BitFramePtr, StackProbeSize);
+    BuildMI(testMBB, DL, TII.get(Opc), StackPtr)
+        .addReg(StackPtr)
+        .addImm(StackProbeSize)
+        .setMIFlag(MachineInstr::FrameSetup);
+  }
+
+  // touch the page
+  addRegOffset(BuildMI(testMBB, DL, TII.get(MovMIOpc))
+                   .setMIFlag(MachineInstr::FrameSetup),
+               StackPtr, false, 0)
+      .addImm(0)
+      .setMIFlag(MachineInstr::FrameSetup);
+
+  // cmp with stack pointer bound
+  BuildMI(testMBB, DL, TII.get(IsLP64 ? X86::CMP64rr : X86::CMP32rr))
+      .addReg(StackPtr)
+      .addReg(FinalStackPtr)
+      .setMIFlag(MachineInstr::FrameSetup);
+
+  // jump
+  BuildMI(testMBB, DL, TII.get(X86::JCC_1))
+      .addMBB(testMBB)
+      .addImm(X86::COND_NE)
+      .setMIFlag(MachineInstr::FrameSetup);
+  testMBB->addSuccessor(testMBB);
+  testMBB->addSuccessor(tailMBB);
+
+  // allocate a block and touch it
+
+  tailMBB->splice(tailMBB->end(), &MBB, MBBI, MBB.end());
+  tailMBB->transferSuccessorsAndUpdatePHIs(&MBB);
+  MBB.addSuccessor(testMBB);
+
+  if (Offset % StackProbeSize) {
+    const unsigned Opc = getSUBriOpcode(Uses64BitFramePtr, Offset);
+    BuildMI(*tailMBB, tailMBB->begin(), DL, TII.get(Opc), StackPtr)
+        .addReg(StackPtr)
+        .addImm(Offset % StackProbeSize)
+        .setMIFlag(MachineInstr::FrameSetup);
+  }
+}
+
+void X86FrameLowering::emitStackProbeInlineWindowsCoreCLR64(
+    MachineFunction &MF, MachineBasicBlock &MBB,
+    MachineBasicBlock::iterator MBBI, const DebugLoc &DL, bool InProlog) const {
+  const X86Subtarget &STI = MF.getSubtarget<X86Subtarget>();
   assert(STI.is64Bit() && "different expansion needed for 32 bit");
   assert(STI.isTargetWindowsCoreCLR() && "custom expansion expects CoreCLR");
   const TargetInstrInfo &TII = *STI.getInstrInfo();
   const BasicBlock *LLVM_BB = MBB.getBasicBlock();
 
   // RAX contains the number of bytes of desired stack adjustment.
   // The handling here assumes this value has already been updated so as to
   // maintain stack alignment.
void X86FrameLowering::emitStackProbeCall(MachineFunction &MF,
 
   if (InProlog) {
     // Apply the frame setup flag to all inserted instrs.
     for (++ExpansionMBBI; ExpansionMBBI != MBBI; ++ExpansionMBBI)
       ExpansionMBBI->setFlag(MachineInstr::FrameSetup);
   }
 }
 
-void X86FrameLowering::emitStackProbeInlineStub(
+MachineInstr *X86FrameLowering::emitStackProbeInlineStub(
     MachineFunction &MF, MachineBasicBlock &MBB,
     MachineBasicBlock::iterator MBBI, const DebugLoc &DL, bool InProlog) const {
 
   assert(InProlog && "ChkStkStub called outside prolog!");
 
-  BuildMI(MBB, MBBI, DL, TII.get(X86::CALLpcrel32))
+  return BuildMI(MBB, MBBI, DL, TII.get(X86::CALLpcrel32))
       .addExternalSymbol("__chkstk_stub");
 }
 
 static unsigned calculateSetFPREG(uint64_t SPAdjust) {
   // Win64 ABI has a less restrictive limitation of 240; 128 works equally well
   // and might require smaller successive adjustments.
   const uint64_t Win64MaxSEHOffset = 128;
   uint64_t SEHFrameOffset = std::min(SPAdjust, Win64MaxSEHOffset);
void X86FrameLowering::emitPrologue(MachineFunction &MF,
   int TailCallReturnAddrDelta = X86FI->getTCReturnAddrDelta();
   if (TailCallReturnAddrDelta && IsWin64Prologue)
     report_fatal_error("Can't handle guaranteed tail call under win64 yet");
 
   if (TailCallReturnAddrDelta < 0)
     X86FI->setCalleeSavedFrameSize(
       X86FI->getCalleeSavedFrameSize() - TailCallReturnAddrDelta);
 
-  bool UseStackProbe = !STI.getTargetLowering()->getStackProbeSymbolName(MF).empty();
+  const bool EmitStackProbeCall =
+      STI.getTargetLowering()->hasStackProbeSymbol(MF);
   unsigned StackProbeSize = STI.getTargetLowering()->getStackProbeSize(MF);
 
   // Re-align the stack on 64-bit if the x86-interrupt calling convention is
   // used and an error code was pushed, since the x86-64 ABI requires a 16-byte
   // stack alignment.
   if (Fn.getCallingConv() == CallingConv::X86_INTR && Is64Bit &&
       Fn.arg_size() == 2) {
     StackSize += 8;
     MFI.setStackSize(StackSize);
     emitSPUpdate(MBB, MBBI, DL, -8, /*InEpilogue=*/false);
   }
 
   // If this is x86-64 and the Red Zone is not disabled, if we are a leaf
   // function, and use up to 128 bytes of stack space, don't have a frame
   // pointer, calls, or dynamic alloca then we do not need to adjust the
   // stack pointer (we fit in the Red Zone). We also check that we don't
   // push and pop from the stack.
-  if (has128ByteRedZone(MF) &&
-      !TRI->needsStackRealignment(MF) &&
+  if (has128ByteRedZone(MF) && !TRI->needsStackRealignment(MF) &&
       !MFI.hasVarSizedObjects() &&             // No dynamic alloca.
       !MFI.adjustsStack() &&                   // No calls.
-      !UseStackProbe &&                        // No stack probes.
+      !EmitStackProbeCall &&                   // No stack probes.
       !MFI.hasCopyImplyingStackAdjustment() && // Don't push and pop.
       !MF.shouldSplitStack()) {                // Regular stack
     uint64_t MinSize = X86FI->getCalleeSavedFrameSize();
     if (HasFP) MinSize += SlotSize;
     X86FI->setUsesRedZone(MinSize > 0 || StackSize > 0);
     StackSize = std::max(MinSize, StackSize > 128 ? StackSize - 128 : 0);
     MFI.setStackSize(StackSize);
   }
void X86FrameLowering::emitPrologue(MachineFunction &MF,
   // stack and adjust the stack pointer in one go.  The 64-bit version of
   // __chkstk is only responsible for probing the stack.  The 64-bit prologue is
   // responsible for adjusting the stack pointer.  Touching the stack at 4K
   // increments is necessary to ensure that the guard pages used by the OS
   // virtual memory manager are allocated in correct sequence.
   uint64_t AlignedNumBytes = NumBytes;
   if (IsWin64Prologue && !IsFunclet && TRI->needsStackRealignment(MF))
     AlignedNumBytes = alignTo(AlignedNumBytes, MaxAlign);
-  if (AlignedNumBytes >= StackProbeSize && UseStackProbe) {
+  if (AlignedNumBytes >= StackProbeSize && EmitStackProbeCall) {
     assert(!X86FI->getUsesRedZone() &&
            "The Red Zone is not accounted for in stack probes");
 
     // Check whether EAX is livein for this block.
     bool isEAXAlive = isEAXLiveIn(MBB);
 
     if (isEAXAlive) {
       if (Is64Bit) {

llvm/lib/Target/X86/X86ISelLowering.h

enum NodeType : unsigned {
       // Windows's _chkstk call to do stack probing.
       WIN_ALLOCA,
 
       // For allocating variable amounts of stack space when using
       // segmented stacks. Check if the current stacklet has enough space, and
       // falls back to heap allocation if not.
       SEG_ALLOCA,
 
+      // For allocating stack space when using stack clash protector.
+      // Allocation is performed by block, and each block is probed.
+      PROBED_ALLOCA,
+
       // Memory barriers.
       MEMBARRIER,
       MFENCE,
 
       // Get a random integer and indicate whether it is valid in CF.
       RDRAND,
 
       // Get a NIST SP800-90B & C compliant random integer and
public:
     unsigned getVectorTypeBreakdownForCallingConv(
         LLVMContext &Context, CallingConv::ID CC, EVT VT, EVT &IntermediateVT,
         unsigned &NumIntermediates, MVT &RegisterVT) const override;
 
     bool isIntDivCheap(EVT VT, AttributeList Attr) const override;
 
     bool supportSwiftError() const override;
 
+    bool hasStackProbeSymbol(MachineFunction &MF) const override;
+    bool hasInlineStackProbe(MachineFunction &MF) const override;
     StringRef getStackProbeSymbolName(MachineFunction &MF) const override;
 
     unsigned getStackProbeSize(MachineFunction &MF) const;
 
     bool hasVectorBlend() const override { return true; }
 
     unsigned getMaxSupportedInterleaveFactor() const override { return 4; }
 
MachineBasicBlock *EmitLoweredAtomicFP(MachineInstr &I,
                                            MachineBasicBlock *BB) const;
 
     MachineBasicBlock *EmitLoweredCatchRet(MachineInstr &MI,
                                            MachineBasicBlock *BB) const;
 
     MachineBasicBlock *EmitLoweredSegAlloca(MachineInstr &MI,
                                             MachineBasicBlock *BB) const;
 
+    MachineBasicBlock *EmitLoweredProbedAlloca(MachineInstr &MI,
+                                               MachineBasicBlock *BB) const;
+
     MachineBasicBlock *EmitLoweredTLSAddr(MachineInstr &MI,
                                           MachineBasicBlock *BB) const;
 
     MachineBasicBlock *EmitLoweredTLSCall(MachineInstr &MI,
                                           MachineBasicBlock *BB) const;
 
     MachineBasicBlock *EmitLoweredRetpoline(MachineInstr &MI,
                                             MachineBasicBlock *BB) const;

llvm/lib/Target/X86/X86ISelLowering.cpp

 // bytes in one go. Touching the stack at 4K increments is necessary to ensure
 // that the guard pages used by the OS virtual memory manager are allocated in
 // correct sequence.
 SDValue
 X86TargetLowering::LowerDYNAMIC_STACKALLOC(SDValue Op,
                                            SelectionDAG &DAG) const {
   MachineFunction &MF = DAG.getMachineFunction();
   bool SplitStack = MF.shouldSplitStack();
-  bool EmitStackProbe = !getStackProbeSymbolName(MF).empty();
+  bool EmitStackProbeCall = hasStackProbeSymbol(MF);
   bool Lower = (Subtarget.isOSWindows() && !Subtarget.isTargetMachO()) ||
-               SplitStack || EmitStackProbe;
+               SplitStack || EmitStackProbeCall;
   SDLoc dl(Op);
 
   // Get the inputs.
   SDNode *Node = Op.getNode();
   SDValue Chain = Op.getOperand(0);
   SDValue Size  = Op.getOperand(1);
   MaybeAlign Alignment(Op.getConstantOperandVal(2));
   EVT VT = Node->getValueType(0);
 
   // Chain the dynamic stack allocation so that it doesn't modify the stack
   // pointer when other instructions are using the stack.
   Chain = DAG.getCALLSEQ_START(Chain, 0, 0, dl);
 
   bool Is64Bit = Subtarget.is64Bit();
   MVT SPTy = getPointerTy(DAG.getDataLayout());
 
   SDValue Result;
   if (!Lower) {
     const TargetLowering &TLI = DAG.getTargetLoweringInfo();
     unsigned SPReg = TLI.getStackPointerRegisterToSaveRestore();
     assert(SPReg && "Target cannot require DYNAMIC_STACKALLOC expansion and"
                     " not tell us which reg is the stack pointer!");
 
-    SDValue SP = DAG.getCopyFromReg(Chain, dl, SPReg, VT);
-    Chain = SP.getValue(1);
     const TargetFrameLowering &TFI = *Subtarget.getFrameLowering();
     const Align StackAlign(TFI.getStackAlignment());
+    if (hasInlineStackProbe(MF)) {
+      MachineRegisterInfo &MRI = MF.getRegInfo();
+
+      const TargetRegisterClass *AddrRegClass = getRegClassFor(SPTy);
+      Register Vreg = MRI.createVirtualRegister(AddrRegClass);
+      Chain = DAG.getCopyToReg(Chain, dl, Vreg, Size);
+      Result = DAG.getNode(X86ISD::PROBED_ALLOCA, dl, SPTy, Chain,
+                           DAG.getRegister(Vreg, SPTy));
+    } else {
+      SDValue SP = DAG.getCopyFromReg(Chain, dl, SPReg, VT);
+      Chain = SP.getValue(1);
       Result = DAG.getNode(ISD::SUB, dl, VT, SP, Size); // Value
+    }
     if (Alignment && Alignment > StackAlign)
       Result =
           DAG.getNode(ISD::AND, dl, VT, Result,
                       DAG.getConstant(~(Alignment->value() - 1ULL), dl, VT));
     Chain = DAG.getCopyToReg(Chain, dl, SPReg, Result); // Output chain
   } else if (SplitStack) {
     MachineRegisterInfo &MRI = MF.getRegInfo();
 
#define NODE_NAME_CASE(NODE) case X86ISD::NODE: return "X86ISD::" #NODE;
   NODE_NAME_CASE(PSADBW)
   NODE_NAME_CASE(DBPSADBW)
   NODE_NAME_CASE(VASTART_SAVE_XMM_REGS)
   NODE_NAME_CASE(VAARG_64)
   NODE_NAME_CASE(WIN_ALLOCA)
   NODE_NAME_CASE(MEMBARRIER)
   NODE_NAME_CASE(MFENCE)
   NODE_NAME_CASE(SEG_ALLOCA)
+  NODE_NAME_CASE(PROBED_ALLOCA)
   NODE_NAME_CASE(RDRAND)
   NODE_NAME_CASE(RDSEED)
   NODE_NAME_CASE(RDPKRU)
   NODE_NAME_CASE(WRPKRU)
   NODE_NAME_CASE(VPMADDUBSW)
   NODE_NAME_CASE(VPMADDWD)
   NODE_NAME_CASE(VPSHA)
   NODE_NAME_CASE(VPSHL)
X86TargetLowering::EmitLoweredSelect(MachineInstr &MI,
 
   // Now remove the CMOV(s).
   ThisMBB->erase(MIItBegin, MIItEnd);
 
   return SinkMBB;
 }
 
 MachineBasicBlock *
+X86TargetLowering::EmitLoweredProbedAlloca(MachineInstr &MI,
+                                           MachineBasicBlock *BB) const {
+  MachineFunction *MF = BB->getParent();
+  const TargetInstrInfo *TII = Subtarget.getInstrInfo();
+  const X86FrameLowering &TFI = *Subtarget.getFrameLowering();
+  DebugLoc DL = MI.getDebugLoc();
+  const BasicBlock *LLVM_BB = BB->getBasicBlock();
+
+  const unsigned ProbeSize = getStackProbeSize(*MF);
+
+  MachineRegisterInfo &MRI = MF->getRegInfo();
+  MachineBasicBlock *testMBB = MF->CreateMachineBasicBlock(LLVM_BB);
+  MachineBasicBlock *tailMBB = MF->CreateMachineBasicBlock(LLVM_BB);
+  MachineBasicBlock *blockMBB = MF->CreateMachineBasicBlock(LLVM_BB);
+
+  MachineFunction::iterator MBBIter = ++BB->getIterator();
+  MF->insert(MBBIter, testMBB);
+  MF->insert(MBBIter, blockMBB);
+  MF->insert(MBBIter, tailMBB);
+
+  unsigned sizeVReg = MI.getOperand(1).getReg();
+
+  const TargetRegisterClass *SizeRegClass = MRI.getRegClass(sizeVReg);
+
+  unsigned tmpSizeVReg = MRI.createVirtualRegister(SizeRegClass);
+  unsigned tmpSizeVReg2 = MRI.createVirtualRegister(SizeRegClass);
+
+  unsigned physSPReg = TFI.Uses64BitFramePtr ? X86::RSP : X86::ESP;
+
+  // test rsp size
+  BuildMI(testMBB, DL, TII->get(X86::PHI), tmpSizeVReg)
+      .addReg(sizeVReg)
+      .addMBB(BB)
+      .addReg(tmpSizeVReg2)
+      .addMBB(blockMBB);
+
+  BuildMI(testMBB, DL,
+          TII->get(TFI.Uses64BitFramePtr ? X86::CMP64ri32 : X86::CMP32ri))
+      .addReg(tmpSizeVReg)
+      .addImm(ProbeSize);
+
+  BuildMI(testMBB, DL, TII->get(X86::JCC_1))
+      .addMBB(tailMBB)
+      .addImm(X86::COND_L);
+  testMBB->addSuccessor(blockMBB);
+  testMBB->addSuccessor(tailMBB);
+
+  // allocate a block and touch it
+
+  BuildMI(blockMBB, DL,
+          TII->get(TFI.Uses64BitFramePtr ? X86::SUB64ri32 : X86::SUB32ri),
+          tmpSizeVReg2)
+      .addReg(tmpSizeVReg)
+      .addImm(ProbeSize);
+
+  BuildMI(blockMBB, DL,
+          TII->get(TFI.Uses64BitFramePtr ? X86::SUB64ri32 : X86::SUB32ri),
+          physSPReg)
+      .addReg(physSPReg)
+      .addImm(ProbeSize);
+
+  const unsigned MovMIOpc =
+      TFI.Uses64BitFramePtr ? X86::MOV64mi32 : X86::MOV32mi;
+  addRegOffset(BuildMI(blockMBB, DL, TII->get(MovMIOpc)), physSPReg, false, 0)
+      .addImm(0);
+
+  BuildMI(blockMBB, DL, TII->get(X86::JMP_1)).addMBB(testMBB);
+  blockMBB->addSuccessor(testMBB);
+
+  // allocate the tail and continue
+  BuildMI(tailMBB, DL,
+          TII->get(TFI.Uses64BitFramePtr ? X86::SUB64rr : X86::SUB32rr),
+          physSPReg)
+      .addReg(physSPReg)
+      .addReg(tmpSizeVReg);
+  BuildMI(tailMBB, DL, TII->get(TargetOpcode::COPY), MI.getOperand(0).getReg())
+      .addReg(physSPReg);
+
+  tailMBB->splice(tailMBB->end(), BB,
+                  std::next(MachineBasicBlock::iterator(MI)), BB->end());
+  tailMBB->transferSuccessorsAndUpdatePHIs(BB);
+  BB->addSuccessor(testMBB);
+
+  // Delete the original pseudo instruction.
+  MI.eraseFromParent();
+
+  // And we're done.
+  return tailMBB;
+}
+
+MachineBasicBlock *
 X86TargetLowering::EmitLoweredSegAlloca(MachineInstr &MI,
                                         MachineBasicBlock *BB) const {
   MachineFunction *MF = BB->getParent();
   const TargetInstrInfo *TII = Subtarget.getInstrInfo();
   DebugLoc DL = MI.getDebugLoc();
   const BasicBlock *LLVM_BB = BB->getBasicBlock();
 
   assert(MF->shouldSplitStack());
X86TargetLowering::EmitInstrWithCustomInserter(MachineInstr &MI,
   case X86::RETPOLINE_TCRETURN32:
   case X86::RETPOLINE_TCRETURN64:
     return EmitLoweredRetpoline(MI, BB);
   case X86::CATCHRET:
     return EmitLoweredCatchRet(MI, BB);
   case X86::SEG_ALLOCA_32:
   case X86::SEG_ALLOCA_64:
     return EmitLoweredSegAlloca(MI, BB);
+  case X86::PROBED_ALLOCA_32:
+  case X86::PROBED_ALLOCA_64:
+    return EmitLoweredProbedAlloca(MI, BB);
   case X86::TLSCall_32:
   case X86::TLSCall_64:
     return EmitLoweredTLSCall(MI, BB);
   case X86::CMOV_FR32:
   case X86::CMOV_FR32X:
   case X86::CMOV_FR64:
   case X86::CMOV_FR64X:
   case X86::CMOV_GR8:
for (auto *Exit : Exits)
           .addReg(NewVR);
   }
 }
 
 bool X86TargetLowering::supportSwiftError() const {
   return Subtarget.is64Bit();
 }
 
+/// Returns true if stack probing through a function call is requested.
+bool X86TargetLowering::hasStackProbeSymbol(MachineFunction &MF) const {
+  return !getStackProbeSymbolName(MF).empty();
+}
+
+/// Returns true if stack probing through inline assembly is requested.
+bool X86TargetLowering::hasInlineStackProbe(MachineFunction &MF) const {
+
+  // No inline stack probe for Windows, they have their own mechanism.
+  if (Subtarget.isOSWindows() ||
+      MF.getFunction().hasFnAttribute("no-stack-arg-probe"))
+    return false;
+
+  // If the function specifically requests inline stack probes, emit them.
+  if (MF.getFunction().hasFnAttribute("probe-stack"))
+    return MF.getFunction().getFnAttribute("probe-stack").getValueAsString() ==
+           "inline-asm";
+
+  return false;
+}
+
 /// Returns the name of the symbol used to emit stack probes or the empty
 /// string if not applicable.
 StringRef
 X86TargetLowering::getStackProbeSymbolName(MachineFunction &MF) const {
+  // Inline Stack probes disable stack probe call
+  if (hasInlineStackProbe(MF))
+    return "";
+
   // If the function specifically requests stack probes, emit them.
   if (MF.getFunction().hasFnAttribute("probe-stack"))
     return MF.getFunction().getFnAttribute("probe-stack").getValueAsString();
 
   // Generally, if we aren't on Windows, the platform ABI does not include
   // support for stack probes, so don't emit them.
   if (!Subtarget.isOSWindows() || Subtarget.isTargetMachO() ||
       MF.getFunction().hasFnAttribute("no-stack-arg-probe"))

llvm/lib/Target/X86/X86InstrCompiler.td

def SEG_ALLOCA_32 : I<0, Pseudo, (outs GR32:$dst), (ins GR32:$size),
                     Requires<[NotLP64]>;
 
 let Defs = [RAX, RSP, EFLAGS], Uses = [RSP] in
 def SEG_ALLOCA_64 : I<0, Pseudo, (outs GR64:$dst), (ins GR64:$size),
                       "# variable sized alloca for segmented stacks",
                       [(set GR64:$dst,
                          (X86SegAlloca GR64:$size))]>,
                     Requires<[In64BitMode]>;
+
+// To protect against stack clash, dynamic allocation should perform a memory
+// probe at each page.
+
+let Defs = [EAX, ESP, EFLAGS], Uses = [ESP] in
+def PROBED_ALLOCA_32 : I<0, Pseudo, (outs GR32:$dst), (ins GR32:$size),
+                      "# variable sized alloca with probing",
+                      [(set GR32:$dst,
+                         (X86ProbedAlloca GR32:$size))]>,
+                    Requires<[NotLP64]>;
+
+let Defs = [RAX, RSP, EFLAGS], Uses = [RSP] in
+def PROBED_ALLOCA_64 : I<0, Pseudo, (outs GR64:$dst), (ins GR64:$size),
+                      "# variable sized alloca with probing",
+                      [(set GR64:$dst,
+                         (X86ProbedAlloca GR64:$size))]>,
+                    Requires<[In64BitMode]>;
 }
 
 // Dynamic stack allocation yields a _chkstk or _alloca call for all Windows
 // targets.  These calls are needed to probe the stack when allocating more than
 // 4k bytes in one go. Touching the stack at 4K increments is necessary to
 // ensure that the guard pages used by the OS virtual memory manager are
 // allocated in correct sequence.
 // The main point of having separate instruction are extra unmodelled effects

llvm/lib/Target/X86/X86InstrInfo.td

 def SDT_X86TLSBASEADDR : SDTypeProfile<0, 1, [SDTCisInt<0>]>;
 
 def SDT_X86TLSCALL : SDTypeProfile<0, 1, [SDTCisInt<0>]>;
 
 def SDT_X86WIN_ALLOCA : SDTypeProfile<0, 1, [SDTCisVT<0, iPTR>]>;
 
 def SDT_X86SEG_ALLOCA : SDTypeProfile<1, 1, [SDTCisVT<0, iPTR>, SDTCisVT<1, iPTR>]>;
 
+def SDT_X86PROBED_ALLOCA : SDTypeProfile<1, 1, [SDTCisVT<0, iPTR>, SDTCisVT<1, iPTR>]>;
+
 def SDT_X86EHRET : SDTypeProfile<0, 1, [SDTCisInt<0>]>;
 
 def SDT_X86TCRET : SDTypeProfile<0, 2, [SDTCisPtrTy<0>, SDTCisVT<1, i32>]>;
 
 def SDT_X86MEMBARRIER : SDTypeProfile<0, 0, []>;
 
 def SDT_X86ENQCMD : SDTypeProfile<1, 2, [SDTCisVT<0, i32>,
                                          SDTCisPtrTy<1>, SDTCisSameAs<1, 2>]>;

 def X86mul_imm : SDNode<"X86ISD::MUL_IMM", SDTIntBinOp>;
 
 def X86WinAlloca : SDNode<"X86ISD::WIN_ALLOCA", SDT_X86WIN_ALLOCA,
                           [SDNPHasChain, SDNPOutGlue]>;
 
 def X86SegAlloca : SDNode<"X86ISD::SEG_ALLOCA", SDT_X86SEG_ALLOCA,
                           [SDNPHasChain]>;
 
+def X86ProbedAlloca : SDNode<"X86ISD::PROBED_ALLOCA", SDT_X86PROBED_ALLOCA,
+                          [SDNPHasChain]>;
+
 def X86TLSCall : SDNode<"X86ISD::TLSCALL", SDT_X86TLSCALL,
                         [SDNPHasChain, SDNPOptInGlue, SDNPOutGlue]>;
 
 def X86lwpins : SDNode<"X86ISD::LWPINS",
                        SDTypeProfile<1, 3, [SDTCisVT<0, i32>, SDTCisInt<1>,
                                             SDTCisVT<2, i32>, SDTCisVT<3, i32>]>,
                        [SDNPHasChain, SDNPMayStore, SDNPMayLoad, SDNPSideEffect]>;
 

llvm/test/CodeGen/X86/stack-clash-dynamic-alloca.ll

+; RUN: llc < %s | FileCheck %s
+
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @foo(i32 %n) local_unnamed_addr #0 {
+
+; CHECK-LABEL: foo:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:   pushq %rbp
+; CHECK-NEXT:   .cfi_def_cfa_offset 16
+; CHECK-NEXT:   .cfi_offset %rbp, -16
+; CHECK-NEXT:   movq  %rsp, %rbp
+; CHECK-NEXT:   .cfi_def_cfa_register %rbp
+; CHECK-NEXT:   movl  %edi, %eax
+; CHECK-NEXT:   leaq  15(,%rax,4), %rax
+; CHECK-NEXT:   andq  $-16, %rax
+; CHECK-NEXT:   cmpq  $4096, %rax # imm = 0x1000
+; CHECK-NEXT:   jl  .LBB0_3
+; CHECK-NEXT:  .LBB0_2: # =>This Inner Loop Header: Depth=1
+; CHECK-NEXT:   subq  $4096, %rax # imm = 0x1000
+; CHECK-NEXT:   subq  $4096, %rsp # imm = 0x1000
+; CHECK-NEXT:   movq  $0, (%rsp)
+; CHECK-NEXT:   cmpq  $4096, %rax # imm = 0x1000
+; CHECK-NEXT:   jge .LBB0_2
+; CHECK-NEXT:  .LBB0_3:
+; CHECK-NEXT:   subq  %rax, %rsp
+; CHECK-NEXT:   movq  %rsp, %rax
+; CHECK-NEXT:   movl  $1, 4792(%rax)
+; CHECK-NEXT:   movl  (%rax), %eax
+; CHECK-NEXT:   movq  %rbp, %rsp
+; CHECK-NEXT:   popq  %rbp
+; CHECK-NEXT:  .cfi_def_cfa %rsp, 8
+; CHECK-NEXT:   retq
+
+  %a = alloca i32, i32 %n, align 16
+  %b = getelementptr inbounds i32, i32* %a, i64 1198
+  store volatile i32 1, i32* %b
+  %c = load volatile i32, i32* %a
+  ret i32 %c
+}
+
+attributes #0 =  {"probe-stack"="inline-asm"}

llvm/test/CodeGen/X86/stack-clash-large.ll

+; RUN: llc < %s | FileCheck %s
+
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @foo() local_unnamed_addr #0 {
+
+; CHECK-LABEL: foo:
+; CHECK:        # %bb.0:
+; CHECK-NEXT: movq  %rsp, %r11
+; CHECK-NEXT: subq  $69632, %r11 # imm = 0x11000
+; CHECK-NEXT:   .LBB0_1:
+; CHECK-NEXT: subq  $4096, %rsp # imm = 0x1000
+; CHECK-NEXT: movq  $0, (%rsp)
+; CHECK-NEXT: cmpq  %r11, %rsp
+; CHECK-NEXT: jne .LBB0_1
+; CHECK-NEXT:# %bb.2:
+; CHECK-NEXT: subq  $2248, %rsp # imm = 0x8C8
+; CHECK-NEXT: .cfi_def_cfa_offset 71888
+; CHECK-NEXT: movl  $1, 264(%rsp)
+; CHECK-NEXT: movl  $1, 28664(%rsp)
+; CHECK-NEXT: movl  -128(%rsp), %eax
+; CHECK-NEXT: addq  $71880, %rsp # imm = 0x118C8
+; CHECK-NEXT: .cfi_def_cfa_offset 8
+; CHECK-NEXT: retq
+
+
+  %a = alloca i32, i64 18000, align 16
+  %b0 = getelementptr inbounds i32, i32* %a, i64 98
+  %b1 = getelementptr inbounds i32, i32* %a, i64 7198
+  store volatile i32 1, i32* %b0
+  store volatile i32 1, i32* %b1
+  %c = load volatile i32, i32* %a
+  ret i32 %c
+}
+
+attributes #0 =  {"probe-stack"="inline-asm"}

llvm/test/CodeGen/X86/stack-clash-medium-natural-probes-mutliple-objects.ll

+; RUN: llc < %s | FileCheck %s
+
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @foo() local_unnamed_addr #0 {
+
+; CHECK-LABEL: foo:
+; CHECK:         # %bb.0:
+; CHECK-NEXT  subq  $4096, %rsp # imm = 0x1000
+; CHECK-NEXT  .cfi_def_cfa_offset 5888
+; CHECK-NEXT  movl  $1, 2088(%rsp)
+; CHECK-NEXT  subq  $1784, %rsp # imm = 0x6F8
+; CHECK-NEXT  movl  $2, 672(%rsp)
+; CHECK-NEXT  movl  1872(%rsp), %eax
+; CHECK-NEXT  addq  $5880, %rsp # imm = 0x16F8
+; CHECK-NEXT  .cfi_def_cfa_offset 8
+; CHECK-NEXT  retq
+
+
+  %a = alloca i32, i64 1000, align 16
+  %b = alloca i32, i64 500, align 16
+  %a0 = getelementptr inbounds i32, i32* %a, i64 500
+  %b0 = getelementptr inbounds i32, i32* %b, i64 200
+  store volatile i32 1, i32* %a0
+  store volatile i32 2, i32* %b0
+  %c = load volatile i32, i32* %a
+  ret i32 %c
+}
+
+attributes #0 =  {"probe-stack"="inline-asm"}

llvm/test/CodeGen/X86/stack-clash-medium-natural-probes.ll

+; RUN: llc < %s | FileCheck %s
+
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @foo() local_unnamed_addr #0 {
+
+; CHECK-LABEL: foo:
+; CHECK:         # %bb.0:
+; CHECK-NEXT: subq  $4096, %rsp             # imm = 0x1000
+; CHECK-NEXT: movq  $0, (%rsp)
+; CHECK-NEXT: subq  $3784, %rsp             # imm = 0xEC8
+; CHECK-NEXT: .cfi_def_cfa_offset 7888
+; CHECK-NEXT: movl  $1, 264(%rsp)
+; CHECK-NEXT: movl  $1, 4664(%rsp)
+; CHECK-NEXT: movl  -128(%rsp), %eax
+; CHECK-NEXT: addq  $7880, %rsp             # imm = 0x1EC8
+; CHECK-NEXT: .cfi_def_cfa_offset 8
+; CHECK-NEXT: retq
+
+
+
+  %a = alloca i32, i64 2000, align 16
+  %b0 = getelementptr inbounds i32, i32* %a, i64 98
+  %b1 = getelementptr inbounds i32, i32* %a, i64 1198
+  store i32 1, i32* %b0
+  store i32 1, i32* %b1
+  %c = load volatile i32, i32* %a
+  ret i32 %c
+}
+
+attributes #0 =  {"probe-stack"="inline-asm"}

llvm/test/CodeGen/X86/stack-clash-medium.ll

+; RUN: llc < %s | FileCheck %s
+
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @foo() local_unnamed_addr #0 {
+
+; CHECK-LABEL: foo:
+; CHECK:      # %bb.0:
+; CHECK-NEXT: subq  $4096, %rsp             # imm = 0x1000
+; CHECK-NEXT: movq  $0, (%rsp)
+; CHECK-NEXT: subq  $3784, %rsp             # imm = 0xEC8
+; CHECK-NEXT: .cfi_def_cfa_offset 7888
+; CHECK-NEXT: movl  $1, 672(%rsp)
+; CHECK-NEXT: movl  -128(%rsp), %eax
+; CHECK-NEXT: addq  $7880, %rsp             # imm = 0x1EC8
+; CHECK-NEXT: .cfi_def_cfa_offset 8
+; CHECK-NEXT: retq
+
+
+
+  %a = alloca i32, i64 2000, align 16
+  %b = getelementptr inbounds i32, i32* %a, i64 200
+  store volatile i32 1, i32* %b
+  %c = load volatile i32, i32* %a
+  ret i32 %c
+}
+
+attributes #0 =  {"probe-stack"="inline-asm"}

llvm/test/CodeGen/X86/stack-clash-no-free-probe.ll

+; RUN: llc < %s | FileCheck %s
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @foo(i64 %i) local_unnamed_addr #0 {
+; CHECK-LABEL: foo:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:  subq $4096, %rsp # imm = 0x1000
+; CHECK-NEXT:  movq $0, (%rsp)
+; CHECK-NEXT:  subq $3784, %rsp # imm = 0xEC8
+; CHECK-NEXT:  .cfi_def_cfa_offset 7888
+; CHECK-NEXT:  movl $1, -128(%rsp,%rdi,4)
+; CHECK-NEXT:  movl -128(%rsp), %eax
+; CHECK-NEXT:  addq $7880, %rsp # imm = 0x1EC8
+; CHECK-NEXT:  .cfi_def_cfa_offset 8
+; CHECK-NEXT:  retq
+
+  %a = alloca i32, i32 2000, align 16
+  %b = getelementptr inbounds i32, i32* %a, i64 %i
+  store volatile i32 1, i32* %b
+  %c = load volatile i32, i32* %a
+  ret i32 %c
+}
+
+attributes #0 =  {"probe-stack"="inline-asm"}
+

llvm/test/CodeGen/X86/stack-clash-small.ll

+; RUN: llc < %s | FileCheck %s
+
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define i32 @foo() local_unnamed_addr #0 {
+; CHECK-LABEL: foo:
+; CHECK:       # %bb.0:
+; CHECK-NEXT:  subq $280, %rsp # imm = 0x118
+; CHECK-NEXT:  .cfi_def_cfa_offset 288
+; CHECK-NEXT:  movl $1, 264(%rsp)
+; CHECK-NEXT:  movl -128(%rsp), %eax
+; CHECK-NEXT:  addq $280, %rsp # imm = 0x118
+; CHECK-NEXT:  .cfi_def_cfa_offset 8
+; CHECK-NEXT:  retq
+
+  %a = alloca i32, i64 100, align 16
+  %b = getelementptr inbounds i32, i32* %a, i64 98
+  store volatile i32 1, i32* %b
+  %c = load volatile i32, i32* %a
+  ret i32 %c
+}
+
+attributes #0 =  {"probe-stack"="inline-asm"}

llvm/test/CodeGen/X86/stack-clash-unknown-call.ll

+; RUN: llc < %s | FileCheck %s
+
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.memset.p0i8.i64(i8* nocapture writeonly, i8, i64, i1 immarg);
+
+define void @foo() local_unnamed_addr #0 {
+
+;CHECK-LABEL: foo:
+;CHECK:         # %bb.0:
+;CHECK-NEXT:  subq  $4096, %rsp # imm = 0x1000
+; it's important that we don't use the call as a probe here
+;CHECK-NEXT:  movq  $0, (%rsp)
+;CHECK-NEXT:  subq  $3912, %rsp # imm = 0xF48
+;CHECK-NEXT:  .cfi_def_cfa_offset 8016
+;CHECK-NEXT:  movq  %rsp, %rdi
+;CHECK-NEXT:  movl  $8000, %edx # imm = 0x1F40
+;CHECK-NEXT:  xorl  %esi, %esi
+;CHECK-NEXT:  callq memset
+;CHECK-NEXT:  addq  $8008, %rsp # imm = 0x1F48
+;CHECK-NEXT:  .cfi_def_cfa_offset 8
+;CHECK-NEXT:  retq
+
+  %a = alloca i8, i64 8000, align 16
+  call void @llvm.memset.p0i8.i64(i8* align 16 %a, i8 0, i64 8000, i1 false)
+  ret void
+}
+
+attributes #0 =  {"probe-stack"="inline-asm"}