For Linux/gnu compatibility, preinclude <stdc-predef.h> if the file is available

[mibintc]

Summary

As reported in llvm bugzilla 32377.
Here’s a patch to add preinclude of stdc-predef.h.

The gcc documentation says “On GNU/Linux, <stdc-predef.h> is pre-included.” See https://gcc.gnu.org/gcc-4.8/porting_to.html;

The preinclude is inhibited with –ffreestanding.

Basically I fixed the failing test cases by adding –ffreestanding which inhibits this behavior.

I fixed all the failing tests, including some in extra/test, there's a separate patch for that which is linked here

[mibintc]

forgot to add the new lit test

[fedor.sergeev]

There is no /usr/include/stdc-predef.h on Solaris and I see no sense in adding this -include for anything except Linux
(as docs are rather clear that this functionality is Linux-only).

Checked gcc on Solaris11/12 both SPARC and X86 - none are doing stdc-predef.h inclusion
(checking with

gcc  - -dI </dev/null -E | grep stdc-predef

)

[fedor.sergeev]

lib/Driver/ToolChains/Gnu.h

219–220

To me this does not belong to GCC-Installation-Detector at all, as it does not have anything to do with detection.

As I see the only reason for adding it here is to check Version vs 4.8.0.
If thats true then adding getVersion() here and using it elsewhere seems a better solution.

[fedor.sergeev]

docs are rather clear that this functionality is Linux-only

Oh, well, gcc implementation is rather generic, it relies on gcc's config.gcc to detect glibc and then it adds a hook that unconditionally provides stdc-predef.h (TARGET_C_PREINCLUDE).
Currently config.gcc does that for the following pattern of triples:
*-*-linux* | frv-*-*linux* | *-*-kfreebsd*-gnu | *-*-knetbsd*-gnu | *-*-gnu* | *-*-kopensolaris*-gnu)

I'm not sure how far into that do you really want to dive.... :-/

[mibintc]

Thanks for your review, Fedor. I moved AddGnuIncludeArgs out of GCCInstallation, like you suggested, and put it along where the library includes are added. I also pulled out the change for MipsLinux since I don't have a way to verify that.

I still have the same 7 test failures when make check-all on Linux. 3 of the fails are from "clangd" unit tests. Not sure why they are failing but i scanned the tests a little and it talks about access only to whitelisted directories, so I assume they aren't working because the dir where std-predef.h lives is not white listed. Maybe I should be sure that "file exists(stdc-predef.h)", e.g. using the system include path, before adding the -include into the command line--what do you think?

[fedor.sergeev]

LGTM wrt your update to sources.
And sorry, I'm not that qualified to answer your question on failing tests.

Probing existence of this header would make a sense, yet you are including it w/o a full path, so how are you going to find it for this probe?

[mibintc]

In D34158#782467, @fedor.sergeev wrote:

LGTM wrt your update to sources.
And sorry, I'm not that qualified to answer your question on failing tests.

Probing existence of this header would make a sense, yet you are including it w/o a full path, so how are you going to find it for this probe?

Yes, I'd have to change this patch to iterate through the system headers. Once I found an instance I'd keep the full path name and -include it using full path name. I'll go ahead and change the patch to do this.

The other thought I had is: don't check for gcc version -- just add this functionality for all gcc versions on Linux. The reason I'd do that is to avoid creating lit test failures when the lit tests are running with gcc versions less than gcc 4.8; thus far I've been able to fix the lit failures by adding "-nostdinc". Adding "-nostdinc" to the tests would work regardless of gcc version. But I'm concerned that I will need to update some of the lit tests by checking for different output. In that case the expected output would differ based on gcc version, and I don't know if it's possible to express "don't run this lit test for gcc< 4.8".

[mibintc]

I'm submitting another revision which checks for the existence of stdc-predef.h before inserting the -include option.

[mibintc]

I want to submit a modified patch

[mibintc]

Here's a modified revision which checks if stdc-predef.h exists before adding a -include option to that file. this is only for Linux using gcc 4.8 and higher. Several tests needed to be fixed because of this change - i fixed them by suppressing the preinclude using ffreestanding or nostdinc. Also i need to change about 10 clang/tools/extra/test/clang-tidy tests. Those test changes aren't in this patch. Please tell me: what's the correct procedure for getting those tests reviewed? Thanks Fedor for your reviews!
--Melanie

[mibintc]

@fedor.sergeev do you have time to review my (hopefully) final revision? Also can you recommend someone to review the extra test changes?

[fedor.sergeev]

can you recommend someone to review the extra test changes?

I'm rather new to the project and thus rather bad at finding reviewers.
Also generally I do not consider myself qualified enough to be the only reviewer, though in this particular case I believe I spent enough time on the toolchain code for my own Solaris work.
I feel less qualified for the tests section...

I will take a look at the final version tomorrow.

[Hahnfeld]

Some comments inline. In general you should consider posting an RFC on cfe-dev because this change will basically affect all compilations on GNU/Linux if the file is present.
Adding Richard (general maintainer) and Renato (ARM Linux) so they are aware.

include/clang/Driver/ToolChain.h

459–462

Why do you need to define this method in the generic ToolChain?

lib/Driver/ToolChains/Gnu.cpp

2342–2359

If this is GNU/Linux only, why not move to the Linux toolchain entirely?

lib/Driver/ToolChains/Gnu.h

331–333

This starts with a lower-case character and won't override the method in ToolChain. Adding the keyword override should reveal this mistake.

test/Driver/gcc-predef.c

2

s/gcc/clang/ ?

10

The driver will only include stdc-predef.h if it can be found in the system. With that, the current version of this test will only run on such Linux system. Maybe add a basic tree in test/Driver/Inputs and test that the corresponding header is included?

15–18

What's the difference to the test above?

[mibintc]

In D34158#792858, @fedor.sergeev wrote: >

I will take a look at the final version tomorrow.

Fedor, let me address the comments from Jonas (with another revision!) before you take a look.

[mibintc]

I'm planning to rework this patch again. sorry for the churn.

[mibintc]

The previous (n-1) version didn't work: I wanted to iterate through the system include directories to check for the existence of stdc-predef.h but clang is using a different kind of system include option for /usr and /usr/include so the iterator didn't find any system includes at all, i don't think there is an iterator which will iterate through these special kind of include directories. [Kind of embarrassing that I posted that patch which didn't work at all, sorry about that.]

This one behaves more like gcc anyway. gcc puts the -include option in there regardless, and to suppress the behavior you are to use the -ffreestanding option.

I responded to the remarks from Jonas Hahnfeld, making the patch only for Linux.

I'm having trouble with one of the existing tests. These are the ones that fail. I don't know how to fix the text. It's a google test and it is using a dummy file system with whitelisted directories. @ilya-biryukov Is there a way to pass in a command line argument? if so, I could fix it these failures by adding -ffreestanding option which suppresses the new behavior. That's how I fixed the many test failures which I encountered with this new behavior.

Extra Tools Unit Tests :: clangd/ClangdTests/ClangdVFSTest.Parse
Extra Tools Unit Tests :: clangd/ClangdTests/ClangdVFSTest.ParseWithHeader
Extra Tools Unit Tests :: clangd/ClangdTests/ClangdVFSTest.Reparse

The other test that fails is my own new test! It fails because I don't know how to set it up so the test thinks it has a gcc toolchain with version > 4.8. I tried using gcc-toolchain set to various other Linux toolchains that i see in the test/Driver/Inputs - none of them cause the gcc version to be in the range. I also tried using -ccc-installation=Inputs/ which I see being used for gcc version parsing. How can I set up the test so that the GCCInstallation has a Version >= 4.8? I test the new functionality from the console on Linux and can confirm it's working.

Failing Tests (4): -- the 3 listed above, as well as

Clang :: Driver/gcc-predef.c -- my new test, the 3rd run which confirms that the include is added

[ilya-biryukov]

Here's a patch to fix ClangdTests: https://reviews.llvm.org/D34936
We should probably land it before your patch to avoid ClangdTests failures between those llvm and clang-tools-extra revisions.

[Hahnfeld]

In D34158#797170, @mibintc wrote:

The other test that fails is my own new test! It fails because I don't know how to set it up so the test thinks it has a gcc toolchain with version > 4.8. I tried using gcc-toolchain set to various other Linux toolchains that i see in the test/Driver/Inputs - none of them cause the gcc version to be in the range. I also tried using -ccc-installation=Inputs/ which I see being used for gcc version parsing. How can I set up the test so that the GCCInstallation has a Version >= 4.8? I test the new functionality from the console on Linux and can confirm it's working.

You could try adding a completely new directory in Inputs/. Add the directory usr/lib/gcc/x86_64-unknown-linux/4.9.0 underneath which should be recognized as the version. Maybe you have to add some more files, I'm not really familiar with the detection mechanism...

lib/Driver/ToolChains/Gnu.cpp

2342–2353

I think this can still be moved to Linux...

test/Driver/gcc-predef.c

28

Maybe it's better to run clang with -### and checking that it passes -include to cc1?

Then you can have:

// CHECK-PREDEF: "-include" "stdc-predef.h"
// CHECK-NO-PREDEF-NOT: "-include" "stdc-predef.h"

and pass the corresponding prefixes to FileCheck. Could you also add a test that it the file is not included if it does not exist? This can be done with the current basic_linux_tree

[mibintc]

In D34158#797968, @Hahnfeld wrote:

In D34158#797170, @mibintc wrote:

The other test that fails is my own new test! It fails because I don't know how to set it up so the test thinks it has a gcc toolchain with version > 4.8. I tried using gcc-toolchain set to various other Linux toolchains that i see in the test/Driver/Inputs - none of them cause the gcc version to be in the range. I also tried using -ccc-installation=Inputs/ which I see being used for gcc version parsing. How can I set up the test so that the GCCInstallation has a Version >= 4.8? I test the new functionality from the console on Linux and can confirm it's working.

You could try adding a completely new directory in Inputs/. Add the directory usr/lib/gcc/x86_64-unknown-linux/4.9.0 underneath which should be recognized as the version. Maybe you have to add some more files, I'm not really familiar with the detection mechanism...

Thanks. I did that by copying the directory from fake_install_tree into a new install directory named stdc-predef. However, this modification didn't result in altering the macro values for gnu-major and gnu-minor. On the other hand, it is necessary to have a gcc installation in the --sysroot directory. The test didn't pass if the gcc installation wasn't present in the Inputs directory. I used the method you suggested and modified the test to use -### and check for the presence or absence of -include stdc-predef.h. Now these new tests are passing. Thank you very much for the suggestion.

[mibintc]

I responded to the review from Jonas Hahnfeld: I moved the entire change into ToolChains/Linux and removed the modifications to Gnu.h and Gnu.cpp; I modified the new tests to use -### with FileCheck, and added a tree in Inputs/ for the new tests. Jonas please see inline reply concerning whether to--or how to--check for existence of stdc-predef.h

Also, note that there are some tests from clang-tidy that need to be updated. I have a separate patch for that since it's in a different repo. I assume it would be good to get those modifications landed before this patch.

[mibintc]

Jonas asked about adding a new test to ensure that "-include stdc-predef.h" does not get added if the file doesn't exist.

I added a reply to that but I can't see where it went. So I'm writing the reply again.

The current version of the patch doesn't check for the existence of stdc-predef.h. As far as I understand, this is consistent with gcc behavior. It is expected that if you are on a system without stdc-predef.h then you can add -ffreestanding.

In a prior revision of this patch (see Diff 4), I attempted to iterate through the system directories to check for the existence of the file before adding -include with a full path name. However that patch didn't work. I was using this call to iterate through the system includes: getAllArgValues(options::OPT_isystem). However, the directory where stdc-predef.h is located, /usr/include, is added into the search path by using the option -internal-externc-isystem. getAllArgValues(options::OPT_isystem) does not iterate through the include directories which were added via -internal-externc-isystem. [Note: is this a bug?]. There is no enumeration value within options::OPT_? which corresponds to -internal-externc-isystem.

I could check for the existence of this file: /usr/include/stdc-predef.h; I don't know whether this would be acceptable or if it's correct.

[jyknight]

lib/Driver/ToolChains/Linux.cpp

755

I don't see why it makes any sense to condition this based on a GCC installation existing and being >= 4.8, since this header comes from libc, and is necessary for standards compliance even if there's absolutely no GCC installed or involved.

Additionally, something like this -- a way for the libc to be involved in setting up predefined macros -- seems probably necessary for *any* libc, if they want to be strictly standards-compliant. Some of the required-to-be-predefined macros like __STDC_ISO_10646__ can only really be provided by the libc, since the appropriate value is dependent on the locale implementation in the libc, and not on the compiler at all.

It's possible that glibc on Linux is the only one who's cared to implement it thus far (and, only when you're using GCC, at that, hence this change...). That seems likely, really, since mostly the impacted macros are nearly useless, so who cares... :) However, if others want to be conforming, I expect they _ought_ to be using this mechanism, as well...

Thus, perhaps it ought to be an unconditional behavior of clang to include the file if it exists, unless -ffreestanding is passed.

[mibintc]

I rewrote the patch the way that Richard suggested, adding a cc1 option "finclude-if-exists" and injecting #if __has_include etc. [OK to use finclude? include-if-exists preferred?]
I responded to James' remarks and removed the gcc version check.
Does this look better?

[jyknight]

lib/Driver/ToolChains/Linux.cpp

755

This is still checking GCCInstallation.isValid(), OPT_nostdinc, and only attempting to load the header on Linux OSes.

I don't think it shouldn't be conditioned by the former two, and I think it should attempt to include stdc-predef.h if it exists on *all* OSes, not just Linux (for the reasons described in the above comment).

[mibintc]

I updated the patch as James directed, moving the preinclude of stdc-predef.h into clang.cpp and out of the Linux toolchain entirely. I also needed to update a couple more tests in tools/extra, so I will need to update that related differential as well.

[jyknight]

This version is still disabling upon -nostdinc, which doesn't make sense to me.

You didn't remove that, nor respond explaining why you think it does make sense?

[hfinkel]

In D34158#803752, @jyknight wrote:

This version is still disabling upon -nostdinc, which doesn't make sense to me.

I agree. If I pass -nostdinc, so that I then arrange include paths for my own libc headers, I'd then like to pick up the predef header from that library (if available). There's no clearly right answer for whether __STDC_NO_THREADS__ is defined, for example, regardless of whether we're using -nostdinc.

You didn't remove that, nor respond explaining why you think it does make sense?

[mibintc]

In D34158#803752, @jyknight wrote:

This version is still disabling upon -nostdinc, which doesn't make sense to me.

You didn't remove that, nor respond explaining why you think it does make sense?

You're right, I should remove the check on nostdinc. Sorry I missed that remark from you initially, that's why I didn't reply. I will upload another patch which removes the check on nostdinc.

[mibintc]

I removed the check on -nostdinc; and made some updates to the test cases

[mibintc]

OK folks, I was off the grid last week but I'm back now, and at my grindstone again.
I haven't received any comments since I updated the patch to remove the checks on "-nostdinc".
Look OK to commit?
Many thanks for all your reviews
--Melanie

[mibintc]

Ping.

[mibintc]

I'm going to rebase the patch to latest.

[mibintc]

Here is an updated diff which is rebased to current trunk per @fedor.sergeev 's suggestion (thank you). make check-all and check-clang are working for me on Linux with no unexpected failures.

The associated patch for test cases in the tools/extra directory is still valid, it doesn't need updating.

[fedor.sergeev]

Hmm... I tried this patch and now the following worries me:

• it passes -finclude-if-exists stdc-predef.h on all platforms (say, including my Solaris platform that has no system stdc-predef.h)
• it searches all the paths, not just "system include" ones

That essentially disallows user to have stdc-predef.h include in my own project, since there is a chance that this user header
will be accidentally included by this hidden machinery.

] cat stdc-predef.h
#error I was not expecting to see this!
] bin/clang hello-world.c
In file included from <built-in>:2:
./stdc-predef.h:1:2: error: I was not expecting to see this!
#error I was not expecting to see this!
 ^
1 error generated.
]

[jyknight]

In D34158#823316, @fedor.sergeev wrote:

Hmm... I tried this patch and now the following worries me:

• it passes -finclude-if-exists stdc-predef.h on all platforms (say, including my Solaris platform that has no system stdc-predef.h)

Right, but Solaris probably _ought_ to add one as well, to define those macros.

• it searches all the paths, not just "system include" ones

That essentially disallows user to have stdc-predef.h include in my own project, since there is a chance that this user header
will be accidentally included by this hidden machinery.

IMO, this is a fairly negligible issue, and so we go *shrug* oh well.

+1 for using a <> include -- that does seem better.

But, note, that will have no effect w.r.t. this issue for most users, since typically people use "cc -Isomepath", which adds 'somepath' to the list which gets searched by both <> and "" includes. Hardly anyone uses -iquote.

[fedor.sergeev]

In D34158#824079, @jyknight wrote:

In D34158#823316, @fedor.sergeev wrote:

Hmm... I tried this patch and now the following worries me:

• it passes -finclude-if-exists stdc-predef.h on all platforms (say, including my Solaris platform that has no system stdc-predef.h)

Right, but Solaris probably _ought_ to add one as well, to define those macros.

Point taken, started internal discussion with Solaris header folks.

+1 for using a <> include -- that does seem better.

This is the only remaining issue that I would like to see fixed here.

[mibintc]

@erichkeane contacted me offline and pointed out I'm twine-ing with " not <. i'm planning to change the option name from "finclude if exists" to "fsystem include if exists", then i'll quote with < not ". hope to get this updated patch into review later today. let me know if you think i'm off track. thanks for all your careful review. i knew there was something wrong with the include like Fedor pointed out but i couldn't see where i went wrong.

[mibintc]

Here's an updated patch which is using angle brackets to do the include, so the search for stdc-predef.h is limited to system directories. Also my previous revision was missing the new test cases since i had gotten a new sandbox but forgot to "svn add". This patch is showing the new test cases.

[fedor.sergeev]

test/Driver/stdc-predef.c

15 ↗

(On Diff #108669)

I would rather see a real check on include file inclusion (say, checking -H output) than a check for a macro.
Exact macro guard name is not a public interface at all. You might be lucky with current stdc-predef.h on Linux, but on other platforms it could be named differently.

[mibintc]

This patch responds to @fedor.sergeev 's feedback from earlier today. This is a change to the test case test/Driver/stdc-predef.c in the situation that the system does not supply a version of stdc-predef.h. Fedor had suggested an option like -H which traces the include files, but the file stdc-predef.h does not appear in the -H standard include tracing even if it is available on the host system. Instead for this case I preprocess the simple test file, with sysroot pointing to a filesystem missing the file stdc-predef.h, and verify with FileCheck that the string stdc-predef.h doesn't appear in the preprocessed output.

[joerg]

I had a long discussion with James about this on IRC without reaching a clear consensus. I consider forcing this behavior on all targets to be a major bug. It should be opt-in and opt-in only:

(1) The header name is not mandated by any standard. It is not in any namespace generally accepted as implementation-owned.
(2) It adds magic behavior that can make debugging more difficult. Partially preprocessed sources for example could be compiled with plain -c before, now they need a different command line.
(3) It seems to me that the GNU userland (and maybe Windows) is the exception to a well integrated tool chain. Most other platforms have a single canonical libc, libm and libpthread implementation and can as such directly define all the relevant macros directly in the driver. Given that many of the macros involved are already reflected by the compiler behavior anyway, they can't be decoupled. I.e. the questionable concept of locale-independent wchar_t is already hard-coded in the front end as soon as any long character literals are used.

As such, please move the command line additions back into the target-specific files for targets that actually want to get this behavior.

[mibintc]

I will prepare another patch responding to joerg's comment:

Quoted Text

I had a long discussion with James about this on IRC without reaching a clear consensus. I consider forcing this behavior on all targets to be a major bug. It should be opt-in and opt-in only:

(1) The header name is not mandated by any standard. It is not in any namespace generally accepted as implementation-owned.
(2) It adds magic behavior that can make debugging more difficult. Partially preprocessed sources for example could be compiled with plain -c before, now they need a different command line.
(3) It seems to me that the GNU userland (and maybe Windows) is the exception to a well integrated tool chain. Most other platforms have a single canonical libc, libm and libpthread implementation and can as such directly define all the relevant macros directly in the driver. Given that many of the macros involved are already reflected by the compiler behavior anyway, they can't be decoupled. I.e. the questionable concept of locale-independent wchar_t is already hard-coded in the front end as soon as any long character literals are used.

As such, please move the command line additions back into the target-specific files for targets that actually want to get this behavior.

[mibintc]

In the last review, it was deemed less controversial if I move these updates back into the gnu/Linux tool chain. This revision is responding to that feedback. I also simplified the test case. I tested on Linux with check-all and check-clang and found no unexpected failures.

The other test case modifications are to fix tests which are broken with this change due to unexpected changes to the preprocessing. I changed all these to pass -ffreestanding which inhibits the new behavior.

In fact I did have trouble writing the new test case to pass with the gnu/Linux toolchain. In the file lib/Driver/ToolChains/Linux.cpp function AddGnuIncludeArgs checks if GCCInstallation.isValid(). I don't know how to set up a Driver/Inputs sysroot tree so that the isValid check passes. (Does anyone know?) I experimented with various existing trees and had success with basic_cross_linux_tree, so I used that one in the test case. However basic_cross_linux_tree doesn't contain stdc-predef.h. Would it be OK to add an include file to that tree? (I don't know if it's acceptable to add a new sysroot tree: it is a lot of new files and directories. On the other hand, I don't know if it's OK to co-opt a tree which was added by a different test case) Perhaps you want me to another test case which verifies that fsystem-include-if-exists works generally. Currently I've added an affirmative test to see that -fsystem-include-if-exists is on the command line, and a test that -ffreestanding inhibits, and a negative test that when given a sysroot without the file, that the preprocessed output shows no indication of stdc-predef.h.

[fedor.sergeev]

In D34158#834298, @mibintc wrote:

In fact I did have trouble writing the new test case to pass with the gnu/Linux toolchain. In the file lib/Driver/ToolChains/Linux.cpp function AddGnuIncludeArgs checks if GCCInstallation.isValid().

You should not be doing stdc-predef.h under GCCInstallation.isValid().
You are handling interaction with libc, so it has nothing to do with the presence or absence of gcc toolchain.

[mibintc]

Responding to @fedor.sergeev 's comment. This is an updated patch which pulls out the "isValid" check on GCCInstallation. Also I'm putting back the dummy sysroot tree which contains stdc-predef.h and adding a new test run to confirm that the new option fsystem-include-if-exists is actually working.

[jyknight]

In D34158#827178, @joerg wrote:

I had a long discussion with James about this on IRC without reaching a clear consensus. I consider forcing this behavior on all targets to be a major bug. It should be opt-in and opt-in only:

(1) The header name is not mandated by any standard. It is not in any namespace generally accepted as implementation-owned.

This is a point. I didn't think it was a big deal, but if you want to argue a different name should be used, that's a reasonable argument. If we can get some agreement amongst other libc vendors to use some more agreeable alternative name, and keep a fallback on linux-only for the "stdc-predef.h" name, I'd consider that as a great success.

(2) It adds magic behavior that can make debugging more difficult. Partially preprocessed sources for example could be compiled with plain -c before, now they need a different command line.

If this is a problem, making it be Linux-only does _nothing_ to solve it. But I don't actually see how this is a substantively new problem? Compiling with plain -c before would get #defines for those predefined macros that the compiler sets, even though you may not have wanted those. Is this fundamentally different?

(3) It seems to me that the GNU userland (and maybe Windows) is the exception to a well integrated tool chain. Most other platforms have a single canonical libc, libm and libpthread implementation and can as such directly define all the relevant macros directly in the driver.

I don't think this is accurate. There's many platforms out there, and for almost none of them do we have exact knowledge of the features of the libc encoded into the compiler. I'd note that not only do you need this for every (OS, libc) combination, you'd need it for every (OS, libc-VERSION) combination.

Given that many of the macros involved are already reflected by the compiler behavior anyway, they can't be decoupled. I.e. the questionable concept of locale-independent wchar_t is already hard-coded in the front end as soon as any long character literals are used.

AFAICT, this example is not actually the case. The frontend only needs to know *a* valid encoding for wide-character literals in some implementation-defined locale (for example, it might always emit them as unicode codepoints, as clang does). Standard says: "the array elements [...] are initialized with the sequence of wide characters corresponding to the multibyte character sequence, as defined by the mbstowcs function with an implementation defined current locale."

On the other hand, I believe the intent of this macro is to guarantee that _no matter what_ the locale is, that a U+0100 character (say, translated with mbrtowc from the locale encoding) gets represented as 0x100.

Thus, it's "fine" for the frontend to always emit wchar_t literals as unicode, yet, the libc may sometimes use an arbitrary different internal encoding, depending on the locale used at runtime. (Obviously using wide character literals with such a libc would be a poor user experience, and such a libc probably ought to reconsider its choices, but that's a different discussion.)

As such, please move the command line additions back into the target-specific files for targets that actually want to get this behavior.

Without even a suggestion of a better solution to use for other targets, I find it is to be a real shame to push for this to be linux-only, and leave everyone else hanging. I find that that _most_ of these defines are effectively library decisions. I further would claim that these are likely to vary over the lifetime of even a single libc, and that as such we would be better served by allowing the libc to set them directly, rather than encoding it into the compiler.

TTBOMK, no targets except linux/glibc/gcc actually comply with this part of the C99/C11 standard today, and so this feature would be useful to have available across all targets.

(I very much dislike that the C standard has started adding all these new predefined macros, instead of exposing them from a header, but there's not much to be done about that...)

Going through the various macros:
__STDC_ISO_10646__:
As discussed above, this is effectively entirely up to the libc. The compiler only need support one possible encoding for wchar_t, and clang always chooses unicode. But it can't define this because the libc might use others as well.

__STDC_MB_MIGHT_NEQ_WC__:
As with __STDC_ISO_10646__, this is up to the libc not the compiler. (At least, I think so... I note that clang currently sets this for freebsd, with a FIXME next to it saying it's only intended to apply to wide character literals. I don't see that the standard says that, however, so, IMO, having it set for freebsd was and is correct).

__STDC_UTF16__, __STDC_UTF32__:
Again, analogous to __STDC_ISO_10646__, except dealing with char16_t/char32_t. this should be set if the libc guarantees that mbrtoc16/mbrtoc32 generate utf16/32-encoded data.

__STDC_ANALYZABLE__
Possibly(?) entirely on the compiler side. (I'm going to bet that nobody will ever implement this, though.)

__STDC_IEC_559__, __STDC_IEC_559_COMPLEX__:
Requires cooperation between library and compiler; some parts of compliance is in the compiler, and some is in the libm routines. GCC defines __GCC_IEC_559 to indicate when the compiler intends to comply, and glibc then sets __STDC_IEC_559__ depending on the GCC define, to indicate the system as a whole is intended to comply.

__STDC_LIB_EXT1__:
Entirely a library issue, nothing to do with the compiler.

__STDC_NO_COMPLEX__:
Given that clang supports complex, what remains is: does libc provides complex.h and the functions within?

__STDC_NO_THREADS__:
Given that clang supports _Atomic (and provides stdatomic.h), what remains is: does libc provide thread.h and the functions within?

__STDC_NO_VLA__:
Entirely on the compiler, nothing to do with libc.

[fedor.sergeev]

In D34158#836026, @jyknight wrote:

In D34158#827178, @joerg wrote:

I had a long discussion with James about this on IRC without reaching a clear consensus. I consider forcing this behavior on all targets to be a major bug. It should be opt-in and opt-in only:

(1) The header name is not mandated by any standard. It is not in any namespace generally accepted as implementation-owned.

This is a point. I didn't think it was a big deal, but if you want to argue a different name should be used, that's a reasonable argument.
If we can get some agreement amongst other libc vendors to use some more agreeable alternative name, and keep a fallback on linux-only for the "stdc-predef.h" name, I'd consider that as a great success.

Perhaps not a big deal yet, but as I have recently described stdc-predef.h idea to Oracle Solaris libc/headers/compilers folks, they generally welcomed the idea..

(3) ...Most other platforms have a single canonical libc, libm and libpthread implementation and can as such directly define all the relevant macros directly in the driver.

I don't think this is accurate. There's many platforms out there, and for almost none of them do we have exact knowledge of the features of the libc encoded into the compiler.

Solaris is a direct example of that...

[joerg]

In D34158#836026, @jyknight wrote:

In D34158#827178, @joerg wrote:

(2) It adds magic behavior that can make debugging more difficult. Partially preprocessed sources for example could be compiled with plain -c before, now they need a different command line.

If this is a problem, making it be Linux-only does _nothing_ to solve it. But I don't actually see how this is a substantively new problem? Compiling with plain -c before
would get #defines for those predefined macros that the compiler sets, even though you may not have wanted those. Is this fundamentally different?

It makes it a linux-only problem. As such, it is something *I* only care about secondary. A typical use case I care about a lot is pulling the crash report sources from my (NetBSD) build machine,
extracting the original command line to rerun the normal compilation with -save-temps. I don't necessarily have the (same) system headers on the machine I use for debugging and that's exactly
the kind of use case this change breaks. All other predefined macros are driven by the target triple and remain stable.

(3) It seems to me that the GNU userland (and maybe Windows) is the exception to a well integrated tool chain. Most other platforms have a single canonical
libc, libm and libpthread implementation and can as such directly define all the relevant macros directly in the driver.

I don't think this is accurate. There's many platforms out there, and for almost none of them do we have exact knowledge of the features of the libc encoded
into the compiler. I'd note that not only do you need this for every (OS, libc) combination, you'd need it for every (OS, libc-VERSION) combination.

Not really. The feature set is rarely changing and generally will have only a cut-off version.

Given that many of the macros involved are already reflected by the compiler behavior anyway, they can't be decoupled. I.e. the questionable
concept of locale-independent wchar_t is already hard-coded in the front end as soon as any long character literals are used.

AFAICT, this example is not actually the case. The frontend only needs to know *a* valid encoding for wide-character literals in some
implementation-defined locale (for example, it might always emit them as unicode codepoints, as clang does). Standard says:
"the array elements [...] are initialized with the sequence of wide characters corresponding to the multibyte character sequence, as
defined by the mbstowcs function with an implementation defined current locale."

I know what the standard says. It doesn't make much sense if you do not have a fixed wchar_t encoding.

On the other hand, I believe the intent of this macro is to guarantee that _no matter what_ the locale is,
that a U+0100 character (say, translated with mbrtowc from the locale encoding) gets represented as 0x100.

Yes, so it is essentially "we have screwed up by creating a language mechanism that adds a major constraint, so let's go all the way".

Thus, it's "fine" for the frontend to always emit wchar_t literals as unicode, yet, the libc may sometimes use an arbitrary different
internal encoding, depending on the locale used at runtime. (Obviously using wide character literals with such a libc would be a poor
user experience, and such a libc probably ought to reconsider its choices, but that's a different discussion.)

One of the biggest opponents of that was Itojun. It's not a decision that should be made here as it is only indirectly related to this discussion.

As such, please move the command line additions back into the target-specific files for targets that actually want to get this behavior.

Without even a suggestion of a better solution to use for other targets, I find it is to be a real shame to push for this to be linux-only,
and leave everyone else hanging. I find that that _most_ of these defines are effectively library decisions. I further would claim that these
are likely to vary over the lifetime of even a single libc, and that as such we would be better served by allowing the libc to set them directly, rather than encoding it into the compiler.

TTBOMK, no targets except linux/glibc/gcc actually comply with this part of the C99/C11 standard today, and so this feature would be useful to have available across all targets.

(I very much dislike that the C standard has started adding all these new predefined macros, instead of exposing them from a header, but there's not much to be done about that...)

Exactly. It's not like this is a lot of target logic. It should be a single call for targets that want to get this functionality. But that's my point -- it should be opt-in, not opt-out.

[hfinkel]

In D34158#837130, @joerg wrote:

In D34158#836026, @jyknight wrote:

In D34158#827178, @joerg wrote:

(2) It adds magic behavior that can make debugging more difficult. Partially preprocessed sources for example could be compiled with plain -c before, now they need a different command line.

If this is a problem, making it be Linux-only does _nothing_ to solve it. But I don't actually see how this is a substantively new problem? Compiling with plain -c before
would get #defines for those predefined macros that the compiler sets, even though you may not have wanted those. Is this fundamentally different?

It makes it a linux-only problem. As such, it is something *I* only care about secondary. A typical use case I care about a lot is pulling the crash report sources from my (NetBSD) build machine,
extracting the original command line to rerun the normal compilation with -save-temps. I don't necessarily have the (same) system headers on the machine I use for debugging and that's exactly
the kind of use case this change breaks. All other predefined macros are driven by the target triple and remain stable.

Don't you use preprocessed source files from a crash?

[jyknight]

Just to restate: the ideal outcome of this discussion for me would be to resolve things such that _ALL_ libc implementations will feel comfortable using this technique to provide the C11-required predefined macros.

I'd love for linux, freebsd, macos, solaris, etc etc libc to all conform to the C standard in this regards, and do so in a common way, without the need to encode information about each libc version into the compiler. I _really_ don't think that scales well.

So I take your comments from FreeBSD's point of view seriously, and would very much like to understand and hopefully resolve them.

In D34158#837130, @joerg wrote:

In D34158#836026, @jyknight wrote:

In D34158#827178, @joerg wrote:

(2) It adds magic behavior that can make debugging more difficult. Partially preprocessed sources for example could be compiled with plain -c before, now they need a different command line.

If this is a problem, making it be Linux-only does _nothing_ to solve it. But I don't actually see how this is a substantively new problem? Compiling with plain -c before
would get #defines for those predefined macros that the compiler sets, even though you may not have wanted those. Is this fundamentally different?

It makes it a linux-only problem. As such, it is something *I* only care about secondary. A typical use case I care about a lot is pulling the crash report sources from my (NetBSD) build machine,
extracting the original command line to rerun the normal compilation with -save-temps. I don't necessarily have the (same) system headers on the machine I use for debugging and that's exactly
the kind of use case this change breaks. All other predefined macros are driven by the target triple and remain stable.

"it's Linux only so I don't care if it's broken." is still not very helpful. :)

But I do think understand what you're saying now, so thanks for the elaboration.

Firstly, let's consider a "clang foo.i" or "clang -x cpp-output foo.c" compilation. In that case, it *clearly* should not be including the predef file. I think the patch as it stands may not do this properly. A test needs to be added for this to this patch, and perhaps the behavior needs to be fixed as well.

(Sidenote: clang doesn't support preprocessed input properly, but that's another bug, and we certainly ought not make it worse. Check out e.g. "int main() { return GNUC; }". it should report that GNUC is undeclared, but instead compiles a program that returns 4.)

But, that's not the case you're talking about above -- you're not talking about compiling preprocessed output, you're talking about taking output that comes from -frewrite-includes.

Let me recap the scenario:

1. Start with a source file foo.c, with this content:

#include <stdlib.h>
#pragma clang __debug parser_crash

2. Run "clang foo.c". It crashes, and dumps a /tmp/foo-XXX.c and a /tmp/foo-XXX.sh script.

The .c file is generated via -frewrite-includes, so it's _not_ already preprocessed, it simply has all includes pulled into a single file. It also _doesn't_ insert the compiler-predefined macros at the top, but it _will_ include the content of this stdc-predef.h file.

OK, so then...
The generated script invokes a -cc1 command line, with all the include arguments stripped out of the command. (TO FIX: We should be stripping the new arg as well: add "-fsystem-include-if-exists" argument to the list of include things in the skipArgs() function in lib/Driver/Job.cpp). Even without that change, it's actually already fine, as there is no include path specified in which to find the file -- but it's cleaner to strip it, so let's do that. The reproducer script will thus run correctly, and not include the file.

Now, the "/tmp/foo-XXX.sh" also has a line labeled "Driver args: " with the original command-line on it. If I understand correctly, you then like to take this simpler Driver command-line, and edit it manually: add -save-temps, and change the input filename to the "/tmp/foo-XXX.c" file, and run that, instead of actually invoking the reproducer foo-XXX.sh.

Since stdc-predef.h is included automatically, it will now be present twice -- first, it will read the one from your system's /usr/include, and then the copy inlined into the /tmp/foo-XXX.c file. That's not what you desired. You wanted nothing from your /usr/include to be used.

The fix for the end-user here is easy: you can add -nostdinc which will suppress all the default include paths, and thus it will not find this predef file from your system include dir.

I'll note that you'd also have had an issue if the original driver command-line had a "-include" option in it, which you would have needed to edit out manually as well. (But I understand that is less common.)

Have I correctly described the situation? I guess my feeling here is that this is somewhat of an edge-case, and that the workaround (-nostdinc) is a sufficient fix.

(3) It seems to me that the GNU userland (and maybe Windows) is the exception to a well integrated tool chain. Most other platforms have a single canonical
libc, libm and libpthread implementation and can as such directly define all the relevant macros directly in the driver.

I don't think this is accurate. There's many platforms out there, and for almost none of them do we have exact knowledge of the features of the libc encoded
into the compiler. I'd note that not only do you need this for every (OS, libc) combination, you'd need it for every (OS, libc-VERSION) combination.

Not really. The feature set is rarely changing and generally will have only a cut-off version.

Yes, but this is information that the compiler has no real need to know, and that for many platforms would be difficult and problematic to coordinate.

[joerg]

In D34158#837281, @hfinkel wrote:

In D34158#837130, @joerg wrote:

In D34158#836026, @jyknight wrote:

In D34158#827178, @joerg wrote:

(2) It adds magic behavior that can make debugging more difficult. Partially preprocessed sources for example could be compiled with plain -c before, now they need a different command line.

If this is a problem, making it be Linux-only does _nothing_ to solve it. But I don't actually see how this is a substantively new problem? Compiling with plain -c before
would get #defines for those predefined macros that the compiler sets, even though you may not have wanted those. Is this fundamentally different?

It makes it a linux-only problem. As such, it is something *I* only care about secondary. A typical use case I care about a lot is pulling the crash report sources from my (NetBSD) build machine,
extracting the original command line to rerun the normal compilation with -save-temps. I don't necessarily have the (same) system headers on the machine I use for debugging and that's exactly
the kind of use case this change breaks. All other predefined macros are driven by the target triple and remain stable.

Don't you use preprocessed source files from a crash?

The crash rewrite tool creates semi-preprocessed output. It resolves includes along all code branches, but keeps macros and CPP conditionals alone.

[joerg]

In D34158#837444, @jyknight wrote:

Now, the "/tmp/foo-XXX.sh" also has a line labeled "Driver args: " with the original command-line on it. If I understand correctly, you then like to take this
simpler Driver command-line, and edit it manually: add -save-temps, and change the input filename to the "/tmp/foo-XXX.c" file, and run that, instead
of actually invoking the reproducer foo-XXX.sh.

It's not so much that it is simpler, but the only reasonable easy way for forcing preprocessor output to written. That one is much nicer for passing to creduce for example (after stripping the remaining # lines).

Since stdc-predef.h is included automatically, it will now be present twice -- first, it will read the one from your system's /usr/include, and then the copy inlined into the
/tmp/foo-XXX.c file. That's not what you desired. You wanted nothing from your /usr/include to be used.

Correct. Worse, the include may file depending on the host system and give different results from the original build, resulting in annoying debugging seasons.

The fix for the end-user here is easy: you can add -nostdinc which will suppress all the default include paths, and thus it will not find this predef file from your system include dir.

Yeah, except you have to remember that when it was never needed before.

I'll note that you'd also have had an issue if the original driver command-line had a "-include" option in it, which you would have needed to edit out
manually as well. (But I understand that is less common.)

Yes, -include is rarely used in the wild.

Have I correctly described the situation? I guess my feeling here is that this is somewhat of an edge-case, and that the workaround (-nostdinc) is a sufficient fix.

(3) It seems to me that the GNU userland (and maybe Windows) is the exception to a well integrated tool chain. Most other platforms have a single canonical
libc, libm and libpthread implementation and can as such directly define all the relevant macros directly in the driver.

I don't think this is accurate. There's many platforms out there, and for almost none of them do we have exact knowledge of the features of the libc encoded
into the compiler. I'd note that not only do you need this for every (OS, libc) combination, you'd need it for every (OS, libc-VERSION) combination.

Not really. The feature set is rarely changing and generally will have only a cut-off version.

Yes, but this is information that the compiler has no real need to know, and that for many platforms would be difficult and problematic to coordinate.

That's kind of my point. There is little coordination involved for almost all platforms since they provide a consistent user environment as a single package. Pushing the changes into Clang (or GCC for that matter) is trivial if you do not have to deal with multiple different versions of libc, the kernel and whatever in a mix-and-match scenario.

[mibintc]

This patch is responding to @jyknight 's concern about preprocessed input. The patch as it stands doesn't have this issue. I added 2 test cases, one using option -x cpp-output, and another for a source file suffixed with .i

Quoting James: "Firstly, let's consider a "clang foo.i" or "clang -x cpp-output foo.c" compilation. In that case, it *clearly* should not be including the predef file. I think the patch as it stands may not do this properly. A test needs to be added for this to this patch, and perhaps the behavior needs to be fixed as well."

[jyknight]

In D34158#838454, @mibintc wrote:

This patch is responding to @jyknight 's concern about preprocessed input. The patch as it stands doesn't have this issue. I added 2 test cases, one using option -x cpp-output, and another for a source file suffixed with .i

Quoting James: "Firstly, let's consider a "clang foo.i" or "clang -x cpp-output foo.c" compilation. In that case, it *clearly* should not be including the predef file. I think the patch as it stands may not do this properly. A test needs to be added for this to this patch, and perhaps the behavior needs to be fixed as well."

Thanks for the test. It wasn't obvious from the code, so I'm glad to hear it was already correct. :)

Did you see the other suggestion I cleverly hid within a big block of commentary? "(TO FIX: We should be stripping the new arg as well: add "-fsystem-include-if-exists" argument to the list of include things in the skipArgs() function in lib/Driver/Job.cpp)"

[mibintc]

Need TO FIX: We should be stripping the new arg as well: add "-fsystem-include-if-exists" argument to the list of include things in the skipArgs() function in lib/Driver/Job.cpp

[mibintc]

@jyknight recommended adding the new option to skipArgs in Job.cpp. This revision makes that change and tests that in crash-report.c; look OK?

[mibintc]

ping

[mibintc]

Another ping. Is it possible to approve this modification, or alternatively, let me know that it won't be approved indefinitely? I understand that it's a controversial change. Thanks!

[mibintc]

I heard that @jyknight is still interested in this functionality, updating the patch to latest sources, to do this i needed to make a small change to a couple of driver tests. Other than that it's the same as previous submission.

[joerg]

This version is fine with me. The only contentious part is whether it should be opt-in or opt-out for platforms, so getting this version in and revisiting the issue again later is OK from my perspective.

[mibintc]

Hey @jyknight I heard from @erichkeane that you may want a couple more changes to this patch. Please let me know if you have some changes to recommend. @joerg thought this was OK for submission. Thanks --Melanie

[jyknight]

I'd still _very_ much like a solution that is acceptable for all libc to use, and have that on by default.

However, I guess this is fine.

Only concern I have is that it seems odd that so many test-cases needed to be changed. What fails without those test changes?

[mibintc]

In D34158#905596, @jyknight wrote:

I'd still _very_ much like a solution that is acceptable for all libc to use, and have that on by default.

However, I guess this is fine.

Only concern I have is that it seems odd that so many test-cases needed to be changed. What fails without those test changes?

Those tests fail because they generate preprocessed output and then check carefully for precise results. Since the preprocessed output is different, the comparison fails. I "fixed" the tests by adding the freestanding option which inhibits the new behavior. I'll have to check out and rebuild everything so I can show you exactly the failure mode, I expect I can post details tomorrow.

[mibintc]

In D34158#905637, @mibintc wrote:

In D34158#905596, @jyknight wrote:

I'd still _very_ much like a solution that is acceptable for all libc to use, and have that on by default.

However, I guess this is fine.

Only concern I have is that it seems odd that so many test-cases needed to be changed. What fails without those test changes?

Those tests fail because they generate preprocessed output and then check carefully for precise results. Since the preprocessed output is different, the comparison fails. I "fixed" the tests by adding the freestanding option which inhibits the new behavior. I'll have to check out and rebuild everything so I can show you exactly the failure mode, I expect I can post details tomorrow.

I created a fresh workspace and applied the patch, then I make "check-clang". I saw one test fail. The test that failed is Driver/crash-report.c. (BTW I need to recheck all the tests and make sure the test modifications are still needed, and likewise in the "extra" tests directory which is in the separate patch file). This is the failure mode for the Driver crash report. Is this sufficient detail to understand the test issue? If not what else can I provide?

llvm/tools/clang/test/Driver/crash-report.c
Exit Code: 1

Command Output (stderr):
--
/site/spt/usr20/mblower/sptel9-ws/llorg/llvm/tools/clang/test/Driver/crash-report.c:20:11: error: expected string not found in input
// CHECK: Preprocessed source(s) and associated run script(s) are located at:

```          ^
<stdin>:1:1: note: scanning from here
<built-in>:1:10: fatal error: '-fobjc-runtime=gcc' file not found
^

If I make this change, adding the freestanding option, and rebuild "check-clang", then no tests are failing:
diff --git a/test/Driver/crash-report.c b/test/Driver/crash-report.c
index a3f1f9e..526e4dd 100644
--- a/test/Driver/crash-report.c
+++ b/test/Driver/crash-report.c
@@ -2,7 +2,7 @@
 // RUN: mkdir %t
 // RUN: not env TMPDIR=%t TEMP=%t TMP=%t RC_DEBUG_OPTIONS=1              \
 // RUN:  CC_PRINT_HEADERS=1 CC_LOG_DIAGNOSTICS=1                         \
-// RUN:  %clang -fsyntax-only %s                                         \
+// RUN:  %clang -fsyntax-only -ffreestanding %s
 // RUN:  -F/tmp/ -I /tmp/ -idirafter /tmp/ -iquote /tmp/ -isystem /tmp/  \
 // RUN:  -iprefix /the/prefix -iwithprefix /tmp -iwithprefixbefore /tmp/ \
 // RUN:  -Xclang -internal-isystem -Xclang /tmp/                         \

[mibintc]

This comment was removed by mibintc.

[mibintc]

The patch to clang is the same as before. The modifications are to the test cases, I verified that all these test cases need modifications. I will also be updating the associated diff for clang/tools/extra, please review that as well. Without the test patches, these are the tests that fail (using check-all) . The failure mode is as described in a note I posted earlier this week, please let me know if you need further details. @jyknight Looking for your review and approval, thanks!

Clang :: Driver/crash-report-header.h
Clang :: Driver/crash-report-spaces.c
Clang :: Driver/crash-report.c
Clang :: Driver/rewrite-map-in-diagnostics.c
Clang :: Driver/stdc-predef.c
Clang :: Index/IBOutletCollection.m
Clang :: Index/annotate-macro-args.m
Clang :: Index/annotate-tokens-pp.c
Clang :: Index/annotate-tokens.c
Clang :: Index/c-index-getCursor-test.m
Clang :: Index/get-cursor-macro-args.m
Clang :: Index/get-cursor.cpp
Clang :: Preprocessor/ignore-pragmas.c
Clang Tools :: pp-trace/pp-trace-pragma-general.cpp
Clang Tools :: pp-trace/pp-trace-pragma-opencl.cpp
Clang-Unit :: Tooling/./ToolingTests/CommentHandlerTest.BasicTest1
Clang-Unit :: Tooling/./ToolingTests/CommentHandlerTest.BasicTest2
Clang-Unit :: Tooling/./ToolingTests/CommentHandlerTest.BasicTest3
Clang-Unit :: Tooling/./ToolingTests/CommentHandlerTest.IfBlock1
Clang-Unit :: Tooling/./ToolingTests/CommentHandlerTest.IfBlock2
Clang-Unit :: Tooling/./ToolingTests/CommentHandlerTest.IfBlock3
Clang-Unit :: Tooling/./ToolingTests/CommentHandlerTest.PPDirectives

[mibintc]

Hi.

I also posted this question on IRC

@erichkeane pushed a fix for D34158 which on Linux does a preinclude of stdc-predef.h; on Monday. Later on Monday he pulled it out because it caused 3 test failures and word-of-mouth that it also caused chromium build to fail. I haven't been watching this IRC so I didn't catch the details about the chromium build fail.

if you know, can you send me details about how/why chromium build fails?

also, the 3 tests (2 lit tests, 1 google test) which failed on buildbot do not fail on the linux servers in-house at Intel. Any clue about why that would be? I have a patched compiler, and verify that the new functionality is present. I build check-clang and there are no fails.

I looked with llvm-lit at each of the 2 failing tests and they don't fail, I checked the google test libclang "skip ranges" and it doesn't fail

also maybe i should redirect this question elsewhere. don't hesitate to let me know

also i have used the self-build for D34158 and likewise those 3 tests do not fail.

i used "llvm-lit -a" on one of the failing tests test/Modules/crash-vfs-path-symlink-component.m to see why the test was failing. i was not able to run the commands directly from the linux shell
the lit command that didn't work is not env FORCE_CLANG_DIAGNOSTICS_CRASH= ...
it's the "not" which is not understood - does that ring a bell? it works OK within llvm-lit
sptel9-ws/llorg1/builds/llorgefi2linux_debug/llvm/bin/count 1 ; the buildbot is showing 2 not 1

include/clang/Driver/ToolChain.h

public:
   /// This checks for presence of the -Ofast, -ffast-math or -funsafe-math flags.
   virtual bool AddFastMathRuntimeIfAvailable(
       const llvm::opt::ArgList &Args, llvm::opt::ArgStringList &CmdArgs) const;
   /// addProfileRTLibs - When -fprofile-instr-profile is specified, try to pass
   /// a suitable profile runtime library to the linker.
   virtual void addProfileRTLibs(const llvm::opt::ArgList &Args,
                                 llvm::opt::ArgStringList &CmdArgs) const;
 
+  /// \brief Add arguments to use system-specific GNU includes.
+  virtual void AddGnuIncludeArgs(const llvm::opt::ArgList &DriverArgs,
+                                  llvm::opt::ArgStringList &CC1Args) const;
+
   /// \brief Add arguments to use system-specific CUDA includes.
   virtual void AddCudaIncludeArgs(const llvm::opt::ArgList &DriverArgs,
                                   llvm::opt::ArgStringList &CC1Args) const;
 
   /// \brief Add arguments to use MCU GCC toolchain includes.
   virtual void AddIAMCUIncludeArgs(const llvm::opt::ArgList &DriverArgs,
                                    llvm::opt::ArgStringList &CC1Args) const;
 

lib/Driver/ToolChain.cpp

if (getTriple().getArch() == llvm::Triple::x86 ||
       getTriple().getArch() == llvm::Triple::arm ||
       getTriple().getArch() == llvm::Triple::aarch64 ||
       getTriple().getArch() == llvm::Triple::wasm32 ||
       getTriple().getArch() == llvm::Triple::wasm64)
     Res |= CFIICall;
   return Res;
 }
 
+void ToolChain::AddGnuIncludeArgs(const ArgList &DriverArgs,
+                                   ArgStringList &CC1Args) const {}
+
 void ToolChain::AddCudaIncludeArgs(const ArgList &DriverArgs,
                                    ArgStringList &CC1Args) const {}
 
 void ToolChain::AddIAMCUIncludeArgs(const ArgList &DriverArgs,
                                     ArgStringList &CC1Args) const {}
 
 static VersionTuple separateMSVCFullVersion(unsigned Version) {
   if (Version < 100)

lib/Driver/ToolChains/Gnu.h

public:
   public:
     explicit GCCInstallationDetector(const Driver &D) : IsValid(false), D(D) {}
     void init(const llvm::Triple &TargetTriple, const llvm::opt::ArgList &Args,
               ArrayRef<std::string> ExtraTripleAliases = None);
 
     /// \brief Check whether we detected a valid GCC install.
     bool isValid() const { return IsValid; }
 
+    void AddGnuIncludeArgs(const llvm::opt::ArgList &DriverArgs,
+                          llvm::opt::ArgStringList &CC1Args) const;
+
     /// \brief Get the GCC triple for the detected install.
     const llvm::Triple &getTriple() const { return GCCTriple; }
 
     /// \brief Get the detected GCC installation path.
     StringRef getInstallPath() const { return GCCInstallPath; }
 
     /// \brief Get the detected GCC parent lib path.
     StringRef getParentLibPath() const { return GCCParentLibPath; }
protected:
 
   bool addLibStdCXXIncludePaths(Twine Base, Twine Suffix, StringRef GCCTriple,
                                 StringRef GCCMultiarchTriple,
                                 StringRef TargetMultiarchTriple,
                                 Twine IncludeSuffix,
                                 const llvm::opt::ArgList &DriverArgs,
                                 llvm::opt::ArgStringList &CC1Args) const;
 
   /// @}
 
 private:
   mutable std::unique_ptr<tools::gcc::Preprocessor> Preprocess;
   mutable std::unique_ptr<tools::gcc::Compiler> Compile;
 };
 
 class LLVM_LIBRARY_VISIBILITY Generic_ELF : public Generic_GCC {
   virtual void anchor();
 
 public:

lib/Driver/ToolChains/Gnu.cpp

 bool Generic_GCC::GCCInstallationDetector::getBiarchSibling(Multilib &M) const {
   if (BiarchSibling.hasValue()) {
     M = BiarchSibling.getValue();
     return true;
   }
   return false;
 }
 
+void Generic_GCC::GCCInstallationDetector::AddGnuIncludeArgs(
+    const ArgList &DriverArgs, ArgStringList &CC1Args) const {
+  if (!DriverArgs.hasArg(options::OPT_ffreestanding) &&
+      !DriverArgs.hasArg(clang::driver::options::OPT_nostdinc) &&
+      !Version.isOlderThan(4, 8, 0)) {
+    CC1Args.push_back("-include");
+    CC1Args.push_back("stdc-predef.h");
+  }
+}
+
 /*static*/ void Generic_GCC::GCCInstallationDetector::CollectLibDirsAndTriples(
     const llvm::Triple &TargetTriple, const llvm::Triple &BiarchTriple,
     SmallVectorImpl<StringRef> &LibDirs,
     SmallVectorImpl<StringRef> &TripleAliases,
     SmallVectorImpl<StringRef> &BiarchLibDirs,
     SmallVectorImpl<StringRef> &BiarchTripleAliases) {
   // Declare a bunch of static data sets that we'll select between below. These
   // are specifically designed to always refer to string literals to avoid any
case llvm::Triple::mips64el:
     if (getTriple().getEnvironment() == llvm::Triple::GNUABI64)
       return true;
     return false;
   default:
     return false;
   }
 }
 
 void Generic_GCC::AddClangCXXStdlibIncludeArgs(const ArgList &DriverArgs,
                                                ArgStringList &CC1Args) const {
   if (DriverArgs.hasArg(options::OPT_nostdlibinc) ||
       DriverArgs.hasArg(options::OPT_nostdincxx))
     return;
 
   switch (GetCXXStdlibType(DriverArgs)) {
   case ToolChain::CST_Libcxx: {
     std::string Path = findLibCxxIncludePath();
     if (!Path.empty())
       addSystemInclude(DriverArgs, CC1Args, Path);
     break;
   }
 
   case ToolChain::CST_Libstdcxx:
     addLibStdCxxIncludePaths(DriverArgs, CC1Args);
     break;
   }
 }
 
 std::string Generic_GCC::findLibCxxIncludePath() const {
   // FIXME: The Linux behavior would probaby be a better approach here.
   return getDriver().SysRoot + "/usr/include/c++/v1";
 }
 
 void

lib/Driver/ToolChains/Linux.h

public:
 
   void
   AddClangSystemIncludeArgs(const llvm::opt::ArgList &DriverArgs,
                             llvm::opt::ArgStringList &CC1Args) const override;
   std::string findLibCxxIncludePath() const override;
   void addLibStdCxxIncludePaths(
       const llvm::opt::ArgList &DriverArgs,
       llvm::opt::ArgStringList &CC1Args) const override;
+  void AddGnuIncludeArgs(const llvm::opt::ArgList &DriverArgs,
+                          llvm::opt::ArgStringList &CC1Args) const;
   void AddCudaIncludeArgs(const llvm::opt::ArgList &DriverArgs,
                           llvm::opt::ArgStringList &CC1Args) const override;
   void AddIAMCUIncludeArgs(const llvm::opt::ArgList &DriverArgs,
                            llvm::opt::ArgStringList &CC1Args) const override;
   bool isPIEDefault() const override;
   SanitizerMask getSupportedSanitizers() const override;
   void addProfileRTLibs(const llvm::opt::ArgList &Args,
                         llvm::opt::ArgStringList &CmdArgs) const override;

lib/Driver/ToolChains/Linux.cpp

if (getTriple().getOS() == llvm::Triple::RTEMS)
     return;
 
   // Add an include of '/include' directly. This isn't provided by default by
   // system GCCs, but is often used with cross-compiling GCCs, and harmless to
   // add even when Clang is acting as-if it were a system compiler.
   addExternCSystemInclude(DriverArgs, CC1Args, SysRoot + "/include");
 
   addExternCSystemInclude(DriverArgs, CC1Args, SysRoot + "/usr/include");
+
+  GCCInstallation.AddGnuIncludeArgs(DriverArgs, CC1Args);
 }
 
 static std::string DetectLibcxxIncludePath(StringRef base) {
   std::error_code EC;
   int MaxVersion = 0;
   std::string MaxVersionString = "";
   for (llvm::sys::fs::directory_iterator LI(base, EC), LE; !EC && LI != LE;
        LI = LI.increment(EC)) {
for (const auto &IncludePath : LibCXXIncludePathCandidates) {
     if (IncludePath.empty() || !getVFS().exists(IncludePath))
       continue;
     // Use the first candidate that exists.
     return IncludePath;
   }
   return "";
 }
 
+void Linux::AddGnuIncludeArgs(const llvm::opt::ArgList &DriverArgs,
+                                     llvm::opt::ArgStringList &CC1Args) const {
+  if (GCCInstallation.isValid())
+    GCCInstallation.AddGnuIncludeArgs(DriverArgs, CC1Args);
+}
+
+
 void Linux::addLibStdCxxIncludePaths(const llvm::opt::ArgList &DriverArgs,
                                      llvm::opt::ArgStringList &CC1Args) const {
   // We need a detected GCC installation on Linux to provide libstdc++'s
   // headers.
   if (!GCCInstallation.isValid())
     return;
 
   // By default, look for the C++ headers in an include directory adjacent to
   // the lib directory of the GCC installation. Note that this is expect to be

lib/Driver/ToolChains/MipsLinux.h

 
 public:
   MipsLLVMToolChain(const Driver &D, const llvm::Triple &Triple,
                     const llvm::opt::ArgList &Args);
 
   void
   AddClangSystemIncludeArgs(const llvm::opt::ArgList &DriverArgs,
                             llvm::opt::ArgStringList &CC1Args) const override;
+  void AddGnuIncludeArgs(const llvm::opt::ArgList &DriverArgs,
+                          llvm::opt::ArgStringList &CC1Args) const override;
 
   CXXStdlibType GetCXXStdlibType(const llvm::opt::ArgList &Args) const override;
 
   std::string findLibCxxIncludePath() const override;
 
   void AddCXXStdlibLibArgs(const llvm::opt::ArgList &Args,
                            llvm::opt::ArgStringList &CmdArgs) const override;
 

lib/Driver/ToolChains/MipsLinux.cpp

void MipsLLVMToolChain::AddClangSystemIncludeArgs(
   const auto &Callback = Multilibs.includeDirsCallback();
   if (Callback) {
     for (const auto &Path : Callback(SelectedMultilib))
       addExternCSystemIncludeIfExists(DriverArgs, CC1Args,
                                       D.getInstalledDir() + Path);
   }
 }
 
+void MipsLLVMToolChain::AddGnuIncludeArgs(const llvm::opt::ArgList &DriverArgs,
+                                     llvm::opt::ArgStringList &CC1Args) const {
+  if (GCCInstallation.isValid())
+    GCCInstallation.AddGnuIncludeArgs(DriverArgs, CC1Args);
+}
+
 Tool *MipsLLVMToolChain::buildLinker() const {
   return new tools::gnutools::Linker(*this);
 }
 
 std::string MipsLLVMToolChain::computeSysRoot() const {
   if (!getDriver().SysRoot.empty())
     return getDriver().SysRoot + SelectedMultilib.osSuffix();
 

lib/Driver/ToolChains/Solaris.cpp

addSystemInclude(DriverArgs, CC1Args,
                      Version.MajorStr + "." +
                      Version.MinorStr +
                      "/include/c++/" + Version.Text);
     addSystemInclude(DriverArgs, CC1Args,
                      getDriver().SysRoot + "/usr/gcc/" + Version.MajorStr +
                      "." + Version.MinorStr + "/include/c++/" +
                      Version.Text + "/" +
                      GCCInstallation.getTriple().str());
+    GCCInstallation.AddGnuIncludeArgs(DriverArgs, CC1Args);
   }
 }

test/Driver/clang_cpp.c

 // Verify that -include isn't included twice with -save-temps.
-// RUN: %clang -S -o - %s -include %t.h -save-temps -### 2> %t.log
+// RUN: %clang -ffreestanding -S -o - %s -include %t.h -save-temps -### 2> %t.log
 // RUN: FileCheck %s < %t.log
 // CHECK: "-include
 // CHECK-NOT: "-include

test/Driver/gcc-predef.c

+// Test that gcc preincludes stdc-predef.h on Linux
+//
+// RUN: %clang -c --target=i386-unknown-linux %s -Xclang -verify
+// expected-no-diagnostics
+#if defined( __GNUC__ )
+  #if __GNUC__ > 0 && (__GNUC__ < 4 || __GNUC__ == 4 && __GNUC_MINOR__ < 8)
+    /* check preinclude for gcc 4.8 and higher */
+  #else
+    #if !defined(  _STDC_PREDEF_H )
+      #error "stdc-predef.h should be preincluded for GNU/Linux 4.8 and higher"
+    #endif
+  #endif
+#endif

test/Index/IBOutletCollection.m

 #define IBOutletCollection(ClassName) __attribute__((iboutletcollection(ClassName)))
 
 @interface Test {
   IBOutletCollection(Test) Test *anOutletCollection;
 }
 @end
 
-// RUN: c-index-test -cursor-at=%s:4:24 %s | FileCheck -check-prefix=CHECK-CURSOR %s
+// RUN: c-index-test -cursor-at=%s:4:24 -ffreestanding %s | FileCheck -check-prefix=CHECK-CURSOR %s
 // CHECK-CURSOR: ObjCClassRef=Test:3:12
 
-// RUN: c-index-test -test-annotate-tokens=%s:4:1:5:1 %s | FileCheck -check-prefix=CHECK-TOK %s
+// RUN: c-index-test -test-annotate-tokens=%s:4:1:5:1 -ffreestanding %s | FileCheck -check-prefix=CHECK-TOK %s
 // CHECK-TOK: Identifier: "IBOutletCollection" [4:3 - 4:21] macro expansion=IBOutletCollection:1:9
 // FIXME: The following token should belong to the macro expansion cursor.
 // CHECK-TOK: Punctuation: "(" [4:21 - 4:22] attribute(iboutletcollection)= [IBOutletCollection=ObjCInterface]
 // CHECK-TOK: Identifier: "Test" [4:22 - 4:26] ObjCClassRef=Test:3:12
 // FIXME: The following token should belong to the macro expansion cursor.
 // CHECK-TOK: Punctuation: ")" [4:26 - 4:27]
 // CHECK-TOK: Identifier: "Test" [4:28 - 4:32] ObjCClassRef=Test:3:12
 // CHECK-TOK: Punctuation: "*" [4:33 - 4:34] ObjCIvarDecl=anOutletCollection:4:34 (Definition)
 // CHECK-TOK: Identifier: "anOutletCollection" [4:34 - 4:52] ObjCIvarDecl=anOutletCollection:4:34 (Definition)

test/Index/annotate-macro-args.m

 // Test without PCH
-// RUN: c-index-test -test-annotate-tokens=%S/annotate-macro-args.h:9:1:10:1 %s -include %S/annotate-macro-args.h | FileCheck -check-prefix=CHECK1 %s
-// RUN: c-index-test -test-annotate-tokens=%S/annotate-macro-args.h:15:1:16:1 %s -include %S/annotate-macro-args.h | FileCheck -check-prefix=CHECK2 %s
+// RUN: c-index-test -test-annotate-tokens=%S/annotate-macro-args.h:9:1:10:1 -nostdinc  %s -include %S/annotate-macro-args.h | FileCheck -check-prefix=CHECK1 %s
+// RUN: c-index-test -test-annotate-tokens=%S/annotate-macro-args.h:15:1:16:1 -nostdinc  %s -include %S/annotate-macro-args.h | FileCheck -check-prefix=CHECK2 %s
 
 // Test with PCH
-// RUN: c-index-test -write-pch %t.pch -x objective-c-header %S/annotate-macro-args.h -Xclang -detailed-preprocessing-record
-// RUN: c-index-test -test-annotate-tokens=%S/annotate-macro-args.h:9:1:10:1 %s -include-pch %t.pch | FileCheck -check-prefix=CHECK1 %s
-// RUN: c-index-test -test-annotate-tokens=%S/annotate-macro-args.h:15:1:16:1 %s -include-pch %t.pch | FileCheck -check-prefix=CHECK2 %s
+// RUN: c-index-test -write-pch %t.pch -x objective-c-header %S/annotate-macro-args.h -Xclang -detailed-preprocessing-record -nostdinc
+// RUN: c-index-test -test-annotate-tokens=%S/annotate-macro-args.h:9:1:10:1 -nostdinc  %s -include-pch %t.pch | FileCheck -check-prefix=CHECK1 %s
+// RUN: c-index-test -test-annotate-tokens=%S/annotate-macro-args.h:15:1:16:1 -nostdinc  %s -include-pch %t.pch | FileCheck -check-prefix=CHECK2 %s
 
 // CHECK1: Identifier: "MACRO" [9:3 - 9:8] macro expansion=MACRO:6:9
 // CHECK1: Punctuation: "(" [9:8 - 9:9]
 // CHECK1: Punctuation: "[" [9:9 - 9:10] ObjCMessageExpr=meth:2:8
 // CHECK1: Identifier: "MyClass" [9:10 - 9:17] ObjCClassRef=MyClass:1:12
 // CHECK1: Identifier: "meth" [9:18 - 9:22] ObjCMessageExpr=meth:2:8
 // CHECK1: Punctuation: "]" [9:22 - 9:23] ObjCMessageExpr=meth:2:8
 // CHECK1: Punctuation: ")" [9:23 - 9:24]
 
 // CHECK2: Identifier: "INVOKE" [15:3 - 15:9] macro expansion=INVOKE:12:9
 // CHECK2: Punctuation: "(" [15:9 - 15:10]
 // CHECK2: Identifier: "meth" [15:10 - 15:14] ObjCMessageExpr=meth:2:8
 // CHECK2: Punctuation: "," [15:14 - 15:15]
 // CHECK2: Identifier: "MyClass" [15:16 - 15:23] ObjCClassRef=MyClass:1:12
 // CHECK2: Punctuation: ")" [15:23 - 15:24]

test/Index/annotate-tokens-pp.c

 
 struct A
 {
 #ifdef SOME_MACRO
   int x;
 #endif
 };
 
-// RUN: c-index-test -test-annotate-tokens=%s:2:1:44:1 -I%S/Inputs %s | FileCheck %s
-// RUN: env CINDEXTEST_EDITING=1 c-index-test -test-annotate-tokens=%s:2:1:44:1 -I%S/Inputs %s | FileCheck %s
+// RUN: c-index-test -test-annotate-tokens=%s:2:1:44:1 -I%S/Inputs -ffreestanding %s | FileCheck %s
+// RUN: env CINDEXTEST_EDITING=1 c-index-test -test-annotate-tokens=%s:2:1:44:1 -I%S/Inputs -ffreestanding %s | FileCheck %s
 // CHECK: Punctuation: "#" [2:1 - 2:2] preprocessing directive=
 // CHECK: Identifier: "define" [2:2 - 2:8] preprocessing directive=
 // CHECK: Identifier: "STILL_NOTHING" [2:9 - 2:22] macro definition=STILL_NOTHING
 // CHECK: Identifier: "NOTHING" [2:23 - 2:30] macro expansion=NOTHING:1:9
 // CHECK: Punctuation: "(" [2:30 - 2:31] macro definition=STILL_NOTHING
 // CHECK: Identifier: "honk" [2:31 - 2:35] macro definition=STILL_NOTHING
 // CHECK: Punctuation: "," [2:35 - 2:36] macro definition=STILL_NOTHING
 // CHECK: Identifier: "warble" [2:36 - 2:42] macro definition=STILL_NOTHING

test/Index/annotate-tokens.c

typedef union {
   };
 } r_t;
 
 void test() {
   r_t reg;
   reg.field = 1;
 }
 
-// RUN: c-index-test -test-annotate-tokens=%s:4:1:37:1 %s | FileCheck %s
+// RUN: c-index-test -test-annotate-tokens=%s:4:1:37:1 -ffreestanding %s | FileCheck %s
 // CHECK: Identifier: "T" [4:3 - 4:4] TypeRef=T:1:13
 // CHECK: Punctuation: "*" [4:4 - 4:5] VarDecl=t_ptr:4:6 (Definition)
 // CHECK: Identifier: "t_ptr" [4:6 - 4:11] VarDecl=t_ptr:4:6 (Definition)
 // CHECK: Punctuation: "=" [4:12 - 4:13] VarDecl=t_ptr:4:6 (Definition)
 // CHECK: Punctuation: "(" [4:14 - 4:15] CStyleCastExpr=
 // CHECK: Identifier: "T" [4:15 - 4:16] TypeRef=T:1:13
 // CHECK: Identifier: "ptr" [4:19 - 4:22] DeclRefExpr=ptr:3:14
 // CHECK: Punctuation: ";" [4:22 - 4:23] DeclStmt=

 // CHECK: Keyword: "__attribute__" [36:70 - 36:83] FunctionDecl=test:36:63 (unavailable)  (always unavailable: "")
 // CHECK: Punctuation: "(" [36:83 - 36:84] FunctionDecl=test:36:63 (unavailable)  (always unavailable: "")
 // CHECK: Punctuation: "(" [36:84 - 36:85] FunctionDecl=test:36:63 (unavailable)  (always unavailable: "")
 // CHECK: Identifier: "unavailable" [36:85 - 36:96] UnexposedAttr=
 // CHECK: Punctuation: ")" [36:96 - 36:97] FunctionDecl=test:36:63 (unavailable)  (always unavailable: "")
 // CHECK: Punctuation: ")" [36:97 - 36:98] FunctionDecl=test:36:63 (unavailable)  (always unavailable: "")
 // CHECK: Punctuation: ";" [36:98 - 36:99]
 
-// RUN: c-index-test -test-annotate-tokens=%s:4:1:165:32 %s | FileCheck %s
-// RUN: c-index-test -test-annotate-tokens=%s:4:1:165:38 %s | FileCheck %s
+// RUN: c-index-test -test-annotate-tokens=%s:4:1:165:32 -ffreestanding %s | FileCheck %s
+// RUN: c-index-test -test-annotate-tokens=%s:4:1:165:38 -ffreestanding %s | FileCheck %s
 
-// RUN: c-index-test -test-annotate-tokens=%s:50:1:55:1 %s | FileCheck %s -check-prefix=CHECK-RANGE1
+// RUN: c-index-test -test-annotate-tokens=%s:50:1:55:1 -ffreestanding %s | FileCheck %s -check-prefix=CHECK-RANGE1
 // CHECK-RANGE1: Keyword: "void" [50:1 - 50:5] FunctionDecl=func1:50:6
 // CHECK-RANGE1: Identifier: "func1" [50:6 - 50:11] FunctionDecl=func1:50:6
 // CHECK-RANGE1: Punctuation: "(" [50:11 - 50:12] FunctionDecl=func1:50:6
 // CHECK-RANGE1: Keyword: "void" [50:12 - 50:16] FunctionDecl=func1:50:6
 // CHECK-RANGE1: Punctuation: ")" [50:16 - 50:17] FunctionDecl=func1:50:6
 // CHECK-RANGE1: Punctuation: ";" [50:17 - 50:18]
 // CHECK-RANGE1: Identifier: "TYPE_INST" [52:1 - 52:10] macro expansion=TYPE_INST:43:9
 // CHECK-RANGE1: Punctuation: "(" [52:10 - 52:11]
 // CHECK-RANGE1: Identifier: "Foo" [52:11 - 52:14] VarDecl=Foo:52:11 (Definition)
 // CHECK-RANGE1: Punctuation: "," [52:14 - 52:15]
 // CHECK-RANGE1: Punctuation: "." [53:5 - 53:6] UnexposedExpr=
 // CHECK-RANGE1: Identifier: "x" [53:6 - 53:7] MemberRef=x:52:1
 // CHECK-RANGE1: Punctuation: "=" [53:8 - 53:9] UnexposedExpr=
 // CHECK-RANGE1: Literal: "0" [53:10 - 53:11] IntegerLiteral=
 // CHECK-RANGE1: Punctuation: "," [53:11 - 53:12] InitListExpr=
 // CHECK-RANGE1: Punctuation: "." [54:5 - 54:6] UnexposedExpr=
 // CHECK-RANGE1: Identifier: "y" [54:6 - 54:7] MemberRef=y:52:1
 // CHECK-RANGE1: Punctuation: "=" [54:8 - 54:9] UnexposedExpr=
 // CHECK-RANGE1: Literal: "1" [54:10 - 54:11] IntegerLiteral=
 // CHECK-RANGE1: Punctuation: "," [54:11 - 54:12] InitListExpr=
 
-// RUN: c-index-test -test-annotate-tokens=%s:54:1:70:1 %s | FileCheck %s -check-prefix=CHECK-RANGE2
+// RUN: c-index-test -test-annotate-tokens=%s:54:1:70:1 -ffreestanding %s | FileCheck %s -check-prefix=CHECK-RANGE2
 // CHECK-RANGE2: Punctuation: "." [54:5 - 54:6] UnexposedExpr=
 // CHECK-RANGE2: Identifier: "y" [54:6 - 54:7] MemberRef=y:52:1
 // CHECK-RANGE2: Punctuation: "=" [54:8 - 54:9] UnexposedExpr=
 // CHECK-RANGE2: Literal: "1" [54:10 - 54:11] IntegerLiteral=
 // CHECK-RANGE2: Punctuation: "," [54:11 - 54:12] InitListExpr=
 // CHECK-RANGE2: Punctuation: "." [55:5 - 55:6] UnexposedExpr=
 // CHECK-RANGE2: Identifier: "z" [55:6 - 55:7] MemberRef=z:52:1
 // CHECK-RANGE2: Punctuation: "=" [55:8 - 55:9] UnexposedExpr=
 // CHECK-RANGE2: Literal: "2" [55:10 - 55:11] IntegerLiteral=
 // CHECK-RANGE2: Punctuation: "," [55:11 - 55:12] InitListExpr=
 // CHECK-RANGE2: Punctuation: ")" [56:1 - 56:2]
 // CHECK-RANGE2: Punctuation: ";" [56:2 - 56:3]
 // CHECK-RANGE2: Keyword: "void" [58:1 - 58:5] FunctionDecl=func2:58:6
 // CHECK-RANGE2: Identifier: "func2" [58:6 - 58:11] FunctionDecl=func2:58:6
 // CHECK-RANGE2: Punctuation: "(" [58:11 - 58:12] FunctionDecl=func2:58:6
 // CHECK-RANGE2: Keyword: "void" [58:12 - 58:16] FunctionDecl=func2:58:6
 // CHECK-RANGE2: Punctuation: ")" [58:16 - 58:17] FunctionDecl=func2:58:6
 // CHECK-RANGE2: Punctuation: ";" [58:17 - 58:18]
 
 // CHECK-RANGE2: Identifier: "reg" [68:3 - 68:6] DeclRefExpr=reg:67:7
 // CHECK-RANGE2: Punctuation: "." [68:6 - 68:7] MemberRefExpr=field:62:9
 // CHECK-RANGE2: Identifier: "field" [68:7 - 68:12] MemberRefExpr=field:62:9
 
-// RUN: c-index-test -test-annotate-tokens=%s:68:15:68:16 %s | FileCheck %s -check-prefix=CHECK-RANGE3
+// RUN: c-index-test -test-annotate-tokens=%s:68:15:68:16 -ffreestanding %s | FileCheck %s -check-prefix=CHECK-RANGE3
 // CHECK-RANGE3: Literal: "1" [68:15 - 68:16] IntegerLiteral=
 // CHECK-RANGE3-NOT: Punctuation: ";"

test/Index/c-index-getCursor-test.m

-// RUN: c-index-test -write-pch %t.ast -arch x86_64 -mmacosx-version-min=10.6 -fblocks -x objective-c %s
+// RUN: c-index-test -write-pch %t.ast -arch x86_64 -mmacosx-version-min=10.6 -fblocks -x objective-c -nostdinc %s
 // RUN: c-index-test -test-file-scan %t.ast %s > %t 2>&1 && FileCheck --input-file=%t %s
 @interface Foo 
 {
 }
 
 - foo;
 + fooC;
 

test/Index/get-cursor-macro-args.m

 // Test without PCH
 // RUN: c-index-test -cursor-at=%S/get-cursor-macro-args.h:9:12 \
 // RUN:              -cursor-at=%S/get-cursor-macro-args.h:9:21 \
 // RUN:              -cursor-at=%S/get-cursor-macro-args.h:9:9 \
 // RUN:              -cursor-at=%S/get-cursor-macro-args.h:9:22 \
 // RUN:              -cursor-at=%S/get-cursor-macro-args.h:15:12 \
 // RUN:              -cursor-at=%S/get-cursor-macro-args.h:15:20 \
-// RUN:       %s -include %S/get-cursor-macro-args.h | FileCheck %s
+// RUN:       -nostdinc %s -include %S/get-cursor-macro-args.h | FileCheck %s
 
 // Test with PCH
-// RUN: c-index-test -write-pch %t.pch -x objective-c-header %S/get-cursor-macro-args.h
+// RUN: c-index-test -write-pch %t.pch -x objective-c-header -nostdinc %S/get-cursor-macro-args.h
 // RUN: c-index-test -cursor-at=%S/get-cursor-macro-args.h:9:12 \
 // RUN:              -cursor-at=%S/get-cursor-macro-args.h:9:21 \
 // RUN:              -cursor-at=%S/get-cursor-macro-args.h:9:9 \
 // RUN:              -cursor-at=%S/get-cursor-macro-args.h:9:22 \
 // RUN:              -cursor-at=%S/get-cursor-macro-args.h:15:12 \
 // RUN:              -cursor-at=%S/get-cursor-macro-args.h:15:20 \
-// RUN:       %s -include-pch %t.pch | FileCheck %s
+// RUN:       -nostdinc %s -include-pch %t.pch | FileCheck %s
 
 // CHECK:      ObjCClassRef=MyClass:1:12
 // CHECK-NEXT: ObjCMessageExpr=meth:2:8
 // CHECK-NEXT: ObjCMessageExpr=meth:2:8
 // CHECK-NEXT: ObjCMessageExpr=meth:2:8
 // CHECK-NEXT: ObjCMessageExpr=meth:2:8
 // CHECK-NEXT: ObjCClassRef=MyClass:1:12

test/Index/get-cursor.cpp

 void test(TestColl coll) {
   for (auto lv : coll) {
     (void)lv;
   }
 }
 
 const int operator""_toint(unsigned long long val) { return int(val); }
 
-// RUN: c-index-test -cursor-at=%s:6:4 %s | FileCheck -check-prefix=CHECK-COMPLETION-1 %s
+// RUN: c-index-test -cursor-at=%s:6:4 -ffreestanding %s | FileCheck -check-prefix=CHECK-COMPLETION-1 %s
 // CHECK-COMPLETION-1: CXXConstructor=X:6:3
 // CHECK-COMPLETION-1-NEXT: Completion string: {TypedText X}{LeftParen (}{Placeholder int}{Comma , }{Placeholder int}{RightParen )}
 
-// RUN: c-index-test -cursor-at=%s:31:16 %s | FileCheck -check-prefix=CHECK-COMPLETION-2 %s
+// RUN: c-index-test -cursor-at=%s:31:16 -ffreestanding %s | FileCheck -check-prefix=CHECK-COMPLETION-2 %s
 // CHECK-COMPLETION-2: CXXMethod=getAnotherX:31:5 (Definition)
 // CHECK-COMPLETION-2-NEXT: Completion string: {ResultType X}{TypedText getAnotherX}{LeftParen (}{RightParen )}
 
-// RUN: c-index-test -cursor-at=%s:12:20 %s | FileCheck -check-prefix=CHECK-VALUE-REF %s
-// RUN: c-index-test -cursor-at=%s:13:21 %s | FileCheck -check-prefix=CHECK-VALUE-REF %s
-// RUN: c-index-test -cursor-at=%s:13:28 %s | FileCheck -check-prefix=CHECK-VALUE-REF %s
-// RUN: c-index-test -cursor-at=%s:14:23 %s | FileCheck -check-prefix=CHECK-VALUE-REF %s
+// RUN: c-index-test -cursor-at=%s:12:20 -ffreestanding %s | FileCheck -check-prefix=CHECK-VALUE-REF %s
+// RUN: c-index-test -cursor-at=%s:13:21 -ffreestanding %s | FileCheck -check-prefix=CHECK-VALUE-REF %s
+// RUN: c-index-test -cursor-at=%s:13:28 -ffreestanding %s | FileCheck -check-prefix=CHECK-VALUE-REF %s
+// RUN: c-index-test -cursor-at=%s:14:23 -ffreestanding %s | FileCheck -check-prefix=CHECK-VALUE-REF %s
 // CHECK-VALUE-REF: DeclRefExpr=value:10:12
 
-// RUN: c-index-test -cursor-at=%s:12:18 %s | FileCheck -check-prefix=CHECK-CONSTRUCTOR1 %s
-// RUN: c-index-test -cursor-at=%s:13:18 %s | FileCheck -check-prefix=CHECK-CONSTRUCTOR2 %s
-// RUN: c-index-test -cursor-at=%s:14:19 %s | FileCheck -check-prefix=CHECK-CONSTRUCTOR1 %s
-// RUN: c-index-test -cursor-at=%s:17:10 %s | FileCheck -check-prefix=CHECK-CONSTRUCTOR3 %s
+// RUN: c-index-test -cursor-at=%s:12:18 -ffreestanding %s | FileCheck -check-prefix=CHECK-CONSTRUCTOR1 %s
+// RUN: c-index-test -cursor-at=%s:13:18 -ffreestanding %s | FileCheck -check-prefix=CHECK-CONSTRUCTOR2 %s
+// RUN: c-index-test -cursor-at=%s:14:19 -ffreestanding %s | FileCheck -check-prefix=CHECK-CONSTRUCTOR1 %s
+// RUN: c-index-test -cursor-at=%s:17:10 -ffreestanding %s | FileCheck -check-prefix=CHECK-CONSTRUCTOR3 %s
 // CHECK-TYPE-REF: TypeRef=struct X:3:8
 // CHECK-CONSTRUCTOR1: CallExpr=X:5:3
 // CHECK-CONSTRUCTOR2: CallExpr=X:6:3
 // CHECK-CONSTRUCTOR3: CallExpr=X:4:3
 
-// RUN: c-index-test -cursor-at=%s:23:3 %s | FileCheck -check-prefix=CHECK-RETTYPE %s
-// RUN: c-index-test -cursor-at=%s:26:1 %s | FileCheck -check-prefix=CHECK-RETTYPE %s
+// RUN: c-index-test -cursor-at=%s:23:3 -ffreestanding %s | FileCheck -check-prefix=CHECK-RETTYPE %s
+// RUN: c-index-test -cursor-at=%s:26:1 -ffreestanding %s | FileCheck -check-prefix=CHECK-RETTYPE %s
 // CHECK-RETTYPE: TypeRef=struct X:3:8
 
-// RUN: c-index-test -cursor-at=%s:23:7 %s | FileCheck -check-prefix=CHECK-MEMFUNC-DECL %s
+// RUN: c-index-test -cursor-at=%s:23:7 -ffreestanding %s | FileCheck -check-prefix=CHECK-MEMFUNC-DECL %s
 // CHECK-MEMFUNC-DECL: CXXMethod=getX:23:5
-// RUN: c-index-test -cursor-at=%s:26:7 %s | FileCheck -check-prefix=CHECK-MEMFUNC-DEF %s
+// RUN: c-index-test -cursor-at=%s:26:7 -ffreestanding %s | FileCheck -check-prefix=CHECK-MEMFUNC-DEF %s
 // CHECK-MEMFUNC-DEF: CXXMethod=getX:26:6
 
-// RUN: c-index-test -cursor-at=%s:26:3 %s | FileCheck -check-prefix=CHECK-TYPEREF-Y %s
+// RUN: c-index-test -cursor-at=%s:26:3 -ffreestanding %s | FileCheck -check-prefix=CHECK-TYPEREF-Y %s
 // CHECK-TYPEREF-Y: TypeRef=struct Y:20:8
 
-// RUN: c-index-test -cursor-at=%s:27:10 %s | FileCheck -check-prefix=CHECK-IMPLICIT-MEMREF %s
-// RUN: c-index-test -cursor-at=%s:31:28 %s | FileCheck -check-prefix=CHECK-IMPLICIT-MEMREF %s
+// RUN: c-index-test -cursor-at=%s:27:10 -ffreestanding %s | FileCheck -check-prefix=CHECK-IMPLICIT-MEMREF %s
+// RUN: c-index-test -cursor-at=%s:31:28 -ffreestanding %s | FileCheck -check-prefix=CHECK-IMPLICIT-MEMREF %s
 // CHECK-IMPLICIT-MEMREF: MemberRefExpr=member:21:7
 
-// RUN: c-index-test -cursor-at=%s:35:5 %s | FileCheck -check-prefix=CHECK-DECL %s
+// RUN: c-index-test -cursor-at=%s:35:5 -ffreestanding %s | FileCheck -check-prefix=CHECK-DECL %s
 // CHECK-DECL: VarDecl=foo:35:5
 
-// RUN: c-index-test -cursor-at=%s:21:3 %s | FileCheck -check-prefix=CHECK-MEMBER %s
+// RUN: c-index-test -cursor-at=%s:21:3 -ffreestanding %s | FileCheck -check-prefix=CHECK-MEMBER %s
 // CHECK-MEMBER: FieldDecl=member:21:7 (Definition)
 // CHECK-MEMBER-NEXT: Completion string: {ResultType int}{TypedText member}
 
-// RUN: c-index-test -cursor-at=%s:38:12 -cursor-at=%s:39:5 %s | FileCheck -check-prefix=CHECK-CXXCATCH %s
+// RUN: c-index-test -cursor-at=%s:38:12 -cursor-at=%s:39:5 -ffreestanding %s | FileCheck -check-prefix=CHECK-CXXCATCH %s
 // CHECK-CXXCATCH: TypeRef=struct X:3:8
 // CHECK-CXXCATCH-NEXT: TypeRef=struct X:3:8
 
-// RUN: c-index-test -test-load-source-usrs local %s | FileCheck -check-prefix=CHECK-USR %s
+// RUN: c-index-test -test-load-source-usrs local -ffreestanding %s | FileCheck -check-prefix=CHECK-USR %s
 // CHECK-USR: get-cursor.cpp c:get-cursor.cpp@472@F@test#@e Extent=[38:12 - 38:15]
 // CHECK-USR: get-cursor.cpp c:get-cursor.cpp@483@F@test#@x Extent=[39:5 - 39:8]
 
-// RUN: c-index-test -cursor-at=%s:45:9 %s | FileCheck -check-prefix=CHECK-LOCALCLASS %s
+// RUN: c-index-test -cursor-at=%s:45:9 -ffreestanding %s | FileCheck -check-prefix=CHECK-LOCALCLASS %s
 // CHECK-LOCALCLASS: 45:9 DeclRefExpr=x:44:11 Extent=[45:9 - 45:10] Spelling=x ([45:9 - 45:10])
 
-// RUN: c-index-test -cursor-at=%s:50:23 -cursor-at=%s:55:23 %s | FileCheck -check-prefix=CHECK-TEMPLPARAM %s
+// RUN: c-index-test -cursor-at=%s:50:23 -cursor-at=%s:55:23 -ffreestanding %s | FileCheck -check-prefix=CHECK-TEMPLPARAM %s
 // CHECK-TEMPLPARAM: 50:23 TypeRef=struct X:3:8 Extent=[50:23 - 50:24] Spelling=struct X ([50:23 - 50:24])
 // CHECK-TEMPLPARAM: 55:23 TypeRef=struct X:3:8 Extent=[55:23 - 55:24] Spelling=struct X ([55:23 - 55:24])
 
-// RUN: c-index-test -cursor-at=%s:66:23 %s | FileCheck -check-prefix=CHECK-TEMPLSPEC %s
+// RUN: c-index-test -cursor-at=%s:66:23 -ffreestanding %s | FileCheck -check-prefix=CHECK-TEMPLSPEC %s
 // CHECK-TEMPLSPEC: 66:23 ClassDecl=TC:66:23 (Definition) [Specialization of TC:59:7] Extent=[66:1 - 66:31] Spelling=TC ([66:23 - 66:25])
 
-// RUN: c-index-test -cursor-at=%s:69:3 -cursor-at=%s:70:11 -cursor-at=%s:73:6 -cursor-at=%s:74:6 -cursor-at=%s:77:8 -cursor-at=%s:78:8 -cursor-at=%s:79:8 -cursor-at=%s:80:8 -cursor-at=%s:81:8 -cursor-at=%s:82:8 -cursor-at=%s:85:6 -cursor-at=%s:86:6 -cursor-at=%s:87:6 -cursor-at=%s:88:6 -cursor-at=%s:91:5 -cursor-at=%s:92:5 -cursor-at=%s:93:5 -cursor-at=%s:94:5 -cursor-at=%s:95:5 -cursor-at=%s:96:5 -cursor-at=%s:97:5 -cursor-at=%s:98:5 -cursor-at=%s:100:5 -cursor-at=%s:101:5 -cursor-at=%s:104:6 -cursor-at=%s:105:6 -cursor-at=%s:106:6 -cursor-at=%s:107:6 -cursor-at=%s:108:6 -cursor-at=%s:109:6 -cursor-at=%s:110:6 -cursor-at=%s:111:6 -cursor-at=%s:113:6 -cursor-at=%s:114:6 -cursor-at=%s:117:8 -cursor-at=%s:118:8 -cursor-at=%s:120:8 -cursor-at=%s:121:8 -cursor-at=%s:122:8 -cursor-at=%s:123:8 -cursor-at=%s:124:8 -cursor-at=%s:125:8 -cursor-at=%s:128:6 -cursor-at=%s:129:6 -cursor-at=%s:130:6 -cursor-at=%s:132:3 -cursor-at=%s:146:15 -std=c++11 %s | FileCheck -check-prefix=CHECK-SPELLING %s
+// RUN: c-index-test -cursor-at=%s:69:3 -cursor-at=%s:70:11 -cursor-at=%s:73:6 -cursor-at=%s:74:6 -cursor-at=%s:77:8 -cursor-at=%s:78:8 -cursor-at=%s:79:8 -cursor-at=%s:80:8 -cursor-at=%s:81:8 -cursor-at=%s:82:8 -cursor-at=%s:85:6 -cursor-at=%s:86:6 -cursor-at=%s:87:6 -cursor-at=%s:88:6 -cursor-at=%s:91:5 -cursor-at=%s:92:5 -cursor-at=%s:93:5 -cursor-at=%s:94:5 -cursor-at=%s:95:5 -cursor-at=%s:96:5 -cursor-at=%s:97:5 -cursor-at=%s:98:5 -cursor-at=%s:100:5 -cursor-at=%s:101:5 -cursor-at=%s:104:6 -cursor-at=%s:105:6 -cursor-at=%s:106:6 -cursor-at=%s:107:6 -cursor-at=%s:108:6 -cursor-at=%s:109:6 -cursor-at=%s:110:6 -cursor-at=%s:111:6 -cursor-at=%s:113:6 -cursor-at=%s:114:6 -cursor-at=%s:117:8 -cursor-at=%s:118:8 -cursor-at=%s:120:8 -cursor-at=%s:121:8 -cursor-at=%s:122:8 -cursor-at=%s:123:8 -cursor-at=%s:124:8 -cursor-at=%s:125:8 -cursor-at=%s:128:6 -cursor-at=%s:129:6 -cursor-at=%s:130:6 -cursor-at=%s:132:3 -cursor-at=%s:146:15 -std=c++11 -ffreestanding %s | FileCheck -check-prefix=CHECK-SPELLING %s
 // CHECK-SPELLING: 69:3 CXXConstructor=A:69:3 (default constructor) Extent=[69:3 - 69:6] Spelling=A ([69:3 - 69:4])
 // CHECK-SPELLING: 70:11 CXXDestructor=~A:70:11 (virtual) Extent=[70:3 - 70:15] Spelling=~A ([70:11 - 70:13])
 // CHECK-SPELLING: 73:6 CXXMethod=operator=:73:6 Extent=[73:3 - 73:25] Spelling=operator= ([73:6 - 73:15])
 // CHECK-SPELLING: 74:6 CXXMethod=operator=:74:6 Extent=[74:3 - 74:29] Spelling=operator= ([74:6 - 74:15])
 // CHECK-SPELLING: 77:8 CXXMethod=operator+:77:8 (const) Extent=[77:3 - 77:25] Spelling=operator+ ([77:8 - 77:17])
 // CHECK-SPELLING: 78:8 CXXMethod=operator-:78:8 (const) Extent=[78:3 - 78:25] Spelling=operator- ([78:8 - 78:17])
 // CHECK-SPELLING: 79:8 CXXMethod=operator~:79:8 (const) Extent=[79:3 - 79:25] Spelling=operator~ ([79:8 - 79:17])
 // CHECK-SPELLING: 80:8 CXXMethod=operator*:80:8 (const) Extent=[80:3 - 80:25] Spelling=operator* ([80:8 - 80:17])

 // CHECK-SPELLING: 124:8 CXXMethod=operator!=:124:8 (const) Extent=[124:3 - 124:34] Spelling=operator!= ([124:8 - 124:18])
 // CHECK-SPELLING: 125:8 CXXMethod=operator==:125:8 (const) Extent=[125:3 - 125:34] Spelling=operator== ([125:8 - 125:18])
 // CHECK-SPELLING: 128:6 CXXMethod=operator[]:128:6 Extent=[128:3 - 128:36] Spelling=operator[] ([128:6 - 128:16])
 // CHECK-SPELLING: 129:6 CXXMethod=operator->:129:6 Extent=[129:3 - 129:18] Spelling=operator-> ([129:6 - 129:16])
 // CHECK-SPELLING: 130:6 CXXMethod=operator():130:6 (const) Extent=[130:3 - 130:37] Spelling=operator() ([130:6 - 130:16])
 // CHECK-SPELLING: 132:12 CXXConversion=operator bool:132:12 (const) Extent=[132:3 - 132:33] Spelling=operator bool ([132:12 - 132:25])
 // CHECK-SPELLING: 146:11 FunctionDecl=operator""_toint:146:11 (Definition) Extent=[146:1 - 146:72] Spelling=operator""_toint ([146:11 - 146:27])
 
-// RUN: c-index-test -cursor-at=%s:141:13 -cursor-at=%s:141:18 -cursor-at=%s:142:11 -std=c++11 %s | FileCheck -check-prefix=CHECK-FORRANGE %s
+// RUN: c-index-test -cursor-at=%s:141:13 -cursor-at=%s:141:18 -cursor-at=%s:142:11 -std=c++11 -ffreestanding %s | FileCheck -check-prefix=CHECK-FORRANGE %s
 // CHECK-FORRANGE: 141:13 VarDecl=lv:141:13 (Definition) Extent=[141:8 - 141:17] Spelling=lv ([141:13 - 141:15])
 // CHECK-FORRANGE: 141:18 DeclRefExpr=coll:140:20 Extent=[141:18 - 141:22] Spelling=coll ([141:18 - 141:22])
 // CHECK-FORRANGE: 142:11 DeclRefExpr=lv:141:13 Extent=[142:11 - 142:13] Spelling=lv ([142:11 - 142:13])

unittests/Tooling/TestVisitor.h

enum Language {
     Lang_OBJC,
     Lang_OBJCXX11,
     Lang_CXX = Lang_CXX98
   };
 
   /// \brief Runs the current AST visitor over the given code.
   bool runOver(StringRef Code, Language L = Lang_CXX) {
     std::vector<std::string> Args;
+    Args.push_back("-nostdinc");
     switch (L) {
       case Lang_C: Args.push_back("-std=c99"); break;
       case Lang_CXX98: Args.push_back("-std=c++98"); break;
       case Lang_CXX11: Args.push_back("-std=c++11"); break;
       case Lang_CXX14: Args.push_back("-std=c++14"); break;
       case Lang_OBJC: Args.push_back("-ObjC"); break;
       case Lang_OBJCXX11:
         Args.push_back("-ObjC++");