PR21174 - clang only searches current working directory for precompiled include file

[nikola]

Summary

Seems that gcc looks for precompiled header in all include directories. Should we be doing this for all extensions?

[kimgr]

My co-worker (who reported the bug) has tested the patch and it works in our build system, so the patch is functional.

I was about to suggest adding the same logic for Clang PCH/PTH, but I see that the .gch handling is considered a special case. So maybe it makes sense to keep it this way for GCC compat only.

Are the PCH paths generally assumed to be absolute?

[kimgr]

Also, is a test possible? I still haven't got my head around the Clang test fx, but it feels like it should be possible to create a .gch in a temporary path, add the path to -I and verify that clang sees it when you attempt to force it in with -include.

[nikola]

Added test.

[nikola]

Ping.

[kimgr]

Thanks! I just looked at this again, and I think I found a bug, see inline.

I can't sign this off, I'm just a happy user, so here's a friendly owner ping!

lib/Driver/Tools.cpp

398

Add a break; here so we don't continue searching after a valid path has been found

[rsmith]

lib/Driver/Tools.cpp

398

Does GCC do this just for the include paths specified on the command line, in the order as written, or should this be using the full set of header search paths?

[kimgr]

lib/Driver/Tools.cpp

398

The GCC docs here [1] say:

If not found there, it is searched for in the remainder of the #include "..." search chain as normal.

I can't tell if the quotes are significant and if they mean only -I is searched. I don't have a GCC environment currently to test with.

[1] https://gcc.gnu.org/onlinedocs/gcc-4.9.3/gcc/Preprocessor-Options.html#index-nostdinc_002b_002b-1026

[kimgr]

lib/Driver/Tools.cpp

398

I found a FreeBSD machine and set up GCC. I had to adjust the repro case to use truss instead of strace, so I hope I didn't mess anything up.

I removed the rule generating the .gch file, so GCC would have to keep searching and it appears to be following the entire include search path, including system paths. Trace below:

...
30860: stat("./precompiled.header.gch",0x7fffffffe540) ERR#2 'No such file or directory'
30860: open("./precompiled.header",O_NOCTTY,0666) ERR#2 'No such file or directory'
30860: stat("./build/include/precompiled.header.gch",0x7fffffffe540) ERR#2 'No such file or directory'
30860: open("./build/include/precompiled.header",O_NOCTTY,0666) ERR#2 'No such file or directory'
30860: stat("./include/precompiled.header.gch",0x7fffffffe540) ERR#2 'No such file or directory'
30860: open("./include/precompiled.header",O_NOCTTY,0666) ERR#2 'No such file or directory'
30860: lstat("/usr",{ mode=drwxr-xr-x ,inode=9,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/local",{ mode=drwxr-xr-x ,inode=115,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/local/lib",{ mode=drwxr-xr-x ,inode=16476,size=1318,blksize=84480 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48",{ mode=drwxr-xr-x ,inode=24335,size=66,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/include",{ mode=drwxr-xr-x ,inode=24336,size=3,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/include/c++",{ mode=drwxr-xr-x ,inode=24337,size=98,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/include/c++/precompiled.header",0x7fffffffd3d8) ERR#2 'No such file or directory'
30860: stat("/usr/local/lib/gcc48/include/c++/precompiled.header.gch",0x7fffffffe540) ERR#2 'No such file or directory'
30860: open("/usr/local/lib/gcc48/include/c++/precompiled.header",O_NOCTTY,0666) ERR#2 'No such file or directory'
30860: lstat("/usr",{ mode=drwxr-xr-x ,inode=9,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/local",{ mode=drwxr-xr-x ,inode=115,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/local/lib",{ mode=drwxr-xr-x ,inode=16476,size=1318,blksize=84480 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48",{ mode=drwxr-xr-x ,inode=24335,size=66,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/include",{ mode=drwxr-xr-x ,inode=24336,size=3,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/include/c++",{ mode=drwxr-xr-x ,inode=24337,size=98,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/include/c++/x86_64-portbld-freebsd10.1",{ mode=drwxr-xr-x ,inode=21305,size=4,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/include/c++/x86_64-portbld-freebsd10.1/precompiled.header",0x7fffffffd3d8) ERR#2 'No such file or directory'
30860: stat("/usr/local/lib/gcc48/include/c++x86_64-portbld-freebsd10.1/precompiled.header.gch",0x7fffffffe540) ERR#2 'No such file or directory'
30860: open("/usr/local/lib/gcc48/include/c++x86_64-portbld-freebsd10.1/precompiled.header",O_NOCTTY,0666) ERR#2 'No such file or directory'
30860: lstat("/usr",{ mode=drwxr-xr-x ,inode=9,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/local",{ mode=drwxr-xr-x ,inode=115,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/local/lib",{ mode=drwxr-xr-x ,inode=16476,size=1318,blksize=84480 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48",{ mode=drwxr-xr-x ,inode=24335,size=66,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/include",{ mode=drwxr-xr-x ,inode=24336,size=3,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/include/c++",{ mode=drwxr-xr-x ,inode=24337,size=98,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/include/c++/backward",{ mode=drwxr-xr-x ,inode=21290,size=10,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/include/c++/backward/precompiled.header",0x7fffffffd3d8) ERR#2 'No such file or directory'
30860: stat("/usr/local/lib/gcc48/include/c++backward/precompiled.header.gch",0x7fffffffe540) ERR#2 'No such file or directory'
30860: open("/usr/local/lib/gcc48/include/c++backward/precompiled.header",O_NOCTTY,0666) ERR#2 'No such file or directory'
30860: lstat("/usr",{ mode=drwxr-xr-x ,inode=9,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/local",{ mode=drwxr-xr-x ,inode=115,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/local/lib",{ mode=drwxr-xr-x ,inode=16476,size=1318,blksize=84480 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48",{ mode=drwxr-xr-x ,inode=24335,size=66,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/gcc",{ mode=drwxr-xr-x ,inode=145103,size=3,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1",{ mode=drwxr-xr-x ,inode=145104,size=3,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1/4.8.4",{ mode=drwxr-xr-x ,inode=145105,size=16,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1/4.8.4/include",{ mode=drwxr-xr-x ,inode=145106,size=64,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1/4.8.4/include/precompiled.header",0x7fffffffd3d8) ERR#2 'No such file or directory'
30860: stat("/usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1/4.8.4/include/precompiled.header.gch",0x7fffffffe540) ERR#2 'No such file or directory'
30860: open("/usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1/4.8.4/include/precompiled.header",O_NOCTTY,0666) ERR#2 'No such file or directory'
30860: lstat("/usr",{ mode=drwxr-xr-x ,inode=9,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/local",{ mode=drwxr-xr-x ,inode=115,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/local/include",{ mode=drwxr-xr-x ,inode=16474,size=218,blksize=14336 }) = 0 (0x0)
30860: lstat("/usr/local/include/precompiled.header",0x7fffffffd3d8) ERR#2 'No such file or directory'
30860: stat("/usr/local/include/precompiled.header.gch",0x7fffffffe540) ERR#2 'No such file or directory'
30860: open("/usr/local/include/precompiled.header",O_NOCTTY,0666) ERR#2 'No such file or directory'
30860: lstat("/usr",{ mode=drwxr-xr-x ,inode=9,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/local",{ mode=drwxr-xr-x ,inode=115,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/local/lib",{ mode=drwxr-xr-x ,inode=16476,size=1318,blksize=84480 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48",{ mode=drwxr-xr-x ,inode=24335,size=66,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/gcc",{ mode=drwxr-xr-x ,inode=145103,size=3,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1",{ mode=drwxr-xr-x ,inode=145104,size=3,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1/4.8.4",{ mode=drwxr-xr-x ,inode=145105,size=16,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1/4.8.4/include-fixed",{ mode=drwxr-xr-x ,inode=145194,size=12,blksize=131072 }) = 0 (0x0)
30860: lstat("/usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1/4.8.4/include-fixed/precompiled.header",0x7fffffffd3d8) ERR#2 'No such file or directory'
30860: stat("/usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1/4.8.4/include-fixed/precompiled.header.gch",0x7fffffffe540) ERR#2 'No such file or directory'
30860: open("/usr/local/lib/gcc48/gcc/x86_64-portbld-freebsd10.1/4.8.4/include-fixed/precompiled.header",O_NOCTTY,0666) ERR#2 'No such file or directory'
30860: lstat("/usr",{ mode=drwxr-xr-x ,inode=9,size=16,blksize=4096 }) = 0 (0x0)
30860: lstat("/usr/include",{ mode=drwxr-xr-x ,inode=111,size=307,blksize=19968 }) = 0 (0x0)
30860: lstat("/usr/include/precompiled.header",0x7fffffffd3d8) ERR#2 'No such file or directory'
30860: stat("/usr/include/precompiled.header.gch",0x7fffffffe540) ERR#2 'No such file or directory'
30860: open("/usr/include/precompiled.header",O_NOCTTY,0666) ERR#2 'No such file or directory'
30860: stat("/usr/share/nls/C/libc.cat",0x7fffffffdf18) ERR#2 'No such file or directory'
30860: stat("/usr/share/nls/libc/C",0x7fffffffdf18) ERR#2 'No such file or directory'
30860: stat("/usr/local/share/nls/C/libc.cat",0x7fffffffdf18) ERR#2 'No such file or directory'
30860: stat("/usr/local/share/nls/libc/C",0x7fffffffdf18) ERR#2 'No such file or directory'
30860: write(2,"cc1plus: fatal error: precompile"...,68) = 68 (0x44)
30860: write(2,"compilation terminated.\n",24) = 24 (0x18)
...

[kimgr]

lib/Driver/Tools.cpp

398

So it seems that GCC *does* scan the entire header search path. Unfortunately, that information doesn't seem to be available to the driver (looks to be set up in CompilerInstance::createPreprocessor.

Should this logic for PCH/PTH scanning move out of the driver and into the frontend?

[thakis]

lib/Driver/Tools.cpp

398

gcc also checks if the gch file is a directory and if so finds the first file with matching language / defines etc. It also uses gch files for #included files instead of just -included files (https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html). I thought clang intentionally did none of this so it doesn't have to stat all these directories.

lib/Driver/Tools.cpp

         if (llvm::sys::fs::exists(P)) {
           FoundPCH = UsePCH;
           FoundPTH = !UsePCH;
+        } else {
+          for (const Arg *A : Args.filtered(options::OPT_I_Group)) {
+            SmallString<128> Path(A->getValue());
+            llvm::sys::path::append(Path, P);
+            if (llvm::sys::fs::exists(Path)) {
+              P = Path;
+              FoundPCH = UsePCH;
+              FoundPTH = !UsePCH;
+            }
+          }
         }
       }
 

test/PCH/additional-include.h

+
+extern int i;

test/PCH/additional-include.cpp

+// RUN: mkdir -p %t.h.gch
+// RUN: %clang -x c++-header %S/additional-include.h -o %t.h.gch/additional-include.h.gch 
+// RUN: %clang -x c++ -include additional-include.h -I %t.h.gch -fsyntax-only %s -Xclang -print-stats 2> %t.log
+// RUN: FileCheck %s < %t.log
+
+int foo() {
+  // CHECK: .h.gch{{[/\\]}}additional-include.h.gch
+  return i;
+}