This is an archive of the discontinued LLVM Phabricator instance.

[Driver] -###: exit with code 1 if hasErrorOccurred
ClosedPublic

Authored by MaskRay on Jul 26 2023, 1:34 PM.

Details

Summary

The exit code for -### is inconsistent. Unrecognized options lead to
exit code 1, as expected. However, most others errors (including invalid
option value) lead to exit code 0, differing from GCC and most utilities.

This is a longstanding quirk of -###, and we didn't fix it because many
driver tests need adjustment.

Change -### to be similar to -fdriver-only -v and exit with code 1.

This requires fixing many driver tests, but the end result gives us
stronger tests.

  • Existing RUN: %clang -### ... tests usually don't use CHECK-NOT: error: or --implicit-check-not=error:. If a change introduces an error, such a change usually cannot be detected.
  • Many folks contributing new tests don't know -fdriver-only -v. To test no driver error/warning for new tests, they can use the familiar -### -Werror instead of -fdriver-only -v -Werror.

An incomplete list of prerequisite test improvement:

Diff Detail

Event Timeline

MaskRay created this revision.Jul 26 2023, 1:34 PM
Herald added a project: Restricted Project. · View Herald Transcript
MaskRay requested review of this revision.Jul 26 2023, 1:34 PM
Herald added a project: Restricted Project. · View Herald Transcript

@jhuber6 @yaxunl you may want to revise some AMDGPU related tests, e.g. amdgcn-gz-options.cl.
Some RUN: %clang lines currently fail with error: cannot find ROCm device library; provide its path via '--rocm-path' or '--rocm-device-lib-path', or pass '-nogpulib' to build without ROCm device library.
They likely need -nogpulib to stop them from depending on the -### quirk (exit code 0), which is going to be fixed by this patch.

dblaikie accepted this revision.Jul 27 2023, 11:30 AM
dblaikie added a subscriber: dblaikie.

FWIW this sounds good to me (though given how wide the patch is, might be worth waiting a few days to a week in case anyone else has thoughts).

I only looked at a sample of the test changes - if there's anything much more interesting than adding not to any tests, might be worth calling those out in the description/review/etc - and maybe discuss what to do with nots that are added but because the test is buggy/mistaken. Them being explicit may confuse future users into thinking they're intentional, rather than pragmatic. Don't really know what to do about that - probably nothing.

This revision is now accepted and ready to land.Jul 27 2023, 11:30 AM
jhuber6 accepted this revision.Jul 27 2023, 11:31 AM

I'd wager a lot of the tests that return non-zero aren't even intentional, so it's probably good to enforce this.

yaxunl accepted this revision.Jul 27 2023, 11:54 AM

LGTM. It is good to be consistent with GCC. Most amdgpu and HIP tests fail due to missing -nogpuinc or -nogpulib. I can help update them.

FWIW this sounds good to me (though given how wide the patch is, might be worth waiting a few days to a week in case anyone else has thoughts).

I only looked at a sample of the test changes - if there's anything much more interesting than adding not to any tests, might be worth calling those out in the description/review/etc - and maybe discuss what to do with nots that are added but because the test is buggy/mistaken. Them being explicit may confuse future users into thinking they're intentional, rather than pragmatic. Don't really know what to do about that - probably nothing.

Thanks for the feedback! I used a script to automatically add not to failing %clang tests and manually fixed some env PATH=... %clang style tests that my script cannot handle.

I noticed debug-options.c had incorrect RUN lines from D54487 but fixed the test separately, and some AMDGPU tests might be better fixed by adding -nogpulib, otherwise there should likely be no tests that require specialized treatment.

This revision was landed with ongoing or failed builds.Jul 28 2023, 8:31 PM
This revision was automatically updated to reflect the committed changes.
MaskRay reopened this revision.Jul 29 2023, 6:30 PM
This revision is now accepted and ready to land.Jul 29 2023, 6:30 PM
MaskRay updated this revision to Diff 545394.Jul 29 2023, 6:30 PM

rebase. fix more existing tests

MaskRay updated this revision to Diff 545397.Jul 29 2023, 7:43 PM
MaskRay edited the summary of this revision. (Show Details)

rebase. mention some dependencies (various test misuses that have to be fixed first)

This revision was landed with ongoing or failed builds.Jul 29 2023, 8:04 PM
This revision was automatically updated to reflect the committed changes.
thakis added a subscriber: thakis.Jul 30 2023, 5:12 AM

http://45.33.8.238/linux/113921/step_7.txt is failing due to this. Is this something on my end?

test/Driver/fsanitize.c contains many %clang RUN lines. Presumably one somehow exits with code 1 , but without using llvm-lit -vv we don't know which command fails, making this almost impossible to investigate...

I guess adding --dump-input=fail to every FileCheck command will be useful as well but it would cluster up the history of fsanitize.c...

dyung added a comment.Jul 30 2023, 2:50 PM

If you cannot reproduce this issue, I can take my buildbot offline and try to reproduce it to pinpoint which command is causing the problem, but I won’t be able to do so until Monday.

We see 14 failing tests on our Linux and macOS builders, and 6 failing tests on Windows builders:

  • Clang :: Driver/csky-toolchain.c
  • Clang :: Driver/env.c
  • Clang :: Driver/fsanitize.c
  • Clang :: Driver/gcc-install-dir.cpp
  • Clang :: Driver/gcc-toolchain.cpp
  • Clang :: Driver/linux-cross.cpp
  • Clang :: Driver/linux-ld.c
  • Clang :: Driver/loongarch-toolchain.c
  • Clang :: Driver/miamcu-opt.c
  • Clang :: Driver/nolibc.c
  • Clang :: Driver/pic.c
  • Clang :: Driver/riscv32-toolchain.c
  • Clang :: Driver/riscv64-toolchain.c
  • Clang :: OpenMP/linking.c

Here is an example of failing builds:

Can we please revert this change while this is being investigated?

MaskRay added a comment.EditedJul 30 2023, 7:41 PM

We see 14 failing tests on our Linux and macOS builders, and 6 failing tests on Windows builders:

  • Clang :: Driver/csky-toolchain.c
  • Clang :: Driver/env.c
  • Clang :: Driver/fsanitize.c
  • Clang :: Driver/gcc-install-dir.cpp
  • Clang :: Driver/gcc-toolchain.cpp
  • Clang :: Driver/linux-cross.cpp
  • Clang :: Driver/linux-ld.c
  • Clang :: Driver/loongarch-toolchain.c
  • Clang :: Driver/miamcu-opt.c
  • Clang :: Driver/nolibc.c
  • Clang :: Driver/pic.c
  • Clang :: Driver/riscv32-toolchain.c
  • Clang :: Driver/riscv64-toolchain.c
  • Clang :: OpenMP/linking.c

Here is an example of failing builds:

Can we please revert this change while this is being investigated?

To the best of my knowledge, no official build exhibits these failures (except Driver/fsanitize.c).
These are non-official builds with quite complex builds. In the end I'd say I'd revert the patch, but my reasoning is different from: the patch broke the unofficial bot, so we justify to revert the patch.
It's more on the basis that these tests indicate issues that would likely happen on official build bots, if an official build bot had such coverage.

FWIW: I have tried -DCLANG_DEFAULT_CXX_STDLIB=libc++ -C clang/cmake/caches/Fuchsia-stage2.cmake -DLLVM_ENABLE_LTO=off to be similar to your build bots appear to be using (I am unfamiliar with it) but unable to reproduce the failures.

I think we need some assistance from Fuchsia to understand why these tests are less portable and what adjustment is needed to make them more portable.


With that said, clang/test/Driver/fsanitize.c is broken on some official build bots (e.g. clang-aarch64-sve-vla https://lab.llvm.org/buildbot/#/builders/197/builds/8677) and on http://45.33.8.238/ (I trust much for representing some configurations that an official build bot would cover), so I think a revert is justified, if we don't want to unsupport clang/test/Driver/fsanitize.c.

If you cannot reproduce this issue, I can take my buildbot offline and try to reproduce it to pinpoint which command is causing the problem, but I won’t be able to do so until Monday.

Thank you, knowing which command caused the failure will be very helpful.
clang/test/Driver/fsanitize.c has the most mysterious failure I have dealt with for this patch.

FWIW I've checked that the test passes even if /usr does not exist (emulated by proot) => there is as if no host GCC installation that could affect Driver's behavior

% cat /tmp/Rel/bin/xxx
#!/bin/zsh
d=$(dirname "$0")
exec proot -R /tmp -b /bin -b /lib64 -b /lib $d/clang --target=aarch64-linux-musl "$@"
% cd clang/test/Driver
% CLANG=/tmp/Rel/bin/xxx /tmp/Rel/bin/llvm-lit -vv fsanitize.c
.. passed
MaskRay reopened this revision.Jul 30 2023, 7:59 PM
This revision is now accepted and ready to land.Jul 30 2023, 7:59 PM
dyung added a comment.Jul 31 2023, 3:45 AM

If you cannot reproduce this issue, I can take my buildbot offline and try to reproduce it to pinpoint which command is causing the problem, but I won’t be able to do so until Monday.

Thank you, knowing which command caused the failure will be very helpful.
clang/test/Driver/fsanitize.c has the most mysterious failure I have dealt with for this patch.

FWIW I've checked that the test passes even if /usr does not exist (emulated by proot) => there is as if no host GCC installation that could affect Driver's behavior

% cat /tmp/Rel/bin/xxx
#!/bin/zsh
d=$(dirname "$0")
exec proot -R /tmp -b /bin -b /lib64 -b /lib $d/clang --target=aarch64-linux-musl "$@"
% cd clang/test/Driver
% CLANG=/tmp/Rel/bin/xxx /tmp/Rel/bin/llvm-lit -vv fsanitize.c
.. passed

I have to run, but I did a quick -vv run with this commit and it seems to fail on the RUN line on 588:

+ : 'RUN: at line 581'
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=i386-apple-ios-simulator -fsanitize=leak /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c -###
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c --check-prefix=CHECK-LSAN-I386-IOSSIMULATOR
+ : 'RUN: at line 584'
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=i386-apple-tvos-simulator -fsanitize=leak /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c -###
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c --check-prefix=CHECK-LSAN-I386-TVOSSIMULATOR
+ : 'RUN: at line 588'
+ not /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=x86_64-linux-gnu -fvisibility=hidden -fsanitize=cfi -flto -c /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c -###
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c --check-prefix=CHECK-CFI

--

********************
********************
Failed Tests (1):
  Clang :: Driver/fsanitize.c

I'll try to look more later when I get more time.

dyung added a comment.Jul 31 2023, 7:01 AM

If you cannot reproduce this issue, I can take my buildbot offline and try to reproduce it to pinpoint which command is causing the problem, but I won’t be able to do so until Monday.

Thank you, knowing which command caused the failure will be very helpful.
clang/test/Driver/fsanitize.c has the most mysterious failure I have dealt with for this patch.

FWIW I've checked that the test passes even if /usr does not exist (emulated by proot) => there is as if no host GCC installation that could affect Driver's behavior

% cat /tmp/Rel/bin/xxx
#!/bin/zsh
d=$(dirname "$0")
exec proot -R /tmp -b /bin -b /lib64 -b /lib $d/clang --target=aarch64-linux-musl "$@"
% cd clang/test/Driver
% CLANG=/tmp/Rel/bin/xxx /tmp/Rel/bin/llvm-lit -vv fsanitize.c
.. passed

I have to run, but I did a quick -vv run with this commit and it seems to fail on the RUN line on 588:

+ : 'RUN: at line 581'
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=i386-apple-ios-simulator -fsanitize=leak /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c -###
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c --check-prefix=CHECK-LSAN-I386-IOSSIMULATOR
+ : 'RUN: at line 584'
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=i386-apple-tvos-simulator -fsanitize=leak /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c -###
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c --check-prefix=CHECK-LSAN-I386-TVOSSIMULATOR
+ : 'RUN: at line 588'
+ not /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=x86_64-linux-gnu -fvisibility=hidden -fsanitize=cfi -flto -c /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c -###
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c --check-prefix=CHECK-CFI

--

********************
********************
Failed Tests (1):
  Clang :: Driver/fsanitize.c

I'll try to look more later when I get more time.

It appears that the clang command in the RUN line returns 0 which the not you added inverts it causing the test to fail:

buildbot@6a3c0c51ae89:~/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver$ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=x86_64-linux-gnu -fvisibility=hidden -fsanitize=cfi -flto -c /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize2.c -###
clang version 18.0.0 (https://github.com/llvm/llvm-project.git 8c3550b1a78fde7bf28f420da8447d9fde37017f)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin
 (in-process)
 "/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang-18" "-cc1" "-triple" "x86_64-unknown-linux-gnu" "-emit-llvm-bc" "-flto=full" "-flto-unit" "-disable-free" "-clear-ast-before-backend" "-main-file-name" "fsanitize2.c" "-mrelocation-model" "pic" "-pic-level" "2" "-pic-is-pie" "-mframe-pointer=all" "-fmath-errno" "-ffp-contract=on" "-fno-rounding-math" "-mconstructor-aliases" "-funwind-tables=2" "-target-cpu" "x86-64" "-tune-cpu" "generic" "-debugger-tuning=gdb" "-fcoverage-compilation-dir=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver" "-resource-dir" "/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/lib/clang/18" "-internal-isystem" "/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/lib/clang/18/include" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include" "-internal-externc-isystem" "/usr/include/x86_64-linux-gnu" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-fdebug-compilation-dir=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver" "-ferror-limit" "19" "-fvisibility=hidden" "-fsanitize=cfi-derived-cast,cfi-icall,cfi-mfcall,cfi-unrelated-cast,cfi-nvcall,cfi-vcall" "-fsanitize-trap=cfi-derived-cast,cfi-icall,cfi-mfcall,cfi-unrelated-cast,cfi-nvcall,cfi-vcall" "-fsanitize-system-ignorelist=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/lib/clang/18/share/cfi_ignorelist.txt" "-fno-sanitize-memory-param-retval" "-fsanitize-cfi-canonical-jump-tables" "-fno-sanitize-address-use-odr-indicator" "-fgnuc-version=4.2.1" "-fcolor-diagnostics" "-fsplit-lto-unit" "-faddrsig" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-o" "fsanitize2.o" "-x" "c" "/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize2.c"
buildbot@6a3c0c51ae89:~/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver$ echo $?
0
MaskRay added a comment.EditedJul 31 2023, 7:47 AM

If you cannot reproduce this issue, I can take my buildbot offline and try to reproduce it to pinpoint which command is causing the problem, but I won’t be able to do so until Monday.

Thank you, knowing which command caused the failure will be very helpful.
clang/test/Driver/fsanitize.c has the most mysterious failure I have dealt with for this patch.

FWIW I've checked that the test passes even if /usr does not exist (emulated by proot) => there is as if no host GCC installation that could affect Driver's behavior

% cat /tmp/Rel/bin/xxx
#!/bin/zsh
d=$(dirname "$0")
exec proot -R /tmp -b /bin -b /lib64 -b /lib $d/clang --target=aarch64-linux-musl "$@"
% cd clang/test/Driver
% CLANG=/tmp/Rel/bin/xxx /tmp/Rel/bin/llvm-lit -vv fsanitize.c
.. passed

I have to run, but I did a quick -vv run with this commit and it seems to fail on the RUN line on 588:

+ : 'RUN: at line 581'
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=i386-apple-ios-simulator -fsanitize=leak /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c -###
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c --check-prefix=CHECK-LSAN-I386-IOSSIMULATOR
+ : 'RUN: at line 584'
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=i386-apple-tvos-simulator -fsanitize=leak /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c -###
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c --check-prefix=CHECK-LSAN-I386-TVOSSIMULATOR
+ : 'RUN: at line 588'
+ not /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=x86_64-linux-gnu -fvisibility=hidden -fsanitize=cfi -flto -c /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c -###
+ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/FileCheck /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize.c --check-prefix=CHECK-CFI

--

********************
********************
Failed Tests (1):
  Clang :: Driver/fsanitize.c

I'll try to look more later when I get more time.

It appears that the clang command in the RUN line returns 0 which the not you added inverts it causing the test to fail:

buildbot@6a3c0c51ae89:~/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver$ /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang --target=x86_64-linux-gnu -fvisibility=hidden -fsanitize=cfi -flto -c /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize2.c -###
clang version 18.0.0 (https://github.com/llvm/llvm-project.git 8c3550b1a78fde7bf28f420da8447d9fde37017f)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin
 (in-process)
 "/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/bin/clang-18" "-cc1" "-triple" "x86_64-unknown-linux-gnu" "-emit-llvm-bc" "-flto=full" "-flto-unit" "-disable-free" "-clear-ast-before-backend" "-main-file-name" "fsanitize2.c" "-mrelocation-model" "pic" "-pic-level" "2" "-pic-is-pie" "-mframe-pointer=all" "-fmath-errno" "-ffp-contract=on" "-fno-rounding-math" "-mconstructor-aliases" "-funwind-tables=2" "-target-cpu" "x86-64" "-tune-cpu" "generic" "-debugger-tuning=gdb" "-fcoverage-compilation-dir=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver" "-resource-dir" "/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/lib/clang/18" "-internal-isystem" "/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/lib/clang/18/include" "-internal-isystem" "/usr/local/include" "-internal-isystem" "/usr/lib/gcc/x86_64-linux-gnu/9/../../../../x86_64-linux-gnu/include" "-internal-externc-isystem" "/usr/include/x86_64-linux-gnu" "-internal-externc-isystem" "/include" "-internal-externc-isystem" "/usr/include" "-fdebug-compilation-dir=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver" "-ferror-limit" "19" "-fvisibility=hidden" "-fsanitize=cfi-derived-cast,cfi-icall,cfi-mfcall,cfi-unrelated-cast,cfi-nvcall,cfi-vcall" "-fsanitize-trap=cfi-derived-cast,cfi-icall,cfi-mfcall,cfi-unrelated-cast,cfi-nvcall,cfi-vcall" "-fsanitize-system-ignorelist=/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/build/lib/clang/18/share/cfi_ignorelist.txt" "-fno-sanitize-memory-param-retval" "-fsanitize-cfi-canonical-jump-tables" "-fno-sanitize-address-use-odr-indicator" "-fgnuc-version=4.2.1" "-fcolor-diagnostics" "-fsplit-lto-unit" "-faddrsig" "-D__GCC_HAVE_DWARF2_CFI_ASM=1" "-o" "fsanitize2.o" "-x" "c" "/home/buildbot/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver/fsanitize2.c"
buildbot@6a3c0c51ae89:~/buildbot-root/llvm-clang-x86_64-gcc-ubuntu/llvm-project/clang/test/Driver$ echo $?
0

Thank you! Theree is an error depending on whether the cfi target is built (whether cfi_ignorelist.txt exists).

% /tmp/Rel/bin/clang --target=x86_64-linux-gnu -fvisibility=hidden -fsanitize=cfi -flto -c fsanitize.c '-###'
clang version 18.0.0
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /tmp/Rel/bin
clang: error: missing sanitizer ignorelist: '/tmp/Rel/lib/clang/18/share/cfi_ignorelist.txt'
 (in-process)
...

It seems that the most portable fix is to add -resource-dir=%S/Inputs/resource_dir .

(%clang --target=x86_64-linux-gnu -fvisibility=hidden -fsanitize=cfi -flto -c fsanitize.c '-###' 2> %t.err || true, then check %t.err would work, but it is ugly)

I am curious what @phosek 's failures were about.

MaskRay updated this revision to Diff 545678.Jul 31 2023, 8:21 AM
MaskRay edited the summary of this revision. (Show Details)

rebase

Matt added a subscriber: Matt.Jul 31 2023, 12:13 PM

I am curious what @phosek 's failures were about.

I created D156792 which should fix those errors, and explains what the errors were.

MaskRay updated this revision to Diff 546183.Aug 1 2023, 12:25 PM
MaskRay edited the summary of this revision. (Show Details)

rebase

Herald added a project: Restricted Project. · View Herald Transcript
This revision was landed with ongoing or failed builds.Aug 1 2023, 1:24 PM
This revision was automatically updated to reflect the committed changes.

Hi, after this change I have three failing tests:

Failed Tests (3):
  Clang :: Driver/lto.cu
  Clang :: Driver/openmp-offload-gpu.c
  Clang :: Driver/openmp-offload-jit.c

I have CUDA installed on my system, not sure if a bot is testing this configuration... Could you elaborate why the clang invocations in these tests are now expected to return a non-zero code?

ro added a subscriber: ro.Aug 2 2023, 1:29 AM

It seems the latest commit of this patch has (re-)introduced two failures on the Solaris/amd64 buildbot:

FAIL: Clang::clang_f_opts.c
FAIL: Clang::lto.c

I cannot really make sense of that.

Also seems to have impacted one of the AMDGPU bots but not the other, not sure why https://lab.llvm.org/staging/#/builders/247/builds/4145.

Clang :: Driver/amdgpu-hip-system-arch.c
Clang :: Driver/cuda-bad-arch.cu
Clang :: Driver/hip-autolink.hip
Clang :: Driver/hip-binding.hip
Clang :: Driver/hip-cuid-hash.hip
Clang :: Driver/hip-cuid.hip
Clang :: Driver/hip-default-gpu-arch.hip
Clang :: Driver/hip-device-compile.hip
Clang :: Driver/hip-host-cpu-features.hip
Clang :: Driver/hip-launch-api.hip
Clang :: Driver/hip-link-bc-to-bc.hip
Clang :: Driver/hip-link-bundle-archive.hip
Clang :: Driver/hip-no-device-libs.hip
Clang :: Driver/hip-options.hip
Clang :: Driver/hip-output-file-name.hip
Clang :: Driver/hip-printf.hip
Clang :: Driver/hip-save-temps.hip
Clang :: Driver/hip-std.hip
Clang :: Driver/hip-syntax-only.hip
Clang :: Driver/hip-toolchain-dwarf.hip
Clang :: Driver/hip-toolchain-features.hip
Clang :: Driver/hip-toolchain-mllvm.hip
Clang :: Driver/hip-toolchain-opt.hip
Clang :: Driver/lto.cu
Clang :: Driver/openmp-offload-gpu.c
Clang :: Driver/openmp-offload-infer.c
Clang :: Driver/rocm-detect.hip

These are the tests I'm seeing fail locally.

Pretty sure most of this is hip returning an error if it can't find ROCm.

Clang :: Driver/amdgpu-hip-system-arch.c
Clang :: Driver/cuda-bad-arch.cu
Clang :: Driver/hip-autolink.hip
Clang :: Driver/hip-binding.hip
Clang :: Driver/hip-cuid-hash.hip
Clang :: Driver/hip-cuid.hip
Clang :: Driver/hip-default-gpu-arch.hip
Clang :: Driver/hip-device-compile.hip
Clang :: Driver/hip-host-cpu-features.hip
Clang :: Driver/hip-launch-api.hip
Clang :: Driver/hip-link-bc-to-bc.hip
Clang :: Driver/hip-link-bundle-archive.hip
Clang :: Driver/hip-no-device-libs.hip
Clang :: Driver/hip-options.hip
Clang :: Driver/hip-output-file-name.hip
Clang :: Driver/hip-printf.hip
Clang :: Driver/hip-save-temps.hip
Clang :: Driver/hip-std.hip
Clang :: Driver/hip-syntax-only.hip
Clang :: Driver/hip-toolchain-dwarf.hip
Clang :: Driver/hip-toolchain-features.hip
Clang :: Driver/hip-toolchain-mllvm.hip
Clang :: Driver/hip-toolchain-opt.hip
Clang :: Driver/lto.cu
Clang :: Driver/openmp-offload-gpu.c
Clang :: Driver/openmp-offload-infer.c
Clang :: Driver/rocm-detect.hip

These are the tests I'm seeing fail locally.

These tests fail because the lit tests expect clang driver to run without ROCm.

I will update these tests so that they pass on systems with or w/o ROCm.

Clang :: Driver/amdgpu-hip-system-arch.c
Clang :: Driver/cuda-bad-arch.cu
Clang :: Driver/hip-autolink.hip
Clang :: Driver/hip-binding.hip
Clang :: Driver/hip-cuid-hash.hip
Clang :: Driver/hip-cuid.hip
Clang :: Driver/hip-default-gpu-arch.hip
Clang :: Driver/hip-device-compile.hip
Clang :: Driver/hip-host-cpu-features.hip
Clang :: Driver/hip-launch-api.hip
Clang :: Driver/hip-link-bc-to-bc.hip
Clang :: Driver/hip-link-bundle-archive.hip
Clang :: Driver/hip-no-device-libs.hip
Clang :: Driver/hip-options.hip
Clang :: Driver/hip-output-file-name.hip
Clang :: Driver/hip-printf.hip
Clang :: Driver/hip-save-temps.hip
Clang :: Driver/hip-std.hip
Clang :: Driver/hip-syntax-only.hip
Clang :: Driver/hip-toolchain-dwarf.hip
Clang :: Driver/hip-toolchain-features.hip
Clang :: Driver/hip-toolchain-mllvm.hip
Clang :: Driver/hip-toolchain-opt.hip
Clang :: Driver/lto.cu
Clang :: Driver/openmp-offload-gpu.c
Clang :: Driver/openmp-offload-infer.c
Clang :: Driver/rocm-detect.hip

These are the tests I'm seeing fail locally.

These tests fail because the lit tests expect clang driver to run without ROCm.

I will update these tests so that they pass on systems with or w/o ROCm.

I've got a patch half done that mostly adds -nogpuinc to all these run lines.

ro added a comment.Aug 2 2023, 10:39 AM
In D156363#4553043, @ro wrote:

It seems the latest commit of this patch has (re-)introduced two failures on the Solaris/amd64 buildbot:

FAIL: Clang::clang_f_opts.c
FAIL: Clang::lto.c

I cannot really make sense of that.

I think I found it: running the matching '*.script' files under bash -x shows the tests ending with:

  • for clang_f_lto.c:
+ : 'RUN: at line 117'
+ /var/llvm/local-amd64-debug-stage2/tools/clang/stage2-bins/bin/clang -### -flto -forder-file-instrumentation /vol/llvm/src/llvm-project/local/clang/test/Driver/clang_f_opts.c
+ /var/llvm/local-amd64-debug-stage2/tools/clang/stage2-bins/bin/FileCheck -check-prefix=CHECK-ORDERFILE-INSTR-LTO /vol/llvm/src/llvm-project/local/clang/test/Driver/clang_f_opts.c
  • for lto.c, it's similar:
+ : 'RUN: at line 19'
+ /var/llvm/local-amd64-debug-stage2/tools/clang/stage2-bins/bin/clang /vol/llvm/src/llvm-project/local/clang/test/Driver/lto.c -flto -save-temps -###
ro@niers 79 > /var/llvm/local-amd64-debug-stage2/tools/clang/stage2-bins/bin/clan

Manually re-running clang gives

clang: error: 'amd64-pc-solaris2.11': unable to pass LLVM bit-code files to linker

in both cases.

In D156363#4554790, @ro wrote:
In D156363#4553043, @ro wrote:

It seems the latest commit of this patch has (re-)introduced two failures on the Solaris/amd64 buildbot:

FAIL: Clang::clang_f_opts.c
FAIL: Clang::lto.c

I cannot really make sense of that.

I think I found it: running the matching '*.script' files under bash -x shows the tests ending with:

  • for clang_f_lto.c:
+ : 'RUN: at line 117'
+ /var/llvm/local-amd64-debug-stage2/tools/clang/stage2-bins/bin/clang -### -flto -forder-file-instrumentation /vol/llvm/src/llvm-project/local/clang/test/Driver/clang_f_opts.c
+ /var/llvm/local-amd64-debug-stage2/tools/clang/stage2-bins/bin/FileCheck -check-prefix=CHECK-ORDERFILE-INSTR-LTO /vol/llvm/src/llvm-project/local/clang/test/Driver/clang_f_opts.c
  • for lto.c, it's similar:
+ : 'RUN: at line 19'
+ /var/llvm/local-amd64-debug-stage2/tools/clang/stage2-bins/bin/clang /vol/llvm/src/llvm-project/local/clang/test/Driver/lto.c -flto -save-temps -###
ro@niers 79 > /var/llvm/local-amd64-debug-stage2/tools/clang/stage2-bins/bin/clan

Manually re-running clang gives

clang: error: 'amd64-pc-solaris2.11': unable to pass LLVM bit-code files to linker

in both cases.

Probably because we're not specifying the --target= I'll add that in my fix for AMDGPU I'm working on and see if it solves the problem.

In D156363#4554790, @ro wrote:
In D156363#4553043, @ro wrote:

It seems the latest commit of this patch has (re-)introduced two failures on the Solaris/amd64 buildbot:

FAIL: Clang::clang_f_opts.c
FAIL: Clang::lto.c

I cannot really make sense of that.

I think I found it: running the matching '*.script' files under bash -x shows the tests ending with:

  • for clang_f_lto.c:
+ : 'RUN: at line 117'
+ /var/llvm/local-amd64-debug-stage2/tools/clang/stage2-bins/bin/clang -### -flto -forder-file-instrumentation /vol/llvm/src/llvm-project/local/clang/test/Driver/clang_f_opts.c
+ /var/llvm/local-amd64-debug-stage2/tools/clang/stage2-bins/bin/FileCheck -check-prefix=CHECK-ORDERFILE-INSTR-LTO /vol/llvm/src/llvm-project/local/clang/test/Driver/clang_f_opts.c
  • for lto.c, it's similar:
+ : 'RUN: at line 19'
+ /var/llvm/local-amd64-debug-stage2/tools/clang/stage2-bins/bin/clang /vol/llvm/src/llvm-project/local/clang/test/Driver/lto.c -flto -save-temps -###
ro@niers 79 > /var/llvm/local-amd64-debug-stage2/tools/clang/stage2-bins/bin/clan

Manually re-running clang gives

clang: error: 'amd64-pc-solaris2.11': unable to pass LLVM bit-code files to linker

in both cases.

Probably because we're not specifying the --target= I'll add that in my fix for AMDGPU I'm working on and see if it solves the problem.

Thank you for fixing these tests. I agree that clang_f_opts.c:117 with -flto needs a --target=, to not cause this error on amd64-pc-solaris2.11'.
This should be the only line. I unfortunately did not catch this because I do not have access to a target using HasNativeLLVMSupport == false`.

For the AMDGPU tests, I do not have local ROCm, so I am unable to catch the issues. I have managed to fix many others before relanding this patch.

Clang :: Driver/amdgpu-hip-system-arch.c
Clang :: Driver/cuda-bad-arch.cu
Clang :: Driver/hip-autolink.hip
Clang :: Driver/hip-binding.hip
Clang :: Driver/hip-cuid-hash.hip
Clang :: Driver/hip-cuid.hip
Clang :: Driver/hip-default-gpu-arch.hip
Clang :: Driver/hip-device-compile.hip
Clang :: Driver/hip-host-cpu-features.hip
Clang :: Driver/hip-launch-api.hip
Clang :: Driver/hip-link-bc-to-bc.hip
Clang :: Driver/hip-link-bundle-archive.hip
Clang :: Driver/hip-no-device-libs.hip
Clang :: Driver/hip-options.hip
Clang :: Driver/hip-output-file-name.hip
Clang :: Driver/hip-printf.hip
Clang :: Driver/hip-save-temps.hip
Clang :: Driver/hip-std.hip
Clang :: Driver/hip-syntax-only.hip
Clang :: Driver/hip-toolchain-dwarf.hip
Clang :: Driver/hip-toolchain-features.hip
Clang :: Driver/hip-toolchain-mllvm.hip
Clang :: Driver/hip-toolchain-opt.hip
Clang :: Driver/lto.cu
Clang :: Driver/openmp-offload-gpu.c
Clang :: Driver/openmp-offload-infer.c
Clang :: Driver/rocm-detect.hip

These are the tests I'm seeing fail locally.

These tests fail because the lit tests expect clang driver to run without ROCm.

I will update these tests so that they pass on systems with or w/o ROCm.

I've got a patch half done that mostly adds -nogpuinc to all these run lines.

Thanks. I will wait for your patch. You can leave the tricky ones to me if you would like. e.g. the rocm-detect.hip

Thanks. I will wait for your patch. You can leave the tricky ones to me if you would like. e.g. the rocm-detect.hip

You can go ahead and fix that one then.

ro added a comment.Aug 2 2023, 12:12 PM

Probably because we're not specifying the --target= I'll add that in my fix for AMDGPU I'm working on and see if it solves the problem.

Thank you for fixing these tests. I agree that clang_f_opts.c:117 with -flto needs a --target=, to not cause this error on amd64-pc-solaris2.11'.
This should be the only line. I unfortunately did not catch this because I do not have access to a target using HasNativeLLVMSupport == false`.

Confirmed: adding -target x86_64-unknown-linux to both clang_f_opts.c:117 and lto.c:19 let both tests PASS on amd64-pc-solaris2.11.

test/Driver/openmp-offload-jit.c was still failing for me with CUDA installed, so I pushed rGcb3136b0d3596f5c3984e4fb49359e2a1a28a547 to add an explicit --cuda-path and make it return exit code 0 on (hopefully) all systems.

zixuan-wu added inline comments.
clang/test/Driver/debug-options.c
245

Here should add not or specify the target with x86 because it fails when only RV target is compiled.

clang: error: -gsplit-dwarf is unsupported with RISC-V linker relaxation (-mrelax)

I am afraid there is also error at clang/test/Driver/as-options.s when default triple is 'riscv32-unknown-elf'