Details
Diff Detail
Event Timeline
Open question: how does %t even work for compiler-rt? In multi-arch setup, the expansion will be the same for different architectures, so two tests are quite likely to race. From what I have seen, the tests seem to be run because they are somehow run sequentially for different architectures?
I think we should follow the approach used by the ASan and TSan tests here.
A configuration directory is created for each target in the test build directory. This prevents any races between the configurations. For example on Darwin take a look at projects/compiler-rt/test/asan/X86_64DarwinConfig/ and projects/compiler-rt/test/asan/IOSSimX86_64Config/ inside the root of your LLVM build tree (this assumes you're doing an in-tree build of compiler-rt).
The ASan/TSan tests go to a lot of trouble to run tests for targets that are not the same of the host. Take a look at test/asan/TestCases/alloca_big_alignment.cc for example.
- Clang is not invoked directly. Instead a %clangxx_asan substitution is used. When the target is the same as the host this just expands to the path to the freshly built clang and the normal ASan flags.
When the target is not the host this expands to something different. For the IOSSimX86_64 target for example the test/sanitizer_common/ios_commands/iossim_compile.py script gets invoked instead (it gets passed what the host compiler invocation would be as arguments).
A similar thing is done for Android devices. The general idea here is the wrapper script should handle any peculiarities for compiling code for the target we want to test.
- The compiled binary is not invoked directly. Instead it is prefixed with the %run substitution. When the host is the same as the target this a no-op. When the host is not equal to the target then %run expands to a script that handles running the compiled binary.
For the IOSSimX86_64 target for example, the test/sanitizer_common/ios_commands/iossim_run.py script is used. In this particular case this script invokes the iOS simulator to run the compiled binary.
This approach seems to work pretty well. Although I've not tested your patch I suspect it doesn't do what I suggest above. My guess is that it assumes that host is equal to the target which is not an assumption we should be making.
A configuration directory is created for each target in the test build directory
Aha, this makes a lot of sense.
I see this in the cmake configuration, must have missed it on the first reading: CONFIG_NAME includes the architecture, and determines the folder where tests are run.
When the target is not the host this expands to something different
Right, so fuzzer tests already have their own clang variable, the function generating the expansion should be just updated to perform the tricks you have mentioned.
Instead it is prefixed with the %run substitution
Yes, this should be done as well, and we should also start running the linting script checking that on libFuzzer tests.
test/fuzzer/afl-driver.test | ||
---|---|---|
18 | Could you please move %run changes into a separate patch? |
Could you please move %run changes into a separate patch?