[asan][test][win] Port more tests to not use clang-cl on MinGW

[alvinhochun]

Summary

Depends on D147432

[alvinhochun]

Fix mistake

[vitalybuka]

compiler-rt/test/asan/TestCases/Windows/crt_initializers.cpp

1–2

I believe it would be much cleaner if done substituting: llvm-project/compiler-rt/test/asan/lit.cfg.py:163

[alvinhochun]

compiler-rt/test/asan/TestCases/Windows/crt_initializers.cpp

1–2

I don't know how one could use substitution for this, since clang and clang-cl use very different command line parameters (especially problematic for the more complex DLL tests which I have yet to port).

Or do you mean using substitution to turn certain RUN: commands into comments? Like:

// RUN: %if_clang_cl %clang_cl_asan -Od %s -Fe%t
// RUN: %if_not_clang_cl %clangxx_asan -O0 %s -o %t

...and conditionally substituting one of %if_clang_cl and %if_not_clang_cl to be : and the other one to be empty?

[mstorsjo]

compiler-rt/test/asan/TestCases/Windows/crt_initializers.cpp

1–2

You could make one substitution for -Fe or -o (note, including the trailing space), another one for -O0 vs -Od etc.

Sorry if it’s obvious if I haven’t followed closely enough, but would it be possible to port the tests to use the gcc driver syntax even with clang in msvc mode? (I understand that that would be a bigger undertaking though.) Or are these tests executed with cl.exe too, so we need to remain compatible with that syntax anyway?

[alvinhochun]

compiler-rt/test/asan/TestCases/Windows/crt_initializers.cpp

1–2

If we need one substitution per flag, I don't know how many substitutions we will end up having, and I may have difficulty naming them. Some flags may also be incompatible (mingw needs -Wl,--out-implib to generate import lib). To me it feels messier than just writing the commands separately.

Rewriting to use gcc driver syntax shouldn't be hard, but I am afraid of breaking them due to obscure differences between the two drivers (things like static/dynamic, debug/release crt). I suppose these tests also serve the purpose of testing asan support in the clang-cl driver so perhaps they should continue to use clang-cl.

It doesn't look like the tests would work with cl.exe out of the box though.

[mstorsjo]

compiler-rt/test/asan/TestCases/Windows/crt_initializers.cpp

1–2

@vitalybuka - Can you elaborate on how you want this to be substituted?

[vitalybuka]

compiler-rt/test/asan/TestCases/Windows/crt_initializers.cpp

1–2

Per flag, like %fPIC llvm-project/compiler-rt/test/asan/lit.cfg.py:234 ?

[vitalybuka]

compiler-rt/test/asan/TestCases/Windows/crt_initializers.cpp

1–2

at least for frequent stuff
for flags which are in 1-2 tests if/else should be fine

[vitalybuka]

compiler-rt/test/asan/TestCases/Windows/crt_initializers.cpp

1–2

@phosek @MaskRay Do you have opinion on %if %else in run lines?
Maybe I am unfairly biased against them, when they are reasonable way to solve the problem?

[alvinhochun]

Use substitution to map clang-cl command and flags to clang gcc-style flags on MinGW

[MaskRay]

compiler-rt/test/asan/TestCases/Windows/crt_initializers.cpp

1–2

I'll certainly avoid complex constructs if possible.
When they provide values, I'll not simply keep them off.

rg '%if target=' compiler-rt/test/ has some examples.

Bonus: our internal lit runner supports many %if constructs.

[mstorsjo]

compiler-rt/test/asan/TestCases/Windows/crt_initializers.cpp

1–2

@MaskRay Yes, all those hits in compiler-rt/test are ones added by @alvinhochun recently. The question is mostly here whether we should prefer adding more %if conditions in the test (older revision of this patch) or add more substitutions, to make parametrized expansions that work across both clang-cl and mingw.

As an example - D147432 (which this patch depends on) changes RUN lines with %clang_cl_asan -Od %s -Fe%t into %clang_cl_asan %Od %s %Fe%t, and for mingw mode, %Od expands to -O0 and %Fe expands to -o. The run lines get a bit more cryptic, but it works. (But for more complex tests it might not be enough with just such substitutions, but actual conditionals might be needed.)

compiler-rt/test/asan/TestCases/Windows/crt_initializers.cpp

-// UNSUPPORTED: target={{.*-windows-gnu}}
-
-// RUN: %clang_cl_asan -Od %s -Fe%t
+// RUN: %if clang-cl %{ %clang_cl_asan -Od %s -Fe%t %} \
+// RUN: %else %{ %clangxx_asan -O0 %s -o %t %}
 // RUN: %run %t | FileCheck %s
 
 // This is a test for http://code.google.com/p/address-sanitizer/issues/detail?id=305
 
 #include <stdio.h>
 
 typedef void (*FPTR)();
 

compiler-rt/test/asan/TestCases/Windows/demangled_names.cpp

-// UNSUPPORTED: target={{.*-windows-gnu}}
-
-// RUN: %clang_cl_asan -Od %s -Fe%t
-// RUN: not %run %t 2>&1 | FileCheck %s
+// RUN: %if clang-cl %{ %clang_cl_asan -Od %s -Fe%t %} \
+// RUN: %else %{ %clangxx_asan -O0 %s -o %t %}
+// RUN: not %run %t 2>&1 | FileCheck %s --check-prefixes=CHECK,%if clang-cl %{MSVC%} %else %{MINGW%}
 //
 // This test makes sure ASan symbolizes stack traces the way they are typically
 // symbolized on Windows.
 #include <malloc.h>
 
 namespace foo {
 // A template function in a namespace.
 template<int x>

 A<char*>::~A() {
   baz<char*, true>(v_);
 }
 
 int main() {
   char *buffer = (char*)malloc(42);
   free(buffer);
   A<char*> a(buffer);
   // CHECK: AddressSanitizer: heap-use-after-free on address [[ADDR:0x[0-9a-f]+]]
   // CHECK: foo::bar<42>{{.*}}demangled_names.cpp
   // CHECK: foo::spam{{.*}}demangled_names.cpp
-// CHECK: baz<char *,{{ *}}1>{{.*}}demangled_names.cpp
-// CHECK: A<char *>::~A<char *>{{.*}}demangled_names.cpp
+  // MSVC:  baz<char *,{{ *}}1>{{.*}}demangled_names.cpp
+  // MINGW: baz<char*, true>{{.*}}demangled_names.cpp
+  // MSVC:  A<char *>::~A<char *>{{.*}}demangled_names.cpp
+  // MINGW: A<char*>::~A(){{.*}}demangled_names.cpp
 }

compiler-rt/test/asan/TestCases/Windows/double_operator_delete.cpp

-// UNSUPPORTED: target={{.*-windows-gnu}}
-
-// RUN: %clang_cl_asan -Od %s -Fe%t
+// RUN: %if clang-cl %{ %clang_cl_asan -Od %s -Fe%t %} \
+// RUN: %else %{ %clangxx_asan -O0 %s -o %t %}
 // RUN: not %run %t 2>&1 | FileCheck %s
 
 #include <malloc.h>
 
 int main() {
   int *x = new int[42];
   delete [] x;
   delete [] x;
   // CHECK: AddressSanitizer: attempting double-free on [[ADDR:0x[0-9a-f]+]]
-// FIXME: The 'operator delete' frame should have [].
-// CHECK-NEXT: {{#0 .* operator delete}}
-// CHECK-NEXT: {{#1 .* main .*double_operator_delete.cpp}}:[[@LINE-4]]
+  // CHECK-NEXT: {{#0 .* operator delete}}[]
+  // CHECK-NEXT: {{#1 .* main .*double_operator_delete.cpp}}:[[@LINE-3]]
   // CHECK: [[ADDR]] is located 0 bytes inside of 168-byte region
   // CHECK-LABEL: freed by thread T0 here:
-// FIXME: The 'operator delete' frame should have [].
-// CHECK-NEXT: {{#0 .* operator delete}}
-// CHECK-NEXT: {{#1 .* main .*double_operator_delete.cpp}}:[[@LINE-10]]
+  // CHECK-NEXT: {{#0 .* operator delete}}[]
+  // CHECK-NEXT: {{#1 .* main .*double_operator_delete.cpp}}:[[@LINE-8]]
   // CHECK-LABEL: previously allocated by thread T0 here:
-// FIXME: The 'operator new' frame should have [].
-// CHECK-NEXT: {{#0 .* operator new}}
-// CHECK-NEXT: {{#1 .* main .*double_operator_delete.cpp}}:[[@LINE-15]]
+  // CHECK-NEXT: {{#0 .* operator new}}[]
+  // CHECK-NEXT: {{#1 .* main .*double_operator_delete.cpp}}:[[@LINE-12]]
   return 0;
 }
 

compiler-rt/test/asan/TestCases/Windows/operator_array_new_left_oob.cpp

-// UNSUPPORTED: target={{.*-windows-gnu}}
-
-// RUN: %clang_cl_asan -Od %s -Fe%t
+// RUN: %if clang-cl %{ %clang_cl_asan -Od %s -Fe%t %} \
+// RUN: %else %{ %clangxx_asan -O0 %s -o %t %}
 // RUN: not %run %t 2>&1 | FileCheck %s
 
 int main() {
   char *buffer = new char[42];
   buffer[-1] = 42;
   // CHECK: AddressSanitizer: heap-buffer-overflow on address [[ADDR:0x[0-9a-f]+]]
   // CHECK: WRITE of size 1 at [[ADDR]] thread T0
   // CHECK-NEXT: {{#0 .* main .*operator_array_new_left_oob.cpp}}:[[@LINE-3]]
   //
   // CHECK: [[ADDR]] is located 1 bytes before 42-byte region
   // CHECK-LABEL: allocated by thread T0 here:
-// FIXME: The 'operator new' frame should have [].
-// CHECK-NEXT: {{#0 .* operator new}}
-// CHECK-NEXT: {{#1 .* main .*operator_array_new_left_oob.cpp}}:[[@LINE-10]]
+  // CHECK-NEXT: {{#0 .* operator new}}[]
+  // CHECK-NEXT: {{#1 .* main .*operator_array_new_left_oob.cpp}}:[[@LINE-9]]
   delete [] buffer;
 }

compiler-rt/test/asan/TestCases/Windows/operator_array_new_right_oob.cpp

-// UNSUPPORTED: target={{.*-windows-gnu}}
-
-// RUN: %clang_cl_asan -Od %s -Fe%t
+// RUN: %if clang-cl %{ %clang_cl_asan -Od %s -Fe%t %} \
+// RUN: %else %{ %clangxx_asan -O0 %s -o %t %}
 // RUN: not %run %t 2>&1 | FileCheck %s
 
 #include <windows.h>
 
 int main() {
   char *buffer = new char[42];
   buffer[42] = 42;
   // CHECK: AddressSanitizer: heap-buffer-overflow on address [[ADDR:0x[0-9a-f]+]]
   // CHECK: WRITE of size 1 at [[ADDR]] thread T0
   // CHECK:   {{#0 .* main .*operator_array_new_right_oob.cpp}}:[[@LINE-3]]
   // CHECK: [[ADDR]] is located 0 bytes after 42-byte region
   // CHECK: allocated by thread T0 here:
-// FIXME: The 'operator new' frame should have [].
-// CHECK:   {{#0 .* operator new}}
-// CHECK:   {{#1 .* main .*operator_array_new_right_oob.cpp}}:[[@LINE-9]]
+  // CHECK:   {{#0 .* operator new}}[]
+  // CHECK:   {{#1 .* main .*operator_array_new_right_oob.cpp}}:[[@LINE-8]]
   delete [] buffer;
 }

compiler-rt/test/asan/TestCases/Windows/operator_array_new_uaf.cpp

-// UNSUPPORTED: target={{.*-windows-gnu}}
-
-// RUN: %clang_cl_asan -Od %s -Fe%t
+// RUN: %if clang-cl %{ %clang_cl_asan -Od %s -Fe%t %} \
+// RUN: %else %{ %clangxx_asan -O0 %s -o %t %}
 // RUN: not %run %t 2>&1 | FileCheck %s
 
 #include <windows.h>
 
 int main() {
   char *buffer = new char[42];
   delete [] buffer;
   buffer[0] = 42;
   // CHECK: AddressSanitizer: heap-use-after-free on address [[ADDR:0x[0-9a-f]+]]
   // CHECK: WRITE of size 1 at [[ADDR]] thread T0
   // CHECK:   {{#0 .* main .*operator_array_new_uaf.cpp}}:[[@LINE-3]]
   // CHECK: [[ADDR]] is located 0 bytes inside of 42-byte region
   // CHECK-LABEL: freed by thread T0 here:
-// FIXME: The 'operator delete' frame should have [].
-// CHECK:   {{#0 .* operator delete}}
-// CHECK:   {{#1 .* main .*operator_array_new_uaf.cpp}}:[[@LINE-9]]
+  // CHECK:   {{#0 .* operator delete}}[]
+  // CHECK:   {{#1 .* main .*operator_array_new_uaf.cpp}}:[[@LINE-8]]
   // CHECK-LABEL: previously allocated by thread T0 here:
-// FIXME: The 'operator new' frame should have [].
-// CHECK:   {{#0 .* operator new}}
-// CHECK:   {{#1 .* main .*operator_array_new_uaf.cpp}}:[[@LINE-14]]
+  // CHECK:   {{#0 .* operator new}}[]
+  // CHECK:   {{#1 .* main .*operator_array_new_uaf.cpp}}:[[@LINE-12]]
   return 0;
 }
 

compiler-rt/test/asan/TestCases/Windows/operator_array_new_with_dtor_left_oob.cpp

-// UNSUPPORTED: target={{.*-windows-gnu}}
-
-// RUN: %clang_cl_asan -Od %s -Fe%t
+// RUN: %if clang-cl %{ %clang_cl_asan -Od %s -Fe%t %} \
+// RUN: %else %{ %clangxx_asan -O0 %s -o %t %}
 // RUN: not %run %t 2>&1 | FileCheck %s
 
 struct C {
   int x;
   ~C() {}
 };
 
 int __attribute__((noinline, optnone)) hide(int x) { return x; }
 
 int main() {
   C *buffer = new C[42];
   buffer[hide(-(1 + (int)sizeof(void*) / 4))].x = 42;
   // CHECK: AddressSanitizer: heap-buffer-overflow on address [[ADDR:0x[0-9a-f]+]]
   // CHECK: WRITE of size 4 at [[ADDR]] thread T0
   // CHECK-NEXT: {{#0 .* main .*operator_array_new_with_dtor_left_oob.cpp}}:[[@LINE-3]]
   //
   // FIXME: Currently it says "4 bytes ... left of 172-byte region",
   //        should be "8 bytes ... left of 168-byte region", see
   //        https://code.google.com/p/address-sanitizer/issues/detail?id=314
   // CHECK: [[ADDR]] is located {{.*}} bytes before {{(172|176)}}-byte region
   // CHECK-LABEL: allocated by thread T0 here:
-// FIXME: The 'operator new' frame should have [].
-// CHECK-NEXT: {{#0 .* operator new}}
-// CHECK-NEXT: {{#1 .* main .*operator_array_new_with_dtor_left_oob.cpp}}:[[@LINE-13]]
+  // CHECK-NEXT: {{#0 .* operator new}}[]
+  // CHECK-NEXT: {{#1 .* main .*operator_array_new_with_dtor_left_oob.cpp}}:[[@LINE-12]]
   delete [] buffer;
 }

compiler-rt/test/asan/TestCases/Windows/sse_misalignment.cpp

-// UNSUPPORTED: target={{.*-windows-gnu}}
-
-// RUN: %clang_cl_asan -Od %s -Fe%t
+// RUN: %if clang-cl %{ %clang_cl_asan -Od %s -Fe%t %} \
+// RUN: %else %{ %clangxx_asan -O0 %s -o %t %}
 // RUN: %env_asan_opts=handle_sigfpe=1 not %run %t 2>&1 | FileCheck %s
 
+// FIXME: On MinGW frame #0 does not include the line number?
+// XFAIxL: target={{.*-windows-gnu}}
+
 // Test the error output from misaligned SSE2 memory access. This is a READ
 // memory access. Windows appears to always provide an address of -1 for these
 // types of faults, and there doesn't seem to be a way to distinguish them from
 // other types of access violations without disassembling.
 
 #include <emmintrin.h>
 #include <stdio.h>
 
 __m128i test() {
   char buffer[17] = {};
   __m128i a = _mm_load_si128((__m128i *)buffer);
   __m128i b = _mm_load_si128((__m128i *)(&buffer[0] + 1));
   return _mm_or_si128(a, b);
 }
 
 int main() {
   puts("before alignment fault");
   fflush(stdout);
   volatile __m128i v = test();
   return 0;
 }
 // CHECK: before alignment fault
 // CHECK: ERROR: AddressSanitizer: access-violation on unknown address {{0x[fF]*}}
 // CHECK-NEXT: The signal is caused by a READ memory access.
-// CHECK-NEXT: #0 {{.*}} in test(void) {{.*}}misalignment.cpp:{{.*}}
+// CHECK-NEXT: #0 {{.*}} in test({{(void)?}}) {{.*}}misalignment.cpp:{{.*}}