[PDB] Make PDB lit tests use the new builder

[aleksandr.urakov]

Summary

This patch makes old PDB plugin tests to use the new builder (see D54914).

There are some problems left:

• Clang called from the builder shows errors on types like char16_t and char32_t. That's why enums-layout.test and typedefs.test are made to use MSVC instead;
• The builder now doesn't support /Gy and /order options, so the old scheme is used in function-level-linking.test;
• The builder now doesn't support compilation of multiple sources and linking them together, so the old scheme is used for linking in func-symbols.test;
• On Windows 32-bit LLDB can debug only 32-bit applications, and 64-bit LLDB can debug only 64-bit applications. That's why the old scheme is left in dynamic tests udt-layout.test, variables-locations.test and vbases.test. If we will always compile the tests as 32-bit (64-bit), then they will fail on the 64-bit (32-bit) platform. That's why we just assume that the user runs the tests from the same command prompt where the build was run.

This one depends on D54914.

[stella.stamenova]

This looks fine, but let's see if Zachary's change lands first and how it looks like after it lands :).

[zturner]

Clang called from the builder shows errors on types like char16_t and char32_t. That's why enums-layout.test and typedefs.test are made to use MSVC instead;

For me enums-layout.test crashes, but I see the same behavior in typedefs.test. This is because lld-link does not emit S_UDT records (note the original test was using %msvc_link, not %lld_link. That's why it passed before). I could probably add a new magic toolchain called clang-cl-with-msvc-linker but I think it's also fine to just keep these using cl.exe.

The builder now doesn't support /Gy and /order options, so the old scheme is used in function-level-linking.test;

Makes sense. Just curious, is the order file strictly necessary for this test? /Gy is the same as -ffunction-sections, so there could be an argument to be made for exposing --function-sections on the command line of build.py. On the other hand, there's no harm in falling back to the old mechanism here since this test isn't intended to be portable.

The builder now doesn't support compilation of multiple sources and linking them together, so the old scheme is used for linking in func-symbols.test;

This one is important to fix.

On Windows 32-bit LLDB can debug only 32-bit applications, and 64-bit LLDB can debug only 64-bit applications. That's why the old scheme is left in dynamic tests udt-layout.test, variables-locations.test and vbases.test. If we will always compile the tests as 32-bit (64-bit), then they will fail on the 64-bit (32-bit) platform. That's why we just assume that the user runs the tests from the same command prompt where the build was run.

Ahh, right. I wonder if we should have something like --arch=lldb that means "match the architecture of LLDB". It's not urgent for this patch, but it could be useful as a followup.

[aleksandr.urakov]

Thanks all!

In D54942#1310224, @zturner wrote:

Makes sense. Just curious, is the order file strictly necessary for this test? /Gy is the same as -ffunction-sections, so there could be an argument to be made for exposing --function-sections on the command line of build.py. On the other hand, there's no harm in falling back to the old mechanism here since this test isn't intended to be portable.

Yes, the order is necessary for the test, because it tests exactly correct processing of source files that are not continuous in the result binary. But I'm not against leaving it as is.

Ahh, right. I wonder if we should have something like --arch=lldb that means "match the architecture of LLDB". It's not urgent for this patch, but it could be useful as a followup.

Yes, I thought about a solution like these too. May be we could even make it the default arch?

[aleksandr.urakov]

Update tests due to D55230

[aleksandr.urakov]

It seems that all the reviews this one depends on are already in. Can we proceed with it?

[stella.stamenova]

enums-layout.test is now failing on Windows:

http://lab.llvm.org:8014/builders/lldb-x64-windows-ninja/builds/1938/steps/test/logs/stdio

[aleksandr.urakov]

It looks strange. The compiler can't open the object file. And in the next build it is ok - may be it was some server failure (e.g. full disk)?

[aleksandr.urakov]

The similar problem with typedefs.test is here: http://lab.llvm.org:8014/builders/lldb-x64-windows-ninja/builds/1940/steps/test/logs/stdio

I have an assumption about the cause of the problem. Are the tests running in parallel? In this case typedefs.test and enums-layout.test are writing to the same object file together, because they both are compiled from the same source.

@zturner Is it possible to specify object file's name in compile-and-link mode? Then we can specify different names in different tests for both object files and executables. But I think that splitting the source or combining the tests would be a better idea.

I will fix it only tomorrow, because I'm already at home today. Feel free to revert it if needed.

lit/SymbolFile/PDB/ast-restore.test

-REQUIRES: system-windows
-RUN: %msvc_cl /Zi /GS- /c %S/Inputs/AstRestoreTest.cpp /Fo%t.obj
-RUN: %msvc_link /debug:full /nodefaultlib /entry:main %t.obj /out:%t.exe
+REQUIRES: system-windows, msvc
+RUN: %build --compiler=msvc --nodefaultlib --source=%S/Inputs/AstRestoreTest.cpp --output=%t.exe
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=ENUM %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=GLOBAL %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=BASE %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=CLASS %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=INNER %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=TEMPLATE %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=FOO %s
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck --check-prefix=MAIN %s

lit/SymbolFile/PDB/calling-conventions.test

 REQUIRES: system-windows, lld
-RUN: %clang_cl -m32 /Zi /GS- /c %S/Inputs/CallingConventionsTest.cpp /o %t.obj
-RUN: lld-link /debug:full /nodefaultlib /entry:main %t.obj /out:%t.exe
+RUN: %build --compiler=clang-cl --arch=32 --nodefaultlib --source=%S/Inputs/CallingConventionsTest.cpp --output=%t.exe
 RUN: lldb-test symbols -dump-ast %t.exe | FileCheck %s
 
 CHECK: Module: {{.*}}
 CHECK-DAG: int (*FuncCCallPtr)();
 CHECK-DAG: int (*FuncStdCallPtr)() __attribute__((stdcall));
 CHECK-DAG: int (*FuncFastCallPtr)() __attribute__((fastcall));
 CHECK-DAG: int (*FuncVectorCallPtr)() __attribute__((vectorcall));
 CHECK-DAG: int (S::*FuncThisCallPtr)() __attribute__((thiscall));

lit/SymbolFile/PDB/class-layout.test

-REQUIRES: msvc
-RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/ClassLayoutTest.cpp /o %T/ClassLayoutTest.cpp.obj
-RUN: %msvc_link %T/ClassLayoutTest.cpp.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/ClassLayoutTest.cpp.exe
+REQUIRES: system-windows, msvc
+RUN: %build --compiler=clang-cl --mode=compile --arch=32 --nodefaultlib --source=%S/Inputs/ClassLayoutTest.cpp --output=%T/ClassLayoutTest.cpp.obj
+RUN: %build --compiler=msvc --mode=link --arch=32 --nodefaultlib --source=%T/ClassLayoutTest.cpp.obj --output=%T/ClassLayoutTest.cpp.exe
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=ENUM %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=UNION %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=STRUCT %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=COMPLEX %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=LIST %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=UNNAMED-STRUCT %s
 RUN: lldb-test symbols %T/ClassLayoutTest.cpp.exe | FileCheck --check-prefix=BASE %s

lit/SymbolFile/PDB/compilands.test

 REQUIRES: system-windows, msvc
-RUN: %clang_cl /Z7 /c %S/Inputs/CompilandsTest.cpp /o %T/CompilandsTest.cpp.obj
-RUN: %msvc_link /debug:full /nodefaultlib /entry:main %T/CompilandsTest.cpp.obj /out:%T/CompilandsTest.cpp.exe
+RUN: %build --compiler=clang-cl --mode=compile --arch=32 --nodefaultlib --source=%S/Inputs/CompilandsTest.cpp --output=%T/CompilandsTest.cpp.obj
+RUN: %build --compiler=msvc --mode=link --arch=32 --nodefaultlib --source=%T/CompilandsTest.cpp.obj --output=%T/CompilandsTest.cpp.exe
 RUN: lldb-test symbols %T/CompilandsTest.cpp.exe | FileCheck %s
 
 ; Link default libraries
 
 CHECK: Module [[CU:.*]]
 CHECK: {{^[0-9A-F]+}}: SymbolVendor ([[CU]])
 CHECK: {{^[0-9A-F]+}}:   CompileUnit{{[{]0x[0-9a-f]+[}]}}, language = "c++", file = '{{.*}}\CompilandsTest.cpp'

lit/SymbolFile/PDB/enums-layout.test

 REQUIRES: system-windows, msvc
-RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/SimpleTypesTest.cpp /o %T/SimpleTypesTest.cpp.enums.obj
-RUN: %msvc_link %T/SimpleTypesTest.cpp.enums.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/SimpleTypesTest.cpp.enums.exe
+RUN: %build --compiler=msvc --arch=32 --nodefaultlib --source=%S/Inputs/SimpleTypesTest.cpp --output=%T/SimpleTypesTest.cpp.enums.exe
 RUN: lldb-test symbols %T/SimpleTypesTest.cpp.enums.exe | FileCheck %s
 
 ; FIXME: PDB does not have information about scoped enumeration (Enum class) so the  
 ; compiler type used is the same as the one for unscoped enumeration.
 
 CHECK: Module [[CU:.*]]
 CHECK-DAG: {{^[0-9A-F]+}}: SymbolVendor ([[CU]])
-CHECK:      Type{{.*}} , name = "Enum", size = 4, decl = SimpleTypesTest.cpp:19, compiler_type = {{.*}} enum Enum {
+CHECK:      Type{{.*}} , name = "Enum", size = 4, decl = simpletypestest.cpp:19, compiler_type = {{.*}} enum Enum {
 CHECK-NEXT:    RED,
 CHECK-NEXT:    GREEN,
 CHECK-NEXT:    BLUE
 CHECK-NEXT:}
 
-CHECK:      Type{{.*}} , name = "EnumConst", size = 4,  decl = SimpleTypesTest.cpp:22, compiler_type = {{.*}} enum EnumConst {
+CHECK:      Type{{.*}} , name = "EnumConst", size = 4,  decl = simpletypestest.cpp:22, compiler_type = {{.*}} enum EnumConst {
 CHECK-NEXT:    LOW,
 CHECK-NEXT:    NORMAL,
 CHECK-NEXT:    HIGH
 CHECK-NEXT:}
 
-CHECK:      Type{{.*}} , name = "EnumEmpty", size = 4,  decl = SimpleTypesTest.cpp:25, compiler_type = {{.*}} enum EnumEmpty {
+CHECK:      Type{{.*}} , name = "EnumEmpty", size = 4,  decl = simpletypestest.cpp:25, compiler_type = {{.*}} enum EnumEmpty {
 CHECK-NEXT:}
 
-CHECK:      Type{{.*}} , name = "EnumUChar", size = 1,  decl = SimpleTypesTest.cpp:28, compiler_type = {{.*}} enum EnumUChar {
+CHECK:      Type{{.*}} , name = "EnumUChar", size = 1,  decl = simpletypestest.cpp:28, compiler_type = {{.*}} enum EnumUChar {
 CHECK-NEXT:    ON,
 CHECK-NEXT:    OFF,
 CHECK-NEXT:    AUTO
 CHECK-NEXT:}
 
 ; Note that `enum EnumClass` is tested instead of `enum class EnumClass`
-CHECK:      Type{{.*}} , name = "EnumClass", size = 4,  decl = SimpleTypesTest.cpp:32, compiler_type = {{.*}} enum EnumClass {
+CHECK:      Type{{.*}} , name = "EnumClass", size = 4,  decl = simpletypestest.cpp:32, compiler_type = {{.*}} enum EnumClass {
 CHECK-NEXT:    YES,
 CHECK-NEXT:    NO,
 CHECK-NEXT:    DEFAULT
 CHECK-NEXT:}
 
-CHECK:      Type{{.*}} , name = "EnumStruct", size = 4,  decl = SimpleTypesTest.cpp:35, compiler_type = {{.*}} enum EnumStruct {
+CHECK:      Type{{.*}} , name = "EnumStruct", size = 4,  decl = simpletypestest.cpp:35, compiler_type = {{.*}} enum EnumStruct {
 CHECK-NEXT:    red,
 CHECK-NEXT:    blue,
 CHECK-NEXT:    black
 CHECK-NEXT:}
 
 CHECK-DAG: {{^[0-9A-F]+}}:   CompileUnit{{[{]0x[0-9a-f]+[}]}}, language = "c++", file = '{{.*}}\SimpleTypesTest.cpp'

lit/SymbolFile/PDB/func-symbols.test

 REQUIRES: system-windows, msvc
-RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/FuncSymbolsTestMain.cpp /o %T/FuncSymbolsTestMain.cpp.obj
-RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/FuncSymbols.cpp /o %T/FuncSymbols.cpp.obj
+RUN: %build --compiler=clang-cl --mode=compile --arch=32 --nodefaultlib --source=%S/Inputs/FuncSymbolsTestMain.cpp --output=%T/FuncSymbolsTestMain.cpp.obj
+RUN: %build --compiler=clang-cl --mode=compile --arch=32 --nodefaultlib --source=%S/Inputs/FuncSymbols.cpp --output=%T/FuncSymbols.cpp.obj
 RUN: %msvc_link %T/FuncSymbolsTestMain.cpp.obj %T/FuncSymbols.cpp.obj /DEBUG /nodefaultlib /Entry:main /OUT:%T/FuncSymbolsTest.exe
 RUN: lldb-test symbols %T/FuncSymbolsTest.exe | FileCheck --check-prefix=CHECK-ONE %s
 RUN: lldb-test symbols %T/FuncSymbolsTest.exe | FileCheck --check-prefix=CHECK-TWO %s
 
 ; Link multiple objects
 ; In this test, We don't check demangled name of a mangled function.
 
 CHECK-ONE: Module [[MD:.*]]

lit/SymbolFile/PDB/function-nested-block.test

 REQUIRES: system-windows, lld
-RUN: %clang_cl /c /Zi %S/Inputs/FunctionNestedBlockTest.cpp /o %t.obj
-RUN: lld-link /debug:full /nodefaultlib /entry:main %t.obj /out:%t.exe
+RUN: %build --compiler=clang-cl --nodefaultlib --source=%S/Inputs/FunctionNestedBlockTest.cpp --output=%t.exe
 RUN: lldb-test symbols -find=function -file FunctionNestedBlockTest.cpp -line 4 %t.exe | FileCheck --check-prefix=CHECK-FUNCTION %s
 RUN: lldb-test symbols -find=block -file FunctionNestedBlockTest.cpp -line 4 %t.exe | FileCheck --check-prefix=CHECK-BLOCK %s
 
 CHECK-FUNCTION: Found 1 functions:
 CHECK-FUNCTION: name = "{{.*}}", mangled = "{{_?}}main"
 
 CHECK-BLOCK: Found 1 blocks:
 CHECK-BLOCK: Blocks: id = {{.*}}, range = {{.*}}
 CHECK-BLOCK:         id = {{.*}}, range = {{.*}}

lit/SymbolFile/PDB/pointers.test

 REQUIRES: system-windows, msvc
-RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/PointerTypeTest.cpp /o %T/PointerTypeTest.cpp.obj
-RUN: %msvc_link %T/PointerTypeTest.cpp.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/PointerTypeTest.cpp.exe
+RUN: %build --compiler=clang-cl --mode=compile --arch=32 --nodefaultlib --source=%S/Inputs/PointerTypeTest.cpp --output=%T/PointerTypeTest.cpp.obj
+RUN: %build --compiler=msvc --mode=link --arch=32 --nodefaultlib --source=%T/PointerTypeTest.cpp.obj --output=%T/PointerTypeTest.cpp.exe
 RUN: lldb-test symbols %T/PointerTypeTest.cpp.exe | FileCheck %s
 RUN: lldb-test symbols %T/PointerTypeTest.cpp.exe | FileCheck --check-prefix=MAIN-ST-F %s
 RUN: lldb-test symbols %T/PointerTypeTest.cpp.exe | FileCheck --check-prefix=MAIN-ST %s
 RUN: lldb-test symbols %T/PointerTypeTest.cpp.exe | FileCheck --check-prefix=MAIN %s
 RUN: lldb-test symbols %T/PointerTypeTest.cpp.exe | FileCheck --check-prefix=F %s
 
 CHECK: Module [[MOD:.*]]
 CHECK: {{^[0-9A-F]+}}:   CompileUnit{{[{]0x[0-9a-f]+[}]}}, language = "c++", file = '{{.*}}\PointerTypeTest.cpp'

lit/SymbolFile/PDB/type-quals.test

 REQUIRES: system-windows, msvc
-RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/TypeQualsTest.cpp /o %T/TypeQualsTest.cpp.obj
-RUN: %msvc_link %T/TypeQualsTest.cpp.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/TypeQualsTest.cpp.exe
+RUN: %build --compiler=clang-cl --mode=compile --arch=32 --nodefaultlib --source=%S/Inputs/TypeQualsTest.cpp --output=%T/TypeQualsTest.cpp.obj
+RUN: %build --compiler=msvc --mode=link --arch=32 --nodefaultlib --source=%T/TypeQualsTest.cpp.obj --output=%T/TypeQualsTest.cpp.exe
 RUN: lldb-test symbols %T/TypeQualsTest.cpp.exe | FileCheck %s
 
 CHECK: Module [[MOD:.*]]
 CHECK-DAG: {{^[0-9A-F]+}}: SymbolVendor ([[MOD]])
 CHECK:      Type{{.*}} , name = "const int", size = 4, compiler_type = {{.*}} const int
 CHECK:      Type{{.*}} , size = 4, compiler_type = {{.*}} const int *
 CHECK:      Type{{.*}} , size = 4, compiler_type = {{.*}} const int **const
 CHECK:      Type{{.*}} , size = 4, compiler_type = {{.*}} const int *const

lit/SymbolFile/PDB/typedefs.test

 REQUIRES: system-windows, msvc
-RUN: %clang_cl -m32 /Z7 /c /GS- %S/Inputs/SimpleTypesTest.cpp /o %T/SimpleTypesTest.cpp.typedefs.obj
-RUN: %msvc_link %T/SimpleTypesTest.cpp.typedefs.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/SimpleTypesTest.cpp.typedefs.exe
+RUN: %build --compiler=msvc --arch=32 --nodefaultlib --source=%S/Inputs/SimpleTypesTest.cpp --output=%T/SimpleTypesTest.cpp.typedefs.exe
 RUN: lldb-test symbols %T/SimpleTypesTest.cpp.typedefs.exe | FileCheck %s
 
 ; Generate 32-bit target
 
 ; FIXME: PDB does not have line information for typedef statements so source
 ; and line information for them is not tested.
 
 ; Note, types `long double` and `double` have same bit size in MSVC and there

lit/SymbolFile/PDB/variables.test

 REQUIRES: system-windows, msvc
-RUN: %clang_cl -m64 /Z7 /c /GS- %S/Inputs/VariablesTest.cpp /o %T/VariablesTest.cpp.obj
-RUN: %msvc_link %T/VariablesTest.cpp.obj /DEBUG /nodefaultlib /ENTRY:main /OUT:%T/VariablesTest.cpp.exe
+RUN: %build --compiler=clang-cl --mode=compile --arch=64 --nodefaultlib --source=%S/Inputs/VariablesTest.cpp --output=%T/VariablesTest.cpp.obj
+RUN: %build --compiler=msvc --mode=link --arch=64 --nodefaultlib --source=%T/VariablesTest.cpp.obj --output=%T/VariablesTest.cpp.exe
 RUN: lldb-test symbols %T/VariablesTest.cpp.exe | FileCheck %s
 
 CHECK: Module [[MOD:.*]]
 CHECK: SymbolVendor ([[MOD]])
 CHECK:     CompileUnit{{.*}}, language = "c++", file = '{{.*}}\VariablesTest.cpp'
 CHECK-DAG:   Variable{{.*}}, name = "g_IntVar"
 CHECK-SAME:  scope = global, location = {{.*}}, external
 CHECK-DAG:   Variable{{.*}}, name = "m_StaticClassMember"

 CHECK-SAME:    artificial
 CHECK-DAG:     Variable{{.*}}, name = "a"
 CHECK-SAME:    scope = parameter
 
 CHECK-DAG:   Function{[[FID4:.*]]}, mangled = ?Func@Class@@QEAAXXZ
 CHECK-NEXT:  Block{[[FID4]]}
 CHECK-DAG:     Variable{{.*}}, name = "this"
 CHECK-SAME:    scope = parameter
-CHECK-SAME:    artificial
+CHECK-SAME:    artificial