This is an archive of the discontinued LLVM Phabricator instance.

[libc++] Create a small DSL for defining Lit features and parameters
ClosedPublic

Authored by ldionne on Apr 17 2020, 9:49 AM.

Details

Summary

This allows defining Lit features that can be enabled or disabled based
on compiler support, and parameters that are passed on the command line.

The main benefits are:

  • Feature detection is entirely based on the substitutions provided in the TestingConfig object, which is simpler and decouples it from the complicated compiler emulation infrastructure.
  • The syntax is declarative, which makes it easy to see what features and parameters are accepted by the test suite. This is significantly less entangled than the current config.py logic.
  • Since feature detection is based on substitutions, it works really well on top of the new format, and custom Lit configurations can be created easily without being based on config.py.

Diff Detail

Event Timeline

ldionne created this revision.Apr 17 2020, 9:49 AM
Herald added a project: Restricted Project. · View Herald TranscriptApr 17 2020, 9:49 AM
Herald added a reviewer: Restricted Project. · View Herald Transcript

This is in itself a NFC change -- only https://reviews.llvm.org/D78382 has a real impact on the config. If there are no comments, I'll go ahead and ship this tomorrow so I can see if the added test passes on Windows. It is expected to, but if it doesn't, that will be valuable information to improve this patch.

EricWF added a subscriber: EricWF.Apr 23 2020, 12:30 PM
EricWF added inline comments.
libcxx/test/libcxx/selftest/dsl/dsl.sh.py
34

A short docstring for each class would be a nice addition.

72

I think the python style guidlines wants 2 spaces between class definitions.

92

Can we watch a more specific exception as to not swallow other errors that may occur?

libcxx/utils/libcxx/test/dsl.py
51

I don't know that this is correct with GCC.
And it doesn't handle -Wno-foo flags.

IIRC the compiler may only report unknown flags as error if another error occurs during compilation.

51

Also, just use os.devnul as the input file, because pipes don't work with popen.

70

Last I checked locales didn't always have a single canonical spelling. Are you handling this case?

ldionne updated this revision to Diff 259986.Apr 24 2020, 2:01 PM
ldionne marked 7 inline comments as done.

Address review comments. Also, harden checking of locales so it should work
on Windows too.

libcxx/utils/libcxx/test/dsl.py
51

I don't know that this is correct with GCC. And it doesn't handle -Wno-foo flags.

It is correct with GCC outside of -Wno-foo flags, but you are correct that it doesn't handle -Wno-foo flags correctly. As-is, it will report that all -Wno-foo flags are supported on GCC. I went ahead and filed https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94745 because I thought it was a GCC bug, but Jonathan Wakely nicely explained to me how this was by design. He also suggested using g++ -Q --help=warnings (we should have that for Clang!).

I was going to go ahead and implement the g++ -Q --help=warnings solution, but then it occurred to me that if GCC reports they "support" a -Wno-foo flag when they don't (and they simply ignore it), then why is it a problem for the test suite to add that warning flag? In other words, why would we want to *not* add a flag that gets ignored? I believe that behavior is more in line with their intent when they made that design choice.

For this reason, I'll suggest we leave it as-is, but if you feel strongly about it, please explain and I can use g++ -Q --help=warnings on GCC to check for supported -Wno-foo flags.

70

This will be handled through trying various different locale names when setting up each locale's feature. This will be in an upcoming patch.

ldionne accepted this revision as: Restricted Project.Apr 27 2020, 6:55 AM

I'm going to go ahead and ship this now so I can see whether it breaks anything.

This revision was not accepted when it landed; it landed in state Needs Review.Apr 27 2020, 6:57 AM
This revision was automatically updated to reflect the committed changes.

I'm going to go ahead and ship this now so I can see whether it breaks anything.

This breaks Fedora 32 (x86_64) building with the system clang/lld (version 10.0):

[0+1] Running all regression tests
llvm-lit: /home/dave/s/lp/libcxxabi/test/lit.cfg:64: note: Using configuration variant: libcxxabi
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:307: note: inferred use_system_cxx_lib as: None
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:333: note: inferred use_clang_verify as: True
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:342: note: enabling thread-safety annotations
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:487: note: inferred language dialect as: c++2a
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale fr_FR.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale ru_RU.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale zh_CN.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale fr_CA.ISO8859-1 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale cs_CZ.ISO8859-2 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:389: note: inferred long_tests as: True
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:153: note: Using compiler: /usr/bin/clang++
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:154: note: Using flags: ['-v']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:158: note: Using compile flags: ['-Werror=thread-safety', '-DLIBCXXABI_NO_TIMER', '-D_LIBCPP_ENABLE_CXX17_REMOVED_UNEXPECTED_FUNCTIONS', '-funwind-tables', '-std=c++2a', '-nostdinc++', '-I/home/dave/s/lp/libcxx/include', '-I/home/dave/s/lp/libcxxabi/include', '-DSTDC_FORMAT_MACROS', '-DSTDC_LIMIT_MACROS', '-DSTDC_CONSTANT_MACROS', '-I/home/dave/s/lp/libcxx/test/support', '-ftemplate-depth=270']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:161: note: Using warnings: ['-Wall', '-Wextra', '-D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER', '-Wuser-defined-warnings', '-Wshadow', '-Wno-unused-command-line-argument', '-Wno-attributes', '-Wno-pessimizing-move', '-Wno-c++11-extensions', '-Wno-user-defined-literals', '-Wno-noexcept-type', '-Wsign-compare', '-Wunused-variable', '-Wunused-parameter', '-Wunreachable-code', '-Wno-unused-local-typedef']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:162: note: Using link flags: ['-L/tmp/_update_lc/r/./lib64', '-Wl,-rpath,/tmp/_update_lc/r/./lib64', '-L/tmp/_update_lc/r/./lib64', '-Wl,-rpath,/tmp/_update_lc/r/./lib64', '-nodefaultlibs', '/tmp/_update_lc/r/./lib64/libc++.a', '/tmp/_update_lc/r/./lib64/libc++abi.a', '-lm', '-lgcc_s', '-lgcc', '-lpthread', '-lrt', '-lc', '-lgcc_s', '-lgcc']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:164: note: Using available_features: ['long_tests', 'fdelayed-template-parsing', 'libc++', 'c++2a', 'diagnose-if-support', 'has-fblocks', '-faligned-allocation', 'clang-10', 'clang', 'locale.en_US.UTF-8', 'linux', 'clang-10.0.0', '-fsized-deallocation', 'thread-safety', 'libcxxabi-has-system-unwinder', 'fcoroutines-ts', 'modules-support', 'clang-10.0']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:170: note: Adding environment variables: {}
llvm-lit: /home/dave/s/lp/libcxxabi/test/lit.cfg:77: note: Using the new libc++ testing format
llvm-lit: /home/dave/s/lp/libcxx/test/lit.cfg:49: note: Using configuration variant: libcxx
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:307: note: inferred use_system_cxx_lib as: None
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:333: note: inferred use_clang_verify as: True
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:342: note: enabling thread-safety annotations
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:487: note: inferred language dialect as: c++2a
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale fr_FR.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale ru_RU.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale zh_CN.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale fr_CA.ISO8859-1 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale cs_CZ.ISO8859-2 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:389: note: inferred long_tests as: True
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:153: note: Using compiler: /usr/bin/clang++
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:154: note: Using flags: ['-v']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:158: note: Using compile flags: ['-Werror=thread-safety', '-std=c++2a', '-include', '/home/dave/s/lp/libcxx/test/support/nasty_macros.h', '-nostdinc++', '-I/home/dave/s/lp/libcxx/include', '-I/tmp/_update_lc/r/projects/libcxx/include/c++build', '-D
STDC_FORMAT_MACROS', '-DSTDC_LIMIT_MACROS', '-DSTDC_CONSTANT_MACROS', '-I/home/dave/s/lp/libcxx/test/support', '-ftemplate-depth=270']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:161: note: Using warnings: ['-Wall', '-Wextra', '-D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER', '-Wuser-defined-warnings', '-Wshadow', '-Wno-unused-command-line-argument', '-Wno-attributes', '-Wno-pessimizing-move', '-Wno-c++11-extensions', '-Wno-user-defined-literals', '-Wno-noexcept-type', '-Wsign-compare', '-Wunused-variable', '-Wunused-parameter', '-Wunreachable-code', '-Wno-unused-local-typedef']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:162: note: Using link flags: ['-L/tmp/_update_lc/r/./lib64', '-Wl,-rpath,/tmp/_update_lc/r/./lib64', '-nodefaultlibs', '-lc++experimental', '/tmp/_update_lc/r/./lib64/libc++.a', '-lc++abi', '-lm', '-lgcc_s', '-lgcc', '-lpthread', '-lrt', '-lc', '-lgcc_s', '-lgcc']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:164: note: Using available_features: ['modules-support', 'clang-10.0', 'fdelayed-template-parsing', 'diagnose-if-support', 'has-fblocks', 'clang-10', 'libatomic', '-faligned-allocation', 'clang', 'locale.en_US.UTF-8', 'c++experimental', 'long_tests', 'libc++', 'c++2a', 'linux', 'clang-10.0.0', 'thread-safety', 'fcoroutines-ts', '-fsized-deallocation']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:170: note: Adding environment variables: {}
llvm-lit: /home/dave/s/lp/libcxx/test/lit.cfg:62: note: Using the new libc++ testing format
llvm-lit: /tmp/_update_lc/r/bin/../../../../home/dave/s/lp/llvm/utils/lit/lit/TestingConfig.py:100: fatal: unable to parse config file '/home/dave/s/lp/libcxx/test/libcxx/selftest/dsl/lit.local.cfg', traceback: Traceback (most recent call last):

File "/tmp/_update_lc/r/bin/../../../../home/dave/s/lp/llvm/utils/lit/lit/TestingConfig.py", line 89, in load_from_path
  exec(compile(data, path, 'exec'), cfg_globals, None)
File "/home/dave/s/lp/libcxx/test/libcxx/selftest/dsl/lit.local.cfg", line 13, in <module>
  escaped = [(k.replace('%{', '%{escaped_'), base64.b64encode(v)) for (k, v) in config.substitutions]
File "/home/dave/s/lp/libcxx/test/libcxx/selftest/dsl/lit.local.cfg", line 13, in <listcomp>
  escaped = [(k.replace('%{', '%{escaped_'), base64.b64encode(v)) for (k, v) in config.substitutions]
File "/usr/lib64/python3.8/base64.py", line 58, in b64encode
  encoded = binascii.b2a_base64(s, newline=False)

TypeError: a bytes-like object is required, not 'str'

FAILED: CMakeFiles/check-all

tpopp added a subscriber: tpopp.Apr 28 2020, 4:38 AM

I'm going to go ahead and ship this now so I can see whether it breaks anything.

This breaks Fedora 32 (x86_64) building with the system clang/lld (version 10.0):

[0+1] Running all regression tests
llvm-lit: /home/dave/s/lp/libcxxabi/test/lit.cfg:64: note: Using configuration variant: libcxxabi
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:307: note: inferred use_system_cxx_lib as: None
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:333: note: inferred use_clang_verify as: True
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:342: note: enabling thread-safety annotations
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:487: note: inferred language dialect as: c++2a
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale fr_FR.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale ru_RU.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale zh_CN.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale fr_CA.ISO8859-1 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale cs_CZ.ISO8859-2 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:389: note: inferred long_tests as: True
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:153: note: Using compiler: /usr/bin/clang++
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:154: note: Using flags: ['-v']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:158: note: Using compile flags: ['-Werror=thread-safety', '-DLIBCXXABI_NO_TIMER', '-D_LIBCPP_ENABLE_CXX17_REMOVED_UNEXPECTED_FUNCTIONS', '-funwind-tables', '-std=c++2a', '-nostdinc++', '-I/home/dave/s/lp/libcxx/include', '-I/home/dave/s/lp/libcxxabi/include', '-DSTDC_FORMAT_MACROS', '-DSTDC_LIMIT_MACROS', '-DSTDC_CONSTANT_MACROS', '-I/home/dave/s/lp/libcxx/test/support', '-ftemplate-depth=270']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:161: note: Using warnings: ['-Wall', '-Wextra', '-D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER', '-Wuser-defined-warnings', '-Wshadow', '-Wno-unused-command-line-argument', '-Wno-attributes', '-Wno-pessimizing-move', '-Wno-c++11-extensions', '-Wno-user-defined-literals', '-Wno-noexcept-type', '-Wsign-compare', '-Wunused-variable', '-Wunused-parameter', '-Wunreachable-code', '-Wno-unused-local-typedef']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:162: note: Using link flags: ['-L/tmp/_update_lc/r/./lib64', '-Wl,-rpath,/tmp/_update_lc/r/./lib64', '-L/tmp/_update_lc/r/./lib64', '-Wl,-rpath,/tmp/_update_lc/r/./lib64', '-nodefaultlibs', '/tmp/_update_lc/r/./lib64/libc++.a', '/tmp/_update_lc/r/./lib64/libc++abi.a', '-lm', '-lgcc_s', '-lgcc', '-lpthread', '-lrt', '-lc', '-lgcc_s', '-lgcc']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:164: note: Using available_features: ['long_tests', 'fdelayed-template-parsing', 'libc++', 'c++2a', 'diagnose-if-support', 'has-fblocks', '-faligned-allocation', 'clang-10', 'clang', 'locale.en_US.UTF-8', 'linux', 'clang-10.0.0', '-fsized-deallocation', 'thread-safety', 'libcxxabi-has-system-unwinder', 'fcoroutines-ts', 'modules-support', 'clang-10.0']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:170: note: Adding environment variables: {}
llvm-lit: /home/dave/s/lp/libcxxabi/test/lit.cfg:77: note: Using the new libc++ testing format
llvm-lit: /home/dave/s/lp/libcxx/test/lit.cfg:49: note: Using configuration variant: libcxx
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:307: note: inferred use_system_cxx_lib as: None
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:333: note: inferred use_clang_verify as: True
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:342: note: enabling thread-safety annotations
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:487: note: inferred language dialect as: c++2a
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale fr_FR.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale ru_RU.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale zh_CN.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale fr_CA.ISO8859-1 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale cs_CZ.ISO8859-2 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:389: note: inferred long_tests as: True
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:153: note: Using compiler: /usr/bin/clang++
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:154: note: Using flags: ['-v']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:158: note: Using compile flags: ['-Werror=thread-safety', '-std=c++2a', '-include', '/home/dave/s/lp/libcxx/test/support/nasty_macros.h', '-nostdinc++', '-I/home/dave/s/lp/libcxx/include', '-I/tmp/_update_lc/r/projects/libcxx/include/c++build', '-D
STDC_FORMAT_MACROS', '-DSTDC_LIMIT_MACROS', '-DSTDC_CONSTANT_MACROS', '-I/home/dave/s/lp/libcxx/test/support', '-ftemplate-depth=270']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:161: note: Using warnings: ['-Wall', '-Wextra', '-D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER', '-Wuser-defined-warnings', '-Wshadow', '-Wno-unused-command-line-argument', '-Wno-attributes', '-Wno-pessimizing-move', '-Wno-c++11-extensions', '-Wno-user-defined-literals', '-Wno-noexcept-type', '-Wsign-compare', '-Wunused-variable', '-Wunused-parameter', '-Wunreachable-code', '-Wno-unused-local-typedef']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:162: note: Using link flags: ['-L/tmp/_update_lc/r/./lib64', '-Wl,-rpath,/tmp/_update_lc/r/./lib64', '-nodefaultlibs', '-lc++experimental', '/tmp/_update_lc/r/./lib64/libc++.a', '-lc++abi', '-lm', '-lgcc_s', '-lgcc', '-lpthread', '-lrt', '-lc', '-lgcc_s', '-lgcc']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:164: note: Using available_features: ['modules-support', 'clang-10.0', 'fdelayed-template-parsing', 'diagnose-if-support', 'has-fblocks', 'clang-10', 'libatomic', '-faligned-allocation', 'clang', 'locale.en_US.UTF-8', 'c++experimental', 'long_tests', 'libc++', 'c++2a', 'linux', 'clang-10.0.0', 'thread-safety', 'fcoroutines-ts', '-fsized-deallocation']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:170: note: Adding environment variables: {}
llvm-lit: /home/dave/s/lp/libcxx/test/lit.cfg:62: note: Using the new libc++ testing format
llvm-lit: /tmp/_update_lc/r/bin/../../../../home/dave/s/lp/llvm/utils/lit/lit/TestingConfig.py:100: fatal: unable to parse config file '/home/dave/s/lp/libcxx/test/libcxx/selftest/dsl/lit.local.cfg', traceback: Traceback (most recent call last):

File "/tmp/_update_lc/r/bin/../../../../home/dave/s/lp/llvm/utils/lit/lit/TestingConfig.py", line 89, in load_from_path
  exec(compile(data, path, 'exec'), cfg_globals, None)
File "/home/dave/s/lp/libcxx/test/libcxx/selftest/dsl/lit.local.cfg", line 13, in <module>
  escaped = [(k.replace('%{', '%{escaped_'), base64.b64encode(v)) for (k, v) in config.substitutions]
File "/home/dave/s/lp/libcxx/test/libcxx/selftest/dsl/lit.local.cfg", line 13, in <listcomp>
  escaped = [(k.replace('%{', '%{escaped_'), base64.b64encode(v)) for (k, v) in config.substitutions]
File "/usr/lib64/python3.8/base64.py", line 58, in b64encode
  encoded = binascii.b2a_base64(s, newline=False)

TypeError: a bytes-like object is required, not 'str'

FAILED: CMakeFiles/check-all

Just a note, this seems due to incompatibilities between python2 and python3. Python2 expects strings like given here while Python3 expects the bytes-like object.

I'm going to go ahead and ship this now so I can see whether it breaks anything.

This breaks Fedora 32 (x86_64) building with the system clang/lld (version 10.0):

[0+1] Running all regression tests
llvm-lit: /home/dave/s/lp/libcxxabi/test/lit.cfg:64: note: Using configuration variant: libcxxabi
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:307: note: inferred use_system_cxx_lib as: None
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:333: note: inferred use_clang_verify as: True
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:342: note: enabling thread-safety annotations
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:487: note: inferred language dialect as: c++2a
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale fr_FR.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale ru_RU.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale zh_CN.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale fr_CA.ISO8859-1 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale cs_CZ.ISO8859-2 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:389: note: inferred long_tests as: True
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:153: note: Using compiler: /usr/bin/clang++
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:154: note: Using flags: ['-v']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:158: note: Using compile flags: ['-Werror=thread-safety', '-DLIBCXXABI_NO_TIMER', '-D_LIBCPP_ENABLE_CXX17_REMOVED_UNEXPECTED_FUNCTIONS', '-funwind-tables', '-std=c++2a', '-nostdinc++', '-I/home/dave/s/lp/libcxx/include', '-I/home/dave/s/lp/libcxxabi/include', '-DSTDC_FORMAT_MACROS', '-DSTDC_LIMIT_MACROS', '-DSTDC_CONSTANT_MACROS', '-I/home/dave/s/lp/libcxx/test/support', '-ftemplate-depth=270']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:161: note: Using warnings: ['-Wall', '-Wextra', '-D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER', '-Wuser-defined-warnings', '-Wshadow', '-Wno-unused-command-line-argument', '-Wno-attributes', '-Wno-pessimizing-move', '-Wno-c++11-extensions', '-Wno-user-defined-literals', '-Wno-noexcept-type', '-Wsign-compare', '-Wunused-variable', '-Wunused-parameter', '-Wunreachable-code', '-Wno-unused-local-typedef']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:162: note: Using link flags: ['-L/tmp/_update_lc/r/./lib64', '-Wl,-rpath,/tmp/_update_lc/r/./lib64', '-L/tmp/_update_lc/r/./lib64', '-Wl,-rpath,/tmp/_update_lc/r/./lib64', '-nodefaultlibs', '/tmp/_update_lc/r/./lib64/libc++.a', '/tmp/_update_lc/r/./lib64/libc++abi.a', '-lm', '-lgcc_s', '-lgcc', '-lpthread', '-lrt', '-lc', '-lgcc_s', '-lgcc']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:164: note: Using available_features: ['long_tests', 'fdelayed-template-parsing', 'libc++', 'c++2a', 'diagnose-if-support', 'has-fblocks', '-faligned-allocation', 'clang-10', 'clang', 'locale.en_US.UTF-8', 'linux', 'clang-10.0.0', '-fsized-deallocation', 'thread-safety', 'libcxxabi-has-system-unwinder', 'fcoroutines-ts', 'modules-support', 'clang-10.0']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:170: note: Adding environment variables: {}
llvm-lit: /home/dave/s/lp/libcxxabi/test/lit.cfg:77: note: Using the new libc++ testing format
llvm-lit: /home/dave/s/lp/libcxx/test/lit.cfg:49: note: Using configuration variant: libcxx
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:307: note: inferred use_system_cxx_lib as: None
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:333: note: inferred use_clang_verify as: True
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:342: note: enabling thread-safety annotations
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:487: note: inferred language dialect as: c++2a
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale fr_FR.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale ru_RU.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale zh_CN.UTF-8 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale fr_CA.ISO8859-1 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/target_info.py:84: warning: The locale cs_CZ.ISO8859-2 is not supported by your platform. Some tests will be unsupported.
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:389: note: inferred long_tests as: True
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:153: note: Using compiler: /usr/bin/clang++
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:154: note: Using flags: ['-v']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:158: note: Using compile flags: ['-Werror=thread-safety', '-std=c++2a', '-include', '/home/dave/s/lp/libcxx/test/support/nasty_macros.h', '-nostdinc++', '-I/home/dave/s/lp/libcxx/include', '-I/tmp/_update_lc/r/projects/libcxx/include/c++build', '-D
STDC_FORMAT_MACROS', '-DSTDC_LIMIT_MACROS', '-DSTDC_CONSTANT_MACROS', '-I/home/dave/s/lp/libcxx/test/support', '-ftemplate-depth=270']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:161: note: Using warnings: ['-Wall', '-Wextra', '-D_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER', '-Wuser-defined-warnings', '-Wshadow', '-Wno-unused-command-line-argument', '-Wno-attributes', '-Wno-pessimizing-move', '-Wno-c++11-extensions', '-Wno-user-defined-literals', '-Wno-noexcept-type', '-Wsign-compare', '-Wunused-variable', '-Wunused-parameter', '-Wunreachable-code', '-Wno-unused-local-typedef']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:162: note: Using link flags: ['-L/tmp/_update_lc/r/./lib64', '-Wl,-rpath,/tmp/_update_lc/r/./lib64', '-nodefaultlibs', '-lc++experimental', '/tmp/_update_lc/r/./lib64/libc++.a', '-lc++abi', '-lm', '-lgcc_s', '-lgcc', '-lpthread', '-lrt', '-lc', '-lgcc_s', '-lgcc']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:164: note: Using available_features: ['modules-support', 'clang-10.0', 'fdelayed-template-parsing', 'diagnose-if-support', 'has-fblocks', 'clang-10', 'libatomic', '-faligned-allocation', 'clang', 'locale.en_US.UTF-8', 'c++experimental', 'long_tests', 'libc++', 'c++2a', 'linux', 'clang-10.0.0', 'thread-safety', 'fcoroutines-ts', '-fsized-deallocation']
llvm-lit: /home/dave/s/lp/libcxx/utils/libcxx/test/config.py:170: note: Adding environment variables: {}
llvm-lit: /home/dave/s/lp/libcxx/test/lit.cfg:62: note: Using the new libc++ testing format
llvm-lit: /tmp/_update_lc/r/bin/../../../../home/dave/s/lp/llvm/utils/lit/lit/TestingConfig.py:100: fatal: unable to parse config file '/home/dave/s/lp/libcxx/test/libcxx/selftest/dsl/lit.local.cfg', traceback: Traceback (most recent call last):

File "/tmp/_update_lc/r/bin/../../../../home/dave/s/lp/llvm/utils/lit/lit/TestingConfig.py", line 89, in load_from_path
  exec(compile(data, path, 'exec'), cfg_globals, None)
File "/home/dave/s/lp/libcxx/test/libcxx/selftest/dsl/lit.local.cfg", line 13, in <module>
  escaped = [(k.replace('%{', '%{escaped_'), base64.b64encode(v)) for (k, v) in config.substitutions]
File "/home/dave/s/lp/libcxx/test/libcxx/selftest/dsl/lit.local.cfg", line 13, in <listcomp>
  escaped = [(k.replace('%{', '%{escaped_'), base64.b64encode(v)) for (k, v) in config.substitutions]
File "/usr/lib64/python3.8/base64.py", line 58, in b64encode
  encoded = binascii.b2a_base64(s, newline=False)

TypeError: a bytes-like object is required, not 'str'

FAILED: CMakeFiles/check-all

Just a note, this seems due to incompatibilities between python2 and python3. Python2 expects strings like given here while Python3 expects the bytes-like object.

Right. Python 2.x was EOLed January 1st, 2020. See: https://www.python.org/dev/peps/pep-0373/#release-schedule

I'm going to revert this due to lack of Python 3 testing.

Right. Python 2.x was EOLed January 1st, 2020. See: https://www.python.org/dev/peps/pep-0373/#release-schedule

I'm going to revert this due to lack of Python 3 testing.

Thanks for the heads up. Note that the LLVM community decided to keep supporting Python 2 until 2021 AFAICT, which means that we need to support both right now. That's a poor decision IMO because it leads to these situations.

Regardless, I added support for Python3 and I'm going to recommit this. Thanks!