This is an archive of the discontinued LLVM Phabricator instance.

[libc++] Unbreak the debug mode
ClosedPublic

Authored by ldionne on Jan 14 2021, 1:43 PM.

Details

Reviewers
rnk
Group Reviewers
Restricted Project
Commits
rG68dba7eae1df: [libc++] Unbreak the debug mode
Summary

When the Debug mode is enabled, we disable extern declarations because we
don't want to use the functions compiled in the library, which might not
have had the debug mode enabled when built. However, some extern declarations
need to be kept, because code correctness depends on it.

31e820378b8a removed those declarations, which had the unintended
consequence of breaking the debug build. This commit fixes that by
re-introducing a separate macro for the required extern declarations,
and adds a comment so that we don't fall into that trap in the future.

Diff Detail

Event Timeline

ldionne created this revision.Jan 14 2021, 1:43 PM
ldionne requested review of this revision.Jan 14 2021, 1:43 PM
Herald added a project: Restricted Project. · View Herald TranscriptJan 14 2021, 1:43 PM
Herald added a reviewer: Restricted Project. · View Herald Transcript

See the discussion in https://reviews.llvm.org/rG1a92de00 for more context.

@rnk Are you able to provide a reproducer we could add to the test suite?

rnk added a comment.Jan 14 2021, 2:29 PM

Thanks, I think this will solve the issue.

It's also worth noting that there is one _LIBCPP_ASSERT in locale:
https://github.com/llvm/llvm-project/blob/b894a9fb237345db64d14ce3881d3195e124df0d/libcxx/include/locale#L4017
But, missing that assert seems preferable to locale registration failure.

Here is the reproducer I came up with. I made a shared library and exe from these two cpp files and linked them with the shell script:

$ cat facet_lib.cpp 
#include <sstream>
std::string __attribute__((visibility("default"))) numToString(int x);
std::string numToString(int x) {
  std::stringstream ss;
  ss << x;
  return ss.str();
}

$ cat facet.cpp 
#include <string>
#include <cstdio>
std::string __attribute__((visibility("default"))) numToString(int x);
int main() {
  std::string s = numToString(42);
  printf("should be 42: %s\n", s.c_str());
}

$ cat linkit.sh 
#!/bin/bash
set -e
clang++ -g -D_LIBCPP_DEBUG=1 -fvisibility=hidden -stdlib=libc++ -c facet_lib.cpp -fPIC -o facet_lib.o
clang++ -stdlib=libc++ -fPIC facet_lib.o  -shared -o libfacet_lib.so
clang++ -g -D_LIBCPP_DEBUG=1 -fvisibility=hidden -stdlib=libc++ facet.cpp libfacet_lib.so -o facet
LD_LIBRARY_PATH=.:lib ./facet

$ ./linkit.sh 
should be 42: 
# output is wrong

objdump can confirm that there is a hidden ::id global in the facet_lib.o, which I think is the issue (double facet registration):

$ llvm-objdump -t facet_lib.o | c++filt | grep num_put.*::id
0000000000000000  w    O .bss._ZNSt3__17num_putIcNS_19ostreambuf_iteratorIcNS_11char_traitsIcEEEEE2idE  0000000000000010 .hidden std::__1::num_put<char, std::__1::ostreambuf_iterator<char, std::__1::char_traits<char> > >::id

After I apply this patch locally and relink, when I check the object file, I see that the id static data member is not defined in this object:

$ llvm-objdump -t facet_lib.o | c++filt | grep num_put.*::id
0000000000000000         *UND*  0000000000000000 std::__1::num_put<char, std::__1::ostreambuf_iterator<char, std::__1::char_traits<char> > >::id

It is provided by the libc++ library instead, which is correct.

curdeius added inline comments.
libcxx/include/__config
887–895

Isn't it possible that both _LIBCPP_DEBUG_LEVEL >= 1 and defined(_LIBCPP_DISABLE_EXTERN_TEMPLATE) are true, and so _LIBCPP_EXTERN_TEMPLATE will be defined twice?

ldionne updated this revision to Diff 317044.Jan 15 2021, 12:13 PM

Add test, fix potential for duplicate macro definition.

ldionne updated this revision to Diff 317045.Jan 15 2021, 12:14 PM

Add comment to explain the purpose of the unit test.

ldionne updated this revision to Diff 317370.Jan 18 2021, 8:48 AM

Add -fPIC when compiling the tests.

ldionne updated this revision to Diff 317396.Jan 18 2021, 11:25 AM

Fix test on GCC and without localization support.

This revision was not accepted when it landed; it landed in state Needs Review.Jan 19 2021, 11:15 AM
This revision was automatically updated to reflect the committed changes.

Hi @ldionne,

here is a problem with extern-templates.sh.cpp test on the windows-to-linux ARM cross builders

http://lab.llvm.org:8011/#/builders/60/builds/1666
http://lab.llvm.org:8011/#/builders/119/builds/1962

the problem looks like

/tmp/libcxx.3WqvVHTiFB/t.tmp.exe: error while loading shared libraries: C:/buildbot/as-builder-2/x-aarch64/build/runtimes/runtimes-bins/libcxx/test/libcxx/debug/Output/extern-templates.sh.cpp.dir/t.tmp.lib: cannot open shared object file: No such file or directory

clang stores a full path to t.tmp.lib into the test executable and this path is unavailable when the test gets running on the remote host (over ssh).

A possible fix is linking the test within local folder and pass a short library path into the linker such as ./t.tmp.lib. I.e. replace the second RUN line with something like

RUN: cd %T && %{cxx} %{flags} %{compile_flags} %s ./t.tmp.lib %{link_flags} -fPIC -DTU2 -D_LIBCPP_DEBUG=1 -fvisibility=hidden -o %t.exe

I have tested these changes on my local build system and it works, but I'm not sure how it will work on the other systems.

Hi @ldionne,

[...]

A possible fix is linking the test within local folder and pass a short library path into the linker such as ./t.tmp.lib. I.e. replace the second RUN line with something like

RUN: cd %T && %{cxx} %{flags} %{compile_flags} %s ./t.tmp.lib %{link_flags} -fPIC -DTU2 -D_LIBCPP_DEBUG=1 -fvisibility=hidden -o %t.exe

I have tested these changes on my local build system and it works, but I'm not sure how it will work on the other systems.

Thanks for the detailed explanation. This should be fixed by 90407b16b1d3e38f1360b6a24ceab801ab9cefc1.