DWZ 05/06: DWZ test mode

[jankratochvil]

Summary

With the new lit mode any DWZ test mode regression will make the whole testcase to fail. So one can regression test LLDB against disabled DWZ (such as mv /usr/bin/dwz /usr/bin/dwz-x, that could be improved).

All DWZ patches are also applied in: git clone -b dwz git://git.jankratochvil.net/lldb

[labath]

The thing to be aware of with this testing strategy is that you will probably be the only person who will be able to run these tests and verify dwz functionality. If you don't setup a buildbot testing this then other developers will never even know if they have broken any dwz functionality in the future. While end-to-end tests are great and we need them, I think it would be worthwhile to think about other testing strategies. These can range from checking in a couple of siple dwz files and making sure we can extract information from them (see unittests/SymbolFile/PDB for code that does something similar) to reimplementing/upstreaming the utilities necessary to build dwz files ourselves (I am not sure what the dwz tool does, but the eu-strip and sepdebugcrcfix tools sound like they would be fairly easy to implement on top of existing llvm libraries).

The other thing that worries me is that your comments seem to indicate that not all dwz tests pass after this. Could you elaborate on the "Many ERRORs are correct" part and how you plan to rectify that?

The third thing I want to say is not really related to your patchset, but your comments seem to indicate that not all regular dwarf tests were passing for you to begin with. Is that true? Would you be willing to help me (probably in a separate thread) to identify the tests that are failing and why? I'd like to move us towards a state where the result of the tests does not depend on the system you're running on and people don't have to resort to differential comparisons.

packages/Python/lldbsuite/test/test_categories.py

42

The way we do these sorts of things is that we explicitly add the category to the list of skipped categories (take a look at checkLibcxxSupport in dotest.py).

[jankratochvil]

you will probably be the only person who will be able to run these tests and verify dwz functionality.

OK; really nobody runs RHEL/CentOS/Fedora? :-)

If you don't setup a buildbot testing

OK, I will try that. I was even running a test bot for GDB.

These can range from checking in a couple of siple dwz files

OK; although during development various bugs required various testfiles, single testfiles does not catch it all but sure better than nothing.

reimplementing/upstreaming the utilities necessary to build dwz files ourselves (I am not sure what the dwz tool does, but the eu-strip and sepdebugcrcfix tools sound like they would be fairly easy to implement on top of existing llvm libraries).

DWZ has its upstream sources.
eu-strip is just a more optimized (and more ELF-details preserving) variant of GNU binutils strip. It should be easy to slightly extend llvm-objcopy (as there is no llvm-strip). Or is it OK to use GNU binutils strip instead?
sepdebugcrcfix is a small utility currently inside upstream rpm package so that could be reimplemented in LLVM.
Although I admit both eu-strip and sepdebugcrcfix are used in the LLDB testsuite only to workaround some DWZ bug it fails for some reason to process unsplit debug info. I did not find DWZ code convenient enough to bugfix it there and its author Jakub Jelínek does not fix its reported bugs in recent years.

The other thing that worries me is that your comments seem to indicate that not all dwz tests pass after this. Could you elaborate on the "Many ERRORs are correct" part and how you plan to rectify that?

There is never any Success->Failure case for _dwarf->_dwz (there were such cases when I had bugs in my LLDB DWZ code). But there are Success->Error cases because in some cases DWZ tool cannot be used such as:

os command: make MAKE_DSYM=NO DWZ=YES ARCH=x86_64 CC="/quad/home/jkratoch/redhat/llvm-git-build-release/bin/clang-6.0"
with pid: 14709
stdout:
stderr: dwz: a.out.debug.dwz: .debug_info section not present
make[4]: *** [../../make/Makefile.rules:538: a.out] Error 1
retcode: 2
ERROR
Error when building test subject.

Unaware what to do with it. It could run the testcase without DWZ but then the _dwarf testcase would be needlessly run twice. Or there could be a list of tests to skip in DWZ mode. Or maybe some list of "ExpectedError" testcases (I guess one cannot defined "ExpectedError" now). This also means make check-lldb prints with DWZ a lot of error messages due to these "expected Errors".

your comments seem to indicate that not all regular dwarf tests were passing for you to begin with. Is that true?

Yes, I haven't investigated why yet. I am used from GDB to only diff results against regressions. RESULT-clean/ or RESULT-clean.tar.xz
BTW with installed system debug info files (/usr/lib/debug/**.debug) the LLDB testsuite times out and fails completely. LLDB could have some option like: gdb -iex 'set debug-file-directory /notexists'

Would you be willing to help me (probably in a separate thread) to identify the tests that are failing and why?

Yes but I would like to put more attention at the DWZ support upstreaming first as it is a LLDB blocker for Fedora/RHEL/CentOS.

[labath]

In D40475#940226, @jankratochvil wrote:

you will probably be the only person who will be able to run these tests and verify dwz functionality.

OK; really nobody runs RHEL/CentOS/Fedora? :-)

There aren't that many lldb developers... I wouldn't be surprised if the number of them having access to a red-hat machine is 1 (you).

If you don't setup a buildbot testing

OK, I will try that. I was even running a test bot for GDB.

These can range from checking in a couple of siple dwz files

OK; although during development various bugs required various testfiles, single testfiles does not catch it all but sure better than nothing.

reimplementing/upstreaming the utilities necessary to build dwz files ourselves (I am not sure what the dwz tool does, but the eu-strip and sepdebugcrcfix tools sound like they would be fairly easy to implement on top of existing llvm libraries).

DWZ has its upstream sources.
eu-strip is just a more optimized (and more ELF-details preserving) variant of GNU binutils strip. It should be easy to slightly extend llvm-objcopy (as there is no llvm-strip). Or is it OK to use GNU binutils strip instead?
sepdebugcrcfix is a small utility currently inside upstream rpm package so that could be reimplemented in LLVM.
Although I admit both eu-strip and sepdebugcrcfix are used in the LLDB testsuite only to workaround some DWZ bug it fails for some reason to process unsplit debug info. I did not find DWZ code convenient enough to bugfix it there and its author Jakub Jelínek does not fix its reported bugs in recent years.

It looks like it could be a fun project to reimplement dwz on top of llvm dwarf library, but I understand that would be a considerable detour for you.

The other thing that worries me is that your comments seem to indicate that not all dwz tests pass after this. Could you elaborate on the "Many ERRORs are correct" part and how you plan to rectify that?

There is never any Success->Failure case for _dwarf->_dwz (there were such cases when I had bugs in my LLDB DWZ code). But there are Success->Error cases because in some cases DWZ tool cannot be used such as:

os command: make MAKE_DSYM=NO DWZ=YES ARCH=x86_64 CC="/quad/home/jkratoch/redhat/llvm-git-build-release/bin/clang-6.0"
with pid: 14709
stdout:
stderr: dwz: a.out.debug.dwz: .debug_info section not present
make[4]: *** [../../make/Makefile.rules:538: a.out] Error 1
retcode: 2
ERROR
Error when building test subject.

Unaware what to do with it. It could run the testcase without DWZ but then the _dwarf testcase would be needlessly run twice. Or there could be a list of tests to skip in DWZ mode. Or maybe some list of "ExpectedError" testcases (I guess one cannot defined "ExpectedError" now). This also means make check-lldb prints with DWZ a lot of error messages due to these "expected Errors".

How many tests are we talking about here? Could you list them here? If these really don't produce any debug info, then we can probably just mark them as @no_debug_info_testcase (that prevents generating N versions of the same test) and be done with it. There is also the ability to skip a test for dwz explicitly.

Also, be aware that we don't really do a good job of distinguising errors and failures. Compilation problems are a common source of Errors, but other issues can come up as errors as well -- it just depends on details of how test is written.

your comments seem to indicate that not all regular dwarf tests were passing for you to begin with. Is that true?

Yes, I haven't investigated why yet. I am used from GDB to only diff results against regressions. RESULT-clean/ or RESULT-clean.tar.xz

Yea, sometimes you have to do that here as well, but it's not a place where we'd like to be.
Looking at the results, they don't seem that bad, really. Most of the failures are in data formatters -- I think a well-placed -fno-limit-debug-info would get those working. The other category is something in the system library confusing the test (e.g. can't find environ or an extra global variable c). I think we could get those resolved fairly quickly.

BTW, is this on current master? I was hoping that some of these (such as TestTopLevelExpressions) would be resolved already.

BTW with installed system debug info files (/usr/lib/debug/**.debug) the LLDB testsuite times out and fails completely. LLDB could have some option like: gdb -iex 'set debug-file-directory /notexists'

Yea, that would be useful, particularly as we want to isolate the test from system specifics. It would also be useful to figure out why does it hang, as it will probably affect your users as well. :)

[davide]

@jankratochvil I run Fedora for now on my desktop, but I don't have time and resources to devote to a buildbot.
IMHO, having a fedora bot would be a great thing for two reasons:

1. I found out Fedora exposes bugs that ubuntu doesn't (e.g. in symbol resolution, due to slightly different naming for globals in shared libraries)
2. We can improve coverage for Linux for lldb.

I think the first step to get this to work is to get the build green there, and then we can iterate, if you're interested.

[zturner]

In D40475#935615, @labath wrote:

The thing to be aware of with this testing strategy is that you will probably be the only person who will be able to run these tests and verify dwz functionality. If you don't setup a buildbot testing this then other developers will never even know if they have broken any dwz functionality in the future. While end-to-end tests are great and we need them, I think it would be worthwhile to think about other testing strategies. These can range from checking in a couple of siple dwz files and making sure we can extract information from them (see unittests/SymbolFile/PDB for code that does something similar) to reimplementing/upstreaming the utilities necessary to build dwz files ourselves (I am not sure what the dwz tool does, but the eu-strip and sepdebugcrcfix tools sound like they would be fairly easy to implement on top of existing llvm libraries).

The other thing that worries me is that your comments seem to indicate that not all dwz tests pass after this. Could you elaborate on the "Many ERRORs are correct" part and how you plan to rectify that?

The third thing I want to say is not really related to your patchset, but your comments seem to indicate that not all regular dwarf tests were passing for you to begin with. Is that true? Would you be willing to help me (probably in a separate thread) to identify the tests that are failing and why? I'd like to move us towards a state where the result of the tests does not depend on the system you're running on and people don't have to resort to differential comparisons.

With the forthcoming introduction of lldb-test, why don't we piggyback off of that? We already have a compiler, so use it to produce a dwz, and then use lldb-test to exercise something about it. It won't be as comprehensive as a full scale test, but it would be a nice complement. We'd have basic test coverage on all platforms right out of the gate.

[jankratochvil]

In D40475#940462, @labath wrote:

It looks like it could be a fun project to reimplement dwz on top of llvm dwarf library, but I understand that would be a considerable detour for you.

I think it would be best to drop DWZ, IIRC its gain is not much bigger than -fdebug-types-section and it is incompatible with some 3rd party debug info tools as DWZ did not really became a standard. I just want to make LLDB compatible with it.

How many tests are we talking about here? Could you list them here?

These tests ERROR due to dwz: a.out.debug.dwz: .debug_info section not present:

functionalities/breakpoint/breakpoint_ignore_count/TestBreakpointIgnoreCount.py
functionalities/data-formatter/data-formatter-ptr-to-array/TestPtrToArrayFormatting.py
functionalities/data-formatter/ptr_ref_typedef/TestPtrRef2Typedef.py
functionalities/data-formatter/typedef_array/TestTypedefArray.py
functionalities/frame_var_scope/TestFrameVariableScope.py
functionalities/signal/TestSendSignal.py
lang/cpp/bool/TestCPPBool.py
lang/cpp/frame-var-anon-unions/TestFrameVariableAnonymousUnions.py
lang/cpp/lambdas/TestLambdas.py
lang/cpp/nsimport/TestCppNsImport.py
linux/builtin_trap/TestBuiltinTrap.py
python_api/rdar-12481949/Test-rdar-12481949.py
sample_test/TestSampleInlineTest.py

If these really don't produce any debug info, then we can probably just mark them as @no_debug_info_testcase

I do not see this keyword in the tree.

Looking at the results, they don't seem that bad, really. Most of the failures are in data formatters -- I think a well-placed -fno-limit-debug-info would get those working. The other category is something in the system library confusing the test (e.g. can't find environ or an extra global variable c). I think we could get those resolved fairly quickly.

BTW, is this on current master? I was hoping that some of these (such as TestTopLevelExpressions) would be resolved already.

No, it was from 26th. I have refreshed the files with today's GIT hash 6513119854292f1992c95073e99d55bf88456adb = SVN 319402.

It would also be useful to figure out why does it hang, as it will probably affect your users as well. :)

I expect because LLDB does not have any Linux DWARF index support, it has only some Apple OSX index support. GDB's .debug_index is not usable for LLDB as it does not contain DIE offsets. I would like to implement DWARF-5 .debug_names for it later.

[jankratochvil]

It now includes also unittests/SymbolFile/DWZ/ which should PASS even on systems without DWZ tool(s).

[jankratochvil]

The way we do these sorts of things is that we explicitly add the category to the list of skipped categories (take a look at checkLibcxxSupport in dotest.py).

I have implemented dwz category like gmodules but even when checkDWZSupport makes configuration.skipCategories.append("dwz") the DWZ-variants of tests are still being run. I will investigate it more.

[davide]

fwiw, you don't need unit tests or python tests to implement this.
If I understand the feature correctly you can probably extend lldb-test (which is run as part of check-lldb-lit).
Up to you though.

[jankratochvil]

In D40475#1018951, @davide wrote:

fwiw, you don't need unit tests or python tests to implement this.
If I understand the feature correctly you can probably extend lldb-test (which is run as part of check-lldb-lit).

I do not understand much yet all the testing frameworks LLDB implements. Could you suggest what you find redundant? So far I do want to run the whole testsuite also in dwz mode.
I understand unittests/SymbolFile/DWZ/SymbolFileDWZTests.cpp could be written also as an extension of tools/lldb-test. But then I do not see why. The unit test of mine looks fine to me.

[jankratochvil]

I have implemented dwz category like gmodules but even when checkDWZSupport makes configuration.skipCategories.append("dwz") the DWZ-variants of tests are still being run. I will investigate it more.

This is fixed now (forgotten update of some items copied from gmodules).

[jankratochvil]

With the new testsuite run it currently reports failures due to missing .debug_info sections in some testcases, I need to make it false-regression-free.

[jankratochvil]

There is no DWZ-tool wrapping script as discussed at: D48782

[jankratochvil]

Replaced by D96241.

packages/Python/lldbsuite/test/dotest.py

def checkLibcxxSupport():
     result, reason = canRunLibcxxTests()
     if result:
         return # libc++ supported
     if "libc++" in configuration.categoriesList:
         return # libc++ category explicitly requested, let it run.
     print("Libc++ tests will not be run because: " + reason)
     configuration.skipCategories.append("libc++")
 
+def canRunDWZTests():
+    from lldbsuite.test import lldbplatformutil
+
+    platform = lldbplatformutil.getPlatform()
+
+    if platform == "linux":
+        import distutils.spawn
+
+        if not os.access("/usr/lib/rpm/sepdebugcrcfix", os.X_OK):
+            return False, "Unable to find /usr/lib/rpm/sepdebugcrcfix"
+        if distutils.spawn.find_executable("eu-strip") is None:
+            return False, "Unable to find executable eu-strip"
+        if distutils.spawn.find_executable("dwz") is None:
+            return False, "Unable to find executable dwz"
+        return True, "/usr/lib/rpm/sepdebugcrcfix, eu-strip and dwz found"
+
+    return False, "Don't know how to build with DWZ on %s" % platform
+
+def checkDWZSupport():
+    result, reason = canRunDWZTests()
+    if result:
+        return # dwz supported
+    if "dwz" in configuration.categoriesList:
+        return # dwz category explicitly requested, let it run.
+    print("dwz tests will not be run because: " + reason)
+    configuration.skipCategories.append("dwz")
+
 def run_suite():
     # On MacOS X, check to make sure that domain for com.apple.DebugSymbols defaults
     # does not exist before proceeding to running the test suite.
     if sys.platform.startswith("darwin"):
         checkDsymForUUIDIsNotOn()
 
     #
     # Start the actions by first parsing the options while setting up the test
def run_suite():
     import lldbsuite.test.lldbutil as lldbutil
     build_dir = configuration.test_build_dir
     lldbutil.mkdir_p(build_dir)
 
     target_platform = lldb.DBG.GetSelectedPlatform().GetTriple().split('-')[2]
 
     checkLibcxxSupport()
 
+    checkDWZSupport()
+
     # Don't do debugserver tests on everything except OS X.
     configuration.dont_do_debugserver_test = "linux" in target_platform or "freebsd" in target_platform or "windows" in target_platform
 
     # Don't do lldb-server (llgs) tests on anything except Linux.
     configuration.dont_do_llgs_test = not ("linux" in target_platform)
 
     #
     # Walk through the testdirs while collecting tests.

packages/Python/lldbsuite/test/lldbinline.py

class InlineTest(TestBase):
     @add_test_categories(["gmodules"])
     def __test_with_gmodules(self):
         self.using_dsym = False
         self.BuildMakefile()
         self.build()
         self.do_test()
     __test_with_gmodules.debug_info = "gmodules"
 
+    def __test_with_dwz(self):
+        self.using_dsym = False
+        self.BuildMakefile()
+        self.build()
+        self.do_test()
+    __test_with_dwz.debug_info = "dwz"
+
     def execute_user_command(self, __command):
         exec(__command, globals(), locals())
 
     def do_test(self):
         exe = self.getBuildArtifact("a.out")
         source_files = [f for f in os.listdir(self.getSourceDir())
                         if source_type(f)]
         target = self.dbg.CreateTarget(exe)
def MakeInlineTest(__file, __globals, decorators=None):
     if test_categories.is_supported_on_platform(
             "dwo", target_platform, configuration.compiler):
         test.test_with_dwo = ApplyDecoratorsToFunction(
             test._InlineTest__test_with_dwo, decorators)
     if test_categories.is_supported_on_platform(
             "gmodules", target_platform, configuration.compiler):
         test.test_with_gmodules = ApplyDecoratorsToFunction(
             test._InlineTest__test_with_gmodules, decorators)
+    if test_categories.is_supported_on_platform(
+            "dwz", target_platform, configuration.compiler):
+        test.test_with_dwz = ApplyDecoratorsToFunction(
+            test._InlineTest__test_with_dwz, decorators)
 
     # Add the test case to the globals, and hide InlineTest
     __globals.update({test_name: test})
 
     # Keep track of the original test filename so we report it
     # correctly in test results.
     test.test_filename = __file
     return test

packages/Python/lldbsuite/test/lldbtest.py

def buildGModules(
             raise Exception("NO_DEBUG_INFO_TESTCASE must build with buildDefault")
 
         module = builder_module()
         dictionary = lldbplatformutil.finalize_build_dictionary(dictionary)
         if not module.buildGModules(self, architecture, compiler,
                                     dictionary, clean, testdir, testname):
             raise Exception("Don't know how to build binary with gmodules")
 
+    def buildDWZ(
+            self,
+            architecture=None,
+            compiler=None,
+            dictionary=None,
+            clean=True):
+        """Platform specific way to build binaries with dwz optimizer."""
+        testdir = self.mydir
+        testname = self.getBuildDirBasename()
+        if self.getDebugInfo() != "dwz":
+            raise Exception("NO_DEBUG_INFO_TESTCASE must build with buildDefault")
+
+        module = builder_module()
+        dictionary = lldbplatformutil.finalize_build_dictionary(dictionary)
+        if not module.buildDWZ(self, architecture, compiler,
+                               dictionary, clean, testdir, testname):
+            raise Exception("Don't know how to build binary with dwz")
+
     def buildGo(self):
         """Build the default go binary.
         """
         exe = self.getBuildArtifact("a.out")
         system([[which('go'), 'build -gcflags "-N -l" -o %s main.go' % exe]])
 
     def signBinary(self, binary_path):
         if sys.platform.startswith("darwin"):
def __new__(cls, name, bases, attrs):
                     @wraps(attrvalue)
                     def gmodules_test_method(self, attrvalue=attrvalue):
                         return attrvalue(self)
                     gmodules_method_name = attrname + "_gmodules"
                     gmodules_test_method.__name__ = gmodules_method_name
                     gmodules_test_method.debug_info = "gmodules"
                     newattrs[gmodules_method_name] = gmodules_test_method
 
+                if "dwz" in supported_categories:
+                    @decorators.add_test_categories(["dwz"])
+                    @wraps(attrvalue)
+                    def dwz_test_method(self, attrvalue=attrvalue):
+                        return attrvalue(self)
+                    dwz_method_name = attrname + "_dwz"
+                    dwz_test_method.__name__ = dwz_method_name
+                    dwz_test_method.debug_info = "dwz"
+                    newattrs[dwz_method_name] = dwz_test_method
+
             else:
                 newattrs[attrname] = attrvalue
         return super(
             LLDBTestCaseFactory,
             cls).__new__(
             cls,
             name,
             bases,
def build(
         elif self.getDebugInfo() == "dwarf":
             return self.buildDwarf(architecture, compiler, dictionary, clean)
         elif self.getDebugInfo() == "dwo":
             return self.buildDwo(architecture, compiler, dictionary,
                                  clean)
         elif self.getDebugInfo() == "gmodules":
             return self.buildGModules(architecture, compiler, dictionary,
                                       clean)
+        elif self.getDebugInfo() == "dwz":
+            return self.buildDWZ(architecture, compiler, dictionary, clean)
         else:
             self.fail("Can't build for debug info: %s" % self.getDebugInfo())
 
     def run_platform_command(self, cmd):
         platform = self.dbg.GetSelectedPlatform()
         shell_command = lldb.SBPlatformShellCommand(cmd)
         err = platform.Run(shell_command)
         return (err, shell_command.GetStatus(), shell_command.GetOutput())

packages/Python/lldbsuite/test/make/Makefile.rules

 # DYLIB_ONLY := YES
 #
 # Also might be of interest:
 # FRAMEWORK_INCLUDES (Darwin only) :=
 # CFLAGS_EXTRAS :=
 # LD_EXTRAS :=
 # SPLIT_DEBUG_SYMBOLS := YES
 # CROSS_COMPILE :=
+# DWZ := YES
 #
 # And test/functionalities/archives/Makefile:
 # MAKE_DSYM := NO
 # ARCHIVE_NAME := libfoo.a
 # ARCHIVE_C_SOURCES := a.c b.c
 
 # Uncomment line below for debugging shell commands
 # SHELL = /bin/sh -x
ifneq "$(OS)" "Darwin"
     override AR = $(ARCHIVER)
 endif
 
 ifdef PIE
     LDFLAGS += -pie
 endif
 
 #----------------------------------------------------------------------
+# Make the dSYM file from the executable if $(DWZ) = "YES"
+#----------------------------------------------------------------------
+ifeq "$(DWZ)" "YES"
+  dwz_strip = \
+    eu-strip --remove-comment -f "$(1).debug" "$(1)" \
+    && cp "$(1).debug" "$(1).debug.dup" \
+    && dwz -m "$(1).debug.dwz" "$(1).debug" "$(1).debug.dup" \
+    && /usr/lib/rpm/sepdebugcrcfix . "$(1)"
+else
+  dwz_strip =
+endif
+
+#----------------------------------------------------------------------
 # Windows specific options
 #----------------------------------------------------------------------
 ifeq "$(OS)" "Windows_NT"
   ifneq (,$(findstring clang,$(CC)))
     # Clang for Windows doesn't support C++ Exceptions
     CXXFLAGS += -fno-exceptions
     CXXFLAGS += -D_HAS_EXCEPTIONS=0
 

 
 #----------------------------------------------------------------------
 # Compile the executable from all the objects.
 #----------------------------------------------------------------------
 ifneq "$(DYLIB_NAME)" ""
 ifeq "$(DYLIB_ONLY)" ""
 $(EXE) : $(OBJECTS) $(ARCHIVE_NAME) $(DYLIB_FILENAME)
   $(LD) $(OBJECTS) $(ARCHIVE_NAME) -L. -l$(DYLIB_NAME) $(LDFLAGS) -o "$(EXE)"
+  $(call dwz_strip,$(EXE))
 else
 EXE = $(DYLIB_FILENAME)
 endif
 else
 $(EXE) : $(OBJECTS) $(ARCHIVE_NAME)
   $(LD) $(OBJECTS) $(LDFLAGS) $(ARCHIVE_NAME) -o "$(EXE)"
+  $(call dwz_strip,$(EXE))
 endif
 
 #----------------------------------------------------------------------
 # Make the dSYM file from the executable if $(MAKE_DSYM) != "NO"
 #----------------------------------------------------------------------
 $(DSYM) : $(EXE)
 ifeq "$(OS)" "Darwin"
 ifneq "$(MAKE_DSYM)" "NO"

 endif
 endif
 else
   $(LD) $(DYLIB_OBJECTS) $(LDFLAGS) -shared -o "$(DYLIB_FILENAME)"
 ifeq "$(SPLIT_DEBUG_SYMBOLS)" "YES"
   $(OBJCOPY) --only-keep-debug "$(DYLIB_FILENAME)" "$(DYLIB_FILENAME).debug"
   $(OBJCOPY) --strip-debug --add-gnu-debuglink="$(DYLIB_FILENAME).debug" "$(DYLIB_FILENAME)" "$(DYLIB_FILENAME)"
 endif
+  $(call dwz_strip,$(DYLIB_FILENAME))
 endif
 
 #----------------------------------------------------------------------
 # Make the precompiled header and compile C++ sources against it
 #----------------------------------------------------------------------
 
 #ifneq "$(PCH_OUTPUT)" ""
 $(PCH_OUTPUT) : $(PCH_CXX_SOURCE)

 .PHONY: clean
 dsym: $(DSYM)
 all:  $(EXE) $(DSYM)
 clean::
   $(RM) -rf $(OBJECTS) $(PREREQS) $(PREREQS:.d=.d.tmp) $(DWOS) $(ARCHIVE_NAME) $(ARCHIVE_OBJECTS) $(CLANG_MODULE_CACHE_DIR)
 ifneq "$(DYLIB_NAME)" ""
   $(RM) -r $(DYLIB_FILENAME).dSYM
   $(RM) $(DYLIB_OBJECTS) $(DYLIB_PREREQS) $(DYLIB_PREREQS:.d=.d.tmp) $(DYLIB_DWOS) $(DYLIB_FILENAME) $(DYLIB_FILENAME).debug
+  $(RM) $(DYLIB_FILENAME).debug.dup $(DYLIB_FILENAME).debug.dwz
 endif
 ifneq "$(PCH_OUTPUT)" ""
   $(RM) $(PCH_OUTPUT)
 endif
 ifneq "$(DSYM)" ""
   $(RM) -r "$(DSYM)"
 endif
 ifeq "$(OS)" "Windows_NT"
 # http://llvm.org/pr24589
   IF EXIST "$(EXE)" del "$(EXE)"
   $(RM) $(wildcard *.manifest *.pdb *.ilk)
 ifneq "$(DYLIB_NAME)" ""
   $(RM) $(DYLIB_NAME).lib $(DYLIB_NAME).exp
 endif
 else
-  $(RM) "$(EXE)"
+  $(RM) "$(EXE)" "$(EXE).debug" "$(EXE).debug.dup" "$(EXE).debug.dwz"
 endif
 
 #----------------------------------------------------------------------
 # From http://blog.melski.net/tag/debugging-makefiles/
 # 
 # Usage: make print-CC print-CXX print-LD
 #----------------------------------------------------------------------
 print-%:
   @echo '$*=$($*)'
   @echo '  origin = $(origin $*)'
   @echo '  flavor = $(flavor $*)'
   @echo '   value = $(value  $*)'
 
 ### Local Variables: ###
 ### mode:makefile ###
 ### End: ###

packages/Python/lldbsuite/test/plugins/builder_base.py

commands.append(getMake(testdir, testname) +
                      getArchSpec(architecture),
                      getCCSpec(compiler),
                      getCmdLine(dictionary)])
 
     lldbtest.system(commands, sender=sender)
     # True signifies that we can handle building with gmodules.
     return True
 
+def buildDWZ(
+        sender=None,
+        architecture=None,
+        compiler=None,
+        dictionary=None,
+        clean=True,
+        testdir=None,
+        testname=None):
+    """Build the binaries with type units (type in  a .debug_types section)."""
+    commands = []
+    if clean:
+        commands.append(getMake(testdir, testname) +
+                        ["clean", getCmdLine(dictionary)])
+    # dwz has a bug being unable to process non-separated debug info.
+    commands.append(getMake(testdir, testname) +
+                    ["MAKE_DSYM=NO",
+                     "DWZ=YES",
+                     getArchSpec(architecture),
+                     getCCSpec(compiler),
+                     getCmdLine(dictionary)])
+
+    lldbtest.system(commands, sender=sender)
+    # True signifies that we can handle building dwz.
+    return True
 
 def cleanup(sender=None, dictionary=None):
     """Perform a platform-specific cleanup after the test."""
     return True

packages/Python/lldbsuite/test/test_categories.py

 
 # Third-party modules
 
 # LLDB modules
 from lldbsuite.support import gmodules
 
 
 debug_info_categories = [
-    'dwarf', 'dwo', 'dsym', 'gmodules'
+    'dwarf', 'dwo', 'dsym', 'gmodules', 'dwz'
 ]
 
 all_categories = {
     'dataformatters': 'Tests related to the type command and the data formatters subsystem',
     'dwarf': 'Tests that can be run with DWARF debug information',
     'dwo': 'Tests that can be run with DWO debug information',
     'dsym': 'Tests that can be run with DSYM debug information',
     'gmodules': 'Tests that can be run with -gmodules debug information',
+    'dwz': 'Tests using DWZ and its DWARF partial units',
     'expression': 'Tests related to the expression parser',
     'libc++': 'Test for libc++ data formatters',
     'objc': 'Tests related to the Objective-C programming language support',
     'pyapi': 'Tests related to the Python API',
     'basic_process': 'Basic process execution sniff tests.',
     'cmdline': 'Tests related to the LLDB command-line interface',
     'dyntype': 'Tests related to dynamic type support',
     'stresstest': 'Tests related to stressing lldb limits',
     'flakey': 'Flakey test cases, i.e. tests that do not reliably pass at each execution',
     'lldb-mi': 'lldb-mi tests',
     'darwin-log': 'Darwin log tests',
     'watchpoint': 'Watchpoint-related tests',
 }
 
 
 def unique_string_match(yourentry, list):
     candidate = None
     for item in list:
         if not item.startswith(yourentry):
             continue
         if candidate:
             return None
         candidate = item
     return candidate
 
 
 def is_supported_on_platform(category, platform, compiler_path):
     if category == "dwo":
         # -gsplit-dwarf is not implemented by clang on Windows.
         return platform in ["linux", "freebsd"]
     elif category == "dsym":
         return platform in ["darwin", "macosx", "ios", "watchos", "tvos", "bridgeos"]
     elif category == "gmodules":
         # First, check to see if the platform can even support gmodules.
         if platform not in ["linux", "freebsd", "darwin", "macosx", "ios", "watchos", "tvos", "bridgeos"]:
             return False
         return gmodules.is_compiler_clang_with_gmodules(compiler_path)
+    elif category == "dwz":
+        return platform in ["linux"]
     return True
 
 
 def validate(categories, exact_match):
     """
     For each category in categories, ensure that it's a valid category (if exact_match is false,
     unique prefixes are also accepted). If a category is invalid, print a message and quit.
        If all categories are valid, return the list of categories. Prefixes are expanded in the

unittests/SymbolFile/CMakeLists.txt

 add_subdirectory(DWARF)
+add_subdirectory(DWZ)
 if (LLVM_ENABLE_DIA_SDK)
   add_subdirectory(PDB)
 endif()

unittests/SymbolFile/DWZ/CMakeLists.txt

+add_lldb_unittest(SymbolFileDWZTests
+  SymbolFileDWZTests.cpp
+
+  LINK_LIBS
+    lldbCore
+    lldbHost
+    lldbSymbol
+    lldbPluginSymbolFileDWARF
+    lldbUtilityHelpers
+    lldbPluginObjectFileELF
+    lldbPluginSymbolVendorELF
+  LINK_COMPONENTS
+    Support
+  )
+
+set(test_inputs
+   dwztest.out
+   dwztest.debug
+   dwztest.debug.dwz)
+
+add_unittest_inputs(SymbolFileDWZTests "${test_inputs}")

unittests/SymbolFile/DWZ/Inputs/dwztest.c

+// gcc -Wall -g -o dwztest.out dwztest.c; eu-strip --remove-comment -f dwztest.debug dwztest.out; cp -p dwztest.debug dwztest.debug.dup; dwz -m dwztest.debug.dwz dwztest.debug dwztest.debug.dup;rm dwztest.debug.dup; /usr/lib/rpm/sepdebugcrcfix . dwztest.out 
+
+struct StructMovedToDWZCommonFile {
+  int i1, i2, i3, i4, i5, i6, i7, i8, i9;
+} VarWithStructMovedToDWZCommonFile;
+static const int sizeof_StructMovedToDWZCommonFile = sizeof(struct StructMovedToDWZCommonFile);
+int main() {
+  return sizeof_StructMovedToDWZCommonFile;
+}

unittests/SymbolFile/DWZ/SymbolFileDWZTests.cpp

+//===-- SymbolFileDWZTests.cpp ----------------------------------*- C++ -*-===//
+//
+//                     The LLVM Compiler Infrastructure
+//
+// This file is distributed under the University of Illinois Open Source
+// License. See LICENSE.TXT for details.
+//
+//===----------------------------------------------------------------------===//
+
+#include "gtest/gtest.h"
+
+#include "TestingSupport/TestUtilities.h"
+
+#include "Plugins/SymbolFile/DWARF/SymbolFileDWARF.h"
+#include "Plugins/ObjectFile/ELF/ObjectFileELF.h"
+#include "Plugins/SymbolVendor/ELF/SymbolVendorELF.h"
+#include "lldb/Core/Module.h"
+#include "lldb/Host/HostInfo.h"
+#include "lldb/Symbol/ClangASTContext.h"
+#include "lldb/Symbol/SymbolVendor.h"
+#include "lldb/Utility/ArchSpec.h"
+#include "lldb/Utility/FileSpec.h"
+
+#if defined(_MSC_VER)
+#include "lldb/Host/windows/windows.h"
+#include <objbase.h>
+#endif
+
+#include <algorithm>
+
+using namespace lldb_private;
+
+class SymbolFileDWZTests : public testing::Test {
+public:
+  void SetUp() override {
+// Initialize and TearDown the plugin every time, so we get a brand new
+// AST every time so that modifications to the AST from each test don't
+// leak into the next test.
+#if defined(_MSC_VER)
+    ::CoInitializeEx(nullptr, COINIT_MULTITHREADED);
+#endif
+
+    HostInfo::Initialize();
+    SymbolFileDWARF::Initialize();
+    ClangASTContext::Initialize();
+    ObjectFileELF::Initialize();
+    SymbolVendorELF::Initialize();
+
+    m_dwztest_out = GetInputFilePath("dwztest.out");
+  }
+
+  void TearDown() override {
+    SymbolVendorELF::Terminate();
+    ObjectFileELF::Terminate();
+    ClangASTContext::Terminate();
+    SymbolFileDWARF::Terminate();
+    HostInfo::Terminate();
+
+#if defined(_MSC_VER)
+    ::CoUninitialize();
+#endif
+  }
+
+protected:
+  std::string m_dwztest_out;
+};
+
+TEST_F(SymbolFileDWZTests, TestSimpleClassTypes) {
+  FileSpec fspec(m_dwztest_out.c_str(), false);
+  ArchSpec aspec("x86_64-pc-linux");
+  lldb::ModuleSP module = std::make_shared<Module>(fspec, aspec);
+
+  SymbolVendor *plugin = module->GetSymbolVendor();
+  EXPECT_NE(nullptr, plugin);
+  SymbolFile *symfile = plugin->GetSymbolFile();
+  EXPECT_NE(nullptr, symfile);
+  EXPECT_EQ(symfile->GetPluginName(), SymbolFileDWARF::GetPluginNameStatic());
+  SymbolContext sc;
+  llvm::DenseSet<SymbolFile *> searched_files;
+  TypeMap results;
+  EXPECT_EQ(1u,
+      symfile->FindTypes(sc, ConstString("StructMovedToDWZCommonFile"), nullptr,
+          false, 0, searched_files, results));
+  EXPECT_EQ(1u, results.GetSize());
+  lldb::TypeSP udt_type = results.GetTypeAtIndex(0);
+  EXPECT_EQ(ConstString("StructMovedToDWZCommonFile"), udt_type->GetName());
+  CompilerType compiler_type = udt_type->GetForwardCompilerType();
+  EXPECT_TRUE(ClangASTContext::IsClassType(compiler_type.GetOpaqueQualType()));
+}