This is an archive of the discontinued LLVM Phabricator instance.

[DebugInfo][OPT] Fixing a couple of DI duplication bugs of CloneModule
ClosedPublic

Authored by rtereshin on Apr 12 2018, 2:40 PM.

Details

Summary

As demonstrated by the regression tests added in this patch, the
following cases are valid cases:

  1. A Function with no DISubprogram attached, but various debug info related to its instructions, coming, for instance, from an inlined function, also defined somewhere else in the same module;
  2. ... or coming exclusively from the functions inlined and eliminated from the module entirely.

The ValueMap shared between CloneFunctionInto calls within CloneModule
needs to contain identity mappings for all of the DISubprogram's to
prevent them from being duplicated by MapMetadata / RemapInstruction
calls, this is achieved via DebugInfoFinder collecting all the
DISubprogram's. However, CloneFunctionInto was missing calls into
DebugInfoFinder for functions w/o DISubprogram's attached, but still
referring DISubprogram's from within (case 1). This patch fixes that.

The fix above, however, exposes another issue: if a module contains a
DISubprogram referenced only indirectly from other debug info
metadata, but not attached to any Function defined within the module
(case 2), cloning such a module causes a DICompileUnit duplication: it
will be moved in indirecty via a DISubprogram by DebugInfoFinder first
(because of the first bug fix described above), without being
self-mapped within the shared ValueMap, and then will be copied during
named metadata cloning. So this patch makes sure DebugInfoFinder
visits DICompileUnit's referenced from DISubprogram's as it goes w/o
re-processing llvm.dbg.cu list over and over again for every function
cloned, and makes sure that CloneFunctionInto self-maps
DICompileUnit's referenced from the entire function, not just its own
DISubprogram attached that may also be missing.

The most convenient way of tesing CloneModule I found is to rely on
CloneModule call from opt -run-twice, instead of writing tedious
unit tests. That feature has a couple of properties that makes it hard
to use for this purpose though:

  1. CloneModule doesn't copy source filename, making opt -run-twice report it as a difference.
  2. opt -run-twice does the second run on the original module, not its clone, making the result of cloning completely invisible in opt's actual output with and without -run-twice both, which directly contradicts opt -run-twices own error message.

This patch fixes this as well.

Diff Detail

Repository
rL LLVM

Event Timeline

rtereshin created this revision.Apr 12 2018, 2:40 PM
rtereshin added inline comments.Apr 12 2018, 2:56 PM
include/llvm/IR/DebugInfo.h
70

extracted from processModule and extended to sync with the CloneBasicBlock's version of the events.

84–85

extracted from processModule

89

NFC, just put in alphabetical order.

lib/IR/DebugInfo.cpp
111

NFC

151

extracted from processModule and extended to sync with the CloneBasicBlock's version of the events.

206

Fixing case 2 (see the summary).

lib/Transforms/Utils/CloneFunction.cpp
64

NFC

178

Fixing case 1 (see the summary).

210

Fixing case 2 (see the summary).

212

NFC

lib/Transforms/Utils/CloneModule.cpp
53

Reducing the number of false-failure reports from opt -run-twice

tools/opt/opt.cpp
775

Moving from

opt:
  Module -> first run -> visible output

opt -run-twice:
  Module -> CloneModule -> first run -> discarded
        \-> second run -> visible output

scheme to

opt:
  Module -> first run -> visible output

opt -run-twice:
  Module -> first run -> discarded
        \-> CloneModule -> second run -> visible output

to

  1. Make sure that opt -run-twice only discards the output of the (full) run that is easily reproducible with a visible output (by just dropping -run-twice in this case)
  2. opt could be used to debug CloneModule
788

NFC

Could you split out the NFC changes into a separate commit (no need for a review) and update the diff? It will make reviewing the actual changes much easier. Thanks!

include/llvm/IR/DebugInfo.h
69

The \brief is unnecessary, we have autobrief enabled in our Doxyfile.

vsk added a subscriber: vsk.Apr 12 2018, 3:52 PM
vsk added inline comments.
lib/Transforms/Utils/CloneFunction.cpp
59

Might be nice to rewrite this as 'for (Instruction *I : *BB)' while we're changing things.

rtereshin marked 2 inline comments as done.Apr 12 2018, 5:41 PM
rtereshin added inline comments.
include/llvm/IR/DebugInfo.h
69

I'm removing \briefs from the entire header in the second NFC-commit you asked for then, thanks!

lib/Transforms/Utils/CloneFunction.cpp
59

Sure, doing that in the follow-up NFC-commit @aprantl asked me for, thanks!

rtereshin updated this revision to Diff 142303.Apr 12 2018, 5:49 PM
rtereshin marked 2 inline comments as done.

The patch is split into 2 commits, FC and a follow-up NFC not included in the review, as requested by @aprantl.
All the comments so far are asking for NFC, addressed in the NFC-commit.

rtereshin added inline comments.Apr 12 2018, 5:54 PM
lib/IR/DebugInfo.cpp
109

Syncing this up with what CloneBasicBlock is doing, extracted as a separate method and re-used in the follow-up NFC commit.

Basically looks good, a few more comments would be nice (see inline).

lib/IR/DebugInfo.cpp
113

Can you add a comment explaining why this is necessary (like what you wrote for the phabricator review)?

139

else llvm_unreachable("unexpected imported entity type")?

206

Let's just add a comment into the code here explaining why this call is necessary.

test/DebugInfo/X86/clone-module-2.ll
11

Can you add a comment explaining what behavior is being tested here?
I.e.: why would running opt invoke the cloner? Is there perhaps a more targeted pass that we could run instead?

tools/opt/opt.cpp
767

Ah I see.

rtereshin added inline comments.Apr 13 2018, 10:22 AM
lib/IR/DebugInfo.cpp
113

Sure, will do.

139

Good idea, I'll try that.

tools/opt/opt.cpp
767

Yeah, I think the fact that -run-twice is invoking cloner is self-evident enough to not elaborate on that, we already have regression tests that rely on it. They aren't very honest, btw, as prior to this patch the opt's output is the original module, not the clone, so their FileCheck's are practically useless. Still works as opt compares the output of the first and second run, so if they differ, it will return with a non-zero return code and the test will still catch it, but anyway that kind of thing is just misleading. I think with these changes it's a tad better as opt will at least output the clone.

As for commenting what tests actually test I think it still needs to be done, thanks.

rtereshin marked 8 inline comments as done.Apr 13 2018, 11:47 AM
rtereshin added inline comments.
lib/IR/DebugInfo.cpp
113

I'm adding the comment to processSubprograms implementation at processCompileUnit's call-site, as the processCompileUnit being a separate method is just an obvious decomposition / DRY thing.

139

Yep, passed the tests locally just fine, doing that.

206

Done.

test/DebugInfo/X86/clone-module-2.ll
11

Done, see below.

tools/opt/opt.cpp
767

I've added the comments.

rtereshin marked 2 inline comments as done.

Addressing the comments.

rtereshin added inline comments.Apr 13 2018, 11:50 AM
lib/IR/DebugInfo.cpp
91

This is all replaced by a processCompileUnit's call in the follow-up NFC commit, as the pieces are identical.

aprantl accepted this revision.Apr 13 2018, 1:19 PM
This revision is now accepted and ready to land.Apr 13 2018, 1:19 PM
This revision was automatically updated to reflect the committed changes.
alberto_magni added inline comments.
llvm/trunk/lib/IR/DebugInfo.cpp
141 ↗(On Diff #142468)

This commit is causing failures in my SPEC2006 tests (xalancbmk among others) with -flto=thin and -g.
The unreachable in this line gets triggered.

This is because the if-cascade above does not handle the case for DIGlobalVariable.
Probably the code that handles GlobalVariables at the top of the function needs to be replicated in one if-case here too.

rtereshin added inline comments.Apr 16 2018, 1:26 PM
llvm/trunk/lib/IR/DebugInfo.cpp
141 ↗(On Diff #142468)

Hi @alberto_magni,

Is there a chance you could extract a regression test from one of those SPEC2006 tests?

Thanks!
Roman

aprantl added inline comments.Apr 16 2018, 2:22 PM
llvm/trunk/lib/IR/DebugInfo.cpp
141 ↗(On Diff #142468)

If it's just a DIGlobalVariable appearing in that list, you should be able to construct one manually without much trouble.

aprantl added inline comments.Apr 16 2018, 2:47 PM
llvm/trunk/lib/IR/DebugInfo.cpp
141 ↗(On Diff #142468)
$ cat /tmp/using.cpp
namespace s {
  int i;
}

using s::i;

int f() { return i; }

Using declarations are represented as imported entities.

aprantl added inline comments.Apr 16 2018, 2:48 PM
llvm/trunk/lib/IR/DebugInfo.cpp
141 ↗(On Diff #142468)

That also means that we'll need to accept pretty much anything in that list :-(

rtereshin added inline comments.Apr 16 2018, 2:51 PM
llvm/trunk/lib/IR/DebugInfo.cpp
141 ↗(On Diff #142468)

Thanks, I'll try that!

So, we gotta remove the unreachable, I suppose?

aprantl added inline comments.Apr 16 2018, 2:53 PM
llvm/trunk/lib/IR/DebugInfo.cpp
141 ↗(On Diff #142468)

Yes, but perhaps think about what else might appear in a using statement that we would need to process here and also whether it can be dangerous to process something twice, because it is reachable via the imported entities and some other means (as in my example above) so we don't end up with duplicates / infinite loops / etc.

rtereshin added inline comments.Apr 16 2018, 3:22 PM
llvm/trunk/lib/IR/DebugInfo.cpp
141 ↗(On Diff #142468)

I see.

I don't know off the top of my head how to turn this "global variable as an imported entity" example into a test that would undeniably break one of the DebugInfoFinder users given that llvm_unreachable is removed. DebugInfoFinder tracks types, and CloneFunction self-maps them for some reason, while DIGlobalVariable references a type, so maybe it may cause some debug info duplication if CloneModule is called?

There are also ModuleDebugInfoPrinter and StripDeadDebugInfo module passes, not sure what exactly they expect from DebugInfoFinder.

That being said, if approached meticulously, this is going to take some time.

Do you think we need to remove llvm_unreachable first right now before we do anything else?

Let's remove the unreachable and add a simple testcase that would trigger it as a first step.