This is an archive of the discontinued LLVM Phabricator instance.

Disallow duplication of imported entities
ClosedPublic

Authored by aaboud on Mar 4 2016, 4:42 AM.

Details

Summary

Fixed DIBuilder to verify that same imported entity will not be added twice to the "imports" list of the DICompileUnit.

Diff Detail

Event Timeline

aaboud updated this revision to Diff 49815.Mar 4 2016, 4:42 AM
aaboud retitled this revision from to Disallow duplication of imported entities.
aaboud updated this object.
aaboud added reviewers: aprantl, dblaikie.
aaboud added a subscriber: llvm-commits.
aaboud added a comment.Mar 4 2016, 4:44 AM

I will add the following test: Clang/test/Codegen/debug-info-imported-entity.cpp

// RUN: %clang_cc1 -triple x86_64-unk-unk -o - -emit-llvm -debug-info-kind=limited %s | FileCheck %s

namespace std { class A; }
using std::A; using ::A;

// CHECK: [[CompileUnit:![0-9]+]] = distinct !DICompileUnit({{.+}} imports: [[Imports:![0-9]+]])
// CHECK: [[Imports]] = !{[[ImportedEntity:![0-9]+]]}
// CHECK: [[ImportedEntity]] = !DIImportedEntity(tag: DW_TAG_imported_declaration, scope: [[CompileUnit]], entity: !"_ZTSSt1A", line: 4)
dblaikie edited edge metadata.Mar 4 2016, 1:47 PM
dblaikie added a subscriber: dblaikie.

(please include the fix in the patch for review)

I think the fix could probably happen at a higher level to avoid the linear
search. Perhaps we could get the canonical decl for a using decl - that way
a redecl wouldn't be a 'new' thing? I suppose we'd still need a map of
them, that perhaps we don't have already.

aprantl edited edge metadata.Mar 4 2016, 1:50 PM

That, or we make AllImportedUnits a SmallSet (which is arguably more expensive).

aaboud added a comment.Mar 4 2016, 4:59 PM

That, or we make AllImportedUnits a SmallSet (which is arguably more expensive).

SmallSet is not good enough, as the order will be arbitrary and we will get non-deterministic order of imported entries that have same context.
I thought to use VectorSet, but it does not have the embrace_push function!

aaboud added a comment.Mar 4 2016, 5:02 PM

(please include the fix in the patch for review)

I am not sure what this comment mean!
What do you want me to include in the patch?

The confusion probably comes from the fact that the test is in CFE, but the change is in LLVM. Maybe it would make sense to also add a unittest. We have a couple for DIBuilder.

hintonda added inline comments.
lib/IR/DIBuilder.cpp
172

Since these are maintained in a DenseMap in LLVMContextImpl, which is already unique, is there a way to use that directly instead of maintaining an additional vector?

hintonda added a comment.EditedMar 5 2016, 8:18 AM

It's actually a DenseSet, which makes it even easier. Here's a partial diff -- the rest is just removing references to AllImportedModules.

diff --git a/lib/IR/DIBuilder.cpp b/lib/IR/DIBuilder.cpp
index b7841fe..9288207 100644
--- a/lib/IR/DIBuilder.cpp
+++ b/lib/IR/DIBuilder.cpp
@@ -19,6 +19,7 @@
 #include "llvm/IR/Module.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/Dwarf.h"
+#include "LLVMContextImpl.h"

 using namespace llvm;
 using namespace llvm::dwarf;
@@ -110,10 +111,10 @@ void DIBuilder::finalize() {
   if (!AllGVs.empty())
     CUNode->replaceGlobalVariables(MDTuple::get(VMContext, AllGVs));

-  if (!AllImportedModules.empty())
+  if (!VMContext.pImpl->DIImportedEntitys.empty())
     CUNode->replaceImportedEntities(MDTuple::get(
-        VMContext, SmallVector<Metadata *, 16>(AllImportedModules.begin(),
-                                               AllImportedModules.end())));
+        VMContext, SmallVector<Metadata *, 16>(VMContext.pImpl->DIImportedEntitys.begin(),
+                                               VMContext.pImpl->DIImportedEntitys.end())));
aaboud added a comment.Mar 6 2016, 4:59 AM

It's actually a DenseSet, which makes it even easier. Here's a partial diff -- the rest is just removing references to AllImportedModules.

diff --git a/lib/IR/DIBuilder.cpp b/lib/IR/DIBuilder.cpp
index b7841fe..9288207 100644
--- a/lib/IR/DIBuilder.cpp
+++ b/lib/IR/DIBuilder.cpp
@@ -19,6 +19,7 @@
 #include "llvm/IR/Module.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/Dwarf.h"
+#include "LLVMContextImpl.h"

 using namespace llvm;
 using namespace llvm::dwarf;
@@ -110,10 +111,10 @@ void DIBuilder::finalize() {
   if (!AllGVs.empty())
     CUNode->replaceGlobalVariables(MDTuple::get(VMContext, AllGVs));

-  if (!AllImportedModules.empty())
+  if (!VMContext.pImpl->DIImportedEntitys.empty())
     CUNode->replaceImportedEntities(MDTuple::get(
-        VMContext, SmallVector<Metadata *, 16>(AllImportedModules.begin(),
-                                               AllImportedModules.end())));
+        VMContext, SmallVector<Metadata *, 16>(VMContext.pImpl->DIImportedEntitys.begin(),
+                                               VMContext.pImpl->DIImportedEntitys.end())));

It is still a set that has non-deterministic order, i.e. the order of the metadata will be different from one run to another...impossible for testing and might add more unwanted results...
The only solution is to have a VectorSet, or a vector + a Set.

Could you use a SetVector or SmallSetVector instead of a DenseSet for DIImportedEntities? It's equivalent to what you've proposed, but since it knows when it inserts a new one, maintaining the vector is much cheaper.

aaboud updated this revision to Diff 49934.Mar 7 2016, 1:03 AM
aaboud edited edge metadata.

Another try to verify that we add imported entity only once.
This is the cheapest solution, though it might not be elegant enough.

Also, added a unittest to check same imported entities are added only once to the "imports" field in DICompileUnit.

aaboud updated this revision to Diff 49973.Mar 7 2016, 9:17 AM

Another try, this time I am uploading suggested solution by Don Hinton.
He is suggesting to make "imports" field in DICompileUnit of new type "unique vector".
Then we can get rid of AllImportedEntitys container and use directly the LLVMContext::impl::DIImportedEntitys instead

Pretty sure we shouldn't be adding things to LLVMContext for this, but I
haven't looked closely at the patch/followed the discussion that got us
there in the first place. Will come back to this when I get a chance
(probably later this week) but I'm mentioning it in case others
reading/interacting on the thread can figure it out before then.

Pretty sure we shouldn't be adding things to LLVMContext for this, but I
haven't looked closely at the patch/followed the discussion that got us
there in the first place. Will come back to this when I get a chance
(probably later this week) but I'm mentioning it in case others
reading/interacting on the thread can figure it out before then.

Thanks David,
Please, when you have time to review this, check also patch 49934 (Diff 2).
It is a simpler solution, but not sure also that it is the right one.

aaboud added a comment.Mar 9 2016, 1:19 AM

I just figure out why this approach will not work.
The idea here was to get rid of AllImportedModules list and use the DIImportedEntitys list from the LLVMContextImpl class, assuming that they contain the same info, which is wrong.
AllImportedModules list is per DICompileUnit, while DIImportedEntitys contains all the imported entities from all Compile units.

We cannot get rid of AllImportedModules, thus I will revert this patch to Diff 2.

aaboud updated this revision to Diff 50112.Mar 9 2016, 1:21 AM

Reverting the patch to Diff 2.

Good catch. LGTM...

I'm confused. This patch looks like it's missing parts - there is no DIImportedEntitys (btw: should be DIImportedEntities, if it does exist/is added) in the LLVMContext in trunk at the moment. This patch seems to use it, but doesn't introduce it. Should I be looking elsewhere, at some preparatory patch?

hintonda added a comment.EditedMar 11 2016, 6:55 PM

It's an instance of the X Macro technique. In this case:

lib/IR/LLVMContextImp.h: 937
#define HANDLE_MDNODE_LEAF_UNIQUABLE(CLASS)                                    \
  DenseSet<CLASS *, CLASS##Info> CLASS##s;
#include "llvm/IR/Metadata.def"

include/llvm/IR/Metadata.def: 110
HANDLE_SPECIALIZED_MDNODE_LEAF_UNIQUABLE(DIImportedEntity)
dblaikie accepted this revision.Mar 11 2016, 7:19 PM
dblaikie edited edge metadata.

Seems like a bit more an implementation detail than we'd really want to depend on, but the cost is so low & isolated, & it's so inobtrusive that if/when this does change, it's not going to cost us anymore to fix it then than it would to fix it now so let's go with it.

Please commit when ready.

This revision is now accepted and ready to land.Mar 11 2016, 7:19 PM
This revision was automatically updated to reflect the committed changes.