Page MenuHomePhabricator

ThinLTO: Add test cases for promote+internalize.
ClosedPublic

Authored by pcc on Jun 30 2016, 5:31 PM.

Details

Summary

This tests the effect of both promotion and internalization on a module,
and helps show that D21883 is NFC wrt promotion+internalization.

Diff Detail

Repository
rL LLVM

Event Timeline

pcc updated this revision to Diff 62447.Jun 30 2016, 5:31 PM
pcc retitled this revision from to ThinLTO: Add -thinlto-action=promote-internalize..
pcc updated this object.
pcc added reviewers: tejohnson, mehdi_amini.
pcc added a subscriber: llvm-commits.
tejohnson added inline comments.Jun 30 2016, 7:14 PM
test/ThinLTO/X86/alias_import.ll
91 ↗(On Diff #62447)

Probably because alias to linkonce odr are the only aliases we will import. So they will be exported and therefore not internalizable. See selectCallee() in FunctionImport.cpp

tools/llvm-lto/llvm-lto.cpp
100 ↗(On Diff #62447)

Rather than add a new type here, can the test just do two invocations of llvm-lto - one invoking promotion and then feeding the resulting bitcode into another invoking internalization?

pcc added inline comments.Jul 1 2016, 1:37 PM
test/ThinLTO/X86/alias_import.ll
91 ↗(On Diff #62447)

Okay, I'll update the comment here

tools/llvm-lto/llvm-lto.cpp
100 ↗(On Diff #62447)

That's what I tried first of all, but I got assertion failures like this:

llvm-lto: ../lib/Transforms/IPO/FunctionImport.cpp:536: auto llvm::thinLTOInternalizeModule(llvm::Module &, const GVSummaryMapTy &)::(anonymous class)::operator()(const llvm::GlobalValue &) const: Assertion `GS != DefinedGlobals.end()' failed.

I think this is caused by the internalization code looking up symbol names by GUID, which will be different as a result of the module identifier being different if the input bitcode is piped in. (I thought the whole point of the source_filename thing was to try to protect against that sort of problem, but evidently it doesn't help here -- even if I apply a diff like this:

diff --git a/test/ThinLTO/X86/Inputs/alias_import.ll b/test/ThinLTO/X86/Inputs/alias_import.ll
index 36e5ad1..5f1324b 100644
--- a/test/ThinLTO/X86/Inputs/alias_import.ll
+++ b/test/ThinLTO/X86/Inputs/alias_import.ll
@@ -1,6 +1,4 @@
-
-
-
+source_filename = "2.c"
 
 @globalfuncAlias = alias void (...), bitcast (void ()* @globalfunc to void (...)*)
 @globalfuncWeakAlias = weak alias void (...), bitcast (void ()* @globalfunc to void (...)*)
diff --git a/test/ThinLTO/X86/alias_import.ll b/test/ThinLTO/X86/alias_import.ll
index d4cc996..c7ffc12 100644
--- a/test/ThinLTO/X86/alias_import.ll
+++ b/test/ThinLTO/X86/alias_import.ll
@@ -2,7 +2,7 @@
 ; RUN: opt -module-summary %p/Inputs/alias_import.ll -o %t2.bc
 ; RUN: llvm-lto -thinlto-action=thinlink -o %t.index.bc %t1.bc %t2.bc
 ; RUN: llvm-lto -thinlto-action=promote -thinlto-index %t.index.bc %t2.bc -o - | llvm-dis -o - | FileCheck %s --check-prefix=PROMOTE
-; RUN: llvm-lto -thinlto-action=promote-internalize -thinlto-index %t.index.bc %t2.bc -o - | llvm-dis -o - | FileCheck %s --check-prefix=PROMOTE-INTERNALIZE
+; RUN: llvm-lto -thinlto-action=promote -thinlto-index %t.index.bc %t2.bc -o - | llvm-lto -thinlto-action=internalize -thinlto-index %t.index.bc -exported-symbol=dummy - -o - | llvm-dis -o - | FileCheck %s --check-prefix=PROMOTE-INTERNALIZE
 ; RUN: llvm-lto -thinlto-action=import -thinlto-index %t.index.bc %t1.bc -o - | llvm-dis -o - | FileCheck %s --check-prefix=IMPORT
 
 ;
@@ -126,6 +126,8 @@
 ; PROMOTE-INTERNALIZE-DAG: define internal void @linkoncefunc()
 ; PROMOTE-INTERNALIZE-DAG: define internal void @weakfunc()
 
+source_filename = "1.c"
+
 define i32 @main() #0 {
 entry:
   call void @globalfuncAlias()

the combined summary still looks like this here:

<ENTRY [...] /> record string = '[...]/Output/alias_import.ll.tmp1.bc'
<ENTRY [...] /> record string = '[...]/Output/alias_import.ll.tmp2.bc

)

tejohnson added inline comments.Jul 1 2016, 5:11 PM
tools/llvm-lto/llvm-lto.cpp
100 ↗(On Diff #62447)

The name in the combined summary is the path to the bitcode (used to locate a function during importing).

However, setting the source_filename in the LLVM assembly should cause the right GUID naming to occur, I'm not sure offhand why it isn't enabling consistent GUID computation. The code in thinLTOInternalizeModule does attempt to get the original name for something that has already been promoted. Can you look at the intermediate bitcode files and see if they contain a SOURCE_FILENAME record?

pcc added inline comments.Jul 6 2016, 1:06 PM
tools/llvm-lto/llvm-lto.cpp
100 ↗(On Diff #62447)

The name in the combined summary is the path to the bitcode (used to locate a function during importing).

Ah, yes, of course.

However, setting the source_filename in the LLVM assembly should cause the right GUID naming to occur, I'm not sure offhand why it isn't enabling consistent GUID computation.

Okay, here's what's going on here.

  • The client is using ModuleSummaryIndex::collectDefinedGVSummariesPerModule to group global value summaries by summary module path [1].
  • We then use the module identifier as a key to look up the global value summary map entry [2].
  • That lookup yields an empty map because the module identifier does not match any summary module path. As a result, none of the lookups in thinLTOInternalizeModule succeed.

I see two possible solutions here:

  • do what I'm doing in this patch
  • provide some way to override the module identifier in the loaded module in order to allow the lookup to succeed

I'm partial to the former because it's more in line with what I see as the end goal here (combining the internalize and promote phases). I don't think we should extend the module summary format (for example by allowing a source filename to be used as a key) for the same reason. Let me know what you think.

[1] http://llvm-cs.pcc.me.uk/lib/IR/ModuleSummaryIndex.cpp#88
[2] http://llvm-cs.pcc.me.uk/lib/LTO/ThinLTOCodeGenerator.cpp#663

mehdi_amini added inline comments.Jul 6 2016, 1:48 PM
tools/llvm-lto/llvm-lto.cpp
100 ↗(On Diff #62447)

Could the option "-thinlto-module-id" to llvm-lto help here?

pcc added inline comments.Jul 6 2016, 1:59 PM
tools/llvm-lto/llvm-lto.cpp
100 ↗(On Diff #62447)

Yes, that looks like exactly what I'm looking for. Let me see if that works.

pcc updated this revision to Diff 62987.Jul 6 2016, 3:36 PM
  • Instead of adding promote-internalize, pipe promote into internalize and use -thinlto-module-id
  • Add promote+internalize test for a linkonce function
pcc updated this revision to Diff 62988.Jul 6 2016, 3:39 PM
pcc marked 2 inline comments as done.
  • Update comment
mehdi_amini edited edge metadata.Jul 6 2016, 3:41 PM

(Title is out-of-sync, just make sure it does not end-up in the final commit)

pcc retitled this revision from ThinLTO: Add -thinlto-action=promote-internalize. to ThinLTO: Add test cases for promote+internalize..Jul 6 2016, 3:44 PM
pcc updated this object.
pcc edited edge metadata.
tejohnson accepted this revision.Jul 6 2016, 3:45 PM
tejohnson edited edge metadata.

LGTM

This revision is now accepted and ready to land.Jul 6 2016, 3:45 PM
This revision was automatically updated to reflect the committed changes.