This is an archive of the discontinued LLVM Phabricator instance.

[Polly] [Refactor] Move isl_ctx into Scop.
ClosedPublic

Authored by etherzhhb on Feb 13 2016, 1:06 AM.

Details

Summary

This patch makes the ScopInfo carry less context, and makes it easier for the new PassManager change.

Diff Detail

Repository
rL LLVM

Event Timeline

etherzhhb updated this revision to Diff 47904.Feb 13 2016, 1:06 AM
etherzhhb retitled this revision from to [Refactor] Use ManagedStatic to manage the global isl_ctx..
etherzhhb updated this object.
etherzhhb added a reviewer: grosser.
etherzhhb set the repository for this revision to rL LLVM.
etherzhhb added a subscriber: Restricted Project.

I think this is not what we want. Globals are always bad when we think about parallel execution of passes. While we still have a global map flying around that nobody cares to remove we... we should not introduce new globals that will break the assumption that non-connected parts of the module can be compiled concurrently. As an alternative I would suggest to move the isl_ctx into the Scop class and let it deal correctly with deallocating all derived objects once it (the Scop) is deleted.

I think this is not what we want. Globals are always bad when we think about parallel execution of passes. While we still have a global map flying around that nobody cares to remove we... we should not introduce new globals that will break the assumption that non-connected parts of the module can be compiled concurrently. As an alternative I would suggest to move the isl_ctx into the Scop class and let it deal correctly with deallocating all derived objects once it (the Scop) is deleted.

I agree that we should move the isl_ctx to Scop.

etherzhhb updated this revision to Diff 47909.Feb 13 2016, 7:40 AM
etherzhhb retitled this revision from [Refactor] Use ManagedStatic to manage the global isl_ctx. to [Refactor] Move isl_ctx into Scop..

I have no opinion whether we should share isl_ctx across SCoPs or should make it becomes SCoP local, because I have no idea what context does it carry and how it is used.

I think make it becomes SCoP local is a good idea since the SCoPs are independent.

However, this patch does *NOT* work, and I get the following stack dump:

#6 0x00007ffff6a01528 abort /build/glibc-3Vu5mt/glibc-2.19/stdlib/abort.c:91:0
#7 0x00007ffff6581856 isl_handle_error /home/ether/work/llvm/tools/polly/lib/External/isl/isl_ctx.c:93:0
#8 0x00007ffff6581d35 isl_ctx_free /home/ether/work/llvm/tools/polly/lib/External/isl/isl_ctx.c:243:0
#9 0x00007ffff648a95c polly::Scop::~Scop() /home/ether/work/llvm/tools/polly/lib/Analysis/ScopInfo.cpp:2777:0
#10 0x00007ffff64a650a std::default_delete<polly::Scop>::operator()(polly::Scop*) const /usr/include/c++/4.9/bits/unique_ptr.h:76:0
#11 0x00007ffff649e6db std::unique_ptr<polly::Scop, std::default_delete<polly::Scop> >::reset(polly::Scop*) /usr/include/c++/4.9/bits/unique_ptr.h:345:0
#12 0x00007ffff6490b85 polly::ScopInfo::clear() /home/ether/work/llvm/tools/polly/lib/Analysis/ScopInfo.cpp:4157:0
#13 0x00007ffff64951de polly::ScopInfo::releaseMemory() /home/ether/work/llvm/tools/polly/include/polly/ScopInfo.h:2223:0
#14 0x000000000169a1dc llvm::PMDataManager::freePass(llvm::Pass*, llvm::StringRef, llvm::PassDebuggingString) /home/ether/work/llvm/lib/IR/LegacyPassManager.cpp:976:0
#15 0x000000000169a0ee llvm::PMDataManager::removeDeadPasses(llvm::Pass*, llvm::StringRef, llvm::PassDebuggingString) /home/ether/work/llvm/lib/IR/LegacyPassManager.cpp:964:0
#16 0x000000000119d602 llvm::RGPassManager::runOnFunction(llvm::Function&) /home/ether/work/llvm/lib/Analysis/RegionPass.cpp:130:0
#17 0x000000000169c75e llvm::FPPassManager::runOnFunction(llvm::Function&) /home/ether/work/llvm/lib/IR/LegacyPassManager.cpp:1550:0
#18 0x000000000169c8f1 llvm::FPPassManager::runOnModule(llvm::Module&) /home/ether/work/llvm/lib/IR/LegacyPassManager.cpp:1571:0
#19 0x000000000169cc7d (anonymous namespace)::MPPassManager::runOnModule(llvm::Module&) /home/ether/work/llvm/lib/IR/LegacyPassManager.cpp:1627:0
#20 0x000000000169d373 llvm::legacy::PassManagerImpl::run(llvm::Module&) /home/ether/work/llvm/lib/IR/LegacyPassManager.cpp:1730:0
#21 0x000000000169d5b3 llvm::legacy::PassManager::run(llvm::Module&) /home/ether/work/llvm/lib/IR/LegacyPassManager.cpp:1762:0
#22 0x0000000000de12b5 main /home/ether/work/llvm/tools/opt/opt.cpp:631:0

I free the IslCtx by "isl_ctx_free(IslCtx)" at the end of the destructor and get the above error. I don't know why this happens, any hit?

etherzhhb updated this revision to Diff 47911.Feb 13 2016, 7:43 AM

Update diff

jdoerfert added inline comments.Feb 13 2016, 8:00 AM
lib/Analysis/ScopInfo.cpp
2800 ↗(On Diff #47911)

You have to free all isl_objects that were created with this Ctx first. Missing are at least:

  • The ScopArrayInfoObjects
  • The Statements
    • The MemoryAccesses

Maybe even more.

Stmts.clear();            // Takes care of all Statement objects
AccFuncMap.clear();       //  ... " " ........ MemoryAccesses objects
ScopArrayInfoMap.clear(); //  ... " " ........ ScopArrayInfo objects

Did that catch all?

I know what happened, let me fix that

etherzhhb marked an inline comment as done.Feb 13 2016, 9:15 AM
etherzhhb added inline comments.
lib/Analysis/ScopInfo.cpp
2800 ↗(On Diff #47911)

We also need to make sure we delete the context after we delete Dependences and IslAst, which also create isl objects with the context.

I am going to use a shared_ptr to manage that.

etherzhhb updated this revision to Diff 47915.Feb 13 2016, 9:22 AM
etherzhhb marked an inline comment as done.

In order to make sure that we delete the isl context after all isl objects are delete, we introduce shared_ptrs to manage the life time of the context. We are going to distribute the shared_ptr to Dependences/IslNode which also create isl objects form the context. By doing this, we make sure the context is delete when we delete the last objects that create isl object from the context.

etherzhhb marked an inline comment as done.Feb 13 2016, 9:36 AM
jdoerfert edited edge metadata.Feb 14 2016, 2:48 PM

LGTM. But I am not sure about the shared pointer, so I would appreciate somebody else looking at the patch.

grosser edited edge metadata.Feb 15 2016, 2:27 AM

Hi ether,

I think moving the isl_ctx into the scop is good, but similarly to Johannes I do not yet understand the use of the shared-ptr. All isl objects take plain pointers, so the shared pointer itself goes very quickly out-of-scope. My feeling is that this very same change should work already without shared pointers. If it does not, then we likely have a memory issue that is just be covered by shared pointers. ether: when not using shared pointers, which element is freed only after we try to free the isl_ctx?

Currently, we build isl object in ScopInfo, DependencyInfo and IslAst.

We must free all isl objects before we free the ctx. And I try to use the reference counter from shared_ptr to figure out the time to free the isl ctx.

Suppose we free ScopInfo, DependencyInfo, IslAst in order:

If we only run ScopInfo, we should free ctx when we free the corresponding Scop.

If we only run ScopInfo and DependencyInfo, we should free ctx when we free the corresponding Dependences.

If we only run ScopInfo, DependencyInfo and IslAst, we should free ctx when we free the corresponding IslAst.

And if ScopInfo, DependencyInfo, IslAst are not free in order, it become a total mess ....

Using the reference counter from the shared_ptr can solve this problem: Everytime we destruct a shared_ptr, the shared_ptr will decrease the reference counter. If the reference counter is decreased to 0, the current analysis result we are destructing are the last one that hold isl objects -- and there are no isl object left, which mean it is safe to free the isl context.

A better solution may be provide a "isl_free_ctx_if_ref_is_zero" function which do not actually free the context until all isl objs are released?

Or we can try to make an SCoP to remember all the created ISL objects, and only free isl objs in the SCoP info?

And if ScopInfo, DependencyInfo, IslAst are not free in order, it become a total mess ....

The pass manager should free IslAst, DependencyInfo, ... before ScopInfo
because they have a dependence on the ScopInfo results. Am I wrong?

And if ScopInfo, DependencyInfo, IslAst are not free in order, it become a total mess ....

The pass manager should free IslAst, DependencyInfo, ... before ScopInfo
because they have a dependence on the ScopInfo results. Am I wrong?

Sorry I am not precise enough. It should be Scop (the result of ScopInfo), Dependences (the result of DependenceInfo) and IslAst (the result of IslAstInfo).

Let me do some experiments to confirm the order with the debug-pass option.

etherzhhb added a comment.EditedFeb 15 2016, 8:01 AM

Actually we free ScopInfo before other scop analyses:
/home/ether/work/llvm_build/bin/opt -load /home/ether/work/llvm_build/lib/LLVMPolly.so -polly-ast -polly-dependences -polly-ast-detect-parallel -analyze /home/ether/work/llvm/tools/polly/test/Isl/Ast/dependence_distance_constant.ll --debug-pass=Executions

[2016-02-15 16:19:24.577937000] 0x380e7c0 Freeing Pass 'Polly - Create polyhedral description of Scops' on Region 'for.cond => for.end8'...
[2016-02-15 16:19:24.578081000] 0x380e7c0 Freeing Pass 'Polly - Generate an AST from the SCoP (isl)' on Region 'for.cond => for.end8'...
[2016-02-15 16:19:24.578174000] 0x380e7c0 Freeing Pass 'Polly - Calculate dependences' on Region 'for.cond => for.end8'...

etherzhhb updated this revision to Diff 48024.Feb 15 2016, 3:34 PM
etherzhhb retitled this revision from [Refactor] Move isl_ctx into Scop. to [Polly] [Refactor] Move isl_ctx into Scop..
etherzhhb edited edge metadata.
etherzhhb removed rL LLVM as the repository for this revision.
etherzhhb added inline comments.
lib/External/isl/include/isl/ctx.h
164 ↗(On Diff #48024)

feel free to suggest a better name

We don't modify the source in External/isl. For any change there, we have to ask Sven Verdoolaege to include it into the mainline isl. In the context of isl only this makes less sense. One would want to have a proper memory management instead of disabling the memory leak detection feature.

Because of missing alternatives, I'd vote for the shared_ptr solution.

include/polly/ScopInfo.h
1282 ↗(On Diff #48024)

The comment still mentions shared_ptr.

I am a little worried about both of these changes. Will we still be warned if we forgot to release a pointer or are will we just stop to free isl_ctx due to the wrong order of freeing items.

Also, I am surprised why this works today without any troubles? For whatever reason today it seems to be ensured that we free everything else before the ScopInfo pass has been freed. Eithers observation of the different pass order somehow suggests that this should not be the case, but it still magically works. Do you understand why this currently works despite the pass freeing being reversed?

etherzhhb added a comment.EditedFeb 16 2016, 7:35 AM

I am a little worried about both of these changes. Will we still be warned if we forgot to release a pointer or are will we just stop to free isl_ctx due to the wrong order of freeing items.

Without shared_ptr, there will be no warning.
With the shared_ptr, we the program abort if we do not free all isl objects.

Also, I am surprised why this works today without any troubles? For whatever reason today it seems to be ensured that we free everything else before the ScopInfo pass has been freed.

Originally, we only free the isl ctx when we destruct/delete the ScopInfo pass itself. With this patch, we free the isl ctx when we destruct/delete the Scop (the analysis result of ScopInfo), in the releaseMemory function of ScopInfo ().

A single ScopInfo pass will produce multiple Scop, we call releaseMemory of the same ScopInfo multiple times, and only delete ScopInfo pass itself once. That is the different. I think the message "Freeing Pass 'Polly - Generate an AST from the SCoP (isl)' on Region 'for.cond => for.end8'" is generated when we call releaseMemory, instead of deleting the pass.

Eithers observation of the different pass order somehow suggests that this should not be the case, but it still magically works. Do you understand why this currently works despite the pass freeing being reversed?

Explained above.

Thanks ether for explaining these things in detail. It seems indeed to be the case that we free passes in a rather unpredictable order. If this is the case, the shared pointer approach seems OK and I now convinced myself that it works and does not prevent us from loosing the ability to understand if we leak memory.

However, if we could understand why this freeing behavior is so unpredictable that would be nice.

Best,
Tobias

They are deleted by "removeDeadPasses".

And the dead passes are collected by "collectLastUses".

Where the last users -- the passes that are going to be freed, are stored in a SmallPtrSet. (See line 569 of LegacyPassManager.cpp).

A SmallPtrSet is not an ordered container, as a result, there is no guarantee on the order we are going to free the dead passes.

another approach is to make all isl objects under Scop's control, and let Scop free them all.

I see. I still would expect that there is an order in which passes are
freed, but if this is not the case let's go with the shared pointer. I
do not see how we can allocate all objects under scop's control. That
seems to be even more complicated.

Best,
Tobias

etherzhhb updated this revision to Diff 48186.Feb 17 2016, 7:17 AM
etherzhhb set the repository for this revision to rL LLVM.

Use shared_ptr again.

After we move isl_ctx into Scop, we need to make sure the isl_ctx
 is free after all isl objects are freed. This is no straightforward
 because we create isl objects in multiple analyses, and the PassManager
 make no guarantee on the order of freeing those passes.

 To solve this problem, we distribute the same isl_ctx to all analyses
 that create isl objects with shared_ptrs that share the same reference
 conter. The idea is to use the reference counter to capture the last
 pass that are being freed, and free the isl_ctx at that time.
grosser accepted this revision.Feb 17 2016, 7:19 AM
grosser edited edge metadata.

LGTM.

Besides one unrelated whitespace change and a couple of typos in the commit message.

Best,
Tobias

lib/Analysis/ScopInfo.cpp
115 ↗(On Diff #48186)

Unrelated withspace change.

This revision is now accepted and ready to land.Feb 17 2016, 7:19 AM
This revision was automatically updated to reflect the committed changes.