This is an archive of the discontinued LLVM Phabricator instance.

[analyzer][NFC] Change LangOptions to CheckerManager in the shouldRegister* functions.
ClosedPublic

Authored by Szelethus on Feb 27 2020, 9:25 AM.

Details

Summary

Some checkers may not only depend on language options but also analyzer options. To make this possible this patch changes the parameter of the shouldRegister* function to CheckerManager to be able to query the analyzer options when deciding whether the checker should be registered.

Diff Detail

Event Timeline

I am so sorry to mention, but we need the AnalysisManager to pass around which manages the analysis, therefore it knows both the LangOptions and AnalyzerOptions. Also this entire callback should be removed ideally: it has to be a virtual function defaulting to return true; and if someone needs this feature could rewrite the behavior. I guess there was some debate whether it should be on by default or not, but for a checker writer and future changes this patch shows that how weak this API is.

PS: The CheckerManager also could serve this behavior as registerXXX() already passing around that manager, but I believe the AnalysisManager supposed to manage the analysis.

It is impossible to use CheckerManager as parameter for shouldRegisterXXX() because CheckerRegistry does not have it. If I add CheckerManager to CheckerRegistry then the printXXX() functions will not work because they do not have CheckerManager at all. This patch does not help in printing error message, see D75171. I need a way to solve 44998 by rejecting the checker if the option is disabled and printing an error message.

It is impossible to use CheckerManager as parameter for shouldRegisterXXX() because CheckerRegistry does not have it. If I add CheckerManager to CheckerRegistry then the printXXX() functions will not work because they do not have CheckerManager at all. This patch does not help in printing error message, see D75171. I need a way to solve 44998 by rejecting the checker if the option is disabled and printing an error message.

Aha, I see as of D75171. Well, @Szelethus felt the same to pass around the CheckerManager. Let us see:

std::unique_ptr<CheckerManager> ento::createCheckerManager(
    ASTContext &context,                                     
    AnalyzerOptions &opts,
    ArrayRef<std::string> plugins,
    ArrayRef<std::function<void(CheckerRegistry &)>> checkerRegistrationFns,
    DiagnosticsEngine &diags) {                                               
  auto checkerMgr = std::make_unique<CheckerManager>(context, opts);
                                                                      
  CheckerRegistry allCheckers(plugins, diags, opts, context.getLangOpts(),
                              checkerRegistrationFns);                      
                                                        
  allCheckers.initializeManager(*checkerMgr);
  allCheckers.validateCheckerOptions();        
  checkerMgr->finishedCheckerRegistration();
                                              
  return checkerMgr;
}
  • from StaticAnalyzer/Frontend/CheckerRegistration.cpp.

Are you sure we cannot pass around the manager like allCheckers.initializeManager(*checkerMgr);? I am also thinking of a glue-layer between the two managers, like CheckerRegistryManager or something, which holds all your needs, like AnalyzerOptions and LangOptions for now.

I think if we ask @Szelethus kindly to design his callback using CheckerManager instead, we could simplify this patch a lot. As he is the code owner, most likely he continues his journey to make sure people enable the checker dependencies, most likely using shouldRegisterXXX().

Szelethus added a reviewer: Charusso.EditedFeb 27 2020, 11:10 AM

I am so sorry to mention, but we need the AnalysisManager to pass around which manages the analysis, therefore it knows both the LangOptions and AnalyzerOptions.

PS: The CheckerManager also could serve this behavior as registerXXX() already passing around that manager, but I believe the AnalysisManager supposed to manage the analysis.

Well, I like to say that "Any manager can be retrieved in clang at arbitrary places if you try hard enough", so I think either that actually satisfies this would be fine :)

Also this entire callback should be removed ideally: it has to be a virtual function defaulting to return true; and if someone needs this feature could rewrite the behavior. I guess there was some debate whether it should be on by default or not, but for a checker writer and future changes this patch shows that how weak this API is.

Yup, that is a very good criticism of the the API. I would prefer to see something a lot less ugly, and I strongly share you sentiment that making changes to it is about as ugly as the initial patch itself. Virtual functions would be okay, the reason why I didn't do that is because where do you put them? Checker objects can't house them, because the point of the entire shouldRegister function is to never create them in the first place. But thinking about it again, the problem isn't really their creation as much as their registration. If we were to create a checker object and ask it whether its fine to do analysis, and then store it, we would be able to get rid of the shouldRegister function. I can see from a mile away that due to the library layout, cyclic linking dependencies, and whatever else would make this seemingly trivial task a lot more invasive and difficult, and I'm just not too sure whether its worth the effort to change an arguably bearable inconvenience.

It is impossible to use CheckerManager as parameter for shouldRegisterXXX() because CheckerRegistry does not have it. If I add CheckerManager to CheckerRegistry then the printXXX() functions will not work because they do not have CheckerManager at all. This patch does not help in printing error message, see D75171. I need a way to solve 44998 by rejecting the checker if the option is disabled and printing an error message.

Aha, I see as of D75171. Well, @Szelethus felt the same to pass around the CheckerManager. Let us see:

std::unique_ptr<CheckerManager> ento::createCheckerManager(
    ASTContext &context,                                     
    AnalyzerOptions &opts,
    ArrayRef<std::string> plugins,
    ArrayRef<std::function<void(CheckerRegistry &)>> checkerRegistrationFns,
    DiagnosticsEngine &diags) {                                               
  auto checkerMgr = std::make_unique<CheckerManager>(context, opts);
                                                                      
  CheckerRegistry allCheckers(plugins, diags, opts, context.getLangOpts(),
                              checkerRegistrationFns);                      
                                                        
  allCheckers.initializeManager(*checkerMgr);
  allCheckers.validateCheckerOptions();        
  checkerMgr->finishedCheckerRegistration();
                                              
  return checkerMgr;
}
  • from StaticAnalyzer/Frontend/CheckerRegistration.cpp.

Are you sure we cannot pass around the manager like allCheckers.initializeManager(*checkerMgr);? I am also thinking of a glue-layer between the two managers, like CheckerRegistryManager or something, which holds all your needs, like AnalyzerOptions and LangOptions for now.

CheckerManager has complete control of CheckerRegistry. You can for sure cascade it down as a parameter. The worry is, again, how CheckerManager is in libStaticAnalysisCore and CheckerRegistry is in libStaticAnalysisFrontend, but given that we wouldn't need to use it one but more then how we use it for registry functions, this should be fine.

Also this entire callback should be removed ideally: it has to be a virtual function defaulting to return true; and if someone needs this feature could rewrite the behavior. I guess there was some debate whether it should be on by default or not, but for a checker writer and future changes this patch shows that how weak this API is.

Yup, that is a very good criticism of the the API. I would prefer to see something a lot less ugly, and I strongly share you sentiment that making changes to it is about as ugly as the initial patch itself. Virtual functions would be okay, the reason why I didn't do that is because where do you put them? Checker objects can't house them, because the point of the entire shouldRegister function is to never create them in the first place. But thinking about it again, the problem isn't really their creation as much as their registration. If we were to create a checker object and ask it whether its fine to do analysis, and then store it, we would be able to get rid of the shouldRegister function. I can see from a mile away that due to the library layout, cyclic linking dependencies, and whatever else would make this seemingly trivial task a lot more invasive and difficult, and I'm just not too sure whether its worth the effort to change an arguably bearable inconvenience.

Thinking back, I did have a number of failed attempts to make something a bit less ugly, but the sharp divide between the 2 libraries makes is really-really difficult, and I don't recall alternative solutions being that much better. Either the checker interface gets worse, or the checker registration interface gets so messy that it would severely hurt further improvements in terms of checker dependency development.

Charusso accepted this revision.Feb 27 2020, 12:00 PM

! In D75271#1896223, @Szelethus wrote:
Thinking back, I did have a number of failed attempts to make something a bit less ugly, but the sharp divide between the 2 libraries makes is really-really difficult, and I don't recall alternative solutions being that much better. Either the checker interface gets worse, or the checker registration interface gets so messy that it would severely hurt further improvements in terms of checker dependency development.

Well, if you cannot solve this problem, most likely we neither, but keep in mind:

If you can't solve a problem, then there is an easier problem you can't solve: find it.

  • George Polya

As far as you guys keep updating your API I will buy it, thanks!

Szelethus requested changes to this revision.Feb 27 2020, 1:44 PM

Let's change this to either CheckerManger or AnalysisManager.

This revision now requires changes to proceed.Feb 27 2020, 1:44 PM

Let's change this to either CheckerManger or AnalysisManager.

Did you read my comments? I tried CheckerManager first, but it does not work, because CheckerRegistry does not have it. If I pass it to CheckerRegistry, then the printXXX() functions will not compile, because they do not have it.

Well, I like to say that "Any manager can be retrieved in clang at arbitrary places if you try hard enough", so I think either that actually satisfies this would be fine :)

This is not always true, especially if the manager does not exist at that particular place. There is not CheckerManager in the printXXX() functions.

Szelethus commandeered this revision.Feb 28 2020, 9:14 AM
Szelethus edited reviewers, added: baloghadamsoftware; removed: Szelethus.

Yoink.

Szelethus updated this revision to Diff 247292.Feb 28 2020, 9:29 AM
Szelethus retitled this revision from [Analyzer][NFC] Add AnalyzerOptions parameter to shouldRegisterXXX() functions to [analyzer][NFC] Change LangOptions to CheckerManager in the shouldRegister* functions..
Szelethus edited the summary of this revision. (Show Details)

Instead of adding AnalyzerOptions as a parameter, pass CheckerManager around. Rebase on top of D75360.

Mind that AnalysisManager, similarly to CheckerManager before D75360, must be constructed with an ASTContext which is not available at the moment when the shouldRegister* functions run, and they must run no later than when the help flags would be honored.

I think this is straightforward after D75360. This was my original attempt as well, but without D75360 it did not work. This solves the problem of both accessing AnalyzerOptions and emitting an error message.

This revision is now accepted and ready to land.Mar 2 2020, 5:49 AM
baloghadamsoftware requested changes to this revision.Mar 5 2020, 10:05 AM

This patch still does not help. Most of the functions of CheckerManager are non-const thus they cannot be called on a const object. E.g. getAnalyzerOptions() is non-const (why?). For error reporting getASTContext() should be const as well (to access the diagnostics engine). Thus either CheckerManager should be changed by setting its getters to const or the shouldRegisterXXX() should take a non-const reference to CheckerManager. I would prefer the former one.

This revision now requires changes to proceed.Mar 5 2020, 10:05 AM

It is even worse: the shouldRegister() functions do not prevent the registration of a checker, at least in cases when they are not enabled directly but via dependencies. See D75171.

Thanks for bringing it up, I'll attend to it -- though both of those changes are out of scope of this change, and I'd be strongly against solving any of those issues within this revision.

OK, I agree, these issues should be solved in other patches. I already solved one, but the other one is strange. I will open a BugZilla ticket for it.

This revision is now accepted and ready to land.Mar 9 2020, 3:48 AM
NoQ accepted this revision.Mar 15 2020, 8:57 PM

Great, thanks!

This revision was automatically updated to reflect the committed changes.