This is an archive of the discontinued LLVM Phabricator instance.

llvm-lto2: Move the LTO::run() action behind a subcommand.
ClosedPublic

Authored by pcc on Mar 27 2017, 4:51 PM.

Details

Summary

Move LTO::run() to a "run" subcommand so that we can introduce new subcommands
for testing different parts of the LTO implementation.

This doesn't use llvm::cl subcommands because it doesn't appear to be currently
possible to pass an argument not associated with a subcommand to a subcommand
(e.g. -lto-use-new-pm, -mcpu=yonah).

Event Timeline

pcc created this revision.Mar 27 2017, 4:51 PM
tejohnson added a subscriber: tejohnson.

+zturner to ask about subcommands support.
I guess the issue is that these other options do not have a cl::sub and you (only) want to handle them for the "run" subcommand?

zturner edited edge metadata.Apr 11 2017, 9:41 AM

So if I understand correctly, these are existing options that are defined in some library somewhere, and you want to be able to use them in a subcommand of your own creation? Yea unfortunately I never considered that use case.

I think one thing you could do to make this work with the least amount of effort possible is to make a new option type instead of cl::opt and cl::list called cl::redefine. It takes an opt or list to its constructor, and a list of modifiers. This way you could do something like:

extern cl::opt<bool> LtoUseNewPmOpt;

cl::SubCommand RunSubcommand("run");

cl::redefine<cl::opt<bool>> LtoUseNewPm2(LtoUseNewPmOpt, cl::sub(RunSubcommand));

There's probably some details to work out, but it seems like it should work in theory. Anyway, hacking it up like this is probably ok as a stopgap.

Actually perhaps better would be to make cl::redefine just another modifier. Maybe call it cl::template_opt or something. This way you would write:

extern cl::opt<bool> UseNewLtoPm;

cl::opt<bool> UseNewLtoPm2(cl::template_opt(UseNewLtoPm), cl::sub(RunSubcommand));
pcc added a comment.Apr 11 2017, 10:34 AM

Actually perhaps better would be to make cl::redefine just another modifier. Maybe call it cl::template_opt or something. This way you would write:

extern cl::opt<bool> UseNewLtoPm;

cl::opt<bool> UseNewLtoPm2(cl::template_opt(UseNewLtoPm), cl::sub(RunSubcommand));

The problem with that (and your other suggestion) is that it would require specifying every possible flag to be passed to this tool. There are already quite a few of them that are required for tests (basically everything in llvm/CodeGen/CommandFlags.h and a few more). Also, as this is a developer tool I may also want to pass arbitrary top-level flags to the tool (e.g. for statistics, extra dumping, etc.) which may not already be covered by tests.

So I think I'd prefer that the API look something like like this (where cl::import_root imports all flags from the "root" subcommand into a given subcommand):

static cl::SubCommand RunSubcommand("run", "Run LTO", cl::import_root());

and use this hack until that is implemented.

tejohnson accepted this revision.Apr 11 2017, 10:36 AM

Going to LGTM this since its confirmed this can't be done with existing subcommand support.

This revision is now accepted and ready to land.Apr 11 2017, 10:36 AM
This revision was automatically updated to reflect the committed changes.
zturner added a comment.EditedApr 11 2017, 11:33 AM

So currently if you were to run llvm-lto2 -help would you already see all the options you want to import? It sounds like yes. If so, what's the difference between llvm-lto2 -lto-use-new-pm and llvm-lto2 run -lto-use-new-pm? Is there some reason you need to import them all rather than just use the existing top-level options?

pcc added a comment.Apr 11 2017, 11:43 AM

So currently if you were to run llvm-lto2 -help would you already see all the options you want to import? It sounds like yes.

Correct.

If so, what's the difference between llvm-lto2 -lto-use-new-pm and llvm-lto2 run -lto-use-new-pm? Is there some reason you need to import them all rather than just use the existing top-level options?

I cannot pass both a top-level option and a subcommand at the same time.

$ ra/bin/llvm-pdbdump -reverse-iterate raw
llvm-pdbdump: Unknown command line argument 'raw'.  Try: 'ra/bin/llvm-pdbdump -help'

I suppose that another possibility is that we can get that to work? It wouldn't be 100% perfect because the top-level flags may not apply to other subcommands, but it would probably be good enough for my use case.

I guess what I mean is, if you just want to import every single top level option into the "run" subcommand, and the tool only works when the run subcommand is given (looking at the source code it just prints usage and exits if there is no run subcommand) why is the run subcommand even necessary? Isn't it equivalent to just not even having a subcommand and running llvm-lto2 -lto-use-new-pm ...?

pcc added a comment.Apr 11 2017, 11:53 AM

I guess what I mean is, if you just want to import every single top level option into the "run" subcommand, and the tool only works when the run subcommand is given (looking at the source code it just prints usage and exits if there is no run subcommand) why is the run subcommand even necessary? Isn't it equivalent to just not even having a subcommand and running llvm-lto2 -lto-use-new-pm ...?

I intend to add another subcommand to this command (see D31920).