[CommandLine] Allow Options to specify multiple OptionCategory's.

[hintonda]

Summary

It's not uncommon for separate components to share common
Options, e.g., it's common for related Passes to share Options in
addition to the Pass specific ones.

With this change, components can use OptionCategory's to simply help
output even if some of the options are shared.

[MaskRay]

May I ask you to share some dumps of some-command -help (a demo how you'll make use of the multiple categories)😁

llvm/lib/Support/CommandLine.cpp

429

Can you explain why Categories cannot be empty? Is it because an option has one category when it is registered? (Sorry as I'm not very familiar with the code)

llvm/unittests/Support/CommandLineTest.cpp

99

Since the file includes STLExtras.h you may use the range-style llvm::find_if to simplify begin/end.

[hintonda]

In D61574#1491471, @MaskRay wrote:

May I ask you to share some dumps of some-command -help (a demo how you'll make use of the multiple categories)😁

Sure, I work up an example. Btw, my ultimate plan is to add categories to the passes so that bugpoint help isn't just a huge mess. That's the motivation for this change. When I started adding them, I found it wasn't really doable without this change since so many passes shared options.

llvm/lib/Support/CommandLine.cpp

429

The ctor for Option adds GeneralCategory. The old way just replaced it if the user provided their own. With this change, it's the first one on the vector. That's why I check index 0.

llvm/unittests/Support/CommandLineTest.cpp

99

Ah, okay. Will do, thanks...

[MaskRay]

llvm/lib/Support/CommandLine.cpp

2421

Use llvm::find (or llvm::count) and delete the otherwise unused CategoriesBegin CategoriesEnd.

[hintonda]

llvm/lib/Support/CommandLine.cpp

2421

I'll update them all. Thanks for point that out.

[hintonda]

In D61574#1491484, @hintonda wrote:

In D61574#1491471, @MaskRay wrote:

May I ask you to share some dumps of some-command -help (a demo how you'll make use of the multiple categories)😁

Sure, I work up an example. Btw, my ultimate plan is to add categories to the passes so that bugpoint help isn't just a huge mess. That's the motivation for this change. When I started adding them, I found it wasn't really doable without this change since so many passes shared options.

There's currently not clean way to do this without being able to capture stdout when calling ParseCommandLineOptions, so I'll add that in a separate patch for that and use it here. Please see D30893 for something similar that was done to capture stderr.

[hintonda]

In D61574#1492410, @hintonda wrote:

In D61574#1491484, @hintonda wrote:

In D61574#1491471, @MaskRay wrote:

May I ask you to share some dumps of some-command -help (a demo how you'll make use of the multiple categories)😁

Sure, I work up an example. Btw, my ultimate plan is to add categories to the passes so that bugpoint help isn't just a huge mess. That's the motivation for this change. When I started adding them, I found it wasn't really doable without this change since so many passes shared options.

There's currently not clean way to do this without being able to capture stdout when calling ParseCommandLineOptions, so I'll add that in a separate patch for that and use it here. Please see D30893 for something similar that was done to capture stderr.

After looking into it, I'm not going to try to capture stdout. So, I'm not going to add a test that compares help output. This is consistent with the status quo -- there isn't a test for that in llvm. I suppose I could add one to clang since I'd have access to %clang in lit, but adding an llvm test to clang seems a bit odd.

[hintonda]

In D61574#1491471, @MaskRay wrote:

May I ask you to share some dumps of some-command -help (a demo how you'll make use of the multiple categories)😁

Here's a dump of the test I was attempting to add with the following options:

cl::OptionCategory Cat1("Category One", "  Category One description");
cl::OptionCategory Cat2("Category Two", "  Category Two description");
cl::opt<bool> GeneralOpt("general-option");
cl::opt<bool> Cat1Opt("category1-option", cl::cat(Cat1));
cl::opt<bool> Cat2Opt("category2-option", cl::cat(Cat2));
cl::opt<bool> Cat1AndCat2Opt("category1-and-category2-option", cl::cat(Cat1), cl::cat(Cat2));

USAGE: prog [options]

OPTIONS:

Category One:
  Category One description

  --category1-and-category2-option    -
  --category1-option                  -

Category Two:
  Category Two description

  --category1-and-category2-option    -
  --category2-option                  -

Color Options:

  --color                             - Use colors in output (default=autodetect)

General options:

  --env-test-opt=<string>             -
  --general-option                    -
  --program-test-string-arg1=<string> -
  --program-test-string-arg2=<string> -
  --time-trace-granularity=<uint>     - Minimum time granularity (in microseconds) traced by time profiler

Generic Options:

  --help                              - Display available options (--help-hidden for more)
  --help-list                         - Display list of available options (--help-list-hidden for more)
  --version                           - Display the version of this program

[hintonda]

• Address comments

llvm/include/llvm/Support/CommandLine.h

class Option {
   unsigned Misc : 5;
   unsigned Position = 0;       // Position of last occurrence of the option
   unsigned AdditionalVals = 0; // Greater than 0 for multi-valued option.
 
 public:
   StringRef ArgStr;   // The argument string itself (ex: "help", "o")
   StringRef HelpStr;  // The descriptive text message for -help
   StringRef ValueStr; // String describing what the value of this option is
-  OptionCategory *Category; // The Category this option belongs to
+  SmallVector<OptionCategory *, 2>
+      Categories;                    // The Categories this option belongs to
   SmallPtrSet<SubCommand *, 4> Subs; // The subcommands this option belongs to.
   bool FullyInitialized = false; // Has addArgument been called?
 
   inline enum NumOccurrencesFlag getNumOccurrencesFlag() const {
     return (enum NumOccurrencesFlag)Occurrences;
   }
 
   inline enum ValueExpected getValueExpectedFlag() const {
public:
   void setDescription(StringRef S) { HelpStr = S; }
   void setValueStr(StringRef S) { ValueStr = S; }
   void setNumOccurrencesFlag(enum NumOccurrencesFlag Val) { Occurrences = Val; }
   void setValueExpectedFlag(enum ValueExpected Val) { Value = Val; }
   void setHiddenFlag(enum OptionHidden Val) { HiddenFlag = Val; }
   void setFormattingFlag(enum FormattingFlags V) { Formatting = V; }
   void setMiscFlag(enum MiscFlags M) { Misc |= M; }
   void setPosition(unsigned pos) { Position = pos; }
-  void setCategory(OptionCategory &C) { Category = &C; }
+  void addCategory(OptionCategory &C);
   void addSubCommand(SubCommand &S) { Subs.insert(&S); }
 
 protected:
   explicit Option(enum NumOccurrencesFlag OccurrencesFlag,
                   enum OptionHidden Hidden)
       : Occurrences(OccurrencesFlag), Value(0), HiddenFlag(Hidden),
-        Formatting(NormalFormatting), Misc(0), Category(&GeneralCategory) {}
+        Formatting(NormalFormatting), Misc(0) {
+    Categories.push_back(&GeneralCategory);
+  }
 
   inline void setNumAdditionalVals(unsigned n) { AdditionalVals = n; }
 
 public:
   virtual ~Option() = default;
 
   // addArgument - Register this argument with the commandline system.
   //

 
 // cat - Specifiy the Option category for the command line argument to belong
 // to.
 struct cat {
   OptionCategory &Category;
 
   cat(OptionCategory &c) : Category(c) {}
 
-  template <class Opt> void apply(Opt &O) const { O.setCategory(Category); }
+  template <class Opt> void apply(Opt &O) const { O.addCategory(Category); }
 };
 
 // sub - Specify the subcommand that this option belongs to.
 struct sub {
   SubCommand &Sub;
 
   sub(SubCommand &S) : Sub(S) {}
 
class alias : public Option {
   void done() {
     if (!hasArgStr())
       error("cl::alias must have argument name specified!");
     if (!AliasFor)
       error("cl::alias must have an cl::aliasopt(option) specified!");
     if (!Subs.empty())
       error("cl::alias must not have cl::sub(), aliased option's cl::sub() will be used!");
     Subs = AliasFor->Subs;
-    Category = AliasFor->Category;
+    Categories = AliasFor->Categories;
     addArgument();
   }
 
 public:
   // Command line options should not be copyable
   alias(const alias &) = delete;
   alias &operator=(const alias &) = delete;
 

llvm/lib/Support/CommandLine.cpp

void Option::setArgStr(StringRef S) {
   if (FullyInitialized)
     GlobalParser->updateArgStr(this, S);
   assert((S.empty() || S[0] != '-') && "Option can't start with '-");
   ArgStr = S;
   if (ArgStr.size() == 1)
     setMiscFlag(Grouping);
 }
 
+void Option::addCategory(OptionCategory &C) {
+  assert(!Categories.empty() && "Categories cannot be empty.");
+  // Maintain backward compatibility by replacing the default GeneralCategory
+  // if it's still set.  Otherwise, just add the new one.  The GeneralCategory
+  // must be explicitly added if you want multiple categories that include it.
+  if (&C != &GeneralCategory && Categories[0] == &GeneralCategory)
+    Categories[0] = &C;
+  else
+    Categories.push_back(&C);
+}
+
 void Option::reset() {
   NumOccurrences = 0;
   setDefault();
   if (isDefaultOption())
     removeArgument();
 }
 
 // Initialise the general option category.
for (std::vector<OptionCategory *>::const_iterator
          I != E; ++I)
       CategorizedOptions[*I] = std::vector<Option *>();
 
     // Walk through pre-sorted options and assign into categories.
     // Because the options are already alphabetically sorted the
     // options within categories will also be alphabetically sorted.
     for (size_t I = 0, E = Opts.size(); I != E; ++I) {
       Option *Opt = Opts[I].second;
-      assert(CategorizedOptions.count(Opt->Category) > 0 &&
+      for (auto &Cat : Opt->Categories) {
+        assert(CategorizedOptions.count(Cat) > 0 &&
                "Option has an unregistered category");
-      CategorizedOptions[Opt->Category].push_back(Opt);
+        CategorizedOptions[Cat].push_back(Opt);
+      }
     }
 
     // Now do printing.
     for (std::vector<OptionCategory *>::const_iterator
              Category = SortedCategories.begin(),
              E = SortedCategories.end();
          Category != E; ++Category) {
       // Hide empty categories for --help, but show for --help-hidden.

 
 iterator_range<typename SmallPtrSet<SubCommand *, 4>::iterator>
 cl::getRegisteredSubcommands() {
   return GlobalParser->getRegisteredSubcommands();
 }
 
 void cl::HideUnrelatedOptions(cl::OptionCategory &Category, SubCommand &Sub) {
   for (auto &I : Sub.OptionsMap) {
-    if (I.second->Category != &Category &&
-        I.second->Category != &GenericCategory)
+    for (auto &Cat : I.second->Categories) {
+      if (Cat != &Category &&
+          Cat != &GenericCategory)
         I.second->setHiddenFlag(cl::ReallyHidden);
     }
   }
+}
 
 void cl::HideUnrelatedOptions(ArrayRef<const cl::OptionCategory *> Categories,
                               SubCommand &Sub) {
   auto CategoriesBegin = Categories.begin();
   auto CategoriesEnd = Categories.end();
   for (auto &I : Sub.OptionsMap) {
-    if (std::find(CategoriesBegin, CategoriesEnd, I.second->Category) ==
-            CategoriesEnd &&
-        I.second->Category != &GenericCategory)
+    for (auto &Cat : I.second->Categories) {
+      if (std::find(CategoriesBegin, CategoriesEnd, Cat) ==
+          Cat != &GenericCategory)
         I.second->setHiddenFlag(cl::ReallyHidden);
     }
   }
+}
 
 void cl::ResetCommandLineParser() { GlobalParser->reset(); }
 void cl::ResetAllOptionOccurrences() {
   GlobalParser->ResetAllOptionOccurrences();
 }
 
 void LLVMParseCommandLineOptions(int argc, const char *const *argv,
                                  const char *Overview) {
   llvm::cl::ParseCommandLineOptions(argc, argv, StringRef(Overview),
                                     &llvm::nulls());
 }

llvm/unittests/Support/CommandLineTest.cpp

StringMap<cl::Option *> &Map =
       cl::getRegisteredOptions(*cl::TopLevelSubCommand);
 
   ASSERT_TRUE(Map.count("test-option") == 1) <<
     "Could not find option in map.";
 
   cl::Option *Retrieved = Map["test-option"];
   ASSERT_EQ(&TestOption, Retrieved) << "Retrieved wrong option.";
 
-  ASSERT_EQ(&cl::GeneralCategory,Retrieved->Category) <<
-    "Incorrect default option category.";
-
-  Retrieved->setCategory(TestCategory);
-  ASSERT_EQ(&TestCategory,Retrieved->Category) <<
-    "Failed to modify option's option category.";
+  ASSERT_NE(Retrieved->Categories.end(),
+            std::find_if(Retrieved->Categories.begin(),
+                         Retrieved->Categories.end(),
+                         [&](const llvm::cl::OptionCategory *Cat) {
+                           return Cat == &cl::GeneralCategory;
+                         }))
+      << "Incorrect default option category.";
+
+  Retrieved->addCategory(TestCategory);
+  ASSERT_NE(Retrieved->Categories.end(),
+            std::find_if(Retrieved->Categories.begin(),
+                         Retrieved->Categories.end(),
+                         [&](const llvm::cl::OptionCategory *Cat) {
+                           return Cat == &TestCategory;
+                         }))
+      << "Failed to modify option's option category.";
 
   Retrieved->setDescription(Description);
   ASSERT_STREQ(Retrieved->HelpStr.data(), Description)
       << "Changing option description failed.";
 
   Retrieved->setArgStr(ArgString);
   ASSERT_STREQ(ArgString, Retrieved->ArgStr.data())
       << "Failed to modify option's Argument string.";
TEST(CommandLineTest, ParseEnvironmentToLocalVar) {
   EXPECT_EQ("hello-local", EnvironmentTestOptionLocal);
 }
 
 #endif  // SKIP_ENVIRONMENT_TESTS
 
 TEST(CommandLineTest, UseOptionCategory) {
   StackOption<int> TestOption2("test-option", cl::cat(TestCategory));
 
-  ASSERT_EQ(&TestCategory,TestOption2.Category) << "Failed to assign Option "
-                                                  "Category.";
+  ASSERT_NE(TestOption2.Categories.end(),
+            std::find_if(TestOption2.Categories.begin(),
+                         TestOption2.Categories.end(),
+                         [&](const llvm::cl::OptionCategory *Cat) {
+                           return Cat == &TestCategory;
+                         }))
+      << "Failed to assign Option Category.";
+}
+
+TEST(CommandLineTest, UseMultipleCategories) {
+  StackOption<int> TestOption2("test-option2", cl::cat(TestCategory),
+                               cl::cat(cl::GeneralCategory));
+
+  ASSERT_NE(TestOption2.Categories.end(),
+            std::find_if(TestOption2.Categories.begin(),
+                         TestOption2.Categories.end(),
+                         [&](const llvm::cl::OptionCategory *Cat) {
+                           return Cat == &TestCategory;
+                         }))
+      << "Failed to assign Option Category.";
+  ASSERT_NE(TestOption2.Categories.end(),
+            std::find_if(TestOption2.Categories.begin(),
+                         TestOption2.Categories.end(),
+                         [&](const llvm::cl::OptionCategory *Cat) {
+                           return Cat == &cl::GeneralCategory;
+                         }))
+      << "Failed to assign General Category.";
+
+  cl::OptionCategory AnotherCategory("Additional test Options", "Description");
+  StackOption<int> TestOption("test-option", cl::cat(TestCategory),
+                              cl::cat(AnotherCategory));
+  ASSERT_EQ(TestOption.Categories.end(),
+            std::find_if(TestOption.Categories.begin(),
+                         TestOption.Categories.end(),
+                         [&](const llvm::cl::OptionCategory *Cat) {
+                           return Cat == &cl::GeneralCategory;
+                         }))
+      << "Failed to remove General Category.";
+  ASSERT_NE(TestOption.Categories.end(),
+            std::find_if(TestOption.Categories.begin(),
+                         TestOption.Categories.end(),
+                         [&](const llvm::cl::OptionCategory *Cat) {
+                           return Cat == &TestCategory;
+                         }))
+      << "Failed to assign Option Category.";
+  ASSERT_NE(TestOption.Categories.end(),
+            std::find_if(TestOption.Categories.begin(),
+                         TestOption.Categories.end(),
+                         [&](const llvm::cl::OptionCategory *Cat) {
+                           return Cat == &AnotherCategory;
+                         }))
+      << "Failed to assign Another Category.";
 }
 
 typedef void ParserFunction(StringRef Source, StringSaver &Saver,
                             SmallVectorImpl<const char *> &NewArgv,
                             bool MarkEOLs);
 
 void testCommandLineTokenizer(ParserFunction *parse, StringRef Input,
                               const char *const Output[], size_t OutputSize) {