[clang][tablegen] adds human documentation to `WarningOption`

[cjdb]

Summary

Building on D126796, this commit adds the infrastructure for being able
to print out descriptions of what each warning does.

[aaron.ballman]

Can you post some examples of the output from this option so we can see what the end results look like more easily?

[cjdb]

In D126832#3553569, @aaron.ballman wrote:

Can you post some examples of the output from this option so we can see what the end results look like more easily?

Right now this doesn't do anything at all, except feed the documentation from tablegen to Clang. It's just taking the text (represented by ...) from code Documentation [{...}], trimming any surrounding spaces, and putting that into DIAG_ENTRY.

e.g.

code Documentation [{
  Hello, world!

  Goodbye, world!
}]

would be forwarded as "Hello, world!\n\n Goodbye, world!"

[aaron.ballman]

In D126832#3554816, @cjdb wrote:

In D126832#3553569, @aaron.ballman wrote:

Can you post some examples of the output from this option so we can see what the end results look like more easily?

Right now this doesn't do anything at all, except feed the documentation from tablegen to Clang. It's just taking the text (represented by ...) from code Documentation [{...}], trimming any surrounding spaces, and putting that into DIAG_ENTRY.

e.g.

code Documentation [{
  Hello, world!

  Goodbye, world!
}]

would be forwarded as "Hello, world!\n\n Goodbye, world!"

Thanks, that helps me to visualize what's going on. Any thoughts on how we could test this functionality, or are we going to assume that the testing comes from its usage when we actually make use of the new information threaded in?

clang/lib/Basic/DiagnosticIDs.cpp

628

clang/utils/TableGen/ClangDiagnosticsEmitter.cpp

1550–1553

Isn't this functionally equivalent? (It means we're using a raw empty string rather than a normal empty string, but that shouldn't matter.)

[cjdb]

applies feedback

[cjdb]

In D126832#3567035, @aaron.ballman wrote:

In D126832#3554816, @cjdb wrote:

In D126832#3553569, @aaron.ballman wrote:

Can you post some examples of the output from this option so we can see what the end results look like more easily?

Right now this doesn't do anything at all, except feed the documentation from tablegen to Clang. It's just taking the text (represented by ...) from code Documentation [{...}], trimming any surrounding spaces, and putting that into DIAG_ENTRY.

e.g.

code Documentation [{
  Hello, world!

  Goodbye, world!
}]

would be forwarded as "Hello, world!\n\n Goodbye, world!"

Thanks, that helps me to visualize what's going on. Any thoughts on how we could test this functionality, or are we going to assume that the testing comes from its usage when we actually make use of the new information threaded in?

Good question. Unless this functionality already has tests (and I'm gathering it doesn't), I think we can rely on Clang being the test. (Though we should consider ourselves on notice for adding robust unit tests at some point.)

clang/utils/TableGen/ClangDiagnosticsEmitter.cpp

1550–1553

Wow, I can no longer say I've never done if (condition) return true; else return false; 😂

[aaron.ballman]

Good question. Unless this functionality already has tests (and I'm gathering it doesn't), I think we can rely on Clang being the test. (Though we should consider ourselves on notice for adding robust unit tests at some point.)

I don't know of tests for it, so I'm okay with that approach. LGTM!

clang/docs/ReleaseNotes.rst

- Improved the diagnostic when accessing a member of an atomic structure or
   union object in C; was previously an unhelpful error, but now issues a
   ``-Watomic-access`` warning which defaults to an error. Fixes
   `Issue 54563 <https://github.com/llvm/llvm-project/issues/54563>`_.
 - Unevaluated lambdas in dependant contexts no longer result in clang crashing.
   This fixes Issues `50376 <https://github.com/llvm/llvm-project/issues/50376>`_,
   `51414 <https://github.com/llvm/llvm-project/issues/51414>`_,
   `51416 <https://github.com/llvm/llvm-project/issues/51416>`_,
   and `51641 <https://github.com/llvm/llvm-project/issues/51641>`_.
-- The builtin function __builtin_dump_struct would crash clang when the target 
+- The builtin function __builtin_dump_struct would crash clang when the target
   struct contains a bitfield. It now correctly handles bitfields.
   This fixes Issue `Issue 54462 <https://github.com/llvm/llvm-project/issues/54462>`_.
 - Statement expressions are now disabled in default arguments in general.
   This fixes Issue `Issue 53488 <https://github.com/llvm/llvm-project/issues/53488>`_.
-- According to `CWG 1394 <https://wg21.link/cwg1394>`_ and 
+- According to `CWG 1394 <https://wg21.link/cwg1394>`_ and
   `C++20 [dcl.fct.def.general]p2 <https://timsong-cpp.github.io/cppwp/n4868/dcl.fct.def#general-2.sentence-3>`_,
   Clang should not diagnose incomplete types in function definitions if the function body is "= delete;".
   This fixes Issue `Issue 52802 <https://github.com/llvm/llvm-project/issues/52802>`_.
 - Unknown type attributes with a ``[[]]`` spelling are no longer diagnosed twice.
   This fixes Issue `Issue 54817 <https://github.com/llvm/llvm-project/issues/54817>`_.
 - Clang should no longer incorrectly diagnose a variable declaration inside of
   a lambda expression that shares the name of a variable in a containing
   if/while/for/switch init statement as a redeclaration.

 - Fixed a false positive diagnostic about an unevaluated expression having no
   side effects when the expression is of VLA type and is an operand of the
   ``sizeof`` operator. Fixes `Issue 48010 <https://github.com/llvm/llvm-project/issues/48010>`_.
 - Fixed a false positive diagnostic about scoped enumerations being a C++11
   extension in C mode. A scoped enumeration's enumerators cannot be named in C
   because there is no way to fully qualify the enumerator name, so this
   "extension" was unintentional and useless. This fixes
   `Issue 42372 <https://github.com/llvm/llvm-project/issues/42372>`_.
-- Clang will now find and emit a call to an allocation function in a 
-  promise_type body for coroutines if there is any allocation function 
+- Clang will now find and emit a call to an allocation function in a
+  promise_type body for coroutines if there is any allocation function
   declaration in the scope of promise_type. Additionally, to implement CWG2585,
   a coroutine will no longer generate a call to a global allocation function
   with the signature (std::size_t, p0, ..., pn).
   This fixes Issue `Issue 54881 <https://github.com/llvm/llvm-project/issues/54881>`_.
 - Implement `CWG 2394 <https://wg21.link/cwg2394>`_: Const class members
   may be initialized with a defaulted default constructor under the same
   conditions it would be allowed for a const object elsewhere.
 - ``__has_unique_object_representations`` no longer reports that ``_BitInt`` types
- A new ARM pass to workaround Cortex-A57 Erratum 1742098 and Cortex-A72 Erratum
   ``-mfix-cortex-a72-aes-1655431``. The pass is enabled when using either of
   these cpus with ``-mcpu=`` and can be disabled using
   ``-mno-fix-cortex-a57-aes-1742098`` or ``-mno-fix-cortex-a72-aes-1655431``.
 - Added the ``-fexperimental-max-bitint-width=`` option to increase the maximum
   allowed bit width of ``_BitInt`` types beyond the default of 128 bits. Some
   operations, such as division or float-to-integer conversion, on ``_BitInt``
   types with more than 128 bits currently crash clang. This option will be
   removed in the future once clang supports all such operations.
+- Added the ``-print-diagnostic-options`` option, which prints a list of
+  warnings the compiler supports.
 
 Deprecated Compiler Flags
 -------------------------
 
 Modified Compiler Flags
 -----------------------
 
 Removed Compiler Flags
- `New CTU implementation
   that keeps the slow-down around 2x compared to the single-TU analysis, even
   in case of complex C++ projects. Still, it finds the majority of the "old"
   CTU findings. Besides, not more than ~3% of the bug reports are lost compared
   to single-TU analysis, the lost reports are highly likely to be false
   positives.
 
 - Added a new checker ``alpha.unix.cstring.UninitializedRead`` this will check for uninitialized reads
   from common memory copy/manipulation functions such as ``memcpy``, ``mempcpy``, ``memmove``, ``memcmp``, `
-  `strcmp``, ``strncmp``, ``strcpy``, ``strlen``, ``strsep`` and many more. Although 
+  `strcmp``, ``strncmp``, ``strcpy``, ``strlen``, ``strsep`` and many more. Although
   this checker currently is in list of alpha checkers due to a false positive.
 
 .. _release-notes-ubsan:
 
 Undefined Behavior Sanitizer (UBSan)
 ------------------------------------
 
 Core Analysis Improvements

clang/include/clang/Basic/DiagnosticCategories.h

 #define CATEGORY(X, ENUM) ENUM,
 #include "clang/Basic/DiagnosticGroups.inc"
 #undef CATEGORY
 #undef GET_CATEGORY_TABLE
       DiagCat_NUM_CATEGORIES
     };
 
     enum class Group {
-#define DIAG_ENTRY(GroupName, FlagNameOffset, Members, SubGroups) GroupName,
+#define DIAG_ENTRY(GroupName, FlagNameOffset, Members, SubGroups, Docs)        \
+  GroupName,
 #include "clang/Basic/DiagnosticGroups.inc"
 #undef CATEGORY
 #undef DIAG_ENTRY
     };
   }  // end namespace diag
 }  // end namespace clang
 
 #endif

clang/include/clang/Basic/DiagnosticIDs.h

public:
   ///
   static bool isBuiltinExtensionDiag(unsigned DiagID, bool &EnabledByDefault);
 
   /// Given a group ID, returns the flag that toggles the group.
   /// For example, for Group::DeprecatedDeclarations, returns
   /// "deprecated-declarations".
   static StringRef getWarningOptionForGroup(diag::Group);
 
+  /// Given a diagnostic group ID, return its documentation.
+  static StringRef getWarningOptionDocumentation(diag::Group GroupID);
+
   /// Given a group ID, returns the flag that toggles the group.
   /// For example, for "deprecated-declarations", returns
   /// Group::DeprecatedDeclarations.
   static llvm::Optional<diag::Group> getGroupForWarningOption(StringRef);
 
   /// Return the lowest-level group that contains the specified diagnostic.
   static llvm::Optional<diag::Group> getGroupForDiag(unsigned DiagID);
 

clang/lib/Basic/DiagnosticIDs.cpp

 #include "clang/Basic/DiagnosticGroups.inc"
 #undef GET_DIAG_ARRAYS
 
 namespace {
   struct WarningOption {
     uint16_t NameOffset;
     uint16_t Members;
     uint16_t SubGroups;
+    StringRef Documentation;
 
     // String is stored with a pascal-style length byte.
     StringRef getName() const {
       return StringRef(DiagGroupNames + NameOffset + 1,
                        DiagGroupNames[NameOffset]);
     }
   };
 }
 
 // Second the table of options, sorted by name for fast binary lookup.
 static const WarningOption OptionTable[] = {
-#define DIAG_ENTRY(GroupName, FlagNameOffset, Members, SubGroups)              \
-  {FlagNameOffset, Members, SubGroups},
+#define DIAG_ENTRY(GroupName, FlagNameOffset, Members, SubGroups, Docs)        \
+  {FlagNameOffset, Members, SubGroups, Docs},
 #include "clang/Basic/DiagnosticGroups.inc"
 #undef DIAG_ENTRY
 };
 
+/// Given a diagnostic group ID, return its documentation.
+StringRef DiagnosticIDs::getWarningOptionDocumentation(diag::Group Group) {
+  return OptionTable[static_cast<int>(Group)].Documentation;
+}
+
 StringRef DiagnosticIDs::getWarningOptionForGroup(diag::Group Group) {
   return OptionTable[static_cast<int>(Group)].getName();
 }
 
 llvm::Optional<diag::Group>
 DiagnosticIDs::getGroupForWarningOption(StringRef Name) {
   const auto *Found = llvm::partition_point(
       OptionTable, [=](const WarningOption &O) { return O.getName() < Name; });

clang/tools/diagtool/DiagnosticNames.cpp

 
 
 #define GET_DIAG_ARRAYS
 #include "clang/Basic/DiagnosticGroups.inc"
 #undef GET_DIAG_ARRAYS
 
 // Second the table of options, sorted by name for fast binary lookup.
 static const GroupRecord OptionTable[] = {
-#define DIAG_ENTRY(GroupName, FlagNameOffset, Members, SubGroups)              \
+#define DIAG_ENTRY(GroupName, FlagNameOffset, Members, SubGroups, Docs)        \
   {FlagNameOffset, Members, SubGroups},
 #include "clang/Basic/DiagnosticGroups.inc"
 #undef DIAG_ENTRY
 };
 
 llvm::StringRef GroupRecord::getName() const {
   return StringRef(DiagGroupNames + NameOffset + 1, DiagGroupNames[NameOffset]);
 }

clang/utils/TableGen/ClangDiagnosticsEmitter.cpp

if (hasDiags) {
       OS << "0, ";
     }
 
     // Subgroups.
     const std::vector<std::string> &SubGroups = I.second.SubGroups;
     const bool hasSubGroups =
         !SubGroups.empty() || (IsPedantic && !GroupsInPedantic.empty());
     if (hasSubGroups) {
-      OS << "/* DiagSubGroup" << I.second.IDNo << " */ " << SubGroupIndex;
+      OS << "/* DiagSubGroup" << I.second.IDNo << " */ " << SubGroupIndex
+         << ", ";
       if (IsPedantic)
         SubGroupIndex += GroupsInPedantic.size();
       SubGroupIndex += SubGroups.size() + 1;
     } else {
-      OS << "0";
+      OS << "0, ";
     }
 
+    std::string Documentation = I.second.Defs.back()
+                                    ->getValue("Documentation")
+                                    ->getValue()
+                                    ->getAsUnquotedString();
+
+    OS << "R\"(" << StringRef(Documentation).trim() << ")\"";
+
     OS << ")\n";
   }
   OS << "#endif // DIAG_ENTRY\n\n";
 }
 
 /// Emit the table of diagnostic categories.
 ///
 /// The table has the form of macro calls that have two parameters. The
 /// category's name as well as an enum that represents the category. The