[clang][docs] Defensively turn off exception behavior in dump_ast_matchers.py

[t-rasmud]

Summary

This patch turns exceptions (that are being triggered in most cases) into defensive prints

[NoQ]

Yes, as far as I understand, the URL request is more of an "assert" thing; it confirms that there's a doxygen page for the class mentioned in the code, but I'd be very surprised if there are any actual classes that you can use in the C++ code for which there's no doxygen page.

This check was running into problems on, uh, certain corporate networks with unusual SSL certificate manipulations, and I think it makes sense to make it less mandatory. It's still checking all URLs by default whenever it can, so it'll start flagging broken URLs anyway if they ever appear, it just unblocks progress for folks who need or prefer it to be offline for some reason. Which is really helpful when it's like your first time running this script and you've no idea what to expect.

clang/docs/tools/dump_ast_matchers.py

62–66

It's probably better to be explicit about what we expect to happen here. This is just a string operation, it probably can't go wrong in many ways.

[t-rasmud]

Address feedback.

[ymandel]

nit: I would suggest splitting the update to the docs from the modification to the script itself, since they are unrelated.

[t-rasmud]

Splitting the update: Updates to LibASTMAtchersReference.html are now in https://reviews.llvm.org/D155304

[jkorous]

LGTM

clang/docs/tools/dump_ast_matchers.py

try:
     from urllib.request import urlopen
 except ImportError:
     from urllib2 import urlopen
 
 CLASS_INDEX_PAGE_URL = "https://clang.llvm.org/doxygen/classes.html"
 try:
     CLASS_INDEX_PAGE = urlopen(CLASS_INDEX_PAGE_URL).read().decode("utf-8")
 except Exception as e:
-    raise Exception("Unable to get %s: %s" % (CLASS_INDEX_PAGE_URL, e))
+    CLASS_INDEX_PAGE = None
+    print("Unable to get %s: %s" % (CLASS_INDEX_PAGE_URL, e))
 
 MATCHERS_FILE = "../../include/clang/ASTMatchers/ASTMatchers.h"
 
 # Each matcher is documented in one row of the form:
 #   result | name | argA
 # The subsequent row contains the documentation and is hidden by default,
 # becoming visible via javascript when the user clicks the matcher name.
 TD_TEMPLATE = """
def link_if_exists(m):
 
         We want to do this only if the page exists, in which case it will be
         referenced from the class index page.
         """
         name = m.group(1)
         url = "https://clang.llvm.org/doxygen/classclang_1_1%s.html" % name
         if url not in doxygen_probes:
             search_str = 'href="classclang_1_1%s.html"' % name
+            if CLASS_INDEX_PAGE is not None:
                 doxygen_probes[url] = search_str in CLASS_INDEX_PAGE
+            else:
+                doxygen_probes[url] = True
             if not doxygen_probes[url]:
                 print("Did not find %s in class index page" % name)
         if doxygen_probes[url]:
             return r'Matcher&lt;<a href="%s">%s</a>&gt;' % (url, name)
         else:
             return m.group(0)
 
     text = re.sub(r"Matcher&lt;([^\*&]+)&gt;", link_if_exists, text)
     return text
def act_on_decl(declaration, comment, allowed_types):
     """Parse the matcher out of the given declaration and comment.
 
     If 'allowed_types' is set, it contains a list of node types the matcher
     can match on, as extracted from the static type asserts in the matcher
     definition.
     """
     if declaration.strip():
 
-        if re.match(r"^\s?(#|namespace|using)", declaration):
+        if re.match(r"^\s?(#|namespace|using|template <typename NodeType> using|})", declaration):
             return
 
         # Node matchers are defined by writing:
         #   VariadicDynCastAllOfMatcher<ResultType, ArgumentType> name;
         m = re.match(
             r""".*Variadic(?:DynCast)?AllOfMatcher\s*<
                        \s*([^\s,]+)\s*(?:,
                        \s*([^\s>]+)\s*)?>