[include-mapping] Parse zombie_names.html into a removed symbols map.

[hokein]

Have you run the script on the zombie_names page? Does it work?

clang/tools/include-mapping/gen_std.py

68

instead adding two CLI flags, I'd suggest extending the existing language flag to something like -symbols = {cpp_symbols, c_symbols, cpp_removed_symbols}, it is easy to extend in the future (e.g. c_removed_symbols).

[VitaNuo]

Address review comments.

[VitaNuo]

Remove extra whitespace.

[VitaNuo]

Adjust argument value set.

[VitaNuo]

Yes, ofc I've run it, I wouldn't send a patch otherwise :)

clang/tools/include-mapping/gen_std.py

68

Sure, good idea!

[hokein]

mostly good, just nits.

clang/tools/include-mapping/gen_std.py

31–33

nit: update the doc.

99–100

nit: can we add a _ between cpp and removed. And maybe move it after the if args.symbols == 'cpp' branch as they are cpp related.

[VitaNuo]

Minor adjustments in accordance with review comments.

[VitaNuo]

Thanks for the comments!

clang/tools/include-mapping/gen_std.py

31–33

Ah thanks for the reminder!

99–100

Ok, sure.

clang/tools/include-mapping/gen_std.py

Usage:
   1. Install BeautifulSoup dependency, see instruction:
        https://www.crummy.com/software/BeautifulSoup/bs4/doc/#installing-beautiful-soup
   2. Download cppreference offline HTML files (e.g. html_book_20181028.zip) at
        https://en.cppreference.com/w/Cppreference:Archives
   3. Unzip the zip file from step 2 (e.g., to a "cppreference" directory). You should
      get a "cppreference/reference" directory.
   4. Run the command:
        // Generate C++ symbols
-       python3 gen_std.py -cppreference cppreference/reference -language=cpp > StdSymbolMap.inc
+       python3 gen_std.py -cppreference cppreference/reference -symbols=cpp > StdSymbolMap.inc
+       // Generate C++ removed symbols
+       python3 gen_std.py -cppreference cppreference/reference -symbols=cpp_removed > RemovedSymbolMap.inc
        // Generate C symbols
-       python3 gen_std.py -cppreference cppreference/reference -language=c > CSymbolMap.inc
+       python3 gen_std.py -cppreference cppreference/reference -symbols=c > CSymbolMap.inc
 """
 
 
 import cppreference_parser
 import argparse
 import datetime
 import os
 import sys

 
 def ParseArg():
   parser = argparse.ArgumentParser(description='Generate StdGen file')
   parser.add_argument('-cppreference', metavar='PATH',
                       default='',
                       help='path to the cppreference offline HTML directory',
                       required=True
                      )
-  parser.add_argument('-language',
+  parser.add_argument('-symbols',
                       default='cpp',
-                      help='Generate c or cpp symbols',
-                      required=True)
+                      help='Generate c or cpp (removed) symbols. One of {cpp, c, cpp_removed}.',
+                      required=True) 
   return parser.parse_args()
 
 
 def main():
   args = ParseArg()
-  if args.language == 'cpp':
+  if args.symbols == 'cpp':
     page_root = os.path.join(args.cppreference, "en", "cpp")
     symbol_index_root = os.path.join(page_root, "symbol_index")
     parse_pages =  [
       (page_root, "symbol_index.html", "std::"),
       # std sub-namespace symbols have separated pages.
       # We don't index std literal operators (e.g.
       # std::literals::chrono_literals::operator""d), these symbols can't be
       # accessed by std::<symbol_name>.
       # FIXME: index std::placeholders symbols, placeholders.html page is
       # different (which contains one entry for _1, _2, ..., _N), we need special
       # handling.
       (symbol_index_root, "chrono.html", "std::chrono::"),
       (symbol_index_root, "filesystem.html", "std::filesystem::"),
       (symbol_index_root, "pmr.html", "std::pmr::"),
       (symbol_index_root, "regex_constants.html", "std::regex_constants::"),
       (symbol_index_root, "this_thread.html", "std::this_thread::"),
     ]
-  elif args.language == 'c':
+  elif args.symbols == 'cpp_removed':
+    page_root = os.path.join(args.cppreference, "en", "cpp")
+    symbol_index_root = os.path.join(page_root, "symbol_index")
+    parse_pages = [(symbol_index_root, "zombie_names.html", "std::")]
+  elif args.symbols == 'c':
     page_root = os.path.join(args.cppreference, "en", "c")
     symbol_index_root = page_root
-    parse_pages = [(page_root, "index.html", None)]
+    parse_pages = [(page_root, "index.html", None)]  
     
   if not os.path.exists(symbol_index_root):
     exit("Path %s doesn't exist!" % symbol_index_root)
 
   symbols = cppreference_parser.GetSymbols(parse_pages)
   
   # We don't have version information from the unzipped offline HTML files.
   # so we use the modified time of the symbol_index.html as the version.
   index_page_path = os.path.join(page_root, "index.html")
   cppreference_modified_date = datetime.datetime.fromtimestamp(
     os.stat(index_page_path).st_mtime).strftime('%Y-%m-%d')
-  print(CODE_PREFIX % (args.language.upper(), cppreference_modified_date))
+  print(CODE_PREFIX % (args.symbols.upper(), cppreference_modified_date))
   for symbol in symbols:
     if len(symbol.headers) == 1:
       # SYMBOL(unqualified_name, namespace, header)
       print("SYMBOL(%s, %s, %s)" % (symbol.name, symbol.namespace,
                                     symbol.headers[0]))
     elif len(symbol.headers) == 0:
       sys.stderr.write("No header found for symbol %s\n" % symbol.name)
     else:
       # FIXME: support symbols with multiple headers (e.g. std::move).
       sys.stderr.write("Ambiguous header for symbol %s: %s\n" % (
           symbol.name, ', '.join(symbol.headers)))
 
 
 if __name__ == '__main__':
   main()