[clang][docs] Remove untracked files from formatted status

[njames93]

Summary

Currently on http://clang.llvm.org/docs/ClangFormattedStatus.html there are format stats on files no actually inside the tree but generated by build scripts. These are usually copied from somewhere else. Right now for example there are files from llvm/utils/release/llvm-package.... Adding these files bloats the list while not giving an accurate representation of how formatted the repo is.
This addresses this issue by checking the git index and ignoring any folder that doesn't contain tracked files.

I'm still unsure whether it would be better to just do away with the os.walk method and just check over every file returned from git ls-index <project-root>.

[MyDeveloperDay]

Thank you for this, I’m happy with this and can take a look at removing the os.walk I agree it might be better/quicker

[MyDeveloperDay]

The net result is an actual reduction in the overall %

[njames93]

In D82707#2118772, @MyDeveloperDay wrote:

The net result is an actual reduction in the overall %

Is that a bad thing?

[MyDeveloperDay]

In D82707#2118796, @njames93 wrote:

In D82707#2118772, @MyDeveloperDay wrote:

The net result is an actual reduction in the overall %

Is that a bad thing?

Not really, just a little disappointed ;-(

[njames93]

In D82707#2119303, @MyDeveloperDay wrote:

In D82707#2118796, @njames93 wrote:

Is that a bad thing?

Not really, just a little disappointed ;-(

I'm gonna hazard a guess the removal of these directories is the reason the % went down

clang/docs/tools/generate_formatted_state.py

 table_row = """\
    * - {path}
      - {style}`{count}`
      - {style}`{passes}`
      - {style}`{fails}`
      - {style2}`{percent}%`
 """
 
+FNULL = open(os.devnull, 'w')
+
 with open(DOC_FILE, 'wb') as output:
     sha = get_git_revision_short_hash()
     today = datetime.now().strftime("%B %d, %Y %H:%M:%S")
     output.write(bytes(rst_prefix.format(today=today,
                                          sha=sha).encode("utf-8")))
 
     total_files_count = 0
     total_files_pass = 0
     total_files_fail = 0
     for root, subdirs, files in os.walk(rootdir):
         for subdir in subdirs:
             if any(sd == subdir for sd in skipped_dirs):
                 subdirs.remove(subdir)
+            else:
+                act_sub_dir = os.path.join(root, subdir)
+                # Check the git index to see if the directory contains tracked
+                # files. Reditect the output to a null descriptor as we aren't
+                # interested in it, just the return code.
+                git_check = subprocess.Popen(
+                    ["git", "ls-files", "--error-unmatch", act_sub_dir],
+                    stdout=FNULL,
+                    stderr=FNULL)
+                if git_check.wait() != 0:
+                    print("Skipping directory: ", act_sub_dir)
+                    subdirs.remove(subdir)
 
         path = os.path.relpath(root, TOP_DIR)
         path = path.replace('\\', '/')
 
-        head, _ = os.path.split(root)
-        while head:
-            head, _ = os.path.split(head)
-
         file_count = 0
         file_pass = 0
         file_fail = 0
         for filename in files:
             file_path = os.path.join(root, filename)
             ext = os.path.splitext(file_path)[-1].lower()
             if not ext.endswith(suffixes):
                 continue