collect_and_build_with_pgo.py: adapt to monorepo

[kristof.beyls]

Summary

This adapts the collect_and_build_with_pgo.py script so that it works with monorepo repositories, i.e. the default after the github move.

[george.burgess.iv]

thanks for this!

i think the preferred way to PGO LLVM is cmake: https://llvm.org/docs/AdvancedBuilds.html

that said, while we have this script & the docs pointing to it, seems good to keep it up-to-date. LGTM

[kristof.beyls]

In D92328#2423327, @george.burgess.iv wrote:

thanks for this!

i think the preferred way to PGO LLVM is cmake: https://llvm.org/docs/AdvancedBuilds.html

that said, while we have this script & the docs pointing to it, seems good to keep it up-to-date. LGTM

Thanks for the quick response!
I had missed https://llvm.org/docs/AdvancedBuilds.html, so I just tried the instructions there to do a PGO build, but it seems it does not work (at least on ToT) currently.
ninja told me it doesn't know target stage2-instrumented-generate-profdata

llvm/utils/collect_and_build_with_pgo.py

def get_make_command(self):
             return ['make', '-j{}'.format(multiprocessing.cpu_count())]
         return ['ninja']
 
     def output_subdir(self, name):
         return os.path.join(self.output_dir, name)
 
     def has_llvm_subproject(self, name):
         if name == 'compiler-rt':
-            subdir = 'projects/compiler-rt'
+            subdir = '../compiler-rt'
         elif name == 'clang':
-            subdir = 'tools/clang'
+            subdir = '../clang'
         else:
             raise ValueError('Unknown subproject: %s' % name)
 
         return os.path.isdir(os.path.join(self.llvm_dir, subdir))
 
     # Note that we don't allow capturing stdout/stderr. This works quite nicely
     # with dry_run.
     def run_command(self,
                     cmd,
                     cwd=None,
                     check=False,
                     silent_unless_error=False):
-        cmd_str = ' '.join(shlex.quote(s) for s in cmd)
         print(
-            'Running `%s` in %s' % (cmd_str, shlex.quote(cwd or os.getcwd())))
+            'Running `%s` in %s' % (cmd, shlex.quote(cwd or os.getcwd())))
 
         if self.dry_run:
             return
 
         if silent_unless_error:
             stdout, stderr = subprocess.PIPE, subprocess.STDOUT
         else:
             stdout, stderr = None, None
def _parse_args():
     args = parser.parse_args()
 
     llvm_dir = os.path.abspath(args.llvm_dir)
     if args.out_dir is None:
         output_dir = os.path.join(llvm_dir, 'out')
     else:
         output_dir = os.path.abspath(args.out_dir)
 
-    extra_args = {'CMAKE_BUILD_TYPE': 'Release'}
+    extra_args = {'CMAKE_BUILD_TYPE': 'Release',
+                  'LLVM_ENABLE_PROJECTS': 'clang;compiler-rt;lld'}
     for arg in args.cmake_extra_arg:
         if arg.startswith('-D'):
             arg = arg[2:]
         elif arg.startswith('-'):
             raise ValueError('Unknown not- -D arg encountered; you may need '
                              'to tweak the source...')
         split = arg.split('=', 1)
         if len(split) == 1: