Make llvm-git (monorepo helper) compatible with both Python 2 and 3

[t.p.northover]

Summary

It seems that the subprocess functions return bytes rather than str for the stdout on Python 3. When this script then tries to combine them with normal strings, an exception is raised.

It seems like we either want to use bytes everywhere, or assume utf-8 encoding. The latter is a much smaller change, and probably reasonable since both git internals and (I assume) our repository text is assumed utf-8. Actual binary data seems to get base64 encoded so I don't think coincidentally-invalid utf-8 is an issue there.

Anyway, with these changes I can commit using both python3.5 and python2.7.

[zturner]

llvm/utils/git-svn/git-llvm

99

This is a bit wrong since it will try to print call str(b'...') behind the scenes. So if the output is "Hello, world" we'll see b'Hello world` printed.

Can you add a decode here?

100

I'd add the decode above this line, that way you can strip it without the b syntax.

106

Probably will need another decode here. Alternatively, you could try hiding this inside of the eprint function so users don't have to remember to do it.

[t.p.northover]

Thanks for the suggestions. I think I've incorporated all of them, and the diff seems much cleaner.

llvm/utils/git-svn/git-llvm

def shell(cmd, strip=True, cwd=None, stdin=None, die_on_failure=True):
     stdout, stderr = p.communicate(input=stdin)
     elapsed = time.time() - start
 
     log_verbose('Command took %0.1fs' % elapsed)
 
     if p.returncode == 0:
         if stderr:
             eprint('`%s` printed to stderr:' % ' '.join(cmd))
             eprint(stderr.rstrip())
         if strip:
-            stdout = stdout.rstrip('\r\n')
-        return stdout
+            stdout = stdout.rstrip(b'\r\n')
+        return stdout.decode('utf-8')
     err_msg = '`%s` returned %s' % (' '.join(cmd), p.returncode)
     eprint(err_msg)
     if stderr:
         eprint(stderr.rstrip())
     if die_on_failure:
         sys.exit(2)
     raise RuntimeError(err_msg)
 
 
 def git(*cmd, **kwargs):
     return shell(['git'] + list(cmd), kwargs.get('strip', True))
 
 
 def svn(cwd, *cmd, **kwargs):
     # TODO: Better way to do default arg when we have *cmd?
     return shell(['svn'] + list(cmd), cwd=cwd, stdin=kwargs.get('stdin', None))
 
-
 def get_default_rev_range():
     # Get the branch tracked by the current branch, as set by
     # git branch --set-upstream-to  See http://serverfault.com/a/352236/38694.
     cur_branch = git('rev-parse', '--symbolic-full-name', 'HEAD')
     upstream_branch = git('for-each-ref', '--format=%(upstream:short)',
                           cur_branch)
     if not upstream_branch:
         upstream_branch = 'origin/master'
if status:
             (rev, status))
 
     for sr in subrepos:
         diff = git('show', '--binary', rev, '--', sr, strip=False)
         svn_sr_path = os.path.join(svn_repo, GIT_TO_SVN_DIR[sr])
         # git is the only thing that can handle its own patches...
         log_verbose('Apply patch: %s' % diff)
         try:
-            shell(['git', 'apply', '-p2', '-'], cwd=svn_sr_path, stdin=diff,
-                  die_on_failure=False)
+            shell(['git', 'apply', '-p2', '-'], cwd=svn_sr_path,
+                   stdin=diff.encode('utf-8'), die_on_failure=False)
         except RuntimeError as e:
             eprint("Patch doesn't apply: maybe you should try `git pull -r` "
                    "first?")
             sys.exit(2)
 
     status_lines = svn(svn_repo, 'status').split('\n')
 
     for l in (l for l in status_lines if l.startswith('?')):