github: Add actions to automate part of the release workflow

[tstellar]

Summary

This adds support for automatically cherry-picking and testing fixes for the
release branch using 'commands' in issue comments. The two supported commands are:

/cherry-pick <commit1> <commit2> ...

Which will backport and test commits from main. And also

/branch owner/repo/branch

Which will test commits from the given branch.

[alexbatashev]

.github/workflows/issue-release-workflow.yml

14

Something like this should avoid interference with LLVM forks and their automation

43

Same as above

[tstellar]

Restrict the action to just the llvm/llvm-project repo.

[alexbatashev]

LGTM now, thanks @tstellar

.github/workflows/issue-release-workflow.yml

43

Typo

[tstellar]

Fix typo.

[kwk]

Again I feel picky and sorry. Due to documentation issues among other issues I have to reject this for the time being. There is literally no comment whatsoever. This would tremendously speed up the review speed because then you can more easily review what's a function is supposed to do and what it actually does.

.github/workflows/issue-release-workflow.yml

14

If this gets too long you might as well do this:

if: >-
    github.repository == 'llvm/llvm-project' &&
    startswith(github.event.comment.body, '/cherry-pick')

Your solution is not wrong.

25

I highly suggest to use a requirements.txt file instead. I suggest you create a requirements.txt.in with this content:

# Convert this file into a requirements.txt file by running:
#
# pip install pip-tools
# pip-compile -o requirements.txt  requirements.txt.in

PyGithub==1.55
GitPython==3.1.26

Then you convert this file locally into a requirements.txt file by running:

pip install pip-tools
pip-compile -o requirements.txt  requirements.txt.in

Find out more about requirements.txt here: https://pip.pypa.io/en/stable/reference/pip_install/#requirements-file-format

One of the added benefits is that you can more easily see when your code is affected by any CVEs and you can maintain direct and indirect dependencies separately. Also local execution of any python code can happen without looking at this nasty github workflow file.

Of course, version both files, requirements.txt.in and requirements.txt.

In this step things would then looks like this:

- pip install PyGithub GitPython
+ pip install -r requirements.txt

43

Same applies here for using multiline ifs if you want.

48

Why curl the file instead of cloning like before? If github-automation.py happens to need another file from the repo you'd have to remember to patch this place.

50

Another benefit of requirements.txt is that you have to maintain dependencies in one place only.

llvm/utils/git/github-automation.py

40

Please have a documentation for these functions.

47

Please describe what this class is supposed to do and use typed-python for return types and parameters. It tremendously will help newcomers and maintainers .

49

I suggest to keep this "private": self._args = args

59

I suggest to make this a property and the rest of the getters that do simple stuff as well:

@property
def issue_number(self) -> int:
    return self._args.issue_number

264

Please use the help='...' parameter to describe what each parameter is doing.

[tstellar]

Thanks for the feedback. I will work on these suggestions.

.github/workflows/issue-release-workflow.yml

25

How important is it to have the version numbers in requirements.txt ? I'm concerned that the versions available in the github actions runner will change often and that will break this workflow if we have explicit versions.

48

I mainly did this because cloning is so slow. I agree cloning would be better. I will try to benchmark with a shallow clone to see how long it takes.

50

That would be nice. I considered putting this step into a reusable workflow, so it could be more easily shared, but I wasn't sure it was worth it for something so small.

[tstellar]

Add some comments and a few other minor fixes

[tstellar]

.github/workflows/issue-release-workflow.yml

48

I dropped this, the shallow checkout seems to be fast enough.

llvm/utils/git/github-automation.py

49

I removed self.args completely.

[kwk]

Thank you @tstellar for accepting so many of my suggestions. I have a few more and then I will give it a full review again.

.github/workflows/issue-release-workflow.yml

25

The version is at least super important and helpful for local development and testing.

I wouldn't bother about the runner because pip is used to install these versions and the actions runner shouldn't have an impact on it. For example, this page lists the version of the library and one should be able to install any of them: https://pypi.org/project/PyGithub/#history .

Here's an example:

kkleine@work:~$ python3 -m venv $PWD/myenv
kkleine@work:~$ cd myenv/
kkleine@work:~/myenv$ source bin/activate
kkleine@work:~/myenv$ pip install PyGithub==1.45
Collecting PyGithub==1.45
  Downloading PyGithub-1.45.tar.gz (113 kB)
...
    Running setup.py install for PyGithub ... done
Successfully installed PyGithub-1.45 certifi-2021.10.8 charset-normalizer-2.0.10 deprecated-1.2.13 idna-3.3

If you want to be super careful, you might employ the idea of the virtual environment to not mess with system installed versions. In fact, I'd really do that! Afterall, these are all the steps needed in the step:

python3 -m venv $PWD/myenv
source myenv/source bin/activate

The only possible downside is that you'd have to source the activate script in all steps.

48

Thank you.

50

That would be nice. I considered putting this step into a reusable workflow, so it could be more easily shared, but I wasn't sure it was worth it for something so small.

I don't think you need a sharable workflow for this. Afterall it's just pip install -r requirements.txt. The point is, that the requirements file bundles everything.

Not sure if I mentioned this, but the requirements.txt file will be beneficial for local testing and if you have the versions in the file, you can be more certain, that github will do the same.

llvm/utils/git/github-automation.py

147

I suggest to change the interface from

def create_branch(self, commits:list):

to

def create_branch(self, commits:list[str]) -> bool:

It is possible to have types for nested elements, so here: list[str] but it seems like it is ignored when executed:

>>> foo([1,2,3])
1
2
3
>>> foo(["1", "2", "3"])
1
2
3

An editor on the other hand nicely presents the correct format to pass into this function:

149–150

I wonder what this sentence is supposed to mean? How is a branch associated with an issue number? By a comment? The description of this change doesn't indicate how this is done.

211–212

Why not return self.create_branch(args.split())?

219–220

Same here, why not return self.create_pull_request(owner, branch)?

[tstellar]

Address more review comments.

[tstellar]

.github/workflows/issue-release-workflow.yml

25

OK, thanks for the explanation. I've added the requirements.txt file.

llvm/utils/git/github-automation.py

147

I tried this, but python gave an error saying this wasn't supported.

[tstellar]

• Added from typing import * for List[Str]
• Reworded some of the error messages.
• Allow commands on lines besides the first.

[kwk]

• More typed python and properties

[kwk]

Looks good overall. Thank for your patience. Please make sure it still works after my patch ;/

.github/workflows/issue-release-workflow.yml

2

Please add a description in YAML comment here so one gets an immediate idea for whom this script is and what it does.

13

Good that you've fixed the OS version.

17

I assume that checking for the /cherry-pick anywhere in the comment body is part of the Allow commands on lines besides the first., right?

32

I never checked this but are you sure the comment body is properly escaped if it contains an apostrophe '? Does it make sense to do parse it here using something like the following? The benefit would be that you can easily test the extraction from the body with tests and that you could make a clean call to the final script.

printf '${{ github.event.comment.body }}' | ./llvm/utils/git/parse-comment-body.py --get-cherry-pick-commit

[tstellar]

• Add comments
• Use environment variables to properly escape comments

[tstellar]

.github/workflows/issue-release-workflow.yml

17

Yes, that's correct.

32

Thanks for catching this. I changed this to use environment variables, which is recommended by the GitHub security documentation. I also test that ' and " are properly escaped.

[kwk]

Minor things in the documentation but I'd say: go for it.

.github/workflows/issue-release-workflow.yml

7–8

COMMIT is <commit>, right? And it probably more than one commit that can be cherry-picked with this command, right? I mean I read the code, so yes.

As a new reader I would ask what the release branch is.

10–12

What is BRANCH in this case? I assume you mean <owner>/<repo>/<branch>?

[ldionne]

I just read the commit summary but this seems very exciting. I'm glad that folks are starting to use Actions, that will open the door to a lot of goodness elsewhere in LLVM too.

.github/workflows/issue-release-workflow.yml

+# This contains the workflow definitions that allow users to test backports
+# to the release branch using comments on issues.
+#
+# /cherry-pick <commit> <...>
+#
+# This comment will attempt to cherry-pick the given commits to the latest
+# release branch (release/Y.x) and if successful push the result to a branch
+# on github.
+#
+# /branch <owner>/<repo>/<branch>
+#
+# This comment will create a pull request from <branch> to the latest release
+# branch.
+
+name: Issue Release Workflow
+
+on:
+  issue_comment:
+    types:
+      - created
+      - edited
+
+env:
+  COMMENT_BODY: ${{ github.event.comment.body }}
+
+jobs:
+  backport-commits:
+    name: Backport Commits
+    runs-on: ubuntu-20.04
+    if: >-
+        (github.repository == 'llvm/llvm-project') &&
+        !startswith(github.event.comment.body, '<!--IGNORE-->') &&
+        contains(github.event.comment.body, '/cherry-pick')
+    steps:
+      - name: Fetch LLVM sources
+        uses: actions/checkout@v2
+        with:
+          repository: llvm/llvm-project
+          fetch-depth: 0
+
+      - name: Setup Environment
+        run: |
+          pip install -r ./llvm/utils/git/requirements.txt
+          ./llvm/utils/git/github-automation.py --token ${{ github.token }} setup-llvmbot-git
+
+      - name: Backport Commits
+        run: |
+          printf "$COMMENT_BODY" |
+          ./llvm/utils/git/github-automation.py \
+          --repo $GITHUB_REPOSITORY \
+          --token ${{ secrets.RELEASE_WORKFLOW_PUSH_SECRET }} \
+          release-workflow \
+          --issue-number ${{ github.event.issue.number }} \
+          auto
+
+  create-pull-request:
+    name: Create Pull Request
+    runs-on: ubuntu-20.04
+    if: >-
+        (github.repository == 'llvm/llvm-project') &&
+        !startswith(github.event.comment.body, '<!--IGNORE-->') &&
+        contains(github.event.comment.body, '/branch')
+
+    steps:
+      - name: Fetch LLVM sources
+        uses: actions/checkout@v2
+
+      - name: Setup Environment
+        run:  |
+          pip install -r ./llvm/utils/git/requirements.txt
+
+      - name: Create Pull Request
+        run: |
+          printf "$COMMENT_BODY" |
+          ./llvm/utils/git/github-automation.py \
+          --repo $GITHUB_REPOSITORY \
+          --token ${{ secrets.RELEASE_WORKFLOW_PUSH_SECRET }} \
+          release-workflow \
+          --issue-number ${{ github.event.issue.number }} \
+          auto

llvm/utils/git/github-automation.py

 #!/usr/bin/env python3
 #
 # ======- github-automation - LLVM GitHub Automation Routines--*- python -*--==#
 #
 # Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 # See https://llvm.org/LICENSE.txt for license information.
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 #
 # ==-------------------------------------------------------------------------==#
 
 import argparse
+from git import Repo # type: ignore
 import github
 import os
+import re
+import sys
+from typing import *
 
 class IssueSubscriber:
 
     @property
     def team_name(self) -> str:
         return self._team_name
 
     def __init__(self, token:str, repo:str, issue_number:int, label_name:str):
         self.repo = github.Github(token).get_repo(repo)
         self.org = github.Github(token).get_organization(self.repo.organization.login)
         self.issue = self.repo.get_issue(issue_number)
         self._team_name = 'issue-subscribers-{}'.format(label_name).lower()
 
     def run(self) -> bool:
         for team in self.org.get_teams():
             if self.team_name != team.name.lower():
                 continue
             comment = '@llvm/{}'.format(team.slug)
             self.issue.create_comment(comment)
             return True
         return False
 
+def setup_llvmbot_git(git_dir = '.'):
+    """
+    Configure the git repo in `git_dir` with the llvmbot account so
+    commits are attributed to llvmbot.
+    """
+    repo = Repo(git_dir)
+    with repo.config_writer() as config:
+        config.set_value('user', 'name', 'llvmbot')
+        config.set_value('user', 'email', '[e-mail]')
+
+class ReleaseWorkflow:
+
+    """
+    This class implements the sub-commands for the release-workflow command.
+    The current sub-commands are:
+        * create-branch
+        * create-pull-request
+
+    The execute_command method will automatically choose the correct sub-command
+    based on the text in stdin.
+    """
+
+    def __init__(self, token:str, repo:str, issue_number:int,
+                       branch_repo_name:str, branch_repo_token:str,
+                       llvm_project_dir:str) -> None:
+        self._token = token
+        self._repo_name = repo
+        self._issue_number = issue_number
+        self._branch_repo_name = branch_repo_name
+        if branch_repo_token:
+            self._branch_repo_token = branch_repo_token
+        else:
+            self._branch_repo_token = self.token
+        self._llvm_project_dir = llvm_project_dir
+
+    @property
+    def token(self) -> str:
+        return self._token
+
+    @property
+    def repo_name(self) -> str:
+        return self._repo_name
+
+    @property
+    def issue_number(self) -> int:
+        return self._issue_number
+
+    @property
+    def branch_repo_name(self) -> str:
+        return self._branch_repo_name
+
+    @property
+    def branch_repo_token(self) -> str:
+        return self._branch_repo_token
+
+    @property
+    def llvm_project_dir(self) -> str:
+        return self._llvm_project_dir
+
+    @property
+    def __repo(self) -> github.Repository.Repository:
+        return github.Github(self.token).get_repo(self.repo_name)
+
+    @property
+    def issue(self) -> github.Issue.Issue:
+        return self.__repo.get_issue(self.issue_number)
+
+    @property
+    def push_url(self) -> str:
+        return 'https://{}@github.com/{}'.format(self.branch_repo_token, self.branch_repo_name)
+
+    @property
+    def branch_name(self) -> str:
+        return 'issue{}'.format(self.issue_number)
+
+    @property
+    def release_branch_for_issue(self) -> Optional[str]:
+        issue = self.issue
+        milestone = issue.milestone
+        if milestone is None:
+            return None
+        m = re.search('branch: (.+)',milestone.description)
+        if m:
+            return m.group(1)
+        return None
+
+    def print_release_branch(self) -> None:
+        print(self.release_branch_for_issue)
+
+    def issue_notify_branch(self) -> None:
+        self.issue.create_comment('/branch {}/{}'.format(self.branch_repo_name, self.branch_name))
+
+    def issue_notify_pull_request(self, pull:github.PullRequest.PullRequest) -> None:
+        self.issue.create_comment('/pull-request {}#{}'.format(self.branch_repo_name, pull.number))
+
+    @property
+    def action_url(self) -> str:
+        if os.getenv('CI'):
+            return 'https://github.com/{}/actions/runs/{}'.format(os.getenv('GITHUB_REPOSITORY'), os.getenv('GITHUB_RUN_ID'))
+        return ""
+
+    def issue_notify_cherry_pick_failure(self, commit:str) -> github.IssueComment.IssueComment:
+        message = "<!--IGNORE-->\nFailed to cherry-pick: {}\n\n".format(commit)
+        action_url = self.action_url
+        if action_url:
+            message += action_url + "\n\n"
+        message += "Please manually backport the fix and push it to your github fork.  Once this is done, please add a comment like this:\n\n`/branch <user>/<repo>/<branch>`"
+        issue = self.issue
+        comment = issue.create_comment(message)
+        issue.add_to_labels('release:cherry-pick-failed')
+        return comment
+
+    def issue_notify_pull_request_failure(self, branch:str) -> github.IssueComment.IssueComment:
+        message = "Failed to create pull request for {} ".format(branch)
+        message += self.action_url
+        return self.issue.create_comment(message)
+
+
+    def create_branch(self, commits:List[str]) -> bool:
+        """
+        This function attempts to backport `commits` into the branch associated
+        with `self.issue_number`.
+
+        If this is successful, then the branch is pushed to `self.branch_repo_name`, if not,
+        a comment is added to the issue saying that the cherry-pick failed.
+
+        :param list commits: List of commits to cherry-pick.
+
+        """
+        print('cherry-picking', commits)
+        branch_name = self.branch_name
+        local_repo = Repo(self.llvm_project_dir)
+        local_repo.git.checkout(self.release_branch_for_issue)
+
+        for c in commits:
+            try:
+                local_repo.git.cherry_pick('-x', c)
+            except Exception as e:
+                self.issue_notify_cherry_pick_failure(c)
+                raise e
+
+        push_url = self.push_url
+        print('Pushing to {} {}'.format(push_url, branch_name))
+        local_repo.git.push(push_url, 'HEAD:{}'.format(branch_name))
+
+        self.issue_notify_branch()
+        return True
+
+
+    def create_pull_request(self, owner:str, branch:str) -> bool:
+        """
+        reate a pull request in `self.branch_repo_name`.  The base branch of the
+        pull request will be choosen based on the the milestone attached to
+        the issue represented by `self.issue_number`  For example if the milestone
+        is Release 13.0.1, then the base branch will be release/13.x. `branch`
+        will be used as the compare branch.
+        https://docs.github.com/en/get-started/quickstart/github-glossary#base-branch
+        https://docs.github.com/en/get-started/quickstart/github-glossary#compare-branch
+        """
+        repo = github.Github(self.token).get_repo(self.branch_repo_name)
+        issue_ref = '{}#{}'.format(self.repo_name, self.issue_number)
+        pull = None
+        release_branch_for_issue = self.release_branch_for_issue
+        if release_branch_for_issue is None:
+            return False
+        try:
+            pull = repo.create_pull(title='PR for {}'.format(issue_ref),
+                                    body='resolves {}'.format(issue_ref),
+                                    base=release_branch_for_issue,
+                                    head='{}:{}'.format(owner, branch),
+                                    maintainer_can_modify=False)
+        except Exception as e:
+            self.issue_notify_pull_request_failure(branch)
+            raise e
+
+        if pull is None:
+            return False
+
+        self.issue_notify_pull_request(pull)
+
+        # TODO(tstellar): Do you really want to always return True?
+        return True
+
+
+    def execute_command(self) -> bool:
+        """
+        This function reads lines from STDIN and executes the first command
+        that it finds.  The 2 supported commands are:
+        /cherry-pick commit0 <commit1> <commit2> <...>
+        /branch <owner>/<repo>/<branch>
+        """
+        for line in sys.stdin:
+            line.rstrip()
+            m = re.search("/([a-z-]+)\s(.+)", line)
+            if not m:
+                continue
+            command = m.group(1)
+            args = m.group(2)
+
+            if command == 'cherry-pick':
+                return self.create_branch(args.split())
+
+            if command == 'branch':
+                m = re.match('([^/]+)/([^/]+)/(.+)', args)
+                if m:
+                    owner = m.group(1)
+                    branch = m.group(3)
+                    return self.create_pull_request(owner, branch)
+
+        print("Do not understand input:")
+        print(sys.stdin.readlines())
+        return False
 
 parser = argparse.ArgumentParser()
-parser.add_argument('--token', type=str, required=True)
-parser.add_argument('--repo', type=str, default=os.getenv('GITHUB_REPOSITORY', 'llvm/llvm-project'))
+parser.add_argument('--token', type=str, required=True, help='GitHub authentiation token')
+parser.add_argument('--repo', type=str, default=os.getenv('GITHUB_REPOSITORY', 'llvm/llvm-project'),
+                    help='The GitHub repository that we are working with in the form of <owner>/<repo> (e.g. llvm/llvm-project)')
 subparsers = parser.add_subparsers(dest='command')
 
 issue_subscriber_parser = subparsers.add_parser('issue-subscriber')
 issue_subscriber_parser.add_argument('--label-name', type=str, required=True)
 issue_subscriber_parser.add_argument('--issue-number', type=int, required=True)
 
+release_workflow_parser = subparsers.add_parser('release-workflow')
+release_workflow_parser.add_argument('--llvm-project-dir', type=str, default='.', help='directory containing the llvm-project checout')
+release_workflow_parser.add_argument('--issue-number', type=int, required=True, help='The issue number to update')
+release_workflow_parser.add_argument('--branch-repo-token', type=str,
+                                     help='GitHub authentication token to use for the repository where new branches will be pushed. Defaults to TOKEN.')
+release_workflow_parser.add_argument('--branch-repo', type=str, default='llvmbot/llvm-project',
+                                     help='The name of the repo where new branches will be pushed (e.g. llvm/llvm-project)')
+release_workflow_parser.add_argument('sub_command', type=str, choices=['print-release-branch', 'auto'],
+                                     help='Print to stdout the name of the release branch ISSUE_NUMBER should be backported to')
+
+llvmbot_git_config_parser = subparsers.add_parser('setup-llvmbot-git', help='Set the default user and email for the git repo in LLVM_PROJECT_DIR to llvmbot')
+
 args = parser.parse_args()
 
 if args.command == 'issue-subscriber':
     issue_subscriber = IssueSubscriber(args.token, args.repo, args.issue_number, args.label_name)
     issue_subscriber.run()
+elif args.command == 'release-workflow':
+    release_workflow = ReleaseWorkflow(args.token, args.repo, args.issue_number,
+                                       args.branch_repo, args.branch_repo_token,
+                                       args.llvm_project_dir)
+    if args.sub_command == 'print-release-branch':
+        release_workflow.print_release_branch()
+    else:
+        if not release_workflow.execute_command():
+            sys.exit(1)
+elif args.command == 'setup-llvmbot-git':
+    setup_llvmbot_git()

llvm/utils/git/requirements.txt

+#
+# This file is autogenerated by pip-compile with python 3.8
+# To update, run:
+#
+#    pip-compile --output-file=requirements.txt requirements.txt.in
+#
+certifi==2021.10.8
+    # via requests
+cffi==1.15.0
+    # via pynacl
+charset-normalizer==2.0.10
+    # via requests
+deprecated==1.2.13
+    # via pygithub
+gitdb==4.0.9
+    # via gitpython
+gitpython==3.1.26
+    # via -r requirements.txt.in
+idna==3.3
+    # via requests
+pycparser==2.21
+    # via cffi
+pygithub==1.55
+    # via -r requirements.txt.in
+pyjwt==2.3.0
+    # via pygithub
+pynacl==1.5.0
+    # via pygithub
+requests==2.27.1
+    # via pygithub
+smmap==5.0.0
+    # via gitdb
+urllib3==1.26.8
+    # via requests
+wrapt==1.13.3
+    # via deprecated

llvm/utils/git/requirements.txt.in

+# Convert this file into a requirements.txt file by running:
+#
+# pip install pip-tools
+# pip-compile -o requirements.txt  requirements.txt.in
+
+PyGithub==1.55
+GitPython==3.1.26