This is an archive of the discontinued LLVM Phabricator instance.

[automation] Add scripts to automate GitHub projects
Needs RevisionPublic

Authored by thieta on Sep 8 2022, 12:25 AM.

Details

Reviewers
tstellar
kwk
Summary

Two new scripts that will later be called from github-actions:

  • github-project.py is using GitHub GraphQL API to query and change the github projects board we use. This can show all the issues in the project and add new issues to the project.
  • sync-release-repo.sh is a bash script that syncs llvm/llvm-project and llvm/llvm-project-release-prs and optionally merges the PR's that are marked as "Needs Merge".

Diff Detail

Event Timeline

thieta created this revision.Sep 8 2022, 12:25 AM
Herald added a project: Restricted Project. · View Herald TranscriptSep 8 2022, 12:25 AM
thieta requested review of this revision.Sep 8 2022, 12:25 AM
Herald added a project: Restricted Project. · View Herald TranscriptSep 8 2022, 12:25 AM
thieta updated this revision to Diff 458670.Sep 8 2022, 12:28 AM

Removed the auto pr merging - it was not ready

kwk requested changes to this revision.Sep 12 2022, 1:52 AM
kwk added a subscriber: serge-sans-paille.

@thieta as much as I like the effort you've made here. I'd like to propose a standalone GraphQL library. You're adding lots of code that is hard to trust without tests. That is why I started to create a library with tests and a release process to pypi. I think much of the things you do here are so generic that we can integrate them into the library at will.

https://github.com/kwk/ghgql

So far I will try to see if we can add the queries you've written nicely into "my" library.

llvm/utils/git/github-project.py
14–36

I would love to see those queries in their own files. Then we can call them manually and verify them using tools etc. But I see that this query itself comes from here: https://docs.github.com/en/issues/planning-and-tracking-with-projects/automating-your-project/automating-projects-using-actions#example-workflow-authenticating-with-a-github-app, right?

78
154

This function is deprecated but still works and requires minimal changes. I'm doing from requests import Session and then self.__session = Session(). For example. BUT: I found out when running my tests, that the session object is never closed. That's why I've implemented my GithubGraphQL class as a context manger so that you can write:

with ghgql.GithubGraphQL() as ghapi:
    ghapi.query_from_file("foo.gql")

and the session stored in ghapi will be closed.

157–159

@thieta please take a look at the function run_graphql_query and the comments for it here: https://reviews.llvm.org/D119412. We've had some discussions on GraphQL already. And I believe we agreed to outsource the queries in the end. At least we discussed to have them clean and use the variables array which you've type here quite nicely.

Anyway I would love to see a sharing of code when it comes to GraphQL. Especially when it comes to making proper calls using an API in Python.

You're potentially raising an HttpError when calling raise_for_status. Is that on purpose or do you want to handle errors locally here?

When you run the PROJECT_FIELDS query in the GraphQL explorer (https://docs.github.com/en/graphql/overview/explorer) you're being greeted with this error:

Although you appear to have the correct authorization credentials, the `llvm` organization has enabled OAuth App access restrictions, meaning that data access to third-parties is limited. For more information on these restrictions, including how to enable this app, visit https://docs.github.com/articles/restricting-access-to-your-organization-s-data/

That is why I believe you should specify a token when making a request to the GraphQL API.

EDIT: I saw that you're setting the bearer token to the global session object directly when parsing the arguments. In the spirit of sharing code for GraphQL usage, can we please just make a class with some nice defaults? Here's an idea for such a class. Then we can get rid of global objects and alike: https://github.com/kwk/ghgql.

I think this incorporates all of the stuff from @tstellar and myself as well as your code and additional tests. I've decided to always return a ghgql.Result even in case of errors. See the tests/ folder for usage information. This way we don't have to mess with exceptions so much and you can call the method on the returned object. All that ghgql.Result does is to wrap dict. What do you think? Can we get this in as a separate patch maybe? I mean not the python package but maybe it makes sense to outsource it like I did and only keep the code in the llvm tree. @serge-sans-paille you do a lot for python-lit if I'm not mistaken. Maybe you can tell us more about the pros and cons.

162–170

I've designed the ghgql.Result class to only raise a RuntimeError if a user tries to access and element from a result but there are errors present (see https://github.com/kwk/ghgql/blob/main/src/ghgql/result.py#L36).

174–188

I like this idea. You can find this idea in my ghgql.Result class here: https://github.com/kwk/ghgql/blob/main/src/ghgql/result.py#L18

This revision now requires changes to proceed.Sep 12 2022, 1:52 AM
kwk added inline comments.Sep 12 2022, 1:55 AM
llvm/utils/git/github-project.py
201–204

According to https://github.blog/2021-11-16-graphql-global-id-migration-update/ you can pass an extra header 'X-Github-Next-Global-ID': '1' and you'll get the id automatically. See here for how I do it: https://github.com/kwk/ghgql/blob/main/src/ghgql/ghgql.py#L70 . I'm not sure if this helps the problem but I sense that we can get rid of some dance with the API.

kwk added inline comments.Sep 12 2022, 1:57 AM
llvm/utils/git/sync-release-repo.sh
12

You're not using SCRIPT_DIR anywhere in this script. Let's not copy snippets from other script that we don't use.

@thieta as much as I like the effort you've made here. I'd like to propose a standalone GraphQL library. You're adding lots of code that is hard to trust without tests. That is why I started to create a library with tests and a release process to pypi. I think much of the things you do here are so generic that we can integrate them into the library at will.

Hi,

No problem - I was actually not that keen on adding all that code, but the only python graphql library I found was graphene and it seemed to be insane overkill. I learned a lot about GraphQL in the process so I don't mind moving it to another library.

https://github.com/kwk/ghgql

So far I will try to see if we can add the queries you've written nicely into "my" library.

Yeah I think the only complicated part of my script was to figure out the GraphQL queries and syntax, the code to request graphql's are pretty straightforward.

The only thing I would make sure we avoid is to make this to-complicated, graphql already seems very over-engineered (to my eyes) and we just need such a small small subset that we shouldn't put to much effort into handling all kinds of queries and edge-cases.

Reading through your comments I think it's fair to say that we should try doing the same things with your library and see if that's working and if it's easier or more annoying. Is that something you have bandwidth for or should I try to use your library? I just want to solve the issue mentioned above so that there is less for me to do for each release, so implementation wise I am less bothered exactly how it's done.

llvm/utils/git/github-project.py
14–36

Yeah agreed.

I did most of the queries using the explorer tool by github.

kwk added a comment.Sep 27 2022, 1:35 AM

[...]
The only thing I would make sure we avoid is to make this to-complicated, graphql already seems very over-engineered (to my eyes) and we just need such a small small subset that we shouldn't put to much effort into handling all kinds of queries and edge-cases.

I agree. That's why I in ghgql I focused on the simplicity of making a and accessing the result. In general I find it weird to handle exceptions in that case which is why I've *deferred* them from the call to GraphQL to the inspection of results. That said, an {"errors": ...}" result doesn't raise an exception but returns a regular Result` object that you can ask for errors or simply just access the thing you're looking for. If you call Result.get("foo.bar") but an "errors" is present, an exception is raised. I had the feeling that this makes more sense.

If possible I'd like to keep ghgql well tested so you can rely on it. It is very very slim and most of the code is for doing the releases. For me this was an experiment to try out python packaging and it works quite nice now. On each successful push to master and tests, a new version is released.

thieta added a comment.Oct 2 2022, 1:02 AM

I agree. That's why I in ghgql I focused on the simplicity of making a and accessing the result. In general I find it weird to handle exceptions in that case which is why I've *deferred* them from the call to GraphQL to the inspection of results. That said, an {"errors": ...}" result doesn't raise an exception but returns a regular Result` object that you can ask for errors or simply just access the thing you're looking for. If you call Result.get("foo.bar") but an "errors" is present, an exception is raised. I had the feeling that this makes more sense.

If possible I'd like to keep ghgql well tested so you can rely on it. It is very very slim and most of the code is for doing the releases. For me this was an experiment to try out python packaging and it works quite nice now. On each successful push to master and tests, a new version is released.

I tried to port my changes over to ghgql - and it seems to work mostly. I filed two issues in the ghgql repo where this one: https://github.com/kwk/ghgql/issues/5 is currently the biggest blocker.

kwk added a comment.EditedOct 4 2022, 5:52 AM

[...]
I tried to port my changes over to ghgql - and it seems to work mostly. I filed two issues in the ghgql repo where this one: https://github.com/kwk/ghgql/issues/5 is currently the biggest blocker.

I have a potential and very simple to solution to the problem in a PR here: https://github.com/kwk/ghgql/pull/7 . Can you look at this please?