[LLVM][TableGen] Notebook describing how to write a Python backend

[DavidSpickett]

Summary

This tutorial uses the dump json option to write a backend for
SQL queries.

It is based on the work of Min-Yih Hsu:

• https://github.com/mshockwave/SQLGen
• https://www.youtube.com/watch?v=UP-LBRbvI_U

I hope that having the same concepts in 3 forms will allow people
to choose the style that fits them.

The main drawback here being that it's in Python. C++ can be used
in a notebook (https://github.com/jupyter-xeus/xeus-cling) but I
decided against it for a few reasons:

• Python is the default for Jupyter, no extra installs needed.
• Having the code in a second language may help people who know one or the other.
• There is no upstream example of a JSON powered backend. (and although we would be unlikely to accept one upstream, I think it's a great option for quick development before translating into C++)

[DavidSpickett]

I should note that this does not require the tablegen kernel, it's the default iPython kernel. You will need to tell it how to find llvm-tblgen though.

@myhsu This is what we talked about over email last year.

There is a licensing issue I need you to clarify if you are ok with. Your C++ work is Apache 2.0 and this would be a derivative work of that, licensed the same. Which is a bit awkward to do here, perhaps it would be enough to include a copy of the license at the start of the tutorial, I'd have to check with an expert.

Alternatively, if you are the sole author (and therefore copyright holder) of SQLGen (which is my understanding) you could give me permission to contribute this to llvm. Where it would be under Apache 2.0 with llvm exceptions and the legacy license (as described here https://llvm.org/docs/DeveloperPolicy.html#relicensing).

That probably sounds like a small concern, but I want to do the right thing here. If you have any reservations about that, I will make my own example backend and base the tutorial on that instead, no problem.

llvm/utils/TableGen/jupyter/sql_query_backend.ipynb

99

Note that in Jupyter, this giant output collapses into a smaller panel with scroll bars. So folks can come back to it if they want detail, if not just carry on reading past it.

[myhsu]

In D142364#4073657, @DavidSpickett wrote:

I should note that this does not require the tablegen kernel, it's the default iPython kernel. You will need to tell it how to find llvm-tblgen though.

@myhsu This is what we talked about over email last year.

There is a licensing issue I need you to clarify if you are ok with. Your C++ work is Apache 2.0 and this would be a derivative work of that, licensed the same. Which is a bit awkward to do here, perhaps it would be enough to include a copy of the license at the start of the tutorial, I'd have to check with an expert.

Interesting...I naively thought that since LLVM's license is a special version of Apache 2.0, it should be easy to authorize my work to LLVM.

Alternatively, if you are the sole author (and therefore copyright holder) of SQLGen (which is my understanding) you could give me permission to contribute this to llvm. Where it would be under Apache 2.0 with llvm exceptions and the legacy license (as described here https://llvm.org/docs/DeveloperPolicy.html#relicensing).

This path definitely sounds easier to me :-) And yes, I'm the sole author and copyright holder of SQLGen. How can I give such permission to you? Send an email?

Thanks for noticing this issue!

[myhsu]

LGTM, thank you for creating this tutorial! As I also mentioned in our previous email communication, I think this really helps the broader community including beginners to learn TableGen programming.

llvm/utils/TableGen/jupyter/sql_query_backend.ipynb

3

I think it will be helpful to show the workflow of normal TG backend (TG parser -> C++ TG backend) and the Python TG backend we're developing here (llvm-tblgen -> JSON -> Python backend) side-by-side using one or two sentences.

[DavidSpickett]

This path definitely sounds easier to me :-) And yes, I'm the sole author and copyright holder of SQLGen. How can I give such permission to you? Send an email?

Just needs to be some record of it, so your comments here count for that.

[DavidSpickett]

Add explanation of C++ vs. JSON/Python backend workflow.

[DavidSpickett]

And FYI I tested SQLGen against 15.0.7 and it works out of the box.

[myhsu]

In D142364#4079397, @DavidSpickett wrote:

This path definitely sounds easier to me :-) And yes, I'm the sole author and copyright holder of SQLGen. How can I give such permission to you? Send an email?

Just needs to be some record of it, so your comments here count for that.

Great!
As the sole copyright holder of SQLGen, I give you the permission to contribute this work to LLVM.

[DavidSpickett]

Minor updates to wording. Tell people what to do if they didn't set up the
llvm-tblgen path prior to running the notebook.