Page MenuHomePhabricator

[LLVM][TableGen] Notebook describing how to write a Python backend
ClosedPublic

Authored by DavidSpickett on Mon, Jan 23, 7:14 AM.

Details

Summary

This tutorial uses the dump json option to write a backend for
SQL queries.

It is based on the work of Min-Yih Hsu:

I hope that having the same concepts in 3 forms will allow people
to choose the style that fits them.

The main drawback here being that it's in Python. C++ can be used
in a notebook (https://github.com/jupyter-xeus/xeus-cling) but I
decided against it for a few reasons:

  • Python is the default for Jupyter, no extra installs needed.
  • Having the code in a second language may help people who know one or the other.
  • There is no upstream example of a JSON powered backend. (and although we would be unlikely to accept one upstream, I think it's a great option for quick development before translating into C++)

Diff Detail

Event Timeline

DavidSpickett created this revision.Mon, Jan 23, 7:14 AM
Herald added a project: Restricted Project. · View Herald TranscriptMon, Jan 23, 7:14 AM
DavidSpickett requested review of this revision.Mon, Jan 23, 7:14 AM
Herald added a project: Restricted Project. · View Herald TranscriptMon, Jan 23, 7:14 AM
DavidSpickett added a subscriber: myhsu.

I should note that this does not require the tablegen kernel, it's the default iPython kernel. You will need to tell it how to find llvm-tblgen though.

@myhsu This is what we talked about over email last year.

There is a licensing issue I need you to clarify if you are ok with. Your C++ work is Apache 2.0 and this would be a derivative work of that, licensed the same. Which is a bit awkward to do here, perhaps it would be enough to include a copy of the license at the start of the tutorial, I'd have to check with an expert.

Alternatively, if you are the sole author (and therefore copyright holder) of SQLGen (which is my understanding) you could give me permission to contribute this to llvm. Where it would be under Apache 2.0 with llvm exceptions and the legacy license (as described here https://llvm.org/docs/DeveloperPolicy.html#relicensing).

That probably sounds like a small concern, but I want to do the right thing here. If you have any reservations about that, I will make my own example backend and base the tutorial on that instead, no problem.

llvm/utils/TableGen/jupyter/sql_query_backend.ipynb
100

Note that in Jupyter, this giant output collapses into a smaller panel with scroll bars. So folks can come back to it if they want detail, if not just carry on reading past it.

myhsu added a comment.Tue, Jan 24, 1:13 PM

I should note that this does not require the tablegen kernel, it's the default iPython kernel. You will need to tell it how to find llvm-tblgen though.

@myhsu This is what we talked about over email last year.

There is a licensing issue I need you to clarify if you are ok with. Your C++ work is Apache 2.0 and this would be a derivative work of that, licensed the same. Which is a bit awkward to do here, perhaps it would be enough to include a copy of the license at the start of the tutorial, I'd have to check with an expert.

Interesting...I naively thought that since LLVM's license is a special version of Apache 2.0, it should be easy to authorize my work to LLVM.

Alternatively, if you are the sole author (and therefore copyright holder) of SQLGen (which is my understanding) you could give me permission to contribute this to llvm. Where it would be under Apache 2.0 with llvm exceptions and the legacy license (as described here https://llvm.org/docs/DeveloperPolicy.html#relicensing).

This path definitely sounds easier to me :-) And yes, I'm the sole author and copyright holder of SQLGen. How can I give such permission to you? Send an email?

Thanks for noticing this issue!

myhsu accepted this revision.Tue, Jan 24, 1:41 PM

LGTM, thank you for creating this tutorial! As I also mentioned in our previous email communication, I think this really helps the broader community including beginners to learn TableGen programming.

llvm/utils/TableGen/jupyter/sql_query_backend.ipynb
4

I think it will be helpful to show the workflow of normal TG backend (TG parser -> C++ TG backend) and the Python TG backend we're developing here (llvm-tblgen -> JSON -> Python backend) side-by-side using one or two sentences.

This revision is now accepted and ready to land.Tue, Jan 24, 1:41 PM

This path definitely sounds easier to me :-) And yes, I'm the sole author and copyright holder of SQLGen. How can I give such permission to you? Send an email?

Just needs to be some record of it, so your comments here count for that.

Add explanation of C++ vs. JSON/Python backend workflow.

DavidSpickett marked an inline comment as done.Wed, Jan 25, 3:55 AM

And FYI I tested SQLGen against 15.0.7 and it works out of the box.

myhsu accepted this revision.Wed, Jan 25, 9:24 AM

This path definitely sounds easier to me :-) And yes, I'm the sole author and copyright holder of SQLGen. How can I give such permission to you? Send an email?

Just needs to be some record of it, so your comments here count for that.

Great!
As the sole copyright holder of SQLGen, I give you the permission to contribute this work to LLVM.

Minor updates to wording. Tell people what to do if they didn't set up the
llvm-tblgen path prior to running the notebook.

This revision was landed with ongoing or failed builds.Thu, Jan 26, 4:30 AM
This revision was automatically updated to reflect the committed changes.