This is an archive of the discontinued LLVM Phabricator instance.

[LLVM][TableGen] Add jupyter kernel for llvm-tblgen
ClosedPublic

Authored by DavidSpickett on Aug 22 2022, 7:55 AM.

Details

Summary

This is based on the MLIR opt kernel:
https://github.com/llvm/llvm-project/tree/main/mlir/utils/jupyter

The inent of this is to enable experimentation and the creation
of interactive tutorials for the basics of tablegen.

Noteable changes from that:

  • Removed the codemirror mode settings since those won't exist for tablegen.
  • Added "%args" "magic" to control arguments sent to llvm-tblgen.

(magics are directives, see
https://ipython.readthedocs.io/en/stable/interactive/magics.html)

For example the following:

%args --print-detailed-records
class Stuff {}

def water_bottle : Stuff {}

Produces:

DETAILED RECORDS for file -

-------------------- Global Variables (0) --------------------

-------------------- Classes (1) --------------------

Stuff  |<stdin>:1|
  Template args: (none)
  Superclasses: (none)
  Fields: (none)

-------------------- Records (1) --------------------

water_bottle  |<stdin>:3|
  Superclasses: Stuff
  Fields: (none)

Diff Detail

Event Timeline

DavidSpickett created this revision.Aug 22 2022, 7:55 AM
Herald added a project: Restricted Project. · View Herald TranscriptAug 22 2022, 7:55 AM
DavidSpickett requested review of this revision.Aug 22 2022, 7:55 AM

First obvious thing, I just chose the closest folder to where MLIR put their kernel. It can go elsewhere of course.

DavidSpickett added a comment.EditedAug 22 2022, 8:11 AM

I'd like to put a notebook next to this that walks through https://llvm.org/docs/TableGen/ProgRef.html with examples but I figure the kernel has some use on its own too.

DavidSpickett added a subscriber: jpienaar.

@jpienaar If you have any feedback since you wrote the kernel for MLIR (and thanks for doing that!).

Correct module name in install command.

  • Added doctests.
  • Added example/testing notebook.
  • Added markdown output for that notebook.

Probably easier to view the markdown at https://github.com/DavidSpickett/llvm-project/blob/5eb2366310b125fca846c9ce339d2d7386c2a434/llvm/utils/TableGen/jupyter/LLVM%20TableGen.md.

Nice, I've found a variant where one just uses Python magic to do most of the same. The pro is not requireing any installation of jupyter kernel, the con is a little bit less output flexibility and error reporting I felt wasn't as nice.

llvm/utils/TableGen/jupyter/tablegen_kernel/kernel.py
82

Can one interleave magic like this in jupyter? I thought one had cell and line magic but both required to be only at start. (Well required is strong word, expected perhaps better)

jpienaar accepted this revision.Aug 24 2022, 12:59 PM
This revision is now accepted and ready to land.Aug 24 2022, 12:59 PM

Nice, I've found a variant where one just uses Python magic to do most of the same.

Another notebook do you mean? Can you post a link to it?

One idea I had was to be able to run python scripts on the JSON tablegen output, so I might end up coming back around to Python.

llvm/utils/TableGen/jupyter/tablegen_kernel/kernel.py
82

Well you already know more than me then :) I did this based on skimming the iPython kernel's docs so I'm still learning the terminology.

So far I guess this would be called "cell magic" and you're right it doesn't make sense for that to be anywhere but the first (or first set of) lines. Since you can't apply cell magic half way through a cell.

I will make this more strict.

Restrict magic that effects the whole cell to only be in the first lines,
or a continuous set of magic lines starting from the first line.

Interepreted as two magic lines:

%args
%args
// Code

Only one because the run of magic was broken:

%args
// Code
%args
DavidSpickett marked an inline comment as done.Aug 25 2022, 1:46 AM

Adding Paul as the current TableGen code owner.

Thanks for doing this!

I've only had the time go over and to run "LLVM TableGen.md" - everything worked fine and the examples make sense to me. If only there was no empty space in the filename ;-) Btw, why do you need an *.md and *.ipynb file?

llvm/utils/TableGen/jupyter/README.md
4

?

32
llvm/utils/TableGen/jupyter/tablegen_kernel/__init__.py
5

I think that I am right, but I might be wrong ;-) (https://www.pristineword.com/grammar-an-l/)

  • Address review comments.
  • Explain what the jupyter console line does.
DavidSpickett marked 3 inline comments as done.Sep 8 2022, 1:57 AM

Btw, why do you need an *.md and *.ipynb file?

The ipynb is the notebook itself which contains the cells and copies of the last output from those cells. This is the one you can re-run and edit in jupyter.

The markdown is a static export of the resulting notebook so you can see what the kernel can do without running it. Also though Phabricator does have support for diffing ipynb, github doesn't and markdown makes it easier to tell what changed without running it locally.

  • Note that there is a markdown file for anyone who just wants to see the results.
awarzynski accepted this revision.Sep 12 2022, 3:04 AM

LGTM, thank you for doing this!

Nice, I've found a variant where one just uses Python magic to do most of the same.

Another notebook do you mean? Can you post a link to it?

Sorry for delay, waa OOO. I'll need to clean it up (well sanitize). I hit a restriction where i couldn't use a custom runtime and then made it as a quick workaround. I'll try and do that shortly (will try today but depends on amount of email I have to sift through :-)). But don't take that as a blocker from me here. (You could always use cell magic to refer to previous cells and so combine this kernel with that).

6yearold added inline comments.
llvm/utils/TableGen/jupyter/tablegen_kernel/assets/kernel.json
10

Is tablgen a typo? I don't see any TableGen modes at https://codemirror.net/5/mode/

DavidSpickett added inline comments.Sep 28 2022, 3:19 AM
llvm/utils/TableGen/jupyter/tablegen_kernel/assets/kernel.json
10

Yes it is. The binary is llvm-tblgen but the language is TableGen, it's endlessly confusing.

And yes there's no style for it in codemirror at this time. I think there is some way to add new languages but I haven't got into it myself.

Correct typo.

This revision was landed with ongoing or failed builds.Jan 23 2023, 6:16 AM
This revision was automatically updated to reflect the committed changes.