This is an archive of the discontinued LLVM Phabricator instance.

[flang] Add new documentation main page
ClosedPublic

Authored by richard.barton.arm on Sep 7 2020, 8:51 AM.

Details

Summary

Add a new index page to be the Flang documentation mainpage instead of
Overview.md, which jumps straight into the compiler Design. The index file
needs to be in .rst format to use the toctree directive to create table of
contents.

Also use the sphinx_markdown_tables extension to generate html tables form
markdown.

A number of additional style changes to the existing docs were needed to make
this work well:

  • Convert all headings to the # style, which works better with toctree's titlesonly option. Ensure that there is only one top-level heading per document.
  • Add a title to documents that don't have one for rendering on the index.
  • Convert the grammar docs from .txt to .md. for better rendering
  • Fixed broken link to a section in another document - sphinx does not seem to support anchor links in markdown files.

Depends on D87226

Diff Detail

Event Timeline

richard.barton.arm requested review of this revision.Sep 7 2020, 8:51 AM

I have a GettingInvolved.md can you include it as well if it makes sense to add it with this patch and the original authors agree.
Most of the content comes from mailing list meeting minutes from Gary(Thanks @gak).

flang/docs/index.rst
1

Is there a hard requirement for using rst, can we instead use links from markdown syntax for generating TOC?
TOC generators could be used initially.

I have a GettingInvolved.md can you include it as well if it makes sense to add it with this patch and the original authors agree.
Most of the content comes from mailing list meeting minutes from Gary(Thanks @gak).

Nice addition! I suggest doing what you have done and making D87270 depend on this change and we submit them in that order.

Is there a hard requirement for using rst, can we instead use links from markdown syntax for generating TOC?
TOC generators could be used initially.

From what I have seen of your recent work @sameeranjoshi, I think it is likely that you know more about this than I do so I went and had a look :-)

The internet suggests the AutoStructify recommonmark extension is the one. I will take a look.

flang/docs/index.rst
1

Is there a hard requirement for using rst, can we instead use links from markdown syntax for generating TOC?
TOC generators could be used initially.

New version that uses recommonmark's AutoStructify plugin to re-write
the index page in markdown with toctrees in embedded rst blocks.

The web-pages seem to work properly for me with below configurations.

Name: sphinx-markdown-tables
Version: 0.0.15
Name: recommonmark
Version: 0.6.0

$ sphinx-build --version
sphinx-build 3.2.0

The TOC tree is displayed in the right indexsidebar, if it could be displayed at the top (see http://llvm.org/docs/CMake.html for reference) of individual pages that would be better and it would help reader initially to navigate to the internal section in the respective page.
Further I went on to apply D87226(contains a fixed side bar index links ) and the TOC disappears probably due to above mentioned issue.

The TOC tree is displayed in the right indexsidebar, if it could be displayed at the top (see http://llvm.org/docs/CMake.html for reference) of individual pages that would be better and it would help reader initially to navigate to the internal section in the respective page.
Further I went on to apply D87226(contains a fixed side bar index links ) and the TOC disappears probably due to above mentioned issue.

Can you help me understand what you mean here? When I build the pages they look like this to me:

Compare that to the reference page you linked me to (same browser, same computer):

I don't see a difference in the two for style. Both seem to have previous | next | index links in the top (and bottom) bars, as well as the indexsidebar as you point out. The indexsidebar seems to be a standard default from sphinx. LLVM uses a special one in llvm/docs/_templates/indexsidebar.html which does not have the previous/next topic links and instead has the permalinks to other articles.

Rebasing over the top of the license header restoration.

The TOC tree is displayed in the right indexsidebar, if it could be displayed at the top (see http://llvm.org/docs/CMake.html for reference) of individual pages that would be better and it would help reader initially to navigate to the internal section in the respective page.
Further I went on to apply D87226(contains a fixed side bar index links ) and the TOC disappears probably due to above mentioned issue.

Can you help me understand what you mean here? When I build the pages they look like this to me:

Compare that to the reference page you linked me to (same browser, same computer):

I don't see a difference in the two for style. Both seem to have previous | next | index links in the top (and bottom) bars, as well as the indexsidebar as you point out. The indexsidebar seems to be a standard default from sphinx. LLVM uses a special one in llvm/docs/_templates/indexsidebar.html which does not have the previous/next topic links and instead has the permalinks to other articles.

I was talking about having the Table Of Contents which is in right pane to be in the center of page the way LLVM has.
Also when D87226 would be applied, it would remove the TOC and will add other contents there.

I don't see a difference in the two for style. Both seem to have previous | next | index links in the top (and bottom) bars, as well as the indexsidebar as you point out. The indexsidebar seems to be a standard default from sphinx. LLVM uses a special one in llvm/docs/_templates/indexsidebar.html which does not have the previous/next topic links and instead has the permalinks to other articles.

I was talking about having the Table Of Contents which is in right pane to be in the center of page the way LLVM has.
Also when D87226 would be applied, it would remove the TOC and will add other contents there.

Ah, so a TOC in each page itself. Fair point.

Pushing a new version which adds a local toc to each document. I held off for a few of them as it did not seem to make sense to add one there (either doc too short, or just had one section.) The options comparison doc had a duplicat section title, so I fixed that up too.

Also fixed up some old-style title headings that slipped past the last rebase.

Thanks ! Builds and pages Look good.

@sameeranjoshi you happy to "accept this revision" :-) ?

sameeranjoshi accepted this revision.Sep 11 2020, 6:08 AM
This revision is now accepted and ready to land.Sep 11 2020, 6:08 AM
This revision was landed with ongoing or failed builds.Sep 11 2020, 6:22 AM
This revision was automatically updated to reflect the committed changes.
hans added a comment.Sep 15 2020, 11:09 AM

Sadly after this commit, I'm no longer able to build the Flang docs, my build failing like this:

cd /work/llvm.monorepo/build.docs/tools/flang/docs && /usr/bin/sphinx-build -b html -d /work/llvm.monorepo/build.docs/tools/flang/docs/_doctrees-flang-html -q -t builder-html /work/llvm.mo
norepo/flang/docs /work/llvm.monorepo/build.docs/tools/flang/docs/html                                                                                                                      
/usr/lib/python3/dist-packages/sphinx/util/compat.py:30: RemovedInSphinx30Warning: The config variable "source_parsers" is deprecated. Please update your extension for the parser and remov
e the setting.                                                                                                                                                                              
  warnings.warn('The config variable "source_parsers" is deprecated. '                                                                                                                      
/usr/lib/python3/dist-packages/sphinx/util/compat.py:36: RemovedInSphinx30Warning: app.add_source_parser() does not support suffix argument. Use app.add_source_suffix() instead.
  app.add_source_parser(suffix, parser)                                                                                                                                                     
/work/llvm.monorepo/flang/docs/ArrayComposition.md:1: WARNING: The "contents" directive may not be used within topics or body elements.                                 
/work/llvm.monorepo/flang/docs/BijectiveInternalNameUniquing.md:1: WARNING: The "contents" directive may not be used within topics or body elements.           
/work/llvm.monorepo/flang/docs/C++17.md:1: WARNING: The "contents" directive may not be used within topics or body elements.                                              
/work/llvm.monorepo/flang/docs/C++style.md:1: WARNING: The "contents" directive may not be used within topics or body elements.                                            
/work/llvm.monorepo/flang/docs/Calls.md:1: WARNING: The "contents" directive may not be used within topics or body elements.                                         
/work/llvm.monorepo/flang/docs/Character.md:1: WARNING: The "contents" directive may not be used within topics or body elements.                              
/work/llvm.monorepo/flang/docs/ControlFlowGraph.md:1: WARNING: The "contents" directive may not be used within topics or body elements.                       
/work/llvm.monorepo/flang/docs/Extensions.md:1: WARNING: The "contents" directive may not be used within topics or body elements.                         
/work/llvm.monorepo/flang/docs/FortranForCProgrammers.md:1: WARNING: The "contents" directive may not be used within topics or body elements.            
/work/llvm.monorepo/flang/docs/FortranIR.md:1: WARNING: The "contents" directive may not be used within topics or body elements.                                      
                                                                                                                                                                                            
Exception occurred:                                                                                                                                                                         
  File "/usr/lib/python3/dist-packages/recommonmark/states.py", line 128, in run_role                                                                                                       
    vec, msg = role_fn(name,                                                                                                                                                                
TypeError: 'NoneType' object is not callable

I think it's only the last part that's actually a critical error, and I think what brought this on is the use of the AutoStructify thing.

Some searching suggests this is due to incompatibilities between certain sphinx and recommonmark versions (e.g. https://github.com/sphinx-doc/sphinx/issues/3800) but I really don't have time to debug this kind of stuff.

For now, my plan is to exclude the Flang docs from the build and link to the release notes as rendered by github here: https://github.com/llvm/llvm-project/blob/release/11.x/flang/docs/ReleaseNotes.md

If someone wants to fix the build, or even just send me the built html files once the release is ready, that would of course work too.

But I don't have time to fiddle around with these python packages, sorry.

Hi @hans thanks for giving it a try. I think it is fair enough that you don't try to fix it yourself.

I would be happy to supply generated .html files if you ping me once the release is signed off. @sameeranjoshi might also be available.

Note also that the Flang docs need to be built with -DSPHINX_WARNINGS_AS_ERRORS=OFF, I was following this when I developed the patch. Is this also the case in your builds when you generate the docs?

@sameeranjoshi - I don't think we can easily work around this without losing the automatic content sections in the markdown docs, which would be a shame.

@hans - there is a hacky fix available online. I have pushed it for review here: https://reviews.llvm.org/D87714 would you mind giving it a try?

If it still causes problems then please can you share the output of

sphinx-build --version
pip3 show recommonmark

then I can try to reproduce.