This is an archive of the discontinued LLVM Phabricator instance.

[Docs] Replace recommonmark with myst-parser
AcceptedPublic

Authored by luke on Jan 20 2023, 4:12 AM.

Details

Summary

Now that recommonmark is deprecated https://github.com/readthedocs/recommonmark/issues/221,
myst-parser seems to be the generally accepted drop in replacement.
This swaps recommonmark out of the sphinx config and upgrades any
markdown document that needs adjustment due to myst-parser's stricter
parsing.
In particular, these things needed addressed as myst-parser complained
about them:

  • Broken references were replaced
  • Invalid lexers in code blocks were replaced or removed (It didn't look like they were being syntax highlighted anyway)
  • Non-contiguous headers (# -> #) were made contiguous (# -> )

Closes #55787

Diff Detail

Event Timeline

luke created this revision.Jan 20 2023, 4:12 AM
Herald added projects: Restricted Project, Restricted Project. · View Herald Transcript
luke requested review of this revision.Jan 20 2023, 4:12 AM
Herald added projects: Restricted Project, Restricted Project. · View Herald Transcript
luke added inline comments.Jan 20 2023, 4:15 AM
flang/docs/DiagnosticsReference.md
2

This is pretty hairy and it will link users to a blank document. Open to any other ideas to workaround this

tstellar added inline comments.
flang/docs/ComplexOperations.md
37

Do these changes have any affected on the output html? Is there another way to specify the language?

luke added inline comments.Jan 20 2023, 5:44 AM
flang/docs/ComplexOperations.md
37

I’m not sure, I think I might be able to add this back in though since I suppressed a warning about highlight errors.
I had to remove the ones that specified MLIR as the lexer though since there didn’t seem to be an option to suppress those warnings. They didn’t seem to receive any syntax highlighting in the output html anyway

luke added inline comments.Jan 23 2023, 4:59 AM
flang/docs/ComplexOperations.md
37

Looks like these weren't being highlighted anyway: https://flang.llvm.org/docs/ComplexOperations.html?highlight=complex#fir-representation

Ideally there would eventually be an MLIR lexer added to pygments. Until then is it better to remove the incorrect lexers?

luke updated this revision to Diff 491318.Jan 23 2023, 5:14 AM

Don't suppress misc.highlighting_failure warning

Instead just fix the flang code block that was failing to highlight

tstellar accepted this revision.Jan 23 2023, 8:17 AM

LGTM. Can you add something to the release notes about this.

This revision is now accepted and ready to land.Jan 23 2023, 8:17 AM
luke updated this revision to Diff 491667.Jan 24 2023, 1:46 AM

Update release notes

luke added a comment.Jan 24 2023, 1:48 AM

LGTM. Can you add something to the release notes about this.

Done. Do any build machines need to be upgraded before this is landed?

The flang website (flang.llvm.org) or flang.llvm.org/docs are built from these. Would you know whether there are changes required are there? https://github.com/llvm/llvm-zorg/blob/main/zorg/buildbot/builders/SphinxDocsBuilder.py

luke added a comment.Jan 24 2023, 2:02 AM

The flang website (flang.llvm.org) or flang.llvm.org/docs are built from these. Would you know whether there are changes required are there? https://github.com/llvm/llvm-zorg/blob/main/zorg/buildbot/builders/SphinxDocsBuilder.py

The docs-flang-html target should remain the same, I quickly checked that script and it seems fine. The main thing is that the builders would need to have myst-parser installed on them. Are these python dependencies installed via a script or is it done manually?

@luke You'll need to contact the bot owners so they can update machine. The workers running the publish-sphinx-docs will need to be updated too. @gkistanova should be able to help.