There are a lot of conventions for writing tests that don't seem to be documented anywhere right now, so this takes a stab at writing down some "best practices".
Details
Diff Detail
- Repository
- rG LLVM Github Monorepo
Event Timeline
Thanks, this is really helpful. One question:
> Avoid unnamed instructions/blocks (such as `%0 or 1:`).
Why so? (It would be nice to clarify this in the doc.)
Added a note. Basically, if you remove an unnamed instruction, you need to renumber all following instructions. Also, generated check lines don't use wildcards for basic block names, so while check lines don't care about var names, they care about block names, which are less stable if you use unnamed blocks in the input.
I have a handy script that I use whenever I need to touch a test with unnamed instructions: https://gist.github.com/nikic/145581a29ce951c31e2650ad322359e4
I could think of a number of additional notes for mir tests (although many of those are really because we have poor defaults)
llvm/docs/TestingGuide.rst | ||
---|---|---|
340 | Maybe also mention minimal pass sets (e.g avoid using opt -O3 in most tests) |
LGTM - see inline for minor edits.
Nice! I was wondering if we could adapt/copy:
https://developers.redhat.com/articles/2022/12/20/how-contribute-llvm#
...to the LLVM doc pages directly. This is a great start.
llvm/docs/TestingGuide.rst | ||
---|---|---|
321–325 | - If the test does not crash, assert, or infinite loop, commit the test with baseline check-lines. That is, the test will show a miscompile or missing optimization. Add a "TODO" or "FIXME" comment to indicate that something is expected to change in a test. - A follow-up patch with code changes to the compiler will then show check-line differences to the tests, so it is easier to see the effect of the patch. Remove TODO/FIXME comments added in the previous step if a problem is solved. - Baseline tests (no-functional-change or NFC patch) may be pushed to main without pre-commit review if you have commit access (link to getting started page?). | |
328–329 | No need to repeat clause about regression tests - this section only applies to those. - Include comments about what is tested/expected in a particular test. If there are relevant issues in the bug tracker add references to those bug reports (for example, "See PR999 for more details"). | |
330–332 | - Avoid undefined behavior or values unless necessary. | |
338–339 | Remove "SSA" - this is true in general for source-level tests too? |
Incorporate review feedback. I've split off a separate section for the precommit workflow, as it's longer than a single bullet point.