This is an archive of the discontinued LLVM Phabricator instance.

[test][tools] Add missing/Improve testing
ClosedPublic

Authored by jhenderson on Dec 6 2019, 6:22 AM.

Details

Summary

Mostly this adds testing for certain aliases in a more explicit ways. There are also a few tidy-ups, and additions of missing testing, where the feature was either not tested at all, or not tested explicitly and sufficiently.

Diff Detail

Event Timeline

jhenderson created this revision.Dec 6 2019, 6:22 AM
Herald added a project: Restricted Project. · View Herald TranscriptDec 6 2019, 6:22 AM
Herald added a subscriber: seiya. · View Herald Transcript
grimar added inline comments.Dec 6 2019, 6:53 AM
llvm/test/tools/llvm-nm/elf-archive.test
13

nosyms confused me at first. It sounds like "no symbols", though -S is "do not build a symbol table".
May be "nosymtable` or alike would be better?

llvm/test/tools/llvm-nm/elf-extern-only.test
6

I do not see -g in the llvm-nm`s help text.
Had to look at GNU.
(this and similar comments above are unrelated to this patch, I've put them just FTR).

9

Not sure the description is clear to me. --no-sort just doesn't change symbols order, right?
(help text says "Show symbols in order encountered")

What if we place one more local symbol at the end of the symbol table? (yes, it is not a valid ELF case probably,
but why not for this test case).

llvm/test/tools/llvm-nm/print-filename.test
0–1

-o %t.o?

2–3

FTR, there is no -A in the help text.

llvm/test/tools/llvm-nm/print-size.test
1

-o %t.o?

3

And no -S in the help too.

llvm/test/tools/llvm-objdump/all-headers.test
1

-o %t.o?

llvm/test/tools/llvm-readobj/elf-dynamic-tags.test
1–4

Add an empty line after the comment?

llvm/test/tools/llvm-readobj/elf-relocations.test
51

I.e. --expand-relocs is no-op for GNU? Perhaps worth to mention that in the comment here and below.

jhenderson updated this revision to Diff 232564.Dec 6 2019, 7:36 AM
jhenderson marked 10 inline comments as done.

Address all inline review comments.

llvm/test/tools/llvm-nm/elf-extern-only.test
6

LLVM's command-line handler by default hides aliases. I don't know why. In some cases, we've explicitly marked them as not hidden, but not in others. I've got cleaning up the help text (a long way down) on my backlog, so hopefully it'll get fixed at some point.

I believe all the aliases should be in the LLVM Command Guide documentation by the way.

9

I've modified the comment a bit to make that point clearer.

llvm/test/tools/llvm-readobj/elf-dynamic-tags.test
1–4

Okay, although I only usually do this where there is more than one case. In this case, there aren't any other blocks covered by this comment.

llvm/test/tools/llvm-readobj/elf-relocations.test
51

That's what the "LLVM style only" bit is meant to say. I'll clarify it a bit more.

rupprecht accepted this revision.Dec 6 2019, 10:25 AM
rupprecht added inline comments.
llvm/test/tools/llvm-nm/elf-extern-only.test
6

+1, would be nice to also see aliases grouped with the thing they're aliasing. I also have it on my list, but very far down :)

50

nit: no newline

This revision is now accepted and ready to land.Dec 6 2019, 10:25 AM
MaskRay accepted this revision.Dec 6 2019, 10:48 AM
MaskRay added inline comments.
llvm/test/tools/llvm-readobj/elf-relocations.test
2

This may replicate some tests in reloc-types-elf-x64.test but I can see that this test mostly checks format and corner case values.

6

Do we usually use cmp to test that alias options have the identical output?

353

Align

llvm/test/tools/llvm-symbolizer/functions.s
17

Unrelated to this patch.

-f can be either --functions or --functions=, and the parsing accounts for the next option. This looks strange.

grimar accepted this revision.Dec 7 2019, 12:20 AM

LGTM with a nit.

llvm/test/tools/llvm-readobj/elf-relocations.test
347
# -> ##

(and below)

llvm/test/tools/llvm-symbolizer/functions.s
17

I am not sure I understood your concern.
It is just the same as --functions none but with the use of alias -f, isn't?

In another test we have a similar check:

# Check --obj aliases --exe, -e
# RUN: llvm-symbolizer 0xa 0xb --exe=%t.o | FileCheck %s
# RUN: llvm-symbolizer 0xa 0xb -e %t.o | FileCheck %s
# RUN: llvm-symbolizer 0xa 0xb -e=%t.o | FileCheck %s
# RUN: llvm-symbolizer 0xa 0xb -e%t.o | FileCheck %s
jhenderson updated this revision to Diff 232801.Dec 9 2019, 3:34 AM
jhenderson marked 6 inline comments as done.

Address review comments (added missing line, fixed indentation, added extra '#' for inline comments).

llvm/test/tools/llvm-readobj/elf-relocations.test
2

Yes, that was my intention. I guess I could use the same relocation type throughout (although I did deliberately want one that was 64-bit and one that wasn't for the SHT_REL relocations). This test is more about the formatting and general behaviour of --relocations, whereas the reloc-types-elf-x64.test is intended to exhaustively test the supported X86_64 relocation types.

6

It seems like we have a bit of a mixed bag on this one. I actually prefer cmp, since it guarantees that the alias really is an alias, but most newer tests written by others don't do that, so I followed their lead.

I'm going to go ahead and commit this as is shortly, but I'm more than happy to go and update them to use cmp instead if you like (and might do that for other tests I see too for the same reasoning).

llvm/test/tools/llvm-symbolizer/functions.s
17

Like @grimar, I'm not sure what it is you are concerned by?

Note that --functions none is NOT the same as --functions=none. The former is actually equivalent to --functions=linkage none. This block of tests demonstrates this for both --functions and -f (which should be identical in behaviour).

grimar added inline comments.Dec 9 2019, 3:44 AM
llvm/test/tools/llvm-symbolizer/functions.s
17

After re-reading the original comment today, I've got it I think.
Fangrui did not mean that this change is unrelated, but pointed to the difference of the behavior,
which is "--functions none is NOT the same as --functions=none". It is indeed looks strange
(I've never saw such things before I think).

jhenderson marked an inline comment as done.Dec 9 2019, 3:54 AM
jhenderson added inline comments.
llvm/test/tools/llvm-symbolizer/functions.s
17

Ah, that would make some sense. The problem basically is because --functions exists in GNU addr2line, but does not take an argument, whereas --functions was implemented in llvm-symbolizer to take an argument. To resolve this incompatibility with GNU, it was decided that --functions arg would function like GNU's --functions, i.e. not consume arg as part of the --functions option. See this commit. I'd be reluctant to change the behaviour again now, even though we now have llvm-addr2line for GNU compatibility, since I don't think there's major benefit to breaking the behaviour for existing users. Note that we in particular need to be careful with something like --functions 0x1234 to make sure that 0x1234 is treated as an address and not a value for --functions.

This revision was automatically updated to reflect the committed changes.

Urrghh... apparently somebody recently changed the default behaviour of llvm-cxxfilt on darwin (see D70250), which means this change broke those tests. I will commit a fix for them, since it's easy to fix, but I'd like a review of it post-commit. I've also updated my herald rules to make sure I get added as a reviewer on such changes in the future.

thakis added a subscriber: thakis.Dec 9 2019, 6:57 AM

Looks like the test is still broken with your fix attempt: http://45.33.8.238/mac/3784/step_11.txt

Can you try another fix, or if it takes a while to investigate, revert in the meantime?

Looks like the test is still broken with your fix attempt: http://45.33.8.238/mac/3784/step_11.txt

Can you try another fix, or if it takes a while to investigate, revert in the meantime?

Thanks. I missed one of the "-n" instances that I had removed in rG2815390, and have fixed this in rG01d8bb49.

thakis added a comment.Dec 9 2019, 7:24 AM

Now everything's passing again. Thanks for the quick fix!