This is an archive of the discontinued LLVM Phabricator instance.

[yaml2obj] Allow variable number of directories
ClosedPublic

Authored by alfonsosanchezbeato on Aug 27 2021, 8:38 AM.

Details

Summary

Allow variable number of directories, as allowed by the
specification. NumberOfRvaAndSize will default to 16 if not specified,
as in the past.

Author: Alfonso Sánchez-Beato <alfonso.sanchez-beato@canonical.com>

Diff Detail

Event Timeline

alfonsosanchezbeato requested review of this revision.Aug 27 2021, 8:38 AM

@mstorsjo, do you have any particular thoughts on this patch at all?

llvm/lib/ObjectYAML/COFFEmitter.cpp
474–475

Don't you need to repeat this multiple times potentially, in case the NumberOfRvaAndSize value is more than one bigger than the number of DataDirectories?

llvm/test/tools/yaml2obj/COFF/variable-number-rva.yaml
2

This test case is insufficient. You need the following cases:

  1. NumberOfRvaAndSize is unspecified (value is the default value).
  2. NumberOfRvaAndSize is specified and matches the default value.
  3. NumberOfRvaAndSize is specified and is lower than the default value (need to check that the relevant DataDirectory content is not written too).
  4. NumberOfRvaAndSize is specified and is higher (more than 1 higher, preferably) than the default value (need to show that the additional zeros are written too).

You may find it useful to use the yaml2obj preprocessor-like functionality. This allows you to specify -D VAR=0, and yaml2obj will replace patterns in the input like the following before processing the document:

# RUN: yaml2obj ... -DVAR=0

---! COFF
<snip>
NumberOfRvaAndSize = [[VAR]]

becomes

NumberOfRvaAndSize = 0

You can also specify a default value using the following: NumberOfRvaAndSize = [[VAR=0]] which would use the value 0 if VAR is not defined on the command-line.

Finally, I believe, but am not certain, you may be able to use the pattern NumberOfRvaAndSize = [[VAR=<none>]], as NumberOfRvaAndSize is a default option. In this case, if VAR is not defined on the command-line, it is as if NumberOfRvaAndSize does not appear in the YAML document and therefore a default value is used (i.e. test case 1 above).

By using the -D option, you should be able to use the same YAML doc in all test cases.

mstorsjo added inline comments.Aug 31 2021, 1:45 AM
llvm/lib/ObjectYAML/COFFEmitter.cpp
461

I find the fused postincrement here a bit possibly confusing - e.g. with NumberOfRvaAndSize = 1, after the loop you would have written one entry, but end up with NumDir = 2. By doing a separate increment after the conditional, the variable would stay better in sync with reality.

474–475

Yeah, making this a while loop should be straightforward and make things more robust.

I guess it'd be a bit unexpected (as CP.Obj.OptionalHeader->DataDirectories is a hardcoded size for all potential types) but if the caller sets a larger value in NumberOfRvaAndSize then we should add dummy padding.

alfonsosanchezbeato updated this revision to Diff 370760.EditedSep 4 2021, 1:34 PM

Comments addressed. I've added tests as described, including a new yaml as NumberOfRvaAndSize = [[VAR=<none>]] does not seem to work.

Comments addressed. I've added tests as described, including a new yaml as NumberOfRvaAndSize = [[VAR=<none>]] does not seem to work.

That's unfortunate, and seems to be an artefact of how the header properties are used internally (they need to be of type Optional<T> to work with <none>). That sounds like something that could be improved in COFF yaml2obj in the future, but not right now. (Also, it should have been NumberOfRvaAndSize: [[VAR=<none>]] FWIW).

llvm/lib/ObjectYAML/COFFEmitter.cpp
461–480

I think the suggested edit is equivalent, and also simpler overall.

llvm/test/tools/yaml2obj/COFF/default-number-rva.yaml
1 ↗(On Diff #370760)

yaml2obj provides the --docnum option to allow multiple YAML documents in the same file. That will allow you to move this test case into the same test file as the variable rva cases. That in turn may enable you to reduce the amount of CHECK statements needed, by sharing them between different FileCheck invocations.

llvm/test/tools/yaml2obj/COFF/variable-number-rva.yaml
6

Does the DataDirectory list get printed (as an empty list) in this case? It would be good to test that if so.

134

You need to show that there are no further attributes printed after those you check for. Applies to all cases.

Diff updated, now there is only one yaml with two documents and I have editted the loop and suggested.

alfonsosanchezbeato marked 6 inline comments as done.Sep 6 2021, 10:00 AM
alfonsosanchezbeato added inline comments.
llvm/test/tools/yaml2obj/COFF/variable-number-rva.yaml
6

It doesn't in this case (both for llvm-readobj and obj2yaml).

Just a few nits remaining, otherwise looks pretty good.

llvm/lib/ObjectYAML/COFFEmitter.cpp
461–467

Now that I see it written out like this, I find NumDir confusing as a name. Could you rename it to I, since it's a basic loop counter.

llvm/test/tools/yaml2obj/COFF/variable-number-rva.yaml
2

I'd add a comment (using ## for the comment marker) before each run of yaml2obj to explain what is significant about each case (i.e. default/higher than normal/etc).

3–4

I'd rename the prefixes in this case to CHECK16 and ROUNDTRIP16, to show what value they are effectively testing.

121–123

Something needs to be added in these two cases to show that there's no DataDirectory dumped. It might be a CHECK-NOT or CHECK-EMPTY directive for the CHECK case, for example.

130–131

As above, use CHECK6 and ROUNDTRIP6.

171

Similar to my comments above, use CHECK18 (and ROUNDTRIP18).

Comments from latest review addressed, thanks.

jhenderson added inline comments.Sep 8 2021, 12:08 AM
llvm/lib/ObjectYAML/COFFEmitter.cpp
462

Nit: please remember to reformat modified/new code.

llvm/test/tools/yaml2obj/COFF/variable-number-rva.yaml
2

Comments are full sentences and should end in a full stop (applies in tests as well as source files).

Same goes for each of your other new comments.

3–4

Ping this comment?

Comment addressed now.

jhenderson accepted this revision.Sep 9 2021, 1:03 AM

LGTM! Do you have commit access yet?

This revision is now accepted and ready to land.Sep 9 2021, 1:03 AM

LGTM! Do you have commit access yet?

Great! No, I don't have commit access...

This revision was landed with ongoing or failed builds.Sep 9 2021, 3:17 AM
This revision was automatically updated to reflect the committed changes.

I've just pushed this. Please pay attention to any build bot failure emails you get, to see if they were caused by this change.