This is an archive of the discontinued LLVM Phabricator instance.

Replace trivial use of external rc.exe by writing our own .res file.
AcceptedPublic

Authored by ecbeckmann on Jun 22 2017, 12:00 PM.

Details

Reviewers
zturner
ruiu
Summary

This patch removes the dependency on the external rc.exe tool by writing a simple .res file using our own library. In this patch I also added an explicit definition for the .res file magic.

Event Timeline

ecbeckmann created this revision.Jun 22 2017, 12:00 PM
ruiu edited edge metadata.Jun 22 2017, 1:29 PM

I think a better way of creating manifest file contents is to construct a temporary std::vector and then copy it to a MemoryBuffer after you append everything to the vector. This way, you don't need to compute the size of the result ahead of time.

lld/COFF/DriverUtils.cpp
380

s/get/create/ as it creates a new instance of MemoryBuffer.

381–386

It feels to me that this is cleaner.

size_t Size = alignTo(object::WIN_RES_MAGIC_SIZE +
                      object::WIN_RES_NULL_ENTRY_SIZE +
                      sizeof(object::WinResHeaderPrefix) +
                      sizeof(object::WinResIDs) +
                      sizeof(object::WinResHeaderSuffix) +
                      Manifest.size(),
                      object::WIN_RES_DATA_ALIGNMENT);
426–428

Why do you want to strip all newlines?

427–436

Since you are not using the contents of Manifest, pass Manifest.size() instead of Manifest.

429

Rename Current Buf and use uint8_t instead of char for consistency

llvm/include/llvm/BinaryFormat/COFF.h
50

Since all the other Magics are of const char[], you want to use char.

llvm/include/llvm/Object/WindowsResource.h
49–88

Remove blank lines. Replace sizeof(uint32_t) with 4 because it is always 4.

67–72

You are not really using this struct. If you need only the size of the struct, define it as a constant.

In D34525#788360, @ruiu wrote:

I think a better way of creating manifest file contents is to construct a temporary std::vector and then copy it to a MemoryBuffer after you append everything to the vector. This way, you don't need to compute the size of the result ahead of time.

Hmmm won't this method require an extra copy? Also, it seems less convenient and more messy to have all these resize() operations before we write anything.

ruiu added a comment.Jun 22 2017, 2:51 PM

It indeed will require extra copy, but that is fine. You don't need to avoid extra copy here since the generated file image is small and this is executed just once.

You can use push_back or something to avoid resize, no? Or, you can use an OutputStream backed up by some buffer and write data using operator<< there.

ecbeckmann marked 5 inline comments as done.

Made some style changes. Also fixed bug in magic test where the full .res magic number was not recognized.

ecbeckmann added inline comments.Jun 22 2017, 3:50 PM
lld/COFF/DriverUtils.cpp
426–428

This mimics the behavior of rc.exe, which strips all the newlines from a manifest before putting it in the .res file. However, spaces are left intact.

429

Hmmm there is already a variable named "Buf".

429

Also const_char to type uint8_t* is not possible, and requires an additional cast.

llvm/include/llvm/Object/WindowsResource.h
67–72

I do use in in DriverUtils.cpp for a reinterpret_cast. It helps I think for writing the resource header.

ruiu added inline comments.Jun 22 2017, 3:56 PM
lld/COFF/DriverUtils.cpp
426–428

If you don't do this and it still works, don't do that. We don't have to mimic all the details.

429

Then please rename the conflicting one. Buf is used as a pointer to uint8_t in LLD very often, so not naming it is a bit confusing.

llvm/include/llvm/Object/WindowsResource.h
67–72

Then maybe you want to remove the default value 0xffff. You are not using it.

ecbeckmann added a comment.EditedJun 22 2017, 4:25 PM
In D34525#788514, @ruiu wrote:

It indeed will require extra copy, but that is fine. You don't need to avoid extra copy here since the generated file image is small and this is executed just once.

You can use push_back or something to avoid resize, no? Or, you can use an OutputStream backed up by some buffer and write data using operator<< there.

How do we use push_back when we are adding all these elements of different sizes? However using a stream may be best.

ruiu added a comment.Jun 22 2017, 4:29 PM

Well, operator<< should still work, but if doing that makes code messy, you don't have to do that.

In D34525#788612, @ruiu wrote:

Well, operator<< should still work, but if doing that makes code messy, you don't have to do that.

Hmm yes I don't think using '<<' necessarily makes it easier. Also reinterpret_cast, increment pointer pattern is well used in LLD already.

zturner added inline comments.Jun 22 2017, 8:03 PM
lld/COFF/DriverUtils.cpp
394

This is going to trigger the msan bot because we'll be writing uninitialized memory. You probably need to write 0s here.

427–428

Yea I would rather not do this as well. (FWIW, you also haven't handled carriage returns, which are likely to be present, so to keep it simple let's just leave it as is)

llvm/unittests/BinaryFormat/TestFileMagic.cpp
79–80

This is modified but no test is modified. Is that normal?

126

Should this be removed?

ecbeckmann marked 4 inline comments as done.

Removed new line stripping for manifests, initialize 0s for null entry.

llvm/include/llvm/Object/WindowsResource.h
67–72

Oh shoot you're right. These fields must be set to 0xffff when the Type and Names are IDs, however. I should make a method for setting them.

llvm/unittests/BinaryFormat/TestFileMagic.cpp
79–80

The change was as follows: I added the WinResMagic constant definition in BinaryFormat/COFF.h. I then modified Magic.cpp to compare based on this constant instead of it's previous method of comparing to a string which, in fact, did not contain all the magic bytes. This broke the test, because the test magic didn't contain the full signature, so it would fail the comparison in Magic.cpp

ruiu added a comment.Jun 23 2017, 8:44 AM

Looks much better. A few minor comments.

llvm/include/llvm/Object/WindowsResource.h
70

Instead of defining setType and setName, you can just set TypeFlag and NameID to 0xffff in writeResEntryHeader, no?

llvm/lib/BinaryFormat/Magic.cpp
54–55

You want to make sure that Magic.data() is at least the same size as WinResMagic.

ecbeckmann marked an inline comment as done.

Check size of magic before comparing

llvm/include/llvm/Object/WindowsResource.h
70

I think it would be better if this level were abstracted away from the user class. After all, it is a detail of the Type and Name fields that the most significant bytes are always set to high when number IDs are used. This is an implementation detail about the .res spec that should be hidden.

ruiu accepted this revision.Jun 23 2017, 11:29 AM

LGTM

This revision is now accepted and ready to land.Jun 23 2017, 11:29 AM

Fix a bug where incorrect characteristic flags are set in section header.

Added a unittest for the embedded manifest case.

Fixed formatting issue.

ruiu added a comment.Jun 26 2017, 8:49 AM

Still LGTM