This is an archive of the discontinued LLVM Phabricator instance.

[clang-offload-bundler] Add option -list
ClosedPublic

Authored by yaxunl on Dec 9 2020, 11:15 AM.

Details

Summary

clang-offload-bundler is not only used by clang driver
to bundle/unbundle files for offloading toolchains,
but also used by out of tree tools to unbundle
fat binaries generated by clang. It is important
to be able to list the bundle IDs in a bundled
file so that the bundles can be extracted.

This patch adds an option -list to list bundle
ID's in a bundled file. Each bundle ID is separated
by new line. If the file is not a bundled file
nothing is output and returns 0.

Diff Detail

Event Timeline

yaxunl requested review of this revision.Dec 9 2020, 11:15 AM
yaxunl created this revision.
tra added inline comments.Dec 9 2020, 11:45 AM
clang/tools/clang-offload-bundler/ClangOffloadBundler.cpp
781

Having to create a temporary file in order to *list* content of the bundle strikes me as rather odd.
It looks like in order to list the content we actually do bundle unpacking, printing bundled content in the process, and discard the results afterwards. Is that so?

Perhaps it would be better to refactor the code a bit and separate iteration over the bundle from what each iteration does.
E.g. make a function forEachBundledFile(input, lambda) and then pass a function that writes things out for normal operations and a function which just prints the info in case of --list. It may simplify the code a bit as right now you have to copy/paste the loops iterating over ReadBundleStart.

yaxunl updated this revision to Diff 310748.Dec 9 2020, 7:39 PM
yaxunl marked an inline comment as done.

Revised by Artem's comments: removing unnecessary output to temporary file, extract forEachBundle.

clang/tools/clang-offload-bundler/ClangOffloadBundler.cpp
781

done. thanks.

yaxunl updated this revision to Diff 310751.Dec 9 2020, 7:56 PM

Remove unnecessary formatting changes.

tra added inline comments.Dec 10 2020, 10:50 AM
clang/tools/clang-offload-bundler/ClangOffloadBundler.cpp
199

Now, if we could save the triple in a BundleInfo when it's parsed, and pass BundleInfo to Func() that would make the iterator more useful.

351–352

I'm still not quite happy with the fact that the listing is interleaving with reading the bundle.
I think the code would benefit from further refactoring that would separate bundle reading from what we do with the bundles we've read.

662

Once BundleInfo carries the triple, ListBundleIDs could then be changed to print the bundle info and moved into the FileHandler class:

if (Error Err = ReadHeader(Input))
    return Err;

return forEachBundle(Input, [](BundleInfo bundle) { llvm::outs() << bundle.triple << '\n'});

All other printouts of the triple should no longer be necessary.

yaxunl marked 3 inline comments as done.Dec 11 2020, 5:30 AM
yaxunl added inline comments.
clang/tools/clang-offload-bundler/ClangOffloadBundler.cpp
199

done

351–352

good point. this can be moved to the lambda without incurring significant overhead.

662

done

yaxunl updated this revision to Diff 311187.Dec 11 2020, 5:33 AM
yaxunl marked 3 inline comments as done.

Revised by Artem's comments.

tra added inline comments.Jan 5 2021, 12:22 PM
clang/tools/clang-offload-bundler/ClangOffloadBundler.cpp
183

Now that listBundleIDs is only used by ListBundleIDsInFile(), perhaps it could all be simplified and moved out of the class.

889

I'd pass InputFileNames as an argument. Makes it easier to tell what the function needs.

1071–1075

Does it make sense to continue once we know that CLI options are wrong?
If we just early-exit with an error that may help simplifying the code below a bit.

1087

Perhaps we can separate list option processing from bundle/unbundle?

I think if we could do something like this it would be more readable:

if (ListBundleIDs) {
  if (Unbundle) {
    error...
    exit.
  }
  ... other list-specific checks...
  ListBundleIDsInFile(InputFileNames)
  exit 0;
}

// complicated bundle/unbundle logic can proceed without having to bother about `list` option.
yaxunl marked 4 inline comments as done.Jan 6 2021, 12:08 PM
yaxunl added inline comments.
clang/tools/clang-offload-bundler/ClangOffloadBundler.cpp
183

listBundleIDsCallback needs to be a virtual function and it is called by listBundleIDs. Keep listBundleIDs as a member function together with listBundleIDsCallback shows their relation and is more readable.

889

done

1071–1075

done

1087

done

yaxunl updated this revision to Diff 314957.Jan 6 2021, 12:09 PM
yaxunl marked 4 inline comments as done.

revised by Artem's comments

tra accepted this revision.Jan 6 2021, 12:37 PM
This revision is now accepted and ready to land.Jan 6 2021, 12:37 PM
This revision was automatically updated to reflect the committed changes.
Herald added a project: Restricted Project. · View Herald TranscriptJan 6 2021, 1:23 PM