Add support for creating static libraries when the input
includes only binaries (and not libraries/archives themselves)
Depends on D82923
Details
Diff Detail
- Repository
- rG LLVM Github Monorepo
Event Timeline
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
---|---|---|
48–62 |
These error functions themselves are using basic error handling functions available in the Support library (e.g logAllUnhandledErrors). If I use createStringError available in the support library, I too would have to pass the error object as an argument to logAllUnhandledErrors. Wouldn't I be doing something similar to this only then (I copied these error functions from llvm-lipo) ? sorry if I missed something. Thanks |
Out of time. Note to self: finish review of the rest of the source code and tests.
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
4–5 | I assume the idea is that these two YAML files will be used by many different tests in the future? If so, I'd rename them simply to input1.yaml and input2.yaml to emphasise their generality (in particular, the main bit suggests it is significant, which I suspect is not true). I'd probably also rename their symbols to symbol1 and symbol2 for the same reason. | |
11 | t is probably sufficient rather than tv since you don't check the other fields. | |
15 | Use -NEXT suffixes where applicable. | |
19 | --implicit-check-not=T sounds flaky to me, as file paths might start appearing in the output. It also won't indicate that there are no other symbols, merely no other global text symbols. What is the actual output here? Also, are you interested in the archive's symbol table or the symbol table of the members, because I believe this might be printing the latter. There's a --print-armap option to do the former. | |
26 | Nit: one more space of indentation, please. Again, I think the --implicit-check-not sounds dangerous to me. You might be better off with something like FORMAT-EMPTY: or FORMAT-NOT: {{.}} after the last check line. | |
28 | Add some extra spacing to make this line up: # FORMAT: __.SYMDEF # FORMAT-NEXT: hello.o # FORMAT-NEXT: main.o | |
33–37 | Similar comments apply here to the lines above. Also, I feel like showing that the ordering has been reversed is a little subtle. Assuming the tool completely replaces the original archive, I'd add just a single object to the new one, and show that the output only has that one object in it. | |
46–49 | Same comments as above, re. --implicit-check-not, symbol table etc. | |
llvm/test/tools/llvm-libtool-darwin/invalid-arguments.test | ||
1 ↗ | (On Diff #274936) | I won't review this now, as I think some of it belongs in the first review, plus I'm running low on time for upstream reviewing today! |
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
39 | Can I confirm that the terminology actually is "statically linked library" in Darwin land? I'd typically use simply "static archive" from my ELF background. | |
41 | Is "shared dynamic library" unnecessarily verbose, or is that the common terminology? I'd expect either "shared library" or "dynamic library" from my background, but am not familiar with Darwin development. | |
44–46 | Can this option be delayed to a later change? I'd tentatively do it by default in this version. FWIW, I think LLVM tools typically set deterministic to true by default, since it makes testing easier. | |
48–62 | I was referring to the WithColor::defaultErrorHandler function, which does the actual printing of the error message and handles the Error. There's also `createFileError which takes an Error and adds a provided file name. You will still need to convert the string to an error using createStringError, but could then pass that directly to defaultErrorHandler. | |
69 | Nit: incorrect comment format. (Use capital letter/full stop). | |
70–71 | More idiomatic and simpler is: if (!ObjOrErr) reportError(Member.MemberName, ObjOrErr.takeError()); | |
73 | Ditto. | |
102–103 | Rather than add this as an unsupported option, I'd just only permit the static mode for now. |
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
19 | you are right, --print-armap is what I wanted. It prints: Archive map _symbol1 in create-static-lib.test.tmp-input1.o _symbol2 in create-static-lib.test.tmp-input2.o /data/users/<complete-path>/create-static-lib.test.tmp.lib(create-static-lib.test.tmp-input1.o): 0000000000000000 T _symbol1 /data/users/<complete-path>/create-static-lib.test.tmp.lib(create-static-lib.test.tmp-input2.o): 0000000000000000 T _symbol2 I have now replaced the check with : # CHECK-SYMBOLS: _symbol1 in # CHECK-SYMBOLS: _symbol2 in # CHECK-SYMBOLS-EMPTY: Would this be fine? | |
33–37 | alrighty, replaced the test by adding a single object to it. | |
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
39 | Yup, I took this from the man pages (https://www.unix.com/man-page/osx/1/libtool/):
| |
41 | For the dynamic option the man page says:
But I can update our description to something else too ? (PS: I have removed the dynamic option for this version) | |
44–46 | ok, I'll set it to true for now as well (and pass it as a default instead of creating an option). | |
70–71 | thanks. | |
102–103 | ok, removed dynamic option above. |
Updating test and source files as per the comments:
- removing dynamic option
- using createStringError and createFileError
- removing --implicit-check-nots from the test
(No time to look at tests today, sorry)
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
---|---|---|
63 | I'd expect deterministic to be capitalized in the function signature. If that is the case, this comment should match. | |
77 | You probably don't need this blank line. | |
83 | Let's not use auto here, as it's not immediately clear what the type of Member is. | |
88–91 | Same comment as above re. capitalization in the comments. | |
98 | What is ToolName used for? | |
102 | return EXIT_FAILURE maybe? What do other tools typically do? |
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
---|---|---|
98 | Nothing anymore. Previously, it was being used for the error message. I have removed it now | |
102 | llvm-objcopy does return 1. However, llvm-lipo does exit(EXIT_FAILURE) whereas some other tools (llvm-ar, llvm-as) use exit(1). I have replaced it with exit(EXIT_FAILURE) for now. Is it ok? |
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
16 | Watch out, here and in other cases, this only shows that there is no output AFTER input2.o. It's possible that there's input before input1.o. Also, you'll not catching name corruptions resulting in a prefix/suffix of the name, since FileCheck only does sub-string matching by default, not full line matching. For example, this would fail if the following was emitted: input-I-really-shouldn't-be-here input1.osuffix prefixinput2.o You probably want to add --implicit-check-not={{.}} to the FileCheck command line, rather than the final CHECK-NAMES-NOT. | |
20 | It would be best to check the symbol is in the right member. You can do this by using FileCheck's -D option, combined with the %t_basename (NB: check the exact name for correctness, but there should be other examples): # RUN: ... FileCheck %s -D FILE1=%t_basename ... # CHECK-SYMBOLS: _symbol1 in [[FILE1]] and so on. This defines a string variable that matches the specified input string, and can be used using the [[VAR]] syntax as shown. Take a look at the FileCheck documentation for details. Also, you should check the Archive map string to ensure there's no symbol before the first. | |
23 | Use CHECK-SYMBOLS-NEXT here to show there's no symbol in between the two symbols. | |
42–43 | This has the same issue as the earlier comment re. CHECK-NOT/--implicit-check-not In fact, if you used OVERWRITE-NAMES and OVERWRITE-SYMBOLS instead of CHECK-NAMES/CHECK-SYMBOLS above, the test will still pass. | |
45–46 | Same as above. | |
55–63 | Same comments here as above. | |
llvm/test/tools/llvm-libtool-darwin/invalid-arguments.test | ||
7 ↗ | (On Diff #275578) | Does the double space match the actual error message? |
9–18 ↗ | (On Diff #275578) | These two checks plus the ELF one below probably belong in the invalid input/output arguments test. |
20 ↗ | (On Diff #275578) | Does this message use error: as a prefix? |
24 ↗ | (On Diff #275578) | Did you mean to use $t.lib? (probably not) |
29–34 ↗ | (On Diff #275578) | Only these lines are required to emit a minimal ELF object. Delete everything else. Also, we prefer to use minimal amounts of padding: --- !ELF FileHeader: Class: ELFCLASS64 Data: ELFDATA2LSB Type: ET_REL Machine: EM_X86_64 |
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
35–39 | I'm not really familiar with the Operation type. What does it look like in the help text? |
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
16 | Thanks, I added --implicit-check-not. Furthermore, I also added -DPREFIX=create-static-lib.test.tmp as the file names are represented by [[PREFIX]]-input1.o and [[PREFIX]]-input2.o | |
20 | Yup, I now check that the symbol is in the right member using the following: ## Check that symbols are present: # RUN: llvm-nm --print-armap %t.lib | \ # RUN: FileCheck %s --check-prefix=CHECK-SYMBOLS -DPREFIX=create-static-lib.test.tmp # CHECK-SYMBOLS: Archive map # CHECK-SYMBOLS-NEXT: _symbol1 in [[PREFIX]]-input1.o # CHECK-SYMBOLS-NEXT: _symbol2 in [[PREFIX]]-input2.o # CHECK-SYMBOLS-EMPTY: Would this be ok? | |
llvm/test/tools/llvm-libtool-darwin/invalid-arguments.test | ||
7 ↗ | (On Diff #275578) | Yes, the actual error msg also has the double space: Library Type: option: must be specified at least once! |
9–18 ↗ | (On Diff #275578) | Ok, I have placed all 4 tests into invalid-input-output-args.test now. Please lemme know in case we needed a separate test file for the first test above ## Missing -static option: |
20 ↗ | (On Diff #275578) | yup, I added error: in the error message too now, thanks! |
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
35–39 | this is what help text looks like: OVERVIEW: llvm-libtool-darwin USAGE: llvm-libtool-darwin [options] <input files> OPTIONS: Color Options: --color - Use colors in output (default=autodetect) Generic Options: --help - Display available options (--help-hidden for more) --help-list - Display list of available options (--help-list-hidden for more) --version - Display the version of this program llvm-libtool-darwin options: -o - Alias for -output --output=<filename> - Specify output filename Library Type: --static - Produce a statically linked library from the input files I created an enum Operation here so that in future we can add support for dynamic operation easily. I can very well make the -static option a boolean flag as well. What do you think? |
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
---|---|---|
78 | @jhenderson Should I replace the type of Member back to auto. clang-tidy raises a warning with StringRef? |
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
12 | It's not widely used, but there is %basename_t which substitutes for just the file name part of %t. This allows the test to not make assumptions about how %t expands, and also keeps the check independent of the test name. I'd recommend trying that. | |
19 | Mostly fine, but use %basename_t here too. | |
35 | Checking -> Check | |
38 | Here and below, use %basename_t too, to avoid baking in file names. | |
45 | Ditto. | |
51–53 | Ditto. | |
llvm/test/tools/llvm-libtool-darwin/invalid-arguments.test | ||
7 ↗ | (On Diff #275578) | Okay, bonus marks for fixing it in another patch if you want. Or file a bug against the CommandLine code. |
llvm/test/tools/llvm-libtool-darwin/invalid-input-output-args.test | ||
27 | Maybe make this more generic, e.g. "Missing library type option:" I don't really know where to put this test. It might want to be its own test case entirely, e.g. "missing-library-type.test" | |
44 | Use %basename_t here too. | |
51 | Use %basename_t here too. | |
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
23 | Adding the categories sounds like a different change to me? You might want to include it alongside a test case to show that unrelated options aren't incldued. | |
35–39 | Your current approach seems fine. I was just making sure it didn't do anything too weird. | |
78 | Well typically you don't want to use a const & for StringRef because it is already a light-weight construct (much the same as you wouldn't use const & to a pointer or primitive type). That is probably the cause of the problem here. | |
98 | See above re. option category change. |
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
12 | thanks, I now have -DPREFIX=%basename_t.tmp as %basename_t substitutes for the last path component of %t but without the .tmp extension | |
llvm/test/tools/llvm-libtool-darwin/invalid-arguments.test | ||
7 ↗ | (On Diff #275578) | 😂I can fix it in another patch. |
llvm/test/tools/llvm-libtool-darwin/invalid-input-output-args.test | ||
27 | Isn't it similar to ## Missing output file? (That's why I thought it would go here). But I have placed it in missing-library-type.test for now. | |
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
23 | yup, I added OptionCategory in order to prevent the general option --safepoint-ir-verifier-print-only from showing up in the help text. Added a test case for it as well. Thanks | |
78 | ok thanks |
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
19 | Might wanna add --match-full-lines here, to ensure there's no unexpected symbol name prefix or member name suffix. | |
33 | You have an underscore instead of a dash :) Is the purpose to ensure that there's no other members? I assume a -EMPTY would work for that. We should also check for the "Archive : " header, to ensure there's no members before the table of contents member. | |
48 | Might be worth noting that cctools libtool warns in this case, and we don't implement that warning yet. | |
llvm/test/tools/llvm-libtool-darwin/hide-unrelated-options.test | ||
1 ↗ | (On Diff #276081) | This seems unrelated to this diff; perhaps it should be in the previous one? |
llvm/test/tools/llvm-libtool-darwin/invalid-input-output-args.test | ||
1 | Can you add -static to all of the tests in this file, so that the only invalid part of the command line is the aspect being tested? | |
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
23 | Yup, this should either go in the previous diff or be its own diff. |
Mostly looks good, barring @smeenai's comments.
llvm/test/tools/llvm-libtool-darwin/invalid-input-output-args.test | ||
---|---|---|
27 | Similar in concept, but different behaviour (it's checking the behaviour of the cl::Required on the Library operation option, which is a different thing to the input and output files). | |
llvm/test/tools/llvm-libtool-darwin/missing-library-type.test | ||
4–6 | I don't think it's necessary to add --strict-whitespace here, as the exact formatting of the message isn't really the tool's responsibility. I'd get rid of the double space, as if the message was correct, and the option, so that you don't have to modify this test when you fix the command line message. | |
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
78 | You can get rid of the const too if you want. StringRef itself is immutable, so all this does is stop you re-assigning Member. |
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
33 | Thanks for catching the underscore. I added a check for "Archive : " header now. However, using FORMAT-EMPTY would just check that the next line (after 2nd member) has nothing on it. What I thought we wanted to check was that there is nothing at all after the second member. For eg, Archive : ... ... __.SYMDEF ...<prefix>-input1.o ...<prefix>-input2.o something here would pass with FORMAT-EMPTY just below the check for second member, right? But we want it to fail. | |
llvm/test/tools/llvm-libtool-darwin/hide-unrelated-options.test | ||
1 ↗ | (On Diff #276081) | Yup, thanks. Moved it to the previous diff D82923 |
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
33 | You're right, that makes sense. |
LGTM, with one optional suggestion.
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
52 | It might be worth adding a 2&>1 and check the output is empty here, to flag up if a warning starts getting emitted. That way, it points to where to add testing for the warning. |
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
52 | ok, added # RUN: llvm-libtool-darwin -static -o %t.lib %t-input1.o %t-input2.o %t-input1.o 2>&1 | \ # RUN: FileCheck %s --allow-empty --implicit-check-not={{.}} I hope --alow-empty is the right way to go about empty input files (found it here http://lists.llvm.org/pipermail/llvm-commits/Week-of-Mon-20140804/229905.html). The flag is not there in the documentation |
llvm/test/tools/llvm-libtool-darwin/help-message.test | ||
---|---|---|
10 | added --implicit-check-not="--safepoint-ir-verifier-print-only as we know the headers won't be present because of LIST-NOT checks below. |
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
---|---|---|
34 | enum class |
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
26 | If I'm not mistaken, the canonical name is "Mach-O" (we use MachO for variable and tool names etc, but the file format is technically Mach-O, so we should use that in comments etc). | |
52 | Sounds like a documentation bug. Feel free to update the doc if you get a minute. By default, FileCheck emits an error if the input is empty, but this can be overridden with the use of --allow-empty. That in turn has got me wondering whether --allow-empty should be replaced by a new option that also implies --implicit-check-not. I've raised that on llvm-dev. | |
llvm/test/tools/llvm-libtool-darwin/help-message.test | ||
10 | Sounds reasonable, but I'd get rid of the -- bit of the implicit-check-not pattern, so that it doesn't matter whether options are printed in the help with one or two dashes. | |
llvm/tools/llvm-libtool-darwin/llvm-libtool-darwin.cpp | ||
52 | Mach-O (same as mentioned elsewhere). | |
73 | Ditto. |
llvm/test/tools/llvm-libtool-darwin/create-static-lib.test | ||
---|---|---|
52 | ok, I'll create a diff for the doc, thanks! |
I assume the idea is that these two YAML files will be used by many different tests in the future? If so, I'd rename them simply to input1.yaml and input2.yaml to emphasise their generality (in particular, the main bit suggests it is significant, which I suspect is not true). I'd probably also rename their symbols to symbol1 and symbol2 for the same reason.