Page MenuHomePhabricator
Feed Advanced Search

Today

chrisjackson updated the summary of D66426: [lld] Enable a watermark of loadable sections to be generated and placed in a note section.
Tue, Aug 20, 2:37 AM · Restricted Project

Yesterday

chrisjackson added reviewers for D66426: [lld] Enable a watermark of loadable sections to be generated and placed in a note section: jhenderson, edd, andrewng.
Mon, Aug 19, 10:00 AM · Restricted Project
chrisjackson created D66426: [lld] Enable a watermark of loadable sections to be generated and placed in a note section.
Mon, Aug 19, 9:56 AM · Restricted Project

Thu, Aug 15

chrisjackson added a comment to D65893: [llvm-objcopy] Allow the visibility of the start, end and size symbols created by --binary to be specified with --binary-symbol-visibility.

Is your GNU objcopy patched to use STV_HIDDEN or STV_PROTECTED for _binary_a_{start,end,size}?

objcopy -I binary -O elf64-x86-64 a b => Symbols in b are STV_HIDDEN/STV_PROTECTED?

Thu, Aug 15, 6:26 AM · Restricted Project
chrisjackson added a comment to D65893: [llvm-objcopy] Allow the visibility of the start, end and size symbols created by --binary to be specified with --binary-symbol-visibility.

I agree that the generalized version is useful and I have been looking at a patch for this. However, in this specific case the symbol names are automatically generated based on the path of the binary, so it makes it difficult to specify these specific symbol names on the command line.

The symbol names are generated but that is unavoidable. You also have to specify the name in the source: extern const char _binary_a_start[];

With regards to the expected semantics, we anticipate standard ELF semantics when combining different visibilities. Our toolchain operates using a default visibility of STV_HIDDEN for definitions unless explicitly exported, in which case they are STV_PROTECTED. This means that declarations need not be explicitly marked with a visibility in order to be exported or not. The new switch allows the replication of the compiler's -fvisibility=hidden switch in llvm-objcopy. As an alternative to the name --binary-symbol-visibility, how about --symbol-visibility? This would affect the visibility of all the new symbols generated by llvm-objcopy, not just those symbols added for binary input.

There are no other options that add new symbols. --add-symbol has its own visibility flags, --symbol-visibility will not help that feature. --new-symbol-visibility might be a better name but people need to be taught it will be overridden by --add-symbol visibility flags. I am still not sure the little amount of saved labor (__attribute__((visibility("protected"))) marks) justifies a new option, given that Keil embedded development tools already require other markers like __declspec(dllexport) and __declspec(dllimport). And another question: how did you survive with GNU objcopy? It doesn't allow you to change the visibility of _binary_xxx_start...

In D65893#1620711, @MaskRay wrote:
Quote gABI:

> Second, if any reference to or definition of a name is a symbol with a non-default visibility attribute, the visibility attribute must be propagated to the resolving symbol in the linked object. If different visibility attributes are specified for distinct references to or definitions of a symbol, the most constraining visibility attribute must be propagated to the resolving symbol in the linked object. The attributes, ordered from least to most constraining, are: STV_PROTECTED, STV_HIDDEN and STV_INTERNAL.

You can achieve the same with: attribute((visibility("hidden"))) extern char _binary_a_start[]; So I'm not sure why another option is needed (I think my another concern is that the option name --binary-symbol-visibility can make people puzzled if they add/delete/change symbols). See D55682.

Thu, Aug 15, 3:00 AM · Restricted Project
chrisjackson committed rGe5cdfbc65cac: [llvm-objcopy] Allow 'protected' visibility to be set when using add-symbol (authored by chrisjackson).
[llvm-objcopy] Allow 'protected' visibility to be set when using add-symbol
Thu, Aug 15, 2:46 AM

Wed, Aug 14

chrisjackson added a comment to D65893: [llvm-objcopy] Allow the visibility of the start, end and size symbols created by --binary to be specified with --binary-symbol-visibility.
  • For our use cases it is more convenient to change the symbol's visibility with an option than to have users alter their source code.

I accepted D65891 because:

  • It deals with an existing option: --add-symbol. protected is a natural extension to the existing default/hidden visibility. Nobody will get puzzled by this addition.
  • According to Changes to symbol visibility between RVCT v3.1 for µVision and RVCT v4.0 for µVision you linked, there are some use cases. I guess it is used for preventing inadvertent preemption in static linking.

    The first bullet is sufficient for me to accept it, but I hope you can elaborate more on the second bullet. How do you use STV_PROTECTED in object files and shared objects? What semantics do you expected when linking STV_DEFAULT/STV_PROTECTED undefined/defined together?

For our use cases it is more convenient to change the symbol's visibility with an option than to have users alter their source code.

--binary-symbol-visibility can cause confusion. As suggested by rupprecht, it should be generalized to --redefine-visibility= if we do want the option.

Wed, Aug 14, 6:35 AM · Restricted Project

Fri, Aug 9

chrisjackson added a comment to D65893: [llvm-objcopy] Allow the visibility of the start, end and size symbols created by --binary to be specified with --binary-symbol-visibility.

Quote gABI:

Second, if any reference to or definition of a name is a symbol with a non-default visibility attribute, the visibility attribute must be propagated to the resolving symbol in the linked object. If different visibility attributes are specified for distinct references to or definitions of a symbol, the most constraining visibility attribute must be propagated to the resolving symbol in the linked object. The attributes, ordered from least to most constraining, are: STV_PROTECTED, STV_HIDDEN and STV_INTERNAL.

You can achieve the same with: __attribute__((visibility("hidden"))) extern char _binary_a_start[]; So I'm not sure why another option is needed (I think my another concern is that the option name --binary-symbol-visibility can make people puzzled if they add/delete/change symbols). See D55682.

Three symbols per embedding in an object results in bloated output.

I don't understand this point. How can the visibility change de-bloat the output?

Preventing this with many attribute statements seems like a clumsy approach.

I don't understand this, either. Your are going to reference _binary_a_start, (_binary_a_end or _binary_a_size) - two undefined symbols. Restrict the undefined symbol with a visibility attribute, then the linked exe/dso will change the symbol bindings accordingly. What's the problem?

With regards to the confusing option, we can just decide on a better name.

We can do that once we decide this option is useful.

Fri, Aug 9, 9:22 AM · Restricted Project
chrisjackson added a comment to D65891: [llvm-objcopy] Allow 'protected' visibility to be set when using add-symbol.

It makes sense from the perspective of completeness but STV_PROTECTED is problematic and can hardly find a justified use case.

People use STV_PROTECTED when they want to avoid GOT/PLT costs in PIC code. However,

A protected STT_OBJECT prevents you from doing copy relocation. ld.bfd silently accepts this, which can cause ODR violations.
A protected STT_FUNC prevents you from making a canonical PLT (-fno-pic code in executable references an external function). If the linker fails to stop you, you'll get pointer equality problems. (It can be argued this case can be fixed by letting the compiler go through GOT when the address of a protected function is taken, but then the GOT indirection will likely break people's expectation. A symbol lookup change is also required on ld.so side.)

I think lld and gold reject both cases. So at the best, a protected symbol can be seen as a promise of the toolchain that the symbol will not be interposed.

Actually, in a DSO, a STV_PROTECTED use case can be easily replaced with the hidden+alias idiom:

__attribute__((visibility("hidden"))) void internal() {} // internal use
__attribute__((alias("internal"))) void external(); // external API

The reservation of STV_INTERNAL was requested by SGI when gABI was formulated. All other parties appear to treat STV_INTERNAL the same as STV_HIDDEN.

Fri, Aug 9, 4:54 AM · Restricted Project
chrisjackson added a comment to D65893: [llvm-objcopy] Allow the visibility of the start, end and size symbols created by --binary to be specified with --binary-symbol-visibility.

Quote gABI:

Second, if any reference to or definition of a name is a symbol with a non-default visibility attribute, the visibility attribute must be propagated to the resolving symbol in the linked object. If different visibility attributes are specified for distinct references to or definitions of a symbol, the most constraining visibility attribute must be propagated to the resolving symbol in the linked object. The attributes, ordered from least to most constraining, are: STV_PROTECTED, STV_HIDDEN and STV_INTERNAL.

You can achieve the same with: __attribute__((visibility("hidden"))) extern char _binary_a_start[]; So I'm not sure why another option is needed (I think my another concern is that the option name --binary-symbol-visibility can make people puzzled if they add/delete/change symbols). See D55682.

Fri, Aug 9, 4:20 AM · Restricted Project

Wed, Aug 7

Herald added a reviewer for D65893: [llvm-objcopy] Allow the visibility of the start, end and size symbols created by --binary to be specified with --binary-symbol-visibility: alexshap.
Wed, Aug 7, 10:26 AM · Restricted Project
Herald added a reviewer for D65891: [llvm-objcopy] Allow 'protected' visibility to be set when using add-symbol: alexshap.
Wed, Aug 7, 10:05 AM · Restricted Project

Thu, Jul 25

chrisjackson added a comment to D65213: [ELF] With --vs-diagnostics, print a separate message for each location of a duplicate symbol..

I don't think we should change the original error message, as the original message seems to make more sense than the new one for users.

The original error message, when --vs-diagnostics is not used, is mostly the same. There is only a small improvement for a case when only one section, either for an old or new definition, is present.

On the other hand, one of the reasons for --vs-diagnostics is to provide a user with means to jump to the mentioned location. That is achieved by placing the location in front of the error message. Thus, as there are usually two source locations, there should be two separate error messages.

Have you considered changing the regexp to parse this error message?

Thu, Jul 25, 7:44 AM · Restricted Project

Jul 18 2019

chrisjackson committed rG0b03429a9111: [lld] Fix vs-diagnostics-version-script test. NFC. (authored by chrisjackson).
[lld] Fix vs-diagnostics-version-script test. NFC.
Jul 18 2019, 2:19 AM

Jul 17 2019

chrisjackson committed rG87886299b468: [lld] Add Visual Studio compatible diagnostics (authored by chrisjackson).
[lld] Add Visual Studio compatible diagnostics
Jul 17 2019, 7:56 AM

Jul 15 2019

chrisjackson added a comment to D58484: [DO NOT SUBMIT] Add -vs-diagnostics..

Does anyone have any thoughts on these additions?

Jul 15 2019, 5:25 AM · Restricted Project

Jun 27 2019

chrisjackson committed rG41e20d21015b: [llvm-nm] Fix for BZ41711 - Class character for a symbol with undefined… (authored by chrisjackson).
[llvm-nm] Fix for BZ41711 - Class character for a symbol with undefined…
Jun 27 2019, 9:29 AM

Jun 26 2019

chrisjackson updated the diff for D58484: [DO NOT SUBMIT] Add -vs-diagnostics..

Additional regexes to match more messages containing source:line information. Also added a test.

Jun 26 2019, 8:22 AM · Restricted Project

Jun 25 2019

chrisjackson updated the diff for D63340: [llvm-nm] Fix for BZ41711 - Class character for a symbol with undefined binding does not match class assigned by GNU nm.

Tidied up the test and added some additional cases. Added an assert explaining the tested binding values.

Jun 25 2019, 6:59 AM

Jun 14 2019

chrisjackson updated the summary of D63340: [llvm-nm] Fix for BZ41711 - Class character for a symbol with undefined binding does not match class assigned by GNU nm.
Jun 14 2019, 10:05 AM
chrisjackson updated the summary of D63340: [llvm-nm] Fix for BZ41711 - Class character for a symbol with undefined binding does not match class assigned by GNU nm.
Jun 14 2019, 9:41 AM
chrisjackson created D63340: [llvm-nm] Fix for BZ41711 - Class character for a symbol with undefined binding does not match class assigned by GNU nm.
Jun 14 2019, 9:34 AM

Jun 13 2019

chrisjackson committed rG7b395133029a: [llvm-nm] Additional lit tests for command line options (authored by chrisjackson).
[llvm-nm] Additional lit tests for command line options
Jun 13 2019, 3:37 AM

Jun 6 2019

chrisjackson added reviewers for D62955: [llvm-nm] Additional lit tests for command line options: MaskRay, rupprecht, mtrent, evgeny777.
Jun 6 2019, 7:43 AM
chrisjackson added a comment to D62296: [Object] object::ELFObjectFile::symbol_begin(): skip symbol index 0.

Have you made sure that there aren't any uses which would result in an undesired change in behaviour because of this? I feel like it's something that people wouldn't expect to change, and therefore wouldn't test explicitly necessarily.

There should also be a test case somewhere (not that I have a good suggestion for where).

I've checked the call sites:

In RuntimeDyld, if (Flags & SymbolRef::SF_Undefined) continue; filters out index 0.
In llvm-objdump, the filtering code is removed in this patch.
In lib/Object/SymbolSize.cpp, I think the code doesn't notice index 0 is included... but it doesn't seem to matter.
In llvm-nm.cpp, I'll leave the test to @chrisjackson in D62148

Jun 6 2019, 7:27 AM · Restricted Project
chrisjackson created D62955: [llvm-nm] Additional lit tests for command line options.
Jun 6 2019, 7:24 AM

May 21 2019

chrisjackson updated subscribers of D62148: [llvm-nm] Omit the symbol table entry at index 0 when --debug-syms is enabled for ELF files.
May 21 2019, 4:25 AM

May 20 2019

chrisjackson created D62148: [llvm-nm] Omit the symbol table entry at index 0 when --debug-syms is enabled for ELF files.
May 20 2019, 8:30 AM

Apr 12 2019

chrisjackson added a comment to D58484: [DO NOT SUBMIT] Add -vs-diagnostics..

Does anyone have any comments on this expanded alternative implementation of vs-diagnostics?

Apr 12 2019, 8:10 AM · Restricted Project

Mar 21 2019

chrisjackson updated the diff for D58484: [DO NOT SUBMIT] Add -vs-diagnostics..

I have expanded Rui's regex implementation to cover the existing tests for Visual Studio diagnostics compatibility patches,

Mar 21 2019, 8:49 AM · Restricted Project
chrisjackson commandeered D58484: [DO NOT SUBMIT] Add -vs-diagnostics..
Mar 21 2019, 8:47 AM · Restricted Project

Mar 12 2019

chrisjackson added a comment to D58484: [DO NOT SUBMIT] Add -vs-diagnostics..

I'm working on extending this to pass the existing tests from D58300. Like @ikudrin , I am not fond of this regex method, but I think we will all be in a better place to make an evaluation once this implementation can pass the tests.

Mar 12 2019, 3:57 AM · Restricted Project

Feb 27 2019

chrisjackson added a comment to D58300: [lld] Enable Visual Studio compatible output.

I like @ruiu 's suggestion in D58484, it is simple and elegant. We don't need performance there, so a regex is fine.
In both cases, yours and @ruiu 's, it would be good to demonstrate usage in the COFF driver as well.
There will be less changes to the LLD codebase with the regex approach. More code changes/additions means more maintenance in the long term.
What do you think on building on @ruiu 's proposal, Chris?

Feb 27 2019, 9:27 AM · lld

Feb 22 2019

chrisjackson updated the diff for D58300: [lld] Enable Visual Studio compatible output.

Used clang-format-diff against the patch is requested by @aganea.

Feb 22 2019, 9:36 AM · lld

Feb 15 2019

chrisjackson created D58300: [lld] Enable Visual Studio compatible output.
Feb 15 2019, 12:13 PM · lld

Jan 30 2019

chrisjackson added a comment to D50560: [LLD] Enable Visual Studio compatible diagnostics..

I will provide a patch for this as I'm very keen to add support for Visual Studio diagnostics, but I'm not currently working on it.

Jan 30 2019, 8:47 AM · Restricted Project, lld
chrisjackson added a comment to D56329: Fix some warnings on MSVC.

With regards to

93 if( CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 19.15 )
Jan 30 2019, 7:32 AM

Sep 19 2018

chrisjackson added a comment to D50560: [LLD] Enable Visual Studio compatible diagnostics..

@ruiu
Hello Rui, do you have any thoughts on this implementation compared to D47540 ?

Sep 19 2018, 10:19 AM · Restricted Project, lld

Sep 7 2018

chrisjackson added a comment to D50560: [LLD] Enable Visual Studio compatible diagnostics..

I have experimented with this implementation and I agree that separating the diagnostic location functions into a distinct library is a good structured approach. I think this would be a reasonable and more explicit support for Visual Studio Diagnostics than D47540 which really just shoehorned in support.

Sep 7 2018, 5:21 AM · Restricted Project, lld

Sep 6 2018

chrisjackson added a comment to D51283: Drop __real_ symbol from the output symbol table..

@ruiu

Sep 6 2018, 3:51 AM

Aug 30 2018

chrisjackson added a reviewer for D51283: Drop __real_ symbol from the output symbol table.: bd1976llvm.
Aug 30 2018, 4:14 AM

Aug 23 2018

chrisjackson added a comment to rL340387: Change how we handle -wrap..

This revision contradicts what was agreed in D34993 because we now emit __real_foo in the symbol table. Is this an intended change in behaviour?

Aug 23 2018, 8:07 AM

Jul 26 2018

chrisjackson added a comment to D47540: [lld] Enable Visual Studio compatible output.

Can we try the approach where the final style of diagnostic messages is determined in the reporting module and all other code just don't care about it?
Right now, we have a bunch of functions here and there which prepare some parts of the message in a specific style. Maybe it will be more semantically clear if there is a distinct place where the resulting message is prepared while all other parts of the linker provide only basic parts of the message (like message text, source location(s), code location(s), etc.) in a structured form. In this case, we can adjust only that distinct place to add any required message styles.

Jul 26 2018, 8:07 AM

Jul 12 2018

chrisjackson updated subscribers of D49184: [lld] Make tests calling llvm-ar more robust.
Jul 12 2018, 9:26 AM
chrisjackson added a comment to D49184: [lld] Make tests calling llvm-ar more robust.

@sbc100
I agree it would be nice to have an option for llvm-ar to start the archive anew.
I'm not sure about changing all the flag settings as you suggest though. I don't want to enforce a standard or accidentally change the way any of the tests work.
Perhaps standardisation of flags could be a different revision.

Jul 12 2018, 4:58 AM
chrisjackson added a reviewer for D49184: [lld] Make tests calling llvm-ar more robust: ruiu.
Jul 12 2018, 4:51 AM

Jul 11 2018

chrisjackson added a reviewer for D49185: [llvm] Increase robustness of tests calling llvm-ar: ruiu.
Jul 11 2018, 7:15 AM
chrisjackson updated the summary of D49185: [llvm] Increase robustness of tests calling llvm-ar.
Jul 11 2018, 7:11 AM
chrisjackson updated the diff for D49185: [llvm] Increase robustness of tests calling llvm-ar.
Jul 11 2018, 7:10 AM
chrisjackson created D49185: [llvm] Increase robustness of tests calling llvm-ar.
Jul 11 2018, 7:09 AM
chrisjackson created D49184: [lld] Make tests calling llvm-ar more robust.
Jul 11 2018, 7:02 AM

Jul 10 2018

chrisjackson updated the diff for D49090: [ThinLTO] Escape module paths when printing.

Followed suggestions to make the test simpler.

Jul 10 2018, 7:19 AM

Jul 9 2018

chrisjackson created D49090: [ThinLTO] Escape module paths when printing.
Jul 9 2018, 10:45 AM

Jul 6 2018

chrisjackson added a comment to D47540: [lld] Enable Visual Studio compatible output.

I think that if we are to add support for Visual Studio (VS) that any
implementation should be explicit and not rely on quietly attempting to
transform any messages to the desired format. As you suggest, the pattern
matching / transform solution is quite hacky. Also, it will implicitly rely on
the diagnostic message string passed having the properties that the
implementation matches on. If we use a pattern matching implementation, have you
considered how we would handle message strings that do not conform to the
expected pattern?

Jul 6 2018, 4:10 AM

Jul 4 2018

chrisjackson updated the diff for D47540: [lld] Enable Visual Studio compatible output.
Jul 4 2018, 7:57 AM
chrisjackson updated the diff for D47540: [lld] Enable Visual Studio compatible output.

Added missing relocation test file in ELF/Inputs/

Jul 4 2018, 7:51 AM
chrisjackson updated the diff for D47540: [lld] Enable Visual Studio compatible output.

There are several modifications to the code and lit test additions based
on the suggestions given. The default origin or source used to prefix any
diagnostics with no source code file is now stored in Driver and passed to
ErrorHandler. There are methods to set and get the private member
'DiagnosticSourceDefault'. This is a StringRef initialised to "lld", resembling
the previous behaviour of ErrorHandler.LogName.

Jul 4 2018, 6:29 AM

Jun 26 2018

chrisjackson updated the diff for D47540: [lld] Enable Visual Studio compatible output.
  • Corrected paranthesised/parenthesized usage in source comments
  • Updated the lit test to remove usage of deprecated %T
Jun 26 2018, 3:05 AM
chrisjackson added a comment to D47540: [lld] Enable Visual Studio compatible output.

I created a program that doesn't link and tried to build it on VS. Linker error messages in the VS window are not clickable -- I couldn't jump to the location by clicking a filename and a linename (e.g. foo\bar\baz.cpp:42) in the linker error message. Is this expected?

Jun 26 2018, 2:09 AM

Jun 22 2018

chrisjackson added inline comments to D47540: [lld] Enable Visual Studio compatible output.
Jun 22 2018, 1:48 AM

Jun 15 2018

chrisjackson added a comment to D47540: [lld] Enable Visual Studio compatible output.

One thing I would have you consider is putting the origin before the message. This doesn't make a super big difference, but it's more consistent with how the code is currently written and the order in which the strings appear.

Jun 15 2018, 7:00 AM

Jun 14 2018

chrisjackson added a comment to D47540: [lld] Enable Visual Studio compatible output.

@chrisjackson,

grep -ER '(error|fatal|warn)\(' lld/ELF

finds a lot of places where we are putting location information in diagnostics. Are you planning to convert those to use the new API/formatting, too?

@inglorion
I think this presents a choice that could affect the implementation. If all the call sites are converted, then potentially we could make the Src parameter non-optional. However, for the cases where a diagnostic lacks file source information we would risk inconsistent output of the Origin component of the message due to the lack of the optional default. One idea in this case is providing some functionality in the errorHandler class that helps enforce consistency e.g.

error("unknown argument: " + Arg->getSpelling(), errorSrcDefault());

Alternatively, we can convert only some of the calls and retain the optional class for Src. I'm assuming that for simplicity we are only considering options that have a single interface each for error/warn/fatal.

Jun 14 2018, 1:48 AM

Jun 12 2018

chrisjackson added a comment to D47540: [lld] Enable Visual Studio compatible output.

Could you share a few real examples of error messages with and without this patch to see how they look like?

Jun 12 2018, 9:20 AM
chrisjackson updated the diff for D47540: [lld] Enable Visual Studio compatible output.

Updated the lit test to show more examples of the the new error function, in response to a request from Ruiu. Also, corrected the setting of errorHandler.LogName in driverutils as LogName was incorrectly being set even if the --vs-diagnostics flag was not set.

Jun 12 2018, 9:14 AM

Jun 1 2018

chrisjackson updated the diff for D47540: [lld] Enable Visual Studio compatible output.

Thanks for reviewing Rui, I've made the changes as described in my responses to your comments. Primarily:

  • removed the extra set of LogName
  • made clearer the extraction of the linker executable name from Args
  • Made a correction to print() in ErrorHandler
  • Removed unnecessary else in inputfiles.cpp
Jun 1 2018, 9:09 AM

May 30 2018

chrisjackson added inline comments to D47540: [lld] Enable Visual Studio compatible output.
May 30 2018, 9:53 AM
chrisjackson created D47540: [lld] Enable Visual Studio compatible output.
May 30 2018, 9:17 AM