# Today

Tue, Aug 20, 2:37 AM · Restricted Project

# Yesterday

Mon, Aug 19, 10:00 AM · Restricted Project

# Thu, Aug 15

Is your GNU objcopy patched to use STV_HIDDEN or STV_PROTECTED for _binary_a_{start,end,size}?

objcopy -I binary -O elf64-x86-64 a b => Symbols in b are STV_HIDDEN/STV_PROTECTED?

Thu, Aug 15, 6:26 AM · Restricted Project

I agree that the generalized version is useful and I have been looking at a patch for this. However, in this specific case the symbol names are automatically generated based on the path of the binary, so it makes it difficult to specify these specific symbol names on the command line.

The symbol names are generated but that is unavoidable. You also have to specify the name in the source: extern const char _binary_a_start[];

With regards to the expected semantics, we anticipate standard ELF semantics when combining different visibilities. Our toolchain operates using a default visibility of STV_HIDDEN for definitions unless explicitly exported, in which case they are STV_PROTECTED. This means that declarations need not be explicitly marked with a visibility in order to be exported or not. The new switch allows the replication of the compiler's -fvisibility=hidden switch in llvm-objcopy. As an alternative to the name --binary-symbol-visibility, how about --symbol-visibility? This would affect the visibility of all the new symbols generated by llvm-objcopy, not just those symbols added for binary input.

There are no other options that add new symbols. --add-symbol has its own visibility flags, --symbol-visibility will not help that feature. --new-symbol-visibility might be a better name but people need to be taught it will be overridden by --add-symbol visibility flags. I am still not sure the little amount of saved labor (__attribute__((visibility("protected"))) marks) justifies a new option, given that Keil embedded development tools already require other markers like __declspec(dllexport) and __declspec(dllimport). And another question: how did you survive with GNU objcopy? It doesn't allow you to change the visibility of _binary_xxx_start...

Quote gABI:

> Second, if any reference to or definition of a name is a symbol with a non-default visibility attribute, the visibility attribute must be propagated to the resolving symbol in the linked object. If different visibility attributes are specified for distinct references to or definitions of a symbol, the most constraining visibility attribute must be propagated to the resolving symbol in the linked object. The attributes, ordered from least to most constraining, are: STV_PROTECTED, STV_HIDDEN and STV_INTERNAL.

You can achieve the same with: attribute((visibility("hidden"))) extern char _binary_a_start[]; So I'm not sure why another option is needed (I think my another concern is that the option name --binary-symbol-visibility can make people puzzled if they add/delete/change symbols). See D55682.

Thu, Aug 15, 3:00 AM · Restricted Project
[llvm-objcopy] Allow 'protected' visibility to be set when using add-symbol

# Wed, Aug 14

• For our use cases it is more convenient to change the symbol's visibility with an option than to have users alter their source code.

I accepted D65891 because:

• It deals with an existing option: --add-symbol. protected is a natural extension to the existing default/hidden visibility. Nobody will get puzzled by this addition.
• According to Changes to symbol visibility between RVCT v3.1 for µVision and RVCT v4.0 for µVision you linked, there are some use cases. I guess it is used for preventing inadvertent preemption in static linking.

The first bullet is sufficient for me to accept it, but I hope you can elaborate more on the second bullet. How do you use STV_PROTECTED in object files and shared objects? What semantics do you expected when linking STV_DEFAULT/STV_PROTECTED undefined/defined together?

For our use cases it is more convenient to change the symbol's visibility with an option than to have users alter their source code.

--binary-symbol-visibility can cause confusion. As suggested by rupprecht, it should be generalized to --redefine-visibility= if we do want the option.

Wed, Aug 14, 6:35 AM · Restricted Project

# Fri, Aug 9

Quote gABI:

Second, if any reference to or definition of a name is a symbol with a non-default visibility attribute, the visibility attribute must be propagated to the resolving symbol in the linked object. If different visibility attributes are specified for distinct references to or definitions of a symbol, the most constraining visibility attribute must be propagated to the resolving symbol in the linked object. The attributes, ordered from least to most constraining, are: STV_PROTECTED, STV_HIDDEN and STV_INTERNAL.

You can achieve the same with: __attribute__((visibility("hidden"))) extern char _binary_a_start[]; So I'm not sure why another option is needed (I think my another concern is that the option name --binary-symbol-visibility can make people puzzled if they add/delete/change symbols). See D55682.

Three symbols per embedding in an object results in bloated output.

I don't understand this point. How can the visibility change de-bloat the output?

Preventing this with many attribute statements seems like a clumsy approach.

I don't understand this, either. Your are going to reference _binary_a_start, (_binary_a_end or _binary_a_size) - two undefined symbols. Restrict the undefined symbol with a visibility attribute, then the linked exe/dso will change the symbol bindings accordingly. What's the problem?

With regards to the confusing option, we can just decide on a better name.

We can do that once we decide this option is useful.

Fri, Aug 9, 9:22 AM · Restricted Project

It makes sense from the perspective of completeness but STV_PROTECTED is problematic and can hardly find a justified use case.

People use STV_PROTECTED when they want to avoid GOT/PLT costs in PIC code. However,

A protected STT_OBJECT prevents you from doing copy relocation. ld.bfd silently accepts this, which can cause ODR violations.
A protected STT_FUNC prevents you from making a canonical PLT (-fno-pic code in executable references an external function). If the linker fails to stop you, you'll get pointer equality problems. (It can be argued this case can be fixed by letting the compiler go through GOT when the address of a protected function is taken, but then the GOT indirection will likely break people's expectation. A symbol lookup change is also required on ld.so side.)

I think lld and gold reject both cases. So at the best, a protected symbol can be seen as a promise of the toolchain that the symbol will not be interposed.

Actually, in a DSO, a STV_PROTECTED use case can be easily replaced with the hidden+alias idiom:

__attribute__((visibility("hidden"))) void internal() {} // internal use
__attribute__((alias("internal"))) void external(); // external API

The reservation of STV_INTERNAL was requested by SGI when gABI was formulated. All other parties appear to treat STV_INTERNAL the same as STV_HIDDEN.

Fri, Aug 9, 4:54 AM · Restricted Project

Quote gABI:

Second, if any reference to or definition of a name is a symbol with a non-default visibility attribute, the visibility attribute must be propagated to the resolving symbol in the linked object. If different visibility attributes are specified for distinct references to or definitions of a symbol, the most constraining visibility attribute must be propagated to the resolving symbol in the linked object. The attributes, ordered from least to most constraining, are: STV_PROTECTED, STV_HIDDEN and STV_INTERNAL.

You can achieve the same with: __attribute__((visibility("hidden"))) extern char _binary_a_start[]; So I'm not sure why another option is needed (I think my another concern is that the option name --binary-symbol-visibility can make people puzzled if they add/delete/change symbols). See D55682.

Fri, Aug 9, 4:20 AM · Restricted Project

# Wed, Aug 7

Wed, Aug 7, 10:26 AM · Restricted Project
Wed, Aug 7, 10:05 AM · Restricted Project

# Thu, Jul 25

I don't think we should change the original error message, as the original message seems to make more sense than the new one for users.

The original error message, when --vs-diagnostics is not used, is mostly the same. There is only a small improvement for a case when only one section, either for an old or new definition, is present.

On the other hand, one of the reasons for --vs-diagnostics is to provide a user with means to jump to the mentioned location. That is achieved by placing the location in front of the error message. Thus, as there are usually two source locations, there should be two separate error messages.

Have you considered changing the regexp to parse this error message?

Thu, Jul 25, 7:44 AM · Restricted Project

# Jul 18 2019

chrisjackson committed rG0b03429a9111: [lld] Fix vs-diagnostics-version-script test. NFC. (authored by chrisjackson).
[lld] Fix vs-diagnostics-version-script test. NFC.

# Jul 17 2019

chrisjackson committed rG87886299b468: [lld] Add Visual Studio compatible diagnostics (authored by chrisjackson).
[lld] Add Visual Studio compatible diagnostics

# Jul 15 2019

Does anyone have any thoughts on these additions?

Jul 15 2019, 5:25 AM · Restricted Project

# Jun 27 2019

[llvm-nm] Fix for BZ41711 - Class character for a symbol with undefined…

# Jun 26 2019

chrisjackson updated the diff for D58484: [DO NOT SUBMIT] Add -vs-diagnostics..

Additional regexes to match more messages containing source:line information. Also added a test.

Jun 26 2019, 8:22 AM · Restricted Project

# Jun 25 2019

Tidied up the test and added some additional cases. Added an assert explaining the tested binding values.

# Jun 13 2019

[llvm-nm] Additional lit tests for command line options

# Jun 6 2019

Have you made sure that there aren't any uses which would result in an undesired change in behaviour because of this? I feel like it's something that people wouldn't expect to change, and therefore wouldn't test explicitly necessarily.

There should also be a test case somewhere (not that I have a good suggestion for where).

I've checked the call sites:

In RuntimeDyld, if (Flags & SymbolRef::SF_Undefined) continue; filters out index 0.
In llvm-objdump, the filtering code is removed in this patch.
In lib/Object/SymbolSize.cpp, I think the code doesn't notice index 0 is included... but it doesn't seem to matter.
In llvm-nm.cpp, I'll leave the test to @chrisjackson in D62148

Jun 6 2019, 7:27 AM · Restricted Project

# Apr 12 2019

Does anyone have any comments on this expanded alternative implementation of vs-diagnostics?

Apr 12 2019, 8:10 AM · Restricted Project

# Mar 21 2019

chrisjackson updated the diff for D58484: [DO NOT SUBMIT] Add -vs-diagnostics..

I have expanded Rui's regex implementation to cover the existing tests for Visual Studio diagnostics compatibility patches,

Mar 21 2019, 8:49 AM · Restricted Project
Mar 21 2019, 8:47 AM · Restricted Project

# Mar 12 2019

I'm working on extending this to pass the existing tests from D58300. Like @ikudrin , I am not fond of this regex method, but I think we will all be in a better place to make an evaluation once this implementation can pass the tests.

Mar 12 2019, 3:57 AM · Restricted Project

# Feb 27 2019

chrisjackson added a comment to D58300: [lld] Enable Visual Studio compatible output.

I like @ruiu 's suggestion in D58484, it is simple and elegant. We don't need performance there, so a regex is fine.
In both cases, yours and @ruiu 's, it would be good to demonstrate usage in the COFF driver as well.
There will be less changes to the LLD codebase with the regex approach. More code changes/additions means more maintenance in the long term.
What do you think on building on @ruiu 's proposal, Chris?

# Feb 22 2019

chrisjackson updated the diff for D58300: [lld] Enable Visual Studio compatible output.

Used clang-format-diff against the patch is requested by @aganea.

# Jan 30 2019

chrisjackson added a comment to D50560: [LLD] Enable Visual Studio compatible diagnostics..

I will provide a patch for this as I'm very keen to add support for Visual Studio diagnostics, but I'm not currently working on it.

Jan 30 2019, 8:47 AM · Restricted Project, lld
chrisjackson added a comment to D56329: Fix some warnings on MSVC.

With regards to

93 if( CMAKE_CXX_COMPILER_VERSION VERSION_GREATER_EQUAL 19.15 )

# Sep 19 2018

chrisjackson added a comment to D50560: [LLD] Enable Visual Studio compatible diagnostics..

@ruiu
Hello Rui, do you have any thoughts on this implementation compared to D47540 ?

Sep 19 2018, 10:19 AM · Restricted Project, lld

# Sep 7 2018

chrisjackson added a comment to D50560: [LLD] Enable Visual Studio compatible diagnostics..

I have experimented with this implementation and I agree that separating the diagnostic location functions into a distinct library is a good structured approach. I think this would be a reasonable and more explicit support for Visual Studio Diagnostics than D47540 which really just shoehorned in support.

Sep 7 2018, 5:21 AM · Restricted Project, lld

# Aug 30 2018

chrisjackson added a reviewer for D51283: Drop __real_ symbol from the output symbol table.: bd1976llvm.

# Aug 23 2018

chrisjackson added a comment to rL340387: Change how we handle -wrap..

This revision contradicts what was agreed in D34993 because we now emit __real_foo in the symbol table. Is this an intended change in behaviour?

# Jul 26 2018

chrisjackson added a comment to D47540: [lld] Enable Visual Studio compatible output.

Can we try the approach where the final style of diagnostic messages is determined in the reporting module and all other code just don't care about it?
Right now, we have a bunch of functions here and there which prepare some parts of the message in a specific style. Maybe it will be more semantically clear if there is a distinct place where the resulting message is prepared while all other parts of the linker provide only basic parts of the message (like message text, source location(s), code location(s), etc.) in a structured form. In this case, we can adjust only that distinct place to add any required message styles.

# Jul 12 2018

chrisjackson updated subscribers of D49184: [lld] Make tests calling llvm-ar more robust.
chrisjackson added a comment to D49184: [lld] Make tests calling llvm-ar more robust.

@sbc100
I agree it would be nice to have an option for llvm-ar to start the archive anew.
I'm not sure about changing all the flag settings as you suggest though. I don't want to enforce a standard or accidentally change the way any of the tests work.
Perhaps standardisation of flags could be a different revision.

chrisjackson added a reviewer for D49184: [lld] Make tests calling llvm-ar more robust: ruiu.

# Jul 11 2018

chrisjackson added a reviewer for D49185: [llvm] Increase robustness of tests calling llvm-ar: ruiu.
chrisjackson updated the summary of D49185: [llvm] Increase robustness of tests calling llvm-ar.
chrisjackson updated the diff for D49185: [llvm] Increase robustness of tests calling llvm-ar.

# Jul 10 2018

chrisjackson updated the diff for D49090: [ThinLTO] Escape module paths when printing.

Followed suggestions to make the test simpler.

# Jul 6 2018

chrisjackson added a comment to D47540: [lld] Enable Visual Studio compatible output.

I think that if we are to add support for Visual Studio (VS) that any
implementation should be explicit and not rely on quietly attempting to
transform any messages to the desired format. As you suggest, the pattern
matching / transform solution is quite hacky. Also, it will implicitly rely on
the diagnostic message string passed having the properties that the
implementation matches on. If we use a pattern matching implementation, have you
considered how we would handle message strings that do not conform to the
expected pattern?

# Jul 4 2018

chrisjackson updated the diff for D47540: [lld] Enable Visual Studio compatible output.
chrisjackson updated the diff for D47540: [lld] Enable Visual Studio compatible output.

Added missing relocation test file in ELF/Inputs/

chrisjackson updated the diff for D47540: [lld] Enable Visual Studio compatible output.

There are several modifications to the code and lit test additions based
on the suggestions given. The default origin or source used to prefix any
diagnostics with no source code file is now stored in Driver and passed to
ErrorHandler. There are methods to set and get the private member
'DiagnosticSourceDefault'. This is a StringRef initialised to "lld", resembling
the previous behaviour of ErrorHandler.LogName.

# Jun 26 2018

chrisjackson updated the diff for D47540: [lld] Enable Visual Studio compatible output.
• Corrected paranthesised/parenthesized usage in source comments
• Updated the lit test to remove usage of deprecated %T
chrisjackson added a comment to D47540: [lld] Enable Visual Studio compatible output.

I created a program that doesn't link and tried to build it on VS. Linker error messages in the VS window are not clickable -- I couldn't jump to the location by clicking a filename and a linename (e.g. foo\bar\baz.cpp:42) in the linker error message. Is this expected?

# Jun 22 2018

chrisjackson added inline comments to D47540: [lld] Enable Visual Studio compatible output.

# Jun 15 2018

chrisjackson added a comment to D47540: [lld] Enable Visual Studio compatible output.

One thing I would have you consider is putting the origin before the message. This doesn't make a super big difference, but it's more consistent with how the code is currently written and the order in which the strings appear.

# Jun 14 2018

chrisjackson added a comment to D47540: [lld] Enable Visual Studio compatible output.
grep -ER '(error|fatal|warn)\(' lld/ELF

finds a lot of places where we are putting location information in diagnostics. Are you planning to convert those to use the new API/formatting, too?

@inglorion
I think this presents a choice that could affect the implementation. If all the call sites are converted, then potentially we could make the Src parameter non-optional. However, for the cases where a diagnostic lacks file source information we would risk inconsistent output of the Origin component of the message due to the lack of the optional default. One idea in this case is providing some functionality in the errorHandler class that helps enforce consistency e.g.

error("unknown argument: " + Arg->getSpelling(), errorSrcDefault());

Alternatively, we can convert only some of the calls and retain the optional class for Src. I'm assuming that for simplicity we are only considering options that have a single interface each for error/warn/fatal.

# Jun 12 2018

chrisjackson added a comment to D47540: [lld] Enable Visual Studio compatible output.

Could you share a few real examples of error messages with and without this patch to see how they look like?

chrisjackson updated the diff for D47540: [lld] Enable Visual Studio compatible output.

Updated the lit test to show more examples of the the new error function, in response to a request from Ruiu. Also, corrected the setting of errorHandler.LogName in driverutils as LogName was incorrectly being set even if the --vs-diagnostics flag was not set.

# Jun 1 2018

chrisjackson updated the diff for D47540: [lld] Enable Visual Studio compatible output.

Thanks for reviewing Rui, I've made the changes as described in my responses to your comments. Primarily:

• removed the extra set of LogName
• made clearer the extraction of the linker executable name from Args
• Made a correction to print() in ErrorHandler
• Removed unnecessary else in inputfiles.cpp

# May 30 2018

chrisjackson added inline comments to D47540: [lld] Enable Visual Studio compatible output.