LangRef: Document the allowed metadata dropping transforms.

[pcc]

Summary

Document the current practice regarding dropping metadata on modules,
functions and global variables.

[mehdi_amini]

(Added a few people that cares about this stuff usually).

So I'm quite worried about this. It slipped in with metadata attachments added to globals, and I'm not sure we really measured the consequence.

I believe we had until now the contract that Metadata attachments (which were on instruction only) were always safely removable: we can always transform the IR without interpreting the Metadata attachment but we have to drop the metadata in the process (if we don't know how to update them).

Adding metadata attachment to global variable is fine by itself, but at the same time it has been used in LLVM 4.0 to express *correctness* related construct: they can't any longer be ignored by the optimizer or the program may break.

I believe the implication of this hasn't been considered enough, and ultimately it means that any pass that looks at a global needs to understand the associated metadata.

It seems critical to me to put a limit on what can be expressed with the metadata (we certainly wouldn't want a metadata saying "any memory access to this variable is volatile", do we?). I'm afraid we're on a slippery slope, and D29104 is going in this direction.

[pcc]

In D29110#655741, @mehdi_amini wrote:

(Added a few people that cares about this stuff usually).

So I'm quite worried about this. It slipped in with metadata attachments added to globals, and I'm not sure we really measured the consequence.

I believe we had until now the contract that Metadata attachments (which were on instruction only) were always safely removable: we can always transform the IR without interpreting the Metadata attachment but we have to drop the metadata in the process (if we don't know how to update them).

Adding metadata attachment to global variable is fine by itself, but at the same time it has been used in LLVM 4.0 to express *correctness* related construct: they can't any longer be ignored by the optimizer or the program may break.

I believe the implication of this hasn't been considered enough, and ultimately it means that any pass that looks at a global needs to understand the associated metadata.

Only passes that rebuild globals without dropping them, and in practice there aren't that many of them. There is already support for copying metadata when rebuilding globals (see GlobalObject::copyMetadata). Look at its callers [0] for how often it is needed. Somehow we've been able to launch features which rely on global-metadata-for-correctness (e.g. CFI and devirtualization on Linux Chrome) with that level of support.

It seems critical to me to put a limit on what can be expressed with the metadata (we certainly wouldn't want a metadata saying "any memory access to this variable is volatile", do we?). I'm afraid we're on a slippery slope, and D29104 is going in this direction.

Agreed that there should be limits, I think we're just trying to figure out what those limits are :)

Ultimately I care about keeping things simple. There are a number of cases where we need to attach "things" to a global for one reason or another. One example of such a "thing" is !type metadata (used for CFI among other things). That metadata is required for correctness, and takes advantage of a number of metadata features (e.g. uniquing based on the value of the isDistinct() flag, which is used to distinguish between classes with external linkage and those with internal linkage). Should we invent some other way of representing type information? I suppose we could, but that would seem to involve a significant parallel infrastructure to metadata and we wouldn't be in much of a better place to where we started, i.e. we would still need to preserve whatever we replace type metadata with, etc.

Perhaps a better way of thinking about whether to keep metadata is by analogy to linkage. We can drop unreferenced globals with internal linkage without breaking semantics, but not external or weak globals. In principle we could add a similar concept for metadata attached to globals, but I don't see a pressing need at this point.

[0] http://llvm-cs.pcc.me.uk/lib/IR/Metadata.cpp/rcopyMetadata

[mehdi_amini]

In D29110#655763, @pcc wrote:

In D29110#655741, @mehdi_amini wrote:

I believe the implication of this hasn't been considered enough, and ultimately it means that any pass that looks at a global needs to understand the associated metadata.

Only passes that rebuild globals without dropping them

What do you mean "without dropping them"? Aren't you making illegal to drop them?

There is already support for copying metadata when rebuilding globals (see GlobalObject::copyMetadata). Look at its callers [0] for how often it is needed. Somehow we've been able to launch features which rely on global-metadata-for-correctness (e.g. CFI and devirtualization on Linux Chrome) with that level of support.

What about merging globals? What about shrinking a global based on the uses? (Int-> bool for example).

Ultimately I care about keeping things simple.

I'm not convinced simplicity has to go before soundness.

Should we invent some other way of representing type information? I suppose we could, but that would seem to involve a significant parallel infrastructure to metadata and we wouldn't be in much of a better place to where we started, i.e. we would still need to preserve whatever we replace type metadata with, etc.

Right, the problem is not that you're using metadata, the problem is that you put a burden on the rest of the compiler to understand your construct, that's a flaw in the way the feature itself is designed.

Perhaps a better way of thinking about whether to keep metadata is by analogy to linkage. We can drop unreferenced globals with internal linkage without breaking semantics, but not external or weak globals. In principle we could add a similar concept for metadata attached to globals, but I don't see a pressing need at this point.

Sorry I don't understand the analogy here. Linkage have a clear semantic and a clear set of rules. Metadata don't.
Dropping them is *not* the only problem, the problem is updating them when mutating the IR and keeping their semantic valid.

[pcc]

In D29110#655887, @mehdi_amini wrote:

In D29110#655763, @pcc wrote:

In D29110#655741, @mehdi_amini wrote:

I believe the implication of this hasn't been considered enough, and ultimately it means that any pass that looks at a global needs to understand the associated metadata.

Only passes that rebuild globals without dropping them

What do you mean "without dropping them"? Aren't you making illegal to drop them?

I meant without dropping the globals.

There is already support for copying metadata when rebuilding globals (see GlobalObject::copyMetadata). Look at its callers [0] for how often it is needed. Somehow we've been able to launch features which rely on global-metadata-for-correctness (e.g. CFI and devirtualization on Linux Chrome) with that level of support.

What about merging globals?

http://llvm-cs.pcc.me.uk/lib/CodeGen/GlobalMerge.cpp#483

What about shrinking a global based on the uses? (Int-> bool for example).

GlobalSplit has custom code for copying type metadata. I was curious to see if there were any other passes like that, and found a global sroa pass in GlobalOpt:
http://llvm-cs.pcc.me.uk/lib/Transforms/IPO/GlobalOpt.cpp#457
So I guess we need to move that code into a central location.

Ultimately I care about keeping things simple.

I'm not convinced simplicity has to go before soundness.

Where did you get the idea that I am trying to make things unsound? Of course I want a design that is both simple and sound.

Should we invent some other way of representing type information? I suppose we could, but that would seem to involve a significant parallel infrastructure to metadata and we wouldn't be in much of a better place to where we started, i.e. we would still need to preserve whatever we replace type metadata with, etc.

Right, the problem is not that you're using metadata, the problem is that you put a burden on the rest of the compiler to understand your construct, that's a flaw in the way the feature itself is designed.

If it is possible to write a pass that copies a global without copying thread_local (or any other semantically significant attribute), would you say that attribute is unsound or flawed? Of course not. And there are ways of copying those attributes (copyAttributesFrom).

Maybe to reduce the overall burden we can teach copyAttributesFrom to copy metadata? Then passes would just need to call one function.

Perhaps a better way of thinking about whether to keep metadata is by analogy to linkage. We can drop unreferenced globals with internal linkage without breaking semantics, but not external or weak globals. In principle we could add a similar concept for metadata attached to globals, but I don't see a pressing need at this point.

Sorry I don't understand the analogy here. Linkage have a clear semantic and a clear set of rules. Metadata don't.

That is, unless we define them. Per above I think it can be done broadly in terms of copying.

Dropping them is *not* the only problem, the problem is updating them when mutating the IR and keeping their semantic valid.

Right, part of that is what copyAttributesFrom does. By calling that function you are saying "I am moving some unspecified part of this global into some unspecified part of that global" (and if you look at the callers that's basically what they are doing). Another part is what copyMetadata does: "I am moving this entire global into this location in this other global". The idea would be to unify them and have a way of saying "I am moving this byte range in this global into this location in this other global".

[mehdi_amini]

In D29110#655948, @pcc wrote:

What about merging globals?

http://llvm-cs.pcc.me.uk/lib/CodeGen/GlobalMerge.cpp#483

I believe this is incorrect.
The metadata being opaque you can't know that they don't prevent some merging in the absolute in the current situation.

Right, the problem is not that you're using metadata, the problem is that you put a burden on the rest of the compiler to understand your construct, that's a flaw in the way the feature itself is designed.

If it is possible to write a pass that copies a global without copying thread_local (or any other semantically significant attribute), would you say that attribute is unsound or flawed? Of course not. And there are ways of copying those attributes (copyAttributesFrom).

This is why we explicitly have this difference between first class construct that can't be ignored and metadata that can be. That provides an easy way to not shoot ourselves in the foot and have an easy set of rules about what we can and can't do.

Dropping them is *not* the only problem, the problem is updating them when mutating the IR and keeping their semantic valid.

Right, part of that is what copyAttributesFrom does. By calling that function you are saying "I am moving some unspecified part of this global into some unspecified part of that global" (and if you look at the callers that's basically what they are doing). Another part is what copyMetadata does: "I am moving this entire global into this location in this other global". The idea would be to unify them and have a way of saying "I am moving this byte range in this global into this location in this other global".

That still does not address that the metadata is opaque and carry any semantic that you don't about and may invalidate. Which is fine as long as metadata don't carry any correctness related semantic.

[mehdi_amini]

Another analogy is instructions vs intrinsics: the former are first class and need specific handling in every pass to understand the semantic of the instruction, while the latter can be opaque because we provide a mechanism of generic "property" (through attributes), and only these attributes / properties needs to be understood by transformations/analyses. Metadata are opaque the same way as intrinsics, but there is no generic mechanism to attach any "semantic property" to them.

[kparzysz]

In D29110#655741, @mehdi_amini wrote:

Adding metadata attachment to global variable is fine by itself, but at the same time it has been used in LLVM 4.0 to express *correctness* related construct: they can't any longer be ignored by the optimizer or the program may break.

I agree. Any information required for correctness is not metadata.

[pcc]

So would it solve the problem if we don't call it metadata? They are "attributes which happen to be represented using the MD* classes".

[mehdi_amini]

In D29110#656480, @pcc wrote:

So would it solve the problem if we don't call it metadata? They are "attributes which happen to be represented using the MD* classes".

You need to define a closed-list of these and their semantic. The key point about metadata is that they are extensible in an opaque way.

[kparzysz]

In D29110#656480, @pcc wrote:

So would it solve the problem if we don't call it metadata? They are "attributes which happen to be represented using the MD* classes".

Would it still look like metadata to someone who has not read that piece of documentation? Would this information invalidate previously correct understanding of the IR?

Imagine that some user somewhere wrote their own LLVM-based compiler that strips all metadata. If that compiler generated a correct code prior to this change, would it potentially generate a wrong code afterwards?

[pcc]

In D29110#656490, @mehdi_amini wrote:

In D29110#656480, @pcc wrote:

So would it solve the problem if we don't call it metadata? They are "attributes which happen to be represented using the MD* classes".

You need to define a closed-list of these and their semantic. The key point about metadata is that they are extensible in an opaque way.

Sure, at the moment those would be type, absolute_symbol and (soon) associated. The idea I have is to define two functions on GlobalObject:

• void copyAttributes(GlobalObject *GO, uint64_t FromBegin, uint64_t FromEnd, uint64_t ToBegin) -- we are moving bytes [FromBegin, FromEnd) to offset ToBegin in "this". Copy "regular" attributes. Copy type, adjusting embedded offsets if necessary. If associated is set, add the global to llvm.compiler.used to make sure it is not dropped. Maybe error out on absolute_symbol (can only be present on declarations and would be replaced by an alias in the IR linker)? Opportunistically also copy dbg if possible (for example by using DW_OP_piece and DW_OP_plus). Copy nothing else.
• void copyMetadata(GlobalObject *GO) -- copy all metadata. This would be used in the IR linker, for example.

[mehdi_amini]

In D29110#656552, @pcc wrote:

In D29110#656490, @mehdi_amini wrote:

In D29110#656480, @pcc wrote:

So would it solve the problem if we don't call it metadata? They are "attributes which happen to be represented using the MD* classes".

You need to define a closed-list of these and their semantic. The key point about metadata is that they are extensible in an opaque way.

Sure, at the moment those would be type, absolute_symbol and (soon) associated.

I'm opposed to associated at this point FWIW.

[mehdi_amini]

In D29110#656551, @kparzysz wrote:

In D29110#656480, @pcc wrote:

So would it solve the problem if we don't call it metadata? They are "attributes which happen to be represented using the MD* classes".

Would it still look like metadata to someone who has not read that piece of documentation? Would this information invalidate previously correct understanding of the IR?

Imagine that some user somewhere wrote their own LLVM-based compiler that strips all metadata. If that compiler generated a correct code prior to this change, would it potentially generate a wrong code afterwards?

I agree and I see this as a problem: writing a pass that would strip all the metadata attachment would not be correct anymore.

[pcc]

In D29110#656551, @kparzysz wrote:

In D29110#656480, @pcc wrote:

So would it solve the problem if we don't call it metadata? They are "attributes which happen to be represented using the MD* classes".

Would it still look like metadata to someone who has not read that piece of documentation? Would this information invalidate previously correct understanding of the IR?

Imagine that some user somewhere wrote their own LLVM-based compiler that strips all metadata. If that compiler generated a correct code prior to this change, would it potentially generate a wrong code afterwards?

You can make the same argument for stripping thread_local (etc). Fundamentally there are always going to be properties that need to be preserved for correctness no matter how we represent them and there's no magical way to make sure that clients don't make mistakes.

If we wanted to help compilers "strip all metadata" correctly, we could provide a function that strips all the metadata except for the "attributes represented with MD*".

[kparzysz]

In D29110#656562, @pcc wrote:

You can make the same argument for stripping thread_local (etc). Fundamentally there are always going to be properties that need to be preserved for correctness no matter how we represent them and there's no magical way to make sure that clients don't make mistakes.

Was there any point in history when stripping thread_local was going to preserve correctness? This isn't about mistakes, this is about making previously correct code wrong. That is not acceptable.

[mehdi_amini]

In D29110#656562, @pcc wrote:

In D29110#656551, @kparzysz wrote:

In D29110#656480, @pcc wrote:

So would it solve the problem if we don't call it metadata? They are "attributes which happen to be represented using the MD* classes".

Would it still look like metadata to someone who has not read that piece of documentation? Would this information invalidate previously correct understanding of the IR?

Imagine that some user somewhere wrote their own LLVM-based compiler that strips all metadata. If that compiler generated a correct code prior to this change, would it potentially generate a wrong code afterwards?

You can make the same argument for stripping thread_local (etc).

And stripping function entirely, and stripping instructions, etc... I don't get your point: there has always been a clear separation between IR and Metadata attachment.

Fundamentally there are always going to be properties that need to be preserved for correctness no matter how we represent them

Right, and there has always been a very clear rule about metadata attachment: *you can't use them for correctness*.

and there's no magical way to make sure that clients don't make mistakes.

See what I mentioned before for intrinsic vs instruction.

Then there is also how one approach the design of a given feature in the first place: for instance the !associated does not have to carry correctness: for instance you can add the variable to compiler.user to model the correctness part, and use the MD to instruct a pass that it can ignore the compiler.used for a specific optimization.
I.e. the MD can be safely dropped, is preserved on a best-effort basis, and optionally passes that are taught about it can optimize better: there's nothing magic about it.

[kparzysz]

In D29110#656562, @pcc wrote:

In D29110#656551, @kparzysz wrote:

Would it still look like metadata to someone who has not read that piece of documentation? Would this information invalidate previously correct understanding of the IR?

[...] there's no magical way to make sure that clients don't make mistakes.

I see what you were referring to. What I meant there was that the design should make it harder for users to make such mistakes. If non-metadata and metadata looks the same, and metadata is safely removable, then the risk of someone accidentally removing non-metadata increases. This is separate from any impact on existing code.

[pcc]

In D29110#656597, @kparzysz wrote:

In D29110#656562, @pcc wrote:

In D29110#656551, @kparzysz wrote:

Would it still look like metadata to someone who has not read that piece of documentation? Would this information invalidate previously correct understanding of the IR?

[...] there's no magical way to make sure that clients don't make mistakes.

I see what you were referring to. What I meant there was that the design should make it harder for users to make such mistakes. If non-metadata and metadata looks the same, and metadata is safely removable, then the risk of someone accidentally removing non-metadata increases. This is separate from any impact on existing code.

Can we solve that problem by renaming getMetadata (etc) to getMDAttributes (etc)? That would make it clear to anyone using the feature that these are just attributes with a special representation.

[sanjoy]

Hi,

Generally I agree with Mehdi and Krzysztof.

I'm also not a big fan of making it "too easy" to add arbitrary semantic attributes to the IR (I think the current design of function attributes is lacking, in this respect). Adding new semantic properties to IR constructs _should_ be a "heavyweight" (of course, up to a point) operation to discourage poorly thought out potentially non-orthogonal design (to be clear, I'm not insinuating the type, absolute_symbol etc. properties are poorly thought out -- I don't have an opinion on them at the moment).

[pcc]

In D29110#656579, @mehdi_amini wrote:

In D29110#656562, @pcc wrote:

In D29110#656551, @kparzysz wrote:

In D29110#656480, @pcc wrote:

So would it solve the problem if we don't call it metadata? They are "attributes which happen to be represented using the MD* classes".

Would it still look like metadata to someone who has not read that piece of documentation? Would this information invalidate previously correct understanding of the IR?

Imagine that some user somewhere wrote their own LLVM-based compiler that strips all metadata. If that compiler generated a correct code prior to this change, would it potentially generate a wrong code afterwards?

You can make the same argument for stripping thread_local (etc).

And stripping function entirely, and stripping instructions, etc... I don't get your point: there has always been a clear separation between IR and Metadata attachment.

I think we can maintain that separation at the API level (see my message to Krzysztof).

Fundamentally there are always going to be properties that need to be preserved for correctness no matter how we represent them

Right, and there has always been a very clear rule about metadata attachment: *you can't use them for correctness*.

Can you show me where that rule is documented for attachments on globals? I know that I did not have this rule in mind when I implemented them. I always viewed them as analogous to global named metadata (which already needs to be preserved for correctness) containing a reference to the global that they are attached to.

and there's no magical way to make sure that clients don't make mistakes.

See what I mentioned before for intrinsic vs instruction.

Then there is also how one approach the design of a given feature in the first place: for instance the !associated does not have to carry correctness: for instance you can add the variable to compiler.user to model the correctness part, and use the MD to instruct a pass that it can ignore the compiler.used for a specific optimization.
I.e. the MD can be safely dropped, is preserved on a best-effort basis, and optionally passes that are taught about it can optimize better: there's nothing magic about it.

I like the idea of teaching GlobalDCE about llvm.compiler.used + associated, it would allow us to move that attribute out of the set of "attributes as MD*" without inhibiting DCE.

I'm not sure how that approach would work for type metadata though. After thinking about it a little I have some ideas for how type information could be represented without metadata. They somewhat resemble my very first proposal to add type information to the IR (http://lists.llvm.org/pipermail/llvm-dev/2015-January/081254.html), but a lot would need to be fleshed out and I'm not sure if it would work. I am also reluctant to go back to the drawing board again after designing/implementing a representation for type information no less than three times already.

[mehdi_amini]

In D29110#656629, @pcc wrote:

Right, and there has always been a very clear rule about metadata attachment: *you can't use them for correctness*.

Can you show me where that rule is documented for attachments on globals? I know that I did not have this rule in mind when I implemented them. I always viewed them as analogous to global named metadata (which already needs to be preserved for correctness) containing a reference to the global that they are attached to.

This is the problem and this is what I told you on IRC... This slipped in with your addition on global MD attachment, without enough scrutiny IMO.
There should have been a dedicated RFC about using Metadata attachment for correctness related thing. I don't believe this happened (At least Duncan and I missed the discussion on this specific aspect if it happened).

Then there is also how one approach the design of a given feature in the first place: for instance the !associated does not have to carry correctness: for instance you can add the variable to compiler.user to model the correctness part, and use the MD to instruct a pass that it can ignore the compiler.used for a specific optimization.
I.e. the MD can be safely dropped, is preserved on a best-effort basis, and optionally passes that are taught about it can optimize better: there's nothing magic about it.

I like the idea of teaching GlobalDCE about llvm.compiler.used + associated, it would allow us to move that attribute out of the set of "attributes as MD*" without inhibiting DCE.

I'm not sure how that approach would work for type metadata though.

I haven't thought about type metadata in particular, and I'm not against attaching a specific property to globals for what is currently the "type metadata", at least on the principle (reusing the MD infrastructure with a different API to differentiate is an option, I haven't given much though about it, and an approach like the one I mentioned for the "associated" is likely preferable when possible).
What I'm worried about is mixing the generic opaque attachment that are considered to always be possible to be ignored with one that can't, for the same reasons exposed by Sanjoy and Krzysztof.

[pcc]

In D29110#656637, @mehdi_amini wrote:

In D29110#656629, @pcc wrote:

Right, and there has always been a very clear rule about metadata attachment: *you can't use them for correctness*.

Can you show me where that rule is documented for attachments on globals? I know that I did not have this rule in mind when I implemented them. I always viewed them as analogous to global named metadata (which already needs to be preserved for correctness) containing a reference to the global that they are attached to.

This is the problem and this is what I told you on IRC... This slipped in with your addition on global MD attachment, without enough scrutiny IMO.
There should have been a dedicated RFC about using Metadata attachment for correctness related thing. I don't believe this happened (At least Duncan and I missed the discussion on this specific aspect if it happened).

Then there is also how one approach the design of a given feature in the first place: for instance the !associated does not have to carry correctness: for instance you can add the variable to compiler.user to model the correctness part, and use the MD to instruct a pass that it can ignore the compiler.used for a specific optimization.
I.e. the MD can be safely dropped, is preserved on a best-effort basis, and optionally passes that are taught about it can optimize better: there's nothing magic about it.

I like the idea of teaching GlobalDCE about llvm.compiler.used + associated, it would allow us to move that attribute out of the set of "attributes as MD*" without inhibiting DCE.

I'm not sure how that approach would work for type metadata though.

I haven't thought about type metadata in particular, and I'm not against attaching a specific property to globals for what is currently the "type metadata", at least on the principle (reusing the MD infrastructure with a different API to differentiate is an option, I haven't given much though about it, and an approach like the one I mentioned for the "associated" is likely preferable when possible).
What I'm worried about is mixing the generic opaque attachment that are considered to always be possible to be ignored with one that can't, for the same reasons exposed by Sanjoy and Krzysztof.

Fair enough. It seems like a reasonable goal to have straightforward ways to

1. copy information from one global to another while preserving correctness (something like the copyAttributes and copyMetadata I proposed earlier)
2. drop all the opaque attachments (metadata, or a subset of it, or whatever) while preserving correctness

As part of getting there, we can figure out the best way to move type and absolute_symbol into the right places in that mechanism.

Do you agree? If so, I will file a bug to keep track of that work.

[mehdi_amini]

In D29110#656660, @pcc wrote:

I like the idea of teaching GlobalDCE about llvm.compiler.used + associated, it would allow us to move that attribute out of the set of "attributes as MD*" without inhibiting DCE.

I'm not sure how that approach would work for type metadata though.

I haven't thought about type metadata in particular, and I'm not against attaching a specific property to globals for what is currently the "type metadata", at least on the principle (reusing the MD infrastructure with a different API to differentiate is an option, I haven't given much though about it, and an approach like the one I mentioned for the "associated" is likely preferable when possible).
What I'm worried about is mixing the generic opaque attachment that are considered to always be possible to be ignored with one that can't, for the same reasons exposed by Sanjoy and Krzysztof.

Fair enough. It seems like a reasonable goal to have straightforward ways to

1. copy information from one global to another while preserving correctness (something like the copyAttributes and copyMetadata I proposed earlier)
2. drop all the opaque attachments (metadata, or a subset of it, or whatever) while preserving correctness

As part of getting there, we can figure out the best way to move type and absolute_symbol into the right places in that mechanism.

Do you agree? If so, I will file a bug to keep track of that work.

Sure.
I don't think there is any emergency on the implementation side, as long as we have somehow an agreement on the direction.

[mehdi_amini]

In D29110#656337, @mehdi_amini wrote:

That still does not address that the metadata is opaque and carry any semantic that you don't about and may invalidate. Which is fine as long as metadata don't carry any correctness related semantic.

I need to clarify, because that was poorly phrased (or even wrong): metadata can already carry correctness related semantic: for example TBAA.
It is not just that the metadata can be ignored, but that they *have to* be dropped if not understood by a transformation. So a metadata-agnostic transformation is always valid as long as it drop the MD.
The issue with these global MDs is not that they affect the correctness per se, but that they can't be dropped by a transformation operating on globals that would invalidate them.

[pcc]

In D29110#656666, @mehdi_amini wrote:

In D29110#656660, @pcc wrote:

I like the idea of teaching GlobalDCE about llvm.compiler.used + associated, it would allow us to move that attribute out of the set of "attributes as MD*" without inhibiting DCE.

I'm not sure how that approach would work for type metadata though.

I haven't thought about type metadata in particular, and I'm not against attaching a specific property to globals for what is currently the "type metadata", at least on the principle (reusing the MD infrastructure with a different API to differentiate is an option, I haven't given much though about it, and an approach like the one I mentioned for the "associated" is likely preferable when possible).
What I'm worried about is mixing the generic opaque attachment that are considered to always be possible to be ignored with one that can't, for the same reasons exposed by Sanjoy and Krzysztof.

Fair enough. It seems like a reasonable goal to have straightforward ways to

1. copy information from one global to another while preserving correctness (something like the copyAttributes and copyMetadata I proposed earlier)
2. drop all the opaque attachments (metadata, or a subset of it, or whatever) while preserving correctness

As part of getting there, we can figure out the best way to move type and absolute_symbol into the right places in that mechanism.

Do you agree? If so, I will file a bug to keep track of that work.

Sure.
I don't think there is any emergency on the implementation side, as long as we have somehow an agreement on the direction.

https://llvm.org/bugs/show_bug.cgi?id=31759

[pcc]

In the meantime, it does at least seem worth documenting the current requirements, so what do you think about committing this?

[mehdi_amini]

llvm/docs/LangRef.rst

3959 ↗

(On Diff #85674)

What about writing:

A transformation is required to drop any metadata attachment that it does not know or know it can't preserve.
Currently there is an exception for metadata attachment to globals for '!type' and '!absolute_symbol' which can't be unconditionally dropped unless the global is itself deleted.

[pcc]

• New wording

[mehdi_amini]

Fine with me! Thanks.

llvm/trunk/docs/LangRef.rst

 
 Metadata can be attached to an instruction. Here metadata ``!21`` is attached
 to the ``add`` instruction using the ``!dbg`` identifier:
 
 .. code-block:: llvm
 
     %indvar.next = add i64 %indvar, 1, !dbg !21
 
-Metadata can also be attached to a function definition. Here metadata ``!22``
-is attached to the ``foo`` function using the ``!dbg`` identifier:
+Metadata can also be attached to a function or a global variable. Here metadata
+``!22`` is attached to the ``f1`` and ``f2 functions, and the globals ``g1``
+and ``g2`` using the ``!dbg`` identifier:
 
 .. code-block:: llvm
 
-    define void @foo() !dbg !22 {
+    declare !dbg !22 void @f1()
+    define void @f2() !dbg !22 {
       ret void
     }
 
+    @g1 = global i32 0, !dbg !22
+    @g2 = external global i32, !dbg !22
+
+A transformation is required to drop any metadata attachment that it does not
+know or know it can't preserve. Currently there is an exception for metadata
+attachment to globals for ``!type`` and ``!absolute_symbol`` which can't be
+unconditionally dropped unless the global is itself deleted.
+
+Metadata attached to a module using named metadata may not be dropped, with
+the exception of debug metadata (named metadata with the name ``!llvm.dbg.*``).
+
 More information about specific metadata nodes recognized by the
 optimizers and code generator is found below.
 
 .. _specialized-metadata:
 
 Specialized Metadata Nodes
 ^^^^^^^^^^^^^^^^^^^^^^^^^^