[WebAssembly] Support `annotate` clang attributes for marking functions.

[brendandahl]

Summary

Annotation attributes may be attached to a function to mark it with
custom data that will be contained in the final Wasm file. The
annotation causes a custom section named
"func_attr.annotate.<name>.<arg0>.<arg1>..." to be created that will
contain each function's index value that was marked with the annotation.

A new patchable relocation type for function indexes had to be created so
the custom section could be updated during linking.

[sbc100]

This change looks really nice. I like the new relocation type, I think we would have had to add that sooner or later anyway.

My only major concern is making this attribute available outside of emscripten (i.e. wasm32-unknown-emscripten). It seems like we should maybe called it em_async or something like that? And make it illegal on other targets?

@dschuff WDYT?

clang/include/clang/Basic/Attr.td

1984 ↗

(On Diff #523110)

Should we call this em_async or emscripten_async since this is an emscripten-specific attribute?

llvm/lib/MC/MCExpr.cpp

366

Maybe this on single line to match the local convention.

668

Was this whole block indented? Maybe limit this to just a single line change?

[aheejin]

Nice! Just some drive-by nitpicking, sorry 😅

lld/test/wasm/async.ll

11 ↗

(On Diff #523110)

21–22 ↗

(On Diff #523110)

lld/test/wasm/merge-async-section.ll

26 ↗

(On Diff #523110)

38–39 ↗

(On Diff #523110)

llvm/test/CodeGen/WebAssembly/async.ll

9 ↗

(On Diff #523110)

13–14 ↗

(On Diff #523110)

llvm/test/MC/WebAssembly/async.s

10–11 ↗

(On Diff #523110)

The intention looks little weird.. Is that what llc emits?

[brendandahl]

Review comments.

clang/include/clang/Basic/Attr.td

1984 ↗

(On Diff #523110)

I wouldn't say this is emscripten specific. You could use this with clang+binaryen without emscripten.

llvm/lib/MC/MCExpr.cpp

668

This was changed by git-clang-format HEAD~1 . I can revert though and only change that line.

llvm/test/MC/WebAssembly/async.s

10–11 ↗

(On Diff #523110)

Yeah, I wasn't really sure how to format, but that's what llc was doing. Open to changing it.

[sbc100]

clang/include/clang/Basic/Attr.td

1984 ↗

(On Diff #523110)

Maybe we can at least say that its web-specific? Would it make any sense for non-web platforms?

If you are using clang + binaryen + web its hard to imagine a non-emscripten triple being used.. but maybe?

lld/test/wasm/async.ll

18 ↗

(On Diff #525348)

For lld tests I've been trying for a while now to move away from .ll and just use .s format.

The only reason we have any .ll files at all in this directory is because for a long time we didn't have a working .s format.

llvm/lib/Target/WebAssembly/WebAssemblyAsmPrinter.cpp

563

Perhaps call this AsyncFuncs or AsyncFuncNames?

569

No need for curlys around one-liners in llvm

llvm/test/MC/WebAssembly/async.s

20 ↗

(On Diff #525348)

Indentation look wrong here.. Index, Offset and Name should align with Type I think?

[aaron.ballman]

This is missing all the usual Sema tests that the attribute only applies to functions, takes no arguments, only works in web assembly targets, etc. Also, the changes should come with a release note so users know about the new attribute.

clang/include/clang/Basic/AttrDocs.td

5608–5612 ↗

(On Diff #525348)

This could be my ignorance of web assembly showing, but the docs don't really help me understand when you'd want to use this attribute. Perhaps a link to what JSPI is and a code example would help a little bit? Or is this more of a low-level implementation detail kind of attribute where folks already know the domain?

clang/lib/CodeGen/TargetInfo.cpp

898 ↗

(On Diff #525348)

clang/lib/Sema/SemaDeclAttr.cpp

7637–7640 ↗

(On Diff #525348)

This should be handled automatically for you by the common attribute handling code, so I think you can remove this.

7645 ↗

(On Diff #525348)

This diagnostic doesn't make sense to me -- how does this attribute relate to ifuncs or aliases? Why should users be prohibited from writing the attribute on a definition?

[brendandahl]

llvm/test/MC/WebAssembly/async.s

10–11 ↗

(On Diff #523110)

Actually, it does look like something got messed up. I'll fix.

[brendandahl]

clang/lib/Sema/SemaDeclAttr.cpp

7645 ↗

(On Diff #525348)

This was from a previous version, it's not needed anymore. I'll remove.

[brendandahl]

Address comments.

[erichkeane]

Couple of CFE comments, otherwise LGTM.

clang/docs/ReleaseNotes.rst

559 ↗

(On Diff #527218)

We require underscores to spell it in code, so I figure we should do it here too.

clang/lib/Sema/SemaDeclAttr.cpp

7638 ↗

(On Diff #527218)

We should probably document that this forces the function to be emitted as well.

[brendandahl]

clang/lib/CodeGen/TargetInfo.cpp

892 ↗

(On Diff #527218)

Whoops, I changed the wrong one here. I'll fix that.

[dschuff]

In D150803#4350554, @sbc100 wrote:

This change looks really nice. I like the new relocation type, I think we would have had to add that sooner or later anyway.

My only major concern is making this attribute available outside of emscripten (i.e. wasm32-unknown-emscripten). It seems like we should maybe called it em_async or something like that? And make it illegal on other targets?

@dschuff WDYT?

Hm, this is interesting because in the long term we plan to have stack switching in wasm, which could allow for similar async behavior that JSPI has, and could be useful in non-web systems. But that's a ways off. The file format we are generating with this CL will be used in emscripten sooner (and we may want to try to stabilize it some point, possibly before pure wasm stack switching is usable in non-web systems).
So overall I kind of feel like I could go either way on this. Curious if @sunfish has had any thoughts about async outside of emscripten.

[sunfish]

In D150803#4389040, @dschuff wrote:

Hm, this is interesting because in the long term we plan to have stack switching in wasm, which could allow for similar async behavior that JSPI has, and could be useful in non-web systems. But that's a ways off. The file format we are generating with this CL will be used in emscripten sooner (and we may want to try to stabilize it some point, possibly before pure wasm stack switching is usable in non-web systems).
So overall I kind of feel like I could go either way on this. Curious if @sunfish has had any thoughts about async outside of emscripten.

When we do have stack switching, would you anticipate still using this wasm_async attribute, or would we switch to something else at that point?

[brendandahl]

Review comments. Add tombstone for unused functions.

[brendandahl]

clang/lib/Sema/SemaDeclAttr.cpp

7638 ↗

(On Diff #527218)

I've removed this since we won't actually always want these functions preserved. I've changed the linker to now put a tombstone in the async section instead. I also added a new test for this.

[brendandahl]

In D150803#4389062, @sunfish wrote:

In D150803#4389040, @dschuff wrote:

Hm, this is interesting because in the long term we plan to have stack switching in wasm, which could allow for similar async behavior that JSPI has, and could be useful in non-web systems. But that's a ways off. The file format we are generating with this CL will be used in emscripten sooner (and we may want to try to stabilize it some point, possibly before pure wasm stack switching is usable in non-web systems).
So overall I kind of feel like I could go either way on this. Curious if @sunfish has had any thoughts about async outside of emscripten.

When we do have stack switching, would you anticipate still using this wasm_async attribute, or would we switch to something else at that point?

It's unclear to me at the moment how we're going to implement full stack switching with emscripten/llvm. I'm not attached to the wasm_async name, so we could be more specific and use wasm_jspi or something else?

[sunfish]

wasm_jspi works for me.

[sbc100]

In D150803#4401061, @sunfish wrote:

wasm_jspi works for me.

The problem with that is that we will also likely want to use this attribute for asyncify (which is the current async approach we have today) as well as JSPI (which is hopefully coming soon)

[sunfish]

In D150803#4401204, @sbc100 wrote:

In D150803#4401061, @sunfish wrote:

wasm_jspi works for me.

The problem with that is that we will also likely want to use this attribute for asyncify (which is the current async approach we have today) as well as JSPI (which is hopefully coming soon)

Ah, I see. Would it be accurate to say that asyncify is acting as a kind of a polyfill for JSPI in this situation, such that wasm_jspi is actually correct?

Alternatively, what would you think about wasm_external_async?

[sbc100]

In D150803#4401552, @sunfish wrote:

In D150803#4401204, @sbc100 wrote:

In D150803#4401061, @sunfish wrote:

wasm_jspi works for me.

The problem with that is that we will also likely want to use this attribute for asyncify (which is the current async approach we have today) as well as JSPI (which is hopefully coming soon)

Ah, I see. Would it be accurate to say that asyncify is acting as a kind of a polyfill for JSPI in this situation, such that wasm_jspi is actually correct?

Sure, that seems reasonable to see that way. I'm ok with that name if others are.

Alternatively, what would you think about wasm_external_async?

[aaron.ballman]

Aside from the suggested changes, the Clang bits LGTM

clang/lib/Sema/SemaDeclAttr.cpp

7636–7639 ↗

(On Diff #529028)

This can be removed entirely (see edit below).

8747 ↗

(On Diff #529028)

[brendandahl]

After some feedback, a few people have indicated they'd like to do something similar to this, but for their own attributes. I've changed to a more generic attribute that allows arbitrary strings. Now a custom section will be created for each unique attribute.

[brendandahl]

Fix few remaining issues.

[dschuff]

backend code looks pretty good. I'll defer to the clang experts for the clang part.

lld/test/wasm/custom-undefine.s

17 ↗

(On Diff #531014)

I don't fully understand how this test is different from custom-attr.s. Bar is still defined, so the only thing I can see that's different is that it's not called. Is it relying on the linker GCing bar so that it's not defined? is the result different when bar is really undefined?

lld/test/wasm/merge-custom-attr-section.ll

1 ↗

(On Diff #531014)

wow I didn't know this utility existed 🤯

7 ↗

(On Diff #531014)

[dschuff]

oh also: please add the new relocation type to the document at https://github.com/WebAssembly/tool-conventions/blob/main/Linking.md

[sbc100]

Nice!

lld/test/wasm/merge-custom-attr-section.ll

47 ↗

(On Diff #531014)

Is there some reason we can't use assembly for this test?

lld/wasm/InputChunks.cpp

522

Can we switch the logic around so that the default return 0; is that last line of the function. Then your new use case could be added right before that line.

e.g.

if (name.startswith(".debug_"))
   return UINT64_C(-1)

llvm/lib/Target/WebAssembly/WebAssemblyAsmPrinter.cpp

570

Are these two lines needed? i.e. does the subscript operator automatically create an empty element when its first used?

[sunfish]

Would it make sense to call this "jspi" in practice, rather than "async"? Even though this isn't clang's decision per se anymore, it seems like the reasoning earlier in this review would still apply.

[sbc100]

In D150803#4419370, @sunfish wrote:

Would it make sense to call this "jspi" in practice, rather than "async"? Even though this isn't clang's decision per se anymore, it seems like the reasoning earlier in this review would still apply.

Did you mean to comment on the old PR? This new PR doesn't propose either "jspi" or "async", but adds that ability to define custom attributes.. since that was deemed more flexible and forward thinking. The decision as to which name emscripten will use for what can then be part of different discussion/PR we hope.

[brendandahl]

lld/test/wasm/custom-undefine.s

17 ↗

(On Diff #531014)

Undefined is a bad name. I'll change to unreferenced or something. This tests when the function isn't referenced a tombstone will be placed in the custom section (e.g. the payload below has a FFFFFFFF).

lld/test/wasm/merge-custom-attr-section.ll

47 ↗

(On Diff #531014)

Nope, I rewrote the other two, but missed this one. I'll fix.

[brendandahl]

Review comments.

[sunfish]

In D150803#4419436, @sbc100 wrote:

Did you mean to comment on the old PR? This new PR doesn't propose either "jspi" or "async", but adds that ability to define custom attributes.. since that was deemed more flexible and forward thinking. The decision as to which name emscripten will use for what can then be part of different discussion/PR we hope.

Ah, I think I somehow was looking at the old PR, and thought there was a test with custom_section.func_attr.async in it.

This patch looks ok, though my initial impression is that if we're going to have a fully general-purpose mechanism like this, we might want to plan ahead for attributes that have parameters. We don't need to implement it now though; if there's a reasonable way to evolve in that direction, then this LGTM.

[brendandahl]

@aaron.ballman or @erichkeane Did you want to re-review after that latest changes (more generic attribute) or are things good to go?

[aaron.ballman]

clang/include/clang/Basic/AttrDocs.td

5608–5612 ↗

(On Diff #525348)

Based on the documentation here, I'm wondering why the annotate attribute doesn't suffice? That attribute lets you specify custom information to associate with a declaration that then gets lowered such that passes can do whatever they want with the info, which seems to be a more generalized version of what this attribute is.

(FWIW, I'm back to having basically no idea when you'd use this attribute or what it would be used for, so my thoughts above might make no sense.)

[brendandahl]

clang/include/clang/Basic/AttrDocs.td

5608–5612 ↗

(On Diff #525348)

I was considering that, but it would require more machinery in the wasm backend to store all the attribute values in the output. For now we only really need a flag associated with function. I think if we find more uses for the annotations in the future we could replace wasm_custom with it.

[aaron.ballman]

clang/include/clang/Basic/AttrDocs.td

5608–5612 ↗

(On Diff #525348)

I was considering that, but it would require more machinery in the wasm backend to store all the attribute values in the output. For now we only really need a flag associated with function. I think if we find more uses for the annotations in the future we could replace wasm_custom with it.

More machinery in the backend is preferred to exposing a new attribute that is this general-purpose; the backend is what needs this functionality, the frontend basically does nothing with it. (I'm assuming this is an implementation detail attribute and not something you expect users to write. If I'm wrong about that, please let me know.)

[aaron.ballman]

Marking as requested changes so it's clear there's more worth discussing, so we don't accidentally land this.

[sbc100]

clang/include/clang/Basic/AttrDocs.td

5608–5612 ↗

(On Diff #525348)

Based on the documentation here, I'm wondering why the annotate attribute doesn't suffice?

I believe that was the original solution that was considered but Benden was told this was perhaps not an appropriate use of "annotate". Brenden can you elaborate on why? IIRC it was something like "the semantics of the program should not depend on annotate attributes but the attribute being added in this case is required for the program to run correctly".. is that about right?

FWIW, I think using the existing annotate attribute would make a lot of sense... if we can get away with it.

5608–5612 ↗

(On Diff #525348)

(I'm assuming this is an implementation detail attribute and not something you expect users to write. If I'm wrong about that, please let me know.)

IIUC user would indeed be specifying these attributes. Kind of like they do for "visibility" today. The initial attribute that motivated this change is "async" which tells the host runtime that a certain function import or export behaves in an async manor (See https://v8.dev/blog/jspi for more details).

[sbc100]

clang/include/clang/Basic/AttrDocs.td

5608–5612 ↗

(On Diff #525348)

I was considering that, but it would require more machinery in the wasm backend to store all the attribute values in the output. For now we only really need a flag associated with function. I think if we find more uses for the annotations in the future we could replace wasm_custom with it.

More machinery in the backend is preferred to exposing a new attribute that is this general-purpose; the backend is what needs this functionality, the frontend basically does nothing with it. (I'm assuming this is an implementation detail attribute and not something you expect users to write. If I'm wrong about that, please let me know.)

I think perhaps there are two alternative implementations being proposed here, and I want to make sure we don't confuse them:

1. Use the existing annotate attribute.
2. Make an even more complex custom attribute that hold key=value pairs rather than the current CL which proposes simply boolean custom attributes (e.g. key=true).

I think we would be happy with (1) if this usage is within scope.

IIUC the thing that would take a lot more work in the backend (and more complex custom section design in the binary format) is (2). I don't think we need (2) today and we should probably wait until we have a use case before designed a more complex version of this attribute.

[brendandahl]

clang/include/clang/Basic/AttrDocs.td

5608–5612 ↗

(On Diff #525348)

(1) could also be more complex since annotate takes a name and optional additional arguments e.g. __attribute__((annotate("x", "arg1", "arg2"))). I suppose we could just handle the case where there's a name for now and effectively do what this patch is currently doing.

5608–5612 ↗

(On Diff #525348)

It's been awhile, but I think I originally didn't go with annotate since 1) we thought it shouldn't affect compilation (in practice that doesn't seem to be true anymore) 2) It was more complex (e.g. args mentioned above). I'll give using annotate with a single attribute a try.

[brendandahl]

Use the annotate attribute to generate custom sections.

[brendandahl]

In D150803#4440802, @aaron.ballman wrote:

Marking as requested changes so it's clear there's more worth discussing, so we don't accidentally land this.

I've switched to using annotate now. Let me know if there's anything else.

[sbc100]

In D150803#4454164, @brendandahl wrote:

In D150803#4440802, @aaron.ballman wrote:

Marking as requested changes so it's clear there's more worth discussing, so we don't accidentally land this.

I've switched to using annotate now. Let me know if there's anything else.

I guess the CL title and description now need updating? Nice to see we no longer need any clang changes here!

[brendandahl]

llvm/lib/Target/WebAssembly/WebAssemblyAsmPrinter.cpp

585–598

This currently supports arguments, but it's somewhat brittle since the arguments aren't escaped. I'm fine with dropping this and only supporting name, but I think this could be useful.

[sbc100]

llvm/lib/Target/WebAssembly/WebAssemblyAsmPrinter.cpp

585–598

I'd suggest going for the simplified version.. and just ignore any annotations that have arguments. My rational here is that we shouldn't add stuff that we don't actually need yet, and also that the code annotation proposal might supersede this mechanism in the future.

llvm/test/MC/WebAssembly/func-attr.s

22

Should these be llvm.func_attr.custom0 perhaps?

I also wonder if its makes sense to preserve all of these. I worry that it could result in production code that contains these extra sections. Can we figure out way to drop them by default on only preserve them in the cases we care about?

How about maybe a linker flag? Something like --preserve-attributes=async.. that emcc could inject?

[brendandahl]

Remove annotate arguments. Change name from func_attr to llvm.func_attr.

[brendandahl]

llvm/test/MC/WebAssembly/func-attr.s

22

I'm planning to have binaryen remove all of these by default, since they're only needed as hints for binaryen.

[aaron.ballman]

In D150803#4454188, @sbc100 wrote:

In D150803#4454164, @brendandahl wrote:

In D150803#4440802, @aaron.ballman wrote:

Marking as requested changes so it's clear there's more worth discussing, so we don't accidentally land this.

I've switched to using annotate now. Let me know if there's anything else.

I guess the CL title and description now need updating? Nice to see we no longer need any clang changes here!

FWIW, I like this approach! I'm going to resign as a reviewer so that my "requested changes" marking no longer matters, but I'll still be subscribed to the review if folks need me to weigh in on anything for some reason.

[sbc100]

lgtm. Can you update the change description now that we decided to not support values?

(Ironically I think have a use for values in pipeline now)

[brendandahl]

Rebase on main.

[MaskRay]

llvm/lib/Target/WebAssembly/WebAssemblyAsmPrinter.cpp

568

"StringMap iteration order, however, is not guaranteed to be deterministic, so any uses which require that should instead use a std::map." https://llvm.org/docs/ProgrammersManual.html#llvm-adt-stringmap-h

I'll fix this in one minute.

lld/test/wasm/func-attr-tombstone.s

+# RUN: llvm-mc -filetype=obj -triple=wasm32-unknown-unknown -o %t.o %s
+# RUN: wasm-ld %t.o -o %t.wasm
+# RUN: obj2yaml %t.wasm | FileCheck %s
+
+        .functype       foo () -> ()
+        .functype       bar () -> ()
+        .functype       _start () -> ()
+        .globl  foo
+        .type   foo,@function
+foo:
+        .functype       foo () -> ()
+        end_function
+
+        .section        .text.bar,"",@
+        .globl  bar
+        .type   bar,@function
+bar:
+        .functype       bar () -> ()
+        end_function
+
+        .globl  _start
+        .type   _start,@function
+_start:
+        .functype       _start () -> ()
+        call    foo
+        end_function
+
+        .section        .custom_section.llvm.func_attr.custom0,"",@
+        .int32  foo@FUNCINDEX
+        .int32  bar@FUNCINDEX
+
+# CHECK:       - Type:            CUSTOM
+# CHECK-NEXT:    Name:            llvm.func_attr.custom0
+# CHECK-NEXT:    Payload:         00000000FFFFFFFF

lld/test/wasm/func-attr.s

+# RUN: llvm-mc -filetype=obj -triple=wasm32-unknown-unknown -o %t.o %s
+# RUN: wasm-ld %t.o -o %t.wasm
+# RUN: obj2yaml %t.wasm | FileCheck %s
+
+        .functype       foo () -> ()
+        .functype       bar () -> ()
+        .functype       _start () -> ()
+        .globl  foo
+        .type   foo,@function
+foo:
+        .functype       foo () -> ()
+        end_function
+
+        .section        .text.bar,"",@
+        .globl  bar
+        .type   bar,@function
+bar:
+        .functype       bar () -> ()
+        end_function
+
+        .globl  _start
+        .type   _start,@function
+_start:
+        .functype       _start () -> ()
+        call    foo
+        call    bar
+        end_function
+
+        .section        .custom_section.llvm.func_attr.custom0,"",@
+        .int32  foo@FUNCINDEX
+        .int32  bar@FUNCINDEX
+
+# CHECK:       - Type:            CUSTOM
+# CHECK-NEXT:    Name:            llvm.func_attr.custom0
+# CHECK-NEXT:    Payload:         '0000000001000000'

lld/test/wasm/merge-func-attr-section.s

+# RUN: split-file %s %t
+# RUN: llvm-mc -filetype=obj -triple=wasm32-unknown-unknown %t/a.s -o %t1.o
+# RUN: llvm-mc -filetype=obj -triple=wasm32-unknown-unknown %t/b.s -o %t2.o
+# RUN: wasm-ld -o %t.wasm %t1.o %t2.o
+# RUN: obj2yaml %t.wasm | FileCheck %s
+
+# Ensure two custom funct_attr sections are concatenated together.
+
+# CHECK:       - Type:            CUSTOM
+# CHECK-NEXT:    Name:            llvm.func_attr.custom0
+# CHECK-NEXT:    Payload:         '000000000100000003000000'
+
+#--- a.s
+
+# Function index 3 (after linking)
+        .functype       baz () -> ()
+# Function index 1
+        .functype       foo () -> ()
+# Function index 2
+        .functype       bar () -> ()
+        .functype       _start () -> ()
+        .globl  foo
+        .type   foo,@function
+foo:
+        .functype       foo () -> ()
+        end_function
+
+        .globl  bar
+        .type   bar,@function
+bar:
+        .functype       bar () -> ()
+        end_function
+
+        .globl  _start
+        .type   _start,@function
+_start:
+        .functype       _start () -> ()
+        call    foo
+        call    bar
+        call    baz
+        end_function
+
+        .section        .custom_section.llvm.func_attr.custom0,"",@
+        .int32  foo@FUNCINDEX
+        .int32  bar@FUNCINDEX
+
+#--- b.s
+        .functype       baz () -> ()
+        .globl  baz
+        .type   baz,@function
+baz:
+        .functype       baz () -> ()
+        end_function
+
+        .section        .custom_section.llvm.func_attr.custom0,"",@
+        .int32  baz@FUNCINDEX

lld/wasm/InputChunks.cpp

for (const WasmRelocation &rel : relocations) {
     case R_WASM_MEMORY_ADDR_SLEB64:
     case R_WASM_MEMORY_ADDR_REL_SLEB64:
     case R_WASM_MEMORY_ADDR_TLS_SLEB64:
       encodeSLEB128(static_cast<int64_t>(value), loc, 10);
       break;
     case R_WASM_TABLE_INDEX_I32:
     case R_WASM_MEMORY_ADDR_I32:
     case R_WASM_FUNCTION_OFFSET_I32:
+    case R_WASM_FUNCTION_INDEX_I32:
     case R_WASM_SECTION_OFFSET_I32:
     case R_WASM_GLOBAL_INDEX_I32:
     case R_WASM_MEMORY_ADDR_LOCREL_I32:
       write32le(loc, value);
       break;
     case R_WASM_TABLE_INDEX_I64:
     case R_WASM_MEMORY_ADDR_I64:
     case R_WASM_FUNCTION_OFFSET_I64:
for (size_t i = 0, e = sec->pieces.size(); i != e; ++i)
         sec->pieces[i].outputOff = builder.getOffset(sec->getData(i));
 }
 
 uint64_t InputSection::getTombstoneForSection(StringRef name) {
   // When a function is not live we need to update relocations referring to it.
   // If they occur in DWARF debug symbols, we want to change the pc of the
   // function to -1 to avoid overlapping with a valid range. However for the
   // debug_ranges and debug_loc sections that would conflict with the existing
-  // meaning of -1 so we use -2.
-  // Returning 0 means there is no tombstone value for this section, and relocation
-  // will just use the addend.
-  if (!name.starts_with(".debug_"))
-    return 0;
+  // meaning of -1 so we use -2.  
   if (name.equals(".debug_ranges") || name.equals(".debug_loc"))
     return UINT64_C(-2);
+  if (name.starts_with(".debug_"))
+    return UINT64_C(-1);
+  // If the function occurs in an function attribute section change it to -1 since
+  // 0 is a valid function index.
+  if (name.starts_with("llvm.func_attr."))
     return UINT64_C(-1);
+  // Returning 0 means there is no tombstone value for this section, and relocation
+  // will just use the addend.
+  return 0;
 }
 
 } // namespace wasm
 } // namespace lld

lld/wasm/InputFiles.cpp

if (reloc.Type == R_WASM_MEMORY_ADDR_LOCREL_I32) {
                    reloc.Offset - segment->getInputSectionOffset();
       value -= p;
     }
     return value;
   }
   case R_WASM_TYPE_INDEX_LEB:
     return typeMap[reloc.Index];
   case R_WASM_FUNCTION_INDEX_LEB:
+  case R_WASM_FUNCTION_INDEX_I32:
     return getFunctionSymbol(reloc.Index)->getFunctionIndex();
   case R_WASM_GLOBAL_INDEX_LEB:
   case R_WASM_GLOBAL_INDEX_I32:
     if (auto gs = dyn_cast<GlobalSymbol>(sym))
       return gs->getGlobalIndex();
     return sym->getGOTIndex();
   case R_WASM_TAG_INDEX_LEB:
     return getTagSymbol(reloc.Index)->getTagIndex();

llvm/docs/ReleaseNotes.rst

* Assembly support was added for the experimental Zfbfmin (scalar BF16
   conversions), Zvfbfmin (vector BF16 conversions), and Zvfbfwma (vector BF16
   widening mul-add) extensions.
 * Added assembler/disassembler support for the experimental Zacas (atomic
   compare-and-swap) extension.
 
 Changes to the WebAssembly Backend
 ----------------------------------
 
-* ...
+* Function annotations (``__attribute__((annotate(<name>)))``)
+  now generate custom sections in the Wasm output file. A custom section
+  for each unique name will be created that contains each function
+  index the annotation applies to.
 
 Changes to the Windows Target
 -----------------------------
 
 Changes to the X86 Backend
 --------------------------
 
 * ``__builtin_unpredictable`` (unpredictable metadata in LLVM IR), is handled by X86 Backend.

llvm/include/llvm/BinaryFormat/WasmRelocs.def

 WASM_RELOC(R_WASM_TABLE_INDEX_SLEB64,     18)
 WASM_RELOC(R_WASM_TABLE_INDEX_I64,        19)
 WASM_RELOC(R_WASM_TABLE_NUMBER_LEB,       20)
 WASM_RELOC(R_WASM_MEMORY_ADDR_TLS_SLEB,   21)
 WASM_RELOC(R_WASM_FUNCTION_OFFSET_I64,    22)
 WASM_RELOC(R_WASM_MEMORY_ADDR_LOCREL_I32, 23)
 WASM_RELOC(R_WASM_TABLE_INDEX_REL_SLEB64, 24)
 WASM_RELOC(R_WASM_MEMORY_ADDR_TLS_SLEB64, 25)
+WASM_RELOC(R_WASM_FUNCTION_INDEX_I32,     26)

llvm/include/llvm/MC/MCExpr.h

enum VariantKind : uint16_t {
     VK_Hexagon_IE,
     VK_Hexagon_IE_GOT,
 
     VK_WASM_TYPEINDEX, // Reference to a symbol's type (signature)
     VK_WASM_TLSREL,    // Memory address relative to __tls_base
     VK_WASM_MBREL,     // Memory address relative to __memory_base
     VK_WASM_TBREL,     // Table index relative to __table_base
     VK_WASM_GOT_TLS,   // Wasm global index of TLS symbol.
+    VK_WASM_FUNCINDEX, // Wasm function index.
 
     VK_AMDGPU_GOTPCREL32_LO, // symbol@gotpcrel32@lo
     VK_AMDGPU_GOTPCREL32_HI, // symbol@gotpcrel32@hi
     VK_AMDGPU_REL32_LO,      // symbol@rel32@lo
     VK_AMDGPU_REL32_HI,      // symbol@rel32@hi
     VK_AMDGPU_REL64,         // symbol@rel64
     VK_AMDGPU_ABS32_LO,      // symbol@abs32@lo
     VK_AMDGPU_ABS32_HI,      // symbol@abs32@hi

llvm/lib/MC/MCExpr.cpp

StringRef MCSymbolRefExpr::getVariantKindName(VariantKind Kind) {
   case VK_Hexagon_LD_PLT: return "LDPLT";
   case VK_Hexagon_IE: return "IE";
   case VK_Hexagon_IE_GOT: return "IEGOT";
   case VK_WASM_TYPEINDEX: return "TYPEINDEX";
   case VK_WASM_MBREL: return "MBREL";
   case VK_WASM_TLSREL: return "TLSREL";
   case VK_WASM_TBREL: return "TBREL";
   case VK_WASM_GOT_TLS: return "GOT@TLS";
+  case VK_WASM_FUNCINDEX: return "FUNCINDEX";
   case VK_AMDGPU_GOTPCREL32_LO: return "gotpcrel32@lo";
   case VK_AMDGPU_GOTPCREL32_HI: return "gotpcrel32@hi";
   case VK_AMDGPU_REL32_LO: return "rel32@lo";
   case VK_AMDGPU_REL32_HI: return "rel32@hi";
   case VK_AMDGPU_REL64: return "rel64";
   case VK_AMDGPU_ABS32_LO: return "abs32@lo";
   case VK_AMDGPU_ABS32_HI: return "abs32@hi";
   case VK_VE_HI32: return "hi";
   case VK_VE_LO32: return "lo";
return StringSwitch<VariantKind>(Name.lower())
     .Case("lo8", VK_AVR_LO8)
     .Case("hi8", VK_AVR_HI8)
     .Case("hlo8", VK_AVR_HLO8)
     .Case("typeindex", VK_WASM_TYPEINDEX)
     .Case("tbrel", VK_WASM_TBREL)
     .Case("mbrel", VK_WASM_MBREL)
     .Case("tlsrel", VK_WASM_TLSREL)
     .Case("got@tls", VK_WASM_GOT_TLS)
+    .Case("funcindex", VK_WASM_FUNCINDEX)
     .Case("gotpcrel32@lo", VK_AMDGPU_GOTPCREL32_LO)
     .Case("gotpcrel32@hi", VK_AMDGPU_GOTPCREL32_HI)
     .Case("rel32@lo", VK_AMDGPU_REL32_LO)
     .Case("rel32@hi", VK_AMDGPU_REL32_HI)
     .Case("rel64", VK_AMDGPU_REL64)
     .Case("abs32@lo", VK_AMDGPU_ABS32_LO)
     .Case("abs32@hi", VK_AMDGPU_ABS32_HI)
     .Case("hi", VK_VE_HI32)
if (SA.isVariable() || SB.isVariable() ||
       return;
 
     // Try to find a constant displacement from FA to FB, add the displacement
     // between the offset in FA of SA and the offset in FB of SB.
     bool Reverse = false;
     if (FA == FB) {
       Reverse = SA.getOffset() < SB.getOffset();
     } else if (!isa<MCDummyFragment>(FA)) {
       Reverse = std::find_if(std::next(FA->getIterator()), SecA.end(),
                              [&](auto &I) { return &I == FB; }) != SecA.end();
     }
 
     uint64_t SAOffset = SA.getOffset(), SBOffset = SB.getOffset();
     int64_t Displacement = SA.getOffset() - SB.getOffset();
     if (Reverse) {
       std::swap(FA, FB);
       std::swap(SAOffset, SBOffset);

llvm/lib/MC/WasmObjectWriter.cpp

if (RelEntry.Type == wasm::R_WASM_TABLE_INDEX_REL_SLEB ||
       return TableIndices[Base] - InitialTableOffset;
     else
       return TableIndices[Base];
   }
   case wasm::R_WASM_TYPE_INDEX_LEB:
     // Provisional value is same as the index
     return getRelocationIndexValue(RelEntry);
   case wasm::R_WASM_FUNCTION_INDEX_LEB:
+  case wasm::R_WASM_FUNCTION_INDEX_I32:
   case wasm::R_WASM_GLOBAL_INDEX_LEB:
   case wasm::R_WASM_GLOBAL_INDEX_I32:
   case wasm::R_WASM_TAG_INDEX_LEB:
   case wasm::R_WASM_TABLE_NUMBER_LEB:
     // Provisional value is function/global/tag Wasm index
     assert(WasmIndices.count(RelEntry.Symbol) > 0 && "symbol not found in wasm index space");
     return WasmIndices[RelEntry.Symbol];
   case wasm::R_WASM_FUNCTION_OFFSET_I32:
case wasm::R_WASM_TABLE_NUMBER_LEB:
       writePatchableU32(Stream, Value, Offset);
       break;
     case wasm::R_WASM_MEMORY_ADDR_LEB64:
       writePatchableU64(Stream, Value, Offset);
       break;
     case wasm::R_WASM_TABLE_INDEX_I32:
     case wasm::R_WASM_MEMORY_ADDR_I32:
     case wasm::R_WASM_FUNCTION_OFFSET_I32:
+    case wasm::R_WASM_FUNCTION_INDEX_I32:
     case wasm::R_WASM_SECTION_OFFSET_I32:
     case wasm::R_WASM_GLOBAL_INDEX_I32:
     case wasm::R_WASM_MEMORY_ADDR_LOCREL_I32:
       patchI32(Stream, Value, Offset);
       break;
     case wasm::R_WASM_TABLE_INDEX_I64:
     case wasm::R_WASM_MEMORY_ADDR_I64:
     case wasm::R_WASM_FUNCTION_OFFSET_I64:

llvm/lib/Object/WasmObjectFile.cpp

while (RelocCount--) {
     Reloc.Offset = readVaruint32(Ctx);
     if (Reloc.Offset < PreviousOffset)
       return make_error<GenericBinaryError>("relocations not in offset order",
                                             object_error::parse_failed);
     PreviousOffset = Reloc.Offset;
     Reloc.Index = readVaruint32(Ctx);
     switch (type) {
     case wasm::R_WASM_FUNCTION_INDEX_LEB:
+    case wasm::R_WASM_FUNCTION_INDEX_I32:
     case wasm::R_WASM_TABLE_INDEX_SLEB:
     case wasm::R_WASM_TABLE_INDEX_SLEB64:
     case wasm::R_WASM_TABLE_INDEX_I32:
     case wasm::R_WASM_TABLE_INDEX_I64:
     case wasm::R_WASM_TABLE_INDEX_REL_SLEB:
     case wasm::R_WASM_TABLE_INDEX_REL_SLEB64:
       if (!isValidFunctionSymbol(Reloc.Index))
         return make_error<GenericBinaryError>(
if (Reloc.Type == wasm::R_WASM_MEMORY_ADDR_LEB64 ||
         Reloc.Type == wasm::R_WASM_MEMORY_ADDR_SLEB64 ||
         Reloc.Type == wasm::R_WASM_MEMORY_ADDR_REL_SLEB64)
       Size = 10;
     if (Reloc.Type == wasm::R_WASM_TABLE_INDEX_I32 ||
         Reloc.Type == wasm::R_WASM_MEMORY_ADDR_I32 ||
         Reloc.Type == wasm::R_WASM_MEMORY_ADDR_LOCREL_I32 ||
         Reloc.Type == wasm::R_WASM_SECTION_OFFSET_I32 ||
         Reloc.Type == wasm::R_WASM_FUNCTION_OFFSET_I32 ||
+        Reloc.Type == wasm::R_WASM_FUNCTION_INDEX_I32 ||
         Reloc.Type == wasm::R_WASM_GLOBAL_INDEX_I32)
       Size = 4;
     if (Reloc.Type == wasm::R_WASM_TABLE_INDEX_I64 ||
         Reloc.Type == wasm::R_WASM_MEMORY_ADDR_I64 ||
         Reloc.Type == wasm::R_WASM_FUNCTION_OFFSET_I64)
       Size = 8;
     if (Reloc.Offset + Size > EndOffset)
       return make_error<GenericBinaryError>("invalid relocation offset",

llvm/lib/Target/WebAssembly/MCTargetDesc/WebAssemblyWasmObjectWriter.cpp

switch (Modifier) {
     case MCSymbolRefExpr::VK_WASM_MBREL:
       assert(SymA.isData());
       return is64Bit() ? wasm::R_WASM_MEMORY_ADDR_REL_SLEB64
                        : wasm::R_WASM_MEMORY_ADDR_REL_SLEB;
     case MCSymbolRefExpr::VK_WASM_TYPEINDEX:
       return wasm::R_WASM_TYPE_INDEX_LEB;
     case MCSymbolRefExpr::VK_None:
       break;
+    case MCSymbolRefExpr::VK_WASM_FUNCINDEX:
+      return wasm::R_WASM_FUNCTION_INDEX_I32;
     default:
       report_fatal_error("unknown VariantKind");
       break;
   }
 
   switch (unsigned(Fixup.getKind())) {
   case WebAssembly::fixup_sleb128_i32:
     if (SymA.isFunction())

llvm/lib/Target/WebAssembly/WebAssemblyAsmPrinter.h

public:
 
   //===------------------------------------------------------------------===//
   // AsmPrinter Implementation.
   //===------------------------------------------------------------------===//
 
   void emitEndOfAsmFile(Module &M) override;
   void EmitProducerInfo(Module &M);
   void EmitTargetFeatures(Module &M);
+  void EmitFunctionAttributes(Module &M);
   void emitSymbolType(const MCSymbolWasm *Sym);
   void emitGlobalVariable(const GlobalVariable *GV) override;
   void emitJumpTableInfo() override;
   void emitConstantPool() override;
   void emitFunctionBodyStart() override;
   void emitInstruction(const MachineInstr *MI) override;
   bool PrintAsmOperand(const MachineInstr *MI, unsigned OpNo,
                        const char *ExtraCode, raw_ostream &OS) override;

llvm/lib/Target/WebAssembly/WebAssemblyAsmPrinter.cpp

 #include "WebAssembly.h"
 #include "WebAssemblyMCInstLower.h"
 #include "WebAssemblyMachineFunctionInfo.h"
 #include "WebAssemblyRegisterInfo.h"
 #include "WebAssemblyRuntimeLibcallSignatures.h"
 #include "WebAssemblyTargetMachine.h"
 #include "llvm/ADT/SmallSet.h"
 #include "llvm/ADT/StringExtras.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/Analysis/ValueTracking.h"
 #include "llvm/BinaryFormat/Wasm.h"
 #include "llvm/CodeGen/Analysis.h"
 #include "llvm/CodeGen/AsmPrinter.h"
 #include "llvm/CodeGen/MachineConstantPool.h"
 #include "llvm/CodeGen/MachineInstr.h"
 #include "llvm/CodeGen/MachineModuleInfoImpls.h"
 #include "llvm/IR/DataLayout.h"
 #include "llvm/IR/DebugInfoMetadata.h"
for (const Metadata *MD : Named->operands()) {
       OutStreamer->switchSection(MySection);
       OutStreamer->emitBytes(Contents->getString());
       OutStreamer->popSection();
     }
   }
 
   EmitProducerInfo(M);
   EmitTargetFeatures(M);
+  EmitFunctionAttributes(M);
 }
 
 void WebAssemblyAsmPrinter::EmitProducerInfo(Module &M) {
   llvm::SmallVector<std::pair<std::string, std::string>, 4> Languages;
   if (const NamedMDNode *Debug = M.getNamedMetadata("llvm.dbg.cu")) {
     llvm::SmallSet<StringRef, 4> SeenLanguages;
     for (size_t I = 0, E = Debug->getNumOperands(); I < E; ++I) {
       const auto *CU = cast<DICompileUnit>(Debug->getOperand(I));
for (auto &F : EmittedFeatures) {
     OutStreamer->emitIntValue(F.Prefix, 1);
     OutStreamer->emitULEB128IntValue(F.Name.size());
     OutStreamer->emitBytes(F.Name);
   }
 
   OutStreamer->popSection();
 }
 
+void WebAssemblyAsmPrinter::EmitFunctionAttributes(Module &M) {
+  auto V = M.getNamedGlobal("llvm.global.annotations");
+  if (!V)
+    return;
+
+  // Group all the custom attributes by name.
+  StringMap<SmallVector<MCSymbol *, 4>> CustomSections;
+  const ConstantArray *CA = cast<ConstantArray>(V->getOperand(0));
+  for (Value *Op : CA->operands()) {
+    auto *CS = cast<ConstantStruct>(Op);
+    // The first field is a pointer to the annotated variable.
+    Value *AnnotatedVar = CS->getOperand(0)->stripPointerCasts();
+    // Only annotated functions are supported for now.
+    if (!isa<Function>(AnnotatedVar))
+      continue;
+    auto *F = cast<Function>(AnnotatedVar);
+
+    // The second field is a pointer to a global annotation string.
+    auto *GV = cast<GlobalVariable>(CS->getOperand(1)->stripPointerCasts());
+    StringRef AnnotationString;
+    getConstantStringInfo(GV, AnnotationString);
+    std::string Name = "annotate." + AnnotationString.str();
+    auto *Sym = cast<MCSymbolWasm>(getSymbol(F));
+    CustomSections[Name].push_back(Sym);
+  }
+
+  // Emit a custom section for each unique attribute.
+  for (const auto &[Name, Symbols] : CustomSections) {
+    MCSectionWasm *CustomSection = OutContext.getWasmSection(
+        ".custom_section.llvm.func_attr." + Name, SectionKind::getMetadata());
+    OutStreamer->pushSection();
+    OutStreamer->switchSection(CustomSection);
+
+    for (auto &Sym : Symbols) {
+      OutStreamer->emitValue(
+          MCSymbolRefExpr::create(Sym, MCSymbolRefExpr::VK_WASM_FUNCINDEX,
+                                  OutContext),
+          4);
+    }
+    OutStreamer->popSection();
+  }
+}
+
 void WebAssemblyAsmPrinter::emitConstantPool() {
   emitDecls(*MMI->getModule());
   assert(MF->getConstantPool()->getConstants().empty() &&
          "WebAssembly disables constant pools");
 }
 
 void WebAssemblyAsmPrinter::emitJumpTableInfo() {
   // Nothing to do; jump tables are incorporated into the instruction stream.

llvm/test/CodeGen/WebAssembly/func-attr-annotate.ll

+; RUN: llc < %s -asm-verbose=false -wasm-keep-registers | FileCheck %s
+
+target triple = "wasm32-unknown-unknown"
+
+@.str = private unnamed_addr constant [8 x i8] c"custom0\00", section "llvm.metadata"
+@.str.1 = private unnamed_addr constant [7 x i8] c"main.c\00", section "llvm.metadata"
+@.str.2 = private unnamed_addr constant [8 x i8] c"custom1\00", section "llvm.metadata"
+@.str.3 = private unnamed_addr constant [8 x i8] c"custom2\00", section "llvm.metadata"
[e-mail] = appending global [3 x { ptr, ptr, ptr, i32, ptr }] [{ ptr, ptr, ptr, i32, ptr } { ptr @test0, ptr @.str, ptr @.str.1, i32 4, ptr null }, { ptr, ptr, ptr, i32, ptr } { ptr @test1, ptr @.str, ptr @.str.1, i32 5, ptr null }, { ptr, ptr, ptr, i32, ptr } { ptr @test2, ptr @.str.2, ptr @.str.1, i32 6, ptr null }], section "llvm.metadata"
+
+define void @test0() {
+  ret void
+}
+
+define void @test1() {
+  ret void
+}
+
+define void @test2() {
+  ret void
+}
+
+define void @test3() {
+  ret void
+}
+
+; CHECK:      .section        .custom_section.llvm.func_attr.annotate.custom0,"",@
+; CHECK-NEXT: .int32  test0@FUNCINDEX
+; CHECK-NEXT: .int32  test1@FUNCINDEX
+; CHECK:      .section        .custom_section.llvm.func_attr.annotate.custom1,"",@
+; CHECK-NEXT: .int32  test2@FUNCINDEX

llvm/test/MC/WebAssembly/func-attr.s

+# RUN: llvm-mc -triple=wasm32-unknown-unknown < %s | FileCheck %s
+# Check that it also comiled to object for format.
+# RUN: llvm-mc -triple=wasm32-unknown-unknown -filetype=obj -o - < %s | obj2yaml | FileCheck -check-prefix=CHECK-OBJ %s
+
+foo:
+    .globl foo
+    .functype foo () -> ()
+    end_function
+
+    .section        .custom_section.llvm.func_attr.custom0,"",@
+    .int32  foo@FUNCINDEX
+
+# CHECK:       .section .custom_section.llvm.func_attr.custom0,"",@
+# CHECK-NEXT: .int32  foo@FUNCINDEX
+
+# CHECK-OBJ:        - Type:            CUSTOM
+# CHECK-OBJ-NEXT:     Relocations:
+# CHECK-OBJ-NEXT:        - Type:            R_WASM_FUNCTION_INDEX_I32
+# CHECK-OBJ-NEXT:          Index:           0
+# CHECK-OBJ-NEXT:          Offset:          0x0
+# CHECK-OBJ-NEXT:     Name:            llvm.func_attr.custom0