Page MenuHomePhabricator

[RISCV][RFC] Add Clang support for RISC-V overlay system
Needs ReviewPublic

Authored by edward-jones on Sep 7 2021, 9:59 AM.

Details

Reviewers
aaron.ballman
Summary

For a description of the riscv-overlay system proposal: https://github.com/riscv-software-src/riscv-overlay/

This is the Clang component of a change to add support for a proposed 'overlay' system for RISC-V to LLVM.

This adds two new attributes, attribute((overlaydata)), and attribute((overlaycall)), which are used to mark functions or global data as only accessible through the overlay engine. Internally they are converted to 'overlay-data' and 'overlay-call' attributes in LLVM IR.

This change also adds the option -fcomrv to enable use of the overlay system. This has the effect of reserving registers x28, x29, x30 and x31 for exclusive used by the overlay engine.

Diff Detail

Unit TestsFailed

TimeTest
50 msx64 debian > ORC-x86_64-linux.TestCases/Linux/x86-64::trivial-cxa-atexit.S
Script: -- : 'RUN: at line 3'; /var/lib/buildkite-agent/builds/llvm-project/build/./bin/clang -m64 -c -o /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/orc/X86_64LinuxConfig/TestCases/Linux/x86-64/Output/trivial-cxa-atexit.S.tmp /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/orc/TestCases/Linux/x86-64/trivial-cxa-atexit.S
60 msx64 debian > ORC-x86_64-linux.TestCases/Linux/x86-64::trivial-static-initializer.S
Script: -- : 'RUN: at line 7'; /var/lib/buildkite-agent/builds/llvm-project/build/./bin/clang -m64 -c -o /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/orc/X86_64LinuxConfig/TestCases/Linux/x86-64/Output/trivial-static-initializer.S.tmp /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/orc/TestCases/Linux/x86-64/trivial-static-initializer.S
60 msx64 debian > ORC-x86_64-linux.TestCases/Linux/x86-64::trivial-tls.S
Script: -- : 'RUN: at line 1'; /var/lib/buildkite-agent/builds/llvm-project/build/./bin/clang -m64 -c -o /var/lib/buildkite-agent/builds/llvm-project/build/projects/compiler-rt/test/orc/X86_64LinuxConfig/TestCases/Linux/x86-64/Output/trivial-tls.S.tmp /var/lib/buildkite-agent/builds/llvm-project/compiler-rt/test/orc/TestCases/Linux/x86-64/trivial-tls.S

Event Timeline

edward-jones created this revision.Sep 7 2021, 9:59 AM
edward-jones requested review of this revision.Sep 7 2021, 9:59 AM
Herald added a project: Restricted Project. · View Herald TranscriptSep 7 2021, 9:59 AM

I don't see anything giving an error if you try and use the new badly-named option for an architecture other than RISC-V (beyond the usual unused argument warning that's only an error with -Werror)?

clang/include/clang/Basic/Attr.td
348

I thought the idea for this proposal was to come up with something that wasn't RISC-V specific (an architecture-independent specification and a RISC-V-specific supplement for how that is mapped to RISC-V) to get industry buy-in?

1793

If you want this to be portable to non-GNU compilers you should consider using a keyword instead (which can still map to an attribute internally). That also tends to get you better errors (there are places where type attributes can get silently ignored currently).

clang/include/clang/Basic/AttrDocs.td
2149

Why not just a single attribute that DTRT based on the type?

clang/include/clang/Driver/Options.td
3232

What on earth is this name?

clang/test/CodeGen/riscv-overlaycall.c
1 ↗(On Diff #371111)

Use update_cc_test_checks.py

clang/test/Driver/riscv-comrv.c
3 ↗(On Diff #371111)

Don't write a driver test that requires the backend, you don't need that to test this

clang/test/Sema/riscv-overlaycall-namespace.cpp
7 ↗(On Diff #371111)

This error message is misleading. The semantics also don't seem great to me.

The name "overlay" is ambiguous. Even after I ruled out Gentoo Overlay and overlayfs, I am thinking whether this has anything to do with OVERLAY description in a linker script: https://sourceware.org/binutils/docs/ld/Overlay-Description.html#Overlay-Description

which are used to mark functions or global data as only accessible through the overlay engine

Can you give more descriptions for folks who don't follow the RISC-V side proposal but need to review your changes? :)

The name "overlay" is ambiguous. Even after I ruled out Gentoo Overlay and overlayfs, I am thinking whether this has anything to do with OVERLAY description in a linker script: https://sourceware.org/binutils/docs/ld/Overlay-Description.html#Overlay-Description

which are used to mark functions or global data as only accessible through the overlay engine

Can you give more descriptions for folks who don't follow the RISC-V side proposal but need to review your changes? :)

Basically hardware-assisted code+rodata banking (I guess either by actually banking ROMs or just paging stuff in and out) that's mostly transparent to software. Functions at the boundary of components (don't know what the granularity is) use a weird indirect calling convention where you instead call into some magic runtime with a unique ID for the callee, it ensures everything's loaded and then tail calls it for you.

aaron.ballman added inline comments.Sep 7 2021, 11:15 AM
clang/include/clang/Basic/Attr.td
1790–1792

This is not typical attribute behavior -- if the target architecture doesn't support the attribute, are the semantics such that an error is appropriate/required? Put another way, why do we want to trigger an error for this attribute while not triggering errors for other target-specific attributes (like calling conventions)?

1794

Does GCC support this attribute? If not, this should be using the Clang spelling (same below).

Also, overlaycall is pretty hard to read; why not overlay_call and overlay_data?

1795

What about Objective-C methods?

clang/include/clang/Basic/DiagnosticDriverKinds.td
36
clang/include/clang/Basic/DiagnosticSemaKinds.td
314
321
clang/lib/Driver/ToolChains/Arch/RISCV.cpp
496
clang/lib/Driver/ToolChains/Clang.cpp
2100
clang/lib/Frontend/CompilerInvocation.cpp
4066 ↗(On Diff #371111)
4461 ↗(On Diff #371111)
clang/lib/Sema/SemaDecl.cpp
3365

You should fix this formatting nit.

clang/lib/Sema/SemaDeclAttr.cpp
2043–2047

This shouldn't be necessary, the automated checking should handle it automatically.

2056–2058

Please use NoInlineAttr::CreateImplicit() instead.

8513–8518

Please write a handle function for this as with all the other attributes (we have hopes to someday do more tablegen in this area, so sticking with the same pattern in this switch is strongly preferred).

The name "overlay" is ambiguous. Even after I ruled out Gentoo Overlay and overlayfs, I am thinking whether this has anything to do with OVERLAY description in a linker script: https://sourceware.org/binutils/docs/ld/Overlay-Description.html#Overlay-Description

which are used to mark functions or global data as only accessible through the overlay engine

Can you give more descriptions for folks who don't follow the RISC-V side proposal but need to review your changes? :)

Basically hardware-assisted code+rodata banking (I guess either by actually banking ROMs or just paging stuff in and out) that's mostly transparent to software. Functions at the boundary of components (don't know what the granularity is) use a weird indirect calling convention where you instead call into some magic runtime with a unique ID for the callee, it ensures everything's loaded and then tail calls it for you.

Yes it's essentially this. The start of the proposal for this 'overlay' system for RISC-V is a FOSDEM talk 'Cacheable Overlay Manager RISC‐V'. That's also the source of the weird name for the -fcomrv option name.

Would something like -foverlay-manager make more sense? (maybe an -m option would actually be more appropriate given this is still very RISC-V specific?). I'm not sure how to disambiguate from the many overloaded meanings of 'overlay'.

Thank's for the feedback. I'll update this and come back with a tidier patch.

clang/include/clang/Basic/Attr.td
1790–1792

If the attribute is being used then the expectation is that it is being supported by an overlay engine, and if that isn't the case then it implies an error in the build setup. That said I need to check this because I'm not convinced it's necessary.

1793

I don't think much consideration has been given to other compilers, but would it be unreasonable for the interface to this feature to not necessarily be identical between GNU and non-GNU compilers?

That said, I'm happy to switch to a keyword, especially if as you mention there are cases where an attribute can silently go missing without error. I'm not sure on the distinction of when to use a keyword vs attribute though, given that keywords are used pretty sparingly in comparison.

clang/include/clang/Basic/AttrDocs.td
2149

Good point. I'll see if I can do that. The fact we used multiple attributes is mainly a consequence of how we put this together rather than any inherent technical need I think.

clang/test/Sema/riscv-overlaycall-namespace.cpp
7 ↗(On Diff #371111)

From recollection the current overlay system only works with externally visible symbols and these semantics are a consequence of that.

That said, it's not documented in the code, and as you point out the error message is just wrong so I'll fix this.

aaron.ballman added inline comments.Sep 8 2021, 6:29 AM
clang/include/clang/Basic/Attr.td
1790–1792

Thanks for looking into it! My preference is for the unknown attribute to be ignored (with diagnostic) on other targets unless there's a strong argument for why it's an error.

1793

This isn't a type attribute, so I don't think there's a risk of the attribute getting lost.

jrtc27 added inline comments.Sep 8 2021, 6:30 AM
clang/include/clang/Basic/Attr.td
1793

This one isn't, but the data one is

aaron.ballman added inline comments.Sep 8 2021, 6:38 AM
clang/include/clang/Basic/Attr.td
1793

Oh, derp, thank you for catching that, somehow I completely missed it! I was thrown off by there being a lot of problems with it, but all the problems made it look like a declaration attribute.

Type attributes come with their own fun set of questions, like: does the attribute impact the type for overloading? SFINAE? Name mangling?

1802

This subject list is incorrect -- the subject is a VarDecl, but that's not a type.

clang/include/clang/Basic/AttrDocs.td
2153

This description is wrong if this is intended to be a type attribute. A global constant is not a type.

clang/lib/Sema/SemaDeclAttr.cpp
8513–8518

Oops, this is entirely wrong -- if this is a type attribute, it should be handled from processTypeAttrs() in SemaType.cpp.

the_o added a comment.Sep 12 2021, 6:22 AM

The name "overlay" is ambiguous. Even after I ruled out Gentoo Overlay and overlayfs, I am thinking whether this has anything to do with OVERLAY description in a linker script: https://sourceware.org/binutils/docs/ld/Overlay-Description.html#Overlay-Description

which are used to mark functions or global data as only accessible through the overlay engine

Can you give more descriptions for folks who don't follow the RISC-V side proposal but need to review your changes? :)

Basically hardware-assisted code+rodata banking (I guess either by actually banking ROMs or just paging stuff in and out) that's mostly transparent to software. Functions at the boundary of components (don't know what the granularity is) use a weird indirect calling convention where you instead call into some magic runtime with a unique ID for the callee, it ensures everything's loaded and then tail calls it for you.

Yes it's essentially this. The start of the proposal for this 'overlay' system for RISC-V is a FOSDEM talk 'Cacheable Overlay Manager RISC‐V'. That's also the source of the weird name for the -fcomrv option name.

Would something like -foverlay-manager make more sense? (maybe an -m option would actually be more appropriate given this is still very RISC-V specific?). I'm not sure how to disambiguate from the many overloaded meanings of 'overlay'.

Thank's for the feedback. I'll update this and come back with a tidier patch.

Adding my two cents:
More info about Overlay can be found on the Overlay standard at 'FOSSi embedded overlay'
For the flag, on RISCV we agreed in -moveraly per the High Level Design section 2.7.1.1 - Flags

Changes here:

    • Merged the separate attributes overlaycall and overlaydata into a single overlay attribute (though I may need to undo this to match the high level document)
    • Fixed to use the correct type of attribute and handle it in a less sloppy way
    • Errors/warnings messages are hopefully more descriptive now
    • Renamed option to -moverlay to match the high level document
    • It's no longer an error to use the attribute without -moverlay, you'll just get "attribute ignored" warnings
  • Updated tests

Still to do:

  • Writing more tests
  • Warn the user to turn on -moverlay as a fix for "attribute ignored" warnings
  • Rebase

I reverted some of the previous changes I made so that this patch matches the spec as currently written - this means it's two attributes again, and the diagnostic messages have been updated a bit too. The two Clang attributes match to the same LLVM attribute internally though.

This is at a stage where more review would be nice. Obviously this is gated on patches to other toolchain components, but I hope that these changes won't change too much now unless the spec also changes.

jrtc27 added a comment.Mon, Nov 1, 4:09 AM

I reverted some of the previous changes I made so that this patch matches the spec as currently written - this means it's two attributes again, and the diagnostic messages have been updated a bit too. The two Clang attributes match to the same LLVM attribute internally though.

This is at a stage where more review would be nice. Obviously this is gated on patches to other toolchain components, but I hope that these changes won't change too much now unless the spec also changes.

The whole point of putting it up for review is so you get feedback about the entire direction of the spec, which was written by people who are not experts when it comes to toolchains. You’re supposed to take our feedback, relay it to them and get the draft spec revised. Otherwise, if the spec written by people who don’t know what makes sense for toolchains is regarded as holy and immutable then I’m just going to NACK this as poorly designed and something LLVM shouldn’t bow to implementing, and you’ve wasted my time asking for a full review.

There are still unhandled comments in the patch, btw.

clang/include/clang/Basic/Attr.td
1794

Still wondering about the naming choice.

1795

And still wondering about ObjC methods for RISCVOverlayCallAttr.

clang/include/clang/Basic/AttrDocs.td
2149

I'm confused as to why this is two attributes again. I think using a single attribute makes more sense if they're both documented to be exactly the same thing.

Also, this is missing information about the data overlay restrictions (that it be a global constant variable), and examples. Further, these docs aren't really telling me why I'd use this attribute. Are there other docs we should be linking to that describe what an overlay is, etc?

clang/lib/CodeGen/CodeGenModule.cpp
1883–1885

We should probably document that this attribute changes the function alignment.

clang/lib/Sema/SemaDeclAttr.cpp
2043–2047

This should be a LangOpt in Attr.td and not use a custom diagnostic. However, there is an interesting feature idea hiding in here for a follow-up patch, which is to generate a better diagnostic about which lang opt is required to enable the attribute.

I reverted some of the previous changes I made so that this patch matches the spec as currently written - this means it's two attributes again, and the diagnostic messages have been updated a bit too. The two Clang attributes match to the same LLVM attribute internally though.

This is at a stage where more review would be nice. Obviously this is gated on patches to other toolchain components, but I hope that these changes won't change too much now unless the spec also changes.

The whole point of putting it up for review is so you get feedback about the entire direction of the spec, which was written by people who are not experts when it comes to toolchains. You’re supposed to take our feedback, relay it to them and get the draft spec revised. Otherwise, if the spec written by people who don’t know what makes sense for toolchains is regarded as holy and immutable then I’m just going to NACK this as poorly designed and something LLVM shouldn’t bow to implementing, and you’ve wasted my time asking for a full review.

Understood, and apologies for requesting a review without first getting the spec clarified. I had relayed feedback about naming of relocs/attributes and 1 vs 2 attributes offline, but that hadn't reached a conclusion so I had reverted to closer to the spec in the meantime. I've opened issues on the riscv-overlay repository to incorporate the feedback here.

https://github.com/riscv-software-src/riscv-overlay/issues/28
https://github.com/riscv-software-src/riscv-overlay/issues/29
https://github.com/riscv-software-src/riscv-overlay/issues/30

clang/include/clang/Basic/Attr.td
1794

I'll push for an underscore in the names if we're going to have two attributes.

I'm also wondering now whether to use overlay_function instead of overlay_call since the attribute is marking that the function exists in an overlay rather than the mechanism by which it is called.

1795

Would simply disallowing this attribute to be used with ObjC be acceptable whilst getting clarification on whether overlays should be usable from ObjC?

clang/include/clang/Basic/AttrDocs.td
2149

Would it be sufficient to document based on the restrictions defined here? As far as I'm aware LLVM documentation doesn't often refer out to external documents so I assume the attribute's documentation needs to be a self-contained summary?

I'm still not sure about one vs two attribute. One attribute is overall a bit simpler, but any diagnostics are going to need to differentiate anyway. Also, I wonder whether a single overlay attribute could confuse users if it was applied to a global function pointer - a naive user might think that the attribute applies to the type of the function rather than the Decl of the pointer.

clang/lib/Sema/SemaDeclAttr.cpp
2043–2047

Yes, the reason for the custom diagnostic is because the diagnostic explicitly requests the user enables -moverlay. If I didn't use the custom diagnostic then I could use the generic 'attribute ignored' warning with an additional diagnostic for 'note: use -moverlay to enable attribute((overlaycall))', but if the generic warning is emitted from generic code I don't know how I would attach the 'note' diagnostic to it.

edward-jones edited the summary of this revision. (Show Details)Thu, Nov 4, 7:35 AM