This is an archive of the discontinued LLVM Phabricator instance.

[Clang] Move assembler into a separate file
ClosedPublic

Authored by aykevl on Jun 26 2019, 4:53 PM.

Details

Summary

This change adds an AssemblerInvocation class, similar to the CompilerInvocation class. It can be used to invoke cc1as directly.

The project I'm working on wants to compile Clang and use it as a static library. For that to work, there must be a way to invoke the assembler programmatically, using the same arguments as you would otherwise pass to cc1as.


This patch is outdated, updating it is fairly trivial. I'd like to have some feedback on the idea first, though.

Diff Detail

Event Timeline

aykevl created this revision.Jun 26 2019, 4:53 PM
Herald added a project: Restricted Project. · View Herald TranscriptJun 26 2019, 4:53 PM
aykevl updated this revision to Diff 206769.Jun 26 2019, 5:00 PM
  • removed useless anonymous namespace

*friendly ping*

aykevl added a comment.Dec 3 2019, 8:52 AM

Ping?
This would be super useful to have: it avoids copying most of cc1as.

aykevl added a comment.Apr 5 2020, 8:32 AM

Ping?
I'm not sure who to add as a reviewer.

aykevl edited the summary of this revision. (Show Details)Apr 5 2020, 8:33 AM

@echristo any chance you could take a look at this?

Hmm. In general I'd like to hear more about what you're trying. Can you give an idea of how you'd like to invoke the library to assemble something via clang? Why not just call into llvm directly? I'm not necessarily against changing the layering here fwiw, just wanted to get a better visual on what this is going to look like at the end and how it'll be used.

Thanks!

(sorry, I missed your comment)

I basically want to run clang by linking to it and calling it directly, without invoking any external commands.
You can see here how I did it:
https://github.com/tinygo-org/tinygo/blob/master/builder/clang.cpp
I copied the cc1as code in the project and modified it a little bit to be callable, essentially what this patch does:
https://github.com/tinygo-org/tinygo/blob/master/builder/cc1as.cpp

Ideally there would be something like the tinygo_clang_driver in Clang. Maybe there is, the last time I looked I couldn't find it.
What is most important is that it will not try to call clang externally, because that will make distribution much harder.

As a sidenote: I have needed to work around some global state in Clang/LLVM by creating a new process (essentially calling argv[0]) and doing the C file compilation in there. As one compiler invocation of TinyGo may need to compile several C files, global state led to problems. This is different from calling clang as I'm still shipping a single statically linked binary instead of several binaries.

The reason to not call into LLVM directly is because I want to use the compiler driver, to be compatible with all the compiler flags. Reimplementing the assembler driver would be a pain.

echristo accepted this revision.Dec 1 2020, 12:36 PM

Thanks for the explanation, lgtm.

-eric

This revision is now accepted and ready to land.Dec 1 2020, 12:36 PM
This revision was automatically updated to reflect the committed changes.

Thanks! I have updated this patch to match LLVM main and committed it.

Well that didn't quite work. I get errors like this:

tools/clang/lib/Frontend/CMakeFiles/obj.clangFrontend.dir/AssemblerInvocation.cpp.o:(.toc+0x0): undefined reference to `vtable for llvm::MCSubtargetInfo'
tools/clang/lib/Frontend/CMakeFiles/obj.clangFrontend.dir/AssemblerInvocation.cpp.o: In function `clang::ExecuteAssembler(clang::AssemblerInvocation&, clang::DiagnosticsEngine&)':
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x418): undefined reference to `llvm::MCTargetOptions::MCTargetOptions()'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x564): undefined reference to `llvm::MCContext::MCContext(llvm::MCAsmInfo const*, llvm::MCRegisterInfo const*, llvm::MCObjectFileInfo const*, llvm::SourceMgr const*, llvm::MCTargetOptions const*, bool)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x5c0): undefined reference to `llvm::MCObjectFileInfo::InitMCObjectFileInfo(llvm::Triple const&, bool, llvm::MCContext&, bool)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x71c): undefined reference to `llvm::MCContext::addDebugPrefixMapEntry(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0xa1c): undefined reference to `llvm::MCAsmBackend::createDwoObjectWriter(llvm::raw_pwrite_stream&, llvm::raw_pwrite_stream&) const'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0xea0): undefined reference to `llvm::MCContext::setGenDwarfRootFile(llvm::StringRef, llvm::StringRef)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x11d8): undefined reference to `llvm::createAsmStreamer(llvm::MCContext&, std::unique_ptr<llvm::formatted_raw_ostream, std::default_delete<llvm::formatted_raw_ostream> >, bool, bool, llvm::MCInstPrinter*, std::unique_ptr<llvm::MCCodeEmitter, std::default_delete<llvm::MCCodeEmitter> >&&, std::unique_ptr<llvm::MCAsmBackend, std::default_delete<llvm::MCAsmBackend> >&&, bool)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x137c): undefined reference to `llvm::createMCAsmParser(llvm::SourceMgr&, llvm::MCContext&, llvm::MCStreamer&, llvm::MCAsmInfo const&, unsigned int)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x1460): undefined reference to `llvm::MCContext::setSymbolValue(llvm::MCStreamer&, llvm::StringRef, unsigned long)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x1500): undefined reference to `llvm::MCContext::setSymbolValue(llvm::MCStreamer&, llvm::StringRef, unsigned long)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x16dc): undefined reference to `llvm::MCContext::~MCContext()'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x1be8): undefined reference to `llvm::createELFStreamer(llvm::MCContext&, std::unique_ptr<llvm::MCAsmBackend, std::default_delete<llvm::MCAsmBackend> >&&, std::unique_ptr<llvm::MCObjectWriter, std::default_delete<llvm::MCObjectWriter> >&&, std::unique_ptr<llvm::MCCodeEmitter, std::default_delete<llvm::MCCodeEmitter> >&&, bool)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x1c08): undefined reference to `llvm::createXCOFFStreamer(llvm::MCContext&, std::unique_ptr<llvm::MCAsmBackend, std::default_delete<llvm::MCAsmBackend> >&&, std::unique_ptr<llvm::MCObjectWriter, std::default_delete<llvm::MCObjectWriter> >&&, std::unique_ptr<llvm::MCCodeEmitter, std::default_delete<llvm::MCCodeEmitter> >&&, bool)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x1d20): undefined reference to `llvm::MCContext::getMachOSection(llvm::StringRef, llvm::StringRef, unsigned int, unsigned int, llvm::SectionKind, char const*)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x1d54): undefined reference to `llvm::MCStreamer::emitZeros(unsigned long)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x1d68): undefined reference to `llvm::MCAsmParser::setTargetParser(llvm::MCTargetAsmParser&)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x2090): undefined reference to `llvm::MCAsmBackend::createObjectWriter(llvm::raw_pwrite_stream&) const'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x2128): undefined reference to `llvm::createNullStreamer(llvm::MCContext&)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x24e0): undefined reference to `llvm::createMachOStreamer(llvm::MCContext&, std::unique_ptr<llvm::MCAsmBackend, std::default_delete<llvm::MCAsmBackend> >&&, std::unique_ptr<llvm::MCObjectWriter, std::default_delete<llvm::MCObjectWriter> >&&, std::unique_ptr<llvm::MCCodeEmitter, std::default_delete<llvm::MCCodeEmitter> >&&, bool, bool, bool)'
AssemblerInvocation.cpp:(.text._ZN5clang16ExecuteAssemblerERNS_19AssemblerInvocationERNS_17DiagnosticsEngineE+0x2500): undefined reference to `llvm::createWasmStreamer(llvm::MCContext&, std::unique_ptr<llvm::MCAsmBackend, std::default_delete<llvm::MCAsmBackend> >&&, std::unique_ptr<llvm::MCObjectWriter, std::default_delete<llvm::MCObjectWriter> >&&, std::unique_ptr<llvm::MCCodeEmitter, std::default_delete<llvm::MCCodeEmitter> >&&, bool)'
collect2: error: ld returned 1 exit status

See:
http://lab.llvm.org:8011/#/builders/57/builds/3704
http://lab.llvm.org:8011/#/builders/112/builds/3216
http://lab.llvm.org:8011/#/builders/121/builds/3900

@echristo do you have an idea what's going on or how to fix this? I suspect I'm not including a required dependency or maybe I've put AssemblerInvocation in the wrong directory/library.
I'm not very familiar with CMake or C++ so I'm not sure how to best fix this.

@aykevl, please check http://lab.llvm.org:8011/#/builders/57/builds/3704: it seems you are missing a change to clang/lib/Frontend/CMakeLists.txt to update the LLVM_LINK_COMPONENTS.

or maybe I've put AssemblerInvocation in the wrong directory/library.

This seems to be a good design question. I think a lot of people consider -cc1 functionality to be the "frontend". I am not sure that goes for -cc1as functionality. This might even be a new library in a sense (a libclangAssemblyFrontend).

Yeah I was wondering the same thing when I saw the failure. Unfortunately such design is a bit outside of my LLVM knowledge. I would just like to use cc1as_main.cpp functionality outside of LLVM without needing to update my copy of cc1as with every LLVM update.