LLVM Driver Multicall tool

[abrachet]

Summary

This patch adds an llvm-driver multicall tool that can combine multiple
LLVM-based tools. The build infrastructure is enabled for a tool by
adding the GENERATE_DRIVER option to the add_llvm_executable CMake
call, and changing the tool's main function to a canonicalized
tool_name_main format (i.e. llvm_ar_main, clang_main, etc...).

As currently implemented llvm-driver contains dsymutil, llvm-ar,
llvm-cxxfilt, llvm-objcopy, and clang (if clang is included in the
build).

llvm-driver can be enabled from builds by setting
LLVM_TOOL_LLVM_DRIVER_BUILD=On.

There are several limitations in the current implementation, which can
be addressed in subsequent patches:

(1) the multicall binary cannot currently properly handle
multi-dispatch tools. This means symlinking llvm-ranlib to llvm-driver
will not properly result in llvm-ar's main being called.
(2) the multicall binary cannot be comprised of tools containing
conflicting cl::opt options as the global cl::opt option list cannot
contain duplicates.

These limitations can be addressed in subsequent patches.

[mehdi_amini]

That's pretty nice! Have you thought about looking into a lit option (triggered by a cmake flag maybe) that would change the substitution for the tools that are enabled by llvm-driver to use the latter instead when running the tests?

[phosek]

llvm/tools/llvm-driver/CMakeLists.txt

18

As was already suggested on D104686, I'd prefer naming this just llvm so you can invoke tools with llvm name which doesn't require any additional typing compared to llvm-name.

[beanz]

In D109977#3006687, @mehdi_amini wrote:

That's pretty nice! Have you thought about looking into a lit option (triggered by a cmake flag maybe) that would change the substitution for the tools that are enabled by llvm-driver to use the latter instead when running the tests?

That's an awesome idea. I had been thinking about having a CMake option to generate symlinks instead of the tools, but you could totally bake this into the lit substitutions. I'll take a look at that.

llvm/tools/llvm-driver/CMakeLists.txt

18

Can do

[beanz]

Renaming llvm-driver binary to llvm.

[aganea]

This is really a great change, thanks @beanz!

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

226

Shouldn't we say:

int objcopy_main(int argc, char **argv) {

here and the other places + all supporting code, if we want llvm objcopy (without the dash) like @phosek suggests?

[beanz]

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

226

I had a different thought for that. I think we want the tools to respond to llvm-objcopy since we will want them to exist in parallel to binutils tools just like the current tools do today.

Many of the current tools also support symlink-redirection, to support that we'll need to have a multiplex where multiple tool names point to the same main function.

Handling that was my point (1) in the main commit message, and I intended to work on it in a follow-on commit.

[phosek]

@beanz Is this ready to land or do you plan on making any more changes?

[MaskRay]

Ping @beanz

[beanz]

Sorry for disappearing on this. I'm working right now on adding support for multi-entry tools (like objcopy). I'll get an update in today.

[beanz]

Adding support for multiple subcommands mapping to the same entry.

[MaskRay]

Hope that someone familiar with CMake can take a look.

As currently implemented llvm-driver contains dsymutil, llvm-ar, llvm-cxxfilt, llvm-objcopy, and clang (if clang is included in the build).

I think either (bundled clang+binary utilities) or (bundled lld+binary utilities) is fine.
Some folks are interested in PGO builds for clang and lld. Not bundling clang and lld together can make both optimized.
The performance of binary utilities likely does not matter that much, so bundling them with either clang or lld should be fine.

---

I have tried the latest diff but -DLLVM_ENABLE_PROJECTS='clang;clang-tools-extra;flang;lldb;lld;compiler-rt;openmp;mlir;cross-project-tests' -DLLVM_TOOL_LLVM_DRIVER_BUILD=on gives me:

CMake Error at cmake/modules/LLVM-Config.cmake:110 (target_link_libraries):                 
  The keyword signature for target_link_libraries has already been used with                
  the target "obj.dsymutil".  All uses of target_link_libraries with a target               
  must be either all-keyword or all-plain.                                                  
                                                                                            
  The uses of the keyword signature are here:                                                                                                 
                                                                                            
   * cmake/modules/AddLLVM.cmake:897 (target_link_libraries)                                
                                                                                            
Call Stack (most recent call first):                                                        
  cmake/modules/LLVM-Config.cmake:95 (explicit_llvm_config)                                 
  cmake/modules/AddLLVM.cmake:898 (llvm_config)                                                                                               
  cmake/modules/AddLLVM.cmake:1264 (add_llvm_executable)                                                                                      
  tools/dsymutil/CMakeLists.txt:21 (add_llvm_tool)

[MaskRay]

In D109977#3282545, @MaskRay wrote:

I have tried the latest diff but -DLLVM_ENABLE_PROJECTS='clang;clang-tools-extra;flang;lldb;lld;compiler-rt;openmp;mlir;cross-project-tests' -DLLVM_TOOL_LLVM_DRIVER_BUILD=on gives me:

CMake Error at cmake/modules/LLVM-Config.cmake:110 (target_link_libraries):                 
  The keyword signature for target_link_libraries has already been used with                
  the target "obj.dsymutil".  All uses of target_link_libraries with a target               
  must be either all-keyword or all-plain.                                                  
                                                                                            
  The uses of the keyword signature are here:                                                                                                 
                                                                                            
   * cmake/modules/AddLLVM.cmake:897 (target_link_libraries)                                
                                                                                            
Call Stack (most recent call first):                                                        
  cmake/modules/LLVM-Config.cmake:95 (explicit_llvm_config)                                 
  cmake/modules/AddLLVM.cmake:898 (llvm_config)                                                                                               
  cmake/modules/AddLLVM.cmake:1264 (add_llvm_executable)                                                                                      
  tools/dsymutil/CMakeLists.txt:21 (add_llvm_tool)

@beanz ⬆️

[phosek]

Another potential future improvement is error reporting for subcommands:

$ ./bin/llvm clang       
llvm: error: no input files
$ ./bin/clang    
clang-15: error: no input files

Ideally, the multicall tool would produce the same error message.

llvm/cmake/modules/AddLLVM.cmake

900

The error that @MaskRay reported is because we use PUBLIC with target_link_libraries() here, but we use plain form of target_link_libraries() later in explicit_llvm_config that is invoked llvm_config (see https://github.com/llvm/llvm-project/blob/1643f01232b41b93e5f3a21d89b111efab0e378a/llvm/cmake/modules/LLVM-Config.cmake#L110). Omitting PUBLIC here appears to be sufficient to avoid the error.

llvm/tools/CMakeLists.txt

59

Should this be guarded by LLVM_TOOL_LLVM_DRIVER_BUILD to behave as the commit message says? Currently the llvm-driver appears to be built regardless of LLVM_TOOL_LLVM_DRIVER_BUILD.

llvm/tools/llvm-driver/llvm-driver.cpp

44

When the tool is invoked without any arguments (that is, simply as ./bin/llvm) this will lead to out-of-bounds array access. We should handle this case explicitly.

[phosek]

@beanz Let me know if you need help, I'm happy to commandeer the change if you're too busy.

[beanz]

In D109977#3405687, @phosek wrote:

@beanz Let me know if you need help, I'm happy to commandeer the change if you're too busy.

Please feel free to commandeer it. I'm really sorry I've held this up so long and I'm totally swamped right now :(.

[abrachet]

I spoke to @beanz offline to make sure he is still ok with the patch being commandeered. Thanks for the patch @beanz, very excited about this :)

[abrachet]

• llvm llvm-$tool -> llvm $tool
• Error messages will now show $tool: error: ... instead of error: ...

[abrachet]

In D109977#3378312, @phosek wrote:

Another potential future improvement is error reporting for subcommands:

$ ./bin/llvm clang       
llvm: error: no input files
$ ./bin/clang    
clang-15: error: no input files

Ideally, the multicall tool would produce the same error message.

It's difficult to make the error message the same, ie clang-15, but hopefully the name the tool was invoked with is enough.

llvm/tools/llvm-driver/llvm-driver.cpp

44

This will now print the help message

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

226

Shouldn't we say:

int objcopy_main(int argc, char **argv) {

here and the other places + all supporting code, if we want llvm objcopy (without the dash) like @phosek suggests?

I've kept name of the function as is, but llvm objcopy is now supported, and llvm llvm-objcopy is not

[MaskRay]

Thanks for picking up the change. I confirm that I can build llvm without an error and it appears to work fine.

llvm-driver can be disabled from builds by setting LLVM_TOOL_LLVM_DRIVER_BUILD=Off.

I think this should be opt-in. The new llvm executable takes a lot of space and not needed by many developers/build bots.
It's useful to some groups (distributions) but they can specify the option themselves.
I think the modified code is quite stable, so don't worry about regressions just because this is not opt-in.

llvm/tools/llvm-driver/llvm-driver.cpp

25

static int unknownMain

26

Prefer OptTable to cl:: for user-facing utilities. If simple, just open coding the help message.

38

https://llvm.org/docs/CodingStandards.html#error-and-warning-messages

51

Some distributions may want to use something like llvm-15. See some binary utilities how the version is handled.

59

If only used once, don't add a typedef (using is preferred)

[MaskRay]

% /tmp/out/custom1/bin/llvm --help
OVERVIEW: llvm compiler driver   ##### llvm toolchain driver ?

USAGE: llvm [subcommand] [options]

SUBCOMMANDS:

  ar                - ar  ###### should be llvm-ar
  bitcode-strip     - bitcode-strip   ### should be llvm-bitcode-strip
  clang             - clang
  clang++           - clang++
  clang-cl          - clang-cl
  clang-cpp         - clang-cpp
  cxxfilt           - cxxfilt   ### ditto
  dlltool           - dlltool
  dsymutil          - dsymutil
  install-name-tool - install-name-tool
  lib               - lib  ##### this is confusing. llvm-lib
  objcopy           - objcopy
  ranlib            - ranlib
  strip             - strip

  Type "llvm <subcommand> --help" to get more help on a specific subcommand

OPTIONS:

Generic Options:

  --help      - Display available options (--help-hidden for more) 
  --help-list - Display list of available options (--help-list-hidden for more)  #### this from cl:: is not useful
  --version   - Display the version of this program

[abrachet]

Better handling of names with version, ie llvm-cxxfilt-15 -> cxxfilt

[abrachet]

In D109977#3558762, @MaskRay wrote:

Thanks for picking up the change. I confirm that I can build llvm without an error and it appears to work fine.

llvm-driver can be disabled from builds by setting LLVM_TOOL_LLVM_DRIVER_BUILD=Off.

I think this should be opt-in. The new llvm executable takes a lot of space and not needed by many developers/build bots.
It's useful to some groups (distributions) but they can specify the option themselves.
I think the modified code is quite stable, so don't worry about regressions just because this is not opt-in.

I didn't update the commits message, now it properly says that it can be opted in by setting LLVM_TOOL_LLVM_DRIVER_BUILD to On

In D109977#3558768, @MaskRay wrote:

% /tmp/out/custom1/bin/llvm --help
...

The output now looks like:

OVERVIEW: llvm toolchain driver

USAGE: llvm [subcommand] [options]

SUBCOMMANDS:

  ar
  clang
  dsymutil
  cxxfilt
  objcopy
  ranlib
  lib
  dlltool
  clang++
  clang-cl
  clang-cpp
  install-name-tool
  bitcode-strip
  strip

  Type "llvm <subcommand> --help" to get more help on a specific subcommand

OPTIONS:

  --help - Display this message

What do you think?

llvm/tools/llvm-driver/llvm-driver.cpp

51

Thank's I've taken this from objcopy's code

[MaskRay]

LGTM from my perspective, but this needs someone else's review.

llvm/tools/llvm-driver/llvm-driver.cpp

51

The format of cl::PrintVersionMessage(); is not so good for user-facing tools. Consider omitting it.

llvm-objcopy --version should probably use a style similar to clang --version but the priority isn't high.

[phosek]

LGTM

llvm/cmake/modules/AddLLVM.cmake

2030

Alternative substitution for + would be x which is used elsewhere in LLVM, for example libcxx, libcxxabi or cxxfilt.

[abrachet]

• Fix bazel build, llvm-driver can't be built by bazel, but the existing tools which lost their main will still work.
• Remove --version
• Fix dsymutil not depending on DsymutilTableGen which caused a dependency ordering problem on the bots

[abrachet]

llvm/cmake/modules/AddLLVM.cmake

2030

Upon closer inspection we are no longer using that parameter of the macro, so I have removed it.

llvm/tools/llvm-driver/llvm-driver.cpp

51

I've just removed --version completely for now. I don't think it is that pressing.

[thakis]

llvm/test/lit.cfg.py

142

This causes llvm-lit: /Users/thakis/src/llvm-project/llvm/utils/lit/lit/llvm/subst.py:126: note: Did not find llvm in /Users/thakis/src/llvm-build-2/bin if llvm-driver isn't enabled (which it isn't by default). Any reason not to add this one only if config.have_llvm_driver?

[abrachet]

llvm/test/lit.cfg.py

142

Fixed in rG553c2af5360b993376d3c000cf53850605940307

[chapuni]

An interesting improvement. Thanks.

llvm/cmake/modules/AddLLVM.cmake

890

Do you have a plan to export the template?

utils/bazel/llvm-project-overlay/clang/BUILD.bazel

1932

Could we unify generated names.cpp between cmake and bazel?

I am running a builder to compare each object file between cmake and bazel. Such a difference appears in a few sections, at least in elf.

[thakis]

llvm/test/lit.cfg.py

142

Thank you!

[nikic]

llvm/cmake/modules/AddLLVM.cmake

890

Unsurprisingly, this breaks standalone builds. However, I think that the multicall tool is fundamentally incompatible with standalone builds, in the sense that all tools included in it must be part of the initial build configuration -- you can't first build the LLVM multicall tool and then add clang to it afterwards.

So I think we just need to guard this code by LLVM_TOOL_LLVM_DRIVER_BUILD, so we don't try to read a non-exported driver-template.cpp.in file even if we're not actually building the multicall tool.

[nikic]

llvm/cmake/modules/AddLLVM.cmake

890

Ah, I misunderstood how this works. Generating the driver file is necessary for normal usage, not for the mutlicall tool, so it's not possible to just guard this code. We do need to export driver-template.cpp.in as part of the LLVM cmake installation.

[nikic]

llvm/cmake/modules/AddLLVM.cmake

890

I've put up https://reviews.llvm.org/D127384 to fix this.

[mgorny]

This seems to have broken standalone builds for Gentoo.

While building Clang, I get:

CMake Error at /usr/lib/llvm/15/lib/cmake/llvm/LLVM-Config.cmake:138 (message):
  Target Sparc is not in the set of libraries.
Call Stack (most recent call first):
  /usr/lib/llvm/15/lib/cmake/llvm/LLVM-Config.cmake:263 (llvm_expand_pseudo_components)
  /usr/lib/llvm/15/lib/cmake/llvm/LLVM-Config.cmake:102 (llvm_map_components_to_libnames)
  /usr/lib/llvm/15/lib/cmake/llvm/LLVM-Config.cmake:95 (explicit_llvm_config)
  /usr/lib/llvm/15/lib/cmake/llvm/AddLLVM.cmake:901 (llvm_config)
  cmake/modules/AddClang.cmake:146 (add_llvm_executable)
  cmake/modules/AddClang.cmake:156 (add_clang_executable)
  tools/driver/CMakeLists.txt:25 (add_clang_tool)

(or a similar message for another target)

[abrachet]

In D109977#3611683, @mgorny wrote:

This seems to have broken standalone builds for Gentoo.

While building Clang, I get:

CMake Error at /usr/lib/llvm/15/lib/cmake/llvm/LLVM-Config.cmake:138 (message):
  Target Sparc is not in the set of libraries.
Call Stack (most recent call first):
  /usr/lib/llvm/15/lib/cmake/llvm/LLVM-Config.cmake:263 (llvm_expand_pseudo_components)
  /usr/lib/llvm/15/lib/cmake/llvm/LLVM-Config.cmake:102 (llvm_map_components_to_libnames)
  /usr/lib/llvm/15/lib/cmake/llvm/LLVM-Config.cmake:95 (explicit_llvm_config)
  /usr/lib/llvm/15/lib/cmake/llvm/AddLLVM.cmake:901 (llvm_config)
  cmake/modules/AddClang.cmake:146 (add_llvm_executable)
  cmake/modules/AddClang.cmake:156 (add_clang_executable)
  tools/driver/CMakeLists.txt:25 (add_clang_tool)

(or a similar message for another target)

Could I have a cmake invocation or a bot link?

[beanz]

In D109977#3652091, @abrachet wrote:

In D109977#3611683, @mgorny wrote:

This seems to have broken standalone builds for Gentoo.

While building Clang, I get:

CMake Error at /usr/lib/llvm/15/lib/cmake/llvm/LLVM-Config.cmake:138 (message):
  Target Sparc is not in the set of libraries.
Call Stack (most recent call first):
  /usr/lib/llvm/15/lib/cmake/llvm/LLVM-Config.cmake:263 (llvm_expand_pseudo_components)
  /usr/lib/llvm/15/lib/cmake/llvm/LLVM-Config.cmake:102 (llvm_map_components_to_libnames)
  /usr/lib/llvm/15/lib/cmake/llvm/LLVM-Config.cmake:95 (explicit_llvm_config)
  /usr/lib/llvm/15/lib/cmake/llvm/AddLLVM.cmake:901 (llvm_config)
  cmake/modules/AddClang.cmake:146 (add_llvm_executable)
  cmake/modules/AddClang.cmake:156 (add_clang_executable)
  tools/driver/CMakeLists.txt:25 (add_clang_tool)

(or a similar message for another target)

Could I have a cmake invocation or a bot link?

@mgorny Have you updated your LLVM with this change?

If not, I suspect this breakage is expected.

@mgorny is building against an installed llvm-15 that if it doesn't have the changes from this patch, mixing that with a ToT clang which does will not work.

In that case the solution is that @mgorny will need to update the LLVM install to handle the revlock.

LLVM & Clang do rev lock from time to time during trunk development, usually when LLVM APIs change, but the build system can cause rev locks too since Clang reuses LLVM's CMake modules.

[mgorny]

In D109977#3652091, @abrachet wrote:

Could I have a cmake invocation or a bot link?

The exact CMake invocation for clang is:

cmake -C /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/x/y/clang-abi_x86_64.amd64/gentoo_common_config.cmake -G Ninja -DCMAKE_INSTALL_PREFIX=/usr -DLLVM_CMAKE_PATH=/usr/lib/llvm/15/lib64/cmake/llvm -DCMAKE_INSTALL_PREFIX=/usr/lib/llvm/15 -DCMAKE_INSTALL_MANDIR=/usr/lib/llvm/15/share/man -DCLANG_RESOURCE_DIR=../../../../lib/clang/15.0.0 -DBUILD_SHARED_LIBS=OFF -DCLANG_LINK_CLANG_DYLIB=ON -DLLVM_DISTRIBUTION_COMPONENTS="clang-cmake-exports;clang-headers;clang-resource-headers;libclang-headers;clang-cpp;libclang;bash-autocomplete;libclang-python-bindings;c-index-test;clang;clang-format;clang-offload-bundler;clang-offload-wrapper;clang-refactor;clang-repl;clang-rename;clang-scan-deps;diagtool;hmaptool;clang-apply-replacements;clang-change-namespace;clang-doc;clang-include-fixer;clang-move;clang-query;clang-reorder-fields;clang-tidy;clang-tidy-headers;clangd;find-all-symbols;modularize;pp-trace;docs-clang-man;docs-clang-tools-man;clang-check;clang-extdef-mapping;scan-build;scan-build-py;scan-view;" -DLLVM_TARGETS_TO_BUILD="SystemZ;WebAssembly;BPF;Hexagon;VE;XCore;X86;Sparc;NVPTX;ARM;AVR;Lanai;AMDGPU;Mips;AArch64;PowerPC;MSP430;RISCV" -DLLVM_BUILD_TESTS=no -DLLVM_ENABLE_EH=ON -DLLVM_ENABLE_RTTI=ON -DCMAKE_DISABLE_FIND_PACKAGE_LibXml2=no -DCLANG_DEFAULT_OPENMP_RUNTIME=libomp -DCMAKE_DISABLE_FIND_PACKAGE_CUDA=ON -DCLANG_DEFAULT_CXX_STDLIB= -DCLANG_DEFAULT_RTLIB= -DCLANG_DEFAULT_LINKER= -DCLANG_DEFAULT_PIE_ON_LINUX=yes -DCLANG_DEFAULT_UNWINDLIB= -DCLANG_ENABLE_ARCMT=yes -DCLANG_ENABLE_STATIC_ANALYZER=yes -DPython3_EXECUTABLE=/usr/bin/python3.10 -DLLVM_BUILD_DOCS=ON -DLLVM_ENABLE_SPHINX=ON -DCLANG_INSTALL_SPHINX_HTML_DIR=/usr/share/doc/clang-15.0.0.9999/html -DCLANG-TOOLS_INSTALL_SPHINX_HTML_DIR=/usr/share/doc/clang-15.0.0.9999/tools-extra -DSPHINX_WARNINGS_AS_ERRORS=OFF -DLLVM_EXTERNAL_CLANG_TOOLS_EXTRA_SOURCE_DIR=/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang-tools-extra -DCLANG_INCLUDE_DOCS=ON -DCLANG_TOOLS_EXTRA_INCLUDE_DOCS=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_TOOLCHAIN_FILE=/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/x/y/clang-abi_x86_64.amd64/gentoo_toolchain.cmake /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang

Though I can reproduce it with much shorter:

mkdir build
cd build
cmake ../clang -G Ninja

(matching installed version of LLVM 15 must be on PATH)

In D109977#3652117, @beanz wrote:

@mgorny Have you updated your LLVM with this change?

Obviously. I always build matching git commit of all LLVM components when testing the most recent state.

[abrachet]

In D109977#3652467, @mgorny wrote:

In D109977#3652091, @abrachet wrote:

Could I have a cmake invocation or a bot link?

The exact CMake invocation for clang is:

cmake -C /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/x/y/clang-abi_x86_64.amd64/gentoo_common_config.cmake -G Ninja -DCMAKE_INSTALL_PREFIX=/usr -DLLVM_CMAKE_PATH=/usr/lib/llvm/15/lib64/cmake/llvm -DCMAKE_INSTALL_PREFIX=/usr/lib/llvm/15 -DCMAKE_INSTALL_MANDIR=/usr/lib/llvm/15/share/man -DCLANG_RESOURCE_DIR=../../../../lib/clang/15.0.0 -DBUILD_SHARED_LIBS=OFF -DCLANG_LINK_CLANG_DYLIB=ON -DLLVM_DISTRIBUTION_COMPONENTS="clang-cmake-exports;clang-headers;clang-resource-headers;libclang-headers;clang-cpp;libclang;bash-autocomplete;libclang-python-bindings;c-index-test;clang;clang-format;clang-offload-bundler;clang-offload-wrapper;clang-refactor;clang-repl;clang-rename;clang-scan-deps;diagtool;hmaptool;clang-apply-replacements;clang-change-namespace;clang-doc;clang-include-fixer;clang-move;clang-query;clang-reorder-fields;clang-tidy;clang-tidy-headers;clangd;find-all-symbols;modularize;pp-trace;docs-clang-man;docs-clang-tools-man;clang-check;clang-extdef-mapping;scan-build;scan-build-py;scan-view;" -DLLVM_TARGETS_TO_BUILD="SystemZ;WebAssembly;BPF;Hexagon;VE;XCore;X86;Sparc;NVPTX;ARM;AVR;Lanai;AMDGPU;Mips;AArch64;PowerPC;MSP430;RISCV" -DLLVM_BUILD_TESTS=no -DLLVM_ENABLE_EH=ON -DLLVM_ENABLE_RTTI=ON -DCMAKE_DISABLE_FIND_PACKAGE_LibXml2=no -DCLANG_DEFAULT_OPENMP_RUNTIME=libomp -DCMAKE_DISABLE_FIND_PACKAGE_CUDA=ON -DCLANG_DEFAULT_CXX_STDLIB= -DCLANG_DEFAULT_RTLIB= -DCLANG_DEFAULT_LINKER= -DCLANG_DEFAULT_PIE_ON_LINUX=yes -DCLANG_DEFAULT_UNWINDLIB= -DCLANG_ENABLE_ARCMT=yes -DCLANG_ENABLE_STATIC_ANALYZER=yes -DPython3_EXECUTABLE=/usr/bin/python3.10 -DLLVM_BUILD_DOCS=ON -DLLVM_ENABLE_SPHINX=ON -DCLANG_INSTALL_SPHINX_HTML_DIR=/usr/share/doc/clang-15.0.0.9999/html -DCLANG-TOOLS_INSTALL_SPHINX_HTML_DIR=/usr/share/doc/clang-15.0.0.9999/tools-extra -DSPHINX_WARNINGS_AS_ERRORS=OFF -DLLVM_EXTERNAL_CLANG_TOOLS_EXTRA_SOURCE_DIR=/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang-tools-extra -DCLANG_INCLUDE_DOCS=ON -DCLANG_TOOLS_EXTRA_INCLUDE_DOCS=ON -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_TOOLCHAIN_FILE=/var/tmp/portage/sys-devel/clang-15.0.0.9999/work/x/y/clang-abi_x86_64.amd64/gentoo_toolchain.cmake /var/tmp/portage/sys-devel/clang-15.0.0.9999/work/clang

This won't work for me because I will need gentoo_common_config.cmake, though I would rather work with the shorter reproducer you have provided.

Though I can reproduce it with much shorter:

mkdir build
cd build
cmake ../clang -G Ninja

(matching installed version of LLVM 15 must be on PATH)

Are you sure it should be in PATH? I have added installed LLVM from ToT to a directory, put that in path yet CMake still finds /usr/lib/llvm-13/cmake/AddLLVM.cmake instead. I can't find any documentation on the standalone build, unfortunately.

[mgorny]

In D109977#3660006, @abrachet wrote:

In D109977#3652467, @mgorny wrote:

Though I can reproduce it with much shorter:

mkdir build
cd build
cmake ../clang -G Ninja

(matching installed version of LLVM 15 must be on PATH)

Are you sure it should be in PATH? I have added installed LLVM from ToT to a directory, put that in path yet CMake still finds /usr/lib/llvm-13/cmake/AddLLVM.cmake instead. I can't find any documentation on the standalone build, unfortunately.

I'm sorry, what does "ToT" mean here? Also, what's /usr/lib/llvm-13/cmake? Perhaps it's picking it up from some other variable. In my case, it's finding the installed LLVM 14 because that's the first PATH entry. Note that CMake looks into ../lib*/cmake and ../share/cmake relative to PATH entries. FWICS it also looks at CMAKE_MODULE_PATH.

[MaskRay]

In D109977#3660734, @mgorny wrote:

In D109977#3660006, @abrachet wrote:

In D109977#3652467, @mgorny wrote:

Though I can reproduce it with much shorter:

mkdir build
cd build
cmake ../clang -G Ninja

(matching installed version of LLVM 15 must be on PATH)

Are you sure it should be in PATH? I have added installed LLVM from ToT to a directory, put that in path yet CMake still finds /usr/lib/llvm-13/cmake/AddLLVM.cmake instead. I can't find any documentation on the standalone build, unfortunately.

I'm sorry, what does "ToT" mean here? Also, what's i? Perhaps it's picking it up from some other variable. In my case, it's finding the installed LLVM 14 because that's the first PATH entry. Note that CMake looks into ../lib*/cmake and ../share/cmake relative to PATH entries. FWICS it also looks at CMAKE_MODULE_PATH.

"Top of Trunk" (or tip?) (or tree?) which indicates the latest commit of origin/main . "Trunk" was a term inherited from the old subversion days.

Debian installs different llvm packages to /usr/lib/llvm-*:

% ls /usr/lib/llvm-* 
/usr/lib/llvm-11:
bin/  build/  cmake@  include/  lib/  share/

/usr/lib/llvm-13:
bin/  build/  cmake@  include/  lib/  share/

/usr/lib/llvm-9:
bin/  build/  cmake@  include/  lib/  share/
% dpkg -S /usr/lib/llvm-13/lib/cmake/llvm/LLVMConfig.cmake 
llvm-13-dev: /usr/lib/llvm-13/lib/cmake/llvm/LLVMConfig.cmake

For downstream reports (e.g. Debian, Gentoo), it is worth stripping the distribution specific knowledge so that a patch author doesn't need to install a particular distribution to reproduce an issue.

Personally I really wish that the build system issue can be fixed in the upcoming 15.0.0 release and encourage distributions to use it to decrease executable sizes.

[abrachet]

Thanks @mgorny I was able to get CMake to run. Although I didn't end up with the same error as you. I simply had warnings:

CMake Warning (dev) at <redacted>/usr/local/lib/cmake/llvm/TableGen.cmake:103 (add_custom_command):
  Policy CMP0116 is not set: Ninja generators transform DEPFILEs from
  add_custom_command().  Run "cmake --help-policy CMP0116" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.
Call Stack (most recent call first):
  cmake/modules/AddClang.cmake:25 (tablegen)
  include/clang/AST/CMakeLists.txt:1 (clang_tablegen)
This warning is for project developers.  Use -Wno-dev to suppress it.

When running just cmake ../clang -G Ninja did you delete CMakeCache.txt? If you didn't then perhaps something from the full command you gave stayed around and caused the error. I suspect then I will need the full command you have supplied above including gentoo_common_config.cmake. Likely I will not need gentoo_toolchain.cmake

In D109977#3660778, @MaskRay wrote:

Personally I really wish that the build system issue can be fixed in the upcoming 15.0.0 release and encourage distributions to use it to decrease executable sizes.

Me too. For these tools to be installable as symlinks to the llvm driver, D127800 will need to land, as well as a follow up after that patch. Maybe I could entice you to take a look :)

[mgorny]

Personally I really wish that the build system issue can be fixed in the upcoming 15.0.0 release and encourage distributions to use it to decrease executable sizes.

Me too, not because of multicall but because right now it's impossible to even build clang with multicall disabled. It's a complete 100% blocker for us.

In D109977#3661581, @abrachet wrote:

When running just cmake ../clang -G Ninja did you delete CMakeCache.txt?

I'm pretty sure I did.

If you didn't then perhaps something from the full command you gave stayed around and caused the error. I suspect then I will need the full command you have supplied above including gentoo_common_config.cmake. Likely I will not need gentoo_toolchain.cmake

I'm going to start by trying against with today's main, maybe (hopefully) something changed. I just hoped I wouldn't have to spend the time I don't really have on this but I guess I have no other option :-(.

[mgorny]

Hmm, I have another idea. Do you have individual LLVM static (or shared) libraries installed? On Gentoo we install only the dylib and a few unavoidable static libs:

/usr/lib/llvm/15/lib64/libRemarks.so
/usr/lib/llvm/15/lib64/libRemarks.so.15git
/usr/lib/llvm/15/lib64/libLLVM-15git.so
/usr/lib/llvm/15/lib64/libLLVM.so
/usr/lib/llvm/15/lib64/libLLVM-15.0.0git.so
/usr/lib/llvm/15/lib64/libLTO.so
/usr/lib/llvm/15/lib64/libLTO.so.15git
/usr/lib/llvm/15/lib64/libLLVMTableGen.a
/usr/lib/llvm/15/lib64/libLLVMSupport.a
/usr/lib/llvm/15/lib64/libLLVMDemangle.a

On my end, I'm going to check if installing full set of static libs changes anything.

[mgorny]

Unfortunately, that doesn't seem to be it. I think at this point it would be easier for both of us if I tried to figure it out myself on Gentoo rather than trying to make it reproducible elsewhere. If you want, you could try grabbing e.g. a Gentoo stage3 image (e.g. via Docker or systemd-nspawn) and trying ACCEPT_KEYWORDS=** emerge -v clang:15 there but I suppose you'd have to figure out some Gentoo basics anyway (and then you'd realize it's the best distro in the world and waste years of your life developing it :-P).

[mgorny]

Ok, I think the key to reproducing it is -DLLVM_LINK_LLVM_DYLIB=ON. We pass that while building LLVM, so it gets passed on to clang but I suppose passing it to clang build might be sufficient to reproduce the problem.

The actual problem is that you're passing ${USE_SHARED} to llvm_config() (in add_llvm_executable) before it's defined. Moving the definition earlier fixed this error but uncovers another problem:

CMake Error at cmake/modules/AddClang.cmake:188 (target_link_libraries):
  The keyword signature for target_link_libraries has already been used with
  the target "obj.clang".  All uses of target_link_libraries with a target
  must be either all-keyword or all-plain.

  The uses of the keyword signature are here:

   * /usr/lib/llvm/15/lib64/cmake/llvm/LLVM-Config.cmake:92 (target_link_libraries)

Call Stack (most recent call first):
  tools/driver/CMakeLists.txt:37 (clang_target_link_libraries)

Now, if you move USE_SHARED definition using the following patch:

diff --git a/llvm/cmake/modules/AddLLVM.cmake b/llvm/cmake/modules/AddLLVM.cmake
index 8e1385e90b82..c37c1c3aa59d 100644
--- a/llvm/cmake/modules/AddLLVM.cmake
+++ b/llvm/cmake/modules/AddLLVM.cmake
@@ -890,6 +890,10 @@ macro(add_llvm_executable name)
     set_target_properties(${obj_name} PROPERTIES FOLDER "Object Libraries")
   endif()
 
+  if (LLVM_LINK_LLVM_DYLIB AND NOT ARG_DISABLE_LLVM_LINK_LLVM_DYLIB)
+    set(USE_SHARED USE_SHARED)
+  endif()
+
   if (ARG_GENERATE_DRIVER)
     string(REPLACE "-" "_" TOOL_NAME ${name})
     foreach(path ${CMAKE_MODULE_PATH})
@@ -964,10 +968,6 @@ macro(add_llvm_executable name)
     add_llvm_symbol_exports( ${name} ${LLVM_EXPORTED_SYMBOL_FILE} )
   endif(LLVM_EXPORTED_SYMBOL_FILE)
 
-  if (LLVM_LINK_LLVM_DYLIB AND NOT ARG_DISABLE_LLVM_LINK_LLVM_DYLIB)
-    set(USE_SHARED USE_SHARED)
-  endif()
-
   set(EXCLUDE_FROM_ALL OFF)
   set_output_directory(${name} BINARY_DIR ${LLVM_RUNTIME_OUTPUT_INTDIR} LIBRARY_DIR ${LLVM_LIBRARY_OUTPUT_INTDIR})
   llvm_config( ${name} ${USE_SHARED} ${LLVM_LINK_COMPONENTS} )

you should be able to reproduce the latter error using a regular in-tree build, e.g.:

cmake ../llvm -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DLLVM_ENABLE_PROJECTS='llvm;clang' -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON

[MaskRay]

In D109977#3662859, @mgorny wrote:

Ok, I think the key to reproducing it is -DLLVM_LINK_LLVM_DYLIB=ON. We pass that while building LLVM, so it gets passed on to clang but I suppose passing it to clang build might be sufficient to reproduce the problem.
...

cmake -Sllvm -B/tmp/out/play -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS='llvm;clang' -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON
ninja -C /tmp/out/play all

works on my Debian testing machine at main (without any patch).

---

If I enable -DLLVM_TOOL_LLVM_DRIVER_BUILD=on, there is an incompatibility due to some cl::opt options registered more than once.
This is a classical error. I think at this point -DLLVM_TOOL_LLVM_DRIVER_BUILD=on users are not supposed to use -DLLVM_BUILD_LLVM_DYLIB=ON,
but it'll be useful to fix this to get more size reduction.

% ninja -C /tmp/out/play bin/llvm
% /tmp/out/play/bin/llvm clang --help
: CommandLine Error: Option 'arm-implicit-it' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options
[1]    160970 IOT instruction  /tmp/out/play/bin/llvm clang --help

LLVM_TOOL_LLVM_DRIVER_BUILD defaults to off, so this should be fine.
Interested distributors can help fix the issue so that they can possibly use LLVM_TOOL_LLVM_DRIVER_BUILD for the 16.0.0 release.

[mgorny]

If you didn't apply my patch, then it works from in-tree build because it uses static libs instead of the dylib.

[abrachet]

In D109977#3662859, @mgorny wrote:

The actual problem is that you're passing ${USE_SHARED} to llvm_config() (in add_llvm_executable) before it's defined. Moving the definition earlier fixed this error but uncovers another problem:

Thanks for looking into it more. As for what @MaskRay said this mode won't work with LLVM_TOOL_LLVM_DRIVER_BUILD=On and it looks like because we were using this variable before setting, it was always empty anyway. For the reproducer that you gave the following diff works. Do you want to try this diff with the full gentoo build that was breaking? If it works then we can commit this.

diff --git a/llvm/cmake/modules/AddLLVM.cmake b/llvm/cmake/modules/AddLLVM.cmake
index 8e1385e90b82..5563a7848ec3 100644
--- a/llvm/cmake/modules/AddLLVM.cmake
+++ b/llvm/cmake/modules/AddLLVM.cmake
@@ -909,7 +909,7 @@ macro(add_llvm_executable name)
     
     set_property(GLOBAL APPEND PROPERTY LLVM_DRIVER_TOOLS ${name})
     target_link_libraries(${obj_name} ${LLVM_PTHREAD_LIB})
-    llvm_config(${obj_name} ${USE_SHARED} ${LLVM_LINK_COMPONENTS} )
+    llvm_config(${obj_name} ${LLVM_LINK_COMPONENTS})
   endif()
 
   add_windows_version_resource_file(ALL_FILES ${ALL_FILES})

[mgorny]

Unsurprisingly, it fails just the same because using an undefined variable is equivalent to not using it at all, so once again you're trying to link to static libraries that are *not there*. When dylib linking is used, you can't link to static libraries here.

[mgorny]

Ok, I've made D130158 as an absolutely minimal change that seems to fix it for me. Basically, I'm skipping all the driver-related logic that's not strictly necessary when LLVM_TOOL_LLVM_DRIVER_BUILD is off. The driver remains broken when dylib is used but at least it shouldn't affect people who don't enable it now.

clang/cmake/modules/AddClang.cmake

 endmacro()
 
 function(clang_target_link_libraries target type)
   if (CLANG_LINK_CLANG_DYLIB)
     target_link_libraries(${target} ${type} clang-cpp)
   else()
     target_link_libraries(${target} ${type} ${ARGN})
   endif()
+  if (TARGET obj.${target})
+    target_link_libraries(obj.${target} ${ARGN})
+  endif()
 
 endfunction()

clang/tools/driver/CMakeLists.txt

add_clang_tool(clang
   driver.cpp
   cc1_main.cpp
   cc1as_main.cpp
   cc1gen_reproducer_main.cpp
 
   DEPENDS
   intrinsics_gen
   ${support_plugins}
+  GENERATE_DRIVER
   )
 
 clang_target_link_libraries(clang
   PRIVATE
   clangBasic
   clangCodeGen
   clangDriver
   clangFrontend

clang/tools/driver/driver.cpp

if (Tool == "-cc1gen-reproducer")
     return cc1gen_reproducer_main(makeArrayRef(ArgV).slice(2), ArgV[0],
                                   GetExecutablePathVP);
   // Reject unknown tools.
   llvm::errs() << "error: unknown integrated tool '" << Tool << "'. "
                << "Valid tools include '-cc1' and '-cc1as'.\n";
   return 1;
 }
 
-int main(int Argc, const char **Argv) {
+int clang_main(int Argc, char **Argv) {
   noteBottomOfStack();
   llvm::InitLLVM X(Argc, Argv);
   llvm::setBugReportMsg("PLEASE submit a bug report to " BUG_REPORT_URL
                         " and include the crash backtrace, preprocessed "
                         "source, and associated run script.\n");
   SmallVector<const char *, 256> Args(Argv, Argv + Argc);
 
   if (llvm::sys::Process::FixupStandardFileDescriptors())

llvm/CMakeLists.txt

if ( LLVM_USE_FOLDERS )
   set_property(GLOBAL PROPERTY USE_FOLDERS ON)
 endif()
 
 include(VersionFromVCS)
 
 option(LLVM_APPEND_VC_REV
   "Embed the version control system revision in LLVM" ON)
 
+option(LLVM_TOOL_LLVM_DRIVER_BUILD "Enables building the llvm multicall tool" OFF)
+
 set(PACKAGE_NAME LLVM)
 set(PACKAGE_STRING "${PACKAGE_NAME} ${PACKAGE_VERSION}")
 set(PACKAGE_BUGREPORT "https://github.com/llvm/llvm-project/issues/")
 
 set(BUG_REPORT_URL "${PACKAGE_BUGREPORT}" CACHE STRING
   "Default URL where bug reports are to be submitted.")
 
 # Configure CPack.

llvm/cmake/driver-template.cpp.in

+//===-- driver-template.cpp -----------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+int @TOOL_NAME@_main(int argc, char **argv);
+
+int main(int argc, char **argv) { return @TOOL_NAME@_main(argc, argv); }

llvm/cmake/modules/AddLLVM.cmake

if (ARG_MODULE)
     set_target_properties(${name} PROPERTIES FOLDER "Loadable modules")
   else()
     set_target_properties(${name} PROPERTIES FOLDER "Libraries")
   endif()
 endmacro(add_llvm_library name)
 
 macro(add_llvm_executable name)
   cmake_parse_arguments(ARG
-    "DISABLE_LLVM_LINK_LLVM_DYLIB;IGNORE_EXTERNALIZE_DEBUGINFO;NO_INSTALL_RPATH;SUPPORT_PLUGINS"
+    "DISABLE_LLVM_LINK_LLVM_DYLIB;IGNORE_EXTERNALIZE_DEBUGINFO;NO_INSTALL_RPATH;SUPPORT_PLUGINS;GENERATE_DRIVER"
     "ENTITLEMENTS;BUNDLE_PATH"
     "DEPENDS"
     ${ARGN})
 
   llvm_process_sources( ALL_FILES ${ARG_UNPARSED_ARGUMENTS} )
 
   list(APPEND LLVM_COMMON_DEPENDS ${ARG_DEPENDS})
 
   # Generate objlib
-  if(LLVM_ENABLE_OBJLIB)
+  if(LLVM_ENABLE_OBJLIB OR ARG_GENERATE_DRIVER)
     # Generate an obj library for both targets.
     set(obj_name "obj.${name}")
     add_library(${obj_name} OBJECT EXCLUDE_FROM_ALL
       ${ALL_FILES}
       )
     llvm_update_compile_flags(${obj_name})
     set(ALL_FILES "$<TARGET_OBJECTS:${obj_name}>")
     if(ARG_DEPENDS)
       add_dependencies(${obj_name} ${ARG_DEPENDS})
     endif()
 
     set_target_properties(${obj_name} PROPERTIES FOLDER "Object Libraries")
   endif()
 
+  if (ARG_GENERATE_DRIVER)
+    string(REPLACE "-" "_" TOOL_NAME ${name})
+    configure_file(
+      ${LLVM_MAIN_SRC_DIR}/cmake/driver-template.cpp.in
+      ${CMAKE_CURRENT_BINARY_DIR}/${name}-driver.cpp)
+
+    list(APPEND ALL_FILES ${CMAKE_CURRENT_BINARY_DIR}/${name}-driver.cpp)
+
+    set_property(GLOBAL APPEND PROPERTY LLVM_DRIVER_COMPONENTS ${LLVM_LINK_COMPONENTS})
+    set_property(GLOBAL APPEND PROPERTY LLVM_DRIVER_DEPS ${ARG_DEPENDS} ${LLVM_COMMON_DEPENDS})
+    set_property(GLOBAL APPEND PROPERTY LLVM_DRIVER_OBJLIBS "${obj_name}")
+    
+    set_property(GLOBAL APPEND PROPERTY LLVM_DRIVER_TOOLS ${name})
+    target_link_libraries(${obj_name} ${LLVM_PTHREAD_LIB})
+    llvm_config(${obj_name} ${USE_SHARED} ${LLVM_LINK_COMPONENTS} )
+  endif()
+
   add_windows_version_resource_file(ALL_FILES ${ALL_FILES})
 
   if(XCODE)
     # Note: the dummy.cpp source file provides no definitions. However,
     # it forces Xcode to properly link the static library.
     list(APPEND ALL_FILES "${LLVM_MAIN_SRC_DIR}/cmake/dummy.cpp")
   endif()
 

 endfunction(create_llvm_tool_options)
 
 function(llvm_add_implicit_projects project)
   set(list_of_implicit_subdirs "")
   file(GLOB sub-dirs "${CMAKE_CURRENT_SOURCE_DIR}/*")
   foreach(dir ${sub-dirs})
     if(IS_DIRECTORY "${dir}" AND EXISTS "${dir}/CMakeLists.txt")
       canonicalize_tool_name(${dir} name)
+      # I don't like special casing things by order, but the llvm-driver ends up
+      # linking the object libraries from all the tools that opt-in, so adding
+      # it separately at the end is probably the simplest case.
+      if("${name}" STREQUAL "LLVM_DRIVER")
+        continue()
+      endif()
       if (${project}_TOOL_${name}_BUILD)
         get_filename_component(fn "${dir}" NAME)
         list(APPEND list_of_implicit_subdirs "${fn}")
       endif()
     endif()
   endforeach()
 
   foreach(external_proj ${list_of_implicit_subdirs})
add_llvm_install_targets(install-${name}
                              DEPENDS ${name} ${dest}
                              COMPONENT ${name}
                              SYMLINK ${dest})
   endif()
 endfunction()
 
 function(add_llvm_tool_symlink link_name target)
   cmake_parse_arguments(ARG "ALWAYS_GENERATE" "OUTPUT_DIR" "" ${ARGN})
+
+  get_property(LLVM_DRIVER_TOOLS GLOBAL PROPERTY LLVM_DRIVER_TOOLS)
+
+  if (${target} IN_LIST LLVM_DRIVER_TOOLS)
+    string(REPLACE "-" "_" tool_entry ${target})
+    string(REPLACE "-" "_" key ${link_name})
+    string(REPLACE "llvm-" "" tool_name ${link_name})
+    set_property(GLOBAL APPEND_STRING PROPERTY
+                 LLVM_EXTRA_DRIVER_ENTRIES "LLVM_DRIVER_TOOL(\"${tool_name}\", ${tool_entry})\n")
+  endif()
   set(dest_binary "$<TARGET_FILE:${target}>")
 
   # This got a bit gross... For multi-configuration generators the target
   # properties return the resolved value of the string, not the build system
   # expression. To reconstruct the platform-agnostic path we have to do some
   # magic. First we grab one of the types, and a type-specific path. Then from
   # the type-specific path we find the last occurrence of the type in the path,
   # and replace it with CMAKE_CFG_INTDIR. This allows the build step to be type

llvm/lib/Support/Path.cpp

 // Include the truly platform-specific parts.
 #if defined(LLVM_ON_UNIX)
 #include "Unix/Path.inc"
 #endif
 #if defined(_WIN32)
 #include "Windows/Path.inc"
 #endif
 
+bool IsLLVMDriver = false;
+
 namespace llvm {
 namespace sys {
 namespace fs {
+
+std::string getMainExecutable(const char *Argv0, void *MainAddr) {
+  if (IsLLVMDriver)
+    return sys::path::stem(Argv0).str();
+  return getMainExecutableImpl(Argv0, MainAddr);
+}
+
 TempFile::TempFile(StringRef Name, int FD)
     : TmpName(std::string(Name)), FD(FD) {}
 TempFile::TempFile(TempFile &&Other) { *this = std::move(Other); }
 TempFile &TempFile::operator=(TempFile &&Other) {
   TmpName = std::move(Other.TmpName);
   FD = Other.FD;
   Other.Done = true;
   Other.FD = -1;

llvm/lib/Support/Unix/Path.inc

getprogpath(char ret[PATH_MAX], const char *bin)
   }
   free(s);
   return nullptr;
 }
 #endif // __FreeBSD__ || __NetBSD__ || __FreeBSD_kernel__
 
 /// GetMainExecutable - Return the path to the main executable, given the
 /// value of argv[0] from program startup.
-std::string getMainExecutable(const char *argv0, void *MainAddr) {
+std::string getMainExecutableImpl(const char *argv0, void *MainAddr) {
 #if defined(__APPLE__)
   // On OS X the executable path is saved to the stack by dyld. Reading it
   // from there is much faster than calling dladdr, especially for large
   // binaries with symbols.
   char exe_path[PATH_MAX];
   uint32_t size = sizeof(exe_path);
   if (_NSGetExecutablePath(exe_path, &size) == 0) {
     char link_path[PATH_MAX];

llvm/lib/Support/Windows/Path.inc

 }
 
 } // end namespace windows
 
 namespace fs {
 
 const file_t kInvalidFile = INVALID_HANDLE_VALUE;
 
-std::string getMainExecutable(const char *argv0, void *MainExecAddr) {
+std::string getMainExecutableImpl(const char *argv0, void *MainExecAddr) {
   SmallVector<wchar_t, MAX_PATH> PathName;
   PathName.resize_for_overwrite(PathName.capacity());
   DWORD Size = ::GetModuleFileNameW(NULL, PathName.data(), PathName.size());
 
   // A zero return value indicates a failure other than insufficient space.
   if (Size == 0)
     return "";
 

llvm/test/CMakeLists.txt

llvm_canonicalize_cmake_booleans(
   LLVM_ENABLE_PLUGINS
   LLVM_BYE_LINK_INTO_TOOLS
   LLVM_HAVE_TF_AOT
   LLVM_HAVE_TF_API
   LLVM_INLINER_MODEL_AUTOGENERATED
   LLVM_RAEVICT_MODEL_AUTOGENERATED
   LLVM_ENABLE_EXPENSIVE_CHECKS
   LLVM_INCLUDE_DXIL_TESTS
+  LLVM_TOOL_LLVM_DRIVER_BUILD
   )
 
 configure_lit_site_cfg(
   ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.py.in
   ${CMAKE_CURRENT_BINARY_DIR}/lit.site.cfg.py
   MAIN_CONFIG
   ${CMAKE_CURRENT_SOURCE_DIR}/lit.cfg.py
   PATHS
set(LLVM_TEST_DEPENDS
           yaml-bench
           yaml2obj
         )
 
 if(TARGET llvm-lto)
   set(LLVM_TEST_DEPENDS ${LLVM_TEST_DEPENDS} llvm-lto)
 endif()
 
+if(TARGET llvm-driver)
+  set(LLVM_TEST_DEPENDS ${LLVM_TEST_DEPENDS} llvm-driver)
+endif()
+
 # If Intel JIT events are supported, depend on a tool that tests the listener.
 if( LLVM_USE_INTEL_JITEVENTS )
   set(LLVM_TEST_DEPENDS ${LLVM_TEST_DEPENDS} llvm-jitlistener)
 endif( LLVM_USE_INTEL_JITEVENTS )
 
 if(TARGET LLVMgold)
   set(LLVM_TEST_DEPENDS ${LLVM_TEST_DEPENDS} LLVMgold)
 endif()

llvm/test/lit.cfg.py

('%llvm-original-di-preservation', "'%s' %s" % (
         config.python_executable, llvm_original_di_preservation_cmd)))
 
 llvm_locstats_tool = os.path.join(config.llvm_tools_dir, 'llvm-locstats')
 config.substitutions.append(
     ('%llvm-locstats', "'%s' %s" % (config.python_executable, llvm_locstats_tool)))
 config.llvm_locstats_used = os.path.exists(llvm_locstats_tool)
 
 tools = [
+    ToolSubst('%llvm', FindTool('llvm')),
     ToolSubst('%lli', FindTool('lli'), post='.', extra_args=lli_args),
     ToolSubst('%llc_dwarf', FindTool('llc'), extra_args=llc_args),
     ToolSubst('%go', config.go_executable, unresolved='ignore'),
     ToolSubst('%gold', config.gold_executable, unresolved='ignore'),
     ToolSubst('%ld64', ld64_cmd, unresolved='ignore'),
     ToolSubst('%ocamlc', ocamlc_command, unresolved='ignore'),
     ToolSubst('%ocamlopt', ocamlopt_command, unresolved='ignore'),
     ToolSubst('%opt-viewer', opt_viewer_cmd),

 # LLVM can be configured with an empty default triple
 # Some tests are "generic" and require a valid default triple
 if config.target_triple:
     config.available_features.add('default_triple')
     # Direct object generation
     if not config.target_triple.startswith(("nvptx", "xcore")):
         config.available_features.add('object-emission')
 
+if config.have_llvm_driver:
+  config.available_features.add('llvm-driver')
+
 import subprocess
 
 
 def have_ld_plugin_support():
     if not os.path.exists(os.path.join(config.llvm_shlib_dir, 'LLVMgold' + config.llvm_shlib_ext)):
         return False
 
     ld_cmd = subprocess.Popen(

llvm/test/lit.site.cfg.py.in

 config.has_plugins = @LLVM_ENABLE_PLUGINS@
 config.linked_bye_extension = @LLVM_BYE_LINK_INTO_TOOLS@
 config.have_tf_aot = @LLVM_HAVE_TF_AOT@
 config.have_tf_api = @LLVM_HAVE_TF_API@
 config.llvm_inliner_model_autogenerated = @LLVM_INLINER_MODEL_AUTOGENERATED@
 config.llvm_raevict_model_autogenerated = @LLVM_RAEVICT_MODEL_AUTOGENERATED@
 config.expensive_checks = @LLVM_ENABLE_EXPENSIVE_CHECKS@
 config.dxil_tests = @LLVM_INCLUDE_DXIL_TESTS@
+config.have_llvm_driver = @LLVM_TOOL_LLVM_DRIVER_BUILD@
 
 import lit.llvm
 lit.llvm.initialize(lit_config, config)
 
 # Let the main config do the real work.
 lit_config.load_config(
     config, os.path.join(config.llvm_src_root, "test/lit.cfg.py"))

llvm/test/tools/llvm-driver/help-passthrough.test

+# REQUIRES: llvm-driver
+# RUN: %llvm cxxfilt --help | FileCheck %s 
+# CHECK: USAGE: cxxfilt

llvm/test/tools/llvm-driver/help.test

+# REQUIRES: llvm-driver
+# RUN: %llvm --help | FileCheck %s
+# CHECK: USAGE: llvm [subcommand]

llvm/test/tools/llvm-driver/symlink-call.test

+## Don't make symlinks on Windows.
+# UNSUPPORTED: system-windows
+# REQUIRES: llvm-driver
+
+# RUN: rm -rf %t
+# RUN: mkdir %t
+# RUN: ln -s %llvm %t/llvm-cxxfilt
+# RUN: %t/llvm-cxxfilt --help | FileCheck %s
+# RUN: ln -s %llvm %t/llvm-cxxfilt-15
+# RUN: %t/llvm-cxxfilt-15 --help | FileCheck %s
+# RUN: ln -s %llvm %t/cxxfilt
+# RUN: %t/cxxfilt --help | FileCheck %s
+# RUN: ln -s %llvm %t/cxxfilt-15
+# RUN: %t/cxxfilt-15 --help | FileCheck %s
+# RUN: ln -s %llvm %t/cxxfilt-15.exe
+# RUN: %t/cxxfilt-15.exe --help | FileCheck %s
+
+# RUN: ln -s %llvm %t/llvm-15
+# RUN: %t/llvm-15 cxxfilt --help | FileCheck %s
+# RUN: ln -s %llvm %t/llvm-15.exe
+# RUN: %t/llvm-15.exe cxxfilt --help | FileCheck %s
+
+# CHECK: OVERVIEW: LLVM symbol undecoration tool

llvm/tools/CMakeLists.txt

 add_llvm_external_project(polly)
 
 # Add subprojects specified using LLVM_EXTERNAL_PROJECTS
 foreach(p ${LLVM_EXTERNAL_PROJECTS})
   add_llvm_external_project(${p})
 endforeach(p)
 
 set(LLVM_COMMON_DEPENDS ${LLVM_COMMON_DEPENDS} PARENT_SCOPE)
+
+if (LLVM_TOOL_LLVM_DRIVER_BUILD)
+  # This is explicitly added at the end _after_ all tool projects so that it can
+  # scrape up tools from other projects into itself.
+  add_subdirectory(llvm-driver)
+endif()

llvm/tools/dsymutil/CMakeLists.txt

add_llvm_tool(dsymutil
   MachODebugMapParser.cpp
   MachOUtils.cpp
   Reproducer.cpp
   SymbolMap.cpp
 
   DEPENDS
   intrinsics_gen
   ${tablegen_deps}
+  DsymutilTableGen
+  GENERATE_DRIVER
   )
 
 if(APPLE)
   target_link_libraries(dsymutil PRIVATE "-framework CoreFoundation")
 endif(APPLE)
 
 if(HAVE_CXX_ATOMICS_WITH_LIB OR HAVE_CXX_ATOMICS64_WITH_LIB)
   target_link_libraries(dsymutil PRIVATE atomic)
 endif()

llvm/tools/dsymutil/dsymutil.cpp

getOutputFileName(StringRef InputFile, const DsymutilOptions &Options) {
   }
 
   sys::path::append(Path, "Contents", "Resources");
   std::string ResourceDir = std::string(Path.str());
   sys::path::append(Path, "DWARF", sys::path::filename(DwarfFile));
   return OutputLocation(std::string(Path.str()), ResourceDir);
 }
 
-int main(int argc, char **argv) {
+int dsymutil_main(int argc, char **argv) {
   InitLLVM X(argc, argv);
 
   // Parse arguments.
   DsymutilOptTable T;
   unsigned MAI;
   unsigned MAC;
   ArrayRef<const char *> ArgsArr = makeArrayRef(argv + 1, argc - 1);
   opt::InputArgList Args = T.ParseArgs(ArgsArr, MAI, MAC);

llvm/tools/llvm-ar/CMakeLists.txt

set(LLVM_LINK_COMPONENTS
   Support
   )
 
 add_llvm_tool(llvm-ar
   llvm-ar.cpp
 
   DEPENDS
   intrinsics_gen
+  GENERATE_DRIVER
   )
 
 add_llvm_tool_symlink(llvm-ranlib llvm-ar)
 add_llvm_tool_symlink(llvm-lib llvm-ar)
 add_llvm_tool_symlink(llvm-dlltool llvm-ar)
 
 if(LLVM_INSTALL_BINUTILS_SYMLINKS)
   add_llvm_tool_symlink(ar llvm-ar)
   add_llvm_tool_symlink(dlltool llvm-ar)
   add_llvm_tool_symlink(ranlib llvm-ar)
 endif()

llvm/tools/llvm-ar/llvm-ar.cpp

for (int i = 1; i < argc; ++i) {
     }
   }
   if (!ArchiveSpecified) {
     badUsage("an archive name must be specified");
   }
   return performOperation(CreateSymTab, nullptr);
 }
 
-int main(int argc, char **argv) {
+int llvm_ar_main(int argc, char **argv) {
   InitLLVM X(argc, argv);
   ToolName = argv[0];
 
   llvm::InitializeAllTargetInfos();
   llvm::InitializeAllTargetMCs();
   llvm::InitializeAllAsmParsers();
 
   Stem = sys::path::stem(ToolName);

llvm/tools/llvm-cxxfilt/CMakeLists.txt

 set(LLVM_LINK_COMPONENTS
   Demangle
   Option
   Support
   )
 
 set(LLVM_TARGET_DEFINITIONS Opts.td)
 tablegen(LLVM Opts.inc -gen-opt-parser-defs)
 add_public_tablegen_target(CxxfiltOptsTableGen)
 
 add_llvm_tool(llvm-cxxfilt
   llvm-cxxfilt.cpp
 
   DEPENDS
   CxxfiltOptsTableGen
+  GENERATE_DRIVER
   )
 
 if(LLVM_INSTALL_BINUTILS_SYMLINKS)
   add_llvm_tool_symlink(c++filt llvm-cxxfilt)
 endif()

llvm/tools/llvm-cxxfilt/llvm-cxxfilt.cpp

if (Split) {
     for (const auto &Word : Words)
       Result += ::demangle(std::string(Word.first)) + Word.second.str();
   } else
     Result = ::demangle(std::string(Mangled));
   OS << Result << '\n';
   OS.flush();
 }
 
-int main(int argc, char **argv) {
+int llvm_cxxfilt_main(int argc, char **argv) {
   InitLLVM X(argc, argv);
   BumpPtrAllocator A;
   StringSaver Saver(A);
   CxxfiltOptTable Tbl;
   ToolName = argv[0];
   opt::InputArgList Args = Tbl.parseArgs(argc, argv, OPT_UNKNOWN, Saver,
                                          [&](StringRef Msg) { error(Msg); });
   if (Args.hasArg(OPT_help)) {

llvm/tools/llvm-driver/CMakeLists.txt

+get_property(LLVM_COMMON_DEPENDS GLOBAL PROPERTY LLVM_DRIVER_DEPS)
+get_property(LLVM_DRIVER_OBJLIBS GLOBAL PROPERTY LLVM_DRIVER_OBJLIBS)
+
+get_property(LLVM_DRIVER_TOOLS GLOBAL PROPERTY LLVM_DRIVER_TOOLS)
+
+foreach(tool ${LLVM_DRIVER_TOOLS})
+  string(REPLACE "-" "_" tool_entry ${tool})
+  string(REPLACE "llvm-" "" tool ${tool})
+  set(def_decl "${def_decl}LLVM_DRIVER_TOOL(\"${tool}\", ${tool_entry})\n")
+endforeach()
+
+get_property(LLVM_EXTRA_DRIVER_ENTRIES GLOBAL PROPERTY LLVM_EXTRA_DRIVER_ENTRIES)
+
+file(WRITE
+          "${CMAKE_CURRENT_BINARY_DIR}/LLVMDriverTools.def"
+          "${def_decl}${LLVM_EXTRA_DRIVER_ENTRIES}#undef LLVM_DRIVER_TOOL\n")
+
+include_directories(${CMAKE_CURRENT_BINARY_DIR})
+
+add_llvm_tool(llvm-driver
+  llvm-driver.cpp
+  )
+
+set_target_properties(llvm-driver PROPERTIES OUTPUT_NAME llvm)
+
+target_link_libraries(llvm-driver PUBLIC ${LLVM_DRIVER_OBJLIBS})
+
+if(APPLE)
+  # dsymutil uses some CoreFoundation stuff on Darwin...
+  target_link_libraries(llvm-driver PRIVATE "-framework CoreFoundation")
+endif(APPLE)

llvm/tools/llvm-driver/llvm-driver.cpp

+//===-- llvm-driver.cpp ---------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ADT/StringExtras.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringSwitch.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/ErrorHandling.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/WithColor.h"
+
+using namespace llvm;
+
+#define LLVM_DRIVER_TOOL(tool, entry) int entry##_main(int argc, char **argv);
+#include "LLVMDriverTools.def"
+
+constexpr char subcommands[] =
+#define LLVM_DRIVER_TOOL(tool, entry) "  " tool "\n"
+#include "LLVMDriverTools.def"
+    ;
+
+static void printHelpMessage() {
+  llvm::outs() << "OVERVIEW: llvm toolchain driver\n\n"
+               << "USAGE: llvm [subcommand] [options]\n\n"
+               << "SUBCOMMANDS:\n\n"
+               << subcommands
+               << "\n  Type \"llvm <subcommand> --help\" to get more help on a "
+                  "specific subcommand\n\n"
+               << "OPTIONS:\n\n  --help - Display this message";
+}
+
+static int findTool(int Argc, char **Argv) {
+  if (!Argc) {
+    printHelpMessage();
+    return 1;
+  }
+
+  StringRef ToolName = Argv[0];
+
+  if (ToolName == "--help") {
+    printHelpMessage();
+    return 0;
+  }
+
+  StringRef Stem = sys::path::stem(ToolName);
+  auto Is = [=](StringRef Tool) {
+    auto I = Stem.rfind_insensitive(Tool);
+    return I != StringRef::npos && (I + Tool.size() == Stem.size() ||
+                                    !llvm::isAlnum(Stem[I + Tool.size()]));
+  };
+
+#define LLVM_DRIVER_TOOL(tool, entry)                                          \
+  if (Is(tool))                                                                \
+    return entry##_main(Argc, Argv);
+#include "LLVMDriverTools.def"
+
+  if (Is("llvm"))
+    return findTool(Argc - 1, Argv + 1);
+
+  printHelpMessage();
+  return 1;
+}
+
+extern bool IsLLVMDriver;
+
+int main(int Argc, char **Argv) {
+  IsLLVMDriver = true;
+  return findTool(Argc, Argv);
+}

llvm/tools/llvm-objcopy/CMakeLists.txt

 
 add_llvm_tool(llvm-objcopy
   ObjcopyOptions.cpp
   llvm-objcopy.cpp
   DEPENDS
   ObjcopyOptsTableGen
   InstallNameToolOptsTableGen
   StripOptsTableGen
+  GENERATE_DRIVER
   )
 
 add_llvm_tool_symlink(llvm-install-name-tool llvm-objcopy)
 add_llvm_tool_symlink(llvm-bitcode-strip llvm-objcopy)
 add_llvm_tool_symlink(llvm-strip llvm-objcopy)
 
 if(LLVM_INSTALL_BINUTILS_SYMLINKS)
   add_llvm_tool_symlink(objcopy llvm-objcopy)
   add_llvm_tool_symlink(strip llvm-objcopy)
 endif()
 
 if(LLVM_INSTALL_CCTOOLS_SYMLINKS)
   add_llvm_tool_symlink(install_name_tool llvm-install-name-tool)
   add_llvm_tool_symlink(bitcode_strip llvm-bitcode-strip)
 endif()

llvm/tools/llvm-objcopy/llvm-objcopy.cpp

if (!Config.SplitDWO.empty())
     if (Error E =
             PermsApplierOrErr->apply(Config.SplitDWO, Config.PreserveDates,
                                      static_cast<sys::fs::perms>(0666)))
       return E;
 
   return Error::success();
 }
 
-int main(int argc, char **argv) {
+int llvm_objcopy_main(int argc, char **argv) {
   InitLLVM X(argc, argv);
   ToolName = argv[0];
 
   // Expand response files.
   // TODO: Move these lines, which are copied from lib/Support/CommandLine.cpp,
   // into a separate function in the CommandLine library and call that function
   // here. This is duplicated code.
   SmallVector<const char *, 20> NewArgv(argv, argv + argc);

utils/bazel/llvm-project-overlay/clang/BUILD.bazel

 # This file is licensed under the Apache License v2.0 with LLVM Exceptions.
 # See https://llvm.org/LICENSE.txt for license information.
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 
 load("//llvm:tblgen.bzl", "gentbl")
 load("//llvm:binary_alias.bzl", "binary_alias")
 load("//llvm:cc_plugin_library.bzl", "cc_plugin_library")
+load("//llvm:template_rule.bzl", "template_rule")
 
 package(
     default_visibility = ["//visibility:public"],
     licenses = ["notice"],
 )
 
 exports_files([
     "tools/clang-format/clang-format.el",
deps = [
         ":frontend",
         ":lex",
         ":parse",
         "//llvm:Core",
         "//llvm:Support",
     ],
 )
 
+template_rule(
+    name = "clang_main",
+    src = "//llvm:cmake/driver-template.cpp.in",
+    out = "clang_main.cpp",
+    substitutions = {
+        "@TOOL_NAME@": "clang"
+    },
+)
+
 cc_library(
     name = "clang-driver",
     srcs = glob([
         "tools/driver/*.cpp",
         "tools/driver/*.h",
-    ]),
+    ]) + ["clang_main.cpp"],
     copts = [
         # Disable stack frame size checks in the driver because
         # clang::ensureStackAddressSpace allocates a large array on the stack.
         "$(STACK_FRAME_UNLIMITED)",
     ],
     deps = [
         ":analysis",
         ":ast",

utils/bazel/llvm-project-overlay/llvm/BUILD.bazel

tbl_outs = [(
         "-gen-opt-parser-defs",
         "tools/dsymutil/Options.inc",
     )],
     tblgen = ":llvm-tblgen",
     td_file = "tools/dsymutil/Options.td",
     td_srcs = ["include/llvm/Option/OptParser.td"],
 )
 
+template_rule(
+    name = "dsymutil_main",
+    src = "cmake/driver-template.cpp.in",
+    out = "dsymutil_main.cpp",
+    substitutions = {
+        "@TOOL_NAME@": "dsymutil"
+    },
+)
+
 cc_binary(
     name = "dsymutil",
     srcs = glob([
         "tools/dsymutil/*.cpp",
         "tools/dsymutil/*.h",
-    ]),
+    ]) + ["dsymutil_main.cpp"],
     copts = llvm_copts,
     stamp = 0,
     deps = [
         ":AllTargetsCodeGens",
         ":BinaryFormat",
         ":CodeGen",
         ":DWARFLinker",
         ":DebugInfo",
deps = [
         ":MCJIT",
         ":Object",
         ":OrcJIT",
         ":Support",
         ":config",
     ],
 )
 
+template_rule(
+    name = "ar_main",
+    src = "cmake/driver-template.cpp.in",
+    out = "ar_main.cpp",
+    substitutions = {
+        "@TOOL_NAME@": "llvm_ar"
+    },
+)
+
 cc_binary(
     name = "llvm-ar",
     srcs = glob([
         "tools/llvm-ar/*.cpp",
         "tools/llvm-ar/*.h",
-    ]),
+    ]) + ["ar_main.cpp"],
     copts = llvm_copts,
     stamp = 0,
     deps = [
         ":AllTargetsAsmParsers",
         ":AllTargetsCodeGens",
         ":Core",
         ":DlltoolDriver",
         ":LibDriver",
tbl_outs = [(
         "-gen-opt-parser-defs",
         "tools/llvm-cxxfilt/Opts.inc",
     )],
     tblgen = ":llvm-tblgen",
     td_file = "tools/llvm-cxxfilt/Opts.td",
     td_srcs = ["include/llvm/Option/OptParser.td"],
 )
 
+template_rule(
+    name = "cxxfilt_main",
+    src = "cmake/driver-template.cpp.in",
+    out = "cxxfilt_main.cpp",
+    substitutions = {
+        "@TOOL_NAME@": "llvm_cxxfilt"
+    },
+)
+
 cc_binary(
     name = "llvm-cxxfilt",
     srcs = glob([
         "tools/llvm-cxxfilt/*.cpp",
         "tools/llvm-cxxfilt/*.h",
-    ]),
+    ]) + ["cxxfilt_main.cpp"],
     copts = llvm_copts,
     stamp = 0,
     deps = [
         ":CxxfiltOptsTableGen",
         ":Demangle",
         ":Option",
         ":Support",
     ],
cc_binary(
     copts = llvm_copts,
     stamp = 0,
     deps = [
         ":Core",
         ":Support",
     ],
 )
 
+template_rule(
+    name = "objcopy_main",
+    src = "cmake/driver-template.cpp.in",
+    out = "objcopy_main.cpp",
+    substitutions = {
+        "@TOOL_NAME@": "llvm_objcopy"
+    },
+)
+
+
 cc_binary(
     name = "llvm-objcopy",
     srcs = glob([
         "tools/llvm-objcopy/*.cpp",
         "tools/llvm-objcopy/*.h",
-    ]),
+    ]) + ["objcopy_main.cpp"],
     copts = llvm_copts,
     stamp = 0,
     deps = [
         ":BinaryFormat",
         ":MC",
         ":ObjCopy",
         ":Object",
         ":ObjectYAML",