This is an archive of the discontinued LLVM Phabricator instance.

Make lld-link work in a non-MSVC shell
ClosedPublic

Authored by pkasting on Jan 24 2022, 1:05 PM.

Details

Summary

Makes lld-link work in a non-MSVC shell by autodetecting MSVC toolchain. Also adds support for /winsysroot and a few other switches.

All this is done by refactoring to share code with clang-cl's existing support for the same.

Diff Detail

Event Timeline

pkasting created this revision.Jan 24 2022, 1:05 PM
pkasting requested review of this revision.Jan 24 2022, 1:05 PM
Herald added a project: Restricted Project. · View Herald TranscriptJan 24 2022, 1:05 PM
thakis added a subscriber: ychen.Jan 25 2022, 1:03 PM

Very very cool.

This needs a test. Maybe you just forgot to git add it?

It'd be nice if the knowledge what the sysroot layout looks like (a Windows\ Kits/10/... dir, a VC/Tools/MSVC/*/ dir, optionally a DIA SDK dir) wasn't duplicated between here and clang. Could we pull that out into some common code in llvm/lib/Support and call that in both places?

(@ychen fyi wrt D117852)

lld/COFF/Driver.cpp
273

Include seems like the wrong directory to look in in the linker (…i think?)

670

there's a bunch of these unrelated sys:: removals sprinkled through the patch. send them in a separate patch, they're unrelated.

703

We shouldn't look at %LIB% here when /winsysroot: is passed.

pkasting updated this revision to Diff 403265.Jan 26 2022, 7:40 AM

Adds autodetection and full support for other clang-cl switches, but doesn't work: config->machine is not set this early, so it's always UNKNOWN.

Herald added a project: Restricted Project. · View Herald TranscriptJan 26 2022, 7:40 AM
pkasting updated this revision to Diff 403316.Jan 26 2022, 9:32 AM
pkasting edited the summary of this revision. (Show Details)Jan 26 2022, 9:32 AM

Functional, but not yet refactored to share code with clang-cl.

pkasting updated this revision to Diff 403679.Jan 27 2022, 8:47 AM

Refactored to share code with clang-cl. Still no tests.

This looks super nice.

Things that can still improve:

  1. This now does way more than just /winsysroot:. It also makes it so that lld-link works in a non-msvc shell by looking up libpaths in either registry or via setup api.
  2. Some of the new lld-link code still looks (from a distance, might be wrong) like it could be shared more
  3. Still needs tests.

@mstorsjo, fyi.

clang/docs/tools/clang-formatted-files.txt
1 ↗(On Diff #403679)

TIL about the existence of clang/docs/tools/clang-formatted-files.txt :)

clang/lib/Driver/ToolChains/MSVC.cpp
465 ↗(On Diff #403679)

Should these be member variables? Looks like we need them in a bunch of places.

lld/COFF/Driver.cpp
230

this function looks kind of similar to one in clang. Maybe the arch paths could be passed in and then this could be shared (?)

703

Oh I see, this is now no longer called if winsysroot is passed.

722

Do you think it's possible to get this logic here (this path if ucrt else this other path, etc) shared between the two places? Maybe a getLibPaths() function that's passed arch etc?

lld/COFF/SymbolTable.cpp
59 ↗(On Diff #403679)

I suppose this is good enough :)

If we wanted to get fancy, we could put in code to explicitly detect the case of a .lib file not being found, and addWinSysRootLibSearchPaths() not having been called yet (due to the machine type not yet being know), and then diag something like ".lib not found. you passed /winsysroot:, try passing /machine: too". But I don't think this will ever happen in practice, so I think it's not worth worrying about.

utils/bazel/llvm-project-overlay/clang/BUILD.bazel
1314 ↗(On Diff #403679)

(fwiw you don't have to update BUILD.bazel files)

pkasting marked 9 inline comments as done.Feb 1 2022, 10:27 AM

Things that can still improve:

  1. This now does way more than just /winsysroot:. It also makes it so that lld-link works in a non-msvc shell by looking up libpaths in either registry or via setup api.

Updated description.

  1. Some of the new lld-link code still looks (from a distance, might be wrong) like it could be shared more

Did what I think I could, see details in comments.

  1. Still needs tests.

Added a few basic tests, please review.

lld/COFF/Driver.cpp
172

Found a way to share these.

273

I think this is OK. It's not obvious to me that we can change this to a .lib check safely, and I think clang and lld should use the same logic to detect ucrt.

703

Yep, added a test for this too.

722

The big problem is that in lld this logic is split between the two phases (pre-run() and post-run()) and can't easily be moved entirely into the post-run() phase. So we'd only be able to share the logic that is here in the pre-run phase (the code that doesn't need the arch), which would make the clang code quite cumbersome.

I think it's not worth trying.

766

Found a way to share this.

lld/COFF/SymbolTable.cpp
59 ↗(On Diff #403679)

I think this can only happen for defaultlibs? ...and those get processed after we set the machine type now, so there can't be a problem here. Or at least, I couldn't find a way to trigger one in practice -- all my attempts led the linker to instead complain "No input files." So I'm gonna call this N/A, but if you think of a command line to trigger it so I can test it, let me know.

utils/bazel/llvm-project-overlay/clang/BUILD.bazel
1314 ↗(On Diff #403679)

Shrug, it's not hard to do so. And already done now, so...

pkasting updated this revision to Diff 404994.Feb 1 2022, 10:30 AM
pkasting marked 5 inline comments as done.
pkasting retitled this revision from Add /winsysroot support to lld to Make lld-link work in a non-MSVC shell.
pkasting edited the summary of this revision. (Show Details)
ychen added a comment.Feb 1 2022, 5:35 PM

Thanks for doing this! Update the release note?

MaskRay added subscribers: compnerd, MaskRay.EditedFeb 1 2022, 5:40 PM

I know that you want a place to be accessed by both clang driver and lld-link but I am a bit nervous about the clang-driver style MSVC library sitting inside llvm/lib/Support/.
Is there a better place? @compnerd @aganea

MaskRay added a subscriber: aganea.Feb 1 2022, 6:00 PM
aganea added a comment.Feb 2 2022, 6:11 AM

I know that you want a place to be accessed by both clang driver and lld-link but I am a bit nervous about the clang-driver style MSVC library sitting inside llvm/lib/Support/.
Is there a better place? @compnerd @aganea

Does it make sense to add a new platform library? LLVMPlatformWindows? LLVMWindows? LLVMWindowsTools? LLVMWindowsABI?

Makes lld-link work in a non-MSVC shell by autodetecting MSVC toolchain. Also adds support for /winsysroot and a few other switches.

@pkasting Note, this patch makes the outcome non-deterministic when the auto-detection kicks in. We are stamping the command-line the .PDB file, see https://github.com/llvm/llvm-project/blob/main/lld/COFF/PDB.cpp#L1402 - and some live-code-patching tools rely on that to reproduce the link. It'd be nice if the detected paths are queued to config->argv, to ensure that we can emit a self-standing LLD cmd-line, a bit like what clang -cc1 does.

lld/COFF/Driver.h
118

You could also use saver().save(...) below and keep this member intact. That would save some memory allocations.

I know that you want a place to be accessed by both clang driver and lld-link but I am a bit nervous about the clang-driver style MSVC library sitting inside llvm/lib/Support/.
Is there a better place? @compnerd @aganea

Does it make sense to add a new platform library? LLVMPlatformWindows? LLVMWindows? LLVMWindowsTools? LLVMWindowsABI?

There is a precedent: llvm/lib/WindowsManifest. So having a library in llvm/lib/ seems fine. I do not know whether it makes sense to combine the Windows driver library with WindowsManifest.

thakis added a comment.Feb 4 2022, 1:10 PM

@pkasting Note, this patch makes the outcome non-deterministic when the auto-detection kicks in. We are stamping the command-line the .PDB file, see https://github.com/llvm/llvm-project/blob/main/lld/COFF/PDB.cpp#L1402 - and some live-code-patching tools rely on that to reproduce the link. It'd be nice if the detected paths are queued to config->argv, to ensure that we can emit a self-standing LLD cmd-line, a bit like what clang -cc1 does.

You mean "non-deterministic" in "cmd.exe / local-computer dependent", yes? If so: There are enough flags that you can pass in to make the output deterministic if this is something you care about. (We care about determinism a lot over in chromium land!) I think the setup matches what we have in clang, right?

pkasting marked an inline comment as done.Feb 4 2022, 2:03 PM

Thanks for doing this! Update the release note?

Done.

I think the setup matches what we have in clang, right?

Yes, I think so.

pkasting updated this revision to Diff 406101.Feb 4 2022, 2:05 PM
pkasting updated this revision to Diff 406526.Feb 7 2022, 10:54 AM

This fixes a failing test.

thakis accepted this revision.Feb 7 2022, 11:04 AM

I think this looks great.

maskray, I kind of see where you're coming from. And I'm _very_ sympathetic to keeping Support small(er). On the other hand, this is a single file that doesn't require any additional dependencies on libs (…I think), so a dedicated library feels a bit overkill to me, maybe. (Are there any "when not to put your stuff in Support" guidelines anywhere?)

lld/COFF/Driver.cpp
187

Maybe this function could be in the shared part too? Looks like basically exactly this code is in both places.

lld/docs/ReleaseNotes.rst
38 ↗(On Diff #406526)

Maybe mention /winsysroot: here too

This revision is now accepted and ready to land.Feb 7 2022, 11:04 AM
pkasting updated this revision to Diff 406570.Feb 7 2022, 12:24 PM
pkasting marked 2 inline comments as done.

Review comments.

MaskRay: Friendly ping.

MaskRay edited reviewers, added: aganea, mstorsjo; removed: MaskRay.Feb 9 2022, 9:56 AM

MaskRay: Friendly ping.

I think I still feel that having this in LLVMSupport is strange, but am happy if the two other folks I added are happy...

This revision was landed with ongoing or failed builds.Feb 11 2022, 10:55 AM
This revision was automatically updated to reflect the committed changes.

I went ahead and landed this for now. It's easy to move the 3 new files to their own library later on if we decide it's something we want to do.

Thanks for the patch!

Unfortunately this seems to break the modular build. Would you mind taking a look? I'm going to revert the patch to get the bots going again.

https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/41306/console

llvm/include/llvm/Support/MSVCPaths.h
15 ↗(On Diff #407955)

This introduces a cyclic dependency that breaks the -DLLVM_ENABLE_MODULES=1 build.

/Volumes/Data/llvm-project/llvm/include/llvm/Option/Arg.h:17:10: fatal error: cyclic dependency in module 'LLVM_Utils': LLVM_Utils -> LLVM_Option -> LLVM_Utils
#include "llvm/ADT/SmallVector.h"
         ^
While building module 'LLVM_Utils' imported from /Volumes/Data/llvm-project/llvm/lib/Support/ARMBuildAttrs.cpp:9:
In file included from <module-includes>:195:
/Volumes/Data/llvm-project/llvm/include/llvm/Support/MSVCPaths.h:14:10: fatal error: could not build module 'LLVM_Option'
#include "llvm/Option/ArgList.h"
 ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~
/Volumes/Data/llvm-project/llvm/lib/Support/ARMBuildAttrs.cpp:9:10: fatal error: could not build module 'LLVM_Utils'
#include "llvm/Support/ARMBuildAttributes.h"
 ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
3 errors generated.

Thanks for the revert!

Is there any documentation for that LLVM_ENABLE_MODULES build config anywhere?

thakis added inline comments.Feb 11 2022, 1:25 PM
llvm/include/llvm/Support/MSVCPaths.h
15 ↗(On Diff #407955)

Ah yeah, support can't depend on Option. Maybe the easiest fix is to just move this to its own library like maskray suggested. Then this isn't a problem.

Relanded in 73e585e44d68cf77e2e3274e98c9615156a7d909, with the new files in a new WindowsDriver library.

dyung added a subscriber: dyung.Feb 11 2022, 4:17 PM

I'm not exactly sure why, but this change seems to be causing the Driver/cl-options.c test to fail on the PS4 Windows bot:

https://lab.llvm.org/staging/#/builders/204/builds/1343

It was also failing previously after the original commit until it was reverted at which point it started to pass again.

It fails on this:

// Validate that the default triple is used when run an empty tools dir is specified
// RUN: %clang_cl -vctoolsdir "" -### -- %s 2>&1 | FileCheck %s --check-prefix VCTOOLSDIR
// VCTOOLSDIR: "-triple" "{{[a-zA-Z0-9_-]*}}-pc-windows-msvc19.20.0"

With this:

C:\buildbot-root\llvm-clang-x86_64-sie-win\llvm-project\clang\test\Driver\cl-options.c:761:16: error: VCTOOLSDIR: expected string not found in input
// VCTOOLSDIR: "-triple" "{{[a-zA-Z0-9_-]*}}-pc-windows-msvc19.20.0"
               ^
<stdin>:1:1: note: scanning from here
clang version 15.0.0 (https://github.com/llvm/llvm-project.git 0574b5fc657451c9d13d3f6d8fe14ea15c23a681)
^
<stdin>:5:68: note: possible intended match here
 "c:\\buildbot-root\\llvm-clang-x86_64-sie-win\\build\\bin\\clang.exe" "-cc1" "-triple" "x86_64-pc-windows-msvc19.29.30137" ...

That test was added in D85998.

I think this is a true regression. findVCToolChainViaCommandLine() used to return true on present-but-empty strings, but llvm::findVCToolChainViaCommandLine() uses the empty string as "not set".

The fix is probably to make llvm::findVCToolChainViaCommandLine() take Optional<StringRef>s instead, and use the Optional value instead of string emptiness.

It's late over here, so I won't get to that myself today. Feel free to revert for now.

I've reverted this and the follow-up change to fix the build in 437d4e01fe4c057509dff30efd560049ad07bc99.

thakis reopened this revision.Feb 13 2022, 8:39 AM

(reopening for fixing things + relanding)

This revision is now accepted and ready to land.Feb 13 2022, 8:39 AM
thakis requested changes to this revision.Feb 13 2022, 8:40 AM

(also marking as "request changes" for the Optional<StringRef> change we need)

This revision now requires changes to proceed.Feb 13 2022, 8:40 AM
pkasting updated this revision to Diff 409046.Feb 15 2022, 1:44 PM

Fix test failure.

thakis accepted this revision.Feb 16 2022, 6:17 AM

Thanks! Relanding.

This revision is now accepted and ready to land.Feb 16 2022, 6:17 AM
This revision was landed with ongoing or failed builds.Feb 16 2022, 6:31 AM
This revision was automatically updated to reflect the committed changes.

The bot that broke last time stayed green on the reland: https://lab.llvm.org/staging/#/builders/204/builds/1591 :)