[lld/mac] Support writing zippered dylibs and bundles

[thakis]

Summary

With -platform_version flags for two distinct platforms,
this writes a LC_BUILD_VERSION header for each.

The motivation is that this is needed for self-hosting with lld as linker
after D124059.

To create a zippered output at the clang driver level, pass

-target arm64-apple-macos -darwin-target-variant arm64-apple-ios-macabi

to create a zippered dylib.

(In Xcode's clang, -darwin-target-variant is spelled just -target-variant.)

(If you pass -target arm64-apple-ios-macabi -target-variant arm64-apple-macos
instead, ld64 crashes!)

This results in two -platform_version flags being passed to the linker.

ld64 also verifies that the iOS SDK version is at least 13.1. We don't do that
yet. But ld64 also does that for other platforms and we don't. So we need to
do that at some point, but not in this patch.

Only dylib and bundle outputs can be zippered.

I verified that a Catalyst app linked against a dylib created with

clang -shared foo.cc -o libfoo.dylib \
      -target arm64-apple-macos \
      -target-variant arm64-apple-ios-macabi \
      -Wl,-install_name,@rpath/libfoo.dylib \
      -fuse-ld=$PWD/out/gn/bin/ld64.lld

runs successfully. (The app calls a function f() in libfoo.dylib
that returns a const char* "foo", and NSLog(@"%s")s it.)

ld64 is a bit more permissive when writing zippered outputs,
see references to "unzippered twins". That's not implemented yet.
(If anybody wants to implement that, D124275 is a good start.)

[thakis]

lld/MachO/Driver.cpp

650

ld64 allows this, but it's not used for anything as far as I can tell. So let's not allow it for now.

653

This is just a warning in ld64 (it uses the last flag in that case). Let's see if we can make it an error.

1150

Moved up so that it's set in time for the new outputType check in parsePlatformVersions

lld/test/MachO/zippered.yaml

60

This matches ld64.

[jhenderson]

I think otool changes should be split off into a separate patch. They should also be tested and documented in isolation.

[thakis]

Sure thing, moved that to D124920

[thakis]

rebase out otool change

[int3]

lgtm!

[thakis]

Thanks :) Want to stamp D124920 too?

lld/MachO/Config.h

struct Configuration {
   uint32_t ltoo = 2;
   llvm::CachePruningPolicy thinLTOCachePolicy;
   llvm::StringRef thinLTOCacheDir;
   bool deadStripDylibs = false;
   bool demangle = false;
   bool deadStrip = false;
   bool errorForArchMismatch = false;
   PlatformInfo platformInfo;
+  llvm::Optional<PlatformInfo> secondaryPlatformInfo;
   NamespaceKind namespaceKind = NamespaceKind::twolevel;
   UndefinedSymbolTreatment undefinedSymbolTreatment =
       UndefinedSymbolTreatment::error;
   ICFLevel icfLevel = ICFLevel::none;
   llvm::MachO::HeaderFileType outputType;
   std::vector<llvm::StringRef> systemLibraryRoots;
   std::vector<llvm::StringRef> librarySearchPaths;
   std::vector<llvm::StringRef> frameworkSearchPaths;

lld/MachO/Driver.cpp

if (platformVersions.empty()) {
     error("must specify -platform_version");
     return PLATFORM_UNKNOWN;
   }
   if (platformVersions.size() > 2) {
     error("must specify -platform_version at most twice");
     return PLATFORM_UNKNOWN;
   }
   if (platformVersions.size() == 2) {
-    // FIXME: If you implement support for this, add a diagnostic if
-    // outputType is not dylib or bundle -- linkers shouldn't be able to
-    // write zippered executables.
-    warn("writing zippered outputs not yet implemented, "
-         "ignoring all but last -platform_version flag");
+    bool isZipperedCatalyst = platformVersions.count(PLATFORM_MACOS) &&
+                              platformVersions.count(PLATFORM_MACCATALYST);
+
+    if (!isZipperedCatalyst) {
+      error("lld supports writing zippered outputs only for "
+            "macos and mac-catalyst");
+    } else if (config->outputType != MH_DYLIB &&
+               config->outputType != MH_BUNDLE) {
+      error("writing zippered outputs only valid for -dylib and -bundle");
+    } else {
+      config->platformInfo.minimum = platformVersions[PLATFORM_MACOS].minimum;
+      config->platformInfo.sdk = platformVersions[PLATFORM_MACOS].sdk;
+      config->secondaryPlatformInfo = PlatformInfo{};
+      config->secondaryPlatformInfo->minimum =
+          platformVersions[PLATFORM_MACCATALYST].minimum;
+      config->secondaryPlatformInfo->sdk =
+          platformVersions[PLATFORM_MACCATALYST].sdk;
     }
+    return PLATFORM_MACOS;
+  }
+
   config->platformInfo.minimum = lastVersionInfo->minimum;
   config->platformInfo.sdk = lastVersionInfo->sdk;
   return lastVersionInfo->platform;
 }
 
 // Has the side-effect of setting Config::target.
 static TargetInfo *createTargetInfo(InputArgList &args) {
   StringRef archName = args.getLastArgValue(OPT_arch);
   if (archName.empty()) {
     error("must specify -arch");
     return nullptr;
   }
 
   PlatformType platform = parsePlatformVersions(args);
   config->platformInfo.target =
       MachO::Target(getArchitectureFromName(archName), platform);
+  if (config->secondaryPlatformInfo) {
+    config->secondaryPlatformInfo->target =
+        MachO::Target(getArchitectureFromName(archName), PLATFORM_MACCATALYST);
+  }
 
   uint32_t cpuType;
   uint32_t cpuSubtype;
   std::tie(cpuType, cpuSubtype) = getCPUTypeFromArchitecture(config->arch());
 
   switch (cpuType) {
   case CPU_TYPE_X86_64:
     return createX86_64TargetInfo();
bool macho::link(ArrayRef<const char *> argsArr, llvm::raw_ostream &stdoutOS,
   }
   if (args.hasArg(OPT_version)) {
     message(getLLDVersion());
     return true;
   }
 
   config = std::make_unique<Configuration>();
   symtab = std::make_unique<SymbolTable>();
+  config->outputType = getOutputType(args);
   target = createTargetInfo(args);
   depTracker = std::make_unique<DependencyTracker>(
       args.getLastArgValue(OPT_dependency_info));
   if (errorCount())
     return false;
 
   if (args.hasArg(OPT_pagezero_size)) {
     uint64_t pagezeroSize = args::getHex(args, OPT_pagezero_size, 0);
bool macho::link(ArrayRef<const char *> argsArr, llvm::raw_ostream &stdoutOS,
   config->headerPad = args::getHex(args, OPT_headerpad, /*Default=*/32);
   config->headerPadMaxInstallNames =
       args.hasArg(OPT_headerpad_max_install_names);
   config->printDylibSearch =
       args.hasArg(OPT_print_dylib_search) || getenv("RC_TRACE_DYLIB_SEARCHING");
   config->printEachFile = args.hasArg(OPT_t);
   config->printWhyLoad = args.hasArg(OPT_why_load);
   config->omitDebugInfo = args.hasArg(OPT_S);
-  config->outputType = getOutputType(args);
   config->errorForArchMismatch = args.hasArg(OPT_arch_errors_fatal);
   if (const Arg *arg = args.getLastArg(OPT_bundle_loader)) {
     if (config->outputType != MH_BUNDLE)
       error("-bundle_loader can only be used with MachO bundle output");
     addFile(arg->getValue(), ForceLoad::Default, /*isLazy=*/false,
             /*isExplicit=*/false,
             /*isBundleLoader=*/true);
   }

lld/MachO/Writer.cpp

public:
   uint32_t getSize() const override {
     return sizeof(build_version_command) + ntools * sizeof(build_tool_version);
   }
 
   void writeTo(uint8_t *buf) const override {
     auto *c = reinterpret_cast<build_version_command *>(buf);
     c->cmd = LC_BUILD_VERSION;
     c->cmdsize = getSize();
+
     c->platform = static_cast<uint32_t>(platformInfo.target.Platform);
     c->minos = encodeVersion(platformInfo.minimum);
     c->sdk = encodeVersion(platformInfo.sdk);
+
     c->ntools = ntools;
     auto *t = reinterpret_cast<build_tool_version *>(&c[1]);
     t->tool = TOOL_LD;
     t->version = encodeVersion(VersionTuple(
         LLVM_VERSION_MAJOR, LLVM_VERSION_MINOR, LLVM_VERSION_PATCH));
   }
 
 private:
template <class LP> void Writer::createLoadCommands() {
   uuidCommand = make<LCUuid>();
   in.header->addLoadCommand(uuidCommand);
 
   if (useLCBuildVersion(config->platformInfo))
     in.header->addLoadCommand(make<LCBuildVersion>(config->platformInfo));
   else
     in.header->addLoadCommand(make<LCMinVersion>(config->platformInfo));
 
+  if (config->secondaryPlatformInfo) {
+    in.header->addLoadCommand(
+        make<LCBuildVersion>(*config->secondaryPlatformInfo));
+  }
+
   // This is down here to match ld64's load command order.
   if (config->outputType == MH_EXECUTE)
     in.header->addLoadCommand(make<LCMain>());
 
   int64_t dylibOrdinal = 1;
   DenseMap<StringRef, int64_t> ordinalForInstallName;
   for (InputFile *file : inputFiles) {
     if (auto *dylibFile = dyn_cast<DylibFile>(file)) {

lld/test/MachO/platform-version.s

 # RUN:     | FileCheck --check-prefix=FAIL-PLATFORM %s
 # RUN: not %no-arg-lld -arch x86_64 -o %t %t.o 2>&1 \
 # RUN:        -platform_version 11 1 5 \
 # RUN:     | FileCheck --check-prefix=FAIL-PLATFORM %s
 # FAIL-PLATFORM: malformed platform: {{.*}}
 # FAIL-PLATFORM-NOT: malformed {{minimum|sdk}} version: {{.*}}
 
 # RUN: not %no-arg-lld -arch x86_64 -o %t %t.o -fatal_warnings 2>&1 \
-# RUN:        -platform_version 'mac catalyst' 14.0 15.0 \
+# RUN:        -platform_version iOS 14.0 15.0 \
 # RUN:        -platform_version macos 12.0 12.0 \
 # RUN:     | FileCheck --check-prefix=FAIL-TODO %s
-# FAIL-TODO: writing zippered outputs not yet implemented, ignoring all but last -platform_version flag
+# FAIL-TODO: lld supports writing zippered outputs only for macos and mac-catalyst
 
 # RUN: not %no-arg-lld -arch x86_64 -o %t %t.o 2>&1 \
 # RUN:        -platform_version bridgeOS 1 5 \
 # RUN:        -platform_version 'mac catalyst' 14.0 15.0 \
 # RUN:        -platform_version macos 12.0 12.0 \
 # RUN:     | FileCheck --check-prefix=FAIL-TOOMANY %s
 # FAIL-TOOMANY: must specify -platform_version at most twice
 
 .text
 .global _main
 _main:
   mov $0, %eax
   ret

lld/test/MachO/zippered.yaml

 # REQUIRES: x86
 # RUN: rm -rf %t; mkdir %t
 # RUN: yaml2obj %s > %t/test.dylib
 # RUN: echo "" | llvm-mc -filetype=obj -triple=x86_64-apple-macos10.15 -o %t/test_macos.o
 # RUN: echo "" | llvm-mc -filetype=obj -triple=x86_64-apple-ios13.15.0-macabi -o %t/test_maccatalyst.o
 # RUN: echo "" | llvm-mc -filetype=obj -triple=x86_64-apple-ios13.15.0 -o %t/test_ios.o
 
+## Test linking against a zippered dylib.
 # RUN: %lld -lSystem -dylib %t/test.dylib %t/test_macos.o -o /dev/null
 # RUN: %no-arg-lld -syslibroot %S/Inputs/MacOSX.sdk -lSystem -dylib -arch x86_64 -platform_version mac-catalyst 13.15.0 14.0 %t/test.dylib %t/test_maccatalyst.o -o /dev/null
 
-
 # RUN: %no-arg-lld -syslibroot %S/Inputs/MacOSX.sdk -lSystem -dylib -arch x86_64 -platform_version mac-catalyst 13.15.0 14.0 %t/test_maccatalyst.o -o /dev/null -framework MacOnly-Indirect
 
 # RUN: not %no-arg-lld -syslibroot %S/Inputs/MacOSX.sdk -lSystem -dylib -arch x86_64 -platform_version mac-catalyst 13.15.0 14.0 %t/test_maccatalyst.o -o /dev/null -framework MacOnly 2>&1 | FileCheck --check-prefix=INCOMPATIBLE %s
 # INCOMPATIBLE: System/Library/Frameworks{{[\\/]}}MacOnly.framework{{[\\/]}}MacOnly.tbd(MacOnly.dylib) is incompatible with x86_64 (macCatalyst)
 
 # RUN: not %no-arg-lld -syslibroot %S/Inputs/MacOSX.sdk -lSystem -dylib -arch x86_64 -platform_version ios 13.15.0 14.0 %t/test.dylib %t/test_ios.o -o /dev/null 2>&1 | FileCheck %s
 # CHECK: test.dylib has platform macOS/macCatalyst, which is different from target platform iOS
 
+## Test creating a zippered dylib.
+# RUN: %no-arg-lld -syslibroot %S/Inputs/MacOSX.sdk -lSystem -dylib -arch x86_64 -platform_version macos 12.0 13.0 -platform_version mac-catalyst 13.15.0 14.0 %t/test.dylib %t/test_macos.o -o %t/test_zippered.dylib
+# RUN: llvm-otool -l %t/test_zippered.dylib | FileCheck --check-prefix=ZIPPERED %s
+# ZIPPERED:           cmd LC_BUILD_VERSION
+# ZIPPERED-NEXT:  cmdsize
+# ZIPPERED-NEXT: platform 1
+# ZIPPERED-NEXT:      sdk 13.0
+# ZIPPERED-NEXT:    minos 12.0
+# ZIPPERED-NEXT:   ntools 1
+# ZIPPERED-NEXT:     tool
+# ZIPPERED-NEXT:  version
+# ZIPPERED-NEXT: Load command
+# ZIPPERED-NEXT:      cmd LC_BUILD_VERSION
+# ZIPPERED-NEXT:  cmdsize
+# ZIPPERED-NEXT: platform 6
+# ZIPPERED-NEXT:      sdk 14.0
+# ZIPPERED-NEXT:    minos 13.15
+# ZIPPERED-NEXT:   ntools 1
+# ZIPPERED-NEXT:     tool
+# ZIPPERED-NEXT:  version
+
+# RUN: %no-arg-lld -syslibroot %S/Inputs/MacOSX.sdk -lSystem -dylib -arch x86_64 -platform_version macos 10.8 13.0 -platform_version mac-catalyst 13.15.0 14.0 %t/test.dylib %t/test_macos.o -o %t/test_zippered.dylib
+# RUN: llvm-otool -l %t/test_zippered.dylib | FileCheck --check-prefix=ZIPPERED-OLD %s
+# ZIPPERED-OLD:           cmd LC_VERSION_MIN_MACOSX
+# ZIPPERED-OLD-NEXT:  cmdsize
+# ZIPPERED-OLD-NEXT:  version 10.8
+# ZIPPERED-OLD-NEXT:      sdk 13.0
+# ZIPPERED-OLD-NEXT: Load command
+# ZIPPERED-OLD-NEXT:      cmd LC_BUILD_VERSION
+# ZIPPERED-OLD-NEXT:  cmdsize
+# ZIPPERED-OLD-NEXT: platform 6
+# ZIPPERED-OLD-NEXT:      sdk 14.0
+# ZIPPERED-OLD-NEXT:    minos 13.15
+# ZIPPERED-OLD-NEXT:   ntools 1
+# ZIPPERED-OLD-NEXT:     tool
+# ZIPPERED-OLD-NEXT:  version
+
+# RUN: not %no-arg-lld -syslibroot %S/Inputs/MacOSX.sdk -lSystem -arch x86_64 -platform_version macos 10.8 13.0 -platform_version mac-catalyst 13.15.0 14.0 %t/test.dylib %t/test_macos.o -o %t/test_zippered.dylib 2>&1 | FileCheck --check-prefix=ZIPPERED-EXE %s
+# ZIPPERED-EXE: writing zippered outputs only valid for -dylib and -bundle
+
+## Reject .o files which only have the secondary platform.
+# RUN: not %no-arg-lld -syslibroot %S/Inputs/MacOSX.sdk -lSystem -dylib -arch x86_64 -platform_version macos 12.0 13.0 -platform_version mac-catalyst 13.15.0 14.0 %t/test.dylib %t/test_maccatalyst.o -o %t/test_zippered.dylib 2>&1 | FileCheck --check-prefix=CAT %s
+# CAT: test_maccatalyst.o has platform macCatalyst, which is different from target platform macOS
+
 --- !mach-o
 FileHeader:
   magic:           0xFEEDFACF
   cputype:         0x1000007
   cpusubtype:      0x3
   filetype:        0x6
   ncmds:           4
   sizeofcmds:      600