[lld-macho] Fix bug in reading of cpusubtype field

[int3]

Is there a test we could write here? How do the other bits in cpusubtype get set?

lld/MachO/InputFiles.cpp

234–241

[thakis]

lld/MachO/InputFiles.cpp

234–241

also cpuType, cpuSubtype

[oontvoo]

addressed review comments + added missing tests

[oontvoo]

In D139572#3979427, @int3 wrote:

Is there a test we could write here? How do the other bits in cpusubtype get set?

Added the test to fat-arch.s
(Without this, it'd throw an error when linking the %t.fat.bundle : ld64.lld: error: unable to find matching architecture in fat-arch.s.tmp.fat.exec.out )

lld/test/MachO/fat-arch.s

15

(removed because this doesn't work for arm and we don't really need it)

[oontvoo]

rebase + fix typo

[int3]

Thanks!

lld/test/MachO/fat-arch.s

18

[oontvoo]

addressed review comment + rebase

[oontvoo]

Thx!

[dyung]

@oontvoo the test you added is failing if the AArch64 target is not built: https://lab.llvm.org/buildbot/#/builders/139/builds/32439

[dyung]

In D139572#3983418, @dyung wrote:

@oontvoo the test you added is failing if the AArch64 target is not built: https://lab.llvm.org/buildbot/#/builders/139/builds/32439

I've marked the test as requiring aarch64 to fix the bots in 9c381c430e41.

[thakis]

This breaks tests on macOS: http://45.33.8.238/macm1/50552/step_10.txt

[thakis]

Reverted in 66692c822aee47baa2cb71f92090d58a8dc01116 for now.

[thakis]

Also fails on Windows: https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8795277354029432113/+/u/package_clang/stdout?format=raw

[oontvoo]

Thanks for the revert!

[oontvoo]

In D139572#3984264, @thakis wrote:

Also fails on Windows: https://logs.chromium.org/logs/chromium/buildbucket/cr-buildbucket/8795277354029432113/+/u/package_clang/stdout?format=raw

seems like we just have to skip tests on windows for now (llvm-mc error)

[oontvoo]

updated tests and reland

[thakis]

This still breaks tests: http://45.33.8.238/macm1/50587/step_10.txt

[oontvoo]

In D139572#3985251, @thakis wrote:

This still breaks tests: http://45.33.8.238/macm1/50587/step_10.txt

reverted - will have a look later. Sorry for the breakage~!

[thakis]

Thanks for the fast revert! :)

[oontvoo]

In D139572#3985254, @thakis wrote:

Thanks for the fast revert! :)

@thakis What kind of Mac do you have? Is it arm64? Just curious since I can't seem to reproduce the crash on my machines ( Mac (x86) and Linux)
Thanks!

[oontvoo]

In D139572#3988704, @oontvoo wrote:

In D139572#3985254, @thakis wrote:

Thanks for the fast revert! :)

@thakis What kind of Mac do you have? Is it arm64? Just curious since I can't seem to reproduce the crash on my machines ( Mac (x86) and Linux)
Thanks!

P.S: oh, never mind - I think this test is bad because otool doesn't really guarantee which slice it picks when printing.

[oontvoo]

reland 2nd attempt

New changes:

Fix tests to dump both slices in the fat-archive because otool
isn't deterministic about which slice it prints across different archs.
(It printed x86 on x86 machines but arm64 on arm64, this was why
the test failed on arm64)

Fingers crossed - 3rd time is 🍀

[thakis]

This broke tests: http://45.33.8.238/macm1/51156/step_10.txt

[oontvoo]

In D139572#4003191, @thakis wrote:

This broke tests: http://45.33.8.238/macm1/51156/step_10.txt

Thanks! Looking at the error more closely, it doesn't make a lot of sense. This bundle (%t.fat.bundle was created with -arch x86_64), and yet it ended up being an arm64 (ie., with CPU=16777228) ...
I suspect we have a bug somewhere that causes the target to be whatever host machine's arch rather than what was passed via -arch 🤔

[oontvoo]

Reland #3: temporarily disabled checking cpu type after link due to suspected bug (described above).
(The current test that the link succeeded should be sufficient because
the link would have failed prior to this patch).

lld/MachO/InputFiles.cpp

std::optional<MemoryBufferRef> macho::readFile(StringRef path) {
 
   for (uint32_t i = 0, n = read32be(&hdr->nfat_arch); i < n; ++i) {
     if (reinterpret_cast<const char *>(arch + i + 1) >
         buf + mbref.getBufferSize()) {
       error(path + ": fat_arch struct extends beyond end of file");
       return std::nullopt;
     }
 
-    if (read32be(&arch[i].cputype) != static_cast<uint32_t>(target->cpuType) ||
-        read32be(&arch[i].cpusubtype) != target->cpuSubtype)
+    uint32_t cpuType = read32be(&arch[i].cputype);
+    uint32_t cpuSubtype =
+        read32be(&arch[i].cpusubtype) & ~MachO::CPU_SUBTYPE_MASK;
+
+    // FIXME: LD64 has a more complex fallback logic here.
+    // Consider implementing that as well?
+    if (cpuType != static_cast<uint32_t>(target->cpuType) ||
+        cpuSubtype != target->cpuSubtype)
       continue;
 
     uint32_t offset = read32be(&arch[i].offset);
     uint32_t size = read32be(&arch[i].size);
     if (offset + size > mbref.getBufferSize())
       error(path + ": slice extends beyond end of file");
     if (tar)
       tar->append(relativeToRoot(path), mbref.getBuffer());

lld/test/MachO/fat-arch.s

-# REQUIRES: x86
+# REQUIRES: x86,aarch64
+## FIXME: The tests doesn't run on windows right now because of llvm-mc (can't produce triple=arm64-apple-macos11.0)
+# UNSUPPORTED: system-windows
+
 # RUN: llvm-mc -filetype=obj -triple=i386-apple-darwin %s -o %t.i386.o
 # RUN: llvm-mc -filetype=obj -triple=x86_64-apple-darwin %s -o %t.x86_64.o
+# RUN: llvm-mc -filetype=obj -triple=arm64-apple-macos11.0 %s -o %t.arm64.o
+
 # RUN: llvm-lipo %t.i386.o %t.x86_64.o -create -o %t.fat.o
 # RUN: %lld -o /dev/null %t.fat.o
-
 # RUN: llvm-lipo %t.i386.o -create -o %t.noarch.o
 # RUN: not %lld -o /dev/null %t.noarch.o 2>&1 | \
 # RUN:    FileCheck %s -DFILE=%t.noarch.o
 # CHECK: error: unable to find matching architecture in [[FILE]]
 
+## Validates that we read the cpu-subtype correctly from a fat exec.
+# RUN: %lld -o %t.x86_64.out %t.x86_64.o
+# RUN: %lld -arch arm64 -o %t.arm64.out %t.arm64.o
+# RUN: llvm-lipo %t.x86_64.out %t.arm64.out -create -o %t.fat.exec.out
+# RUN: %lld %t.x86_64.o -bundle_loader %t.fat.exec.out -bundle -o %t.fat.bundle
+
+# RUN: llvm-otool -h %t.fat.exec.out %t.fat.bundle| FileCheck %s --check-prefix=CPU-SUB
+# CPU-SUB:            magic     cputype      cpusubtype   caps      filetype   ncmds sizeofcmds      flags
+# CPU-SUB-NEXT:  0xfeedfacf   [[#CPU:]]   [[#CPU_SUB:]]   0x{{.+}}    {{.+}}  {{.+}}    {{.+}}      {{.+}}
+# CPU-SUB-NEXT: Mach header
+# CPU-SUB-NEXT:       magic     cputype       cpusubtype       caps    filetype ncmds sizeofcmds      flags
+# CPU-SUB-NEXT:  0xfeedfacf    [[#CPU]]     [[#CPU_SUB]]   0x{{.+}}    {{.+}}  {{.+}}    {{.+}}      {{.+}}
+
 .text
 .global _main
 _main:
-  mov $0, %eax
   ret