Differential D92537
[lld-macho] Implement -object_path_lto Authored by int3 on Dec 2 2020, 8:47 PM.
Details
This makes it possible for STABS entries to reference the debug info I'm not sure how to test the file mtime within llvm-lit -- GNU and BSD
Diff Detail
Event TimelineComment Actions What is the main reason for this? Is this so you can run the linker on the same set of input .o files and end up with a binary whose MD5 checksum doesn't change? The danger of this approach is that we will end up allowing any lto.o file to be loaded and linked with dsymutil or lldb. If the lto.o file is modified, we will have no way of knowing if we are loading the right file. Can we add an option for this if this is solely needed for testing? I think people in the real world are going to want the modification file set correctly. If we fix https://reviews.llvm.org/D92736 to generate the UUID with the MD5 of all of the sections that don't change with where or when the binary was built, you can end up using the UUID for verification that things are the same or correct.
Comment Actions
Yeah pretty much... but you raise good points. I'll change it and have the test just skip checking for the exact timestamp Comment Actions FWIW we're very interested in having deterministic build outputs (in the senses of https://blog.llvm.org/2019/11/deterministic-builds-with-clang-and-lld.html), and putting file mtimes in the linker output defeats that. Could we put content hashes in the mtime instead? (We could make the compiler produce a content hash of its .o output at compile time so that we could just read the content off out the .o file instead of computing it if computing it is a perf issue.) Comment Actions That sounds like it'd need a corresponding change in LLDB. (And possibly gdb too?) Seems out of scope for now... Comment Actions If I understood @clayborg's earlier comments, LLDB will match nonzero STABS values with the mtime, so a content hash would result in a mismatch. But a zero STABS value will match any file (at the given path), so I guess we could have a flag to enable that behavior in order to get reproducible builds? Comment Actions We can't change the value of the N_OSO symbol without a change to LLDB and GDB. The debuggers really want to have this timestamp because someone can recompile a .o file and then try to debug a program that refers to this .o file which now has completely different contents. If we use a hash, then each time you debug, the debugger must fully hash each .o file to figure out if the hash matches which we really do not want to do as that will force it to touch every page in each .o file each time you debug and that will kill performance. If we can come up with a solution that doesn't adversely affect performance, the N_OSO currently must have the "n_desc" field of each symbol set to 1. We could set this to 2 and know that the value in the symbol would be different. But that again requires an update to the debugger and anyone else that tries to load a binary with this change into an older debugger will get no debug info with no explanation of what is going wrong. Comment Actions zeroing one the value of the N_OSO can cause the debugger to just load any version of the .o file even if it is out of date. BTW: dsymutil does support loading the debug map from a yaml file (or maybe JSON?). They use this for testing. So there is nothing that states that the debug map must be in the binary itself. We could teach dsymutil to check for N_OSO entries in the binary it is trying to link and if there are none, check for a JSON or YAML file right next to the binary and load the debug map from there automatically. Another option is to add a new STAB entry that specifies the path to the YAML file, and this could be made relative for repro builds. | |||||||||||||