[cmake] Add custom command to touch archives so ninja won't rebuild them.

[hintonda]

Summary

clang and ninja use high-resolutions timestamps, but libtool
doesn't, so the archive will often get an older timestamp than the
last object that was added or updated. To fix this, we add a custom
command to touch archive after it's been built so that ninja won't
rebuild it unnecessarily the next time it's run.

[beanz]

@thakis Didn't we patch Ninja for this?

[hintonda]

In D62172#1509516, @beanz wrote:

@thakis Didn't we patch Ninja for this?

Here's what I'm running on my Mac:

local:/Users/dhinton/projects/llvm_project/monorepo/llvm-project $ ninja --version
1.9.0
local:/Users/dhinton/projects/llvm_project/monorepo/llvm-project $ cmake --version
cmake version 3.14.0

[hintonda]

In D62172#1509521, @hintonda wrote:

In D62172#1509516, @beanz wrote:

@thakis Didn't we patch Ninja for this?

Here's what I'm running on my Mac:

local:/Users/dhinton/projects/llvm_project/monorepo/llvm-project $ ninja --version
1.9.0
local:/Users/dhinton/projects/llvm_project/monorepo/llvm-project $ cmake --version
cmake version 3.14.0

btw, here's some samples of what I get when I don't touch them:

local:/Users/dhinton/projects/llvm_project/monorepo/build/Debug $ /usr/local/opt/coreutils/libexec/gnubin/ls --full-time lib/lib*.a
-rw-r--r--  1 dhinton staff     17232 2019-05-20 18:03:27.000000000 -0700 libDynamicLibraryLib.a
-rw-r--r--  1 dhinton staff   2822800 2019-05-20 18:01:34.000000000 -0700 libLLVMBinaryFormat.a
-rw-r--r--  1 dhinton staff 605424256 2019-05-20 18:05:56.000000000 -0700 libLLVMCodeGen.a
-rw-r--r--  1 dhinton staff 115587352 2019-05-20 18:04:26.000000000 -0700 libLLVMCore.a

and what I get when I do:

local:/Users/dhinton/projects/llvm_project/monorepo/build/Debug $ /usr/local/opt/coreutils/libexec/gnubin/ls --full-time lib/lib*.a
-rw-r--r-- 1 dhinton staff     17232 2019-05-20 20:57:50.000000000 -0700 lib/libDynamicLibraryLib.a
-rw-r--r-- 1 dhinton staff   2822800 2019-05-20 20:57:50.002389000 -0700 lib/libLLVMBinaryFormat.a
-rw-r--r-- 1 dhinton staff 605424256 2019-05-20 20:58:06.902285000 -0700 lib/libLLVMCodeGen.a
-rw-r--r-- 1 dhinton staff 115587352 2019-05-20 20:58:04.538669000 -0700 lib/libLLVMCore.a

Note that DynamicLibrary doesn't use llvm_add_library...

[beanz]

I have a long story about this issue... Ask me about it sometime :).

We should get @thakis to weigh in here, but Ninja 1.8.2 works on Darwin because it uses the older filesystem APIs and ignores the high-precision timestamps. If we take this patch we should restrict it more. I suspect that libtool on Linux probably behaves correctly. Darwin's binutils fork is literally decades old.

My guess at restricting the patch would be only on Darwin (which is the only place we use libtool), only if the Ninja version > 1.8.2, and Darwin version > 15.6.0 (I think 16.0.0 was when APFS went in which causes this issue).

[hintonda]

In D62172#1509541, @beanz wrote:

I have a long story about this issue... Ask me about it sometime :).

We should get @thakis to weigh in here, but Ninja 1.8.2 works on Darwin because it uses the older filesystem APIs and ignores the high-precision timestamps. If we take this patch we should restrict it more. I suspect that libtool on Linux probably behaves correctly. Darwin's binutils fork is literally decades old.

My guess at restricting the patch would be only on Darwin (which is the only place we use libtool), only if the Ninja version > 1.8.2, and Darwin version > 15.6.0 (I think 16.0.0 was when APFS went in which causes this issue).

That sound right. I just did a quick check on a linux box and it didn't have the problem.

[hintonda]

• Restrict to Darwin > 15.6.0. Ninja version not available, so will add code to get that and hide all of this under a new variable in the next version of this patch.

[hintonda]

• Get Ninja version and add LLVM_TOUCH_STATIC_LIBRARIES.

[hintonda]

• Remove unneeded else branch.

[beanz]

It is silly that CMake doesn't expose the ninja version, but it is easy enough to get it. It should be something like:

if(CMAKE_GENERATOR STREQUAL "Ninja")
  execute_process(COMMAND ${RunCMake_MAKE_PROGRAM} --version
      OUTPUT_VARIABLE ninja_version OUTPUT_STRIP_TRAILING_WHITESPACE)
  if(ninja_version VERSION_GREATER 1.8.2)
    set(ninja_greater_1_8_2 On)
  endif()
endif()

I know this is a bunch of seemingly odd boiler plate, but since we're working around tools bugs in non-LLVM tools which will hopefully someday be fixed, I'd like to restrict this fix to only places where we know the bug is present.

[hintonda]

• Don't put new variable in the cache.

[hintonda]

In D62172#1510515, @beanz wrote:

It is silly that CMake doesn't expose the ninja version, but it is easy enough to get it. It should be something like:

if(CMAKE_GENERATOR STREQUAL "Ninja")
  execute_process(COMMAND ${RunCMake_MAKE_PROGRAM} --version
      OUTPUT_VARIABLE ninja_version OUTPUT_STRIP_TRAILING_WHITESPACE)
  if(ninja_version VERSION_GREATER 1.8.2)
    set(ninja_greater_1_8_2 On)
  endif()
endif()

I know this is a bunch of seemingly odd boiler plate, but since we're working around tools bugs in non-LLVM tools which will hopefully someday be fixed, I'd like to restrict this fix to only places where we know the bug is present.

I already added it to llvm/cmake/config-ix.cmake. Please let me know if it looks okay.

[hintonda]

• Remove tab.

[beanz]

One small change needed, otherwise LGTM.

llvm/cmake/config-ix.cmake

569

Instead of checking CMAKE_SYSTEM_NAME you should probably just check CMAKE_HOST_APPLE. Not that people really are targeting Darwin from other hosts, but CMAKE_SYSTEM_NAME is actually the target system not the host.

[hintonda]

• Use CMAKE_HOST_APPLE when checking host system.

[ABataev]

llvm/trunk/cmake/config-ix.cmake

558 ↗

(On Diff #200543)

This line breaks cmake, seems to me must be CMakeNinjaFindMake

llvm/cmake/config-ix.cmake

else()
     endif()
   endif()
 endif()
 
 find_program(GOLD_EXECUTABLE NAMES ${LLVM_DEFAULT_TARGET_TRIPLE}-ld.gold ld.gold ${LLVM_DEFAULT_TARGET_TRIPLE}-ld ld DOC "The gold linker")
 set(LLVM_BINUTILS_INCDIR "" CACHE PATH
   "PATH to binutils/include containing plugin-api.h for gold plugin.")
 
+if(CMAKE_GENERATOR STREQUAL "Ninja")
+  include(CMakeNInjaFindMake)
+  if(CMAKE_MAKE_PROGRAM)
+    execute_process(COMMAND ${CMAKE_MAKE_PROGRAM} --version
+      OUTPUT_VARIABLE NINJA_VERSION
+      OUTPUT_STRIP_TRAILING_WHITESPACE)
+    set(NINJA_VERSION ${NINJA_VERSION} CACHE STRING "Ninja version number" FORCE)
+  endif()
+endif()
+
+if(CMAKE_GENERATOR STREQUAL "Ninja" AND
+    NINJA_VERSION VERSION_GREATER_EQUAL "1.9.0" AND
+    CMAKE_SYSTEM_NAME MATCHES "Darwin" AND
+    CMAKE_HOST_SYSTEM_VERSION VERSION_GREATER "15.6.0")
+  set(LLVM_TOUCH_STATIC_LIBRARIES ON)
+endif()
+
 if(CMAKE_HOST_APPLE AND APPLE)
   if(NOT CMAKE_XCRUN)
     find_program(CMAKE_XCRUN NAMES xcrun)
   endif()
   if(CMAKE_XCRUN)
     execute_process(COMMAND ${CMAKE_XCRUN} -find ld
       OUTPUT_VARIABLE LD64_EXECUTABLE
       OUTPUT_STRIP_TRAILING_WHITESPACE)

llvm/cmake/modules/AddLLVM.cmake

foreach(objlib ${objlibs})
       add_dependencies(${objlib} ${LLVM_COMMON_DEPENDS})
     endforeach()
   endif()
 
   if(ARG_SHARED OR ARG_MODULE)
     llvm_externalize_debuginfo(${name})
     llvm_codesign(${name} ENTITLEMENTS ${ARG_ENTITLEMENTS})
   endif()
+  # clang and newer versions of ninja use high-resolutions timestamps,
+  # but older versions of libtool on Darwin don't, so the archive will
+  # often get an older timestamp than the last object that was added
+  # or updated.  To fix this, we add a custom command to touch archive
+  # after it's been built so that ninja won't rebuild it unnecessarily
+  # the next time it's run.
+  if(ARG_STATIC AND LLVM_TOUCH_STATIC_LIBRARIES)
+    add_custom_command(TARGET ${name}
+      POST_BUILD
+      COMMAND touch ${LLVM_LIBRARY_DIR}/${CMAKE_STATIC_LIBRARY_PREFIX}${name}${CMAKE_STATIC_LIBRARY_SUFFIX}
+      )
+  endif()
 endfunction()
 
 function(add_llvm_install_targets target)
   cmake_parse_arguments(ARG "" "COMPONENT;PREFIX" "DEPENDS" ${ARGN})
   if(ARG_COMPONENT)
     set(component_option -DCMAKE_INSTALL_COMPONENT="${ARG_COMPONENT}")
   endif()
   if(ARG_PREFIX)