Support: Add a VCSRevision.h header file.

[pcc]

Summary

This is a magic header file supported by the build system that provides a
single definition, LLVM_REVISION, containing an LLVM revision identifier,
if available. This functionality previously lived in the LTO library, but
I am moving it out to lib/Support because I want to also start using it in
lib/Object to create the IR symbol table.

This change also fixes a bug where LLVM_REVISION was never actually being
used in lib/LTO because the macro HAS_LLVM_REVISION was never defined (it
was misspelled as HAVE_SVN_VERSION_INC in lib/LTO/CMakeLists.txt, and was
only being defined in a non-existent file Version.cpp).

I also changed the code to use "git rev-parse --git-dir" to locate the .git
directory, instead of looking for it in the LLVM source root directory,
which makes this compatible with monorepos as well as git worktrees.

[pcc]

Remove some dead code

[beanz]

LGTM.

[• qneill2014]

FYI this breaks downstream builders that rely on the "repo" utility, since the file ".git/logs/HEAD" may not exist.

[pcc]

In D31985#732540, @qneill2014 wrote:

FYI this breaks downstream builders that rely on the "repo" utility, since the file ".git/logs/HEAD" may not exist.

Hi Quentin, please let me know whether this patch fixes the problem in your scenario.

diff --git a/llvm/include/llvm/Support/CMakeLists.txt b/llvm/include/llvm/Support/CMakeLists.txt
index b4b99370574..f1901e55004 100644
--- a/llvm/include/llvm/Support/CMakeLists.txt
+++ b/llvm/include/llvm/Support/CMakeLists.txt
@@ -17,7 +17,9 @@ macro(find_first_existing_vc_file out_var path)
     OUTPUT_VARIABLE git_dir)
   if(git_result EQUAL 0)
     string(STRIP "${git_dir}" git_dir)
-    set(${out_var} "${git_dir}/logs/HEAD")
+    if(EXISTS "${git_dir}/logs/HEAD")
+      set(${out_var} "${git_dir}/logs/HEAD")
+    endif()
   else()
     find_first_existing_file(${out_var}
       "${path}/.svn/wc.db"   # SVN 1.7

[• qneill2014]

In D31985#732557, @pcc wrote:

In D31985#732540, @qneill2014 wrote:

FYI this breaks downstream builders that rely on the "repo" utility, since the file ".git/logs/HEAD" may not exist.

Hi Quentin, please let me know whether this patch fixes the problem in your scenario.

diff --git a/llvm/include/llvm/Support/CMakeLists.txt b/llvm/include/llvm/Support/CMakeLists.txt
index b4b99370574..f1901e55004 100644
--- a/llvm/include/llvm/Support/CMakeLists.txt
+++ b/llvm/include/llvm/Support/CMakeLists.txt
@@ -17,7 +17,9 @@ macro(find_first_existing_vc_file out_var path)
     OUTPUT_VARIABLE git_dir)
   if(git_result EQUAL 0)
     string(STRIP "${git_dir}" git_dir)
-    set(${out_var} "${git_dir}/logs/HEAD")
+    if(EXISTS "${git_dir}/logs/HEAD")
+      set(${out_var} "${git_dir}/logs/HEAD")
+    endif()
   else()
     find_first_existing_file(${out_var}
       "${path}/.svn/wc.db"   # SVN 1.7

Thanks. That fixes the build breakage, but it still won't work with "repo" managed repositories. The "repo" utility manages .git/logs directories itself and doesn't seem to change ".git/logs/HEAD" at all when using manifests that track remote branches (no local branches are used).

I think you could just use ".git/HEAD" itself. If this feature is merely tracking a dependency to detect when to regenerate the header, that should work perfectly.

diff --git a/include/llvm/Support/CMakeLists.txt b/include/llvm/Support/CMakeLists.txt
index b4b9937..377462a 100644
--- a/include/llvm/Support/CMakeLists.txt
+++ b/include/llvm/Support/CMakeLists.txt
@@ -17,7 +17,7 @@ macro(find_first_existing_vc_file out_var path)
     OUTPUT_VARIABLE git_dir)
   if(git_result EQUAL 0)
     string(STRIP "${git_dir}" git_dir)
-    set(${out_var} "${git_dir}/logs/HEAD")
+    set(${out_var} "${git_dir}/HEAD")
   else()
     find_first_existing_file(${out_var}
       "${path}/.svn/wc.db"   # SVN 1.7

[• qneill2014]

In D31985#733509, @qneill2014 wrote:

Thanks. That fixes the build breakage, but it still won't work with "repo" managed repositories....

(Just in case my comment edits above didn't get noticed - reposting the last part with my proposed diff again.)

I think you could just use ".git/HEAD" itself. If this feature is merely tracking a dependency to detect when to regenerate the header, that should work perfectly.

This works for the "repo" case. Let me know @pcc if that works for your tests, and if you will post the fix or if you would like me to do it.

diff --git a/include/llvm/Support/CMakeLists.txt b/include/llvm/Support/CMakeLists.txt
index b4b9937..377462a 100644
--- a/include/llvm/Support/CMakeLists.txt
+++ b/include/llvm/Support/CMakeLists.txt
@@ -17,7 +17,7 @@ macro(find_first_existing_vc_file out_var path)
     OUTPUT_VARIABLE git_dir)
   if(git_result EQUAL 0)
     string(STRIP "${git_dir}" git_dir)
-    set(${out_var} "${git_dir}/logs/HEAD")
+    set(${out_var} "${git_dir}/HEAD")
   else()
     find_first_existing_file(${out_var}
       "${path}/.svn/wc.db"   # SVN 1.7

[pcc]

Thanks for looking into this, I agree that we should be using .git/HEAD and your fix LGTM.

Do you have commit access?

[pcc]

Actually, I think that won't work because it looks like .git/HEAD does not get updated if a commit is made on a branch.

An alternative solution would be to touch .git/logs/HEAD if it does not exist.

[• qneill2014]

In D31985#734032, @pcc wrote:

Actually, I think that won't work because it looks like .git/HEAD does not get updated if a commit is made on a branch.

Hmm, you are right, because .git/HEAD holds a ref

An alternative solution would be to touch .git/logs/HEAD if it does not exist.

I checked and this seems to work, it doesn't seem to affect our typical usage ('repo init' ... 'repo sync').

To answer your other question, I haven't committed anything for a couple of years, so practically I don't have commit access; please submit a patch

FWIW I am testing this locally to see if anything else pops up:

diff --git a/include/llvm/Support/CMakeLists.txt b/include/llvm/Support/CMakeLists.txt
index b4b9937..41e2668 100644
--- a/include/llvm/Support/CMakeLists.txt
+++ b/include/llvm/Support/CMakeLists.txt
@@ -18,6 +18,10 @@ macro(find_first_existing_vc_file out_var path)
   if(git_result EQUAL 0)
     string(STRIP "${git_dir}" git_dir)
     set(${out_var} "${git_dir}/logs/HEAD")
+    # some branchless cases (e.g. 'repo') may not yet have .git/logs/HEAD
+    if (not EXISTS "${git_dir}/logs/HEAD")
+      file(WRITE "${git_dir}/logs/HEAD" "")
+    endif()
   else()
     find_first_existing_file(${out_var}
       "${path}/.svn/wc.db"   # SVN 1.7

[pcc]

In D31985#737643, @qneill2014 wrote:

In D31985#734032, @pcc wrote:

Actually, I think that won't work because it looks like .git/HEAD does not get updated if a commit is made on a branch.

Hmm, you are right, because .git/HEAD holds a ref

An alternative solution would be to touch .git/logs/HEAD if it does not exist.

I checked and this seems to work, it doesn't seem to affect our typical usage ('repo init' ... 'repo sync').

To answer your other question, I haven't committed anything for a couple of years, so practically I don't have commit access; please submit a patch

FWIW I am testing this locally to see if anything else pops up:

diff --git a/include/llvm/Support/CMakeLists.txt b/include/llvm/Support/CMakeLists.txt
index b4b9937..41e2668 100644
--- a/include/llvm/Support/CMakeLists.txt
+++ b/include/llvm/Support/CMakeLists.txt
@@ -18,6 +18,10 @@ macro(find_first_existing_vc_file out_var path)
   if(git_result EQUAL 0)
     string(STRIP "${git_dir}" git_dir)
     set(${out_var} "${git_dir}/logs/HEAD")
+    # some branchless cases (e.g. 'repo') may not yet have .git/logs/HEAD
+    if (not EXISTS "${git_dir}/logs/HEAD")
+      file(WRITE "${git_dir}/logs/HEAD" "")
+    endif()
   else()
     find_first_existing_file(${out_var}
       "${path}/.svn/wc.db"   # SVN 1.7

Thanks, I have committed your patch as r301565.

[• qneill2014]

In D31985#739662, @pcc wrote:

In D31985#737643, @qneill2014 wrote:

In D31985#734032, @pcc wrote:

Actually, I think that won't work because it looks like .git/HEAD does not get updated if a commit is made on a branch.

Hmm, you are right, because .git/HEAD holds a ref

An alternative solution would be to touch .git/logs/HEAD if it does not exist.

I checked and this seems to work, it doesn't seem to affect our typical usage ('repo init' ... 'repo sync').

To answer your other question, I haven't committed anything for a couple of years, so practically I don't have commit access; please submit a patch

FWIW I am testing this locally to see if anything else pops up:

diff --git a/include/llvm/Support/CMakeLists.txt b/include/llvm/Support/CMakeLists.txt
index b4b9937..41e2668 100644
--- a/include/llvm/Support/CMakeLists.txt
+++ b/include/llvm/Support/CMakeLists.txt
@@ -18,6 +18,10 @@ macro(find_first_existing_vc_file out_var path)
   if(git_result EQUAL 0)
     string(STRIP "${git_dir}" git_dir)
     set(${out_var} "${git_dir}/logs/HEAD")
+    # some branchless cases (e.g. 'repo') may not yet have .git/logs/HEAD
+    if (not EXISTS "${git_dir}/logs/HEAD")
+      file(WRITE "${git_dir}/logs/HEAD" "")
+    endif()
   else()
     find_first_existing_file(${out_var}
       "${path}/.svn/wc.db"   # SVN 1.7

Thanks, I have committed your patch as r301565.

Thanks! Cheers.

llvm/trunk/cmake/modules/VersionFromVCS.cmake

if( Subversion_FOUND )
       if( Project_WC_REVISION )
         set(SVN_REVISION ${Project_WC_REVISION} PARENT_SCOPE)
         set(result "${result}-r${Project_WC_REVISION}")
       endif()
       if( Project_WC_URL )
         set(LLVM_REPOSITORY ${Project_WC_URL} PARENT_SCOPE)
       endif()
     endif()
-  elseif( EXISTS ${SOURCE_DIR}/.git )
-    set(result "${result}git")
-    # Try to get a ref-id
+  else()
     find_program(git_executable NAMES git git.exe git.cmd)
 
     if( git_executable )
-      if( EXISTS ${SOURCE_DIR}/.git/svn )
+      # Run from a subdirectory to force git to print an absoute path.
+      execute_process(COMMAND ${git_executable} rev-parse --git-dir
+        WORKING_DIRECTORY ${SOURCE_DIR}/cmake
+        RESULT_VARIABLE git_result
+        OUTPUT_VARIABLE git_dir)
+      if(git_result EQUAL 0)
+        # Try to get a ref-id
+        string(STRIP "${git_dir}" git_dir)
+        set(result "${result}git")
+        if( EXISTS ${git_dir}/svn )
           # Get the repository URL
           execute_process(COMMAND
             ${git_executable} svn info
             WORKING_DIRECTORY ${SOURCE_DIR}
             TIMEOUT 5
             RESULT_VARIABLE git_result
             OUTPUT_VARIABLE git_output)
           if( git_result EQUAL 0 )
             string(REGEX MATCH "URL: ([^ \n]*)" svn_url ${git_output})
             if(svn_url)
               set(LLVM_REPOSITORY ${CMAKE_MATCH_1} PARENT_SCOPE)
             endif()
           endif()
 
           # Get the svn revision number for this git commit if one exists.
           execute_process(COMMAND ${git_executable} svn find-rev HEAD
             WORKING_DIRECTORY ${SOURCE_DIR}
             TIMEOUT 5
             RESULT_VARIABLE git_result
             OUTPUT_VARIABLE git_head_svn_rev_number
             OUTPUT_STRIP_TRAILING_WHITESPACE)
           if( git_result EQUAL 0 AND git_output)
             set(SVN_REVISION ${git_head_svn_rev_number} PARENT_SCOPE)
             set(git_svn_rev "-svn-${git_head_svn_rev_number}")
           else()
             set(git_svn_rev "")
           endif()
         endif()
 
         # Get the git ref id
         execute_process(COMMAND
           ${git_executable} rev-parse --short HEAD
           WORKING_DIRECTORY ${SOURCE_DIR}
           TIMEOUT 5
           RESULT_VARIABLE git_result
           OUTPUT_VARIABLE git_ref_id
           OUTPUT_STRIP_TRAILING_WHITESPACE)
 
         if( git_result EQUAL 0 )
           set(GIT_COMMIT ${git_ref_id} PARENT_SCOPE)
           set(result "${result}${git_svn_rev}-${git_ref_id}")
         else()
           set(result "${result}${git_svn_rev}")
         endif()
       endif()
     endif()
+  endif()
   set(${VERS} ${result} PARENT_SCOPE)
 endfunction(add_version_info_from_vcs)

llvm/trunk/include/llvm/CMakeLists.txt

 add_subdirectory(IR)
+add_subdirectory(Support)
 
 # If we're doing an out-of-tree build, copy a module map for generated
 # header files into the build area.
 if (NOT "${CMAKE_SOURCE_DIR}" STREQUAL "${CMAKE_BINARY_DIR}")
   configure_file(module.modulemap.build module.modulemap COPYONLY)
 endif (NOT "${CMAKE_SOURCE_DIR}" STREQUAL "${CMAKE_BINARY_DIR}")

llvm/trunk/include/llvm/Support/CMakeLists.txt

+# Figure out if we can track VC revisions.
+function(find_first_existing_file out_var)
+  foreach(file ${ARGN})
+    if(EXISTS "${file}")
+      set(${out_var} "${file}" PARENT_SCOPE)
+      return()
+    endif()
+  endforeach()
+endfunction()
+
+macro(find_first_existing_vc_file out_var path)
+  find_program(git_executable NAMES git git.exe git.cmd)
+  # Run from a subdirectory to force git to print an absolute path.
+  execute_process(COMMAND ${git_executable} rev-parse --git-dir
+    WORKING_DIRECTORY ${path}/cmake
+    RESULT_VARIABLE git_result
+    OUTPUT_VARIABLE git_dir)
+  if(git_result EQUAL 0)
+    string(STRIP "${git_dir}" git_dir)
+    set(${out_var} "${git_dir}/logs/HEAD")
+  else()
+    find_first_existing_file(${out_var}
+      "${path}/.svn/wc.db"   # SVN 1.7
+      "${path}/.svn/entries" # SVN 1.6
+    )
+  endif()
+endmacro()
+
+find_first_existing_vc_file(llvm_vc "${LLVM_MAIN_SRC_DIR}")
+
+# The VC revision include that we want to generate.
+set(version_inc "${CMAKE_CURRENT_BINARY_DIR}/VCSRevision.h")
+
+set(get_svn_script "${LLVM_CMAKE_PATH}/GenerateVersionFromCVS.cmake")
+
+if(DEFINED llvm_vc)
+  # Create custom target to generate the VC revision include.
+  add_custom_command(OUTPUT "${version_inc}"
+    DEPENDS "${llvm_vc}" "${get_svn_script}"
+    COMMAND
+    ${CMAKE_COMMAND} "-DSOURCE_DIR=${LLVM_MAIN_SRC_DIR}"
+                     "-DNAME=LLVM_REVISION"
+                     "-DHEADER_FILE=${version_inc}"
+                     -P "${get_svn_script}")
+
+  # Mark the generated header as being generated.
+  set_source_files_properties("${version_inc}"
+    PROPERTIES GENERATED TRUE
+               HEADER_FILE_ONLY TRUE)
+else()
+  file(WRITE "${version_inc}" "")
+endif()
+
+add_custom_target(llvm_vcsrevision_h DEPENDS "${version_inc}")

llvm/trunk/lib/LTO/CMakeLists.txt

-# Figure out if we can track VC revisions.
-function(find_first_existing_file out_var)
-  foreach(file ${ARGN})
-    if(EXISTS "${file}")
-      set(${out_var} "${file}" PARENT_SCOPE)
-      return()
-    endif()
-  endforeach()
-endfunction()
-
-macro(find_first_existing_vc_file out_var path)
-  find_first_existing_file(${out_var}
-    "${path}/.git/logs/HEAD" # Git
-    "${path}/.svn/wc.db"     # SVN 1.7
-    "${path}/.svn/entries"   # SVN 1.6
-    )
-endmacro()
-
-find_first_existing_vc_file(llvm_vc "${LLVM_MAIN_SRC_DIR}")
-
-# The VC revision include that we want to generate.
-set(version_inc "${CMAKE_CURRENT_BINARY_DIR}/LLVMLTORevision.h")
-
-set(get_svn_script "${LLVM_CMAKE_PATH}/GenerateVersionFromCVS.cmake")
-
-if(DEFINED llvm_vc)
-  # Create custom target to generate the VC revision include.
-  add_custom_command(OUTPUT "${version_inc}"
-    DEPENDS "${llvm_vc}" "${get_svn_script}"
-    COMMAND
-    ${CMAKE_COMMAND} "-DSOURCE_DIR=${LLVM_MAIN_SRC_DIR}"
-                     "-DNAME=LLVM_REVISION"
-                     "-DHEADER_FILE=${version_inc}"
-                     -P "${get_svn_script}")
-
-  # Mark the generated header as being generated.
-  set_source_files_properties("${version_inc}"
-    PROPERTIES GENERATED TRUE
-               HEADER_FILE_ONLY TRUE)
-
-  # Tell Version.cpp that it needs to build with -DHAVE_SVN_VERSION_INC.
-  set_source_files_properties(Version.cpp
-    PROPERTIES COMPILE_DEFINITIONS "HAVE_SVN_VERSION_INC")
-else()
-  # Not producing a VC revision include.
-  set(version_inc)
-endif()
-
-
 add_llvm_library(LLVMLTO
   Caching.cpp
   LTO.cpp
   LTOBackend.cpp
   LTOModule.cpp
   LTOCodeGenerator.cpp
   UpdateCompilerUsed.cpp
   ThinLTOCodeGenerator.cpp
-  ${version_inc}
 
   ADDITIONAL_HEADER_DIRS
   ${LLVM_MAIN_INCLUDE_DIR}/llvm/LTO
 
   DEPENDS
   intrinsics_gen
+  llvm_vcsrevision_h
 )

llvm/trunk/lib/LTO/LTO.cpp

 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/SHA1.h"
 #include "llvm/Support/SourceMgr.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/Threading.h"
+#include "llvm/Support/VCSRevision.h"
 #include "llvm/Support/raw_ostream.h"
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Target/TargetOptions.h"
 #include "llvm/Transforms/IPO.h"
 #include "llvm/Transforms/IPO/PassManagerBuilder.h"
 #include "llvm/Transforms/Utils/SplitModule.h"
 
 #include <set>
static void computeCacheKey(
   // Compute the unique hash for this entry.
   // This is based on the current compiler version, the module itself, the
   // export list, the hash for every single module in the import list, the
   // list of ResolvedODR for the module, and the list of preserved symbols.
   SHA1 Hasher;
 
   // Start with the compiler revision
   Hasher.update(LLVM_VERSION_STRING);
-#ifdef HAVE_LLVM_REVISION
+#ifdef LLVM_REVISION
   Hasher.update(LLVM_REVISION);
 #endif
 
   // Include the parts of the LTO configuration that affect code generation.
   auto AddString = [&](StringRef Str) {
     Hasher.update(Str);
     Hasher.update(ArrayRef<uint8_t>{0});
   };

llvm/trunk/lib/LTO/ThinLTOCodeGenerator.cpp

 //===-ThinLTOCodeGenerator.cpp - LLVM Link Time Optimizer -----------------===//
 //
 //                     The LLVM Compiler Infrastructure
 //
 // This file is distributed under the University of Illinois Open Source
 // License. See LICENSE.TXT for details.
 //
 //===----------------------------------------------------------------------===//
 //
 // This file implements the Thin Link Time Optimization library. This library is
 // intended to be used by linker to optimize code at link time.
 //
 //===----------------------------------------------------------------------===//
 
 #include "llvm/LTO/legacy/ThinLTOCodeGenerator.h"
 
-#ifdef HAVE_LLVM_REVISION
-#include "LLVMLTORevision.h"
-#endif
-
 #include "llvm/ADT/Statistic.h"
 #include "llvm/ADT/StringExtras.h"
 #include "llvm/Analysis/ModuleSummaryAnalysis.h"
 #include "llvm/Analysis/ProfileSummaryInfo.h"
 #include "llvm/Analysis/TargetLibraryInfo.h"
 #include "llvm/Analysis/TargetTransformInfo.h"
 #include "llvm/Bitcode/BitcodeReader.h"
 #include "llvm/Bitcode/BitcodeWriter.h"

 #include "llvm/Support/Debug.h"
 #include "llvm/Support/Error.h"
 #include "llvm/Support/Path.h"
 #include "llvm/Support/SHA1.h"
 #include "llvm/Support/TargetRegistry.h"
 #include "llvm/Support/ThreadPool.h"
 #include "llvm/Support/Threading.h"
 #include "llvm/Support/ToolOutputFile.h"
+#include "llvm/Support/VCSRevision.h"
 #include "llvm/Target/TargetMachine.h"
 #include "llvm/Transforms/IPO.h"
 #include "llvm/Transforms/IPO/FunctionImport.h"
 #include "llvm/Transforms/IPO/Internalize.h"
 #include "llvm/Transforms/IPO/PassManagerBuilder.h"
 #include "llvm/Transforms/ObjCARC.h"
 #include "llvm/Transforms/Utils/FunctionImportUtils.h"
 
auto AddUnsigned = [&](unsigned I) {
       Data[1] = I >> 8;
       Data[2] = I >> 16;
       Data[3] = I >> 24;
       Hasher.update(ArrayRef<uint8_t>{Data, 4});
     };
 
     // Start with the compiler revision
     Hasher.update(LLVM_VERSION_STRING);
-#ifdef HAVE_LLVM_REVISION
+#ifdef LLVM_REVISION
     Hasher.update(LLVM_REVISION);
 #endif
 
     // Hash the optimization level and the target machine settings.
     AddString(TMBuilder.MCpu);
     // FIXME: Hash more of Options. For now all clients initialize Options from
     // command-line flags (which is unsupported in production), but may set
     // RelaxELFRelocations. The clang driver can also pass FunctionSections,