Added optional validation of svn sources to Dockerfiles.

[ilya-biryukov]

Summary

This commit also adds a script to compute sha256 hashes of llvm checkouts.

[mehdi_amini]

Would this be all obsolete with git repos?

Also all the python code seems complicated to me for what is basically find | grep -v '/\.git/' | grep -v '/\.svn/' | LC_ALL=C sort | tar -cf - -T - --no-recursion | sha1sum, can you elaborate why this is needed?

[ilya-biryukov]

In D37099#852251, @mehdi_amini wrote:

Would this be all obsolete with git repos?

I would think so.

Also all the python code seems complicated to me for what is basically find | grep -v '/\.git/' | grep -v '/\.svn/' | LC_ALL=C sort | tar -cf - -T - --no-recursion | sha1sum, can you elaborate why this is needed?

That's roughly what I started with, but's it's more complicated if you want to compute separate checksums for projects inside single tree source checkout (i.e. llvm, llvm/tool/clang, llvm/projects/lldb, etc.).
The use-case is rather bizzare:

1. You continuously import and review commits from svn repo for a set a llvm projects (e.g., llvm, clang, lldb). You use single tree source checkout for that.
2. You want to build docker images for a subset of those projects (e.g., only for llvm and clang). However, you want to checkout source code from official svn repo, not your mirror.
3. You want to make sure the code you checkout matches the code you reviewed in step 1.

There is also a problem that some files inside LLVM repo use svn substitutions. The substitutions are mostly fine, but $Date$ and $LastChangedDate$ are locale-specific and we have to account for that.

I totally agree with you, though, the code is quite complicated. Wish it was simpler.

[mehdi_amini]

In D37099#852298, @ilya-biryukov wrote:

In D37099#852251, @mehdi_amini wrote:

Would this be all obsolete with git repos?

I would think so.

Also all the python code seems complicated to me for what is basically find | grep -v '/\.git/' | grep -v '/\.svn/' | LC_ALL=C sort | tar -cf - -T - --no-recursion | sha1sum, can you elaborate why this is needed?

That's roughly what I started with, but's it's more complicated if you want to compute separate checksums for projects inside single tree source checkout (i.e. llvm, llvm/tool/clang, llvm/projects/lldb, etc.).

Well first using a flat layout makes it easier: checkout clang next to llvm instead of inside llvm/tools (and use -DLLVM_ENABLE_PROJECTS).
Then it is a matter of wrapping the command I gave in a loop around the directories and producing one entry per project.

There is also a problem that some files inside LLVM repo use svn substitutions. The substitutions are mostly fine, but $Date$ and $LastChangedDate$ are locale-specific and we have to account for that.

OK that's terrible :(

I totally agree with you, though, the code is quite complicated. Wish it was simpler.

Yeah, like just using git ;)

Anyway I'm fine with adding this, but I won't have time to review deeply the python code right now.

[klimek]

utils/docker/scripts/llvm_checksum/llvm_checksum_utils.py

2

Why 2 files? (generally, I dislike files named "util" :)

28–29

Why's the algo not part of the content_hasher?
Can't we just use a hasher that has the exact interface we need?

49

Call it process_file? (proc_file sounds like a file in /proc :P)

83–84

Can't we put that in the same line?

123–124

Given that we feed in paths, this seems redundant?

132–134

Why not just feed the content (file content or symlink target)?

[ilya-biryukov]

Addressed review comments.

• Renamed proc_file to process_file.
• Don't feed number of files to the hasher, it's redundant.
• Removed next_dirs local var.

utils/docker/scripts/llvm_checksum/llvm_checksum_utils.py

2

Totally agree, util is a bad choice.
One file(util) is a source code of a library, the other is a source code of an executable.
The idea is to split the command argument parsing logic from an actual library code.
How about _lib instead of _util? Or do you think merging the files is better?

28–29

• content_hasher is responsible for reading files and replacing the contents of the file, while hash_algo is responsible for providing hash functions.
• we don't call content_hasher for broken symlinks and use hash_algo on the symlink target instead. It does not make sense to do any replacements in there, as it's not a file.
• it's nice that this function controls the lifetime of hash_algo() objects. Otherwise, there's a chance the client will create hasher only once and use that for all subsequent calls.

It also seems nice to have the checksumming algorithm name as part of the function interface.

49

All short names are already taken by Linux :-(
Renamed to process_file.

123–124

Totally agree, removed it.

132–134

To distinguish between a broken symlink and a file with content equivalent to the broken symlink's target.

[klimek]

utils/docker/scripts/llvm_checksum/llvm_checksum_utils.py

28–29

I think my main concern is that we're mixing levels of abstraction; generally, we have a strategy how to do the hashing, and we have an algorithm that does the visitation (giving project structure, whitelist, etc).

I'd probably just do a visit_project function that takes a class with perhaps 2 functions:
visit_file(path, content)
visit_symlink(path, link)

Then we'd have a Checksum class that implements these 2 functions, has a hasher member and does the hashing in the visitation, cutting at the responsibilities.

That way, we also could split this up into multiple files more easily:
project_tree.py (perhaps with a better name :) allows to visit all files in the project.
llvm_checksum.py - can now be the main method and the checksum'ing parts of the algo

132–134

Why do we care?

[ilya-biryukov]

utils/docker/scripts/llvm_checksum/llvm_checksum_utils.py

28–29

Good point, will do.

132–134

Frankly, we don't. Broken symlinks and files are different beasts, but for our use-case this is probably irrelevant.
Could remove this logic altogether and make the script simpler.

[ilya-biryukov]

• Separated project tree walking and checksumming implementations.

[ilya-biryukov]

Followed @klimek's suggestions and split the checksumming and project walking logic.
Also simplified the logic of dealing with broken symlinks - now we simply hash the target of the symlink, therefore it's indistinguishable from file with the same content. This makes the code a little simpler.

[klimek]

utils/docker/scripts/llvm_checksum/llvm_checksum.py

87

Substitute subsitutions for substitutions.

115

So the main reason we hash each file is for debugging purposes?

[ilya-biryukov]

• Rename read_replacing_substituions to read_and_collapse_svn_substitutions.

[ilya-biryukov]

utils/docker/scripts/llvm_checksum/llvm_checksum.py

87

Substituted with a name mentioning less substitutions.

115

Yes, makes debugging when checksums don't match easier.

An alternative I was thinking about is to just feed all files to a single hasher, but that would probably allow to easily craft two different directory trees with the same hashes.

[klimek]

LG

[jlebar]

Are the checksums intended to be valid across multiple platforms? I recall that @zturner noticed that we use SVN in weird ways to munge newlines for some files on Windows.

[ilya-biryukov]

In D37099#873837, @jlebar wrote:

Are the checksums intended to be valid across multiple platforms? I recall that @zturner noticed that we use SVN in weird ways to munge newlines for some files on Windows.

Currently the checksums are only intended to match on Linux. I haven't checked, but some files will definitely have different line-endings on Windows.

utils/docker/build_docker_image.sh

LLVM-specific:
     -p|--llvm-project   name of an svn project to checkout. Will also add the
                         project to a list LLVM_ENABLE_PROJECTS, passed to CMake.
                         For clang, please use 'clang', not 'cfe'.
                         Project 'llvm' is always included and ignored, if
                         specified.
                         Can be specified multiple times.
     -i|--install-target name of a cmake install target to build and include in
                         the resulting archive. Can be specified multiple times.
+    -c|--checksums      name of a file, containing checksums of llvm checkout.
+                        Script will fail if checksums of the checkout do not
+                        match.
 
 Required options: --source and --docker-repository, at least one
   --install-target.
 
 All options after '--' are passed to CMake invocation.
 
 For example, running:
 $ build_docker_image.sh -s debian8 -d mydocker/debian8-clang -t latest \ 
$ ./build_docker_image.sh -s debian8 -d mydocker/clang-debian8 -t "latest" \
     -- \ 
     -DLLVM_TARGETS_TO_BUILD=Native -DCMAKE_BUILD_TYPE=Release \ 
     -DBOOTSTRAP_CMAKE_BUILD_TYPE=Release \ 
     -DCLANG_ENABLE_BOOTSTRAP=ON \ 
     -DCLANG_BOOTSTRAP_TARGETS="install-clang;install-clang-headers"
 EOF
 }
 
+CHECKSUMS_FILE=""
 SEEN_INSTALL_TARGET=0
 while [[ $# -gt 0 ]]; do
   case "$1" in
     -h|--help)
       show_usage
       exit 0
       ;;
     -s|--source)
-t|--docker-tag)
       ;;
     -i|--install-target|-r|--revision|-b|--branch|-p|--llvm-project)
       if [ "$1" == "-i" ] || [ "$1" == "--install-target" ]; then
         SEEN_INSTALL_TARGET=1
       fi
       BUILDSCRIPT_ARGS="$BUILDSCRIPT_ARGS $1 $2"
       shift 2
       ;;
+    -c|--checksums)
+      shift
+      CHECKSUMS_FILE="$1"
+      shift
+      ;;
     --)
       shift
       BUILDSCRIPT_ARGS="$BUILDSCRIPT_ARGS -- $*"
       shift $#
       ;;
     *)
       echo "Unknown argument $1"
       exit 1

 
 BUILD_DIR=$(mktemp -d)
 trap "rm -rf $BUILD_DIR" EXIT
 echo "Using a temporary directory for the build: $BUILD_DIR"
 
 cp -r "$SOURCE_DIR/$IMAGE_SOURCE" "$BUILD_DIR/$IMAGE_SOURCE"
 cp -r "$SOURCE_DIR/scripts" "$BUILD_DIR/scripts"
 
+mkdir "$BUILD_DIR/checksums"
+if [ "$CHECKSUMS_FILE" != "" ]; then
+  cp "$CHECKSUMS_FILE" "$BUILD_DIR/checksums/checksums.txt"
+fi
+
 if [ "$DOCKER_TAG" != "" ]; then
   DOCKER_TAG=":$DOCKER_TAG"
 fi
 
 echo "Building from $IMAGE_SOURCE"
 echo "Building $DOCKER_REPOSITORY-build$DOCKER_TAG"
 docker build -t "$DOCKER_REPOSITORY-build$DOCKER_TAG" \
   --build-arg "buildscript_args=$BUILDSCRIPT_ARGS" \

utils/docker/debian8/build/Dockerfile

 # Install build dependencies of llvm.
 # First, Update the apt's source list and include the sources of the packages.
 RUN grep deb /etc/apt/sources.list | \
     sed 's/^deb/deb-src /g' >> /etc/apt/sources.list
 
 # Install compiler, python and subversion.
 RUN apt-get update && \
     apt-get install -y --no-install-recommends ca-certificates gnupg \
-          build-essential python2.7 wget subversion ninja-build && \
+           build-essential python wget subversion ninja-build && \
     rm -rf /var/lib/apt/lists/*
 
 # Import public key required for verifying signature of cmake download.
 RUN gpg --keyserver hkp://pgp.mit.edu --recv 0x2D2CEF1034921684
 
 # Download, verify and install cmake version that can compile clang into /usr/local.
 # (Version in debian8 repos is is too old)
 RUN mkdir /tmp/cmake-install && cd /tmp/cmake-install && \
     wget "https://cmake.org/files/v3.7/cmake-3.7.2-SHA-256.txt.asc" && \
     wget "https://cmake.org/files/v3.7/cmake-3.7.2-SHA-256.txt" && \
     gpg --verify cmake-3.7.2-SHA-256.txt.asc cmake-3.7.2-SHA-256.txt && \
     wget "https://cmake.org/files/v3.7/cmake-3.7.2-Linux-x86_64.tar.gz" && \
     ( grep "cmake-3.7.2-Linux-x86_64.tar.gz" cmake-3.7.2-SHA-256.txt | \
       sha256sum -c - ) && \
     tar xzf cmake-3.7.2-Linux-x86_64.tar.gz -C /usr/local --strip-components=1 && \
     cd / && rm -rf /tmp/cmake-install
 
+ADD checksums /tmp/checksums
+ADD scripts /tmp/scripts
+
 # Arguments passed to build_install_clang.sh.
 ARG buildscript_args
 
 # Run the build. Results of the build will be available as /tmp/clang.tar.gz.
-ADD scripts/build_install_llvm.sh /tmp
-RUN /tmp/build_install_llvm.sh ${buildscript_args}
+RUN /tmp/scripts/build_install_llvm.sh ${buildscript_args}

utils/docker/example/build/Dockerfile

 FROM ubuntu
 
 # FIXME: Change maintainer name
 LABEL maintainer "Maintainer <maintainer@email>"
 
 # FIXME: Install llvm/clang build dependencies. Including compiler to
 # build stage1, cmake, subversion, ninja, etc.
 
-# Arguments to pass to build_install_clang.sh.
+ADD checksums /tmp/checksums
+ADD scripts /tmp/scripts
+
+# Arguments passed to build_install_clang.sh.
 ARG buildscript_args
 
 # Run the build. Results of the build will be available as /tmp/clang.tar.gz.
-ADD scripts/build_install_llvm.sh /tmp
-RUN /tmp/build_install_llvm.sh ${buildscript_args}
+RUN /tmp/scripts/build_install_llvm.sh ${buildscript_args}

utils/docker/nvidia-cuda/build/Dockerfile

 
 LABEL maintainer "LLVM Developers"
 
 # Arguments to pass to build_install_clang.sh.
 ARG buildscript_args
 
 # Install llvm build dependencies.
 RUN apt-get update && \
-    apt-get install -y --no-install-recommends ca-certificates cmake python2.7 \
+    apt-get install -y --no-install-recommends ca-certificates cmake python \
         subversion ninja-build && \
     rm -rf /var/lib/apt/lists/*
 
+ADD checksums /tmp/checksums
+ADD scripts /tmp/scripts
+
+# Arguments passed to build_install_clang.sh.
+ARG buildscript_args
+
 # Run the build. Results of the build will be available as /tmp/clang.tar.gz.
-ADD scripts/build_install_llvm.sh /tmp
-RUN /tmp/build_install_llvm.sh ${buildscript_args}
+RUN /tmp/scripts/build_install_llvm.sh ${buildscript_args}

utils/docker/scripts/build_install_llvm.sh

 
 if [ $CLANG_TOOLS_EXTRA_ENABLED -ne 0 ]; then
   echo "Checking out https://llvm.org/svn/llvm-project/clang-tools-extra to $CLANG_BUILD_DIR/src/clang/tools/extra"
   svn co -q $SVN_REV_ARG \
     "https://llvm.org/svn/llvm-project/clang-tools-extra/$LLVM_BRANCH" \
     "$CLANG_BUILD_DIR/src/clang/tools/extra"
 fi
 
+CHECKSUMS_FILE="/tmp/checksums/checksums.txt"
+
+if [ -f "$CHECKSUMS_FILE" ]; then
+  echo "Validating checksums for LLVM checkout..."
+  python "$(dirname $0)/llvm_checksum/llvm_checksum.py" -c "$CHECKSUMS_FILE" \
+    --partial --multi_dir "$CLANG_BUILD_DIR/src"
+else
+  echo "Skipping checksumming checks..."
+fi
+
 mkdir "$CLANG_BUILD_DIR/build"
 pushd "$CLANG_BUILD_DIR/build"
 
 # Run the build as specified in the build arguments.
 echo "Running build"
 cmake -GNinja \
   -DCMAKE_INSTALL_PREFIX="$CLANG_INSTALL_DIR" \
   -DLLVM_ENABLE_PROJECTS="$CMAKE_LLVM_ENABLE_PROJECTS" \

utils/docker/scripts/llvm_checksum/llvm_checksum.py

+#!/usr/bin/python
+""" A small program to compute checksums of LLVM checkout.
+"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import logging
+import sys
+from argparse import ArgumentParser
+from llvm_checksum_utils import *
+
+
+def main():
+  parser = ArgumentParser()
+  parser.add_argument(
+      "-v", "--verbose", action="store_true", help="enable debug logging")
+  parser.add_argument("-c", "--check", metavar="reference_file",
+                      help="read checksums from reference_file and " +
+                           "check they match checksums of llvm_path.")
+  parser.add_argument("--partial", action="store_true",
+                      help="ignore projects from reference_file " +
+                           "that are not checked out in llvm_path.")
+  parser.add_argument("--multi_dir", action="store_true",
+                      help="indicates llvm_path contains llvm, checked out " +
+                            "into multiple directories, as opposed to a " +
+                            "typical single source tree checkout.")
+  parser.add_argument("llvm_path")
+
+  args = parser.parse_args()
+  if args.check is not None:
+    with open(args.check, "r") as f:
+      reference_checksums = ReadLLVMChecksums(f)
+  else:
+    reference_checksums = None
+
+  if args.verbose:
+    logging.basicConfig(level=logging.DEBUG)
+
+  llvm_projects = CreateLLVMProjects(not args.multi_dir)
+  checksums = ComputeLLVMChecksums(args.llvm_path, llvm_projects)
+
+  if reference_checksums is None:
+    WriteLLVMChecksums(checksums, sys.stdout)
+    sys.exit(0)
+
+  if not ValidateChecksums(reference_checksums, checksums, args.partial):
+    sys.stdout.write("Checksums differ.\nNew checksums:\n")
+    WriteLLVMChecksums(checksums, sys.stdout)
+    sys.stdout.write("Reference checksums:\n")
+    WriteLLVMChecksums(reference_checksums, sys.stdout)
+    sys.exit(1)
+  else:
+    sys.stdout.write("Checksums match.")
+
+
+if __name__ == "__main__":
+  main()

utils/docker/scripts/llvm_checksum/llvm_checksum_utils.py

+"""Contains helper functions to compute checksums for LLVM checkouts.
+"""
+from __future__ import absolute_import
+from __future__ import division
+from __future__ import print_function
+
+import hashlib
+import logging
+import os
+import os.path
+import re
+import sys
+
+SVN_DATES_REGEX = re.compile(r"\$(Date|LastChangedDate)[^\$]+\$")
+
+
+class FileKind(object):
+  """ Kind of file in results of checksum_recursively.
+  """
+  FILE = 0
+  VALID_SYMLINK = 1
+  BROKEN_SYMLINK = 2
+
+
+def checksum_recursively(path,
+                         is_ignored,
+                         content_hasher,
+                         hash_algo=hashlib.sha256):
+  """ Computes checksums for all files and symlinks under path.
+
+  Results also include files and symlinks in all recursive subdirectories.
+
+  Args:
+    path: a directory for computing the checksum.
+    is_ignored: a function to indicate parts of directory tree should be ignored
+      during checksumming.
+    content_hasher: a function to compute checksums of the file and symlink
+      contents.
+    hash_algo: a function that creates a hasher object with a preferred hashing
+      algorithm. An 'update' method will be called on created object to
+      populate it with the contents to hash.
+
+  Returns:
+    List of tuple of the form (kind, path, checksum). The `kind` field has type
+    FileKind.
+  """
+
+  def process_file(fullpath):
+    if os.path.islink(fullpath):
+      # Compute checksum of symlink's contents.
+      hasher = hash_algo()
+      symlink_exists = os.path.exists(fullpath)
+      if symlink_exists:
+        # Use contents of the symlink for valid symlinks.
+        content_hasher(fullpath, hasher)
+      else:
+        # Use target for broken symlinks.
+        target = os.readlink(fullpath)
+        hasher.update(target)
+      symlink_checksum = hasher.hexdigest()
+      logging.debug("Checksum %s for %s symlink %s", symlink_checksum, "valid"
+                    if symlink_exists else "broken", fullpath)
+      return (FileKind.VALID_SYMLINK if symlink_exists else
+              FileKind.BROKEN_SYMLINK, fullpath, symlink_checksum)
+    else:
+      # Compute checksum of the file.
+      hasher = hash_algo()
+      content_hasher(fullpath, hasher)
+      file_checksum = hasher.hexdigest()
+      logging.debug("Checksum %s for file %s", file_checksum, fullpath)
+      # Remember the checksum
+      return (FileKind.FILE, fullpath, file_checksum)
+
+  def raise_error(err):
+    raise err
+
+  # Compute symlinks for all files inside dir_path.
+  def process_dir(dir_path):
+    checksums = list()
+    for root, dirs, files in os.walk(dir_path, onerror=raise_error):
+      # Don't recurse into ignored subdirectories.
+      dirs[:] = [d for d in dirs if not is_ignored(os.path.join(root, d))]
+      # Process files.
+      for f in files:
+        fullpath = os.path.join(root, f)
+        if is_ignored(fullpath):
+          continue
+        checksums.append(process_file(fullpath))
+    return checksums
+
+  results = process_dir(path)
+  # Sort results by path.
+  results.sort(key=lambda x: x[1])
+  return results
+
+
+def dir_checksum(path, is_ignored, content_hasher, hash_algo=hashlib.sha256):
+  """ Computes a checksum of the directory.
+
+  Calls checksum_recursively and combines results into a single checksum.
+
+  Args:
+    path: a directory for computing the checksum.
+    is_ignored: a function to check whether the path should be ignored when
+      calculating the hash code.
+    content_hasher: a function to compute checksums of the file and symlink
+      contents.
+    hash_algo: a function that creates a hasher object with the preferred
+      hashing algorithm. An 'update' method will be called on created object to
+      populate it with the contents to hash.
+
+  Returns:
+    Result of calling hasher.hexdigest(), where hasher was created by calling
+    hash_algo().
+  """
+
+  # Computes checksums for files and symlinks under path.
+  file_checksums = checksum_recursively(path, is_ignored, content_hasher,
+                                        hash_algo)
+  hasher = hash_algo()
+  for kind, file_path, checksum in file_checksums:
+    # Feed path of the file to hasher.
+    relpath = os.path.relpath(file_path, path)
+    hasher.update(relpath)
+    # Feed a kind of the file (symlink, file, broken symlink) to the hasher
+    if kind == FileKind.FILE:
+      hasher.update("@file")
+    elif kind == FileKind.VALID_SYMLINK:
+      hasher.update("@symlink")
+    else:
+      assert kind == FileKind.BROKEN_SYMLINK
+      hasher.update("@broken_symlink")
+    # Feed checksum to hasher.
+    hasher.update(checksum)
+  return hasher.hexdigest()
+
+
+class LLVMProject(object):
+  """An LLVM project with a descriptive name and a relative checkout path.
+  """
+
+  def __init__(self, name, relpath):
+    self.name = name
+    self.relpath = relpath
+
+
+def CreateLLVMProjects(single_tree_checkout):
+  """Returns a list of LLVMProject instances, describing relative paths of a typical LLVM checkout.
+
+  Args:
+    single_tree_checkout:
+      When True, relative paths for each project points to a typical single
+        source tree checkout.
+      When False, relative paths for each projects points to a separate
+        directory. However, clang-tools-extra is an exception, its relative path
+        will always be 'clang/tools/extra'.
+  """
+  # FIXME: cover all of llvm projects.
+
+  # Projects that reside inside 'projects/' in a single source tree checkout.
+  ORDINARY_PROJECTS = [
+      "compiler-rt", "dragonegg", "libcxx", "libcxxabi", "libunwind",
+      "parallel-libs", "test-suite"
+  ]
+  # Projects that reside inside 'tools/' in a single source tree checkout.
+  TOOLS_PROJECTS = ["clang", "lld", "lldb", "llgo"]
+
+  if single_tree_checkout:
+    projects = [LLVMProject("llvm", "")]
+    projects += [
+        LLVMProject(p, os.path.join("projects", p)) for p in ORDINARY_PROJECTS
+    ]
+    projects += [
+        LLVMProject(p, os.path.join("tools", p)) for p in TOOLS_PROJECTS
+    ]
+    projects.append(
+        LLVMProject("clang-tools-extra",
+                    os.path.join("tools", "clang", "tools", "extra")))
+  else:
+    projects = [LLVMProject("llvm", "llvm")]
+    projects += [LLVMProject(p, p) for p in ORDINARY_PROJECTS]
+    projects += [LLVMProject(p, p) for p in TOOLS_PROJECTS]
+    projects.append(
+        LLVMProject("clang-tools-extra", os.path.join("clang", "tools",
+                                                      "extra")))
+  return projects
+
+
+def ComputeLLVMChecksums(root_path, projects):
+  """Compute checksums for LLVM sources checked out using svn.
+
+  Args:
+    root_path: a directory of llvm checkout.
+    projects: a list of LLVMProject instances, which describe checkout paths,
+      relative to root_path.
+
+  Returns:
+    A dict mapping from project name to project checksum.
+  """
+  project_dirs = set([os.path.join(root_path, p.relpath) for p in projects])
+
+  def is_ignored(path):
+    # Don't recurse into llvm subprojects.
+    if path in project_dirs:
+      return True
+    # Don't recurse into .svn and .git subdirectories.
+    dirname, basename = os.path.split(path)
+    if basename == ".svn" or basename == ".git":
+      return True
+    return False
+
+  def replace_svn_substitutions(contents):
+    # Replace svn substitutions for $Date$ and $LastChangedDate$.
+    # Unfortunately, these are locale-specific for local machine.
+    return SVN_DATES_REGEX.sub("$\1$", contents)
+
+  def hash_ignoring_subsitutions(file_path, hasher):
+    with open(file_path, "rb") as f:
+      contents = f.read()
+      new_contents = replace_svn_substitutions(contents)
+      if contents != new_contents:
+        logging.debug("Replaced svn keyword substitutions in %s", file_path)
+        logging.debug("\n\tBefore\n%s\n\tAfter\n%s", contents, new_contents)
+      hasher.update(new_contents)
+
+  project_checksums = dict()
+  # Hash each project using dir_checksum.
+  for proj in projects:
+    fullpath = os.path.join(root_path, proj.relpath)
+    if os.path.exists(fullpath):
+      logging.info("Computing checksum for %s", proj.name)
+      checksum = dir_checksum(
+          fullpath, is_ignored, content_hasher=hash_ignoring_subsitutions)
+      project_checksums[proj.name] = checksum
+    else:
+      logging.info("Folder %s doesn't exist, skipping project %s", proj.relpath,
+                   proj.name)
+  return project_checksums
+
+
+def WriteLLVMChecksums(checksums, f):
+  """Writes checksums to a text file.
+
+  Args:
+    checksums: a dict mapping from project name to project checksum (result of
+      ComputeLLVMChecksums).
+    f: a file object to write into.
+  """
+
+  for proj in sorted(checksums.keys()):
+    f.write("{} {}\n".format(checksums[proj], proj))
+
+
+def ReadLLVMChecksums(f):
+  """Reads checksums from a text file, produced by WriteLLVMChecksums.
+
+  Returns:
+    A dict, mapping from project name to project checksum.
+  """
+  checksums = {}
+  while True:
+    line = f.readline()
+    if line == "":
+      break
+    checksum, proj = line.split()
+    checksums[proj] = checksum
+  return checksums
+
+
+def ValidateChecksums(reference_checksums,
+                      new_checksums,
+                      allow_missing_projects=False):
+  """Validates that reference_checksums and new_checksums match.
+
+  Args:
+    reference_checksums: a dict of reference checksums, mapping from a project
+      name to a project checksum.
+    new_checksums: a dict of checksums to be checked, mapping from a project
+      name to a project checksum.
+    allow_missing_projects:
+      When True, reference_checksums may contain more projects than
+        new_checksums. Projects missing from new_checksums are ignored.
+      When False, new_checksums and reference_checksums must contain checksums
+        for the same set of projects. If there is a project in
+        reference_checksums, missing from new_checksums, ValidateChecksums
+        will return False.
+
+  Returns:
+    True, if checksums match with regards to allow_missing_projects flag value.
+    False, otherwise.
+  """
+  if not allow_missing_projects:
+    if len(new_checksums) != len(reference_checksums):
+      return False
+
+  for proj, checksum in new_checksums.iteritems():
+    # We never computed a checksum for this project.
+    if proj not in reference_checksums:
+      return False
+    # Checksum did not match.
+    if reference_checksums[proj] != checksum:
+      return False
+
+  return True