[lit] Allow llvm's build and test systems to support paths with spaces

[bd1976llvm]

Summary

Most major platforms support paths which can contain spaces. Paths with spaces are rare on some systems but they are common on Windows.
LLVM should embrace paths with spaces!

My motivation for this change is that we had a CI system, which we were obliged to use but over which we didn't have control, that checked out source code into sub-directories below a path that contained spaces.

Details

On Windows the build worked out of the box.

On Linux I had to make a few small changes to CMake and bash build scripts to get the build working.

In order to get the lit tests to run I modified lit to escape spaces in substitutions, this change allowed the vast majority of tests to run. I provided a new set of default substitutions containing the windows '^' escape character e.g: %^s, %^t to allow a test writer to use a substitution which does not escape spaces in the (extremely rare) cases where this is required. The main use case for this is where a test writer needs to use a substitution inside of quotation marks - as this would preserve the escape character. I also had to modify tests that used echo to emit lines to a file that is later piped into a process. These tests have been modified to echo quotation marks to surround any paths as these might contain spaces.

One test is still failing: BugPoint/compile-custom.ll. Bugpoint simply does not work with spaces in paths due to naive tokenization of the custom compiler string. See the function lexCommand in bugpoint/ExecutionDriver.cpp.

Testing

I tested this on windows via cmd and and msys and on Linux (bash - ubuntu 14.04).

I also ran the lit testsuite on Linux (It doesn't work on windows.)

[gbedwell]

I also ran the lit testsuite on Linux (It doesn't work on windows.)

Can you clarify what you mean by this? Are you referring to test-suite ( http://llvm.org/docs/TestingGuide.html#test-suite ) or something else?

docs/GettingStartedVS.rst

59

I'm not convinced by this message. LLVM as a whole includes various different projects which new users may come to expect from this document to also work in paths with spaces, but AFAICT this commit only affects the base llvm project itself. I'm also not entirely comfortable with the idea that it's essentially saying "it may work, unless we break it". I make no judgement as to whether it's worth making everything work with paths with spaces, but we should probably err on the side of caution and just document that it's not supported if we can't guarantee it.

[bd1976llvm]

In D29185#657747, @gbedwell wrote:

I also ran the lit testsuite on Linux (It doesn't work on windows.)

Can you clarify what you mean by this? Are you referring to test-suite ( http://llvm.org/docs/TestingGuide.html#test-suite ) or something else?

lit has it's own set of tests that have to be run separately to the llvm lit tests, see comments in llvm/utils/lit/README.txt

[MatzeB]

I trust that this patch fixes the issues we have today and that bugpoint issue really should be fixed.

On the other hand I am not sure we will be able to keep this working consistently in a project with hundreds of comitters. So maybe we should just make our life easier and declare this an unsupported configuration. I would propose to add this to our toplevel CMakeLists.txt:

if(CMAKE_SOURCE_DIR MATCHES " " OR CMAKE_BINARY_DIR MATCHES " ")
  message(WARNING "Spaces in the path to the source or build directory are"
  " supported on a best-effort basis. We recommend to use paths without spaces.")
endif()

[MatzeB]

Would it make sense to split this into

• cmake changes (look good to me)
• test changes (they all look good to me)
• lit changes (I have to find more time to read/think about those)
• bash script changes (they look good to me)

test/lit.cfg

334

Escaping spaces is probably a first step, but wouldn't we rather need a (posix?) shell escape mechanism? (maybe python shlex.quote/pipes.quote?)

[bd1976llvm]

In D29185#657844, @MatzeB wrote:

I trust that this patch fixes the issues we have today and that bugpoint issue really should be fixed.

On the other hand I am not sure we will be able to keep this working consistently in a project with hundreds of comitters. So maybe we should just make our life easier and declare this an unsupported configuration. I would propose to add this to our toplevel CMakeLists.txt:

if(CMAKE_SOURCE_DIR MATCHES " " OR CMAKE_BINARY_DIR MATCHES " ")
  message(WARNING "Spaces in the path to the source or build directory are"
  " supported on a best-effort basis. We recommend to use paths without spaces.")
endif()

Really like this. However, I link the idea of adding a space in the path to one of the build bots (which @rnk states would be achievable) better. Therefore, I will simply remove the paragraph instead.

In D29185#657870, @MatzeB wrote:

Would it make sense to split this into

• cmake changes (look good to me)
• test changes (they all look good to me)
• lit changes (I have to find more time to read/think about those)
• bash script changes (they look good to me)

Ok - great suggestion. However, I will wait until this review is accepted before splitting up the change. In the interest of keeping everything together for review. Reducing the potential for confusion is important :)

docs/GettingStartedVS.rst

59

This document only refers to building LLLVM not any of the other projects e.g: clang, lld etc...
CMake works with paths with spaces. This change adds the ability for lit to support them. Therefore, I think the "has support for" is justified. However, I don't think that there are any build bots that are testing this. That's why I wanted a line saying that the tests might have rotted.

[rnk]

Thanks for tackling this!

docs/GettingStartedVS.rst

59

Honestly I would remove this paragraph altogether. The best documentation is the kind that doesn't need to exist. It'd be good to add a space to the build directory of some Linux buildbot at some point to ensure that this doesn't regress. I don't think we need a special bot configuration for paths with spaces.

test/lit.cfg

334

+1, it should be the opposite of whatever the lit shell does for tokenization, which is roughly to follow bash quoting rules minus variable expansion.

utils/lit/lit/TestRunner.py

736

I can't find any tests using these substitutions. Any reason not to remove them? At least don't add %^:s without a clear use case for it.

[bd1976llvm]

test/lit.cfg

334

We don't need one right now. Why not accept this as an improvement and leave the implementation of a more general escaping mechanism for when/if it is required in the future?

[bd1976llvm]

utils/lit/lit/TestRunner.py

736

I wanted to add %^ versions for all of the existing substitutions so that the ability to cope with spaces in paths is available. I actually can't find any tests using the %:[sSptT] substitutions. Do you think that I can remove these. If out of tree tools need them they can add them to local lit config files.

[bd1976llvm]

utils/lit/lit/TestRunner.py

736

Actually, %:[sSptT] are used in a few tests in LLD.

[bd1976llvm]

Removed paths in spaces warning from the getting started with llvm and visual studio doc. This is because it would be better to simply change one of the build bots to test this and commit to supporting it going forward!

[bd1976llvm]

test/lit.cfg

334

Actually, the lit shell already knows that escape characters are different on windows and Linux. See: https://gitlab.crest.iu.edu/lvoufo/conceptclang/commit/e9201f5ced4ba3b19d4e548deb8e8d3b461f77ab. This patch just extends that idea to allow escape characters on windows rather than just ignoring them.

[bd1976llvm]

I did some more testing of this patch. It turns out that in my testing on linux I had a space in the path to the llvm source code directory but not a space in the path to the build directory. Adding a space in the path to the build directory revealed some problems with the original patch that this new patch addresses.

1. Needed to quote one more argument in the AddLLVM.cmake for --version-script
2. On Linux the "%/[STpst]" substitutions were replacing the '\' escape character with '/'. I have reworked the default substitutions code so that normalization is done before escaping.
3. I have escaped spaces in paths for more substitutions in lit.cfg

[bd1976llvm]

ping

[chandlerc]

So, we have a system that does *not* handle spaces in paths well. I would really like to continue to use the more restrictive rules for lit test names.

I'm happy to have fixes that let things be checked out and built in a path with spaces, but I'd like to require the paths *within* LLVM to continue to not have spaces. Those will work on all systems, whereas paths with spaces will make some users have a (much) harder time integrating with LLVM.

[bd1976llvm]

In D29185#675161, @chandlerc wrote:

So, we have a system that does *not* handle spaces in paths well. I would really like to continue to use the more restrictive rules for lit test names.

I'm happy to have fixes that let things be checked out and built in a path with spaces, but I'd like to require the paths *within* LLVM to continue to not have spaces. Those will work on all systems, whereas paths with spaces will make some users have a (much) harder time integrating with LLVM.

I like this constraint. Unfortunately, as a side-effect of supporting things being checked out and built in a path with spaces, this patch does add the support needed to commit code into paths with spaces within LLVM.

Do you think I need to modify the patch so that CMake/lit actively reject paths with spaces - to prevent such paths getting into llvm?

[mgorny]

I would like to point out that if you proactively reject all paths with spaces, you wouldn't be able to add additional tests that would test support for paths with spaces ;-). While I agree that we should avoid spaces whenever possible, I think it would be reasonable to add tests for paths with spaces wherever we expect them to be well-supported and want to avoid regressions.

[bd1976llvm]

In D29185#677308, @mgorny wrote:

I would like to point out that if you proactively reject all paths with spaces, you wouldn't be able to add additional tests that would test support for paths with spaces ;-). While I agree that we should avoid spaces whenever possible, I think it would be reasonable to add tests for paths with spaces wherever we expect them to be well-supported and want to avoid regressions.

True :) it would be hard to write explicit tests for paths with spaces.

An earlier reviewer (@rnk) suggested that we could modify one of the build bots to add a space in the path to the llvm source tree and the build directory. This would allow the support to be maintained without explicit tests.

[chandlerc]

In D29185#677308, @mgorny wrote:

I would like to point out that if you proactively reject all paths with spaces, you wouldn't be able to add additional tests that would test support for paths with spaces ;-). While I agree that we should avoid spaces whenever possible, I think it would be reasonable to add tests for paths with spaces wherever we expect them to be well-supported and want to avoid regressions.

You could add to lit's own tests a simple test tree under a path with a space and that would seem fine?

My goal is that *within* a lit test tree, the paths of the files that lit is walking don't have spaces.

[gbreynoo]

After the discussion regarding Bugpoint I've made the change myself in patch D29940. Any input would be appreciated.

[bd1976llvm]

In D29185#677753, @chandlerc wrote:

In D29185#677308, @mgorny wrote:

I would like to point out that if you proactively reject all paths with spaces, you wouldn't be able to add additional tests that would test support for paths with spaces ;-). While I agree that we should avoid spaces whenever possible, I think it would be reasonable to add tests for paths with spaces wherever we expect them to be well-supported and want to avoid regressions.

You could add to lit's own tests a simple test tree under a path with a space and that would seem fine?

My goal is that *within* a lit test tree, the paths of the files that lit is walking don't have spaces.

Right. Thanks for the comments.

I was initially going to go back to the position that we don't want space in paths anywhere inside the svn repro and we don't need any tests.

However, that means that any work on lit could easily break support for spaces in paths.

Therefore, I will add a "spaces in paths" test to the lit test suite and update the patch.

I am now opposed to adding explicit checks to lit and CMake for spaces in paths within llvm as this is just not needed. People are just not going to start doing this IMO.

[bd1976llvm]

Added tests in "utils/lit/tests/sp aces" to check that spaces in paths are handled correctly.

This does mean that there is now a directory in the repository with a space in its name.

[bd1976llvm]

Apologies for the slow update to this :(

[bd1976llvm]

ping.

[rnk]

@chandlerc are you happy with the lit test as is checked in with paths with spaces? It's out of the way for any consumer who isn't trying to test lit directly.

Other than that I think this is good.

utils/lit/tests/sp aces/spaces-in-paths.py

3

We could avoid checking in paths with spaces by doing something like:

RUN: mkdir "path with space"
RUN: cp %S/Inputs/check-for-spaces.py "path with space"
RUN: cd "path with space"
RUN: %{lit} -j1 -v check-for-spaces.py

[bd1976llvm]

utils/lit/tests/sp aces/spaces-in-paths.py

3

Nice suggestion! Lets see what others think about allowing *just one space* into the repo :)

cmake/modules/AddLLVM.cmake

add_custom_command(OUTPUT ${native_export_file}
       DEPENDS ${export_file}
       VERBATIM
       COMMENT "Creating export file for ${target_name}")
     if (${CMAKE_SYSTEM_NAME} MATCHES "SunOS")
       set_property(TARGET ${target_name} APPEND_STRING PROPERTY
                    LINK_FLAGS "  -Wl,-M,${CMAKE_CURRENT_BINARY_DIR}/${native_export_file}")
     else()
       set_property(TARGET ${target_name} APPEND_STRING PROPERTY
-                   LINK_FLAGS "  -Wl,--version-script,${CMAKE_CURRENT_BINARY_DIR}/${native_export_file}")
+                   LINK_FLAGS "  -Wl,--version-script,\"${CMAKE_CURRENT_BINARY_DIR}/${native_export_file}\"")
     endif()
   else()
     set(native_export_file "${target_name}.def")
 
     add_custom_command(OUTPUT ${native_export_file}
       COMMAND ${PYTHON_EXECUTABLE} -c "import sys;print(''.join(['EXPORTS\\n']+sys.stdin.readlines(),))"
         < ${export_file} > ${native_export_file}
       DEPENDS ${export_file}
function(llvm_setup_rpath name)
   elseif(UNIX)
     set(_install_rpath "\$ORIGIN/../lib${LLVM_LIBDIR_SUFFIX}" ${extra_libdir})
     if(${CMAKE_SYSTEM_NAME} MATCHES "(FreeBSD|DragonFly)")
       set_property(TARGET ${name} APPEND_STRING PROPERTY
                    LINK_FLAGS " -Wl,-z,origin ")
     elseif(${CMAKE_SYSTEM_NAME} STREQUAL "Linux" AND NOT LLVM_LINKER_IS_GOLD)
       # $ORIGIN is not interpreted at link time by ld.bfd
       set_property(TARGET ${name} APPEND_STRING PROPERTY
-                   LINK_FLAGS " -Wl,-rpath-link,${LLVM_LIBRARY_OUTPUT_INTDIR} ")
+                   LINK_FLAGS " -Wl,-rpath-link,\"${LLVM_LIBRARY_OUTPUT_INTDIR}\" ")
     endif()
   else()
     return()
   endif()
 
   set_target_properties(${name} PROPERTIES
                         BUILD_WITH_INSTALL_RPATH On
                         INSTALL_RPATH "${_install_rpath}"

docs/GettingStartedVS.rst

 
 You will also need the `CMake <http://www.cmake.org/>`_ build system since it
 generates the project files you will use to build with.
 
 If you would like to run the LLVM tests you will need `Python
 <http://www.python.org/>`_. Version 2.7 and newer are known to work. You will
 need `GnuWin32 <http://gnuwin32.sourceforge.net/>`_ tools, too.
 
-Do not install the LLVM directory tree into a path containing spaces (e.g.
-``C:\Documents and Settings\...``) as the configure step will fail.
-
 
 Getting Started
 ===============
 Here's the short story for getting up and running quickly with LLVM:
 
 1. Read the documentation.
 2. Seriously, read the documentation.
 3. Remember that you were warned twice about reading the documentation.
 4. Get the Source Code
 
    * With the distributed files:

test/BugPoint/compile-custom.ll

-; RUN: bugpoint -load %llvmshlibdir/BugpointPasses%shlibext --compile-custom --compile-command="%python %s.py arg1 arg2" --output-prefix %t %s | FileCheck %s
+; RUN: bugpoint -load %llvmshlibdir/BugpointPasses%shlibext --compile-custom --compile-command="%python %^s.py arg1 arg2" --output-prefix %t %s | FileCheck %s
 ; REQUIRES: loadable_module
 
 ; Test that arguments are correctly passed in --compile-command.  The output
 ; of bugpoint includes the output of the custom tool, so we just echo the args
 ; in the tool and check here.
 
 ; CHECK: Error: arg1 arg2
 
 define void @noop() {
     ret void
 }

test/DebugInfo/llvm-symbolizer-split-dwarf-empty.test

 REQUIRES: shell
 RUN: cd %T
 RUN: cp %p/Inputs/split-dwarf-empty.dwo %T
-RUN: echo "%p/Inputs/split-dwarf-empty.o 0xdeadbeef" > %t.input
+RUN: echo "\"%^p/Inputs/split-dwarf-empty.o\" 0xdeadbeef" > %t.input
 
 RUN: llvm-symbolizer --functions=linkage --inlining --demangle=false \
 RUN:    --default-arch=i386 < %t.input | FileCheck %s
 
 CHECK: ??
 CHECK: ??:0:0

test/DebugInfo/llvm-symbolizer.test

-RUN: echo "%p/Inputs/dwarfdump-test.elf-x86-64 0x400559" > %t.input
-RUN: echo "%p/Inputs/dwarfdump-test.elf-x86-64.debuglink 0x400559" >> %t.input
-RUN: echo "%p/Inputs/dwarfdump-test.elf-x86-64 0x400436" >> %t.input
-RUN: echo "%p/Inputs/dwarfdump-test.elf-x86-64 0x400528" >> %t.input
-RUN: echo "%p/Inputs/dwarfdump-test.elf-x86-64 0x400586" >> %t.input
-RUN: echo "%p/Inputs/dwarfdump-test2.elf-x86-64 0x4004e8" >> %t.input
-RUN: echo "%p/Inputs/dwarfdump-test2.elf-x86-64 0x4004f4" >> %t.input
-RUN: echo "%p/Inputs/dwarfdump-test4.elf-x86-64 0x62c" >> %t.input
-RUN: echo "%p/Inputs/dwarfdump-inl-test.elf-x86-64 0x8dc" >> %t.input
-RUN: echo "%p/Inputs/dwarfdump-inl-test.elf-x86-64 0xa05" >> %t.input
-RUN: echo "%p/Inputs/dwarfdump-inl-test.elf-x86-64 0x987" >> %t.input
-RUN: echo "%p/Inputs/dwarfdump-inl-test.high_pc.elf-x86-64 0x568" >> %t.input
-RUN: echo "\"%p/Inputs/dwarfdump-test3.elf-x86-64 space\" 0x640" >> %t.input
-RUN: echo "\"%p/Inputs/dwarfdump-test3.elf-x86-64 space\" 0x633" >> %t.input
-RUN: echo "\"%p/Inputs/dwarfdump-test3.elf-x86-64 space\" 0x62d" >> %t.input
-RUN: echo "%p/Inputs/macho-universal 0x1f84" >> %t.input
-RUN: echo "%p/Inputs/macho-universal:i386 0x1f67" >> %t.input
-RUN: echo "%p/Inputs/macho-universal:x86_64 0x100000f05" >> %t.input
-RUN: echo "%p/Inputs/llvm-symbolizer-dwo-test 0x400514" >> %t.input
-RUN: echo "%p/Inputs/fission-ranges.elf-x86_64 0x720" >> %t.input
-RUN: echo "%p/Inputs/arange-overlap.elf-x86_64 0x714" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-test.elf-x86-64\" 0x400559" > %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-test.elf-x86-64.debuglink\" 0x400559" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-test.elf-x86-64\" 0x400436" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-test.elf-x86-64\" 0x400528" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-test.elf-x86-64\" 0x400586" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-test2.elf-x86-64\" 0x4004e8" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-test2.elf-x86-64\" 0x4004f4" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-test4.elf-x86-64\" 0x62c" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-inl-test.elf-x86-64\" 0x8dc" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-inl-test.elf-x86-64\" 0xa05" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-inl-test.elf-x86-64\" 0x987" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-inl-test.high_pc.elf-x86-64\" 0x568" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-test3.elf-x86-64 space\" 0x640" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-test3.elf-x86-64 space\" 0x633" >> %t.input
+RUN: echo "\"%^p/Inputs/dwarfdump-test3.elf-x86-64 space\" 0x62d" >> %t.input
+RUN: echo "\"%^p/Inputs/macho-universal\" 0x1f84" >> %t.input
+RUN: echo "\"%^p/Inputs/macho-universal:i386\" 0x1f67" >> %t.input
+RUN: echo "\"%^p/Inputs/macho-universal:x86_64\" 0x100000f05" >> %t.input
+RUN: echo "\"%^p/Inputs/llvm-symbolizer-dwo-test\" 0x400514" >> %t.input
+RUN: echo "\"%^p/Inputs/fission-ranges.elf-x86_64\" 0x720" >> %t.input
+RUN: echo "\"%^p/Inputs/arange-overlap.elf-x86_64\" 0x714" >> %t.input
 RUN: cp %p/Inputs/split-dwarf-test.dwo %T
-RUN: echo "%p/Inputs/split-dwarf-test 0x4005d4" >> %t.input
-RUN: echo "%p/Inputs/split-dwarf-test 0x4005c4" >> %t.input
-RUN: echo "%p/Inputs/cross-cu-inlining.x86_64-macho.o 0x17" >> %t.input
+RUN: echo "\"%^p/Inputs/split-dwarf-test\" 0x4005d4" >> %t.input
+RUN: echo "\"%^p/Inputs/split-dwarf-test\" 0x4005c4" >> %t.input
+RUN: echo "\"%^p/Inputs/cross-cu-inlining.x86_64-macho.o\" 0x17" >> %t.input
 
 RUN: llvm-symbolizer --functions=linkage --inlining --demangle=false \
 RUN:    --default-arch=i386 < %t.input | FileCheck --check-prefix=CHECK --check-prefix=SPLIT --check-prefix=DWO %s
 
 Ensure we get the same results in the absence of gmlt-like data in the executable but the presence of a .dwo file
 
-RUN: echo "%p/Inputs/split-dwarf-test-nogmlt 0x4005d4" >> %t.input
-RUN: echo "%p/Inputs/split-dwarf-test-nogmlt 0x4005c4" >> %t.input
+RUN: echo "\"%^p/Inputs/split-dwarf-test-nogmlt\" 0x4005d4" >> %t.input
+RUN: echo "\"%^p/Inputs/split-dwarf-test-nogmlt\" 0x4005c4" >> %t.input
 RUN: llvm-symbolizer --functions=linkage --inlining --demangle=false \
 RUN:    --default-arch=i386 < %t.input | FileCheck --check-prefix=SPLIT --check-prefix=DWO %s
 
 Ensure we get gmlt like results in the absence of a .dwo file but the presence of gmlt-like data in the executable
 
 RUN: rm %T/split-dwarf-test.dwo
-RUN: echo "%p/Inputs/split-dwarf-test 0x4005d4" >> %t.input
-RUN: echo "%p/Inputs/split-dwarf-test 0x4005c4" >> %t.input
+RUN: echo "\"%^p/Inputs/split-dwarf-test\" 0x4005d4" >> %t.input
+RUN: echo "\"%^p/Inputs/split-dwarf-test\" 0x4005c4" >> %t.input
 RUN: llvm-symbolizer --functions=linkage --inlining --demangle=false \
 RUN:    --default-arch=i386 < %t.input | FileCheck --check-prefix=SPLIT --check-prefix=NODWO %s
 
 CHECK:       main
 CHECK-NEXT: /tmp/dbginfo{{[/\\]}}dwarfdump-test.cc:16
 
 CHECK:       main
 CHECK-NEXT: /tmp/dbginfo{{[/\\]}}dwarfdump-test.cc:16

 CHECK-NEXT: main
 CHECK-NEXT: /tmp{{[/\\]}}cross-cu-inlining.c:11:0
 
 RUN: echo "unexisting-file 0x1234" > %t.input2
 RUN: llvm-symbolizer < %t.input2 2>&1 | FileCheck %s --check-prefix=MISSING-FILE
 
 MISSING-FILE: LLVMSymbolizer: error reading file: {{[Nn]}}o such file or directory
 
-RUN: echo "%p/Inputs/macho-universal 0x1f84" > %t.input3
+RUN: echo "\"%^p/Inputs/macho-universal\" 0x1f84" > %t.input3
 RUN: llvm-symbolizer < %t.input3 | FileCheck %s --check-prefix=UNKNOWN-ARCH
 
 UNKNOWN-ARCH-NOT: main
 UNKNOWN-ARCH: ??
 UNKNOWN-ARCH-NOT: main
 
 RUN: echo "0x400559" > %t.input4
 RUN: echo "0x400436" >> %t.input4

 BINARY_C:      {{g$}}
 
 RUN: echo "0x1f1" > %t.input6
 RUN: llvm-symbolizer --obj %p/Inputs/shared-object-stripped.elf-i386 < %t.input6 \
 RUN:   | FileCheck %s --check-prefix=STRIPPED
 
 STRIPPED:  global_func
 
-RUN: echo "%p/Inputs/dwarfdump-test4.elf-x86-64 0x62c" > %t.input7
+RUN: echo "\"%^p/Inputs/dwarfdump-test4.elf-x86-64\" 0x62c" > %t.input7
 RUN: llvm-symbolizer --functions=short --demangle=false < %t.input7 \
 RUN:    | FileCheck %s --check-prefix=SHORT_FUNCTION_NAME
 
 SHORT_FUNCTION_NAME-NOT: _Z1cv

test/LibDriver/libpath.test

 RUN: mkdir -p %T/a %T/b
 RUN: llvm-mc -triple=x86_64-pc-windows-msvc -filetype=obj -o %T/a/foo.obj %S/Inputs/a.s
 RUN: llvm-mc -triple=x86_64-pc-windows-msvc -filetype=obj -o %T/b/foo.obj %S/Inputs/b.s
 
-RUN: env "LIB=%T/a;%T/b" llvm-lib /out:%t1.lib foo.obj
+RUN: env "LIB=%^T/a;%^T/b" llvm-lib /out:%t1.lib foo.obj
 RUN: llvm-nm %t1.lib | FileCheck --check-prefix=A %s
 
 RUN: llvm-lib /out:%t2.lib /libpath:%T/a /libpath:%T/b foo.obj
 RUN: llvm-nm %t2.lib | FileCheck --check-prefix=A %s
 
 RUN: env LIB=%T/a llvm-lib /libpath:%T/b /out:%t3.lib foo.obj
 RUN: llvm-nm %t3.lib | FileCheck --check-prefix=B %s
 
 A: T a
 B: T b

test/Object/mri-addmod.test

 ; RUN: echo create %t.a > %t.mri
-; RUN: echo "addmod  \"%p/Inputs/trivial-object-test.elf-x86-64\" " >> %t.mri
+; RUN: echo "addmod  \"%^p/Inputs/trivial-object-test.elf-x86-64\" " >> %t.mri
 ; RUN: echo save >> %t.mri
 ; RUN: echo end >> %t.mri
 
 ; RUN: llvm-ar -M  < %t.mri
 ; RUN: llvm-nm -M %t.a | FileCheck %s
 
 ; CHECK:      Archive map
 ; CHECK-NEXT: main in trivial-object-test.elf-x86-64

test/Object/nm-archive.test

 RUN: llvm-ar rcs %t2 %t1
 RUN: llvm-nm %t2 | FileCheck %s -check-prefix BITCODE
 
 RUN: rm -f %t2
 RUN: llvm-lib /out:%t2 %t1
 RUN: llvm-nm %t2 | FileCheck %s -check-prefix BITCODE
 
 RUN: rm -f %t2
-RUN: echo /out:%t2 %t1 > %t.rsp
+RUN: echo "\"/out:%^t2\" \"%^t1\"" > %t.rsp
 RUN: llvm-lib @%t.rsp
 RUN: llvm-nm %t2 | FileCheck %s -check-prefix BITCODE
 
 BITCODE:          U SomeOtherFunction
 BITCODE-NEXT:          T main
 BITCODE-NEXT:          U puts
 BITCODE-NEXT:          D var
 

test/Other/ResponseFile.ll

 ; Test that we can recurse, at least a little bit.  The -time-passes flag here
 ; is a hack to make sure that neither echo nor the shell expands the response
 ; file for us.  Tokenization with quotes is tested in unittests.
 ; On Windows, paths contain \ characters, which are escape characters in
 ; GNU-style response files.  So replace \ with \\ to make the tests work there.
-; RUN: echo %s | sed -e 's:\\:\\\\:g' > %t.list1
-; RUN: echo "-time-passes @%t.list1" | sed -e 's:\\:\\\\:g' > %t.list2
+; RUN: echo "\"%^s\"" | sed -e 's:\\:\\\\:g' > %t.list1
+; RUN: echo "-time-passes \"@%^t.list1\"" | sed -e 's:\\:\\\\:g' > %t.list2
 ; RUN: llvm-as @%t.list2 -o %t.bc
 ; RUN: llvm-nm %t.bc 2>&1 | FileCheck %s
 
 ; When the response file begins with UTF8 BOM sequence, we shall remove them.
 ; Neither command below should return a "Could not open input file" error.
 ; RUN: llvm-as @%S/Inputs/utf8-response > /dev/null
 ; RUN: llvm-as @%S/Inputs/utf8-bom-response > /dev/null
 
 ; CHECK: T foobar
 
 define void @foobar() {
   ret void
 }

test/ThinLTO/X86/prefix_replace.ll

 ; Check that changing the output path via prefix-replace works
 ; Use of '/' in paths created here make this unsuitable for Windows.
 ; RUN: mkdir -p %T/oldpath
 ; RUN: opt -module-summary %s -o %T/oldpath/prefix_replace.o
 ; Ensure that there is no existing file at the new path, so we properly
 ; test the creation of the new file there.
 ; RUN: rm -f %T/newpath/prefix_replace.o.thinlto.bc
 
 ; RUN: llvm-lto -thinlto-action=thinlink -o %t.index.bc %T/oldpath/prefix_replace.o
-; RUN: llvm-lto -thinlto-action=distributedindexes -thinlto-prefix-replace="%T/oldpath/;%T/newpath/" -thinlto-index %t.index.bc %T/oldpath/prefix_replace.o
+; RUN: llvm-lto -thinlto-action=distributedindexes -thinlto-prefix-replace="%^T/oldpath/;%^T/newpath/" -thinlto-index %t.index.bc %T/oldpath/prefix_replace.o
 
 ; RUN: ls %T/newpath/prefix_replace.o.thinlto.bc
 
 define void @f() {
 entry:
   ret void
 }

test/Transforms/GCOVProfiling/function-numbering.ll

 ; Test that GCOV instrumentation numbers functions correctly when some
 ; functions aren't emitted.
 
 ; Inject metadata to set the .gcno file location
-; RUN: echo '!14 = !{!"%/T/function-numbering.ll", !0}' > %t1
+; RUN: echo '!14 = !{!"%^/T/function-numbering.ll", !0}' > %t1
 ; RUN: cat %s %t1 > %t2
 
 ; RUN: opt -insert-gcov-profiling -S < %t2 | FileCheck --check-prefix GCDA %s
 ; RUN: llvm-cov gcov -n -dump %T/function-numbering.gcno 2>&1 | FileCheck --check-prefix GCNO %s
 ; RUNN: rm %T/function-numbering.gcno
 
 ; RUN: opt -passes=insert-gcov-profiling -S < %t2 | FileCheck --check-prefix GCDA %s
 ; RUN: llvm-cov gcov -n -dump %T/function-numbering.gcno 2>&1 | FileCheck --check-prefix GCNO %s

test/Transforms/GCOVProfiling/global-ctor.ll

-; RUN: echo '!16 = !{!"%/T/global-ctor.ll", !0}' > %t1
+; RUN: echo '!16 = !{!"%^/T/global-ctor.ll", !0}' > %t1
 ; RUN: cat %s %t1 > %t2
 ; RUN: opt -insert-gcov-profiling -disable-output < %t2
 ; RUN: not grep '_GLOBAL__sub_I_global-ctor' %T/global-ctor.gcno
 ; RUN: rm %T/global-ctor.gcno
 
 ; RUN: opt -passes=insert-gcov-profiling -disable-output < %t2
 ; RUN: not grep '_GLOBAL__sub_I_global-ctor' %T/global-ctor.gcno
 ; RUN: rm %T/global-ctor.gcno

test/Transforms/GCOVProfiling/linezero.ll

-; RUN: sed -e 's|PATTERN|%/T|g' %s | opt -insert-gcov-profiling -disable-output
+; RUN: sed -e 's|PATTERN|%^/T|g' %s | opt -insert-gcov-profiling -disable-output
 ; RUN: rm %T/linezero.gcno
 
-; RUN: sed -e 's|PATTERN|%/T|g' %s | opt -passes=insert-gcov-profiling -disable-output
+; RUN: sed -e 's|PATTERN|%^/T|g' %s | opt -passes=insert-gcov-profiling -disable-output
 ; RUN: rm %T/linezero.gcno
 
 ; This is a crash test.
 
 target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
 target triple = "x86_64-unknown-linux-gnu"
 
 %struct.vector = type { i8 }

test/Transforms/GCOVProfiling/linkagename.ll

-; RUN: echo '!9 = !{!"%/T/linkagename.ll", !0}' > %t1
+; RUN: echo '!9 = !{!"%^/T/linkagename.ll", !0}' > %t1
 ; RUN: cat %s %t1 > %t2
 ; RUN: opt -insert-gcov-profiling -disable-output < %t2
 ; RUN: grep _Z3foov %T/linkagename.gcno
 ; RUN: rm %T/linkagename.gcno
 
 ; RUN: opt -passes=insert-gcov-profiling -disable-output < %t2
 ; RUN: grep _Z3foov %T/linkagename.gcno
 ; RUN: rm %T/linkagename.gcno

test/Transforms/GCOVProfiling/return-block.ll

 ; Inject metadata to set the .gcno file location
-; RUN: echo '!19 = !{!"%/T/return-block.ll", !0}' > %t1
+; RUN: echo '!19 = !{!"%^/T/return-block.ll", !0}' > %t1
 ; RUN: cat %s %t1 > %t2
 
 ; By default, the return block is last.
 ; RUN: opt -insert-gcov-profiling -disable-output %t2
 ; RUN: llvm-cov gcov -n -dump %T/return-block.gcno 2>&1 | FileCheck -check-prefix=CHECK -check-prefix=RETURN-LAST %s
 
 ; But we can optionally emit it second, to match newer gcc versions.
 ; RUN: opt -insert-gcov-profiling -gcov-exit-block-before-body -disable-output %t2

test/Transforms/GCOVProfiling/three-element-mdnode.ll

-; RUN: echo '!10 = !{!"%/T/aaa.gcno", !"%/T/bbb.gcda", !0}' > %t1
+; RUN: echo '!10 = !{!"%^/T/aaa.gcno", !"%/T/bbb.gcda", !0}' > %t1
 ; RUN: cat %s %t1 > %t2
 ; RUN: opt -insert-gcov-profiling -S -o %t3 < %t2
 ; RUN: grep _Z3foov %T/aaa.gcno
 ; RUN: grep bbb.gcda %t3
 ; RUN: rm %T/aaa.gcno
 
 define void @_Z3foov() !dbg !5 {
 entry:

test/Transforms/GCOVProfiling/version.ll

-; RUN: echo '!9 = !{!"%/T/version.ll", !0}' > %t1
+; RUN: echo '!9 = !{!"%^/T/version.ll", !0}' > %t1
 ; RUN: cat %s %t1 > %t2
 ; RUN: opt -insert-gcov-profiling -disable-output < %t2
 ; RUN: head -c8 %T/version.gcno | grep '^oncg.204'
 ; RUN: rm %T/version.gcno
 ; RUN: not opt -insert-gcov-profiling -default-gcov-version=asdfasdf -disable-output < %t2
 ; RUN: opt -insert-gcov-profiling -default-gcov-version=407* -disable-output < %t2
 ; RUN: head -c8 %T/version.gcno | grep '^oncg.704'
 ; RUN: rm %T/version.gcno

test/lit.cfg

 
 # Similarly, have a macro to use llc with DWARF even when the host is win32.
 llc_dwarf = 'llc'
 if re.search(r'win32', config.target_triple):
   llc_dwarf += ' -mtriple='+config.target_triple.replace('-win32', '-mingw32')
 config.substitutions.append( ('%llc_dwarf', llc_dwarf) )
 
 # Add site-specific substitutions.
-config.substitutions.append( ('%gold', config.gold_executable) )
-config.substitutions.append( ('%go', config.go_executable) )
-config.substitutions.append( ('%llvmshlibdir', config.llvm_shlib_dir) )
-config.substitutions.append( ('%shlibext', config.llvm_shlib_ext) )
-config.substitutions.append( ('%exeext', config.llvm_exe_ext) )
-config.substitutions.append( ('%python', config.python_executable) )
-config.substitutions.append( ('%host_cc', config.host_cc) )
+config.substitutions.append( ('%gold', lit.util.escapeSpaces(config.gold_executable)) )
+config.substitutions.append( ('%go', lit.util.escapeSpaces(config.go_executable)) )
+config.substitutions.append( ('%llvmshlibdir', lit.util.escapeSpaces(config.llvm_shlib_dir)) )
+config.substitutions.append( ('%shlibext', lit.util.escapeSpaces(config.llvm_shlib_ext)) )
+config.substitutions.append( ('%exeext', lit.util.escapeSpaces(config.llvm_exe_ext)) )
+config.substitutions.append( ('%python', lit.util.escapeSpaces(config.python_executable)) )
+config.substitutions.append( ('%host_cc', lit.util.escapeSpaces(config.host_cc)) )
 
 # Provide the path to asan runtime lib if available. On darwin, this lib needs
 # to be loaded via DYLD_INSERT_LIBRARIES before libLTO.dylib in case the files
 # to be linked contain instrumented sanitizer code.
 ld64_cmd = config.ld64_executable
 asan_rtlib = get_asan_rtlib()
 if asan_rtlib:
   ld64_cmd = "DYLD_INSERT_LIBRARIES={} {}".format(asan_rtlib, ld64_cmd)
for pattern in [r"\bbugpoint\b(?!-)",
                 # for during testing.
                 r"\| \bcount\b",
                 r"\| \bnot\b"]:
     tool_name, tool_path, tool_pipe = find_tool_substitution(pattern)
     if not tool_path:
         # Warn, but still provide a substitution.
         lit_config.note('Did not find ' + tool_name + ' in ' + llvm_tools_dir)
         tool_path = llvm_tools_dir + '/' + tool_name
-    config.substitutions.append((pattern, tool_pipe + tool_path))
+    config.substitutions.append((pattern, tool_pipe + lit.util.escapeSpaces(tool_path)))
 
 # For tools that are optional depending on the config, we won't warn
 # if they're missing.
 for pattern in [r"\bllvm-go\b",
                 r"\bKaleidoscope-Ch3\b",
                 r"\bKaleidoscope-Ch4\b",
                 r"\bKaleidoscope-Ch5\b",
                 r"\bKaleidoscope-Ch6\b",
                 r"\bKaleidoscope-Ch7\b",
                 r"\bKaleidoscope-Ch8\b"]:
     tool_name, tool_path, tool_pipe = find_tool_substitution(pattern)
     if not tool_path:
         # Provide a substitution anyway, for the sake of consistent errors.
         tool_path = llvm_tools_dir + '/' + tool_name
-    config.substitutions.append((pattern, tool_pipe + tool_path))
+    config.substitutions.append((pattern, tool_pipe + lit.util.escapeSpaces(tool_path)))
 
 
 ### Targets
 
 config.targets = frozenset(config.targets_to_build.split())
 
 for arch in config.targets_to_build.split():
     config.available_features.add(arch.lower() + '-registered-target')

test/tools/llvm-profdata/input-dir.test

 # Create an input file.
 RUN: echo "#" > %t.input
-RUN: echo "%t.dir1" >> %t.input
-RUN: echo "2,%t.dir2" >> %t.input
+RUN: echo "%^t.dir1" >> %t.input
+RUN: echo "2,%^t.dir2" >> %t.input
 
 RUN: mkdir -p %t.dir1
 RUN: mkdir -p %t.dir2/subdir
 
 RUN: echo ' ' > %t.dir1/bar
 RUN: echo ' ' > %t.dir1/foo
 RUN: echo ' ' > %t.dir2/bar
 RUN: echo ' ' > %t.dir2/foo

test/tools/llvm-profdata/input-filenames.test

 # Create an input file.
 RUN: echo '# comment 1' > %t.input
 RUN: echo ' # comment 2' >> %t.input
-RUN: echo " %t.bar" >> %t.input
-RUN: echo " %t.baz" >> %t.input
-RUN: echo "2,%t.weighted" >> %t.input
+RUN: echo " %^t.bar" >> %t.input
+RUN: echo " %^t.baz" >> %t.input
+RUN: echo "2,%^t.weighted" >> %t.input
 
 RUN: echo ' ' > %t.weighted
 RUN: echo ' ' > %t.foo
 RUN: echo ' ' > %t.bar
 RUN: echo ' ' > %t.baz
 
 RUN: llvm-profdata merge -f %t.input -dump-input-file-list -o /dev/null %t.foo | FileCheck %s
 RUN: llvm-profdata merge -input-files %t.input -dump-input-file-list -o /dev/null %t.foo | FileCheck %s
 
 CHECK: 1,{{.*}}.foo
 CHECK-NEXT: 1,{{.*}}.bar
 CHECK-NEXT: 1,{{.*}}.baz
 CHECK-NEXT: 2,{{.*}}.weighted

test/tools/llvm-symbolizer/coff-dwarf.test

 RUN: grep '^ADDR:' %s | sed -s 's/ADDR: //' \
-RUN:     | llvm-symbolizer --inlining --relative-address -obj="%p/Inputs/coff-dwarf.exe" \
+RUN:     | llvm-symbolizer --inlining --relative-address -obj="%^p/Inputs/coff-dwarf.exe" \
 RUN:     | FileCheck %s
 
 This test relies on UnDecorateSymbolName, which is Windows-only.
 REQUIRES: target-windows
 
 ADDR: 0x5009
 ADDR: 0x5038
 
 CHECK: foo(void)
 CHECK: coff-dwarf.cpp:7
 CHECK: bar(void)
 CHECK: coff-dwarf.cpp:13
 CHECK: main
 CHECK: coff-dwarf.cpp:17

test/tools/llvm-symbolizer/coff-exports.test

 RUN: grep '^ADDR:' %s | sed -s 's/ADDR: //' \
-RUN:   | llvm-symbolizer --inlining --relative-address -obj="%p/Inputs/coff-exports.exe" \
+RUN:   | llvm-symbolizer --inlining --relative-address -obj="%^p/Inputs/coff-exports.exe" \
 RUN:   | FileCheck %s
 
 This test relies on UnDecorateSymbolName, which is Win32-only.
 REQUIRES: system-windows
 REQUIRES: target-windows
 FIXME: This test depends on host, not target.
 
 ADDR: 0x500A

test/tools/llvm-symbolizer/dsym.test

-RUN: echo "%p/Inputs/dsym-test-exe 0x0000000100000f90" > %t.input
-RUN: echo "%p/Inputs/dsym-test-exe-second 0x0000000100000f90" >> %t.input
+RUN: echo "\"%^p/Inputs/dsym-test-exe\" 0x0000000100000f90" > %t.input
+RUN: echo "\"%^p/Inputs/dsym-test-exe-second\" 0x0000000100000f90" >> %t.input
 RUN: llvm-symbolizer < %t.input | FileCheck %s --check-prefix=CHECK-NOHINT
 RUN: llvm-symbolizer -dsym-hint=%p/Inputs/dsym-test-exe-differentname.dSYM < %t.input | FileCheck %s --check-prefix=CHECK-HINT
 
 CHECK-NOHINT: main
 CHECK-NOHINT: dsym-test.c
 CHECK-NOHINT: main
 CHECK-NOHINT: ??:0:0
 
 CHECK-HINT: main
 CHECK-HINT: dsym-test.c
 CHECK-HINT: main
 CHECK-HINT: dsym-test.c

test/tools/sanstats/elf.test

 # RUN: yaml2obj %s > %t1.o
 # RUN: yaml2obj %s > %t2.o
 
 # RUN: printf "\x04" > %t.stats
 
-# RUN: printf "%%s" "%t1.o" >> %t.stats
+# RUN: printf "%%s" %t1.o >> %t.stats
 # RUN: printf "\x00" >> %t.stats
 # RUN: printf "\x01\x00\x00\x00\x01\x00\x00\x00" >> %t.stats
 # RUN: printf "\x11\x00\x00\x00\x02\x00\x00\x20" >> %t.stats
 # RUN: printf "\x21\x00\x00\x00\x03\x00\x00\x40" >> %t.stats
 # RUN: printf "\x01\x00\x00\x00\x04\x00\x00\x60" >> %t.stats
 # RUN: printf "\x11\x00\x00\x00\x05\x00\x00\x80" >> %t.stats
 # RUN: printf "\x21\x00\x00\x00\x06\x00\x00\xa0" >> %t.stats
 # RUN: printf "\x00\x00\x00\x00\x00\x00\x00\x00" >> %t.stats
 
-# RUN: printf "%%s" "%t2.o" >> %t.stats
+# RUN: printf "%%s" %t2.o >> %t.stats
 # RUN: printf "\x00" >> %t.stats
 # RUN: printf "\x21\x00\x00\x00\x07\x00\x00\x00" >> %t.stats
 # RUN: printf "\x11\x00\x00\x00\x08\x00\x00\x20" >> %t.stats
 # RUN: printf "\x01\x00\x00\x00\x09\x00\x00\x40" >> %t.stats
 # RUN: printf "\x21\x00\x00\x00\x0b\x00\x00\x60" >> %t.stats
 # RUN: printf "\x11\x00\x00\x00\x0c\x00\x00\x80" >> %t.stats
 # RUN: printf "\x01\x00\x00\x00\x0e\x00\x00\xa0" >> %t.stats
 # RUN: printf "\x00\x00\x00\x00\x00\x00\x00\x00" >> %t.stats

utils/GetRepositoryPath

 #!/bin/sh
 
 usage() {
   echo "usage: $0 <source root>"
   echo "  Prints the source control repository path of the given source"
   echo "  directory, the exact format of the revision string depends on the"
   echo "  source control system. If the source control system isn't known,"
   echo "  the output is empty and the exit code is 1."
   exit 1
 }
 
-if [ $# != 1 ] || [ ! -d $1 ]; then
+if [ $# != 1 ] || [ ! -d "$1" ]; then
   usage;
 fi
 
-cd $1
+cd "$1"
 if [ -d .svn ]; then
   svn info | grep '^URL:' | cut -d: -f2-
 elif [ -f .git/svn/.metadata ]; then
   git svn info | grep 'URL:' | cut -d: -f2-
 elif [ -d .git ]; then
   git remote -v | grep 'fetch' | awk '{ print $2 }' | head -n1
 else
   exit 1;
 fi
 
 exit 0

utils/GetSourceVersion

 #!/bin/sh
 
 usage() {
   echo "usage: $0 <source root>"
   echo "  Prints the source control revision of the given source directory,"
   echo "  the exact format of the revision string depends on the source "
   echo "  control system. If the source control system isn't known, the output"
   echo "  is empty and the exit code is 1."
   exit 1
 }
 
-if [ $# != 1 ] || [ ! -d $1 ]; then
+if [ $# != 1 ] || [ ! -d "$1" ]; then
   usage;
 fi
 
-cd $1
+cd "$1"
 if [ -d .svn ]; then
   svnversion | sed -e "s#\([0-9]*\)[A-Z]*#\1#"
 elif [ -f .git/svn/.metadata ]; then
   git svn info | grep 'Revision:' | cut -d: -f2-
 elif [ -d .git ]; then
   git log -1 --pretty=format:%H
 else
   exit 1;
 fi
 
 exit 0

utils/lit/lit/ShUtil.py

 from __future__ import absolute_import
 import itertools
 
 import lit.util
+from lit.util import escapeChar
 from lit.ShCommands import Command, Pipeline, Seq
 
 class ShLexer:
     def __init__(self, data, win32Escapes = False):
         self.data = data
         self.pos = 0
         self.end = len(data)
         self.win32Escapes = win32Escapes
def maybe_eat(self, c):
         if self.data[self.pos] == c:
             self.pos += 1
             return True
         return False
 
     def lex_arg_fast(self, c):
         # Get the leading whitespace free section.
         chunk = self.data[self.pos - 1:].split(None, 1)[0]
-        
+        escape_char = escapeChar(self.win32Escapes) 
+
         # If it has special characters, the fast path failed.
         if ('|' in chunk or '&' in chunk or 
             '<' in chunk or '>' in chunk or
             "'" in chunk or '"' in chunk or
-            ';' in chunk or '\\' in chunk):
+            ';' in chunk or escape_char in chunk):
             return None
         
         self.pos = self.pos - 1 + len(chunk)
         return chunk
         
     def lex_arg_slow(self, c):
         if c in "'\"":
             str = self.lex_arg_quoted(c)
def lex_arg_slow(self, c):
                 assert isinstance(tok, tuple) and len(tok) == 1
                 return (tok[0], num)                    
             elif c == '"':
                 self.eat()
                 str += self.lex_arg_quoted('"')
             elif c == "'":
                 self.eat()
                 str += self.lex_arg_quoted("'")
-            elif not self.win32Escapes and c == '\\':
-                # Outside of a string, '\\' escapes everything.
+            elif c == escapeChar(self.win32Escapes):
+                # Outside of a string, '\\' (or '^' on windows) escapes everything.
                 self.eat()
                 if self.pos == self.end:
                     lit.util.warning(
                         "escape at end of quoted argument in: %r" % self.data)
                     return str
                 str += self.eat()
             else:
                 str += self.eat()

utils/lit/lit/TestRunner.py

 from __future__ import absolute_import
 import os, signal, subprocess, sys
 import re
 import platform
 import tempfile
 import threading
 
 import lit.ShUtil as ShUtil
 import lit.Test as Test
 import lit.util
-from lit.util import to_bytes, to_string
+from lit.util import to_bytes, to_string, escapeSpaces
 from lit.BooleanExpression import BooleanExpression
 
 class InternalShellError(Exception):
     def __init__(self, command, message):
         self.command = command
         self.message = message
 
 kIsWindows = platform.system() == 'Windows'
def getTempPaths(test):
     root, not test source root."""
     execpath = test.getExecPath()
     execdir,execbase = os.path.split(execpath)
     tmpDir = os.path.join(execdir, 'Output')
     tmpBase = os.path.join(tmpDir, execbase)
     return tmpDir, tmpBase
 
 def getDefaultSubstitutions(test, tmpDir, tmpBase, normalize_slashes=False):
-    sourcepath = test.getSourcePath()
-    sourcedir = os.path.dirname(sourcepath)
+    sourcePath = test.getSourcePath()
+    sourceDir = os.path.dirname(sourcePath)
+
+    normSourcePath = sourcePath.replace('\\', '/')
+    normSourceDir = sourceDir.replace('\\', '/')
+    normTmpDir = tmpDir.replace('\\', '/')
+    normTmpBase = tmpBase.replace('\\', '/')
 
     # Normalize slashes, if requested.
     if normalize_slashes:
-        sourcepath = sourcepath.replace('\\', '/')
-        sourcedir = sourcedir.replace('\\', '/')
-        tmpDir = tmpDir.replace('\\', '/')
-        tmpBase = tmpBase.replace('\\', '/')
+        sourcePath = normSourcePath
+        sourceDir = normSourceDir
+        tmpDir = normTmpDir
+        tmpBase = normTmpBase
+
+    # handle spaces in paths
+    escapedSourcePath = escapeSpaces(sourcePath)
+    escapedSourceDir = escapeSpaces(sourceDir)
+    escapedTmpDir = escapeSpaces(tmpDir)
+    escapedTmpBase = escapeSpaces(tmpBase)
 
     # We use #_MARKER_# to hide %% while we do the other substitutions.
     substitutions = []
     substitutions.extend([('%%', '#_MARKER_#')])
     substitutions.extend(test.config.substitutions)
-    substitutions.extend([('%s', sourcepath),
-                          ('%S', sourcedir),
-                          ('%p', sourcedir),
+    substitutions.extend([('%s', escapedSourcePath),
+                          ('%S', escapedSourceDir),
+                          ('%p', escapedSourceDir),
                           ('%{pathsep}', os.pathsep),
-                          ('%t', tmpBase + '.tmp'),
-                          ('%T', tmpDir),
+                          ('%t', escapedTmpBase + '.tmp'),
+                          ('%T', escapedTmpDir),
                           ('#_MARKER_#', '%')])
 
+    # "%^[STpst]" should not have spaces escaped.
+    substitutions.extend([
+            ('%\^s', sourcePath),
+            ('%\^S', sourceDir),
+            ('%\^p', sourceDir),
+            ('%\^t', tmpBase + '.tmp'),
+            ('%\^T', tmpDir),
+            ])
+
     # "%/[STpst]" should be normalized.
     substitutions.extend([
-            ('%/s', sourcepath.replace('\\', '/')),
-            ('%/S', sourcedir.replace('\\', '/')),
-            ('%/p', sourcedir.replace('\\', '/')),
-            ('%/t', tmpBase.replace('\\', '/') + '.tmp'),
-            ('%/T', tmpDir.replace('\\', '/')),
+            ('%/s', escapeSpaces(normSourcePath)),
+            ('%/S', escapeSpaces(normSourceDir)),
+            ('%/p', escapeSpaces(normSourceDir)),
+            ('%/t', escapeSpaces(normTmpBase + '.tmp')),
+            ('%/T', escapeSpaces(normTmpDir)),
+            ])
+
+    # "%^/[STpst]" should be normalized and unescaped.
+    substitutions.extend([
+            ('%\^/s', normSourcePath),
+            ('%\^/S', normSourceDir),
+            ('%\^/p', normSourceDir),
+            ('%\^/t', normTmpBase + '.tmp'),
+            ('%\^/T', normTmpDir),
+            ])
+
+    # "%:[STpst]" are escaped paths without colons.
+    if kIsWindows:
+        substitutions.extend([
+                ('%:s', re.sub(r'^(.):', r'\1', escapedSourcePath)),
+                ('%:S', re.sub(r'^(.):', r'\1', escapedSourceDir)),
+                ('%:p', re.sub(r'^(.):', r'\1', escapedSourceDir)),
+                ('%:t', re.sub(r'^(.):', r'\1', escapedTmpBase) + '.tmp'),
+                ('%:T', re.sub(r'^(.):', r'\1', escapedTmpDir)),
+                ])
+    else:
+        substitutions.extend([
+                ('%:s', escapedSourcePath),
+                ('%:S', escapedSourceDir),
+                ('%:p', escapedSourceDir),
+                ('%:t', escapedTmpBase + '.tmp'),
+                ('%:T', escapedTmpDir),
                 ])
 
-    # "%:[STpst]" are paths without colons.
+    # "%^:[STpst]" are unescaped paths without colons.
     if kIsWindows:
         substitutions.extend([
-                ('%:s', re.sub(r'^(.):', r'\1', sourcepath)),
-                ('%:S', re.sub(r'^(.):', r'\1', sourcedir)),
-                ('%:p', re.sub(r'^(.):', r'\1', sourcedir)),
-                ('%:t', re.sub(r'^(.):', r'\1', tmpBase) + '.tmp'),
-                ('%:T', re.sub(r'^(.):', r'\1', tmpDir)),
+                ('%\^:s', re.sub(r'^(.):', r'\1', sourcePath)),
+                ('%\^:S', re.sub(r'^(.):', r'\1', sourceDir)),
+                ('%\^:p', re.sub(r'^(.):', r'\1', sourceDir)),
+                ('%\^:t', re.sub(r'^(.):', r'\1', tmpBase) + '.tmp'),
+                ('%\^:T', re.sub(r'^(.):', r'\1', tmpDir)),
                 ])
     else:
         substitutions.extend([
-                ('%:s', sourcepath),
-                ('%:S', sourcedir),
-                ('%:p', sourcedir),
-                ('%:t', tmpBase + '.tmp'),
-                ('%:T', tmpDir),
+                ('%\^:s', sourcePath),
+                ('%\^:S', sourceDir),
+                ('%\^:p', sourceDir),
+                ('%\^:t', tmpBase + '.tmp'),
+                ('%\^:T', tmpDir),
                 ])
     return substitutions
 
 def applySubstitutions(script, substitutions):
     """Apply substitutions to the script.  Allow full regular expression syntax.
     Replace each matching occurrence of regular expression pattern a with
     substitution b in line ln."""
     def processLine(ln):

utils/lit/lit/util.py

 def convert_string(bytes):
     try:
         return to_string(bytes.decode('utf-8'))
     except AttributeError: # 'str' object has no attribute 'decode'.
         return str(bytes)
     except UnicodeError:
         return str(bytes)
 
+def escapeChar (win32Escapes):
+    if win32Escapes:
+        return '^'
+    else:
+        return '\\'
+
+def escapeSpaces (s):
+    """
+    Escapes spaces in strings. Igores whitespace at the start and end of the
+    string.
+    """
+    first = len(s) - len(s.lstrip())
+    last = len(s.rstrip())
+
+    escape = escapeChar('win32' == sys.platform)
+    escapedStr = s[first:last].replace(' ', escape + ' ')
+
+    return s[:first] + escapedStr + s[last:]
+
 def detectCPUs():
     """
     Detects the number of CPUs on a system. Cribbed from pp.
     """
     # Linux, Unix and MacOS:
     if hasattr(os, "sysconf"):
         if "SC_NPROCESSORS_ONLN" in os.sysconf_names:
             # Linux & Unix:

utils/lit/tests/lit.cfg

 # test_source_root: The root path where tests are located.
 config.test_source_root = os.path.dirname(__file__)
 config.test_exec_root = config.test_source_root
 
 config.target_triple = '(unused)'
 
 src_root = os.path.join(config.test_source_root, '..')
 config.environment['PYTHONPATH'] = src_root
-config.substitutions.append(('%{src_root}', src_root))
-config.substitutions.append(('%{inputs}', os.path.join(
-            src_root, 'tests', 'Inputs')))
+
+# escaped versions
+config.substitutions.append(('%{src_root}', lit.util.escapeSpaces(src_root)))
+config.substitutions.append(('%{inputs}', lit.util.escapeSpaces(os.path.join(
+            src_root, 'tests', 'Inputs'))))
 config.substitutions.append(('%{lit}', "%%{python} %s" % (
-            os.path.join(src_root, 'lit.py'),)))
-config.substitutions.append(('%{python}', sys.executable))
+            lit.util.escapeSpaces(os.path.join(src_root, 'lit.py')),)))
+config.substitutions.append(('%{python}', lit.util.escapeSpaces(sys.executable)))
 
 # Enable coverage.py reporting, assuming the coverage module has been installed
 # and sitecustomize.py in the virtualenv has been modified appropriately.
 if lit_config.params.get('check-coverage', None):
     config.environment['COVERAGE_PROCESS_START'] = os.path.join(
         os.path.dirname(__file__), ".coveragerc")
 
 # Add a feature to detect the Python version.

utils/lit/tests/sp aces/check-for-spaces.py

+# check that spaces in paths are escaped properly
+# RUN: python %s %s | FileCheck %s --check-prefix ESCAPED
+# ESCAPED: sp aces
+
+# check that ^ substitutions do not escape spaces
+# RUN: python %s %^s | FileCheck %s --check-prefix SPLIT
+# SPLIT-NOT: sp aces
+
+import sys
+print sys.argv [-1]
+
+

utils/lit/tests/sp aces/spaces-in-paths.py

+# check that lit can deal with a path continaing spaces 
+
+# RUN: %{lit} -j 1 -v check-for-spaces.py

utils/lit/tests/unit/ShUtil.py

def test_quoting(self):
         self.assertEqual(self.lex(""" he"llo wo"rld """),
                          ["hello world"])
         self.assertEqual(self.lex(""" a\\ b a\\\\b """),
                          ["a b", "a\\b"])
         self.assertEqual(self.lex(""" "" "" """),
                          ["", ""])
         self.assertEqual(self.lex(""" a\\ b """, win32Escapes = True),
                          ['a\\', 'b'])
+        self.assertEqual(self.lex(""" a^ b """, win32Escapes = True),
+                         ['a b'])
 
 class TestShParse(unittest.TestCase):
     def parse(self, str):
         return ShParser(str).parse()
 
     def test_basic(self):
         self.assertEqual(self.parse('echo hello'),
                          Pipeline([Command(['echo', 'hello'], [])], False))