Add .init_array support to Wasm LLD

[ncw]

Summary

I hope this is the right approach!

What I've done is added some (hacky?) plumbing so that for Wasm we export the start/end of .init_array and .fini_array using the well-known __{init,fini}_array_{start,end} symbols.

(Aside: the way this is implemented on desktop (Linux) platforms is that the driver which invokes LLD adds some special files "crtbegin.o" and "crtend.o" to the link line. Those are shipped as part of the platform, and register dummy functions with the magic names __{init,fini}_array_{start,end}, and in turn libc has a weak dependency on those symbols so that it can traverse the array of symbols.)

I had a go with the same approach as on desktop Linux - but it didn't work quite as simply as I'd like, plus there's the nuisance of actually shipping and maintaining the crtbegin/end.o files...

The approach I've gone for instead is to explicitly fudge in those symbols in LLD, and export their addresses manually.

It all works - which is perhaps the only redeeming virtue of this sort of special-casing.

Note this approach should be portable to other libc implementations (not just Musl), since the symbols __init_array_start etc are mandated by the ABI (Musl doesn't create/define those, it relies on GCC to supply them). So, creating those symbols ourselves in LLD is OK, we can add a note to the C ABI document on GitHub to say that crt1.o and __init_array_start from Musl's point of view will work exactly as on desktop linux.

Testing

I've tested with Musl, and the tests in lld/tests/wasm showcase a basic C++ file that uses a global static constructor and a C function annotated with __attribute__((constructor)). The Wasm file at the end does indeed run nicely in the browser, calling each global constructor exactly once.

[sunfish]

I'm not too bothered by the special-casing here.

That said, I have an idea for a different approach and I'm curious what you and others think. Instead of traditional .init_array sections, what if we taught the linker to link multiple start sections, by synthesizing a new start function that calls all the start functions from the input? So if you link a.o and b.o, and each have a start function, the output would contain a start function containing two calls, one to a.o's start function and one to b.o's. Start functions never have arguments or return values, so the synthesized start function would just look like "0x10 <first callee> 0x10 <second callee> ..." (0x10 is the call opcode).

This would require changes to how eg. musl and other things work. However, it would fit well within the spirit of using the wasm format for .o files in a natural way.

(There's also .fini_array, but we can talk about that if the overall idea is worth pursuing.)

[ruiu]

I wonder if you can do everything in the loader. We can let lld create .init_array and .fini_array (which is default behavior) and make a change to the loader so that the loader executes each function pointer in .init_array and .fini_array sections, no?

[ncw]

@sunfish, I think that's an interesting idea. Here are some thoughts I've had:

Pros of building "composite start function" on link:

• Uses native WebAssembly information rather than custom section. That is a genuinely nice change.

Cons:

• Adds asymmetry to destructor handling, when it feels like they should be mirror features
• Won't work out of the box with musl (needless friction with upstream?)
• Less extensible to other sections. I don't expect any other sections to need this treatment, currently, but maybe other languages apart from C have initialisation concerns - like Haskell or Rust...? Sticking as close to ELF as possible for C sets a good precedent for making those other languages easy to integrate. If you're suggesting adapting Musl's startup to WebAssembly, are we going to have to adapt the runtimes of other languages too?
• Assumes that start functions can be run in any order, but that's actually not exactly the case. In particular, libc does its own initialisation _before_ any user-defined constructors, so that thread-local storage and printf are usable in all user-supplied code. It's a bit cheeky of libc to get special privileges like that (by virtue of supplying the entrypoint, it's able to do its own initialisation first). There would have to be a way to force this behaviour to continue - but it wouldn't be insurmountable, you'd just to have some "loose" object file (eg. "crt2.o") that we could guarantee appears first on the link-line.

In D40614#939712, @ruiu wrote:

I wonder if you can do everything in the loader. We can let lld create .init_array and .fini_array (which is default behavior) and make a change to the loader so that the loader executes each function pointer in .init_array and .fini_array sections, no?

I'm not sure what you mean by the "loader" - is that the runtime WebAssembly interpreter/AOT-compiler? At runtime, the "sections" don't exist anymore, they're only available here to the linker. At runtime, you're right we want the interpreter to execute each function, and that's only achievable in WebAssembly via the START function.

[sunfish]

Cons:
Adds asymmetry to destructor handling, when it feels like they should be mirror features

Actually, it's not entirely clear that .fini_array is desirable for current wasm. It's nice to be compatible with tools that expect it, but it's usually better to use atexit/__cxa_atexit instead, because that makes it feasible to run destructors in reverse order of the runtime order of the corresponding constructors. For example, C++ uses __cxa_atexit and not .fini_array on ELF, in part for this reason.

Won't work out of the box with musl (needless friction with upstream?)

It would be some friction. I don't think it's needless though :-).

Less extensible to other sections. I don't expect any other sections to need this treatment, currently, but maybe other languages apart from C have initialisation concerns - like Haskell or Rust...? Sticking as close to ELF as possible for C sets a good precedent for making those other languages easy to integrate. If you're suggesting adapting Musl's startup to WebAssembly, are we going to have to adapt the runtimes of other languages too?

ELF is a standard that some tools support, so by emulating ELF, we make it easier to port these tools.

On the other hand, there are people looking to port and build tools for wasm that have no knowledge of ELF. If we do things the wasm way, rather than the ELF way, it'll be easier to promote interoperability between all these tools on wasm, because wasm is the thing they're all guaranteed to have in common.

Assumes that start functions can be run in any order, but that's actually not exactly the case. In particular, libc does its own initialisation _before_ any user-defined constructors, so that thread-local storage and printf are usable in all user-supplied code. It's a bit cheeky of libc to get special privileges like that (by virtue of supplying the entrypoint, it's able to do its own initialisation first). There would have to be a way to force this behaviour to continue - but it wouldn't be insurmountable, you'd just to have some "loose" object file (eg. "crt2.o") that we could guarantee appears first on the link-line.

We'd just do the same thing that ELF does: order the functions in the combined start function in the order they appear on the link command line. Then we just put the initialization in crt1.o/crti.o/crtbegin.o and but those before any user code, and we're good, just like ELF.

[ncw]

(Rebased.)

We probably need some move votes then to reach consensus. On the one hand, Dan's idea is maybe a bit cleaner for C/C++ in the long term. There should probably be a principle on how much we want to emulate ELF, which we would then stick to. Choosing not to implement support for .fini_array (__attribute__((destructor)) on exit()) also seems a bit premature, although I admit it's probably hardly-used. Potentially a good number of Emscripten programs do call exit() though.

At the moment, LLD is far, far from being able to link "generic" Wasm files together. The entire model revolves around custom sections providing ELFy data defined in the "Tool conventions", so using a start section instead of .init_array really isn't going to a huge difference for generic tools that want to produce inputs for LLD to process. I can appreciate the desire to reduce the ELFyness of the conventions though.

Either way, we could still do this (smaller) change right now, and come back and do Dan's change later, once we've got C++ working.

[ncw]

Rubbish, when I rebased I must have sent the wrong diff... OK, here's the correct rebased diff.

[sbc100]

I've also been working on a similar change: https://reviews.llvm.org/D40760

Sorry its been in the works for a while but I hadn't uploaded it yet.

[ncw]

CHANGES:

• Rebased...
• Integrated Sam's changes from D40760, in particular his tests and his renaming/refactoring (where possible)

I suspect there'll be some more conflicts as soon as D40859 is merged. I'm quite happy for D40859 to be merged first, and I can rebase this patch.

(NB. My entrypoint changes are "Ready to Land" in Phabricator, so I've rebased on top of those, since surely they'll be merged sooner than this. I'm just trying to minimise the amount of rebasing and conflicts.)

[ncw]

As expected, it needed rebasing after D40859. I don't know whether this will be merged - but I've done the rebase anyway just so we're at least looking at the right change.

Now that Sam's Global changes are merged, it's a shorter diff.

[ncw]

Abandoning - but! I need to remember to keep the test files, which might still be useful.

include/llvm/ADT/Triple.h

bool isARM() const {
     return getArch() == Triple::arm || getArch() == Triple::armeb;
   }
 
   /// Tests whether the target is AArch64 (little and big endian).
   bool isAArch64() const {
     return getArch() == Triple::aarch64 || getArch() == Triple::aarch64_be;
   }
 
-  /// Tests wether the target supports comdat
+  /// Tests whether the target supports comdat
   bool supportsCOMDAT() const {
-    return !isOSBinFormatMachO() && !isOSBinFormatWasm();
+    return !isOSBinFormatMachO();
   }
 
   /// @}
   /// @name Mutators
   /// @{
 
   /// setArch - Set the architecture (first) component of the triple
   /// to a known type.

lib/CodeGen/TargetLoweringObjectFileImpl.cpp

     return;
 
   // TODO(sbc): At some point we may need COMDAT support but currently
   // they are not supported.
   report_fatal_error("WebAssembly doesn't support COMDATs, '" + C->getName() +
                      "' cannot be lowered.");
 }
 
+static SectionKind getWasmKindForNamedSection(StringRef Name, SectionKind K) {
+  // If we're told we have function data, then use that.
+  if (K.isText() || Name == ".text.__startup")
+    return SectionKind::getText();
+
+  // Otherwise, ignore whatever section type the generic impl detected and use
+  // a plain data section.
+  return SectionKind::getData();
+}
+
 MCSection *TargetLoweringObjectFileWasm::getExplicitSectionGlobal(
     const GlobalObject *GO, SectionKind Kind, const TargetMachine &TM) const {
   StringRef Name = GO->getSection();
   checkWasmComdat(GO);
-  return getContext().getWasmSection(Name, SectionKind::getData());
+  Kind = getWasmKindForNamedSection(Name, Kind);
+  return getContext().getWasmSection(Name, Kind);
 }
 
 static MCSectionWasm *selectWasmSectionForGlobal(
     MCContext &Ctx, const GlobalObject *GO, SectionKind Kind, Mangler &Mang,
     const TargetMachine &TM, bool EmitUniqueSection, unsigned *NextUniqueID) {
   StringRef Group = "";
   checkWasmComdat(GO);
 

lib/MC/WasmObjectWriter.cpp

class WasmObjectWriter : public MCObjectWriter {
   void startSection(SectionBookkeeping &Section, unsigned SectionId,
                     const char *Name = nullptr);
   void endSection(SectionBookkeeping &Section);
 
 public:
   WasmObjectWriter(std::unique_ptr<MCWasmObjectTargetWriter> MOTW,
                    raw_pwrite_stream &OS)
       : MCObjectWriter(OS, /*IsLittleEndian=*/true),
-        TargetObjectWriter(std::move(MOTW)) {}
+        TargetObjectWriter(std::move(MOTW)) { /*llvm::DebugFlag = true;*/ } // XXX
 
 private:
   ~WasmObjectWriter() override;
 
   void reset() override {
     CodeRelocations.clear();
     DataRelocations.clear();
     TypeIndices.clear();

 static void addData(SmallVectorImpl<char> &DataBytes,
                     MCSectionWasm &DataSection) {
   DEBUG(errs() << "addData: " << DataSection.getSectionName() << "\n");
 
   DataBytes.resize(alignTo(DataBytes.size(), DataSection.getAlignment()));
 
   size_t LastFragmentSize = 0;
   for (const MCFragment &Frag : DataSection) {
-    if (Frag.hasInstructions())
+    if (Frag.hasInstructions()) {
+      Frag.dump(); // XXX
+      errs() << "\n";
+      DataSection.dump();
+      errs() << "\n";
+      const SectionKind Kind = DataSection.getKind();
+      if (Kind.isMetadata()) errs() << "isMetadata" << "\n";
+      if (Kind.isText()) errs() << "isText" << "\n";
+      if (Kind.isExecuteOnly()) errs() << "isExecuteOnly" << "\n";
+      if (Kind.isReadOnly()) errs() << "isReadOnly" << "\n";
+      if (Kind.isMergeableCString()) errs() << "isMergeableCString" << "\n";
+      if (Kind.isMergeable1ByteCString()) errs() << "isMergeable1ByteCString" << "\n";
+      if (Kind.isMergeable2ByteCString()) errs() << "isMergeable2ByteCString" << "\n";
+      if (Kind.isMergeable4ByteCString()) errs() << "isMergeable4ByteCString" << "\n";
+      if (Kind.isMergeableConst()) errs() << "isMergeableConst" << "\n";
+      if (Kind.isMergeableConst4()) errs() << "isMergeableConst4" << "\n";
+      if (Kind.isMergeableConst8()) errs() << "isMergeableConst8" << "\n";
+      if (Kind.isMergeableConst16()) errs() << "isMergeableConst16" << "\n";
+      if (Kind.isMergeableConst32()) errs() << "isMergeableConst32" << "\n";
+      if (Kind.isWriteable()) errs() << "isWriteable" << "\n";
+      if (Kind.isThreadLocal()) errs() << "isThreadLocal" << "\n";
+      if (Kind.isThreadBSS()) errs() << "isThreadBSS" << "\n";
+      if (Kind.isThreadData()) errs() << "isThreadData" << "\n";
+      if (Kind.isGlobalWriteableData()) errs() << "isGlobalWriteableData" << "\n";
+      if (Kind.isBSS()) errs() << "isBSS" << "\n";
+      if (Kind.isBSSLocal()) errs() << "isBSSLocal" << "\n";
+      if (Kind.isBSSExtern()) errs() << "isBSSExtern" << "\n";
+      if (Kind.isCommon()) errs() << "isCommon" << "\n";
+      if (Kind.isData()) errs() << "isData" << "\n";
+      if (Kind.isReadOnlyWithRel()) errs() << "isReadOnlyWithRel" << "\n";
       report_fatal_error("only data supported in data sections");
+    }
 
     if (auto *Align = dyn_cast<MCAlignFragment>(&Frag)) {
       if (Align->getValueSize() != 1)
         report_fatal_error("only byte values supported for alignment");
       // If nops are requested, use zeros, as this is the data section.
       uint8_t Value = Align->hasEmitNops() ? 0 : Align->getValue();
       uint64_t Size = std::min<uint64_t>(alignTo(DataBytes.size(),
                                                  Align->getAlignment()),

utils/lit/lit/TestRunner.py

 from __future__ import absolute_import
+import difflib
+import errno
+import functools
+import itertools
+import getopt
 import os, signal, subprocess, sys
 import re
+import stat
 import platform
+import shutil
 import tempfile
 import threading
 
 try:
     from StringIO import StringIO
 except ImportError:
     from io import StringIO
 
def executeBuiltinEcho(cmd, shenv):
 
     for (name, mode, f, path) in opened_files:
         f.close()
 
     if not is_redirected:
         return stdout.getvalue()
     return ""
 
+def executeBuiltinMkdir(cmd, cmd_shenv):
+    """executeBuiltinMkdir - Create new directories."""
+    args = expand_glob_expressions(cmd.args, cmd_shenv.cwd)[1:]
+    try:
+        opts, args = getopt.gnu_getopt(args, 'p')
+    except getopt.GetoptError as err:
+        raise InternalShellError(cmd, "Unsupported: 'mkdir':  %s" % str(err))
+
+    parent = False
+    for o, a in opts:
+        if o == "-p":
+            parent = True
+        else:
+            assert False, "unhandled option"
+
+    if len(args) == 0:
+        raise InternalShellError(cmd, "Error: 'mkdir' is missing an operand")
+
+    stderr = StringIO()
+    exitCode = 0
+    for dir in args:
+        if not os.path.isabs(dir):
+            dir = os.path.realpath(os.path.join(cmd_shenv.cwd, dir))
+        if parent:
+            lit.util.mkdir_p(dir)
+        else:
+            try:
+                os.mkdir(dir)
+            except OSError as err:
+                stderr.write("Error: 'mkdir' command failed, %s\n" % str(err))
+                exitCode = 1
+    return ShellCommandResult(cmd, "", stderr.getvalue(), exitCode, False)
+
+def executeBuiltinDiff(cmd, cmd_shenv):
+    """executeBuiltinDiff - Compare files line by line."""
+    args = expand_glob_expressions(cmd.args, cmd_shenv.cwd)[1:]
+    try:
+        opts, args = getopt.gnu_getopt(args, "wbu", ["strip-trailing-cr"])
+    except getopt.GetoptError as err:
+        raise InternalShellError(cmd, "Unsupported: 'diff':  %s" % str(err))
+
+    filelines, filepaths = ([] for i in range(2))
+    ignore_all_space = False
+    ignore_space_change = False
+    unified_diff = False
+    strip_trailing_cr = False
+    for o, a in opts:
+        if o == "-w":
+            ignore_all_space = True
+        elif o == "-b":
+            ignore_space_change = True
+        elif o == "-u":
+            unified_diff = True
+        elif o == "--strip-trailing-cr":
+            strip_trailing_cr = True
+        else:
+            assert False, "unhandled option"
+
+    if len(args) != 2:
+        raise InternalShellError(cmd, "Error:  missing or extra operand")
+
+    stderr = StringIO()
+    stdout = StringIO()
+    exitCode = 0
+    try:
+        for file in args:
+            if not os.path.isabs(file):
+                file = os.path.realpath(os.path.join(cmd_shenv.cwd, file))
+            filepaths.append(file)
+            with open(file, 'r') as f:
+                filelines.append(f.readlines())
+
+        def compose2(f, g):
+            return lambda x: f(g(x))
+
+        f = lambda x: x
+        if strip_trailing_cr:
+            f = compose2(lambda line: line.rstrip('\r'), f)
+        if ignore_all_space or ignore_space_change:
+            ignoreSpace = lambda line, separator: separator.join(line.split())
+            ignoreAllSpaceOrSpaceChange = functools.partial(ignoreSpace, separator='' if ignore_all_space else ' ')
+            f = compose2(ignoreAllSpaceOrSpaceChange, f)
+
+        for idx, lines in enumerate(filelines):
+            filelines[idx]= [f(line) for line in lines]
+
+        func = difflib.unified_diff if unified_diff else difflib.context_diff
+        for diff in func(filelines[0], filelines[1], filepaths[0], filepaths[1]):
+            stdout.write(diff)
+            exitCode = 1
+    except IOError as err:
+        stderr.write("Error: 'diff' command failed, %s\n" % str(err))
+        exitCode = 1
+
+    return ShellCommandResult(cmd, stdout.getvalue(), stderr.getvalue(), exitCode, False)
+
+def executeBuiltinRm(cmd, cmd_shenv):
+    """executeBuiltinRm - Removes (deletes) files or directories."""
+    args = expand_glob_expressions(cmd.args, cmd_shenv.cwd)[1:]
+    try:
+        opts, args = getopt.gnu_getopt(args, "frR", ["--recursive"])
+    except getopt.GetoptError as err:
+        raise InternalShellError(cmd, "Unsupported: 'rm':  %s" % str(err))
+
+    force = False
+    recursive = False
+    for o, a in opts:
+        if o == "-f":
+            force = True
+        elif o in ("-r", "-R", "--recursive"):
+            recursive = True
+        else:
+            assert False, "unhandled option"
+
+    if len(args) == 0:
+        raise InternalShellError(cmd, "Error: 'rm' is missing an operand")
+
+    def on_rm_error(func, path, exc_info):
+        # path contains the path of the file that couldn't be removed
+        # let's just assume that it's read-only and remove it.
+        os.chmod(path, stat.S_IMODE( os.stat(path).st_mode) | stat.S_IWRITE)
+        os.remove(path)
+
+    stderr = StringIO()
+    exitCode = 0
+    for path in args:
+        if not os.path.isabs(path):
+            path = os.path.realpath(os.path.join(cmd_shenv.cwd, path))
+        if force and not os.path.exists(path):
+            continue
+        try:
+            if os.path.isdir(path):
+                if not recursive:
+                    stderr.write("Error: %s is a directory\n" % path)
+                    exitCode = 1
+                shutil.rmtree(path, onerror = on_rm_error if force else None)
+            else:
+                if force and not os.access(path, os.W_OK):
+                    os.chmod(path,
+                             stat.S_IMODE(os.stat(path).st_mode) | stat.S_IWRITE)
+                os.remove(path)
+        except OSError as err:
+            stderr.write("Error: 'rm' command failed, %s" % str(err))
+            exitCode = 1
+    return ShellCommandResult(cmd, "", stderr.getvalue(), exitCode, False)
+
 def processRedirects(cmd, stdin_source, cmd_shenv, opened_files):
     """Return the standard fds for cmd after applying redirects
 
     Returns the three standard file descriptors for the new child process.  Each
     fd may be an open, writable file object or a sentinel value from the
     subprocess module.
     """
 
def _executeShCmd(cmd, shenv, results, timeoutHelper):
     if cmd.commands[0].args[0] == 'export':
         if len(cmd.commands) != 1:
             raise ValueError("'export' cannot be part of a pipeline")
         if len(cmd.commands[0].args) != 2:
             raise ValueError("'export' supports only one argument")
         updateEnv(shenv, cmd.commands[0])
         return 0
 
+    if cmd.commands[0].args[0] == 'mkdir':
+        if len(cmd.commands) != 1:
+            raise InternalShellError(cmd.commands[0], "Unsupported: 'mkdir' "
+                                     "cannot be part of a pipeline")
+        cmdResult = executeBuiltinMkdir(cmd.commands[0], shenv)
+        results.append(cmdResult)
+        return cmdResult.exitCode
+
+    if cmd.commands[0].args[0] == 'diff':
+        if len(cmd.commands) != 1:
+            raise InternalShellError(cmd.commands[0], "Unsupported: 'diff' "
+                                     "cannot be part of a pipeline")
+        cmdResult = executeBuiltinDiff(cmd.commands[0], shenv)
+        results.append(cmdResult)
+        return cmdResult.exitCode
+
+    if cmd.commands[0].args[0] == 'rm':
+        if len(cmd.commands) != 1:
+            raise InternalShellError(cmd.commands[0], "Unsupported: 'rm' "
+                                     "cannot be part of a pipeline")
+        cmdResult = executeBuiltinRm(cmd.commands[0], shenv)
+        results.append(cmdResult)
+        return cmdResult.exitCode
+
     procs = []
     default_stdin = subprocess.PIPE
     stderrTempFiles = []
     opened_files = []
     named_temp_files = []
     # To avoid deadlock, we use a single stderr stream for piped
     # output. This is null until we have seen some output using
     # stderr.

utils/lit/tests/Inputs/shtest-shell/check_path.py

+#!/usr/bin/env python
+
+import os
+import sys
+
+
+def check_path(argv):
+    if len(argv) < 3:
+        print("Wrong number of args")
+        return 1
+
+    type = argv[1]
+    paths = argv[2:]
+    exit_code = 0
+
+    if type == 'dir':
+        for idx, dir in enumerate(paths):
+            print(os.path.isdir(dir))
+    elif type == 'file':
+        for idx, file in enumerate(paths):
+            print(os.path.isfile(file))
+    else:
+        print("Unrecognised type {}".format(type))
+        exit_code = 1
+    return exit_code
+
+if __name__ == '__main__':
+    sys.exit (check_path (sys.argv))

utils/lit/tests/Inputs/shtest-shell/diff-error-0.txt

+# Check error on a unsupported diff (cannot be part of a pipeline).
+#
+# RUN: diff diff-error-0.txt diff-error-0.txt | echo Output

utils/lit/tests/Inputs/shtest-shell/diff-error-1.txt

+# Check error on a unsupported diff (not support the -B option).
+#
+# RUN: diff -B temp1.txt temp2.txt

utils/lit/tests/Inputs/shtest-shell/diff-error-2.txt

+# Check error on an internal shell error (missing tofile)
+#
+# RUN: diff temp.txt

utils/lit/tests/Inputs/shtest-shell/diff-error-3.txt

+# Check error on an internal shell error (unable to find compared files)
+#
+# RUN: diff temp.txt temp1.txt

utils/lit/tests/Inputs/shtest-shell/diff-error-4.txt

+# Check exit code error (file's contents are different)
+#
+# RUN: echo "hello-first" > %t
+# RUN: echo "hello-second" > %t1
+# RUN: diff %t %t1

utils/lit/tests/Inputs/shtest-shell/diff-error-5.txt

+# Check error on an internal shell error (missing operand)
+#
+# RUN: diff -u

utils/lit/tests/Inputs/shtest-shell/diff-error-6.txt

+# Check error on an internal shell error (extra operand)
+#
+# RUN: diff -u a.txt b.txt c.txt

utils/lit/tests/Inputs/shtest-shell/mkdir-error-0.txt

+# Check error on a unsupported mkdir (cannot be part of a pipeline).
+#
+# RUN: mkdir -p temp | rm -rf temp

utils/lit/tests/Inputs/shtest-shell/mkdir-error-1.txt

+# Check error on a unsupported mkdir (only does not support -m option).
+#
+# RUN: mkdir -p -m 777 temp

utils/lit/tests/Inputs/shtest-shell/mkdir-error-2.txt

+# Check error on a unsupported mkdir (missing operand).
+#
+# RUN: mkdir -p

utils/lit/tests/Inputs/shtest-shell/rm-error-0.txt

+# Check error on a unsupported rm. (cannot be part of a pipeline)
+#
+# RUN: rm -rf temp | echo "hello"

utils/lit/tests/Inputs/shtest-shell/rm-error-1.txt

+# Check error on a unsupported rm (only does not support -v option).
+#
+# RUN: rm -f -v temp

utils/lit/tests/Inputs/shtest-shell/rm-error-2.txt

+# Check error on a unsupported rm (only supports -f option and in combination with -r).
+#
+#### RUN: rm -r hello
+# RUN: rm hello

utils/lit/tests/Inputs/shtest-shell/rm-error-3.txt

+# Check error on a unsupported rm (can't remove test since it is a directory).
+#
+# RUN: mkdir -p test
+# RUN: rm -f test

utils/lit/tests/Inputs/shtest-shell/valid-shell.txt

+# Check rm file operations.
+# Check force remove commands success whether the file does or doesn't exist.
+#
+# RUN: rm -f %t.write
+# RUN: "%{python}" %S/check_path.py file %t.write > %t.out
+# RUN: FileCheck --check-prefix=REMOVE-FILE < %t.out %s
+# RUN: echo "create a temp file" > %t.write
+# RUN: "%{python}" %S/check_path.py file %t.write > %t.out
+# RUN: FileCheck --check-prefix=FILE-EXIST < %t.out %s
+# RUN: rm -f %t.write
+# RUN: "%{python}" %S/check_path.py file %t.write > %t.out
+# RUN: FileCheck --check-prefix=REMOVE-FILE < %t.out %s
+#
+# REMOVE-FILE: False
+# FILE-EXIST: True
+#
+# Check mkdir and rm folder operations.
+# Check force remove commands success whether the directory does or doesn't exist.
+#
+# Check the mkdir command with -p option.
+# RUN: rm -f -r %T/test
+# RUN: "%{python}" %S/check_path.py dir %T/test > %t.out
+# RUN: FileCheck --check-prefix=REMOVE-PARENT-DIR < %t.out %s
+# RUN: mkdir -p %T/test
+# RUN: "%{python}" %S/check_path.py dir %T/test > %t.out
+# RUN: FileCheck --check-prefix=MAKE-PARENT-DIR < %t.out %s
+# RUN: rm -f %T/test || true
+# RUN: rm -f -r %T/test
+# RUN: "%{python}" %S/check_path.py dir %T/test > %t.out
+# RUN: FileCheck --check-prefix=REMOVE-PARENT-DIR < %t.out %s
+#
+# MAKE-PARENT-DIR: True
+# REMOVE-PARENT-DIR: False
+#
+# Check the mkdir command without -p option.
+#
+# RUN: rm -rf %T/test1
+# RUN: mkdir %T/test1
+# RUN: "%{python}" %S/check_path.py dir %T/test1 > %t.out
+# RUN: FileCheck --check-prefix=MAKE-DIR < %t.out %s
+# RUN: cd %T/test1 && mkdir foo
+# RUN: "%{python}" %S/check_path.py dir %T/test1 > %t.out
+# RUN: FileCheck --check-prefix=MAKE-DIR < %t.out %s
+# RUN: cd %T && rm -rf %T/test1
+# RUN: "%{python}" %S/check_path.py dir %T/test1 > %t.out
+# RUN: FileCheck --check-prefix=REMOVE-DIR < %t.out %s
+#
+# MAKE-DIR: True
+# REMOVE-DIR: False
+#
+# Check creating and removing multiple folders and rm * operation.
+#
+# RUN: rm -rf %T/test
+# RUN: mkdir -p %T/test/test1 %T/test/test2
+# RUN: "%{python}" %S/check_path.py dir %T/test %T/test/test1 %T/test/test2 > %t.out
+# RUN: FileCheck --check-prefix=DIRS-EXIST < %t.out %s
+# RUN: mkdir %T/test || true
+# RUN: echo "create a temp file" > %T/test/temp.write
+# RUN: echo "create a temp1 file" > %T/test/test1/temp1.write
+# RUN: echo "create a temp2 file" > %T/test/test2/temp2.write
+# RUN: "%{python}" %S/check_path.py file %T/test/temp.write %T/test/test1/temp1.write %T/test/test2/temp2.write> %t.out
+# RUN: FileCheck --check-prefix=FILES-EXIST < %t.out %s
+# RUN: rm -r -f %T/*
+# RUN: "%{python}" %S/check_path.py dir %T/test > %t.out
+# RUN: FileCheck --check-prefix=REMOVE-ALL < %t.out %s
+#
+# DIRS-EXIST: True
+# DIRS-EXIST-NEXT: True
+# DIRS-EXIST-NEXT: True
+# FILES-EXIST: True
+# FILES-EXIST-NEXT: True
+# FILES-EXIST-NEXT: True
+# REMOVE-ALL: False
+#
+# Check diff operations.
+#
+# RUN: echo "hello" > %t.stdout
+# RUN: echo "hello" > %t1.stdout
+# RUN: diff %t.stdout %t1.stdout
+# RUN: diff -u %t.stdout %t1.stdout
+# RUN: echo "hello-2" > %t1.stdout
+# RUN: diff %t.stdout %t1.stdout || true
+#
+# RUN: mkdir -p %T/dir1 %T/dir2
+# RUN: cd %T/dir1 && echo "hello" > temp1.txt
+# RUN: cd %T/dir2 && echo "hello" > temp2.txt
+# RUN: diff temp2.txt ../dir1/temp1.txt

utils/lit/tests/max-failures.py

 # Check the behavior of --max-failures option.
 #
 # RUN: not %{lit} -j 1 -v %{inputs}/max-failures > %t.out
 # RUN: not %{lit} --max-failures=1 -j 1 -v %{inputs}/max-failures >> %t.out
 # RUN: not %{lit} --max-failures=2 -j 1 -v %{inputs}/max-failures >> %t.out
 # RUN: not %{lit} --max-failures=0 -j 1 -v %{inputs}/max-failures 2>> %t.out
 # RUN: FileCheck < %t.out %s
 #
 # END.
 
-# CHECK: Failing Tests (3)
+# CHECK: Failing Tests (17)
 # CHECK: Failing Tests (1)
 # CHECK: Failing Tests (2)
 # CHECK: error: Setting --max-failures to 0 does not have any effect.

utils/lit/tests/shtest-shell.py

 # Check the internal shell handling component of the ShTest format.
 #
 # RUN: not %{lit} -j 1 -v %{inputs}/shtest-shell > %t.out
 # FIXME: Temporarily dump test output so we can debug failing tests on
 # buildbots.
 # RUN: cat %t.out
 # RUN: FileCheck --input-file %t.out %s
 #
 # END.
 
 # CHECK: -- Testing:
 
+
+# CHECK: FAIL: shtest-shell :: diff-error-0.txt
+# CHECK: *** TEST 'shtest-shell :: diff-error-0.txt' FAILED ***
+# CHECK: $ "diff" "diff-error-0.txt" "diff-error-0.txt"
+# CHECK: # command stderr:
+# CHECK: Unsupported: 'diff' cannot be part of a pipeline
+# CHECK: error: command failed with exit status: 127
+# CHECK: ***
+
+# CHECK: FAIL: shtest-shell :: diff-error-1.txt
+# CHECK: *** TEST 'shtest-shell :: diff-error-1.txt' FAILED ***
+# CHECK: $ "diff" "-B" "temp1.txt" "temp2.txt"
+# CHECK: # command stderr:
+# CHECK: Unsupported: 'diff': option -B not recognized
+# CHECK: error: command failed with exit status: 127
+# CHECK: ***
+
+# CHECK: FAIL: shtest-shell :: diff-error-2.txt
+# CHECK: *** TEST 'shtest-shell :: diff-error-2.txt' FAILED ***
+# CHECK: $ "diff" "temp.txt"
+# CHECK: # command stderr:
+# CHECK: Error:  missing or extra operand
+# CHECK: error: command failed with exit status: 127
+# CHECK: ***
+
+# CHECK: FAIL: shtest-shell :: diff-error-3.txt
+# CHECK: *** TEST 'shtest-shell :: diff-error-3.txt' FAILED ***
+# CHECK: $ "diff" "temp.txt" "temp1.txt"
+# CHECK: # command stderr:
+# CHECK: Error: 'diff' command failed
+# CHECK: error: command failed with exit status: 1
+# CHECK: ***
+
+# CHECK: FAIL: shtest-shell :: diff-error-4.txt
+# CHECK: *** TEST 'shtest-shell :: diff-error-4.txt' FAILED ***
+# CHECK: Exit Code: 1
+# CHECK: # command output:
+# CHECK: diff-error-4.txt.tmp
+# CHECK: diff-error-4.txt.tmp1
+# CHECK: *** 1 ****
+# CHECK: ! hello-first
+# CHECK: --- 1 ----
+# CHECK: ! hello-second
+# CHECK: ***
+
+# CHECK: FAIL: shtest-shell :: diff-error-5.txt
+# CHECK: *** TEST 'shtest-shell :: diff-error-5.txt' FAILED ***
+# CHECK: $ "diff"
+# CHECK: # command stderr:
+# CHECK: Error:  missing or extra operand
+# CHECK: error: command failed with exit status: 127
+# CHECK: ***
+
+# CHECK: FAIL: shtest-shell :: diff-error-6.txt
+# CHECK: *** TEST 'shtest-shell :: diff-error-6.txt' FAILED ***
+# CHECK: $ "diff"
+# CHECK: # command stderr:
+# CHECK: Error:  missing or extra operand
+# CHECK: error: command failed with exit status: 127
+# CHECK: ***
+
 # CHECK: FAIL: shtest-shell :: error-0.txt
 # CHECK: *** TEST 'shtest-shell :: error-0.txt' FAILED ***
 # CHECK: $ "not-a-real-command"
 # CHECK: # command stderr:
 # CHECK: 'not-a-real-command': command not found
 # CHECK: error: command failed with exit status: 127
 # CHECK: ***
 
 # FIXME: The output here sucks.
 #
 # CHECK: FAIL: shtest-shell :: error-1.txt
 # CHECK: *** TEST 'shtest-shell :: error-1.txt' FAILED ***
 # CHECK: shell parser error on: 'echo "missing quote'
 # CHECK: ***
 
 # CHECK: FAIL: shtest-shell :: error-2.txt
 # CHECK: *** TEST 'shtest-shell :: error-2.txt' FAILED ***
 # CHECK: Unsupported redirect:
 # CHECK: ***
 
+# CHECK: FAIL: shtest-shell :: mkdir-error-0.txt
+# CHECK: *** TEST 'shtest-shell :: mkdir-error-0.txt' FAILED ***
+# CHECK: $ "mkdir" "-p" "temp"
+# CHECK: # command stderr:
+# CHECK: Unsupported: 'mkdir' cannot be part of a pipeline
+# CHECK: error: command failed with exit status: 127
+# CHECK: ***
+
+# CHECK: FAIL: shtest-shell :: mkdir-error-1.txt
+# CHECK: *** TEST 'shtest-shell :: mkdir-error-1.txt' FAILED ***
+# CHECK: $ "mkdir" "-p" "-m" "777" "temp"
+# CHECK: # command stderr:
+# CHECK: Unsupported: 'mkdir': option -m not recognized
+# CHECK: error: command failed with exit status: 127
+# CHECK: ***
+
+# CHECK: FAIL: shtest-shell :: mkdir-error-2.txt
+# CHECK: *** TEST 'shtest-shell :: mkdir-error-2.txt' FAILED ***
+# CHECK: $ "mkdir" "-p"
+# CHECK: # command stderr:
+# CHECK: Error: 'mkdir' is missing an operand
+# CHECK: error: command failed with exit status: 127
+# CHECK: ***
+
 # CHECK: PASS: shtest-shell :: redirects.txt
+
+# CHECK: FAIL: shtest-shell :: rm-error-0.txt
+# CHECK: *** TEST 'shtest-shell :: rm-error-0.txt' FAILED ***
+# CHECK: $ "rm" "-rf" "temp"
+# CHECK: # command stderr:
+# CHECK: Unsupported: 'rm' cannot be part of a pipeline
+# CHECK: error: command failed with exit status: 127
+# CHECK: ***
+
+# CHECK: FAIL: shtest-shell :: rm-error-1.txt
+# CHECK: *** TEST 'shtest-shell :: rm-error-1.txt' FAILED ***
+# CHECK: $ "rm" "-f" "-v" "temp"
+# CHECK: # command stderr:
+# CHECK: Unsupported: 'rm': option -v not recognized
+# CHECK: error: command failed with exit status: 127
+# CHECK: ***
+
+# CHECK: FAIL: shtest-shell :: rm-error-2.txt
+# CHECK: *** TEST 'shtest-shell :: rm-error-2.txt' FAILED ***
+# CHECK: $ "rm" "-r" "hello"
+# CHECK: # command stderr:
+# CHECK: Error: 'rm' command failed
+# CHECK: error: command failed with exit status: 1
+# CHECK: ***
+
+# CHECK: FAIL: shtest-shell :: rm-error-3.txt
+# CHECK: *** TEST 'shtest-shell :: rm-error-3.txt' FAILED ***
+# CHECK: Exit Code: 1
+# CHECK: ***
+
 # CHECK: PASS: shtest-shell :: sequencing-0.txt
 # CHECK: XFAIL: shtest-shell :: sequencing-1.txt
-# CHECK: Failing Tests (3)
+# CHECK: PASS: shtest-shell :: valid-shell.txt
+# CHECK: Failing Tests (17)