Add info on running remote test suite to the website

[zturner]

I know this is more work, but would it be possible to split all the testing
stuff out into a totally separate page on the sidebar?

[labath]

That sounds like a good idea. I have created a separate test page. Please take another look.

[tberghammer]

Looks good with 2 minor comment

www/test.html

72–73 ↗

(On Diff #23406)

It can be also specified by '-t' or by '--threads'

86–88 ↗

(On Diff #23406)

Probably add a note that running the test suit against a remote target with multiple threads isn't fully supported yet.

[labath]

Address review comments.

[tberghammer]

LGTM

lldb/trunk/www/build.html

<div id="content">
               the second allows Python to locate the LLDB extension module.
             </p>
             <p>
               Steps 6 and 7 of <a href="#WindowsPreliminaries">Preliminaries</a> describe a method for simplifying
               this setup.
             </p>
             <p>Finally, when you are ready to build LLDB, generate CMake with the following command line:</p>
             <code>cmake -G Ninja &lt;cmake variables&gt; &lt;path to root of llvm src tree&gt;</code>
-            <p>and run <code>ninja</code> to build LLDB, and <code>ninja check-lldb</code> to run LLDB's test suite.</p>
+            <p>and run <code>ninja</code> to build LLDB. Information about running the LLDB test suite can be found on the <a href="test.html">test</a> page.</p>
             <p>
               Following is a description of some of the most important CMake variables which you are likely to encounter.
               A variable <code>FOO</code> is set by adding <code>-DFOO=value</code> to the CMake command line.
             </p>
             <ul>
               <li>
                 <b>LLDB_TEST_DEBUG_TEST_CRASHES</b> (Default=0): If set to 1, will cause Windows to generate a crash
                 dialog whenever lldb.exe or the python extension module crashes while running the test suite.  If set to
                 0, LLDB will silently crash.  Setting to 1 allows a developer to attach a JIT debugger at the time of
                 a crash, rather than having to reproduce a failure or use a crash dump.
               </li>
               <li>
                 <b>PYTHON_LIBRARY</b>: Path to python27.lib.  If doing a debug build, note that the library is called
                 python27_d.lib.  Generally this should be set to <div align="center">&lt;python src dir&gt;\PCBuild\python27(_d).lib</div>
               </li>
               <li>
                 <b>PYTHON_INCLUDE_DIR</b>: Path to python include directory.  Generally this should be set to
                 <div align="center">&lt;python src dir&gt;\Include</div>
               </li>
               <li>
                 <b>PYTHON_EXECUTABLE</b>: Path to python.exe.  If doing a debug build of LLDB, note that the executable
                 is called python_d.exe.  Generally this should be set to <div align="center">&lt;python src dir&gt;\PCBuild\python(_d).exe</div>
               </li>
-              <li>
-                <b>LLDB_TEST_COMPILER</b>: The test suite only supports testing executables that were compiled with clang.  This specifies
-                the path to the copy of clang that you wish to use to compile test executables.  To use the version
-                of clang that you compiled alongside LLDB, set this to <div align="center">&lt;folder where CMake build files are generated&gt;\bin\clang.exe</div>
-              </li>
             </ul>
           </div>
         </div>
         <div class="post" id="BuildingLldbOnMacOSX">
           <h1 class="postheader">Building LLDB on Mac OS X</h1>
           <div class="postcontent">
             <p>Building on Mac OS X is as easy as downloading the code and building the Xcode project or workspace:</p>
           </div>
<div id="content">
               necessary to perform a top-level <tt>make</tt> to rebuild changes made only to LLDB.
               You can run <tt>make</tt> from the <tt>build/tools/lldb</tt> subdirectory as well.
             </p>
             <p>
               If you wish to build with libc++ instead of libstdc++ (the default), run configure with the
               <tt>--enable-libcpp</tt> flag.
             </p>
             <p> If you wish to build a release version of LLDB, run configure with the <tt>--enable-optimized</tt> flag.</p>
-            <h2>Testing</h2>
-            <p>
-              By default, the <tt>check-lldb</tt> target builds the 64-bit variants of the test programs with the same
-              compiler that was used to build LLDB. It is possible to customize the architecture and compiler by appending -A and
-              -C options respectively to the CMake variable <tt>LLDB_TEST_ARGS</tt>. For example, to test LLDB against 32-bit binaries
-              built with a custom version of clang, do:
-            </p>
-            <code>
-              <br />&gt; cmake -DLLDB_TEST_ARGS="-A i386 -C /path/to/custom/clang" -G Ninja
-              <br />&gt; ninja check-lldb
-            </code>
-            <p>Note that multiple -A and -C flags can be specified to <tt>LLDB_TEST_ARGS</tt>.</p>
-            <p>
-              In addition to running all the LLDB test suites with the "check-lldb" CMake target above, it is possible to
-              run individual LLDB tests. For example, to run the test cases defined in TestInferiorCrashing.py, run:
-            </p>
-            <code>
-              <br />&gt; cd $lldb/test
-              <br />&gt; python dotest.py --executable &lt;path-to-lldb&gt;  -p TestInferiorCrashing.py
-            </code>
-            <p>
-              In addition to running a test by name, it is also possible to specify a directory path to <tt>dotest.py</tt>
-              in order to run all the tests under that directory. For example, to run all the tests under the
-              'functionalities/data-formatter' directory, run:
-            </p>
-            <code>
-              <br />&gt; python dotest.py --executable &lt;path-to-lldb&gt; functionalities/data-formatter
-            </code>
-            <p>
-              To dump additional information to <tt>stdout</tt> about how the test harness is driving LLDB, run
-              <tt>dotest.py</tt> with the <tt>-t</tt> flag. Many more options that are available. To see a list of all of them, run:
-            </p>
-            <code>
-              <br />&gt; python dotest.py -h
-            </code>
+
             <h2>Building API reference documentation</h2>
             <p>
               LLDB exposes a C++ as well as a Python API. To build the reference documentation for these two APIs, ensure you have
               the required dependencies installed, and build the <tt>lldb-python-doc</tt> and <tt>lldb-cpp-doc</tt> CMake targets.
             </p>
             <p> The output HTML reference documentation can be found in <tt>&lt;build-dir&gt;/tools/lldb/docs/</tt>.</p><p>
               <h2>Additional Notes</h2>
             </p>

lldb/trunk/www/sidebar.incl

<div class="menu">
     <div class="urbangreymenu">
     <h1 class="headerbar">Resources</h1>
     <ul>
       <li><a href="/download.html">Download</a></li>
       <li><a href="/python_reference/index.html">Python API Documentation</a></li>
       <li><a href="/cpp_reference/html/index.html">C++ API Documentation</a></li>
       <li><a href="/source.html">Source</a></li>
       <li><a href="/build.html">Build</a></li>
+      <li><a href="/test.html">Test</a></li>
       <li><a href="/lldb-coding-conventions.html">Coding Conventions</a></li>      
       <li><a href="/SB-api-coding-rules.html">SB API Coding Rules</a></li>
       <li><a href="http://llvm.org/bugs">Bug Reports</a></li>
       <li><a href="http://llvm.org/svn/llvm-project/lldb/trunk">Browse SVN</a></li>
       <li><a href="http://llvm.org/viewvc/llvm-project/lldb/trunk">Browse ViewVC</a></li>
     </ul>
     </div>
   </div>
 </div>

lldb/trunk/www/test.html

+<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+<html xmlns="http://www.w3.org/1999/xhtml">
+<head>
+  <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
+  <link href="style.css" rel="stylesheet" type="text/css" />
+  <title>Testing LLDB</title>
+</head>
+<body>
+  <div class="www_title">
+    The <strong>LLDB</strong> Debugger
+  </div>
+
+  <div id="container">
+    <div id="content">
+
+      <!--#include virtual="sidebar.incl"-->
+
+        <div class="post">
+          <h1 class="postheader">Testing LLDB</h1>
+          <div class="postcontent">
+            <p>
+              The LLDB test suite consists of Python scripts located under the
+              <tt>test</tt> directory. Each script contains a number of test cases and is usually
+              accompanied by a C (C++, ObjC, etc.) source file. Each test first compiles the
+              source file and then uses LLDB to debug the resulting executable. The tests verify
+              both the LLDB command line interface and the scripting API.
+            </p>
+
+            <p>
+              The easiest way to run the LLDB test suite is to use the <tt>check-lldb</tt> build
+              target. By default, the <tt>check-lldb</tt> target builds the test programs with
+              the same compiler that was used to build LLDB. To build the tests with a different
+              compiler, you can set the <tt>LLDB_TEST_COMPILER</tt> CMake variable. It is possible to
+              customize the architecture of the test binaries and compiler used by appending -A
+              and -C options respectively to the CMake variable <tt>LLDB_TEST_USER_ARGS</tt>. For
+              example, to test LLDB against 32-bit binaries
+              built with a custom version of clang, do:
+            </p>
+            <code>
+              <br />&gt; cmake -DLLDB_TEST_ARGS="-A i386 -C /path/to/custom/clang" -G Ninja
+              <br />&gt; ninja check-lldb
+            </code>
+            <p>Note that multiple -A and -C flags can be specified to <tt>LLDB_TEST_USER_ARGS</tt>.</p>
+            <p>
+              In addition to running all the LLDB test suites with the "check-lldb" CMake target above, it is possible to
+              run individual LLDB tests. For example, to run the test cases defined in TestInferiorCrashing.py, run:
+            </p>
+            <code>
+              <br />&gt; cd $lldb/test
+              <br />&gt; python dotest.py --executable &lt;path-to-lldb&gt;  -p TestInferiorCrashing.py
+            </code>
+            <p>
+              In addition to running a test by name, it is also possible to specify a directory path to <tt>dotest.py</tt>
+              in order to run all the tests under that directory. For example, to run all the tests under the
+              'functionalities/data-formatter' directory, run:
+            </p>
+            <code>
+              <br />&gt; python dotest.py --executable &lt;path-to-lldb&gt; functionalities/data-formatter
+            </code>
+            <p>
+              To dump additional information to <tt>stdout</tt> about how the test harness is driving LLDB, run
+              <tt>dotest.py</tt> with the <tt>-t</tt> flag. Many more options that are available. To see a list of all of them, run:
+            </p>
+            <code>
+              <br />&gt; python dotest.py -h
+            </code>
+
+            <p>
+              Besides <code>dotest.py</code>, there is also <code>dosep.py</code>, which runs
+              multiple instances of <code>dotest.py</code> in parallel, thereby greatly
+              decreasing the time it takes to run the full testsuite. The number of concurrent
+              tests is controlled by the <code>LLDB_TEST_THREADS</code> environment variable or
+              the <code>--threads</code> command line parameter. The default value is the number
+              of CPUs on your system. To pass additional options to <code>dotest.py</code>,
+              specify those options as an <code>-o</code> argument to <code>dosep.py</code>. For
+              example, the command
+            </p>
+            <code>python dosep.py -o "--executable bin/lldb -C bin/clang"</code>
+            <p>
+              will specify the lldb and clang executables to test for each dotest invocation.
+              <code>ninja check-lldb</code> is wrapper around <code>dosep.py</code>.
+            </p>
+
+            <h3>Running the test-suite remotely</h3>
+
+            <p>
+              Running the test-suite remotely is similar to the process of running a local test
+              suite, but there are two things to have in mind:
+            </p>
+            <ul>
+              <li>
+                You must have the <code>lldb-server</code> running on the remote system, ready to
+                accept multiple connections. For more information on how to setup remote
+                debugging see the <a href="remote.html">Remote debugging</a> page.
+              </li>
+              <li>
+                You must tell the test-suite how to connect to the remote system. This is
+                achieved using the <code>--platform-name</code>, <code>--platform-url</code> and
+                <code>--platform-working-dir</code> parameters to <code>dotest.py</code>. These
+                parameters correspond to the <code>platform select</code> and <code>platform
+                connect</code> LLDB commands. You will usually also need to specify the compiler and
+                architecture for the remote system.
+              </li>
+            </ul>
+            <p>
+              Currently, running the remote test suite is supported only with
+              <code>dotest.py</code> (or <code>dosep.py</code> with a single thread), but we
+              expect this issue to be adressed in the near future.
+            </p>
+
+          </div>
+          <div class="postfooter"></div>
+        </div>
+    </div>
+  </div>
+</body>
+</html>