Update website for building and testing on Windows

[zturner]

Summary

Lots of new info about how to build / test on Windows.

1. Update for VS 2015 and Python 3.5
2. Remove old info about building with Python 2.7
3. Reformat the testing page to make it easier to read.
4. Add info about installing and using PTVS on Windows.

[zturner]

Update to include full context.

[amccarth]

Could you post the patch with full context? It's easier to see the changes in context.

The information looks right. It's nice to see all the build-Python-yourself stuff simplified away.

www/build.html

63

typo: "on your own" not "own your own"

67

Wasn't there a particular version (like 3.0.5) that had a specific bug? I seem to recall you needed something older or something very recent.

www/test.html

67

delete "that"

139

Did you mean "managed" here? Would it be clearer to say you can step between the test script and the native LLDB code?

"Seamlessly" is a stretch. ;-)

160

It might be useful to give examples here. For one thing, it might not be obvious at this point that the Path and Windows Path fields should point do the _d version of the Python executable.

[Honsik]

Hello, newcomer here.

Thanks zturner for you support.
Shouldn't best workflow for VS also be mentioned? Like one solution for code editing generated by CMake -G "Visual Studio 14" that lacks directory structure (mostly CMake + VS issue) and another with custom project containg test case in llvm/tools/lldb/packages directory for creating test cases and running them with setting zturner has posted?

I am currently using different approach, editing code in Eclipse (it follows directory structure) and using VS only for debugging.

Or documentation shouldn't contain these recommendations?

www/test.html

62

Hi, I have currently problem with running python dotest.py here.
When I have Debug build, it just fails to find _lldb module to import:

Traceback (most recent call last):
  File "d:\workspace\build\ninja\lib\site-packages\lldb\__init__.py", line 43, in swig_import_helper
    fp, pathname, description = imp.find_module('_lldb', [dirname(__file__)])
  File "d:\workspace\python\lib\imp.py", line 296, in find_module
    raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named '_lldb'

So i had to use python_d and then its working correctly

[zturner]

In D17521#359245, @Honsik wrote:

Hello, newcomer here.

Thanks zturner for you support.
Shouldn't best workflow for VS also be mentioned? Like one solution for code editing generated by CMake -G "Visual Studio 14" that lacks directory structure (mostly CMake + VS issue) and another with custom project containg test case in llvm/tools/lldb/packages directory for creating test cases and running them with setting zturner has posted?

I am currently using different approach, editing code in Eclipse (it follows directory structure) and using VS only for debugging.

Or documentation shouldn't contain these recommendations?

Yea I can probably mention that.

www/test.html

62

Good point, it has to be python_d on windows. I'll fix this note.

[zturner]

Updated with more info about using VS and ninja together, and a note about python_d on Windows.

[Eugene.Zelenko]

Committed in r261795.

www/build.html

<div id="content">
           </ul>
         </div>
         <div class="postfooter"></div>
         <div class="post" id="BuildingLldbOnWindows">
           <h1 class="postheader">Building LLDB on Windows</h1>
           <div class="postcontent">
             <h2>Required Dependencies</h2>
             <ul>
-              <li>Visual Studio 2012 or greater</li>
-              <li>Windows SDK 8.0 or higher</li>
+              <li>Visual Studio 2015 or greater</li>
+              <li>Windows SDK 8.0 or higher.  In general it is best to use the latest available version.</li>
               <li>
-                <a href="https://www.python.org/download/releases/2.7/">Python 2.7</a>. Note that you <b>must</b>
-                compile Python from source.  See <a href="#WindowsPreliminaries">Preliminaries</a> for more
-                information.
+                <a href="https://www.python.org/downloads/windows/">Python 3.5 or higher</a> or higher.  Earlier
+                versions of Python can be made to work by compiling your own distribution from source,
+                but this workflow is unsupported and you are own your own.
               </li>
               <li><a href="http://martine.github.io/ninja/">Ninja build tool</a> (strongly recommended)</li>
               <li><a href="http://gnuwin32.sourceforge.net/">GnuWin32</a></li>
-              <li><a href="http://www.swig.org/download.html">SWIG for Windows</a></li>
+              <li><a href="http://www.swig.org/download.html">SWIG for Windows (version 3+)</a></li>
+            </ul>
+            <h2>Optional Dependencies</h2>
+            <ul>
+              <li><a href="https://github.com/Microsoft/PTVS/releases">Python Tools for Visual Studio</a>.  If you
+              plan to debug test failures or even write new tests at all, PTVS is an indispensable debugging extension
+              to VS that enables full editing and debugging support for Python (including mixed native/managed debugging)</li>
             </ul>
             <h2 id="WindowsPreliminaries">Preliminaries</h2>
             <p>
               This section describes how to set up your system and install the required dependencies such that
               they can be found when needed during the build process.  The steps outlined here only need to
               be performed once.
             </p>
             <ol>
               <li><p>Install Visual Studio and the Windows SDK.</p></li>
-              <li>
-                <p>
-                  Build Python from source using the solution file supplied with the Python 2.7 source
-                  distribution.
-                </p>
-                <p>
-                  Because LLDB functionality is compiled into a Python extension module,
-                  the extension module must be compiled with the same version of Visual Studio that
-                  Python itself was compiled with.  The binary release of Python 2.7 is compiled with
-                  Visual Studio 2008, so it is incompatible with linking against LLDB.
-                </p>
-                <p>
-                  Note that if you plan to do both debug and release builds of LLDB, you will need to
-                  compile both debug and release builds of Python.  The same applies if you plan to build
-                  both x86 and x64 configurations of LLDB
-                </p>
-              </li>
-              <li>
-                <p>Copy &lt;python src dir&gt;\PC\pyconfig.h to &lt;python src dir&gt;\Include.</p>
-                <p>
-                  This is necessary because pyconfig.h is a hand-maintained file which is platform specific,
-                  so multiple copies of this file are included with each source distribution.  It appears to
-                  be up to the person building Python to move the correct version of pyconfig.h to the Include
-                  folder.
-                </p>
-              </li>
-              <li>
-                <p>
-                  Run lldb/scripts/install_custom_python.py so to "install" your custom build of Python to a
-                  canonical directory structure.
-                </p>
-              </li>
-              <li><p>Install GnuWin32, making sure &lt;GnuWin32 install dir&gt;\bin is added to your PATH environment variable.</p></li>
-              <li><p>Install SWIG for Windows, making sure &lt;SWIG install dir&gt; is added to your PATH environment variable.</p></li>
+              <li><p>Install GnuWin32, making sure <code>&lt;GnuWin32 install dir&gt;\bin</code> is added to your <code>PATH</code> environment variable.</p></li>
+              <li><p>Install SWIG for Windows, making sure <code>&lt;SWIG install dir&gt;</code> is added to your <code>PATH</code> environment variable.</p></li>
             </ol>
             <h2>Building LLDB</h2>
             <p>
               Any command prompt from which you build LLDB should have a valid Visual Studio environment setup.
-              This means you should run vcvarsall.bat or open an appropriate Visual Studio Command Prompt
+              This means you should run <code>vcvarsall.bat</code> or open an appropriate Visual Studio Command Prompt
               corresponding to the version you wish to use.
             </p>
             <p>Finally, when you are ready to build LLDB, generate CMake with the following command line:</p>
             <code>cmake -G Ninja &lt;cmake variables&gt; &lt;path to root of llvm src tree&gt;</code>
             <p>and run <code>ninja</code> to build LLDB. Information about running the LLDB test suite can be found on the <a href="test.html">test</a> page.</p>
             <p>
               Following is a description of some of the most important CMake variables which you are likely to encounter.
               A variable <code>FOO</code> is set by adding <code>-DFOO=value</code> to the CMake command line.
             </p>
             <ul>
               <li>
                 <b>LLDB_TEST_DEBUG_TEST_CRASHES</b> (Default=0): If set to 1, will cause Windows to generate a crash
                 dialog whenever lldb.exe or the python extension module crashes while running the test suite.  If set to
                 0, LLDB will silently crash.  Setting to 1 allows a developer to attach a JIT debugger at the time of
                 a crash, rather than having to reproduce a failure or use a crash dump.
               </li>
               <li>
-                <b>PYTHON_HOME</b> (Required): Path the folder you specified in the --dest argument to install_custom_python.py.
-                Note that install_custom_python.py will create x86 and x64 subdirectories under this folder.  PYTHON_HOME should
-                refer to the correct architecture-specific folder.
+                <b>PYTHON_HOME</b> (Required): Path to the folder where the Python distribution is installed.  For example,
+                C:\Python35
               </li>
               <li>
                 <b>LLDB_RELOCATABLE_PYTHON</b> (Default=0): When this is 0, LLDB will bind statically to the location specified
                 in the PYTHON_HOME CMake variable, ignoring any value of PYTHONHOME set in the environment.  This is most useful for
                 developers who simply want to run LLDB after they build it.  If you wish to move a build of LLDB to a different
                 machine where Python will be in a different location, setting LLDB_RELOCATABLE_PYTHON to 1 will cause Python to
                 use its default mechanism for finding the python installation at runtime (looking for installed Pythons, or using
                 the PYTHONHOME environment variable if it is specified).
               </li>
-            </ul>
+              <li>
+                <b>LLDB_TEST_COMPILER</b>: The test suite needs to be able to find a copy of clang.exe that it can use to compile
+                inferior programs.  Note that MSVC is not supported here, it <strong>must</strong> be a path to a clang executable.
+                Note that using a release clang.exe is strongly recommended here, as it will make the test suite run much faster.
+                This can be a path to any recent clang.exe, including one you built yourself.
+              </li>
+            </ul>
+            Sample command line:<br/>
+            <code>cmake -G Ninja -DLLDB_TEST_DEBUG_TEST_CRASHES=1 -DPYTHON_HOME=C:\Python35 -DLLDB_TEST_COMPILER=d:\src\llvmbuild\ninja_release\bin\clang.exe ..\..\llvm</code>
+            <h2>Working with both Ninja and MSVC</h2>
+            <p>
+              Compiling with <code>ninja</code> is both faster and simpler than compiling with MSVC, but chances are you still want
+              to debug LLDB with MSVC (at least until we can debug LLDB on Windows with LLDB!).  One solution to this is to run 
+              <code>cmake</code> twice and generate the output into two different folders.  One for compiling (the <code>ninja</code>
+              folder), and one for editing / browsing / debugging (the MSVC folder).
+            </p>
+            <p>
+              To do this, simply run <code>`cmake -G Ninja &lt;arguments&gt;`</code> from one folder, and 
+              <code>`cmake -G "Visual Studio 14 2015" &lt;arguments&gt;`</code> in another folder.  Then you can open the .sln file
+              in Visual Studio, set <code>lldb</code> as the startup project, and use F5 to run it.  You need only edit the project
+              settings to set the executable and the working directory to point to binaries inside of the <code>ninja</code> tree.
+            </p>
           </div>
         </div>
         <div class="post" id="BuildingLldbOnMacOSX">
           <h1 class="postheader">Building LLDB on Mac OS X</h1>
           <div class="postcontent">
             <p>Building on Mac OS X is as easy as downloading the code and building the Xcode project or workspace:</p>
           </div>
           <div class="postcontent">

www/test.html

<div id="content">
           <div class="postcontent">
             <p>
               The LLDB test suite consists of Python scripts located under the
               <tt>test</tt> directory. Each script contains a number of test cases and is usually
               accompanied by a C (C++, ObjC, etc.) source file. Each test first compiles the
               source file and then uses LLDB to debug the resulting executable. The tests verify
               both the LLDB command line interface and the scripting API.
             </p>
-
+          </div>
+          <h1 class="postheader">Running tests</h1>
+          <div class="postcontent">
+            <h2>Running the full test suite</h2>
+            <p>
+              <strong>Windows Note</strong>: In the examples that follow, any invocations of <code>python</code>
+              should be replaced with <code>python_d</code>, the debug interpreter, when running the test
+              suite against a debug version of LLDB.
+            </p>
             <p>
               The easiest way to run the LLDB test suite is to use the <tt>check-lldb</tt> build
               target. By default, the <tt>check-lldb</tt> target builds the test programs with
               the same compiler that was used to build LLDB. To build the tests with a different
-              compiler, you can set the <tt>LLDB_TEST_COMPILER</tt> CMake variable. It is possible to
+              compiler, you can set the <strong>LLDB_TEST_COMPILER</strong> CMake variable. It is possible to
               customize the architecture of the test binaries and compiler used by appending -A
-              and -C options respectively to the CMake variable <tt>LLDB_TEST_USER_ARGS</tt>. For
+              and -C options respectively to the CMake variable <strong>LLDB_TEST_USER_ARGS</strong>. For
               example, to test LLDB against 32-bit binaries
               built with a custom version of clang, do:
             </p>
             <code>
               <br />&gt; cmake -DLLDB_TEST_ARGS="-A i386 -C /path/to/custom/clang" -G Ninja
               <br />&gt; ninja check-lldb
             </code>
             <p>Note that multiple -A and -C flags can be specified to <tt>LLDB_TEST_USER_ARGS</tt>.</p>
             <p>Note that on NetBSD you must export <tt>LD_LIBRARY_PATH=$PWD/lib</tt> in your environment. This is due to lack of
                the <tt>$ORIGIN</tt> linker feature.</p>
+            <h2>Running a specific test or set of tests</h2>
             <p>
               In addition to running all the LLDB test suites with the "check-lldb" CMake target above, it is possible to
               run individual LLDB tests. For example, to run the test cases defined in TestInferiorCrashing.py, run:
             </p>
             <code>
               <br />&gt; cd $lldb/test
-              <br />&gt; python dotest.py --executable &lt;path-to-lldb&gt;  -p TestInferiorCrashing.py
+              <br />&gt; python dotest.py --executable &lt;path-to-lldb&gt;  -p TestInferiorCrashing.py ../packages/Python/lldbsuite/test
             </code>
             <p>
-              In addition to running a test by name, it is also possible to specify a directory path to <tt>dotest.py</tt>
-              in order to run all the tests under that directory. For example, to run all the tests under the
-              'functionalities/data-formatter' directory, run:
+              If the test is not specified by name (e.g. if you leave the <code>-p</code> argument off), LLDB will run all tests in
+              that directory:
             </p>
             <code>
               <br />&gt; python dotest.py --executable &lt;path-to-lldb&gt; functionalities/data-formatter
             </code>
             <p>
-              To dump additional information to <tt>stdout</tt> about how the test harness is driving LLDB, run
-              <tt>dotest.py</tt> with the <tt>-t</tt> flag. Many more options that are available. To see a list of all of them, run:
+              Many more options that are available. To see a list of all of them, run:
             </p>
             <code>
-              <br />&gt; python dotest.py -h
+              &gt; python dotest.py -h
             </code>
 
             <p>
               The dotest.py script runs tests in parallel by default.
               To disable the parallel test running feature, use the
               <code>--no-multiprocess</code> flag. The number of
               concurrent tests is controlled by
               the <code>LLDB_TEST_THREADS</code> environment variable
<div id="content">
               given directory serially, but will run multiple
               directories concurrently.  Thus, as a test writer, we
               provide serialized test run semantics within a
               directory.  Note child directories are considered
               entirely separate, so two child directories could be
               running in parallel with a parent directory.
             </p>
 
-            <h3>Running the test-suite remotely</h3>
+            <h2>Running the test-suite remotely</h2>
 
             <p>
               Running the test-suite remotely is similar to the process of running a local test
               suite, but there are two things to have in mind:
             </p>
             <ul>
               <li>
                 You must have the <code>lldb-server</code> running on the remote system, ready to
<div id="content">
             <p>
               Currently, running the remote test suite is supported only with
               <code>dotest.py</code> (or <code>dosep.py</code> with a single thread), but we
               expect this issue to be addressed in the near future.
             </p>
 
           </div>
           <div class="postfooter"></div>
+          <h1 class="postheader">Debugging test failures</h1>
+          <div class="postcontent">
+            <h2>Non-Windows platforms</h2>
+            <p>
+              On non-Windows platforms, you can use the <code>-d</code> option to <code>dotest.py</code> which will cause the script to wait
+              for a while until a debugger is attached.
+            </p>
+            <h2>Windows</h2>
+            <p>
+              On Windows, it is strongly recommended to use <a href="https://github.com/Microsoft/PTVS/releases">Python Tools for Visual Studio</a>
+              for debugging test failures.  It can seamlessly step between native and managed code, which is very helpful when you need to step
+              through the test itself, and then into the LLDB code that backs the operations the test is performing.  A quick guide to getting
+              started with PTVS is as follows:
+              <ul>
+                <li>Install PTVS</li>
+                <li>
+                  Create a Visual Studio Project for the Python code.
+                  <ul>
+                    <li>Go to File -> New -> Project -> Python -> From Existing Python Code.</li>
+                    <li>Choose <code>llvm/tools/lldb</code> as the directory containing the Python code.</li>
+                    <li>
+                      When asked where to save the <code>.pyproj</code> file, choose the folder <code>llvm/tools/lldb/pyproj</code>.
+                      This is a special folder that is ignored by the <code>.gitignore</code> file, since it is not checked in.
+                    </li>
+                  </ul>
+                </li>
+                <li>Set <code>test/dotest.py</code> as the startup file</li>
+                <li>
+                  Make sure there is a Python Environment installed for your distribution.  For example, if you installed Python to
+                  <code>C:\Python35</code>, PTVS needs to know that this is the interpreter you want to use for running the test suite.
+                  <ul>
+                    <li>Go to Tools -> Options -> Python Tools -> Environment Options</li>
+                    <li>Click Add Environment, and enter <code>Python 3.5 Debug</code> for the name.  Fill out the values correctly.</li>
+                  </ul>
+                </li>
+                <li>
+                  Configure the project to use this debug interpreter.
+                  <ul>
+                    <li>Right click the Project node in Solution Explorer</li>
+                    <li>In the <code>General</code> tab, Make sure <code>Python 3.5 Debug</code> is the selected Interpreter.</li>
+                    <li>In <code>Debug/Search Paths</code>, enter the path to your <code>ninja/lib/site-packages</code> directory.</li>
+                    <li>
+                      In <code>Debug/Environment Variables</code>, enter<br/>
+                      <code>VCINSTALLDIR=C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\</code>
+                    </li>
+                    <li>
+                      If you want to enabled mixed mode debugging, check <code>Enable native code debugging</code> (this slows down debugging,
+                      so enable it only on an as-needed basis.)
+                    </li>
+                  </ul>
+                </li>
+                <li>
+                  Set the command line for the test suite to run.
+                  <ul>
+                    <li>Right click the project in solution explorer and choose the <code>Debug</code> tab.</li>
+                    <li>Enter the arguments to <code>dotest.py</code>.  Note you must add <code>--no-multiprocess</code></li>
+                    <li>
+                      Example command options:
+                      <code>
+                        <br/># quiet mode
+                        <br/>-q 
+                        <br />--arch=i686
+                        <br /># Path to debug lldb.exe
+                        <br />--executable D:/src/llvmbuild/ninja/bin/lldb.exe 
+                        <br /># Directory to store log files
+                        <br />-s D:/src/llvmbuild/ninja/lldb-test-traces 
+                        <br />-u CXXFLAGS -u CFLAGS
+                        <br /># If a test crashes, show JIT debugging dialog.
+                        <br />--enable-crash-dialog  
+                        <br /># Path to release clang.exe
+                        <br />-C d:\src\llvmbuild\ninja_release\bin\clang.exe  
+                        <br /># Path to the particular test you want to debug.
+                        <br />-p TestPaths.py   
+                        <br /># Root of test tree
+                        <br />D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test 
+                        <br /># Required in order to be able to debug the test.
+                        <br />--no-multiprocess 
+                      </code>
+                    </li>
+                    <li>
+                      As copy-pastable command line:<br/>
+                      <code>
+                        -q --arch=i686 --executable D:/src/llvmbuild/ninja/bin/lldb.exe -s D:/src/llvmbuild/ninja/lldb-test-traces 
+                        -u CXXFLAGS -u CFLAGS --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe 
+                        -p TestPaths.py D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --no-multiprocess
+                      </code>
+                    </li>
+                  </ul>
+                </li>
+              </ul>
+            </p>
+          </div>
+          <div class="postfooter"></div>
         </div>
       </div>
     </div>
   </div>
 </body>
 </html>