-
Pexpect makes Python a better tool for controlling other
-applications.
-
Pexpect is a pure Python module for spawning child applications;
-controlling them; and responding to expected patterns in their output.
-Pexpect works like Don Libes' Expect. Pexpect allows your script to
-spawn a child application and control it as if a human were typing
-commands.
-
Pexpect can be used for automating interactive applications such as
-ssh, ftp, passwd, telnet, etc. It can be used to a automate setup
-scripts for duplicating software package installations on different
-servers. It can be used for automated software testing. Pexpect is in
-the spirit of Don Libes' Expect, but Pexpect is pure Python. Unlike
-other Expect-like modules for Python, Pexpect does not require TCL or
-Expect nor does it require C extensions to be compiled. It should work
-on any platform that supports the standard Python pty module. The
-Pexpect interface was designed to be easy to use.
-
-
-
- | Send questions to: |
-  |
-
-
-
-
-
License: MIT style
-
-Free, open source, and all that good stuff.
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
-IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
-DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
-OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
-USE OR OTHER DEALINGS IN THE SOFTWARE.
-
-Pexpect Copyright (c) 2008 Noah Spurrier
-http://pexpect.sourceforge.net/
-
-
-
-
-
Download the
-current version here from the SourceForge site. Grab the current Pexpect tarball.
-
-
Installing Pexpect
-
The Pexpect tarball is a standard Python Distutil distribution.
-
- - download pexpect-2.4.tar.gz
- - tar zxf pexpect-2.4.tar.gz
- - cd pexpect-2.4
- - python setup.py install do this as root
-
-
Examples
-
-Under the pexpect-2.4 directory you should find
-the examples directory.
-This is the best way to learn to use Pexpect.
-See the descriptions of Pexpect Examples.
-
-
API Documentation
-
-
-pexpect This is the main module that you want.
-pxssh Pexpect SSH is an extension of 'pexpect.spawn' that specializes in SSH.
-
-the following are experimental extensions to Pexpect
-
-fdpexpect fdpexpect extension of 'pexpect.spawn' that uses an open file descriptor.
-SCREEN This represents a virtual 'screen'.
-ANSI This parses ANSI/VT-100 terminal escape codes.
-FSM This is a finite state machine used by ANSI.
-
-
-
-
Project Status
-
Automated pyunit tests reach over 80%
-code coverage on pexpect.py. I regularly test on Linux and BSD
-platforms. I try to test on Solaris and Irix.
-
-
-
Requirements for use of Pexpect
-
Python
-
- Pexpect was written and tested with Python 2.4. It should work on
-earlier versions that have the pty module. I
-sometimes even manually test it with Python 1.5.2, but I can't easily
-run the PyUnit test framework against Python 1.5.2, so I have less
-confidence in Pexpect on Python 1.5.2.
-
-
pty module
-
- Any POSIX system (UNIX) with a working pty
-module should be able to run Pexpect. The pty
-module is part of the Standard Python Library, so if you are running on
-a POSIX system you should have it. The pty
-module does not run the same on all platforms. It should be solid on Linux
-and BSD systems. I have taken effort to try to smooth the wrinkles out of the different platforms. To learn more
-about the wrinkles see Bugs and Testing.
-
-
Pexpect does not currently work on the standard Windows Python (see
-the pty requirement); however, it seems to work fine using Cygwin. It is possible to build
-something like a pty for Windows, but it would have to use a different
-technique that I am still investigating. I know it's possible because
-Libes' Expect was ported to Windows. If you have any ideas or
-skills to contribute in this area then I would really appreciate some
-tips on how to approach this problem.
-
-
Overview
-
Pexpect can be used for automating interactive applications such as
-ssh, ftp, mencoder, passwd, etc. The Pexpect interface was designed to be
-easy to use. Here is an example of Pexpect in action:
-
- # This connects to the openbsd ftp site and
# downloads the recursive directory listing.
import pexpect
child = pexpect.spawn ('ftp ftp.openbsd.org')
child.expect ('Name .*: ')
child.sendline ('anonymous')
child.expect ('Password:')
child.sendline ('noah@example.com')
child.expect ('ftp> ')
child.sendline ('cd pub')
child.expect('ftp> ')
child.sendline ('get ls-lR.gz')
child.expect('ftp> ')
child.sendline ('bye')
-
-
Obviously you could write an ftp client using Python's own ftplib module, but this is just a demonstration.
-You can use this technique with any application. This is especially
-handy if you are writing automated test tools.
-
-
There are two important methods in Pexpect -- expect()
-and send() (or sendline()
-which is like send() with a linefeed).
-The expect() method waits for the child application
-to return a given string. The string you specify is a regular expression, so
-you can match complicated patterns. The send() method
-writes a string to the child application. From the child's point of
-view it looks just like someone typed the text from a terminal. After
-each call to expect() the before and after
-properties will be set to the text printed by child application. The before property will contain all text up to
-the expected string pattern. The after string
-will contain the text that was matched by the expected pattern.
-The match property is set to the re MatchObject.
-
-
-
An example of Pexpect in action may make things more clear. This example uses
-ftp to login to the OpenBSD site; list files
-in a directory; and then pass interactive control of the ftp session to
-the human user.
-
- import pexpect
child = pexpect.spawn ('ftp ftp.openbsd.org')
child.expect ('Name .*: ')
child.sendline ('anonymous')
child.expect ('Password:')
child.sendline ('noah@example.com')
child.expect ('ftp> ')
child.sendline ('ls /pub/OpenBSD/')
child.expect ('ftp> ')
print child.before # Print the result of the ls command.
child.interact() # Give control of the child to the user.
-
-
Special EOF and TIMEOUT patterns
-
-There are two special patterns to match the End Of File or a Timeout condition.
-You you can pass these patterns to expect().
-These patterns are not regular expressions. Use them like predefined constants.
-
-
If the child has died and you have read all the child's output then ordinarily
-expect() will raise an EOF
-exception. You can read everything up to the EOF without generating an
-exception by using the EOF pattern expect(pexpect.EOF).
-In this case everything the child has output will be available in the before property.
-
The pattern given to expect() may be a
-regular expression or it may also be a list of regular expressions.
-This allows you to match multiple optional responses. The expect()
-method returns the index of the pattern that was matched. For example,
-say you wanted to login to a server. After entering a password you
-could get various responses from the server -- your password could be
-rejected; or you could be allowed in and asked for your terminal type;
-or you could be let right in and given a command prompt. The following
-code fragment gives an example of this:
-
- child.expect('password:')
child.sendline (my_secret_password)
# We expect any of these three patterns...
i = child.expect (['Permission denied', 'Terminal type', '[#\$] '])
if i==0:
print 'Permission denied on host. Can't login'
child.kill(0)
elif i==2:
print 'Login OK... need to send terminal type.'
child.sendline('vt100')
child.expect ('[#\$] ')
elif i==3:
print 'Login OK.'
print 'Shell command prompt', child.after
-
-
If nothing matches an expected pattern then expect will eventually
-raise a TIMEOUT exception. The default time is 30 seconds, but you can
-change this by passing a timeout argument to expect():
-
- # Wait no more than 2 minutes (120 seconds) for password prompt.
child.expect('password:', timeout=120)
-
-
Find the end of line -- CR/LF conventions
-Matching at the end of a line can be tricky
-$ regex pattern is useless.
-
-
Pexpect matches regular expressions a little differently than what
-you might be used to.
-
-
The $ pattern for end of line match is useless.
-The $ matches the end of string, but Pexpect reads from the child
-one character at a time, so each character looks like the end of a line.
-Pexpect can't do a look-ahead into the child's output stream.
-In general you would have this situation when using regular expressions
-with any stream.
-Note, pexpect does have an internal buffer, so reads are faster
-than one character at a time, but from the user's perspective the regex
-patterns test happens one character at a time.
-
The best way to match the end of a line is to look for the
-newline: "\r\n" (CR/LF). Yes, that does appear to be DOS-style.
-It may surprise some UNIX people to learn that terminal TTY device drivers
-(dumb, vt100, ANSI, xterm, etc.) all use the CR/LF combination to signify
-the end of line. Pexpect uses a Pseudo-TTY device to talk to the child application, so
-when the child app prints "\n" you actually see "\r\n".
-
-
UNIX uses just linefeeds to end lines of text, but not when it
-comes to TTY devices! TTY devices are more like the Windows world.
-Each line of text end with a CR/LF combination. When you intercept data
-from a UNIX command from a TTY device you will find that the TTY device
-outputs a CR/LF combination. A UNIX command may only write a linefeed
-(\n), but the TTY device driver converts it to CR/LF. This means that
-your terminal will see lines end with CR/LF (hex 0D 0A).
-Since Pexpect emulates a terminal, to match ends of lines you have to
-expect the CR/LF combination.
-
- child.expect ('\r\n')
-
-
If you just need to skip past a new line then expect
-('\n') by itself will work, but if you are expecting a specific
-pattern before the end of line then you need to explicitly look for the
-\r. For example the following expects a word at the end of a line:
-
- child.expect ('\w+\r\n')
-
-
But the following would both fail:
-
- child.expect ('\w+\n')
-
-
And as explained before, trying to use '$' to match the end of line
-would not work either:
-
- child.expect ('\w+$')
-
-
So if you need to explicitly look for the END OF LINE, you want to
-look for the CR/LF combination -- not just the LF and not the $ pattern.
-
This problem is not limited to Pexpect. This problem happens any
-time you try to perform a regular expression match on a stream. Regular
-expressions need to look ahead. With a stream it is hard to look ahead
-because the process generating the stream may not be finished. There is no
-way to know if the process has paused momentarily or is finished and
-waiting for you. Pexpect must implicitly always
-do a NON greedy match (minimal) at the end of a input {### already said
-this}.
-
Pexpect compiles all regular expressions with the DOTALL flag. With
-the DOTALL flag a "." will match a newline. See the Python documentation
-
Beware of + and * at the end of input.
-
Remember that any time you try to match a pattern that needs
-look-ahead that you will always get a minimal match (non greedy). For
-example, the following will always return just one character:
-
- child.expect ('.+')
-
-
This example will match successfully, but will always return no
-characters:
-
- child.expect ('.*')
-
-
Generally any star * expression will match as little as possible
-
One thing you can do is to try to force a non-ambiguous character at
-the end of your \d+ pattern. Expect that
-character to delimit the string. For example, you might try making the
-end of your pattrn be \D+ instead of \D*. That means number digits alone would not
-satisfy the (\d+) pattern. You would need
-some number(s) and at least one \D at the
-end.
-
Matching groups
-
You can group regular expression using parenthesis. After a match,
-the match parameter of the spawn object will
-contain the Python Match object.
-
Examples
-
Using "match" and groups...
-
Debugging
-
If you get the string value of a pexpect.spawn object you will get
-lots of useful debugging information. For debugging it's very useful to
-use the following pattern:
-
try:
- i = child.expect ([pattern1, pattern2, pattern3,
-etc])
-except:
- print "Exception was thrown"
- print "debug information:"
- print str(child)
-
-
It is also useful to log the child's input and out to a file or the
-screen. The following will turn on logging and send output to stdout
-(the screen).
-
-
child = pexpect.spawn (foo)
- child.logfile = sys.stdout
-
-
-
-
Exceptions
-
EOF
-
Note that two flavors of EOF Exception may be thrown. They are
-virtually identical except for the message string. For practical
-purposes you should have no need to distinguish between them, but they
-do give a little extra information about what type of platform you are
-running. The two messages are:
-
- End Of File (EOF) in read(). Exception style platform.
- End Of File (EOF) in read(). Empty string style
-platform.
-
-
Some UNIX platforms will throw an exception when you try to read
-from a file descriptor in the EOF state. Other UNIX platforms instead
-quietly return an empty string to indicate that the EOF state has been
-reached.
-
Expecting EOF
-
If you wish to read up to the end of the child's output without
-generating an EOF exception then use the expect(pexpect.EOF) method.
-
TIMEOUT
-
The expect() and read()
-methods will also timeout if the child does not generate any output for
-a given amount of time. If this happens they will raise a TIMEOUT exception. You can have these method
-ignore a timeout and block indefinitely by passing None for the timeout
-parameter.
-
- child.expect(pexpect.EOF, timeout=None)
-
-
-
FAQ
-
Q: Why don't shell pipe and redirect (| and >) work when I
-spawn a command?
-
-
-A: Remember that Pexpect does NOT interpret shell meta characters such as
-redirect, pipe, or wild cards (>, |, or *). That's done by a shell not the
-command you are spawning. This is a common mistake. If you want to run a
-command and pipe it through another command then you must also start a shell.
-For example:
-
-
- child = pexpect.spawn('/bin/sh -c "ls -l | grep LOG > log_list.txt"')
- child.expect(pexpect.EOF)
-
-
-The second form of spawn (where you pass a list of arguments) is useful in
-situations where you wish to spawn a command and pass it its own argument list.
-This can make syntax more clear. For example, the following is equivalent to
-the previous example:
-
-
- shell_cmd = 'ls -l | grep LOG > log_list.txt'
- child = pexpect.spawn ('/bin/sh', ['-c', shell_cmd])
- child.expect (pexpect.EOF)
-
-
-
-
Q: Isn't there already a Python Expect?
-
A: Yes, there are several of them. They usually require you to
-compile C. I wanted something that was pure Python and preferably a
-single module that was simple to install. I also wanted something that
-was easy to use. This pure Python expect only recently became possible
-with the introduction of the pty module in the standard Python library.
-Previously C extensions were required.
-
-
Q: The before and after properties sound weird.
-
Originally I was going to model Pexpect more after Expect, but then
-I found that I could never remember how to get the context of the stuff
-I was trying to parse. I hate having to read my own documentation. I
-decided that it was easier for me to remember what before and after
-was. It just so happens that this is how the -B and -A options in grep
-works, so that made it even easier for me to remember. Whatever makes
-my life easier is what's best.
-
-
Q: Why not just use Expect?
-
A: I love it. It's great. I has bailed me out of some real jams, but
-I wanted something that would do 90% of what I need from Expect; be 10%
-of the size; and allow me to write my code in Python instead of TCL.
-Pexpect is not nearly as big as Expect, but Pexpect does everything I
-have ever used Expect for.
-
-
-
Q: Why not just use a pipe (popen())?
-
A: A pipe works fine for getting the output to non-interactive
-programs. If you just want to get the output from ls,
-uname, or ping
-then this works. Pipes do not work very well for interactive programs
-and pipes will almost certainly fail for most applications that ask for
-passwords such as telnet, ftp, or ssh.
-
There are two reasons for this.
-
First an application may bypass stdout and print directly to its
-controlling TTY. Something like SSH will do this when it asks you for a
-password. This is why you cannot redirect the password prompt because
-it does not go through stdout or stderr.
-
The second reason is because most applications are built using the C
-Standard IO Library (anything that uses #include
-<stdio.h>). One of the features of the stdio library is
-that it buffers all input and output. Normally output is line
-buffered when a program is printing to a TTY (your terminal
-screen). Every time the program prints a line-feed the currently
-buffered data will get printed to your screen. The problem comes when
-you connect a pipe. The stdio library is smart and can tell that it is
-printing to a pipe instead of a TTY. In that case it switches from line
-buffer mode to block buffered. In this mode the
-currently buffered data is flushed when the buffer is full. This causes
-most interactive programs to deadlock. Block buffering is more
-efficient when writing to disks and pipes. Take the situation where a
-program prints a message "Enter your user name:\n" and then waits for
-you type type something. In block buffered mode, the stdio library will
-not put the message into the pipe even though a linefeed is printed.
-The result is that you never receive the message, yet the child
-application will sit and wait for you to type a response. Don't confuse
-the stdio lib's buffer with the pipe's buffer. The pipe buffer is
-another area that can cause problems. You could flush the input side of
-a pipe, whereas you have no control over the stdio library buffer.
-
More information: the Standard IO library has three states for a
-FILE *. These are: _IOFBF for block buffered; _IOLBF for line buffered;
-and _IONBF for unbuffered. The STDIO lib will use block buffering when
-talking to a block file descriptor such as a pipe. This is usually not
-helpful for interactive programs. Short of recompiling your program to
-include fflush() everywhere or recompiling a custom stdio library there
-is not much a controlling application can do about this if talking over
-a pipe.
-
The program may have put data in its output that remains unflushed
-because the output buffer is not full; then the program will go and
-deadlock while waiting for input -- because you never send it any
-because you are still waiting for its output (still stuck in the
-STDIO's output buffer).
-
The answer is to use a pseudo-tty. A TTY device will force line
-buffering (as opposed to block buffering). Line buffering means that
-you will get each line when the child program sends a line feed. This
-corresponds to the way most interactive programs operate -- send a line
-of output then wait for a line of input.
-
I put "answer" in quotes because it's ugly solution and because
-there is no POSIX standard for pseudo-TTY devices (even though they
-have a TTY standard...). What would make more sense to me would be to
-have some way to set a mode on a file descriptor so that it will tell
-the STDIO to be line-buffered. I have investigated, and I don't think
-there is a way to set the buffered state of a child process. The STDIO
-Library does not maintain any external state in the kernel or whatnot,
-so I don't think there is any way for you to alter it. I'm not quite
-sure how this line-buffered/block-buffered state change happens
-internally in the STDIO library. I think the STDIO lib looks at the
-file descriptor and decides to change behavior based on whether it's a
-TTY or a block file (see isatty()).
-
I hope that this qualifies as helpful.
-
-
Don't use a pipe to control another application...
-
Pexpect may seem similar to os.popen() or
-commands module. The main difference is that
-Pexpect (like Expect) uses a pseudo-TTY to talk to the child
-application. Most applications do no work well through the system()
-call or through pipes. And probably all applications that ask a user to
-type in a password will fail. These applications bypass the stdin and
-read directly from the TTY device. Many applications do not explicitly
-flush their output buffers. This causes deadlocks if you try to control
-an interactive application using a pipe. What happens is that most UNIX
-applications use the stdio (#include <stdio.h>) for input and
-output. The stdio library behaves differently depending on where the
-output is going. There is no way to control this behavior from the
-client end.
-
-
-
Q: Can I do screen scraping with this thing?
-
A: That depends. If your application just does line-oriented output
-then this is easy. If it does screen-oriented output then it may work,
-but it could be hard. For example, trying to scrape data from the 'top'
-command would be hard. The top command repaints the text window.
-
I am working on an ANSI / VT100 terminal emulator that will have
-methods to get characters from an arbitrary X,Y coordinate of the
-virtual screen. It works and you can play with it, but I have no
-working examples at this time.
-
-
Bugs
-
Threads
-
On Linux (RH 8) you cannot spawn a child from a different thread and
-pass the handle back to a worker thread. The child is successfully
-spawned but you can't interact with it. The only way to make it work is
-to spawn and interact with the child all in the same thread. [Adam
-Kerrison]
-
Timing issue with send() and sendline()
-
This problem has been addressed and should not effect most users.
-
It is sometimes possible to read an echo of the string sent with send() and sendline().
-If you call sendline() and then immediately
-call readline() you may get part of your
-output echoed back. You may read back what you just wrote even if the
-child application does not explicitly echo it. Timing is critical. This
-could be a security issue when talking to an application that asks for
-a password; otherwise, this does not seem like a big deal. But why
-do TTYs do this?
-
People usually report this when they are trying to control SSH or
-some other login. For example, if your code looks something like this:
-
child.expect ('[pP]assword:')
child.sendline (my_password)
-
-
-1. SSH prints "password:" prompt to the user.
-2. SSH turns off echo on the TTY device.
-3. SSH waits for user to enter a password.
-
-When scripting with Pexpect what can happen is that Pexpect will response to the "password:" prompt
-before SSH has had time to turn off TTY echo. In other words, Pexpect sends the password between
-steps 1. and 2., so the password gets echoed back to the TTY. I would call this an SSH bug.
-
-
-Pexpect now automatically adds a short delay before sending data to a child process.
-This more closely mimics what happens in the usual human-to-app interaction.
-The delay can be tuned with the 'delaybeforesend' attribute of the spawn class.
-In general, this fixes the problem for everyone and so this should not be an issue
-for most users. For some applications you might with to turn it off.
- child = pexpect.spawn ("ssh user@example.com")
- child.delaybeforesend = 0
-
-
-
-
Try changing it to look like the following. I know that this fix
-does not look correct, but it works. I have not figured out exactly
-what is happening. You would think that the sleep should be after the
-sendline(). The fact that the sleep helps when it's between the
-expect() and the sendline() must be a clue.
-
child.expect ('[pP]assword:')
child.sendline (my_password)
-
Timing issue with isalive()
-
Reading the state of isalive() immediately after a child exits may
-sometimes return 1. This is a race condition. The child has closed its
-file descriptor, but has not yet fully exited before Pexpect's
-isalive() executes. Addings a slight delay before the isalive() will
-help. In the following example isalive()
-sometimes returns 1:
-
- child = pexpect.spawn('ls')
child.expect(pexpect.EOF)
print child.isalive()
-
-
But if there is any delay before the call to isalive()
-then it will always return 0 as expected.
-
- child = pexpect.spawn('ls')
child.expect(pexpect.EOF)
time.sleep(0.1)
print child.isalive()
-
-
-
Truncated output just before child exits
-
So far I have seen this only on older versions of Apple's MacOS X.
-If the child application quits it may not flush its output buffer. This
-means that your Pexpect application will receive an EOF even though it
-should have received a little more data before the child died. This is
-not generally a problem when talking to interactive child applications.
-One example where it is a problem is when trying to read output from a
-program like 'ls'. You may receive most of
-the directory listing, but the last few lines will get lost before you
-receive an EOF. The reason for this is that 'ls'
-runs; completes its task; and then exits. The buffer is not flushed
-before exit so the last few lines are lost. The following example
-demonstrates the problem:
-
-
- child = pexpect.spawn ('ls -l')
child.expect (pexpect.EOF)
print child.before
-
-
-
-
Controlling SSH on Solaris
-
Pexpect does not yet work perfectly on Solaris.
-One common problem is that SSH sometimes will not allow TTY password
-authentication. For example, you may expect SSH to ask you for a
-password using code like this:
-
-
child = pexpect.spawn ('ssh user@example.com')
child.expect ('assword')
child.sendline ('mypassword')
-You may see the following error come back from a spawned
-child SSH:
-
-
Permission denied (publickey,keyboard-interactive).
-
-This means that SSH thinks it can't access the TTY to ask you for your
-password.
-The only solution I have found is to use public key authentication with
-SSH.
-This bypasses the need for a password. I'm not happy with this
-solution.
-The problem is due to poor support for Solaris Pseudo TTYs in the
-Python
-Standard Library.
-
-
CHANGES
-
Current Release
-
Fixed OSError exception when a pexpect object is cleaned up.
-Previously you might have seen this exception:
-
- Exception exceptions.OSError: (10, 'No child processes')
in <bound method spawn.__del__ of
<pexpect.spawn instance at 0xd248c>> ignored
-
-
You should not see that anymore. Thanks to Michael Surette.
-
Added support for buffering reads. This greatly improves speed when
-trying to match long output from a child process. When you create an
-instance of the spawn object you can then set a buffer size. For now
-you MUST do the following to turn on buffering -- it may be on by
-default in future version.
-
- child = pexpect.spawn ('my_command')
child.maxread=1000 # Sets buffer to 1000 characters.
-
-
-
I made a subtle change to the way TIMEOUT and EOF exceptions behave.
-Previously you could either expect these states in which case pexpect
-will not raise an exception, or you could just let pexpect raise an
-exception when these states were encountered. If you expected the
-states then the 'before' property was set to everything before the
-state was encountered, but if you let pexpect raise the exception then
-'before' was not set. Now the 'before' property will get set either way
-you choose to handle these states.
-
Older changes...
-
The spawn object now provides iterators for a file-like interface.
-This makes Pexpect a more complete file-like object. You can now write
-code like this:
-
- child = pexpect.spawn ('ls -l')
for line in child:
print line
-
-
I added the attribute exitstatus. This
-will give the exit code returned by the child process. This will be set
-to None while the child is still alive. When
-isalive() returns 0 then exitstatus
-will be set.
-
I made a few more tweaks to isalive() so
-that it will operate more consistently on different platforms. Solaris
-is the most difficult to support.
-
-
You can now put TIMEOUT in a list of
-expected patterns. This is just like putting EOF
-in the pattern list. Expecting for a TIMEOUT
-may not be used as often as EOF, but this
-makes Pexpect more consitent.
-
Thanks to a suggestion and sample code from Chad J. Schroeder I
-added the ability for Pexpect to operate on a file descriptor that is
-already open. This means that Pexpect can be used to control streams
-such as those from serial port devices. Now you just pass the integer
-file descriptor as the "command" when contsructing a spawn open. For
-example on a Linux box with a modem on ttyS1:
-
- fd = os.open("/dev/ttyS1", os.O_RDWR|os.O_NONBLOCK|os.O_NOCTTY)
m = pexpect.spawn(fd) # Note integer fd is used instead of usual string.
m.send("+++") # Escape sequence
m.send("ATZ0\r") # Reset modem to profile 0
rval = m.expect(["OK", "ERROR"])
-
-
Pexpect now tests itself on Compile Farm!
-
I wrote a nice script that uses ssh to connect to each machine on
-Source Forge's Compile Farm and then run the testall.py script for each
-platform. The result of the test is then recorded for each platform.
-Now it's easy to run regression tests across multiple platforms.
-
Pexpect is a file-like object
-
The spawn object now provides a file-like interface. It
-supports most of the methods and attributes defined for Python File
-Objects.
-
I changed write and writelines() so that they no longer return a
-value. Use send() if you need that functionality. I did this to make
-the Spawn object more closely match a file-like object.
-
read() was renamed to read_nonblocking(). I added a new read()
-method that matches file-like object interface. In general, you should
-not notice the difference except that read() no longer allows you to
-directly set the timeout value. I hope this will not effect any
-existing code. Switching to read_nonblocking() should fix existing code.
-
I changed the name of set_echo() to setecho().
-
I changed the name of send_eof() to sendeof().
-
I modified kill() so that it checks to
-make sure the pid isalive().
-
I modified spawn() (really called from __spawn())so that it does not raise an expection
-if setwinsize() fails. Some platforms such
-as Cygwin do not like setwinsize. This was a constant problem and since
-it is not a critical feature I decided to just silence the error.
-Normally I don't like to do that, but in this case I'm making an
-exception.
-
Added a method close() that does what you
-think. It closes the file descriptor of the child application. It makes
-no attempt to actually kill the child or wait for its status.
-
Add variables __version__ and __revision__ (from cvs) to the pexpect modules.
-This is mainly helpful to me so that I can make sure that I'm testing
-with the right version instead of one already installed.
-
Logging changes
-
- log_open() and log_close()
-have been removed. Now use setlog(). The setlog() method takes a file object. This is far
-more flexible than the previous log method. Each time data is written
-to the file object it will be flushed. To turn logging off simply call setlog() with None.
-
-
isalive changes
-
- I renamed the isAlive() method to isalive() to match the more typical naming style
-in Python. Also the technique used to detect child process status has
-been drastically modified. Previously I did some funky stuff with
-signals which caused indigestion in other Python modules on some
-platforms. It's was a big headache. It still is, but I think it works
-better now.
-
-
attribute name changes
-
- The names of some attributes have been changed. This effects the
-names of the attributes that are set after called the expect() method.
-
-
-
EOF changes
-
- The expect_eof() method is gone. You
-can now simply use the expect() method to
-look for EOF.
- Was:
-
- p.expect_eof ()
-
- Now:
-
- p.expect (pexpect.EOF)
-
-
-
-
TESTING
-
The following platforms have been tested:
-
-
-
-
Add an option to add a delay after each expect() or before each
-read()/readline() call to automatically avoid the echo
-bug.
-
-
-
-
-
-
- | |
-
-
-
-
-
Pexpect makes Python a better tool for controlling other
-applications.
-
Pexpect is a pure Python module for spawning child applications;
-controlling them; and responding to expected patterns in their output.
-Pexpect works like Don Libes' Expect. Pexpect allows your script to
-spawn a child application and control it as if a human were typing
-commands.
-
Pexpect can be used for automating interactive applications such as
-ssh, ftp, passwd, telnet, etc. It can be used to a automate setup
-scripts for duplicating software package installations on different
-servers. It can be used for automated software testing. Pexpect is in
-the spirit of Don Libes' Expect, but Pexpect is pure Python. Unlike
-other Expect-like modules for Python, Pexpect does not require TCL or
-Expect nor does it require C extensions to be compiled. It should work
-on any platform that supports the standard Python pty module. The
-Pexpect interface was designed to be easy to use.
-
-
-
- | Send questions to: |
- |
-
-
-
-
-
License: MIT style
-
-Free, open source, and all that good stuff.
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
-IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
-DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
-OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
-USE OR OTHER DEALINGS IN THE SOFTWARE.
-
-Pexpect Copyright (c) 2008 Noah Spurrier
-http://pexpect.sourceforge.net/
-
-
-
-
-
Download the
-current version here from the SourceForge site. Grab the current Pexpect tarball.
-
-
Installing Pexpect
-
The Pexpect tarball is a standard Python Distutil distribution.
-
- - download pexpect-VERSION.tar.gz
- - tar zxf pexpect-VERSION.tar.gz
- - cd pexpect-VERSION
- - python setup.py install do this as root
-
-
Examples
-
-Under the pexpect-VERSION directory you should find
-the examples directory.
-This is the best way to learn to use Pexpect.
-See the descriptions of Pexpect Examples.
-
-
API Documentation
-
-
-pexpect This is the main module that you want.
-pxssh Pexpect SSH is an extension of 'pexpect.spawn' that specializes in SSH.
-
-the following are experimental extensions to Pexpect
-
-fdpexpect fdpexpect extension of 'pexpect.spawn' that uses an open file descriptor.
-SCREEN This represents a virtual 'screen'.
-ANSI This parses ANSI/VT-100 terminal escape codes.
-FSM This is a finite state machine used by ANSI.
-
-
-
-
Project Status
-
Automated pyunit tests reach over 80%
-code coverage on pexpect.py. I regularly test on Linux and BSD
-platforms. I try to test on Solaris and Irix.
-
-
-
Requirements for use of Pexpect
-
Python
-
- Pexpect was written and tested with Python 2.4. It should work on
-earlier versions that have the pty module. I
-sometimes even manually test it with Python 1.5.2, but I can't easily
-run the PyUnit test framework against Python 1.5.2, so I have less
-confidence in Pexpect on Python 1.5.2.
-
-
pty module
-
- Any POSIX system (UNIX) with a working pty
-module should be able to run Pexpect. The pty
-module is part of the Standard Python Library, so if you are running on
-a POSIX system you should have it. The pty
-module does not run the same on all platforms. It should be solid on Linux
-and BSD systems. I have taken effort to try to smooth the wrinkles out of the different platforms. To learn more
-about the wrinkles see Bugs and Testing.
-
-
Pexpect does not currently work on the standard Windows Python (see
-the pty requirement); however, it seems to work fine using Cygwin. It is possible to build
-something like a pty for Windows, but it would have to use a different
-technique that I am still investigating. I know it's possible because
-Libes' Expect was ported to Windows. If you have any ideas or
-skills to contribute in this area then I would really appreciate some
-tips on how to approach this problem.
-
-
Overview
-
Pexpect can be used for automating interactive applications such as
-ssh, ftp, mencoder, passwd, etc. The Pexpect interface was designed to be
-easy to use. Here is an example of Pexpect in action:
-
- # This connects to the openbsd ftp site and
# downloads the recursive directory listing.
import pexpect
child = pexpect.spawn ('ftp ftp.openbsd.org')
child.expect ('Name .*: ')
child.sendline ('anonymous')
child.expect ('Password:')
child.sendline ('noah@example.com')
child.expect ('ftp> ')
child.sendline ('cd pub')
child.expect('ftp> ')
child.sendline ('get ls-lR.gz')
child.expect('ftp> ')
child.sendline ('bye')
-
-
Obviously you could write an ftp client using Python's own ftplib module, but this is just a demonstration.
-You can use this technique with any application. This is especially
-handy if you are writing automated test tools.
-
-
There are two important methods in Pexpect -- expect()
-and send() (or sendline()
-which is like send() with a linefeed).
-The expect() method waits for the child application
-to return a given string. The string you specify is a regular expression, so
-you can match complicated patterns. The send() method
-writes a string to the child application. From the child's point of
-view it looks just like someone typed the text from a terminal. After
-each call to expect() the before and after
-properties will be set to the text printed by child application. The before property will contain all text up to
-the expected string pattern. The after string
-will contain the text that was matched by the expected pattern.
-The match property is set to the re MatchObject.
-
-
-
An example of Pexpect in action may make things more clear. This example uses
-ftp to login to the OpenBSD site; list files
-in a directory; and then pass interactive control of the ftp session to
-the human user.
-
- import pexpect
child = pexpect.spawn ('ftp ftp.openbsd.org')
child.expect ('Name .*: ')
child.sendline ('anonymous')
child.expect ('Password:')
child.sendline ('noah@example.com')
child.expect ('ftp> ')
child.sendline ('ls /pub/OpenBSD/')
child.expect ('ftp> ')
print child.before # Print the result of the ls command.
child.interact() # Give control of the child to the user.
-
-
Special EOF and TIMEOUT patterns
-
-There are two special patterns to match the End Of File or a Timeout condition.
-You you can pass these patterns to expect().
-These patterns are not regular expressions. Use them like predefined constants.
-
-
If the child has died and you have read all the child's output then ordinarily
-expect() will raise an EOF
-exception. You can read everything up to the EOF without generating an
-exception by using the EOF pattern expect(pexpect.EOF).
-In this case everything the child has output will be available in the before property.
-
The pattern given to expect() may be a
-regular expression or it may also be a list of regular expressions.
-This allows you to match multiple optional responses. The expect()
-method returns the index of the pattern that was matched. For example,
-say you wanted to login to a server. After entering a password you
-could get various responses from the server -- your password could be
-rejected; or you could be allowed in and asked for your terminal type;
-or you could be let right in and given a command prompt. The following
-code fragment gives an example of this:
-
- child.expect('password:')
child.sendline (my_secret_password)
# We expect any of these three patterns...
i = child.expect (['Permission denied', 'Terminal type', '[#\$] '])
if i==0:
print 'Permission denied on host. Can't login'
child.kill(0)
elif i==2:
print 'Login OK... need to send terminal type.'
child.sendline('vt100')
child.expect ('[#\$] ')
elif i==3:
print 'Login OK.'
print 'Shell command prompt', child.after
-
-
If nothing matches an expected pattern then expect will eventually
-raise a TIMEOUT exception. The default time is 30 seconds, but you can
-change this by passing a timeout argument to expect():
-
- # Wait no more than 2 minutes (120 seconds) for password prompt.
child.expect('password:', timeout=120)
-
-
Find the end of line -- CR/LF conventions
-Matching at the end of a line can be tricky
-$ regex pattern is useless.
-
-
Pexpect matches regular expressions a little differently than what
-you might be used to.
-
-
The $ pattern for end of line match is useless.
-The $ matches the end of string, but Pexpect reads from the child
-one character at a time, so each character looks like the end of a line.
-Pexpect can't do a look-ahead into the child's output stream.
-In general you would have this situation when using regular expressions
-with any stream.
-Note, pexpect does have an internal buffer, so reads are faster
-than one character at a time, but from the user's perspective the regex
-patterns test happens one character at a time.
-
The best way to match the end of a line is to look for the
-newline: "\r\n" (CR/LF). Yes, that does appear to be DOS-style.
-It may surprise some UNIX people to learn that terminal TTY device drivers
-(dumb, vt100, ANSI, xterm, etc.) all use the CR/LF combination to signify
-the end of line. Pexpect uses a Pseudo-TTY device to talk to the child application, so
-when the child app prints "\n" you actually see "\r\n".
-
-
UNIX uses just linefeeds to end lines of text, but not when it
-comes to TTY devices! TTY devices are more like the Windows world.
-Each line of text end with a CR/LF combination. When you intercept data
-from a UNIX command from a TTY device you will find that the TTY device
-outputs a CR/LF combination. A UNIX command may only write a linefeed
-(\n), but the TTY device driver converts it to CR/LF. This means that
-your terminal will see lines end with CR/LF (hex 0D 0A).
-Since Pexpect emulates a terminal, to match ends of lines you have to
-expect the CR/LF combination.
-
- child.expect ('\r\n')
-
-
If you just need to skip past a new line then expect
-('\n') by itself will work, but if you are expecting a specific
-pattern before the end of line then you need to explicitly look for the
-\r. For example the following expects a word at the end of a line:
-
- child.expect ('\w+\r\n')
-
-
But the following would both fail:
-
- child.expect ('\w+\n')
-
-
And as explained before, trying to use '$' to match the end of line
-would not work either:
-
- child.expect ('\w+$')
-
-
So if you need to explicitly look for the END OF LINE, you want to
-look for the CR/LF combination -- not just the LF and not the $ pattern.
-
This problem is not limited to Pexpect. This problem happens any
-time you try to perform a regular expression match on a stream. Regular
-expressions need to look ahead. With a stream it is hard to look ahead
-because the process generating the stream may not be finished. There is no
-way to know if the process has paused momentarily or is finished and
-waiting for you. Pexpect must implicitly always
-do a NON greedy match (minimal) at the end of a input {### already said
-this}.
-
Pexpect compiles all regular expressions with the DOTALL flag. With
-the DOTALL flag a "." will match a newline. See the Python documentation
-
Beware of + and * at the end of input.
-
Remember that any time you try to match a pattern that needs
-look-ahead that you will always get a minimal match (non greedy). For
-example, the following will always return just one character:
-
- child.expect ('.+')
-
-
This example will match successfully, but will always return no
-characters:
-
- child.expect ('.*')
-
-
Generally any star * expression will match as little as possible
-
One thing you can do is to try to force a non-ambiguous character at
-the end of your \d+ pattern. Expect that
-character to delimit the string. For example, you might try making the
-end of your pattrn be \D+ instead of \D*. That means number digits alone would not
-satisfy the (\d+) pattern. You would need
-some number(s) and at least one \D at the
-end.
-
Matching groups
-
You can group regular expression using parenthesis. After a match,
-the match parameter of the spawn object will
-contain the Python Match object.
-
Examples
-
Using "match" and groups...
-
Debugging
-
If you get the string value of a pexpect.spawn object you will get
-lots of useful debugging information. For debugging it's very useful to
-use the following pattern:
-
try:
- i = child.expect ([pattern1, pattern2, pattern3,
-etc])
-except:
- print "Exception was thrown"
- print "debug information:"
- print str(child)
-
-
It is also useful to log the child's input and out to a file or the
-screen. The following will turn on logging and send output to stdout
-(the screen).
-
-
child = pexpect.spawn (foo)
- child.logfile = sys.stdout
-
-
-
-
Exceptions
-
EOF
-
Note that two flavors of EOF Exception may be thrown. They are
-virtually identical except for the message string. For practical
-purposes you should have no need to distinguish between them, but they
-do give a little extra information about what type of platform you are
-running. The two messages are:
-
- End Of File (EOF) in read(). Exception style platform.
- End Of File (EOF) in read(). Empty string style
-platform.
-
-
Some UNIX platforms will throw an exception when you try to read
-from a file descriptor in the EOF state. Other UNIX platforms instead
-quietly return an empty string to indicate that the EOF state has been
-reached.
-
Expecting EOF
-
If you wish to read up to the end of the child's output without
-generating an EOF exception then use the expect(pexpect.EOF) method.
-
TIMEOUT
-
The expect() and read()
-methods will also timeout if the child does not generate any output for
-a given amount of time. If this happens they will raise a TIMEOUT exception. You can have these method
-ignore a timeout and block indefinitely by passing None for the timeout
-parameter.
-
- child.expect(pexpect.EOF, timeout=None)
-
-
-
FAQ
-
Q: Why don't shell pipe and redirect (| and >) work when I
-spawn a command?
-
-
-A: Remember that Pexpect does NOT interpret shell meta characters such as
-redirect, pipe, or wild cards (>, |, or *). That's done by a shell not the
-command you are spawning. This is a common mistake. If you want to run a
-command and pipe it through another command then you must also start a shell.
-For example:
-
-
- child = pexpect.spawn('/bin/sh -c "ls -l | grep LOG > log_list.txt"')
- child.expect(pexpect.EOF)
-
-
-The second form of spawn (where you pass a list of arguments) is useful in
-situations where you wish to spawn a command and pass it its own argument list.
-This can make syntax more clear. For example, the following is equivalent to
-the previous example:
-
-
- shell_cmd = 'ls -l | grep LOG > log_list.txt'
- child = pexpect.spawn ('/bin/sh', ['-c', shell_cmd])
- child.expect (pexpect.EOF)
-
-
-
-
Q: Isn't there already a Python Expect?
-
A: Yes, there are several of them. They usually require you to
-compile C. I wanted something that was pure Python and preferably a
-single module that was simple to install. I also wanted something that
-was easy to use. This pure Python expect only recently became possible
-with the introduction of the pty module in the standard Python library.
-Previously C extensions were required.
-
-
Q: The before and after properties sound weird.
-
Originally I was going to model Pexpect more after Expect, but then
-I found that I could never remember how to get the context of the stuff
-I was trying to parse. I hate having to read my own documentation. I
-decided that it was easier for me to remember what before and after
-was. It just so happens that this is how the -B and -A options in grep
-works, so that made it even easier for me to remember. Whatever makes
-my life easier is what's best.
-
-
Q: Why not just use Expect?
-
A: I love it. It's great. I has bailed me out of some real jams, but
-I wanted something that would do 90% of what I need from Expect; be 10%
-of the size; and allow me to write my code in Python instead of TCL.
-Pexpect is not nearly as big as Expect, but Pexpect does everything I
-have ever used Expect for.
-
-
-
Q: Why not just use a pipe (popen())?
-
A: A pipe works fine for getting the output to non-interactive
-programs. If you just want to get the output from ls,
-uname, or ping
-then this works. Pipes do not work very well for interactive programs
-and pipes will almost certainly fail for most applications that ask for
-passwords such as telnet, ftp, or ssh.
-
There are two reasons for this.
-
First an application may bypass stdout and print directly to its
-controlling TTY. Something like SSH will do this when it asks you for a
-password. This is why you cannot redirect the password prompt because
-it does not go through stdout or stderr.
-
The second reason is because most applications are built using the C
-Standard IO Library (anything that uses #include
-<stdio.h>). One of the features of the stdio library is
-that it buffers all input and output. Normally output is line
-buffered when a program is printing to a TTY (your terminal
-screen). Every time the program prints a line-feed the currently
-buffered data will get printed to your screen. The problem comes when
-you connect a pipe. The stdio library is smart and can tell that it is
-printing to a pipe instead of a TTY. In that case it switches from line
-buffer mode to block buffered. In this mode the
-currently buffered data is flushed when the buffer is full. This causes
-most interactive programs to deadlock. Block buffering is more
-efficient when writing to disks and pipes. Take the situation where a
-program prints a message "Enter your user name:\n" and then waits for
-you type type something. In block buffered mode, the stdio library will
-not put the message into the pipe even though a linefeed is printed.
-The result is that you never receive the message, yet the child
-application will sit and wait for you to type a response. Don't confuse
-the stdio lib's buffer with the pipe's buffer. The pipe buffer is
-another area that can cause problems. You could flush the input side of
-a pipe, whereas you have no control over the stdio library buffer.
-
More information: the Standard IO library has three states for a
-FILE *. These are: _IOFBF for block buffered; _IOLBF for line buffered;
-and _IONBF for unbuffered. The STDIO lib will use block buffering when
-talking to a block file descriptor such as a pipe. This is usually not
-helpful for interactive programs. Short of recompiling your program to
-include fflush() everywhere or recompiling a custom stdio library there
-is not much a controlling application can do about this if talking over
-a pipe.
-
The program may have put data in its output that remains unflushed
-because the output buffer is not full; then the program will go and
-deadlock while waiting for input -- because you never send it any
-because you are still waiting for its output (still stuck in the
-STDIO's output buffer).
-
The answer is to use a pseudo-tty. A TTY device will force line
-buffering (as opposed to block buffering). Line buffering means that
-you will get each line when the child program sends a line feed. This
-corresponds to the way most interactive programs operate -- send a line
-of output then wait for a line of input.
-
I put "answer" in quotes because it's ugly solution and because
-there is no POSIX standard for pseudo-TTY devices (even though they
-have a TTY standard...). What would make more sense to me would be to
-have some way to set a mode on a file descriptor so that it will tell
-the STDIO to be line-buffered. I have investigated, and I don't think
-there is a way to set the buffered state of a child process. The STDIO
-Library does not maintain any external state in the kernel or whatnot,
-so I don't think there is any way for you to alter it. I'm not quite
-sure how this line-buffered/block-buffered state change happens
-internally in the STDIO library. I think the STDIO lib looks at the
-file descriptor and decides to change behavior based on whether it's a
-TTY or a block file (see isatty()).
-
I hope that this qualifies as helpful.
-
-
Don't use a pipe to control another application...
-
Pexpect may seem similar to os.popen() or
-commands module. The main difference is that
-Pexpect (like Expect) uses a pseudo-TTY to talk to the child
-application. Most applications do no work well through the system()
-call or through pipes. And probably all applications that ask a user to
-type in a password will fail. These applications bypass the stdin and
-read directly from the TTY device. Many applications do not explicitly
-flush their output buffers. This causes deadlocks if you try to control
-an interactive application using a pipe. What happens is that most UNIX
-applications use the stdio (#include <stdio.h>) for input and
-output. The stdio library behaves differently depending on where the
-output is going. There is no way to control this behavior from the
-client end.
-
-
-
Q: Can I do screen scraping with this thing?
-
A: That depends. If your application just does line-oriented output
-then this is easy. If it does screen-oriented output then it may work,
-but it could be hard. For example, trying to scrape data from the 'top'
-command would be hard. The top command repaints the text window.
-
I am working on an ANSI / VT100 terminal emulator that will have
-methods to get characters from an arbitrary X,Y coordinate of the
-virtual screen. It works and you can play with it, but I have no
-working examples at this time.
-
-
Bugs
-
Threads
-
On Linux (RH 8) you cannot spawn a child from a different thread and
-pass the handle back to a worker thread. The child is successfully
-spawned but you can't interact with it. The only way to make it work is
-to spawn and interact with the child all in the same thread. [Adam
-Kerrison]
-
Timing issue with send() and sendline()
-
This problem has been addressed and should not effect most users.
-
It is sometimes possible to read an echo of the string sent with send() and sendline().
-If you call sendline() and then immediately
-call readline() you may get part of your
-output echoed back. You may read back what you just wrote even if the
-child application does not explicitly echo it. Timing is critical. This
-could be a security issue when talking to an application that asks for
-a password; otherwise, this does not seem like a big deal. But why
-do TTYs do this?
-
People usually report this when they are trying to control SSH or
-some other login. For example, if your code looks something like this:
-
child.expect ('[pP]assword:')
child.sendline (my_password)
-
-
-1. SSH prints "password:" prompt to the user.
-2. SSH turns off echo on the TTY device.
-3. SSH waits for user to enter a password.
-
-When scripting with Pexpect what can happen is that Pexpect will response to the "password:" prompt
-before SSH has had time to turn off TTY echo. In other words, Pexpect sends the password between
-steps 1. and 2., so the password gets echoed back to the TTY. I would call this an SSH bug.
-
-
-Pexpect now automatically adds a short delay before sending data to a child process.
-This more closely mimics what happens in the usual human-to-app interaction.
-The delay can be tuned with the 'delaybeforesend' attribute of the spawn class.
-In general, this fixes the problem for everyone and so this should not be an issue
-for most users. For some applications you might with to turn it off.
- child = pexpect.spawn ("ssh user@example.com")
- child.delaybeforesend = 0
-
-
-
-
Try changing it to look like the following. I know that this fix
-does not look correct, but it works. I have not figured out exactly
-what is happening. You would think that the sleep should be after the
-sendline(). The fact that the sleep helps when it's between the
-expect() and the sendline() must be a clue.
-
child.expect ('[pP]assword:')
child.sendline (my_password)
-
Timing issue with isalive()
-
Reading the state of isalive() immediately after a child exits may
-sometimes return 1. This is a race condition. The child has closed its
-file descriptor, but has not yet fully exited before Pexpect's
-isalive() executes. Addings a slight delay before the isalive() will
-help. In the following example isalive()
-sometimes returns 1:
-
- child = pexpect.spawn('ls')
child.expect(pexpect.EOF)
print child.isalive()
-
-
But if there is any delay before the call to isalive()
-then it will always return 0 as expected.
-
- child = pexpect.spawn('ls')
child.expect(pexpect.EOF)
time.sleep(0.1)
print child.isalive()
-
-
-
Truncated output just before child exits
-
So far I have seen this only on older versions of Apple's MacOS X.
-If the child application quits it may not flush its output buffer. This
-means that your Pexpect application will receive an EOF even though it
-should have received a little more data before the child died. This is
-not generally a problem when talking to interactive child applications.
-One example where it is a problem is when trying to read output from a
-program like 'ls'. You may receive most of
-the directory listing, but the last few lines will get lost before you
-receive an EOF. The reason for this is that 'ls'
-runs; completes its task; and then exits. The buffer is not flushed
-before exit so the last few lines are lost. The following example
-demonstrates the problem:
-
-
- child = pexpect.spawn ('ls -l')
child.expect (pexpect.EOF)
print child.before
-
-
-
-
Controlling SSH on Solaris
-
Pexpect does not yet work perfectly on Solaris.
-One common problem is that SSH sometimes will not allow TTY password
-authentication. For example, you may expect SSH to ask you for a
-password using code like this:
-
-
child = pexpect.spawn ('ssh user@example.com')
child.expect ('assword')
child.sendline ('mypassword')
-You may see the following error come back from a spawned
-child SSH:
-
-
Permission denied (publickey,keyboard-interactive).
-
-This means that SSH thinks it can't access the TTY to ask you for your
-password.
-The only solution I have found is to use public key authentication with
-SSH.
-This bypasses the need for a password. I'm not happy with this
-solution.
-The problem is due to poor support for Solaris Pseudo TTYs in the
-Python
-Standard Library.
-
-
CHANGES
-
Current Release
-
Fixed OSError exception when a pexpect object is cleaned up.
-Previously you might have seen this exception:
-
- Exception exceptions.OSError: (10, 'No child processes')
in <bound method spawn.__del__ of
<pexpect.spawn instance at 0xd248c>> ignored
-
-
You should not see that anymore. Thanks to Michael Surette.
-
Added support for buffering reads. This greatly improves speed when
-trying to match long output from a child process. When you create an
-instance of the spawn object you can then set a buffer size. For now
-you MUST do the following to turn on buffering -- it may be on by
-default in future version.
-
- child = pexpect.spawn ('my_command')
child.maxread=1000 # Sets buffer to 1000 characters.
-
-
-
I made a subtle change to the way TIMEOUT and EOF exceptions behave.
-Previously you could either expect these states in which case pexpect
-will not raise an exception, or you could just let pexpect raise an
-exception when these states were encountered. If you expected the
-states then the 'before' property was set to everything before the
-state was encountered, but if you let pexpect raise the exception then
-'before' was not set. Now the 'before' property will get set either way
-you choose to handle these states.
-
Older changes...
-
The spawn object now provides iterators for a file-like interface.
-This makes Pexpect a more complete file-like object. You can now write
-code like this:
-
- child = pexpect.spawn ('ls -l')
for line in child:
print line
-
-
I added the attribute exitstatus. This
-will give the exit code returned by the child process. This will be set
-to None while the child is still alive. When
-isalive() returns 0 then exitstatus
-will be set.
-
I made a few more tweaks to isalive() so
-that it will operate more consistently on different platforms. Solaris
-is the most difficult to support.
-
-
You can now put TIMEOUT in a list of
-expected patterns. This is just like putting EOF
-in the pattern list. Expecting for a TIMEOUT
-may not be used as often as EOF, but this
-makes Pexpect more consitent.
-
Thanks to a suggestion and sample code from Chad J. Schroeder I
-added the ability for Pexpect to operate on a file descriptor that is
-already open. This means that Pexpect can be used to control streams
-such as those from serial port devices. Now you just pass the integer
-file descriptor as the "command" when contsructing a spawn open. For
-example on a Linux box with a modem on ttyS1:
-
- fd = os.open("/dev/ttyS1", os.O_RDWR|os.O_NONBLOCK|os.O_NOCTTY)
m = pexpect.spawn(fd) # Note integer fd is used instead of usual string.
m.send("+++") # Escape sequence
m.send("ATZ0\r") # Reset modem to profile 0
rval = m.expect(["OK", "ERROR"])
-
-
Pexpect now tests itself on Compile Farm!
-
I wrote a nice script that uses ssh to connect to each machine on
-Source Forge's Compile Farm and then run the testall.py script for each
-platform. The result of the test is then recorded for each platform.
-Now it's easy to run regression tests across multiple platforms.
-
Pexpect is a file-like object
-
The spawn object now provides a file-like interface. It
-supports most of the methods and attributes defined for Python File
-Objects.
-
I changed write and writelines() so that they no longer return a
-value. Use send() if you need that functionality. I did this to make
-the Spawn object more closely match a file-like object.
-
read() was renamed to read_nonblocking(). I added a new read()
-method that matches file-like object interface. In general, you should
-not notice the difference except that read() no longer allows you to
-directly set the timeout value. I hope this will not effect any
-existing code. Switching to read_nonblocking() should fix existing code.
-
I changed the name of set_echo() to setecho().
-
I changed the name of send_eof() to sendeof().
-
I modified kill() so that it checks to
-make sure the pid isalive().
-
I modified spawn() (really called from __spawn())so that it does not raise an expection
-if setwinsize() fails. Some platforms such
-as Cygwin do not like setwinsize. This was a constant problem and since
-it is not a critical feature I decided to just silence the error.
-Normally I don't like to do that, but in this case I'm making an
-exception.
-
Added a method close() that does what you
-think. It closes the file descriptor of the child application. It makes
-no attempt to actually kill the child or wait for its status.
-
Add variables __version__ and __revision__ (from cvs) to the pexpect modules.
-This is mainly helpful to me so that I can make sure that I'm testing
-with the right version instead of one already installed.
-
Logging changes
-
- log_open() and log_close()
-have been removed. Now use setlog(). The setlog() method takes a file object. This is far
-more flexible than the previous log method. Each time data is written
-to the file object it will be flushed. To turn logging off simply call setlog() with None.
-
-
isalive changes
-
- I renamed the isAlive() method to isalive() to match the more typical naming style
-in Python. Also the technique used to detect child process status has
-been drastically modified. Previously I did some funky stuff with
-signals which caused indigestion in other Python modules on some
-platforms. It's was a big headache. It still is, but I think it works
-better now.
-
-
attribute name changes
-
- The names of some attributes have been changed. This effects the
-names of the attributes that are set after called the expect() method.
-
-
-
EOF changes
-
- The expect_eof() method is gone. You
-can now simply use the expect() method to
-look for EOF.
- Was:
-
- p.expect_eof ()
-
- Now:
-
- p.expect (pexpect.EOF)
-
-
-
-
TESTING
-
The following platforms have been tested:
-
-
-
-
Add an option to add a delay after each expect() or before each
-read()/readline() call to automatically avoid the echo
-bug.
-
-
-
-
-
-
- | |
-
-
-
-
' + response[i:]
-
-
-def main():
-
- try:
- optlist, args = getopt.getopt(
- sys.argv[
- 1:], 'h?d', [
- 'help', 'h', '?', 'hostname=', 'username=', 'password=', 'port=', 'watch'])
- except Exception as e:
- print str(e)
- exit_with_usage()
-
- command_line_options = dict(optlist)
- options = dict(optlist)
- # There are a million ways to cry for help. These are but a few of them.
- if [elem for elem in command_line_options if elem in [
- '-h', '--h', '-?', '--?', '--help']]:
- exit_with_usage(0)
-
- hostname = "127.0.0.1"
- port = 1664
- username = os.getenv('USER')
- password = ""
- daemon_mode = False
- if '-d' in options:
- daemon_mode = True
- if '--watch' in options:
- watch_mode = True
- else:
- watch_mode = False
- if '--hostname' in options:
- hostname = options['--hostname']
- if '--port' in options:
- port = int(options['--port'])
- if '--username' in options:
- username = options['--username']
- print "Login for %s@%s:%s" % (username, hostname, port)
- if '--password' in options:
- password = options['--password']
- else:
- password = getpass.getpass('password: ')
-
- if daemon_mode:
- print "daemonizing server"
- daemonize()
- # daemonize('/dev/null','/tmp/daemon.log','/tmp/daemon.log')
-
- sys.stdout.write('server started with pid %d\n' % os.getpid())
-
- virtual_screen = ANSI.ANSI(24, 80)
- child = pxssh.pxssh()
- child.login(hostname, username, password)
- print 'created shell. command line prompt is', child.PROMPT
- #child.sendline ('stty -echo')
- # child.setecho(False)
- virtual_screen.write(child.before)
- virtual_screen.write(child.after)
-
- if os.path.exists("/tmp/mysock"):
- os.remove("/tmp/mysock")
- s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
- localhost = '127.0.0.1'
- s.bind('/tmp/mysock')
- os.chmod('/tmp/mysock', 0o777)
- print 'Listen'
- s.listen(1)
- print 'Accept'
- #s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
- #localhost = '127.0.0.1'
- #s.bind((localhost, port))
- # print 'Listen'
- # s.listen(1)
-
- r = roller(0.01, endless_poll, (child, child.PROMPT, virtual_screen))
- r.start()
- print "screen poll updater started in background thread"
- sys.stdout.flush()
-
- try:
- while True:
- conn, addr = s.accept()
- print 'Connected by', addr
- data = conn.recv(1024)
- if data[0] != ':':
- cmd = ':sendline'
- arg = data.strip()
- else:
- request = data.split(' ', 1)
- if len(request) > 1:
- cmd = request[0].strip()
- arg = request[1].strip()
- else:
- cmd = request[0].strip()
- if cmd == ':exit':
- r.cancel()
- break
- elif cmd == ':sendline':
- child.sendline(arg)
- # child.prompt(timeout=2)
- time.sleep(0.2)
- shell_window = str(virtual_screen)
- elif cmd == ':send' or cmd == ':xsend':
- if cmd == ':xsend':
- arg = arg.decode("hex")
- child.send(arg)
- time.sleep(0.2)
- shell_window = str(virtual_screen)
- elif cmd == ':cursor':
- shell_window = '%x%x' % (
- virtual_screen.cur_r, virtual_screen.cur_c)
- elif cmd == ':refresh':
- shell_window = str(virtual_screen)
-
- response = []
- response.append(shell_window)
- #response = add_cursor_blink (response, row, col)
- sent = conn.send('\n'.join(response))
- if watch_mode:
- print '\n'.join(response)
- if sent < len(response):
- print "Sent is too short. Some data was cut off."
- conn.close()
- finally:
- r.cancel()
- print "cleaning up socket"
- s.close()
- if os.path.exists("/tmp/mysock"):
- os.remove("/tmp/mysock")
- print "done!"
-
-
-def pretty_box(rows, cols, s):
- """This puts an ASCII text box around the given string, s.
- """
-
- top_bot = '+' + '-' * cols + '+\n'
- return top_bot + \
- '\n'.join(['|' + line + '|' for line in s.split('\n')]) + '\n' + top_bot
-
-
-def error_response(msg):
-
- response = []
- response.append ("""All commands start with :
-:{REQUEST} {ARGUMENT}
-{REQUEST} may be one of the following:
- :sendline: Run the ARGUMENT followed by a line feed.
- :send : send the characters in the ARGUMENT without a line feed.
- :refresh : Use to catch up the screen with the shell if state gets out of sync.
-Example:
- :sendline ls -l
-You may also leave off :command and it will be assumed.
-Example:
- ls -l
-is equivalent to:
- :sendline ls -l
-""")
- response.append(msg)
- return '\n'.join(response)
-
-
-def parse_host_connect_string(hcs):
- """This parses a host connection string in the form
- username:password@hostname:port. All fields are options expcet hostname. A
- dictionary is returned with all four keys. Keys that were not included are
- set to empty strings ''. Note that if your password has the '@' character
- then you must backslash escape it. """
-
- if '@' in hcs:
- p = re.compile(
- r'(?P