1. Computing

Discuss in my forum

The Linux / Unix Command: expect

Official Software Documentation (Man Page) Reformatted

By

spawn [args] program [args]
creates a new process running "program args" . Its stdin, stdout and stderr are connected to Expect, so that they may be read and written by other Expect commands. The connection is broken by close or if the process itself closes any of the file identifiers.

When a process is started by spawn , the variable spawn_id is set to a descriptor referring to that process. The process described by spawn_id is considered the "current process" . spawn_id may be read or written, in effect providing job control.

user_spawn_id is a global variable containing a descriptor which refers to the user. For example, when spawn_id is set to this value, expect behaves like expect_user .

.I error_spawn_id is a global variable containing a descriptor which refers to the standard error. For example, when spawn_id is set to this value, send behaves like send_error .

tty_spawn_id is a global variable containing a descriptor which refers to /dev/tty. If /dev/tty does not exist (such as in a cron, at, or batch script), then tty_spawn_id is not defined. This may be tested as:

    if {[info vars tty_spawn_id]} {
        # /dev/tty exists
    } else {
        # /dev/tty doesn't exist
        # probably in cron, batch, or at script
    }

spawn returns the UNIX process id. If no process is spawned, 0 is returned. The variable spawn_out(slave,name) is set to the name of the pty slave device.

By default, spawn echoes the command name and arguments. The -noecho flag stops spawn from doing this.

The -console flag causes console output to be redirected to the spawned process. This is not supported on all systems.

Internally, spawn uses a pty, initialized the same way as the user's tty. This is further initialized so that all settings are "sane" (according to stty(1)). If the variable stty_init is defined, it is interpreted in the style of stty arguments as further configuration. For example, "set stty_init raw" will cause further spawned processes's terminals to start in raw mode. -nottycopy skips the initialization based on the user's tty. -nottyinit skips the "sane" initialization.

Normally, spawn takes little time to execute. If you notice spawn taking a significant amount of time, it is probably encountering ptys that are wedged. A number of tests are run on ptys to avoid entanglements with errant processes. (These take 10 seconds per wedged pty.) Running Expect with the -d option will show if Expect is encountering many ptys in odd states. If you cannot kill the processes to which these ptys are attached, your only recourse may be to reboot.

If program cannot be spawned successfully because exec(2) fails (e.g. when program doesn't exist), an error message will be returned by the next interact or expect command as if program had run and produced the error message as output. This behavior is a natural consequence of the implementation of spawn . Internally, spawn forks, after which the spawned process has no way to communicate with the original Expect process except by communication via the spawn_id.

The -open flag causes the next argument to be interpreted as a Tcl file identifier (i.e., returned by open .) The spawn id can then be used as if it were a spawned process. (The file identifier should no longer be used.) This lets you treat raw devices, files, and pipelines as spawned processes without using a pty. 0 is returned to indicate there is no associated process. When the connection to the spawned process is closed, so is the Tcl file identifier. The -leaveopen flag is similar to -open except that -leaveopen causes the file identifier to be left open even after the spawn id is closed.

The -pty flag causes a pty to be opened but no process spawned. 0 is returned to indicate there is no associated process. Spawn_id is set as usual.

The variable spawn_out(slave,fd) is set to a file identifier corresponding to the pty slave. It can be closed using "close -slave".

The -ignore flag names a signal to be ignored in the spawned process. Otherwise, signals get the default behavior. Signals are named as in the trap command, except that each signal requires a separate flag.

strace level
causes following statements to be printed before being executed. (Tcl's trace command traces variables.) level indicates how far down in the call stack to trace. For example, the following command runs Expect while tracing the first 4 levels of calls, but none below that.

    expect -c "strace 4" script.exp

The -info flag causes strace to return a description of the most recent non-info arguments given.

stty args
changes terminal modes similarly to the external stty command.

By default, the controlling terminal is accessed. Other terminals can be accessed by appending "< /dev/tty..." to the command. (Note that the arguments should not be grouped into a single argument.)

Requests for status return it as the result of the command. If no status is requested and the controlling terminal is accessed, the previous status of the raw and echo attributes are returned in a form which can later be used by the command.

For example, the arguments raw or -cooked put the terminal into raw mode. The arguments -raw or cooked put the terminal into cooked mode. The arguments echo and -echo put the terminal into echo and noecho mode respectively.

The following example illustrates how to temporarily disable echoing. This could be used in otherwise-automatic scripts to avoid embedding passwords in them. (See more discussion on this under EXPECT HINTS below.)

    stty -echo
    send_user "Password: "
    expect_user -re "(.*)\n"
    set password $expect_out(1,string)
    stty echo

system args
gives args to sh(1) as input, just as if it had been typed as a command from a terminal. Expect waits until the shell terminates. The return status from sh is handled the same way that exec handles its return status.

In contrast to exec which redirects stdin and stdout to the script, system performs no redirection (other than that indicated by the string itself). Thus, it is possible to use programs which must talk directly to /dev/tty. For the same reason, the results of system are not recorded in the log.

timestamp [args]
returns a timestamp. With no arguments, the number of seconds since the epoch is returned.

The -format flag introduces a string which is returned but with substitutions made according to the POSIX rules for strftime. For example %a is replaced by an abbreviated weekday name (i.e., Sat). Others are:

    %a      abbreviated weekday name
    %A      full weekday name
    %b      abbreviated month name
    %B      full month name
    %c      date-time as in: Wed Oct  6 11:45:56 1993
    %d      day of the month (01-31)
    %H      hour (00-23)
    %I      hour (01-12)
    %j      day (001-366)
    %m      month (01-12)
    %M      minute (00-59)
    %p      am or pm
    %S      second (00-61)
    %u      day (1-7, Monday is first day of week)
    %U      week (00-53, first Sunday is first day of week one)
    %V      week (01-53, ISO 8601 style)
    %w      day (0-6)
    %W      week (00-53, first Monday is first day of week one)
    %x      date-time as in: Wed Oct  6 1993
    %X      time as in: 23:59:59
    %y      year (00-99)
    %Y      year as in: 1993
    %Z      timezone (or nothing if not determinable)
    %%      a bare percent sign
Other % specifications are undefined. Other characters will be passed through untouched. Only the C locale is supported.

The -seconds flag introduces a number of seconds since the epoch to be used as a source from which to format. Otherwise, the current time is used.

The -gmt flag forces timestamp output to use the GMT timezone. With no flag, the local timezone is used.

trap [[command] signals]
causes the given command to be executed upon future receipt of any of the given signals. The command is executed in the global scope. If command is absent, the signal action is returned. If command is the string SIG_IGN, the signals are ignored. If command is the string SIG_DFL, the signals are result to the system default. signals is either a single signal or a list of signals. Signals may be specified numerically or symbolically as per signal(3). The "SIG" prefix may be omitted.

With no arguments (or the argument -number), trap returns the signal number of the trap command currently being executed.

The -code flag uses the return code of the command in place of whatever code Tcl was about to return when the command originally started running.

The -interp flag causes the command to be evaluated using the interpreter active at the time the command started running rather than when the trap was declared.

The -name flag causes the trap command to return the signal name of the trap command currently being executed.

The -max flag causes the trap command to return the largest signal number that can be set.

For example, the command "trap {send_user "Ouch!"} SIGINT" will print "Ouch!" each time the user presses ^C.

By default, SIGINT (which can usually be generated by pressing ^C) and SIGTERM cause Expect to exit. This is due to the following trap, created by default when Expect starts.

    trap exit {SIGINT SIGTERM}
If you use the -D flag to start the debugger, SIGINT is redefined to start the interactive debugger. This is due to the following trap:
    trap {exp_debug 1} SIGINT
The debugger trap can be changed by setting the environment variable EXPECT_DEBUG_INIT to a new trap command.

You can, of course, override both of these just by adding trap commands to your script. In particular, if you have your own "trap exit SIGINT", this will override the debugger trap. This is useful if you want to prevent users from getting to the debugger at all.

If you want to define your own trap on SIGINT but still trap to the debugger when it is running, use:

    if {![exp_debug]} {trap mystuff SIGINT}
Alternatively, you can trap to the debugger using some other signal.

trap will not let you override the action for SIGALRM as this is used internally to Expect . The disconnect command sets SIGALRM to SIG_IGN (ignore). You can reenable this as long as you disable it during subsequent spawn commands.

See signal(3) for more info.

wait [args]
delays until a spawned process (or the current process if none is named) terminates.

wait normally returns a list of four integers. The first integer is the pid of the process that was waited upon. The second integer is the corresponding spawn id. The third integer is -1 if an operating system error occurred, or 0 otherwise. If the third integer was 0, the fourth integer is the status returned by the spawned process. If the third integer was -1, the fourth integer is the value of errno set by the operating system. The global variable errorCode is also set.

Additional elements may appear at the end of the return value from wait . An optional fifth element identifies a class of information. Currently, the only possible value for this element is CHILDKILLED in which case the next two values are the C-style signal name and a short textual description.

The -i flag declares the process to wait corresponding to the named spawn_id (NOT the process id). Inside a SIGCHLD handler, it is possible to wait for any spawned process by using the spawn id -1.

The -nowait flag causes the wait to return immediately with the indication of a successful wait. When the process exits (later), it will automatically disappear without the need for an explicit wait.

The wait command may also be used wait for a forked process using the arguments "-i -1". Unlike its use with spawned processes, this command can be executed at any time. There is no control over which process is reaped. However, the return value can be checked for the process id.

LIBRARIES

Expect automatically knows about two built-in libraries for Expect scripts. These are defined by the directories named in the variables exp_library and exp_exec_library. Both are meant to contain utility files that can be used by other scripts.

exp_library contains architecture-independent files. exp_exec_library contains architecture-dependent files. Depending on your system, both directories may be totally empty. The existence of the file $exp_exec_library/cat-buffers describes whether your /bin/cat buffers by default.

PRETTY-PRINTING

A vgrind definition is available for pretty-printing Expect scripts. Assuming the vgrind definition supplied with the Expect distribution is correctly installed, you can use it as:
    vgrind -lexpect file

EXAMPLES

It many not be apparent how to put everything together that the man page describes. I encourage you to read and try out the examples in the example directory of the Expect distribution. Some of them are real programs. Others are simply illustrative of certain techniques, and of course, a couple are just quick hacks. The INSTALL file has a quick overview of these programs.

The Expect papers (see SEE ALSO) are also useful. While some papers use syntax corresponding to earlier versions of Expect, the accompanying rationales are still valid and go into a lot more detail than this man page.

CAVEATS

Extensions may collide with Expect's command names. For example, send is defined by Tk for an entirely different purpose. For this reason, most of the Expect commands are also available as "exp_XXXX". Commands and variables beginning with "exp", "inter", "spawn", and "timeout" do not have aliases. Use the extended command names if you need this compatibility between environments.

Expect takes a rather liberal view of scoping. In particular, variables read by commands specific to the Expect program will be sought first from the local scope, and if not found, in the global scope. For example, this obviates the need to place "global timeout" in every procedure you write that uses expect . On the other hand, variables written are always in the local scope (unless a "global" command has been issued). The most common problem this causes is when spawn is executed in a procedure. Outside the procedure, spawn_id no longer exists, so the spawned process is no longer accessible simply because of scoping. Add a "global spawn_id" to such a procedure.

If you cannot enable the multispawning capability (i.e., your system supports neither select (BSD *.*), poll (SVR>2), nor something equivalent), Expect will only be able to control a single process at a time. In this case, do not attempt to set spawn_id , nor should you execute processes via exec while a spawned process is running. Furthermore, you will not be able to expect from multiple processes (including the user as one) at the same time.

Terminal parameters can have a big effect on scripts. For example, if a script is written to look for echoing, it will misbehave if echoing is turned off. For this reason, Expect forces sane terminal parameters by default. Unfortunately, this can make things unpleasant for other programs. As an example, the emacs shell wants to change the "usual" mappings: newlines get mapped to newlines instead of carriage-return newlines, and echoing is disabled. This allows one to use emacs to edit the input line. Unfortunately, Expect cannot possibly guess this.

You can request that Expect not override its default setting of terminal parameters, but you must then be very careful when writing scripts for such environments. In the case of emacs, avoid depending upon things like echoing and end-of-line mappings.

The commands that accepted arguments braced into a single list (the expect variants and interact ) use a heuristic to decide if the list is actually one argument or many. The heuristic can fail only in the case when the list actually does represent a single argument which has multiple embedded \n's with non-whitespace characters between them. This seems sufficiently improbable, however the argument "-nobrace" can be used to force a single argument to be handled as a single argument. This could conceivably be used with machine-generated Expect code. Similarly, -brace forces a single argument to be handle as multiple patterns/actions.

BUGS

It was really tempting to name the program "sex" (for either "Smart EXec" or "Send-EXpect"), but good sense (or perhaps just Puritanism) prevailed.

On some systems, when a shell is spawned, it complains about not being able to access the tty but runs anyway. This means your system has a mechanism for gaining the controlling tty that Expect doesn't know about. Please find out what it is, and send this information back to me.

Ultrix 4.1 (at least the latest versions around here) considers timeouts of above 1000000 to be equivalent to 0.

Digital UNIX 4.0A (and probably other versions) refuses to allocate ptys if you define a SIGCHLD handler. See grantpt page for more info.

IRIX 6.0 does not handle pty permissions correctly so that if Expect attempts to allocate a pty previously used by someone else, it fails. Upgrade to IRIX 6.1.

Telnet (verified only under SunOS 4.1.2) hangs if TERM is not set. This is a problem under cron, at and in cgi scripts, which do not define TERM. Thus, you must set it explicitly - to what type is usually irrelevant. It just has to be set to something! The following probably suffices for most cases.

    set env(TERM) vt100

Tip (verified only under BSDI BSD/OS 3.1 i386) hangs if SHELL and HOME are not set. This is a problem under cron, at and in cgi scripts, which do not define these environment variables. Thus, you must set them explicitly - to what type is usually irrelevant. It just has to be set to something! The following probably suffices for most cases.

    set env(SHELL) /bin/sh
    set env(HOME) /usr/local/bin

Some implementations of ptys are designed so that the kernel throws away any unread output after 10 to 15 seconds (actual number is implementation-dependent) after the process has closed the file descriptor. Thus Expect programs such as

    spawn date
    sleep 20
    expect
will fail. To avoid this, invoke non-interactive programs with exec rather than spawn . While such situations are conceivable, in practice I have never encountered a situation in which the final output of a truly interactive program would be lost due to this behavior.

On the other hand, Cray UNICOS ptys throw away any unread output immediately after the process has closed the file descriptor. I have reported this to Cray and they are working on a fix.

Sometimes a delay is required between a prompt and a response, such as when a tty interface is changing UART settings or matching baud rates by looking for start/stop bits. Usually, all this is require is to sleep for a second or two. A more robust technique is to retry until the hardware is ready to receive input. The following example uses both strategies:

    send "speed 9600\r";
    sleep 1
    expect {
        timeout {send "\r"; exp_continue}
        $prompt
    }

trap -code will not work with any command that sits in Tcl's event loop, such as sleep. The problem is that in the event loop, Tcl discards the return codes from async event handlers. A workaround is to set a flag in the trap code. Then check the flag immediately after the command (i.e., sleep).

The expect_background command ignores -timeout arguments and has no concept of timeouts in general.

"EXPECT HINTS"

There are a couple of things about Expect that may be non-intuitive. This section attempts to address some of these things with a couple of suggestions.

A common expect problem is how to recognize shell prompts. Since these are customized differently by differently people and different shells, portably automating rlogin can be difficult without knowing the prompt. A reasonable convention is to have users store a regular expression describing their prompt (in particular, the end of it) in the environment variable EXPECT_PROMPT. Code like the following can be used. If EXPECT_PROMPT doesn't exist, the code still has a good chance of functioning correctly.

    set prompt "(%|#|\\$) $"          ;# default prompt
    catch {set prompt $env(EXPECT_PROMPT)}
    expect -re $prompt
I encourage you to write expect patterns that include the end of whatever you expect to see. This avoids the possibility of answering a question before seeing the entire thing. In addition, while you may well be able to answer questions before seeing them entirely, if you answer early, your answer may appear echoed back in the middle of the question. In other words, the resulting dialogue will be correct but look scrambled.

Most prompts include a space character at the end. For example, the prompt from ftp is 'f', 't', 'p', '>' and . To match this prompt, you must account for each of these characters. It is a common mistake not to include the blank. Put the blank in explicitly.

If you use a pattern of the form X*, the * will match all the output received from the end of X to the last thing received. This sounds intuitive but can be somewhat confusing because the phrase "last thing received" can vary depending upon the speed of the computer and the processing of I/O both by the kernel and the device driver.

In particular, humans tend to see program output arriving in huge chunks (atomically) when in reality most programs produce output one line at a time. Assuming this is the case, the * in the pattern of the previous paragraph may only match the end of the current line even though there seems to be more, because at the time of the match that was all the output that had been received.

expect has no way of knowing that further output is coming unless your pattern specifically accounts for it.

Even depending on line-oriented buffering is unwise. Not only do programs rarely make promises about the type of buffering they do, but system indigestion can break output lines up so that lines break at seemingly random places. Thus, if you can express the last few characters of a prompt when writing patterns, it is wise to do so.

If you are waiting for a pattern in the last output of a program and the program emits something else instead, you will not be able to detect that with the timeout keyword. The reason is that expect will not timeout - instead it will get an eof indication. Use that instead. Even better, use both. That way if that line is ever moved around, you won't have to edit the line itself.

Newlines are usually converted to carriage return, linefeed sequences when output by the terminal driver. Thus, if you want a pattern that explicitly matches the two lines, from, say, printf("foo\nbar"), you should use the pattern "foo\r\nbar".

A similar translation occurs when reading from the user, via expect_user . In this case, when you press return, it will be translated to a newline. If Expect then passes that to a program which sets its terminal to raw mode (like telnet), there is going to be a problem, as the program expects a true return. (Some programs are actually forgiving in that they will automatically translate newlines to returns, but most don't.) Unfortunately, there is no way to find out that a program put its terminal into raw mode.

Rather than manually replacing newlines with returns, the solution is to use the command "stty raw", which will stop the translation. Note, however, that this means that you will no longer get the cooked line-editing features.

interact implicitly sets your terminal to raw mode so this problem will not arise then.

It is often useful to store passwords (or other private information) in Expect scripts. This is not recommended since anything that is stored on a computer is susceptible to being accessed by anyone. Thus, interactively prompting for passwords from a script is a smarter idea than embedding them literally. Nonetheless, sometimes such embedding is the only possibility.

Unfortunately, the UNIX file system has no direct way of creating scripts which are executable but unreadable. Systems which support setgid shell scripts may indirectly simulate this as follows:

Create the Expect script (that contains the secret data) as usual. Make its permissions be 750 (-rwxr-x---) and owned by a trusted group, i.e., a group which is allowed to read it. If necessary, create a new group for this purpose. Next, create a /bin/sh script with permissions 2751 (-rwxr-s--x) owned by the same group as before.

The result is a script which may be executed (and read) by anyone. When invoked, it runs the Expect script.

"SEE ALSO"

Tcl (3), libexpect (3)
"Exploring Expect: A Tcl-Based Toolkit for Automating Interactive Programs" by Don Libes, pp. 602, ISBN 1-56592-090-2, O'Reilly and Associates, 1995.
"expect: Curing Those Uncontrollable Fits of Interactivity" by Don Libes, Proceedings of the Summer 1990 USENIX Conference, Anaheim, California, June 11-15, 1990.
.I "Using expect to Automate System Administration Tasks" by Don Libes, Proceedings of the 1990 USENIX Large Installation Systems Administration Conference, Colorado Springs, Colorado, October 17-19, 1990.
.I "Tcl: An Embeddable Command Language" by John Ousterhout, Proceedings of the Winter 1990 USENIX Conference, Washington, D.C., January 22-26, 1990.
.I "expect: Scripts for Controlling Interactive Programs" by Don Libes, Computing Systems, Vol. 4, No. 2, University of California Press Journals, November 1991.
.I "Regression Testing and Conformance Testing Interactive Programs", by Don Libes, Proceedings of the Summer 1992 USENIX Conference, pp. 135-144, San Antonio, TX, June 12-15, 1992.
.I "Kibitz - Connecting Multiple Interactive Programs Together", by Don Libes, Software - Practice & Experience, John Wiley & Sons, West Sussex, England, Vol. 23, No. 5, May, 1993.
.I "A Debugger for Tcl Applications", by Don Libes, Proceedings of the 1993 Tcl/Tk Workshop, Berkeley, CA, June 10-11, 1993.

AUTHOR

Don Libes, National Institute of Standards and Technology

ACKNOWLEDGMENTS

Thanks to John Ousterhout for Tcl, and Scott Paisley for inspiration. Thanks to Rob Savoye for Expect's autoconfiguration code.

The HISTORY file documents much of the evolution of expect . It makes interesting reading and might give you further insight to this software. Thanks to the people mentioned in it who sent me bug fixes and gave other assistance.

Design and implementation of Expect was paid for in part by the U.S. government and is therefore in the public domain. However the author and NIST would like credit if this program and documentation or portions of them are used.

  1. About.com
  2. Computing
  3. Linux
  4. Linux / Shell Commands
  5. The Linux / Unix Command: expect

©2014 About.com. All rights reserved.