The Perl Shell

Gregor N. Purdy

When I first started writing programs in Perl, I was glad to finally work with a scripting language with real data structures. I had recently written some large Korn shell scripts (upwards of seventy pages when printed) that would have been much more easily implemented in Perl. Ever since then, I've always wished I could work in a shell environment that used Perl syntax so I didn't have to remember both shell constructs and Perl constructs. In addition, I wanted to be able to execute experimental Perl fragments directly rather than having to invoke the interpreter manually.

For a long time, I just kept these thoughts to myself, and didn't spend any time trying to solve the problem. But one day last July I was having lunch with some of my coworkers, the conversation turned to Perl. I was comparing and contrasting Perl and the Korn shell when one of the group asserted that it would be great to have a shell that had all the nice features of Perl. I couldn't stand to sit on the idea anymore, so that night in my hotel room I banged out what soon became known as version 0.001 of the Perl Shell.

This first version was extremely simple. The synopsis of the documentation read:

A fairly simple read-eval loop. The -w flag and use strict are not employed so that the user is not bound by their stipulations. Setting $^W = 1 will turn on warnings, and calling use strict will do the usual thing if called by the user.

This early version simply immediately evaluated each line of input. However, it did have a couple of interesting additional features. First, it supported multiline input through Perl's 'here-document' syntax, so you could type larger Perl fragments at it:

  psh$ <<FOO
  print "Hello";
  print ", world!\n";
  FOO

Also, an input line beginning with an exclamation point would be handed off to system(), providing very basic shell-like functionality. This first version even supported "dotting in" a file, and a .pshrc file, analogous to typical shell behavior.

All of this was pretty basic, but the one feature of the first version that I thought showed the potential of a Perl shell was the ability to set the prompt string. Version 0.001 allowed you to set the prompt string to either a constant string value or to a subroutine reference. If you did the latter, the subroutine would be called each time a prompt was needed, allowing you to do whatever processing you wanted to generate a string to display. I provided this very simple example:

$psh::prompt = sub { $i++; "psh [$i]\$ "; }

which made the prompt "psh [1]$ ", followed by "psh [2]$ ", and so on.

The reason I was so excited about this simple capability was that it was an example of going a step beyond basic expected shell behavior by providing extensibility in a way that is natural to Perl programmers. I had seen the prompt variables of bash and other shells, but hadn't used a code hook like this in any other shell.

The next few versions saw contributions by other people, and the addition of ReadLine support and prompt string variables (a la bash).

By November 1999, Version 0.002 Patch 5 was released, which added support for built-in commands (initially just cd, exit, and which), custom prompt variables (another fun extensibility feature), and history handling.

It was after this release that things really began to heat up. I had been managing the psh-dev and psh-announce mailing lists in ultra low-tech mode with my PC and basic mail client because I didn't have control over a server for managing mailing lists. By late November it was becoming clear that I had to do something. I had heard about SourceForge, and one of the list subscribers suggested I check it out when I put out a request for comments on moving to a mailing list service. In early December, the Perl Shell lists were converted over to SourceForge, and I imported all the code into CVS there as well.

December 1999 was a big month for the Perl Shell, with contributions for background jobs, signal handling, filename completion, pipelines, more builtins, internationalization, configurable evaluation strategies, a major modularization of the code (which had been monolithic up until this point), and a badly needed documentation update. These and many later changes were due largely to the efforts of what is now the Perl Shell Core Team, which consists of Markus Peter (the most prodigious contributor of them all), Simon Huggins, Omer Shenker, and Glen Whitney. Late in the month (version 0.005) saw the addition of a Win32 port, filename and custom completions, globbing, and more built-in commands.

January of 2000 saw the release of version 0.006, which contained recursive globbing and the manifest filters described below. With February came version 0.007, which included some enhancements to builtins and evaluation strategies.

Evaluation Strategies

The Perl Shell aims to be a highly extensible shell, and has numerous hooks allowing the user to customize or extend its behavior. Even the methods used to process input are customizable via the evaluation strategies mechanism. The Perl Shell comes configured with a basic set of prioritized strategies for evaluating the user's input. Each time the shell processes a command, it presents the text of the command to the strategies in order until one of them claims it. Then, the strategy consumes the input, usually taking some action on the user's behalf.

For instance, there is one strategy for detecting comment lines, one that handles lines beginning with "!", one that sends code starting with an open curly brace to Perl's eval() function, one that handles all the built-in commands (the list of which is extensible), one that handles Perl function calls with shell-like syntax of arguments, one that handles general executables, one that can provide fallback implementations of programs that are not present on some platforms but easily emulated, and finally a catch-all strategy that just sends the input to Perl's eval().

These strategies, when combined in this order, make for a very usable environment. There are some additional strategies that can be added to the list if desired (they are not enabled by default). The perlscript strategy (no relation to PerlScript, which embeds Perl commands in web pages) detects executables that are Perl scripts using the same interpreter as the Perl Shell itself, and simply fork()s and runs the new script rather than going through system().

Globbing

The Perl Shell's globbing works mostly as would be expected, although it has the advanced recursive globbing feature not found on all shells. The example from the manpage illustrates this well:

grep foo lib/**/*.pm

will search for "foo" in all *.pm files which are somewhere (recursively) within the lib directory.

Manifest Filters

Manifest filters are chunks of code that create filter processes. They are handy for creating simple one-time filters because they don't require creating a program file, setting permissions, and so on. There are three kinds: quick filters, grep filters, and substitution filters.

A quick filter consists of a block of code surrounded by curly braces, with a trailing 'q' modifier. The Perl Shell turns this into a line-by-line filter. For the code in the braces, $_ will contain the line as it was read from standard input (including any end-of-line character). The filter block should print any lines it wants to appear on standard output. Here's an example that displays a directory listing with line numbers:

ls | { print ++$i, ": $_"; }q

A grep filter consists of a block of code surrounded by curly braces, with a trailing 'g' modifier. The Perl Shell turns this into a line-by-line filter, printing lines for which the code in the braces returns true. For the code in the braces, @_ will contain the results of splitting $_ with the pattern \s+. Here's a grep filter that displays the lines of netstat for which the second column is greater than two:

netstat | { $_[1]>2; }g

A substitution filter consists of a Perl-style s/// operation. The Perl Shell turns this into a line-by-line filter that performs the substitution on each line, and then prints it:

ls | s/y/k/

A substitution filter is logically equivalent to a block filter containing the substitution and a statement to print the resulting line. So the example above is equivalent to:

ls | { s/y/k/; print; }q

Comparison To Other Shells

psh is just one of a number of Perl shells you can choose from. In this section, I'll briefly describe the others.

Larry Wall's Perl Shell

Page 161 of the Camel Book (Programming Perl, O'Reilly & Associates, 2nd Edition) shows this simple Perl shell:

  while (<>) { eval; print $@; }

lpsh

Lee Eakin wrote the Fancy Poor Man's Perl SHell (called lpsh for Lee's Perl Shell), a simple shell derived from Larry Wall's Shell that he has used for a number of years now. He has added some numeric conversion functions for use as a calculator. This shell is available at the URL http://www.dfw.nostrum.com/~leakin/psh. For brief documentation (and a reference to the main Perl Shell site), see http://www.dfw.nostrum.com/~leakin/psh.README.

Perl Debugger Shell

Surprisingly few people seem to know that Perl comes with its own symbolic debugger that functions as a shell:

perl -de 1;

perlish

Hiroo Hayashi wrote perlsh, a one-line Perl evaluator with line editing and variable name completion, as an example of his Term::ReadLine::Gnu module.

PSH.pm

Jan Krynicky maintains a Perl shell module called PSH.pm at http://jenda.krynicky.cz/. It's quite similar to psh, and is designed to provide a command line that can be called inside another program via PSH::prompt. A small file, psh.pl, is also included that uses PSH to provide a standalone shell.

softlist

Some versions of the Perl FAQ mention an interactive Perl shell called SoftList, which can be found at http://www.mit.edu/afs/sipb/contrib/perl/SoftList/. It predates Term::Readline and was apparently last touched in 1993, and so would seem to be obsolete.

timtosh

Tim Newsome, has developed a shell he calls timtosh (There Is More Than One SHell), available at http://www.wiw.org/~drz/timtosh. It's a shell written entirely in Perl. The goal is a shell which is extensible in Perl and applies other Perl features to shell tasks, like Perl regex file matching. Tim says that timtosh "is focused quite differently than psh is, but is currently still waiting for a rewrite of the command line parsing. (It has been for almost a year now)."

vbsh

Tom Christiansen and Nathan Torkington's book Perl Cookbook, published by O'Reilly in 1998, has "Example 15-4. vbsh" on page 531 for section 15.11 (Editing Input). Its name stands for "Very Bad SHell".

Comparison Of The Perl Shells

As an aid to comparing/contrasting these different shells, a brief table of features is shown in Table 1.

Resources