Win32 Module Installation

Bill Birthisel

This is an article about making easy jobs easy. It is also about MakeMaker, cross-platform installation issues, and a few other details that are not so easy.

Microsoft Windows does not, by default, install a make utility -- something that can figure out how (and in what order) to compile files into working programs. While several Win32 versions of make are freely available, they are incompatible with each other. Most Win32 Perl users aren't able to compile or build programs at all -- they installed the binary version of Perl, and lack a compiler or make equivalent. But as I'll show in this article, Perl is all you need to build many modules on Win32 -- providing they've been prepared as I describe below.

The primary ingredient is MakeMaker, bundled with Perl and available as ExtUtils::MakeMaker. Normally, MakeMaker expects that the tools used to build Perl will still be around -- a bad assumption for end users who installed Perl in binary form. MakeMaker also expects to find information about the build environment in Config.pm; again, a bad assumption for users who didn't build their own Perl.

For our purposes, we're going to consider modules with the following characteristics:

1. It's written entirely in Perl with no binary components.

2. It works without any modification on a wide variety of operating systems, including Windows.

3. It includes test programs and test data files in a t subdirectory. The tests might create additional files.

4. It might include example programs in an eg subdirectory.

5. Most of the target users don't consider themselves programmers, and don't frequently use software development tools. They would never entertain the idea of building Perl from source.

The module we will use as an example, MARC.pm, fits all of these criteria. MARC (MAchine Readable Catalog) is available on the CPAN, and its primary use is converting between the electronic interchange formats used by library catalogs and book distributors, and other useful formats such as HTML, XML, and plain text. There are many useful applications for this module, and the internal structure is interesting as well, but all that is beyond the scope of this article. We are concerned with how to distribute and install it.

MakeMaker was designed to make this easy on systems with complete sets of development tools. The installation instructions for nearly every Perl module looks like this:

  perl Makefile.PL # Create the Makefile
  make             # Create temporary directories and copy files
  make test        # Confirm it works as expected
  make install     # Put files where they belong
 

For a Perl-only module, the command h2xs -A -X -n MARC will even create a usable Makefile.PL for us:

  use ExtUtils::MakeMaker;
  # See lib/ExtUtils/MakeMaker.pm for details of how to influence
  # the contents of the Makefile that is written.
  WriteMakefile(
      'NAME'    => 'MARC',
      'VERSION_FROM' => 'MARC.pm', # finds $VERSION
  );

That certainly qualifies as easy -- unless, like many Windows users, you don't have a copy of make. You could grab one off the net, but not just any make will do -- you have to match whichever one is named in Config.pm. Or maybe you have the right make, but some other essential parameter differs between your configuration and the configuration of the person who created the module. Librarians may be good at managing structured data -- but if MARC had to provide instructions for hand-tweaking Config.pm, it wouldn't qualify as easy.

Let's take a step back. Our module is 100% pure Perl, yet we're required to use a utility traditionally used to compile programs. We're not even analyzing any dependencies -- it's just a single module that we want to test and install. So what function is make providing for us? If we look under the hood, make test does essentially this:

  PERL_DL_NONLAZY=1 $(FULLPERL) -I$(INST_LIB) -I$(PERL_LIB)
      -e 'use Test::Harness qw(&runtests); runtests @ARGV;' t/*.t
 

In other words, make test sets up an environment and runs a Perl script. The equivalent section in the Makefile adds a couple directories to @INC as well as support for $Test::Harness::verbose. The environment variable PERL_DL_NONLAZY isn't even used on Win32.

For make install, the script resolves to a call to the install() method in ExtUtils::Install. This suggests that we can do everything we need with a sequence of all-Perl commands:

  perl Makefile.PL    # Create temporary directories, copy files
                      # and PLUS generate test.pl and install.pl    
  perl test.pl        # Confirm it works as expected
  perl install.pl     # Put files where they belong
 

Now that's EASY. And it resembles the traditional sequence, too. While there is no support for developer options like make clean and make dist, end users won't miss them. This method also won't scale to cover the general cases and arbitrary degrees of complexity like make and MakeMaker, and sometimes those are the best tools. However, for our example module, we can get away with this simple installation process.

Let's look at a Makefile.PL that provides this install sequence. The first few lines will be familiar:

  # Makefile.PL
  #
  use ExtUtils::MakeMaker;
  # See lib/ExtUtils/MakeMaker.pm for details of how to influence
  # the contents of the Makefile that is written.
  
  require 5.004;
  
  unless ($^O eq "MSWin32") {
      WriteMakefile(
          'NAME'         => 'MARC',
          'VERSION_FROM' => 'MARC.pm', # finds $VERSION
          'SKIP'         => [qw(tool_autosplit)],
          'clean'        => {FILES => "t/output*.* output*.*"},
      );
      exit;
  }
  

Perl 5.004 is a prerequisite for various methods in MARC.pm. It is also the earliest Windows port with MakeMaker support -- and we still use MakeMaker tools, just not called via make. The exit makes things act the traditional way on systems other than Win32. SKIP and clean are explained in the MakeMaker documentation.

  
  use File::Copy;
  use File::Path;
  use Pod::Html;
  
  # a low-fat version of parse_version from ExtUtils::MM_Unix.
  my $version = simple_version("MARC.pm");
  my $INST_LIBDIR  = "./lib";
  my $INST_HTMLDIR = "./html";
  my $INST_FILES  = "MARC.pm";
  my $INST_NAME  = "MARC";
  my @HTML_FILES  = "MARC";
  
  # probably should auto-generate @cleanfiles
  my @cleanfiles = qw( output.txt output.html output.xml output.urls
           output2.html t/output.txt t/output.html
           t/output.xml t/output.urls t/output2.html
           output.isbd t/output.isbd );
 

On Win32, you want to avoid depending on the shell for anything; for instance, MakeMaker's make clean translates into rm -rf list_with_wildcards, which the Win32 shell will reject. Consequently, it's best to maintain an explicit list of the files to be cleaned, which we do above with the @cleanfiles array.

  print <<INTRO3;
              				MARC version $version
  
             			No 'Makefile' will be created
             			Test with:    perl test.pl
             			Install with: perl install.pl

    
  INTRO3
 

This displays a little reminder of the installation sequence for those who didn't read the README. The distribution includes two README files -- a README with linefeeds for CPAN, and a README.txt with both carriage returns and linefeeds for Win32 editors. Since Win32 Perl ports work fine with linefeed-only termination, and some other operating systems dislike the carriage returns, the rest of the files we use are linefeed-only.

Next, we create the make-target-replacement scripts:

  my $dfile = "test.pl";
  unlink $dfile, @cleanfiles;        # a bit like "make clean"
  print "Creating new $dfile\n";
  open (DEFAULT, "> $dfile") or die "Can't create $dfile: $!\n";
  
  print DEFAULT <<"TEST4";    # double quotes -- need interpolation
  # Created by Makefile.PL
  # $INST_NAME Version $version


  TEST4

"Here" documents (in the snippet above, everything between the two TEST4 tokens constitutes a here document) are quoted in the same way as the initial token. The first TEST4 is double quoted, so the entire here document is double quoted, which means that $INST_NAME and $version are replaced by their values. Since the rest of test.pl does not interpolate variables, single quoting is used.

print DEFAULT <<'TEST4';    # single quotes -- minimize character quoting
use Test::Harness;
runtests ("t/test1.t");

print "\nTo run individual tests, type:\n";
print "    C:\\> perl t/test1.t Page_Pause_Time (0..5)\n";
print "See README and other documentation for additional information.\n\n";

TEST4

close DEFAULT;

The optional Page_Pause_Time just adds a delay after each page of output, since some Win32 shells don't redirect STDERR properly.

  
  unless (-d $INST_LIBDIR) {
      File::Path::mkpath([ "$INST_LIBDIR" ],1,0777) or
          die "ERROR creating directories: ($!)\n";
  }
  unless (-d $INST_HTMLDIR) {
      File::Path::mkpath([ "$INST_HTMLDIR" ],1,0777) or
          die "ERROR creating directories: ($!)\n";
  }
  File::Copy::copy($INST_FILES,$INST_LIBDIR) or
      die "ERROR copying files: ($!)\n";
	  

This is most of what a plain make actually does on simple modules like ours.

  foreach $source (@HTML_FILES) {
      pod2html(
           "--norecurse",
           "--infile=$source.pm",
           "--outfile=$INST_HTMLDIR/$source.html"
          );
  }

On systems with standard support for online manual pages, the make builds new manual pages from pod embedded in the module. On Win32, HTML versions are created by the pod2html invocation above.

  $dfile = "install.pl";
  unlink $dfile, "pod2html-itemcache","pod2html-dircache";
  print "Creating new $dfile\n";
  open (DEFAULT, "> $dfile") or die "Can't create $dfile: $!\n";
  
  print DEFAULT <<"INST5";
  # Created by Makefile.PL
  # $INST_NAME Version $version
  
  
  INST5
  

Now we create a single-quoted string containing the program code we are writing. This means we don't have to escape every $, making it easier to read and debug. We can use printf to interpolate a variable into this string.

  my $template = <<'INST5';        # a single-quoted string
  use Config qw(%Config);
  use strict;
  use ExtUtils::Install qw( install );
  
  my $FULLEXT = "%s";        #### << printf puts $INST_NAME here
  my $INST_LIB = "./lib";
  my $HTML_LIB = "./html";
  

This next section unveils a nasty little secret. The key in %Config used to determine where to install the HTML changed from 5.004 to 5.005. No users have reported hitting the die below, so the autodetection seems adequate for all the releases I've made to date.

  my $html_dest = "";    # edit real html base here if autodetect fails
  
  if (exists $Config{installhtmldir} ) {
      $html_dest = "$Config{installhtmldir}";
  } elsif (exists $Config{installprivlib} ) {
      $html_dest = "$Config{installprivlib}";
      $html_dest =~ s%\\lib%\\html%;
  }
  
  if ( length ($html_dest) ) {
      $html_dest .= '\lib\site';
  } else {
      die "Can't find html base directory. Edit install.pl manually.\n";
  }
  

After all the smoke and mirrors (and make and blib and install_default), the final destination is the same for us as for MakeMaker. Both sequences copy files to the Promised Land and update the Holy Books with essentially identical install commands.

  install({
         read => "$Config{sitearchexp}/auto/$FULLEXT/.packlist",
           write => "$Config{installsitearch}/auto/$FULLEXT/.packlist",
         $INST_LIB => "$Config{installsitelib}",
         $HTML_LIB => "$html_dest"
        },1,0);


 __END__
  INST5
  
  printf DEFAULT $template, $INST_NAME;
  close DEFAULT;

By using printf with our single-quoted string as a template, we limit interpolation to the specific location intended. The actual Makefile.PL adds the simple_version() subroutine at the end. But its internals are not relevant to this article. Did we make an easy job easy? Let's review what the user sees and does.

    traditional MakeMaker         Win32 sequence

    perl Makefile.PL	          perl Makefile.PL
    make
    make test	                  perl test.pl
    make install	          perl install.pl

Looks easy to me. Many Perl Win32 users want to become more adept at managing their Perl installation, but are daunted by the initial learning curve. When we can succeed at making easy jobs easy, we help open Perl to a whole new group of users.

__END__