Java Programming on the Sharp Zaurus

The Sharp Zaurus (SL-5000) is a geek's dream device. A color, touch-sensitive screen, a tiny QWERTY keyboard, and the Linux operating system are crammed into a palm-sized package. A CF card slot allows for an 802.11 or Bluetooth wireless networking card. Best of all, the Zaurus can run Java applications. The Zaurus was on sale at the 2002 JavaOne Conference for a steep discount. If you've picked one up, you might be wondering what you can do with the Java platform on the Zaurus.

This article describes how to program your Zaurus using the Java programming language. It provides a quick demonstration, then discusses the details of the Personal Profile and its close cousin, the PersonalJava platform. The article concludes by showing how to package a Java application for the Zaurus.

Cheap Thrills

Talk is cheap. This section describes how you can get some Java code running on your Zaurus quickly. Later on, I'll discuss the details.

I assume you have a Java compiler handy on your desktop computer. If you don't, you should skip this section and read the rest of the article before you start writing code. Copy the following source code into a text editor and save it as HelloPP.java. (You may download the file if you wish.)

import java.awt.*; import java.awt.event.*; public class HelloPP extends Canvas { public void paint(Graphics g) { Dimension d = getSize(); int cx = d.width / 2; int cy = d.height / 2; for (int x = 0; x < d.width; x++) { if (x % 2 == 0) g.setColor(Color.black); else g.setColor(Color.white); g.drawLine(cx, cy, x, 0); g.drawLine(cx, cy, x, d.height); } for (int y = 0; y < d.height; y++) { if (y % 2 == 0) g.setColor(Color.black); else g.setColor(Color.white); g.drawLine(cx, cy, 0, y); g.drawLine(cx, cy, d.width, y); } } public static void main(String[] args) { final Frame f = new Frame("HelloPP"); Dimension d = Toolkit.getDefaultToolkit().getScreenSize(); f.setLocation(0, 0); f.setSize(d.width, d.height); Component c = new HelloPP(); f.add(c); f.addWindowListener(new WindowAdapter() { public void windowClosing(WindowEvent we) { f.dispose(); System.exit(0); } }); f.setVisible(true); } }

Now compile this source code with this command:

javac HelloPP.java

If you are using J2SDK 1.4.0, make sure to generate the older style of classfiles:

javac -target 1.1 HelloPP.java

If there are no errors, go ahead and run this example on your desktop:

java HelloPP

You should see a screen like this:

HelloPP in action

Now that HelloPP works on your desktop, I'll show you how to get it running on the Zaurus quickly. This process results in an ugly way to run the application. Later on I'll show you a way to deploy it that looks nicer to the user.

The first step is to transfer the class files from your desktop to the Zaurus. You can do this any way you like. I have an 802.11 card for the Zaurus, so I just use FTP to copy the files. Alternatively, you can use the Qtopia desktop software to transfer files through the USB cradle. The files you need are HelloPP.class and HelloPP$1.class. You can put them anywhere you want on the Zaurus; I placed them in /home/root.

Once you've transferred the class files to the Zaurus, use the Terminal application to open a command line. Navigate to the directory where you placed the class files. Type the following:

evm HelloPP

The same design should appear on your Zaurus. (evm is the Insignia Jeode PersonalJava runtime environment.)

Understanding the Personal Profile and PersonalJava

The Personal Profile (JSR 62) is a J2ME profile, a specification for a standard Java runtime environment. It is part of a software stack (see Introduction to Wireless Java Technology) designed for devices like the Sharp Zaurus and Compaq iPaq. It is built on the Personal Basis Profile (JSR 129), the Foundation Profile (JSR 46), and ultimately the Connected Device Configuration (JSR 36).

Personal Profile software stack

The Personal Profile is the J2ME incarnation of an older Java runtime environment, PersonalJava, which resembles the Java Development Kit version 1.1.8. PersonalJava runtime environments use a Java virtual machine¹ (JVM), just as Java 2, Standard Edition does. Since the heyday of JDK 1.1.8, however, PersonalJava and J2SE have evolved separately. In terms of target device capacity, PersonalJava sits between J2SE and the CLDC/MIDP stack of J2ME. (Read Introduction to Wireless Java Technology for background information on J2ME.)

The Personal Profile software stack is built on the Connected Device Configuration (CDC), which specifies both a virtual machine and a basic set of APIs. The CDC is a superset of the Connected, Limited Device Configuration (CLDC) familiar to MIDlet programmers. It contains fundamental APIs from J2SE, including java.io, java.lang, java.net, java.security, and java.util. CDC implementations must support local files and datagram communications.

The Foundation Profile builds on the CDC, fleshing out its packages with support for socket and HTTP connection, among other things. The Personal Basis Profile adds AWT support with the exception of heavyweight components. The Personal Profile rounds out the software stack with support for AWT heavyweight components.

Developing Personal Profile Applications

Because the Personal Profile closely resembles the PersonalJava platform, you can use PersonalJava tools to get a quick start on Personal Profile development. Eventually Personal Profile tools will be built, but in the short term the PersonalJava tools are handy, available, and work for Personal Profile development.

The J2ME Personal Profile (and the PersonalJava platform before it) includes a JVM, just as J2SE does. Consequently, Personal Profile or PersonalJava development is very similar to J2SE development. The tricky part is that the Personal Profile's set of APIs differs from J2SE's. Setting up a development environment is a matter of installing a JDK that is close to the version of Personal Profile or PersonalJava you want to use, then obtaining compatibility classes that include the APIs that the JDK lacks.

Which JDK to use depends on which version of the Personal Profile or PersonalJava you prefer. The PersonalJava Web site contains a chart that explains the relationship between PersonalJava versions and JDK versions .

The compatibility classes are packaged as an archive; simply add this archive to the classpath you use for compiling and testing your applications.

Bear in mind that once you install the appropriate JDK and compatibility classes, many APIs may be available during development that are not actually present in a Personal Profile runtime environment. One simple example: the JDK may include the Swing user interface classes, while the Personal Profile does not. A Swing application that runs in your J2SE development environment will not run in a Personal Profile runtime environment. How do you make sure you're using only Personal Profile classes? You can check the specification as you code, but there are also some tools to help you.

The PersonalJava Emulation Environment (PJEE) tool simulates a PersonalJava device. You can use this tool to test your application in an environment that complies with the PersonalJava specification. Various versions of the PJEE are available, corresponding to different versions of the PersonalJava specification, different host platforms, and different levels of support for graphic user interface. See the PersonalJava Web site for more details. Because the Personal Profile and PersonalJava environments are so closely related, you can use PJEE as a tool for developing Personal Profile applications.

Another handy tool is JavaCheck. JavaCheck performs a comprehensive analysis of the class files in your application to see whether it complies with the specification. Like the PJEE, JavaCheck has different versions for the different PersonalJava specification versions.

PersonalJava on the Zaurus

The Zaurus comes with a PersonalJava runtime environment already installed, Jeode from Insignia Solutions. At the command line, evm invokes the Jeode virtual machine. This is the virtual machine you used to run the simple example at the beginning of this article.

Personal Profile for Zaurus

Sun has created a highly tuned Personal Profile implementation for the Zaurus. To learn how to download this implementation, send an email to ppti@sun.com. You will receive an automatic response that contains instructions for downloading and installing the software. Note that this software is for evaluation only; it is not supported and cannot be used in a product. This release is for you if you are interested in the leading edge of Personal Profile technology and would like to see a high-performance, optimized implementation.

To install the Personal Profile implementation for the Zaurus, follow the instructions in the email response from ppti@sun.com.

If you installed cvm into a directory in your path, you can type cvm at the command line to invoke the virtual machine. Use -Djava.class.path to specify a classpath. If you already have the HelloPP class installed, you can run it using cvm HelloPP.

Pretty Packaging for the Zaurus

As you've seen, it's a straightforward process to install .class files on the Zaurus and run them from the command line using evm or cvm. While this is fine for an individual developer, a packaged application should have a much smoother interface.

In this section I'll show you how to install a Java application in a way that adds it to the Zaurus's application menu. The Zaurus knows how to install .ipk files, which are archives. The archive is really a GZipped tar archive, as shown here:

.ipk package structure

The trick to packaging your application for the Zaurus is to use the right directory structure to assemble the pieces. Create the following hierarchy somewhere on your computer:

Project Directory Structure

All of the class and resource files for the application should go in the home/QtPalmtop/java directory. Copy this application's two class files, HelloPP.class and HelloPP$1.class, into home/QtPalmtop/java.

Next you need to tell the Zaurus how to run your application. Create the following script and save it as home/QtPalmtop/bin/run_hellozaurus.

. /home/QtPalmtop/bin/installdir.sh
$QPEDIR/bin/cvm -Djava.class.path=$INSTALLDIR/java HelloPP

The first line runs a script that initializes the INSTALLDIR variable. Then the second line simply calls cvm, just as you did manually earlier. (Use $QPEDIR/bin/evm -cp $INSTALLDIR/java HelloPP if you're still using the Jeode PersonalJava environment.)

If you want your application to look nice, you'll need an icon file. Create an icon (32x32 pixels, PNG format) or download my icon and save it as home/QtPalmtop/pics/hellozaurus.png.

The next thing to create is a desktop entry file. This file tells the Zaurus how to show your application in the application menu and how to run it. Save the following file as home/QtPalmtop/Games/hellozaurus.desktop.

[Desktop Entry] Comment=Simple Moire Exec=run_hellozaurus Icon=hellozaurus.png Type=Application Name=HelloZaurus

Notice how the desktop entyr points to the files you've already installed: the Exec= line to run the run_hellozaurus script in the bin subdirectory, and the Icon= line to the icon file in pics.

Finally, the application installation archive needs a control file. This is another text file that contains some simple information about the archive. Save the following text as control, in the same directory as home.

Package: HelloZaurus Installed-Size: 3k Filename: ./hellozaurus-cvm_1.0_arm.ipk Version: 1.0 Architecture: Arm Maintainer: Jonathan Knudsen Description: Simple Moire Application Section: Java

Building the archive is straightforward if you have tools that create .tar and .gz files. Linux, Solaris, Mac OS X and other Unix-like operating systems include tar and gzip programs. On MS Windows platforms, you need to obtain appropriate tools; PKZIP, ARJ, WinZip and other are readily available. On my OS X machine, I use the following script to create the .ipk file.

rm *.ipk rm *.gz tar -cvf control.tar ./control tar -cvf data.tar ./home gzip control.tar gzip data.tar tar -cvf hellozaurus.tar ./control.tar.gz ./data.tar.gz gzip hellozaurus.tar mv hellozaurus.tar.gz hellozaurus-cvm_1.0_arm.ipk

Installing Your Application

Once you create the application archive, it's a simple matter to install it:

1. Transfer the .ipk file to the /home/root/Documents/apps/ipkg directory on the Zaurus.
2. Select the Add/Remove Software item from the Settings tab.
3. Click on Install Packages. You will see HelloZaurus listed. Click on it to install it.
4. Exit from the Package Installer and exit from Add/Remove Software.

Running the application is now a simple matter of choosing the Games tab and tapping on the HelloZaurus icon.

A Perl Hacker's Foray into .NET

What Is .NET?

When something's as incredibly hyped as Microsoft's .NET project, it's hard to convince people that there's a real working technology underneath it. Unfortunately, Microsoft doesn't do itself any favors by slapping the .NET moniker on anything they can. So let's clarify what we're talking about.

.NET is applied to anything with the broad notion of "Web services" -- from the Passport and Hailstorm automated privacy-deprivation services and the Web-service-enabled versions of operating systems and application products to the C# language and the Common Language Runtime. But there is an underlying theme and it goes like this: The .NET Framework is an environment based on the Common Language Runtime and (to some extent) the C# language, for creating portable Web services.

C# Essentials, 2nd Edition By Ben Albahari, Peter Drayton, Brad Merrill Table of Contents Index Sample Chapter

So for our exploration, the components of the .NET Framework that we care about are the Common Language Runtime and the C# language. And to nail it down beyond any doubt, these are things that you can download and use today. They're real, they exist and they work.

The .NET CLR

Let's begin with the CLR. The CLR is, in essence, a virtual machine for C# much like the Java VM, but which is specifically designed to allow a wide variety of languages other than C# to run on it. Does this ring any bells with Perl programmers? Yes, it's not entirely dissimilar to the idea of the Parrot VM, the host VM for Perl 6 but designed to run other languages as well.

But that's more or less where the similarity ends. For starters, while Parrot is chiefly intended to be ran as an interpreted VM but has a "bolted-on" JIT, CLR is expected to be JITted from the get-go. Microsoft seems to want to avoid the accusations of slowness leveled at Java by effectively requiring JIT compilation.

Another "surface" distinction between Parrot and CLR is that the languages supported by the CLR are primarily statically typed languages such as C#, J#, (a variant of Java) and Visual Basic .NET. The languages Parrot aims to support are primarily dynamically typed, allowing run-time compilation, symbolic variable access, (try doing ${"Package::$var"} in C#...) closures, and other relatively wacky operations.

To address these sorts of features, the Project 7 research project was set up to provide .NET ports for a variety of "academic" languages. Unfortunately, it transpires that this has highlighted some limitations of the CLR, and so almost all of the implementations have had to modify their target languages slightly or drop difficult features. For instance, the work on Mercury turned up some deficiencies in CLR's Common Type System that would also affect a Perl implementation. We'll discuss these deficiencies later when we examine how Perl and the .NET Framework can interact.

But on the other hand, let's not let this detract from what the CLR is good at - it can run a variety of different languages relatively efficiently, and it can share data between languages. Let's now take a look at C#, the native language of the CLR, and then see how we can run .NET executables on our favourite free operating systems.

C#

C# is Microsoft's new language for the .NET Framework. It shares some features with Java, and in fact looks extremely like Java at first glance. Here's a piece of C# code:

using System;

class App {
  public static void Main(string[] args) {
     Console.WriteLine("Hello World");
     foreach (String s in args) {
        Console.WriteLine("Command-line argument: " + s);
     }
  }
} 

Naturally, the Java-like features are quite obvious to anyone who's seen much Java - everything's in a class, and there's an explicitly defined Main function. But what's this - a Perl-like foreach loop. And that using declaration seems strangely familiar.

Now, don't get me wrong. I'm not trying to claim that C# is some bastard offspring of Perl and Java, or even that C# really has that much in common with Perl; it doesn't. But it is a well-designed language that does have a bunch of "programmer-friendly" language features that traditionally made "scripting" languages like Perl or Python faster for rapid code prototyping.

Here's some more code, which forms part of a game-of-life benchmarking tool we used to benchmark the CLR against Parrot.

   static String generate(String input) {
       int cell, neighbours;
       int len = input.Length;
       String output = "";
       cell = 0;
       do {
           neighbours = 0;
           foreach (int offset in new Int32[] {-16, -15, -14, -1, 1, 14, 15, 16}) {
               int pos = (offset + len + cell) % len;
               if (input.Substring(pos, 1) == "*")
                   neighbours++;
           }
           if (input.Substring(cell, 1) == "*") {
               output += (neighbours <> 3) ? " " : "*";
           } else {
               output += (neighbours == 3) ? "*" : " ";
           }
       } while (++cell < len);
       return output;
   } 

This runs one generation of the game of life, taking an input playing field and building an output string. What's remarkable about this is that I wrote it after a day of looking at C# code, with no prior exposure to Java. C# is certainly easy to pick up.

What can Perl learn from C#? That's an interesting question, especially as the Perl 6 design project is ongoing. Let's have a a quick look at some of the innovations in C# and how we might apply them to Perl.

Strong Names

We'll start with an easy one, since Larry has already said that something like this will already be in Perl 6: To avoid versioning clashes and interface incompatibilities, .NET has the concept of "strong names." Assemblies -- the C# equivalent of Java's jar files -- have metadata containing their name, version number, md5sum and cryptographic signature, meaning you can be sure you're always going to get the definitions and behavior you'd expect from any third-party code you run. More generally, assemblies support arbitrary metadata that you can use to annotate their contents.

This approach to versioning and metadata in Perl 6 was highlighted in Larry's State of the Onion talk this year, and is also the solution used by JavaScript 2.0, as described by Waldemar Horwat at his LL1 presentation, so it seems to be the way the language world is going.

Properties

C# supports properties, which are class fields with explicit get/set methods. This is slightly akin to Perl's tying, but much, much slicker. Here's an example:

   private int MyInt;
   public int SomeInt {
       get {
           Console.WriteLine("I was got.\n");
           return MyInt;
       }
       set {
           Console.WriteLine("I was set.\n");
           MyInt = value;
       }
   } 

Whenever we access SomeInt, the get accessor is executed, and returns the value of the underlying MyInt variable; when we write to it, the corresponding set accessor is called. Here's one suggested way we could do something similar in Perl 6:

     my $myint;
     our $SomeInt :get(sub{ print "I was got!\n"; $myint })
                  :set(sub{ print "I was set!\n"; $myint = $^a });
   

C# actually takes this idea slightly further, providing "indexers", which are essentially tied arrays:

   private String realString;
   public String substrString[int idx] {
       get {
           return realString.Substring(idx, 1);
       }
       set {
           realString = realString(0, idx) + value + realString(idx+1);
       }
   }

   substrString[12] = "*"; // substr($string, 12, 1) = "*"; 

Object-Value Duality

Within the CLR type system, (CTS) there are two distinct types (as it were) of types: reference types and value types. Value types are the simple, honest-to-God values: integers, floating point numbers, strings, and so on. Reference types, on the other hand, are objects, references, pointers and the like.

Now for the twist: Each value type has an associated reference type, and you can convert values between them. So, if you've got an int counter;, then you can "box" it as an object like so: Object CounterObj = counter. More specifically, int corresponds to Int32. This gives us the flexibility of objects when we need to, for instance, call methods on them, but the speed of fixed values when we're doing tight loops on the stack.

While Perl is and needs to remain an essentially untyped language, optional explicit typing definitions combined with object-value duality could massively up Perl's flexibility as well as bringing some potential optimizations.

Bringing Java into Perl

In this article, I will show how to bring Java code into a Perl program with Inline::Java. I won't probe the internals of Inline or Inline::Java, but I will tell you what you need to make a Java class available in a program or module. The program/module distinction is important only in one small piece of syntax, which I will point out.

The article starts with the Java code to be glued into Perl, then shows several approaches for doing so. First, the code is placed directly into a Perl program. Second, the code is placed into a module used by a program. Finally, the code is accessed via a small Perl proxy in the module.

The Java Code

Consider the following Java class:

   public class Hi {

       String greeting;

       public Hi(String greeting) {
           this.greeting = greeting;
       }

       public void setGreeting(String newGreeting) {
           greeting = newGreeting;
       }

       public String getGreeting() {
           return greeting;
       }
   }

This class is for demonstration only. Each of its objects is nothing but a wrapper for the string passed to the constructor. The only operations are accessors for that one string. Yet with this, we will learn most of what we need to know to use Java from Perl. Later, we will add a few features, to show how arrays are handled. That's not as interesting as it sounds, since Inline::Java almost always does all of the work without help.

A Program

Since we're talking about Perl, there is more than one way to incorporate our trivial Java class into a Perl program. (Vocabulary Note: Some people call Perl programs "scripts." I try not to.) Here, I'll show the most direct approach. Subsequent sections move to more and more indirect approaches, which are more often useful in practice.

Not surprisingly, the most direct approach is the simplest to understand. See if you can follow this:

   #!/usr/bin/perl
   use strict; use warnings;

   use Inline Java => <<'EOJ';
   public class Hi {
       // The class body is shown in the Java Code above
   }
   EOJ

   my $greeter = Hi->new("howdy");
   print $greeter->getGreeting(), "\n";

The Java class is the one above, so I have omitted all but the class declaration. The Perl code just wraps it, so it is tiny. To use Inline::Java, say use Inline Java => code where code tells Inline where to look for the code. In this case, the code follows inline (clever naming, huh?). Note that single-quote context is safest here. There are other ways to include the code; we'll see my favorite way later. The overly curious are welcome to consult the perldoc for all of the others.

Once Inline::Java has worked its magic -- and it is highly magical -- we can use the Java Hi class as if it was a Perl package. Inline::Java provides several ways to construct Java objects. I usually use the one shown here; namely, I pretend the Java constructor is called new, just like many Perl constructors are. In honor of Java, you might rather say my $greeter = new Hi("howdy");, but I usually avoid this indirect object form. You can even call the constructor by the class name as in my $greeter = Hi->Hi("howdy"); (or, you could even say the pathological my $greeter = Hi Hi("howdy");). Class methods are accessed just like the constructor, except that their names are the Java method names. Instance methods are called through an object reference, as if the reference were a Perl object.

Note that Inline::Java performs type conversions for us, so we can pass and receive Java primitive types in the appropriate Perl variables. This carries over to arrays, etc. When you think about what must be going on under the hood, you'll realize what a truly magical module this is.

A Module

I often say that most Perl code begins life in a program. As time goes by, the good parts of that code, the ones that can be reused, are factored out into modules. Suppose our greeter is really popular, so many programs want to use it. We don't want to have to include the Java code in each one (and possibly require each program to compile its own copy of the class file). Hence, we want a module. My module looks a lot like my earlier program, except for two features. First, I changed the way Inline looks for the code, which has nothing to do with whether the code is in a program or a module. Second, reaching class methods from any package other than main requires careful -- though not particularly difficult -- qualification of the method name.

   package Hi;
   use strict; use warnings;

   use Inline Java => "DATA";

   sub new {
       my $class    = shift;
       my $greeting = shift;
       return Hi::Hi->new($greeting);
   }

   1;

   __DATA__
   __Java__
   public class Hi {
       // The class body is shown in The Java Code above
   }

The package starts like all good packages, by using strict and warnings. The use Inline statement is almost like the previous one, but the code lives in the __DATA__ segment instead of actually being inline. Note that when you put the code in the __DATA__ segment, you must include a marker for your language so that Inline can find it. There are usually several choices for each language's marker; I chose __Java__. This allows Inline to glue from multiple languages into one source file.

The constructor is needed so that the caller does not need to know they are interfacing with Inline::Java. They call the constructor with Hi->new("greeting") as they would for a typical package called Hi. Yet, the module's constructor must do a bit of work to get the right object for the caller. It starts by retrieving the arguments, then returns the result of the unusual call Hi::Hi->new(...). The first Hi is for the Perl package and the second is for the Java class; both are required. Just as in the program from the last section, there are multiple ways to call the constructor. I chose the direct method with the name new. You could use the indirect object form and/or call the method by the class name. The returned object can be used as normal, so I just pass it back to the caller. All instance methods are passed directly through Inline::Java without help from Hi.pm. If there were class methods (declared with the static keyword in Java), I would either have to provide a wrapper, or the caller would have to qualify the names. Neither solution is particularly difficult, but I favor the wrapper, to keep the caller's effort to a minimum. This is my typical laziness at work. Since there will likely be several callers, and I will have to write them, I want to push any difficult parts into the module.

If you need to adapt the behavior of the Java object for your Perl audience, you may insert routines in Hi.pm to do that. For instance, perhaps you want a more typical Perl accessor, instead of the get/set pair used in the Java code. In this case, you must make your own genuine Perl object and proxy through it to the Java class. That might look something like this:

   package Hi2;
   use strict; use warnings;

   use Inline Java => "DATA";

   sub new {
       my $class    = shift;
       my $greeting = shift;
       bless { OBJECT => Hi2::Hi->new($greeting) }, $class;
   }

   sub greeting {
       my $self      = shift;
       my $new_value = shift;
       if (defined $new_value) {
           $self->{OBJECT}->setGreeting($new_value);
       }
       return $self->{OBJECT}->getGreeting();
   }

   1;

   __DATA__
   __Java__
   public class Hi {
       // Body omitted again
   }

Here, the object returned from Inline::Java, which I'll call the Java object for short, is stored in the OBJECT key of a hash-based Hi2 object that is returned to the caller. The distinction between the Perl package and the Java class is clear in this constructor call. The Perl package comes first, then the Java class, then the class method to call.

The greeting method, shifts in the $new_value, which the caller supplies if she wants to change the value. If $new_value is defined, greeting passes the set message to the Java object. In either case, it returns the current value to the caller, as Perl accessors usually do.

A Pure Proxy

In the last section, we saw how to make a Perl module access Java code. We also saw how to make the Perl module adapt between the caller's expectation of Perl objects and the underlying Java objects. Here, we will see how to access Java classes that can't be included in the Perl code.

There are a lot of Java libraries. These are usually distributed in compiled form in so-called .jar (java archive) files. This is good design on the part of the Java community, just as using modules is good design on the part of the Perl community. Just as we wanted to make the Hi Java class available to lots of programs -- and thus placed it in a module -- so the Java people put reusable code in .jars. (Yes, Java people share the bad pun heritage of the Unix people, which brought us names like yacc, bison, more, and less.)

Suppose that our humble greeter is so popular that it has been greatly expanded and .jarred for worldwide use. Unless we provide an adapter like the one shown earlier, the caller must use the .jarred code from Perl in a Java-like way. So I will now show three pieces of code: 1) an expanded greeter, 2) a Perl driver that uses it, and 3) a mildly adapting Perl module the driver can use.

Here's the expanded greeter; the two Perl pieces follow later:

   import java.util.Random;
   public class Higher {
       private static Random myRand = new Random();
       private String[] greetings;

       public Higher(String[] greetings) {
           this.greetings = greetings;
       }

       public void setGreetings(String[] newGreetings) {
           greetings = newGreetings;
       }

       public String[] getGreetings() {
           return greetings;
       }

       public void setGreeting(int index, String newGreeting) {
           greetings[index] = newGreeting;
       }

       public String getGreeting() {
           float randRet = myRand.nextFloat();
           int   index   = (int) (randRet * greetings.length);
           return greetings[index];
       }
   }

Now there are multiple greetings, so the constructor takes an array of Strings. There are get/set pairs for the whole list of greetings and for single greetings. The single get accessor returns one greeting at random. The single set accessor takes the index of the greeting to replace and its new value.

Note that Java arrays are fixed-size; don't let Inline::Java fool you into thinking otherwise. It is very good at making you think Java works just like Perl, even though this is not the case. Calling setGreeting with an out-of-bounds index will be fatal unless trapped. Yes, you can trap Java exceptions with eval and the $@ variable.

This driver uses the newly expanded greeter through Hi3.pm:

   #!/usr/bin/perl
   use strict; use warnings;

   use Hi3;

   my $greeter = Hi3->new(["Hello", "Bonjour", "Hey Y'all", "G'Day"]);
   print $greeter->getGreeting(), "\n";
         $greeter->setGreeting(0, "Howdy");
   print $greeter->getGreeting(), "\n";

The Hi3 module (directly below) provides access to the Java code. I called the constructor with an anonymous array. An array reference also works, but a simple list does not. The constructor returns a Java object (at least, it looks that way to us); the other calls just provide additional examples. Note, in particular, that setGreeting expects an int and a String. Inline::Java examines the arguments and coerces them into the best types it can. This nearly always works as expected. When it doesn't, you need to look in the documentation for "CASTING."

Finally, this is Hi3.pm (behold the power of Perl and the work of the Inline developers):

   package Hi3;
   use strict; use warnings;

   BEGIN {
       $ENV{CLASSPATH} .= ":/home/phil/jar_home/higher.jar";
   }
   use Inline Java  => 'STUDY',
              STUDY => ['Higher'];

   sub new {
       my $class = shift;
       return Hi3::Higher->new(@_);
   }

   1;

To use a class hidden in a .jar I need to do three things:

1. Make sure an absolute path to the .jar file is in the CLASSPATH, before using Inline. A well-placed BEGIN block makes this happen.
2. Use STUDY instead of providing Java source code.
3. Add the STUDY directive to the use Inline statement. This tells Inline::Java to look for named classes. In this case, the list has only one element: Higher. Names in this list must be fully qualified if the corresponding class has a Java package.

The constructor just calls the Higher constructor through Inline::Java, as we have seen before.

Yes, this is the whole module, all 15 lines of it.

If you need an adapter between your caller and the Java library, you can put it in either Perl or Java code. I prefer to code such adapters in Perl when possible, following the plan we saw in the previous section. Yet occasionally, that is too painful, and I resort to Java. For example, the glue module Java::Build::JVM uses both a Java and a Perl adapter to ease communication with the genuine javac compiler. Look at the Java::Build distribution from CPAN for details.

Anatomy of Automated Compiling: A Brief Discussion

So what is Inline::Java doing for us? When it finds our Java code, it makes a copy in the .java file of the proper name (javac is adamant that class names and file names match). Then it uses our Java compiler to build a compiled version of the program. It puts that version in a directory, using an MD5 sum to ensure that recompiling happens when and only when the code changes.

You can cruise through the directories looking at what it did. If something goes wrong, it will even give you hints about where to look. Here's a tour of some of those directories. First, there is a base directory. If you don't do anything special, it will be called _Inline, under the working directory from which you launched the program. If you have a .Inline directory in your home directory, all Inline modules will use it. If you use the DIRECTORY directive in your use Inline statement, its value will be used instead. For ease of discussion, I'll call the directory _Inline.

Under _Inline is a config file that describes the various Inline languages available to you. More importantly, there are two subdirectories: build and lib. If your code compiles, the build directory will be cleaned. (That's the default behavior; you can include directives in your use Inline statement to control this.) If not, the build directory has a subdirectory for your program, with part of the MD5 sum in its name. That directory will hold the code in its .java file and the error output from javac in cmd.out.

Code that successfully compiles ends up in lib/auto. The actual .class files end up in a subdirectory, which is again named by class and MD5 sum. Typically, there will be three files there. The .class file is as normal. The other files describe the class. The .inl file has an Inline description of the class. It contains the full MD5 sum, so code does not need to be recompiled unless it changes. It also says when the code was compiled, along with a lot of other information about the Inline::Java currently installed. The .jdat file is specific to Inline::Java. It lists the signatures of the methods available in the class. Inline::Java finds these using Java's reflection system (reflection is the Java term for symbolic references).

Open Source Music Software & The AGNULA Project

If you use a computer at all in your recording setup, you'll know that the performance and stability of machines running general-purpose operating systems hasn't yet come close to that of dedicated studio hardware. That dedicated hardware might be like a computer inside, but it runs an 'embedded' operating system that is tailored for the purpose, with rock-solid reliability.

Most of the desktop computers in the world run a derivative of Windows 95 or the Mac's System 7, and it's a fact that the manufacturers of both know that their time is up. In the case of PC users, Microsoft would like to see their customers move to the NT-based Windows XP, while Apple has also abandoned its existing codebase with the launch of the UNIX-based OS X.

In the network era, computers that crash, lose data and work inconsistently are no longer acceptable. As more and more people have come to depend on computers for earning a living, a migration of some sort from the legacy operating systems becomes essential. Meanwhile, the computing power available to individuals continues to grow at rapid rate, meaning that off-the-shelf hardware can now process multiple tracks of digital audio in real time, something that not even the most well-equipped studio could do a generation ago.

If Linux Is So Good, How Come It's Free?
There are two traditions in software development: the work done in universities and by individual developers, often shared among research teams, and the proprietary code created by commercial entities which is usually a trade secret. Each tradition has its own ethos, and licenses the software it creates in very different ways.

The UNIX family, which includes Mac OS X and Linux, has long been the mainstay of university computer departments, but the original UNIX concept has fragmented into many incompatible proprietary versions. Because Linux belongs to everyone and no-one, it is now being invested in by many of the big UNIX companies, including IBM, Sun, SGI and HP. By contrast, Mac OS X belongs to Apple, so we are unlikely to see other companies contributing significantly to it.

It's a common misconception that Linux was created by a lone student in Finland. In fact Linus Torvalds still leads the project to this day, more than 10 years after he began it, but the operating system has been contributed to by thousands of people. It builds on the considerable work of developers who had dreamt for many years of making a free UNIX, many of whom worked on the GNU project.

The sense in which 'free' is used by the GNU project is not the same as in 'freeware': it refers to developer and user freedom rather than zero cost. Confusingly, Free Software can be sold, while software that can be downloaded at no cost often places restrictions on user freedom. But there are plenty of complete Linux systems that you can download for free, or buy on CD-ROMs for near the cost of production.

The familiar software licence that comes with a Windows or Mac application only allows you to use that software in specific ways, and non-compliance with the conditions of the licence can even lead to prosecution. Usually the software you get is a binary — an executable file made of ones and zeros that isn't human-readable. It's like a sealed black box, and if anything goes wrong or you don't like a certain feature, there's not much you can do about it except complain to the vendor. The vendor's response can only be that you should upgrade the binary, since the source code from which the program is made is a trade secret.

With 'open source' software, the source code of the program is made available for those who need it. This not only helps the software developers, who can fix problems directly, but also those users who can hire a developer to sort the program out. There are usually fewer restrictions on binaries too — for example, you can often install open source software on multiple machines without infringing the licence.

The next step in the process is what separates Mac OS X from Linux. OS X is based on a very liberally licensed codebase known as BSD, which allows developers who make fixes to keep the improvements to themselves — if they want to. This is one of the reasons why you can't just download OS X for free, even though it's based on freely available software.

The licence under which Linux and the majority of software for it is released is known as the GNU GPL, or General Public Licence. It insists that any improvements to GPL-licenced source code have to be made available under the same terms, and generally patches are sent back to the original developer. This continual feedback process has lead to the refinement of most of the software that runs the Internet — the Apache web server, for example — as well as Linux itself.

Of course, there is nothing to stop programs from the proprietary tradition being made available for Linux. As long as those programs don't make secret changes to Linux itself, then they are perfectly acceptable to most users and developers. The message that anyone porting their program to Linux will be forced to make it Free Software has been put about by the likes of Microsoft — but this is clearly not the case. As one spokesperson for the company put it, IBM has a lot of intellectual property and a lot of lawyers, and they aren't worried about the possibility.

For more information on Free Software, see www.gnu.org.


The Linux Advantage

At the pinnacle of creative power computing, Hollywood studios are squeezing every processor cycle out of the fastest hardware currently available to design and render ever-greater special effects scenes. If these computers are capable of creating the graphics used in The Two Towers, the second film in Lord Of The Rings trilogy for example, then they should have the power and stability to be able to handle any sound recording job we can throw at them. The suprise is that the film studios are not all using super-expensive hardware and esoteric operating systems to perform this mission-critical work. Increasingly, they are using off-the-shelf hardware that is not fundamentally different from today's desktop computers, and they are running the freely available Linux operating system.

One obvious advantage of Linux is lower acquisition cost, and while this is definitely a factor for sites with large numbers of computers and limited budgets (for example schools), it's not actually the major reason for most migrations. In the case of a multi-million dollar movie studio, it's unlikely to be the price of Linux alone that makes it attractive. These studios can afford to buy any system they want, so why are they choosing Linux? The answer is in part the lower cost of Intel processor hardware compared to the traditional SGI UNIX platform, and the fact that it's much easier to move programs from SGI to Linux than from SGI to Windows.

But a far more compelling reason for the migration is the quality and flexibility of Linux. Machines powered by Linux have been known to run for months or even years without needing a reboot, let alone crashing. In the UNIX world that's not unusual, but that kind of reliability hasn't been seen on desktop computers before. And the open source development model means that users can get the software they want, rather than just choose from what's on offer.

Another factor often cited by people who have migrated to Linux is the supportive and knowledgeable user community. If you have a problem with your Linux machine, there are lots of places to ask for help — both with local user groups and on the Internet. Linux users tend to be self-documenting: when they find the solution to a problem, they will often create a web page describing the fix to share their knowledge.

Ardour is a multitrack hard disk recorder which offers an interface modelled on hardware units such as the Tascam MX2424, and also boasts DAW-style on-screen editing
People who choose Linux also appreciate the benefit of open standards. This makes vendor lock-in — the phenomenon where customers of a particular computer company are prevented from having a free choice — much more difficult. Many Apple and Microsoft users across the creative industries know exactly how this feels, when they find their projects can't easily be moved to a different platform. Open standards are particularly useful to the creators of embedded systems, as they mean manufacturers don't have to start from scratch every time they invent a new digital device. Linux can be customised for embedded systems, which often provide a highly resource-constrained environment, to the extent that Linux has already been successfully installed by IBM Research on a prototype digital watch.

Could the recording industry be the next creative sector to adopt Linux? Mac hardware is firmly entrenched in professional studios for the time being, but due to the open source nature of Linux, the free OS is available for many different processors, including the PowerPC architecture of the G3 and G4 chips. While most Linux users have Intel Pentium or AMD Athlon-based computers, several companies already make packaged versions of Linux for PowerPC machines, including Mandrake, SuSE and Yellow Dog. This makes the PC/Mac debate somewhat irrelevant in the Linux world. Because it's part of the UNIX family, Linux could be the closest thing to Mac OS X that someone who currently has a Windows machine could get.

The Stanford University Centre for Computer Research in Music and Acoustics (CCRMA) already produces pre-packaged audio applications and system components for Linux, as part of its Planet CCRMA project. However, this isn't much help to people working in the sound industry with little or no UNIX experience. Although both Linux and Mac OS X hide UNIX quite well under some very elegant graphical interfaces, there is no doubt that UNIX can be confusing to someone from a Windows or traditional Mac background. It's probably no more difficult than moving from Mac to Windows or vice versa, but nevertheless an element of retraining will be required when migrating to any UNIX-based operating system. With a different version of Windows or Mac OS coming out every couple of years, though, people will have to retrain anyway. And the Linux desktop can be themed to resemble Windows or Mac OS for a smoother transition.

AGNULA

The AGNULA project, the name of which is an acronym for A GNU/Linux Audio Distribution, has been created to design and build aversion of Linux specifically for professional musicians and recording engineers. AGNULA is a consortium of several European universities, the Red Hat Linux company and the Free Software Foundation. The idea is that all the software needed for professional audio use will be on one set of CD-ROMs, which will include a tuned Linux operating system. This doesn't mean that it just comes with an extra driver or two and a few tweaked settings here and there. While Windows and Mac OS will always remain general-purpose systems, the open source philosophy means that Linux can be customised for individual requirements at the most fundamental level.

As far as the musician and pro audio user is concerned, there are four key components of a Linux system. The kernel — the actual Linux core — can be customised for very low latency, which is essential for any kind of multitracking or synchronisation. The Advanced Linux Sound Architecture or ALSA provides drivers for pro and consumer soundcards (see table on the last page of this article). JACK is the internal audio system which connects between applications at a low level, while LADSPA stands for the Linux Audio Developers Simple Plug-in API. LADSPA is a bit like Steinberg's VST plug-in system, but in an open source style — the many VST plug-ins available to download from the Internet at no cost don't generally include source code.

The individual components of AGNULA are already available, but the project is creating two integrated packages based on either the Red Hat Linux or Debian distributions. Each distribution (or version) of Linux has its followers: the Red Hat company is more business-orientated, but Debian is built by volunteers and has a reputation for very high quality. While each distribution has a slightly different way of doing things, they remain broadly compatible. Published standards and source code availability mean that a program created on one distribution is usually available to all the others.

Beta versions of the two AGNULA distributions are due shortly, with stable releases expected in 2003. In the Linux community, the widest possible testing of beta versions is encouraged to enable high standards of quality control. With modern hard disks being so large, there is usually plenty of room to install Linux in its own partition, and keep the existing operating system intact. Disk partitioning is well worth doing for Windows machines anyway, since it keeps your sound data well away from the operating system, and therefore safe if you ever have to reformat your C drive. Most current Linux distributions feature a wizard to help you set up partitions and dual-booting, so it's likely that AGNULA will too.

Since AGNULA will cost very little to try out, except perhaps in the time taken to learn about the software, there's not much to lose. Both AGNULA distributions will be free for download, intially built for the Intel x86 architecture (Pentium and Athlon), and will be probably be available on CD-ROM too. The project also plans to make the distributions available for PowerPC chips and the new generation of 64-bit processor systems later next year.

Audacity is another Linux-based audio recording and editing package.
Applications

The drawback to any new desktop operating system is usually a lack of applications. Fortunately, although Linux is relatively new to the sound industry, it has already been under development for a decade. While Windows and Mac applications can run under Linux on the relevant processor using various kinds of emulator and virtual machine, lots of native Linux audio software is already available. The full range of programs available on other platforms is covered, from notation editors to DJ software. Many excellent free music applications for Linux are stable or close to stable release, and will be included with AGNULA. Here are just a few of them:
Rosegarden is a MIDI + Audio sequencer which includes notation and audio editing. Version 4 is probably the closest native equivalent to Cubase for Linux, and has recently been released as a beta after two years of active development. Unlike Windows and Mac OS where there is only one kind of desktop for each system, Linux developers have a wide choice of graphical toolkits to build applications from. Rosegarden is designed with the KDE interface, but it will run on any Linux machine with the right libraries installed. Rosegarden features include MIDI and audio playback and recording using ALSA and JACK, real-time audio plug-in effects via LADSPA, score, piano-roll and track overview editors, high-quality score printing and MIDI file input/output.

Software engineer and musician Richard Bown comes from London, and is one of the lead developers on the Rosegarden project. He's been working on Rosegarden over the last seven years, but until recently he still needed to use Windows or a Mac to record and produce his music. While making his last album, Richard wondered if Rosegarden would one day allow him to do the same on Linux, and now believes it can.

When Rosegarden was originally written for UNIX machines about 10 years ago, Linux was just beginning to emerge, and it seemed a natural fit. Since there were already plenty of good sequencers and notation editors available for the Mac and Windows, the Rosegarden team concentrated on making something new. "We're now establishing Rosegarden as a product in its own right and hope to make it something that will make music people start to think about Linux more seriously," comments Richard. "If studios or individuals want a boxed solution they can approach us and we can put it together for them, but it's true that anyone can package and distribute our software. AGNULA doesn't need our permission and we don't get any explicit kickback apart from a little publicity. As a small company we couldn't afford to develop, market and distribute a closed-source solution. As an open-source project we do sign away our rights to saying what people do with the source code but we do gain testers, developers, marketers, friends and a warm feeling along the way."

Sweep is a multitrack audio editor with a difference: it can also be used for virtual scratching.
Richard adds "The warm glow bit is important to me. While I'd like to be paid to write this code, I'm also proud to be a part of it for what it is. The fact that Rosegarden is now becoming a quality solution to rival those on other platforms is a great bonus."
Ardour is a multi-channel hard disk recorder and digital audio workstation, capable of the simultaneous recording of 24 or more channels of 32-bit audio at the 48kHz sample rate. Currently in heavy development, Ardour visually resembles the UNIX software available on platforms such as SGI. Linux machines running Ardour are intended to replace dedicated studio hardware such as the Mackie HDR, the Tascam 2424 and ADAT systems. Ardour is also intended to rival proprietary software applications such as Pro Tools, Samplitude, Logic Audio, Nuendo and Cubase VST. It supports MIDI Machine Control, and can therefore be used with any MMC-compliant digital mixer.
Audacity is a deceptively simple audio editor and multitrack hard disk recorder. It has a clean interface with large buttons in the style of a tape machine, but very precise edits are possible by drawing envelopes directly on the waveform using the mouse. Audacity breaks large audio files into small chunks for its native file format, which makes multitrack recording and editing quite feasible on modest hardware.

Many Free Software effects plug-ins are available under the LADSPA standard.
An added bonus with Audacity is that the program is available not just for Linux, but for Mac and Windows too. This means that mixed-computer environments can have a standard free format for multitrack recordings. Version 1.0.0 is the current stable release, while the development version 1.1.0 contains many new features, including support for 24-bit and 32-bit samples with automatic real-time resampling. The new version also includes LADSPA plug-in capability, an XML-based project format and full Ogg Vorbis (a licence-free alternative to MP3) import/export.
Sweep, at first glance, is a conventional multi-channel audio file editor. However, a virtual stylus instead of the normal cursor makes the program quite unique, on Linux at least. The idea is that you can scrub through a file to hear the exact place where you want to make an edit — but the virtual stylus, known as Scrubby, has been programmed with the physics of a real turntable. Throwing the mouse to the left results in a spin-back effect, decelerating Scrubby to a full stop.

This makes Sweep possibly the first audio program that could be used both in a serious production environment and as a performance tool for fully digital DJs. It was developed with the support of animation studio Pixar, who presumably needed a high-quality audio editor which could be used on Linux and Sun UNIX workstations (Sweep is available freely for both). This application might be the first benefit to trickle down from the big movie studios to the Linux audio community, but it almost certainly won't be the last.

Plug-ins Under Linux

Steve Harris wrote many of the LADSPA effects plug-ins. He works at Southampton University and started writing experimental electronic music about 10 years ago. Steve takes up the story: "In order to expand the palette of sounds available to me I started writing simple music programs, but the computers I had then weren't fast enough to work in real time. I only learnt real signal processing skills a few years ago, when Linux started to become a useful audio system and I noticed the lack of plug-ins. I chose LADSPA mostly because I liked the design, and there are some licensing issues which make writing Free Software VST plug-ins difficult."

Will My Soundcard Work With Linux?
This is just a small selection of soundcards and other hardware supported by ALSA. Some drivers have been available in stable versions for a long time, while others are still in development.

Mark Of The Unicorn
Micro Express.
Midi Express XT.
Midi Time Piece AV.

M Audio
DMAN PCI.
Delta 44, 66, 410, 1010 and 1010LT.
Audiophile 2496.
Delta DiO 2448 and 2496.
USB Audio Duo, Quattro and Omni Studio.
USB Keystations.
USB Midisport 1x1, 2x2, 4x4 and 8x8.

RME
Digi32, Digi32/8 and Digi32 Pro.
Hammerfall, Hammerfall Light and Hammerfall DSP.
Digi96, Digi96/8, Digi96/8 PRO, PST and PAD.

Roland/Edirol
PC300.
SC8820 and SC8850.
SCD70.
SD20, SD80 and SD90.
SK500.
U8.
UA1A, UA100, UA100G and UA700.
UM1, UM1S, UM2(E), UM4/Super MPU64, UM550 and UM880.
XV5050.

Turtle Beach
Daytona.
Malibu.
Montego II.
Tropez and Tropez Plus.

Most USB devices are supported, as long as they are standards-compliant. For a full list, see www.alsa-project.org.

Steve gives his plug-ins away for free, including the source code — and he isn't worried about being ripped off. "I encourage people to reuse my work. Many of the core routines used in DSP programming are difficult to get right and very tedious to write, so it makes sense that they should only have to be written once. I write audio software so that people will have access to high-quality, free (in both senses of the word) audio tools. It doesn't make sense to go out of my way to deliberately hinder music software development by hiding the source code."

Where's The Catch?

So is there any reason why we shouldn't all migrate to AGNULA and Linux tomorrow? Well, if a studio has a large capital investment in proprietary systems, some of this won't be able to be carried over to a Linux platform. An example would be a large collection of plug-ins, or application software that won't run adequately in a virtual machine or emulator. However, this is often outweighed by the cost of continual upgrades of the existing system — and upgrades will almost certainly be forced on both Windows and Mac users eventually.

A greater problem could be back-catalogue work stored in proprietary formats. Most of the audio formats from Windows, Mac and UNIX are supported by the equivalent Linux programs, but complex projects combining multitrack audio and MIDI could be a problem. If the original software vendor supported open standards, it wouldn't be difficult to create a tool to transfer the project from one platform to another. Where that file format is binary and a trade secret, however, the user may have no choice but to fall back to standard file types for exporting projects, and some of the information might be lost.

This is also a problem when moving complex projects between proprietary programs on the same platform, of course. It could be argued that since binary and secret formats keep the user hostage to their creator, they are best left behind anyway. What if you need to get that project out of the archive for a remix in 20 years' time, and the company who made the software no longer exists? Unfortunately, this is not a hypothetical situation for many institutions who have data going back to the 1970s on unreadable open-reel tapes.

It may be early days for Linux desktop audio applications, and both proprietary embedded hardware and software based on trade secrets aren't going to disappear from studios overnight. A more likely scenario is that Linux machines will be introduced to perform specific tasks, and gradually increase their profile in the world of sound. Companies in the industry are almost certainly using Linux, perhaps without realising, in their computer networks already — a record label with a busy web site, for example, or a studio saving its work on an embedded file storage appliance that doesn't even look like a computer. Even if you've got no time to try it out at the moment, Linux is well worth keeping an eye on

An A-Z Index of the Linux BASH command line

alias Create an alias
apropos Search Help manual pages (man -k)
awk Find and Replace text, database sort/validate/index
break Exit from a loop
builtin Run a shell builtin
bzip2 Compress or decompress named file(s)

cal Display a calendar
case Conditionally perform a command
cat Display the contents of a file
cd Change Directory
cfdisk Partition table manipulator for Linux
chgrp Change group ownership
chmod Change access permissions
chown Change file owner and group
chroot Run a command with a different root directory
cksum Print CRC checksum and byte counts
clear Clear terminal screen
cmp Compare two files
comm Compare two sorted files line by line
command Run a command - ignoring shell functions
continue Resume the next iteration of a loop
cp Copy one or more files to another location
cron Daemon to execute scheduled commands
crontab Schedule a command to run at a later time
csplit Split a file into context-determined pieces
cut Divide a file into several parts

date Display or change the date & time
dc Desk Calculator
dd Data Dump - Convert and copy a file
declare Declare variables and give them attributes
df Display free disk space
diff Display the differences between two files
diff3 Show differences among three files
dig DNS lookup
dir Briefly list directory contents
dircolors Colour setup for `ls'
dirname Convert a full pathname to just a path
dirs Display list of remembered directories
du Estimate file space usage

echo Display message on screen
egrep Search file(s) for lines that match an extended expression
eject Eject removable media
enable Enable and disable builtin shell commands
env Environment variables
ethtool Ethernet card settings
eval Evaluate several commands/arguments
exec Execute a command
exit Exit the shell
expand Convert tabs to spaces
export Set an environment variable
expr Evaluate expressions

false Do nothing, unsuccessfully
fdformat Low-level format a floppy disk
fdisk Partition table manipulator for Linux
fgrep Search file(s) for lines that match a fixed string
file Determine file type
find Search for files that meet a desired criteria
fmt Reformat paragraph text
fold Wrap text to fit a specified width.
for Expand words, and execute commands
format Format disks or tapes
free Display memory usage
fsck File system consistency check and repair
ftp File Transfer Protocol
function Define Function Macros

gawk Find and Replace text within file(s)
getopts Parse positional parameters
grep Search file(s) for lines that match a given pattern
groups Print group names a user is in
gzip Compress or decompress named file(s)

hash Remember the full pathname of a name argument
head Output the first part of file(s)
history Command History
hostname Print or set system name

id Print user and group id's
if Conditionally perform a command
ifconfig Configure a network interface
import Capture an X server screen and save the image to file
install Copy files and set attributes

join Join lines on a common field

kill Stop a process from running

less Display output one screen at a time
let Perform arithmetic on shell variables
ln Make links between files
local Create variables
locate Find files
logname Print current login name
logout Exit a login shell
look Display lines beginning with a given string
lpc Line printer control program
lpr Off line print
lprint Print a file
lprintd Abort a print job
lprintq List the print queue
lprm Remove jobs from the print queue
ls List information about file(s)
lsof List open files

make Recompile a group of programs
man Help manual
mkdir Create new folder(s)
mkfifo Make FIFOs (named pipes)
mkisofs Create an hybrid ISO9660/JOLIET/HFS filesystem
mknod Make block or character special files
more Display output one screen at a time
mount Mount a file system
mtools Manipulate MS-DOS files
mv Move or rename files or directories

netstat Networking information
nice Set the priority of a command or job
nl Number lines and write files
nohup Run a command immune to hangups
nslookup Query Internet name servers interactively

passwd Modify a user password
paste Merge lines of files
pathchk Check file name portability
ping Test a network connection
popd Restore the previous value of the current directory
pr Prepare files for printing
printcap Printer capability database
printenv Print environment variables
printf Format and print data
ps Process status
pushd Save and then change the current directory
pwd Print Working Directory

quota Display disk usage and limits
quotacheck Scan a file system for disk usage
quotactl Set disk quotas

ram ram disk device
rcp Copy files between two machines.
read read a line from standard input
readonly Mark variables/functions as readonly
remsync Synchronize remote files via email
return Exit a shell function
rm Remove files
rmdir Remove folder(s)
rsync Remote file copy (Synchronize file trees)

screen Terminal window manager
scp Secure copy (remote file copy)
sdiff Merge two files interactively
sed Stream Editor
select Accept keyboard input
seq Print numeric sequences
set Manipulate shell variables and functions
sftp Secure File Transfer Program
shift Shift positional parameters
shopt Shell Options
shutdown Shutdown or restart linux
sleep Delay for a specified time
sort Sort text files
source Run commands from a file `.'
split Split a file into fixed-size pieces
ssh Secure Shell client (remote login program)
strace Trace system calls and signals
su Substitute user identity
sum Print a checksum for a file
symlink Make a new name for a file
sync Synchronize data on disk with memory

tail Output the last part of files
tar Tape ARchiver
tee Redirect output to multiple files
test Evaluate a conditional expression
time Measure Program running time
times User and system times
touch Change file timestamps
top List processes running on the system
traceroute Trace Route to Host
trap Run a command when a signal is set(bourne)
tr Translate, squeeze, and/or delete characters
true Do nothing, successfully
tsort Topological sort
tty Print filename of terminal on stdin
type Describe a command

ulimit Limit user resources
umask Users file creation mask
umount Unmount a device
unalias Remove an alias
uname Print system information
unexpand Convert spaces to tabs
uniq Uniquify files
units Convert units from one scale to another
unset Remove variable or function names
unshar Unpack shell archive scripts
until Execute commands (until error)
useradd Create new user account
usermod Modify user account
users List users currently logged in
uuencode Encode a binary file
uudecode Decode a file created by uuencode

v Verbosely list directory contents (`ls -l -b')
vdir Verbosely list directory contents (`ls -l -b')
vi Text Editor

watch Execute/display a program periodically
wc Print byte, word, and line counts
whereis Report all known instances of a command
which Locate a program file in the user's path.
while Execute commands
who Print all usernames currently logged in
whoami Print the current user id and name (`id -un')
Wget Retrieve web pages or files via HTTP, HTTPS or FTP

xargs Execute utility, passing constructed argument list(s)
yes Print a string until interrupted

.period Run commands from a file
### Comment / Remark

LAMP (software bundle)

The acronym LAMP refers to a solution stack of software programs, commonly open source programs, used together to run dynamic Web sites or servers. The original expansion is as follows:

* Linux, referring to the operating system;
* Apache, the Web server;
* MySQL, the database management system (or database server);
* PHP, the programming language.

The combination of these technologies is used primarily to define a web server infrastructure, define a programming paradigm of developing software, and establish a software distribution package. More recently, the P has come to refer frequently to Perl or Python as alternate programming languages. See Variants, below.

Though the originators of these open source programs did not design them all to work specifically with each other, the combination has become popular because of its low acquisition cost and because of the ubiquity of its components (which come bundled with most current Linux distributions). When used in combination they represent a solution stack of technologies that support application servers. Other such stacks include unified application development environments such as Apple Computer's WebObjects, Java/Java EE, Grails, and Microsoft's .NET architecture.

The scripting component of the LAMP stack has its origins in the CGI web interfaces that became popular in the early 1990s. This technology allows the user of a web browser to execute a program on the web server, and to thereby receive dynamic as well as static content. Programmers used scripting languages with these programs because of their ability to manipulate text streams easily and efficiently, even when they originate from disparate sources. For this reason system designers often referred to such scripting systems as glue languages.

Michael Kunze coined the acronym LAMP in an article for the German computing magazine c't in 1998 (12/98, page 230). The article aimed to show that a bundle of free software could provide a viable alternative to commercial packages. Knowing about the IT-world's love of acronyms, Kunze came up with LAMP as a marketing-like term to increase the popularity of free software.[citation needed] O'Reilly and MySQL AB have made the term popular among English-speakers. Indeed, MySQL AB has since based some of its marketing efforts on the popularity of the LAMP stack.

Control structures

More interesting possiblities arise when we introduce control structures and looping. Perl supports lots of different kinds of control structures which tend to be like those in C, but are very similar to Pascal, too. Here we discuss a few of them.

foreach
To go through each line of an array or other list-like structure (such as lines in a file) Perl uses the foreach structure. This has the form

foreach $morsel (@food) # Visit each item in turn
# and call it $morsel
{
print "$morsel\n"; # Print the item
print "Yum yum\n"; # That was nice
}

The actions to be performed each time are enclosed in a block of curly braces. The first time through the block $morsel is assigned the value of the first item in the array @food. Next time it is assigned the value of the second item, and so until the end. If @food is empty to start with then the block of statements is never executed.

Testing
The next few structures rely on a test being true or false. In Perl any non-zero number and non-empty string is counted as true. The number zero, zero by itself in a string, and the empty string are counted as false. Here are some tests on numbers and strings.

$a == $b # Is $a numerically equal to $b?
# Beware: Don't use the = operator.
$a != $b # Is $a numerically unequal to $b?
$a eq $b # Is $a string-equal to $b?
$a ne $b # Is $a string-unequal to $b?

You can also use logical and, or and not:

($a && $b) # Is $a and $b true?
($a || $b) # Is either $a or $b true?
!($a) # is $a false?

for
Perl has a for structure that mimics that of C. It has the form

for (initialise; test; inc)
{
first_action;
second_action;
etc
}

First of all the statement initialise is executed. Then while test is true the block of actions is executed. After each time the block is executed inc takes place. Here is an example for loop to print out the numbers 0 to 9.

for ($i = 0; $i < 10; ++$i) # Start with $i = 1
# Do it while $i < 10
# Increment $i before repeating
{
print "$i\n";
}

while and until
Here is a program that reads some input from the keyboard and won't continue until it is the correct password

#!/usr/local/bin/perl
print "Password? "; # Ask for input
$a = ; # Get input
chop $a; # Remove the newline at end
while ($a ne "fred") # While input is wrong...
{
print "sorry. Again? "; # Ask again
$a = ; # Get input again
chop $a; # Chop off newline again
}

The curly-braced block of code is executed while the input does not equal the password. The while structure should be fairly clear, but this is the opportunity to notice several things. First, we can we read from the standard input (the keyboard) without opening the file first. Second, when the password is entered $a is given that value including the newline character at the end. The chop function removes the last character of a string which in this case is the newline.

To test the opposite thing we can use the until statement in just the same way. This executes the block repeatedly until the expression is true, not while it is true.

Another useful technique is putting the while or until check at the end of the statement block rather than at the beginning. This will require the presence of the do operator to mark the beginning of the block and the test at the end. If we forgo the sorry. Again message in the above password program then it could be written like this.

#!/usr/local/bin/perl
do
{
"Password? "; # Ask for input
$a = ; # Get input
chop $a; # Chop off newline
}
while ($a ne "fred") # Redo while wrong input

Exercise
Modify the program from the previous exercise so that each line of the file is read in one by one and is output with a line number at the beginning. You should get something like:

1 root:oYpYXm/qRO6N2:0:0:Super-User:/:/bin/csh
2 sysadm:*:0:0:System V Administration:/usr/admin:/bin/sh
3 diag:*:0:996:Hardware Diagnostics:/usr/diags:/bin/csh
etc

You may find it useful to use the structure

while ($line = )
{
...
}

When you have done this see if you can alter it so that line numbers are printed as 001, 002, ..., 009, 010, 011, 012, etc. To do this you should only need to change one line by inserting an extra four character