If you use the shell for serious programming, as I do, speed of execution is an important issue. A script should not appear sluggish; it should not be noticeably slower than a program written in Perl or Python — or even C. One of the major contributors to slowdown of scripts is starting a new process, whether it is an external command, or command substitution1 . (All shells except KornShell 93 create a new process for command substitution.)

When I started writing Unix shell scripts, I used a Bourne shell. It was far more powerful than the Amiga or MS-DOS shells I had used previously, but it still relied on external commands for most useful work. There was no arithmetic in the shell; I used expr and awk for calculations. I used expr, cut, tr, basename, and various other commands to manipulate strings.

With the Korn shell, and the later POSIX/SUS standardization, string chopping (via parameter expansion: ${var%PATTERN}, ${var#PATTERN}, etc.) and integer arithmetic were brought into the shell itself, speeding up many operations. It became possible to write a large number of useful programs without calling any external commands.

This still left many trivial operations requiring external commands (converting uppercase letters to lowercase, for example). Bash has a solution: commands that can be compiled and loaded at run time if and when needed.

Compiling and Loading Bash Built-Ins

The bash source package has a directory full of examples ready to be compiled. To do that, download the source from ftp://ftp.cwru.edu/pub/bash/bash-3.1.tar.gz. Unpack the tarball, cd into the top level directory, and run the configure script.

The configure script creates Makefiles throughout the source tree, including one in examples/loadables. In that directory are the source files for built-in versions of a number of standard commands "whose execution time is dominated by process startup time". You can cd into that directory, and run make: 

cd examples/loadables
make -k   ##  I use -k because I get some errors.

You'll now have a number of commands ready to load into your bash shell. These include: 

logname  basename  dirname   tee
head     mkdir     rmdir     uname
ln       cat       id        whoami

There are also some useful new commands: 

print     ## Compatible with the ksh print command
finfo     ## Print file information
strftime  ## Format date and time

These built-ins can be loaded into a running shell with: 

enable -f filename built-in-name

They include documentation, and the help command can be used with them, just as with other built-in commands: 

$ enable ./strftime strftime
$ help strftime
strftime: strftime format [seconds]
Converts date and time format to a string and displays it on the
standard output.  If the optional second argument is supplied, it
is used as the number of seconds since the epoch to use in the
conversion, otherwise the current time is used.

Modifying Loadable Built-Ins

With the strftime command, I can now do date arithmetic without external commands. For example, to get yesterday's date (a very frequently asked question in the newsgroups): 

strftime %Y-%m-%d $(( $(strftime %s) - 86400 ))

That script has one drawback: it uses command substitution. The timing of commands must not be taken too literally (they can vary a great deal even on the same system, depending on what else is running at the time), but they give a useful basis for comparison. The difference between using the built-in strftime (with command substitution) and the GNU date command is surprisingly small: 

$ time strftime %Y-%m-%d $(( $(strftime %s) - 86400 ))
2006-04-04

real    0m0.006s
user    0m0.000s
sys     0m0.005s
$ time date -d yesterday +%Y-%m-%d
2006-04-04

real    0m0.007s
user    0m0.000s
sys     0m0.007s

In absolute terms, it's not very long, but in a script there may be many such commands and they may be repeated many times. Since built-in commands are executed in the current shell, why not have it set a variable instead of printing the result? I added an option to strftime to store the result in a variable rather than printing it on stdout. The difference was significant: 

$ time {
strftime -v now %s
strftime %Y-%m-%d $(( $now - 86400 ))
}
2006-04-04

real    0m0.000s
user    0m0.000s
sys     0m0.000s

The changes to strftime.c are relatively minor. First, I included the header for bash's internal options parser: 

#include "bashgetopt.h"

Then I declared two variables: 

int ch;
char *var = NULL;

The longest piece of code parses the options, which are passed as a linked list and parsed by bash's own function: 

reset_internal_getopt ();
while ((ch = internal_getopt (list, "p:")) != -1)
switch(ch) {
case 'p':
var = list_optarg; /* should add check for valid variable name */
break;
default:
builtin_usage();
return (EX_USAGE);
}
list = loptend;

The bind_variable function stores the result in a shell variable if the -v option was used: 

if ( var )
bind_variable (var, tbuf, 0);
else

Finally, two lines to add to the documentation. The first is added to the array of strings that are printed when help strftime command is used: 

"OPTION: -v VAR - Store the result in shell variable VAR",

The second is the short documentation or usage string, and modifies the existing string: 

"strftime [-v VAR] format [seconds]",   /* usage synopsis; becomes short_doc */

The final strftime.c file is in strftime.c.

Writing New Bash Built-Ins

To write your own loadable commands, create a directory for them and copy the Makefile and the template.c files from bash-3.1/examples/loadables into it. The Makefile, which was created by running ./configure at the root of the bash source tree, contains the location of that source so that header files can be found. Make sure that top_dir points to the same place as BUILD_DIR. I also strip out all that I don't need. My resulting Makefile looks like this:

The template.c file is compilable and has the bare bones necessary to write a dynamically loadable built-in, plus the skeleton for adding command-line options. There are three necessary sections:

These are outlined in the examples/loadables/hello.c:

To write a new built-in command, I use newbi-sh to change the references to template in template.c to the name of my built-in, and add it to the Makefile:

One of the scripts most frequently requested in the Unix and Linux newsgroups converts filenames from uppercase (or partly uppercase) to lowercase. This usually means calling tr once for every file. (ksh has typeset -u, but it's non-standard and not implemented in bash.)

A shell function is quite efficient for converting short strings:

...but it drags when used for long words, and approaches the execution time of tr. A built-in command would be an order of magnitude faster, so I wrote lcase:

Having done that, I added the inverse, ucase to convert lowercase to uppercase. Then icase to convert upper to lower and lower to upper. Next came pattern creation to match either upper or lower case: 

$ icase "John Doe"
jOHN dOE
$ ncase qwerty
[Qq][Ww][Ee][Rr][Tt][Yy]

Finally, I added cap, to capitalize the first letters of words. I amalgamated all of these into a single file ( case.c ), and they are all enabled with a single command: 

enable -f $HOME/src/loadables/case ucase lcase icase ncase cap
1I first noticed how long command substitution takes when I was writing a script to print a form on the screen. A function converted dates in ISO form (YYYY-MM-DD) to a "human friendly" DD Month YYYY with a_date=$(format_date "$date"). There were only four date fields in the form, but with the conversion there was a noticeable delay; without it, the screen was redrawn immediately.