 Systems
Administration with Scsh
Evan Sarmiento
Shell scripting languages provide systems administrators with
powerful tools to automate mundane tasks, saving hours of work.
Most of these languages, however, do not provide a well-designed
object system, nor are they optimized for recursion and abstraction.
Scheme, a dialect of LISP, incorporates all the features of a good
programming language as listed above. Because Scheme cannot interact
with UNIX primitives alone, there is a Scheme software package called
scsh (the scheme shell). Scsh provides a set of macros for the Scheme
programming language, adding the ability to call on all standard
UNIX system calls and even adds regular expressions.
In this article, I will briefly introduce and describe the important
features of the Scheme language and present some of my own scripts
written in scsh, which exhibit many of the listed features. This
article presumes that you know how to program using Scheme. If you
are new to Scheme, read the Notes section at the end of this article,
which provides numerous learning resources.
You can download scsh from:
http://www.swiss.ai.mit.edu/ftpdir/scsh/
There are easy-to-follow installation instructions on the site. For
FreeBSD users, scsh is present in the ports collection, /usr/ports/shells/scsh.
Scheme comes in many variants, such as MIT Scheme, Scheme 48,
MzScheme, and others. Scsh is based on Scheme 48. However, I find
MIT Scheme to be the most complete. You can download MIT Scheme
from:
http://www.swiss.ai.mit.edu/projects/scheme/
Scheme tutorials and the book Structure and Interpretation of Programs
(Second Edition by Harold Abelson and Gerald Jay Sussman, with Julie
Sussman, The MIT Press) are provided online at:
http://sicp.ai.mit.edu
You can also find out more about each variant by looking at:
http://www.Schemers.org
Running Scsh
When you execute scsh, you're running in interactive mode,
which is similar to the interactive mode for Python. You can type
in individual commands, and the interpreter will evaluate them line-by-line.
To use scsh for batch processing, append this header to the top
of your scsh script files:
#! /usr/local/bin/scsh -s
!#
Benefits of Scsh
Programming in scsh provides numerous benefits. You will have
access to both a scripting language and a systems-programming language
when using scsh, because it is just a set of macros for Scheme.
scsh uses ''process notation'', which allows
easy access to the command line. Scsh also provides low-level access
to the operating system. The current release of scsh provides full
access to Posix, which includes: fork, exec, wait, sockets, write,
open, close, seek, tell, chmod, chmod, chgrp, chown, locking, and
more. Another benefit of using scsh is that it is very portable.
It is based on Scheme and, like Java, Scheme code is executed through
a byte-code interpreter. There are Scheme 48 virtual machines for
numerous operating systems, including Windows, which allow you to
run scsh in almost any environment.
The Scheme Philosophy
Before discussing the specifics of scsh, it is useful to understand
the peculiar syntax and fundamental data structures of Scheme. Scheme
was derived from Lisp, improving upon many of the Lisp features.
The Lisp programming language was mainly used in two fields: math
and artificial intelligence. Scheme is a dialect of Lisp that stresses
elegance and simplicity. It is a stripped-down version of Common
Lisp. In fact, the language specification is about 50 pages, while
the Common Lisp's specification is about 1,300 pages. Scheme
is often used in computer science courses to teach abstraction and
programming concepts. Even though it is used primarily as a teaching
language, Scheme has numerous applications -- the GIMP allows
users to write GIMP-loadable modules in Guile, a GNU Scheme variant,
and Scwm, a Window manager, allows users to write their own extensions
in Scheme.
The elegance and readability of Scheme is what prompted Olin Shivers
to write scsh, a set of macros that provide access to low-level
operating system functions and usual shell commands. Sh provides
a terse, inflexible, and hard to read syntax, which does not provide
access to lower level functions like sockets or signals, unlike
scsh, which encompasses both a shell scripting language and a systems
programming language.
Prefix Notation
Scheme may be hard to learn at first, but learning scheme will
certainly increase your programming skills. What separates Scheme
from other languages is its use of Prefix Notation. In Scheme, operations
are placed before operands. For example, you would normally write
this equation as such:
Regular: 5 / 3 * 10
Prefix Notation: (* (/ 5 3) 10)
Even simple arithmetic is changed. For example, 1 + 1 + 1 + 1
becomes:
(+ 1 1 1 1)
The concept of Prefix Notation is important to understand in order
to use scsh. All operations in Scheme are done through Prefix Notation
syntax, which may seem cumbersome at first, but simplifies coding.
Lists
Lists are an integral part of the Scheme programming languages.
A list is just a data structure. The most obvious application of
a List would be in a loop, where your program needs to take an element
from the front of the list and perform some evaluations on it. When
I program in scsh, I commonly use lists to parse output from programs.
I can also specify that the program output be translated into a
list of strings. Another benefit of Lists is that you can have functions
return more than one value using a list. This is how the (pipe)
system call works -- it returns a list consisting of two new
ports.
Lists are manipulated through the functions: list, cons, car,
cdr, and append. Scheme itself is based on the idea of a list. When
the Scheme interpreter tries to execute the command (+ 1 1 1
1), the interpreter thinks of it as a list, where the first
element of the list is the operator. You create a list by using
the list function.
The following examples are performed in interactive mode:
(define q (list 1 2 3 4 5))
This creates a list assigned to the name q. You do not have to assign
lists to names; you can pass them anonymously to functions.
You can retrieve the item off the front of the list by using car:
(car q)
1
You can remove one item from the list by using cdr. Using cdr
on a list creates a temporary copy of the list in memory with the
first item removed. The original list is still intact:
(cdr q)
'(2 3 4 5)
cons adds an item to the beginning of a list, while append
adds a list to the end of a list:
(cons 5 q)
'(5 1 2 3 4 5)
(append q (list 5))
'(1 2 3 4 5 5)
What is Scsh?
As stated before, scsh is a set of Scheme macros that allow you
to interact with the operating system, even providing access to
system calls. Scsh is different from other shell scripting languages
because it works with Scheme and uses a syntax that many people
are unfamiliar with. However, Scheme does provide recursion, abstraction,
object orientation, and readability features, which make it perfect
to use for systems programming or shell scripting.
Keep in mind that scsh is not a "command language";
it is a systems programming language, meaning that it encompasses
both a way to interact with the shell, and a way to interact with
low-level functions of the operating system. A programming language
is just a notation for expressing computation. Scsh has its own
way of expressing commands, called S-expressions, which will be
described in the next section.
Ports
A port, which appears throughout all scsh documentation, is like
a file descriptor, from which you can read and write data. In scsh,
there are certain functions that either require ports as arguments,
or return ports. You will probably be using functions that return
ports.
When I'm programming in scsh, I usually need to parse output
from programs or text-files. There is a specific function in scsh
that allows you to run a program and have all output returned on
a certain port. You can then read all the data from the port and
turn it into any sort of data structure. I usually turn output into
a list.
Ports are also used when opening files. When a user calls the
(open-file) system call, a port is returned and you can perform
subsequent operations, such as reading and writing, on that port.
Extended Process Notation
Interaction with the operating system is done through S-expressions.
S-expressions themselves are just data structures for representing
complex data. They can be either byte strings or lists of S-expressions.
Usually, while coding in scsh, you will be dealing with lists of
S-expressions. An S-expression could look like this:
(belong (all (your "base")) (to "us"))
This S-expression has no meaning and is only an example of the syntax.
When analyzing this S-expression, you can see that belong is
a function that acts on the return value of (all (your "base"))
and (to "us")).
Scsh has a notation, taking the form of S-expressions, for controlling
UNIX processes, which are called process forms. An extended process
form is a specification of a UNIX process to run, in a particular
environment:
(pf redir_1 redir_n)
where pf is the process form and redir_i are arguments.
Listed below is a set of process forms that do the normal set of commands,
such as creating a pipe, running a process in the background, redirecting
output into the calling process or out to a file:
| -- fork/pipe.
& -- Run a process in background.
begin -- Runs code in a fork.
< -- Open a file for read.
> -- Open a file for crease/truncate.
The list below provides scsh functions and examples that are commonly
used in shell scripting. Note that ">" means
that we are working at the interactive scsh prompt:
(run . EPF) -- Runs a process; returns an exit code.
For example:
(run (echo Hello))
Hello
(run/sexps . EPF) -- Run a process and return its
output as a list of strings. This is very useful as shown later.
This example executes ps | tail -1, transforming the output
to a list for further parsing:
> (define output (run/sexps (| (ps) (tail -1))))
> output
'(61661 v0 S 0:00.45 aterm -tr -tn xterm-color -bg black -fg white -sh 30 -font nexus)
> (car output)
61661
(argv n) -> string -- Returns the Nth argument
given to the program.
(begin . EPF) -- Runs code within a fork. For example:
(begin (run (ls)))
This forks a new process to execute ls.
(with-cwd directory . EPF) -- Changes the current
directory to directory and executes the other arguments.
For example:
(with-cwd "/usr/src"
(run (ls)))
(& . EPF) -- Runs a process in the background.
For example, compare these two commands. The first is a command
written in Sh, the second is written in scsh. It is a simple operation
and both commands get the last ten lines of /var/log/smb.log
and grep for the string File.
Sh: tail -10 /var/log/log.smb | grep File
Scsh: (run (| (tail -10 /var/log/log.smb) (grep File)))
System Calls
As noted before, scsh provides access to low-level system calls,
like dup, sockets, open, close, write, read, etc. All of these operations
are done through the ports. Below is a table of system calls supported
in scsh. I took a sampling from each section (not all of the system
calls are listed), and you can find the rest in the documentation.
UNIX I/O
(dup fd/port) -> fd/port -- This function duplicates
a file descriptor or a port, returning the newly created descriptor
or port:
> (define port (open-file "/etc/inetd.conf" open/read))
> (define port-dup (dup port))
(open-file fname flags [perms]) -> port -- This
function opens the filename referenced by fname and returns a
port. Perms is an optional argument, allowing you to set permissions
on the file you are opening.
The are numerous flags you can use, which are listed in the scsh reference
or in cheat.txt, however, the most commonly used flags are:
open/read
open/write
open/append
> (define port (open-file "/etc/inetd.conf" open/read))
(open-input-file fname [flags]) -> port
(open-output-file fname [flag perms]) -> port
(open-fdes fname flags [perms]) -> integer
This returns a true file descriptor; ports are more useful and do
all the same things.
(set-fdes-flags fd/port flags) -- Allows you to
set flags on a file descriptor or a port. You can find the flags
in the reference manual:
> (define port (open-file "/etc/inetd.conf" open/read))
> (set-fdes-flags port open/async)
open/async -- Enables Asynchronous writing on the
port.
(pipe) -> [rport wport] -- This returns two unused
ports, rport and a wport. All output on rport
is sent to wport.
(read-line [fd/port retain-newline?]) -> string or eof-object
-- When a file descriptor or port is specified, it returns
the first line waiting on the port:
> (define port (open-file "/etc/inetd.conf" open/read))
> (read-line port)
"# $FreeBSD: src/etc/inetd.conf,v 1.44.2.1 2000/03/25 22:09:59 jhb Exp $"
> (read-line port)
"#"
> (read-line port)
"# Internet server configuration database"
(read-string nbytes [fd/port]) -> string or #f --
Returns a string from a specified port:
> (read-string 30 port)
"# $FreeBSD: src/etc/inetd.conf"
(write-string string [fd/port start end]) -- Writes
a string to a specified port:
> (define port (open-file "/home/evms/.zshrc" open/append))
> (write-string "echo Boo" port)
Locking
(make-lock-region exclusive? start len [whence]) -> lock-region
-- Returns a lock-region record. Records are similar to a
struct in C:
> (define q (make-lock-region 0 30 50))
(lock-region fdes lock) -- Locks a file descriptor
according to the lock region created with make-lock-region.
File System
(create-directory fname [perms override?]) -- Creates
a directory fname.
(delete-directory fname) -- Deletes a directory
fname.
(delete-file fname) -- Deletes a file fname.
Processes
(exec prog arg1 ...) -- Executes prog with
args:
> (exec "/bin/ls" "-al")
(fork) -> proc -- Forks a new process and returns
a process object.
Process State
(umask) -> fixnum -- Returns the current umask:
> (umask)
18
(set-uid uid) -- Sets the processes UID:
> (set-uid 0)
Signals
(signal-process proc/pid sig) -- Signals a process
whose pid is pid with a signal specified by sig.
You can use almost any signal, but the most common ones are:
signal/kill
signal/stop
signal/alrm
signal/hup
(signal-process (cadr (run/sexps (| (ps aux) (grep inetd) (tail -1)))) signal/hup)
The (cadr ...) evaluates to the pid of the inetd
process. As you recall, run/sexps turns the output into a list
and cadr takes the second element from the list and returns
it.
These system calls are the ones I use most. There are socket system
calls in scsh, but they are complicated and won't be explained
in this article. The scsh documentation, however, has great examples
on sockets.
OO in Scheme
Object oriented programming is becoming a dominant trend for systems
programming, and it can also be used for administration tasks. For
example, you could create a computer object, which has the locally
defined functions ping, restore, reboot, and
local variables that indicate the IP address and hostname. When
used in a network environment, this script would be modular and
powerful. I would use Scheme to write this because it has the capacity
for OO.
OO in Scheme is mainly done through the use of state variables
and lambda. All variables in Scheme maintain state. Here, I will
use the example of a piggy bank object. You can deposit, withdraw,
and display the amount of change within the piggy bank. Of course,
you can modify this code to do something that is actually useful:
(define (create-piggybank)
(let ((balance 0))
(define (deposit amount)
(set! balance (+ balance amount)))
(define (withdraw amount)
(if (<= (- balance amount) 0)
(display "Error: you have no balance!")
(set! balance (- balance amount))))
(define (print_balance)
(display balance))
(define (dispatch operation)
(if (eq? operation 'deposit)
deposit
(if (eq? operation 'withdraw)
withdraw
(if (eq? operation 'print_balance)
print_balance))))
dispatch))
See Figure 1.
System Automation
Scsh is especially useful for system automation. Below are some
of the shell scripts I've created using scsh, comparing them
to a script written in sh that performs the same function.
A jail is similar to chroot, but with more restrictions. Jail
is implemented within FreeBSD. This script initializes all of the
jails within /jail. It is used on bootup. Within the /jail
directory, note that the jails are named by their IP addresses rather
than their hostnames.
Bourne Shell:
#! /bin/sh
JAIL="/usr/sbin/jail"
for dir in $(ls /export); do
echo $dir
$JAIL /export/$dir $dir $dir /bin/sh /etc/rc
done
Scsh:
#! /usr/local/bin/scsh -s
!#
(with-cwd "/jail"
(define (jail dir host ip)
(run (/usr/sbin/jail ,dir ,host ,ip /bin/sh /etc/rc)))
(for-each (lambda (j)
(jail j j j))
(directory-files)))
Comparison
The sh version of the script looks substantially shorter and easier
to read, but, there are substantial benefits in the scsh version.
One benefit is the ability to define functions within regions like
(with-cwd), which allowed me to keep the (jail) function
hidden from users of the script and I think it makes it easier to
read. (directory-files) is also much shorter than writing
for dir in $(ls /export); do....
(lambda) is used to create anonymous functions in Scheme.
As shown, I am passing a function as an argument to (for-each)
to be applied to every item of the list (directory-files).
Lambda is very powerful and can be used for a variety of purposes.
Using lambda, you can write a scsh script that returns functions
or creates functions from a template.
This second shell script, similar to one in ''Useful
Scripts for Overworked Administrators'' by Mark Prager
(Sys Admin, June 2001) pings a list of servers to test their
availability. If the servers are down, the administrator is contacted.
Bourne Shell:
#!/bin/sh
hosts="next-station postfix ns router"
for i in $hosts; do
if [ "$(ping -c 1 $i | grep "100% packet loss")" != "" ]; then
/home/evms/sms "$i is down."
echo "$i is down" | /usr/bin/mail kaworu
fi
done
scsh:
#! /usr/local/bin/scsh -s
!#
(for-each (lambda (host)
(define (alive?)
(run/strings (| (/sbin/ping -c 1 ,host) (grep "100% packet loss"))))
(if (alive?)
'()
(begin
(run (/home/evms/sms ", host is down."))
(run (| (echo ", host is down") (/usr/bin/mail kaworu))))))
(list "next-station" "postfix" "ns"
"router" "teqnix" "next" "logsrv"))
This example shows that I am also employing abstraction by defining
a function within lambda. This example is a bit larger than the sh
script, but it is certainly more readable.
Through the vnconfig program in FreeBSD, you can create
mountable partitions through disk images. This scsh shell script
accepts two arguments: the directory that you want to back up, and
the path to where the disk image should be stored:
#! /usr/local/bin/scsh -s
!#
(begin
(if (eq? (length (command-line)) 1)
(run (echo
"Usage: ./createbackup <tree> <file>
For example, ./createbackup /usr creates a self-mounting image <file>
containing all files within /usr"))
(let ((totalsize (car (run/sexps (| (du -c ,(argv 1)) (grep "total"))))))
(run (echo "It appears that" ,(argv 1) "is" ,totalsize "K. Starting the backup process.."))
(begin
(&& (dd if=/dev/zero of=,(argv 2) bs=1k count=,totalsize)
(vnconfig -s labels -c vn0 ,(argv 2))
(disklabel -r -w vn0 auto)
(newfs vn0c)
(rmdir /backup_mount)
(mkdir /backup_mount)
(mount /dev/vn0c /backup_mount)
(cp -R -f ,(argv 1) /backup_mount)
(umount /backup_mount)
(rmdir /backup_mount))))
(run (echo "Backup complete."))))
This example illustrates the advantage of Prefix Notation. Instead
of having to include a && after each command, I only
have to include it once, which makes the script easier to read. Also,
let is a very important constructor that allows you to define
variables within a region so that it can not be altered by the programmer.
This is another example of abstraction.
Conclusion
There are many shells available for the various *NIX operating
systems. Each one encompasses both an interactive prompt and a batch
mode where more complicated commands can be written. Most shell
scripting languages are archaic and have limited access to the operating
system. Scsh merges shell scripting and systems programming to produce
easy to read, portable, and fast code. When comparing the scsh shell
scripts to their sh counterparts, you can see the difference scsh
makes.
References
Shivers, Olin, "A Scheme Shell". 1994. MIT Laboratory
for Computer Science.
Shivers, Olin, and Brian D. Carlstrom. "Scsh Reference Manual".
1994. SCSH 0.5.2 Documentation.
Kelsey, R. and J. Reeves and M. Sperber. "The Incomplete
Scheme 48 Reference Manual for release 0.57", Scheme 48 0.57
Documentation.
Abelson, H., and G. J. Sussman and J. Sussman. "Structure
and Interpretation of Computer Programs 2nd edition". 1996.
MIT Press.
IEEE Working Group. "IEEE Standard for the Scheme Programming
Language". 1991.
Evan Sarmiento is a tenth-grade student at Boston University
Academy. He enjoys FreeBSD kernel hacking and network administration.
He can be contacted at: evms@bu.edu.
| 
