Cover V02, I06
Listing 1
Listing 2


Building a Secure Journal/Logging Utility with Encryption

Leor Zolman

I don't usually "recycle" ideas from my past writings in other publications, but for this article I decided to make an exception. Back in the June, 1991 issue of The C Users Journal I presented a C program named j, a personal journal utility (as in diary, not the computer science sense of "journal"). That program provided an easy-to-use interface for the creation of dated, sequential entries in cumulative ASCII text files, with support for multiple subject categories. While I created it specifically with personal journal writing in mind, one user later pointed out how the program could be equally useful as a software or system logging mechanism. Such a utility allows a system administrator (or team of administrators) to easily create log entries documenting any information deemed worthy of recording.

The j program turned into one of those perpetual projects for me. When I sat down to make entries in my electronic journal, about as often as not I'd first take some time to modify how j worked; often that left me no time to actually make a journal entry.

I chose C as the original implementation language for j because I was writing for CUJ and wanted to stress portability (in this case, between DOS and UNIX on the source code level). Eventually, I decided to abandon the DOS version and focus on the UNIX implementation in order to take advantage of UNIX's encryption utilities. Without the requirement for DOS support, it became apparent that j is an application better suited to implementation in shell script rather than in C. After all, most of the "work" is done by external programs. All j does is check the environment and command line for controlling parameters, initialize/concatenate files, and call up the text editor and encryption programs as needed. After loading up the original C version with features and struggling to make them all work together, I found that a total rewrite in shell script resulted in a much more coherent piece of code. That shell version is what I present in this article.

Basic Use

The j script has two links, named j and jsee. When invoked as j, it adds new journal text to the current cumulative monthly text file under the invoking user's $HOME/.Journ directory. The jsee link is used to examine and/or modify either the current cumulative text file, or any cumulative text file created by j.

Setting aside the encryption features for now, here is a description of what j does. The first time it is invoked by a user, j checks whether or not the $HOME/.Journ directory exists, and creates it if necessary. Then, j checks if a name argument was given on the command line. If not, j assumes the user wants to add some text to the current default cumulative monthly text file named $HOME/.Journ/yy-mm.ext, where yy is the current two-digit year, mm is the current two-digit month, and ext is an extension dependent upon encryption status (described later). If a name argument was given on the command line, that name is interpreted as the name of a subdirectory of the $HOME/.Journ directory in which the cumulative text file resides (or is to reside). In that case, the full pathname of the cumulative text file to use becomes: $HOME/.Journ/subdir-name/yy-mm.ext.

Once j has figured out which cumulative text file to use, it creates a text file containing a header line with the current date and encryption status, then starts up the text editor to let the user add text to the file to form a complete journal entry. When the user exits the editor, j checks to see if the file has been modified (e.g., the user has saved the work.) If not, no further action is taken -- j assumes the user has decided to abort the journal entry for whatever reason. If the file has changed, then the entire contents are appended to the cumulative text file identified above.

To bring up a previous journal entry, you use jsee. When invoked with no arguments, jsee brings up your text editor with the contents of the current default cumulative monthly journal file, $HOME/.Journ/yy-mm.ext. You may then examine the file's contents and/or edit the file as desired. jsee accepts an optional subdirectory name command-line parameter, similar to j. Unlike j, however, jsee also accepts a straight pathname of a cumulative journal text file, minus the extension, as an argument. If such a name is supplied to jsee, the appropriate encryption extension is tacked on and the result is treated as the pathname of the journal file you wish to examine. For example, the command:

jsee 93-07

would bring up the contents of a file named ./93-07.ext, where ext is an extension dependent upon encryption status. On the other hand, the command:

jsee project1

would bring up the file $HOME/.Journ/project1/yy-mm.ext, where yy-mm are the current year/month, and ext is dependent upon encryption status.

For everyday use as a personal journal manager, the operation of j/jsee couldn't be simpler. To create a new entry, just enter the command:


To review the current month's entries, enter:


Encryption Support

To make my journal entries as secure as possible, I decided to take advantage of UNIX's crypt command to implement automatic encryption/decryption of journal text. This turned into a bigger Pandora's box than I ever could have imagined, due primarily to the inconsistency of crypt features across UNIX platforms. My primary work with j/jsee has been under SCO XENIX and SCO UNIX (two products from the same vendor), yet the crypts from just these two platforms contain a surprising number of dissimilarities that I've had to build in support for. And the crypt command I've experimented with on yet a third Unix platform, Encore UMAX, differs slightly from both the others.

Vanilla crypt

First, I'll to take a look at the common, standard usage of the crypt command. In its basic form, crypt encrypts the plain text provided on its standard input stream and sends the encrypted text to its standard output. The user can specify the encryption key as the only option on the command line. Thus, the command:

crypt foobar <file.1 >file.2

creates a file named file.2 having exactly the same length as the existing file file.1 and containing the text from file.1 encrypted with the key "foobar". To decrypt file.2, the command to use is:

crypt foobar <file.2 >whatever

After execution, a file named whatever should contain exactly the same plain text as was originally contained in file.1.

If the encryption key is omitted, then crypt prompts the user for the key before performing the encryption. This is probably the most secure way of using crypt; however, the way j/jsee works often requires multiple runs of crypt within a single process pipeline, and using the "no key" format is incompatible with such a usage. The only way to avoid the pipeline would be to create temporary files, and that kind of approach opens its own can of security worms.

Up to this point, all versions of crypt I've used display the same behavior, and if this was all the functionality I expected out of crypt, then there wouldn't be any problem. However, invoking crypt with the key on the command line is highly insecure under XENIX, and only marginally secure (sort of like "only a little bit" pregnant) under UNIX. Under XENIX, anyone can list the system process table with full detail using the command

ps -ef

during the entire time that the crypt process is executing. This would reveal the entire command line used to start up crypt -- including the actual encryption key specified by the crypt user. Using SCO's XENIX-specific version of crypt, I've found no way around this "hole".

The UNIX version is a little more careful to wipe out any trace of the encryption key entered on the command line. It accomplishes this by taking advantage of a new feature of crypt specific to the UNIX version: the -k option. In this version, specifying -k in place of a literal encryption key causes crypt to read the actual key out of an environment variable named CrYpTkEy. The assumption in this case is that you've defined and exported CrYpTkEy before running crypt.

The advantage of getting the key from an environment variable is that there is no simple way for any user, including the super-user, to sneak looks into another user's environment. When supplied with a literal encryption key, the UNIX crypt can quickly stuff the string into the required environment variable and then exec a new crypt with -k to overlay the original crypt process and perform the encryption. There would still be a short window of opportunity when someone running ps -ef might be able to see the encryption key, but a user would have to be real fast and lucky to catch a glimpse through that window. Of more concern would be whether or not process accounting is enabled on your system; if so, the system administrator might have access to a record of the original crypt command line with the explicit encryption key spelled out.

The best way I've found to take advantage of the -k option is to provide a way of prompting the user for an encryption key that leaves no revealing process accounting trails for nosy administrators to follow (not that any of us self-respecting administrators would ever stoop that low, of course...), and then to initialize the CrYpTkEy environment variable for use by subsequent crypt invocations. Under XENIX, where -k isn't supported, I've made the interface similar for at least the convenience, if not the security, of the user.

Configuring j/jsee

The greatest complexity of j/jsee is a result of providing the flexibility to adapt to both the differing forms of the crypt command and the varying security needs of users. While you might sense a bit of "overkill" in the range of configuration options, the end result is that once you've set it up to fit your needs, the program will be both easy to use and about as secure as you need it to be.

The comments at the start of the script (Listing 1) provide a complete reference to internal configuration variables and sensitivity to external environment variables. There are three areas involved: encryption control (whether or not to encrypt), getting the encryption key, and supplying that key to crypt as necessary.

Encryption Control

To determine whether or not the user wants the journal text to be encrypted, j/jsee examines its command line, the JCRYPT external environment variable, and the ASK_CRYPT configuration variable.

JCRYPT, the external variable, can be set to Y or N. If Y, j/jsee uses encryption unless (a) the - option appears on the command line, or (b) ASK_CRYPT is Y and the user answers "no" to the encryption prompt.

  • If JCRYPT is N, then encryption is disabled.

  • If the ASK_CRYPT configuration variable is set to Y, then j/jsee prompts the user to verify encryption when JCRYPT is set to Y. When JCRYPT is N, ASK_CRYPT has no effect.

  • When JCRYPT is undefined and no command-line options appear, encryption is disabled. If -e or -ekey is used on the command line, encryption is enabled.

  • If - appears on the command line, encryption is always disabled, regardless of any other configuration parameters.

    Whenever encryption is in effect, the extension on the cumulative journal text files manipulated during that session is .txe. Without encryption, the extension is .txt. Forcing different extensions based on encryption status helps prevent the inadvertent mixing of plain and encrypted text within the same file. Even though j/jsee always checks encrypted journal files for a signature line at the beginning to avoid encryption key conflicts, the distinct extension names let users determine whether or not journal files are encrypted without having to examine file contents.

    Getting the Encryption Key

    SCO UNIX's crypt uses an environment variable named CrYpTkEy to store the encryption key in the user's environment (for use with -k), while the Encore's crypt uses CRYPTKEY for the same purpose. To avoid having to do a conditional global replace as part of the j/jsee configuration process, I've created a configuration variable named CRYPT_KEY_VAR that lets you change just one line and forget it.

    When running j/jsee, the user is responsible for initializing the variable named by CRYPT_KEY_VAR if the local crypt supports -k (and the use of that feature is desired). Although the user can insert an assignment statement into his or her startup file to perform the initialization, that is not a good idea from a security standpoint. It would be better if there were no "hard copy" of the encryption key anywhere on the system. Alternatively, the user can enter the shell assignment explicitly before using j/jsee, e.g.,

    $ CrYpTkEy=cowabunga_dude

    In this case, the startup profile should include the export statement

    export CrYpTkEy

    so the user does not have to type it in every time. Since typing out the word CrYpTkEy can get pretty annoying, I've written a shell function named key (Listing 2) for inclusion in the startup profile. This function prompts the user for a new encryption key, reads the new key value without echoing the characters to the screen, and exports the variable.

    Screen echo is disabled during key input, so a user interrupt during the read statement would cause a return to the system prompt with screen echo still disabled. For key to be robust, it needs to respond intelligently to a user pressing the INTR key during encryption key input. key deals with this issue by defining a shell variable named TRAPPED upon receipt of a user interrupt. After the input line has been read, key tests this variable and takes appropriate action if an interrupt had indeed been processed.

    Passing the Encryption Key to crypt

    j/jsee needs to know whether or not your particular version of crypt supports the -k option. You supply this information via the internal configuration variable CRYPT_HAS_K. If set to N, then j/jsee is obliged to supply the literal encryption key on the crypt command line every time it invokes crypt.

    If CRYPT_HAS_K is set to Y, then j/jsee makes sure the environment variable named by CRYPT_KEY_VAR is set to the encryption key and the -k option is used with crypt.

    Note: even if you use the key function with a version of crypt that does not support -k, the encryption key is still stored in the environment variable named by CRYPT_KEY_VAR; in this case, j/jsee simply gets the key value out of the environment variable and explicitly includes it on the crypt command line when needed. If you should someday move to a system where -k is supported, you'd reconfigure CRYPT_HAS_K to Y and notice no change at all in any aspect of j/jsee's user interface.

    Command-Line Processing

    I didn't want to restrict the order in which things have to appear on the j/jsee command line. The format I chose for the encryption options (-e, taking an optional argument; -, having meaning, etc.), however, precluded use of the standard getopts mechanism for command-line parsing. To allow the encryption option to appear either before or after the journal-file or journal-dir arguments, I basically check for all permutations via an exhaustive brute-force approach. Since my format supports a maximum of two distinct command-line arguments, it is not prohibitively difficult to check for the five possible permutations (no args, encryption option only, name option only, or both in either order). Lines 287-320 figure out what the user is trying to say on the command line.

    j/jsee Shell Functions

    Lines 159-266 contain several shell functions used internally within j/jsee. Here are the descriptions of these functions:

    usage: Displays a usage summary.

    ed_invoke: Invokes the user's text editor on the named plain-text ASCII file. If the editor is microEMACS or Epsilon, then ed_invoke uses some editor-specific techniques to enhance the user interface. Before editing the file, ed_invoke does an ls -l on the file and saves the output. After the editing session, another ls -l is run and the result is compared to the first one. If they are identical, then a message indicating that fact is displayed and j/jsee terminates.

    get_key: Assigns the encryption key to the environment variable key, obtaining the value either indirectly through CRYPT_KEY_VAR or through direct keyboard input from the user.

    test_k: If -k is supported, "moves" the encryption key out of the key variable and into the appropriate environment variable supported by the local crypt command, then sets key to -k for use on the crypt command line.

    ask: A general purpose yes/no utility function.

    Code Walk-Through

    Execution of j/jsee begins at line 269 with a banner message, and lines 271-285 create the master journal directory, if necessary.

    Then, lines 287-320 process the command-line arguments, initializing the following environment variables:

    OPTS: the option string specified, if any

    name_given: Y if a filename or subdirectory name was specified, else N

    jdir: the pathname of the directory containing the target journal file

    jfile: the filename of the target journal file.

    Lines 326-354 process the encryption options, initializing the following environment variables:

    crypt: Y if encrypting, else N

    key: if -k is not supported, this is set to the literal value of the encryption key. If -k is supported, then this becomes -k and the external environment variable named by CRYPT_KEY_VAR is set to the encryption key value

    EXT: set to txe if encryption is in effect, else to txt.

    Lines 356-372 perform some additional processing to determine the cumulative journal text filename when jsee is invoked. This is basically to support jsee's ability to accept a journal filename as an argument.

    Lines 383-398 check for the existence of any named subdirectory supplied on the j command line, and create it if necessary. jsee does not support subdirectory creation, since its purpose is to examine existing journal text files only.

    Lines 400-409 complete the specification of the journal text file by adding an appropriate extension onto the pathname, if required. If running as jsee, then a filename argument would already have been processed into a directory and filename (with appropriate extension) by the code in lines 362-369.

    Lines 417-436 create a brand new cumulative monthly text file. This happens whenever it is time for the first entry of a new month, in the main $JOURN_DIR directory or any of its subdirectories.

    If the cumulative journal file already exists and encryption is in effect, then lines 437-445 check encryption key synchronization. Since appending text onto an encrypted data file necessarily involves decrypting the original file, appending the new text to the plain text of the original file, and encrypting the entire resulting file, there is a potential for disaster if a different key is used during the append process than was used to encrypt the earlier data. If j didn't check to make sure that the current encryption key successfully decrypts the existing cumulative text file, the result of misspelling an encryption key would be total loss of all previous cumulative monthly data. Whenever a new cumulative file is initialized, a signature line containing the word "Journal" is written at the beginning of the file. Thereafter, any time you run j/jsee to create a new entry or examine an existing cumulative file, lines 438-444 decrypt that file with your currently selected encryption key and make sure the word Journal is there before letting you do anything else.

    With all these preliminaries out of the way, j/jsee can finally do its real work. If the command was invoked as jsee, then lines 447-464 execute the jsee logic to update the specified cumulative journal file. If the command was invoked as j, then lines 466-489 create a skeletal journal entry, let the user edit it, and then append the edited text onto the selected cumulative journal file.

    Last Thoughts

    There are some subtle "gotchas" involved in using crypt portably, and I've tried to cover the most relevant issues in the j/jsee showcase. You may find j/jsee useful in its own right as a journal and/or logging utility, or you can extract whichever pieces you may need to help construct your own encryption-related applications. I'll probably keep on tinkering with this code for years -- if you're curious about what it might evolve into after another year or two, just e-mail me then and I'll be glad to send you back the latest (greatest?) version.

    About the Author

    Leor Zolman wrote BDS C, the first C compiler targeted exclusively for personal computers. Leor is currently an instructor on UNIX topics for Boston University's Corporate Education Center, a regular contributor to The C Users Journal and Sys Admin magazines, and "Tech Tips" editor for Windows/DOS Developer's Journal. His first book, Illustrated C, was recently published by R&D Publications, Inc. He may be contacted at 74 Marblehead St., North Reading, MA 01864, or on Usenet/Internet as: