A termcap/terminfo Description for xterm

Hendrik-Jan Thomassen

Not all xterm implementations are the same. Especially if your implementation of xterm was vendor-delivered, it may have features not present in the X Consortium distribution. For example, to give applications a more PC-console-like appearance, manufacturers have been adding color support to their versions for some time now.

When the xterm implementation changes, the xterm termcap and terminfo definitions should also change. Even when vendors distribute the standard X Consortium version of xterm, they occasionally feel compelled to change the termcap/terminfo definition. Unfortunately, vendor-delivered xterm-definitions may differ quite a lot from the one delivered by the X Consortium, in both compatibility and quality.

While the definition from the X Consortium is correct, it is also quite terse. Moreover, because related capabilities are not grouped logically, the standard definition is more difficult to understand than it needs to be. This makes it difficult to successfully "borrow" features from the standard definition for your vendor-supplied definition. This article explains how to re-create a full-fledged xterm termcap/terminfo definition that is complete, correct, and well-organized.

This article focuses on xterm and its capabilities, but in doing so, it explains many of the termcap/terminfo capabilities. Thus it could also serve as a useful tutorial for writing all kinds of termcap/terminfo definitions.

Although I explain the individual capabilities in detail, I assume that you are familiar with the terminfo/termcap notation, and that you know how to handle commands like tic, infocmp (or untic), and tput. Consult the manual pages terminfo(4) and termcap(5) for additional information. The tables present both the termcap and terminfo entries; in the running text, however, I use only the terminfo capability names.

Cursor Positioning

The first few lines of my xterm definitions describe xterm's cursor positioning capabilities (see Table 1). Notice that the xterm "hardware" doesn't need any delays whatsoever in its capability descriptions.

The Boolean capabilities xm and xenl belong together. They specify that this terminal has automatic margins, i.e., that the "hardware" will handle a string which would extend over the rightmost margin. A terminal with automatic margins will cut that string into two pieces and display the second piece at the beginning of the next screen line. Automatic margins are the opposite of the behavior where all characters of a long line get printed at the rightmost position of the line, on top of each other, such that the last printed character stays visible eventually. The xenl capability says that lines of exactly 80 printable characters will cause only one line feed -- that is, the automatic line feed will not be generated after the 80th printable character, but before the 81st, if there is one. Thus when the line consists of exactly 80 printable characters plus a line feed, the terminal does not insert an automatic line feed.

The counts for the lines and columns are obvious -- until you have to allow for dynamically resized windows. See the sidebar "Static vs. Dynamic Windows" for more on this subject.

Lines 2 and 3 of Table 1 describe the one-step cursor movements (CUrsor Backward, CUrsor Down, CUrsor Up, CUrsor Forward). The most cryptic name is probably the termcap nd, which stands for "non-destructive space," i.e., a space-step which retains the character already present at the new location where the cursor moves to. Cursor home (upper left corner) is also obvious. The clear screen operation of a VT100 does not affect the position of the cursor. But the curses library wants the cursor to be at the home position after a clear screen. Therefore the clear screen capability contains the home-string as a substring. The ind capability indicates what to send to the terminal when the cursor is at the bottom line of the screen, and the user wants to invoke a one-line scrolling step upward. If this capability is not present, curses software will refuse to go into full-screen mode.

The curses software does all its calculations on the assumption that the upper left corner of the screen is (0,0). But the ANSIX3.64-1979 standard (on which the control strings for the VT100 and higher numbered members of that family as well as the PCconsole and ANSImode are based), decided to call this position (1,1). The %i indicator makes the curses software aware of this discrepancy, so it will apply a last moment adaptation to every calculated result just before the command string is sent out to the hardware. Note that this adaptation takes place only on calculated results for the capability involved.

Tab-Related Capabilities

To get the tab capabilities right (Table 2), you must realize that the curses package disables the stty/ioctl tab handling. Thus, the device driver will not expand tabs to spaces, but will pass those tabs unmodified. If your terminal is unable to handle tabs properly, the it capability should be absent, so that curses will expand tabs to the appropriate number of space characters. The xterm handles tabs properly, so the it capability is present.

The ht capability tells which character this terminal uses for tabs. The default, written here explicitly, is of course the standard ASCII tab character Ctrl-I.

Both termcap and terminfo can handle programmable tab settings. Termcap generates these by just pumping out the contents of the file named with the rf capability. That file, whose name may differ for your system, simply contains an "all tabs clear" command, followed by groups of eight spaces, interleaved with "set a tab-stop here" commands.

Terminfo is more intelligent: it wants to know what the current tab settings are. The it#8 means: tab stops are set at every eighth column. Next, terminfo will use the hts capability (horizontal tab set) to program a tab-stop at the current cursor position, preceded by a tbc (clear all tab-stops) first.

Video Effects and Graphic Characters

Table 3 shows video effect and graphic character capabilities. Line 10 gives the commands to "start mode underlining" and to stop (reset) it again. Line 11 is about "standout mode." An application programmer who is using curses but is unsure about (or interested in) in the exact possibilities of the terminal may request this standout mode. curses defines standout mode as some decent way of distinguishing text if the terminal is able to do at least something of this kind. The example uses a combination of bold and inverse video, which is somewhat noisy and overdone.

Line 12 gives explicit capabilities for bold and inverse video. Unlike a real VT100, an xterm is not able to show blinking text. xterm will map requests for blinking mode to bold display behavior. However, because this xterm definition doesn't specify an explicit blinking capability, curses will do the same mapping before sending the output to xterm. The sgr0 capability serves to turn all these extra modes off in one step. The curses package knows that VT100 terminals just have a single, non-discriminatory command for this.

If the terminal description defines an "alternate character set," then applications can use special characters to draw horizontal and vertical box lines, and the corners and intersections needed with them. Support for alternate character sets was added to terminfo-curses about ten years ago; a termcap-curses implementation may not support it at all. The string to switch to the alternate character set is in capability smacs, and switching back is done with rmacs. Note that the rmacs string is also part of the general cleanup capability sgr0. The enacs capability, necessary to initialize the alternate character setting, must be given once in the lifetime of an xterm. The enacs example given here specifies that the standard seven-bit ASCII character set will become the "normal" character set ( \E(B in VT100 language), and the Special Graphics character set the "alternate" one (\E)0 for the VT100).

The acsc capability indicates which character codes must be sent to the terminal to draw lines, under the assumption that the alternate character set is currently selected. The string in this capability should be read as pairs of two characters each. The first character of every pair defines the function (e.g., k is an upper right corner, m is a lower left corner, x is a vertical line). The function definition has been taken from the standard VT100 definition, and you need a VTxxx manual to find out which is which. The second character of every pair tells which character to send to this specific terminal in order to get the functionality of the corresponding VT100 character just in front of it. Because the xterm is VT100 compatible, the string in this capability consists of matching pairs of characters.

The msgr capability tells curses that cursor movements over a piece of text displayed in some special mode are without complications. Some terminals keep so much information in their video memories that step-counting for the cursor is almost impossible. Other terminals show visible effects if the cursor jumps from a position inside some special display area to another position outside it. The xterm does not have any of these quirks. Terminals that did have them are probably extinct by now.

Terminfo supports another possibility to control various combinations of video effects in one single command. The main purpose of this feature is to cut down transmission volume. In the sgr capability in Table 4 (no termcap equivalent), you find parameters %p1 ... %p9, which should each be either 0 or 1, to turn on/off the following attributes (in this sequence): standout, underline, inverse video, blink, half intensity, double intensity, invisible characters, protected characters (cannot be overwritten), and the alternate character set.

The strings in the example sgr definition use an if-then-else-if-then-else-if-then...else notation. For example, whether the ninth parameter (alternate character set) is on or off depends on the string %?%p9%t^N%e^O%; which must be read as: if p9 is not equal to zero (%?%p9 with the "not equal to zero" implicitly assumed just as in C), then (%t) send ^N, else (%e) send ^O, fi (%;). The first part of the sgr definition just composes a command of the form \E[0;x;x;x;x;m with digits 1 (bold, for p1 or p6), 4 (underline, for p2), 7 (inverse, for p1 or p3), and 5 (blinking, for p4) filled in at positions of the x characters if requested. The terminfo(4) manual page gives more details about this notation.

Special Keys

Tables 5, 6a, 6b, and 7 handle the recognition of special keys: the erase key, the four arrow keys, the numeric keypad, the function keys, and the extra keypad with, among others, the keys for next and previous page.

Except for the erase key, all keys produce a sequence of bytes, and the curses software does string matching on the incoming bytes to recognize which key got hit. Line 19 (Table 5) starts with the Boolean km. It indicates that this keyboard has a meta-key, which is a term dating from the days when keyboards were not available in proper 8-bit versions for the support of various national character sets. A meta-key, if present, acted as a kind of shift which in combination with another key generated the other key's code, with the eighth bit added to it. The default behavior of older curses implementations was to consider every eighth bit as a parity bit, and to strip it off. Modern curses implementations accept what modern keyboards give to them, which makes the km capability a dummy. If proper 8-bit operation is desired, you just make sure that the underlying terminal driver passes all 8-bit patterns on: stty cs8 -istrip -parenb -parodd.

Line 19 also shows that the erase key generates a backspace (^H -- other terminals may produce a delete 0x7F with this key). Lines 20 and 21 show the patterns generated by the arrow keys: "key cursor backward one step," "key cursor down one step," etc. A VT100 can switch the arrow keys between two different modes: the cursor mode and the application mode. The same holds for the numeric keypad: it has a normal (numeric) mode and an application mode. Depending on their current mode, these special keys will produce different strings. Arrow keys in cursor mode generate an escape, followed by a [, followed by A, B, C, or D. In application mode these keys generate an escape, followed by an O (capital Ooh) followed by A, B, C, or D. Table 5 assumes cursor mode to be the default, but some xterm implementations may have application mode as their default. The default for the numeric keypad is always numeric mode: the keys generate digits, a dot, a carriage return, etc. Application mode results in a more function-key-like behavior.

The smkx capability is the (combined) switching command for these modes. The rmkx capability switches back again. (Note that which mode smkx switches to depends on how the default is defined.) I prefer to keep all keys permanently in the same setting. Therefore, I defined both smkx and rmkx to set cursor mode for the arrow keys (\E[?1l) and normal mode for the keypad (\E>). If you have non-cursor based applications which assume a different setting, you may need to do some experimentation to find out when to have what installed.

The arrow keys can be set to application mode with \E[?1h, and \E= sets the numeric keypad to application mode. By the way: many terminfo descriptions don't define smkx and rmkx. When these two capabilities are omitted, the keys are set via the initialization string capabilities (covered near the end of this article, in Table 12) or the keys are kept in their default modes permanently. Just be sure to have your definition of smkx compatible with the strings in every individual key definition. A little tip: use the od -c command on standard input to find out which key generates what, but keep the translations at the ioctl level in mind.

Function keys are not switchable like the arrow keys or the numeric keypad. Depending on your xterm implementation, you get function keys with VT100 behavior or with VT200 behavior. These two behaviors are quite different: your keystrings should match your actual situation. Table 6a shows the VT200 style strings. Note the irregularities in the number pattern. Table 6b gives the corresponding VT100 style.

The special PF function keys found on VT-style keyboards are handled in exactly the same way, and their appropriate capability names can be found in the manual. Note in Table 6b that the code for key F12 equals the code for one of the arrow keys in application mode. This may cause conflicts during use.

If your X terminal has a keyboard with a PC layout, it will have keys for Insert, Delete, Home, End, PageUp (previous page), and PageDown (next page). VT-style keyboards have Find, Select, Insert Here, PrevScrn, Remove, and NextScrn, which I will not cover here. However, they must be handled in exactly the same way. The Delete key (capability name dch1) may cause several complications. Not all versions of tic translate ^? (the usual notation for the ASCII delete code) to 0x7F. You may get 0x3F (which is consistent with the translation of all other ^-notations, but should be treated as an exceptional case). This inconsistency may even lead to asymmetric behavior between the tic and infocmp commands. Furthermore, the functionality of the Delete key interferes with the stty/ioctl-setting of the erase character. Therefore, it is usually best to not specify this key at the termcap/terminfo level, but to leave its operation to the ioctl level.

The Home and End keys will not always produce a unique string, or even any character at all. Table 7 defines these keys only for the VT100 mode (line 31b), and leaves them undefined for VT200 mode.

Some curses applications use the 3x3 group of digit-keys in the numeric keypad (in application mode) as arrow keys, with the arrows on the middle keys of each side. vi doesn't use this key overloading, but the curses library supports it. The corner keys and the center key are left: curses terminology names them the ka1 (upper left), ka3 (upper right), kb2 (center), kc1 (lower left), and kc3 (lower right). The numeric keypad also has an enter key which does not just produce a carriage return when in application mode. The kent capability specifies what it does produce. On a VT-style keyboard it may be necessary to mirror the top and bottom row definitions (ka1=\EOq, ka3=\EOs, kc1=\EOw, kc3=\EOy). You may define other keys on this keypad in a similar way. The terminfo manual lists capability names for an almost infinite number of different keys.

If you have an application that wants to program the function keys itself, you need more information than I give in this article, since xterm does not support such behavior. It may be necessary to call upon xmodmap for help in this case. You also need xmodmap if a special key does not produce any string at all, or in case two keys produce the same output string.

Cursor Movement Optimization

Table 8 provides the basic information for the optimization of cursor movements. All these capabilities are optional, but their effect on speed and transmission volume is great. Leaving out the el capability may provoke buggy behavior from vi.

An xterm is able to remember its current cursor position, and restore it later upon request. This feature is not a stack-based mechanism, but just a one-deep memory. sc (save cursor) and rc (restore cursor) do the work.

The cub/cud/cuu/cuf are the multi-step equivalents of the cub1/cud1 etc. capabilities from lines 2 and 3 in Table 1. These capabilities make the cursor jump in the direction indicated over a specified number of positions. Their effect on efficiency is moderate because the command takes almost the same amount of transmission time as a complete cursor movement (the cup of line 7). I also define several erase capabilities in Table 7. An xterm terminal can clear part of its screen, from the cursor position to the end of its line (el), from the cursor to the start of the line (el1), or from the cursor to the end of the screen (ed).

The "change scrolling region" capability on line 42 plays a very important role in optimization. A good example is a dd command in vi. The optimization routines in curses are even clever enough to reverse the scrolling direction, if the terminal supports it. Line 43 (ri) reverts the scrolling direction. It is very instructive to set a terminal at a slow speed (e.g., 1200 bps) and look at the effects if you take out one or a few of the lines 40-44.

Line 44 (mir) indicates that the terminal calculates properly when tabs on a line interfere with cursor steps. Like msgr of line 14, it is a throwback to the times when terminals had too much hardware and not enough software aboard.

Insert and Delete

The definitions in Table 9 also exist mainly for efficiency reasons. These entries define separate capabilities for efficient delete and insert operations. Separate cases are made for one line, a number of lines, one character, or a number of characters. Theoretically, there could be eight cases, but because of a bug in many xterms it is not usual to define the character insert capabilities ich and ich1. These two capabilities may produce one superfluous space for every actual character. Line 48 switches to insert mode (smir), and back (rmir) again to replace mode. Replace means that a newly written character erases the character already present at that position.

The Mouse

Not all xterm implementations support a mouse in a way that curses can use. This paragraph is only relevant for xterms that have the proper support. An example is the xterm as delivered with Novell's UnixWare, from which the command strings of Table 10 were taken. This table specifies that the mouse has three buttons. The reqmp capability requests the current mouse position, and the answer is expected to begin with the kmous string. The getm capability is used by curses to inquire about mouse button events.

Colors

Color support in xterm is a very recent development. Several manufacturers have added color support already to their versions of xterm, to make curses applications resemble PC-monitor-based software. The general form of the color commands (ISO6429) is: \E[3xm to set the foreground to color number x, and \E[4xm to set the background to color number x. Color numbers are defined as follows: 0, black; 1, red; 2, green; 3, yellow; 4, dark blue; 5, magenta (purple); 6, cyan (light blue); 7, white. (It is easy to remember these as all possible combinations of Blue+Green+Red, in that sequence, expressed as a three-bit number.) Colors are controlled with the capabilities of Table 11. Do not include these definitions if your xterm does not have color support.

Table 11 shows that eight colors are supported, and that curses will maintain bookkeeping space for up to 64 color pairs (fg+bg). curses applications can compose these pairs themselves. The setaf and setab modify foreground and background color in an ANSI-defined way. The eight color numbers mentioned above are the defaults for applications that do not want to define additional colors themselves. If application programmers want more colors than the basic eight, they must define them explicitly, they must use hardware that supports this (an X terminal does, of course), and the colors capability must be defined at a higher level (64 max). Usually, the background color of an xterm is set to some other color than the standard white, so color 7 is not the same as "the default background."

If an application wants to change foreground or background colors, curses prefers to use the setf and setb capabilities. These definitions look horrifying, because of the if-then-else construct used. Basically, system administrators would define their own translation tables for the choices made by the application programmer. The example reads (from left to right): if the programmer has chosen color 1, then I want a 4; if a 3 is chosen, then I want a 6; instead of a 4, I want a 1; instead of a 6, I want a 3, and in all other cases give me what the programmer did choose.

The final color capability is op: go back to the original fg/bg color pair. This capability prevents colors chosen by a certain application from persisting after the application has terminated. I must admit that I do not know whether op capability is standard in color-supporting environments; this example comes from a UnixWare environment.

Initialization and Reset

Table 12 handles a variety of initializations and resets.

Line 57 contains codes specific for xterm, which are documented in the xterm Control Sequences Document. These are the first definitions I've used which are not VT-compatible. The smcup capability specifies a string which will be sent to the terminal at the start of an application which intends to use cursor position commands. rmcup is the corresponding closing string. xterm has not been designed for such a requirement, but "misuses" these capabilities in a very creative way.

The VT-compatible part of xterm has two different display surfaces (glass screens). (The Tektronix screen is in addition to these two VT component screens.) The smcup/rmcup sequences switch between the two VT-screens.

When a curses application (like vi) is started, it will first switch screens (with smcup and then perform a "clear screen"). By switching screens first, the pre-session display is saved and will reappear when vi exits. The rmcup capability encodes this behavior by clearing the second screen again (\E[2J) and the current line on the first screen (\r\E[M). You may encounter versions of smcup/rmcup that switch the modes of your cursor keys and numeric keypad keys, as covered earlier in this article (line 23). If you follow my advice to stick to one key mode permanently, you should remove those mode-switching strings.

The is2 capability is also an initialization string. This string is sent to the terminal if a user asks for it with the tic command. This request may be useful in a .profile if you want to be sure that your session starts with the terminal in a well-defined mode. The example definition (line 58) defines the entire screen to be the scrolling region (\E[r), turns off all video effects (\E[m), and does some mode-sets and resets (to be explained shortly).

The terminfo man page has a long section about initialization capabilities (is1, is2, is3). You can find related information in the description of the init parameter of the tput(1) command. Generally speaking, is2 is the key initialization string, and the others only serve special, nonessential purposes. It is important to keep is2 compatible with the arrow key and numeric keypad modes selected by smkx.

Finally, there are the reset capabilities, available as a numbered collection just like the initialization capabilities. From the manual you can learn that rs2 is the workhorse. Again, the description of the reset parameter of the tput(1) command includes more information. The example reset capabilities have been kept very similar to the initialization capabilities. Make sure that rs2 leaves the key modes as you want to have them for non-curses based operation.

The terminal mode sets and resets distinguish between ANSI-standard based properties and vendor-specific properties. The latter category is a mixture of DEC VT-definitions, and specific xterm extensions. This gives (among others) the following combinations:

\E[ ... l       reset ansi modes
\E[ ... h       set ansi modes
\E[? ... l      reset DEC and xterm modes
\E[? ... h      set DEC and xterm modes

At the ellipsis you fill in numbers that indicate the specific properties. Multiple numbers are separated with semicolons. Not all possibilities are of equal importance. The following list is more than sufficient for almost any situation. In general, the reset situation is the default one.

ANSI modes: withE[ ...

4 reset is replace mode: new characters write over old ones, which is the usual situation. Set is insert mode: characters behind an insertion point shift to the right as new characters are being inserted.

20 reset is non auto linefeed behavior. Set adds a linefeed to every carriage return (which you never want because curses does this job already).

DEC and xterm modes: withE[? ...

1 reset is ANSI (cursor) mode for the arrow keys. We have covered this with the smkx and is2 capabilities. Set is application mode.

3 reset is 80 columns screen; set is 132 columns.

4 reset is jump scroll; set is smooth scroll.

5 reset is normal video; set is inversed video.

6 reset means that the screen origin is at the home position; set means that it is at the upper left of the currently defined scroll region.

7 reset is: auto line wrap off. Set is on. In our examples I specified the am capability, which means that I expect a set status for this property. Since set is the UNIX default not only for a curses environment but for line-oriented commands as well, I set it explicitly in the initialization and reset capabilities.

8 reset means auto-repeat key behavior off; set means on. Since auto repeat on is not the default but is generally preferred, we set this explicitly in the initialization and reset.

38 reset means: VT100 mode. set means: Tektronix 4014 mode.

44 reset means: margin bell off. Set is margin bell on.

46 reset means: xterm logging enabled. Set is logging disabled.

47 reset is: use the first VT screen buffer (as discussed with the smcup/rmcup capabilities in line 57). Set enables the second screen buffer.

In real life, the is and rs capability strings are usually a mess. Every copy you encounter will have other choices for what to set, reset, or ignore. Often you see several separate commands inside a single capability which could have been bundled into one. In any case the initialization and reset strings give you a chance to fix problems with applications that don't clean up properly when they stop.

Conclusion

Obviously, the cautious reader should not switch to my description on blind faith. However, if you compare this xterm description to your vendor-supplied version, you may be able to augment the latter by adding selected bits and pieces from this story.

Because this definition groups related capabilities in a way that makes clear which parts belong together and which parts are independent of each other, it should be relatively easy to add specific portions to your existing xterm definition.

For Further Reading

B. Goodheart. UNIX Curses Explained. Prentice Hall, ISBN 0-13-931957-3.

E. Moy, S. Gildea. Xterm Control Sequences. MIT X Consortium.

About the Author

Hendrik-Jan Thomassen has been a full-time UNIX system administrator and programmer for the last 21 years. Back in the days of UNIX 5th Edition he was one of the first handful of UNIX users in Europe. He co-founded AT Computing in Nijmegen, The Netherlands. This company now performs training throughout The Netherlands in UNIX, C, the X Window System, and a variety of other topics all related to the UNIX system. He can be contacted via email at hjt@atcmp.nl.