Option Directories -- Rationale

Usage

In more modern SysV R4 Unix systems and their derivatives, the /opt directory has begun to be used as a location for installation of software "option" packages--things that aren't part of the "base" OS installation. So far, we have seen this directory used for

• OS vendor add-ons, such as HP's ANSI C compiler,
• OS vendor applications, such as HP's OpenView, and
• Third-party vendor applications, such as Rational's ClearCase.

• Freely-available software packages, acquired and supported mostly through the Internet, such as the GNU tools, and
• Locally developed and maintained software packages.

We feel that the latter two categories are sufficiently similar to the other sorts of "option" packages that they should be treated in a similar fashion, and located in the same directory tree.

The Package Philosophy

When using /usr/local or similar installation points, a package is typically distributed throughout several parallel directory trees: /usr/local/bin for the binaries, /usr/local/lib for any related libraries, and often for configuration information, /usr/local/man for manual pages, and so on. Upgrading or uninstalling a package can be a tricky and time-consuming proposition.

The /opt package concept attempts to rectify this scattering of files by collecting all installed files related to a given package into a single directory, normally referred to as /opt/appname. Thus an application's executable binaries would be in /opt/appname/bin, its library files in /opt/appname/lib, its man pages in /opt/appname/man, and so on.

This single-tree approach greatly simplifies the management of multiple versions of a given package, including upgrades, and also makes uninstalling a package a snap.

Addressing Common Problems

The obvious approach would be therefore to simply create an /opt/appname directory, install an application into it, and be done with it. However, experience has shown that this simplistic method leads to several maintenance problems:

• Maintaining multiple concurrent versions of an application is not possible with this naming scheme. This has several ramifications, such as being able to fall back to a known working version if an upgrade turns out badly.
• The /opt directory quickly becomes littered with dozens or even hundreds of directories which can be difficult to categorize. Most sysadmins would like to be able to tell whether a package is locally maintained or came from outside the company, and whether it is supported commercial software or is a free tool.
• Installing an application in its own directory on each machine in a network quickly becomes unmanageable. This problem grows seemingly exponentially with the number of machines in the network.
• Some methods of centralizing control (to address the previous issue) can lead to problems if different platforms are being supported from the same servers.

Open Questions

We haven't solved all the issues surrounding this question, and some of the parts described in the following sections are still open to some debate. You may wish to skip this section on your first reading, and refer back to it as you read the rest of the document.

Some of the problems and questions we still have:

• The complexity of the scheme can be daunting. However, it really is fairly simple once understood, and is very easy to work with. Our answers to this issue are currently
  1. this document, which attempts to clarify the structure and the reasons behind it, and
  2. the repln script, which automates much of the creation and propagation of necessary links.
• Many people are horrified at the prospect of having softlinks pointing to their applications, not to mention links pointing to links. In most cases at our installation, application binary foo is a softlink /opt/bin/foo which points to /opt/foo/bin/foo. In turn, /opt/foo is a softlink mstr/foo which points across an NFS mount point. And on the master node, foo is another softlink pkg/foo-1.23 which points to the actual application directory. That's three levels of symbolic link which must be resolved before foo can begin to load. Yet if the network is up, we don't even notice any lag, even on the slowest workstations on our network.
• One school of thought holds that the source subdirectory within the /opt/appname directory tree should be called "src". Another feels that it should keep the name it was given in the distribution tarfile. Which, of course, only applies to packages which have distribution tarfiles, such as freely-available tools, and even then, only to those tarfiles which do assign a subdirectory name to their contents. (Most do.) Our current practice is the latter--if the tarfile wants to put the source into an app-1.2.3 directory, we don't afterwards rename it to src. Still others feel that the source directory tree should not be contained within the /opt/appname tree at all, but should be somewhere else entirely. Which leads to the next point:
• One must consider exactly what is meant by all application files, when stating the philosophy that all such files be contained within the /opt/appname tree. Early MS-DOS applications (and even many modern MS-Windows apps, too) wanted everything related to an application to be contained within a single directory--binaries, configuration and other support files, and files the user created using that application. Word processor documents would be down in the wp's directory. Spreadsheets were in your 1-2-3 directory. This interpretation of the word "all" probably derives from the removable-media architecture of the earliest personal computer applications, where one had everything pertaining to an application on a floppy disk or cassette tape, and one loaded that medium when one wished to use that application. It was natural to keep the data files in the same place, a practice encouraged by the fact that only that application could operate on those data files anyway. Our current practice is to keep all installed files--those which came from the distribution archive, or were created during the build step(s)--in the /opt/appname tree. Log files (which are often kept in /var), data files, user configuration files, and other such files with changeable content are kept elsewhere. System configuration files are a bit of a gray area--we usually keep those in something like /opt/appname/etc, but as the next section shows, that's not always the obvious choice.
• Where to put the distribution tarfiles, for those packages which have them? This is a variant of the "all files" question. We originally placed them down into the /opt/appname directory, but now keep them all in /opt/pkg/archives.
• What is an "application", anyway? After we had installed the Apache web server on one system, we were then faced with the question of where to place the web document tree, server configuration files, and CGI binaries. We ultimately solved that problem by declaring the web site to be an "application", and created a separate /opt/web directory tree to contain those files. A large database application might be approached in the same way. But we occasionally still go looking in /opt/apache/conf for the configuration files. (That directory contains only the originals as distributed with Apache--the "real" ones are in /opt/web/conf.)
• What to do with files distributed in RPM or other packaging formats, which already contain their directory structures within them, from the top level down? And some applications expect or even demand to be installed in "common locations", or to find other things there. Well, nothing is perfect. RPM and other packaging programs usually offer their own methods of tracking packages, files, and so on, and we respect that. Forcing something into an /opt suit just because you can usually isn't the right approach. We only use this design where it can provide some benefit; if the software fights too hard against it, we give in quite easily. However, we're not above creating an /opt/appname directory even for obstreperous applications, and simply keep our local installation notes there. Sometimes we supplement this local documentation with one or more symbolic links which point to the application's actualinstallation directories. At least we always know where to look!
• Along the lines of the previous matter is the question of where X11 application default files should go. Our current usage places them in /opt/appname/lib/X11/app-defaults, with links in the convenience directory /opt/lib/X11/app-defaults. The newer X11 search path variables such as XUSERFILESEARCHPATH make such searches easier, but in practice, we find ourselves creating links in /usr/lib/X11/app-defaults just to keep things running right. All these "lib/X11/app-defaults" bits hark back to the day when /usr was king and X11 was a relatively new package that may not catch on. And then there's Jamie Zawinski's tirade about how app-defaults files are an abomination anyway. So this is still very much an open question.
• What should be the interaction between software configuration management systems (such as RCS, CVS, or ClearCase) and the /opt directory structure? Should the repositories be kept in /opt? How about working directories, for systems which support such a concept? For us, this only applies to locally-maintained packages (those in the site subdirectory), since we do not keep net-supported packages under configuration control. Our current practice is to keep the repositories (ClearCase VOBs or CVS roots) outside the /opt/appname tree, but place "working" versions of the files into the /opt/site/appname-n.nn directory, and "publish" them through an /opt/appname link as usual. However, in one case (one of our developers maintains a "net-supported" package himself) the CVS repository is kept in /opt/appname/cvs, and the source is in /opt/appname/src. This probably isn't too wise, and he has been planning to change it.