Configuration management

Introduction

Configuration management is quickly becoming a very important issue. Having programs which do cool stuff is great, but we need to store their configuration as well. We see more and more different configuration systems being introduced all the time, which is not very practical. This text introduces a general configuration management system which flexible enough to be used for all kinds of applications.

Configuration Data

The configuration space

All configuration information is stored in what I call the configuration space. This is a database with a special design which resembles the method we look at configuration information. This is done by defining a hierarchy of information. Each package receives its own space in the hierarchy. Each package is free to use a flat space, or divide its space further into sub-hierarchies. If multiple packages share a common purpose they may use a shared toplevel hierarchy, preferably with the same name as a shared (virtual) package name (for example, both mutt and elm can use mail-reader, strn and nn could use news-reader). This shared tree can also be used as a default, ie a variable news-reader/nntpserver can be used by strn if strn/nntpserver does not exist.

Each variable in the configuration space has some information associated with it. Most importantly, it has a value. It also may have a set of flags and a set of substitution data.

Templates

Each variable in the configuration space is associated with some meta-data. The minimum meta-data associated with a variable is: long and short description, type, and default value. The meta-data is essentially static; the protocol described below does not allow it to be changed.

The meta-data exists in a space with similar naming properties to the configuration space described above, and typically one variable in the configuration space will have associated with it metadata with the same name in the meta-data space. However, this need not be the case; many different variables can all be associated with the same meta-data. In effect the meta-data serves as a template for the configuration variable.

Template information

So, what do we need to store in a variable template? Of course we need a name to identify the template. Template names are made up of components separated by the character `/' (slash). Each component is limited to alphanumerics and `+' `-' `.' `_' (plus, minus, full stop, underscore).

A type is also needed so data can be verified. Here is a table of common types; implementations are free to make up more.

Table 1. Available data types

Type	Description
string	Holds any arbitrary string of data.
boolean	Holds "`true`" or "`false`".
select	Holds one of a finite number of possible values. These values must be specified in a field named `Choices:`. Separate the possible values with commas and spaces, like this: `Choices: yes, no, maybe`
multiselect	Just like the select data type, except the user can choose any number of items from the list. This means that the `Default:` field and the actual value of the question may be a comma and space delimited list of values, just like the `Choices:` field.
note	This template is a note that can be displayed to the user. As opposed to text, it is something important, that the user really should see. If it is not possible to display it, it might be saved to a log file or mailbox for them to see later.
text	This template is a scrap of text that can be displayed to the user. It's intended to be used for mostly cosmetic reasons, touching up around other questions that are asked at the same time. Unlike a note, it isn't treated as something the user should definitely see. Less complex frontends may refuse to ever display this type of element.
password	Holds a password. Use with caution. Be aware that the password the user enters will be written to a database. You should consider clearing that value out of the database as soon as is possible.
title	Holds a (short) string that can be displayed using the SETTITLE command. Only the value of the short description will be used.

Of course a default value is useful as well, and finally we need a description of the variable. We actually use two descriptions: a short one (limited to 50 characters or so) and an extended one.

The extended description may be word-wrapped by the FrontEnd. To make separate paragraphs in it, use . on a line by itself to separate them. Text in the extended description that is prefaced by additional whitespace will not be wordwrapped. Both the description and extended description may have substitutions embedded in them. Ie, ${foo}. These will be expanded when the descriptions are displayed.

This information is stored in a template file that consists of stanzas in a rfc-822 compliant format, separated by blank lines. Here is an example:

Template: hostname
Type: string
Default: debian
Description: unqualified hostname for this computer
 This is the name by which this computer will be known on the network. It
 has to be a unique name in your domain.

Template: domain
Type: string
Description: domain for this computer
  This is the domain your computer is a member of. Typically it is
  something like "mycompany.com" or "myuniversity.edu".

For localization, the description field (and also the choices field of a select or multiselect type question, and the default field of a string or password type question) can be supplemented with versions for other languages. These are named Description-ll, Description-ll_LL, Description-ll_LL.encoding and so on.

Configuration frontends

Of course applications can use the database and meta-database directly. But there should be a simple system to interact with the user that is simple and modular enough to be used with systems ranging from shell-scripts to Fortran programs. To do this we define a general frontend that can be driven using the simplest and most common form of communication: stdin and stdout.

Using this simple form of communication gives us a great advantage: it becomes easy to change the frontend. That means the user can switch between a console, a graphical or even a web-interface at will.

Besides being able to switch between types of frontends there is another important aspect of a good user interface: user friendliness. We have to account for the fact that some users know more then others and change the information we show or ask from the user. We do this by giving everything a priority and giving the user control over what kind of questions he wants to see. Experts can request to see everything, while novices get the option of only seeing only important questions. Finally there is an option to simply skip all questions, so it becomes possible to do automatic configuration using default values or values that are downloaded into the database from a remote location. This makes it simple for example to install and manage clusters or lab rooms or do installs for dummies.

Communication with the frontend

This communication between the frontend and the application should be as simple as possible. Since most IO implementations default to line-buffered IO, so we use a simple language where each command is exactly one line.

After sending each command to stdout, the client should read one line from stdin. This is the response to the command, and it will be in the form of a number followed by whitespace and an optional string of text. The number is the status code, while the text provides additional information.

Table 2. Numeric status codes

Range	Description
0	success
1-9	reserved
10-19	invalid parameters
20-29	syntax errors
30-99	command-specific return codes
100-109	internal errors
110-255	reserved

Here are the currently supported commands.

VERSION number

This exchanges with the frontend the protocol version number that is being used. The current version is 2.1. Versions in the 2.x series will be backwards-compatible. You may specify the protocol version number you are speaking. The frontend will return the version of the protocol it speaks. If the version you specify is too low, this command will return the numeric return code 30.

CAPB capabilities

This exchanges with the frontend a list of supported capabilities Capabilities both the frontend and your confmodule support may be used; the capabilities supported by the frontend are returned by this command.

Table 3. Currently used capabilities

capability	description
backup	Backing up to a previous step is supported.
escape	The frontend expects commands sent to it to have backslashes and newlines quoted as `\\` and `\n` respectively and will in turn quote backslashes and newlines in its replies. See debconf-escape(1).
multiselect	The multiselect data type is supported. You do not need to check this capability if you depend on any modern version of debconf.

SETTITLE template

You can use this command to set a title in the frontend. This may appear in different ways, depending on the frontend being used, for example it might change the title of the frontend's window. If you don't specify anything, a title will automatically be generated.

Using a template has the advantage that titles are translatable and that they can be maintained in the same place as other text displayed to users.
TITLE string

Similar to SETTITLE, but takes a string instead of a template as parameter. Consequence is that the title will not be translatable, unless some other mechanism (like gettext) is used.
STOP

This command tells the frontend you're done talking to it. Typically the frontend can detect the termination of your program and this command is not necessary.

INPUT priority question

This tells the frontend to display a question (or other type of item) to the user. question is the name of the item to display, all other information about the item is retrieved from the templates described previously. priority is how important it is that the user be prompted. The frontend need only ask this question if the priority is high enough. The question is not displayed until a go command is given. This allows us to ask multiple questions in a single screen. Once a question has been displayed to the user and the user has provided input, the frontend will set the seen flag.

Table 4. Supported priorities

Priority	Description
low	Very trivial items that have defaults that will work in the vast majority of cases.
medium	Normal items that have reasonable defaults.
high	Items that don't have a reasonable default.
critical	Items that will probably break the system without user intervention.

Note that the frontend decides if the user is actually prompted or not. If the user has already answered a question, they are normally not asked it again even if input is called again. And if the user is ignoring low priority items, they will not see them. In either of these cases, this command returns the numeric return code 30.

BEGINBLOCK
ENDBLOCK

Some frontends are able to display a number of items to the user at once. To do this, they need to be given blocks of input commands, enclosed in the BEGINBLOCK and ENDBLOCK commands. Blocks can be nested and very advanced frontends may use this as a user interface hint.

Note: There is an implicit block around any set of INPUT commands that are not enclosed in an explicit block.
GO

Shows the current set of accumulated items to the user and lets them fill in values, etc. If the backup capability is supported and the user indicates they want to back up a step, this command returns the numeric return code 30.
CLEAR

Clears the accumulated set of INPUT commands without displaying them to the user.
GET question

Ask the frontend to tell you how the user answered a question. The value is returned to you.
SET question value

Set the answer of a question to a value.
RESET question

Reset the question to its default value. This includes resetting flags to their defaults.
SUBST question key value

Questions (and other items) can have substitutions embedded in their descriptions (and, currently in their choices fields). These substitutions look like "${key}". When the question is displayed, the substitutions are replaced with their values. This command can be used to set the value of a substitution.
FGET question flag

Questions (and other items) can have flags associated with them. The flags have a value of "true" or "false". This command returns the value of a flag.
FSET question flag value

This sets the state of a flag on a question. Valid states for the flag are "true" and "false".

One common flag is the "seen" flag. It is normally only set if a user already seen a question. Typically, frontends only display questions to users if they have the seen flag set to "false". Sometimes you want the user to see a question again -- in these cases you can set the seen flag to false to force the frontend to redisplay it.

Note that as a special convenience behavior, frontends will redisplay already seen questions if the question was first seen by the user in the same confmodule run. This makes it easy for a confmodule to back up to previous questions without having to reset the seen flag.
METAGET question field

This returns the value of any field of a question (the description, for example).
REGISTER template question

This creates a new question that is bound to a template. By default each template has an associated question with the same name. However, any number of questions can really be associated with a template, and this lets you create more such questions.
UNREGISTER question

This removes a question from the database.
PURGE

Call this in your postrm when your package is purged. It removes all templates and questions your package has generated.

Debian install-time configuration

Debian has had an excellent packaging system for a long time now. There is one thing missing though: a system to handle the configuration of packages so we don't have to stop the installation every time a package needs some data from the user or wants to show some information.

We want to make a package which does not break older dpkg's, and we want to be able to get the configuration information before the package is unpacked. To do this we add two new files, config and templates, to the control.tar.gz of a .deb package. Since all installation-software (apt, dselect, dpkg) download the package before installing it, we can extract this before the package is unpacked.

The templates file lists the templates for variables that this package uses. This is done using the format as used in the example in the section on templates.

The config-file contains a new element, which I call the configmodule. This is a program that will determine the configuration before the package is unpacked. This means it is usually run before the preinst, and before the package is unpacked!

Note: Please see debconf-devel(7) for details.

This is done to make sure that we can use the desired configuration in the preinst if necessary.

How does the configmodule get its information? The configmodule needs a way to retrieve information from the configuration space, ask the user for information if necessary, etc. But we don't want to implement a user interface for each package. To solve this we use a separate frontend as specified in the section on frontends.