Advanced Module Development

This section carries on from WebminModuleDevelopment? , and explains some of the more advanced parts of module development such as access control, logging and integration with the Users and Groups module.

Module Access Control

Webmin supports a standard method for restricting which features of a module a user can access. For example, the Apache module allows a Webmin user to be restricted to managing selected virtual servers, and the BIND module allows user to be limited to editing records only in certain domains.

This kind of detailed access control is separate from the first level ACLs that control which users have access to which modules. As long as your module calls init_config, the Webmin API will automatically block users who do not have access to the entire module.

Module access control options are set in the Webmin Users module by clicking on a username and then on the name of a module. The options available are generated by code from the module itself (except for the Can edit module configuration? option, which is always present). When the user clicks on Save the form parameters are also parsed by code from the module being configured, before being saved in the Webmin configuration directory.

A module wanting to use access control must contain a file called acl_security.pl in its directory. This file must contain two Perl functions :

• acl_security_form(acl) This function takes a reference to a hash containing the current ACL options for this user, and must output HTML for form inputs to edit those ACL options. You must use the ui_table_row function to format your output.

• acl_security_save(acl, inputs) This function must fill in the given hash reference with values from the form created by acl_security_form. Form inputs are available in the second parameter to the function, which is in the same format as the %in hash created by the ReadParse function.

An example acl_security.pl file looks like :

require "foomod-lib.pl";

sub acl_security_form
{
my ($access) = @_;
print ui_table_row("Allow creation of websites?",
  ui_yesno_radio("create", $access->{'create'}));
}

sub acl_security_save
{
my ($access, $in) = @_;
$access->{'create'} = $in->{'create'};
}

Because these functions are called in the context of your module, the acl_security.pl file can require the common functions file used by other CGI programs in the module. This gives you access to all the standard Webmin functions, and allows you to provide more meaningful inputs. For example, when setting ACL options for the Apache module a list of virtual servers from the Apache configuration is displayed for the user to select from.

If a user has not yet had any ACL options set for a module, a default set of options will be used. These are read from the file defaultacl in the module directory, which must contain name_=_value pairs one per line. These options should allow the user to do anything, so that the admin or master Webmin user is not restricted by default.

To actually enforced the chosen ACL options for each user, your module programs must use the get_module_acl function to get the ACL for the current user, and then verify that each action is allowed. When called with no parameters this function will return a hash containing the options set for the current user in the current module, which is almost always what you want. For example :

#!/usr/local/bin/perl 
require 'foobar-lib.pl'; 
%access = &get_module_acl();
$access{'create'} || error("You are not allowed to create new websites"); 

When designing a module that some users will have limited access to, remember the user can enter any URL, not just those that you link to. For example, just doing ACL checking in the program that displays a form is not enough - the program that processing the form should do all the same checks as well. Similarly, CGI parameters should never be trusted, even hidden parameters that cannot normally be input by the user.

User and Group Update Notification

Webmin has a feature that allows the Users and Groups module tonotify other modules when a Unix user or group is added, updated or deleted. This can be useful if your module deals with additional information that is associated with users. For example, the Disk Quotas module sets default quotas when new users are created, and the Samba Windows File Sharing module keeps the Samba password file in sync with the Unix user list.

To have your module notified when a user is added, updated or deleted you must create a Perl script called useradmin_update.pl in your module directory. This file must contain three functions :

• useradmin_create_user(user) This function is called when a new Unix user is created. The user parameter is a hash containing the details of the new user, described in more detail below.

• useradmin_modify_user(user, olduser) This function is called when an existing Unix user is modified in any way. The user parameter is a hash containing the new details of the user, and olduser the details of the user before he was modified.

• useradmin_delete_user(user) This function is called when a Unix user is deleted. Like the other functions, the user hash contains the user's details.

The hash reference passed to each of the three functions has the following keys :

• user - The Unix username.
• pass - Encrypted password, perhaps using MD5 or DES.
• uid - User's ID.
• gid - User's primary group's ID.
• real - Real name for the user. May also contain office phone, home phone and office location, comma-separated.
• home - User's home directory.
• shell - Shell command to run when the user logs in.
• passmode - Set to 0 if the user has no password, 1 for a lock password, 2 for a pre-encrypted password, 3 if a new password was entered, or 4 if the password was not changed. * plainpass - The user's plain-text password, if available

In addition, if the system supports shadow passwords it may also have the keys :

• change - Days since 1970 the password was last changed.
• min - Days before password may be changed.
• max - Days after which password must be changed.
• warn - Days before password is to expire that user is warned.
• inactive - Days after password expires that account is disabled.
• expire - Days since Jan 1, 1970 that account is disabled.

When your functions are called, they will be in the context of your module. This means that your useradmin_update.pl script can require the file of common functions used by other CGI programs. The functions can perform any action you like in order to update other configuration files or whatever, but should not generate any output on STDOUT, or take too long to execute. An example useradmin_update.pl might look like :

do 'foobar-lib.pl'; 

sub useradmin_create_user
{
my ($user) = @_;
my $lref = &read_file_lines($users_file);
push(@$lref, "$user->{'user'}:$user->{'pass'}"); 
&flush_file_lines($users_file);
} 

Groups update information can also be passed to your module if the useradmin_update.pl script contains the functions useradmin_create_group , useradmin_modify_group and useradmin_delete_group. These take group hash references as parameters, which contain the keys :

• group - The group name.
• pass - Rarely-used encrypted password, in DES or MD5 format.
• gid - Unix ID for the group.
• members - A comma-separated list of secondary group members.

Internationalisation

Webmin provides module writers with functions for generating different text and messages depending on the language selected by the user. Each module that wishes to use this feature should have a subdirectory called lang which contains a translation file for each language supported. Each line of a translation file defines a message in that language in the format messagecode_=_Message in this language

The default language for Webmin is English (code en), so every module should have at least a file called lang/en. If any other language is missing a message, the English one will be used instead. Check the file lang_list.txt for all the languages currently supported and their codes. To change the current language, go into the Webmin Configuration module and click on the Language icon.

When your module calls the init_config function, all the messages from the appropriate translation file will be read into the hash %text. Thus instead of generating hard-coded text like this :

print "Click here to start the server<p>\n"; 

Your module should use the %text hash like so :

print $text{'index_startmsg'},"<p>\n"; 

The lang/en file would then have a line like :

index_startmsg=Click here to start the server

Messages from the appropriate file in the top-level lang directory are also included in %text. Several useful messages such as save, delete and create are thus available to every module.

In some cases, you may want to include some variable text in a message. Because the position of the variable may differ depending on the language used, message strings can include place-markers like $1, $2 or $3. The function text should be used to replace these place-markers with actual values like so :

print &text('servercount', $count),"<p>\n"; 

Your module's module.info file can also support multiple languages by adding a line with the key =desc=_code_ for each language, where code is the language code. So the German description for your module would be specified with a link like :

desc_de=Verwalten von Benutzer und Gruppen

You can also have a separate config.info file for each language, whose filename has the language code appended. So the file for german would be named config.info.de , and might contain the contents :

users_file=Die Benutzer-Datei,8
groups_file=Gruppen-Datei,8
show_groups=Details anzeigen Gruppe?,1,1-Ja,0-Nein

Help files can also be translated for each language, by creating separate files with the same prefixes as the English help, but with a language code before the .html extension. So the introductory help page for your module in German might be named intro.de.html .

In all cases, if there is no translation for the user's chosen language then the default (English) will be used instead.

File Locking

Webmin's API has several simple functions for locking files to prevent multiple programs from writing to them at the same time. Module programmers should make use of these functions in order to prevent the corruption or overwriting of configuration files in cases where two users are using the same module at the same time.

Locking is done by the function lock_file, which takes the name of a file as a parameter and obtains and exclusive lock on that file by creating a file with the same name but with .lock appended. Similarly, the function unlock_file removes the lock on the file given as a parameter. Because the .lock file stores the PID of the process that locked the file, any locks a CGI program holds will be automatically released when it exits. However, it is recommended that locks be properly released by calling unlock_file or unlock_all_files before exiting.

The following code shows how the locking functions might be used :

lock_file("/etc/something.conf"); 
open(CONF, ">>/etc/something.conf");
print CONF "some new directive\n"; 
close(CONF);
unlock_file("/etc/something.conf"); 

Locking should be done as soon as possible in the CGI program, ideally before reading the file to be changed and definitely before writing to it. Files can and should be locked during creation and deletion as well, as should directories and symbolic links before creation or removal. While this is not really necessary to prevent file corruption, it does make the logging of file changes performed by the program more complete, as explained below.

Many other programs also use .lock files for the same purpose, but most do not put their process ID in the file. If the lock_file function encounters a lock like this, it will wait until it is completely removed before obtaining its own lock, as there is no way to tell if the original process is still running or not.

If you want to just read from a file while being sure that no other process is corrupting it by writing to it, the lock_file function takes an optional second parameter that can be set to 1 to indicate a read-only lock. This will prevent other Webmin processes from writing to the same file, but will not block read locks by other scripts.

Safe File Writes

If your module writes to critical system configuration files, you should use IO functions built into the Webmin API instead of Perl's standard open function. These protect files from problems like the failure of a script part way through writing a file, lack of disk space, or un-expected termination.

To open a file for writing safely, use the open_tempfile function. This writes to a temporary file in the same directory until it is closed with close_tempfile, at which point the target file is over-written. For example :

open_tempfile(CONFIG, ">/etc/foo.conf");
print_tempfile(CONFIG, "foo bar\n");
close_tempfile(CONFIG);

The print_tempfile function behaves like Perl's built-in print, but immediately calls error to terminate the script if the write fails due to lack of disk space or some other reason.

Functions in the Webmin API that write to files like flush_file_lines , write_file and replace_file_line already call the safe file IO functions internally.

Action Logging

Webmin has support for detailed logging by CGI programs of the actions performed by users for later viewing in the Webmin Actions Log module. Logs are also written to the file /var/webmin/miniserv.log, this does not contain the information required to work out exactly what each Webmin user had been doing. To improve on this, Webmin now logs detailed information to the file /var/webmin/webmin.log and optionally to files in the directory /var/webmin/diffs. Note that nothing will be recorded in this file if logging is not enabled in the *Webmin Configuration* module.

The function webmin_log should be called by CGI programs after they have successfully completed all processing and file updates. The parameters taken by the function are :

• action - A short code for the action being performed, like 'create'.
• type - A code for the type of object the action is performed to, like 'user'.
• object - A short name for the object, like 'joe' if the Unix user 'joe' was just created.
• params - A hash ref of additional information about the action.
• module - Name of the module in which the action was performed, which defaults to the current module.
• host - Remote host on which the action was performed. You should never need to set this (or the following two parameters), as they are used only for remote Webmin logging.
• script-on-host - Script name like create_user.cgi on the host the action was performed on.
• client-ip - IP address of the browser that performed the action.

All of these parameters can contain any information you want, as they are merely logged to the actions log file and not interpreted by webmin_log in any way. For example, a module might call the function like this :

lock_file("/etc/foo.users"); 
open(USERS, ">>/etc/foo.users"); 
print USERS "$in{'username'} $in{'password'}\n";
close(USERS);
unlock_file("/etc/foo.users");
webmin_log("create", "user", $in{'username'}, \%in); 

Because the raw log files are not easy to understand, Webmin also provides support for converting detailed action logs into human-readable format. The Webmin Actions Log module makes use of a Perl function in the file log_parser.pl in each module's subdirectory to convert logs records from that module into a readable message.

This file must contain the function parse_webmin_log, which is called once for each log record for this module. It will be called with the following parameters :

• user - The Webmin user who run the program that generated this log record.
• script - The filename of the CGI script that generated this log, without the directory.
• action - Whatever was passed as the action parameter to webmin_log to create this log record.
• type - Whatever was passed as the type parameter to webmin_log.
• object - Whatever was passed as the object parameter to webmin_log.
• parameters - A reference to a hash the same as the one passed to webmin_log.
• long - If non-zero, this indicates that the function is being called to create the description for the Action Details page, and thus can return a longer message than normal. You can ignore this if you like.

The function should return a text string based on the parameters passed to it that converts them into a readable description for the user. For example, your log_parser.pl file might look like :

require 'foobar-lib.pl'; 

sub parse_webmin_log 
{
my ($user, $script, $action, $type, $object, $params, $long) = @_;
if ($action eq 'create') {
    return &text('log_create', $user);
    }
elsif ($action eq 'delete') {
    return &text('log_delete', $user);
    }
else {
    return undef;
    }
} 

Because the log_parser.pl file is read and executed in a similar way to how the acl_security.pl file is handled by the Webmin Users module, it can require the module's own library of functions just like any module CGI program would. This means that the text function and %text hash are available for accessing the module's translated text strings, as in the example above.

Webmin can also be configured to record exactly what file changes have been made by each CGI program before calling webmin_log. Under Logging in the Webmin Configuration module is a checkbox labeled Log changes made to files by each action which when enabled will cause the webmin_log function to use the diff command to find changes made to any file locked by each program.

When logging of file changes is enabled, the Action Details page in the actions log module will show the diffs for all files updates, creations and deletions by the chosen action. If locking of directories and symbolic links is done as well, it will show their creations and modifications too.

As well as having their file changes logged, programs can also use the common functions system_logged, kill_logged and rename_logged which take the same parameters as the Perl system, kill and rename functions, but also record the event for viewing on the *Action Details* page. There is also a backquote_logged function which works similar to the Perl backquote operator (it takes a command and executes it, returning the output), but also logs the command. If these functions are used they must be called before webmin_log for the logging to be actually recorded, as in this example :

if ($pid) { 
    kill_logged('TERM', $pid); </blockquote>
    }
else {
    system_logged("/etc/init.d/foo stop");
    }
webmin_log("stop"); 

Pre and Post Install Scripts

Webmin allows modules to define scripts that will be run after a module is installed and before it is un-installed. If your module contains a file called postinstall.pl , the Perl function module_install in this file will be called after the install of your module is complete. Because it is executed in the module's directory, it can make use of the common functions library, like so :

require 'foobar-lib.pl'; 

sub module_install 
{
if (!-r "$config_directory/somefile") {
    copy_source_dest("$module_root_directory/somefile", "$config_directory/somefile");
    }
}

The function will be called when a module is installed from the Webmin Configuration or Cluster Webmin Servers modules, when a module RPM or Debian package is installed, or when the install-module.pl command is used. It will also be called when your module is upgraded or when Webmin is upgraded, so make sure it doesn't over-write.

Similarly, if your module contains a file called uninstall.pl, the Perl function module_uninstall in that file will be called just before the module is deleted. This can happen when it is deleted using the Webmin Users or Cluster Webmin Servers modules, or when the entire of Webmin is uninstalled. The uninstall function should clean up any configuration that will no longer work when the module is uninstalled, such as Cron jobs that reference scripts in the module.

Installed Checks

Webmin module writers can call the API function foreign_installed to check if the server or service managed by some other module is installed on the system. If you are writing a module that manages some server, you can add a file to your module's directory that provides this information to callers. In addition, this determines if your module appears under Un-used Modules on the left menu.

This is done by creating a script called install_check.pl that contains the single Perl function is_installed. This function takes a mode parameter with the same meaning as the parameter passed to foreign_installed, and must interpret it in the same way. Because most modules don't require an extra level of configuration before use, your function can just return 0 if the server is not installed, or mode + 1 if it is.

This example code shows how an is_installed function might be written :

do 'foobar-lib.pl';

sub is_installed
{
local $mode = $_[0];
if (!-r $config{'foo_config_file'}) {
        return 0;
        }
else {
        return $mode + 1;
        }
}

Functions in Other Modules

The standard Webmin modules contain a vast number of useful functions for parsing and manipulating the configuration files for Apache, BIND, Unix Users and so on. If your module needs to configure these servers as well in some way, it makes sense to make use of existing functions in the standard modules.

Because the standard modules have typically already been configured with the correct paths for files like httpd.conf and squid.conf, their functions will use those paths when you call them to read and write configuration files. The actual %config settings for another module can also be accessed, so that your module knows what commands to use to apply changes to or start some server like Apache or Squid.

When you first load the library for some other module with the foreign_require function, it is actually executed in a separate Perl module namespace. All of your module's CGI programs and its library will be in the their own namespace, but other foreign module's functions will be put in a namespace with the same name as the Webmin module. This means that you can call those functions with code like useradmin::list_users() , and access global variables like $useradmin::config{'passwd_file'}. This Perl namespace separation ensures that functions and globals with the same names can exist in both your and the foreign module, without any clashes. Some things are shared between all modules though, such as caches used by get_system_hostname, load_language, read_file_cached and get_all_module_infos, so that loading the library of a new module with foreign_require is not too slow.

Documentation on functions available in other modules can be found on the page TheWebminAPI.

Remote Procedure Calls

Webmin has several API functions for executing code on remote Webmin servers. They are used by some of the standard modules (such as those in the Cluster category) to control multiple servers from a single interface, and may be useful in your own modules as well. These functions, all of which have names starting with remote, let you call functions, evaluation Perl code, and transfer data to and from other system running Webmin.

Before a "master" server can make RPC calls to a remote host, it must be registered in the Webmin Servers Index module on the master system. The Link type field must be set to Login via Webmin and a username and password entered. The user specified should be root or admin, as others are not by default allowed to accept RPC calls.

RPC is usually used to call functions in other modules on a remote system, or common functions. This is done with the remote_foreign_call function, but before it can be used remote_foreign_require must be called to load the library for the module that you want to call. This is very similar to calling functions in other local modules with the foreign functions, explained above.

A piece of code that edits a user on a remote system might look like :

$server = "www.example.com"; 
$user = "joe";
remote_foreign_require($server, "useradmin", "user-lib.pl");
@users = remote_foreign_call($server, "useradmin", "list_users");
($joe) = grep { $_->{'user'} eq $user } @users;
if ($joe) {
    $joe->{'real'} = "Joe Bloggs";
    &remote_foreign_call($server, "useradmin", "modify_user", $joe, $joe);
    }

Of course, you need to be familiar with the available functions in other modules, and also to be sure that the module that you want to call is actually installed and of the right version.

All parameters passed to remote functions are converted to a serialized text form for transfer to the remote server, and any return value is also sent back in serialized form. The API functions serialize_variable and unserialize_variable are used, but the process is hidden from both the caller and the remote function - they only see scalars and references in their original format. One thing to look out for is circular references though - trying to send a structure that contains links to itself (such as a doubly-linked list) will fail due to the shortcomings of the serialize_variable function. Also, try to avoid using extremely large parameters, such as strings over 1 MB in size, as serialization may make them massive.

Parameters that are references to hashes, arrays or scalars that would normally be filled in by the function will not be transferred properly. For example, the read_file function normally fills in the hash referenced by its second argument with the contents of a file. This will not work when it is called remotely, as all parameters and anything that they refer to are 'copied' to the other system.

The remote_eval function can be used to execute an arbitrary block of Perl code on a remote system, which allows you to do things that calls to remote functions cannot. It is the only way to call native Perl functions such as unlink, to read and write arbitrary format files, set global variables and properly call functions that set their parameters. Whatever the Perl code evaluates to will be sent back returned by this function. This example shows remote_eval in use :

$data = &remote_eval($server, "useradmin",
    "rename('/etc/foo', '/etc/bar');\n".
    "local \%data;\n".
    "&read_file('/etc/bar', \\%data);\n".
    "return \\%data;\n");
&write_file('/etc/foo', $data);

As you can see, proper quoting is necessary when constructing the Perl code string, so that any variable symbols (such as $, % and @) are escape, as is the \ character. The second module parameter to remote_eval can be set to undef, which indicates that the code should be executed in the global Webmin context, rather than in any module's.

The functions remote_read and remote_write can be used to transfer the contents of an entire file between the master and remote systems. They are must faster than reading in the file and encoding it for use in the remote_foreign_call or remote_eval functions, as the file is transferred un-encoded over a separate TCP connection.

If your module makes RPC calls, you may want the user to select a system to make calls to from a menu. A list of the names of all those available can be obtained from the Webmin Servers Index module with code like this :

foreign_require("servers", "servers-lib.pl"); 
@allservers = servers::list_servers();
@rpcservers = map { $_->{'host'} } grep { $_->{'user'} } @allservers;

In addition, all of the remote functions will accept undef for the server parameter. This indicates that the local system should be used, which never needs to be defined in the Webmin Servers Index module. This is how all of the Cluster category modules can include the this server option in their lists of hosts to manage.

Creating Usermin Modules

Usermin has a very similar architecture to Webmin, and so its modules have an almost identical design to Webmin modules. The main difference is that Usermin is designed to be used by any Unix user on a server to perform tasks that they could perform from the command line. Any third-party Usermin modules should be written with this in mind.

By default, module CGI programs are run as root, just like in Webmin. This is necessary because some tasks (like changing passwords) can only be done as root. However, most Usermin modules do not need super-user privileges and so should call the switch_to_remote_user API function just after calling init_config , in order to lower privileges to those of the logged-in user.

Usermin module can have global configuration variables that are initially set from the config files in the module directory, and are available in %config. However, these variables are never editable by the user - they can only be set in the Usermin Configuration module in Webmin.

Per-user configurable options are supported though, using a different mechanism. When the standard create_user_config_dirs function is called, the global hash %userconfig will be filled with values from the following sources, with later sources overriding earlier ones :

1. The defaultuconfig file in the module directory This should contain the default options for this module for all users, to be used if no other settings are made by the user or system administrator.
2. The file defaultuconfig in the module's directory under /etc/usermin . This contains defaults for the module on this system, as set by the system administrator using the second form in the Usermin Module Configuration page feature in the Usermin Configuration Webmin module.
3. The file config in the modules' directory in .usermin under the user's home directory. This contains options chosen by users themselves.

The editors for the system-wide and per-user configuration variables are defined by the uconfig.info file in the module directory. This file has the exact same format as the config.info file used for Webmin and Usermin global configuration, explained elsewhere in this document.

If you create your own Usermin module, it should be packaged in exactly the same way as a Webmin module (as a .tar or .tar.gz file). However, the module.info file must contain the line usermin=1 so that it cannot be installed into Webmin where it would not work properly.

If your module needs to store additional data in the user's .usermin directory, it should call the create_user_config_dirs API function first to ensure that directory exists. This in turn sets the $user_config_directory and $user_module_config_directory global variables, which contain paths to the .usermin directory and its per-module sub-directory.