 mquota
-- A Passive Disk Quota System
Michael Montero
** missed drop char **quota was developed at Community Connect
Inc. in November 2000. We needed a quota system under reiserFS 3.5.26
(running Linux kernel 2.2.16), and reiserFS itself did not support
such a facility. Furthermore, we had special requirements for our
quota system because most of our users would be accessing files
using Samba and atalk rather than logging directly into a Linux
server.
Quota systems are essential management tools when preserving disk
space is a concern and when resources are shared among many users.
For example, Internet Service Providers (ISPs) have long relied
on quota systems to manage user accounts. Quota systems essentially
allocate a fixed amount of disk space for a particular user (or
group) and ensure that the amount of data saved never exceeds the
allotted amount. Under Linux, the built-in Quota system (http://www.linuxdoc.org/HOWTO/Quota.html)
is an effective method for implementing such policies.
Linux's Quota allows an administrator to set up a total number
of inodes or disk blocks for a user or group of users to utilize.
Attempts to save data when the quota limit is reached or exceeded
will fail, thus prompting the user to make space for new data by
deleting or moving files. Quota supports soft limits (the user is
issued warnings about space consumption), hard limits (the user
is no longer allowed to write to disk), and grace periods (an amount
of time during which the soft limit will not be enforced). Quota
has been incorporated in the Linux kernel since 1.3.8x and is now
standard for the 2.0 kernel. A quota facility is available for ext2,
ext3, and reiserFS (http://www.namesys.com/).
How mquota Works
mquota (http://www.bsdpro.com/scripts.php) works by summing
up disk utilization at various times throughout the day (through
cron) and taking action based on a user's or group's consumption.
mquota is therefore a passive quota system because it will not immediately
impede a user's ability to write data. (This is in contrast
to an active quota system like Quota that can be configured to prevent
a user or group from writing data as soon as the quota is reached
or exceeded.)
Since users may not be logged into the server running mquota,
we devised two methods for informing users of their disk consumption
and availability. The first method incorporates sending email messages
to a user when they are 75% or 90% of their quota. This message
is used as a warning to notify the user of the available disk space.
The second method involves writing a YOUR_QUOTA file in every directory
being managed by mquota. The YOUR_QUOTA file contains the directory
being monitored, the total disk space used, the total quota, and
the percent used.
In addition to the report sent to a user or a group, centralized
reports are created so an administrator can quickly observe disk
utilization. Each time quotas are calculated, mquota writes out
two files:
/var/log/mquota/mquota_dept.report
/var/log/mquota/mquota_user.report
To clarify the use of the term "dept", we aligned our Linux
groups (in /etc/group) with our functional business units or
departments. To keep the terminology consistent, we prefer the use
of "dept" instead of "group". The report files
contain the directory being monitored, the total disk space consumed,
the total quota, and the percent used. We have used these reports
in the past to determine whether disk space was becoming tight or
whether we were too strict or too lenient with quota sizes.
At the heart of mquota is the Linux command:
/usr/bin/du
du is used to estimate file-space usage. When passed the -b
option, du summarizes its output in bytes. Specifying -c
will produce a grand total. An example execution of du is:
/usr/bin/du -bc /some/user/or/group/directory
The output appears like this (in this case, we ran it against /tmp):
4069 /tmp/directory1
8192 /tmp/directory2
40960 /tmp
40960 total
Two other options to use with du are -s to create a
summary, and -h to put the results in "human-readable"
form. The output of:
/usr/bin/du -bcsh /tmp
Looks like this:
40k /tmp
40k total
Note that mquota only uses the -b, -c, and -s
options.
To put it all together, I'll start by listing all of the
files and scripts we use and describe each.
Configuration
/etc/mquota/mquota.dept -- Contains information on
department directories to monitor, their total quota in megabytes,
and the contact person for notification emails.
/etc/mquota/mquota.user -- Contains a user name, the
path to his home directory, and his total quota in megabytes.
Libraries
mquota.pl -- Perl library containing global variables
and subroutines.
Scripts
adddeptmquota -- An administrative script for adding
a new dept quota to the configuration file.
addusermquota -- An administrative script for adding
a new user quota to the configuration file.
checkdeptquota -- cron that sums up the disk space
used by the depts and takes action based on configuration file.
checkuserquota -- cron that sums up the disk space
used by the users and takes action based on configuration file.
resetdeptdir -- Used to give write permission back
to a dept directory after it has been disabled by mquota.
resetuserdir -- Used to give write permission back
to a user directory after is has been disabled by mquota.
(Several of the other supporting scripts written were omitted
because of their complexity or because they were solutions to problems
for which we had very specific requirements.)
Before discussing the configuration files, it is worth mentioning
that all of the scripts are "intelligent". For instance,
they will not allow you to add a user if the user does not have
an UNIX-level account, or if the user already has an mquota. In
the latter instance, the user will not be added twice.
Both configuration files are in plain text and can be easily modified
manually. The header for these files read:
# mquota.dept
#
# This file is used to manage the quotas for specific departments.
# To add or remove values to this file, either manually add an entry
# or use the administrative scripts located in /usr/local/cci/mquota.
# The format for this file is:
#
# [department]\t[directory]\t[quota in megabytes]\t[contact user name]\n
#
# mquota.user
#
# This file is used to manage the quotas for specific users. To add or
# remove values to this files, either manually add an entry or use the
# administrative scripts located in /usr/local/cci/mquota. The format
# for this file is:
#
# [user name]\t[directory]\t[quota in megabytes]\n
#
You might be wondering how we send email messages without email addresses.
The user names on our servers are unified with user names throughout
our system. Therefore, my user name would be mmontero and appending
@mail.communityconnect.com would create my email address. mquota
is designed to do just this.
At the heart of the system are checkdeptquota and checkuserquota,
both of which are set to run out of cron. We have chosen to check
quotas hourly (and conveyed the significance of this to our users)
but could modify how often user and group directories are affected
by increasing the frequency with which these two scripts execute.
In this way, the "activeness" of mquota can be specified.
Our cron entries look like this:
############################################################
#
# Crons for mquota disk quota system.
#
# Michael Montero Nov 29, 2000
#
#
# Check department drives for quota limits.
#
0 * * * * /usr/local/cci/mquota/checkdeptquota >
/var/log/mquota/checkdeptquota.log 2>&1
#
# Check user drives for quota limits.
#
0 * * * * /usr/local/cci/mquota/checkuserquota >
/var/log/mquota/checkuserquota.log 2>&1
############################################################
Since these two scripts produce output about what they are doing,
we redirect the output to log files so we can examine them if things
need investigation. Note that the output from these is not the reports
we generate for an administrator. One of the functions these scripts
perform is writing the report directly to a predetermined report file.
checkdeptquota and checkuserquota perform several
tasks during each execution. The first is to open the configuration
file specified in mquota.pl defined in either $_MQUOTA_DEPT_FILE
or $_MQUOTA_USER_FILE. The report file specified in mquota.pl
defined in either $_MQUOTA_DEPT_REPORT or $_MQUOTA_USER_REPORT
is then opened for writing. Each line of the configuration is then
read and parsed. If any data are missing for an entry, the script
immediately recognizes an error and aborts. If all data are provided,
the du command is executed:
$command = "$_DU $deptDir |" ;
if ( !open( COMMAND, $command ) )
{
print "$0::main() -> fatal, cannot execute command $command. Aborted.\n" ;
exit -1 ;
}
$junk = <COMMAND> ;
$total = <COMMAND> ;
close( COMMAND ) ;
($_DU is defined in mquota.pl as /usr/bin/du -sbc.)
The -s option produces two lines of output: the first is the
total disk usage and the name of the directory passed to du,
and the second is the total disk usage and the word "total".
We discard the first line (assigning it to a variable called $junk)
and assign the total to $total. We parse the $total
variable to extract only the amount of disk space used. Because we
can be dealing with disk spaces in several hundred megabytes and larger,
we use the Math::BigInt and Math::BigFloat libraries of Perl. Various
mathematical operations are done on these numbers, and these libraries
provide subroutines for doing so.
Next, the disk space being used by either the user or group is
compared against the allotted value specified in the configuration
file. We determine the percentage of disk space used to see whether
it is between $_MQUOTA_PERCENT_WARN and $_MQUOTA_PERCENT_FATAL,
over $_MQUOTA_PERCENT_FATAL, but not quota or over quota.
$_MQUOTA_PERCENT_WARN is currently defined as 75%, and $_MQUOTA_PERCENT_FATAL
as 90%. If any of these thresholds are reached, email messages are
sent to the user name extracted from the configuration file. If
the user or group is over quota, their directory permissions for
writing are revoked using this subroutine:
sub mquota_overquota
{
my $dir = $_[ 0 ] ; # the directory to disable
$command = "( cd $dir ; $_MQUOTA_FIND_COMMAND . -type d -exec /bin/chmod 2550 {} \\; )" ;
system( $command ) ;
chmod( $_MQUOTA_DIR_OVER_PERMS, $dir ) ;
}
$_MQUOTA_FIND_COMMAND is defined as /usr/bin/find and
$_MQUOTA_DIR_OVER_PERMS as 02550 in mquota.pl.
After these checks have been performed, the YOUR_QUOTA file
is written to the directory extracted from the configuration file.
printf() is used to provide clean output:
printf YOURQUOTA "%-45s %12s %12s %7s\r\n",
"Directory", "Total Used", "Quota", "% Used" ;
printf YOURQUOTA "%-45s %12s %12s %7s\r\n\r\n",
"-----------", "-------------", "--------", "---------" ;
printf YOURQUOTA "%-45s %12s %12s %7s\r\n",
$directory, $total, $total_quota, $percent ;
The file generated by the code above is readable under Windows, Mac,
or Linux and appears the same in all text file readers (that we've
tested and are accustomed to using). Using almost identical code,
a line is added to the summarization report generated.
The YOUR_QUOTA file for my home directory looks like this:
Directory Total Used Quota % Used
--------- ---------- ----- ------
/mnt/raid/RAID00_5/people/mmontero +407769088 +524288000 77.77%
The summarization report for depts looks like this (I provide only
a snippet below):
Directory Total Used Quota % Used
--------- ---------- ----- ------
/mnt/raid/RAID00_5/depts/cci +4238495744 +5242880000 80.84%
/mnt/raid/RAID00_5/depts/mgmt +463360 +104857600 .4418%
/mnt/raid/RAID00_5/depts/bizdev +4608 +262144000 .0017%
/mnt/raid/RAID00_5/depts/mservices +129280512 +262144000 49.31%
/mnt/raid/RAID00_5/depts/hr +129034752 +524288000 24.61%
/mnt/raid/RAID00_5/depts/admins +4608 +52428800 .0087%
/mnt/raid/RAID00_5/depts/pdt +69592576 +1048576000 6.636%
/mnt/raid/RAID00_5/depts/aa +229409792 +8912896000 2.573%
/mnt/raid/RAID00_5/depts/bp +52784640 +8912896000 .5922%
/mnt/raid/RAID00_5/depts/mg +328064000 +8912896000 3.680%
/mnt/raid/RAID00_5/depts/pr +251612160 +524288000 47.99%
/mnt/raid/RAID00_5/depts/engineering +4463104 +2621440000 .1702%
/mnt/raid/RAID00_5/depts/sa +46080 +10485760000 .0004%
There are some disadvantages to mquota. We have gotten around many
of them because of how we have structured our servers, permissions,
groups, and users. But shortcomings are still worth discussing.
The first obvious shortcoming is how "write permission"
is turned off. Merely changing the permissions on the users'
or groups' directories isn't always effective. It is still
possible for a user to change the permissions back. We were able
to circumvent this issue because most of the users of our systems
do not have command-line access. Our users are typically reading
and writing files through either Samba or atalk. A better enhancement
to this would be to chown all of the directories and files
to root, thus completely prohibiting users from their data.
Since we scheduled the crons to run every hour, it is possible
for a single user to consume all of the disk space available. As
long as they've filled up the disks before the cron runs, they
will effectively use all the resources available. One obvious solution
to this is to run the check scripts more frequently. A better solution,
though, is to actively monitor the amount of data written and removed
from disk.
Once a user has exceeded his quota, he is unable to continue to
write or remove data until he has seen an administrator to reset
his directory. A better implementation of a quota system would allow
files to be removed (thus bringing oneself under quota) but not
written. This was not essential to us because it gave us the ability
to monitor our users' habits and educate them on the significance
of file management (which we felt was far more important then not
exceeding a quota). Typically, we have had issues where a particular
user will require our intervention two or three times before our
lecture becomes boring. Thereafter, they choose to more effectively
manage their files.
We now install mquota on all of our servers regardless of the
file system. mquota gives us quota control regardless of what operating
system users currently use, will use in the future, or how they
access their data. It has allowed us centralized management of our
quota system -- we decide the standards for quotas and enforce
them across multiple servers by synchronizing configuration files
from a central server (with the rsync facility). mquota has allowed
us to educate our users on the significance of data management and
to enforce the behavior. So far, we have not had to purchase more
storage space, which allows us to project our costs for data storage
and back up and stay within our projections. Furthermore, configuring
mquota is very simple (two plain-text configuration files) and is
written entirely in Perl. Because most administrators know Perl
(or find that it's easy to learn), it is easy to debug or expand
mquota as our needs arise. It relies on proven UNIX commands like
du and find, so we understand exactly what we are
getting. Additionally, these commands are available on any flavor
of UNIX (with varying output from each command).
mquota filled a specific need for our organization and grew into
a significant tool for managing our systems. If there were one piece
of advice I could impart about the experience, I would recommend
implementing a quota system before users get into the habit of having
"unlimited" disk capacity. In the end, the people maintaining
the servers and the users who learn to manage their data will thank
you.
Michael Montero is the Chief Technology Officer and a co-founder
of Community Connect Inc., the leading publisher of online ethnic
communities. He has been administering UNIX machines for more than
eight years. Michael manages a brilliant staff of 20 helping to
build and manage AsianAvenue.com, BlackPlanet.com, and MiGente.com.
He can be contacted at: mmontero@mail.communityconnect.com.
| 
