Install and set up

Install DRSM on a monitor host, give it keyed SSH access to the hosts it watches, add the hosts, and schedule the runs. Plan on about 30 minutes.

Requirements

# RHEL, Rocky, Alma and Fedora
# dnf install curl iputils procps-ng s-nail

# Debian and Ubuntu
# apt install curl iputils-ping procps bsd-mailx

Create the DRSM user

DRSM can run from your own account or from a system admin account. The default is sysadmin; the name is set in ~/.drsm.sh.

# groupadd -g 3500 sysadmin
# useradd -u 3500 -g 3500 sysadmin
# passwd sysadmin
# su - sysadmin
$ ssh-keygen -t ed25519

Create the same account on each host DRSM checks, then copy the public key to it:

$ for s in server1 server2 server3; do
>   ssh -tq $s "sudo groupadd -g 3500 sysadmin; sudo useradd -u 3500 -g 3500 sysadmin; sudo passwd sysadmin"
>   ssh-copy-id -i ~/.ssh/id_ed25519.pub sysadmin@$s
> done

If a host needs a different user, port or key, as most cloud images do, set it in ~/.ssh/config. See Cloud servers.

Install

1. Download the source

   $ mkdir -p ~/git && cd ~/git
   $ wget https://datareel.com/downloads/drsm/dr_system_monitor-<version>.tar.gz
   $ tar xzf dr_system_monitor-<version>.tar.gz
   $ mv dr_system_monitor-<version> dr_system_monitor
   $ cd dr_system_monitor

2. Create your configuration

   Copy the config and set the Web directory, user and group. etc/drsm_dev.sh is a version that runs from your own account and serves from ~/public_html/sysadmin.

   $ cp etc/drsm.sh ~/.drsm.sh
   $ vi ~/.drsm.sh
   
   export WWWdir=/var/www/html/sysadmin
   export SYSADMIN_USERNAME=sysadmin
   export SYSADMIN_GROUPNAME=sysadmin
   export TEMPdir="/tmp/${SYSADMIN_USERNAME}/drsm"

3. Create the Web directory

   # mkdir -p /var/www/html/sysadmin
   # chmod 775 /var/www/html/sysadmin
   # chown sysadmin:sysadmin /var/www/html/sysadmin

4. Run the installer

   DRSM installs to ~/drsm. Set DRSM_ACCEPT_EULA=yes to skip the license prompt in a provisioning script.

   $ cd ~/git/dr_system_monitor/utils
   $ ./install.sh
   Accept EULA (Yes/no)> Yes
   ...
   Install complete

   Reinstalling updates the code and keeps your etc/hosts files, plugins/local, $WWWdir/index.php and $WWWdir/site.

Add the hosts to monitor

Each host gets one file in ~/drsm/etc/hosts, named <host>.conf. Settings shared by every host go in _defaults.conf, so a host file only lists what is different.

$ vi ~/drsm/etc/hosts/web1.conf

DESCRIPTION="Public Web server"
IMPACT="Company website is down"
GROUP=production
PROBES="network/ping network/https"
network_https_URL=https://www.example.com/
storage_disk_WARN=85

The files are plain KEY=value lines that DRSM reads but never runs. All the keys are in Host files.

Coming from systems.dat

Releases before the plugin system kept hosts in $CONFIGdir/systems.dat and dev_systems.dat, with alert levels in generated <host>_profile.sh scripts. Convert them all in one step:

$ ~/drsm/utils/migrate_systems_dat.sh
$ ~/drsm/bin/drsm_run.sh -l

Until any host file exists, drsm_run.sh reads systems.dat directly, and system_check.sh and system_report.sh still work. The systems.dat format is:

# hostname,description,impact,is_web_server,is_linux,is_cluster_ip,can_ping,can_ssh
cms,CMS server,Web CMS is affected,yes:HTTP HTTPS,yes,yes,yes,yes
cms1,Primary CMS server,Web CMS is affected,no,yes,no,yes,yes

Test a run

$ source ~/.drsm.sh
$ ~/drsm/bin/drsm_run.sh -l           # list hosts and plugins
$ ~/drsm/bin/drsm_run.sh -n           # run everything, no alerts
$ ~/drsm/bin/drsm_run.sh -n -H web1   # one host

Then open http://<monitor host>/sysadmin in a browser.

Web server

# RHEL, Rocky, Alma and Fedora
# dnf install httpd php
# systemctl enable --now httpd

# Debian and Ubuntu
# apt install apache2 libapache2-mod-php
# systemctl enable --now apache2

The archived report links use Apache directory listings, so allow Options Indexes on $WWWdir/reports.

# /etc/httpd/conf.d/sysadmin.conf, or /etc/apache2/conf-available/sysadmin.conf
<Directory /var/www/html/sysadmin>
    AuthType Basic
    AuthName "DRSM"
    AuthUserFile /etc/httpd/drsm.htpasswd
    Require valid-user
</Directory>

# htpasswd -c /etc/httpd/drsm.htpasswd sysadmin
# firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.122.0/24" service name="https" accept'
# firewall-cmd --reload
# Debian and Ubuntu
# ufw allow from 192.168.122.0/24 to any port 80,443 proto tcp

Schedule the runs

Add the runs to the DRSM user's crontab. Probes run every 30 minutes, the full health run at quarter past and quarter to the hour, and a status report goes out each weekday at 13:15 even when everything passes.

$ crontab -e

*/30 * * * * /home/sysadmin/drsm/bin/drsm_run.sh -m probes &> /dev/null
15,45 * * * * /home/sysadmin/drsm/bin/drsm_run.sh &> /dev/null
15 13 * * mon-fri /home/sysadmin/drsm/bin/drsm_run.sh -e &> /dev/null
55 09-16 * * mon-fri /home/sysadmin/drsm/bin/drsm_run.sh -g dev &> /dev/null

# Archive reports nightly, purge old ones hourly
59 23 * * * /home/sysadmin/drsm/bin/archive.sh &> /dev/null
00 * * * * /home/sysadmin/drsm/bin/purge.sh &> /dev/null

Sites still on the older scripts use system_check.sh and system_report.sh in the same slots; both versions are in crontabs/crontab_example.txt. For a cluster, see Production cluster.

Next steps

• Set up email and text alerts: Email and text alerts
• Tune alert levels per host: Thresholds and alerts
• Brand the dashboard: Customizing the dashboard
• Write a check for your own application: Writing a plugin