EMG Watchdog

EMG Watchdog is a perl script (emg_watchdog.pl) that is run as a background process on the same server as EMG and which can monitor and control the emgd process. It is released with EMG 7 but can be used to control EMG 6 as well.

It implements a simple API based on HTTP and JSON for starting, stopping EMG and reading EMG server log files.

EMG Watchdog can be used from EMG Portal to control the EMG process via the web-based interface.

Installation

The script provided is named “emg_watchdog.pl.sample”. It is intended to be copied to “emg_watchdog.pl” and can be run in the background with “perl emg_watchdog.pl &” as the same user that runs the “emgd” process, normally “emg”.

It depends on several CPAN modules that need to be installed before the script can be run.

cpan Email::Sender::Simple Email::Sender::Transport::SMTP Email::Simple Email::Simple::Creator Filesys::Df HTTP::Daemon HTTP::Status IPC::Shareable JSON Net::Subnet URI::QueryParam

Before running the watchdog the configuration variables described below must be modified to match the local environment.

The latest version of the EMG Watchdog can be found on the URL below:
https://docs.nordicmessaging.se/wp-content/uploads/documents/emg_watchdog.pl.sample.txt

Configuration

The most important variables below are “$emg_dir”, “@notify_recipients” and “$mail_from”.

Configuration variables in the script:

VariableDefaultDescription
$emg_dir/home/emg/etcDirectory where EMG configuration (server.cfg) is located
$logfile$emg_dir/log/emg_watchdog.logLog file for EMG watchdog
$emg_quickstop1Use “quick stop” for stopping emgd (signal emgd to stop, wait a few seconds and then kill it)
$check_interval60Specifies how often (in seconds) checks should be run
$notify_interval21600Specifies how often (in seconds) notifications should be sent for persistent error (21600 secs = 6 hours)
$fs_to_check/File system to check for used space
$fs_limit80File system usage limit (in %), notification will be sent when limit exceeded
$listen_port3000Port on which integrated web server listens for incoming requests
@notify_recipients(”)Array with recipients for e-mail notifications (empty by default)
$mail_fromchangeme_from@example.comE-mail notification “From” address
$mail_subject_prefix‘EMG watchdog’E-mail notification subject prefix (consider adding hostname for easier identification)
$smtp_server127.0.0.1IP address of SMTP server to use for sending e-mails
$smtp_port25Port of SMTP server to use for sending e-mails
$smtp_usernameundefUsername for SMTP server authentication (undef = no auth)
$smtp_passwordundefPassword for SMTP server authentication (undef = no auth)
$allowed_client_ips127.0.0.1/32 192.168.0.0/24Allowed client ip addresses / subnets for web server access

Notifications

When specific events occur the watchdog will send e-mail notifications to the addresses configured ($notify_recipients).

  • Monitor detected EMG was not running
  • EMG manual stop requested via API
  • EMG stopped
  • EMG manual startup requested via API
  • EMG started
  • EMG startup failed
  • Used disk space in monitored file system ($fs_to_check) exceeds configured percent (fs_limit)

Using the watchdog API

The watchdog features a built-in web server that by default listens on port 3000 which implements an api.

Web server port must be protected in order to make sure it cannot be accessed directly via the internet. Access must be protected either by a local firewall in the server (iptables) or by an external firewall in front of the EMG server.

URIDescription and parameters (M – Mandatory, O – Optional)
/api/pingCheck server status
/api/emg_startStart EMG server
/api/emg_stopStop EMG server
/api/emg_statusGet status of EMG server
/api/get_log_file_listGet list of log files
/api/get_log_fileGet contents of log file
Parameters: file – Log file name (M)
maxrows – Max number of rows to return (O, default: 100)
search_string – Only return rows that include search_string, multiple terms can be separated by space (O)

Sample http request using wget

wget -q -O - http://127.0.0.1:3000/api/ping

should return

{"status":"ok"}