Skip to end of metadata
Go to start of metadata

Overview

Shutdown design

When your UPS batteries get low, the operating system needs to be brought down cleanly. Also, the UPS load should be turned off so that all devices that are attached to it are forcibly rebooted.

Here are the steps that occur when a critical power event happens:

  1. The UPS goes on battery
  2. The UPS reaches low battery (a "critical" UPS), that is to say upsc displays:

    ups.status: OB LB

    The exact behavior depends on the specific device, and is related to:

    • battery.charge and battery.charge.low
    • battery.runtime and battery.runtime.low


  3. The upsmon master notices and sets "FSD" - the "forced shutdown" flag to tell all slave systems that it will soon power down the load.

    (If you have no slaves, skip to step 6)

  4. upsmon slave systems see "FSD" and:

    • generate a NOTIFY_SHUTDOWN event
    • wait FINALDELAY seconds - typically 5
    • call their SHUTDOWNCMD
    • disconnect from upsd


  5. The upsmon master system waits up to HOSTSYNC seconds (typically 15) for the slaves to disconnect from upsd. If any are connected after this time, upsmon stops waiting and proceeds with the shutdown process.
  6. The upsmon master:

    • generates a NOTIFY_SHUTDOWN event
    • waits FINALDELAY seconds - typically 5
    • creates the POWERDOWNFLAG file - usually /etc/killpower
    • calls the SHUTDOWNCMD


  7. On most systems, init takes over, kills your processes, syncs and unmounts some filesystems, and remounts some read-only.
  8. init then runs your shutdown script. This checks for the POWERDOWNFLAG, finds it, and tells the UPS driver(s) to power off the load.
  9. The system loses power.
  10. Time passes. The power returns, and the UPS switches back on.
  11. All systems reboot and go back to work.


Proposed Architecture

For this setup, we will be setting up our UPS master on a raspberry pi with slaves setup on a QNAP NAS and a Ubuntu Server.



Master Setup on Raspberry Pi


Connect UPS using USP cable

Installation

$ sudo apt-get install nut

Configuring the UPS/UPSD


$ vi /etc/nut/ups.conf


Add the following config

[qnapups]
  driver = usbhid-ups
    port = auto
    desc = "Back-UPS ES 750"

Configure upsd.conf

$ sudo vi /etc/nut/upsd.conf

Add the following config:

LISTEN 0.0.0.0 3493

Update Credentials

$ sudo vi /etc/nut/upsd.users
[admin]
  password = 123456
  actions = SET
  instcmds = ALL

[upsmon]
  password = 123456
  upsmon master

[deepthought]
  password = 123456
  upsmon slave

Update Nut.Conf

$ sudo vi /etc/nut/nut.conf 

Set Mode=netserver

MODE=netserver


Configuring UPS Monitor

$ vi /etc/nut/upsmon.conf

Add the following line

MONITOR qnapups@localhost 1 upsmon 123456 master
FINALDELAY 5


FINALDELAY seconds

When running in master mode, upsmon waits this long after sending the NOTIFY_SHUTDOWN to warn the users. After the timer elapses, it then runs your SHUTDOWNCMD. By default this is set to 5 seconds.

If you need to let your users do something in between those events, increase this number. Remember, at this point your UPS battery is almost depleted, so don’t make this too big.

Alternatively, you can set this very low so you don’t wait around when it’s time to shut down. Some UPSes don’t give much warning for low battery and will require a value of 0 here for a safe shutdown.



Verify Hardware Config

$ sudo upsdrvctl start 


Restart the Service

$ sudo service nut-server restart


Get Status

$ sudo service nut-server status

● nut-server.service - Network UPS Tools - power devices information server
   Loaded: loaded (/lib/systemd/system/nut-server.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2021-08-24 20:28:53 EDT; 7s ago
  Process: 12995 ExecStart=/sbin/upsd (code=exited, status=0/SUCCESS)
 Main PID: 12996 (upsd)
    Tasks: 1 (limit: 877)
   CGroup: /system.slice/nut-server.service
           └─12996 /lib/nut/upsd

Aug 24 20:28:53 pihole systemd[1]: Starting Network UPS Tools - power devices information server...
Aug 24 20:28:53 pihole upsd[12995]: fopen /var/run/nut/upsd.pid: No such file or directory
Aug 24 20:28:53 pihole upsd[12995]: listening on 127.0.0.1 port 3493
Aug 24 20:28:53 pihole upsd[12995]: listening on ::1 port 3493
Aug 24 20:28:53 pihole upsd[12995]: listening on 127.0.0.1 port 3493
Aug 24 20:28:53 pihole upsd[12995]: Connected to UPS [homeups]: usbhid-ups-homeups
Aug 24 20:28:53 pihole upsd[12995]: listening on ::1 port 3493
Aug 24 20:28:53 pihole upsd[12995]: Connected to UPS [homeups]: usbhid-ups-homeups
Aug 24 20:28:53 pihole systemd[1]: Started Network UPS Tools - power devices information server.
Aug 24 20:28:53 pihole upsd[12996]: Startup successful


Query the UPS

$ upsc qnapups@localhost

$ upsc qnapups@192.168.1.52

battery.charge: 100
battery.charge.low: 10
...
ups.vendorid: 051d


Client Setup QNAP

From the QNAP UI, open the control panel and navigate to External Sevices → UPS

Select Remote UPS Slave and enter the IP address of the master.

Click apply. The UPS information should now be updated. See screenshots below.


From a shell connection on the QNAP server, you can verify connection via the following command: (not required)

$ upsc qnapups@192.168.1.52

battery.charge: 100
...
ups.vendorid: 051d


Client Setup (Ubuntu)

Installation

Install nut-client

$ sudo apt-get install nut-client


Configure

Update nut-config

$ vi /etc/nut/nut.conf


Set mode to netclient

MODE=netclient


Update upsmon.conf

$ vi /etc/nut/upsmon.conf


RUN_AS_USER root
MONITOR qnapups@192.168.1.52 1 deepthought 123456 slave
NOTIFYCMD /usr/sbin/upssched
NOTIFYFLAG ONLINE WALL+EXEC
NOTIFYFLAG ONBATT WALL+EXEC

On our master we created a slave user called deepthought.

  • upsname = qnapups
  • user = deepthought
  • password = 123456


Setup Timer to Shutdown after 1 Minute

Edit the upssched.conf file

$ sudo vi /etc/nut/upssched.conf


Set  the following

CMDSCRIPT /etc/nut/upssched-cmd
  
PIPEFN /var/run/nut/upssched.pipe
LOCKFN /var/run/nut/upssched.lock

# Timer to shutdown machine after 30 seconds
AT ONBATT * START-TIMER onbattwarn 30
AT ONLINE * CANCEL-TIMER onbattwarn


Create the Command Script

$ sudo vi /etc/nut/upssched-cmd


Add the following content

case $1 in
        onbattwarn)
                logger -t upssched-cmd "Timer On Battery Warning has been triggered - Shutting Down!"
                wall "UPS timer expired: shutting down..."
                shutdown -h now
                ;;
        *)
                logger -t upssched-cmd "Unrecognized command: $1"
                ;;
esac



Restart Nut Client

Restart nut client:

$ systemctl restart nut-client


Get Status of Nut Client

Get Status:

$ systemctl status nut-client


● nut-monitor.service - Network UPS Tools - power device monitor and shutdown controller
Loaded: loaded (/lib/systemd/system/nut-monitor.service; enabled; vendor preset: enabled)
Active: active (running) since Wed 2021-08-25 13:11:09 EDT; 2s ago
Process: 376582 ExecStart=/sbin/upsmon (code=exited, status=0/SUCCESS)
Main PID: 376585 (upsmon)
Tasks: 2 (limit: 19043)
Memory: 1.1M
CGroup: /system.slice/nut-monitor.service
├─376583 /lib/nut/upsmon
└─376585 /lib/nut/upsmon

Aug 25 13:11:09 deepthought systemd[1]: Starting Network UPS Tools - power device monitor and shutdown controller...
Aug 25 13:11:09 deepthought upsmon[376582]: fopen /run/nut/upsmon.pid: No such file or directory
Aug 25 13:11:09 deepthought upsmon[376582]: Using power down flag file /etc/killpower
Aug 25 13:11:09 deepthought upsmon[376582]: UPS: qnapups@192.168.1.52 (slave) (power value 1)
Aug 25 13:11:09 deepthought systemd[1]: nut-monitor.service: Can't open PID file /run/nut/upsmon.pid (yet?) after start: Operati>
Aug 25 13:11:09 deepthought upsmon[376583]: Startup successful
Aug 25 13:11:09 deepthought systemd[1]: nut-monitor.service: Supervising process 376585 which is not our child. We'll most likel>
Aug 25 13:11:09 deepthought systemd[1]: Started Network UPS Tools - power device monitor and shutdown controller.


Check connection 

Check Connection:

$ upsc qnapups@192.168.1.52

References

  • No labels