Overview
Shutdown design
When your UPS batteries get low, the operating system needs to be brought down cleanly. Also, the UPS load should be turned off so that all devices that are attached to it are forcibly rebooted.
Here are the steps that occur when a critical power event happens:
- The UPS goes on battery
The UPS reaches low battery (a "critical" UPS), that is to say upsc displays:
ups.status: OB LB
The exact behavior depends on the specific device, and is related to:
- battery.charge and battery.charge.low
- battery.runtime and battery.runtime.low
The upsmon master notices and sets "FSD" - the "forced shutdown" flag to tell all slave systems that it will soon power down the load.
(If you have no slaves, skip to step 6)
upsmon slave systems see "FSD" and:
- generate a NOTIFY_SHUTDOWN event
- wait FINALDELAY seconds - typically 5
- call their SHUTDOWNCMD
- disconnect from upsd
- The upsmon master system waits up to HOSTSYNC seconds (typically 15) for the slaves to disconnect from upsd. If any are connected after this time, upsmon stops waiting and proceeds with the shutdown process.
The upsmon master:
- generates a NOTIFY_SHUTDOWN event
- waits FINALDELAY seconds - typically 5
- creates the POWERDOWNFLAG file - usually /etc/killpower
- calls the SHUTDOWNCMD
- On most systems, init takes over, kills your processes, syncs and unmounts some filesystems, and remounts some read-only.
- init then runs your shutdown script. This checks for the POWERDOWNFLAG, finds it, and tells the UPS driver(s) to power off the load.
- The system loses power.
- Time passes. The power returns, and the UPS switches back on.
- All systems reboot and go back to work.
Proposed Architecture
For this setup, we will be setting up our UPS master on a raspberry pi with slaves setup on a QNAP NAS and a Ubuntu Server.
Master Setup on Raspberry Pi
Connect UPS using USP cable
Installation
$ sudo apt-get install nut
Configuring the UPS/UPSD
$ vi /etc/nut/ups.conf
Add the following config
[qnapups]
driver = usbhid-ups
port = auto
desc = "Back-UPS ES 750"
Configure upsd.conf
$ sudo vi /etc/nut/upsd.conf
Add the following config:
LISTEN 0.0.0.0 3493
Update Credentials
$ sudo vi /etc/nut/upsd.users
[admin] password = 123456 actions = SET instcmds = ALL [upsmon] password = 123456 upsmon master [deepthought] password = 123456 upsmon slave
Update Nut.Conf
$ sudo vi /etc/nut/nut.conf
Set Mode=netserver
MODE=netserver
Configuring UPS Monitor
$ vi /etc/nut/upsmon.conf
Add the following line
MONITOR qnapups@localhost 1 upsmon 123456 master FINALDELAY 5
FINALDELAY seconds
When running in master mode, upsmon waits this long after sending the NOTIFY_SHUTDOWN to warn the users. After the timer elapses, it then runs your SHUTDOWNCMD. By default this is set to 5 seconds.
If you need to let your users do something in between those events, increase this number. Remember, at this point your UPS battery is almost depleted, so don’t make this too big.
Alternatively, you can set this very low so you don’t wait around when it’s time to shut down. Some UPSes don’t give much warning for low battery and will require a value of 0 here for a safe shutdown.
Verify Hardware Config
$ sudo upsdrvctl start
Restart the Service
$ sudo service nut-server restart
Get Status
$ sudo service nut-server status
● nut-server.service - Network UPS Tools - power devices information server
Loaded: loaded (/lib/systemd/system/nut-server.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2021-08-24 20:28:53 EDT; 7s ago
Process: 12995 ExecStart=/sbin/upsd (code=exited, status=0/SUCCESS)
Main PID: 12996 (upsd)
Tasks: 1 (limit: 877)
CGroup: /system.slice/nut-server.service
└─12996 /lib/nut/upsd
Aug 24 20:28:53 pihole systemd[1]: Starting Network UPS Tools - power devices information server...
Aug 24 20:28:53 pihole upsd[12995]: fopen /var/run/nut/upsd.pid: No such file or directory
Aug 24 20:28:53 pihole upsd[12995]: listening on 127.0.0.1 port 3493
Aug 24 20:28:53 pihole upsd[12995]: listening on ::1 port 3493
Aug 24 20:28:53 pihole upsd[12995]: listening on 127.0.0.1 port 3493
Aug 24 20:28:53 pihole upsd[12995]: Connected to UPS [homeups]: usbhid-ups-homeups
Aug 24 20:28:53 pihole upsd[12995]: listening on ::1 port 3493
Aug 24 20:28:53 pihole upsd[12995]: Connected to UPS [homeups]: usbhid-ups-homeups
Aug 24 20:28:53 pihole systemd[1]: Started Network UPS Tools - power devices information server.
Aug 24 20:28:53 pihole upsd[12996]: Startup successful
Query the UPS
$ upsc qnapups@localhost $ upsc qnapups@192.168.1.52 battery.charge: 100 battery.charge.low: 10 ... ups.vendorid: 051d
Client Setup QNAP
From the QNAP UI, open the control panel and navigate to External Sevices → UPS
Select Remote UPS Slave and enter the IP address of the master.
Click apply. The UPS information should now be updated. See screenshots below.
From a shell connection on the QNAP server, you can verify connection via the following command: (not required)
$ upsc qnapups@192.168.1.52 battery.charge: 100 ... ups.vendorid: 051d
Client Setup (Ubuntu)
Installation
Install nut-client
$ sudo apt-get install nut-client
Configure
Update nut-config
$ vi /etc/nut/nut.conf
Set mode to netclient
MODE=netclient
Update upsmon.conf
$ vi /etc/nut/upsmon.conf
RUN_AS_USER root MONITOR qnapups@192.168.1.52 1 deepthought 123456 slave NOTIFYCMD /usr/sbin/upssched NOTIFYFLAG ONLINE WALL+EXEC NOTIFYFLAG ONBATT WALL+EXEC
On our master we created a slave user called deepthought.
- upsname = qnapups
- user = deepthought
- password = 123456
Setup Timer to Shutdown after 1 Minute
Edit the upssched.conf file
$ sudo vi /etc/nut/upssched.conf
Set the following
CMDSCRIPT /etc/nut/upssched-cmd PIPEFN /var/run/nut/upssched.pipe LOCKFN /var/run/nut/upssched.lock # Timer to shutdown machine after 30 seconds AT ONBATT * START-TIMER onbattwarn 30 AT ONLINE * CANCEL-TIMER onbattwarn
Create the Command Script
$ sudo vi /etc/nut/upssched-cmd
Add the following content
case $1 in
onbattwarn)
logger -t upssched-cmd "Timer On Battery Warning has been triggered - Shutting Down!"
wall "UPS timer expired: shutting down..."
shutdown -h now
;;
*)
logger -t upssched-cmd "Unrecognized command: $1"
;;
esac
Restart Nut Client
Restart nut client:
$ systemctl restart nut-client
Get Status of Nut Client
Get Status:
$ systemctl status nut-client
● nut-monitor.service - Network UPS Tools - power device monitor and shutdown controller Loaded: loaded (/lib/systemd/system/nut-monitor.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2021-08-25 13:11:09 EDT; 2s ago Process: 376582 ExecStart=/sbin/upsmon (code=exited, status=0/SUCCESS) Main PID: 376585 (upsmon) Tasks: 2 (limit: 19043) Memory: 1.1M CGroup: /system.slice/nut-monitor.service ├─376583 /lib/nut/upsmon └─376585 /lib/nut/upsmon Aug 25 13:11:09 deepthought systemd[1]: Starting Network UPS Tools - power device monitor and shutdown controller... Aug 25 13:11:09 deepthought upsmon[376582]: fopen /run/nut/upsmon.pid: No such file or directory Aug 25 13:11:09 deepthought upsmon[376582]: Using power down flag file /etc/killpower Aug 25 13:11:09 deepthought upsmon[376582]: UPS: qnapups@192.168.1.52 (slave) (power value 1) Aug 25 13:11:09 deepthought systemd[1]: nut-monitor.service: Can't open PID file /run/nut/upsmon.pid (yet?) after start: Operati> Aug 25 13:11:09 deepthought upsmon[376583]: Startup successful Aug 25 13:11:09 deepthought systemd[1]: nut-monitor.service: Supervising process 376585 which is not our child. We'll most likel> Aug 25 13:11:09 deepthought systemd[1]: Started Network UPS Tools - power device monitor and shutdown controller.
Check connection
Check Connection:
$ upsc qnapups@192.168.1.52
Installing NUT on Mac
> brew install nut ==> Downloading https://ghcr.io/v2/homebrew/core/nut/manifests/2.8.1 ######################################################################### 100.0% ==> Fetching nut ==> Downloading https://ghcr.io/v2/homebrew/core/nut/blobs/sha256:e89241f392bb9f ######################################################################### 100.0% ==> Pouring nut--2.8.1.arm64_sonoma.bottle.tar.gz Warning: The post-install step did not complete successfully You can try again using: brew postinstall nut ==> Caveats To start nut now and restart at login: brew services start nut Or, if you don't want/need a background service you can just run: /opt/homebrew/opt/nut/sbin/upsmon -D ==> Summary 🍺 /opt/homebrew/Cellar/nut/2.8.1: 94 files, 12MB ==> Running `brew cleanup nut`...
Didn't work.....
Troubleshooting
| Problem | Details |
|---|---|
| UPS Shutting Down | Sometimes the UPS shutdown for no reason. No power outage. cat /var/log/syslog |grep ups Mar 16 02:09:28 pihole upsmon[390]: UPS qnapups@localhost on battery Unplugging the power and querying the UPS showed a charge of 0! > upsc qnapups@localhost |grep battery
|
| Get NUT status | sudo service nut-server status Mar 16 03:10:11 pihole upsd[374]: fopen /var/run/nut/upsd.pid: No such file or directory
|
| Querying UPS | upsc qnapups@localhost $ upsc qnapups@192.168.1.52 battery.charge: 100 battery.charge.low: 10 ... ups.vendorid: 051d |
References
| Reference | URL |
|---|---|
| Network UPS Tools User Manual | https://networkupstools.org/docs/user-manual.chunked/index.html |
| Configuration | https://networkupstools.org/docs/user-manual.chunked/ar01s06.html |
| UPSMON Man Page | https://networkupstools.org/docs/man/upsmon.conf.html |
NUT Introduction to Network UPS Tools - Configuration Examples | http://rogerprice.org/NUT/ConfigExamples.A5.pdf |
| Raspberry Pi Home Server v2: Network UPS Tools | https://melgrubb.com/2016/12/11/rphs-v2-ups/ |