An External Watchdog for the Iris Fleet
A small shell-based external watchdog that supervises the iris fleet from outside the processes it is meant to protect.
This one is small and I want to keep the writeup honest to that. iris-watchdog is an external watchdog for the iris fleet, written in shell, tracked against issues #1017 and #1036. There is not much more the source will honestly support, so I will talk about the idea rather than pad it out.
Why external matters
The word doing the work here is external. A process cannot reliably supervise itself. If iris hangs, deadlocks, or wedges in a state where it is technically alive but no longer doing its job, any health logic living inside that same process is exactly as stuck as the process is. The only supervisor you can trust is one running somewhere else, watching from the outside, with no shared fate.
That is the whole reason this exists as a separate shell tool rather than a feature baked into iris. It observes the fleet, and because it lives outside the thing it watches, it stays standing when the thing it watches falls over.
Keeping it in shell
Choosing shell for a watchdog is a deliberate bet on fewer moving parts. A watchdog earns its keep by being more reliable than what it supervises, which means it should have the smallest possible dependency surface and no runtime that can itself break. Shell plus the core system utilities is about as close to always-available as you get on a host, so a watchdog written that way is one you rarely have to babysit.
The takeaway is boring on purpose, and that is the point. The most valuable pieces of infrastructure are frequently the least glamorous: a few lines of shell that sit outside your fleet, notice when something has gone quiet, and act. You do not think about them until the day they save you.
Have something that needs building, or stabilizing?
These notes are the work log. The paid work runs through Moonshine Labs, my product and engineering studio. Tell us what you're building, or grab a call.
Work with us →