Your Backup Script Can Fail Silently — Here's How to Actually Find Out
A cron job that fails loudly is annoying. A cron job that fails silently is the one that actually hurts you. If you followed our earlier guide on backing up a VPS to Cloudflare R2 with restic, you have a nightly backup running unattended — which also means nothing is watching to confirm it actually ran. Restic can fail for reasons that have nothing to do with your code: a flaky network mid-upload, an expired R2 API token, a full disk. Cron doesn't email you when a scheduled job silently stops succeeding; it only tells you what the job explicitly reports, and a job that never runs reports nothing at all.
Self-hosted Healthchecks closes that gap for the cost of one more small container. Here's the setup, how to wire it into the restic script from the earlier guide, and the one design mistake that quietly defeats the whole point.
The problem restic alone doesn't solve
Restic exits with a non-zero status when a backup fails, but an unmonitored cron job doesn't do anything useful with that exit code by default — it just vanishes into cron's mail spool, which on most VPS setups isn't configured to go anywhere. A dead man's switch flips this: instead of watching for something bad to happen, it watches for something good to stop happening. Your backup script pings a URL every time it finishes successfully; if that ping doesn't arrive within an expected window, the monitor assumes something's wrong and alerts you — even if the failure mode is "the whole VPS silently stopped responding" rather than a clean error.
Why self-host it instead of using the free hosted tier
Healthchecks is open source under the BSD 3-clause license, and per its own FAQ, self-hosting is a reasonable option if you want to run everything in-house or you're monitoring enough separate checks (backups across multiple VPS instances, multiple cron jobs per server) that a hosted free tier's check limit becomes a real constraint. If you only have one or two things to monitor, the hosted free tier is genuinely fine and simpler — self-hosting only pays off once you're running several servers or several scheduled jobs worth tracking centrally.
Setting it up
Healthchecks ships an official Docker Compose configuration. On a small VPS (a separate one from what you're backing up — more on why below):
git clone https://github.com/healthchecks/healthchecks.git
cd healthchecks/docker
cp .env.example .env
Edit .env to set at minimum ALLOWED_HOSTS, SITE_ROOT, SECRET_KEY, and your SMTP details for outbound alert email, then run docker compose up -d. The stack doesn't handle TLS itself, so put it behind Caddy or Nginx with a real certificate, the same way you would any other self-hosted app — a reverse proxy in front of the container on port 8000 is the standard setup.
Create a check in the web UI (New Check → name it "restic backup — appserver" → set Period to 24 hours and Grace Time to something that comfortably exceeds how long your backup actually takes). Healthchecks hands you a unique ping URL like https://hc.yourdomain.com/ping/<uuid>.
Wiring it into the restic backup script
Extend the cron job from the earlier restic guide to report start, success, and failure distinctly, instead of just success. Add a ping at the start, a success ping on the normal path, and a fail ping in the error path of the same script.
HC_URL="https://hc.yourdomain.com/ping/YOUR-UUID-HERE"
curl -fsS -m 10 --retry 3 "$HC_URL/start"
Then run the backup and prune as usual, and branch on the result:
source /root/.config/restic/r2.env
restic backup /var/www /root/db-backups
BACKUP_EXIT=$?
restic forget --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune
Finally, ping success or failure based on the backup's exit code:
if [ $BACKUP_EXIT -eq 0 ]; then
curl -fsS -m 10 --retry 3 "$HC_URL"
else
curl -fsS -m 10 --retry 3 "$HC_URL/fail"
fi
This gives you three distinct outcomes instead of one: the backup ran and succeeded, the backup ran and explicitly failed (an immediate alert with the failure state, not just silence), or the backup never ran at all (Healthchecks notices the missing ping once Period + Grace Time elapses and alerts you the same way). Connect an integration — email, Slack, ntfy, whatever you actually check — under the check's settings so the alert reaches you somewhere you'll notice it.
The catch: don't monitor your only server from itself
This is the mistake that quietly defeats the entire setup: self-hosting Healthchecks on the exact same VPS whose backup it's supposed to monitor. If that server has a total failure — disk dies, provider outage, kernel panic — your backup obviously stops running, but so does the thing that was supposed to tell you it stopped. You get silence on both ends, which is indistinguishable from everything being fine until you actually need the backup.
The fix is straightforward: run the Healthchecks instance somewhere independent of the servers it watches. A second cheap VPS, a free-tier instance from a different provider, or even the hosted healthchecks.io free tier used purely as the external monitor for your self-hosted infrastructure all work — the point is that the thing watching for "did my server go silent" can't be hosted on the server that might go silent.
Who this is for
Worth setting up if you're backing up one or more VPS instances with cron and restic (or any scheduled job you can't afford to have fail quietly) and want the alerting under your own control rather than depending on a third-party SaaS staying up and free forever. Skip the self-hosting step and just use the hosted free tier if you only have a check or two to monitor — the operational overhead of running and maintaining your own monitoring service isn't worth it until you're managing enough checks to hit real limits on the free plan.
FAQ
What happens if the Healthchecks server itself goes down?
You stop getting any alerts, including for real backup failures, until it's back up — which is exactly why it shouldn't share a host with what it's monitoring. Treat the monitor's own uptime as a dependency worth checking on periodically, the same way you'd check anything else critical.
Can I monitor more than just backups with this?
Yes — anything that runs on a schedule and should report success is a candidate: certificate renewal cron jobs, database dumps, scheduled data syncs, health-check scripts. Each gets its own check and ping URL.
How do I pick the right Grace Time so I don't get false alarms?
Time a few real runs of the job first, then set Grace Time comfortably above the slowest observed run — for a backup that normally takes 20 minutes but occasionally runs 40 during a large weekly prune, a grace time of at least an hour avoids paging you over normal variance.
Does self-hosting Healthchecks require a lot of server resources?
No — it's a Django app with SQLite or PostgreSQL as the backing store, and comfortably runs on the smallest VPS tier from any major provider alongside a handful of other small containers.
Is there a way to see the actual restic output, not just success/fail?
Yes — Healthchecks lets you attach a request body to the ping, so piping the last few lines of restic's output into the ping's POST body gives you the actual error text in the alert, not just a red status.
Bottom line
A restic backup you never check on is a backup you're assuming works. Self-hosted Healthchecks turns that assumption into an actual signal, for the price of one more small container — as long as it's running somewhere other than the server whose silence it's supposed to catch.
Sources: Healthchecks GitHub repository; Healthchecks self-hosted Docker documentation; Healthchecks FAQ. Verified against Healthchecks' official documentation on August 26, 2026.
Comments 0
Be the first to comment.
Leave a comment