Renewing a Cert From Behind a Residential ISP

I found out my certificate had expired the way everybody finds out: a browser full of red on a service I use every day. That was annoying. What actually bothered me was the next thing I learned, which was that certbot.timer on that host had been failing on every run for months, and not one thing had told me about it.

The host is a Ubuntu 24.04 box on my home connection running nginx as a reverse proxy in front of a stack of Docker services. Residential WAN, which means inbound port 80 is blocked or in use by another tech minded person. That single fact is the root of the whole story, because port 80 is exactly what Let's Encrypt's HTTP-01 challenge needs. The ACME server has to reach http://yourdomain/.well-known/acme-challenge/<token> on port 80. It will follow a redirect once it gets there, but the first request is always to port 80 and there is no way to move it. If your ISP swallows that port, HTTP-01 is not something you tune. It is something you cannot use.

Nothing was going to shout at me about it either. Let's Encrypt ended its expiration notification emails on June 4, 2025, so the safety net that used to catch exactly this failure is gone. A silent renewal failure now stays silent right up until the certificate dies and your browser tells you.

So I needed two things: a challenge type that never touches port 80, and an alerting path loud enough that the next failure reaches me instead of the journal. This post is the full walkthrough of both, in the order you would actually do them on your own host. Everything is in the companion repo at certbot-discord.

The half-solution that was already there

Before the rewrite, the host used certbot --manual --preferred-challenges dns, which meant pasting TXT records into Cloudflare by hand every ninety days. That works exactly once, in a terminal, with a human present. Under the systemd timer it produced this on every run:

An authentication script must be provided with --manual-auth-hook
when using the manual plugin non-interactively.

certbot was doing the right thing. The manual plugin genuinely cannot run unattended without a hook. But the failure went into the journal and stopped there, and a timer that fails identically twice a day looks a lot like a timer that is doing nothing at all.

Why DNS-01 fixes it

DNS-01 proves control of a domain by publishing a TXT record at _acme-challenge.yourdomain, which Let's Encrypt then looks up. Validation happens entirely over your DNS provider's API. Port 80 is never involved, your firewall is never involved, and your ISP's opinions about inbound traffic stop mattering.

Two consequences worth understanding before you start, because they change what this is useful for:

It issues wildcards. HTTP-01 cannot issue *.example.com at all. Let's Encrypt only offers wildcards over DNS-01. Once you have one, adding a new service subdomain never involves the certificate again, which on a homelab is the difference between "spin up a container" and "spin up a container and then remember the cert dance."

It works for hosts the internet cannot reach. Because nothing has to connect to your server, you can get a publicly trusted certificate for a service that only listens on your LAN, or sits behind a VPN, or is reachable only through a tunnel. The only requirement is that you control the DNS zone. How your users actually reach the service is a completely separate question from how you get the certificate.

What you need before you start

  • A domain whose nameservers point at Cloudflare. It does not matter whether individual records are proxied (orange cloud) or not; DNS-01 only needs the API.
  • A Linux host with nginx and certbot installed. This was written on Ubuntu 24.04, and sudo apt install certbot is enough. The setup script installs the Cloudflare plugin itself but assumes certbot is already there.
  • Root on that host.
  • Outbound HTTPS. The host talks to api.cloudflare.com and to Let's Encrypt. Nothing inbound is required for issuance.
  • A Discord server where you can create webhooks, meaning the Manage Webhooks permission on it.

Clone the repo somewhere on the host, then deal with the placeholders before running anything. Every script ships with example.com, admin@example.com and /home/youruser in it, and none of them are secrets, they are just not real. The README has a table mapping each file and line, but the table is a map rather than a substitute for checking:

grep -rn 'example\.com\|admin@example\|/home/youruser' scripts/ hooks/

Fix every hit before you go further. A missed one usually surfaces later as a monitor watching a certificate path that does not exist.

Step 1: notifications first

It is tempting to do the certificate first, because that is the actual problem. Do the notifications first anyway. Every step after this one can fail, and if the alerting is already in place you find out about those failures immediately instead of discovering them the next time something breaks.

Why a webhook rather than a notification service

There are perfectly good tools for this. Notifiarr, Pushbullet, ntfy, Gotify, healthchecks.io: all of them do notifications more thoroughly than a shell script does, and if you are already running one, use it. But each is another service to stand up, another account to hold, another key to rotate, and in several cases another container sitting right next to the containers it is meant to be watching. I already had a Discord server for the homelab with a channel nobody posts in, and a Discord webhook avoids all of that. It is a URL that accepts an HTTP POST. No SDK, no daemon, no auth flow, no OAuth refresh to break at three in the morning. curl is the entire client.

That is not only convenience. The thing doing the alerting has to be simpler than the thing it is watching, or you have just moved the silent failure somewhere else. A webhook has almost no moving parts, and the ones it does have live in a script short enough to read in one sitting.

Creating the webhook

In Discord: Server Settings, then Integrations, then Webhooks, then New Webhook. Pick the channel you want alerts in, give it a name, and hit Copy Webhook URL. You get something shaped like https://discord.com/api/webhooks/<numeric id>/<token>.

Treat that URL as a credential. It grants nothing beyond posting to that one channel, but anyone holding it can post there, so it goes to /etc/discord-webhook with mode 0600 and is in the repo's .gitignore next to the Cloudflare token. If it ever leaks, regenerate the webhook in Discord and the old URL dies immediately.

Now install the notifier:

sudo ./scripts/install-discord-notify.sh

It puts discord-notify in /usr/local/bin, then runs discord-notify --setup, which prompts for the URL, stores it and posts a test message. If a webhook is already configured it keeps the existing one, so re-running the installer is safe.

The store itself is unremarkable:

umask 077
printf '%s\n' "$URL" > "$CONF"
chmod 600 "$CONF"
echo "Saved to $CONF (0600). Sending a test..."

Between the paste and that write there is a step I did not expect to need. Most terminals wrap pasted text in bracketed-paste escape sequences, and some editors and password managers add stray control characters or wrapping quotes. None of it is visible, all of it lands in the file, and the resulting POST fails in a way that looks exactly like Discord rejecting a valid URL:

URL=$(printf '%s' "$RAW" | python3 -c '
import re, sys
s = sys.stdin.read()
s = re.sub(r"\x1b\[[0-9;]*[~a-zA-Z]", "", s)   # ANSI / bracketed paste
s = re.sub(r"[\x00-\x1f\x7f]", "", s)          # any other control chars
print(s.strip().strip("\"").strip("\x27").strip())')

Then it validates the cleaned result before saving anything, because the most common mistake by a wide margin is copying the channel link or a server invite instead of the webhook URL:

if ! printf '%s' "$URL" | grep -qE \
    '^https://(canary\.|ptb\.)?discord(app)?\.com/api/(v[0-9]+/)?webhooks/[0-9]+/[A-Za-z0-9_.-]+$'

The canary. and ptb. alternatives cover Discord's beta clients, which hand out webhook URLs on those hostnames, and discordapp.com is the legacy domain that still shows up in older documentation. If validation fails, the script tells you how many characters it ended up with and shows the first thirty-four, which is usually enough to see what went wrong without printing the token.

Posting to it

Sending a message is a JSON POST. The body gets wrapped in an embed so alerts carry a coloured stripe, which is what makes a red renewal failure distinguishable from a grey routine notice while you are scrolling past on a phone:

PAYLOAD=$(printf '{"embeds":[{"title":%s,"description":%s,"color":%s}]}' \
    "$(printf '%s' "$TITLE" | json_escape)" \
    "$(printf '```\n%s\n```' "$BODY" | json_escape)" \
    "$COLOR")

color is a decimal integer rather than a hex string, which is easy to get wrong the first time. The script accepts names and maps them, so callers never think about it:

red)    COLOR=15158332 ;;
green)  COLOR=3066993  ;;
yellow) COLOR=16776960 ;;
grey|gray) COLOR=9807270 ;;

Escaping the body through python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()))' instead of hand-rolling quoting is the other thing worth copying wholesale. Journal output contains quotes, backslashes and newlines as a matter of routine, and every one of them will eventually break a JSON string you assembled with printf.

The result is a command any script on the box can use with one pipe:

echo "body text" | discord-notify "Title" --color red
discord-notify "Title" --file /var/log/something.log

Wiring it to systemd

With a notifier that is just a command, the installer can hook it to anything systemd runs, using OnFailure and a template unit:

mkdir -p "/etc/systemd/system/${base}.service.d"
cat > "/etc/systemd/system/${base}.service.d/discord-on-failure.conf" <<'EOF'
[Unit]
OnFailure=discord-failure@%N.service
EOF

%N expands to the unit name without its suffix, so the drop-in on certbot.service instantiates discord-failure@certbot.service. That template runs a small helper which collects the unit's state, result and exit code, then the last sixty journal lines, and pipes the lot to discord-notify. The same drop-in works for any unit, which is why the installer takes extra unit names as arguments:

sudo ./scripts/install-discord-notify.sh nginx docker

Sixty journal lines is usually more than Discord will take inline. Message content caps at 2000 characters and an embed description at 4096, and the script uses a conservative 1900 to leave room for the code fence it wraps around the body. Anything longer is uploaded as a .txt attachment with a twelve-line preview in the embed, rather than being truncated at whatever turns out to be the interesting part.

That attachment path holds the one gotcha in this whole setup that cost me a real outage's worth of confidence. The payload has to go through --form-string, not -F:

RESP=$(curl -sS -m 60 -X POST \
    --form-string "payload_json=$PAYLOAD" \
    -F "files[0]=@${ATTACH};type=text/plain" \
    "$URL" -w '\n%{http_code}' 2>&1)

With -F, curl reads a ; inside the value as the start of a modifier such as ;type=, truncates the JSON there, and Discord rejects the whole post with a complaint that payload_json is not valid JSON. Journal output is full of semicolons, so this fails on every real alert while passing every short test message. The file part still uses -F, because it genuinely needs the ;type= modifier.

Which is the lesson, and it is the reason this step comes first: a notification system is proven when you have watched a real alert land, not when the setup test succeeded. The two took different code paths here and only one of them worked. Fire a real one on purpose before you trust it:

sudo systemctl start discord-failure@certbot.service

That posts an actual failure-shaped alert for certbot.service, journal tail and all. If it does not appear in your channel, fix that now, while you still have the context.

Step 2: the Cloudflare API token

In the Cloudflare dashboard, go to My Profile, then API Tokens, then Create Token, and choose the "Edit zone DNS" template. Three things matter on that page.

Permissions. The template gives you Zone, DNS, Edit, and the plugin documentation confirms that is all certbot needs.

Zone Resources. Change this from all zones to Include, Specific zone, and pick the one zone. This is the difference between a credential that can edit one domain's DNS and one that can edit everything in your account.

TTL. Set an expiry date. This feels like it is making life harder, and it is the reason step 4 exists, but an API token with no expiry is a credential that lives on a home server forever. Give it a year, or at least a cadence different than the certs you are renewing, and let the monitor remind you.

Use a token. Not the Global API Key, which authenticates as your entire account and cannot be scoped to anything.

Cloudflare shows the token exactly once. The setup script will prompt for it, or you can pass it in the environment for a non-interactive run:

sudo ./scripts/setup-cloudflare-certbot.sh
# or
sudo CF_TOKEN=xxxx ./scripts/setup-cloudflare-certbot.sh

It lands where the plugin expects it, at mode 0600:

mkdir -p "$SECRET_DIR"
chmod 700 "$SECRET_DIR"
umask 077
cat > "$SECRET_FILE" <<EOF
# Cloudflare API token for certbot DNS-01. Keep mode 0600.
dns_cloudflare_api_token = ${CF_TOKEN}
EOF
chmod 600 "$SECRET_FILE"

The permissions are not cosmetic. certbot emits an "Unsafe permissions on credentials configuration file" warning on every single run if that file is readable by anyone else, including every renewal, and the only way to silence it is to fix the mode.

Step 3: issue the certificate

The same script continues into issuance. First it installs the plugin, but only if it is missing:

if ! certbot plugins 2>/dev/null | grep -q dns-cloudflare; then
    echo "==> Installing python3-certbot-dns-cloudflare"
    apt-get update -qq
    apt-get install -y python3-certbot-dns-cloudflare
fi

Then it builds the certbot invocation once and uses it twice:

CERTBOT_ARGS=(
    certonly
    --dns-cloudflare
    --dns-cloudflare-credentials "$SECRET_FILE"
    --dns-cloudflare-propagation-seconds 30
    --cert-name "$CERT_NAME"
    -d "$DOMAIN"
    -d "*.${DOMAIN}"
    --email "$EMAIL"
    --agree-tos
    --non-interactive
    --key-type ecdsa
)

echo "==> Dry run (no rate limit consumed)"
certbot "${CERTBOT_ARGS[@]}" --dry-run

echo "==> Dry run passed. Issuing the real certificate."
certbot "${CERTBOT_ARGS[@]}"

Worth understanding rather than copying blindly:

--dry-run runs the whole flow against Let's Encrypt's staging environment, so a typo in your domain or a wrongly scoped token costs you nothing. Production rate limits are not generous enough to debug against.

--dns-cloudflare-propagation-seconds 30 is the wait between writing the TXT record and telling Let's Encrypt to look for it. The default is shorter and you can lose that race, which shows up as a validation failure that succeeds on retry, which is the most annoying class of bug there is. Thirty seconds has been reliable for me. If you see intermittent failures, raise it rather than retrying.

-d "$DOMAIN" -d "*.${DOMAIN}" is the payoff. One lineage, apex plus wildcard.

--key-type ecdsa is explicit but not actually a change: certbot has defaulted to ECDSA P-256 for new certificates since version 2.0. I keep it in the script because a lineage's key type is recorded at issuance and inherited on renewal, so being explicit about it is worth four words.

--cert-name pins the lineage directory name, which matters because everything downstream, nginx and the monitor both, hardcodes /etc/letsencrypt/live/<cert-name>/. Without it you can end up with a -0001 suffix and a lot of confusion.

When it finishes it prints the expiry and the SANs so you can see what you actually got:

openssl x509 -in "/etc/letsencrypt/live/${CERT_NAME}/fullchain.pem" \
    -noout -enddate -ext subjectAltName

The deploy hook, and a hook path that eats an afternoon

Reloading nginx after renewal belongs in a deploy hook, which certbot runs only when a certificate was actually replaced:

DEPLOY_HOOK="/etc/letsencrypt/renewal-hooks/deploy/reload-nginx.sh"
mkdir -p "$(dirname "$DEPLOY_HOOK")"

Note the path carefully. certbot only executes hooks that live in the pre/, post/ or deploy/ subdirectories. A script sitting loose in renewal-hooks/ is silently ignored, with no warning, and you will watch it not run several times before you notice.

The distinction between the three matters too. Post-hooks run on every attempt, including failures. Deploy hooks run only when a certificate was genuinely replaced. Reloading nginx from a post-hook means reloading it twice a day forever for no reason; reloading from a deploy hook means reloading roughly six times a year, exactly when the files on disk changed.

One ordering gotcha in this repo. Both setup-cloudflare-certbot.sh and install-discord-notify.sh write that same deploy hook path, and they write different versions of it. The setup script installs a plain reload; the Discord installer installs a version that reloads and then posts a green "certificate renewed" notice, or a red one if the reload failed. Since I have you installing notifications first, the setup script overwrites the notifying version with the plain one. Re-run the notification installer after issuance to put it back:

sudo ./scripts/install-discord-notify.sh

It is idempotent and it keeps your existing webhook, so this costs nothing. Those renewal success posts are worth having. Six messages a year is a heartbeat telling you the whole chain still works, which is precisely the signal whose absence started this entire project.

Step 4: point nginx at the certificate

The certificate exists but nothing is serving it yet. In each TLS vhost:

ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

Because the lineage is a wildcard, every vhost points at the same two files regardless of subdomain. Then the usual:

sudo nginx -t && sudo systemctl reload nginx

Verify from the host rather than from your laptop, so DNS and proxying do not confuse the result:

echo | openssl s_client -connect 127.0.0.1:443 -servername sonarr.example.com 2>/dev/null \
  | openssl x509 -noout -subject -ext subjectAltName -enddate

You want to see the wildcard in the SANs and an expiry roughly ninety days out.

If you had older per-subdomain certificates before this, delete them now. They will fail on every renewal run otherwise, and a timer that always fails is a timer whose failures you learn to ignore:

sudo certbot certificates                              # see what exists
sudo certbot delete --cert-name overseerr.example.com  # then remove the dead ones

Do that only after nginx is pointed at the new lineage, since deleting a certificate nginx still references will break the config test.

Step 5: confirm unattended renewal actually works

This is the step that was broken for months, so verify it properly.

sudo certbot renew --dry-run

That exercises the real renewal path, plugin, credentials and hooks included, against staging. If it passes, the thing that was failing is fixed.

Then look at the timer, because on Ubuntu it is not what people assume:

systemctl list-timers certbot.timer
systemctl cat certbot.timer

The packaged unit is OnCalendar=*-*-* 00,12:00:00 with RandomizedDelaySec=43200 and Persistent=true. So it runs twice a day at a randomised offset, not daily, and certbot only actually renews a certificate inside the last thirty days of its life. Most of those runs do nothing, which is the design: the frequency is there so a transient failure has many chances to correct itself before anything expires.

Which is also why the failure mode is so quiet. Twice a day, every day, that unit was failing on my host, and the only trace was a journal entry nobody reads. Now it posts to Discord.

Step 6: watch the token that everything now depends on

Switching to DNS-01 traded a blocked port for an API token, and I gave that token an expiry date on purpose in step 2. If it lapses or gets revoked, renewal breaks silently and I am back at the start of this post.

sudo ./scripts/install-cf-token-monitor.sh

That installs the check as a systemd timer (Monday 08:00, Persistent=true, fifteen minutes of jitter) rather than a cron job, specifically so the check itself gets an OnFailure handler. If the monitor breaks, you hear about that too, instead of it quietly stopping and the expiry warning simply never arriving. A monitor that can fail silently is not a monitor.

The check asks Cloudflare directly rather than trusting a date in a file:

RESP=$(curl -sS -m 30 -H "Authorization: Bearer ${TOKEN}" \
    "https://api.cloudflare.com/client/v4/user/tokens/verify" 2>&1)

The response carries status and expires_on, so rotating the token updates the monitor for free with nothing to edit. A FALLBACK_EXPIRY constant is used only when the API cannot be reached at all. If Cloudflare reports the token as invalid or expired, that is a red alert immediately, because renewal is broken right now rather than at some future date.

One threshold in there is deliberately odd:

WARN_DAYS=35                     # start warning this many days out
CRITICAL_DAYS=7                  # escalate inside this window

Thirty-five, not thirty, because the check runs weekly. A 30-day threshold can first fire on the run that lands at 24 days remaining, since the previous week's run was at 31 and stayed quiet. Thirty-five guarantees the first warning arrives between 29 and 35 days out. The general rule is that whenever a check interval and a warning threshold are close together, the threshold has to absorb a full interval or you quietly lose a cycle of notice.

Prove the alert path, same as before:

sudo /usr/local/bin/cloudflare-token-monitor --test

Caveat if you adapt this. /user/tokens/verify only accepts user API tokens. Cloudflare's newer account-owned tokens verify at /accounts/{account_id}/tokens/verify and return code 1000, "Invalid API Token", at the user endpoint even when they are perfectly valid and correctly scoped. If your token came from Manage Account rather than My Profile, you need the other endpoint, and the monitor as written would tell you your working token is dead.

There is also scripts/cert-monitor.sh, which checks the certificate itself rather than the token and alerts if it is inside thirty days and renewal fails. Note that nothing in the repo installs it: nginx-config-manager.sh cert-status expects it at ~/scripts/cert-monitor.sh, so copy it there yourself, and add your own timer if you want it running on a schedule. Its 30-day threshold overlaps certbot's own renewal window on purpose, so it is a backstop rather than the primary path.

Step 7: the emergency config

Everything above is about not having an outage. This last piece is about the outage you have anyway.

When a certificate problem is actively blocking access, either an expired cert that browsers refuse or missing cert files that make nginx -t fail so nginx will not reload at all, there is a TLS-free copy of the vhosts you can swap in. Services come back over plain HTTP while you fix the real problem.

./nginx-config-manager.sh emergency     # back up live config, swap in HTTP-only, reload
./nginx-config-manager.sh normal        # restore the backup, or fall back to the TLS config
./nginx-config-manager.sh test          # curl every vhost in the live config, resolved locally
./nginx-config-manager.sh cert-status   # run cert-monitor.sh
./nginx-config-manager.sh cert-renew    # certbot renew, then switch back to HTTPS on success

This one has no installer on purpose. Copy it to ~/scripts/ and run it from there, because the one time you need it may well be the time systemd is unhappy.

It had a quiet bug worth describing, because the shape of it is common. If the emergency config file was missing, the copy failed, the live config stayed in place, nginx -t then validated that untouched live config, and the script printed success. During an outage, a false success is the worst answer available. Now it refuses:

if [ ! -f "$EMERGENCY_CONFIG" ]; then
    echo "ERROR: emergency config not found at $EMERGENCY_CONFIG" >&2
    echo "Nothing was changed. Live config is untouched." >&2
    exit 1
fi

And if the emergency config is present but fails validation, the backup is restored and nginx reloaded before it exits non-zero. Failing without leaving the box worse than you found it is most of the job.

Why the fallback generates itself

The bigger problem was that the emergency config used to be a hand-maintained snapshot. That is the wrong shape for a file only ever read during an outage: add, rename or remove a service and it goes stale silently, and you discover it at the exact moment you need it to be correct.

So generate-emergency-config.sh builds it from the live config instead. It pulls server_name and proxy_pass out of each server block and emits an HTTP-only equivalent:

for blk in re.findall(r'server\s*\{.*?\n\}', src, re.S):
    m = re.search(r'server_name\s+([^;]+);', blk)
    p = re.search(r'proxy_pass\s+(http://[\d.]+:\d+);', blk)
    if not m or not p:
        continue          # redirect-only vhosts have nothing to proxy to
    seen[m.group(1).strip()] = p.group(1)

Two constraints hide in those two regexes, and you need to know them before pointing this at your own config. The block pattern expects top-level server { blocks that close with a } at the start of a line, which is conventional nginx formatting but not guaranteed. And the proxy_pass pattern only matches numeric http://IP:port upstreams, so if you proxy to Docker service names or named upstream blocks, those vhosts get skipped. Tweak the pattern for your setup, then check the vhost count in the log against what you expect.

Each generated vhost keeps the ACME challenge location, so renewal still works while you are in emergency mode:

server {
    listen 80;
    server_name sonarr.example.com;

    location /.well-known/acme-challenge/ {
        root /var/www/html;
    }

    location / {
        proxy_pass http://10.0.0.5:8989;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto http;
    }
}

The generator is mostly refusals, which is the interesting part of it. Each exists so it cannot make an outage worse:

Situation What it does
Live config has no TLS Emergency mode is already applied, so the live config is the emergency config. Skips, rather than regenerating from itself and possibly overwriting a good copy with a truncated one.
Output fails sanity checks Empty file, unbalanced braces, no server blocks, or any ssl_certificate or listen 443 that leaked through. Candidate discarded, existing file untouched, red alert sent.
Redirect-only vhosts Omitted deliberately. No proxy_pass means nothing to serve over HTTP.

Install it behind both triggers:

sudo ./scripts/install-emergency-sync.sh

That gives you a .path unit that regenerates the moment the live config changes, and a weekly .timer at Monday 07:30 as a backstop for anything inotify does not see, such as the path unit being masked or the file being replaced by a method that raises no change event. Both report their own failures through the same Discord path.

Useful afterwards:

sudo /usr/local/bin/generate-emergency-config --check   # report drift, change nothing, exit 3 if stale
sudo systemctl start emergency-config-sync.service      # force a resync
systemctl status emergency-config-sync.path             # is the watch live
systemctl list-timers emergency-config-sync.timer       # next backstop run

The emergency file is now a generated artifact. Edit the live config and it follows; edit the emergency copy by hand and your changes vanish on the next run.

Then actually rehearse it once, on a quiet evening rather than during an outage: run emergency, confirm the services answer over HTTP, run normal, confirm HTTPS is back. A fallback you have never exercised is a hypothesis.

What ends up running

Unit Schedule Purpose
certbot.timer Twice daily, randomised Renews inside the last 30 days. Alerts red on failure.
certbot deploy hook On actual renewal Reloads nginx, posts green. Roughly six a year.
cloudflare-token-monitor.timer Monday 08:00 Warns at 35 days, escalates at 7, red if revoked.
emergency-config-sync.path On config change Regenerates the HTTP-only fallback.
emergency-config-sync.timer Monday 07:30 Backstop for the above.
discord-failure@.service On any wired failure Posts the failed unit's journal tail.

When something goes wrong

Symptom Likely cause
Renewal fails with an authorization error from Cloudflare Token scope. It needs Zone, DNS, Edit, and the zone has to be in Zone Resources. A token from the wrong Cloudflare account looks identical and fails the same way.
Intermittent validation failures that pass on retry TXT propagation. Raise --dns-cloudflare-propagation-seconds.
"Unsafe permissions on credentials configuration file" on every run chmod 600 the cloudflare.ini. This cannot be silenced any other way.
Token monitor reports a valid token as invalid Account-owned token hitting the user verify endpoint. Use /accounts/{account_id}/tokens/verify.
Hook never runs It is in renewal-hooks/ rather than renewal-hooks/deploy/.
Discord returns "Invalid Form Body" Usually an embed field over its limit, or -F where --form-string belongs.
Setup test posts fine but real alerts never arrive The attachment path. Long bodies take different code than short ones. Fire discord-failure@certbot.service to test the real path.
No renewal success posts, failures work fine The deploy hook got overwritten by the setup script. Re-run the notification installer.

Adapting it

Almost none of this is Cloudflare-specific. certbot ships DNS plugins for Route 53, DigitalOcean, Google Cloud DNS, Linode, RFC 2136 and plenty more, and the only parts that change are the plugin name, the credentials file format, and the token monitor, which is genuinely Cloudflare-only. Everything else, the deploy hook, the OnFailure wiring, the generated fallback config, cares about none of it.

The notification side is even more portable. discord-notify is one script whose entire interface is a title, a colour and a body on stdin. Point it at a Slack or Teams incoming webhook by changing the payload shape, or swap it for an ntfy publish, and every caller keeps working because none of them know what is on the other end.

What this actually bought

The renewal problem itself was solved by DNS-01 in about an hour. Everything after that was the harder half: making sure that when this breaks again, and it will, something tells me before a browser does. Renewal alerts on failure and posts on success. The token warns a month before it lapses. The emergency fallback keeps itself current and refuses to lie to me about whether it worked.

If you are running anything real on a residential connection, DNS-01 is very likely your only sane path to a certificate, and Cloudflare's API makes issuance a fifteen-minute job. Just do not stop there. An automated system that fails silently is worse than a manual chore you remember to do, because at least you know the chore exists.

And if the alerting half is what has been putting you off, it is smaller than it looks. A webhook URL in a file with mode 0600, a curl wrapper, and OnFailure= on the units you care about is maybe an hour of work with nothing new running afterwards. Whether the messages land in Discord, Slack, Teams or ntfy barely matters. What matters is that a failed timer stops being a line in a journal nobody reads.