Systemd-resolved.service Watchdog timeout

Virtualmin
1
SYSTEM INFORMATION
OS type and version Ubuntu Linux 22.04.5
Webmin version 2.660
Usermin version 2.560
Virtualmin version 8.2.0 GPL
Theme version 26.60

Since several days I have a new problem with systemd-resolved.service: Watchdog timeout, it shows these errors in the systemd log:

Sep  2 10:13:40 mysite systemd[1]: systemd-resolved.service: Watchdog timeout (limit 3min)!
Sep  2 10:13:40 mysite systemd[1]: systemd-resolved.service: Killing process 391473 (systemd-resolve) with signal SIGABRT.
Sep  2 10:13:40 mysite systemd[1]: systemd-resolved.service: Main process exited, code=dumped, status=6/ABRT
Sep  2 10:13:40 mysite systemd[1]: systemd-resolved.service: Failed with result 'watchdog'.
Sep  2 10:13:40 mysite systemd[1]: systemd-resolved.service: Scheduled restart job, restart counter is at 7.
Sep  2 10:13:40 mysite systemd-resolved[396752]: Positive Trust Anchors:
Sep  2 10:13:40 mysite systemd-resolved[396752]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Sep  2 10:13:40 mysite systemd-resolved[396752]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa corp home internal intranet lan local private test
Sep  2 10:13:40 mysite systemd-resolved[396752]: Using system hostname 'mysite.net'.
Sep  2 10:13:50 mysite systemd-resolved[396752]: Using degraded feature set UDP instead of UDP+EDNS0 for DNS server 1.1.1.1.
Sep  2 10:13:55 mysite systemd-resolved[396752]: Using degraded feature set UDP instead of UDP+EDNS0 for DNS server 8.8.8.8.
Sep  2 10:13:55 mysite systemd-resolved[396752]: Using degraded feature set TCP instead of UDP for DNS server 1.1.1.1.
Sep  2 10:14:00 mysite systemd-resolved[396752]: Using degraded feature set TCP instead of UDP for DNS server 8.8.8.8.

Rebooting the server fixes it for about 1 day, then it returns.

I’m not using the BIND.

/etc/systemd/resolved.conf has this:

[Resolve]
DNS=1.1.1.1 8.8.8.8
FallbackDNS=1.0.0.1 8.8.4.4
DNSSEC=no

Any suggestions for remedies would be gratefully received.

2

After lengthy research it turns out that Awstats was launching too many reverse IP lookups, triggering a bug in resolveD that tries to use stale file handles in its socket “graveyard”.

In case anyone else ever runs into this issue, what fixed it was replacing resolveD by dnsmasq.

3

Wish RHEL 10 (EPEL) did not remove Awstats, users liked it but since it is not maintained anymore they removed it.

4

In the modern web, AWstats and Webalizer can’t tell you very much about your usage.

It can’t see traffic served by CDNs or from local caches. It can’t see SPA interactions. It doesn’t know all that much about your users (relatively speaking, it just sees User-Agent, Referer, and IP), and it’s impossible to track “visits” when a user might travel across a bunch of IP addresses. If someone is browsing your site on a train, they’ll look like a dozen visits.

In some regards, this is a good thing. I’d rather more sites knew less about me. But, complicated web apps can have bugs and without data about usage, they might go unnoticed for a long time. And, if you want to best serve your users, you have to know a few things about them and actions they take.

There’s a reason nobody is spending much effort on tools like that, anymore. I don’t love the alternatives (a JavaScript bug that can track your every interaction feels bad, and it feels much worse when it feeds back to Google and/or Meta and a few thousand other “data partners”).