Hardening Ubuntu / Vmin / Wmin / Umin With Systemd

Hello All,

I am currently hardening a server and I want to request & share knowledge about hardening services with Systemd.

I realize that hardening services with Systemd doesn’t seem to be a popular technique, so I guess my first question is… Why isn’t it? (for reasons other than being tedious)

Should I invest the energy in AppArmor Instead?

Logically, it seems that hardening LAMP (specifically php) would be a popular thing since most hacks come from out-dated CMS, that doesn’t seem to be the case with systemd and php. I can find virtually nothing on the topic.

In any case, if there are some more experienced admins with thought on this matter, I would appreciate them.

As for sharing… I managed to to secure apache2 via using the following:

nano /etc/systemd/system/apache2.service.d/apache2-custom.conf, containing:

[Service]
NoNewPrivileges=yes
LockPersonality=yes
ProtectClock=yes
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectControlGroups=yes
RemoveIPC=yes
RestrictAddressFamilies=AF_UNIX AF_INET AF_NETLINK
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
SystemCallArchitectures=native
SystemCallFilter=@system-service
SystemCallFilter=~@keyring

And I have partially hardened PHP-FPM with:
nano /etc/systemd/system/apache2.service.d/php7.3-fpm-custom.conf containing

NoNewPrivileges=yes
PrivateTmp=yes
PrivateDevices=yes
DevicePolicy=closed
ProtectControlGroups=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
RestrictAddressFamilies=AF_UNIX AF_INET AF_NETLINK
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
LockPersonality=yes

Feedback, tips, admonishments for stupidity… all welcome :wink:

The question not asked or answered here is “secured from what?”

Yes, PHP apps are a very common source of exploit. But, I don’t think any of this will prevent or mitigate those kinds of exploits.

Also, Apache has very rarely been an exploit vector. It has an excellent security record (a few modules over the years have had problems, but the core service and the most popular modules are very good). Most of these assume the service itself is untrustworthy…i.e. someone can bust the stack and make the service do something it should not, maybe before it drops privileges, so it can provide escalation. Feels like if these were both effective and didn’t cause any problems in common deployments, CentOS (at least) would already use them, and it mostly does not. (Ubuntu and Debian trail CentOS on server security stuff, in general, or have chosen a less intrusive path. SELinux vs. AppArmor…SELinux does more, but is a complete pain in the ass. AppArmor is set it and forget it, but won’t stop a lot of attacks that SELinux theoretically can.)

Let’s go through some of these (I don’t know what most of them do, but there are a few I do):

Apache user already can’t touch the hardware clock. This is a no-op.

Apache user already cannot do anything to kernel modules. Can’t explicitly load/unload them. Can’t modify them. Can’t sniff them. This does nothing.

Again, Apache user can’t do anything to the kernel. This is a no-op.

Only users that have been granted rt privileges can use rt features, as far as I know. Probably another no-op.

Some of them sound like they might be useful, though. But, since I don’t know what they do or what attack vector they’re intended to address, I don’t have anything to say about them. PrivateTmp sounds like a good thing, for example…though I worry it might break user apps. That’d be my biggest concern here, is that you’re going to find yourself with problems running apps and no idea why. Maybe a month from now, when you’ve forgotten all the changes you made, and you can’t make something work.

@Joe thanks for the feedback. What I am getting from your post is that I don’t know enough about the apps themselves to accurately choose the systemd attributes to secure them. Therefore, I am probably better off setting an AppArmor profile that I can easily turn on and off while troubleshooting bugs. Fair enough. (the prospect of systemd hardening every service was/is not exciting me)… though I still plan to tinker with the important services php, etc. etc… until I get settings, I am confident in… thanks, I now have a road map.

[funny-sidetopic]
removing systemd would be a good security-wise start :slight_smile:

[back to topic]
kernel tweaks (sysctl) would be a better way to implement system accesses, mod_security for apache, chrooted apache/php jails, and probably many more things… but still imho, keeping systems/websites updated is the best defense.

I’ve heard this advice before… but for web servers I think this philosophy can create a very false sense of security for an admin.

Anybody in this group https://security.stackexchange.com/ would laugh if you -only-depended on the OS and CMS updates for all of your security needs.

While keeping the OS and CMS up to date is a basic rule that everyone should practice, in relation to securing a web server against potential attack it seems very much incomplete.

IMHO, I believe web servers are analogous to museums. In any country you have “National Museums” and “Local Museums”, which in web terminology could be thought of as “Bare Metal Servers” and “Virtual Private Servers”.

The bigger more, invested the museum, the bigger, more invested the security.

You would very rarely in the real world find a museum that only relies on common deadbolt locks on the doors and closing/locking the windows for their only security, which is analogous to only updating your OS & CMS on a webserver.

Inside of museums you have priceless works of art that required artists to spend hours, days, months, even years to create & eventually gather admiring looks, or rather “user views” & museum concessions sales/donation revenues .

On web servers you have web projects that required developers (artists) to invest significant periods of time to create & maintain. And their “user views” and “sensitive user data” (revenues) merit at least some security consideration beyond just “locking the door & closing the windows.”

For “noobs” just entering the webhosting arena, I think there is no better tool for learning how to secure your system than Lynis.

From this single tool, I learned about sysstat, psacct, auditd, sysdig, closing unused ports, kernel hardening, PAM hardening, removing unnecessary information from banners, and removing unused/outdated network protocols, which more often than not contain security vulnerabilities, that knowledgeable hackers can readily exploit.

Simply updating the OS is insufficient when the OS itself includes exploitable security holes.

As a developer entering the vps webhosting arena, Virtualmin/Wmin/Usermin is by far the sane choice of control panel for usability, documentation, & built-in security features.

However, I wouldn’t consider myself even an intermediate level sysadmin until I can fully harden the OS against attack, (Lynis score of 90, perfect is impossible), properly apply all of the free and readily available security tools such as fail2ban, modsecurity, maldet, etc, readily protect system services with a security framework such as AppArmor, Selinux, etc. maintain an offsite daily backup regimen, easily/readily determine if rkhunter, chkrootkit, & maldet hits are “false positives” or “real hits”, and get my disaster & recovery time to under 12 hours.

Consider server hardening from the financial viewpoint. Sure, you can use Virtualmin to get your webserver up, running, and ready for webhosting in X days. But why not invest X additional days to learn how to properly secure it? Because -after- your system is hacked by a high school kid, who did it for fun & practice or even worst, a pro hacker using a ransomware… its too late.

If you choose to only “lock the door & close the windows” on a system containing works of art, often worth more than system itself, it will cost you a lot more money & time hiring a higher level sysadmin to help clean & restore your system in the long-run.

In a nutshell, if you are going to do something… do it right.

That’s not at all correct. systemd natively supports control groups, among other things. Very few people are aware of it or making advanced use of it, thus far, but already you’re almost certainly getting a more secure system by default with systemd…at least now that it’s had a decade or so for the bugs to fall out and for OS vendors to start taking advantage of its capabilities.

My argument was not that systemd doesn’t have security features, it’s just that the security features are useful in specific contexts, and often they’re already being used by the default systemd unit files, if they make sense and won’t break common use cases.

+1, almost there after 2 decades of managing systems. (there’s no such thing as 100% prepared/secure).

this is one advice, not a full security checklist. if you’re going to be security-paranoid, you shouldn’t be using a control panel for starters… so many open ports/services = security nightmare.
my experience so far in simple websites hosted in virtualmin, always showed older/unmaintained php scripts/websites as responsible for security holes. imho, sophisticated web server attacks don’t attack small websites like the ones web control panels usually manage.


systemd adds a similar big attack surface, and it’s security history log is full… so, imho again, minimal/simpler applications help sysadmins keep systems more secure, more easily, without spending hundreds of work hours debugging software beasts…
control groups is a kernel feature, there are other ways of implementing them, without using a 5?million-lines code…
(not even gonna dig into foss -& init- diversity which is more important for me personally)

anyway just my 2c, not into starting flamewar or anything…
not while real war is on my/our neighborhood.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.