Improving your server is the best single way to get performance gains. Better processors, more memory, higher bus speeds. That’s what makes everything faster. You’re not going to simply install a new OS and VOILA! More speed.
Wordpress is just slow. It has been for the last 6 or 8 years and getting slower all the time because of it’s popularity. It’s how they sell things. Make Wordpress so bloated and dysfunctional that people will start buying plugins to make it run better.
The single best thing you can do to Wordpress is go through and increase everything in your PHP.INI files. Memory, upload timeout, upload numbers, upload size, etc. All of it. Then make sure you increase it in Wordpress as well to match.
That will do more for you than any OS change.