without being too specific:
what are these sites doing in php?
are they caching requests correctly - if at all (eg memcache)?
is there enough memory/space - is this one /or multiple sites/sub affected?
It is almost certainly not PHP-FPM that is slowing down. It’s almost certainly something in your app, probably database usage.
Check the logs. What errors when you have a time out?
And, since it’s usually a database thing, some app or plugin probably has a pathological query that takes a long time, causing everything to back up. So, turn on slowquery logging in Mariadb and find those slow queries and fix them, or disable the offending plugin.
You may be able to configure Mariadb to mitigate the problem and accommodate whatever database abuse is happening. There’s mysqltuner for getting the most out of Mariadb on your particular system.
Anyway, it might not be database, it might be something else, but there’s almost certainly going to be some evidence if it’s something else. So, look at the FPM logs, check the error_log. What specific requests are hanging around long enough to be a problem.
This might help trying to find the slow query. The relevant settings need turning on which I believe are now present in the MariaDB/MySQL Webmin module or you can do it via the old fashioned way by editing the config files.