TLDR; I confirmed that there are two log rotate entries for apache and then I only deleted the daily log rotate entry for /var/log/apache/*.log (I kept the log rotate entries for /virtualmin/).
The problem happens when two or more restarts are done consecutively, this makes Apache assume something is wrong with itself and is crashing as soon as it starts and then it prevents itself from starting which leads to the following messages.
[Mon Nov 08 06:25:09.847585 2021] [mpm_event:notice] [pid 1885:tid 139788868089728] AH00493: SIGUSR1 received. Doing graceful restart
[Mon Nov 08 06:25:11.902396 2021] [core:notice] [pid 1885] AH00060: seg fault or similar nasty error detected in the parent process
I had no traffic coming on to the default apache site which is why I deleted the log rotate entries.
You could also try changing the schedule for the /apache/*.log log rotate entry or maybe add a 5 or more second sleep before the graceful restart occurs. (This would be more ideal but I thought of this later :P). Do let me know if you think the same / if you try this and works.
If anyone knows, can Virtualmin (or any other package) re-create the log rotate entries or modify the schedule back to default?