… just to be clear - quotas are enabled in Virtualmin Configuration?
… and you re-run config check, and you don’t have a message saying it’s okay?
...
Both user and group quotas are enabled for home and email directories.
...
It’s unclear why it doesn’t work for you.
You could try running the following command to patch all needed files for Virtualmin to setup quotas (as it’s done upon initial setup):
virtualmin-config-system -i=Quotas