Thanks to all.
Indeed i think it was my router.
I was excluded this posibility upfront.
I had activated remote control over the internet for my Asus Router, and i think his firewall was blocking my ports to server, but i had it on DMZ, so that why i thought it is not my router’s fault.
I checked logs and i saw brute force attacks straight on my wan.
I experienced 2-3 more cutoffs on my virtualmin server and i rebooted my Asus router and issues where fixed only by that.
I still had connection to internet from server during cutoffs, but i think only 80 and 443 ports where blocked, or dns’s.
For now i have had no issues for last 3 days.
My backup server is on a different ISP and another router, that’s why i had no issues there.
Is either my isp, or those brute attacks straight on my WAN.
If i experience more of those cutoffs, i will check ports to see if my router blocking only certain ports, but if is on DMZ, i don’t think is that the issue.