Hi:
In my office I have a Centos 6.5 server with Virtualmin 4.06.gpl GPL in which I host a couple of websites. The server is losing connectivity to the internet several times a day. Sometimes, I have to restart the server in order for it to work again. Other times, it takes a couple of hours for it to fix itself.
When the server loses conectivity to the web, I ping the external IP and get a response. Also, if I’m on my office and the server loses connectivity, I connect my laptop directly to the same router that the server connects to and I’m able to access the internet.
In the status windows, the ClamAV Virus Scanning Server is Down. I just enabled it and the error is gone, but the service will get down again as it usually does. there have been times when trying to start the clamav service it gives me an error which I don’t remember now.
First step as always: check the logfiles in /var/log for errors at the time when the problem occurs. This issue can have any number of reasons, hard to guess. Could be out of memory issues.
What exactly do you mean when you say the server loses connectivity? Can you still reach it on SSH? Can the server ping to the outside?
You can check logs from Webmin by going into Webmin -> System -> System Logs.
You could also try logging in via the console, in case it’s a network problem.
However, once the server is back online, you could also check the logs then to see if any errors are showing up during the time that it was down earlier.
I’ve lost connectivity from the outside to my server for more than an hour. I have tried restarting the server twice, but was not successful after that.
Via console, when I go to /var/log there are a couple of directories. I have clamav, audit, httpd, mailman, proftpd, and virtualmin. Also, many log files.
What should I look for?
Ok, I got connectivity now. I stopped and restarted csf firewall and I’m in. But, I’m not sure if I was already connected before stopping csf.
I was looking at this csf log when I lost connectivity again now:
Mar 12 14:23:15 apollo lfd[1936]: iptables appears to have been flushed - running csf startup…
Mar 12 14:23:20 apollo lfd[1936]: csf startup completed
Mar 12 14:23:20 apollo lfd[20070]: Suspicious Process PID:19108 PPID:2025 User:mysql Uptime:64 secs EXE:/usr/libexec/mysqld CMD:/usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
Mar 12 14:23:21 apollo lfd[20070]: Suspicious Process PID:19151 PPID:2309 User:apache Uptime:63 secs EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: Suspicious Process PID:19160 PPID:2309 User:apache Uptime:63 secs EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: Suspicious Process PID:19161 PPID:2309 User:apache Uptime:63 secs EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: Suspicious Process PID:19162 PPID:2309 User:apache Uptime:63 secs EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: Suspicious Process PID:19163 PPID:2309 User:apache Uptime:63 secs EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: Suspicious Process PID:19164 PPID:2309 User:apache Uptime:63 secs EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: Suspicious Process PID:19165 PPID:2309 User:apache Uptime:63 secs EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: Suspicious Process PID:19166 PPID:2309 User:apache Uptime:63 secs EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: Suspicious Process PID:19167 PPID:2309 User:apache Uptime:63 secs EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: User Processing PID:19162 Kill:1 User:apache VM:428(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: User Processing PID:19167 Kill:1 User:apache VM:428(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: User Processing PID:19166 Kill:1 User:apache VM:428(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: User Processing PID:19163 Kill:1 User:apache VM:428(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: User Processing PID:19108 Kill:1 User:mysql VM:840(MB) EXE:/usr/libexec/mysqld CMD:/usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
Mar 12 14:23:21 apollo lfd[20070]: User Processing PID:19160 Kill:1 User:apache VM:428(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: User Processing PID:19164 Kill:1 User:apache VM:428(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: User Processing PID:19151 Kill:1 User:apache VM:312(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: User Processing PID:19165 Kill:1 User:apache VM:428(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:21 apollo lfd[20070]: User Processing PID:19240 Kill:1 User:pcelements VM:235(MB) EXE:/usr/bin/php-cgi CMD:/usr/bin/php-cgi
Mar 12 14:23:21 apollo lfd[20070]: User Processing PID:19161 Kill:1 User:apache VM:428(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:23:22 apollo lfd[20070]: User Processing PID:19250 Kill:1 User:jorgepi VM:314(MB) EXE:/usr/bin/php-cgi CMD:/usr/bin/php-cgi
Mar 12 14:24:16 apollo lfd[20313]: User Processing PID:20196 Kill:1 User:apache VM:429(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:24:16 apollo lfd[20313]: User Processing PID:20195 Kill:1 User:apache VM:429(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:24:16 apollo lfd[20313]: User Processing PID:20308 Kill:1 User:fashionboxpr VM:244(MB) EXE:/usr/bin/php-cgi CMD:/usr/bin/php-cgi
Mar 12 14:24:16 apollo lfd[20313]: User Processing PID:20134 Kill:1 User:mysql VM:840(MB) EXE:/usr/libexec/mysqld CMD:/usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/lib/mysql/mysql.sock
Mar 12 14:24:16 apollo lfd[20313]: User Processing PID:20305 Kill:1 User:jorgepi VM:314(MB) EXE:/usr/bin/php-cgi CMD:/usr/bin/php-cgi
Mar 12 14:24:16 apollo lfd[20313]: User Processing PID:20199 Kill:1 User:apache VM:429(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:24:16 apollo lfd[20313]: User Processing PID:20198 Kill:1 User:apache VM:429(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:24:16 apollo lfd[20313]: User Processing PID:20197 Kill:1 User:apache VM:429(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:24:16 apollo lfd[20313]: User Processing PID:20194 Kill:1 User:apache VM:429(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:24:16 apollo lfd[20313]: User Processing PID:20200 Kill:1 User:apache VM:429(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:24:16 apollo lfd[20313]: User Processing PID:20187 Kill:1 User:apache VM:313(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Mar 12 14:24:16 apollo lfd[20313]: User Processing PID:20193 Kill:1 User:apache VM:429(MB) EXE:/usr/sbin/httpd CMD:/usr/sbin/httpd
Hi:
I uninstalled csf firewall and the server is working normally now. I still have to keep checking the Clamav server to see if something is not right or if the error was due to csf also.
Thanks for your help!
Maybe LFD was blocking your IP for “suspicious process” or something? I use it too and you can easily get yourself banned if you don’t set it up properly.
Please enclose your shell listing in [code][/code] tags, because it’s unreadable in the current form.
Could be out of memory issues.